← Back to DevOps

Jenkins CI/CD Pipeline: Complete Beginner's Guide

Jenkins is an open-source automation server that streamlines continuous integration and continuous deployment. It automates parts of software development like building, testing, and deploying applications, reducing manual work and catching bugs early.

What Is Jenkins and Why Does It Matter?

Jenkins has been the backbone of CI/CD automation for over a decade. It's free, extensible, and works with virtually every development stack. When you push code to your repository, Jenkins automatically triggers builds, runs tests, and can deploy to production—all without human intervention.

Without Jenkins or similar tools, developers manually build, test, and deploy code. This approach is slow, error-prone, and doesn't scale. Jenkins eliminates these bottlenecks by automating the entire workflow. Teams using CI/CD pipelines catch integration issues within minutes instead of days, and they deploy code multiple times per day with confidence.

The core value proposition: faster feedback, fewer bugs, quicker releases, and happier teams.

Understanding CI/CD Fundamentals

Before diving into Jenkins specifics, you need to understand the concepts it orchestrates.

Continuous Integration (CI)

CI means developers integrate code into a shared repository frequently—often multiple times per day. Each integration triggers automated builds and tests. If something breaks, the team knows immediately. This prevents the "integration hell" that happens when teams work in isolation for weeks.

Continuous Deployment (CD)

CD takes integration further. Every successful build automatically deploys to production (or staging). Your code goes from developer's laptop to live servers in minutes. Some teams use "Continuous Delivery" instead, which means code is *ready* to deploy but requires manual approval first.

Continuous Delivery vs. Continuous Deployment

Continuous Delivery stops before production—a human clicks a button to deploy. Continuous Deployment goes all the way. Jenkins supports both patterns, and you'll configure whichever matches your risk tolerance.

Getting Started: Jenkins Installation and Setup

Installing Jenkins is straightforward. Here's what you need to know.

System Requirements

Jenkins runs on Linux, Windows, and macOS. It needs Java 11 or higher. For development, 512MB RAM is minimum; production requires 1-2GB+. Disk space depends on your artifact storage needs—plan for at least 10GB.

Docker Installation (Recommended for Learning)

The fastest way to start learning is Docker:

docker run -d -p 8080:8080 -p 50000:50000 \
  -v jenkins_home:/var/jenkins_home \
  jenkins/jenkins:lts

This starts Jenkins on http://localhost:8080. The first run generates an admin password displayed in the logs.

Traditional Installation on Linux

wget -q -O - https://pkg.jenkins.io/debian/jenkins.io.key | sudo apt-key add -
sudo sh -c 'echo deb https://pkg.jenkins.io/debian-stable binary/ > /etc/apt/sources.list.d/jenkins.list'
sudo apt update
sudo apt install jenkins
sudo systemctl start jenkins

After installation, Jenkins starts on port 8080. Navigate to localhost:8080 and retrieve the initial admin password from /var/lib/jenkins/secrets/initialAdminPassword.

Initial Configuration

During first setup, Jenkins prompts you to install recommended plugins. Accept this—it includes Git integration, pipeline support, and essential tools. You'll then create an admin account and configure the Jenkins URL (important for webhook callbacks from GitHub/GitLab).

Building Your First Jenkins Pipeline

Jenkins supports two pipeline styles: Declarative and Scripted. Declarative is friendlier for beginners.

Creating a Declarative Pipeline

Go to Jenkins homepage → New Item → Pipeline. Name it "HelloWorld" and paste this:

pipeline {
    agent any
    stages {
        stage('Checkout') {
            steps {
                echo 'Checking out code...'
            }
        }
        stage('Build') {
            steps {
                echo 'Building application...'
                sh 'echo "Build successful!"'
            }
        }
        stage('Test') {
            steps {
                echo 'Running tests...'
                sh 'echo "Tests passed!"'
            }
        }
        stage('Deploy') {
            steps {
                echo 'Deploying to production...'
            }
        }
    }
    post {
        always {
            echo 'Pipeline finished.'
        }
        success {
            echo 'Pipeline succeeded!'
        }
        failure {
            echo 'Pipeline failed!'
        }
    }
}

Click "Build Now" and watch Jenkins execute each stage. The post section runs after all stages, with branches for success/failure scenarios.

Connecting to a Real Git Repository

Most pipelines start by pulling code from Git. Modify the Checkout stage:

stage('Checkout') {
    steps {
        git branch: 'main', 
            url: 'https://github.com/yourusername/yourrepo.git'
    }
}

If your repo is private, you'll need to add credentials in Jenkins (Manage Jenkins → Manage Credentials) and reference them:

stage('Checkout') {
    steps {
        git branch: 'main',
            credentialsId: 'github-credentials',
            url: 'https://github.com/yourusername/yourrepo.git'
    }
}

Running Tests in the Pipeline

The Test stage is where automated tests run. For a Node.js project:

stage('Test') {
    steps {
        sh 'npm install'
        sh 'npm test'
    }
}

For Python:

stage('Test') {
    steps {
        sh 'pip install -r requirements.txt'
        sh 'pytest'
    }
}

If tests fail, Jenkins stops the pipeline and marks it red. This prevents broken code from reaching production.

Triggering Pipelines Automatically

Manually clicking "Build Now" doesn't scale. Real CI/CD is event-driven.

Poll SCM

Jenkins can periodically check your Git repo for changes. In the Pipeline configuration, enable "Poll SCM" and set a schedule:

H/15 * * * *

This checks every 15 minutes. It's simple but inefficient—Jenkins repeatedly polls even when nothing changed.

Webhooks (Better Approach)

GitHub/GitLab can push events to Jenkins instantly. On GitHub:

  1. Go to your repo → Settings → Webhooks → Add webhook
  2. Payload URL: http://your-jenkins-server:8080/github-webhook/
  3. Content type: application/json
  4. Select "Push events" and save

Now every push triggers your pipeline immediately. In Jenkins, enable "GitHub hook trigger for GITScm polling" in the Pipeline configuration.

Scheduled Builds

Some teams run nightly or weekly builds regardless of code changes (e.g., security scans, dependency checks):

triggers {
    cron('H 2 * * *')  // 2 AM daily
}

Managing Artifacts and Build History

Pipelines produce artifacts—compiled binaries, Docker images, reports. Jenkins stores these so you can download them later or inspect test results.

Archiving Artifacts

After building, save artifacts:

stage('Build') {
    steps {
        sh 'mvn clean package'
    }
    post {
        success {
            archiveArtifacts artifacts: 'target/*.jar', 
                           allowEmptyArchive: false
        }
    }
}

Jenkins stores these in the job's workspace. You can download them from the UI or fetch via API.

Test Reports

Jenkins integrates with testing frameworks. For JUnit tests:

post {
    always {
        junit 'target/surefire-reports/**/*.xml'
    }
}

Jenkins parses XML reports and displays pass/fail statistics and trends over time.

Essential Jenkins Plugins and Extensions

Jenkins alone is powerful, but plugins supercharge it. Here are must-haves for beginners:

Install plugins via Manage Jenkins → Manage Plugins → Available. Search and click Install.

Security Best Practices

Jenkins often has access to sensitive data and production systems. Secure it properly.

Credential Management

Never hardcode passwords or API keys in pipelines. Use Jenkins Credentials:

stage('Deploy') {
    environment {
        AWS_CREDENTIALS = credentials('aws-prod-credentials')
    }
    steps {
        sh 'aws s3 cp app.jar s3://my-bucket/'
    }
}

The credentials() function retrieves encrypted secrets stored in Jenkins. They're never printed in logs.

Authentication and Authorization

Enable authentication immediately. Go to Manage Jenkins → Security → Enable security. Set up role-based access control so developers can't access production deployments.

Regular Updates

Jenkins releases security patches frequently. Check for updates weekly and apply them promptly.

Troubleshooting Common Pipeline Issues

Pipelines fail. Here's how to debug.

Build Logs

The build log is your primary diagnostic tool. Each pipeline execution has a clickable log showing every command and its output. Most issues are visible there.

Timeout Issues

If tests or deployments hang, add timeouts:

stage('Test') {
    options {
        timeout(time: 10, unit: 'MINUTES')
    }
    steps {
        sh 'npm test'
    }
}

Workspace Cleanup

Old build artifacts consume disk space. Enable automatic cleanup:

options {
    buildDiscarder(logRotator(numToKeepStr: '10', artifactNumToKeepStr: '5'))
}

This keeps the last 10 builds and 5 with artifacts.

Agent Availability

agent any uses any available agent. If no agents are free, jobs queue. Monitor agent status in the Jenkins dashboard and add agents if needed.

Advanced Pipeline Patterns

Once comfortable with basics, these patterns unlock powerful