Jenkins Tutorial: Changes to Github for CI/CD

Yes, I admit it, i was once a "Cowboy Coder," one of those programmers who worked alone, and didn't bother with paltry things like "version control."  To be fair, tools like Jenkins, which make it easy to automatically build an application, didn't exist back then. Fortunately, today I can set up a Jenkins server to create a Continuous Integration/Continuous Deployment (CI/CD) pipeline, automatically building, testing, and deploying changes to an application any time they're committed to a Git repository.
It's a good thing, too, because today it's not just about applications; even infrastructure is frequently so complex that it needs to be treated like source code, with instructions for creating it versioned and stored so that it can be run automatically.
So it's time to learn about how CI/CD works.  And where better to start than with Jenkins, which frequently sits at the heart of the matter? In this series of articles, we're going to look at how to perform various tasks, such as Jenkins pipeline, allowing for human intervention, and performing tests and security scanning.
Today we'll start at the very beginning, with how to install Jenkins and get it to build your project whenever someone pushes changes to Github repositories.
In general, we're talking about four steps:

1. Install the prerequisites
2. Install and initialize Jenkins
3. Connect Jenkins to Github
4. Create a project to build your application

Before we start, however, let's talk a little bit about what Jenkins actually is and what it does for us.

What is Jenkins?

Jenkins is arguably the standard tool used for automatic deployment of applications.  Packaged as a self-contained Java application, Jenkins enables you to create "jobs" that carry out specific tasks when certain events occur.
For example, you might configure a job that runs unit tests against your codebase once a day, sending an email with the results to a specified list of people. You might set it up so that it acts as a distributor of the "latest" version of your application at any given time.  Or you might, as we are going to do in this tutorial, simply set it up to build your application whenever changes are pushed to Github, thus making certain that code doesn't get too badly out of sync.
Jenkins comes with a number of different plugins that help integrate with various applications and systems, and hundreds of plugins are available for it. There are probably very few CI/CD-related tasks that Jenkins can't do.  (In fact, I can't think of any off the top of my head.)
You can even create complicated pipelines for Jenkins to follow, including multiple "stages". For example, Mirantis Cloud Platform (MCP) uses Jenkins as part of its CI/CD process, pushing code to a staging environment automatically but requiring human approval before changes are pushed to production.
In this Jenkins tutorial, we’ll run you through the installation and initialization steps to get you up and running.

Installing Jenkins

The first step, of course, is to install and initialize Jenkins.  As a Java application, it can run pretty much anywhere but the Jenkins site does have instructions and downloads that make installation easy on a number of different platforms, including various Linux flavors, Windows, and MacOS.  In our case, we're going to use a fresh Ubuntu 16.04 VM to make sure that we haven't left out any steps.
To get up and running, we're going to execute the following steps:

1. Install Git
2. Create a local test repository to make sure Git is working, and to have something to test Jenkins with
3. Install Java8
4. Install Jenkins
5. Initialize Jenkins

Fortunately, we can use Ubuntu's built-in package management system, apt-get, to do most of what we need to do.

Install Git

To install Git on Ubuntu, we just have to execute two steps:

sudo apt-get update
sudo apt-get install git

Once Git is finished installing, you can test it by checking out an existing Github repository.  For example, in my case, I'm going to use:

git clone https://www.github.com/nickchase/jenkins-example

This repo consists of two files:

Jenkinsfile
README.md

The files themselves aren't very important at this stage; they don't actually do anything significant. We just want to have the repo ready so we can see that Jenkins builds the project when we make changes.  To do that, we do need to create a simple Jenkinsfile.  In my case, I'm using:

pipeline {
  agent any
  stages {
    stage('myStage'){
      steps {
        sh 'ls -la' 
      }
    }
    stage('Build') {
      steps { 
        sh 'ls' 
      }
    }
  }
}

Note also that while Jenkins runs in Java and is therefore platform independent, the build scripts themselves are not; if you are running Jenkins on a Windows machine, replaceNote that this file doesn't actually do much of anything, except output the contents of the directory (twice). It certainly doesn't build anything. We'll talk more about creating a Jenkinsfile that actually builds and deploys an app in future articles, but for now we just want to make sure everything works, so don't worry about trying to figure it out.

Note also that while Jenkins runs in Java and is therefore platform independent, the build scripts themselves are not; if you are running Jenkins on a Windows machine, replace the commands:

sh 'ls -la'

and

sh 'ls'

with

bat 'dir'

Install Java 8

Because Jenkins is a Java application, we will, of course, have to have Java installed.  Unfortunately, it does require Java 8, so it's not as simple as just installing the default JDK, which is, at the time of this writing, Java 7. Fortunately, though, it's not overly difficult.  
To install Java 8, start by referencing Oracle's repos, then update apt-get so it knows where to look. Once you've done that, you can simply install Java 8 directly:

sudo add-apt-repository ppa:webupd8team/java
sudo apt-get update
 sudo apt-get install oracle-java8-installer

You'll need to follow the prompts and agree to Oracle's license agreement.

 

Install Jenkins

The process for installing Jenkins is similar to that for Java 8, in that you'll need to add a new repository, update apt-get, then install.  You do, however, first need to add the Jenkins key to your system using the following commands:

wget -q -O - https://pkg.jenkins.io/debian-stable/jenkins.io.key | sudo apt-key add -

Once you've done that, you'll need to manually add the repo by adding the line:

deb https://pkg.jenkins.io/debian-stable binary/

To your /etc/apt/sources.list file. To do this, simply edit the file, as in:

sudo vi /etc/apt-get/sources.list

Once you've done that, update apt-get and install, as in:

sudo apt-get update
sudo apt-get install jenkins

Initialize Jenkins

Before you can actually use Jenkins, you'll need to initialize it.  To do that, point your browser at http://localhost:8080. You'll be faced with a screen asking for the administrator password.

The easiest way to get this password is to just copy it from the initialAdminPassword file:

sudo cat /var/lib/jenkins/secrets/initialAdminPassword | pbcopy

Next Jenkins will ask you to decide what plugins you want installed; everything we need is in the "recommended" set, so just choose that.
 

Once the plugins are installed, you'll need to create a new user:

It's not necessary to use the same credentials you use for GitHub.

Jenkins Github Integration

Our ultimate goal is to create a project that builds the application when changes are submitted to Github, but in order to do that, we need to connect Jenkins to Github via the Github API.  To do that, we need to create a personal access token using our Github account.

Create a Github access token

To create a Github access token, log into your Github account and go to Settings/Personal Access Token. From there, you can click Generate New Token.

Github enables you to control the capabilities of your new access token with a pretty fine level of granularity, and in a real world situation it’s best to choose the bare minimum of capabilities.  In this case, however, give the token all privileges and click Generate Token.

Make sure to copy your new token; once you navigate away from this page, you won't be able to get it back.

Add Github access token to Jenkins

To add your new token to Jenkins, go back to the Jenkins window and click Credentials.

Select the Jenkins provider, then the Global Credentials (unrestricted) domain. Click Add Credentials.
In this case, we want to create a Secret Text credential, so set the Kind, then add the text and name it using the ID field.
Click OK to save your changes.

Add Github credentials to Jenkins

We're also going to need to use our regular credentials to connect to Github, so click Add Credentials again.
This time, choose Username with Password for the Kind and enter your GitHub username and password. You don't need to name this one (though you can); it'll show up under the username.

Click OK to save your changes.

Restart Jenkins

Before we can actually connect to Github, we'll want to restart Jenkins so the credentials are available to us. To do that, point your browser at:
http://localhost:8080/restart

Confirm the restart and wait for Jenkins to come back up.  When it does, log in with the username and password you created earlier.
Now we can actually go ahead and make the connection to GitHub.

Connect Jenkins to Github

In order for Jenkins to know that a change has been submitted (and thus that it needs to build the app) it has to know to check with the Github API. In order to make that happen, we need to configure that capability from within Jenkins itself.
To start, click Manage Jenkins, then Configure System.

Scroll down to the GitHub section and under GitHub Servers, click Add GitHub Server, then GitHub Server again.

Give the server a name, then choose your secret text credentials and click Test Connection to make sure everything is configured properly.

Scroll down and click Save.

Now we’re ready to create the actual Jenkins job that builds the app when changes are pushed.

Create a Jenkins that builds when changes are pushed to Github

We're almost finished. We just need to create a job that automatically triggers when Jenkins detects Git repo changes.
Start by clicking New Item.

Specify a name for the project and choose Multibranch Pipeline, then click OK.

Scroll down to Branch Sources, then click Add Source, then GitHub.

Choose the credentials you entered earlier, then specify the owner of the repo you're going to be using. This will make it possible for you to choose the specific repo to monitor and build.

If you scroll down, you'll notice that the default is to use build instructions in the Jenkinsfile, which we added earlier.

Finally, under Scan Multibranch Pipeline Triggers tick the box for Periodically if not otherwise run and set the time appropriately. This is how often Jenkins will look for changes.

In our case we're going to use 1 minute just to make sure it's working, but you probably don't need to build that often.
Finally, click Save.

Test it out

When you save the project, it will automatically scan the repo and build to make sure everything is working.

You can see the results of the build by click the build number

To see the results, you can click Console output

Now let's make sure the actual integration is working; go ahead and make a change to one of the files in the repo and push the change to Github, as in:

vi README.md

(make change)

git add README.md
git commit -m "Fixed README"
git push

You may need to enter your GitHub credentials on the command line to get the change to go through.
Once the change has been pushed to GitHub, within the next 60 seconds or so, the Jenkins Git changes job should trigger and you’ll see a new build starting:

So that gets us started! Next time, we'll talk about some of the more complex tasks you can accomplish with Jenkins.  What would you like to learn how to do?  Let us know in the comments!