I recently struggled a lot to get a self-hosted Jenkins instance and GitHub to work together, at work.
After having finally made it, I thought to myself that maybe, someone else might have the same problems that I faced.
Hope you enjoy and take something useful for yourself.
In tech, there are still a lot of words and terms that have a somehow negative attitude when used together.
With Jenkins, for example, the terms 'master' and 'slave', together, usually describe a coordinating instance that does management work and worker nodes that the coordinating instance can offload work to.
Due to to the recent events and due to my personal feelings, that those old-school terms (when used together) do not reflect an inclusive and welcoming world, I will use the following words to replace them:
- master -> coordinator
- slave -> worker
Some context always helps. Not everyone has the same requirements or the same ability to do as they like.
Doing something for yourself has other requirements than at your job, where you might have security policies that prevent you from doing certain things.
Even companies can differ greatly in those regards. A startup will most likely be able to move faster than an established enterprise organization, where everything has to be requested by ticket, mail, personally, sacrifice, etc.
Please keep this in mind while reading. I hope the context I give you about the company I work for, helps you understand why some things were done as they are.
We are still mainly a Java shop (there's more and more Node we use, e.g. and also other languages like Go or Rust), and we do enterprise stuff.
We mostly deploy micro services these days, which are (again) mostly Java-based HTTP-REST services, that communicate with one another and the outside world.
Getting new technology into banks and the financial industry is actually quite difficult sometimes. You have to do a lot of advocating and evangelism to get a bank's IT to agree on deploying something they don't know, yet.
This is why we don't use too much cloud technology, and e.g. still run with Jenkins.
Our usual setup consists of a lot of on-premise hardware, mainly VMs, that are spread across a multitude of machines.
What we use:
- Bitbucket (self-hosted) for our source code, internal network
- Jenkins coordinators, with workers, internal network
- A sonatype Nexus Repository, internal network
- VMs to run our software (some containerized, some not), internal network We sometimes leverage AWS for cloud workflows. Whenever we're mobile (laptop, smartphone), we leverage a VPN to connect to our internal systems.
We recently started to slowly migrate more of our infrastructure to the cloud and free up internal resources for other work, starting with our source code management.
Our new setup looks like this:
- GitHub for our source code (private repositories within an organzation)
- An IP-range protected instance of a sonatype Nexus Repository to host our deployed packages (Maven, npm, etc.) within our own DMZ
- An IP-range protected Jenkins instance with worker nodes within our own DMZ
- IP-range protected VMs to run our software (for testing purposes) within our own DMZ
"[...]a physical or logical subnetwork that contains and exposes an organization's external-facing services to an untrusted, usually larger, network such as the Internet. The purpose of a DMZ is to add an additional layer of security to an organization's local area network (LAN): an external network node can access only what is exposed in the DMZ, while the rest of the organization's network is firewalled. The DMZ functions as a small, isolated network positioned between the Internet and the private network and, if its design is effective, allows the organization extra time to detect and address breaches before they would further penetrate into the internal networks."
Taken from this Wikipedia article
First of all, we wanted Jenkins to build all of our Pull Requests and GitHub to have checks for those builds enabled.
Whenever a build fails, no one should be able to merge, thus making a successful build a basic requirement, next to the approval of at least one team member.
For this article, let's assume that you already have a working instance of Jenkins installed. Getting a coordinator and some workers up is worth an article for itself.
At the time of writing this, the version of Jenkins used is 2.241
Your Jenkins coordinator must be accessible from the internet, as GitHub will use a webhook to trigger the build.
If you are doing this from home, or on some remote machine you own, you will have to expose Jenkins to the internet.
After some googling, I found this article here on Medium which might help you.
We carefully exposed our Jenkins to the internet by:
- Making it authenticated
- Using a reverse proxy to only make the Jenkins webhook accessible without an IP check (we had to do this because GitHub has too many ips to whitelist them all). The web interface itself is unreachable for everyone that doesn't try to access it with a white-listed ip address.
First of all, you will need some plugins to get Jenkins to work with GitHub at all.
Those are (with the versions used):
(Actually, the Plugin defines all its dependencies pretty well, but these are the ones you will definitely need.)
This is the plugin that handles everything related to your pull request.
For me, all traditional approaches didn't work and this was the only plugin that actually did what I wanted it to do.
Keep in mind: There might be other ways to achieve this!
This plugin will expose a webhook which GitHub can later use to send meta data to.
The webhook is exposed at:
A request arriving at the hook is then used to identify a Jenkins build which is actually run then.
The plugin will (when configured):
- add a comment to your PR and ask for a review
- add a merge check which you can add to the branch protection requirements for a review
- start the build and send the result back to GitHub using its API
As of now, the plugin is looking for someone to take over maintenance and has a security issue open.
The issue was none for me, as the machine the Jenkins runs on is safe on-premise and only reachable by HTTPS due to firewall settings. Internal access is secured, as well. Not everyone has access to the machine itself.
But this could however be an issue for you:
"GitHub access tokens stored in in build.xml"
The uncertainty regarding the future of the plugin might also be a turn-off for you.
You will need credentials stored in Jenkins for two things:
- Pulling your source code from GitHub
- Using the GitHub API to make comments and push the result of the merge check
You could also work with an API token here, but for the sake of simplicity, username and password is just more straight forward.
Before doing anything more, enter your GitHub project URL.
Many posts or articles do not mention this enough, but the url is part of the routing mechanism, that decides what project to actually build when your webhook receives a message.
If this field is left blank, no build will ever run!
In order for your Jenkins to actually build your project, you have to add the git coordinates.
Go to your project and just copy the HTTPS clone link.
After that, choose the credentials you created earlier from the dropdown.
In your case, that red error message should, of course, not pop up.
If it does, there is something wrong with your credentials (check your password again!) or your Jenkins can't reach GitHub.
The goal is to build individual Pull Requests before they are merged into your main branch. As they originate from another branch as your main one, Jenkins somehow needs to get told which specific branch to build.
By adding a refspec and a branch specifier, including variables that are set by the plugin, the job will always build what is specified by the webhook trigger, that comes from GitHub itself.
You are nearly finished with setting up the Jenkins job now.
It's time to activate the plugin within your job and make all settings necessary for it to work properly.
Enable "GitHub Pull Request Builder" and tick the checkbox "Use github hooks for build triggering".
Those two are the basic settings you need to get everything working.
However, for security reasons, not everyone that creates a PR in your repository can automatically also trigger a build.
That's a great feature to prevent misuse, spam, ddos, etc.
As you can see, you have several options here to enable certain people to being able to trigger a build.
- Admin List
- Adding specific GitHub user names
- White List
- Also adding specific GitHub user names
- List of organizations
- By adding an organization, all members automatically become white listed
You could, however, just tick the option 'Build every pull request automatically without asking (Dangerous!).' and whitelist basically everybody, but I would advice against it.
It's better to add an organization or add people individually, and only yourself as the Admin.
Save your build, and you are done with Jenkins, for now.
Whenever Jenkins receives the correct payload at
/ghprbhook/, your build will get triggered.
What you now have to do is setting up your GitHub project to actually use the webhook.
Finally, under "Which events would you like to trigger this webhook?" choose "Let me select individual events." and select:
- Commit comments
- Pull requests Interestingly, if you have given the user you use for your Jenkins job admin rights (on GitHub, to the repo), the GitHub Pull Request Builder will add those hook settings itself.
Make any commit to your project on any feature branch and just create a Pull Request for testing purposes.
You should now see your Jenkins instance starting the job you created. Let it run through and ensure that it runs successfully.
If you don't have a protection rule, yet, create a new one.
Target your main/master branch or whatever is your base branch.
The important setting you want to enable is "Require status checks to pass before merging".
Under that, you should see your "Jenkins" context, if the pipe has run at least once.
Check the box and you are finally finished.
Congratulations, you've made it!
If you liked this article, let's consider becoming friends on Twitter