GitHub Webhooks To Local Jenkins Instance With Smee

Using webhooks to let a GitHub action initiate a Jenkins deploy is an extremely useful feature. The problem is when the Jenkins instance is a local deployment. This would potentially mean opening up a port to the public internet to get things working. Thankfully smee.io has a solution that allows this to work. This article will look at using a local Jenkins, Smee, and GitHub to make the webhook solution work seamlessly.

Security Warning

As the code repository's README states, this is not a production solution. It's only meant for local development situations. Corporate solutions should consider self hosted GitHub and Jenkins inside a VPN accessible internal network. If this might not be suitable cost wise for you consider working with your networking and security teams for a customized solution.

GitHub Token Creation

The first thing that will be required is having an access token available for Jenkins to authenticate with. Direct your browser to Fine-grained tokens. I find the access control naming conventions here are a lot easier to work with. Expiration is up to you, but I generally go with 90 days. Make sure to select "All Repositories" so Jenkins can work with private repos as well. Now the options you need to concern yourself with (everything else can be left as-is):

• Commit statuses: Read and Write (so Jenkins can provide build status updates for commits)
• Contents: Read and Write (I allow write here for things like automated release publishing)
• Deployments: Read and Write
• Issues: Read and Write (For anything where I want to automated around issues)
• Pull Requests: Read and Write (Adding PR comments/labeling/etc.)
• Webhooks: Read and Write (This will allow Jenkins to setup webhooks for us, which I'll be going over)

Once you're done note down the token when it appears as it won't be visible anymore once you leave the page (and if you forget you'll have to create another one from scratch).

Jenkins GitHub Setup

Now comes the rather fun part that will be full of weird things. First off go to your Jenkins instance and:

1. Select Manage Jenkins
2. Click on "Credentials" under the security category
3. Click on anyone one of the (global) links that show up
4. Then click Add Credentials in the upper right

The credential should be created with:

• Kind: Secret Text
• Scope: System
• Secret: The GitHub token
• ID: Identifier of choice for the secret (jenkins-gh-main-token for example)
• Description: At your discretion

Now create another credential (I told you this was going to be weird):

• Kind: Username and Password
• Scope: System
• Username: git
• Secret: The GitHub token
• ID: Identifier of choice for the secret (jenkins-gh-repo-token for example)
• Description: At your discretion

The reason why this is necessary is because the main server setup requires the token as secret text. Repo checkouts on the other hand require a username/password credential (don't ask me why I have no idea either). Now that this is done it's time to setup the GitHub service in Jenkins. Go back to the "Manage Jenkins" top page and select "Plugins" under "System Configuration". Then click on "Installed Plugins" and verify you have the following ones (versions may be off):

• GitHub API Plugin
• GitHub Branch Source Plugin
• GitHub plugin
• Pipeline: GitHub Groovy Libraries

The last one is somewhat of an optional if you don't plan to store shared library code on GitHub. If you're missing any plugins simply install them under the "Available Plugins" link on the left. Once you have the plugins, head back to "Manage Jenkins" and select "System":

1. Take note of the the "Home Directory" at top, you'll (unfortunately) need this shortly
2. Scroll down until you see the "GitHub" header
3. Select "Add GitHub Server"
4. Give it a name
5. Leave API URL as-is
6. For the credentials, select the Secret Text version of the token (it's filtered by type anyways so not hard to miss)
7. Click "Test Connection" on the right to verify everything is solid
8. Click on "Manage hooks"
9. Click "Save" at the very bottom

Smee Setup

Now it's time to setup the smee service. There's no login so things are pretty quick to setup. At the main site page simply click "Start a new channel". This will take you to a dedicated page with your URL. it will look something like https://smee.io/[random string here]. Now as this is an npm package you'll need to install NodeJS. Mac and Windows users can utilize the installer. *NIX type systems can utilize a supported package. If you already have NodeJS you can skip this step entirely. Now to install smee (this should be done on the instance where Jenkins lives):

$ npm install -g smee-client

Before starting here we're going to have to make an adjustment to Jenkins due to a 3 year old Jenkins bug with a PR that no one seems to want to merge yet. Basically when you save your custom smee URL as the webhook URL it doesn't actually save the value due to type mismatch... So remember when I said to note down Jenkins home directory? Well open that directory up and look for a file called github-plugin-configuration.xml. Now edit it to add in your smee URL as the webhook override:

  </configs>
  <hookUrl>https://smee.io/[random string here]</hookUrl> 

Replacing the value of hookUrl with the random string URL that smee generated for you. Once this is done restart the Jenkins instance. If everything went well you should see the hook URL if you expand "Advanced" underneath "Add GitHub Server" in the system configuration page. Now back to the smee part, run:

$ smee -u https://smee.io/[random-string-here] --path /github-webhook/ --port 8080

Now this will redirect to localhost:8080/github-webhook/. If you use a different setup you can indicate a full path like:

$ smee -t http://192.168.1.7:8080/github-webhook/ -u https://smee.io/[random-string-here]

This will have your client setup to redirect to the local Jenkins instance.

Pipeline Creation

Now navigate to the main Jenkins page. Click on "New Item" to the left. Give it a name and we'll go with Multibranch Pipeline. This is an easy to setup way for being able to work off multiple Git branches. You can also filter which branches you want to target. From here:

• Provide a display name
• Under "Branch Sources" select "Add Source"
• Select "GitHub" from the dropdown
• Under "Credentials" select the PAT username and password credentials
• Under "Repository URL" enter the HTTPS clone URL from GitHub for the repository ( You can fork my sample Jenkins repo if you want something quick to test with )
• For "Branch Discovery" I generally select "All Branches" so I can work with PRs as well
• For "Discover pull requests from forks" I generally just select "Nobody" for Trust since I don't do forking on my personal projects
• Once that's done just select "Save" at the bottom

This will do an initial scan of the repository and do a build of all the branches that have a Jenkinsfile in them. It will also setup the webhook. You can check this by going to your repository settings and selecting webhook to get something like this:

Jenkins did this all automatically for us via the token permissions. The running smee client should also show something like this:

Now when pushes are done the GitHub repository the Jenkins job will scan it and start building the changes. Also note that if you edit the webhook in the repository settings you can also make the hook proc on other events like discussions or issues.

Conclusion

This concludes using smee.io to integrate with Jenkins and GitHub so webhooks work with local dev environments. Remember this is meant for local dev work and not production. If you're up for it as well Gitea can also be installed locally so the public access is less of an issue.