GitHub Webhook: our own “server” in NodeJs to receive Webhook events over the internet.

We are writing an HTTP “server” in NodeJs to receive GitHub Webhook events. We use the ngrok program to make our server publicly accessible over the internet. Finally, we set up a GitHub repo and define some Webhook on this repo, then see how our now public NodeJs server handles GitHub Webhook's notifications.

GitHub enables subscribing to receive activities occur on repositories. This is known as Webhooks. This is the official documentation page About webhooks.

To subscribe, we must have a public HTTP endpoint which understands how to process notifications from GitHub Webhook's events. We are going to write our own “server” application, in NodeJs, which implements this endpoint: all it does is logging the received notifications to the console.

To make our “server” public, GitHub recommends using ngrok -- this application enables localhost applications accessible over the internet.

Table of contents

• Environments
• Our “server” in NodeJs
• Install ngrok for Ubuntu 22.10 kinetic
• Set up GitHub Webhook and test our server

Environments

In this post, I'm using Ubuntu version 22.10 kinetic, and NodeJs version 18.7.0, and ngrok version 3.1.1.

But, please note, both NodeJs and ngrok are available under Windows 10. All material discussed in this post should also work in Windows 10, I have not tested it, but I have done something similar (using Python) under Windows 10.

Our “server” in NodeJs

The primary objective is to demonstrate the flow -- how everything works together: I'm keeping it to a minimum demonstrable piece of functionality.

After we subscribe to an event in GitHub, and whenever that event has occurred, GitHub will POST a notification to the Payload URL that we specify when setting up the Webhook. In the context of this post, the Payload URL is just simply a POST route that we implement on the server.

The Payload URL method is extremely simple: it just prints to the console whatever GitHub gives it, and sends back a text response so that GitHub knows the notification has been successfully received.

The default root route (/) is a GET, and will just simply send back a “Hello, World!” message.

I have the code running under /home/behai/webwork/nodejs.

Content of /home/behai/webwork/nodejs/package.json:

{
    "name": "Git Webhook",
    "version": "0.0.1",
    "dependencies": {
        "express": "latest",
        "body-parser": "latest"
    },
    "author": "Van Be Hai Nguyen",
    "description": "Learn Git Webhook Server"
}

We use the latest versions of Express web framework and the middleware body-parser.

To install the packages, while within /home/behai/webwork/nodejs, run:

$ npm i

Content of /home/behai/webwork/nodejs/webhook.js:

const express = require( 'express' );
const bodyParser = require("body-parser")

const app = express();

app.use(bodyParser.json())
app.use(bodyParser.urlencoded({ extended: false }))

app.get( '/', function ( req, res ) {
        res.send("Hello, World!")
    }
);

app.post('/git-webhook', function(req, res) {
    let data = req.body;
    console.log(data);
    res.send('Received!');
})

const port = 8008;

app.listen( port, function() {
  console.log( `App listening on port ${port}!` )
});

• The server listens on port 8008.
• The Payload URL's route is /git-webhook.That means the full URL on localhost is http://localhost:8008/git-webhook.
• The Payload URL method's response is simply Received!.
• The default route http://localhost:8008 responds with Hello, World!.

Run it with:

$ node webhook.js

On the Ubuntu machine, curl http://localhost:8008 on a command line, and http://localhost:8008 via a browser should respond with Hello, World!.

Install ngrok for Ubuntu 22.10 kinetic

The official page Getting Started with ngrok, describes the installation process for different operating systems. I skipped Step 1 of this instruction, since I already have a web server application of my own.

In Step 2: Install the ngrok Agent, I just ran the long and scary looking command listed under "For Linux, use Apt:".

I then completed all the instructions described under Step 3: Connect your agent to your ngrok account.

Please read through Step 4: Start ngrok. Since our server above listens on port 8008, provided that it is still running, we start ngrok with:

$ ngrok http 8008

The screen should look like the following:

https://53a0-58-109-142-244.au.ngrok.io/ is the public URL for our server above: anybody with this URL can access our server running on our private network.

The GitHub Payload URL is then https://53a0-58-109-142-244.au.ngrok.io/git-webhook.

Please note that, since we're running the free version of ngrok, every time we start ngrok, we'll have a different URL! Please be mindful of that, but for our learning purpose, this is not a problem.

From my Windows 10 machine, I request https://53a0-58-109-142-244.au.ngrok.io/ using Postman, (but a browser would do, too), I get the expected response, as seen:

ngrok also logs the request:

It appears ngrok works okay with our “server”. We can now set up GitHub Webhook, and test our https://53a0-58-109-142-244.au.ngrok.io/git-webhook endpoint.

Set up GitHub Webhook and test our server

Webhooks are local to each GitHub repo. We'll create a new repo learn-git for this purpose.

When learn-git has been created, click on Settings on the top right hand corner, then on Webhooks on left hand side, then Add webhook button on the top right hand.

For Payload URL, specify https://53a0-58-109-142-244.au.ngrok.io/git-webhook. For Content type, select application/json:

Leave everything else at default, click the green Add webhook button:

Note under Which events would you like to trigger this webhook?, we leave it at the default Just the push event. That means, this Webhook will notify our server only when we check something into this repo.

GitHub tells us that it has sent our server (i.e. to the Payload URL), a ping event:

According to the above screen, our server should have received this ping event with no problem: indeed, it logs some JSON data, and ngrok also logs a new POST request to /git-webhook endpoint:

At this point, the repo is still empty. Let's do some check in, i.e. push. The Webhook should trigger.

D:\learn-git</code> has some files. Let's initialise the repo and check them in. Note the check in message “Initial checking should have two files.” (I meant “check in” 😂):


D:\learn-git>git init

D:\learn-git>git config user.name "behai-nguyen"
D:\learn-git>git config user.email "behai_nguyen@hotmail.com"

D:\learn-git>git add .
D:\learn-git>git commit -m "Initial checking should have two files."

D:\learn-git>git branch -M main
D:\learn-git>git remote add origin https://github.com/behai-nguyen/learn-git.git
D:\learn-git>git push -u origin main

The Webhook does trigger, our server logs the notification data, note that the logged message matches the check in message above; and also ngrok records another new POST request to /git-webhook endpoint:

Back to GitHub learn-git repo, go back to Webhook area, click on the payload link as pointed to by the arrow in the following screen:

Click on Recent Deliveries tab, there are two (2) events, push and ping as we've gone through above:

Pick on the push event, then click on Response 200 tab, under Body, we should see the text Received!, which is the response from our NodeJs server:

Note that, the Request tab has two sections, Headers and Payload. The data that gets posted to our server is the Payload data: GitHub Webhook documentation should help us understand what this data means, so can we can use it correctly.

Pick a file in https://github.com/behai-nguyen/learn-git.git, edit it directly and commit. This should trigger a push event. It does. Our server does get notified, note that the messages match:

Let's sync js1.js, edit it locally and check it in properly. Command to sync:

D:\learn-git>git pull

Make some changes to js1.js locally; then check it in. Note the two messages “Test Webhook.” and “Check in from local machine via command.”:

D:\learn-git>git add js1.js
D:\learn-git>git commit -m "Test Webhook." -m "Check in from local machine via command."
D:\learn-git>git push -u origin main

We get the expected response to our server. And ngrok records four (4) POST requests to /git-webhook endpoint:

The Recent Deliveries tab (discussed before), should now have four (4) entries.

Through the screen captures presented throughout this post, it should be apparent that we can change properties of an existing Webhook, including the Payload URL.

Due to the fact that our so-called server is so simple, it will work happily with other Webhook events beside push. I have tested with Send me everything., and raised issues to the learn-git repo, the server logs notifications as it does for push. This little server is good for examining the structure of the payloads we get for different Webhook events. The GitHub documentation should have this info, but for me personally, visualising the data makes reading these documents easier.

This concludes this post. I hope you find it helpful and useful. Thank you for reading, and stay safe as always.