How to use the new Symfony Maker command to work with GitHub Webhooks

Recently I've been working on a tool that would gather some open-source contribution metrics from our teams. We mostly focus on contributions on GitHub, so I started studying the API to see how I could get the relevant data I needed to process. If I wanted to get fresh data on a regular basis, I would have sent requests periodically. But polling API endpoints is not ideal, especially when the services you are accessing can come to you instead !

Enter Webhooks !

Webhooks are a pretty common way for services from the outside world to communicate with your own application. It is quite similar to the event subscriber in its design :

• A remote service declares a list of steps in its lifecycle (for github: an issue has been opened, a comment has been made on a PR, ...), and for each of theses steps it will dispatch an event containing relevant data.
• You can subscribe to any of these events, and you'll get notified when they are dispatched.

The main difference with your local event based system resides in the transport : events are sent over the network to a custom endpoint where you implement your own logic to handle the events.

Nowadays, Webhooks are widely used for a lot of different purposes (getting information on a mail delivery, get notified of the steps of a payment process, ...), and the process to create a webhook is somehow always the same :

• Expose an endpoint.
• Check if the request should be processed.
• Check if the request if authorized and well formed.
• Process the request.

To make things easier for developers, Symfony released the Webhook and RemoteEvent components in Symfony 6.3. The Webhook component focuses on making the creation of endpoint and validation of request easy, while RemoteEvent is about making the event's payload transit on Messenger and be handled by a RemoteEventConsumer, where your logic will live.

To install these components, run :

$ composer require symfony/webhook

How does this work ?

Okay, now we've installed the component, where should we start ?

First of all, let's set up our webhook so that we can effectively handle the requests that will be sent to us by GitHub.

At the time of writing, the component's documentation isn't fully released yet, so it may be a little bit confusing at first. But don't sweat : to make your life easier, a new Maker command was introduced !

To create a new Webhook, run :

$ symfony console make:webhook

The maker will ask you for the webhook name. It will be used to generate the webhook url (https://example.com/webhook/the_name_goes_here). Let’s call it “github”.

Next you'll be asked you for the RequestMatchers to use. For GitHub, we know that the events are sent via POST requests and the format is JSON, so we'll add MethodRequestMatcher and IsJsonRequestMatcher.

Now we can see that the command added some config to config/packages/webhook.yaml and created two files in our project source dir : src/Webhook/GithubRequestParser.php and src/RemoteEvent/GithubWebhookConsumer.
Hooray 🎉 ! Now we have some basis to work on.

Tweaking the code

Let's dive into the generated class src/Webhook/GithubRequestParser.php and see what changes we have to make to fit our needs.

<?php
// src/Webhook/GithubRequestParser.php
namespace App\Webhook;

use Symfony\Component\HttpFoundation\ChainRequestMatcher;
use Symfony\Component\HttpFoundation\Exception\JsonException;
use Symfony\Component\HttpFoundation\Request;
use Symfony\Component\HttpFoundation\RequestMatcher\IsJsonRequestMatcher;
use Symfony\Component\HttpFoundation\RequestMatcher\MethodRequestMatcher;
use Symfony\Component\HttpFoundation\RequestMatcherInterface;
use Symfony\Component\HttpFoundation\Response;
use Symfony\Component\RemoteEvent\RemoteEvent;
use Symfony\Component\Webhook\Client\AbstractRequestParser;
use Symfony\Component\Webhook\Exception\RejectWebhookException;

final class GithubRequestParser extends AbstractRequestParser
{
    protected function getRequestMatcher(): RequestMatcherInterface
    {
        return new ChainRequestMatcher([
            new IsJsonRequestMatcher(),
            new MethodRequestMatcher('POST'),
        ]);
    }

    /**
     * @throws JsonException
     */
    protected function doParse(
        Request $request,
        #[\SensitiveParameter] string $secret
    ): ?RemoteEvent
    {
        // TODO: Adapt or replace the content of this method to fit your need.

        // Validate the request against $secret.
        $authToken = $request->headers->get('X-Authentication-Token');
        if ($authToken !== $secret) {
            throw new RejectWebhookException(Response::HTTP_UNAUTHORIZED, 'Invalid authentication token.');
        }

        // Validate the request payload.
        if (!$request->getPayload()->has('name')
            || !$request->getPayload()->has('id')) {
            throw new RejectWebhookException(Response::HTTP_BAD_REQUEST, 'Request payload does not contain required fields.');
        }

        // Parse the request payload and return a RemoteEvent object.
        $payload = $request->getPayload()->all();

        return new RemoteEvent(
            $payload['name'],
            $payload['id'],
            $payload,
        );
    }
}

We can see that the RequestMatchers previously selected were added to a ChainRequestMatcher. We don't have anything else to do in this method 🥳.

Now in the doParse method, we see that three main steps are hinted by the comments :

• Validating the request against the secret.
• Check the request is well formed (mandatory fields are present, the expected format is respected ...).
• Returning a remote event holding the payload.

GitHub has an interesting documentation on request validation, with some snippets in Ruby, JavaScript, Python ... but no PHP 😢. No worries, I made the translation for you :

$signature = $request->headers->get('X-Hub-Signature-256');  
if (!is_string($signature)  
    || !str_starts_with($signature, 'sha256=')  
    || !hash_equals('sha256='.hash_hmac('sha256', $request->getContent(), $secret), $signature)) {  
    throw new RejectWebhookException(Response::HTTP_UNAUTHORIZED, 'Invalid authentication token.');  
}

Now, for the validation : we're expecting a variety of events to knock at our webhook's door, so we won't be too strict on format validation.

To create a RemoteEvent we'll need a name (action) and an id. That will be our minimum requirements :

if (!$request->getPayload()->has('action') || !$request->getPayload()->has('number')) {  
    throw new RejectWebhookException(Response::HTTP_BAD_REQUEST, 'Request payload does not contain required fields.');  
}

Then all that's left to do is to create and return a RemoteEvent :

$payload = $request->getPayload()->all();  

return new RemoteEvent(  
    $payload['action'],  
    $payload['number'],  
    $payload,  
);

This event will be passed over to Messenger, that in turn will pass it to your GithubWebhookEventConsumer (thanks to the #[AsRemoteEventConsumer('github')] attribute on the class).

<?php
// src/RemoteEvent/GithubWebhookConsumer.php
namespace App\RemoteEvent;

use Symfony\Component\RemoteEvent\Attribute\AsRemoteEventConsumer;
use Symfony\Component\RemoteEvent\Consumer\ConsumerInterface;
use Symfony\Component\RemoteEvent\RemoteEvent;

#[AsRemoteEventConsumer('github')]
final class GithubWebhookConsumer implements ConsumerInterface
{
    public function __construct()
    {
    }

    public function consume(RemoteEvent $event): void
    {
        // Implement your own logic here
    }
}

Here is where you'll put your custom logic : mapping to DTO, persisting to database, ... whatever fits your needs.

Finally, head to config/packages/webhook.yaml and set up a secret (a random string of text with high entropy). This being sensitive data, it should be referenced here but stored in an environment variable (see Symfony documentation).

# config/packages/webhook.yaml

framework:

  webhook:

    routing:

      github:

        service: App\Webhook\GithubRequestParser

        secret: '%env(GITHUB_WEBHOOK_SECRET)%'

Call me back

Now that we're ready to handle requests, all we need to do is ask GitHub to send us some.
The official documentation is really good so we won't detail the process here. You'll need the endpoint url (https://example.com/webhook/github) and your secret. Just be aware that you can only create webhooks for resources that you own. If you want to be notified on actions performed on a repository you don't own, you'll have to ask the owner to set it up for you. If this is not an option, you'll have to rely on good old API polling.

Going further

Is that even useful ?

You may be tempted to say that all we've done is exposing an endpoint to process a request, and that it could have been done without the webhook component.
And you would be right : you can achieve the same result with a custom controller.

But let's see the benefits of doing it the way we did :

• A single conf file with minimal and simple configuration for all our exposed endpoints.
• A clean implementation of AbstractRequestParser to handle request authorization, validation and event dispatching.
• Any service can be turned into a remote event consumer just by using #[AsRemoteEventConsumer] and ConsumerInterface.
• A seamless integration with Symfony Messenger, Notifier, Mailer (and more to come).
• All of the above was done by running a single command and writing less than 10 lines of code 🤯.

What's next ?

You may want to take a look at Github's Best practices for using webhooks .
That could make you want to :

• Add an IpsRequestMatcher to check if the request is sent from one of GitHub's official IPs.
• Use an async transport to reduce the request process time.
• Handle re-deliveries
• ...

You may even want to contribute to Symfony by improving the component or the maker, creating a Bridge to spare some trouble to future developers, ... It's all up to you to make Symfony even better !

Tips : Local webhooks

When developing, you may want to receive requests to test your code. You'll need a webhook proxy for this. I would suggest using smee.io :

• Start a new channel.
• Copy the link to the "Payload URL" in the GitHub webhook configuration form.
• Download the smee client on your local machine.
• Run smee -u https://smee.io/thispartisrandom --port 8000 --path /webhhok/github *

*For a Symfony application running on localhost:8000

Accessing the smee url from your browser will allow you to visualize webhooks deliveries : headers, payload, ... and you'll be able to replay them.
You can take advantage of this and copy the payloads and headers to create fixtures to test your webhooks !