Implementing PSR-18 and extending it with middleware

With the current trend of using microservices chances are that the majority of your projects either perform http calls, or use one or more packages that do so. And thus chances are that you are either using Guzzle or another http request library or abstraction, directly or via a package. This causes a hard dependency to this library and maybe even to multiple libraries. You might even run into the issue of being unable to use or upgrade a package due to a version mismatch. The PHP-FIG was founded with the intent to handle these kind of situations. And with the introduction of PSR-18 the entire HTTP stack was "standardized" via PSR's; with PSR-7, PSR-15, PSR-17 and PSR-18 it is now possible to make completely framework agnostic packages and applications. And for us the introduction of PSR-18 came at the ideal moment as we were preparing to migrate from Guzzle 5 to Guzzle 6. When we learned about PSR-18 we decided to skip Guzzle 6 and go directly for a PSR-18 compliant implementation.

So the issue that PSR-18 solves is having a hard dependency to a proprietary package and subsequently a proprietary interface¹. But an issue that PSR-18 deliberately does not solve is the configuration of the clients. This was deemed too complex and implementation bound. And I agree with this. This "lack of configuration" is used by some to dismiss the usefulness of PSR-18 over for example Guzzle. Features like retry mechanisms and logging are not considered in PSR-18. Why introduce a standard to substitute a pretty much defacto standard like Guzzle? But that does not mean the majority of these functionalities are not easily added to a PSR-18 compliant client. The solution I eventually opted for was middleware.

A small disclaimer. I personally very much like the single purposeness of the PRS-18 interface - perform a http request - over the extended functionality Guzzle offers. That does not mean that I want everyone to immediately switch away from Guzzle to PSR-18. For me and my colleagues switching from Guzzle 5 to PSR-18 - powered underneath by Guzzle - felt like the right move. More so because we also use and maintain multiple internal Composer packages that also perform http requests. By introducing a new interface we were able to gradually migrate away from Guzzle 5. If you are not keen on PSR-18 or adamant to keep using Guzzle, maybe this article is not for you. No hard feelings.

Introducing PSR-18

The issue deliberately not solved in PSR-18 is creation and thus configuration of the client. But I want to be able to inject an instance of ClientInterface into my code. The logical solution for this is to create a factory and have the dependency injection container create clients for me. An additional benefit is that I can change properties of my client per environment. A first investigation of the existing code revealed usage of only three options; timeout, connect timeout and base url. Easy enough:

<?php

declare(strict_types=true);

use Psr\Http\Client\ClientInterface;

final class ClientFactory 
{
    public function create(
        float $timeout, 
        float $connectTimeout, 
        string $baseUrl
    ): ClientInterface
    {
        // ...
    }
}

And in my dependency injection container²:

services:
  http.client.factory:
      class: ClientFactory

  http.client.my_service:
      class: ClientInterface
      factory: ['@http.client.factory', 'create']
      arguments:
       - '%http.client.my_service.timeout%'
       - '%http.client.my_service.connect_timeout%'
       - '%http.client.my_service.base_url%'

The reason I create a client per service is that we work with a lot of people, that work on multiple teams in the same large codebase. By creating a client per service I can minimise the possibility of a change by one team unwillingly affects the code of another team. It is of course possible to vary on the implementation; you can create default values and specify them on the constructor of the factory or use these defaults explicitly in the creation of your own client. Additionally this allows for different configurations per environment.

This approach will make that the code that calls the client does not have to worry about these configuration options. This will lead to simpler and cleaner code, as you don't have to worry about the configuration options of your client.

Soon after migrating the first Guzzle clients to PSR-18 compliant clients I was wondering whether logging of any client exceptions could be centralized. And maybe I could add monitoring to the clients. And how about authentication? I didn't want to enable these functionalities by adding more parameters to the factory. In an attempt to extract these functionalities I investigated the possibilities of decorators. This lead to a nesting structure and so with incomplete services in the container. Looking for an alternative I took inspiration from PSR-15. More specific in the middleware.

Introducing middleware

To log any errors I need to execute code after the actual http call, to add authentication before the http call and to add monitoring - timing, success rate - both before and after the http call. This flexibilty is offered by middleware. Inspired by PSR-15's middleware I came up with a simple interface:

<?php

declare(strict_types=1);

namespace Coolblue\Http\Client;

use Psr\Http\Client\ClientInterface;
use Psr\Http\Message\RequestInterface;
use Psr\Http\Message\ResponseInterface;

interface MiddlewareInterface
{
    public function process(RequestInterface $request, ClientInterface $client): ResponseInterface;
}

And I updated the factory to accommodate the addition of middleware to a client:

<?php

declare(strict_types=true);

use Psr\Http\Client\ClientInterface;

final class ClientFactory 
{
    public function create(
        float $timeout, 
        float $connectTimeout, 
        string $baseUrl, 
        ClientMiddlewareInterface ...$middleware
    ): ClientInterface
    {
        // ...
    }
}

Now when I need authentication on my request my dependency injection container would look like this:

services:
  http.client.factory:
      class: ClientFactory

  my_service.http.middleware.authentication:
      class: MyService\AuthenticationMiddleware
      arguments: ~    

  http.client.my_service:
      class: ClientInterface
      factory: ['@http.client.factory', 'create']
      arguments:
       - '%http.client.my_service.timeout%'
       - '%http.client.my_service.connect_timeout%'
       - '%http.client.my_service.base_url%'
       - '@my_service.http.middleware.authentication'

The nice thing about this is that the code that uses the client doesn't need to be changed. Do remember however that - just as with PSR-15's middleware - the order in which the middleware is executed can change the results of the http call, as the middlewares are called sequentially.

As I was working on this approach I started to think about what configuration options could not be achieved using middleware. I think the configuration options are divideable into two group; low level client configuration, eg. timeouts, and higher level client configuration, eg. base url, logging and retry mechanisms. For the first group there is no other way than to configure this on the actual client. The creation of a client is easily achieved by using the factory pattern. The second - and largest - group can be achieved by modifying the outgoing request, the calling mechanism or modifying the incoming response.

Taking these criteria to heart I removed the base url from the factory and moved it to a middleware. This introduces another service per client, but also introduces a more transparent call stack; if the base url middleware is set before any logging middleware I can always get the complete and full url that is being requested. Something that would be hard if the base url is embedded in the client as there is no method in the interface to request the base url from the client:

<?php 

declare(strict_type=1);

use Coolblue\Http\Client\ClientMiddlewareInterface;

final class BaseUrlMiddleware implements ClientMiddlewareInterface
{
    /** @var string */
    private $baseUrl;

    public function __construct(string $baseUrl)
    {
        $this->baseUrl = $baseUrl;
    }

    public function process(RequestInterface $request, ClientInterface $client): ResponseInterface
    {
        // Very naive implementation of adding a base url
        return $client->sendRequest($request->withUri(
            $this->baseUrl . $request->getUri()
        ));
    }
}

Taking the approach of using middleware I am able to have a true single purpose class. This typically means a smaller and reusable class and a class that is easy testable.

Conclusion

PSR-18 offers a very clean approach of performing http calls. This also means that the additional built-in features, that a library like Guzzle offers, are not part of the PSR and will therefore not be a part of a PSR-18 compliant client. But a lot of this functionality can be achieved by using middleware on the ClientInterface. By implementing a simple interface the separate middleware can be built following the SOLID principles.

Whether you should use PSR-18 over a library like Guzzle, with of without the addition of middleware, is a matter of preference. From what I read on sources like Reddit and Twitter not everyone is convinced of the usefulness of PSR-18. I personally feel that PSR-18 offers a very clean approach of performing http calls. And due to the way the ClientInterface - and RequestInterface - is constructed it discourages configuration of the client in the code that also performs the request. This keeps your code simpler. But this leaves the issue of configuring a client. You can opt for a factory that allows for a multitude of configuration options. Or you can opt for using middleware.

We have been using middleware for a few weeks now and so far we have had no issues in functionality of performance. By using the approach described in this article I found the client creation process more flexible. By using the DI container to configure the client modifying the client per environment is no (longer) part of the calling code, but part of the configuration. As it should be in my opinion.

Addendum

At the time this article was published the middleware solution was still an internal solution. Early 2020 we decided to open source our solution. The package coolblue/http-client-middleware is available on Packagist and Github. It contains both the MiddlewareInterface and a PSR-18 compliant client implementation for using middleware implementations.

At Coolblue we've been using the middleware in multiple codebases for over a year now. During this year we have come to some insights about this middleware solution. In the article BaseUrlMiddleware is used as example. This makes that the code using the client does not have to worry about the base url per environment. Another middleware we used was an authentication middleware that would add an authentication header to requests. This made the code smaller and simpler, but at the same time this moved a vital part of the functionality to configuration of the dependency injection container instead of tying it directly to the code that called the client. This can be seen as a hidden requirement and we've deemed that undesirable. We now have the requirement that middleware should only add non-essential functionality. In other words: the code should still function if middleware is not added. This still leaves a lot of use cases for which we have been using middleware: retrying of failing requests, adding logging and monitoring to requests, caching of responses based on cache policies and caching of responses based on response headers. That way we are still benefiting from the middleware while also keeping our repositories and services responsible for the entire request flow.

¹ This risk that comes with a proprietary interface of course also exist with the interface defined by a PSR. But since PSR-18 consists of a relative simple interface I think this interface will suffice in our needs. And if needed we will be able to quickly replace it with a custom interface just like it.

² I am using Symfony's Service Container component for this.