Testing an OpenAPI specification in PHP

Introduction

OpenAPI has become the de facto standard for API specifications. According to its description, "It is a specification language for HTTP APIs that defines structure and syntax in a way that is not wedded to the programming language the API is created in". The specification is independent of the programming language we use. Therefore, any client can check the specification and integrate with our API. The specification is a YAML or JSON file.

However, we do not need to write the specification by hand, as there are GUI editors to perform that task. We show a couple of examples of Spotlight, which provides an easy-to-use interface:

Therefore, following API design first, the developer's task is to implement the required endpoints to comply with the specification. But, how can we be sure that we are complying with the specification?

Someone could argue that it is an easy task to accomplish: we only need to follow the specification and check that our responses match the specification. But, what would happen if we refactored our code with collateral effects? In this case, we could break the contract of the API, causing failures for the clients.

The solution consists of testing the responses of our API against the specification file. We can take advantage of the functional tests to validate their responses against the specification. If we do not have functional tests, this could be a good moment to add them, even though the only thing they do is validate the responses against the specification.

In this post, we will see how to test an OpenAPI specification with Symfony's testing tools.

Implementation

Libraries

There is a package within The PHP League that allows validation of an OpenAPI specification: league/openapi-psr7-validator. This package validates requests and responses to the PSR-7 specification.

However, Symfony uses the HTTP Foundation component, which does not implement PSR-7. Therefore, we need an extra package to convert Symfony requests and responses to and from PSR-7: symfony/psr-http-message-bridge.

As the documentation states, this package only performs the conversion, so we would need a PSR-7 and a PSR-17 implementation to convert the objects to and from PSR-7. We can use the library the documentation recommends, nyholm/psr7, but there are others.

Thus, we install these three packages as dev dependencies in our project:

composer require --dev league/openapi-psr7-validator nyholm/psr7 
symfony/psr-http-message-bridge

OpenApiResponseAssert

We will implement a service that, given a Symfony Response, a route and a request method, validates those responses against the OpenAPI specification. If it does not pass the validation, it will throw an exception.

To simplify the testing, we will convert the Symfony Response to PSR-7 within this service. Thus, our tests will remain clean and will only have the logic they are testing.

The implementation of this service is as follows:

<?php

declare(strict_types=1);

namespace rubenrubiob\Tests\Common\Validation\OpenApi;

use League\OpenAPIValidation\PSR7\Exception\Validation\AddressValidationFailed;
use League\OpenAPIValidation\PSR7\Exception\ValidationFailed;
use League\OpenAPIValidation\PSR7\OperationAddress;
use League\OpenAPIValidation\PSR7\ResponseValidator;
use League\OpenAPIValidation\PSR7\ValidatorBuilder;
use Nyholm\Psr7\Factory\Psr17Factory;
use Symfony\Bridge\PsrHttpMessage\Factory\PsrHttpFactory;
use Symfony\Component\HttpFoundation\Response;

use function strtolower;

final readonly class OpenApiResponseAssert
{
    private PsrHttpFactory $psrHttpFactory;
    private ResponseValidator $validator;

    public function __construct()
    {
        $psr17Factory = new Psr17Factory();
        $this->psrHttpFactory = new PsrHttpFactory($psr17Factory, $psr17Factory, $psr17Factory, $psr17Factory);
        $this->validator = (new ValidatorBuilder())
            ->fromYamlFile(__DIR__ . '/../../../../specs/reference/blog-src.yaml')
            ->getResponseValidator();
    }

    /** @throws ValidationFailed */
    public function __invoke(Response $response, string $route, string $method): void
    {
        $psrResponse = $this->psrHttpFactory->createResponse($response);
        $operation = new OperationAddress($route, strtolower($method));

        try {
            $this->validator->validate($operation, $psrResponse);
        } catch (AddressValidationFailed $e) {
            $class = $e::class;

            throw new $class(
                $e->getVerboseMessage(),
                $e->getCode(),
                $e,
            );
        }
    }
}

We create the OpenAPI validator in the constructor, specifying the path to the specification file. It is not a good practice to have logic within a constructor. However, as this service is only for the testing environment, we allow it.

In the __invoke method is where:

• Convert a Symfony Response to PSR-7.
• Validate this Response against the OpenAPI specification for the route and request method passed as arguments.
• If there is an error in the validation, we throw an exception.

Functional test

We can now write a functional test. In this case, a functional test is a request with the whole stack, including the database and any other external service we need. We will use Symfony's WebTestCase, which simulates a web server. Therefore, we will have a client to perform requests, the Symfony\Bundle\FrameworkBundle\KernelBrowser service.

We will write a test for the request to get a book we saw in a previous post:

The test is the following:

<?php

declare(strict_types=1);

namespace rubenrubiob\Tests\Functional\Book;

use rubenrubiob\Tests\Common\Validation\OpenApi\OpenApiResponseAssert;
use rubenrubiob\Tests\Functional\FunctionalBaseTestCase;
use Symfony\Bundle\FrameworkBundle\KernelBrowser;
use Symfony\Bundle\FrameworkBundle\Test\WebTestCase;
use Symfony\Component\HttpFoundation\Response;

use function Safe\json_decode;
use function sprintf;

final class GetBookTest extends WebTestCase
{
    private const EXISTING_BOOK_ID     = '080343dc-cb7c-497a-ac4d-a3190c05e323';
    private const NON_EXISTING_BOOK_ID = '4bb201e9-2c07-4a3a-b423-eca329d2f081';
    private const INVALID_BOOK_ID      = 'foo';
    private const EMPTY_BOOK_ID        = ' ';

    private const REQUEST_METHOD = 'GET';

    private readonly KernelBrowser $client;
    private readonly OpenApiResponseAssert $openApiResponseAssert;

    protected function setUp(): void
    {
        parent::setUp();

        $this->client                = static::createClient();
        $this->openApiResponseAssert = new OpenApiResponseAssert();
    }

    public function test_amb_non_existing_book_retorna_404(): void
    {
        $url = $this->url(self::NON_EXISTING_BOOK_ID);

        $this->client->request(self::REQUEST_METHOD, $url);

        $response = $this->client->getResponse();

        self::assertSame(Response::HTTP_NOT_FOUND, $response->getStatusCode());

        $this->openApiResponseAssert->__invoke($response, $url, self::REQUEST_METHOD);
    }

    public function test_amb_book_retorna_resposta_valida(): void
    {
        $url = $this->url(self::EXISTING_BOOK_ID);

        $this->client->request(self::REQUEST_METHOD, $url);

        $response        = $this->client->getResponse();
        $responseContent = json_decode($this->client->getResponse()->getContent(), true);

        self::assertSame(Response::HTTP_OK, $response->getStatusCode());
        self::assertEquals(
            [
                'id'     => self::EXISTING_BOOK_ID,
                'title'  => 'Curial e Güelfa',
                'author' => 'Anònim',
            ],
            $responseContent,
        );

        $this->openApiResponseAssert->__invoke($response, $url, self::REQUEST_METHOD);
    }

    private function url(string $bookId): string
    {
        return sprintf(
            '/books/%s',
            $bookId
        );
    }
}

We have two tests:

• One for validating a response with an 404 HTTP code.
• Another to check a valid response, with a 200 HTTP code.

In both cases, we validate the response against the OpenAPI specification with the following line:

$this->openApiResponseAssert->__invoke($response, $url, self::REQUEST_METHOD);

We have to take into account that in this example, we should also test the response with the HTTP code 400. We do not show how to persist the fixtures required.

FunctionalBaseTestCase

To simplify our test classes, we can create a TestCase from which all other functional tests extend from:

<?php

declare(strict_types=1);

namespace rubenrubiob\Tests\Functional;

use rubenrubiob\Tests\Common\Validation\OpenApi\OpenApiResponseAssert;
use Symfony\Bundle\FrameworkBundle\KernelBrowser;
use Symfony\Bundle\FrameworkBundle\Test\WebTestCase;

abstract class FunctionalBaseTestCase extends WebTestCase
{
    protected readonly KernelBrowser $client;
    protected readonly OpenApiResponseAssert $openApiResponseAssert;

    protected function setUp(): void
    {
        parent::setUp();

        $this->client = static::createClient();
        $this->openApiResponseAssert = new OpenApiResponseAssert();
    }
}

Conclusions

With these functional tests, we tie the implementation to the documentation. It is the best way to always have the documentation up-to-date, as our code will always have to comply with the specification.

Especially if we use a CI/CD pipeline, the code in the production environment will always follow the specification; otherwise, it would not be deployed.

Summary

• We briefly reviewed the OpenAPI standard and how to write its specifications.
• We saw the problem that may arise if we do not comply to the API specification.
• We listed the libraries that test a Symfony Response against the OpenAPI specification, converting it to the PSR-7 standard.
• We implemented a service that implements a Symfony Response, converts it to PSR-7 and validates against the OpenAPI specification.
• We used this service in a functional test and generated a base class to reuse it in all our functional tests.