Overwriting error handling for third-party libraries in PHPUnit

This is a copy of an old post on my private blog from August 2022.

For a while now, PHPUnit's default configuration has been to convert notices and warnings to exceptions. Any code that would raise an error of one of the following levels: E_NOTICE, E_USER_NOTICE, E_STRICT, E_WARNING, E_USER_WARNING; will result in an exception.

We don't want to change this default behavior. If our code triggered it, we have a problem. We might not see the problem yet, but it's there, and we need to fix it before it causes a runtime issue. And this is easy to do. Since we are talking about our code base, we are in a position to address those issues.

The challenge starts when we start calling into third-party libraries. Those libraries might as well trigger the same errors. The thing is - we do not have any control over them. Sure, we could open a PR to fix the issue in upstream, but this might not be a viable solution if this library is no longer maintained or if it takes a lot of time to merge. Another option is to fork the library, patch it and use the forked for as long as it's not fixed in upstream. If a library is no longer maintained, it might be a good idea to replace it with something different.

While I think the above options are the way to go in the long term, it is sometimes not feasible to do any of that. Forking and fixing would take too much time, and we must deploy now. Yet we can't because a third-party library raises a warning which gets converted into an exception by PHPUnit, and our test fails. And it might not even mean that the other library does not work. It is usually a design issue. Someone wrote the library assuming that users will ignore such errors.

Can we somehow overwrite the default behavior but only for some parts of our application?

There is nothing built into PHPUnit. We configure the behavior we want for the whole application, and that's it. But let's have a look at how the sausage is made.

A warning: this is not how we should approach this problem under normal circumstances. I mentioned the "right" way to do it earlier. But sometimes, we need to get pragmatic and do things that are more like a temporary hack and not an actual solution.

Just before PHPUnit invokes our test case, it registers an error handler configured according to what we have in our phpunit.xml configuration file.

$this->startTest($test);

if ($this->convertDeprecationsToExceptions || $this->convertErrorsToExceptions || $this->convertNoticesToExceptions || $this->convertWarningsToExceptions) {
    $errorHandler = new ErrorHandler(
        $this->convertDeprecationsToExceptions,
        $this->convertErrorsToExceptions,
        $this->convertNoticesToExceptions,
        $this->convertWarningsToExceptions
    );

    $errorHandler->register();
}

// ...
$timer = new Timer;
$timer->start();

try {
    $invoker = new Invoker;

    // ...
    $invoker->invoke([$test, 'runBare'], [], $_timeout);
}

Based on the configuration, the error handler mentioned above will convert things like notices or warnings to exceptions. That is implemented using PHP's built-in function set_error_handle(). As an argument, it takes a callback that will get executed when an error happens. What is also important is that this function will return the previous handler upon registering a new one. That will be important for the final solution.

As I mentioned, the registration happens right before the invoker starts our tests. Every test can have a custom setup and teardown logic implemented via protected functions setUp() and tearDown(). That is where we will hook in to implement our solution.

Let's quickly have a look at what argument the error handler callback accepts:

handler(
    int $errno,
    string $errstr,
    string $errfile = ?,
    int $errline = ?,
    array $errcontext = ?
): bool

The third argument errfile represents a full path to a PHP file when the error occurred. Now, you may have noticed that this argument is optional. But this will only happen when an error is triggered from within a PHP extension. When it comes from a userland code, it will always have a full path. That is exactly how we can detect an error from our own code or a third-party library. For example, if we use composer to deal with dependencies, it will put all files under the vendor folder of our project.

To implement the final solution, we have to do the following:

1. Before the test: get the current error handler (the original one, registered by PHPUnit)
2. Before the test: register our error handler
3. In handler: if a path is inside a third-party library - do nothing (read: ignore errors)
4. In handler: if a path is in our code - call the previous error handler (so the original handler registered by PHPUnit
5. After the test: restore the original error handler (so that PHPUnit still works correctly)

This whole thing could be added to an abstract base class that would then be used as a base for all other tests:

abstract class MyBaseTestCase
{
    protected function setUp(): void
    {
        $phpUnitErrorHandler = set_error_handler(function(){});
        restore_error_handler();

        set_error_handler(
            static function (
                int $errorNr,
                string $errorStr,
                string $errorFile,
                int $errorLine,
                array $errorContext = []
            ) use ($phpUnitErrorHandler): bool {
                $vendorDir = realpath(__DIR__ . "/../vendor/");
                if (str_contains($vendorDir, $errorFile) && $errorNr === E_NOTICE) {
                    return true;
                }
                return call_user_func_array(
                    $phpUnitErrorHandler,
                    [$errorNr, $errorStr, $errorFile, $errorLine, $errorContext]
                );
            }
        );
    }
}

Because PHP doesn't have a function to get the current error handler, we register an empty handler, and as a return value, we will get the previous handler. In this case, the PHPUnit's error handler. Then, we register our error handler and pass the PHPUnit's handler to use it as a fallback. Inside our handler, we check if the file where the error happened is a third-party library. We do this by testing if it is in the vendor folder of our root project. If so, we ignore it (if it's also a notice). If not, we call the original PHPUnit handler with the original arguments.

All that's left is to unregister our custom handler on tearDown():

protected function tearDown(): void
{
    restore_error_handler();
}

As we can see, the implementation is straightforward. We only need to remember that this bypasses PHPUnit's internals. It shouldn't backfire, but please use it with caution.