My team and I have a lot of micro services to develop and maintain, and, since the key of success for this kind of architecture is performance, it is our main concern while coding.
Unfortunalty, Symfony does not provide a simple way for monitoring performances metrics other than the Profiler.
TL;DR:
We wanted to be able to perform automatic checks on our performances:
- Number of database queries
- Execution time (and not only the response time, which include too much variables)
- Memory used
How to be fast ?
That's not the subject of this post, but I'll do a quick reminder about few requirements for a fast API, in a Symfony context:
- Have as less database queries as possible
- Don't have too much injected services
- Use as less as memory as possible
- Disable/remove all unused extra Symfony packages
- If possible, don't use Listeners
- ...
But, how'd you check that ?
Symfony provides a very handfull tool to check some of the previous bullet points.
It allows you to:
- Know the number of database queries
- Know how much memory your app is using
- Have information about performances
Amazing, right ?
When I'm coding or doing a code review, it's one of the first thing I check.
It's easy when you have a route to get all rows of a table, with a OneToMany relation to have an impressive number of queries, and that should not go to production !
What about functional tests ?
Well, your code is OK on your machine, you have the right number of queries, you know your route don't consume too much memory and that it's fast enough.
But I'm a paranoid developer, I don't trust code, not even mine. That's why I write unit tests, functional tests and integration tests.
And none of this tests allow me to assert that my route has less than 5 database queries, and that the execution time (which if different than response time) is fast enough.
I looked over, and I didn't find any easy way to do so. BlackFire.io do it, but it's expensive and a little bit too much for me.
But, hey, I'm a developer, so let's do it myself, that'll be fun !
First, where do I want to have access to this data ?
During unit tests ?
Nah, I don't want to do database calls during thoses tests, and, I prefer not have to boot the kernel
During integration tests ?
That might do the trick, but for now we don't go through CI before commiting, so that'll be triggered only before being push on production
During functional tests ?
Yes ! That'd be perfect.
We do that using Postman (and its CLI tool "newman").
I love this tool because it provide a way to test your work in progress and, when you're done, you just write tests in Javascript.
We are used to run the postman tests before commiting, during the code review (it's part of it), and, of course, before any push on production.
Accessing the data
Ok, I know when I want to have access to the metrics, I still have to figure how.
Postman cannot access the profiler, since it'd require a secondary HTTP call, and to crawl the HTTP response. So we had to figure out another way.
Why not pushing those metrics in the response header ?
- Postman can read them and perform some tests on it
- We can access them with ease, even from Chrome dev tool
- It's pretty sexy, let's admit it !
We decided that we want to have:
- The name of the controller and method called by the router
- Have the number of database query
- The memory used
- The execution time
Execution-Controller → App\Controller\MyController -> myMethod
Execution-Doctrine-Queries → 8
Execution-Max-Memory → 4194304 bytes
Execution-Time → 206 ms
How it works ?
Well, it's really simple.
- A StopWatch is started when any Controller is triggered using the
onKernelController
event, then stoped on KernelResponse. This StopWatch give us the Execution-Time AND the Max-Memory - We use the DoctrineDataCollector (as the Profiler does) to count the number of queries
Add it to the headers, and that's it !
Write some tests in Postman or whatever tool you are using, and you can assert that your route is using less than 2mb of memory and require less than 5 database queries !
laboutiqueofficielle / MonitHeaderBundle
Add monitoring data in the response's header
MonitHeaderBundle
This Symfony 3 Bundle add some extra informations to the response's header
Installation
Simple add this line to AppKernel.php
public function registerBundles()
{
$bundles = array_merge($this->servicesBundles($this->getEnvironment(), $this), [
[...]
new Lbo\Bundle\MonitHeaderBundle\MonitHeaderBundle(),
Usage
Then, inspect your headers with, for example, Postman
Execution-Controller →App\Controller\MyController -> myMethod
Execution-Doctrine-Queries →8
Execution-Max-Memory →4194304 bytes
Execution-Time →206 ms
Top comments (4)
Let me suggest a useful solution to microservices. Please look at spiral/roadrunner. It can give you high performance.
Don't forget about using APP_ENV = prod, because log and db profiling can just burn your memory :)
Well, if you do so before running your functional tests, you won't be able to use the the MonitHeaderBundle since it's based on the same components than the profiler ;)
That will require extra work but profiling performance on dev is nearly useless :/