Edit:

I went ahead and ran some performance tests again with optimizations with better results.

Preface

This is the first post in a series of articles I plan to write where I will be looking at a variety of server side web frameworks or using no framework in several languages.

Goals

My primary goal is to highlight a few server side frameworks and to give enough information so that you, the reader, can make a judgement as to whether or not you should do more investigation for your use case.

Here are some goals:

• Test flexibility
  • Does it impede dev?
• SQL compatibility
  • Can you use raw sql
  • Is the ORM, if present, good for almost any type of query?
• Quick response time performance for a multi join api endpoint
  • How many concurrent users can the framework support before response time is too high for good UX.

Setup

I have done everything in docker to make it easy to keep things separated. For load testing I will be using k6.

Repo: https://github.com/buphmin/symfony4-impressions
Load tester:

docker pull loadimpact/k6

As a reference for the performance portion here is how I am testing

Hardware/Hardware Config

My hardware is faster than the majority of people so your mileage may vary on the benchmarks. I am also running everything within a linux mint 19 VM. Specs are as reserved by the VM.

i7 8700k running at 4.9ghz 6 cores
8GB ram running at 3600mhz

Cores 0-1 reserved for the web server
Cores 2-3 reserved for the database
Cores 4-5 reserved for k6

Impressions

So as a disclaimer I have worked with Symfony for many years, so this is hardly a first impression for me. I do feel, however, that I can objectively qualify Symfony's strengths and weaknesses in a way that you can determine whether or not it warrants your attention. You should always judge a technology for yourself and not blindly think it is good just because (insert famous person here) said so.

What is Symfony

Symfony is a full featured full stack framework designed to give you an all in one solution for web development. Symfony attempts to be a one size fits all type of framework with built in authentication, server side rendering/templating, api support, etc.

Flexibility

In terms of flexibility Symfony is quite flexible, especially for a fully featured framework. That does not mean Symfony is perfect though. You have to live with the config files that Symfony requires and set them up in the Symfony way. Additionally by default Symfony expects you to put your files in specific locations forcing you to

Symfony allows you to override many of it's conventions but you will likely find troublesome. For instance you can override the directory structure, but then likely be confused when the docs assume you are using the defaults.

For SQL Symfony ships with Doctrine for use of both the ORM and DBAL (DataBase Abstraction Layer). Both can be used exclusively or simultaneously. The doctrine ORM is pretty good and will allow you to use most database structures. A notable weakness is composite foreign key relations as primary keys which either cannot be done or will take some creativity to work around. The doctrine mapping can take some time to setup but Symfony comes with some tools to make the process simpler if you have already structured your database. On the flip side you can use the Doctrine DBAL for raw SQL. The DBAL is just a light wrapper around PDO, PHP's native database connection library that provides some convenience methods. Symfony also has the ability to use other connection interfaces, such as to connect to MongoDB, but the Symfony docs shy away from that and I will not cover it here. Between the the ORM and DBAL you have ways to work with a SQL database however you need.

Here are some examples of raw SQL vs ORM endpoints with multi join:

    /**
     * @param integer $id
     * @param Connection $conn
     * @return JsonResponse
     * @Route(path="/league_players/raw/{id}", name="league_player_raw")
     */
    public function getLeaguePlayerRawAction($id, Connection $conn)
    {
        $sql = "
            select * from 
            league_player lp
            join league l on lp.league_id = l.id
            join player p on lp.player_id = p.id
            where lp.id = :id
        ";

        $leaguePlayers = $conn->fetchAll($sql, [
            'id' => $id
        ]);

        if (count($leaguePlayers) === 0) {
            throw new NotFoundHttpException('Player not found');
        }

        return new JsonResponse($leaguePlayers[0]);
    }

    /**
     * @param LeaguePlayer $leaguePlayer
     * @param SerializerInterface $serializer
     * @return Response
     * @Route(path="/league_players/{leaguePlayer}", name="league_player")
     */
    public function getLeaguePlayerAction(LeaguePlayer $leaguePlayer, SerializerInterface $serializer)
    {
        $data = $serializer->serialize($leaguePlayer, 'json');
        return new Response($data);
    }

To end the flexibility section it is interesting to note that the Symfony framework is build from a motley of independent reusable components. Each of these can be used outside of the Symfony framework as a whole. For instance if you just need a router utility then you can use the Symfony routing component and manage the rest of the stack yourself.

Performance

I will be judging the two endpoints above to judge the raw SQL implementation and the ORM implementation. The ORM implementation is slightly different, but is a realistic implementation of what you might do with an ORM endpoint.

1 Concurrent User

Raw:

docker run --network host --cpuset-cpus "4-5" -v $PWD/loadTests:/src -i loadimpact/k6 run --vus 1 --duration 10s /src/getLeaguePlayerRaw.js

Result:

http_req_duration..........: avg=4.51ms  min=3.51ms  med=4.46ms  max=5.76ms   p(90)=5.09ms  p(95)=5.42ms

Pretty good performance, 11.73ms duration. Obviously since this is local to local this is not exactly "real" world performance but it gives us a rough idea.

ORM:

docker run --network host --cpuset-cpus "4-5" -v $PWD/loadTests:/src -i loadimpact/k6 run --vus 1 --duration 10s /src/getLeaguePlayer.js

Result:

http_req_duration..........: avg=16.74ms min=15.06ms med=15.98ms max=21.15ms  p(90)=19.43ms p(95)=20.29ms

The ORM implementation is close to double the request time in this scenario.

10 Concurrent Users

Change --vus to 10

Raw:

 http_req_duration..........: avg=8.81ms  min=3.24ms med=8.35ms max=19.11ms  p(90)=12.5ms  p(95)=17.18ms

ORM:

http_req_duration..........: avg=51.44ms min=14.81ms med=51.05ms max=82.89ms  p(90)=76.4ms  p(95)=80.5ms

The extra load of the ORM method is becoming evident.

50 Concurrent Users

Change --vus to 50

Raw:

http_req_duration..........: avg=34.75ms  min=4.57ms med=31.57ms max=90.81ms  p(90)=58.4ms  p(95)=76.5ms

ORM:

http_req_duration..........: avg=201.94ms min=15ms    med=216.65ms max=1.69s    p(90)=334.99ms p(95)=369.24ms

Edit:

Performance is pretty good with raw SQL in my tests at 50 concurrent users. I will note that I saw a very wide variation between runs. Most runs were between 28-45ms but some shot to 400ms, which could just be the limitations of my testing environment. With the implemented optimizations performance I was able to get around 130ms response time at 200 concurrent users. This is good but still trails behind anything in node.js as we will see in future articles ;)

Here we are at the limit of what 2 cores can achieve. 500ms is too much for good UX in most circumstances. We find that the ORM adds extra overhead over the raw SQL implementation which is to be expected.

Results

My impression of Symfony is that it quite flexible and gives you a lot of ways to go about your business.

Pros

• flexibility
• good docs
• mostly stays out of your way

Cons

• some limitations with the ORM
• high concurrent user performance is not great

Recommendation:

I would say Symfony is worth taking a look at for most people who are interested in a framework. It offers good flexibility and pretty good low load performance which makes it ideal for an internal company app.

Let me know if you have any insights or questions as this will be my first fairly in depth article.