The Flutter team announced at the Google IO event that was celebrated recently some web-related improvements that Flutter 3 brings. In the official keynote, they reported on the possibility of running Flutter in headless mode, and after that they showed some performance tests that presumably increase Flutter's performance on the web by using the latest standards regarding image decoding.
But is this true? And if it is, are the improvements they've made substantial enough for an end-user to notice?
Since I always like to check things out for myself, I've decided to run some performance tests comparing the difference between the previous version of Flutter, the version 2.10.5 with the new version 3 of Flutter.
In this article I am going to explain to you the specific tests that I have carried out, as well as the final result so that you can see and judge for yourself how Flutter 3 performs on the web.
Benchmark methodology
In the official keynote, the Flutter team showed some screenshots in which a Flutter application appeared to load a bunch of GIFs. I have decided to follow the same methodology and take advantage of one of the open-source projects that I already have created in which the Giphy API is consumed and a lot of Gifs are displayed. I have cloned it and made some adjustments to it to add the web platform. After that, I executed it in debug, since Flutter web does not support profile mode, and I have observed the frames per second indicator. Then I have run the same test after upgrading to Flutter 3. I have repeated this set of tests on Windows, Mac and Linux to see if there is a performance improvement on all or just some platforms.
Before starting I am going to tell you the specific specs of the machines on which I am going to run the benchmarks. All the tests will be executed in Google Chrome, in its version 101. For the Windows and Linux tests I will use a laptop configured with dual boot with an Intel i7-8750H, Nvidia GeForce GTX 1070 Mobile and 16 GB of RAM. The tests on macOS will be executed with a MacBook Pro M1 from the year 2020 with 16 GB of RAM.
The tests
Note: for a better visualization of the tests, I recommend you to watch the video version of this article.
Starting with Windows 10 we can see the previous version of Flutter loading all the trending GIFs and reaching the figure of 35 FPS. After upgrading to Flutter 3 and running the same test, I've seen the indicator go up around 39 FPS; with which we can already see that in this first test there is already an improvement over the previous version.
I have repeated this same test with Chrome on my Mac 13'' with an M1 chip. Here we can see how it reaches 45 FPS, and after upgrading we can see how it goes up to almost 60 FPS, a very substantial improvement and superior to that observed on Windows, although we have to take into account that this is a different machine with different hardware.
To finish, I have executed this benchmark on the same machine where I have Windows installed, but this time on another partition where I have Linux Mint mounted. 32 FPS, not bad although it is lower than Windows. After upgrading there is a small improvement up to 38 FPS.
Results
Now I'm going to give you a summary of all the tests. On Windows we have seen how we went from about 35 FPS to 39, which is an improvement of approximately 11%. On macOS is where I've seen the biggest difference, going from 45 FPS to 58, which equates to a 28% improvement. Finally, on Linux we have gone from about 32 FPS to 38, which represents an improvement of about 18%.
By the way, keep in mind that these tests are being executed in debug mode, without minified code, so it is possible that the performance is higher in release mode.
After having seen all this, I think we can say that there has certainly been a huge improvement in the performance of Flutter apps published in a browser. It remains to be seen if these improvements are also replicated in other browsers such as Firefox or Safari, but at least we know that it will be noticeable in Chromium browsers, so my most sincere congratulations to the entire Flutter team and its contributors for having made this excellent work.
Thank you for reading, and have a great rest of your day.
Top comments (6)
Testing performance in debug mode isn't accurate. Absolutely need to test in release mode.
I have done the tests in debug mode because profile mode is not supported on the web. I agree with you that a real serious test should be done with external tools in release mode, but for the subject of this article (which is to show the performance differences between Flutter 2 and Flutter 3) I believe that doing the test in debug mode is sufficient enough.
It really is not. Testing Flutter in debug mode is unfortunately a totally meaningless measure. As a Flutter developer I have to say it just does not say anything about production speed.
Might as well read the tea leaves.
So it would be good to repeat this test with release mode. Then it makes sense.
If you look closely at the article you will see that I do not intend to demonstrate anything about the speed in production, in fact my experiment goes in another direction, which is to observe the differences between Flutter 2 and Flutter 3, that is, to measure relatively how much has improved.
Both tests have been executed under the same parameters, with which the result is the performance differential between the two.
The adjustment that you are commenting on makes sense if what is intended is to demonstrate production performance, but that is not the objective of my article.
I am saying only production performance matters.
Dev environment performance is irrelevant. Why would the Flutter team take time to improve dev performance at all? There's no need for it and no reason, therefore even if they had made production 20x faster, the dev environment may be the exact same as before. Any speed improvements that "happen" to dev are accidental. Therefore it doesn't make sense to measure it.
It's a bit like a speed race with no tires on. You can race some cars with no tires on , and some will work better than others without tires, but it's not really measuring anything that has meaning.
Doesn't perform well in safari browser, Even worst issue with back button.