One of my recent tasks has been to take the current test suite for a large-ish Rails app and speed it up as much as possible. This is the story of how I managed to do so without compromising the integrity of the tests.
At the time of starting, our test suite contained 1347 assertions, and took approximately 12 minutes to run on a 2018 Macbook Pro.
The setup is as follows:
- The testing framework is
MiniTest, which comes set up by default in new Rails applications.
- The database is populated using thoughtbot’s
- Feature tests are done using
- Browser driver used for Capybara is
I was able to get a breakdown for how long each test took by running the following command:
bundle exec rake test TESTOPTS="-v"
The output then looked as the following example:
Displaying orders for today::Given the company does not allows soup orders#test_Soup_orders_being_hidden = 1.02 s = .
With this, I was able to figure out which tests were taking longer than others.
Normally, our tests are run using headless Firefox, as mentioned above, with
geckodriver. However, I was able to run the tests with an attached window and monitor which steps were taking particularly long.
I entered the following line into
Rails.logger.level = Logger::DEBUG
I then could watch SQL commands during my tests with the following:
RAILS_ENABLE_TEST_LOG=1 bundle exec rails test
Our biggest bottleneck when starting out was the feature tests. I was able to fix a lot of these by looking deeper into the Capybara documentation.
For example, when checking that a piece of text wasn’t visible, I’d run the following:
Which does work! It turns out, however, that this was causing Capybara to look for the text on the page until its timeout, and then return
false. When having lots of these calls in a test, it adds up quickly.
A much faster way is to do the following:
There are lots of examples for this, such as
#has_no_select?, and even
Going through our tests and replacing these checks sped things up greatly!
Don’t get me wrong,
FactoryBot is an incredibly helpful gem. The problem was with the way I was using it!
See, a lot, if not all, of the tests were using a similar database setup in terms of data.
What I was doing previously was using
FactoryBot to create a series of sample data used by tests.
This created a huge overload in terms of setting up the database. There are things that can be done, such as using transactional fixtures to dump the database between tests, but setting up the stage still takes a lot of time.
I then re-took a look at Rails fixtures. With these, I can set up a series of sample data that can be loaded for every test.
What I didn’t know , though, is that using fixtures significantly lowers the number of SQL
INSERT statements, as opposed to a single one for each factory, reducing the amount of time needed to set the stage!
These are the two biggest factors that contributed to a faster suite. Got it down from 12 minutes to 3!
This doesn’t by any means indicate that I’m done, but definitely a great set of steps forward.
There are certainly other paths I could take to speed up our test suite, such as:
- Re-considering the exhaustive nature of certain feature tests. Do I really need to check certain aspects in such detail?
- Parallel testing. We’re not on Rails 6 yet but would be nice to have!
Yo, do you have any favourite techniques to make your Rails tests faster? Do please let me know!