At Mekari, we highly encourage that developers should write tests on every projects. Testing prevents bugs in production, forces better code, and encourages developer confidence when making changes to existing code bases. We use rspec, selenium with capybara as our standard unit tests and integration tests tools.
Rspec best practice
If you're new to ruby on rails and never used rspec before, here are some links which will give little guidance to start.
Working with Large Code Bases
In a large code base, testing becomes even more important. While some argue that testing slows development down, we argue that it increases development velocity over time, especially when refactoring existing code and eliminating technical debt.
Test Driven Development(TDD) encourages running your tests frequently and often. What you might find in a large code base when running the full test suite are it may takes 25+ minutes to run 2500+ tests. That will make you unproductive when you push changes and wait test suite to finish.
In this post, I will share how we accelerate our pipeline test. For my project, we use dedicated CI (Continuous Integration) server bitbucket for running the full test suite. Every minutes, developers make changes on a branch, commit, and push changes to bitbucket. Then, create PR (Pull Request) regarding ticket and bitbucket will pull down these changes and run the entire test suite.
Increase Hardware capability
When we want to achieve performance without doing much, our first option come to improve hardware specifications. You can easily see double the non parallel performance on your test suite by upgrading your CI server from an old or low spec machine into faster one.
Parallel Tests
Running specs across multiple CPU cores can further decrease the test time for your suite. We use parallel_tests gem for this. It uses a database per thread and can run on CI servers like bitbucket. Running test suite in parallel also make hardware upgrades even more effective, scaling out to machines with more cores. Single core performance is no longer your bottleneck.
When parallelizing your suite, beware of shared state between tests that lives outside of the database such as Elasticsearch. These interdependencies need to be addressed too in parallel.
This solution solve your issue about slow running your test suite, but they come at a price.
Profiling The Specs
When you have implemented the above solutions, but you still feel slow with your test suite. Then, what you should try to consider is your database strategy setup and test suite.
Database Cleaner configuration
If you write ruby or rails code that interacts with a database and write unit test, chances are you have heard of or used the database_cleaner gem. It's a terrific gem that abstracts away the various ORM APIs for getting the DB into a "blank slate" state. Some setup needed to use it in optimal mode. Here are some article that give you idea to setup your own:
- Configuring database_cleaner with Rails, Rspec, Capybara, and Selenium
- Testing with Rspec, FactoryGirl, Faker and Database cleaner
Factory cascade
Let's say you have User model and each user has one Contact and one Location record. Your factory should be looking similar like this:
factory :user do
contact
location
end
Now, try to guess how many records are created in the database once you call create(:user)?
Each time you call create(:user), then 2 additional records are created. Yes, even with build. Now imagine that you are using this base factory across all tests. There is 2 ways to eliminate this issue.
- Explicit associations The first thing that comes to mind is to remove all (or almost all) associations from our factories:
factory :user do
# do not declare associations
# contact
# location
end
With this approach, you have to explicitly specify all required associations when using a factory
create(:user, contact: contact, location: location)
# But!!
create(:user) # => raises ActiveRecord::InvalidRecord
- Traits Your base factory should always have only these attributes that are needed to pass validation. If you sometimes need associations then consider using traits
factory :user do
name { "Zuned #{Faker::Name.unique.name}" }
username { "#{ name.parameterize.underscore}_#{ SecureRandom.hex(3) }" }
email { "#{ username + Random.rand(1000).to_s }@example.com"}
trait :with_location do
location
end
end
and you can call your factory using
create :user, :with_location
let_it_be and Anyfixture
Our specs always have relations or required company and 2 or more employees (with association to user, job title, location, and job level). Imagine if we call create :company and so on for every specs. 1 spec need 1 company + 2 employee + 2 locations + 2 job_title + 2 job_level, it means 9 records only for initial data. Can we generate those 9 records only once for every specs?
Yes, here comes anyfixture. Built in fixture are different with anyfixture. Both represent different approach. Fixture declare a static state of the data that is loaded into a test database right away and usually persists between test runs. But with anyfixture, we able to hard reload record to get fresh copy of record via refind method.
Here are our implementation with (Anyfixture)[https://test-prof.evilmartians.io/#/any_fixture].
require 'test_prof/any_fixture/dsl'
require 'test_prof/ext/active_record_refind'
using TestProf::AnyFixture::DSL
using TestProf::Ext::ActiveRecordRefind
# Fixtures are kept to be outside db transaction such that it can be leveraged
# for specs. https://github.com/palkan/test-prof/blob/master/docs/any_fixture.md
RSpec.shared_context 'company with staffs' do
before(:all) do
# register to fixture
fixture(:company) { FactoryBot.create(:company) }
fixture(:location) { FactoryBot.create(:location, company: fixture(:company)) }
fixture(:job_title_ceo) {
FactoryBot.create(:job_title, :ceo, company: fixture(:company))
}
fixture(:job_title_hr) {
FactoryBot.create(
:job_title,
title: 'HR Manager', parent: fixture(:job_title_ceo), company: fixture(:company)
)
}
fixture(:job_title_staff) {
FactoryBot.create(
:job_title,
title: 'Staff', parent: fixture(:job_title_ceo), company: fixture(:company)
)
}
fixture(:org_general) {
FactoryBot.create(
:organisation_node,
name: 'General', company_id: fixture(:company).id
)
}
fixture(:org_hr) {
FactoryBot.create(
:organisation_node,
name: 'HR', company_id: fixture(:company).id
)
}
fixture(:job_level_ceo) {
FactoryBot.create(:job_level, :CEO, company: fixture(:company))
}
fixture(:job_level_hr) {
FactoryBot.create(:job_level, name: 'HR', company: fixture(:company))
}
fixture(:job_level_staff) {
FactoryBot.create(:job_level, name: 'Staff', company: fixture(:company))
}
fixture(:owner) {
FactoryBot.create(
:employee, :owner,
company: fixture(:company),
location: fixture(:location),
job_level: fixture(:job_level_ceo),
organisation_node: fixture(:org_general),
job_title: fixture(:job_title_ceo)
)
}
fixture(:hr_admin) {
FactoryBot.create(
:employee, :hr_admin,
company: fixture(:company),
location: fixture(:location),
job_level: fixture(:job_level_hr),
organisation_node: fixture(:org_hr),
job_title: fixture(:job_title_hr)
)
}
fixture(:staff) {
FactoryBot.create(
:employee, :staff,
company: fixture(:company),
location: fixture(:location),
job_level: fixture(:job_level_staff),
organisation_node: fixture(:org_general),
job_title: fixture(:job_title_staff)
)
}
end
# Ensure that every example has a fresh copy of a record
%i[
company location job_title_ceo job_title_hr org_general org_hr
job_level_ceo job_level_hr owner hr_admin staff
].each do |object|
let(object) { fixture(object).refind }
end
end
Imagine this example too.
describe User do
let!(:user) { FactoryGirl.create :user }
it 'does something' do
# test with user
end
it 'does something' do
# test with user
end
it 'does something' do
# test with user
end
end
How many User records are created in the database? One record for each example so the answer is 3 records. To avoid it you can use (let_it_be)[https://test-prof.evilmartians.io/#/let_it_be]. This helper uses Rails transactional tests feature which means that the record is created only once at the beginning of the test and removed when the test is finished.
describe User do
let_it_be(:user) { FactoryGirl.create :user }
it 'does something' do
# test with user
end
it 'does something' do
# test with user
end
it 'does something' do
# test with user
end
end
Additional suggestion and resource
There are some condition where we doesn't need data to persists first into db like checking custom validation on model and unit test on view or mailer specs. Then if this happer, we can utilize build and build_stubbed create strategy.
let(:company) { FactoryBot.build_stubbed(:company) }
some usefull links:
- (Test Profiling)[https://test-prof.evilmartians.io]
- (Factory therapy for your ruby tests)[https://evilmartians.com/chronicles/testprof-2-factory-therapy-for-your-ruby-tests-rspec-minitest#bonus-anyfixture]
- (build and build_stubbed)[https://thoughtbot.com/blog/use-factory-girls-build-stubbed-for-a-faster-test]
Top comments (0)