I've worked on many codebases that used what I consider to be one of the worst patterns in automated test architecture:
Fixtures.
Yuck.
Fixtures are a set of statically defined objects which might be injected into your database during your test run, or might just be passed to various functions in your tests. They'll either be raw .json files, or they could be defined in code, looking something like this:
export const REPOS = [
{
id: "1",
slug: "use-firestore",
starCount: 100,
owner: "erikpukinskis",
},
{
id: "1",
slug: "codedocs",
starCount: 0,
owner: "erikpukinskis",
}
]
They will often be associated with one another, so that when they're injected into your database there will be some referential integrity. So in the example above we might also have fixtures like:
export const USERS = [
{
id: "1",
username: "erikpukinskis",
}
]
The problem is these fixtures will tend to be used to set up many different tests. And so the fixture suite gets bigger and bigger, with lots of relationships between fixtures. It either gets really complicated to get the database into a "good" state, or you end up with a function like this:
import { setUpAllTheThings } from "~/test/helpers"
describe("my new test suite", () => {
beforeAll(async () => {
await setUpAllTheThings()
})
})
That's easy enough, but it misses out on one of the Properties of a Good Test:
- A Good Test describes its preconditions
- A Good Test describes its expectations
- A Good Test only fails when things are broken
The problem with setUpAllTheThings() is it doesn't tell you what the conditions for success were. It just gestures at a big pile of fixtures that you have to dig through and guess what was important.
The solution
Instead of static fixtures, your data-dependent tests should use functions called factories. Over the years I have come to identify some important features for great test factories.
This post will teach you those patterns in three parts:
What makes a factory?
1. Factories are imperative
Unlike "fixtures" which are set up once, globally for all tests, factories are called imperatively during test setup:
import * as factory from "~/test/factory"
test("should render a repo heading", async () => {
const { repo } = await factory.setUpRepo(testApp)
const { findByRole } = render(<Repo id={repo.id} />)
await findByRole("heading", repo.name)
})
This means that your test can specify the smallest number of records that need to be set up for the test to work. When the test fails, you can immediately see what the specific conditions being tested were, eliminating a bunch of guesswork.
2. You can override everything in a factory
When you use record from a test fixture, it has every single field set. It also probably has every possible association specified. And so it's never clear which fields or associations are required for the test to make sense.
With factories, you can specify exactly which fields are preconditions for the test's success:
This lets us make a small improvement on the test above, making it more explicit that we're testing a repo with a specific name:
import * as factory from "~/test/factory"
test("should render a repo heading", async () => {
const { repo } = await factory.setUpRepo(testApp, {
name: "My Test Repo"
})
const { findByRole } = render(<Repo id={repo.id} />)
await findByRole("heading","My Test Repo)
})
3. Factories auto-generate associations
One of the key jobs of a factory is to ensure data consistency by creating any and all associations needed for the database schema to be happy.
This allows you to specify only the objects needed for your test preconditions without having to bootstrap a bunch of context:
import { User } from "~/server/data"
test("repos should have owners", async () => {
const { repo, owner } = await factory.setUpRepo(testApp)
expect(owner).toBeInstanceOf(User)
expect(repo.ownerId).toBe(owner.id)
})
This gives you all the ease-of-use of a giant setUpAllTheThings() function, without over-specifying your preconditions.
Factories let you override associations
But even better, factories can pick and choose when to do automatic setup and when to set up relations manually.
For example, you might want to test some aspects of how repos and and users work together. The setUpRepo factory can automatically create a user for you, but if your test depends on a user with some specific traits you can also do it yourself:
test("only owners can see repo details", async () => {
const { user: owner } = await factory.setUpUser(testApp)
const { user: otherUser } = await factory.setUpUser(testApp)
const { repo } = await factory.setUpRepo(testApp, {
owner,
name: "My Test Repo",
})
const { findByRole, queryByRole, findByText, rerender } = render(
<SignInProvider userId={owner.id}>
<Repo id={repo.id} />
</SignInProvider>
)
await findByRole("heading", "My Test Repo")
rerender(
<SignInProvider userId={otherUser.id}>
<Repo id={repo.id} />
</SignInProvider>
)
await findByRole("heading", "404 Not Found")
expect(queryByRole("heading", "My Test Repo")).toBe(null)
})
All of this is in service of the same goal: Tests with explicit preconditions.
What my factory functions look like
This is the typical pattern I use for setting up a factory:
export type Repo = {
id: string
slug: string
starCount: number
tagIds: string[]
ownerId: string
}
let repoCount = 0
type RepoOverrides = Omit<Partial<Repo>, "id"> & {
owner?: User
}
export async function setUpRepo(
app: FirebaseApp,
overrides: RepoOverrides = {}
) {
// It's useful to number the calls to the factory, to have a unique identifier
// that also indicates the order of creation:
const uniqueId = ++repoCount
// Split out the associations from the object properties:
let { owner, ...propertyOverrides } = overrides
// Create any associations that are missing:
if (!owner) {
owner = await setUpUser()
}
// Now create the primary object
const ref = await addDoc(collection(getFirestore(app), "repos"), {
// Provide fallback values:
slug: `repo-${uniqueId}`,
starCount: 0,
tagIds: [],
// Then the overrides from the user:
...propertyOverrides,
// And lastly any associations:
ownerId: owner.id,
})
const repo = {
id: ref.id,
...propertyOverrides,
} as Repo
// Finally, return the primary object along with all of its assocations
return { repo, owner }
}
And that's it. You can build out a large factory system using just those few patterns.
I don't tend to use factory libraries like Fishery. That's a great library, but I don't find it saves much code.
Additional factory patterns I love
Factories should return a set of associated objects
Sometimes folks will set up factories to return just a single object, or maybe an object with a bunch of associations attached to it.
Because "factories auto-generate associations", I recommend returning all of the associations along with the primary object. This allows you use those associations elsewhere in your test:
test("can navigate to repos path", async () => {
const { repo, owner } = await factory.setUpRepo(testApp, {
name: "My Test Repo"
})
await signInAs(owner.email, owner.password)
await goToPage(`/repos/${repo.id}`)
expect(screen).toHaveRole("heading", "My Test Repo")
})
Conclusion
A solid set of test factories can really accelerate your team in a few ways:
First, they speed up debugging time when tests fail. They make it super clear what scenario is under test, which helps developers spend less time trying to understand a test failure.
Second, they make it much easier to bang out new tests. This gives you better coverage which usually means more stability in your code. It also makes developers more likely to delete obsolete tests, since they know it'll be easy to add back the correct ones.
And lastly, factories reduce the dependencies between tests, which means less time playing whack-a-mole when you change a fixture to get one test to pass, and it breaks another test.
Top comments (0)