When testing your code you’ll come across a lot of different problems to solve. Unfortunately, an application does usually not work with strictly pure functions and without any side effects at all. So we have to mock stuff, or simulate environments, inject dependencies and a bunch of other things. If you’ve ever worked on the frontend side and tried to write tests you will have noticed that it is particularly hard to test components. One possible solution are snapshot tests.
What is snapshot testing?
The idea of snapshot testing is pretty simple. You create your component and take a snapshot if you’re done. From now on the test will fail when it detects any change in the output of the component. In those cases you have two possible solutions: Fix the component to behave as before or create a new snapshot.
What is wrong with snapshot testing?
Well, it sounds intriguing, does it not? But there are some problems with snapshot testing.
First of all, snapshot testing assumes the component was doing the right thing when the snapshot was taken. This might lead to a false sense of security. Also, snapshot tests do not say anything about the expected behaviour of a component. It’s just a simple “It does what it did before”.
And then you’d have to think about what should happen if s snapshot test fails in the future. When you’ve changed the component has been changed, then of course it should fail. But do you know if it does now what it should do? No, not based on the snapshot test. What does a developer do now? He/she just assumes everything is okay with the changes, updates the snapshot and pushes the change. Done.
So, the only thing a snapshot test is able is to notify of changes that happened in the component. Well, duh, that’s what you’d expect when a developer works on the code.
What should you do instead?
Plain and simple: Write specific tests for specific expected behaviours. Try to catch edge cases and improve the tests as you go. And especially make sure you write tests for all bugs and issues you find in the future.
Of course, we can not assume developers will think about all the possible edge cases and error states. Snapshot testing won’t catch those either. To test for ‘unpredicted’ behaviour there are techniques like random tests or fuzz testing (fuzzing).
Are snapshot tests always wrong?
No. As long as you make sure they don’t replace ‘normal’ tests. They can be helpful in addition to your usual tests to check for unexpected changes which you might not have caught otherwise. Beyond that their usefulness is in my opinion very limited, especially since they might even fail on small ‘technical’ changes - like whitespace - which would not have any impact on the functionality or actual presentation of a component.
Thanks for reading. I'm curious to hear your take on snapshot testing below. Did I miss some use cases? Let me know.
Top comments (8)
There's a big use case for snapshot tests that is not covered here and has a huge practical application:
You inherit a codebase from someone. The code is messy and ugly and old. There are no tests. You want, for example, to start refactoring it by migrating some of the files to TypeScript from JS. The application is old, people use it and it's stable. You don't have a lot of resources and want to make stuff at least a bit better. So you write a snapshot test, record how the app behaves now, and refactor while checking if something broke, adding more specialised tests in the process (because you are discovering the app yourself). If you change something, add a specialised test for it and update the snapshot, knowing that it just got slightly less important.
People will say: just bite the bullet bro, write detailed specs and write test cases for them, you will have to do it eventually. Yeah, like the people who delivered a working app without tests before me had to do it. And of course I have all the time in the world.
Snapshot tests are a great way to quickly add some security when making changes, and some is better than none.
I had exactly this issue a couple of weeks ago - we migrated our l10n lib and a given date that was returning as UTC started returning with timezone, so I didn't know which one was correct... in the end after a discussiom with the tech lead we found out that considering the timezone was correct, and fortunately this field was still not being used
One could say "At least you did catch the error." However this would be caught with explicit tests as well. Having said that... Testing dates and timezones is a topic in and of itself.
I agree, and i'll add that i even think you should not even try to use them.
Frontend testing has matured a lot.
Yeah, I guess just not using snapshot tests is where you end up if you balance effort and worth...
Thanks for mentioning axe. Especially since I was always on projects which wouldn't give a f--- about accessibility (or only said they would until it came down to actually testing or/and enforcing it).
And how about regarding ensuring not breaking features like when updating libraries?
Yeah, this is a big maybe. You might catch that something has changed... but would you know what exactly it is (maybe just a whitespace) and if it actually matters for your codebase? Write test against your actual use cases. Those should catch breaking changes in 3rd party libraries as well, if they actually matter for your usage. (Other than that we can try to use only mature and responsible enough libraries with well tested code and proper release cycles using semver...)
If you write specific tests to cover each functionality I believe it would caught breaking changes as well, wouldn't it? Because you would for example, be passing now unexpected params, or the return would be different so it would fail anyway