Let's talk about how to stabilize a complex application, back to front.
Recently I was working on a strategy to text a complex application, get some checks in place so that bugs in core processes would be caught before they could go live, even as I worked on the underlying systems (core creation and updating of information, generally).
To put it another way, I kept on breaking my own core application functionality in weird ways while trying to make improvements.
I can't tell you the number of times that I have had some core functionality (login, forgot password, the homepage, the first page a user sees) break and it took some time for me to catch it. I have broken http://ninjawars.net more times than I can count, over it's 19 years of being in existence. Usually in totally new ways.
So I wanted to get a check in place to try to prevent breakage, and get that check automated in the Continous Integration.
Well, it is time to get some stability on this beast. I built some unit tests via TDD while I was coding things, I created some basic integration tests run through python to check urls that are deployed in a local instance and check that they load. What I needed now was some End-2-End tests, with a real, true browser involved.
The Testing Pyramid
Before I get into what I did, I just want to mention that my approach was informed by the testing pyramid:
You have a lot of unit tests at the bottom, some integration tests in the middle, and fewer E2E tests at the top of the testing pyramid.
Some people I have spoken to get a little too caught up in the pyramid metaphor and say "Ok, so you have to write lots of unit tests first, right?" but nah, don't get too hung up on it being a pyramid like the pyramid of giza. Ironically the tiny tip of the pyramid (a few E2E tests) is more valuable than the wide base of the pyramid (lots of E2E tests).
DON'T GIVE IN, TEST FROM THE OUTSIDE IN!
Don't give in, test from the outside in! By this I mean don't do what's easy and write a few unit tests and stop at the layer of unit tests (there are always more unit tests you could write, so it's easy to get lost at the unit test stage). Instead consider ignoring integration tests until you know you need them, delay unit tests later, just skip right to writing E2E tests first! You can get a lot of stability value from working down from the top and up from the bottom, until you meet in the middle.
Cypress for E2E tests
So I started to set up cypress for E2E browser tests. Modern, powerful, and integrates well with continuous integration.
I ran through the core paths of the system, login, forgot password, player account profile, homepage and main page.
E2E tests are actually nice for this because knowing how to use the app allows us to create tests. We don't need as much to know how the infrastructure, the react components, the code itself is working, as long as we have a good sense of the behavior of the app itself, and how we expect it to behave. Also almost everyone is familiar with login, so testing a login page with E2E tests is highly familiar ground, a good place to start.
Test Flow
Homepage --> Login --> Main Page
Login --> Main Page --> Profile Page
Login --> Forgot Password
Login --> Main Page --> Core Page
This set of test flows for E2E tests cover a LOT of ground, if these pieces of your app are operational, at least people can get in and see your core functionality, and try to use it. So these tests flows are high value. Potentially much more valuable than testing 5-10 units of unit test functionality.
Forment Revolution
You don't have to do what they tell you. Write your E2E tests first, do unit tests later, if you can.
Just a brief final anecdote, I once worked at a company with thousands of unit tests (it had a full QA team dedicated to creating unit tests)... ...and the app still kept breaking because the app was much much more complex than the sum of it's units. If your current tests aren't working for you, don't do more of the same, get some E2E tests in place.
Thoughts on these ideas?
Top comments (0)