I’m a software engineer in a small company. In short, this means I do everything related to the software used in the business, from dev-ops and CI setup, to quality assurance. Most of my time is spent developing REST API’s for our company’s clients. The only problem I have is this: I don’t feel confident in the quality of my REST API’s.
I guess some people would recommend using TDD, make a test-coverage of 100% and you’re sorted. I’ve done all of that, but it will not rescue you in production environment.
In my experience, unit tests and test-driven development work pretty well for small encapsulated segments of code. It works if you’re developing some library or part of code that performs math, for example. Generally speaking, the code that doesn’t depends on some external services. If you’re developing REST API’s, you’ll mainly rely on database or queue or cache services, e.g. an external service.
Maybe smoke tests? Yes, they’ll alert you if a known problem occurs, but they won’t solve the problem completely.
What I actually need is some type of integration test. This is the only one you can trust as it works in the production environment and covers the whole process, from the client request to the client response.
What is the best way to carry out the integration test?
The standard way is to run a script that will imitate the client’s behaviour and call the API’s methods in a specific order. This script runs on the “stage” or “production” environment under a special test account. After each run, you’ll also need to delete all the records created by the test account or recreate the whole database. Sounds difficult. You’ll also need to come up with test scenarios and describe them in code, as well as supporting these scenarios after each small change in the API’s interface. I think this approach is good for an enterprise level company with a dedicated QA team, but not for an indie developer. Indie developers need a more simple and efficient approach.
Maybe monitoring?
After looking around, my first idea was to use some monitoring service to solve my problem. Actually, my teammate wrote a whole article about Monitoring in 2019.
For a time, we tried APMs (like NewRelic and ELK stack) and “pinging” services (like PingDom, Assertible or Checkly).
The APM solutions have an agent that captures all the required data, but lacks the ability to quickly and easily define assertions. As proof, try to set up alerting in ELK stack for some unobvious case. I also found that NewRelic’s pricing policy is a bit too high for adding a batch of REST APIs.
Alternatively, there are several “pinging” services that have absolutely wonderful ways of defining assertions for each and every method of REST API, but lack access to the client’s requests and responses.
That’s why I came up with an idea:
What if I could capture each and every client request and corresponding response to my API with the help of an agent library (like an APM service)? But then let the developer define a set of assertions on each of the REST API methods (like a “pinging” service).
I call it the “indie hacker way” of doing an integration test for the whole API in just two or three clicks.
What are the pros?
You don’t need to create test scenarios and then describe them in code, as the actions of your clients are recorded for you. And it would always be up to date because the client is always right :) I mean if a client can’t call your API in the proper order, then it probably means that your documentation sucks.
Secondly, your test coverage will be around 100% because every request will go through assertions. You could argue that there would be methods that clients do not call and these would not be tested. Ok, that’s true, but do you need such methods?
Another feature of "the indie hacker way" is the possibility of rate testing. For example, you can define an assertion for the minimum or maximum number of requests per second for each and every method of your API. If something goes over the threshold, you’ll be notified.
Last but not least, you can assert response times for the customer and be confident that all requests are running under 100ms, for example.
What are the cons?
Yes, it’s not perfect. As I said above, if you have dedicated QA team then it’s better to go with integration tests in the usual sense, or configure a proper APM solution.
First of all, you’re adding some computational load to your app as a result of capturing and sending each and every request. It’s not a big deal, but for some apps it may be a problem.
Secondly, you can’t define any custom logic or dependency between methods in the test.
Implementation
Inspired by this idea, I quickly developed a web service to define assertions and receive notifications. It’s called SLAO. I use it on a daily basis and have added it to all my REST APIs.
How does it work?
- Install agent to capture requests (now only node.js/express is supported)
- Define assertions in simple web interface
- Receive notification if something goes wrong
If you like the idea, try it now on 📊 SLAO and provide your feedback via chat or via email hello@slao.io
Originally posted: https://slao.io/blog/posts/new-way-of-testing/
Top comments (0)