Automated testing has become an essential part of software development.
However, I still very much dislike writing tests and recently, I found out I’m not alone. I posted a poll on Reddit to see if other devs think like me and, thankfully, I’m not alone.
76% out of 1046 developers hate writing automated tests.
Imagine choosing between:
- writing automated tests for a feature you created
- or improving the feature / starting the implementation of another feature
Most of us would prefer not to write tests but rather work on the core codebase but we know that a disaster awaits us if we don’t have any written tests. So, we just bite the bullet and write them.
All of this got me thinking if all of this is really necessary. Having tests definitely is necessary but do we really have to write them manually?
So, I got to work with my friend and now, 2 months later, we released Pythagora – an open source npm package that creates automated integration tests by recording and analyzing server activity without you having to write a single line of code. Within 1 hour of recording, it can generate tests with 90% code coverage.
In this blog post, I’ll show you how you can create hundreds of automated integration tests within minutes and how does all that work.
If you’re a video person, here’s a quick 2 minute video of how Pythagora works:
🏗️ How it works
Pythagora works by capturing API requests (with responses) and server activity during the processing of the request. From that data, it creates automated tests that you can run from any environment/machine.
When an API request is being captured, Pythagora saves all database documents used during the request (before and after each db query).
When you run the test, Pythagora first connects to a temporary pythagoraDb database and restores all saved documents. This way, the database state is the same during the test as it was during the capture so the test can run on any environment while NOT changing your local database. Then, Pythagora makes an API request tracking all db queries and checks if the API response and db documents are the same as they were during the capture.
For example, if the request updates the database – then, after the API returns the response, Pythagora checks the database to see if it was updated correctly.
⚙️ Setup
It’s super simple to set up Pythagora for a Node.js server. Just install the npm package with
npm install pythagora
and add one line of code right after you initialize Express. For example, if you initialize it with const app = express(); – then you add the following line of code.
if (global.Pythagora) global.Pythagora.setApp(app);
That’s it, you’re ready now to start capturing requests.
🎥 Capturing requests
Now, you just need to run the pythagora command and pass the .js file you’re usually running when starting your server. For example, if you run your server with node ./path/to/your/server.js, you would run Pythagora like this:
npx pythagora --initScript ./path/to/your/server.js
This command will start your server but wrapped with Pythagora. Now, to get Pythagora to create automated tests, you just need to send API requests to your server.
You can do this in any way you’re usually sending requests while developing or testing. You can use your browser and click around your frontend, send requests through Postman or cURL or any other way you’re used to.
An example of running a server with Pythagora and clicking around the frontend to trigger the API requests – notice how the console is showing all requests that are being captured in real time
Pythagora records all requests to endpoints of the app with the response and everything that’s happening during the request. Currently, that means all Mongo and Redis queries with responses (in the future 3rd party API requests, disk IO operations, etc.).
You can see all captured tests with metadata in the pythagora_data folder in the root of your project. By having this folder present, anyone from your team can run these tests regardless of the database they are connected to.
🚦Running tests
When running tests, it doesn’t matter what database is your Node.js connected to or what is the state of that database. Actually, that database is never touched or used —> instead, Pythagora creates a special, ephemeral pythagoraDb database, which it uses to restore the data before each test is executed, which was present at the time when the test was recorded. Because of this, tests can be run on any machine or environment.
If a test does an update to the database, Pythagora also checks the database to see if it was updated correctly.
So, after you captured all requests you want, you just need to add the mode parameter --mode test to the Pythagora command to run the captured requests.
npx pythagora --initScript ./path/to/your/server.js --mode test
📝 Code coverage report
Code coverage is a great metric while building automated tests as it shows us which lines of code are covered by the tests. Pythagora uses nyc to generate a report about code that was covered with Pythagora tests. By default, Pythagora will show you the basic code coverage report summary when you run tests.
If you want to generate a more detailed report, you can do so by running Pythagora with --full-code-coverage-report flag. For example:
npx pythagora --initScript ./path/to/your/server.js --mode test --full-code-coverage-report
You can find the code coverage report inside pythagora_data folder in the root of your repository. You can open the HTML view of the report by opening pythagora_data/code_coverage_report/lcov-report/index.html.
In case you don’t want the code coverage to be shown at all while running tests, you can run the tests with --no-code-coverage parameter.
🎚 Current limitations
Right now, Pythagora is quite limited and it supports only Node.js apps with Mongoose and Express but we’re working hard to extend this support to other technologies and languages. In case Mongoose is not used in the codebase, Pythagora will capture API requests and responses without testing the database.
To support the most important technologies, it would mean a lot to us if you leave a comment or send us an email at hi@pythagora.io with technologies you use that you would like to be supported.
Conclusion
Most developers, including me, don’t enjoy the task of creating and maintaining tests. This is the problem we’re trying to solve with Pythagora.
Pythagora is an open-source npm package that can record and analyze server activity to create tests that mimic the behavior of your codebase. Using it, you can save time and effort and generate up to 90% code coverage in as little as one hour. This means you can spend more time working on the core codebase and less time worrying about writing and maintaining tests.
You don’t have to worry about what database your server is connected to, or the state of that database. Pythagora creates a special, ephemeral pythagoraDb database, which it uses to restore the data before each test is executed, which was present at the time when the test was recorded. This means that tests can be run on any machine or environment, which makes testing much more efficient.
Anyway, that’s it. I hope you liked it. We’re still in a super early stage so any feedback is much appreciated.
How do you write integration tests for your API server? Would you consider using Pythagora instead/along with your system?
If not, I’d love to hear what are your concerns and why this wouldn’t work for you?
Also, if you like Pythagora, it would mean a lot if you could give us a star on Github – https://github.com/Pythagora-io/pythagora
To get an update about the beta release or to give a suggestion on tech (framework / database) you want Pythagora to support you can 👉 add your email / comment here 👈
Top comments (0)