As a developer, I’m always on the lookout for new tools and technologies that can make my work easier, more efficient, and enjoyable. So, after taking a 1-year break after my last company was acquired, I decided to build the next product with my best friend.
So, over the last year, we created four different dev tools in search of the perfect fit. We tested all of them on the market and while each tool had its strengths and weaknesses, none of them quite hit the mark until I s Pythagora. With over 400 stars on Github in just two weeks and 2k upvotes on Reddit, Pythagora proved to be the solution we had been searching for.
Iteration #1: Responsiveye
Responsiveye is a testing tool that finds visual bugs on web pages from a design file. The idea was to upload a #Figma design from which, our ML model could predict how the page should look on different resolutions and show the errors before new changes are deployed.
The feedback we got from the first iteration was that functional bugs were much more important. So, we sketched a prototype that would go around the page by itself, clicking around and checking if all buttons and forms work (basically a fully automatic monkey testing).
Iteration #2: Responsiveye functional testing
So, we built the second version of Responsiveye which could find functional bugs as well. While it could find meaningful bugs (eg. a form on a site not working), there were 2 problems.
Firstly, the tests were incredibly slow — it would take around an hour to test just one page. Secondly, because the algorithm simply tried random actions on the page, users were unable to discern which specific actions had been tested, making it difficult to track the testing process. The second problem was a deal breaker for most users since both devs and QAs wanted to know what was tested by the automated tests so it was clear that we needed to make significant changes to our testing approach.
Iteration #3: Responsiveye functional testing from user interactions
So, we sketched a tool that creates functional tests based on user actions, rather than simply clicking around the site at random. To achieve this, we envisioned a snippet that would be embedded on a website, similar to Google Analytics integration. This snippet would track user actions (like Hotjar or Fullstory do) and provide us with the exact steps that users are doing from which we can create E2E tests.
While this was received extremely well, the problem became technical. Most tests are related to a user state in the database. For example, if a user has a premium subscription, they will have different buttons to click on the site. This can't be replicated without having control over the backend.
Iteration #4: SafetyTest
While being depressed about how much money we spent testing all these products, we decided to build something simple with a clear value proposition. SafetyTest is a Shopify app that tests store's functions (eg. adding to cart or checkout) by itself.
There wasn't any actual problem with this but we built it in parallel with Pythagora where we wanted to see if Pythagora is technically feasible. When we realized it was, we released it on Reddit and got overwhelmingly positive feedback which pushed us to continue with it.
Iteration #5: Pythagora
After extensive research, experimentation, and iteration, we finally created Pythagora, a tool designed for backend developers and QAs that generates automated integration tests in seconds by recording server activity without the need for writing any code. With Pythagora, users can achieve up to 80% code coverage within just 30 minutes of using the tool and 90% within an hour.
As a result of the Pythagora launch, we decided to stop further development on SafetyTest and focus entirely on the continued growth and improvement of Pythagora. We continue to invest heavily in research and development, working tirelessly to improve the tool’s capabilities and performance.
Currently, Pythagora is quite limited and it supports only Node.js apps with Express and Mongo database but we’re working on supporting more databases (PostgreSQL likely coming out soon) and frameworks.
Conclusion
If you really came this far, thank you so much for taking the time to read about our journey in creating Pythagora. I hope that our story has been informative and inspiring and that it has shed some light on the challenges and opportunities involved in developing new dev tools.
⭐ Pythagora is open source so it would mean a lot if you could star it on Github.
📨 You can also add your email here to get notifications about updates on Pythagora.
Top comments (0)