Planning Your Next Cypress.io Tests with Testomat.io

It is so easy to start writing tests with Cypress.io. It seems that Cypress opened doors to testing for everyone with basic JavaScript knowledge. However, when the number of tests grows writing new tests becomes harder. And the issue is not just about writing code. Planning new test scenarios, refactoring, configuring pipelines, and getting test run reports. The more people involved in the product development the more visibility the testing process needs.

To keep end 2 end tests organized, following questions should be answered:

• what tests do we have?
• what parts of an application are covered with tests?
• what new test scenarios should be implemented?
• what is the current state for passed and failed tests?
• if there are flaky tests on a project?

🤔

Cypress.io is just a test runner, but it can be paired with Testomat.io, a test management system for automated tests.
Let's see how Testomat.io can answer the questions risen above. We will use Cypress Realworld App, which is a real-world project (as you might have guessed) with a well-written Cypress tests.

🤔 What tests do we have?

But answering the very first question is not as simple as it seems.

All Cypress.io end 2 end tests are written as JavaScript code so to read them a person should be at least familiar with JavaScript, GitHub, etc. So this already clothes doors for business analysts, product managers, and manual QAs, for all these folks who do not actually code.

That's why we can import a Cypress realworld app into a newly created project on Testomat.io.

Importing happens with a magical npx check-tests command which scans all the tests on a project and imports them into UI without executing them. This way Testomat.io can present those tests to non-tech folks.

To import only UI tests from realworld app project, we need to run check-tests command with --typescript flag (as the project is written in TypeScript) specifying that we use Cypress.io and the path to tests via --dir parameter:

TESTOMATIO=a9ae6798atoj npx check-tests@latest Cypress.io "**/*{.,_}{test,spec}.ts" --dir cypress/tests

a9ae6798atoj is an API key of a project you are importing to, so in your case, it will be a different one.

If you use Import from Source button, you will get this command generated. Take note, that by default it imports tests from cypress/integration folder which doesn't exist in cypress-realworld project. So you need to change path and use cypress/tests instead.

Importing is fast, it takes just a few seconds to get this output.

Then all imported tests can be reviewed.

That's great that all tests are finally visible to team members. But is there a way they could be linked to specifications they are testing? Yes, and that answers the next question:

🤔 What parts of an application are covered with tests?

If the development management happens in Jira, Testomatio can link automated tests to Jira issue. You can link tests from a Cypress spec to an Issue:

Now all tests will be available within Jira, so can be taken into the development process. Revealing automated tests to Jira Issues helps managers and business analysts to understand more information on quality control. They don't need to leave Jira to see those automated tests!

The only thing needed, is a Testomatio Jira plugin should be installed from a marketplace and a Jira Project to be connected to Testomatio project.

And we are almost ready to answer the next question:

🤔 What new test scenarios should be implemented?

A project manager or a business analyst knows what features are coming next so they are the right person to propose the next tests.

As they can't be asked to write a code, or open an issue for each test, they could create in Testomat.io test directly from Jira. After that, a developer can take the written scenario and implement it using Cypress.io.

As you see on the screenshot, step autocompletion can be used to help non-tech folks write scenarios efficiently.

The test with a description is treated as a manual test, but once it is implemented as Cypress test, and re-imported it will be marked as automated.

🤔 What is the current state for passed and failed tests

Testomat.io can display and store Cypress reports. To send run reports to Testomat.io a @testomatio/reporter package should be installed and enabled in cypress/plugins/index.ts.

// inside cypress/plugins/index.ts
// require testomatio reporter package:
const testomatioReporter = require('@testomatio/reporter/lib/adapter/cypress-plugin');


export default (on, config) => {
  testomatioReporter(on, config);

  // ....

This reporter can be added to any Cypress.io project, following instructions from Testomat.io

Then tests should be executed via npx cypress run with few extra parameters:

TESTOMATIO=a9ae6798atoj npx cypress run

a9ae6798atoj is an API key of a project to which reports will be sent.

A report will contain a list of passed and failed tests, with videos and screenshots. External S3 storage from one of the cloud providers (AWS, DigitalOcean, Google Cloud, etc) can be used to keep unlimited screenshots and videos.

Videos and screenshots can be enabled in Testomatio by adding S3 credentials into .env file of Cypress Realworld project:

S3_ACCESS_KEY_ID=aws-key
S3_SECRET_ACCESS_KEY=aws-secret
S3_REGION=ap-south-1
S3_BUCKET=testomatio-test

A complete historical data of all Cypress.io test executions are also available:

If we have this data the final question can be answered:

🤔 Are there flaky tests on a project?

Testomat.io has an analytics dashboard where all tests are analyzed to provide valuable insights. For instance, it is important to keep track of flaky tests, and slowest tests, and check for the most common defects across tests.

Final Thoughts

Developing end-2-end tests with Cypress.io is fun. But making them first-class citizens in project development is also important. Cypress End 2 end tests must be visible to the whole team, so even team members who are not directly involved in testing could track the progress, propose new changes to tests, and see the actual test coverage.

Testing is a crucial part of the development process. The more clear it is and the more team members are aware of it, the more quality software can be delivered. While Cypress.io helps build efficient and fast tests for the modern web, Testomat.io provides a feature-rich test management solution for all kinds of automated tests.

P.S. Testomat.io is absolutely free to use for personal projects. Try it out with Cypress realworld app, or any other Cypress project.