DEV Community

Cover image for Startup Testing Dilemma: E2E or Not E2E?
Jeremy SFEZ
Jeremy SFEZ

Posted on • Originally published at argos-ci.com

Startup Testing Dilemma: E2E or Not E2E?

In a fast-growing startup I'm part of, a question sparked intense debates among senior developers: with all the setup and maintenance time it cost, is E2E testing really worth our time?

We all prioritize fast delivery, and figuring out how to keep that momentum in long term was the crux of our chats

Quick note: The other perspective would be to bypass E2E, let a few bugs sneak by QA, and patched things up when they showed in production.

The E2E Test Paradox in Startups

End-to-end (E2E) tests are often seen as the gold standard in quality assurance. They replicate real user actions and make sure everything in the application runs smoothly. But in a startup environment, they can be a double-edged sword.

Imagine investing hours crafting a detailed E2E test suite for a feature. The next week, based on new insights, that feature undergoes a revamp or is replaced entirely. Now, not only do you need to rewrite the feature, but you also have the unenviable task of overhauling the test suite. This cycle can be both mentally exhausting and time-consuming.

Developer struggling with E2E testing

To counter the challenges posed by E2E testing and its associated maintenance costs, we explored an alternative: visual testing.

🧿 Visual Testing: Agile’s Best Friend

In the dynamic landscape of startups, where changes are frequent, visual testing emerges as the optimal solution. Here's why:

  1. Trust in Every Change: With visual testing, changes become transparent. Approve expected updates and reject unintended alterations.

  2. Minimal Effort, Maximum Coverage: Efficiently use one line of code to visually verify a full page's layout and content, eliminating the need for extensive scripts.

  3. Cut the Maintenance Chain: Even if your product evolves constantly, no need to update the test code. Simply approve or reject the updated screenshots.

⚒ Our Go-To Tool

We've been using Argos for visual testing. It's a dev tool that allows to review the changes a pull request brings. It's easy to onboard and it slides right into our workflow.

// Homepage test file. No maintenance needed! 
import { test } from "@playwright/test";
import { argosScreenshot } from "@argos-ci/playwright";

test('homepage', async ({ page }) => {
  await page.goto("http://localhost:3000");
  await argosScreenshot(page, "homepage");
});
Enter fullscreen mode Exit fullscreen mode

Argos demo

An example of Argos build, where the dev can review the PR's visual changes.

⚖ Balancing Act: E2E and Visual Testing

While visual tests shine in adaptability, the value of E2E tests remains unmatched for mission-critical functionalities. The key is balance. For core features with less frequent changes, E2E tests guarantee stability. For others that are in flux, visual tests provide adequate coverage without the maintenance headache.

Conclusion

In the fast-paced world of startup development, agility is critical. While robust testing remains essential, the type of testing needs to align with the product's stage and volatility. Visual testing provides the flexibility startups need without compromising on quality, making it a crucial part of any developer's toolkit today.

Have you faced similar testing dilemmas in your startup? How did you navigate them? Share your experiences in the comments.

Top comments (3)

Collapse
 
gregberge profile image
Greg Bergé

Visual Testing offers an unparalleled cost-to-value ratio in the realm of software testing. It stands as the pinnacle in the testing hierarchy, delivering comprehensive results without the overhead typically associated with exhaustive testing methods.

Image description

Collapse
 
codehost profile image
CodeHost

I totally agree with your recommendation! E2E testing is crucial before launching your start-up.

Collapse
 
dylanmerigaud profile image
Dylan Merigaud

Thanks for sharing! đŸ’Ș