Stop Test Suite Execution with ‘X’ Failed Tests | Fail-fast Principle

The Fail-fast Principle in Software Testing

In software development, a fail-fast system is a tool that will notify when a failure is likely to occur. The philosophy of the fail-fast principle is formed around the idea that a system should stop normal operations rather than trying to continue working around a flawed piece of code.

The fail-fast principle makes software bugs easier to detect earlier in the development cycle. It allows for faster strengthening of the code under development, limits the number of bugs that make it into production, and reduces the intrinsic cost of letting bugs in the code.

In short, it is better to fail early in the development than later when the product is about to be released.

One of the benefits of test automation is the ability to test early, often, and anywhere in the development cycle. Katalon Studio 8.1 delivers a vital feature that aims to increase the operations and cost-efficiency of your automation cycle. Leveraging the fail-fast principle, Studio 8.1 provides users with the ability to stop Test Suite Executions when a particular failure threshold is reached.

This new feature will help make more efficient use of testing devices and improve infrastructure costs, especially for cloud users.

What This Looks Like in Studio

Failure thresholds in KS 8.1 allow users to terminate the Test Case or Test Suite Collection execution when there is a significant number of test cases in an execution that have failed, enabling them to provide feedback earlier in the cycle rather than waiting until the test cycle is completed.

To leverage this feature, we have included a new command option, -maxFailedTests= , where T is the maximum number of failed executions allowed before the test is stopped. Once the Test Execution stops, Katalon Studio will generate the report containing all the insights. This feature is only available for tests executed through the CLI or CI using the runtime engine.

First, find the option to set the threshold within the Generate Command for Console Mode dialog box – set the level of failure tolerance by ticking the box “Terminate the execution once the total number of test failures reaches this threshold” and typing a number in the text field.

Generate the CLI command, which in this case will include the -maxFailedTests= parameter in it.

Run the execution and once it reaches the threshold it will provide a message in the CLI indicating that it had stopped following the rules.

You can also take a look at the results in the Test Cases Table within reports that Studio will generate

To learn more about how to leverage this feature in your testing environment, check out the Katalon Studio 8.1 release notes.

Wrapping Up

We are thrilled about Studio 8.1, and we hope you are too. We encourage you to download Studio 8.1 to start taking advantage of its many new features and improvements.

For a complete list of new features, improvements, and fixes, please visit the release notes. As always, please post any questions, ideas or concerns on our community site, we are eager to hear from you.

The post Stop Test Suite Execution with ‘X’ Failed Tests | Fail-fast Principle appeared first on Katalon Solution.