Can we retry failed tests with testng? IRetryAnalyzer

Automation runs could fail for an endless number of reasons:

• Flaky tests,
• Running environment not steady,
• Bugs,
• Changes in selectors,
• Framework problems,
• And so on and so forth...

One of the aspects we definitely need to take into count is to investigate the root causes why the testcases fail, but in this post I want to bring up another topic.

If the team wants to rerun the failing tests under certain conditions, the different frameworks and programming languages usually offer diverse mechanisms that could let us re-run testcases programmatically.

Knowing that "Selenium-java" is still relevant and many people still use frameworks such as testng, I want to bring up the specific feature that testng offers, because it's a feature that's been available for quite a long time, and one I hadn't taken advantage of it, up to recently.

It's pretty straight forward, all what we have to do is to add the retryAnalyzer property to the test decorator and create the class that handles the logic for the retry.

Let's look at the following chunck of code as an example:

1. First we add the property.

@Test(retryAnalyzer = DynamicRetryAnalyzer.class)
    public void verifyWidgetOnPage()  {
        runTheTest();
    }

1. Then we create the class with the logic to retry your tests.

import org.testng.IRetryAnalyzer;
import org.testng.ITestResult;

public class DynamicRetryAnalyzer implements IRetryAnalyzer {

    int count = 1;
    int maxRetryCount=2;

    @Override
    public boolean retry(ITestResult iTestResult) {
        if(count<maxRetryCount){
            count++;
            return true;
        }
        return false;
    }
}

In this specific example the framework will retry a fail test an extra time no matter what (just if it fails). The example is kind of simple, but, I think we get the picture. If we need any specific logic to evaluate wether we should rerun a test or not, we do it by changing the logic inside the public boolean retry(ITestResult iTestResult) method.

The only thing we need to make sure is to implement the IRetryAnalyzer interface and override the retry(ITestResult iTestResult) method with the logic that we need to evaluate if we should rerun the test or not.

I hope this helps anyone!

"A test which fails is not always flaky!"