Change is a vital process in software maintenance, and during changes, new bugs might occur. Even if you carried out a thousand different test scenarios before releasing the first version of the software, it is not guaranteed that the second version does not require any of those tests to be re-executed. This is because fixing new bugs might cause some older bugs that were fixed in an earlier release to reappear.
So, to ensure that old bugs have not resurfaced, we conduct regression testing. Regression testing is the testing that is done before finalizing any changes in the source code of the software.
There are several levels of tests, for example, unit tests, integration tests, end-to-end tests, etc. And regression tests could comprise any of these tests. It is not about the level of tests or the type of tests, but about the time when the tests are done and the depth of tests. It is about time because the regression test is done before any changes are merged or deployed. As for the depth, the greater the coverage, the better the quality assurance.
Nevertheless, it may become very tedious to manually test all the scenarios from a test plan if the number of test scenarios is very high. A solution to this problem is automated regression testing.
You might have already heard some buzzwords around this topic, such as BDD, TDD, test report, bug tracking, etc. Many tools and technologies can indeed be used to automate testing your software. But, my advice is that you should not get intimidated by these tools and technologies while you are a beginner in regression testing.
Regression testing is mainly about re-executing the tests several times. Sometimes, it might not be possible to execute all the tests, so you might have to categorize the test cases based on their priorities and context as a tester.
If you are in the starting phase of your software testing career, then I can give you the following tips to perform regression testing in the best way possible.
1. Automate as much as possible
One of the seven principles of software testing states that “Exhaustive testing is not possible”; however, it is always better to test as much as we can. And automating as many tests as possible makes the process of regression testing a lot faster. So you should try to increase test coverage with more automated tests.
And add varieties of tests. Do not just write unit tests, although they catch more bugs than any other tests. You should write end-to-end tests as well that reflect the user stories.
2. Run tests in some CI servers and run them parallelly
Automated regression tests should be executed before any pull request is merged into a base branch. As the number of tests increases, these tests might take hours to complete execution. So, instead of running the tests each time in a local machine, we run them in the CI servers, such as Travis, Drone, GitHub Actions, etc. Finally, as far as possible, run the tests concurrently so that developers do not have to wait long for the test to be done.
3. Maintain your test because the AUT is changing the behavior
Once you automate your test scenarios, you should run the test on each commit and maintain the taste according to the changes made in each commit. If the change is unintentional, then the taste fails, which will notify the developer that the regression test detected a new bug. And if the change is intentional, then the test might fail, in which case software testers should maintain the test scenario to comply with the new behavior of the application under test (AUT).
4. Priorities the tests because running each test becomes time-consuming
After you automate a lot of test cases, you should prioritize those tests so that the most important tests are executed in every commit. For this purpose, you can tag your tests based on the area that it covers. One of the popular tags is “smokeTest”, which is used to indicate that the test is very basic and there is no point in executing other tests if the smoke test fails. If you tag your tests based on the context, you can execute the tests only if the changes are related to that context. It will save a significant amount of time.
5. Don't skip the test that caught some bugs previously
The defect clustering principle of software testing explains that almost 80% of the defects are centered around 20% of the code (or module). So, if a test finds some bug, you should run that test on each regression test because it is highly likely that the same test will catch a similar bug.
Conclusion
In a nutshell, regression testing requires tests to be executed multiple times throughout the SDLC. So, to avoid boredom and increase the efficiency of your work, you should try to automate as many tests as you can. Similarly, classify the tests and set the priorities so that you would know which tests must be executed for change in particular modules.
Oldest comments (0)