#TestCulture 🦅 Episode 19 - Optimize regression testing

𝐖𝐡𝐚𝐭 𝐢𝐬 𝐫𝐞𝐠𝐫𝐞𝐬𝐬𝐢𝐨𝐧 𝐭𝐞𝐬𝐭𝐢𝐧𝐠?
Regression testing refers to testing a previously tested program, after a modification. It ensures that defects have not been introduced or discovered in unmodified parts of the software, as a result of the modifications made. These tests are performed when the software or its environment is modified. The objective of a regression campaign is to check whether the changes made to the software (or to its environment) have not caused major bugs in the functionality already in place... And therefore whether these changes have caused major regressions.
This definition highlights important principles such as software testing that has 𝚊𝚕𝚛𝚎𝚊𝚍𝚢 𝚋𝚎𝚎𝚗 𝚝𝚎𝚜𝚝𝚎𝚍 𝚋𝚎𝚏𝚘𝚛𝚎 or the subsequent modification of the same software. A regression test 𝗿𝗮𝗿𝗲𝗹𝘆 stands alone. It is part of a coherent set of several tests, each with its own characteristics and objectives.

𝐖𝐡𝐲 𝐢𝐬 𝐫𝐞𝐠𝐫𝐞𝐬𝐬𝐢𝐨𝐧 𝐭𝐞𝐬𝐭𝐢𝐧𝐠 𝐜𝐡𝐚𝐥𝐥𝐞𝐧𝐠𝐢𝐧𝐠?
👉🏼 𝐓𝐢𝐦𝐞 & 𝐜𝐨𝐬𝐭 𝐦𝐚𝐧𝐚𝐠𝐞𝐦𝐞𝐧𝐭. Regression tests involve running tests that have already passed. Time spent can be hard to justify to stakeholders. Especially when it comes to creating and running tests, as well as the high cost of test-related services and infrastructure. Especially when stakeholders don't appreciate the value it adds.
👉🏼 𝐂𝐨𝐦𝐩𝐥𝐞𝐱 𝐦𝐚𝐢𝐧𝐭𝐞𝐧𝐚𝐧𝐜𝐞. Regression tests inevitably become large and complex over the course of a complex development project. The number of test cases, as well as the functions and processes that they check, can easily number in the thousands. It makes it impractical for quality assurance engineers to test manually without consuming significant time and resources.
👉🏼 𝐓𝐨𝐨 𝐦𝐚𝐧𝐲 𝐟𝐥𝐚𝐤𝐲 𝐭𝐞𝐬𝐭𝐬. Regression tests are great when they identify issues before a production build. But there's nothing more frustrating than a test suite that throws constant false positives — or tests that fail when they should pass. Flaky tests that generate false positives are frustrating for stakeholders, developers and testers and could make everyone second-guess the process.

📚 𝕊𝕠𝕦𝕣𝕔𝕖𝕤:
🦅 https://agilitest.com/challenges/multi-device-testing

Thread on Twitter.