#TestCulture 🦅 Episode 14 - Test coverage problems

𝗪𝗵𝗮𝘁 𝗶𝘀 𝘁𝗲𝘀𝘁 𝗰𝗼𝘃𝗲𝗿𝗮𝗴𝗲? Test coverage helps determine whether a test case is covering the application code and how much code is exercised when those test cases are runned.

Test coverage enables testers to identify gaps at early stages, get better coverage and to remove redundant cases. It basically saves 𝚝𝚒𝚖𝚎 and 𝚖𝚘𝚗𝚎𝚢.

👉 Function coverage: 𝙷𝚊𝚜 𝚎𝚟𝚎𝚛𝚢 𝚏𝚞𝚗𝚌𝚝𝚒𝚘𝚗 𝚒𝚗 𝚝𝚑𝚎 𝚙𝚛𝚘𝚐𝚛𝚊𝚖 𝚋𝚎𝚎𝚗 𝚌𝚊𝚕𝚕𝚎𝚍?

👉 Statement coverage: 𝙷𝚊𝚜 𝚎𝚟𝚎𝚛𝚢 𝚕𝚒𝚗𝚎 𝚘𝚏 𝚌𝚘𝚍𝚎 𝚋𝚎𝚎𝚗 𝚎𝚡𝚎𝚌𝚞𝚝𝚎𝚍 𝚊𝚗𝚍 𝚟𝚎𝚛𝚒𝚏𝚒𝚎𝚍?

👉 Condition coverage: 𝙰𝚛𝚎 𝚊𝚕𝚕 𝚌𝚘𝚗𝚍𝚒𝚝𝚒𝚘𝚗𝚜 𝚎𝚡𝚎𝚌𝚞𝚝𝚎𝚍 𝚊𝚗𝚍 𝚟𝚎𝚛𝚒𝚏𝚒𝚎𝚍?

👉 Path coverage: 𝙷𝚊𝚜 𝚎𝚟𝚎𝚛𝚢 𝚙𝚘𝚜𝚜𝚒𝚋𝚕𝚎 𝚛𝚘𝚞𝚝𝚎 𝚋𝚎𝚎𝚗 𝚎𝚡𝚎𝚌𝚞𝚝𝚎𝚍 𝚊𝚗𝚍 𝚟𝚎𝚛𝚒𝚏𝚒𝚎𝚍?

⚠️ Some methods are related, for example:

• Path coverage implies both instruction coverage and test point coverage;
• Instruction coverage does not imply test point coverage.

But test coverage is challenging…
1️⃣ With test coverage, it is hard to access accurate test results. Testers are having a hard time to understand why a test is returned 𝕒𝕤 𝕂𝕆 or to increase coverage through 𝕡𝕠𝕤𝕚𝕥𝕚𝕧𝕖 𝕒𝕟𝕕 𝕟𝕖𝕘𝕒𝕥𝕚𝕧𝕖 𝕥𝕖𝕤𝕥 𝕕𝕒𝕥𝕒. We must get rid of 𝗳𝗹𝗮𝗸𝘆 𝘁𝗲𝘀𝘁𝘀 and maintain tests based on the evolution of user stories. 👍

2️⃣ 𝙲𝚞𝚜𝚝𝚘𝚖𝚎𝚛𝚜 can now use complex user paths with multiple data, on multiple devices and with multiple navigators. The need to perform multiple types of data tests, on access control of an application, on object' transitions subject to a workflow 𝗶𝗺𝗽𝗹𝗶𝗲𝘀 𝗱𝗲𝗽𝘁𝗵 𝗼𝗻 𝗳𝘂𝗻𝗰𝘁𝗶𝗼𝗻𝗮𝗹 𝘁𝗲𝘀𝘁𝘀. But also the ability to manage 𝗮 𝗵𝗶𝗴𝗵 𝘃𝗼𝗹𝘂𝗺𝗲 𝗼𝗳 𝘀𝘁𝗲𝗽𝘀. ☝️

3️⃣ Manual validation is made by developers or project managers and takes 𝘁𝗼𝗼 𝗺𝘂𝗰𝗵 𝘁𝗶𝗺𝗲 𝗰𝗼𝗺𝗽𝗮𝗿𝗲𝗱 𝘁𝗼 𝗱𝗲𝘃𝗲𝗹𝗼𝗽𝗺𝗲𝗻𝘁 𝗮𝗰𝘁𝗶𝘃𝗶𝘁𝗶𝗲𝘀. ⌛ On the other hand, regular changes of UI makes it hard to maintain automated tests.

Time spent 𝗯𝗲𝗰𝗼𝗺𝗲𝘀 𝗵𝗮𝗿𝗱 𝘁𝗼 𝗷𝘂𝘀𝘁𝗶𝗳𝘆 𝘁𝗼 𝘀𝘁𝗮𝗸𝗲𝗵𝗼𝗹𝗱𝗲𝗿𝘀 when it comes to creating and running tests, as well as the high cost of test-related services and infrastructure. 😨

📚 𝕊𝕠𝕦𝕣𝕔𝕖𝕤:
🦅 https://agilitest.com/challenges/test-coverage
🎴 http://agilitest.com/cards/understand-what-to-test

Thread on Twitter.