Originally authored by Prashant Hegde [https://testsigma.com/blog/a-testers-guide-to-deal-with-missed-bugs/]
Even with the increasing adoption of Agile and DevOps, many organizations still consider testers the gatekeepers of quality and are solely held responsible for the quality and approving releases. Such organizations blame testers for a missed production bug.
No matter how much you test, a few bugs can still slip through the testing phase & reach production. Every software tester would have come across a situation of a missed bug. For many testers, it’s a terrifying nightmare. Such situations can be overwhelmingly stressful and scary!
This article will discuss how a tester can deal with a bug in production.
1. FIRST OFF, STAY CALM – FOCUS ON WHAT IS WITHIN YOUR CONTROL
Dealing with your angry customers and your annoyed managers while figuring out what went wrong requires you to stay calm. Get some breathing space where you can think peacefully about the next steps.
Do not get involved in the blame game; it won’t help anyone.
If you missed testing something, it’s totally fine to admit the oversight and take corrective actions. Accept the fact that no one is perfect all the time. Owning your mistakes will only increase your credibility.
Don’t Panic. Understand that no software is bug-free. Exhaustive testing is impossible with limited time, resources and budget. Additionally, your users may use your application in ways you never imagined.
Stop worrying about the past – Free your mind to focus on what is within your control. Help your team by –
i. Reproducing the issue – is the most significant contribution you can make to your team to resolve the issue at the earliest.
ii. Gathering more information – look at the support tickets, talk to the customers or support team, look at logs or monitoring tools
iii. Communicate with different teams and Identify the changes released recently. Verify if those releases introduced the bug.
iv. Identify a workaround – A user can complete some action in your application in multiple ways. If there is a workaround, unblock your users and let them know alternative ways of achieving the task. If required, work with customer success and support teams to communicate the issue and the workaround with your customers. Every missed bug to production need not have to be fixed immediately.
v. Roll out a quick fix – Work with your team to solve the problem as quickly as possible. Do not look for a perfect solution while sending a patch, but rather one that unblocks the customer soon. Ensure that you test the patch before the fix. Sometimes the fixes introduce new bugs. Identify the impact areas and explore them for risks before releasing the fix.
vi. Don’t shy away from asking for help from other testers to accelerate the release of the patch.
2. LEARN FROM YOUR MISTAKES – TURN THEM TO YOUR ADVANTAGE
See mistakes as an opportunity to learn. Think about ways to prevent similar mistakes from happening in the future.
i. After the patch is released, conduct a root cause analysis and investigate how they got through the testing phase.
ii. Brainstorm with your team about the possible ways to prevent similar issues in the future
iii. Identify the gaps in your process to avoid similar bugs in the future. Strengthen or change your process as per your context.
iv. Use the learnings to update and improve your test strategy, plan, test cases, and automated tests.
v. Use open-source tools like Testsigma, and start automating your happy flows. When automated tests help you find the obvious bugs, you can focus on deeper and more creative tests to uncover hidden bugs. Furthermore, give early feedback to your developers by seamlessly integrating with your favorite CI/CD tools.
vi. Ask yourself, could this issue be found in earlier phases of the software development. For instance, could this issue be caught during the requirements phase? Or during code review?
vii. Sometimes you will notice that some features/modules are difficult to test. Poor testability leads to more bugs. Improve the testability of such features/modules.
viii. Document and share your learnings with your entire team—track action items to prevent similar issues in the future.
ix. Introspect into what you could have done better or different. Seek help from your team if you do not clearly understand a feature/system/module.
3. ANTICIPATE ISSUES AND BUILD A SAFETY NET
Though bugs are inevitable, the consequences can be severe for the business.
So always anticipate bugs!
Have a rollback strategy in place to prevent or minimize damage in case of a bug leaks to production. Employ one or more of the following shift-right strategies or some rollback strategies to avoid or reduce the impact in case of a defect.
i. Feature flags – Have the flexibility to toggle on/off the new feature without deploying new code. In case of an issue, switch off the new feature.
ii. Staged rollouts – roll out the update to a subset of users first. Upon gaining confidence, you release it incrementally to the rest of the users.
iii. Blue-Green Deployments – runs two identical production environments called Blue and Green. At any time, only one of the environments is live and serves the production traffic. While Blue is live, green is idle, and vice versa. Let’s say blue is serving the traffic currently. When releasing a new feature, you can deploy and safely test on Green (which is idle). Upon completing testing, you can switch all incoming traffic to green. Now Blue goes idle. This technique reduces risk in case of an issue: if something unexpected happens with your new feature on Green, you can immediately roll back to the last version by switching back to Blue.
iv. A/B testing – Divide your customers into two groups (Eg – Free and Premium). One group(free customers) gets the new version of the software, while the other group(premium customers) remains in the older version. Monitor production until you gain confidence. Later roll out the new version of the software to premium customers. This strategy can minimize the impact in case of bugs.
v. Beta testing – Get the application tested in a real-world environment by early adopters.
4. EDUCATE YOUR STAKEHOLDERS – STRIVE TO BUILD A TESTING CULTURE
Why do decision-makers and executives think that “Testers are responsible for missed bugs” or “Quality is the sole responsibility of a tester”? It’s primarily because of a lack of awareness about testing and the role of a tester. The first step to building a testing culture is to educate your organization about testing by debunking myths.
Testers explore, learn, investigate and test the software to provide information about the quality of the software and the associated risks to the business. This valuable information enables businesses to make informed decisions. Some executives and managers have the old-school mentality that testers are gatekeepers of quality. Several managers are aware that “Testing is everyone’s responsibility,” but they do not know how to involve developers, product managers, and other non-testers in testing activities.
As a tester, coach the whole team has to take up testing responsibilities within the software development. Below are some ideas –
- Hold three amigo meetings – A meeting where the Product Manager, Developer, and tester meet to discuss the requirements before development.
i. Enables everyone to be on the same page about requirements and the scope of the testing.
ii. Collaborate to find ambiguities at the early stages(requirement, design, etc.) to avoid rework and save costs.
Make unit testing a part of the definition of done. Furthermore, start measuring and publishing unit test coverage in your test reports.
Encourage quality practices like Test-Driven Development, code reviews, test case reviews, etc.
Teach developers to test – conduct pair testing sessions with your developers.
Tools like Testsigma enable the entire team, including devs, QA teams, & PMs, to contribute to automation without requiring advanced programming skills.
Conduct bug bashes – bug bashing fosters a shared sense of quality by involving the entire organization in testing and finding problems before the release.
Eat your own dog food – have your organization use its product in your day-to-day operations. Have a mechanism to collect feedback and improve continuously.
CONCLUSION
When testing complex applications, you cannot find every possible bug in limited time, resources and budget. Implement risk-based testing – prioritize testing those areas of the application that are most critical to the business. Realize that QA cannot assure quality but only assess the quality of a product. Anticipate issues and have a plan to rollback without impacting the customers. Furthermore, build a culture where everyone contributes to testing.
As long as you learn from your mistakes and constantly improve – A missed production is a blessing in disguise. A missed bug teaches you the lessons you will not forget, which no testing books teach you.
Top comments (0)