“Bug bash” is a term thrown around often in software engineering and product development. In general, it means all stakeholders in a project placing all of their focus into finding (and resolving) any outstanding bugs in a feature or product.

Some companies or teams conduct bug bashes in a reactionary manner, meaning they pause everything when they’ve accumulated too many bugs in the product. At that point, they tell everyone involved to go out and “bash” as many bugs as they can. Here at Monolist, we like to conduct them proactively: sitting down and working through everything together, in an organized manner, right before we release something new to our users.

We already have (QA|automated testing). Do we really need to do bug bashes too?

If your company has dedicated QA engineers whose job it is to manually test each and every new feature from top to bottom, then you’ve got it made (and if you’re one of them, I’m sorry). This does probably mean a bug bash, as we define it at Monolist, is unnecessary for your team. But they can never hurt!

Automated test coverage, on the other hand, is the direction that more and more software companies are moving. This often means a combination of unit testing and integration testing (among various other approaches). Although we are firm believers in automating everything here at Monolist, we still aim to bug bash every new feature we ship as a team to do the following:

1. Catch unknown edge cases.

No matter how comprehensive your test suite is, it’s still likely to have some gaps. With unit tests, as is hinted at by its name, you’re testing individual units and missing any of the logic in between. Integration testing aims to solve this, but still requires you to define and properly test every possible user path. Although you should always strive for that goal, it may not always be realistic.

2. Get all stakeholders in front of the final product.

We all dream of the perfect product process in which the feature is well defined from the beginning, no requirements change, and everything is delivered as expected. In an ideal world, all product questions should be raised and addressed before you reach the bug bash. Unfortunately, that’s rarely reality.

A final bug bash with all stakeholders involved (product, design, engineering, etc.) ensures that everyone is getting to see the product in its (near) final state. This should help to ensure that any discrepancies or miscommunications that have slipped through the preceding product process can be ironed out before public release.

How to maximize stress and minimize efficacy

The very nature of a bug bash lends itself to stress and anxiety. After all, you’re sitting down with your whole team, poring over the product with a fine-toothed comb, finding and resolving any number of new bugs in a rushed environment. I don’t have the stats, but I assume that the number of engineers excited about that prospect would be fairly low.

Due to the stress-inducing nature, there are a number of things that you should avoid when conducting a bug bash.

Coming unprepared or without a test plan.

Throwing a group of people into a room with the single instruction of “break it” is a pretty ineffective strategy from the get-go. Without coordination between parties, you’re likely to end up with:

• Multiple reports of the same bug, wasting the time and energy of both reporter and reportee.
• Missed bugs, because without a plan, there’s no chance your team will get through all of the test-cases.
• A stressed and confused team trying to play whack-a-mole with the bugs they’re hearing.

Lacking organization around the bug discovery/reporting process.
When you do have multiple people all looking to find and report issues with a product, this can quickly devolve into any number of them vying for the attention of the relevant stakeholder, or calling out their issues, or asking the people around them if they’re seeing the same thing. This all distracts from the actual bug bash, and makes it more likely that the team will miss real issues.

Allowing non-stakeholders to participate (or even be in the room).

Other parties who are not involved in the building and shipping of the feature should not be in the room for the bug bash (sorry). Without the context involved with the project, they’re likely to interject thoughts or opinions that have no place in a bug bash, and that are likely to cause stress, confusion, or misdirection late in the process.

Trying to discuss or address feedback unrelated to bugs, such as on the design or user experience.

When issues arise in the bug bash unrelated to actual bugs, i.e. design tweaks or UX concerns, it’s easy to get distracted and dive into a conversation about that specific issue. Although it should be tracked and addressed, a bug bash is not the best forum.

How to do it right

Now that we’ve discussed the various things you should not do when organizing and running a bug bash, here are some of the things we’ve codified at Monolist to ensure all bug bashes run smoothly, are frustration-free, and address as many outstanding issues as possible.

Come prepared with a formal list of user stories and/or specific paths that need to be tested.

These should be as granular as possible, as if to serve as a step-by-step guide for the tester(s). If you’ve already written a thorough product spec outlining the various user stories, this can probably double as your testing plan. If you haven’t already written a product spec (tsk tsk), writing the test plan might help.

Below is a screenshot of our actual test plan for our recent Asana integration revamp. As you can see, it is quite simple. All it does is lay out clearly which specific scenarios need to be tested, and what the expected outcome is. You can even tell we had a bug bash follow-up from the story outlined in red.

Assign roles.

As discussed earlier, it’s not ideal to throw everyone into a room and have them start bashing away. There should be a set of pre-assigned roles for the stakeholders, dividing responsibilities to ensure a smooth process. At Monolist, we use the roles below:

“Actor” - One person responsible for acting out all of the outlined user stories within your application.
“Conductor” - One person responsible for walking the actor (and larger group) through the test plan in the necessary order.
“Recorder” - One person responsible for recording and assigning any and all bugs that do occur.

If you have any remaining stakeholders involved in the bug bash, they should be paying close attention for any possible product issues, visual bugs, or anything else that should be noted.

Create a task for any issue that you encounter. Assign it immediately.

As the actor proceeds through the test plan, the recorder should be paying close attention for any bugs that occur or issues that are raised. These should immediately be tracked and assigned in whatever task management system your team uses. This is to ensure that nothing is forgotten about later.

If necessary, pause and immediately schedule the follow-up bug bash.

There’s always that occasional bug bash where the first bug that occurs is quite early in the experience, preventing the bug bash from continuing. If the cause is not immediately obvious or the bug cannot quickly be resolved, the bug bash should be paused. You don’t want to make everyone wait or put the audience pressure on the responsible engineer. Before pausing, though, be sure to schedule the follow-up at a time when all parties will again be present.

Always start from the top.

When your team is ready for a follow-up bug bash, always be sure to start from the very beginning of the test plan regardless of how far you made it last time. Any tweaks or bug fixes could have affected previously tested paths, and so everything should be covered.

In Conclusion...

As your team continues to conduct bug bashes, always pay attention to how effective they are and how the team feels about them. The process can and should be tweaked along the way.

Although bug bashes are a manual effort and should never fully replace your other testing approaches, they’re a worthwhile method of testing your new feature end-to-end just as a user would. The key is to have a system and stick to it, ensuring your bug bashes are efficient and valuable for everyone involved.

Monolist is a global inbox for engineers. Try it out now!