There is a child's fable that I have always loved, The boy who cried wolf.
It goes more or less like this:
There was once a young shepherd living in the mountains. One day he grew so bored that to have fun he started yelling
"Wolf! Wolf! The wolf is attacking my sheep! Help!".At the sound of the yells, people in the nearby village grabbed their pitchforks and shovels and rushed to help the young shepherd,
only to find him rolling on the floor laughing.Rinse and repeat several times, depending on the attention span of your kids.
One day, the wolf actually showed up and the young shepherd yelled and yelled and yelled, but villagers thought:
"Oh that boy! We won't be fooled again. Lets keep to our tasks".And the wolf enjoyed a great feast of mutton and shepherd flesh.
What does this fable have to do with software development?
Lets give it a try:
There was once a unit testing suite living in a CI environment. One day the suite started to shout "I found a bug! A bug! Help!"
At the view of the red status, developers in the nearby cubicle, grabbed their laptops and coffee cups and rushed to fix bug,
only to find out the build green again.Rinse and repeat several times, depending on the attention span of your developers.
One day, an actual bug was introduced and the test suite yelled and yelled and yelled, but the developers thought
"Oh those flaky tests! We won't be fooled again. Lets deploy to production".And the bug enjoyed a great feast of long and stressful developer nights.
Another one:
There was once a log file living in a production server. One day the log file started to shout "WARN! WARN! Something is wrong in production! WARN!"
At the view of the error report, developers jumped from their beds, grabbed their laptops and coffee cups and rushed to investigate
the issue, only to find out that it was a one-off innocuous issue.Rinse and repeat several times, depending on the attention span of your developers.
One day, an actual issue happened and the log file yelled and yelled and yelled, but the developers thought "Oh that noisy log file! We won't
be fooled again. Lets stay on bed".And the issue enjoyed a great feast of manual and costly data fixes.
Flaky tests and incorrect log level on a log file are just technical debt.
The cost of that debt is initially paid with the time it takes to rerun the test suite, or the time that it takes to investigate the error reports, but it also includes the resulting loss of confidence that your team has in the test suite and alerts.
Following the words of the Frederick Brooks on The Mythical Man-Month:
How does a project get to be one year late?
... One day at a time.
How a test suite get abandoned? One flaky test at a time.
How does an error report get completely ignored? One noisy line at a time.
Top comments (0)