In my last blog, I talked about the Next Wave of DevOps, specifically around the threads associated with this second wave of innovation. I shared a little about some of the companies that are part of this wave. Out of that came some really great discussions with other folks who are clearly pushing changes in how Application Developers, DevOps, and AppSec teams work both separately and together.
A core problem we are all solving for is toil. Companies are finally starting to acknowledge the cost. This article by AppMap’s CEO, Elizabeth Lawler, “AppMap: A map to reduce developer toil” does a phenomenal job of mapping out where your toil lives and the impact it has on the product's health, security, and the people who care for it. The article states:
“Wasted developer time and effort, which includes inefficient processes and code rework, is often referred to as “developer toil.” According to Stripe, developers spend up to half their work week in toil, with some estimates as high as 80%. Toil costs businesses over $85 billion per year in wasted developer effort.”
This idea of toil is not new. We’ve had folks raising the alarm for years. John Allspaw used the term Dark Debt* for this in his talk "How to keep your systems running day after day". When talking about dark debt he states:
“Here are my main points that I'm going to give you. One, we have to start taking human performance seriously in this industry. If we don't, we will continue to see brittle systems with ever-increasing impacts on our businesses and on society.”
There are many brilliant talks about the consequences of ignoring toil in your organization, like John Willis' DevOpsDay ATL talk, Burnout. He digs into the signs of burnout, recovery, and ways companies can track how teams are faring from a socio perspective.
The trouble toil creates
Knowing this theoretically is a great step, so let’s talk about some concrete examples of toil that you, your team, or your company at large might be experiencing:
- Incident Oncall: When an incident channel is opened, how many people are included? Is it greater than 5 people?
- Archeological dig: Do you have a principal or five working on researching and documenting how your technical stack works?
- Find your fixer: Does AppSec or DevOps do extra cycles to identify who owns it?
- Application work: How many lines of code does your dev have to navigate, plan, migrate, and check?
- Vulnerability Remediation: Did your entire eng team work over the weekend on Log4j?
What is the opportunity cost associated with the DevOps toil? Are we ok with it? Or do we want something better for ourselves and our teams? Which small step will you take to identify and reduce your toil?
NextOps and you
The good news is there are many companies that are addressing the above problems and doing so without forcing your teams to become YAML wizards. They are doing this by making the hard parts more approachable whether that’s via your application, systems, or security. They are auto-magically capturing and graphically providing you with a mapping of your unknowns. They are providing you with the insight you need without having to pull 40 people into a channel or conference chat.
Without these tools you have toil, both hidden and known, that Application Developers, DevOps, and AppSec folks slog through on a daily basis.
Just imagine what it would be like if folks were able to focus on the fun complex areas of building, maintaining, securing, and scaling a product. To have junior folks work in an environment where it’s safe to try things.
Shameless plug: Try our beta to see if it helps reduce your DevOps toil.
*The term dark debt came out of the OSU Cognitive Systems Engineering Lab's Stella report here It's a brilliant read. I encourage you to check it out if you haven't already.
Photo Credit #WOCinTech
Top comments (0)