A Criticism of Scrum

Okay, look... scrum is just terrible. It's frustrating, mind-numbing, and absolutely kills productivity.

Don't get me wrong, on paper scrum seems amazing collaboration framework. And maybe in an ideal world, it is amazing. But down on planet earth, I have yet to be apart of a team or company that actually saw a net-benefit from the scrum framework.

This topic is something that I've been thinking about off-and-on for much of the past decade as agile frameworks have become more popular. And I debated posting this article knowing full well that scrum or scrum-like collaboration frameworks are used in MANY (probably most) software engineering teams nowadays. Surely it's all in my head, right? Maybe I'm just doing it wrong? Maybe I'm thinking about it wrong? Why would scrum be so popular if it didn't actually work?

So to come clean, this really is not so much a criticism of the theory or formula of scrum, but instead more of a cry for help on how it's been implemented in practice, at least in my experience. I can't possibly be the only one struggling with this, can I?

I guess we'll see. Let's dig in.

Sprint Towards Mediocrity

Everyone (I assume) who has followed the scrum framework will be familiar with sprints. But in case you are not, sprints are designed to be a fixed duration of time to complete a bunch of tasks, usually 2 weeks long. The goal (I think) is to timebox tasks to a specific period of time to incentivize teams to keep tasks small and achievable. The hope is at the end of a sprint you end up with some sort of deliverable that can actually be demonstrated to others--something to show for your work.

Prior to a sprint, most scrum teams will have a series of sprint planning and backlog refinement meetings where the team discusses the list of tasks (typically called "User Stories") that need to be completed during sprints. During that discussion, the team is asked to apply an arbitrary number value to that task, usually called "Story Points". Most teams, for whatever reason, use numbers from the Fibonacci sequence: 0, 1, 2, 3, 5, 8, 13, 21... with larger numbers applied to user stories that are more difficult, require more effort, or have a lot of uncertainty attached. Once user stories are assigned a numeric value, the team makes a determination of how many user stories can be completed within a sprint by looking at their average velocity from previous sprints (how many story points worth of work were completed previously), and their upcoming velocity (how many story points they expect to be able to complete). Cool.

Except here is the kicker: This whole story points business is nonsense. It's like the TV show Whose Line Is It Anyway?: The rules are made up and the points don't matter. It is extremely rare to complete the same task twice (and if you are, you should probably automate it). So really the question being asked is this:

"Hey can you assign an arbitrary point value to a task that you've never done before so we know roughly how long it will take and difficult it will be?"

No, not really.

So what do engineers do in the face of such a question? Well, in my experience most will answer it in one of two ways:

1. Make up some absurdly high number to ensure they are confident they can complete it within a sprint. This usually triggers the typical 10-15 minute sidebar conversation where everyone on the team reiterates all the things we don't yet know. Seems like a good use of time.
2. Give it a "normal" number so the Project Manager and/or Scrum Master just accepts it and moves on to something else without discussion.

Regardless, once the user stories are pointed and assigned out to engineers, the story points attached to those stories tend to be static. It is extremely rare for a story to be re-pointed after it has been completed to reflect the true complexity. The tasks that were unexpectedly simple were over-estimated, while the tasks that ended up being unexpectedly difficult get taken out of the sprint and assigned to the new sprint. The end result? The team's overall velocity is incredibly inaccurate and not a reflection of the actual amount of work being done. Not only does this hurt the team's performance when reporting up to the people that actually care about that stuff, it also makes planning future sprints a fool's errand. You're assigning arbitrary units of work to tasks you know very little about, all while trying to keep it the overall workload under an imaginary capacity number that is estimated from previous velocity numbers that don't actually reflect the team's true output. Brilliant, just brilliant.

But that's not even the worst of it. The biggest issue is the combination of sprints and story points breeds mediocrity within a team. No one likes to carry incomplete user stories between sprints. Some teams (or, more accurately, their managers) consider it a failure if user stories do not get completed within a sprint. This does nothing but incentivize scrum teams to over-point their user stories, and/or under-estimate their capacity. In an ideal world, if an engineer finishes with their user stories before the end of a sprint, they should simply pull the next user story off the top of the backlog. In practice, there is a much larger incentive to work on a side project, or even just idly sit there twiddling their thumbs while waiting for the end of the sprint. After all... you wouldn't want to pump up your velocity numbers. That could cause you to over-commit in the future and burn yourself with a user story that ends up being unexpectedly difficult. Ugh.

I'll freely admit I don't know a better way to estimate work load and work capacity. But what I do know is that, despite what all of the agile coaches will tell you, story points are really a way to measure how long a task will take. And if engineers were to answer that question honestly they would probably answer it in one of three ways:

• "It will take about 15 minutes"
• "It'll take longer than it should"
• "I have no clue how long it will take"

Facts. (As I see them, anyway.)

Backlog Shoveling

To your left is a large pile of unfinished "to-do" items, typically known as the backlog. In most scrum teams, this is the domain of the product owner or the project manager who is responsible for acting as the liaison between the engineering teams and the business stakeholders. Their job, roughly speaking, is to maximize the value the team delivers each sprint by ensuring the team is always working on the items that are most important to the stakeholders. They are effectively the person who determines the priority of the to-do list. At most companies, these individuals are typically less technical in nature but much more business-savvy. Which, if you've never seen a group of engineers attempt to communicate with business people, is definitely not a bad thing.

The rub with this setup is engineers can't always be working on the items that provide direct benefit for the stakeholders. There are many necessary evils in IT that engineers must devote their time and energy to in order to keep the engine churning. It is the omni-present threat of crippling technical debt. Software currency is perhaps the easy example to point at when it comes to technical debt, but it comes in many forms. And it is extremely difficult for engineers to adequately communicate exactly how important it is to address technical debt. The conversations usually goes something like this:

Engineers: "Hey this piece of software is end-of-life, and we really need to have time this sprint to update it.

Product Owner: "Okay, what happens if we don't update it?"

Egrs: "Well... nothing right now. But it'll be a problem someday."*

PO: "When exactly is 'someday'?"

Egrs: "Hard to say, but it'll be a problem, and we need to address it."

PO: "Will it be a problem in the next sprint or two?"

Egrs: "Probably not, but someday it'll bite us."

This conversation happens every. single. time. there is backlog refinement meeting on all of the user stories that don't clearly provide direct business value. The product owner, who has no idea when "someday" will occur, will naturally lean towards deprioritizing the technical debt user stories in favor of stories that provide more immediate value to the stakeholders because it is their job to do so. At first the decision seems correct and all is fine & dandy... until one day it isn't. There will be a day when the volcano of technical debt explodes, your team is caught in the lava flow, and you're looking at a solid 6 months of slogging through that molted lava of debt. When it happens, and it will happen, there will be a predictable chorus of I-told-you-so's as everyone takes a step back to wonder why they are running production Ubuntu 14.04 servers in the year 2024 (true story).

Now obviously this situation is a problem that many organizations encounter regardless of whether they are operating under the scrum framework or not. The temptation to ignore technical debt is real, and it is not a uniquely scrum problem. However, working under a scrum framework means you are continually refining the backlog, which in-turn makes it easier to encounter these issues. The two or three sprints you initially reserved for tackling technical debt tend to disappear as the associated user stories get refined to the bottom of the pile and completely forgotten about, never to see the light of day until the volcano explodes.

It is not the fault of the product owner, scrum master, or the engineers that these issue pops up. It's ultimately a shared responsibility between all parties to ensure these sorts of issues do not happen. But for whatever reason it is a shared responsibility that seems to be exceedingly easy to ignore while operating under scrum, especially when your backlog reaches the size of "mountain".

Stand-ups... or stand-downs?

Part of the scrum formula calls for daily scrums, which most people in my experience refer to as "stand-ups" regardless of whether you are standing or not. The scrum is facilitated by a scrum master to discuss the sprint's progress or discuss blockers that are preventing progress. In theory, this meeting is supposed to last less than 15 minutes, although in my personal experience it typically lasts around 30-40 minutes to give us ample time to once again talk about all the things we don't yet know. I'm guessing the goal of this meeting is to keep everyone engaged with each other's work, ensure progress is being made, and increase team synergy. But in practice it's just a means for the manager to complete his/her daily check-list so they can update superiors on the work being done. Most engineers don't actually listen to what their team members are doing, they just silently plug away at their tasks (or make themselves coffee) with their microphone muted until their name is called by the scrum master.

Admittedly a "true" stand-up, in which everyone is actually standing in the same room, would largely resolve this problem. But in a world where hybrid and remote work is commonplace, we need something better than these mindless daily scrums. And if no alternative can be found, just let us place an update in the Teams/Slack channel so we can go about our day.

Not Another Agile Ceremony

The biggest issue I have with scrum (and agile in general) is the sheer amount of meetings. Sprint plannings that can take hours, daily scrums that are rarely 15 minutes or less, post-sprint retrospectives that take hours and often turn into complaining festivals. And worst of all, the increment planning meetings—intended to allow organizations to plan out several months in advance—can easily chew up several days and suck away your will to do anything remotely productive for an entire week. These meetings got so bad at a previous employer that my boss at the time coined the term "NAAC", short for "Not Another Agile Ceremony". "Oh, sorry, can't talk right now I have to go to another NAAC."

It would be one thing if these meetings were productive. But when you can feel scrum isn't working... when you know your velocity numbers are nonsense, when you start throwing out random meaningless story points so the meeting ends quicker, when you discuss the same unresolved issues in the retrospectives sprint after sprint after sprint, it just becomes an utterly painful exercise. Judging by the level of interaction (or lack thereof), I would guess a good chunk of engineers out there have stopped expecting to gain value from these NAACs, and instead opt to silently sit in the meeting while doing something more productive, whether that be working on unfinished stories from previous sprints, or even finishing up their laundry.

But don't worry! Bring in the agile coaches to tell us that we're doing it wrong (thanks captain obvious), and let's go to more NAACs to talk about how we're actually supposed to do scrum. Maybe, just maybe, when we've spent more time talking about how to organize the work than it actually takes to do the work, all will be fixed! The punishments will continue until morale improves!!

(Save me.)

But can it be saved?

Can scrum work? Honestly, I don't know.

Unless there is some grand conspiracy to reduce productivity industry wide, I imagine that scrum is truly working for a lot of people, and I've just been unlucky with the 5-6 teams that I've been apart over the past decade. Or maybe I'm just expecting too much from scrum. Or maybe we should be using SAFe instead. I honestly have no idea.

The fundamental issue, I suspect, is switching from a more traditional waterfall-like workflow to one of the many agile workflows requires a fundamental change in how the business is structured and operates. On top of that, it only can truly work if the culture of the company itself changes. And honestly, that's hard to do. These companies are big ships with a lot of built-up inertia. It requires a tremendous amount of leadership to change the direction of that ship. It's just straight-up a hard thing to do, and very few leaders are capable of making it happen. Most leaders seem to prefer the "agile-in-name-only" approach rather than truly putting in the work to fundamentally change the company.

But regardless of the exact reason, my scrum master announced last week that the team will temporarily be changing to a Kanban workflow after an internal company reorganization, and I was more excited than I care to admit. Not that Kanban is some magical framework--many of the issues I've discussed with Scrum also apply to Kanban. But at least there is no longer a pressure to tightly squeeze everything into 2-week units of work. No more of taking square-peg-shaped units of work and shoving it into a round-hole-shaped sprints. And for that alone, I'm thankful.