We've all heard someone say something like:
Estimation is impossible.
What's the point in estimating?
Every estimate is just a SWAG.
One of my early experiences as a professional software developer was when a project manager brought me aside and told me that the way to make estimates was by the following process:
- Write down how long you think it will take. Example: Two weeks.
- Double it. Example: Four weeks.
- Increase the unit. Example: Four months.
What is estimation?
If all of these things are true, what even is estimation? I think this unanswered question is core to the myth of "estimation is impossible." Whenever I am in a conversation about estimation, I always come away with the strong sense that everyone is talking about different things. One person says estimation is possible because their company delivers to deadlines all the time, proving so. Another person is adamant that developers can't make estimates because too much is outside of their control or even outside of their knowledge. A third will say that software is an amorphous intangible abstraction so you can't estimate it.
None of these people defined estimation. Is it a deadline (a date) when something is delivered? Does estimation inherently involve knowing exactly what that something is? Is estimation inclusive of how many people, how much money, and for how long a company will work on one "thing?"
Estimation techniques
Besides the lack of clarity, there's another problem I find in these conversations. They only involve the following estimation technique:
Sit down and stare at the wall for awhile, then write down how long you think it will take to deliver the functionality in question.
At best, people discuss "planning poker" or having a conversation among the entire team about the delivery of a feature. Besides just arbitrarily throwing out numbers, this is the only game in town.
Learning to estimate
Did you know that the average developer tenure of tech companies is less than 2 years?
I bet if you think of all the people you've ever worked with, the colleague who made the most accurate estimates was either someone who had worked at your company for a long time or someone who worked in the same general industry (maybe e-commerce or SaaS).
This suggests that you can learn to estimate.
It also suggests why no one knows how to estimate.
So how do you?
Steve McConnell, the author of Code Complete, also wrote an oft-ignored book called Software Estimation: Demystifying the Black Art. It tells you exactly how to make good estimates. It defines what estimation is. It gives you more techniques than throwing numbers out. And it boils down to this: every organization can learn to estimate, if they actually kept data on their projects.
McConnell writes the following:
The most important reason to use historical data from your own organization is that it improves estimation accuracy. The use of historical data, or “documented facts,” is negatively correlated with cost and schedule overruns—that is, projects that have been estimated using historical data tend not to have overruns (Lederer and Prasad 1992).
My bet is that no one reading this article has ever worked in an organization that has historical project data from their own organization at hand when they make estimates. I make this bet confidently based on my own experience at six large companies as well as the numerous conversations I've had with software professionals from dozens of other companies including Amazon, Microsoft, and Google.
It sounds shocking that companies have ignored this simple practice, but they have.
Ending on a low note
If you would like to make better estimates, I highly recommend reading McConnell's book, because it will help. Using even the simplest tools that McConnell covers like work-breakdown structures will improve your estimates.
On the other hand, I have bad news. Since you're not likely to work in an organization that collects project data and makes it available to everyone, accurate estimation is effectively impossible, even if it is emphatically not impossible.
I wish you the best of luck.
Top comments (21)
We’ll there is one thing you are wrong about…we actually look at our historical data in my team. :D
I 100% agree with what you wrote in this aritcle. Although I would definitely emphasize that it is not something you learn alone. It’s something that you learn as a team. And in every new team, you will have to relearn it (perhaps not from ground up, but it will be different every time). And even if you have, when new a new member joins the team, with new insights, I think even then the way you estimate is liable to change and evolve.
That's amazing! You are the first team I've ever heard who actually does this. I'm flabbergasted, in a good way. Great points! I wonder if you might write an article about it? I think I would learn a lot from it.
I will definitely talk to my team at @storyblokcom!!
@lorenaramonda what do you think? Maybe we can work together on this!
Sure! I think we definitely can! 😁
People and Teams estimate LARGE tasks, there is a lot of uncertainty as there are unknowns in big things that are not fully detailed. When you break down large items to MEDIUM ones you get close to the work you are doing, so you could still estimate but you haven't delved down enough. What needs to be down is break the LARGE items down to MEDIUM blocks of work that can be separated out, and then make SMALL tasks that define the work at a human level. SMALL can be given a time, or if you like an estimate that is Measurable (like in that SMART thing) and can be referred back to as you work. But you can add up all those SMALL parts that have time associated and then get an idea of the LARGE.
The thing is you have time to break it down and get an idea of how long those SMALL items will take, when you are given a LARGE and moments to say a time then its either "when are you telling me to have it done?" or "would you like me to break it down and give you a realistic idea of how long it will take?"
I agree. Sounds exactly like a work breakdown structure which is also covered in Demystifying the Black Art. Few people working in software are aware of even the concept in my experience, let alone actually practice it.
At one job, I put a work breakdown structure together for a task and emailed it to my manager and project manager. It gave an estimate ranging between a best and worst case scenario. Both times were longer than all other proposed estimates. They nodded politely and ignored it until after the project was complete and it had taken a time hitting about the mid-point of the estimated range.
Then they came back to me and said "Can you make one of these for the next project?"
It's just something I have done in many jobs for years, working in Agile environments we needed small tasks and over time our ability to determine how long those small tasks can take got better. Honestly not hard to do, and to me seemed rather obvious.
I am reading this and my team has collected a lot of historical data like how many story points finished, how many planned, how many unplanned tasks, bugs, sick leaves and what not. But we are still nowhere if having good estimations in the sense of someone takes that to plan when feature X can be shipped to a customer. One thing based on the data that we collect that correlates very well with performance of the team is developer happiness. We ask every dev every Retro about their current feeling.
So, one question I would ask before “what is an estimation” that is “why do we need estimations?”
Estimations will not make you faster. It will not enable you to deliver anything earlier or better. So why not just work on the thing with the highest business value? Maybe, when in doubt one could use a rough estimation (X is bigger than Y) to decide on what feature with the same business value to start with?
I agree. I think that when the estimation relates to "When can we release this feature to customers?" but there's no other contingency on it than that, it really makes it difficult to find a compelling reason why you would need an estimate. Generally, you would think that what truly matters is exactly what you said: what has the highest business value?
There are a few times I have seen teams trying to coordinate with each other and communicate with various stakeholders where the estimate is meaningful as far as allocating resources and ordering work at the layer above. Think of a situation where a team is needed to support the efforts of several other teams, and thus they want to know if team A will need them only three weeks or six weeks, because team B will need them in three weeks.
A lot of the time such situations can be avoided if teams are self-sufficient, but in reality there are situations where that is not possible, e.g. when you are resource-constrained and trying to work lean.
Nice read, good insights. I see devs estimate on essential complexity but real develop time is massively dominated by accidental complexity, so estimations are inherently compromised.
J.B. Rainsberger touches on this in a wonderful ~7 minute lightning talk from 2013: t.co/wg1Rwt5qrw
When you suggest using past performance to inform estimates that really changes things, because it moves the estimation out of the realm of dev-theory and -feelings into the sharp uncompromising light of data. Estimations become less squishy, and the process is capable of learning, and those are gamechanging differences.
The best and simplest I've achieved was to record the start and end dates of stories, and then focusing the team on right-sizing all their stories to about an even level of complexity (that is: slice the stories as thinly as they can, but no thinner). That sets up a cadence of delivery that can be predicted quite well exclusively based on the completion metrics, so no planning-poker etc. is required. It's just a matter of doing the work and checking the "speedometer" to forecast future progress.
There are many ways to skin this particular cat and ultimately whatever works for a team and org. is fine, but I am deeply skeptical of participating in estimation-games that have no basis in data or ways of demonstrably self-improve.
FWIW I also blogged about this topic here: dev.to/jonlauridsen/estimates-dont...
Scott, have you tried estimating with twigflo? It integrates the work-breakdown into a fluid real-time visualization process that can come together at the speed of a brainstorming session. Here is an example -- linkedin.com/feed/update/urn:li:ac...
I have not. I'm going to check it out. Thanks Frank!
Historical data can tell you how accurate you were with your estimates. But what can it do to help you estimage a new, unknown piece of work?
If you've historically been under by 50% on one and over by 50% on the next, should you assume your next guess is going to be bang on the nose?
Breaking down a project into smaller pieces that you're better at estimating is a good idea, until it takes up more time than doing the work in the first place (which happens). And what you usually get is the transition from one big piece of work you can't estimate into 50 tiny pieces you can and 1 big blob you still can't. That's potentially useful, but...
I think you've named two arguments against making estimations, if I'm reading correctly.
Estimates can't be made from historical data because the historical data varies too widely. (Not clear if you are saying estimates never or just sometimes can't be made from historical data.)
Estimates (sometimes) can't be made because doing so would take longer than the actual task that's being estimated.
I think these are legitimate concerns.
I would respond by saying that, first, valid estimates always incorporate risks, and when risks are included, it means that estimates are in a range from "none of the risks are actualized" (no blockers at all during the project and everything works exactly as planned the first time) to "all of the risks are actualized" (not only does nothing ever work the first time, your company is also sold, or a natural disaster hits your area on top of everything else).
This is important because it addresses the first concern. There's more to the story than "my last two estimates for my last two projects were off by 50% either way so I guess estimates are always off." It means you have to incorporate into your historical data which risks occurred and how exactly they impacted the project.
Again, this is all covered in Demystifying the Black Art.
For the second concern, my impression is that a task that is small enough to execute faster than it can be estimated is obvious to everyone involved so I don't anticipate this happening except in dysfunctional orgs. I think bright, professional people working together on a project will know when to estimate automatically.
Even if estimations are possible and even if you can get better with time, my question is why do you need them in the first place. If you work on small user stories, get constant feedback, re-prioritize daily, and pick the topmost to work on, there’s no way you’re doing the wrong thing. If things are not delivered at the expected time, estimates wouldn’t have changed that anyway because the team was doing what mattered, ir order of importance. I'm assuming the team is not putting extra hours or doing nasty shortcuts just to finish on time (my full opinion).
I think that's a great question also. It is one of the questions underlying the "what even is estimation?" question. What you've said here is also something I often hear in the conversation. It shows that different people have a different idea about what estimation is necessary based on how they work. If you work in an organization that is free to "discover" the product through experimentation, then what you've said makes sense and explains why there is a segment of the tech industry that created the #NoEstimates hashtag.
Apart from that, though, I think there's maybe a blind spot in the idea expressed. Let me describe what I'm seeing and see what you think:
What is a small user story? There seems to be invisible estimation that happens to determine this?
If you "re-prioritize daily" is there not estimation involved? People usually take some kind of estimate into a priority consideration. I've personally never experienced a team determining the priority of a story without an estimate of some kind as a criteria.
let me first state that I don't want to be extremist and say that estimations are never useful. I've been in a few places where they were not useful and it worked out fine.
regarding your points: yes, maybe there's an implicit estimation under-the-hood. but:
also, beware that the division/slicing that I mentioned happens when kicking-off the story (or close to it); not in a separate dedicated phase (that in some cases feels a bit waterfallish).
regarding prioritization, other factors come into play, but yes, size can also be implicitly there (to recognize low hanging fruits for example), although again, not as a first-citizen but as a secondary subject.
That's all fair. There's certainly an element of toxicity around estimates in many (all?) orgs. You summarize them well.
Estimation can have a purpose, but to make any sense you need to know what exactly you are estimating.
You should know, if you are estimating e.g. with time or complexity.
If your team consists of members with different experience and focuses, it makes a difference, if you are estimating with certain people in mind or not.
Also knowledge of the business it not necessarily what you need, knowledge with the technology and solving problems with it are important.
The part with historical data makes sense, if you have enough data and are doing similar work with similar team members.
Estimating complexity always sounds good on paper - after all, it's independent of developer skill, and considers the work involved more. But I have found it to be a double-edged sword.
For one, if you have substantial skill differences on the team, it does prove to be a depressing force for the weaker members, as they are inevitably compared to the stronger members, even if the difference stems from something as simple as experience.
The other problem is that you need to estimate with a thorough understanding of the problem, which can often be difficult. Adding a research modifier to resolve that only muddies the estimate, so you end up with either very clear solution plans before ever estimating (which leads to tension when it comes to implementation), or very difficult and often very wrong estimates when assumptions don't line up with the reality of the problem.
I tend to estimate for the weakest team member, but in terms of "how long should it take?", translated to a bit of a blend between time and complexity estimates. If a task turns out to be a lot more intensive than anticipated, with this framework it generally implies flaws in the codebase to be resolved - bad DX leading to slow resolution, essentially.
Usually, I'm bang on the money with my worst-case estimates (when problems do arise), but the recorded estimate tends to drift a bit between worst and best. It's far from ideal, but it provides better feedback to stakeholders than any other method I've found so far.