Time Is Costly, Estimation Is Key: How To Estimate Projects

There’s no controversial topic greater than estimation among developers and product managers.

There are people who believe it’s an absolute waste of time to estimate.

And then, there are others who think proceeding without estimating is as good as moving forward without a goal.

But the one thing both of them agree is, it's extremely hard to get developer teams to come even close to estimating accurately.

The thing about estimates is, they’re almost always wrong because the tasks estimated are new, vague, and unknown.

Almost every time you think you’re finally going to ship on time, you’re told by your engineering team that they ran into a series of unexpected bugs and issues, and you’re not going to launch on time, again.

It’s frustrating, right?

I’ve been through it too.

Every time I’m told we can’t launch on schedule, a part of my heart cries and wishes there was a way to estimate more accurately.

And the other part, well, I hate to admit it, but it tries to rationalize that estimating is a lot like predicting the future - sometimes, just sometimes, you get it right. And a lot of times it’s as accurate as a dart-throwing chimp.

Since we started building Zepel, we’ve been working hard to push features on time so teams could plan, track, and ship their projects without all the hassle. That meant, we needed to deliver value and deliver it consistently.

So when we set out to find a solution for our estimation problem, we read several blog posts, saw how other teams were tackling this problem, and tried a couple of solutions ourselves.

We came across a few articles that came up with a generic formula, like: “Think how long you’ll need to build this feature, and triple it.”

Image credits: xkcd

And it obviously, wasn’t accurate.

During this process of trying various solutions, we learnt the hard way that there are no tricks or shortcuts; just processes we must go through.

Here’s the four-step process we use to estimate our projects:

The 4 Step Process On How To Estimate With Better Accuracy

1. Allocate Enough Time To Estimate:

People want to move quickly.

Product managers want to ship new features on-time so they can satisfy the needs of the customers. Developers want to build and ship products without issues at a higher frequency. And marketers want to run multiple campaigns to acquire users and gather data quickly for their next campaign.

In a world where people are constantly wanting to get things done quickly, it’s easy to want to dive head-first and try to tackle the problem the minute a task is assigned.

But if you want to make sure your teams get their estimates accurate, don’t start your conversation with your team by telling them the deadline.

When deadlines come from the top, which is usually from business people who aren’t involved in the day-to-day complexities that come with developing the feature itself, it leads to setting unrealistic deadlines that cannot be met.

And when developers are forced to meet the deadlines, they’re most likely going to not think through the problem completely, underestimate the complexities, and come up with estimates that will satisfy the manager.

Image credits: Dilbert

Often times, product managers can fall into the trap of prioritizing a list of features, go to their team and say, “We need to ship this, this, and this, by the end of next month”, when in reality it would take them four months to build it without any issues or bugs.

When you start by telling your team that you’re going to spend the next half-a-day (depending on how big your project is, you could allocate more time) thinking through the problem, you allow them to think clearly of all the complexities and come up with a reasonable estimate.

2. Break It Down:

At the start of your project, you probably have very little data on what’s needed to be done to complete the project. Any number of days you come up for your estimate will most likely be off by a lot.

But when you think through the various aspects of your project and break it down into tiny pieces of actionable tasks, the uncertainty in your project reduces, and gives you a more accurate estimate.

Breaking down your project down to smaller, actionable tasks forces you to figure out what you’re going to do, and to a certain extent, even help you think through how you should be executing it.

In his blog post, Joel Spolsky talks about why it’s important to break down a task:

"When you haven’t thought about what you’re going to do, you can’t know how long it will take."

~ Joel Spolsky, CEO of Stack Overflow

A good rule of thumb to follow when breaking down your project to smaller tasks is to break them down further till the estimation of your tasks are less than 8 - 10 hours.

If you have tasks that are, say, 16 hours worth of work, by not break it down further, you can easily get misguided to think it’s less than a day’s worth of work, but in reality, 16 hours is at least two working days.

3. Group Tasks Under Buckets And Research:

Before you start working on those bite-sized, actionable tasks that you’ve successfully gathered by breaking down your project, take a step back and look at all your tasks. You’ll notice that you have a bunch of work that you can start working on immediately, few that look vague, and a couple that look as mysterious as the dark side of the moon.

Categorize them as:

A. Known Tasks:

These are tasks for which you have knowledge of all the ins and outs of what is required to execute efficiently. Ideally, you know how to work on this task and what you have to do to complete it within the estimated time.

B. Partially Known Tasks:

You have some knowledge on how to execute on these tasks, but need guidance or require to research further to confidently execute.

Normally, these are tasks that you’d say you need 15-30 mins to look around on the internet to see how others are solving this problem, or talk to another developer who has already worked on a similar problem.

C. Unknown Tasks:

Usually, these are tasks that will require you to spend anywhere between an hour to at least half a day to understand the technology that needs to go into it and think about how you’re going to execute on it.

Building software is rarely about building the same thing you already know how to execute. It’s mostly about finding ways to make the different set of tools (APIs and libraries) work for you, the way you want them to. That means research plays a big role in getting estimates accurately.

With all your tasks categorized, start researching tasks under Partially Known Tasks bucket and work your way down to Unknown Tasks.

The goal is to move all your tasks to Known Tasks bucket so you can estimate with all the knowledge about your work and give more accurate estimates.

4. Re-estimate:

Now that all your work is under the Known Tasks bucket, you have more information on how you’re going to implement them, making it easy for you to estimate more accurately.

We can understand this better by looking at a popular concept known as The Cone of Uncertainty which states that the more uncertain you are about how to implement a project, the more uncertain your estimates will be.

The Cone Of Uncertainty. Image credits: InformIT

The added advantage with re-estimating tasks is, you tend to think of different aspects of your problem and ask yourself questions that you missed earlier while researching.

And every time you get new information/knowledge on how you’re going to solve your problem, you can use it to re-estimate and get your estimates accurate.

4 Things To Keep In Mind During Project Estimation

1. Watch Out For The “Just”:

Have you noticed tasks you think will take “just” few minutes always end up taking a lot of time? These are usually the tasks that end up causing delays and missed deadlines.

Other variations of “just”, include:

• It’s just a small task.
• I’ll only need 5 mins to fix it.
• It shouldn’t take me more than 15 mins.

These “just” tasks end up being the primary cause of delays because they normally come up unexpectedly during a random conversation or stand up meetings. And if it’s a task that the developer did not think through, they will underestimate the complexity of the task and end up giving an estimate that will not be accurate.

2. Developers Estimate, Not Managers:

The developer who is assigned to work on the task is often the best person to think through the tiniest of tiny details that are required to implement the feature.

While managers are the ones who come up with the requirements, developers know it best when it comes to thinking through which APIs or third-party libraries they’ll need, giving them an idea of how long it might take for them to implement the feature.

3. Don’t Forget The Miscellaneous Tasks:

It’s easy to miss out on the miscellaneous tasks or even not think too much of them as they’re “just!” miscellaneous tasks. But when you have to check for bugs, fix them, review code, and deploy to production, you know you’ve got a hand full of tasks that could end up eating up a lot of time.

4. Consider Unexpected Delays:

People get sick, they go on vacations, public holidays come in between, and bugs show up like an unwelcome guest to a house party. Remember to count them into your final estimate based on your calendar and past experiences.

While there is no perfect number to add to your final estimate, you should ideally be able to take the average number of days your previous projects were delayed by and use them to give you an estimate.

Conclusion

When it comes to estimating, hitting the bullseye is hard.

Some say it’s impossible and think it’s worthless to spend any time even trying. And many understand the importance of estimating and try hard to get as close as possible to nailing it.

While we were researching, learning, and trying new ideas, we learned a key lesson about estimation:

The problem with estimation isn’t about estimating at all. It’s about creating a clear understanding and having a full picture of how to solve the problem.

Both developers and managers gain new information at different phases of the development process. Keeping everyone updated on new information helps the entire team to plan, re-estimate, and get a more accurate picture of when you’re likely to ship.

We’ve also learned that estimation isn’t only to help teams get a picture of when the next feature is expected to release.

In the process of trying to estimate accurately, it opened up an entire universe of ideas and solutions in our heads for us to build our features in a shorter time frame.

If you’re working with a team that collaborates with several team members to ship products and features on time, estimation can help foresee unexpected challenges...

And unexpected challenges are a good thing to know if you want to ship on time.