I'm on the faculty at Boston University in the computer science department, where I teach software engineering, intro courses, and application architecture and development. Also a bit of a Deadhead.
I can't argue that the way we estimate software projects is a problem, but it's the way most non-trivial projects are managed. We normally don't call the extra time padding...really it is a buffer to protect what really IS important in most projects, which is the completion date. That date often is set by product or marketing management, and if it is missed, the result can be as bad as losing market share in a quickly moving market, or missing revenue trargets for a quarter or more.
Typically projects are broken down into work units and laid out as a connected acyclic graph that relfects their dependencies. Estimates are applied to each, both for an optomistic and a pessimistic schedule. The graph will usually have many paths that converge to a delivery in the future. One or more of those paths will be very tight schedule-wise...it has the least tolerance for late starts on tasks such that ANY late start will cause the delivery date to slip. This path through the project is called the 'critical path' and is closely managed since it directly affects the end date.
The critical path is determined by analyzing start and end estimates across the entire graph; we assign a value called 'slack' to each task that reflects how late that task can start before it causes the next task to slip. For example, if I have a task that takes five days, but the following task converges into a path that doesn't need to start for another ten days, I have five days of slack, meaning that I can delay the start of the earlier task by that much without delaying anything else. In some cases I can reassign the resources on the shorter task to another task that needs help.
Since the critical path so directly drives the delivery date, and the delivery date i often a hard stop, when planning a project we'll add a risk buffer to the critical path (and thus the entire project). The critical path has little or no slack, and so that extra time tends to be arbitrary. It's like an insurance policy...I'm happy to pay $1,500 to protect against missing a delivery date that might cost me $10,000.
A method that isn't quite so arbitrary is called Critical Chain, in which slack times across each unique path are gathered into mini-buffers at the end of each of these feeder paths, then rolle into a buffer that sits prior to the delivery date. Managing a Critical Chain project is a little easier than a Critical Path project since we can examine the size and burn rate of the buffer at the end to get a sense of how the project is going overall.
Both methods still require estimates for early and late starts. The project team, and especially the project manager, need to provide reasonable estimates and be able to recognize estimates that are outliers.
I'm a self-employed developer who doesn't really like primarily defining himself by his income source. Check out my website and other stuff to get a better - but still woefully incomplete - idea of...
This description of project management graphs doesn't begin to address the fact that estimates of non-trivial, unique software-creating tasks are a total wild guess when given at the start of a project. The only way to provide anything close to a reasonable estimate is by knowing as much as possible about everything involved (and even then it could be way off) and that state of knowledge is never present at the outset. You never know less than you do on Day 1 - therefore your estimates only ever improve as you go on in a software project.
This is the core principle of Agile and the reason why big up front designs and their resulting estimates are disasters. How many software projects need to suffer the same fate before everyone is finally willing to admit this?
You very succinctly identified (and ignored) the crux of the problem:
[the completion date] often is set by product or marketing management
How can a date be set by a team that has no knowledge of the process involved in completing the tasks that must be completed by that date? If I give a contractor a completion date of 2 weeks to build my house, I'm the one to blame when the contractor misses the ill-informed deadline.
PS this isn't even taking into account user/product owner feedback, which can (and should) entirely re-steer a given project, making all initial estimates less than worthless and indeed harmful.
I'm on the faculty at Boston University in the computer science department, where I teach software engineering, intro courses, and application architecture and development. Also a bit of a Deadhead.
I'm guessing you haven't worked on any non-agile projects in your career. I like agile methodologies -- I'm a fan of the Rational Unified Process in particular -- and there is certainly a large class of projects where agile makes a lot of sense. A web site? Sure. A back-end equities trading platform? Not so much.
That said, most of my development and consulting career was spent in Fortune 500 companies (primarily financial services, some healthcare) where projects spanned 12 to 24 months, and exactly zero of the projects I was involved in for those clients used agile methodologies.
Day 1 estimates are not wild guesses, as you put it. They are informed by experience (of both the group and the developers), vetted by management, and in my experience are typically quite close to the actual effort. A large project usually starts with a lengthy requirements gathering stage, during which the architecture and functionality of the system are agreed on by both the business and the technology team.
Project planning at this scale is necessary to coordinate resources, equipment, schedules, marketing, QA/testing, purchasing, deployment, and so many other things. Individual contributors (especially devs) are expected to be capable of providing reasonable estimates; if they can't, they won't be working for long. I've pulled developers from projects and sent thm back to the bench for training when they've shown that they haven't learned how to do estimation yet.
And to that point, you can't just plop a new dev into a project and expect them to be able to provide proper estimates; it's a skill, just like coding, and it takes time and experience to get good at it.
To your last point on feedback from product management changing the project, that's not how it happens at this level. Once the project plan and requirements docs are signed off, there are no further ad-hoc changes. If the product team wants to make a change, the request goes through an approval process (I've often seen them denied) that might require new estimates and a new schedule that must also be signed off on by all parties if the change is significant.
I'm absolutely not saying that agile doesn't have its place, and in fact many of the larger projects I've worked on probably could have been done more quickly if parts of them were done that way. I am saying that not all projects are appropriate for agile and require significant planning and management effort. As a developer I think it's a mistake to brush off non-agile methodologies as too hard to estimate, or having too much overhead, or taking too long. Doing so is to shut off a large, lucrative chunk of software development opportunities.
BTW props for moving to the countryside, I'm not that far away and spend quite a bit of time in the summers down in LC and Westport. Beautiful place to live.
I'm a self-employed developer who doesn't really like primarily defining himself by his income source. Check out my website and other stuff to get a better - but still woefully incomplete - idea of...
I'm guessing you haven't worked on any non-agile projects in your career.
Sort of yes, sort of no. I've worked on clusterf*** projects that had no methodology/approach/management, and I've worked on projects that tried to be agile. The latter were always a bit smoother, but still suffered from lots of over-estimation and over-promising. The former were mostly some version of poorly-implemented Waterfall where the final result looked nothing like the initial plans.
[...]projects where agile makes a lot of sense. A web site? Sure. A back-end equities trading platform? Not so much.
True that I only have experience with web sites, web applications, simple command line apps, etc.
Day 1 estimates are not wild guesses, as you put it. They are informed by experience (of both the group and the developers), vetted by management, and in my experience are typically quite close to the actual effort. A large project usually starts with a lengthy requirements gathering stage, during which the architecture and functionality of the system are agreed on by both the business and the technology team.
I guess if the problems being approached mostly take the shape of other previously-approached problems then this is can be reliable (given an experienced team. I've also not had the joy of working on a very experienced team.) But what about for entirely novel problems, especially ones that are subject to great change once some initial assumptions start being tested in the hands of users?
Project planning at this scale is necessary to coordinate resources, equipment, schedules, marketing, QA/testing, purchasing, deployment, and so many other things.
This is an understandable problem. One that I don't have a solution for (
you can't just plop a new dev into a project and expect them to be able to provide proper estimates; it's a skill, just like coding, and it takes time and experience to get good at it.
This is an important point. But I find it suspicious that it can't be learned or studied other than gathering experience and learning from the experience of others. If tasks are so well-defined as to be accurately estimated by developers (and get the same estimate from disparate devs), then why can't the underlying principles guiding those estimates be extracted and taught from first principles - or even used to train managers and take the responsibility of estimating away from the devs?
Again, it brings me back to: if the tasks are so normalized and akin to other tasks in history, then why would estimates be needed in the first place? If you must estimate, it seems like that is because there is an element of the unknown or the unique involved, and those are - by definition - wildcard, variable-by-orders-of-magnitude factors to be adding into your clean math of dates and times.
To your last point on feedback from product management changing the project, that's not how it happens at this level. Once the project plan and requirements docs are signed off, there are no further ad-hoc changes. If the product team wants to make a change, the request goes through an approval process (I've often seen them denied) that might require new estimates and a new schedule that must also be signed off on by all parties if the change is significant.
Yep, it's either this very slow, cumbersome, expensive process you're describing - or do away with the estimates and approval teams altogether and embrace change. The time and money spent on this insane funnel would be better spent on some variance in time/cost, don't you think?
The obvious caveat here being that large corporations, governments, etc. won't be ok with that. But does that justify it? Haven't there been many instances of large organizations setting up little agile bubbles within their walls exactly in order to escape the process you're describing?
I'm absolutely not saying that agile doesn't have its place, and in fact many of the larger projects I've worked on probably could have been done more quickly if parts of them were done that way. I am saying that not all projects are appropriate for agile and require significant planning and management effort. As a developer I think it's a mistake to brush off non-agile methodologies as too hard to estimate, or having too much overhead, or taking too long. Doing so is to shut off a large, lucrative chunk of software development opportunities.
I'm taking your word for it that you're correct. But I've never seen that side of it and it seems incomprehensible to me. I've also read articles and books from folks who claim that agile methodologies as well as #noEstimates approaches can work in those situations... Since my bias is clearly in that direction those claims seem viable to me...
BTW props for moving to the countryside, I'm not that far away and spend quite a bit of time in the summers down in LC and Westport. Beautiful place to live.
IT'S HEAVEN! :D I'm happy to hear that a previously-anonymous avatar on the internet appreciates the beauty of this area. cheers!
I'm on the faculty at Boston University in the computer science department, where I teach software engineering, intro courses, and application architecture and development. Also a bit of a Deadhead.
Most projects, especially in large companies, really aren't all that innovative and are often just an extension of what's already there. In those cases you can use metrics to crunch most of the numbers to provide a fair estimate. Moderately complex database tables are 30 hours, DB triggers are 10 hours each, integrating a new data feed is 80, and so on. Once you have a large enough set of requirements you can just extract the pieces and do the math.
On the business side it gets even more interesting. We do projects to make money (why else do them?) and so companies want to run as many projects as possible to maximize profit. In a given year there might be 100 potential projects, but resources to run only a few...which ones? We use high-level estimates and ROI analysis to prioritize the projects. One company I worked with required a 10x ROI to fund a project.
Those high-level estimates also give an indication of how many resources you need. If it's 10,000 hours of work and I have six months, I'm gonna need ten devs minimum (roughly 2,000 man-hours in a year so 1,000 hours in six months). If I'm billing devs internally at $200/hour, I get a rough budget, too, in this case $2m. I know that the up-front phases (requirements and design) are typically 10% of the project, and QA another 20%, plus 10% contingency, and so I have a rough budget of $2.8m. That's enough to let me compare projects and prioritize them, and if a project gets green-lighted I can refine the estimates during the discovery phase.
The flip side is that if you grossly overestimate, all hell can break loose. If I'm managing that $2.8m project and get it done early and under budget, that is not necessarily good news! Let's say I brought the project in at $1.75m. Great! I just saved the company $1m! Unfortunately C-level management (CTO, CEO, etc) sees that as $1m tied up that couldv'e been used to fund other projects (and recall that more projects means more profit). Hence, a $2.8m project miraculously always comes in at $2.8m (unless there are issues that drive the number up, which is another story).
A truly unique endeavor would probably be funded out of an R&D budget rather than operations or product, treated as high-risk, and would be estimated / managed differently.
BTW that's just for internal projects. If it's being done by some other firm (consultancy, vendor, etc) there are additional costs, and the type of contract (fixed cost vs time-and-material or cost-plus, for example) will drastically change the quoted price. If I'm in a T&M situation there will be a cap on spend and schedule, beyond which requires C-level signoff, and if it's fixed-cost I'm going to add ANOTHER 10% to cover my risk (the risk is that it'll take longer, which means more spend on my part, even though I'm getting a fixed price). I love fixed-cost since there's an opportunity to work hard, finish early, and keep the extra dollars.
Next time you'r grabbing some ice cream at Gray's, keep going down main road about a mile, and on the left you'll see the old Friend's Meeting House. Across the road from it is the John Irish house, which my ancestor Jonathan attempted to burn down in the late 1600s. John Irish is buried on the property...I stopped at the grave once to apologize for Jonathan.
For small shops, startups, and the like there aren't a lot of competing projects (often just one), and so the budget is the internal cost of dev times however long it takes. There isn't a lot of overhead and requirements can be revised as you dev and learn.
For further actions, you may consider blocking this person and/or reporting abuse
We're a place where coders share, stay up-to-date and grow their careers.
I can't argue that the way we estimate software projects is a problem, but it's the way most non-trivial projects are managed. We normally don't call the extra time padding...really it is a buffer to protect what really IS important in most projects, which is the completion date. That date often is set by product or marketing management, and if it is missed, the result can be as bad as losing market share in a quickly moving market, or missing revenue trargets for a quarter or more.
Typically projects are broken down into work units and laid out as a connected acyclic graph that relfects their dependencies. Estimates are applied to each, both for an optomistic and a pessimistic schedule. The graph will usually have many paths that converge to a delivery in the future. One or more of those paths will be very tight schedule-wise...it has the least tolerance for late starts on tasks such that ANY late start will cause the delivery date to slip. This path through the project is called the 'critical path' and is closely managed since it directly affects the end date.
The critical path is determined by analyzing start and end estimates across the entire graph; we assign a value called 'slack' to each task that reflects how late that task can start before it causes the next task to slip. For example, if I have a task that takes five days, but the following task converges into a path that doesn't need to start for another ten days, I have five days of slack, meaning that I can delay the start of the earlier task by that much without delaying anything else. In some cases I can reassign the resources on the shorter task to another task that needs help.
Since the critical path so directly drives the delivery date, and the delivery date i often a hard stop, when planning a project we'll add a risk buffer to the critical path (and thus the entire project). The critical path has little or no slack, and so that extra time tends to be arbitrary. It's like an insurance policy...I'm happy to pay $1,500 to protect against missing a delivery date that might cost me $10,000.
A method that isn't quite so arbitrary is called Critical Chain, in which slack times across each unique path are gathered into mini-buffers at the end of each of these feeder paths, then rolle into a buffer that sits prior to the delivery date. Managing a Critical Chain project is a little easier than a Critical Path project since we can examine the size and burn rate of the buffer at the end to get a sense of how the project is going overall.
Both methods still require estimates for early and late starts. The project team, and especially the project manager, need to provide reasonable estimates and be able to recognize estimates that are outliers.
This description of project management graphs doesn't begin to address the fact that estimates of non-trivial, unique software-creating tasks are a total wild guess when given at the start of a project. The only way to provide anything close to a reasonable estimate is by knowing as much as possible about everything involved (and even then it could be way off) and that state of knowledge is never present at the outset. You never know less than you do on Day 1 - therefore your estimates only ever improve as you go on in a software project.
This is the core principle of Agile and the reason why big up front designs and their resulting estimates are disasters. How many software projects need to suffer the same fate before everyone is finally willing to admit this?
You very succinctly identified (and ignored) the crux of the problem:
How can a date be set by a team that has no knowledge of the process involved in completing the tasks that must be completed by that date? If I give a contractor a completion date of
2 weeksto build my house, I'm the one to blame when the contractor misses the ill-informed deadline.PS this isn't even taking into account user/product owner feedback, which can (and should) entirely re-steer a given project, making all initial estimates less than worthless and indeed harmful.
I'm guessing you haven't worked on any non-agile projects in your career. I like agile methodologies -- I'm a fan of the Rational Unified Process in particular -- and there is certainly a large class of projects where agile makes a lot of sense. A web site? Sure. A back-end equities trading platform? Not so much.
That said, most of my development and consulting career was spent in Fortune 500 companies (primarily financial services, some healthcare) where projects spanned 12 to 24 months, and exactly zero of the projects I was involved in for those clients used agile methodologies.
Day 1 estimates are not wild guesses, as you put it. They are informed by experience (of both the group and the developers), vetted by management, and in my experience are typically quite close to the actual effort. A large project usually starts with a lengthy requirements gathering stage, during which the architecture and functionality of the system are agreed on by both the business and the technology team.
Project planning at this scale is necessary to coordinate resources, equipment, schedules, marketing, QA/testing, purchasing, deployment, and so many other things. Individual contributors (especially devs) are expected to be capable of providing reasonable estimates; if they can't, they won't be working for long. I've pulled developers from projects and sent thm back to the bench for training when they've shown that they haven't learned how to do estimation yet.
And to that point, you can't just plop a new dev into a project and expect them to be able to provide proper estimates; it's a skill, just like coding, and it takes time and experience to get good at it.
To your last point on feedback from product management changing the project, that's not how it happens at this level. Once the project plan and requirements docs are signed off, there are no further ad-hoc changes. If the product team wants to make a change, the request goes through an approval process (I've often seen them denied) that might require new estimates and a new schedule that must also be signed off on by all parties if the change is significant.
I'm absolutely not saying that agile doesn't have its place, and in fact many of the larger projects I've worked on probably could have been done more quickly if parts of them were done that way. I am saying that not all projects are appropriate for agile and require significant planning and management effort. As a developer I think it's a mistake to brush off non-agile methodologies as too hard to estimate, or having too much overhead, or taking too long. Doing so is to shut off a large, lucrative chunk of software development opportunities.
BTW props for moving to the countryside, I'm not that far away and spend quite a bit of time in the summers down in LC and Westport. Beautiful place to live.
Sort of yes, sort of no. I've worked on clusterf*** projects that had no methodology/approach/management, and I've worked on projects that tried to be agile. The latter were always a bit smoother, but still suffered from lots of over-estimation and over-promising. The former were mostly some version of poorly-implemented Waterfall where the final result looked nothing like the initial plans.
True that I only have experience with web sites, web applications, simple command line apps, etc.
I guess if the problems being approached mostly take the shape of other previously-approached problems then this is can be reliable (given an experienced team. I've also not had the joy of working on a very experienced team.) But what about for entirely novel problems, especially ones that are subject to great change once some initial assumptions start being tested in the hands of users?
This is an understandable problem. One that I don't have a solution for (
This is an important point. But I find it suspicious that it can't be learned or studied other than gathering experience and learning from the experience of others. If tasks are so well-defined as to be accurately estimated by developers (and get the same estimate from disparate devs), then why can't the underlying principles guiding those estimates be extracted and taught from first principles - or even used to train managers and take the responsibility of estimating away from the devs?
Again, it brings me back to: if the tasks are so normalized and akin to other tasks in history, then why would estimates be needed in the first place? If you must estimate, it seems like that is because there is an element of
the unknownorthe uniqueinvolved, and those are - by definition - wildcard, variable-by-orders-of-magnitude factors to be adding into your clean math of dates and times.Yep, it's either this very slow, cumbersome, expensive process you're describing - or do away with the estimates and approval teams altogether and embrace change. The time and money spent on this insane funnel would be better spent on some variance in time/cost, don't you think?
The obvious caveat here being that large corporations, governments, etc. won't be ok with that. But does that justify it? Haven't there been many instances of large organizations setting up little agile bubbles within their walls exactly in order to escape the process you're describing?
I'm taking your word for it that you're correct. But I've never seen that side of it and it seems incomprehensible to me. I've also read articles and books from folks who claim that agile methodologies as well as #noEstimates approaches can work in those situations... Since my bias is clearly in that direction those claims seem viable to me...
IT'S HEAVEN! :D I'm happy to hear that a previously-anonymous avatar on the internet appreciates the beauty of this area. cheers!
Most projects, especially in large companies, really aren't all that innovative and are often just an extension of what's already there. In those cases you can use metrics to crunch most of the numbers to provide a fair estimate. Moderately complex database tables are 30 hours, DB triggers are 10 hours each, integrating a new data feed is 80, and so on. Once you have a large enough set of requirements you can just extract the pieces and do the math.
On the business side it gets even more interesting. We do projects to make money (why else do them?) and so companies want to run as many projects as possible to maximize profit. In a given year there might be 100 potential projects, but resources to run only a few...which ones? We use high-level estimates and ROI analysis to prioritize the projects. One company I worked with required a 10x ROI to fund a project.
Those high-level estimates also give an indication of how many resources you need. If it's 10,000 hours of work and I have six months, I'm gonna need ten devs minimum (roughly 2,000 man-hours in a year so 1,000 hours in six months). If I'm billing devs internally at $200/hour, I get a rough budget, too, in this case $2m. I know that the up-front phases (requirements and design) are typically 10% of the project, and QA another 20%, plus 10% contingency, and so I have a rough budget of $2.8m. That's enough to let me compare projects and prioritize them, and if a project gets green-lighted I can refine the estimates during the discovery phase.
The flip side is that if you grossly overestimate, all hell can break loose. If I'm managing that $2.8m project and get it done early and under budget, that is not necessarily good news! Let's say I brought the project in at $1.75m. Great! I just saved the company $1m! Unfortunately C-level management (CTO, CEO, etc) sees that as $1m tied up that couldv'e been used to fund other projects (and recall that more projects means more profit). Hence, a $2.8m project miraculously always comes in at $2.8m (unless there are issues that drive the number up, which is another story).
A truly unique endeavor would probably be funded out of an R&D budget rather than operations or product, treated as high-risk, and would be estimated / managed differently.
BTW that's just for internal projects. If it's being done by some other firm (consultancy, vendor, etc) there are additional costs, and the type of contract (fixed cost vs time-and-material or cost-plus, for example) will drastically change the quoted price. If I'm in a T&M situation there will be a cap on spend and schedule, beyond which requires C-level signoff, and if it's fixed-cost I'm going to add ANOTHER 10% to cover my risk (the risk is that it'll take longer, which means more spend on my part, even though I'm getting a fixed price). I love fixed-cost since there's an opportunity to work hard, finish early, and keep the extra dollars.
Next time you'r grabbing some ice cream at Gray's, keep going down main road about a mile, and on the left you'll see the old Friend's Meeting House. Across the road from it is the John Irish house, which my ancestor Jonathan attempted to burn down in the late 1600s. John Irish is buried on the property...I stopped at the grave once to apologize for Jonathan.
For small shops, startups, and the like there aren't a lot of competing projects (often just one), and so the budget is the internal cost of dev times however long it takes. There isn't a lot of overhead and requirements can be revised as you dev and learn.