Estimating software tasks is a very challenging. Inspired by a recent podcast and blog post, I want to pass along some ideas that may help.
Planning poker is a tool that I’ve used successfully in the past. Mike Cohn was an early advocate of this approach and has an excellent overview at https://www.mountaingoatsoftware.com/agile/planning-poker. There are multiple reasons why I like the approach:
- It’s a way to quickly survey several expert opinions.
- The conversation that comes from any initial delta in the individual estimates leads to a significant increase in clarity.
- Using the fibonacci series approach for the estimates (1,2,3,5,8,13,21) drives appropriate precision in the estimates. A larger estimate is less precise and using the series acknowledges this.
Another approach that I’ve used is to make two estimates – expected case and worst case. The difference between these estimates is a good indicator of the risk. A 2 day estimate with a 3 day worst case is a significantly different task than one with a 2 day estimate with a 2 week worst case.
The reason why this second approach works came up in last week’s Linear Digressions podcast. The podcast was discussing Erik Bernhardsson’s blog post, “Why software projects take longer than you think – a statistical model“. In his post, Erik asserts that the statistical distribution of a software estimate is a log normal distribution.
I brought up a log normal distribution in Simulation in R, Part 2 and have included a sample plot below. Log normal distributions model software estimates well because a) they are never less than 0, b) will not be significantly less than the median estimate (the tallest bar), and c) have long tails of values that are more common than we would like in the software engineering world.
Another characteristic of a log normal actual distribution is that the mean estimate is higher than the median estimate. In the example above, the mean estimate is about 10 and the median is about 8. While that 25% difference doesn’t seem that high, those differences can add up across many estimates. There are also tasks with a higher standard deviation that result in a bigger difference.
In his blog post, Erik makes a good numeric case demonstrating the impact of this distribution on software estimates. He makes a couple of observations that I agree with. First, software engineers are much better at estimating the median than the mean. Second, tasks with the most uncertainty will dominate the project resulting in a high “blow up factor”.
Armed with the simulations work from my Operations Research class last fall, it would be very interesting to model a project’s estimates with the proper distribution and run some simulations to find the actual mean along with result ranges. In Simulation in R, Part 2, simulations of my 15 week home project (median estimate) showed that it would take 18 weeks on average and be completed between 13 and 21 weeks about half the time. And that was with several tasks using distributions with smaller potential blow up factors than a log normal distribution.
Using a estimation simulation result enables you to choose which percentile that you want to use when planning the project. A project with a large penalty would encourage you to use a conservative estimate from the 90th percentile, for example. Going back to my home project example, a smart person tackling a home project before a major event like a graduation might start the project using the 90th result in the simulation to increase the likelihood of completion. Of course, an argument could be made that a ‘smart person’ wouldn’t tackle any home project before a major event… but we know better than that!
If nothing else, thinking about software estimates using a statistical approach will improve your process and help you understand why they so frequently go wrong. Give a listen to Linear Digressions podcast and /or read Erik Bernhardsson’s blog post, “Why software projects take longer than you think – a statistical model“, to learn more!
Picture details: “Splotchy sunrise”, 5/10/2019, Canon Power Sot SD4000, f/4, 1/200 s, ISO-1600, –1 step