The case for estimation, capacity planning and forecasting: these are not the same!
Updated: Jan 12, 2020
When will that be ready? It is a question we all either asked or had to answer at least once, personally or professionally. And it makes sense: we, humans, exist within the notion of time and understanding where things fit in time is part of how we carry our lives.
In a less philosophical explanation, we just all live by dates. We have to deliver a paper by Friday, we need to buy a Christmas gift before December 25th, etc. Estimation even involve budget: the paper cannot be more than 10 pages long, the gift cannot cost more that 50 bucks. So, we can see that estimating is practical. And we all do it. But what is it?
The meaning of the word estimation as per Google:
Meaning, a guess. Look at your favorite dictionary to verify the same answer. When we estimate, we have a guess. It is not a science, it is not precise, it cannot be, from its very nature. As an example, you might buy that gift for 55 bucks instead of 50 because it was the best gift you found for your aunt Corine before Christmas and that would be an actual better ROI for your money. One might want to be able to meet with friends this weekend but emergency in the family prevented the gathering. One cannot control all elements surrounding the estimation. Therefore, it is actually difficult to become better at estimating per se.
Now, that does not make estimating useless. Diving into the Agile world, I particularly find estimating an extremely important exercise with all my teams, never to be skipped. I like the idea of estimation as understanding. We estimate to understand the task ahead. I also like relative estimation, such as t-shirt size and story points in Fibonacci. When Sara says that that login page is of medium complexity and Allan says it is extra-small, very rarely are they talking about the exact same set of steps to complete it. As they each explain their rationale, we discover that for Sara this involves also the link "Forgot your password"? which involves generating a new secure link for the user to reset the password, whereas Allan was basically thinking username and password with a "Submit" button and a "Sign in with Google" option.
Some teams and individuals like to estimate in days. Many times that is an attempt to understand if they can hit a certain milestone and accept or reject a delivery date. I also find it very useful, provided we understand that this is again, not a a certainty. It is also sometimes dangerous to estimate in days because some people forget that a lot goes in a day, not just the time actually focusing on deliverable work. There are meetings, unforeseen troubles, absence in the team. So, estimating a day as in 4 to 6 hours and giving it some extra buffer is actually a healthy exercise. It explains the rationale of a team, for the team itself and also for its stakeholders. It sets the intention. It makes the assumptions clear, and the assumptions in it go beyond than just the date itself. So, estimating that something takes five days to complete does not mean that if I start the activity Monday I will finish on Friday. It only means that, to the best of my ability, as far as I can see today, I believe that in good circumstances the task at hand takes me about 5 days and that is rather safe to put it in the sprint.
So, that being understood, I do make a case for never skipping estimation. Now, what about forecasting?
What I think many coaches have a hard time to correct in most teams and even beyond the teams themselves is the notion of forecasting. Many times what teams and management are explaining as estimation is the actual forecasting practice.
Forecast is being able to predict when something will happen. In most cases for us, when something will be ready or delivered. Forecast is 100% independent of estimation. I can estimate that it takes me 3 days to build a website from scratch but that can take me anywhere from 1 day or less to a never completed website. It might even take me exactly 3 days. But it does not takes 3 days because I said so. It might take that in average. It might take that, let's say, 80% of the time. But it might fail big time and take six times more time!
What is important to leverage forecasting is to have in place a system, manual or _ideally_automated, to collect actual data of when I have been finishing similar tasks. That is why the way we break down work is so important and why we need to focus on helping our teams to understand how to find their ideal breakdown, categories and characteristics of work. With such things in place we can look back in time and use from simple averages and medians to complex algorithms and simulations (such as Monte Carlo) and then try and predict when something will be done. Those simulations look at was actually done and when. Not just when Sara finished coding that page, but actually when Sara finished it, waited for the external UAT team to approve it, the usual delays introduced from the deployment handled by the IS team. And that is what really matter. This is a probabilistic approach to forecast delivery dates based on team's throughput.
Usually those computations are not easy and that is why an automated system is more appropriate. One single task can take more or less time depending on the state of mind of a developer, the amount of trouble in the hand-off of the task and all sort of random events impacting the days of work. And one cannot determine any of those facts. The nature of forecasting is NOT deterministic. Deterministic approaches assume infinity autonomy and complete isolation from external influences, which is something that not even all scientific experiments can guarantee. Even for writing our own names it takes a variation on the number of milliseconds to do so. Because the granularity in this time-frame is so small, we perceive it as non-existent. We ignore it. But the variation is there, making it non-deterministic. That is when we say that things fall into a range. That is when we say we have 75% confidence into something. It is not deterministic, it is probabilistic. But I will take that sort of studied uncertainty any day of the week instead of a random individual guess for a date I will not make it for real!
We do not even need to get creative on that front. Two very smart folks have been developing tools that make it easy for teams (or individuals) to forecast their work based on actual data they have about their work. One of them offers a free tool in the form of an Excel spreadsheet, Troy Magennis. Troy makes an important case for metrics in general and he has a famous free tool in which you literally enter your data (the user stories and when they were finished or the breakdown data you might have) and get forecast based on percentiles, so it is up to you to decide if you want to be 100% sure of the final date, if you are a risk taker and you will use the date on the 50% percentile or if you are going to take a more balanced approach and stay on the 85% percentile, where usually most things will be done.
Another great guy with didactic and fun material on forecasting form the Kanban world approach (cycle time, throughput, etc.) is Daniel Vacanti. His book is an absolute must-read and he also developed a tool that is now integrated in Jira via plugin, called Actionable Agile. Among other things, we can understand the teams cycle time, throughput and have an idea of when things will be ready, once again based on actual historical data. So, irregardless on how complex things were, the tools look at what is delivered then.
That is actually the beauty of most throughput analysis: all you need to start is an item with an end date. What most analysis, automated or manual, do is some sort of variation of incrementing a counter for a stage when something arrived at that certain stage. You can look at your whole process or you can look at specific stages, such as in development, ready for production, etc.
Forecasting is really attempting to answer the question: when will it be done? And honestly it has very little to do with what the team thinks that can be done or not. If the team never managed to deliver a fully functional page per sprint, although not impossible, it is unlikely to happen if all circumstances remain unchanged, like adding more people or simplifying the page. But past performance does not limit what a team can do and should never be used to squash ambition to do better. It should be used as a tool through which fruitful discussion can take place on how the team can improve and serve as well as a solid material for negotiating probable dates with clients. Probable. Remind your client on a probabilities 101 session! Forecast dates are the ones we give to the best of our abilities. They still can deviate. But they are still better than the guess of a team.
Capacity planning is also an important element to take into consideration, despite the old school nomenclature. The capacity of a team is based on factual and intangible data. Factual data is how many people we have in the team? Will there be any holiday? Intangibles have huge impact that we cannot exactly predict, but we know will hit us: do we need to onboard a new team member? Is a team member going through personal problems in their life? Are we changing technology and no one is very skilled on that?
I like to think of capacity planning as yet another layer for the conversation when planning as a team. Bringing things to the surface that help us to prepare and place a strategic bet on how we want to commit and operate for the given iteration, or sprint, or project.
Capacity planning, forecasting and estimation are NOT the same thing. They are all useful elements of a team planning their iteration. They are misunderstood but they are actually rather simple: Understand the context and constraints (capacity), understand the problem (estimate) and commit to a plausible date (forecasting).
To summarize, we could use with an example from outside of the usual suspects of the Agile world. In 2016 I changed my house's front door for something more beautiful and technologically more efficient, weather-wise. I did not do so on a whim. I checked with several companies and I want to know the prices and probable dates. Sound familiar?
The process from the company perspective was to come and measure my door, my entrance, check the materials my house is built with. Based on that they can estimate. They can understand their work. Door of such size, with such material; usually costs so much. Measures. Estimation. After which, they used their knowledge of their team schedule and their suppliers to understand who would be available on what time. That is their capacity planning. Based on their data on how long it usually takes, for my type of door and materials (it was a new model and brand) they could forecast a date. And for the engineers who love to say that in more engineering, standardized types of work things work better than in software and knowledge work, I am sad to say that the door unfortunately was delivered with 100% of error on the final date, arriving two weeks after the estimated two weeks delivery time they thought possible. So yes, estimation is not forecasting and forecasting is not deterministic.
This post was not a crash course in forecast and estimation techniques, as we sure have lots of them already and from reputable sources and every Agile gathering will have at least one workshop in which you can experience them live. And you should! What this post had the intention was to separate Estimation from Forecasting from Capacity Planning and to incite a discussion for these three elements as important components for a disciplined team, pursuing high performance, whenever they plan, whenever they intend to improve their delivery performance.