From #NoEstimates to #YesPrototypes
Massimo, how is it possible to do without estimates? Every single job starts with an estimate! Should we just go random?
(Pietro on first reading Massimo’s slides on #NoEstimates)
The problem starts right from the last sentence: “should we just go random?”.
Just a couple of weeks ago, I was asked to provide a “first estimate” for a complex project. I did my first estimate draft, and sent it to the customer. It resulted that I was asking too much. They wanted to have more details about the features included, so that they could pick the ones they were most interested in and delay the others for a following phase. I went more in detail, and what I knew from experience would happen, happened this time too: at each estimate revision, as I spelled out more detail, the amount total grew and grew.
We don’t think this is a rare experience. Estimates for non trivial projects go in large part random right from their inception, and Dilbert presented this in very clear detail.
The problem of reliable estimates
Estimates have the initial bug of being asked at the beginning of a project. In the classical perspective the feature set (which in reality will change several times) is assumed as exhaustive, and each feature is assumed as completely defined (no accidental changes, no deployment accidents and so on), isolated from the context where it is produced. But unfortunately cognitive biases will hit you hard whether you are aware about it or not, so resulting estimates quickly become detached from the production process.
The estimation problem will not be solved by revisions: tendentially estimates are not revised as they quickly mutate and take the form of contracts, so that the initial bug becomes fatally persistent.
This below is the the cone of uncertainty which describes the error delta in estimations during the project’s evolution in time. As you see below we statistically have 400% possible under and over estimation at the beginning of a project, when they usually morph into contracts.
What we see above is in the unrealistic case of requirements not changing during project development, so it is actually an optimistic representation.
Actually our latest experience with waterfall projects goes more like this:
We are aware of the existing literature and attempts to refine the Black art of Estimation.
The traditional focus on initial estimation and contract simplifies life for accountants (i.e. bureaucrats) and makes life hell for everyone else. E.g. if you ever had to deal with requesting European Union funding, you see this in action in its worse form.
Providing an overall estimate is a perfect tool to inhibit any kind of mature dialogue. It is like an arranged marriage, where communication with the bride is forbidden. A way to create intrinsically immature relationships.
From #NoEstimates to #YesPrototypes
#NoEstimates is not about ditching estimates. It is about improving the way we work such that estimates become redundant Neil Killick
The heart of our idea is this: the running dialogue with the customer is initiated by providing a prototype as soon as possible, not an estimate for an imaginary complete project. Building the prototype will elicit a learning and discovery process leading to a possibly wildly different prototype. The only form of classical estimate regards providing the prototype, it’s not about the entire the solution, not even a first running solution.
Our point is that in complex domains you don’t even know what you don’t know. This evidences how important is the learning process as part of the production process.
The idea is that complex domains do not get properly analyzed by meetings, but mostly through experimentation, by throwing models at the problem and seeing how they work.
How could we then reform the workflow from customer request to released product?
What we propose here is to always work prototype based. Any estimation is replaced by a “working prototype”. Any development cycle ends with a working prototype. This way we make the amount of money required for each step small and with a close and well defined objective — and so properly estimated.
This fits well with some agile contracts where your commitment is for a short development cycle that ends with the release of the prototype, and so the customer minimizes economical risk.
What is a prototype? We are aware that it is not trivial to distinguish between prototype and working solution, but it is essential to draw a line. We leave you dear reader with this easy exercise, as most appropriately defined for you domain. Usually a prototype is not something that ends up in production; it is not a working solution. It ends up set aside once working state is reached.
A prototype by being functionally explicit allows defining a slicing model and hence facilitates producing working and limited releases.
A serious case
In thinking about this story we kept in mind a large serious game project in which Pietro was involved a year ago as game designer.
Note that the case of serious games is particularly relevant as the domain is the fusion of several unrelated ones by definition and hence the resulting projects are always frontier ones, i.e. non trivial.
The project was estimated from start to end initially and was to be carried through a classical waterfall procedure. The budget was about one million Euro, was allocated completely at start and determined through a huge effort. As often the case in such projects, after the first review meeting with the customer, user requirements were changed deeply, making the first estimation obsolete (it was a work of fantasy anyway).
In this experience meetings with the customer resulted in a learning process and in acquiring (sometimes vaguely defined) information, refining domain knowledge and hence deeply changing the requirements. What could be a positive knowledge exchange experience got ruined by the looming estimate, mutated in zombie fixed price / fixed scope contract.
This kind of contract model is well described in this comment from a post on fixed price fixed scope contract:
…we called them You-f*ck-me, I-f*ck-you contracts simply because of the type of attitude these contracts foster…
Side note: The colorful name for fixed priced contracts was derived from the following scenario: You-f*ck-me by demanding the largest possible scope, in the shortest possible time, for the lowest possible cost. I-f*ck-you by charging a fortune for every change no matter how minor to the original specification (which we both know will be wrong because no one can predict the future).
This estimating obsession is reinforced in a negative loop by the customer asking for certainties where they do not actually exist, tempting towards un-ethical behaviour. The customer asks for lies, being uncomfortable with the unpredictability of the domain. The worse mistake is to model the process in order to satisfy this misleading fear. The process is designed for #epicFail instead of #happyPath.
It seems obvious (but it isn’t) that the procedure would have benefitted greatly by following a different sample approach:
Step 1. First prototype
The only commitment with the customer is to create a prototype (say using a tool like Balsamiq which helps in not confusing functional prototypes with visual concepts).
This would involve customer and production right from the start, removing the need of intermediaries and actively hindering misunderstanding due to bad translation between domain knowledge and technology.
This step could enter iteration, making say the prototype linked and including parts of the concept design.
Step 2. Working prototype
Provide a low technology working prototype, e.g. a JavaScript browser based engine covering the game mechanics, user feedback, connected web pages / social networks.
This could enter iterations allowing an exploration of the complex domain. One of the wonderful aspects of this perspective is that it brings into the picture learning and knowledge acquisition as relevant work components. Actually this is inspired by Deliberate Discovery. Notice that by having the working prototype you avoided the accumulation of uninspected black-box “user stories”: everything you have is working and open to exploration and debate.
All this allows very detailed minimally scoped plans, no long term estimating, and continuous revisions. You keep staying on a maximally efficient point.
Step 3. Minimal feature working solution
Having the working prototype from step 2 also allows easier feature slicing as features are functions; so the last step towards the first working prototype can be done with a minimal effort by picking a minimal set of functions and focusing in making the base real technological solution first run.
Finale
When a consolidated myth gets revealed as such, scandal ensues. We find it gorgeously funny that some people are made so unhappy by the destruction of the estimates myth, but they’ll survive :-)
The “no estimates” idea is simply a reality check: it makes sense for those whose experience matches with the problem. Reforming the contract would make it possible to redefine delivery in honest terms, maximizing transparency.
We are aware that the simple realization that not all contracts have to be “fixed price fixed scope” will not be incorporated easily by large organizations, unless their managers come from a different planet.
Closing up, there is one point which we would like to stress: that in all this reflections, the most important takeaway is becoming sensitive to the learning process which permeates the dialogue between customer and developers, which traditionally has not been considered as a dimension in determining costs and processes, and can instead be leveraged for project success.
As pointed out by Woody Zuill, what we are pointing out is merely …
Which is here:
We are here
This story is by Massimo Iacolare (.NET and web developer, more about him here; since this post he also has a future in infographics, as you surely noticed) and Pietro Polsinelli (game designer and developer, more about him here). We are available for providing complete estimates on your next, big development project :-D
Responsibility for any view or opinion here presented is entirely ours. We are open to revise the contents of this post, just send us a note.
References
#NoEstimates by Oleg Shanyuk
#NoEstimates by Massimo Iacolare
Software Estimation: Demystifying the Black Art by Steve McConnell
Thinking, Fast and Slow by Daniel Kahneman
The Twitter stream of Woody Zuill
Management science’s impossible quest: in search of predictability by Vasco Duarte
Neil Killick Blog
Relevant tags: estimates, no estimates, prototyping, agile, scrum, project management.
