One in six IT projects has an average cost overrun of 200% and a schedule overrun of 70%. Well I guess no one ever said the life of a project manager was going to be easy...
So how do we overcome such a seemingly insurmountable problem?
In November of last year I attended a Meetup organised by Agile-Lean Ireland, which included a presentation by Vasco Duarte, a Product Manager, Agile Practitioner and author of ‘No Estimates — How to measure project progress without estimating’. The #NoEstimates movement and book builds on agile and lean concepts, encouraging minimal use of estimates while focusing on activities that bring value to the end user. Attending the lecture got me thinking about estimates in software.
Nathaniel Branden, American Psychologist noted ‘The first step toward change is awareness’. So in an effort to build awareness, here’s some reasons why accurate estimates can be so hard.
“It always takes longer than you expect, even when you take into account Hofstadter’s Law.”
If you haven’t already, check out this excellent article by Michael Wolfe. It’s a great example of Hofstadter’s Law in practice, showing how, despite the best of intentions, tasks take more time to do than originally planned.
“Work expands so as to fill the time available for its completion.”
I would always include a factor of safety or buffer in my estimates. But how much to use? And where does that leave me with Parkinson’s Law? Striking a balance between risk and remaining competitive is key.
The risk profile and general outlook (positive/negative) of the individual will always affect their view of how long a task will take. And those providing estimates may not always be the individuals carrying out a particular task; senior engineer giving an estimate vs junior engineer completing the work. Often, estimates are required before a project team or project scope is even finalised.
For me, one of the key lessons from the #NoEstimates book was the concept that software systems have ‘nonlinear causality’ or in other words, the impact of a small action can be large. Before I joined the software industry, I worked as a Project Manager in the security and design industries. But the foundation of my project management experience came during a number of years working as a structural engineer in the construction industry. My more linear and constrained engineering brain has struggled at times adjusting to a software environment; the requirement for regression and assessing the impact of a current change on completed functionality. The visibility of the impacts of change and the constraints on flexibility are less pronounced in software and as such, harder to estimate.
Introduced by J.B. Rainsberger, Software Development Consultant and Technology writer in 2013, Accidental complication impacts the cost of adding a feature or functionality to an existing project. It is defined as the complication that creeps into a feature from business and organisational structures or issues working with an existing codebase, hard to define and hard to estimate.
“There are known knowns. There are things we know that we know. There are known unknowns. That is to say, there are things that we now know we don’t know. But there are also unknown unknowns. There are things we do not know we don’t know.”
One of my favourite quotes from Donald Rumsfeld. While applicable to all types of projects, I think it’s enhanced in a software environment. We learn much about features and functionality as we are developing them and new issues are also uncovered in this process. Some examples of this taken from the book include:
So now we are aware of the difficulties, what can we change?
Agile project management has helped in many ways to address the issues above. Breaking items down into smaller manageable stages and releases enables frequent review and assessment of progress, timelines and costs. Estimates, features and priorities can be evaluated based on the most up to date information available so any deviations or issues can be communicated early.
An article by Jasim A Basheer noted that the accuracy of an estimate is proportionate to the time and effort spent creating it. Giving sufficient time at the beginning of a project to allow for investigation and elaboration may delay providing an estimate upfront but can be more beneficial in the long run.
The #NoEstimates book suggests that we should not view additional features/functionality as scope creep but as value discovered, changing a negative to a positive. As we progress our development, we get greater understanding of what our software needs to deliver and the problems surrounding it. Yes this results in unforeseen and additional work but if these items are bringing value to customers and improving our software, it should be viewed as a positive, right? Maybe, keeping an open mind to change and discovery will help us make this adjustment.
Software development will have unknowns and estimates will change. Similar to risks, being aware of the potential issues can help us prepare for them. We can work to try and improve our estimating ability but maybe the best start is to change our attitude and expectations around estimates; moving from waterfall to agile, from static to dynamic. If our estimates are open to uncertainty and change, is it still right for us to measure our performance and more significantly, attribute our project costs against them? If the first obstacle is our outlook on estimates, the next challenge will be reviewing how procurement and budgeting methodologies can adjust to cater for this new viewpoint.
Adjusting our attitude and expectations may not solve all the issues noted above, but maybe the next collection of project management statistics might just look that little bit better.
We’re ready to start the conversation however best suits you - on the phone at
+353 (0)1 833 7392 or by email
