The Problems with Project Risk Analysis and How to Fix Them

Major capital projects are famously late and over-budget. Why is this? And what can be done about it?

According to the International Journal of Innovation, Management and Technology (April 2017), over 85% of major public or private construction projects come in over budget and late. The average cost overrun is 28% above the estimate including contingency. Traditional risk analysis has failed in its promise to improve these statistics, resulting in challenging economics and inability of leadership to make reliable predictions about project outcomes. The purpose of this post is to summarize common shortcomings of traditional risk analysis approaches and to provide a description of best practices that have been proven to overcome these failure modes.

Shortcomings of Traditional Cost or Schedule Risk Analyses

* Project plans are often created using an ‘everything-goes-right’ mentality
* Deterministic estimates require making assumptions; typically, experts will not include any adverse events in their base (planning) assumptions * Motivated and high-functioning teams typically have an optimistic bias – “Despite the historical evidence, we have learned and that won’t happen on my watch”
* Probabilistic math shows how intuition about uncertainty can lead teams to underestimate risks.
* Schedules are often built on a “p50” basis, meaning there is a 50 percent chance of the duration actually being shorter or longer than the estimate. If four parallel independent activities are required to complete a project and each has a 50% chance of achieving their deadline, then the overall chance of meeting the project deadline is (0.5)^4 or only 6.25%, not 50%.
* Assessed ranges of uncertainty are often much too narrow. Well known cognitive biases drive people to put too much weight on what they know and not enough weight on what they don’t know, resulting in consistently and dramatically underestimating. * A focus on range of uncertainty for each individual cost item and schedule activity, neglect large, macro risk drivers. For example, labor productivity uncertainty might be assessed as plus or minus 10%, but the weather, strikes, labor unrest, simultaneous operation interference, permitting issues, etc. and their compounding effects are often under-represented in the risk analysis. * Many risk models are a black box to the project teams and the logic of how the experts’ inputs will be used leads to poor communication between expert and risk modeler. Results are often provided in a written report days or weeks after assessments from subject matter experts, rather than in a live group feedback, resulting in lost opportunities to iterate, enhance communications, and act on the analysis.
To address these common failure modes, best practices include the following items:

* Risk assessments should be facilitated in a group / team setting by an expert risk analyst who understand cognitive and behavioral biases and how to mitigate them. * Use historical data and neutral subject matter experts who are not on the project should provide unbiased input and critique. * Risk models should be aggregated to a higher level of granularity to capture dependencies, incorporate macro-level risk events, and avoid central bias that results in narrow outcomes. * Risk modeling needs to be transparent so experts understand directly how their inputs affect the risk profiles of the cost or schedule distributions. * Facilitation should include real-time analysis to provide timely feedback and opportunities to iterate on the analysis until team buy-in is achieved. * Cost risk needs to be integrated with schedule risk; one significant contributor to cost overruns is schedule delays, which are often ignored in standard cost contingency calculations. * Post analysis risk mitigation planning should be motivated by the major drivers of risk revealed in the analysis. What are the major cost risk drivers? What are the major choke points creating schedule delays? These items form the priority for risk mitigation and risk management strategies to deliver the project on time and on budget.
Comments and questions welcome.

Leave Comment

Your email address will not be published. Required fields are marked *