Alvarez & Marsal: 3 disciplines for managing IT projects
How can you tell whether an IT project will be a boon or a bomb? And how can you spot the land mines in time to change course? Three disciplines can help, explains Ron Bisaccia from Alvarez & Marsal.
As you read this, danger lurks in IT project portfolios around the world—perhaps even yours. Don’t think so? Neither did Bridgestone, Kmart, Hawaiian Telcom, JCPenney or Avon. At least, they didn't think so before their failed IT projects brought down the CIO, the company or both.
We’ve all heard these IT project horror stories. Huddled around the glow of the conference room projector, we shake our heads at the lack of foresight (even hubris) of those who went before us. “How could they not see it coming?” we wonder. How, indeed? After 20 years of being called in to rescue very large and very troubled IT projects, I can assure you: The overwhelming majority do not see it coming, despite excruciatingly detailed projections and planning. How can you tell whether an IT project will be a boon or a bomb? More importantly, how can you spot the land mines in time to change course? The following three disciplines can help.
1. Measure effort, not cost
Let’s start with understanding just how much risk your project poses. Over the past decade, most experts have equated the level of project risk in proportion to its size. But what determines a project’s size and how it affects the outcome has never been clear. Is it budget, schedule, team size (full-time equivalents [FTEs]) or effort (person-months)?
Budget (or cost) is a popular yardstick. Conventional wisdom says that projects above $10 million to $15 million have a much lower success rate than those with smaller budgets. Should we then focus risk mitigation on projects greater than $15 million and breathe easy about the rest? No.
As a measure of risk, budget is too easily skewed. For example, does the project involve expensive yet simple-to-implement hardware or software? Is the work performed onshore or offshore? Are the costs of internal FTEs included in the budget? The answers to these questions can have a dramatic impact on your ability to rely on budget to gauge risk.
Instead, our experience is much more aligned with a 2007 study that examined the relationship of all four of the size dimensions to the risk of project failure. It concluded that cost is a poor indicator of risk. The best indicator? Effort level. In fact, the study found that any project estimated to exceed 2,400 person-months of effort level is virtually guaranteed to blow its budget and schedule.
2. Know your risk tipping point
Perhaps more relevant, project risk does not change linearly. Instead, it follows a fairly pedestrian arc in relation to the effort level until it reaches a certain threshold—a tipping point. Beyond that, the risk of project failure screams into the stratosphere.
Projects that fall below the tipping point have a much lower risk of failure and can usually be estimated using traditional techniques. However, projects above the tipping point have a much higher risk of failure and therefore require a risk-adjusted approach to project estimation.
Where is this tipping point? That is the hard part, because it varies based on the organization. However, its general position can be found with these three dimensions:
- Estimated work: 2,000 person-months
- Schedule: more than 18 months
- Team size: more than 20 FTEs
On the surface, these dimensions appear to measure the overall size of a project. Yet size is really a proxy for the true driver of risk: complexity. Executing a complex project within the tight schedule and budget constraints of most corporate environments is fraught with peril. (And despite popular belief, methodologies such as agile versus waterfall have no appreciable effect on large project success rates).
The first risk of complexity stems from ambiguous design and specification. The more complex the project, the more design effort is needed to accurately identify and estimate scope. Unfortunately, the pace of business demands a “figure-out-the-difficult-parts-later” approach to estimation, which virtually ensures that you’ll bake some special surprises into your project.
Surprises can include not truly understanding the business or functional requirements until the project is well under way; or relying on the collaboration of software modules that end up not playing nicely together; or underestimating the difficulty of merging and migrating legacy data.
Next, traditional estimation methods assume a stable environment, but corporate environments are anything but. Too many potential changes are outside your control, such as strategic realignments and/or key personnel losses that can dramatically affect resource availability and slow your project to a crawl. Also, external changes such as customer need shifts or competitive market entries can alter your target space, and therefore the functionality and scope of your initiative.
Finally, complex projects take time to complete—usually more than 18 months. During that time, projects are subject to the so-called “unknown unknowns”: the types of events that are extremely difficult to anticipate, yet are entirely possible during long projects. We’ve seen projects suffer delays due to hurricanes, tornadoes and blackouts. However, the original project plans did not allow for any of those events.
3. Simulate your risks
Traditional estimation begins with brainstorming the sequence of tasks required to complete a project. Unfortunately, it also implicitly assumes that “unknowns” will not occur, and that, as we’ve seen, is a bad bet to make.
How then do we estimate projects on the high side of the tipping point? By using a different approach—particularly one that assesses risk in terms of probabilities. In our work, we’ve found Monte Carlo simulations (algorithms that rely on repeated random sampling to obtain numerical results) to be very effective at both compensating for the unpredictable and accounting for complexity.
The simulations are simple to perform. Your project team develops best-case, worst-case and most-likely duration estimates for each task. Then, using any number of inexpensive Monte Carlo software packages, you simulate the project multiple times. (We’ve found that doing 5,000 simulations works well.)
The primary result is a readout of possible project outcomes from which you can choose the duration that best fits your risk tolerance. Then plan your project accordingly. Also important is a sensitivity analysis report that details which tasks are the most risky. This allows you to mitigate the risks of each task before you take action.
Several years ago, American Express used this approach before an overhaul of its online membership rewards presence. The financial services giant wanted to completely rework not only the look and feel of its site, but also the underlying functionality available to its consumers.
The project would demand decommissioning existing systems, while others would need to be revised and new systems added. Initially, size estimates focused purely on scheduling, which determined the project would take approximately 33 months to complete.
Fortunately, company leaders recognized that earlier projects this big had underperformed in terms of schedule and budget. So they tried a new approach to forecasting and structuring large projects: Monte Carlo.
The simulation predicted that the project was more likely to consume 45 months—a full year more than expected. Armed with this knowledge, American Express used a series of risk-mitigation steps and completed the project on time and on budget.
Risk is inevitable, especially when it comes to large IT projects. Yet it's far better to know just how much and what type of risks you’re dealing with before it’s too late. Therefore, insist on estimates that measure effort level (e.g., person-months) so you’ll know on which side of the risk tipping point the project falls. If it’s on the high side, use a risk-simulation approach such as Monte Carlo to understand the true schedule risks and restructure your project plan accordingly.
Follow this simple yet disciplined approach, and you can avoid costly surprises and be much more confident that your project will be completed on or very near schedule and budget. Even more important, you can rewrite what might have been an IT project horror story into one with a happy ending.
An article from Ron Bisaccia, Managing Director at Alvarez & Marsal, a global professional services firm that delivers performance improvement, turnaround management and business advisory services.
