SPPMG Event 18 th February 2010 Why (IT) Projects Fail

1. SPPMG Event 18th February 2010Why (IT) Projects Fail Malcolm Bronte-Stewart School of Computing University of the West of Scotland

2. Objectives This presentation will: • Discuss some of the costs and causes of technology project failure • Challenge the way in which project failure is judged • Suggest pre and post project evaluation techniques that may be useful to SMEs M B-S

3. Cobb’s Paradox “We know why projects fail, we know how to prevent their failure – so why do they still fail?” Martin Cobb, Treasury Board of Canada Secretariat in Unfinished Voyages, a follow to the CHAOS report, The Standish Group M B-S

4. Lyytinen and Robey (2000) “Organisations fail to learn from their experience in systems development because of limits of organisational intelligence, disincentives for learning, organisational designs and educational barriers. Not only have many organisations failed to learn they also have learned to fail. Over time they accept and expect poor performance while creating organisational myths that perpetuate short-term optimisation.” M B-S

5. Project Failure IT projects tend to go over budget and / or over time schedule and / or do not meet expectations • Research by the BCS, Royal Academy of Engineering, OASIG, Oxford University and others suggest that only about 15% to 30% are successful • Conservative estimates put the cost of IT project failure at tens of billions of Euros across the EU Jaques, 2004 (142 billion euros in 2004 McManus & Wood-Harper, 2008) and around $500 billion wasted on IT purchases that fail to reach their objectives worldwide Feld & Stoddard, 2004 • We go on making expensive mistakes M B-S

6. M B-S

7. M B-S

8. M B-S

9. M B-S

10. M B-S

11. M B-S

12. M B-S

13. M B-S

14. M B-S

15. M B-S

16. IT Development Project Costs • $55 billion wasted on cancelled and over-run IT projects in the USA in 2002 (Standish, March 2003) • “Public sector IT expenditure over past 12 months is in excess of £12.4 billion with a significant proportion at risk of being wasted” according to a House of Commons select committee (published July 2004) • “This year (2006) organisations and governments will spend and estimated $1 trillion on IT hardware, software and services worldwide”. (Charette, IEEE) M B-S

17. IT project success statistics • 75% of IT projects are challenged and 10% are abandoned (Oxford University, August 2003) • Only 7% of the 1000 firms Critical Research surveyed thought they were using IT effectively and 75% of these firms think that their IT systems are not providing a return on investment • Of 3,682 projects in 365 firms – 31% were cancelled, 53% had cost overruns and poor functionality and only 12% were on-time and budget (Johnson) • Three quarters of large systems may be considered failures (Laudon & Laudon) M B-S

18. For example - some Headlines • 2010 New HMRC inland revenue tax system producing wrong codes “ we’ve heard it all before”. • 2010 Labour accused of wasting 26bn on failed IT projects “stupendous incompetence”. • 2009 Rural payments scheme put out to grass “a display of scant regard for protecting public money”. • 2009 C-nomis offender management system “a master class in sloppy project management” • 2008 Edinburgh Fringe Box-office system “weak”, “fundamentally flawed” “insufficient planning, lack of risk management, inadequate communications and no authorised business case”. • 2005 Strathclyde Police Computer System “a complete disaster” • 2003 Libra the IT system for magistrates courts “One of the worst projects I have ever seen”. M B-S

19. “Prison IT system guilty of 'basic' project management failures” (2009) • The £234m C-Nomis IT system for Prisons failed in almost every possible way . • The NAO concluded that the technical complexity had been “significantly underestimated”. C-Nomis was treated as an IT project and not a business-change programme, project management was poor, and contracts with suppliers were weak.  • Edward Leigh, chairman of the Public Accounts Committee, said of C-Nomis that “kindergarten mistakes” had been repeated: “This Committee hears of troubled government projects all too frequently. But the litany of failings in this case are in a class of their own. All of this mess could have been avoided if good practice in project management had been followed.” M B-S

20. IT project failures • The Defence Information Infrastructure project to incorporate 150,000 terminals for 300,000 users at over 2000 defence sites is 18 months late and running more than £180m over budget. • The official cost of the controversial national identity system has soared to over £5bn. • The NHS IT scheme [NPfIT] was initially estimated to cost around £2.3bn. The most recent estimate is £12.7bn. M B-S

21. IT Project Failure • Sainsbury’s, the supermarket giant, installed an automated fulfilment system to increase efficiency and streamline operations in 2003. • The system ran into "horrendous" barcode-reading errors. Yet in 2005 the company claimed the system was operating as intended. • Two years later, the entire project was scrapped, and Sainsbury's wrote off £150 million* in IT costs. * (One report claims it was a $526m investment) M B-S

22. Avon County Council installed a new computer program to pay staff wages. • The spree started in a small way paying a caretaker £75 an hour. • Then it didn’t pay canteen workers at all for 7 weeks. • Next it paid a janitor £2,600 for a week’s work. • A deputy headmistress received her year’s annual salary once a month. • Heads of department earned less than their assistants. • Some people had more tax deducted in a week than they earned all year. • By February 280 council employees were out on strike. (Stephen Pile) M B-S

23. The misguided torpedo assumption • To prevent the possibility of a torpedo returning and exploding against the ship it was fired from • a safety system built so that it would self-destruct if it turned 180 degrees • Unfortunately, during trials, a torpedo jammed in its tube on board the ship, the test was abandoned and the ship turned round to go home …. BANG! M B-S

24. London Ambulance System New Software “Put Lives At Risk!” • Main Findings of investigation • Inexperienced procurement team • Staff mistrust and opposition • Over-ambitious timetable • Price put before quality • Incomplete and untested software • Andersen Consulting report suppressed • Management failed to identify and solve problems • Users “did things wrong” “...a faulty system implemented with undue haste, by a management determined to impose its will...” M B-S

25. IT Project Research • Jones, in a study of 250 large software projects at or above 10,000 function points* between 1995 and 2004 found that about 25 were successful, 50 had minor delays and 175 had major delays and overruns or had been terminated without completion. • He notes that the main problems were not technical but were due to aspects of poor project management. *(equivalent to around1,250,000 statements in the C programming language) M B-S

26. Wider effects of IT project failure • Perceptions of poor success rates and wasted resources affect decision making • The more IT projects are seen to go wrong the more: • the public become cynical • staff learn to expect problems and delays • Developers wonder if a lot of their work is likely to be wasted effort • business people become nervous of technology change • those holding the purse strings may view IT as a worry and a poor return on investment M B-S

27. What are the main causes of failure? • 6 major research studies reach remarkably similar conclusions about the significant causes that threaten project success: • OASIG • Standish (CHAOS) • Select Committee • OGC / NAO • Schmidt, Lyytinen, Keil & Cule, • Fortune and White M B-S

28. OASIG Report (Clegg et al,1996) • Outcomes from IT investment : • 80% to 90% do not meet goals • 80% delivered late and over budget • 40% fail or are abandoned • Under 40% address training and skills enough • Less than 25% integrate business and technological objectives properly • Only 10% - 20% meet all success criteria M B-S

29. IT Project Failure (OASIG) • Main reasons why IT projects fail • Management agenda is too limited • most IT investments are technology led • main investment motive is to only cut costs • This narrow focus on technical capabilities and efficiency goals means that inadequate attention is given to the human and organisational issues that can determine a project’s ultimate success. • Users don’t influence development enough • Senior managers don’t understand the links between technical and organisational change • Project management techniques and IT approaches are too technical • Companies fail to organise work or design jobs/roles properly M B-S

30. The Standish Group (1995) Of the $250 billion spent each year on IT application development • 31.1% of projects cancelled before completion (failed). • 52.7% of projects were challenged: over-budget, over the time estimate, and offers fewer features and functions than originally specified. • 16.2% of software projects were successful - completed on time, on-budget and to specification, (but some of the largest of which have only approximately 42% of the originally-proposed features and functions). • Recently these results have improved somewhat but remain disappointing. M B-S

31. Standish Group CHAOS Survey Results M B-S

32. M B-S

33. CHAOS Project Success Factors • User involvement • Executive management support • Clear and firm statement of requirements • Proper planning and formal methodology • Realistic expectations • Minimised scope and smaller project milestones • Competent, skilled staff • Ownership • Clear vision and business objectives • Hard-working, focussed staff • Experienced project managers • Reliable estimates M B-S

34. Select Committee on Public Accounts Report on Improving the delivery of Government IT projects (Study of 25 projects 1990 to 2000) • “Implementing IT systems has proved difficult” • Frequent cases of delay, confusion and inconvenience for the citizen and poor value for money for the tax payer. M B-S

35. Select Committee Report conclusions • Decisions about IT are Business not technical, they have profound effects on customer service and must involve senior management. • End users and their business needs must be identified before the project commences so that clear objectives are taken into account fully during design and development • Scale and complexity – how ambitious? Can it be undertaken in one go? • Skilled Project Managers are essential • Sound methodologies and well conceived risk management are called for • Need for a high degree of professionalism in the definition, negotiation and management of IT contracts • Training can take up considerable resources but must address the needs of the users and of those maintaining the systems if the anticipated benefits are to be realised • Contingency plans should be in place • Organisations should learn lessons from project undertaken. A post implementation review should establish the extent to which they secured the proposed business benefits, user expectations and technical requirements. M B-S

36. OGC and NAO Best Practice 2005Common causes of project failure • 1. Lack of clear links between the project and the organisation's key strategic priorities, including agreed measures of success. • 2. Lack of clear senior management and Ministerial ownership and leadership. • 3. Lack of effective engagement with stakeholders. • 4. Lack of skills and proven approach to project management and risk management. • 5. Too little attention to breaking development and implementation into manageable steps. • 6. Evaluation of proposals driven by initial price rather than long-term value for • money (especially securing delivery of business benefits). • 7. Lack of understanding of, and contact with the supply industry at senior levels in the organisation. • 8. Lack of effective project team integration between clients, the supplier team and the supply chain. M B-S

37. Keil, Cule, Lyytinen & Schmidt (1998) These researchers recruited 3 panels of experienced project managers in different places – Finland, Hong Kong & U.S.A. – and asked them to identify and rank specific risk factors. • Lack of top management commitment to the project • Failure to gain user commitment • Misunderstanding the requirements • Lack of adequate user involvement • Failure to manage end user expectations • Changing scope / objections • Lack of required knowledge / skills in project personnel • Lack of frozen requirements • Introduction of new technology • Conflict between user departments M B-S

38. Fortune and White (2006) Fortune and White reviewed 63 publications that focus on project Critical Success Factors • The top ten (in order of count of citations) of the 27 they quote are: • Support from senior management • Clear realistic objectives • Strong / detailed plan kept up to date • Good communications / feed back • User / client involvement • Skilled / suitably qualified / sufficient staff / team • Effective change management • Competent project manager • Strong business case / sound basis for project • Sufficient / well allocated resources M B-S

39. M B-S

40. Are IT Projects Different? • IT and software-based projects have certain characteristics which make them different • specification • requirements creep • invisibility • complexity • flexibility • estimation • training / churn • changing technologies • communication and conflict • resistance to change • testing M B-S

41. Risk Analysis • Consideration of these recognised common causes of project failure, taken with reports, case studies, project management experience and previous research, build a picture of the variety and scope of important risk factors. • Instead of expecting individuals to guess the likely risks for each project we can specify a standard, comprehensive collection of a variety of aspects and issues that threaten successful IT project outcomes. M B-S

42. Risk Analysis Technique (This Environment) A technique has been developed that is intended to help project managers and others to gauge and take account of these risk factors. It requires little training and gives a structured list of relevant questions that invite the user to consider the extent to which each element threatens the project under review . These can be separated into 3 (or 4) groups: Some concern the firm itself and are slow to change Some concern the project in question Some concern the details of the options being assessed and compared To create a series of Risk Tables This Organisation This Project This Option M B-S

43. M B-S

44. M B-S

45. M B-S

46. Risk or Threat Level • Each factor may be awarded its own weighting (or ignored) according to its perceived significance • Multiplying the weights by the scores gives a total value for each factor and these can be summed to arrive at the overall value • Comparing your risk analysis values with those of others provides a focus and common language to bring out concerns, specific issues, enlightened evaluation and assessment of likely dangers and potential problems • The technique may become more valuable the more it is used as experience of the results and the model’s inadequacies develops M B-S

47. Other issues There are at least two other aspects that might usefully be taken into account: • The wider environment, trends and market conditions, ie the level above the first table that is outside the control of the organisation in question but influences project choice • The degree of uncertainty in the estimates of risk factors So these tables provide an IT project risk analysis ready-reckoner in a standardised and quantifiable form that can be applied before the project begins, or during a feasibility study But how do we judge the finished project? M B-S

48. Measuring Success • Three factors are used to decide if a project is a success or a failure soon after its closure: • Time / on schedule? • Money / within budget? • Delivered product / to specification? • Fail any one of these tests and it may be labelled a failed project (even though it passes the other two) } Yes or No M B-S

49. Quality / Scope / Functionality Time Cost Three factors – “The Iron Triangle” Good? Did the project come in on or under budget and schedule? AND did the project meet customer requirements? Cheap? Quick? M B-S

50. Failures? What about the extent and type of the failure? • Some are abandoned, some are incomplete but in use • Some just overrun budget or time, others are total disasters • The project management process may be a mess AND / OR • The delivered product may be useless or unwanted • How do we differentiate and categorise the relative success / failure of IT projects? M B-S