1 / 38

Understanding Technology Project Risks and Predicting Project Performance

This project aims to advance our understanding of IT project performance and how to influence it. It analyzes different project types, identifies project characteristics, and explores factors that contribute to project success or failure.

tschuler
Download Presentation

Understanding Technology Project Risks and Predicting Project Performance

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Understanding Technology Project Risks and Predicting Project Performance Dr. Andrew Gemino Simon Fraser University gemino@sfu.ca www.PMPerspectives.org IS SIG Meeting, April 8, 2008

  2. Agenda Introduction Part 1: IT Project Performance Part 2: Elements of Performance Discussion: How does your company evaluate performance Part 3: Predicting IT Project Performance Part 4: PM Practice and Performance IS SIG Meeting, April 8, 2008

  3. Motivation for My Research Application of Information Technology plays an important role in Canada’s capacity to innovate The Info. And Comm. Technology Sector Contributes $61 bil to Canadian GDP (in 1997 constant dollars) Comprises 5.8% of Canadian GDP Employs over 560,000 Canadians Responsible for 38% of all private sector R&D http://strategis.ic.gc.ca/epic/site/ict-tic.nsf/en/h_it06143e.html Our focus is to advance our ability to understand and influence IT project performance. IS SIG Meeting, April 8, 2008

  4. The Team Blaize Horner Reich, Simon Fraser University Chris Sauer, Oxford University, UK Andrew Gemino, Simon Fraser University Website: www.PMPerspectives.org Funding: Social Sciences and Humanities Research Council (SSHRC) Initiative for the New Economy (2003-2006) Research Grant funding (2007-2010) INE Communication Grant Natural Sciences and Engineering Research Council (NSERC) Research Grant Funding IS SIG Meeting, April 8, 2008

  5. Some Recent Output IS SIG Meeting, April 8, 2008 • Sauer, C., Gemino, A, and Reich, B.H. “Managing Projects for Success: The Impact of Size and Volatility on IT Project Performance”, Communications of the ACM, 60:11, Nov. 2007, pp. 79-84. • Gemino, A., Horner-Reich, B. and Sauer, C. "A Temporal Model for IT Project Management", Journal of MIS, Winter 2007–8, Vol. 24, No. 3, pp. 9–44 (awaiting publication). • Gemino, A., Reich, B. and Sauer, C. "Beyond Chaos: Examining IT Project Performance" Proceedings of the 2nd Annual International Research Workshop for the Special Interest Group for Information Technology Project Management (SIGITProjMgmt), Montreal, Canada, Dec. 8, 2007. • Reich, B.H., and Sauer, C. “Myths About Information Technology Project Performance”, Proceedings of the Administrative Sciences Association of Canada (ASAC) 2007, Ottawa, Canada, June 2-5, 2007. • Gemino, A., Reich, B.H. “Estimating Risk in Information Technology Projects”, Proceeding if the Americas Conference on Information Systems, Keystone, Colorado, Aug. 9-12, 2007. 5

  6. PART 1: IT Project Performance IS SIG Meeting, April 8, 2008

  7. The Standish Report The Chaos Report (1994) Many have heard the numbers On time, budget and scope – 16% Challenged 53% Abandoned 31% Average Performance Cost overruns 189% Time overruns 222% % of Original Specs 61% http://www.standishgroup.com/sample_research/chaos_1994_1.php IS SIG Meeting, April 8, 2008

  8. Standish Surveys Show Improvement (complied from press releases from www.standishgroup.com ) IS SIG Meeting, April 8, 2008

  9. Some Questions about Standish Figures • “Most such academic papers and guru reports cite the same source for their crisis concern—a study published by the Standish Group more than a decade ago” Robert Glass, CACM, 2006 • Reasons to doubt the Standish figures • Inherent bias due to perspective on “failure” • Tough test for “success” – all targets at 100% • Sampling method is very unclear • what projects are in the sample? IS SIG Meeting, April 8, 2008 9

  10. Study 1: Computer Weekly UK Web based survey of Readers and PM’s Asked IT project managers about their last project (completed or abandoned) 421 full responses Average 17 years industry experience Average 9 years as project manager Study 2: PMI Chapters in Ohio Web based survey 3 PMI chapters in Ohio Asked IT project managers about their last project (completed or abandoned) 194 full responses Average age, 43 Average years experience, 15 Average PM training: 34 days Survey Research IS SIG Meeting, April 8, 2008 10

  11. What we found IS SIG Meeting, April 8, 2008 • 65% of projects in the UK and 66% US samples had good performance • Delivered within a reasonab;le contingency (approx 7%) of ALL targets. • Compare with 63% challenged or abandoned in Standish • What’s the difference? • Experienced Project Managers • Data collection • We asked for actual variances from original goal • Cluster Analysis • Didn’t use hard line (99% is not challenged) • “let the data speak for themselves” 11

  12. IT Project Types: UK Study IS SIG Meeting, April 8, 2008

  13. UK - Project Characteristics IS SIG Meeting, April 8, 2008

  14. IT Project Types: US Study IS SIG Meeting, April 8, 2008

  15. US - Project Characteristics IS SIG Meeting, April 8, 2008

  16. Surprises:Some IT Projects Exceed Expectations IS SIG Meeting, April 8, 2008 • The IT Performance story is not all bad • 2/3rds of projects are performing well • Some IT projects exceed expectations • 7% of UK • 17% of US projects • These projects that exceed expectations are not mentioned in Standish Group Reports • We found them because we did not constrain “success” in the data collection 16

  17. IT Project Performance The Impact of Size and Volatility IS SIG Meeting, April 8, 2008

  18. Risk Associated with Project Size IS SIG Meeting, April 8, 2008

  19. Risk Associated with Project Size IS SIG Meeting, April 8, 2008

  20. Risk Associated with Project Size IS SIG Meeting, April 8, 2008

  21. Risk Associated with Volatility IS SIG Meeting, April 8, 2008

  22. Risk Associated with Volatility IS SIG Meeting, April 8, 2008

  23. Recommendations from UK Study IS SIG Meeting, April 8, 2008

  24. Recommendations from UK Study IS SIG Meeting, April 8, 2008

  25. Recommendations from UK Study IS SIG Meeting, April 8, 2008

  26. PART 2: Group Discussion Elements of Performance: Let’s hear from you – How does your company measure Project Performance? IS SIG Meeting, April 8, 2008

  27. Are Budget and Schedule variances good measures of IT Project Performance? http://www.codinghorror.com/blog/images/software_engineering_explained.gif IS SIG Meeting, April 8, 2008 27

  28. How should performance be measured? Two basic outcomes in project Process Outcomes On time On budget On specs Product Outcomes Value delivered Benefits Quality PM’s are often measured here But are expected to deliver here IS SIG Meeting, April 8, 2008

  29. Question Does your company consider both product and process when considering evaluating performance? Should this be done? And if so, how? IS SIG Meeting, April 8, 2008

  30. PART 3: Predicting IT Project Performance What factors determine IT Project Performance? IS SIG Meeting, April 8, 2008

  31. What Affects Performance Risk Factors and Resources Initial (A-priori) Factors Knowledge resources Team. PM, Sponsor, Clients Structural Factors Technical complexity, budget, duration, effort Emergent Factors Organizational Resources Top Management Support, User participation Volatility Governance changes, Target Changes, Environment changes IS SIG Meeting, April 8, 2008

  32. What Affects Performance Project Management Practice Expertise Coordination Who are knowledge leaders How can they be accessed Horizontal Communication Across team and clients PM Methods and Tools Traditional PM IS SIG Meeting, April 8, 2008

  33. US Study: Results Emergent Performance A-Priori IS SIG Meeting, April 8, 2008

  34. Results With this model we can explain Approximately 40% of the variance in Process performance Approximately 22% of the variance in Product performance We can also show the strength of the relationships between risk and resource factors associated with IT Projects. Strength and direction is given by the number associated with each line. IS SIG Meeting, April 8, 2008

  35. Results IS SIG Meeting, April 8, 2008 • Two Broad Forces Acting on Projects • Forces of Evil • Structural risk is strongly related to volatility and volatility is negatively related to process performance • Large projects often have a bumpy ride and are challenged in regards to being on time and budget. 35

  36. Results • 2. Forces of Good • Increased knowledge is related to higher organizational support and higher level of PM Practices. • Increased PM Practice is related to better process and PRODUCT performance IS SIG Meeting, April 8, 2008

  37. Summary IT Project performance is a critical issue in Canada’s ability to innovate. We have a good idea of the factors that affect process performance. More work needs to be done. It is important to consider initial factors as well as emergent factors when considering performance. We do not yet have a good understanding of how these factors affect product performance More work required IS SIG Meeting, April 8, 2008

  38. Questions? If you are interested in more information please visit: www.PMPerspectives.org IS SIG Meeting, April 8, 2008

More Related