Glen L. Gray, California State University, Northridge, USA

1. EAA 2011: Rome, Italy Improving System Development ProjectSuccess: How InternalAuditors Add Value ThroughProcess Involvement &Measurement Glen L. Gray, California State University, Northridge, USA Anna H. Gold, VU University, The Netherlands Christopher G. Jones, California State University, Northridge, USA David W. Miller, California State University, Northridge, USA

2. Overview • Background • SDP failures and the dismal rate of SDP success • Control issues • Research objective • Internal auditor’s role in SDP success • Research questions, methods, and summary of findings

3. Many SDP failures… • December 2002: McDonald’s abandons major project after two years. Cost: US$170 million • November 2004: Sainsbury (UK super-market chain) writes off a £260 millionIT investment in its supply chain • February 2008: Los Angeles Unified School District’s faulty US$95 million payroll system goes live. For months afterward, thousands are overpaid, underpaid, or not paid at all. • November 2010: FBI spent $405 million of the $451 million budgeted for new Sentinel case-management system, but, as of September, it’s two years behind schedule and $100 million over budget

4. Few SDP Successes… Standish Group [2009]

5. Costly Conundrum • How do failing or challenged projects go undetected? • Where were the ‘red flags’? • Missed, dismissed, or ignored all together? • Who’s responsible for monitoring the controls and raising these red flags?

6. Research Objective • To explore how internal auditorscurrently do and potentially can provide value-added support to proactively help identify and monitor system development project controls to either: • Help get these projects back on track toward success or • Stop projects when the investment in the projects is still relatively low

7. Post-SOX Changes? • Pre-SOX: internal auditors usually came into a system development project after the project was completed to evaluate the internal controls—bayoneting the wounded • Post SOX: internal auditors are more frequently active members of major system development projects, but— • auditor focuses on controls for the specific processes being automated, not the system development controls Gray [2004, 2007]

8. Research Questions RQ1: When and how should internal auditors become involved in SDPs? RQ2: For which factors critical to system success can internal auditors add the most value? RQ3: What metrics should be used to monitor SDPs?

9. Mixed-mode Research Method • Review IS and internal auditing literature • CSFs and CFFs • Conduct internal auditor focus groups exploring RQ1 – RQ3. • Qualitative • Develop CSF taxonomy from an internal auditing perspective • Qualitative • Survey a sample of The IIA membership • Quantitative

10. Critical Success Factors • Literately, hundreds of success/failure factors • However, many different ways to say same things • From both professional and academic literature • Mostly opinions/observations vs. rigors analysis • Mostly not stated as measurable factor/metric(e.g., adequate user involvement) • Our next task: reduce factors to manageable set.

11. Critical Success Factor Taxonomy

12. Critical Success Factors Project Management • Systems Development Methodology • Quality Assurance • Change Management • Monitoring SDP Process • Financial Management • Tools and Infrastructure • Agile Optimization Project • System Requirements • Systems Interoperability People • Executive Support • Project Personnel • Project Management Expertise • Conflict Management Organization • User Involvement • Business Alignment Externalities • Vendor Relationship Management

13. Summary of Findings (1) RQ 1 Internal Auditor’s Role • Waiting until post-implementation review is too late. • Greenberg & Murphy, 1989

14. Summary of Findings (2) RQ 1 Internal Auditor’s Role • It’s OK to invite yourself to the party.

15. Summary of Findings (3) RQ 2 Where Internal Auditors Add Value • Some CSFs more critical than others. • Criticality transforms.

16. Summary of Findings (4) RQ 3 Monitoring SDP Success • Metrics abound but dashboards uncommon. • Conventional wisdom evolving.

17. Internal Auditor Involvement • Three basic approaches to the auditor’s involvement in SDPs: • Auditor approachwould be the more traditional auditing function by monitoring the SDP on a milestone basis to monitor how the project is progressing on behalf of management and the board. • Consultant approachwhere the internal auditors are advising the SDP team on an as-needed basis regarding controls. • Embedded approachwhere internal auditors are integrated in the SDP team functioning as the control experts.

18. Embedded Consultant Auditor Internal Auditor Involvement [Large] Internal AuditDepartmentSize [Small] • [Audit] IT Skill Portfolio [IT]

19. The Final Survey Question Q: What is the one best way for internal auditors to improve the success rate of SDPs? A:“Be included, be involved, and participate regularly in the process from project inception.”

20. Questions? Thank You! Grazie Mille! Glen L. Gray [glen.gray@csun.edu] Anna H. Gold [a.h.gold@vu.nl] Christopher G. Jones [christopher.jones@csun.edu] David W. Miller [david.w.miller@csun.edu]