Faking it: How accepted practice in project scoping and assessment has conned us

1. Faking it:How accepted practice in project scoping and assessment has conned us John Smyrk Visiting Fellow School of Management, Marketing & International Business ANU College of Business & Economics AIPM Conference Hobart 2007

2. Theme • Conventional practice ignores target outcomes in projects. • Two unsatisfactory consequences: • Criteria for gauging project success are flawed. • There is no reliable method for solving the scoping problem. • Both problems can be addressed with the author’s ITO model. John Smyrk: Australian National University

3. The conventional test for project success • A project: • Is successful if it meets its objectives. • Has three objectives: • To deliver outputs of agreed quality. • To do this on time. • To do this within an agreed budget. • Is therefore successful if outputs are delivered with agreed quality, on time and within budget. • This test is provably flawed. John Smyrk: Australian National University

4. Assessing a project • Ex ante • Assessment is called appraisal. • Used to inform the funding decision. • Documented in a business case. • Ex post • Assessment is called evaluation. • Used to gauge project success. • Documented in a closure report. John Smyrk: Australian National University

5. The ultimate test for project success • Would the funder have approved the project knowing in detail how it turned out? • If yes, the project is successful. • If no the project is unsuccessful. • A project’s “equation of worth”: • Worth = F(benefits, disbenefits, cost) • Benefits ~ magnitude & timing • Disbenefits ~ magnitude & timing • Costs ~ magnitude & timing John Smyrk: Australian National University

6. Ex ante: Target outcomes Expected undesirable outcomes Anticipated cost Anticipated timeframe Risk (a measure of the reliability of these parameters). Ex post: Actual desirable outcomes Actual undesirable outcomes Actual cost Actual timeframe Outputs delivered fit-for-purpose (binary). A project’s assessment parameters John Smyrk: Australian National University

7. The flaw in conventional evaluation • How would a funder view a project in which: • Outputs were delivered: • Fit-for-purpose. • Under budget. • Within an agreed timeframe. • No desirable outcomes were realised. • Undesirable outcomes were unacceptable. • Under the previous rule this project would be judged a failure—but conventional wisdom classifies it as successful. John Smyrk: Australian National University

8. Project scope • Under the conventional view, scope: • Has to be set. • Is about constraining the work in a project. • Determined by the project’s deliverables. • I propose the following scoping principle: • A project is scoped if and only if its outputs are defined. • A project’s outputs are defined when: • They are listed. • Their fitness-for-purpose features are set. John Smyrk: Australian National University

9. The scoping problem • A proposed project has a tentative list of outputs: A, B, C & D—but: • “Output E” has been proposed for addition to the existing scope. • “Output A” has been proposed for removal from the existing scope. • How are these issues to be resolved? • The conventional wisdom appears to say “Ask someone”! John Smyrk: Australian National University

10. The central role of target outcomes • The concept of target outcomes is central to: • Judging a project (ex ante & ex post). • Solving the scoping problem. • Target outcomes are: • Are the desired end-effects we seek from the project. • Effectively the same thing as benefits. • Intangible but measurable. John Smyrk: Australian National University

11. The IPO model of a project • The conventional view of a project is based on an IPO (Input-Process-Output) model: INPUTS PROJECT OUTPUTS Resources Work Products John Smyrk: Australian National University

12. Incorporating outcomes into a project ITO = “Input-Transform-Outcome”. OUTCOMES INPUTS PROJECT UTILISATION OR CONSUMPTION OUTPUTS John Smyrk: Australian National University

13. Stakeholding and the ITO model • The Project Owner is accountable (to the Funder) for the realisation of target outcomes. • In utilising outputs, the Project’s Customers “cause” outcomes to emerge. • The Project Manager is accountable to the Project Owner for delivery of project outputs. John Smyrk: Australian National University

14. Expressing outcomes & outputs • Outputs: • Deliverables are always “things” (artifacts). • Always expressed as nouns. • Can be guaranteed (because the work is controllable). • Outcomes: • Changes in measurable characteristics of the world. • Usually expressed as nouns —qualified by an “–ed” word. • Increased efficiency. • Improved access. • Cannot be guaranteed (because utilisation is not controllable). John Smyrk: Australian National University

15. Appraising a project • Target outcomes drive benefits. • Undesirable outcomes drive disbenefits. • Outputs drive work. • Work drives timeframes and costs. • Uncertainty about the above parameters drives risk. John Smyrk: Australian National University

16. Scoping a project with the ITO model • A statement of scope for a project: • Lists its outputs. • Lists its target outcomes. • Provides an overarching statement of objective. • Issues: • How are we to confirm that the list of outputs and the list of outcomes are internally consistent? • Can the ITO model be used analytically? John Smyrk: Australian National University

17. List of outputs List of target outcomes The structure of a statement of scope • Statement of objective: • Short • Begins with “To …” • Outcomes oriented. • Indicates intent John Smyrk: Australian National University

18. Scoping with the ITO model • The Customer Map is a model of Utilisation in the ITO model in which: • Columns are associated with target outcomes. • Rows are associated with outputs. • Entries identify those customers who utilise the outputs on the left to generate the outcomes at the top. • The CM quickly reveals “surplus” or “missing” outcomes John Smyrk: Australian National University

19. The customer map (CM). Names of outcomes go in here Names of linking customers (only) go in here Names of outputs go in here John Smyrk: Australian National University

20. Entries in the Customer Map • Each cell contains a list of the customers who utilise the output on the left to make an appreciable contribution to the outcomes above. • Three sorts of entry: • Null (empty). • One customer. • Multiple customers. John Smyrk: Australian National University

21. Significant patterns of entries • The one (and only one) customer appears in all cells—trivial model. • A row is empty: • Implications? • Qualifications? • A column is empty: • Implications? • Qualifications? John Smyrk: Australian National University

22. Conclusions • Conventional approaches to gauging success and project scoping are both flawed. • The ITO model points to approaches that can resolve both issues. John Smyrk: Australian National University