Revealed causal mapping of IT/IS project risk

1. Revealed causal mapping of IT/IS project risk Robert T. Hughes, University of Brighton, UK

2. Overview of the talk • Locating project risk management in the research world • Metrics versus management information • Need for causal models • Challenge of quantification • Future directions

3. Locating risk management research Hirschheim and Smithson framework Objective/rational Efficiency zone Hardware/software monitor, Simulation, Code inspection, Software metrics, Quality assurance, TQM Effectiveness zone System usage, Cost benefit analysis, Critical success factors, Risk analysis, Resource utilisation, Economics, Management, Understanding zone Personal constructs, Context-content-process, Political analysis, Organisational behaviour Subjective/political

4. Software metrics – in my day… • Focus on engineering approach • Would like software to be physical product • Measurements tied to specific ‘physical’ entities • Focus on measurement practice • Often quite negative – demonstrating why measurements invalid • Dearth of positive achievement

5. Need for ‘management information’ • Management is concerned with allocation of resources • This allocation needs to be seen to be fair and just • Therefore needs to be linked to objective indicators of need, merit, productivity etc., etc. • This implies that measurement needs to be based on theories of causation

6. Revealed causal mapping • Strongly influenced by Kelly’s concept of personal constructs – sees human behaviour based on ‘model building’ about cause and effect • Each construct has a positive and negative pole • A network of constructs and the cause and effect relationships between them can be built • Note that the maps describe perceptions – hence often called ‘cognitive’ mapping

7. Fragment of a RCM + experience of developers product quality – remedial work – time pressure

8. Deadlines met… missed Unstable environment…unstable Experienced staff … inexperienced High salaries …low High productivity… low low staff turn-over… high Heavy management pressure…low Uncertain user requirements … certain High costs…low costs Requirements prototype…not

9. What RCMs can illustrate • ‘Tail’ constructs are those that have no prior cause within the scope of the map • Tails could be: • Environment e.g. ‘low staff turnover…high’ • Policy e.g. ‘requirements prototype…not’ • Constructs can be subject to both negative and positive influences – shows uncertainty • Between two constructs there can be both positive and negative relationships

10. Comparison with other mapping approaches – physical models • Systems dynamics • SD involves building a mathematical model which attempts to represents the real world system • Question of validation of each relationship identified in SD model • Very labour intensive • Root cause analysis • Analysis of the circumstances of a particular situation: actual events rather than situational factors

11. Comparison with other approaches - perceptual • Cognitive mapping – people’s perceptions: But how do you know they are telling the truth? >>> ‘Revealed’ causal mapping • Reasoning maps - the way people make decisions • Accuracy of predicting actual decisions? • Effectiveness of decisions? • Healthy people’s perceptions should be close to reality?

12. Reasoning maps: problem of indeterminate outcomes Need for some kind of quantification + experience of developers product quality – remedial work – time pressure

13. Approaches to ‘intermediate’ quantification • Fuzzy cognitive maps • Allow values to be set for tail nodes and edges • Can execute the model and study dynamic behaviour • The presentation of FCMs can be off-putting for non-mathematicians • Reasoning maps are a ‘user-friendly’ alternative

14. Using ordinal indicators • Allow us to model quantification without actual measurement • Allocate ordinal values to tail nodes • For example • Very strong, strong, medium, weak, very weak • Allocate value to the strength of the causal links • Very strong, strong, medium, weak, very weak

15. Propagating values • Methods can vary e.g. • Where there is a single cause, use: minimum (node_value, edge_value) medium strong medium

16. Where there is more than one causal link strong weak medium • Identify minima as before • Take the maximum result • This implies an independent ‘OR’ relationship between two causal factors • Other relationships possible e.g. compensatory factors – take the median medium strong

17. Our use of RCMs • As a teaching and learning tool – can be used in text analysis • Project risk management – used to study failed projects retrospectively • Asked participants to map causes of failure individually – large differences in perception • Follow-up by consensual map-building – group consensus appeared to be relatively easy • Differences in perceptions of managers and development staff • Attempt at building a ‘core model’ of project risk

18. Our use of RCMs • As a method of designing service management information systems • Identify the problem domain • Stakeholders collaboratively build the model • Identify measurements: • That can corroborate model • That can act as predictive and summative performance indicators • Examine what effects performance indicators might have

19. Future work: Generic project risk model Policy Contingency preparation Risk situation… Contingency action ✚ ✚ ✚ ✚ ✚ ▬ ✚ Policy Risk Avoidance… commitment Risk Exposure… Risk occurrence Damage.. benefit ✚ ▬ ✚ ✚ ✚ ▬ Policy Risk reduction ✚ Covers risk tactics of avoidance, reduction and mitigation

20. Future plans • Developing tools e.g. analogy seeking • ‘White fly experiment’ – risk model for student projects

21. Further details • Al-Shehab, A., Hughes R.T. and Winstanley G . (2006). CorMod: A causal mapping approach to identifying project development risk. European & Mediterranean Conference on Information Systems (EMCIS) 2006, July 6-7, Alicante, Spain. • Hughes R.T., Al-Shehab A., and Winstanley G. (2006). Obstacles to the modelling of the causes of project success and failure. In Dan Remenyi (Ed), Proceedings of the 5th European Conference on Research Methods in Business & Management, Trinity Colledge, Dublin, Ireland, 17-18 July, pp 179-186. • Al-Shehab A.,Hughes R.T. and Winstanley G. (2005). Modelling Risks in IS/IT Projects through Causal and Cognitive Mapping. Electronic Journal of Information Systems Evaluation (EJISE), Vol.8, No.1, pp 1-10, January 2005 • Hughes R.T., Al-Shehab A., and Winstanley G. (2005). The use of casual mapping in the design of management information systems. Proceedings of the 4th International Conference on Research Methodology for Business and Management Studies (ECRM-05), held at the University of Paris-Dauphine, France, 21-22 April 2005 • Al-Shehab A., Huhghes R.T. and Winstanley G. (2004). Using Causal Mapping Methods to Identify and Analyse Risk in Information System Projects as a Post-Evaluation Process. Proceedings of the 11th European Conference on Information Technology Evaluation (ECITE 2004) held at the Royal Netherlands Academy of Arts and Sciences, Amsterdam, 11-12 November 2004 • R.T.Hughes, A. Al Shehab, M.Eastwood. ‘The use of cognitive causal mapping as an aid to professional reflection’. CHI workshop on ‘The Reflective Practitioner’ Vienna, April 2004. See - http://www.cmis.brighton.ac.uk/research/cig/