ICT-Action Design Research method

1. ICT-Action Design Research method Raimo Hälinen (2012) Lahti 14.12.2012

2. Agenda • 12.00 – 13.30 Action Design Research • 13.30 – 14.00 Cafee break • 14.00 – 15.00 Using Action Design Reseach method • Discussion how to apply ADR-method • 15.00 – 16.00 Comparison DSR, AR and ADR methods • how to generalize results using by deduction and induction • Summary and conclusion

3. Information systems design science research frameworks Source: Walls et al (1992), van Aken (2004), Carlsson (2006), Venable (2006)

4. Paradigmatic dimension “Considering the practical implications of this analysis, we further identify that the employment of AR in the conduct of DSR needs to be done with care, especially where there is the potential for significant risk to the client or other stakeholders.” Source: Iivari and Venable (2009)

5. Basic assumptions Source: Cole, Purao, Rossi and Sein (2005)

6. Similarities of AR and DSR Source: Järvinen P: (2005, 2007)

7. Action Design Reserch Problem Formulation Principle 1. Practice-Inspired research Principle 2. Theory-Ingrained Artifact Principle 3. Data-inspired research 3. Relfection and learning Principle 6. Guided Emergence 2. Building, Intervention, and Evaluation Principle 3. Reciprocal Shaping Principle 4. Mutually Influential Roles Principle 5. Authentic and Concurrent Evaluation 4. Formalization of Learning Principle 7. Generalized Outcomes

8. Tasks in the problem formulation • Identify and concepualize the research opportunity • Formulate initial research questions • Cast the problem as an instance of a class of problems • Indentify contributing theoretical bases and prior technology advances • Secure long-term organizational commitment • Set up roles and responsibilities Orlikowski and Iacono (2001) defined IT artifacts as the “bundles of material and cultural properties packaged in some socially recognizable form such as hardware and/or software”.

9. Problem Formulation • Principle 1. Practice-Inspired Research The principle emphasizes to consider field problems (real-world problems) Action design research method is applied to study intersection of technological and organizational domains. The researcher can investigate how technological solutions can be used to improve organizational processes and achieve better results. An action design Researchers should try to generate knowledge that can be applied at the class of problems. (The cases are examples of the recognized class of problem). The result from the research activity is problem-inspired.

10. Problem Formulation ... • Principle 2. Theory-Ingrained Artifact The principle emphasizes that created artifacts are based on theories. Gregor (2006) explored systems of statements that allow generalization and abstraction to be theories. Gregor’s theory of Type IV (explanation and prediction theories) or Type V (design theories) The applied theories can be universal laws of natural science or ADR-researcher can utilize the specific theory (e.g. TAM, IS-succes) . • Identify problem, 2. Identify potential solutions, 3. design guide. Ingrained artifact is subject to organizational practice and it provides basis for cycles of intervention, evaluation and further reshaping.

11. Building-intervention and evaluation Design principles Researcher(s) Contribution to the specific ensemble being designed Artifact ADR team Alpha version Practitioners Utility for the users Beta version End-users The Generic Schema for IT-dominant building, intervention and evaluation

12. Discover initial knowledge creation target Select or customize BIE form Execute BIE cycle(s) Assess need for additional cycles, repeat Design principles Researcher(s) Contribution to the specific ensemble being designed Implementation of Artifact ADR team Practitioners Implementation of an artifact can lead to modification of organizational processes and work flows. Utility for the users Alpha version Beta version End-users The Generic schema for Organization-Dominant BIE

13. Building-intervention-evaluation Researchers Design principles Contribution to the specific database being designed ARD team Database Practitioners Utility for the users Alpha version Beta versin End-users The Generic Schema for Data-Dominant BIE

14. Principle 3: Reciprocal Shaping The iterative process that are described is based on DeGrace and Stahl (1990) recursive cycles. Scrum process Scrum meeting 30 min. What did you do yesterday? What will you do today? What obstacles got in your way? The roles of Scrum project The product owner The Team The Scrum Master The basic concecpt The product backlog The Sprint Sprint planning Daily scrum meeting Sprint review and retrospective Source: scrumalliange.org, DeGrace P and Stahl L. (1990), Sutherland J. (2010)

15. Principle 4: Mutually Influential Roles The action design researcher’s role is to share the knowledge of theory and technological advances. The practitioner’s role is to consider practical hypotheses and knowlegde of organizational work practices. Researchers and practitioners’ role are complementary and complate each others. The clear assigment of these responsibilities is important that reflection and experience can be utilized during the research projects.

16. Principle 5: Authentic and Concurrent Evaluation The evaluation is essential activities in building, intervention and evaluation process. The evaluation is not a separate stage as it is e.g. March and Smith (1995) state-gate models or compared to Peffers et al.’s (2008) model. The evaluation is ongoing and continues from start to the end of the project. The style of evaluation is formative (Remenyi and Sherwood-Smith (1999). The summative evaluation is utilized for beta version evaluation. The controlled evaluation may be difficult to achieve, so it should be carried as part of natural controls where possible.

17. Reflection and Learning • Principle 6: Guided Emergence The designed artifact will reflect preliminary design and ongoing shaping by organizational use, perspective and participants. Kernel theories Kernel theories Meta-requiremens Meta-design Desing method Testable desing product Testable design process Source: Walls et al. (1992, 2004)

18. Formalization of Learning • The objective is to formalize the learning. • According to Van Aken (2004), the situated learning should develop general solution concepts for a class of field problems. • Principle 7: Generized Outcomes • Generalization of the problem instance • Generalization of the solution instance • derivation of design principles from the design research outcomes.

19. Tasks in the formalization of learning • Absract the learning into concepts for a class of real-world problems. • Share outcomes and assessment with practitioners • Articulate outcomes as design principles • Articulate learning in light of theories selected • Formalize results for dissemination Tsang and Williams (2012) Definition of statements: a statement is empiricalwhen and only when it cannot be ascertained to be true or false without experience or observation. a statement is theoretical when and only when it is a generalization that purports to predict and explain the phenomena to which it refers.

20. Mid-range theorizing in DR Evidence Theory development can be confirmed by leads to revision of Explanatory statement Kernel theories Effect Cause Might lead to Articact evaluation Mid-range theories Prescribtive statement Prescribed action Goal Design theories Is installed to lead to Kuechler and Vaishnavi (2008) pointed out to need for to create mid-range theories. The mid-range theories act as a bridge between kernel theories and design theories. Source: Kuechler and Vaishnavi (20008)

21. Levels of generalization from DR Four levels in conceptual process: Generalization of the problem instance Generalization of the solution instance Emerging design knowledge in the form of design principles Feedback to design theory Design Theory Level 4 Class of Problems Design Principles Class of Solutions Level 3 Level 1 Level 2 Design process Solution as an artifact in use Problem Rossi, Purao and Sein (2012)

22. Yin’s conception of generalization • Trying to generalize results of the research project can be characterized using by Yin’s process(1994, p.31). Theory Rival theory Level-2 inference Experimental findings Population characteristics Case study findings Level-1 inference Subjects Sample of research

23. A generalizability framework • Lee and Baskerville (2003) considered generalization problem and proposed the framework. EE = emprical to emprical, ET = empirical to theory TE = theory to empirical, TT = theoretical to theory Accroding to Järvinen P review this framework may be problematic?

24. Types of induction Theory Empirical level Same population Theoretical generalization Within-population generalization Different population Sample Cross-population generalization Inductive analogy Different context Contextual generalization Particular instance Different time Temporal generalization Statistical syllogism Source: modfied from Tsang and Williams (2012)

25. Definition of types of induction Source: adopted from Tsang and Williams (2012

26. Comparison of DR, AR and ADR Source: Henfidsson (2011)

27. Comparison of Quantitative and Qualitative research approaches Source: Modified from Thomas (2010)

28. Contingency theory of MIS Contingency variables Management information system Management information system Performance Firm Performance Strategy Structure Size Environment Technology Individual Task Design Management implementation investment Use Implementation Use Satisfaction Success Effectiveness Perception Financial Source: Weil and Olson (1987)

29. Case studies Source: Järvinen (2011)

30. Stages of case study Real-world situation A single case Source: modified from Benbasat et al. (1987), Eisenhardt K.M. (1989) Järvinen P. (2012)

31. Case studies ... • How and Why questions are important • The focus is contemporal • The purpose of case studies: • Description of phenomena • Grounded theory can be applied to explore phenomena • Explorative approach is used to validate, confirm or falsify developed theory or model based on collected data.

32. Requirements of Case studies • Conceptual requirements: • Construct validity • Internal validity • External validity • Reliability • Structural requirements: • Relevance for the audience • Voyage of discovery • Controversies • Include all necessary data but not too much • Formal structure and elaboration

33. User centred design activities Practitioners Users, managers and executices participates to define needed features. Plan the human centred process Specify user requirements Evaluate appication against user requirements Specify the context of use Information related work systems Evaluate application against organizational requirements Specify organizational requirements Demonstrate application against speficied features (Field experiment) Design and develop an application Specify main features of an application Testprocesses module test Incremental Acceptance test verification

34. Scientific rigour in action ressearch • Baskerville and Wood-Harper’sstrategies: • Establish an ethicalclient-systeminfrastructure • Design data gatheringprocesscarefully • Follow the plannediterativephases: • plan action • take action • evaluate action • Promotecollaborationby the subject and supporttheirsubjects’ learningcycles. • Write the reportthatdisseminate the scientificknowledge in a waythatit is possible to carry out futureresearch and bythiswayconfirmorrefuteanycausalsuggestionsorclaims of generilizedtheory.

35. Action design research studies • The following slides include examples, how ADR-method has been applied by other researchers. • Saarinen L. (2012) Dissertation in Aalto University • Rothengatter D. (2012) Dissertation • Modified action design reserch method: • Bilandzic and Venable (2011): Participatory Action Design Research method, a new method applied to urban informatics in Australia • Wieringa and Morali (2012) Technical Action Research as a Validation Method in ISDS. • Papas, O’Keefe and Seltsikas (2012) The Action research vs design science debate: reflection from an intervention in eGovernment. They point out that ADR-method is meta-method.

36. Saarinen’s ADR process Source: Saarinen (2012) Dissertation

37. Rothengatter’s ARD process Source: Rothengatter (2012) Dissertation

38. Rothengatter’s Meta-level process in ADR Problem formulation Rothengatter’smeta-level process in ADR included two building, intervention and evaluation states. First iteration stage focus is to develop version 1. Second iteration stage includes an improvement of version one after evaluation is carried out, and results are available. Reflection and learning include three analyses. Building, implementing and evaluation Literature review Donain analysis Initial design Implementation of initial design in case Building, intervention and evalluaution Evaluation Domain analysis Design updating based on evaluation Implementation of improved design Reflection and learning Evaluation of improved desing Analysis of initial design Rothengatter applied contingency theory to achieve IS performance and organizational performance of Information system design (ISD). Formalization of learning Analysis of evaluation of desing Update of underlying theories Analysis of improved desing Formalization of final design

39. Participatory Action Design Research Urban informatics Community, Urban dwellers Social good Well-being, healt, social connectedness Government, public institution Open, mobile, diverse Public, access for all Public Completely discretionary Essential Participative problem setting Ethnographic study Diagnosis and Problem formulation Action planning Opportunity identification Participative planning Action taking: design Participative design Prototyping and installation Impact evaluation Ethnographic study Participative evaluation Reflection and learning Participative client learning Design theorizing for UI Source: Bilandzic and Venable (2011)

40. Technical action reserch processes IT-artefact problem investigation Research problem investigation Client-system problem investigation IT-artefact design Research design Treatment design It-artefact design validation Research design validation Design validation It-artefact implementation Research execution Implementation in the Client-system Implementation evaluation Analysis of research results and publishing Implentation evaluation and applying results

41. Techical Action research cycles Engineering cycle Engineering cycle o o o o Idealizing assumptions Realistic assumptions Environ-ment IS design sccience Knowledge base Goals Improvement problem solving Knowledge question investigation Knowledge Artifact Framework for IS design science

42. The multiple realms of ADR Practical Realm Academic Realm Knowledge base and values Real-world practices and values Design Research Practice Reserch Practices Local Practices Contribution to general practice Contribution to knowledge base Purely practical contribution Practical and scientific contribution Practical advancement Purely scientific contribution No contribution Scientific advancement Source: modified from Sjöström and Donellan (2012

43. Reflective question to select research method Source: modified from Papas et al. (2012)

44. Development of design theories and knowledge A realist Information Systems intervention Identify real-world situations (P) and desired outcomes (O) O=f(p,i,c,m), where P = problem I = IS initiative M = Mechanism C = Context O = Outcomes Review (kernel theories) and previous research Propose/refine design theory Test design theory Source: Carlsson et al. (2007), Hrastinski et al. (2007)

45. Future research area in DSR Source: modified from Hevner et al. (2010)

46. References • Bilandzic, M., & Venable, J. (2011). Towards Participatory Action Design Research: Adapting Action Research and Design Science Research Methods for Urban Informatics. Journal of Community Informatics. Special Issue: Research in Action: Linking Communities and Universities, 7(3). • Carlsson, S.A. (2006): Towards an Information Systems Design Research Framework: A Critical Realist Perspective, Proceedings of the First International Conference on Design Science in Information Systems and Technology (DESRIST 2006), 192-212. • Henfridsson O. (2010) Action Design Research, presentation slides, University of Oslo • Hevner A and Chatterjee S. (2010) Design Research in Information Systems, Theory and Practice, Integrated Series in Information Systems 22, Springer. • Järvinen P. (2012) On baundaries between field experiment, action research and design research, University of Tamper, School of Information Sciences, Reports in Information Sciences 14, Tampere • Järvinen P. (2010) IS reviews, Department of Computer Sciences, University of Tampere, DE-2010-16, Tampere • Lee, A. S., and Baskerville, R. L. 2003. “Generalizing Generalizability in Information Systems Research,” Information Systems Research (14:3), pp. 221-243. • Rothengatter D. (2012) Engineering situational methods for professional service organization, An action design research approach, CTIT Ph D. Thesis Series No. 11-225, Enchede. • Saarinen L. (2012) Enhancing ICT Supported Distributed Learning through Action Design Research, Department of Information and Service Economy, Aalto University, Doctoral Dissertation. • Sein, Henfridsson, Purao, Rossi and Lindgren (2011), Action Design research, MIS Quarterly, Vol. 35, No. 1, pp. 37-56 • Sjörström and Donellan (2012) Design research practice: A product semantics interpretation, The International workshop on IT Artefact Design & Workpractice Intervention, 10 June 2012, Barcelona • Tsang E. W. K. and Williams J.N. (2012) Generalization and induction: Clarifications, and a classification of induction, MIS Quarterly, Vol. 36, No. 3, pp. 729-748. • Yin R.K, (2003 ) Case Study Resaarch, 3rd edition, Sage Publication.