Introduction to Design Research: a Methodological Background for Scientific Work

1. Introduction to Design Research: a Methodological Background for Scientific Work Elena Paslaru Bontas Semantic Web PhD Network Berlin Brandenburg 30.09.2005

2. Outline • Motivation • Types of research • Design Research Basics • Evaluation in Design Research • Conclusion

3. Motivation • Motivation for research: • pure research: enhance understanding of phenomena • instrumentalist research: a problem needs a solution • applied research: a solution needs application fields • Motivation for research methodology • (qualitatively) control research process • validate research results • compare research approaches • respect rules of good scientific practice

4. Research: A Definition • Research: • an activity that contributes to the understanding of a phenomenon [Kuhn, 1962; Lakatos, 1978] • phenomenon: a set of behaviors of some entity(ies) that is found interesting by a research community • understanding: knowledge that allows prediction of the behavior of some aspect of the phenomenon • activities considered appropriate to the production of understanding (knowledge) are the research methods and techniques of a research community • paradigmatic vs multi-paradigmatic communities (agreement on phenomena of interest and research methods)

5. Scientific Disciplines • Types of research [Simon, 1996]: • natural sciences: phenomena occurring in the world (nature or society) • design sciences ~ sciences of the artificial: • all or part of the phenomena may be created artificially • studies artificial objects or phenomena designed to meet certain goals • social sciences: structural level processes of a social system and its impact on social processes and social organization • behavioural sciences: the decision processes and communication strategies within and between organisms in a social system

6. phenomena design sciences Semantic Web (CS) activities [Owen,1997]

7. Design research basics • Process model • Artifact types: • result of the research work • Artifact structure • content of the research approach • Evaluation: • evaluation criteria • evaluation approach

8. Process model • a problem-solving paradigm: • seeks to create innovations that define the ideas, practices, technical capabilities, and products through which the analysis, design, implementation, and use of information systems can be effectively and efficiently accomplished [Tsichritzis 1997; Denning 1997]

9. Design research process knowledge flows + operation and goal knowledge circumscription process steps Awareness of problem Suggestion Development Evaluation Conclusion logical formalism abduction deduction [Takeda,1990]

10. Artifacts • are not exempt from natural laws or behavioral theories • artifacts rely on existing "kernel theories" that are applied, tested, modified, and extended through the experience, creativity, intuition, and problem solving capabilities of the researcher [Walls et al. 1992; Markus et al. 2002]

11. Design research outputs[March & Smith, 1995] • Constructs • conceptual vocabulary of a problem/solution domain • Methods • algorithms and practices to perform a specific task • Models • a set of propositions or statements expressing relationships among constructs • abstractions and representations • Instantiations • constitute the realization of constructs, models and methods in a working system • implemented and prototype systems • Better theories • artifact construction Thesis output

12. Design research outputs constructs better theories models emergent theory about embedded phenomena abstraction models methods constructs better theories abstraction knowledge as operational principles abstraction artifact as situated implementation instatiations methods constructs [Purao , 2002]

13. Examples • Open up a new area • Provide a unifying framework • Resolve a long-standing question • Thoroughly explore an area • Contradict existing knowledge • Experimentally validate a theory • Produce an ambitious system • Provide empirical data • Derive superior algorithms • Develop new methodology • Develop a new tool • Produce a negative result

14. Artifact structure • Structure of the artifact • the information space the artifact spans • basis for deducing all required information about the artifact • determines the configurational characteristics necessary to enable the evaluation of the artifact Content of the thesis

15. Evaluation criteria • Evaluation criteria • the dimensions of the information space which are relevant for determining the utility of the artifact • can differ on the purpose of the evaluation Test cases

16. Evaluation approach • Evaluation approach • the procedure how to practically test an artifact • defines all roles concerned with the assessment and the way of handling the evaluation • result is a decision whether or not the artifact meets the evaluation criteria based on the available information. Testing method

17. Evaluation approach (2) • Quantative evaluation: • originally developed in the natural sciences to study natural phenomena • approaches: • survey methods • laboratory experiments • formal methods (e.g. econometrics) • numerical methods (e.g. mathematical modeling)

18. Evaluation approach (3) • Qualitative evaluation: • developed in the social sciences to enable researchers to study social and cultural phenomena • approaches: • action research • case study research • ethnography • grounded theory • qualitative data sources: • observation and participant observation (fieldwork) • interviews and questionnaires • documents and texts • the researcher’s impressions and reactions

19. Constructs

20. Methods

21. Models

22. Instantiations

23. Conclusion Good research results require a careful design of the research methodology and considerable evaluation efforts

24. References • „DFG Rules of Good Scientific Practice“ available at www.dfg.de, last seen September 2005 • Tsichritzis, D. "The Dynamics of Innovation," Beyond Calculation: The Next Fifty Years of Computing, Copernicus, 1997, pp. 259-265 • Denning, P.J. "A New Social Contract for Research," Communications of the ACM (40:2), February 1997, pp. 132-134 • Simon, H.A. The Sciences of the Artificial, 3rd Edition, MIT Press, Cambridge, MA, 1996 • Markus, M.L., Majchrzak, A., and Gasser, L., "A Design Theory for Systems that Support Emergent Knowledge Processes," MIS Quarterly (26:3), September, 2002, pp. 179-212 • Walls, J.G., Widmeyer, G.R., and El Sawy, O.A. "Building an Information System Design Theory for Vigilant EIS," Information Systems Research (3:1), March 1992, pp. 36-59 • Kuhn, T.S. The Structure of Scientific Revolutions, 3rd Edition, University of Chicago Press, 1996 • March, S.T. and Smith, G. “Design and Natural Science Research on Information • Technology,” Decision Support Systems (15:4), December 1995, pp. 251-266 • Lakatos, I. „The Methodology of Scientific Research Programmes“, John Worral and Gregory Currie, Eds., Cambridge, Cambridge University Press, 1978 • Wikipedia available at www.wikipedia.org, last seen Semptember 2005 • Purao, S. “Design Research in the Technology of Information Systems: Truth or Dare.” GSU Department of CIS Working Paper. Atlanta, 2002

25. Danke für die AufmerksamkeitViel Erfolg für die Promotionpaslaru@inf.fu-berlin.de