430 likes | 777 Views
Educational Design Research: appropriate to address problems in educational practice. Tjeerd Plomp Professor Emeritus University of Twente Enschede, The Netherlands.
E N D
Educational Design Research:appropriate to address problemsin educational practice Tjeerd Plomp Professor Emeritus University of Twente Enschede, The Netherlands ICREAMS
Van den Akker (1999; director of Netherlands Institute for Curriculum Development) about research in domain of curriculum development and implementation: many ‘traditional’ research approaches such as experiments, surveys, correlational analyses, with their emphasis on description hardly provide prescriptions that are useful for design and development problems in education ICREAMS
In this presentation: Research functions and research designs What is design research Research question Quality criteria for interventions Generalizability in design research New book on design research ICREAMS
Focus of research project Dependent on research question/goal research may address a need to describe to explain, understand, predict to compare to evaluate to design/develop Called: research functions ICREAMS
Research approaches/designsto realize research functions e.g.: Survey: to describe, compare, evaluate Case studies: to describe, to explain Experiments: to explain, to compare Ethnography: to describe, to understand Correlational research: e.g. to describe, to explain Evaluation research: e.g. to determine the effectiveness of a program ICREAMS
Research designs, e.g.: Design research: to design/develop an intervention (e.g. program, T/L strategy, process, product, system, etc) with the purpose to improve practice andcontribute to body of knowledge OR to develop a theory (and possibly: to inform decision making and policy development) Note:intervention is container term ICREAMS
What is Educational Design Research (EDR)? Distinction between: Development studies aimed at research-based solutions for complex problems in educational practice Validation studies aimed at development or validation of a theory Research on interventions versus research through interventions (McKenney & Reeves, 2012) Often both angles in one study! ICREAMS
What is Educational Design Research (EDR)? Educational design research – type development studies- is the systematic analysis, design, development and evaluation of educational interventions with the dual aim of generating research-based solutions for complex problems in educational practice, and advancing our knowledge about the characteristics of these interventions and the process of designing and developing them. ICREAMS
What is Educational Design Research (EDR)? Educational design research – type validation studies - is the study of educational interventions – such as learning processes, learning environments and the like – with the purpose to develop and validate theories about such processes and how these can be designed. ICREAMS
Educational Design Research encompasses systematic educational design processes. The reverse is not true: Not all systematic educational design can be called research ICREAMS
Iterations of systematic design cycles PROBLEM design & develop prototype analysis evaluation Revision: yes? No? STOP
ID of problems with practitioners Development of prototype solutions Design research methodology can be expressed as follows: Revision Testing solutions in context Reflection to produce design principles (adapted from Reeves, UoGeorgia, Athens, USA) ICREAMS
Susan McKenney, Univ of Twente: ICREAMS
Phases in Design Research Preliminary phase: needs and content analysis, review of state-of-art of literature => conceptualisation & design specifications Development or Prototyping phase (iterative design phase + formative evaluation) <=> micro-cycles of research Assessment phase (semi-summative evaluation) ALL Phases: systematic reflection and documentation (resulting in design principles) ICREAMS
On terminology: (Educational) Design Research stands for a ‘family’ of related research approaches Design studies, design experiments Design-based (implementation) research Development research Participative action research Formative research Engineering research (e.g. Van den Akker et al., 2006): But a number of characteristics in common! ICREAMS
Common characteristics Interventionist Iterative Process oriented Utility oriented: merit of design measured Theory oriented: based upon & contributing to Involvement of practitioners ICREAMS
Context Z: A closer look: Intervention X Input => Process Outcomes Y1, Y2, …, Yn In general: Research goal may vary, but always: (i) design/development of intervention (ii) design principles or theory validation ICREAMS
Research question in design research If research goal is development of intervention: What are the characteristics of an <intervention X> to realize outcomes Y in context Z OR in case of developing/validating a theory (tentatively): What is an adequate learning and teaching strategy for <topic B> in context C in order to cope with the main difficulties in the learning of this topic ICREAMS
Examples of research questions #1 what are the characteristics of an effective in-service programme for mathematics teachers through which they develop the ability to apply student-centred pedagogical methods (Tecle, 2003) ICREAMS
Examples of research questions #2 What is an adequate learning and teaching strategy for genetics in upper secondary biology education in order to cope with the main difficulties in learning and teaching genetics, and to promote the acquisition of a meaningful and coherent understanding of hereditary phenomena? ( Knippels, 2002) ICREAMS
Quality criteriafor interventions: • Relevance or Content Validity: • need for the intervention • based on contemporary scientific insights 2. Consistency or Construct Validity 3. Practicality - can be used in settings for which developed 4. Effectiveness: - yields desired results - cost-benefit ratio #3&4: Expected versus Actual ICREAMS
Shift in emphasis on criteria ICREAMS
Formative evaluation e.g.: ICREAMS
Yin (2003) about case study research: striving to generalize findings to some broader theory: Generalizability in Design Research: Context Z: Outcomes Y1, Y2, …, Yn Intervention X Input => Process design principles as intervention theory local instruction theory
Generalizability in Design Research: Yin (2003) about case study research: striving to generalize findings to some broader theory A theory (design principles or local theories) must be tested through replications of findings in a 2nd, 3rd or more cases: if same results in various cases, then …. the theory (= design principles or local theories) might be accepted for a much larger number of contexts. This replication logic is the same that underlies design research! Analytical generalization. (compare with ‘transferability’ in qualitative research) ICREAMS
Editedby Tjeerd Plomp & Nienke NieveenPublishedby: SLO – Netherlands Institutefor Curriculum Development. Enschede; 2013Part A: anIntroductionPart B: Illustrative casesFree available from:http://international.slo.nl/edr ICREAMS
Part B: 51 Cases of Design Research Case Selection Tool: Countries Educational Sectors Educational domains (>1) Main aim Educational focus Type of intervention (>1) See: http://international.slo.nl/edr ICREAMS
Part B: 51 Cases of Design Research Case Selection Tool: 22 countries: CHI, SIN, IND, KOR – USA (14), NET (11) Educational Sectors: all Educ domains (>1): curriculum (16), learning & instruction (29), subject-related pedagogy (34), school organization (3), instructional technology (7), ICT in education (11) Main aim: development studies (36); validation studies (14); implemtation studies (1) Educational focus Type of intervention (>1) ICREAMS
Part B: 51 Cases of Design Research Case Selection Tool: Countries Educational Sectors Educ domains (>1) Main aim Educational focus: teaching-learning methods; curriculum planning; ICT in education; school management & leadership; monitoring quality of educ; professional development; workplace learning Type of intervention:curr unit/course (34), educprogramme (12), learning task (9), monitoring system (4), assessment task (2), other (4) ICREAMS
Thank you!(t.plomp@utwente.nl) ICREAMS
Quotes illustrate that many researchers in various ‘corners’ of our field belief that there is a need for an alternative research approach that is directly relevant for problems in educational practice. ICREAMS
NOTE: One resq =>more than oneres function Example: if aim of research is to design and develop a teaching-learning strategy for acquiring the competency of mathematical modelling (in grade 11 & 12), then researchers may first want to understand and carefully describe what barriers students experience with mathematical modelling. But: there is primary research function ICREAMS
Why embark on Design Research? Various reasons – e.g.: Need to solve a complex problem in education practice for which no how-to-do guidelines are available Need to elaborate and/or validate a theory (e.g. whether the theory of Realistic Mathematics Education is valid in a certain context) ICREAMS
Problems may exist at all levels - for example: System level: Need for system for e-learning to serve specific group of students Institutional level: What are effective methods for collaborative learning ICREAMS
When is Design Research Appropriate? Eamonn Kelly (2009): Initial state(s) unknown or unclear Goal state(s) unknown or unclear Operators to move from initial states to goal states are unknown or how to apply the operators is unclear ICREAMS
Generalizability in Design Research: So: design principles must be seen as ‘heuristic’ statements providing guidance and direction, but do not give ‘certainties’. When we give proper weight to local conditions, any generalization is a working hypothesis, not a conclusion! (Lee Cronbach, 1975, cited by Tom Reeves) ICREAMS