Experimental Research Methods in Language Learning

1. Experimental Research Methods in Language Learning Chapter 3 Experimental Research Paradigm and Processes

2. Leading Questions • Have you ever heard of the term ‘research paradigm”? Do you know what it is about? • What are key criteria you have in mind when you read a research article? • Do you know steps researchers take to complete their experimental study?

3. Research Paradigms in Experimental Research • Guba and Lincoln (2005) define a research paradigm as a set of related beliefs/assumptions that underlie an approach to research and its relationship to the world. • A paradigm is related to how researchers see the world, what they believe constitutes knowledge about the world and how this knowledge can be attained.

4. Ontology, Epistemology and Methodology • Ontology: What is a reality or truth? • Epistemology: What is our relationship to the established reality or truth? • Methodology: How do we know about the reality or truth?

5. Ontology, Epistemology and Methodology • At an ontological level, we ask ‘what is reality?’ How do we know what we think we know is real? • At an epistemological level, we seek to establish the relationship between ourselves and what we aim to know (e.g., research constructs and their causal-like relationships). Questions includes objective-subjective judgements in our observations.

6. Ontology, Epistemology and Methodology • At a methodological level, we ask ‘how do we go about our pursuit of knowledge?’ This level is related to the research methods we employ to understand reality. We ask ‘how do we collect data that are suitable for our research questions?’, and ‘how do we analyze them?’

7. Positivist Paradigm • A realist perspective, which believes that the object of an inquiry really exists ‘out there’ in the world • Reality is governed by immutable laws and mechanisms essentially independent of whom, when and how it is being examined. • Takes an objectivist stance toward an inquiry. • Tries to completely remove their influence from the research setting. • Attempts to control variables and manipulate the research setting

8. Postpositivist Paradigm • A modified positivism and takes similar stances to those of the positivists. • Unlike the positivists, the postpositivists maintain that the object of their inquiry can never be perceived with total accuracy, though existing outside and independent of their minds. • Assumes that objectivity is nearly impossible to achieve in research, but the notion of objectivity is retained as an ideology to regulate their research.

9. Postpositivist Paradigm • Postpositivists modify the positivists’ position by encouraging the use of multiple strategies for gathering and analyzing data (including qualitative data) • Multiple strategies can allow researchers to gain a more complete understanding of the subject matter.

10. Constructivist Paradigm • Does not share the realist or critical realist perspective. • Takes the relativist stance that realities are multiple and exist in people’s minds. • Multiple realities exist and are constructed by individual observers • Takes the subjectivist position that a reality is inherently and unavoidably subjective. • Adopts a non-experimental, non-manipulative set of research procedures.

11. Constructivist Paradigm • Ethnographic studies, for example, employ a range of techniques associated with prolonged fieldwork, such as participant observations and in-depth interviews. • Dialogic approaches are methods that allow an interaction with participants in the research setting. • Hermeneutic describes a research process in which the researcher forms interpretations based on their observation notes, and interview recordings, etc

12. Experimental Research Processes • Figure 3.1 presents the key stages of research processes. • The figure suggests that in practice, these processes are iterative or cyclical (i.e., going back and forth).

13. Figure 3.1 Experimental Research Processes

14. Experimental Research Processes 1. Choos a Topic: To find important, researchable and feasible topic 2. Identify a Research Problem: To have a clear, well-identified research problem or gap as suggested by the existing literature 3. Do a Literature Review: To conduct and write a comprehensive and critical review of the relevant literature including theories and previous studies. To ask research questions or set research hypotheses.

15. Experimental Research Processes 4. Design an Experimental Study: To plan and organize the study; to consider a suitable research design that can address the research problem; to consider research instruments and data collection procedure; to consider ways in which data will be analyzed. 5. Consider Ethics: To consider any physical or psychological harms to research participants; to have consent from participants or guardians; to gain permission from the authority to conduct the study; to obtain institutional ethics approval.

16. Experimental Research Processes 5. Consider Ethics: To consider any physical or psychological harms to research participants; to have consent from participants or guardians; to gain permission from the authority to conduct the study; to obtain institutional ethics approval. 6. Collect Data: To conduct an experiment (provide treatment(s)), as planned and collect all necessary data to address the research problem/answer the research questions; to use research instruments appropriate for the study.

17. Experimental Research Processes 7. Analyze Data: To check data for completeness prior to a preliminary data analysis; to conduct reliability analysis; to perform statistical analysis such as a t-test or an analysis of variance (ANOVA); if qualitative data is also collected, to analyze it following a particular qualitative framework; to empirically answer the research questions

18. Experimental Research Processes 8. Interpret Data: To go beyond raw data in order to conclude about the effect of the treatment or condition on something; to discuss both statistical and practical significance of the study 9. Discuss Findings: To discuss results in relation to the theories and previous research; to conclude the study and consider implications and limitations of the study. 10. Write up a Report: Todisseminate the research in a written report.

19. Discussion • Do you find it useful to distinguish the positivist, from the postpositivist and the constructivist? Why or why not? • What are difficult concepts about research paradigms that you have had so far? • In your view, which experimental stage(s) is the most critical to good experimental research?