Experimental Research Methods in Language Learning

1. Experimental Research Methods in Language Learning Chapter 2 Experimental Research Basics

2. Leading Questions • Do you believe in a cause-effect relationship in language learning? Why or why not? • What do you think are characteristics of an experimental research study? • What kind of research questions do you think experimental researchers ask?

3. Experimental Research Design • A useful research methodology for those studies that aim to address a causal-like relationship. • It allows researchers to strictly control the influence of factors that are not of interest by setting them constant across groups, but to vary the degree of a factor under study across groups of learners in order to undestand a causal-like relationship.

4. Key Characteristics of Experimental Research • Causal or causal-like relationships • Research questions in experimental research • Independent and dependent variables • Dichotomous and continuous variables • Measurement scales of variables • Constructs in experimental research • Manipulation and control in experimental research

5. Causal or Causal-like Relationships • Researchers can examine whether their hypothesis about a causal-like relationship is supported by empirical data • Reasons for using ‘causal-like’ in place of ‘causal’ are that there is no direct proof that an experimental finding indicates a causal relationship (i.e. We can only make inferences). • Statistical analysis in experimental research is not a method to discover causes.

6. Examples of Research Titles • Baralt and Gurzynski-Weiss (2011): Comparing learners’ state anxiety during task-based interaction in computer-mediated and face-to-face communication • Park (2010): The influence of pretask instructions and pretask planning on focus on form during Korean EFL task-based interaction • Takimoto (2006): The effects of explicit feedback on the development of pragmatic proficiency

7. Research Questions • Research questions are used to frame a research focus and method to answer them • Two types of research questions: Theoretical questions, and practical or pedagogical questions. • Theoretical questions are connected with basic research which seeks empirical evidence that can inform a new theory, or refine and extend existing theories. • Example: What is motivation in L2 learning?

8. Research Questions • Practical/pedagogical research questions are connected with applied research which aims to address a practical/pedagogical problem in a particular setting or to apply relevant theories or recommendations to real practice. • Example: How can teachers help students memorize a grammar rule?

9. Examples of Research Questions • Do adult and child dyads respond differently to the amount of implicit negative feedback provided to NNSs [non-native English speakers] during task-based interaction? (Mackey, Oliver & Leeman 2003, p.44) • To what extent is Lexical Focus-on-Form beneficial during a focus on meaning activity (such as listening comprehension) in terms of students’ receptive vocabulary learning? (Tian & Macaro 2012, p. 373)

10. Independent and Dependent Variables • Variable is an aspect or characteristic of something that can take different values or scores. • Examples: age, gender, first language, length of learning, intelligence, English language proficiency, motivation and anxiety • An independent variable is a variable that exists freely, and is hypothesized to have an effect on other variables that are described as dependent variables

11. Independent and Dependent Variables • Independent variables are factors that influence certain behaviors or psychological processes. • A dependent variable is a variable that changes as the independent variable being examined changes. • Example: Gass and Mackey (1999): The effects of task repetition [IV] on linguistic output [DV]

12. Dichotomous and Continuous Variables • Categorical variables are used to group non-overlapping variables such as English proficiency levels (beginning, intermediate and advanced). • A dichotomous variable is the simplest type of categorical variable. It has only two classes (e.g., male or female, pass or fail). • Continuous variables can be arranged from lowest to highest (e.g. age, length of residency)

13. Measurement Scales of Variables • Nominal scales use numbers to label or classify variables into categories • Ordinal scales are rank-order scales; used for ranking some quality or ability (e.g. grade point average (GPA)). • Interval scales have the features of both ordinal scales and equal distances or intervals (e.g. language test scores, personality scores). • Ratio scales are measurements with all the properties of nominal, ordinal and interval scales, and also possess a true zero

14. Constructs in Experimental Research revisited • Constructs are aspects or abstract concepts that researchers seek to understand (e.g. intelligence, language proficiency, memory, language aptitude). • Constructs cannot be seen directly or be easily measured, thereby needing a sound theory to help us define and measure them.

15. Constitutive versus Operational Constructs • A constitutive construct is one defined using the general definition of a term (e.g., motivation, self-regulation, language learning strategies, and self-efficacy). • An operational construct definition is one defined by researchers for a particular study. It is often concerned with exactly what a researcher means in a research setting and how it can be measured using a designed instrument.

16. Manipulation and Control in Experimental Research • Manipulation is related to how experimental researchers manipulate independent variables. • It aims to rule out other plausible rival explanations of the research outcomes. • Manipulation helps researchers control confounding variables as well as to systematically vary the independent variable for testing its effect.

17. Manipulation and Control in Experimental Research • Confounding variables are those independent variables which are not of interest but can interact with the independent variable to co-affect the dependent variable. • Manipulation can also take place when researchers hold several conditions for two or more groups of comparisons constant, to avoid the potential confounding effects.

18. Discussion • What are the ways in which experimental researchers can produce evidence of causality in language learning? • What is random assignment? Why is it essential for a true experimental study? • Can you think of a situation in which you can do an experimental study? Think of a situation in which you can have control over your research. What would be the potential confounding variables that can influence your findings?