Statistics and Research methods

1. Statistics and Research methods Wiskunde voor HMI Bijeenkomst 3 Relating statistics and experimental design

2. Contents • Multiple regression • Inferential statistics • Basic research designs • Hypothesis testing • Learn to select the appropriate statistical test in a particular research design

3. Multiple Regression • Multiple correlation • The association between a criterion variable and two or more predictor variables • Multiple regression • Making predictions with two or more predictor variables

4. Multiple Regression • Multiple regression prediction models • Each predictor variable has its own regression coefficient • e.g., Z-score multiple regression formula with three predictor variables: Standardized regression coefficients

5. Multiple Regression • Note: the betas are not the same as the correlation coefficients for each predictor variable (because predictors “overlap”) • Standardized regression coefficient (Beta) of a variable: about unique, distinctive contribution of that variable (overlap excluded) • There is also a corresponding raw score prediction formula for multiple regression: Ŷ = a + (b1)(X1) + (b2)(X2) + (b3)(X3)

6. Multiple correlation coefficient • R • In SPSSoutput: Multiple R • R is usually smaller than the sum of individual correlation coefficients in bivariate regression • R2 is proportionate reduction in error = proportion of variance accounted for

7. ResearchExample

8. Inferential Statistics • Make decisions about populations based on information in samples (as opposed to descriptive statistics, which summarize the attributes of known data) • Notations in statistical test theory

9. Sample and population

10. The Normal Distribution (Z-scores) • Normal curve and percentage of scores between the mean and 1 and 2 standard deviations from the mean

11. Basic research methods • Experimental method • manipulation of variables and measure effects • Field studies – observation • No outside intervention, e.g. ethnography • Quasi-experimental method • Combination of elements of other two We concentrate on experiments and quasi-experiments

12. Experimental method • Manipulation of (levels of) one or more independent variables (e.g. medication: pill or placebo; different versions of a user interface)  experimental conditions • Control (keep constant) other possibly intervening variables • Measure dependent variables (e.g. effectiveness, performance, satisfaction) • Test for differences between the conditions

13. Experimental design How to assign subjects to conditions? • Between-subjects design • a subject is assigned to only one of the conditions • Within-subjects design orRepeated measures design • Each subjects receives all the experimental conditions

14. Between-subjects design • Randomization: assign subject at random to different conditions • Matching: random assignment but control for variable that is expected to be very relevant Example: (if sex is important) seperately assign men to experimental groups assign women to experimental groupsEqual amount of men and women in conditions.“the subjetcs in each condition were matched on sex”

15. Between-subjects design (continued) • Matched pairs • Two subjects that are similar (on relevant variable(s)) assigned to different conditions • Randomized blocks design • Extension of matched pairs for more than two conditions, e.g. 3 conditions • Form blocks of 3 similar subjects • Assign subjects in one block randomly to different conditions

16. Between-subjects design (continued) • Factorial designs • More than one independent variable • Study separate effects of each variable (main effects) but also interaction between variables • Interaction effect: the impact of one variable depends on the level of the other variable • Two-way factorial research design (two independent variables); three-way with three indep. variables • 2x2 if independent variables have two levels (condions) or 3x3 with three levels

17. Within-subjects design • Same subjects in each experimental condition • Repeated measures design • Within-subjects design required if change is measured as a consequence of an experimental treatment (e.g. testscores before and after a training) • In other situations: carryover effects • experimental conditions need to be counterbalanced • One half sequence AB the other half BA

18. Quasi-experimental method • Combination of elements from experimental methods and field research

19. Hypotheses Testing • H0: Null hypothesis – No difference • The Independent variable has no effect e.g. pill or placebo make no difference • H1 (or Ha): Alternative hypothesis – Significant difference • The Independent variable has an effect

20. Hypothesis Testing Errors • Type I Error: • Null hypothesis is rejected but true. • Alpha (α) probability of making type I error • Type II Error: • Null hypothesis is not rejected but false. • Beta (β) probability of making type II error No effect, but you say there is. Real effect, but you say there’s not.

21. Type I and II errors αusually0.05or 0.01 βusually 0.20

22. Statistical Power Power: The probability that a test will correctly reject a false null hypothesis (1- β )

23. An Example of Hypothesis Testing • A person claims to be able to identify people of above-average intelligence (IQ) with her eyes closed • We devise a test – take her to a stadium full of randomly selected people from the population and ask her to pick someone with her eyes closed who is of above average IQ. • If she does, we’ll be convinced. But she might pick someone with an above-average IQ just by chance.

24. Distribution of IQ Scores

26. The Hypothesis Testing Process • Restate the question as a research hypothesis and a null hypothesis about the populations • Population 1 • Population 2 • Research hypothesis or alternative hypothesis • Null hypothesis

27. The Hypothesis Testing Process • Determine the characteristics of the comparison distribution • Comparison distribution: distribution of the sort you would have if the null-hypothesis were true.

28. The Hypothesis Testing Process • Determine the cutoff sample score on the comparison distribution at which the null hypothesis should be rejected • Cutoff sample score • Conventional levels of significance: p < .05, p < .01

29. The Hypothesis Testing Process • Determine your sample’s score on the comparison distribution • Decide whether to reject the null hypothesis

30. One-Tailed and Two-Tailed Hypothesis Tests • Directional hypotheses • One-tailed test • Nondirectional hypotheses • Two-tailed test

31. Determining Cutoff Points With Two-Tailed Tests • Divide up the significance between the two tails