Quasi-Experimental Research in Education: Designs and Methodology

1. Basically, you need to know how experimental units are assign to groups how level of treatments are assign to groups nature of X what are under the control of experimenter These are characteristics of true experiment If you can’t do true experiment, use designs that can give you the most control  quasi-experimental design edu5900 ab. rahim bakar

2. QUASI-EXPERIMENTAL RESEARCH • In education, difficult to do true experiment because of the difficulty to have randomization, no control on the scheduling of treatment. • Lost part of the power due to lack of randomization (assignment of subjects) • Control of extraneous variable, difficult. • External validity is enhanced ―> may be moving toward real world setting edu5900 ab. rahim bakar

3. Use when true experiment is not possible Any design that does not randomly assigned subjects to the group is known as quasi-experimental designs. Researchers do not use randomization but rely instead on other techniques to control threat to internal validity edu5900 ab. rahim bakar

4. Quasi Experimental Designs This type of design involves a treatment (manipulation ) and an outcome but lacks one of the other two properties that characterize a true experiment: randomization or a control group. Example: if you want to study the effects of smoking on a variable, you cannot randomly assign people to smoking vs nonsmoking group. edu5900 ab. rahim bakar

5. Quasi-experimental Methodology  In this methodology, the essential controls are not administered, such as treatments not being randomized. It is otherwise similar in structure to a true experimental design. edu5900 ab. rahim bakar

6. Designs frequently used: • NON-EQUIVALENT CONTROL GROUP DESIGN (NONRANDOMIZED Control group, pretest-post test design) E O1 X1 O2 ----------------------- C O3 X2 O4 edu5900 ab. rahim bakar

7. Not equivalent because no randomization Naturally formed group Contoh? Class sections>differ Randomized the treatmet not the subjects will make the design stronger Assess before and after treatment Experiment occupies the same time period Control almost all IV problem Possible problem > regression edu5900 ab. rahim bakar

8. Choose two classes of form five students (your school may have 5 classess) Choose two classes that appear to be equivalent (no remedial and no advance classes) Give pretest (comprehension test) to both experimental and control groups Administer the treatment to improve students comprehension to the experimental group only Look for difference on posttest scores E C , can you conclude, treat is effective? edu5900 ab. rahim bakar

9. No randomization, not equivalent group High score could be due to high ability to begin with Thus, there is a selection bias and it threaten the internal validity, but Pretest can overcome this threat. It checks if the groups are equivalent to begin with. If no different, selection threat is eliminated. If yes, use ANCOVA to adjust posttest score for the pretest differences edu5900 ab. rahim bakar

10. Other threats, NO, why? Subjects take the same pretest and posttest They occupy the same time period This design does not control for interaction of selection and maturation, interaction of selection and regression, and interaction of selection and instrumentation edu5900 ab. rahim bakar

11. What is interaction of selection and maturation? One of the selected group is more subject to maturation than the other (will it happen in our proposed study?) edu5900 ab. rahim bakar

12. But, what if edu5900 ab. rahim bakar

13. Non equivalent control groups - other than the absence of randomly assigned groups, these designs are similar to experimental designs . However, lack of random assignment to control and experimental groups, can not assure that the groups are equal. The researcher must do everything possible to show that there are no differences. For example, a pretest may show that there is no difference. If the study is done on "after only data", this control is not present edu5900 ab. rahim bakar

14. COUNTERBALANCED DESIGN • Another design that can be used with intact group • Rotate the group at interval during the experiment • Group 1 & 2 might use method A, respectively for the first of the experiment, then exchange the method for the second half edu5900 ab. rahim bakar

15. edu5900 ab. rahim bakar

16. After the study compute the column mean and compare the result to determin the effectiveness of the methods Because the treatment is administered to all groups, the results can’t be attributed to the existing differences Main disadvantage is the carryover effect form one X to another X edu5900 ab. rahim bakar

17. Time siries design • One Group Time Series Design • O1 O2 O3 X O4 O5 O6 edu5900 ab. rahim bakar

18. Ambil ukuran beberapa kali sebelum dan selepas treatment. Check for the stability of repeated measures Still lack the control Rules out maturation, testing, regression but not history Should specify the time interval in advance X mestilah dibawah kawalan penyelidik Show contoh? Must have sufficient number of repeated measures * Study the effect of a major change in administrative policy upon disiplinary incidents * The effect of a motivation programs on students’ attitudes towards school edu5900 ab. rahim bakar

19. A * * * * * * * * * B * * * * * * * * C * * * * * * * D edu5900 ab. rahim bakar X yi y2 y3 y4 y5 y6 y7 y8

20. Difference between Y4 and Y5 on A justify you to suggest that X affect Y Pattern B, possible temporary effect Patter C, could be the result of maturation Pattern D could be some influences of extraneous variables. edu5900 ab. rahim bakar

21. Threats: • History • Instrumentation • Testing • Pretest treatment interaction edu5900 ab. rahim bakar

22. Control group time series design • E Y1 Y2 Y3 Y4 X Y5 Y6 Y7 Y8 • C Y1 Y2 Y3 Y4 - Y5 Y6 Y7 Y8 edu5900 ab. rahim bakar

23. Time series designs - tests for changes over time. There is no randomization and there is no control group. However, the researcher can use each person as his own control. If possible, test the group several times prior to introducing the intervention. This method gives more reliability to the study edu5900 ab. rahim bakar

24. PRE TRUE EXPERIMENTAL QUASI edu5900 ab. rahim bakar

25. edu5900 ab. rahim bakar

26. Ex post facto research edu5900 ab. rahim bakar

27. We want to know why certain behavior the way it is. Why are some children better readers than the others? What is the effect of single-parent home on achievement Here we normally deal with attribute variables or variable that can be manipulated but not manipulated edu5900 ab. rahim bakar

28. When a researcher use attribute independent variable that he or she can’t manipulate, ex post facto research sometimes called causal-comparative research must be used. Also when variable can be manipulated but not manipulated edu5900 ab. rahim bakar

29. Two basic mode of ex post facto research Begin with subjects who differ on an independent variable and try to determine the consequences of this differences (A) Begin with subjects who differ on an dependent variable and try to determine the antecedent (cause) of the difference. (B) edu5900 ab. rahim bakar

30. Example of (A): investigating the effect of single-parent versus that of two-parent on… Example of (B): compare students who graduated from … and those who drop-out on independent variables such as motivation, clarity of goals, and social maturation edu5900 ab. rahim bakar

31. edu5900 ab. rahim bakar

32. Design 1 Look for the consequences of differences on independent variable One group posses the characteristics and the others do not A study to compare the academic performance of creative and non-creative college students Hypothesis: creative college students will exhibit greater speed and accuracy on a problem-solving task than will noncreative college students. edu5900 ab. rahim bakar

33. What is the independent variable? Creativity. Can we manipulate creativity? Or randomly assign students to creative and noncreative group? We start with two groups that are known to differ on independent and compare them on dependent variable. You must identify other independent variables that might have an impact on dependent variable. edu5900 ab. rahim bakar

34. Design 2 Look for the cause(s) of difference in the dependent variable Research on gender differences in performance on standardized mathematics test. What factors might account for the observed difference. edu5900 ab. rahim bakar

35. “The Effect Of Students Anxiety In An Achievement Testing Situation On Examination Performance” Apa yang berlaku di sini? Bolehkah kita menggunakan kajian experiment ? Why not or why shouldn’t we? Why do you go for ex post facto? You can have either ex post facto or experimental study dalam kes ini edu5900 ab. rahim bakar

36. Ex post facto → ukur tahap kerisauan sebelum examination Then what do you do ? high and low anxiety compare the exam result if test result significant→what does it mean? Here you do not induce anxiety→natural occurring Experimental way >how? edu5900 ab. rahim bakar

37. What’s the different between ex post facto and experimental? Begin with the measure of the present situation like student achievement then try to determine the factors that may contribute to the achievement Manipulation of X Assignment to groups Both methods require at least one categorical variable (group membership) Both compare group performance edu5900 ab. rahim bakar

38. Frequently used design: static group comparison X 1 O X 2 O Other designs? Use quasi-experimental designs such as: + time series + non-equivalent control group design + multiple time-series design Jika berupaya cadang guna salah satu daripada rekabentuk quasi-experiment edu5900 ab. rahim bakar

39. DESIGNING EX POST FACTO STUDY • Begin with research hypothesis to be tested • [major hypotheses]-educational level and commitment • construct alternative hypotheses • [extraneous variable and the relationship with dependent variable are tested] • those EXTRaneous that are significantly correlated with DV pose threat to the internal validity of the study • Analyse edu5900 ab. rahim bakar

40. If that is the case, test the relationship antara significant EXT var dan major independent variable If there exist significant correlation between the two, threat to internal validity A study was conducted to determine the effect of type of institution and program effectiveness edu5900 ab. rahim bakar

41. Major hypothesis: Variable: types of institution and program effectiveness Control hypotheses: Variable: age, data processing experience Example :age X program effectiveness edu5900 ab. rahim bakar

42. If there is significant correlation, test the hypothesis between age and type of institution. No correlation is good, but yes correlation no good edu5900 ab. rahim bakar

43. edu5900 ab. rahim bakar

44. There may be significant correlation between age and type of institution There may be correlation between data processing experience and type of institution Difference of effectiveness may be because of those two extraneous variables edu5900 ab. rahim bakar

45. Therefore, major hypothesis may not be supported even if the difference exist. Graduate of public institution could be older and have more data processing experience than those from private institution edu5900 ab. rahim bakar

46. Do automobile and scholarship mix? No straight A students use car 15% of B students drive to school 41% of C students drive to school 71% of D students drive to school 83% of E students drive to school there is a relationship between scholarship and the use of car!!! But can we conclude that penggunaan kereta ke sekolah boleh menjatuhkan pencapaian akademik? edu5900 ab. rahim bakar

47. Look at common cause> ada tak pembolehubah yang mungkin mempengaruhi penggunaan otomobil dan juga pencapaian akademik? Bagaimana dengan status sosioekonomi? SES mungkin mempengaruhi penggunaan otomobil edu5900 ab. rahim bakar

48. Reverse causality> is it possible that pencapaian yang rendah menyebabkan pertambahan kadar penggunaan otomobil. Pelajar mungkin mencar satu jalan untuk mengurangkan frustration. Pelajar mungkin mencari pengangkutan alternatif untuk menjimatkan masa perjalanan dari dan ke sekolah edu5900 ab. rahim bakar

49. Lain-lain pembolehubah bebas> could it be that teachers perception about car users. Could it be that the students are poor students to begin with therefor using or not using oyomobil has no effect on the grade edu5900 ab. rahim bakar

50. BEBERAPA JENIS KAJIAN EX POST FACTO • Exploration Of Effect (Dependent Variable) • Penerokaan Kesan Yang Disebabkan Oleh Keahlian Dalam Sesuatu Kumpulan • Q-Adakah terdapat perbezaan rancangan kerjaya antara pelajar berdasarkan pengaliran (streaming) • contohnya? exploration of causes (independent variable) edu5900 ab. rahim bakar