Quasi-experimental Designs & Survey Research

1. Quasi-experimental Designs & SurveyResearch

2. Quasi-experiments • Hedrick, Bickman & Rog (1993) “a quasi-experimental design is not the method of choice, but rather a fallback strategy” • Cannot infer cause and effect, BUT well designed quasi-experiments enable you to demonstrate that rival interpretations are rendered unlikely

3. Non-equivalent Group Designs • Posttest only nonequivalent control group design (aka static group comparison) X O (treatment grp) O (nonequivalent control grp) X=treatment O=measurement/observation • Because there is no random assignment to groups, confounding variables may explain any difference observed

4. Pre-test/Post-test Non-equivalent Control Group O X O treatment group O O non-equivalent control group Pre Post

5. Example: Research Methods and Reasoning Ability • Intervention: critical thinking seminar • Research Methods students receive the intervention (i.e., participate in the critical thinking seminar) • Developmental Psychology students are used as a nonequivalent control group (i.e., do not attend the seminar)

6. Interrupted Time Series Designs • Extension of the simple one group pre and post design • Participants are pre-tested a number of times and then post-tested a number of times after being exposed to the treatment intervention • O1 O2 O3 O4 X O5 O6 O7 O8

7. Interrupted Time Series Designs • Useful when: • cannot randomize participants • it is possible to obtain a series of assessments of the DV before and after treatment

8. Example: Interrupted Time Series Designs • Intervention: Course to change students’ study habits, implemented during the summer (after semester 4). • DV: semester GPA Post-tx Baseline X

9. How Many Measurements Are Needed for a Time Series Design? • Depends on: • amount of random fluctuation (noise) that may occur in the outcome being measured • how much of an impact the intervention is expected to have

10. Multiple Time Series Designs • Add a comparison group to the simple interrupted time series design: O1 O2 O3 O4 X O5 O6 O7 O8 ------------------------------------------------------------------ O1 O2 O3 O4 O5 O6 O7 O8 • Campbell & Stanley consider this an excellent quasi-experimental design

11. Survey Research

12. Surveys Items are composed of 2 parts: stem & response options The university should not raise tuition under any circumstances (stem) Agree Disagree (response options)

13. Question Wording Important consideration when writing questions: Simplicity Avoid jargon and technical terms Avoid abbreviations Be specific Avoid double-barreled questions -

14. Responses to Questions Open-ended Closed-ended

15. Open vs. Closed Questions “…closed questions should be used where alternative replies are known, are limited in number, and are clear cut. Open ended questions are used where the issue is complex, where relevant dimensions are not known, and where process is being explored.” Stacey (1969)

16. Rating Scales Likert Scales Semantic Differential Graphic Rating Scale Non-verbal scale

17. Question Placement Very important Confusing questionnaire = demoralizing to respondents Can affect people’s responses to them

18. Layout of Questionnaire Use closed-ended questions Use a consistent item format Don’t crowd questions together Don’t split questions or response options between pages It is most likely to be completed if easy on the eye, relevant, logical and as short as possible.

19. Response Biases Social desirability Acquiescence response - tendency either to agree/disagree with statements Yea-Saying Nay-Saying

20. Checklist for Survey Use Do the items provide a representative sample of the behaviour under study? Do the items ask for irrelevant information? Does the question have the same meaning for researcher and respondent? Is the question worded clearly?Are the words too difficult? Does the question concern a sensitive issue?

21. Checklist for Survey Use Does the question suggest a socially desirable answer? Open vs closed questions? How many questions are necessary to obtain the required information? How might the questions be arranged to encourage the respondent’s participation?