1 / 37

Non-Experimental Methods I

Non-Experimental Methods I. Class 22. Why Alternatives to Controlled Experiment? Advantage of controlled experiment? Advantages to not using controlled experiment?. Controlled Experiment: Great, but Not Always the Best Ethical Concerns Practical Concerns Methodological Concerns

afric
Download Presentation

Non-Experimental Methods I

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Non-Experimental Methods I Class 22

  2. Why Alternatives to Controlled Experiment? Advantage of controlled experiment? Advantages to not using controlled experiment?

  3. Controlled Experiment: Great, but Not Always the Best Ethical Concerns Practical Concerns Methodological Concerns Generalizability Internal analyses Not all interesting questions are causal

  4. Types of non-experimental studies a. Correlational b. Pseudo-experimental c. Quasi-experimental

  5. Correlational Studies FIASCO STUDY: Shared meals will reduce major fights Correlation of meals to major fights = -. 37  Interpretations: 1. ??????????? 2. ??????????? 3. ???????????

  6. Less overtime at work Correlational Studies FIASCO STUDY: Shared meals will reduce major fights Correlation of meals to major fights = -. 37  Interpretations: Dinners -----------------> Fewer fights Fewer fights ----------> More dinners more dinners fewer fights

  7. ??? Panel Design Do shared meals reduce the frequency of major family conflicts? Time 1 Time 2 (3 years) Dinners Dinners r = +.24 A r = - .37 r = - .31 B r = +.40 Fights Fights

  8. Cross-Lagged Panel Design Do shared meals reduce the frequency of major family conflicts? Time 1 Time 2 (3 years) Dinners Dinners r = +.24 A r = - .37 r = - .31 B r = +.40 Fights Fights

  9. Cross-Lagged Panel Design Do shared meals reduce the frequency of major family conflicts? Time 1 Time 2 (3 years) Dinners Dinners r = +.24 r = - .37 r = -.15 r = - .31 r = -.39 r = +.40 Fights Fights (Pssst: Ask Kent about Walter Michel study)

  10. ????? Designs 1. One-shot (post-test only) case study 2. One-group pretest-posttest design 3. Static-group comparison design

  11. Pseudo-experimental Designs 1. One-shot (post-test only) case study 2. One-group pretest-posttest design 3. Static-group comparison design

  12. One Shot Case Study Design: Observe population of interest Examples: Client case studies (client case studies, famous people) Group case studies (focus groups) Population case studies (anthropological studies) FIASCO: Study families at dinner time, observe dinner together --> less conflicts.  Problems?

  13. One Shot Case Study Design: Observe population of interest Examples: Client case studies (client case studies, famous people) Group case studies (focus groups) Population case studies (anthropological studies) FIASCO: Study families at dinner time, observe dinner together --> less conflicts.  Problems: No comparison: with other families displaying same behavior with same family under different conditions.

  14. Excellent Case Studies De Tocqueville in America Freud’s studies of Anna O, and the Wolfman Cantril’s study of NJ after Mars Invasion Wertheimer on train  Gestalt psychology

  15. One Group Pretest-Posttest Design Observe before and after treatment Pretest and posttest provide basis for comparison Example: * Monitor families before and after dinner * Record number/intensity of conflicts pre vs. post dinner. * Finding: Less tension after dinner Problems with this design???

  16. Threats to Internal Validity 1. History effects: ????? 2. ????: People change over time 3. Testing: ???? 4. ?????: Experimenter, apparatus change. 5. Mortality: ???? 6. Selection: ????

  17. Threats to Internal Validity 1. History effects: Things co-occur with TX 2. Maturation: People change over time 3. Testing: People react to being observed 4. Instrumentation: Experimenter, apparatus change. 5. Mortality: People drop out before study ends. 6. Selection: People volunteering for study may differ from those who did not volunteer.

  18. Selection. Maybe non-dinner family found that not sharing dinner helps prevent conflicts. Static Group Comparison Design General idea: Take advantage of naturally occurring comparison group. Examples: School intervention: Look at students not in new program FIASCO: Look at families nearby, that don't have dinner together. Advantage: Provides some basis for comparison. Problem?

  19. ???-Experiments Features * Control administration of treatment (TX) * Control the collection of DV measures * BUT, no random assignment to TX or control WITHIN GROUP QUASI-DESIGNS Simple time series designs Equivalent time-samples design BETWEEN GROUP QUASI-DESIGN Non-equivalent Control-group design Multiple-group time-series designs

  20.   Quasi-Experiments Features * Control administration of treatment (TX) * Control the collection of DV measures * BUT, no random assignment to TX or control WITHIN GROUP QUASI-DESIGNS Simple time series designs Equivalent time-samples design BETWEEN GROUP QUASI-DESIGN Non-equivalent Control-group design Multiple-group time-series designs

  21. The Arith-Matisse Project Problem: Many students have math anxiety. Also, they don't know how to engage in visual skills that might improve their math achievement. Solution: Provide students 1 hour of drawing / fine arts instruction to help develop visual skills, and also to relax them before math class. Art class must immediately precede math class. Prediction: Art class preceding math class will lead to better math performance.

  22. Art Hour Offered Day Score on Daily Math Quiz No Monday 60% No Tuesday 57% No Wednedsay 63% Yes Thursday 84% No Friday 74% No Monday 64% No Tuesday 59% Time Series Experiment: Simple Time Series Design Effect of art hour on math class performance

  23. Art Hour Offered Day Score on Daily Math Quiz No Friday 81% No Monday 60% No Monday 87% No Tuesday 57% No Tuesday 90% No Wednedsay 63% Yes Thursday 84% Why It’s Necessary to Monitor Outcome Beyond Treatment Administration Question: Why not stop monitoring after treatment (e.g., Thurs)? Reason: If stopped on Thurs., would miss continuation of change.

  24. Problems with Simple Time Series • Hawthorne Effect: Being observed can, in itself, be motivating or otherwise affect change. • History Effect Events co-occurring with treatment could be influencing outcomes.

  25. Problems with Simple Time Series • Hawthorne Effect: ???? • History Effect: ????

  26. Art Hour Offered Day Score on Daily Math Quiz Time Series Experiment: Equivalent Time Samples Design No Monday 60% No Tuesday 57% No Wednesday 63% Yes Thursday 84% No Friday 74% No Monday 64% No Tuesday 59% Yes Wednesday 85% No Thursday 62% No Friday 65% Effect of art hour on math class performance

  27. Art Hour Offered Day Score on Daily Math Quiz Problems With Equivalent Time Samples Design? No Monday 60% No Tuesday 57% No Wednesday 63% Yes Thursday 84% No Friday 74% No Monday 64% No Tuesday 59% Yes Wednesday 85% No Thursday 62% No Friday 65%

  28. Art Hour Offered Day Score on Daily Math Quiz Problems With Equivalent Time Samples Design? No Monday 60% No Tuesday 57% No Wednesday 63% Yes Thursday 84% No Friday 74% No Monday 64% No Tuesday 59% Yes Wednesday 85% No Thursday 62% No Friday 65% No comparison condition. Example: Students do well on math in order to retain art class, but not b/c art helps with math

  29. Psychologists Sans Data William JamesSolomon Asch Father of Amer. psychology Perception The Self Conformity, group dynamics Emotions Group Dynamics Gordon Allport Perception Founds Personality Theory Prejudice research Sigmund Freud Unconscious Processes Humanists Psychosomatics Victor Frankl Social, Personality, Developmental Rolo May Abraham Maslow Kurt Lewin Carl Rogers Group Processes Social Systems The Self

  30. “Invasion From ???”: The ‘Big Bad Boo’ of 1938 Orson Well broadcast of invasion from Mars—as if a breaking news story, Oct. 30, 1938 Approximately 30 million people affected, to varying degrees Historical context a. ??? relatively new—and trusted b. World on brink of ???

  31. “Invasion From Mars”: The ‘Big Bad Boo’ of 1938 Orson Wells broadcast of invasion from Mars—as if a breaking news story, Oct. 30, 1938 Approximately 30 million people affected, to varying degrees Historical context a. Radio relatively new—and trusted b. World on brink of world war

  32. Cantril Study (1940) Rationale: Rare event; oppty to study mass behavior Event structures research: * How people respond during panic/stress * Power of real-life event, outside the lab Method: * Multi-method approach—provides converging evidence: -- Interviews (n = 135) -- Unobtrusive measures (e.g., phone company) -- Commercial survey research (Gallop, AIPO) -- Newspapers

  33. Observations from Invasion from Mars Study Recovery from panic * Prolonged anxiety – sometimes “I was sick in bed for 3 days after the broadcast” * “Nevermind” – sometimes “We listened to the rest of the play and went out dancing” Role of religion and faith * “I held a crucifix in my hand and prayed while looking out of my window for falling meteors”

  34. Observations from Invasion from Mars Study (continued) Role of purposeful action in coping * “...having to concentrate on the driving held me together somewhat” Fear of unknown * I didn’t (know) what I was fleeing from, and that made me all the more afraid.” Background fears based on reality influence immediate, ambiguous threat. * “Don’t you know New Jersey is destroyed by the Germans...” Final note: Open-ended interviews provide invaluable historical record.

More Related