1 / 19

Chapter 14

Chapter 14. External Validity of Research. External Validity. Concept of External Validity Structural Component Functional & Conceptual Components Assessing External Validity Lab Research, Natural Setting & External Validity Summary. External Validity: Concept.

fitzgerald-trevino
Download Presentation

Chapter 14

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Chapter 14 External Validity of Research

  2. External Validity • Concept of External Validity • Structural Component • Functional & Conceptual Components • Assessing External Validity • Lab Research, Natural Setting & External Validity • Summary

  3. External Validity:Concept • Do results generalize “across”: • Other settings • Populations • Times • Do results generalize “to”: • Particular setting • Particular population • ? Which is more important to • Basic researchers? • Across • Applied researchers? • To • ? Why? • Give examples of limits to from your project

  4. External validity:concepts • Boundary conditions? • Limitations of generalization • Situations where the theory doesn’t apply • E.g. Newtonian physics in outer space • “Reinforcement theory” when behavior is constrained • ? What are some boundary conditions in your study? • Ecological validity? • Similarity of conditions to the natural setting • Substance abuse in half-way house. v. living at home • Training offsite v. on the job

  5. External Validity(Whitley’s view) • Generalizing Across • “generalizability” • Generalizing to • “ecological validity”

  6. External Validity Components: • Structural • Methods, how its carried out • Setting, procedures, sample • Functional • How it operates; works • Similarity of psych process in study and natural setting • Conceptual • How important in real world • Faking on personality tests used for applicant selection • Marihuana use at work

  7. External Validity:Components (Vidmar, ’79) • Structural • Method • Procedure • Sample • Setting • Example: • Does student participation in GS -> task satisfaction? • Generalize to other students? • To workers in a factory?

  8. External ValidityComponents (con’t) (Vidmar, 79) • Functional (primarily ecological validity) • How close do the psychological processes in study match natural setting? • E.g. mock jury v. real jury • Conceptual (primarily ecological validity) • How close does the problem in study match the natural setting? • E.g. crowding in lab study = crowding in prison?

  9. Structural Components:Generalize findings to other • Settings • Participant samples • Research procedures • Times • Cultures • ? How are the structural components of your study related to external validity?

  10. Structural Component • Settings (Physical & Social) • Physical (e.g. Wickland) • Reactivity (behavior changes when being observed) • Research attributes (of E) • Co-participant attributes (when they interact) • Ecological validity (mundane realism) • What factors in your project could interact with IVs?

  11. Structural Component • Participant sample factors • Convenience sample • availability v. representative • Restricted sampling • E.g. Young, white, lesbian female postal carrier • Volunteer participants • Would you “devolunteer”? • Person by person situation interactions • Personal characteristics may interact with IV • Ecological validity • Are college students representative of…?

  12. Research Procedure Factors • Artificiality • Operational Definitions (use multiple op defs) • Levels of IV (what is the range of values?) • Ecological Validity (Too many college Ss?) • Cultural Factors (Cross cultural factors) • Time Factors • (how many psychotherapy sessions are optimal?)

  13. Functional and Conceptual Componentsverisimilitude = appearance of truth • Functional • Do the psychological processes act the same way in other settings, populations? • E.g. when the stakes are high in jury decisions v. mock juries? • Conceptual • E.g. “sub-clinical” depression in college Ss = form of depression in clinical population? (Kazdin, ’97) • Researchers v. policy makers • Estimator vars cannot be changed (e.g. level of victim anxiety) • Policy vars can (e.g. time between crime and testimony) • Utility of DV • E.g. actual prisoner violence v. perceptions of control (theoretical interest)

  14. Relationships Among Components • Components of structural, functional, conceptual are interrelated • Acute v. chronic manipulations • Related to both functional and conceptual • E.g. are long and short term anxiety the same?

  15. Assessing External Validity • Assessing Generalizability • 1. include generalizability factors • E.g. moderators such as age, gender, race • 2. replicate findings • 3. acknowledge limitations of study • ? What are some of yours? • Assessing Ecological Validity – consider: • Conceptual (policy relevance of IV; utility of DV) • Functional (similar psychological processes?) • Structural (naturalism?)

  16. Lab v. natural Settings & Ecological Validity • Lab research & Ecological Validity • Defending lab research (Purposes are different) • Lab research • Tests causal hypotheses (Berkowitz 7 Donnerstein, ’82) • “ A behavioral phenomenon reliably deonstrated is a genuine phenomenon” • Falsify theoretical porpositions (Higgens & Marlatt, ’73) • Alcoholics don’t drink to reduce tension • Dissect complex phenomena (Haney, Banks, & Zimbardo, ’73) • Role assignment alone can determine behaviors • Discover new phenomena (Henshel, ’80) • Would ecological research have discovered that biofeedback works?

  17. Lab v. Natural Settings (con’t) • Ecological Validity as Empirical Question • Can be tested • (e.g. meta of meta-analysis shows effect size r=.73) • Natural Settings and Generalizability • No guarantee of generalizability for natural setting research • Structural v. Functional & Conceptual Verisimilitude • Ecological focuses on structural, ignores functional/conceptual (Locke, ’86) • Analog Research • Reproducing natural setting in lab • What most of you would like to do with your manipulation • Philosophies of Science and Naturalism in Research • A matter of preference! Basic or Applied? • What’s in your wallet?

  18. External Validity & Internal Valitidy • It’s a trade off, but… • Internal is more important • Must rule out other plausible causes to draw conclustions • E.g. which diet is best? Carbs or Fat? • Who really knows? • ?Which is more important for your project?

  19. External Validity: Summary • Concept of External Validity • Structural Component • Functional & Conceptual Components • Assessing External Validity • Lab Research, Natural Setting & External Validity

More Related