1 / 17

Lecture 6

Lecture 6. Experiments II: Validity and Design Considerations. Example: Trust-Building Study. Question: Do increased risk-taking behaviors over time increase interpersonal trust?. Trust-Building Study . Independent Variable Experiment Condition (2 conditions):

caroun
Download Presentation

Lecture 6

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Lecture 6 Experiments II: Validity and Design Considerations

  2. Example: Trust-Building Study • Question: Do increased risk-taking behaviors over time increase interpersonal trust?

  3. Trust-Building Study • Independent Variable • Experiment Condition (2 conditions): • Fixed partner on every trial, cannot control amount to entrust to partner • Fixed partner on every trial, can control amount to entrust to partner • Assignment • Random assignment of participants to one of the 2 conditions. • Same experiment conducted in Japan and US, and comparisons made between the two studies. • Dependent Variable • Cooperation rate (i.e., whether they returned the coins to the partner or not)

  4. Measurement and Design Validity • Measurement Concerns • Construct Validity • Design Concerns • Internal Validity • External Validity • Ecological Validity

  5. Construct Validity • “Face” validity deals with subjective judgement of appropriate operationalization • “Content” validity is a more direct check against relevant content domain for the given construct. How do we know that our independent variable is reflecting the intended causal construct and nothing else?

  6. Internal Validity Internal Validity deals with questions about whether changes in the dependent variable were caused by the treatment.

  7. ? ? Cause Effect ? ?

  8. Threats to Internal Validity • History • additional I.V. that occurs between pre-test and post-test • Maturation • Subjects simply get older and change during experiment • Testing • Subjects “get used” to being tested • Regression to the Mean • Issue with studies of extremes on some variable

  9. Contamination and Internal Validity • Demand Characteristics • Anything in the experiment that could guide subjects to expected outcome • Experimenter Expectancy • Researcher behavior that guides subjects to expected outcome (self-fulfilling prophecy)

  10. General Demand Characteristics • Evaluation Apprehension • Solutions • Double-blind experiments • Experiments in natural setting (i.e., subjects do not know they are in an experiment) • Cover stories • Hidden measurements

  11. Reducing the role of the experimenter: solving expectancy effects • Naïve experimenter • Those conducting study are not aware of theory or hypotheses in the experiment • Blind • Researcher is unaware of the experiment condition that he/she is administering • Standardization • Experimenter follows a script, and only the script • “Canned” Experimenter • Audio/Video/Print material gives instructions

  12. And More! • Selection Bias • Issue with non-random selection of subjects • Mortality • Departure of subjects in the experiment • Diffusion, Sharing of Treatments • Control group unexpectedly obtains treatment • Other ‘social’ threats? • Compensatory rivalry, resentful demoralization, etc.

  13. Three threats to external validity (generalizability) in experiments • Setting • Population • History External Validity– How far does the given experiment generalize to similar groups, individuals, etc?

  14. Ecological Validity

  15. The Validity Tradeoff: Truth and Myth Internal Validity External & Ecological Validity Balance is important between the types of validity, but internal validity is usually (if not always) the more important factor.

  16. Pro’s and Con’s of Experiments • Pro’s • Gives researcher tight control over independent factors • Allows researcher to test key relationships with as few confounding factors as possible • Allows for direct causal testing • Con’s • Usually a smaller N than surveys • Sometimes give up large amounts of external validity in favor of construct validity and direct causal analysis • Require a large amount of planning, training, and time– sometimes to test relationship between only 2 factors!

  17. Additional considerations before using experiments • Cost and Effort • Is the effort worth it to test the concepts you are interested in? • Manipulation and Control • Will you actually be able to manipulate the key concept(s)? • Importance of Generalizability • Are you testing theory, or trying to establish a real-world test?

More Related