1 / 32

EXPERIMENTS AND EXPERIMENTAL DESIGN

EXPERIMENTS AND EXPERIMENTAL DESIGN. ETHICAL ISSUES IN RESEARCH. Logic of causality. If X, then Y and If not X, then not Y Or, colloquially: If the stimulus is given, then the outcome occurs and If the stimulus is not given, then the outcome does not occur.

nicholsj
Download Presentation

EXPERIMENTS AND EXPERIMENTAL DESIGN

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. EXPERIMENTS AND EXPERIMENTAL DESIGN ETHICAL ISSUES IN RESEARCH

  2. Logic of causality If X, then Y • and If not X, then not Y Or, colloquially: If the stimulus is given, then the outcome occurs • and If the stimulus is not given, then the outcome does not occur

  3. Basic requirements of a true experiment: - At least 2 groups: one experimental & one control group - Random assignment of subjects to groups. Every case has: (1) the same chance of being selected (i.e. equal probability) (2) the chance of selection does not change (i.e. constant probability) - Manipulation of the IV. - Measurement of the DV. - Except for the experimental manipulation, the 2 groups = treated exactly alike

  4. Results • controlling for pre-existing subject differences; differences are to be neutralized/distributed approximately evenly among the groups random assignment of subjects to treatment & control groups - making sure that the events occurring within each experimental group are exactly the same, except for the manipulated independent variable  controlling for extraneous factors & experiences during the experiment

  5. Randomization for groups

  6. Tests • Use tests of statistical significance to determine the likelihood that the results occurred by chance rather than being caused by the treatment variable (i.e. significant differences btw. the groups). • A result is statistically significant at the 0.05 level: it means that only 5% of the time (5 times in 100 if we were to repeat the experiment) would differences btw. the experimental & control groups occur by chance when the experimental variable actually has no effect.

  7. Tests • Differences between the mean values for T and C • The differences between slopes for T and C

  8. Mean values for T and C

  9. Slope

  10. Staging of an experiment  1. Introduction: Instructions and Cover Story 2. Manipulation of the IV 3. Measurement of the DV (verbal reports; behavioral observation) 4. Debriefing (explain purpose of experiment; explore subjects’ reactions; alleviate subjects’ anxiety about their performance/ participation)

  11. Validity • Internal Validity: the approximate truth about inferences regarding the tested causal relationships in the experimental setting • External Validity: the degree to which the conclusions in the study would hold for other persons in other places & at other times (i.e. how generalizable are the results of the study)

  12. Issues to consider • Pre-testing of experiments • Laboratory experiments vs. experiments in natural settings (field experiments) • Trade-off btw. increase in external validity & control over extraneous variables & manipulation of the IV • Subjects’ reaction to the social meaning of the situation: often a problem for laboratory experiments

  13. Types of experiments • Pre-test – post-test control group design • Post-test-only control group design • The Solomon four group design • Factorial design • Quasi-experimental design • Experimental Design in Survey

  14. Pre-test – post-test control group design Tt1 X Tt2 R Ct1 -- Ct2 R = random assignment of subjects to groups T = Treatment group,C = Control group t1 = first measurement, before experimental treatment (pretest), t2 = 2nd measurement, after experimental treatment (post-test)

  15. Post-test-only control group design X Tt1 R -- Ct1 Many variations on this experimental design: See www.socialresearchmethods.net - various experimental designs.

  16. The Solomon four group design Tat1 X Tat2 Cat1 -- Cat2 R X Tbt1 -- Cbt1

  17. Outcome I

  18. Outcome II

  19. Source: www.socialresearchmethods.netFactorial design

  20. The Main Effects and Interaction Effects • A main effect is an outcome that is a consistent difference between levels of a factor. • Because of the way we combine levels in factorial design, it also enables us to examine the interaction effects that exist between factors. An interaction effect exists when differences on one factor depend on the level of another factor.

  21. Explanation of effects, I

  22. Explanation of effects, II

  23. Explanation of effects, III

  24. Explanation of effects, IV

  25. Explanation of effects, V

  26. Quasi-experimental design (QED) QED takes an experimental approach without having full experimental control; lacks some feature of true experiments, generally randomization • - Separate sample pretest-posttest design (both groups receive the treatment, but one is tested before the treatment, and the other afterward) • - None-equivalent control group design; control group but without random assignment; • - Interrupted time series-design: series of observations before and after the treatment; • - Multiple time series design: extends the one above by adding a series of observations on a non-equivalent control group.

  27. Experimental Design in Surveys • Differently worded questions: As part of a larger survey: direct a question with one wording to a randomly selected sub-sample of respondents; and a differently worded question to the remaining respondents. • Vignettes: Short, detailed and concrete descriptions of situations that contain references to the factors that are thought to be important in decision making. To test experimentally which factors affect the subject’s decision making, a number of versions of the vignettes are developed by systematically varying the details of the situation. The different versions are then randomly assigned to respondents. • Constructing different sets of questionnaires

  28. ETHICAL ISSUES IN RESEARCH Basic question: For what to to whoe the researchers in the social sciences are resposible? Human subjests Peers Society

  29. Ethics of treatment of human subjects • - Ethical considerations preclude withholding treatment from a control group: education, correction, clinical psychology, medicine frequently the case. • Institutional review boards assess the ethics of the proposed research (including on risk vs. benefits). Overall guiding principle: potential benefits of research should be weighed against potential costs (risks).

  30. General rules: • - Researcher should not expose participants to substantial risk of physical or psychological harm, unless the benefits of participation exceed the risks, and the subject knowingly chooses to participate; • - Subjects should be informed that their participation is voluntary throughout the entire research; should be told about any aspects of the research that may influence their willingness to participate; • - If deception is necessary- appropriate debriefing as soon as possible after their participation; • - Confidentiality and anonymity: researchers should use all possible means to protect the confidentiality of information participants provide (identity).

  31. Ethics of data collection and analysis • - Honesty and accuracy in collection & analysis of data, and reporting of results • - Scientific misconduct, if identified, kills researcher’s career

  32. Ethics of responsibility to society • Awareness of potential use and misuse of scientific results by others

More Related