1 / 18

Design (2): Experimental designs

Explore the characteristics and types of experimental designs to establish cause-effect relationships between variables. Learn about manipulation checks, setting types, and different designs like treatment-control and factorial designs.

chuey
Download Presentation

Design (2): Experimental designs

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Design (2): Experimental designs Learning outcomes • State the characteristics of an experiment • State and define different types of experimental design

  2. Outline • What is an experiment? • Elements of experimental design • Setting • Experimental manipulation • Some elementary designs

  3. Experiment • Defining characteristics: • Establish cause-effect relationship between IV(s) and DV as stated in experimental hypothesis • Manipulation of IV(s) • Random assignment of participants to levels (categories) of the IV(s) • If no random assignment then quasi-experiment • If neither manipulation nor random assignment then non-experiment

  4. Setting • Laboratory experiment • Setting especially created for the study • Potentially high internal validity, but at the expense of external validity • Experimental realism: an experimental setting in which participants become involved, e.g. computer games and usability tests (http://www.useit.com/alertbox/20050214.html) • Field experiment – existing setting • Potentially, but not necessarily, high external validity • Mundane realism: research setting that is likely to occur in the normal course of participants’ lives, i.e. in the ‘real world’; however, this is NOT a guarantee for importance of events to or involvement of participants • ‘[The] purpose [of an experiment] is to remake the outside world so that it duplicates as closely as possible the experimental world’ (Henshel)

  5. Manipulation checks • Check whether the manipulation of the IV had its intended effects • Example: • IV frustration; DV aggression • Check that the manipulation of frustration is effective, that an allegedly frustration-inducing treatment - e.g. a frustrating task - increases the level of frustration • Possible unintended effect: the measurement of the effectiveness of the IV (the check) may affect the DV indirectly through other variables, including manipulation check • Therefore, the effectiveness of the manipulation of should be checked separately in advance, in a pilot study, in order to avoid effects of the check on the DV

  6. Independent measures versus repeated measures • Independent measures/between groups/independent groups/unrelated • Separate groups for each of the different experimental conditions for one ore more IVs • Each participant tested once • Repeated measures/within subjects/related • Each participant exposed to all the conditions for one or more IVs • Each participant tested several times • Mixed designs • A combination of independent-measures IVs and repeated-measures IVs

  7. Independent measures • Make sure there as few differences between the groups as possible, e.g. by randomization or matching • Advantages • Simplicity – allocate each participant to one condition • Less chance of practice and fatigue effects • Useful if it is impossible for each participant to take part in all experimental conditions • Disadvantages • Expense in terms of time, effort and participant numbers • Insensitivity to experimental manipulations as a result of individual differences • Some examples of independent-measures designs follow

  8. 1 Treatment-control. Post-measure only • Control group: unethical to have (withholding treatment) or unethical NOT to have (no comparison)? • Validity: • Major threats to internal validity controlled by having a control group • Because no pre-measure is taken, no sensitisation by pre-testing and no interaction of pre-testing with treatments • Other aspects of validity depend on specific details of the design • Analysis: unrelated t test (interval DV), Mann-Whitney U test (ordinal DV) or simple regression (nominal IV, interval DV)

  9. 2 Categorical or continuous IV. Post-measure only • Design 1 is a special case of Design 2 • Analysis: one-way independent measures analysis of variance (interval DV), Kruskal-Wallis test (ordinal DV) or multiple regression (nominal IV, interval DV)

  10. 3 Treatment-control. Pre-measures and post-measures • Validity: • Internal validity as for Design 1 • Because pre-measure is taken, possible sensitisation by pre-testing and possible interaction of pre-testing with treatments • Analysis: analysis of co-variance; analysis of change scores is NOT appropriate

  11. 4 Treatment and concomitant variables • Concomitant variable (CV): related to the DV; participants’ attributes relevant to outcome measure • Purpose: control for CV (‘individual differences’ variable) and thereby reduce error term in analysis • Design 3 is a special case of Design 4 • The CV should not be influenced by treatment and should be measured before treatment is administered • Analysis: analysis of co-variance

  12. 5 Factorial designs • At least one IV needs to be manipulated and participants need to be randomly assigned to treatments • Aim for equal numbers of participants in the non-manipulated IV (if there is one) when assigning levels of the IV to treatments, e.g. gender (‘stratified randomization’) • Analysis: factorial analysis of variance, e.g. two-way independent measures analysis of variance (ANOVA) • ‘Main’ effect of IV1 (ignoring the effect of IV2) • ‘Main’ effect of IV2 (ignoring the effect of IV1) • ‘Interaction’ effect (is the effect of IV1 the same across all levels of IV2?)

  13. 6 Solomon four-group design • Two IVs: pre-measure (pre-test or not) and treatment (treatment given or not) • Can establish whether pre-measure interacts with treatment • More than one analysis required

  14. 7 Attribute-treatment interaction • Attribute: participants’ characteristic that may moderate the effect of treatment • Equivalent to Design 4, but with different purpose • Purpose: establish if an individual difference IV (attribute, A) interacts with treatment • Analysis: analysis of co-variance

  15. Repeated measures • Each participant acts as their own control • Advantages • Economy • Sensitivity • Disadvantages • Order effects and carry-over effects from one condition to another; solution: counterbalancing, e.g. using Latin squares • The need for conditions to be reversible • Some examples of repeated-measures designs follow

  16. Some repeated measures designs • Repeated measures, post-test only, two conditions; analysis: related t test (interval DV) or Wilcoxon test (ordinal DV) • Repeated measures, post-test only, more than two conditions; analysis: one-way RM ANOVA (interval DV) or Friedman test (ordinal DV) • Factorial repeated measures; analysis: e.g. two-way RM ANOVA • Two-factor mixed design; this is a design in its own right, but is sometimes called repeated measures, e.g. in SPSS; analysis: two-way mixed ANOVA

  17. Preparation for next practical class • Study key experimental research designs • Reading: Pedhazur: Ch. 12 • Clark-Carter: Ch. 1 • Lecture notes

  18. Summary • Experimental designs differ from quasi-experimental and non-experimental designs • Experiments can be conducted in different types of setting • Manipulation checks are important and should be conducted in a pilot study • Various elementary designs differ in their validity and data analysis

More Related