Basic Logic of Experimentation

1. Basic Logic of Experimentation • The design of an Internally valid experimental procedure requires us to: • Form Equivalent Groups • Treat Groups Identically in All Respects Except for the IV • Assess the Significance of Differences in DV Scores Between Conditions • A conclusion is based on the logic that • if the groups did not pre-experimentally differ with respect to the DV • then the groups were treated identically in all ways except for one (the IV) • and now the DV scores for the groups do significantly differ • There is only one way to account for the differences in the DV scores, they must be due to the influence of the IV

2. Matching Matching involves initially premeasuring subjects on some attribute (believed to be strongly correlated with the DV) and assigning subjects to conditions based on these scores. • How to match: • Collect data on the matching variable (MV) • Rank order subjects according to MV scores • Segregate subjects into matched pairs (or matched trios, or matched quartettes, etc.) • Randomly assign one pair member to a condition of the experiment • The assignment of the second member of the pair is dictated

3. Matching • Matching involves initially premeasuring subjects on some attribute (believed to be strongly correlated with the DV) and assigning subjects to conditions based on these scores. • Matching always produces groups which are more equivalent than random assignment. But how much more? That depends on the degree of correlation between MV and DV. • If the MV is not correlated with the DV, matching will produce no effect on equivalence. • How to choose a matching variable: • Consult earlier published studies – there may be reported observed correlations between your DV and possible MV’s • Conduct pilot research to determine which possible MV correlate best with your DV • Choose a MV on logical grounds. (Example: if your DV involves complex problem solving, its logical to assume that performance is correlated with intelligence).

4. Matching Since matching always produces groups which are more equivalent than random assignment, why not always use matching? Only Disadvantage of Matching – Extra work and time of conducting the matching procedure prior to commencement of actual experimentation (plus the increased possibility of mortality with the greater demands being placed on the subject). Ultimately, the decision to use matching hinges on whether the advantages to be gained justify the time and effort required to do the matching. Examples: A common practice in early psychological experimentation was to routinely match on the basis of intelligence under the belief that intelligence is related to everything we do. (Abandoned as a general practice). Using rats in an instrumental task where body weight reduction is the definition of the motivating factor – deprivation level. Matching on pre-deprivation body weight is a simple easy practice which is therefore routinely done.

5. More Advanced Matching Matching always produces groups which are more equivalent than random assignment. (Match by correlated criterion design) Even greater equivalence can be attained by matching on multiple variables simultaneously. (Match by correlated criteria) You can match on the basis of one criterion, two criteria, three criteria,… But would it ever be possible to match on the basis of all criteria? The ultimate match in correlated criteria would be the case of identical twins (the so called “Co-Twin Design”) Who would be the ultimate match for you? The Within-Subject design is in essence the ultimate matched groups design. The same exact people are used in all conditions of the experiment. Not exactly – due to progressive error. If someone participates in two conditions they have to participate in one of them first. This changes the person in some way so they are not exactly the same person in the second condition that they were before the first condition. And, of course, if they participate in a third condition they are changed even more by their previous participation in the first two conditions.

6. Independent Conditions vs Dependent Conditions Independent Conditions are called that because the group compositions are arrived at independent of each other. Who is assigned to one condition is in no way determined by the assignment of subjects to the other condition. There is no systematic relationship between the subjects in one group vs the other. (E.G., Random Assignment) Dependent Conditions are formed when there is some dependence or relationship between those assigned to one condition vs another. There are three situations where dependent conditions are formed: Matched-groups designs Within-subject designs Yoked-control designs Matching done on variables that are organismic Matching done on stimulus variables It is essential that the distinction between independent condition and dependent conditions be recognized as it dictates the appropriate statistical test for the analysis of differences between conditions.

7. Yoked-Control Group Procedure in which the environmental conditions affecting a control subject are manipulated so that they exactly correspond to those affecting an experimental subject

8. Designs which Create “Independent Groups” Designs which Create “Dependent Groups” • Matched-Groups Designs • Co-Twin Investigations • Within-Subjects Designs • Yoked-Control Designs • Randomized Groups Designs • Any Between-Subjects Design Lacking a Logical Connection Between Subjects in its Conditions Appropriate Analysis (two-groups design) T-test for Dependent Means T-test for Independent Means