Issues still worth emphasizing…

1. Issues still worth emphasizing… Designs & clarifications

2. Repeated Measures Designs • Repeated measures (RM) variable: • Participants are measured multiple times • E.g.: caffeine variable (X1 = dose level 1, etc): • Non RM: • R X1 O • R X2 O • R X3 O • RM • X1 O X2 O X3 O • Which is stronger in terms of internal validity? • Order effects • Treatment carry-over effects

3. Repeated Measures Designs • Repeated measures (RM) variable: • Take this example: • A) Non RM: • R X1 O • R X2 O • B) RM • X1 O X2 O • A is a multiple group experiment. What is B? • Remember internal validity is about differences between the non-treatment and treatment conditions, which are not necessarily in different groups of people • In B, factors affecting the one group of people concern external validity. Factors affecting measure 1 vs. measure 2 are relevant to internal validity.

4. Repeated Measures Designs • The multiple times constitute the RM • Goal of internal validity – assess the degree to which groups differ on treatment alone. • RM variables can rule out differences due to individual difference • Differences due to time/order must be ruled out (order effects) • Spreading of treatment must be ruled out (for the caffeine variable, has the caffeine left the body before the no caffeine measurement is taken?)

5. Case study/multiple baseline designs here the comparison is between different people, rather than groups Each person is given the treatment at different times Examining the change in the dv to see if it changes if and only if the treatment is introduced is the way of ascertaining cause.

6. Questionnaire designs • Give everyone a questionnaire or two. • Questionnaires measure several variables • Authors subsequently study relationships between these variables • All measures taken at same time • No temporal precedence • Direction of causation open to interpretation

7. Questionnaire designs • Coincidence of high responses in one variable with high/low responses in other variables are studied for possible causal inference • Alternative interpretations? • Any other variables that might co-vary with the 2 variables being considered are candidates • No alternative interpretations are ruled out unless these variables are also measured and “controlled for” • E.G. • Churches and crime rate • # of fire trucks and damage done by fire

8. “intact group” designs • Where an investigator assigns an entire class to a condition, and an entire other class to a different condition • Consider all the things that could have caused each group to be “intact” • Can any of these systematic biases be proposed as alternate causes?

9. General mistakes on classification • Into construct section: • Anything pertaining to the idea of and the operationalization of the study variables • Generalization of operationalization back to the idea of the variable • Into External validity section: • Generalization of anything else • Interaction of different settings and relationship • Interaction of different times and relationship • Interaction of different samples and relationship • Random selection

10. General mistakes on classification • Into Internal validity section • design and causality only • Random assignment • Internal validity is a zero-generalizability concern.

11. Other comments • Critique = bad andgood (& overall evaluation of quality) • In construct section, evaluate each variable separately • An unknown factor is not a weakness, it’s an unknown factor (use conditional arguments) • And: • Follow the structure of the rubric (!), but write in full sentences/paragraphs. Please don’t use the numbering from the rubric. This is a nightmare to follow! • Evaluate plausibility of arguments (don’t make poorly reasoned claims) • Be specific to the relationship under investigation • “results could change if other people were used” • Provide reasons • What results, how would they change, and why.