Chapter 21

1. Chapter 21 Research Design Applications: Randomized Groups and Correlated Groups

2. Simple Randomized Subjects Design • The general design paradigm (designated as Design 20.1) is shown in p.502. • Research example • Dolinski and Nawrat (1998): Fear-then-Relief and Compliance. • They claim that compliance is due to the reduction of fear and not the fear itself. A one-way analysis of variance is used for the study. Table 21.1

3. Factorial Designs • Factorial design is the structure of research in which two or more independent variables are juxtaposed in order to study their independent and interactive effects on a dependent variable. • The simplest factorial design, the 2*2, has three possibilities: both A and B active; A active, B attribute (or vice versa); and both A and B attribute. (The last possibilities, both independent variables attributes, is the nonexperimental case. As indicated earlier, however, it is probably not appropriate to use analysis of variance with nonexperimental independent variables.)

4. Factorial Designs with More than Two Variables • Each cell must have at least two participants, and preferably many more. (It is possible, but not very sensible, to have only one participant per cell if one can have more. Of course, there are designs that have only one participant per cell. This is covered in Chapter 22.)

5. Research Examples of Factorial Designs • Sigall and Ostrove (1975): Attractiveness and Crime. • They asked the question: Ho is the physical attractiveness of a criminal defendant related to juridic sentences, and does the nature of the crime interact with attractiveness? • Table 21.2

6. Research Examples of Factorial Designs • Hoyt (1955): Teacher Knowledge and Pupil Achievement. • The research question was: What are the effects on the achievement and attitudes of pupils if teachers are given knowledge of the characteristics of their pupils? • A 3*3*2*2 factorial design with variables Treatment, Ability, Sex, and School. Figure 21.1 and Table 21.3.

7. Evaluation of Randomized Subjects Designs • Randomized subjects designs are all variants or extensions of Design 20.1, the basic experimental group-control group design in which participants are assigned to the experimental and control groups at random. • As such they have the strengths of the basic design, the most important of which is the randomization feature, and the consequent ability to assume the preexperimental approximate equality of the experimental groups in all possible independent variables.

8. Evaluation of Randomized Subjects Designs • Two other strengths of these designs, springing from the many variations possible, are flexibility and applicability. These can be used to help solve many behavioral research problems, since they seem to be peculiarly well suited to the types of design problems that arise from social scientific and educational problems and hypotheses.

9. Evaluation of Randomized Subjects Designs • There are also weakness. One criticism has been that randomized subjects designs do no permit tests of the equality of group, as do before-after (pretest-post test) designs. Actually, this is not a valid criticism for two reasons: (1) with enough participants and randomization, it can be assumed that the groups are equal, as we have seen; an (2) it is possible to check the groups for equality on variables other than Y, the dependent variable.

10. Evaluation of Randomized Subjects Designs • Another weakness is statistical. One should have equal numbers of cases in the cells of factorial designs. It is possible to work with unequal ns, but it is both clumsy and a threat to interpretation. Multiple regression is a better solution for this problem.

11. Evaluation of Randomized Subjects Designs • Compared to matched groups designs, randomized subjects designs are usually less precise; that is, the error term is ordinarily larger, other things being equal.

12. Correlated Groups • A basic principle is behind all correlated groups designs: there is systematic variance in the dependent variable measures due to the correlation between the groups on some variable related to the dependent variable. • This correlation and its concomitant variance can be introduced into the measures, and the design, in three ways:

13. Correlated Groups • 1. use the same units, for example, participants, in each of the experimental groups, • 2. match units on one or more independent variables that are related to the dependent variable, and • 3. use more than one group of units, like classes or schools, in the design.

14. The General Paradigm • The word group should be taken to indicate set of scores. Then there is no confusion when repeated trials experiment is classified as a multigroup design. • Figure 21.2 • It can be seen that there are two sources of systematic variance: that due to column, or treatments, and that due to rows (individual or unit differences).

15. Units • The word unit is deliberately used to emphasize that units can be persons or participants, classes, schools, districts, cities, even nations. • In Figure 21.3. On the left is a factorial design and on the right a correlated groups design, but they look the same! They are the same, in variance principle. (The only differences might be numbers of scores in the cells and statistical treatment.)

16. One Group, Repeated Trials Design • It was said earlier that the best possible matching of participants is to match the participant with himself or herself. One of the difficulties using this solution resembles pretest sensitization, which may produce an interaction between the pretest and the experimentally manipulated variable. Another is that participants mature and learn over time.

17. One Group, Repeated Trials Design • The problem of how individuals learn, or become unduly sensitized during an experiment, is difficult to solve. In short, history, maturation, and sensitization are possible weakness of repeated trials. The regression effect can also be a weakness. • The simplest case of this kind of designs is one group, Before-After design, Design 19.2 (a).

18. Two Groups, Experimental Group-Control Group Designs • This design is described as Design 20.2. • The most common variant of the group, experimental group-control group design is the Before-After, two group design [see Design 20.3 (b)]

19. Research Examples of Correlated Group Designs • Miller and DiCara: Learning of Autonomic Functions • Table 21.4 • The research design is a variant of Design 20.3 (a) • The difference is that ~X, which in the design means absence of experimental treatment for the control group, now means reward for decrease of urine secretion. The usual analysis of the after-training measures of the two group is therefore altered.

20. Research Examples of Correlated Group Designs • Tipper, Eissengberg, and Weaver (1992): Effects of Practice on Selective Attention • A completely within-subjects design. All of the participants experienced all of the treatment conditions.

21. Multigroup Correlated Groups Designs • Units Variance • Until recently, the variances due to differences between classes, schools, school systems, and other “natural” units have not been well controlled or often used in the analysis of data. • The educational investigator has to be alert to these unit differences, as well as to individual differences.

22. Multigroup Correlated Groups Designs • Factorial Correlated Groups • Figure 21.5 • Suedfeld and Rank (1976): Revolutionary and Conceptual Complexity • Table 21.5

23. Multigroup Correlated Groups Designs • Perrine, Lisle, and Tucker (1995): Offer of Help and Willingness to Seek Support • The design was a 3*2*2 factorial design. It contained one manipulated (active) independent variable, one measured (attribute) independent variable and one within-subjects (correlated) independent variable. Figure 21.6 • It usually referred to as mixed ANOVA when at least one independent variable is between-subjects and at least one other is within-subjects.

24. Analysis of Covariance • Analysis of covariance is a form of analysis of variance that tests the significance of the differences among means of experimental groups after taking into account initial differences among the groups, and the correlation of the initial measures and the dependent variable measures. The measure used as a control variable—the pretest or pertinent variable—is called a covariate.

25. Analysis of Covariance • Clark and Walberg (1968): Massive Reinforcement and Reading Achievement • Table 21.7

26. Research Design and Analysis: Concluding Remarks • Four major objectives have dominated the organization and preparation of Part Six. • 1. to acquaint the student with the principal designs of research. • 2. to convey a sense of the balanced structure of good research designs, to develop sensitive feeling for the architecture of design. • 3. to help the reader understand the logic of experimental inquiry and the logic of the various design. • 4. to help the student understand the relation between the research design and statistics.