RESEARCH STATISTICS

1. Analysis of Variance RESEARCH STATISTICS Jobayer Hossain Larry Holmes, Jr, November 20, 2008

2. Experimental Design Terminology • An Experimental Unit is the entity on which measurement or an observation is made. For example, subjects are experimental units in most clinical studies. • Homogeneous Experimental Units: Units that are as uniform as possible on all characteristics that could affect the response. • A Block is a group of homogeneous experimental units. For example, if an investigator had reason to believe that age might be a significant factor in the effect of a given medication, he might choose to first divide the experimental subjects into age groups, such as under 30 years old, 30-60 years old, and over 60 years old.

3. Experimental Design Terminology • A Factor is a controllable independent variable that is being investigated to determine its effect on a response. E.g. treatment group is a factor. • Factors can be fixed or random • Fixed -- the factor can take on a discrete number of values and these are the only values of interest. • Random -- the factor can take on a wide range of values and one wants to generalize from specific values to all possible values. • Each specific value of a factor is called a level.

4. Experimental Design Terminology • A covariate is an independent variable not manipulated by the experimenter but still affecting the response. E.g. in many clinical experiments, the demographic variables such as race, gender, age may influence the response variable significantly even though these are not the variables of interest of the study. These variables are termed as covariate. • Effect is the change in the average response between two factor levels. That is, factor effect = average response at one level – average response at a second level.

5. Experimental Design Terminology • Interaction is the joint factor effects in which the effect of one factor depends on the levels of the other factors. No interaction effect of factor A and B Interaction effect of factor A and B

6. Experimental Design Terminology • Randomization is the process of assigning experimental units randomly to different experimental groups. • It is the most reliable method of creating homogeneous treatment groups, without involving potential biases or judgments.

7. Experimental Design Terminology • A Replication is the repetition of an entire experiment or portion of an experiment under two or more sets of conditions. • Although randomization helps to insure that treatment groups are as similar as possible, the results of a single experiment, applied to a small number of experimental units, will not impress or convince anyone of the effectiveness of the treatment. • To establish the significance of an experimental result, replication, the repetition of an experiment on a large group of subjects, is required. • If a treatment is truly effective, the average effect of replicated experimental units will reflect it. • If it is not effective, then the few members of the experimental units who may have reacted to the treatment will be negated by the large numbers of subjects who were unaffected by it. • Replication reduces variability in experimental results and increases the significance and the confidence level with which a researcher can draw conclusions about an experimental factor.

8. Experimental Design Terminology • A Design (layout) of the experiment includes the choice of factors and factor-levels, number of replications, blocking, randomization, and the assignment of factor –level combination to experimental units. • Sum of Squares (SS): Let x1, …, xn be n observations. The sum of squares of these n observations can be written as x12 + x22 +…. xn2. In notations, ∑xi2. In a corrected form this sum of squares can be written as (xi -x )2. • Degrees of freedom (df): Number of quantities of the form – Number of restrictions. For example, in the following SS, we need n quantities of the form (xi -x ). There is one constraint (xi -x ) = 0. So the df for this SS is n – 1. • Mean Sum of Squares (MSS): The SS divided by it’s df.

9. Experimental Design Terminology • The analysis of variance (ANOVA) is a technique of decomposing the total variability of a response variable into: • Variability due to the experimental factor(s) and… • Variability due to error (i.e., factors that are not accounted for in the experimental design). • The basic purpose of ANOVA is to test the equality of several means. • A fixed effect model includes only fixed factors in the model. • A random effect model includes only random factors in the model. • A mixed effect model includes both fixed and random factors in the model.

10. The basic ANOVA situation • Two type of variables: Quantitative response • Categorical predictors (factors), • Main Question: Do the (means of) the quantitative variable depend on which group (given by categorical variable) the individual is in? • If there is only one categorical variable with only 2 levels (groups): • 2-sample t-test • ANOVA allows for 3 or more groups

11. One-way analysis of Variance • One factor of k levels or groups. E.g., 3 treatment groups in a drug study. • The main objective is to examine the equality of means of different groups. • Total variation of observations (SST) can be split in two components: variation between groups (SSG) and variation within groups (SSE). • Variation between groups is due to the difference in different groups. E.g. different treatment groups or different doses of the same treatment. • Variation within groups is the inherent variation among the observations within each group. • Completely randomized design (CRD) is an example of one-way analysis of variance.

12. One-way analysis of variance Consider a layout of a study with 16 subjects that intended to compare 4 treatment groups (G1-G4). Each group contains four subjects. S1 S2 S3 S4 G1 Y11 Y12 Y13 Y14 G2 Y21 Y22 Y23 Y24 G3 Y31 Y32 Y33 Y34 G4 Y41 Y42 Y43 Y44

13. One-way Analysis: Informal Investigation • Graphical investigation: • side-by-side box plots • multiple histograms • Whether the differences between the groups are significant depends on • the difference in the means • the standard deviations of each group • the sample sizes • ANOVA determines P-value from the F statistic

14. One-way Analysis: Side by Side Boxplots

15. One-way analysis of Variance • Model: • Assumptions: • Observations yij are independent. • eij are normally distributed with mean zero and constant standard deviation. • The second (above) assumption implies that response variable for each group is normal (Check using normal quantile plot or histogram or any test for normality) and standard deviations for all group are equal (rule of thumb: ratio of largest to smallest are approximately 2:1).

16. One-way analysis of Variance • Hypothesis: Ho: Means of all groups are equal. Ha: At least one of them is not equal to other. • doesn’t say how or which ones differ. • Can follow up with “multiple comparisons” • Analysis of variance (ANOVA) Table for one way classified data A large F is evidence against H0, since it indicates that there is more difference between groups than within groups.

17. Pooled estimate for variance The pooled variance of all groups can be estimated as the weighted average of the variance of each group: so MSE is the pooled estimate of variance

18. Multiple comparisons • If the F test is significant in ANOVA table, then we intend to find the pairs of groups are significantly different. Following are the commonly used procedures: • Fisher’s Least Significant Difference (LSD) • Tukey’s HSD method • Bonferroni’s method • Scheffe’s method • Dunn’s multiple-comparison procedure • Dunnett’s Procedure

19. One-way ANOVA - Demo • MS Excel: • Put response data (hgt) for each groups (grp) in side by side columns (see next slides) • Select Tools/Data Analysis and select Anova: Single Factor from the Analysis Tools list. Click OK. • Select Input Range (for our example a1: c21), mark on Group by columns and again mark labels in first row. • Select output range and then click on ok.

20. One-way ANOVA MS-Excel Data Layout

21. One-way ANOVA MS-Excel output: height on treatment groups

22. One-way ANOVA - Demo • SPSS: • Select Analyze > Compare Means > One –Way ANOVA • Select variables as Dependent List: response (hgt), and Factor: Group (grp) and then make selections as follows-click on Post Hoc and select Multiple comparisons (LSD, Tukey, Bonferroni, or Scheffe), click options and select Homogeneity of variance test, click continue and then Ok.

23. One-way ANOVA SPSS output: height on treatment groups

24. Analysis of variance of factorial experiment (Two or more factors) • Factorial experiment: • The effects of the two or more factors including their interactions are investigated simultaneously. • For example, consider two factors A and B. Then total variation of the response will be split into variation for A, variation for B, variation for their interaction AB, and variation due to error.

25. Analysis of variance of factorial experiment (Two or more factors) • Model with two factors (A, B) and their interactions: • Assumptions: The same as in One-way ANOVA.

26. Analysis of variance of factorial experiment (Two or more factors) • Null Hypotheses: • Hoa: Means of all groups of the factor A are equal. • Hob: Means of all groups of the factor B are equal. • Hoab:(αβ)ij = 0, i. e. two factors A and B are independent

27. Analysis of variance of factorial experiment (Two or more factors) • ANOVA for two factors A and B with their interaction AB.

28. Two-factor with replication - Demo • MS Excel: • Put response data for two factors like in a lay out like in the next page. • Select Tools/Data Analysis and select Anova: Two Factor with replication from the Analysis Tools list. Click OK. • Select Input Range and input the rows per sample: Number of replications (excel needs equal replications for every levels). Replication is 2 for the data in the next page. • Select output range and then click on ok.

29. Two-factor ANOVA MS-Excel Data Layout

30. Two-factor ANOVA MS-Excel output: height on treatment group, shades, and their interaction

31. Two-factor ANOVA - Demo • SPSS: • Select Analyze > General Linear Model > Univariate • Make selection of variables e.g. Dependent varaiable: response (hgt), and Fixed Factor: grp and shades. • Make other selections as follows-click on Post Hoc and select Multiple comparisons (LSD, Tukey, Bonferroni, or Scheffe), click options and select Homogeneity of variance test, click continue and then Ok.

32. Two-factor ANOVA SPSS output: height on treatment group, shades, and their interaction

33. Repeated Measures • The term repeated measures refers to data sets with multiple measurements of a response variable on the same experimental unit or subject. • In repeated measurements designs, we are often concerned with two types of variability: • Between-subjects - Variability associated with different groups of subjects who are treated differently (equivalent to between groups effects in one-way ANOVA) • Within-subjects - Variability associated with measurements made on an individual subject.

34. Repeated Measures • Examples of Repeated Measures designs: • Two groups of subjects treated with different drugs for whom responses are measured at six-hour increments for 24 hours. Here, DRUG treatment is the between-subjects factor and TIME is the within-subjects factor. • Students in three different statistics classes (taught by different instructors) are given a test with five problems and scores on each problem are recorded separately. Here CLASS is a between-subjects factor and PROBLEM is a within-subjects factor. • Volunteer subjects from a linguistics class each listen to 12 sentences produced by 3 text to speech synthesis systems and rate the naturalness of each sentence. This is a completely within-subjects design with factors SENTENCE (1-12) and SYNTHESIZER

35. Repeated Measures • When measures are made over time as in example A we want to assess: • how the dependent measure changes over time independent of treatment (i.e. the main effectof time) • how treatments differ independent of time (i.e., the main effect of treatment) • how treatment effects differ at different times (i.e. the treatment by time interaction). • Repeated measures require special treatment because: • Observations made on the same subject are not independent of each other. • Adjacent observations in time are likely to be more correlated than non-adjacent observations

36. Repeated Measures Response Time

37. Repeated Measures • Methods of repeated measures ANOVA • Univariate - Uses a single outcome measure. • Multivariate - Uses multiple outcome measures. • Mixed Model Analysis - One or more factors (other than subject) are random effects. • We will discuss only univariate approach

38. Repeated Measures • Assumptions: • Subjects are independent. • The repeated observations for each subject follows a multivariate normal distribution • The correlation between any pair of within subjects levels are equal. This assumption is known as sphericity.

39. Repeated Measures • Test for Sphericity: • Mauchley’s test • Violation of sphericity assumption leads to inflated F statistics and hence inflated type I error. • Three common corrections for violation of sphericity: • Greenhouse-Geisser correction • Huynh-Feldt correction • Lower Bound correction • All these three methods adjust the degrees of freedom using a correction factor called Epsilon. • Epsilon lies between 1/k-1 to 1, where k is the number of levels in the within subject factor.

40. Repeated Measures - SPSS Demo • Analyze > General Linear model > Repeated Measures • Within-Subject Factor Name: e.g. time, Number of Levels: number of measures of the factors, e.g. we have two measurements PLUC.pre and PLUC.post. So, number of level is 2 for our example. > Click on Add • Click on Define and select Within-Subjects Variables (time): e.g. PLUC.pre(1) and PLUC.pre(2) • Select Between-Subjects Factor(s): e.g. grp

41. Repeated Measures ANOVA SPSS output:

42. Repeated measures ANOVA SPSS output

43. Questions