Lecture 10

1. Lecture 10 • Review Rank Sum test (Chapter 4.2) • Welch t-test for comparing two normal populations with unequal spreads (Chapter 4.3.2) • Practical and statistical significance (Chapter 4.5.1)

2. Rank Sum Test Review • Let F and G be the two distributions. The rank sum test is a test of vs. . Validity depends only on having independent samples from two populations. • Useful when sample sizes are small (<30) and populations appear nonnormal. • Implementation: Compute T=sum of ranks in Group 1. Compute where mean(T) and SD(T) calculated under H0. p-value = 2*Prob(Z>|z|) where Z = standard normal r.v.

3. Example • Study of role of vitamin C in schizophrenia. • Twenty schizophrenic patients and 15 controls with a diagnosis of neurosis of different origin were selected for study. • Dose of vitamin C was given to schizophrenic patients and controls. Total amount of urinary vitamin C excreted during a six hour period was measured. • Data in schizovitaminc.JMP.

5. Meaning of null hypothesis in rank sum test • The rank sum test is most useful if the following condition holds: • Condition: Two population distributions F and G are the same, except that they are shifted by a constant so that G is higher than F. • If condition holds, the rank sum test is a test of whether or not two populations have same center (and are equal). • If condition does not hold, then the null hypothesis of the rank sum test might not be true (and hence will be rejected in large samples) even if populations have same center.

6. Cognitive Load in Teaching • Case Study 4.1.2 • A randomized experiment was done to compare (i) a conventional approach to teaching coordinate geometry in which presentation is split into diagram, text and algebra with (ii) a modified approach in which algebraic manipulations and explanations are presented as part of the graphical display. Students’ performance on a test was compared after being taught by two methods. • Both distributions are highly skewed. In addition, there were five students who did not come to any solution in the five minutes allotted so that their solution times are censored (all that is known about them is that they exceed 300 seconds).

8. Welch t-test for comparing normal pops. with unequal spread • t-test assumes populations have equal standard deviations. Rule of thumb: t-test remains approximately valid if ratio of larger standard deviation to smaller standard deviation is less than 2. • Welch’s t-test for unequal spreads: Rather than pooling to obtain single estimate of population SD, use individual sample SD’s to estimate respective population SD’s, resulting in different formula for standard error of :

9. Welch’s t-test • Welch’s t-test assumes populations are normal but doesn’t assume equal SD. • Welch’s t-test has different degrees of freedom, p-value only approximate • JMP implementation: Analyze, Fit Y by X, click on unequal variances under red triangle next to Oneway Analysis. • Vitamin C in schizophrenia study:

10. Value of equal spread model • If two populations have same spread and same shape, then difference in means is entirely adequate summary of their difference. • If two populations have different means and different standard deviations, then difference in means may be inadequate summary. See Display 4.11.

11. Practical and Statistical Significance • Section 4.5.1 • p-values indicate statistical significance, the extent to which a null hypothesis is contradicted by data • This must be distinguished from practical significance, the practical importance of the finding.

12. Example • Investigators compare WISC vocabulary scores for big city and rural children. • They take a simple random sample of 2500 big city children and an independent simple random sample of 2500 rural children. • The big city children average 26 on the test and their SD is 10 points; the rural children average only 25 and their SD is 10 points • Two sample t-test: , p-value .00005 • Difference between big city children and rural children is highly significant, rural children are lagging behind in development of language skills and the investigators launch a crusade to pour money into rural schools.

13. Example Continued • Confidence interval for mean difference between rural and big city children: Approximate 95% CI = . • WISC test – 40 words child has to define. Two points given for correct definition, one for partially correct definition. • Likely value of mean difference between big city and rural children is about one partial understanding of a word out of forty. • Not a good basis for a crusade. Actually investigators have shown that there is almost no difference between big city and rural children on WISC vocabulary scale.

14. Practical vs. Statistical Significance • The p-value of a test depends on the sample size. With a large sample, even a small difference can be “statistically significant,” that is hard to explain by the luck of the draw. This doesn’t necessarily make it important. Conversely, an important difference may not be statistically significant if the sample is too small. • Always accompany p-values for tests of hypotheses with confidence intervals. Confidence intervals provide information about the likely magnitude of the difference and thus provide information about its practical importance.