LISA Short Course: A Tutorial in t-tests and ANOVA using JMP

1. LISA Short Course: A Tutorial in t-tests and ANOVA using JMP Anne Ryan Assistant Professor of Practice Department of Statistics, VT agryan@vt.edu Laboratory for Interdisciplinary Statistical Analysis

2. Laboratory for Interdisciplinary Statistical Analysis LISA helps VT researchers benefit from the use ofStatistics Designing Experiments • Analyzing Data • Interpreting ResultsGrant Proposals • Using Software (R, SAS, JMP, Minitab...) Walk-In Consulting Available 1-3 PM: Mon—Fri in the GLC Video Conference Room for questions requiring <30 mins See our website for additional times and locations. Collaboration From our website request a meeting for personalized statistical advice Great advice right now:Meet with LISA before collecting your data Short Courses Designed to help graduate students apply statistics in their research All services are FREE for VT researchers. We assist with research—not class projects or homework. www.lisa.stat.vt.edu

3. Hypothesis Testing and Criminal Trial Analogy

4. Criminal Trial • Defense: • Prosecution: • What’s the Assumed Conclusion? Represent the accused (defendant) Hold the “Burden of Proof”—obligation to shift the assumed conclusion from an oppositional opinion to one’s own position through evidence • ANSWER: The accused is innocent until proven guilty. • Prosecution must convince the judge/jury that the defendant is guilty beyond a reasonable doubt

5. Similarities between Criminal Trials and Hypothesis Testing Burden of Proof—Obligation to shift the conclusion using evidence Hypothesis Test Trial Accept the status quo (what is believed before) until the data suggests otherwise Innocent until proven guilty

6. Similarities between Criminal Trials and Hypothesis Testing Decision Criteria Hypothesis Test Trial Occurs by chance less than 100α% of the time (ex: 5%) Evidence has to convincing beyond a reasonable

7. Inferential Statistics is defined as …a procedure that allows us to make statements about a general population using the results of a random sample from that population. • Two Types of Inferential Statistics: • Hypothesis Testing • Estimation • Point estimates • Confidence intervals

8. What is hypothesis testing ? Hypothesis testing is a detailed protocol for decision-making concerning a population by examining a sample from that population.

9. Steps in a Hypothesis Test • Test • Assumptions • Hypotheses • Mechanics • Conclusion

10. One Sample t-Test Used to test whether the population mean is different from a specified value.

11. Example 1: Motivating Example • In a glaucoma study, the following intraocular pressure (mm Hg) values were recorded from a sample of 21 elderly subjects. Based on this data, can we conclude that the mean intraocular pressure of the population from which the sample was drawn differs from 14 mm Hg?* *Wayne, D. Biostatistics: A Foundation for Analysis in the Health Sciences. 5th ed. New York: John Wiley & Sons, 1991.

12. 1. Test • Example 1: • Test: One sample t test for • State the name of the testing method to be used • It is important to not be off track in the very beginning

13. 2. Assumptions • Example 1: • 2. Assumptions • Simple random sample (SRS) was used to collect data • The population distribution from which the sample is drawn is normal or approximately normal. • List all the assumptions required for your test to be valid. • All tests have assumptions • Even if assumptions are not met you should still comment on how this affects your results.

14. 3. Hypotheses Claims versus suspicions: • The “null hypothesis” is a statement describing a claim about a population constant. - The null hypothesis is denoted as . • The “alternative hypothesis” is a statement describing the researcher’s suspicions about the claim. Also called “research hypothesis”. - The alternative hypothesis is denoted as . • Examples of possible hypotheses:

15. 3. Hypotheses • For hypothesis testing there are three versions for testing that are determined by the context of the research question. • Left Tailed Hypothesis Test (less than) • Right Tailed Hypothesis Test (greater than) • Two Tailed or Two Sided Hypothesis Test (not equal to)

16. 3. Hypotheses Left Tailed Hypothesis Test: • Researchers are only interested in whether the true value is below the hypothesized value. • Example— Administrators of a health care center want to know if the mean time spent by patients in the waiting room is less than 20 minutes. Right Tailed Hypothesis Test: • Researchers are only interested in whether the True Value is above the hypothesized value. • Example— Administrators of a health care center want to know if the mean time spent by patients in the waiting room is greater than 20 minutes.

17. 3. Hypotheses • Two Tailed or Two Sided Hypothesis Test: • The researcher is interested in looking above and belowtheir hypothesized value. • Example— Administrators of a health care center want to know if the mean time spent by patients in the waiting room differsfrom 20 minutes. • Note: The direction of the alternative hypothesis will be used when determining the p-value at a later step.

18. 3. Hypotheses • Example 1: • 3. Hypotheses • In a glaucoma study, the following intraocular pressure (mm Hg) values were recorded from a sample of 21 elderly subjects. Based on this data, can we conclude that the mean intraocular pressure of the population from which the sample was drawn differs from 14 mm Hg?* • What are the hypotheses for Example 1? Where is the true intraocular pressure

19. 4. Mechanics • Computational Part of the Test • Parts of the Mechanics Step • Stating the Significance Level • Finding the Rejection Rule • Computing the Test Statistic • Computing the p-value

20. 4. Mechanics • Example 1: • 4. Mechanics: • Significance Level: • *We use here because the significance level was not given in the problem. • *Note: The Type I error would be concluding that the true mean intraocular pressure differs from 14 mm Hg, when in fact the pressure is 14 mm Hg. • Significance Level: Here we choose a value to use as the significance level, which is the level at which we are willing to start rejecting the null hypothesis. • Denoted by α which corresponds to the Type 1 Error for the test. • Type 1 Error is error committed when the true null hypothesis is rejected. Ex: You reject when is true. • * Default value is α=.05, use α=.05 unless otherwise noted!

21. 4. Mechanics • Example 1: • 4. Mechanics: • Rejection Rule: • Reject if • Rejection Rule: State our criteria for rejecting the null hypothesis • Reject the null hypothesis () if the p-value • p-value: The chance of observing your sample results or more extreme results assuming that the null hypothesis is true. If this chance is “small” then you may decide the claim in the null hypothesis is false.

22. 4. Mechanics • Test Statistic: Compute the test statistic, which is usually a standardization of your point estimate. • Translates your point estimate, a statistic, to follow a known distribution so that is can be used for a test. A point estimate is a single numerical value used to estimate the corresponding population parameter. • is the point estimate for

23. Deriving the Test Statistic for Example 1 Test statistic for a one sample t-test • In many cases, including Example 1, the population standard deviation is unknown because it is a parameter from the population that must be estimated. • The best estimate for is . • Our standardized value becomes : hypothesized mean : sample mean : sample standard deviation : sample size : observed t test statistic This t observed ( test statistic follows a tdistribution with degrees of freedom.

24. 4. Mechanics • Example 1: • 4. Mechanics • Test Statistic: • *In the example it was given that and .

25. 4. Mechanics • Example 1: • 4. Mechanics: • P-value (in words): • The probability of observing a sample mean of mm hg or a value more extreme assuming the true mean pressure is 14 mm hg. • p-value: After computing the test statistic, now you can compute the p-value. • A p-value is the probability of obtaining a point estimate as “extreme” as the current value where the definition of “extreme” is taken from the alternative hypothesis assuming the null hypothesis is true. • The p-value depends on the alternative hypothesis, so there are three ways to compute p-values. • p-value: The chance of observing your sample results or more extreme results assuming that the null hypothesis is true. If this chance is “small” then you may decide the claim in the null hypothesis is false.

26. 4. Mechanics • The p-value is determined based on the sign of the alternative hypothesis. • . If this is the case, then the p-value is the area in both tails of the t distribution.

27. 4. Mechanics • The p-value is determined based on the sign of the alternative hypothesis. • . If this is the case, then the p-value is the area to the left of the observed test statistic.

28. 4. Mechanics • The p-value is determined based on the sign of the alternative hypothesis. • . If this is the case, then the p-value is the area to the right of the observed test statistic.

29. Example 1: • 4. Mechanics • p-value: • *In the example the hypotheses are:

30. Example 1: • 4. Mechanics • p-value:

31. Example 1: • 4. Mechanics • p-value: • JMP will give the 3 p-values and you must select the correct p-value based on your alternative hypothesis

32. Example 1: • 5. Conclusion: • With a p-value=0.0398, which is less than 0.05, we reject . There is sufficient sample evidence to conclude that the true mean intraocular pressure differs from 14 mm Hg. 5. Conclusion • Conclusion: Last step of the hypothesis test. • Conclusions should always include: • Decision:reject or fail to reject (not accept ). • When conducting hypothesis tests, we assume that is true, therefore the decision cannot be to accept the null hypothesis. • Context: what your decision means in context of the problem. Note: The significance level can be thought of as a tolerance for things happening by chance. If we set α=.05 then we are saying that we are willing to say what we observe may be out of the ordinary, but unless it is something that occurs less that 5% of the time we will attribute it to chance.

33. Summary of One Sample t-test • Possible Hypotheses: • Test Statistic: • Degrees of Freedom: • Assumption: The population from which the sample is drawn is normal or approximately normal.

34. P-values for One Sample t-test Let T be a trandom variable with and • Left-tailed test • *written as Prob<t in jmp • Right-tailed test • *written as Prob>t in jmp • Two-tailed tests • *written as Prob>|t| in jmp

35. Solution to Example 1 re-visited: T: One sample t-test for A: i) SRS was used ii)The population from which the sample is drawn is normal or approximately normal. H: is the true mean intraocular pressure M: Reject if p-value0.05 p-value=(calculated using JMP: Prob>|t|) C: With a p-value less than 0.05, we reject . There is sufficient sample evidence to conclude that the true mean intraocular pressure differs from 14 mm Hg. • In a glaucoma study, the following intraocular pressure (mm Hg) values were recorded from a sample of 21 elderly subjects. Based on this data, can we conclude that the mean intraocular pressure of the population from which the sample was drawn differs from 14 mm Hg?*

36. Hypothesis Test for a Single Mean in JMP • JMP Demonstration • Open Pressure.jmp • AnalyzeDistribution • Complete the dialog box as shown and select OK. • Select the red arrow next to “Pressure” and select Test Mean. • Complete Dialog box as shown and select OK. • Select the red arrow next to “Pressure” and select Confidence Interval->0.95.

37. JMP Output • The normal quantile plot may also be created in JMP to check the normality assumption. The assumption is met if the points fall close to the red line.

38. Two Sample t-Test Two sample t-tests are used to determine whether the population mean of one group is equal to, larger than or smaller than the population mean of another group.

39. Two Sample T-Test • The major goal is to determine whether a difference exists between two populations. • Examples: • Compare blood pressure for male and females. • Compare the proportion of smokers and nonsmokers with lung cancer. • Compare weight before and after treatment. • Is the mean cholesterol of people taking drug A lower than the mean cholesterol of people taking drug B?

40. Step 1: Formulate the Hypotheses • The population means of the two groups are not equal. H0: μ1 = μ2 Ha: μ1 ≠ μ2 • The population mean of group 1 is greater than the population mean of group 2. H0: μ1 = μ2 Ha: μ1 > μ2 • The population mean of group 1 is less than the population mean of group 2. H0: μ1 = μ2 Ha: μ1 < μ2

41. Step 2: Check the Assumptions The two samples are random and independent. The populations from which the samples are drawn are either normal or the sample sizes are large. The populations have the same standard deviation.

42. Steps 3-5 Step 3: Calculate the test statistic where Step 4: Calculate the appropriate p-value. Step 5: Write a Conclusion.

43. Summary of two sample t-test • Possible Hypotheses: • Test Statistic: Assumption: The populations from which both samples are drawn are normal or approximately normal.

44. Two Sample Example http://en.wikipedia.org/ wiki/Iris_flower_data_set http://en.wikipedia.org/ wiki/Iris_versicolor A researcher would like to know whether the mean sepal width of setosa irises is different from the mean sepal width of versicolor irises. The researcher randomly selects 50 setosa irises and 50 versicolor irises and measures their sepal widths. Step 1 Hypotheses: H0: μsetosa = μversicolor Ha: μsetosa ≠ μversicolor

45. JMP Steps 2-4: JMP Demonstration: Analyze  Fit Y By X Y, Response: Sepal Width X, Factor: Species Means/ANOVA/Pooled t Normal Quantile Plot  Plot Actual by Quantile

46. JMP Output • Step 5 Conclusion: There is strong evidence (p-value < 0.0001) that the mean sepal widths for the two varieties are different.

47. Paired t-Test The paired t-test is used to compare the population means of two groups when the samples are dependent.

48. Paired T-Test The objective of paired comparisons is to minimize sources of variation that are not of interest in the study by pairing observations with similar characteristics. Example: A researcher would like to determine if background noise causes people to take longer to complete math problems. The researcher gives 20 subjects two math tests one with complete silence and one with background noise and records the time each subject takes to complete each test.

49. Step 1: Formulate the Hypotheses • The population mean difference is not equal to zero. H0: μdifference = 0 Ha: μdifference ≠ 0 • The population mean difference is greater than zero. H0: μdifference = 0 Ha: μdifference > 0 • The population mean difference is less than a zero. H0: μdifference = 0 Ha: μdifference < 0

50. Step 2: Check the assumptions The sample is random. The data is matched pairs. The differences have a normal distribution or the sample size is large.