1 / 35

Krisztina Boda PhD Department of Medical Informatics, University of Szeged

Biostatistics, statistical software V . Statistical errors, one-and two sided tests. One-way and multifactor analysis of variance. Krisztina Boda PhD Department of Medical Informatics, University of Szeged. One- and two tailed (sided) tests. Two tailed test H 0 : there is no change

penha
Download Presentation

Krisztina Boda PhD Department of Medical Informatics, University of Szeged

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Biostatistics, statistical software V.Statistical errors, one-and two sided tests. One-way and multifactor analysis of variance. Krisztina Boda PhD Department of Medical Informatics, University of Szeged

  2. One- and two tailed (sided) tests • Two tailed test • H0: there is no change • Ha: There is change (in either direction) • One-tailed test • H0: the change is negative or zero • Ha: the change is positive p-values: p(one-tailed)=p(two-tailed)/2 INTERREG

  3. Significance • Significant difference – if we claim that there is a difference (effect), the probability of mistake is small (maximum - Type I error ). • Not significant difference – we say that there is not enough information to show difference. Perhaps • there is no difference • There is a difference but the sample size is small • The dispersion is big • The method was wrong • Even is case of a statistically significant difference one has to think about its biological meaning INTERREG

  4. Statistical errors TruthDecision do not reject H0 reject H0 (significance) H0 is true correct Type I. errorits probability: Ha is true Type II. error correct its probability: INTERREG

  5. Error probabilities • The probability of type I error is known ( ). • The probability of type II error is not known () • It depends on • The significance level (), • Sample size, • The standard deviation(s) • The true difference between populations • others (type of the test, assumptions, design, ..) • The power of a test: 1-  ability to detect a real effect; probability to have a significant p-value INTERREG

  6. The power of a test on case of fixed sample size and , with two alternative hypotheses INTERREG

  7. ANOVAAnalysis of Variance • Comparison the mean of of several (>2), normally distributed samples • Types: • One-way: • Control, treatment I, treatment II. • Two-way (treatment + sex) • Any „way” (factor) can be • „independent” („between-subjects”) sex, treatments • „repeated measures” („within-subjects”) data measured on the same patient INTERREG

  8. Why not t-test (pair wise)?We can get significant result only by chance at every 20th case INTERREG

  9. The increase of type I error • It can be shown that when t tests are used to test for differences between multiple groups, the chance of mistakenly declaring significance (Type I Error) is increasing. For example, in the case of 5 groups, if no overall differences exist between any of the groups, using two-sample t tests pair wise, we would have about 30% chance of declaring at least one difference significant, instead of 5% chance. • In general, the t test can be used to test the hypothesis that two group means are not different. To test the hypothesis that three ore more group means are not different, analysis of variance should be used. INTERREG

  10. Each statistical test produces a ‘p’ value • If the significance level is set at 0.05 (false positive rate) and we do multiple significance testing on the data from a single clinical trial, • then the overall false positive rate for the trial will increase with each significance test. INTERREG

  11. False positive rate for each test = 0.05 • Probability of incorrectly rejecting ≥ 1 hypothesis out of N testings • = 1 – (1-0.05)N=1-(1-)n INTERREG

  12. INTERREG

  13. Compound hypotheses • (H01 and H02 and... H0n ) null hypotheses, the significance levels are 1, 2, …, n • How to choose i-s so that the level of the compound hypothesis (H01 and H02 and ... H0n ) would be no greater than  ? (0,1) INTERREG

  14. Bonferroni correction • The  is divided by the number of comparisons. (H01and H02 and H0n ) is rejected, if at least one pi</n • In case of many comparisons, this is too conservative (will not show real differences). INTERREG

  15. Holm-modification (SAS: step-down Bonferroni) • The pi-s are sorted. p1p2...pn • H0i is tested at level • If any of them is significant, then reject (H01 and H02 and... H0n ) . • Pl. n=5 • p1/5=0.01 if p1 is not smaller, then finish • p2/4=0.0125 ha p2 is not smaller, then finish • p3/3=0.0166 is not smaller, then finish • p4/2=0.025 …. • p5/1=0.05 INTERREG

  16. FDR (false discovery rate) • p1p2...pn • Begin with the greatest p-value, it remains the same • The next is tested at level • Pl. n=5 • p5 • p4/(4*5) • p3/(3*5) • p2/(2*5) • p1/(1*5)=0.05 INTERREG

  17. Correction of unique p-values The SAS System The Multtest Procedure p-Values False Stepdown Discovery Test Raw Bonferroni Hochberg Rate 1 0.9999 1.0000 0.9999 0.9999 2 0.2318 0.9272 0.9272 0.5795 3 0.3771 1.0000 0.9999 0.6285 4 0.8231 1.0000 0.9999 0.9999 5 0.0141 0.0705 0.0705 0.0705 INTERREG

  18. One-Way ANOVA • Let us suppose that we have t independent samples (t “treatment” groups) drawn from normal populations with equal variances ~N(µi,). • Assumptions: • Independent samples • normality • Equal variances • Null hypothesis: population means are equal, µ1=µ2=.. =µt INTERREG

  19. http://lib.stat.cmu.edu/DASL/Stories/CancerSurvival.html.Cameron, E. and Pauling, L. (1978) Supplemental ascorbate in the supportive treatment of cancer: re-evaluation of prolongation of survival times in terminal human cancer. Proceedings of the National Academy of Science USA, 75, 4538Ð4542. Original Square root transformed INTERREG

  20. Method • If the null hypothesis is true, then the populations are the same: they are normal, and they have the same mean and the same variance. This common variance is estimated in two distinct ways: • between-groups variance • within-groups variance • If the null hypothesis is true, then these two distinct estimates of the variance should be equal • ‘New’ (and equivalent) null hypothesis: 2between=2within • their equality can be tested by an F ratio test • The p-value of this test: • if p>0.05, then we accept H0. The analysis is complete. • if p<0.05, then we reject H0 at 0.05 level. There is at least one group-mean different from one of the others INTERREG

  21. INTERREG

  22. The ANOVA table INTERREG

  23. Pairwise comparisons • As the two-sample t-test is inappropriate to do this, there are special tests for multiple comparisons that keep the probability of Type I error as . The most often used multiple comparisons are the modified t-tests. • Modified t-tests(LSD) • Bonferroni: α/(number of comparisons) • Scheffé • Tukey • Dunnett: a test comparing a given group (control) with the others INTERREG

  24. Examplehttp://lib.stat.cmu.edu/DASL/Stories/ReadingComprehension.htmlExamplehttp://lib.stat.cmu.edu/DASL/Stories/ReadingComprehension.html • Researchers at Purdue University conducted an experiment to compare three methods of teaching reading. • Students were randomly assigned to one of the three teaching methods, and their reading comprehension was tested before and after they received the instruction. Several different measures of reading comprehension, from both the pre- and posttests are included in the dataset. • Reference: Moore, David S., and George P. McCabe (1989). Introduction to the Practice of Statistics. Original source: study conducted by Jim Baumann and Leah Jones of the Purdue University Education Department. INTERREG

  25. INTERREG

  26. INTERREG

  27. INTERREG

  28. Nonparametric one-way ANOVAKruskal-Wallis test. • As a result, it gives one p-value. If it is nit significant, the null hypothesis is accepted. • If the null hypothesis is rejected, further tests are required to make pairwise comparisons. These pairwise comparisons are generally not available in standard statistical packages. Pairwise comparisons can be performed by Mann Whitney U tests and p-values can be corrected by Bonferroni correction INTERREG

  29. Two-way ANOVA, example Does systolic blood pressure depend on • Diabetes or not • Male or female Independent factors INTERREG

  30. Two-way repeated measurements ANOVA • Does QT widening in the Langendorff-perfused rat heart represent the effect of repolarization delay or conduction slowing? J Cardiovasc Pharmacol. 42 (2003) 612-21 INTERREG

  31. Effect of regional ischemia and K+ content of the perfusion solution on the QT90 interval (A) and heart rate (B)in drug-free isolated rat hearts (n = 12 hearts per group). (mean ± SEM) INTERREG

  32. Frequently, separate univariate analyses are used for every time point and take no account the fact that data are related in time. A second problem is the frequent occurrence of missing values in the data. A repeated measurement ANOVA model is more appropriate (Brown and Prescott). • repeated testing is taking place and therefore a significant effect is more likely to occur at some time point by chance. INTERREG

  33. Repeated measurement ANOVA model • We can examine: • The treatment effect (K+) • Time-effect • Their interaction * * * In high potassium concentration the heart rate is significantly higher, independently of the time it was measured INTERREG

  34. Review questions and exercises • Problems to be solved by hand-calculations • ..\Handouts\Problems hand V.doc • Solutions • ..\Handouts\Problems hand V solutions.doc • Problems to be solved using computer • ..\Handouts\Problems comp V.doc, • ..\Handouts\Problems comp V solutions.doc INTERREG

  35. Useful WEB pages • http://www-stat.stanford.edu/~naras/jsm • http://www.ruf.rice.edu/~lane/rvls.html • http://my.execpc.com/~helberg/statistics.html • http://www.math.csusb.edu/faculty/stanton/m262/index.html INTERREG

More Related