1 / 68

Nonparametric tests II

Nonparametric tests II. as randomisation tests. Lecture Outline. Background: Nonparametric tests as randomisation tests The sign test The Wilcoxon signed ranks test The Mann-Whitney test General remarks on randomisation tests Brief Review of the course so far. after before 640.0 1050.0

varana
Download Presentation

Nonparametric tests II

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Nonparametric tests II as randomisation tests

  2. Lecture Outline • Background: Nonparametric tests as randomisation tests • The sign test • The Wilcoxon signed ranks test • The Mann-Whitney test • General remarks on randomisation tests • Brief Review of the course so far

  3. after before 640.0 1050.0 70.0 84.0 83.0 77.0 64.0 110.0 420.0 440.0 6.4 4.8 26.0 48.0 2.2 16.0 75.0 340.0 16.0 430.0

  4. after before change 640.0 1050.0 -410.0 70.0 84.0 -14.0 83.0 77.0 6.0 64.0 110.0 -46.0 420.0 440.0 -20.0 6.4 4.8 1.6 26.0 48.0 -22.0 2.2 16.0 -13.8 75.0 340.0 -265.0 16.0 430.0 -414.0

  5. after before change 640.0 1050.0 -410.0 70.0 84.0 -14.0 83.0 77.0 6.0 64.0 110.0 -46.0 420.0 440.0 -20.0 6.4 4.8 1.6 26.0 48.0 -22.0 2.2 16.0 -13.8 75.0 340.0 -265.0 16.0 430.0 -414.0 schange -414.0 -410.0 -265.0 -46.0 -22.0 -20.0 -14.0 -13.8 1.6 6.0

  6. MTB > stest 'change' Sign Test for Median Sign test of median=0.000 versus N.E. 0.000 N BELOW EQUAL ABOVE P-VALUE MEDIAN change 10 8 0 2 0.1094 -21.00 MTB >

  7. MTB > stest 'change' Sign Test for Median Sign test of median=0.000 versus N.E. 0.000 N BELOW EQUAL ABOVE P-VALUE MEDIAN change 10 8 0 2 0.1094 -21.00 MTB >

  8. MTB > stest 'change' Sign Test for Median Sign test of median=0.000 versus N.E. 0.000 N BELOW EQUAL ABOVE P-VALUE MEDIAN change 10 8 0 2 0.1094 -21.00 MTB > which added = number of non-zero datapoints (in this case there are no zeroes)

  9. So if we take ten items that might be plus or minus,

  10. So if we take ten items that might be plus or minus, and randomly choose them, we get the set of relevant comparisons for our dataset of 8 minus and 2 plus. This is the randomisation part of the test.

  11. So if we take ten items that might be plus or minus, and randomly choose them, we get the set of relevant comparisons for our dataset of 8 minus and 2 plus. This is the randomisation part of the test. To decide whether our actual dataset is extreme in the distribution, we calculate the test statistic in each case - just the number of plusses. We count in what fraction of cases, the relevant comparison has a more extreme number of plusses, that is, either 2 or fewer, or 8 or more.

  12. MTB > stest 'change' Sign Test for Median Sign test of median=0.000 versus N.E. 0.000 N BELOW EQUAL ABOVE P-VALUE MEDIAN change 10 8 0 2 0.1094 -21.00 MTB >

  13. The truth about confidence intervals . .. . . .: .. ---+---------+---------+---------+---------+---------+---change -400 -320 -240 -160 -80 0

  14. MTB > stest 'change' Sign Test for Median Sign test of median=0.000 versus N.E. 0.000 N BELOW EQUAL ABOVE P-VALUE MEDIAN change 10 8 0 2 0.1094 -21.00 MTB >

  15. MTB > stest 'change' Sign Test for Median Sign test of median=0.000 versus N.E. 0.000 N BELOW EQUAL ABOVE P-VALUE MEDIAN change 10 8 0 2 0.1094 -21.00 MTB >

  16. MTB > stest 0 c3 Sign Test for Median: C3 Sign test of median = 0.00000 versus not = 0.00000 N Below Equal Above P Median C3 10 8 0 2 0.1094 -21.00 MTB > stest 10 c3 Sign Test for Median: C3 Sign test of median = 10.00 versus not = 10.00 N Below Equal Above P Median C3 10 10 0 0 0.0020 -21.00 . .. . . .: .. ---+---------+---------+---------+---------+---------+---change -400 -320 -240 -160 -80 0

  17. H0median N Below Equal Above P Median -50 10 3 0 7 0.3438 -21.00 -40 10 4 0 6 0.7539 -21.00 -35 10 4 0 6 0.7539 -21.00 -30 10 4 0 6 0.7539 -21.00 -25 10 4 0 6 0.7539 -21.00 -20 10 5 1 4 1.0000 -21.00 -15 10 6 0 4 0.7539 -21.00 -10 10 8 0 2 0.1094 -21.00 -5 10 8 0 2 0.1094 -21.00 0 10 8 0 2 0.1094 -21.00 5 10 9 0 1 0.0215 -21.00 10 10 10 0 0 0.0020 -21.00 15 10 10 0 0 0.0020 -21.00 . .. . . .: .. ---+---------+---------+---------+---------+---------+---change -400 -320 -240 -160 -80 0

  18. H0median N Below Equal Above P Median -50 10 3 0 7 0.3438 -21.00 -40 10 4 0 6 0.7539 -21.00 -35 10 4 0 6 0.7539 -21.00 -30 10 4 0 6 0.7539 -21.00 -25 10 4 0 6 0.7539 -21.00 -20 10 5 1 4 1.0000 -21.00 -15 10 6 0 4 0.7539 -21.00 -10 10 8 0 2 0.1094 -21.00 -5 10 8 0 2 0.1094 -21.00 0 10 8 0 2 0.1094 -21.00 5 10 9 0 1 0.0215 -21.00 10 10 10 0 0 0.0020 -21.00 15 10 10 0 0 0.0020 -21.00 . .. . . .: .. ---+---------+---------+---------+---------+---------+---change -400 -320 -240 -160 -80 0

  19. H0median N Below Equal Above P Median -50 10 3 0 7 0.3438 -21.00 -40 10 4 0 6 0.7539 -21.00 -35 10 4 0 6 0.7539 -21.00 -30 10 4 0 6 0.7539 -21.00 -25 10 4 0 6 0.7539 -21.00 -20 10 5 1 4 1.0000 -21.00 -15 10 6 0 4 0.7539 -21.00 -10 10 8 0 2 0.1094 -21.00 -5 10 8 0 2 0.1094 -21.00 0 10 8 0 2 0.1094 -21.00 5 10 9 0 1 0.0215 -21.00 10 10 10 0 0 0.0020 -21.00 15 10 10 0 0 0.0020 -21.00 . .. . . .: .. ---+---------+---------+---------+---------+---------+---change -400 -320 -240 -160 -80 0

  20. H0median N Below Equal Above P Median -50 10 3 0 7 0.3438 -21.00 -40 10 4 0 6 0.7539 -21.00 -35 10 4 0 6 0.7539 -21.00 -30 10 4 0 6 0.7539 -21.00 -25 10 4 0 6 0.7539 -21.00 -20 10 5 1 4 1.0000 -21.00 -15 10 6 0 4 0.7539 -21.00 -10 10 8 0 2 0.1094 -21.00 -5 10 8 0 2 0.1094 -21.00 0 10 8 0 2 0.1094 -21.00 5 10 9 0 1 0.0215 -21.00 10 10 10 0 0 0.0020 -21.00 15 10 10 0 0 0.0020 -21.00 . .. . . .: .. ---+---------+---------+---------+---------+---------+---change -400 -320 -240 -160 -80 0

  21. H0median N Below Equal Above P Median -50 10 3 0 7 0.3438 -21.00 -40 10 4 0 6 0.7539 -21.00 -35 10 4 0 6 0.7539 -21.00 -30 10 4 0 6 0.7539 -21.00 -25 10 4 0 6 0.7539 -21.00 -20 10 5 1 4 1.0000 -21.00 -15 10 6 0 4 0.7539 -21.00 -10 10 8 0 2 0.1094 -21.00 -5 10 8 0 2 0.1094 -21.00 0 10 8 0 2 0.1094 -21.00 5 10 9 0 1 0.0215 -21.00 10 10 10 0 0 0.0020 -21.00 15 10 10 0 0 0.0020 -21.00 . .. . . .: .. ---+---------+---------+---------+---------+---------+---change -400 -320 -240 -160 -80 0

  22. H0median N Below Equal Above P Median -50 10 3 0 7 0.3438 -21.00 -40 10 4 0 6 0.7539 -21.00 -35 10 4 0 6 0.7539 -21.00 -30 10 4 0 6 0.7539 -21.00 -25 10 4 0 6 0.7539 -21.00 -20 10 5 1 4 1.0000 -21.00 -15 10 6 0 4 0.7539 -21.00 -10 10 8 0 2 0.1094 -21.00 -5 10 8 0 2 0.1094 -21.00 0 10 8 0 2 0.1094 -21.00 5 10 9 0 1 0.0215 -21.00 10 10 10 0 0 0.0020 -21.00 15 10 10 0 0 0.0020 -21.00 . .. . . .: .. ---+---------+---------+---------+---------+---------+---change -400 -320 -240 -160 -80 0

  23. . .. . . .: .. ---+---------+---------+---------+---------+---------+---change -400 -320 -240 -160 -80 0

  24. . .. . . .: .. ---+---------+---------+---------+---------+---------+---change -400 -320 -240 -160 -80 0 . .. . . .: .. ---+---------+---------+---------+---------+---------+---change -400 -320 -240 -160 -80 0 The green values cannot be rejected at the 5% level, while the red values can.

  25. . .. . . .: .. ---+---------+---------+---------+---------+---------+---change -400 -320 -240 -160 -80 0 . .. . . .: .. ---+---------+---------+---------+---------+---------+---change -400 -320 -240 -160 -80 0 The green values cannot be rejected at the 5% level, while the red values can. The range of green values is therefore the 95% confidence interval for the median based on the sign test.

  26. The real definition of 95% confidence interval • is “the set of values of a parameter that cannot be rejected at the 5% level” • is therefore not “the set of values that the parameter has a 95% chance of belonging to”, as many textbooks claim. (This is called a “fiducial interval”.)

  27. MTB > sinterval 'change' Sign Confidence Interval Sign confidence interval for median ACHIEVED POSI N MEDIAN CONFIDENCE CONFIDENCE INTERVAL TION change 10 -21.000 0.8906 (-265.000, -13.800) 3 0.9500 (-314.640, -8.528) NLI 0.9785 (-410.000, 1.600) 2 MTB >

  28. H0median N Below Equal Above P Median -50 10 3 0 7 0.3438 -21.00 -40 10 4 0 6 0.7539 -21.00 -35 10 4 0 6 0.7539 -21.00 -30 10 4 0 6 0.7539 -21.00 -25 10 4 0 6 0.7539 -21.00 -20 10 5 1 4 1.0000 -21.00 -15 10 6 0 4 0.7539 -21.00 -10 10 8 0 2 0.1094 -21.00 -5 10 8 0 2 0.1094 -21.00 0 10 8 0 2 0.1094 -21.00 5 10 9 0 1 0.0215 -21.00 10 10 10 0 0 0.0020 -21.00 15 10 10 0 0 0.0020 -21.00 . .. . . .: .. ---+---------+---------+---------+---------+---------+---change -400 -320 -240 -160 -80 0

  29. after before change 640.0 1050.0 -410.0 70.0 84.0 -14.0 83.0 77.0 6.0 64.0 110.0 -46.0 420.0 440.0 -20.0 6.4 4.8 1.6 26.0 48.0 -22.0 2.2 16.0 -13.8 75.0 340.0 -265.0 16.0 430.0 -414.0 schange -414.0 -410.0 -265.0 -46.0 -22.0 -20.0 -14.0 -13.8 1.6 6.0 97.85% 89.06% 89.06% 97.85%

  30. MTB > sinterval 'change' Sign Confidence Interval Sign confidence interval for median ACHIEVED POSI N MEDIAN CONFIDENCE CONFIDENCE INTERVAL TION change 10 -21.000 0.8906 (-265.000, -13.800) 3 0.9500 (-314.640, -8.528) NLI 0.9785 (-410.000, 1.600) 2 MTB > . .. . . .: .. ---+---------+---------+---------+---------+---------+---change -400 -320 -240 -160 -80 0

  31. Why does Minitab give three confidence intervals for the sign test? • the p-value for rejecting a value changes in a step function at observed values • so exact confidence intervals are given between observed values, at whatever level of confidence is attained • the NLI (Non-Linear Interpolation) confidence interval is a confidence trick

  32. Lecture Outline • Background: Nonparametric tests as randomisation tests • The sign test • The Wilcoxon signed ranks test • The Mann-Whitney test • General remarks on randomisation tests • Brief Review of the course so far

  33. after before change 640.0 1050.0 -410.0 70.0 84.0 -14.0 83.0 77.0 6.0 64.0 110.0 -46.0 420.0 440.0 -20.0 6.4 4.8 1.6 26.0 48.0 -22.0 2.2 16.0 -13.8 75.0 340.0 -265.0 16.0 430.0 -414.0 schange -414.0 -410.0 -265.0 -46.0 -22.0 -20.0 -14.0 -13.8 1.6 6.0

  34. MTB > stest 'change' Sign Test for Median Sign test of median=0.000 versus N.E. 0.000 N BELOW EQUAL ABOVE P-VALUE MEDIAN change 10 8 0 2 0.1094 -21.00 MTB > wtest 'change' Wilcoxon Signed Rank Test TEST OF MEDIAN = 0.000 VERSUS MEDIAN N.E. 0.000 N FOR WILCOXON ESTIMATED N TEST STATISTIC P-VALUE MEDIAN change 10 10 3.0 0.014 -46.00 MTB >

  35. MTB > stest 'change' Sign Test for Median Sign test of median=0.000 versus N.E. 0.000 N BELOW EQUAL ABOVE P-VALUE MEDIAN change 10 8 0 2 0.1094 -21.00 MTB > wtest 'change' Wilcoxon Signed Rank Test TEST OF MEDIAN = 0.000 VERSUS MEDIAN N.E. 0.000 N FOR WILCOXON ESTIMATED N TEST STATISTIC P-VALUE MEDIAN change 10 10 3.0 0.014 -46.00 MTB >

  36. MTB > stest 'change' Sign Test for Median Sign test of median=0.000 versus N.E. 0.000 N BELOW EQUAL ABOVE P-VALUE MEDIAN change 10 8 0 2 0.1094 -21.00 MTB > wtest 'change' Wilcoxon Signed Rank Test TEST OF MEDIAN = 0.000 VERSUS MEDIAN N.E. 0.000 N FOR WILCOXON ESTIMATED N TEST STATISTIC P-VALUE MEDIAN change 10 10 3.0 0.014 -46.00 MTB > The Wilcoxon test is more powerful than the Sign Test

More Related