1 / 26

Computer Algebra and Rank Statistics Presentation

Explore nonparametric methods, computer algebra, Mann-Whitney statistic, moments computation, generating functions, and practical applications in statistics.

tao
Download Presentation

Computer Algebra and Rank Statistics Presentation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Computer algebra and rank statistics Alessandro Di Bucchianico HCM Workshop Coimbra November 5, 1997

  2. How to run this presentation? • the presentation runs itself most of the time • click the mouse if you want to continue • type S to stop or restart the presentation • underlined items are hyperlinks to files on the World Wide Web (usually Postscripts files of technical reports) Enjoy my presentation!

  3. Outline • General remarks on nonparametric methods • What is computer algebra? • Case study: the Mann-Whitney statistic • Critical values of rank test statistics • Moments of the Mann-Whitney statistic • Conclusions

  4. General remarks on nonparametric methods Practical problems • tables (limited, errors, not exact,…) • limited availability in statistical software • procedures in statistical software often only based on asymptotics

  5. General remarks on nonparametric methods Mathematical problems • in general no closed expression for distribution function • direct enumeration only feasible for small sample sizes • recurrences are time-consuming

  6. What is computer algebra?

  7. Case study: Mann-Whitney statistic independent samples X1,…,Xm and Y1,…,Yn continuous distribution functions F, G resp. (hence, no ties with probability one) order the pooled sample from small to large

  8. Mann-Whitney (continued) Wilcoxon: Wm,n= Sirank(Xi) Mann-Whitney: Mm,n = #{(i,j) | Yj < Xi} Wm,n = Mm,n + ½ m (m+1) What is the distribution of Mm,n under H0:F=G?

  9. Under H0, we have:

  10. Computational speed (Pentium 133 MHz) Exact: P(M5,5 4) = 1/21  0.0476 computing time: 0.05 sec (generating function: degree 25) P(M5,5 4)  0.0384 Exact: P(M20,20 138) = 0.0482 (rounded) computing time: 8.5 sec (generating function: degree 400) P(M20,20 138)  0.0475 Asymptotics and exact calculations are bothuseful!

  11. Other examples of nonparametric test statistics with closed form for generating function include: • Wilcoxon signed rank statistic • Kendall rank correlation statistic • Kolmogorov one-sample statistic • Smirnov two-sample statistic • Jonckheere-Terpstra statistic Consult the combinatorial literature! What to do if there is no generating function?

  12. Linear rank statistics Z = 1 if th order statistic in the pooled sample is an X-observation, and 0 otherwise Streitberg & Röhmel 1986 (cf. Euler 1748): Branch-and-bound algorithm (Van de Wiel)

  13. Moments of Mann-Whitney statistic Mann and Whitney (1947) calculated 4th central moment Fix and Hodges (1955) calculated 6th central moment Computations are based on recurrences Can we improve? computer algebra and generating functions 21th century solution:

  14. Computing moments of Mm,n recompute E(Mm,n) (following René Swarttouw)

  15. Hence, it remains to calculate for 1  k  m : After some simplifications:

  16. L’Hôpital’s rule yields that the limit equals: It is tedious to perform these computations by hand. Alternative: compute moments using Mathematica.

  17. Mathematica procedures for moments of Mm,n:

  18. 8th central moment of Mm,n

  19. Conclusions • generating functions are also useful in nonparametric statistics • computer algebra is a natural tool for mathematicians • asymptotics and exact calculations complement each other

  20. Topics under investigation • tests for censored data • power calculations • nonparametric ANOVA (Kruskal-Wallis, block designs, multiple comparisons) • Spearman’s  (rank correlation) • multimedia/ World Wide Web implementation Click on underlined items to obtain Postscript file of technical report

  21. References • A. Di Bucchianico, Combinatorics, computer algebra and the Wilcoxon-Mann-Whitney test, to appear in J. Stat. Plann. Inf. • B. Streitberg and J. Röhmel, Exact distributions for permutation and rank tests: An introduction to some recently published algorithms, Stat. Software Newsletter 12 (1986), 10-18

  22. References (continued) • M.A. van de Wiel, Exact distributions of nonparametric statistics using computer algebra, Master’s Thesis, TUE, 1996 • M.A. van de Wiel and A. Di Bucchianico, The exact distribution of Spearman’s rho, technical report • M.A. van de Wiel, A. Di Bucchianico and P. van der Laan, Exact distributions of nonparametric test statistics using computer algebra, technical report

  23. References (continued) • M.A. van de Wiel, Edgeworth expansions with exact cumulants for two-sample linear rank statistics , technical report • M.A. van de Wiel,Exact distributions of two-sample rank statistics and block rank statistics using computer algebra , technical report

  24. The End

More Related