1 / 79

Object Orie’d Data Analysis, Last Time

Object Orie’d Data Analysis, Last Time. DiProPerm Test Direction – Projection – Permutation HDLSS hypothesis testing NCI 60 Data Particulate Matter Data Perou 500 Breast Cancer Data OK for subpop’ns found by clustering??? Started Investigation of Clustering Simple 1-d examples.

iain
Download Presentation

Object Orie’d Data Analysis, Last Time

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Object Orie’d Data Analysis, Last Time • DiProPerm Test • Direction – Projection – Permutation • HDLSS hypothesis testing • NCI 60 Data • Particulate Matter Data • Perou 500 Breast Cancer Data • OK for subpop’ns found by clustering??? • Started Investigation of Clustering • Simple 1-d examples

  2. Clustering Important References: • McQueen (1967) • Hartigan (1975) • Gersho and Gray (1992) • Kaufman and Rousseeuw (2005),

  3. K-means Clustering Notes on Cluster Index: • CI = 0 when all data at cluster means • CI small when gives tight clustering (within SS contains little variation) • CI big when gives poor clustering (within SS contains most of variation) • CI = 1 when all cluster means are same

  4. K-means Clustering Clustering Goal: • Given data • Choose classes • To miminize

  5. 2-means Clustering Study CI, using simple 1-d examples • Varying Standard Deviation

  6. 2-means Clustering

  7. 2-means Clustering

  8. 2-means Clustering Study CI, using simple 1-d examples • Varying Standard Deviation • Varying Mean

  9. 2-means Clustering

  10. 2-means Clustering

  11. 2-means Clustering Study CI, using simple 1-d examples • Varying Standard Deviation • Varying Mean • Varying Proportion

  12. 2-means Clustering

  13. 2-means Clustering

  14. 2-means Clustering

  15. 2-means Clustering

  16. 2-means Clustering

  17. 2-means Clustering

  18. 2-means Clustering

  19. 2-means Clustering

  20. 2-means Clustering

  21. 2-means Clustering

  22. 2-means Clustering

  23. 2-means Clustering

  24. 2-means Clustering

  25. 2-means Clustering

  26. 2-means Clustering

  27. 2-means Clustering Study CI, using simple 1-d examples • Over changing Classes (moving b’dry)

  28. 2-means Clustering

  29. 2-means Clustering

  30. 2-means Clustering

  31. 2-means Clustering

  32. 2-means Clustering

  33. 2-means Clustering

  34. 2-means Clustering

  35. 2-means Clustering

  36. 2-means Clustering

  37. 2-means Clustering

  38. 2-means Clustering Study CI, using simple 1-d examples • Over changing Classes (moving b’dry) • Multi-modal data  interesting effects • Multiple local minima (large number) • Maybe disconnected • Optimization (over ) can be tricky… (even in 1 dimension, with K = 2)

  39. 2-means Clustering

  40. 2-means Clustering Study CI, using simple 1-d examples • Over changing Classes (moving b’dry) • Multi-modal data  interesting effects • Can have 4 (or more) local mins (even in 1 dimension, with K = 2)

  41. 2-means Clustering

  42. 2-means Clustering Study CI, using simple 1-d examples • Over changing Classes (moving b’dry) • Multi-modal data  interesting effects • Local mins can be hard to find • i.e. iterative procedures can “get stuck” (even in 1 dimension, with K = 2)

  43. 2-means Clustering Study CI, using simple 1-d examples • Effect of a single outlier?

  44. 2-means Clustering

  45. 2-means Clustering

  46. 2-means Clustering

  47. 2-means Clustering

  48. 2-means Clustering

  49. 2-means Clustering

  50. 2-means Clustering

More Related