Download
object orie d data analysis last time n.
Skip this Video
Loading SlideShow in 5 Seconds..
Object Orie’d Data Analysis, Last Time PowerPoint Presentation
Download Presentation
Object Orie’d Data Analysis, Last Time

Object Orie’d Data Analysis, Last Time

105 Views Download Presentation
Download Presentation

Object Orie’d Data Analysis, Last Time

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. Object Orie’d Data Analysis, Last Time • DiProPerm Test • Direction – Projection – Permutation • HDLSS hypothesis testing • NCI 60 Data • Particulate Matter Data • Perou 500 Breast Cancer Data • OK for subpop’ns found by clustering??? • Started Investigation of Clustering • Simple 1-d examples

  2. Clustering Important References: • McQueen (1967) • Hartigan (1975) • Gersho and Gray (1992) • Kaufman and Rousseeuw (2005),

  3. K-means Clustering Notes on Cluster Index: • CI = 0 when all data at cluster means • CI small when gives tight clustering (within SS contains little variation) • CI big when gives poor clustering (within SS contains most of variation) • CI = 1 when all cluster means are same

  4. K-means Clustering Clustering Goal: • Given data • Choose classes • To miminize

  5. 2-means Clustering Study CI, using simple 1-d examples • Varying Standard Deviation

  6. 2-means Clustering

  7. 2-means Clustering

  8. 2-means Clustering Study CI, using simple 1-d examples • Varying Standard Deviation • Varying Mean

  9. 2-means Clustering

  10. 2-means Clustering

  11. 2-means Clustering Study CI, using simple 1-d examples • Varying Standard Deviation • Varying Mean • Varying Proportion

  12. 2-means Clustering

  13. 2-means Clustering

  14. 2-means Clustering

  15. 2-means Clustering

  16. 2-means Clustering

  17. 2-means Clustering

  18. 2-means Clustering

  19. 2-means Clustering

  20. 2-means Clustering

  21. 2-means Clustering

  22. 2-means Clustering

  23. 2-means Clustering

  24. 2-means Clustering

  25. 2-means Clustering

  26. 2-means Clustering

  27. 2-means Clustering Study CI, using simple 1-d examples • Over changing Classes (moving b’dry)

  28. 2-means Clustering

  29. 2-means Clustering

  30. 2-means Clustering

  31. 2-means Clustering

  32. 2-means Clustering

  33. 2-means Clustering

  34. 2-means Clustering

  35. 2-means Clustering

  36. 2-means Clustering

  37. 2-means Clustering

  38. 2-means Clustering Study CI, using simple 1-d examples • Over changing Classes (moving b’dry) • Multi-modal data  interesting effects • Multiple local minima (large number) • Maybe disconnected • Optimization (over ) can be tricky… (even in 1 dimension, with K = 2)

  39. 2-means Clustering

  40. 2-means Clustering Study CI, using simple 1-d examples • Over changing Classes (moving b’dry) • Multi-modal data  interesting effects • Can have 4 (or more) local mins (even in 1 dimension, with K = 2)

  41. 2-means Clustering

  42. 2-means Clustering Study CI, using simple 1-d examples • Over changing Classes (moving b’dry) • Multi-modal data  interesting effects • Local mins can be hard to find • i.e. iterative procedures can “get stuck” (even in 1 dimension, with K = 2)

  43. 2-means Clustering Study CI, using simple 1-d examples • Effect of a single outlier?

  44. 2-means Clustering

  45. 2-means Clustering

  46. 2-means Clustering

  47. 2-means Clustering

  48. 2-means Clustering

  49. 2-means Clustering

  50. 2-means Clustering