1 / 30

Manifold Learning

Manifold Learning. Kai Yang sadoii@163.com. 1.12.2015. Machine Learning Problem. (Training Data). f. C 罗. We always think X and Y are in Euclidean space. f:X → Y. Outline. What’s manifold and manifold learning? What’s classical methods and its application? Summary and thought.

kchapin
Download Presentation

Manifold Learning

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Manifold Learning Kai Yang sadoii@163.com 1.12.2015

  2. Machine Learning Problem (Training Data) f C罗 We always think X and Y are in Euclidean space f:X→Y Manifold learning

  3. Outline What’s manifold and manifold learning? What’s classical methods and its application? Summary and thought. Manifold learning

  4. What’s manifold and manifold learning? • Background • Motivation Manifold learning

  5. Background • Manifold • Manifold = Many + Fold • 流形学习——北大数学系江泽涵教授 Manifold learning

  6. Background • Dimensionality reduction • Manifold learning • The geometry and topology of data manifold • Study on machine learning problem under manifold assumption Manifold learning

  7. PCA LDA Motivation Data:Euclidean space • Traditional dimensionality reduction • Principal Component Analysis(PCA) • Linear Discriminant Analysis(LDA) Manifold learning

  8. Motivation PCA Not Work! Traditional method in manifold Manifold learning

  9. Manifold learning Manifold Dimensionality Reduction It’s an dimensionality reduction method based on manifold space Manifold learning

  10. Manifold learning dimensionality reduction Maintain a certain geometric properties (principle) Low-dimensional embedding / coordinate space High-dimensional data / observation space [王瑞平,流形学习专题介绍] Manifold learning

  11. Outline What’s classical methods and its application? • 等距离映射 • (Isometric MaPPing,ISOMAP) • 局部线性嵌入 • (Locally Linear Embedding,LLE) Manifold learning

  12. Professor of Computer Science and the Princeton Neuroscience Institute Professor in School of Engineering and Applied Science at the University of Pennsylvania The manifold ways of perception[H. Sebastian Seung, Daniel D. Lee,2000,science] Manifold learning

  13. Isometric Feature Mapping J.B. Tenenbaum, V. de Silva, and J. C. Langford. A global geometric framework for nonlinear dimensionality reduction. Science, vol. 290, pp. 2319--2323, 2000. Cited Views: 7424 Manifold learning

  14. Euclidean distance vs. geodesic distance Shortest path approximate geodesic distance Embedding space of dimensionality reduction Isometric Feature Mapping geodesic distance Manifold learning

  15. yi gij xi xj yj Isometric Feature Mapping dij Mapping gij dij • The basic idea • After the reduction, the distance between any two points in low-dimensional space should be same with distance in the original high-dimensional space Manifold learning

  16. Isometric Feature Mapping 16 Manifold learning

  17. Cited Views: 7660 Locally linear Embedding S. T. Roweis and L. K. Saul. Nonlinear dimensionality reduction by locally linear embedding. Science, vol. 290, pp. 2323--2326, 2000. Cited Views: 7660 Manifold learning

  18. Locally linear Embedding approximately a Euclidean space locally Data samples on manifold • The basic idea • Sampling data with low-dimensional manifold is linear Manifold learning

  19. Locally linear Embedding Manifold learning

  20. Locally linear Embedding Manifold learning

  21. Application Data visualization Information retrieval Image process Pattern recognition …… Manifold learning

  22. Outline Summary and Thoughts. Compare ISOMAP and LLE Conclusion Resources and Reference Manifold learning

  23. ISOMAP vs LLE • Similar • They all can keep the geometrical properties of manifold for the same purpose. Manifold learning

  24. ISOMAP vs LLE • Different • Isomap wants to maintain the geodesic distance between any two points while LLE hope to maintain local linear relationship Manifold learning

  25. ISOMAP vs LLE Too Hard Better Isomap:global LLE:local Different Manifold learning

  26. Conclusion • Advantage • Based on the geometry structure of the manifold, can keep the original information • Disadvantages • The assumption of manifold structure • Neighborhood parameter k Manifold learning

  27. Resources • Isomap • http://isomap.stanford.edu/ • LLE • http://www.cs.nyu.edu/~roweis/lle/publications.html • Mani fold Learning Matlab Demo • http://www.math.ucla.edu/~wittman/mani/index.html • Comparison of Manifold Learning methods • http://scikit-learn.org/stable/auto_examples/ manifold/plot_compare_methods.html • http://people.cs.uchicago.edu/~xiaofei/ Manifold learning

  28. Reference • Xiaofei He ：manifold learning http://www.cad.zju.edu.cn/reports/%C1%F7%D0%CE%D1%A7%CF%B0.pdf • Homepage: http://people.cs.uchicago.edu/~xiaofei/ • Joshua B. Tenenbaum, Vin de Silva, John C. Langford. A Global Geometric Framework for Nonlinear Dimensionality Reduction • Sam T. Roweis and Lawrence K. Saul. Nonlinear Dimensionality Reduction by Locally Linear Embedding • Chunguang LI. Manifold Learning and its Application in Pattern Recognition • Yingke Lei. The study of Manifold Learning Algorithms and Their Applications • Ruiping Wang. Manifold Learning presentations • http://blog.csdn.net/xywlpo/article/details/6450632 • http://blog.sciencenet.cn/blog-722391-583413.html Manifold learning

  29. Question and Answer? 2011年11月1日

  30. Thanks! Manifold learning

More Related