1 / 11

Manifold Learning: ISOMAP Alan O'Connor April 29, 2008

Manifold Learning: ISOMAP Alan O'Connor April 29, 2008. Manifold Learning. Dimensionality Reduction. Multidimensional Scaling. Convert a matrix of distances between vectors || x i – x j || in R n into a matrix of inner products.

cate
Download Presentation

Manifold Learning: ISOMAP Alan O'Connor April 29, 2008

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Manifold Learning: ISOMAP Alan O'Connor April 29, 2008

  2. Manifold Learning Dimensionality Reduction

  3. Multidimensional Scaling Convert a matrix of distances between vectors || xi – xj || in Rn into a matrix of inner products. Use the eigen-decomposition to find the coordinates of an optimal d-dimensional embedding.

  4. Kernel ISOMAP Algorithm 1. Build a graph with edges connecting datapts. within an epsilon neighborhood. 2. Use Dijkstra's or Floyd's algorithm to compute approximate geodesic distances. 3. Apply kernel PCA for K given by the centered matrix of squared geodesic distances. 4. Project test points onto principal components as in kernel PCA.

  5. Example “Swiss Roll” dataset 3D data 2D coord chart Error vs. dimensionality of coordinate chart

  6. Convergence Thm.

  7. Convergence Thm. Theorem (Bernstein et al, 2000). Let M be a compact submanifold of Euclidean space, and let {xi } be a set of n data points in M. Given a graph G on {x_i} i=1...n, numbers 0 < λ1 , λ2 < 1, and positive numbers k, K and δ , and supposing that 1. The graph G contains every edge xy for which | x − y | ≤ k; 2. All edges of G have length | x − y | ≤ K ; 3. For ∀m ∈ M, ∃x ∈ {xi} s.t. dM(m, x) < δ ; 4. The submanifold M is geodesically convex; 5. k < s0 , where s0 is the minimum branch separation of M ; 6. K ≤ 2/π r0 sqrt{24 λ1} , for r0 the minimum radius of curvature of M ; 7. δ ≤ λ2 k /4, then it follows that for ∀x, y ∈ M , we have the inequality ( 1 − λ1 ) dM( x, y ) ≤ dG( x, y ) ≤ ( 1 + λ2 ) dM( x, y ).

  8. Limitations of ISOMAP 3D data: Sphere 2D coord chart Error vs. dimensionality of coordinate chart

  9. Limitations of ISOMAP 3D data: Hemisphere 2D coord chart Error vs. dimensionality of coordinate chart

  10. Conclusion Despite the technical limitation of ISOMAP's applicability only to intrinsically flat manifolds, the literature abounds with examples of its application to more general problems. It is a good example of the application of kernel PCA methods to real problems.

  11. References H. Choi and S. Choi. Kernel Isomap on Noisy Manifold. Proc. CDL, 2005. T.F. Cox and M.A.A Cox. Multidimensional Scaling, 1994. M. Bernstein, V. de Silva, J.C. Langford and J.B. Tenenbaum. Graph approximations to geodesics on embedded manifolds,Technical report, 2000. J. B. Tenenbaum, V. de Silva and J. C. Langford. A Global Geometric Framework for Nonlinear Dimensionality Reduction,Science, 2000 .

More Related