Nonlinear dimensionality reduction frameworks
1 / 22

Nonlinear Dimensionality Reduction Frameworks - PowerPoint PPT Presentation

  • Updated On :

Nonlinear Dimensionality Reduction Frameworks. Rong Xu Chan su Lee. Outline. Intuition of Nonlinear Dimensionality Reduction(NLDR) ISOMAP LLE NLDR in Gait Analysis. Intuition: how does your brain store these pictures?. Brain Representation. Brain Representation. Every pixel?

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Nonlinear Dimensionality Reduction Frameworks' - MikeCarlo

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Outline l.jpg

  • Intuition of Nonlinear Dimensionality Reduction(NLDR)


  • LLE

  • NLDR in Gait Analysis

Brain representation5 l.jpg
Brain Representation

  • Every pixel?

  • Or perceptually meaningful structure?

    • Up-down pose

    • Left-right pose

    • Lighting direction

      So, your brain successfully reduced the high-dimensional inputs to an intrinsically 3-dimensional manifold!

Manifold learning l.jpg
Manifold Learning

  • A manifold is a topological space which is locally Euclidean

  • An example of nonlinear manifold:

Manifold learning10 l.jpg

Discover low dimensional representations (smooth manifold) for data in high dimension.

Linear approaches(PCA, MDS) vs Non-linear approaches (ISOMAP, LLE)

Manifold Learning





Linear approach pca l.jpg
Linear Approach- PCA for data in high dimension.

  • PCA Finds subspace linear projections of input data.

Linear approach mds l.jpg
Linear Approach- MDS for data in high dimension.

  • MDS takes a matrix of pairwise distances and gives a mapping to Rd. It finds an embedding that preserves the interpoint distances, equivalent to PCA when those distance are Euclidean.

  • BUT! PCA and MDS both fail to do embedding with nonlinear data, like swiss roll.

Nonlinear approaches isomap l.jpg

Constructing neighbourhood graph G for data in high dimension.

For each pair of points in G, Computing shortest path distances ---- geodesic distances.

Use Classical MDS with geodesic distances.

Euclidean distance Geodesic distance

Nonlinear Approaches- ISOMAP

Josh. Tenenbaum, Vin de Silva, John langford 2000

Sample points with swiss roll l.jpg
Sample points with Swiss Roll for data in high dimension.

  • Altogether there are 20,000 points in the “Swiss roll” data set. We sample 1000 out of 20,000.

Construct neighborhood graph g l.jpg
Construct neighborhood graph G for data in high dimension.

K- nearest neighborhood (K=7)

DG is 1000 by 1000 (Euclidean) distance matrix of two neighbors (figure A)

Compute all points shortest path in g l.jpg
Compute all-points shortest path in G for data in high dimension.

Now DG is 1000 by 1000 geodesic distance matrix of two arbitrary points along the manifold(figure B)

Use mds to embed graph in r d l.jpg
Use MDS to embed graph in R for data in high dimension.d

  • Find a d-dimensional Euclidean space Y(Figure c) to minimize the cost function:

Linear approach classical mds l.jpg
Linear Approach-classical MDS for data in high dimension.

  • Theorem: For any squared distance matrix ,there exists of points xi and,xj, such that

  • So

Solution l.jpg
Solution for data in high dimension.

  • Y lies in Rd and consists of N points correspondent to the N original points in input space.

Pca md vs isomap l.jpg
PCA, MD vs ISOMAP for data in high dimension.

Isomap advantages l.jpg
Isomap: Advantages for data in high dimension.

  • Nonlinear

  • Globally optimal

    • Still produces globally optimal low-dimensional Euclidean representation even though input space is highly folded, twisted, or curved.

  • Guarantee asymptotically to recover the true dimensionality.

Isomap disadvantages l.jpg
Isomap: Disadvantages for data in high dimension.

  • May not be stable, dependent on topology of data

  • Guaranteed asymptotically to recover geometric structure of nonlinear manifolds

    • As N increases, pairwise distances provide better approximations to geodesics, but cost more computation

    • If N is small, geodesic distances will be very inaccurate.