Out of sample extension and reconstruction on manifolds
1 / 23

Out-of-Sample Extension and Reconstruction on Manifolds - PowerPoint PPT Presentation

  • Uploaded on

Out-of-Sample Extension and Reconstruction on Manifolds. Bhuwan Dhingra Final Year (Dual Degree) Dept of Electrical Engg. Introduction. An m - dimensional manifold is a topological space which is locally homeomorphic to the m -dimensional E uclidean space

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about ' Out-of-Sample Extension and Reconstruction on Manifolds' - hugh

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Out of sample extension and reconstruction on manifolds

Out-of-Sample Extension and Reconstruction on Manifolds

BhuwanDhingraFinal Year (Dual Degree)Dept of Electrical Engg.


  • An m-dimensional manifold is a topological space which is locally homeomorphic to the m-dimensional Euclidean space

  • In this work we consider manifolds which are:

    • Differentiable

    • Embedded in a Euclidean space

    • Generated from a set of m latent variables via a smooth function f


n >> m

Non linear dimensionality reduction
Non-Linear Dimensionality Reduction

  • In practice we only have a sampling on the manifold

  • Y is estimated using a Non-Linear Dimensionality Reduction (NLDR) method

  • Examples of NLDR methods –ISOMAP, LLE, KPCA etc.

  • However most non-linear methods only provide the embedding Y and not the mappings f and g


  • p is the nearest neighbor of x*

  • Only the points in are used for extension and reconstruction


  • The tangent plane is estimated from the k-nearest neighbors of p using PCA

Out of sample extension
Out-of-Sample Extension

  • A linear transformation Aeis learnt s.t Y = AeZ

  • Embedding for new point y* = Aez*




Out of sample reconstruction
Out-of-Sample Reconstruction




  • A linear transformation Aris learnt s.t Z = ArY

  • Projection of reconstruction on tangent plane z* = Ary*

Principal components analysis
Principal Components Analysis

  • Covariance matrix of neighborhood:

  • Let be the eigenvector and eigenvalue matrixes of Mk

  • Then

  • Denote then the projection of a point x onto the tangent plane is given by:

Linear transformation
Linear Transformation

  • Y and Z are both centered around and

  • Then Ae =BeRewhere Be and Re are scale and rotation matrices respectively

  • If is the singular value decomposition of ZTY, then

Error analysis
Error Analysis

  • We don’t know the true form of f or g to compare our estimates against

  • Reconstruction Error: For a new point x* its reconstruction is computed as , and the reconstruction error is

Sampling density
Sampling Density

  • To show: As the sampling density of points on the manifold increases, reconstruction error of a new point goes to 0

  • In a k-NN framework, the sampling density can increase in two ways:

    • k remains fixed and the sampling width decereases

    • remains fixed and

  • We consider the second case

Neighborhood parameterization
Neighborhood Parameterization

  • Assume that the first m-canonical vectors of are along

Reconstruction error
Reconstruction Error

  • But ArAe = I, hence

Reconstruction error1
Reconstruction Error

  • Tyagi, Vural and Frossard (2012) derive conditions on k s.t the angle between and is bounded

  • They show that as

  • Equivalently, where Rm is an aribitrarym-dimensional rotation matrix

  • and

Reconstruction error2
Reconstruction Error

  • Hence the reconstruction approaches the projection of x* onto

Smoothness of manifold
Smoothness of Manifold

  • If the manifold is smooth then all will be smooth

  • Taylor series of :

  • As because x* will move closer to p

Results extension
Results - Extension

  • Out of sample extension on the Swiss-Roll dataset

  • Neighborhood size = 10

Results extension1
Results - Extension

  • Out of sample extension on the Japanese flag dataset

  • Neighborhood size = 10

Results reconstruction
Results - Reconstruction

  • Reconstructions of ISOMAP faces dataset (698 images)

  • n = 4096, m = 3

  • Neighborhood size = 8

Reconstruction error v number of points on manifold
Reconstruction error v Number of Points on Manifold

  • ISOMAP Faces dataset

  • Number of cross validation sets = 5

  • Neighborhood size = [6, 7, 8, 9]