out of sample extension and reconstruction on manifolds n.
Skip this Video
Download Presentation
Out-of-Sample Extension and Reconstruction on Manifolds

Loading in 2 Seconds...

play fullscreen
1 / 23

Out-of-Sample Extension and Reconstruction on Manifolds - PowerPoint PPT Presentation

  • Uploaded on

Out-of-Sample Extension and Reconstruction on Manifolds. Bhuwan Dhingra Final Year (Dual Degree) Dept of Electrical Engg. Introduction. An m - dimensional manifold is a topological space which is locally homeomorphic to the m -dimensional E uclidean space

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Out-of-Sample Extension and Reconstruction on Manifolds' - hugh

Download Now An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
out of sample extension and reconstruction on manifolds

Out-of-Sample Extension and Reconstruction on Manifolds

BhuwanDhingraFinal Year (Dual Degree)Dept of Electrical Engg.

  • An m-dimensional manifold is a topological space which is locally homeomorphic to the m-dimensional Euclidean space
  • In this work we consider manifolds which are:
    • Differentiable
    • Embedded in a Euclidean space
    • Generated from a set of m latent variables via a smooth function f
non linear dimensionality reduction
Non-Linear Dimensionality Reduction
  • In practice we only have a sampling on the manifold
  • Y is estimated using a Non-Linear Dimensionality Reduction (NLDR) method
  • Examples of NLDR methods –ISOMAP, LLE, KPCA etc.
  • However most non-linear methods only provide the embedding Y and not the mappings f and g
  • p is the nearest neighbor of x*
  • Only the points in are used for extension and reconstruction
  • The tangent plane is estimated from the k-nearest neighbors of p using PCA
out of sample extension
Out-of-Sample Extension
  • A linear transformation Aeis learnt s.t Y = AeZ
  • Embedding for new point y* = Aez*




out of sample reconstruction
Out-of-Sample Reconstruction




  • A linear transformation Aris learnt s.t Z = ArY
  • Projection of reconstruction on tangent plane z* = Ary*
principal components analysis
Principal Components Analysis
  • Covariance matrix of neighborhood:
  • Let be the eigenvector and eigenvalue matrixes of Mk
  • Then
  • Denote then the projection of a point x onto the tangent plane is given by:
linear transformation
Linear Transformation
  • Y and Z are both centered around and
  • Then Ae =BeRewhere Be and Re are scale and rotation matrices respectively
  • If is the singular value decomposition of ZTY, then
error analysis
Error Analysis
  • We don’t know the true form of f or g to compare our estimates against
  • Reconstruction Error: For a new point x* its reconstruction is computed as , and the reconstruction error is
sampling density
Sampling Density
  • To show: As the sampling density of points on the manifold increases, reconstruction error of a new point goes to 0
  • In a k-NN framework, the sampling density can increase in two ways:
    • k remains fixed and the sampling width decereases
    • remains fixed and
  • We consider the second case
neighborhood parameterization
Neighborhood Parameterization
  • Assume that the first m-canonical vectors of are along
reconstruction error
Reconstruction Error
  • But ArAe = I, hence
reconstruction error1
Reconstruction Error
  • Tyagi, Vural and Frossard (2012) derive conditions on k s.t the angle between and is bounded
  • They show that as
  • Equivalently, where Rm is an aribitrarym-dimensional rotation matrix
  • and
reconstruction error2
Reconstruction Error
  • Hence the reconstruction approaches the projection of x* onto
smoothness of manifold
Smoothness of Manifold
  • If the manifold is smooth then all will be smooth
  • Taylor series of :
  • As because x* will move closer to p
results extension
Results - Extension
  • Out of sample extension on the Swiss-Roll dataset
  • Neighborhood size = 10
results extension1
Results - Extension
  • Out of sample extension on the Japanese flag dataset
  • Neighborhood size = 10
results reconstruction
Results - Reconstruction
  • Reconstructions of ISOMAP faces dataset (698 images)
  • n = 4096, m = 3
  • Neighborhood size = 8
reconstruction error v number of points on manifold
Reconstruction error v Number of Points on Manifold
  • ISOMAP Faces dataset
  • Number of cross validation sets = 5
  • Neighborhood size = [6, 7, 8, 9]