Out-of-Sample Extension and Reconstruction on Manifolds

Download Presentation

Out-of-Sample Extension and Reconstruction on Manifolds

Loading in 2 Seconds...

- 70 Views
- Uploaded on
- Presentation posted in: General

Out-of-Sample Extension and Reconstruction on Manifolds

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Out-of-Sample Extension and Reconstruction on Manifolds

BhuwanDhingraFinal Year (Dual Degree)Dept of Electrical Engg.

- An m-dimensional manifold is a topological space which is locally homeomorphic to the m-dimensional Euclidean space
- In this work we consider manifolds which are:
- Differentiable
- Embedded in a Euclidean space
- Generated from a set of m latent variables via a smooth function f

n >> m

- In practice we only have a sampling on the manifold
- Y is estimated using a Non-Linear Dimensionality Reduction (NLDR) method
- Examples of NLDR methods –ISOMAP, LLE, KPCA etc.
- However most non-linear methods only provide the embedding Y and not the mappings f and g

g

y*

x*

f

- p is the nearest neighbor of x*
- Only the points in are used for extension and reconstruction

- The tangent plane is estimated from the k-nearest neighbors of p using PCA

- A linear transformation Aeis learnt s.t Y = AeZ
- Embedding for new point y* = Aez*

z*

y*

Ae

z*

y*

Ar

- A linear transformation Aris learnt s.t Z = ArY
- Projection of reconstruction on tangent plane z* = Ary*

- Covariance matrix of neighborhood:
- Let be the eigenvector and eigenvalue matrixes of Mk
- Then
- Denotethen the projection of a point x onto the tangent plane is given by:

- Y and Z are both centered around and
- Then Ae =BeRewhere Be and Re are scale and rotation matrices respectively
- If is the singular value decomposition of ZTY, then

- We don’t know the true form of f or g to compare our estimates against
- Reconstruction Error: For a new point x* its reconstruction is computed as , and the reconstruction error is

- To show: As the sampling density of points on the manifold increases, reconstruction error of a new point goes to 0
- In a k-NN framework, the sampling density can increase in two ways:
- k remains fixed and the sampling width decereases
- remains fixed and

- We consider the second case

- Assume that the first m-canonical vectors of are along

- But ArAe = I, hence

- Tyagi, Vural and Frossard (2012) derive conditions on k s.t the angle between and is bounded
- They show that as
- Equivalently, where Rm is an aribitrarym-dimensional rotation matrix
- and

- Hence the reconstruction approaches the projection of x* onto

- If the manifold is smooth then all will be smooth
- Taylor series of :
- As because x* will move closer to p

- Out of sample extension on the Swiss-Roll dataset
- Neighborhood size = 10

- Out of sample extension on the Japanese flag dataset
- Neighborhood size = 10

- Reconstructions of ISOMAP faces dataset (698 images)
- n = 4096, m = 3
- Neighborhood size = 8

- ISOMAP Faces dataset
- Number of cross validation sets = 5
- Neighborhood size = [6, 7, 8, 9]