1 / 23

# Out-of-Sample Extension and Reconstruction on Manifolds - PowerPoint PPT Presentation

Out-of-Sample Extension and Reconstruction on Manifolds. Bhuwan Dhingra Final Year (Dual Degree) Dept of Electrical Engg. Introduction. An m - dimensional manifold is a topological space which is locally homeomorphic to the m -dimensional E uclidean space

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

## PowerPoint Slideshow about ' Out-of-Sample Extension and Reconstruction on Manifolds' - hugh

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

### Out-of-Sample Extension and Reconstruction on Manifolds

BhuwanDhingraFinal Year (Dual Degree)Dept of Electrical Engg.

• An m-dimensional manifold is a topological space which is locally homeomorphic to the m-dimensional Euclidean space

• In this work we consider manifolds which are:

• Differentiable

• Embedded in a Euclidean space

• Generated from a set of m latent variables via a smooth function f

n >> m

• In practice we only have a sampling on the manifold

• Y is estimated using a Non-Linear Dimensionality Reduction (NLDR) method

• Examples of NLDR methods –ISOMAP, LLE, KPCA etc.

• However most non-linear methods only provide the embedding Y and not the mappings f and g

g

y*

x*

f

• p is the nearest neighbor of x*

• Only the points in are used for extension and reconstruction

• The tangent plane is estimated from the k-nearest neighbors of p using PCA

• A linear transformation Aeis learnt s.t Y = AeZ

• Embedding for new point y* = Aez*

z*

y*

Ae

z*

y*

Ar

• A linear transformation Aris learnt s.t Z = ArY

• Projection of reconstruction on tangent plane z* = Ary*

• Covariance matrix of neighborhood:

• Let be the eigenvector and eigenvalue matrixes of Mk

• Then

• Denote then the projection of a point x onto the tangent plane is given by:

• Y and Z are both centered around and

• Then Ae =BeRewhere Be and Re are scale and rotation matrices respectively

• If is the singular value decomposition of ZTY, then

• We don’t know the true form of f or g to compare our estimates against

• Reconstruction Error: For a new point x* its reconstruction is computed as , and the reconstruction error is

• To show: As the sampling density of points on the manifold increases, reconstruction error of a new point goes to 0

• In a k-NN framework, the sampling density can increase in two ways:

• k remains fixed and the sampling width decereases

• remains fixed and

• We consider the second case

• Assume that the first m-canonical vectors of are along

• But ArAe = I, hence

• Tyagi, Vural and Frossard (2012) derive conditions on k s.t the angle between and is bounded

• They show that as

• Equivalently, where Rm is an aribitrarym-dimensional rotation matrix

• and

• Hence the reconstruction approaches the projection of x* onto

• If the manifold is smooth then all will be smooth

• Taylor series of :

• As because x* will move closer to p

• Out of sample extension on the Swiss-Roll dataset

• Neighborhood size = 10

• Out of sample extension on the Japanese flag dataset

• Neighborhood size = 10

• Reconstructions of ISOMAP faces dataset (698 images)

• n = 4096, m = 3

• Neighborhood size = 8

• ISOMAP Faces dataset

• Number of cross validation sets = 5

• Neighborhood size = [6, 7, 8, 9]