# Out-of-Sample Extension and Reconstruction on Manifolds - PowerPoint PPT Presentation

1 / 23

Out-of-Sample Extension and Reconstruction on Manifolds. Bhuwan Dhingra Final Year (Dual Degree) Dept of Electrical Engg. Introduction. An m - dimensional manifold is a topological space which is locally homeomorphic to the m -dimensional E uclidean space

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

Out-of-Sample Extension and Reconstruction on Manifolds

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

## Out-of-Sample Extension and Reconstruction on Manifolds

BhuwanDhingraFinal Year (Dual Degree)Dept of Electrical Engg.

### Introduction

• An m-dimensional manifold is a topological space which is locally homeomorphic to the m-dimensional Euclidean space

• In this work we consider manifolds which are:

• Differentiable

• Embedded in a Euclidean space

• Generated from a set of m latent variables via a smooth function f

n >> m

### Non-Linear Dimensionality Reduction

• In practice we only have a sampling on the manifold

• Y is estimated using a Non-Linear Dimensionality Reduction (NLDR) method

• Examples of NLDR methods –ISOMAP, LLE, KPCA etc.

• However most non-linear methods only provide the embedding Y and not the mappings f and g

g

y*

x*

f

### Outline

• p is the nearest neighbor of x*

• Only the points in are used for extension and reconstruction

### Outline

• The tangent plane is estimated from the k-nearest neighbors of p using PCA

### Out-of-Sample Extension

• A linear transformation Aeis learnt s.t Y = AeZ

• Embedding for new point y* = Aez*

z*

y*

Ae

### Out-of-Sample Reconstruction

z*

y*

Ar

• A linear transformation Aris learnt s.t Z = ArY

• Projection of reconstruction on tangent plane z* = Ary*

### Principal Components Analysis

• Covariance matrix of neighborhood:

• Let be the eigenvector and eigenvalue matrixes of Mk

• Then

• Denotethen the projection of a point x onto the tangent plane is given by:

### Linear Transformation

• Y and Z are both centered around and

• Then Ae =BeRewhere Be and Re are scale and rotation matrices respectively

• If is the singular value decomposition of ZTY, then

### Error Analysis

• We don’t know the true form of f or g to compare our estimates against

• Reconstruction Error: For a new point x* its reconstruction is computed as , and the reconstruction error is

### Sampling Density

• To show: As the sampling density of points on the manifold increases, reconstruction error of a new point goes to 0

• In a k-NN framework, the sampling density can increase in two ways:

• k remains fixed and the sampling width decereases

• remains fixed and

• We consider the second case

### Neighborhood Parameterization

• Assume that the first m-canonical vectors of are along

### Reconstruction Error

• But ArAe = I, hence

### Reconstruction Error

• Tyagi, Vural and Frossard (2012) derive conditions on k s.t the angle between and is bounded

• They show that as

• Equivalently, where Rm is an aribitrarym-dimensional rotation matrix

• and

### Reconstruction Error

• Hence the reconstruction approaches the projection of x* onto

### Smoothness of Manifold

• If the manifold is smooth then all will be smooth

• Taylor series of :

• As because x* will move closer to p

### Results - Extension

• Out of sample extension on the Swiss-Roll dataset

• Neighborhood size = 10

### Results - Extension

• Out of sample extension on the Japanese flag dataset

• Neighborhood size = 10

### Results - Reconstruction

• Reconstructions of ISOMAP faces dataset (698 images)

• n = 4096, m = 3

• Neighborhood size = 8

### Reconstruction error v Number of Points on Manifold

• ISOMAP Faces dataset

• Number of cross validation sets = 5

• Neighborhood size = [6, 7, 8, 9]