Out of sample extension and reconstruction on manifolds
This presentation is the property of its rightful owner.
Sponsored Links
1 / 23

Out-of-Sample Extension and Reconstruction on Manifolds PowerPoint PPT Presentation


  • 59 Views
  • Uploaded on
  • Presentation posted in: General

Out-of-Sample Extension and Reconstruction on Manifolds. Bhuwan Dhingra Final Year (Dual Degree) Dept of Electrical Engg. Introduction. An m - dimensional manifold is a topological space which is locally homeomorphic to the m -dimensional E uclidean space

Download Presentation

Out-of-Sample Extension and Reconstruction on Manifolds

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Out of sample extension and reconstruction on manifolds

Out-of-Sample Extension and Reconstruction on Manifolds

BhuwanDhingraFinal Year (Dual Degree)Dept of Electrical Engg.


Introduction

Introduction

  • An m-dimensional manifold is a topological space which is locally homeomorphic to the m-dimensional Euclidean space

  • In this work we consider manifolds which are:

    • Differentiable

    • Embedded in a Euclidean space

    • Generated from a set of m latent variables via a smooth function f


Introduction1

Introduction

n >> m


Non linear dimensionality reduction

Non-Linear Dimensionality Reduction

  • In practice we only have a sampling on the manifold

  • Y is estimated using a Non-Linear Dimensionality Reduction (NLDR) method

  • Examples of NLDR methods –ISOMAP, LLE, KPCA etc.

  • However most non-linear methods only provide the embedding Y and not the mappings f and g


Problem statement

Problem Statement

g

y*

x*

f


Outline

Outline

  • p is the nearest neighbor of x*

  • Only the points in are used for extension and reconstruction


Outline1

Outline

  • The tangent plane is estimated from the k-nearest neighbors of p using PCA


Out of sample extension

Out-of-Sample Extension

  • A linear transformation Aeis learnt s.t Y = AeZ

  • Embedding for new point y* = Aez*

z*

y*

Ae


Out of sample reconstruction

Out-of-Sample Reconstruction

z*

y*

Ar

  • A linear transformation Aris learnt s.t Z = ArY

  • Projection of reconstruction on tangent plane z* = Ary*


Principal components analysis

Principal Components Analysis

  • Covariance matrix of neighborhood:

  • Let be the eigenvector and eigenvalue matrixes of Mk

  • Then

  • Denotethen the projection of a point x onto the tangent plane is given by:


Linear transformation

Linear Transformation

  • Y and Z are both centered around and

  • Then Ae =BeRewhere Be and Re are scale and rotation matrices respectively

  • If is the singular value decomposition of ZTY, then


Final estimates

Final Estimates


Error analysis

Error Analysis

  • We don’t know the true form of f or g to compare our estimates against

  • Reconstruction Error: For a new point x* its reconstruction is computed as , and the reconstruction error is


Sampling density

Sampling Density

  • To show: As the sampling density of points on the manifold increases, reconstruction error of a new point goes to 0

  • In a k-NN framework, the sampling density can increase in two ways:

    • k remains fixed and the sampling width decereases

    • remains fixed and

  • We consider the second case


Neighborhood parameterization

Neighborhood Parameterization

  • Assume that the first m-canonical vectors of are along


Reconstruction error

Reconstruction Error

  • But ArAe = I, hence


Reconstruction error1

Reconstruction Error

  • Tyagi, Vural and Frossard (2012) derive conditions on k s.t the angle between and is bounded

  • They show that as

  • Equivalently, where Rm is an aribitrarym-dimensional rotation matrix

  • and


Reconstruction error2

Reconstruction Error

  • Hence the reconstruction approaches the projection of x* onto


Smoothness of manifold

Smoothness of Manifold

  • If the manifold is smooth then all will be smooth

  • Taylor series of :

  • As because x* will move closer to p


Results extension

Results - Extension

  • Out of sample extension on the Swiss-Roll dataset

  • Neighborhood size = 10


Results extension1

Results - Extension

  • Out of sample extension on the Japanese flag dataset

  • Neighborhood size = 10


Results reconstruction

Results - Reconstruction

  • Reconstructions of ISOMAP faces dataset (698 images)

  • n = 4096, m = 3

  • Neighborhood size = 8


Reconstruction error v number of points on manifold

Reconstruction error v Number of Points on Manifold

  • ISOMAP Faces dataset

  • Number of cross validation sets = 5

  • Neighborhood size = [6, 7, 8, 9]


  • Login