Manifold Learning Using Geodesic Entropic Graphs

1 / 20

# Manifold Learning Using Geodesic Entropic Graphs - PowerPoint PPT Presentation

Manifold Learning Using Geodesic Entropic Graphs. Research supported in part by: ARO-DARPA MURI DAAD19-02-1-0262. Alfred O. Hero and Jose Costa Dept. EECS, Dept Biomed. Eng., Dept. Statistics University of Michigan - Ann Arbor hero@eecs.umich.edu http://www.eecs.umich.edu/~hero.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

## PowerPoint Slideshow about 'Manifold Learning Using Geodesic Entropic Graphs' - charla

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

### Manifold Learning Using Geodesic Entropic Graphs

Research supported in part by: ARO-DARPA MURI DAAD19-02-1-0262

Alfred O. Hero and Jose Costa

Dept. EECS, Dept Biomed. Eng., Dept. Statistics

University of Michigan - Ann Arbor hero@eecs.umich.edu

http://www.eecs.umich.edu/~hero

• Manifold Learning and Dimension Reduction
• Entropic Graphs
• Examples
1.Dimension Reduction and Pattern Matching
• 128x128 images of three vehicles over 1 deg increments of 360 deg azimuth at 0 deg elevation
• The 3(360)=1080 images evolve on a lower dimensional imbedded manifold in R^(16384)

HMMV

Truck

T62

Courtesy of Center for Imaging Science, JHU

Land VehicleImage Manifold

Quantities

Of Interest

Embediing (extrinsic) Dimension: D

Manifold (intrinsic) Dimension: d

Entropy:

Sampling on a Domain Manifold

Assumption:

is a conformal mapping

2dim manifold

Embedding

Sampling distribution

Sampling

A statistical sample

Background on Manifold Learning
• Manifold intrinsic dimension estimation
• Local KLE, Fukunaga, Olsen (1971)
• Nearest neighbor algorithm, Pettis, Bailey, Jain, Dubes (1971)
• Fractal measures, Camastra and Vinciarelli (2002)
• Packing numbers, Kegl (2002)
• Manifold Reconstruction
• Isomap-MDS, Tenenbaum, de Silva, Langford (2000)
• Locally Linear Embeddings (LLE), Roweiss, Saul (2000)
• Laplacian eigenmaps (LE), Belkin, Niyogi (2002)
• Hessian eigenmaps (HE), Grimes, Donoho (2003)
• Characterization of sampling distributions on manifolds
• Statistics of directional data, Watson (1956), Mardia (1972)
• Data compression on 3D surfaces, Kolarov, Lynch (1997)
• Statistics of shape, Kendall (1984), Kent, Mardia (2001)
MST and Geodesic MST
• For a set of points in D-dimensional Euclidean space, the Euclidean MST with edge power weighting gamma is defined as
• edge lengths of a spanning tree over
• When pairwise distances are geodesic distances on obtain Geodesic MST
• For dense samplings GMST length = MST length
Convergence of Euclidean MST

Beardwood, Halton, Hammersley Theorem:

Convergence Theorem for GMST

Ref: Costa&Hero:TSP2003

Special Cases
• Isometric embedding ( distance preserving)
• Conformal embedding ( angle preserving)
Joint Estimation Algorithm
• Convergence theorem suggests log-linear model
• Use bootstrap resampling to estimate mean MST length and apply LS to jointly estimate slope and intercept from sequence
• Extract d and H from slope and intercept
Bootstrap Estimates of GMST Length

Bootstrap SE bar (83% CI)

Dimension and Entropy Estimates
• From LS fit find:
• Intrinsic dimension estimate
• Alpha-entropy estimate ( )
• Ground truth:
Application to Faces
• Yale face database 2
• Photographic folios of many people’s faces
• Each face folio contains images at 585 different illumination/pose conditions
• Subsampled to 64 by 64 pixels (4096 extrinsic dimensions)
• Objective: determine intrinsic dimension and entropy of a typical face folio
GMST for 3 Face Folios

Ref: Costa&Hero 2003

Conclusions

Advantages of Geodesic Entropic Graph Methods

• Characterizing high dimension sampling distributions
• Standard techniques (histogram, density estimation) fail due to curse of dimensionality
• Entropic graphs can be used to construct consistent estimators of entropy and information divergence
• Robustification to outliers via pruning
• Manifold learning and model reduction
• LLE, LE, HE estimate d by finding local linear representation of manifold
• Entropic graph estimates d from global resampling
• Computational complexity of MST is only n log n
References
• A. O. Hero, B. Ma, O. Michel and J. D. Gorman, “Application of entropic graphs,” IEEE Signal Processing Magazine, Sept 2002.
• H. Neemuchwala, A.O. Hero and P. Carson, “Entropic graphs for image registration,” to appear in European Journal of Signal Processing, 2003.
• J. Costa and A. O. Hero, “Manifold learning with geodesic minimal spanning trees,” accepted in IEEE T-SP (Special Issue on Machine Learning), 2004.
• A. O. Hero, J. Costa and B. Ma, "Convergence rates of minimal graphs with random vertices," submitted to IEEE T-IT, March 2001.
• J. Costa, A. O. Hero and C. Vignat, "On solutions to multivariate maximum alpha-entropy Problems", in Energy Minimization Methods in Computer Vision and Pattern Recognition (EMM-CVPR), Eds. M. Figueiredo, R. Rangagaran, J. Zerubia, Springer-Verlag, 2003