manifold learning using geodesic entropic graphs l.
Skip this Video
Download Presentation
Manifold Learning Using Geodesic Entropic Graphs

Loading in 2 Seconds...

play fullscreen
1 / 20

Manifold Learning Using Geodesic Entropic Graphs - PowerPoint PPT Presentation

  • Uploaded on

Manifold Learning Using Geodesic Entropic Graphs. Research supported in part by: ARO-DARPA MURI DAAD19-02-1-0262. Alfred O. Hero and Jose Costa Dept. EECS, Dept Biomed. Eng., Dept. Statistics University of Michigan - Ann Arbor

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Manifold Learning Using Geodesic Entropic Graphs' - charla

Download Now An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
manifold learning using geodesic entropic graphs

Manifold Learning Using Geodesic Entropic Graphs

Research supported in part by: ARO-DARPA MURI DAAD19-02-1-0262

Alfred O. Hero and Jose Costa

Dept. EECS, Dept Biomed. Eng., Dept. Statistics

University of Michigan - Ann Arbor

  • Manifold Learning and Dimension Reduction
  • Entropic Graphs
  • Examples
1 dimension reduction and pattern matching
1.Dimension Reduction and Pattern Matching
  • 128x128 images of three vehicles over 1 deg increments of 360 deg azimuth at 0 deg elevation
  • The 3(360)=1080 images evolve on a lower dimensional imbedded manifold in R^(16384)




Courtesy of Center for Imaging Science, JHU

land vehicle image manifold
Land VehicleImage Manifold


Of Interest

Embediing (extrinsic) Dimension: D

Manifold (intrinsic) Dimension: d


sampling on a domain manifold
Sampling on a Domain Manifold


is a conformal mapping

2dim manifold


Sampling distribution


A statistical sample

background on manifold learning
Background on Manifold Learning
  • Manifold intrinsic dimension estimation
    • Local KLE, Fukunaga, Olsen (1971)
    • Nearest neighbor algorithm, Pettis, Bailey, Jain, Dubes (1971)
    • Fractal measures, Camastra and Vinciarelli (2002)
    • Packing numbers, Kegl (2002)
  • Manifold Reconstruction
    • Isomap-MDS, Tenenbaum, de Silva, Langford (2000)
    • Locally Linear Embeddings (LLE), Roweiss, Saul (2000)
    • Laplacian eigenmaps (LE), Belkin, Niyogi (2002)
    • Hessian eigenmaps (HE), Grimes, Donoho (2003)
  • Characterization of sampling distributions on manifolds
    • Statistics of directional data, Watson (1956), Mardia (1972)
    • Data compression on 3D surfaces, Kolarov, Lynch (1997)
    • Statistics of shape, Kendall (1984), Kent, Mardia (2001)
mst and geodesic mst
MST and Geodesic MST
  • For a set of points in D-dimensional Euclidean space, the Euclidean MST with edge power weighting gamma is defined as
  • edge lengths of a spanning tree over
  • When pairwise distances are geodesic distances on obtain Geodesic MST
  • For dense samplings GMST length = MST length
convergence of euclidean mst
Convergence of Euclidean MST

Beardwood, Halton, Hammersley Theorem:

convergence theorem for gmst
Convergence Theorem for GMST

Ref: Costa&Hero:TSP2003

special cases
Special Cases
  • Isometric embedding ( distance preserving)
  • Conformal embedding ( angle preserving)
joint estimation algorithm
Joint Estimation Algorithm
  • Convergence theorem suggests log-linear model
  • Use bootstrap resampling to estimate mean MST length and apply LS to jointly estimate slope and intercept from sequence
  • Extract d and H from slope and intercept
bootstrap estimates of gmst length
Bootstrap Estimates of GMST Length

Bootstrap SE bar (83% CI)

dimension and entropy estimates
Dimension and Entropy Estimates
  • From LS fit find:
  • Intrinsic dimension estimate
  • Alpha-entropy estimate ( )
    • Ground truth:
application to faces
Application to Faces
  • Yale face database 2
    • Photographic folios of many people’s faces
    • Each face folio contains images at 585 different illumination/pose conditions
    • Subsampled to 64 by 64 pixels (4096 extrinsic dimensions)
  • Objective: determine intrinsic dimension and entropy of a typical face folio
gmst for 3 face folios
GMST for 3 Face Folios

Ref: Costa&Hero 2003


Advantages of Geodesic Entropic Graph Methods

  • Characterizing high dimension sampling distributions
    • Standard techniques (histogram, density estimation) fail due to curse of dimensionality
    • Entropic graphs can be used to construct consistent estimators of entropy and information divergence
    • Robustification to outliers via pruning
  • Manifold learning and model reduction
    • LLE, LE, HE estimate d by finding local linear representation of manifold
    • Entropic graph estimates d from global resampling
    • Computational complexity of MST is only n log n
  • A. O. Hero, B. Ma, O. Michel and J. D. Gorman, “Application of entropic graphs,” IEEE Signal Processing Magazine, Sept 2002.
  • H. Neemuchwala, A.O. Hero and P. Carson, “Entropic graphs for image registration,” to appear in European Journal of Signal Processing, 2003.
  • J. Costa and A. O. Hero, “Manifold learning with geodesic minimal spanning trees,” accepted in IEEE T-SP (Special Issue on Machine Learning), 2004.
  • A. O. Hero, J. Costa and B. Ma, "Convergence rates of minimal graphs with random vertices," submitted to IEEE T-IT, March 2001.
  • J. Costa, A. O. Hero and C. Vignat, "On solutions to multivariate maximum alpha-entropy Problems", in Energy Minimization Methods in Computer Vision and Pattern Recognition (EMM-CVPR), Eds. M. Figueiredo, R. Rangagaran, J. Zerubia, Springer-Verlag, 2003