Applications
This presentation is the property of its rightful owner.
Sponsored Links
1 / 30

Applications PowerPoint PPT Presentation


  • 71 Views
  • Uploaded on
  • Presentation posted in: General

Applications. Shmuel Peleg and Joshua Herman, “ Panoramic Mosaics by Manifold Projection ”, Computer Vision and Pattern Recognition (CVPR), 1997 Wolfgang Heidrich and Hans-Peter Seidel , “ View-independent environment maps ”, SIGGRAPH / Eurographics Workshop on Graphics Hardware, 1998

Download Presentation

Applications

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Applications

Applications

Shmuel Peleg and Joshua Herman,

“Panoramic Mosaics by Manifold Projection”, Computer Vision and Pattern Recognition (CVPR), 1997

Wolfgang Heidrich and Hans-Peter Seidel,

“View-independent environment maps”, SIGGRAPH / Eurographics Workshop on Graphics Hardware, 1998

Matthew Brand,

“Charting a Manifold”, Mitsubishi tech report, 2003

K.Grochow, S. Martin, A. Hertzmann, and Z. Popovic

Style-based Inverse Kinematics, Siggraph 2004

Richard Souvenir and Robert Pless. Manifold clustering. ICCV, pp. 648-653, 2005


Applications using manifolds

Applications using manifolds

  • Many problems can be phrased in manifold terminology

    • Provides an alternative way of viewing the problem

    • Also provides some formalism

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/


Applications1

Applications

  • Image-based rendering

  • Environment mapping

  • Animation

  • Surfaces

    • Parameterization

    • Modeling

    • Fitting

      • Consistent parameterization

      • Multiple, different parameterizations

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/


Application panoramas

Application: Panoramas

  • Problem statement:

    • Given images from a known camera movement

      • Rotation about camera axis

      • “Push-broom” pan (assumes negligible depth)

    • “Glue” images together into a single image

Rover, nasa.gov

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/


Push broom vertical slit camera

Push broom/vertical slit camera

Peleg and Herman

  • Translation of camera

    • Image slit perpendicular to camera motion

      • Need not travel in straight line

    • Depth differences negligible

      • Parallax

    • Manifold is part of ground plane viewed by camera

Direction of travel

Ground plane

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/


Camera rotation

Camera rotation

  • Final image can be rendered on a cylinder

    • No parallax

    • Each image samples some number of pixels on cylinder (manifold) image

Peleg and Herman

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/


Practical problem

Practical problem

  • How to line up individual images to create one seamless image?

    • Manifold: Final image (3D function RGB on 2D manifold)

    • Charts: Individual images (2D charts)

    • Overlap regions/transition functions: Unknown

      • Assume translation

      • (Account for optical effects of camera)

  • Note: Only works for these two camera motions

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/


General solution

General solution

  • Define a format for the transition function

    • E.g., translation in x,y

  • Define an error metric that measures how well two overlap regions agree

    • E.g., pixel difference

  • Optimize over free parameters in transition function

    • E.g., x,y shift between all pairs of overlapping images

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/


Solving for overlaps transitions

Solving for overlaps, transitions

  • Find translation that minimizes pixel differences

    • Find y that minimizes || I0(s) – I1(y(s))||

    • y(s) = s + Ds, where Ds is unknown

0 1

0 1

Ds

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/


Final image

Final image

  • Transition functions align images (abstract manifold)

    • Final image colors? (RGB function on manifold)

  • Blend and embedding functions for each chart

    • Embedding function: Original image

    • Blending function: How much to use of each overlapping image

      • Usually favor very short blend regions

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/


Application environment mapping

Application: Environment mapping

  • Place scene/model inside sphere

  • Light intensity/color found by intersecting normal with sphere

    • 1-1 mapping between normal direction and sphere

    • Every point on sphere assigned light intensity/color

  • Implementation

    • Store colors in one (or more) texture maps (2D)

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/


Parameterization

Parameterization

  • Surface normal (point on sphere) to point in texture map

    • Atlas/local parameterization

  • Desirable properties

    • Even sampling of sphere

      • Adaptive

    • Partition

      • Overlap (mip mapping, continuity)

    • Simple to compute

      • Amenable to GPU implementation

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/


Approach 1

Approach 1

  • Single texture map

    • Not unique (poles)

    • Poor sampling

    • Simple to compute

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/


Approach ii

Approach II

  • Cube mapping

    • Six charts

    • Discontinuities at edges

    • Sampling better at center of faces than edges

    • Simple (plane) computation

      • Which plane?

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/


Approach iii

Approach III

  • Parabolic mapping

    • Chart functions use parabolic function

    • Better sampling

    • Slightly more computation

    • Less-noticeable seams

Heidrich and Hans-Peter Seidel

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/


Approach iv

Approach IV

  • Use chart approach

    • Allows for adaptive sampling (more detail where needed)

    • Chart sizes uniform: Tile texture map

    • Include overlap

      • Minimal extra texture map

      • Mip-mapping/down sampling

    • Example: 6 charts like cube

      • Charts extend into others

      • GPU implementation

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/


Application animation

Application: Animation

  • Human configuration space lies on a manifold of dimension n embedded in m dimensional space, where n << m

  • Articulated skeleton: over 40 degrees of freedom (shoulders, knees, hips, etc., each 1-3 degrees of rotation)

  • Individual motions (reaching, walking) certainly lie on lower dimension manifolds

    • End-point of reach plus time

  • Shape of manifold of all possible human motions?

    • Who knows?

K.Grochow, S. Martin, A. Hertzmann, and Z. Popovic

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/


Overview

Overview

  • Manifold learning

    • Data samples (e.g., motion capture, key frames)

      • Interpolation equals manifold construction

      • Editing equals manifold editing

  • 2D animation example

  • Manifolds in animation

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/


2d illustration

2D illustration

  • Two joint angles

    • Circle X Circle manifold (torus)

  • Animation

    • Repetitive motion

    • Joint angle plot

      • Circle manifold

  • Animation is a 1D manifold embedded in 2D

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/


2d illustration1

2D illustration

  • Two joint angles

    • Circle X Circle manifold (torus)

  • Animation

    • Repetitive motion

    • Joint angle plot

      • Circle manifold

  • Animation is a 1D manifold embedded in 2D

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/


Manifold construction

Manifold construction

  • Input: Sample points in Rm

    • E.g., Motion capture sequence, each pose is a data point, m is number of dof of joints

      • 2D example: q,f for each pose

  • Assume data lies on a manifold of dimension n

    • Constraints on manifold shape/geometry (e.g., linear, no self-intersections)

  • Goal: Find/build manifold

    • Multiple manifolds

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/


Manifold construction techniques

Manifold construction techniques

  • Principal components analysis (PCA), Independent components analysis (ICA)

    • Hyper planes

  • Support vector machines (SVM)

    • Deformed hyper planes

  • Isomap, Local linear embedding (LLE), Semi-definite embedding (SDE)

    • Planar, cylinder, sphere

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/


Isomap lle sde cont

Isomap, LLE, SDE cont.

  • Non-obvious failure modes

    • Circular/repetitive data sets

    • Self-intersections

Modified : 1D embedding

Raw result: 2D embedding

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/


Manifold construction as learning

Manifold construction as learning

  • Use K neighbors to define chart domains (Uc)

    • Charts are “squished” Gaussians

      • Center, tangent vectors

  • Find transition functions (affine transformations)

    • Transformation takes tangent vectors into Rn

    • Aligns free vectors with neighbors

Uc

Matthew Brand, “Charting a Manifold”, Mitsubishi tech report, 2003

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/


Video segmentation

Video Segmentation

Manifold Clustering, Souvenir, Pless, ICCV 2005

  • Motion Capture Data

    • 175 markers in 3D

      • 525 dimensions

    • 2212 frames

  • Accuracy: 94.8%

  • No domain knowledge

  • No human motion model

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/


Uses of animation manifold

Uses of animation manifold

  • General idea:

    • Construct (implicitly or explicitly) a manifold representing valid human poses

    • Create a new animation sequence

      • Foot must touch here, reach here, etc.

        • Not sufficient to constrain all degrees of freedom (dof)

      • Project on to manifold to fill in remaining dof

K. Grochow, S. L. Martin, A. Hertzmann, Z. Popovic,

Style-Based Inverse Kinematics, Siggraph 2004

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/


Re sequencing as embedded manifold

Re-sequencing as embedded manifold

  • Goal: Given existing sequence (samples), add more/change samples

  • Assumptions:

    • Samples come from some smooth manifold

    • Some form of interpolation gives new samples on manifold

  • Current approaches: Interpolation between neighboring samples in sequence for given new time

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/


Re phrasing problem

Re-phrasing problem

  • Manifold learning or sequence timing provides parameterization/abstract manifold

  • Embed manifold with smooth function

    • Parameterization

    • Use function fitting

  • Re-sequencing: Evaluate embedding function

E(M)

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/


Caveats

Caveats

  • What makes animation data difficult?

    • “Distance” loses meaning in >> 10 dimensions

      • Every point equally far away

      • Can’t enumerate

    • Noise

      • Error in capture process

      • Skeleton only approximates human motion

    • Joint angle representation

      • Don’t explicitly deal with manifold, parameterization

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/


Summary

Summary

  • Manifolds provide a formalism for breaking a problem into manageable pieces

    • Charts provide local parameterization

      • Planar domains

    • Overlaps: Natural mechanism for moving between parameterizations

      • Blend functions instead of geometric constraints

      • No boundary condition problems

  • Explicitly encapsulating/representing manifold is beneficial

    • Cleaner algorithm specifications

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/


  • Login