applications
Download
Skip this Video
Download Presentation
Applications

Loading in 2 Seconds...

play fullscreen
1 / 30

Applications - PowerPoint PPT Presentation


  • 107 Views
  • Uploaded on

Applications. Shmuel Peleg and Joshua Herman, “ Panoramic Mosaics by Manifold Projection ”, Computer Vision and Pattern Recognition (CVPR), 1997 Wolfgang Heidrich and Hans-Peter Seidel , “ View-independent environment maps ”, SIGGRAPH / Eurographics Workshop on Graphics Hardware, 1998

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Applications' - manning


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
applications

Applications

Shmuel Peleg and Joshua Herman,

“Panoramic Mosaics by Manifold Projection”, Computer Vision and Pattern Recognition (CVPR), 1997

Wolfgang Heidrich and Hans-Peter Seidel,

“View-independent environment maps”, SIGGRAPH / Eurographics Workshop on Graphics Hardware, 1998

Matthew Brand,

“Charting a Manifold”, Mitsubishi tech report, 2003

K.Grochow, S. Martin, A. Hertzmann, and Z. Popovic

Style-based Inverse Kinematics, Siggraph 2004

Richard Souvenir and Robert Pless. Manifold clustering. ICCV, pp. 648-653, 2005

applications using manifolds
Applications using manifolds
  • Many problems can be phrased in manifold terminology
    • Provides an alternative way of viewing the problem
    • Also provides some formalism

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/

applications1
Applications
  • Image-based rendering
  • Environment mapping
  • Animation
  • Surfaces
    • Parameterization
    • Modeling
    • Fitting
      • Consistent parameterization
      • Multiple, different parameterizations

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/

application panoramas
Application: Panoramas
  • Problem statement:
    • Given images from a known camera movement
      • Rotation about camera axis
      • “Push-broom” pan (assumes negligible depth)
    • “Glue” images together into a single image

Rover, nasa.gov

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/

push broom vertical slit camera
Push broom/vertical slit camera

Peleg and Herman

  • Translation of camera
    • Image slit perpendicular to camera motion
      • Need not travel in straight line
    • Depth differences negligible
      • Parallax
    • Manifold is part of ground plane viewed by camera

Direction of travel

Ground plane

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/

camera rotation
Camera rotation
  • Final image can be rendered on a cylinder
    • No parallax
    • Each image samples some number of pixels on cylinder (manifold) image

Peleg and Herman

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/

practical problem
Practical problem
  • How to line up individual images to create one seamless image?
    • Manifold: Final image (3D function RGB on 2D manifold)
    • Charts: Individual images (2D charts)
    • Overlap regions/transition functions: Unknown
      • Assume translation
      • (Account for optical effects of camera)
  • Note: Only works for these two camera motions

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/

general solution
General solution
  • Define a format for the transition function
    • E.g., translation in x,y
  • Define an error metric that measures how well two overlap regions agree
    • E.g., pixel difference
  • Optimize over free parameters in transition function
    • E.g., x,y shift between all pairs of overlapping images

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/

solving for overlaps transitions
Solving for overlaps, transitions
  • Find translation that minimizes pixel differences
    • Find y that minimizes || I0(s) – I1(y(s))||
    • y(s) = s + Ds, where Ds is unknown

0 1

0 1

Ds

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/

final image
Final image
  • Transition functions align images (abstract manifold)
    • Final image colors? (RGB function on manifold)
  • Blend and embedding functions for each chart
    • Embedding function: Original image
    • Blending function: How much to use of each overlapping image
      • Usually favor very short blend regions

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/

application environment mapping
Application: Environment mapping
  • Place scene/model inside sphere
  • Light intensity/color found by intersecting normal with sphere
    • 1-1 mapping between normal direction and sphere
    • Every point on sphere assigned light intensity/color
  • Implementation
    • Store colors in one (or more) texture maps (2D)

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/

parameterization
Parameterization
  • Surface normal (point on sphere) to point in texture map
    • Atlas/local parameterization
  • Desirable properties
    • Even sampling of sphere
      • Adaptive
    • Partition
      • Overlap (mip mapping, continuity)
    • Simple to compute
      • Amenable to GPU implementation

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/

approach 1
Approach 1
  • Single texture map
    • Not unique (poles)
    • Poor sampling
    • Simple to compute

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/

approach ii
Approach II
  • Cube mapping
    • Six charts
    • Discontinuities at edges
    • Sampling better at center of faces than edges
    • Simple (plane) computation
      • Which plane?

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/

approach iii
Approach III
  • Parabolic mapping
    • Chart functions use parabolic function
    • Better sampling
    • Slightly more computation
    • Less-noticeable seams

Heidrich and Hans-Peter Seidel

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/

approach iv
Approach IV
  • Use chart approach
    • Allows for adaptive sampling (more detail where needed)
    • Chart sizes uniform: Tile texture map
    • Include overlap
      • Minimal extra texture map
      • Mip-mapping/down sampling
    • Example: 6 charts like cube
      • Charts extend into others
      • GPU implementation

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/

application animation
Application: Animation
  • Human configuration space lies on a manifold of dimension n embedded in m dimensional space, where n << m
  • Articulated skeleton: over 40 degrees of freedom (shoulders, knees, hips, etc., each 1-3 degrees of rotation)
  • Individual motions (reaching, walking) certainly lie on lower dimension manifolds
    • End-point of reach plus time
  • Shape of manifold of all possible human motions?
    • Who knows?

K.Grochow, S. Martin, A. Hertzmann, and Z. Popovic

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/

overview
Overview
  • Manifold learning
    • Data samples (e.g., motion capture, key frames)
      • Interpolation equals manifold construction
      • Editing equals manifold editing
  • 2D animation example
  • Manifolds in animation

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/

2d illustration
2D illustration
  • Two joint angles
    • Circle X Circle manifold (torus)
  • Animation
    • Repetitive motion
    • Joint angle plot
      • Circle manifold
  • Animation is a 1D manifold embedded in 2D

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/

2d illustration1
2D illustration
  • Two joint angles
    • Circle X Circle manifold (torus)
  • Animation
    • Repetitive motion
    • Joint angle plot
      • Circle manifold
  • Animation is a 1D manifold embedded in 2D

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/

manifold construction
Manifold construction
  • Input: Sample points in Rm
    • E.g., Motion capture sequence, each pose is a data point, m is number of dof of joints
      • 2D example: q,f for each pose
  • Assume data lies on a manifold of dimension n
    • Constraints on manifold shape/geometry (e.g., linear, no self-intersections)
  • Goal: Find/build manifold
    • Multiple manifolds

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/

manifold construction techniques
Manifold construction techniques
  • Principal components analysis (PCA), Independent components analysis (ICA)
    • Hyper planes
  • Support vector machines (SVM)
    • Deformed hyper planes
  • Isomap, Local linear embedding (LLE), Semi-definite embedding (SDE)
    • Planar, cylinder, sphere

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/

isomap lle sde cont
Isomap, LLE, SDE cont.
  • Non-obvious failure modes
    • Circular/repetitive data sets
    • Self-intersections

Modified : 1D embedding

Raw result: 2D embedding

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/

manifold construction as learning
Manifold construction as learning
  • Use K neighbors to define chart domains (Uc)
    • Charts are “squished” Gaussians
      • Center, tangent vectors
  • Find transition functions (affine transformations)
    • Transformation takes tangent vectors into Rn
    • Aligns free vectors with neighbors

Uc

Matthew Brand, “Charting a Manifold”, Mitsubishi tech report, 2003

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/

video segmentation
Video Segmentation

Manifold Clustering, Souvenir, Pless, ICCV 2005

  • Motion Capture Data
    • 175 markers in 3D
      • 525 dimensions
    • 2212 frames
  • Accuracy: 94.8%
  • No domain knowledge
  • No human motion model

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/

uses of animation manifold
Uses of animation manifold
  • General idea:
    • Construct (implicitly or explicitly) a manifold representing valid human poses
    • Create a new animation sequence
      • Foot must touch here, reach here, etc.
        • Not sufficient to constrain all degrees of freedom (dof)
      • Project on to manifold to fill in remaining dof

K. Grochow, S. L. Martin, A. Hertzmann, Z. Popovic,

Style-Based Inverse Kinematics, Siggraph 2004

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/

re sequencing as embedded manifold
Re-sequencing as embedded manifold
  • Goal: Given existing sequence (samples), add more/change samples
  • Assumptions:
    • Samples come from some smooth manifold
    • Some form of interpolation gives new samples on manifold
  • Current approaches: Interpolation between neighboring samples in sequence for given new time

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/

re phrasing problem
Re-phrasing problem
  • Manifold learning or sequence timing provides parameterization/abstract manifold
  • Embed manifold with smooth function
    • Parameterization
    • Use function fitting
  • Re-sequencing: Evaluate embedding function

E(M)

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/

caveats
Caveats
  • What makes animation data difficult?
    • “Distance” loses meaning in >> 10 dimensions
      • Every point equally far away
      • Can’t enumerate
    • Noise
      • Error in capture process
      • Skeleton only approximates human motion
    • Joint angle representation
      • Don’t explicitly deal with manifold, parameterization

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/

summary
Summary
  • Manifolds provide a formalism for breaking a problem into manageable pieces
    • Charts provide local parameterization
      • Planar domains
    • Overlaps: Natural mechanism for moving between parameterizations
      • Blend functions instead of geometric constraints
      • No boundary condition problems
  • Explicitly encapsulating/representing manifold is beneficial
    • Cleaner algorithm specifications

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/

ad