- 107 Views
- Uploaded on

Download Presentation
## PowerPoint Slideshow about 'Applications' - manning

**An Image/Link below is provided (as is) to download presentation**

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

### Applications

Shmuel Peleg and Joshua Herman,

“Panoramic Mosaics by Manifold Projection”, Computer Vision and Pattern Recognition (CVPR), 1997

Wolfgang Heidrich and Hans-Peter Seidel,

“View-independent environment maps”, SIGGRAPH / Eurographics Workshop on Graphics Hardware, 1998

Matthew Brand,

“Charting a Manifold”, Mitsubishi tech report, 2003

K.Grochow, S. Martin, A. Hertzmann, and Z. Popovic

Style-based Inverse Kinematics, Siggraph 2004

Richard Souvenir and Robert Pless. Manifold clustering. ICCV, pp. 648-653, 2005

Applications using manifolds

- Many problems can be phrased in manifold terminology
- Provides an alternative way of viewing the problem
- Also provides some formalism

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/

Applications

- Image-based rendering
- Environment mapping
- Animation
- Surfaces
- Parameterization
- Modeling
- Fitting
- Consistent parameterization
- Multiple, different parameterizations

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/

Application: Panoramas

- Problem statement:
- Given images from a known camera movement
- Rotation about camera axis
- “Push-broom” pan (assumes negligible depth)
- “Glue” images together into a single image

Rover, nasa.gov

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/

Push broom/vertical slit camera

Peleg and Herman

- Translation of camera
- Image slit perpendicular to camera motion
- Need not travel in straight line
- Depth differences negligible
- Parallax
- Manifold is part of ground plane viewed by camera

Direction of travel

Ground plane

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/

Camera rotation

- Final image can be rendered on a cylinder
- No parallax
- Each image samples some number of pixels on cylinder (manifold) image

Peleg and Herman

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/

Practical problem

- How to line up individual images to create one seamless image?
- Manifold: Final image (3D function RGB on 2D manifold)
- Charts: Individual images (2D charts)
- Overlap regions/transition functions: Unknown
- Assume translation
- (Account for optical effects of camera)
- Note: Only works for these two camera motions

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/

General solution

- Define a format for the transition function
- E.g., translation in x,y
- Define an error metric that measures how well two overlap regions agree
- E.g., pixel difference
- Optimize over free parameters in transition function
- E.g., x,y shift between all pairs of overlapping images

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/

Solving for overlaps, transitions

- Find translation that minimizes pixel differences
- Find y that minimizes || I0(s) – I1(y(s))||
- y(s) = s + Ds, where Ds is unknown

0 1

0 1

Ds

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/

Final image

- Transition functions align images (abstract manifold)
- Final image colors? (RGB function on manifold)
- Blend and embedding functions for each chart
- Embedding function: Original image
- Blending function: How much to use of each overlapping image
- Usually favor very short blend regions

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/

Application: Environment mapping

- Place scene/model inside sphere
- Light intensity/color found by intersecting normal with sphere
- 1-1 mapping between normal direction and sphere
- Every point on sphere assigned light intensity/color
- Implementation
- Store colors in one (or more) texture maps (2D)

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/

Parameterization

- Surface normal (point on sphere) to point in texture map
- Atlas/local parameterization
- Desirable properties
- Even sampling of sphere
- Adaptive
- Partition
- Overlap (mip mapping, continuity)
- Simple to compute
- Amenable to GPU implementation

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/

Approach 1

- Single texture map
- Not unique (poles)
- Poor sampling
- Simple to compute

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/

Approach II

- Cube mapping
- Six charts
- Discontinuities at edges
- Sampling better at center of faces than edges
- Simple (plane) computation
- Which plane?

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/

Approach III

- Parabolic mapping
- Chart functions use parabolic function
- Better sampling
- Slightly more computation
- Less-noticeable seams

Heidrich and Hans-Peter Seidel

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/

Approach IV

- Use chart approach
- Allows for adaptive sampling (more detail where needed)
- Chart sizes uniform: Tile texture map
- Include overlap
- Minimal extra texture map
- Mip-mapping/down sampling
- Example: 6 charts like cube
- Charts extend into others
- GPU implementation

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/

Application: Animation

- Human configuration space lies on a manifold of dimension n embedded in m dimensional space, where n << m
- Articulated skeleton: over 40 degrees of freedom (shoulders, knees, hips, etc., each 1-3 degrees of rotation)
- Individual motions (reaching, walking) certainly lie on lower dimension manifolds
- End-point of reach plus time
- Shape of manifold of all possible human motions?
- Who knows?

K.Grochow, S. Martin, A. Hertzmann, and Z. Popovic

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/

Overview

- Manifold learning
- Data samples (e.g., motion capture, key frames)
- Interpolation equals manifold construction
- Editing equals manifold editing
- 2D animation example
- Manifolds in animation

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/

2D illustration

- Two joint angles
- Circle X Circle manifold (torus)
- Animation
- Repetitive motion
- Joint angle plot
- Circle manifold
- Animation is a 1D manifold embedded in 2D

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/

2D illustration

- Two joint angles
- Circle X Circle manifold (torus)
- Animation
- Repetitive motion
- Joint angle plot
- Circle manifold
- Animation is a 1D manifold embedded in 2D

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/

Manifold construction

- Input: Sample points in Rm
- E.g., Motion capture sequence, each pose is a data point, m is number of dof of joints
- 2D example: q,f for each pose
- Assume data lies on a manifold of dimension n
- Constraints on manifold shape/geometry (e.g., linear, no self-intersections)
- Goal: Find/build manifold
- Multiple manifolds

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/

Manifold construction techniques

- Principal components analysis (PCA), Independent components analysis (ICA)
- Hyper planes
- Support vector machines (SVM)
- Deformed hyper planes
- Isomap, Local linear embedding (LLE), Semi-definite embedding (SDE)
- Planar, cylinder, sphere

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/

Isomap, LLE, SDE cont.

- Non-obvious failure modes
- Circular/repetitive data sets
- Self-intersections

Modified : 1D embedding

Raw result: 2D embedding

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/

Manifold construction as learning

- Use K neighbors to define chart domains (Uc)
- Charts are “squished” Gaussians
- Center, tangent vectors
- Find transition functions (affine transformations)
- Transformation takes tangent vectors into Rn
- Aligns free vectors with neighbors

Uc

Matthew Brand, “Charting a Manifold”, Mitsubishi tech report, 2003

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/

Video Segmentation

Manifold Clustering, Souvenir, Pless, ICCV 2005

- Motion Capture Data
- 175 markers in 3D
- 525 dimensions
- 2212 frames
- Accuracy: 94.8%
- No domain knowledge
- No human motion model

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/

Uses of animation manifold

- General idea:
- Construct (implicitly or explicitly) a manifold representing valid human poses
- Create a new animation sequence
- Foot must touch here, reach here, etc.
- Not sufficient to constrain all degrees of freedom (dof)
- Project on to manifold to fill in remaining dof

K. Grochow, S. L. Martin, A. Hertzmann, Z. Popovic,

Style-Based Inverse Kinematics, Siggraph 2004

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/

Re-sequencing as embedded manifold

- Goal: Given existing sequence (samples), add more/change samples
- Assumptions:
- Samples come from some smooth manifold
- Some form of interpolation gives new samples on manifold
- Current approaches: Interpolation between neighboring samples in sequence for given new time

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/

Re-phrasing problem

- Manifold learning or sequence timing provides parameterization/abstract manifold
- Embed manifold with smooth function
- Parameterization
- Use function fitting
- Re-sequencing: Evaluate embedding function

E(M)

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/

Caveats

- What makes animation data difficult?
- “Distance” loses meaning in >> 10 dimensions
- Every point equally far away
- Can’t enumerate
- Noise
- Error in capture process
- Skeleton only approximates human motion
- Joint angle representation
- Don’t explicitly deal with manifold, parameterization

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/

Summary

- Manifolds provide a formalism for breaking a problem into manageable pieces
- Charts provide local parameterization
- Planar domains
- Overlaps: Natural mechanism for moving between parameterizations
- Blend functions instead of geometric constraints
- No boundary condition problems
- Explicitly encapsulating/representing manifold is beneficial
- Cleaner algorithm specifications

Siggraph 2006, 7/31/2006

www.cs.wustl.edu/~cmg/

Download Presentation

Connecting to Server..