115 Views

Download Presentation
##### Applications

**An Image/Link below is provided (as is) to download presentation**

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -

**Applications**Shmuel Peleg and Joshua Herman, “Panoramic Mosaics by Manifold Projection”, Computer Vision and Pattern Recognition (CVPR), 1997 Wolfgang Heidrich and Hans-Peter Seidel, “View-independent environment maps”, SIGGRAPH / Eurographics Workshop on Graphics Hardware, 1998 Matthew Brand, “Charting a Manifold”, Mitsubishi tech report, 2003 K.Grochow, S. Martin, A. Hertzmann, and Z. Popovic Style-based Inverse Kinematics, Siggraph 2004 Richard Souvenir and Robert Pless. Manifold clustering. ICCV, pp. 648-653, 2005**Applications using manifolds**• Many problems can be phrased in manifold terminology • Provides an alternative way of viewing the problem • Also provides some formalism Siggraph 2006, 7/31/2006 www.cs.wustl.edu/~cmg/**Applications**• Image-based rendering • Environment mapping • Animation • Surfaces • Parameterization • Modeling • Fitting • Consistent parameterization • Multiple, different parameterizations Siggraph 2006, 7/31/2006 www.cs.wustl.edu/~cmg/**Application: Panoramas**• Problem statement: • Given images from a known camera movement • Rotation about camera axis • “Push-broom” pan (assumes negligible depth) • “Glue” images together into a single image Rover, nasa.gov Siggraph 2006, 7/31/2006 www.cs.wustl.edu/~cmg/**Push broom/vertical slit camera**Peleg and Herman • Translation of camera • Image slit perpendicular to camera motion • Need not travel in straight line • Depth differences negligible • Parallax • Manifold is part of ground plane viewed by camera Direction of travel Ground plane Siggraph 2006, 7/31/2006 www.cs.wustl.edu/~cmg/**Camera rotation**• Final image can be rendered on a cylinder • No parallax • Each image samples some number of pixels on cylinder (manifold) image Peleg and Herman Siggraph 2006, 7/31/2006 www.cs.wustl.edu/~cmg/**Practical problem**• How to line up individual images to create one seamless image? • Manifold: Final image (3D function RGB on 2D manifold) • Charts: Individual images (2D charts) • Overlap regions/transition functions: Unknown • Assume translation • (Account for optical effects of camera) • Note: Only works for these two camera motions Siggraph 2006, 7/31/2006 www.cs.wustl.edu/~cmg/**General solution**• Define a format for the transition function • E.g., translation in x,y • Define an error metric that measures how well two overlap regions agree • E.g., pixel difference • Optimize over free parameters in transition function • E.g., x,y shift between all pairs of overlapping images Siggraph 2006, 7/31/2006 www.cs.wustl.edu/~cmg/**Solving for overlaps, transitions**• Find translation that minimizes pixel differences • Find y that minimizes || I0(s) – I1(y(s))|| • y(s) = s + Ds, where Ds is unknown 0 1 0 1 Ds Siggraph 2006, 7/31/2006 www.cs.wustl.edu/~cmg/**Final image**• Transition functions align images (abstract manifold) • Final image colors? (RGB function on manifold) • Blend and embedding functions for each chart • Embedding function: Original image • Blending function: How much to use of each overlapping image • Usually favor very short blend regions Siggraph 2006, 7/31/2006 www.cs.wustl.edu/~cmg/**Application: Environment mapping**• Place scene/model inside sphere • Light intensity/color found by intersecting normal with sphere • 1-1 mapping between normal direction and sphere • Every point on sphere assigned light intensity/color • Implementation • Store colors in one (or more) texture maps (2D) Siggraph 2006, 7/31/2006 www.cs.wustl.edu/~cmg/**Parameterization**• Surface normal (point on sphere) to point in texture map • Atlas/local parameterization • Desirable properties • Even sampling of sphere • Adaptive • Partition • Overlap (mip mapping, continuity) • Simple to compute • Amenable to GPU implementation Siggraph 2006, 7/31/2006 www.cs.wustl.edu/~cmg/**Approach 1**• Single texture map • Not unique (poles) • Poor sampling • Simple to compute Siggraph 2006, 7/31/2006 www.cs.wustl.edu/~cmg/**Approach II**• Cube mapping • Six charts • Discontinuities at edges • Sampling better at center of faces than edges • Simple (plane) computation • Which plane? Siggraph 2006, 7/31/2006 www.cs.wustl.edu/~cmg/**Approach III**• Parabolic mapping • Chart functions use parabolic function • Better sampling • Slightly more computation • Less-noticeable seams Heidrich and Hans-Peter Seidel Siggraph 2006, 7/31/2006 www.cs.wustl.edu/~cmg/**Approach IV**• Use chart approach • Allows for adaptive sampling (more detail where needed) • Chart sizes uniform: Tile texture map • Include overlap • Minimal extra texture map • Mip-mapping/down sampling • Example: 6 charts like cube • Charts extend into others • GPU implementation Siggraph 2006, 7/31/2006 www.cs.wustl.edu/~cmg/**Application: Animation**• Human configuration space lies on a manifold of dimension n embedded in m dimensional space, where n << m • Articulated skeleton: over 40 degrees of freedom (shoulders, knees, hips, etc., each 1-3 degrees of rotation) • Individual motions (reaching, walking) certainly lie on lower dimension manifolds • End-point of reach plus time • Shape of manifold of all possible human motions? • Who knows? K.Grochow, S. Martin, A. Hertzmann, and Z. Popovic Siggraph 2006, 7/31/2006 www.cs.wustl.edu/~cmg/**Overview**• Manifold learning • Data samples (e.g., motion capture, key frames) • Interpolation equals manifold construction • Editing equals manifold editing • 2D animation example • Manifolds in animation Siggraph 2006, 7/31/2006 www.cs.wustl.edu/~cmg/**2D illustration**• Two joint angles • Circle X Circle manifold (torus) • Animation • Repetitive motion • Joint angle plot • Circle manifold • Animation is a 1D manifold embedded in 2D Siggraph 2006, 7/31/2006 www.cs.wustl.edu/~cmg/**2D illustration**• Two joint angles • Circle X Circle manifold (torus) • Animation • Repetitive motion • Joint angle plot • Circle manifold • Animation is a 1D manifold embedded in 2D Siggraph 2006, 7/31/2006 www.cs.wustl.edu/~cmg/**Manifold construction**• Input: Sample points in Rm • E.g., Motion capture sequence, each pose is a data point, m is number of dof of joints • 2D example: q,f for each pose • Assume data lies on a manifold of dimension n • Constraints on manifold shape/geometry (e.g., linear, no self-intersections) • Goal: Find/build manifold • Multiple manifolds Siggraph 2006, 7/31/2006 www.cs.wustl.edu/~cmg/**Manifold construction techniques**• Principal components analysis (PCA), Independent components analysis (ICA) • Hyper planes • Support vector machines (SVM) • Deformed hyper planes • Isomap, Local linear embedding (LLE), Semi-definite embedding (SDE) • Planar, cylinder, sphere Siggraph 2006, 7/31/2006 www.cs.wustl.edu/~cmg/**Isomap, LLE, SDE cont.**• Non-obvious failure modes • Circular/repetitive data sets • Self-intersections Modified : 1D embedding Raw result: 2D embedding Siggraph 2006, 7/31/2006 www.cs.wustl.edu/~cmg/**Manifold construction as learning**• Use K neighbors to define chart domains (Uc) • Charts are “squished” Gaussians • Center, tangent vectors • Find transition functions (affine transformations) • Transformation takes tangent vectors into Rn • Aligns free vectors with neighbors Uc Matthew Brand, “Charting a Manifold”, Mitsubishi tech report, 2003 Siggraph 2006, 7/31/2006 www.cs.wustl.edu/~cmg/**Video Segmentation**Manifold Clustering, Souvenir, Pless, ICCV 2005 • Motion Capture Data • 175 markers in 3D • 525 dimensions • 2212 frames • Accuracy: 94.8% • No domain knowledge • No human motion model Siggraph 2006, 7/31/2006 www.cs.wustl.edu/~cmg/**Uses of animation manifold**• General idea: • Construct (implicitly or explicitly) a manifold representing valid human poses • Create a new animation sequence • Foot must touch here, reach here, etc. • Not sufficient to constrain all degrees of freedom (dof) • Project on to manifold to fill in remaining dof K. Grochow, S. L. Martin, A. Hertzmann, Z. Popovic, Style-Based Inverse Kinematics, Siggraph 2004 Siggraph 2006, 7/31/2006 www.cs.wustl.edu/~cmg/**Re-sequencing as embedded manifold**• Goal: Given existing sequence (samples), add more/change samples • Assumptions: • Samples come from some smooth manifold • Some form of interpolation gives new samples on manifold • Current approaches: Interpolation between neighboring samples in sequence for given new time Siggraph 2006, 7/31/2006 www.cs.wustl.edu/~cmg/**Re-phrasing problem**• Manifold learning or sequence timing provides parameterization/abstract manifold • Embed manifold with smooth function • Parameterization • Use function fitting • Re-sequencing: Evaluate embedding function E(M) Siggraph 2006, 7/31/2006 www.cs.wustl.edu/~cmg/**Caveats**• What makes animation data difficult? • “Distance” loses meaning in >> 10 dimensions • Every point equally far away • Can’t enumerate • Noise • Error in capture process • Skeleton only approximates human motion • Joint angle representation • Don’t explicitly deal with manifold, parameterization Siggraph 2006, 7/31/2006 www.cs.wustl.edu/~cmg/**Summary**• Manifolds provide a formalism for breaking a problem into manageable pieces • Charts provide local parameterization • Planar domains • Overlaps: Natural mechanism for moving between parameterizations • Blend functions instead of geometric constraints • No boundary condition problems • Explicitly encapsulating/representing manifold is beneficial • Cleaner algorithm specifications Siggraph 2006, 7/31/2006 www.cs.wustl.edu/~cmg/