Learning Models of Shape from 3D Range Data. Dragomir Anguelov Artificial Intelligence Lab Stanford University. Shape Models for Animation. Animation. Biomechanics. F z. M z. M x. F y. F x. M y. [Gollum - Time Warner]. Shape Models for Motion Estimation.

Download Presentation

Learning Models of Shape from 3D Range Data

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

Simulation-based models [Aubel ‘02] [Wilhelms, Van Gelder ‘97] Example-based models [Allen et al. ‘02] [Allen et al. ‘03] Machine Learning for Model Construction Artist-designed models [Poser – Curious Labs] [Lucasfilm] [Dreamworks]

Marker motion capture Markerless motion capture Physical measurement [Bregler et al. ‘98] [Braune,Fischer 1892] [Polar Express] [Cheung et al ’03] Machine Learning for Motion Estimation

Shape Models from 3D Scans • Object models: • Discover object parts • Model pose variation in terms of parts • Class models: • Model shape variation within class Pose variation Body-shape variation

3D Range Scans • Cyberware Scans • 4 views, • ~125k polygons • ~65k points each • Problems • Missing surface • Drastic shape changes

Articulated Template 2. Fit Template to Scans 3. Interpolation Standard Modeling Pipeline A lot of human intervention Pose or body shape deformations modeled, but not both [Allen, Curless, Popovic 2002] Similar to: [Lewis et al. ‘00] [Sloan et al. ’01] [Mohr, Gleicher ’03], …

3D Scans Registration Recover skeleton Learn model of deformations Contributions Unsupervised non-rigid registration Automatic articulated model recovery Modeling pose and body shape deformations 3D Scan segmentation and object detection(not in thesis)

Talk outline • Automating the data processing pipeline • Unsupervised non-rigid registration • Recovering an articulated model • Modeling the space of human deformations • Scene understanding (not in thesis)

Registration Task: Establish correspondences between two surfaces

C Correspondences Transformation Q Model mesh X Transformed mesh X’ Correspondence ck specifies which point x’i generated point zk Generative Model Scan meshZ Goal: Given model mesh X and scan mesh Z, recover transformation Q and correspondences C

Q Transformation Gaussian noise model Deformation Model Treat each link as a spring, resisting stretching and twisting Model X

Non-rigid Iterative Closest Point • Algorithm • Assume initial alignment Qknown • Compute correspondences C (given Q) • For each point zk , find its nearest neighbor x’i • Solve for transformation Q (given C) which • Brings matching pairs together • Minimizes deformation [Shelton ’00], [Chui & Rangarajan ’02], [Allen et al. ’03], [Hähnel et al.’03]

Correspondences for different points computed independently c1 c2 Poor correspondences Poor transformations Nonrigid ICP Experiment X Z

Z X Correlated Correspondence Algorithm Z X Correlated Correspondence Algorithm Computes an embedding of mesh Z into mesh X Output: Correspondences Input: Pair of scans • The embedding enforces: • • Minimal surface deformation • Similar local surface appearance • Preservation of geodesic distance [Anguelov, Srinivasan, Pang, Koller, Thrun, Davis ‘04]

Single potential Pairwise potential (C1, C2) C1 C2 (C2) (C1) (C3) (C1,C3) (C2,C3) C3 • Joint probability distribution Markov Network • Markov network

Model Point x2 Model Point x1 Link Link Local appearance Local appearance Deformation potential Appearance potential 1 2… N (Ci,Cj) Ci Cj (Ci) (Cj) Scan Point zj Scan Point zi Local appearance Local appearance Correlated Correspondence Model

(C1, C2) C1 C2 (C2) (C1) (C3) (C1,C3) (C2,C3) C3 CC Potentials • Markov network Want a good consistent assignment for all correspondences C ! • Local appearance potentials (Ci=k) • Use spin images [Johnson+Hebert ’97] to quantify the surface similarity around two matching points • Deformation potentials D(Ci=k,Cj=l) = P(e’ij| ekl)

(C1, C2) C1 C2 (C2) (C1) (C3) (C1,C3) (C2,C3) C3 Markov Network Inference • Markov network • Inference is Markov Nets is generally intractable • Exponential search space • Loopy Belief Propagation (LBP) [Pearl ’88] is an efficient algorithm for search in exponential spaces • Converges to a local minimum (of the Bethe free energy)

Geodesic Potentials: far -> far Z X • Distant points in Z must be distant in X • Constraint between each pair of distant points zk, zl (farther than 5r) r resolution of mesh X

Talk outline • Automating the data processing pipeline • Unsupervised non-rigid registration • Recovering an articulated model • Modeling the space of human deformations • Scene understanding (not in thesis)

combine Skeleton: 9 parts Recovering Articulation: State of the art • Algorithm assigns points to parts independently; ignoring the correlations between the assignments • Prone to local minima Each joint is estimated from a separate sequence [Cheung et al., ‘03]

1 2… P Transformations T1 a1 aN Part labels … Model … xN x1 Points … TP Transformed Model yN y1 … cK c1 Point corrs Instance zK z1 … Points Probabilistic Model

a1 a2 a3 Contiguity Prior • Parts are preferably contiguous regions • Adjacent points on the surface should have similar labels • Enforce this with a Markov network: Penalizes large number of parts

Clustering algorithm • Algorithm • Given transformations , perform min-cut* inference to get • Given labels , solve for rigid transformations If a part doesn’t contribute to the likelihood, it will be automatically dropped *[Greig et al. 89], [Kolmogorov & Zabih 02]

Talk outline • Automating the data processing pipeline • Unsupervised non-rigid registration • Recovering an articulated skeleton • Modeling the space of human shapes • Pose and body shape deformations • Application: shape completion • Scene understanding (not in thesis) • Discriminative Markov networks for scan segmentation • Articulated object detection

[Allen et al. ‘03] [Seo & Thalmann‘03] Body shape deformation = point displacements from average shape Deformation Transfer Problem [Lewis et al. ‘00] [Sloan et al. ‘01] [Allen et al. ‘02] [Wang & Phillips ‘02] [Mohr & Gleicher ‘03] [Sand et al. ‘03] Pose deformation = point displacements from articulated template How do you combine two displacement-based models? • displacements cannot be multiplied • adding displacements ignores notion of object scale

Deformed polygon Template polygon Pose deformation Body shape deformation Rigid part rotation Predict from nearby joint angles Linear subspace (PCA) Predicting Human Deformation [Anguelov, Srinivasan, Koller, Thrun, Rodgers, Davis ‘05]

Deformed polygon Template polygon Pose deformation Body shape deformation Rigid part rotation We have: To reconstruct the entire mesh Y, solve: Reconstructing the Shape Related work: [Sumner & Popovic ’04]

Joint angles Deformations output Regression function Linear regression from two nearest joints Pose Deformation input