1 / 30

3D Face Modeling

3D Face Modeling. Michaël De Smet. Topics to Discuss. 3D Morphable Models 3D face reconstruction Face recognition Lip synchronization. Topics to Discuss. 3D Morphable Models 3D face reconstruction Face recognition Lip synchronization. 3D Morphable Models.

apria
Download Presentation

3D Face Modeling

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. 3D Face Modeling Michaël De Smet

  2. Topics to Discuss • 3D Morphable Models • 3D face reconstruction • Face recognition • Lip synchronization

  3. Topics to Discuss • 3D Morphable Models • 3D face reconstruction • Face recognition • Lip synchronization

  4. 3D Morphable Models • Statistical model of shape and texture • Derived from laser scans • USF DARPA HumanID 3D Face Database • Processing • Hole filling • Surface smoothing • Albedo estimation • Dense correspondence

  5. -2s +2s -2s +2s … … 3D Morphable Models

  6. Topics to Discuss • 3D Morphable Models • 3D face reconstruction • Face recognition • Lip synchronization

  7. 3D Face Reconstruction Fitting the 3DMM to one or more images of the same face • Scale • Rotation • Translation • Illumination ? ? • Shape • Texture Optimization problem with > 100 parameters

  8. 3D Face Reconstruction Feature points Feature alignment Fitting result

  9. 3D Face Reconstruction

  10. Dealing with Occlusions

  11. Dealing with Occlusions Without occlusion handling With occlusion handling

  12. 3D Face Reconstruction

  13. 3D Face Reconstruction

  14. 3D Face Reconstruction

  15. 3D Face Reconstruction

  16. 3D Face Reconstruction

  17. 3D Face Reconstruction

  18. Topics to Discuss • 3D Morphable Models • 3D face reconstruction • Face recognition • Lip synchronization

  19. Face Recognition Fit the 3DMM to an image of an unknown face • Scale • Rotation • Translation • Illumination ? ? • Shape • Texture Compare to database Recognition

  20. Pose 1 100.0% Pose 2 100.0% Pose 3 N/A Pose 4 100.0% Pose 5 95.7% Face Recognition • In controlled settings, almost perfect recognition is possible Training view

  21. Face Recognition • Uncontrolled environments are challenging • Face orientation unknown • Difficult illumination • Facial expressions • Occlusions • Low resolution • Motion blur • …

  22. Face Recognition European parliament video: 21 persons, 86% correct

  23. Face Recognition • VRT news broadcasts: 12 persons

  24. Face Recognition • VRT news broadcasts: 12 persons 82.3% correct recognition

  25. Topics to Discuss • 3D Morphable Models • 3D face reconstruction • Face recognition • Lip synchronization

  26. Lip Synchronization • Speech driven animation • Texture based, i.e. shape is fixed • Strategy: • Extract 3D model of speaker’s face • Track rigid motion of the face in video • Extract texture for each frame • Compute PCA model of texture • Train ANN to link phonemes and PCA coefficients (visemes)

  27. System Overview Automatic Phone Recognition Neural Network Face Synthesis Speech feature vectors Facial feature vectors

  28. Training Setup Automatic Phone Recognition Neural Network Training Face Analysis Speech feature vectors Facial feature vectors

  29. Video Processing • 3D face model acquisition • Rigid motion tracking • Normalized texture extraction • Texture feature extraction (PCA)

  30. Conclusion • 3DMMs are a very powerful tool for face modeling • Many applications in computer vision and computer graphics

More Related