3D Face Reconstruction from Monocular or Stereo Images. - PowerPoint PPT Presentation

3d face reconstruction from monocular or stereo images l.jpg
Download
1 / 51

3D Face Reconstruction from Monocular or Stereo Images. . Thomas Vetter. Universit y of Basel. Switzerland . http://gravis.cs.uni bas.ch. Change Your Image . Analysis by Synthesis. model parameter. Analysis. Image Model. Synthesis. Image. 3D World.

Related searches for 3D Face Reconstruction from Monocular or Stereo Images.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

Download Presentation

3D Face Reconstruction from Monocular or Stereo Images.

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


3d face reconstruction from monocular or stereo images l.jpg

3D Face Reconstruction from Monocular or Stereo Images.

Thomas Vetter

University ofBasel

Switzerland

http://gravis.cs.unibas.ch


Change your image l.jpg

Change Your Image ...


Analysis by synthesis l.jpg

Analysis by Synthesis

model parameter

Analysis

Image Model

Synthesis

Image

3D

World

Image Description


Approach example based modeling of faces l.jpg

Approach: Example based modeling of faces

2D Image 3D Face Models

2D Image 2D Face Examples

= w1 * + w2 * + w3 * + w4 * +. . .


Morphing 3d faces l.jpg

Morphing 3D Faces

1

__

2

3D Blend

3D Morph

1

__

=

+

2


Shape and texture vectors l.jpg

Shape and Texture Vectors

Reference Head

Example i


Surface registration which representation l.jpg

Surface registration: Which representation?


Registration in different representations l.jpg

Registration in different representations

  • Curvature Guided Level Set Registration using Adaptive Finite ElementAndreas Dedner, Marcel Lüthi, Thomas Albrecht and Thomas Vetter IN: Proceedings DAGM'07: Heidelberg 2007

  • Optimal Step Nonrigid ICP Algorithms for Surface RegistrationBrian Amberg, Sami Romdhani and Thomas Vetter IN: Proceedings, CVPR'07, Minneapolis, USA 2007.

  • A Morphable Model for the Synthesis of 3D Faces. Volker Blanz and Thomas VetterIN: SIGGRAPH'99 Conference Proceedings, 187-194

  • Implicit:

  • Triangulated:

  • Parameterized:


Database of 3d faces l.jpg

Database of 3D Faces


Vector space of 3d faces l.jpg

Vector space of 3D faces.

  • A Morphable Model can generate new faces.

a1 * + a2 * + a3 * + a4 * +. . .

=

b1 * + b2 * + b3 * + b4 * +. . .


Manipulation of faces l.jpg

Manipulation of Faces

Modeler


Continuous modeling in face space l.jpg

Continuous Modeling in Face Space

Caricature

Original

Average

Anti Face


Model l ing the appearance of faces l.jpg

Modelling the Appearance of Faces

A face is represented as a point in face space.

  • Which directions code for specific attributes ?


Learning from labeled example faces l.jpg

Learning from Labeled Example Faces

Fitting a regression function


Facial attributes l.jpg

Facial Attributes

Weight

Subjective Attractiveness

Gender

Original


3d shape from images l.jpg

3D Shape from Images

Face

Analyzer

Input Image

3D Head


Matching a morphable 3d face model l.jpg

Matching a Morphable 3D-Face-Model

  • R = Rendering Function

  • = Parameters for Pose, Illumination, ...

    Find optimal a, b, r !


Automated parameter estimation l.jpg

Automated Parameter Estimation

Ambient: intensity, color

Parallel: intensity, color,direction

Color: contrast, gains, offsets

  • Face Parameters

  • 150 shape coefficients ai

  • 150 texture coefficients bi

head position

head orientation

focal length

  • 3D Geometry

  • Light and Color


Image formation at each vertex k l.jpg

Image Formation: at each Vertex k

  • Rigid Transformation

  • Normals

  • Phong Illumination

  • Perspective Projection

  • Color Transformation

  • bi

  • ai


Error function l.jpg

Error Function

  • Image difference (pixel intensity cost function)

  • Plausible parameters

  • Minimize


Slide21 l.jpg

animation by Volker Blanz.


Using multiple features l.jpg

Using Multiple Features


Which feature to use l.jpg

Which Feature to use?

someEdge

detector


Edge feature l.jpg

Edge Feature

  • Rigid Transformation

  • Normals

  • Phong Illumination

  • Perspective Projection

  • Color Transformation

  • bi

  • ai


Edge fitting results l.jpg

Edge Fitting Results


Multi features fitting algorithm l.jpg

Multi-Features Fitting Algorithm


Multi features fitting algorithm27 l.jpg

Multi-Features Fitting Algorithm

1

2

3

4

5

At stage 4:


Recognition from images l.jpg

Recognition from Images

Complex Changes in Appearance

Images: CMU-PIE database.


3d computer graphics l.jpg

3D Computer Graphics


Correct identification 1 out of 68 l.jpg

Correct Identification “1 out of 68” (%)

  • 99.5

  • 83.0

  • 97.8

  • 86.2

  • 79.5

  • 85.7

  • 92.3

  • 95.0

  • 89.0

  • gallery

  • front

  • side

  • profile

  • probe

  • front

  • 99.8

  • side

  • 99.9

  • profile

  • 98.3

  • total

CMU-PIE database: 4488 images of 68 individuals

3 poses x 22 illuminations = 66 images per individual


Reanimation of images l.jpg

Reanimation of Images

V. Blanz, C. Basso, T. Poggio & T. Vetter

Reanimating Faces in images and Video

Proc. of Eurographics 2003


Expression transfer l.jpg

Expression Transfer

Fitting

Fitting

Rendering


Analysis by synthesis33 l.jpg

Analysis by Synthesis

model parameter

  • Image Processing

    • Edges

    • Highlights

    • Segmentation

    • ……

Image Model

some ║ ║X

Analysis

Synthesis

3D

World

Image Description

Image


Segmenting hair a general requirement l.jpg

Segmenting hair a general requirement ?

No outlier detection

with outlier mask


Skin segmentation l.jpg

Skin segmentation

  • We need to mask out non-skin regions / outliers

  • 3DMM is not sufficient


Shading problem l.jpg

Shading Problem

  • Skin regions contain strong intensity gradients that make a segmentation difficult!


Illumination compensation l.jpg

Illumination Compensation


Illumination compensation38 l.jpg

Illumination Compensation

  • Skin Detail Analysis for Face RecognitionJean Sebastian Pierrard , Thomas Vetter CVPR 2007

Local fitting


Segmentation results l.jpg

Segmentation Results

GrabCut

  • Skin Detail Analysis for Face RecognitionJean Sebastian Pierrard , Thomas Vetter CVPR 2007

Thresholding


Try new hairstyles l.jpg

Try New Hairstyles

3D Angle, Position

Illumination,

Foreground,

Background

3D Shape

and Texture


More hairstyles l.jpg

More Hairstyles

3D Shape

and Texture

3D Angle, Position

Illumination,

Foreground,

Background


Using more than a single image l.jpg

Using more than a single image ?

Reconstructing High Quality Face-Surfaces using Model Based Stereo Brian Amberg, Andrew Blake, Andrew Fitzgibbon, Sami Romdhani and Thomas Vetter  IN: Proceedings ICCV 2007 Rio de Janeiro, Brazil


Model based stereo l.jpg

Model Based Stereo


Model based stereo44 l.jpg

Model Based Stereo


Silhouette term l.jpg

Silhouette Term


Colour difference term l.jpg

Colour Difference Term


Results l.jpg

Results


Results48 l.jpg

Results


Slide50 l.jpg

Results on Flash Data

Ground Truth Monocular Stereo


Acknowledgement l.jpg

Acknowledgement

Volker Blanz

Sami Romdhani

Brian Amberg

Jaen Sabastian Pierrard


  • Login