3d face reconstruction from monocular or stereo images
Download
1 / 51

change your image ... - PowerPoint PPT Presentation


  • 270 Views
  • Updated On :

3D Face Reconstruction from Monocular or Stereo Images. . Thomas Vetter. Universit y of Basel. Switzerland . http://gravis.cs.uni bas.ch. Change Your Image . Analysis by Synthesis. model parameter. Analysis. Image Model. Synthesis. Image. 3D World.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'change your image ...' - Angelica


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
3d face reconstruction from monocular or stereo images l.jpg

3D Face Reconstruction from Monocular or Stereo Images.

Thomas Vetter

University ofBasel

Switzerland

http://gravis.cs.unibas.ch



Analysis by synthesis l.jpg
Analysis by Synthesis

model parameter

Analysis

Image Model

Synthesis

Image

3D

World

Image Description


Approach example based modeling of faces l.jpg
Approach: Example based modeling of faces

2D Image 3D Face Models

2D Image 2D Face Examples

= w1 * + w2 * + w3 * + w4 * +. . .


Morphing 3d faces l.jpg
Morphing 3D Faces

1

__

2

3D Blend

3D Morph

1

__

=

+

2


Shape and texture vectors l.jpg
Shape and Texture Vectors

Reference Head

Example i



Registration in different representations l.jpg
Registration in different representations

  • Curvature Guided Level Set Registration using Adaptive Finite ElementAndreas Dedner, Marcel Lüthi, Thomas Albrecht and Thomas Vetter IN: Proceedings DAGM'07: Heidelberg 2007

  • Optimal Step Nonrigid ICP Algorithms for Surface RegistrationBrian Amberg, Sami Romdhani and Thomas Vetter IN: Proceedings, CVPR'07, Minneapolis, USA 2007.

  • A Morphable Model for the Synthesis of 3D Faces. Volker Blanz and Thomas VetterIN: SIGGRAPH'99 Conference Proceedings, 187-194

  • Implicit:

  • Triangulated:

  • Parameterized:



Vector space of 3d faces l.jpg
Vector space of 3D faces.

  • A Morphable Model can generate new faces.

a1 * + a2 * + a3 * + a4 * +. . .

=

b1 * + b2 * + b3 * + b4 * +. . .



Continuous modeling in face space l.jpg
Continuous Modeling in Face Space

Caricature

Original

Average

Anti Face


Model l ing the appearance of faces l.jpg
Modelling the Appearance of Faces

A face is represented as a point in face space.

  • Which directions code for specific attributes ?


Learning from labeled example faces l.jpg
Learning from Labeled Example Faces

Fitting a regression function


Facial attributes l.jpg
Facial Attributes

Weight

Subjective Attractiveness

Gender

Original


3d shape from images l.jpg
3D Shape from Images

Face

Analyzer

Input Image

3D Head


Matching a morphable 3d face model l.jpg
Matching a Morphable 3D-Face-Model

  • R = Rendering Function

  • = Parameters for Pose, Illumination, ...

    Find optimal a, b, r !


Automated parameter estimation l.jpg
Automated Parameter Estimation

Ambient: intensity, color

Parallel: intensity, color,direction

Color: contrast, gains, offsets

  • Face Parameters

  • 150 shape coefficients ai

  • 150 texture coefficients bi

head position

head orientation

focal length

  • 3D Geometry

  • Light and Color


Image formation at each vertex k l.jpg
Image Formation: at each Vertex k

  • Rigid Transformation

  • Normals

  • Phong Illumination

  • Perspective Projection

  • Color Transformation

  • bi

  • ai


Error function l.jpg
Error Function

  • Image difference (pixel intensity cost function)

  • Plausible parameters

  • Minimize




Which feature to use l.jpg
Which Feature to use?

someEdge

detector


Edge feature l.jpg
Edge Feature

  • Rigid Transformation

  • Normals

  • Phong Illumination

  • Perspective Projection

  • Color Transformation

  • bi

  • ai




Multi features fitting algorithm27 l.jpg
Multi-Features Fitting Algorithm

1

2

3

4

5

At stage 4:


Recognition from images l.jpg
Recognition from Images

Complex Changes in Appearance

Images: CMU-PIE database.



Correct identification 1 out of 68 l.jpg
Correct Identification “1 out of 68” (%)

  • 99.5

  • 83.0

  • 97.8

  • 86.2

  • 79.5

  • 85.7

  • 92.3

  • 95.0

  • 89.0

  • gallery

  • front

  • side

  • profile

  • probe

  • front

  • 99.8

  • side

  • 99.9

  • profile

  • 98.3

  • total

CMU-PIE database: 4488 images of 68 individuals

3 poses x 22 illuminations = 66 images per individual


Reanimation of images l.jpg
Reanimation of Images

V. Blanz, C. Basso, T. Poggio & T. Vetter

Reanimating Faces in images and Video

Proc. of Eurographics 2003


Expression transfer l.jpg
Expression Transfer

Fitting

Fitting

Rendering


Analysis by synthesis33 l.jpg
Analysis by Synthesis

model parameter

  • Image Processing

    • Edges

    • Highlights

    • Segmentation

    • ……

Image Model

some ║ ║X

Analysis

Synthesis

3D

World

Image Description

Image


Segmenting hair a general requirement l.jpg
Segmenting hair a general requirement ?

No outlier detection

with outlier mask


Skin segmentation l.jpg
Skin segmentation

  • We need to mask out non-skin regions / outliers

  • 3DMM is not sufficient


Shading problem l.jpg
Shading Problem

  • Skin regions contain strong intensity gradients that make a segmentation difficult!



Illumination compensation38 l.jpg
Illumination Compensation

  • Skin Detail Analysis for Face RecognitionJean Sebastian Pierrard , Thomas Vetter CVPR 2007

Local fitting


Segmentation results l.jpg
Segmentation Results

GrabCut

  • Skin Detail Analysis for Face RecognitionJean Sebastian Pierrard , Thomas Vetter CVPR 2007

Thresholding


Try new hairstyles l.jpg
Try New Hairstyles

3D Angle, Position

Illumination,

Foreground,

Background

3D Shape

and Texture


More hairstyles l.jpg
More Hairstyles

3D Shape

and Texture

3D Angle, Position

Illumination,

Foreground,

Background


Using more than a single image l.jpg
Using more than a single image ?

Reconstructing High Quality Face-Surfaces using Model Based Stereo Brian Amberg, Andrew Blake, Andrew Fitzgibbon, Sami Romdhani and Thomas Vetter  IN: Proceedings ICCV 2007 Rio de Janeiro, Brazil








Slide50 l.jpg

Results on Flash Data

Ground Truth Monocular Stereo


Acknowledgement l.jpg
Acknowledgement

Volker Blanz

Sami Romdhani

Brian Amberg

Jaen Sabastian Pierrard


ad