3d face reconstruction from monocular or stereo images
Download
1 / 51

3D Face Reconstruction from Monocular or Stereo Images. - PowerPoint PPT Presentation


3D Face Reconstruction from Monocular or Stereo Images. . Thomas Vetter. Universit y of Basel. Switzerland . http://gravis.cs.uni bas.ch. Change Your Image . Analysis by Synthesis. model parameter. Analysis. Image Model. Synthesis. Image. 3D World.

Related searches for 3D Face Reconstruction from Monocular or Stereo Images.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha

Download Presentation

3D Face Reconstruction from Monocular or Stereo Images.

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


3D Face Reconstruction from Monocular or Stereo Images.

Thomas Vetter

University ofBasel

Switzerland

http://gravis.cs.unibas.ch


Change Your Image ...


Analysis by Synthesis

model parameter

Analysis

Image Model

Synthesis

Image

3D

World

Image Description


Approach: Example based modeling of faces

2D Image 3D Face Models

2D Image 2D Face Examples

= w1 * + w2 * + w3 * + w4 * +. . .


Morphing 3D Faces

1

__

2

3D Blend

3D Morph

1

__

=

+

2


Shape and Texture Vectors

Reference Head

Example i


Surface registration: Which representation?


Registration in different representations

  • Curvature Guided Level Set Registration using Adaptive Finite ElementAndreas Dedner, Marcel Lüthi, Thomas Albrecht and Thomas Vetter IN: Proceedings DAGM'07: Heidelberg 2007

  • Optimal Step Nonrigid ICP Algorithms for Surface RegistrationBrian Amberg, Sami Romdhani and Thomas Vetter IN: Proceedings, CVPR'07, Minneapolis, USA 2007.

  • A Morphable Model for the Synthesis of 3D Faces. Volker Blanz and Thomas VetterIN: SIGGRAPH'99 Conference Proceedings, 187-194

  • Implicit:

  • Triangulated:

  • Parameterized:


Database of 3D Faces


Vector space of 3D faces.

  • A Morphable Model can generate new faces.

a1 * + a2 * + a3 * + a4 * +. . .

=

b1 * + b2 * + b3 * + b4 * +. . .


Manipulation of Faces

Modeler


Continuous Modeling in Face Space

Caricature

Original

Average

Anti Face


Modelling the Appearance of Faces

A face is represented as a point in face space.

  • Which directions code for specific attributes ?


Learning from Labeled Example Faces

Fitting a regression function


Facial Attributes

Weight

Subjective Attractiveness

Gender

Original


3D Shape from Images

Face

Analyzer

Input Image

3D Head


Matching a Morphable 3D-Face-Model

  • R = Rendering Function

  • = Parameters for Pose, Illumination, ...

    Find optimal a, b, r !


Automated Parameter Estimation

Ambient: intensity, color

Parallel: intensity, color,direction

Color: contrast, gains, offsets

  • Face Parameters

  • 150 shape coefficients ai

  • 150 texture coefficients bi

head position

head orientation

focal length

  • 3D Geometry

  • Light and Color


Image Formation: at each Vertex k

  • Rigid Transformation

  • Normals

  • Phong Illumination

  • Perspective Projection

  • Color Transformation

  • bi

  • ai


Error Function

  • Image difference (pixel intensity cost function)

  • Plausible parameters

  • Minimize


animation by Volker Blanz.


Using Multiple Features


Which Feature to use?

someEdge

detector


Edge Feature

  • Rigid Transformation

  • Normals

  • Phong Illumination

  • Perspective Projection

  • Color Transformation

  • bi

  • ai


Edge Fitting Results


Multi-Features Fitting Algorithm


Multi-Features Fitting Algorithm

1

2

3

4

5

At stage 4:


Recognition from Images

Complex Changes in Appearance

Images: CMU-PIE database.


3D Computer Graphics


Correct Identification “1 out of 68” (%)

  • 99.5

  • 83.0

  • 97.8

  • 86.2

  • 79.5

  • 85.7

  • 92.3

  • 95.0

  • 89.0

  • gallery

  • front

  • side

  • profile

  • probe

  • front

  • 99.8

  • side

  • 99.9

  • profile

  • 98.3

  • total

CMU-PIE database: 4488 images of 68 individuals

3 poses x 22 illuminations = 66 images per individual


Reanimation of Images

V. Blanz, C. Basso, T. Poggio & T. Vetter

Reanimating Faces in images and Video

Proc. of Eurographics 2003


Expression Transfer

Fitting

Fitting

Rendering


Analysis by Synthesis

model parameter

  • Image Processing

    • Edges

    • Highlights

    • Segmentation

    • ……

Image Model

some ║ ║X

Analysis

Synthesis

3D

World

Image Description

Image


Segmenting hair a general requirement ?

No outlier detection

with outlier mask


Skin segmentation

  • We need to mask out non-skin regions / outliers

  • 3DMM is not sufficient


Shading Problem

  • Skin regions contain strong intensity gradients that make a segmentation difficult!


Illumination Compensation


Illumination Compensation

  • Skin Detail Analysis for Face RecognitionJean Sebastian Pierrard , Thomas Vetter CVPR 2007

Local fitting


Segmentation Results

GrabCut

  • Skin Detail Analysis for Face RecognitionJean Sebastian Pierrard , Thomas Vetter CVPR 2007

Thresholding


Try New Hairstyles

3D Angle, Position

Illumination,

Foreground,

Background

3D Shape

and Texture


More Hairstyles

3D Shape

and Texture

3D Angle, Position

Illumination,

Foreground,

Background


Using more than a single image ?

Reconstructing High Quality Face-Surfaces using Model Based Stereo Brian Amberg, Andrew Blake, Andrew Fitzgibbon, Sami Romdhani and Thomas Vetter  IN: Proceedings ICCV 2007 Rio de Janeiro, Brazil


Model Based Stereo


Model Based Stereo


Silhouette Term


Colour Difference Term


Results


Results


Results on Flash Data

Ground Truth Monocular Stereo


Acknowledgement

Volker Blanz

Sami Romdhani

Brian Amberg

Jaen Sabastian Pierrard


ad
  • Login