3d face reconstruction from monocular or stereo images
Download
Skip this Video
Download Presentation
3D Face Reconstruction from Monocular or Stereo Images.

Loading in 2 Seconds...

play fullscreen
1 / 51

change your image ... - PowerPoint PPT Presentation


  • 273 Views
  • Uploaded on

3D Face Reconstruction from Monocular or Stereo Images. . Thomas Vetter. Universit y of Basel. Switzerland . http://gravis.cs.uni bas.ch. Change Your Image . Analysis by Synthesis. model parameter. Analysis. Image Model. Synthesis. Image. 3D World.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'change your image ...' - Angelica


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
3d face reconstruction from monocular or stereo images

3D Face Reconstruction from Monocular or Stereo Images.

Thomas Vetter

University ofBasel

Switzerland

http://gravis.cs.unibas.ch

analysis by synthesis
Analysis by Synthesis

model parameter

Analysis

Image Model

Synthesis

Image

3D

World

Image Description

approach example based modeling of faces
Approach: Example based modeling of faces

2D Image 3D Face Models

2D Image 2D Face Examples

= w1 * + w2 * + w3 * + w4 * +. . .

morphing 3d faces
Morphing 3D Faces

1

__

2

3D Blend

3D Morph

1

__

=

+

2

shape and texture vectors
Shape and Texture Vectors

Reference Head

Example i

registration in different representations
Registration in different representations
  • Curvature Guided Level Set Registration using Adaptive Finite ElementAndreas Dedner, Marcel Lüthi, Thomas Albrecht and Thomas Vetter IN: Proceedings DAGM\'07: Heidelberg 2007
  • Optimal Step Nonrigid ICP Algorithms for Surface RegistrationBrian Amberg, Sami Romdhani and Thomas Vetter IN: Proceedings, CVPR\'07, Minneapolis, USA 2007.
  • A Morphable Model for the Synthesis of 3D Faces. Volker Blanz and Thomas VetterIN: SIGGRAPH\'99 Conference Proceedings, 187-194
  • Implicit:
  • Triangulated:
  • Parameterized:
vector space of 3d faces
Vector space of 3D faces.
  • A Morphable Model can generate new faces.

a1 * + a2 * + a3 * + a4 * +. . .

=

b1 * + b2 * + b3 * + b4 * +. . .

continuous modeling in face space
Continuous Modeling in Face Space

Caricature

Original

Average

Anti Face

model l ing the appearance of faces
Modelling the Appearance of Faces

A face is represented as a point in face space.

  • Which directions code for specific attributes ?
learning from labeled example faces
Learning from Labeled Example Faces

Fitting a regression function

facial attributes
Facial Attributes

Weight

Subjective Attractiveness

Gender

Original

3d shape from images
3D Shape from Images

Face

Analyzer

Input Image

3D Head

matching a morphable 3d face model
Matching a Morphable 3D-Face-Model
  • R = Rendering Function
  • = Parameters for Pose, Illumination, ...

Find optimal a, b, r !

automated parameter estimation
Automated Parameter Estimation

Ambient: intensity, color

Parallel: intensity, color,direction

Color: contrast, gains, offsets

  • Face Parameters
  • 150 shape coefficients ai
  • 150 texture coefficients bi

head position

head orientation

focal length

  • 3D Geometry
  • Light and Color
image formation at each vertex k
Image Formation: at each Vertex k
  • Rigid Transformation
  • Normals
  • Phong Illumination
  • Perspective Projection
  • Color Transformation
  • bi
  • ai
error function
Error Function
  • Image difference (pixel intensity cost function)
  • Plausible parameters
  • Minimize
which feature to use
Which Feature to use?

someEdge

detector

edge feature
Edge Feature
  • Rigid Transformation
  • Normals
  • Phong Illumination
  • Perspective Projection
  • Color Transformation
  • bi
  • ai
recognition from images
Recognition from Images

Complex Changes in Appearance

Images: CMU-PIE database.

correct identification 1 out of 68
Correct Identification “1 out of 68” (%)
  • 99.5
  • 83.0
  • 97.8
  • 86.2
  • 79.5
  • 85.7
  • 92.3
  • 95.0
  • 89.0
  • gallery
  • front
  • side
  • profile
  • probe
  • front
  • 99.8
  • side
  • 99.9
  • profile
  • 98.3
  • total

CMU-PIE database: 4488 images of 68 individuals

3 poses x 22 illuminations = 66 images per individual

reanimation of images
Reanimation of Images

V. Blanz, C. Basso, T. Poggio & T. Vetter

Reanimating Faces in images and Video

Proc. of Eurographics 2003

expression transfer
Expression Transfer

Fitting

Fitting

Rendering

analysis by synthesis33
Analysis by Synthesis

model parameter

  • Image Processing
    • Edges
    • Highlights
    • Segmentation
    • ……

Image Model

some ║ ║X

Analysis

Synthesis

3D

World

Image Description

Image

segmenting hair a general requirement
Segmenting hair a general requirement ?

No outlier detection

with outlier mask

skin segmentation
Skin segmentation
  • We need to mask out non-skin regions / outliers
  • 3DMM is not sufficient
shading problem
Shading Problem
  • Skin regions contain strong intensity gradients that make a segmentation difficult!
illumination compensation38
Illumination Compensation
  • Skin Detail Analysis for Face RecognitionJean Sebastian Pierrard , Thomas Vetter CVPR 2007

Local fitting

segmentation results
Segmentation Results

GrabCut

  • Skin Detail Analysis for Face RecognitionJean Sebastian Pierrard , Thomas Vetter CVPR 2007

Thresholding

try new hairstyles
Try New Hairstyles

3D Angle, Position

Illumination,

Foreground,

Background

3D Shape

and Texture

more hairstyles
More Hairstyles

3D Shape

and Texture

3D Angle, Position

Illumination,

Foreground,

Background

using more than a single image
Using more than a single image ?

Reconstructing High Quality Face-Surfaces using Model Based Stereo Brian Amberg, Andrew Blake, Andrew Fitzgibbon, Sami Romdhani and Thomas Vetter  IN: Proceedings ICCV 2007 Rio de Janeiro, Brazil

slide50

Results on Flash Data

Ground Truth Monocular Stereo

acknowledgement
Acknowledgement

Volker Blanz

Sami Romdhani

Brian Amberg

Jaen Sabastian Pierrard

ad