aam based face tracking with temporal matching and face segmentation l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
AAM based Face Tracking with Temporal Matching and Face Segmentation PowerPoint Presentation
Download Presentation
AAM based Face Tracking with Temporal Matching and Face Segmentation

Loading in 2 Seconds...

play fullscreen
1 / 26

AAM based Face Tracking with Temporal Matching and Face Segmentation - PowerPoint PPT Presentation


  • 294 Views
  • Uploaded on

AAM based Face Tracking with Temporal Matching and Face Segmentation. Dalong Du. Outline. Author Introduction AAM Introduction Abstract Method and Theory Experiment. Author Introduction. Mingcai Zhou Institute of Automation Chinese Academy of Sciences Lin Liang Microsoft Research Asia.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'AAM based Face Tracking with Temporal Matching and Face Segmentation' - derrick


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
outline
Outline
  • Author Introduction
  • AAM Introduction
  • Abstract
  • Method and Theory
  • Experiment
author introduction
Author Introduction
  • Mingcai Zhou
    • Institute of Automation Chinese Academy of Sciences
  • Lin Liang
    • Microsoft Research Asia
author introduction4
Author Introduction
  • Jian Sun
    • Microsoft Research Asia
      • joined in July, 2003.
    • Educational background
      • BS degree, MS degree and Ph.D degree from Xian Jiaotong University in 1997, 2000 and 2003
    • Current research interests
      • Interactive compute vision (user interface + vision)
      • Internet compute vision (large image collection + vision)
      • stereo matching and computational photography
author introduction5
Author Introduction
  • Yangsheng Wang
    • Director of Digital Interactive

Media Lab, Institute of Automation

Chinese Academy of Sciences

    • Educational background
      • BS degree, MS degree and Ph.D degree from  Huazhong University of Science and Technology
aam introduction
AAM Introduction
  • Shape Model
  • Appearance (Texture) Model
  • AAM Model Search
aam shape model
AAM—Shape Model
  • Face Q consists of N landmark points
    • The geometry information of Q decouples into two parts:
      • A shape S
        • Shape is the geometric information

invariant to a particular class of

transformations

        • e.g.Or other

linear or nonlinear methods

      • A transformation
        • θ
        • e.g. similarity s, R, t Or

Affine or others.

        • Similarity

x = (x1,y1, … , xn, yn)T

b

θ

Same shape

Different shape

aam shape model8
AAM—Shape Model
  • Shape Model Building
    • Given a set of shapes
    • Align shapes into common frame
      • Procrustes analysis
    • Estimate shape distribution p(x)
      • Use PCA

The aligned shapes

aam shape model9
AAM—Shape Model
  • Shape Model Building, continued
    • Given aligned shapes, { }
    • Apply PCA
      • Compute mean and eigenvectors of covar.
    • P – First t eigenvectors of covar. matrix
    • b – Shape model parameters
aam texture model
AAM—Texture Model
  • Building Texture Models
    • For each example, extract texture vector
    • Normalise vectors (as for eigenfaces)
    • Build eigen-model

Warp to

mean

shape

Texture, g

aam texture model12
AAM—Texture Model
  • Warp method, continued
aam model search
AAM—Model Search
  • Find the optimal shape parameters and appearance parameters to minimize the difference between the warped-back appearance and synthesized appearance

Computed by the inverse

Compositional parameter

Update technique

map every pixel x in the model coordinate to its corresponding image point

abstract
Abstract
  • Problems
    • Generalization problem
    • images with cluttered background
  • How to do?
    • A temporal matching constraint in AAM fitting
      • Enforce an inter-frame local appearance constraint between frames
    • Introduce color-based face segmentation as a soft constraint
method and theory
Method and Theory
  • Extend basic AAM to Multi-band AAM
    • The texture(appearance) is a concatenation of three texture band values
      • The intensity (b)
      • X-direction gradient strength (c)
      • Y-direction gradient strength (d)
method and theory16
Method and Theory
  • Temporal Matching Constraint
    • Select feature points with salient local appearances at previous frame
    • Optimize the shape parameters to match the local appearances at current frame
method and theory17
Method and Theory
  • Temporal Matching Constraint, continued
    • : a set of feature points
      • Selected by a corner detector and some semantic points
    • : the face appearance of frame t-1
    • : the local patch corresponding to the j-th feature point
    • : the average intensity of j-th patches of frame t-1 and t respectively

Normalize the illuminations of two patches

method and theory18
Method and Theory
  • Temporal Matching Constraint, continued
    • Add a new term to the AAM cost function
      • Empirically,

Can be efficiently minimized based

on inverse compositional algorithm

method and theory19
Method and Theory
  • Temporal Matching Constraint, continued
    • Be resistant to global illumination changes
      • Match local patches
    • Do not suffer from the mismatched points
      • Feature matching is continuously refined by updating the shape parameters during AAM fitting
method and theory20
Method and Theory
  • Initialize shape
    • Good initial parameters -> good AAM fitting
    • Method
      • Selected feature points at frame t-1
      • Matched feature points at frame t
      • Remaining feature points after main direction filter
method and theory21
Method and Theory
  • Initialize shape, continued
    • M matched points
    • Estimate the initial shape parameters
      • represents the consistency of feature points I’s direction
      • is the estimated position of the point I given the shape parameters p
      • are the vertex coordinate of the triangle
      • are the triangle coordinate

Gauss-Newton algorithm

method and theory22
Method and Theory
  • Face Segmentation Constrained AAM
    • Problem: AAM tends to fit the face outline to the background edges
    • Method: segment the face region using an adaptive color model and constrain AAM fitting
method and theory23
Method and Theory
  • Formalization
    • Where are the locations of the selected outline points in the model coordinate

Wc = 0.01

experiments
Experiments
  • RI: robust initialization
  • TO: temporal matching constraint
  • FS: face segmentation