Aam based face tracking with temporal matching and face segmentation
This presentation is the property of its rightful owner.
Sponsored Links
1 / 22

AAM based Face Tracking with Temporal Matching and Face Segmentation PowerPoint PPT Presentation


  • 69 Views
  • Uploaded on
  • Presentation posted in: General

CVPR 2010. AAM based Face Tracking with Temporal Matching and Face Segmentation. Mingcai Zhou 1 、 Lin Liang 2 、 Jian Sun 2 、 Yangsheng Wang 1. 1 Institute of Automation Chinese Academy of Sciences, Beijing, China. 2 Microsoft Research Asia Beijing, China. Outline. AAM Introduction

Download Presentation

AAM based Face Tracking with Temporal Matching and Face Segmentation

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Aam based face tracking with temporal matching and face segmentation

CVPR 2010

AAM based Face Tracking with Temporal Matching and Face Segmentation

Mingcai Zhou1、 Lin Liang2、 Jian Sun2、Yangsheng Wang1

1Institute of Automation Chinese Academy of Sciences, Beijing, China

2Microsoft Research Asia Beijing, China


Outline

Outline

  • AAM Introduction

  • Related Work

  • Method and Theory

  • Experiment


Aam introduction

AAM Introduction

  • A statistical model of shape and grey-level appearance

Shape model

Appearance model


Shape model building

Shape Model Building

:mean shape

:shape bases

,shape parameters

learn by PCA

generate mean shape、

shape bases


Texture model building

Texture Model Building

:mean appearance

:appearance bases

:appearance parameters

灰階值

W(x)

Mean shape

Shape-free patch


Aam model building

AAM Model Building


Aam model search

AAM Model Search

  • Find the optimal shape parameters and appearance parameters to minimize the difference between the warped-back appearance and synthesized appearance

map every pixel x in the model coordinate to its corresponding image point


Problems aam tracker

Problems- AAM tracker

  • Difficultly generalize to unseen images

  • Clutterd backgrounds


How to do

How to do?

  • A temporal matching constraint in AAM fitting

    -Enforce an inter-frame local appearance constraint

    between frames

    • Introduce color-based face segmentation as a soft constraint


Related work

Related Work

-feature-based (mismatched local feature)

Integrating multiple visual cues for robust real-time

3d face tracking,W.-K. Liao, D. Fidaleo, and G. G. Medioni. 2007

-intensity-based (fast illumination changes)

Improved face model fitting on video sequences, X. Liu, F.

Wheeler, and P. Tu. 2007

temporal matching constraint


Method and theory

Method and Theory

  • Extend basic AAM to Multi-band AAM

    • The texture(appearance) is a concatenation of three texture band values

      • The intensity (b)

      • X-direction gradient strength (c)

      • Y-direction gradient strength (d)


Temporal matching constraint

Temporal Matching Constraint

  • Select feature points with salient local appearances at previous frame

  • I(t−1) to the Model coordinate and get the appearance A(t-1)

  • Use warping function W(x;pt) maps R(t-1) to a patch R(t) at frame t


Shape parameter initialization

Shape parameter Initialization

Face Motion Direction

,


Shape parameter initialization1

Shape parameter Initialization

When r reaches the noise level expected in the

correspondences, the algorithm stops


Shape parameter initialization2

Shape parameter Initialization

-Comparison

Motion direction

Previous frame’s shape

Feature matching


Face segmentation constraint

Face Segmentation Constraint

Where are the locations of the selected outline points in the model coordinate


Face segmentation constraint1

Face Segmentation Constraint

-Face Segmentation


Face segmentation constraint2

Face Segmentation Constraint


Experiments

Experiments

Lost frame num


Experiments1

Experiments


Conclusion

Conclusion

─ Our tracking algorithm accurately localizes the facial

components, such as eyes, brows, noses and mouths,

under illumination changes as well as large expression

and pose variations.

─ Our tracking algorithm runs in real-time. On a

Pentium-43.0G computer, the algorithm’s speed is

about 50 fps for thevideo with 320 × 240 resolution


Future work

Future Work

─ Our tracker cannot robustly track profile views

with large angles

─ The tracker’s ability to handle large occlusion also

needs to be improved


  • Login