1 / 24

Advanced Visual Face and Head Tracking Techniques

Explore methods for localizing and tracking faces in image sequences amidst challenges like lighting, occlusion, and rotation. Learn about visual cues, density propagation, and solutions for fast and accurate face tracking. Dive into 3D face tracking advancements and solutions.

fwingate
Download Presentation

Advanced Visual Face and Head Tracking Techniques

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Face/Head Tracking Ying Wu yingwu@ece.northwestern.edu Electrical and Computer Engineering Northwestern University, Evanston, IL http://www.ece.northwestern.edu/~yingwu

  2. Tracking Heads? Courtesy of Y. Wu, 2001 • The task: • Localize faces and track them in image sequences • Challenges: • Lighting, occlusion, rotation, etc.

  3. Outline Motivation What is tracking? One solution (Birchfield_CVPR98) Other methods and open issues

  4. Motivation • Why tracking? • The complexity of face detection • scan all the pixel positions and several scales • The limitation of face detection • hard to handle out-of-plane rotation • Can we maintain the identity of the faces? • although face recognition is the ultimate solution for this, we may not need it, if not necessary • Objectives • fast (frame-rate) face/head localization • handle 360o out-of-plane rotation

  5. Visual Tracking

  6. Four Elements • Infer target states in video sequences • Target states vs. image observations • Visual cues and modalities • Four elements • Target representation X • Observation representation Z • Hypotheses measurement p(Zt|Xt) • Hypotheses generating p(Xt|Xt-1)

  7. Ground Truth Hypothesis Prediction Estimation Visual Tracking

  8. P(Xt|Xt-1) P(Zt|Xt) Dynm. Mdl Obsrv. Mdl Formulating Visual Tracking

  9. Tracking as Density Propagation Posterior Prob. State space Xt Posterior Prob. State space Xt+1

  10. One Solution(Birchfield_CVPR98) • Framework • Search strategy • Edge cue • Color cue

  11. color matching score hypotheses Edge matching score Framework • s = (x,y,) • Tracking is treated as a local search based on the prediction

  12.  is the search step size Search Strategy • Local exhaustive search • Do you have better ideas?

  13. The the magnitude of the gradient at perimeter pixel i of the ellipse s. # of pixels on the perimeter of the ellipse unit vector normal to the ellipse at pixel i. Edge Cue • Method I • Method II • Which is better?

  14. Normalization • Why do we need normalization? • How good is it?

  15. # of bins Model histogram Color Cue • Histogram intersection

  16. Color Cue • Color space • B-G • G-R • R+G+B (why do we need that) • 8 bins for B-G and G-R, 4 for R+G+B • Training the model histogram • Normalization

  17. Comments • Can the rotation be handled? • Can the scaling issue be handled? • Is the search strategy good enough? • Is the color module good? • Is the motion prediction enough? • Is the combination of the two cues good? • Can it handle occlusion? • Can it cope with multiple faces • Coalesce • Switch ID

  18. Other Solutions • Condensation algorithm • 3D head tracking

  19. Tracking as Density Propagation Posterior Prob. State space Xt Posterior Prob. State space Xt+1

  20. Sequential Monte Carlo • P(Xt|Zt) is represented by a set of weighted samples • Sample weights are determined by P(Zt(n)|Xt(n)) • Hypotheses generating is controlled by P(Xt|Xt-1)

  21. Challenge to Condensation • Curse of dimensionality • What to track? • Positions, orientations • Shape deformation • Color appearance changing • The dimensionality of X • The number of hypotheses grows exponentially

  22. 3D Face Tracking: The Problem • The goal: • Estimate and track 3D head poses • The challenges: • Side view • Back view • Poor illumination • Low resolution • Different users

  23. Annotated Pose Ellipsoid Model Estimator Predictor Cropped Input Image Prepro-cessing Feature Extraction Final Pose Motion Model 3D Face Tracking: A Solution Courtesy of Y. Wu and K. Toyama, 2000

  24. 3D Face Tracking: some results

More Related