1 / 40

Amir Hosein Omidvarnia Spring 2007

Principles of 3D Face Recognition. Amir Hosein Omidvarnia Spring 2007. Outline. Fundamentals 3D face processing stages Illumination Cone method Structured-light pattern method Elastic bunch graph method Open issues and challenges Conclusion. 3D FRT vs. 2D FRT.

lexi
Download Presentation

Amir Hosein Omidvarnia Spring 2007

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Principles of 3D Face Recognition Amir Hosein Omidvarnia Spring 2007

  2. Outline • Fundamentals • 3D face processing stages • Illumination Cone method • Structured-light pattern method • Elastic bunch graph method • Open issues and challenges • Conclusion

  3. 3D FRT vs. 2D FRT • 2D face recognition still requires help • Pose, expression, illumination variations • Promises of 3D facial recognition • High-security applications • 3D shape information invariance • Pose and illumination problems can be solved • Better facial feature localization

  4. Challenges in FRT • The recent FERET test has revealed that there are at least two major challenges: • The illumination variation problem • The pose variation problem

  5. Illumination variation • Images of the same face appear differently due to the change in lighting • Naive Solution: • discarding the first few eigenfaces

  6. Pose Variation • Basically, the existing solution can be divided into three types: • multiple images in both training stage and recognition stage • multiple images in training stage, but only one image in recognition stage • single image based methods

  7. A Typical 3D FR System • A4Vision Core Technology

  8. 3D Facial Recognition Pipeline Face Normalization/Alignment 3D Face Detection Pre-proc. Features Pattern Classifier Point Clouds Depth Images Cropping Coarse Alignment Noise Removal Hole Filling Smoothing Landmark Finding Fine Alignment

  9. 3D Face Detection • This problem has not been touched so far! • Simple heuristics such as nose tip • In complex scenes, curvature analysis is generally used

  10. Pre-processing • Artifact removal • Noise removal: spikes (filters), clutter (manually), noise (median filter) • Holes filling (Gaussian smoothing, linear interpolation, symmetrical interpolation)

  11. Face Normalization/Alignment • Coarse alignment by • Centre of mass, • Plane fitted to the data • Facial landmarks (eyes, nosetip) • Fine alignment • ICP (Iterative Conditional Proc.) • Warping • Elastic deformations

  12. 3D Acquisition Systems • Face specific • Biometrics • A4Vision • Geometrix • Modeling • Cyberware • Genex • Inspeck • Medeim • Breuckmann

  13. 3D Face Databases • UND • 275 subjects, 943 scans • Shape + texture • FRGC • 400 subjects, 4007 scans • Shape + texture • 3D_RMA • 120 subject, 6 scans • Shape only • GavabDB • 61 subjects (9 scans) • Shape only • Pose, expression variations • USF database • 357 scans • 3DFS generator • Custom face databases • 12 persons to ~6000 persons (A4Vision) UND GavabDB 3DFS

  14. 3D Face Recognition Approaches • Appearance-Based Methods • Feature-Based Methods • Model-Based Methods

  15. An Appearance-Based Method Illumination Cone Method

  16. Lambertian Model Lambertian • Lambertian shading assumes that the incoming light is reflected equally in all directions, without bias. The angle of incoming light has no effect on the direction in which it is reflected. Phong

  17. Illumination cone • For a Lambertian surface: • Image x superpositioned with k light sources can be written as:

  18. Illumination cone • Database images of Yale University: Different Poses Different Illuminations

  19. Illumination cone • Image Acquisition

  20. Illumination cone • Least Square estimation is used to find normal unit vectors.

  21. Illumination cone • Illumination cone is a subspace covers the variation in illumination. Synthetic images Basis images • Reconstructed surface by means of GBR Ambiguity

  22. Illumination cone • Representations and Algorithms for Face Recognition • Constructing 117 (19x7) different poses by means of planar transformations (non-linear warping) • Constructing the Illumination Cone of each pose from 80-120 different lighting conditions

  23. Illumination cone • Decreasing the number of lighting conditions using PCA dimension reduction down to 11 • Applying SVD based methods to reduce the number of database images (11x117) down to 100 • These 100 images form the image basis space for each person

  24. A Feature-Based Method Structured-Light Pattern Method

  25. Structured-Light Surface Rendering

  26. Structured-Light Surface Rendering • Striped images • Reconstructed surface

  27. Structured-Light Surface Rendering • Curvature Analysis for Surface Matching

  28. Triangulation-Interpolation

  29. A Model-Based Method Elastic Bunch Graph Matching

  30. Elastic Bunch Graph • use Gabor wavelet transform to extract face features so that the recognition performance can be invariant to the variation in poses.

  31. Elastic Bunch Graph • Gabor wavelet decomposition • Gabor kernels

  32. Gabor Filters

  33. Jets • Small patch gray values • Wavelet transform

  34. Comparing Jets • Amplitude similarity • Phase similarity

  35. Comparing Jets

  36. Face Bunch Graphs (FBG) • Stack like general representation • Graph similarity function

  37. Graph Extraction • Step 1: find approximate face position • Step 2: refine position and size • Step 3: refine size and find aspect ratio • Step 4: local distortion

  38. Recognition • Comparing image graph • Recognized for highest similarity

  39. Open Issues & Challenges • Uncontrolled acquisition • Non-cooperative • Different lighting conditions • Texture map + shape map inconsistencies • Real-time 3D video data • Computational complexity • Issues related to performance assessment • Publicly available standard face databases • Quality (resolution) of the data • Artifacts such as eyeglasses

  40. Conclusions • 3D face recognition systems were proposed to overcome expression, illumination, and pose challenges • Illumination correction is simpler • Facial landmark localization is better • The core algorithm, ICP, has limited capabilities • Not suitable for non-rigid deformations

More Related