Face Recognition

1. Face Recognition

2. Introduction • Why we are interested in face recognition? • Passport control at terminals in airports • Participant identification in meetings • System access control • Scanning for criminal persons

3. Face Recognition • Face is the most common biometric used by humans • Applications range from static, mug-shot verification to a dynamic, uncontrolled face identification in a cluttered background • Challenges: • automatically locate the face • recognize the face from a general view point under different illumination conditions, facial expressions, and aging effects

4. Authentication vs Identification • Face Authentication/Verification (1:1 matching) • Face Identification/recognition(1:n matching)

5. Applications • It is not fool proof – many have been fooled by identical twins • Because of these, use of facial biometrics for identification is often questioned.

6. Application • Video Surveillance (On-line or off-line) • http://www.crossmatch.com/facesnap-fotoshot.php locates and extracts images from video footage for identification and verification

7. Why is Face Recognition Hard? • Many faces of Madonna

8. Why is Face Recognition Hard?

9. Face Recognition Difficulties • Identify similar faces (inter-class similarity) • Accommodate intra-class variability due to: • head pose • illumination conditions • expressions • facial accessories • aging effects • Cartoon faces

10. Inter-class Similarity • Different persons may have very similar appearance Twins Father and son

11. Intra-class Variability • Faces with intra-subject variations in pose, illumination, expression, accessories, color, occlusions, and brightness

12. Sketch of a Pattern RecognitionArchitecture

13. Example: Face Detection • Scan window over image. • Classify window as either: • Face • Non-face

14. Profile views • Schneiderman’s Test set as an example

15. Example: Finding skinNon-parametric Representation of CCD • Skin has a very small range of (intensity independent) colors, and little texture • Compute an intensity-independent color measure, check if color is in this range, check if there is little texture (median filter) • See this as a classifier - we can set up the tests by hand, or learn them. • get class conditional densities (histograms), priors from data (counting) • Classifier is

17. Face Detection Algorithm

18. Face Recognition

19. Face Recognition: 2-D and 3-D

20. Image as a Feature Vector • Consider an n-pixel image to be a point in an n-dimensional space, • Each pixel value is a coordinate of x.

21. Nearest Neighbor Classifier • Rj is the training dataset • The match for I is R1, who is closer than R2

22. Comments • Sometimes called “Template Matching” • Variations on distance function • Multiple templates per class- perhaps many training images per class. • Expensive to compute k distances, especially when each image is big (N dimensional). • May not generalize well to unseen examples of class. • Some solutions: • Bayesian classification • Dimensionality reduction

23. Face Recognition Solutions • Holistic or Appearance-based Face recognition • EigenFace • LDA • Feature-based

24. EigenFace

25. EigenFace • Use Principle Component Analysis (PCA) to determine the most discriminating features between images of faces. • The principal component analysis or Karhunen-Loeve transform is a mathematical way of determining that linear transformation of a sample of points in L-dimensional space which exhibits the properties of the sample most clearly along the coordinate axes.

26. PCA • http://www.cs.otago.ac.nz/cosc453/student_tutorials/principal_components.pdf

27. More New Techniques in Face Biometrics • Facial geometry, 3D face recognition http://www-users.cs.york.ac.uk/~nep/research/3Dface/tomh/3DFaceDatabase.html 3D reconstruction

28. Skin pattern recognition • using the details of the skin for authentication http://pagesperso-orange.fr/fingerchip/biometrics/types/face.htm

29. Facial thermogram • Facial thermogram requires an (expensive) infrared camera to detect the facial heat patterns that are unique to every human being. Technology Recognition Systems worked on that subject in 1996-1999. Now disappeared. http://pagesperso-orange.fr/fingerchip/biometrics/types/face.htm

30. Side effect of Facial thermogram • can detect lies • The image on the left shows his normal facial thermogram, and the image on the right shows the temperature changes when he lied. http://pagesperso-orange.fr/fingerchip/biometrics/types/face.htm

31. Smile recognition • Probing the characteristic pattern of muscles beneath the skin of the face. • Analyzing how the skin around the subject's mouth moves between the two smiles. • Tracking changes in the position of tiny wrinkles in the skin, each just a fraction of a millimetre wide. • The data is used to produce an image of the face overlaid with tiny arrows that indicate how different areas of skin move during a smile. http://pagesperso-orange.fr/fingerchip/biometrics/types/face.htm

32. Dynamic facial features • They track the motion of certain features on the face during a facial expression (e.g., smile) and obtain a vector field that characterizes the deformation of the face. http://pagesperso-orange.fr/fingerchip/biometrics/types/face.htm