Introduction • Why we are interested in face recognition? • Passport control at terminals in airports • Participant identification in meetings • System access control • Scanning for criminal persons
Face Recognition • Face is the most common biometric used by humans • Applications range from static, mug-shot verification to a dynamic, uncontrolled face identification in a cluttered background • Challenges: • automatically locate the face • recognize the face from a general view point under different illumination conditions, facial expressions, and aging effects
Authentication vs Identification • Face Authentication/Verification (1:1 matching) • Face Identification/recognition(1:n matching)
Applications • It is not fool proof – many have been fooled by identical twins • Because of these, use of facial biometrics for identification is often questioned.
Application • Video Surveillance (On-line or off-line) • http://www.crossmatch.com/facesnap-fotoshot.php locates and extracts images from video footage for identification and verification
Why is Face Recognition Hard? • Many faces of Madonna
Face Recognition Difficulties • Identify similar faces (inter-class similarity) • Accommodate intra-class variability due to: • head pose • illumination conditions • expressions • facial accessories • aging effects • Cartoon faces
Inter-class Similarity • Different persons may have very similar appearance Twins Father and son
Intra-class Variability • Faces with intra-subject variations in pose, illumination, expression, accessories, color, occlusions, and brightness
Example: Face Detection • Scan window over image. • Classify window as either: • Face • Non-face
Profile views • Schneiderman’s Test set as an example
Example: Finding skinNon-parametric Representation of CCD • Skin has a very small range of (intensity independent) colors, and little texture • Compute an intensity-independent color measure, check if color is in this range, check if there is little texture (median filter) • See this as a classifier - we can set up the tests by hand, or learn them. • get class conditional densities (histograms), priors from data (counting) • Classifier is
Image as a Feature Vector • Consider an n-pixel image to be a point in an n-dimensional space, • Each pixel value is a coordinate of x.
Nearest Neighbor Classifier • Rj is the training dataset • The match for I is R1, who is closer than R2
Comments • Sometimes called “Template Matching” • Variations on distance function • Multiple templates per class- perhaps many training images per class. • Expensive to compute k distances, especially when each image is big (N dimensional). • May not generalize well to unseen examples of class. • Some solutions: • Bayesian classification • Dimensionality reduction
Face Recognition Solutions • Holistic or Appearance-based Face recognition • EigenFace • LDA • Feature-based
EigenFace • Use Principle Component Analysis (PCA) to determine the most discriminating features between images of faces. • The principal component analysis or Karhunen-Loeve transform is a mathematical way of determining that linear transformation of a sample of points in L-dimensional space which exhibits the properties of the sample most clearly along the coordinate axes.
PCA • http://www.cs.otago.ac.nz/cosc453/student_tutorials/principal_components.pdf
More New Techniques in Face Biometrics • Facial geometry, 3D face recognition http://www-users.cs.york.ac.uk/~nep/research/3Dface/tomh/3DFaceDatabase.html 3D reconstruction
Skin pattern recognition • using the details of the skin for authentication http://pagesperso-orange.fr/fingerchip/biometrics/types/face.htm
Facial thermogram • Facial thermogram requires an (expensive) infrared camera to detect the facial heat patterns that are unique to every human being. Technology Recognition Systems worked on that subject in 1996-1999. Now disappeared. http://pagesperso-orange.fr/fingerchip/biometrics/types/face.htm
Side effect of Facial thermogram • can detect lies • The image on the left shows his normal facial thermogram, and the image on the right shows the temperature changes when he lied. http://pagesperso-orange.fr/fingerchip/biometrics/types/face.htm
Smile recognition • Probing the characteristic pattern of muscles beneath the skin of the face. • Analyzing how the skin around the subject's mouth moves between the two smiles. • Tracking changes in the position of tiny wrinkles in the skin, each just a fraction of a millimetre wide. • The data is used to produce an image of the face overlaid with tiny arrows that indicate how different areas of skin move during a smile. http://pagesperso-orange.fr/fingerchip/biometrics/types/face.htm
Dynamic facial features • They track the motion of certain features on the face during a facial expression (e.g., smile) and obtain a vector field that characterizes the deformation of the face. http://pagesperso-orange.fr/fingerchip/biometrics/types/face.htm