1 / 53

Eigenfaces (3)

Eigenfaces (3). Plan of the lecture. Eigenfaces-based methods Fisherfaces Bayesian Matching Local PCA Face relevance maps Error function minimisation Eigenfaces – feature extraction definition of recognition error optimal masks and weights. Eigenfaces – drawbacks. Main drawbacks:

kolina
Download Presentation

Eigenfaces (3)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Eigenfaces (3) Face Recognition and Biometric Systems

  2. Plan of the lecture • Eigenfaces-based methods • Fisherfaces • Bayesian Matching • Local PCA • Face relevance maps • Error function minimisation • Eigenfaces – feature extraction • definition of recognition error • optimal masks and weights Face Recognition and Biometric Systems

  3. Eigenfaces – drawbacks • Main drawbacks: • holistic method • face topology not taken into account • statistical analysis of differences between images in the training set • character of differences not taken into account Face Recognition and Biometric Systems

  4. Example Face Recognition and Biometric Systems

  5. Example: PCA Face Recognition and Biometric Systems

  6. Example: PCA not helpful Face Recognition and Biometric Systems

  7. Example: Linear Discriminant Analysis Face Recognition and Biometric Systems

  8. Fisherfaces • PCA finds main directions of variance • class identity not utilised • Methods based on PCA which utilise class identity: • Linear Discriminant Analysis (LDA) • Fisherfaces Face Recognition and Biometric Systems

  9. Fisherfaces • Principal Component Analysis: • training set  covariance matrix • Linear Discriminant Analysis: • classified training set  two covar. matrices • within-class covariance matrix • between-class covariance matrix • orthogonal basis from two matrices Face Recognition and Biometric Systems

  10. Fisherfaces • Between-class matrix CB – between-class covariance matrix C – number of classes Mi – number of images in i-th class  – average image i – average image of i-th class Face Recognition and Biometric Systems

  11. Fisherfaces • Within-class covariance matrix CW – within-class covariance matrix C – number of classes Xi – set of images which belong to i-th class xk – k-th image which belongs to i-th class i – average image of i-th class Face Recognition and Biometric Systems

  12. Fisherfaces • PCA:  - eigenvectors matrix (vectors in columns) • LDA: Face Recognition and Biometric Systems

  13. Fisherfaces • LDA – hard to find inverse matrix • Fisherfaces – improved approach: • PCA for dimensionality reduction • LDA for finding optimal orthogonal basis Face Recognition and Biometric Systems

  14. Fisherfaces • Feature extraction in the Fisherfaces: • Feature vector calculated by PCA • normalised image as an input • dimensionality reduction • Feature vector calculated by LDA • PCA feature vector as an input • rotation of feature vector • no dimensionality reduction Face Recognition and Biometric Systems

  15. Bayesian Matching • Vectors similarity based on probability of their difference classification • I – set of intra-personal pairs • E – set of extra-personal pairs Face Recognition and Biometric Systems

  16. Bayesian Matching • P(|) – probability of observing a given difference in a defined set of differences • function of PCA back projection error – () Face Recognition and Biometric Systems

  17. Bayesian Matching • Two classes of image pairs • intra- and extra-personal • Differences generated from pairs • two classes of pairs • PCA used for both classes separately • two image spaces Face Recognition and Biometric Systems

  18. Face Recognition and Biometric Systems

  19. Bayesian Matching • Image difference recognition • Dual Eigenfaces • Difference distance from two image spaces • Bayesian Matching – a slow method • image difference calculated for every comparison • possibility of applying other method for selecting candidates (n most similar images) Face Recognition and Biometric Systems

  20. Local PCA • Based on detected features • eyes, nose, mouth • PCA for features • small part of face image • analysis of small images (eigeneyes, eigennoses, etc.) • Less dimensional spaces • Lower effectiveness, but supports the Eigenfaces Face Recognition and Biometric Systems

  21. Local PCA K1 K2 K3 K4 Face Recognition and Biometric Systems

  22. Other methods • Local Feature Analysis • 2D PCA, 2D LDA • Independent Component Analysis Face Recognition and Biometric Systems

  23. Face relevance map • Face topology • eyes & nose – extra-personal differences • mouth & cheeks – intra-personal differences • Nature of features concerned with location Face Recognition and Biometric Systems

  24. Face relevance map • Face relevance map • enhance influence of extra-personal features • decrease influence of intra-personal features • Feature extraction with a map (m) Face Recognition and Biometric Systems

  25. Face relevance map • „T” map • artificial map for eyes and nose • binary values • Results: • FeretA: 423 -> 445 (3,7%) • Conclusion: good approach, need for better map generation methods Face Recognition and Biometric Systems

  26. Face relevance map • Difference map – statistical analysis • Pairs of images: • intra-personal • extra-personal • Average differences between images: • average intra-personal difference • average extra-personal difference • Map obtained by subtracting intra-personal difference from extra-personal one • Results for FeretA: 423 -> 462 (6,4%) Face Recognition and Biometric Systems

  27. Face relevance map • Colour data • information lost during conversion to GS • low distinctiveness • can be used for map generation • Colour used for detection • eye and mouth map • masks based on detection maps Face Recognition and Biometric Systems

  28. Face relevance map • Desired effect: • higher values around eyes and nose • lower values in the area of mouth • Maps deliver information about features location • Two possible approaches: • image-> feature maps-> face relevance map • image-> feature maps-> features -> f.r.m. Face Recognition and Biometric Systems

  29. Face relevance map • Maps from points • Nose location derived from eye & mouth • weighted mean eye(R): (15, 24) eye(L): (49, 24) mouth: (32, 58) Face Recognition and Biometric Systems

  30. Face relevance map • Single point influence • r – radius, mmax – maximal map value • Map – summed influence of the points • eye, nose – positive weights • mouth – negative weights Face Recognition and Biometric Systems

  31. Face relevance map • Maps from colour • improvement comparable to difference maps • colour data carry information concerning nature of face areas • generated for every image • Map may be imposed during normalisation Face Recognition and Biometric Systems

  32. Face relevance map • Maps from colour - examples Face Recognition and Biometric Systems

  33. Face relevance map • Maps from colour - examples Face Recognition and Biometric Systems

  34. Face relevance map • Back-projection based dynamic map • dynamic – created for every image • Back projection: • map of local projection error • higher error = lower importance • map should be smoothed • Good for occluded images Face Recognition and Biometric Systems

  35. Face relevance map • Back-projection based dynamic map • examples of occluded face images Face Recognition and Biometric Systems

  36. Recognition error • Maps take into account difference nature basing on face topology • Differences not concerned with location • lighting • Eigenfaces – appearance interpretation • various types of information • some responsible for lighting • Weights assigned to eigenvectors: Face Recognition and Biometric Systems

  37. Recognition error • Eigenvectors weights • lower values for intra-personal directions of variance • How to obtain the weights? • visual assessment – may be incorrect • the same procedure as in the case of difference masks Face Recognition and Biometric Systems

  38. Recognition error • A better method for obtaining maps and eigenvector weights: • error function minimisation Face Recognition and Biometric Systems

  39. Recognition error • Definition of recognition problem: • M vectors, C classes and C base vectors (ui1) • Mi vectors in i-th class (uij) • classification of non-base vectors (j > 1) • Single comparison • similarity to home class and foreign class • classes represented by base vectors Face Recognition and Biometric Systems

  40. Recognition error • Single comparison error: uij– a vector which is being recognised ui1 – home class base vector uk1 – foreign class base vector S – similarity between vectors Face Recognition and Biometric Systems

  41. Recognition error • Single comparison: • correct if • incorrect if Face Recognition and Biometric Systems

  42. Recognition error • Error for comparison with all classes: • Error for the whole set: Face Recognition and Biometric Systems

  43. Eigenfaces: feature extraction K1 K2 K3 ... ... Scalar products between normalised image and eigenvectors Feature vector Face Recognition and Biometric Systems

  44. Eigenfaces: feature extraction • Feature vector element ( ): - dimensionality of feature vector - normalised face image - i-th eigenvector • Improvements to the Eigenfaces • face relevance masks • eigenvector weights Face Recognition and Biometric Systems

  45. Eigenfaces: feature extraction • Improved feature extraction: - i-th eigenvector weight - j-th element (pixel) of the mask - j-th element of the i-th eigenvector - i-th element of the feature vector Face Recognition and Biometric Systems

  46. Eigenfaces: feature extraction • Similarity based on Euclidean distance: Face Recognition and Biometric Systems

  47. Error minimisation • Recognition error is a function of mask and eigenvector weights • The function may be minimised • optimal mask • optimal eigenvector weights • Example of mask optimisation... Face Recognition and Biometric Systems

  48. Error minimisation • Optimised dataset • problem of overfitting • How to avoid overfitting? • large datasets • optimisation can be stopped • Advantages of overfitting • overfitting to a group of people Face Recognition and Biometric Systems

More Related