Download Presentation
## Face Recognition

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -

**Face Recognition**Image Understanding Xuejin Chen**Face Recogntion**• Good websites • http://www.face-rec.org/ • Eigenface [Turk & Pentland] Image Understanding, Xuejin Chen**Eigenface**• Projecting a new image into the subspace spanned by the eigenfaces (face space) • Classifying the face by comparing its position in face space with the positions of known individuals Image Understanding, Xuejin Chen**Face image used as the training set**Image Understanding, Xuejin Chen**Average face**• 7 eigenfaces calculated from the training images Image Understanding, Xuejin Chen**Calculating Eigenfaces**Face image : intensity matrix -D vector PCA: to find the vectors that best account for the distribution of face images within the entire image space Training images Average face Each face differs from the average by Seeks a set of M orthogonal vectors that best describe the distribution of the data The kth vector is chosen such that Image Understanding, Xuejin Chen**Calculating Eigenfaces**• The vector and scalar are the eigenvectors and eigenvalues, respectively, of the covariance matrix Intractable task for typical image size Need a computationally feasible method to find these eigenvectors Image Understanding, Xuejin Chen**Calculating Eigenfaces**• If image number M < N^2, only M meaningful eigenvectors • Consider eigenvectors vi of A’A such that 16x16 16384*16384 (an 128x128 image) Solve MxM matrix Image Understanding, Xuejin Chen**Calculating Eigenfaces**• Construct MxM matrix • Compute M eigenvectors of L • The M eigenfaces is the linear combination of the M eigenvectors The computations are greatly reduced Image Understanding, Xuejin Chen**Average face**• 7 eigenfaces calculated from the training images Image Understanding, Xuejin Chen**Classify a Face Image**• 40 eigenfaces are sufficient for a good description for an ensemble of M=115 images [Sirovich and Kirby 1987] • A smaller M’ is sufficient for identification • C • hoose M’ significant eigenvectors of L matrix (M=16, M’=7) Image Understanding, Xuejin Chen**Classify a Face Image**• New image : transformed into its eigenface components Image Understanding, Xuejin Chen**Classify a Face Image**• describe the contribution of each eigenface in representing the input image, treating the eigenfaces as a basis set for face image • Find a class of the face • A simple distance Average of a class or individual Image Understanding, Xuejin Chen**Classify a Face Image**• Creating the weight vector is: projecting the original face image onto the low-D face space • Distance from image to face space Image Understanding, Xuejin Chen**Four possibilities for an image and pattern vector**• Near face space and near a face class • Near face space but not near a known face class • Distant from face space but near a face class • Distant from face space and not near a known face class Image Understanding, Xuejin Chen**Distance to face space**• 29.8 • 58.5 • 5217.4 Image Understanding, Xuejin Chen**Summary of Eigenface Recognition**• Collect a set of characteristic face image of the known individuals. • The set should include a number of images for each person, with some variation in expression and lighting (M=40) • Calculate the 40x40 matrix L, find its eigenvalues and eigenvectors, choose M’(~10) eigenvectors with highest eigenvalues • Compute eigenfaces • For each known individual, calculate the class vector by averaging the eigenface pattern vector . • Choose a threshold that defines the maximum allowable distance from any face class and • a threshold that defines the maximum allowable distance from face space Image Understanding, Xuejin Chen**Summary of Eigenface Recognition**• For each new face image to be identified, calculate • its pattern vector , • the distance to face space , • the distance to each known class . • if the minimum distance • Classify the input face as the individual associated with class vector • If the minimum distance • The image may be classified as unknown, and • Optionally used to begin a new face class • Optionally, if the new image is classified as a known individual, the image may be added to the original set of familiar face image, and the eigenfaces may be recalculated (steps 1-4) Image Understanding, Xuejin Chen**Locating and Detecting Faces**• Assume a centered face image, the same size as the training images and the eigenfaces • Using face space to locate the face in image • Images of faces do not change radically when projected into the face space, while the projection of nonface images appears quite different • > detect faces in a scene Image Understanding, Xuejin Chen**Use face space to locate face**• At every location in the image, calculate the distance between the local subimage and face space, which is used as a measure of ‘faceness’ a face map Expensive calculation Image Understanding, Xuejin Chen**Face Map**• A subimage at (x,y) Image Understanding, Xuejin Chen**Face Map**• : linear combination of orthogonal eigenface vectors Image Understanding, Xuejin Chen**Face Map**Correlation operator Image Understanding, Xuejin Chen**Face Map**Precomputed L+1 correlations Can be implemented by a simple neural networks Image Understanding, Xuejin Chen**Learning to Recognize New Faces**• An image is close to face space, but not classified as one of the familiar faces, labeled as “unknown” • If a collection of “unknown” pattern vectors cluster in the pattern space, a new face is postulated • Check similarity: the distance from each image to the mean is smaller than a threshold • Add the new face to database (optionally) Image Understanding, Xuejin Chen**Background Issue**• Eigenface analysis can not distinguish the face from background • Segmentation? • Multiply the image by a 2D Gaussian window centered on the face • Deemphasize the outside of the face • Also practical for hairstyle changing Image Understanding, Xuejin Chen**Scale and Orientation Issue**• Recognition performance decreases quickly as the size is misjudged • Motion estimation? • Multiscaleeigenfaces / multiscale input image • Non-upright faces • Orientation estimation using symmetry operators Image Understanding, Xuejin Chen**Distribution in Face Space**• Nearest-neighbor classification assumes Gaussian distribution of an individual feature vector • No prior to assume any distribution • Nonlinear networks to learn the distribution by example [Fleming and Cottrell, 1990] Image Understanding, Xuejin Chen**Multiple Views**• Define a number of face classes for each person • Frontal view • Side view at ± 45° • Right and left profile views Image Understanding, Xuejin Chen**Experiments**• Database • Over 2500 face images under controlled conditions • 16 subjects • All combinations of 3 head orientations, 3 head sizes, 3 lighting conditions • Construct 6-level Gaussian pyramid from 512x512 to 16x16 Image Understanding, Xuejin Chen**Variation of face images for one individual**Image Understanding, Xuejin Chen**Experiments with Lighting, Size, Orientation**• Training sets • One image of each person, under the same lighting condition, size, orientation • Use seven eigenfaces • Mean accuracy as the difference between the training conditions, test conditions • Difference in illumination • Image size, • Head orientation • Combinations of illumination, size, orientation Image Understanding, Xuejin Chen**Changing lighting conditions --- few errors**• Image size changing -- performance dramatically drops • Need multiscale approach (a) Lighting 96% (b) Size 85% (c) Orientation 64% (d) Orientation & lighting (e) Orienation & Size 1 (f) Orientation & Size 2 (g) Size & Lighting 1 (h) Size & Lighting 2 Image Understanding, Xuejin Chen**Experiments with varying thresholds**• Smaller threshold: • Few errors, but more false negative • Larger threshold • More errors • To achieve 100% accurate recognition, boost unknown rate to • 19% while varying lighting • 39% for orientation • 60% for size • Set the unknown rates to 20%, the correct recognition rate • 100% for lighting • 94% for orientation • 74% for size Image Understanding, Xuejin Chen**Neural Networks**• Can be implemented using parallel computing elements Image Understanding, Xuejin Chen**Collection of networks to implement computation of the**pattern vector, projection into face space, distance from face space, and identification Image Understanding, Xuejin Chen**Conclusion**• Not general recognition algorithm • Practical and well fitted to face recognition • Fast and simple • Do not require perfect identification • Low false-positive rate • A small set of likely matches for user-interaction Image Understanding, Xuejin Chen**Eigenface**• Tutorial Image Understanding, Xuejin Chen**Bayesian Face Recognition**BabackMoghaddam, Tony Jebaraand Alex Pentland Pattern Recognition 33(11), Nov. 2000**Novelty**• A direct visual matching of face images • Probabilistic measure of similarity • Bayesian (MAP) analysis of image differences • Simple computation of nonlinear Bayesian similarity**A Bayesian Approach**• Many face recognition systems rely on similarity metrics • nearest-neighbor, cross-correlation • Template matching • Which types of variation are critical in expressing similarity?**Probabilistic Similarity Measure**• Intensity difference • Two classes of facial image variations • Intrapersonal variations • Extrapersonal variations • Similarity measure Non-Euclidean similarity measure Can be estimated using likelihoods given by Bayes rule**A Bayesian Approach**• First instance of non-Euclidean similarity measure for face recognition • A generalized extension of • Linear Discriminant Analysis • FisherFace • Has computational and storage advantages over most linear methods for large database**Probabilistic Similarity Measures**• Previous Bayesian analysis of facial appearance • 3 different inter-image representations were analyzed using the binary formulation • XYI-warp modal deformation spectra • XY-warp optical flow fields • Simplified I-(intensity)-only image-based difference**Probabilistic Similarity Measures**• Intrapersonal variations • Images of the same individual with different expression, lighting conditions.. • Extrapersonal variations • Variations when matching two individuals • Both are Gaussian-distributed, learn the likelihoods**Probabilistic Similarity Measures**• Similarity score • The priors can be set as the portion of image number in the database or specified knowledge • Maximum Posterior (MAP)**Probabilistic Similarity Measures**• M individuals: M classes • Many classification -> binary pattern classification • Maximum likelihood measure • Almost as effective as MAP in most cases**Subspace Density Estimation**• Intensity difference vector: high dimensional • No sufficient independent training examples • Computational cost is very large • Intrinsic dimensionality or major degree-of-freedom of intensity difference is significantly smaller than N • PCA • Divides the vector space R^N into two complementary subspaces [Moghaddam & Pentaland]**Subspace Density Estimation**• Two complementary subspaces A typical eigenvalue spectrum