Applying Computer Vision Techniques to the Problem of Face Recognition

1. Applying Computer Vision Techniques to the Problem of Face Recognition By OlamideAkintewe and Jacob Hamilton

2. Table of Contents Introduction Obtaining the Training Set/Face Database Obtaining the Eigenface Basis Testing and Results Light Regulator Circuit Conclusion

3. Abstract • There have been many advances made in computers and imaging hardware in the past few years. These advances have given birth to an array of computer vision related projects. • Automated computer vision systems that capture and process image data have been up until now mostly defense and military related. These include surveillance, targeting, and biometrics. • Future developments include virtual simulators, games and entertainment, improved computer interfaces, collision avoidance, and home robotics. The list is limited only to the imagination.

4. Introduction • The project entails recognition of an input face by comparing facial features and structure to that of a known person’s. • The method involved is a simple data set dimensionality reduction via Principle Component Analysis. • This method, when applied to face recognition, is known as the Eigenface approach. • With minimal modification, our algorithm can be applied to almost anything of interest.

5. Objective • Our objective is to create a system that takes an individual’s input image using a camera and being able to automatically recognize who the person is, if it is someone in the Database. • The uses include keyless access to areas of restricted access, whether it be a building, plane, car, room, laptop, etc.

6. Obtaining the Training Set/Face Database

7. Obtaining the Training Set/Face Database • The pictures used for the training set consisted of 24 8-bit grayscale frontal images, each of size 200 by 150. • These pictures were taken in a controlled environment under incandescent light all with the same camera. • The pictures were then converted from RGB to grayscale in MATLAB.

8. Obtaining the Training Set/Face Database • Upon conversion, the images had to be registered to the same base for optimum performance of the algorithm. • This registration was performed by picking one of the training set images (Base), resizing and cropping it to the desired 200 by 150 format, and lastly, registering all other images in the database to the Base for spatial and geometric coherence. • The registration was achieved by using a linear conformal geometric transform, alongside the control point selection tool in MATLAB. Each transform was unique to the pair of images involved.

9. Registration of Database Images

10. Obtaining the Eigenface Basis

11. Obtaining the Eigenface Basis • Each image in the database is arranged in a column wise format to avoid unnecessary dimensions during computation. Essentially, all 150 columns (containing 200 pixels ea.) of an image are concatenated into a single column vector of length 30000 pixels. • When all 24 images in the database are arranged in this format, one ends up with a 30000 by 24 matrix (A) encompassing all image data in the database.i.e. The 3rd column of A is the 3rd (Γj) face of the database. • At this point, we are faced with the challenge of representing each face as a linear combination of a basis set of faces, collectively known as the Eigenfaces of the system. • In order to do this, we need to effectively compute the eigenvectors of A.

12. Obtaining the Eigenface Basis • We begin by finding the mean face of the database (Ψ).Ψ = Sum(Γ1Γ2 … Γ24)/24 • Then we compute the zero mean images. Basically we subtract Ψ from every Γ in A to give us a zero mean database matrix (B) that has all its columns as database faces minus the mean face. B = A – Ψ • We attempted to compute the covariance matrix of our database using the formula Covariance = B*BT and solve for its eigenvectors. However, the resulting matrix (C) is of size 30000 by 30000, this computation was too memory intensive, even for MATLAB. This is where we employ Principal Component Analysis (PCA). Instead of attempting to compute all 30000 eigenvectors of C, we can get away with just 24. • Therefore, we compute a Surrogate matrix (S) = BT*B. For PCA tells us that where Vi is an eigenvector of S, B* Vi is an eigenvector of C. This is how the eigenvectors of C and thus the Eigenfaces of the database are computed.

13. Relevant Eigenfaces of Database These are the top 5 Eigenfaces collectively contributing 76.38% energy of the entire Eigenface set. Eigenface 1 = 32.5% Eigenface 2 = 16.73% Eigenface 3 = 11.68% Eigenface 4 = 8.03% EigenFace 5 = 7.43%

14. Relative Energy of each Eigenface

15. Testing & Results

16. Disguised Detection and Visualization of three dimensional Eigenspace Input Image Input Image Input Image Detected Image Detected Image Detected Image

17. The Eigenspace [1,2]

18. The Eigenspace [1,2,3]

19. The Eigenspace [1,2,3]

20. The Eigenspace [22,23,24]

21. Algorithms Used for Detection • Nearest Neighbor. • As the name suggests, this algorithm takes in an input picture, projects it onto the Eigenspace and computes the distance between the input image and every image in the database. • As every individual has 3 images in the database, it simply computes the 3 smallest distances between the input image and all images in the database. • Of the 3 closest images to the input, a majority rule is applied. That is, if 2 of the 3 closest images are person A and the other is person B, the algorithm detects person A (even if person B happens to the closest). • In the rare event that the algorithm experiences a 3 way tie, the algorithm detects the person who had the closest proximity to the input image.

22. Algorithms Used for Detection • Mean. • This algorithm takes in an input picture, projects it onto the Eigenspace and computes the distance between the input image and the mean projection of every unique individual in the database. • It accounts for stability and is less sensitive to extreme images in the training set. However, it does lose out on the variety of options that is made available to the Nearest Neighbor algorithm. • Both algorithms performed satisfactory; however, the Nearest Neighbor algorithm performed slightly better in testing.

23. The Effect of Number of Eigenfaces on Rate of Recognition • Naturally, the rate of recognition increases as one increases the dimensions used in linear combination of faces. However, there is a threshold at which the use of additional Eigenfaces does nothing to improve recognition rate. • Theoretically, this threshold is Number of images in database – Number of unique individuals in database. Which would be 16 Eigenfaces in our case. • However, we reached this threshold using a mere 5 Eigenfaces • This is most likely due to the nature of our database (Very diverse, High image per person to number of unique persons ratio)

24. Recognition Rate of Nearest Neighbor Algorithm

25. Recognition Rate of Mean Algorithm

26. Theoretical Recognition Rates

27. Effect of Input Image Decimation on Recognition rates Scaled 80% Scaled 60% Scaled 40% No Decimation Scaled 20% Scaled 10% Scaled 5% Scaled 3%

28. Effect of Input Image Decimation on Recognition rates • Decimation had no real effect on recognition rate until 5%, and even then it often times worked. • Ideally this should not be the case; however, our database is so diverse that even images decimated to that extent can be recognized because of the limited choices the algorithm has to make. • Moreover, as previously described, an increase in the number of optimum Eigenfaces (above 5) played no significant role in aiding recognition of decimated images.

29. Light Regulator Circuit

30. Explanation of Circuit Based on the intensity of the light present in the room, the circuit would output the desired amount of light needed to capture a good quality image. If the room is darker, we need to output more light. The circuit consisted of 4 elements with different thresholds that varied their sensitivity to light. The rightmost element required two hands to block the light and output a high in most cases simulating a very dark room, while the leftmost element needed a flashlight to turn the output off simulating daylight. We accomplished our functionality with the use of only phototransistors, comparators, op-amps, LEDs, and resistors.

31. VO

32. Light Output To demonstrate functionality we used 3 multicolored LEDs as our output. In reality, we need a 120V, 60W incandescent light bulb such as the one we used for lighting in obtaining our database pictures. To do this, we use a typical 12V to 120V transform circuit shown in the next slide. It utilizes a center tapped transformer, with our input being the output from the previous circuit, 4 discrete voltage levels, which is proportional to the current flowing through the secondary branch.

33. Circuit Output The output is the average of the four 11V elements VO= 0, 2.75, 5.5, 8.24, 11 V VSecondary= 0, 30, 60, 90, 120 V VO

34. Conclusion

35. Challenges Setting a rejection threshold is possible for our system only if the lighting is held fixed. From the decimation testing it is clear our system is very robust if the lighting is constant. This is because the algorithm compares relative intensities between pictures and having a different lighting condition essentially makes for a different face to the computer. Therefore, our product is only practical for indoor uses such as entry to rooms, etc where the light is fixed.

36. Potential Ethical Issues • This technology could possibly be used by the government or other party in a way that invades personal privacy. • It also could be used to further the technology of guided weaponry such as targeting missiles and projectiles.

37. Questions? • We will answer questions now, thank you for your time and have a good summer!