以區域二元圖樣與部分比對為基礎之人臉辨識 Face Recognition with Local Binary Patterns and Partial Matching

1. 以區域二元圖樣與部分比對為基礎之人臉辨識Face Recognition with Local Binary Patterns andPartial Matching Presenter: 施佩汝 Advisor: 歐陽明 教授

2. Outlines • Motivation • Implementation • Result • Conclusion

3. Motivation

4. Motivation

5. Publication • Che-Hua Yeh, Pei-Ruu Shih, Kuan-Ting Liu, Yin-Tzu Lin, Huang-Ming Chang, Ming Ouhyoung. A Comparison of Three Methods of Face Recognition for Home Photos. ACM Siggraph, poster, 2009.

6. Problem Statement

7. Main Contribution • Improve Local Binary Patterns by using Partial Matching Metric • Better Performance in Home Photos

8. Implementation

9. System Overview Images Descriptors Pre-Processing Build Descriptor Clustering Calculate LBP Build Descriptor

10. System Overview Images Descriptors Pre-Processing Build Descriptor Clustering Calculate LBP Build Descriptor

11. Pre-Processing

12. System Overview Images Descriptors Pre-Processing Build Descriptor Clustering Calculate LBP Build Descriptor

13. System Overview Images Descriptors Pre-Processing Build Descriptor Clustering Calculate LBP Build Descriptor

14. Local Binary Patterns [PAMI2006] • An operator to encode the relationship of a pixel and its neighbors.

15. Local Binary Patterns [PAMI2006] • An operator to encode the relationship of a pixel and its neighbors.

16. Local Binary Patterns [PAMI2006] • An operator to encode the relationship of a pixel and its neighbors.

17. Local Binary Patterns [PAMI2006] • An operator to encode the relationship of a pixel and its neighbors.

18. Local Binary Patterns [PAMI2006] • An operator to encode the relationship of a pixel and its neighbors.

19. Local Binary Patterns [PAMI2006] • An operator to encode the relationship of a pixel and its neighbors.

20. Local Binary Patterns [PAMI2006] • An operator to encode the relationship of a pixel and its neighbors.

21. Local Binary Patterns [PAMI2006] • An operator to encode the relationship of a pixel and its neighbors.

22. Local Binary Patterns [PAMI2006] • An operator to encode the relationship of a pixel and its neighbors. LBP = 11010011

23. System Overview Images Descriptors Prepared-Works Build Descriptor Clustering Calculate LBP Build Descriptor

24. Facial Image Descriptor • They use Spatially Enhanced Histogram in original Local Binary Pattern. [PAMI2006]

25. Local Patches • We sample a patch for every s pixels. • There are S patches for one image. s s

26. Spatial Block [CVPR2007] • We use three concentric circles to describe a patch.

27. Descriptor • Build a descriptor for one face.

28. System Overview Images Descriptors Pre-Processing Build Descriptor Clustering Calculate LBP Build Descriptor

29. System Overview Images Descriptors Pre-Processing Build Descriptor Complete-Linkage Clustering Calculate LBP Build Descriptor Compute all the similarities Clustering

30. System Overview Images Descriptors Pre-Processing Build Descriptor Complete-Linkage Clustering Calculate LBP Build Descriptor Compute all the similarities Clustering

31. Similarity • They use the weighted Chi-Square distance in original Local Binary Pattern. [PAMI2006]

32. Partial Matching [ICCV2009] , • Step1: • Compute the similarity of each patch from one image with the nearby patches in another image. Image 1: I(1) Image 2: I(2)

33. Partial Matching [ICCV2009] • Step2: • Sort the similarities of all patches. • dαSis the similarity of I(1) to I(2).

34. Partial Matching [ICCV2009] • Step3: • Calculate the similarity of I(2) to I(1)

35. Partial Matching [ICCV2009] • Step4: • Use the maximum of two similarity

36. System Overview Images Descriptors Pre-Processing Build Descriptor Complete-Linkage Clustering Calculate LBP Build Descriptor Compute all the similarities Clustering

37. Hierarchical Clustering • Build a tree-based hierarchical taxonomy (dendrogram) from a set of documents. Material Selected from Introduction to Information Retrieval, Christopher D. Manning, Prabhakar Raghavan and Hinrich Schütze, 2008.

38. Hierarchical Clustering • Clustering obtained by cutting the dendrogram at a desired level: each connected connected component forms a cluster. Material Selected from Introduction to Information Retrieval, Christopher D. Manning, Prabhakar Raghavan and Hinrich Schütze, 2008.

39. Hierarchical Complete-Linkage Clustering • Similarity of the “furthest” points. • Makes “tighter,” spherical clusters that are typically preferable. Material Selected from Introduction to Information Retrieval, Christopher D. Manning, Prabhakar Raghavan and Hinrich Schütze, 2008.

40. Performance Optimization • 4 threads in Quad-Core system Images Descriptors Pre-Processing Build Descriptor Complete-Link Clustering Calculate LBP Build Descriptor Compute all the similarities Clustering

41. Performance Optimization • 4 threads in Quad-Core system • 3 times faster than single thread. • 73 minutes to 24 minute for 309 images. Images Descriptors Pre-Processing Build Descriptor Complete-Link Clustering Calculate LBP Build Descriptor Compute all the similarities Clustering

42. Result

43. FERET Result • fa: gallery, 994 images • fb: alternative facial expression, 992 images • dup1: the photos taken after later, 736 images • dup2: the photos taken at least one year after the gallery, 228 images

44. FERET Result ※ The time results are computed in multithreads version.

45. Experiments • Home Photo Dataset I • 309 images, 5 subjects • Home Photo Dataset II • 838 images, 8 subjects

46. Evaluation • Cluster Number • Unknown Number • Pair-wise Precision • Pair-wise Rand Index • Executing time

47. Unknown Number • The number of clusters which contain only one component.

48. Precision/Rand Index

49. Dataset-I Result LBP: Local Binary Pattern, PM: Partial Matching, SB: Spatial Block ※ The time results are computed in multithreads version.

50. Dataset-I Result LBP Our Result