EE368Group04 Face Detection by color segmentation and template matching

1. EE368Group04Face Detection by color segmentation and template matching Scott Tan Boonping Lau Chun Hui Weng

2. Definition of Problem • Multiple face detection in images • With cluttered background, high degree of occlusion • Limited scale and rotational invariance • Constant lighting condition • GOAL: design easily extendable to general image

3. Approach • Color segmentation + texture filtering • Multi-resolution face + eyes template matching

4. Input image For each resolution… Pre-processing eyes template Eye template matching Face template matching face template Pair eye/face hits Color segmentation Skin area test color mask face size mask Clustering Clustering output from other resolutions output face positions

5. Color Segmentation • YCbCr color space • Histogram model from pixels under ref*.png masks • Threshold=900 pixels (empirical) • If histo(Cb,Cr)>threshold => skin pixel • Problem: skin pixel sample base small, model over-specific !!

6. Color Segmentation (Cont.) • Solution: • Elliptical boundary model • Build more general histogram on larger skin pixel sample base under different conditions • Compromise between performance under test image condition and more general conditions

7. Color Segmentation: Enhancements • Texture filtering Grayscale variance threshold=1000 (emp.) If region variance<=threshold => non-skin • Binary operations Close, fill holes, remove small regions • Aspect ratio test Unreliable under high degree of occlusion/clutter

8. Color Segmentation Mask Example

9. Template Matching • Why? • Flexibility of implementation • Extensibility despite limited sample base • Predictability under occlusion

10. Template Matching (Cont.) • Face template built from averaging pre-processed faces • Pre-processing: local mean removed => lighting condition invariance • Correlation done in spatial frequency domain on subsampled image and templates => reduce processing time • Soft thresholding: mean+n*standard deviation • Post-processing: reject border hits

11. Face+Eyes Template Matching • Face hit marked base of nose • Eyes hit marked between eyes • Pair dilated hits (E hit above F hit) • Advantages: 1) reduce individual false hit rates 2) reduce thresholds: detect under occlusion 3) limited stretch/rotational invariance

12. Face and Eyes Templates

15. Skin area test • Removes false and non-centered hits on edge of face • For each hit, take a oval-shaped region around it and count # of skin pixels (from skin segmentation mask) • If less than 75% of skin pixels, then reject • Dependent on quality of segmentation mask

17. Clustering • After skin area test, cluster remaining hits within a certain distance apart (1/2 a face distance) • Hits weighted by correlation • Each cluster represented by a single pixel at centroid => slight translation of detection pixel

19. Multi-resolution • Limited scale invariance needed • Use 4 different scales: 90%, 100%, 120%, 130% • Vary image size vs vary template size • Latter can be done ahead of time • Logical OR to collapse multi-resolution template matching results • Clustering algorithm to eliminates hits at different resolutions

22. Results

29. Conclusion • Excellent overall results, even with occluded faces, slightly rotated faces, different face sizes etc. • Template matching detects 162/164 faces • Methods are extendable to any general image • Skin color mask is limiting factor