Since 2008 Google street view has blurred faces for privacy.

1. Since 2008 Google street view has blurred faces for privacy.

2. The system mostly works well, but does have both false positive and false negatives

3. Let us see if we could solve this problem, with what we have learned in this class. First, a small insight. Let us solve the skin detection problem, instead of the face detection problem. What features can we use? Every color pixel in an image, is represented as a combination of three colors, typically RedGreenBlue, but there are other possibilities, HSV (hue, saturation, value), HSL (hue, saturation, lightness/luminance) etc. R B G

4. R B G

5. R B G R = If we project a skin pixel into 3D space, then you could use the following, human coded rule to classify skin pixels: If (R > 95) AND (G > 40) AND (B > 20) AND (max{R, G, B} − min{R, G, B} > 15) AND (|R − G| > 15) AND (R > G) AND (R > B) Then IsSkin However, this rule is not perfect (next slide) G = B =

6. The rule works pretty good here, but it has a lot of false positives, the plate on the wall, the stripy sweater, parts of the couch on the bottom left etc.

7. Let us try to do better, with machine learning. First we need to get training data to build our classifier. Recall that for machine learning problems, it can sometimes be useful to transform features. Instead of height and weight, use weight/ height2 Maybe there is a transformation of the color space that makes this problem easer? We could ask an expert, or just compare all color spaces using leave-one-out classification.

8. 250 200 150 100 50 0 0 50 100 150 200 250  In YCbCr, the Y is the brightness (luma), Cb is blue minus luma (B-Y) and Cr is red minus luma (R-Y). Y = Cr Cb = Cb Cr =

9. 250 200 150 100 50 0 0 50 100 150 200 250 Cr > 124? Cr Cr < 174? Not Skin Cb < 90? Not Skin Cb Cb > 140? Not Skin Skin Not Skin

10. 250 200 150 100 50 0 0 50 100 150 200 250 Cr Cb RGB-H-CbCr Skin Colour Model for Human Face Detection.Nusirwan Anwar Abdul Rahim, Kit Chong Wei, John See.MMU International Symposium on Information & Communications Technologies (M2USIC 2006),

11. 250 200 150 100 50 0 0 50 100 150 200 250 This is a hand coded piecewise linear model Cr ≤ 1.5862 × Cb + 20 Cr ≥ 0.3448 × Cb + 76.2069 Cr ≥ -4.5652 × Cb + 234.5652 Cr ≤ -1.15 × Cb + 301.75 Cr ≤ -2.2857 × Cb + 432.85 Cr Cb RGB-H-CbCr Skin Colour Model for Human Face Detection.Nusirwan Anwar Abdul Rahim, Kit Chong Wei, John See.MMU International Symposium on Information & Communications Technologies (M2USIC 2006),

12. A simple nearest neighbor algorithm in the YCbCr space works very well. How could we make it better?

13. Extensions For example, if we wanted to find examples of FedEx trucks on streetview, we could build a classifier for each color in the logo (Purple, Orange and White). If a small area of the image had all three colors, we could classify truck seen. Why not just “read” the text? Consider the below… We can use basic classifiers as a building blocks for more complex questions

15. Less we get too complacent I How well could you do on this problems? Google “Karen Zack” for more examples

16. Less we get too complacent II How well could you do on this problems?