1. Simon Fraser University Computational Vision Lab Lilong Shi, Brian Funt and Tim Lee Studies in Appearance of Skin

2. Overview • Studies of factors affecting skin colour • Simple and linear model of skin • Modelling Skin appearance under lights • Applications: • Estimate melanin and hemoglobin concentrations • Correct imaged skin tones for lighting conditions

3. Applications Skin tone correction Tone correction Preserve melanin Melanin/Hemoglobin separation

4. Appearance of Human Skin • Appearance of human skin determined by • Biological factors • pigmentation, blood microcirculation, roughness, etc.. • Viewing conditions • Inducing lights • Acquisition devices • Cones in retina, RGB sensors of CCD digital cameras

5. Schematic Model of Human Skin • Two-layered Skin Model [2] • Epidermis Layer: melanin absorbance • Dermis Layer: hemoglobin absorbance • A layer has properties of an optical filter

6. Biological Factors • Various skin colour <= melanin + hemoglobin • Genetic: Race • Temporary: • Exposure to UV • Hot bath • Mixture varying by 2 independent factors • Analyse melanin and hemoglobin factors

7. Blind Source Separation • Estimate melanin and hemoglobin concentration • Independent Component Analysis (ICA) • Statistical technique for revealing “hidden” factors • To “unmix” or “separate” signals composed of multiple sources • Independent and linear mixing • Related to Eigen-vector analysis

8. Independent Component Analysis (ICA) Original Source Signals Mixing Observed Signals 70% v1 0% 20% 80% 30% s1 s2 100% v2 v3 s × A = v

9. Independent Component Analysis (ICA) Melanin Melanin Hemoglobin Hemoglobin Skin samples

10. A Example of Skin basis • Typical skin spectrum • Visible wavelength 400nm – 700nm • Extract skin bases from observed spectrum by ICA (left) 33 skin spectrum after normalization; (right) two independent basis spectrum – the melanin and hemoglobin, and the spectrum of chromophores other than melanin and hemoglobin pigments. ICA

11. Linear Model of Skin • Arbitrary skin spectrum can be approximated are variables constru

12. Signal from Digital Camera • Human vision • 3 types of Photoreceptors • L, M and S Cones • Digital Cameras • 3 sensors • Red, Green, and Blue • Reflectance spectrum recorded by 3 sensors => three values (R, G, B) for a skin colour

13. Skin Model in Camera Space Given a pixel from skin, compute by projecting log(R,G,B) onto Possible skin colours lie within plane

14. Real Image Result Melanin Image Hemoglobin Image Input Image [3]

15. Results on 33 Skin Spectra - Inverse melanin concentration - Inverse hemoglobin concentration

16. Modelling Skin + Illumination • Skin appearance greatly affected by lights • Reveal true skin colour by removing illum. • Common lights blackbody radiation • e.g. tungsten/halogen lamps, sunrise/sunset, etc • Varying colour temperature T • Redish -> white -> bluish

17. Skin-Illumination Model • Colour: illumination times reflectance • In log space, multiplication => addition: Illum. basis

18. Simplified Skin-Illum Model • In practice • Drop hemoglobin basis • Small angle between Illum and hemoglobin axes • Ignore brightness • Skin colour varying by T and

19. Result based on Synthesized Data • 384 real skin reflectances times • 67 real light sources • => 25728 samples

20. Result of Tone Correction • Skin tone correction example (UOPB DB [4]) 16 different illumination + camera settings Tone correction Preserve melanin

21. Result of Tone Correction • Skin tone correction example (UOPB DB [4])

22. Conclusion • Skin colour modelling: • Melanin and Hemoglobin concentration • Linear model in logarithm space • Estimation by Independent Component Analysis • Skin appearance + Light modelling: • Estimates light source • Preserves skin colour by melanin value • Applied to digital images from CCD cameras

23. Reference • [1] Shi, L., and Funt, B., "Skin Colour Imaging That Is Insensitive to Lighting," Proc. AIC (Association Internationale de la Couleur) Conference on Colour Effects & Affects, Stockholm, June 2008 • [2] Angelopoulou, E., Molana, R., and Daniilidis, K. “Multispectral skin color modeling,” In IEEE Conf. on Computer Vision and Pattern Recognition, volume 2, pages 635-642, Kauai, Hawaii, Dec. 2001. • [3] Shimizu, H., Uetsuki, K., Tsumura, N., and Miyake, Y. Analyzing the effect of cosmetic essence by independent component analysis for skin color images. In 3rd Int. Conf. on Multispectral Color Science, pages 65-68, Joensuu, Finland, June 2001. • [4] Martinkauppi, B. “Face color under varying illumination-analysis and applications,” Ph.D. Dissertation, University of Oulu, 2002.