The Measurement of Highlights in Color Images Gudrun J. Klinker, Steven A. Shafer

1. The Measurement of Highlights in Color ImagesGudrun J. Klinker, Steven A. Shafer Presented by: Huizhuan Wu Nov 30th, 04

2. Outline An introduction to light physics Dichromatic Reflection Model Color images with Real Cameras Detecting and Removing Highlight Discussions Besides modeling the light reflection in a theoretical physical model. The observed pixels values are also influenced by the characteristics of the recording camera. I will coer the influences of such camera characteristics on the image data. Besides modeling the light reflection in a theoretical physical model. The observed pixels values are also influenced by the characteristics of the recording camera. I will coer the influences of such camera characteristics on the image data.

3. An introduction to light physics Laws of Reflection, Refraction and Transmission Dielectric Materials Geometric and Photometric Properties of Body Reflection Dichromatic Reflection Model Color images with Real Cameras Detecting and Removing Highlight Discussions Where are we??

4. The physics of Light Reflection and Refraction Law of Reflection: ?i = ?r Snell's Law: n1sin(?i) = n2sin(?t) Physics Need to redo this slide. Come up with more informative and precise sentences. Make clear the two component of lights Reflection Refraction Defines incident light, exit, refrat, relationship btw angles Intensity issues: Intensity of rays: Reflection + refraction = incident ray Define Transmittance coefficience Transmittance coefficient is independent of the intensity of the incident light (Lambert’s law) May want to mention Beer’s law, which states that the absorbed light is proportional to the density of the pigment and the depth travelled Absorption ( scattering Snell's Law describes the relationship between the angles and the velocities of the waves. Snell's law equates the ratio of material velocities n1 and n2 to the ratio of the sine's of incident (theta i) and refraction (theta t) angles. Fresnel's equations describe the reflection and transmission of electromagnetic waves at an interface. Lambert’s law says that the fraction of transmitted light is independent of the the incident light intensity. Beer’s law: light absorption is propotional to the concentration© of the absorbant medium and the thickness. Physics Need to redo this slide. Come up with more informative and precise sentences. Make clear the two component of lights Reflection Refraction Defines incident light, exit, refrat, relationship btw angles Intensity issues: Intensity of rays: Reflection + refraction = incident ray Define Transmittance coefficience Transmittance coefficient is independent of the intensity of the incident light (Lambert’s law) May want to mention Beer’s law, which states that the absorbed light is proportional to the density of the pigment and the depth travelled Absorption ( scattering Snell's Law describes the relationship between the angles and the velocities of the waves. Snell's law equates the ratio of material velocities n1 and n2 to the ratio of the sine's of incident (theta i) and refraction (theta t) angles. Fresnel's equations describe the reflection and transmission of electromagnetic waves at an interface. Lambert’s law says that the fraction of transmitted light is independent of the the incident light intensity. Beer’s law: light absorption is propotional to the concentration© of the absorbant medium and the thickness.

5. Fresnel's Equations  Fresnel’s Law: compute the fraction of a light wave reflected and transmitted Reflection coefficients r? = tan(?i-?t)/tan(?i+?t) r?= -sin(?i-?t)/sin(?i+?t) Transmission coefficients t? = 2sin?tcos?i/sin(?i+?t)cos(?i-?t) t?= 2sin?tcos?i/sin(?i+?t)

6. Quantitation of Light Transmissions Lambert’s Law: T(?) = Itrans(?)/Iinc(?) Beer’s Law: A(?) = e(?)Cd The intensity of transmitted light is proportional to the intensity of incident light. The ration T, is called transmittance, which is independent of the light intensity and can depend on the wave length. (Labert’s law) The Aborbance is the logarithm of the inverse of T. It is proportional to the concentration of the absorbant/pigment, and the thickness/depth of the medim. The proportional constant (epsilon) is a function of wavelength, and it is a intrinsic property of the color pigment. The intensity of transmitted light is proportional to the intensity of incident light. The ration T, is called transmittance, which is independent of the light intensity and can depend on the wave length. (Labert’s law) The Aborbance is the logarithm of the inverse of T. It is proportional to the concentration of the absorbant/pigment, and the thickness/depth of the medim. The proportional constant (epsilon) is a function of wavelength, and it is a intrinsic property of the color pigment.

7. Image Understanding Method Drop the word Goal? Why IUM? Redraw the rays, May want to label the art clip, or just omit them completely. Say them instead. Drop the word Goal? Why IUM? Redraw the rays, May want to label the art clip, or just omit them completely. Say them instead.

8. Light reflection of dielectric materials Use autoshape to cover the intelligible fuzzy labels. Retype them. Use autoshape to cover the intelligible fuzzy labels. Retype them.

9. Geometric and Photometric Properties of Body Reflection The transmitting properties of the medium The scattering and absorption properties of pigments The shape and distribution of the pigments Influences the amount and color the reflected light Randomly distributed -> same color over the entire surface Capitalize the 1st letter of the real words in the title to be consistant The random assumptionsCapitalize the 1st letter of the real words in the title to be consistant The random assumptions

11. Assumptions on Dichromatic Reflection Model Pigments randomly distributed and completely embedded Single spectrum of body reflection One light source No ambient light or inter-reflection Drop Assumptions Rearrange bullets: 4, 3, 1, 2Drop Assumptions Rearrange bullets: 4, 3, 1, 2

12. Dichromatic Reflection Model L(?, i, e, g) = Ls(?, i, e, g) + Lb(?, i, e, g) i: incidence angle e: exit angle g: phase angle Ls: surface reflected light Lb: body reflected light Indicate where the senser is.Indicate where the senser is.

13. Dichromatic Reflection Model L(?, i, e, g) = ms(i, e, g) Cs(?) + mb(i, e, g) Cb(?) What is C and m, and geometric factor? What is C and m, and geometric factor?

14. The shape of the spectral cluster for a cylindrical object Succinctly label the two graphs (short titles) Succinctly label the two graphs (short titles)

15. Spectral Cluster on the Dichromatic Plane

16. Where are we??

17. Problems associated with Real Images Data Representing a continuous light spectrum as a finite-dimensional vector Limit dynamic rage of cameras Changes in the camera’s responsivity as a function of wavelength and intensity

18. Sensor Model Spectral Integration Impossible to measure continuous light spectrum Sample measurements L(?,i,e,g) – amount of incoming light tf(?) – spectral transmittance of filter s(?) – spectral responsivity of the camera Light spectrum is continuous, analogue in nature Sampling on R, G, B components, and discrete digitalized valueLight spectrum is continuous, analogue in nature Sampling on R, G, B components, and discrete digitalized value

19. Dichromatic reflection model in 3-Dimensional Color Space C(x, y) = ms(i, e, g) Cs + mb(i, e, g) Cb  Upper Case 11 and 14 use same title, change to DRM in RGG color space Swap the bullet and the graph, move the bottom line on to the top of the graphUpper Case 11 and 14 use same title, change to DRM in RGG color space Swap the bullet and the graph, move the bottom line on to the top of the graph

20. Color cluster in the color cube UppcaseUppcase

21. Spectral responsivity SR of CCD cameraSR of CCD camera

22. Color balancing Color balancing Aperture control IR suppressor Gamma-correction Upper case May want split into more slides Use tables as the quantitive measureUpper case May want split into more slides Use tables as the quantitive measure

23. Spectral Linearization

24. Color Cluster from Real Color Images What title? What do you want to say here?What title? What do you want to say here?

25. Orange cup underyellow light(Blooming effects) Blooming effects Please describe. May want to exchange 18 and 19Blooming effects Please describe. May want to exchange 18 and 19

26. Where are we??

27. Determining the Body and Surface Reflection Vectors Algorithm(Stage One) Not able to distinguish between several color clusters in a color cube Recursive Line Splitting method (i) Choose L (ii) Divide the plane into strips (iii) Compute d(s) (iV) Determine max d(si) (v) Smooth d(s) (vi) d(si) > threshold, split L into two segments (vii) goto step iii, recompute d(s) Algorithm Deriving/computing the reflection vectors (or you can use the subsection title) You can give an overview (properties, pros and cons of this algorithm), them move to the specifics of the algorithm. So, split into 2 slides Change the title to Strip, in the text stripes Fonts not consistent Iv should be break the points at point mas d(si) Or, you should swap iV and v. You smooth to remove noise, then pick the max, and them break the line on l. for each new line segment, repeat the process. The threshold will determine how many lines you end with. We want 2 or 3 lines. So adjust the threshold to meet this requirement. Algorithm Deriving/computing the reflection vectors (or you can use the subsection title) You can give an overview (properties, pros and cons of this algorithm), them move to the specifics of the algorithm. So, split into 2 slides Change the title to Strip, in the text stripes Fonts not consistent Iv should be break the points at point mas d(si) Or, you should swap iV and v. You smooth to remove noise, then pick the max, and them break the line on l. for each new line segment, repeat the process. The threshold will determine how many lines you end with. We want 2 or 3 lines. So adjust the threshold to meet this requirement.

28. The Algorithm (Stage One Result) Assigning pixels to different zones, corresponding to the vectors Assigning pixels to different zones, corresponding to the vectors

29. The Algorithm(Stage Two) Classify pixels into three classes: Matte Highlight Clipped Refit each class of pixels to obtain the respective vectors using Principal Component Analysis Obtain Cb and Cs Parallel to matte and highlight lines respectively Starting from the origin of the Dichromatic Plane Title Use vector instead of lineTitle Use vector instead of line

30. Generating Intrinsic Reflection Images C(x, y) = msCs + mbCb A new coordinate system Cs , Cb , Cs* Cs c = [R, G, B] d = Tc where T affine transformation matrix d = [mb, ms, ?]T where ? noise ms component of all transformed pixels -> surface reflection image ms component of all transformed pixels -> body reflection image People do not use * for cross product, the right one is either x or ? T is the aft matrix Type ms and mbPeople do not use * for cross product, the right one is either x or ? T is the aft matrix Type ms and mb

31. Results Results

32. Matte and highlight lines of the three plastic cups under yellow light Fitted Lines: Plastic Cups under Yellow Light matte line highlight line clipped line green cup from ( 5. 1. 1) from ( 24. 54. 9) ---- to ( 23. 56. 9) to (177,192. 52) yellow cup from ( 2. 0. 2) from (185.108. 21) ---- to (189.106. 19) to (255.175. 41) orange cup from ( 1. 0. 2) from (155. 41. 17) from (220.101. 34) to (159. 36. 15) to (220.101. 34) to (238.191. 54)

35. Restoring the Colors of Clipped and Bloomed Pixels Occurs only in one or two color bands Replaced with pixels on the matte or highlight line that have the same value in the undistorted band Visible intensity changes Shading information lost Since all clipped pixels and all bloomed highlight pixels are restored to lie exactly on the highlight line. They are all projected into the same body reflection component. We thus lost all shading inform. Since all clipped pixels and all bloomed highlight pixels are restored to lie exactly on the highlight line. They are all projected into the same body reflection component. We thus lost all shading inform.

37. The surface reflection images locate very well the highlights in the images. Note the gradual change of the amount of surface reflection on the paper folders. Since surface relfection images captures this gradual change, it may provide the means for a quantitative approach to determine object shape from hightlightsThe surface reflection images locate very well the highlights in the images. Note the gradual change of the amount of surface reflection on the paper folders. Since surface relfection images captures this gradual change, it may provide the means for a quantitative approach to determine object shape from hightlights

39. Body and surface reflection vectors of the three plastic cups under white light

40. Discussions Lack a quantitative evaluation methodology A possible means to determine object shape from highlight ?? Investigate why larger variations in surface reflection vector under white light Fuzzy clipping threshold Imprecise classifications of matte and highlight pixels