Formation et Analyse d’Images Session 2

1. Formation et Analyse d’ImagesSession 2 Daniela Hall 7 October 2004

2. Course Overview • Session 1: • Homogenous coordinates and tensor notation • Image transformations • Camera models • Session 2: • Camera models • Reflection models • Color spaces • Session 3: • Review color spaces • Pixel based image analysis • Gaussian filter operators • Session 4: • Scale Space

3. Course overview • Session 5: • Contrast description • Hough transform • Session 6: • Kalman filter • Tracking of regions, pixels, and lines • Session 7: • Stereo vision • Epipolar geometry • Session 8: exam

4. Session Overview • Camera model • Light • Reflection models • Color spaces

5. Camera model • Projective model Scene coordinates Camera coordinates Image coordinates

6. Camera model • Transformation from scene to camera coordinates • Projection of camera coordinates to retina coordinates • Transformation from retina coordinates to image coordinates • Composition (camera model) The camera model is the composition of the transformations that transform Ps to Pi

7. Transformation Scene - Camera • (xs,ys,zs) is position of the origin of the camera system with respect to the scene coordinates (translation). • R is the orientation of the camera system with respect to the scene system (3d rotation).

8. 3d rotation • Around x-axis (counter-clockwise) • Around y-axis • Around z-axis • General

9. (xc ,zc ) xr z F O x Projection Camera-Retina • Imagine a 1D camera in a 2D space. • The transformation MRc can be found by considering similar triangles

10. Projection Camera-Retina

11. (0,0) i columns (i-1,j-1) j rows Transformation Retina-Image • A frame: the image is composed of pixels (picture elements) • Pixels are in general not squared. There physical sizes depends on the used material.

12. Intrinsic camera parameters • F: focal distance • Ci , Cj : Optical image center (in pixels) • Di , Dj : Physical size of the pixel on the retina (in pixel/mm) • i, j : image coordinates (in pixels) • Transformation Retina-Image

13. Camera model Equation: Image coordinates

14. Transformation image-scene • Problem: we need to know depth zs for each image position. • This process of finding MSI is called calibration. • MSI has 12 coefficients. • MSI is homogenous. 11 degrees of freedom.

15. Calibration • Construct a calibration object whose 3D position is known. • Measure image coordinates • Determine correspondences between 3D point RSk and image point PIk. • We have 11 DoF. We need at least 5 ½ correspondences.

16. Calibration • For each correspondence scene point RSk and image point PIk • which gives following equations for k=1, ..., 6 • from wich MIS can be computed

17. Calibration using many points • For k=5 ½ M has one solution. • Solution depends on precise measurements of 3D and 2D points. • If you use another 5 ½ points you will get a different solution. • A more stable solution is found by using large number of points and do optimisation.

18. Calibration using many points • For each point correspondence we know (i,j) and R. • We want to know MIS. Solve equation with your favorite algorithm (least squares, levenberg-marquart, svd,...)

19. Application: Rectifying images

20. Applications

21. Homographie: projection from one plane to another • Homographie HBA is bijective QB = HBA PA

22. Homography computation • H can be computed from 4 point correspondences. Rd1 Rd2 Ps1 Ps2 Ps3 Ps4 Rd3 Rd4 Destination image (rectified) Source image (observed)

23. Homography computation • H is 3x3 matrix and has 8 degrees of freedom (homogenous coordinates) • gives 8 equations and one solution for H.

24. Session Overview • Camera model • Light • Reflection models • Color spaces

25. light N e i camera g Light • N: surface normal • i angle between incoming light and normal • e angle between normal and camera • g angle between light and camera

26. Spectrum • Light source is characterised by its spectrum. • The spectrum consists of a particular quantity of photons per frequency. • The frequency is described by its wavelength • The visible spectrum is 380nm to 720nm • Cameras can see a larger spectrum depending on their CCD chip

27. light N e i camera g Albedo • Albedo is the fraction of light that is reflected by a body or surface. • Reflectance function:

28. Session Overview • Camera model • Light • Reflection models • Color spaces

29. Reflectance functions • Specular reflection • example mirror • Lambertian reflection • diffuse reflection, example paper, snow

30. Specular reflection light N e camera i g

31. Lambertian reflection

32. Di-chromatic reflectance model • the reflected light R is the sum of the light reflected at the surface Rs and the light reflected from the material body RL • Rs has the same spectrum as the light source • The spectrum of Rl is « filtered » by the material (photons are absorbed, this changes the emitted light) • Luminance depends on surface orientation • Spectrum of chrominance is composed of light source spectrum and absorption of surface material.

33. Color perception • The retina is composed of rods and cones. • Rods - provide "scotopic" or low intensity vision. • Provide our night vision ability for very low illumination, • Are a thousand times more sensitive to light than cones, • Are much slower to respond to light than cones, • Are distributed primarily in the periphery of the visual field.

34. Color perception • Cones - provide "photopic" or high acuity vision. • Provide our day vision, • Produce high resolution images, • Determine overall brightness or darkness of images, • Provide our color vision, by means of three types of cones: • "L" or red, long wavelength sensitive, • "M" or green, medium wavelength sensitive, • "S" or blue, short wavelength sensitive. • Cones enable our day vision and color vision. Rods take over in low illumination. However, rods cannot detect color which is why at night we see in shades of gray. • source: http://www.hf.faa.gov/Webtraining/VisualDisplays/

35. Color perception • Rod Sensitivity- Peak at 498 nm. • Cone Sensitivity- Red or "L" cones peak at 564 nm. - Green or "M" cones peak at 533 nm.  - Blue or "S" cones peak at 437 nm. • This diagram shows the wavelength sensitivities of the different cones and the rods. Note the overlap in sensitivity between the green and red cones.

36. S(λ) CCD vidicon λ 400 600 800 1000 nm Camera sensitivity • observed light intensity depends on: • source spectrum: S(λ) • reflectance of the observed point (i,j): P(i,j,λ) • receptive spectrum of the camera: c(λ) • p0 is the gain

37. Classical RGB camera • The filters follow a convention of the International Illumination Commission. • They are functions of λ: r(λ), g(λ), b(λ) • They are close to the sensitivity of the human color vision system.

38. Color pixels

39. Color bands (channels) • It is not possible to perceive the spectrum directly. • Color is a projection of the spectrum to the spectrum of the sensors.

40. Session Overview • Camera model • Light • Reflection models • Color spaces

41. Color spaces • RGB color space • CMY color space • YIQ color space • HLS color space

42. RGB color space • A CCD camera provides RGB images • The luminance axis is r=g=b (diagonal)

43. CMY color space • Cyan, magenta, yellow • CMYK: CMY + black color channel

44. YIQ color space • This is an approximation of • Y: luminance, • I: red – cyan, • Q: magenta - green • Used US TVs (NTSC coding). Black and white TVs display only Y channel.

45. HLS space • Hue, luminance, saturation space. • L=R+G+B • S=1-3*min(R,G,B)/L L T S

46. Color distribution • Color distribution can be studied by histograms. • A histogram is a multi-dimensional table. • We define a function from the continuous color space to the discrete histogram space. • Then each pixel of the image increments a cell in the histogram. • Example: We define a histogram of RGB (3D) with 32 cells/dimension. The pixel value (210,180,100) increments cell (6,5,3)

47. Colors of a surface • A reflection has the color of the light source (Rs) • Which color is near the border of the reflection? • Rs and Rb are mixed. • A color histogram can be used to study this mix. • The histogram should contain two axis (in theory). • But reflectance in the real world is more complex than only Rs and Rb. You also have inter reflectance between neighboring objects.