1 / 44

Exploring Computational Photography: From Human Eyes to Digital Cameras

This presentation, delivered by Alexei Efros at CMU in Fall 2005, delves into the fascinating field of computational photography. It covers the comparison between human vision and digital camera technology, including the functioning of sensors like CCD and CMOS, the role of photoreceptors in the human eye, and the complexities of light perception. The lecture highlights concepts like dynamic range, image formation, and color sensing in cameras, drawing connections to how we perceive and interpret light in our environment.

lang
Download Presentation

Exploring Computational Photography: From Human Eyes to Digital Cameras

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Capturing Light… in man and machine 15-463: Computational Photography Alexei Efros, CMU, Fall 2005 Some figures from Steve Seitz, Steve Palmer, Paul Debevec, and Gonzalez et al.

  2. Image Formation Digital Camera Film The Eye

  3. Digital camera • A digital camera replaces film with a sensor array • Each cell in the array is light-sensitive diode that converts photons to electrons • Two common types • Charge Coupled Device (CCD) • CMOS • http://electronics.howstuffworks.com/digital-camera.htm

  4. Sensor Array CMOS sensor

  5. Sampling and Quantization

  6. Interlace vs. progressive scan http://www.axis.com/products/video/camera/progressive_scan.htm

  7. Progressive scan http://www.axis.com/products/video/camera/progressive_scan.htm

  8. Interlace http://www.axis.com/products/video/camera/progressive_scan.htm

  9. The Eye • The human eye is a camera! • Iris - colored annulus with radial muscles • Pupil - the hole (aperture) whose size is controlled by the iris • What’s the “film”? • photoreceptor cells (rods and cones) in the retina

  10. The Retina

  11. Light Retina up-close

  12. Two types of light-sensitive receptors Cones cone-shaped less sensitive operate in high light color vision Rods rod-shaped highly sensitive operate at night gray-scale vision © Stephen E. Palmer, 2002

  13. Rod / Cone sensitivity The famous sock-matching problem…

  14. Distribution of Rods and Cones Night Sky: why are there more stars off-center? © Stephen E. Palmer, 2002

  15. Electromagnetic Spectrum Human Luminance Sensitivity Function http://www.yorku.ca/eye/photopik.htm

  16. Visible Light Why do we see light of these wavelengths? …because that’s where the Sun radiates EM energy © Stephen E. Palmer, 2002

  17. The Physics of Light Any patch of light can be completely described physically by its spectrum: the number of photons (per time unit) at each wavelength 400 - 700 nm. © Stephen E. Palmer, 2002

  18. The Physics of Light Some examples of the spectra of light sources © Stephen E. Palmer, 2002

  19. Yellow Blue Purple Red 400 700 400 700 400 700 400 700 The Physics of Light Some examples of the reflectance spectra of surfaces % Photons Reflected Wavelength (nm) © Stephen E. Palmer, 2002

  20. mean area variance The Psychophysical Correspondence There is no simple functional description for the perceived color of all lights under all viewing conditions, but …... A helpful constraint: Consider only physical spectra with normal distributions © Stephen E. Palmer, 2002

  21. # Photons Wavelength The Psychophysical Correspondence Mean Hue © Stephen E. Palmer, 2002

  22. # Photons Wavelength The Psychophysical Correspondence Variance Saturation © Stephen E. Palmer, 2002

  23. # Photons Wavelength The Psychophysical Correspondence Area Brightness © Stephen E. Palmer, 2002

  24. Physiology of Color Vision Three kinds of cones: • Why are M and L cones so close? • Are are there 3? © Stephen E. Palmer, 2002

  25. More Spectra metamers

  26. Color Sensing in Camera (RGB) • 3-chip vs. 1-chip: quality vs. cost • Why more green? Why 3 colors? http://www.cooldictionary.com/words/Bayer-filter.wikipedia

  27. Practical Color Sensing: Bayer Grid • Estimate RGBat ‘G’ cels from neighboring values http://www.cooldictionary.com/words/Bayer-filter.wikipedia

  28. RGB color space • RGB cube • Easy for devices • But not perceptual • Where do the grays live? • Where is hue and saturation?

  29. HSV • Hue, Saturation, Value (Intensity) • RGB cube on its vertex • Decouples the three components (a bit) • Use rgb2hsv() and hsv2rgb() in Matlab

  30. White Balance White World / Gray World assumptions

  31. Programming Assignment #1 • How to compare R,G,B channels? • No right answer • Sum of Squared Differences (SSD): • Normalized Correlation (NCC):

  32. Image Pyramids (preview) • Known as a Gaussian Pyramid [Burt and Adelson, 1983] • In computer graphics, a mip map [Williams, 1983] • A precursor to wavelet transform

  33. Image Formation f(x,y) = reflectance(x,y) * illumination(x,y) Reflectance in [0,1], illumination in [0,inf]

  34. Problem: Dynamic Range The real world is High dynamic range 1 1500 25,000 400,000 2,000,000,000

  35. Is Camera a photometer? Image pixel (312,284) = 42 42 photos?

  36. Long Exposure 10-6 106 High dynamic range Real world 10-6 106 Picture 0 to 255

  37. Short Exposure 10-6 106 High dynamic range Real world 10-6 106 Picture 0 to 255

  38. ò Image Acquisition Pipeline Lens Shutter scene radiance (W/sr/m ) sensor irradiance sensor exposure 2 Dt CCD ADC Remapping analog voltages digital values pixel values Camera is NOT a photometer!

  39. Varying Exposure

  40. What does the eye sees? The eye has a huge dynamic range Do we see a true radiance map?

  41. Eye is not a photometer! • "Every light is a shade, compared to the higher lights, till you come to the sun; and every shade is a light, compared to the deeper shades, till you come to the night." • — John Ruskin, 1879

  42. Cornsweet Illusion

  43. Sine wave Campbell-Robson contrast sensitivity curve

  44. Metamers Eye is sensitive to changes (more on this later…)

More Related