Presentation Transcript

1. Photometric Image Formation

   CSE 559: Computer Vision Guest Lecturer: Austin Abrams Images/Demo from Steve Seitz, Wikipedia
2. How are images made? One half: geometric vision “how the pixel projected onto the image” Today: photometric vision (aka radiometric) “how the pixel got its color”
3. Vision and Graphics Computer Graphics Properties of a scene Image Vision
4. Image Formation Approach Come up with a model for how the scene was created Given images, find the most likely properties that fit that model
5. Diffuse Surfaces Brightness of a pixel depends on: object color lighting direction surface normal But NOT view direction!
6. Lambertian Cosine Law The intensity of an observed diffuse object is proportional to the cosine of the angle between the normal and lighting direction I = ρcosθ L θ N = ρ |L||N| cosθ = ρLN
7. =  L  N = L N
8. = x I = ρ L N
9. Recovering Albedo and Normals Can you decompose a single image into its albedo and normal images?
10. x = x x
11. Photometric Stereo Given multiple images taken with varying illumination, recover albedo and normals. take pictures in dark room with varying illumination. estimate lighting directions L. recover albedo and normals.
12. Side note 1: How to get the lighting direction? Put a shiny sphere in the scene Sphere’s geometry (normals) are known Find specular highlight
13. Side-note 2: Why “Stereo”? Surface normals provide constraints on depth differences
14. Photometric Stereo If L is known, and albedo is grayscale this is a linear problem. I = ρ(L N) = ρ (Lx Nx + Ly Ny+ LzNz) = Lx Nxρ + Ly Nyρ+ LzNzρ = Lxa + Lyb+ Lzc
15. I = ρ(L  N) = Lxa+ Lyb+ Lzc For each pixel: Lx1 Ly1 Lz1 Lx2 Ly2 Lz2 Lx3 Ly3 Lz3 … LxnLyn Lzn I1 I2 I3 … In a b c = Then: ρ = sqrt(a2+ b2+ c2) N = (a,b,c) / ρ
16. Demo
17. When does this model fail? I ≠ ρ (L  N)
18. Attached shadows L  N = 0 L  N > 0 L  N < 0 I = ρmax(L N, 0)
19. Cast Shadows, Ambient Light I = ρ (S L  N + a) S = 0 or 1
20. Radiometric Camera Calibration Pixel intensities are usually not proportional to the energy that hit the CCD RAW image Published image
21. Radiometric Camera Calibration Published f RAW
22. Radiometric Camera Calibration Observed = f(RAW) (Grossberg and Nayar) f-1 (Observed) = RAW
23. Radiometric Camera Calibration How do you model f-1? f-1(x) =xγ f-1(x) = c0 + c1x + c2x2 + c3x3 + … f-1(x) = f0(x) + f1(x) c1 + f2(x)c2+ … mean camera curve basis camera curves
24. Radiometric Camera Calibration I = f(ρ (S L  N + a)) Adding exposure: I = f(eρ (S L  N + a))
25. Heliometric Stereo Given lots of images from a stable webcam, use lighting from the sun to recover: I = f(eρ (S L N + a))
26. Heliometric Stereo
27. Heliometric Stereo
28. Heliometric Stereo