1 / 71

Pixels and Image Filtering

08/30/19. Pixels and Image Filtering. Computational Photography Derek Hoiem. Graphic: http://www.notcot.org/post/4068/. Administrative stuff. Any questions? Next week reminders Labor day on Monday Derek is out Wed. Jae will teach. Recording from earlier year available.

hperkins
Download Presentation

Pixels and Image Filtering

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. 08/30/19 Pixels and Image Filtering Computational Photography Derek Hoiem Graphic: http://www.notcot.org/post/4068/

  2. Administrative stuff • Any questions? • Next week reminders • Labor day on Monday • Derek is out Wed. Jae will teach. Recording from earlier year available. • Yuan will give tutorial Thurs at 5pm in DCL 1320

  3. Pixels and Image Filtering Computational Photography Derek Hoiem Graphic: http://www.notcot.org/post/4068/

  4. Today’s Class: Pixels and Linear Filters • What is a pixel? How is an image represented? • What is image filtering and how do we do it? • Introduce Project 1: Hybrid Images

  5. Next three classes • Image filters in spatial domain • Smoothing, sharpening, measuring texture • Image filters in the frequency domain • Denoising, sampling, image compression • Templates and Image Pyramids • Detection, coarse-to-fine registration

  6. Image Formation

  7. Digital camera Digital camera replaces film with a sensor array • Each cell in the array is light-sensitive diode that converts photons to electrons • http://electronics.howstuffworks.com/digital-camera.htm

  8. Sensor Array CCD sensor

  9. The raster image (pixel matrix) Photo by Phil Greenspun used with permission

  10. The raster image (pixel matrix)

  11. Perception of Intensity from Ted Adelson

  12. Perception of Intensity from Ted Adelson

  13. Digital Color Images https://commons.wikimedia.org/wiki/File:BayerPatternFiltration.png

  14. Color Image R G B

  15. Images in Python im = cv2.imread(filename) # read image im = cv2.cvtColor(im, cv2.COLOR_BGR2RGB) # order channels as RGB im = im / 255 # values range from 0 to 1 • RGB image imis a H x W x 3 matrix (numpy.ndarray) • im[0,0,0]= top-left pixel value in R-channel • im[y, x, c] = y+1 pixels down, x+1 pixels to right in the cth channel • im[H-1, W-1, 2] = bottom-right pixel in B-channel column row R G B

  16. Image filtering • Image filtering: compute function of local neighborhood at each position • Really important! • Enhance images • Denoise, resize, increase contrast, etc. • Extract information from images • Texture, edges, distinctive points, etc. • Detect patterns • Template matching

  17. Example: box filter 1 1 1 1 1 1 1 1 1 Slide credit: David Lowe (UBC)

  18. 1 1 1 1 1 1 1 1 1 Image filtering Credit: S. Seitz

  19. 1 1 1 1 1 1 1 1 1 Image filtering Credit: S. Seitz

  20. Image filtering 1 1 1 1 1 1 1 1 1 Credit: S. Seitz

  21. Image filtering 1 1 1 1 1 1 1 1 1 Credit: S. Seitz

  22. Image filtering 1 1 1 1 1 1 1 1 1 Credit: S. Seitz

  23. Image filtering 1 1 1 1 1 1 1 1 1 ? Credit: S. Seitz

  24. Image filtering 1 1 1 1 1 1 1 1 1 ? Credit: S. Seitz

  25. Image filtering 1 1 1 1 1 1 1 1 1 Credit: S. Seitz

  26. Image filtering 1 1 1 1 1 1 1 1 1 Informally, what does the filter do? Credit: S. Seitz

  27. 1 1 1 1 1 1 1 1 1 Box Filter • What does it do? • Replaces each pixel with an average of its neighborhood • Achieve smoothing effect (remove sharp features) Slide credit: David Lowe (UBC)

  28. Smoothing with box filter

  29. One more by hand… = *

  30. Practice with linear filters 0 0 0 0 1 0 0 0 0 ? Original Source: D. Lowe

  31. Practice with linear filters 0 0 0 0 1 0 0 0 0 Original Filtered (no change) Source: D. Lowe

  32. Practice with linear filters 0 0 0 0 0 1 0 0 0 ? Original Source: D. Lowe

  33. Practice with linear filters 0 0 0 0 0 1 0 0 0 Original Shifted left By 1 pixel Source: D. Lowe

  34. Practice with linear filters 0 1 0 1 0 1 1 0 2 1 0 1 1 0 1 0 0 1 - ? (Note that filter sums to 1) Original Source: D. Lowe

  35. Practice with linear filters 0 1 0 1 0 1 1 0 2 1 0 1 1 0 1 0 0 1 - Original • Sharpening filter • Accentuates differences with local average Source: D. Lowe

  36. Sharpening Source: D. Lowe

  37. Other filters 1 0 -1 2 0 -2 1 0 -1 Sobel Vertical Edge (absolute value)

  38. Other filters Q? 1 2 1 0 0 0 -1 -2 -1 Sobel Horizontal Edge (absolute value)

  39. How could we synthesize motion blur? theta = 30 len = 21 mid = (len-1)/2 fil = np.zeros((len,len)) fil[:,int(mid)] = 1/len R = cv2.getRotationMatrix2D((mid,mid),theta,1) fil = cv2.warpAffine(fil,R,(len,len)) im_fil = cv2.filter2D(im, -1, fil)

  40. Correlation vs. Convolution • 2d correlation im_fil = cv2.filter2d(im, -1, fil) • 2d convolution im_fil = scipy.signal.convolve2d(im, fil, [opts]) • “convolve” mirrors the kernel, while “filter” doesn’t cv2.filter2D(im, -1, cv2.flip(fil,-1)) same as signal.convolve2d(im,fil,mode='same‘,boundary=‘symm’)

  41. Key properties of linear filters Linearity: filter(f1 + f2) = filter(f1) + filter(f2) Shift invariance: same behavior regardless of pixel location filter(shift(f)) = shift(filter(f)) Any linear, shift-invariant operator can be represented as a convolution Source: S. Lazebnik

  42. More properties • Commutative: a * b = b * a • Conceptually no difference between filter and signal (image) • Associative: a * (b * c) = (a * b) * c • Often apply several filters one after another: (((a * b1) * b2) * b3) • This is equivalent to applying one filter: a * (b1 * b2 * b3) • Distributes over addition: a * (b + c) = (a * b) + (a * c) • Scalars factor out: ka * b = a * kb = k (a * b) • Identity: unit impulse e = [0, 0, 1, 0, 0],a * e = a Source: S. Lazebnik

  43. Important filter: Gaussian • Weight contributions of neighboring pixels by nearness 0.003 0.013 0.022 0.013 0.003 0.013 0.059 0.097 0.059 0.013 0.022 0.097 0.159 0.097 0.022 0.013 0.059 0.097 0.059 0.013 0.003 0.013 0.022 0.013 0.003 5 x 5,  = 1 Slide credit: C. Rasmussen

  44. Smoothing with Gaussian filter

  45. Smoothing with box filter

  46. Gaussian filters • Remove “high-frequency” components from the image (low-pass filter) • Images become more smooth • Convolution with self is another Gaussian • So can smooth with small-width kernel, repeat, and get same result as larger-width kernel would have • Convolving two times with Gaussian kernel of width σ is same as convolving once with kernel of width • Separable kernel • Factors into product of two 1D Gaussians Source: K. Grauman

  47. Separability of the Gaussian filter Source: D. Lowe

  48. Separability example * = = * 2D filtering(center location only) The filter factorsinto a product of 1Dfilters: Perform filteringalong rows: Followed by filteringalong the remaining column: Source: K. Grauman

  49. Separability • Why is separability useful in practice?

  50. Some practical matters

More Related