cs558 computer vision n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
CS558 Computer Vision PowerPoint Presentation
Download Presentation
CS558 Computer Vision

Loading in 2 Seconds...

play fullscreen
1 / 79

CS558 Computer Vision - PowerPoint PPT Presentation


  • 118 Views
  • Uploaded on

CS558 Computer Vision. Lecture IV : Image Filter and Edge Detection. Slides adapted from S. Lazebnik. Recap of Lecture III. Light and Shade Radiance and irradiance Radiometry of thin lens Bidirectional reflectance distribution function (BRDF) Photometric stereo Color What is color?

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'CS558 Computer Vision' - chiko


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
cs558 computer vision

CS558 Computer Vision

Lecture IV: Image Filter and Edge Detection

Slides adapted from S. Lazebnik

recap of lecture iii
Recap of Lecture III
  • Light and Shade
    • Radiance and irradiance
    • Radiometry of thin lens
    • Bidirectional reflectance distribution function (BRDF)
    • Photometric stereo
  • Color
    • What is color?
    • Human eyes
    • Trichromacy and color space
    • Color perception
outline
Outline
  • Image filter and convolution
    • Convluationand linear filter
    • Gaussian filter, box filter, and median filter
    • Application: Hybrid images
  • Edge detection
    • Image derivatives
    • Gaussian derivative filters
    • Canny edge detector
outline1
Outline
  • Image filter and convolution
    • Convolution and linear filter
    • Gaussian filter, box filter, and median filter
    • Application: Hybrid images
  • Edge detection
    • Image derivatives
    • Gaussian derivative filters
    • Canny edge detector
motivation image denoising
Motivation: Image denoising
  • How can we reduce noise in a photograph?
moving average

1

1

1

1

1

1

1

1

1

“box filter”

Moving average
  • Let’s replace each pixel with a weighted average of its neighborhood
  • The weights are called the filter kernel
  • What are the weights for the average of a 3x3 neighborhood?

Source: D. Lowe

outline2
Outline
  • Image filter and convolution
    • Convolution and linear filter
    • Gaussian filter, box filter, and median filter
    • Application: Hybrid images
  • Edge detection
    • Image derivatives
    • Gaussian derivative filters
    • Canny edge detector
defining convolution

f

Defining convolution
  • Let f be the image and g be the kernel. The output of convolving f with g is denoted f *g.
  • MATLAB functions: conv2, filter2, imfilter

Source: F. Durand

key properties
Key properties
  • Linearity:filter(f1 + f2) = filter(f1) + filter(f2)
  • Shift invariance: same behavior regardless of pixel location: filter(shift(f)) = shift(filter(f))
  • Theoretical result: any linear shift-invariant operator can be represented as a convolution
properties in more detail
Properties in more detail
  • Commutative: a * b = b * a
    • Conceptually no difference between filter and signal
  • Associative: a * (b * c) = (a * b) * c
    • Often apply several filters one after another: (((a * b1) * b2) * b3)
    • This is equivalent to applying one filter: a * (b1 * b2 * b3)
  • Distributes over addition: a * (b + c) = (a * b) + (a * c)
  • Scalars factor out: ka * b = a * kb = k (a * b)
  • Identity: unit impulse e = […, 0, 0, 1, 0, 0, …],a * e = a
annoying details
Annoying details
  • What is the size of the output?
  • MATLAB: filter2(g, f, shape)
    • shape = ‘full’: output size is sum of sizes of f and g
    • shape = ‘same’: output size is same as f
    • shape = ‘valid’: output size is difference of sizes of f and g

full

same

valid

g

g

g

g

f

f

f

g

g

g

g

g

g

g

g

annoying details1
Annoying details
  • What about near the edge?
    • the filter window falls off the edge of the image
    • need to extrapolate
    • methods:
      • clip filter (black)
      • wrap around
      • copy edge
      • reflect across edge

Source: S. Marschner

annoying details2
Annoying details
  • What about near the edge?
    • the filter window falls off the edge of the image
    • need to extrapolate
    • methods (MATLAB):
      • clip filter (black): imfilter(f, g, 0)
      • wrap around: imfilter(f, g, ‘circular’)
      • copy edge: imfilter(f, g, ‘replicate’)
      • reflect across edge: imfilter(f, g, ‘symmetric’)

Source: S. Marschner

practice with linear filters

0

0

0

0

1

0

0

0

0

Practice with linear filters

?

Original

Source: D. Lowe

practice with linear filters1

0

0

0

0

1

0

0

0

0

Practice with linear filters

Original

Filtered

(no change)

Source: D. Lowe

practice with linear filters2

0

0

0

0

0

1

0

0

0

Practice with linear filters

?

Original

Source: D. Lowe

practice with linear filters3

0

0

0

0

0

1

0

0

0

Practice with linear filters

Original

Shifted left

By 1 pixel

Source: D. Lowe

practice with linear filters4

1

1

1

1

1

1

1

1

1

Practice with linear filters

?

Original

Source: D. Lowe

practice with linear filters5

1

1

1

1

1

1

1

1

1

Practice with linear filters

Original

Blur (with a

box filter)

Source: D. Lowe

practice with linear filters6

0

1

0

1

1

0

1

0

1

2

1

0

1

0

1

0

1

0

Practice with linear filters

-

?

(Note that filter sums to 1)

Original

Source: D. Lowe

practice with linear filters7

0

1

0

1

1

0

1

0

1

2

1

0

1

0

1

0

1

0

Practice with linear filters

-

Original

  • Sharpening filter
    • Accentuates differences with local average

Source: D. Lowe

sharpening
Sharpening

Source: D. Lowe

sharpening1

=

detail

smoothed (5x5)

original

Let’s add it back:

+

=

original

detail

sharpened

Sharpening
  • What does blurring take away?

smoothing with box filter revisited
Smoothing with box filter revisited
  • What’s wrong with this picture?
  • What’s the solution?

Source: D. Forsyth

smoothing with box filter revisited1
Smoothing with box filter revisited
  • What’s wrong with this picture?
  • What’s the solution?
    • To eliminate edge effects, weight contribution of neighborhood pixels according to their closeness to the center

“fuzzy blob”

outline3
Outline
  • Image filter and convolution
    • Convluationand linear filter
    • Gaussian filter, box filter, and median filter
    • Application: Hybrid images
  • Edge detection
    • Image derivatives
    • Gaussian derivative filters
    • Canny edge detector
gaussian kernel
Gaussian Kernel
  • Constant factor at front makes volume sum to 1 (can be ignored when computing the filter values, as we should renormalize weights to sum to 1 in any case)

0.003 0.013 0.022 0.013 0.003

0.013 0.059 0.097 0.059 0.013

0.022 0.097 0.159 0.097 0.022

0.013 0.059 0.097 0.059 0.013

0.003 0.013 0.022 0.013 0.003

5 x 5,  = 1

Source: C. Rasmussen

gaussian kernel1
Gaussian Kernel
  • Standard deviation : determines extent of smoothing

σ = 2 with 30 x 30 kernel

σ = 5 with 30 x 30 kernel

Source: K. Grauman

choosing kernel width
Choosing kernel width
  • The Gaussian function has infinite support, but discrete filters use finite kernels

Source: K. Grauman

choosing kernel width1
Choosing kernel width
  • Rule of thumb: set filter half-width to about 3σ
gaussian filters
Gaussian filters
  • Remove “high-frequency” components from the image (low-pass filter)
  • Convolution with self is another Gaussian
    • So can smooth with small- kernel, repeat, and get same result as larger- kernel would have
    • Convolving two times with Gaussian kernel with std. dev. σis same as convolving once with kernel with std. dev.
  • Separable kernel
    • Factors into product of two 1D Gaussians

Source: K. Grauman

separability example

*

=

=

*

Separability example

2D convolution(center location only)

The filter factorsinto a product of 1Dfilters:

Perform convolutionalong rows:

Followed by convolutionalong the remaining column:

Source: K. Grauman

why is separability useful
Why is separability useful?
  • What is the complexity of filtering an n×n image with an m×mkernel?
    • O(n2 m2)
  • What if the kernel is separable?
    • O(n2 m)
noise
Noise
  • Salt and pepper noise: contains random occurrences of black and white pixels
  • Impulse noise: contains random occurrences of white pixels
  • Gaussian noise: variations in intensity drawn from a Gaussian normal distribution

Source: S. Seitz

gaussian noise
Gaussian noise
  • Mathematical model: sum of many independent factors
  • Good for small standard deviations
  • Assumption: independent, zero-mean noise

Source: M. Hebert

reducing gaussian noise
Reducing Gaussian noise

Smoothing with larger standard deviations suppresses noise, but also blurs the image

reducing salt and pepper noise
Reducing salt-and-pepper noise

3x3

5x5

7x7

  • What’s wrong with the results?
alternative idea median filtering
Alternative idea: Median filtering
  • A median filter operates over a window by selecting the median intensity in the window
  • Is median filtering linear?

Source: K. Grauman

median filter
Median filter
  • What advantage does median filtering have over Gaussian filtering?
    • Robustness to outliers

Source: K. Grauman

median filter1
Median filter

Median filtered

Salt-and-pepper noise

  • MATLAB: medfilt2(image, [h w])

Source: M. Hebert

gaussian vs median filtering
Gaussian vs. median filtering

3x3

5x5

7x7

Gaussian

Median

sharpening revisited
Sharpening revisited

Source: D. Lowe

sharpening revisited1

=

detail

smoothed (5x5)

original

Let’s add it back:

+ α

=

original

detail

sharpened

Sharpening revisited
  • What does blurring take away?

unsharp mask filter

unit impulse

Gaussian

Laplacian of Gaussian

Unsharp mask filter

image

unit impulse(identity)

blurredimage

outline4
Outline
  • Image filter and convolution
    • Convolution and linear filter
    • Gaussian filter, box filter, and median filter
    • Application: Hybrid images
  • Edge detection
    • Image derivatives
    • Gaussian derivative filters
    • Canny edge detector
application hybrid images
Application: Hybrid Images
  • A. Oliva, A. Torralba, P.G. Schyns, “Hybrid Images,” SIGGRAPH 2006
application hybrid images1
Application: Hybrid Images

Gaussian Filter

  • A. Oliva, A. Torralba, P.G. Schyns, “Hybrid Images,” SIGGRAPH 2006

Laplacian Filter

outline5
Outline
  • Image filter and convolution
    • Convoluation and linear filter
    • Gaussian filter, box filter, and median filter
    • Application: Hybrid images
  • Edge detection
    • Image derivatives
    • Gaussian derivative filters
    • Canny edge detector
edge detection
Edge detection
  • Goal: Identify sudden changes (discontinuities) in an image
    • Intuitively, most semantic and shape information from the image can be encoded in the edges
    • More compact than pixels
  • Ideal: artist’s line drawing (but artist is also using object-level knowledge)

Source: D. Lowe

origin of edges
Origin of edges
  • Edges are caused by a variety of factors:

surface normal discontinuity

depth discontinuity

surface color discontinuity

illumination discontinuity

Source: Steve Seitz

characterizing edges

intensity function(along horizontal scanline)

first derivative

edges correspond toextrema of derivative

Characterizing edges
  • An edge is a place of rapid change in the image intensity function

image

outline6
Outline
  • Image filter and convolution
    • Convoluation and linear filter
    • Gaussian filter, box filter, and median filter
    • Application: Hybrid images
  • Edge detection
    • Image derivatives
    • Gaussian derivative filters
    • Canny edge detector
derivatives with convolution
Derivatives with convolution

For 2D function f(x,y), the partial derivative is:

For discrete data, we can approximate using finite differences:

To implement above as convolution, what would be the associated filter?

Source: K. Grauman

partial derivatives of an image
Partial derivatives of an image

-1 1

1 -1

-1 1

or

Which shows changes with respect to x?

finite difference filters
Finite difference filters
  • Other approximations of derivative filters exist:

Source: K. Grauman

image gradient
Image gradient
  • The gradient of an image:

The gradient points in the direction of most rapid increase in intensity

  • How does this direction relate to the direction of the edge?

The gradient direction is given by

The edge strength is given by the gradient magnitude

Source: Steve Seitz

outline7
Outline
  • Image filter and convolution
    • Convolution and linear filter
    • Gaussian filter, box filter, and median filter
    • Application: Hybrid images
  • Edge detection
    • Image derivatives
    • Gaussian derivative filters
    • Canny edge detector
effects of noise

Where is the edge?

Effects of noise
  • Consider a single row or column of the image
    • Plotting intensity as a function of position gives a signal

Source: S. Seitz

solution smooth first

g

f * g

  • To find edges, look for peaks in
Solution: smooth first

f

Source: S. Seitz

derivative theorem of convolution

f

Derivative theorem of convolution
  • Differentiation is convolution, and convolution is associative:
  • This saves us one operation:

Source: S. Seitz

derivative of gaussian filter
Derivative of Gaussian filter
  • Are these filters separable?

y-direction

x-direction

derivative of gaussian filter1
Derivative of Gaussian filter
  • Which one finds horizontal/vertical edges?

y-direction

x-direction

scale of gaussian derivative filter
Scale of Gaussian derivative filter
  • Smoothed derivative removes noise, but blurs edge. Also finds edges at different “scales”

1 pixel

3 pixels

7 pixels

Source: D. Forsyth

review smoothing vs derivative filters
Review: Smoothing vs. derivative filters
  • Smoothing filters
    • Gaussian: remove “high-frequency” components; “low-pass” filter
    • Can the values of a smoothing filter be negative?
    • What should the values sum to?
      • One: constant regions are not affected by the filter
  • Derivative filters
    • Derivatives of Gaussian
    • Can the values of a derivative filter be negative?
    • What should the values sum to?
      • Zero: no response in constant regions
    • High absolute value at points of high contrast
outline8
Outline
  • Image filter and convolution
    • Convolution and linear filter
    • Gaussian filter, box filter, and median filter
    • Application: Hybrid images
  • Edge detection
    • Image derivatives
    • Gaussian derivative filters
    • Canny edge detector
the canny edge detector
The Canny edge detector

original image

The story of Lena

Slide credit: Steve Seitz

the canny edge detector1
The Canny edge detector

norm of the gradient

the canny edge detector3
The Canny edge detector

How to turn these thick regions of the gradient into curves?

thresholding

non maximum suppression
Non-maximum suppression

Check if pixel is local maximum along gradient direction, select single max across width of the edge

requires checking interpolated pixels p and r

the canny edge detector4
The Canny edge detector

Problem: pixels along this edge didn’t survive the thresholding

thinning

(non-maximum suppression)

hysteresis thresholding
Hysteresis thresholding
  • Use a high threshold to start edge curves, and a low threshold to continue them.

Source: Steve Seitz

hysteresis thresholding1

high threshold

(strong edges)

low threshold

(weak edges)

hysteresis threshold

Hysteresis thresholding

original image

Source: L. Fei-Fei

recap canny edge detector
Recap: Canny edge detector
  • Filter image with derivative of Gaussian
  • Find magnitude and orientation of gradient
  • Non-maximum suppression:
    • Thin wide “ridges” down to single pixel width
  • Linking and thresholding(hysteresis):
    • Define two thresholds: low and high
    • Use the high threshold to start edge curves and the low threshold to continue them
  • MATLAB: edge(image, ‘canny’);

J. Canny, A Computational Approach To Edge Detection, IEEE Trans. Pattern Analysis and Machine Intelligence, 8:679-714, 1986.

edge detection is just the beginning
Edge detection is just the beginning…

human segmentation

image

gradient magnitude

  • Berkeley segmentation database:http://www.eecs.berkeley.edu/Research/Projects/CS/vision/grouping/segbench/
low level edges vs perceived contours
Low-level edges vs. perceived contours

Shadows

Background

Texture

Kristen Grauman, UT-Austin