- By
**chiko** - Follow User

- 113 Views
- Uploaded on

Download Presentation
## PowerPoint Slideshow about 'CS558 Computer Vision' - chiko

Download Now**An Image/Link below is provided (as is) to download presentation**

Download Now

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

OutlineOutline

Recap of Lecture III

- Light and Shade
- Radiance and irradiance
- Radiometry of thin lens
- Bidirectional reflectance distribution function (BRDF)
- Photometric stereo
- Color
- What is color?
- Human eyes
- Trichromacy and color space
- Color perception

Outline

- Image filter and convolution
- Convluationand linear filter
- Gaussian filter, box filter, and median filter
- Application: Hybrid images
- Edge detection
- Image derivatives
- Gaussian derivative filters
- Canny edge detector

Outline

- Image filter and convolution
- Convolution and linear filter
- Gaussian filter, box filter, and median filter
- Application: Hybrid images
- Edge detection
- Image derivatives
- Gaussian derivative filters
- Canny edge detector

Motivation: Image denoising

- How can we reduce noise in a photograph?

1

1

1

1

1

1

1

1

“box filter”

Moving average- Let’s replace each pixel with a weighted average of its neighborhood
- The weights are called the filter kernel
- What are the weights for the average of a 3x3 neighborhood?

Source: D. Lowe

Outline

- Image filter and convolution
- Convolution and linear filter
- Gaussian filter, box filter, and median filter
- Application: Hybrid images
- Edge detection
- Image derivatives
- Gaussian derivative filters
- Canny edge detector

Defining convolution

- Let f be the image and g be the kernel. The output of convolving f with g is denoted f *g.

- MATLAB functions: conv2, filter2, imfilter

Source: F. Durand

Key properties

- Linearity:filter(f1 + f2) = filter(f1) + filter(f2)
- Shift invariance: same behavior regardless of pixel location: filter(shift(f)) = shift(filter(f))
- Theoretical result: any linear shift-invariant operator can be represented as a convolution

Properties in more detail

- Commutative: a * b = b * a
- Conceptually no difference between filter and signal
- Associative: a * (b * c) = (a * b) * c
- Often apply several filters one after another: (((a * b1) * b2) * b3)
- This is equivalent to applying one filter: a * (b1 * b2 * b3)
- Distributes over addition: a * (b + c) = (a * b) + (a * c)
- Scalars factor out: ka * b = a * kb = k (a * b)
- Identity: unit impulse e = […, 0, 0, 1, 0, 0, …],a * e = a

Annoying details

- What is the size of the output?
- MATLAB: filter2(g, f, shape)
- shape = ‘full’: output size is sum of sizes of f and g
- shape = ‘same’: output size is same as f
- shape = ‘valid’: output size is difference of sizes of f and g

full

same

valid

g

g

g

g

f

f

f

g

g

g

g

g

g

g

g

Annoying details

- What about near the edge?
- the filter window falls off the edge of the image
- need to extrapolate
- methods:
- clip filter (black)
- wrap around
- copy edge
- reflect across edge

Source: S. Marschner

Annoying details

- What about near the edge?
- the filter window falls off the edge of the image
- need to extrapolate
- methods (MATLAB):
- clip filter (black): imfilter(f, g, 0)
- wrap around: imfilter(f, g, ‘circular’)
- copy edge: imfilter(f, g, ‘replicate’)
- reflect across edge: imfilter(f, g, ‘symmetric’)

Source: S. Marschner

1

0

1

1

0

1

0

1

2

1

0

1

0

1

0

1

0

Practice with linear filters-

Original

- Sharpening filter
- Accentuates differences with local average

Source: D. Lowe

Sharpening

Source: D. Lowe

detail

smoothed (5x5)

original

Let’s add it back:

+

=

original

detail

sharpened

Sharpening- What does blurring take away?

–

Smoothing with box filter revisited

- What’s wrong with this picture?
- What’s the solution?

Source: D. Forsyth

Smoothing with box filter revisited

- What’s wrong with this picture?
- What’s the solution?
- To eliminate edge effects, weight contribution of neighborhood pixels according to their closeness to the center

“fuzzy blob”

Outline

- Image filter and convolution
- Convluationand linear filter
- Gaussian filter, box filter, and median filter
- Application: Hybrid images
- Edge detection
- Image derivatives
- Gaussian derivative filters
- Canny edge detector

Gaussian Kernel

- Constant factor at front makes volume sum to 1 (can be ignored when computing the filter values, as we should renormalize weights to sum to 1 in any case)

0.003 0.013 0.022 0.013 0.003

0.013 0.059 0.097 0.059 0.013

0.022 0.097 0.159 0.097 0.022

0.013 0.059 0.097 0.059 0.013

0.003 0.013 0.022 0.013 0.003

5 x 5, = 1

Source: C. Rasmussen

Gaussian Kernel

- Standard deviation : determines extent of smoothing

σ = 2 with 30 x 30 kernel

σ = 5 with 30 x 30 kernel

Source: K. Grauman

Choosing kernel width

- The Gaussian function has infinite support, but discrete filters use finite kernels

Source: K. Grauman

Choosing kernel width

- Rule of thumb: set filter half-width to about 3σ

Gaussian filters

- Remove “high-frequency” components from the image (low-pass filter)
- Convolution with self is another Gaussian
- So can smooth with small- kernel, repeat, and get same result as larger- kernel would have
- Convolving two times with Gaussian kernel with std. dev. σis same as convolving once with kernel with std. dev.
- Separable kernel
- Factors into product of two 1D Gaussians

Source: K. Grauman

Separability of the Gaussian filter

Source: D. Lowe

=

=

*

Separability example2D convolution(center location only)

The filter factorsinto a product of 1Dfilters:

Perform convolutionalong rows:

Followed by convolutionalong the remaining column:

Source: K. Grauman

Why is separability useful?

- What is the complexity of filtering an n×n image with an m×mkernel?
- O(n2 m2)
- What if the kernel is separable?
- O(n2 m)

Noise

- Salt and pepper noise: contains random occurrences of black and white pixels
- Impulse noise: contains random occurrences of white pixels
- Gaussian noise: variations in intensity drawn from a Gaussian normal distribution

Source: S. Seitz

Gaussian noise

- Mathematical model: sum of many independent factors
- Good for small standard deviations
- Assumption: independent, zero-mean noise

Source: M. Hebert

Reducing Gaussian noise

Smoothing with larger standard deviations suppresses noise, but also blurs the image

Alternative idea: Median filtering

- A median filter operates over a window by selecting the median intensity in the window

- Is median filtering linear?

Source: K. Grauman

Median filter

- What advantage does median filtering have over Gaussian filtering?
- Robustness to outliers

Source: K. Grauman

Sharpening revisited

Source: D. Lowe

detail

smoothed (5x5)

original

Let’s add it back:

+ α

=

original

detail

sharpened

Sharpening revisited- What does blurring take away?

–

Outline

- Image filter and convolution
- Convolution and linear filter
- Gaussian filter, box filter, and median filter
- Application: Hybrid images
- Edge detection
- Image derivatives
- Gaussian derivative filters
- Canny edge detector

Application: Hybrid Images

- A. Oliva, A. Torralba, P.G. Schyns, “Hybrid Images,” SIGGRAPH 2006

Application: Hybrid Images

Gaussian Filter

- A. Oliva, A. Torralba, P.G. Schyns, “Hybrid Images,” SIGGRAPH 2006

Laplacian Filter

Outline

- Image filter and convolution
- Convoluation and linear filter
- Gaussian filter, box filter, and median filter
- Application: Hybrid images
- Edge detection
- Image derivatives
- Gaussian derivative filters
- Canny edge detector

Edge detection

- Goal: Identify sudden changes (discontinuities) in an image
- Intuitively, most semantic and shape information from the image can be encoded in the edges
- More compact than pixels
- Ideal: artist’s line drawing (but artist is also using object-level knowledge)

Source: D. Lowe

Origin of edges

- Edges are caused by a variety of factors:

surface normal discontinuity

depth discontinuity

surface color discontinuity

illumination discontinuity

Source: Steve Seitz

intensity function(along horizontal scanline)

first derivative

edges correspond toextrema of derivative

Characterizing edges- An edge is a place of rapid change in the image intensity function

image

Outline

- Image filter and convolution
- Convoluation and linear filter
- Gaussian filter, box filter, and median filter
- Application: Hybrid images
- Edge detection
- Image derivatives
- Gaussian derivative filters
- Canny edge detector

Derivatives with convolution

For 2D function f(x,y), the partial derivative is:

For discrete data, we can approximate using finite differences:

To implement above as convolution, what would be the associated filter?

Source: K. Grauman

Image gradient

- The gradient of an image:

The gradient points in the direction of most rapid increase in intensity

- How does this direction relate to the direction of the edge?

The gradient direction is given by

The edge strength is given by the gradient magnitude

Source: Steve Seitz

- Image filter and convolution
- Convolution and linear filter
- Gaussian filter, box filter, and median filter
- Application: Hybrid images
- Edge detection
- Image derivatives
- Gaussian derivative filters
- Canny edge detector

Effects of noise

- Consider a single row or column of the image
- Plotting intensity as a function of position gives a signal

Source: S. Seitz

Derivative theorem of convolution

- Differentiation is convolution, and convolution is associative:
- This saves us one operation:

Source: S. Seitz

Scale of Gaussian derivative filter

- Smoothed derivative removes noise, but blurs edge. Also finds edges at different “scales”

1 pixel

3 pixels

7 pixels

Source: D. Forsyth

Review: Smoothing vs. derivative filters

- Smoothing filters
- Gaussian: remove “high-frequency” components; “low-pass” filter
- Can the values of a smoothing filter be negative?
- What should the values sum to?
- One: constant regions are not affected by the filter
- Derivative filters
- Derivatives of Gaussian
- Can the values of a derivative filter be negative?
- What should the values sum to?
- Zero: no response in constant regions
- High absolute value at points of high contrast

- Image filter and convolution
- Convolution and linear filter
- Gaussian filter, box filter, and median filter
- Application: Hybrid images
- Edge detection
- Image derivatives
- Gaussian derivative filters
- Canny edge detector

The Canny edge detector

norm of the gradient

The Canny edge detector

thresholding

Non-maximum suppression

Check if pixel is local maximum along gradient direction, select single max across width of the edge

requires checking interpolated pixels p and r

The Canny edge detector

Problem: pixels along this edge didn’t survive the thresholding

thinning

(non-maximum suppression)

Hysteresis thresholding

- Use a high threshold to start edge curves, and a low threshold to continue them.

Source: Steve Seitz

(strong edges)

low threshold

(weak edges)

hysteresis threshold

Hysteresis thresholdingoriginal image

Source: L. Fei-Fei

Recap: Canny edge detector

- Filter image with derivative of Gaussian
- Find magnitude and orientation of gradient
- Non-maximum suppression:
- Thin wide “ridges” down to single pixel width
- Linking and thresholding(hysteresis):
- Define two thresholds: low and high
- Use the high threshold to start edge curves and the low threshold to continue them
- MATLAB: edge(image, ‘canny’);

J. Canny, A Computational Approach To Edge Detection, IEEE Trans. Pattern Analysis and Machine Intelligence, 8:679-714, 1986.

Edge detection is just the beginning…

human segmentation

image

gradient magnitude

- Berkeley segmentation database:http://www.eecs.berkeley.edu/Research/Projects/CS/vision/grouping/segbench/

Download Presentation

Connecting to Server..