csce 641 computer graphics image processing l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
CSCE 641 Computer Graphics: Image Processing PowerPoint Presentation
Download Presentation
CSCE 641 Computer Graphics: Image Processing

Loading in 2 Seconds...

play fullscreen
1 / 111

CSCE 641 Computer Graphics: Image Processing - PowerPoint PPT Presentation


  • 179 Views
  • Uploaded on

CSCE 641 Computer Graphics: Image Processing. Jinxiang Chai. Review. 2D Fourier Transform Nyquist sampling theory - Sampling Rate ≥ 2 * max frequency in the signal Antialiasing - prefiltering Gaussian pyramid - Mipmap. Review. 2D Fourier Transform Nyquist sampling theory

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'CSCE 641 Computer Graphics: Image Processing' - christmas


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
review
Review

2D Fourier Transform

Nyquist sampling theory

- Sampling Rate ≥ 2 * max frequency in the signal

Antialiasing

- prefiltering

Gaussian pyramid

- Mipmap

review3
Review

2D Fourier Transform

Nyquist sampling theory

- Sampling Rate ≥ 2 * max frequency in the signal

Antialiasing

- prefiltering

Gaussian pyramid

- Mipmap

review 2d fourier transform
Review: 2D Fourier Transform

Fourier Transform:

Inverse Fourier transform:

review nyquist sampling rate

f(x)

F(u)

x

fmax

u

-fmax

Fs(u)

fs(x)

x

fmax

u

-fmax

-2T

-T

0

T

2T

Review: Nyquist Sampling Rate

Fourier transform

Inverse Fourier transform

Sampling Rate ≥ 2 * max frequency in the signal!

review antialiasing

fmax

u

-fmax

Review: Antialiasing

Increase the sampling rate to above twice the highest frequency

Introduce an anti-aliasing filter to reduce fmax

Fs(u)

fmax

u

-fmax

review antialiasing7

fmax

u

-fmax

Review: Antialiasing

Increase the sampling rate to above twice the highest frequency

Introduce an anti-aliasing filter to reduce fmax

  • - Sinc filter
  • Mean filter
  • - Triangular filter
  • - Gaussian filter

Fs(u)

fmax

u

-fmax

slide8

Review: Image Pyramids

Known as a Gaussian Pyramid

  • MipMap (Williams, 1983)
outline
Outline

Image Processing

- Median filtering

- Bilateral filtering

- Edge detection

- Corner detection

filtering
Filtering

input

Gaussian filter

median filter
Median Filter
  • For each neighbor in image, sliding the window
  • Sort pixel values
  • Set the center pixel to the median
median filter12
Median Filter

input

Gaussian filter

Median filter

median filter examples
Median Filter Examples

input

Median 7X7

median filter examples14
Median Filter Examples

Median 11X11

Median 3X3

median filter examples15
Median Filter Examples

Straight edges kept

Median 11X11

Median 3X3

Sharp features lost

median filter properties
Median Filter Properties

Can remove outliers (peppers and salts)

Window size controls size of structure

Preserve some details but sharp corners and edges might get lost

comparison of mean gaussian and median18
Comparison of Mean, Gaussian, and Median

original

Gaussian with 6 pixels

comparison of mean gaussian and median19
Comparison of Mean, Gaussian, and Median

original

Median with 6 pixels

common problems
Common Problems

Mean: blurs image, removes simple noise, no details are preserved

common problems21
Common Problems

Mean: blurs image, removes simple noise, no details are preserved

Gaussian: blurs image, preserves details only for small σ.

common problems22
Common Problems

Mean: blurs image, removes simple noise, no details are preserved

Gaussian: blurs image, preserves details only for small σ.

Median: preserves some details, good at removing strong noise

common problems23
Common Problems

Mean: blurs image, removes simple noise, no details are preserved

Gaussian: blurs image, preserves details only for small σ.

Median: preserves some details, good at removing strong noise

Can we find a filter that not only smooths regions but preserves edges?

common problems24
Common Problems

Mean: blurs image, removes simple noise, no details are preserved

Gaussian: blurs image, preserves details only for small σ.

Median: preserves some details, good at removing strong noise

Can we find a filter that not only smooths regions but preserves edges?

- yes, bilateral filter

outline25
Outline

Image Processing

- Median filtering

- Bilateral filtering

- Edge detection

- Corner detection

what is bilateral filter
What Is Bilateral Filter?

Bilateral

- Affecting or undertaken by two sides equally

Property:

- Convolution filter

- Smooth image but preserve edges

- Operates in the domain and the range of image

bilateral filter example
Bilateral Filter Example

Gaussian filter

Bilateral filter

bilateral filter example28
Bilateral Filter Example

Gaussian filter

Bilateral filter

slide29

1D Graphical Example

Center Sample u

I(p)

Neighborhood

p

It is clear that in weighting this neighborhood, we would like to preserve the step

slide31

Filtered Values

I(p)

Filtered value

p

slide32

Edges Are Smoothed

I(p)

Filtered value

p

slide33

What Causes the Problem?

I(p)

Filtered value

p

slide34

What Causes the Problem?

Same weights for these two pixels!!

I(p)

Filtered value

p

slide36

Bilateral Filtering

Denoise

Feature preserving

Bilateral filter

Normalization

slide37

Kernel Properties

  • Per each sample, we can define a ‘Kernel’ that averages its neighborhood
  • This kernel changes from sample to sample!
  • The sum of the kernel entries is 1 due to the normalization,
  • The center entry in the kernel is the largest,
  • Subject to the above, the kernel can take any form (as opposed to filters which are monotonically decreasing).
slide38

Filter Parameters

As proposed by Tomasi and Manduchi, the filter is controlled by 3 parameters:

The filter can be applied for several iterations in order to further strengthen its edge-preserving smoothing

N(u) – The neighbor size of the filter support,

c – The variance of the spatial distances,

s – The variance of the value distances,

bilateral filter43
Bilateral Filter

input

Ws*Wc

Wc

Ws

bilateral filter44
Bilateral Filter

input

Output

Ws*Wc

Wc

Ws

bilateral filter results46
Bilateral Filter Results

σc = 3, σs = 3

bilateral filter results47
Bilateral Filter Results

σc = 6, σs = 3

bilateral filter results48
Bilateral Filter Results

σc = 12, σs = 3

bilateral filter results49
Bilateral Filter Results

σc = 12, σs = 6

bilateral filter results50
Bilateral Filter Results

σc = 15, σs = 8

additional comments
Additional Comments

The bilateral filter is a powerful filter:

- Can work with any reasonable distances function Ws and Wc,

- Easily extended to higher dimension signals, e.g. Images, video, mesh, animation data etc.

- Easily extended to vectored-signals, e.g. Color images, etc.

Bilateral Mesh Denoising

[Fleishman et al, siggraph 03]

outline52
Outline

Image Processing

- Median filtering

- Bilateral filtering

- Edge detection

- Corner detection

edge detection
Edge Detection

What are edges in this image?

edge detection54
Edge Detection

What are edges in this image?

edge detection55
Edge Detection

One of the most important image processing problems

- really easy for human

- really difficult for computers

- fundamental for computer vision

- very important for many graphics applications

How to tell a pixel is on the edge?

edge detection56
Edge Detection

One of the most important image processing problems

- really easy for human

- really difficult for computers

- fundamental for computer vision

- very important for many graphics applications

How to tell a pixel is on the edge?

edge detection57
Edge Detection

One of the most important image processing problems

- really easy for human

- really difficult for computers

- fundamental for computer vision

- very important for many graphics applications

How to tell a pixel is on the edge?

edge detection58
Edge Detection

One of the most important image processing problems

- really easy for human

- really difficult for computers

- fundamental for computer vision

- very important for many graphics applications

How to tell a pixel is on the edge?

  • Related to the first-derivative.
  • - How to compute the first-derivatives of an image?
gradient
Gradient

A vector variable

- Direction of the maximum growth of the function

- Magnitude of the growth

- Perpendicular to the edge direction

how to calculate gradient
How to Calculate Gradient?

Intensity

How to compute ?

Pixel column

the good ole taylor series
The Good ole’ Taylor Series

or…

Subtracting the second from the first we obtain

discrete gradient estimation
Discrete Gradient Estimation

For discrete functions, we can use the first order approximation of the gradient

where h corresponds to the step size

discrete gradient estimation63
Discrete Gradient Estimation

For discrete functions, we can use the first order approximation of the gradient

where h corresponds to the step size

For our purposes, h corresponds to the width of 1 pixel =>

discrete gradient estimation64
Discrete Gradient Estimation

So how can we compute the image gradient efficiently?

- Using our good old friend convolution!

- Dropped off the “divide by 2” for speed considerations.

- This only scales the gradient.

basic edge detection step 1

Horizontal

[-1 0 1]

INPUT IMAGE

1) Edge

Enhancement

Vertical

[-1 0 1]T

Basic Edge Detection Step 1

But isn’t edge detection susceptible to noise?

basic edge detection steps 1 2

1) Noise

Smoothing

Basic Edge Detection Steps 1-2

Horizontal

[-1 0 1]

INPUT IMAGE

2) Edge

Enhancement

Vertical

[-1 0 1]T

discrete gradient estimation68
Discrete Gradient Estimation

Remember that the gradient is a vector and we have calculated the coefficients in the x and y directions at each point in the image

After convolving, we get the magnitude of the gradient from at each point (pixel) from

In practice, we often sum the absolute values of the components for computational efficiency

basic edge detection cont d

1) Noise

Smoothing

Basic Edge Detection (cont’d)

Horizontal

[-1 0 1]

INPUT IMAGE

2) Edge

Enhancement

Vertical

[-1 0 1]T

“GRADIENT” IMAGE

basic edge detection cont d70

1) Noise

Smoothing

Basic Edge Detection (cont’d)

Horizontal

[-1 0 1]

INPUT IMAGE

2) Edge

Enhancement

Vertical

[-1 0 1]T

What does the gradient image mean?

“GRADIENT” IMAGE

basic edge detection cont d71

1) Noise

Smoothing

Basic Edge Detection (cont’d)

Horizontal

[-1 0 1]

INPUT IMAGE

2) Edge

Enhancement

Vertical

[-1 0 1]T

What does the gradient image mean?

- Magnitude of intensity changes around each pixel

“GRADIENT” IMAGE

discrete gradient estimation72
Discrete Gradient Estimation

So how do we segment the edges from the rest of the scene?

- Thresholding!

the effects of thresholding
The Effects of Thresholding

While edge features are independent of illumination, the edge strength is not!

Results from threshold values of 50 and 100

basic edge detection summary

1) Noise

Smoothing

Basic Edge Detection Summary

Horizontal

[-1 0 1]

INPUT IMAGE

2) Edge

Enhancement

Vertical

[-1 0 1]T

3)Threshold

EDGE IMAGE

“GRADIENT” IMAGE

the effects of filtering noise
The effects of Filtering Noise

Threshold

20

Threshold

50

Unsmoothed Edges

Gaussian Smoothing

sobel edge detection
Sobel Edge Detection

One of the dominant edge detection schemes uses the Sobel operators

Convolving each of these with the original image generates horizontal and vertical gradient images that are combined as before

sobel edge detection77
Sobel Edge Detection

One of the dominant edge detection schemes uses the Sobel operators

  • - Can be approximated as a derivative of Gaussian
  • First Gaussian smoothing and then compute derivatives
sobel edge detection78
Sobel Edge Detection

One of the dominant edge detection schemes uses the Sobel operators

  • - Can be approximated as a derivative of Gaussian
  • First Gaussian smoothing and then compute derivatives

Why?

sobel edge detection79
Sobel Edge Detection

One of the dominant edge detection schemes uses the Sobel operators

  • - Can be approximated as a derivative of Gaussian
  • First Gaussian smoothing and then compute derivatives
  • - In practice we may still need to smooth for noise
robert and prewitt edge detectors
Robert and Prewitt Edge Detectors

The Prewitt is similar to the Sobel, but uses a different kernel

Roberts was an early edge detector kernel

second derivative edge detector
Second Derivative Edge Detector

Sobel Operator can produce thick edges; ideally we are looking for infinitely thin boundaries

second derivative edge detector83
Second Derivative Edge Detector

Sobel Operator can produce thick edges; ideally we are looking for infinitely thin boundaries

An alternative approach is to look for local extrema in the first derivative

A peak in the first derivative corresponds what in the second derivative?

second derivative edge detector84
Second Derivative Edge Detector

Sobel Operator can produce thick edges; ideally we are looking for infinitely thin boundaries

An alternative approach is to look for local extrema in the first derivative

A peak in the first derivative corresponds what in the second derivative?

localization with laplacian
Localization with Laplacian

An equivalent measure of the second derivative in 2D is the Laplacian:

Numerically, we approximate the Laplacian using the following filtering kernel:

Zeros crossing of the filter corresponds to positions of maximum gradient.

- can be used for detecting edges

- might be sensitive to noise

- need to filter the image

laplacian edge detector
Laplacian Edge Detector

Java applet: Click here

outline89
Outline

Image Processing

- Median filtering

- Bilateral filtering

- Edge detection

- Corner detection

corner detection
Corner Detection

Why do we need to detect corners?

- Corners correspond to points in both the world and image spaces

- Very important for image based rendering and modeling

- Very useful in computer vision

what are corners
What are Corners?

Point features

Where two edges come together

Where the image gradient has significant components in the x and y direction

We will establish corners from the gradient rather than the edge images

basic ideas
Basic Ideas

What are gradients along x and y directions?

basic ideas93
Basic Ideas

What are gradients along x and y directions?

basic ideas94
Basic Ideas

What are gradients along x and y directions?

How to measure corners based on the gradient images?

basic ideas95
Basic Ideas

What are gradients along x and y directions?

How to measure corners based on the gradient images?

How to measure corners based on the gradient images?

- two major axes in the local window!

corner detection algorithm
Corner Detection Algorithm

1. Compute the image gradients

2. Define a neighborhood size as an area of interest around each pixel

3x3

neighborhood

slide97

Corner Detection Algorithm (cont’d)

  • For each image pixel (i,j), construct the following matrix from it and its neighborhood values

e.g.

Similar to covariance matrix (Ix,Iy)T!

corner detection algorithm cont d
Corner Detection Algorithm (cont’d)
  • For each matrix C(i,j), determine the 2 eigenvalues λ(i.j)=[λ1, λ2].

Simple case:

  • This means dominant gradient direction aligns with x or y axis.
  • If either λ1 or λ2 is close to zero, then this is not a corner.
corner detection algorithm cont d99
Corner Detection Algorithm (cont’d)
  • For each matrix C(i,j), determine the 2 eigenvalues λ(i.j)=[λ1, λ2].

Simple case:

Interior Region

Edge

Corner

Isolated pixels

Large λ1 and small λ2

Large λ1 and large λ2

small λ1 and small λ2

λ1, λ2=0

corner detection algorithm cont d100
Corner Detection Algorithm (cont’d)
  • For each matrix C(i,j), determine the 2 eigenvalues λ(i.j)=[λ1, λ2].

General case:

  • This is just a rotated version of the one on last slide
  • If either λ1 or λ2 is close to zero, then this is not a corner.
eigen values and corner
Eigen-values and Corner
  • λ1 is large
  • λ2 is large
eigen values and corner102
Eigen-values and Corner
  • λ1 is large
  • λ2 is small
eigen values and corner103
Eigen-values and Corner
  • λ1 is small
  • λ2 is small
corner detection algorithm cont d104
Corner Detection Algorithm (cont’d)
  • For each matrix C(i,j), determine the 2 eigenvalues λ(i.j)=[λ1, λ2].

5. If both λ1 and λ2 are big, we have a corner (Harris also checks the ratio of λs is not too high)

ISSUE: The corners obtained will be

a function of the threshold !

image gradients106
Image Gradients

Closeup of image orientation at each pixel

the orientation field
The Orientation Field

Corners are detected where both λ1 and λ2 are big

the orientation field108
The Orientation Field

Corners are detected where both λ1 and λ2 are big

corner detection sample results
Corner Detection Sample Results

Threshold=25,000

Threshold=10,000

Threshold=5,000

summary
Summary

Image Processing

- Median filtering

- Bilateral filtering

- Edge detection

- Corner detection

next lecture
Next Lecture

Image composition and blending

Image warping

Image morphing