- 175 Views
- Uploaded on

Download Presentation
## PowerPoint Slideshow about 'CSCE 641 Computer Graphics: Image Processing' - christmas

**An Image/Link below is provided (as is) to download presentation**

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

### CSCE 641 Computer Graphics:Image Processing

Jinxiang Chai

Review

2D Fourier Transform

Nyquist sampling theory

- Sampling Rate ≥ 2 * max frequency in the signal

Antialiasing

- prefiltering

Gaussian pyramid

- Mipmap

Review

2D Fourier Transform

Nyquist sampling theory

- Sampling Rate ≥ 2 * max frequency in the signal

Antialiasing

- prefiltering

Gaussian pyramid

- Mipmap

F(u)

x

fmax

u

-fmax

Fs(u)

fs(x)

x

fmax

u

-fmax

…

…

-2T

-T

0

T

2T

Review: Nyquist Sampling RateFourier transform

Inverse Fourier transform

Sampling Rate ≥ 2 * max frequency in the signal!

u

-fmax

Review: AntialiasingIncrease the sampling rate to above twice the highest frequency

Introduce an anti-aliasing filter to reduce fmax

Fs(u)

fmax

u

-fmax

u

-fmax

Review: AntialiasingIncrease the sampling rate to above twice the highest frequency

Introduce an anti-aliasing filter to reduce fmax

- - Sinc filter
- Mean filter
- - Triangular filter
- - Gaussian filter

Fs(u)

fmax

u

-fmax

Outline

Image Processing

- Median filtering

- Bilateral filtering

- Edge detection

- Corner detection

Median Filter

- For each neighbor in image, sliding the window
- Sort pixel values
- Set the center pixel to the median

Median Filter Properties

Can remove outliers (peppers and salts)

Window size controls size of structure

Preserve some details but sharp corners and edges might get lost

Common Problems

Mean: blurs image, removes simple noise, no details are preserved

Common Problems

Mean: blurs image, removes simple noise, no details are preserved

Gaussian: blurs image, preserves details only for small σ.

Common Problems

Mean: blurs image, removes simple noise, no details are preserved

Gaussian: blurs image, preserves details only for small σ.

Median: preserves some details, good at removing strong noise

Common Problems

Mean: blurs image, removes simple noise, no details are preserved

Gaussian: blurs image, preserves details only for small σ.

Median: preserves some details, good at removing strong noise

Can we find a filter that not only smooths regions but preserves edges?

Common Problems

Mean: blurs image, removes simple noise, no details are preserved

Gaussian: blurs image, preserves details only for small σ.

Median: preserves some details, good at removing strong noise

Can we find a filter that not only smooths regions but preserves edges?

- yes, bilateral filter

What Is Bilateral Filter?

Bilateral

- Affecting or undertaken by two sides equally

Property:

- Convolution filter

- Smooth image but preserve edges

- Operates in the domain and the range of image

Center Sample u

I(p)

Neighborhood

p

It is clear that in weighting this neighborhood, we would like to preserve the step

- Per each sample, we can define a ‘Kernel’ that averages its neighborhood
- This kernel changes from sample to sample!
- The sum of the kernel entries is 1 due to the normalization,
- The center entry in the kernel is the largest,
- Subject to the above, the kernel can take any form (as opposed to filters which are monotonically decreasing).

As proposed by Tomasi and Manduchi, the filter is controlled by 3 parameters:

The filter can be applied for several iterations in order to further strengthen its edge-preserving smoothing

N(u) – The neighbor size of the filter support,

c – The variance of the spatial distances,

s – The variance of the value distances,

Bilateral Filter

input

Bilateral Filter

input

Bilateral Filter Results

Original

Bilateral Filter Results

σc = 3, σs = 3

Bilateral Filter Results

σc = 6, σs = 3

Bilateral Filter Results

σc = 12, σs = 3

Bilateral Filter Results

σc = 12, σs = 6

Bilateral Filter Results

σc = 15, σs = 8

Additional Comments

The bilateral filter is a powerful filter:

- Can work with any reasonable distances function Ws and Wc,

- Easily extended to higher dimension signals, e.g. Images, video, mesh, animation data etc.

- Easily extended to vectored-signals, e.g. Color images, etc.

Bilateral Mesh Denoising

[Fleishman et al, siggraph 03]

Outline

Image Processing

- Median filtering

- Bilateral filtering

- Edge detection

- Corner detection

Edge Detection

What are edges in this image?

Edge Detection

What are edges in this image?

Edge Detection

One of the most important image processing problems

- really easy for human

- really difficult for computers

- fundamental for computer vision

- very important for many graphics applications

How to tell a pixel is on the edge?

Edge Detection

One of the most important image processing problems

- really easy for human

- really difficult for computers

- fundamental for computer vision

- very important for many graphics applications

How to tell a pixel is on the edge?

Edge Detection

One of the most important image processing problems

- really easy for human

- really difficult for computers

- fundamental for computer vision

- very important for many graphics applications

How to tell a pixel is on the edge?

Edge Detection

One of the most important image processing problems

- really easy for human

- really difficult for computers

- fundamental for computer vision

- very important for many graphics applications

How to tell a pixel is on the edge?

- Related to the first-derivative.
- - How to compute the first-derivatives of an image?

Gradient

A vector variable

- Direction of the maximum growth of the function

- Magnitude of the growth

- Perpendicular to the edge direction

Discrete Gradient Estimation

For discrete functions, we can use the first order approximation of the gradient

where h corresponds to the step size

Discrete Gradient Estimation

For discrete functions, we can use the first order approximation of the gradient

where h corresponds to the step size

For our purposes, h corresponds to the width of 1 pixel =>

Discrete Gradient Estimation

So how can we compute the image gradient efficiently?

- Using our good old friend convolution!

- Dropped off the “divide by 2” for speed considerations.

- This only scales the gradient.

[-1 0 1]

INPUT IMAGE

1) Edge

Enhancement

Vertical

[-1 0 1]T

Basic Edge Detection Step 1But isn’t edge detection susceptible to noise?

Smoothing

Basic Edge Detection Steps 1-2Horizontal

[-1 0 1]

INPUT IMAGE

2) Edge

Enhancement

Vertical

[-1 0 1]T

Discrete Gradient Estimation

Remember that the gradient is a vector and we have calculated the coefficients in the x and y directions at each point in the image

After convolving, we get the magnitude of the gradient from at each point (pixel) from

In practice, we often sum the absolute values of the components for computational efficiency

Smoothing

Basic Edge Detection (cont’d)Horizontal

[-1 0 1]

INPUT IMAGE

2) Edge

Enhancement

Vertical

[-1 0 1]T

“GRADIENT” IMAGE

Smoothing

Basic Edge Detection (cont’d)Horizontal

[-1 0 1]

INPUT IMAGE

2) Edge

Enhancement

Vertical

[-1 0 1]T

What does the gradient image mean?

“GRADIENT” IMAGE

Smoothing

Basic Edge Detection (cont’d)Horizontal

[-1 0 1]

INPUT IMAGE

2) Edge

Enhancement

Vertical

[-1 0 1]T

What does the gradient image mean?

- Magnitude of intensity changes around each pixel

“GRADIENT” IMAGE

Discrete Gradient Estimation

So how do we segment the edges from the rest of the scene?

- Thresholding!

The Effects of Thresholding

While edge features are independent of illumination, the edge strength is not!

Results from threshold values of 50 and 100

Smoothing

Basic Edge Detection SummaryHorizontal

[-1 0 1]

INPUT IMAGE

2) Edge

Enhancement

Vertical

[-1 0 1]T

3)Threshold

EDGE IMAGE

“GRADIENT” IMAGE

Sobel Edge Detection

One of the dominant edge detection schemes uses the Sobel operators

Convolving each of these with the original image generates horizontal and vertical gradient images that are combined as before

Sobel Edge Detection

One of the dominant edge detection schemes uses the Sobel operators

- - Can be approximated as a derivative of Gaussian
- First Gaussian smoothing and then compute derivatives

Sobel Edge Detection

One of the dominant edge detection schemes uses the Sobel operators

- - Can be approximated as a derivative of Gaussian
- First Gaussian smoothing and then compute derivatives

Why?

Sobel Edge Detection

One of the dominant edge detection schemes uses the Sobel operators

- - Can be approximated as a derivative of Gaussian
- First Gaussian smoothing and then compute derivatives
- - In practice we may still need to smooth for noise

Robert and Prewitt Edge Detectors

The Prewitt is similar to the Sobel, but uses a different kernel

Roberts was an early edge detector kernel

Sobel Edge Detector

Java Applet

Second Derivative Edge Detector

Sobel Operator can produce thick edges; ideally we are looking for infinitely thin boundaries

Second Derivative Edge Detector

Sobel Operator can produce thick edges; ideally we are looking for infinitely thin boundaries

An alternative approach is to look for local extrema in the first derivative

A peak in the first derivative corresponds what in the second derivative?

Second Derivative Edge Detector

Sobel Operator can produce thick edges; ideally we are looking for infinitely thin boundaries

An alternative approach is to look for local extrema in the first derivative

A peak in the first derivative corresponds what in the second derivative?

Localization with Laplacian

An equivalent measure of the second derivative in 2D is the Laplacian:

Numerically, we approximate the Laplacian using the following filtering kernel:

Zeros crossing of the filter corresponds to positions of maximum gradient.

- can be used for detecting edges

- might be sensitive to noise

- need to filter the image

Laplacian Edge Detector

Java applet: Click here

Corner Detection

Why do we need to detect corners?

- Corners correspond to points in both the world and image spaces

- Very important for image based rendering and modeling

- Very useful in computer vision

What are Corners?

Point features

Where two edges come together

Where the image gradient has significant components in the x and y direction

We will establish corners from the gradient rather than the edge images

Basic Ideas

What are gradients along x and y directions?

Basic Ideas

What are gradients along x and y directions?

Basic Ideas

What are gradients along x and y directions?

How to measure corners based on the gradient images?

Basic Ideas

What are gradients along x and y directions?

How to measure corners based on the gradient images?

How to measure corners based on the gradient images?

- two major axes in the local window!

Corner Detection Algorithm

1. Compute the image gradients

2. Define a neighborhood size as an area of interest around each pixel

3x3

neighborhood

Corner Detection Algorithm (cont’d)

- For each image pixel (i,j), construct the following matrix from it and its neighborhood values

e.g.

Similar to covariance matrix (Ix,Iy)T!

Corner Detection Algorithm (cont’d)

- For each matrix C(i,j), determine the 2 eigenvalues λ(i.j)=[λ1, λ2].

Simple case:

- This means dominant gradient direction aligns with x or y axis.
- If either λ1 or λ2 is close to zero, then this is not a corner.

Corner Detection Algorithm (cont’d)

- For each matrix C(i,j), determine the 2 eigenvalues λ(i.j)=[λ1, λ2].

Simple case:

Interior Region

Edge

Corner

Isolated pixels

Large λ1 and small λ2

Large λ1 and large λ2

small λ1 and small λ2

λ1, λ2=0

Corner Detection Algorithm (cont’d)

- For each matrix C(i,j), determine the 2 eigenvalues λ(i.j)=[λ1, λ2].

General case:

- This is just a rotated version of the one on last slide
- If either λ1 or λ2 is close to zero, then this is not a corner.

Eigen-values and Corner

- λ1 is large
- λ2 is large

Eigen-values and Corner

- λ1 is large
- λ2 is small

Eigen-values and Corner

- λ1 is small
- λ2 is small

Corner Detection Algorithm (cont’d)

- For each matrix C(i,j), determine the 2 eigenvalues λ(i.j)=[λ1, λ2].

5. If both λ1 and λ2 are big, we have a corner (Harris also checks the ratio of λs is not too high)

ISSUE: The corners obtained will be

a function of the threshold !

Image Gradients

Closeup of image orientation at each pixel

The Orientation Field

Corners are detected where both λ1 and λ2 are big

The Orientation Field

Corners are detected where both λ1 and λ2 are big

Download Presentation

Connecting to Server..