Fourier Transform in Image Processing

1 / 102

# Fourier Transform in Image Processing - PowerPoint PPT Presentation

Fourier Transform in Image Processing. Review - Image as a Function. We can think of an image as a function, f , f: R 2  R f ( x, y ) gives the intensity at position ( x, y ) Realistically, we expect the image only to be defined over a rectangle, with a finite range:

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

## PowerPoint Slideshow about 'Fourier Transform in Image Processing' - melanie-ashley

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Review - Image as a Function
• We can think of an image as a function, f,
• f:R2R
• f (x, y)gives the intensity at position (x, y)
• Realistically, we expect the image only to be defined over a rectangle, with a finite range:
• f: [a,b]x[c,d]  [0,1]
• A color image is just three functions pasted together.
• We can write this as a “vector-valued” function:
Image as a Function
• function, f,
• f:R2R
• f (x, y)gives the intensity at position (x, y)
Transformations on Images
• Define a new image gin terms of an existing image f
• We can transform:
• either the domain
• or the range of f
• Range transformation:
• What kinds of operations on the image can this transformation perform?
• What t transformations are good for what?

Operations that change the domain of image

• Some operations preserve the range but change the domain of f
• Here is an example that changes the domain:
• What kinds of operations can this perform?
• Still other operations operate on both the domain and the range of f .

In general we are interested in all these transformations

Specifically we want them to have some nice (like linear) properties

Linear Shift Invariant Systems (LSIS)

Linearity:

Shift invariance:

LSIS

Example of LSIS: ideal lens

Linear Shift Invariant Systems (LSIS)

Defocused image ( g ) is a processed version of the focused image ( f )

Ideal lens is a LSIS

Linearity: Brightness variation

Shift invariance: Scene movement

(this is not valid for lenses with non-linear distortions)

LSIS is Doing a Convolution

Linear Shift Invariant Systems (LSIS)

convolutionis linear and shift invariant

Continuous convolution defined by integral

Discrete convolution from previous lectures defined by finite sums

We negate (mirror) the kernel of convolution h

kernel h

Convolution – two more Examples

Here we show two more examples of convolution of functions f and g

We use two non-idea approximation of Dirac’s Delta function as a signal model

From Eric Weinstein’s Math World

Convolution – Another Example

1

1

-1

1

-1

1

Here we convolve signals a and b into convolved signal c

1

-2

-1

1

2

Convolution Kernel – Impulse Response
• We ask question “What h will give us g = f ?”

Dirac Delta Function (Unit Impulse)

Sifting property:

We convolve f with Dirac Delta:

And we get this

Therefore g=f when we convolve with delta

Optical

System

point source

• However, optical systems are never ideal.
• Point spread function of Human Eyes

Now we apply information from last slide to optical system

Optical

System

scene

image

• Ideally, the optical system should be a Dirac delta function. (no distortion)

PSF stands for Point Spread Function

If the PSF of human eye is different (not a good approximation of delta) then we have eye troubles, related to this PSF type

PSF stands for Point Spread Function

normal vision

myopia

hyperopia

Tell me what your PSF is and I will tell you what is your eye problem.

astigmatism

Images by Richmond Eye Associates

Properties of Convolution

Convolution operator

• Commutative
• Associative

This is unlike matrix multiplication and Kronecker (tensor) products

We used in cascading filters in previous lectures. Now we have a theoretical base

These are general properties of convolution

, not related to signal representation

• However signals for convolution can be represented in various ways.
• So the question is, “how to represent signals?”
• 1D
• 2D
• 3D?
How to Represent Signals?
• Option 1: Taylor series represents any function using polynomials.
• Polynomials are not the best - unstable and not very physically meaningful. (they still have some applications)
• Easier to talk about “signals” in terms of its “frequencies”

(how fast/often signals change, etc).

Jean Baptiste Joseph Fourier (1768-1830)
• Any periodic function can be rewritten as a weighted sum of Sines and Cosines of different frequencies.
• Don’t believe it?
• Neither did Lagrange, Laplace, Poisson and other big wigs
• Not translated into English until 1878!
• But it’s true!
• called Fourier Series
• Possibly the greatest tool

used in Engineering

A Sum of Sinusoids
• Our building block:
• Add enough of them to get any signal f(x) you want!
• How many degrees of freedom?
• What does each control?
• Which one encodes the coarse vs. fine structure of the signal?
Math Review - Complex numbers
• Real numbers:

1

-5.2

• Complex numbers

4.2 + 3.7i

9.4447 – 6.7i

-5.2 (-5.2 + 0i)

We often denote in EE i by j

Complex numbers

4.2 + 3.7i

9.4447 – 6.7i

-5.2 (-5.2 + 0i)

General Form

Z = a + bi

Re(Z) = a

Im(Z) = b

Amplitude

A = | Z | = √(a2 + b2)

Phase

 =  Z = tan-1(b/a)

Math Review - Complex numbers

Real and imaginary parts

b

a

Math Review – Complex Numbers
• Polar Coordinate

Z = a + bi

• Amplitude

A = √(a2 + b2)

• Phase

 = tan-1(b/a)

A

Math Review – Complex Numbers and Cosine Waves
• Cosine wave has three properties
• Frequency
• Amplitude
• Phase
• Complex number has two properties
• Amplitude
• Wave
• Complex numbers to represent cosine waves at varying frequency
• Frequency 1: Z1 = 5 +2i
• Frequency 2: Z2 = -3 + 4i
• Frequency 3: Z3 = 1.3 – 1.6i

Simple but great idea !!

Inverse Fourier

Transform

Fourier

Transform

F(w)

f(x)

F(w)

f(x)

Fourier Transform Idea
• We want to understand the frequency w of our signal. So, let’s reparameterizethe signal by w instead of x:
• For every wfrom 0 to infinity, F(w) holds the amplitude A and phase f of the corresponding sine
• How can F hold both? Complex number trick!

All info preserved

Math Review - Periodic Functions

If there is some a, for a function f(x), such that

f(x) = f(x + na)

then function is periodicwith the period a

a

2a

3a

0

Amplitude

Phase

f(x) = cos (x)

f(x) = 5 cos (x)

f(x) = 5 cos (x + 3.14)

Math Review - Attributes of cosine wave
Amplitude

Phase

Frequency

f(x) = 5 cos (x)

f(x) = 5 cos (x + 3.14)

f(x) = 5 cos (3 x + 3.14)

Math Review - Attributes of cosine wave
f(x) = cos (x)

Amplitude, Frequency, Phase

Math Review - Attributes of cosine wave
• f(x) = A cos (kx + )

This math is base of Fourier Analysis and Fourier Synthesis

Time and Frequency
• example : g(t) = sin(2 f t) + (1/3)sin(2(3f) t)

Our periodic signal is a mixture of two frequencies

Before we go to using complex numbers and formulas let us get intuition about time and frequency

Time and Frequency
• example : g(t) = sin(2 f t) + (1/3)sin(2(3f) t)

=

+

Frequency Spectra
• example : g(t) = sin(2pf t) + (1/3)sin(2p(3f) t)

=

+

In time domain

In frequency domain

Frequency Spectra
• Usually, frequency is more interesting than the phase

We will decompose it to frequencies,

we will add one more coefficient based base function in each slide

Frequency Spectra

First base function

second base function

=

+

=

Sum of first and second base functions is already not bad approximation

Frequency Spectra

=

+

=

Sum of first , second and third base functions is even better approximation

Frequency Spectra

=

In time domain

In frequency domain

Frequency Spectra

Observe the negative values of domain that appear in spectrum

Fourier Transform is Just a change of basis

M can be a matrix, a decision diagram, a butterfly, a system (physical or simulated)

M * f(x) = F(w)

=

*

.

.

.

IFT: Just a change of basis

Inverse Fourier Transform is similarly only a change of basis

M-1 * F(w)= f(x)

=

*

.

.

.

Image

Fourier Transform

Fourier Transform is an invertible operator

FT

v2 will display image or its transform, Matlab examples.

Image

Fourier Transform

Ny ⁄ 2

Ny

y

Nx ⁄ 2

0

Nx

x

{f(x,y)} = F(kx,ky)

{F(kx,ky)} = f(x,y)}

Fourier Transform is an invertible operator

f(x,y)

F(kx,ky)

Similarly as for Hadamard-Walsh or Reed-Muller Transforms we can talk about transform pairs.

f(x)

= F(s)

Euler’s Formula

Continuous Fourier Transform

Various normalizations of pairs

Image Domain

Forward Transform

f(x,y,z)

g(x)

F

Fourier Domain

Reciprocal space

Fourier Space

K-space

Frequency Space

Spectral domain

ReverseTransform, Inverse Transform

F(kx,ky,kz)

G(s)

F

Some Conventions for Fourier Transforms
Fourier Analysis

Decompose f(x) into a series of cosine waves that when summed reconstruct f(x)

Fourier Analysis in 1D. Audio signals

Amplitude Only

5

10

15

(Hz)

5

10

15

(Hz)

Fourier Analysis in 1D. Audio signals

5

10

15

(Hz)

Fourier Analysis in 1D. Spectrum Analyzer.

iTunes performs fourier analysis.

We know it from our Hi-Fi equipment

The same can be done with standard images or radar images or any other images, in any number of dimensions

Fourier Synthesis

Summing cosine waves reconstructs the original function

Periodic Boxcar

Example: Fourier Synthesis of Boxcar Function

Boxcar function

Can this function be reproduced with cosine waves?

k=1. One cycle per period

A1·cos(2kx + 1)

k=1

1

Ak·cos(2kx + k)

k=1

k=2. Two cycles per period

A2·cos(2kx + 2)

k=2

2

Ak·cos(2kx + k)

k=1

k=3. Three cycles per period

A3·cos(2kx + 3)

k=3

3

Ak·cos(2kx + k)

k=1

Fourier Synthesis. N Cycles

A3·cos(2kx + 3)

k=3

N

Ak·cos(2kx + k)

k=1

An image is two dimensional data.

Intensities as a function of x,y

White pixels represent the highest intensities.

Greyscale image of iris

128x128 pixels

Example: Fourier Synthesis of a 2D Function
Fourier Synthesis of a 2D Function

F(2,3)

We select coefficients, multiply them by weights and add

We can create artificial images

Fourier Filters
• Fourier Filters in 1D data (sound) change the image by changing which frequencies of cosine waves go into the image
• Represented by 1D spectral profile
• 2D Specteral Profile is rotationally symmetrized 1D profile
Low and High Frequency Image Filters
• Low frequency terms
• Close to origin in Fourier Space
• Changes with great spatial extent (like ice gradient), or particle size
• High frequency terms
• Closer to edge in Fourier Space
• Necessary to represent edges or high-resolution features
Frequency-based Filters
• Low-pass Filter (blurs)
• Restricts data to low-frequency components
• High-pass Filter (sharpens)
• Restricts data to high-frequency-components
• Band-pass Filter
• Restrict data to a band of frequencies
• Band-stop Filter
• Suppress a certain band of frequencies

We will show many practical examples soon

Cutoff Low-pass Filter

Image is blurred

Sharp features are lost

Ringing artifacts

Filter characteristics

bandpass

How the filter looks in 2D

unprocessed

spectra

lowpass

highpass

Convolution again

Our goal is now to approach the most important results of spectral processing

Convolution

Convolution of some function f(x) with some kernel g(x)

Continuous

Discrete

*

=

Convolution in 2D

image

kernel

result

x

x

x

=

x

x

x

x

x

x

x

x

x

x

x

x

x

=

Convolution Theorem

Convolution in image domain

Is equivalent to multiplication in fourier domain

f  g = {FG}

f = FG

G

Can be derived from above because FG/G = F and f = F-1 (F)

Lowpass Filtering by Convolution
• f  g = {FG}
• Camera shake

We showed examples of this using standard convolution, but it is done in modern equipment through digitally calculated FFT and IFFT

2D gaussian is example of kernel

2D gaussian as a matrix of a kernel

Power spectrum

PS =

Incoherant average of transform

obs(x) = f(x)  psf(x)  env(x) + n(x)

observed image

noise

f(x) for true particle

envelope function

Contrast Theory

F2(s) CTF2(s) Env2(s) + N2(s)

5 waves

25 waves

125 waves

Gibbs Ringing

Even with 125 waves we still have some small Gibbs Ringing

Arbitrary function

Single Analytic Expression

Spatial Domain (x)

Frequency Domain (u)

Fourier Transform – more formally

Represent the signal as an infinite weighted sum

of an infinite number of sinusoids

Note:

(Frequency Spectrum F(u))

Inverse Fourier Transform (IFT)

Fourier Transform

• Also, defined as:

Fourier Transform

Note:

• Inverse Fourier Transform (IFT)
Fourier Transform Pairs (I)

Fourier Transform

Note that these are derived using

angular frequency ( )

Fourier Transform Pairs (I)

Fourier Transform

Note that these are derived using

angular frequency ( )

Fourier Transform Pairs (I)

Note that these are derived using

angular

frequency ( )

Convolution in spatial domain

Multiplication in frequency domain

Fourier Transform and Convolution

Let

Here we derive the most important formula of image processing and other applications of Fourier Transform

Then

From definition

Separate domains

Linearity

Scaling

Shifting

Symmetry

Conjugation

Convolution

Differentiation

Properties of Fourier Transform 1

Spatial Domain (x)

Frequency Domain (u)

Note that these are derived using

frequency ( )

Properties of Fourier Transform 2

Star will be used for conjugation in complex plane

Substitute f instead of g in this formula to get the upper formula

For instance, when f is R then F is RE, IO,. etc

RO – real odd

IE – imaginary even

IO – imaginary odd

RE – Real even

CE – Complex even

CO – Complex odd

We explained even-odd properties for Walsh, we will explain again for other transforms

2. Let us use a Gaussian kernel

• 3. Then
Example use of Fourier: Smoothing/Blurring

We want a smoothed function of f(x)

But we do this in spectral domain

H(u) attenuates high frequencies in F(u) (Low-pass Filter)!

Digital Images

The scene is

• projected on a 2D plane,
• sampled on a regular grid, and each sample is
• quantized(rounded to the nearest integer)

Image as a matrix

Fourier Transform is invertible operator

Math Review

Periodic functions

Amplitude, Phase and Frequency

Complex number

Amplitude and Phase

Fourier Analysis (Forward Transform)Decomposition of periodic signal into cosine waves

Fourier Synthesis (Inverse Transform)Summation of cosine waves into multi-frequency waveform

Fourier Transforms in 1D, 2D, 3D, ND

Image Analysis

Image (real-valued)

Transform (complex-valued, amplitude plot)

Fourier Filters

Low-pass

High-pass

Band-pass

Band-stop

Convolution Theorem

Deconvolute by Division in Fourier Space

All Fourier Filters can be expressed as real-space Convolution Kernels

Lens does Fourier transforms

Diffraction Microscopy

Review
Where we are in this class?
• We defined continuous Fourier Transform
• We defined discrete Fourier Transform DFT
• DFT is calculated by matrix multiplication
• It is slow
• We will need factorizations, Kroneckers, recursions and butterflies to calculate it faster.
• Only then FT will become a really useful tool, FFT.
• Wikipedia
• Mathworld
• The Fourier Transform and its Applications. Ronald Bracewell
LowPass Filters

filtered=image.process(‘filter.lowpass.guass’, {‘sigma’:0.10})

filtered=image.process(‘filter.lowpass.butterworth’, {‘low_cutoff_frequency’:0.10, ‘high_cutoff_frequency’:0.35})

filtered=image.process(‘filter.lowpass.tanh’, {‘cutoff_frequency’:0.10, ‘falloff’:0.2})

HighPass Filters

filtered=image.process(‘filter.highpass.guass’, {‘sigma’:0.10})

filtered=image.process(‘filter.highpass.butterworth’, {‘low_cutoff_frequency’:0.10, ‘high_cutoff_frequency’:0.35})

filtered=image.process(‘filter.highpass.tanh’, {‘cutoff_frequency’:0.10, ‘falloff’:0.2})

BandPass Filters

filtered=image.process(‘filter.bandpass.guass’, {‘center’:0.2,‘sigma’:0.10})

filtered=image.process(‘filter.bandpass.butterworth’, {‘low_cutoff_frequency’:0.10, ‘high_cutoff_frequency’:0.35})

filtered=image.process(‘filter.bandpass.tanh’, {‘cutoff_frequency’:0.10, ‘falloff’:0.2})

Filtering with EMAN2

### Sources of slides

S. Narasimhan

Mike Marsh

National Center for Macromolecular Imaging

Baylor College of Medicine