Presents - PowerPoint PPT Presentation

Presents

The Story of Wavelets

Robi PolikarDept. of Electrical & Computer EngineeringRowan University

• Technical Overview
• But…We cannot do that with Fourier Transform….
• Time - frequency representation and the STFT
• Continuous wavelet transform
• Multiresolution analysis and discrete wavelet transform (DWT)
• Application Overview
• Conventional Applications: Data compression, denoising, solution of PDEs, biomedical signal analysis.
• Unconventional applications
• Yes…We can do that with wavelets too…
• Historical Overview
• 1807 ~ 1940s: The reign of the Fourier Transform
• 1940s ~ 1970s: STFT and Subband Coding
• 1980s & 1990s: The Wavelet Transform and MRA

• Transform: A mathematical operation that takes a function or sequence and maps it into another one
• Transforms are good things because…
• The transform of a function may give additional /hidden information about the original function, which may not be available /obvious otherwise
• The transform of an equation may be easier to solve than the original equation (recall your fond memories of Laplace transforms in DFQs)
• The transform of a function/sequence may require less storage, hence provide data compression / reduction
• An operation may be easier to apply on the transformed function, rather than the original function (recall other fond memories on convolution).

Jean B. Joseph Fourier

(1768-1830)

“An arbitrary function, continuous or with discontinuities, defined in a finite interval by an arbitrarily capricious graph can always be expressed as a sum of sinusoids”

J.B.J. Fourier

• Basis functions
• Using only a few blocks  Compressed representation
• Using sinusoids as building blocks  Fourier transform
• Frequency domain representation of the function

• Recall that FT uses complex exponentials (sinusoids) as building blocks.
• For each frequency of complex exponential, the sinusoid at that frequency is compared to the signal.
• If the signal consists of that frequency, the correlation is high  large FT coefficients.
• If the signal does not have any spectral component at a frequency, the correlation at that frequency is low / zero,  small / zero FT coefficient.

F

F

F

F

Complex exponentials (sinusoids) as basis functions:

F

An ultrasonic A-scan using 1.5 MHz transducer, sampled at 10 MHz

• FT identifies all spectral components present in the signal, however it does not provide any information regarding the temporal (time) localization of these components. Why?
• Stationary signals consist of spectral components that do not change in time
• all spectral components exist at all times
• no need to know any time information
• FT works well for stationary signals
• However, non-stationary signals consists of time varying spectral components
• How do we find out which spectral component appears when?
• FT only provides what spectral components exist , not where in time they are located.
• Need some other ways to determine time localization of spectral components

• Stationary signals’ spectral characteristics do not change with time
• Non-stationary signals have time varying spectra

Concatenation

X4(ω)

Perfect knowledge of what

frequencies exist, but no

information about where

these frequencies are

located in time

X5(ω)

Shortcomings of the FT

• Sinusoids and exponentials
• Stretch into infinity in time, no time localization
• Instantaneous in frequency, perfect spectral localization
• Global analysis does not allow analysis of non-stationary signals
• Need a local analysis scheme for a time-frequency representation (TFR) of nonstationary signals
• Windowed F.T. or Short Time F.T. (STFT) : Segmenting the signal into narrow time intervals, narrow enough to be considered stationary, and then take the Fourier transform of each segment, Gabor 1946.
• Followed by other TFRs, which differed from each other by the selection of the windowing function

• Choose a window function of finite length
• Place the window on top of the signal at t=0
• Truncate the signal using this window
• Compute the FT of the truncated signal, save.
• Incrementally slide the window to the right
• Go to step 3, until window reaches the end of the signal
• For each time location where the window is centered, we obtain a different FT
• Hence, each FT provides the spectral information of a separate time-slice of the signal, providing simultaneous time and frequency information

Frequency

parameter

Time

parameter

Signal to

be analyzed

FT Kernel

(basis function)

STFT of signal x(t):

Computed for each

window centered at t=t’

Windowing

function

Windowing function

centered at t=t’

t’=-8 t’=-2

t’=4 t’=8

• STFT provides the time information by computing a different FTs for consecutive time intervals, and then putting them together
• Time-Frequency Representation (TFR)
• Maps 1-D time domain signals to 2-D time-frequency signals
• Consecutive time intervals of the signal are obtained by truncating the signal using a sliding windowing function
• How to choose the windowing function?
• What shape? Rectangular, Gaussian, Elliptic…?
• How wide?
• Wider window require less time steps  low time resolution
• Also, window should be narrow enough to make sure that the portion of the signal falling within the window is stationary
• Can we choose an arbitrarily narrow window…?

Two extreme cases:

• W(t) infinitely long:  STFT turns into FT, providing excellent frequency information (good frequency resolution), but no time information
• W(t) infinitely short:

 STFT then gives the time signal back, with a phase factor. Excellent time information (good time resolution), but no frequency information

Wide analysis window poor time resolution, good frequency resolution

Narrow analysis windowgood time resolution, poor frequency resolution

Once the window is chosen, the resolution is set for both time and frequency.

Frequency resolution: How well two spectral components can be separated from each other in the transform domain

Time resolution: How well two spikes in time can be separated from each other in the transform domain

Both time and frequency resolutions cannot be arbitrarily high!!! We cannot precisely know at what time instance a frequency component is located. We can only know what interval of frequencies are present in which time intervals

• Overcomes the preset resolution problem of the STFT by using a variable length window
• Analysis windows of different lengths are used for different frequencies:
• Analysis of high frequencies Use narrower windows for better time resolution
• Analysis of low frequencies  Use wider windows for better frequency resolution
• This works well, if the signal to be analyzed mainly consists of slowly varying characteristics with occasional short high frequency bursts.
• Heisenberg principle still holds!!!
• The function used to window the signal is called the wavelet

A normalization

constant

Translation parameter, measure of time

Scale parameter, measure of frequency

Signal to be analyzed

Continuous wavelet transform of the signal x(t) using the analysis wavelet (.)

The mother wavelet. All kernels are obtained by translating (shifting) and/or scaling the mother wavelet

Scale = 1/frequency

High frequency (small scale)

Low frequency (large scale)

• CWT computed by computers is really not CWT, it is a discretized version of the CWT.
• The resolution of the time-frequency grid can be controlled (within Heisenberg’s inequality), can be controlled by time and scale step sizes.
• Often this results in a very redundant representation
• How to discretize the continuous time-frequency plane, so that the representation is non-redundant?
• Sample the time-frequency plane on a dyadic (octave) grid

• Dyadic sampling of the time –frequency plane results in a very efficient algorithm for computing DWT:
• Subband coding using multiresolution analysis
• Dyadic sampling and multiresolution is achieved through a series of filtering and up/down sampling operations

H

x[n]

y[n]

x[n]

x[n]

~

G

2

2

2

2

2

2

2

2

2

2

~

~

G

H

G

G

+

+

~

H

H

H

Decomposition

Reconstruction

Down-sampling

Up-sampling

Half band high pass filter

Half band low pass filter

G

H

2-level DWT decomposition. The decomposition can be continues as long as there are enough samples for down-sampling.

|H(jw)|

w

/2

-/2

2

2

2

2

2

Length: 512

B: 0 ~ 

g[n]

h[n]

Length: 256

B: 0 ~ /2 Hz

Length: 256

B: /2 ~  Hz

a1

|G(jw)|

d1: Level 1 DWT

Coeff.

g[n]

h[n]

Length: 128

B: 0 ~  /4 Hz

w

Length: 128

B: /4 ~ /2 Hz

-/2

/2

-



a2

d2: Level 2 DWT

Coeff.

g[n]

h[n]

2

Length: 64

B: 0 ~ /8 Hz

Length: 64

B: /8 ~ /4 Hz

…a3….

d3: Level 3 DWT

Coeff.

Level 3 approximation

Coefficients

Choose wavelet

and number

of levels

Load signal

Hit Analyze

button

s=a5+d5+…+d1

Approx. coef.

at level 5

Level 1 coeff.

Highest freq.

(Wavedemo_signal1)

Applications of

Wavelets

• Compression
• De-noising
• Feature Extraction
• Discontinuity Detection
• Distribution Estimation
• Data analysis
• Biological data
• NDE data
• Financial data

• DWT is commonly used for compression, since most DWT are very small, can be zeroed-out!

First, analyze the signal with appropriate wavelets

Hit

Denoise

(Noisy Doppler)

Choose thresholding

method

Choose noise type

Choose thrsholds

Hit

Denoise

(microdisc.mat)

(microdisc.mat)

• Data Compression
• Wavelet Shrinkage Denoising
• Source and Channel Coding
• Biomedical Engineering
• EEG, ECG, EMG, etc analysis
• MRI
• Nondestructive Evaluation
• Ultrasonic data analysis for nuclear power plant pipe inspections
• Eddy current analysis for gas pipeline inspections
• Numerical Solution of PDEs
• Study of Distant Universes
• Galaxies form hierarchical structures at different scales

• Wavelet Networks
• Real time learning of unknown functions
• Learning from sparse data
• Turbulence Analysis
• Analysis of turbulent flow of low viscosity fluids flowing at high speeds
• Topographic Data Analysis
• Analysis of geo-topographic data for reconnaissance / object identification
• Fractals
• Daubechies wavelets: Perfect fit for analyzing fractals
• Financial Analysis
• Time series analysis for stock market predictions

• 1807, J.B. Fourier:
• All periodic functions can be expressed as a weighted sum of trigonometric function
• Denied publication by Lagrange, Legendre and Laplace
• 1822: Fourier’s work is finally published
• …
• …
• …
• …
• 1965, Cooley & Tukey: Fast Fourier Transform

143 years

• 1946, Gabor: STFT analysis:
• high frequency components using a narrow window, or
• low frequency components using a wide window, but not both
• Late 1970s, Morlet’s (geophysical engineer) problem:
• Time - frequency analysis of signals with high frequency components for short time spans and low frequency components with long time spans
• STFT can do one or the other, but not both Solution: Use different windowing functions for sections of the signal with different frequency content
• Windows to be generated from dilation / compression of prototype small, oscillatory signals  wavelets
• Criticism for lack of mathematical rigor !!!
• Early 1980s, Grossman (theoretical physicist): Formalize the transform and devise the inverse transformation  First wavelet transform !
• Rediscovery of Alberto Calderon’s 1964 work on harmonic analysis

• 1984, Yeves Meyer :
• Similarity between Morlet’s and Colderon’s work, 1984
• Redundancy in Morlet’s choice of basis functions
• 1985, Orthogonal wavelet basis functions with better time and frequency localization
• Rediscovery of J.O. Stromberg’s 1980 work the same basis functions (also a harmonic analyst)
• Yet re-rediscovery of Alfred Haar’s work on orthogonal basis functions, 1909 (!).
• Simplest known orthonormal wavelets

Ingrid Daubechies:

• Discretization of time and scale parameters of the wavelet transform
• Wavelet frames, 1986
• Orthonormal bases of compactly supported wavelets (Daubechies wavelets), 1988
• Liberty in the choice of basis functions at the expense of redundancy

Stephane Mallat:

• Multiresolution analysis w/ Meyer, 1986

Ph.D. dissertation, 1988

• Discrete wavelet transform
• Cascade algorithm for computing DWT

• Decomposition of a discrete into dyadic frequencies (MRA) , known to EEs under the name of “Quadrature Mirror Filters”, Croisier, Esteban and Galand, 1976 (!)

Martin Vetterli & Jelena Kovacevic

• Wavelets and filter banks, 1986
• Perfect reconstruction of signals using FIR filter banks, 1988
• Subband coding
• Multidimensional filter banks, 1992

• Equivalence of QMF and MRA, Albert Cohen, 1990
• Compactly supported biorthogonal wavelets, Cohen, Daubechies, J. Feauveau, 1993
• Wavelet packets, Coifman, Meyer, and Wickerhauser, 1996
• Zero Tree Coding, Schapiro 1993 ~ 1999
• Search for new wavelets with better time and frequency localization properties.
• Super-wavelets
• Matching Pursuit, Mallat, 1993 ~ 1999

• Zero crossing representation
• signal classification
• computer vision
• data compression
• denoising
• Super wavelet
• Linear combination of known basic wavelets
• Zero Tree Coding, Schapiro
• Matching Pursuit , Mallat
• Using a library of basis functions for decomposition
• New MPEG standard

The Story of Wavelets