Digital Image Processing

1. Digital Image Processing 4. Frequency Domain transformations Digital Image Processing - Lecturer: Prof. Alexander Bronstein, TA: Maria Tunik-Schmidt

2. Session 1 Linear Algebra Basics: • Notation (how do we present things in our world) – scalar, vector matrix. • Linear and Affine spaces. • Vector Inner Product • Vector Norm • The Euclidian Norm • The Lp Norms Family • Matrices Operations • Eigen - Values and Eigen - Vectors • Eigen – Decomposition

3. Session 2 • Linear Algebra: • Quadratic Form Eigen values and Eigen vectors • Multivariate Differential Calculus: • Functions • Derivatives • Directional Derivatives • Gradients

4. Session 3 • Digital Filters: • Image Representation • Convolution • Linear Filtering

5. Session 4 Frequency domain operations: • FREQUENCY AND SPACE DOMAINS • DFT • FFT • APPLICATION IN IMAGE PROCESSING

6. An image is a function in the spatial domain • An image I is represented as a function of location I(x,y), where x and y are integers. Thus an image I(x,y) is a matrix of pixels. • Gray-scale images - 2D-matrix with a measurement of gray-scale intensity. RGB color images or a 3D vector of 2D matrices of red/green/blue intensity. • You can change the image if you change the magnitude of intensity at any point in the plane. • This state of 2D matrices that depict the intensity is called Spatial Domain.

7. Every image can be described and represented as a function You can look at it as a discrete measurements (the intensity) that you measure on fixed intervals of an image “Pixels” intervals.

8. Function in Frequency domain • Every Function equals a sum of it’s underlying frequency components – sines and cosines (sinusoids) => That is called the Fourier Series • Any signal can be written as the sum of phase-shifted sines and cosines of different frequencies.

9. The Fourier series • Any function that periodically repeats itself can be expressed as a sum of sines and cosines of different frequencies each multiplied by a different coefficient • The Fourier series breaks down a periodic function into the sum of sinusoidal functions.

10. Example • For example: Square wave as a sum of • Sine waves. Any signal decomposed into sum of signals with different frequencies

11. Fourier Transform • Fourier analysis lets us determine the frequency content of an image • A way to represent a function as the sum of those frequency components (Fourier Series) is the Fourier transform. • The Fourier transform decomposes any periodic function or periodic signal from the spatial (x) domain to the frequency (u) domain by breaking it into the sum (possibly infinite) of it’s frequency components - a set of simple oscillating functions, namely sines and cosines. • Transforms a signal (i.e., function) from the spatial (x) domain to the frequency (u) domain (or equivalently, complex exponentials because)

12. Complex Numbers • According to Euler's formula for any real number x, • A point in a “Complex Plain” / Any complex number • z = x + iyand its complex conjugate, z = x − iy • can be represented by a complex number • Can be written as

13. Time/Space & Frequency domains • We call the original “Image” / Domain – The space domain • And the signs/ cosines domain – the frequency domain. • The result of performing the desired operations • On Any function in the frequency domain, • Can be transformed back to the space domain.

14. Continuous Fourier Transform (FT) in 1-D • Transforms a continuoussignal (i.e., function) • From the spatial (x) domain • To the frequency (u) domain.

15. Continuous Functions FT in 2-D

16. Do you remember filtering? • Neighborhood (area-based) operators, where each new pixel’s value depends on a small number of neighboring input values. • To filter an image in the spatial domain a Convolution operation was required. • A convenient tool to analyze (and sometimes accelerate) such neighborhood operations is the Fourier Transform when the function is transformed to the frequency domain.

17. Frequency Domain Filtering & Spatial Domain Filtering • Similar jobs can be done in the spatial and frequency domains • Filtering in the spatial domain can be easier to understand • Filtering in the frequency domain can be much faster – especially for large image • The operation of differentiation in the space domain corresponds to multiplication by the frequency. so some differential equations are easier to analyze in the frequency domain. • Also, convolution in the space domain corresponds to ordinary multiplication in the frequency domain.

18. The Discrete Fourier Transform (DFT) • As we talk about images and pixels, to get to the frequency domain we use the DFT. • The Discrete Fourier Transform of f(x, y), for: • x = 0, 1, 2…M-1 • y = 0,1,2…N-1, • denoted by F(u, v) , u = 0, 1, 2…M-1 , v = 0, 1, 2…N-1. • is given by the equation:

19. Fast Fourier Transform • Is a mathematically efficient way to perform Discrete Fourier transform. • Computation for direct DFT is O (N^2) • Computation of FFT is O(N*Log2N) • The basic idea is to break up transform of length N to 2 transforms of N/2 based on the equation: Using the Fast Fourier Transform we can perform large-kernel convolutions in time that is independent of the kernel’s size.

20. The Inverse Discrete Fourier Transform • As we talk about images and pixels, to get to the frequency domain we use the DFT. • The Discrete Fourier Transform of f(x, y), for: • x = 0, 1, 2…M-1 • y = 0,1,2…N-1, • denoted by F(u, v) , u = 0, 1, 2…M-1 , v = 0, 1, 2…N-1. • is given by the equation:

21. Special Case – N*N Image

22. Main Uses of DFT • Easier to remove undesirable frequencies. To remove certain frequencies, set their corresponding F(u) coefficients to zero! • Faster performance for certain operations in the frequency domain than in the spatial domain.

23. How frequencies show up in an image? • Low frequencies correspond to slowly varying information (e.g., continuous surface). • High frequencies correspond to quickly varying information (e.g., edges)

24. DFT and Images • The DFT of a two dimensional image can be visualized by showing the spectrum of the images component frequencies Usually we visualize the magnitude of the frequency (spectrum). DFT each point represents a particular frequency contained in the spatial domain image

25. To Process an Image with DFT 1. To filter an image in the frequency domain: Compute the F(u,v) matrix of the DFT of the image. 2. Then filter it: Multiply F(u,v) by a filter function H(u,v) 3. Inverse the filtered result to get the image after filtering – Compute the inverse DFT of the result

26. Smoothing In the Frequency Domain (LPF) • Smoothing is achieved in the frequency domain by LOW PASS FILTERS that are dropping out the high frequency components . • In the frequency domain we can easily through the high frequencies… (just a multiplication) • The basic model for filtering is: The Filter Transform Function The Fourier Transform of the image Low pass filters – only pass the low frequencies, drop the high ones.

27. Sharpening in the Frequency Domain • Edges and fine detail in images are achieved in the frequency domain are associated with high frequency components and achieved by HIGH PASS FILTERS that are dropping out the LOW frequency components . • High pass filters are precisely the reverse of low pass filters, so

28. If we’ll try to visualize it The image in the special domain - The original signal is a rectangular pulse with added noise. 1. Rectangular Pulse 2. White Noise

29. In the frequency domain • We take every pixel value in the spatial domain and transform it to it’s value in the frequency domain (u,v). • 1. The Fourier transform of the rectangular pulse is the two dimensional equivalent of the sync function • 2. The Fourier transform of white noise is a constant. 1. Rectangular Pulse 2. White Noise

30. Low pass filter in the frequency domain Applying a high pass filter in a frequency domain is the opposite to the low pass filter. All the frequencies above some cut-off radius are removed. It means zeroing all frequency components above a cut-off frequency. Which means removing the high frequency components. This is similar to what one would do in a 1 dimensional case except now the ideal filter is a cylindrical "can" instead of a rectangular pulse.

31. Back to the original Image • The result transformed back into the spatial domain. • The rectangular pulse is "rounded“ or more “Smooth” since high frequency components were required for the transition. This is an explanation to why High pass filters are used to “Smooth” images.

32. Additional Low pass Filtering Examples A low pass Gaussian filter is used to connect broken text

33. LPF #1 - Ideal Low Pass Filter

34. Low Pass Filter – Blurring

35. The Problem - Ringing

36. LPF #2 - Butterworth Filter

37. LPF #3 - Gaussian Filter

38. Visual LPF Comparison

39. High pass filter in the frequency domain Applying a high pass filter in a frequency domain is the opposite to the low pass filter. All the frequencies below some cut-off radius are removed.

40. Back to the special domain Transformed back into the spatial domain shows that the noise field is retained as well as the transitions (edges) of the rectangular pulse. This is how the high pass filter is used to sharpen the images.

41. HPF #1 - Gaussian Filter

42. HPF #2 - Butterworth Filter

43. LPF #3 - Gaussian Filter

44. HPF - Comparison

45. High Frequency Emphasis Maintain Low Frequencies and Mean + Emphasize high frequencies!

46. Comparison of HPF

47. Example – Convolution in Space Vs. Frequency • Board Example

48. Thank you! Digital Image Processing - Lecturer: Prof. Alexander Bronstein, TA: Maria Tunik-Schmidt