Smoothing Techniques in Image Processing

1. Smoothing Techniques in Image Processing Prof. PhD. Vasile Gui Polytechnic University of Timisoara

2. Content • Introduction • Brief review of linear operators • Linear image smoothing techniques • Nonlinear image smoothing techniques

3. Introduction Why do we need image smoothing? What is “image” and what is “noise”? • Frequency spectrum • Statistical properties

4. Brief Review of Linear Operators[Pratt 1991] • Generalized 2D linear operator • Separable linear operator: • Space invariant operator: Convolution sum

5. h22 h20 h21 L-1 h12 h11 h10 2 L-1 h02 h01 h00 h22 h20 h21 H 2 h12 h11 h10 h02 h01 h00 L-1 h22 G h20 h21 2 L-1 h12 h11 h10 2 h02 h01 h00 N-L+1 F N N+L-1 h22 h20 h21 h12 h11 h10 h22 h20 h21 h02 h12 h01 h00 h11 h10 h02 h01 h00 Brief Review of Linear Operators • Geometrical interpretation of 2D convolution

6. Brief Review of Linear Operators • Matrix formulation • Unitary transform • Separable transform • Transform is invertible

7. Brief Review of Linear Operators • Basis vectors are orthogonal • Inner product and energy are conserved • Unitary transform: a rotation in N-dimensional vector space

8. x2 2 1 x1 Brief Review of Linear Operators • 2D vector space interpretation of a unitary transform

9. Brief Review of Linear Operators • DFT unitary matrix

10. Brief Review of Linear Operators • 1D DFT • 2D DFT

11. Brief Review of Linear Operators • Circular convolution theorem • Periodicity of f,h and g with N are assumed

12. Linear Image smoothing techniquesBox filters. Arithmetic mean LL operator

13. m,n  + L pixels m,n+1 Linear Image smoothing techniquesBox filters. Arithmetic mean LL operator • Separable • Can be computed recursively, resulting in roughly 4 operations per pixel

14. Linear Image smoothing techniquesBox filters. Arithmetic mean LL operator Optimality properties • Signal and additive white noise • Noise variance is reduced N times

15. Linear Image smoothing techniquesBox filters. Arithmetic mean LL operator • Unknown constant signal plus noise • Minimize MSE of the estimation g:

16. Linear Image smoothing techniquesBox filters. Arithmetic mean LL operator • i.i.d. Gaussian signal with unknown mean. • Given the observed samples, maximize • Optimal solution: arithmetic mean

17. Linear Image smoothing techniquesBox filters. Arithmetic mean LL operator • Frequency response: Non-monotonically decreasing with frequency

18. Linear Image smoothing techniquesBox filters. Arithmetic mean LL operator • An example

19. Linear Image smoothing techniquesBox filters. Arithmetic mean LL operator • Image smoothed with 33, 55, 99 and 11 11 box filters

20. Linear Image smoothing techniquesBox filters. Arithmetic mean LL operator Lena image filtered with 5x5 box filter Original Lena image

21. Linear Image smoothing techniquesBinomial filters [Jahne 1995] • Computes a weighted average of pixels in the window • Less blurring, less noise cleaning for the same size • The family of binomial filters can be defined recursively • The coefficients can be found from (1+x)n

22. Linear Image smoothing techniquesBinomial filters. 1D versions As size increases, the shape of the filter is closer to a Gaussian one

23. Linear Image smoothing techniquesBinomial filters. 2D versions

24. Linear Image smoothing techniquesBinomial filters. Frequency response Monotonically decreasing with frequency

25. Linear Image smoothing techniquesBinomial filters. Example Original Lena image Lena image filtered with binomial 5x5 kernel Lena image filtered with box filter 5x5

26. Step edge Result of size L box filter Size L binomial L Linear Image smoothing techniquesBinomial and box filters. Edge blurring comparison • Linear filters have to compromise smoothing with edge blurring

27. f(1) f1 f(2) f2 . . . f(m) . . . Order samples select f(N) fN median Nonlinear image smoothingThe median filter [Pratt 1991] • Block diagram N is odd

28. 3 7 8 2 3 7 4 6 7 2, 3, 3, 4, 6, 7, 7, 7, 8 6 Nonlinear image smoothingThe median filter • Numerical example

29. Nonlinear image smoothingThe median filter • Nonlinearity median{f1 + f2}  median{ f1} + median{ f2}. However: median{ c f} = c median{ f }, median{ c + f} = c + median{ f}. • The filter selects a sample from the window, does not average • Edges are better preserved than with liner filters • Best suited for “salt and pepper” noise

30. Nonlinear image smoothingThe median filter Noisy image 5x5 median filtered 5x5 box filter

31. Nonlinear image smoothingThe median filter • Optimality • Grey level plateau plus noise. Minimize sum of absolute differences: • Result: • If 51% of samples are correct and 49% outliers, the median still finds the right level!

32. Test images Results of 33 pixel median filter Nonlinear image smoothingThe median filter • Caution: points, thin lines and corners are erased by the median filter

33. Results of the 9 pixel cross shaped window median filter Nonlinear image smoothingThe median filter • Cross shaped window can correct some of the problems

34. Nonlinear image smoothingThe median filter • Implementing the median filter • Sorting needs O(N2) comparisons • Bubble sort • Quick sort • Huang algorithm (based on histogram) • VLSI median

35. x1 x(1) x2 x(2) x3 x(3) a Min(a,b) x4 x(4) b Max(a,b) x5 x(5) x6 x(6) x7 x(7) Nonlinear image smoothingThe median filter • VLSI median block diagram

36. Nonlinear image smoothingThe median filter • Color median filter • There is no natural ordering in 3D (RGB) color space • Separate filtering on R,G and B components does not guarantee that the median selects a true sample from the input window • Vector median filter, defined as the sample minimizing the sum of absolute deviations from all the samples • Computing the vector median is very time consuming, although several fast algorithms exist

37. Nonlinear image smoothingThe median filter • Example of color median filtering • 5x5 pixels window • Up: original image • Down: filtered image

38. Nonlinear image smoothingThe weighted median filter • The basic idea is to give higher weight to some samples, according to their position with respect to the center of the window • Each sample is given a weight according to its spatial position in the window. • Weights are defined by a weighting mask • Weighted samples are ordered as usually • The weighted median is the sample in the ordered array such that neither all smaller samples nor all higher samples can cumulate more than 50% of weights. • If weights are integers, they specify how many times a sample is replicated in the ordered array

39. 1 2 1 2 2 3 1 2 1 3 7 8 2 3 7 6 4 6 7 2, 2, 3, 3, 3, 3, 4, 6, 6, 7, 7, 7, 7, 7, 8 Nonlinear image smoothingThe weighted median filter • Numerical example for the weighted median filter

40. R3 R2 R4       m5      R1       mi = median( Ri ), i =1,2,3,4. Result = median( m1,m2,m3,m4,m5 ) Nonlinear image smoothingThe multi-stage median filter • Better detail preservation

41. weights f(1) f1 a1 f(2) f2 a2 y . . . . . . ordering sum f(N) fN aN y = a1f(1) + a2f(2)+...+ aNf(N) Nonlinear image smoothingRank-order filters (L filters) • Block diagram

42. a1=1 . . . Min aN=1 . . . Max am=1 . . . . . . Median a1=.5 aN=.5 . . . Mid range Nonlinear image smoothingRank-order filters (L filters) • Some examples Percentile filters (rank selection): 0%, 50% 100% Particular cases of grey level morphological filters

43. Nonlinear image smoothingRank-order filters (L filters) • Optimality Minkovski distance Case r = 1: best estimator ismedian. Caser = 2, best estimator is arithmetic mean. Caser , best estimator is mid-range.

44. Nonlinear image smoothingRank-order filters (L filters) • Alpha-trimmed mean filter •  = (2Q + 1) / N, defines de degree of averaging • = 1 corresponds to arithmetic mean  = 1 / N corresponds to the median filter Properties: in between mean and median

45. Nonlinear image smoothingRank-order filters (L filters) Median of absolute differences trimmed mean • Better smoothing than the median filter and good edge preservation M2 is a robust estimator of the variance

46. Nonlinear image smoothingRank-order filters (L filters) • k nearest neighbour (kNN) median filter • Median of the k grey values nearest by rank to the central pixel • Aim: same as above

47. reg 2 reg 1 reg 4 reg 3 reg 6 reg 5 reg 8 reg 7 Nonlinear image smoothingSelected area filtering [Nagao 1980] • Kuwahara type filtering • Form regions Ri • Compute mean, mi and variance, Si for each region. • Result = mi : mi < mj for all j different from i, i,e. themean of the most homogeneous region • Matsujama & Nagao

48. Nonlinear image smoothingConditional mean • Pixels in a neighbourhood are averaged only if they differ from the central pixel by less than a given threshold: L is a spacescale parameter and th is a range scale parameter

49. Nonlinear image smoothingConditional mean • Example with L=3, th=32

50. Nonlinear image smoothingBilateral filter [Tomasi 1998] • Space and range are treated in a similar way • Space and range similarity is required for the averaged pixels • Tomasi and Manduchi [1998] introduced soft weights to penalize the space and range dissimilarity. s() and r() are space and range similarity functions (Gaussian functions of the Euclidian distance between their arguments).