Mean Shift A Robust Approach to Feature Space Analysis

1. Mean ShiftA Robust Approach toFeature Space Analysis Kalyan Sunkavalli 04/29/2008 ES251R

2. An Example Feature Space

3. An Example Feature Space

4. An Example Feature Space Parametric Density Estimation?

5. Mean Shift • A non-parametric technique for analyzing complex multimodal feature spaces and estimating the stationary points (modes) of the underlying probability density function without explicitly estimating it.

6. Outline • Mean Shift • An intuition • Kernel Density Estimation • Derivation • Properties • Applications of Mean Shift • Discontinuity preserving Smoothing • Image Segmentation

7. Outline • Mean Shift • An intuition • Kernel Density Estimation • Derivation • Properties • Applications of Mean Shift • Discontinuity preserving Smoothing • Image Segmentation

8. Region of interest Intuitive Description Center of mass Mean Shift vector Slide Credit: Yaron Ukrainitz & Bernard Sarel Objective : Find the densest region Distribution of identical billiard balls

9. Region of interest Intuitive Description Center of mass Mean Shift vector Objective : Find the densest region Distribution of identical billiard balls

10. Region of interest Intuitive Description Center of mass Mean Shift vector Objective : Find the densest region Distribution of identical billiard balls

11. Region of interest Intuitive Description Center of mass Mean Shift vector Objective : Find the densest region Distribution of identical billiard balls

12. Region of interest Intuitive Description Center of mass Mean Shift vector Objective : Find the densest region Distribution of identical billiard balls

13. Region of interest Intuitive Description Center of mass Mean Shift vector Objective : Find the densest region Distribution of identical billiard balls

14. Region of interest Intuitive Description Center of mass Objective : Find the densest region Distribution of identical billiard balls

15. Outline • Mean Shift • An intuition • Kernel Density Estimation • Derivation • Properties • Applications of Mean Shift • Discontinuity preserving Smoothing • Image Segmentation

16. Parametric Density Estimation The data points are sampled from an underlying PDF Estimate from data Assumed Underlying PDF Data Samples

17. Non-parametric Density Estimation PDF value Data point density Assumed Underlying PDF Data Samples

18. Non-parametric Density Estimation Assumed Underlying PDF Data Samples

19. Parzen Windows Kernel Properties • Bounded • Compact support • Normalized • Symmetric • Exponential decay

20. Kernels and Bandwidths • Kernel Types • Bandwidth Parameter (product of univariate kernels) (radially symmetric kernel)

21. Various Kernels Epanechnikov Normal Uniform

22. Outline • Mean Shift • An intuition • Kernel Density Estimation • Derivation • Properties • Applications of Mean Shift • Discontinuity preserving Smoothing • Image Segmentation

23. Density Gradient Estimation Modes of the probability density Epanechnikov  Uniform Normal  Normal

24. Mean Shift KDE Mean Shift Mean Shift Algorithm • compute mean shift vector • translate kernel (window) by mean shift vector

25. Mean Shift • Mean Shift is proportional to the normalized density gradient estimate obtained with kernel • The normalization is by the density estimate computed with kernel

26. Outline • Mean Shift • An intuition • Kernel Density Estimation • Derivation • Properties • Applications of Mean Shift • Discontinuity preserving Smoothing • Image Segmentation

27. Properties of Mean Shift • Guaranteed convergence • Gradient Ascent algorithms are guaranteed to converge only for infinitesimal steps. • The normalization of the mean shift vector ensures that it converges. • Large magnitude in low-density regions, refined steps near local maxima  Adaptive Gradient Ascent. • Mode Detection • Let denote the sequence of kernel locations. • At convergence • Once gets sufficiently close to a mode of it will converge to the mode. • The set of all locations that converge to the same mode define the basin of attraction of that mode.

28. Properties of Mean Shift • Smooth Trajectory • The angle between two consecutive mean shift vectors computed using the normal kernel is always less that 90° • In practice the convergence of mean shift using the normal kernel is very slow and typically the uniform kernel is used.

29. Mode detection using Mean Shift • Run Mean Shift to find the stationary points • To detect multiple modes, run in parallel starting with initializations covering the entire feature space. • Prune the stationary points by retaining local maxima • Merge modes at a distance of less than the bandwidth. • Clustering from the modes • The basin of attraction of each mode delineates a cluster of arbitrary shape.

30. Mode Finding on Real Data initialization tracks detected mode

31. Mean Shift Clustering

32. Outline • Mean Shift • Density Estimation • What is mean shift? • Derivation • Properties • Applications of Mean Shift • Discontinuity preserving Smoothing • Image Segmentation

33. Joint Spatial-Range Feature Space • Concatenate spatial and range (gray level or color) information

34. Discontinuity Preserving Smoothing

35. Discontinuity Preserving Smoothing

36. Discontinuity Preserving Smoothing

37. Discontinuity Preserving Smoothing

38. Outline • Mean Shift • Density Estimation • What is mean shift? • Derivation • Properties • Applications of Mean Shift • Discontinuity preserving Smoothing • Image Segmentation

39. Clustering on Real Data

40. Image Segmentation

41. Image Segmentation

42. Image Segmentation

43. Image Segmentation

44. Image Segmentation

45. Acknowledgements • Mean shift: A robust approach toward feature space analysis. D Comaniciu, P Meer Pattern Analysis and Machine Intelligence, IEEE Transactions on, Vol. 24, No. 5. (2002), pp. 603-619. • http://www.caip.rutgers.edu/riul/research/papers.html • Slide credits: Yaron Ukrainitz & Bernard Sarel

46. Thank You