Skeleton of Presentation

2. Skeleton of Presentation • Probability density function (pdf) estimation using isocontours/isosurfaces • Application to Image Registration • Application to Image Filtering • Circular/spherical density estimation in Euclidean space

3. PDF Estimation: Contemporary techniques Kernel density estimate Histograms Mixture model Parameter selection: bin-width/bandwidth/number of components Bias/variance tradeoff: large bandwidth: high bias, low bandwidth: high variance) Sample-based methods Do not treat a signal as a signal

4. New approach: isocontours Trace out isocontours of the intensity function I(x,y) at several intensity values. Continuous image representation: using some interpolant.

6. New approach: isocontours Assume a uniform density on (x,y) Random variable transformation from (x,y) to (I,u) Every point in the image domain contributes to the density. u = direction along the level set (dummy variable) Integrate out u to get the density of intensity I Published in CVPR 2006, PAMI 2009.

7. Joint density: isocontours

8. Joint density: isocontours Relationships between geometric and probabilistic entities.

9. Related work • Similar density estimator developed by Kadir and Brady (BMVC 2005) independently of us. • Similar idea: several differences in implementation, motivation, derivation of results and applications.

10. Densities may not exist: distributions do. Densities (derivatives of the cumulative) do not exist where image gradients are zero, or where image gradients run parallel. Compute cumulative interval measures.

11. Computational issues: No binning problem

12. 1024 bins 128 bins 256 bins 512 bins 32 bins 64 bins Standard histograms Isocontour Method

14. Histogramming with upsampling? • Randomized/digital approximation to area calculation. • Strict lower bound on the accuracy of the isocontour method, for a fixed interpolant. • Computationally more expensive than the isocontour method.

15. Histogramming with Random upsampling? 128 x 128 bins

16. Choice of interpolant? • Simplest one: linear interpolant to each half-pixel (level curves are segments). • Low-order polynomial interpolants: high bias, low variance. • High-order polynomial interpolants: low bias, high variance.

17. Choice of interpolant? Accuracy of estimated density improves as signal is sampled with finer resolution. Polynomial Interpolant Bandlimited analog signal, Nyquist-sampled digital signal: Accurate reconstruction by sincinterpolant! (Whitaker-Shannon Sampling Theorem) Assumptions on signal: better interpolant

18. Skeleton of Presentation • Probability density function (pdf) estimation using isocontours • Application to Image Registration • Application to Image Filtering • Circular/spherical density estimation in Euclidean space

19. Image Registration: Problem Definition Mutual Information: Well known image similarity measure Viola and Wells (IJCV 1995) and Maes et al (TMI 1997). • Given two images of an object, to find the geometric transformation that “best” aligns one with the other, w.r.t. some image similarity measure. Insensitive to illumination changes: useful in multimodality image registration

20. Marginal Probabilities Joint Probability Marginal entropy Joint entropy Conditional entropy

21. Hypothesis: If the alignment between images is optimal then Mutual information is maximum. Functions of Geometric Transformation

22. Registration experiments: Rotation

23. 32 bins 128 bins PVI=partial volume interpolation (Maes et al, TMI 1997)

24. Experiments: Affine registration PD slice Warped T2 slice T2 slice Warped and Noisy T2 slice Brute force search for the maximum of MI

25. MI with standard histograms MI with our method Par. of affine transf. 25

26. 32 BINS

27. Skeleton of Presentation • Probability density function (pdf) estimation using isocontours • Application to Image Registration • Application to Image Filtering • Circular/spherical density estimation in Euclidean space

28. Anisotropic neighborhood filters (Kernel density based filters): Grayscale images K: a decreasing function (typically Gaussian) Parameter σ controls the degree of anisotropicity of the smoothing Central Pixel (a,b): Neighborhood N(a,b) around (a,b)

29. Anisotropic Neighborhood filters: Problems Sensitivity to the SIZE of the Neighborhood Sensitivity to the parameter Does not account for gradient information

30. Anisotropic Neighborhood filters: Problems Treat pixels as independent samples

31. Continuous Image Representation Interpolate in between the pixel values

32. Continuous Image Representation Areas between isocontours at intensity α and α+Δ (divided by area of neighborhood)= Pr(α < Intensity < α+Δ|N(a,b))

33. Areas between isocontours: contribute to weights for averaging. Published in EMMCVPR 2009

34. Extension to RGB images Joint Probability of R,G,B = Area of overlap of isocontour pairs from R, G, B images

35. Mean-shift framework • A clustering method developed by Fukunaga & Hostetler (IEEE Trans. Inf. Theory, 1975). • Applied to image filtering by Comaniciu and Meer (PAMI 2003). • Involves independent update of each pixel by maximization of local estimate of probability density of joint spatial and intensity parameters.

36. Mean-shift framework • One step of mean-shift update around (a,b,c) where c=I(a,b).

37. Our Method in Mean-shift Setting Y(x,y)=y X(x,y)=x I(x,y)

38. Our Method in Mean-shift Setting Facets of tessellation induced by isocontours and the pixel grid = Centroid of Facet #k. = Intensity (from interpolated image) at . = Area of Facet #k.

39. Experimental Setup: Grayscale Images • Piecewise-linear interpolation used for our method in all experiments. • For our method, Kernel K = pillbox kernel, i.e. • For discrete mean-shift, Kernel K = Gaussian. • Parameters used: neighborhood radius ρ=3, σ=3. • Noise model: Gaussian noise of variance 0.003 (scale of 0 to 1). If |z| <= σ If |z| > σ

40. Original Image Noisy Image Denoised (Isocontour Mean Shift) Denoised (Gaussian Kernel Mean Shift)

41. Denoised (Isocontour Mean Shift) Denoised (Std.Mean Shift) Original Image Noisy Image

42. Experiments on color images • Use of pillbox kernels for our method. • Use of Gaussian kernels for discrete mean shift. • Parameters used: neighborhood radius ρ= 6, σ = 6. • Noise model: Independent Gaussian noise on each channel with variance 0.003 (on a scale of 0 to 1).

43. Experiments on color images • Independent piecewise-linear interpolation on R,G,B channels in our method. • Smoothing of R, G, B values done by coupled updates using joint probabilities.

44. Original Image Noisy Image Denoised (Isocontour Mean Shift) Denoised (Gaussian Kernel Mean Shift)

45. Denoised (Isocontour Mean Shift) Denoised (Gaussian Kernel Mean Shift) Original Image Noisy Image

46. Observations • Discrete kernel mean shift performs poorly with small neighborhoods and small values of σ. • Why? Small sample-size problem for kernel density estimation. • Isocontour based method performs well even in this scenario (number of isocontours/facets >> number of pixels). • Large σ or large neighborhood values not always necessary for smoothing.

47. Observations • Superior behavior observed when comparing isocontour-based neighborhood filters with standard neighborhood filters for the same parameter set and the same number of iterations.

48. Skeleton of Presentation • Probability density function (pdf) estimation using isocontours • Application to Image Registration • Application to Image Filtering • Circular/spherical density estimation in Euclidean space

49. Circular/spherical density estimation in Euclidean space. • Examples of unit vector data: 1. Chromaticity vectors of color values: 2. Hue (from the HSI color scheme) obtained from the RGB values.

50. Conventional approach Convert RGB values to unit vectors Estimate density of unit vectors voMF mixture models Banerjee et al (JMLR 2005) Other popular kernels: Watson, cosine.