A New Method of Probability Density Estimation for Mutual Information Based Image Registration

1. A New Method of Probability Density Estimation for Mutual Information Based Image Registration Ajit Rajwade, Arunava Banerjee, Anand Rangarajan. Dept. of Computer and Information Sciences & Engineering, University of Florida.

2. Image Registration: problem definition • Given two images of an object, to find the geometric transformation that “best” aligns one with the other, w.r.t. some image similarity measure.

3. Mutual Information for Image Registration • Mutual Information (MI) is a well known image similarity measure ([Viola95], [Maes97]). • Insensitive to illumination changes; useful in multimodality image registration.

4. Mathematical Definition for MI Marginal entropy Joint entropy Conditional entropy

5. Calculation of MI • Entropies calculated as follows: Marginal Probabilities Joint Probability

6. Joint Probability Functions of Geometric Transformation

7. Estimating probability distributions Histograms How do we select bin width? Too large bin width: Over-smooth distribution Too small bin width: Sparse, noisy distribution

8. Estimating probability distributions Parzen Windows Choice of kernel Choice of kernel width Too small: Noisy, spiky Too large: Over-smoothing

9. Estimating probability distributions Mixture Models [Leventon98] How many components? Local optima Difficult optimization in every step of registration.

10. Direct (Renyi) entropy estimation Minimal Spanning Trees, Entropic kNN Graphs [Ma00, Costa03] Requires creation of MST from complete graph of all samples

11. Cumulative Distributions Entropy defined on cumulatives [Wang03] Extremely Robust, Differentiable

12. A New Method What’s common to all previous approaches? Obtain approximation to the density Take samples More accurate approximation More samples

13. A New Method Uncountable infinity of samples taken Assume uniform distribution on location Transformation Location Intensity Each point in the continuum contributes to intensity distribution Distribution on intensity Image-Based

14. Other Previous Work • A similar approach presented in [Kadir05]. • Does not detail the case of joint density of multiple images. • Does not detail the case of singularities in density estimates. • Applied to segmentation and not registration.

15. A New Method Continuous image representation (use some interpolation scheme) No pixels! Trace out iso-intensity level curves of the image at several intensity values.

17. Analytical Formulation: Marginal Density • Marginal density expression for image I(x,y) of area A: • Relation between density and local image gradient (u is the direction tangent to the level curve):

18. Joint Probability

19. Joint Probability

20. Analytical Formulation: Joint Density • The joint density of images and with area of overlap A is related to the area of intersection of the regions between level curves at and of , and at and of as . • Relation to local image gradients and the angle between them ( and are the level curve tangent vectors in the two images):

21. Practical Issues • Marginal density diverges to infinity, in areas of zero gradient (level curve does not exist!). • Joint density diverges • in areas of zero gradient of either or both • image(s). • in areas where gradient vectors of the • two images are parallel.

22. Work-around • Switch from densities (infinitesimal bin width) to distributions (finite bin width). • That is, switch from an analytical to a computational procedure.

23. Binning without the binning problem! More bins = more (and closer) level curves. Choose as many bins as desired.

24. 1024 bins 128 bins 256 bins 512 bins 32 bins 64 bins Standard histograms Our Method

25. Pathological Case: regions in 2D space where both images have constant intensity

26. Pathological Case: regions in 2D space where only one image has constant intensity

27. Pathological Case: regions in 2D space where gradients from the two images run locally parallel

28. Registration Experiments: Single Rotation • Registration between a face image and its 15 degree rotated version with noise of variance 0.1 (on a scale of 0 to 1). • Optimal transformation obtained by a brute-force search for the maximum of MI. • Tried on a varied number of histogram bins.

29. MI Trajectory versus rotation: noise variance 0.1 Our Method Standard Histograms 128 bins 16 bins 32 bins 64 bins

30. MI Trajectory versus rotation: noise variance 0.8 Our Method Standard Histograms 128 bins 16 bins 32 bins 64 bins

31. Affine Image Registration BRAINWEB PD slice Warped and Noisy T2 slice Warped T2 slice T2 slice Brute force search for the maximum of MI

32. Affine Image Registration MI with standard histograms MI with our method

33. Directions for Future Work • Our distribution estimates are not differentiable as we use a computational (not analytical) procedure. • Differentiability required for non-rigid registration of images.

34. Directions for Future Work • Simultaneous registration of multiple images: efficient high dimensional density estimation and entropy calculation. • 3D Datasets.

35. References • [Viola95]“Alignment by maximization of mutual information”, P. Viola and W. M. Wells III, IJCV 1997. • [Maes97]“Multimodality image registration by maximization of mutual information”, F. Maes, A. Collignon et al, IEEE TMI, 1997. • [Wang03] “A new & robust information theoretic measure and its application to image alignment”, F. Wang, B. Vemuri, M. Rao & Y. Chen, IPMI 2003. • [BRAINWEB]http://www.bic.mni.mcgill.ca/brainweb/

36. References • [Ma00]“Image registration with minimum spanning tree algorithm”, B. Ma, A. Hero et al, ICIP 2000. • [Costa03]“Entropic graphs for manifold learning”, J. Costa & A. Hero, IEEE Asilomar Conference on Signals, Systems and Computers 2003. • [Leventon98]“Multi-modal volume registration using joint intensity distributions”, M. Leventon & E. Grimson, MICCAI 98. • [Kadir05] “Estimating statistics in arbitrary regions of interest”, T. Kadir & M. Brady, BMVC 2005.

37. Acknowledgements • NSF IIS 0307712 • NIH 2 R01 NS046812-04A2.

38. Questions??