Medical Image Registration

1. Medical Image Registration Yujun Guo Dept.of CS Kent State University

2. Outline • Why registration • Registration basics • Rigid registration • Non-rigid registration • Applications

3. Modalities in Medical Image • Computed Tomography (CT), Magnetic Resonance (MR) imaging, Ultrasound, and X-ray give anatomic information. • Positron Emission Tomography (PET) and Single Photon Emission CT (SPECT) give functional information.

4. Registration • Monomodality: • A series of same modality images (CT/CT, MR/MR, Mammogram pairs,…). • Images may be acquired weeks or months apart; taken from different viewpoints. • Aligning images in order to detect subtle changes in intensity or shape • Multimodality: • Complementary anatomic and functional information from multiple modalities can be obtained for the precise diagnosis and treatment. • Examples:PET and SPECT (low resolution, functional information) need MR or CT (high resolution, anatomical information) to get structure information.

5. q = (912,632) q = T(p;a) p = (825,856) Pixel location in first image Homologous pixel location in second image Pixel location mapping function Registration Problem Definition

6. Example Mapping Function q = (912,632) p = (825,856) Pixel scaling and translation

7. Image Registration • Define a transform T that will map one image onto another image of the same object such that some image quality criterion is maximized. • A mapping between two images both spatially and with respect to intensity • I2 = g (T(I1))

8. Registration Scheme

9. Components • Feature Space • Search Space or transformation • Similarity Metric • Search Strategy

10. Feature Space • Geometric landmarks: Points Edges Contours Surfaces, etc. • Intensities: Raw pixel values Feature-based Intensity-based • 35 • 56

11. Input image Output image Transformation Original shape Rigid transformation Affine transformation Image transformations Rigid Non-rigid

12. Similarity Metric • Absolute difference • SSD (Sum of Squared Difference) • Correlation Coefficient • Mutual Information / Normalized Mutual Information

13. Search Strategy • Powell’s direction set method • Downhill simplex method • Dynamic programming • Relaxation matching • Hierarchical techniques

14. Multi-modality Brain image registration • Intensity-based • 3D/3D Rigid transformation, DOF=6 (3 translations, 3 rotations) • Maximization of Normalized Mutual Information • Simplex Downhill • Multi-resolution • Dataset: Vanderbilt University http://www.vuse.vanderbilt.edu/~image/registration/results.html

15. Mutual Information as Similarity Measure • Mutual informationis applied to measure the statistic dependence between the image intensities of corresponding voxels in both images, which is assumed to be maximal if the images are geometrically aligned.

16. Normalized Mutual Information • Extension of Mutual Information Maes et. al.: Studholme et. Al.: • Compensate for the sensitivity of MI to changes in image overlap

17. Geometry Transformation • Image Coordinate transform: • The features (dimension, voxel size, slice spacing, gantry tilt, orientation) of images, which are acquired from different modalities, are not the same. • From voxel units (column, row, slice spacing) to millimeter units with its origin in the center of the image volume.

18. j j i i Template Image Grid Target ImageGrid y’ y Space Transform x’ x Template ImagePhysical Coordinates Target ImagePhysical Coordinates Target Image & Template Image

19. MRI-T2 PET 128 x 128 pixels 256 x 256 pixels Images from the same patient Target Image ? Template Image ? Images provided as part of the project: “Retrospective Image Registration Evaluation”, NIH, Project No. 8R01EB002124-03, Principal Investigator, J. Michael Fitzpatrick, Vanderbilt University, Nashville, TN.

20. Interpolation • Nearest Neighbor • Tri-linear Interpolation • Partial-Volume Interpolation • Higher order partial-volume interpolation

21. Evaluating similarity measure for each transformation y y Transform x x Template Image Target Image

22. Optimization • Powell’s Direction Set method • Downhill Simplex method

23. Multi-resolution • Why Multi-resolution • Methods for detecting optimality can not guarantee that a global optimal value will be found. • Time to evaluate the registration criterion is proportional to the number of voxels. • The result at coarser level is used as the starting point for the finer level. • Currently multi-resolution approaches: • Sub-sampling • Averaging • Wavelet

24. Registration Result (I) A typical superposition of CT-MR images. Left : before registration Right: after registration.

25. Rigid transformation (II) A typical superposition of MR-PET images. Left : before registration Right: after registration.

26. Mammography • Breast cancer is the second leading cause of death among women in USA. • Detected in its early stage, breast cancer is most treatable. • Mammography is the main tool for detection and diagnosis of breast malignances. • It reduces breast cancer mortality by 25% to 30% for women in the 50 to 70 age group

27. Mammogram Registration • Temporal/bilateral mammograms vary • Breast compression • Breast position • Imaging Technique • Change in Breast

28. Mammogram registration techniques • Whole breast area vs. regional • Nipple location • Control-point location • Rigid & non-rigid registration

29. Non-rigid Mammogram Registration • Intensity-based • Elastic transformation • Multi-resolution • Demons algorithm (Thirion, 1996)

30. Demons Transform Scene (Target) Model (Template)

31. Demons (Cont.) Transform Scene Forces Model

32. Demons (Cont.) Current Estimation Intensity Space Gradient Desired Displacement Scene

33. Demons • From Optical Flow • Scene: f, Model: g • Assumption: The intensity of a moving object is constant with time (1) (2)

34. Description of the Approach • Select demon points. • Compute the force uon the model at each of the selected demons • Determine a global transformation based on the computed u and apply it to the model • If the model images is now registered to the scene image, stop. Else, go to Step 2.

35. Registration Components • Image Intensities • Non-rigid transformation, one displacement vector for each pixel • Bilinear interpolation • Absolute difference as similarity metric • Multi-resolution • Dataset: MIAS,DDSM

36. Demons Results (I) Synthetic Images Level=2 Level=3 Level=5 Level=4

37. Demons Result (II) MIAS Original images Before registration After rigid registration After non-rigid registration

38. Ongoing registration topics • Trade-off of computation and accuracy • Evaluation of registration results • Visualization of registration

39. Applications: Change Detection • Images taken at different times • Following registration, the differences between the images may be indicative of change • Deciding if the change is really there may be quite difficult

40. Other Applications • Multi-subject registration to develop organ variation atlases. • Used as the basis for detecting abnormal variations • Object recognition - alignment of object model instance and image of unknown object (segmentation)

41. References • Maes F,Collignon A, et al. “Multimodality image registration by maximization of mutual information.” IEEE Trans. Med. Imaging. 1997, V16,pp187-198 • L.G.Brown, “A survey of image registration techniques,” ACM Computing Surveys, vol. 24, no. 4, pp. 325–376, 1992. • Jean-Philippe Thirion, “Non-Rigid Matching Using Demons,” IEEE Conference on Computer Vision and Pattern Recognition,1996