9.4.08

1. Timothy R. TuinstraDissertation Defense:Automatic segmentation of small pulmonary nodules in computed tomography imagery using a radial basis function neural network with application to volume estimation 9.4.08

2. Background • Lung cancer is the leading cause of cancer death in the U.S. • 215,000 new cases will be diagnosed this year. • About 160,000 people will die of lung cancer in the U.S. alone in 2008. • Computer aided detection and diagnosis (CADD) tools are critical to the fight.

3. Research Goals • Primary Goal: To develop a robust automatic segmentation algorithm for pulmonary nodules that compares well with radiologist segmented data. • Secondary Goal: To make use of 3D nodule segmentations to automatically estimate nodule volumes.

4. Foundational Work • Zhao et al: • Create candidate segmentations • Measure features such as sphericity and gradient strength • Apply heuristic rule to choose a segmentation • Kostis et al: • Use a constant threshold and morphological processing with a fixed structuring element radius over a set of nodules.

5. The need for an adaptive approach: • “…although a static structuring kernel size ay be chosen that yields reasonably good results over a wide range of nodules, a better approach is to choose a structuring kernel of the appropriate size for a particular case.” Kostis et al. 2003

6. Segmentation Engine Input VOI Output Segmentation Threshold Connectivity Requirement Morphological Processing Connectivity Requirement R T T - Intensity Threshold R – Structuring Element radius LIVE DEMO!

7. Morphological Processing:Opening Small Radius: Large Radius: Resulting Segmentation: Resulting Segmentation:

8. Big Ideas: • Create candidate segmentations • Predict the quality of the candidates and pick the best one

9. How good is a Segmentation? Overlap Measure: Truth Segmentation kth Candidate Segmentation

10. Volume of interest Structuring Element Radius Intensity Threshold Candidate Segmentation Feature Vector Predicted Overlap Prediction of Overlap Artificial Neural Network Segmentation Engine Extract Features

11. The Neural Network • We experimented with both multilayer perceptrons and radial basis function(RBF) networks to predict the overlap. • General regression radial basis function networks (GRNN) were chosen because of the speed of training and consistency of results. • GRNN networks are known for their ability to approximate functions. • GRNN networks use examples to respond to previously unseen inputs.

12. The Neural Network • Generalized Regression Radial Basis Function Network: maps kth feature vector jth “center” jth radial basis function jth weight

13. φ1 φ2 Σ φM The Network Architecture

14. RBF Network Exampleσ=1

15. RBF Network Exampleσ = 0.5

16. Measured Segmentation Features Network Centers Network Weights Desired Network Outputs (Desired Overlap) Radial Basis Functions Gaussian Function Training the Network Select M training vectors together with corresponding desired outputs.

17. The Training Data • 72 nodules were from the Early Lung Cancer Action Program (ELCAP). • 1.25 mm slice spacing and thickness • 76 nodules were from the University of Texas Medical Branch (UTMB). • 2.5 and 5 mm slice spacing and thickness • These nodules were manually segmented using T & R method.

18. 76 Nodules 72 Nodules SFS weights SFS weights SFS weights 76 Nodules The Training Data • The training data was partitioned into two sets: • One set was used to determine weights. • The second set was used to select features.

19. Feature Selection • Features were selected from a pool of 50 using Sequential Forward Selection (SFS). • SFS: • Train a network with each of the 50 features. • Select the one that provides the best mean overlap across the training set and include as winning feature #1. • Create networks using #1 and each one of the remaining features. • Find winner #2 and include in feature set. • Continue until adding additional features no longer helps.

20. Winning features • 1) Mean convergence index (MCI) • 2) Mean surface gradient • 3) Mean separation in the axial direction • 4) Radial Deviation

21. Estimate the 3-D gradient of the intensity volume of interest: Create a radial vector field: Divide these two vector fields by their magnitudes to find unit vector fields. Compute the inner product (scalar field): Mean Convergence Index

22. 2-D Example of Mean Convergence Index

23. Compute Inner Product =

24. Candidate 1 Candidate 2 Resulting Scalar Field The Mean Convergence Index is the mean of the pixels contained in the candidate segmentation. Obviously, the green segmentation has a higher MCI than the red segmentation candidate.

25. Mean intensity in segmentation Mean intensity axial boundary voxels Other Features • Mean Surface Gradient: Number of voxels on the surface of the segmentation • Mean Intensity Separation in Axial Direction:

26. Other Features • Radial Deviation: Computed in the same way as MCI except only for boundary voxels.

27. Automatic Selection of T & R Output Segmentation Input VOI Threshold Connectivity Requirement Morphological Processing Connectivity Requirement Artificial Neural Network Feature Extractor Update T & R

28. Updating T & R • “Exhaustive” Search • Test candidates on a uniformly spaced grid of T & R and choose largest predicted overlap. • Simulated Annealing • Move through the T & R space in a purposeful manner “looking” for the largest predicted overlap. • Reduces computation time. • Golden Section Search • Apply across thresholds. • Reduces computation time.

29. Minimizing via simulated annealing • Choose an initial value for R & T • Compute predicted quality for this R & T: • Compute Segmentation • Compute Features • Use NN to compute cost • Take a random step in R & T space • Max step size based on previous cost • High quality yields small step (we must be close) • Low quality yields larger step

30. Minimizing via simulated annealing • At the new location, compute the segmentation and evaluate the predicted quality. • If it is less than the previous one…keep it. • If it is more…keep it with some small probability that gets smaller with # of iterations. This keeps us from getting stuck in local minima. • Repeat until some criteria is satisfied. • Always keep the highest quality output seen as the best R & T • Example trajectories are shown in the next few slides.

31. Minimizing via Golden Section Search • Apply this search along T for 4 different values for R. • Find the 4 minima. • Choose the smallest of these 4.

32. What data was used for segmentation testing? • Lung Imaging Database Consortium (LIDC) data • 69 Nodules • Manual segmentations of up to 4 radiologists are contained in the data. • Nodules with 3 or 4 radiologists segmentations were used. • Voxels considered to be part of the nodule by 3 or 4 radiologists comprise the truth mask to which we will compare.

33. Segmentation Example

34. Segmentation Example

35. Example Segmentation LIDC #34: Overlap = 81.5% Quality Function

36. Example Segmentation LIDC #36: Overlap = 40.6% Quality Function

37. Example SegmentationLIDC #41 Quality Function Overlap = 68.2%

38. Live Segmentation Demo!

39. Segmentation Overlap Results for LIDC

40. Overlap Distribution for Simulated Annealing Results 67% of nodules with 60% or greater overlap!

41. Comparison of Overlap Results Using LIDC Data 64% 63% 58% 51% 51%

42. Modified Winer-Muram Volume Estimation • Volume estimation of nodules using thick-slice data tends to produce a volume magnification effect. • Winer-Muram et al. suggested a magnification of the form: A constant Slice Thickness Actual volume (what we are trying to find)

43. Modified Winer-Muram Volume Estimation • The relationship of the true volume to the apparent volume is • Putting the two equations together yields the equation:

44. Modified Winer-Muram Volume Estimation • This equation may be solved numerically. • It actually also has an analytic solution by converting it to a cubic and using Cardano’s method.

45. MWM testing • This volume estimation computation was tested using images of 13 nodule phantoms imaged at various thicknesses. • The truth volumes of these phantoms was known ahead of time and used for comparison. • The results were compared with perimeter method and minimax method.

46. Modified Winer-Muram: Results 1.25 mm 5 mm 8 mm 10 mm

47. Comparison Methods Perimeter Method: Minimax Method: Number of segmented voxels x-dimension voxel size Estimated Column height y-dimension voxel size Number of segmented columns Slice spacing

48. Modified Winer-Muram: Results 1.25 mm 5 mm

49. Modified Winer-Muram: Results 8 mm 10 mm

50. Modified Winer-Muram: Results