Medical Imaging Projects @ DePaul CDM

1. Medical Imaging Projects@ DePaul CDM Daniela S. Raicu, PhD Associate Professor Email: draicu@cs.depaul.edu Lab URL: http://facweb.cs.depaul.edu/research/vc/

2. Outline Medical Imaging (Computed Tomography) • Content-based and semantic-based image retrieval • Projects 1 and 2 • Mappings from low-level image features to semantic concepts • Projects 3 and 4 • Liver segmentation • Project 5

3. Content-based medical image retrieval (CBMS) systems - • Definition of Content-based Image Retrieval: • Content-based image retrieval is a technique for retrieving images on the basis of automatically derived image features such as texture and shape. • Applications of Content-based Image Retrieval: • Teaching • Research • Diagnosis • PACS and Electronic Patient Records

4. Image Features [D1, D2,…Dn] Feature Extraction Image Database Similarity Retrieval Query Image Feedback Algorithm User Evaluation Query Results Diagram of a CBIR

5. CBIR as a Diagnosis Aid An image retrieval system can help when the diagnosis depends strongly on direct visual properties of images in the context of evidence-based medicine or case-based reasoning.

6. CBIR as a Teaching Tool An image retrieval system will allow students/teachers to browse available data themselves in an easy and straightforward fashion by clicking on “show me similar images”. Advantages: - stimulate self-learning and a comparison of similar cases - find optimal cases for teaching • Teaching files: • Casimage: http://www.casimage.com • myPACS: http://www.mypacs.net

7. CBIR as a Research Tool • Image retrieval systems can be used: • to complement text-based retrieval methods • for visual knowledge management whereby the images and associated textual data can be analyzed together • multimedia data mining can be applied to learn the unknown links between visual features and diagnosis or other patient information • for quality control to find images that might have been misclassified

8. CBIR as a tool for lookup and reference in CT chest/abdomen • Case Study: lung nodules retrieval • Lung Imaging Database Resource for Imaging Research http://imaging.cancer.gov/programsandresources/Inf ormationSystems/LIDC/page7 • 29 cases, 5,756 DICOM images/slices, 1,143 nodule images • 4 radiologists annotated the images using 9 nodule characteristics: calcification, internal structure, lobulation, malignancy, margin, sphericity, spiculation, subtlety, and texture • Goals: • Retrieve nodules based on image features: • Texture, Shape, and Size • Find the correlations between the image features and the radiologists’ annotations

9. LIDC Semantic Concepts

10. Extracted Image Features

11. Lung nodule representation

12. Choose a nodule

13. Choose an image feature& a similarity measure

14. Retrieved Images

15. CBIR systems: challenges & REU projects • Type of features • image features: • - texture features: statistical, structural, model and filter-based • - shape features • textual features (such as physician annotations) • Project 1: Feature reduction for medical image processing • Investigate how many features with respect to the number of unique nodules • Investigate what the most important low-level image features are with respect to the retrieval process • Investigate the uniformity of the features with respect to the same type of nodules.

16. CBIR systems: challenges & REU projects (cont.) • Similarity measures • -point-based and distribution based metrics • Retrieval performance: • precision and recall • clinical evaluation • Project 2: Evaluation of CBIR and SBIR systems • Perform a literature review on the current techniques used to evaluate CBIR systems both for the general and medical domain • Investigate ways to include radiologists’ feedback in the retrieval process • Investigate ways to evaluate the retrieval process by varying various numbers of parameters such as number of images retrieved, cutoff value for acceptable precision and recall, and minimum number of radiologists/observers needed to evaluate the system.

17. Correlations between Image Features and Concepts

18. Automatic Mappings Extraction Step-wise multiple regression analysis was applied to generateprediction models for each characteristic ci based on all image features fk: where p is the # of used image features, are the regression coefficients, and are the prediction errors per model. Goodness of fit for the regression model:

19. Regression Models

20. Texture Regression Model

21. Malignancy Regression Model

22. Lobulation Regression Model

23. Spiculation Regression Model

24. Image Features – Semantics Mappings: challenges & REU projects • Project 3: Multi-view learning classifier for lung nodule classification • Investigate which image features are the best for individual semantic characteristics, build classifiers for each one of the individual classifiers, and combine the individual classifies for optimal learning/classification of lung nodules • Project 4: Bridging the semantic gap in lung nodule interpretation • Investigate ways to clinically evaluate the mappings from low-level image features to semantic characteristics • Investigate the effect of the imaging acquisition parameters (such as pitch, FOV, and reconstruction kernel) on the proposed mappings

25. Pixel Level Texture Extraction Pixel Level Classification Liver Segmentation in CT images - Pixel-level Classification: - tissue segmentation - context-sensitive tools for radiology reporting Organ Segmentation

26. Original Image Initial Seed at 90% Split & Merge at 85% Split & Merge at 80% Liver Segmentation in CT images Example of Liver Segmentation: (J.D. Furst, R. Susomboon, and D.S. Raicu, "Single Organ Segmentation Filters for Multiple Organ Segmentation",IEEE 2006 International Conference of the Engineering in Medicine and Biology Society (EMBS'06)) Region growing at 70% Region growing at 60% Segmentation Result

27. Liver Segmentation using Automatic Snake a) d) a) b) c) Figure: a) Gradient vector flow segmentation; b) Traditional vector field segmentation; c) and,d) Respective segmentations overlaid on ground truth (white). • Project 5: Automatic selection of initial points for snake-based segmentation

28. uestions ?