Brian Canada Academic Computing Fellow and PhD Candidate in Integrative Biosciences

1. Automated Segmentation and Classification of Zebrafish Histology Images forHigh-Throughput Phenotyping Brian Canada Academic Computing Fellow and PhD Candidate in Integrative Biosciences Jake Gittlen Cancer Research Institute | Penn State College of Medicine October 22, 2007

2. The zebrafish (danio rerio): A powerful functional genomics tool • Vertebrate • Develop tumors • Hundreds of eggs per clutch • Rapid, ex vivo development • Most organ systems differentiated before 7 days post-fertilization • Transparent embryos • Reverse genetics • Morpholinos for gene “knock-down”

3. Zebrafish histology Adult zebrafish (sagittal plane view) with papilloma Zebrafish larval array hht mutant 7dpf (days post-fertilization)

4. “High-Throughput”Zebrafish Histology Embedding in agarose Processing into paraffin Sectioning, staining, mounting onto slides Fixation The “rate-limiting step” Scoring and Annotation Scanning Digitization • What can be done to improve the speed and reliability of scoring images? • Can we score abnormalities quantitatively?

5. Current efforts in automated zebrafish image analysis • Stephen T.C. Wong and colleagues at Harvard developed methods for quantitative assessment of neuron loss and automated detection of somites • In principle, such automated methods should be scalable to allow high-throughput phenotyping Retinal cell detection for studying neurogenesis Detection of Rohon-Beard sensory neurons Liu T.L., “A quantitative zebrafish phenotyping tool for developmental biology and disease modeling,” IEEE Signal Processing Magazine, Jan 2007.

6. Building on interdisciplinary expertise Keith Cheng, MD, PhDZebrafish Functional Genomics James Z. Wang, PhDContent-Based Image Retrieval,Automatic Image Annotation

7. SHIRAZ: System for Histological Image Retrievaland Annotation for Zoopathology [ ] IPL_Compactness = 9.8137 IPL_Eccentricity = 0.9019 IPL_Solidity = 0.3086 IPL_Contrast = 0.9375 IPL_Homogeneity = 0.0093 LENS_COMPACTNESS = 1.1262 LENS_eccentricity = 0.3530 … …    ImagePre-processing Creation ofVirtual Slides Extract feature vector for each image Image segmentation  Use feature database to train model for image classification (K-means clustering, Classification trees, Support Vector Machine, etc.) Automatically classify and annotate previously uncharacterized images   Repeat for allimages in database

8. SHIRAZ: System for Histological Image Retrievaland Annotation for Zoopathology • Prototype implemented in MATLAB for segmentation and classification of eye and gut images • Eye and gut tissues have a polar or directional organization that is deformed or disrupted on mutation • To our knowledge, we are the first group to publish material on automated zebrafish histology image analysis • Canada, B.A., Thomas, G.K., Cheng ,K.C., Wang, J.Z., “Automated Segmentation and Classification of Zebrafish Histology Images For High-Throughput Phenotyping,”Proc IEEE-NIH Life Science Systems And Applications (LISSA) Workshop 2007

9. Image pre-processing    Manually crop eye and gut images from selected larvae Take snapshot of selected H&E-stained specimens in ImageScope Aperio T2 Scanner for Creation ofVirtual Slides (120 slide capacity)  To reduce computational costs, convert to grayscale 512 x 512 matrix (pad with white pixels if needed)

10. Ganglion Cell Layer (GCL) Inner Plexiform Layer (IPL) Lens Retinal Pigmented Epithelium (RPE) PhotoreceptorLayer (PRL) Inner Nuclear Layer (INL) Example of wild-typeeye segmentation

11. Example of mutant eye segmentation

12. Filled area Perimeter Compactness Eccentricity Extent Solidity Fractal dimension Seven moment invariants Four gray level co-occurrence features: Contrast Correlation Energy Homogeneity Eye feature extraction Yields vector of 92 features per eye image

13. Gut segmentationand feature extraction 30 features extracted per gut image, e.g.: • Thickness and shape of the epithelial lining • Polarity of the epithelial cells (position of nuclei relative to basement membrane) • Number of distinct villi (folds) of the lumen • Amount and “granularity” of cellular debris and mucous in lumen Epithelial lining Lumen Cell nuclei

14. Classification algorithm: CART (Classification And Regression Trees) • Advantages: • “White-box” model • Helps provide a sense of objectivity and direction to histological assessment • Disadvantages: • May not be as accurate as other classification methods (e.g. SVM, GMM, ANN) • “Splits” can only be performed on one dimension at a time (not really a problem in this case)

15. Preliminary Results

16. Discussion and Conclusions • Preliminary results are encouraging • Potential opportunities for improvement: • Analyze different larval ages separately • Improve segmentation accuracy • Use color images instead of grayscale • Experiment with different classifiers (SVM, for example) • Minimize manual preprocessing • Increase overall size of datasets • Future: • Direct integration into laboratory pipeline • Parallel image processing for higher throughput • Automatic image annotation and retrieval

17. Current collaborators • Georgia Thomas, Graduate Student • Keith Cheng, co-PI (Functional Genomics) • James Z. Wang, co-PI (Info Science & Tech) • Prof. Yanxi Liu (PSU Computer Science dept.) • Prof. Nancy Hopkins (MIT)