Angela Jean

1. Angela Jean Bioinformatics Group School of Chemical and Life Sciences 8th September 2009, SYMBIO 2009 Automated Scoring of Her2/neu Status in Breast Carcinomas

2. Content • Motivation • Image Informatics • Bioimaging • Carcinoma scoring • Process overview • Method • Results • Conclusion • Question & Answer

3. Translational Research & Medicine • To improve human health, scientific discoveries must be translated into practical applications. • Bench to beside - Such discoveries typically begin at “the bench” with basic research — in which scientists study disease at a molecular or cellular level — then progress to the clinical level, or the patient's “bedside.” -- NIH http://clinicalcenter.nih.gov/ccc/btb/images/b2b.gif

4. Image Informatics • Process of mining and refining knowledge derived from images • Consists of • Capturing high quality images • Extracting raw data • Analyzing the data • Presenting it in an easily comprehensible manner • In High Content Screening • It is mining and refining knowledge from many images at one time a lot of data at one time

5. Image Informatics • In Life Sciences • Such images are obtained in a process known as bioimaging • Provides images of various resolution and quality • Large amount of images may be attained at one time • Results in more images to process to extract “hidden information”

6. Technological Advancements • Fast digital cameras with higher resolution • Automated motorized microscopes • Quantum dots • New fluorophores (e.g. EGFP) • Increased computational powers • Better stains? http://en.wikipedia.com/

7. Why is Bioimaging important? • The ability to visualize, trace and quantify cellular morphologies at high spatial and temporal resolutions is becoming essential • For the understanding of biological processes • For the development of effective therapeutic agents. http://www.skewsme.com

8. Workflow in Bioimaging • An example of a workflow in a bioimaging process Xiaobo Zhou, Stephen T.C. Wong, A Primer on Image Informatics of High Content Screening, High Content Screening: Science, Techniques and Applications, 2008(3), 43 - 84

9. Challenges in Bioimaging • Although genome-scale experiments are now routinely performed, the difficulty of interpreting such large-scale image datasets varies with the apparatus’ readouts. • An example • Immunohistochemistry reading of cancer cells

10. Carcinoma Samples IHC 0 IHC 1+ IHC 2+ IHC 3+

11. Carcinoma Scoring • Score IHC 0 • No observable staining or membranous staining in less than 10% of tumour cells • Score IHC 1+ • Faint or barely perceptible membrane staining in less than 10% of cells • Cells are stained only in part of the cell

12. Carcinoma Scoring • Score IHC 2+ • Weak to moderate complete membrane staining in less than 10% of cells • Score IHC 3+ • Strong complete membrane staining in less than 10% of cells

13. Automated Scoring Overview

14. Algorithm • Classification Module • Color system conversion  Measurements  Post processing  Classification

15. Algorithm • Unstained Membrane Image Processing Module • Segmentation  Post processing  Measurement (Pre-processing)  Classification

16. Algorithm • Stained Membrane Image Processing Module • As per previous module  Bounding box measurement  Classification

17. Focus

18. Method – Classification • Bio-image process • Encompassing classical methods such as cell edge detection and color segmentation • Typically fast for a couple of cells • But computationally intensive for processing hundreds to thousands of images on a regular basis; each with over 100 cells • Classification Module • Provide some form of (pre-)classification so that computational time spent downstream processes can be more efficiently used

19. Example: Cell Detection • Bounding Box (measurement algorithm)

20. Example: Cell Detection • Bounding Box (first we find the nucleus)

21. Example: Cell Detection • Bounding Box (then we find the corresponding cell wall)

22. Example: Cell Detection • Bounding Box (and we do this for the rest of the cells in the sample)

23. Example: Cell Detection • Bounding Box (doing it for a tissue microarray)

24. Example: Cell Detection • Bounding Box (doing it for a tissue microarray)

25. Method – Basis • Generally, different grades of cancer cell “appear different” • Different sets of colors • E.g. IHC 3+ sample will have more “orange/brown” than an IHC 1+ sample • From a computational and bio-image informatics perspective • If numbers can be assigned to colors • Then, for e.g., IHC 3+ samples will have a different set of color values than IHC 1+

26. Method – Basis • Here, the pixels of each sample image is represented using the CIELAB color space • L – Intensity • A – A Channel • B – B Channel • By plotting the different (A, B) pixel values of the image on a 2D histogram, the “concentration” or the spread of the color values of each image can be obtained and visualized http://www.newsandtech.com/issues/2002/02-02/ifra/02-02_greybalance.htm

27. Scoring Results (2D histograms) 0 1 2 3

28. Scoring Results (2D histograms) 0 1 2 3

29. Scoring Results (Contour plots) 0 1 2 3

30. Obtaining the orientation • Orientation of ellipse = Angle of major axis against x-axis

31. Obtaining the orientation • Traditionally… Orientation angle

32. Obtaining the orientation • But with modern day technology…

33. Orientation values • Obtained through unsupervised learning • Value set varies with different cancer samples and staining methods • Breast Carcinoma • IHC 0 • Values are greater than 0, i.e. positive values • IHC 1+, 2+, 3+ • Values are negative • Increased negativity denotes higher scoring

34. Orientation values • Breast Carcinoma • IHC 1+, 2+, 3+ • Values are negative • Increased negativity denotes higher scoring • Typically • IHC 1+ • - 10 ± 5 • IHC 2+ • - 20 ± 5 • IHC 3+ • - 30 ± 5

35. Results • We processed about 70 samples obtained from a histopathologist on a typical day:

36. Patent pending • Process and Device for Automated Grading of Bio‐Specimens, PCT/SG2008/000397, 2008 

37. Downstream Process • Images that are pre-classified as IHC 0+ and/or possibly IHC 1+ do not have to go through computationally intensive processing • Read: Days (and nights) of potato chips and coffee • More images can be processed at a shorter time

38. Conclusion • The (pre-)classification method • Can effectively sieve out images of IHC 0 samples • Is more objective • Color values are objectively processed and not subjected to inter-observer variability and fatigue • Can possibly pre-score an image for better scrutiny • A normalization process can effectively allow the computer to analyze faded or old images accurately • Which presents the method as a useful tool to the histopathologist

39. A final note • The automated scoring/grading methods are not meant to replace your flesh-and-blood histopathologist • This means that you are unlikely to be treated/diagnosed by a robot any time in the near future • The tools are meant to help your doctors make a more accurate diagnosis – just like what X-ray machines and MRIs are used for

40. Thank you • Question & Answer http://en.wikipedia.org/wiki/File:WasatchMountainsSaltLakeCountyWestSide.jpg