Analysis of Microvascular Lesions in Brain and Retina for Early CVD Detection

1. Large Analysis of microvascular lesions in the brain and retina using MRI and colourfundus imaging for early detection of CVD Prof. Ramamohanarao Kotagiri Department of Computing and Information Systems The University of Melbourne, Australia 3010 Email: kotagiri@unimelb.edu.au 1

2. Presentation Outline • Motivation • Brief Background • (Magnetic Resonance Imaging) MRI • Retinal Imaging • Brain microvascular lesions • White matter lesions (WMLs) • Brain infarcts • Retinal microvascular lesions • Arteriovenus nicking (AV nicking) • Focal arteriolar narrowing (FAN) • Objective • Proposed Method • Segmentation and quantification of WMLs • Detection and quantification of AV nicking • Detection and quantification of FAN • Summary 2

3. Vessel Calibre: A New Biomarker Retinal Circulation = ‘Window’ to Brain Circulation • Only place blood vessels can be viewed & monitored non-invasively • Damage to the retinal circulation may reflect impact of both recognised & unrecognised risk factors, and susceptibility • Indicator of structural vascular damage

4. Human Retina

5. Retinal Imaging • A retinal camera is used to capture image • A picture is taken showing the optic nerve (i.e., disc), fovea, surrounding vessels, and the retinal layer A retinal camera (left) retinal fundus image (right)

6. Significance of Retinal ImagingRetinal vascular signs may predict Cardiovascular Diseases and Diabetes! • CVD (Cardiovascular Disease) (heart disease & stroke) is the most common cause of death in the developed world • 36% all deaths in Australia (2004) • Kills one Australian every 10 minutes.

7. Diagnostic Tools • Currently available prediction tools based on assessment of traditional risk factors (e.g., blood pressure, cholesterol, smoking history, MRI after the symptoms of a stroke) • Account for only 50% of CVD cases Need for improved diagnostic tools

8. New Diagnostic Tools Traditional CVD Risk Factors e.g. blood pressure Unknown CVD Risk Factors e.g. genetic factors Sub clinical vascular damage Retinal Vascular Signs CARDIOVASCULAR DISEASE AND DEATH

9. Retinal Signs Predict CVD

10. COMPUTER-BASED RETINAL IMAGING PROGRAM FOR IDENTIFICATION OF CARDIOVASCULAR DISEASE RISK Patient’s retina photographed Photos transmitted to reading centre Retinal vascular scan report generated Retinal grading

11. Blood Vessels • Artery (Red) • Vein (Blue) Retinal Blood VesselAnalysis

12. Arterial Vane (AV) Nicking AV nicking

13. Focal Arterial Narrowing (FAN)

14. Cheap Way to image Retina Using smartphone –lens adaptor cost$5.00

15. Magnetic Resonance Imaging (MRI) • MRI is used for non invasively visualizing internal structures of the body . • Does not have any side effects of radiation . • Property of nuclear magnetic resonance (NMR) is used to image nuclei of atoms inside the body. • Provides higher details about soft-tissues 15

16. Magnetic Resonance Imaging (MRI) Different modalities are used to characterize and discriminate among tissues • T1 weighted MRI is effective for visualizing various anatomical structures such as White matter, Gray matter and CSF very useful for identifying the location of tissues • T2 weighted MRI is effective for visualizing pathologies such as lesions and tumors • Fluid attenuated inversion recovery (FLAIR) is effective in suppressing CSF (Cerebral Spinal Fluid) and enhancing lesions. 16

17. T1 MRI Anatomical Structures are clearly identifiable. Gray Matter White Matter CSF(Cerebral Spinal Fluid) WMLs 17

18. Pathology (WMLs) looks more clearer. T2 MRI WMLs 18

19. Flair MRI Intensity of CSF is suppressed WMLs 19

20. Motivation Relationship between WMLs and Risk of Stroke1 [1] Vermeer, Sarah E., et al. "Silent brain infarcts and white matter lesions increase stroke risk in the general population The Rotterdam Scan Study." Stroke 34.5 (2003): 1126-1129. 20

21. Motivation Correlation between severity of Sub-Cortical WMLs and AV nicking2 Sub-cortical WMLs Load • Q# is the severity scale of sub-cortical WMLs. • # Reference group; * p<0.05; § p<0.01. • [2] Qiu, Chengxuan, et al. "Retinal and cerebral microvascular signs and diabetes the age, gene/environment susceptibility-reykjavi study." Diabetes 57.6 (2008): 1645-1650. 21

22. Motivation Correlation between severity of Periventricular WMLs and AV nicking2 Periventricular WMLs Load • T# is the severity scale of peri-ventricular WMLs. • # Reference group; * p<0.05; § p<0.01. • [2] Qiu, Chengxuan, et al. "Retinal and cerebral microvascular signs and diabetes the age, gene/environment susceptibility-reykjavi study." Diabetes 57.6 (2008): 1645-1650. 22

23. Motivation Correlation between severity of Sub-Cortical WMLs WMLs and FAN2 Sub-cortical WMLs Load • Q# is the severity scale of sub-cortical WMLs. • # Reference group; * p<0.05; § p<0.01. • [2 ] Qiu, Chengxuan, et al. "Retinal and cerebral microvascular signs and diabetes the age, gene/environment susceptibility-reykjavi study." Diabetes 57.6 (2008): 1645-1650. 23

24. Motivation Correlation between severity of Sub-Cortical WMLs WMLs and FAN2 Periventricular WMLs Load • T# is the severity scale peri-ventricular WMLs. • # Reference group; * p<0.05; § p<0.01. • [2 ]Qiu, Chengxuan, et al. "Retinal and cerebral microvascular signs and diabetes the age, gene/environment susceptibility-reykjavi study." Diabetes 57.6 (2008): 1645-1650. 24

25. Motivation Retinal microvascular lesions (AV nicking and FAN) can be an important bio-marker to predict the severity of WMLs load. • Current approach of quantification and correlation analysis is Manual • Limitations • Highly subjective • Expensive • Time consuming • High intra and inter-grader variability 25

26. Objective • Automatic segmentation and quantification of WMLs • Automatic detection and quantification of AV nicking • Automatic detection and quantification of FAN • Quantify the correlation between retinal and brain microvascular lesions • Develop a retinal image based brain microvascular lesions prediction model 26

27. Automated WMLs segmentation method T1 MRI Flair MRI Pre-processing Feature extraction Classification of WMLs Post-processing using MRF 27

28. Automated WMLs segmentation method • Pre-processing for noise reduction and spatial normalization • Co-registration of T1 and Flair MRI using Statistical parametric mapping (SPM8)3 • Brain skull removal using Brain Extraction Tool (BET) 4 • Intensity normalization using dynamic maximum boundary5 • Feature extraction • Multimodal MRI (T1 and Flair) intensity • Tissue probability mask (PWM, PGM, PCSF) • Normalized spatial coordinate (X, Y, Z) • Global neighbourhood based contrast [3] J. Ashburner and K. J. Friston, “Unified segmentation,”Neuroimage, vol. 26, no. 3, pp. 839–851, 2005. [4] V. Popescu, M. Battaglini, W. Hoogstrate, S. Verfaillie, I. Sluimer, R. Van Schijndel, B. van Dijk, K. Cover, D. Knol, M. Jenkinson et al., “Optimizing parameter choice for fsl-brain extraction tool (bet) on 3d t1 images in multiple sclerosis,”Neuroimage, vol. 61, no. 4, pp. 1484– 1494, 2012. [5] Liang, Xi, Kotagiri Ramamohanarao et al. "Nat. ICT Australia (NICTA), Eveleigh, SA, Australia." Digital Image Computing Techniques and Applications (DICTA), 2012 International Conference on. IEEE, 2012. 28

29. Proposed automated WMLs segmentation method Multimodal MRI (T1 and Flair) intensity T1 MRI Flair MRI 29

30. Automated WMLs segmentation method • Tissue probability mask • Multiple atlas construction from healthy T1 MRI using FAST6 • Non-linear registration of atlases with input T1 MRI • Tissue mask construction based on multi atlas voting PWM PCSF PGM [6] Y. Zhang, M. Brady, and S. Smith, “Segmentation of brain mr images through a hidden markov random field model and the expectationmaximization algorithm,”Medical Imaging, IEEE Transactions on, vol. 20, no. 1, pp. 45–57, 2001. 30

31. Automated WMLs segmentation method • Normalized spatial coordinate • Subject patient’s T1 MRI is linearly registered with Montreal Neurological Institute (MNI ) space • The voxel wise corresponding Montreal Neurological Institute (MNI) space is warped back to the experimental subject space. • This results in normalized X, Y and Z space comparable between patients. X Y Z 31

32. Automated WMLs segmentation method • Global neighbourhood based contrast • Create global neighbours for each pixel • Mask neighbours of candidate pixel using PWM. • A box filter of size m is applied on candidate and its neighbour pixels • Global neighbourhood based contrast (GNC) is computed using Here, X represent candidate pixel and Xnrepresents its neighbours. 32

33. Automated WMLs segmentation method Classification of Lesion using Random Forest 33

34. Automated WMLs segmentation method Post processing using Markov random field (MRF) Neighbourhood structure • represents lesion probability computed by RF classifier • represents edge potential computed by Eq. 2. • xi and yi represent ith pixel intensity and RF given class label. • is the penalty constant. • Loopy belief propagation (LBP) 6 is used to infer the outcome of MRF [6] K. P. Murphy, Y. Weiss, and M. I. Jordan, “Loopy belief propagation for approximate inference: An empirical study,” in Proceedings of the Fifteenth conference on Uncertainty in artificial intelligence. Morgan Kaufmann Publishers Inc., 1999, pp. 467–475. 34

35. Automated WMLs segmentation method • Segmentation outcome after applying MRF Initial Segmentation MRF given Segmentation Ground Truth Segmentation 35

36. Automated WMLs segmentation method • Classification Parameters • Classifier : Random Forest • Number of trees in RF : 200 • Data • Number of Subject : 24 • Source of Data: ENVISion study7 • Size : 256 × 256 × 36 • Resolution : 0.94 × 0.94 × 4 mm3 • Training and Testing procedures • 4 Fold cross-validation (4 times) • 2 fold for training • 1 fold for parameter selection • 1 Fold for testing • [7] C. M. Reid, E. Storey, T. Y.Wong, R. Woods, A. Tonkin, J. J. Wang, A. Kam, A. Janke, R. Essex,W. P. Abhayaratna et al., “Aspirin for the prevention of cognitive decline in the elderly: rationale and design of a neuro-vascular imaging study (envis-ion),”BMC neurology, vol. 12, no. 1, p. 3, 2012. 36

37. Training of Random Forest Classifier • Number of subjects : • Resolution of each Subject • Number of data points (voxels) after skull stripping • Number of features for each data point: 26 • The number of training sample is very large = 810,00*12 = 9.7 million voxels • To reduce the training time we randomly sample 20,000 voxels to build a random forest tree and build about 200 random forest trees. This method provides adequate accuracy!

38. [1] A. Baraldi and F. Parmiggiani, “An investigation of the textural characteristics associated with gray level cooccurrence matrix statistical parameters,” Geoscience and Remote Sensing, IEEE Transactions on, vol. 33, no. 2, pp. 293–304, 1995. [2] M. M. Galloway, “Texture analysis using gray level run lengths,” Computer graphics and image processing, vol. 4, no. 2, pp. 172–179, 1975. [3] P. Bankhead, C. N. Scholfield, J. G. McGeown, and T. M. Curtis, “Fast retinal vessel detection and measurement using wavelets and edge location refinement,”PloS one, vol. 7, no. 3, p. e32435, 2012.

39. Automated WMLs segmentation method • Evaluation metrics • Sensitivity (SEN) • Positive predictive value (PPV) • Dice Similarity Index (SI) 39

40. Automated WMLs segmentation method • WMLs Load Category • High Lesion Load (HLL [>10ml]) • Medium Lesion Load (MLL [between 6 ml to 10 ml]) • Low Lesion Load (LLL [between 1 ml to 5 ml ]) 40

41. Proposed automated WMLs segmentation method Results 41

42. Proposed automated WMLs segmentation method Comparison with state of the art methods 8 9 For both methods p-value< 0.05. [8] M. D. Steenwijk, P. J. Pouwels, M. Daams, J. W. van Dalen, M. W. Caan, E. Richard, F. Barkhof, and H. Vrenken, “Accurate white matter lesion segmentation by k nearest neighbor classification with tissue type priors (knn-ttps),”NeuroImage: Clinical, vol. 3, pp. 462–469, 2013. [9] P. Schmidt, C. Gaser, M. Arsic, D. Buck, A. F¨orschler, A. Berthele, M. Hoshi, R. Ilg, V. J. Schmid, C. Zimmer et al., “An automated tool for detection of flair-hyperintense white-matter lesions in multiple sclerosis,”Neuroimage, vol. 59, no. 4, pp. 3774–3783, 2012. 42

43. Automatic quantification of AVN and FAN Retinal Image Pre-processing Feature extraction Quantification of AVN and FAN 43

44. Automatic quantification of AVN and FAN • Pre-processing • Vessel Segmentation • Vessel Cross-over point detection • Candidate vessel region selection • Vessel width measurement 44

45. First to Analyze the Vessel: Segmentation Although many methods have been proposed, significant improvement is still a necessity due to the limitations in state of the-art methods, which include: • Poor segmentation in the presence of central reflex (i.e., bright strip along the centre of a vessel). • Poor segmentation at bifurcation and crossover points. • The merging of close vessels. • The missing of small vessels. • False vessel detection at the optic disk and pathological regions. 45

46. Some Limitations of Existing Methods • a portion of a retinal image showing the presence of central reflex vessels (white solid arrows), close vessels (white dashed arrows), artery-vein crossing regions (black solid arrows), and small vessels (black dashed arrows) and segmentations obtained by • (b) Staal et al. method; (c) Soares et al. method; (d) Ricci-line method ; (e) Ricci-svm method (f) Our proposed method. Uyen et. all (2013), Pattern Recognition 46

47. OurProposedMethod • A linear combination of line detectors at different scales to produce the vessel segmentation for each retinal image. • A basic line detector uses a set of approximated rotated straight lines to detect the vessels at different angles. • The difference between the average gray level of the winning line (the line with maximum average gray level) and the average gray level of the surrounding window provides a measure of ‘vesselness’ of each image pixel. 47

48. VesselSegmentationAccuracy Segmentation results on some selective regions showing the improvements of the proposed method over existing methods in terms of segmentation quality: (a) original image; segmentations of (b) Staal et al. method; (c) Soares et al. method; (d) Ricci-line method; and (e) proposed method. 48

49. Vessel Segmentation Accuracy 49

50. Vessel Segmentation Accuracy (Cont.) 50