ROC Based Evaluation and Comparison of Classifiers for IVF Implantation Prediction

1. ROC Based Evaluation and Comparison ofClassifiers for IVF Implantation Prediction Aslı Uyar, Ayşe Bener Boğaziçi University, Department of Computer Engineering, Istanbul Nadir Çıray, Mustafa Bahçeci IVF Unit of German Hospital, Istanbul PhD Thesis 2nd Progress Presentation 15th June 2009

2. IN-VITRO FERTILIZATION (IVF) • IVF is a common infertility treatment method duringwhich female germ cells (oocytes) are inseminated by sperm under laboratoryconditions. • Fertilized oocytes are cultured between 2-6 days in special medicalequipments and embryonic growth is observed and recorded by embryologists. • Finally, selected embryo(s) are transferred into the woman's womb.

3. PROBLEM STATEMENT • In-vitro fertilization (IVF) • A complex and costly process requiring automated decision support • Many critical decisions affecting the success of treatment • Analysis of factors affecting treatment outcome • Number of embryos to be transferred • Embryo selection • Decision of transfer day • Initial consideration • Implantation prediction of individual embryos • Yielding reliable elective single embryo transfer (eSET) avoiding multiple pregnancies

4. MOTIVATIONAND OBJECTIVES • MOTIVATION: • Lack of reliable eSET criteria and public IVF datasets • Limited number of machine learning based implantation prediction studies • Conflicting prediction results • No consensus on: • input feature sets • training and testing strategies • performance measures • OBJECTIVES • Dataset construction • Benchmarking machine learning based predictor models • Enhancement of prediction results by methodological improvements or novel techniques • Statistical validation and generalization of proposed methods

5. METHODOLOGY

6. DATASET • Database of German Hospital, Bahceci IVF Center • Cycles performed from January 2007 through August 2008 • Two classes of total 2453 embryos • Each embryo was represented as an individual record with 18 input features related to clinical patient and embryo variables • 1853 embryos with proven negative implantation (cycles with negative outcome) • 270 embryos with proven positive implantation outcome (cycles in which number of visualized pregnancy sacs were equal to number of transferred embryos) • Imbalanced class distribution with 89% negative and 11% positive cases • Considering both FP and FN rates

7. DATASET FEATURES

8. PERFORMANCE MEASURES AND ROC ANALYSIS • A single performance measure representing discriminative power of binary classification • Default threshold t_0 = 0.5 • Ideal case: (0,1) left corner of ROC curve • Optimum threshold t_opt: threshold value that maps to nearest point to (0,1) Accuracy = TP + TN / (TP + FN + FP + TN) Sensitivity = TP / (TP + FN) ~ TPR Specificity = TN / (TN + FP) (1 – Specificity) ~ FPR

9. BENCHMARKING CLASSIFIERS Only NB and RBF provided acceptable discrimination with 0.7 ≤ AUC < 0.8. (Hosmer and Lemeshow, 2000)

10. Thanks for listening!