Introdução à Medicina II 2011/2012 Class 12 1 Advisors: Rosa Oliveira 2 and Daniel Martins 3

1. Case-mix and Optimization of Mortality Prognostic Scores in Portuguese Pediatric Intensive Care Units Introdução à Medicina II 2011/2012 Class 121 Advisors: Rosa Oliveira2andDaniel Martins3 1. alunosmedicina12@gmail.com; 2. rcoliveira@med.up.pt; 3. daniel_artur@hotmail.com;

2. Introduction • Contextualization • State of the Art • Background • Are scores discriminative? And calibrated? • Differences in calibration… Why? • Should scores be recalibrated? • Problems… • Research Questions • Aims • Methods • Expected results • Results • References • Acknowledgments Table of Contents Case-mix and Optimization of Mortality Prognostic Scores in Portuguese Pediatric Intensive Care Units

3. Introduction Introduction Introduction State of the Art State of the Art Background Background • Contextualization • A Pediatric Intensive Care Unit (PICU)is an area within a hospital specialized in the care of critically ill infants, children, and teenagers. • The type and number of patients and the provided health care differs according to its location. Research Questions Research Questions Aims Aims Methods Methods Results References References Acknowledgements Acknowledgements Frankel, Lorry R; DiCarlo, Joseph V (2003). “Pediatric Intensive Care”. In Bernstein, Daniel; Shelov, Steven P. Pediatrics for Medical Students (2nd ed.). Case-mix and Optimization of Mortality Prognostic Scores in Portuguese Pediatric Intensive Care Units PICUs – Pediatric Intensive Care Units

4. Introduction Introduction Introduction State of the Art State of the Art Portuguese PICUs Background Background Research Questions Research Questions Consume human and financial resources Aims Aims Methods Methods Results Provide adequate treatment for severely ill children References References Acknowledgements Acknowledgements PICUs – Pediatric Intensive Care Units Case-mix and Optimization of Mortality Prognostic Scores in Portuguese Pediatric Intensive Care Units PICUs – Pediatric Intensive Care Units PICUs – Pediatric Intensive Care Units

5. Introduction Introduction Introduction State of the Art State of the Art Portuguese PICUs Background Background Research Questions Research Questions Provide adequate treatment for severely ill children Aims Aims Major health quality resource Pacient stratification Methods Methods Evaluation Results References References Acknowledgements Mortality rate Assistance quality Acknowledgements PICUs – Pediatric Intensive Care Units Case-mix and Optimization of Mortality Prognostic Scores in Portuguese Pediatric Intensive Care Units PICUs – Pediatric Intensive Care Units PICUs – Pediatric Intensive Care Units

6. Introduction Introduction Introduction State of the Art State of the Art Portuguese PICUs Background Background Research Questions Research Questions Provide adequate treatment for severely ill children Aims Aims Major health quality resource Pacient stratification Methods Methods Evaluation Results References References Acknowledgements Mortality rate Assistance quality Acknowledgements PICUs – Pediatric Intensive Care Units Case-mix and Optimization of Mortality Prognostic Scores in Portuguese Pediatric Intensive Care Units PICUs – Pediatric Intensive Care Units PICUs – Pediatric Intensive Care Units

7. Introduction Introduction Introduction State of the Art State of the Art Portuguese PICUs Background Background Research Questions Research Questions Provide adequate treatment for severely ill children Aims Aims Major health quality resource Pacient stratification Methods Methods Evaluation Results References References Acknowledgements Mortality rate Assistance quality Acknowledgements PICUs – Pediatric Intensive Care Units Case-mix and Optimization of Mortality Prognostic Scores in Portuguese Pediatric Intensive Care Units PICUs – Pediatric Intensive Care Units PICUs – Pediatric Intensive Care Units

8. Introduction Introduction Introduction State of the Art State of the Art Portuguese PICUs Background Background Research Questions Research Questions Provide adequate treatment for severely ill children Aims Aims Major health quality resource Pacient stratification Methods Methods Evaluation Results References References Acknowledgements Mortality rate Assistance quality Acknowledgements PICUs – Pediatric Intensive Care Units Case-mix and Optimization of Mortality Prognostic Scores in Portuguese Pediatric Intensive Care Units PICUs – Pediatric Intensive Care Units PICUs – Pediatric Intensive Care Units

9. Introduction Introduction Introduction State of the Art State of the Art Portuguese PICUs Background Background Research Questions Research Questions Provide adequate treatment for severely ill children Aims Aims Major health quality resource Pacient stratification Methods Methods Evaluation Results References References Acknowledgements Mortalit rate Assistance quality Acknowledgements PICUs – Pediatric Intensive Care Units Case-mix and Optimization of Mortality Prognostic Scores in Portuguese Pediatric Intensive Care Units PICUs – Pediatric Intensive Care Units PICUs – Pediatric Intensive Care Units

10. Introduction Introduction Introduction State of the Art State of the Art Portuguese PICUs Background Background Research Questions Research Questions Aims Provide adequate treatment for severely ill children Aims Methods Methods Results Consume human and financial resources References References Acknowledgements Acknowledgements PICUs – Pediatric Intensive Care Units Case-mix and Optimization of Mortality Prognostic Scores in Portuguese Pediatric Intensive Care Units PICUs – Pediatric Intensive Care Units PICUs – Pediatric Intensive Care Units

11. Introduction Introduction State of the Art Background Mortality Prognostic Scores Research Questions Aims Methods PIM PRISM Results References Acknowledgements PIM – Pediatric Index of Mortality; PRISM – Pediatric Risk of Mortality Case-mix and Optimization of Mortality Prognostic Scores in Portuguese Pediatric Intensive Care Units PICUs – Pediatric Intensive Care Units PICUs – Pediatric Intensive Care Units

12. Introduction Introduction State of the Art Background Mortality Prognostic Scores Research Questions Aims Methods PIM PRISM Results References Acknowledgements PIM – Pediatric Index of Mortality; PRISM – Pediatric Risk of Mortality Case-mix and Optimization of Mortality Prognostic Scores in Portuguese Pediatric Intensive Care Units PICUs – Pediatric Intensive Care Units PICUs – Pediatric Intensive Care Units

13. Introduction Introduction State of the Art Background Research Questions Logit = (-4.873) + (values * Beta) + (0.021 * (absolute(SBP-120))) + (0.071 * (absolute base excess)) + (0.415 * (FiO2/PaO2)) Mortality Prediction Value= eLogit/ (1+eLogit) Aims Methods Expected results • Amountofinformation; • Duration of theobservationperiod; • Momentinwhichtheinformationiscollected.1 References PIM – Pediatric Index of Mortality; PRISM – Pediatric Risk of Mortality Acknowledgements 1. Slater, A.M.B., FRACP, FJFICM; Shann, Frank MB, BS, MD, FRACP, FJFICM; the ANZICS Paediatric Study Group, The suitability of the Pediatric Index of Mortality (PIM), PIM2, the Pediatric Risk of Mortality (PRISM), and PRISM III for monitoring the quality of pediatric intensive care in Australia and New Zealand. Pediatric Critical Care Medicine, 2004. 5(5): p. 447-453. Case-mix and Optimization of Mortality Prognostic Scores in Portuguese Pediatric Intensive Care Units PICUs – Pediatric Intensive Care Units PICUs – Pediatric Intensive Care Units

14. State of the Art Introduction Introduction State of the Art State of the Art Background Background Research Questions Research Questions Aims Aims Methods Methods Results References References Acknowledgements Acknowledgements Carmen Ferández, J.L.-H., Jose C. Flores, Dolores Galaviz, Marta Rupérez, Kay B. Brandstrup, Amaya Bustinza, Prognosis in critically ill children requiring continuous renal replacement therapy. IPNA, 2005. Case-mix and Optimization of Mortality Prognostic Scores in Portuguese Pediatric Intensive Care Units

15. Background Introduction Introduction State of the Art State of the Art Are scores discriminative? And calibrated? Background Background Research Questions Research Questions “Both PRISM and PIM scores have a good discriminatory performance. The calibration with PRISM score is good but the PIM score displays poor calibration.” Aims Aims Methods Methods Results References References Acknowledgements Acknowledgements PIM – Pediatric Index of Mortality; PRISM – Pediatric Risk of Mortality Taori RN, L. K., Tullu MS (2010). "Performance of PRISM (Pediatric Risk of Mortality) score and PIM (Pediatric Index of Mortality) score in a tertiary care pediatric ICU." Indian J Pediatr 77(3): 267-271. Case-mix and Optimization of Mortality Prognostic Scores in Portuguese Pediatric Intensive Care Units

16. Background Introduction Introduction State of the Art State of the Art Diferences in calibration... Why? Background Background Research Questions Research Questions Different regions Aims Aims • Population case-mix • Type of PICU • Admissionpolicy • … Missing values Methods Methods Recalibrations Results References References Acknowledgements Acknowledgements PICUs – Pediatric Intensive Care Units Case-mix and Optimization of Mortality Prognostic Scores in Portuguese Pediatric Intensive Care Units

17. Background Introduction Introduction State of the Art State of the Art Should scores be recalibrated? Background Background “ In general, scoring systems should be used only in populations similar to the reference population in which the prediction model was developed.”1 Research Questions Research Questions Aims Aims “Subsequent validation of variables could improve the sensitivity and the value of severity scoring in the future. Nevertheless, illness severity scores will never be indicative of absolute irreversibility of disease or impossibility of survival...”2 Methods Methods Results References References Acknowledgements Wells M, R.-F.J., Luyt DK, Dance M, Lipman J., Poor discriminatory performance of the Pediatric Risk of Mortality (PRISM) score in a South African intensive care unit. Crit Care Med., 1996. 24(9) Unertl K, K.B., Prognostic scores in intensive care. Anaesthesist., 1997. 46(6): p. 471-80. Acknowledgements Case-mix and Optimization of Mortality Prognostic Scores in Portuguese Pediatric Intensive Care Units

18. Background Introduction Introduction State of the Art State of the Art Problems... Background Background Research Questions Research Questions • Inclusion and exclusion criteria of the data-base; • Time of collection; • Human resources; • Algorithm descalibration through time; • … Aims Aims Methods Methods Results References References Acknowledgements Acknowledgements Case-mix and Optimization of Mortality Prognostic Scores in Portuguese Pediatric Intensive Care Units

19. Background Introduction Recalibration Introduction PIM-2 PIM State of the Art PIM-3 State of the Art PRISM III PRISM Background Background Background Assessment and Optimization of Mortality Pronostic Scores in Portuguese Pediatric Intensive Care “Are they doing a good job?” MARTINS, D. A. A. 1; SANTOS, L. R.1; FIGUEIREDO, M. A. R. M.1; GUERRA, M. D. L.1; MAGALHÃES, R. S.1; FRANCISCO, R. M. H.1; REBELO, C. A. B. S. T.1; ABREU, C. A. M. L. M.1; LAIGINHAS, A. R. A.1; DIAS, M. O.1; CHAVES, J. G. A. C.1; OLIVEIRA, R. C. S.1,2; Aim: To evaluate the Pediatric Risk of Mortality (…) and the Pediatric Index of Mortality (…) performance at Portuguese Pediatric Intensive Care Units, where PIM2-R is the recalibrated PIM2 model using the Australia and New Zealand Pediatric Intensive Care 2008 coefficients (which is for the first time applied and analyzed in Portuguese context). Research Questions Research Questions Aims Aims Lack of calibration and discrimination in Portuguese reality Methods Methods Results References References Acknowledgements Acknowledgements PIM – Pediatric Index of Mortality; PRISM – Pediatric Risk of Mortality Case-mix and Optimization of Mortality Prognostic Scores in Portuguese Pediatric Intensive Care Units

20. Expected results Introduction State of the Art PIM AND PRISM underestimated the risk of death in other realities Background We expect... Research Questions Underestimated risk of death in Portugal Aims We look forward to... Methods Results Optimize of the quality of the health care provided References Acknowledgements PIM – Pediatric Index of Mortality; PRISM – Pediatric Risk of Mortality CarmenFerández, J.L.-H., Jose C. Flores, Dolores Galaviz, Marta Rupérez, Kay B. Brandstrup, AmayaBustinza, Prognosis in critically ill children requiringcontinuous renal replacementtherapy. IPNA, 2005. Case-mix and Optimization of Mortality Prognostic Scores in Portuguese Pediatric Intensive Care Units

21. Research Questions Introduction Introduction State of the Art State of the Art Background Background • How do Pediatric Mortality Prognostic Scores (PIM-2) behave in Portuguese PICUs’ specific groups of diagnosis? • Is it possible to enhance the performance of PIM-2 with a new recalibration? • With what purpose are scores used in Portuguese PICUs’? Research Questions Research Questions Aims Aims Methods Methods Results References References Acknowledgements Acknowledgements PICUs – Pediatric Intensive CareUnits; PIM – PediatricIndexofMortality Case-mix and Optimization of Mortality Prognostic Scores in Portuguese Pediatric Intensive Care Units

22. Specificaims Introduction Introduction State of the Art State of the Art • To assess calibration and discrimination of Pediatric Mortality Prognostic Scores in Portuguese PICUs and in the specific diagnosis groups. • To recalibratePIM-2, in the broad-spectrum patients and to specific diagnosis group. • To evaluate the new model concerning calibration and discrimination • To understand clinicians awareness ofthe purpose and use ofprognostic mortality scores in Portuguese PICUs. Background Background Research Questions Research Questions Aims Aims Methods Methods Results References References Acknowledgements Acknowledgements PICUs – Pediatric Intensive CareUnits; PIM – PediatricIndexofMortality Case-mix and Optimization of Mortality Prognostic Scores in Portuguese Pediatric Intensive Care Units

23. Methods Introduction Introduction Target Population State of the Art State of the Art Background Background Research Questions Research Questions The target population will be all children admitted in Portuguese PICUs. Aims Aims Methods Methods Results References References Acknowledgements Acknowledgements PICUs – Pediatric Intensive CareUnits Case-mix and Optimization of Mortality Prognostic Scores in Portuguese Pediatric Intensive Care Units 23

24. Methods Introduction Introduction Sample State of the Art State of the Art Background Sources: PICUs from Hospital de Santa Maria, Hospital de D. Estefânia, Hospital Fernando da Fonseca, Hospital Geral de S. João, Hospital Pediátrico de Coimbra, Hospital Garcia de Orta, Hospital Geral de Santo António, Hospital da Misericórdia de Paredes and Hospital de Faro. Background Research Questions Research Questions Aims Aims Methods Methods Results Children wereincludedin different groups of diagnosis. References References Acknowledgements Acknowledgements PICUs – Pediatric Intensive Care Units Case-mix and Optimization of Mortality Prognostic Scores in Portuguese Pediatric Intensive Care Units

25. Methods Introduction Introduction Sample State of the Art State of the Art Background Age(mean): 2005-2006: 6 y 1 m; 2007-2008: 6 y 1 m. Lenghtof stay(mean): 2005-2006: 182h; 2007-2008: 127h. Background Research Questions Research Questions Aims Aims Methods Methods Results References References Acknowledgements Acknowledgements Case-mix and Optimization of Mortality Prognostic Scores in Portuguese Pediatric Intensive Care Units

26. Methods Introduction Introduction Sample State of the Art State of the Art Background Background Research Questions Research Questions Aims Aims Methods Methods Results References References Acknowledgements Acknowledgements Case-mix and Optimization of Mortality Prognostic Scores in Portuguese Pediatric Intensive Care Units

27. Methods Introduction Introduction Validation State of the Art State of the Art Background Standardized Mortality Ratio (SMR) Calibration Discrimination Background Research Questions Research Questions Aims Aims Methods Methods Results References References Acknowledgements Acknowledgements Case-mix and Optimization of Mortality Prognostic Scores in Portuguese Pediatric Intensive Care Units

28. Methods Introduction Introduction Validation State of the Art State of the Art Background Standardized Mortality Ratio (SMR) Calibration Discrimination Background Research Questions Research Questions Observed deaths / expected deaths Aims Aims Methods Methods Results References References Acknowledgements Acknowledgements Case-mix and Optimization of Mortality Prognostic Scores in Portuguese Pediatric Intensive Care Units

29. Methods Introduction Introduction Validation State of the Art State of the Art Background Standardized Mortality Ratio (SMR) Calibration Discrimination Background Research Questions Research Questions Hosmer-Lemeshow Test Aims Aims Methods Methods Results References References Acknowledgements Acknowledgements Case-mix and Optimization in Mortality Prognostic Scores in Portuguese PICUs Case-mix and Optimization of Mortality Prognostic Scores in Portuguese Pediatric Intensive Care Units

30. Methods Introduction Introduction Validation State of the Art State of the Art Background Standardized Mortality Ratio (SMR) Calibration Discrimination Background Research Questions Research Questions Relative Operating Characteristic (ROC) Curves Aims Aims Methods Methods Results Sensitivity vs. 1-Specificityfor every value of the score References References Acknowledgements Acknowledgements Case-mix and Optimization of Mortality Prognostic Scores in Portuguese Pediatric Intensive Care Units

31. Methods Introduction Introduction State of the Art State of the Art Validation Background Background Standardized Mortality Ratio Calibration Discrimination Research Questions Research Questions Aims Relative Operating Characteristic (ROC) Curves Aims Methods Methods Results References Sensitivity vs. 1-Specificityfor every value of the score References Area under the (AUC) ROC Curve Acknowledgements Acknowledgements Case-mix and Optimization of Mortality Prognostic Scores in Portuguese Pediatric Intensive Care Units

32. Methods Introduction State of the Art State of the Art Background Background Research Questions Research Questions Aims Aims Methods Methods Results References References Acknowledgements Acknowledgements AUC – Area Under the ROC Curve Case-mix and Optimization of Mortality Prognostic Scores in Portuguese Pediatric Intensive Care Units

33. Methods Introduction State of the Art Background Surveys of use Research Questions Main study Aims Methods Results Grounding of conclusions References Acknowledgements https://docs.google.com/spreadsheet/viewform?pli=1&formkey=dHZpbU9hX1h6cFEyWExnTzNzRHplcnc6MQ#gid=0 Case-mix and Optimization of Mortality Prognostic Scores in Portuguese Pediatric Intensive Care Units 33

34. Introduction Results State of the Art Background Research Questions Aims Methods Results References Acknowledgements *1 - Total disagreement, …, 5 - Total agreement

35. Introduction Results State of the Art Background Research Questions Aims Methods Results References Acknowledgements *1 - Total disagreement, …, 5 - Total agreement

36. Introduction Results State of the Art Background Research Questions Aims Methods Results References Acknowledgements *1 - Total disagreement, …, 5 - Total agreement

37. Introduction Results State of the Art Background Research Questions Aims Methods Results References Acknowledgements *1 - Total disagreement, …, 5 - Total agreement

38. Introduction Results State of the Art Background Research Questions Aims Methods Results References Acknowledgements

39. Introduction Results State of the Art Background Research Questions Aims Methods Results References Acknowledgements

40. Introduction Results State of the Art Background Research Questions Sensitivity Aims 2005/2006 Methods Results References 1-Specificity Acknowledgements AUC – Area Under the Curve; PIM – Pediatric Index ofMortality

41. Introduction Results State of the Art Background Research Questions Sensitivity Aims 2007/2008 Methods Results References 1-Specificity Acknowledgements AUC – Area Under the Curve; PIM – Pediatric Index ofMortality

42. Introduction Results State of the Art Background Research Questions Aims Methods Results References Acknowledgements

43. Introduction Results State of the Art Background Research Questions Aims Methods Results References Acknowledgements

44. Introduction Results State of the Art Background Research Questions Aims Methods Results References Acknowledgements

45. Introduction Results State of the Art Background Research Questions Aims Methods Results References Acknowledgements AUC – Area Under the Curve; PIM – Pediatric Index ofMortality

46. Introduction Results State of the Art Background Research Questions Aims Methods Results References Acknowledgements AUC – Area Under the Curve; PIM – Pediatric Index ofMortality

47. Introduction Results State of the Art Background Research Questions Aims Methods Results References Acknowledgements AUC – Area Under the Curve; PIM – Pediatric Index ofMortality

48. Introduction Results State of the Art • New Algorithm: • PIM-2 IM=(-0.015 x SBP)+(4.270 x PUPIL REACTION)+ - 0.011 x mm)+(0.040 x BASE EXCESS)+(- 0.695 x VENTILATION)+(- 3.019 x PLANED ADMISSION)+(-1.099 x POST OPERATORY)+(19.805 x CARDIAC SURGERY)+(0.702 x RECOVERY)+(0.426 x DIGESTIVE) + 2.779 Background Research Questions Aims Methods Results References Acknowledgements

49. Introduction Results State of the Art 2007/2008 Background Sensitivity Research Questions Aims 1-Specificity Methods Results References Acknowledgements AUC – Area Under the Curve; PIM – Pediatric Index ofMortality

50. Introduction Results State of the Art Background Research Questions Aims Methods Results References Acknowledgements