Gabe Peterson, MBA, PMP | Director, Clinical Research Informatics

1. Clinical Research Informatics:Evaluation and Implementation of anElectronic Data Capture System for Clinical Research in an Academic Medical Center Gabe Peterson, MBA, PMP | Director, Clinical Research Informatics Gwen Alofaituli Sale, MPH | Lead Business Systems Analyst, EDC Based on presentation at the Summit on Clinical Research Informatics at AMIA (American Medical Informatics Association) in March 2011, and featured in an International webinar titled “In Search of the Right Electronic Data Capture (EDC) Solution” with Medidata Solutions Worldwide in June 2011

2. Topic Discussion Leader Time Clinical Research Informatics Core Mission Services Clinical Research Informatics Strategic Roadmap Provide context for where EDC fits into the overall strategy EDC Project Timeline Provide overview of major phases of project EDC Evaluation and Selection Process Evaluation criteria Key benefits Gabe 10min EDC Implementation Pilot studies EDC Operational Support Model Study build process EDC study dashboard EDC Governance EDC Workgroup EDC System Integrations Gwen 10min Demonstration Gwen, Gabe 10min Questions & Answers Frequently asked questions Gabe, Gwen 5min Presentation Roadmap

3. CRIC Mission Apply the science of information management to enable the effective and efficient execution of Clinical Research: • Clinical trials • Observational and outcomes research

4. CRIC Services • Service • Description • Key Systems / Projects • Clinical Trials Management Systems (CTMS) • Provide informatics solutions to manage the planning, execution, and reporting of clinical trials, such as protocol authoring and subject enrollment • CTMS Evaluation Project • MIDAS (legacy CTMS) • Clinical Trials Online (CTOL) • Electronic Data Capture (EDC) • Provide electronic data capture (EDC) systems with electronic case report forms (eCRFs) or computer-scannable CRFs to capture study data, such as adverse events and tumor response • Medidata Rave EDC • InTeleScans (built on Teleforms technology) • Clinical Research Decision Support • Transform data to decisions using business intelligence tools to report and analyze study-related data • Clinical Research Portal • SAS • InTeleScan Work Bench • Clinical Trials Online (CTOL) • Medidata Rave EDC (Business Objects) • Automated Lab-Based Adverse Event Grader

5. CRIC Guiding Principles • Enable Research  CRIC is a service organization to enable Research by partnering with researchers to transform data into decisions • Deliver on Commitments  To ensure credibility with researchers, CRIC must deliver on its commitments • Manage the system lifecycle  CRIC applies industry-recognized software lifecycle management best practices • Silos = 0  CRIC uses cross-functional teams to implement and support systems

6. Enabling Research: Research Continuum • Basic Research • Biology • Disease areas • Medicinal chemistry • High throughput screening • Proteomics • Genomics • Translational Research • Safety sciences • Pharmacokinetics and drug metabolism • Biological samples • Immunology • Medical imaging • Pharmaceutical sciences • Clinical Research • Clinical trials • Observational studies • Outcomes research

7. Enabling Research: Informatics Continuum • Discovery Informatics • Sequence Analysis • High throughput screening and microarray data capture and management • Cheminformatics • Translational Res. Informatics • Laboratory information management • Biologic sample inventory management • Electronic lab notebooks • Integrate clinical and basic research data • Clinical Research Informatics • Clinical trials management systems (CMTS) • Electronic Data Capture (EDC) • Clinical Research Decision Support

8. CRIC Organizational Structure

9. CRIC Strategic Roadmap Timeline

10. Implementation Timeline • Phase 1 • Evaluation • EDC project initiated by Biostatistics to support COH in-house trial data needs • Ten EDC/CTMS vendors identified for consideration • Two leading EDC systems selected for deep dive • Phase 2 • Pilot • Conduct pilot of two finalist EDC systems • Medidata Rave selected as recommended system for COH • Phase 3 • Create Buy-In • Medidata approved by COH leadership • Negotiated contract with Medidata • Identified two pilot studies • Phase 4 • Implementation • Trained CRIC staff to build studies in Medidata EDC • Implement two pilot studies • Gene-therapy study (07082) • Breast cancer study (08166) • Offer Medidata EDC as a service to COH investigators in April 2010

11. Evaluation Criteria • Evaluation Dimension • Description • Usability: Interface • Conformance to user requirements. Fit for purpose for data capture and study management • Usability: Reporting • Standard reports and report building tools for decision support • Usability: Scheduling Reports/Exports • Ability to automate delivery of reports and exports • Development Tool Usability • Maturity and flexibility of tools for developing studies • Study Rollout Capabilities • Ability to rapidly promote studies from test to production • Data Integration • Interfaces and web services for integrating with existing COH systems • Help Desk Support • Ability to meet service level agreements • Professional Services • Breadth and depth of service offerings, such as project management and subject matter expertise • Metadata Integration • Ability to export metadata from the system for integration with COH’s metadata repository • Vendor Stability • Vendor’s financial viability and ability to support their product in the future • Cost • Must be in alignment with COH mission as a not for profit medical center • 21 CFR Part 11 Compliance • Conformance to FDA regulations

12. Benefit Business Impact for City of Hope Web-Based Enables multi-site studies with role-based security Rapid deployment of studies and mid-study changes Enables more timely data collection Data Validation Upfront data validation using eCRFs Reduced queries and improved data quality Reporting and Data Analysis Built-in reporting tools to eliminate “data black holes” Integration with SAS for data analysis Patient Enrollment Tracking More efficient tracking of patient enrollment for studies employing the EDC Regulatory Compliance Leading EDC vendor-based products have been used to support drug applications, which is the gold standard for 21 CFR Part 11 compliance EDC Key Benefits

13. Pilot Studies

14. 6 Phases Roles Design Study Team QA BSA Form Dev Form Developers BSA Prog EDC Programmer BSA Report Decision Support BSA UAT Study Team EDC Prog BSA Go Live EDC Programmer BSA EDC Study Implementation Process Development 6 Phases Roles

15. Phase Key Activities Inputs Deliverables Expected Time eCRF Requirements /Design Create mock forms Gather form requirements Create Data Grid Create Data Dictionary Protocol in final draft stages Form Requirements Edit Check Requirements Data Grid Data Dictionary 2-3 weeks eCRF Development Develop eCRFs Data Dictionary complete eCRF PDFs 3-5 days eCRF Programming Create edit checks/custom functions Update eCRFs eCRF PDF approved eCRF development complete Edit Check/ Custom Function Specification Document complete Functioning edit check and custom functions 1-2 weeks Report Development Gather report requirements Create report eCRF development complete Functioning Report 5-8 days User Acceptance Testing (UAT) Complete training (eLearning) Review UAT Plan Study development complete UAT Plan document Signed UAT Plan 5-8 days Production Conduct study-specific EDC Training Go Live UAT Plan sign-off Production URL 1-2 days Study Implementation SOP Summary Approximately 9 Weeks

16. Study Implementation Process Map

17. Completed Pilot Studies in May 2010 and offered the EDC to COH Clinical Researchers EDC Study Implementation Team (also responsible for studies implemented in our legacy EDC system, Teleform) consists of: 2 Business Systems Analysts 2 Forms Developers 2 EDC Programmers 1 Report Developer 11 Protocols Currently in Production 4 NCI-Sponsored Multi-Centered Protocols (CDUS Reporting) Diverse therapeutic indications (Carcinoma, Lymphoma, Vaccine, Diabetes) 9 Protocols Currently in Development EDC Dashboard

18. EDC Workgroup

19. EDC System Integration

20. Role Responsibilities FTEs Person Executive Sponsor Provide research resources and support As needed Joyce Niland Jeff Longmate Project Director Provided technical resources Manages scope 0.1 Gabe Peterson Service Lead Accountable for managing service delivery Manage time and activities of team members 0.2 Gwen Sale Business Lead Provide direction on business requirements 0.2 Adina Londrc Business Systems Analyst Facilitate business requirements sessions Perform and document functional requirements analysis Develop and execute QA and UAT test plans 0.6 0.6 Gwen Sale Jennifer Boesen Technical Lead Creates technical approach, strategy, and design 0.2 Gary Cumming Scientific Developer Develops edit checks and custom functions 0.6 0.4 Gary Cumming Chris Lanning Medidata Rave Resources

21. Demo

22. 1. Q: Can you integrate with other systems? A: Yes. We are currently working on integrating the EDC with the patient registration system, currently MIDAS. Medidata Rave can also integrate with electronic patient diaries, IVRS, laboratory data, ECG and other third-party applications and databases. 2. Q: What steps do I take to get my study into the EDC? A: Send an email along with a copy of your protocol to CRIC_Help@coh.org and a representative from the EDC team will contact you. 3. Q: When should I submit a request to the CRIC team regarding my protocol? A: It normally takes 8-10 weeks to implement your study in the EDC. We recommend that you contact us once your protocol has CPRMC approval and is ready to go to the IRB. 4. Q: How long does it take to deploy a study? A: It normally takes 8-10 weeks to implement your study in the EDC but our timetable will depends on our customer's timeline and the complexity of the study. 5. Q: What are your reporting capabilities? What standard reports are available? A: In addition to numerous standard built-in reports which provide users with study metrics and listings, the EDC also has the ability to provide customizable reports and ad hoc tools for user-driven report creation. 6. Q: What security measures are in place to ensure that no unauthorized users have access to the system? A: The EDC grants access to information only if and when users are allowed to see it, based on their roles and responsibilities. Frequently Asked Questions

23. 7. Q: Are other Cancer Centers using Medidata Rave? A: Yes. The Mayo Clinic and Memorial Sloan Kettering are among the peer Cancer Centers implementing Rave. However, City of Hope has implemented the most studies as of October 2011, based on assessment from Medidata Solutions. 8. Q: Do you look at EDC system X (where X is the questioner’s favorite system)? A: We did deep and broad evaluation of commercial and open source solutions. Medidata Rave is the industry leader, so City of Hope can take assurance that we are using a best-in-class EDC. 9. Q: Did you do a buy-versus-build analysis? A: Yes, build would have been at least 10x more costly. The Clinical Research Informatics strategy is to leverage commercial solutions where available and focus our development on solutions that are not commercially available to give City of Hope researchers a competitive advantage. 10. Q: Is there a chargeback model in place for Medidata Rave or other CRIC services? A: Not currently. However, we are starting to do notional chargebacks (i.e., this is what it would have cost if you had to pay for it) as first step to transitioning to an actual chargeback model in the future, subject to Cancer Center leadership approval. Frequently Asked Questions (continued)

24. Executive Team Michael Friedman, MD President & CEO, City of Hope Richard Jove, PhD Director, Beckman Research Institute John Zaia, PhD Associate Director, Clinical Research Executive Sponsors Joyce Niland, PhD Chair, Department of Information Sciences Jeffrey Longmate, PhD Director, Department of Biostatistics Business Lead Adina Londrc, MD, MPH Director, Clinical Research Information Services PI’s for Pilot Studies BehnamBadie, MD ThehangLuu, MD EDC Work Group Vincent Chung, MD (PI) Edward Newman, PhD (PI) Paul Frankel, PhD (Biostatistician) Suzette Blanchard, PhD (Biostatistician) Adina Londrc, MD, MPH (CRIS) Stella Khoo(Consortium) Dina Johnson (CTO) Dorie Garcia (CRA) Lisa Bening(CRA) Susan Hmwe(Data Management) Gary Cumming (EDC Programmer) Gwen Sale (EDC Lead) Gabe Peterson (CRIC) Thank You!

25. EDC Implementation Team Gabe Peterson, Director, Clinical Research Informatics Gwen Sale, EDC Lead Gary Cumming, Sr. Scientific Programmer Chris Lanning, Sr. Scientific Programmer Jennifer Boesen, Business Systems Analyst Susan Pannoni, Project Manager Thank You! (continued)

26. Questions