Strength in numbers: Finding meaning in Big Data ( and Little Data too ) Dana E. Connors

1. Strength in numbers: Finding meaning in Big Data (and Little Data too) Dana E. Connors 2017 Project Management Symposium Tuesday, September 12, 2017 Sheraton Tysons Hotel

2. Information

3. Instruction

4. Big Data

5. Big data is…

6. Big data is … Insert your data here

7. Summary How do we organize Big Data? Step 1: Establish an objective.

8. Step 1. Establish an objective

9. Summary How do we organize Big Data? Step 1: Establish an objective. Step 2: Create a plan.

10. Step 2. Create a plan

11. jake.and.franny

12. Summary How do we organize Big Data? Step 1: Establish an objective. Step 2: Create a plan. Step 3: Understand what you have.

13. trello.com

14. 3. Understand what you have

15. Week 1

16. Week 2

17. Week 3

18. 3 week average

19. What do we have?

20. What do we have?

21. Summary How do we organize Big Data? Step 1: Establish an objective. Step 2: Create a plan. Step 3: Understand what you have. Important Lesson Number One: Do not spend your time proving to yourself what you already know.

22. Summary How do we organize Big Data? Step 1: Establish an objective. Step 2: Create a plan. Step 3: Understand what you have. Important Lesson Number One: Do not spend your time proving to yourself what you already know. Step 4: Initiate appropriate analysis.

23. Foundation for the National Institutes of Health $1billion raised by the FNIH since 1996 1996 established by Congress to support NIH in its mission by 501(c)(3) Non-governmental not-for-profit & independent Board of Directors $0.94 $0.93 advancing biomedical research and training collaborations Over 550 projects supported 93% of funds directly support programs 120+ active research partnerships, scientific education/training, conferences/events and capital programs 13 years of outstanding Charity Navigator ratings among government, universities, industry and not-for-profit organizations

24. Biomarkers Consortium • Mission: To discover, develop, and seek regulatory approval for biomarkers to support and accelerate development of new drugs, preventive medicine, and medical diagnostics by combining the forces of the public and private sectors. • Fosters the exchange of knowledge and expertise among industry, academic, and government leaders • Works to qualify biomarkers for specific applications in diagnosing disease, predicting therapeutic response, and improving clinical practice • Generates information useful to inform regulatory decision-making • Employs rigorous, inclusive governance and project management with clearly defined goals and milestones • Addresses a broad range of disease / therapeutic areas • Pre-competitive; makes consortium project results broadly available to the entire scientific community

25. Biomarkers Consortium

26. Biomarkers Consortium Contributing Partners

27. Biomarkers Consortium Governance Structure Executive Committee NIH / FDA / CMS / Industry / FNIH Metabolic Disorders Steering Committee Neuroscience Steering Committee Cancer Steering Committee Inflammation & Immunity Steering Committee Multiple Project Teams Representatives from NIH, FDA, Industry, Subject experts from academia

28. ProjectDevelopmentProcess IdeasaresubmittedandreviewedbySteeringCommittees WorkingGroup,EC/SC RFA/RFP,or External SubmissiontoSC Biomarkers Consortium Executive Committee FNIH Development Group (workingwith ProjectTeam) Steering Committee/Project Team Steering Committee ProjectTeam FNIH 2 1 6 5 3 4 Approved Project Concept Project Plan Approved Project Project Concept FNIHFundraising Launch Projectplansare developed byProjectTeamsand reviewedbytheSteeringCommitteesandExecutiveCommittee Projectsare launchedafterfundingissecuredbyFNIH

29. CSC in Development • Concepts in Development • Immunotherapy Proteogenomics (IOPG) • To be re-presented to the CSC in Q3 2017 • HD-SCA – 2 • HD-SCA Project team developing a proteomics phase II proposal for review by CSC in Q4 2017 • MRD in Multiple Myeloma (MRD in MM) • Whitepaper defining the landscape of MRD in MM published by FNIH working group in Clinical Cancer Research • Data aggregation for conducting a per patient meta-analysis was defined as the highest priority • Single cell mass accumulation rate (MAR) as an ex vivo biomarker to predict drug susceptibility (SMR) • Re-present concepts to the CSC in Q4 2017 • Prostate Cancer Study of Active Surveillance in Low Grade Prostate Cancer • Working Groups • High Density Content Integration • Whitepaper in development on applications of high content, single cell analysis in clinical studies submitted to Science Translational Medicine • Clinical Utility of Liquid Biopsy • Clinical studies geared to establish clinical validity and clinical utility for key assays using ctDNA, CTC and/or exosomes. Present to the CSC in Q1/2 2018 • Radiomics

30. ProjectDevelopmentProcess IdeasaresubmittedandreviewedbySteeringCommittees WorkingGroup,EC/SC RFA/RFP,or External SubmissiontoSC Biomarkers Consortium Executive Committee FNIH Development Group (workingwith ProjectTeam) Steering Committee/Project Team Steering Committee ProjectTeam FNIH 2 1 6 5 3 4 Approved Project Concept Project Plan Approved Project Project Concept FNIHFundraising Launch Projectplansare developed byProjectTeamsand reviewedbytheSteeringCommitteesandExecutiveCommittee Projectsare launchedafterfundingissecuredbyFNIH

31. CSC Active Portfolio • Active Projects • FDG-PET Lung and Lymphoma Trials (FDG-PET) • FDA submission packet to qualify FDG-PET as a response biomarker will be submitted in Q4 2017. • Minimal Residual Disease in Acute Lymphoblastic Leukemia (MRD in ALL) • FDA submission packets are being prepared to have COG 6-color assay recognized as predicate data for newly developed 8-color flow cytometric assay. Manuscript was published in Cytometry on 05/2017. FDA working on per patient MRD meta-analysis to build data for MRD qualification. • High Density Single-Cell Analysis in Metastatic Colorectal Cancer (HD-SCA in mCRC) • Team accrued necessary patients with a CTC-positive draw and is demonstrating performance of a 4 color assay. • Advanced Metrics and Modeling with Volumetric CT for Precision Analysis of Clinical Trial results (Vol-PACT) • Analysis of 4 existing data sets is ongoing, with imaging measurement of an additional 3 along with continued data acquisition efforts. Methods manuscript being prepared for submission in Q3 2017. • Approved Plans • Chemotherapeutic Impact on the Immune Microenvironment: Determining the impact of chemotherapy on tumor immunity by systematic dissection of the tumor microenvironment with single cell genomics (ChIIME) • EC Approved. Fundraising just beginning late Q2 2017, seeing definite interest in the science and a recognized need for the work. • Developing an Analytically and Clinically Validated Reference Material for ctDNA Testing (ctDNA) • The project plan was approved for funding at the 8/16/2017 EC meeting.

32. Vol-PACT • Vol-PACT: Advanced Metrics and Modeling with Volumetric CT for Precision Analysis of Clinical Trial results • Public Private Partnership: Industry, NCI, FDA, Academia and Research Institutions • In-Kind Donations of Phase II and III Trial Data from Industry

33. The Vol-PACT Objective • Develop a systematic approach to compare RECIST and irRECIST, assist in standardizing irRECIST, and validate alternative imaging metrics for drug development using original trial images for tumor measurements • Address Drug Failure: A review of 253 phase III trials of drugs for treatment of solid tumors published between 2005 and 2009 showed that 158 (62.5%) were negative.* • millions of dollars • thousands of patients • A large proportion of negative phase III trials suggests a failure of phase II trial analysis in oncology • This is potentially stemming from inadequate endpoints (progression free survival and overall survival) • *Gan HK, You B, Pond GR, et al: Assumptions of Expected Benefits in Randomized Phase III Trials Evaluating Systemic Treatments for Cancer. J Natl Cancer Inst 104:590-598, 2012

34. RECIST Criteria Cross-product (WHO) 2D Diameter (RECIST) 1D Volume 3D

35. Hypothesis • Quantitative analysis of tumor response as a continuous variable will improve the ability of randomized phase II trials to accurately predict phase III results • Detailed assessment of the entire tumor burden using volumetric CT will improve efficiency and accuracy of phase II trial analysis

36. Summary How do we organize Big Data? Step 1: Establish an objective. Step 2: Create a plan. Step 3: Understand what you have. Important Lesson Number One: Do not spend your time proving to yourself what you already know. Step 4: Initiate appropriate analysis. Step 5: Assemble the right team.

37. Vol-PACT Project Team Private Sector Public Sector Image analysis EMD Serono Data Genentech Regeneron Takeda Statistical analysis Amgen NCI BI Merck FDA Clinical relevance Sanofi GSK / Novartis Columbia University Inova Dana Farber MSKCC Funding NIH FDA Government

38. Vol-PACT Project Process Company Funding and Data Sharing • FNIH Company Imaging Data Company Clinical Data • Processed Imaging Data • CRO • Columbia University • Inova Scher Cancer Research Institute • Memorial Sloan Kettering Cancer Center • Dana Farber Cancer Institute • Imaging Data Receipt, Volumetric Assessments and Analysis • Project Oversight and Data Analysis • Statistical and Clinical and Processed Imaging Data Analysis • Clinical Data Receipt, Processing, and Data Quality • Processed Clinical Data

39. Data Sharing Agreements

40. Agreements Required CDA – Confidentiality Agreement (as needed) MOU - Memorandum of Understanding (2) LoA - Letters of Agreement (7) DSA - Data Sharing Agreement (5) MTA – Material Transfer Agreement (5) RCA - Research Collaboration Agreement (4)

41. IP and Publication Considerations

42. Summary How do we organize Big Data? Step 1: Establish an objective. Step 2: Create a plan. Step 3: Understand what you have. Important Lesson Number One: Do not spend your time proving to yourself what you already know. Step 4: Initiate appropriate analysis. Step 5: Assemble the right team. Important Lesson Number Two: Communicate. Communicate. Communicate.

43. Data Transfer • patient de-identification • offsetting of key data values for anonymization • test data transfer • data transferred separately in three files: • the patient-level de-identified DICOM images • baseline clinical data recorded in CRFs • study-specific clinical data including measurements and observations • source anonymization key destroyed

44. Data Access

45. Summary How do we organize Big Data? Step 1: Establish an objective. Step 2: Create a plan. Step 3: Understand what you have. Important Lesson Number One: Do not spend your time proving to yourself what you already know. Step 4: Appropriate analysis. Step 5: Assemble the right team. Important Lesson Number Two: Communicate. Communicate. Communicate. Step 6: Insure Data Quality

46. Quality Control

47. Quality Control • data dictionary including study ID, subject ID, and date • initial quality check: • number of subject provided against publications • readability / measurability of the DICOM images • number of subjects and scan time points • each subject ID in the transmittal form matches the DICOM imaging files and the clinical metadata files and vice versa • once these initial quality control steps complete imaging processing/analysisbegins

48. Data Analysis • each scan at each time point is measured using semi-automated algorithms (diameter, perpendicular diameter, and volume) • fields provided to statistical team: • Subject ID • CIAL ID • Study Date • Target Lesion # • Target Lesion Site • Uni (mm) • Perp (mm) • Bi (mm2) (the product of the maximal diameter and the maximal perpendicular diameters) • Volume (mm3) • Comment • (patients who had no target lesions measured at baseline were excluded)

49. Imaging Data Studied Number of patients analyzed changes primarily because: 1) no target lesions found at baseline 2) only one time-point scan 3) secondary captured images

50. Developing Metrics new endpoints generated systematically by aggregating three types of metrics: lesion measurement (longest-diameter, bi-dimensional and volumetric) and log-transformation of lesion measurement time points (two fixed points, maximum and average over the study duration) including baseline and first cycle or second cycle or midpoint of planned treatment (6 months if the plan is to treat until progression) aggregation (of time burden, lesion profile, and heterogeneity in response) the transition from Phase II to Phase III trials is simulated by generating 1,000 randomized Phase II trials from a given Phase III data.