Using genomics in clinical trial design
Download
1 / 62

Using Genomics in Clinical Trial Design - PowerPoint PPT Presentation


  • 307 Views
  • Updated On :

Using Genomics in Clinical Trial Design. Richard Simon, D.Sc. Chief, Biometric Research Branch National Cancer Institute http://linus.nci.nih.gov. BRB Website http://linus.nci.nih.gov/brb. Powerpoint presentations and audio files Reprints & Technical Reports BRB-ArrayTools software

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Using Genomics in Clinical Trial Design' - LeeJohn


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Using genomics in clinical trial design l.jpg

Using Genomics in Clinical Trial Design

Richard Simon, D.Sc.

Chief, Biometric Research Branch

National Cancer Institute

http://linus.nci.nih.gov


Brb website http linus nci nih gov brb l.jpg
BRB Websitehttp://linus.nci.nih.gov/brb

  • Powerpoint presentations and audio files

  • Reprints & Technical Reports

  • BRB-ArrayTools software

  • BRB-ArrayTools Data Archive

  • Sample Size Planning for Targeted Clinical Trials


Slide3 l.jpg


Genomic targeting l.jpg
Genomic Targeting the patients to which they are administered

  • Enables patients to be treated with drugs that actually work for them

  • Avoids false negative trials for heterogeneous populations

  • Avoids erroneous generalizations of conclusions from positive trials


Biomarkers l.jpg
“Biomarkers” the patients to which they are administered

  • Surrogate endpoints

    • A measurement made before and after treatment to determine whether the treatment is working

  • Predictive classifiers

    • A measurement made before treatment to select good patient candidates for the treatment


Validation fit for purpose l.jpg
Validation=Fit for Purpose the patients to which they are administered

  • FDA terminology of “valid biomarker” and “probable valid biomarker” are not applicable to predictive classifiers

  • “Validation” has meaning only as fitness for purpose and the purpose of predictive classifiers are completely different than for surrogate endpoints


Slide8 l.jpg


New drug developmental strategy i l.jpg
New Drug predict Developmental Strategy (I)

  • Develop a diagnostic classifier that identifies the patients likely to benefit from the new drug

  • Develop a reproducible assay for the classifier

  • Use the diagnostic to restrict eligibility to a prospectively planned evaluation of the new drug

  • Demonstrate that the new drug is effective in the prospectively defined set of patients determined by the diagnostic


Slide10 l.jpg

Develop Predictor of Response to New Drug predict

Using phase II data, develop predictor of response to new drug

Patient Predicted Responsive

Patient Predicted Non-Responsive

Off Study

New Drug

Control


Applicability of design i l.jpg
Applicability of Design I predict

  • Primarily for settings where the classifier is based on a single gene whose protein product is a target of the drug

    • Herceptin

  • With substantial biological basis for the classifier, it will often be unacceptable ethically to expose classifier negative patients to the new drug


Slide12 l.jpg

We don’t think that this drug will help you because your tumor is test negative. But we need to show the FDA that a drug we don’t think will help test negative patients actually doesn’t


Evaluating the efficiency of strategy i l.jpg
Evaluating the Efficiency of Strategy (I) tumor is test negative. But we need to show the FDA that a drug we don’t think will help test negative patients actually doesn’t

  • Simon R and Maitnourim A. Evaluating the efficiency of targeted designs for randomized clinical trials. Clinical Cancer Research 10:6759-63, 2004.

  • Maitnourim A and Simon R. On the efficiency of targeted clinical trials. Statistics in Medicine 24:329-339, 2005.

  • reprints and interactive sample size calculations at http://linus.nci.nih.gov/brb


Two clinical trial designs l.jpg
Two Clinical Trial Designs tumor is test negative. But we need to show the FDA that a drug we don’t think will help test negative patients actually doesn’t

  • Un-targeted design

    • Randomized comparison of T to C without screening for expression of molecular target

  • Targeted design

    • Assay patients for expression of target

    • Randomize only patients expressing target


Slide17 l.jpg

  • Efficiency relative to trial of unselected patients depends on proportion of patients test positive, and effectiveness of drug (compared to control) for test negative patients

  • When less than half of patients are test positive and the drug has little or no benefit for test negative patients, the targeted design requires dramatically fewer randomized patients


No treatment benefit for assay patients n std n targeted l.jpg
No treatment Benefit for Assay - Patients on proportion of patients test positive, and effectiveness of drug (compared to control) for test negative patientsnstd / ntargeted


Treatment benefit for assay pts half that of assay pts n std n targeted l.jpg
Treatment Benefit for Assay – Pts Half that of Assay + Pts on proportion of patients test positive, and effectiveness of drug (compared to control) for test negative patients nstd / ntargeted


Trastuzumab l.jpg
Trastuzumab on proportion of patients test positive, and effectiveness of drug (compared to control) for test negative patients

  • Metastatic breast cancer

  • 234 randomized patients per arm

  • 90% power for 13.5% improvement in 1-year survival over 67% baseline at 2-sided .05 level

  • If benefit were limited to the 25% assay + patients, overall improvement in survival would have been 3.375%

    • 4025 patients/arm would have been required

  • If assay – patients benefited half as much, 627 patients per arm would have been required


Slide21 l.jpg
Comparison of Targeted to Untargeted Design on proportion of patients test positive, and effectiveness of drug (compared to control) for test negative patientsSimon R,Development and Validation of Biomarker Classifiers for Treatment Selection, JSPI


Randomized ratio sensitivity specificity 0 9 l.jpg
Randomized Ratio on proportion of patients test positive, and effectiveness of drug (compared to control) for test negative patientssensitivity=specificity=0.9


Screened ratio sensitivity specificity 0 9 l.jpg
Screened Ratio on proportion of patients test positive, and effectiveness of drug (compared to control) for test negative patientssensitivity=specificity=0.9


Web based software for comparing sample size requirements l.jpg
Web Based Software for Comparing Sample Size Requirements on proportion of patients test positive, and effectiveness of drug (compared to control) for test negative patients

  • http://linus.nci.nih.gov/brb/


Developmental strategy ii l.jpg

Develop Predictor of on proportion of patients test positive, and effectiveness of drug (compared to control) for test negative patients

Response to New Rx

Predicted Responsive

To New Rx

Predicted Non-responsive to New Rx

New RX

Control

New RX

Control

Developmental Strategy (II)


Developmental strategy ii26 l.jpg
Developmental Strategy (II) on proportion of patients test positive, and effectiveness of drug (compared to control) for test negative patients

  • Do not use the diagnostic to restrict eligibility, but to structure a prospective analysis plan

  • Having a prospective analysis plan is essential; “stratifying” (balancing) the randomization is not except that stratification ensures that all randomized patients will have tissue available

  • The purpose of the study is to evaluate the new treatment overall and for the pre-defined subsets; not to modify or refine the classifier

  • The purpose is not to demonstrate that repeating the classifier development process on independent data results in the same classifier


Analysis plan a l.jpg
Analysis Plan A on proportion of patients test positive, and effectiveness of drug (compared to control) for test negative patients

  • Compare the new drug to the control for classifier positive patients

    • If p+>0.05 make no claim of effectiveness

    • If p+ 0.05 claim effectiveness for the classifier positive patients and

      • Compare new drug to control for classifier negative patients using 0.05 threshold of significance


Sample size for analysis plan a l.jpg
Sample size for Analysis Plan A on proportion of patients test positive, and effectiveness of drug (compared to control) for test negative patients

  • 88 events in classifier + patients needed to detect 50% reduction in hazard at 5% two-sided significance level with 90% power

  • If test is predictive but not prognostic, and if 25% of patients are positive, then when there are 88 events in positive patients there will be about 264 events in negative patients

    • 264 events provides 90% power for detecting 33% reduction in hazard at 5% two-sided significance level

    • Sequential futility monitoring may have enabled early cessation of accrual of classifier negative patients

      • Not much earlier with time-to-event endpoint


Slide29 l.jpg


Analysis plan b l.jpg
Analysis Plan B analysis plan A

  • Compare the new drug to the control overall for all patients ignoring the classifier.

    • If poverall 0.03 claim effectiveness for the eligible population as a whole

  • Otherwise perform a single subset analysis evaluating the new drug in the classifier + patients

    • If psubset 0.02 claim effectiveness for the classifier + patients.


Slide31 l.jpg


Sample size for analysis plan b l.jpg
Sample size for Analysis Plan B for having developed a classifier

  • To have 90% power for detecting uniform 33% reduction in overally hazard at 3% two-sided level requires 297 events (instead of 263 for similar power at 5% level)

  • If test is predictive but not prognostic, and if 25% of patients are positive, then when there are 297 total events there will be approximately 75 events in positive patients

    • 75 events provides 75% power for detecting 50% reduction in hazard at 2% two-sided significance level

    • By delaying evaluation in test positive patients, 80% power is achieved with 84 events and 90% power with 109 events


Song chi refinement of testing procedure for plan b l.jpg
Song & Chi Refinement of Testing Procedure for Plan B for having developed a classifier

  • Specify 1 <  < 1*

    • e.g. =.025, 1=.02, 1*=.10

    • calculate + =.013

  • Reject overall null hypothesis if

    • Poverall≤1 or

    • P+≤ + and Poverall≤

  • Reject null hypothesis in test positive subset if

    • P+≤ + and Poverall≤1*

  • e.g. =.025, 1=.02, 1*=.10, + =.013


Adaptively modifying the types of patients accrued wang o neill hung l.jpg
Adaptively Modifying the Types of Patients Accrued for having developed a classifier Wang, O’Neill, Hung

  • Plan RCT to accrue N total patients

  • Interim futility analysis of test negative patients

  • If accrual of test negative patients is terminated, replace them with test positive patients to achieve the planned total N

    • May prolong duration of trial substantially

  • Futility for test negative patients declared only if efficacy for control group is superior to treatment group by specified amount

    • Limited opportunity to reduce number of test negative patients


Analysis plan c l.jpg
Analysis Plan C for having developed a classifier

  • Test for interaction between treatment effect in test positive patients and treatment effect in test negative patients

  • If interaction is significant at level int then compare treatments separately for test positive patients and test negative patients

  • Otherwise, compare treatments overall


Sample size planning for analysis plan c l.jpg
Sample Size Planning for Analysis Plan C for having developed a classifier

  • 88 events in classifier + patients needed to detect 50% reduction in hazard at 5% two-sided significance level with 90% power

  • If test is predictive but not prognostic, and if 25% of patients are positive, then when there are 88 events in positive patients there will be about 264 events in negative patients

    • 264 events provides 90% power for detecting 33% reduction in hazard at 5% two-sided significance level


Simulation results for analysis plan c l.jpg
Simulation Results for Analysis Plan C for having developed a classifier

  • Using int=0.10, the interaction test has power 93.7% when there is a 50% reduction in hazard in test positive patients and no treatment effect in test negative patients

  • A significant interaction and significant treatment effect in test positive patients is obtained in 88% of cases under the above conditions

  • If the treatment reduces hazard by 33% uniformly, the interaction test is negative and the overall test is significant in 87% of cases


The roadmap l.jpg
The Roadmap for having developed a classifier

  • Develop a completely specified genomic classifier of the patients likely to benefit from a new drug

  • Establish reproducibility of measurement of the classifier

  • Use the completely specified classifier to design and analyze a new clinical trial to evaluate effectiveness of the new treatment with a pre-defined analysis plan.


Guiding principle l.jpg
Guiding Principle for having developed a classifier

  • The data used to develop the classifier must be distinct from the data used to test hypotheses about treatment effect in subsets determined by the classifier

    • Developmental studies are exploratory

      • And not closely regulated by FDA

    • Studies on which treatment effectiveness claims are to be based should be definitive studies that test a treatment hypothesis in a patient population completely pre-specified by the classifier


Use of archived samples l.jpg
Use of Archived Samples for having developed a classifier

  • From a non-targeted “negative” clinical trial to develop a binary classifier of a subset thought to benefit from treatment

  • Test that subset hypothesis in a separate clinical trial

    • Prospective targeted type I trial

    • Using archived specimens from a second previously conducted clinical trial


Development of genomic classifiers l.jpg
Development of Genomic Classifiers for having developed a classifier

  • Single gene or protein based on knowledge of therapeutic target

  • Empirically determined based on evaluation of a set of candidate classifiers

    • e.g. EGFR assays

  • Empirically determined based on genome-wide correlating gene expression or genotype to patient outcome after treatment


Development of genomic classifiers42 l.jpg
Development of Genomic Classifiers for having developed a classifier

  • During phase II development or

  • After failed phase III trial using archived specimens.

  • Adaptively during early portion of phase III trial.


Biomarker adaptive threshold design l.jpg

Biomarker Adaptive Threshold Design for having developed a classifier

Wenyu Jiang, Boris Freidlin & Richard Simon

JNCI 99:1036-43, 2007


Biomarker adaptive threshold design44 l.jpg
Biomarker Adaptive Threshold Design for having developed a classifier

  • Randomized phase III trial comparing new treatment E to control C

  • Survival or DFS endpoint


Biomarker adaptive threshold design45 l.jpg
Biomarker Adaptive Threshold Design for having developed a classifier

  • Have identified a predictive index B thought to be predictive of patients likely to benefit from E relative to C

  • Eligibility not restricted by biomarker

  • No threshold for biomarker determined


Analysis plan l.jpg
Analysis Plan for having developed a classifier

  • S(b)=log likelihood ratio statistic for treatment versus control comparison in subset of patients with Bb

  • Compute S(b) for all possible threshold values

  • Determine T=max{S(b)}

  • Compute null distribution of T by permuting treatment labels

    • Permute the labels of which patients are in which treatment group

    • Re-analyze to determine T for permuted data

    • Repeat for 10,000 permutations


Slide47 l.jpg


Adaptive biomarker threshold design l.jpg
Adaptive Biomarker Threshold Design the permutation null distribution of T, then reject null hypothesis that E is ineffective

  • Sample size planning methods described by Jiang, Freidlin and Simon, JNCI 99:1036-43, 2007


Slide51 l.jpg

Adaptive Signature Design the permutation null distribution of T, then reject null hypothesis that E is ineffective An adaptive design for generating and prospectively testing a gene expression signature for sensitive patients

Boris Freidlin and Richard Simon

Clinical Cancer Research 11:7872-8, 2005


Adaptive signature design end of trial analysis l.jpg
Adaptive Signature Design the permutation null distribution of T, then reject null hypothesis that E is ineffective End of Trial Analysis

  • Compare E to C for all patients at significance level 0.03

    • If overall H0 is rejected, then claim effectiveness of E for eligible patients

    • Otherwise


Slide53 l.jpg

  • Otherwise: the permutation null distribution of T, then reject null hypothesis that E is ineffective

    • Using only the first half of patients accrued during the trial, develop a binary classifier that predicts the subset of patients most likely to benefit from the new treatment E compared to control C

    • Compare E to C for patients accrued in second stage who are predicted responsive to E based on classifier

      • Perform test at significance level 0.02

      • If H0 is rejected, claim effectiveness of E for subset defined by classifier


Slide54 l.jpg
Treatment effect restricted to subset. the permutation null distribution of T, then reject null hypothesis that E is ineffective 10% of patients sensitive, 10 sensitivity genes, 10,000 genes, 400 patients.


Slide55 l.jpg
Overall treatment effect, no subset effect. the permutation null distribution of T, then reject null hypothesis that E is ineffective 10% of patients sensitive, 10 sensitivity genes, 10,000 genes, 400 patients.


Conclusions l.jpg
Conclusions the permutation null distribution of T, then reject null hypothesis that E is ineffective

  • New technology makes it increasingly feasible to identify which patients are likely or unlikely to benefit from a specified treatment

  • Targeting treatment can greatly improve the therapeutic ratio of benefit to adverse effects


Conclusions57 l.jpg
Conclusions the permutation null distribution of T, then reject null hypothesis that E is ineffective

  • Some of the conventional wisdom about how to develop predictive classifiers and how to use them in clinical trial design and analysis is flawed

  • Prospectively specified analysis plans for phase III studies are essential to achieve reliable results

    • Biomarker analysis does not mean exploratory analysis except in developmental studies

    • Prospective analysis of previously conducted trials can provide reliable conclusions


Conclusions58 l.jpg
Conclusions the permutation null distribution of T, then reject null hypothesis that E is ineffective

  • Achieving the potential of new technology requires paradigm changes in “correlative science” and in some aspects of design and analysis of clinical trials


Collaborators l.jpg
Collaborators the permutation null distribution of T, then reject null hypothesis that E is ineffective

  • Boris Freidlin

  • Aboubakar Maitournam

  • Kevin Dobbin

  • Wenu Jiang

  • Yingdong Zhao


Using genomic classifiers in clinical trials l.jpg
Using Genomic Classifiers In Clinical Trials the permutation null distribution of T, then reject null hypothesis that E is ineffective

  • Dupuy A and Simon R. Critical review of published microarray studies for clinical outcome and guidelines for statistical analysis and reporting, Journal of the National Cancer Institute 99:147-57, 2007

  • .

  • Dobbin K and Simon R. Sample size planning for developing classifiers using high dimensional DNA microarray data. Biostatistics 8:101-117, 2007.

  • Dobbin K, Zhao Y and Simon R. How large a training set is needed to develop a classifier for microarray data? Clinical Cancer Research (In Press).

  • Simon R. Development and validation of therapeutically relevant predictive classifiers using gene expression profiling, Journal of the National Cancer Institute 98:1169-71, 2006.

  • Simon R. Validation of pharmacogenomic biomarker classifiers for treatment selection. Cancer Biomarkers 2:89-96, 2006.

  • Simon R. Guidelines for the design of clinical studies for development and validation of therapeutically relevant biomarkers and biomarker classification systems. In Biomarkers in Breast Cancer, Hayes DF and Gasparini G, pp 3-15, Humana Press, 2006.

  • Simon R. A checklist for evaluating reports of expression profiling for treatment selection. Clinical Advances in Hematology and Oncology 4:219-224, 2006.

  • Simon R. Identification of pharmacogenomic biomarker classifiers in drug development. In Pharmacogenomics, Anti-cancer Drug Discovery and Response, F Innocenti (ed), Humana Press (In Press).

  • Simon R. New challenges for 21st century clinical trials, Controlled Clinical Trials 4:167-169, 2007.

  • Simon R. Development and validation of biomarker classifiers for treatment selection, Journal of Statistical Planning and Inference 138:308-320, 2008.


Using genomic classifiers in clinical trials61 l.jpg
Using Genomic Classifiers In Clinical Trials the permutation null distribution of T, then reject null hypothesis that E is ineffective

  • .

  • Simon R and Maitnourim A. Evaluating the efficiency of targeted designs for randomized clinical trials. Clinical Cancer Research 10:6759-63, 2004.

  • Maitnourim A and Simon R. On the efficiency of targeted clinical trials. Statistics in Medicine 24:329-339, 2005.

  • Simon R. When is a genomic classifier ready for prime time? Nature Clinical Practice – Oncology 1:4-5, 2004.

  • Simon R. An agenda for Clinical Trials: clinical trials in the genomic era. Clinical Trials 1:468-470, 2004.

  • Simon R. Development and validation of therapeutically relevant multi-gene biomarker classifiers. Journal of the National Cancer Institute 97:866-867, 2005..

  • Simon R. A roadmap for developing and validating therapeutically relevant genomic classifiers. Journal of Clinical Oncology 23:7332-41,2005.

  • Freidlin B and Simon R. Adaptive signature design. Clinical Cancer Research 11:7872-78, 2005.

  • Simon R. and Wang SJ. Use of genomic signatures in therapeutics development in oncology and other diseases, The Pharmacogenomics Journal 6:166-73, 2006.

  • Trepicchio WL, Essayan D, Hall ST, Schechter G, Tezak Z, Wang SJ, Weinreich D, Simon R. Designing prospective clinical pharmacogenomic trials- Effective use of genomic biomarkers for use in clinical decision-making. The Pharmacogenomics Journal 6:89-94,2006.

  • Simon R. Designs and adaptive analysis plans for pivotal clinical trials of therapeutics and companion diagnostics. Expert Opinion in Medical Diagnostics (In Press)

  • Simon R. Using genomics in clinical trial design. Clinical Cancer Research (In Press).


Using genomic classifiers in clinical trials62 l.jpg
Using Genomic Classifiers In Clinical Trials the permutation null distribution of T, then reject null hypothesis that E is ineffective

  • .

  • Simon R Challenges of microarray data and the evaluation of gene expression profile signatures. Cancer Investigation (In Press)

  • Simon R. Lost in translation: Problems and pitfalls in translating laboratory observations to clinical utility. European Journal of Cancer (In Press)


ad