slide1
Download
Skip this Video
Download Presentation
Cross-trial estimation of control effect and false positive issues in active-control trials

Loading in 2 Seconds...

play fullscreen
1 / 33

Cross-trial estimation of control effect and false positive issues in active-control trials - PowerPoint PPT Presentation


  • 153 Views
  • Uploaded on

Cross-trial estimation of control effect and false positive issues in active-control trials. Abdul J Sankoh, PhD sanofi-aventis Bridgewater, NJ 08807, USA [email protected] Tel: (908) 231 2825; fax: (908) 231 2151. Presentation Outline.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Cross-trial estimation of control effect and false positive issues in active-control trials' - fathi


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
slide1
Cross-trial estimation of control effect and false positive issues in active-control trials

Abdul J Sankoh, PhD

sanofi-aventis

Bridgewater, NJ 08807, USA

[email protected]

Tel: (908) 231 2825; fax: (908) 231 2151

Active control trials: Princeton-Trenton ASA-2006

presentation outline
Presentation Outline

Background and presentation motivation

 Types of active-control study designs

Notation and hypotheses setting

Cross-trial estimation of control effect

 Properties & simulation results summary on efficiency of two most popular (PE&CI) methods for quantifying inferiority margin

Other plausible approaches- an overview

 Multiplicative odds ratio, a real clinical data example

 Bayesian Modeling- an example

1 background
1. Background

In a two-arm active-control trial,

•evensuperiority of experimental drug to active-control doesn’t guarantee superiority of either drug to placebo!

To minimize chances of approving drugs not different than placebo, active-control candidate must have

•Substantialhistorical clinical and statistical evidence from R, DB, PC, and well-conducted trials

° that show similarly designed and conducted trials of active-control regularly demonstrated superiority to placebo with substantial and similar treatment effect sizes,

Assay sensitivity of the trial!

2 types of active control designs
2. Types of active-control designs

Two-arm active-control design (E vs. C) – most common:

•Assay sensitivity is not measurable in the active trial:

 cannot show study’s ability to distinguish active from inactive therapy.

Three-or more-arm active-control study design- not common:

Dose-ranging active-control study design (E1, …, EK vs. C, 2K, where E1 is safe but sub-optimal dose)

Concurrent active- and placebo-control study design (E vs. P; C vs. P; E vs. C).

• Assay sensitivity is measurable in the active trial:

 can show study’s ability to distinguish active from inactive therapy –Generally preferred by agencies!

assay sensitivity
Assay sensitivity:-

R. Temple: Fundamental problems with non-inferiority/ equivalence trials: “Critical assumption that trial had ‘assay sensitivity’”

This assumption:

• not necessarily true for all effective drugs

•not directly testable in the (2-arm) active-control trial

• thus requires an active-control study to have elements of a historically controlled study.

3 notation formulation
3. Notation & formulation
  •  = event rate = Pr(Event occurs|Treatment) –

P = Pr(Event occurs|P) - placebo group response rate (historical)

C = Pr(Event occurs|C) - control group response rate (both)

E = Pr(Event occurs|E) – experimental group response rate

 Define treatment difference :

C-P = C - P = control effect (efficacy of C)- historical trials

E-C = E - C = experimental effect (relative efficacy of E)- acive

E-P = E-C+C-P= treatment effect (E effect)- cross-trial inference

some notation formulation
Some notation & formulation

 Note: Treatment = Control + Experimental

(E - P) = (C - P) + (E - C) i.e, E-P = C-P + E-C

 E-C = E-P - C-P __________________________________________________________

 Let N = M - C<0 = non-inferiority margin,

Mis (imaginary) response level at boundary of acceptable and unacceptable level of inferiority

i.e., N = maximum allowable loss of efficacy associated with E relative to C in active trial.

hypotheses formulation on the e c e c
Hypotheses formulation on the E-C = (E-C)

 = event rate = Pr[event occurs|treatment] –

Non-inferiority: H0: E-CN vs. HA: E-C < N

Superiority: H0: E-C 0 vs. HA: E-C < 0

 E is non-inferior E is inferior 

|=================|===========)||

0 N

 E is superior E is not different or worse 

|=========|========|||

-S 0

accounting for p in m percent of c effect retained to ensure efficacy of e assume small is better
Accounting for P in M: Percent of C effect retained to ensure efficacy of E : Assume small  is better

• Effect of C: C - P = C-P;

• Effect of E: E - P= E-P; let (0,1)

• Proportion of C effect retained by E:  = C-P/E-P

• Proportion of C effect lost by E: 1- =E-C/E-P

Ex: C-P = 15% & E-P = 20%; so = 15/20 =75%; 1- = 25%.

HA: (E-P)< (C-P) – (E retains at least % of C effect)

E-C> (1-)C-P – (E loses  (1-)% of C effect)

• Since M-C= (1-)C-P,

HA: C-E> (1-)M-C – (E effect relative to C  (1-)%)

E-C < N

4 two most popular approaches for c hoosing non inferiority margin n
4. Two most popular approaches forchoosing non-inferiority margin N

Assume statistical approach is toconclude non-inferiority of E to C if N < LL of 100(1-2)% confidence interval (CI) on E-C.

 Generally, estimate C-P from historical studies (meta-analysis) and choose N using either:

Point estimate (PE): |N| r|C-P|, 0

CI: N < LL of 100(1-’)% CI on C-P (1%  ’2.5%)(useful in addressing within study variability)

Efficiency issue: PE is too liberal,CI too conservative for controlling type I error rate for concluding E effect!

slide13
Efficiency and type I error rate with estimation of C and E effect in active trial using PE approach -

 Point estimate (PE): |N| r|C-P|, 0

- efficient under constancy and 100% retention of control effect

- but type I error rate inflation for concluding efficacy of E if constancy assumption not tenable or C effect in active trial is less than historical trials(see Fig. 1)

 In general, type I error rate for concluding E effect w/ 97.5% PE

Pr[E-P  C-P|H0]  1-(1.96f); E[|H0]= 

f = [1+{(1-)0-1E-C}2]-1/2<1

 20=2C-P estimate of control effect variance from historical data.

 2E-C estimated from active trial;  is observed &  truediff.

slide14
Fig. 1.- Overall type I error rate w/ PE approach for concluding efficacy of E and non-inferiority to C for given % retention and efficiency ratio(C-P/E-C= 1.5, 1.1, 1.0, 0.91, 0.67)
slide15
Efficiency and type I error rate for estimation of C and E effect in active trial using CI approach-Preferred by Agencies?(Hung et al, Wang et al, Chen et al, Tsong et al, Rothman et al, Ng et al)

 Confidence interval (CI): N< LL of 100(1-’)% CI on C-P; - ultra-conservative even with constancy of control effect

- generally serious deflation of type I error rate unless 100% retention

of control effect is achieved (Fig. 2).

 In general, type I error rate for concluding E effect w/97.5% CI

Pr[E-P  C-P|H0]  1- (1.96h); E[|H0]= 

h = [1+(1- )0-1E-C][1+{(1-)0-1E-C}2]-1/2 >1

 20=2C-P estimate of control effect variance from historical data

 2E-C estimated from active trial;  is observed &  truediff.

slide16
Fig. 2.- CI approach overall type I error rate for concluding efficacy of E and non-inferiority to C for given % retention and efficiency ratio(C-P/E-C=1.1, 1.0, 0.91, 0.67)
summary of simulation results figs 1 2
Summary of simulation results (Figs. 1&2)

 In general, nominal type I error rate () is maintained if

Historical control effect (0C-P)=Active control effect (C-P)!

 Generally, when N is fixed known constant,

Pr[E-P  C-P|H0]  ; E[|H0]= 

For both methods, efficiency of active study highly dependent on external factors - outside control of active study.  

 For PE, higher historical (C) variability vs. E more liberal.

 For CI, higher historical (C) variability vs. E more conservative.

 For both, smaller historical (C) variability vs. E more efficient active study.

any plausible less conservative approach critical path initiative
?Any plausible less conservative approach – Critical Path Initiative

FDA Considering "Less Conservative" Approaches To Non-Inferiority Trials - PINK-SHEET, June 27, 2005, pp 10.

• "One of the things we've been thinking about is whether there are somewhat less conservative approaches" to non-inferiority trials "than we've been inclined to use," Temple said at a 6/15/2005 FDA's Cardio-Renal ACM.

• FDA has "thought about things like narrowing the confidence interval for certain measures, using less stringent insistence on convention in a wide variety of ways".

• One way to narrow confidence intervals would be "to incorporate prior data".

 BAYESIAN approach?

slide19
Numerical Ex 2:- ESSENCE- Active control, DB, R, PG study comparing Enoxoparin+Aspirin vs. Heparin + Aspirin in UA pts. 2ndary Endpoint: Composite of Death or MI @ Day 14.

 2-sided p-value = 0.019 for OR of E+A vs. H+A (for primary composite endpoint of recurrent angina, MI, or death).

 For 2ndary composite endpoint of MI or death:

• pH+A = 96/1564 = 0.06 = Pr(MI or Death|C  H+A)

• pE+A =79/1607 = 0.05 = Pr(MI or Death|E  E+A)

• OR(E+A) v (H+A)=0.791; 95% CI (0.582,1.074); p-value =0.132.

Q: Is E+A better than A alone, had there been A arm in active trial?

 To answer, cross-trial estimation of Aspirin response?

slide20
Numerical Examples:1. ESSENCE- Active control, DB, R, PG study comparing Enoxoparin+Aspirin vs. Heparin + Aspirin in UA pts. 2ndary Endpoint: Composite of Death or MI @ Day 14.

▪ Used meta-analysis to estimate H+A ( C0 ) effects from historical data on reduction in Death or MI in UA patients.

Incidence of MI or Death and associated OR from 6 historical studies

Historical Data Active

Study Aspirin H+A H+A E+A

(A  P) (H+A  C0) (C) (E)

1. 4/121 2/12 96/1564 79/1607

2. 7/189 3/210 (0.06) (0.05)

3. 1/32 0/37 -------------------------

4. 9/109 4/105 OR(E+A) vs. (H+A)= 0.791

5. 40/131 42/154 95% CI: (0.58, 1.07)

6. 7/73 4/70 2-sided p-value = 0.132

Total 68/655 55/698 (0.104) (0.079)

 OR(H+A) vs. A =0.665; 95% CI: (0.443, 0.992); p=.045 (6 22 tables - StatXact)

slide21
Ex 2: Active control, DB, R, PG study comparing E+A vs. H+A2nd Endpoint: Composite of Death or MI @ Day 14.

Multiplicative Odds Ratio (OR) - Epidemiology

To estimate OR of E+A vs. A alone

OR(E+A) vs. A = [OR(E+A) vs. (H+A)][OR(H+A) vs. A]

Active Historical

= 0.791 0.665 = 0.52

 95% CI on log(OR) – (0.31, 0.89);

2-sided p-value = 0.016 (ESSENCE Study FDA Statistical review, 1998).

bayesian modeling in two arm active design
?Bayesian Modeling in two-arm active design

Bayesian:- Been used in traditional two-arm active-control designs to show implicit superiority of test drug over (putative) placebo (Simon, Biometrics 1999).

 Determine (predictive) posterior probabilities based on appropriate priors and conclude non-inferiority if predictive posterior probability greater than 0.975.

 If Pr[E effect relative to putative P|data]  0.95, then E>>P

 Pr[C effect +ve & E preserves  100 (1-)% C effect]0.975,

then conclude E preserves  (1-)% of C effect.

 Compute one-sided p-value = 1 - posterior probability.

p redictive posterior models for calculation of posterior probabilities
Predictive (posterior) models for calculation of posterior probabilities

Ex: One sample predictive (posterior) categorical model:

 From prior clinical info, estimate N = 30 (1:1 to 2 treatment grps).

▪ Partition sample space:(yj1, yj2, yj3) = (xj1, xj2, xj3) + (zj1, zj2, zj3) (y’s categorical & exchangeable) observedunobserved

Assume yjr|pj = pjr & pjr~Dirichlet(jr); r=1,…,M

Given sample info xj = (xj1, xj2, xj3); 1r3xjr= nj, then

yjr|(pj, xj) Dir(jr+xjr).

▪pjr jr/j, j=j1+…+jM – prior to sample info

▪ pjr|x (jr+xjr)/(j+nj), j+nj = j1+xj1…+jM+xjM,- given sample

ex one sample of the c ategorical model continued
Ex:One sample of the Categorical model- continued

 Assume at interim look, of 7 randomized and treated with treatment j, we observe xj=(2, 4, 1). If jr =2, then

pjr|j {0.333} =jr/j, = {2/(2+2+2)}, forall r; is our prior estimate before sample info, and

pj|(j,x) (.307, .462, .230) = {(j1+xj1)/(j+nj)} is our updated estimate given sample info xj=(2, 4, 1)

 Compare with frequentist’s estimate pj= (.286, .571, .143), jr=0.

 For Nj=20, z=(5, 5, 3),

{j} = (1, 3, 2);

{jr/j}= (.167, .500, .333); {(j1+xj1)/(j+nj)} = (.133, .467, .267)

slide25
Fig. 3.-For N=15, predictive PP[z|x;] vs. multinomial Mn[z;p] future success probabilities for prior info: a=(2, 2, 2) & b=(2, 3, 1); sample info x = (2, 4, 1); future success z=(z11, z12, z13), z11+z12+z13=8
ex extension to non categorical model
Ex: Extension to non-categorical model-

yjk = outcome of kth subject in treatment j (j=1,2 and k=1,…, Nj), yjk any value on real line, R.

  • yj= (yj1,…, yjN1) is a random sample from Fj

If Fj j(.)/ j(R), a Dirichlet process with shape measure j(.) defined on R, j(R) = j(-, )

  • Then yjis Dirichlet process with shape measure j(.)
  • Given xj= (x11…, x1n1) sample information,

yj|xDir(j(.)+nj(.)), nj(.) = kx(.), x unit mass at x

strength of prior clinical evidence
Strength of prior clinical evidence

(R)  0 is measure of strength of prior clinical evidence (e.g., Phase I/IIa, historical trial sample size)

  • • (R) = 0 reflecting complete lack of clinical information.
  • • Thus (R) = (0, )
  • • (R) could be dominated by sample information
posterior estimates
Posterior estimates

Let g() denote mean treatment difference:

g() = [(y11+…+y1N1)/N1 - (y21+…+y2 N2)/N2].

  • • n = E[g()|x] = n+ (1-)0,01= (mean of observed)

+ (1-)(mean of unobserved) –

 *(stage 1 estimate)+(1-*)(stage 2 estimate) - frequentist‘s weighted statistic

• n = Var[g()|x] = w1s2 + w2(R)(n-0)2) + w32 = wns2 for (R) = 0 . Where s2 & 2 = sample & prior variances; n & 0 = sample & prior means; w’s constants.

establishing non inferiority
Establishing non-inferiority
  • With posterior model (categorical/non-categorical)-

 can determine (predictive) posterior probabilities and conclude non-inferiority if predictive posterior probability greater than 0.975.

 If Pr[E effect relative to putative P|data]  0.95, then E>>P

 Pr[C effect +ve & E preserves  100 (1-)% C effect]0.975,

then conclude E preserves  (1-)% of C effect.

 Compute one-sided p-value = 1 - posterior probability

other possible applications
Other possible Applications

Given clinically acceptable treatment effect size,

 For fixed N (=1j JNj), test for futility possible– group sequential

 If N not fixed, sample size re-estimation possible – adaptive

Dynamic treatment allocation – play the winner

concluding remarks
Concluding Remarks

•Generally difficult to design and conduct efficient active control trials

• Active control trial design and hypotheses formulation not only disease, endpoint, and analysis method dependent, but too dependent on historical.

• Equivalence or non-inferiority trials always raise a question:

Is at least % of control effect size retained in active trial?

If not, equivalence or non-inferiority conclusion is meaningless as experimental drug (E) could have no effect at all.

• Thus if control effect size is not “substantial enough” and/or “constant”, superiority and NOT non-inferiority objective should be pursued; else stringent requirement of‘100% retention of control effect’ is inescapable!

• Constancy assumption for C effect and trial conditions is not time-invariant and unrealistic for active-control trial success

concluding remarks32
Concluding Remarks

• Simulation results show adequate type I error rate control for concluding E effect with point estimate for inferiority margin (N) selection when  75% retention of C effect.

But CI method ultra-conservative for concluding E effect even with  75% retention of C effect.

• Since primary concern is assay sensitivity, regulators should provide lists of compounds (by class) with acceptable assay sensitivity and thus ok to focus on establishing non-inferiority of E to C and not demonstration of E effect.

• For compounds with poor assay sensitivity, agencies should recommend three-arm dose-ranging active design, else clearly document minimum C effect that must be reproduced in active trial to ensure efficacy of E in active study.

Need to explore alternative approaches to establish E effect!

some references
Some References
  • Tsong Y, Zhang J (2005). Testing superiority and non-inferiority hypotheses in active controlled active trials. Biometrical Journal, 47, 62-74.
  • Hung HMJ, Wang S-J, Tsong Y, Lawrence J, O’Neil RT (2003). Some fundamental issues with non-inferiority testing in active controlled trials. Statistics in Med., 22, 213-225.
  • ICH Efficacy Document No. E-9 (1997). Statistical Principles of Clinical Trials. http://www.fda.gov/cder/guidance.
  • DerSimonian R, Laird N (1986). Meta-analysis in clinical trials. CCT, 177-188.
  • Sankoh, AJ, Huque, MF. Impact of Multiple Endpoints on Type I Error Rate and Power of Test Statistic in Non-superiority Clinical Trials. Far East Journal of Theoretical Statistics, 13 (1), 47-65.
  • Sankoh AJ, Al-Osh M, Huque FM (1999). On the utility of the Dirichlet distribution for meta-analysis of clinical studies. JBS, 9: 289-306.
  • Rohmel J (1998). Therapeutic equivalence investigations: statistical consideration. Stats in Med, 17, 1703-1714
  • Holmgreen EB (1999). Establishing equivalence by showing that a specified percentage of the effect of the active control over placebo is maintained JBS 9(4), 651-659
ad