Trial Objectives. Superiority, Noninferiority, and Equivalence. Questions of Interest. Is the new treatment better than the control treatment that I am using now? (superiority trial)
Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.
Superiority, Noninferiority, and Equivalence
Noninferiority and equivalence trials are usually
considered when there is an active control.
GAO1798 Evidence from Clinical Trials
JAMA 2010; 303:951958
“The study was powered to assess treatment equivalence for the primary endpoint (i.e., a plasma HIV RNA level <= 400 copies/mL at week 48 for the intent totreat population). For the primary end point, treatments were considered equivalent if the 95% confidence interval was within the bound 12% to 12%.”
New Treatment
But is it as effective?
Superiority and NonInferiority in One Trial and Equivalence Trials(Usually concurrent placebo arm is absent, but this may be practical in some shortterm studies)
Randomize
Drug A
Control
Drug B
Experimental
Placebo
Superiority
Noninferiority
Randomize
Sertraline
Active Control
St. John’s Wort
Experimental
Placebo
Control
Neither sertraline or St. John’s Wort was significantly different
from placebo in this 8 week study. The authors noted “without a placebo,
hypericum could easily have been considered as effective as sertraline…”
JAMA 2002; 287:18071814.
In the absence of a concurrent placebo, have to provide assurance that the active control would have been superior to placebo, if it had been used, and the test treatment would have beat placebo had it been used (indirect inference).
Assay Sensitivity (def.) – ability to demonstrate a difference between active and inactive treatments
Constancy (def.)
Hung and O’Neill, Encyclopedia of Clinical Trials
Would like to convince people that if you had used placebo you would have won!
How do you prove two treatments are equal? and Defining the NonInferiority or Equivalence Margin
Cannot prove HO: Δ=0
“It is never correct to claim that treatments have no effect or that there is no difference in the effects of treatments. It is impossible to prove … that two treatments have the same effect. There will always be uncertainty surrounding estimates of treatment effects, and a small difference can never be excluded… An analysis of 45 reports of trials purporting to test equivalence found that only a quarter set boundaries on their equivalence.”
Alderson P, Chalmers I. BMJ 2003:326:16918.
The noninferiority/equivalence margin must
be specified in the protocol!
Superiority strongly
shown
p=0.002
p=0.05
Superiority shown
p=0.20
Superiority not shown
ControlBetter
0
New Agent Better
Treatment Difference
CHD (95% CI:0.911.08)
CVD Composite (95% CI:
1.051.16)
ESRD (95% CI: 0.881.38)
Lisinopril better
Chlorthalidone better
1.00
HR (Lisinopril/Chlorthalidone)
In ALLHAT, 15,255 participants were randomized to chlorthalidone and 9,000+
participants were randomized to each of 3 other treatments.
JAMA 2002;288:29812997.
CONVINCE equivalence bounds (0.861.16)
CONVINCE Trial result
CAPPP Trial result
Overview (9 trials)
1.00
Calcium Channel Blocker better
SOC better
HR (Verapamil/SOC) for CONVINCE
(Captopril/SOC) for CAPPP
CAPPP = Captopril Primary Prevention Project. Authors
concluded: “captopril and conventional treatment did not differ in efficacy.”
See JAMA 2003;289: 20732082 for Convince Trial
Lancet 2004, 363:125363
‘Antiviral therapy with nevirapine or efavirenz showed similar efficacy, so tripledrug regimens with either … are valid for firstline treatment’
Lancet 2004, 363:125363
Estimated benefit of
standard drug over placebo
Zone of
noninferiority
Test drugbetter
Standarddrug better
0.6
0.7
0.8
0.9
1.0
1.1
1.2
1.3
1.4
A
Superiority
B
C
Noninferiority(i.e., Equivalence)
Inferiority
D
E
Underpowered trial
F
Anteman EM, Circulation 2001;103:e101e104.
Estimated benefit of
standard drug over placebo
Zone of
noninferiority
Test drugbetter
Standarddrug better
0.6
0.7
0.8
0.9
1.0
1.1
1.2
1.3
1.4
A
Superiority
B
C
Noninferiority(i.e., Equivalence)
Inferiority
D
E
Underpowered trial
F
A = Test drug is superior to standard
Estimated benefit of
standard drug over placebo
Zone of
noninferiority
Test drugbetter
Standarddrug better
0.6
0.7
0.8
0.9
1.0
1.1
1.2
1.3
1.4
A
Superiority
B
C
Noninferiority(i.e., Equivalence)
Inferiority
D
E
Underpowered trial
F
B = Test drug is better than standard and can be considered“noninferior” to standard
Estimated benefit of
standard drug over placebo
Zone of
noninferiority
Test drugbetter
Standarddrug better
0.6
0.7
0.8
0.9
1.0
1.1
1.2
1.3
1.4
A
Superiority
B
C
Noninferiority(i.e., Equivalence)
Inferiority
D
E
Underpowered trial
F
C = Test drug is worse than standard but not that much worse,and can be considered “noninferior” to standard
Estimated benefit of
standard drug over placebo
Zone of
noninferiority
Test drugbetter
Standarddrug better
0.6
0.7
0.8
0.9
1.0
1.1
1.2
1.3
1.4
A
Superiority
B
C
Noninferiority(i.e., Equivalence)
Inferiority
D
E
Underpowered trial
F
D = Test drug is inferior to standard and noninferiority
criteria not satisfied.
Estimated benefit of
standard drug over placebo
Zone of
noninferiority
Test drugbetter
Standarddrug better
0.6
0.7
0.8
0.9
1.0
1.1
1.2
1.3
1.4
A
Superiority
B
C
Noninferiority(i.e., Equivalence)
Inferiority
D
E
Underpowered trial
F
E = Test drug is very inferior to standard (noninferiority
criteria not satisfied)
Estimated benefit of
standard drug over placebo
Zone of
noninferiority
Test drugbetter
Standarddrug better
0.6
0.7
0.8
0.9
1.0
1.1
1.2
1.3
1.4
A
Superiority
B
C
Noninferiority(i.e., Equivalence)
Inferiority
D
E
Underpowered trial
F
F = Trial is inconclusive due to small size and resultant wide CI
Absence of proof of a treatment difference does
not constitute proof of an absence of a treatment difference.
A = new treatment; B = standard;PA and PB = event rates (failure rate)
If Ho is rejected, treatments are “equivalent”
Roles of null and alternative hypotheses are reversed. In practice, this is confusing to people.
Blackwelder W, Cont Clin Trials 1982
2
Note: If Δ=0, then this
is equivalent to
superiority trial to detect
δo with 90% power.
4 0 132
4 +2 525
4 2 58
δO = 4 mmHg
Δ = 0, 2 (A better) and +2 (B better)
σ2 = 100; α = 0.025 (1sided); 1β = 0.90
1:1 allocation
No. per
group
δO
Δ
A (newtreatmentbetter)
B (standardtreatmentbetter)
A (newtreatmentbetter)
B (standardtreatmentbetter)
Prob (upper limit of CI exceeds effect or that there is no difference in the effects of treatments. It is impossible to prove … that two treatments have the same effect. There will always be uncertainty surrounding estimates of treatment effects, and a small difference can never be excluded… An analysis of 45 reports of trials purporting to test equivalence found that only a quarter set boundaries on their equivalence.”
d
when
d
<
d
)
=
0
0
é
ù
^
^
Prob
(
P

P
)
+
Z
s
>
d
=
b
ê
ú
a
A
B
ë
û
0
1

2
é
ù
^
^
(
P

P
)

(
P

P
)
d

(
P

P
)
ê
ú
A
B
A
B
A
B
Prob
>

Z
=
b
0
ê
ú
a
s
s
1

ê
ú
2
ë
û
P
(1

P
)
P
(1

P
)
s 2
=
+
A
A
B
B
N
N
A
B
Sample Size for EquivalenceDesign Based on CI LimitsA = New Treatment; B = Standard(
)
Makuch and Simon (Cancer Treatment Reports, 1978) suggest a = 0.10 (1sided) and b = 0.20; I like a = .05 (and usually 2sided)
Stat Med 1990; 9:14471454
0.05 0.05 0.01 9,972 10,032
0.10 0.10 0.05 756 775
0.15 0.15 0.05 1,071 1,080
0.20 0.20 0.05 1,344 1,348
0.20 0.20 0.10 336 340
Sample Size per Group
Makuchand Simon
Farrington
and Manning
PA(PE)
PB(PC)
δO
* α = 0.025 (1sided)
1β = 0.90
1:1 allocation
0.10 0.10 0.05 756 775
0.125 0.10 0.05 3,343 3,379
0.10 0.125 0.05 371 384
Sample Size per Group
Makuchand Simon
Farrington
and Manning
PA(PE)
PB(PC)
δO
* α = 0.025 (1sided)
1β = 0.90
1:1 allocation
Sample Size for Proportions: Superiority Trial with Makuch and Simon versus Farrington and Manning (PSpecified Delta or Inferiority with Farrington and Manning(1:1 allocation and 1β = 0.90)
0.05 0.05 0.01 9,021 10,032 8,174
0.10 0.10 0.05 581 775 630
0.15 0.15 0.05 917 1,080 880
0.20 0.20 0.05 1,211 1,349 1,099
0.20 0.20 0.10 266 340 277
Sample Size per Group
Superiority*
Farrington
and Manning**
PA(PE)
PB(PC)
δO
* α = 0.05 (2sided)
PE=PC  δO
** α = 0.025 (1sided)
in 1st column;
α = 0.05 (1sided)
in 2nd column
General Approach Makuch and Simon versus Farrington and Manning (P
RRo
RR
NewTreatmentBetter
StandardTreatmentBetter
RRo chosen so that if upper limit
< RRo, we conclude
“equivalence”
RRo usually ≠ 1.0
50% Makuch and Simon versus Farrington and Manning (P
67%
No PI
Treatment
Example:CPCRA Study of Nelfinavir (NFV) and Ritonavir (RTV) for Patients with CD4+ <100RR (RTV/Placebo) = 0.5 Directly est.
RR (NFV/RTV) = 1.33 Direct est.
RR (NFV/Placebo) = 0.67 Indirectly est.
0.75
Lower
limit of
equiv.
1.0
No diff.
1.33
Upper
limit of
equiv.
RTV
Better
NFV
Better
Equivalence
Inconclusive
The 95% CI for the difference between the control and the intervention are all >δ,i.e. noninferiority demonstrated.
In this case both noninferiority and superiority have been demonstrated
δ
0
No difference
The 95% CI for the difference between the control and the intervention are all >δ,i.e. noninferiority demonstrated.
In this case both noninferiority and superiority have been demonstrated
In this case both noninferiority and inferiority have been demonstrated
δ
0
No difference
N Engl J Med 2011; 365:21552166.
Gusto IStudy(N = 41,021)
Background
Lower 95% CI limit for 30 day mortality difference for accelerated infusion of alteplase vs. streptokinase = 0.4% (30 day mortality: 6.3 vs 7.3%) (need N = 50,000 to rule out difference this big)
Double bolus alteplase vs. accelerated infusion of alteplase (N = 7,169)
7.98% vs. 7.53%
1sided 95% CI (∞ to 1.49)
Not equivalent
Cobalt Study
30 day
mortality
Conclusion
Gusto III Trial
30 day
mortality
Conclusion
Double bolus reteplase vs. accelerated infusion of alteplase (N = 15,059)
7.47% vs. 7.24%
95% CI (0.66 to 1.10)
Similar efficacy
control  new intervention is both > δ and < δ, the new treatment must be neither worse nor better than the control by a fixed amount.
+ Builds on CONSORT guidelines for superiority trials.