progress and regression in clinical trials l.
Skip this Video
Loading SlideShow in 5 Seconds..
Progress and Regression in clinical trials PowerPoint Presentation
Download Presentation
Progress and Regression in clinical trials

Loading in 2 Seconds...

play fullscreen
1 / 52

Progress and Regression in clinical trials - PowerPoint PPT Presentation

  • Uploaded on

Progress and Regression in clinical trials 1950-1990: False POSITIVES increasingly well controlled by randomisation 1990-2000: False NEGATIVES increasingly well controlled by “mega-trials” and “meta-analyses”

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Progress and Regression in clinical trials' - Audrey

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
progress and regression in clinical trials
Progress and Regression in clinical trials

1950-1990: False POSITIVES increasingly well controlled by randomisation

1990-2000: False NEGATIVES increasingly well controlled by “mega-trials” and “meta-analyses”

2000 & beyond: Increasing regulation,complexity and costs may prevent many important public health questions from being answered reliably (REGRESSION)


effect sizes diminishing effects
Effect Sizes: Diminishing Effects

In the current era, where multiple effective therapies for a condition already exist, the incremental effects of a new treatment may be harder to detect:

  • Benefits may be more moderate when added to other treatments
  • Benefits may be more moderate when tested against established treatments (e.g. 10% RRR, not 20% RRR)
  • Adverse effects may be more than established treatments


examples of smaller treatment effects in the modern era
Examples of “Smaller” Treatment Effects in the Modern Era

Antiplatelet agents: Chronic CAD:ASA vs Control: 25% RRR in vascular events, Thienopyridine vs ASA : 10% RRR, Oral GP IIb/IIIa inhibitors vs ASA : no diff. AMI: 20% RRR of ASA v plac but Clop v plac on top of ASA:10% RRR.

Thrombolytic agents: SK vs Control: 25% RRR in mortality tPA vs SK: 10% RRR (in mortality/disabling strokes)

Bolus new agents vs infusion: No diff in mortality, but increase in intracranial bleeds by 30%

Thrombin inhibitors: UFH/LMWH vs Control in UA: 45% RRR, Fonda v Enox : 0%(20%) RRR but 50% RR in major bleeds.

potential cumulative impact of 4 simple secondary prevention treatments
Potential Cumulative Impact of 4 Simple Secondary Prevention Treatments



questions that require substantial efficiency in costs for large trials to be conducted
Questions that Require Substantial Efficiency in Costs for Large Trials to be Conducted
  • Non-pharmaceutical : Lifestyle modification (e.g. wt reduction), surgical procedures, eval diagnostic procedures, nutrition supplements or mod (e.g. vitamins,breast v formula), health care delivery (e.g. parameds to provide antenatal care/deliveries, handwashing)
  • Generic drugs: New uses of old drugs (e.g. HOPE)
  • Combination therapies (Polypill)/Extending duration of therapies (Duration of tamoxifen in breast cancer)
  • Developing country questions: e.g. Chagas disease,HIV,TB ,antithrombotics in resource poor settings.
key elements of a good trial answering an important question reliably
Key Elements of a Good Trial:Answering an important question reliably.


Large No. Events

Good adherence and complete followup

Unbiased Evaluation

complexities of a trial
Complexities of a Trial

Voluminous data/patient




Large No. Events

Unbiased Evaluation

Detailed Eligibility

Proliferating Committees


Multiple approvals

Complex Monitoring

Complex “informed” consent procedures

Proliferation of laws and“guidelines”may make trial results LESS reliable(and so harm, not help, patients)

Clinical trial conduct:ICH Guideline for GCPEU Clinical Trials Directive

NHS Research Governance

Data access/confidentiality:1998 Data Protection ActGMC guidance on confidentialityHealth & Social Care Act/PIAG

Ethics & consent:Helsinki Declaration

public s attitudes vs legal regulatory restrictions
Public’s Attitudes vs Legal/Regulatory Restrictions
  • Over 98% of the general public do not have concerns on data misuse or violation of confidentiality in research studies where ethics committees have reviewed the protocol.
  • In CRASH, only 1/10,008 enrolled withdrew the consent initially provided by the relative. In PAC-MAN, only 3.3% refused consent when they regained capacity.
privacy and confidentiality laws on clinical trials
Privacy and Confidentiality Laws on Clinical Trials
  • Allow use of medical records to screen patients for trials: Facilitated by informing all patients that this is the case, but they can “opt out”.

-IRBs/ethics committees should be encouraged to agree to this

  • Use patient identifiers for follow-up (within and beyond the trial) through central mechanisms (coordinating center, national registries, etc).
  • Access records to confirm events.

2 and 3 can be facilitated by obtaining the consent of the participant

good clinical practice
Good Clinical Practice?
  • Chiefly a bureaucratic document that is related to documentation and mechanics of research and is neither good, clinical nor practical for clinical trials.
  • Reaction to perceived/documented(rare) sloppy data collection, suspicions that investigators and sponsors may be dishonest
  • While simple guidelines on ensuring unbiasedness and accuracy of data are reasonable, current guidelines are far too defensive and make many trials of good questions (especially non industry) almost impossible to conduct.
  • Suggestion: Need new set of sensible guidelines by an independent Professional Body (eg.Society for Clinical Trials)
  • NB:Most trials that have changed practice have NOT used GCP.

Reg requirements that can be substantially simplified/eliminated

  • Multiple IRB/REB approvals ( central per country /reciprocity)
  • Approved informed consent forms(simplify)
  • All REB amendment approvals(simplify and only major changes thru a central website)
  • All future REB annual reviews(?eliminate/post progress on a website)
  • Contracts(simplify/standardize)
  • Lab certif and ref normal ranges(only for special tests)
  • Import licenses and HQP inspections(?elimin/simplify)
MRC review: Potential for EU Clinical Trials Directive (2001) to be a major obstacle to important trials
  • Increased bureaucracy due to requirement for single sponsor (possibly the funding source)
  • Burdensome drug authorisation and supply (GMP & labelling) processes
  • Threat to trials of emergency treatments for patients unable to give consent
  • Rigid approach to pharmacovigilance and site monitoring (through over-interpretation both by regulators, pharma beaureacrats and recently IRBs)
  • Substantial cost increases may result in fewer important trials being conducted
impact of eu clinical trials directive 2001 on non commercial cancer trials in uk eur j cancer 2006
Impact of EU Clinical Trials Directive(2001)on non-commercial cancer trials in UK(Eur J Cancer 2006)
  • Doubling in costs of running non-commercial cancer trials and 6-12 month delays to starting
  • Major concerns about correct interpretation due to lack of central guidance, lack of clarity regarding interpretation of guidance notes, and increased documentation
  • Clinical trial units unable or unwilling to start in non-UK centres due to different interpretations in different European countries
new eu directive 2005 28 ec recital 11 simplified procedures for non commercial trials
New EU Directive 2005/28/EC (Recital 11):simplified procedures for non-commercial trials

“Non-commercial clinical trials conducted by researchers without the participation of the pharmaceutical industry may be of great benefit to the patients concerned……. The conditions under which the non-commercial research is conducted by public researchers, and the places where this research takes place, make the application ofcertain of the details of good clinical practice unnecessary or guaranteed by other means.”

eu definition of non commercial trials
EU definition of “non-commercial” trials
  • Sponsor is university, hospital, public scientific organisation, non-profit institution, patient organisation or researcher;
  • Data from trial belongs to this non-commercial sponsor;
  • Design, conduct, recording and reporting under their control;
  • Impractical:

No agreement in place between sponsor and third parties that allows use of trial data for regulatory or marketing purposes; and

  • Trial should not be part of the development programme for a marketing authorisation of a medicinal product.
ich gcp guidance on monitoring
ICH GCP: Guidance on monitoring

“… extent and nature of monitoring should be based on considerations such as the objectives, purpose, design, complexity, blinding, size and endpoints of the trial. In general there is a need for on-site monitoring before, during and after the trial; however … central monitoring …can assure appropriate conduct of the trial in accordance with GCP”

ICH GCP 5.18.3

on site monitoring
On-site monitoring

”(...) the trial management procedures ensuring validity and reliability of the results are vastly more important than absence of clerical errors. Yet, it is clerical inconsistencies referred to as ’errors’ that are chased by the growing GCP-departments.”

Refs: Lörstad, ISCB-27, Geneva, August 28-31, 2006

Grimes et al, Lancet 2005;366:172

examples of cost escalations that damage the feasibility of trials
Examples of Cost Escalations that Damage the Feasibility of Trials
  • CREATE-ECLA(20,000 AMI eval GIK/LMWH): Costs for a CRO in India to obtain regulatory approvals, import and distribute drugs exceeded the entire study budget
  • UNNAMED TRIAL: A trial of 20,000(statin/combo BP lowering) followed for 5 years proposed at a total cost of $80 million. Funding approved by XYZ company.

-Added complexities related to monitoring,and perceived regulatory requirements , escalated costs to $140 million. Funding withdrawn.

-Revised simple trial with 10,000 (higher risk) individuals ongoing at $30 million

quality assurance why
Quality Assurance – why ?

The purpose of quality assurance is not to ensure that individual data items are 100% error-free.

Its purpose is to ensure that the clinical trial results are reliable, i.e.

  • observed treatment effects are real
  • their estimated magnitude is unbiased
a taxonomy of errors
A taxonomy of errors

Random errors (Random with respect to treatment assignment and unlikely to materially influence study results , unless large).

  • Measurement errors (eg due to assay precision or frequency of visits)
  • Errors due to transcription errors,variations in entry criteria.
  • Some types of fraud (“fabrication” of non key data items)

Systematic errors(Differential with respect to treatment assignment and could substantiallybias results).

  • Design flaws (eg post rand exclusion,biases in event ascertainment,failure to use intent to treat)
  • Some types of fraud (usually with knowledge of treatment assignment as in a single center study)
a randomized study of the impact of on site monitoring
A randomized study of the impact of on-site monitoring

Stratify by

- Type (Academic vs Private)

- Location (Paris vs Province)




in trial



Group A (site visits)

Group B (no visits)

Ref: Liénard et al, Clinical Trials 2006;3:1-7

prevalence of fraud
Prevalence of fraud?
  • Industry (Hoechst, 1990-1994)1 case of fraud in 243 (0.43%) randomly selected centers
  • FDA (1985-1988)1%of 570 routine audits led to a for-cause investigation
  • CALGB (1982-1992)2 cases of fraud in 691 (0.29%) on-site audits
  • SWOG (1983-1990)no case (0%) of fraud in 1,751 patients
  • McMaster:(1992-2007) 2 (0.1%) in 46 trials involving >250,000 patients from over 5000 centers.

fraud is probably rare (but possible underestimation esp in era of paying more than the costs of trials?)

Ref: Buyse et al, Statist in Med 1999;18:3435

impact of fraud
Impact of fraud

Most frauds have little impact on the trial results(unless widespread or at central) because:

  • they introduce random but not systematic errors (i.e. noise but no bias) in the analyses
  • may affect secondary analysis (e.g. subgroup analyses if baseline data are incorrect)
  • their magnitude is too small to have an influence (one site and/or few patients)

Refs: Altman, Practical Statistics for Medical Research 1991

Peto et al, Controlled Clin Trials 1997;18:1

example of fraud detection thru central checks
Example of “fraud” detection thru central checks
  • Invitation to collaborate in being a coauthor(by Dr XY thru well respected invest and friend) on a paper demonstrating the marked benefits of a Vit in individuals with specific genotype in preventing CVD.(Result would have major implications, attract great attention, and targetted for a leading journal).
  • Requested the data base: randomization did not match various “explanations of process and sequence”,diff in events marked and early(both implausible), event rate patterns and rates not clinically consistent ,indication of an independent DSMB who stopped the trial but “named” chair unaware of even being on the DSMB(who was named as the last author and thru whom my involvement as a coauthor was requested).
statistical approaches to data checking
Statistical approaches to data checking
  • Humans are poor random number generators  test randomness (e.g. Benford’s law)
  • Plausible multivariate data are hard to fabricate  test correlation structure
  • Clinical trial data are highly structured compare expected vs observed
  • Clinical trial data are rich in meaning test plausibility (e.g. dates)
  • Fraud or gross errors usually occur at one center compare centers
approaches to study monitoring 1
Approaches to Study Monitoring (1)
  • Trial oversight: Operations or Trial Management Committees, Steering Committee, Independent Data Monitoring Committee
  • Central Monitoring:

-fax consent forms centrally

-central faxing of key documents (e.g. ECGs, laboratory reports, discharge summaries)

Statistical Monitoring:

approaches to study monitoring 2
Approaches to Study Monitoring (2)

3. On Site Monitoring:

a) Random and infrequent

b) Guided by Central Monitoring

c) Onsite “mentoring” instead of “monitoring”

A combined approach of central with directed onsite monitoring(random and for cause) is likely to be both efficient and effective.

safety monitoring reporting and reviewing aes
Safety Monitoring, Reporting and Reviewing AEs


Any event including those that are the outcomes of interest,and events that are common in the condition of that age are considered to be AE

-They(sometimes even the primary outcome) are often recorded, reported (expedited)

-Reviewed individually,so difficult to discern patterns

-Unblinded, and SAEs in the active group only are reported to investigators, their IRBs and to regulatory authorities

safety monitoring reporting and review
Safety Monitoring, Reporting and Review


-Enormous amount of effort (upto about 25 hrs/SAE reported) BUT is it useful and worthwhile?

-Misleading as to the “safety” situation of the trial,as no between group comparisons are possible and impossible to reliably attribute causality on a case by case basis (except perhaps for very unusual events eg thrombocytopenia or liver failure).

-Lack of a balance between safety and efficacy, OASIS 5, catheter thrombosis (excess of 0.2%), bleeding (reduced 2.5%), mortality (reduced 1.0%) with fonda in OASIS-5

alternative approach to adverse event reporting and review
Alternative Approach to Adverse Event Reporting and Review
  • Report all relevant data to a coordinating center, which regularly shares SAE and efficacy to the DSMB.
  • Report to regulatory bodies and centers, only if the DSMB judges that “harm” exceeds “benefit”


Central Events Committe versus Site Investigator?

  • 11,751 events reviewed in VALIANT
  • Events identified by Site Investigator

# Events % Agee

Cause of Death 2897 66%

CHF 3841 73%

MI 2159 63%

Resucitated Death 636 27%

Stroke 541 91%

Who is correct? Investigator with more clinical info

or the comm thousands of miles away?


Effect of Adjudication on Estimated Treatment Effect: McMaster Experience(108,000 pts)

Adjud. Investigator OR

OASIS-1 0.70 0.71 0.99

OASIS-2 0.90 0.85 1.06

HOPE 0.78 0.80 0.98

HOPE-2 0.95 0.93 1.02

CURE 0.82 0.80 1.03

OASIS-5 1.01 1.01 1.00

OASIS-6 0.86 0.86 1.00

CREATE 0.87 0.87 1.00

WAVE 0.82 0.94 0.87

ACTIVE-W 1.43 1.42 1.01


Primary Endpoint Results According to Adjudication

Treated Placebo P-value


Adjudicated 8.3% 12.8% 0.009

Investigator 9.0% 12.4% 0.120


Adjudicated 9.2% 11.4% 0.063

Investigator 5.5% 7.8% 0.018


Adjudicated 8.9% 9.8% 0.058

Investigator 8.4% 9.6% 0.016


Adjudicated 14.2% 15.7% 0.04

Investigator 8.0% 10.0% 0.0007


Adjudicated 22.0% 24.3% 0.12

Investigator 21.4% 24.7% 0.028

central adjudication of events when and to what degree
Central Adjudication of Events: When and to What Degree?
  • Not needed: “Hard” endpoints, especially blinded studies
  • Needed for open trials, especially when the outcome may be subject to interpretation
  • Occasionally: Central screening for “missed events”

Fundamental need is to avoid BIASES and LARGE MISCLASSIFICATIONS

Accuracy may be enhanced by collecting information on CRFs on outcomes in a structure than matches protocol definitions

factorial designs
Factorial Designs
  • Increased efficiency as “two” for the price of “one”
  • Only way to answer generic questions, by piggybacking it to a more “fundable” question
  • Avoid overfactorializing (e.g. 2x2x2x2)
  • Our approach: ALWAYS try to factor a generic question(Vit, fish oils,GIK,intervention type or procedures, lifestyles, vaccines,etc), unless impossible.
isis 2 study
ISIS-2 Study


Lancet 1988

increasing clinical trials in disadvantaged vulnerable populations 1
Increasing Clinical Trials in Disadvantaged/Vulnerable Populations (1)
  • 90% of the global burden of disease occurs in LIC + MIC; yet only 10% of the $70 billion of the research expenditures occur in these countries, e.g. Chagas disease affects 18 mill people in L. America; yet only 4 RCTs involving 800 individuals followed for 3 to 6 months are available .BENEFIT: 3000 patients x 3 yrs, evaluating benznidazole; T.B. pericarditis (50% 6 mo mortality), no major trials.IMPI: steroids and a vaccine in 800( 3000) people.
  • Neglected populations e.g. children, vulnerable groups, etc.
  • Neglected conditions, e.g. road traffic accidents (CRASH: steroids in head injuries), cardiac arrest, other critical illness
intensive care management of severe head injury
Intensive care management of severe head injury

Percent therapy for intracranial hypertension

1995 USA 1996 UK No.pts(trials) OR(CI)

Barbiturates 33% 56% 208(3) 1.09(.91,1.47)

Corticosteroids 64% 49% 2119(16) 0.96(.85,1.08)

CSF drainage 44% - 0(0) --

Hyperventilation 83% 100% 77(1) 0.73(.36,1.49)

Mannitol 83% 100% 44(1) 1.75(.48,6.35)




893 / 4 979


1 052 / 4 985


1.18 (1.09–1.27)

p < 0.001






Corticosteroid better

Corticosteroid worse

CRASH Trial :Death within 14 days

increasing clinical trials in disadvantaged vulnerable populations 2
Increasing Clinical Trials in Disadvantaged/Vulnerable Populations (2)

Consider trials with consent of relatives or surrogates and when time is of the essence; perhaps without any consent but IRB approvals


globalization of trials in cvd50
Globalization of Trials in CVD

1. Most CVD trials include LIC/MIC to reduce costs and speed enrollment, and are primarily aimed at answering questions relevant to the West.

BUT, these trials may also have applicability in LIC/MIC if the condition is common and the treatments are affordable.

2. Need trials in LIC & MIC of locally relevant conditions and treatments, and locally conducted.

barriers to funding global trials
Barriers to Funding Global Trials
  • No (e.g. in LIC/MIC) or modest investment in clinical trials by governmental/charitable bodies even in Western countries (usually 2% to 8%, in Canada about 4%)
  • Balkanization in funding (e.g. HSFS in Canada or need special justification to include foreign centers) and restrictions in their use by territory (e.g. by province or country)
  • Misperceptions about the importance of large clinical trials (cookbook and mundane) vs “basic science” (more fundamental and exciting
  • Perception that clinical trials funding should be left to industry
improving global health thru reliable trials of important questions
Improving global health thru reliable trials of important questions
  • Randomize a large number of (high risk) individuals
  • Minimize data collection per subject drastically
  • Minimize complexities (e.g. adjudicatIion, monitoring, standardization, AE reporting, approvals, etc.)
  • MANY MORE trials with factorial designs.
  • Need more trials in vulnerable populations and developing countries, where the disease burdens are the largest