Progress and Regression in clinical trials 1950-1990: False POSITIVES increasingly well controlled by randomisation 1990-2000: False NEGATIVES increasingly well controlled by “mega-trials” and “meta-analyses” 2000 & beyond: Increasing regulation,complexity and costs may prevent many important public health questions from being answered reliably (REGRESSION) URGENT NEED TO SIMPLIFY TRIALS TO ENHANCE THE CONDUCT OF IMPORTANT TRIALS ESP IN VULNERABLE AND UNDERSERVED POPULATIONS BOTH IN DEVELOPED AND DEVELOPING COUNTRIES
Effect Sizes: Diminishing Effects In the current era, where multiple effective therapies for a condition already exist, the incremental effects of a new treatment may be harder to detect: • Benefits may be more moderate when added to other treatments • Benefits may be more moderate when tested against established treatments (e.g. 10% RRR, not 20% RRR) • Adverse effects may be more than established treatments THEREFORE, THE FUTURE GENERATION OF TRIALS COMPARING TWO ACTIVE AGENTS MAY HAVE TO BE SEVERAL TIMES LARGER THAN INITIALTRIALS OF ACTIVE VSCONTROL OR FOR NON INF TRIALS.
Examples of “Smaller” Treatment Effects in the Modern Era Antiplatelet agents: Chronic CAD:ASA vs Control: 25% RRR in vascular events, Thienopyridine vs ASA : 10% RRR, Oral GP IIb/IIIa inhibitors vs ASA : no diff. AMI: 20% RRR of ASA v plac but Clop v plac on top of ASA:10% RRR. Thrombolytic agents: SK vs Control: 25% RRR in mortality tPA vs SK: 10% RRR (in mortality/disabling strokes) Bolus new agents vs infusion: No diff in mortality, but increase in intracranial bleeds by 30% Thrombin inhibitors: UFH/LMWH vs Control in UA: 45% RRR, Fonda v Enox : 0%(20%) RRR but 50% RR in major bleeds.
Potential Cumulative Impact of 4 Simple Secondary Prevention Treatments CUMULATIVE BENEFITS ARE LIKELY TO BE IN EXCESS OF 75% RRR, WHICH IS SUBSTANTIAL
Questions that Require Substantial Efficiency in Costs for Large Trials to be Conducted • Non-pharmaceutical : Lifestyle modification (e.g. wt reduction), surgical procedures, eval diagnostic procedures, nutrition supplements or mod (e.g. vitamins,breast v formula), health care delivery (e.g. parameds to provide antenatal care/deliveries, handwashing) • Generic drugs: New uses of old drugs (e.g. HOPE) • Combination therapies (Polypill)/Extending duration of therapies (Duration of tamoxifen in breast cancer) • Developing country questions: e.g. Chagas disease,HIV,TB ,antithrombotics in resource poor settings.
Key Elements of a Good Trial:Answering an important question reliably. Randomization Large No. Events Good adherence and complete followup Unbiased Evaluation
Complexities of a Trial Voluminous data/patient Adjudication Regulators Randomization Large No. Events Unbiased Evaluation Detailed Eligibility Proliferating Committees Audits Multiple approvals Complex Monitoring Complex “informed” consent procedures
Proliferation of laws and“guidelines”may make trial results LESS reliable(and so harm, not help, patients) Clinical trial conduct:ICH Guideline for GCPEU Clinical Trials Directive NHS Research Governance Data access/confidentiality:1998 Data Protection ActGMC guidance on confidentialityHealth & Social Care Act/PIAG Ethics & consent:Helsinki Declaration
Public’s Attitudes vs Legal/Regulatory Restrictions • Over 98% of the general public do not have concerns on data misuse or violation of confidentiality in research studies where ethics committees have reviewed the protocol. • In CRASH, only 1/10,008 enrolled withdrew the consent initially provided by the relative. In PAC-MAN, only 3.3% refused consent when they regained capacity.
Privacy and Confidentiality Laws on Clinical Trials • Allow use of medical records to screen patients for trials: Facilitated by informing all patients that this is the case, but they can “opt out”. -IRBs/ethics committees should be encouraged to agree to this • Use patient identifiers for follow-up (within and beyond the trial) through central mechanisms (coordinating center, national registries, etc). • Access records to confirm events. 2 and 3 can be facilitated by obtaining the consent of the participant
Good Clinical Practice? • Chiefly a bureaucratic document that is related to documentation and mechanics of research and is neither good, clinical nor practical for clinical trials. • Reaction to perceived/documented(rare) sloppy data collection, suspicions that investigators and sponsors may be dishonest • While simple guidelines on ensuring unbiasedness and accuracy of data are reasonable, current guidelines are far too defensive and make many trials of good questions (especially non industry) almost impossible to conduct. • Suggestion: Need new set of sensible guidelines by an independent Professional Body (eg.Society for Clinical Trials) • NB:Most trials that have changed practice have NOT used GCP.
Reg requirements that can be substantially simplified/eliminated • Multiple IRB/REB approvals ( central per country /reciprocity) • Approved informed consent forms(simplify) • All REB amendment approvals(simplify and only major changes thru a central website) • All future REB annual reviews(?eliminate/post progress on a website) • Contracts(simplify/standardize) • Lab certif and ref normal ranges(only for special tests) • Import licenses and HQP inspections(?elimin/simplify)
MRC review: Potential for EU Clinical Trials Directive (2001) to be a major obstacle to important trials • Increased bureaucracy due to requirement for single sponsor (possibly the funding source) • Burdensome drug authorisation and supply (GMP & labelling) processes • Threat to trials of emergency treatments for patients unable to give consent • Rigid approach to pharmacovigilance and site monitoring (through over-interpretation both by regulators, pharma beaureacrats and recently IRBs) • Substantial cost increases may result in fewer important trials being conducted
Impact of EU Clinical Trials Directive(2001)on non-commercial cancer trials in UK(Eur J Cancer 2006) • Doubling in costs of running non-commercial cancer trials and 6-12 month delays to starting • Major concerns about correct interpretation due to lack of central guidance, lack of clarity regarding interpretation of guidance notes, and increased documentation • Clinical trial units unable or unwilling to start in non-UK centres due to different interpretations in different European countries
New EU Directive 2005/28/EC (Recital 11):simplified procedures for non-commercial trials “Non-commercial clinical trials conducted by researchers without the participation of the pharmaceutical industry may be of great benefit to the patients concerned……. The conditions under which the non-commercial research is conducted by public researchers, and the places where this research takes place, make the application ofcertain of the details of good clinical practice unnecessary or guaranteed by other means.”
EU definition of “non-commercial” trials • Sponsor is university, hospital, public scientific organisation, non-profit institution, patient organisation or researcher; • Data from trial belongs to this non-commercial sponsor; • Design, conduct, recording and reporting under their control; • Impractical: No agreement in place between sponsor and third parties that allows use of trial data for regulatory or marketing purposes; and • Trial should not be part of the development programme for a marketing authorisation of a medicinal product.
ICH GCP: Guidance on monitoring “… extent and nature of monitoring should be based on considerations such as the objectives, purpose, design, complexity, blinding, size and endpoints of the trial. In general there is a need for on-site monitoring before, during and after the trial; however … central monitoring …can assure appropriate conduct of the trial in accordance with GCP” ICH GCP 5.18.3
On-site monitoring ”(...) the trial management procedures ensuring validity and reliability of the results are vastly more important than absence of clerical errors. Yet, it is clerical inconsistencies referred to as ’errors’ that are chased by the growing GCP-departments.” Refs: Lörstad, ISCB-27, Geneva, August 28-31, 2006 Grimes et al, Lancet 2005;366:172
Examples of Cost Escalations that Damage the Feasibility of Trials • CREATE-ECLA(20,000 AMI eval GIK/LMWH): Costs for a CRO in India to obtain regulatory approvals, import and distribute drugs exceeded the entire study budget • UNNAMED TRIAL: A trial of 20,000(statin/combo BP lowering) followed for 5 years proposed at a total cost of $80 million. Funding approved by XYZ company. -Added complexities related to monitoring,and perceived regulatory requirements , escalated costs to $140 million. Funding withdrawn. -Revised simple trial with 10,000 (higher risk) individuals ongoing at $30 million
Quality Assurance – why ? The purpose of quality assurance is not to ensure that individual data items are 100% error-free. Its purpose is to ensure that the clinical trial results are reliable, i.e. • observed treatment effects are real • their estimated magnitude is unbiased
A taxonomy of errors Random errors (Random with respect to treatment assignment and unlikely to materially influence study results , unless large). • Measurement errors (eg due to assay precision or frequency of visits) • Errors due to transcription errors,variations in entry criteria. • Some types of fraud (“fabrication” of non key data items) Systematic errors(Differential with respect to treatment assignment and could substantiallybias results). • Design flaws (eg post rand exclusion,biases in event ascertainment,failure to use intent to treat) • Some types of fraud (usually with knowledge of treatment assignment as in a single center study)
VeGF trial for macular degeneration:Median simulated P-values (and IQ range)
A randomized study of the impact of on-site monitoring Stratify by - Type (Academic vs Private) - Location (Paris vs Province) Centers accruing patients in trial AERO B2000 Group A (site visits) Group B (no visits) Ref: Liénard et al, Clinical Trials 2006;3:1-7
RCT of onsite monitoing:Impact of initiation visits on patient accrual No difference
Impact of initiation visits on volume of data submitted No difference
Impact of initiation visits on quality of data submitted No difference
Prevalence of fraud? • Industry (Hoechst, 1990-1994)1 case of fraud in 243 (0.43%) randomly selected centers • FDA (1985-1988)1%of 570 routine audits led to a for-cause investigation • CALGB (1982-1992)2 cases of fraud in 691 (0.29%) on-site audits • SWOG (1983-1990)no case (0%) of fraud in 1,751 patients • McMaster:(1992-2007) 2 (0.1%) in 46 trials involving >250,000 patients from over 5000 centers. fraud is probably rare (but possible underestimation esp in era of paying more than the costs of trials?) Ref: Buyse et al, Statist in Med 1999;18:3435
Impact of fraud Most frauds have little impact on the trial results(unless widespread or at central) because: • they introduce random but not systematic errors (i.e. noise but no bias) in the analyses • may affect secondary analysis (e.g. subgroup analyses if baseline data are incorrect) • their magnitude is too small to have an influence (one site and/or few patients) Refs: Altman, Practical Statistics for Medical Research 1991 Peto et al, Controlled Clin Trials 1997;18:1
Example of “fraud” detection thru central checks • Invitation to collaborate in being a coauthor(by Dr XY thru well respected invest and friend) on a paper demonstrating the marked benefits of a Vit in individuals with specific genotype in preventing CVD.(Result would have major implications, attract great attention, and targetted for a leading journal). • Requested the data base: randomization did not match various “explanations of process and sequence”,diff in events marked and early(both implausible), event rate patterns and rates not clinically consistent ,indication of an independent DSMB who stopped the trial but “named” chair unaware of even being on the DSMB(who was named as the last author and thru whom my involvement as a coauthor was requested). • WE PROVIDED A REPORT OF OUR FINDINGS AND DECLINED PARTICIPATION
COMMIT: Example of central checks indicating problem at one of 1250 participating hospitals
Statistical approaches to data checking • Humans are poor random number generators test randomness (e.g. Benford’s law) • Plausible multivariate data are hard to fabricate test correlation structure • Clinical trial data are highly structured compare expected vs observed • Clinical trial data are rich in meaning test plausibility (e.g. dates) • Fraud or gross errors usually occur at one center compare centers
Approaches to Study Monitoring (1) • Trial oversight: Operations or Trial Management Committees, Steering Committee, Independent Data Monitoring Committee • Central Monitoring: -fax consent forms centrally -central faxing of key documents (e.g. ECGs, laboratory reports, discharge summaries) Statistical Monitoring:
Approaches to Study Monitoring (2) 3. On Site Monitoring: a) Random and infrequent b) Guided by Central Monitoring c) Onsite “mentoring” instead of “monitoring” A combined approach of central with directed onsite monitoring(random and for cause) is likely to be both efficient and effective.
Safety Monitoring, Reporting and Reviewing AEs Current: Any event including those that are the outcomes of interest,and events that are common in the condition of that age are considered to be AE -They(sometimes even the primary outcome) are often recorded, reported (expedited) -Reviewed individually,so difficult to discern patterns -Unblinded, and SAEs in the active group only are reported to investigators, their IRBs and to regulatory authorities
Safety Monitoring, Reporting and Review Problems: -Enormous amount of effort (upto about 25 hrs/SAE reported) BUT is it useful and worthwhile? -Misleading as to the “safety” situation of the trial,as no between group comparisons are possible and impossible to reliably attribute causality on a case by case basis (except perhaps for very unusual events eg thrombocytopenia or liver failure). -Lack of a balance between safety and efficacy, e.g.in OASIS 5, catheter thrombosis (excess of 0.2%), bleeding (reduced 2.5%), mortality (reduced 1.0%) with fonda in OASIS-5
Alternative Approach to Adverse Event Reporting and Review • Report all relevant data to a coordinating center, which regularly shares SAE and efficacy to the DSMB. • Report to regulatory bodies and centers, only if the DSMB judges that “harm” exceeds “benefit”
VALIANT Central Events Committe versus Site Investigator? • 11,751 events reviewed in VALIANT • Events identified by Site Investigator # Events % Agee Cause of Death 2897 66% CHF 3841 73% MI 2159 63% Resucitated Death 636 27% Stroke 541 91% Who is correct? Investigator with more clinical info or the comm thousands of miles away?
Effect of Adjudication on Estimated Treatment Effect: McMaster Experience(108,000 pts) Adjud. Investigator OR OASIS-1 0.70 0.71 0.99 OASIS-2 0.90 0.85 1.06 HOPE 0.78 0.80 0.98 HOPE-2 0.95 0.93 1.02 CURE 0.82 0.80 1.03 OASIS-5 1.01 1.01 1.00 OASIS-6 0.86 0.86 1.00 CREATE 0.87 0.87 1.00 WAVE 0.82 0.94 0.87 ACTIVE-W 1.43 1.42 1.01
Primary Endpoint Results According to Adjudication Treated Placebo P-value EPIC Adjudicated 8.3% 12.8% 0.009 Investigator 9.0% 12.4% 0.120 IMPACT II Adjudicated 9.2% 11.4% 0.063 Investigator 5.5% 7.8% 0.018 GUSTO IIB Adjudicated 8.9% 9.8% 0.058 Investigator 8.4% 9.6% 0.016 PURSUIT Adjudicated 14.2% 15.7% 0.04 Investigator 8.0% 10.0% 0.0007 CHARM-Preserved Adjudicated 22.0% 24.3% 0.12 Investigator 21.4% 24.7% 0.028
Central Adjudication of Events: When and to What Degree? • Not needed: “Hard” endpoints, especially blinded studies • Needed for open trials, especially when the outcome may be subject to interpretation • Occasionally: Central screening for “missed events” Fundamental need is to avoid BIASES and LARGE MISCLASSIFICATIONS Accuracy may be enhanced by collecting information on CRFs on outcomes in a structure than matches protocol definitions
Factorial Designs • Increased efficiency as “two” for the price of “one” • Only way to answer generic questions, by piggybacking it to a more “fundable” question • Avoid overfactorializing (e.g. 2x2x2x2) • Our approach: ALWAYS try to factor a generic question(Vit, fish oils,GIK,intervention type or procedures, lifestyles, vaccines,etc), unless impossible. • FACTORIAL DESIGNS ARE UNDERUTILIZED DUE TO BOTH ACADEMIC, INDUSTRY AND REGULATORY FEARS THAT QUALITATIVE INTERACTIONS WILL OCCUR THAT ARE LARGELY MISPLACED.
ISIS-2 Study ISIS 2 Lancet 1988
Increasing Clinical Trials in Disadvantaged/Vulnerable Populations (1) • 90% of the global burden of disease occurs in LIC + MIC; yet only 10% of the $70 billion of the research expenditures occur in these countries, e.g. Chagas disease affects 18 mill people in L. America; yet only 4 RCTs involving 800 individuals followed for 3 to 6 months are available .BENEFIT: 3000 patients x 3 yrs, evaluating benznidazole; T.B. pericarditis (50% 6 mo mortality), no major trials.IMPI: steroids and a vaccine in 800( 3000) people. • Neglected populations e.g. children, vulnerable groups, etc. • Neglected conditions, e.g. road traffic accidents (CRASH: steroids in head injuries), cardiac arrest, other critical illness
Intensive care management of severe head injury Percent therapy for intracranial hypertension 1995 USA 1996 UK No.pts(trials) OR(CI) Barbiturates 33% 56% 208(3) 1.09(.91,1.47) Corticosteroids 64% 49% 2119(16) 0.96(.85,1.08) CSF drainage 44% - 0(0) -- Hyperventilation 83% 100% 77(1) 0.73(.36,1.49) Mannitol 83% 100% 44(1) 1.75(.48,6.35)
Placebo-allocated Corticosteroid-allocated 893 / 4 979 (17.9%) 1 052 / 4 985 (21.1%) 1.18 (1.09–1.27) p < 0.001 0.8 1 1.2 1.4 1.6 Corticosteroid better Corticosteroid worse CRASH Trial :Death within 14 days
Increasing Clinical Trials in Disadvantaged/Vulnerable Populations (2) Consider trials with consent of relatives or surrogates and when time is of the essence; perhaps without any consent but IRB approvals CRASH
ACTIVE: Recruitment, Compliance & Data Quality by High & Middle Income Countries
Globalization of Trials in CVD 1. Most CVD trials include LIC/MIC to reduce costs and speed enrollment, and are primarily aimed at answering questions relevant to the West. BUT, these trials may also have applicability in LIC/MIC if the condition is common and the treatments are affordable. 2. Need trials in LIC & MIC of locally relevant conditions and treatments, and locally conducted.