Randomized controlled trials allocation concealment
1 / 48

Randomized Controlled Trials: Allocation Concealment - PowerPoint PPT Presentation

  • Updated On :

Randomized Controlled Trials: Allocation Concealment. Kenneth F. Schulz, PhD. Fixation on Sequence Generation. Traditionally, many medical researchers mistakenly consider the sequence generation process as “randomization” They frequently ignore allocation concealment

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Randomized Controlled Trials: Allocation Concealment' - Thomas

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Fixation on sequence generation l.jpg
Fixation on Sequence Generation

  • Traditionally, many medical researchers mistakenly consider the sequence generation process as “randomization”

  • They frequently ignore allocation concealment

  • Without adequate allocation concealment, proper random sequences can be subverted

    • For example, a properly generated sequence posted on a bulletin board

Allocation concealment l.jpg
Allocation Concealment

  • Prevents selection bias by concealing the allocation sequence from those assigning participants to intervention groups

    • Until the moment of assignment

Mechanisms for selection bias in rcts l.jpg
Mechanisms for Selection Biasin RCTs

  • If those responsible for entering participants know, or can detect, the upcoming treatment allocations:

    • Decide entrance based on that knowledge

    • Channel participants with a better prognosis to the experimental group and those with a poorer prognosis to the control group, or vice versa

  • In any case, they introduce bias into the treatment comparison

Do not confuse allocation concealment with blinding l.jpg
Do Not Confuse Allocation Concealment with Blinding

  • Allocation concealment seeks to prevent selection bias, protects assignment sequence before and until allocation, and can always be successfully implemented

  • In contrast, blinding seeks to prevent ascertainment bias, protects sequence after allocation, and cannot always be successfully implemented

Importance of allocation concealment l.jpg

Unclearly concealed and inadequately concealed trials, compared to adequately concealed trials, exaggerated the estimates of an intervention’s effectiveness by 30% to 40%, on average

Importance of Allocation Concealment

Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias: dimensions of methodological quality associated with estimates of treatment effects in controlled trials.JAMA 1995;273:408-412.

Similar results on allocation concealment in a separate study l.jpg
Similar Results on Allocation Concealment in a Separate Study

  • Replicated our methods in different subject areas (digestive & circulatory diseases and mental health)

  • Examined 11 meta-analyses

  • Inadequately concealed trials exaggerated estimates of effectiveness by 37%

[Moher D, et al. Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses? Lancet 1998; 352: 609-13.]

Allocation concealment summary l.jpg
Allocation Concealment: Summary Study

  • Adequate allocation concealment emerges from our analyses as crucial to reducing bias

  • Without it, the whole point of randomisation vanishes and bias may distort results

  • Our results support Mosteller and his colleagues

“When the randomization leaks, the trial’s guarantee of lack of bias runs down the drain”

Slide9 l.jpg

Method of allocation of treatment in 208 controlled trials in head injury

Dickinson K, et al., BMJ 2000;320:1308-1311.

Method of allocation No. of trials

Actual allocation method with the method just specified as random l.jpg
Actual allocation method with the method just specified as “random”

  • Difficult (Peto and I. Chalmers experience)

  • MA of OCs

  • David G. knew the PI who put him in contact with the statistician

  • Described a “great” method . . . Post-it notes!

    • Seemed indignant . . . We asked for clarification

    • Perfect also for stratification—different colors!!

Personal accounts of deciphering assignment sequences l.jpg
Personal Accounts of Deciphering Assignment Sequences “random”

  • Conducted over 20 epidemiological workshops for medical residents and medical junior faculty

  • Each included 20-25 participants

  • Asked how many of the participants had deciphered, or had witnessed someone else decipher, an assignment sequence

Personal accounts of deciphering assignment sequences cont l.jpg
Personal Accounts of Deciphering Assignment Sequences “random”(cont.)

  • With assurance of anonymity, more than half related at least one instance of deciphering

  • Not half of all trials

    • No accurate denominator

  • Nevertheless, not a rare occurrence

Decipherings run the gamut from simple to intricate l.jpg
Decipherings Run the Gamut From Simple to Intricate “random”

  • Simple the most frequent and usually take advantage of inadequate allocation concealment schemes:

    • Posting sequence on a bulletin board

    • Opening unsealed envelopes

    • Translucent envelopes to a light bulb

    • Opening many envelopes that were not sequentially numbered

Intricate decipherings less frequent l.jpg
Intricate Decipherings Less Frequent “random”

  • Needed to circumvent more adequate allocation schemes

    • With sequentially numbered drug containers

      • Based on appearance of tablets in unsealed containers

      • Appearance of the label

    • With central allocation, obtaining the next few allocations at once

    • Envelopes to a “hot light” in radiology

Attempted to decipher a numbered container scheme l.jpg
Attempted to Decipher a Numbered Container Scheme “random”

  • Gave up

  • Found attending physician rifling the P.I.’s files for the assignment sequence

  • Horrified? No, impressed with his brilliance and proceeded to help

  • Should be kept in a locked location

Slide16 l.jpg

Ignorance Is Not Bliss “random”

“At the annual meeting of the Society for Clinical Trials, held in Houston, … Schulz reported data showing that the experts’ fears about ignorance are justified. Schulz examined 250 reports of clinical trials and came up with an intriguing finding…”

[Rachel Nowak. Problems in clinical trials go far beyond misconduct. Science 1994; 264: 1539.]

Slide17 l.jpg

Ignorance Is Not Bliss “random”

“Most physicians are not trained in basic scientific principles, let alone clinical trials”

Stephen George, Duke University, Chair of the Statistics Committee for NCI, NIH Cancer Clinical Cooperative Groups

“I fantasize that in the future (training) will become a requirement for running clinical trials”

John Gallin, Director of NIH’s Clinical Center

Rigorous trials annoy humans l.jpg
Rigorous Trials Annoy Humans “random”

  • Investigators

    • Certain Ps to benefit

    • May want the results of study to reveal the “truth”

  • Proper trial procedures attempt to impede human inclinations

    • Hence, aspects of properly conducted RCTs annoy investigators

Without methodological rigour l.jpg
Without Methodological Rigour “random”

  • The challenge of deciphering may frequently become too great a temptation to resist

    • Even without intent to bias

  • “The only way to get rid of a temptation is to yield to it.” Oscar Wilde

  • Deciphering may just reflect human inquisitiveness and ingenuity

  • Does not necessarily indicate scientific malevolence

Without methodological rigour20 l.jpg
Without Methodological Rigour “random”

  • Or deciphering may reflect deliberate acts to alter findings

  • Whatever the motivations, innocent or naïve or deliberate, those actions undermine the validity of the trial

  • Investigators must devote diligent, persistent attention to randomization

Minimal standards common allocation concealment approaches l.jpg
Minimal Standards: Common Allocation Concealment Approaches “random”

  • Sequentially numbered, opaque, sealed envelopes (SNOSE)

  • Pharmacy control

  • Numbered or coded containers

  • Central randomization

  • Realistically, these standards should be exceeded

Envelopes for allocation concealment l.jpg
Envelopes for Allocation Concealment “random”

  • More susceptible to manipulation through human ingenuity

  • Less than ideal method of allocation concealment

  • If used, investigators must diligently develop and monitor the process

Envelopes cont l.jpg
Envelopes “random”(Cont.)


  • Ensure that the envelopes are opened sequentially

  • only after the P’s name and other details are written on the appropriate envelope

Envelopes cont d l.jpg
Envelopes “random”(Cont’d)

  • Pressure-sensitive or carbon paper inside

    • Supports enrolment

    • Encourages orderly opening

    • Promotes proper assignment

    • Creates valuable audit trail

  • Cardboard or aluminum foil inside envelope

Pharmacy controlled l.jpg
“Pharmacy Controlled” “random”

  • Compliance with randomization/allocation concealment methods questionable

  • Aware of gross distortions

    • Pharmacy ran out of one drug . . .

    • Alternate assignment

  • Should not assume pharmacists knowledge

  • Ensure they, and all research partners, follow proper trial procedures

Numbered containers l.jpg
“Numbered Containers” “random”

  • Excellent – prevents foreknowledge

    • Must take proper precautions

  • Appearance – exactly the same

  • Weight – exactly the same

  • Sound when shaken – exactly the same

  • Lettering – exactly the same

  • Odor – exactly the same

  • Sealed

    • Difference between concealment and blinding

Central randomization l.jpg
“Central Randomization” “random”

  • Frequently telephone

    • Fax

    • Email

    • Internet

    • “Computer”

  • Assignments only provided after participants irrevocably enrolled

  • Must check procedures for leaks

  • Monitor process for adherence

Rcts anathema to the human spirit l.jpg
RCTs: Anathema to the Human Spirit “random”

  • Must acknowledge the vagaries of human nature

  • Must establish methodological safeguards that thwart attempts to contaminate trials with bias

Subversions of sequences l.jpg
Subversions of Sequences “random”

  • Conscious or subconscious?

  • Direct or compensatory?

  • Doesn’t matter

    • Biases the trial

Slide30 l.jpg

Participants “random”



New Oral Drug

25% Non-compliance

75% Compliance

60% Compliance

40% Non-compliance

Group representing the policy of no treatment

Group representing the policy of oral treatment


Slide31 l.jpg

RCT Compared the Effectiveness of Clofibrate in Preventing Cardiac Deaths in Men Who Had Survived a Myocardial Infarction

  • Authors state that:

  • One can justify almost any conclusion, dependent upon the analysis chosen

  • Manipulating deviates leads to severe bias

  • Can you ever do so?

Exclusions after randomization l.jpg
Exclusions After Randomization Cardiac Deaths in Men Who Had Survived a Myocardial Infarction

  • Can introduce bias and should be carefully scrutinized

  • All randomized patients should be analyzed, and analyzed as part of the group to which they were initially assigned

  • ITT (Intention-to-treat)

Exclusions of lfu damage internal validity l.jpg
Exclusions of LFU Damage Internal Validity Cardiac Deaths in Men Who Had Survived a Myocardial Infarction

  • Without outcomes from those lost to follow-up, investigators have little choice but to exclude them from the analysis

  • Any losses damage internal validity

    • However, differential rates of loss among comparison groups cause major damage

  • Investigators must minimize their losses to follow-up

Retention of trial participants l.jpg
Retention of Trial Participants Cardiac Deaths in Men Who Had Survived a Myocardial Infarction

  • Minimizing losses exudes difficulties

  • Eliminating losses may be impossible

    • But investigators too frequently profess insurmountable difficulties

  • Most should work harder to obtain higher follow-up rates

    • Investigators must commit adequate attention and resources to develop and implement procedures to minimize losses

Innovative twists that cultivate high follow up rates l.jpg
Innovative Twists that Cultivate High Follow-up Rates Cardiac Deaths in Men Who Had Survived a Myocardial Infarction

  • Establish many conveniently placed FU facilities

    • Too often investigators expect participants to

      visit a single, inconvenient location

  • Shortening data collection form caters to the participants’ wishes

    • Long sessions lead them to vote with their feet

    • Be considerate

    • Foster follow-up by not overburdening

    • Also may engender higher quality data on the

      main items

Slide36 l.jpg
END Cardiac Deaths in Men Who Had Survived a Myocardial Infarction

Slide37 l.jpg

Meta-analysis of Polyglycolic Cardiac Deaths in Men Who Had Survived a Myocardial Infarction

Meta-analysis of polyglycolic acid versus catgut for perineal repair on short term pain

1 adequately concealed trial … OR = .89

4 unclearly concealed trials … OR = .44

Protective effect of polyglycotic acid was 51% lower in the unclearly concealed trial

(ROR = .49; 95% CI of .35 - .69)

Slide38 l.jpg

Meta-analysis of Corticosteriods Cardiac Deaths in Men Who Had Survived a Myocardial Infarction

Meta-analysis of corticosteroids vs. none, after preterm rupture of membranes, on respiratory distress syndrome (RDS)

3 adequately concealed trials … OR = 0.72

4 unclearly concealed trials … OR = 0.53

Protective effect of corticosteroids was exaggerated by 27% in the unclearly concealed trials

ROR = 0.73; 95% CI of 0.35 - 1.50

Randomization process p 138b l.jpg
Randomization Process (P 138B) Cardiac Deaths in Men Who Had Survived a Myocardial Infarction

In the 4 OB/GYN Journals, found the 2 treatment group sizes to be much too similar

Separation of executer and generator l.jpg
Separation of Executer and Generator Cardiac Deaths in Men Who Had Survived a Myocardial Infarction

  • Person(s) who prepare the randomization scheme should not determine eligibility, administer treatment, or assess outcome

  • Neglected critical element

  • Obviously important—regardless of methodological quality, access to sequence and thus opportunity to introduce bias

  • Faults in this element may be the crack though which much of the bias seeps into controlled trials

Slide41 l.jpg

“Not quite as random as I pretended” Cardiac Deaths in Men Who Had Survived a Myocardial Infarction

[Martyn C. Not quite as random as I pretended. Lancet 1996; 347:70.]

“Bias in the way treatments are assigned can be a more powerful determinant of the outcome of the trial than the treatments that are being investigated.”

“Trialists … often find it hard to prevent their hopes that a new treatment might prove to be beneficial from evolving into

a belief that it really is so.”

Rct in norway l.jpg
RCT in Norway Cardiac Deaths in Men Who Had Survived a Myocardial Infarction

  • Examined the effect of Chlorhexidine compared to placebo on postpartum infection and Strep B infections in neonates

  • Reported as a randomized controlled trial in the published article

Insert published article

Rct in norway43 l.jpg
RCT in Norway Cardiac Deaths in Men Who Had Survived a Myocardial Infarction

  • Midwives knew on which shelf the Chlorhexidine and placebo were kept in the cabinet and just proceeded to administer whatever they deemed appropriate

  • Even if they forgot the shelf, they could smell the difference

  • No monitoring of the allocation process

Letter from a physical therapist l.jpg
Letter from a Physical Therapist Cardiac Deaths in Men Who Had Survived a Myocardial Infarction

  • A “friend of mine” conducted an RCT comparing 2 training regimens following surgical repair of the knee for anterior cruciate ligament damage

    • “randomization method used was alternating assignment”

  • Referred to one patient as “difficult” and asked the friend to which group she was assigned

    • Friend responded “that she was in the group that required slightly less contact and that her inclusion was by his choice”

Not so blind after all l.jpg
“Not so blind, after all” Cardiac Deaths in Men Who Had Survived a Myocardial Infarction

Sometimes subverting can work against a treatment’s apparent effectiveness

“Medical staff … may try to help their sickest patients by sneaking them into the treatment group instead of the control group.”

“The drug would then have to be significantly better than conventional treatment just to appear equal in efficacy”

[Wallich P. Not so blind after all. Scientific American; May 1996.]

Rcts anathema to the human spirit46 l.jpg
RCTs: Anathema to the Human Spirit Cardiac Deaths in Men Who Had Survived a Myocardial Infarction

  • Scientific need to obtain unbiased data from an inherently biased source – human beings

  • Many trialists are unaware of the rationale behind RCTs

    • And the need for trial rigor

  • Some may intellectually grasp the need, but once immersed in a trial, have many contradictory interests (e.g. episiotomy trial in Canada)

Anathema a psychologist s view l.jpg
Anathema: A Psychologist’s View Cardiac Deaths in Men Who Had Survived a Myocardial Infarction

  • ”Fundamental conflict” between scientific and human imperatives … “Many probably think they already know what treatment is best.”

    • Geoffrey Scobie, psychologist at the University of Glasgow; Doctors cheat because they know what’s best

      New Scientist, 16 December 95, p. 10

  • Of course, what we think we ‘know’ is frequently wrong

Slide48 l.jpg
Reporting of Exclusions, Double-blinding, and Schedule Generation Related to the Level of Allocation Concealment for 250 Trials