randomized controlled trials allocation concealment l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Randomized Controlled Trials: Allocation Concealment PowerPoint Presentation
Download Presentation
Randomized Controlled Trials: Allocation Concealment

Loading in 2 Seconds...

play fullscreen
1 / 48

Randomized Controlled Trials: Allocation Concealment - PowerPoint PPT Presentation


  • 407 Views
  • Uploaded on

Randomized Controlled Trials: Allocation Concealment. Kenneth F. Schulz, PhD. Fixation on Sequence Generation. Traditionally, many medical researchers mistakenly consider the sequence generation process as “randomization” They frequently ignore allocation concealment

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Randomized Controlled Trials: Allocation Concealment' - Thomas


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
fixation on sequence generation
Fixation on Sequence Generation
  • Traditionally, many medical researchers mistakenly consider the sequence generation process as “randomization”
  • They frequently ignore allocation concealment
  • Without adequate allocation concealment, proper random sequences can be subverted
    • For example, a properly generated sequence posted on a bulletin board
allocation concealment
Allocation Concealment
  • Prevents selection bias by concealing the allocation sequence from those assigning participants to intervention groups
    • Until the moment of assignment
mechanisms for selection bias in rcts
Mechanisms for Selection Biasin RCTs
  • If those responsible for entering participants know, or can detect, the upcoming treatment allocations:
    • Decide entrance based on that knowledge
    • Channel participants with a better prognosis to the experimental group and those with a poorer prognosis to the control group, or vice versa
  • In any case, they introduce bias into the treatment comparison
do not confuse allocation concealment with blinding
Do Not Confuse Allocation Concealment with Blinding
  • Allocation concealment seeks to prevent selection bias, protects assignment sequence before and until allocation, and can always be successfully implemented
  • In contrast, blinding seeks to prevent ascertainment bias, protects sequence after allocation, and cannot always be successfully implemented
importance of allocation concealment
Unclearly concealed and inadequately concealed trials, compared to adequately concealed trials, exaggerated the estimates of an intervention’s effectiveness by 30% to 40%, on averageImportance of Allocation Concealment

Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias: dimensions of methodological quality associated with estimates of treatment effects in controlled trials.JAMA 1995;273:408-412.

similar results on allocation concealment in a separate study
Similar Results on Allocation Concealment in a Separate Study
  • Replicated our methods in different subject areas (digestive & circulatory diseases and mental health)
  • Examined 11 meta-analyses
  • Inadequately concealed trials exaggerated estimates of effectiveness by 37%

[Moher D, et al. Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses? Lancet 1998; 352: 609-13.]

allocation concealment summary
Allocation Concealment: Summary
  • Adequate allocation concealment emerges from our analyses as crucial to reducing bias
  • Without it, the whole point of randomisation vanishes and bias may distort results
  • Our results support Mosteller and his colleagues

“When the randomization leaks, the trial’s guarantee of lack of bias runs down the drain”

slide9

Method of allocation of treatment in 208 controlled trials in head injury

Dickinson K, et al., BMJ 2000;320:1308-1311.

Method of allocation No. of trials

actual allocation method with the method just specified as random
Actual allocation method with the method just specified as “random”
  • Difficult (Peto and I. Chalmers experience)
  • MA of OCs
  • David G. knew the PI who put him in contact with the statistician
  • Described a “great” method . . . Post-it notes!
    • Seemed indignant . . . We asked for clarification
    • Perfect also for stratification—different colors!!
personal accounts of deciphering assignment sequences
Personal Accounts of Deciphering Assignment Sequences
  • Conducted over 20 epidemiological workshops for medical residents and medical junior faculty
  • Each included 20-25 participants
  • Asked how many of the participants had deciphered, or had witnessed someone else decipher, an assignment sequence
personal accounts of deciphering assignment sequences cont
Personal Accounts of Deciphering Assignment Sequences (cont.)
  • With assurance of anonymity, more than half related at least one instance of deciphering
  • Not half of all trials
    • No accurate denominator
  • Nevertheless, not a rare occurrence
decipherings run the gamut from simple to intricate
Decipherings Run the Gamut From Simple to Intricate
  • Simple the most frequent and usually take advantage of inadequate allocation concealment schemes:
    • Posting sequence on a bulletin board
    • Opening unsealed envelopes
    • Translucent envelopes to a light bulb
    • Opening many envelopes that were not sequentially numbered
intricate decipherings less frequent
Intricate Decipherings Less Frequent
  • Needed to circumvent more adequate allocation schemes
    • With sequentially numbered drug containers
      • Based on appearance of tablets in unsealed containers
      • Appearance of the label
    • With central allocation, obtaining the next few allocations at once
    • Envelopes to a “hot light” in radiology
attempted to decipher a numbered container scheme
Attempted to Decipher a Numbered Container Scheme
  • Gave up
  • Found attending physician rifling the P.I.’s files for the assignment sequence
  • Horrified? No, impressed with his brilliance and proceeded to help
  • Should be kept in a locked location
slide16

Ignorance Is Not Bliss

“At the annual meeting of the Society for Clinical Trials, held in Houston, … Schulz reported data showing that the experts’ fears about ignorance are justified. Schulz examined 250 reports of clinical trials and came up with an intriguing finding…”

[Rachel Nowak. Problems in clinical trials go far beyond misconduct. Science 1994; 264: 1539.]

slide17

Ignorance Is Not Bliss

“Most physicians are not trained in basic scientific principles, let alone clinical trials”

Stephen George, Duke University, Chair of the Statistics Committee for NCI, NIH Cancer Clinical Cooperative Groups

“I fantasize that in the future (training) will become a requirement for running clinical trials”

John Gallin, Director of NIH’s Clinical Center

rigorous trials annoy humans
Rigorous Trials Annoy Humans
  • Investigators
    • Certain Ps to benefit
    • May want the results of study to reveal the “truth”
  • Proper trial procedures attempt to impede human inclinations
    • Hence, aspects of properly conducted RCTs annoy investigators
without methodological rigour
Without Methodological Rigour
  • The challenge of deciphering may frequently become too great a temptation to resist
    • Even without intent to bias
  • “The only way to get rid of a temptation is to yield to it.” Oscar Wilde
  • Deciphering may just reflect human inquisitiveness and ingenuity
  • Does not necessarily indicate scientific malevolence
without methodological rigour20
Without Methodological Rigour
  • Or deciphering may reflect deliberate acts to alter findings
  • Whatever the motivations, innocent or naïve or deliberate, those actions undermine the validity of the trial
  • Investigators must devote diligent, persistent attention to randomization
minimal standards common allocation concealment approaches
Minimal Standards: Common Allocation Concealment Approaches
  • Sequentially numbered, opaque, sealed envelopes (SNOSE)
  • Pharmacy control
  • Numbered or coded containers
  • Central randomization
  • Realistically, these standards should be exceeded
envelopes for allocation concealment
Envelopes for Allocation Concealment
  • More susceptible to manipulation through human ingenuity
  • Less than ideal method of allocation concealment
  • If used, investigators must diligently develop and monitor the process
envelopes cont
Envelopes (Cont.)
  • SNOSE
  • Ensure that the envelopes are opened sequentially
  • only after the P’s name and other details are written on the appropriate envelope
envelopes cont d
Envelopes (Cont’d)
  • Pressure-sensitive or carbon paper inside
    • Supports enrolment
    • Encourages orderly opening
    • Promotes proper assignment
    • Creates valuable audit trail
  • Cardboard or aluminum foil inside envelope
pharmacy controlled
“Pharmacy Controlled”
  • Compliance with randomization/allocation concealment methods questionable
  • Aware of gross distortions
    • Pharmacy ran out of one drug . . .
    • Alternate assignment
  • Should not assume pharmacists knowledge
  • Ensure they, and all research partners, follow proper trial procedures
numbered containers
“Numbered Containers”
  • Excellent – prevents foreknowledge
    • Must take proper precautions
  • Appearance – exactly the same
  • Weight – exactly the same
  • Sound when shaken – exactly the same
  • Lettering – exactly the same
  • Odor – exactly the same
  • Sealed
    • Difference between concealment and blinding
central randomization
“Central Randomization”
  • Frequently telephone
    • Fax
    • Email
    • Internet
    • “Computer”
  • Assignments only provided after participants irrevocably enrolled
  • Must check procedures for leaks
  • Monitor process for adherence
rcts anathema to the human spirit
RCTs: Anathema to the Human Spirit
  • Must acknowledge the vagaries of human nature
  • Must establish methodological safeguards that thwart attempts to contaminate trials with bias
subversions of sequences
Subversions of Sequences
  • Conscious or subconscious?
  • Direct or compensatory?
  • Doesn’t matter
    • Biases the trial
slide30

Participants

Randomize

Placebo

New Oral Drug

25% Non-compliance

75% Compliance

60% Compliance

40% Non-compliance

Group representing the policy of no treatment

Group representing the policy of oral treatment

OutcomeOutcome

slide31

RCT Compared the Effectiveness of Clofibrate in Preventing Cardiac Deaths in Men Who Had Survived a Myocardial Infarction

  • Authors state that:
  • One can justify almost any conclusion, dependent upon the analysis chosen
  • Manipulating deviates leads to severe bias
  • Can you ever do so?
exclusions after randomization
Exclusions After Randomization
  • Can introduce bias and should be carefully scrutinized
  • All randomized patients should be analyzed, and analyzed as part of the group to which they were initially assigned
  • ITT (Intention-to-treat)
exclusions of lfu damage internal validity
Exclusions of LFU Damage Internal Validity
  • Without outcomes from those lost to follow-up, investigators have little choice but to exclude them from the analysis
  • Any losses damage internal validity
    • However, differential rates of loss among comparison groups cause major damage
  • Investigators must minimize their losses to follow-up
retention of trial participants
Retention of Trial Participants
  • Minimizing losses exudes difficulties
  • Eliminating losses may be impossible
    • But investigators too frequently profess insurmountable difficulties
  • Most should work harder to obtain higher follow-up rates
    • Investigators must commit adequate attention and resources to develop and implement procedures to minimize losses
innovative twists that cultivate high follow up rates
Innovative Twists that Cultivate High Follow-up Rates
  • Establish many conveniently placed FU facilities
    • Too often investigators expect participants to

visit a single, inconvenient location

  • Shortening data collection form caters to the participants’ wishes
    • Long sessions lead them to vote with their feet
    • Be considerate
    • Foster follow-up by not overburdening
    • Also may engender higher quality data on the

main items

slide37

Meta-analysis of Polyglycolic

Meta-analysis of polyglycolic acid versus catgut for perineal repair on short term pain

1 adequately concealed trial … OR = .89

4 unclearly concealed trials … OR = .44

Protective effect of polyglycotic acid was 51% lower in the unclearly concealed trial

(ROR = .49; 95% CI of .35 - .69)

slide38

Meta-analysis of Corticosteriods

Meta-analysis of corticosteroids vs. none, after preterm rupture of membranes, on respiratory distress syndrome (RDS)

3 adequately concealed trials … OR = 0.72

4 unclearly concealed trials … OR = 0.53

Protective effect of corticosteroids was exaggerated by 27% in the unclearly concealed trials

ROR = 0.73; 95% CI of 0.35 - 1.50

randomization process p 138b
Randomization Process (P 138B)

In the 4 OB/GYN Journals, found the 2 treatment group sizes to be much too similar

separation of executer and generator
Separation of Executer and Generator
  • Person(s) who prepare the randomization scheme should not determine eligibility, administer treatment, or assess outcome
  • Neglected critical element
  • Obviously important—regardless of methodological quality, access to sequence and thus opportunity to introduce bias
  • Faults in this element may be the crack though which much of the bias seeps into controlled trials
slide41

“Not quite as random as I pretended”

[Martyn C. Not quite as random as I pretended. Lancet 1996; 347:70.]

“Bias in the way treatments are assigned can be a more powerful determinant of the outcome of the trial than the treatments that are being investigated.”

“Trialists … often find it hard to prevent their hopes that a new treatment might prove to be beneficial from evolving into

a belief that it really is so.”

rct in norway
RCT in Norway
  • Examined the effect of Chlorhexidine compared to placebo on postpartum infection and Strep B infections in neonates
  • Reported as a randomized controlled trial in the published article

Insert published article

rct in norway43
RCT in Norway
  • Midwives knew on which shelf the Chlorhexidine and placebo were kept in the cabinet and just proceeded to administer whatever they deemed appropriate
  • Even if they forgot the shelf, they could smell the difference
  • No monitoring of the allocation process
letter from a physical therapist
Letter from a Physical Therapist
  • A “friend of mine” conducted an RCT comparing 2 training regimens following surgical repair of the knee for anterior cruciate ligament damage
    • “randomization method used was alternating assignment”
  • Referred to one patient as “difficult” and asked the friend to which group she was assigned
    • Friend responded “that she was in the group that required slightly less contact and that her inclusion was by his choice”
not so blind after all
“Not so blind, after all”

Sometimes subverting can work against a treatment’s apparent effectiveness

“Medical staff … may try to help their sickest patients by sneaking them into the treatment group instead of the control group.”

“The drug would then have to be significantly better than conventional treatment just to appear equal in efficacy”

[Wallich P. Not so blind after all. Scientific American; May 1996.]

rcts anathema to the human spirit46
RCTs: Anathema to the Human Spirit
  • Scientific need to obtain unbiased data from an inherently biased source – human beings
  • Many trialists are unaware of the rationale behind RCTs
    • And the need for trial rigor
  • Some may intellectually grasp the need, but once immersed in a trial, have many contradictory interests (e.g. episiotomy trial in Canada)
anathema a psychologist s view
Anathema: A Psychologist’s View
  • ”Fundamental conflict” between scientific and human imperatives … “Many probably think they already know what treatment is best.”
    • Geoffrey Scobie, psychologist at the University of Glasgow; Doctors cheat because they know what’s best

New Scientist, 16 December 95, p. 10

  • Of course, what we think we ‘know’ is frequently wrong
slide48
Reporting of Exclusions, Double-blinding, and Schedule Generation Related to the Level of Allocation Concealment for 250 Trials