follow up and compliance outline n.
Skip this Video
Loading SlideShow in 5 Seconds..
Follow-up and Compliance: Outline PowerPoint Presentation
Download Presentation
Follow-up and Compliance: Outline

Loading in 2 Seconds...

play fullscreen
1 / 46

Follow-up and Compliance: Outline - PowerPoint PPT Presentation

  • Uploaded on

Follow-up and Compliance: Outline. Follow-up Contents, frequency Importance of complete follow-up Compliance/adherence Importance How to measure and maximize Reported follow-up and compliance CONSORT guidelines ITT and other issues in analysis Definitions Different flavors.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

Follow-up and Compliance: Outline

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
follow up and compliance outline
Follow-up and Compliance: Outline
  • Follow-up
    • Contents, frequency
    • Importance of complete follow-up
  • Compliance/adherence
    • Importance
    • How to measure and maximize
  • Reported follow-up and compliance
    • CONSORT guidelines
  • ITT and other issues in analysis
    • Definitions
    • Different flavors
follow up in rct s
Follow-up in RCT’s
  • What happens after randomization
  • Carefully lay out procedures to be followed
  • Describe on forms and in Operations Manual
  • First reaction: do everything on everyone at every visit
    • e.g. labs at all visits
    • But great opportunities for efficiencies and cost savings
    • Ask the following:
      • Do only at some visits?
      • Do only on a subset?
      • Don’t do at all
follow up styles
Follow up “Styles”
  • Intensity of follow up depends on study needs (and budget)
  • Maximal
    • In patient GCRC-type study
    • Phase I and (sometimes phase II) studies
    • Lots of measurements done frequently
  • Minimalist
    • Few follow up visits and measurements
  • Advantages and disadvantages of each
large and simple trials
Large and Simple Trials
  • Get a whole lot of people
  • Randomize, do as few follow-up measurements as possible
  • Can be difficult to carry out in practice
  • Examples
    • Physicians’ Health study: Randomize to aspirin or placebo, mail out drugs, follow-up by mail
    • Use data collected for other purposes for follow-up/endpoints
      • Population mortality
      • Medical systems (Medicare, Kaiser in U.S.; Gov’t. health in Europe)
    • Internet follow-up direct from participants
large and simple trials vitamin d
Large and Simple Trials: Vitamin D
  • 2700 men and women from general medical practice
  • Oral vitamin D (100,000 units) given 4 x per year (oral) for prevention of fractures in UK
  • Sent oral D (and placebo) via mail
  • Ascertained fractures via “post” (and National Health Service data base)
  • Mortality via National statistics
  • Found decrease in fracture risk (possibly mortality)

Effect of four monthly oral vitamin D3 (cholecalciferol) supplementation on fractures and mortality in men and women living in the community: randomised double blind controlled trial.BMJ. 2003 Mar 1; 326(7387): 469.

vitamin d summary of 5 yr results
Vitamin D: Summary of 5 Yr. Results


(n=1345) (n=1341)

Any fracture 8.8% 11.1% 0.8 (.04)

Hip, wrist, vert. 4.5% 6.5% 0.67 (.02)

Mortality 16.7% 18.4% 0.88 (.18)

CONCLUSION: Four monthly supplementation with 100 000 IU oral vitamin D may prevent fractures without adverse effects in men and women living in the general community

BMJ. 2003 Mar 1; 326(7387): 469.

large and simple trials1
Large and Simple Trials
  • Efficiencies:
    • No selection criteria (other than age)
    • No follow-up clinic visits or measurements
    • No follow-up labs or measurements
    • Outcomes from self-assessed mail-in
  • Study very efficient (VERY) and very generalizable
    • Cost of Vitamin D trial: about $500,000.
    • Cost of WHI: about $1,000,000,000.
compliance or mpc adherence
Compliance or (mpc) adherence
  • Trial is meaningless unless participants adhere to interventions
  • Think in very precise terms about meaning
    • Term “drop out” often used but is ambiguous
  • Several aspects
    • 1. Adherence to medications/interventions
    • 2. Adherence to visit schedules/reporting
    • 3. Adherence to protocol (eg. Admitting only eligibles)
  • Lack of adherence leads to:
    • Bias in either efficacy or safety
    • Decreased power
    • Uninterpretable results
impact of loss to follow up
Impact of Loss to Follow up
  • Can lead to bias especially when LTFU varies by treatment
    • Eg. Treated drop out more than placebo
  • Other forms of bias may exist in those lost
    • Older, younger, sicker
  • Even random loss to follow up could have impact
potential effect of incomplete visit follow up on results in clinical trials
Potential effect of incomplete visit follow-up on results in clinical trials

- Fracture Intervention Trial (alendronate vs. placebo)

- X-rays obtained at baseline, 2 years, 3 years

- Vertebral fractures defined from changes in radiographs

- FU radiographs on 97% of participants @ year 3

Time (yrs)Relative risk (CI)

BL to 2 0.34 (66% reduction)

BL to 3 0.49 ( 51% reduction)

effect of incomplete follow up virtual experiment
Effect of Incomplete Follow-up: Virtual Experiment
  • FIT I: Follow-up x-rays on 97% of surviving participants at year 3
  • What if follow-up less complete?
  • Randomly “lose” 50% between year 2 and 3
use of survival analysis for x rays in fit i virtual experiment
Use of Survival Analysis for X-Rays in FIT I: Virtual Experiment

Time (yrs)Relative risk

2 0.34

3 0.49

3 (50% LTFU) 0.37

LTFU = Lost to follow-up

effect of high rate of loss to follow up on results
Effect of High Rate of Loss to Follow-up on Results
  • If early results differ from later results, could create bias when comparing one study to another
  • Even a “random” (therefore unbiased) loss to follow-up can affect results
  • Clearly most losses to follow-up are not unbiased so more reasons for concern
effect of high rate of loss to follow up on results1
Effect of High Rate of Loss to Follow-up on Results
  • More generally, people lost to follow-up may be different than those who remain
  • Could be differential in two treatment groups
    • Due to treatment (e.g. estrogen)
  • Advantages of randomization may be lost
    • Groups may no longer be comparable
  • Could bias results
measuring adherence
Measuring adherence
  • Medication-taking
    • Just ask! (self report)
    • Pill counts
    • Biochemical assays for some drugs
    • High tech pill bottles
      • Can count number of times bottle actually opened
  • Visit schedule
    • N missed visits
    • Visits within schedule
    • etc.
adherence goals
Adherence goals
  • Ideal: all participants continue to take medication (perfectly) throughout the trial and attend all follow-up visits until the very end
  • Why might participants stop medication?
    • Side effects (real or perceived)
    • Complex regimens
    • Want to take true active medication
      • New info on old medication
      • New competing medication
    • Want to stop active medication
      • New info on old medication (e.g, ERT increases BC risk)
some examples of bad adherence days
Some Examples of “Bad Adherence Days”
  • Women’s Health Initiative
    • After first year, letter sent to all participants “observed a small increase in cardiovascular disease among ppts on HRT”…
    • Many stopped medications
  • PROOF trial (effect of Calcitonin on osteoporosis)
    • 1994 to 1999
    • 1997: Alendronate approved with significant marketing and excellent results
effect of stopping medication classical interpretation
Effect of Stopping Medication: Classical interpretation
  • Placebo’s start active medication==>become more like actives
  • Actives stop active medication and start “inactive”==>become more like placebo
  • Two groups become more similar
  • Treatment effect is underestimated/conservative
    • Comforting
  • “Classical interpretation” may not hold:
    • Example: patients stop study meds to take a medication that is better than active study medication
strategies to enhance compliance
Strategies to enhance compliance
  • Warm and fuzzy stuff
    • Participants to feel appreciated
    • Staff in clinic spend enough time
    • Sensitive to ppts. scheduling needs
  • Ease of logistics/transportation to clinics
    • Ex. Van for transportation: encourage group solidarity
  • Parties/events with all participants
  • Birthday cards
  • Gifts (e.g. gift cards for moderate amounts)
  • Information, Newsletters, other
strategies to enhance compliance ii
Strategies to enhance compliance II
  • Most drop outs occur in early study period
    • FIT (4 years total); 2/3 of drop outs occurred in first year, most of those in first 6 months
  • Care in recruitment strategies
  • Make certain that ppt’s understand study requirements
  • Run-in period
    • Trial run of drug/treatment
    • Typically 2-4 weeks, usually of placebo (not always)
    • Value controversial
adherence to follow up visits
Adherence to Follow-up visits
  • Goal: visits all on time (within window)
  • Set appointments flexibly
  • Reminders prior to appt.
  • Give study calendar
  • Listen to concerns/problems
study management methods to enhance adherence
Study Management Methods to Enhance Adherence
  • Monitor adherence during study
  • Important to assess as go along
    • Spot systematic or individual problems (and make corrections)
  • Can manage via study website
example of visit adherence monitoring
Example of Visit Adherence Monitoring
  • Horizon study
  • IV dosing of zoledronic acid
  • 1 dose per year
  • Endpoint: hip and vertebral fractures
  • 7400 women in 23 countries, 269 sites
  • Adherence to visit schedule monitored via IVRS
    • Interactive Voice Response System (telephone “punch 1, punch 2”) system
  • Reports on web
adherence to medication is not the same as adherence to visit schedule
Adherence to medication is not the same as adherence to visit schedule
  • “Drop out” is very vague term
  • Can have perfect visit adherence (come to all visits on time) but--
    • Not take a single study med pill
    • Take only 60% of pills
  • If miss visits or stop coming to visits, then generally don’t take study medication
    • Exceptions do occur: Trial of once-yearly infusion treatment. May have perfect medication compliance but poor visit compliance
follow up visits for those who have stopped study medications
Follow-up visits for those who have stopped study medications?
  • Practice varies dramatically across studies
  • Option 1: Stop follow-up as soon as drug stops
  • Option 2: Continue to collect follow-up info
  • Advantages of each
    • ??
follow up visits for those who have stopped study medications1
Follow-up visits for those who have stopped study medications?
  • Practice varies dramatically across studies
  • Option 1: Stop follow-up as soon as drug stops
  • Option 2: Continue to collect follow-up info

Saves money

Won’t see long term negative effects

Have full follow-up on all participants

Can do all sorts of subsets and per protocol analyses

Increase costs

intention to treat analysis itt
Intention to Treat Analysis (ITT)
  • ITT coined by AB Hill in textbook on Stat (1961)
  • One of the main Commandments of RCT bible
  • Original definition “All subjects will be analyzed according to the treatment group they were originally intended by the randomization process”
  • “generally interpreted as including all patients, regardless of whether they satisfied entry criteria, treatment was actually received or withdrawal or deviation from protocol”-- Hollis BMJ article
true meaning of itt is ambiguous
True Meaning of ITT is Ambiguous
  • Most rigorous interpretation includes the following principles (Hollis, discussion):
    • All patients included in analysis even if they didn’t start intervention
    • Patients who were randomized but did not meet inclusion criterion should be included in ITT
    • All patients included regardless of compliance
    • Ascertainment for primary outcome on all randomized
two purposes of itt from hollis
Two Purposes of ITT (from Hollis)
  • 1. Maintain validity of original randomization
    • Groups can only differ by chance
    • Exclusion of some subjects post-randomization (e.g. didn’t take any pills) could create bias
    • Medical/surgery example in Hollis (table 1)
  • 2. Makes clinical trial more like real-world situation
    • Clinicians in real world have been known to deviate from protocols
    • (Probably a bit naive to compare any trial to the real clinical world)
beware of we did an itt analysis
Beware of “we did an ITT analysis”
  • Generally considered sacred, almost god-like virtue
  • The term “ITT” used differently in different studies
  • Probably does mean that all randomized (or most) were included in analysis
  • Examples of differences in meanings:
    • In practice, ITT does NOT always mean that participants were followed beyond stopping study medications
how are itt principles applied in real trials
How are ITT Principles Applied in Real Trials?
  • Hollis (1999) evaluated application of ITT in 249 trials in BMJ, JAMA, Lancet, NEJM
    • About 50% (119 trials) reported using ITT
  • Found that most trials with (ITT claimed) violated one or more of the 4 ITT principles
per protocol analyses as alternatives to itt
Per protocol analyses as alternatives to ITT

If all ppts. are followed regardless of adherence to medications, several types of options

Various flavors of per protocol:

  • Include only those patients who took all study medications and completed all visits
  • Include all patients but only for the time that they remained on study medications

If obtain complete follow-up on all ppts., can run several different types of analyses and any discrepancies could be informative.

analysis based on post randomization variables
Analysis based on post-randomization variables
  • Per-protocol limits analysis to adherers
  • Per-protocol is one example of analysis which stratifies based on post-randomization experience
    • Other examples?
  • More generally, subgroup analyses by post-rand. factors are biased, sometimes extremely biased--BEWARE
can analysis be restricted to those who adhere coronary drug project cdp nejm 1980
Can analysis be restricted to those who adhere? Coronary Drug Project (CDP, NEJM 1980)

5 year mortality

Overall Adherence

> 80% (2/3)< 80% (1/3)

Clofibrate (n=1065) 18% 15% 25%

can analysis be restricted to those who adhere coronary drug project cdp nejm 19801
Can analysis be restricted to those who adhere? Coronary Drug Project (CDP, NEJM 1980)

5 year mortality

Overall Adherence

> 80% (2/3)< 80% (1/3)

Clofibrate (n=1065) 18% 15% 25%

Placebo (n=2695) 19% 15% 28%


  • Unknown/unmeasured confounders associated with compliance
  • Differ in placebo and active groups
  • Be wary of post-randomization groupings
problems with itt full follow up approach
Problems with ITT/full follow-up approach
  • ITT/full follow-up not holy grail
  • Does not estimate full biologic efficacy of drug/intervention
    • Advising individual patients may depend on efficacy
    • Utility underestimated
  • May be anti-conservative for adverse effects
    • per-protocol may be preferred
consort guidelines for reporting results of trials
CONSORT Guidelines for Reporting Results of Trials
  • “The CONSORT statement is an important research tool that takes an evidence-based approach to improve the quality of reports of randomized trials. The statement is available in several languages and has been endorsed by prominent medical journals such as The Lancet, Annals of Internal Medicine, and the Journal of the American Medical Association”
  • Includes guidelines for reporting trials and the “Consort Flowchart”

Example of CONSORT Diagram:

Trial of Zoledronic acid vs. placebo

Publication revision to be submitted ~ 2/8/07

summary of itt from hollis in bmj
Summary of ITT from Hollis in BMJ:
  • The Brits: they don’t know how to cook nor run a monarchy but….
  • They sure do talk gooder than us Americans
itt key messages hollis
ITT: Key Messages (Hollis)
  • ITT gives a pragmatic estimate of benefit of treatment policy rather benefit in patients who receive treatment exactly as planned
  • Full application of ITT possible only when complete outcome available on all randomized
  • Many trials that claim ITT varied in handling of missing data, deviations from protocol, etc.
  • ITT often inadequately described and applied
recommendations for itt hollis
Recommendations for ITT (Hollis)
  • ITT best regarded as complete trial strategy (design, conduct, analysis) and not simply analysis
  • Design
    • Justify in advance any inclusion which if violated merit exclusion from ITT
  • Conduct
    • Minimise (sic) missing response on primary outcome
    • Follow up subjects who withdraw from treatment
  • Analysis
    • Investigate potential effects of missing response
recommendations for itt
Recommendations for ITT
  • Reporting
    • Specify how ITT used (explicitly describe handling of deviations from randomization and missing responses)
    • Report deviations and missingness
    • Discuss potential effects of missing reponse
    • Base conclusions on ITT analyses
follow up and adherence summary
Follow-up and Adherence: summary
  • Best trial:
    • All participants remain on medication
    • All participants are followed until end of study
    • Pre-planned analysis and handling of deviations from protocol
  • Design and implemention of study can help move toward that goal
  • Important to report details about adherence and how it could affect results