Can addiction research be trusted
1 / 59

Can Addiction Research Be Trusted? - PowerPoint PPT Presentation

  • Uploaded on

Can Addiction Research Be Trusted?. Introducing EBAM* (*Evidence Based Addiction Medicine). Stewart B. Leavitt, PhD, Editor Addiction Treatment Forum Sponsored by an educational grant from Mallinckrodt Inc. EBAM Guides Available. “Can Addiction Research Be Trusted?” – report

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Can Addiction Research Be Trusted?' - eudora

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Can addiction research be trusted l.jpg

Can Addiction Research Be Trusted?

Introducing EBAM*(*Evidence Based Addiction Medicine)

Stewart B. Leavitt, PhD, EditorAddiction Treatment Forum

Sponsored by an educational grant from Mallinckrodt Inc.

Ebam guides available l.jpg
EBAM Guides Available

  • “Can Addiction Research Be Trusted?” – report

  • Available online at

  • Look under Addiction Resources tab

  • Slides from this presentation also available

    • Called “EBAM AATOD Slides”

    • PowerPoint format

Ebam for practitioners l.jpg
“EBAM for Practitioners”

  • Available at… under “Addiction Resources” tab.

  • A more technical explanation for persons who need to critically assess research evidence.

Can addiction research be trusted yes no sometimes l.jpg
Can Addiction ResearchBe Trusted?Yes? No? Sometimes?

  • Research must be critically examined and interpreted for particular clinical needs and patient benefits.

  • Just because something appears in print does not mean it is worthwhile or valid.

Why should you care l.jpg
Why Should You Care?

  • Research directly or indirectly affects the practice of addiction medicine on a daily basis.

  • There is an urgent need forgood science to overcome stigma, prejudice, and misunderstanding.

Trepanation once was a “scientific cure” formental illness.

What is ebam evidence based addiction medicine l.jpg
What is EBAM?Evidence-Based Addiction Medicine

  • The use of current best evidence in making decisions about the care of patients.

  • Involves combining clinical expertise with valid research on a topic of concern.

  • Differentiates between practices based on sound evidence versus those founded more on 1) tradition, 2) outdated beliefs, or 3) “junk” science.

Why ebam l.jpg

EBAM approaches allow you to respond more convincingly to the following questions….

  • Where did you hear about that treatment?

  • How do you know the information is worthwhile and true?

  • What do you propose doing and what results do you expect?

  • What are the costs versus benefits?

Essential understandings l.jpg
Essential Understandings

  • Medical research is an imperfect science.

  • Medical writing/reporting is often biased.

  • Scientific research does not “prove” anything to be true or not true.

The nature of clinical research l.jpg
The Nature of Clinical Research

  • Compares experimental treatment (intervention) with standard therapy or placebo.

  • Research subjects represent only a tiny portion of the population in question – statistical analyses estimate effects in the larger population.

  • All things are possible (some patients recover without treatment), and chance can play an important role in the results.

Interpreting research l.jpg
Interpreting Research

  • Interpretation is a matter of perception.

  • Not everyone “sees” a research study in the same way.

  • Becoming a more critical consumer of research is essential.

Problems with medical research articles overall l.jpg
Problems with Medical Research Articles Overall

Surveys of research have found startling results…

  • On average, half or more of journal articles contain errors – from missing data to major design flaws.

  • Roughly, up to 2/3 of article abstracts may contain data that is either inconsistent with or missing from the main body of the article.

  • In one investigation, 80% of systematic reviews & meta-analyses examined had serious flaws.

Concerns about addiction research studies l.jpg
Concerns About Addiction Research Studies

  • Many articles appear to be written for other researchers, rather than practitioners.

  • There are often too few subjects and/or inadequate effects for accurately detecting significant group differences.

  • Subjects are often selected to maximize outcome results, but they are not typical patients.

  • Cases are presented as if they are evidence of causality or applicable to broader groups.

  • There may be a biased leap of interpretation from laboratory experiments to potential effects in humans.

What really matters l.jpg
What Really Matters?

  • “Not everything that can be counted counts, and not everything that counts can be counted. What really mattersis not always obvious.”– Albert Einstein

  • Just because data were collected, analyzed, and reported does not mean they are relevant to the clinical question at hand – but it mattered to somebody.

What are significant outcomes l.jpg
What Are Significant Outcomes?

  • Statistical Significance – the likelihood (probability) that the results represent true differences between groups.

  • Clinical Significance – importance of the results for providing meaningful benefits to patients.

  • Statistical significance does not automatically transfer to clinical significance.

Taking a closer look at clinical research articles l.jpg

Taking A Closer LookAtClinical Research Articles

Anatomy of a research article l.jpg
Anatomy of a Research Article







Abstract l.jpg

  • Presents overview: who, what, where, when, why, and how.

  • Structured abstract is a mini-article…

    • Summarizes major sections of article.

    • Limited to 250 words.

  • Abstract may be inaccurate & misleading.

Introduction l.jpg

  • States purpose of study, including:

    • clinical background,

    • review of previous research,

    • rationale for current investigation.

  • Identifies questions (hypotheses) to be investigated in present study.

Methods l.jpg

  • Defines the patients (subjects) included in the study and the sample sizes.

  • Describes in detail how the study was conducted (treatment protocol).

  • Defines outcomes of importance and how they were measured.

  • Describes how data was analyzed (statistics).

Results l.jpg

  • Describes in detail what was found:

    • differences in treatment effects (outcomes),

    • their significance (p values),

    • and, confidence limits.

  • Should show how statistical procedures were performed and any data adjustments (eg, for noncompliant/dropout subjects).

Discussion l.jpg

  • Recaps results (the “payoff”).

  • Interprets findings in comparison with other research on the topic.

  • Describes study limitations.

  • Notes other factors that might have influenced outcomes.

Conclusion l.jpg

  • Presents case for “external validity” – how, when, where, and in which patients the results might be applied in clinical settings.

  • Often provides suggestions for further research on the subject.

Hierarchy of evidence clinical therapy research l.jpg
Hierarchy of EvidenceClinical Therapy Research

Systematic Reviews / Meta-Analyses of RCTs.

Randomized Controlled Trials (RCTs).

Cohort Studies

Case-Control Studies

Cross-Sectional Surveys

Case Reports

“Perspectives” Articles

Perspectives articles l.jpg
“Perspectives” Articles

  • Aka* overviews, reports, commentary, editorials, or interviews. *Also known as…

  • Most common in addiction field; often cited as evidence.

  • Summarize or comment on research, and are the most subject to bias.

  • Usually, favors one viewpoint (perspective) over another.

The emperor s new perspective clothes l.jpg
“The Emperor’s NewPerspective (Clothes)”

  • “He thought it better to continue the illusion that anyone who couldn’t see his clothes was either stupid or incompetent.” – H.C. Anderson

  • Sometimes, the higher the “authority” of the source, theless its perspective can be trusted as being valid and unbiased.

Case reports l.jpg
Case Reports

  • Aka case histories, case series, or anecdotes.

  • Draw upon personal observations or medical records to report unexpected events.

  • Subject to errors in observation or interpretation, or inadequate documentation.

  • May arrive at unsupported conclusions.

  • “The plural of anecdote is not evidence.” – Leshner

Cross sectional surveys l.jpg
Cross-Sectional Surveys

  • Aka prevalence or epidemiology studies.

  • Examines relationships of medical conditions, treatments, and/or other factors in a patient population at a particular time.

  • Can establish associations but not causality.

  • May be problems with recall and extraneous factors (confounders).

Case control studies l.jpg
Case-Control Studies

  • Aka case-referent, case-comparison, or retrospective studies.

  • Identifies patients with outcome of interest (Cases) and Control patients without the outcome.

  • Researchers look back in time to see how the two groups differed (especially, regarding treatment or exposure received).

  • Subject to recall bias or inconsistent records.

A classic case control study l.jpg
A “Classic” Case-Control Study

  • 25 persons (Cases) exposed to Tutankhamen’s tomb in 1923-26.

  • Compared with 11 explorers in Egypt at that time not exposed (Controls).

Results: No significant differences in mean age at death or in years survival after exposure.

Conclusion: Mummy’s curse was an urban legend.

Cohort studies l.jpg
Cohort Studies

  • Aka followup, incidence, longitudinal, observational, or prospective studies.

  • Most common trial in addiction medicine.

  • An Experimental group (cohort) receives treatment of interest; Control group does not.

  • Outcome differences between groups are compared and contrasted.

  • Nonrandomized, many factors uncontrolled.

A first clinical cohort trial l.jpg
A First (?) Clinical Cohort Trial

  • In 1774, a ship’s doctor took 12 sailors with scurvy and treated them daily, 2 at a time, with either…

    • …cider, elixir vitriol, vinegar, sea water, nutmeg, or oranges/lemons.

    • Compared with untreated patients.

  • Citrus fruit provided rapid relief.

Randomized controlled trials l.jpg
Randomized Controlled Trials

  • RCTs considered “gold standard.”

  • Subjects randomized to Experimental and Control conditions; then followed for outcomes.

  • Extraneous factors (confounders) are equally distributed across groups.

  • Subjects/investigators usually “blinded.”

  • May be ethical concerns about Control condition.

Systematic reviews meta analyses l.jpg
Systematic Reviews& Meta-Analyses

  • Gather highest quality evidence on a subject.

  • Compare and contrast studies to reach summary statements and conclusions.

  • Meta-analyses combine statistical evidence from multiple studies for greater precision.

  • Conclusions tend to be reliable and accurate.

  • Clinical guidelines often come from these studies.

A closer look at clinical research validity l.jpg

A Closer LookAtClinical Research Validity

What is validity l.jpg
What is Validity?

  • The degree to which the research represents the most accurate and bias-free treatment effects.

  • Internal Validity – refers to effects that are valid for the subjects in the particular study.

  • External Validity – refers to the relevance of a study for typical addiction treatment settings.

  • Validity is determined by looking at a study’s design and sources of potential bias.

What is bias l.jpg
What is Bias?

  • Bias is anything that influences conclusions about the groups under investigation and potentially distorts comparisons.

  • The question is not if there is bias, buthow much and whether it is sufficient to make the study invalid for a particular purpose.

Sources of potential bias 1 l.jpg

Publication Bias – Trials with “positive” results are more likely to be submitted and published more quickly.

Patient Selection – Who was excluded or included? Were groups equivalent?

Subject Management – Were groups treated exactly the same, except for the intervention under study?

Sources of Potential Bias - 1

Sources of potential bias 2 l.jpg
Sources of Potential Bias - 2

  • Randomization – Subjects should have an equal chance of assignment to each study condition.

  • Blinding (Masking) – Avoids preconceived ideas about treatment effects.

  • Placebo Effects – Mere expectation of benefit (or harm) may influence outcomes.

Sources of potential bias 3 l.jpg
Sources of Potential Bias - 3

  • Confounding Factors – Extraneous factors, known or unknown, may cause certain effects to occur or not to occur.

  • Compliance/Follow-up – Adherence to protocol and retention in study are critical.

  • Completeness of Data – Are all or only some subjects analyzed? (ITT vs Per Protocol)

Overcoming statistophobia l.jpg
Overcoming “Statistophobia”

  • When most people hear the wordstatistics they want to hide.

  • It is not necessary to be a biostatistician to understand and assess addiction research.

Example how large is the treatment benefit l.jpg
Example: How Large Is TheTreatment Benefit?

  • RCT comparing 8 mg/d buprenorphine (BUP) vs 80 mg/d methadone (M80).

  • Average illicit-opioid+ urinalyses at 26 weeks.

  • M80 produced significant 17% reduction.(55% - 38% = 17%; p < .01)

Ling et al. Arch Gen Psychiatry. 1996;53:401-407.

Example how many patients need to be treated l.jpg
Example: How Many Patients Need To Be Treated?

  • Treatment effect = 17% reduction with M80.

  • For every 100 patients treated with M80, rather than BUP, 17 more would benefit.

  • That is… …for every 6 patients (100 ÷ 17) treated with M80, one additional patient would remain illicit-opioid abstinent than if treated with BUP.

What the article didn t say l.jpg
What The Article Didn’t Say

In the 8 mg/d BUP vs 80 mg/d Methadone trial –

  • The patients selected were ideal; rather than typical.

  • There was a 95% chance that the “true” treatment effect was between 7% and 28% (vs 17% reported).

  • Not enough subjects enrolled (n=150) to accurately detect the relatively small treatment effect of 17%.

  • So many subjects dropped out that the validity should be questioned. (48% M80, 65% BUP gone at 26 wks)

Why does good research go bad l.jpg

Why DoesGood ResearchGo Bad?

Researcher writers face challenges l.jpg
Researcher-WritersFace Challenges

  • Insufficient funding – researchers may choose to do “something” when funds are not available to do it “right.”

  • Inadequate training in research & reporting.

  • Lack of technical support – many researchers do not have access to data processing and statistical support staff.

Researcher writers face challenges cont l.jpg
Researcher-WritersFace Challenges (Cont.)

  • “Publish or perish” pressures – influence poor quality in design and reporting.

  • Political agendas – promote certain lines of research looking for “desired” results.

  • Journal space limitations – foster inadequate descriptions of analyses and results.

  • “Too many cooks” – multiple authors and peer reviewers complicate the process.

Research writing run amok l.jpg
Research Writing Run Amok

When too many people are involved in a research article, it reminds us that…

“A camel is a horse that was designed by a committee.”

Dangers of data dredging l.jpg
Dangers of “Data Dredging”

  • Searching for “significant results” after the study is over and the findings were not as expected or desired (post hoc analysis).

  • Analyzing the same data in different ways until something “important” turns up.

  • Cranking-out several articles from a single small clinical study.

Laboratory vs clinical research l.jpg
Laboratory vs Clinical Research

  • Test tube & animal results do not necessarily apply to people.

  • “Bench work” can be invaluable for guiding clinical trials in humans.

  • But the ultimate test is in patients “at the bedside.”

©1980, David Sinclair, PhD - used by permission

Distortions in the mass media medical propaganda l.jpg
Distortions in the Mass Media“Medical Propaganda”

  • Medical journalists usually lack understanding of research methods and shortcomings.

  • News editors seek sensational health stories to grab the public’s attention.

  • Research organizations and publications should be more cautious in their reporting of research to the press.

  • Readers need to become more critical.

Putting addiction research into practice l.jpg

PuttingAddiction ResearchInto Practice

Judging everyday relevance l.jpg
Judging Everyday Relevance

  • Is the article of high quality and valid?

  • Does the study relate to your clinical needs?

  • Are the participants similar to your own patients?

  • Is the approach used in the study clearly explained and free of bias?

  • Are the results understandable and significant?

  • Do the conclusions make sense and suggest clear benefits for patient care?

Develop a healthy skepticism l.jpg
Develop A Healthy Skepticism

  • Learn to trust your educated instincts.

  • If it sounds like a duck, and walks like a duck, but smells like a fish… …something is probably a bit “fishy.”

What else can you do l.jpg
What Else Can You Do?

  • Only support/subscribe to publications offering high-quality, well-documented research evidence.

  • Lobby NIDA & other funding organizations to provide clinically-relevant research of the best quality in design and reporting.

  • Challenge “fishy” reports in the media or from other sources – write letters.

Bible for medical authors l.jpg
“Bible” for Medical Authors

  • AMA style book is the “gold standard” for medical writing.

  • Depicts correct formats for research papers, plus a wealth of other info.

  • Differs from other styles used in behavioral and social sciences.

E bm guidebook l.jpg
E-BM Guidebook

  • Users’ Guides to the Medical Literature(Textbook with CD-ROM)

  • Available at:



  • Presents all aspects of evidence-based medicine.

Ebam for practitioners57 l.jpg
“EBAM for Practitioners”

  • Available at… under “Addiction Resources” tab.

  • A more technical explanation for persons who need to critically assess research evidence.

Can addiction research be trusted usually if you learn what to look for l.jpg
Can Addiction Research Be Trusted?Usually!IF you learn what tolook for.

“We have met the enemy and he is us!” - Pogo (Walt Kelly)