is a discussion on are observational studies any good any good n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Is a “Discussion” on “Are Observational Studies Any Good” Any Good PowerPoint Presentation
Download Presentation
Is a “Discussion” on “Are Observational Studies Any Good” Any Good

Loading in 2 Seconds...

play fullscreen
1 / 9

Is a “Discussion” on “Are Observational Studies Any Good” Any Good - PowerPoint PPT Presentation


  • 109 Views
  • Uploaded on

Is a “Discussion” on “Are Observational Studies Any Good” Any Good . Don Hoover May 2, 2014. Everyone Already Knows Observational Studies Are Not Perfect … Right?. But who thinks the real type 1 error is 0.55 when the nominal is 0.05? The real coverage of a 95% confidence interval is 25%?

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Is a “Discussion” on “Are Observational Studies Any Good” Any Good' - aelan


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
is a discussion on are observational studies any good any good

Is a “Discussion” on “Are Observational Studies Any Good” Any Good

Don Hoover

May 2, 2014

everyone already knows observational studies are not perfect right
Everyone Already Knows Observational Studies Are Not Perfect … Right?
  • But who thinks
    • the real type 1 error is 0.55when the nominal is 0.05?
    • The real coverage of a 95% confidence interval is 25%?
    • That’s what David Madigan and the OMAP team find
  • This obviously makes such results meaningless
  • But how many papers with these properties are being (and will continue to be) published ???
but does david s talk really apply to all observational studies
But Does David’s Talk Really Apply to ALL Observational Studies?
  • They Only Look at Observational Studies of Drug Use and Adverse Consequences
  • There’s other kinds of Observational Studies … on HIV, Epi, Health Behaviors, Nutrition, etc.
    • No one has looked at these types of studies
      • These other studies must have similar problems
      • Maybe ata smaller magnitude
    • But there are no “negative controls” for these settings … so no one can check this
the approach h ere i s creative and innovative
The Approach here is Creative and Innovative
  • Finding Negative Control Exposures or Outcomes to derive empirical distribution of the test statistic somewhat equalizes assumptions and unmeasured confounding
  • With a given Drug Use as the exposure and agiven Disease the outcome, such negative controls are readily available in many data sets
  • So maybe something like it should be used when possible
  • But now some questions ……
q1 w hy were negative control drugs more associated with outcomes than by chance
Q1- Why were Negative Control Drugs More Associated With Outcomes than by Chance?
  • People put on Any Drug are Sicker?
  • Those receiving a negative (control) drug are more likely to receive some other positive drug?
  • Those apriori more likely to have a given disease outcome are steered to the negative drugs?
  • Incorrect statistical models used?
q2 i s this approach practical
Q2- Is this Approach Practical?
  • A lot more work to fit many models than the standard approach which only fits one
    • More money as well - A grant application using it would be less likely to get funded
    • More work also means more chance for error in implementation
q3 how does one interpret a positive drug with empirical p 0 05
Q3 – How does one interpret a positive drug with empirical P < 0.05?

Calibrated Normal Scores of Negative Controls

Positive Drug with empirical P < 0.05

The use of an “empirical” approach acknowledges we do not know what is going on so maybe the P < 0.05 is from model artifact not causal

q4 what is done with negative drugs more extreme than the positive o ne
Q4 – What is done with “Negative Drugs” more extreme than the Positive One

Calibrated Normal Scores of Negative Controls

Positive Drug with P < 0.05

Should these Negative Controls all be Examined for Causal Association as their Signal is larger than the positive drug?

q5 how to handle heterogeneity in denominator of calibration statistic
Q5 - How to handle Heterogeneity in Denominator of Calibration Statistic

From Schumie … Madigan Stat Med 2014 33; 209-18

Variance may introduce Apples to Oranges comparisons especially if

although such does not appear to be the case in the examples David used