Trends and Updates : Research on Research Integrity

1. Trends and Updates: Research on Research Integrity Nicholas H. Steneck, PhD University of Michigan 2009 ORI RRI Conference: THE CONFERENCE CENTER NIAGARA FALLS, NY May 15-17, 2009 RRI

2. Origins of RRI at ORI • 1999 • Chris Pascal / Larry Rhoades request $1M for RRI • HHS (NIH source of funds) approves • Mary Scheetz, ORI program officer • now Cynthia Ricard <Cynthia.Ricard@hhs.gov> • Contract to study of RRI literature • Contacted researchers for advice • July 1999, contact N. Steneck regarding four projects: • November advisory committee meeting • National RRI Conference • RRI program • RRI Literature review

3. ORI justification (Pascal 2000) • Since its establishment in 1992, the Office of Research Integrity (ORI) has conducted several studies on research misconduct and research integrity in an attempt to develop a knowledge base on important issues, such as the impact of misconduct allegations on exonerated scientists, the experience of whistleblowers in the aftermath of making allegations, the research guidelines adopted by medical schools, and the causes of research misconduct. • Over time, it became apparent to ORI that a more comprehensive, coordinated effort in collaboration with extramural research scholars was needed to develop the science base on research integrity issues. This recognition led to development of this "Research Conference on Research Integrity" and the related "Research on Research Integrity" program announcement jointly issued by ORI and the National Institute of Neurological Disorders and Stroke (RFA: NS-01-008).

4. Planning Committee • Ruth Fischbach, Ph.D., M.P.E., National Institutes of Health • Mark Frankel, Ph.D., American Association for the Advancement of Science • Paul Friedman, M.D., UC, San Diego • Edward Hackett, Ph.D., Arizona State University • Stanley Korenman, University of California, San Diego • Francis Macrina, Ph.D., Virginia Commonwealth University • Barry Markovsky, Ph.D. University of Iowa • Kathleen Montgomery, Ph.D., UC, Riverside • John Perhonis, Ph.D., National Science Foundation • Drummond Rennie, M.D., F.R.C.P., F.A.C.P., UC, San Francisco • Allan Shipp, M.H.A., Association of American Medical Colleges • Peter Yeager, Ph.D., Boston University

5. Major events • RRI (& RCR) Conferences • November 2000, Bethesda MD (Marriott) • November 2002, Potomac MD (Bolger Conference Center) • November 2004, San Diego CA (Paradise Point Resort) • UC San Diego School of Medicine • December 2006, Tampa FL (Safety Harbor Resort) • University of South Florida College of Medicine • April 2007, St. Louis MO (Washington University) • Washington University in St. Louis School of Medicine • May 2009, Niagara Falls NY (Conference Center) • Roswell Park Cancer Institute • RRI Program, December 15, 2000, deadline, 1st round • ORI does not have granting authority • Early 2000 explored administration options

6. May NIH presentation, IC Directors

7. Closing slides • NINDS agreed to be 1st co-sponsor

8. RRI Program History • 2001, RO1, two-year, $100,000/year, direct cost • 2004, three-year, $250,000/year direct cost • Most proposals received • Lowest percentage of proposal funded • 2005, two-year, $175,00 2008, R22

9. Funding rate average = 18% 31% 28% 16% 32% 19% 10% 12% 13%

10. Source of funds / totals NIH provides over 50%

11. ORI / Other funding (60% / 40%)

12. Primary focus, all proposals

13. Areas: Proposed / Funded

14. RRI apart from ORI • NIH funds training and research ethics, not RRI • Ethics = bioethics, not research ethics/integrity • RCR funded through training grants • RRI can be considered but few applications • NSF broad array of programs • STS –> EVS, HPS, SSS, and SPS • Research ethics training (REU, IGERT, etc.) • Peer Review Congresses • Research on peer review and biomedical publication • Peer generated and supported; no funding

15. NSF vs. NIH, RCR education • ORI/NIH RCR Enhancement Initiative (2002) • Budget • National planning and coordination effort ($1.1m), • Institutional Support Program ($11.5m), and • Instructional Enhancement Program ($6.1m). • Approved by NIH Director in 2004, never implemented • National Science Foundation, research ethics • 2001-2009, 328 “research ethics” awards • $270,616,429.00 total funding • Most to support instructional development • No special attention to or coordination of RRI * ORI has provided some RCR support

16. Global interest in RRI? • Some interest in Europe: • Early pioneer, Croatia, now joined by other countries • European Science Foundation has RRI committee • Just getting organized • UK RIO has considered, but not yet undertaken • No support from European Commission • China is supporting some RRI • Developing an instrument to assess integrity • Has investigated plagiarism • Elsewhere, support for RRI is scattered

17. Observations: Professional development • RRI has become a recognized field of research • Established researchers anxious to pursue • Peers have reviewed and endorsed proposals • Funding agencies provide some support • Literature and professional community has emerged • Influences on professional development of RRI • FUNDING ~ interest grows/declines with support • Funding mechanism ~ RO1 vs. R22 • External events • Human subjects concerns • Conflict of interest

18. Observations: Tensions • Researcher-funder conflict • Researchers interested in broad (basic) studies • Funders interested in IC-specific (applied) topics • Research-policymaker conflict • Researchers interested in broader (basic) studies • Policy makers want studies that are relevant to policy making • Professional apathy / hostility • RRI is “important” but low priority • … • RRI is conspiracy to undermine science

19. RRI track record (ORI & other) • Misconduct “worldview” 1970s • Serious misconduct is rare • Self-regulation keeps in check • Misconduct is difficult to detect • Misconduct cannot be prevented • Apart from misconduct, standards are high • RRI has cast doubt on all five assumptions • FFP ~ 0.1–>1.0%; QRP ~ 5% –> 50% • Self-regulation has serious shortcomings • Better self-regulation would prevent FFP / QRP • Research integrity can be improved (considerably)

20. Challenge ~ demonstrating relevance • 2006 Study (NEJM) • Early detection study • CT scan vs. X-ray • With CT scan, 80% of cases could be cured • March 28, 2008 NY Times* • Did not report study partly funded by tobacco company • Vector Group (Liggett) —>$$$—> Foundation for Lung Cancer: • Foundation listed in NEJM article, along with 31 additional funding sources • Did not report held patents on CT-scan-related technology • Impact/profit: 48M former smokers / 40M current smokers (US) • Others involved as members of Foundation for … Early Detection • Dean & Vice-Chair Board of Overseers, Weill Cornell Medical College • Controversial cases get attention *see also Paul Goldberg, Cancer Letter Inc. (2008)

21. Comparison to careful RRI study • Gorman, Evaluation & Program Planning (2007) • study of National Registry of Effective & Promising Programs for drug and alcohol use prevention • 78% of published evaluations had developer as an author • Developers had a significant financial interest in outcomes • Evaluation in one program had serious methodological flaws • Texas spent $35M on one questioned program in 2007 • Why is this story not national headline news? • COI –> wasted funds and probably loss of life • Conclusion: future of RRI depends on relevance • Need more studies of impacts (economic / human) • RRI community must become proactive

22. Thanks & on with the conference nsteneck@umich.edu