Trends and Updates: Research on Research Integrity

1. Trends and Updates: Research on Research Integrity Nicholas H. Steneck, PhD University of Michigan 2009 ORI RRI Conference: THE CONFERENCE CENTER NIAGARA FALLS, NY May 15-17, 2009

2. Origins of RRI at ORI 1999 Chris Pascal / Larry Rhoades request $1M for RRI HHS (NIH source of funds) approves Mary Scheetz, ORI program officer now Cynthia Ricard <Cynthia.Ricard@hhs.gov> Contract to study of RRI literature Contacted researchers for advice July 1999, contact N. Steneck regarding four projects: November advisory committee meeting National RRI Conference RRI program RRI Literature review

3. ORI justification (Pascal 2000) Since its establishment in 1992, the Office of Research Integrity (ORI) has conducted several studies on research misconduct and research integrity in an attempt to develop a knowledge base on important issues, such as the impact of misconduct allegations on exonerated scientists, the experience of whistleblowers in the aftermath of making allegations, the research guidelines adopted by medical schools, and the causes of research misconduct. Over time, it became apparent to ORI that a more comprehensive, coordinated effort in collaboration with extramural research scholars was needed to develop the science base on research integrity issues. This recognition led to development of this "Research Conference on Research Integrity" and the related "Research on Research Integrity" program announcement jointly issued by ORI and the National Institute of Neurological Disorders and Stroke (RFA: NS-01-008).

4. Planning Committee Ruth Fischbach, Ph.D., M.P.E., National Institutes of Health Mark Frankel, Ph.D., American Association for the Advancement of Science Paul Friedman, M.D., UC, San Diego Edward Hackett, Ph.D., Arizona State University Stanley Korenman, University of California, San Diego Francis Macrina, Ph.D., Virginia Commonwealth University Barry Markovsky, Ph.D. University of Iowa Kathleen Montgomery, Ph.D., UC, Riverside John Perhonis, Ph.D., National Science Foundation Drummond Rennie, M.D., F.R.C.P., F.A.C.P., UC, San Francisco Allan Shipp, M.H.A., Association of American Medical Colleges Peter Yeager, Ph.D., Boston University

5. Major events RRI (& RCR) Conferences November 2000, Bethesda MD (Marriott) November 2002, Potomac MD (Bolger Conference Center) November 2004, San Diego CA (Paradise Point Resort) UC San Diego School of Medicine December 2006, Tampa FL (Safety Harbor Resort) University of South Florida College of Medicine April 2007, St. Louis MO (Washington University) Washington University in St. Louis School of Medicine May 2009, Niagara Falls NY (Conference Center) Roswell Park Cancer Institute RRI Program, December 15, 2000, deadline, 1st round ORI does not have granting authority Early 2000 explored administration options

6. May NIH presentation, IC Directors

7. Closing slides NINDS agreed to be 1st co-sponsor

8. RRI Program History 2001, RO1, two-year, $100,000/year, direct cost 2004, three-year, $250,000/year direct cost Most proposals received Lowest percentage of proposal funded 2005, two-year, $175,00

9. Funding rate average = 18%

10. Source of funds / totals

11. ORI / Other funding (60% / 40%)

12. Primary focus, all proposals

13. Areas: Proposed / Funded

14. RRI apart from ORI NIH funds training and research ethics, not RRI Ethics = bioethics, not research ethics/integrity RCR funded through training grants RRI can be considered but few applications NSF broad array of programs STS �> EVS, HPS, SSS, and SPS Research ethics training (REU, IGERT, etc.) Peer Review Congresses Research on peer review and biomedical publication Peer generated and supported; no funding

15. NSF vs. NIH, RCR education ORI/NIH RCR Enhancement Initiative (2002) Budget National planning and coordination effort ($1.1m), Institutional Support Program ($11.5m), and Instructional Enhancement Program ($6.1m). Approved by NIH Director in 2004, never implemented National Science Foundation, research ethics 2001-2009, 328 �research ethics� awards $270,616,429.00 total funding Most to support instructional development No special attention to or coordination of RRI

16. Global interest in RRI? Some interest in Europe: Early pioneer, Croatia, now joined by other countries European Science Foundation has RRI committee Just getting organized UK RIO has considered, but not yet undertaken No support from European Commission China is supporting some RRI Developing an instrument to assess integrity Has investigated plagiarism Elsewhere, support for RRI is scattered

17. Observations: Professional development RRI has become a recognized field of research Established researchers anxious to pursue Peers have reviewed and endorsed proposals Funding agencies provide some support Literature and professional community has emerged Influences on professional development of RRI FUNDING ~ interest grows/declines with support Funding mechanism ~ RO1 vs. R22 External events Human subjects concerns Conflict of interest

18. Observations: Tensions Researcher-funder conflict Researchers interested in broad (basic) studies Funders interested in IC-specific (applied) topics Research-policymaker conflict Researchers interested in broader (basic) studies Policy makers want studies that are relevant to policy making Professional apathy / hostility RRI is �important� but low priority � RRI is conspiracy to undermine science

19. RRI track record (ORI & other) Misconduct �worldview� 1970s Serious misconduct is rare Self-regulation keeps in check Misconduct is difficult to detect Misconduct cannot be prevented Apart from misconduct, standards are high RRI has cast doubt on all five assumptions FFP ~ 0.1�>1.0%; QRP ~ 5% �> 50% Self-regulation has serious shortcomings Better self-regulation would prevent FFP / QRP Research integrity can be improved (considerably)

20. Challenge ~ demonstrating relevance 2006 Study (NEJM) Early detection study CT scan vs. X-ray With CT scan, 80% of cases  could be cured March 28, 2008 NY Times* Did not report study partly funded by tobacco company Vector Group (Liggett) �>$$$�> Foundation for Lung Cancer: Foundation listed in NEJM article, along with 31 additional funding sources Did not report held patents on CT-scan-related technology Impact/profit: 48M former smokers / 40M current smokers (US) Others involved as members of Foundation for � Early Detection Dean & Vice-Chair Board of Overseers, Weill Cornell Medical College Controversial cases get attention

21. Comparison to careful RRI study Gorman, Evaluation & Program Planning (2007) study of National Registry of Effective & Promising Programs for drug and alcohol use prevention 78% of published evaluations had developer as an author Developers had a significant financial interest in outcomes Evaluation in one program had serious methodological flaws Texas spent $35M on one questioned program in 2007 Why is this story not national headline news? COI �> wasted funds and probably loss of life Conclusion: future of RRI depends on relevance Need more studies of impacts (economic / human) RRI community must become proactive