Responsible Conduct of Research Lecture 7 Peer Review Responsible Authorship and Publication

1. Responsible Conduct of Research Lecture 7Peer ReviewResponsible Authorship and Publication Elizabeth Langdon-Gray Director of Research Policy and Education Ken Carson Assistant Provost for Research Policy October 19, 2012

2. Peer Review Cornerstone of Review of scientific Proposals and Publications

3. Peer Review – ProposalsBackground • NIH peer review mandated by statute: "Scientific Peer Review of Research Grant Applications and Research and Development Contract Projects" (42 CFR Part 52h) “NIH policy is intended to ensure that grant applications submitted to the NIH are evaluated on the basis of a process that is fair, equitable, timely, and free of bias.” (http://grants.nih.gov/grants/peer_review_process.htm) • NSF “Merit Review” (http://www.nsf.gov/bfa/dias/policy/meritreview/) • Study sections consider merits of proposal + public health needs + goals of the agency • Peer review of proposals vs peer review of articles = evaluation of “competitive position” vs evaluation of “intrinsic soundness” Lenard, J., “Two facets of peer review and the proper role of study sections”, Accountability in Research, 13:277-283, 2006

4. Potential Impact = Responsibility • Make or break reputations • Influence public policy • Ensure the success or failure of research programs • Affect public health initiatives; environmental and safety regulations RESPONSIBILITY to be: Timely Free from bias Constructive Thorough Respectful of the need for confidentiality

5. Peer Review – ProposalsCriticisms • Enormous administrative burden • Stifling of innovation • Complaint that reviewers tend to underscore that which they do not understand = more conservative applications • Exacerbated by more competitive funding environment • Increasingly collaborative research landscape not always matched by expertise of reviewers • Bias, conflicts of interest, plagiarism: obligation of reviewers to maintain integrity of process (those best able to judge are often in direct competition) National Institutes of Health 2007-2008 Peer Review Self-Study – Final Draft, 2-29-2008, http://enhancing-peer-review.nih.gov/meetings/NIHPeerReviewReportFINALDRAFT.pdf

6. Peer Review – PublicationsHistory • Pre-peer review, practice of keeping results secret so others could not take credit – undermines common purpose of scientific endeavor • Peer review of journal articles uncommon until the mid-20th century. • Tied to importance of publications: • Allows for prioritization of articles (elite journals seen as indicator of quality of science) • Publication major factor in hiring, promotion, reputation, recognition Casadevall, A. and Ferric, Fang C., “Is Peer Review Censorship?”, INFECTION AND IMMUNITY, April, 2009, p.1273-1274

7. Peer Review – PublicationsBackground The review process is widely used in: • Journal articles • Multi-authored books • Congress proceedings • Book series with invited articles • Scientific meetings (poster/oral presentations) See: Johannes Schramm, Some Thoughts on the Review Process, World Neurosurgery, Volume 76, Issues 1-2, July-August 2011, Pages 43-44, ISSN 1878-8750, 10.1016/j.wneu.2011.05.038. • Publication and authorship practices vary by field

8. Peer Review – PublicationsObligations Obligation to Review • Contribution to science and science as a collaborative endeavor Obligations of the Reviewer Broad effects of review – on progress of science, careers and reputations of authors, influence on public policy • Timeliness • Objectivity • Sensitivity to potential/perceived conflicts of interest • Preservation of confidentiality (e.g. insider trading) Obligations of the Journal • Pick the right reviewers • Provide clear guidance • Provide recognition (most reviews uncompensated = need for other forms of recognition) Groves, T. “Quality and value: How can we get the best out of peer review? A recipe for good peer review”, Nature (2006), doi:10.1038/nature04995

9. Peer Review – PublicationsReview Criteria From The British Medical Journal’s “Guidance for Peer Reviewers” (http://resources.bmj.com/bmj/reviewers/peer-reviewers-guidance) For all articles:Is the article important? Will it help our readers to make better decisions and, if so, how? Will the article add enough to existing knowledge? Does the article read well and make sense? Does it have a clear message? For research articles please comment on: • Originality — does the work add enough to what is already in the published literature? If so, what does it add? Please cite relevant references to support your comments on originality. • Importance of the work to general readers — does this work matter to clinicians, researchers, policymakers, educators, or patients? Will it help our readers to make better decisions and, if so, how?  Is a general medical journal the right place for it? • Scientific reliability • Research question — clearly defined and appropriately answered? • Overall design of study — appropriate and adequate to answer the research question? • Participants — adequately described, their conditions defined, inclusion and exclusion criteria described? How representative were they of patients whom this evidence might affect? • Methods — adequately described? Main outcome measure clear? For randomised trials, systematic reviews, observational studies, health economics studies - reported in line with the appropriate reporting statement or checklist (see below)? Was the study ethical (this may go beyond simply whether the study was approved by an ethics committee or IRB)? • Results — answer the research question? Credible? Well presented? • Interpretation and conclusions — warranted by and sufficiently derived from/focused on the data? Discussed in the light of previous evidence? Message clear? • References — up to date and relevant? Any glaring omissions? • Abstract/summary/key messages/what this paper adds — reflect accurately what the paper says? • Documents in the supplemental files eg checklists for these statements CONSORT, PRISMA, MOOSE, STROBE; the BMJ health economics checklist; and the protocol for an RCT - do these contain information that should be better reported in the manuscript, or raise questions about the work?

10. Peer Review – PublicationsCriticisms and Shortcomings …“neither Newton or Darwin had to submit to the indignity of peer review!”… Casadevall, A. and Ferric, Fang C., “Is Peer Review Censorship?”, INFECTION AND IMMUNITY, April, 2009, p.1273-1274 • Claims of “ethical transgressions” e.g. • Breach of confidentiality • Theft of ideas • Insider trading • Personal attacks • Bias / conflicts of interest Resnik, D.B., Gutierrez-Ford, C., Peddada, S., “Perceptions of Ethical Problems with Scientific Journal Peer Review: An Exploratory Study”, Sci Eng Ethics, 2008 September; 14(3): 305-310. • Failure to detect errors and fraud Peer review is “an inherently conservative process … [that] … encourages the emergence of self-serving cliques of reviewers” - UK Parliamentary Office of Science and Technology …“it’s a lousy system, but it’s the best one we have”…

11. What could go wrong? “In incidents involving four scientists … journal editors say authors got to critique their own papers by suggesting reviewers with contact emails that actually went to themselves.” “Blame lies with those journals … that allow authors to nominate their own reviewers and don’t check credentials and contacts.”

12. Peer Review – Publications Open Peer Review – A Panacea? • Reviewers’ names and affiliations made available to authors • Reviewers asked to declare any potential conflicts of interest • Should authors (and readers) see reviewers comments? • Should readers themselves be able to comment? • New models being tested at journals See Nature’s archive of articles on the peer-review debate: http://www.nature.com/nature/peerreview/debate/index.html

13. Peer Review – PublicationsThe Role of Technology • Open peer review made easier by internet • Allows for dialog between authors, reviewers, editors and readers; post-publication commentary Open peer review cousin to: • Open access • Databases allow for sharing of raw data • Open access to journal articles more prevalent • NIH policy “requires scientists to submit final peer-reviewed journal manuscripts that arise from NIH funds to the digital archive PubMed Centralupon acceptance for publication.” http://publicaccess.nih.gov/ • Open access at HU promoted and facilitated by Office for Scholarly Communication (http://osc.hul.harvard.edu/)

14. Open Access: the end of Peer Review? • PLoS One • Limited Peer Review (Has the science in this paper been done well enough to warrant it being entered into the scientific literature as a whole?”) + • Post-publication open comment

15. Authorship Allocating credit Assigning responsibility Avoiding conflict See Harvard Workshop for resources: http://projects.iq.harvard.edu/attribution_workshop

16. Authorship • List of authors = apportionment of credit (and accountability) • Credit in publications critical to career advancement, reputation, recognition • Authorship conventions (first, last, second etc.) vary by discipline and research group • Problems: ghost-writing, honorary authors, allocation of credit and responsibility • Proliferation of issues with increased collaboration/multi-site studies

17. Authorship as Code for Contribution "If scientists want to convey this information by the way their names are ordered, the method is similar to sending smoke signals, in code, on a dark, windy night." --Drummond Rennie, Deputy Editor, JAMA

18. Putting Authorship Issues in ContextCassandra Extavour, OEB, Harvard

19. Putting Authorship Issues in ContextCassandra Extavour, OEB, Harvard

20. Putting Authorship Issues in ContextCassandra Extavour, OEB, Harvard

21. AuthorshipCase Study 1 • 2002 J. Hendrik Schön – Physicist at Bell Labs – author or co-author of more than 90 scientific papers • Investigative panel concluded scientific misconduct in 17 of the 24 allegations stemming from examination of 25 papers “Except for the provision of starting materials by others, all device fabrication, physical measurement and data processing in the work in question were carried out (with minor exceptions) by HendrikSchön alone, with no participation by any coauthor or other colleague. None of the most significant physical results was witnessed by any coauthor or other colleague.” “In addition to addressing the question of scientific misconduct, the Committee also addressed the question whether the coauthors of HendrikSchön exercised appropriate professional responsibility in ensuring the validity of data and physical claims in the papers in question. By virtue of their coauthorship, they implicitly endorse the validity of the work. There is no implication here of scientific misconduct; the issue is one of professional responsibility.” September, 2002 Report of the Investigative Committee on the Possibility of Scientific Misconduct in the Work of HendrikSchön and Coauthors: http://publish.aps.org/reports/lucentrep.pdf

22. AuthorshipCase Study 2 • Dr. Gerald Schatten – claimed senior authorship on one of two papers retracted by Woo Suk Hwang in South Korea claiming generation of embryonic stem cells by somatic cell nuclear transfer. • But when work was found to be fraudulent, Schatten “attempted to distance himself from it” • Admitted to have helped write the paper, but not to have participated in the experiments, and not to have interacted with other collaborators/authors • No evidence of misconduct, but Schatten found guilty of “scientific misbehavior” by University of Pittsburgh investigation. • Authorship was acceptable, based on his work on paper, but not senior authorship and co-corresponding author status • Highlights linked notions of credit and responsibility. “Dr. Schatten’s listing as the last author not only conferred considerable credibility to the paper within the international scientific community, but directly benefitted Dr. Schatten in numerous ways, including enhancement of his scientific reputation, improved opportunities for additional research funding, enhanced positioning for pending patent applications, and considerable personal financial benefit. “University of Pittsburgh investigators also examined a Nature paper coauthored by Hwang and Schatten that described the cloning of a dog. The so-called “Snuppy paper” was not fraudulent, but the investigating committee questioned Schatten’s assumption of coauthorship since his only contribution was to suggest “that a professional photographer be engaged so that Snuppy would appear with greater visual appeal.” Strange, K., Authorship: why not just toss a coin? American Journal of Physiology – Cell Physiology 295: C567-C575, 208

23. AuthorshipICMJE Standards • International Committee of Medical Journal Editors enacted new guidelines at its May, 2000 meeting in Copenhagen: Uniform Requirements for Manuscripts Submitted to Biomedical Journals • “Authorship credit should be based on • Substantial contributions to conception and design, acquisition of data, or analysis and interpretation of data; • Drafting the article or revising it critically for important intellectual content; and • Final approval of the version to be published. Authors should meet conditions 1, 2 and 3.” www.icmje.org/ethical_1author.html Some journals require that authors indicate specific contributions

24. AuthorshipAvoiding Problems / Resolving Disagreements Some suggestions… • Discuss authorship and allocation of credit at the beginning of the project – in writing, if possible • Continue discussion throughout project • Avoid superficial collaborations • Communicate • Seek arbitration / guidance from third party / institutional committee Strange, K., Authorship: why not just toss a coin? American Journal of Physiology – Cell Physiology 295: C567-C575, 208

25. Framework for Awkward Conversations Criteria for Authorship: Stephen M. Kosslyn • Idea (250 points) • Design (100 points) • Implementation (100 points) • Conducting the experiment (100 points) • Data analysis (200 points) • Writing (250 points)

26. One Taxonomy of Authorship OrderCassandra Extavour, OEB, Harvard University • First Author: Carried out bulk of experimental / analytical work; usually grad student or postdoc • Second Author: Likely did experimental or analytical work; usually a student • Other Middle Author: could have contributed in any way, how is unclear; less experimental / analytical than 1st, 2nd • Last Author: Senior Author = PI; Major contributor of intellectual content and funding