Peer review – how to deal with with reviewers ? Ana Marušić

1. Workshop: EditorialProcess Peerreview – how to dealwithwithreviewers? Ana Marušić Editor inchief, Journal of Global Health Editor emerita, Croatian Medical Journal University of Split School of Medicine, Split, Croatia

2. Peerreview How to findgoodreviwersanddealwiththem? How to obtainbetterperformancefromreviewers? How to dealwithcompletelydifferentviewsaboutthe same paper?

3. Is training of peer reviewers effective? Ann Emerg Med. 2002 Sep;40(3):323-8. Effect of structured workshop training on subsequent performance of journal peer reviewers. Callaham ML, Schriger DL. CONCLUSIONS: • Among invited peer reviewers, voluntary attendance at a highly structured and interactive workshop was low and did not improve the quality of subsequent reviews, contrary to the predictions of attendees. Efforts to aggressively recruit average reviewers to a second workshop were time consuming, had low success rates, and showed a similar lack of effect on ratings, despite improvement in scores on a test instrument. Workshop teaching formats, although traditional, are of unproven efficacy.

4. Is training of peer reviewers effective? PLoS Med. 2007 Jan;4(1):e40. The relationship of previous training and experience of journal peer reviewers to subsequent review quality. Callaham ML, Tercier J. CONCLUSIONS: • Our study confirms that there are no easily identifiable types of formal training or experience that predict reviewer performance. Skill in scientific peer review may be as ill defined and hard to impart as is "common sense." Without a better understanding of those skills, it seems unlikely journals and editors will be successful in systematically improving their selection of reviewers. This inability to predict performance makes it imperative that all but the smallest journals implement routine review ratings systems to routinely monitor the quality of their reviews (and thus the quality of the science they publish).

5. Is training of peer reviewers effective? Ann Emerg Med. 2011 Feb;57(2):141-8. Epub 2010 Nov 12. Longitudinal trends in the performance of scientific peer reviewers. Callaham M, McCulloch C. CONCLUSIONS: • This study, one of few tracking expert performance longitudinally, demonstrates that most journal peer reviewers received lower quality scores for article assessment over the years. This could be due to deteriorating performance (caused by either cognitive changes or competing priorities) or, to a partial degree, escalating expectations; other explanations were ruled out. This makes monitoring reviewer quality even more crucial to maintain the mission of scientific journals.

6. Is training of peer reviewers effective? BMC Med Educ. 2012 Aug 28;12(1):83. [Epub ahead of print] Does mentoring new peer reviewers improve review quality? A randomized trial. Callaham M, Green S, Houry D. CONCLUSIONS: • A structured training intervention of pairing newly recruited medical journal peer reviewers with senior reviewer mentors did not improve the quality of their subsequent reviews.

7. Is training of peer reviewers effective? J ClinEpidemiol. 2012 Mar;65(3):247-52. Epub 2011 Nov 8. Medical journal editorslackedfamiliaritywithscientificpublicationissuesdespitetrainingandregularexposure. Wong VS, Callaham ML. CONCLUSIONS: • Our study presents a current look at editors of major clinical medical journals. Most editors reported training in medical editing topics, saw ethical issues regularly, and were aware of scientific publication organizations, but their knowledge of four common and well-disseminated publication ethics topics appears poor.

8. WorldAssociation of MedicalEditors EUROPEAN ASSOCIATION OF SCIENCE EDITORS Integrity of peerreview www.councilscienceeditors.org www.ease.org.uk www.icmje.org www.wame.org www.publicationethics.org.uk

9. Authors and Reviewers • Manuscripts must be reviewed with due respect for authors’ confidentiality. In submitting their manuscripts for review, authors entrust editors with the results of their scientific work and creative effort, on which their reputation and career may depend. Authors’ rights may be violated by disclosure of the confidential details during review of their manuscript. Reviewers also have rights to confidentiality, which must be respected by the editor. Confidentiality may have to be breached if dishonesty or fraud is alleged but otherwise must be honored. • Editors must not disclose information about manuscripts (including their receipt, content, status in the reviewing process, criticism by reviewers, or ultimate fate) to anyone other than the authors and reviewers. This includes requests to use the materials for legal proceedings.

10. Authors and Reviewers • Editors must make clear to their reviewers that manuscripts sent for review are privileged communications and are the private property of the authors. Therefore, reviewers and members of the editorial staff must respect the authors’ rights by not publicly discussing the authors’ work or appropriating their ideas before the manuscript is published. Reviewers must not be allowed to make copies of the manuscript for their files and must be prohibited from sharing it with others, except with the editor’s permission. Reviewers should return or destroy copies of manuscripts after submitting reviews. Editors should not keep copies of rejected manuscripts. • Reviewer comments should not be published or otherwise publicized without permission of the reviewer, author, and editor.

11. Guidelines for reviewers? • Goodresource: BMJ Resources for reviewers: http://www.bmj.com/about-bmj/resources-reviewers

12. http://www.cmj.hr/2005/46/2/15849858.htm

13. Einstein’s answer to PhysicalReviewin 1936

14. Obrigada! ana.marusic@mefst.hr http://3.bp.blogspot.com/-oLwi2iC6-pk/UAbBLqUQR_I/AAAAAAAABS0/BmcmHauPTtE/s1600/poor-review-and-peer-review.jpg