First aid for the peer review process l.jpg
This presentation is the property of its rightful owner.
Sponsored Links
1 / 55

FIRST AID FOR THE PEER REVIEW PROCESS PowerPoint PPT Presentation


  • 62 Views
  • Uploaded on
  • Presentation posted in: General

FIRST AID FOR THE PEER REVIEW PROCESS. Giuseppe Biondi Zoccai Division of Cardiology , University of Turin , Turin , Italy Meta-analysis and Evidence-based medicine Training in Cardiology (METCARDIO), Ospedaletti , Italy.

Download Presentation

FIRST AID FOR THE PEER REVIEW PROCESS

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


First aid for the peer review process l.jpg

FIRST AID FOR THE PEER REVIEW PROCESS

Giuseppe Biondi Zoccai

DivisionofCardiology, UniversityofTurin, Turin, Italy

Meta-analysis and Evidence-based medicine Training in Cardiology(METCARDIO), Ospedaletti, Italy

How to develop effective abstracts and manuscripts in interventional cardiology

GISE 2009 – 22 October 2009 – 13.25-13.50


Learning goals l.jpg

LEARNING GOALS

  • What is peer review, and why have we to survive it through?

  • What should you do and not do when actually peer reviewing other colleagues’ works?

  • What should you do and not do when surviving through peer review?

  • Case studies


Why am i giving you this lecture l.jpg

WHY AM I GIVING YOU THIS LECTURE?

Peer reviews I have completed since 2003

expected


Learning goals4 l.jpg

LEARNING GOALS

  • What is peer review, and why have we to survive it through?

  • What should you do and not do when actually peer reviewing other colleagues’ works?

  • What should you do and not do when surviving through peer review?

  • Case studies


Definition l.jpg

DEFINITION

Peer

vb intr. 1. to look intently with or as if with difficulty. 2. to appear partially or dimly.

n. 1. a person who is an equal in social standing, rank, age, etc.

Collins Dictionary of the English Language,

London & Glasgow: Collins, 1979


Origins of peer review l.jpg

ORIGINS OF PEER REVIEW

  • Peer review was born in the 17th century in the UK, likely thanks to Henry Oldenburg, the founder of Philosophical Abstractions (1665)

  • He originally introduced the practice of soliciting opionions on manuscripts from more knowledgeable external colleagues


Notable exceptions l.jpg

NOTABLE EXCEPTIONS

  • Not all manuscripts undergo thorough peer review. Notable examples?

  • Albert Einstein’s “AnnusMirabilis” papers, published in 1905 in AnnalenderPhysikbyMax Planck(fatherof quantum theory and Nobel prizewinner), whoread the papers and decidedaltogethertopublishthem

  • more humbly…Biondi-Zoccaiet al, Int J Cardiol2005;100:119-23


What s wrong with peer review l.jpg

WHAT’S WRONG WITH PEER REVIEW?

  • Unreliable

  • Unfair

  • Fails to truly validate or authenticate

  • Unstandardized

  • Idiosyncratic

  • Open to every sort of bias


What s even worse l.jpg

WHAT’S EVEN WORSE?

  • Stifles innovation

  • Perpetuates the status quo

  • Rewards the prominent but punishes the weak

  • Unnecessarily delays dissemination

  • Very expensive

  • Insufficiently tested


Yes but l.jpg

YES, BUT…

  • It has been said that democracy is the worst form of government except all those other forms that have been tried from time to time – W. Churchill

  • The sameappliestopeerreview


What are its pros l.jpg

WHAT ARE ITS PROS?

  • Filters bad from good, and better from less good, as well as possibly identifying misleading and false research

  • Protects patients

  • Guides authors to improve the quality of their article, and improve their whole research approach

  • Authenticates work, assuring quality

  • Improves readability

  • Broadens participation and dialogue


Paraphrasing groucho l.jpg

PARAPHRASING GROUCHO

I would never enter into a club that would accept me as a member…

Groucho Marx, 1980-1977


Learning goals13 l.jpg

LEARNING GOALS

  • What is peer review, and why have we to survive it through?

  • What should you do and not do when actually peer reviewing other colleagues’ works?

  • What should you do and not do when surviving through peer review?

  • Case studies


Are peer reviewers unbiased l.jpg

ARE PEER REVIEWERS UNBIASED?

Mahoneyet al, CTR 1977


Will peer review let you sleep l.jpg

WILL PEER REVIEW LET YOU SLEEP?

On average a review will take you around 3 hours (actually I usually complete one in 45’)

McNuttet al, JAMA 1990


Should you be blindfolded l.jpg

SHOULD YOU BE BLINDFOLDED?

Godleeet al, JAMA 1998


Who are the best reviewers l.jpg

WHO ARE THE BEST REVIEWERS?

  • Other (also weak) predictors:

  • Coming from good institutions

  • Known to the editors

  • Had methodological training (statistics & epidemiology)

Klieveret al, AJR 2005


Who are the worst reviewers l.jpg

WHO ARE THE WORST REVIEWERS?

Italian peer reviewers don’t like Italian manuscripts!

Opthofet al, CardiovascRes 2002


Three pivotal questions for peer review l.jpg

THREE PIVOTAL QUESTIONS FOR PEER REVIEW

  • Do I understand it? Are the question and the methods clearly explained?

  • Do I believe it? Are the conclusions justified by the data and are the methods valid?

  • Do I care? Is the question important and interesting?

Wageret al, Howtosurvivepeerreview. BMJ Books 2002


Checklist for peer reviewers issues for comment l.jpg

CHECKLIST FOR PEER REVIEWERS: ISSUES FOR COMMENT

  • Importance of research question

  • Originality of work

  • Delineation of strengths and weaknesses of methodology/experimental/statistical approach/interpretation of results

  • Writing style and figure/table presentation

  • Ethical concerns (animal/human)

Benoset al, AdvanPhysiolEduc 2003


Subtleties of peer review priority i e how to kill a paper l.jpg

SUBTLETIES OF PEER REVIEW: PRIORITY – i.e. how to kill a paper

  • Priority means novelty, originality, and likelihood of generating interest, irrespective from quality, validity, and methodology

  • Manuscript can be judged as low, mid, high, or top priority

  • Some journals (e.g. the Journal of Cardiovascular Medicine) use scores, e.g. ranging from 0 to 100, with 100 meaning top priority


Subtleties of peer review decision l.jpg

SUBTLETIES OF PEER REVIEW: DECISION

  • Decision means recommending a specific editorial handling of the manuscript, and can be distinguished in:

    • accept as is

    • accept/reconsider after minor revisions

    • accept/reconsider after major revisions

    • reject but reconsider on a de novo basis

    • reject


Subtleties of peer review decision23 l.jpg

SUBTLETIES OF PEER REVIEW: DECISION

  • Accept as is: it can be published as it stands

  • Accept after minor revisions: it can be published EVEN IF not all my comments are taken into account

  • Accept/reconsider after major revisions: it can be published ONLY IF all my comments are taken into account

  • Reject but reconsider on a de novo basis: it must be changed altogether, and priority also reappraised after resubmission

  • Reject: just send it back, it ain’t worth it


Example of active peer review l.jpg

EXAMPLE OF ACTIVE PEER REVIEW


Learning goals25 l.jpg

LEARNING GOALS

  • What is peer review, and why have we to survive it through?

  • What should you do and not do when actually peer reviewing other colleagues’ works?

  • What should you do and not do when surviving through peer review?

  • Case studies


Slide26 l.jpg

FIRST, DON’T PANIC!


Should you trust peer reviewers l.jpg

SHOULD YOU TRUST PEER REVIEWERS?

Opthofet al, CardiovascRes 2002


Identifying the most appropriate target l.jpg

IDENTIFYING THE MOST APPROPRIATE TARGET

Wageret al, Howtosurvivepeerreview. BMJ Books 2002


Cover letter l.jpg

COVER LETTER


Replying to reviewers comments l.jpg

REPLYING TO REVIEWERS COMMENTS

  • Do not ignore comments, but make most if not all of changes

  • Be calm, objective, and polite even if reviewer’s comments were harsh

  • Deconstruct each of the messages into individual items

  • Respond to each item thoughtfully

  • Make responses clear


Replying to reviewers comments31 l.jpg

REPLYING TO REVIEWERS COMMENTS

  • Don’t have to fully accept suggestions but must give reasons that will convince editor your opinion is reasonable

  • Be pragmatic and not dismissive of reviewer’s work

  • Explain just enough to enable you to survive

  • Benefit from it, and learn also how to become a competent reviewer


Rebuttal letter l.jpg

REBUTTAL LETTER


After resubmission of revision l.jpg

AFTER RESUBMISSION OF REVISION

  • No guarantee will be published

  • Editor will consider new version and your replies to comments

  • Editorial process can be subjective, and sometimes downgrade priority -> rejection

  • Reviewer’s comments only one factor

  • Editor may reject paper even if reviewer’s comments were minor

  • Editor has absolute discretion


If rejected can appeal l.jpg

IF REJECTED CAN APPEAL

  • If you think reviewer’s overlooked or misunderstood something important

  • Appeal by writing a letter stating your case

    –> appeal letter

  • Rare decision overturned but it does happen

  • If appealing—send new copy of paper—rejected papers do not remain on file (i.e. de novo submission)


What to do after rejection l.jpg

WHAT TO DO AFTER REJECTION

  • Reviewers critical of basic methods—may need to rethink study and do further data analysis

  • Reviewers critical on style and presentation—fix problems before resubmitting to another journal

  • Three repeat rejections—completely reassess entire approach or search for appropriate target


When accepted l.jpg

WHEN ACCEPTED

  • Receive page proofs—typeset copy of work—how looks in journal

  • May take several months to receive

  • Time for final check

  • Journal usually send proof reading instructions you must follow

  • Usually standard proofreading marks


Typical reasons for final rejection l.jpg

TYPICAL REASONS FOR FINAL REJECTION

  • Fundamentally weak hypothesis

  • Lack of clinical relevance

  • Old knowledge with no new or useful material

  • Two or three of reasons 1–3

  • Reasonable text, but images are of very poor quality, are inappropriate, or are incorrectly interpreted

  • Too many methodologic errors

  • Hypothesis adequate, but poor study design, methodology, or statistics

  • Lacking in logic; initial premise not logically supported by methods and results

Eharaet al, AJR 2007


Typical reasons for final rejection cont l.jpg

TYPICAL REASONS FOR FINAL REJECTION (CONT.)

  • None of the other reasons, but reviewers do not like the article

  • Previously published

  • Sample population too small or biased to justify results and conclusion

  • Well written but better suited for another journal

  • Major language problems; English not primary language of author

  • Too poorly written, phrased, or presented

  • Failure to follow journal guidelines

  • Lack of correlation between purpose and results

  • Poor statistics, beyond salvage

Eharaet al, AJR 2007


Ten tips to for survival l.jpg

TEN TIPS TO FOR SURVIVAL

  • Properly organize the manuscript

  • Clearly state the study question and rationale

  • Explain methods in a systematic fashion

  • Structure methods and results in a similar manner

  • Make the discussion session concise

  • Explain if – and why – your study results are important

  • Avoid overinterpreting

  • Explain the limitations

  • Account for unexpected findings

  • Fully incorporate reviewers’ suggestions

Provenzale et al, AJR 2007


Learning goals40 l.jpg

LEARNING GOALS

  • What is peer review, and why have we to survive it through?

  • What should you do and not do when actually peer reviewing other colleagues’ works?

  • What should you do and not do when surviving through peer review?

  • Case studies


The loverboy reviewer l.jpg

THE LOVERBOY REVIEWER

Lotrionteet al, Am J Cardiol 2008


The loverboy reviewer42 l.jpg

THE LOVERBOY REVIEWER

Lotrionteet al, Am J Cardiol 2008


The inquiring reviewer l.jpg

THE INQUIRING REVIEWER

Biondi-Zoccaiet al, Am Heart J 2008


The inquiring reviewer44 l.jpg

THE INQUIRING REVIEWER

Biondi-Zoccaiet al, Am Heart J 2008


The delusional reviewer l.jpg

THE DELUSIONAL REVIEWER

Sheibanet al, J Am CollCardiol 2009


The delusional reviewer46 l.jpg

THE DELUSIONAL REVIEWER

Sheibanet al, J Am CollCardiol 2009


The nasty reviewer l.jpg

THE NASTY REVIEWER

Sheibanet al, J IntervenCardiol 2008


The nasty reviewer48 l.jpg

THE NASTY REVIEWER

Sheibanet al, J IntervenCardiol 2008


The pessimistic reviewer l.jpg

THE PESSIMISTIC REVIEWER

Biondi-Zoccaiet al, Am Heart J 2008


The pessimistic reviewer50 l.jpg

THE PESSIMISTIC REVIEWER

Biondi-Zoccaiet al, Am Heart J 2008


The ask too much reviewer l.jpg

THE ASK-TOO-MUCH REVIEWER

27 COMMENTS JUST FROM THE STATISTICAL REVIEWER!!!

HOW CAN YOU TACKLE THIS?

Biondi-Zoccaiet al, BMJ 2006


The ask too much reviewer52 l.jpg

THE ASK-TOO-MUCH REVIEWER

27 COMMENTS JUST FROM THE STATISTICAL REVIEWER!!!

HOW CAN YOU TACKLE THIS?

WITH A LOT OF PATIENCE!!!

Biondi-Zoccaiet al, BMJ 2006


Take home messages l.jpg

TAKE HOME MESSAGES

What a very bad reviewer once told Amir Lerman (Mayo Clinic, USA):

Delete the first phrase of the second section of the Discussion (“It remains to be speculated that…”).

Better, print it, put it in a frame and hang it in front of your desk to remind you never to repeat such mistake….


Take home messages54 l.jpg

TAKE HOME MESSAGES

  • Peer review remains a time-consuming and painful process, but still the best available

  • Active peer review is best summarized by Matthew 19, 16-19: Thou shalt love thy neighbor as thyself

  • Passive peer review should be based on patience, perserverance, and sound methods


Slide55 l.jpg

Thank you for your attentionFor any correspondence: [email protected] these and further slides on these topics feel free to visit the metcardio.org website:http://www.metcardio.org/slides.html


  • Login