first aid for the peer review process
Download
Skip this Video
Download Presentation
FIRST AID FOR THE PEER REVIEW PROCESS

Loading in 2 Seconds...

play fullscreen
1 / 55

FIRST AID FOR THE PEER REVIEW PROCESS - PowerPoint PPT Presentation


  • 100 Views
  • Uploaded on

FIRST AID FOR THE PEER REVIEW PROCESS. Giuseppe Biondi Zoccai Division of Cardiology , University of Turin , Turin , Italy Meta-analysis and Evidence-based medicine Training in Cardiology (METCARDIO), Ospedaletti , Italy.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'FIRST AID FOR THE PEER REVIEW PROCESS' - season


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
first aid for the peer review process

FIRST AID FOR THE PEER REVIEW PROCESS

Giuseppe Biondi Zoccai

DivisionofCardiology, UniversityofTurin, Turin, Italy

Meta-analysis and Evidence-based medicine Training in Cardiology(METCARDIO), Ospedaletti, Italy

How to develop effective abstracts and manuscripts in interventional cardiology

GISE 2009 – 22 October 2009 – 13.25-13.50

learning goals
LEARNING GOALS
  • What is peer review, and why have we to survive it through?
  • What should you do and not do when actually peer reviewing other colleagues’ works?
  • What should you do and not do when surviving through peer review?
  • Case studies
why am i giving you this lecture
WHY AM I GIVING YOU THIS LECTURE?

Peer reviews I have completed since 2003

expected

learning goals4
LEARNING GOALS
  • What is peer review, and why have we to survive it through?
  • What should you do and not do when actually peer reviewing other colleagues’ works?
  • What should you do and not do when surviving through peer review?
  • Case studies
definition
DEFINITION

Peer

vb intr. 1. to look intently with or as if with difficulty. 2. to appear partially or dimly.

n. 1. a person who is an equal in social standing, rank, age, etc.

Collins Dictionary of the English Language,

London & Glasgow: Collins, 1979

origins of peer review
ORIGINS OF PEER REVIEW
  • Peer review was born in the 17th century in the UK, likely thanks to Henry Oldenburg, the founder of Philosophical Abstractions (1665)
  • He originally introduced the practice of soliciting opionions on manuscripts from more knowledgeable external colleagues
notable exceptions
NOTABLE EXCEPTIONS
  • Not all manuscripts undergo thorough peer review. Notable examples?
  • Albert Einstein’s “AnnusMirabilis” papers, published in 1905 in AnnalenderPhysikbyMax Planck(fatherof quantum theory and Nobel prizewinner), whoread the papers and decidedaltogethertopublishthem
  • more humbly…Biondi-Zoccaiet al, Int J Cardiol2005;100:119-23
what s wrong with peer review
WHAT’S WRONG WITH PEER REVIEW?
  • Unreliable
  • Unfair
  • Fails to truly validate or authenticate
  • Unstandardized
  • Idiosyncratic
  • Open to every sort of bias
what s even worse
WHAT’S EVEN WORSE?
  • Stifles innovation
  • Perpetuates the status quo
  • Rewards the prominent but punishes the weak
  • Unnecessarily delays dissemination
  • Very expensive
  • Insufficiently tested
yes but
YES, BUT…
  • It has been said that democracy is the worst form of government except all those other forms that have been tried from time to time – W. Churchill
  • The sameappliestopeerreview
what are its pros
WHAT ARE ITS PROS?
  • Filters bad from good, and better from less good, as well as possibly identifying misleading and false research
  • Protects patients
  • Guides authors to improve the quality of their article, and improve their whole research approach
  • Authenticates work, assuring quality
  • Improves readability
  • Broadens participation and dialogue
paraphrasing groucho
PARAPHRASING GROUCHO

I would never enter into a club that would accept me as a member…

Groucho Marx, 1980-1977

learning goals13
LEARNING GOALS
  • What is peer review, and why have we to survive it through?
  • What should you do and not do when actually peer reviewing other colleagues’ works?
  • What should you do and not do when surviving through peer review?
  • Case studies
are peer reviewers unbiased
ARE PEER REVIEWERS UNBIASED?

Mahoneyet al, CTR 1977

will peer review let you sleep
WILL PEER REVIEW LET YOU SLEEP?

On average a review will take you around 3 hours (actually I usually complete one in 45’)

McNuttet al, JAMA 1990

should you be blindfolded
SHOULD YOU BE BLINDFOLDED?

Godleeet al, JAMA 1998

who are the best reviewers
WHO ARE THE BEST REVIEWERS?
  • Other (also weak) predictors:
  • Coming from good institutions
  • Known to the editors
  • Had methodological training (statistics & epidemiology)

Klieveret al, AJR 2005

who are the worst reviewers
WHO ARE THE WORST REVIEWERS?

Italian peer reviewers don’t like Italian manuscripts!

Opthofet al, CardiovascRes 2002

three pivotal questions for peer review
THREE PIVOTAL QUESTIONS FOR PEER REVIEW
  • Do I understand it? Are the question and the methods clearly explained?
  • Do I believe it? Are the conclusions justified by the data and are the methods valid?
  • Do I care? Is the question important and interesting?

Wageret al, Howtosurvivepeerreview. BMJ Books 2002

checklist for peer reviewers issues for comment
CHECKLIST FOR PEER REVIEWERS: ISSUES FOR COMMENT
  • Importance of research question
  • Originality of work
  • Delineation of strengths and weaknesses of methodology/experimental/statistical approach/interpretation of results
  • Writing style and figure/table presentation
  • Ethical concerns (animal/human)

Benoset al, AdvanPhysiolEduc 2003

subtleties of peer review priority i e how to kill a paper
SUBTLETIES OF PEER REVIEW: PRIORITY – i.e. how to kill a paper
  • Priority means novelty, originality, and likelihood of generating interest, irrespective from quality, validity, and methodology
  • Manuscript can be judged as low, mid, high, or top priority
  • Some journals (e.g. the Journal of Cardiovascular Medicine) use scores, e.g. ranging from 0 to 100, with 100 meaning top priority
subtleties of peer review decision
SUBTLETIES OF PEER REVIEW: DECISION
  • Decision means recommending a specific editorial handling of the manuscript, and can be distinguished in:
    • accept as is
    • accept/reconsider after minor revisions
    • accept/reconsider after major revisions
    • reject but reconsider on a de novo basis
    • reject
subtleties of peer review decision23
SUBTLETIES OF PEER REVIEW: DECISION
  • Accept as is: it can be published as it stands
  • Accept after minor revisions: it can be published EVEN IF not all my comments are taken into account
  • Accept/reconsider after major revisions: it can be published ONLY IF all my comments are taken into account
  • Reject but reconsider on a de novo basis: it must be changed altogether, and priority also reappraised after resubmission
  • Reject: just send it back, it ain’t worth it
learning goals25
LEARNING GOALS
  • What is peer review, and why have we to survive it through?
  • What should you do and not do when actually peer reviewing other colleagues’ works?
  • What should you do and not do when surviving through peer review?
  • Case studies
should you trust peer reviewers
SHOULD YOU TRUST PEER REVIEWERS?

Opthofet al, CardiovascRes 2002

identifying the most appropriate target
IDENTIFYING THE MOST APPROPRIATE TARGET

Wageret al, Howtosurvivepeerreview. BMJ Books 2002

replying to reviewers comments
REPLYING TO REVIEWERS COMMENTS
  • Do not ignore comments, but make most if not all of changes
  • Be calm, objective, and polite even if reviewer’s comments were harsh
  • Deconstruct each of the messages into individual items
  • Respond to each item thoughtfully
  • Make responses clear
replying to reviewers comments31
REPLYING TO REVIEWERS COMMENTS
  • Don’t have to fully accept suggestions but must give reasons that will convince editor your opinion is reasonable
  • Be pragmatic and not dismissive of reviewer’s work
  • Explain just enough to enable you to survive
  • Benefit from it, and learn also how to become a competent reviewer
after resubmission of revision
AFTER RESUBMISSION OF REVISION
  • No guarantee will be published
  • Editor will consider new version and your replies to comments
  • Editorial process can be subjective, and sometimes downgrade priority -> rejection
  • Reviewer’s comments only one factor
  • Editor may reject paper even if reviewer’s comments were minor
  • Editor has absolute discretion
if rejected can appeal
IF REJECTED CAN APPEAL
  • If you think reviewer’s overlooked or misunderstood something important
  • Appeal by writing a letter stating your case

–> appeal letter

  • Rare decision overturned but it does happen
  • If appealing—send new copy of paper—rejected papers do not remain on file (i.e. de novo submission)
what to do after rejection
WHAT TO DO AFTER REJECTION
  • Reviewers critical of basic methods—may need to rethink study and do further data analysis
  • Reviewers critical on style and presentation—fix problems before resubmitting to another journal
  • Three repeat rejections—completely reassess entire approach or search for appropriate target
when accepted
WHEN ACCEPTED
  • Receive page proofs—typeset copy of work—how looks in journal
  • May take several months to receive
  • Time for final check
  • Journal usually send proof reading instructions you must follow
  • Usually standard proofreading marks
typical reasons for final rejection
TYPICAL REASONS FOR FINAL REJECTION
  • Fundamentally weak hypothesis
  • Lack of clinical relevance
  • Old knowledge with no new or useful material
  • Two or three of reasons 1–3
  • Reasonable text, but images are of very poor quality, are inappropriate, or are incorrectly interpreted
  • Too many methodologic errors
  • Hypothesis adequate, but poor study design, methodology, or statistics
  • Lacking in logic; initial premise not logically supported by methods and results

Eharaet al, AJR 2007

typical reasons for final rejection cont
TYPICAL REASONS FOR FINAL REJECTION (CONT.)
  • None of the other reasons, but reviewers do not like the article
  • Previously published
  • Sample population too small or biased to justify results and conclusion
  • Well written but better suited for another journal
  • Major language problems; English not primary language of author
  • Too poorly written, phrased, or presented
  • Failure to follow journal guidelines
  • Lack of correlation between purpose and results
  • Poor statistics, beyond salvage

Eharaet al, AJR 2007

ten tips to for survival
TEN TIPS TO FOR SURVIVAL
  • Properly organize the manuscript
  • Clearly state the study question and rationale
  • Explain methods in a systematic fashion
  • Structure methods and results in a similar manner
  • Make the discussion session concise
  • Explain if – and why – your study results are important
  • Avoid overinterpreting
  • Explain the limitations
  • Account for unexpected findings
  • Fully incorporate reviewers’ suggestions

Provenzale et al, AJR 2007

learning goals40
LEARNING GOALS
  • What is peer review, and why have we to survive it through?
  • What should you do and not do when actually peer reviewing other colleagues’ works?
  • What should you do and not do when surviving through peer review?
  • Case studies
the loverboy reviewer
THE LOVERBOY REVIEWER

Lotrionteet al, Am J Cardiol 2008

the loverboy reviewer42
THE LOVERBOY REVIEWER

Lotrionteet al, Am J Cardiol 2008

the inquiring reviewer
THE INQUIRING REVIEWER

Biondi-Zoccaiet al, Am Heart J 2008

the inquiring reviewer44
THE INQUIRING REVIEWER

Biondi-Zoccaiet al, Am Heart J 2008

the delusional reviewer
THE DELUSIONAL REVIEWER

Sheibanet al, J Am CollCardiol 2009

the delusional reviewer46
THE DELUSIONAL REVIEWER

Sheibanet al, J Am CollCardiol 2009

the nasty reviewer
THE NASTY REVIEWER

Sheibanet al, J IntervenCardiol 2008

the nasty reviewer48
THE NASTY REVIEWER

Sheibanet al, J IntervenCardiol 2008

the pessimistic reviewer
THE PESSIMISTIC REVIEWER

Biondi-Zoccaiet al, Am Heart J 2008

the pessimistic reviewer50
THE PESSIMISTIC REVIEWER

Biondi-Zoccaiet al, Am Heart J 2008

the ask too much reviewer
THE ASK-TOO-MUCH REVIEWER

27 COMMENTS JUST FROM THE STATISTICAL REVIEWER!!!

HOW CAN YOU TACKLE THIS?

Biondi-Zoccaiet al, BMJ 2006

the ask too much reviewer52
THE ASK-TOO-MUCH REVIEWER

27 COMMENTS JUST FROM THE STATISTICAL REVIEWER!!!

HOW CAN YOU TACKLE THIS?

WITH A LOT OF PATIENCE!!!

Biondi-Zoccaiet al, BMJ 2006

take home messages
TAKE HOME MESSAGES

What a very bad reviewer once told Amir Lerman (Mayo Clinic, USA):

Delete the first phrase of the second section of the Discussion (“It remains to be speculated that…”).

Better, print it, put it in a frame and hang it in front of your desk to remind you never to repeat such mistake….

take home messages54
TAKE HOME MESSAGES
  • Peer review remains a time-consuming and painful process, but still the best available
  • Active peer review is best summarized by Matthew 19, 16-19: Thou shalt love thy neighbor as thyself
  • Passive peer review should be based on patience, perserverance, and sound methods
slide55
Thank you for your attentionFor any correspondence: [email protected] these and further slides on these topics feel free to visit the metcardio.org website:http://www.metcardio.org/slides.html
ad