Peer reviewers provide editors with information about the scientific merit of a manuscript and provide authors with suggestions to strengthen the presentation, content, and clarity of their manuscript.1,2  Yet, training on how to provide high-quality reviews is often limited and not standardized.

Emerging literature provides insights on training peer reviewers. An analysis of 116 journals revealed that 35% of journals provided online reviewer instructions, and typically, these instructions focused on the logistics, professionalism, and ethics of the review process.3  Studies evaluating training interventions (eg, structured workshops led by journal editors, independent learning modules, checklists, and mentoring) have been sparse.4  Peer review is strengthened per a consensus of perspectives when reviewers clearly understand the goals of, and their role in, the peer-review process, along with the expected content and format of the review report.1,5,6  This Rip Out complements the Journal of Graduate Medical Education's (JGME's) instructions for peer reviewers,1  which highlight these features by providing best practices from prior reviewers, now JGME editors.

Rip Out Action Items
  • 1. 

    Serve as a peer reviewer—it is vital to advancing medical education, and you will learn too.

  • 2. 

    Before accepting an invitation to review, consider potential conflicts, the match between your expertise and the paper, and your time.

  • 3. 

    Continue to enhance your peer-review skills by using a structured reviewer report format, seeking feedback, and mentoring others.

  • 1. 

    Volunteer to be a reviewer for a journal that you read. This usually requires an e-mail to the editorial office with a brief summary of your background and interests.

  • 2. 

    Actively address the ethical issues of being a peer reviewer. When a review invitation arrives in your inbox, determine if you have a conflict of interest or a competing interest (1 or more authors are a friend, family member, or current collaborator). If you have a conflict of interest, decline the review. If you are unsure if there is a conflict, contact the journal editor for advice.

  • 3. 

    Exhibit professionalism. If you have insufficient expertise in the topic area, decline the review. If you cannot submit the report within the requested time frame, ask for an extension. If you accept the review, do not disclose any information from the manuscript to anyone without the editor's permission.

  • 4. 

    Understand the journal, its audience, and publication criteria. Each journal has a unique audience, focus, manuscript categories, and criteria (word limit, number of tables and figures, section headings). Review past issues and guidelines.

  • 5. 

    Read the paper through and organize your review. Does the paper address a topic of current interest and relevance to the journal's audience? Present original information? Cite relevant literature? Use journal format and organization? See the “Assessing Key Elements of a Manuscript” box and online supplemental material.

  • 6. 

    Organize your written review. With a courteous, supportive tone, provide a 1- to 2-sentence overall summary of the paper's purpose. Then, section by section, provide numbered comments to which the author(s) should respond. Label these comments with section, page number, or line number(s) to direct authors. Focus on content, clarity, and organization rather than grammar, punctuation, or spelling. However, for papers that need an overall rewrite for correct English use, flag the editor.

  • 7. 

    Provide your recommendation to editors regarding accept, revise, reject. Do not provide a decision to authors. Editors can best determine, given other papers in the pipeline, disposition of the paper. Do provide confidential comments to editors, when appropriate.

Assessing Key Elements of a Manuscript

Ask yourself as a peer reviewer, does the paper:

  • Address a topic of current interest and relevance to the journal's audience?

  • Present new information or corroborate information that is not well accepted?

  • Cite and discuss relevant literature to justify the need for the study?

  • Provide a clear research purpose or question?

  • Describe methods in sufficient detail so that others could replicate the study?

  • Analyze the results with accepted quantitative or qualitative methods?

  • Provide the results in easy to read (text) or view (tables, figures) formats?

  • Discuss the results in light of what is already known, along with the methodological limitations?

  • Provide conservative conclusions from the study findings, without editorializing?

  • Fit within the journal's format and category restrictions?

  • Tell the story in a clear manner with good flow?

  • 1. 

    Improve your reviews by asking for editors' feedback on your reviews. If the journal blind copies you on the author's letter, compare your reviews with other reviewers' and editors' comments.

  • 2. 

    Read your own manuscripts as if you were a “reviewer.”

  • 3. 

    Seek advice from senior colleagues. Mentor junior colleagues as peer reviewers. Consider joining or forming a medical education journal club or group reviewer program.7 

  • 4. 

    Become a member of a journal editorial board after honing your skills as a peer reviewer.

1
Journal of Graduate Medical Education
.
Guidance for JGME peer reviewers
.
Updated March 2014.
,
2017
.
2
Mulligan
A,
Hall
L,
Raphael
E.
Peer review in a changing world: an international study measuring the attitudes of researchers
.
J Am Soc Inf Sci Tech
.
2013
;
64
(
1
):
132
161
.
3
Hirst
A,
Altman
DG.
Are peer reviewers encouraged to use reporting guidelines? A survey of 116 health research journals
.
PLoS One
.
2012
;
7
(
4
):
e3561
.
4
Bruce
R,
Chauvin
A,
Tinquart
L,
et al.
Impact of interventions to improve the quality of peer review of biomedical journals: a system review and meta-analysis
.
BMC Med
.
2016
;
14
(
1
):
85
.
5
Azer
SA,
Ramani
S,
Peterson
R.
Becoming a peer reviewer to medical education journals
.
Med Teach
.
2012
;
34
(
9
):
698
704
.
6
Academic Medicine
.
2017
.
7
Yarris
LM,
Simpson
D,
Ilgen
JS,
et al.
Team-based coaching approach to peer review: sharing service and scholarship
.
J Grad Med Educ
.
2017
;
9
(
1
):
127
128
.

Author notes

Editor's Note: The online version of this article contains the peer reviewer worksheet and a list of additional readings.

Supplementary data