Reviewer: You could be making waffles with your family, but instead you are writing a review for a manuscript sent to you 2 weeks ago. The paper is in an area of strong interest, but it has been hard to find the time to give it attention. You wonder if the effort is worth it.

Author: Rather than having brunch, you are digesting the reviews that you received regarding your manuscript. You are anxious to revise your paper, the product of many months of writing, yet you wish the reviewers were more explicit and clear about their concerns.

Editor: You are debating whether to search for yet another reviewer for a manuscript that is growing as stale as your bagel. The paper seems to have value, but it needs a careful review to translate the authors' work for the average reader.

Peer review is the foundation of science dissemination, and high-quality reviews are essential to this process. Few would argue with the collective value of peer-review activities. Yet many ponder the question What's in it for me? when asked to review a paper. Conducting an effective peer review requires time, effort, and training.1,2  When faced with a multitude of competing demands, it can be challenging for busy professionals to see the personal value in reviewing papers, particularly if they feel unskilled in or unprepared for these activities.

As Journal of Graduate Medical Education (JGME) editors, we know that the work of our reviewers is the bedrock upon which high-quality medical education scholarship is built. We propose a paradigm shift in how we think about this activity to maximize the benefits of peer review for reviewers, authors, and editors. In this editorial, we assess the benefits of traditional reviewing, and suggest a new process of shared peer review activities that can add value.

Old School: Peer Review as a Solitary Activity

In the traditional paradigm, peer review occurs as a solitary activity in which the reviewer judges the manuscript's potential contribution to the field and provides guidance to strengthen its value. For individuals seeking to build their medical education research skills and reputation, this traditional paradigm offers numerous opportunities for professional development (box 1). Examining submitted manuscripts in their prepublication form exposes reviewers to a variety of approaches to how authors' frame questions, construct methods, present data analysis, interpret results, and build cogent discussions. Moreover, many journals, including JGME, carbon copy reviewers when decision letters are e-mailed to authors; reviewers then can use these letters to check their impressions against those of their peers, evaluate how their critiques align with editorial feedback, and follow the progress of articles toward publication. Because most journals rank reviewers' performance on each review,3  motivated individuals can solicit feedback directly from editors.

box 1 Benefits of Individual Peer Review
  • Learn content through exposure to emerging questions and background literature

  • Learn theory, research approaches, and curricular designs by evaluating conceptual frameworks, methods, and novel programs

  • Enhance writing skills through exposure to successful and less successful styles

  • Stay up to date with new research initiatives or curricular innovations

  • Experience personal satisfaction in contributing to the field, and a sense of reciprocation for prior reviews of one's own work

  • Earn recognition as an outstanding reviewer, with potential opportunities to join editorial boards

  • Gain opportunities to write commentaries

While peer review offers an efficient way to advance medical education scholarship while gaining feedback on research and writing skills, conducting peer review as a solitary activity has several drawbacks. First, reviewing requires a diverse background of knowledge and skills, and new reviewers may feel ill prepared to tackle this work independently. There are journal-specific, online, and academic society resources that provide preliminary guidance for performing peer review.37  However, these contain general advice and may address neither the unique challenges of each manuscript nor the idiosyncratic knowledge deficits of each reviewer. Second, seeing the blind carbon copy of reviewer and editor feedback for a manuscript reviewed 2 weeks prior requires the reviewer to take the time to comb through comments to identify the differences between his or her review and those of other reviewers and the editor(s). This feedback may not yield the level of specific detail needed for individual reviewer improvement. Reviewers can seek specific editorial feedback regarding the quality of their reviews, but this entails additional effort, and feedback is not always provided. Although an invitation to review another article implies some degree of positive feedback, reviewers still lack specific guidance for how they might strengthen future reviews.

New School: Peer Review as Community Engagement and Mentorship

If we view the acquisition of peer-review skills through the lens of deliberate practice,8  we find that reviewers seeking to identify and remediate their knowledge gaps require opportunities for frequent repetition with timely and specific feedback. Lacking this structure, the individual-driven peer-review paradigm may not consistently produce high-quality reviews, particularly for new reviewers. How might this change if we viewed peer review as an opportunity to engage local or virtual communities of educators in a shared scholarly activity?

Most medical educators participate in various forms of a “journal club” as a means of reviewing emerging literature for potential translation into practice. A similar format can be adapted as a form of “group peer review” with individuals who are geographically or virtually connected, as has been modeled by the JACC Heart Failure Fellows Program review of published articles.2  Led by an individual who has been approached by a journal to review an article, group members independently review the article before meeting to discuss and debate their ratings. The group leader then submits the consolidated review to the journal. These review groups can be structured in a hierarchical fashion, with novice reviewers tackling the peer review before sending their impressions to experienced mentors, who subsequently provide feedback and additional insights.

Several JGME editors have employed this model successfully within postgraduate training or faculty development programs. Alternatively, review groups can function as a shared experience among peers. While reviewers should check with other journals whether such a group review format is acceptable, JGME supports this practice and requires no advance permission. We require that the same standards of confidentiality apply to these group reviews, and that reviewers cite the members of their review team when a review is submitted.

A group peer-review format offers benefits to the individual, community, and journal (box 2). Formally discussing a manuscript review exposes individuals to a diversity of perspectives and provides opportunities to give and receive immediate feedback. Furthermore, it helps to strengthen the identity of a medical education community, forges bonds among educators, unmasks shared interests, exposes underlying assumptions and biases, and highlights how issues addressed in scholarly articles might inform similar challenges in local contexts. Analogous to how traditional journal clubs have piloted virtual formats,9  group peer-review structures allow individuals in different locations to connect and discuss articles confidentially within closed virtual communities. Group peer reviews may generate feedback that is more diverse and nuanced than the opinions of an individual, which ultimately strengthens the rigor of the final review. Most importantly, tackling articles at this formative stage can be fun!

box 2 Benefits of Group Peer Review
  • Gain exposure to a diversity of perspectives

  • Provide and receive immediate and more detailed feedback

  • Strengthen medical education communities

  • Create bonds among educators and unmask shared interests

  • Uncover underlying personal assumptions and biases

  • Provide ongoing asynchronous discussions in virtual communities

  • Produce a more rigorous and comprehensive final review product

  • Provide mentorship to junior colleagues and trainees

  • Enrich a formal instructional program

  • Have fun discussions with your colleagues!

For faculty and trainees involved in formal educational programs,2,10  group peer review can be an avenue for mentorship and a valuable instructional approach. By challenging junior colleagues to read and critique medical education scholarship, mentors can assess and address their mentees' knowledge gaps. Experienced reviewers can broaden junior reviewers' perspectives by offering feedback specific to the article itself (such as relevant background literature, methods, or interpretations of the data), and impart practical tips for writing efficiently and constructively (box 3). These tips may include providing comments with page and line numbers for easy reference; suggesting additional relevant citations; providing guidance for clearer data presentation aligned with journal guidelines; and maintaining a file of “review phrases” for responses to common manuscript problems. Experienced reviewers can share their reviewer phrases and model how to provide constructive feedback using nonjudgmental language that conveys an implicit commitment to help authors strengthen their work.3,5,11  In our experience, high-quality peer-review comments require a nuanced language that often doesn't come naturally to junior faculty—or even to some senior faculty.

box 3 Tips for Structuring a High-Quality Review
  • Provide a brief 1- to 2-sentence summary of your general impressions

  • Highlight your major concerns

  • Number each section of feedback so authors can refer to them when they respond

  • Keep your feedback concise: sentences, not paragraphs

  • Be specific: list specific page and line numbers or quote prose from the manuscript

  • Maintain a collegial but objective tone; avoid sounding harsh or judgmental

  • Include references and citations to support your critiques when relevant

  • Keep a file of common reviewer phrases, references, or explanations

  • Use the “Comments to the Editor” section for overall impressions of the article, particularly regarding acceptance or rejection

  • Avoid telling authors in the “Comments to Author” section that the paper should be accepted or rejected, as this is an editorial decision

  • Respond quickly to editorial requests to review; use the “now or never” approach

Note: Several suggestions in this list have been adapted from DeMaria.3 

We recognize that finding a local peer-review mentor may not be feasible for all individuals. As is the case with other journals,12  JGME editors may be able to identify more experienced reviewers to assist novice reviewers seeking mentorship. Developing virtual communities that support “coached peer review,” similar to processes described for the review of online educational content,13  offers another promising solution for training the next generation of reviewers.

Conclusion

Peer review requires practice and feedback. While many individuals develop these skills by regularly performing—and seeking feedback on—their own reviews, reconceptualizing peer review as a community of practice offers rich possibilities for coaching, bridge building, and strengthening educational communities.14 

We invite your thoughts regarding how we might create reviewer communities within graduate medical education sponsoring organizations and programs. If you are interested, please e-mail us (jgme@acgme.org) a description of what you would like to do (along with the participants and areas of interest) and an editor will work with your community to identify an appropriate article for group review.

Finally, many thanks to the past, current, and future reviewers of JGME, particularly because we know that this work may come at the expense of your Sunday brunch!

References

References
1
Evans
AT,
McNutt
RA,
Fletcher
SW,
et al.
The characteristics of peer reviewers who produce good-quality reviews
.
J Gen Intern Med
.
1993
;
8
(
8
):
422
428
.
2
Cooper
LB,
Bellam
N,
Vaduganathan
M,
et al.
Educating the next generation of peer reviewers
.
J Am Coll Cardiol
.
2016
;
67
(
17
):
2079
2082
.
3
DeMaria
AN.
What constitutes a great review?
J Am Coll Cardiol
.
2003
;
42
(
7
):
1314
1315
.
4
Azer
SA,
Ramani
S,
Peterson
R.
Becoming a peer reviewer to medical education journals
.
Med Teach
.
2012
;
34
(
9
):
698
704
.
5
Roediger
HL
III.
Twelve tips for reviewers
.
Observer
.
April 2007.
.
6
Lucey
B.
Peer review: How to get it right—10 tips
.
The Guardian
.
September 27, 2013.
.
7
Durning
SJ,
Carline
JD,
eds
.
Review Criteria for Research Manuscripts. 2nd ed
.
Washington, DC
:
Association of American Medical Colleges;
2015
.
8
Ericsson
KA.
Deliberate practice and acquisition of expert performance: a general overview
.
Acad Emerg Med
.
2008
;
15
(
11
):
988
994
.
9
Chan
TM,
Thoma
B,
Radecki
R,
et al.
Ten steps for setting up an online journal club
.
J Contin Educ Health Prof
.
2015
;
35
(
2
):
148
154
.
10
Tekian
A,
Harris
I.
Preparing health professions education leaders worldwide: a description of masters-level programs
.
Med Teach
.
2012
;
34
(
1
):
52
58
.
11
Benson
E.
The perils of peer review
.
gradPSYCH
.
May 2003.
.
12
Eva
KW.
The reviewer is always right: peer review of research in medical education
.
Med Educ
.
2009
;
43
(
1
):
2
4
.
13
Sidalak
D,
Purdy
E,
Luckett-Gatopoulos
S,
et al.
Coached peer review: developing the next generation of authors
.
Acad Med
.
2016
May
17
.
Epub ahead of print.
14
Wenger
E.
Communities of Practice: Learning, Meaning, and Identity
.
Cambridge, UK
:
Cambridge University Press;
1998
.