Reviewer: You could be making waffles with your family, but instead you are writing a review for a manuscript sent to you 2 weeks ago. The paper is in an area of strong interest, but it has been hard to find the time to give it attention. You wonder if the effort is worth it.
Author: Rather than having brunch, you are digesting the reviews that you received regarding your manuscript. You are anxious to revise your paper, the product of many months of writing, yet you wish the reviewers were more explicit and clear about their concerns.
Editor: You are debating whether to search for yet another reviewer for a manuscript that is growing as stale as your bagel. The paper seems to have value, but it needs a careful review to translate the authors' work for the average reader.
Peer review is the foundation of science dissemination, and high-quality reviews are essential to this process. Few would argue with the collective value of peer-review activities. Yet many ponder the question What's in it for me? when asked to review a paper. Conducting an effective peer review requires time, effort, and training.1,2 When faced with a multitude of competing demands, it can be challenging for busy professionals to see the personal value in reviewing papers, particularly if they feel unskilled in or unprepared for these activities.
As Journal of Graduate Medical Education (JGME) editors, we know that the work of our reviewers is the bedrock upon which high-quality medical education scholarship is built. We propose a paradigm shift in how we think about this activity to maximize the benefits of peer review for reviewers, authors, and editors. In this editorial, we assess the benefits of traditional reviewing, and suggest a new process of shared peer review activities that can add value.
Old School: Peer Review as a Solitary Activity
In the traditional paradigm, peer review occurs as a solitary activity in which the reviewer judges the manuscript's potential contribution to the field and provides guidance to strengthen its value. For individuals seeking to build their medical education research skills and reputation, this traditional paradigm offers numerous opportunities for professional development (box 1). Examining submitted manuscripts in their prepublication form exposes reviewers to a variety of approaches to how authors' frame questions, construct methods, present data analysis, interpret results, and build cogent discussions. Moreover, many journals, including JGME, carbon copy reviewers when decision letters are e-mailed to authors; reviewers then can use these letters to check their impressions against those of their peers, evaluate how their critiques align with editorial feedback, and follow the progress of articles toward publication. Because most journals rank reviewers' performance on each review,3 motivated individuals can solicit feedback directly from editors.
Learn content through exposure to emerging questions and background literature
Learn theory, research approaches, and curricular designs by evaluating conceptual frameworks, methods, and novel programs
Enhance writing skills through exposure to successful and less successful styles
Stay up to date with new research initiatives or curricular innovations
Experience personal satisfaction in contributing to the field, and a sense of reciprocation for prior reviews of one's own work
Earn recognition as an outstanding reviewer, with potential opportunities to join editorial boards
Gain opportunities to write commentaries
While peer review offers an efficient way to advance medical education scholarship while gaining feedback on research and writing skills, conducting peer review as a solitary activity has several drawbacks. First, reviewing requires a diverse background of knowledge and skills, and new reviewers may feel ill prepared to tackle this work independently. There are journal-specific, online, and academic society resources that provide preliminary guidance for performing peer review.3–7 However, these contain general advice and may address neither the unique challenges of each manuscript nor the idiosyncratic knowledge deficits of each reviewer. Second, seeing the blind carbon copy of reviewer and editor feedback for a manuscript reviewed 2 weeks prior requires the reviewer to take the time to comb through comments to identify the differences between his or her review and those of other reviewers and the editor(s). This feedback may not yield the level of specific detail needed for individual reviewer improvement. Reviewers can seek specific editorial feedback regarding the quality of their reviews, but this entails additional effort, and feedback is not always provided. Although an invitation to review another article implies some degree of positive feedback, reviewers still lack specific guidance for how they might strengthen future reviews.
New School: Peer Review as Community Engagement and Mentorship
If we view the acquisition of peer-review skills through the lens of deliberate practice,8 we find that reviewers seeking to identify and remediate their knowledge gaps require opportunities for frequent repetition with timely and specific feedback. Lacking this structure, the individual-driven peer-review paradigm may not consistently produce high-quality reviews, particularly for new reviewers. How might this change if we viewed peer review as an opportunity to engage local or virtual communities of educators in a shared scholarly activity?
Most medical educators participate in various forms of a “journal club” as a means of reviewing emerging literature for potential translation into practice. A similar format can be adapted as a form of “group peer review” with individuals who are geographically or virtually connected, as has been modeled by the JACC Heart Failure Fellows Program review of published articles.2 Led by an individual who has been approached by a journal to review an article, group members independently review the article before meeting to discuss and debate their ratings. The group leader then submits the consolidated review to the journal. These review groups can be structured in a hierarchical fashion, with novice reviewers tackling the peer review before sending their impressions to experienced mentors, who subsequently provide feedback and additional insights.
Several JGME editors have employed this model successfully within postgraduate training or faculty development programs. Alternatively, review groups can function as a shared experience among peers. While reviewers should check with other journals whether such a group review format is acceptable, JGME supports this practice and requires no advance permission. We require that the same standards of confidentiality apply to these group reviews, and that reviewers cite the members of their review team when a review is submitted.
A group peer-review format offers benefits to the individual, community, and journal (box 2). Formally discussing a manuscript review exposes individuals to a diversity of perspectives and provides opportunities to give and receive immediate feedback. Furthermore, it helps to strengthen the identity of a medical education community, forges bonds among educators, unmasks shared interests, exposes underlying assumptions and biases, and highlights how issues addressed in scholarly articles might inform similar challenges in local contexts. Analogous to how traditional journal clubs have piloted virtual formats,9 group peer-review structures allow individuals in different locations to connect and discuss articles confidentially within closed virtual communities. Group peer reviews may generate feedback that is more diverse and nuanced than the opinions of an individual, which ultimately strengthens the rigor of the final review. Most importantly, tackling articles at this formative stage can be fun!
Gain exposure to a diversity of perspectives
Provide and receive immediate and more detailed feedback
Strengthen medical education communities
Create bonds among educators and unmask shared interests
Uncover underlying personal assumptions and biases
Provide ongoing asynchronous discussions in virtual communities
Produce a more rigorous and comprehensive final review product
Provide mentorship to junior colleagues and trainees
Enrich a formal instructional program
Have fun discussions with your colleagues!
For faculty and trainees involved in formal educational programs,2,10 group peer review can be an avenue for mentorship and a valuable instructional approach. By challenging junior colleagues to read and critique medical education scholarship, mentors can assess and address their mentees' knowledge gaps. Experienced reviewers can broaden junior reviewers' perspectives by offering feedback specific to the article itself (such as relevant background literature, methods, or interpretations of the data), and impart practical tips for writing efficiently and constructively (box 3). These tips may include providing comments with page and line numbers for easy reference; suggesting additional relevant citations; providing guidance for clearer data presentation aligned with journal guidelines; and maintaining a file of “review phrases” for responses to common manuscript problems. Experienced reviewers can share their reviewer phrases and model how to provide constructive feedback using nonjudgmental language that conveys an implicit commitment to help authors strengthen their work.3,5,11 In our experience, high-quality peer-review comments require a nuanced language that often doesn't come naturally to junior faculty—or even to some senior faculty.
Provide a brief 1- to 2-sentence summary of your general impressions
Highlight your major concerns
Number each section of feedback so authors can refer to them when they respond
Keep your feedback concise: sentences, not paragraphs
Be specific: list specific page and line numbers or quote prose from the manuscript
Maintain a collegial but objective tone; avoid sounding harsh or judgmental
Include references and citations to support your critiques when relevant
Keep a file of common reviewer phrases, references, or explanations
Use the “Comments to the Editor” section for overall impressions of the article, particularly regarding acceptance or rejection
Avoid telling authors in the “Comments to Author” section that the paper should be accepted or rejected, as this is an editorial decision
Respond quickly to editorial requests to review; use the “now or never” approach
Note: Several suggestions in this list have been adapted from DeMaria.3
We recognize that finding a local peer-review mentor may not be feasible for all individuals. As is the case with other journals,12 JGME editors may be able to identify more experienced reviewers to assist novice reviewers seeking mentorship. Developing virtual communities that support “coached peer review,” similar to processes described for the review of online educational content,13 offers another promising solution for training the next generation of reviewers.
Peer review requires practice and feedback. While many individuals develop these skills by regularly performing—and seeking feedback on—their own reviews, reconceptualizing peer review as a community of practice offers rich possibilities for coaching, bridge building, and strengthening educational communities.14
We invite your thoughts regarding how we might create reviewer communities within graduate medical education sponsoring organizations and programs. If you are interested, please e-mail us (email@example.com) a description of what you would like to do (along with the participants and areas of interest) and an editor will work with your community to identify an appropriate article for group review.
Finally, many thanks to the past, current, and future reviewers of JGME, particularly because we know that this work may come at the expense of your Sunday brunch!