Group Peer Review: The Breakfast of Champions

In the traditional paradigm, peer review occurs as a solitary activity in which the reviewer judges the manuscript's potential contribution to the field and provides guidance to strengthen its value. For individuals seeking to build their medical education research skills and reputation, this traditional paradigm offers numerous opportunities for professional development (box 1). Examining submitted manuscripts in their prepublication form exposes reviewers to a variety of approaches to how authors' frame questions, construct methods, present data analysis, interpret results, and build cogent discussions. Moreover, many journals, including JGME, carbon copy reviewers when decision letters are e-mailed to authors; reviewers then can use these letters to check their impressions against those of their peers, evaluate how their critiques align with editorial feedback, and follow the progress of articles toward publication. Because most journals rank reviewers' performance on each review,³  motivated individuals can solicit feedback directly from editors.

While peer review offers an efficient way to advance medical education scholarship while gaining feedback on research and writing skills, conducting peer review as a solitary activity has several drawbacks. First, reviewing requires a diverse background of knowledge and skills, and new reviewers may feel ill prepared to tackle this work independently. There are journal-specific, online, and academic society resources that provide preliminary guidance for performing peer review.^3–7  However, these contain general advice and may address neither the unique challenges of each manuscript nor the idiosyncratic knowledge deficits of each reviewer. Second, seeing the blind carbon copy of reviewer and editor feedback for a manuscript reviewed 2 weeks prior requires the reviewer to take the time to comb through comments to identify the differences between his or her review and those of other reviewers and the editor(s). This feedback may not yield the level of specific detail needed for individual reviewer improvement. Reviewers can seek specific editorial feedback regarding the quality of their reviews, but this entails additional effort, and feedback is not always provided. Although an invitation to review another article implies some degree of positive feedback, reviewers still lack specific guidance for how they might strengthen future reviews.

If we view the acquisition of peer-review skills through the lens of deliberate practice,⁸  we find that reviewers seeking to identify and remediate their knowledge gaps require opportunities for frequent repetition with timely and specific feedback. Lacking this structure, the individual-driven peer-review paradigm may not consistently produce high-quality reviews, particularly for new reviewers. How might this change if we viewed peer review as an opportunity to engage local or virtual communities of educators in a shared scholarly activity?

Most medical educators participate in various forms of a “journal club” as a means of reviewing emerging literature for potential translation into practice. A similar format can be adapted as a form of “group peer review” with individuals who are geographically or virtually connected, as has been modeled by the JACC Heart Failure Fellows Program review of published articles.²  Led by an individual who has been approached by a journal to review an article, group members independently review the article before meeting to discuss and debate their ratings. The group leader then submits the consolidated review to the journal. These review groups can be structured in a hierarchical fashion, with novice reviewers tackling the peer review before sending their impressions to experienced mentors, who subsequently provide feedback and additional insights.

Several JGME editors have employed this model successfully within postgraduate training or faculty development programs. Alternatively, review groups can function as a shared experience among peers. While reviewers should check with other journals whether such a group review format is acceptable, JGME supports this practice and requires no advance permission. We require that the same standards of confidentiality apply to these group reviews, and that reviewers cite the members of their review team when a review is submitted.

A group peer-review format offers benefits to the individual, community, and journal (box 2). Formally discussing a manuscript review exposes individuals to a diversity of perspectives and provides opportunities to give and receive immediate feedback. Furthermore, it helps to strengthen the identity of a medical education community, forges bonds among educators, unmasks shared interests, exposes underlying assumptions and biases, and highlights how issues addressed in scholarly articles might inform similar challenges in local contexts. Analogous to how traditional journal clubs have piloted virtual formats,⁹  group peer-review structures allow individuals in different locations to connect and discuss articles confidentially within closed virtual communities. Group peer reviews may generate feedback that is more diverse and nuanced than the opinions of an individual, which ultimately strengthens the rigor of the final review. Most importantly, tackling articles at this formative stage can be fun!

For faculty and trainees involved in formal educational programs,^2,10  group peer review can be an avenue for mentorship and a valuable instructional approach. By challenging junior colleagues to read and critique medical education scholarship, mentors can assess and address their mentees' knowledge gaps. Experienced reviewers can broaden junior reviewers' perspectives by offering feedback specific to the article itself (such as relevant background literature, methods, or interpretations of the data), and impart practical tips for writing efficiently and constructively (box 3). These tips may include providing comments with page and line numbers for easy reference; suggesting additional relevant citations; providing guidance for clearer data presentation aligned with journal guidelines; and maintaining a file of “review phrases” for responses to common manuscript problems. Experienced reviewers can share their reviewer phrases and model how to provide constructive feedback using nonjudgmental language that conveys an implicit commitment to help authors strengthen their work.^3,5,11  In our experience, high-quality peer-review comments require a nuanced language that often doesn't come naturally to junior faculty—or even to some senior faculty.

We recognize that finding a local peer-review mentor may not be feasible for all individuals. As is the case with other journals,¹²  JGME editors may be able to identify more experienced reviewers to assist novice reviewers seeking mentorship. Developing virtual communities that support “coached peer review,” similar to processes described for the review of online educational content,¹³  offers another promising solution for training the next generation of reviewers.

Peer review requires practice and feedback. While many individuals develop these skills by regularly performing—and seeking feedback on—their own reviews, reconceptualizing peer review as a community of practice offers rich possibilities for coaching, bridge building, and strengthening educational communities.¹⁴ 

We invite your thoughts regarding how we might create reviewer communities within graduate medical education sponsoring organizations and programs. If you are interested, please e-mail us ([email protected]) a description of what you would like to do (along with the participants and areas of interest) and an editor will work with your community to identify an appropriate article for group review.

Finally, many thanks to the past, current, and future reviewers of JGME, particularly because we know that this work may come at the expense of your Sunday brunch!