Background 

Promoting resident scholarship is important to programs. Positive Peer-Pressured Productivity (P-QUAD) is a dual incentive model that combines increased transparency through awareness of peers' engagement in scholarship, with a weighted cash lottery where tickets are earned for various dimensions of academic success (ie, 1 point/ticket for an abstract submission up to 6 for manuscript acceptance).

Objective 

We explored whether a weighted lottery system contributes to sustained increases in academic productivity in a residency program.

Methods 

We implemented P-QUAD in 1 pediatrics residency program in July 2015. Residents reported their scholarship submissions/acceptances for the prior year, establishing a program baseline. During the 2-year intervention, residents logged their academic submissions/acceptances on a web interface where they could view real-time scores and the work of their peers. At the end of each academic year, we compared P-QUAD points for each category to baseline.

Results 

During the intervention, 31% of residents (68 of 218) reported engaging in scholarship. Using P-QUAD was acceptable to most residents. Engagement in scholarship across the program, as measured by total P-QUAD score, increased 53% from baseline (329 versus 504 points per year). Mean submission and acceptance rates for individual residents reporting research through P-QUAD increased across all categories, ranging from 19% for abstract submissions (1.62 to 1.93 per year) to 275% (0.24 to 0.90 per year) for accepted manuscripts.

Conclusions 

The residency program sustained gains in academic productivity at the program-wide and participating resident level in the 2 years since implementing P-QUAD.

Building skills and habits in scholarship during residency is a critical foundation for physicians, including those who do not pursue careers as researchers.1  Although many programs have infrastructure in place to promote scholarly activity, creating a flourishing culture of resident scholarship can be challenging.2,3 

Programs have aimed to increase scholarly productivity with various interventions, including protecting residents' time, creating dedicated research tracks or curricula, or assigning research mentors.38  These interventions often require substantial time commitments and resources. We implemented a novel approach to promote resident scholarship that would not require a new curriculum or changes in resident schedules.

We describe the implementation and outcomes of a novel dual incentive program called Positive Peer-Pressured Productivity (P-QUAD). The intervention sought to leverage positive peer pressure associated with increased awareness of colleagues' scholarly activity and used a lottery-based incentive where more productivity leads to more chances to win cash prizes.9 

Intervention

We previously developed P-QUAD for faculty as a peer-mentoring model with financial incentives, where points are earned based on degrees of academic productivity. Point values include 1 point for an abstract submission, 2 points for a poster acceptance or rejection or manuscript resubmission, 3 points for a platform or workshop acceptance, 4 points for a manuscript submission, and 6 points for a manuscript acceptance. These points translate into tickets for a semiannual raffle, with a greater chance of winning based on an individual's degree of productivity. During the faculty pilot, we saw a statistically significant doubling of abstract submissions.9 

As we brought P-QUAD to residents, we wanted them to be able to see a leaderboard of submissions in real time, both to promote their work and to leverage the gamification aspect where the positive peer pressure of friendly competition can encourage outcomes.1013  We created a web-based platform (www.pquad.org), where residents could voluntarily log their submissions, see their P-QUAD score, and search through peers' submissions. We used a free web-based survey software (Google Forms, Google Inc, Mountain View, CA) for residents to log their P-QUAD points and ultimately contracted with a web designer to create a more polished web interface for P-QUAD logging and leaderboard display. We promoted its use at residency meetings and via e-mail (figure 1). We held cash drawings every 6 months for $750 in prizes. The only direct cost associated with P-QUAD was the cash prizes, which came from philanthropic funds already earmarked for supporting resident research.

Figure 1

Sample of Explanatory Flyer to Introduce and Promote Use of P-QUAD

Figure 1

Sample of Explanatory Flyer to Introduce and Promote Use of P-QUAD

Close modal

At the completion of the intervention, we sent residents an anonymous electronic survey for program evaluation. The survey was developed by the authors and did not undergo validity testing. We used Likert scale questions to determine the degree to which residents found the various aspects of the program to be motivating and/or stressful as well as multiple-choice questions to ask residents if they anticipated logging future scholarship achievements via P-QUAD or if they had connected with peers about scholarship because of what they had seen.

The study received Institutional Review Board approval from the University of Minnesota.

Analysis

We implemented P-QUAD in our pediatrics residency program on July 1, 2015, and collected data through June 30, 2017. Residents' scholarship submissions and acceptances for the prior academic year provided the baseline data. At the end of each academic year, we compared mean P-QUAD points reported for each category, comparing groups of residents at the same level of training as well as the overall residency program.

Engagement in scholarship across the program, as measured by the total P-QUAD score for each year, increased 53% from baseline, from 329 points across the program preimplementation to 504 after the second year, with sustained increases in all categories except abstracts submitted.

In the preintervention year, 34% of residents (37 of 109) reported engaging in scholarly activity as measured by submissions and/or acceptances of academic work. During the collective 2-year intervention period, 31% of residents (68 of 218) reported engaging in scholarship. There were increases in mean scores from baseline across all categories, ranging from a 19% increase (1.62 to 1.93 per year) in abstract submissions per resident engaged in research to a 275% increase (0.24 to 0.90 per year) in accepted manuscripts per resident (figure 2).

Figure 2

Comparison of Average Annual Scholarship Contributions per Resident

Figure 2

Comparison of Average Annual Scholarship Contributions per Resident

Close modal

Fifty-four percent (59 of 110) of residents provided feedback via the survey. Most (78%, 46 of 59) agreed that P-QUAD increased their awareness of peers' scholarly work, with some (12%, 7 of 59) reporting that they reached out to a resident to collaborate specifically because of what they had seen logged; and 41% (24 of 59) agreed that P-QUAD motivated them to engage in scholarship, with (34%, 20 of 59) reporting that the increased transparency was motivating and 28% (16 of 59) indicating that the opportunity to win cash was a motivation for engagement. Some residents (25%, 15 of 59) indicated that P-QUAD produced stress due to the competition; more than twice that number disagreed with this sentiment (54%, 32 of 59). No respondents indicated that they were opposed to submitting their future work via P-QUAD.

Implementing a transparent, engaging platform that highlights resident scholarship was associated with increased scholarly productivity in our program. Although we did not see an overall increase in the number of residents who reported scholarly work, we did notice increased productivity for the residents who engaged with the lottery system, and for the program as a whole, suggesting that this approach may be most effective in motivating residents already inclined to participate in research. As previous work has shown a strong association between productivity during residency and plans to pursue research in a future career,14  furthering the productivity of these engaged residents during their training may be beneficial.

An important finding is the increase in residents submitting manuscripts. Fostering an environment that leads to residents climbing the scholarship ladder from abstracts to manuscripts is an important goal for all training programs. Previous work has demonstrated that requiring research within residency leads to an increase in manuscript publications.15  Our study demonstrates, however, that similar increases can be attained via a voluntary incentive-based system. Interestingly, abstract submissions were relatively stagnant. It is possible the increased emphasis on scholarship motivated some residents to advance their prior work to a full-fledged publication.

While many residents indicated that P-QUAD motivated them to engage in scholarship, more study is needed to determine if the explicit emphasis (via increased point values in the lottery) on manuscript submission was a motivator for residents or if engaging more frequently with scholarship naturally led them to increased productivity.

Our study has limitations. Our program served as its own control, and it is possible that scholarship among residents may be increasing across all programs. We recognize that some residents who engaged in scholarship may have chosen not to log their submissions. Future study will include tracking scholarly productivity over time for graduates who participated in P-QUAD compared with graduates prior to the intervention.

Implementation of a transparent weighted lottery system that reported and incentivized scholarly work was associated with increased scholarly productivity in our residency program. The greatest increase was seen in the category of submitted manuscripts.

1
Accreditation Council for Graduate Medical Education
.
Common Program Requirements
.
July 1, 2017.
.
2
Stevenson
MD,
Smigielski
EM,
Naifeh
MM,
et al.
Increasing scholarly activity productivity during residency: a systematic review
.
Acad Med
.
2017
;
92
(
2
):
250
266
.
3
Vinci
RJ,
Bauchner
H,
Finkelstein
J,
et al.
Research during pediatric residency training: outcome of a senior resident block rotation
.
Pediatrics
.
2009
;
124
(
4
):
1126
1134
.
4
Abramson
EL,
Naifeh
MM,
Stevenson
MD,
et al.
Research training among pediatric residency programs: a national assessment
.
Acad Med
.
2014
;
89
(
12
):
1674
1680
.
5
Kanna
B,
Deng
C,
Erickson
SN,
et al.
The research rotation: competency-based structured and novel approach to research training of internal medicine residents
.
BMC Med Educ
.
2006
;
6
(
1
):
52
.
6
Stevenson
MD,
Smigielski
EM,
Naifeh
MM,
et al.
Increasing scholarly activity productivity during residency
.
Acad Med
.
2017
;
92
(
2
):
250
266
.
7
Crawford
P,
Seehusen
D.
Scholarly activity in family medicine residency programs: a national survey
.
Fam Med
.
2011
;
43
(
5
):
311
317
.
8
West
CP,
Halvorsen
AJ,
McDonald
FS.
Scholarship during residency training: a controlled comparison study
.
Am J Med
.
2011
;
124
(
10
):
983.e1
987.e1
.
9
Pitt
MB,
Furnival
RA,
Zhang
L,
et al.
Positive Peer-Pressured Productivity (P-QUAD): novel use of increased transparency and a weighted lottery to increase a division's academic output
.
Acad Pediatr
.
2017
;
17
(
2
):
218
221
.
10
Pitt
MB,
Borman-Shoap
EC,
Eppich
WJ.
Twelve tips for maximizing the effectiveness of game-based learning
.
Med Teach
.
2015
;
37
(
11
):
1013
1017
.
11
Lindsley
JE,
Morton
DA,
Pippitt
K,
et al.
The two-stage examination: a method to assess individual competence and collaborative problem solving in medical students
.
Acad Med
.
2016
;
91
(
10
):
1384
1387
.
12
Fox
LA,
Walsh
KE,
Schainker
EG.
The creation of a pediatric hospital medicine dashboard: performance assessment for improvement
.
Hosp Pediatr
.
2016
;
6
(
7
):
412
419
.
13
Scoggins
CR,
Crockett
T,
Wafford
L,
et al.
Improving clinical productivity in an academic surgical practice through transparency
.
J Am Coll Surg
.
2013
;
217
(
1
):
46
51
.
14
Abramson
EL,
Naifeh
MM,
Stevenson
MD,
et al.
Scholarly activity training during residency: are we hitting the mark? A national assessment of pediatric residents
.
Acad Pediatr
.
2018
Feb
12
.
pii: S1876-2859(18)30072-X.
doi: . Epub ahead of print.
15
Geyer
BC,
Kaji
AH,
Katz
ED,
et al.
A national evaluation of the scholarly activity requirement in residency programs: a survey of emergency medicine program directors
.
Acad Emerg Med
.
2015
;
22
(
11
):
1337
1344
.

Author notes

Funding: Research reported in this article was supported by the National Center for Advancing Translational Sciences of the National Institutes of Health Award Number UL1TR000114.

Competing Interests

Conflict of interest: The authors declare they have no competing interests.

This work was presented as a poster at the Pediatric Academic Societies Meeting, San Francisco, California, May 6–9, 2017.

The authors would like to thank Abe Jacob, Ron Furnival, Ann Marie Weber-Main, and Nancy Raymond for their roles in the development and initial pilot of P-QUAD and Brian Krohn for his assistance in creating the web-based platform.

The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.