Improving Residents' Safe Opioid Prescribing for Chronic Pain Using an Objective Structured Clinical Examination

Chronic pain is one of the most common reasons patients seek medical care.^1,2  During the past 2 decades, more aggressive chronic pain management with opioid analgesics³  has been associated with an increase in prescription opioid misuse.⁴  There are numerous safe opioid prescribing guidelines;⁵  however, adherence with these guidelines remains low.^6–8 

Physicians struggle to balance benefits and harms of prescription opioids,⁹  which is exacerbated by inadequate education.¹⁰  Medical trainees report low levels of confidence managing chronic pain.^11,12  Safe opioid prescribing education can improve residents' attitudes,¹³  knowledge,^14,15  and confidence.¹⁴  Residents prefer skills-based opioid prescribing education.¹⁴ 

To improve residents' opioid prescribing practices, we developed a skills-based educational program that included a lecture followed by an objective structured clinical examination (OSCE), allowing residents to practice skills in a realistic setting. OSCEs utilize principles of cognitive apprenticeship¹⁶  through coaching, feedback, and allowing learners to reflect on their skills.¹⁷  They have been used to assess trainees' pain and addiction management practices,^18,19  yet we are not aware of prior reports of using OSCEs to teach safe opioid prescribing skills.

The setting was an urban, academic, hospital-based internal medicine residency program with approximately 150 residents. Prescription opioid monitoring tools (eg, agreements, urine drug testing [UDT]) were available to residents but were not widely used.⁸ 

Utilizing a quasi-experimental design, residents were assigned to either a control or 1 of 3 intervention subgroups: 1-hour lecture with immediate OSCE, lecture with 4-month delayed OSCE, or lecture only (figure 1). A United States National Institute on Drug Abuse grant covered the costs associated with program development and implementation.

Lecture: The 1-hour lecture developed by experts (D.P.A., A.H.J.) in pain, addiction, and education was guideline-based and covered the assessment of pain and opioid misuse risk, the monitoring of benefits and harms, and the modifying of treatment plans when appropriate.

Two OSCE sessions were completed postlecture (immediate or a delay of 4 months). The total time for OSCE administration was 4 hours, 30 minutes. Each session had three 20-minute stations (table 1), which included (1) the resident reading the case summary and specific tasks (2 minutes); (2) interviewing the standardized patient (SP; 10 minutes); (3) verbal self-assessment (1 minute); (4) SP feedback (1 minute); (5) faculty feedback (5 minutes); and (6) poststation evaluation (1 minute). The OSCE materials are available as online supplemental material.

All 5 faculty members proctoring the OSCE were experienced in primary care, safe opioid prescribing, and medical education. Faculty attended a 1-hour, 30-minute orientation to review station-specific tasks and logistics. During OSCE sessions, faculty observed residents and gave feedback based on an assessment tool that included communication skills and station-specific tasks. Faculty were paid $500 for 6 hours of participation during nonclinical protected time.

The SPs were actors who were given detailed patient roles and attended a 1-hour, 30-minute orientation with faculty, which included general approaches to the interview and how to give constructive feedback. Three SPs were paid $240 each for 6 hours of participation.

Residents in the OSCE groups attended the 1-hour lecture, a 30-minute OSCE orientation, and all 3 OSCE stations. Three residents completed the 3 OSCE stations per hour. After residents completed the follow-up surveys, they were remunerated with a $50 gift card.

Resident Surveys: Residents completed baseline (prelecture) and 4- and 8-month (postlecture) follow-up surveys (figure 1), which assessed knowledge, confidence, and self-reported practices. The surveys were developed by experts. To allow for maximum time between the intervention and follow-up, we report 8-month outcomes. We used the 4-month survey to identify residents who were not opioid prescribers and focused our 8-month practice change survey on residents who prescribed opioids. Residents in the OSCE groups evaluated the OSCE.

The assessments included:

• Knowledge: 4 multiple-choice questions on opioid efficacy, misuse risk factors, and management of aberrant medication-taking behaviors. Score ranged from 0 to 4.

• Confidence: 8 items utilizing 5-point scales (1, not at all confident, to 5, very confident) in specific safe opioid prescribing practices (table 2). The confidence score was calculated as the average of the 8 items.

• Practice: 3 items utilizing 5-point scales (1, never/rarely, to 5, always) in frequency of self-reported practices, calculated as an average (table 3).

The Boston University Medical Campus Institutional Review Board determined this evaluation to be exempt from further review. Residents signed a consent outlining the voluntary nature of this study.

The internal consistency of the 8-item summary confidence and 3-item summary practice scales at baseline was described using Cronbach's α. Exploratory analyses with the 4 groups examined changes from baseline to 8-month follow-up on the knowledge score, individual confidence and practice items, and summary confidence and summary practice scales using a paired sample t test. Changes with 2-tailed P < .05 are reported as significant. Given our sample size, these within-group comparisons are not adjusted for multiple comparisons and should be interpreted with caution.

Primary analyses examined differences across the 4 groups on mean changes in confidence and practice scales from baseline to 8-month follow-up; they were made through 1-factor analysis of variance (ANOVA), followed by Tukey's pairwise comparison, which accounts for multiple comparisons to identify specific differences if the overall ANOVA found significant differences. We reported effect sizes using Cohen's d to describe the magnitude of observed differences, calculated as the difference in the mean change score for an intervention group versus the control group, divided by the pooled standard deviation of the change score from the 4 groups. Given the small sample, we focused on effect sizes and statistical significance in reporting differences among study groups.

This study included 39 internal medicine residents assigned to 4 groups: lecture with immediate OSCE (L + iO, n = 9), lecture with 4-month delayed OSCE (L + dO, n = 8), lecture only (L, n = 12), or control (C, n = 10). All residents completed all surveys and were similar at baseline (provided as online supplemental material) for mean number of patients managed with chronic pain (9.1, SD = 7.00) and on opioid analgesics (3.9, SD = 3.53). They had similar prior opioid prescribing training (1, none; 2, some; and 3, a lot; mean = 1.68; SD = 0.47) and opioid prescribing confidence (1, not at all, to 5, very; mean = 2.5; SD = 0.72).

All 4 groups showed an increase in knowledge at 8 months (mean [SD] score increased from 2.8 [1.1] to 3.2 [0.7], P < .01), with no significant difference across the 4 groups (P = .74).

The 8 confidence items had strong reliability (Cronbach's α = 0.86). Within-group analysis showed significant improvement at 8 months in the summary confidence score for the L + iO group only. The L + iO group had significant increases on 4 of 8 individual confidence items (interpreting UDT, discussing unexpected UDT, discussing aberrant medication taking behaviors, dealing with patients' possible anger; table 2). The L + dO and L groups each showed a significant increase only on a single item (interpreting UDT and discussing opioid risks/benefits, respectively), while there were no significant increases on any items for the control group. ANOVA found no significant difference in change in confidence across the 4 study groups (P = .36). Based on Cohen's d, there were moderate increases in confidence for L + iO and L + dO groups relative to controls and a small increase in confidence for the L group relative to control.

Although there were some significant improvements in confidence within intervention groups, no significant changes were observed between groups. However, further analysis (Cohen's d) suggests meaningful improvements for both OSCE groups compared to the control.

The 3 practice items had strong reliability (Cronbach's α = 0.80). We excluded 5 participants from this practice outcome (1 each from L + iO, L, and C groups and 2 from the L + dO group) as they indicated no opioid prescribing in the months prior to both 4- and 8-month assessments and were, therefore, unable to make opioid prescribing changes. In the remaining sample (n = 34), only the L + iO group showed significant improvement on the summary practice score at 8 months (figure 2), with a significant improvement on 1 item (conducting pill counts; table 3). ANOVA found no significant difference in improvement across the 4 groups (P = .54). Based on Cohen's d, there was a moderate increase relative to controls on the summary practice score for the L + iO group.

While there was significant improvement in self-reported practice within the immediate OSCE group, there were no significant changes in self-reported practice between groups. However, further analysis (Cohen's d) suggests meaningful improvement for the immediate OSCE group compared to the control.

All participants reported the OSCE helped identify strengths and weaknesses, and 94% (16 of 17) reported the OSCE taught them something new and stimulated further learning.

A safe opioid prescribing educational program that includes a lecture followed by an OSCE may improve internal medicine residents' safe opioid prescribing confidence and self-reported practices. Having the OSCE immediately follow the lecture appeared to have the greatest impact.

This improvement can be explained by learners having an opportunity to practice challenging skills and receive feedback allowing for rapid integration, refinement, and solidification of new skills. Previous resident safe opioid prescribing education using role playing has demonstrated the importance of skills practice.¹⁴  Trainees respond favorably to using SPs,^20–22  which are more realistic but are a logistically challenging skills practice method.^20,23  Our program findings are similar to a program that used OSCEs to teach addiction management skills.^19,24 

While this grant-funded program was highly regarded by both residents and residency program leadership, it was not sustained after grant support ended. Understanding the specific and unique challenges internal medicine residents face when implementing new safe opioid prescribing practices helps put our results in context. Opioid prescribing guidelines, based on expert opinion^5,25–28  rather than scientific evidence, result in wide variation in clinical practice and inconsistent faculty preceptors' guidance to residents. Moreover, faculty preceptors may disregard residents' new opioid prescribing practices if they differ from their own. Finally, residents often inherit patients already on opioids and may feel pressured to continue the treatment plan.²⁹  Despite these challenges, our educational program improved residents' self-reported safe opioid prescribing practices.

Our study has several limitations. The nonrandom distribution of residents across groups could lead to confounding, since the L + iO and L + dO groups were more likely to favor primary care careers. The small sample makes it difficult to identify significant differences among groups. Self-reported data may have introduced a social desirability bias, and the follow-up times after OSCE completion varied between the 2 OSCE groups. It is unclear whether more or less time before follow-up would influence the likelihood of change. Our study did not assess improvements in patient-level outcomes. Although the survey instruments were developed by content experts, they did not undergo validity testing.

Future research might include a more robust practice change and patient-level outcomes, including chart reviews and/or patient interviews. Due to the importance of faculty preceptors' role on resident practices, future safe opioid prescribing interventions should include faculty development and/or faculty and resident co-training.

A safe opioid prescribing educational intervention that included a lecture followed by immediate skills practice using an OSCE appeared to improve residents' confidence and self-reported practices and was highly regarded by resident learners.