ABSTRACT

 Internal medicine residents care for a sizable number of patients with chronic pain. Programs need educational strategies to promote safe opioid prescribing.Background

 To describe a safe opioid prescribing education program utilizing an objective structured clinical examination (OSCE) and report the resulting impact on residents' knowledge, confidence, and self-reported practices.Objective

 Using a quasi-experimental design, 39 internal medicine residents from an urban academic medical center were assigned to 1 of 4 groups: 1-hour lecture only, lecture followed by immediate OSCE, lecture followed by 4-month delayed OSCE, and control. Safe opioid prescribing knowledge, confidence, and self-reported practices were assessed at baseline and at 8 months.Methods

 At 8 months, knowledge, confidence, and self-reported practices improved in the control and in all 3 intervention groups. The immediate OSCE group had the greatest improvements in combined confidence scores within group (0.74, P = .01) compared to controls (0.52, P = .05), using a 5-point scale. This group also had the greatest improvement in self-reported practice changes (1.04, P = .04), while other groups showed nonsignificant improvements—delayed OSCE (0.43, P = .44), lecture only (0.66, P = .24), and control (0.43, P = .19).Results

 Safe opioid prescribing education that includes a lecture immediately followed by an OSCE had an impact on residents' confidence and self-reported practices greater than those for delayed OSCE or lecture only groups. There was no difference in knowledge improvement among the groups. Lecture followed by an OSCE was highly regarded by residents, but required additional resources.Conclusions

What was known and gap

Internal medicine residents care for patients with chronic pain, yet they report low confidence and self-efficacy in safe opioid prescribing.

What is new

Residents were assigned to a control and 3 intervention groups for opioid prescribing education, with assessment at baseline and 8 months later.

Limitations

Single site, single specialty study; small sample; and lack of randomization may limit generalizability.

Bottom line

A lecture followed by an objective structured clinical examination had the largest impact on resident confidence and self-reported practices, and is well accepted, but requires added resources.

Introduction

Chronic pain is one of the most common reasons patients seek medical care.1,2  During the past 2 decades, more aggressive chronic pain management with opioid analgesics3  has been associated with an increase in prescription opioid misuse.4  There are numerous safe opioid prescribing guidelines;5  however, adherence with these guidelines remains low.68 

Physicians struggle to balance benefits and harms of prescription opioids,9  which is exacerbated by inadequate education.10  Medical trainees report low levels of confidence managing chronic pain.11,12  Safe opioid prescribing education can improve residents' attitudes,13  knowledge,14,15  and confidence.14  Residents prefer skills-based opioid prescribing education.14 

To improve residents' opioid prescribing practices, we developed a skills-based educational program that included a lecture followed by an objective structured clinical examination (OSCE), allowing residents to practice skills in a realistic setting. OSCEs utilize principles of cognitive apprenticeship16  through coaching, feedback, and allowing learners to reflect on their skills.17  They have been used to assess trainees' pain and addiction management practices,18,19  yet we are not aware of prior reports of using OSCEs to teach safe opioid prescribing skills.

Methods

Setting and Participants

The setting was an urban, academic, hospital-based internal medicine residency program with approximately 150 residents. Prescription opioid monitoring tools (eg, agreements, urine drug testing [UDT]) were available to residents but were not widely used.8 

Intervention

Utilizing a quasi-experimental design, residents were assigned to either a control or 1 of 3 intervention subgroups: 1-hour lecture with immediate OSCE, lecture with 4-month delayed OSCE, or lecture only (figure 1). A United States National Institute on Drug Abuse grant covered the costs associated with program development and implementation.

Educational Program Description

Lecture: The 1-hour lecture developed by experts (D.P.A., A.H.J.) in pain, addiction, and education was guideline-based and covered the assessment of pain and opioid misuse risk, the monitoring of benefits and harms, and the modifying of treatment plans when appropriate.

Objective Structured Clinical Examination: 

Two OSCE sessions were completed postlecture (immediate or a delay of 4 months). The total time for OSCE administration was 4 hours, 30 minutes. Each session had three 20-minute stations (table 1), which included (1) the resident reading the case summary and specific tasks (2 minutes); (2) interviewing the standardized patient (SP; 10 minutes); (3) verbal self-assessment (1 minute); (4) SP feedback (1 minute); (5) faculty feedback (5 minutes); and (6) poststation evaluation (1 minute). The OSCE materials are available as online supplemental material.

table 1

Objective Structured Clinical Examination Station Patient Profile and Clinical Tasks

Objective Structured Clinical Examination Station Patient Profile and Clinical Tasks
Objective Structured Clinical Examination Station Patient Profile and Clinical Tasks

Faculty Observers: 

All 5 faculty members proctoring the OSCE were experienced in primary care, safe opioid prescribing, and medical education. Faculty attended a 1-hour, 30-minute orientation to review station-specific tasks and logistics. During OSCE sessions, faculty observed residents and gave feedback based on an assessment tool that included communication skills and station-specific tasks. Faculty were paid $500 for 6 hours of participation during nonclinical protected time.

Standardized Patients: 

The SPs were actors who were given detailed patient roles and attended a 1-hour, 30-minute orientation with faculty, which included general approaches to the interview and how to give constructive feedback. Three SPs were paid $240 each for 6 hours of participation.

Resident Learners: 

Residents in the OSCE groups attended the 1-hour lecture, a 30-minute OSCE orientation, and all 3 OSCE stations. Three residents completed the 3 OSCE stations per hour. After residents completed the follow-up surveys, they were remunerated with a $50 gift card.

Outcomes

Resident Surveys: Residents completed baseline (prelecture) and 4- and 8-month (postlecture) follow-up surveys (figure 1), which assessed knowledge, confidence, and self-reported practices. The surveys were developed by experts. To allow for maximum time between the intervention and follow-up, we report 8-month outcomes. We used the 4-month survey to identify residents who were not opioid prescribers and focused our 8-month practice change survey on residents who prescribed opioids. Residents in the OSCE groups evaluated the OSCE.

The assessments included:

  • Knowledge: 4 multiple-choice questions on opioid efficacy, misuse risk factors, and management of aberrant medication-taking behaviors. Score ranged from 0 to 4.

  • Confidence: 8 items utilizing 5-point scales (1, not at all confident, to 5, very confident) in specific safe opioid prescribing practices (table 2). The confidence score was calculated as the average of the 8 items.

  • Practice: 3 items utilizing 5-point scales (1, never/rarely, to 5, always) in frequency of self-reported practices, calculated as an average (table 3).

table 2

Change in Safe Opioid Prescribing Confidence

Change in Safe Opioid Prescribing Confidence
Change in Safe Opioid Prescribing Confidence
table 3

Change in Safe Opioid Prescribing Self-Reported Practices

Change in Safe Opioid Prescribing Self-Reported Practices
Change in Safe Opioid Prescribing Self-Reported Practices

The Boston University Medical Campus Institutional Review Board determined this evaluation to be exempt from further review. Residents signed a consent outlining the voluntary nature of this study.

Analysis

The internal consistency of the 8-item summary confidence and 3-item summary practice scales at baseline was described using Cronbach's α. Exploratory analyses with the 4 groups examined changes from baseline to 8-month follow-up on the knowledge score, individual confidence and practice items, and summary confidence and summary practice scales using a paired sample t test. Changes with 2-tailed P < .05 are reported as significant. Given our sample size, these within-group comparisons are not adjusted for multiple comparisons and should be interpreted with caution.

Primary analyses examined differences across the 4 groups on mean changes in confidence and practice scales from baseline to 8-month follow-up; they were made through 1-factor analysis of variance (ANOVA), followed by Tukey's pairwise comparison, which accounts for multiple comparisons to identify specific differences if the overall ANOVA found significant differences. We reported effect sizes using Cohen's d to describe the magnitude of observed differences, calculated as the difference in the mean change score for an intervention group versus the control group, divided by the pooled standard deviation of the change score from the 4 groups. Given the small sample, we focused on effect sizes and statistical significance in reporting differences among study groups.

Results

This study included 39 internal medicine residents assigned to 4 groups: lecture with immediate OSCE (L+iO, n = 9), lecture with 4-month delayed OSCE (L+dO, n = 8), lecture only (L, n = 12), or control (C, n = 10). All residents completed all surveys and were similar at baseline (provided as online supplemental material) for mean number of patients managed with chronic pain (9.1, SD = 7.00) and on opioid analgesics (3.9, SD = 3.53). They had similar prior opioid prescribing training (1, none; 2, some; and 3, a lot; mean = 1.68; SD = 0.47) and opioid prescribing confidence (1, not at all, to 5, very; mean = 2.5; SD = 0.72).

Knowledge

All 4 groups showed an increase in knowledge at 8 months (mean [SD] score increased from 2.8 [1.1] to 3.2 [0.7], P < .01), with no significant difference across the 4 groups (P = .74).

Confidence

The 8 confidence items had strong reliability (Cronbach's α = 0.86). Within-group analysis showed significant improvement at 8 months in the summary confidence score for the L+iO group only. The L+iO group had significant increases on 4 of 8 individual confidence items (interpreting UDT, discussing unexpected UDT, discussing aberrant medication taking behaviors, dealing with patients' possible anger; table 2). The L+dO and L groups each showed a significant increase only on a single item (interpreting UDT and discussing opioid risks/benefits, respectively), while there were no significant increases on any items for the control group. ANOVA found no significant difference in change in confidence across the 4 study groups (P = .36). Based on Cohen's d, there were moderate increases in confidence for L+iO and L+dO groups relative to controls and a small increase in confidence for the L group relative to control.

Although there were some significant improvements in confidence within intervention groups, no significant changes were observed between groups. However, further analysis (Cohen's d) suggests meaningful improvements for both OSCE groups compared to the control.

Self-Reported Practices

The 3 practice items had strong reliability (Cronbach's α = 0.80). We excluded 5 participants from this practice outcome (1 each from L+iO, L, and C groups and 2 from the L+dO group) as they indicated no opioid prescribing in the months prior to both 4- and 8-month assessments and were, therefore, unable to make opioid prescribing changes. In the remaining sample (n = 34), only the L+iO group showed significant improvement on the summary practice score at 8 months (figure 2), with a significant improvement on 1 item (conducting pill counts; table 3). ANOVA found no significant difference in improvement across the 4 groups (P = .54). Based on Cohen's d, there was a moderate increase relative to controls on the summary practice score for the L+iO group.

While there was significant improvement in self-reported practice within the immediate OSCE group, there were no significant changes in self-reported practice between groups. However, further analysis (Cohen's d) suggests meaningful improvement for the immediate OSCE group compared to the control.

OSCE Evaluation

All participants reported the OSCE helped identify strengths and weaknesses, and 94% (16 of 17) reported the OSCE taught them something new and stimulated further learning.

figure 1

Objective Structured Clinical Examination (OSCE) Study Design

Abbreviations: L+iO, 60-minute lecture followed by immediate OSCE; L+dO, 60-minute lecture followed by delayed OSCE 4 months later; L, 60-minute lecture only; C, control, no lecture, no OSCE.

figure 1

Objective Structured Clinical Examination (OSCE) Study Design

Abbreviations: L+iO, 60-minute lecture followed by immediate OSCE; L+dO, 60-minute lecture followed by delayed OSCE 4 months later; L, 60-minute lecture only; C, control, no lecture, no OSCE.

figure 2

Combined Within-Group Change From Baseline to 8 Months in 3 Safe Opioid Prescribing Self-Reported Practices (n = 34)

Abbreviations: L+iO, 60-minute lecture followed by immediate OSCE; L+dO, 60-minute lecture followed by delayed OSCE 4 months later; L, 60-minute lecture only; C, control, no lecture, no OSCE; OSCE, objective, structured clinical examination.

aP < .05.

figure 2

Combined Within-Group Change From Baseline to 8 Months in 3 Safe Opioid Prescribing Self-Reported Practices (n = 34)

Abbreviations: L+iO, 60-minute lecture followed by immediate OSCE; L+dO, 60-minute lecture followed by delayed OSCE 4 months later; L, 60-minute lecture only; C, control, no lecture, no OSCE; OSCE, objective, structured clinical examination.

aP < .05.

Discussion

A safe opioid prescribing educational program that includes a lecture followed by an OSCE may improve internal medicine residents' safe opioid prescribing confidence and self-reported practices. Having the OSCE immediately follow the lecture appeared to have the greatest impact.

This improvement can be explained by learners having an opportunity to practice challenging skills and receive feedback allowing for rapid integration, refinement, and solidification of new skills. Previous resident safe opioid prescribing education using role playing has demonstrated the importance of skills practice.14  Trainees respond favorably to using SPs,2022  which are more realistic but are a logistically challenging skills practice method.20,23  Our program findings are similar to a program that used OSCEs to teach addiction management skills.19,24 

While this grant-funded program was highly regarded by both residents and residency program leadership, it was not sustained after grant support ended. Understanding the specific and unique challenges internal medicine residents face when implementing new safe opioid prescribing practices helps put our results in context. Opioid prescribing guidelines, based on expert opinion5,2528  rather than scientific evidence, result in wide variation in clinical practice and inconsistent faculty preceptors' guidance to residents. Moreover, faculty preceptors may disregard residents' new opioid prescribing practices if they differ from their own. Finally, residents often inherit patients already on opioids and may feel pressured to continue the treatment plan.29  Despite these challenges, our educational program improved residents' self-reported safe opioid prescribing practices.

Our study has several limitations. The nonrandom distribution of residents across groups could lead to confounding, since the L+iO and L+dO groups were more likely to favor primary care careers. The small sample makes it difficult to identify significant differences among groups. Self-reported data may have introduced a social desirability bias, and the follow-up times after OSCE completion varied between the 2 OSCE groups. It is unclear whether more or less time before follow-up would influence the likelihood of change. Our study did not assess improvements in patient-level outcomes. Although the survey instruments were developed by content experts, they did not undergo validity testing.

Future research might include a more robust practice change and patient-level outcomes, including chart reviews and/or patient interviews. Due to the importance of faculty preceptors' role on resident practices, future safe opioid prescribing interventions should include faculty development and/or faculty and resident co-training.

Conclusion

A safe opioid prescribing educational intervention that included a lecture followed by immediate skills practice using an OSCE appeared to improve residents' confidence and self-reported practices and was highly regarded by resident learners.

References

References
1
Institute of Medicine Committee on Advancing Pain Research C, Education
.
The National Academies Collection: Reports funded by National Institutes of Health
.
Relieving Pain in America: A Blueprint for Transforming Prevention, Care, Education, and Research
.
Washington, DC
:
National Academies Press
;
2011
.
2
Gureje
O,
Von Korff
M,
Simon
GE,
Gater
R.
Persistent pain and well-being: a World Health Organization Study in primary care
.
JAMA
.
1998
;
280
(
2
):
147
151
.
3
Daubresse
M,
Chang
HY,
Yu
Y,
Viswanathan
S,
Shah
ND,
Stafford
RS,
et al.
Ambulatory diagnosis and treatment of nonmalignant pain in the United States, 2000–2010
.
Med Care
.
2013
;
51
(
10
):
870
878
.
4
Dart
RC,
Surratt
HL,
Cicero
TJ,
Parrino
MW,
Severtson
SG,
Bucher-Bartelson
B,
et al.
Trends in opioid analgesic abuse and mortality in the United States
.
N Engl J Med
.
2015
;
372
(
3
):
241
248
.
5
Nuckols
TK,
Anderson
L,
Popescu
I,
Diamant
AL,
Doyle
B,
Di Capua
P,
et al.
Opioid prescribing: a systematic review and critical appraisal of guidelines for chronic pain
.
Ann Intern Med
.
2014
;
160
(
1
):
38
47
.
6
Starrels
JL,
Becker
WC,
Weiner
MG,
Li
X,
Heo
M,
Turner
BJ.
Low use of opioid risk reduction strategies in primary care even for high risk patients with chronic pain
.
J Gen Intern Med
.
2011
;
26
(
9
):
958
964
.
7
Sekhon
R,
Aminjavahery
N,
Davis
CN
Jr,
Roswarski
MJ,
Robinette
C.
Compliance with opioid treatment guidelines for chronic non-cancer pain (CNCP) in primary care at a Veterans Affairs Medical Center (VAMC)
.
Pain Med
.
2013
;
14
(
10
):
1548
1556
.
8
Khalid
L,
Liebschutz
JM,
Xuan
Z,
Dossabhoy
S,
Kim
Y,
Crooks
D,
et al.
Adherence to prescription opioid monitoring guidelines among residents and attending physicians in the primary care setting
.
Pain Med
.
2015
;
16
(
3
):
480
487
.
9
Upshur
CC,
Luckmann
RS,
Savageau
JA.
Primary care provider concerns about management of chronic pain in community clinic populations
.
J Gen Intern Med
.
2006
;
21
(
6
):
652
655
.
10
Mezei
L,
Murinson
BB;
Johns Hopkins Curriculum Development Team. Pain education in North American medical schools
.
J Pain
.
2011
;
12
(
12
):
1199
1208
.
11
Chen
JT,
Fagan
MJ,
Diaz
JA,
Reinert
SE.
Is treating chronic pain torture? Internal medicine residents' experience with patients with chronic nonmalignant pain
.
Teach Learn Med
.
2007
;
19
(
2
):
101
105
.
12
Yanni
LM,
Weaver
MF,
Johnson
BA,
Morgan
LA,
Harrington
SE,
Ketchum
JM.
Management of chronic nonmalignant pain: a needs assessment in an internal medicine resident continuity clinic
.
J Opioid Manag
.
2008
;
4
(
4
):
201
211
.
13
Roth
CS,
Burgess
DJ.
Changing residents' beliefs and concerns about treating chronic noncancer pain with opioids: evaluation of a pilot workshop
.
Pain Med
.
2008
;
9
(
7
):
890
902
.
14
Elhwairis
H,
Reznich
CB.
An educational strategy for treating chronic, noncancer pain with opioids: a pilot test
.
J Pain
.
2010
;
11
(
12
):
1368
1375
.
15
Smith
CD.
A curriculum to address family medicine residents' skills in treating patients with chronic pain
.
Int J Psychiatry Med
.
2014
;
47
(
4
):
327
336
.
16
Stalmeijer
RE,
Dolmans
DH,
Snellen-Balendong
HA,
van Santen-Hoeufft
M,
Wolfhagen
IH,
Scherpbier
AJ.
Clinical teaching based on principles of cognitive apprenticeship: views of experienced clinical teachers
.
Acad Med
.
2013
;
88
(
6
):
861
865
.
17
Stalmeijer
RE.
When I say . . . cognitive apprenticeship
.
Med Educ
.
2015
;
49
(
4
):
355
356
.
18
Weiner
DK,
Morone
NE,
Spallek
H,
Karp
JF,
Schneider
M,
Washburn
C,
et al.
E-learning module on chronic low back pain in older adults: evidence of effect on medical student objective structured clinical examination performance
.
J Am Geriatr Soc
.
2014
;
62
(
6
):
1161
1167
.
19
Parish
SJ,
Ramaswamy
M,
Stein
MR,
Kachur
EK,
Arnsten
JH.
Teaching about substance abuse with objective structured clinical exams
.
J Gen Intern Med
.
2006
;
21
(
5
):
453
459
.
20
Lane
C,
Rollnick
S.
The use of simulated patients and role-play in communication skills training: a review of the literature to August 2005
.
Patient Educ Couns
.
2007
;
67
(
1–2
):
13
20
.
21
Eagles
JM,
Calder
SA,
Nicoll
KS,
Walker
LG.
A comparison of real patients, simulated patients and videotaped interview in teaching medical students about alcohol misuse
.
Med Teach
.
2001
;
23
(
5
):
490
493
.
22
Koponen
J,
Pyörälä
E,
Isotalus
P.
Comparing three experiential learning methods and their effect on medical students' attitudes to learning communication skills
.
Med Teach
.
2012
;
34
(
3
):
e198
e207
.
23
Fletcher
JD,
Wind
AP.
Cost considerations in using simulations for medical training
.
Mil Med
.
2013
;
178
(
suppl 10
):
37
46
.
24
Parish
SJ,
Stein
MR,
Hahn
SR,
Goldberg
U,
Arnsten
JH.
Teaching and assessing residents' skills in managing heroin addiction with objective structured clinical examinations (OSCEs)
.
Subst Abus
.
2013
;
34
(
4
):
350
355
.
25
Federation of State Medical Boards
.
Model policy on the use of opioid analgesics in the treatment of chronic pain July 2013
. .
26
Chou
R,
Fanciullo
GJ,
Fine
PG,
Adler
JA,
Ballantyne
JC,
Davies
P,
et al.
Clinical guidelines for the use of chronic opioid therapy in chronic noncancer pain
.
J Pain
.
2009
;
10
(
2
):
113
130
.
27
Franklin
GM;
American Academy of Neurology. Opioids for chronic noncancer pain: a position paper of the American Academy of Neurology
.
Neurology
.
2014
;
83
(
14
):
1277
1284
.
28
Manchikanti
L,
Abdi
S,
Atluri
S,
Balog
CC,
Benyamin
RR,
Boswell
MV,
et al.
American Society of Interventional Pain Physicians (ASIPP) guidelines for responsible opioid prescribing in chronic non-cancer pain: part 2—guidance
.
Pain Physician
.
2012
;
15
(
suppl 3
):
67
116
.
29
Gourlay
DL,
Heit
HA.
Universal precautions revisited: managing the inherited pain patient
.
Pain Med
.
2009
;
10
(
suppl 2
):
115
123
.

Author notes

Funding: Program support was provided by the United States National Institute on Drug Abuse (N01DA-1142).

Competing Interests

Conflict of interest: The authors declare they have no competing interests.

The authors would like to thank Timothy Heeren, PhD, for his statistical guidance in revising this manuscript.

Editor's Note: The online version of this article contains information for OSCE implementation, including educational objectives and procedures, station-specific resident instructions and clinical tasks, and faculty, standardized patient, and resident assessment tools, and a table of baseline participant characteristics.

Supplementary data