ABSTRACT

Background

Despite increasing use of telehealth, there are limited published curricula training primary care providers in utilizing telehealth to deliver complex interdisciplinary care.

Objective

To describe and evaluate a telehealth curriculum with a longitudinal objective structured clinical examination (OSCE) to improve internal medicine residents' confidence and skills in coordinating complex interdisciplinary primary care via televisits, electronic consultation, and teleconferencing.

Methods

In 2019, 56 first- and third-year residents participated in a 3-part, 5-week OSCE training them to use telehealth to manage complex primary care. Learners conducted a standardized patient (SP) televisit in session 1, coordinated care via inter-visit e-messaging, and led a simulated interdisciplinary teleconference in session 2. Surveys measured confidence before session 1 (pre), post-session 1 (post-1), and post-session 2 (post-2). SP televisit checklists and investigators' assessment of e-messages evaluated residents' telehealth skills.

Results

Response rates were pre 100%, post-1 95% (53 of 56), and post-2 100%. Post-intervention, more residents were “confident/very confident” in adjusting their camera (33%, 95% CI 20–45 vs 85%, 95% CI 75–95, P < .0001), e-messaging (pre 36%, 95% CI 24–49 vs post-2 80%, 95% CI 70–91, P < .0001), and coordinating interdisciplinary care (pre 35%, 95% CI 22–47 vs post-2 84%, 95% CI 74–94, P < .0001). More residents were “likely/very likely” to use telemedicine in the future (pre 56%, 95% CI 43–69, vs post-2 79%, 95% CI 68–89, P = .001).

Conclusions

A longitudinal, interdisciplinary telehealth simulation is feasible and can improve residents' confidence in using telemedicine to provide complex patient care.

What was known and gap

There are limited curricula for resident telehealth training that focuses on longitudinal, interdisciplinary primary care.

What is new

We report a longitudinal simulation to teach interdisciplinary telemedicine and e-communication skills.

Limitations

Single center study and resource-intensive intervention limit generalizability.

Bottom line

Interdisciplinary telehealth training is feasible and can improve residents' confidence in using telemedicine and e-communication to provide complex patient care.

Introduction

Telehealth utilization has increased in the United States, with over 60% of health care systems incorporating telemedicine technology,1  and this is likely to expand post-COVID-19. Telehealth is multifaceted and requires both synchronous and asynchronous skills in using technology to provide effective care.1  As telehealth usage expands, the American Medical Association has encouraged training in telemedicine for students and residents.2  However, curricula do not address how longitudinal and complex primary care can be delivered utilizing telehealth and electronic technologies.

In the literature on telehealth education, over a quarter of medical schools delivered preclinical telemedicine curricula, and almost half provided clinical exposure through routine practice or telehealth electives.3  Other specialties, such as dermatology,4,5  psychiatry,6  and neurology,7,8  have reported telemedicine training for residents and fellows. There are a few reports of primary care telehealth training, with limited uses of telehealth technologies. Family medicine program directors report limited and infrequent use of telehealth for e-visits and store-and-forward services among residents.9  A 3-year internal medicine telehealth curriculum provided online didactic content and experience in remote patient monitoring.10  Lee et al also highlighted the importance of incorporating electronic consultation in graduate medical education to improve value-based access to specialty care.11  Although objective structured clinical examinations (OSCEs) have been used in telehealth training, most interventions are limited to single encounter e-visits at the undergraduate medical level of training.1214  Only one study reported that an OSCE with nurse practitioners and occupational/physical therapy students utilized an interdisciplinary teaching model.15 

Primary care can be challenging, and for patients with complex psychosocial needs, we should use interdisciplinary telehealth modalities to manage patients. The Interprofessional Education Collaborative highlights communication technologies to facilitate health professional engagement in shared patient-centered problem solving.16 

The study goal was to describe and evaluate a telehealth curriculum with a longitudinal OSCE to improve internal medicine (IM) residents' confidence and skills in coordinating complex, interdisciplinary primary care via televisits, electronic consultation, and teleconferencing.

Methods

Setting and Participants

In 2019, 56 first- and third-year IM residents participated in mandatory training sessions during their ambulatory block at a university-based residency program in Stony Brook, New York. The residency follows a 4+1 block schedule, and each session had 10 to 12 participants. Due to scheduling conflicts, second-year residents did not participate.

Intervention

Residents attended two 120-minute sessions that were 5 weeks apart. Each session was repeated for 5 consecutive weeks in order to capture all ambulatory groups. Session 1 was conducted in the clinical simulation center (CSC) and required 4 standardized patients (SPs) per session for televisit OSCEs. The inter-visit simulation occurred during the 4 weeks between sessions 1 and 2, and electronic messaging was done in a training domain of Cerner PowerChart. The session 2 teleconference was held virtually using Skype for Business (figure). Two IM faculty members delivered lectures and facilitated sessions. Faculty from social work, pain management, psychiatry/addiction medicine, and nursing participated via Skype for the session 2 teleconference.

The intervention was funded by an internal department grant, which covered the cost of a TytoCare medical exam kit ($2,750) and CSC use ($4,000). No protected time was allotted for IM faculty investigators, who facilitated 10 sessions. The CSC technicians set up Skype for Business for OSCE televisits. One IM investigator initiated teleconferences and was present to troubleshoot technological issues. Psychiatry, pain management, social work, and nursing faculty volunteered five 1-hour blocks to participate in virtual teleconferences. Residents had 2 half-day clinic sessions blocked to participate in the simulation. Electronic messages and responses were sent by a study investigator weekly to resident Cerner PowerChart electronic inboxes, and a reminder email was sent 1 week after session 1 to respond to messages.

Training Description

Session 1: 

Residents received an orientation with didactic on screening for social determinants of health and OSCE instructions to address psychosocial factors, then rotated through three 30-minute stations: (1) televisit OSCE, (2) e-consultation and e-messaging, and (3) virtual examination tools.

Inter-visit Simulation: 

Over 4 weeks between sessions, the simulation continued via e-messaging. A nursing e-message was sent to participants regarding patient decompensation, prompting a series of e-consultations between the participant and specialty providers. The exercise required the learner to address the nursing message and to communicate with the social worker, pain management physician, and psychiatry providers using e-messages (activity provided as online supplemental material).

Session 2: 

This session included (1) a 20-minute didactic on interdisciplinary care and teleconferencing; (2) a 40-minute group preparation for the teleconference; (3) a 45-minute interdisciplinary teleconference; and (4) a 15-minute debriefing (provided as online supplemental material). Please email the corresponding author for simulation materials.

Evaluation

Residents voluntarily completed anonymous Qualtrics surveys before session 1 (pre) and after session 1 (post-1) and session 2 (post-2). The surveys evaluated residents' confidence with telehealth and interdisciplinary care using 5-point Likert scales (1, strongly disagree, to 5, strongly agree, and 1, very unconfident, to 5, very confident). Due to an error in survey format, data on knowledge and attitudes toward telehealth (presurvey questions 8–17) were not included in this report. SP checklists to evaluate televisit skills were based on locally developed templated checklists from the CSC library, with additional case-specific tasks in assessing psychosocial factors17  and pain using the Pain, Enjoyment of life, and General activity (PEG) scale (provided as online supplemental material).18  E-messaging evaluations were based on a consultation model from Podolsky et al19  and were scored independently by 2 IM faculty investigators for internal validity. Each e-message was scored as unsatisfactory, satisfactory, or superior (evaluation provided as online supplemental material).

Twenty survey questions were based on prior studies on the impact of telemedicine simulation training,13  telehealth usability,20  and working in multidisciplinary teams.21  Additional investigator-developed questions were not tested prior to use: 6 on demographics, 3 on satisfaction and likelihood of using telehealth in the future, and 8 on learner confidence in interdisciplinary care coordination using e-messaging and teleconferences (surveys provided as online supplemental material).

Statistical analysis was conducted in SAS 9.4 with chi-square test of independence for categorical variables and Kruskal–Wallis for continuous variables. The study was approved by the Institutional Review Board, and residents reviewed a consent form prior to participation.

Results

Survey response rates were pre 100% (56 of 56), post-1 95% (53 of 56), and post-2 100% (56 of 56). Demographics are described in table 1.

After the OSCE in session 1, more residents reported being “confident/very confident” in adjusting their camera (33%, 95% CI 20–45 vs 85%, 95% CI 75–95, P < .0001), troubleshooting videoconferences (18%, 95% CI 8–28 vs 58%, 95% CI 45–72, P < .0001), and speaking in front of a camera (42%, 95% CI 29–55 vs 76%, 95% CI 68–90, P = .001). They also reported improved confidence in explaining a televisit (29%, 95% CI 17–41 vs 74%, 95% CI 62–85, P = .002), establishing rapport (38%, 95% CI 25–51 vs 91%, 95% CI 83–98, P < .0001), and communicating effectively (33%, 95% CI 20–45% vs 79%, 95% CI 68-90%, P < .0001; table 2).

Regarding resident skills during the televisit, SPs reported that most residents introduced their role (98%, 55 of 56), explained televisit expectations (96%, 54 of 56), and redirected the patient for optimal camera viewing (80%, 45 of 56). Most residents addressed SP needs (95%, 53 of 56), portrayed empathy (100%), and 98% of SPs reported that they would return to their trainee for care. There was no significant difference in mean OSCE grades between third-year and first-year learners (68 of 100, SD 10.7 vs 65.4 of 100, SD 12.1). Despite orientation instructions to address psychosocial factors, residents rarely asked about making ends meet (16%, 9 of 56), food security (11%, 6 of 56), housing stability (5%, 3 of 56), and transportation (20%, 11 of 56), with no significant difference between resident levels.

For most evaluations of resident e-consultations, there was no significant difference in scores between 2 independent IM faculty evaluators. Resident response rate decreased with consecutive messages to providers from pain management (82%, 46 of 56), social work (66%, 37 of 56), and psychiatry (34%, 19 of 56). Residents struggled with several tasks, where the e-messaging skills most frequently scored as unsatisfactory: indicating urgency of consultation (50%–93%), documenting plan for follow-up (49%–57%), and communicating plan of care (46%–68%; table 3).

After the simulated interdisciplinary teleconference in session 2, there was an increase in residents who were “confident/very confident” in using e-messaging to communicate with specialists (36%, 95% CI 24–49 vs 80%, 95% CI 70–91, P < .0001) and coordinating interdisciplinary care (35%, 95% CI 22–47 vs 84%, 95% CI 74–94, P < .0001). Confidence increased in presenting to another provider using telemedicine (35%, 95% CI 22–47 vs 79%, 95% CI 68–89, P < .0001), facilitating an interdisciplinary teleconference (25%, 95% CI 14–37 vs 88%, 95% CI 79–96, P < .0001), and formulating an interdisciplinary care plan (42%, 95% CI 29–55 vs 91%, 95% CI 84–99, P < .0001; table 4).

Overall, residents rated sessions as “good” or “excellent” (post-1 96%, post-2 95%) and wanted to participate in additional telemedicine training (post-1 92%, post-2 84%). More residents reported that they were “likely/very likely” to use telemedicine in the future (pre 56%, 95% CI 43–69 vs post-2 79%, 95% CI 68–89, P = .001).

Discussion

A longitudinal interdisciplinary simulation improved IM residents' self-reported confidence in using telemedicine, e-messaging, and teleconferences, and increased their reported likelihood of using telemedicine in the future. For patients with complex needs, provider skills in team management and synchronous and asynchronous communication are important. Residents were least skilled in screening for psychosocial barriers to care and with documenting urgency of consults, plan of care, and follow-up on e-consultation.

As expertise becomes more specialized and care becomes more decentralized, residents will require training in telemedicine and remote expert consultation. Although we conducted the OSCE in a simulation center, the case was virtual and only required access to software such as Skype for Business. For future telemedicine training in the post-COVID-19 era, interactive computer-based virtual cases,22  virtual SP, and interdisciplinary cases could be used to increase the reach of simulation centers, expand access in settings without simulation centers, or to accommodate social distancing requirements. E-consultation and teleconferences, especially in complex disease, provides an opportunity to evaluate and expand the boundaries of traditional Accreditation Council for Graduate Medical Education and Interprofessional Education Collaborative competencies. While we did not evaluate the change in e-consultation skills before and after intervention, it was interesting that there were specific areas of e-consultation that residents struggled with. We hypothesize that there were deficiencies in these skills because our residents do not routinely write e-messages to consultants between ambulatory blocks. This may be an opportunity for future educational interventions and evaluation. Although interdisciplinary teleconferences in primary care may seem futuristic, the concept is familiar in complex care of cancer through tumor boards and hospice interdisciplinary meetings. By educating residents in leading interdisciplinary teleconferences, we aim to develop leadership skills and create demand for better practices in caring for complex patients.

Our study had several limitations. The intervention was delivered to only one program, and this simulation may not be generalizable to other specialties or programs. We did not measure the duration of training effect or assess patient-level outcomes. Due to funding and time limitations, we were unable to assess resident teleconference skills with an OSCE or change in e-messaging skills after intervention. Resident participation in the e-messaging exercise declined with each sequential message and future interventions should limit the number of required tasks to avoid learner disengagement.

Nevertheless, we learned many valuable lessons. We found that the success of the televisits and teleconferences depended on technicians or faculty to troubleshoot videoconferencing issues. Anecdotally, the residents commented that the SPs faced many of the challenges their continuity patients experience, and as the case progressed we found opportunity to teach different skills in telemedicine and e-messaging. To increase feasibility of reproducing this simulation, educators may consider deconstructing components of the simulation. Individual modules could focus on conducting televisits, e-messaging, and interdisciplinary teleconferencing. This curriculum could also be used with different patient cases and subspecialties. As educators, we should consider how future care will be delivered using telehealth technology and expand curricula to mimic real-world interdisciplinary care.

Conclusions

A longitudinal interdisciplinary telehealth simulation is feasible and can improve residents' confidence in using telemedicine and e-messaging tools to provide complex patient care.

References

1.
Tuckson
RV,
Edmunds
M,
Hodgkins
ML.
Telehealth
.
N Engl J Med
.
2017
;
377
(16)
:
1585
1592
.
2.
American Medical Association
.
AMA encourages telemedicine training for medical students, residents: new policy build upon the AMA's efforts to create the medical school of the future
.
2020
.
3.
Waseh
S,
Dicker
AP.
Telemedicine training in undergraduate medical education: mixed-methods review
.
JMIR Med Educ
.
2019
;
5
:
e12515
.
4.
Wanat
KA,
Newman
S,
Finney
KM,
Kovarik
CL,
Lee
I.
Teledermatology education: current use of teledermatology in US residency programs
.
J Grad Med Educ
.
2016
;
8
(2)
:
286
287
.
5.
Boyers
LN,
Schultz
A,
Baceviciene
R,
Blaney
S,
Marvi
N,
Dellavalle
RP,
et al.
Teledermatology as an educational tool for teaching dermatology to residents and medical students
.
Telemed J E Health
.
2015
;
21
(4)
:
312
314
.
6.
Sunderji
N,
Crawford
A,
Jovanovic
M.
Telepsychiatry in graduate medical education: a narrative review
.
Acad Psychiatry
.
2015
;
39
(1)
:
55
62
.
7.
Govindarajan
R,
Anderson
ER,
Hesselbrock
RR,
Madhavan
R,
Moo
LR,
Mowzoon
N,
et al.
Developing an outline for teleneurology curriculum: AAN Telemedicine Work Group recommendations
.
Neurology
.
2017
;
89
(9)
:
951
959
.
8.
Jagolino
AL,
Jia
J,
Gildersleeve
K,
Ankrom
C,
Cai
C,
Rahbar
M,
et al.
A call for formal telemedicine training during stroke fellowship
.
Neurology
.
2016
;
86
(19)
:
1827
1833
.
9.
Moore
MA,
Jetty
A,
Coffman
M.
Over half of family medicine residency program directors report use of telehealth services
.
Telemed J E Health
.
2019
;
25
(10)
:
933
939
.
10.
Kirkland
E,
DuBose-Morris
R,
Duckett
A.
Telehealth for the internal medicine resident: a 3-year longitudinal curriculum
.
J Telemed Telecare
.
2019
;
25
(2)
:
106
122
.
11.
Lee
M,
Namudiri
V.
Integrating telemedicine into training: adding value to graduate medical education through electronic consultations
.
J Grad Med Educ
.
2019
;
11
(3)
:
251
254
.
12.
Cantone
R,
Palmer
R,
Dodson
L,
Biagioli
F.
Insomnia telemedicine OSCE (TeleOSCE): a simulated standardized patient video-visits case for clerkship students
.
MedEdPORTAL
.
2019
;
15
:
10867
.
13.
Palmer
R,
Rdesinski
R,
Galper
A,
Cantone
R,
Shaver
J,
Symkowick
M,
et al.
Assessing the impact of a telemedicine simulation on clinical learners
.
J Family Med Commun Health
.
2017
;
4
(5)
:
112
.
14.
Sartori
D,
Olsen
S,
Weinshel
E,
Zabar
S.
Preparing trainees for telemedicine: a virtual OSCE pilot
.
Med Educ
.
2019
;
53
(5)
:
517
518
.
15.
Shortridge
A,
Steinheider
B,
Ciro
C,
Randall
K,
Costner-Lark
A.
Simulation interprofessional geriatric patient care using telehealth: a team-based learning activity
.
MedEdPORTAL
.
2016
;
12
:
10415
.
16.
Interprofessional Education Collaborative
.
Core competencies for interprofessional collaborative practice: report of an expert panel
.
2020
.
17.
Greater New York Hospital Association
.
Training primary care residents on social determinants of health
.
2020
.
18.
Krebs
E,
Lorenz
K,
Bair
M,
Damush
T,
Wu
J,
Sutherland
J,
et al.
Development and initial validation of the PEG, a three-item scale assessing pain intensity and interference
.
J Gen Intern Med
.
2009
;
24
(6)
:
733
738
.
19.
Podolsky
A,
Stern
D,
Peccoralo
L.
The courteous consult: a consult card and training to improve resident consults
.
J Grad Med Educ
.
2015
;
7
(1)
:
113
117
.
20.
Parmanto
B,
Lewis
A,
Graham
K,
Bertolet
M.
Development of the Telehealth Usability Questionnaire (TUQ)
.
Int J Telerehabil
.
2016
;
8
(1)
:
3
10
.
21.
Thistlewaite
J,
Moran
M,
Dunston
R,
Roberts
C,
Eley
D,
Bognossian
F,
et al.
Introducing the Individualized Teamwork Observation and Feedback Tool (iTOFT): development and description of a new teamwork measure
.
J Interprof Care
.
2016
;
30
(4)
:
526
528
.
22.
Oliven
A,
Nave
R,
Gilad
D,
Barch
A.
Implementation of a web-based interactive virtual patient case simulation as a training and assessment tool for medical students
.
Stud Health Technol Inform
.
2011
;
169
:
233
237
.

Author notes

Editor's Note: The online version of this article contains the intersession electronic messaging activity, objective structured clinical examination case and learner clinical tasks, standardized patient OSCE checklist, e-consultation/e-messaging evaluation, and pre-intervention, post-session 1, and post-session 2 surveys.

Funding: This study was supported by the Stony Brook Department of Medicine Pilot Grant.

Competing Interests

Conflict of interest: The authors declare they have no competing interests.

The authors would like to thank the Stony Brook Department of Medicine, Clinical Skills Center at the Renaissance School of Medicine of Stony Brook University, and Sara Steinfeld, Suffolk Care Collaborative Social Worker.

This research was presented as a poster at the Society of General Internal Medicine Annual Meeting, Washington, DC, May 9, 2019, Association of Program Directors in Internal Medicine Fall Meeting, Denver, Colorado, October 18, 2019, and American Telemedicine Association Virtual Conference, June 22–26, 2020.

Supplementary data