The Virtual Visit: A Telehealth Curriculum for Internal Medicine Residents Featuring a Virtual Physical Examination

Prior to March 2020, telehealth was a rapidly developing practice of medicine with evidence demonstrating improved health outcomes for patients with chronic medical disease including diabetes, hypertension, and heart failure.^1-3  Health care systems, such as the US Department of Veterans Affairs, led efforts to incorporate telehealth care for patients with multimorbid chronic medical disease.⁴  When the COVID-19 pandemic began, telehealth became a vital tool to deliver care to patients with acute and chronic illnesses. At the University of California Los Angeles (UCLA) Health, telemedicine visits in the Department of Medicine increased from <1% to 55% of total health care visits from March 9 to April 18, 2020.⁵ 

During the forced expansion of telehealth, medical institutions had to adapt and train physicians on telehealth simultaneously. However, there were no clear guidelines or standard platforms for telehealth medical education.⁶  While many virtual physical examination guides exist,^7-9  residents’ familiarity with the virtual physical examination was low.^10-12  In response, some residency programs developed ambulatory telehealth training programs to address residents’ concerns about their ability to deliver high-quality virtual care.^10,11 

In July 2020 we performed a needs-based assessment within the UCLA internal medicine (IM) and internal medicine/pediatrics (IM/P) residency programs to assess residents’ comfort level and knowledge on the delivery of telehealth to address gaps in knowledge. After delivery of a telehealth curriculum to UCLA IM and IM/P residents, we hypothesized that residents’ confidence and knowledge in delivering virtual care, including performing a virtual physical examination, would significantly improve.

In July 2020, a formalized needs-based assessment (provided as online supplementary data) was conducted within the UCLA IM and IM/P residency programs. Our approach followed the Kern’s framework for curricular development. The initial survey determined attitudes toward telehealth and identified specific areas in which residents felt additional education would be beneficial. A total of 111 of 180 (62%) UCLA IM and IM/P residents participated (36% postgraduate year 1 (PGY-1); 31.5% PGY-2; 32.4% PGY-3/4). Eighty-one percent of residents (90 of 111) had no formal training on telehealth. Respondents demonstrated subjective deficiencies in understanding the legal guidelines associated with telehealth care, performing a virtual examination, and using remote monitoring devices (online supplementary data).

We designed a formal telehealth curriculum using the frameworks of Ericsson’s theory of deliberate practice and Kolb’s theory of experiential learning.^13,14  Our curriculum contained core learning objectives and sub-objectives (online supplementary data) with specific competencies focused on improving residents’ knowledge and confidence in (1) legal guidelines of telehealth, (2) patient-assisted virtual physical examination, (3) health equity and access to telehealth, and (4) telehealth chronic disease management.

End-point results focused on self-reported confidence intervals. Power calculations determined approximately 123 participants (n) were needed for our population of 180 (N=total size of IM and IM/P residents [α=0.5; Z=1.96]).

Our telehealth curriculum was taught in 3 parts to 180 eligible UCLA IM and IM/P residents from November 2020 through March 2021. Our curriculum incorporated virtual didactic and experiential pedagogies. The telehealth legal guidelines, health equity, and chronic disease management topics were each respectively presented to the entire IM and IM/P residency programs during one-hour didactic sessions (noon conferences) on Zoom and included interactive case scenarios. The virtual physical examination curriculum was presented during a dedicated one-hour time block allocated for outpatient practice management education. It was delivered on Zoom to 5 small groups, each comprised of 35 to 40 residents. Residents received demonstrations, handouts, and verbal coaching on adapting physical examinations to a virtual setting. Additionally, resident pairs performed case-based scenarios, as a physician or a patient, to simulate telemedicine examinations, specifically the musculoskeletal and the head, ears, eyes, nose, and throat examinations (Figure).

Pre- and post-intervention learner assessments were conducted via Likert scale surveys (graded 1-5, least to most confident) and knowledge-based examinations (multiple choice or true/false questions; provided as online supplementary data). Results were analyzed via a t test (Table). Feasibility was measured by tracking estimated time to create the curriculum and prepare faculty to deliver it.

UCLA Research Administration Institution Review Board approval was obtained. Residents were not required to consent to participate in the curriculum or assessments.

The results detailed below are from the virtual physical examination curriculum. Outcomes related to other curricular components were not sufficiently powered and therefore not included. Ninety-four percent (146 of 156) of residents completed the pre-survey and the post-survey (27.4% PGY-1; 37.6% PGY-2; 34.9% PGY-3; 0% PGY-4). Surveys were not uniquely linked to individuals.

Following the virtual examination curriculum, residents reported statistically significant increased confidence levels in performing a targeted virtual physical examination (P<.001; 95% CI 0.97-1.35), engaging patients or caregivers to assist in virtual examinations (P<.001; 95% CI 0.76-1.21), and using remote monitoring devices (P<.001; 95% CI 0.58-1.03; Table).

Three instructors developed and delivered the curriculum. It took 4 hours to create each didactic session, 1 hour to plan content delivery logistics, and 1 hour to teach each session.

Following the COVID-19 pandemic, the health care system adopted a hybrid of virtual and in-person care, supporting the future utility of telehealth.^15,16  We demonstrated within our UCLA IM and IM/P residency programs that a formalized telehealth curriculum significantly improves residents’ confidence in delivering virtual care.

Our curriculum was novel and developed prior to the Association of American Medical Colleges and the Accreditation Council for Graduate Medical Education (ACGME) updating the Milestones and competencies evaluation to include telehealth skills.^17,18  Notably, the ACGME program requirements for graduate medical education in internal medicine do not specifically highlight the virtual physical examination, which requires a unique approach and skillset compared to the in-person physical examination.¹⁹  Furthermore, these accreditation guidelines do not comment on how residents should be taught virtual care. While data on outcomes of using tele-education compared to face-to-face learning exist,²⁰  there has not been extensive research studying the most effective methods of teaching a virtual physical examination.

Studies from multiple subspecialties show that residents’ confidence levels improved with formal education on the virtual physical examination.^10,21-23  The results from our curriculum support teaching the virtual physical examination in a virtual learning environment, using simulation training, and incorporating real-time peer feedback. Studies support that experiential learning through simulations leads to more sustained practice change.²⁴  It is feasible for other residency programs to duplicate our telehealth curriculum with the use of an online interactive synchronous didactic program.

Limitations of our study include the use of self-reported data, sample size of a single academic internal medicine residency program, and the short time interval of the educational intervention and post-survey that occurred on the same day. Assessment over a larger time period would allow for analysis of sustained confidence and true practice change.

We have subsequently expanded our curriculum to medical students at the David Geffen School of Medicine at UCLA and intend to incorporate an objective structured clinical examination for additional practical assessments.

Our study contributes to the growing body of evidence that supports telehealth education in residency. We hope our experience can guide other residency programs in developing a formalized telehealth curriculum including the virtual physical examination.

We created a curriculum that taught residents how to perform a virtual physical examination in a virtual learning environment, using simulation training and incorporating real-time peer feedback. Our results demonstrate that a formalized telehealth curriculum significantly improved residents’ confidence in delivering virtual care.

The online version of this article contains the needs-based assessment tool, demographics of study participants and summary of needs-based assessment, telehealth curriculum core learning objectives, virtual physical examination checklist, and virtual physical examination pre-assessment tool.

Funding: The authors report no external funding source for this study.

Conflict of interest: The authors declare they have no competing interests.

Disclaimer: The views expressed in the article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs or the US Government.