The Acceptability of Avatar Patients for Teaching and Assessing Pediatric Residents in Communicating Medical Ambiguity

All residency programs are required to report resident competence to the Accreditation Council for Graduate Medical Education (ACGME) through the Milestone Project.^1,2  Certain Pediatric Milestones, such as those addressing challenging communication skills like ambiguity and shared decision-making in patient care, are difficult to assess.³  Medical ambiguity has been defined as a lack of clarity in the clinical decision-making process.⁴  Pediatric residency program directors and simulation experts identified communication of medical ambiguity as appropriate for simulation-based assessment.³  Assessing these discussions is challenging in the clinical setting as they may not occur during faculty observation.⁵  First-year internal medicine residents indicated that only 5% of their first patient experiences delivering bad news occurred with faculty observation.⁶  When faculty are present, they are more likely to take the lead in challenging conversations rather than mentoring the trainee to do so.⁵ 

Although standardized patients (SPs) are an effective educational and assessment strategy for improving communication skills,^7-11  recruiting pediatric SPs may be problematic for difficult medical discussions because of ethical and developmental considerations and the limited availability of child actors.¹²  When children have served as SPs, the focus has been on clinical reasoning rather than communication, and feedback from these SPs on their emotional experience was mixed.¹³  Realistic communication encounters in pediatrics are challenging to coordinate when multiple SPs are needed to represent both caregivers and their children.

To circumvent these challenges, alternative simulation modalities may be preferable. Clinicians can interact with computerized “virtual patients” via software branching algorithms. Virtual patients have been used in health care teaching^14-20  and assessment^18,19,21,22  of clinical reasoning and shared decision-making,²³  but have more limited use in assessing communication since computer algorithms do not yet allow for the nuanced conversations between live people.

Avatar patients (APs) are realistic animated representations of people who are voiced and controlled by live, trained actors. APs have taught communication skills to nursing students^24,25  and pediatric residents to recognize respiratory distress in babies.²⁶  There are little data on the use of APs for teaching or assessing resident communication skills. One study used prerecorded AP responses that were perceived as realistic and beneficial for learning.²⁷  For complex discussions, such as communicating medical ambiguity and initiating shared decision-making, APs with prerecorded answers are unlikely to reproduce realistic encounters.

We developed a curriculum using APs to assess resident competency in communicating medical ambiguity. We sought to determine if APs are acceptable, provide a meaningful educational experience, and are realistic for teaching and assessment of a complex communication topic.

We conducted the study from 2019 to 2021 at a medium-sized pediatric residency program in a large Northeastern US city. We recruited participants from the categorical pediatric, preliminary child neurology, child psychiatry, and combined medicine-pediatric residencies over 2 academic years.

We partnered with Mursion, an educational technology company that developed screen-based APs voiced by live actors in the roles of children and adults. These APs interact with trainees through verbal and nonverbal communication during a simulated scenario. Encounters required a computer with Zoom, camera, microphone, and speaker for the participants and AP actors to communicate. Mursion provided free access to their software and actors in exchange for the ability to use the study scenarios with other clients. The actor was paid $30/hour through grant funding. Mursion was not involved in study development, analysis, or manuscript writing. The standard Mursion start-up cost for the first 500 hours of simulation is $50,000 with additional sessions at $140/hour.

The authors designed 2 commonly encountered pediatric scenarios (Figure) featuring ambiguous medical situations without clear-cut answers. In scenario 1, the objective was to discuss the decision to perform a lumbar puncture on a febrile 30-day-old infant in the emergency department (online supplementary data). In scenario 2, the objective was to discuss next steps for a teenager expressing suicidal ideation in the outpatient setting (online supplementary data). Shared decision-making was crucial to both scenarios. The scenarios were piloted with trained APs by the study leader (A.F.V.) playing the participant role and either achieving or missing the predefined desired behaviors for each case; see the cases (online supplementary data) for behavior definitions. One actor responded to the participant in real time using voice modulating technology to play the roles of 2 people in each scenario.

All residents (n=89) in the categorical pediatric, preliminary child neurology, child psychiatry, and combined medicine-pediatric residency programs were scheduled for both AP scenarios, followed by an acceptability survey as part of their required curriculum. No residents opted out of study inclusion. In both scenarios, all relevant medical guidelines were provided immediately beforehand so that the focus would not be participant medical knowledge. Participants met their APs on Zoom, where the actor introduced the scenarios. Total intervention time was 45 minutes. After each scenario, participants participated in a brief debriefing session with the actor and completed an acceptability survey.

We developed a 13-question survey on demographics, perceived realism of the APs, emotions evoked by APs, and the educational value of APs (provided as online supplementary data). To develop this, we conducted a literature review to identify and incorporate previously developed survey questions that were applicable to our study. A survey developed by Mursion²⁸  for teachers using their technology to simulate teaching experiences was adapted for question 4. A survey comparing different educational technologies was used for question 3, and a survey designed for medical students to compare SPs to manikin-based simulation was used for question 7.²⁹  The latter 2 have validity evidence supporting its use in this capacity.³⁰  Twelve questions were answered on scales. One free text question was included in which respondents were asked to give 1- to 2-word answers. Introductory information, question order, survey layout, and wording were designed to maximize response rate while decreasing measurement error using internet survey design best practices.³¹  The final acceptability survey was reviewed by the entire team and distributed using Qualtrics software.

Descriptive analysis was conducted for the quantitative survey questions using SPSS Statistics, version 24 (IBM Corp). One free-text survey question asked for 1- to 2-word responses describing emotions raised by the scenarios. Two study authors (A.F.V., L.M.) independently coded the responses by putting like responses into categories and counting how many times each category was mentioned. Minor coding differences were reconciled.

This study was approved by the Institutional Review Boards at Mass General Brigham, Weill Cornell Medicine, and Maine Medical Center.

Sixty-five out of 89 (73%) residents from the categorical pediatric, preliminary child neurology, child psychiatry, and combined medicine-pediatric residencies completed the experience. Sixty-five residents were eligible in 2019-2020 and an additional 24 residents were eligible in 2020-2021. Reasons for non-participation included session cancellation during the COVID-19 pandemic, being called into work unexpectedly, Zoom challenges, personal emergency, or graduation. Sixty-one of 65 (93.8%) residents who completed the AP experience completed the survey. Table 1 describes resident respondent characteristics including postgraduate year and program type.

A majority of respondents (78.7%, 48 of 61) reported being emotionally “invested” or “extremely invested” on a 5-point scale of extremely uninvested to extremely invested (Table 2). When asked what emotions were evoked by the scenario, anxiety, uncertainty, concern, empathy, sadness, stress, and fear were the most often cited emotions (Table 3). Overall, the majority of respondents rated the AP environment and the conversations in the scenarios as realistic (Table 2).

APs were rated as “beneficial” or “very beneficial” by the majority of respondents for learning to communicate with patients about medically ambiguous situations and increasing their confidence in these discussions. Most would recommend the use of APs and would be comfortable with faculty using AP experiences to assess them for a complex communication milestone (Table 4).

Most pediatric residents felt emotionally invested in conversations using AP technology. The majority of residents indicated their acceptance of the AP format, suggesting that APs may be a viable technology to improve complex communication skills in formative training and summative assessment.

Our findings underscore the impact of APs in creating emotional investment and conversational realism in simulated clinical scenarios. Nursing students^24,25  and pediatric residents^27,32  find APs to be realistic and helpful for teaching communication skills. Few studies have used VPs or APs voiced by a live person rather than branching narrative algorithms or natural language processing.^27,32,33  Without a live person, such technology deprives learners of the authenticity that comes with reciprocal conversation and nonverbal communication skills, particularly for nuanced topics. Unlike other studies where medical personnel voice VP patients,^34-36  we used lay person actors to increase the realism of these conversations. Importantly, using live actors on Zoom permitted the APs to see and respond to resident nonverbal facial cues. We believe these helped increase the realism and emotional investment residents felt. Most residents found APs helpful for improving confidence in communicating ambiguity and engaging in shared decision-making. Moreover, most would repeat the AP experience for their learning, be comfortable (or at least neutral) with its use for assessment, and would recommend it for difficult conversations.

In terms of feasibility, the financial and resource costs to using a software platform for AP encounters such as Mursion include purchasing a minimum of 500 hours of simulation for $35,000, plus approximately $15,000 for actor training in the first year. Each 1-hour simulation session beyond the first 500 hours costs $140. The total cost of our project would have been $50,000 even though we only used 65 of the 500 purchased hours. On the other hand, when SPs are employed in communication simulations, the costs may be considerable when multiple SPs are needed for a cast of characters. Training one SP and then conducting 2 cases for 65 residents is estimated to cost from $2,000 to $10,000.^37-39  That cost would approximately double to portray a parent and a child together. Neither includes travel costs for the SPs if remote Zoom technology is not used. In contrast, a single adult actor using voice modification can portray multiple APs. In cases involving children, communicating with both child and parent is often imperative to demonstrating a communication skill. Recruiting child SPs can be challenging, and the use of minors raises important ethical considerations.^12,40  Training APs and SPs is resource intensive. In our study, we trained the actor on both cases over 1 hour, and then piloted both cases with performance feedback over 2 hours.

Our study has limitations. The study was conducted at one site in one clinical field and findings may not be generalizable. The actor training was brief and may have been insufficient for the broad range of resident behaviors displayed. This may have led to the better actor responses over time and the experience being different for residents who participated later in the study. Residents in this study only had a single exposure to APs. Comfort and perceived value with APs could increase with further exposure. While our survey used many of the best practices in survey design,³¹  we did not perform cognitive interviewing prior to implementation. Some survey questions may be perceived as leading residents to answer affirmatively, such as asking “how” rather than “whether” the technology was beneficial. We asked residents to respond in a few words to describe their emotions during the AP experience, making a more nuanced qualitative analysis impossible. We asked for short responses to increase response rate on the resident survey. Further, it is not clear if the emotions raised related to the communication content or the AP technology.

Our next step is to assess resident performance on the AP experience. We hope to develop a tool that allows for assessment of competency in coping with ambiguity.

We evaluated realism, acceptability, and educational value of APs to pediatric residents for communication involving medical ambiguity. Residents were emotionally invested and found the AP modality to be realistic and beneficial for their learning and would repeat it and recommend it to others. Importantly, they agreed with its use to assess competency.

The authors wish to thank Katherine Sparger, MD, for her help in writing the suicidal ideation scenario, and the MGfC residents for their participation in this study.