Pilot Study Evaluating Use of Bug-in-Ear Technology as a Novel Resident Feedback Tool Open Access

Feedback is a key component of medical education; however, changing physician behavior is challenging.¹  Typically, resident feedback follows a patient encounter and can then be incorporated in future patient interactions. This approach is not ideal since it does not allow residents to apply the feedback in real time, resulting in missed opportunities for practice and behavior change.

Bug-in-ear (BIE) technology is a novel feedback tool that allows educators to provide real-time feedback to residents through an earpiece, increasing opportunities for immediate implementation and practice of desired behaviors. BIE feedback (BIEF) has been used effectively in other professions and is considered an evidence-based practice for professional development in education, with increasing frequency and accuracy of teaching behaviors and maintenance of behavior change reported.²  However, BIEF has had limited use in medical education, and further evaluation of BIEF in medical training is warranted.

First-year pediatric residents (21 eligible), faculty preceptors, and volunteer standardized patients (SPs) participated in 2 consecutive simulated shared decision-making (SDM) case encounters during intern orientation in 2021 and were randomly assigned into groups. One group received traditional feedback after each case, and the other received real-time feedback from faculty preceptors throughout the encounters using a $26 Bluetooth earpiece. The BIEF intervention was conceptualized using a deliberate practice framework, with residents receiving real-time feedback on SDM skills that could then be implemented throughout the consecutive cases. Faculty members providing BIEF received a 1-hour training on best practices for real-time feedback. The encounters were video recorded.

All participants completed surveys assessing feasibility and acceptability of BIEF (provided as online supplementary data). We measured residents’ SDM skills (OPTION 5, an observer rating instrument which measures SDM-specific physician behaviors) after cases; recordings were reviewed and scored by trained raters. Residents’ SDM attitudes (Patient-Practitioner Orientation Scale [PPOS]) and self-efficacy (SE-12 questionnaire) were measured before, immediately after, and 4 months after case encounters. We evaluated feasibility and acceptability using descriptive statistics and compared outcomes between resident feedback groups using mixed models.

Eighteen of 21 eligible residents (86%), all 10 faculty, and all 9 SPs agreed to study participation; 8 residents, 3 faculty, and 6 SPs were in BIE groups. All residents who received BIEF found it to be effective feedback, and 75% (6 of 8 residents) felt it enhanced their learning. Residents described the opportunity to adjust behavior in real time as a noteworthy advantage of BIEF. Disadvantages included distraction (38%, 3 of 8 residents) and earpiece discomfort (13%, 1 of 8 residents). All faculty members who used BIE technology found it effective for providing real-time feedback; 67% (2 of 3 faculty) felt it was beneficial for the residents. Faculty described benefits as including the ability to provide immediate guidance while also building autonomy, but noted adjustment time was necessary for effective BIEF. All 6 SPs reported willingness to see a clinician receiving BIEF in the future.

Resident PPOS and SE-12 scores showed no significant change over time for either group. Mean scores on the OPTION 5 were higher in the BIE group (52.5 [SD=14] vs 48.7 [SD=10.3]), but this difference was not statistically significant (P=.34). The small sample size limited our ability to detect differences between groups. Additionally, our intervention occurred for a brief time, which may not have allowed for sufficient practice time or an adequate BIE “dose.”

BIE technology offers an opportunity to provide real-time feedback in a manner that is feasible and acceptable to most residents, faculty members, and SPs involved in simulated encounters. Use of BIE technology has a steep learning curve, and adequate practice is necessary to maximize the impact of feedback and minimize distractions. Evaluation with a larger sample is needed to explore the ideal BIEF dose to change outcomes, use of BIEF in clinical encounters, and the impact of BIEF on patient care.

The online supplementary data contains the survey used in the study with results, comments, and acceptability of bug-in-ear feedback.