Analysis of Supervisors' Feedback to Residents on Communicator, Collaborator, and Professional Roles During Case Discussions

In medical training, every clinical case seen by a resident is discussed, at least briefly, with a supervisor (clinical teacher or preceptor).¹  Although it is a recurrent teaching opportunity, supervisors acknowledge the challenge to offer verbal feedback during case discussions on residents' roles as communicator, collaborator, and professional.^2,3  Renamed “intrinsic roles” in the CanMEDS framework, the roles of communicator, collaborator, professional, manager, scholar, and health advocate are intertwined with the central role of medical expert.^4,5 

Supervisors who offer feedback on intrinsic roles help residents embody those roles in the workplace.^6,7  However, the proportion of verbal feedback on intrinsic roles (not limited to medical expertise) remains unknown.^2,8–11  Observations from clinical settings in internal medicine revealed that the CanMEDS roles were rarely explicitly named or used to structure daily interactions between residents and supervisors.^8,11  Discourse analysis of the written feedback given in the workplace further suggested that supervisors often misinterpreted the meaning of intrinsic roles.⁹  When interviewed, residents and supervisors agreed that only a limited number of intrinsic roles can be addressed during most clinical supervisions.⁸ 

To analyze and improve the feedback supervisors give to residents about their roles of communicator, collaborator, and professional, the literature relies only on analyses of written feedback or what residents and supervisors self-report about what was said during the discussions.^9,12  However, self-reports may be incomplete or biased, and analyses of written feedback may not apply to the verbal feedback given to residents throughout their training. In a review published in 2017, only 11 studies used content analysis of audiotapes or videotapes of feedback interactions. None of them focused on intrinsic roles.^12–14 

This study provides the first detailed analysis of the verbal feedback supervisors give to residents about intrinsic roles when discussing in-hospital clinical cases. The objectives of this study were to determine the importance of case discussions to trigger feedback on intrinsic roles and identify areas for improvement.

Scripted clinical cases challenged resident roles of communicator, collaborator, and professional. Each case represented a particular intrinsic role. Ubiquitous in all competency frameworks, these roles overlap with the core competencies of interpersonal and communication skills and professionalism from the Accreditation Council for Graduate Medical Education (ACGME).¹⁵  We sought to:

1. quantify how much verbal feedback from supervisors, as a percentage of total feedback time, targeted intrinsic roles (ie, not strictly limited to medical expertise), and how well feedback was aligned with the specific intrinsic role targeted by the case (specific role time as a proportion of total intrinsic role time);

2. determine the educational goals of the feedback on intrinsic roles; and

3. determine, in the embedded randomized experiment, if feedback on an intrinsic role differed in terms of length, alignment, and educational goals when residents explicitly asked for it.

From 2017 to 2019, we recruited French-speaking university-affiliated clinical supervisors in 2 large medical residency programs in the Faculty of Medicine at Université Laval (Canada): general internal medicine (n = 15) and diagnostic radiology (n = 15). This sample size was calculated to be sufficient to detect a moderate effect of explicitly asking for feedback in a within-subjects experiment based on previous studies by Côté and colleagues.¹⁶ 

Because our goal was to analyze supervisors' natural behaviors, the study did not include pretraining, reminders, or feedback tools. Supervisors were unaware of the study's focus. Optional professional development on providing feedback had been offered for many years. Supervisors in both programs have used the CanMEDS framework since 2007. Transition to a competency-based approach took effect after the study in both specialities.

We chose diagnostic radiology and internal medicine because they largely differ in their work. In internal medicine, the aging population, among other factors, mandates a biopsychosocial approach and longer encounters with patients (and their families), often targeting many intrinsic roles.¹⁷  In contrast, patient interactions in radiology are usually brief and diagnostic oriented.¹⁸ 

As shown in Table 1, we designed each clinical case to target an intrinsic role, either communicator (case A), collaborator (case B), or professional (case C). Both in internal medicine and radiology, supervisors considered role modeling to be relevant and necessary to learn these 3 competencies but felt uneasy providing feedback due to a lack of confidence.^18,19  This opinion was shared by our program directors and department heads.

We trained 3 residents in their second or third year. They recruited a convenience sample of supervisors throughout the year in hospital services where they were doing rotations, without involvement of researchers. Residents did not recruit supervisors with whom they had longitudinal supervision. All supervisors who were approached agreed to participate and to be audio recorded by the resident. Case discussions occurred in the workplace. Supervisors were told to meet their usual time constraints and decided when to conclude each case discussion.

A resident discussed each of the 3 clinical cases (A, B, and C) with a supervisor (Table 1). We designed cases based on Milestones specific to each CanMEDS role.^5,20  In terms of medical expertise, residents performed as expected for their level. We validated and standardized the cases in a pilot study with senior residents in both specialities. All aspects of the case were scripted, including descriptions of residents' uncertainties and patients' reactions. Residents mentioned all clinical details necessary for diagnostic and management (eg, lab values, diameter of the ovary, social context). We provided mock images, laboratory results, consultation reports, and summary charts. For example, radiologists and residents were looking at anonymized ultrasound images on their workstations when discussing the case. Residents were not strictly bound to their script and engaged in a discussion with the supervisor. The supervisor could, at any time, take the lead in the discussion.

The study included 90 case discussions conducted between 15 radiologist-resident pairs and 15 internist-resident pairs. As displayed in Figure 1, the order of the cases and inclusion of explicit statements were randomized to minimize order effects such as carryover. The only difference between implicit and explicit cases was one segment mentioned near the end of the discussion by the resident (presented in Table 1). For example, in case A targeting the communicator role, if so randomized, the resident would make explicit their need for feedback aligned with the communicator role by saying, “I didn't know what to tell the patient. [Brief pertinent details.] What do you think?”

We transcribed all supervisor sentences that contained at least a subject and a verb, noting time stamps in seconds to quantify time length of feedback. A research assistant had previously erased sentences that would compromise authors' blinding (eg, explicit statements) or participants' anonymity (eg, name of supervisor). Three coders (A.L., L.C., C.S. for radiology; A.L., L.C., P.L. for internal medicine) conducted the content analysis of the transcripts. Each coder independently analyzed all 90 supervisor-resident discussions, provided to each in a distinct random order. Two authors (A.L. and L.C.) are clinical supervisors with experience in content analysis.

We analyzed the feedback on intrinsic roles using a deductive approach, coding in NVivo 12 (QSR International [Americas] Inc, Burlington, MA). Our framework for analysis of supervisors' verbal feedback was based on 3 dimensions: proportion of supervisors' verbal feedback time targeting intrinsic roles, alignment with the role targeted by the case, and educational goals of feedback (provided as online supplementary data).

The focus of feedback is often selected by supervisors based on their own expertise or interest and may therefore only partially be aligned with the case itself or resident needs.^2,19,21,22  Renting and colleagues⁹  observed that patient-centered and teamwork approaches were rarely discussed with students. Rather, feedback on efficiency, resource management, and directive leadership predominated.⁹  Even when using techniques to structure feedback on many roles, feedback was mostly limited to the medical expert and communicator roles.^23–25 

Analyzing recordings of physicians' verbal feedback on medical expertise during case discussions, Kennedy and Lingard,²⁶  followed by Côté and colleagues,^16,27  observed the following educational goals: asking clarifying questions about the case or the resident's educational needs, asking probing or challenging questions, giving advice, and reflecting about the resident or oneself as a supervisor. We added one more category: expressions of agreement or disagreement with a resident's opinion. We confirmed in our pilot study that this new classification was exhaustive and mutually exclusive.

Residents' active engagement in identifying and explicitly expressing their learning needs is a key factor for eliciting supervisor feedback during clinical supervision.^8,28,29  Students who explicitly expressed educational needs regarding medical expertise are more likely to have their specific needs met.³⁰  With respect to intrinsic roles, qualitative evidence suggested that if students were to explicitly state their educational needs, supervisors would deploy a wide range of responses aligned with those needs.^16,27  However, these studies were based on what supervisors imagined they would do when discussing vignettes with a researcher. Results can therefore be overestimated due to the tendency for study participants to change their behavior simply as a result of being observed.

We calculated the proportion of supervisors' verbal feedback spent on each CanMEDS role by dividing the time spent on that role by the total feedback time (intrinsic roles + medical expertise). We calculated alignment as the time spent on the specific role targeted by each case divided by the feedback time spent on all intrinsic roles. Supervisors' verbal feedback time spent on each educational goal of feedback was also divided by the feedback time on all intrinsic roles.

We used SPSS Statistics 21 (IBM Corp, Armonk, NY) for statistical analysis. Interrater agreement was calculated with intraclass correlation coefficients (2-way mixed; absolute agreement). We used Student's t tests to determine if feedback differed in terms of duration, alignment, and goals when residents explicitly asked. Because each participant did all 3 cases consecutively, in order to avoid multiplication of type I errors, the P values were compared with thresholds of .0083 (Bonferroni correction; 0.05 divided by 6).

After the 3 case discussions, each supervisor completed a written questionnaire (provided as online supplementary data). We collected participants' characteristics and their perceptions of the study setting (Table 2), while avoiding questions endangering anonymity. Five questions were answered on a Likert scale from 0 to 10 (0, totally disagree, to 10, totally agree). Three multiple-choice questions probed supervisors' insights on each case. They sought the following information:

1. For each case, the supervisor had to identify which role was targeted by the case. We compared their answer with Table 1.

2. We asked the supervisor on which role feedback was given. We verified in our data set if feedback time was mainly spent on this role.

3. We asked the supervisor if, and on which role, the resident explicitly asked for feedback. We verified if this discussion was assigned by randomization as explicit.

After reviewing our protocol, the Research Ethics Committee of Laval University, applying rule 2.5 for quality improvement of educational projects, waived ethical approval.

Supervisors' characteristics and perception of the study setting are presented in Table 2. On average, radiologists (n = 15) dedicated 22% of case discussion time to providing feedback on intrinsic roles (mean 61 seconds, SD 52), as detailed in Table 3 and Figure 2. The feedback on intrinsic roles was aligned at 48% with the role targeted by the case (mean 29 seconds, SD 29). As detailed in Table 3 and Figure 3, internists (n = 15) dedicated, on average, 70% of case discussion time to providing feedback on intrinsic roles (mean 116 seconds, SD 80), 56% of which was aligned with the role targeted by the case (mean 65 seconds, SD 62).

With respect to educational goals of feedback, radiologists offered 66% advice (mean 40 seconds, SD 40), 21% reflections (13 seconds, SD 18), and 7% agreements (4 seconds, SD 9). Internists offered 41% advice (47 seconds, SD 46), 40% reflections (46 seconds, SD 59), and 10% clarifying questions (12 seconds, SD 16). Detailed results are presented in Figures 2 and 3, and as online supplementary data. Interrater agreements of the coders on the duration and alignment of feedback for each intrinsic role were 0.93 for radiology cases and 0.92 for internal medicine cases. Interrater agreements on the goals of feedback were 0.91 and 0.92, respectively.

When residents explicitly requested feedback on an intrinsic role, we saw no consistent effects in both specialities on the length, alignment, or educational goals of feedback (see footnotes to Table 3 and online supplementary data).

Radiologists identified the intrinsic role targeted by the case in 15 of the 45 case discussions (33%). For example, they correctly identified that case A was targeting the communicator role. Radiologists rightly identified on which role they provided the most feedback in 49% (22 of 45) of the cases. In 100% (26 of 26) of the cases in which residents explicitly asked for feedback about an intrinsic role, radiologists heard an explicit need, and in 58% (15 of 26) of cases identified correctly for which role the resident had explicitly asked. Internists identified the intrinsic role targeted by 36% (16 of 45) of the cases, and for which role they provided the most feedback in 42% (19 of 45) of cases. In 95% (23 of 24) of the cases, internists heard the explicit need and in 42% (10 of 24) of cases identified for which role.

In case discussions across 2 different specialties, supervisors' verbal feedback to residents on communicator, collaborator, and professional roles was substantial and largely aligned with the role targeted by the clinical case. A possible misconception is that this feedback time was not spent discussing medicine.⁹  As shown in figures 2 and 3, all feedback statements on intrinsic roles were tied to the context of the case and to the medical expert role. Supervisors were still discussing how to manage the case medically while reinforcing important aspects of interdisciplinary and patient-centered care.³¹ 

Confirming previous studies,^8,11  in our data the intrinsic roles were never explicitly named by supervisors to structure their feedback or make explicit their role modeling efforts. Nonetheless, although the CanMEDS framework did not appear to guide supervisors at the metacognitive level (difficulty to identify implicit or explicit needs, partial insight on the feedback given), at the cognitive level, intrinsic roles occupied a substantial part of case discussions. Our data provide a rare opportunity for researchers to analyze verbal discussions on intrinsic roles in the workplace. New research angles include the use of verbal/written reminders for intrinsic roles, the perspective of the learner, or the comparison of the discussions on medical expertise versus intrinsic roles.

In terms of alignment, similar patterns emerged in both specialities. The cases targeting the communicator and professional roles triggered a meaningful combination of feedback on communication, collaboration, and professionalism. In contrast, cases targeting the collaborator role triggered feedback almost exclusively on this role. This is partly explained by the CanMEDS definition of the collaborator role, which includes professional communications with colleagues (provided as online supplementary data).²⁰  Corroborating the findings of Renting and colleagues,¹¹  if not targeted by the clinical case, supervisors barely discussed the roles of leader, health advocate, and scholar. These findings emphasize the need for teaching resources targeting those roles specifically.³¹ 

Our measures of verbal feedback time and previous studies on occurrence (eg, number of sentences) concur that feedback is predominantly a monologue of advice, reflections, and agreements or disagreements.^13,14,32  Blatt and colleagues³³  found that through interaction analysis only 13% of feedback was formulated as questions. Using content analysis, Holmboe and colleagues¹³  showed that in 34% of direct observation sessions, faculty asked interns for self-assessment. In our samples of internists and radiologists, questions represented 14% and 6% of the feedback on intrinsic roles, respectively. Surveyed by Olmos-Vega and colleagues,³⁴  senior residents preferred to be questioned and to engage in meaningful dialogues. Further content or interaction analysis of verbal feedback could help understand supervisors' difficulty in establishing trusting dialogues with learners, especially on intrinsic roles.^13,33 

In our experiment, residents' explicit need for feedback on intrinsic roles did not result in similar responses to those described in studies based on written feedback systems or interviews with supervisors.^10,16,27  In previous studies, supervisors could more easily deduce that intrinsic roles were the focus of the researchers and adapt their behavior accordingly. Another explanation for the absence of effect is that supervisors were already providing lengthy feedback without the explicit statements and therefore had relatively little room for improvement. This hypothesis would benefit from further research using individual interviews with supervisors with think-aloud protocols. We designed this study to detect a moderate to large effect. Studies with more participants are needed, but in the meantime our results set realistic expectations with respect to the added value of explicitly asking for feedback on intrinsic roles.

As also reported by Saucier and colleagues,⁸  supervisors have limited insights on their performance regarding feedback on intrinsic roles. Most supervisors could not identify the role targeted by the case and on which role they mainly gave feedback. Therefore, studies relying only on self-reports present a risk of misjudging the feedback on intrinsic roles. Our findings should persuade supervisors and residents of the opportunity that case discussions represent. Having a better insight of the feedback they give on intrinsic roles could motivate supervisors to improve their teaching skills. Our data represent individualized performance evaluations of 30 supervisors. Based on our observations, supervisors could improve by aligning feedback with residents' explicit needs, and using more clarifying and probing-challenging questions to establish a meaningful dialogue especially with senior residents.^26,34  Further research is needed to confirm if sharing these observations with supervisors would improve their teaching and self-assessment abilities.^2,19,22,35 

There are limitations to this study. The absolute length of time spent on targeted feedback and, to a lesser extent its proportions, can be overestimated if supervisors change their behavior as a result of being observed. We minimized this effect by not disclosing the focus of the study, involving residents of the program, and using realistic clinical cases with the typical time constraints of in-hospital supervision. We based our analysis on the amount of time rather than occurrence. Duration of feedback is not a guarantee of quality and presents drawbacks, especially when looking at the goal of feedback. Probing or clarifying questions may be inherently short; they may be undervalued if we use units of time as a measure of their educational impact.^13,14,32 

We designed cases that represented a challenge for 3 intrinsic roles; therefore, our results cannot be extrapolated to all roles or all clinical encounters, especially “simpler” ones. We did not provide feedback support tools (eg, encounter cards), which are known to increase the proportion of feedback on intrinsic roles, but difficult to adopt in every case discussion.^24,25  As for the generalizability of our observations, many factors influence supervisors' feedback-giving behavior: the clinical organization, the department's feedback culture, the educational program, as well as supervisors' training, educational goals, communication skills, or well-being.³⁶  In particular, our sample size did not allow an analysis of the impact of supervisors' experience and training.²⁷ 

Case discussions represent frequent opportunities for substantial feedback on intrinsic roles, largely aligned with the clinical case.

The authors would like to thank the supervisors for graciously giving of their time; Dr. Patricia Lemieux (internal medicine resident), Dr. Sara Di-Battista (internal medicine resident), Dr. Marc Vincent (radiology resident) for their major contributions as team members. Dr. Isabelle Martin-Zément, Dr. Marie-Pier Champagne, Dr. Caroline Simard, Mr. Pascal Lalancette, Ms. Anush Kirakosyan and Ms. Claudie Michaud-Couture for their help with the analysis; The Department of Medical Imaging (Dr. André Lamarre) and the Department of Medicine (Dr. Pierre Leblanc and Dr. Jacques Couët) for supporting this project; Mr. Muhamy Akbar for editing the figures, Mr. Keith Goddard for proofreading the first version of the manuscript, and Mr. Douglas Michael Massing for improving the final version.