A Unique Lens: Understanding What Nurses Are Best Positioned to Assess About Residents

Resident physicians interact with multiple health care professionals throughout their training, including nurses, peers, supervising residents, chief residents, and attending physicians; however, much of the structured feedback elicited for residents has historically been from attending physicians.¹  Feedback from nurses, specifically, has been found to provide a perspective that differs from other assessors² ; however, there is limited understanding of the specific resident behaviors that nurses are best equipped to assess and how best to approach the implementation of a nurse-to-resident feedback system.

Multisource feedback is the method through which individuals are assessed by multiple stakeholders, including nurses, peers, and patients on key behaviors, and it is increasingly recognized by the medical community as a valid and critical means by which to inform resident development.^3,4  While feedback from attending physicians provides data on medical knowledge and clinical skills,¹  feedback from nurses, peers, and patients, when elicited, provides valuable information on professionalism, teamwork, and communication.²  Further, attending physicians have previously indicated that they were less able to provide feedback on behaviors requiring direct observation with patients, such as performing the physical examination, explaining problems, sharing decisions, and listening carefully.⁵  While multisource feedback can enrich the development of residents, and Accreditation Council for Graduate Medical Education requirements state that programs must use multiple evaluators in resident evaluation, implementation has been limited.⁶ 

Nurses, specifically, have perspectives that differ from those of attending physicians.⁷  Given that many of a resident's workplace activities are not directly observed by a physician supervisor, nursing assessments can provide reliable and different perspectives on competencies such as communication and professionalism.⁸  For example, recent studies found that internal medicine and emergency medicine nurses could provide feedback on residents' efficiency, kindness, communication, advocacy, leadership, collaboration, and professionalism.^7,9,10  A randomized controlled trial found that nursing feedback did lead to improvement in pediatric residents' communication skills and professional behavior.¹¹  Importantly, residents have found nursing feedback to be valuable.⁹  Though physicians perceive nursing feedback favorably, research has shown that residents are hesitant to accept feedback if they perceive that the evaluators do not understand the resident role or are being asked to comment on areas outside their expertise.¹²  The prior studies on nurses' perspectives are largely focused on the broad categories of evaluation topics on which nurses can provide feedback (eg, communication); however, the specific behaviors within those categories remain unclear. Understanding what specific behaviors nurses are best suited to assess is necessary for nurses to successfully provide and residents to be willing to receive nursing feedback.

The aim of this study was to understand the specific resident behaviors that nurses are best positioned to assess, from the perspectives of both nurses and residents.

We performed a qualitative study employing an inductive approach to thematic analysis of focus groups to explore resident and nurse perspectives on the resident behaviors on which nurses are best equipped to provide feedback.^13,14 

We invited by email pediatric residents and inpatient nurses who work at Boston Children's Hospital, a free-standing urban children's hospital. Nurses were eligible for participation if they had worked on their unit for at least 2 years and had experience working with residents. We invited nurses who were in the roles of charge nurse, preceptor nurse (nurses who support and educate newly hired staff by providing clinical orientation),¹⁵  or resource nurse (experienced nurses who support patient care and nursing workflow through consultation, education, or assistance).¹⁶  We chose nurses in these leadership roles given their significant experience in frontline nursing positions in addition to the fact that they provide feedback frequently (to nurses and other interprofessional clinicians). Characteristics of the participants are shown in Table 1. Verbal informed consent was obtained from all participants.

We developed 2 focus group guides—one for resident focus groups and one for nursing focus groups (provided as online supplementary data). The resident focus group guide included questions on multiple types of evaluators, including nurses, peers, fellows, and attendings; this study focuses exclusively on their responses to the questions about nurses. We chose focus groups to encourage open discussion and idea sharing about nursing feedback. K.S.D., a senior (postgraduate year [PGY]-3) resident at the time, piloted the resident guide with a group of recent residency graduates who were close enough to their residency experiences to reflect on the questions. D.P.D., a nurse educator, piloted the nursing guide with a group of nurses with experience working with residents but who were no longer in direct patient care roles. The pilot participants responded to the questions and gave feedback on the questions. The guides were subsequently refined based on this feedback.

In July and August 2020, 2 authors (K.S.D., D.P.D.) who have training in focus group facilitation each conducted five 60-minute focus groups with 3 to 5 pediatric residents and pediatric nurses, respectively. We intentionally had a resident lead the resident focus groups and a nurse lead the nurse focus groups to ensure that participants could speak freely about their perspectives. K.S.D. was a senior resident (PGY-3) at the time of conducting the focus groups. We purposely conducted the resident focus groups prior to her commencing chief residency so that the focus group facilitator was someone uninvolved with resident evaluation. There were 2 intern focus groups and 3 resident (PGY-2+) focus groups. First-year residents participated in focus groups separately from second- and third-year residents to capture potential differences in perspectives and to ensure interns could speak freely. We conducted virtual focus groups via the Zoom video-based conferencing platform. Focus groups were audio recorded, transcribed, and anonymized prior to analysis.

The primary coding team consisted of 2 investigators (K.S.D., C.H.M.) who used thematic analysis to inductively generate codes. K.S.D. and C.H.M. independently read 2 resident and 2 nursing focus group transcripts to create an initial codebook which then they used to analyze and code all the transcripts, meeting multiple times throughout the process to discuss new codes and address areas of discrepancy. After the initial coding process, we organized the codes and constructed them into themes. We performed iterative data analysis, returning to transcripts that had already been reviewed if new insights emerged during analysis. We reached thematic saturation prior to analyzing the final transcript. The larger research group met to discuss theme development. We used the qualitative analysis software Dedoose, version 8.3.35 (SocioCultural Research Consultants LLC) to facilitate data organization and analysis. We kept an audit trail to ensure the study's trustworthiness by maintaining documents with each iteration of the interview guide during its development and codebook during its creation with our motives for any changes. Representative quotations for the manuscript were collected during coding and final review of the transcripts. Final quotations were then selected through iterative discussion with the larger research group based on those which were most demonstrative of themes and subthemes.

We maintained awareness of reflexivity as we analyzed the data. K.S.D. was a pediatric senior resident at the time of conducting the focus groups and chief resident at the time of the data analysis. As chief resident, she often provided feedback to residents, received informal feedback from nurses about residents, and interpreted faculty evaluations of residents. C.H.M. and A.S.W. are pediatric hospitalists and residency associate program directors who often provide feedback to residents and must frequently interpret evaluations and recognize missing data in their roles on the clinical competency committee. L.E.C. had recently graduated from the pediatric residency program and had frequent experiences with nurses. Thus, given our personal experiences with residents clinically, with nurses clinically and in informal discussions, with resident feedback, and with the evaluation system as a whole, we have our own opinions about how nurses' expertise can and should be utilized in the evaluation process. D.P.D. is a registered nurse at Boston Children's Hospital and a nursing professional development specialist who has had frequent experiences with residents and personal reflections on what he personally could assess. The study team regularly revisited the data to ensure that our interpretations were from the data itself rather than influenced by our own experiences. To strengthen the study's credibility, we conducted the focus groups with both residents and nurses to enable data triangulation, and the multiple roles represented in our investigator group enabled investigator triangulation. Member checking was not performed.

The study was deemed exempt from review by the Boston Children's Hospital Institutional Review Board.

Twenty residents and 17 nurses participated in 10 focus groups. Quotations are identified by nurse (N) or resident (R), followed by focus group number and participant number. For example, participant 2 in nursing focus group 4 is represented by N4-2. While the focus group guides primed participants to focus on written feedback, focus group discussions focused both on written and verbal feedback.

Residents and nurses agreed that nurses offer a distinctive and important perspective. We identified 3 major themes about the areas on which both resident and nurse participants agreed that nurses are able to provide feedback: (1) Nurses are uniquely positioned to provide feedback on residents' interprofessional collaborative practice; (2) Nurses are uniquely positioned to provide feedback on residents' communication with patients and their families; and (3) Nurses are uniquely positioned to provide feedback on behalf of patients and families. Within each of these, we identified subthemes noting specific behaviors on which nurses can provide residents with feedback. Additionally, we identified a fourth theme encompassing 2 topics that were discussed but thought to be areas that may not be ideal for nursing feedback—medical decision-making and resident responsiveness to nursing concerns. There were no differences noted between what first-year residents and more senior residents reported. Themes 1 to 3 and subthemes with representative quotes are listed in Tables 2-4 and further elaborated on below.

Both nursing and resident participants felt that the extensive interaction between residents and nurses provides nurses with perspective on the efficacy of residents' interprofessional collaborative practice, especially communication with colleagues. Nurses can provide feedback on many different facets of communication, including coordination of communication with families, communication surrounding orders, and communication around escalation of care.

Nurses expressed that they have an ability to provide feedback on whether residents take advantage of opportunities to collaborate, such as asking for nursing input during rounds or trying to interview a patient and family together during an admission. Nurses also noted that they could give residents feedback on their communication about orders, including the presence of any new orders and the rationale behind orders. Additionally, nurses felt they could give feedback to residents on the respectfulness of their communication: “Nursing can provide unique feedback on all sorts of aspects of communication. Is it respectful? It is timely? Are [residents] closing the loop? Are they being clear when communicating?” (N3-2).

Similarly, residents desired feedback from nurses on how well they communicate their thought processes regarding patients' clinical trajectories, changes to patients' plans, order clarity, and ability to perform closed-loop communication. Residents also desired feedback on how well they collaborate during situations requiring escalation of care, such as end-of-life care or transfer to the intensive care unit; nurses also expressed desire to provide this feedback. Finally, residents desired feedback from nurses on how they handle situations in which they were faced with high patient volumes with a “higher potential for something to happen or be missed” (R5-2) and when responsible for caring for patients with whom they were less familiar, such as overnight.

Nursing and resident participants felt nurses were positioned to provide residents with feedback regarding their communication with patients and their families since nurses are often part of the conversation—either as an observer or direct participant. “Sometimes we'll be in the room either giving meds or doing other tasks where [residents] might not necessarily think that we're watching or listening, because we're busy doing other things. Because we're at the bedside for 12 hours…we're always sort of the eyes and ears and see interactions with the families and can see when things go well, when things go not so well” (N4-6).

Specifically, nurse participants noted that they can provide residents with feedback on their bedside manner (eg, listening skills, eye contact, decision to sit down, or turn on the lights) and visibility to families, meaning how often they check in with patients at the bedside. Residents agreed that nurses could provide feedback on their communication skills with families, including their ability to defuse difficult situations or deliver bad news. They also wanted feedback on their ability to explain medical concepts in a family-centered way, and both nurses and residents felt that nurses were well positioned to provide this feedback.

Resident and nurse participants described that nurses' days are spent largely at the bedside and therefore they get to know families in a different way than residents. Nurses noted that they care for patients over a series of days through a long admission or across multiple admissions and thus have insight into specific patients that is helpful in medical decision-making (eg, what medications typically work for a patient in pain or how a patient is best examined). Nurses' extensive time at the bedside may allow them to better understand patients' clinical trajectories, such as whether a patient's pain is controlled: “We're at the bedside, we see the whole picture. We're the ones that interact with—see the patients throughout the entire day...We see it from a different perspective as well—not just the doctor's perspective but the whole picture” (N2-3). This insight was felt by nurses to enable them to provide residents with patient-specific coaching and feedback. Resident participants also reported that families sometimes provide honest feedback to nurses about their impressions of their interactions with residents that families would not tell residents directly, so nurses may be able to serve as a proxy for feedback from patients and families.

Resident participants did not think nurses should provide feedback on their medical decision-making. One resident participant noted “feedback should be…more communication based and less actual medical decision-making” (R3-5). Residents cited differences in training such that they felt it was an unfair topic of evaluation for both parties. In line with this perception, no nurses specifically commented on medical decision-making as a potential area of feedback.

Finally, “responsiveness” (ie, responding to nurses' pages and concerns about patients) was brought up as a potentially important area for feedback by nurse and resident participants but was overall polarizing because responsiveness was noted often to be confounded by external factors. Resident participants recognized the importance of responding to nurses in a timely manner, although expressed concern that the desired action was not always clear based on a nurse's initial outreach. For example, one resident noted: “Sometimes nurses will page me, and I don't know exactly what they want from the page—if it's really just a true FYI or if it's ‘we want you to come to the bedside'” (R3-1). Residents expressed concern that nurses might not always be aware of the acuity of their other tasks that might prevent them from being immediately responsive. Residents and nurses were concerned that nurses might not understand the scope of residents' role and responsibilities, which makes it challenging to provide feedback and puts their degree of responsiveness into context: “It's difficult for [nurses] to really give feedback when we don't necessarily know exactly what they are doing…we don't even realize the amount of things that they have to deal with during the day” (N1-1).

In this qualitative study, pediatric resident and nurse participants identified specific resident behaviors on inpatient rotations on which nurses are best positioned to provide feedback. Both groups agreed that nurses have the potential to provide valuable perspectives on resident interdisciplinary collaboration, communication, and interaction with patients and families, and offered specific behaviors within these domains for potential feedback.

The results of our study complement existing literature on nurse-to-resident feedback. Prior studies have found that nurses are poised to provide residents with feedback on their communication skills, collaboration, and professionalism.^7,9,10,17-19  Our study similarly found collaboration and communication as domains on which nurses are well-equipped to provide feedback and further adds to the literature by elucidating the specific behaviors within these broader categories that nurses can assess. While residents are often assessed on their medical knowledge by supervising attendings and peers, both nurse and resident participants felt that nurses should not be asked to assess residents on their medical knowledge or clinical decision-making, which also aligns with prior work.²⁰ 

A novel finding in our study was that participants said that nurses might be able to provide feedback on behalf of the patients and families, acting as a proxy for the family experience. Patient feedback adds a new perspective that differs from that of faculty evaluations²¹ ; however, patient feedback is the most difficult perspective to capture in multisource feedback.²²  There are numerous factors contributing to this challenge including patient/family availability and difficulty in identifying specific residents.²³  Given these challenges, our finding that nurses might be able to provide feedback on behalf of the patient could potentially lend itself to a more feasible interim way to obtain this important perspective, while ongoing research clarifies the best means to facilitate direct patient feedback. This finding should be approached cautiously as our research did not explore if nursing feedback does in fact align with the feedback that patients/families would give themselves.

Resident and nurse participants agreed that asking nurses to assess residents would add an alternate perspective and enhance the working relationship between residents and nurses. While supervising attending physicians are the most frequent resident assessors, they do not directly observe residents performing most of their daily activities. Further, formal direct observation performed by faculty has been shown to be flawed, partially due to an “observer effect” whereby trainees operate differently than their normal practice.²⁴  Including nurses in the feedback process provides feedback obtained via more authentic observation. Additionally, as competency-based medical education becomes increasingly prevalent, programs will need to ensure robust and frequent assessments about residents' performance.²⁵  As there is likely missing data about residents when mostly assessed by attending physicians, nursing evaluation can be a valuable component of competency-based medical education assessment.

On the other hand, in our study, residents and nurses were concerned that nurses might not have a full understanding of residents' day-to-day activities or the scope of their role. Residents felt this may impact nurses' ability to give feedback on their responsiveness without knowledge of the entire context of their jobs. Thus, it would be important to mitigate these concerns by providing nurses with this context prior to the assessment process or by not asking about resident responsiveness. Further investigation into interventions that would address this concern might be meaningful.

Nurses are best able to assess specific resident behaviors and prior studies have found that residents are more likely to accept feedback if they believe the questions asked were within the expertise of the evaluator.¹²  Thus, it would be beneficial to have distinct nursing feedback forms with questions that are specific to what nurses are best-suited to assess. Given varied perspectives of the different evaluators who work with residents, one generalized form for all evaluators (eg, attendings, peers, nurses, etc) is insufficient.

There are limitations to our study. The study population included pediatric residents and nurses from one residency training program and one children's hospital in an inpatient environment; therefore, our study might be less transferrable to residents in different training environments such as adult hospital, outpatient, or emergency room settings. We sought to mitigate this by recruiting nurses with experience on multiple floors and from multiple services across the hospital. Additionally, many of the nurses in our focus groups had numerous years of experience and held leadership roles, so it is uncertain if these results are similarly applicable to early-stage nurses. Finally, our study only included residents and nurses, who may lack educational expertise. Gathering the perspectives of individuals with specific educational expertise, such as residency program directors, is a potential area of future study.

Nurses and residents value nursing feedback in resident development and express that nursing feedback provides a perspective not captured by other individuals, such as supervising attendings or peers. Our participants described how nurses can provide residents with feedback on their interprofessional collaboration skills, their communication with patients and their families, and on behalf of patients and families. Finally, medical decision-making and resident responsiveness may not be ideal topics for nursing feedback.

The authors would like to thank the participants of the focus groups for their contribution to this research study.