ICRE Top Research Abstracts

Introduction: A recent independent review on diversity and inclusivity in the surgical profession highlighted concerns that barriers to career progression exist for some groups of individuals. Successful completion of the Membership Examination of the Surgical Royal Colleges of Great Britain and Ireland (MRCS) is prerequisite for entry into higher surgical residency training in the United Kingdom. We aimed to characterize the relationship between socio-demographic differences and MRCS success to investigate whether this gatekeeper examination is a cause of inequity in surgical career progression.

Methods: Candidates attempting MRCS Part A (n=5780) and Part B (n=2600) between 2013 and 2019 were linked to detailed socio-demographic data using the UK Medical Education Database (UKMED). Chi-square tests established univariate associations with MRCS success. Multiple logistic regression identified independent predictors of success at first attempt, adjusted for medical school performance.

Results: Differences in pass rates were statistically significant according to age, sex, ethnicity, childhood socioeconomic status, and educational background (all P<.05). Men (OR 2.34, 95% CI 1.87-2.92) and non-graduates (those who studied medicine without a prior degree) (OR 1.98, 95% CI 1.44-2.74) were significantly more likely to pass MRCS Part A. Non-graduates (OR 1.77, 95% CI 1.15-2.71) and those who attended a fee-paying (non-state funded) high school (OR 1.51, 95% CI 1.08-2.10) were more likely to pass MRCS Part B. Black and minority ethnic groups were significantly less likely to pass MRCS Part B (OR 0.41, 95% CI 0.18-0.92 for Black candidates and OR 0.49, 95% CI 0.35-0.69 for Asian candidates) compared to White candidates.

Conclusions: We found significant group-level differential attainment at MRCS that is not attributable to learner deficit. Training providers now have a responsibility to identify and address the causes of these attainment differences. This use of the novel UKMED database also demonstrates a new gold standard by which differential attainment should be investigated in all high-stakes international examinations.

R. Ellis,¹ P. Brennan,² A. Lee,¹ D. Scrimgeour,³ J. Cleland⁴

¹University of Aberdeen, Aberdeen, United Kingdom; ²Queen Alexandra Hospital, Portsmouth, United Kingdom; ³Aberdeen Royal Infirmary, Aberdeen, United Kingdom; ⁴Nanyang Technological University, Singapore, Singapore

Introduction: Worldwide, medical education pivoted to maintain core functions during the COVID-19 pandemic, accreditation included. Under principles endorsed by the Canadian Residency Accreditation Consortium (CanRAC) for continuing accreditation activities during the pandemic, the accrediting colleges conducted regular accreditation reviews virtually during 2020-2021. The feasibility, effectiveness, rigor, and satisfaction of the virtual process were evaluated.

Methods: Virtual accreditation reviews were conducted using secure file sharing and video conferencing technology for 2 large Canadian institutions and each of their active programs. A 32-question survey evaluated the virtual review process and experience for postgraduate office staff, program directors, residents, faculty, and surveyors (n=593). Focus groups were held with college staff (n=9).

Results: Stakeholders reported that the virtual process was feasible; however, there were some differences in perceived effectiveness between the 2 accreditation reviews. Many surveyors (47%) reported challenges reading body language but remained largely able to prehensively evaluate the program (86%). Most stakeholders (69%) reported that the virtual review had the same level of rigor as an in-person review, with 9% indicating an increased level of rigor. Staff focus groups revealed the additional complexities (volume of work, technology demands) and impacts (well-being, communications, team dynamics) of conducting large-scale regular accreditation reviews virtually.

Conclusions: Regular accreditation reviews can effectively shift to the virtual environment without compromising the integrity of the process. However, stakeholders receiving accreditation perceived the virtual environment more positively than the providers, including volunteer surveyors and college staff. There was an overall sentiment that large-scale regular accreditation reviews be conducted in-person, when possible.

A. Ronson,¹ S. Taber,¹ A. Ray,¹ T. Phillips,¹ H. Plant,² R. Almond,² L. Samson³

¹Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada; ²College of Family Physicians of Canada, Toronto, ON, Canada; ³College des médecins du Quebec, Montreal, QC, Canada

Introduction: Entrustable professional activities (EPAs) are the building blocks of assessment within competency-based medical education (CBME) in Canada. Although CBME aims to be learner-centered, the current literature largely focuses on faculty or administrator perspectives. However, understanding the resident perspective is essential, particularly since assessments perceived to be inauthentic or inaccurate may interfere with learning and professional development. Our study explores residents' real-world perspectives of EPAs, including their perceived impact on learning.

Methods: Using a constructivist grounded theory approach, we conducted 18 semi-structured interviews with residents from disciplines that have implemented CBME at one tertiary care academic center in Canada. Data collection and analysis occurred iteratively, and themes were entified using constant comparative analysis.

Results: Residents' perspectives were mixed, seemingly influenced by the attitudes and behaviors of their respective programs and faculty. In programs where EPAs were perceived as valuable, residents described a culture where EPAs were seamlessly integrated into daily routine. Faculty were both capable and willing to do these assessments, and residents intentionally sought EPAs for skillsets in which they were less confident, with the aim of eliciting specific, actionable feedback. Conversely, other programs were described as having less faculty buy-in, and residents perceived EPAs as a strictly quantitative requirement that ultimately interfered with learning and caused undue stress. In turn, these residents completed EPAs solely to fulfill perceived program requirements and advance to the next stage.

Conclusions: Authenticity matters. For EPAs to be perceived as truly reflective of residents' lived experiences and as a feasible means of formative feedback and accurate assessments, the CBME learner-centered approach necessitates a supportive educational environment with palpable faculty buy-in. Residency programs must build a culture that incorporates frequent low-stakes EPA assessments into daily routine and communicate EPAs as authentic facilitators of formative learning rather than arbitrary checklists for generating quantitative metrics.

E. Ahn, K. LaDonna, J. Landreville, W. Cheung

University of Ottawa, Ottawa, ON, Canada

Editor's Note: The following are the Top Resident Research abstracts selected by the Journal of Graduate Medical Education and the Royal College of Physicians and Surgeons of Canada for the 2022 International Conference on Residency Education (ICRE).

Introduction: With the adoption of competency-based medical education there have been calls for greater use of direct observation for feedback and assessment. However, a considerable amount of assessment occurs through means other than directly watching a trainee perform a task. The nuances regarding the value afforded by these indirect observations have been scarcely researched. Therefore, this study examined how residents and attendings understand and operationalize direct and indirect observation.

Methods: A constructivist grounded theory approach was used to interview 10 attendings and 8 residents at The Ottawa Hospital about their experiences with direct and indirect observation. The role of each type of observation and its perceived impact on learning was explored. Data collection and analysis were conducted iteratively. Themes were identified using constant comparative analysis.

Results: Direct observation was preferred for higher acuity situations and assessment of communication and leadership skills. Indirect observation was consistently utilized across specialties and operationalized in many forms. Attendings felt they could provide an accurate assessment of the learner's clinical reasoning and management plans. Resident physicians valued indirect observation to further their independent practice. However, resident physicians did not consistently find feedback based on indirect observation to be credible. This tension seemed to stem from residents not being aware of how indirect observation informed attending physicians' judgements. Resident physicians with more insight on the methods of indirect observation perceived this feedback as credible.

Conclusions: Consistent with prior literature, this study found that direct observation was preferred in certain situations. However, participants identified that indirect observation may be more useful and appropriate for the assessment of clinical reasoning. Residents with a deeper understanding of how indirect methods of observation informed attending physicians' assessments found this feedback more credible. For indirect observation to be effective, learners should be made aware of how indirect observation can inform judgments of their performance.

J. Landreville, K. O'Connell, N. Dudek, A. McDougall, M. Reed, W. Cheung

University of Ottawa, Ottawa, ON, Canada

Introduction: While Competence by Design (CBD) guidelines explicitly propose the academic advisor (AA) role, each residency program must decide whether and how to implement this new role. Such advice likely generates uncertainty and heterogeneity in how AAs are trained and how they function to “monitor and advise” residents. We sought to clarify how AA-resident dyads experienced their meetings as opportunities to influence residents' approaches to learning.

Methods: Closely following each of their 6 meetings during 2 years of internal medicine residency, we conducted individual debriefs with AAs and with residents from 9 dyads. Transcripts were analyzed using a theory-informed, abductive framework analysis with self-regulated learning sensitizing concepts.

Results: We analyzed 46 resident and 39 AA debriefs (15-20 minutes) and produced a rich description of how AAs and residents engaged in self- and co-regulated learning. Participants developed their own expectations for the AA role, which did not always align. The “stances” they adopted served to guide their perceptions and govern their activities with each other, and in the CBD system more broadly. We observed unique patterns in how stances shifted or evolved, in how this impacted the co-construction of meaning and strategies, and in the factors influencing the dyad over the 2 years.

Conclusions: We observed important variation in how AAs and residents come together to deliberately and intentionally self- and co-regulate each resident's learning progress. In studying how this dyadic relationship forms and functions over time, our findings have implications for how to better prepare AAs and residents for learning-focused interactions in CBD.

L. Branfield Day,¹ D. Butler,² R. Shah,¹ A. Kuper,¹ W. Tavares,¹ S. Ginsburg,¹ L. Stroud,¹ R. Brydges¹

¹University of Toronto, Toronto, ON, Canada; ²University of British Columbia, Vancouver, BC

Introduction: Surgical training programs have a high prevalence of trainee stress and burnout. Formal mentorship programs (FMP) have been shown to alleviate these factors and improve quality of life (QOL) in short-term follow-up. This study aims to determine the long-term effects of an FMP on the well-being of a single center cohort of surgical trainees.

Methods: In this prospective interventional study, a voluntary FMP was established at a surgical training program comprised of 8 residents. The primary outcome was the well-being of the trainee participating in the FMP. To quantitatively measure stress and burnout, the Perceived Stress Survey (PSS) and Maslach Burnout Inventory (MBI) were administered at baseline, 3, 6, 9, 12, 18, and 24 months. The World Health Organization Quality of Life (WHOQOL-BREF) questionnaire was administered at baseline, 12 months, and 24 months. Continuous data were expressed as means and analyzed using analysis of variance. Comparisons of proportions were performed using the chi-square test.

Results: All 8 residents participated in the FMP (100% response rate). Baseline levels of stress and burnout were high among residents with an average PSS of 18.5 and MBI of 47.6, 50.6, and 16.5 for the emotional, depersonalization, and personal achievement domains, respectively. After FMP implementation, PSS was reduced to 7.9 at 12 months (P=.001). These scores were sustained at 24 months (6.8, P>.99). MBI scores improved in emotional exhaustion (14.9, P<.0001), depersonalization (20.1, P<.0001), and personal achievement (40.1, P<.0001) at 12 months. All these benefits were sustained at 24 months. Baseline overall WHOQOL scores reflected low QOL (71.9). These significantly improved at 12 months (37.5, P=.002) with further improvement by 24 months (17.2, P=.03).

Conclusions: Long-term follow-up of a previously successful academic surgical FMP showed lasting improvements in stress, burnout and overall QOL, despite new life challenges. A limitation of this study is that it lacks a control group.

S. Mohajeri,¹ H. Seikaly,² Y. Alrajhi,² E. Wright,² H. El-Hakim,² H. Zhang¹

¹McMaster University, Hamilton, ON, Canada; ²University of Alberta, Edmonton, AB, Canada

Introduction: Recruitment and retention of women in surgery is limited by a reputation that surgery is not family friendly. Recent work has focused on why women leave surgery, and there is little data on how most manage to stay. In this qualitative phenomenological study, we interviewed 11 female senior trainees and junior surgical consultants in the United Kingdom and conducted a thematic analysis of their challenges and solutions, impact on professional identity, and possible solutions.

Methods: Regarding work-life balance, most participants relied on their partner and family for wraparoldcare, as the unpredictable nature of operating meant leaving work on time created tension with trainers. For those who hadn't moved near family, childcare was a significant financial burden, which meant for some they couldn't afford to restart training in another specialty. Many found their new parental responsibilities honed their time management, and the joy of motherhood balanced well with the satisfaction of surgical careers.

Results: Lack of a 2-way conversation with training program directors was a significant stressor, creating conflict during the annual assessment process which made trainees more assertive in protecting their right to training. Participants described a shift in professional identity away from a “rising star” persona to someone with changed priorities needing to defend their life balance. Lack of local knowledge regarding occupational health and maternity pay meant trainees rely on informal networking to have safe working conditions and correct pay during the maternity period. Most cited a single conversation with a supportive supervisor that prevented them leaving the profession.

Conclusions: This study shows that balancing motherhood and surgery in the United Kingdom is feasible, but interventions in deaneries and training organizations could lead us to a future where they are better supported. Normalizing motherhood is needed to normalize modern family structures and is essential for gender equity.

R. Fisher, S. Smith

Imperial College London, London, United Kingdom

Introduction: Residents play a pivotal role in providing feedback to medical students in clinical education, but most studies have focused on faculty-student encounters. There is also conflicting evidence as to the value of resident feedback from the student perspective, which sits in contrast to the central role residents play in student learning. We undertook a qualitative study to explore medical students' perceptions of feedback experiences with residents, addressing how and why resident feedback plays a role in their clinical education.

Methods: We used qualitative interpretive description, informed by phenomenology, as our guiding methodology. We conducted 24 semi-structured interviews with third- and fourth-year medical students at one institution. The interviews aimed to foster rich discussion about students' feedback experiences with residents during clinical rotations. Data collection and analysis proceeded iteratively. Initial interviews were independently open coded by 3 investigators and then collaboratively refined and organized into themes. During the final stages of analysis, we refined our themes by drawing on a sociocultural perspective to illuminate the role of relationships in students' experiences.

Results: We identified several themes that reflected how and why students valued feedback conversations with residents. Residents, both by virtue of their role as well as active strategies they undertook, were able to build strong interpersonal relationships with students. The interpersonal relationship shaped students' receptivity to feedback conversations. Resident feedback influenced how students approached learning and working in the clinical environment, as well as students' visions of their future selves.

Conclusions: Residents are uniquely positioned to create supportive clinical learning environments for students in which feedback conversations can flourish. The resident-student relationship embodies many elements of the “ideal” feedback relationship that has been so elusive in faculty-learner encounters. Focusing educational efforts on resident feedback conversations has the potential to significantly impact the feedback culture of our clinical environments.

S. Wong, C. Luo, G. MacDonald, R. Hatala

University of British Columbia, Vancouver, BC, Canada