ICRE Top Research Papers

Background: The advent of competency-based education has led to concerns regarding reductionism in the assessment of clinical competence. This apprehension stems from a frequently utilized approach using the assessment of isolated competencies to build a complete picture of clinical competence. Others argue that the EPA framework complements the construct of competencies, as EPAs describe a unit of work and require a global approach to the assessment of the activities of a physician.

Methods: We designed a simulation-based workshop to discern whether the assessment of separate competencies subsequently aggregated is equivalent to the global assessment of EPAs. We assessed each student using individual competencies mapped to the core EPAs, a modified supervision scale, and a global statement regarding entrustment and readiness for residency. These assessments were compared to aggregate workplace-based assessment data on the various individual competencies from the core clerkships.

Results: Assessment data obtained using the individual competencies mapped to the EPAs were highly correlated with the assessment of individual competencies obtained in the workplace during core clerkships. However, these individual competency-based assessments did not correlate with EPA-based global supervision scale ratings, entrustment decisions, or perceived readiness for residency.

Conclusion: The global assessment of EPAs and the judgement of entrustment appear to be separate processes from aggregating the assessment of individual competencies. This may reflect variations in the approach to global assessment when compared to the assessment of individual competencies and the need to consider the construct of trustworthiness in addition to the learner's ability to perform the individual activities.

H. Caretta-Weyer¹

¹Stanford University School of Medicine, Palo Alto, CA

Background/Objective: Women in medicine report alarming rates of discrimination based on their gender, race, ethnicity, or sexual orientation. While formally reporting these events may be stigmatizing, the #MeToo movement empowered women physicians and physicians-in-training to share their experiences on Twitter. We explored these discussions to understand how women describe and grapple with discriminatory forces in the workplace.

Methods: We conducted a content analysis of approximately 10,000 Tweets linked with hashtags including, but not limited to, #WomenInMedicine, #BlackWomenInMedicine, and #LGBTQInMedicine. Tweets were systematically analyzed to identify thematic patterns.

Results: Tweets contained instances of being mistaken for support staff, being overlooked for—or discouraged from pursuing—professional opportunities, and feeling marginalized within medicine. Women also sought and provided support about experiences such as motherhood, infertility, and work-life integration. More commonly, however, women seemed to use Twitter to challenge stereotypes by sharing photographs representing the diversity of women working in medicine, to demand change, to seek allies to address discrimination, and to work to advance women's careers by both promoting their peers and recognizing their achievements.

Discussion: Twitter appears to provide a space for women in medicine to not only share discriminatory work experiences but also to collaborate and develop strategies to resist discrimination. Given the potential risks to women's careers when they identify workplace discrimination, Twitter may be an important venue for both creating support networks and advocating for change. Future research will explore connections between emerging findings and known impediments to personal and professional well-being such as inaccurate self-assessment and burnout.

K. LaDonna¹, R. Pack², K. Chien¹, S. Ginsburg³, L. Madrazo⁴, C. Watling⁵, E. Field⁵

¹University of Ottawa, Ottawa, ON, Canada; ²Centre for Education Research & Innovation, Western University, London, ON, Canada; ³University of Toronto, Toronto, ON, Canada; ⁴Schulich School of Medicine & Dentistry, Western University, London, ON, Canada; ⁵Western University, London, ON, Canada

Introduction: The Australian Orthopaedic Association (AOA) is responsible for educating and training orthopaedic surgeons in Australia. The new training curriculum (AOA 21) involves advanced technological support for assessment of residents' performance, including 4 types of workplace-based assessment (WBA), brief feedback episodes and a national electronic logbook of procedures. Differences in performance on in-training assessments based on the resident and supervisor gender is demonstrated in the surgical, medical, and non-medical education literature.

Methods: Over 15,000 WBAs, 10,000 feedback episodes, and almost 300,000 elogbook entries for approximately 270 Australian orthopaedic residents over a 3-year period (2017–2019) were analyzed. Our primary outcome variable was the performance rating of trainees, as provided by their supervisors. We performed a regression analysis of the association between trainee gender and trainee rating, and the association between supervisor gender and trainee rating. We also performed a stratified analysis to test for interaction between supervisor gender and trainee gender.

Results: Regression analyses adjusted for region, year, and case complexity showed that male trainees received higher ratings than female trainees, and that male supervisors rated trainees lower than female supervisors. There were some differences in the stratified analyses, but formal tests for interaction were not statistically significant. eLogbook differences reduced as the resident progressed through the AOA 21 training program. However, WBA and feedback episode differences did not reduce with resident seniority.

Discussion: Gender differences in assessment, predicted by the current literature, are confirmed to occur in Australian orthopaedic surgery training. Possible causes and management strategies are discussed.

I. W. Incoll¹, J. N. Atkin², J. Green³, I. Harris⁴

¹Australian Orthopaedic Association, Erina Fair, NSW, Australia; ²Nikta Projects, Sydney, NSW, Australia; ³Australian Orthopaedic Association, Sydney, NSW, Australia; ⁴UNSW Sydney, Sydney, NSW, Australia

Editor's Note: The following are the Top Resident Research abstracts selected by the JGME and the Royal College of Physicians and Surgeons of Canada for the 2020 ICRE kickoff. The Top QI abstracts will be published in the April 2021 issue, along with the full listing of submitted abstracts online. Underlined author names indicate presenting author at the conference.

Top Resident Winner

Introduction: Assessment of residents is reported to differ by gender, yet little is known about the potential effects of these variations. We sought to understand if male and female internal medicine (IM) residents perceive differences in their experiences of being assessed and receiving feedback.

Methods: We used constructivist grounded theory as an approach to data collection and interpretation, which occurred iteratively and simultaneously. We conducted semi-structured focus groups and interviews with IM residents, divided by gender and by training level. Twenty-two residents participated (8 male, 14 female).

Results: We found a profound difference in the experience of receiving feedback and assessments between male and female residents. Themes of power, authority, display of confidence, and clothing/appearance diverged between male and female residents. For example, in contrast to their male counterparts, women relied on symbols such as a white coat, stethoscope, and demure clothing to establish and justify their physicianship. Women also encountered conflicting feedback from supervisors regarding confidence and assertiveness (eg, told to be more or less assertive), often resulting in self-censorship, whereas men rarely received similar feedback.

Conclusions: Gendered differences in the experiences of working and being assessed on IM wards may not be easily captured by standard assessments. Our findings present opportunities for hypothesis generation and further study. We plan to triangulate these findings with scores and comments from PGY-1 EPAs. Future directions include formally recognizing gender inequities in assessment and feedback and putting in place measures to address such biases.

M. J. Billick, J. C. Rassos, S. Ginsburg

University of Toronto, Toronto, ON, Canada

Introduction: Inclusion of patients in the medical educational environment can increase professional behavior, mitigate demeaning language, increase humanistic care, and demonstrate role modelling opportunities—all components of a positive learning environment (Branch et al, 2001; Weissman et al, 2006). However, the patient, who is the focus of our care, has often been systematically excluded from discourse in the learning environment. In this study, we explored the ways in which patient inclusion affected dynamics of the educational and clinical environment.

Methods: This study took place in the Outpatient Internal Medicine Clinic at Vancouver General Hospital in British Columbia. Previously, trainees (residents and medical students) often reviewed their cases with staff physicians outside the patient room. In this study, all case review occurred in the room with the patient present. The preliminary sample included 9 staff physicians, 12 trainees, 9 patients, and 1 family member. One-on-one semi-structured interviews were then used to understand the experiences of the learner, the faculty, and most importantly, the patient.

Results: Descriptive thematic analysis revealed 3 themes: (1) Traditional teaching opportunities on medical knowledge were perceived to be limited; (2) Other teaching opportunities, especially on clinical skills and professionalism, were more apparent; and (3) Patients felt more comfortable, respected, and informed about their care.

Conclusion: This study showed that although inclusion of the patient in the medical learning environment may reduce traditional teaching opportunities, it can also enhance medical education by introducing other teaching opportunities and reinforcing positive professional values that importantly contributes to patient centeredness.

M. Li, E. Amari, H. Buckley, C. Canfield, B. Cheema, C. Cuncic, D. Ho, L. Nimmon, K. Veerapen, A. van Enk, K. Wisener, C. Holmes

University of British Columbia, Vancouver, Canada

Introduction: Podcasts are used as educational tools by physicians across multiple subspecialties and levels of training. However, there has been limited research understanding the ways in which trainees incorporate podcasts into informal medical education. In this study, we examine the motivation behind the use of internal medicine (IM) podcasts in the early years of postgraduate medical education.

Methods: We conducted semi-structured interviews (n = 8) with internal medicine residents in the PGY-1 and PGY-2 level. Interpretive description methodology was applied to identify pertinent themes across interviews. Data collection is ongoing.

Results: Early thematic analysis has revealed that “study guilt” is a main motivator for using IM podcasts. Participants endorse factors driving motivation to use podcasts including the desire for productivity, maximizing efficiency in spare time, remaining up-to-date in their knowledge, and the desire to expand their knowledge for both patient care and clinical performance.

Conclusion: Residents identify “study guilt” as a main motivator to use internal medicine podcasts in the informal medical education. Our participants used IM podcasts as a method to increase their overall productivity and assuage study guilt. Podcasts are uniquely positioned to target study guilt given the easy access to abundant sources and the requirement only for auditory attention, which allows them to be used for learning while multitasking. Understanding how and why residents reach for podcasts in their medical education will allow for the development of future medical resources designed specifically to meet these needs.

C. S. Leurer¹, L. Melvin², J. Fu¹, A. Lai¹

¹University of Toronto, Toronto, ON, Canada; ²Toronto Western Hospital, Department of General Internal Medicine, Toronto, ON, Canada

Background/Objective: There is an increasing emphasis on ambulatory care in Canadian general internal medicine programs. However, outpatient training opportunities are often limited. To maximize trainee educational experiences, we must define the competencies of ambulatory practice, including discharging patients from ambulatory clinics. We sought to understand how practicing physicians engage in discharge decision-making to patients' primary care providers in the outpatient setting.

Methods: We purposively sampled general internists who attend in the ambulatory clinic from 6 academic hospitals in Toronto. Twenty-three semi-structured interviews were conducted from October 2019 to January 2020. Data collection and analysis were iterative with constant comparison according to constructivist grounded theory.

Results: Several themes were identified that illuminate the decision to discharge a patient from the ambulatory clinic centered around the value added to the patient's care. These include: (1) Stability of the medical condition and/or diagnostic closure; (2) Accessibility of appropriate care; (3) Individual physician factors including risk tolerance and self-identified scope; and (4) Individual patient factors including frailty, comorbidity burden, and vulnerability. Trainee education around these decisions is informal and often relies on role modeling and supporting trainee independent decision-making under guidance.

Conclusions: Our study describes the physician-, patient-, and systems-related factors that influence general internists' discharge practices from ambulatory clinic. We believe these factors can be used as a framework for educating trainees regarding transitions of care decisions in ambulatory internal medicine. Explicitly defining important components of practice in the ambulatory setting will enhance internal medicine trainees' competencies in ambulatory practice.

S. Halani¹, L. Melvin², R. Cavalcanti¹

¹University of Toronto, Toronto, ON, Canada; ²Toronto Western Hospital, Toronto, ON, Canada

Background: The educational experience in operating rooms (ORs) plays a central role in the transformation of a trainee into a surgeon. As Canadian residency programs transition to CBME, and since most surgical competencies are attained in the OR, we investigated the perception of Canadian surgical residents about their OR learning environment.

Methods: An online questionnaire, including the validated 40-item Operating Room Educational Environment Measure (OREEM) inventory, was sent to all surgical residency programs in Canada. The OREEM assesses trainees' perceptions of the teaching surgeon, learning opportunities, operating room atmosphere, and workload. Each question is ranked from 1 (strongly disagree) to 5 (strongly agree).

Results: Four hundred thirty residents were included for final analysis. The overall mean OREEM score was 3.72 ± 0.4. “Atmosphere in the OR” was the subscale with the highest mean score (3.87 ± 0.5), while “supervision, workload, and support” had the lowest subscale mean score (3.49 ± 0.5). “I get paged during operations” was the item with the lowest score (1.98 ± 1.0), while “I feel discriminated against in the OR because of my race” was the item with the highest score (4.47 ± 0.8). The overall OREEM mean score for junior and senior residents was 3.67 ± 0.4 and 3.80 ± 0.4, respectively (P = .001). No significant differences were seen in the mean OREEM score between men and women or different surgical programs. Nevertheless, general surgery had the lowest “supervision, workload, and support” subscale score (3.27 ± 0.5; P < .001).

Conclusions: Residents' perception of overall learning environment in ORs may be considered satisfactory; however, several areas for potential improvement are identified.

L. Luo¹, Y. Marwan², A. Toobaie³, T. Benaroch¹, L. Snell²

¹Division of Orthopaedic Surgery, McGill University Health Centre, Montreal, QC, Canada; ²McGill University, Montreal, QC, Canada; ³Division of Orthopaedic Surgery, Western University, Londo, ON, Canada