Looking Beyond the Numbers: A Comparison of Operative Self-Efficacy, Supervision, and Case Volume in General Surgery Residency Open Access

The development of self-efficacy is essential for continued growth and lifelong learning, particularly in task- or procedurally oriented specialties such as surgery. Self-efficacy, or one’s belief in their ability to accomplish a specific task, is a concept described by cognitive psychologist Albert Bandura in the 1970s and comes from a combination of mastery experiences, social persuasion (coaching, feedback), vicarious experiences, and psychological responses.¹  However, there is evidence to suggest that surgery residents may be lacking in operative self-efficacy. A survey on self-efficacy of graduating postgraduate year (PGY) 5 residents was linked to the 2020 American Board of Surgery In-Training Examination (ABSITE), to ensure that all PGY-5 residents who took the examination had the option to complete the survey. It revealed that only 7.7% of graduating residents reported self-efficacy in all 10 commonly performed operations.²  This presents a challenge when considering graduating residents’ readiness for independent clinical practice.

While this study identified a significant deficit in resident self-efficacy, it also presents further questions as to what is contributing to the deficit and how it can be addressed. Kearse et al attempted to answer this question and identified a combination of institutional and individual factors, such as male sex, emphasis on autonomy, and peer/faculty socialization associated with higher self-efficacy.³  Performing operations with greater levels of resident responsibility and in a teaching assistant role were also correlated with higher self-efficacy.³  While this takes a step toward better understanding and thereby addressing the self-efficacy deficit, it provides little information about case-level experiences that might shape a resident’s perceptions of their own abilities. These case-level experiences include elements of both quantity and quality, such as number of cases performed, frequency of performance, and degree of operative independence when performing the case. At present, surgery residents log their operative cases as the primary method to track operative experience during training. While such logs document case volume, they do not capture other elements of the case-level experiences that may be important to consider in preparing residents for independent clinical practice.

These case-level experiences are important to evaluate in the context of Bandura’s self-efficacy theory, where there are many different facets that contribute to the development of self-efficacy.¹  Most of these facets are centered around prior experience, such number of times a case was performed, operative entrustment, vicarious experiences watching someone else perform an operation, and the physiological response associated with these experiences. Thus, the theory of self-efficacy would suggest that experience shapes perceptions. We sought to evaluate how operative independence and case volume contribute to self-efficacy; operative independence speaks to the quality of the experience, while case volume speaks more to quantity. We hypothesized that self-efficacy would be positively correlated with both operative independence and case volume.

To make comparisons between operative self-efficacy, operative independence, and case volume, data were compared from 3 different and independent data sources as follows: (1) self-reported operative self-efficacy data obtained through a survey taken by graduating PGY-5 residents; (2) self-reported operative independence or autonomy data obtained through the SIMPL OR phone application (Procedural Learning and Safety Collaborative); and (3) case numbers obtained through the publicly available 2019-2020 Accreditation Council for Graduate Medical Education (ACGME) case log data. Throughout this study, operation refers to a specific surgery (eg, laparoscopic appendectomy or trauma thoracotomy), while case refers to a particular instance of performing an operation. To provide consistency between the distinctly unique datasets, data were obtained from general surgery residents graduating in 2020 for all 3 sources. We will now describe the 3 datasets in more detail.

Self-efficacy survey data from the graduating general surgery PGY-5 cohort was obtained from an institutional review board (IRB)-approved survey included as a posttest section on the 2020 ABSITE. The ABSITE is offered annually, and survey questions are often included at the end of the examination; our institution spearheaded the self-efficacy survey in 2020. The authors selected a total of 10 common general surgery operations for the focus of the study, which included 4 laparoscopic cases (laparoscopic appendectomy, cholecystectomy, right hemicolectomy, and diagnostic laparoscopy) and 6 open cases (wide local excision, breast biopsy, thyroidectomy, inguinal hernia repair, trauma exploratory laparotomy, and trauma thoracotomy). Individuals were asked to rate their ability to perform these procedures without attending assistance at this time using a 5-point scale (1=not able to; 2=likely able to; 3=more likely able to; 4=most likely able to; 5=definitely able to). Consistent with previous literature, self-efficacy was considered to be a binary construct where a score of 5 was considered self-efficacious and a score of 1 to 4 was considered not self-efficacious. Complete methods for case selection and survey design are described in Anderson et al.² 

Data were also obtained through the SIMPL OR application on the same operations queried in the 2020 ABSITE survey through a research-based deidentified data use agreement. On the SIMPL OR application, residents report their perceptions of operative independence for individual cases (eg, a specific laparoscopic appendectomy). Thus, each resident could theoretically submit multiple individual cases for a given operation (eg, 3 trauma thoracotomies in 5 years). SIMPL OR is not used universally across all general surgery residency programs; however, the programs that do use SIMPL OR are diverse in program type (academic vs academic-affiliated vs independent). When using the SIMPL OR application, both trainees and attending surgeons answered 3 key questions related to the case regarding complexity, performance, and supervision. For supervision, individuals selected a score from 1 to 4 based on the Zwisch scale: 1=show and tell; 2=active help; 3=passive help; and 4=supervision only.⁴  Case supervision data obtained from the 5 years of residency for 2020 graduates were reviewed for the aforementioned 10 operations because many of the operations included in the survey are more commonly performed at the junior resident stage; thus, operative independence could theoretically be achieved at any year of training. Operative independence was also considered to be a binary construct where a Zwisch score of 4 (supervision only) was considered independent and a Zwisch score of 1 to 3 was considered nonindependent.

The publicly available ACGME 2019-2020 National Data Report was used to compare graduating resident mean case numbers (surgeon total including surgeon chief and surgeon junior) for the same 10 operations.⁵  Case numbers reflect the average number of times (cases) a given operation was performed by a resident over 5 years of training. Cases logged through the ACGME Case Log System were not granular enough to be able to identify 3 of the 10 operations: diagnostic laparoscopy, wide local excision, and laparoscopic right hemicolectomy.

The 10 operations were grouped into 3 tiers (high, middle, and low) based on resident-reported self-efficacy. Descriptive statistics were used to evaluate the frequency of reported operative independence per operation and mean frequency of operative independence per self-efficacy tier. Descriptive statistics were also used to calculate mean case volume per self-efficacy tier. Analysis of variance (ANOVA) was used to assess for differences in the percentage of cases completed with operative independence and the number of cases completed per the ACGME National Data Report based on self-efficacy tier. The Bonferroni calculation was used for post-hoc analysis. Correlations were used to evaluate the relationships between self-efficacy, operative independence, and case volume. Independent samples proportions tests were used to compare trainee and attending perceptions of operative independence. All statistical analyses were conducted using SPSS statistical software (IBM Corp).

The study was reviewed and approved by the Stanford University IRB.

The 10 operations previously described by Anderson et al were organized into 3 self-efficacy tiers (3 to 4 operations per tier) based on the data obtained through the post-ABSITE 2020 resident survey.²  High self-efficacy operations included laparoscopic cholecystectomy, laparoscopic appendectomy, diagnostic laparoscopy, and wide local excision. The middle self-efficacy operations included breast biopsy, inguinal hernia, and trauma exploratory laparotomy. The low self-efficacy operations included thyroidectomy, laparoscopic right hemicolectomy, and trauma thoracotomy. See Table 1 for a breakdown of self-efficacy per operation with sample sizes in addition to tiered means.

The percentage of cases which residents reported as completing with operative independence was calculated based on SIMPL OR data for the same 10 operations.⁴  The sample sizes vary by operation; see Table 1 for a full breakdown of operative independence per operation including sample sizes.

The mean number of cases reported per graduating resident (N=1257) on the 2020 ACGME National Data Report was reviewed for 7 of 10 operations. Due to lack of granularity in case reporting, diagnostic laparoscopy, wide local excision, and laparoscopic right hemicolectomy were excluded from the mean calculations. See Table 1 for a full breakdown of case volume.

ANOVA revealed significant differences in mean self-efficacy between all tiers: high vs mid (87.7% vs 68.3%, P=.008 [95% CI 6.2, 32.7]), high vs low (87.7% vs 25.4%, P<.001 [49.1, 75.6]), and mid vs low (68.3% vs 25.4%, P<.001 [28.7, 57.1]). The percentage of cases completed with operative independence followed similar trends with 32.7% vs 13.8%, P=.006 [6.4, 31.4] for high vs mid; 32.7% vs 4.9%, P<.001 [15.3, 40.3] for high vs low; and 13.8% vs 4.9%, P=.23 [-4.5, 22.3] for mid vs low. While the total volume of cases on average decreased from high (91.8) to mid (20.8) to low (11.1) self-efficacy tiers, this did not reach statistical significance on post-hoc analysis (P=.10 [-16.0, 157.8] for high vs mid, P=.09 [-14.4, 175.9] for high vs low, P>.99 [-77.1, 96.7] for mid vs low).

Correlations were calculated to evaluate the relationships between self-efficacy, supervision, and case volume. Self-efficacy and supervision showed a strong correlation (r=0.84, P=.002 [0.5, 1.0]). Self-efficacy and case volume showed no significant correlation (r=0.61, P=.14 [-0.3, 0.9]). Supervision and case volume showed a strong significant correlation (r=0.79, P=.04 [0.1, 1.0]). See Table 2 for a summary of these correlations.

Attending surgeon perceptions of operative independence demonstrated similar trends by self-efficacy tier as compared to resident perceptions: while attending surgeons generally perceived higher degrees of operative independence, this was statistically significant for only 3 of 10 procedures (laparoscopic cholecystectomy, laparoscopic appendectomy, and thyroidectomy); see Table 3. The Pearson correlation between resident and attending perceptions of operative independence was strong (r=0.95, P<.001 [0.8, 1.0]). The Pearson correlation between attending-reported operative independence and resident-reported self-efficacy was also strong (r=0.82, P=.003 [0.4, 1.0]).

These study findings support the idea that prior case-specific experience impacts perceptions of self-efficacy. The strong correlation between level of supervision and self-efficacy is logical given the role of social persuasion (coaching, feedback, entrustment, etc) as an important contributor in self-efficacy theory. While there are certainly many different factors that contribute to a resident’s perception of their self-efficacy, these findings suggest that level of supervision is one of many potential targets for improving the operative self-efficacy deficit. When residents have more opportunities for operative independence, they can further develop their own self-efficacy, which is essential for personal growth. This creates a positive self-efficacy cycle through social persuasion with further opportunities for operative independence as depicted in the Figure.

Though the study findings support the self-efficacy/operative independence relationship, it is worth noting the stark differences in the percentages of residents reporting procedural self-efficacy compared to the percentage of cases being completed with operative independence. A Zwisch score of 4 likely underestimates true self-efficacy, as there could be several reasons why a self-efficacious resident does not perform a case with operative independence (faculty preference, patient preference, time constraints, billing requirements, institution requirements, etc). Additionally, cases were included from all years of training, which should theoretically represent a range and continuum of independence levels. Despite this, these differences do suggest that some self-efficacious residents are not being given opportunities to perform cases with operative independence, thereby inhibiting further personal growth. It also suggests that there are likely factors outside of operative independence that contribute to development of self-efficacy.

Case volume is one such factor that may affect self-efficacy through mastery experiences, despite not showing a significant correlation with self-efficacy in this study. It did, however, show a strong correlation with the level of supervision, suggesting that the number of cases completed likely contributes to the amount of operative independence granted. It is difficult to ascertain whether this is due to competence, performance, level of training, resident-faculty relationship, or other trainee- or institution-related factors.⁶  This correlation illustrates why competency-based medical education (CBME) is critical; we cannot rely on case volume alone. CBME encourages learner-centered graduated levels of trainee responsibility based on outcomes and abilities, which is not synonymous with time in training or volume of prior experience.^7-9  While case volume certainly contributes to a body of surgical experience, that alone does not equate to self-efficacy.

The comparison of trainee and attending perceptions of operative independence granted is consistent with existing literature suggesting an “autonomy gap” in which residents tend to underrate themselves and perceive needing more supervision compared to attending physician reports.¹⁰  While there is still work to be done to better align trainee/attending perceptions, the findings from the current study suggest that attending-reported supervision level and resident-reported self-efficacy remain strongly correlated. Thus, the lack of operative independence identified in this study cannot simply be explained as a trainee perception issue.

There are several limitations associated with this study. The tiered classification for self-efficacy was used to group cases for statistical analysis purposes instead of considering each operation independently. However, ANOVA did show significant differences between each of the tiers, supporting the notion that these do represent distinct groups. Additionally, the ACGME case log granularity prevents perfect alignment with the operations chosen for the initial self-efficacy post-ABSITE survey. Thus, the case volume sample size was limited for each of the self-efficacy tiers because several operations had to be excluded from the analysis. There are also varying degrees of difficulty within elements of a single operation; some residents may be facile at a portion of the operation but unable to do the entire case with operative independence, which is not captured by this survey. Similarly, the post-ABSITE self-efficacy survey did not specify the level of case difficulty when asking about operative self-efficacy, which could have led to differences in how the question was perceived by respondents. The use of SIMPL OR also presents a limitation to the generalizability of the study, as not all general surgery programs use the platform.

Future areas of study could seek to design interventions that target opportunities for operative independence and case volume to help facilitate the positive self-efficacy cycle previously described. For cases in the lower self-efficacy tiers, which tend to confer less operative independence and volume, targeted efforts in simulation or video-based education could be considered. Other elements of social persuasion, such as operative feedback and coaching, could also be targeted.

This study shows a strong correlation between operative independence and self-efficacy. Operative independence confers a high degree of resident entrustment through social persuasion, a key tenant of self-efficacy theory. However, case volume or quantity, which address the role of mastery experience in contributing to perceptions of self-efficacy, did not have a significant effect.

The authors would like to thank the Procedural Learning and Safety Collaboration (Boston, Massachusetts, USA) for providing SIMPL OR data.

This was presented as an oral presentation at the Accreditation Council for Graduate Medical Education Annual Educational Conference, February 23-25, 2023, Nashville, Tennessee, USA.

Funding: The authors report no external funding source for this study.

Conflict of interest: The authors declare they have no competing interests.