Exploring the Value of an Assessment for the Professional Coaching of Residents

In recent years, professional coaching has gained increasing prominence as an individualized intervention to foster personal and professional development of health professionals.^1,2  Professional coaching is defined as “a collaborative solution-focused, result-orientated and systematic process in which the coach facilitates the enhancement of life experience and goal attainment in the personal and/or professional life of normal, nonclinical clients.”^3,4  Professional coaching is distinctly different from mentoring as there is no formal hierarchy between the learner and coach.⁵  Furthermore, it differs from clinical coaching, which is exclusively focused on the development of clinical skills.⁶ 

Coaching has been proposed as a “proactive solution to the growing epidemic of physician distress and burnout”⁶  as it can strengthen personal resources, thereby reducing vulnerability to burnout.^4,6-8  Recognizing that residency training presents a challenging transformational period in physicians’ development, integrating coaching into the training curriculum has been put forward as a promising strategy to support residents’ learning journey towards becoming competent and resilient physicians.⁹  Therefore, there is a compelling need for evidence to inform the effective integration of coaching and tools that can support the coaching process in residency training programs.

Numerous heterogeneous studies from various professional settings, including the medical field, have demonstrated the positive effects of coaching.^1,2,4,5,7,10  These studies demonstrated that coaching can enhance performance and skills; improve work-life balance, job satisfaction, and engagement; and foster psychological capital and autonomy, while reducing emotional exhaustion and distress, burnout symptoms, and job insecurity.^1,2,4,5,7,10  Moreover, coaching has been associated with improvements in overall quality of life and resilience.^1,2,5,7,10  However, the understanding regarding the underlying mechanisms and the role of tools that can support coaching remains limited.^1,5,11  Gaining insight into what works in coaching and how tools from the workplace could be applied to the coaching process to foster professional and personal development in residents is crucial for shaping its implementation in the training curriculum.

In the East-Netherlands Education and Training District for postgraduate medical training, residency applicants participate in a selection assessment, which measures cognitive capacities, personality, motivational drivers, and competencies (provided as supplementary data). Our previous research investigating the learning value of this selection assessment for pediatric residency applicants demonstrated that applicants considered the assessment a valuable learning tool.¹²  It fostered self-reflection, self-awareness, self-acceptation, and the identification of development goals.¹²  Additionally, it provided insights into professional fit and motivational drivers in work. However, we found that the learning effect remained underutilized, as the assessment is currently strongly linked to the selection procedure and lacks integration into the training curriculum.¹²  Drawing on these insights, we speculated that integrating the assessment as a tool to support coaching of residents could benefit the coaching process as well as learning from the assessment, better aligning it with the goals of the learning curriculum.

In this study, we set out to explore the perceived value and limitations of using the selection assessment for professional coaching in medical postgraduate training, as experienced by residents and coaches. Understanding the perceived value and limitations of the assessment for coaching is valuable for formulating evidence-informed recommendations to shape coaching practices in the postgraduate training curriculum.

The research was conducted in the East-Netherlands Education and Training District for postgraduate medical training. Postgraduate medical training in the Netherlands comprises 7 Education and Training Districts and 30 specialties, with approximately 7000 residents in training at any given time. Training programs are accessed through district-based open-market selection procedures, which vary between districts. Medical graduates often work as junior doctors before entering the 4 to 6 years of residency training, which includes rotations in various clinical settings, including university medical centers and affiliated general hospitals. Despite the growing interest in professional coaching, it is not currently formally integrated in the training program.

The electronic assessment (EA), an online aptitude test (PiCompany), is used in the selection process for residency training in the East-Netherlands Education and Training District. It comprises 4 elements: cognitive capacity (“fluid” intelligence compared to a Dutch control population at the university master level), personality (evaluated using the 5-factor model¹³ ), drivers (motivational orientation), and a composed score reflecting the developability of a set of relevant competencies for the respective medical specialist practice (provided as supplementary data). As part of the selection procedure, applicants receive a detailed report outlining their results for each element and engage in a meeting with a Human Resources advisor affiliated with the training program who explains the results and encourages applicants to reflect on them. Further details on the use of the EA in the selection procedure are provided in our previous work.¹²  The Figure provides an illustrative example of the result overview page. After the selection process, residents are free to use the EA for their personal and professional development. Although the EA can serve both personnel selection and professional development purposes, it was initially introduced and presented primarily as a selection instrument and is presently not explicitly integrated into the training curriculum.

For this study, we assembled a pool of 4 accredited coaches from diverse backgrounds, each experienced in coaching health professionals. The coaches were not involved in the research team or selection procedure and had no affiliation with the training program or any other formal authority over the residents. Coaches were equipped with background information about the EA and engaged in one online meeting with other study coaches to exchange ideas on the use of the EA in coaching.

We recruited residents to participate in a single coaching session using the EA to facilitate the coaching process. We purposefully recruited residents from 3 different training programs for supportive, surgical and nonsurgical specialties (anesthesiology, gynecology, pediatrics) and at varying stages of their training, recognizing that diverse context and training experiences and duration may influence perceptions. Residents were invited by the secretarial offices of the 3 training programs via email. A sample from different training years was invited, with replacements invited from the same training year in case of decline or nonresponse.

After an introductory meeting, residents and coaches engaged in a single coaching session, lasting approximately 1.5 hours. Since coaches have their personal style and preferences, and learners can present a wide range of coaching and learning needs and preferences,⁷  we intentionally refrained from providing explicit instructions on how to apply the EA in the coaching process to allow for personalized application. Residents were allowed to choose their preferred coach. Residents shared the EA report with the coach prior to the coaching session. Residents were permitted to continue their coaching trajectory outside the study procedures, if desired. The coaching intervention was proactively offered only to potential study participants; however, participation in the intervention was not contingent upon their agreement to participate in the interviews, and it was also accessible to residents who sought it outside of study procedures.

Grounded in a constructivist paradigm, we designed a qualitative study to explore the perceived value of the EA for coaching. We chose a qualitative approach because it best allowed us to gain an in-depth understanding of the participants’ perspectives.¹⁴  We employed individual semistructured interviews to delve deeply into residents’ experiences, and a focus group interview to explore and compare coaches’ experiences.

The residents participated in 2 individual interviews, which facilitated an in-depth exploration of each resident’s unique experiences and perspectives. The first interview occurred approximately 1 week after the coaching session to capture immediate reflections and insights from the coaching experience with the EA. The second interview took place after 3 months, allowing for a deeper understanding of how the residents incorporated the insights from the coaching with the EA into their work practice. All individual interviews were conducted by the main researcher (L.T.) using a semistructured interview guide (provided as supplementary data). Interviews were conducted either face-to-face or through a secured video-calling platform, based on the resident’s preferences. Interviews took place from August 2022 to April 2023 and were recorded and transcribed verbatim.

Data collection and analysis were conducted iteratively. Initially, 14 residents agreed to participate. However, 7 residents did not proceed with scheduling the coaching session and/or the study interview, and their study participation was discontinued due to a lack of response. One resident did not participate in the second interview due to a work sabbatical. After interviewing the 7 included residents, the research team established that the data contained sufficient breadth and depth to answer the research questions. In reaching this conclusion, several aspects were considered, including the focused study aim, the depth reached in the interviews, and the variation of perspectives and coaching perspective already represented in the sample.^15-17 

Three out of 4 coaches from the study pool coached at least 1 resident during the study period and participated in the focus group interview. The focus group interview was conducted in February 2023, after the majority of interviews with residents was conducted, allowing it to be informed by the insights obtained in those interviews. We opted for a focus group interview, because we expected that an interactive discussion between the coaches would enrich and deepen our findings.¹⁸  We used a semistructured interview guide (provided as supplementary data) to explore the coaches’ perspectives on the value of the EA in their coaching experiences. Additionally, it aimed to explore differences and similarities between the coaches’ experiences as well as between different residents they coached. During the focus group interview, L.T. served as the main moderator, while B.V. acted as an observer, focusing on participants’ responses and asking follow-up questions to enhance insights. The focus group interview was recorded and transcribed verbatim.

Interview transcripts were imported into ATLAS.ti (ATLAS.ti GmbH) to facilitate data analysis. Our analysis followed the principles of thematic analysis proposed by Braun and Clarke,¹⁹  an iterative method, involving extensive reading and re-reading of the interview transcripts, coding relevant text fragments, and identifying themes through collaborative discussions.

After immersing themselves in the data through repeated reading, the main researchers (L.T. and B.V.) independently conducted inductive coding. After each interview, they engaged in a joint review and discussion of the coded transcripts, working collaboratively to develop a codebook. Disagreements were resolved through discussion, and a third researcher (E.C.) was consulted when necessary. This collaborative approach led to a richer understanding and interpretation of the data fragments.

Subsequently, L.T. and B.V. organized the codes into potential themes, gathering relevant data to underpin and substantiate these themes. In an interactive meeting, a subgroup of the research team (L.T., E.C., B.V., F.U., B.T., J.V.) reviewed and evaluated the emerging themes and underlying supporting data. Afterward, each of the other team members critically and independently reviewed the potential themes and the findings from this meeting. Through an iterative process, L.T. and B.V. restructured and refined the themes in collaboration with the entire team, carefully reviewing and analyzing themes alongside supporting data, resulting in the comprehensive findings presented in this manuscript. ChatGPT, version 3.5, an advanced language model developed by OpenAI, was employed for language editing of the manuscript to enhance clarity and cohesiveness. The generated outputs were carefully reviewed and revised by the authors to ensure the integrity of the scientific content.

The research team encompassed diverse perspectives, summarized in Table 1. The diverse backgrounds and expertise actively contributed to fostering reflexivity throughout the research process, as some team members could draw on their insider experience as resident, coach, or program director to contextualize and give meaning to the experiences reflected in the data, while other team members could act as “outsiders” to critically question assumptions that may otherwise have been taken for granted. Before commencing the research, the researchers acknowledged and discussed their own assumptions. During the various team meetings and online discussions, the research team intentionally engaged in critical discussions regarding each other’s notions and underlying assumptions, always anchoring the discussions with supporting study data. Reflexivity and confirmability were further stimulated by reflective journalling by the main researchers (L.T., B.V.) who documented both their individual reflections during data collection and analysis, as well as the discussions with the team.

This research was conducted in adherence to the Declaration of Helsinki. The Netherlands Association for Medical Education (NVMO) Ethical Review Board approved this study (file number 2021.8.5). We provided all participants with information about their rights, the study’s objectives and how their data is protected. Written informed consent was obtained from all participants.

Resident participant characteristics are provided in Table 2. A central finding was that residents and coaches considered the EA a supportive tool but not a core focus in the coaching process. We found that the perceived value of the tool for the coaching process was affected by residents’ and coaches’ attitudes towards the EA.

Residents and coaches commented that the residents’ coaching goals always remained central in the coaching process, and while the EA served as a supportive tool, it was never the core focus (Quotation [Q] 1 and 2, see Table 3). In the interviews, residents often initially reported that the EA did not play an explicit role during the coaching session and they believed they could have eventually achieved similar results without the EA (Q3). However, further probing uncovered instances where (parts of) the EA were indeed used, such as when coaches referred to motivational drivers from the EA or prompted residents to reflect on their inherent qualities.

We found that the EA facilitated residents’ reflections in the coaching process, especially during the preparation phase. All residents revisited the EA as part of their preparation for the coaching session. Residents indicated that this process heightened their self-awareness of their personal attributes and motivational drivers and offered insights into their personal growth (Q4-5). Some considered it supportive in recognizing or clarifying challenges they encountered in their professional or personal life. As a result, they found it helpful to use the EA to formulate or clarify their coaching goals (Q6). Others expressed more difficulty in identifying information within the EA that could be applied to their coaching goals (Q7).

During the coaching session, residents experienced that the EA helped them articulate and concretize their reflections. For some residents, the combination of insights from the EA with reflections during the coaching session contributed to increased self-acceptance or a clearer understanding of why certain things were challenging or important to them (Q8).

Both residents and coaches expressed that the EA helped accelerate the coaching process and allowed them to delve into deeper aspects more rapidly (Q9). They presented a range of reasons for this, including the perception that the EA provided a solid foundation to build upon, as well as both residents and coaches noting that the residents appeared more prepared for the coaching session (Q10). A coach noted that residents’ openness in sharing the EA, which they considered to be something intimate, seemed to lay a foundation of mutual trust that contributed to the acceleration of the coaching process (Q11). Furthermore, residents reported feeling a sense of familiarity and experienced that the coach already had a better understanding of who they are, resulting in less need for extensive exploration and, therefore, enabling them to reach the core of the matter more quickly with the coach (Q12). Consistent with this, coaches reported referring to the EA during the coaching conversation, employing various components of the EA for this purpose (Q13).

As a caveat, coaches expressed concerns about potential negative effects of preconceived notions brought forth by the EA; however, in practice, they did not find this to be the case, a sentiment echoed by residents. Coaches also contemplated whether the coaching process might have been accelerated due to the instruction to conduct a single coaching session, deviating from their usual practice of multiple sessions.

We found that residents’ and coaches’ attitudes affected how the EA was used in the coaching process and its perceived value. While attitudes toward the EA varied among residents, ranging from questioning its credibility and relevance, maintaining a more neutral standpoint to attributing great value to the instrument, none exhibited reluctance to share the EA or use it during the coaching session (Q14-15). Individuals who attributed greater value to the EA also appeared to emphasize it more while formulating their coaching goals, while those who held reservations did not initially emphasize the tool as much. During the interviews, some residents reflected that they had previously associated the EA primarily with the selection process or did not consider it relevant for their professional development. However, their perspective shifted after the coaching with the EA, leading them to recognize and appreciate the EA as a learning instrument (Q16).

Coaches expressed distinct preferences for using specific EA elements in the coaching process, guided by their perceptions and past experiences. For instance, one coach favored focusing on motivational drivers, considering them central in the coaching process, while another leaned toward competencies due to their familiarity with the psychometric properties.

This study revealed that the EA played a supportive rather than a dominant role in the coaching process. Residents and coaches reflected that the EA supported the coaching process by facilitating residents’ reflection and accelerating the process. We found that its use and perceived value were affected by residents’ and coaches’ preconceived notions about the EA. Specifically, some residents initially questioned the EA’s credibility or viewed it as tied to selection and only recognized it as a learning tool through the coaching, underscoring the importance of positioning the tool as a learning instrument.

Previous research has predominantly focused on coaching outcomes, leaving a gap in understanding how and why specific coaching tools can contribute.^1,5,11  Coaches have a wide array of tools at their disposal to support residents, but little guidance exists on their effective usage.²⁰  Existing studies on comparable assessment tools have primarily examined their application in selection processes, neglecting their potential for resident professional development.¹²  This study gives insight into how personality motivation and competency assessments can be applied in the professional coaching of residents. Placing our findings within the classification of coaching tools and their purposes from Richter et al,²⁰  we observed that this type of assessment tool can help residents and coaches to identify personal strengths and pitfalls, support self-reflection and goal setting, and can contribute to establishing rapport. Our study revealed that residents and coaches purposefully customized the tool’s use to address specific coaching goals and unique needs and preferences of each resident.

It has been argued that for coaching to be effective, multiple sessions may be necessary.⁵  However, review studies did not identify a significant correlation between the number of coaching sessions and coaching outcomes.^1,5  In our study, both residents and coaches noted that the EA accelerated the coaching process, which enabled them to delve into deeper aspects more rapidly. Concerns regarding the risk of making overly great leaps or harboring preconceived notions were raised but were not substantiated by participant experiences. These findings suggest that tools providing comprehensive insights into learners’ characteristics and motivational drivers may hold potential to reduce the required duration of coaching interventions. This seems of particular interest in the context of postgraduate medical training, where budget and time constraints may restrict coaching opportunities.

A limitation of this study is the potential influence of selection bias, as it is probable that only residents and coaches who were open to using this tool chose to participate. Due to ethical constraints, we were unable to explore why 7 residents were lost to follow-up. Despite this limitation, we did observe a variety of attitudes toward the EA. While our decision to allow participants to customize coaching sessions aligned with the principles of individualized training, it may pose challenges to the transferability of our findings due to the heterogeneity of the coaching sessions. This study was conducted in a single context, and we acknowledge that our findings may be culturally dependent. It should be acknowledged that the EA was already implemented as part of the selection procedure. Introducing a novel assessment tool into the training curriculum would likely incur additional costs.

Future research could quantitively assess the effectiveness of using the EA for coaching to form a perspective regarding the cost-benefit implications of applying such tools in educational practices. Further research could explore how coaches and residents can effectively employ of the various other learning tools offered by the workplace and in the training curriculum, and comparing and contrasting different methods of applying the EA in coaching could yield valuable insights. Understanding when and why certain tools are valuable in specific coaching scenarios will contribute to a more comprehensive understanding to form evidence-informed recommendations to shape coaching practices in postgraduate medical training.

The cognitive capacities, personality, motivation, and competency selection assessment played a supportive but nonprominent role in the professional coaching of residents. Its supportive value included fostering self-reflection and accelerating the coaching process within a single coaching session. Residents’ and coaches’ attitudes affected the use and perceived value of the tool, highlighting the importance of how coaching tools are presented.

The authors would like to thank the study participants, Christiaan Keijzer for his contributions to the conception of this study and participant recruitment, and Thomas Hengeveld for translating the interview quotes.

The online version of this article contains assessment information and the interview guides.

Funding: This research was funded by the Radboudumc Innovation Fund for Postgraduate Medical Education.

Conflict of interest: The authors declare they have no competing interests.