Ready, Set, Goal: A Mixed Methods Study of a Goal-Setting Intervention on 2 Competency-Based Geriatric Medicine Rotations

Globally, postgraduate medical education programs have transitioned to competency-based medical education (CBME).^1,2  Through the assessment of entrustable professional activities (EPAs), CBME programs produce data that coaches can use to cocreate learning goals with residents. Preliminary research in Canada suggests that EPAs may not sufficiently motivate residents to engage with core content on a given clinical rotation.³  Consequently, clinician teachers may need to supplement EPAs with other strategies, such as goal-setting interventions, to support how residents self-regulate their clinical learning (ie, experiment with and learn how to use learning strategies effectively). Goal-setting has been acknowledged as a key learning strategy that associates strongly with successful learning outcomes, and tailored goal-setting may support efforts to optimize residents’ learning in CBME programs.^4-6 

Goals represent the desired outcomes and related standards against which learners measure their progress and achievement.^7,8  In medical education, programs regularly implement goal-setting interventions to improve trainee engagement, performance, and collaboration with faculty.^6,9  Generally, such goal-setting interventions have been integrated into trainees’ longitudinal, individualized learning plans or specific clinical rotation objectives.^10-13  The reported limitations of longitudinal goal-setting interventions include that goals might not be revisited and can be forgotten over time.^10,12  In contrast, rotation-specific goal interventions often address such issues by focusing on relevant day-to-day objectives but are challenged by shorter timelines to choose, enact, adapt, and strategize goal achievement.^6,14 

Few studies examining goal-setting interventions in clinical environments have looked at how they can be best leveraged in CBME programs with EPAs.^15-17  Generally, these studies provide limited insight regarding how goal-setting influences key outcomes, such as trainees’ learning processes, or their perceptions of faculty members’ feedback quality. Therefore, studies could focus more on producing an in-depth understanding of how residents perceive the influences of goal-setting on their overall educational experiences in competency-based clinical learning environments.

After transitioning to a CBME curriculum, we prospectively implemented an iteratively refined rotation-specific goal-setting intervention¹⁸  and aimed to explore how residents perceived the intended effects on their learning climate and learning experiences.

We conducted a concurrent mixed methods study involving the simultaneous collection and analysis of qualitative and quantitative data with a convenience sample of postgraduate residents. We chose 2 geriatric medicine rotations using a CBME/EPA curriculum as the specific learning environment and participants’ self-regulated attempts at implementing goal-setting into their learning practice as the specific learning experiences under study.

From November 2019 to June 2021, all residents (eg, internal medicine, family medicine, geriatric medicine subspecialty residents and others) assigned to and electively placed on geriatric medicine rotations at 2 academic hospital sites were expected to complete a goal-setting intervention during the rotations. The geriatric medicine rotations are inpatient-based and 4 weeks in duration. The clinical team at each site involves the geriatrician attending, trainees, and a clinical nurse specialist or nurse practitioner. Residents typically have access to 2 or 3 geriatrician attendings during the rotation. Each rotation has a geriatrician site director. There usually are 2 or 3 trainees per rotation of mixed levels, from medical students to postgraduate year (PGY) 1 through 6 medical residents from core training programs such as internal medicine and subspecialty training programs such as geriatric medicine. Most trainees are internal medicine residents on their mandatory geriatric medicine rotations. The research coordinator (E.K.) sent a recruitment email during the first week of each potential participant’s geriatric medicine rotation to invite them to participate in this study.

The goal-setting intervention included: (1) completing a prerotation structured goal-setting form; (2) reviewing and clarifying prerotation goals with a clinician teacher (by email, telephone, or in-person meeting); and (3) joining a midrotation check-in, over the telephone, email, or in-person, with the site director. The latter intended to allow trainees to review goal progress, receive feedback, and collaboratively (re)strategize goal achievement.

The goal-setting form prompted residents to generate 3 learning goals for their rotation. We developed the form using recommendations for implementing goal-setting interventions in postgraduate medical education.¹⁶  We presented the form as a fillable PDF, including examples of appropriate goals that covered geriatric medicine topics and instructions on setting important SMART (Specific, Measurable, Achievable, Relevant, and Time-Bound) goals (online supplementary data Appendix 1). The form accompanied rotation orientation materials, including standard rotation objectives. Due to differing administrative practices at each hospital site, residents received and completed the goal-setting form via an introductory email on either the first day of their rotation or 1 week before. At both sites, near the beginning of the rotation, residents’ goal forms were reviewed by the site lead or the teaching faculty lead. Goals were then summarized and emailed to all faculty working with the trainee, along with a suggested plan for maximizing goal achievement. A lead faculty (J.A., D.G.) gave all residents feedback on their goals and discussed potential strategies for achieving them in person or by email.

While all residents rotating at each site completed the goal-setting intervention, only those who consented to participate in this study voluntarily were requested to complete the Dutch Residency Educational Climate Test (D-RECT) questionnaire (online supplementary data Appendix 2) and qualitative interviews. All residents who volunteered to participate provided written consent, as approved by both hospitals’ research ethics boards. We specifically sampled residents and excluded medical students, who would not have been trained in CBME curricula. Regarding our sampling strategy, while all residents were chosen purposively because they were completing a geriatric medicine rotation at the 2 study sites, our sampling strategy amounted to a convenience sampling approach in that we included all residents who volunteered to participate.

We administered the D-RECT to assess how residents perceived the overall educational functioning of the 2 geriatric medicine rotations. Researchers have collected validity evidence supporting the use of the D-RECT to measure learning climate, including but not limited to support for the scoring system using exploratory factor analyses and generalizability metrics.^19-22  Previous studies suggest that at least 11 residents provide sufficiently robust data for comparative analyses.²⁰ 

Given nearly all clinical rotations at our 2 hospital sites had moved to CBME curricula, our goal-setting intervention represented a key rotation-specific difference in residents’ learning climate. For the prerotation D-RECT, we asked participants to rate the learning climate of their most immediately recent clinical rotation, no matter the specialty. For the postrotation D-RECT, participants rated their geriatric medicine rotation. Hence, we aimed to understand if the D-RECT was sensitive to potential shifts in the overall learning climate that could be associated with the many facets of the goal-setting intervention.

Author E.K. conducted the 30- to 60-minute semistructured interviews over the telephone or using video conferencing software. Participants described their experiences with the goal-setting intervention, shared their thoughts on the learning climate, and discussed any challenges they experienced with the intervention or the rotation. Our team developed the interview guide using sensitizing concepts from the literature on goal-setting in medical education,^6,9-13  as well as concepts relating to goal-setting, self-monitoring, adaptation, and strategic learning from the self-regulated learning (SRL) literature (online supplementary data Appendix 3).^4-6,8,9  We piloted the guide via several team meetings and revised it iteratively throughout the study. We audio-recorded each interview, and a paid service transcribed them verbatim, with E.K. removing any identifying information before team analyses.

We followed standard practice to analyze the D-RECT questionnaire data by aggregating all 35 items into a mean score for each participant. We then analyzed participants’ pre- and postrotation scores using a paired-sample t test to evaluate whether and how scores were compared within each participant’s personal experiences.

We analyzed all interview transcripts using constant comparative analysis principles. Our thematic analysis involved an iterative process of inductively coding data to challenge and refine our initial deductive coding (ie, using our sensitizing concepts).^23,24  First, we engaged in open coding to identify preliminary codes. Second, we developed focused codes by condensing frequently occurring initial codes and then organized codes into relevant thematic categories. All authors were involved in the initial coding of all transcripts. After multiple meetings, we agreed on a final coding structure, and author E.K. applied the final codebook to all transcripts. The entire study team regularly debriefed throughout the analytic process to discuss the rigor of the process and the representativeness of the themes.

We continued with data collection and participant recruitment until our analysis of new transcripts yielded less meaningful data to answer our research questions.²⁵  We used Nvivo qualitative data analysis software (QSR International).

Our team included members with expertise in geriatric medicine (J.A., D.G.), qualitative research methods (E.K., R.B.), SRL theory, and goal-setting research (R.B.). E.K. and R.B. had no relationship with trainees. J.A. and D.G. both attend as geriatricians and supervise trainees in their rotations. Their experiences allowed them to clarify clinical contextual information, such as influences on how the clinical team and training program are structured. Collectively, we adopted a realist perspective, which we interpret as participants’ learning experiences being bound by context and consequently situated. Yet, we believed we could develop a representative understanding of participants’ experiences within our selected situations. One investigator (E.K.) kept electronic memos to track how the analyses evolved, to keep the team updated, and to inform judgements of how participants’ lived experiences may have influenced the data collection and analysis. These memos ensured that the team viewed the data from different perspectives, contributing to analytic rigor.²⁶ 

This study was approved by the ethics boards of St. Michael’s Hospital and Sunnybrook Health Sciences Centre.

Fifty-eight trainees completed the goal-setting intervention between the 2 sites and were eligible to participate. We recruited 12 residents (20.7%) from this population to complete the D-RECT forms and interviews. Trainee level included 2 PGY-1 residents, 3 PGY-2s, 4 PGY-3s, and 3 PGY-4s. All PGY-4s were geriatric medicine residents, whereas PGY-1 to PGY-3 included 1 neurology resident and 8 internal medicine residents.

Eleven of the 12 participants completed the pre- and postrotation D-RECT questionnaires. Analysis of the D-RECT data helped us set the context for interpreting our interview data. According to the paired t test (t10=4.04, P=.002), we found that D-RECT scores for the preceding rotation (M=3.84±0.44) were significantly lower than scores for the geriatric medicine rotation (M=4.29±0.37). These scores reflect participants’ judgements of the entire learning climate and thus provide an encouraging signal that the geriatric medicine rotation, including all components built around the goal-setting intervention, offered a relatively positive educational experience. As a form of triangulation, we explored participants’ interview data to understand what contributed to and detracted from this relatively positive learning climate.

Our analysis of the interview data yielded 3 themes relating to how residents perceived the goal-setting intervention and overall learning climate. We provide representative quotes below to explore how each theme pertains to participants’ experiences.

Participants reported multiple reasons for setting their goals, including to focus their self-regulated learning, to make the rotation more “educational,” to help operationalize the rotation’s formal objectives, to create opportunities for receiving feedback, and to produce a sense of accomplishment at the end of the rotation: “…it’s a way to personalize a rotation…to make sure that your goals are met. You can also look at it from, a monitoring perspective in terms of did I achieve what I wanted: yes or no? Are they helping me in my learning?” (S02)

Each resident’s interests, previous knowledge levels, and clinical specialty also affected goal selection. For those with advanced geriatric medicine knowledge, their goals were most focused; whereas, for others, goals tended to focus on general knowledge. Nearly all reported that the structure and the listed examples were helpful: “…the form forces you to sit there and think about it for a little while… it forced you to put a pen to paper and think of more specifics in relation to your goals.” (S012) By contrast, some participants reported that the structure constrained the goals they could set. For example, some expressed that the “measurable” requirement was not helpful: “It didn’t seem particularly valuable to say I need to do 5 physical exams to be successful…” (S04)

Beyond the form, some participants reported that the processes lacked clarity about the intervention’s purpose: “I don’t know, was the onus on me to be doing it? No one told me what the point of this was, so I didn’t know should I have had my goal sheet with me all the time to show people, like, we need to tick this box off kind of today?” (S011)

Most participants described a self-regulated and independent process in documenting their goals. However, some reported cocreating their goals with someone else, for example, a clinical staff member who they felt knew the rotation’s learning opportunities well (eg, a nurse practitioner). Participants reported placing high importance on ensuring staff, team members (including other trainees), and other practitioners were aware of their chosen goals, as they thought this would ensure successful goal achievement: “If I can think about how those goals are achieved, it really is tied to the people that are there with you. Not the policy or program. There is nothing built into the rotation that reminds me oh I have these goals. It’s really when people remind you and give you an opportunity to speak and to focus on [your goals].” (S09)

Participants noted that if neither they nor the staff worked to be clear on their goals, or if time was not set aside to discuss them, they were less likely to achieve their goals: “Did I actually keep them in mind? Not so much. I would say—yeah, and part of it was, we never really revisited them much. I think it would have been helpful to [do so].” (S011) As a result, early adequate coconstruction of goals between supervisors and residents appeared to dissipate over the 4-week rotation for some residents. In addition to factors associated with staff interactions, participants often mentioned how, during busier times, their learning took a backseat to patient care, which limited the availability of clinical opportunities that could be dedicated to their personal learning goals.

Many participants recognized that identifying clear and relevant measures of goal achievement was challenging given the personal nature of their goals: “But I don’t think that it should be something where somebody else is trying to measure where you achieved your goals, because learning goals are so personal.” (S01) Consequently, many had to rely on their self-perceptions to assess the “…blurry line between am I comfortable with my level of knowledge or not?” (S01) when judging their progress. Others mentioned relying partially on external feedback to judge their success, for example, if they received corrective feedback less frequently over time. For some residents, external feedback also took the form of EPAs in the CBME curriculum: “The other 2 [goals] were easy, because I could effectively track them with the consults I did or the EPAs I got…but, the last one required a bit more reflection. There was no specific EPA, target, etc. The last one was more difficult to measure.” (S03)

While some participants noted the intervention lacked formal, structured opportunities to assess goal achievement, others mentioned the value of reflecting on their goal-setting at the midpoint meeting. Participants also reported needing to be better informed about whether and how to adapt or adjust their goals: “If I had known I could have changed the goal I probably could have chosen another clinical goal. I have one in mind right now that might have had a higher chance of coming up and is still a goal of mine. I didn’t know that we could change our goals midway.” (S01) Related to this, participants commented on the challenge of setting goals before the rotation, and thus before they had developed comfort with the rotation’s procedures, policies, team, content, and patient population. Some noted that this early goal-setting could limit rather than expand learning opportunities: “I find it quite challenging to set some of the goals before you know the rotation, and it might also have the ironic negative impact of making you focus on those things and not open to the flexibility of learning some of the other things.” (S05)

We found that a structured goal-setting intervention implemented during a 4-week CBME-based geriatric medicine rotation prompted a primarily resident-dependent process in which they engaged variably with teaching faculty and other team members. Our mixed methods design offers insights into how residents experienced the overall climate associated with our goal-setting intervention (ie, the D-RECT data) and how it influenced their learning and collaboration with teaching faculty.

Our first theme suggests that the SMART format, like any structured approach, provided guidance while limiting flexibility. Like other studies, such a structured approach may be more difficult for junior trainees,^10,27  whose difficulty might be related to their needing to develop more familiarity with rotational content and what to expect on the rotation. By contrast, we found that geriatric medicine residents tended to report that the format mediated their setting of specific, refined goals. This impact of resident seniority might suggest that independent goal-setting may be more suitable for residents who know what to expect on a rotation.

Our second theme showed that, despite our attempted forecasting and anticipation of residents’ and supervisors’ challenges and time restrictions, the presence and absence of clinical opportunities affected goal-setting, learning plan enactment, and goal achievement.^6,17  This finding aligns with those of others who have reported that prioritizing patient care above goal-setting can limit some staff and residents from planning, reviewing, adapting, and debriefing goal pursuit.^6,27  When enacting their goals, we found that participants needed to establish goal clarity with clinician teachers, and all supervising clinicians on the rotation needed a broad awareness of the participants’ goals. Like other studies of rotational goal-setting interventions, when implemented well, these actions appeared to provide a sense of accountability, direct learning efforts, and facilitate collaboration with teaching faculty.^6,14,17,27 

Our third theme demonstrated how an unstructured assessment process could lead to uncertainty regarding whether and how goals have been successfully achieved. Participants suggested that educators could focus more on establishing clear measures of goal achievement. Indeed, many described having to rely on their self-assessment and self-perceived comfort. Some reported that attending the structured midrotation “check-in” gave them greater clarity regarding their goal progress and achievement. In auditing the intervention, we found that some participants did not attend their assigned midrotation debrief, limiting these check-ins to email exchanges only. Given the realities of busy clinical rotations, future research could focus on how to support residents in cocreating meaningful measures of goal achievement with their peers and supervisors beyond relying solely on the notably limited self- and EPA-based assessments.

Our findings, integrated with contemporary evidence, provide implications for other programs considering the implementation of rotation-specific goal-setting interventions. First, we suggest clinician teachers benefit from considering each resident’s stage and familiarity with the rotation when deciding how to prompt goal-setting. Second, it seems that more adaptable goals with more cocreation with supervisors may prevent residents from feeling engaged in misdirected learning efforts. When practical, allowing time to acclimatize to the rotation before selecting goals may be beneficial.¹⁶  Third, goal-setting interventions will likely function best with clearly outlined and transparent processes and expectations for how trainees set goals, enact plans, and measure goal achievement. Fourth, all teaching faculty would likely benefit from training to develop cocreation skills and to overcome likely challenges to goal-setting, which may ensure they can collaboratively help trainees set and sustain their efforts toward goal achievement.^6,27,28  For example, the WOOP (wish/outcome/obstacle/plan) SRL strategy, incorporated in a study of residents on critical care rotations, may structure how trainees and teachers collaboratively identify potential barriers to goal achievement and strategize around them.²⁹  Fifth, programs could consider the benefits and drawbacks of different goal criteria when choosing a goal-setting framework (eg, SMART vs achievement goals vs “life goals”). Recent literature on goal-setting theory suggests trainees may benefit from prompts to set life goals and achievement goals with which they identify and can relate to their future selves.^30,31 

Transferability of our findings may be limited given we examined only 1 specialty at 2 sites. In choosing an interviewer (E.K.) without program status, we likely encouraged candid interviews; however, her nonclinical background may also have affected her ability to probe for key responses. The ongoing pandemic may have influenced residents’ well-being, the learning climate, clinical encounters, and other learning opportunities. By delivering the D-RECT tool before the participants’ geriatric medicine rotation, we may have influenced how they viewed their subsequent experiences. However, the direction of that influence was likely variable (ie, negative, positive, or neutral). Finally, we suggest carefully interpreting the D-RECT findings, given our sample size just met the requirements of a prior power analysis.²⁰ 

We did not interview faculty, whom future researchers could study to provide a more comprehensive understanding of the perceived impacts of goal-setting interventions. Additional avenues for research include exploring the influences of alternative approaches to goal-setting (ie, beyond or in addition to SMART), as well as different approaches to delivering any goal-setting intervention (eg, timing of goal-setting prompts, frequency of check-ins, and so on).

An independently administered, rotation-specific, goal-setting intervention appeared to engage residents by stimulating and directing learning on a geriatric medicine rotation. Time restrictions, reliance on unpredictable clinical opportunities, and a desire for explicit measures of goal achievement all arose as challenges to residents’ goal enactment and achievement. Practical implications that others might consider when designing supplements to EPA assessments and other assessment measures within CBME programs include ensuring deliberate goal-prompting processes for both trainees and faculty, clearly outlining rotation procedures and expectations, and offering clinician teachers faculty development in how to effectively coregulate residents’ SRL processes, like goal-setting.

The authors would like to thank all the administrative team members, the clinical faculty members on the geriatric medicine units, and the trainees who contributed their time, effort, and perspectives to this research. Without the collective, these individual units of work cannot be realized.

The online supplementary data contains the goal-setting form, D-RECT questionnaire, and semistructured interview guide used in the study.

Funding: Funding for this study was provided by an anonymous donor.

Conflict of interest: The authors declare they have no competing interests.