Background

Primary care forms a critical part of pediatricians' practices, yet the most effective ways to teach primary care during residency are not known.

Objective

We established a new primary care curriculum based on Malcolm Knowles' theory of andragogy, with brief clinical content that is easily accessible and available in different formats.

Methods

We used Kern's model to create a curriculum. In 2013, we implemented weekly e-mails with links to materials on our learning management system, including moderators' curricular content, resident-developed quizzes, and podcasts. After 3 years, we evaluated the curriculum with resident focus groups, retrospective pre-/post-resident surveys, faculty feedback, a review of materials accessed, and resident attendance.

Results

From content analysis of focus groups we learned that residents found the curriculum beneficial, but it was not always possible to do the pre-work. The resident survey, with a response rate of 87% (71 of 82), showed that residents perceived improvement in 37 primary care clinical skills, with differences from 0.64 to 1.46 for scales 1–5 (P < .001 for all). Faculty feedback was positive regarding curriculum organization and structure, but patient care often precluded devoting time to discussing the curriculum. In other ways, our results were disappointing: 51% of residents did not access the curriculum materials, 51% did not open their e-mails, only 37% completed any of the quizzes, and they attended a weekly conference 46% of the time.

Conclusions

Although residents accessed the curriculum less than expected, their self-assessments reflect perceptions of improvement in their clinical skills after implementation.

What was known and gap

Primary care is an important part of practicing as a pediatrician, but the most effective ways for teaching primary care are not known.

What is new

A primary care curriculum based on Malcolm Knowles' theory of andragogy.

Limitations

Curriculum was implemented at 1 institution for 1 year, limiting generalizability; survey was not tested for validity.

Bottom line

Providing residents a structured pediatric primary care curriculum based on the theory of andragogy resulted in limited engagement, but residents perceived an increase in their clinical skills after implementation.

Primary care is the foundation of practice for every pediatrician. In 2017, a survey showed that 63% of pediatric fellows were in primary care.1  A longitudinal learning experience in a primary care setting with an associated curriculum is an Accreditation Council for Graduate Medical Education requirement.2  Two pediatric programs, Yale School of Medicine and The Johns Hopkins University School of Medicine, have published comprehensive primary care curricula; however, evidence indicates that comprehensive curriculum implementation that adequately prepares residents in primary care is lacking.37 

We interviewed 5 prominent pediatric programs as a part of a needs assessment, and they reported their primary care education was loosely structured and resident engagement variable, unreliable, and suboptimal. This is consistent with survey results from 75% of accredited pediatric residencies nearly 20 years earlier.8  We wondered if lack of progress might be due to an insufficient focus on deliberately structuring curriculum to make it relevant, engaging, and doable for residents in busy clinical service environments.

Our goal was to develop a comprehensive pediatric primary care curriculum (PPCC) for pediatric residents (using Kern's 6-step model of curriculum development)9  that maximized resident engagement by providing brief clinical content that is easily accessible and available in different formats.

The 87 pediatric residents in our program during academic year 2015–2016, 90% of whom had continuity clinic in urban hospital-based clinics and 10% in private practices, participated in the curriculum.

The overarching aims of our new primary care curriculum were for pediatric residents to (1) gain general pediatric primary care knowledge, skills, and attitudes to answer general questions for patients, families, friends, and neighbors; (2) apply basic concepts of general pediatric care to whatever pediatric career they pursued; and (3) maintain an interest in the health of children at a population level.

We carefully planned a new curricular structure based on Malcolm Knowles' theory of andragogy,10  which emphasizes that learners are self-directed and the primary driver for learning is internal, anticipating that residents would participate in a thoughtful, engaging, well-planned curriculum.

We based our curriculum on Yale's because of its affordability, clear learning objectives, and succinct case-based topic reviews.11  Our version has 49 weekly educational modules/topics, each designed to have 10 to 15 minutes of reading completed before a 15-minute pre-continuity clinic (CC) educational session. The goal was for every resident to have enough knowledge of the topic to engage in higher-level group discussion with other residents and the faculty about clinical application of the material. To increase learner buy-in and ensure that the curriculum considered the learner's perspective, we created a PPCC resident steering committee with 1 resident leader per year and 5 to 10 resident members. The steering committee met quarterly to review materials and created new formats of content delivery including quizzes and “fast facts.” Curricular content was planned and sequenced to align with patient concerns and common conditions seen at specific times during the year (eg, “School Readiness” in August, “Identification of the Sick Child” in October) and to complement other teaching in the residency. All curricular materials were housed on a learning management system (LMS)12  that allowed online access anytime and anywhere. We communicated with the residents using the free Mailchimp13  e-mail platform, created humorous topic-related Bitmojis14  (figure), gave links to the annual schedule and the week's learning materials, and highlighted that reading was to be completed before the educational session.

figure

Sample Weekly E-Mail With Learning Objectives and Links to Content

figure

Sample Weekly E-Mail With Learning Objectives and Links to Content

Close modal

We evaluated the curriculum with resident focus groups, retrospective pre-post surveys about curricular content, informal feedback from faculty, Mailchimp and LMS usage data, and resident attendance at CC teaching sessions. Focus group questions were based on Kirkpatrick's model of evaluation (levels 1–3)15  and asked residents to describe what they liked and disliked about the curriculum (reaction), what they learned, and how their behavior had changed. Focus groups were audio-recorded and transcribed, then transcripts were reviewed by the curriculum director who performed a content analysis and annotated the text with labels for recurring points. Another author (J.L.H.) reviewed the content analysis and confirmed identified themes.

Retrospective pre-post surveys (developed by the authors without further testing) were completed by the residents in March 2016 when 32 of 49 topics had been delivered. Thirty-seven seminal statements with Likert-type scale responses (1, strongly disagree to 5, strongly agree) that described desired resident behaviors at Miller's Pyramid16  level of “does” were written, 1 to 2 for each of the 32 surveyed curricular topics (provided as online supplemental material). Residents were asked to self-assess their ability to perform these skills before and after the curriculum, which provided a retrospective self-assessment of their learning from the curriculum. The survey asked the residents to answer only the questions for which they had received teaching of any kind. To maximize response rate, we hand-delivered paper copies of the survey to each resident. Residents delivered anonymous completed surveys to a folder in the chief residents' office. Survey data were analyzed using a paired t test. We applied a Bonferroni correction to control for multiple comparisons, which provided a P value of .001 to meet criteria for statistical significance.

The residency director and PPCC director interviewed 20 faculty CC preceptors in person or by telephone about their perceptions of the new curriculum. Usage data were downloaded from the electronic platforms, and attendance records taken by the PPCC director were summarized.

This educational intervention was deemed exempt by the University of Colorado Institutional Review Board.

Content analysis of focus groups revealed that residents liked the LMS, multiple formats of content delivery were appreciated, active participation during pre-CC educational sessions was beneficial, topics aligned with patient presentations they saw frequently in clinic, and it was not always possible to do the readings.

The retrospective pre-post self-assessment survey was completed by 71 of 82 residents (87%), and indicated that 51 of those 71 residents had received teaching in some form weekly. The survey results indicated that residents performed well in “Rhinorrhea and Congestion,” “Fever in Children Less than 36 Months,” and “Antibiotic Use in Primary Care” before experiencing the curriculum (table), so these topics were replaced. Residents reported minimal learning about “Temperament in the Pediatric Visit” and “Identification of the Sick Child,” so teaching was modified for these topics.

table

Retrospective Pre-Post Self-Assessment Resultsa

Retrospective Pre-Post Self-Assessment Resultsa
Retrospective Pre-Post Self-Assessment Resultsa

Faculty feedback was positive regarding the curriculum's organization and structure. However, faculty in private practices commented that patient service demands often precluded formal discussion on curricular topics.

Review of usage data from Mailchimp and the LMS revealed that 51% (44 of 87) of residents never accessed the curriculum on the LMS, and 51% (44 of 87) of residents did not open the e-mails. Only 3 of 87 residents accessed half or more of the curricular topics on the LMS. Those who viewed topics on average looked at 7 of the 49 topics. Only 37% (32 of 87) of residents completed any quizzes, and the average completion rate of the quizzes was 8% (4 of 49). Attendance data revealed that pre-CC educational sessions were attended by residents scheduled for CC 46% of the time.

Development of the curriculum relied heavily on the work of the PPCC director (approximately 4 hours per week over 2 years) and resident steering committee (2 to 4 hours annually per resident). However, faculty time spent on implementation was estimated to be the same for the prior primary care education program. The pediatric department had already purchased an LMS platform that was available to the pediatric residency. Sending the weekly topic materials required administrative support similar to prior communication efforts.

This pediatric primary care curriculum, developed in partnership with residents and focused on intrinsic motivation, was feasible and generally well-accepted by faculty and residents, who self-assessed improvements in knowledge and skills. However, actual usage of all curriculum components was low.

At Yale, implementation of a structured primary care curriculum increased interns' participation, satisfaction, and confidence. At our institution, despite a generally positive response to the new multifaceted curriculum, most of the residents did not engage with all components of the curriculum. It was challenging for an attending or resident to cover a topic thoroughly (eg, “autism”) when many residents had done no pre-CC self-study. In addition, some clinic sites did not routinely use the curriculum despite having access to the LMS and engaging in faculty development. Non-uniform implementation meant that residents at these sites who did not do pre-work received no primary care curriculum except during their clinic month at the main site. Residents face similar time restraints across disciplines, so the lessons of this curriculum likely apply to other pediatric programs as well as other primary care specialties. One study of a web-based outpatient curriculum in internal medicine found low levels of engagement—only 60% of residents knew what the topic was when they arrived at the teaching session and only 35% had read pre-session materials—even though the curriculum was viewed favorably.17 

Our data indicated that our reliance on Knowles' model was inadequate. We concluded that the residents still required extrinsic motivators to engage in learning. Ryan and Deci's self-determination theory18  differs from andragogy by concentrating on the learning environment rather than the learner, creating opportunities for behavior that is fully self-determined and addresses psychological needs for autonomy, competence, and relatedness. External motivators that support these 3 needs can help engage adult learners. While the motivators begin as extrinsic, they become internalized and then part of the learner's self-regulation.

Limitations of this study include that it was conducted at 1 institution over 1 academic year and may reflect a cultural context (only 15 minutes of pre-CC group discussion) rather than a failure of our rigorous approach. The survey did not have validity evidence and questions may not have been perceived as intended. In addition, self-assessment is weaker than external assessment as a measure of applied knowledge and clinical performance; however, there are data to support that this survey model correlates more closely to performance than traditional pretests and posttests (due to a reduction in response shift bias).19,20 

In the next iteration of this curriculum, during specific weekly times, residents will reflect on their performance, record these reflections, and write related individual learning goals in MedHub,21  which will be linked to faculty evaluations and curriculum topics, to facilitate intrinsic motivation and autonomous regulation. Future research should study the effectiveness of these extrinsic motivators in facilitating change in residents' learning and practice behavior. Reasons for limited use of the curriculum at community CC sites also require further exploration.

Providing residents a structured PPCC based on the theory of andragogy, with easily accessible curricular materials, resulted in limited engagement in our setting. However, residents perceived an increase in their clinical skills after implementation.

1
American Academy of Pediatrics
.
Pediatrician's Practice and Personal Characteristics: US Only, 2017
. ,
2019
.
2
Accreditation Council for Graduate Medical Education
.
Pediatrics
. ,
2019
.
3
Talwalkar
JS,
Fenick
AM.
Evaluation of a case-based primary care pediatric conference curriculum
.
J Grad Med Educ
.
2011
;
3
(
2
):
224
231
. doi:.
4
Physician Education and Assessment Center
.
Pediatrics Ambulatory Care Curriculum
.
https://ped.peaconline.org. Accessed October 16
,
2019
.
5
Camp
BW,
Gitterman
B,
Headley
R,
Ball
V.
Pediatric residency as preparation for primary care practice
.
Arch Pediatr Adolesc Med
.
1997
;
151
(
1
):
78
83
. doi:.
6
Freed
GL,
Jones
MD,
Dunham
KM,
Switalski
KE,
McGuinness
GA.
Recently trained pediatric subspecialists: perspectives on training and scope of practice
.
Pediatrics
.
2009
;
123
(
1 suppl
):
44
49
. doi:.
7
Rosenberg
AA,
Kamin
C,
Glicken
AD,
Jones
MD.
Training gaps for pediatric residents planning a career in primary care: a qualitative and quantitative study
.
J Grad Med Educ
.
2013
;
3
(
3
):
309
314
. doi:.
8
Dumont-Driscoll
MC,
Barbian
LT,
Pollock
BH.
Pediatric residents' continuity clinics: how are we really doing?
Pediatrics
.
1995
;
96
(
4
):
616
621
.
9
Thomas
PA,
Kern
DE,
Hughes
MT,
Chen
BY,
eds
.
Curriculum Development for Medical Education: A Six-Step Approach. 3rd ed
.
Baltimore, MA
:
Johns Hopkins University Press;
2016
.
10
Knowles
MS,
Holton
EF,
Swanson
RA.
The Adult Learner: The Definitive Classic in Adult Education and Human Resource Development. 8th ed
.
New York, NY
:
Routledge;
2015
.
11
Yale School of Medicine
.
Yale Primary Care Pediatrics Curriculum, 12th ed
.
http://pcpc.yale.edu. Accessed October 29
,
2019
.
12
Desire2Learn
.
Brightspace Learning Environment
. ,
2019
.
13
Mailchimp
.
https://mailchimp.com. Accessed October 29
,
2019
.
14
Bitmoji
.
https://www.bitmoji.com. Accessed October 29
,
2019
.
15
Kirkpatrick
JD,
Kirkpatrick
WK.
Kirkpatrick's Four Levels of Training Evaluation
.
Alexandria, VA
:
ATD Press;
2016
.
16
Miller
GE.
The assessment of clinical skills/competence/performance
.
Acad Med
.
1990
;
65
(
9 suppl
):
63
67
. doi:.
17
Embi
PJ,
Bowen
JL,
Singer
E.
A web-based curriculum to improve residents' education in outpatient medicine
.
Acad Med
.
2001
;
76
(
5
):
545
.
18
Ryan
RM,
Deci
EL.
Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being
.
Am Psychol
.
2000
;
55
(
1
):
68
78
. doi:.
19
Hoogstraten
J.
Influence of objective measures on self-reports in a retrospective pretest-posttest design
.
J Experimental Educ
.
1985
;
53
(
4
):
207
210
.
20
Skeff
KM,
Stratos
GA,
Bergen
MR.
Evaluation of a medical faculty development program
.
Eval Health Prof.
1992
;
15
(
3
);
350
366
.
21
MedHub
.
Healthcare Education Management
.
http://www.medhub.com. Accessed October 29
,
2019
.

Author notes

Editor's Note: The online version of this article contains the pre-post surveys used in the study.

Funding: The authors report no external funding source for this study.

Competing Interests

Conflict of interest: The authors declare they have no competing interests.

This work was previously presented at the Pediatric Academic Societies Annual Meeting, San Francisco, California, May 6–9, 2017, and AMEE, Basel, Switzerland, August 25–29, 2018.

Supplementary data