Women's Health Curriculum for Internal Medicine Residents: Development, Implementation, and Evaluation

Women's health is a public health priority.1 Women have unique health care needs, are affected by some diseases more frequently than men, and can present differently than men with the same condition.2,3 Provider lack of knowledge may contribute to disparities in the care of women.4–7 Many believe medical education in women's health is not adequate, and several national organizations have called for increased training.8–15 Internal medicine physicians, specifically, may not be well prepared to care for women,16,17 and internal medicine residents have reported a lower satisfaction level with their women's health training compared with family medicine and obstetrics-gynecology residents.18 In early 2006, the Morgantown West Virginia University (WVU) internal medicine residency program did not have a formal curriculum for meeting gender-specific training requirements. The objective of this project was to develop, implement, and evaluate a women's health curriculum for an internal medicine residency program.

The residency program faced several barriers to providing women's health education. Specialists in women's health in the internal medicine department were limited; involving appropriate faculty would necessitate crossing specialty lines and might make lecture scheduling difficult. There also was no external funding for this project. However, the institution had recently been recognized as a National Center of Excellence in Women's Health, which helped to identify a multidisciplinary group of women's health faculty. The new center also provided a culture of support for women's health teaching, as education was a major focus area of the National Center of Excellence in Women's Health. Also, WVU had its own “Secure On-Line Environment” (SOLE) web-based educational portal for online teaching. During the planning process, an opportunity arose for obtaining a small amount of internal funding to strengthen the curriculum's evaluation.

The internist course director and the internal medicine program director planned the curriculum with help from a faculty member experienced in educational evaluations. This project included a web-based tutorial, as this method of training is increasingly being used in medical education.19–24 Including a web-based tutorial with a multidisciplinary clinical rotation would provide broad clinical experience and consistent didactic education with a limited time required from faculty. The SOLE portal could house the web-based tutorial at no added cost, and the inclusion of tests and surveys in the tutorial would simplify assessment of learning outcomes. Neither a multidisciplinary clinical rotation nor a web-based tutorial had been used previously in this residency program.

We contacted faculty representing multiple clinical specialties and who had an interest in women's health. We asked them to contribute teaching materials (new or already developed) to the web-based tutorial and/or to precept for the clinical rotation. The faculty included physicians from obstetrics-gynecology, reproductive medicine, general internal medicine, family medicine, urology, neurology, psychiatry, breast surgery, hematology/oncology, and a nurse practitioner from gynecology. The topics for the curriculum were chosen from a comprehensive set of women's health topics recommended by the Federated Council for Internal Medicine Task Force13 and the American Board of Internal Medicine.11 A multifaceted evaluation was planned (table 1). An incentive of $35 was offered to participating residents and controls to encourage completion of the curriculum evaluation. The WVU institutional review board exempted this project from review as it was categorized as program evaluation.

The curriculum was implemented in September 2006. This was a mandatory senior resident rotation with no call, and 1 to 2 residents participated monthly. A total of 24 residents completing the program during 16 months of data collection. The clinical rotation was shared with a geriatrics rotation to satisfy 2 training requirements in 1 month and to provide scheduling flexibility. The learning objectives, requirements, and evaluations of the 2 rotations were completely independent. Participants consisted of residents of both genders with a variety of future career plans such as primary care, hospital medicine, or fellowships. The chief resident assigned residents to half-days in various ambulatory settings, primarily in the core disciplines of internal medicine and gynecology. Other assignments included urogynecology, eating disorders, breast cancer oncology, and breast surgery clinic. Several half-days were also designated for independent study and could be used for completion of the tutorial. The curriculum covered most of the women's health topics recommended by Federated Council for Internal Medicine (76%) and American Board of Internal Medicine (85%).

We made the tutorial available through the SOLE portal, which could be accessed 24 hours per day. It contained 16 teaching modules in a variety of formats such as short written chapters, narrated PowerPoint (Microsoft Corp, Redmond, WA) lectures, and self-assessment quizzes (table 2). After implementation, the web-based tutorial teaching modules were revised for publication in a national repository for medical teaching materials, where they can be widely accessed.25 A pretest and identical posttest consisting of 46 board-style questions displayed in random order were administered through the web-based tutorial. The tests were composed of existing questions from several sources; new questions were written for areas in which major content was not addressed. Residents were instructed to take the pretest before completing the teaching modules and to take the posttest at the end of the rotation, approximately 1 month later. Scores were given immediately on completion of the tests. Residents could see the questions, their answers, and whether they were right, but were not given the correct answer when wrong. After the posttest, residents scoring 80% or better were given automatic feedback congratulating them on mastering the material. Those scoring lower were advised that they should review course content, although 70% was the assigned cut-off for the tutorial to record as “passing.” Taking the tests was mandatory and was enforced by the course director withholding resident performance evaluations until completion. However, test scores were not part of these evaluations and a passing score was not required to earn credit for the rotation. Time spent on the teaching modules was at the residents' discretion, and assessment of module completion was self-reported.

Mean scores on pretests and posttests were compared with independent sample t tests using the JMP statistical software package (JMP-SAS Institute, Cary, NC). We analyzed data for 20 participating residents (3 of these completed the pretest only), excluding 4 residents who did not take the tests or took the pretest after completing the tutorial. To reduce test familiarity as a potential threat to validity, control residents also took pretests and posttests on SOLE 1 month apart. The number of potential internal controls was limited by the fact that the rotation was required, so internal medicine residents from the WVU School of Medicine program in Charleston (which did not offer a dedicated women's health curriculum) were chosen. To maximize the number of controls (with a goal of 15 to 20), all 44 Charleston residents were offered participation. Although 10 agreed, only 6 completed the tests (1 of these completed the pretest only).

A statistically significant increase was seen in participating residents' mean posttest scores (71.7%, [SD 10.2%]) compared with pretest scores (61.1% [7.7%]) (difference, 10.7%; 95% confidence interval [CI]: 4.7–16.6; P  =  .0009). No difference was seen in control residents' mean posttest scores (56.5% [7.7%]) versus pretest scores (57.2% [8.8%]) (difference, −0.7%; 95% CI: −12.1–10.7; P  =  .9). There was no statistically significant difference in mean participant and control pretest scores (difference, 3.8%; 95% CI: −3.8–11.5; P  =  .3). A significant difference was seen in mean participant and control posttest scores (difference, 15.2%; 95% CI: 4.9–25.6; P  =  .006). The Cohen's d for participating residents' post versus pretest score difference was 1.2, a large effect size. Of the 17 participating residents who took the posttest, 10 (59%) achieved at least the passing score of 70%, with 5 (29%) of those achieving the “mastery” score of 80%. In comparison, on the pretest, only 2 (10%) achieved a passing score and none achieved a mastery score. Question-specific statistics from the posttests were evaluated to identify areas of curriculum weakness. The questions were grouped into 10 topics. The topics with lower average scores were osteoporosis, screening/prevention, breast problems, pregnancy, and pap screening. The 6 questions that were answered correctly by less than 50% of participants included 2 osteoporosis questions, 2 pregnancy questions, and 2 pap screening questions.

An evaluation survey was developed and incorporated into the tutorial to obtain more information than what the standard web-based rotation evaluations could provide. The survey was pretested and revised before implementation. The rotation evaluation and survey results were analyzed with univariate descriptive statistics and qualitative comments were compiled. Mean rotation evaluation responses ranged from 7.09 to 7.45 on a 9-point scale as shown in table 3. The women's health rotation was also ranked in comparison with other rotations by using a ranking feature of the rotation evaluation program. Three ranking methods were conducted to increase validity. All methods placed the women's health rotation in the middle tertile. The majority (77%) of survey respondents reported completing all or almost all of the tutorial. All agreed that the curriculum was well organized, that it increased their awareness of women's health issues, and that the addition of the web-based tutorial to the clinical rotation was beneficial. The majority (93%) agreed that the curriculum increased their skills in caring for women. table 4 shows participants' assessments of program components that enhanced learning. It also identifies topics that residents felt confident discussing with patients, conditions they felt comfortable treating, and procedures they felt confident performing after the rotation.

Our novel women's health education program was associated with improvements in learning outcomes. Posttest scores significantly improved, and residents felt confident discussing and treating many conditions related to women's health after curriculum completion. The average ranking of the rotation was considered to be satisfactory because it was mandatory for residents who were not necessarily interested in women's health. Residents' survey responses were mostly positive. For example, the residents unanimously agreed with the survey items stating “the curriculum was well organized” and “the web-based tutorial was beneficial.” Although there were fewer controls than planned, this rotation's evaluation contained several techniques that strengthen medical curriculum evaluations: controls were used, effect size was included, and multiple outcomes were assessed.26–30 The program may have other benefits that were not directly assessed, such as pretests and posttests stimulating self-directed learning or helping with board preparation. On the other hand, we identified topics in which many residents lacked confidence, which allowed for targeted program improvement.

The experience at our institution and our learning about curriculum needs and improvements may be useful for other programs looking to develop a women's health curriculum. More opportunities to prescribe contraception and to prepare and review vaginal wet mounts would be helpful as less than half of residents were confident in those skills after completing the program. Those topics and several others with low posttest scores (osteoporosis, screening/prevention, pap screening) are basic areas of women's health. Although the multidisciplinary nature of the clinical rotation is a strength, a larger emphasis on primary care and gynecology may improve the learning of these core topics. The lower rankings of the specialty sites by residents may reflect a perception of less practical usefulness, and one half-day in each of those areas would probably be sufficient to provide exposure. The majority of nationally recommended women's health content areas were covered. However the curriculum could be strengthened by the addition of others, such as care of women during pregnancy, as this topic was covered only on the tutorial, and questions relating to pregnancy received low posttest scores. Supplemental face-to-face teaching could take the place of some of the tutorial's PowerPoint lectures because that teaching format was ranked relatively low.

This curriculum satisfied important women's health training requirements that can be difficult for programs to meet. The web-based tutorial provided standardized didactics at no added cost. Minimal time was required after initial program development, primarily the time needed for scheduling the rotation. The tutorial's tests and survey simplified learning outcome assessments. After successfully implementing this approach for training internal medicine residents, the educational program was adapted for use as a medical student elective.

In summary, the format of a multidisciplinary clinical rotation in conjunction with this or another web-based tutorial may be a useful model for women's health education.