Balancing Knowledge Among Resident Specialties: Lecture-Based Training and the OUCH Card to Treat Children's Pain

There are unique concerns for residents and fellows treating hospitalized children in pain. Some children cannot necessarily communicate their pain,1,2 and the dosages for children differ dramatically by age and weight.3 Finally, because of a fear of adverse effects, the undertreatment of pain (ie, giving too little medicine) has been reported.4,5 

Lapses in the treatment of children's pain have been attributed, in part, to the following: systematic and logistical barriers within institutions providing pediatric care,6 knowledge-based barriers including insufficient knowledge of pain assessment and treatment in children,7 and inadequate pediatric pain management (PPM) educational requirements for residency programs.8 These inadequacies have also been shown to be widespread across a variety of pediatric settings such as cardiac surgery, emergency medicine, and terminal cancer.5,9–16 Pediatric residents have also self-identified their lack of knowledge in pain and symptom management,17 and pediatric attending physicians have rated the need for resident training for pediatric pain, pain assessment, and pharmacology of analgesics as moderate to high.18 

To further evaluate this perceived educational need, we recently developed and preliminarily validated a resident PPM knowledge assessment in a multiple-choice test format.8 In using this instrument, we showed that there were clear PPM knowledge deficiencies among the resident specialties; specifically, pediatric and orthopedic residents performed significantly poorer on this knowledge assessment when compared to a group of anesthesiology residents.8 

Despite identifying the PPM issue in the literature, there are only a few known educational interventions that demonstrate ways to improve clinicians' knowledge of acute-pain assessment. Existing studies have focused on the teaching of behavioral techniques19,20 or pharmacologic management of children in pain.21 In an attempt to minimize the PPM knowledge deficiencies of pediatric residents, we developed the OUCH card—a portable reference for acute-pain management in hospitalized children. While the goal of increasing pediatric residents' PPM knowledge is worthwhile, it is even more important for them to have access to information that will provide answers. Such access will allow the most commonly asked questions to be answered correctly at the time the information is needed (eg, at the bedside). Moreover, the introduction of readily available reference cards in clinical settings has been described as useful in environments such as adult heart care, palliative care, and internal medicine.22–25 

The present study addresses the issue of pediatric residents' comparably low PPM knowledge.8 It tests the extent to which providing pediatric residents with lecture-based PPM training (to increase their knowledge), as well as a targeted PPM memory aid at the point of decision during their clinical activities (to decrease their reliance on memory or limited experience), has a positive and long-term effect. Specifically, through training and a memory aid, it is predicted that they will perform no differently than their generally more knowledgeable and more experienced anesthesiology resident colleagues.

Participants were a convenience sample of pediatric, orthopedic, and anesthesiology residents who were (1) in attendance at their respective PPM lecture and (2) willing to take a follow-up PPM knowledge assessment test 4 weeks later. The PPM lecture was part of each specialty's residency core lecture series that on-site residents are required to attend. Institutional Review Board exemption was granted for all resident groups.

All lecture participants completed a performance assessment prior to its start. The lecture then began and “OUCH” cards were distributed. Participants, located in their home departments and available for reassessment, completed the alternate form of the knowledge assessment approximately 4 weeks following the lecture-based educational intervention. Comparison of knowledge assessment scores by resident specialty before the lecture and again 4 weeks after the lecture constitutes the primary outcome of interest. Usage of the OUCH card was encouraged during completion of the posttest to simulate its use as a clinical tool.

To construct the PPM knowledge test, we reviewed the International Association for the Study of Pain outline curriculum for medical schools26 as it pertains to children and infants. Using this as a reference, we constructed items that were measurable under 2 general categories related to acute pain in hospitalized children: (1) pediatric pain assessment and treatment and (2) recognition and treatment of adverse drug events (ADEs).

Ten specific knowledge domains were empirically defined based on the types of questions that are commonly asked of the PPM service at our institution and based on our review of the related literature.8,27 Six domains related to pediatric pain assessment and treatment, and 4 related to recognition and treatment of ADEs. We wrote 2 multiple-choice questions for each of the 10 knowledge domains, resulting in a total of 20 items, and then constructed 2 parallel test forms (Form A and Form B). Psychometric analyses consisted of calculation of total mean difficulty indices, which were comparable8 between Form A (0.62) and Form B (0.63). The discrimination index, which reflects the relative likelihood that high-scoring respondents will endorse the item compared to low-scoring respondents and which can range from −1.00 to a perfect score of 1.00, had a positive value (0.08–0.56) for 19 out of 20 questions.

The introduction of readily available reference cards has been reported as useful in pain management,25 palliative care,22 acute heart care,23 and continuing medical education.24 In the year prior to the implementation of the PPM education initiative, the pediatric residents at our institution created laminated portable reference cards for their pediatric subspecialty rotations and inpatient general pediatric rotations. Based on the popularity of the resident-created cards, the PPM faculty created the OUCH card, a portable reference for acute-pain management in hospitalized children.

Analgesic dosing information, which is consistent with published guidelines,28–30 appears on the front of the first card. In addition, we selected equivalency ratios for intravenous morphine and intravenous hydromorphone, based on the equianalgesic ratio, that have been shown to be applicable in children.31 Pain-assessment tools, including the Faces Pain Scale–Revised32 and the Face, Leg, Activity, Cry, Consolability Scale,33,34 which are well-established measures according to reviews of pediatric-pain measures,35,36 are shown on the back of the first card. The Faces Pain Scale–Revised may be used with developmentally normal children from approximately 3 to 7 years of age, and the Face, Leg, Activity, Cry, Consolability Scale may be used for nonverbal and cognitively impaired children. The treatment of opioid-related ADEs, such as nausea, vomiting,37 pruritus, and constipation, appears on the front of the second card. These dosing recommendations are also consistent with published guidelines.29,30 Constipation guidelines are also based on practices developed during a quality initiative, led by the Child Health Corporation of America, to reduce opioid-related ADEs in children; these have been added since the card's initial development. On the back of the second card, instructions regarding how to request a PPM consult are listed. Phone numbers for a pediatric pain or anesthesiology consultation were also included but are represented by X's. The cards are small enough at 15 cm (width) × 13 cm (height) to fit in a resident's white-coat pocket, and they match the format and size of the resident-created cards. Printing and lamination of the first 200 cards cost $2.50 per card.

One instructor (J.S.) presented the designated PPM content in a didactic slide-based lecture for each of the 3 subspecialty groups of interest. Distribution of the OUCH card occurred immediately after participants completed the pre-lecture PPM knowledge assessment and just prior to the PPM lecture. During the PPM lecture, the OUCH card content was reviewed. Proper usage of the pain intensity measurement scales was taught. Finally, attendees were encouraged to use the OUCH card in daily practice.

The alternate-form posttest was administered approximately 4 weeks later. The investigator requested that willing participants who completed Form B on the pretest complete Form A on the posttest and vice versa. Since our intended goal was to improve access to PPM assessment and pharmacotherapy, the residents were allowed to use the OUCH cards while completing this assessment. Use of the OUCH card on the posttest was indicated by the participant.

Following participants' completion of the posttest, a survey was immediately administered to assess their overall satisfaction with the content and the utility of the OUCH card. On a 5-point Likert scale (1  =  strongly disagree, 5  =  strongly agree), each participant ranked portability, legibility, organization, and content of the OUCH card.22 Space for written comments about the OUCH card was also included.

Initially, to test whether the psychometric characteristics of the PPM knowledge test were similar to the original study,8 analyses were conducted by calculating difficulty and discrimination indices to establish item and assessment form reliability for the present samples of residents. The comparison of performance scores by resident specialty at repeat testing (pretest and posttest) is the primary analysis of interest. More specifically, both the pretest and posttest scores are presented as the total number of questions answered correctly out of 10. Since the level of training for residents ranged from postgraduate year 1 to postgraduate year 5, we included it as a covariate measure to control for years of experience. Thus, our design for analysis is a 3 (resident specialty) × 2 (pretest/posttest) repeated measures mixed analysis of covariance. We tested main effects for specialty and test, as well as for the interaction term (ie, whether some specialties improved pretest to posttest and not others). Scores are presented as means ± SE. Data were analyzed using SPSS 14.0 (SPSS Inc, Chicago, IL). A P value of <.05 was deemed significant. For the OUCH card satisfaction questionnaire, the distribution, means, and standard deviations of the Likert responses were calculated.

Thirty-nine residents (n  =  39; 15 anesthesiology, 13 orthopedic, and 11 pediatric) completed both the PPM pretest and posttest. This is a subgroup of 60 residents (15 anesthesiology, 19 orthopedic, and 26 pediatric) who completed the PPM pretest prior to the PPM lecture and distribution of the OUCH card.8 Thus, of all the residents who completed the pretest, 65% (39 of 60) also completed the posttest. On average, participants completed the posttest 4 weeks after the pretest and lecture (SD  =  13 days). Based on our previous research8 showing that the alternate PPM test forms had comparable difficulty indices, data from Forms A and B were combined prior to the calculation of pretest and posttest means.

In the psychometric analyses, none of the items on the 2 forms had a negative discriminating index. Difficulty and discrimination indices are presented in table 1. Of the 20 questions, 19 had a positive value (0.08–0.56) for the discrimination index. Of note, comparison of mean difficulty indices from pretest to posttest on both Forms A and B improved (ie, the number of correct responses increased) for questions related to pediatric pain assessment and management. Mean difficulty indices for questions related to analgesic-related ADEs remained relatively unchanged.

For the repeated measures analysis of covariance, the covariate of residency year was significant with a small effect size (F_1,35  =  5.44, P  =  .026, η²  =  .135; see table 2). After accounting for the intrasubject variance based on residency year, the interaction term of residency specialty and test performance was significant with a moderate to low effect size (F_{2, 35}  =  5.23, P  =  .01, η²  =  .23). The adjusted mean scores for the 39 residents by specialty are shown in figure 3.

As previously mentioned, the performance of the 39 residents reported in the present study is a subset of 60 resident participants.8 For each specialty, all 15 anesthesiology residents completed the pretest and posttest, as did 13 of the 19 orthopedics residents. Due to the low response rate among the pediatrics residents (11 of 26), an analysis of their pretest scores was conducted to test whether the scores of those who completed the pretest and posttest were somehow different from those who completed only the pretest. There were no significant pretest differences found between these two groups of pediatrics residents (P  =  .47), suggesting that pediatrics residents completing the posttest were no different in PPM knowledge—at pretest—as compared to their colleagues. In addition, there were no significant differences in age or mean resident year of training in these 2 groups. Finally, while 25 of the 39 residents (64%) reported using the OUCH card in the posttest, average posttest scores did not differ significantly between the use and no-use groups (P  =  .105).

Participants' generally favorable ratings of the portability, legibility, content, and organization of the OUCH cards are presented in table 3. Three participants wrote in comments, one relating to format (“may be easier to use if it was a single sheet, folded in half, rather than two sheets), one relating to content (“include infusion doses for patient-controlled analgesia”), and one relating to size (“wallet size would be nice”).

The OUCH card was developed as a clinical and instructional aid to be distributed in conjunction with PPM lecture. Readily accessible PPM assessment and treatment was meant to reinforce valuable information presented in lecture, which is not always easily recalled when needed the most. In an effort to assess the effect of the OUCH card and/or lecture on completion of a knowledge assessment, residents from 3 different specialties completed a posttest approximately 4 weeks after their pretest. Our results show preliminary evidence that a PPM lecture, combined with the use of a portable reference card (the OUCH card), may reduce differences in residents' performance on a knowledge assessment of acute PPM in hospitalized children.

There is also preliminary support for the effect of lecture plus OUCH card usage on posttest performance because of the crossover examination design of the study (which avoided the possibility of test-retest error through alternate forms). Item analyses showed good psychometric properties in pretest as well as in posttest knowledge assessments on Forms A and B. Improvement of the difficulty indices for questions related to PPM compared to analgesic ADEs lends further confirmation to the effect of lecture plus OUCH card on resident performance on the posttest knowledge assessment.

To improve performance by allowing residents to use the OUCH card while completing the posttest knowledge assessment may be considered an obvious finding or a fundamental weakness in study design. To the contrary, we purport that the means by which the residents obtained the right answer is less important than the fact that they were able to obtain the correct answer to a clinically relevant PPM question.

In terms of usability, 80% or more of posttest completers rated the OUCH card as portable, easy to read, well-organized, and containing appropriate content. For this reason, only updates and additions to content have been made. The suggestion to include patient-controlled analgesia settings in children is being considered for an OUCH card distributed to anesthesiology residents who rotate on the acute PPM inpatient service and write these orders.

Based on the beneficial results of residents having easily accessible, accurate PPM information in the setting of knowledge assessment, one future endeavor may be the transfer of PPM content to a portable digital assistant—a preferred vehicle for text references by emergency medicine and pediatric residents in clinical settings.38 Regardless of venue, if tools like the OUCH card are incorporated into clinical practice, yearly review by pharmacists and PPM faculty in lieu of any changes in best practice should be planned. For example, the next iteration of the OUCH card will include the revised Face, Leg, Activity, Cry, Consolability Scale observational pain tool, which shows improved reliability and validity for pain assessment in children with cognitive impairment.39 

Finally, determining best practices for the most efficient and cost-effective means of delivering pain management education is a greatly needed step40 that may be discernible in subsequent studies. The effect of lecture versus OUCH card distribution (as well as their combination) merits further investigation in order to better understand the optimal content and format of training programs. Future research on improvement in pain control should be done in multiple institutions to show that the present results can be generalized. Larger samples may also permit an analysis of whether frequency of OUCH card usage influences posttest performance.

Despite multiple efforts to obtain posttest completion by participants, the response rate of those who took the posttest was less than the pretest and less than other before-and-after studies of palliative care22 or pain management.25 Pediatric residents who rotate through multiple clinical sites in and around our institution were especially difficult to include in the posttest. Only 11 of 26 pediatric residents completed the posttest. In comparison, all anesthesiology residents (15 of 15) completed both the pretest and posttest, and two-thirds of orthopedic residents (13 of 19) completed both. Such convenience sampling is relatively common in educational research on “difficult to obtain” populations like residents, but it nevertheless raises questions of bias.8 

The lower response rate of pediatric residents through attrition and the increased accessibility of anesthesiology and orthopedic residents are threats to the internal validity of the study (eg, selection-mortality). It is impossible to know whether our performance results overestimated or underestimated pediatric residents' knowledge of PPM and whether these findings are specific to only this group. This limits how applicable our results are to other settings. The nonsignificant difference in the comparison of pretest scores, age, and year of training among pediatric completers of the posttest versus noncompleters partially mitigates the effect of the large attrition rate. Our approach to PPM education (lecture plus learning aid distribution) does not address the efficacy of each intervention alone. This is pertinent to those involved in resident and faculty education when decisions about allocation of resources and time are made.

We did not measure whether there was any improvement, clinically, in the actual pain management of the children cared for by the participating residents. In the absence of this clinical data it is not possible to show the ultimate effectiveness of the intervention. Future research must address this. Perhaps the recent development of electronic entry for the prescribing of medications to patients who are being discharged from clinic or hospital offers a new avenue of clinically relevant before-and-after measurement of analgesic dosing errors.

This study raises several practical implications about how to approach teaching and testing of pain management knowledge. First, distribution of aids like the OUCH card, in addition to lecture on use of the card, may improve instruction of resident pain management education by reinforcing the “take home” points.

Second, faculty should consider adopting the “open book” approach used in this study. For example, at the beginning of a new academic year, faculty could encourage distribution of preselected clinical information succinctly presented and portable (though such information would need to be subject to quality control and expert agreement). We do not believe that making available such “cheat notes” constitutes “cheating,” especially when issues of children's pain management are involved. Instead, distribution of learning aids such as the OUCH card may help residents with relatively little clinical experience in PPM to learn useful material more quickly and make better decisions, possibly improving clinical care.

Third, faculty may also include the OUCH card as a clinical tool for their own practice. For an attending on rounds with residents, for example, using the OUCH card to measure pain intensity in a child accomplishes 2 goals simultaneously: assessing a child's pain and role modeling pain measurement.

Finally, and though only speculative at this point because of the limitations of our study, there was notable improvement in single-item performance for use of pain assessment, opioid conversion, and dosing of acetaminophen. In terms of practical significance, the improvement of pediatric-pain assessment and treatment item performance suggests that if a pediatric resident has access to this information at the time of treating a child, then the widespread underdosing and overdosing of analgesics and underassessment of children in pain may be reduced.

Introduction of a knowledge aid (the OUCH card) within the context of PPM training led to a significant improvement in knowledge-assessment performance by the resident population of interest, pediatrics. The topics specifically related to PPM in a hospitalized child showed the most improvement. Further investigation of PPM knowledge aids (such as the OUCH card) is merited for PPM knowledge assessment and clinical practice. Faculty members who supervise residents charged with treating hospitalized children who experience pain should consider incorporating a similar learning aid into their clinical practice and teaching.