Preparing for Oral Scientific and Clinical Vignette Presentations

The dissemination of scholarly work is vital to furthering medical practice and academic advancement. The Accreditation Council for Graduate Medical Education requires residents to participate in scholarly activities, including presenting at local and national professional meetings.1,2 

Accomplished speakers provide content that is relevant and specific, using slides that are clear and readable, and a presentation style that is well paced and engaging.3 Strategies to deliver an outstanding oral presentation offer guidance on “what” to teach,4–11 but empirical evidence is scarce regarding “how” to teach these skills,12–14 and specific recommendations vary widely.15–17 

To our knowledge, no study has examined how presenters, particularly residents and fellows, practice or receive feedback prior to an oral scientific presentation. We sought to categorize presenters' practice styles and the impact of feedback received prior to an oral presentation at academic meetings. Our findings may assist program directors and mentors in institutionalizing practice sessions.

 Using an iterative process, 5 clinician-educators developed a 9-item survey based on recommendations from the literature.4–11 The survey was pilot-tested at a regional meeting in 2008, and 2 self-efficacy questions were added from 2009 to 2010. Questions asked about sex, training level, presentation type (research or clinical vignette), practice frequency (ie, no practice, 1–3 times, 4–6 times, and >6 times), practice settings (practice alone; send slides to a mentor for review; practice in front of a nonexperienced group [nonmedical friends, medical peers]; practice in front of a group of more experienced colleagues; and other [open-ended]), the practice setting perceived as most helpful, and changes made in response to feedback (content changes [content and focus of the presentation, learning objectives, conclusions]; visual changes [font and text, number of words, slide background, graphics, animation]; delivery style changes [presentation style, mannerisms]; and other [open-ended]).

We computed the number of practice settings (5, maximum) and the number of change types (10, maximum) for analyses. We based these answer choices on previous work with oral presentations by Estrada et al.3 The 2 self-efficacy questions, scored using a Likert-type scale, asked about the perceived quality of the responder's presentation relative to other presentations at the same meeting (ranging from 1  =  far worse than others to 5  =  much better) and the impact of feedback on the preparation for the presentation (where 1  =  very detrimental to 5  =  very useful). Finally, responders could identify their own presentation with 2 optional items. The survey is provided as online supplemental material. The University of Alabama at Birmingham Institutional Review Board approved the study. No incentives were offered for participation.

 We conducted a cross-sectional study at 3 regional (2008–2010) and 2 national meetings (2009–2010) of the Society of General Internal Medicine. After each meeting, we invited all oral research and clinical vignette presenters to complete a website-based survey.

 We used analysis of variance (ANOVA) to compare mean scores among training levels, the t test to compare means, and the χ² test to compare proportions.

The response rate was 63% (333 of 525 responders). Responders represented 59 institutions from across the United States (98%; 137/140 responders who provided their institutions), Canada, and Switzerland (table 1).

 The most common practice settings were the least helpful ones (figure). While 268 responders (81%) reported practicing alone, only 82 responders (25%) reported it was the most helpful setting. One hundred fifty-two responders (46%) reported practicing with more experienced colleagues or mentors, and 132 responders (40%) reported this was the most helpful practice setting (80%; 122/152 responders). Faculty reported practicing fewer times in fewer settings (both, P < .001) (table 2).

 The most commonly reported type of changes were those made to visual aids (68%; 227/333 responders), followed by changes to content (55%; 183/333 responders) and to delivery style (31%; 103/333 responders). On average, responders reported making 2.7 (±SD, 1.9) change types as a result of feedback. The mean number of change types reported by faculty were fewer than those made by residents and fellows (2.3 ± 1.8, and 3.1 ± 2.0, respectively; P < .001).

The type of changes made by practice setting is shown in figure 2. Responders who either practiced in front of a group or sent slides for review reported making significantly more change types in each domain than those who did not. However, practicing alone did not result in any significant change types made to content, visual display, or delivery style.

 Responders who practiced in front of a group of more experienced colleagues reported their presentations were of a higher quality than those of responders who did not (3.7 ± 0.8, and 3.5 ± 0.7, respectively; P  =  .03); however, other practice settings were not associated with higher quality (eg, those who practiced alone, P  =  .80; by slide review, P  =  .34; who practiced with nonexperienced group, P  =  .16; and data not shown).

Responders who practiced in front of a group of more experienced colleagues reported the feedback from this setting had a greater impact on their presentations than that reported by responders who practiced in other settings (4.7 ± 0.6, and 4.3 ± 0.8, respectively; P < .001); similarly, other practice settings were also associated with greater feedback impact on presentations (those practiced alone, P < .001; by slide review, P < .001; those practiced with a nonexperienced group, P < .001; and data not shown).

At a general internal medicine academic meeting, most presenters utilized the most convenient yet least helpful practice settings: practicing alone, slide review, and practicing in front of a nonexperienced group. Practicing in front of a group of experienced colleagues was perceived as the most helpful but least utilized setting. Practicing alone was not associated with any changes in content, visual display, or delivery style, while other practice settings did result in significant changes in each domain.

Our study has important implications for program directors and mentors. First, practicing in front of a group of experienced colleagues requires institutional support, infrastructure, and organization. It is unrealistic to expect residents and fellows to arrange these practice sessions. Even faculty did not frequently utilize a group practice session, likely due to logistical barriers.

Second, we observed that changes made to presentations varied based on practice setting. The educational literature suggests that self-assessment is often not correlated with external assessment of oral presentations,18,19 perhaps due to a lack of insight or overconfidence. Feedback from others is critical to identify areas that need clarification and simplification, as well as difficulties with delivery style. Across all settings, delivery style was less likely to change. Delivery style may be determined both by the presenter's “intrinsic” factors (eg, personality, experience, knowledge of topic) and “extrinsic” factors (audience, setting). Finally, presenters may not have the knowledge or skills required to make changes, even when problems are recognized.

Because teachers are often viewed as role models, faculty would also benefit from practicing presentations in multiple settings. In our study, faculty practiced in the least number of settings. Practicing alone may be insufficient, even with substantial public speaking experience and “expert” level knowledge of the topic.

The benefits of practicing in front of a group of experienced colleagues is based on a social cognitive perspective on self-directed learning theoretical framework.12 Learning any new skill requires 4 cognitive processes: (1) attention, (2) retention, (3) production, and (4) motivation. In attention, learners are exposed to earlier versions of an oral presentation, which develop over time into more polished versions. Retention is increased by the use of “multichannel” instruction, such as video, open discussion, and rubrics, to rate performance. This instruction process is based on the “dual-channel” theory, where each channel (ie, visual and auditory) has limited capacity for learning. Production occurs during the repeated practice sessions with internal and external feedback. Finally, internal and external motivators tend to improve performance.12 

Study limitations include the fact that our data represent one academic society's meetings, the potential for recall bias, which may have influenced some responses, and use of a nonvalidated survey.

Practicing in front of a group of experienced colleagues was the most helpful practice setting in which to prepare for an oral academic meeting presentation, but it is not universally utilized. Practicing alone was utilized frequently but did not lead to changes in content, visual display, or delivery style.