Your program evaluation team has completed its evaluation of a comprehensive diversity, equity, and inclusion (DEI) workshop series for program residents and faculty. The workshops were designed to foster dialogue about DEI and build skillsets, including ways to address microaggressions. You are now scheduled to preview the evaluation results individually with your program director and your vice chair for DEI, and then present the full report at the next combined resident/fellow and faculty meeting. As each of these stakeholders have different perspectives and their availability to meet varies from 15 to 45 minutes, the challenge is how to approach presenting the evaluation results to each of these groups.
Reporting the results of a program evaluation must explicitly consider how to:
Align the results with the original evaluation questions and stakeholders' inputs.
Make it actionable: evaluations are conducted to inform decisions.
Adapt report to stakeholder audience and present using multiple formats and media.
Follow up to ascertain changes associated with evaluation.
What Is Known
Systematically designed evaluations yield information about the value of a program, project, or initiative to inform key stakeholders' decisions regarding the program (eg, continuation, revision, expansion). Typically, evaluation reports include the results of the evaluation (“What”), interpretation of results (“So What”), and recommendations for continuing and improving the educational activity (“Now What”). How evaluation findings are communicated directly influences how stakeholders understand and react to the data and ultimately their decisions. Evaluators use both comprehensive reports and targeted presentations to address stakeholders' information needs—evidence that matters to them. Targeted presentations can be as short as an “elevator pitch” or as involved as a visual abstract.1 Using data visualizations (graphic or pictorial formats) for quantitative results (graphs, pie charts, diagrams) enables decision-makers to quickly grasp difficult concepts or identify new patterns.2 Qualitative data can be effectively presented through word clouds, photos, and quotations.
How You Can Start TODAY
What: Return to your evaluation questions and evaluation standards. Use the questions generated by key stakeholders to organize your presentation. Consider framing the results by levels of data (reaction, learning, behavior, results). Highlight the key findings for all groups then focus on areas by stakeholder interest. Make sure your findings are accurate and your recommendations are useful; demonstrate integrity by differentiating results from opinion.
Now What: Adapt your report to the stakeholder audience. Presentations and reports may differ by stakeholder group. Do you want to inform or persuade? Choose written or verbal reports that best tell the “evaluation story.”2 Use simple, plain writing. Include quotations, specific examples, and/or a case study. Metaphors help statistics have practical impact and build the story. Use data visualizations.2 Evaluation snapshots (short summaries) or storybooks3 are useful formats for quick review of key points. Emphasize data specific to your stakeholder input(s) during the planning stage (Figure). Presentations by resident members of your evaluation team can be powerful.
Now What: Reinforce use of evaluation results. Seek to present the findings in multiple forums to reinforce messages. Identify who needs or wants to see or hear the evaluation findings. Consider existing forums, such as regularly occurring meetings (eg, residency curriculum committee, clinical competency committee, graduate medical education council) and new forums.
So What: Make your report actionable. Make sure recommendations are relevant, useful, aligned with stakeholder values, and actionable.
What You Can Do LONG TERM
What: Maintain an evaluation master file. Include all the various presentations and reports your team has created. Consider using your evaluation findings to populate the annual program evaluation or self-study, and sponsoring institution's documents.
So What: Revisit your evaluation report in a year. Check the utility of your evaluation by stakeholder groups. Was there action taken based on the report? Were your recommendations followed?
Now What: Consider disseminating your evaluation as scholarship. Your evaluation results might contribute to a larger conversation about learning in graduate medical education. Learn about how program evaluations are best presented4 or consult with an evaluation expert.