A researcher wonders how to navigate conflicting recommendations from 2 senior mentors on her research project, which explores how residents decide which patients to follow after hospital discharge. One mentor insists she define specific hypotheses about which motivational factors influenced residents to follow patients after discharge. The other strongly encourages her to begin with an exploratory approach, by collecting and analyzing residents' descriptions of the process of following patients after discharge, to identify patterns and themes in decision-making processes. The researcher is uncertain how to proceed and is concerned she is falling behind on her project timeline.
In a 2016 editorial, Picho and Artino1 offered a checklist of recommended practices for the responsible conduct of research. Their recommendations are enticingly clear and concise; they identify specific actions that can help researchers avoid the consequences associated with the “7 deadly sins” in health professions education research. Yet, as illustrated in the dilemma above, decisions in research are not always straightforward judgments of right or wrong. As researchers primarily involved in qualitative inquiry, we are keenly aware of the nuance, complexity, and uncertainty in the research process. As Varpio and Meyer2(p155) wrote, “There are few, if any, universally applicable ‘gold standards' for qualitative research.” Many aspects of the research process (quantitative or qualitative) require careful judgment that takes context, circumstances, competing priorities, and values into consideration.
In this editorial, we attend to some of the knotty facets of health professions education research by offering a set of virtues, or behaviors typically associated with high ethical standards, to complement the “7 deadly sins.” Virtues are influenced by contexts and philosophical traditions, and are often aspirational and less rigid than rules. Researchers must thoughtfully navigate competing virtues to ensure integrity in their work. They must also recognize that practices deemed a “sin” in one context or philosophical tradition may constitute a virtue in another context or tradition.3 For example, researchers trained in biomedical research with positivist traditions (eg, a single objective reality) will agree that robust research begins with a solid hypothesis, while researchers trained in medical anthropology might disagree, as a hypothesis defies core assumptions of naturalistic traditions (eg, the coexistence of multiple subjective realities).
Although our stance is primarily qualitative, the virtues we discuss are based on insights we have accumulated through years of experience engaging with a variety of qualitative and quantitative research initiatives. We acknowledge that the 6 virtues are grounded in personal research experiences and are far from exhaustive. The virtues are organized around phases of the research process in which each virtue might be particularly applicable. Although we present the virtues in a linear fashion, they tend to overlap and interconnect (Table).
Conception: Planning the Research
Virtue 1: Thoughtfully Convening a Research Team
Health professions education researchers often undertake research projects in collaborative teams.4 We may invite collaborators from our own or other departments, other disciplines, other institutions, or even other countries.5 For example, researchers desiring to describe group dynamics at the patient bedside may invite a business school colleague to contribute alternate theoretical framing, a qualitative researcher to conduct interviews, and a patient advocate to share his or her experiences working with patients. Research collaborations such as this can provide a research project with a diversity of approaches, ideas, and skill sets—plus extra hands to distribute the work.
We convene research teams for a variety of reasons. We select individuals who will contribute the necessary content knowledge, methodological expertise, alternate perspectives, and access to populations or data; these types of contributions help us answer our research questions and address project goals. We also recognize that teams form because members enjoy working together, which can be another critically important factor in a research project's success.
Virtue 2: Prudently Setting the Scope and Timeline of the Research Project
The scope of the research project generally includes determining the type of study to be undertaken (eg, descriptive study, experiment), perspective(s) to be included (eg, learners, patients), and data sources to be collected (eg, surveys, interviews). We assess what is already known about the topic of interest, what perspectives should be heard, and how these perspectives will be captured. And we align these elements with the research question(s) and, when necessary, use them to revise the question(s).
Setting the scope of a research project entails balancing what are often lofty project goals with practical concerns. We try to be realistic about what each team member, in addition to the principal investigator, can contribute to the research project based on knowledge, skills, personal interest, and, perhaps most importantly, time. To help manage the team's time effectively, we create a timeline that allocates time to different components of the research depending on study type. For example, in experimental or quasi-experimental designs, researchers may allocate a significant amount of time to designing the measures or experiment, whereas a researcher undertaking a qualitative study may dedicate much of his or her time to data collection and analysis. The timeline should allow for elements within the researchers' control (formulation of research question, data analysis, writing, Institutional Review Board submission) and should also be adjustable for elements outside the researchers' control (eg, delays in Institutional Review Board review, illness or life circumstances of team members, requests for revisions and resubmissions, deadlines for other projects).
We also consider the resources needed to move the research project forward and ensure that they are incorporated into project planning. These might include consultation on survey design, statistical analysis, recording equipment, transcription services, and software. It is important to take into consideration team member expectations, such as whether the member plans to commit sufficient time and effort to the project to warrant authorship and how he or she expects to contribute.
Execution: Conducting Research
Virtue 3: Carefully Considering All Data and Interpretations
Nearly every seminal text on research methods treats the research question and/or related hypotheses as the lynchpin of a good study. In addition to guiding the study design, scope, and timeline, the research question also helps to focus the research team while collecting and analyzing data. Yet, many times when we begin data collection and analysis, we discover something unexpected, discover something a bit outside the scope of the initial research question, or something about the methods selected does not go as planned. This presents a dilemma: Do we stay the course, or do we adjust?
Published versions of studies make the research process seem linear. In truth, rigorous research often involves complex decisions and nonlinear processes. We find it critical to maintain openness to alternative interpretations and to allow flexibility in the research question and process of inquiry. To be clear, we are advocating neither the “merciless torture of data”1(p485) nor sloppy research; rather, we are advocating a systematic approach to ensure rigorous exploration of unexpected findings without excessive rigidity.6
For example, suppose a research team designed a study to examine how residents formulate learning goals prior to clinic sessions. The team followed recommended procedures for survey development and used self-regulated learning theory as a guiding framework. However, when they launched the survey they discovered that 60% of the residents did not consistently formulate learning goals, even though they were “required” to do so. Several residents provided written comments explaining why. As a next step, the research team has to decide what to do. Should it adjust its research question to explore the barriers to formulating learning goals? Should it just focus on the responses from the 40% who did formulate learning goals? Should it try to intervene to encourage residents to write learning goals? To decide how to proceed, the research team needs to consider the purpose of its research (eg, to test and refine a generalizable theory, to evaluate an intervention, and/or to improve an intervention), examine its relationship and ethical responsibility to the study participants, and contemplate the strengths and limitations of its data with respect to the research question (ie, validity, power, sufficiency, etc).
Virtue 4: Mindfully Checking Assumptions
Everyone has assumptions. We must be aware of our assumptions and be vigilant about checking them, not only when collecting and analyzing data or writing up our methods, but throughout the whole process, including but not limited to: assembling the research team, choosing which literature to reference, clarifying the philosophical tradition, designing methods, and writing up the findings.7,8
Research in some contexts and philosophical traditions aims to minimize or avoid bias, defined as assumptions that can distort the results of a research study. Bias is usually due to conscious or unconscious acts by the researcher that yield faulty conclusions skewed toward the researcher's own assumptions.9 Some traditions purport that there is a single, objective truth or a right and wrong answer; thus, bias needs to be kept in check. In contrast, other traditions view research as a dynamic, meaning-making process situated in particular contexts and circumstances and sensitive to different perspectives. Correspondingly, replication of findings and proof of truth are unrealistic goals.10,11 Rather than dismissing these contextual, personal, and interpersonal contingencies as bias to be avoided, researchers in this tradition advocate transparency and reflexivity, considering how researchers' perspectives and processes influence decisions and interpretations throughout the study.12
A common thread among all research endeavors is the virtue of honesty about what we believe to be true versus what the data suggest. Identifying and acknowledging ambiguity in data and uncertainty in our interpretations provides transparency and enhances trustworthiness. Patton13 recommended researchers consider 3 types of reflexive questions to stay vigilant about assumptions and values that may influence data interpretation:
Self-reflexive: What do I know? How do I know what I know?
Reflexive about study participants: How do those studied know what they know?
Reflexive about audience: How do those who receive my findings make sense of them?13(p72)
Keeping careful track of decisions about all aspects of the research process along the way and reflecting on the assumptions underlying those decisions help researchers maintain a reflexive stance.12
Presentation: Writing Up the Research
Virtue 5: Cautiously Chunking Data to Tell a Story
Reporting on large qualitative or quantitative research projects, or projects that collect large amounts of data over time, can be overwhelming. Lingard and Watling14 helpfully explained that a manuscript needs to tell a compelling story, particularly in the introduction and discussion sections, while also providing an accurate description of the study in the methods and results sections. With page after page of rich qualitative data or multiple variables and relationships to analyze from survey data, a study can launch multiple conversations. We find it works well to choose 1 story and construct a paper around it. There may be additional stories in the data that can contribute substantially to the field through separate papers, if authors can clearly articulate how and why the stories are sufficiently distinct to warrant separate publications.15,16
In telling the story, we should consider whose voice resonates and be able to answer the following questions. Are the research participants the main characters? Or have the researchers' interpretations overshadowed the research participants' voice? While this matter of voice may not be particularly problematic for research with a strong positivist stance, it may be inconsistent for qualitative research that seeks to understand participants' point of view. We try to be clear about highlighting participants' voice (eg, using block quotes, italics, etc) and clearly portraying our interpretation of their voice as a researcher.
Virtue 6: Cohesively Telling a Story
Once we have decided what story we want to tell, we need to ensure it is coherent and consistent, and that the pieces of the manuscript fit well together. Sections of a manuscript are easy enough to follow based on journal guidelines, but writing each section so the content flows logically, stays tethered to the research purpose and questions, and uses consistent terminology requires diligence.
In the introduction, the literature review needs to clearly address the problem and frame the research project. Next, the research question needs to connect to the problem and the literature review.17 Typically, a conceptual framework will also be referenced in the introduction, although depending on the guiding tradition (eg, inductive or constructivist), this may be fairly general—with elaboration and refinement in the discussion. The methods section explains how guiding theories or conceptual frameworks informed study design, data collection instruments and protocols, and analysis. Readers should easily see how the choice of methods fits the research question and the guiding framework.
In the results section, the data presented should address the research question. In some studies, unanticipated findings may arise that go beyond the research question. While these are often some of the most interesting findings, it must be clear how the researchers identified these findings and why they are important to the story. The discussion speaks to the problem, the conceptual framework, and the research question(s) presented in the introduction, and connects findings to previous empirical and theoretical work. New findings should not be presented in the discussion. While this seems straightforward, we often find tangents (“this point is so important, even if it's not really what our study was about”) and/or additional results (“this is such an interesting finding, we have to include it somehow”) slipping into the discussion.
Finally, we check for uniform terminology throughout the manuscript. This is particularly important when multiple authors write different sections. While synonyms add flavor to prose, they often cause confusion when describing key components of the research study (eg, referring to subjects, participants, and trainees interchangeably).
We have presented 6 virtues for researchers to consider when conducting research projects. We resist the notion of a right or wrong way to do research. Instead, we hold ourselves to guiding principles such as those described in these virtues. When caught between conflicting recommendations, our response to the scholar at the beginning of this article, and to readers in general, might be: What resources do you bring to a research project and, if these are limited, what resources are available to you? Who can you collaborate with to investigate this question? How firmly grounded are you in your own beliefs about this topic (eg, following patients after discharge)? What is the scope of research you are prepared to take on?
Asking questions such as these and incorporating the 6 virtues into the research process helps us be mindful of and intentional about complex research decisions. We hope that making these virtues part of our regular practices can enhance the quality and impact of our educational research and potentially guard against the 7 deadly sins.
Bridget C. O'Brien, PhD, is Associate Professor, Department of Medicine, and Educational Researcher, Center for Faculty Educators, University of California, San Francisco; Dorene F. Balmer, PhD, is Associate Professor, Department of Pediatrics, University of Pennsylvania, and Director of Research on Pediatric Education, The Children's Hospital of Philadelphia; and Lauren A. Maggio, MS(LIS), PhD, is Associate Professor, Department of Medicine, Uniformed Services University of the Health Sciences.
The views expressed in this article are those of the authors and do not necessarily reflect the official policy or position of the Uniformed Services University of the Health Sciences, the US Department of Defense, or the US government.
All 3 authors contributed equally to this work.