Understanding the Differences That Differentiate: A Model for Deciding Which Literature Review to Conduct

In choosing a literature review, the first step is to clarify expectations about the kind of knowledge to be developed about a phenomenon and how rigorous, adequate, and credible research is defined for the research team. In other words, choosing a literature review starts by clarifying one’s epistemological orientation²⁵ : Is it an objectivist or subjectivist research orientation?

The objectivist orientation is familiar to many clinician educators because it is commonly employed in the natural sciences. It aims to generate unbiased knowledge of the phenomena being studied. Objectivist research seeks to create knowledge that is not influenced by an individual’s perceptions or understandings of the phenomenon under study.²⁶  Literature reviews that are aligned with an objectivist position seek to summarize the evidence available in the published literature in a value-free way, meaning that the synthesis is disconnected from the values of the scholar doing the review and those of the people being studied.

For example, if a program director were interested in understanding resident burnout as a phenomenon that can be objectively observed and recorded, they may seek out or conduct a systematic literature review of the peer-reviewed literature. This review could ask a narrow question, such as: Are skills-based programs effective for reducing resident burnout?²⁷  The process would include well-justified inclusion and exclusion criteria to ensure that a specific body of literature was reviewed. To determine these criteria, the scholar might ask, for example: Are all specialties included in the review? Are data from all countries? What type of data is be included in the review? The study would also assess the quality of the included studies so that only the best evidence was used to inform the conclusions. These considerations help to ensure that the findings generated from the review are objective—unbiased, verifiable, and generalizable. If any other researcher carried out the same systematic review, they should come to the same conclusions as those presented in the original review, demonstrating the replicability of the analysis and conclusions. This approach can be valuable when making decisions about narrowly defined, specific interventions and populations with precise outcomes.

The objectivist orientation is so pervasive in research that it is often considered the norm²⁸ ; however, it is only one way of thinking about knowledge and research. Many social sciences are founded in a subjectivist orientation that aims to understand the meaning that people make about an experience or phenomenon.²⁵  Subjectivist research takes individuals’ perceptions and understandings as the foundation of investigations; the perspectives of the populations studied and the scholars carrying out the investigation are recognized as important and valuable. Literature reviews that work in the subjectivist tradition seek to summarize the available evidence to offer an interpretation of the literature from a particular perspective. The insights generated from such reviews are intimately connected to the subjective perspectives of the researcher(s) who carried out the synthesis.

For example, if a program director interested in resident burnout wants to understand how the leading researchers in this field interpret the body of known knowledge to date, they would not turn to a literature review carried out in the objectivist tradition. Instead, they would turn to a review that sits in the subjectivist tradition, one of the narrative review types.^29,30  Here the expertise of the researcher(s) shapes the synthesis and interpretation of the literature.³¹  In the subjectivist orientation, the literature review is informed by the researcher’s previous knowledge, their experiences, and their subjective readings of the literature. The findings of this synthesis could not be replicated by another scholar, because the unique perceptions, experiences, and insights of the author shaped the review. The researcher’s synthesis might reflect how modern understandings of resident burnout have historically evolved (eg, via a state-of-the-art review), or how theories of intersectionality could offer important insights into the literature (eg, via a critical review). Their synthesis might focus on the qualitative research carried out to date and offer a meta-level interpretation of this data (eg, a meta-ethnographic review). Alternatively, they might want to include in the synthesis a wide variety of publications—newspaper reports, commentaries in journals, reports from graduate medical education (GME) accrediting bodies, and other non-evidence-based manuscripts—to map what kinds of information are presently shaping the discussions of resident burnout (eg, via a scoping review). The reviewer might decide to take all of the theories currently available about burnout from all disciplines and synthesize them into a new theory that addresses resident burnout (eg, via a theoretical integrative review). These reviews can offer insightful and generative interpretations of research on resident burnout, but none of them aim to generate one objective or true answer. The markers of rigor for these reviews are not replicability nor verifiability; instead, they use other markers such as listing the factors shaping their interpretations.^29,30  Subjectivist-oriented literature reviews are shaped by the researcher’s perspectives, which makes them uniquely informative.

Objectivist and subjectivist research can be conceptualized as sitting at 2 ends of a continuum. Between these 2 poles, different kinds of literature reviews can be placed as having foundations that variably rely on the premises of one pole or the other (see Figure). We plot the 8 kinds of literature reviews addressed in JGME’s literature review series³²  along this continuum to illustrate how these literature syntheses rely on different traditions. As the articles in this series explain, understanding which of these 8 reviews to use is contingent on whether the kind of knowledge sought is objective or subjective. If you are interested in generating objective knowledge through a narrowly focused exploration and assessment of existing evidence-based literature, then a systematic review^13,14  is most closely aligned with that goal. In contrast, if you are interested in having your subjective perceptions and understandings shape the interpretations and critique of a body of literature, you will select from one of the narrative review types.^29,30 

Five review types sit within the narrative category, each with a different purpose. First, scoping reviews generate a map of the broad body of literature that exists about a specific phenomenon or topic.^1,2  The scope of literature included in scoping reviews is often far-reaching, to include manuscripts in popular media, policy documents, and commentary publications. Second, state-of-the-art reviews offer historical overviews of how modern conceptualizations of a phenomenon or topic came to hold dominance.^7,8  These reviews present a 3-part synthesis of the literature: This is where we are now; this is how we got here; this is where we could go next. Meta-ethnographic reviews focus on synthesizing qualitative data by engaging in interpretation of the data from across manuscripts to generate new insights. Next, critical reviews seek to offer new ways of thinking about a phenomenon by drawing on the theories, findings, and/or literature from different disciplines or academic fields that bring new perspectives on the phenomenon.^5,6  The outcome of a critical review is a new way of thinking about the phenomenon that questions long-held assumptions and proposes new research directions. Finally, theoretical integrative reviews involve the analysis and synthesis of different theories that address the same phenomenon.^9,10 

Standing between the objectivist and subjectivist poles are realist reviews.^3,4  Realist reviews look across complex interventions that address a particular problem or goal. These reviews examine the contexts in which the interventions were carried out, the mechanisms at work in the contexts, and the outcomes of the intervention. Realist reviews harness subjectivist orientations because they recognize that each context is unique, and so the intervention will work differently in different contexts. However, a realist review also harnesses objectivist orientations because this review generates generalizable abstractions that describe what works, for whom, and in what contexts. The realist review may generate a theory or a new understanding of the causal mechanisms that can be applied across defined contexts.

Understanding the objectivist/subjectivist foundations of different review types and being savvy with types across this spectrum are important goals for GME scholars to embrace. GME research relies on many different methods because the issues affecting the field span tangible, quantifiable topics (eg, quantifying the association between the amount of rural training during family medicine residencies and subsequent rural work)³³  to more abstract and experiential topics (eg, exploring the effects of longitudinal coaching on relationships between fellows and faculty).³⁴  Thus, a broad range of research methodologies and methods are needed to study the diversity of topics. Unsurprisingly, qualitative research methods are increasingly populating our journals. In 1998, only 1.2% of original research manuscripts published in general medicine used qualitative approaches.³⁵  By 2007 that percentage was up to 4.1%.³⁵  By 2015, 34% of the original research publications in JGME, Academic Medicine, and Medical Education relied on qualitative methods and 5% used mixed methods.³⁶  Therefore, if we were to synthesize the literature on a GME topic now, it is likely that nearly 40% of research publications would be rejected during a systematic review process for not having suitable quantitative data. However, a review approach under the narrative review umbrella could synthesize the full breadth of available knowledge while aligning with the epistemological orientation of a wider range of studies. Therefore, to more fully encompass the breadth of modern GME research, we must be able to use synthesis approaches other than systematic reviews.

While some typologies have identified up to 25 or more different kinds of literature reviews,²⁴  we addressed just 8 in our model and in JGME’s literature review series. This decision might leave readers wondering why we did not engage in a more expansive project. Our response is that we selected review types that we considered to be particularly useful for synthesizing the medical education literature. For some of these review types, long traditions of methodological descriptions, how-to guides, and best practices exist (eg, systematic reviews, realist reviews, and scoping reviews). However, for the overall category of narrative reviews, as well as several of the subtypes within it, there was little literature to draw on—or even none (eg, critical reviews, state-of-the-art reviews, and theoretical integrative reviews). For these latter types, the series’ authors engaged in a kind of reverse-engineering process, to understand the underpinning traditions, purposes, methods, markers of rigor, and outcomes of these reviews. This methodological mapping work will need to be done for other kinds of literature reviews in the context of medical education for them to be positioned into our model.

JGME’s literature review series seeks to disambiguate popular and lesser-known types of reviews, while aiming to dispel unhelpful myths about the possible value and rigor of the methodologies employed across literature syntheses for GME. We have made a start. There is more work to do in enhancing understanding of the many valuable literature synthesis approaches in health professions education.