A Lesson From the Qualitative Rip Out Series: Let Go of Expectations for Universally Applicable “Gold Standards” for Qualitative Research

Historically, there is no denying that medical education research and scholarship has been predominantly a quantitative field. Given that tradition, it is not surprising that randomized controlled trials (RCTs) have long been considered the field's “gold standard” for research, and that objectivity and replicability are hailed as hallmarks of research rigor.

But as the field grows, tackles new problems, and explores new areas of inquiry, researchers have increasingly needed to use different kinds of research approaches to answer their questions. Scholars have had to look beyond their quantitative comfort zones to see how social scientists use qualitative research approaches to develop new knowledge.^1,2  In 1998, only 1.2% of original studies published in journals of general medicine used qualitative approaches.³  By 2002, that percentage had risen to 2.5%, and by 2007 it was up to 4.1%.³  This is an impressive 2.9% increase in the short span of just 10 years—translating to a 3.4-fold relative increase.³  A recent scoping review⁴  of medical education research in family medicine, which looked at studies between 2002 and 2012, found that 15% of published original studies used qualitative research approaches, with the annual number of publications using qualitative methods increasing from 8 manuscripts in 2002 to 17 in 2011.

Curious about the representation of qualitative work in the most recent medical education publications, we read the abstracts of original research articles published in 2015 in the Journal of Graduate Medical Education (JGME), Academic Medicine, and Medical Education, and compiled some descriptive statistics. We excluded any articles that did not report original research and so did not include short manuscripts (such as Viewpoints, Last Pages, Commentaries, Brief Reports, etc) or manuscripts that focused on describing educational innovations (such as Innovations, Really Good Stuff, etc). Across these 3 journals, 200 original research papers were published, of which 67 (34%) used qualitative methods and 10 (5%) used mixed methods. While quantitative research remains the norm (n = 123, 62%), qualitative inquiry has clearly established itself as a valid form of inquiry worthy of dissemination in medical education journals.

In reading these abstracts we noted that qualitative researchers are using a wide array of methodologies. As can be seen in the table, among the qualitative and mixed methods reports, grounded theory is the most commonly used methodology (n = 15, 19%). However, phenomenology (n = 4), discourse analysis (n = 3), ethnography (n = 1), and other methodologies are also reported. We are excited by this diversity since it suggests that, within the field of medical education research, researchers are becoming increasingly aware of the range of available qualitative methodologies and the unique insights each methodology can offer them. In contrast, however, we noted that 42 publications did not report the methodology that framed the qualitative or mixed methods studies. We predict that journal editors and reviewers will soon expect authors to report the methodology underlying qualitative studies. As Tehrani et al^5(p669) assert in their JGME Rip Out, alignment between a study's methodology and research question “is a prerequisite for rigorous qualitative research.”

Similarly, we noted a variety of qualitative data collection methods being used in these publications. While interviews (n = 48, 62%) and focus groups (n = 20, 26%) are clearly preferred, we were pleased to see that texts were regularly used as data sources (n = 18, 23%), and both video/audio data (n = 2, 3%) and observations (n = 1, 1%) had been reported as well. This diversity of data is a welcomed advance, since an assortment of data sources can enable medical education researchers to investigate different problems in multifaceted ways.

This expanded use of qualitative methodologies and data collection methods has been fruitful. Among the bevy of contributions made to medical education over the years, qualitative studies have helped researchers understand the phenomenon of “slowing down when you should,”⁶  the moral distress experienced by residents in palliative care contexts,⁷  and the lessons learned by residents via informal education in clinical contexts.⁸  Making these discoveries demanded that researchers develop and refine their qualitative skills.

The Journal of Graduate Medical Education is committed to helping its readership participate in this expansion into qualitative research approaches. Over the past 7 issues, JGME has published a series of Qualitative Rip Outs that answer some of graduate medicine's most commonly asked questions about qualitative research—from understanding how qualitative research differs from quantitative research⁹  to selecting a methodological orientation⁵ ; from selecting a data collection method¹⁰  to analyzing existing evaluation data¹¹ ; and from the particular ethical concerns of engaging in qualitative research¹²  to incorporating theory into a qualitative study.¹³  Collectively, the topics that JGME's Qualitative Rip Outs address are highly relevant to graduate medical education scholars.

In addition to answering common qualitative research questions, these Rip Outs have also offered lists of suggested readings that can help scholars develop deep understanding of the intricacies of qualitative research. These additional readings are very important because, just as scientists who specialize in RCT research need years to hone their research methods skills, medical education scholars interested in qualitative research must also invest considerable time honing their qualitative skill set. We hope these Qualitative Rip Outs will help JGME readers begin that honing work—but they are just a beginning. To engage in rigorous qualitative research, medical educators will need to read up on the methodologies, methods, theories, and epistemologies of qualitative inquiry. They are well-advised to find collaborators skilled in qualitative research to assist them in conducting their investigations.

Medical educators who take on the challenge of learning qualitative approaches must remember that there are few, if any, universally applicable “gold standards” for qualitative research. The processes and structures of RCTs don't transfer to the vast majority of qualitative studies. The criteria of objectivity and replicability generally don't apply to qualitative inquiry.

As these Qualitative Rip Outs demonstrate, engagement in qualitative research is rarely buttressed with simple “do this” instruction. Instead, the more common advice is “it depends.” While this reply is admittedly frustrating, that variability is the very power of qualitative research. It is part of the new way of thinking about, conducting, and evaluating research that medical educators will need to embrace to engage in rigorous qualitative research.