“Past and Present” features excerpts from an article that appeared previously in the pages of the Journal or its predecessor, the Federation Bulletin, coupling them with an accompanying contemporary essay. This edition of “Past and Present” draws upon the April 1917 edition of the Federation Bulletin. Dr. Hannah Hurd, a member of the Minnesota State Board of Medical Examiners, shared that state board's experience with a “practical” exam as part of the medical licensing examination.
“Practical examinations were introduced by the Minnesota State Board of Medical Examiners about eight years ago. The effect was so satisfactory that we have been stimulated to elaborate and improve them from time to time in so far as has been compatible with our means and opportunity…My purpose today is to tell you not only the kind of examination we are now giving but how they could be much improved…The applicant is given one hospital patient with the greatest number of physical findings possible. About one and a half hours is allowed for his examination of the patient. The laboratory tests…are shown if asked for, and a written record of the case is required. Following this, an oral examination is given covering not only the case in hand but similar conditions, questioning closely on the points of differential diagnosis…
The chief change our board would like to make is a closer personal supervision during the period of the bedside diagnosis…it is probable that two cases would be a fairer test, had we the time, means and opportunity which at present are not granted to us…
The examination of next importance is that of the clinical laboratory. At present it consists of a very simple urinalysis… It might be well to add other clinical laboratory procedures…for instance, a differential blood count…Only one other examination is left to be mentioned, namely, histology, pathology and bacteriology. I see the chance for much improvement over [our] present methods, where are: The first hour is for an examination of four microscopic slides, two being histological and two pathological and bacteriological. The last hour is for the examination of six gross pathological specimens…
If you will consider this carefully I believe it will appeal to you as being truly practical; nearer the ideal to which we should aspire — that of making the examinations as nearly similar as possible to the real test of actual practice…”
A human tendency in reflecting upon the past is to view those time periods preceding ours as somehow less sophisticated or lacking the complexity we perceive as uniquely characteristic of our own era. Reading Dr. Hannah Hurd's description of the Minnesota practical exam provides a strong corrective against the hubris of the present. Minnesota's efforts at a practical examination for licensure beginning in 1908 were remarkable both then and now.
At that time, medical licensing examinations were commonly multi-day cognitive assessments, focusing heavily on the body of medical knowledge possessed by the licensure applicant—an important skill set in an era lacking on-demand access to medical libraries, resources and databases. These were usually written examinations, though sometimes they were administered with an oral or interview component as well. The Minnesota board's examination featured a written examination spanning two days.
Open-end or essay questions represented the predominant testing format for most state medical boards. Typically, candidates for licensure faced a battery of exams in multiple subject areas, usually 10 questions in each area. Minnesota's written exam covered three non-compensatory major topics (medicine, surgery and OB/GYN) and several minor topics (anatomy and physiology; therapeutics; materia medica; diseases of the eye, ears, nose, throat; medical jurisprudence).
Dr. Hurd and her colleagues on the Minnesota board understood that any written examination served as an imperfect proxy for observing physicians interacting with real patients. While they might assess medical knowledge as the foundation for a physician's preparedness to deliver safe and effective care, the written exam could only simulate the application of this knowledge into hands-on patient care. Thus, in addition to assessing cognitive knowledge, the Minnesota board aspired to “making the examinations as nearly similar as possible to the real test of actual practice.”
The board's introduction of its “practical” exam in 1908–1909 represented an ambitious undertaking. Minnesota's practical exam was among the first in the country—at that time, only Massachusetts and Ohio administered this format as well. Minnesota's practical exam involved real-world physician tasks directly related to patient care: taking a bedside history; conducting a physical examination; rendering a diagnosis. All of these activities then served as the basis for an interview. Within that setting the interviewing board member(s) might explore the clinical reasoning and judgment of the individual examinee.
The ambitiousness of this undertaking is all the more remarkable considering the first administrations of Minnesota's practical examination pre-dated by nearly a decade the first efforts of the National Board of Medical Examiners (NBME) to assess clinical skills in 1915. It would be another century before full-scale efforts at clinical skills assessment figured prominently in a nationally administered examination for medical licensure—specifically, the United States Medical Licensing Examination and the Comprehensive Osteopathic Medical Licensing Examination in 2004.
In relating the parameters for Minnesota's practical examination, Hurd shared several observations for improving the exam. Her critique reflected an intuitive concern for what today we would identify as examination validity, i.e., the appropriateness of the decisions being made based upon the assessment. While the technical language and concepts of assessment (validity, reliability) were not commonly a part of licensing discourse in 1917, Hurd signaled an awareness of the concerns underlying these concepts. Notice that she stated, “two cases would be a fairer test.” This brief observation touched on a two-fold concern. Hurd appears to acknowledge that an assessment informed by a single patient encounter represented a hit-or-miss affair for the examinee. The examinee might benefit from (or be disadvantaged by) their familiarity with the clinical condition presented by the patient. Only a second encounter (or a third or a fourth) could mitigate this randomness.
Second, Hurd's brief but telling comment signals an understanding that the unique aspects of the patient case presented to a single examinee determined the overall parameters and assessment focus for that encounter. In other words, while the board might attempt to standardize the experience for examinees by providing comparable patient cases, it was impossible to do so using a single case for each examinee.
Additionally, Hurd acknowledged the desirability for a “closer supervision” of examinee interactions with the patient. In her reference to time and resource constraints, she echoed a concern raised by contemporaries elsewhere. While 17 state boards utilized a practical component to their licensing examination in 1920, this may well have been the high-water mark for the period. The logistical issues identified by Hurd ultimately plagued all state board attempts at practical examinations. Thus, even as the science of assessment evolved rapidly after World War I, state medical boards began quietly scaling back from these assessments.
This excerpt from a century ago reminds us that assessment has served as a critical element supporting the licensing decisions rendered by state medical boards over the past 150 years. The ongoing collaboration of medical regulators and assessment experts should continue as the medical licensing examination carries on its role as an independent, external audit of the medical education and training systems of this country.
About the Author
David Alan Johnson, MA, is Senior Vice President, Federation of State Medical Boards. He is the author of Diploma Mill: The Rise and Fall of Dr. John Buchanan and the Eclectic Medical College of Pennsylvania (Kent State University Press, 2018).