Artificial Intelligence in Graduate Medical Education Applications

I am not a robot, and I can prove it…just by clicking on all the picture tiles with bicycles or buses. These CAPTCHA tests, or Completely Automated Public Turing tests to tell Computers and Humans Apart, protect websites from bots and spam. Will residency program directors soon need their own form of CAPTCHA to protect the integrity of the residency application process from artificial intelligence (AI)? The 2023-2024 residency application season is the first in which generative AI has been a popularly recognized and easily accessed tool. While its use has advantages, it also threatens the sincerity and reliability of important data points in application evaluation.

In November 2022, the San Francisco-based tech company OpenAI released ChatGPT—a free online chat bot capable of writing prose that is nearly indistinguishable from human-generated text. It is now one of several such services available online. Generative Pre-trained Transformer (GPT) describes large language model artificial neural networks that are “trained” on large data sets and can perform natural language processing tasks such as generating new text and analyzing text for specified content.¹  After the March 2023 update, OpenAI announced on its website:

The use of AI was quickly adopted by university students. An informal survey of nearly 5000 Stanford students found that 17% used ChatGPT in fall 2022 for assignments and examinations (the term in which ChatGPT became publicly available).³  Over 25% felt that AI use, even if answers are unedited, was not a violation of honor code.³  While there are no data on AI use by health professional students, one must assume parallels with the general university population.

Because AI use is widespread at universities, many applicants will consider using it when writing a personal statement (PS). The PS has long been the applicant’s opportunity to narrate their journey through medical school and to their specialty of choice, and to highlight their unique qualities and motivations.⁴  Using AI to assist in writing them can improve efficiency, help with brainstorming, and improve communication for people for whom English is a second language.⁵  It also improves equity in access to application assistance services, which can cost in the thousands of dollars.⁶ 

On the other hand, multiple online forums advise against using AI for the PS because it seems “inauthentic and unoriginal—and therefore not as good as” an applicant’s own work.⁷  An AI-written PS lacks the narrative voice, context, and specific details that make it personal.⁸  Another notable drawback of AI-generated work is its propensity for lengthy and potentially excessive text, a factor that poses a barrier to reader efficiency. No data yet exist on the applicant’s perspective on the use of AI in PS generation.

Through letters of recommendation (LORs), faculty advocate for students and highlight their longitudinal relationships.⁹  Similarly to the PS, AI can assist in generating an LOR, which can improve writing efficiency and reduce the use of biased language,¹⁰  with which LORs have long been fraught.¹¹  There are no studies of how frequently AI is used to generate a professional school LOR. An article in The Atlantic, “The End of Recommendation Letters: Professors, Like Their Students, Use ChatGPT to Get Out of Doing Their Assignments,” reveals one popular sentiment against the use of AI in letter writing.¹²  Because AI-generated letters usually lack the personal connection, emotional intelligence, nuances, and subjective judgement of a human-written letter,¹³  heavy editing is advisable if AI is used for the first draft. If AI is used in text generation, authorship is not attributed to the AI; at least this is the stance for scientific journals.¹⁴  Ultimately, the signer of an LOR takes responsibility for its content, which has always been the case.

Program directors may also look for assistance from AI. Application inflation necessitates improved efficiency in application review.^15,16  AI could streamline application screening by looking for specific combinations of content. AI could mitigate human bias, for example, by identifying biased language in an LOR,¹⁷  but could also propagate it. AI must be “trained” on the attractive features of an application, and if bias exists in the training data, that bias can be codified. When Amazon piloted the use of AI in application screening, the training data set included a majority of male employees, which unintentionally led to weighting “non-maleness” as a deficit in the application. Amazon discontinued use of this faulty paradigm once the bias was discovered.¹⁸ 

While AI offers advantages and threats to the application process, identifying its use is challenging. The majority of program directors reviewing the PS found AI-generated ones acceptable and did not suspect that they were not human-written.⁶  Faculty with an MD or PhD could also not consistently determine which medical research abstracts were written by humans vs AI.¹⁹  Software for AI detection is publicly available and often free, but would require additional time in the already compressed application review period, as each piece of prose would need to be uploaded or pasted into the detection site. Additionally, AI-detecting software may be inaccurate, particularly when evaluating text from a writer for whom English is a second language.²⁰  The Mozilla Foundation, whose mission is to maintain open and publicly accessible internet services, warns: “Detector tools will always be imperfect, which makes them nearly useless…One cannot accuse a student of using AI…based on the output of a detector tool that you know has a 10% chance of giving a false positive.”²¹  At least for now, it seems impossible to know with certainty if the content of prose is the product of the author and contains accurate information. However, this has always been the case. There have always been threats to authenticity and concern over the ethics of using such potentially flawed datapoints in admissions.²²  AI only exacerbates them.

Currently the only guideline from the Association of American Medical Colleges on the use of AI in residency applications states: “[the PS] must be your own work and not the work of another author or the product of artificial intelligence.”²³  Applicants submitting applications to medical school through American Medical College Application Service are required to attest that “all written passages…are my own and have not been written, in part or in whole, by another author and are not the product of artificial intelligence.”²⁴  However, such an attestation is not required for residency application. These policies do not reflect the complexities of AI use. For example, is it allowable to brainstorm using AI and then edit the text heavily? Can AI be used to edit one’s own work to improve grammar and readability?

Until there are more nuanced and generally agreed-upon guidelines at a national level on the use of AI in application preparation, graduate medical education (GME) programs should consider posting explicit policies on their websites regarding the use of AI in preparing application materials.²⁵  AI could even aid in the generation of such a policy (Box 1). However, author-generated guidelines are clearer and more concise, which highlights the difference in word efficiency between AI and human text (Box 2). Any limitations should be stated with a purpose behind them, although such policies are presently unenforceable. There have always been concerns over the use of the PS and the LOR in applicant evaluation; AI only exacerbates these and may ultimately lead to a decline in the importance of these data points in the application package. The national GME community needs to revisit the methods of holistic application review in light of the challenges posed by AI.