ABSTRACT
New Jersey's COVID-19 Temporary Emergency Reciprocity Licensure Program provided temporary licenses to more than 31000 out-of-state healthcare practitioners, over a quarter of whom were mental health providers. As the need for mental health care accelerated during the pandemic, especially among health disparity populations, expanding mental health provider pools may be a critical tool to increase access to care. In January 2021, we surveyed New Jersey's temporary licensees. We analyzed over 4500 mental health provider responses to examine the impact of the temporary licensure program on access to mental health care overall and on enhancing a diverse mental health workforce. Over 3700 respondents used their temporary license to provide mental health care to New Jersey patients. About 7% of respondents self-identified as Hispanic, 12% Black, 6% Asian, 1% American Indian or Alaska Native, and 0% (more than 5) Native Hawaiian or other Pacific Islander. They treated about 30100 New Jersey patients, 40% of whom were new to the provider, and 81% delivered care exclusively using telehealth. Respondents conversed with patients in at least 13 languages. About 53% served at least one patient from an underserved racial/ethnic minority group. Our findings suggest that temporary out-of-state mental health providers helped enhance mental health care continuity and access.
Introduction
States govern and oversee healthcare provider licensing and regulation, but, in most instances, states do not offer licensure reciprocity for interstate professional practice.1,2 Public health emergencies can cause localized surges in demand for healthcare services, triggering a need for an increased healthcare provider supply. A key strategy to help meet these demands is temporary modifications of professional licensure laws, which can allow for the expansion of provider pools, methods of care delivery, and scopes of practice.3 During the COVID-19 pandemic, at least 45 US states modified their healthcare provider licensure laws through public health emergency legislation and regulatory mechanisms.4,5
On March 9, 2020, as the State of New Jersey became one of the first COVID-19 hotspots in the US, New Jersey Governor Philip D. Murphy signed Executive Order 103, authorizing the state Division of Consumer Affairs (DCA), which houses the professional licensing boards, to “waive, suspend or modify any existing rule, where the enforcement of the rule would be detrimental to the public welfare during the emergency.”8 Via a series of regulatory actions, including administrative orders and waivers pursuant to the authority granted by Executive Order 103, the Division of Consumer Affairs enacted a Temporary Emergency Reciprocity Licensure program on March 20, 2020. This program allowed out-of-state, licensed healthcare providers to obtain a temporary license to provide care to New Jersey residents via telehealth6 or in-person similar to a plenary license.7–11 Although policy decisions regarding licensure reciprocity and professional practice boundaries are historically politically-fraught, the pandemic-related provider supply crisis yielded swift emergency policy response by most states. The goal of these programs was to increase the healthcare workforce supply to meet pandemic-related surges in demand for both COVID-19 and non-COVID-19 care. As part of New Jersey's program, licensing fees and criminal background checks were waived for individuals holding current, out-of-state licenses in good standing. Twenty-three categories of providers were eligible to participate in the program, including physicians, nurses, and mental health providers.9 As of January 2021, the DCA had issued over 30000 temporary licenses to out-of-state healthcare providers. Simultaneous to the establishment of the temporary licensure program, New Jersey and the federal government introduced multiple telehealth expansion efforts,6,12 offering broader modalities for care delivery during the COVID-19 emergency.
Expanding provider pools and telehealth offerings is critical, particularly for mental health care, as the already urgent need for mental health providers accelerated during the pandemic. As of 2015, mental health and substance use disorders together surpassed cancer and cardiovascular disease as the leading cause of disease burden in the US.13 The US has experienced mental health workforce shortages for years,14 especially in populations vulnerable to health disparities, which includes rural populations15 as well as communities from racial and ethnic minority backgrounds.16 The pandemic exacerbated care gaps and disparities attributable to workforce shortages, as studies show that many mental health providers left their jobs for work outside of healthcare or work with higher pay during this time period.17 Attrition may also be increasing due to workforce burnout,18 as psychologists report having a higher number of patients and increased workload in 2021 compared to 2020, with the burden particularly acute among psychologists who are younger, female, and racial/ethnic minorities.19 Many psychologists have also been working at or above capacity, with nearly half experiencing burnout and 41% unable to meet treatment demand.19 Innovative approaches, including interstate licensure reciprocity, are needed to increase the mental health workforce to mitigate growing gaps in mental health demand and supply, which disproportionately burden health disparity populations.
Emergency licensure reciprocity programs implemented during the pandemic thus offer a unique opportunity to examine the impact of more flexible and portable licensure on health services. New Jersey's COVID-19 Temporary Emergency Reciprocity Licensure program is well-positioned for study, as New Jersey was one of the initial COVID-19 hotspots in the US and is one of the most diverse states in the country.20,21 The goal of this study was to investigate the impact of emergency licensure reciprocity on access to mental health services overall and access to a diverse mental health workforce in particular. This study expands upon our prior work which analyzed the survey data across six broad health practitioner categories.22 In the previous study, we found that mental health providers were the highest overall utilizers of the New Jersey temporary license. Compared to the other practitioner groups, mental health providers were the most likely to serve existing patients, most likely to use telehealth to deliver care to patients, and least likely to converse with patients in a language other than English. These findings motivated the present analysis to more closely examine trends and variation among the different types of mental health providers who used a temporary license, particularly to serve New Jersey's diverse population.23 This is the first study to examine the impact of emergency licensure reciprocity on a state's mental health provider supply during the COVID-19 emergency. Findings may inform future licensure policies that aim to enhance mental health care continuity and increase access to diverse provider pools.
Methods
We conducted a cross-sectional survey of New Jersey temporary licensees. The survey instrument was developed by officials from the New Jersey DCA and researchers from Rutgers University and drew upon validated survey questions and theoretical frameworks24 to capture the impact of the program on healthcare workforce supply. See Appendix A for the survey questions. The DCA fielded the survey January 7–21, 2021 by emailing a survey link that allowed for anonymous reply and 2 reminder emails to all providers (N=31805) who obtained a New Jersey temporary license between March 20, 2020 and January 6, 2021.25 There were 10449 total survey respondents (33% response rate, which is similar to prior provider surveys in New Jersey26 and nationally27 )—4689 of whom were mental health providers and are the focus of this study. Additional results from the survey are reported elsewhere.22
We created 5 mental health provider respondent sub-groupings: (1) psychologists; (2) social workers; (3) professional counselors; (4) mental health prescribers, which includes psychiatrists, nurse practitioners (including Advanced Practice Nurses), and physician assistants who indicated a psychiatry specialty; and (5) other mental health providers, which includes alcohol and drug counselors, marriage and family therapists, and nurses specializing in psychiatry.
We performed descriptive analyses to identify the reach and impact of the mental health providers who used the temporary license. Analyses were conducted using R statistical software.28 Counts less than 10 were denoted as “less than 10” to ensure respondent anonymity, per recommended best practice.29 The study was approved by the Rutgers University Institutional Review Board.
Results
Among the 4689 total mental health provider respondents, 3726 (79.5%) reported that they used their temporary license to provide mental health services to at least one New Jersey-based patient. These 3726 respondents are hereinafter referred to as “license-using respondents” or the “license-using group.” The remaining 963 mental health provider respondents obtained but had not, at the time of the survey, used their temporary license. Among the 518 who provided a reason for not using the temporary license, the top reasons were: did not receive treatment requests (eg, no referrals for new patients, existing patients who relocated did not seek care; 54.6%) and no job placement (unable to find employment; 13.7%). The license-using group is the focus of this study. For a breakdown of the provider types who responded to the survey, see Appendix B. The license-using respondents held primary licenses from every state in the US except Mississippi. Most were licensed in New York (1409; 37.8%), Pennsylvania (932; 25.0%), California (173; 4.6%), Massachusetts (138; 3.7%), and Maryland (120; 3.2%). These five states accounted for almost 75% of all the license-using respondents.
Table 1 describes the self-identified demographics of the license-using respondents. The majority were non-Hispanic, white women between the ages of 18–59. However, there were many respondents from underrepresented racial/ethnic groups. About 7% of respondents identified as Hispanic, 12% Black, 6% Asian, 1% American Indian or Alaska Native, and 0% (more than 5) Native Hawaiian or other Pacific Islander. About 75% of Black respondents were either social workers or professional counselors. About 45% of Asian respondents were psychologists.
Table 2 shows the number of patients treated by the license-using respondents during the study period. In total, license-using respondents reported treating an estimated 30096 New Jersey patients. Respondents served a mixture of new and existing patients, with about 40% of patients new to the provider and 60% existing patients. Psychologists, social workers, and providers in the “other mental health providers” category treated more existing patients than new patients compared to professional counselors and mental health prescribers who treated similar percentages of both.
Table 3 shows telehealth use by provider type, including the share of patient care time using telehealth and the share of the telehealth time by modality (ie, audio vs. video). All licensing-using respondents spent most of their New Jersey patient care time using telehealth; in fact, 80.9% of respondents spent all of their patient care time using telehealth. The group that reported the lowest share of time using telehealth was “other mental health providers;” 21.3% of which spent none of their patient care time using telehealth. Across all 5 groups, respondents were more likely to use video-based telehealth than audio-only telehealth. Respondents in the “other mental health providers” category (which includes nurses) were slightly more likely to use audio-based telehealth compared to the other respondent groups.
Table 4 shows the languages that respondents used to converse with their New Jersey patients. Most spoke only English with their patients. However, respondents also reported conversing with their patients in Spanish (165 respondents), Arabic, Chinese, Haitian Creole, Hindi, Italian, Korean, Gujarati, Polish, Portuguese, Russian, and Tagalog.
Table 5 shows characteristics of patients served by license-using respondents, which highlights ways in which respondents provided care to health disparity populations (eg, racial and ethnic minorities and those with less privileged socioeconomic status).30 While most respondents indicated that their patients had insurance, some also served New Jersey's Medicaid and uninsured population—331 (7.1%) reported serving at least 1 Medicaid or uninsured patient. Forty-four percent of these respondents (147 of the 331) indicated that more than 25% of their patients were Medicaid enrollees or uninsured. About two-thirds of the license-using respondents (n=2559) reported serving at least 1 patient with a chronic or severe mental health diagnosis or substance use disorder—1655 of which had more than 25% of their patients in this category. About 53% (n=1975) reported serving at least 1 patient from an underserved racial or ethnic minority population—1011 of which had more than 25% of their patients in this category. About 18% (n=683) reported serving at least 1 patient with a physical disability—201 of which had more than 25% of their patients in this category.
Discussion
Our survey results revealed that at least 3700 mental health providers who held primary licenses in other states used their temporary New Jersey license to provide mental health services to over 30000 New Jersey-based patients during the study period (March 2020–January 2021). The program enabled patients with already-established care to maintain care continuity and patients seeking new care to have increased access to mental health services. With increasing deaths of despair (ie, deaths attributable to drugs, alcohol, and suicide)31,32 and a growing shortage of mental health providers across the US,14,17 especially from health disparity backgrounds,33 increased access to mental health services for even 1 individual, let alone 30000, has the potential to save lives.34 The expansion of interstate reciprocity licensure represents a significant public health emergency response initiative with potentially far-reaching impact for mental health care accessibility.
Importantly, our survey results revealed that the New Jersey Temporary Emergency Reciprocity Licensure Program enabled at least 689 mental health providers from underrepresented racial and ethnic groups to provide care in New Jersey in at least 13 different languages. New Jersey has a dense and diverse population, with approximately 21.5% of its population identifying as Hispanic/Latino, 15.3% Black or African American, 10.3% Asian, 0.7% American Indian or Alaska Native, and 0.1% Native Hawaiian or other Pacific Islander.23 The influx of non-white providers may have increased the supply of mental health providers who were linguistically, racially, or ethnically congruent to New Jersey patients. Our findings also show that 331 of those mental health providers served at least 1 patient who was uninsured or covered by Medicaid, and 2559 were treating patients diagnosed with serious mental illness or substance use disorders. Given the known disparities in access to mental health services for patients who are uninsured or have Medicaid coverage, as well as the shortage in mental health providers who work with patients with serious mental illness,35 increasing the number of providers who work with these patient populations is key for improving access to mental health care for all Americans, not just the wealthy, healthy, and insured.
Due to the lack of diversity among healthcare providers in the US in general, relative to non-Hispanic whites, patients from racial and ethnic minorities have lower rates of racial/ethnic concordance with their healthcare providers.36–38 People often appear to prefer receiving treatment from a healthcare provider matching their own racial or ethnic background, especially groups in the early stages of acculturation and language acquisition.39–42 However, finding providers that match patient preferences can be a barrier to receiving health care.41 In race-, ethnicity-, or language-discordant patient-provider consultations, there are lower levels of patient satisfaction and higher rates of miscommunication.38,43 In mental health intake visits, concordant patient-provider dyads engage in more patient-centered communication and have better continuance rates.44 While there have been many attempts to provide cultural competency training to providers to overcome discordance between providers and patients, evidence is limited in the effectiveness of the trainings.45 Our study suggests that introducing flexibility into professional licensure through emergency reciprocity licensure expanded the availability of providers from health disparity backgrounds in New Jersey, a state with large numbers of residents from health disparity backgrounds—potentially improving health equity.
While telehealth has great potential to expand the reach and availability of mental health providers, the limitations of telehealth, especially as they pertain to health equity, are also important to note. Our study revealed that out-of-state mental health providers mainly used telehealth, particularly video-based modalities. Even prior to the pandemic, mental health providers who used telehealth were increasingly relying on video-based modalities.46 Historically, however, health disparity populations have limited digital access and digital literacy,47–49 and during the pandemic, socially vulnerable communities were more likely to use audio-only telehealth.50 Mental health providers, including those working across state lines, should be aware of how telehealth modalities may mitigate or perpetuate disparities in relation to “digital divides,” and policymakers should consider how telehealthrelated regulatory approaches and reimbursement changes51,52 may effect health equity.
Additionally, 60% of the mental health providers in our study indicated that they utilized the temporary licensure program to see existing patients who are now located in New Jersey (eg, the patient moved to New Jersey during the pandemic and sought care via telehealth), thus allowing patients to maintain continuity of care. It should be noted that federal and New Jersey telehealth waivers6,12 permitted out-of-state providers to treat existing patients via telehealth without a temporary license; however, our findings suggest that some of our survey respondents may have obtained a New Jersey temporary license as a precaution to ensure they could maintain care continuity, even though they did not need one.
Continuity of care in mental health is a critical aspect of successful care delivery and health improvement.53,54 There are a number of reasons why patients may temporarily relocate, for example, college students going home due to COVID-19 closures or summer break,55 seasonal workers performing time-limited work,56 and older adults migrating to warmer climates for the winter.57 Patients feel more supported and have better health outcomes when they can meet with their mental health providers consistently and in a timely manner.58 COVID-19 federal and state telehealth waivers6,12 relaxed originating site requirements which led to increased adoption of telehealth during the pandemic and helped to facilitate continuity of care.59 Our findings suggest that by offering mental health providers a pathway to temporary reciprocity licensure in tandem with telehealth waivers, providers were able to follow patients across state lines via telehealth, thus ensuring continuity of care and best practices in mental health treatment.
Several pre-pandemic licensure mechanisms designed to facilitate flexibility and portability may be informed by lessons from the COVID-19 temporary licensure programs.60 One mechanism that has been widely-discussed by researchers during the pandemic is the interstate licensure compact.61–63 Interstate licensure compacts predate the pandemic and are different from reciprocity-based licensure. Interstate licensure compacts are typically designed to allow healthcare providers to have primary or home state licensure plus either a multistate license or an option for expedited additional licenses in order to practice in their home state as well as other participating compact states. For example, as of February 2023, the Psychology Interjurisdictional Compact (PSYPACT), which serves as an interstate compact for psychologists, has been enacted in 35 US states and territories (including Washington, DC) and fully implemented in 33 (including New Jersey), which serves as an interstate compact for psychologists.64 Twenty-three of those states/territories enacted the PSYPACT during the pandemic (ie, after March 2020). Other non-psychologist mental health providers are working on similar programs, including professional counselors who are in process of establishing the Counseling Compact, with 15 states already enacting it into law.65 The National Association for Social Workers has initiated the development of the Social Work Licensure Compact.66 From an access-to-care perspective, the utility of such interstate compacts will likely depend on how they—and related reimbursement policies—impact, facilitate, or hinder telehealth-based practice.
It is worth noting the potential pitfalls of licensure reciprocity. Ultimately, licensure reciprocity is a “zero-sum game” for workforce supply—providers shift into areas of need or demand, while the total volume of providers does not change. Evaluation and optimization of interstate licensure does not satisfy pressing needs or approaches to increase the overall number of providers. In addition, there is limited research on the effects of licensure reciprocity or the relative value and drawbacks of compact-based licensure structures. State-specific licensing allows significant variability across regulations, and further research is encouraged to establish the extent to which variation or standardization impact key metrics. A qualitative study of licensure stakeholders reported that areas meriting further study include: care quality delivered by out-of-state providers; impact on care costs (related to relocation packages); and impact of increased market competition on smaller provider organizations.67 At the same time, evaluation of telehealth-based mental health care continues. Additional visibility into modality-specific quality, access, and affordability considerations will likely prove valuable. We encourage further research on licensure reciprocity and interstate licensing compacts to help inform considerations for licensure policy change.
Limitations
This study had limitations. First, the New Jersey program was implemented early in the pandemic when COVID-19 cases were concentrated in a few US regions, including New Jersey. Program results may have differed if the program was implemented during a time in which pandemic-related workforce needs were more widespread across the country. Second, this was a cross-sectional survey that aimed to estimate the impact of the program on healthcare workforce supply, conducted in January 2021. We are unable to offer insights into temporary license usage beyond that point in the pandemic, and the survey did not include specific questions about interstate licensure compacts. We were also unable to assess whether or how providers holding primary licenses in New Jersey obtained and/or used temporary licenses through programs offered by other states. Third, the survey had a 33% response rate, 45% of whom were mental health providers. The response rate is similar to prior surveys of healthcare providers in New Jersey26 and nationally,27 but care should be taken not to generalize beyond the study sample, as non-respondents' characteristics are unknown. Fourth, the survey may overrepresent certain mental health provider types who received a New Jersey temporary license. For example, psychologists and social workers are 64% of the license-using respondent pool, while mental health prescribers are only 6% (see Appendix B). We stratified our analyses where possible to reduce bias from overrepresented groups and found that trends were generally similar across the provider types. Fifth, respondents self-reported how they used their temporary license and the characteristics of their patients. There isn't a centralized source to confirm the self-reports although claims data could provide additional insight in some areas. Further efforts would be needed to create and maintain a database to capture licensee activities specific to this study population.
Conclusion
New Jersey's Temporary Emergency Reciprocity Licensure program was highly utilized by mental health providers with primary licenses from outside of New Jersey—many of whom are from underrepresented racial and ethnic groups. This increased provider supply resulted in at least 30000 New Jersey individuals, many of whom are also from health disparity populations, receiving mental health care provided by the temporary licensees. The use of New Jersey's temporary licensure program suggests that this program was a timely policy response that helped increase access to mental healthcare and access to a diverse mental health workforce during the pandemic. As our nation continues to battle a burgeoning healthcare workforce crisis,18 there is an urgent need to increase identification, implementation, and study of innovative licensure policy approaches in order to enhance mental health care continuity, access, and equity.
Acknowledgments:
The views expressed here are the authors' and do not necessarily reflect the views of the State of New Jersey or the Robert Wood Johnson Foundation. We thank colleagues at the New Jersey Division of Consumer Affairs and Rutgers University for initial review of the survey instrument and findings, as well as survey participants for sharing their experiences using temporary New Jersey licenses.
References
Appendices
Appendix A. List of survey questions used in the analysis of mental health providers who used the New Jersey Temporary Emergency Licensure Reciprocity Program
Characteristics
1. What is your profession?
○ Acupuncturist
○ Advanced Practice Nurse
○ Alcohol and Drug Counselor
○ Athletic Trainer
○ Audiologist
○ Dentist
○ Genetic Counselor
○ Hearing Aid Dispenser
○ Marriage and Family Therapist
○ Nurse (other than Advanced Practice Nurse)
○ Occupational Therapist
○ Optician (Ophthalmic Dispenser or Ophthalmic Technician)
○ Optometrist
○ Perfusionist
○ Pharmacist
○ Physical Therapist
○ Physician
○ Physician Assistant
○ Professional Counselor
○ Psychoanalyst
○ Psychologist
○ Respiratory Therapist
○ Social Worker
○ Other: ______
2. Have you used your NJ temporary license to provide care to New Jersey residents during the pandemic?
○ Yes (skip to Q In which months…)
○ No
3. Why didn't you use the NJ temporary license? ______ (open text), then skip to Demographics section
4. In which months have you cared for NJ patients? (check all that apply)
○ March
○ April
○ May
○ June
○ July
○ August
○ September
○ October
○ November
○ December
5. What types of COVID-related care have you / are you providing to NJ patients? (check all that apply)
○ Testing (PCR, rapid test, and antibody)
○ Treating COVID+ patients in inpatient settings (hospitalized)
○ Treating COVID+ patients in outpatient settings
○ Providing or facilitating connections to social services (eg, food banks)
○ Providing mental health or behavioral health services
○ Other (specify): _____
6. What types of non-COVID-related care have you / are you providing to NJ patients? (check all that apply)
○ Lab tests
○ Treating patients in inpatient settings (hospitalized)
○ Treating patients in outpatient settings
○ Providing or facilitating connections to social services
○ Providing mental health or behavioral health services
○ Other (specify): ____
○ None
New Jersey Patients
7. Approximately how many NJ patients have you served using your NJ temporary license? _____ (integer value)
8. About what share of your NJ patients was… new to you? (optional)
○ ___%
9. About what share of your NJ patients… were covered by Medicaid/NJ Family Care/Uninsured?
○ None
○ 1–10%
○ 11–25%
○ > 25%
○ Don't know
10. About what share of your patient care time for NJ patients was telehealth (phone, video, or e-visit)?
○ ___%
11. Of the care delivered to NJ patients via telehealth, about what share was by AUDIO only (ie, not FaceTime or other video interface)?
○ ____ %
12. Of the care delivered to NJ patients via telehealth, about what share was by VIDEO (eg, FaceTime, Zoom, patient portal, or comparable platforms)?
○ ____ %
13. In what languages have you conversed with NJ patients, other than English? (check all that apply)
○ Arabic
○ Chinese (including Mandarin and Cantonese)
○ Haitian Creole
○ Hindi
○ Italian
○ Korean
○ Gujarati
○ Polish
○ Portuguese
○ Russian
○ Spanish
○ Tagalog (including Filipino)
○ Other (specify): ____
○ None
14. How are you getting paid for the services you are providing to NJ patients using your NJ temporary license? (Check all that apply)
○ Reimbursed by private insurer
○ Government program (Medicare, Medicaid)
○ Salary or locum tenens
○ Volunteer
○ Other (specify): ______
○ Don't know
15. About what share of your NJ patients… have a chronic or severe mental health diagnosis or substance use disorder?
○ None
○ 1–10%
○ 11–25%
○ >25%
○ Don't know
16. About what share of your NJ patients… are from underserved racial/ethnic minority populations (eg, Black/African American, Hispanic/Latinx, immigrant, etc.)?
○ None
○ 1–10%
○ 11–25%
○ >25%
○ Don't know
17. About what share of your NJ patients… have physical disabilities?
○ None
○ 1–10%
○ 11–25%
○ >25%
○ Don't know
Demographics
18. In which state is your primary out-of-state license issued?
○ Dropdown list of states
19. Do you consider yourself Hispanic or Latino?
○ Yes
○ No
○ Prefer Not to Answer
20. Which category or categories best describe your race? (check all that apply)
○ American Indian or Alaska Native
○ Asian
○ Black or African American
○ Native Hawaiian or Other Pacific Islander
○ White
○ Prefer Not to Answer
○ Other: ________
21. What is your gender?
○ Female
○ Male
○ Undesignated/Non-binary (Note: “Undesignated/Non-binary” encompasses all gender identities that are not exclusively male or female, including, but not limited to: intersex, agender, amalgagender, androgynous, bigender, demigender, genderfluid, genderqueer, neutrois, nonbinary, pangender, third sex, transsexual, Two Spirit, or otherwise unspecified.)
○ Prefer Not to Answer
22. What is your age range?
○ 18–29 years old
○ 30–39 years old
○ 40–49 years old
○ 50–59 years old
○ 60–69 years old
○ 70 years or older
○ Prefer Not to Answer
Appendix B. Distribution of the participants who responded to a survey of out-of-state healthcare practitioners who were granted a New Jersey temporary license between March 2020 and January 2021
Funding/support: Support for Rutgers Center for State Health Policy research was provided by the Robert Wood Johnson Foundation (Grant ID# 78350).
Other disclosures: Magda Schaler-Haynes served as Senior Advisor at the New Jersey Office of the Attorney General based in the Division of Consumer Affairs during the establishment of the emergency licensure program and the survey.
Author notes
Author contributions: MSH conceived of the licensure program evaluation. MSH and AMN contributed to the design and execution of the study protocol, which included designing and conducting the survey. AMN, EK, and MSH conceptualized the specific research questions. JC performed the data analysis, with review and iterative discussions with AMN and EK. All authors reviewed the findings and identified key findings and discussion points. AMN, EK, MN, and DHL drafted the majority of the first draft, with critical review and edits by MSH and JC in subsequent drafts. All authors reviewed and approved the final version.