Why Quantifying Recidivism After Remediation Is So Difficult: The Experience of an Education Provider

When regulators hold licensees to account for problematic professional conduct, members of the public and the professions themselves must be able to trust that sanctions are appropriate and impactful and will make a lifelong impression. Often regulator-ordered sanctions include remedial coursework to address ethics, boundary or professionalism lapses. Understandably, regulators and the public want to feel confident that remedial coursework is effective and consider observed rates of recidivism as a marker for effectiveness. Because the professional literature is limited in this regard, organizations that provide remedial courses are often asked the question, “How do you know that what you do works?”

PBI Education (PBI) is one such organization. Founded in 2001, PBI has assisted nearly 15,000 health care professionals across the United States and Canada to examine the process of how and why they committed their infractions, understand why they are being held to account, and then to rehabilitate and recommit to professional ideals. Despite such considerable experience, we have found that there is no straightforward way to measure recidivism after remediation. Therefore, we undertook to develop a more nuanced understanding of the disciplinary process, experience, and outcomes for licensees, regulators and educators. We analyzed a subset of PBI course graduates for whom we had six to 11 years of follow up. This effort revealed many regulatory and logistical obstacles to arriving at a specific, accurate answer to the recidivism question. By describing our investigative process and lessons learned, this paper provides some benchmarks by which the effectiveness of a remedial intervention may be gauged and offers recommendations that may improve the tracking and support of remediated licensees for the purposes of determining recidivism.

We tracked all participants from the state of California who passed PBI’s Professional Boundaries and Ethics, Medical Ethics and Professionalism, Pharmacy Ethics and Professionalism, or Maintenance and Accountability Seminar courses taken between January 1, 2010, and December 31, 2014, for any subsequent interactions with their California regulator. Those who failed their course did not graduate and were not included in this analysis. We defined “subsequent interactions” with a regulator as any public formal accusation that led to a regulatory action, petition, citation, decision, or order listed on the regulator’s website for each individual in our study sample. This definition was purposefully broad so as to capture any potential wrongdoing that might suggest recidivist activity. For example, it was necessary to review formal accusations made as a result of findings of an investigation and leading to the disciplinary action because the reasons for disciplinary action and coursework referral are not spelled out in the orders themselves; accusations appear in separate appendices. Simple accusations that did not lead to regulatory action were not considered for the purposes of identifying recidivism. Any subsequent conduct of concern could then be compared to the original infraction for which course attendance was ordered.

We tracked these participants in two stages. First, for the purposes of presenting a poster at the 2019 annual meeting of the Federation of State Medical Boards (FSMB), we provided descriptive data on the participants through December 31, 2018. This provided a follow-up period that ranged from a minimum of four years (for those participants who attended our course in December 2014) to a maximum of nine years (for those participants who attended our course in January 2010). Second, for the purposes of providing extended follow-up of the same cohort of participants, we tracked them for an additional two years through December 31, 2020. Thus, the follow-up period ranged from a minimum of six to a maximum of 11 years. The descriptive data presented in the Results section reflect the extended follow-up period.

California licensees were selected for two reasons: 1) California referrals represent PBI’s largest sample size, compared to referrals from other jurisdictions, and 2) the California Department of Consumer Affairs (DCA), the state-wide regulator and licensing entity, maintains websites for each clinical discipline (e.g., medical, physician assistant, pharmacy, etc.) that provide current, easily searchable licensee information, allowing accurate and timely tracking of California graduates of our courses.

All course graduates’ licenses were checked against the DCA website for any subsequent, post-course interaction with their regulator. All participants with a post-course interaction were examined independently on a case-by-case basis by reviewing all publicly available documents on the participant’s regulator’s website. Participants were searched for by name, with name spelling, middle initials, license numbers, clinical specialty and year of original licensure double-checked to ensure accuracy and eliminate duplicates. The initial cohort, followed through December 31, 2018, was examined by three PBI staff members (one RN, one MD, and one Master of Fine Arts). The second cohort, followed through December 31, 2020, was examined by two PBI staff members (the same MD as in the initial analysis and one Bachelor of Science). All examiners followed the same process to search for documents.

Participants with any post-course interaction with their regulator were assigned to one of three groups:

• Primary recidivism, defined as re-offenses with the same conduct of concern that occasioned their original referral to a remedial course.

• Secondary recidivism, defined as offenses that differed from the conduct of concern that occasioned the original referral to a remedial course. We chose to consider such offenses as recidivism based on the assumption that graduation from a course in ethics, boundaries, or professionalism, in addition to other sanctions and the entire experience of being held to account by a regulator, should serve to sensitize licensees to matters of professional conduct that extend beyond the particular issue for which course attendance was mandated.

• Other, defined as some other issue that brought them back again before their regulator.

All staff members’ group assignments were compared. Where there was uncertainty or disagreement, a final determination was made by consensus.

Between January 1, 2010, and December 31, 2014, 322 California clinicians graduated from PBI’s Professional Boundaries and Ethics, Medical Ethics and Professionalism, Pharmacy Ethics and Professionalism, or Maintenance and Accountability Seminar courses. Of those, 25 duplicate records were found due to clinicians taking more than one PBI course. The duplications were eliminated and those clinicians were only counted once, during the year that they took their first course, thereby maximizing their follow-up window. An additional 87 did not have accusations, citations, decisions, or orders that set forth the reasons why they took the course. Without knowing those reasons, recidivist activity could not be determined. Thus, 210 par ticipants met inclusion criteria (Table 1). These participants were licensed by 10 different boards (Table 2). Medical doctors (MDs) and pharmacists accounted for nearly 85% of the sample. Of the entire sample, six (2.9%) relapsed with the same misconduct as occasioned their initial board action, meeting criteria for primary recidivism. Fourteen (6.7%) had lapses in their professional conduct of a different sort after graduating from their PBI course — including violations of the terms and conditions of an order of probation, such as failure to attend for a fluid-sample collection or failure to obey a law, or findings of clinical incompetence after taking a boundaries course for a sexual boundary violation — meeting our definition of secondary recidivism. Another 28 (13.3%) were categorized as “other” for reasons such as opting to surrender their license rather than go through the entire disciplinary process, voluntary surrender for unstated reason, non-renewal of license due to retirement, revocation due to failure of a competence assessment, or additional accusations before the mandated course was taken. Although we found no evidence of any recidivism within this group, the time period of follow-up was necessarily truncated, preventing us from determining whether recidivism might have occurred had they been followed for the full study period (Table 3).

After six to 11 years of follow-up, graduates from PBI’s Professional Boundaries and Ethics, Medical Ethics and Professionalism, Pharmacy Ethics, or Maintenance and Accountability Seminar courses had a primary recidivism rate of 2.9% and a secondary recidivism rate of 6.7%. These rates compare favorably to other published recidivism rates, as detailed below. However, direct comparisons should be made with caution because of differences in methodology, cohort types and sizes, inconsistent inclusion of remedial education in sanctions and lack of distinguishing primary from secondary recidivism.

For example, in one study, physicians who had received either mild or medium-to-severe sanctions had recidivism rates of 9.7% and greater than 20%, respectively.¹  Further, in one case-control study of nurses followed for up to five years, the recidivism rate was 39%, although specific educational requirements represented only 10% of probationary terms and conditions.²  Another larger study of nurses followed for up to five years showed a recidivism rate of 26.6%. Only 24.1% of the study sample was required to complete specific educational programs as part of probation, and the kind of education was not specified.³  Of note, we could find no relevant literature from outside the United States. In our personal communication with colleagues in Canada and Australia, there is great interest in the topic of recidivism, but publications are lacking.

When providers of remedial courses are asked the very legitimate question about recidivism among course graduates, an underlying assumption is that the better the course, the lower the recidivism rate. Although that assumption makes intuitive sense, it fails to recognize that remedial coursework is almost always mandated by a regulator in the context of a disciplinary order that includes several other sanctions. True, courses should be impactful by virtue of their content, process, and the quality of the background and referral documentation available to the course provider, but a course’s impact can also be bolstered by impactful discipline.^1,4  Thus, whether a licensee reoffends or not could be due simply to the impact of the remedial course alone or in combination with the monitoring, therapy, supervision, professional health program involvement, and/or other mandated interventions.

Another way course providers assess the effectiveness of remedial coursework is from course participant feedback and plans. For example, during PBI’s extended follow-up Maintenance and Accountability Seminars, faculty hear many course graduates express that what they learned about the process of committing their offenses, and the required Personalized Protection Plans they developed, are instrumental in preventing relapses. They share the many ways in which they have elevated their accountability post-course. Between January 1 and July 30, 2021, 498 (95.8%) of 520 PBI course graduates opted to provide narrative responses to a post-course evaluation question regarding plans for clinical practice change following their remedial course with specific, concrete examples, some of which had been implemented within days of completing their course.⁵  Moreover, PBI courses address known predictors of and risk factors for recidivism, such as high defensiveness (e.g., denial, rationalization, other blaming) and disdain for professional standards in general,⁶  as well as the development of victim empathy, self-awareness, a personalized plan of rules and safeguards to improve self-regulation, mentoring relationships, avoidance of professional isolation and enrollment in an extended follow-up program.^1,7,8  Addressing these predictors and risks is especially important, since previous board action is observed to be a potent risk factor for subsequent board sanction.^1,9 

In order to determine whether a specific remedial course, or remedial courses in general, are effective in preventing recidivism, one would need to design studies to track outcomes of disciplined licensees who either took remedial courses without any other sanctions or had board-ordered sanctions that did not include coursework. Trying to answer the recidivism question in this way would be inadvisable, as returning a disciplined licensee to practice without the benefit of remedial instruction or other interventions is unlikely to ensure the safety of the public.

One might think that it shouldn’t be so difficult to quantify infractions and then track reoccurrences. However, the paucity of published studies on the topic from the United States and elsewhere, plus the challenges encountered while following PBI course graduates for the purposes of determining whether recidivism occurred, suggest otherwise. Our work has uncovered several reasons that contribute to why answering the recidivism question is difficult:

1. Defining recidivism. Recidivism can mean different things to different people. We believe the regulatory community and the public would be interested chiefly in what we call primary recidivism — that is, the rate of relapse of the same offense that occasioned the disciplinary action. However, we also believe that future problems of a different sort would be of additional interest, particularly if those problems reflected impaired judgment or failures in professional conduct. That is why we also considered them a type of recidivism, but others might disagree and not consider them recidivism at all. Moreover, because we found that not all subsequent problems are significant (e.g., failing to attend for a single fluid collection because the lab closed early that day) or reflect a problem in an area that was not addressed in the remedial course taken (e.g., documentation or prescribing problems after taking a boundaries course), we believed it is valuable to distinguish primary from secondary. Not all studies of recidivism make this distinction.

2. Significance of apparent recidivism. We discovered that what may appear to be recidivist behavior, such as noncompliance with probationary terms and conditions, actually was a clue to an underlying mental or physical health issue. Further, we noted instances where a licensee was successfully remediated in the area of ethics or boundaries but later had problems in a different domain, such as medical records. Does the new problem signal a licensee with a more global lack of accountability — in which case one might argue that this is recidivism — or should the new problem be considered a completely separate matter — in which case this would not be an example of recidivism? Regulators and researchers should ask themselves these questions as they design studies and interpret recidivism data.

3. Labor-intensive process. Tracking previously disciplined licensees’ subsequent interactions with their regulator requires examining all publicly available documents to accurately classify whether the actions that triggered those interactions constituted primary or secondary recidivism or no recidivism at all. The regulatory language often found in board orders, such as unprofessional conduct, negligence, gross negligence, and similarly broad and vague terms do not specify what sort of conduct needed to be remediated in the first place for the purposes of determining whether recidivism occurred later. The process of examining regulatory documents to discover the exact reason why a licensee took a remedial course and the date the infractions occurred with respect to the course date, and to track all probationary terms and conditions for potential future violations, requires reviewing all formal accusations (to find those that led to a regulatory action), citations, decisions, and orders, an extremely labor-intensive process.

4. Identifying the purpose of remediation. As we saw in the large number of cases that could not be included in our data analysis, many licensees took our courses without an antecedent accusation or order. Therefore, it was impossible to know the purpose of the remedial intervention in their cases. Without knowing their original offense made it impossible to know whether a re-offense occurred.

5. Confounders. More than 13% of the course graduates in our data analysis had subsequent inter actions with their regulator for reasons we categorized as “other.” That percentage exceeds our percentages of primary and secondary recidivism combined. In many of those “other” cases, we could not ascertain why they voluntarily surrendered their licenses or had them revoked, or the surrender or non-renewal did not reflect recidivism. Other published studies of recidivism included all sub sequent board actions^1–3  which would have considered such cases as recidivists. We believe those cases represent confounders that may spuriously elevate recidivism calculations.

6. Multiple jurisdictions. Many health care professionals hold licenses in multiple jurisdictions. After professional discipline in one state, licensees may choose not to practice there any longer. If a re-offense occurs in another state, that state may not recognize it as recidivism (unless they are already apprised of the original offense). Similarly, researchers may not appreciate an offense as recidivism if they do not know that a similar offense occurred elsewhere.

   What about clinicians who have licenses in multiple states that participate in an interstate licensure compact? Occupational licensure compacts offer multistate practice for U. S. physicians, nurses and other health care professionals.¹⁰  Although a history of disciplinary actions renders a physician ineligible to participate in the Interstate Medical Licensure Compact (IMLC),¹¹  professional wrongdoing can occur after licensure in IMLC-participating states. The centralized database of investigative and disciplinary information available to IMLC-participating boards could provide a mechanism for detecting recidivism posed by licensure in multiple jurisdictions.

7. Inconsistencies across jurisdictions and professions. The same offense can be handled very differently in one state compared to another, or in one health care profession compared to another. For a given offense, some regulatory actions rise to the level of formal, reportable discipline, whereas others do not. Similarly, some disciplinary orders are comprehensive, whereas others are not. Therefore, analyses of recidivism rates would need to take these inconsistencies into account.

8. Access to disciplinary documents. Not all regulators’ websites provide easy and timely access to public disciplinary documents. This presents an obstacle to more widespread analyses of post-disciplinary issues that may represent recidivism.

9. Duration of follow-up. How long a licensee should be followed for recidivism is unclear, although most would probably agree that the longer the time period, the better. One study reports an average follow-up period of approximately two years.¹²  The lag time between the filing of an accusation and subsequent board action that includes mandated course attendance can be many years, arguing for a much longer duration of follow-up. Our data analysis picked up recidivists when followed for six to 11 years.

10. Who reports and collects the data. Some studies rely on licensee self-reports of recidivism. This source of information may be inferior to data collection performed by neutral third parties or by regulatory staff. Further, a regulator may not become aware of re-offenses that occur in the workplace if those offenses do not rise to the level of board notification. Even if such re-offenses do not prompt institutional discipline and/or adversely affect clinical privileges, they still may qualify as examples of recidivism.

11. Quality and appropriateness of remedial course. Some licensees may be ordered to take a remedial course that is not adequate or appropriate to address their infraction. For example, a licensee with a sexual boundary violation may be ordered to take an ethics or professionalism course that may only include a brief module on boundaries. Although it is true that a sexual boundary violation represents a failure of professional ethics and conduct, impactful remediation in this case would require a course that is specifically targeted to boundary violations. Moreover, regulator-approved courses must meet certain expectations for remediation, such as evidence that the licensee recognizes the harms they have caused, takes responsibility for their actions, cultivates insights and motivation to enact behavioral change and develops plans to ensure that they will not reoffend when encountering inevitable future stresses.

12. Impact of remedial course or disciplinary action. When tracking licensees for post-disciplinary recidivism, it is important not to draw conclusions about the remedial intervention itself, but to consider the components of the entire regulatory order as a whole. If a licensee reoffends, it could represent a failure of the remedial course, but it could also represent a lack of adequate safeguards, assessments, and treatments in the regulatory order.

As with any reporting of descriptive data, limitations exist. We looked only at small-group, live educational interventions and not at other types, such as 1:1 tutorials or coaching. Despite a longer follow-up period than in other published studies, it is possible that additional participants in this cohort might relapse after the snapshot of them six to 11 years post-course. Moreover, we looked only at California licensees; our results may have been different had we studied participants from other jurisdictions. Nevertheless, looking at one state provided important consistency in how professional wrongdoing was handled.

Some of our participants may, indeed, have relapsed without our knowing. One reason may be that, had allegations been reported to their board within the study time period, investigations may have been underway, and formal accusations not yet filed. Further, for participants who may be licensed in other states, we did not look for new formal accusations or disciplinary reports outside California.

The size of our sample dictated that participants with a range of infractions be considered together as a single cohort. In a much larger sample, it may be illuminating to analyze the outcomes of participants according to their infraction (e.g., sexual boundary violation, addiction-related violations, negligence, etc.).

Follow-up of 210 California licensees who passed PBI courses in ethics, professionalism, and/or boundaries for up to 11 years yielded a primary recidivism rate of 2.9% and a secondary recidivism rate of 6.7%. Although very favorable numbers, the process of quantifying recidivism in this cohort revealed many logistical and practical reasons why recidivism is an exceedingly difficult area to study carefully and to quantify accurately. Further, a remedial course is but one component of a disciplinary package that, together with monitoring, supervision, assessments, mental and physical health treatment, and other measures, can prevent recidivism.

Evidence used by remedial course faculty that the remediation process has begun includes observed development of empathy for those the licensee has harmed or exploited, expressions of remorse and self-assessment, insights into the risk factors, vulnerabilities, and resistance that increased the licensees’ potential to commit a professional offense and the development of a sound plan to lower their violation and relapse potential. Participation in an extended follow-up program keeps them engaged in the remediation process and supports them in enacting their plans, items that contribute to the prevention of recidivism.^1,6–8 

It would be naive to say that every errant health care professional can avoid recidivism. Although every disciplined clinician has the potential to re-offend, the vast majority can avoid recidivism, assuming that the disciplinary action is impactful and includes rigorous remedial education when indicated. And yet, simply taking a remedial course is insufficient; regulators require successful completion. Participants do fail PBI courses and similarly rigorous remedial courses offered by other entities. Those who fail return to their regulator for further disposition. In such cases, even though the attempt to pass the course was unsuccessful, the remediation process itself successfully identified licensees who are unable, unwilling or simply not ready to resume practice safely. It is the goal of the health care professions to ensure that the public can trust clinicians who have come through the disciplinary and remediation process successfully, and to prevent the unsuccessful ones from returning to practice.

Our work contributes to the conversation among regulators, educators, professional organizations and schools, researchers and the public that recidivism can be prevented. Toward that end, we offer the following recommendations to assist in improving data collection and long-term follow up, and generating additional research questions and avenues for exploration:

• Support regulators’ efforts to post timely, publicly accessible reports of disciplinary actions, including formal accusations and citations. Membership in a professional interstate licensure compact may facilitate these efforts.

• Develop a practical and reasonable system for long-term monitoring of clinicians with a disciplinary history. For example, a Workgroup on Recidivism could be assembled with representatives from the FSMB, the IMLC Commission, the National Practitioner Data Bank, and the Federation of Medical Regulatory Authorities of Canada. The collective experience and existing data possessed by the members of such a workgroup position them well to develop a best-practice model on how the monitoring should be done, by whom, and for how long. In addition, the input of other professional organizations with a longstanding interest in recidivism, such as the National Council of State Boards of Nursing,^2,3  could provide valuable, novel insights and approaches in a collegial, inter-professional activity — a model of teamwork at a metalevel.

• Statutory language in some board orders does not describe the exact infraction, which impedes accurate categorization and tracking of re-offenses. Consider information technology solutions so that exact infractions could be computer coded and tabulated and then paired with the educational interventions ordered to facilitate longitudinal follow-up of outcomes.

• Other potential sources of data on recidivism after remediation include hospitals, other health care organizations and professional schools. However, the reasons why such entities mandate attendance in a remedial course typically are memorialized in private, confidential documents. This process limits researcher access and makes it difficult to know whether the infractions that occasion these referrals might differ qualitatively from the infractions of concern to a regulator. These entities often look to regulators for guidance on how to handle professional wrongdoing. A benefit of developing a body of more robust studies of recidivism using data from regulators would be to further inform the disciplinary dispositions in these other entities.

• In future studies with an adequate sample size, participants could be categorized by clinical specialty, infraction type, size of community where they practice and practice type (e.g., solo, multi-specialty group, hospitalist). Such categorization may lead to important insights into both quantity and quality of professional risks among these sub-groups that would merit additional investigation and yield recommendations for impactful sanctions.

• Make background and referral information available to course providers, allowing them to target their efforts to each licensee’s unique underlying issues and full range of concerns of the regulator.

• Order remedial instruction to be completed within a short timeframe (e.g., within 60–90 days). Immediacy of educational intervention improves the connection between wrongdoing and corrective action, hastens change and prevents resistance from becoming entrenched, thereby increasing the impact of the educational intervention and reducing the likelihood of reoffending.

• Apply quality standards when vetting remedial courses, using the factors reported in the professional literature on recidivism to mitigate against predictors and risks (e.g., development of victim empathy, self-awareness, a plan to improve self-regulation, extended follow up).

• Because previous board action is a potent risk factor for subsequent board sanction, explore how support for remediated clinicians from professional societies, peer groups of other remediated clinicians, workplaces and elsewhere might decrease their isolation, assist in maintaining their accountability and thereby reduce their risk of recidivism.

The author gratefully acknowledges the contributions of the PBI Education team in data collection, synthesis and review and oversight of final content.