Abstract

Introduction

Many health care institutions are working to improve depression screening and management with the use of the Patient Health Questionnaire 9 (PHQ-9). Clinical decision support (CDS) within the EHR is one strategy, but little is known about effective approaches to design or implement such CDS. The purpose of this study is to compare implementation outcomes of two versions of a CDS tool to improve PHQ-9 administration for patients with depression.

Methods

This was a retrospective, observational study comparing two versions of a CDS. Version 1 interrupted clinician workflow, and version 2 did not interrupt workflow. Outcomes of interest included reach, adoption, and effectiveness. PHQ-9 administration was determined by chart review. Chi-square tests were used to evaluate associations between PHQ-9 administration with versions 1 and 2.

Results

Version 1 resulted in PHQ-9 administration 77 times (15.3% of 504 unique encounters) compared with 49 times (9.8% of 502 unique encounters) with version 2 (P = .011).

Discussion

An interruptive CDS tool may be more effective at increasing PHQ-9 administration, but a noninterruptive CDS tool may be preferred to minimize alert fatigue despite a decrease in effectiveness.

Introduction

Depression affects nearly 300 million people worldwide and can significantly interfere with quality of life.1  The World Health Organization1  reports that depression is a significant cause for disability leading to emotional and physical challenges. Unfortunately, depression remains underdiagnosed and undertreated.2,3 

To assist in diagnosis and treatment, there are many screening tools available. The most widely used tool for clinical care is the validated Patient Health Questionnaire-9 (PHQ-9), which has become best practice for identifying depression.4  The PHQ-9 is used to help diagnose depression and monitor symptom severity.4  To facilitate integration into clinician workflow and clinical documentation, PHQ-9 scores are often documented in structured fields within EHRs. However, despite the widespread availability of the PHQ-9, fewer than 3% of US adults are screened for depression during office visits.5 

The Centers for Medicare and Medicaid Services6  and the US Preventive Services Task Force7  both recommend routine depression screening in adults and encourage use of the PHQ-9. Furthermore, in value-based reimbursement models, Centers for Medicare and Medicaid Services financially incentivizes practices to use the PHQ-9. As such, health care institutions are implementing interventions to improve test administration and documentation of PHQ-9 scores. One potential intervention is to incorporate clinical decision support (CDS) tools within EHRs. Here, we describe the design and implementation outcomes of the addition of a CDS tool to improve PHQ-9 administration in a primary care setting.

Methods

This is a retrospective study comparing implementation outcomes of two versions of a clinician-facing CDS tool to improve PHQ-9 administration. Version 1 was implemented in May 2016 for approximately 18 months. After the trial period of version 1, version 2 was designed with input from clinician end users and implemented in November 2017. Outcomes of the alerts from version 1 during the month of February 2017 were compared with outcomes of version 2 in February 2018. This study compares reach (patients and clinicians exposed), adoption (clinician did not dismiss), and effectiveness (documentation of PHQ-9 scores) of the 2 versions of the CDS tool from the perspective of the clinician.

Description of the Health System and CDS Tools

This study took place across 13 primary care practices within a large health system. The health system has used 1 integrated EHR, Epic Systems, since 2011.

The goal of both versions of the CDS tool was to increase PHQ-9 administration and documentation of scores when the most recent PHQ-9 score was elevated and a repeat PHQ-9 was clinically indicated. Version 1 was designed with input from a multidisciplinary team representing primacy care clinicians, psychiatry, clinical informaticians, analysts, and population health experts. It was designed as an active CDS that would interrupt clinicians (advanced practice providers, residents, physicians) each time they opened a clinical encounter for a patient meeting the following criteria: most recent PHQ-9 was ≥9 and last alert was ≥3 months ago. The alert that appeared recommended administration of the PHQ-9 and provided 8 response options (Table 1), including the option to update the PHQ-9 score or to indicate the PHQ-9 was not appropriate. Once a clinician selected 1 of the 8 response options, the CDS would no longer be visible for that encounter; otherwise, it would continue to interrupt whenever a provider entered the patient encounter.

TABLE 1

Design features of both versions of the clinical decision support (CDS) toola

Design features of both versions of the clinical decision support (CDS) toola
Design features of both versions of the clinical decision support (CDS) toola

Interim evaluation of version 1, including clinician feedback, revealed that revisions were needed to address concerns of excessively frequent and sometimes inappropriate alerts. For example, it was identified that version 1 would sometimes alert more than once for a given clinician-patient visit (if a response option was not selected). Clinicians also stated that version 1 would sometimes alert when the most recent PHQ-9 was several years old and, thus, no longer relevant. During the interim evaluation, the clinician dismissal rate for version 1 was estimated to be 94%.

The redesigned CDS tool (version 2) incorporated the following changes: it was passive, only considered PHQ-9 scores within the past 18 months, and had a simplified user interface with fewer response options. As a passive tool, version 2 did not interrupt workflow, but was available on demand for clinicians through 2 different pathways: 1) by clicking on an icon in the top banner/header of the EHR screen or 2) navigating to the dedicated CDS section available when documenting clinical encounters. As a passive CDS, it was available throughout the patient encounter and allowed access to the information at the clinician's discretion. Table 1 compares the design specifications of the 2 CDS tools.

Parallel to implementation of versions 1 and 2 of the clinician-facing CDS tool, 2 versions of a medical assistant–facing CDS tool were also implemented. The medical assistant–facing CDS tool was implemented to support clinical workflows that encouraged medical assistants to administer PHQ-9 surveys to patients during the rooming process. Similar to the clinician-facing CDS tools, version 1 of the medical assistant–facing CDS tool was active with multiple response options and an estimated 99% dismissal rate. Version 2 of the medical assistant–facing CDS was subsequently designed to be passive with few response options.

Data Collection

Instances of the CDS alerts, the identity of the patient, and clinician responses to the CDS were collected using an analytics reporting tool within the EHR. Patient and visit characteristics were collected via manual chart review. PHQ-9 completion was reviewed via manual chart review as responses to a CDS are not always reflective of actual action taken. Completion of the PHQ-9 was defined as documentation within structured fields of the EHR. A 20% sample of the data collected was reviewed by an independent investigator to validate accuracy.

Outcomes

The primary outcome of this study was effectiveness, defined as PHQ-9 administration for patients at an outpatient encounter when the CDS tool alerted (see Table 1). Because both versions could alert more than once for the same patient and encounter, the total number of alerts for both versions of the tool were calculated based on the number of unique patient encounters. A secondary outcome was time to PHQ-9 administration using the most recent baseline PHQ-9 prior to the alert as the comparator. The time to administration was determined by taking patients with a documented PHQ-9 score on the same day of their visit and calculating the difference between that date and the date of their most recent baseline PHQ-9 score.

Statistical Analysis

Clinician response to the CDS, patient demographics, and visit characteristics were summarized descriptively. Differences in categorical demographics and visit characteristics between versions 1 and 2 were compared using chi-square tests of independence. A two-sample t test was used to test for differences in age, and Wilcoxon rank sum tests were used to evaluate differences in the distribution of number of medications between versions. Chi-square tests were used to evaluate associations between PHQ-9 administration with each version. SAS 9.4 (SAS Institute Inc) was used for analyses. The study was approved by the Colorado Multiple IRB.

Results

In February 2017, there were a total of 1987 alerts for 504 unique patient encounters with version 1 (median; IQR alerts per encounter: 3; 2, 5). In February 2018, there were a total of 917 alerts for 502 unique encounters with version 2 (median; IQR alerts per encounter: 2; 1, 2). Because version 2 was passive, the number of alerts does not reflect the number of clinicians who opted to review the alert. The mean age of patients was 51 years, and the majority were female, White, and insured. More patients exposed to version 2 had a documented diagnosis of dysthymia or depression compared with version 1 (78% vs 64%). Baseline characteristics of patients and encounters exposed to each of the tools are described in Table 2.

TABLE 2

Characteristics of patients and encounters with an alert

Characteristics of patients and encounters with an alert
Characteristics of patients and encounters with an alert

Adoption and Responses to the CDS

Of the 1987 alerts with version 1, 1870 (94.1%) were not adopted (dismissed) with no documented reason, a reason was documented in 105 (5.3%) instances, and in 12 (0.6%) instances, clinicians stated they would or did complete the PHQ-9. The most common reason provided for not adopting the CDS was PHQ-9 was not appropriate (Table 3). Because version 2 was passive and did not request or allow clinicians to provide a reason for not following the recommendation, adoption rates and reasons for not adopting are not applicable.

TABLE 3

Clinician-stated responses to clinical decision support (CDS) version 1

Clinician-stated responses to clinical decision support (CDS) version 1
Clinician-stated responses to clinical decision support (CDS) version 1

PHQ-9 Administration Frequency and Timing

Version 2 of the tool resulted in fewer PHQ-9 administrations compared with version 1. Version 1 resulted in PHQ-9 administration 77 times (15.3% of 504 unique encounters) compared with 49 times (9.8% of 502 unique encounters) with version 2 (P = .011).

For both tools, a PHQ-9 was more likely to be administered when there was a documented antidepressant or psychotropic medication change during a given encounter (version 1: 4% with no medication change vs 22% with a change, P < .001; version 2: 2% with no medication change vs 10% with a change, P < .002). Documentation of dysthymia or chronic depression was significantly associated with PHQ-9 administration with version 2 but not version 1 (version 1: 63% documented PHQ-9 in absence of dysthymia or chronic depression vs 74% documented PHQ-9 in presence of dysthymia or chronic depression, P = .053; version 2: 76% documented PHQ-9 in absence of dysthymia or chronic depression versus 92% documented PHQ-9 in presence of dysthymia or chronic depression, P = .013). Encounters in which a referral to a psychiatrist was placed or a psychiatric indication was billed for were more likely to result in PHQ-9 administration with version 1 (version 1: 0% documented PHQ-9 in absence of referral or billing for psychiatric indication vs 6% documented PHQ-9 in presence of referral or billing for psychiatric indication, P < .001; version 2: 1% documented PHQ-9 in absence of referral or billing for psychiatric indication vs 0% documented PHQ-9 in presence of referral or billing for psychiatric indication, P = .509). Associations between encounter characteristics and PHQ-9 administration are summarized in Table 4.

TABLE 4

Associations between visit characteristics and Patient Health Questionnaire-9 (PHQ-9) administration for both versions of the clinical decision support (CDS) tool

Associations between visit characteristics and Patient Health Questionnaire-9 (PHQ-9) administration for both versions of the clinical decision support (CDS) tool
Associations between visit characteristics and Patient Health Questionnaire-9 (PHQ-9) administration for both versions of the clinical decision support (CDS) tool

For the secondary outcome of time to PHQ-9 administration, the median (IQR) number of days between baseline and follow-up PHQ-9 was 264 (IQR: 163, 479) days with version 1 and 229 (IQR: 144, 334) with version 2 (P = .1055).

Discussion

This study suggests CDS reminders can improve administration frequency of PHQ-9. This study also highlights how changes to specific design features can improve specificity of CDS tools with minimal impact on effectiveness. When comparing the interruptive version 1 of the tool to the noninterruptive version 2, frequency of alerts and PHQ-9 administration decreased with the noninterruptive version 2. However, the decrease in effectiveness was not in proportion to the decrease in alert frequency. Alerts decreased in frequency by more than 2-fold with version 2, yet the decrease in PHQ-9 administration was modest. Further, given that version 2 was passive, the alert frequency of version 2 overestimates the instances in which clinicians opted to view the alert. As such, the decreased effectiveness is more likely influenced by the passive, noninterruptive nature of version 2 than the decrease in alert frequency.

Others have compared active and passive tools. In general, active CDS tools are found to be more effective at changing behavior compared with passive CDS tools.8,9  Our study is consistent with these prior findings. The current study expands the body of literature evaluating the effectiveness of passive versus active CDS tools and suggests active CDS tools may be more effective but potentially at the cost of higher dissatisfaction and alert fatigue.10  Active CDS tools interrupt users' workflow at a predefined time point, which is not always aligned with the planned activity. This misalignment is a source of user frustration and can potentiate alert fatigue.

In our health system, the decrease in effectiveness with version 2 was outweighed by the decrease in alert frequency and intrusiveness; thus, version 2 has been continued. For other clinical situations of higher acuity, such a trade-off may not be optimal. Measures to decrease alert fatigue are critical in all settings, including primary care in which clinicians are flooded with alerts. Although alert fatigue was not directly measured, reducing the frequency of alerts and inappropriate alerts can improve clinician attention and response to alerts when they do appear, including those that require more urgent action.11  In our case, we added a lookback period of 18 months for the most recent PHQ-9 scores because clinician users felt this would increase the appropriateness of the CDS alerts, which also led to a substantial decrease in alert frequency.

Our findings may not be generalizable across all settings, including those with different institutional norms surrounding use of CDS tools. Health systems have different standards for the implementation and design of CDS tools as well as different capabilities based on their technical infrastructure. The findings from our study also have some limitations. As with all CDS tools, we are not able to establish causality of the change in PHQ-9 administration behavior directly with the addition of a CDS tool, but comparing the 2 tools does provide insights into the value of the different CDS designs (notably, active vs passive design). Further, we are unable to attribute changes in effectiveness of documenting PHQ-9 scores to the clinician- or medical assistant–facing CDS tools. However, our goal was to evaluate the clinician-facing CDS tool from the perspective of the impact on the clinician, including frequency of alerts and instances of adopting the CDS (did not dismiss the CDS).

Conclusion

CDS tools can improve PHQ-9 administration within primary care settings. An interruptive CDS tool may be more effective at increasing PHQ-9 administration, but a noninterruptive CDS tool may be preferred to minimize alert fatigue despite a decrease in effectiveness. The findings from this study can be considered by other institutions that are designing CDS tools to improve PHQ-9 administration and depression management.

References

1. 
Depression fact sheet [Internet].
World Health Organization
.
c2020 [cited 2020 June 17]. Available from: https://www.who.int/news-room/fact-sheets/detail/depression
2. 
Wang
PS,
Berglund
P,
Olfson
M,
Pincus
HA,
Wells
KB,
Kessler
RC.
Failure and delay in initial treatment contact after first onset of mental disorders in the National Comorbidity Survey Replication
.
Arch Gen Psychiatry
.
2005
;
62
(6)
:
603
-
13
.
3. 
Hirschfeld
RM,
Keller
MB,
Panico
S,
Arons
BS,
Barlow
D,
Davidoff
F,
et al
The National Depressive and Manic-Depressive Association consensus statement on the undertreatment of depression
.
JAMA
.
1997
;
277
(4)
:
333
-
40
.
PubMed PMID: 9002497.
4. 
Kroenke
K,
Spitzer
RL,
Williams
JBW.
The PHQ-9
.
J Gen Intern Med
.
2001
;
16
(9)
:
606
-
13
.
5. 
Depression screening rates among adults increased slightly in recent years, but remain low [Internet]
.
American Psychiatric Association
.
6. 
Decision memo for screening for depression in adults (CAG-00425N) [Internet].
Centers for Medicare & Medicaid Services
.
7. 
Siu
AL;
US Preventive Services Task Force (USPSTF),
Bibbins-Domingo
K,
Grossman
DC,
Baumann
LC,
Davidson
KW,
et al
Screening for depression in adults: US Preventive Services Task Force recommendation statement
.
JAMA
.
2016
;
315
(4)
:
380
-
7
.
DOI: 10.1001/jama.2015.18392. PubMed PMID: 26813211.
8. 
Pevnick
JM,
Li
X,
Grein
J,
Bell
DS,
Silka
P.
A retrospective analysis of interruptive versus non-interruptive clinical decision support for identification of patients needing contact isolation
.
Appl Clin Inform
.
2013
;
4
(4)
:
569
-
82
.
9. 
Blecker
S,
Austrian
JS,
Horwitz
LI,
Kuperman
G,
Shelley
D,
Ferrauiola
M,
et al
Interrupting providers with clinical decision support to improve care for heart failure
.
Int J Med Inform
.
2019
;
131
(12)
:
103956
.
DOI: 10.1016/j.ijmedinf.2019.103956. PubMed PMID: 31525580; PubMed Central PMCID: PMC6994190.
10. 
Blecker
S,
Pandya
R,
Stork
S,
Mann
D,
Kuperman
G,
Shelley
D,
et al
Interruptive versus noninterruptive clinical decision support: usability study
.
JMIR Hum Factors
.
2019
;
6
(2)
:
e12469
.
DOI: 10.2196/12469. PubMed PMID: 30994460; PubMed Central PMCID: PMC6492060.
11. 
Patient safety primer: Alert fatigue [Internet]
.
Agency for Healthcare Research and Quality
.
c2019 [cited 2020 June 17]. Available from: https://psnet.ahrq.gov/primer/alert-fatigue

Competing Interests

Disclosures: K.E.T. was supported by the National Heart Lung and Blood Institute K12 Training grant (no. K12HL137862). The authors declare that there is no conflict of interest.

This is an open access article distributed under the terms of the Creative Commons Attribution-NonCommercial 3.0 License, which permits non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.