OBJECTIVE

Often we call the patient’s pharmacy to obtain a refill history to assess inhaled corticosteroid (ICS) adherence. The purpose of this project was to determine the accuracy of refill histories for ICS (with or without long-acting beta agonist) listed in Epic’s Medication Dispense History.

METHODS

We evaluated 61 patients and used data from 38 who met the following criteria: 1) under the care of the UF Pediatric Severe Asthma Clinic; 2) taking the same dose of the same ICS product for 6 months before the patient’s last clinic visit; and 3) having data available from the pharmacy where the last ICS prescription was electronically sent. We called the pharmacies to obtain a verbal report of their refill record. Then, we compared the number of refills reported to the number listed in Epic’s records using a Wilcoxon matched-pairs signed-ranks test.

RESULTS

Of the 293 refill dates listed in Epic, 157 were duplicates, giving a 54% error. After deleting duplicates, the mean (SD) number of refills listed in Epic was 3.6 (2.0) compared with 3.3 (2.0) in pharmacies over a period of 6 months (p < 0.0001). After removing duplicates Epic correctly reported the total number of refills for 30 of the 38 patients (78.9%). Seven of the remaining patients had more refills listed in Epic while 1 patient had more refills dispensed.

CONCLUSION

This study indicates that our version of Epic over-reports refills thus limiting assessment of adherence. In contrast, absence of refills in Epic is a clear indication of poor adherence.

Regular treatment with ICS is the mainstay of care for children with persistent asthma. ICS are effective in reducing bronchial hyper-responsiveness, airway inflammation, asthma symptoms and exacerbations.1  Poor adherence to ICS has been associated with an increase in asthma-related emergency department visits and hospitalizations, as well as more frequent oral steroid use.2  Also, poor ICS adherence has been identified as a risk factor for asthma deaths.3 

Unfortunately, many patients with asthma have poor overall ICS adherence.4,5  For example, in a previous study of 116 patients under the care of the University of Florida (UF) Pediatric Pulmonary Division, only 61% of the ICS prescriptions were filled.6  When compared with Florida Medicaid payments for these patients, the pharmacies provided accurate information 92% of the time. Interestingly, physicians were able to identify only 49% of patients who had an adherence rate of <50%. In a subsequent study of different patients in the same clinic, adherence to ICS averaged 44% compared with 59% for montelukast, a once daily oral tablet.7  As a result of these studies, providers in our clinic routinely call the patients’ pharmacy to evaluate adherence before stepping-up therapy in patients who are poorly controlled on ICS monotherapy.

Epic (Epic Systems Corporation; Verona, WI) is the electronic medical record (EMR) software used by UF Health. The system includes the “Medication Dispense History” feature that provides refill histories for each prescription filled by a pharmacist for a given patient, and is available to prescribers, clinical pharmacists, and nurses. Of note, this information is collected and provided to Epic by a third party, Surescripts, a company that aligns nearly all electronic health records, vendors, pharmacy benefit managers, pharmacies, and clinicians. Since calling the patient’s pharmacy to obtain a refill history is time consuming, it would be more efficient for providers to use Epic’s Medication Dispense History feature to evaluate adherence. However, there is no information on the accuracy of this feature.

Study Design and Study Population. The primary objective of the study was to determine the accuracy of refill histories for ICS (with or without long-acting beta agonist) listed in Epic’s Medication Dispense History. The secondary objective was to determine how frequently duplicate records occurred and whether they could be filtered out. Sixty-one charts were initially reviewed by the first author between May and July 2021. The inclusion criteria were as follows: 1) under the care of the UF Pediatric Severe Asthma Clinic; 2) taking the same ICS product at the same dose for 6 months before the patient’s last clinic visit; and 3) having data available from the pharmacy where the last ICS prescription was electronically sent. The exclusion criteria were a change in ICS product or dose, or the pharmacy or Epic did not display the dispense records for the period of review.

Data Collection and Analysis. Pharmacies were called that received each patient’s last electronic ICS prescription to obtain a verbal report of the patient’s prescription refill record for the 6 months prior to the last Pediatric Pulmonary Clinic visit. The pharmacy profile served as reference. All the information contained in the prescription is retained by the pharmacy software when a label for the prescription is generated. The central profile of each pharmacy was searched and if the prescription had been transferred to a different pharmacy, that pharmacy was also called.

Each patient’s refill dates, total number of refills recorded, and days’ supply dispensed, according to the pharmacy’s records, were compared with those of the Epic records. Percent accuracy was defined as the number of patients that had matching refill records divided by the total number of patients included. Based on the previous study where pharmacies were 92% accurate in reporting refills,6  we selected 90% as the threshold for Epic accurately reporting total refills. Also, the study determined the number of duplicate entries in the Epic record as well as any additional discrepancies.

Duplicates are prescription refills on the same day, or within a day or so of each other (Figure 1). These cannot be separate refills since most third-party payers will only approve payment for a 30-day supply. Duplicate records were counted as inaccuracies in Epic’s report if 2 dates were less than 23 days apart since many prescription benefit plans will allow an early refill beginning 7 days before the actual refill date.

Figure 1.

Information contained in screen display of Epic for a patient taking an ICS with long-acting beta ­agonist. Note that except for 1/4/2023, each refill has 1 duplicate. The provider and pharmacy names have been deleted to de-identify the data.

Figure 1.

Information contained in screen display of Epic for a patient taking an ICS with long-acting beta ­agonist. Note that except for 1/4/2023, each refill has 1 duplicate. The provider and pharmacy names have been deleted to de-identify the data.

Close modal

Statistical Methods. Duplications were excluded from statistical analysis and, since the data were not normally distributed, the nonparametric Wilcoxon matched-pairs signed-ranks test for the number of refills recorded by each method was used. Also, the numbers of refills ­recorded from each method were plotted along the 45-degree line of identity to determine how many data points fell above or below the line.

From the initial 61 charts reviewed, 38 met the inclusion criteria. Eleven patients were excluded because of a change in ICS product or dose and 12 patients were excluded because their pharmacy or Epic did not ­display or provide the dispense records for the period of interest. For the 38 patient records that qualified for inclusion, there were 125 unique refills listed in pharmacy records compared with 293 listed in Epic for the same 6-month period (an accuracy of 43%). After removal of the 157 duplicate entries in Epic, there were 136 refills remaining in the Epic records. The mean (SD) number of refills was 3.6 (2.0) compared with 3.3 (2.0) in pharmacies (p < 0.0001) for the 6-month period. In both sources, records for 30 patients were identical, resulting in an overall accuracy rate (after removal of duplicates) of 79%. Records of seven patients indicated more refills in Epic, while only one patient had more refills in the pharmacy record compared with Epic (Figure 2).

Figure 2.

Distribution of the number of refills indicated in Epic after duplicates were removed and the true number reported by pharmacies plotted along the line of identity (equality). The numbers in parentheses indicate the number of patients at each data point. There are a total of 38 patients. The data points above the line indicate over-reporting in Epic.

Figure 2.

Distribution of the number of refills indicated in Epic after duplicates were removed and the true number reported by pharmacies plotted along the line of identity (equality). The numbers in parentheses indicate the number of patients at each data point. There are a total of 38 patients. The data points above the line indicate over-reporting in Epic.

Close modal

The Epic Medication Dispense History feature in our version of Epic did not report refill histories accurately when compared against pharmacy dispensing records. On average, the information in Epic overestimates the true refill rate and thus gives the impression that some patients are receiving the medication when they are not. The overreporting is particularly misleading for patients with poor adherence (defined as <50% of prescribed doses).5 

The total number of duplicate records recorded for the 38 included patients was 157. Duplicate records are confusing for providers. If they do not know to disregard duplicates, they may mistakenly think that the pharmacy dispensed multiple inhalers instead of one and misinterpret the data. Duplicates could lead providers to overestimate a patient’s adherence and overlook a missed refill, or to think that the patient is misusing the medication and requiring multiple inhalers per month. However, even after the duplicates were removed, Epic over-reported refills in seven of the 38 patients (18%). The cause of this over-reporting is unlikely a result of prescriptions that were refilled but not picked up. When this happens, pharmacies return the medications to stock and reverse the charge, which in turn is supposed to remove the refill date from the Epic record. Another potential cause of over-reporting after duplicates are removed is under-reporting by the pharmacy. In fact, in a previous study the accuracy of telephoning the patient’s pharmacy to get a refill history was 92%; the 8% inaccuracy was a result of pharmacies failing to report a refill that was reimbursed by Medicaid, who provided the information directly to the investigators.

A few limitations were noted in conducting this Quality Improvement Project. The reason for focusing on time periods before each patient’s last Pediatric Pulmonary office visit was because at the time, Epic’s Medication Dispense feature only updated with each office visit. However, we have now learned that refill information can be accessed through another feature, “Reconcile Outside Information”, and have alerted providers to this option. Interestingly, this feature does not list duplicates.

Another limitation of this study is the small sample size of 38 patients. This was a result of having to ­exclude 11 patients who had a change in ICS product or dose and the other was 12 patients for whom either the pharmacy or Epic did not have data. Florida state law requires that pharmacies retain records of dispensed medications for 2 years (Chapter 64B16–28), and Epic only retains data for 1 year. Thus, patients requiring data outside of 1 year from the start of data collection were excluded.

Next Steps. The results of this study prompted our co-author, Dr Anzeela Schentrup, Coordinator for Clinical Documentation at University of Florida Physicians, to investigate the cause of the duplication. She found that Epic had been receiving refill information from multiple sources of data. For example, Surescripts reported refills from third party payers and some chain pharmacies also reported refills to Surescripts which became duplicates. This issue was addressed, and both Epic and Surescripts now have algorithms to remove the duplications. Another alternative is to use the Reconcile Outside Information function in place of the Medication Dispense History. This is the current method supported by Epic to obtain a refill history; however, it was only available when a patient had unreconciled outside medications (so not at all times). A change was made so that the data with duplications removed from Reconcile Outside Information is now available both within active patient encounters and within patient charts without the need for there to be unreconciled medications. After this change, there needs to be a subsequent study comparing the accuracy of both methods now that the duplications have been removed. Also, the providers sent a letter requesting that the refill histories be available to the in-patient staff when a patient is admitted to the hospital. This feature is available in a limited way to our inpatient staff at this time; however, we are looking into making this available in a wider number and type of workflows so that all providers and appropriate staff can easily utilize the information.

After excluding duplications, Epic’s Medication Dispense History only listed refill numbers with 78.9% accuracy, and therefore has limited usefulness unless the record indicates infrequent refills. If a patient is not refilling the ICS, they cannot be taking it. Improving the accuracy of our organization’s method of reporting refill history will allow the medical team to more efficiently evaluate patient adherence to guide patient care. Other institutions need to evaluate the accuracy of how their EMR reports this information.

EMR

electronic medical record;

ICS

inhaled corticosteroid with or without long-acting beta agonist;

UF

University of Florida

We thank the pharmacists and technicians who provided refill histories.

1.
Cloutier
MM
,
Baptist
AP
,
Blake
KV
,
et al.
2020 Focused updates to the asthma management guidelines: a report from the National Asthma Education and Prevention Program Coordinating Committee Expert Panel Working Group
.
J Allergy Clin Immunol
.
2020
;
146
(
6
):
1217
1270
.
2.
Williams
LK
,
Pladevall
M
,
Xi
H
,
et al.
Relationship between adherence to inhaled corticosteroids and poor outcomes among adults with asthma
.
J Allergy Clin Immunol
.
2004
;
114
(
6
):
1288
1293
.
3.
Ernst
P
,
Spitzer
WO
,
Suissa
S
,
et al.
Risk of fatal and near-fatal asthma in relation to inhaled corticosteroid use
.
JAMA
1992
;
268
;
3462
3464
.
4.
Morton
RW
,
Everard
ML
,
Elphick
HE
.
Adherence in childhood asthma: the elephant in the room
.
Arch Dis Child
.
2014
;
99
(
10
):
949
953
.
5.
Lindsay
JT
,
Heaney
LG
.
Nonadherence in difficult asthma: facts, myths, and a time to act
.
Patient Prefer Adherence
.
2013
;
7
:
329
336
.
6.
Sherman
J
,
Hutson
A
,
Baumstein
S
,
Hendeles
L
.
Telephoning the patient's pharmacy to assess adherence with asthma medications by measuring refill rate for prescriptions
.
J Pediatr
.
2000
;
136
(
4
):
532
536
.
7.
Sherman
J
,
Patel
P
,
Hutson
A
,
et al.
Adherence to oral montelukast and inhaled fluticasone in children with persistent asthma
.
Pharmacotherapy
.
2001
;
21
(
12
):
1464
1467
.

Disclosure. The authors declare no conflicts or financial interests. The authors had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Ethical Approval and Informed Consent. The authors assert that all procedures contributing to this work comply with the ethical standards of UF’s Quality Improvement Project Registry (Project ID# 1576) and that Institutional Review Board (IRB) approval was not required.