## ABSTRACT

Introduction

The study investigated the use of defects per million medication orders (DPMMO) as a SMART (specific, measurable, achievable, realistic, and timely) indicator for monitoring medication safety in admission and discharge orders. The study aimed to develop and test a new indicator as an investigator of medication safety.

Methods

The study was conducted in 2018 at King Saud Medical City in Riyadh City in Saudi Arabia. A retrospective cross-sectional design was used. The research sample had 292 patients. The selected medication orders included two types of medication orders (admission and discharge order). After sufficient data had been gathered from the hospitals, a statistical analysis was carried out.

Results

Analysis of admission and discharge orders indicated that defects per million opportunities (DPMO) and DPMMO count were slightly low, while the sigma level for admission orders was slightly high. Thus, the admission order process was slightly better than the discharge order process.

Conclusion

The DPMMO indicator could serve as a SMART indicator of medical safety. It can be used as a standardized indicator in any healthcare facility, which serves as a recommendation guide in monitoring and evaluating healthcare processes or systems that affect the safety and outcomes of patients.

## INTRODUCTION

Medication safety has become one of the hot topics of discussion in the field of healthcare. Quantitative indexes to measure medication safety can serve as an important indicator of the overall performance of a hospital, highlighting the areas demanding attention. In this field, there is an important part of the wider quality and safety improvement programs. Improper use of medication can influence patient outcomes; therefore, harmless and appropriate utilization of medicines is an important component of patients' well-being in hospitals.[1] Many indicators are used by different health facilities to measure medication safety, the majority of which depend on counting instances of medication error. Based on the nature of the research, the pharmacist who reviews the appropriateness of each prescription detects errors at the prescribing phase. Rates of drug prescription errors (such as prescribing wrong dosage forms, prescribing drugs that cause allergic reactions in patients, and dosing errors) are measured by different ways, either by admission or order: for example, errors per 100 error opportunities; avoidable adverse drug events in every 1000 admissions[2]; errors in every 1000 orders[3]; and errors in every 1000 admissions.[4]

Medication discrepancy is a type of medication error that is commonly found in the hospital setting. There are unintentional differences between regimens of the drug (ie, between drugs prescribed when admitted into the healthcare facility and a patient's domestic regimen).[5] In general, the prescribing process is considered a critical part of any medication use system,[6] and reliable procedures are needed to quantify medication safety.[7]

Different types of measures are considered indicators and have been used to monitor medication safety; e.g., by counting the medication errors. Process indicators provide information about the quality of work and consist of inputs and outputs; these indicators can be considered as a tool to monitor the process.[8] Among current clinical indicators, the use of the quality metric defects per million medication order (DPMMO) has been proposed by the author for monitoring medication safety. This DPMMO metric can be used to measure the performance of a medication management process. It will reflect the safety dimension of the process. Research has examined the effectiveness of defects per million opportunities (DMPO) in medication safety. For example, Kuwaiti[9] summarized that the six-sigma methodology represented in DPMO, and this was considered important in decreasing the reported medication errors and ensuring patient safety. In addition, it has been discovered that DPMO is effective in identifying factors critical to quality in medical transcription.[10] This study was designed to propose medication safety indicator, DPMMO, derived from the well-known DPMO and corresponding sigma level for medication-related processes. Sigma levels allow performance to be compared throughout an entire organization because they are process independent.[11] Such a metric is considered crucial to stimulate the leaders and propel the owner to enhance patients' safety quality, patient care, as well as medication safety. Therefore, the objective of the current work is to conduct a field test of the DPMMO indicator as an investigator of medication safety, testing its application to admission and discharge orders thereby verifying DPMMO as a specific, measurable, attainable, relevant, and time-bound (SMART) indicator.

### Operational Definitions

#### Defect

The on-conformance of a specific quality characteristic to its planned specification. In the study checklist, a defect is assumed when one of the nine elements is missing. Those elements are as follows:

1. Diagnosis/indication

2. Drug generic name

3. Drug dose

4. Drug frequency

5. Drug route

6. Drug duration

7. Physician signature

8. Physician stamp

9. Date

#### Opportunities

The total number of possible defects. In this research, the investigators considered each medication order as an opportunity.

#### Defect per million opportunities

DPMO is a metric used to measure the performance of a specific process.

#### Defects per million medication orders

DPMMO is a newly proposed matrix developed by the investigators. The value of DPMMO is equal to the value of DPMO. The name is tailored for medication order.

#### Specific, measurable, achievable, realistic, and timely

SMART is a framework used to identify quality indicators.

## METHODS

A retrospective design cross-sectional was used in current research. The study was conducted in 2018 at King Saud Medical City in Riyadh City in Saudi Arabia. This is a tertiary medical institution that includes three hospitals (general, maternity, and pediatric) with a total of 1500 beds across all specialties. Files of the patients discharged from the center were reviewed during the period of 6 months (June to December 2017).

The authors considered the ethical approach per the King Saud Medical City policy. This work was carried out by means of a retrospective chart review. There was no direct contact with patients; however, it was approved by the institutional review board committee in King Saud Medical City.

### Sample Size

For sufficient statistical information, the data were based on the average number of discharge patients as of December 2017; 3891 with 5% alpha and a 95% CI, the research sample was 292 patients. A total of 292 files were selected using systematic random sampling and reviewed for patients discharged from June to December of 2017. The admission and discharge medication orders in each selected file were checked. Two medication orders were subsequently selected from each file for review. Consequently, we examined 584 medication orders based on a 6-month period.

### Inclusion Criteria

All the patients' files containing admission and discharge medication orders were included in the study. The cases where the medication orders include narcotics, standing, range, hidden, refill, body surface area–based, weight-based, titrated, tapering, restricted antibiotics, transfer, blanket, telephone, verbal, and total parenteral nutrition order. The patients who left the hospitals (general, maternity, and pediatric) or those who were either transferred, absconded, or discharged against medical advice were also excluded. Further, the patients without admission or discharge medication orders in their records were also not considered in the study.

### Sampling Procedures

For this research, records of 292 patients from the three hospitals (general, maternity, and pediatric) were randomly selected. The number of patients from each hospital was proportional to the total number of discharges in each hospital. The procedures were as follows:

• Step 1.

Find the total number of normally discharged patients from the three hospitals as follows: Total normal discharges = general hospital + pediatric hospital + maternity hospital (total normal discharges = 2014 + 685 + 886 = 3585).

• Step 2.
Find the percent of normal discharge in each hospital out of the total (approximated to the nearest hundred) as follows:
$$\def\upalpha{\unicode[Times]{x3B1}}$$$$\def\upbeta{\unicode[Times]{x3B2}}$$$$\def\upgamma{\unicode[Times]{x3B3}}$$$$\def\updelta{\unicode[Times]{x3B4}}$$$$\def\upvarepsilon{\unicode[Times]{x3B5}}$$$$\def\upzeta{\unicode[Times]{x3B6}}$$$$\def\upeta{\unicode[Times]{x3B7}}$$$$\def\uptheta{\unicode[Times]{x3B8}}$$$$\def\upiota{\unicode[Times]{x3B9}}$$$$\def\upkappa{\unicode[Times]{x3BA}}$$$$\def\uplambda{\unicode[Times]{x3BB}}$$$$\def\upmu{\unicode[Times]{x3BC}}$$$$\def\upnu{\unicode[Times]{x3BD}}$$$$\def\upxi{\unicode[Times]{x3BE}}$$$$\def\upomicron{\unicode[Times]{x3BF}}$$$$\def\uppi{\unicode[Times]{x3C0}}$$$$\def\uprho{\unicode[Times]{x3C1}}$$$$\def\upsigma{\unicode[Times]{x3C3}}$$$$\def\uptau{\unicode[Times]{x3C4}}$$$$\def\upupsilon{\unicode[Times]{x3C5}}$$$$\def\upphi{\unicode[Times]{x3C6}}$$$$\def\upchi{\unicode[Times]{x3C7}}$$$$\def\uppsy{\unicode[Times]{x3C8}}$$$$\def\upomega{\unicode[Times]{x3C9}}$$$$\def\bialpha{\boldsymbol{\alpha}}$$$$\def\bibeta{\boldsymbol{\beta}}$$$$\def\bigamma{\boldsymbol{\gamma}}$$$$\def\bidelta{\boldsymbol{\delta}}$$$$\def\bivarepsilon{\boldsymbol{\varepsilon}}$$$$\def\bizeta{\boldsymbol{\zeta}}$$$$\def\bieta{\boldsymbol{\eta}}$$$$\def\bitheta{\boldsymbol{\theta}}$$$$\def\biiota{\boldsymbol{\iota}}$$$$\def\bikappa{\boldsymbol{\kappa}}$$$$\def\bilambda{\boldsymbol{\lambda}}$$$$\def\bimu{\boldsymbol{\mu}}$$$$\def\binu{\boldsymbol{\nu}}$$$$\def\bixi{\boldsymbol{\xi}}$$$$\def\biomicron{\boldsymbol{\micron}}$$$$\def\bipi{\boldsymbol{\pi}}$$$$\def\birho{\boldsymbol{\rho}}$$$$\def\bisigma{\boldsymbol{\sigma}}$$$$\def\bitau{\boldsymbol{\tau}}$$$$\def\biupsilon{\boldsymbol{\upsilon}}$$$$\def\biphi{\boldsymbol{\phi}}$$$$\def\bichi{\boldsymbol{\chi}}$$$$\def\bipsy{\boldsymbol{\psy}}$$$$\def\biomega{\boldsymbol{\omega}}$$$$\def\bupalpha{\bf{\alpha}}$$$$\def\bupbeta{\bf{\beta}}$$$$\def\bupgamma{\bf{\gamma}}$$$$\def\bupdelta{\bf{\delta}}$$$$\def\bupvarepsilon{\bf{\varepsilon}}$$$$\def\bupzeta{\bf{\zeta}}$$$$\def\bupeta{\bf{\eta}}$$$$\def\buptheta{\bf{\theta}}$$$$\def\bupiota{\bf{\iota}}$$$$\def\bupkappa{\bf{\kappa}}$$$$\def\buplambda{\bf{\lambda}}$$$$\def\bupmu{\bf{\mu}}$$$$\def\bupnu{\bf{\nu}}$$$$\def\bupxi{\bf{\xi}}$$$$\def\bupomicron{\bf{\micron}}$$$$\def\buppi{\bf{\pi}}$$$$\def\buprho{\bf{\rho}}$$$$\def\bupsigma{\bf{\sigma}}$$$$\def\buptau{\bf{\tau}}$$$$\def\bupupsilon{\bf{\upsilon}}$$$$\def\bupphi{\bf{\phi}}$$$$\def\bupchi{\bf{\chi}}$$$$\def\buppsy{\bf{\psy}}$$$$\def\bupomega{\bf{\omega}}$$$$\def\bGamma{\bf{\Gamma}}$$$$\def\bDelta{\bf{\Delta}}$$$$\def\bTheta{\bf{\Theta}}$$$$\def\bLambda{\bf{\Lambda}}$$$$\def\bXi{\bf{\Xi}}$$$$\def\bPi{\bf{\Pi}}$$$$\def\bSigma{\bf{\Sigma}}$$$$\def\bPhi{\bf{\Phi}}$$$$\def\bPsi{\bf{\Psi}}$$$$\def\bOmega{\bf{\Omega}}$$$${\rm{General\ hospital\mbox{:}\quad}}2014 \div 3585 \times 100 = 56.18\% .$$
$${\rm{Pediatric\ hospital\mbox{:}\quad}}685 \div 3585 \times 100 = 19.11\% .$$
$${\rm{Maternity\ hospital\mbox{:}\quad}}886 \div 3585 \times 100 = 24.71\% .$$
• Step 3.
Find the number of the sample from each hospital by multiplying the percent by the total sample size as follows:
$${\rm{General\ hospital\mbox{:}\quad}}56.18\% \times 292 = 164.$$
$${\rm{Pediatric\ hospital\mbox{:}\quad}}19.11\% \times 292 = 56.$$
$${\rm{Maternity\ hospital\mbox{:}\quad}}24.71\% \times 292 = 72.$$
• Step 4.

The first sample was systematic randomly selected from the first 12 medical records at different intervals. A total of 292 files were screened and 61 patients (20.89%) were excluded owing to file incompletion (not having any discharge medication or admission orders). For the second round, patients were randomly selected from the first six medical records at various intervals, with 42 patients (14.38%) excluded. In the third round, patients were selected randomly by including every sixth medical record. Figure 1 shows the sampling flow chart.

Figure 1

Flow chart for sampling patient file. GH: general hospital; PedH: pediatric hospital; MatH: maternity hospital; DAMA: discharge against medical advice.

Figure 1

Flow chart for sampling patient file. GH: general hospital; PedH: pediatric hospital; MatH: maternity hospital; DAMA: discharge against medical advice.

### Data Collection Tool

A checklist was used to collect information from screened patients' medical records. The checklist consists of the two selected types of order items (admission order and discharge order).

### Data Collection Procedure

A team of two healthcare professionals (EKA and PJP) examined each selected record; medication entry and defects in the sampled medication orders were reviewed and counted. From these data, the values of DPMMO were calculated, and the results obtained were converted into sigma values using a conversion table.

### Data Analysis

Patient data were analyzed descriptively using Minitab 17. Mann-Whitney U test was used for nonnormality tests for dichotomous outcomes. A significance level of p < 0.05 was used. The value of sigma was calculated for each type of order. The value of sigma was calculated for each type of order and the corresponding DPMMO. The Pareto analysis was used to analyze the type of defect in each type of order.[12,13]

## RESULTS

### Defects per Million Medication Orders

The 584 orders (admission and discharge) from 292 patient records were screened, and 1055 defects were observed out of 5256 opportunities, resulting in 200,723 defects per million opportunities which is similar to DPMMO with a 2.33 sigma level and p = 0.009. Table 1 displays the statistics related to medication orders.

Table 1

Analysis of DPMMO at three hospitals (general, maternity, and pediatric hospital)

### Analysis of Admission and Discharge Orders

Further analysis (Table 2) showed the DPMMO in the overall discharge orders in the three hospitals was higher (213,470) than the overall admission in the three hospitals, which was lesser (191,444). The highest DPMMO was in the discharge orders of the general hospital (226,287), while the lowest was in the discharge orders in the pediatric hospital (182,540). In the pediatric hospital, the DPMMO in discharge orders was lower (182,540) than that of the admission orders (378,968). However, in maternity, the DPMMO in admission orders was lower (191,358) than the discharge order (194,444). The highest sigma level of 2.41 was obtained for discharge orders from the pediatric hospital, while the lowest sigma level of 2.25 was found for the general hospital. For maternity (Table 3), the number of defects for the admission order and the discharge order was 124 and 126, respectively. Therefore, the sigma levels for admission orders and discharge orders are 2.37 and 2.36, respectively. Conversely for the general hospital (Table 2), the number of defects was 280 for the admission orders and 334 for the discharge orders. The sigma levels are, therefore, 2.38 and 2.25 for admission and discharge orders, respectively. This means that for general and maternity, the sigma levels were higher for admission orders than discharge orders. However, for the pediatric hospital, the sigma level for admission orders is lower than that for discharge orders (Table 4).

Table 2

Analysis of DPMMO for AO and DO in the general hospital

Table 3

Analysis of DPMMO for AO and DO in maternity hospital

Table 4

DPMMO for AO and DO in pediatric hospital

Table 5

Developing indicators: a SMART criteria checklist[15]

Analysis of results from admission and discharge orders indicates that the DMPMO for admission orders is slightly lower and their corresponding sigma level (slightly higher). Consequently, the DPMMO values for both admission orders and discharge orders are high and the sigma level for both processes are quite low.

### Comparing Admission and Discharge Orders

The significance level for normality tests and hypothesis testing was 5%, meaning that the tests were performed at a 95% CI. The low value of p (<0.05) in both cases indicated the data are nonnormal and a Mann-Whitney U test was used (Figure 2).

Figure 2

Figure 2

The data for admission and discharge order defects were also not normally distributed. There were 292 admission orders and 292 discharge others; hence the Mann-Whitney U test was used above, finding a significant difference between defects in discharge and admission orders at a 5% level of significance (U = 90274.50, p = 0.0174).

### Pareto Analysis

The Pareto analysis showed that, in admission orders, the maximum number of defects were related to duration (n = 165), followed by route of administration (n = 143) (Figure 3).

Figure 3

Pareto chart of defects in admission orders.

Figure 3

Pareto chart of defects in admission orders.

In discharge orders, the most common defect was physician signatures (n = 188) followed by the incorrect drug generic name (n = 111) (Figure 4). Overall, physician signatures, route, drug generic name, and duration were the most common defects in both admission and discharge orders.

Figure 4

Pareto chart of defects in overall discharge orders.

Figure 4

Pareto chart of defects in overall discharge orders.

## DISCUSSION

In the present study, we examined the admission and discharge orders in the medical records of 292 patients from three different hospitals (general, maternity, and pediatrics), identifying 1064 defects through retrospective chart review. It was discovered that the DPMMO fits the criteria for a SMART indicator and can be used as a standardized indicator for the purpose of monitoring and evaluating the safety of medication use processes in different facilities.

The use of the measure of DPMO when tailored to medication orders resulted in the newly proposed DPMMO. The latter is suggested for use as a standardized indicator to measure medication safety rather than depending on reported medication error, which is a volunteer activity. The use of such an indicator can enable organizations to longitudinally monitor changes[14] in the process of medication management in response to any strategy or intervention designed to improve medication safety.

Previous efforts to measure medication safety have failed to settle on one standardized indicator. Using the DPMMO as an indicator of medication safety requires minimal support. Because it can be easily adopted with chart review training sessions and the method of calculating the DPMMO from medication orders, it is applicable to gathering data and aggregation across the facility as a broad scope for healthcare organizations and procedures.

The DPMMO indicator can potentially become standardized across every healthcare facility, acting as a tribute to monitor and evaluate medication safety. The suggested approach is simple to introduce and implement, while the needed basic data can be gathered by a minimum number of samples from medication orders. The only commitment is to allocate time for data collection through chart review. It is also relatively generic; hence, it can be adapted by healthcare facilities of any scope of service. It should be used in local programs, corporate, or regional levels. In addition, it can be used as a national and/or international indicator for medication safety.

### Limitations of the Study

In the process of using DPMMO to monitor medication safety, there were some factors perceived as limitations. First, the data used were collected from the prescribing process, which is only one of the five main phases of the process of managing medication use. Second, the data were collected for six months, and the proposed indicator applied to only two types of medication orders (admission and discharge orders).

The lack of a standardized indicator raises concern over the current variation among those used to monitor and evaluate medication safety. In addition, our interobserver variation must be considered as a possible DPMMO indicator limitation, but the provided data involved three hospitals (general, maternity, and pediatric) inside one healthcare organization, King Saud Medical City, implying that the exact variation in the practice is relatively small. We anticipate that DPMMO represents a good proxy for the evaluation of medical safety. However, the implementation of the DPMMO in its current form leaves behind other opportunities for errors/defects, related to the storage and administration of the prescribed medicine.

### Conclusion

This study indicated that DPMMO is a SMART indicator to test admission and discharge orders and is appropriate for quantifying and evaluating medication safety in healthcare facilities. The DPMMO depends on the prescribing phase. The DPMMO indicator has a good probability of becoming standardized throughout every healthcare facility, acting as a point of reference for monitoring and evaluation in medication management processes that affect patient safety.

## References

References
1.
Hughes
R.
Patient Safety and Quality: An Evidence-Based Handbook for Nurses
.
Agency for Healthcare Research and Quality
;
2008
.
2.
Neuspiel
DR,
Taylor
MM.
Reducing the risk of harm from medication errors in children
.
Health Serv Insights
.
2013
;
6
:
47
59
.
3.
Elden
NMK,
Ismail
A.
The importance of medication errors reporting in improving the quality of clinical care services
.
Glob J Health Sci
.
2016
;
8
:
54510
.
4.
Patel
N,
Desai
M,
Shah
S,
Patel
P,
Gandhi
A.
A study of medication errors in a tertiary care hospital
.
Perspect Clin Res
.
2016
;
7
:
168
173
.
5.
Wollersheim
H,
Hermens
R,
Hulscher
M,
et al.
Clinical indicators: development and applications
.
Neth J Med
.
2007
;
65
:
15
22
.
6.
L,
Semple
S,
Rosenfeld
E.
Literature review: medication safety in Australia
.
Sydney
:
Australian Commission on Safety and Quality in Health Care
.
Accessed September 23,
2018
.
7.
Classen
DC,
Metzger
J.
Improving medication safety: the measurement conundrum and where to start
.
Int J Qual Health Care
.
2003
;
15 Suppl 1:i41–i147.
8.
Mainz
J.
Defining and classifying clinical indicators for quality improvement
.
Int J Qual Health Care
.
2003
;
15
:
523
530
.
9.
Kuwaiti
AA.
Application of six sigma methodology to reduce medication errors in the outpatient pharmacy unit: a case study from the King Fahd University Hospital, Saudi Arabia
.
Int J Qual Res
.
2016
;
10
:
267
278
.
10.
Nandakumar
PM,
Anitha
HS,
Singh
BP.
Six sigma methodology utilization in medical transcription-A DMAIC process to identify six sigma projects
.
Int J Sci Eng Technol
.
2014
;
3
:
574
578
.
11.
Huysal
K,
Budak
YU.
Application of sigma metrics for the assessment of quality assurance using the MQ-2000 PT HbA1c analyzer
.
Biochemia Medica
.
2015
:
416
420
.
12.
Haughey
D.
Pareto analysis step by step
.
London
:
Project Smart
.
Accessed April 8,
2020
.
13.
Sanders
R.
The Pareto principle: its use and abuse
.
J Prof Serv Mark
.
1987
;
1
:
37
40
.
14.
Smeulers
M,
Verweij
L,
JM,
et al.
Quality indicators for safe medication preparation and administration: a systematic review
.
PLoS One
.
2015
;
10
:
e0122695
.
15.
Glahn
C,
Specht
M,
Koper
R.
Smart indicators on learning interactions
.
In:
Duval
E,
Klamma
R,
Wolpers
M.
Creating New Learning Experiences on a Global Scale
.
Springer
;
2007
:
56
70
.

## Competing Interests

Source of Support: King Saud Medical City. Conflict of Interest: None.