Exploring the application of the Charlson Comorbidity Index to assess the patient population seen in a Veterans Affairs chiropractic residency program

Chiropractic care is increasingly being delivered in US hospitals and other medical settings.^1,2  The largest integrated US health care system, the Veterans Health Administration of the Department of Veterans Affairs (VA),³  has seen a 822% increase in the number of patients receiving chiropractic care at VA facilities from 2004 to 2015,⁴  and the system continues to add chiropractic clinics at additional facilities.⁵ 

In general, patients seen in hospitals have multiple and more severe medical problems. For instance, it is estimated up to 0.7% of acute low back pain patients seen in general primary care settings have underlying malignancy, compared with up to 7% of those seen in hospitals.⁶  One specific population, veterans receiving VA health care, have substantially worse health status, more medical conditions, and higher use of medical resources compared to the general patient population.^7–9  These patients are more than 14 times more likely to have poor health status and to have 5 or more medical conditions.⁸  Multimorbidity—the presence of 2 or more chronic conditions in an individual—can be associated with greater challenges and complexity of case management approaches.¹⁰ 

The VA Chiropractic Residency Program is a novel hospital-based training program providing postgraduate training in integrated clinical practice, the delivery of chiropractic care in an integrated hospital system.¹¹  Patient care is the primary component of the residency, and 1 aim is for a resident to gain experience managing a range of multimorbid musculoskeletal cases. Each VA Chiropractic Residency Program includes a case log, with data manually entered by residents, among its quantitative measures to evaluate a resident's training.¹²  Although resident case logs are used to track some comorbid diagnoses, to our knowledge there have been no published reports on the characteristics of these patients.

Previous work has explored the degree of exposure to multimorbid patients that occurs during doctor of chiropractic (DC) education in the United States and examined the prevalence of specific comorbid conditions among populations receiving chiropractic care.^13,14  However, a validated comorbidity measure was not used to assess for the concurrent burden of disease, and to our knowledge no studies have yet reported the exposure of DC resident trainees to multimorbid patients.

Several approaches to assessing patient comorbidity from administrative data have been described, including the Charlson Comorbidity Index (CCI). CCI is a measure of overall burden of illness based on the number of comorbid conditions existing along with a patient's principal diagnosis.^15,16  It is the most widely used comorbidity index and has been validated in patient populations with various diagnoses or undergoing various surgical procedures.¹⁵  CCI scores have been shown to predict risk of 1-year mortality,^16,17  10-year mortality,¹⁸  health service cost,¹⁹  and health service use, including physician visits, prescription drugs, and hospitalizations.²⁰ 

Since VA chiropractic residency training aims to provide clinical experience managing multimorbid cases, using a validated measure of multimorbidity to assess resident caseload can be helpful for program evaluation. Additionally, since it is not known how closely resident case morbidity tracks with that of staff DCs, an assessment method for this is needed. The purpose of this project was to test 2 approaches to obtaining CCI scores in a VA resident caseload and then to compare the CCI scores of resident patients with those of staff DCs at 1 VA medical center.

This project was conducted by the Chiropractic Section of the Department of Physical Medicine and Rehabilitation at the VA Connecticut Healthcare System (VACHS). The facility Research Department designated this a program assessment activity not requiring institutional review board review.

As summarized in Table 1, the CCI score is calculated based on weighted values that are assigned to 17 diagnostic categories representing comorbid conditions and weighted values assigned to decade of age.^16,17,21  A total score, ranging from 0 to 37 points, is calculated from the sum of the weighted conditions and age.¹⁶ 

We developed and tested 2 processes of data collection to calculate CCI scores: a manual method and an automated method. Data to calculate CCI scores for both methods were obtained at the veteran's index chiropractic visit, defined as the first visit in which a patient presented to a VACHS chiropractor.

For the manual method, a modified version of the existing VACHS residency case log was created using Microsoft Excel (Microsoft Corporation, Redmond, WA). Fields to document each CCI comorbid condition and a formula to calculate the CCI score following previously validated methods were added to the resident case log.¹⁷  Additionally, a reference glossary including comorbid condition descriptions was included.¹⁷  CCI comorbid conditions were identified through manual chart review of VA's electronic health record. This included review of a patient's electronic problem list, a list of previously diagnosed conditions chosen to be entered by a health care professional, and review of recent encounter notes. Data were then manually entered into the modified case log by 1 investigator (VL), and a patient's CCI score was calculated.

For the automated method, all International Classification of Diseases, Tenth Revision (ICD-10) diagnostic codes from both inpatient and outpatient encounters in a veteran's VA medical record, from October 1, 2015, through the veteran's index chiropractic visit, were obtained from VA's Corporate Data Warehouse. Data processing was performed using Python 3.5 (Python Software Foundation, Beaverton, OR) to (1) identify comorbid conditions based on previously published ICD-10 code groupings²¹  and (2) calculate a CCI score following previously validated methods.^16,17,21 

Data from veterans who were seen by the Chiropractic Section at VACHS from July 2018 to February 2019 were used. This encompassed a 3-week pilot period to compare methods to obtain CCI scores and a 6-month retrospective observation of VACHS Chiropractic Section as a whole.

The manual and automated methods were evaluated by 2 measures: (1) the time required to obtain CCI scores and (2) the agreement of CCI score between methods. CCI scores were obtained using both methods for consecutive, identical patients seen by a chiropractic resident during a 3-week period from January 28, 2019, to February 15, 2019. This was a pilot project so sample size estimates to ensure appropriate power were not performed. Duration of time required to log patient information with the manual method and time required to retrieve necessary data with the automated method for identical patients was recorded. Agreement of CCI scores between methods was defined as the matching numerical CCI score between the manual and automated methods. For comparison of agreement, a free-marginal kappa statistic was used and interpreted using standard criteria.^22,23  A kappa value < 0.20 was interpreted as poor agreement; 0.21–0.40 as fair; 0.41–0.60 as moderate; 0.61–0.80 as good; and 0.81–1.00 as very good.²² 

CCI scores were assessed for the entire VACHS Chiropractic Section, which at the time the study was conducted, included 7 staff DCs (3 full time and 4 part-time) and 1 resident. The distribution of CCI scores of patients seen by the resident DC was compared to that of all staff DCs. We retrospectively looked at a 6-month period of patients seen by the entire chiropractic clinic from July 1, 2018, to December 31, 2018. We obtained the CCI score for distinct patients seen by each DC using the automated method. The automated method was chosen because, after comparing both methods, it required less time and person-work to obtain CCI scores than did the manual method. The distribution of CCI for patients seen by individual DCs was described as the proportion of his or her patient caseload.

Distribution of CCI scores and mean CCI score of the entire VACHS Chiropractic Section was assessed. Additionally, the effect of resident and staff DC status on the distribution of CCI score was examined as categorical data. Fisher's exact test was used with a significance level of α = .05. This was preferred to a χ² analysis, given the data were not normally distributed and the expected counts for higher comorbidity score categories failed to meet those necessary for usual χ² test assumptions.²⁴  Monte Carlo simulation of the Fisher's exact test²⁵  was used due to the size of the contingency table (17 rows × 2 columns) and was repeated across 100 simulation iterations to identify a mean p value and 95% confidence interval (CI) for comparing the distributions. Statistical analyses were completed using R Version 3.6.2 (R Foundation, Vienna, Austria) and Microsoft Excel.

In the 3-week assessment period, 22 of the resident's patient charts were reviewed. Using the manual method, chart review and manual data entry for CCI scores took a total of 33.3 minutes for 22 patients, an average of 1.3 minutes per patient. With the automated method, executing the processing code took 4.4 minutes for 22 patients, an average of 13 seconds per patient. The automated method was approximately 7 times faster than the manual method. Agreement between methods was evaluated by comparing CCI scores obtained by each method for each identical patient. Of the 22 patients, 15 CCI scores (68.2%) matched between the manual and automated methods. Free-marginal kappa was good at 0.67 (95% CI, 0.47–0.88).

From July 1, 2018, to December 31, 2018, VACHS DCs saw 1387 distinct patients (219 by the resident DC). Median age was 57.5 years, with a range from 21 to 94 years. CCI scores ranged from 0 to 16, with a mean CCI score for all patients seen by VACHS DCs of 2.73. Overall, 70.0% of VACHS patients had a CCI score >0, and 10.0% of patients had a score ≥7. Distribution of CCI scores by proportion of each DC's patient panel is shown in Table 2. The distribution of CCI scores of patients seen by the resident DC and by all staff DCs each followed a quasi-negative binomial distribution (Figure 1). The Fisher's exact test simulation comparing resident DC patient CCI score distribution to CCI score distribution of all patients seen by staff DCs failed to demonstrate a significant difference across 100 simulation iterations (mean p = .377; 95% CI, .375–.379).

This project reports on the development and analysis of 2 methods to obtain CCI scores in a VA chiropractic residency program. To our knowledge, this is the first description of CCI scores of a chiropractic trainee's caseload. Previous work examining student clinical training case mix emulating that of practicing chiropractors has been mixed. Report from 1 college student clinic found that patients are dissimilar in demographic and clinical characteristics,¹⁴  while another college found that their teaching clinics patients are similar in demographics to practicing clinic patients.¹³  To our knowledge, there has been no report of a multimorbidity measure used as an assessment tool to compare between teaching clinics and practicing clinics. Using CCI may be a way to assess trainees, students, or residents; to better understand the case mix they are exposed to; and to allow for comparison between clinical training and practice.

We were able to obtain CCI scores by performing manual chart review and data entry into a spreadsheet with the manual method, as well as by querying and processing administrative data with the automated method. We found the latter method was faster (78 vs 13 seconds per patient). On average, VACHS residents encounter over 1000 patients during their 1-year residency. Based on our findings, using the automated method in a complete VA residency period would save an estimated 21.6 hours when compared to the manual method.

We found that agreement between our manual and automated methods is good, with a free-marginal kappa of 0.67. This agreement is higher than reports from previous work. In a systematic review of 7 studies, agreement between CCI calculations from chart review and administrative data was poor to fair, with kappa ranging from 0.30 to 0.56.²²  Our findings may be due to the relatively small number of charts we reviewed or the characteristics of ICD codes in our population. In any case, our results suggest a manual method of CCI calculation may be usable for assessing chiropractic resident caseload in small samples of patients, particularly in settings without electronic databases or the informatics expertise to analyze such databases. However, when considering overall speed and the ability to process larger datasets, our results in a chiropractic resident patient population are similar to previous work in other populations, favoring automated assessment of administrative data rather than manual chart review.¹⁵As there is no gold standard method of obtaining CCI scores, determining the appropriate method to use should consider sample size, access to medical records, and time and resource availability.

Additionally, this project reports on CCI scores among all patients receiving chiropractic care at VACHS. The mean CCI score is 2.73, with 70.0% of patients with a CCI score > 0. The CCI scores we obtained for all chiropractic patients at this facility are higher than a previous report of several chiropractic patient populations using Medicare (mean range 1.3–1.5).²⁶  This is consistent with data showing VA patients have more medical conditions than the general population.⁸  Our values were also higher than a report from a national population of younger VA users (16% of cases > 0).²⁷  This may be explained by our population including all age ranges and many comorbidities contributing to the CCI calculation developing more frequently in older individuals.

When assessing CCI score distribution between patients who were seen by the VACHS resident DC and staff DCs, we did not find a statistically significant difference in CCI score distribution (mean p = .377; 95% CI: .375–.379). This suggests that the patients being seen by the resident may not substantially differ from those being seen by staff DCs at this facility during the study period, and the clinical training of the resident may be congruent with the practice experience of the staff. To our knowledge, this is the first report comparing a feature of a resident's patient population to that of staff DCs at a VA facility.

This work presents a method to examine chiropractic resident case multimorbidity applicable in the VA training system but may also be useful in other chiropractic postgraduate or undergraduate settings. Chiropractic residency accreditation standards require programs to prepare graduates to serve in their area of advanced or focused practice.²⁸  Exposing trainees to similarly multimorbid populations as the populations they will serve, in theory, will prepare them for managing these cases in future practice.^13,14  Further exploration of the multimorbidity of patient populations in VA and other hospital or integrated systems can provide benchmarks against which the multimorbidity of patient populations in chiropractic educational settings—VA residency programs, other residency programs, DC degree programs—can be compared.

This report adds to the few reports of multimorbidity, using CCI, in the chiropractic population. Additional comparison of CCI score distributions to varying populations, health care systems, chiropractic residencies, and/or chiropractic college clinical setting populations may be helpful to understand the frequency and severity of comorbidity present within different chiropractic patient populations. More work is needed to determine the importance and effects of trainee exposure to patients with comorbidity during chiropractic clinical training.

This project analyzed a small sample size at 1 VA facility. Future work with a larger sample across multiple facilities is needed. The time for completing the automated method may be dependent on the computational processing speed available; therefore, variations in completion time may be expected. There are inherent limitations to use of ICD diagnostic code–based indices, including coding error, omissions, and bias,²⁹  but currently, diagnostic codes remain a common practice in identifying comorbidity and quantifying disease burden. We do not know if CCI is the best measure to assess chiropractic patient multimorbidity. Multiple other indices exist, but we chose CCI as it is the most widely recognized and validated comorbidity index.¹⁵  Additionally, there is no known model or optimal multimorbidity distribution for chiropractic patient populations in general or in chiropractic training programs for comparison.

CCI scores of a VA chiropractic resident's patients are measurable with both manual and automated methods, although automated may be preferred for its time efficiency. At the facility studied, the resident and staff DCs did not see patients with significantly different distributions of CCI scores. Applying CCI may give better insight into the characteristics of DC trainee patient populations.

This material is based upon work supported by the Department of Veterans Affairs, Veterans Health Administration, Office of Academic Affiliations, and by the Palmer College Foundation, with resources and the use of facilities at the VACHS. The authors have no conflicts of interest to disclose. The contents of this manuscript represent the views of the authors and do not necessarily reflect the position or policy of the US Department of Veterans Affairs or the United States Government.