Objective

Chiropractic trainees require exposure to a diverse patient base, including patients with multiple medical conditions. The Veterans Affairs (VA) Chiropractic Residency Program aims for its doctor of chiropractic (DC) residents to gain experience managing a range of multimorbid cases, yet to our knowledge there are no published data on the comorbidity characteristics of patients seen by VA DC residents. We tested 2 approaches to obtaining Charlson Comorbidity Index (CCI) scores and compared CCI scores of resident patients with those of staff DCs at 1 VA medical center.

Methods

Two processes of data collection to calculate CCI scores were developed. Time differences and agreement between methods were assessed. Comparison of CCI distribution between resident DC and staff DCs was done using 100 Monte Carlo simulation iterations of Fisher's exact test.

Results

Both methods were able to calculate CCI scores (n = 22). The automated method was faster than the manual (13 vs 78 seconds per patient). CCI scores agreement between methods was good (κ = 0.67). We failed to find a significant difference in the distribution of resident DC and staff DC patients (mean p = .377; 95% CI, .375–.379).

Conclusion

CCI scores of a VA chiropractic resident's patients are measurable with both manual and automated methods, although automated may be preferred for its time efficiency. At the facility studied, the resident and staff DCs did not see patients with significantly different distributions of CCI scores. Applying CCI may give better insight into the characteristics of DC trainee patient populations.

Chiropractic care is increasingly being delivered in US hospitals and other medical settings.1,2  The largest integrated US health care system, the Veterans Health Administration of the Department of Veterans Affairs (VA),3  has seen a 822% increase in the number of patients receiving chiropractic care at VA facilities from 2004 to 2015,4  and the system continues to add chiropractic clinics at additional facilities.5 

In general, patients seen in hospitals have multiple and more severe medical problems. For instance, it is estimated up to 0.7% of acute low back pain patients seen in general primary care settings have underlying malignancy, compared with up to 7% of those seen in hospitals.6  One specific population, veterans receiving VA health care, have substantially worse health status, more medical conditions, and higher use of medical resources compared to the general patient population.79  These patients are more than 14 times more likely to have poor health status and to have 5 or more medical conditions.8  Multimorbidity—the presence of 2 or more chronic conditions in an individual—can be associated with greater challenges and complexity of case management approaches.10 

The VA Chiropractic Residency Program is a novel hospital-based training program providing postgraduate training in integrated clinical practice, the delivery of chiropractic care in an integrated hospital system.11  Patient care is the primary component of the residency, and 1 aim is for a resident to gain experience managing a range of multimorbid musculoskeletal cases. Each VA Chiropractic Residency Program includes a case log, with data manually entered by residents, among its quantitative measures to evaluate a resident's training.12  Although resident case logs are used to track some comorbid diagnoses, to our knowledge there have been no published reports on the characteristics of these patients.

Previous work has explored the degree of exposure to multimorbid patients that occurs during doctor of chiropractic (DC) education in the United States and examined the prevalence of specific comorbid conditions among populations receiving chiropractic care.13,14  However, a validated comorbidity measure was not used to assess for the concurrent burden of disease, and to our knowledge no studies have yet reported the exposure of DC resident trainees to multimorbid patients.

Several approaches to assessing patient comorbidity from administrative data have been described, including the Charlson Comorbidity Index (CCI). CCI is a measure of overall burden of illness based on the number of comorbid conditions existing along with a patient's principal diagnosis.15,16  It is the most widely used comorbidity index and has been validated in patient populations with various diagnoses or undergoing various surgical procedures.15  CCI scores have been shown to predict risk of 1-year mortality,16,17  10-year mortality,18  health service cost,19  and health service use, including physician visits, prescription drugs, and hospitalizations.20 

Since VA chiropractic residency training aims to provide clinical experience managing multimorbid cases, using a validated measure of multimorbidity to assess resident caseload can be helpful for program evaluation. Additionally, since it is not known how closely resident case morbidity tracks with that of staff DCs, an assessment method for this is needed. The purpose of this project was to test 2 approaches to obtaining CCI scores in a VA resident caseload and then to compare the CCI scores of resident patients with those of staff DCs at 1 VA medical center.

This project was conducted by the Chiropractic Section of the Department of Physical Medicine and Rehabilitation at the VA Connecticut Healthcare System (VACHS). The facility Research Department designated this a program assessment activity not requiring institutional review board review.

As summarized in Table 1, the CCI score is calculated based on weighted values that are assigned to 17 diagnostic categories representing comorbid conditions and weighted values assigned to decade of age.16,17,21  A total score, ranging from 0 to 37 points, is calculated from the sum of the weighted conditions and age.16 

Table 1-

Charlson Comorbidity Index Components

Charlson Comorbidity Index Components
Charlson Comorbidity Index Components

We developed and tested 2 processes of data collection to calculate CCI scores: a manual method and an automated method. Data to calculate CCI scores for both methods were obtained at the veteran's index chiropractic visit, defined as the first visit in which a patient presented to a VACHS chiropractor.

For the manual method, a modified version of the existing VACHS residency case log was created using Microsoft Excel (Microsoft Corporation, Redmond, WA). Fields to document each CCI comorbid condition and a formula to calculate the CCI score following previously validated methods were added to the resident case log.17  Additionally, a reference glossary including comorbid condition descriptions was included.17  CCI comorbid conditions were identified through manual chart review of VA's electronic health record. This included review of a patient's electronic problem list, a list of previously diagnosed conditions chosen to be entered by a health care professional, and review of recent encounter notes. Data were then manually entered into the modified case log by 1 investigator (VL), and a patient's CCI score was calculated.

For the automated method, all International Classification of Diseases, Tenth Revision (ICD-10) diagnostic codes from both inpatient and outpatient encounters in a veteran's VA medical record, from October 1, 2015, through the veteran's index chiropractic visit, were obtained from VA's Corporate Data Warehouse. Data processing was performed using Python 3.5 (Python Software Foundation, Beaverton, OR) to (1) identify comorbid conditions based on previously published ICD-10 code groupings21  and (2) calculate a CCI score following previously validated methods.16,17,21 

Data from veterans who were seen by the Chiropractic Section at VACHS from July 2018 to February 2019 were used. This encompassed a 3-week pilot period to compare methods to obtain CCI scores and a 6-month retrospective observation of VACHS Chiropractic Section as a whole.

Comparison of Methods to Obtain CCI Scores

The manual and automated methods were evaluated by 2 measures: (1) the time required to obtain CCI scores and (2) the agreement of CCI score between methods. CCI scores were obtained using both methods for consecutive, identical patients seen by a chiropractic resident during a 3-week period from January 28, 2019, to February 15, 2019. This was a pilot project so sample size estimates to ensure appropriate power were not performed. Duration of time required to log patient information with the manual method and time required to retrieve necessary data with the automated method for identical patients was recorded. Agreement of CCI scores between methods was defined as the matching numerical CCI score between the manual and automated methods. For comparison of agreement, a free-marginal kappa statistic was used and interpreted using standard criteria.22,23  A kappa value < 0.20 was interpreted as poor agreement; 0.21–0.40 as fair; 0.41–0.60 as moderate; 0.61–0.80 as good; and 0.81–1.00 as very good.22 

Assessment of CCI Scores

CCI scores were assessed for the entire VACHS Chiropractic Section, which at the time the study was conducted, included 7 staff DCs (3 full time and 4 part-time) and 1 resident. The distribution of CCI scores of patients seen by the resident DC was compared to that of all staff DCs. We retrospectively looked at a 6-month period of patients seen by the entire chiropractic clinic from July 1, 2018, to December 31, 2018. We obtained the CCI score for distinct patients seen by each DC using the automated method. The automated method was chosen because, after comparing both methods, it required less time and person-work to obtain CCI scores than did the manual method. The distribution of CCI for patients seen by individual DCs was described as the proportion of his or her patient caseload.

Distribution of CCI scores and mean CCI score of the entire VACHS Chiropractic Section was assessed. Additionally, the effect of resident and staff DC status on the distribution of CCI score was examined as categorical data. Fisher's exact test was used with a significance level of α = .05. This was preferred to a χ2 analysis, given the data were not normally distributed and the expected counts for higher comorbidity score categories failed to meet those necessary for usual χ2 test assumptions.24  Monte Carlo simulation of the Fisher's exact test25  was used due to the size of the contingency table (17 rows × 2 columns) and was repeated across 100 simulation iterations to identify a mean p value and 95% confidence interval (CI) for comparing the distributions. Statistical analyses were completed using R Version 3.6.2 (R Foundation, Vienna, Austria) and Microsoft Excel.

Comparison of Methods to Obtain CCI Scores

In the 3-week assessment period, 22 of the resident's patient charts were reviewed. Using the manual method, chart review and manual data entry for CCI scores took a total of 33.3 minutes for 22 patients, an average of 1.3 minutes per patient. With the automated method, executing the processing code took 4.4 minutes for 22 patients, an average of 13 seconds per patient. The automated method was approximately 7 times faster than the manual method. Agreement between methods was evaluated by comparing CCI scores obtained by each method for each identical patient. Of the 22 patients, 15 CCI scores (68.2%) matched between the manual and automated methods. Free-marginal kappa was good at 0.67 (95% CI, 0.47–0.88).

Assessment of CCI Scores

From July 1, 2018, to December 31, 2018, VACHS DCs saw 1387 distinct patients (219 by the resident DC). Median age was 57.5 years, with a range from 21 to 94 years. CCI scores ranged from 0 to 16, with a mean CCI score for all patients seen by VACHS DCs of 2.73. Overall, 70.0% of VACHS patients had a CCI score >0, and 10.0% of patients had a score ≥7. Distribution of CCI scores by proportion of each DC's patient panel is shown in Table 2. The distribution of CCI scores of patients seen by the resident DC and by all staff DCs each followed a quasi-negative binomial distribution (Figure 1). The Fisher's exact test simulation comparing resident DC patient CCI score distribution to CCI score distribution of all patients seen by staff DCs failed to demonstrate a significant difference across 100 simulation iterations (mean p = .377; 95% CI, .375–.379).

Table 2-

CCI Score Distribution by Percentage for All DCs at VACHS, July–December 2019

CCI Score Distribution by Percentage for All DCs at VACHS, July–December 2019
CCI Score Distribution by Percentage for All DCs at VACHS, July–December 2019
Figure 1-

Charlson Comorbidity Index score distribution by percentage for all staff doctors of chiropractic vs residents at Veterans Affairs Connecticut Healthcare System July 1 to December 31, 2018, N = 1387.

Figure 1-

Charlson Comorbidity Index score distribution by percentage for all staff doctors of chiropractic vs residents at Veterans Affairs Connecticut Healthcare System July 1 to December 31, 2018, N = 1387.

This project reports on the development and analysis of 2 methods to obtain CCI scores in a VA chiropractic residency program. To our knowledge, this is the first description of CCI scores of a chiropractic trainee's caseload. Previous work examining student clinical training case mix emulating that of practicing chiropractors has been mixed. Report from 1 college student clinic found that patients are dissimilar in demographic and clinical characteristics,14  while another college found that their teaching clinics patients are similar in demographics to practicing clinic patients.13  To our knowledge, there has been no report of a multimorbidity measure used as an assessment tool to compare between teaching clinics and practicing clinics. Using CCI may be a way to assess trainees, students, or residents; to better understand the case mix they are exposed to; and to allow for comparison between clinical training and practice.

We were able to obtain CCI scores by performing manual chart review and data entry into a spreadsheet with the manual method, as well as by querying and processing administrative data with the automated method. We found the latter method was faster (78 vs 13 seconds per patient). On average, VACHS residents encounter over 1000 patients during their 1-year residency. Based on our findings, using the automated method in a complete VA residency period would save an estimated 21.6 hours when compared to the manual method.

We found that agreement between our manual and automated methods is good, with a free-marginal kappa of 0.67. This agreement is higher than reports from previous work. In a systematic review of 7 studies, agreement between CCI calculations from chart review and administrative data was poor to fair, with kappa ranging from 0.30 to 0.56.22  Our findings may be due to the relatively small number of charts we reviewed or the characteristics of ICD codes in our population. In any case, our results suggest a manual method of CCI calculation may be usable for assessing chiropractic resident caseload in small samples of patients, particularly in settings without electronic databases or the informatics expertise to analyze such databases. However, when considering overall speed and the ability to process larger datasets, our results in a chiropractic resident patient population are similar to previous work in other populations, favoring automated assessment of administrative data rather than manual chart review.15As there is no gold standard method of obtaining CCI scores, determining the appropriate method to use should consider sample size, access to medical records, and time and resource availability.

Additionally, this project reports on CCI scores among all patients receiving chiropractic care at VACHS. The mean CCI score is 2.73, with 70.0% of patients with a CCI score > 0. The CCI scores we obtained for all chiropractic patients at this facility are higher than a previous report of several chiropractic patient populations using Medicare (mean range 1.3–1.5).26  This is consistent with data showing VA patients have more medical conditions than the general population.8  Our values were also higher than a report from a national population of younger VA users (16% of cases > 0).27  This may be explained by our population including all age ranges and many comorbidities contributing to the CCI calculation developing more frequently in older individuals.

When assessing CCI score distribution between patients who were seen by the VACHS resident DC and staff DCs, we did not find a statistically significant difference in CCI score distribution (mean p = .377; 95% CI: .375–.379). This suggests that the patients being seen by the resident may not substantially differ from those being seen by staff DCs at this facility during the study period, and the clinical training of the resident may be congruent with the practice experience of the staff. To our knowledge, this is the first report comparing a feature of a resident's patient population to that of staff DCs at a VA facility.

This work presents a method to examine chiropractic resident case multimorbidity applicable in the VA training system but may also be useful in other chiropractic postgraduate or undergraduate settings. Chiropractic residency accreditation standards require programs to prepare graduates to serve in their area of advanced or focused practice.28  Exposing trainees to similarly multimorbid populations as the populations they will serve, in theory, will prepare them for managing these cases in future practice.13,14  Further exploration of the multimorbidity of patient populations in VA and other hospital or integrated systems can provide benchmarks against which the multimorbidity of patient populations in chiropractic educational settings—VA residency programs, other residency programs, DC degree programs—can be compared.

This report adds to the few reports of multimorbidity, using CCI, in the chiropractic population. Additional comparison of CCI score distributions to varying populations, health care systems, chiropractic residencies, and/or chiropractic college clinical setting populations may be helpful to understand the frequency and severity of comorbidity present within different chiropractic patient populations. More work is needed to determine the importance and effects of trainee exposure to patients with comorbidity during chiropractic clinical training.

Limitations

This project analyzed a small sample size at 1 VA facility. Future work with a larger sample across multiple facilities is needed. The time for completing the automated method may be dependent on the computational processing speed available; therefore, variations in completion time may be expected. There are inherent limitations to use of ICD diagnostic code–based indices, including coding error, omissions, and bias,29  but currently, diagnostic codes remain a common practice in identifying comorbidity and quantifying disease burden. We do not know if CCI is the best measure to assess chiropractic patient multimorbidity. Multiple other indices exist, but we chose CCI as it is the most widely recognized and validated comorbidity index.15  Additionally, there is no known model or optimal multimorbidity distribution for chiropractic patient populations in general or in chiropractic training programs for comparison.

CCI scores of a VA chiropractic resident's patients are measurable with both manual and automated methods, although automated may be preferred for its time efficiency. At the facility studied, the resident and staff DCs did not see patients with significantly different distributions of CCI scores. Applying CCI may give better insight into the characteristics of DC trainee patient populations.

This material is based upon work supported by the Department of Veterans Affairs, Veterans Health Administration, Office of Academic Affiliations, and by the Palmer College Foundation, with resources and the use of facilities at the VACHS. The authors have no conflicts of interest to disclose. The contents of this manuscript represent the views of the authors and do not necessarily reflect the position or policy of the US Department of Veterans Affairs or the United States Government.

1. 
Lisi
AJ,
Salsbury
SA,
Twist
EJ,
Goertz
CM.
Chiropractic integration into private sector medical facilities: a multisite qualitative case study
.
J Altern Complement Med
.
2018
;
24
(8)
:
792
800
.
2. 
Branson
RA.
Hospital-based chiropractic integration within a large private hospital system in Minnesota: a 10-year example
.
J Manipulative Physiol Ther
.
2009
;
32
(9)
:
740
748
.
3. 
Kizer
KW,
Demakis
JG,
Feussner
JR.
Reinventing VA health care: systematizing quality improvement and quality innovation
.
Med Care
.
2000
;
38
(6 Suppl 1)
:
I7
16
.
4. 
Lisi
AJ,
Brandt
CA.
Trends in the use and characteristics of chiropractic services in the Department of Veterans Affairs
.
J Manipulative Physiol Ther
.
2016
;
39
(5)
:
381
386
.
5. 
US Department of Veterans Affairs.
Rehabilitation and Prosthetic Services. Chiropractic Care Facility Locations 2019
.
Updated January 9, 2020.
6. 
Verhagen
AP,
Downie
A,
Maher
CG,
Koes
BW.
Most red flags for malignancy in low back pain guidelines lack empirical support: a systematic review
.
Pain
.
2017
;
158
(10)
:
1860
1868
.
7. 
Kazis
LE,
Miller
DR,
Clark
J,
et al
Health-related quality of life in patients served by the Department of Veterans Affairs: results from the Veterans Health Study
.
Arch Intern Med
.
1998
;
158
(6)
:
626
632
.
8. 
Agha
Z,
Lofgren
RP,
VanRuiswyk
JV,
Layde
PM.
Are patients at Veterans Affairs medical centers sicker? A comparative analysis of health status and medical resource use
.
Arch Intern Med
.
2000
;
160
(21)
:
3252
3257
.
9. 
Selim
AJ,
Berlowitz
DR,
Fincke
G,
et al
The health status of elderly veteran enrollees in the Veterans Health Administration
.
J Am Geriatr Soc
.
2004
;
52
(8)
:
1271
1276
.
10. 
Wallace
E,
Salisbury
C,
Guthrie
B,
Lewis
C,
Fahey
T,
Smith
SM.
Managing patients with multimorbidity in primary care
.
BMJ
.
2015
;
350
:
h176.
11. 
US Department of Veterans Affairs.
Rehabilitation and Prosthetic Services
.
VA Chiropractic Education and Training.
https://www.rehab.va.gov/chiro/Residency_Programs.asp. Updated April 23, 2020. Accessed November 6, 2019.
12. 
VA Connecticut Healthcare System.
Chiropractic Residency Program: Integrated Clinical Practice Program Handbook
.
New Haven
:
VA Connecticut Healthcare System
;
2019
:
15.
13. 
Puhl
AA,
Reinhart
CJ,
Injeyan
HS,
Tibbles
A.
Description of the case mix experienced by chiropractic students during a clinical internship
.
J Chiropr Educ
.
2017
;
31
(2)
:
132
139
.
14. 
Kaeser
MA,
Hawk
C,
Anderson
ML,
Reinhardt
R.
Community-based free clinics: opportunities for interprofessional collaboration, health promotion, and complex care management
.
J Chiropr Educ
.
2016
;
30
(1)
:
25
29
.
15. 
Yurkovich
M,
Avina-Zubieta
JA,
Thomas
J,
Gorenchtein
M,
Lacaille
D.
A systematic review identifies valid comorbidity indices derived from administrative health data
.
J Clin Epidemiol
.
2015
;
68
(1)
:
3
14
.
16. 
Charlson
ME,
Pompei
P,
Ales
KL,
MacKenzie
CR.
A new method of classifying prognostic comorbidity in longitudinal studies: development and validation
.
J Chronic Dis
.
1987
;
40
(5)
:
373
383
.
17. 
Deyo
RA,
Cherkin
DC,
Ciol
MA.
Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases
.
J Clin Epidemiol
.
1992
;
45
(6)
:
613
619
.
18. 
Charlson
M,
Szatrowski
TP,
Peterson
J,
Gold
J.
Validation of a combined comorbidity index
.
J Clin Epidemiol
.
1994
;
47
(11)
:
1245
1251
.
19. 
Charlson
M,
Wells
MT,
Ullman
R,
King
F,
Shmukler
C.
The Charlson comorbidity index can be used prospectively to identify patients who will incur high future costs
.
PLoS One
.
2014
;
9
(12)
:
e112479
.
20. 
Dominick
KL,
Dudley
TK,
Coffman
CJ,
Bosworth
HB.
Comparison of three comorbidity measures for predicting health service use in patients with osteoarthritis
.
Arthritis Rheum
.
2005
;
53
(5)
:
666
672
.
21. 
Quan
H,
Sundararajan
V,
Halfon
P,
et al
Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data
.
Med Care
.
2005
;
43
(11)
:
1130
1139
.
22. 
Leal
JR,
Laupland
KB.
Validity of ascertainment of co-morbid illness using administrative databases: a systematic review
.
Clin Microbiol Infect
.
2010
;
16
(6)
:
715
721
.
23. 
Brennan
RL,
Prediger
DJ.
Coefficient kappa: some uses, misuses, and alternatives
.
Educ Psychol Meas.
1981
;
(41)3:
687
699
.
24. 
McHugh
ML.
The chi-square test of independence
.
Biochem Med (Zagreb)
.
2013
;
23
(2)
:
143
149
.
25. 
Mehta
CR,
Patel
NR.
A network algorithm for performing Fisher exact test in R × C contingency-tables
.
J Am Stat Assoc
.
1983
;
78
(382)
:
427
434
.
26. 
Weeks
WB,
Leininger
B,
Whedon
JM,
et al
The association between use of chiropractic care and costs of care among older Medicare patients with chronic low back pain and multiple comorbidities
.
J Manipulative Physiol Ther
.
2016
;
39
(2)
:
63
75
.
27. 
Herman
PM,
Yuan
AH,
Cefalu
MS,
et al
The use of complementary and integrative health approaches for chronic musculoskeletal pain in younger US veterans: an economic evaluation
.
PLoS One
.
2019
Jun
5
;
14
(6)
:
e0217831
.
28. 
The Council on Chiropractic Education.
Residency Program Accreditation Standards–Principles, Processes & Requirements for Accreditation
.
Scottsdale, AZ
:
The Council;
2017
.
29. 
Iezzoni
LI.
Assessing quality using administrative data
.
Ann Intern Med
.
1997
;
127
(8 Pt 2)
:
666674
.

Author notes

Vivian T. Ly is a postdoctoral fellow in the Chiropractic Section, VA Connecticut Healthcare System and Yale Center for Medical Informatics, Yale School of Medicine (300 George Street, Suite 501, New Haven, CT 06520; Vivian.ly@va.gov). Brian C. Coleman is an associate research scientist at the Yale Center for Medical Informatics, Yale School of Medicine, Yale University and the Pain Research, Informatics, Multimorbidities, and Education (PRIME) Center, VA Connecticut Healthcare System (300 George Street, Suite 501, New Haven, CT 06520; Brian.Coleman@yale.edu). Christopher M. Coulis is a staff chiropractor in the Chiropractic Section of the VA Connecticut Healthcare System (950 Campbell Avenue, West Haven, CT 06516; Christopher.coulis@va.gov). Anthony J. Lisi is Chief of the Chiropractic Section at the VA Connecticut Healthcare System, and associate research scientist at the Yale Center for Medical Informatics, Yale School of Medicine, and Pain Research, Informatics, Multimorbidities, and Education (PRIME) Center, VA Connecticut Healthcare System (950 Campbell Avenue, West Haven, CT 06516; Anthony.lisi@va.gov).

Concept development: VTL, BCC, CMC, AJL. Design: VTL, BCC, AJL. Supervision: AJL. Data collection/processing: VTL, BCC. Analysis/interpretation: VTL, BCC, AJL. Literature search: VTL. Writing: VTL, BCC, AJL. Critical review: VTL, BCC, CMC, AJL.