Generalizability of a Composite Student Selection Procedure at a University-Based Chiropractic Program

Purpose: Non-cognitive admission criteria are typically used in chiropractic student selection to supplement grades. The reliability of non-cognitive student admission criteria in chiropractic education has not previously been examined. In addition, very few studies have examined the overall test generalizability of composites of non-cognitive admission variables in admission to health science programs. The aim of this study was to estimate the generalizability of a composite selection to a chiropractic program, consisting of: application form information, a written motivational essay, a common knowledge test, and an admission interview. Methods: Data from 105 Chiropractic applicants from the 2007 admission at the University of Southern Denmark were available for analysis. Each admission parameter was double scored using two random, blinded, and independent raters. Variance components for applicant, rater and residual effects were estimated for a mixed model with the restricted maximum likelihood method. The reliability of obtained applicant ranks (generalizability coefficients) was calculated for the individual admission criteria and for the composite admission procedure. Results: Very good generalizability was found for the common knowledge test (G = 1.00) and the admission interview (G = 0.88). Good generalizability was found for application form information (G = 0.75) and moderate generalizability (G = 0.50) for the written motivation essay. The generalizability of the final composite admission procedure, which was a weighted composite of all 4 admission variables was good (G_c = 0.80). Conclusion: Good generalizability for a composite admission to a chiropractic program was found. Optimal weighting and adequate sampling are important for obtaining optimal generalizability. Limitations and suggestions for future research are discussed.