This investigation presents a review of all of the clinical outcome measures used by authors and published in the Journal of the American Podiatric Medical Association and the Journal of Foot and Ankle Surgery from January 1, 2011, to December 31, 2015. Of 1,336 articles published during this time frame, 655 (49.0%) were classified as original research and included in this analysis. Of these 655 articles, 151 (23.1%) included at least one clinical outcome measure. Thirty-seven unique clinical outcome scales were used by authors and published during this period. The most frequently reported scales in the 151 included articles were the American Orthopaedic Foot and Ankle Society scales (54.3%; n = 82), visual analog scale (35.8%; n = 54), Medical Outcomes Study Short Form Health Survey (any version) (10.6%; n = 16), Foot Function Index (5.3%; n = 8), Maryland Foot Score (4.0%; n = 6), and Olerud and Molander scoring system (4.0%; n = 6). Twenty-four articles (15.9%) used some form of original/subjective measure of patient satisfaction/expectation. The results of this investigation detail the considerable variety of clinical outcome measurement tools used by authors in the Journal of the American Podiatric Medical Association and the Journal of Foot and Ankle Surgery and might support the need for a shift toward the consistent use of a smaller number of valid, reliable, and clinically useful scales in the podiatric medical literature.

In 2013, Hunt and Hurwit1  published a review of patient-reported clinical outcome measures used in foot and ankle research during a 10-year period from Foot and Ankle International, The Journal of Bone and Joint Surgery-American Volume, The Journal of Bone and Joint Surgery-British Volume, The American Journal of Sports Medicine, Clinical Orthopedics and Related Research, and the Journal of Orthopaedic Trauma. Of the 16,513 total articles published in these journals during this period, they identified 878 specific to the foot and ankle that used at least one patient-reported outcome measure. They found that these 878 articles included 139 unique clinical outcome scales. The five scales most frequently used by authors were the American Orthopaedic Foot and Ankle Society (AOFAS) scales, the visual analog scale, the Medical Outcomes Study Short Form Health Survey, the Foot Function Index, and the American Academy of Orthopaedic Surgeons outcome instruments. These authors concluded that this considerable variety had the potential to be detrimental to evidence-based decision making, and they encouraged a “paradigm shift toward the consistent use of valid and reliable outcome measures for foot and ankle clinical research.”1(p8-9)

These findings might be particularly relevant in contemporary foot and ankle practice because US health-care centers, hospitals, and third-party payers are working toward value-based and outcome-based reimbursement strategies. Andrawis et al2  have criticized that the orthopedic specialties are generally lacking with respect to outcome-based evaluation because of a paucity of accepted definitions, relatively vague indications for surgical intervention, and use of a multitude of different outcome measures to evaluate similar parameters.

Because Hunt and Hurwit1  did not include any podiatric medicine–specific journals in their analysis, the aim of this study was to determine which clinical outcome measures were the most commonly published by authors in the Journal of the American Podiatric Medical Association and the Journal of Foot and Ankle Surgery.

Methods

All articles published in the Journal of the American Podiatric Medical Association and the Journal of Foot and Ankle Surgery during a 5-year retrospective period (January 1, 2011, to December 31, 2015) were manually reviewed for the use of clinical outcome measures. The inclusion criteria consisted of all original research articles. These articles were then reviewed for the use of clinical outcome rating instruments and whether a comparison was performed between preintervention and postintervention measurements. Descriptive statistics and frequency counts are provided.

Results

During the 5-year period of this investigation, the Journal of the American Podiatric Medical Association and the Journal of Foot and Ankle Surgery offered 1,336 publications (original research, case reports, reviews, instructional courses, editorials, letters to the editor, special communications, tips/quips/pearls, etc). Of these 1,336 articles, 655 (49.0%) were classified as original research. Of these 655 articles, 151 used at least one clinical outcome measure, representing 23.1% of the original research articles and 11.3% of all publications. For reference, Hunt and Hurwit1  found that in Foot and Ankle International, 32.3% of original research articles used a patient-reported outcome measure; The Journal of Bone and Joint Surgery-American Volume, 52.4%; The Journal of Bone and Joint Surgery-British Volume, 48.2%; The American Journal of Sports Medicine, 47.3%; Clinical Orthopedics and Related Research, 47.6%; and the Journal of Orthopaedic Trauma, 51.8%.

Seventy-eight of the 151 articles we reviewed (51.7%) included a comparison of preintervention and postintervention measurements for at least one of the outcomes. The remainder of the articles primarily recorded a postintervention measurement without comparison.

We observed 37 unique clinical outcomes scales used by authors during this period (Table 1). The most frequently reported scales (as a percentage of the original research articles) were the AOFAS scales (54.3%; n = 82), the visual analog scale (35.8%; n = 54), the Medical Outcomes Study Short Form Health Survey (10.6%; n = 16), the Foot Function Index (5.3%; n = 8), the Maryland Foot Score (4.0%; n = 6), and the Olerud and Molander scoring system (4.0%; n = 6). Twenty-four articles (15.9%) used some form of original/subjective measure of patient satisfaction or expectation.

Table 1

Clinical Outcome Measures Used by Authors in the Journal of the American Podiatric Medical Association and the Journal of Foot and Ankle Surgery

Clinical Outcome Measures Used by Authors in the Journal of the American Podiatric Medical Association and the Journal of Foot and Ankle Surgery
Clinical Outcome Measures Used by Authors in the Journal of the American Podiatric Medical Association and the Journal of Foot and Ankle Surgery

Discussion

The results of this investigation detail the considerable variety of clinical outcome measurement tools used by authors and published in the Journal of the American Podiatric Medical Association and the Journal of Foot and Ankle Surgery and might support the need for a shift toward the consistent use of a smaller number of valid, reliable, and clinically useful outcome scales in the podiatric medicine foot and ankle literature.

These clinical outcome measures are important because they provide physicians with an objective means to evaluate their interventions. Particularly when considering the foot and ankle, other objective measures that do not take into account at least a subjective component provided by the patient might be relatively easier to perform but do not provide information with respect to overall patient function. A common example of this is a radiographic measurement. Although it might be relatively easy to objectively compare measurement of the first intermetatarsal angle before and after a surgical procedure for hallux abducto valgus deformity, this measurement does not provide any relevant information about the patient's functional outcome or response to surgery. As US health-care centers, hospitals, and third-party payers work toward value-based and outcome-based reimbursement strategies, effective and efficient use of these tools will become increasingly important to demonstrate the effect of our interventions.3-7  These measurements have the potential to provide evidence in support of the therapies that we are recommending and performing, and should help us make evidence-based decisions when considering the relative effectiveness of similar therapies and interventions.

These assessment tools might have the most value to investigators and critical readers when they 1) are established to be reliable and valid for a specific pathologic disorder; 2) include a subjective measure of patient symptoms, satisfaction, or perceived disability; and 3) include an objective measure of patient functional status. Several authors have provided commendable reviews of the different and most commonly used clinical outcome measures for the lower extremity.8-13  These references are recommended for foot and ankle surgeons and critical readers who are interested in the pursuit of clinical science (Table 2).

Table 2

Recommended References for Foot and Ankle Surgeons and Critical Readers on Clinical Outcome Measures

Recommended References for Foot and Ankle Surgeons and Critical Readers on Clinical Outcome Measures
Recommended References for Foot and Ankle Surgeons and Critical Readers on Clinical Outcome Measures

Note that just because an outcome measure has been frequently used and published does not necessarily mean that it is valid and reliable. Similar to the investigation by Hunt and Hurwit,1  we observed that the clinical outcome measure most frequently used by authors and published in podiatric medical journals were the AOFAS scales. We found this interesting because there is substantial evidence indicating that this is an unreliable outcome measure.14-16  In fact, in 2011, the AOFAS published a position statement stating that “scores from the AOFAS Clinical Scoring Systems have not been found to be valid or reliable, and their continued use is not recommended.”17(p842)

The present results additionally demonstrated that approximately half (51.7%) of articles publishing a clinical outcome measure did so with a preintervention versus postintervention comparison. We believe that this comparison is important to further demonstrate the effect of the intervention, as opposed to simply reporting postintervention findings.

As with any scientific investigation, critical readers are encouraged to review the study design and results to reach their own conclusions, although the preceding represented our conclusions based on the specific results. We also embrace the fact that all investigations have limitations, and this one has several to consider. First, we recognize that not every investigation and study design would be improved with the addition of an appropriate clinical outcome measure. Although we reported that the clinical outcome measures in the Journal of the American Podiatric Medical Association and the Journal of Foot and Ankle Surgery were used less frequently than has been reported in the orthopedic foot and ankle literature,1  this is not necessarily a negative finding. The use of clinical outcome measures is indicated for some study designs but not for others. Second, we offered no analysis or opinion about whether the published clinical outcome measures were performed appropriately for the included published articles. Third, we did not perform an analysis of the use of clinical outcome measures over time. It is possible that these tools were more frequently published in 2015 versus 2011, for example.

In conclusion, the results of this investigation provide evidence of a wide quantity and variety of clinical outcome measures used by authors and published in the Journal of the American Podiatric Medical Association and the Journal of Foot and Ankle Surgery. Particularly when considering the potential for outcome-based reimbursement in the future, we believe that these results should encourage our national organizations and publications to educate their members on this topic and more consistently use a smaller number of valid and reliable measures.

Acknowledgment: We acknowledge the editors of the Journal of the American Podiatric Medical Association and the Journal of Foot and Ankle Surgery, as well as the respective organizations of the American Podiatric Medical Association and the American College of Foot and Ankle Surgeons, for agreeing to this collaborative effort.

Financial Disclosure: None reported.

Conflict of Interest: None reported.

References

1
Hunt
KJ,
Hurwit
D:
Use of patient-reported outcome measures in foot and ankle research
.
J Bone Joint Surg Am
95
:
e118
,
2013
.
2
Andrawis
JP,
Chenok
KE,
Bozic
KJ:
Health policy implications of outcomes measurement in orthopaedics
.
Clin Orthop Relat Res
471
:
3475
,
2013
.
3
Snyder
CF,
Aaronson
NK:
Use of patient-reported outcomes in clinical practice
.
Lancet
374
:
369
,
2009
.
4
Chow
A,
Mayer
EK,
Darzi
AW,
et al:
Patient-reported outcome measures: the importance of patient satisfaction in surgery
.
Surgery
146
:
435
,
2009
.
5
Shirley
ED,
Sanders
JO:
Patient satisfaction: implications and predictors of success
.
J Bone Joint Surg Am
95
:
e69
,
2013
.
6
Bosco
JA
III,
Sachdev
R,
Shapiro
LA,
et al:
Measuring quality in orthopaedic surgery: the use of metrics in quality management
.
Instr Course Lect
63
:
47
,
2014
.
7
Katz
G,
Ong
C,
Hutzler
L,
et al:
Applying quality principles to orthopaedic surgery
.
Instr Course Lect
63
:
465
,
2014
.
8
Martin
RL,
Irrgang
JJ:
A survey of self-reported outcome instruments for the foot and ankle
.
J Orthop Sports Phys Ther
37
:
72
,
2007
.
9
Button
G,
Pinney
S:
A meta-analysis of outcome rating scales in foot and ankle surgery: is there a valid, reliable, and responsive system?
Foot Ankle Int
25
:
521
,
2004
.
10
Smith
MV,
Klein
SE,
Clohisy
JC,
et al:
Lower extremity-specific measures of disability and outcome in orthopaedic surgery
.
J Bone Joint Surg Am
95
:
468
,
2012
.
11
Shultz
S,
Olszewski
A,
Ramsey
O,
et al:
A systematic review of outcome tools used to measure lower leg conditions
.
Int J Sports Phys Ther
8
:
838
,
2013
.
12
Pynsent
PB:
Choosing an outcome measure
.
J Bone Joint Surg Br
83
:
792
,
2001
.
13
Goldstein
CL,
Schemitsch
E,
Bhandari
M,
et al:
Comparison of different outcome instruments following foot and ankle trauma
.
Foot Ankle Int
31
:
1075
,
2010
.
14
SooHoo
NF,
Shuler
M,
Fleming
LL;
American Orthopaedic Foot and Ankle Society: Evaluation of the validity of the AOFAS Clinical Rating Systems by correlation to the SF-36
.
Foot Ankle Int
24
:
50
,
2003
.
15
Baumhauer
JF,
Nawoczenski
DA,
DiGiovanni
BF,
et al:
Reliability and validity of the American Orthopaedic Foot and Ankle Society Clinical Rating Scale: a pilot study for the hallux and lesser toes
.
Foot Ankle Int
27
:
1014
,
2006
.
16
Madeley
NJ,
Wing
KJ,
Topliss
C,
et al:
Responsiveness and validity of the SF-36, Ankle Orthoarthritis Scale, AOFAS Ankle Hindfoot Score, and Foot Function Index in end stage ankle arthritis
.
Foot Ankle Int
33
:
57
,
2012
.
17
Pinsker
E,
Daniels TR: AOFAS position statement regarding the future of the AOFAS Clinical Rating Systems
.
Foot Ankle Int
32
:
842
,
2011
.

Author notes

Publisher's Note: Copyright 2017 by the American Podiatric Medical Association and the American College of Foot and Ankle Surgeons. This article is being published jointly in both the Journal of the American Podiatric Medical Association and the Journal of Foot and Ankle Surgery. Either citation can be used when citing this article.