Abstract

The purpose of this study was to identify and quantify the availability of orthodontic literature for evidence-based clinical decision-making (ie, sound clinical studies of etiology, diagnosis, treatment, or prognosis meeting basic methodologic criteria for direct clinical use). This is a first step toward developing online decision analysis systems. A search strategy based on Medical Subject Headings (MeSH) for orthodontics was developed to examine MEDLINE using the Ovid Web Gateway search engine. Sensitive and specific methodologic search filters were then employed to identify the 4 categories of information. The results were then subdivided by year to identify trends and sorted to identify source of publications. In the period 1990 to 1998, the MEDLINE searches identified 6938 English-language articles about orthodontics. The mean number of articles (±SD) per year ranged from 42 ± 25 for specific searches to 314 ± 214 for sensitive searches. The number of articles identified by the specific or sensitive searches increased 14% to 21% annually. When subdivided by clinical category, the mean numbers of articles per year for specific and sensitive searches were respectively: etiology 19 ± 15 and 91 ± 37, diagnosis 11 ± 5 and 80 ± 35, therapy 3 ± 1 and 50 ± 23, and prognosis 10 ± 7 and 93 ± 33. Five dental journals accounted for nearly half of these publications. These results provide several key findings: (1) there is a substantial literature of clinically relevant information in orthodontics upon which to base clinical decisions; (2) the information appears to be balanced between etiology, diagnosis, treatment, and prognosis; (3) approximately 45% of the articles reside in 5 journals, whereas the remainder reside in approximately 66 other journals, making it difficult to stay current; (4) the number of articles is increasing significantly each year; (5) to stay current, one would need to read between 1 and 6 articles per week, 52 weeks per year; (6) these trends suggest the need for computer-based clinical knowledge systems; and (7) the methods used here can be immediately employed to identify the best and most current clinical orthodontic evidence.

INTRODUCTION

With increasing emphasis on the development of evidence-based clinical decision-making,1–4 more focus is being placed on the availability of high quality evidence.5,6 The reason for this is that access to computer-based communication networks and online, critically appraised medical information can potentially improve clinical decision making by increasing information availability.7 Evidence-based clinical decision-making, however, requires the presence of, and access to, a large volume of high quality clinical information.

To identify and verify the availability of clinical orthodontic information for evidence-based clinical decision-making, we conducted a benchmarking study of MEDLINE. We employed bibliometric methods—the analysis of a literature using statistical methods to reveal the historical development of subjects and patterns of authorship, use, and publications.8 Similar methods have been successfully employed for similar purposes in medicine. Bibliometric analysis methods are currently being used by the United States National Academy of Sciences for evaluating research programs9 and have been used for evaluating clinical progress in cardiology,10 audiology,11 mental health,12 epilepsy,13 emergency medicine,14 family practice,15 allied health,16 arthritis,17 and dentistry.4 

The objectives of the current study were to develop and implement MEDLINE search strategies that would access the orthodontic literature, and then to estimate the availability of literature that one can potentially use to make clinical decisions and examine trends. In particular, we were interested in the availability of articles on etiology, diagnosis, therapy, and prognosis. This is a first step toward developing computer-based clinical knowledge systems to assist in clinical decision analysis. We hypothesize that there is a substantial orthodontic literature upon which to make evidence-based decisions.

MATERIALS AND METHODS

Literature search

A comprehensive manual review of the Tree Hierarchy of MEDLINE was performed online (http://gateway.ovid.com/re1410/server2/ovidweb.cgi) to identify the Medical Subject Heading (MeSH) terms and their related subheading describing orthodontics. MeSH terms “orthodontics,” “cephalometry,” and “malocclusion” were deemed most relevant and selected to capture orthodontic journal articles. From this search a vocabulary in orthodontics was developed (Table 1). A literature search was then developed and applied using these terms (Table 2). The search strategy queried MEDLINE from year 1966 to September 1999, week 2. The search was performed using the Ovid Web Gateway (Ovid Technologies Inc, NY, NY) Internet interface for MEDLINE (http://gateway.ovid.com). Several search commands were applied to these subject headings. “Explode” (abbreviated “exp”) was applied to topics “orthodontics” and “malocclusion” for the most comprehensive search, thus all conceptually related subtopics were included in the search. “Explode” was not applied to cephalometry because doing so would retrieve irrelevant subtopics such as “craniometry.” “Focus (abbreviated “*”) was applied to “cephalometery” instead. Table 2 alphabetically lists all 3 orthodontic topics and their subtopics covered by the literature search. To limit the topics only to orthodontics, “Temporomandibular joint dysfunction syndrome” was eliminated, as it is an overlapping topic spanning many dental specialties. This was accomplished using the Boolean operator “not,” as shown in step 6 of the search. The identified literature was then limited to humans, to articles written in English, and from the year 1990 to the year 1998. Validated sensitive and specific methodologic filters were then used to identify 4 clinical categories of information: etiology, diagnosis, therapy, and prognosis (Table 3).18 As used here, a sensitive search indicates a search strategy that retrieves the largest number of relevant articles but also includes some irrelevant ones. A specific search indicates a search strategy that identifies a small number of the most relevant articles, but also excludes some relevant articles and most irrelevant articles. The results of the search were then subdivided by year from 1990 to 1998 to identify trends.

TABLE 1.

Orthodontic Terms Included in MeSH Headings

Orthodontic Terms Included in MeSH Headings
Orthodontic Terms Included in MeSH Headings
TABLE 2.

Orthodontic Search Strategy Based on MeSH Headings (1966 to September 1999 week 2)

Orthodontic Search Strategy Based on MeSH Headings (1966 to September 1999 week 2)
Orthodontic Search Strategy Based on MeSH Headings (1966 to September 1999 week 2)
TABLE 3.

Methodological Filters for Searching

Methodological Filters for Searching
Methodological Filters for Searching

The search results were also stratified to identify dental journals that published the greatest number of articles per year in all 4 clinical categories during the past 5 years (1994–1998). First, for a given year, citations in each clinical category and each search type (for example, 1994 diagnosis and sensitive search) were tallied according to their journal sources. Then, for each clinical category and each search type, the mean number of articles published/year (±SD) were calculated. The journals that published these articles were then stratified from those that published most to least.

Statistical analysis

The data obtained from the literature search in each category were statistically analyzed using InStat 2.01 for Macintosh (Graphpad Software Inc, San Diego, Calif). One-way Analysis of Variance (ANOVA) was performed with Tukey-Kramer corrections for multiple comparisons to compare the sensitive and specific search strategies for the 4 clinical topics. Linear regression and Spearman rank correlation were used to determine differences over time.

RESULTS

The results of the implemented search strategy quantify the availability of orthodontic literature. The data (Table 2) indicate that, over the period between 1966 and September 1999, week 2, there were 34,450 articles published on orthodontics. Of these, approximately 20% or 6938 articles were published between 1990 and 1998 (inclusive) in English and addressed human orthodontics.

To determine the number of articles per year and the distribution of these articles, sensitive and specific methodologic filters were applied (Table 3) to identify articles addressing etiology, diagnosis, therapy, and prognosis. Figure 1 and Table 4 present summary data for sensitive and specific searches for each of the 4 clinical categories over the 9-year period. The mean number of articles published per year in all 4 categories combined ranged from 42 (±26) for specific searches to 313 (±124) for sensitive searches. We further examined the difference between sensitive search (a strategy that retrieves the largest number of relevant articles but also includes some irrelevant ones) and specific search (a strategy that identifies a small number of the most relevant articles, but also excludes some relevant articles and most irrelevant articles). As for individual category, the number of articles per year for specific and sensitive searches ranged between 19 ± 15 and 91 ± 37 for etiology, between 11 ± 5 and 80 ± 35 for diagnosis, between 3 ± 1 and 50 ± 23 for therapy, and between 10 ± 7 and 93 ± 33 for prognosis. Within each category, sensitive searches identified more articles per year than specific searches (all P < .0002, paired t-test). Comparison between the 2 means indicates that, as expected, sensitive search consistently identified more articles than specific search (P < .0001, paired t-test).

FIGURE 1.

Box plots indicating number of articles in 4 clinical categories between 1990 and 1998. The squares indicate the average, while the horizontal lines indicate the 10th, 25th, 50th, 75th, and 90th percentiles. The graphic displays indicate that the sensitive searches identify more articles than specific searches. The diagnosis, etiology, or prognosis categories had significantly more articles than therapy in sensitive search while etiology category had more articles than prognosis or therapy in specific search

FIGURE 1.

Box plots indicating number of articles in 4 clinical categories between 1990 and 1998. The squares indicate the average, while the horizontal lines indicate the 10th, 25th, 50th, 75th, and 90th percentiles. The graphic displays indicate that the sensitive searches identify more articles than specific searches. The diagnosis, etiology, or prognosis categories had significantly more articles than therapy in sensitive search while etiology category had more articles than prognosis or therapy in specific search

TABLE 4.

Mean Number of Articles (± SD) Per Year 1990–1998A. Percentage Indicates the Average Percent Increase of Articles Per Year

Mean Number of Articles (± SD) Per Year 1990–1998A. Percentage Indicates the Average Percent Increase of Articles Per Year
Mean Number of Articles (± SD) Per Year 1990–1998A. Percentage Indicates the Average Percent Increase of Articles Per Year

Comparing publication quantity reveals the relative emphasis placed on different aspects of clinical information in orthodontic literature. Data in Figure 1 and Table 4 were further examined to determine the relative publication quantity of 4 clinical categories. For sensitive searches, the data suggest a publication quantity of (in the decreasing order): prognosis, etiology, diagnosis, and therapy. Statistical analysis indicated that search results of prognosis, etiology, and diagnosis were greater than that of therapy, with no significant difference among the first 3 categories (all P < .001, ANOVA). For specific searches, the data suggest a publication quantity of (in decreasing order): etiology followed by diagnosis, prognosis, and therapy. Statistical analysis indicated that etiology was greater than those of prognosis and therapy (both P < .005, ANOVA), but there was no significant difference between diagnosis, prognosis, and therapy (all P < .05, ANOVA). Overall, the clinical articles on 4 categories are comparable in publication quantity, with no particular order of ranking in quantity. This finding suggests that the body of orthodontic literature is balanced in clinical content.

To better understand the dynamics of information evolution in the 4 clinical categories, the searches were stratified by publication year. Figures 2–5 illustrate the time course of diagnosis, therapy, etiology, and prognosis publications, respectively. The figures confirm the consistency with which sensitive searches identified more articles than specific searches. The figures also suggest that for all clinical categories the number of articles generally increased each year for sensitive and specific searches. Linear regression substantiates this impression in that the slopes were all positive in sensitive and specific searches respectively: 12.1 and 4.8 for etiology, 12.4 and 1.4 for diagnosis, 7.9 and 0.4 for therapy, and 10.4 and 2.4 for prognosis. These slopes were significantly different from 0 for all 4 categories in both sensitive and specific searches (all P < .02, linear regression). Spearman rank correlation of publication number with year (r2), which approximated slopes of liner regression, substantiated the impression that all 4 categories increased significantly each year for both sensitive and specific searches.

FIGURE 2.

The time course of etiology publications indicates that: (1) for all years the sensitive search identified more articles than the specific search; and (2) the number of published diagnostic articles identified by both the sensitive and specific search strategies increased during 1990–1998

FIGURE 2.

The time course of etiology publications indicates that: (1) for all years the sensitive search identified more articles than the specific search; and (2) the number of published diagnostic articles identified by both the sensitive and specific search strategies increased during 1990–1998

From the above results, calculations were made to determine the average percent increase of articles over the 9-year period. The average percent increases by category for sensitive and specific searches were: 13% and 25% for etiology, 16% and 13% for diagnosis, 16% and 13% for therapy, and 11% and 24% for prognosis. Therefore, the composite average percent increase of articles per year (sum of slopes divided by sum of percent increase) was 14% for sensitive searches and 21% for specific searches over the past 9 years.

From a clinical viewpoint, the ultimate goal of clinical activities is to improve the prognosis of a patient's condition(s). In a body of balanced orthodontic literature, publications on prognosis would be expected to constitute (1) approximately one-fourth of all publications, or (2) increase over time as the evidence base of other clinical information grows to support more research on prognosis. To test this hypothesis, we examined the ratio of the number of prognosis articles to the total number of articles in all 4 clinical categories. This was done for the 9-year period. The mean percentage of prognosis articles ranged from 22% (±7%) for specific searches to 30% (±4%) for sensitive searches. Figure 6 illustrates that the number of prognosis articles relative to those of other categories has remained steady. These results fulfill ones expectation.

FIGURE 6.

Time course of prognosis publications with respect to all publications combined. The time course indicates that the percent of published prognosis articles decreased for sensitive search strategy and increased for specific search strategy, but overall remained steady during 1990–1998

FIGURE 6.

Time course of prognosis publications with respect to all publications combined. The time course indicates that the percent of published prognosis articles decreased for sensitive search strategy and increased for specific search strategy, but overall remained steady during 1990–1998

To examine the source of information in the 4 clinical categories, journals that published the articles were identified. Table 5 shows the 5 journals that yielded the greatest mean number of articles per year in all 4 categories combined over the past 5 years in the respective sensitive and specific searches: American Journal of Orthodontics & Dentofacial Orthopedics (81 ± 25 and 14 ± 7), European Journal of Orthodontics (27 ± 8 and 5 ± 2), Angle Orthodontist (25 ± 7 and 4 ± 2), British Journal of Orthodontics (20 ± 3 and 4 ± 2), and International Journal of Adult Orthodontics & Orthognathic Surgery (14 ± 3 and 2 ± 3). These 5 journals totaled 167 (±33) and 30 (±13) articles per year in sensitive and specific searches respectively, which correspond to 41% (±8%) and 48% (±21%) of the mean total number of articles per year; approximately 66 other dental journals published the remainder of all articles.

TABLE 5.

The 5 Journals Publishing the Most Number of Orthodontic Articles Per Year (1994–1998)

The 5 Journals Publishing the Most Number of Orthodontic Articles Per Year (1994–1998)
The 5 Journals Publishing the Most Number of Orthodontic Articles Per Year (1994–1998)

DISCUSSION

The current study was conducted to identify and quantify the availability of orthodontic literature upon which one can potentially make clinical decisions. This is a first step toward developing online knowledge systems for clinical decision analysis. The results indicated that over the last 9 years there is a significant body of clinically relevant articles published in orthodontics.

The results, while informative, have several important implications. First, there are, on average, between 42 (specific search) and 314 (sensitive search) articles published per year addressing the etiology, diagnosis, prognosis, and therapy of orthodontics. Second, if all of these publications are of high clinical applicability, these results also suggest that one would need to read, digest, and implement into clinical practice between 1 and 6 articles per week, 52 weeks per year to keep current. Third, all 4 categories of clinical information significantly increased over the 9-year period. The average increase of articles per year is 14% for sensitive search and 21% for specific search. One can expect this trend to continue. Fourth, there was no strong evidence of relative emphasis given to any particular category of clinical information, pointing to a balanced body of orthodontic literature. Fifth, the evolution of an increasing body of literature predicts a subsequent development of additional diagnostic and therapeutic procedures and potentially the need for additional codes. Just as conceivable will be the parallel development of a combined literature that will enhance prognostics and advanced understanding of disease etiology. Finally, while libraries (personal or public) may subscribe to the “top 5” journals and cover approximately 45% of the clinical information, it is sobering to note that this misses 55% of the literature. This can lead to wide variation in “standards” of care.

This study has several limitations. First, certain relevant articles may have been omitted, while other irrelevant ones may have been included. It was with this issue in mind that the search strategies attempted to “bound” the available literature by using sensitive and specific searches. As expected, a sensitive search, in all 4 categories, retrieved more articles than a specific search. Second, the classification of “orthodontic” articles and their isolation from other specialties was made based on the authors' judgement. For example, “temporomandibular joint dysfunction syndrome” was an overlapping topic in multiple dental specialties. To isolate a potential overlap from articles of other disciplines, articles on temporomandibular joint dysfunction syndrome were not included in this analysis. This should be the focus of another study. Third, the key words used in the MEDLINE search were limited to MeSH vocabulary. The word selection was meant to be inclusive, but it may have excluded some relevant articles. Fourth, the data presented here are probably upper estimates of number of the articles that provide valid, clinically important, and clinically applicable information at a high level of evidence. Further assessments are needed to critically appraise the identified articles. Fifth, the level of evidence needs to be assessed. For example, the Agency Health Care Policy16 and the Centre for Evidence-based Medicine (http://cebm.jr2.ox.ac.uk/docs/levels.html) categorize evidence by quality levels.

In conclusion, while the search strategies, methodological filters, and results demonstrate a substantial and increasing clinical orthodontic literature, this approach to bibliometric assessment may also be useful for information retrieval and stratification. Thus, the next steps are to sample, critically appraise, stratify, and electronically catalog the identified literature to provide an accessible and ongoing electronic database. Such a database will be useful for multiple audiences. These audiences include: patients and clinicians making decisions about clinical care; academics concerned about the evidence base for curricular decisions; researchers interested in identifying gaps in the available knowledge base; corporate entities interested in developing new products; policy makers who fund clinical research; health care purchasers who make decisions about care compensation; and finally for professional societies that seek to provide guidance for their membership.

FIGURE 3.

The time course of diagnosis publications indicates that: (1) for all years the sensitive search identified more articles than the specific search; and (2) the number of published diagnostic articles identified by both the sensitive and specific search strategies increased during 1990–1998

FIGURE 3.

The time course of diagnosis publications indicates that: (1) for all years the sensitive search identified more articles than the specific search; and (2) the number of published diagnostic articles identified by both the sensitive and specific search strategies increased during 1990–1998

FIGURE 4.

The time course of therapy publications indicates that: (1) for all years the sensitive search identified more articles than the specific search; and (2) the number of published diagnostic articles identified by both the sensitive and specific search strategies increased during 1990–1998

FIGURE 4.

The time course of therapy publications indicates that: (1) for all years the sensitive search identified more articles than the specific search; and (2) the number of published diagnostic articles identified by both the sensitive and specific search strategies increased during 1990–1998

FIGURE 5.

The time course of prognosis publications indicates that: (1) for all years the sensitive search identified more articles than the specific search; and (2) the number of published diagnostic articles identified by both the sensitive and specific search strategies increased during 1990–1998

FIGURE 5.

The time course of prognosis publications indicates that: (1) for all years the sensitive search identified more articles than the specific search; and (2) the number of published diagnostic articles identified by both the sensitive and specific search strategies increased during 1990–1998

Acknowledgments

The authors acknowledge the Dean and the Office of Dental Education at the Harvard School of Dental Medicine for providing an environment for this work to proceed. This work was supported in part by grants from NIH/NICDR.

REFERENCES

1
Haynes
,
R. B.
,
D. L.
Sackett
,
J. A.
Gray
,
D. J.
Cook
, and
G. H.
Guyatt
.
Transferring evidence from research into practice: overcoming barriers to application.
ACP J Club
1997
.
126
:
A14
15
.
2
Sackett
,
D. L.
,
W. M.
Rosenberg
,
J. M.
Gray
,
R. B.
Haynes
, and
W. S.
Richardson
.
Evidence-based medicine: what is and what it isn't.
BMJ
1996
.
312
:
71
72
.
3
Richards
,
D.
and
A.
Lawrence
.
Evidence-based dentistry.
Br Dent J
1995
.
179
:
270
273
.
4
Niederman
,
R.
and
R.
Badovinac
.
Tradition-based care and evidence-based care.
J Dent Res
1999
.
78
:
1288
1291
.
5
Greenhalgh
,
T.
How to read a paper: the MEDLINE database.
BMJ
1997
.
315
:
180
183
.
6
Greenhalgh
,
T.
How to read a paper: assessing the methodological quality of published papers.
BMJ
1997
.
315
:
305
308
.
7
Lindberg
,
D. A. B.
and
B. L.
Humphreys
.
Medicine and health on the Internet: the good the bad and the ugly.
JAMA
1998
.
280
:
1303
1304
.
8
Young
,
H.
ALA Glossary of Library and Information Science.
Chicago, Ill: American Library Association
.
1983
.
9
Griffiths
,
P. A.
,
B. M.
Alberts
, and
P.
Diamond
.
et al.
Evaluating Federal Research Programs.
Washington, DC: National Academy Press
.
1999
.
10
Yosipovitch
,
G.
and
B.
Belhassen
.
Bibliometric study of cardiology in Israel in 1978, 1983 and 1988.
Isr J Med Sci
1991
.
27
:
234
238
.
11
Lutman
,
M.
Bibliometric analysis as a measure of scientific output.
Br J Audiol
1992
.
26
:
323
234
.
12
Adams
,
C. E.
,
A.
Power
,
K.
Frederick
, and
C.
Lefebvre
.
An investigation of the adequacy of MEDLINE searches for randomized controlled trials (RCTs) of the effects of mental health care.
Psych Med
1994
.
24
:
741
748
.
13
Marson
,
A. G.
and
D. W.
Chadwick
.
How easy are randomized controlled trials in epilepsy to find on Medline? The sensitivity and precision of two Medline searches.
Epilepsia
1996
.
37
:
377
380
.
14
Singer
,
A. J.
,
C. S.
Homan
,
M. J.
Stark
,
M. C.
Werblud
,
H. C. Jr,
Thode
, and
J. E.
Hollander
.
Comparison of types of research articles published in emergency medicine and non-emergency medicine journals.
Acad Emerg Med
1997
.
4
:
1153
1158
.
15
Van der Weijden
,
T.
,
C. J.
Ijzermans
,
G. J.
Dinant
,
N. P.
vanDuijn
,
R.
de Vet
, and
F.
Buntinx
.
Identifying relevant diagnostic studies in MEDLINE. The diagnostic value of the erythrocyte sedimentation rate (ESR) and dipstick as an example.
Fam Pract
1997
.
14
:
204
208
.
16
Schloman
,
B. F.
Mapping the literature of allied health: project overview.
Bull Med Libr Assoc
1997
.
85
:
271
277
.
17
Lewison
,
G.
and
M. E.
Devey
.
Bibliometric methods for the evaluation of arthritis research.
Rheumatology (Oxford)
1999
.
38
:
13
20
.
18
Haynes
,
R. B.
,
N.
Wilczynski
,
K. A.
McKibbon
,
C. J.
Walker
, and
J. C.
Sinclair
.
Developing optimal search strategies for detecting clinically sound studies in MEDLINE.
J Am Med Inform Assoc
1994
.
1
:
447
458
.