Intra- and Interexaminer Repeatability of Diagnostic Peri-Implant Clinical Measurement: A Pilot Study

Calibrating operators to achieve intra- and interexaminer repeatability is an essential component of quality clinical study and interdisciplinary practice when multiple providers are involved in patient care. Several investigations have discussed the factors that might influence periodontal measurements.^1–5  However, although the number of clinical studies on dental implants has increased exponentially over the past years, there is limited information on examiner repeatability for peri-implant measurements.⁶  Repeatability, defined as the closeness of agreement between independent measurements obtained by the same operators with equivalent methodology, contributes substantially to both diagnosis and the course of therapy.^7,8  The repeatability of clinical periodontal measurements plays an important role in the accuracy of the diagnosis, and if imprecise or unreliable, this may jeopardize clinical judgement and treatment outcomes. While prior human studies have traditionally focused on periodontal measurements,⁹  the repeatability of peri-implant measurements is less well understood, often analyzed by translating existing knowledge on probing measurements from the periodontal to the peri-implant field.

There are several differences in terms of the measurements around teeth and implants. Histological evaluation of probing depth (PD) is usually deeper for peri-implant compared with periodontal tissues.¹⁰  For example, the probe tip generally stops within the junctional epithelium (JE) around healthy gingiva and within the coronal portion of the connective tissue for healthy peri-implant mucosa.¹¹  Histological evaluation of clinical PD showed different degrees of penetration depending on the presence of inflammation.^3,10–12  Inflammation strongly influences the depth of penetration, with the probe tip reaching the connective tissue compartment close to the bone in case of peri-implantitis.¹²  Probing peri-implant tissues is a traumatic event that influences the homeostasis of the peri-implant mucosal seal; however, complete regeneration of the JE and establishment of a new epithelial adhesion was observed as early as 5 days after clinical probing.¹³ 

In addition to variability among different examiners, the determination of factors that influence clinical measurements specifically around implants has not been adequately described in the literature. Time between examinations, different examiners, and probing sequence may be potential sources of error. In addition, implant characteristics such as restoration contour, implant diameter, and severity of peri-implant disease may also influence measurement error.

Despite the crucial clinical relevance attributed to peri-implant clinical diagnostic measurements, little is known about their repeatability. Thus, the present pilot study aimed to investigate the intra- and interexaminer repeatability of clinical diagnostic measurements around dental implants as well as any potential factor that might influence its consistency.

This human study was reviewed and approved by the University of Michigan Institutional Review Board for ethical approval of human studies (HUM00124386). Subjects aged between 18 and 85 years, healthy, or with well-controlled systemic conditions (American Society of Anesthesiologists I or II) were included if they presented with at least 1 osseointegrated dental implant with a rough surface and in function for more than 6 months.

Subjects were not considered eligible in case of implant mobility, pain or abscess originating from the peri-implant mucosa, implant malpositioning, pregnancy, or if unable to understand and sign an informed consent. Written consents were obtained from each enrolled subject.

The following clinical variables were investigated: PD, recession (REC), and gingival index (GI). The PD was defined as the linear distance in millimeters from the free mucosal margin to the probe tip. The REC was defined as the linear distance in millimeters from the crown-abutment margin to the free mucosal margin. Positive values were assigned in cases of mucosa receding from the crown margin; in contrast, negative REC values were assigned when the mucosa was located coronally to the crown margin and the abutment was not visible. The above measurements were rounded up when the value was between 2 numbers. The GI was coded from 0 to 3 according to visual signs of inflammation and bleeding, as described by Löe¹⁴ : 0 was used for physiologic healthy mucosa, 1 described visual signs of inflammation without bleeding, 2 was used for localized bleeding after probing, and 3 was used in case of profuse bleeding.

To control the effect of probe type on periodontal measurements,^5,7  all operators used the CP-15 UNC periodontal probe (PCPUNC157; Hu-Friedy Qulix, Chicago, Ill). Every implant was probed at 6 sites (distobuccal, midbuccal, mesiobuccal, mesiolingual, midlingual, and distolingual). The probe was inserted into the peri-implant sulcus with light force until mild resistance was felt.¹⁵  Care was made to maintain contact with the implant surface during apical insertion of the probe. Angled positioning of the probe was allowed in case of overcontoured prostheses. Gentle irrigation with water and/or air was performed to improve visibility in cases of profuse bleeding.

A training calibration session was conducted with 1 patient who was not included in the record. All 4 examiners were present during the training session. First, definitions of PD, REC, and GI were reviewed and agreed upon by all examiners. Then, all examiners measured PD, REC, and GI together on 1 subject until 100% agreement was achieved. Finally, the official calibration session was started.

Four examiners participated in the data recording. Each patient was screened by 3 different examiners to investigate the interoperator agreement and twice from a single examiner to investigate the intraexaminer agreement (Supplementary Table 1). Each implant received a total of 4 turns of measurements. The 2 turns evaluated by the same operator were not consecutive. Measurements were dictated to a dental assistant, who recorded them on a concealed table, unavailable to the examiners. Therefore, all examiners were blinded to measurements recorded by themselves or other operators.

Files of the same patients were then reviewed by 2 investigators to retrieve additional data regarding radiographic marginal bone levels (RMBLs), radiographic angle of restoration contour (RARC),¹⁶  crown height, implant diameter, and demographic data.

Intraclass correlation coefficients (ICCs; 95% confidence interval [CI]) were calculated using ratios of variance estimates from linear mixed models. Mixed linear and logistic regressions were used to assess the influence of variables on inter- and intraexaminer agreement using all pairable measurements. Confidence intervals for ICCs with 95% CIs were determined as the 0.025 and 0.975 quantiles from a parametric bootstrap distribution generated by simulating 10 000 new samples from the underlying linear mixed models. The standard deviations of these bootstrap distributions were also used to estimate standard errors for ICCs. These estimated standard errors were used to formulate standard errors for Wald tests to compare ICCs where needed. All hypothesis tests and P values were 2 sided. When appropriate, the Holm method was used to control the family-wise error rate across multiple comparisons. Analyses were carried out in R version 3.6.0 using the lme4 package. The sample size was based on the comparable study by Merli et al,⁶  which included 27 implants examined by 3 operators. The statistical analysis and methodology were reviewed and approved by an independent statistician (J.H.).

The calibration session was conducted smoothly, and patients were dismissed uneventfully. Thirty-three bone level implants from 12 patients (7 men, 5 women; aged 63.8 ± 10.4 years) were examined, accounting for 804 measurements from 198 sites for each diagnostic variable.

The ICC for overall interexaminer agreement was 0.80 for PD (0.74–0.84; 95% CI) and 0.78 for REC (0.69–0.84; 95% CI), without statistically significant differences between the 2 variables (P = .62). There was significantly lower interexaminer agreement for GI compared with PD (P < .001) and REC (P = .002), with an ICC for GI of 0.45 (0.26–0.62; 95% CI). A comparison of site-specific ICCs did not find statistically significant differences between locations (distobuccal, midbuccal, mesiobuccal, mesiolingual, midlingual, or distolingual) for any of the considered clinical variables, after adjustment for multiple comparisons. Overall and site-specific interexaminer agreements are presented in Table 1.

The overall intraexaminer agreement, calculated using the ICC, showed a reliability of 0.81 for PD (0.69–0.87; 95% CI) and 0.80 for REC (0.59–0.91; 95% CI). Intraexaminer agreement was not significantly different between PD and REC (P = .93) or between REC and GI (P = .08), but PD achieved higher reliability than GI did (P = .03), for which intraexaminer agreement was 0.57 (0.33–0.72; 95% CI). Pairwise analysis of different operators did not detect significant differences for PD or REC but did detect statistically significant differences between examiner 1 and examiner 2 (adjusted P < .001) and between examiner 1 and examiner 4 (adjusted P = .04). The intraexaminer agreements for PD, REC, and GI are shown by operator in Table 2.

The values for PD, REC, and GI averaged 4.38 ± 0.67, −0.30 ± 0.73, and 1.62 ± 0.58, respectively. The values of REC and GI, but not PD, influenced the repeatability of the measurements of the same site (Figure 1). The magnitude of the PD measurements was not associated with PD agreement (slope [95% CI]: −0.008 [−0.056, 0.041]; P = .74; Figure 1A). On the contrary, increasing disagreement was noted for positive REC (OR: 3.0; slope [95% CI]: 0.177 [0.049, 0.284]) as well as negative REC (OR: 4.8; slope [95% CI]: −0.299 [−0.404, −0.193]) as compared with no REC. Higher disagreement was noted for negative REC (tissue overgrowth) than for positive REC (Figure 1B; P = .04). Agreement for GI was influenced by GI measurements (Figure 1C), such that better agreement was observed in case of higher GI (profuse bleeding) and lower agreement was noted for lower GI (tissue health; OR: 4.4; slope [95% CI]: −0.423 [−0.564, −0.294]).

The relationship between GI and PD was investigated using a linear mixed model under the hypothesis that GI increases with deeper PD. GI increased significantly with PD, with a GI that was 0.113 units higher for each millimeter of increase in PD (slope: 0.113 units/mm; 0.066−0.161, 95% CI; P < .001).

Multiple sequential measurements affected the result of repeated readings of PD and GI so that, on average, PD measurements were 0.19 mm (95% CI; 0.11–0.26 mm) deeper and GI measurements were 0.08 (95% CI; 0.05-0.12) higher for each additional examination. Overall, additional readings for PD were 23% deeper and shallower in only 6% of the cases (Figure 2A). For GI, additional readings recorded higher values for 23% of the cases and lower inflammation for 14% of the cases (Figure 2B).

Radiographic marginal bone loss (RMBL; mean: 0.86 mm, SD: 0.98) correlated with PD, and each additional millimeter of RMBL was associated with an increase of 0.172 mm (0.011–0.332 mm, 95% CI; P = .04) in PD. RMBL had no effect on PD agreement (slope: −0.006 mm (95% CI: [−0.134, 0.121]) so that PD agreement was not influenced by the severity of bone loss for that site.

The radiographic angle of the restoration contour (RARC; mean: 152.56°, SD: 16.87°), measured as the external angle between the long axis of the implant and the crown–abutment emergence, was not associated with PD agreement (OR: 0.989; 0.965–1.014, 95% CI; P = .39). Variation in the angle between the crown and the implant did not influence the accuracy of measurements. An angle of 150° was retrospectively used to cluster wide and narrow crowns; regardless of the RARC, there was no association with PD disagreement (OR: 0.551; 0.150–2.018, 95% CI; P = 0.37).

Finally, neither implant diameter (4.58 ± 0.64 mm) nor crown length (10.63 ± 1.97 mm) were significantly correlated with PD agreement (P = .557 and P = .265, respectively).

As explicitly reported in European workshops and during the 2017 World Workshop, probing peri-implant tissues is safe and mandatory to evaluate their state of health or disease.^17–21  However, despite the daily use of probing-based diagnostics, limited data are available regarding the repeatability of peri-implant measurements; available data on inter- and intraexaminer agreement for peri-implant probing measurements are also scarce.^22,23  Therefore, the present pilot study was conducted to assess the repeatability of peri-implant clinical diagnostic measurements and to investigate what factors influenced measurement agreement.

How PD values are associated with measurement inaccuracy has been the subject of discussion in classical studies on periodontal probing, although it is still unexplored for peri-implant measurements. Deeper pockets may potentially lead to more challenging probing and greater interexaminer disagreement regarding PD measurements. Studies on periodontal probing did not find a relationship between PD and repeatability of measurements.^24–27  In agreement with the periodontal literature, the present study documented no association between peri-implant PD and repeatability and suggested that deeper pockets are not at increased risk of diagnostic inaccuracy than shallower ones with trained examiners. Regarding REC, measurement disagreement scaled directly with the magnitude of positive or negative REC, so that higher agreement was found when the free mucosal margin was located at the crown margin, whereas both positive and negative RECs were associated with reduced diagnostic repeatability. In other words, there was more agreement when the mucosal margin was located at the crown-abutment junction and lower agreement existed when operators were asked to estimate the extent of the positive REC or the height of the mucosa covering the crown (negative REC). As for GI, agreement increased linearly for higher GI. Operators were more likely to agree (higher repeatability) when there was severe bleeding, and they were more likely to disagree (lower repeatability) on the definition of erythema or visual health. The idea that indices aimed to quantify gingival inflammation are affected by high subjective components is well described in the periodontal literature.²⁸  To reduce this subjective component, multiple indices have been proposed, such as the Sulcus Bleeding Index,²⁹  Gingival Bleeding Index,^28,30  Papillary Bleeding Index,³¹  Papillary Bleeding Score,³²  Bleeding Time Index,³³  Eastman Interdental Bleeding Index,³⁴  Quantitative Gingival Bleeding Index,³⁵  Modified Gingival Index,³⁶  Modified Gingival Index,³⁷  and Bleeding on Interdental Brushing Index,³⁸  but none of them resulted in higher diagnostic reliability. Because of its universal acceptance and simplicity of use, Löe's¹⁴ GI was used in the present analysis. However, the overall low agreement for GI highlights the need for better technology to measure this important clinical parameter.

The present human study documented a small increase in PD and GI after additional remeasurements. Peri-implant tissues are weak to mechanical trauma, and repeated probing is likely to disrupt the existing adhesion with deeper penetration of the probe tip into the connective tissue, which can explain the increased mean values for PD and GI. A classical study by Etter et al¹³  reported how the probe tip passed the most apical extent of the JE, causing separation of the JE from the implant surface. However, epithelial attachment was completely regenerated after 5 days without irreversible damage noted for the connective tissue compartment. Another study by Lang et al¹²  clarified that apical probe penetration increases for inflamed peri-implant tissues and confirmed that even in health, the probe tip reaches the connective tissue compartment. It is worth mentioning that, while in animal studies it is possible to have experimental categories of health, mucositis, and peri-implantitis, patients included in the present human trial were not prepared with any prophylaxis sessions and presented at the examination with a heterogeneous extent of inflammation.

The present article provided new information on the repeatability of relevant routinely used peri-implant measurements. The increased bleeding in deeper pockets opens the field to interesting speculations on a possible role of the deep transmucosal tunnel on mucosal inflammation.³⁹  Deeper probing after multiple remeasurements also raises the need to reduce tissue trauma during diagnostics through careful and gentle probing.¹³  Despite the clinical significance of the reported findings, the present study still has limitations. Most of the implants included during the calibration sessions were considered healthy with limited PD and low-grade inflammation, which may have a significant effect on measurement results. In addition, external generalizability is limited by the small sample size. Future studies will require larger sample sizes with inclusion of more severe peri-implant disease conditions with various conditions. Finally, particular care should be given for parameters with higher variability, such as lower values of GI and negative REC. Future development of tools to measure these clinical markers should also be developed to improve diagnostic reproducibility.

After assessment of the variability of commonly used peri-implant clinical measurements by multiple operators within the limitation of the present pilot study, it was concluded that (1) inter- and intraexaminer agreement was significantly lower for GI when it was compared with PD and REC, especially for lower values of GI; (2) PD and GI were observed to increase after multiple measurements; and (3) both positive and negative value of RECs were associated with increasing disagreement, especially for negative REC values when peri-implant mucosal tissue was covering the crown margin.

Abbreviations

CI:

confidence interval

GI:

gingival index

ICC:

intraclass correlation coefficient

JE:

junctional epithelium

OR:

odds ratio

PD:

probing depth

RARC:

radiographic angle of restoration contour

REC:

recession

RMBL:

radiographic marginal bone level

SD:

standard deviation

The study was supported by the POM Clinical Research Fund. The authors thank James Henderson, independent statistician and consultant at the Consulting for Statistics, Computing, and Analytics Research (CSCAR), for the statistical support. All authors provided significant contribution for the realization of the study.

All authors do not have any conflict of interests, neither directly nor indirectly, related to the study.