We were interested to read the article by Maney and colleagues1 published in this issue of the Journal of Oral Implantology. The authors aimed to assess the reliability of implant surgical guides based on soft-tissue models and reported that the accuracy was evaluated using software. I found the article's title to be incorrect and misleading. Moreover, they reported that 22 sites (46.81%) were considered accurate (13 of 24 maxillary and 9 of 23 mandibular sites). As the authors point out in their conclusion, soft-tissue models do not always provide sufficient accuracy for fabricating implant surgical guides.
Reliability (precision) and validity (accuracy) are different methodological issues in researches. Sensitivity, specificity, positive predictive value, negative predictive value, likelihood ratio positive (true positive/false positive), and likelihood ratio negative (false negative/true negative), as well as odds ratio (true results/false results, preferably more than 50), are among the tests used to evaluate the validity (accuracy) of a single test compared to a gold standard.2–4 It is not clear that the reported 22 sites (46.81%) that were considered accurate related to any the aforementioned estimates for validity analysis. Reliability (repeatability or reproducibility) is being assessed by different statistical tests, such as Pearson r, least square, and paired t test, all of which result in common mistakes in reliability analysis.5 Briefly, for quantitative variable intraclass correlation coefficient and for qualitative variables, weighted kappa should be used with caution because kappa has its own limitation, too. Regarding reliability or agreement, it is good to know that for computing kappa value, only concordant cells are being considered, whereas discordant cells should also be taken into account to reach a correct estimation of agreement (weighted kappa).2–4 As a take-home message, for reliability and validity analysis, appropriate tests should be applied.
