Comparison of the accuracy of digital face scans obtained by two different scanners:An in vivo study

In recent years, the use of three-dimensional (3D) imaging techniques has increased considerably in the medical field, including the disciplines of orthodontics and maxillofacial surgery. Orthodontic applications for 3D analysis include the study of dentoskeletal relationships and overall facial esthetics, as well as the morphology of hard and soft tissues.^1,2  Instrumental 3D analysis has evolved from computed tomography, which first appeared in 1972, to cutting-edge 3D laser scanning.^1,3  The literature increasingly reports that 3D facial scanners can be used in the dental field, demonstrating a high degree of precision and accuracy.⁴  Recent innovations in 3D facial analysis have led to the development of noninvasive techniques based on optical properties and digital processing,^5,6  as in the field of 3D facial scans. Studies^2,7  reported less than 1 mm in overall linear error, as compared with 2 mm of error found in another study.⁸  Interestingly, the studies compared scans made with the same scanning system. Facial scanning is a high-resolution, economical, fast, and repeatable technique for capturing the outer surface of the face that allows manipulation of the scan in all directions. The results of treatment can be compared with the pretreatment records, especially in children, in which routine radiology examinations are ethically questionable.^1,6,8–11  The scans also allow integration of 3D analysis obtained from computed tomography and superimposition of images of the hard tissues onto scans of the facial surface.¹ 

The two facial scan systems compared in this study employed different principles. The Dental Pro application (Bellus 3D, Inc, Campbell, Calif) used the TrueDepth camera technology built into Apple devices, and the frames obtained were integrated with the creation of a complete 3D model of the face with up to 500,000 points. The Face Hunter (Zirkonzahn, Gais [BZ], South Tyrol, Italy) scanner used the structured light system, projecting a light pattern on the model, analyzing the deformation of the light on the surface to derive the geometry of the object.

Because of the great evolution in 3D techniques since the advent of digital technology, there is now a need to determine the accuracy and reproducibility of scanned models to reliably use facial scanning as a clinical tool for diagnostic evaluation.^12–15  Hence, the purpose of this study was to compare the degree of accuracy of facial scanners as routine diagnostic tests. The facial scanner was compared with manual measurements, and the effectiveness of the two different scan systems were compared: a scanner established in the scientific world (Face Hunter) was compared with the data obtained with the images of a scanner coming from the Dental Pro application.

After the approval of the institutional review board of the Postgraduate School of Orthodontics of the University of Ferrara and the informed consent release, 25 patients, 11 men and 14 women, between the ages of 25 and 48 years, were measured manually and scanned using each device. The inclusion criteria were Caucasian subjects who had finished growing and were older than 25 years but not older than 50 years. Men with beards and subjects with scars, facial esthetic surgery, or skin blemishes were excluded.

The scanners to be tested were the Face Hunter facial scanner and the Dental Pro facial scan application. The Face Hunter facial scanner allows for planning the work of the dental technician and dentist through the use of the patient's real physiognomy. It uses a Dell M318WL projector and Basler ac780 and ac1600 cameras and has a scan speed of 0.3 seconds. The Bellus 3D Dental Pro app uses the TrueDepth camera built into Apple devices running iOS 12.2 and later for iPad Pro and iPhone X to scan and render a subject's 3D face in less than 15 seconds to capture 3D measurements from many different directions. Once the scan is complete, the Bellus 3D dental function automatically generates the lip line and co-registers the 3D facial scan with up to 500,000 points, creating a high-resolution 4K texture map.

Six reference markers were placed on each subject's face at the cephalometric points using a special cross-shaped mold. The distances between the Tr–Na′, Na′–Prn, Prn–Pog′, and left–right Zyg points were manually measured using a digital caliper (Juning Caliper) as a reference (Figure 1). All subjects were made to sit on a chair against a backrest, so as not to allow backward or forward motion of the torso and head, maintaining the correct natural position of the head.^16–18  A careful, quality-control assessment was performed to verify the differences in head posture or facial expression that could bias the measurements in this study.

The first scan was made using Bellus 3D's Dental Pro app, downloaded via the Apple app store. To perform the scan, each subject held up the phone with their dominant hand, making sure that their arches were always in occlusion. The precise distance between the subject and the phone and the exact tilt of the head were automatically corrected by the application, which indicated correct positioning via the appearance of a green oval around the face on the screen and incorrect positioning or distance via a red oval. Next, a robotic voice guided the subject to move their head.

The second scan was made using Zirkonzahn's Face Hunter scanner. All subjects were made to sit on a chair with a backrest at an arm's length from the scanner, maintained throughout the scan. The operator correctly positioned the patient in the direction of the scanner camera, which reproduced the image on the computer screen so that the correct positioning of the subject could be ascertained. Five static facial scans were performed with occluded arches: one from the forehead, one from each side, and left and right 3/4 profiles; the reference markers in these scans were subsequently lined up by the technician and processed by the software into a single 3D scan.

Digital measurement software was used to calculate the distances between the cephalometric reference points on each of the scans, and sets of measurements were compared with each other and the manual reference.

To verify the percentage of the surface of the areas that coincided in the two scans of the same subject, they were uploaded to Geomagic X Control software (3D Systems Inc, Rock Hill, SC; Figure 2). The program was used to superimpose the scans, automatically determining the best-fit alignment by means of the preestablished reference points, and to calculate the percentage of overlapping surfaces within the tolerance ranges as follows:

• 0.5 mm to 0 mm and 0 mm to −0.5 mm (highly reproducible)

• 1 mm to 0.5 mm and −0.5 mm to −1 mm (moderately reproducible)

• 1.5 mm to 1 mm and −1 mm to −1.5 mm (poorly reproducible)

• >1.5 mm and <−1.5 mm (not reproducible)

The following statistical analyses were performed on the data set: the intraclass correlation coefficient (ICC) index, which, if greater than .9, indicated whether the measurements were consistent and reproducible; t-tests for paired samples to indicate whether the two samples were comparable in terms of means; and Bland–Altman¹⁹  plots to graphically assess the replicability of the measures. Finally, the tolerance values as a function of the five ranges and the face side were modeled using repeated-measures analysis.²⁰  The resulting model was used to obtain marginal means and corresponding confidence intervals.²¹  The analyses were performed using the R Statistical Software, and statistical significance was assessed using a threshold for type I error of α = .05.

Table 1 shows the number of subjects exhibiting the different margins of error in the different cephalometric measurements compared with manual values. The measurements of the distances in millimeters between the cephalometric points obtained via digital orthodontic caliper directly on the subjects' face were compared with those measured on 3D models obtained from the scans by the Dental Pro application and the Face Hunter facial scanner using dedicated measurement software. Individual comparisons of the four distances measured anthropometrically and on the scans, namely, trichion–nasion, nasion–pronasion, pronasion–pogonion, and left–right zygomatic points, all yielded an ICC index greater than .9.

From the results of the t-test for matched samples, it was possible to compare the averages obtained for the different distances directly on the face with the averages of the same distances measured on the scans. In most comparisons, the t-test yielded a P value below the significance threshold (P = .05); therefore, average distances were not statistically different from those measured via digital calipers, with the exception of the trichion–nasion distance measured by the Dental Pro and the nasion–pronasion distance on the Face Hunter scans (Table 2).

The Bland–Altman plots (Figure 3) revealed that the average values of both sets of scanned measurements fell into a relatively narrow range, with few exceptions (ie, the points lying outside the confidence bands that were generally at the extremes of the measurement ranges). As shown in the side-by-side plot pairs, the Face Hunter scanner generally had smaller confidence intervals than those of the Dental Pro application. In fact, the latter often had one or more values well above the threshold of the confidence bands, as illustrated in Graph AA, in which the average trichion–nasion distance was within a range of about −2 to 3 but one of the values was 6.

Percentage overlap of scanned surfaces with the five ranges of tolerance are shown in Table 3. The marginal mean and the corresponding confidence intervals (lower CL and upper CL) between the two scans are summarized in Table 4.

Because of the evolution in 3D scanning techniques, scanners may be a viable alternative to the classic 2D data collection modes, saving time and avoiding exposing patients to high doses of radiation for diagnosis and follow-up.^1,9,10  However, before facial scanning can be reliably recommended as a routinely diagnostic and clinical assessment tool, it is necessary to determine the accuracy and reproducibility of scanned models.^12–15 

In this regard, the t-test analysis for all eight distances (4 × manual vs Dental Pro and 4 × manual vs Face Hunter) yielded statistically significant P values (ie, P > .05), meaning that the three data sets were essentially comparable. The only exceptions to this were the trichion–nasion distance measured by the Dental Pro and the nasion–pronasion distance on the Face Hunter. However, this finding was in agreement with the results obtained by Aung et al.,¹²  who found that distances related to the trichion were less reproducible; indeed, the trichion is a cephalometric point located at the hairline, which can make this area difficult to capture for scanners, leading to a poorly defined image and probable measurement errors. Nevertheless, although the two measured trichion–nasion distances were statistically different, this difference was not clinically relevant. The discrepancy in the nasion–pronasion measurement was neither statistically nor clinically significant. This finding was supported by the conclusions of the study by Hajeer et al.,²  who proposed seven cephalometric points that have proved to be reproducible in superimposition of two scans of the same subject, including nasion. That study²  showed that not only was nasion a reproducible point with a standard deviation of less than 0.5 mm but also that it could be defined as a stable cephalometric point over time and after maxillofacial surgery.

As for the individual distance measurements obtained from the scans, Bland–Altman plots showed that both scanners yielded values that were generally within a confidence range not exceeding 2 and −2. The exceptions were a few Dental Pro scanner values that fell beyond the confidence range, reaching measurement errors of 6. In other words, it was clear that the margin of error of the Face Hunter in a sample of 25 subjects was lower and more constant than that of the Dental Pro, which yielded only a few values that deviated from the average range. This, however, may have been ascribable to the small size of the sample or to external environmental factors that were not predictable at the time of the scans.

The analysis of superimpositions revealed the presence of areas that were more reproducible than others. The right and left cheeks had the highest average value (almost 60%) of the surface in the highly reproducible category. The tip of the nose, despite having lower values than the cheeks in the measuring range 0.5 mm to −0.5 mm, was the area that had values lower than the tolerance limits, reaching more than 80% of the surface in the highly/moderately reproducible category, and 16 of 25 subjects reached 100% of the surface within the tolerance limits. In agreement with the results obtained, Eidson et al.⁴  superimposed facial scans on the areas considered stable: the intercanthal region, dorsum of the nose, temporal region, and upper zygoma zones, which showed a mean difference between the two images of only 0.14 mm. The better result obtained could have been due to the different scanning system, the 3dMDface stereo camera system, but it should be noted that the superimposed scans in that study were made by the same scanning system, whereas, in the current study, two different systems were compared.

The forehead area was poorly reproducible in most subjects, as opposed to the assumption in the early stages. In fact, only an average 11.08% of the surface fell within the range from 0.5 mm to −0.5 mm (Table 4) and, in eight subjects, the value of this range was about 0% (Table 3). It is likely that, despite being an extremely smooth area, its reproducibility was conditioned by facial expressions and strongly influenced by the position of the head during the scan.^6,17  These results were in line with findings by Toma et al.⁷  showing that the most reproducible points were those related to the lips. Those that were poorly reproducible included the glabella and the upper eyelid point. In fact, that study led to the conclusion that there was great variability of reproducibility among points; those placed on well-defined edges showed a higher degree of accuracy than those on curved surfaces.

These results contrasted with the findings of Kau et al.¹³  in their study comparing facial scans over time. Superimposing three scans revealed that, despite never exceeding a margin of difference of 1.35 mm, in most subjects, the greatest errors were detected in the lower third of the face. Kau et al.¹³  argued that this occurred because the jaw was the only moving bone, joined to the skull by soft tissues, which were subject to contractions and modifications. In this study, however, the chin and the lower part of the cheeks were found to be highly reproducible in about 50% of the subjects on average. This discrepancy may be explained, first, by the fact that the time between the two scans was extremely short (about 1 minute), and, second, that subjects were advised to keep the arches in occlusion before performing the scan.

With reference to the position of the head, the authors confirmed the importance of maintaining positions during scanning because, if a subject changes posture, the soft tissues are contracted or relaxed, leading to major changes in facial morphology. However, while immobilizing the head and preventing its movement in the three planes of space would certainly improve the quality of the scans, it would not prevent changes in facial expression and therefore apparent morphology.^4,10,18  On the other hand, movement of the head was inevitable, as the scanners tested were actually fixed and the subjects had to rotate their heads to capture the different views of the face.

The limits of this study were that the age of the subjects was narrow and lay subjects participated; therefore, the study did not include patients undergoing orthodontic or surgical treatment. Future studies should evaluate the effectiveness of facial scanners as diagnostic tools for monitoring soft tissue healing over longer time intervals in patients undergoing orthognathic surgery.

• Three-dimensional scans of the facial surface provide an excellent analytical tool for routine clinical evaluation.

• The Face Hunter facial scanner and the Dental Pro facial scanning application reproduce 3D subjects with comparable, consistent, and reproducible measurements as compared with manual measurements, and it does not appear that one or the other of the measuring tools is systematically more accurate.

• The comparison of the overlays in the areas of the scans shows that the cheeks are the area with the highest average percentage of surface in the highly reproducible range (about 60%), followed by the chin and the tip pf the nose. The forehead was the only area in which most of the surface falls within the poorly reproducible range, with more than 20% of values out of tolerance.