Three Dimensional Analysis of Facial Movement in Normal Adults: Influence of Sex and Facial Shape

Facial appearance during function has a major impact on how a person is perceived in society. Consequently, for individuals with facial functional or movement impairments, methods for analyzing these impairments are useful in diagnosis, treatment planning, and outcome assessment of surgical rehabilitative procedures. For these reasons, researchers have attempted through various strategies to quantify facial functional deficits. Until recently, the only tools available for the evaluation of facial function/movement were based on either subjective scaling assessments1– 3 or two-dimensional (2-D) measurements.4–10 Subjective assessments have the drawback that they are based on scales that are discontinuous and ambiguous11 and, although 2-D measurements are objective, recent work has cast doubt on the validity of such measurements.12–17 For example, researchers have analyzed facial movements in the 2-D frontal (full-face view) perspective only.4–10 From this perspective, movements expressed in the vertical and lateral dimensions should be measured accurately; however, movements expressed in the antero-posterior dimension would be omitted. In addition, it might be expected that, as facial shapes of individuals vary, movements could be expressed to a different extent in each dimension: antero-posterior, lateral, or vertical. If one considers the facial movement during smiling, subjects with narrow faces might be expected to have movements on the lower face expressed in the antero-posterior rather than the vertical or lateral dimensions.

In this regard, we have developed a three-dimensional (3-D) video-based technique that is capable of objective measurements of facial movement and provides a means of evaluating soft-tissue functional problems.12–17 There have been several other 3-D methods proposed,18–20 but these have not been tested or validated in a large clinical population. Furthermore, there is little available information on objective measures of facial motor function in normal individuals. This information would be vital to establish the normal range of variation. Thus, a component of this study will be to collect and characterize baseline data in normal subjects.

Because individuals with facial impairments are likely to have obvious distortions at the extremities of movements due to tissue restrictions (scaring) and compensatory movements of unrestricted tissues,15 we will focus on maximum or border movements. Additionally, although previous studies have alluded to the existence of certain confounding factors, such as sex, that would influence facial movements,10,21 there has been no systematic investigation of any such factor. The aim of this study, therefore, is to quantify 3-D facial movements in a large sample of normal adults and to investigate the influence of possible confounders of sex and facial shape on facial border movements. Our primary hypothesis is that there are differences in facial border movements due to sex. A secondary hypothesis is that there are differences in the expression (direction) of these movements due to differences in facial shape.

The sample consisted of 50 healthy adult subjects (25 males, 25 females) with a mean age of 27.3 years (range: 23–39). The sample size was based on a power (1 − β) of 0.85 to detect a sex difference of 4-mm displacement at P = .05 and SD = 4.6 mm. The standard deviation selected was the largest value recorded in pilot studies.12 Subjects were recruited from the students, staff, and faculty at The University of Michigan School of Dentistry, Ann Arbor, Mich. The inclusion criteria were a willingness to participate in the study, age between 20 and 40 years, and no known facial impairment. Exclusion criteria were the presence of orthodontic appliances and/or facial hair that would interfere with marker placement. Approval for the study was obtained from the Institutional Review Board at the University of Michigan, and informed consent was obtained from each subject before data collection.

A video-based tracking system (Motion Analysis Corporation, Santa Rosa, Calif) was used to record facial animations. This system tracks 4-mm diameter spherical, retroreflective markers attached to specific landmarks on the face (Figure 1).12–15 Under ideal conditions, only 2 cameras are necessary to track a marker in 3 dimensions; however, the 2 additional cameras served as backups in case the markers showed inadequate spatial separation or were carried outside the field of view of the primary cameras. Camera optics consisted of lenses with a focal length of 25 mm. Prior to each recording session, the space where the subject's head was to be positioned was calibrated by way of a cube-shaped metal space-frame (200 mm on each edge) fitted with an array of 12 markers whose positions in space were certified to an accuracy of +7.6 nm by Dimensional Inspection Laboratories (Fremont, Calif). Lens distortion was corrected by means of a translation table provided with each of the lenses, and high definition resolution enhancement techniques were employed on a frame-by-frame basis. Under the conditions of this study, lens distortion, as determined by a 3-cm test object positioned at the center and corners of the measurement space, produced a mean error of 0.53 mm (±0.45 mm). The position of each marker on the patient's face was referenced to the calibration cube, and a tracking algorithm that used a target search area 0.9 nm in diameter was used to estimate the spatial position of each marker. Data from the video cameras were recorded in real time on 4 analogue video recorders for later, off-line digitization and processing. Computations were executed by a computer workstation (Sun Sparc TM, Sun Corporation, Palo Alto, Calif). Off-line digitization of the video data was effected 1 data stream at a time. Channels were synchronized by timing cues stored on all 4 analogue tapes. Each frame was digitized at a horizontal and vertical resolution of 245 × 245 pixels. Data for each of the markers then were stored on hard disk for subsequent analysis.

The subject was positioned with his/her head within the calibrated measurement field. Before data collection, subjects were instructed during a short practice session on how to make each animation. Following the practice session, subjects were requested to perform 7 maximum instructed facial animations that were performed in the following order: smile, grimace, lip purse, cheek puff, eye closure, eye opening, and mouth opening. Each subject repeated an animation 3 times before performing the next animation. Subjects were given ample rest time between animations (approximately 1 minute). The entire tracking session for each subject was approximately 20 minutes. No subject reported concerns of fatigue.

Because landmark movement was recorded relative to the calibrated space frame, any head movement would confound soft-tissue landmark movement. Therefore, to obtain a valid measure of the 3-D soft-tissue landmark movement, 3 stable and widely separated markers were used to control for head movement. These markers were secured firmly to a facebow and then attached to a maxillary occlusal splint that was constructed for each subject. The arms of the facebow were adjusted so that they were away from the lips and rested comfortably between the upper and lower lips in order to ensure the least possible interference during animations. These dentition-supported markers have been demonstrated to be stable throughout facial movements,12 and the movement of the centroid of these markers was subtracted from the movement of each soft-tissue landmark to obtain the true landmark movement. Additionally, in order to standardize head position among subjects, the outer arms of the facebow were adjusted parallel to the Frankfort horizontal (FH) plane. Before further analyses, all faces were oriented on the FH plane.

Vectors of maximum landmark displacement from rest position characterized facial mobility. The displacement of each landmark from its initial rest position (as determined from the median of the first 10 frames of data recorded at rest) was calculated with respect to the calibrated space frame. Then the coordinates of the centroid of the dentition- supported markers were subtracted from that of each skin- based landmark to give the facial soft-tissue movement during each animation. For each landmark, the 3-D coordinates (vectors) at the point of maximum displacement for all repetitions and all subjects relative to the origin or rest position were calculated and plotted together. These vectors of displacement for each marker during each of the 7 animations then were described in terms of Mahalanobis percentiles or scores, which rate the degree of marker movement in terms of both magnitude and direction (see Trotman et al13 for further explanation).13 Those landmarks with Mahalanobis scores approaching unity would have the most pronounced movement.

Because differences in maximum displacement may be due to sex differences in facial size, measuring the mean distance of the landmarks from the centroid of all the landmarks and averaging this distance over all frames and motions obtained an estimate of the facial size of each subject. Then the mean face size for males and females was calculated. The mean face size for males was 50.7 mm (range 46.4–56.4), and the mean face size for females was 50.6 mm (range 45.7–57.0). Thus, there was very little difference in facial size between males and females in this sample; however, to eliminate any residual variance in facial size among subjects, the 3-D displacement vectors were scaled by the mean centroid size for each subject to that of the average over all subjects.

After plotting the 3-D coordinates (vectors) at the point of maximum displacement for all repetitions and all subjects relative to the origin or rest position, the Hotelling's T² multivariate statistic was used to test for significant sex differences in landmark displacement.

To assess the effect of facial shape on facial movement, linear and angular facial measurements were obtained from standardized frontal and profile photographs of each subject recorded at rest and in natural head position. Natural head position was achieved by having subjects look into a mirror that was set at a constant distance and focus on their own eyes. Ten sets of photographs were selected randomly and remeasured 1 week later. The repeat measurements were subjected to an intraclass correlation coefficient (ICC) analysis in order to determine the intraexaminer reliability in recording the measurements.

To provide a summary measure of facial shape, 3 facial indices were calculated based on ratios of the facial measurements (Figure 2) and included (1) bizygomatic width (stz-stz) to face height (stn-stm), (2) mandibular width (stg- stg) to lower anterior face height (nlj-stm), and (3) bizygomatic width to mandibular width (stg-stg). For each subject and animation, the 3-D landmark displacements were calculated. Pearson product moment correlation coefficients then were computed between each of the 3 facial indices and the landmark displacements in the antero-posterior, lateral, and vertical dimensions. Index 1 was found to have the strongest association with movement and was selected as the final measure of facial shape. Because of the large volume of data, subsequent analysis of the relationship between facial shape and the direction of displacement (ie, antero-posterior, lateral, and vertical) was limited to a pair of symmetrical landmarks for each animation (Table 1). The landmark pair that was selected for each of the 7 animations displayed a large, if not the largest, movement.

The 3-D maximum displacement values for each landmark pair then were analyzed using a mixed-model analysis of variance with facial shape, sex, and the interaction between facial shape and sex as fixed effects and subject as a random effect. Sex was included in the model because it could have a significant effect on displacement direction and, therefore, omitting this factor would decrease the precision of the estimates of the effect of facial shape. Because there were no significant differences in symmetry between the landmark pairs, paired landmarks were combined in the same regression. Therefore, no distinction was made between left and right landmarks.

Examples of the male and female Mahalanobis scores for the smile and grimace animations are summarized in Figures 3 and 4 (figures for the remaining animations are available from the authors on request). In order to facilitate a description of where movements occurred during a given animation, landmarks were grouped into upper (landmarks 1–7), middle (landmarks 8–16), and lower facial regions (landmarks 17–30). The eye closure, eye opening, and grimace animations had larger Mahalanobis scores in the middle and upper facial regions. The cheek puff, lip purse, and mouth opening animations had larger scores in the lower face region, whereas the smile animation had larger scores in the middle and lower facial regions. The asterisks above the Mahalanobis scores in Figures 3 and 4 denote those landmarks that displayed significant sex differences in 3-D displacement. These latter results were based on the Hotelling's T² test.

The ICCs for the repeated photographic facial measurements ranged between 0.965 and 0.994. The results of the regression analysis (Table 1) show that, with the exception of mouth opening, the within- and between-subject standard deviation in landmark displacement was small (within-subject = 0.7–1.8 mm; between-subject = 0.0–2.8 mm). On average, the between-subject standard deviations were 70% greater than the within-subject standard deviations; and both standard deviations were much greater for the mouth opening animation. For the effects of facial shape on landmark displacement, the numbers in Table 1 represent the effect on displacement of a 1 standard deviation (SD) increase in the shape index above the mean shape index value. Thus, if an individual had a shape index 1 SD greater than the mean shape index value of the 50 subjects, then during lip purse and eye closure, the 3-D landmark displacement increased significantly by 0.5 mm and 1.1 mm during lip purse, cheek puff, and eye closure. The displacement in only the vertical dimension increased significantly by 0.6 mm, 0.6 mm, and 1.1 mm, respectively, and during lip purse and eye closure, the displacement in only the lateral dimension decreased significantly by 0.6 mm and 0.7 mm. Sex had significant effects on 3-D landmark displacement during lip purse, cheek puff, and eye opening animations, with males having 1.0 mm, 1.2 mm, and 2.8 mm greater displacement than females, respectively. Additionally, sex had significant effects on the direction of displacement in that, during eye opening, displacement was greater for males in the vertical (2.3 mm) dimension only, while during eye closure and mouth opening, females had greater displacement than males in the antero-posterior (1.1 mm) and lateral (1.3) dimensions, respectively. Finally, there were no significant interaction effects between facial shape and sex for any of the landmarks.

In this study, we quantified maximum/border movements of the face in a sample of normal adults. The data revealed that a given animation was characterized by increased movement in specific regions of the face, a finding that confirms the preliminary results of Trotman and coworkers13–15 and Weeden 16 and Mendez.17 Although these present findings are largely descriptive, they emphasize regions of the face that may be targeted in future studies of facial movement. For example, those landmarks with Mahalanobis scores approaching unity would have the most pronounced movement and would be best suited for analysis of movement deficits or abnormalities as well as for the evaluation of changes in movement after surgical reconstructive procedures. The feasibility of such an approach was demonstrated in a recent study in which Mahalanobis scores for landmarks on the circumoral region of patients with different forms of facial functional impairments were calculated and compared with scores for a limited number (n = 5) of normal patients.13 Obvious deficits in the maximum or border movements of these patients were evident. Similarly, the large database compiled in this study will provide a valuable normative sample that can be used to assess region-specific movement deficits in patients.

The main aim of this study was to assess the influence of sex and facial shape on facial movement. The finding that males had greater 3-D displacement than females is similar to the findings of Paletz and coworkers,10 who argued that the greater maximum movements in males may be due to sex differences in facial size.22–27 Our results, however, do not support Paletz's argument given that we found that males had greater displacement after scaling the faces to eliminate the effects of facial size.

The importance of the effect of facial shape as a confounder of facial movement bears directly on the conclusions of several recent studies that were based on measurements of movement in 1 plane of space only,4–10 eg, in the frontal plane. We contend that such an analysis is not sufficiently sensitive to detect movement deficits. In our study, we found that, as the facial shape changed, so too did the magnitude of 3-D displacement as well as the displacement in specific dimensions. For example, as the face became broader, the following changes in displacement occurred: the 3-D movement increased during lip purse and eye closure; the expression of movement in the vertical dimension increased during lip purse, cheek puff, and eye closure; and the expression of movement in the lateral dimension decreased during lip purse and eye closure. One important finding is that sex also had an effect on the expression of movement. The clinical relevance of these findings, however, may be questionable because of the small magnitudes of change. In this regard, 2 important caveats should be noted. First, it is highly probable that the facial shape index employed in this study may not have been appropriate. During various facial animations, the soft-tissue facial contours change (eg, bulge and furrow) in various dimensions due to the activity of the underlying mimetic muscles. At the point of maximum displacement, the facial tissue contours would be very different from that at rest. The facial index that we used in this study was measured at rest and calculated from the 2-D data and therefore may not have been appropriate to capture the effect of facial shape. Second, this aspect of the analysis only included 1 set of paired landmarks for each animation. It is possible that landmark displacement at other facial sites may have been more significantly affected by changes in facial shape.

To illustrate the utility of this method to detect differences in facial movement between normal and impaired subjects, the facial movements of a patient with severe mandibular retrognathia were matched to those of the 50 normal subjects. These results were based on the mean of 3 replications of the smile animation. The landmarks for all subjects were scaled to the initial or start position (signified by the landmark number in Figure 5a,b,c); thus, all faces were scaled to a similar size. The solid line represents the mean direction of movement from rest (rest position is represented by the landmark number) for the 50 normal subjects during the smile animation. The gray ellipses represent the 95th percentiles of movement. We would expect that 95% of the normal motion would be in this gray region. The mean movement of the patient from rest (rest position is represented by the landmark number) is represented by the dotted line. It is clear that the patient's midfacial movements were within the normal range; however, movements of the lower lip corners (numbers 25 and 27) were outside the normal range. Additionally, movements of the upper lip (numbers 21 and 22) were limited in the antero-posterior dimension when compared with the normals. Several interesting questions are generated by observation of these plots: (1) Would the surgical plan for a mandibular advancement in this patient lead to more normal functional movements of the perioral region postsurgery? And (2) would normalization of these movements have an important bearing on future surgical stability? These questions are the subject of further investigations in our laboratory.

Some limitations to this methodology will now be discussed. First, we address the possible effects of skin vs muscle movements of the face. The facial muscles insert directly into the skin of the face. As such, movements of the face are due mainly to the movement of the underlying muscle, and any skin movement can be expected to be minimal compared with the muscle movement.

Second, we will address possible interference to facial movement caused by the markers. The markers used for this study were 4 mm in diameter and constructed of light plastic hollow balls with reflective tape on the exterior. Each marker was fixed to a flat plastic base and weighed approximately 0.080 g. After a short period (10 minutes or so), patients were not really aware of the markers. In later studies, we have changed to 2-mm diameter markers, not because of the decreased weight (0.035 g) but because more markers can be placed in a specific area. Last, the facebow used to account for head movements may interfere with facial movements. Although the facebow is in the area of the perioral region where movements of the lower face occur, it is a very effective method to control for head movements once care has been taken to properly adjust its position. The facebow was positioned away from the lips and rested comfortably between the upper and lower lip to ensure the least possible interference. The stability of the facebow attached in this manner was investigated in one of our early studies.12 In our subsequent approaches to this type of analysis in which absolute measurements are calculated, the facebow has been eliminated in favor of a statistical approach to account for head movements.15 

Individuals with facial disfigurement may have associated functional deficits that may impair facial movement. Many of these patients will elect to have reconstructive or orthognathic surgeries. To assess the severity of the functional deficits and the outcomes of surgical rehabilitation, valid and reliable measurement instruments are needed. A video-based motion analysis system, capable of quantifying facial movements, was used to assess the facial movements in a sample of normal adults and to investigate the effects of sex and facial type (shape) on these movements.

We concluded the following:

1. A characterization of facial movements during specific animations was obtained and specific regions of the face were representative of movement during different animations.

2. Males and females showed differences in maximum facial movements after adjusting for differences in facial size. In general, males had greater movement than females.

3. Facial type/shape had a small but significant effect on facial movement. This effect was more evident when the movements were analyzed in a single dimension rather than in three.

This study was supported in part by grant DE 05215-2151 from the National Institute of Dental Research.