Knowledge-Dependent Pattern Classification of Human Nasal Profiles

Facial appearance is a key to our instantaneous but comprehensive recognition of another individual.1 Conversely, the extent of recognition of an individual's own face by others exerts a great influence, sociopsychologically, on that individual's sense of acceptance by his or her community. The size and shape of the eyes, mouth, and nose are major elements in the recognition of a face. Because the nose is located in the center of the face, it serves, together with the lips and the chin, to characterize the facial appearance unique to each individual.2 

The degree of prominence of the nose, in combination with the anteroposterior position of the chin, has been used clinically as an index for judging the degree of convexity or concavity of the lips.3 In developing an orthodontic treatment plan for those malocclusions involving the upper incisors, evaluation of the inclination of the nasal base is indispensable in considering whether orthodontic movement of the incisors alone (camouflage) or a combined surgical orthodontic approach will more precisely obtain an improved posttreatment facial appearance. Based on these clinical requirements, a systematic method for the evaluation and classification of morphological traits of nasal profiles would be of great value in orthodontic diagnosis.

So far, however, no standard objective criterion for classifying the human nasal form has been developed, although a qualitative classification based on subjective visual judgment has been reported.4 The elements of a feature classification system will significantly influence the results of classification. In the discipline of engineering, techniques are available to extract sets of linear elements sampled from the contour line of the targeted object, but whether such techniques would be effective in classifying human nasal shapes has remained untested. In addition, a technique for modeling the feature extraction process that takes into account the knowledge and thought processes employed by oral health experts in linguistically describing morphological traits of the nose has yet to be established.

The purposes of the present study were (1) to determine feature vector elements (geometric pattern parameters) that are valid for describing human nasal profiles, (2) to determine the number of code vectors mathematically optimized by applying the vector quantization method to each feature vector extracted in each individual subject, and (3) to describe the morphological characteristics of each code thus determined.

Two hundred Japanese women in the prediagnosis stage (mean age, 26 years 9 months; range, 18 years 0 month to 52 years 8 months) who had visited the university dental hospital between 2000 and 2005 were selected consecutively from the patient database in order of their dates of registration at the hospital. All had permanent dentition, and none had any facial congenital anomalies or history of surgery or trauma to the face.

Lateral facial photographs (right side views) taken with the head fixed by ear rods and with the Frankfort horizontal plane parallel to the ground were employed. The recordings had been made with a digital camera equipped with a 2.3-megapixel effective CCD sensor (FinePix 2900Z; Fuji Film, Kanagawa, Japan) and a 70-mm telescopic lens and with a recording distance of 110 cm between camera and patient. The patient was positioned with the teeth in the habitual maximum intercuspation position and the lips in repose. A ring strobe was employed as a light source. Proportions of the subjects in terms of sagittal skeletal classification5 as determined from standard digitally recorded lateral head films of each patient were skeletal class 1, 49.5%; skeletal class 2, 30.0%; and skeletal class 3, 20.5%.

An orthodontist with 23 years of experience performed the visual inspections. Each facial record was displayed on a 17-in. LCD monitor (1701FP; Dell Inc, Round Rock, Tex) scaled down to 75% of the actual size.

A set of multiple points that constitute the contour of the facial profile segmented between the forehead and the inferior part of the chin on the facial image was extracted automatically using customized software.6 A set of x and y coordinates located between the y coordinates of se and sn (mean, 258 points; range, 201–420 points) was used as contour data. Positions of soft tissue landmarks7 (Table 1) were identified by visual inspection of the image and registered using the computer's mouse and cursor.

The Frankfort horizontal plane has been considered useful for characterizing the contour of the soft tissue of the face.8 It is difficult to define the Frankfort horizontal plane on standard facial photographs, however. In a pilot study, it was confirmed that the line connecting the porion and the geometric center (g) of po, sn, and ex on photographs closely approximated the Frankfort horizontal with a mean difference between the two lines of approximately 3°. Consequently, we employed the po-g line as the horizontal reference line for establishing the coordinate systems in the measurement of the soft tissue facial profile. Sellion was defined as the origin of the feature system. The x-axis was defined as a line parallel to the po-g line and passing through the origin. The y-axis was defined as a line perpendicular to the x-axis and passing through the origin. The positions of the soft tissue landmarks and the contour data were defined mathematically and normalized with respect to the difference in y coordinate values between sellion and subnasale.

The image of the facial profile was equally divided vertically into 24 segments to employ horizontal coordinate values for each segmented profile contour as vector elements, which comprised V^(we) (Figure 1; for details, see Appendix 1).

This method employed as vector elements detailed anatomic characteristics of the nose provided by oral health experts. Twenty-nine descriptive parameters expressed linguistically t (t = 1, 2, …, 29) used for classifying nasal shapes were collected from a panel of three orthodontists who provided judgments based on their experience and knowledge (Table 2).

From the linguistic descriptions of the nasal shapes, 21 mathematical descriptions (ie, vector elements vk [k = 1, 2, …, 21] and subsets of the vector elements) were defined (Table 3, Figures 2-1, 2-2, 2-3, and 2-4).

Single-vector elements were assigned to each of the 4 kinds of linguistic descriptions (t = 9, 10, 11, 24). Two subsets of vector elements existed for the 5 kinds of descriptions (t = 7, 8, 23, 25, 29). For each of the three kinds of linguistic descriptions (t = 15, 16, 27), a single-vector element and a single subset of vector elements were assigned. For the seven kinds of descriptions (t = 1, 2, 3, 4, 5, 6, 26), a single-vector element and two subsets of vector elements were assigned. For the nine kinds of descriptions (t = 12, 13, 14, 17, 18, 19, 20, 21, 28), two vector elements and a single subset of vector elements were assigned.

Feature vectors that designate the nasal profile were generated using the vector elements v1, v2, …, v21 or their combinations, with the following conditions as presumptions:

• Condition 1: A single-vector element or a subset of vector elements adequately fulfills the condition to form a single linguistic description.

• Condition 2: Vectors that feature the lateral nasal contour should include a single-vector element or a single subset of vector elements that corresponds to every linguistic description.

• V^(w) (w = 1, 2, …, 6) that feature the contour of the lateral nose were thus determined. Figure 3 shows the feature vectors that meet both conditions 1 and 2.

The feature vector sets V^(we) and V^(w) (w = 1, 2, …, 6) were computed for all facial records.

To determine the number of nasal profile patterns that are mathematically optimized, the vector quantization (VQ) method on the basis of the generalized Lloyd algorithm9,10 was applied to the feature vector sets V^(wx) (wx = we, w) with convergence number N (N = 3, 4, …, 13), and code vectors V^*_N(i) (i = 1, 2, …, N) were obtained. The optimum number of codes, Nopt^(wt), was defined and computed as described in Appendix 2 of this report.

To optimize the feature vector generation method, a digital facial image d (d = 1, 2, …, 200) in actual size was displayed on a computer monitor, and an examiner j (j = 1, 2, 3, the panel of three orthodontists described earlier) separately categorized the shapes of the nasal profiles into one of three classes (“matched,” “not matched,” or “neither”) in light of the previously described linguistic description t (t = 1, 2, …, 29, shown in Table 2), to which judgment scores of 1, −1, and 0 were assigned.

The judgment score of the classified nasal form corresponding to the code vector V^*_Nopt(wx)(i), JS(wx, i)[t], was also calculated. (For detailed computational procedures used in calculating the judgment scores, please see Appendix 3.)

Whether judgment scores {JS(wx, 1)[t], JS(wx, 2)[t], …, JS(wx, Nopt^(wx))[t]} showed significant differences was tested for each wx using the one-way analysis of variance (ANOVA). A matching score S(wx, t) was given by

The matching score for the feature vector generation method wx, S(wx), and the highest matching score S_highest were determined. (Also, see Appendix 3.) The wx that achieved the highest matching score S_highest was defined as the optimum feature vector extraction method wopt. Mean nasal profiles were reconstructed by averaging the approximated curves connecting se and sn for each code vector obtained by classification with the convergence number N = Nopt^(wopt).

A one-way ANOVA was performed to examine whether there was a significant difference between sets of records categorized in each code for each element of the feature vector extracted (P < .01). In addition, the Tukey-Kramer test was performed to determine the codes that showed significant differences between vector elements (P < .01).

The matching score S(we) computed for the feature vector V^(we) and the matching scores S(w) for the V^(w) (w = 1, 2, …, 6) are shown in Table 4. S(we) was found to be 3, whereas S(w) showed values equal to or greater than 9. When w = 2, the maximum matching score was achieved. Because S(2) = 14 and S(we) = 3, respectively, we obtained S(2) > S(we) and, consequently, wopt = 2. In other words, the optimum feature vector V^(wopt) was found to be an eight-dimensional vector whose elements were v8, v9, v10, v11, v12, v13, v14, and v15. This is equivalent to saying that the eight-dimensional feature vector thus determined is an optimum knowledge description that distinguishes among nasal profiles of Japanese women most effectively in accord with linguistic descriptions t = 1, 2, …, 29, descriptions that are assumed to reflect human knowledge and judgment.

Figure 4 shows mean intercode vector distances D(n) determined for the optimum feature vector V⁽²⁾ by applying the VQ method with the number of classes, n = 3, 4, …, 13. The number of codes optimized mathematically was found to be six. In other words, six representative nasal profile patterns were found for the current sample. Proportions of the number of vectors classified into each code were 25.5%, 24.5%, 21.5%, 15.0%, 10.0%, and 3.5% for codes 1 to 6, respectively.

Mean nasal profiles corresponding to each of the six code vectors are shown in Figures 5-1 and 5-2. Code vectors that were expressed in the real space are given in Table 5. With regard to vector elements of the feature vectors extracted, significant differences were found between records of six codes for all elements. Table 6 gives comparisons between codes for each vector element.

Morphological traits for each code are summarized as follows. The code 1 nose is characterized by the upper part of the dorsum descending straightforwardly and the middle part of the dorsum concave in appearance, curving down and posteriorly, with a pointed tip and gently sloping base. The code 2 nose is characterized by a concave middle part of the dorsum with a sharp pointed tip and upturned base. The code 4 nose has a moderately prominent upper dorsum that descends straight, accompanied by an upturned base. The code 5 nose has a prominent upper part but is straight in the middle and lower parts of the dorsum, with a relatively rounded tip and moderately flat base. The code 6 nose is characterized by a straight dorsum and rounded tip. Code 3 represents an intermediate pattern characterized by a curved dorsum with moderate downward deflection.

Nasal profiles have been evaluated quantitatively using specific linear and angular measures,^11-13 but the anatomic sites evaluated have been limited and fragmental. In contrast, studies using a Fourier series approximation14–16 have extracted morphological traits of the nose expressed in terms of a set of amplitude and phase coefficients. These studies are important in the sense that they have quantified the overall shape of the nasal profile, but, regrettably, they have neglected to describe subtle but often important differences in nasal shapes. In addition, their findings did not take into account the intuitive perception of the nasal profile because they were not based on the knowledge and perceptual patterns shared by humans and used in recognition of facial forms through visual inspection.

In recent years, automated pattern recognition systems that employ a process similar to that used by the human brain have begun to reach a level of practical use.17,18 Generalized methods for disassembling information such as principal component analysis and discriminative analysis have been tested,17,19 but it is well accepted that finding properties inherent in the data by understanding the nature of the information or the subject of recognition is more likely to obtain good results.17 

The present study demonstrates that the feature extraction processing method, which takes into account the knowledge and experience of human experts to linguistically describe morphological traits of the nose, is more effective in classifying nasal profiles than the simple mechanical sectioning method. In the present study, we also investigated mathematically which of the six knowledge-dependent methods was the optimum (ie, the best feature vector extraction method). We found that the method that employed a combination of the eight kinds of knowledge was the one that best reflected the experts' knowledge and experience.

This is the first report to classify human nasal shapes objectively and quantitatively. The current sample of nasal profiles was found to be divided into six code vectors (patterns) according to morphological similarity, whose number was determined to be the most stable mathematically. In other words, it was found that the morphological differences between codes were maximized at the root, dorsum, tip, and base of the nose.

Direct comparison of the current results with previous reports on nasal profiles is difficult because of the methodological difference in classifying shapes of the nose, but the proportions of various types of dorsum shape may be compared. According to a previous report,20 documenting the shapes of the nasal dorsum on the basis of examiners' subjective judgments, the nasal dorsum can be classified into three subtypes, the concave, the convex, and the straight, with proportions of 33%, 37%, and 29%, respectively. In the present study, given that code 1, code 2, and code 3 are categorized as concave type; code 4 and code 5 convex type; and code 6 the straight type, ratios for those subtypes would be 71.5%, 25%, and 3.5%, respectively. Such differences in the proportions of subtypes may be due to racial variations. It is anticipated that the classification method we have developed will facilitate exact and objective comparisons of nasal and other facial forms between and among different racial groups, genders, and age groups.

The objective standard for classifying nasal profiles established in the current study is clinically relevant. The shape of the nose is known to be altered by the upward rotation of its dorsum, development of the prominence of the dorsum, and the downward rotation of the nasal base.21 Both code 2 and code 4 share the common trait of an anteriorly and upwardly tilted base of the nose. In addition, with code 4, the nose exhibits a rounded tip. In planning an impaction and forward repositioning of the maxilla by means of a LeFort I osteotomy in patients with such nasal profile characteristics, as well as skeletal anterior cross-bite due to a retro-positioned maxilla, the treatment plan should be designed carefully because an osteotomy may lead to superior displacement of the nasal tip22–24 and a resultant more upward deflection of the base of the nose.25 

When upper premolars are extracted to enable retraction of upper incisors palatally, the upper lip may retract following the reduction of the overjet, which sometimes results in elongation and posterior inclination of the naso-labial sulcus, the so-called dished-in-type facial profile. In such a case, subnasale may be displaced posterio-superiorly to alter the cant of the base of the nose horizontally. In patients having a moderately flat base of the nose as with code 1, code 3, and code 5, excessive retraction of the upper incisors is not recommended. In contrast, in patients with the code 2 or code 4 nose, which has a steeply sloped base, the retraction of the upper anteriors may be recommended because the cant of the base is anticipated to become gentle. When planning orthopedic inhibition of forward maxillary growth in young patients with Class II division 1 malocclusion and a code 2 or code 4 nasal profile, the opportunity for enhancing the steep inclination of the nasal base may be decreased. On the other hand, orthopedic stimulation of growth or surgical forward displacement of the maxilla in children with a code 2 or code 4 nasal profile and a skeletal Class III malocclusion primarily due to recessive growth of the maxilla may cause the base of the nose to be more anteriorly tilted upward after treatment.

In summary, morphological evaluation and classification of nasal profiles of each individual patient on the basis of the current prediction model should allow practitioners to develop more precise orthodontic treatment plans, taking into account possible posttreatment changes in facial profile.

• The feature extraction method of classifying nasal profile patterns, based on knowledge from a linguistic description, was found to be more effective than a method based on uniform sectioning.

• An eight-dimensional feature vector representation is shown to be the optimum knowledge description for distinguishing among nasal profiles.

• Six mathematically optimized code vectors (ie, patterns) were found.

• The differences between vectors were maximized by morphological traits of the root, dorsum, tip, and base of the nose.

The 24-dimensional vector whose elements were the x coordinate values corresponding to y = (q × 1)/ 25, (q × 2)/25, …, (q × 24)/25 for the contour data composed of q coordinate values, where 201 ≤ q ≤ 420 was defined as V^(we) (Figure 2).

A distance between V^*_N(i) and V^*_N(j)⁠, D_ij(N) was calculated, where I = 1, 2, …, N; j = 1, 2, …, N; j ≠ i. The minimum of D_ij(N) where I = I, D_I min(N), was given by

The mean intercode vector distance D_(N) and ΔD_(N) for the code vector set V^*_N were computed as follows:

where N = 3, 4, …, 12. The N for which ΔD_(N) took maximum value was assigned as M. M + 1 was defined as the optimum number of codes Nopt^(wx).

The judgment score on a facial image d for a linguistic description t, JS[d, t] was given by the following equation:

where JS[j, d, t] is the judgment score on a facial image d for a linguistic description t given by judge j. The judgment score of the classified nasal form corresponding to the code vector V^*_Nopt(wx)(i), JS(wx, i)[t] was calculated as follows:

where Di is a set of indices of facial images corresponding to the feature vectors classified into the code vector V^*_Nopt(wx)(i)⁠. The judgment score group JSG(wx, t) was defined as follows:

The statistical significance of the differences between the judgment score groups JSG(wx, t), where t = 1, 2, …, 29, was tested for each wx by the one-way analysis of variance, and a matching score S(wx, t) was given by

The matching score for the feature vector generation method wx, S(wx) and the highest matching score S_highest were determined by the equations given below:

The wx that achieved the highest matching score S_highest was defined as the optimum feature vector extraction method wopt.