ORIGINAL ARTICLE
The Asher-McDade Aesthetic Index in Comparison With Two Scoring Systems in Nonsyndromic Complete Unilateral Cleft Lip and Palate Patients D.G.M. Mosmuller, MD, C.L. Bijnen, MD,y G.J.C. Kramer, DDS, PhD,z M.A. Disse, DDS,z C. Prahl, DDS, PhD,z D.J. Kuik, MSc,§ F.B. Niessen, MD, PhD, and J.P.W. Don Griot, MD, PhD Objective: To compare the Asher-McDade aesthetic index with 2 systems used to score the appearance of the nasolabial area in patients with a complete cleft lip and palate. Design: Retrospective analysis of the results of complete unilateral cleft lip and palate patients. Setting: Academic Center for Dentistry of Amsterdam and the VU University Medical Center. Patients: Six-year-olds with complete unilateral cleft lip and palate. Main outcome measures: Cleft lip and palate patients assessed using the scoring system proposed by Prahl et al, a 5-point ordinal scale, and the scoring system proposed by Asher-McDade et al by 6 judges, 3 orthodontists, and 3 plastic surgeons. A calculation of intra- and interobserver reliability was made. A comparison was made of all the assessment methods using Kendalls’ tau. Results: Photographs of 55 children (38 boys and 17 girls) with complete unilateral cleft lip and palate were assessed. For the scoring system of Prahl et al, interobserver reliability varied from 0.43 to 0.53, for the 5-point scale between 0.45 and 0.57, and for the scoring system by Asher-McDade et al these varied between 0.52 and 0.66. Multiple significant correlations were found between the used scoring systems. Conclusion: It can be concluded that the Asher-McDade aesthetic index is still superior to the other scoring systems used in this study. However, all 3 scoring systems can reliably be used when 3 or more observers are used. Key Words: Assessment, cleft lip and palate, facial appearance, nasolabial appearance, scoring system, symmetry (J Craniofac Surg 2015;26: 1242–1245)
From the Department of Plastic, Reconstructive and Hand Surgery, VU University Medical Center Amsterdam, HV Amsterdam, The Netherlands; yDepartment of Maxillofacial Surgery, Shenzhen’s Childrens Hospital, Fu Tian District, Shenzhen, China; zDepartment of Orthodontics, Academic Center for Dentistry, LA Amsterdam; and §Department of Medical Statistics, VU University Medical Center Amsterdam, HV Amsterdam, The Netherlands. Received April 17, 2014. Accepted for publication October 16, 2014. Address correspondence and reprint requests to D.G.M. Mosmuller, MD, Department of Plastic, Reconstructive and Hand Surgery, VU University Medical Center Amsterdam, De Boelelaan 1117, 1081 HV Amsterdam, The Netherlands; E-mail: [email protected] The authors report no conflicts of interest. Copyright # 2015 by Mutaz B. Habal, MD ISSN: 1049-2275 DOI: 10.1097/SCS.0000000000001784
1242
C
left lip and palate has an overall incidence of approximately 1 in 700 live births.1 An isolated cleft lip is seen in less than 20% of these cases, whereas combined cleft lip and palate is the most common presentation with 50%.2 Many factors affect the etiology of cleft lip and palate, and it can also be part of a syndrome. Around the age of 3 to 6 months children will undergo their first cleft lip and palate operation, the surgical repair of the cleft lip, which is performed in most cleft palate centers. The purpose of this operation is to make the lip and nose perfectly symmetrical when the face is at rest and when it is moving. For some patients, however, the initially achieved symmetry will eventually be disturbed by contraction and/ or even hypertrophy of the scar on the lip, resulting in an outcome that is unfavorable.3– 6 Research by Al-Omari et al5 and by Mosmuller et al6 indicates that reliability tests have been conducted for only a few of the currently available scoring systems used to assess the results of cleft lip surgery. To perform this assessment of cleft-related facial deformities the majority of studies use 2-dimensional photographs that are rated using an ordinal scale.5,6 Asher-McDade et al7 have developed one of the most commonly used methods used to measure the aesthetic results of surgical cleft lip and nose repair. Their method involves showing masked photographs of the nasolabial area of patients to assessors who are to use a 5-point ordinal scale to rate the following aspects: the vermilion border, nasal form, nasal symmetry, and nasal profile including the upper lip. It is important to note that the assessors are given strict instructions when using this scoring system. When assessing the upper lip, for example, the assessors may only rate the vermilion. This however constitutes only a minor part of the total cleft lip deformity. Moreover, this system requires that for every patient 4 different features must be assessed, making it very time consuming. The final disadvantage is that, for all research that makes use of this system, single-observer reliability is insufficient.5,6 However, AsherMcDade’s scoring system is still thought to be the most reliable method using 2-dimensional photographs for the assessment of cleftrelated deformities.5,6 Prahl et al8 developed another scoring system that also uses photographs. These have a visual analog scale as well as a numerical scale (0–200) that allow the assessor to give an overall rating of cleft lip and palate patients at 18 months of age; using a numerical scale was found to make this method more reliable.8,9 A previous study tested the reliability of Prahl et al’s scorings system8 as well as a simplified version of this system.9 The patient’s nose and lip were also assessed separately using these 2 systems. The reliability scores of these tests are presented in Tables 1 and 2. In this study, these scoring systems will be compared with the scoring system by Asher-McDade et al7 to assess which of these scoring systems is the most reliable.
MATERIALS AND METHODS Fifty-five postoperative pictures from over the past 30 years from the database of the department of orthodontics of the
The Journal of Craniofacial Surgery
Volume 26, Number 4, June 2015
Copyright © 2015 Mutaz B. Habal, MD. Unauthorized reproduction of this article is prohibited.
The Journal of Craniofacial Surgery
Volume 26, Number 4, June 2015
Nasolabial Appearance in CLP Patients
TABLE 1. Numerical Scoring System12 Measure Numerical Scoring System Sample Size Cronbachs’ alpha Interobserver score 95% confidence interval Intraobserver score Estimated reliability 3 examinersy y
Nose and Lip N ¼ 54
Nose Numerical Score N ¼ 54
Lip Numerical Score N ¼ 54
Total Numerical Score N ¼ 54
0.87 0.53 0.40–0.65 0.71 0.78
0.86 0.51 0.38–0.64 0.77 0.80
0.82 0.43 0.30–0.58 0.71 0.70
0.84 0.47 0.44–0.60 0.74 0.78
Obtained using the Intraclass correlation coefficient. Obtained using the Spearman Brown formula.
Academic Center for Dentistry of Amsterdam were assessed. The photographs were taken when the patients were 6-years old. The assessment only included patients who had a complete unilateral cleft lip; patients who had more than 1 lip repair surgery before the picture was taken, and patients with syndromes were excluded. The photographs were cropped so that one could only see a circle with the nose and mouth of the patient (Fig. 1). Besides the frontal view photograph, a profile photograph (Fig. 1) of each patient was also assessed, resulting in 110 photographs to be assessed. The appearance of the patients’ nose and lip was independently assessed by 6 judges using the Asher-McDade aesthetic index: (1) (2) (3) (4)
Nasal form (frontal view), Deviation of the nose (frontal view), Shape of the vermilion border, and Nasal profile including upper lip (lateral view). All 4 items had to be scored on a 5-point scale:
(1) (2) (3) (4) (5)
Very good appearance Good appearance Fair appearance Poor appearance Very poor appearance
the experimental assessments, the judges were asked to assess 6 photographs showing patients who were not included in the actual experimental trial. Two weeks after the first assessment, the same judges assessed the same number of pictures, once again in random order. In an earlier study the same judges performed an assessment of all the included patients using the scoring system developed by Prahl et al,8 and a simplified version (5-point scale) of this scoring system.9
Statistical Analysis The Intraclass correlation coefficient was used to measure the degree of agreement between judges (interobserver score) as well as to determine the intraobserver scores. The internal consistency of the scales were measured by means of Cronbachs’ alpha. The reliability of a panel of 3 and 6 judges for the different scores were estimated by using the Spearman Brown formula. Kendalls’ tau was used to find the amount of correlation between the 3 separate scoring systems. Finally, the Wilcoxon test was used to determine the differences in scoring between the plastic surgeons and the orthodontists. The data were analyzed with the statistical program SPSS for Windows 15.0.
RESULTS
Three of the judges were plastic surgeons and the other 3 judges were orthodontists. Photographs were presented simultaneously as color slides in a PowerPoint presentation in a darkened room. Memory effects were reduced in 2 ways. First, the photographs were presented in a random order each time. Second, the clefts were always shown to the assessors as left-sided clefts; Adobe Photoshop (Adobe Systems Inc., San Jose, CA) was used to provide a mirror image of the photograph without causing any distortion. Steps were taken to make the judges more familiar with both the type of photographs they would be assessing and the scoring system. Before
Postoperative photographs of 55 children (6 years of age, 38 boys and 17 girls) with complete unilateral cleft lip and palate were assessed. Of these patients 48 had a left-sided cleft lip and 7 had a right-sided cleft lip. As mentioned before, in a previous study the scorings system by Prahl et al and a simplified version (5-point scale) were tested on its reliability.9 The results of this study are presented in Tables 1 and 2. The method of assessment we tested in this study was the scoring system as proposed by Asher-McDade et al7 Table 3 shows that a high level of overall reliability was obtained (Cronbachs’ alpha 0.92). The highest agreement between the observers was found in the assessment of the nasal form and in the assessment of the profile photographs (both 0.60). The total score resulting from summing
TABLE 2. Five-Point Scale12 Measure 5-Point Scale Sample Size Cronbachs’ alpha Interobserver score 95% confidence interval Intraobserver score Estimated reliability 3 examinersy y
#
Nose and Lip N ¼ 55
Nose 5-Point Scale N ¼ 55
Lip 5-Point Scale N ¼ 55
Total 5-Point Scale N ¼ 55
0.88 0.55 0.43–0.66 0.67 0.79
0.89 0.57 0.44–0.69 0.75 0.83
0.83 0.45 0.32–0.60 0.75 0.73
0.87 0.53 0.41–0.66 0.77 0.80
Obtained using the Intraclass correlation coefficient. Obtained using the Spearman Brown formula.
2015 Mutaz B. Habal, MD
1243
Copyright © 2015 Mutaz B. Habal, MD. Unauthorized reproduction of this article is prohibited.
The Journal of Craniofacial Surgery
Mosmuller et al
FIGURE 1. Cropped photographs, frontal and lateral view.
the 4 subscores produced a higher level of reliability (0.67), meaning that the sum of the subscores is more reliable than the subscores alone. This can also be said for the intraobserver reliability. Besides this, the intraobserver score for the nasal deviation was the lowest (0.65). The interobserver reliability of this method of assessment is insufficient, but the estimated reliability of 3 examiners was found to be excellent (0.89). The intraobserver scores of this scoring system were sufficient to good (see Table 3). An interesting finding is that the orthodontists scored significantly more negative in this scoring system (P < 0.05) a finding that was also found in our previous study. A significant correlation was found for the scoring of the nose with the use of the 5-point scale with the scoring of the nasal form and the total score. Besides this a significant correlation was found in the scoring of the nose and lip together (5-point scale and numerical scale) with the scorings of the vermilion border. This last finding correlates with the fact that the judges’ assessment of the lip is given more weight than their assessment of the nose, as found in our previous study. Lastly, a correlation was found between the numerical scoring of the lip and the assessment of the vermilion border.
DISCUSSION Over the years several scholars have attempted to construct reliable and objective methods to assess facial deformities including cleft lip. In general, these assessments have been based on 2-dimensional photographs.5,6 Most studies use a 5 to 9 point ordinal scale. The reliability of the scale used in the studies is rarely calculated. Prahl et al8 tested the overall reliability of their assessment system and found that it was very high, with a Cronbachs’ alpha >0.97. Their assessment used a visual analog scale and numerical scale, a reference photograph was used and the assessments were done by 45 judges. In a previous study, it was found that the simplified version of the scoring system by Prahl et al8 measures the same outcome as the original system and that both systems have almost the same reliability.9 In this study, the Cronbachs’ alpha was found
Volume 26, Number 4, June 2015
to be a little lower (0.88), the reason for this might be in the fact that in the study by Prahl et al8 more observers were used (45 judges). The scoring system by Asher-McDade et al7 was not modified and the results of this study are even better than the results of the original study by Asher-McDade et al.7 Although arguably the best way to measure the symmetry of someone’s nostrils is to do so from a ‘‘worm’s-eye view,’’ from this perspective one cannot clearly see the attractiveness of the whole nasolabial area of a patient. Moreover, this perspective is rarely relevant in social circumstances.10 The 3/4 oblique view is not commonly used in assessing cleftrelated facial deformities and is difficult to crop to exclude disturbing surrounding factors. Although Eurocleft recommends using frontal, both laterals, 3/4 oblique, and inferior columellar views to assess the results of surgery in cleft lip and palate patients, in the Eurocleft study itself only the Asher-McDade rating system was used to assess the nasolabial appearance because in most centers only frontal and lateral views were available.11,12 An interesting finding in this and the previous studies was the fact that, regardless of the method of assessment, the scores given by the orthodontists were significantly more negative than those given by the plastic surgeons. This may be because plastic surgeons look at the photographs from a surgical perspective and compare the results with the preoperative population, while orthodontists are more likely to compare the results with the normal population. The photographs used in this study were cropped so that only a circle was left revealing the patient’s nose and mouth. Masked photographs, such as those used by Asher-McDade et al,7 can affect the ratings because symmetry artifacts of the face can still be seen in some of the masked photographs. In cropped photographs with only a circle showing the patient’s nose and mouth, it is much less likely that a symmetry artifact will occure.8,13 Previous studies have shown that assessors score photographs showing the patient’s entire face much more positively than photographs that are cropped.8,14,15 Further, numerical scoring systems allow assessors to be much more discriminative compared with a visual analog scale.8 Moreover, a visual analog scale makes it more difficult to interpret the results of the assessment.8 Cleft lip and palate facial deformities can also be assessed by using video recordings. Morrant and Shaw15 discovered that the pooled panel scores for different aspects of the nose and the lip had a poor to excellent reliability. However, this assessment method is not appropriate for infants because the subject must fully cooperate during the video recording. Also measurements on digital photographs are used to score the results of cleft lip and palate surgery. For example, programs such as Photoshop16– 18 SymNose,19 LabVIEW 6.1, and NI0IMAQ Vision20 can be used to make measurements on photographs. The disadvantage, however, is that measurements on 2-dimensional
TABLE 3. Asher-McDade Aesthetic Index Measure Sample size Cronbachs’ alpha Interobserver scores Asher-McDade et al12 95% confidence interval Intraobserver scores Estimated reliability 3 examinersy
Nasal Form N ¼ 55
Nasal Deviation N ¼ 55
Vermilion Border N ¼ 55
Profile N ¼ 55
Total N ¼ 55
0.91 0.64 0.44 0.54–0.74 0.75 0.82
0.88 0.55 0.43 0.44–0.66 0.66 0.84
0.87 0.52 0.49 0.41–0.64 0.73 0.82
0.89 0.58 0.56 0.47–0.69 0.71 0.81
0.92 0.66 0.60 0.56–0.76 0.79 0.89
y
Obtained using the Intraclass correlation coefficient. Obtained using the Spearman Brown formula.
1244
#
2015 Mutaz B. Habal, MD
Copyright © 2015 Mutaz B. Habal, MD. Unauthorized reproduction of this article is prohibited.
The Journal of Craniofacial Surgery
Volume 26, Number 4, June 2015
pictures are affected by differences in lighting and head orientation. Moreover, the distance between the camera and the subject can vary, which makes the measurements unreliable. To overcome this problem, some have proposed the use of standardized photographs.21 However, it is very difficult to get standardized photographs because children (and especially infants) tend to move during the photo shoot. A new development in the field of patient appearance assessment is the use of 3-dimensional media. This tool seems to provide the most reliable method to score cleft lip and palate patients.5,6 There are, however, some disadvantages associated with 3-dimensional media. One of these disadvantages is that expensive equipment is required. This equipment can be difficult to obtain and can only be operated by specially trained operators. Rating of 2-dimensional photographs is simple, inexpensive, and noninvasive. Moreover, taking such photographs is a standard part of the diagnosis procedure, and thus they are available for nearly every patient. However, the most important and frequently mentioned disadvantage is that distortions can occur when the image of a 3-dimensional object is projected onto a 2-dimensional surface. An additional problem with having assessors rate a patient’s facial appearance is that such assessments are based on subjective notions of attractiveness. For all of these reasons, assessing patients’ facial appearance can be a complex process involving several different variables. In the present study, multiple scoring systems were compared to see which scoring system is more reliable. It can be concluded that the scoring system by Asher-McDade et al7 is superior to the scoring system by Prahl et al8 and to the simplified version of this system.9 However, it can be said that all used scoring system are able to score cleft lip and palate-related deformities in children of 6-years old. Another conclusion is that there still is a need for a more reliable scorings system using 2-dimensional photographs because the most frequently used scoring system (Asher-McDade et al7) can be considered as not reliable enough when only 1 observer is used. However, when 3 or more observers are used, all 3 scoring systems are reliable.
REFERENCES 1. Mossey P, Castillia E. Global Registry and Database on Craniofacial Anomalies. Geneva: World Health Organization; 2003 2. Young G. Cleft Lip and Palate. Available at: http://www2.utmb.edu/ otoref/Grnds/Cleft-lip-palate-9801/Cleft-lip-palate-9801.htm. Accessed February 17, 2015. 3. van der Veer WM, Bloemen MC, Ulrich MM, et al. Potential cellular and molecular causes of hypertrophic scar formation. Burns 2009;35:15–29
#
2015 Mutaz B. Habal, MD
Nasolabial Appearance in CLP Patients
4. Molsted K. Treatment outcome in cleft lip and palate: issues and perspectives. Crit Rev Oral Biol Med 1999;10:225–239 5. Al-Omari I, Millet DT, Ayoub AF. Methods of assessment of cleftrelated facial deformity: a review. Cleft Palate Craniofac J 2005;42:145–156 6. Mosmuller DGM, Don Griot JPW, Bijnen CL, et al. Scoring systems of cleft related facial deformities: a review of literature. Cleft Palate Craniofac J 2013;50:286–296 7. Asher-McDade C, Roberts C, Shaw WC, et al. Development of a method for rating nasolabial appearance in patients with clefts of the lip and the palate. Cleft Palate Craniofac J 1991;28:385–391 8. Prahl C, Prahl-Andersen B, van’t Hof MA, et al. Infant orthopedics and facial appearance: a randomized clinical trial (Dutchcleft). Cleft Palate Craniofac J 2006;43-6:659–664 9. Mosmuller DGM, Bijnen CL, Don Griot JPW, et al. Comparison of two scoring systems in the assessment of nasolabial appearance in cleft lip and palate patients. J Craniofac Surg 2014;25:1222–1225 10. Tobiasen JM. Scaling facial impairment. Cleft Palate J 1989;26:249– 254 11. Shaw WC, Semb G, Nelson P, et al. The Eurocleft Project 1996-2000. Standards of Care for Cleft Lip and Palate in Europe. Amsterdam: IOS Press; 2000 12. Brattstro¨m V1, Mølsted K, Prahl-Andersen B, et al. The Eurocleft study: intercenter study of treatment outcome in patients with complete cleft lip and palate. Part 2:craniofacial form and nasolabial appearance. Cleft Palate Craniofac J 2005;42:69–77 13. Kuijpers-Jagtman AM, Nollet PJ, Semb G, et al. Reference photographs for nasolabial appearance rating in unilateral cleft lip and palate. J Craniofac Surg 2009;20:1683–1686 14. Tobiasen JM. Scaling facial impairment. Cleft Palate J 1989;26:249– 254 15. Morrant DG, Shaw WC. Use of standardized video recordings to assess cleft surgery outcome. Cleft Palate Craniofac J 1996;33:134–142 16. Reddy SG, Nagy K, Mommaerts MY, et al. Primary septoplasty in the repair of unilateral complete cleft lip and palate. Plast Reconstr Surg 2011;127:761–767 17. Chang CS, Por YC, Liou EJ, et al. Long-term comparison of four techniques for obtaining nasal symmetry in unilateral complete cleft lip patients: a single surgeon’s experience. Plast Reconstr Surg 2010;126:1276–1284 18. Nagy K, Mommaerts MY. Analysis of the cleft-lip nose in submentalvertical view, Part I – reliability of a new measurement instrument. J Craniomaxillofac Surg 2007;35:265–277 19. Pigott R, Pigott B. Quantitative Measurement of Symmetry From Photographs Following Surgery for Unilateral Cleft Lip and Palate. Cleft Palate Craniofac J 20091. (in press) 20. Kim SC, Nam KC, Rah DK, et al. Assessment of the cleft nasal deformity using a regression equation. Cleft Palate Craniofac J 2009;46:197–203 21. Vegter F, Hage JJ. Standardized facial photography of cleft patients: just fit the grid. Cleft Palate Craniofac J 2000;37:435–440
1245
Copyright © 2015 Mutaz B. Habal, MD. Unauthorized reproduction of this article is prohibited.
The Asher-McDade Aesthetic Index in Comparison With Two Scoring Systems in Nonsyndromic Complete Unilateral Cleft Lip and Palate Patients D.G.M. Mosmuller, MD, C.L. Bijnen, MD,y G.J.C. Kramer, DDS, PhD,z M.A. Disse, DDS,z C. Prahl, DDS, PhD,z D.J. Kuik, MSc,§ F.B. Niessen, MD, PhD, and J.P.W. Don Griot, MD, PhD Objective: To compare the Asher-McDade aesthetic index with 2 systems used to score the appearance of the nasolabial area in patients with a complete cleft lip and palate. Design: Retrospective analysis of the results of complete unilateral cleft lip and palate patients. Setting: Academic Center for Dentistry of Amsterdam and the VU University Medical Center. Patients: Six-year-olds with complete unilateral cleft lip and palate. Main outcome measures: Cleft lip and palate patients assessed using the scoring system proposed by Prahl et al, a 5-point ordinal scale, and the scoring system proposed by Asher-McDade et al by 6 judges, 3 orthodontists, and 3 plastic surgeons. A calculation of intra- and interobserver reliability was made. A comparison was made of all the assessment methods using Kendalls’ tau. Results: Photographs of 55 children (38 boys and 17 girls) with complete unilateral cleft lip and palate were assessed. For the scoring system of Prahl et al, interobserver reliability varied from 0.43 to 0.53, for the 5-point scale between 0.45 and 0.57, and for the scoring system by Asher-McDade et al these varied between 0.52 and 0.66. Multiple significant correlations were found between the used scoring systems. Conclusion: It can be concluded that the Asher-McDade aesthetic index is still superior to the other scoring systems used in this study. However, all 3 scoring systems can reliably be used when 3 or more observers are used. Key Words: Assessment, cleft lip and palate, facial appearance, nasolabial appearance, scoring system, symmetry (J Craniofac Surg 2015;26: 1242–1245)
From the Department of Plastic, Reconstructive and Hand Surgery, VU University Medical Center Amsterdam, HV Amsterdam, The Netherlands; yDepartment of Maxillofacial Surgery, Shenzhen’s Childrens Hospital, Fu Tian District, Shenzhen, China; zDepartment of Orthodontics, Academic Center for Dentistry, LA Amsterdam; and §Department of Medical Statistics, VU University Medical Center Amsterdam, HV Amsterdam, The Netherlands. Received April 17, 2014. Accepted for publication October 16, 2014. Address correspondence and reprint requests to D.G.M. Mosmuller, MD, Department of Plastic, Reconstructive and Hand Surgery, VU University Medical Center Amsterdam, De Boelelaan 1117, 1081 HV Amsterdam, The Netherlands; E-mail: [email protected] The authors report no conflicts of interest. Copyright # 2015 by Mutaz B. Habal, MD ISSN: 1049-2275 DOI: 10.1097/SCS.0000000000001784
1242
C
left lip and palate has an overall incidence of approximately 1 in 700 live births.1 An isolated cleft lip is seen in less than 20% of these cases, whereas combined cleft lip and palate is the most common presentation with 50%.2 Many factors affect the etiology of cleft lip and palate, and it can also be part of a syndrome. Around the age of 3 to 6 months children will undergo their first cleft lip and palate operation, the surgical repair of the cleft lip, which is performed in most cleft palate centers. The purpose of this operation is to make the lip and nose perfectly symmetrical when the face is at rest and when it is moving. For some patients, however, the initially achieved symmetry will eventually be disturbed by contraction and/ or even hypertrophy of the scar on the lip, resulting in an outcome that is unfavorable.3– 6 Research by Al-Omari et al5 and by Mosmuller et al6 indicates that reliability tests have been conducted for only a few of the currently available scoring systems used to assess the results of cleft lip surgery. To perform this assessment of cleft-related facial deformities the majority of studies use 2-dimensional photographs that are rated using an ordinal scale.5,6 Asher-McDade et al7 have developed one of the most commonly used methods used to measure the aesthetic results of surgical cleft lip and nose repair. Their method involves showing masked photographs of the nasolabial area of patients to assessors who are to use a 5-point ordinal scale to rate the following aspects: the vermilion border, nasal form, nasal symmetry, and nasal profile including the upper lip. It is important to note that the assessors are given strict instructions when using this scoring system. When assessing the upper lip, for example, the assessors may only rate the vermilion. This however constitutes only a minor part of the total cleft lip deformity. Moreover, this system requires that for every patient 4 different features must be assessed, making it very time consuming. The final disadvantage is that, for all research that makes use of this system, single-observer reliability is insufficient.5,6 However, AsherMcDade’s scoring system is still thought to be the most reliable method using 2-dimensional photographs for the assessment of cleftrelated deformities.5,6 Prahl et al8 developed another scoring system that also uses photographs. These have a visual analog scale as well as a numerical scale (0–200) that allow the assessor to give an overall rating of cleft lip and palate patients at 18 months of age; using a numerical scale was found to make this method more reliable.8,9 A previous study tested the reliability of Prahl et al’s scorings system8 as well as a simplified version of this system.9 The patient’s nose and lip were also assessed separately using these 2 systems. The reliability scores of these tests are presented in Tables 1 and 2. In this study, these scoring systems will be compared with the scoring system by Asher-McDade et al7 to assess which of these scoring systems is the most reliable.
MATERIALS AND METHODS Fifty-five postoperative pictures from over the past 30 years from the database of the department of orthodontics of the
The Journal of Craniofacial Surgery
Volume 26, Number 4, June 2015
Copyright © 2015 Mutaz B. Habal, MD. Unauthorized reproduction of this article is prohibited.
The Journal of Craniofacial Surgery
Volume 26, Number 4, June 2015
Nasolabial Appearance in CLP Patients
TABLE 1. Numerical Scoring System12 Measure Numerical Scoring System Sample Size Cronbachs’ alpha Interobserver score 95% confidence interval Intraobserver score Estimated reliability 3 examinersy y
Nose and Lip N ¼ 54
Nose Numerical Score N ¼ 54
Lip Numerical Score N ¼ 54
Total Numerical Score N ¼ 54
0.87 0.53 0.40–0.65 0.71 0.78
0.86 0.51 0.38–0.64 0.77 0.80
0.82 0.43 0.30–0.58 0.71 0.70
0.84 0.47 0.44–0.60 0.74 0.78
Obtained using the Intraclass correlation coefficient. Obtained using the Spearman Brown formula.
Academic Center for Dentistry of Amsterdam were assessed. The photographs were taken when the patients were 6-years old. The assessment only included patients who had a complete unilateral cleft lip; patients who had more than 1 lip repair surgery before the picture was taken, and patients with syndromes were excluded. The photographs were cropped so that one could only see a circle with the nose and mouth of the patient (Fig. 1). Besides the frontal view photograph, a profile photograph (Fig. 1) of each patient was also assessed, resulting in 110 photographs to be assessed. The appearance of the patients’ nose and lip was independently assessed by 6 judges using the Asher-McDade aesthetic index: (1) (2) (3) (4)
Nasal form (frontal view), Deviation of the nose (frontal view), Shape of the vermilion border, and Nasal profile including upper lip (lateral view). All 4 items had to be scored on a 5-point scale:
(1) (2) (3) (4) (5)
Very good appearance Good appearance Fair appearance Poor appearance Very poor appearance
the experimental assessments, the judges were asked to assess 6 photographs showing patients who were not included in the actual experimental trial. Two weeks after the first assessment, the same judges assessed the same number of pictures, once again in random order. In an earlier study the same judges performed an assessment of all the included patients using the scoring system developed by Prahl et al,8 and a simplified version (5-point scale) of this scoring system.9
Statistical Analysis The Intraclass correlation coefficient was used to measure the degree of agreement between judges (interobserver score) as well as to determine the intraobserver scores. The internal consistency of the scales were measured by means of Cronbachs’ alpha. The reliability of a panel of 3 and 6 judges for the different scores were estimated by using the Spearman Brown formula. Kendalls’ tau was used to find the amount of correlation between the 3 separate scoring systems. Finally, the Wilcoxon test was used to determine the differences in scoring between the plastic surgeons and the orthodontists. The data were analyzed with the statistical program SPSS for Windows 15.0.
RESULTS
Three of the judges were plastic surgeons and the other 3 judges were orthodontists. Photographs were presented simultaneously as color slides in a PowerPoint presentation in a darkened room. Memory effects were reduced in 2 ways. First, the photographs were presented in a random order each time. Second, the clefts were always shown to the assessors as left-sided clefts; Adobe Photoshop (Adobe Systems Inc., San Jose, CA) was used to provide a mirror image of the photograph without causing any distortion. Steps were taken to make the judges more familiar with both the type of photographs they would be assessing and the scoring system. Before
Postoperative photographs of 55 children (6 years of age, 38 boys and 17 girls) with complete unilateral cleft lip and palate were assessed. Of these patients 48 had a left-sided cleft lip and 7 had a right-sided cleft lip. As mentioned before, in a previous study the scorings system by Prahl et al and a simplified version (5-point scale) were tested on its reliability.9 The results of this study are presented in Tables 1 and 2. The method of assessment we tested in this study was the scoring system as proposed by Asher-McDade et al7 Table 3 shows that a high level of overall reliability was obtained (Cronbachs’ alpha 0.92). The highest agreement between the observers was found in the assessment of the nasal form and in the assessment of the profile photographs (both 0.60). The total score resulting from summing
TABLE 2. Five-Point Scale12 Measure 5-Point Scale Sample Size Cronbachs’ alpha Interobserver score 95% confidence interval Intraobserver score Estimated reliability 3 examinersy y
#
Nose and Lip N ¼ 55
Nose 5-Point Scale N ¼ 55
Lip 5-Point Scale N ¼ 55
Total 5-Point Scale N ¼ 55
0.88 0.55 0.43–0.66 0.67 0.79
0.89 0.57 0.44–0.69 0.75 0.83
0.83 0.45 0.32–0.60 0.75 0.73
0.87 0.53 0.41–0.66 0.77 0.80
Obtained using the Intraclass correlation coefficient. Obtained using the Spearman Brown formula.
2015 Mutaz B. Habal, MD
1243
Copyright © 2015 Mutaz B. Habal, MD. Unauthorized reproduction of this article is prohibited.
The Journal of Craniofacial Surgery
Mosmuller et al
FIGURE 1. Cropped photographs, frontal and lateral view.
the 4 subscores produced a higher level of reliability (0.67), meaning that the sum of the subscores is more reliable than the subscores alone. This can also be said for the intraobserver reliability. Besides this, the intraobserver score for the nasal deviation was the lowest (0.65). The interobserver reliability of this method of assessment is insufficient, but the estimated reliability of 3 examiners was found to be excellent (0.89). The intraobserver scores of this scoring system were sufficient to good (see Table 3). An interesting finding is that the orthodontists scored significantly more negative in this scoring system (P < 0.05) a finding that was also found in our previous study. A significant correlation was found for the scoring of the nose with the use of the 5-point scale with the scoring of the nasal form and the total score. Besides this a significant correlation was found in the scoring of the nose and lip together (5-point scale and numerical scale) with the scorings of the vermilion border. This last finding correlates with the fact that the judges’ assessment of the lip is given more weight than their assessment of the nose, as found in our previous study. Lastly, a correlation was found between the numerical scoring of the lip and the assessment of the vermilion border.
DISCUSSION Over the years several scholars have attempted to construct reliable and objective methods to assess facial deformities including cleft lip. In general, these assessments have been based on 2-dimensional photographs.5,6 Most studies use a 5 to 9 point ordinal scale. The reliability of the scale used in the studies is rarely calculated. Prahl et al8 tested the overall reliability of their assessment system and found that it was very high, with a Cronbachs’ alpha >0.97. Their assessment used a visual analog scale and numerical scale, a reference photograph was used and the assessments were done by 45 judges. In a previous study, it was found that the simplified version of the scoring system by Prahl et al8 measures the same outcome as the original system and that both systems have almost the same reliability.9 In this study, the Cronbachs’ alpha was found
Volume 26, Number 4, June 2015
to be a little lower (0.88), the reason for this might be in the fact that in the study by Prahl et al8 more observers were used (45 judges). The scoring system by Asher-McDade et al7 was not modified and the results of this study are even better than the results of the original study by Asher-McDade et al.7 Although arguably the best way to measure the symmetry of someone’s nostrils is to do so from a ‘‘worm’s-eye view,’’ from this perspective one cannot clearly see the attractiveness of the whole nasolabial area of a patient. Moreover, this perspective is rarely relevant in social circumstances.10 The 3/4 oblique view is not commonly used in assessing cleftrelated facial deformities and is difficult to crop to exclude disturbing surrounding factors. Although Eurocleft recommends using frontal, both laterals, 3/4 oblique, and inferior columellar views to assess the results of surgery in cleft lip and palate patients, in the Eurocleft study itself only the Asher-McDade rating system was used to assess the nasolabial appearance because in most centers only frontal and lateral views were available.11,12 An interesting finding in this and the previous studies was the fact that, regardless of the method of assessment, the scores given by the orthodontists were significantly more negative than those given by the plastic surgeons. This may be because plastic surgeons look at the photographs from a surgical perspective and compare the results with the preoperative population, while orthodontists are more likely to compare the results with the normal population. The photographs used in this study were cropped so that only a circle was left revealing the patient’s nose and mouth. Masked photographs, such as those used by Asher-McDade et al,7 can affect the ratings because symmetry artifacts of the face can still be seen in some of the masked photographs. In cropped photographs with only a circle showing the patient’s nose and mouth, it is much less likely that a symmetry artifact will occure.8,13 Previous studies have shown that assessors score photographs showing the patient’s entire face much more positively than photographs that are cropped.8,14,15 Further, numerical scoring systems allow assessors to be much more discriminative compared with a visual analog scale.8 Moreover, a visual analog scale makes it more difficult to interpret the results of the assessment.8 Cleft lip and palate facial deformities can also be assessed by using video recordings. Morrant and Shaw15 discovered that the pooled panel scores for different aspects of the nose and the lip had a poor to excellent reliability. However, this assessment method is not appropriate for infants because the subject must fully cooperate during the video recording. Also measurements on digital photographs are used to score the results of cleft lip and palate surgery. For example, programs such as Photoshop16– 18 SymNose,19 LabVIEW 6.1, and NI0IMAQ Vision20 can be used to make measurements on photographs. The disadvantage, however, is that measurements on 2-dimensional
TABLE 3. Asher-McDade Aesthetic Index Measure Sample size Cronbachs’ alpha Interobserver scores Asher-McDade et al12 95% confidence interval Intraobserver scores Estimated reliability 3 examinersy
Nasal Form N ¼ 55
Nasal Deviation N ¼ 55
Vermilion Border N ¼ 55
Profile N ¼ 55
Total N ¼ 55
0.91 0.64 0.44 0.54–0.74 0.75 0.82
0.88 0.55 0.43 0.44–0.66 0.66 0.84
0.87 0.52 0.49 0.41–0.64 0.73 0.82
0.89 0.58 0.56 0.47–0.69 0.71 0.81
0.92 0.66 0.60 0.56–0.76 0.79 0.89
y
Obtained using the Intraclass correlation coefficient. Obtained using the Spearman Brown formula.
1244
#
2015 Mutaz B. Habal, MD
Copyright © 2015 Mutaz B. Habal, MD. Unauthorized reproduction of this article is prohibited.
The Journal of Craniofacial Surgery
Volume 26, Number 4, June 2015
pictures are affected by differences in lighting and head orientation. Moreover, the distance between the camera and the subject can vary, which makes the measurements unreliable. To overcome this problem, some have proposed the use of standardized photographs.21 However, it is very difficult to get standardized photographs because children (and especially infants) tend to move during the photo shoot. A new development in the field of patient appearance assessment is the use of 3-dimensional media. This tool seems to provide the most reliable method to score cleft lip and palate patients.5,6 There are, however, some disadvantages associated with 3-dimensional media. One of these disadvantages is that expensive equipment is required. This equipment can be difficult to obtain and can only be operated by specially trained operators. Rating of 2-dimensional photographs is simple, inexpensive, and noninvasive. Moreover, taking such photographs is a standard part of the diagnosis procedure, and thus they are available for nearly every patient. However, the most important and frequently mentioned disadvantage is that distortions can occur when the image of a 3-dimensional object is projected onto a 2-dimensional surface. An additional problem with having assessors rate a patient’s facial appearance is that such assessments are based on subjective notions of attractiveness. For all of these reasons, assessing patients’ facial appearance can be a complex process involving several different variables. In the present study, multiple scoring systems were compared to see which scoring system is more reliable. It can be concluded that the scoring system by Asher-McDade et al7 is superior to the scoring system by Prahl et al8 and to the simplified version of this system.9 However, it can be said that all used scoring system are able to score cleft lip and palate-related deformities in children of 6-years old. Another conclusion is that there still is a need for a more reliable scorings system using 2-dimensional photographs because the most frequently used scoring system (Asher-McDade et al7) can be considered as not reliable enough when only 1 observer is used. However, when 3 or more observers are used, all 3 scoring systems are reliable.
REFERENCES 1. Mossey P, Castillia E. Global Registry and Database on Craniofacial Anomalies. Geneva: World Health Organization; 2003 2. Young G. Cleft Lip and Palate. Available at: http://www2.utmb.edu/ otoref/Grnds/Cleft-lip-palate-9801/Cleft-lip-palate-9801.htm. Accessed February 17, 2015. 3. van der Veer WM, Bloemen MC, Ulrich MM, et al. Potential cellular and molecular causes of hypertrophic scar formation. Burns 2009;35:15–29
#
2015 Mutaz B. Habal, MD
Nasolabial Appearance in CLP Patients
4. Molsted K. Treatment outcome in cleft lip and palate: issues and perspectives. Crit Rev Oral Biol Med 1999;10:225–239 5. Al-Omari I, Millet DT, Ayoub AF. Methods of assessment of cleftrelated facial deformity: a review. Cleft Palate Craniofac J 2005;42:145–156 6. Mosmuller DGM, Don Griot JPW, Bijnen CL, et al. Scoring systems of cleft related facial deformities: a review of literature. Cleft Palate Craniofac J 2013;50:286–296 7. Asher-McDade C, Roberts C, Shaw WC, et al. Development of a method for rating nasolabial appearance in patients with clefts of the lip and the palate. Cleft Palate Craniofac J 1991;28:385–391 8. Prahl C, Prahl-Andersen B, van’t Hof MA, et al. Infant orthopedics and facial appearance: a randomized clinical trial (Dutchcleft). Cleft Palate Craniofac J 2006;43-6:659–664 9. Mosmuller DGM, Bijnen CL, Don Griot JPW, et al. Comparison of two scoring systems in the assessment of nasolabial appearance in cleft lip and palate patients. J Craniofac Surg 2014;25:1222–1225 10. Tobiasen JM. Scaling facial impairment. Cleft Palate J 1989;26:249– 254 11. Shaw WC, Semb G, Nelson P, et al. The Eurocleft Project 1996-2000. Standards of Care for Cleft Lip and Palate in Europe. Amsterdam: IOS Press; 2000 12. Brattstro¨m V1, Mølsted K, Prahl-Andersen B, et al. The Eurocleft study: intercenter study of treatment outcome in patients with complete cleft lip and palate. Part 2:craniofacial form and nasolabial appearance. Cleft Palate Craniofac J 2005;42:69–77 13. Kuijpers-Jagtman AM, Nollet PJ, Semb G, et al. Reference photographs for nasolabial appearance rating in unilateral cleft lip and palate. J Craniofac Surg 2009;20:1683–1686 14. Tobiasen JM. Scaling facial impairment. Cleft Palate J 1989;26:249– 254 15. Morrant DG, Shaw WC. Use of standardized video recordings to assess cleft surgery outcome. Cleft Palate Craniofac J 1996;33:134–142 16. Reddy SG, Nagy K, Mommaerts MY, et al. Primary septoplasty in the repair of unilateral complete cleft lip and palate. Plast Reconstr Surg 2011;127:761–767 17. Chang CS, Por YC, Liou EJ, et al. Long-term comparison of four techniques for obtaining nasal symmetry in unilateral complete cleft lip patients: a single surgeon’s experience. Plast Reconstr Surg 2010;126:1276–1284 18. Nagy K, Mommaerts MY. Analysis of the cleft-lip nose in submentalvertical view, Part I – reliability of a new measurement instrument. J Craniomaxillofac Surg 2007;35:265–277 19. Pigott R, Pigott B. Quantitative Measurement of Symmetry From Photographs Following Surgery for Unilateral Cleft Lip and Palate. Cleft Palate Craniofac J 20091. (in press) 20. Kim SC, Nam KC, Rah DK, et al. Assessment of the cleft nasal deformity using a regression equation. Cleft Palate Craniofac J 2009;46:197–203 21. Vegter F, Hage JJ. Standardized facial photography of cleft patients: just fit the grid. Cleft Palate Craniofac J 2000;37:435–440
1245
Copyright © 2015 Mutaz B. Habal, MD. Unauthorized reproduction of this article is prohibited.
