The Journal of Foot & Ankle Surgery 54 (2015) 46–50
Contents lists available at ScienceDirect
The Journal of Foot & Ankle Surgery journal homepage: www.jfas.org
Validity, Reliability, and Responsiveness of the Korean Version of American Academy of Orthopedic Surgeons Foot and Ankle Questionnaire Ji-Beom Kim, MD 1, Je Kyun Kim, MD 1, Sang Gyo Seo, MD 2, Dong Yeon Lee, MD 3 1
Orthopedic Surgeon, Department of Orthopedic Surgery, Seoul National University Hospital, Yeongeon-dong, Jongno-gu, Seoul, Korea Instructor, Department of Orthopedic Surgery, Seoul National University Hospital, Yeongeon-dong, Jongno-gu, Seoul, Korea 3 Associate Professor (Clinical), Department of Orthopedic Surgery, Seoul National University Hospital, Yeongeon-dong, Jongno-gu, Seoul, Korea 2
a r t i c l e i n f o
a b s t r a c t
Level of Clinical Evidence: 3
The American Academy of Orthopaedic Surgeons Foot and Ankle Questionnaire (AFAQ) reflects patients’ subjective disorder due to foot and ankle conditions. We evaluated the validity, reliability, and responsiveness of the Korean version of the AFAQ, after translation and transcultural adaptation of the original AFAQ into the Korean language. A total of 206 patients were enrolled, including 152 with chronic problems (experimental group) and 54 with acute problems (control group). We used the intraclass correlation coefficient to assess the test–retest reliability and Cronbach’s a to assess internal reliability. Pearson’s correlation coefficient was used to assess the criterion validity by correlating the Korean AFAQ scores with those from other validated scales (American Orthopaedic Foot and Ankle Society Hallux-Metatarsophalangeal-Interphalangeal scale, American Orthopaedic Foot and Ankle Society Ankle-Hindfoot scale, and visual analog scale for pain). To analyze discriminant validity, we evaluated the difference between the experimental and control groups using the Student t test. Of the 152 patients in the experimental group, 29 revisited our clinic postoperatively and repeated the Korean AFAQ. To analyze responsiveness, we used paired t tests to evaluate postoperative changes. In terms of test–retest reliability, the intraclass correlation coefficient ranged from 0.979 to 0.999. In terms of internal reliability, Cronbach’s a was 0.528 for the stiffness and swelling subscale and greater than 0.7 for all other subscales. In terms of criterion validity, Pearson’s correlation coefficient ranged from 0.492 to 0.699. The probability of the null hypothesis for discriminant validity and responsiveness was statistically significant (p < .001 and p ¼ .021, respectively). These results showed that the Korean version of the AFAQ had the same concept and intention as the original version and is reliable, valid, and responsive. Ó 2015 by the American College of Foot and Ankle Surgeons. All rights reserved.
Keywords: AAOS discriminant validity outcome assessment test-retest reliability transcultural adaptation translation VAS pain
Disorders involving the foot and ankle increase with age owing to degenerative changes involving bone, joint, neuromuscular tissues, and tendon (1). Active lifestyles, shoe gear, and an increased body mass can influence the prevalence of foot and ankle disorders (2). In fact, foot and ankle disorders are usually not correlated directly to age; rather, they correlate with quality of life. However, assessing the effect of foot and ankle disorders on each individual can be quite difficult, because, even in cases of similar severity, they could have different effects, depending on the lifestyle of the patient (3). Financial Disclosure: This study was supported by Basic Science Research Program through the National Research Foundation of Korea funded by the Ministry of Science, ICT and Future Planning (grant NRF-2012R1A1A2044067). Conflict of Interest: None reported. Address correspondence to: Dong Yeon Lee, MD, Associate Professor (Clinical), Department of Orthopedic Surgery, Seoul National University Hospital, Yeongeondong, Jongno-gu, Seoul, Korea, 110-744. E-mail address: [email protected] (D.Y. Lee).
To assess foot and ankle problems, determining the subjective malfunction and severity of pain is as important as finding the anatomic malformation (4). Various questionnaires have been used as an instrument to measure the subjective discomfort due to the disorders and/or the improvement after treatment (5–10). However, no reference standard instrument has yet been determined for assessment of the foot and ankle. For example, the RAND Medical Outcomes Study Short Form-36 questionnaire was supposed to represent the subjective discomfort of patients well, but it does not focus on foot and ankle problems (11) but, instead, is a general health measurement. The American Orthopaedic Foot and Ankle Society (AOFAS) scales focus on foot and ankle problems and seem to be the most widely used scales in foot and ankle research in recent years (6). The AOFAS scale studies reported the scales could evaluate the outcomes of surgery (12), and their subjective component had acceptable validity (13). However, other studies have reported that the AOFAS scales are difficult to perform and lack reliability (14), validity, and precision (15–17).
1067-2516/$ - see front matter Ó 2015 by the American College of Foot and Ankle Surgeons. All rights reserved. http://dx.doi.org/10.1053/j.jfas.2014.08.011
J.-B. Kim et al. / The Journal of Foot & Ankle Surgery 54 (2015) 46–50
The American Academy of Orthopaedic Surgeons (AAOS) (9) developed an instrument for subjective patient assessment (i.e., the AAOS Lower Limb Instrument), and the AAOS Foot and Ankle Questionnaire (AFAQ) is one of the major parts of the AAOS Lower Limb Instrument, focusing on foot and ankle problems. The AFAQ is a self-administered questionnaire designed for efficient collection of patient-oriented data. It consists of 25 items and combines the Global Foot and Ankle scale (GFAS) and the Shoe Comfort scale (SCS). The GFAS is formatted into 4 subscales that address pain, function, stiffness and swelling, and “giving way” (9). “Giving way” of an ankle means the ankle is unable to support the person’s weight unexpectedly and suddenly. The SCS is formatted into 5 questions about the type of shoes that are comfortable to the patient. An assessment instrument should be examined for reliability, validity, and responsiveness. After an assessed instrument has been determined to be reliable, valid, and responsive, physicians and surgeons can trust the instrument and use it. The AFAQ was expected to reflect a patient’s subjective discomfort well and to produce meaningful data. Although AAOS group insisted that the AFAQ showed high reliability and validity (9), similar support for the AFAQ in the published data, to our knowledge, is nonexistent. The original AFAQ was developed in the English language and was intended for use in English-speaking countries (9). To validate the AFAQ in Korea, the original AFAQ had to be translated into the Korean language. Although it is not easy to translate an English-language questionnaire into another language without losing some of the meaning in translation, we created the Korean version of the AFAQ with adaptation of the cultural nuances in accordance with internationally published recommendations (18,19). The purpose of the present report was to demonstrate the reliability, validity, and responsiveness of the Korean version of the AFAQ. Patients and Methods Our hospital’s institutional review board approved the present study, and all enrolled patients provided written informed consent. Before beginning the present study, we ensured that the original AFAQ had not already been translated into the Korean language. In addition, we contacted Johanson, the author of the AAOS Lower Limb Instrument (9), and he granted us permission to produce the Korean version of the AFAQ. We performed 2 steps in the present study. The first step was to create the Korean version of the AFAQ by translation and transcultural adaptation. The second step was to examine the reliability, validity, and responsiveness of the Korean version of the AFAQ. The original AFAQ was translated into Korean by 2 Koreans, an orthopedic surgeon and a nonmedical person. Both translators were native Koreans and fluent in English. They created the Korean version of the AFAQ separately. To reconcile these 2 Korean versions of the AFAQ, the 2 translators and 1 Korean nurse met and discussed the translations. After producing a reconciled Korean version of the AFAQ, 2 bilingual speakers (Korean–English) translated the reconciled Korean version of the AFAQ into English together, without any background information pertaining to the original AFAQ. A committee consisting of 2 back translators and 2 forward translators met to compare the back-translated version of the AFAQ with the original AFAQ. The committee determined that some expressions in the back-translated version of the AFAQ were not equivalent to those of the original AFAQ. To improve these expressions, the reconciled Korean version of the AFAQ was revised. Finally, 20 Korean outpatients from our clinic reviewed the revised Korean version of the AFAQ and were asked whether they had any problems in understanding and answering the revised Korean version. After the pilot study, several ambiguous expressions were modified further. Errors in the revised Korean version of the AFAQ were then corrected, and the Korean version was finalized. To conduct the second step of our investigation, we recruited patients who had undergone an operation at our foot and ankle clinic from January 2012 to March 2013. The patients who had had a chronic foot and ankle problem for more than 1 year were enrolled in the experimental group, and those with an acute foot and ankle problem that had been present for less than 1 month were enrolled in the control group. After admission, the patients enrolled in our study completed the Korean version of the AFAQ for the first time and the visual analog scale for pain (VAS pain). Twenty-four hours later, they completed the Korean version of the AFAQ for the second time. An orthopedic surgeon investigator then visited the patients in the experimental group and completed the other, previously validated, foot-related quality-of-life scales. If patients had a forefoot problem, the AOFAS Hallux-Metatarsophalangeal-Interphalangeal (Hallux-MTP-IP) scale was completed. If patients had a hindfoot or ankle problem,
47
the AOFAS Ankle-Hindfoot scale was completed. After leaving the hospital, the patients in the experimental group visited our clinics and again completed the Korean version of the AFAQ. The patients in the control group completed the Korean AFAQ 1 time at admission. The patients in the experimental group completed the Korean AFAQ 3 times, at admission, 24 hours after completing the Korean AFAQ at admission, and after the operation. We assessed the test–retest reliability and internal reliability to understand the reliability of the Korean version of the AFAQ. Test–retest reliability was evaluated for 2 of the questionnaires, which were completed at a 24-hour interval by the patients in the experimental group before surgery. We calculated the intraclass correlation coefficient as a measure of test–retest reliability. Internal reliability was evaluated for the different items in each of the subscales. We calculated Cronbach’s a coefficient to understand the internal reliability of the Korean version of the AFAQ. The criterion validity and discriminant validity were also assessed to understand the validity of the Korean version of the AFAQ. Criterion validity was analyzed using the correlation between the mean standardized score of the Korean version of the GFAS and other validated scales, including the AOFAS Hallux-MTP-IP scale, AOFAS Ankle-Hindfoot scale, and VAS pain scale. We calculated the Pearson correlation coefficients to assess criterion validity. To assess discriminant validity, we used Student’s t test to determine the difference in the mean standardized score of the Korean version of the AFAQ between the control and experimental groups. Responsiveness indicates the sensitiveness of an instrument to detect clinically meaningful changes in the state of patients. Postoperatively, the patients visited our clinic again to check their status. They also completed the Korean version of the AFAQ a second time. We used paired t tests to compare the differences between the preoperative mean standardized score of the Korean version of the GFAS and the postoperative mean standardized score of the Korean version of the GFAS. The Statistical Package for Social Sciences, version 18.0 (SPSS, Chicago, IL), was used for statistical analysis.
Results A total of 206 patients were included in our study, of whom 152 (73.79%) were enrolled in the experimental group and 54 (26.21%) in the control group. In the experimental group, 152 patients (73.79%) provided us with useable questionnaire information from the first test after enrollment. After a 24-hour interval, of the 152 patients in the experimental group, 144 (94.73% of the experimental group) provided useable questionnaire information from the second test. All 54 patients (26.21%) in the control group provided useable information. All 152 patients in the experimental group provided VAS pain scale data, 38 (25.00% of the experimental group) provided chronic forefoot information by completing the AOFAS Hallux-MTP-IP scale, and 46 (30.26% of the experimental group) provided chronic ankle and/or hindfoot information by completing the AOFAS Ankle-Hindfoot scale. In the experimental group, 29 patients (19.07% of experimental group) visited our clinic again after the operation and completed the Korean version of the AFAQ. The mean average interval from surgery to the follow-up visit when the Korean AFAQ was completed again was 8.2 (range 5.1 to 13.9) months. Of the 152 patients in the experimental group, 61 were male (40.13% of the experimental group) and 91 were female (59.87% of the experimental group), and their mean age was 49 (range 14 to 81) years. Of the 54 patients in the control group, 31 were male (57.41% of the control group) and 23 were female (42.59% of the control group), and their mean age was 46 (range 17 to 76) years. All the patients enrolled in our study were literate and appeared to understand the questionnaires. A statistical description of the prevalence of the diagnoses by patient group is given in Table 1. Test–retest reliability was calculated for the 144 patients (69.9%) in the experimental group who had provided us useable information after completing the questionnaire twice. Internal reliability was calculated for the 152 patients (73.79%) in the experimental group who had provided us with useable questionnaire results after completing the questionnaire the first time after enrollment into the study. The mean standardized scores of the subscales and statistical reliability coefficients are presented in Table 2. The test–retest reliability was considered excellent, because the intraclass correlation coefficient ranged from 0.979 to 0.999. Internal reliability was considered moderate to high, because Cronbach’s a ranged from 0.528
48
J.-B. Kim et al. / The Journal of Foot & Ankle Surgery 54 (2015) 46–50
Discussion
Table 1 Distribution of diagnoses (N ¼ 206 patients) Diagnosis
n (%)
Experimental group (n ¼ 152) Hallux valgus Osteochondral lesion of the talus Mass* Osteoarthritis Ankle instability Foot deformity Nonunion Chronic osteomyelitis Miscellaneousy Control group (n ¼ 54) Fracture Metal removal Miscellaneousy * y
32 23 22 19 11 10 6 6 23
(15.53) (11.17) (10.68) (9.22) (5.34) (4.85) (2.91) (2.91) (11.17)
37 (17.96) 11 (5.34) 6 (2.91)
Bone or soft tissue mass. Diagnoses that included fewer than 5 patients.
to 0.926. The stiffness and swelling subscale showed the lowest Cronbach’s a coefficient, which was 0.528. To assess criterion validity, the results of the Korean version of the AFAQ were compared with those of the AOFAS Hallux-MTP-IP scale, AOFAS Ankle-Hindfoot scale, and the VAS pain scale. To evaluate the correlation between the Korean version of the AFAQ and the other validated scales, we used the Korean version of the GFAS; the measures of criterion validity are presented in Table 3. The correlation between the Korean version of the GFAS and the 2 AOFAS scales were considered moderate, with a Pearson’s correlation coefficient of 0.492 and 0.699, respectively. The correlation between the Korean version of the AFAQ and the VAS pain scale was considered moderate, with a Pearson’s correlation coefficient of 0.687. The higher score of the Koran version of the AFAQ indicated a less severe problem; in contrast, a higher VAS pain score indicated more severe pain. Hence, a moderate inverse correlation coefficient (0.687) was calculated in the present study. To assess discriminant validity, we compared the results of the Korean version of the GFAS between the experimental and control groups using unpaired Student’s t test. The mean standardized score of the Korean version of the GFAS in the experimental group was 67 (range 15 to 100) and that in the control group was 83 (range 41 to 100); this difference was statistically significant (p < .001). To assess the responsiveness of the Korean version of the AFAQ, we compared the results of the Korean version of the GFAS from preoperatively and postoperatively, using paired Student’s t tests for 29 patients (19.07% of the experimental group). The mean standardized score for the preoperative Korean version of the GFAS was 65 (range 15 to 93) and that postoperatively was 77 (range 50 to 100); this difference was also statistically significant (p ¼ .021).
We have demonstrated that the Korean version of the AFAQ, after translation and transcultural adaptation from its original English version, is a reliable instrument for evaluation of the subjective functional status in patients with disorders involving the foot and ankle. Diverse clinical outcome questionnaires for measuring foot and ankle problems have been invented and refined. For these to be useful in the clinical setting, they must demonstrate reliability, validity, and responsiveness. Several groups have reported that the AOFAS scales are reliable, valid, and responsive to change (6,9,13). Thus, the AOFAS scales have become widely used (5). Experienced foot and ankle surgeons developed the AOFAS scales, and their purpose was to provide a reliable measurement of the influence that foot and ankle problems have on an individual (6). The AOFAS scales include 4 different scales that focus on different anatomic regions: the Ankle-Hindfoot, Midfoot, Hallux-MTP-IP, and MTP-IP-Lesser Toe scale. Each scale can be used independently and has 3 subscales (i.e., pain, function, and alignment subscales). The subscales are weighted differently in the AOFAS Ankle-Hindfoot scale than in the other scales. The pain subscale has 1 subjective question in each scale. However, the function subscale has a different number of questions in each scale. The Ankle-Hindfoot scale has 7 questions, the Midfoot scale has 6, and the Hallux-MTP-IP and MTP-IP-Lesser Toe scales both have 5 questions in the function subscale. The alignment subscale consists of 1 question. Patients can complete the pain subscale alone; however, patients cannot complete the function subscale and the alignment subscale without the help of a physician. Each of the AOFAS scales consists of subjective questions related to discomfort and objective elements related to function. Although the AOFAS scales have been widely used, the objective symptom elements have some limitations. First, they require the presence of a physician. Second, the data obtained with these questions might not accurately represent the patient’s status, because the physician’s expectations could influence the data. Third, if the physician completing these objective elements has not been properly trained, the reliability of the results could be threatened (14). In contrast, the AFAQ consists entirely of questions related to subjective discomfort; thus, it is advantageous regarding the procurement of data that effectively measures the patient’s status. Although many foot and ankle surgeons expected the AFAQ to be an alternative health measurement instrument to the AOFAS scales, and although the AFAQ has been shown to be reliable, valid, and responsive to change (9), follow-up studies reinforcing the validity of the AFAQ are nonexistent to our knowledge. Thus, we believe the AFAQ has not been used as widely as the AOFAS scales. In the present report, we have described our findings related to the reliability, validity, and responsiveness of the Korean version of the AFAQ, and we compared our results with those of the AAOS group’s study.
Table 2 Internal reliability and test–retest reliability of the Korean version of the AAOS foot and ankle questionnaire subscales Scale or Subscale
Questions (n)
First AFAQ Score (n ¼ 152)
Second AFAQ Score (n ¼ 144)
Test–Retest Reliability (n ¼ 144) (ICC)
Internal Reliability (n ¼ 152) (Cronbach’s a)
Global Foot and Ankle scale* Pain Function Stiffness and swelling Giving way Shoe Comfort scale
20 9 6 2 3 5
67 64 6 71 55 46
66 64 6 72 56 46
0.998y 0.999y 0.999y 0.995y 0.979y 0.999y
0.926 0.867 0.868 0.528 0.709 NA
(15 to 100) (15 to 100) (9 to 100) (12 to 100) (0 to 100) (0 to 100)
(15 to 100) (15 to 100) (9 to 100) (12 to 100) (0 to 100) (0 to 100)
Abbreviations: AAOS, American Academy of Orthopaedic Surgeons; AFAQ, AAOS Foot and Ankle Questionnaire; ICC, intraclass correlation coefficient; NA, not applicable. Data presented as standardized mean (range), unless otherwise noted. * The Global Foot and Ankle Scale consists of 4 subscales (pain, function, stiffness and swelling, and giving way). y p < .001.
J.-B. Kim et al. / The Journal of Foot & Ankle Surgery 54 (2015) 46–50
Table 3 Criterion validity (measured using Pearson’s correlation coefficient) between the Global Foot and Ankle scale score and other selected scales Scale Correlation With GFAS
Pearson’s Correlation Coefficient*
AOFAS Hallux-MTP-IP scale (n ¼ 38 patients) AOFAS Ankle-Hindfoot scale (n ¼ 46 patients) VAS Pain (n ¼ 152 patients)
0.492 0.699 0.610
Abbreviations: AOFAS, American Orthopaedic Foot and Ankle Society; GFAS, Global Foot and Ankle Scale; IP, interphalangeal; MTP, metatarsophalangeal; VAS, visual analog scale. * All correlations were statistically significant (p < .01).
We translated the original AFAQ into the Korean language and performed transcultural adaptation using published guidelines to validate the AFAQ in Korea (18,19). Because the AFAQ is a patientassessed form, patients cannot request a professional explanation from the physician when completing the AFAQ. Thus, it is easy for patients to misunderstand an item in the questionnaire. To prevent misunderstandings, we made an effort to use easy-to-understand expressions when we translated the original AFAQ into the Korean language. Despite this effort, patients in the pilot study had difficulty interpreting the term “give way” and confused it with “muscle weakness” or “pain.” This possible misunderstanding by the patients was addressed and discussed in an effort to achieve a consensus. Subsequently, we finalized the Korean version of the AFAQ. Before discussing the results of our study, the study limitations should be mentioned. First, to evaluate the test–retest reliability, we used 2 questionnaires, the second completed 24 hours after the first, although a 1- to 2-week interval (20) and a minimum interval of 48 hours (21) has been recommended to limit remembrance of the previous questionnaire. Although the 24-hour interval decreased the likelihood that the patient’s status would change, it might not have been long enough to reduce the effect of recent memory. We believe, however, that this methodologic limitation could not have been avoided without imparting excessive inconvenience to the study participants. Furthermore, although the AFAQ was designed to reflect the influence that a patient’s foot and ankle has on their status during the previous week, not the previous 1 to 2 days, the control group was composed of patients with acute foot and ankle problems. Hence, our findings could have been subject to biases related to insufficient discriminant validity. To reduce this problem, we highlighted and underlined the words “During the past one week” in the Korean version of the AFAQ. Still further, we used the AOFAS Hallux-MTP-IP scale and AOFAS Ankle-Hindfoot scale to evaluate criterion validity. We did this, with appreciation of the differences between the AOFAS scales and the AFAQ. In particular, the function and alignment subscales of the AOFAS scales are physician-assessed, but the AFAQ is entirely patient-assessed. This difference might have caused a low correlation between the AFAQ and AOFAS scales. We also appreciate the potential biases associated with the comparison of one’s results with those of previously published studies as historical controls and that the comparison of our results with those from the AAOS study group could be limited in this regard. In the AAOS group’s study, the mean standardized score of the original GFAS was 69 (9). In our study, the mean standardized score of the Korean version of the GFAS was 67. It seemed that patients in our study had more severe foot and ankle problems than did those in the AAOS study, and this difference in patient characteristics could have contributed to this observation. First, osteochondral lesions of talus (n ¼ 34) and ankle instability (n ¼ 11) were included in our study but were excluded from the AAOS group’s study. Because patients with these diagnoses presented mainly with symptoms of the ankle “giving way,” the results of our study revealed lower mean standardized scores on the “giving way” subscale (55 in our study, and 89 in the
49
AAOS group’s study). Furthermore, just as with the AAOS group’s study, we thought that the SCS was not an appropriate measure of internal reliability, because the scoring of some items in the SCS (Shoe Comfort scale component of the AFAQ) was contingent on biases inherent in the questionnaire (9). Second, patients who had had chronic foot and ankle problems for at least 1 year were included in the AAOS group’s study. However, in our study, the patients who required surgery to solve their chronic foot and ankle problem were included. Therefore, we believe, the patients in our study might have had more severe foot and ankle problems than those in the AAOS group’s study. These differences in the diagnosis and chronicity of symptoms likely contributed to the statistically significant differences we observed between our study group and those in the AAOS group’s study in the GFAS questionnaire. The test–retest reliability value in our study was more than 0.9 on all subscales. However, the short interval between the first and second times the participants completed the questionnaire might have influenced this high test–retest reliability value. The results of the internal reliability in our study were comparable to those observed in the AAOS group’s study. It is known that Cronbach’s a coefficient should be greater than 0.7 and less than 0.9, because high Cronbach’s a coefficients could mean that the correlated items are redundant (colinear or the same) (22). In our study, the Cronbach’s a coefficient for the Korean version of the GFAS was 0.926, and those for all the other subscales were less than 0.9. In the AAOS group’s study, Cronbach’s a coefficient for the original GFAS was greater than 0.9, similar to the findings in our study. Also, Cronbach’s a coefficient for the stiffness and swelling subscale was the lowest and the only value that was less than 0.7 in both studies. It was 0.528 in our study and 0.61 in the AAOS group’s study. The stiffness and swelling subscale consists of 2 items, 1 for stiffness and 1 for swelling. Because the correlation is low between stiffness and swelling, a low Cronbach’s a coefficient of stiffness and swelling subscale is inevitable. In terms of internal reliability, the results of our study were similar to those of the AAOS group’s study. To assess criterion validity, we calculated Pearson’s correlation coefficients between the Korean version of the GFAS and other previously validated scales (studies know to produce valid information). Because the AOFAS scales have been widely use and previously validated by several groups (5,13), we used 2 of the AOFAS scales to assess the criterion validity of the Korean version of the AFAQ. It is known that moderate correlation is evident when Pearson’s correlation coefficient ranges from 0.3 to 0.7 (23). The physician-assessed function in the AAOS group’s study (i.e., the function subscale and alignment subscales of the AOFAS scales) include physician-assessed elements that do not depend solely on a patient’s subjective assessment of their status. In the AAOS group’s study, the original GFAS had moderate correlation with physician-assessed function (r ¼ .43). In our study, the Korean version of the GFAS also showed a correlation with the AOFAS Hallux-MTP-IP scale (r ¼ .492) and the AOFAS Ankle-Hindfoot scale (r ¼ .699). The VAS pain scale and Short Form-36 questionnaire are patient-assessed forms. In the AAOS group’s study, the original GFAS had moderate correlation with the Short Form-36 questionnaire (r ¼ .65). In our study, the Korean version of the GFAS displayed a moderate inverse correlation with the VAS pain scale (r ¼ .610; again, inverse because better foot function is associated with a higher GFAS score and lower VAS pain score). The Korean version of the AFAQ showed moderate correlation with the other validated scales, and this finding was comparable to that of the AAOS group’s study in connection with both the physician-assessed form and the patientassessed form. To assess discriminant validity, we compared the results of the Korean version of the GFAS between the experimental and control groups. The experimental group had a lower mean standardized score in the Korean version of the GFAS than did the control group, and this difference was statistically significant (p < .0001). This
50
J.-B. Kim et al. / The Journal of Foot & Ankle Surgery 54 (2015) 46–50
result suggested that the patients in the experimental group had more severe foot and ankle conditions than did those in the control group and that the Korean version of the AFAQ could distinguish these differences. Also, 29 patients who revisited our foot and ankle clinic postoperatively showed higher postoperative results with the Korean version of the AFAQ when the postoperative and preoperative results were compared. This difference was also statistically significant (p ¼ .021). This result showed the responsiveness of the Korean version of the AFAQ. The AAOS group did not study the discriminant validity and the responsiveness of the original AFAQ, and we believe that these 2 findings were a strong point of our study. From these observations, we determined that the Korean version of the AFAQ has satisfactory validity and responsiveness. We translated and transculturally adapted the original AFAQ into the Korean language using strict international guidelines. After finalizing the Korean version of the AFAQ, we accepted that each item in the Korean version of the AFAQ conveyed the same concept and intention as the original English-language version. The test–retest reliability of each subscale was satisfactory, and so too was the internal reliability of each subscale. In terms of criterion validity, our study showed moderate correlations between the Korean version of the AFAQ and other validated scales. Discriminant validity and responsiveness were evident in our study. The results of our study showed that the Korean version of the AFAQ had satisfactory reliability, validity, and responsiveness for patients with foot and ankle problems. In conclusion, after translation and transcultural adaptation, we considered that each item of the Korean version of the AFAQ had the same concept and intention as the original version. We have also demonstrated that the Korean version of the AFAQ has satisfactory reliability, validity, and responsiveness. We believe that the Korean version of the AFAQ can be used as an instrument to assess subjective function in patients with foot and ankle disorders. Acknowledgments We thank Hyo Ji Ma for collecting data from the study participants. References 1. Thomas MJ, Roddy E, Zhang W, Menz HB, Hannan MT, Peat GM. The population prevalence of foot and ankle pain in middle and old age: a systematic review. Pain 152:2870–2880, 2011. 2. Balint GP, Korda J, Hangody L, Balint PV. Regional musculoskeletal conditions: foot and ankle disorders. Best Pract Res Clin Rheumatol 17:87–111, 2003. 3. Janwantanakul P, Pensri P, Jiamjarasrangsi W, Sinsongsook T. Biopsychosocial factors are associated with high prevalence of self-reported musculoskeletal
4.
5.
6.
7. 8.
9.
10.
11. 12.
13.
14.
15. 16. 17.
18.
19. 20.
21. 22. 23.
symptoms in the lower extremities among office workers. Arch Med Res 40: 216–222, 2009. Sangeorzan BJ, Beskin JL, Britt SM, Brage M, Holt S, Kadel NJ, Legro MW, Sands AK, SooHoo NF, Stroud CC. Issues of importance to patients seeking care from members of the AOFAS: a preliminary report of the outcomes committee of the AOFAS. Foot Ankle Int 26:638–644, 2005. Button G, Pinney S. A meta-analysis of outcome rating scales in foot and ankle surgery: is there a valid, reliable, and responsive system? Foot Ankle Int 25: 521–525, 2004. Kitaoka HB, Alexander IJ, Adelaar RS, Nunley JA, Myerson MS, Sanders M. Clinical rating systems for the ankle-hindfoot, midfoot, hallux, and lesser toes. Foot Ankle Int 15:349–353, 1994. Budiman-Mak E, Conrad KJ, Roach KE. The Foot Function Index: a measure of foot pain and disability. J Clin Epidemiol 44:561–570, 1991. Martin RL, Irrgang JJ, Burdett RG, Conti SF, Van Swearingen JM. Evidence of validity for the Foot and Ankle Ability Measure (FAAM). Foot Ankle Int 26: 968–983, 2005. Johanson NA, Liang MH, Daltroy L, Rudicel S, Richmond J. American Academy of Orthopaedic Surgeons Lower Limb outcomes assessment instruments: reliability, validity, and sensitivity to change. J Bone Joint Surg Am 86: 902–909, 2004. Morris C, Liabo K, Wright P, Fitzpatrick R. Development of the Oxford ankle foot questionnaire: finding out how children are affected by foot and ankle problems. Child Care Health Dev 33:559–568, 2007. Ware JE Jr, Sherbourne CD. The MOS 36-item short-form health survey (SF-36). I: conceptual framework and item selection. Med Care 30:473–483, 1992. Toolan BC, Wright Quinones VJ, Cunningham BJ, Brage ME. An evaluation of the use of retrospectively acquired preoperative AOFAS clinical rating scores to assess surgical outcome after elective foot and ankle surgery. Foot Ankle Int 22:775–778, 2001. Ibrahim T, Beiri A, Azzabi M, Best AJ, Taylor GJ, Menon DK. Reliability and validity of the subjective component of the American Orthopaedic Foot and Ankle Society clinical rating scales. J Foot Ankle Surg 46:65–74, 2007. Madeley NJ, Wing KJ, Topliss C, Penner MJ, Glazebrook MA, Younger AS. Responsiveness and validity of the SF-36, Ankle Osteoarthritis scale, AOFAS Ankle Hindfoot score, and Foot Function Index in end stage ankle arthritis. Foot Ankle Int 33:57–63, 2012. SooHoo NF, Shuler M, Fleming LL. Evaluation of the validity of the AOFAS clinical rating systems by correlation to the SF-36. Foot Ankle Int 24:50–55, 2003. Guyton GP. Theoretical limitations of the AOFAS scoring systems: an analysis using Monte Carlo modeling. Foot Ankle Int 22:779–787, 2001. Baumhauer JF, Nawoczenski DA, DiGiovanni BF, Wilding GE. Reliability and validity of the American Orthopaedic Foot and Ankle Society clinical rating scale: a pilot study for the hallux and lesser toes. Foot Ankle Int 27:1014–1019, 2006. Guillemin F, Bombardier C, Beaton D. Cross-cultural adaptation of health-related quality of life measures: literature review and proposed guidelines. J Clin Epidemiol 46:1417–1432, 1993. Beaton DE, Bombardier C, Guillemin F, Ferraz MB. Guidelines for the process of cross-cultural adaptation of self-report measures. Spine 25:3186–3191, 2000. Deyo RA, Andersson G, Bombardier C, Cherkin DC, Keller RB, Lee CK, Liang MH, Lipscomb B, Shekelle P, Spratt KF. Outcome measures for studying patients with low back pain. Spine 19:2032S–2036S, 1994. Dawson J, Fitzpatrick R, Carr A. Questionnaire on the perceptions of patients about shoulder surgery. J Bone Joint Surg Br 78:593–600, 1996. Fitzpatrick R, Davey C, Buxton MJ, Jones DR. Evaluating patient-based outcome measures for use in clinical trials. Health Technol Assess 2, 1998. i–iv, 1–74. Engelberg R, Martin DP, Agel J, Obremsky W, Coronado G, Swiontkowski MF. Musculoskeletal function assessment instrument: criterion and construct validity. J Orthop Res 14:182–192, 1996.
Contents lists available at ScienceDirect
The Journal of Foot & Ankle Surgery journal homepage: www.jfas.org
Validity, Reliability, and Responsiveness of the Korean Version of American Academy of Orthopedic Surgeons Foot and Ankle Questionnaire Ji-Beom Kim, MD 1, Je Kyun Kim, MD 1, Sang Gyo Seo, MD 2, Dong Yeon Lee, MD 3 1
Orthopedic Surgeon, Department of Orthopedic Surgery, Seoul National University Hospital, Yeongeon-dong, Jongno-gu, Seoul, Korea Instructor, Department of Orthopedic Surgery, Seoul National University Hospital, Yeongeon-dong, Jongno-gu, Seoul, Korea 3 Associate Professor (Clinical), Department of Orthopedic Surgery, Seoul National University Hospital, Yeongeon-dong, Jongno-gu, Seoul, Korea 2
a r t i c l e i n f o
a b s t r a c t
Level of Clinical Evidence: 3
The American Academy of Orthopaedic Surgeons Foot and Ankle Questionnaire (AFAQ) reflects patients’ subjective disorder due to foot and ankle conditions. We evaluated the validity, reliability, and responsiveness of the Korean version of the AFAQ, after translation and transcultural adaptation of the original AFAQ into the Korean language. A total of 206 patients were enrolled, including 152 with chronic problems (experimental group) and 54 with acute problems (control group). We used the intraclass correlation coefficient to assess the test–retest reliability and Cronbach’s a to assess internal reliability. Pearson’s correlation coefficient was used to assess the criterion validity by correlating the Korean AFAQ scores with those from other validated scales (American Orthopaedic Foot and Ankle Society Hallux-Metatarsophalangeal-Interphalangeal scale, American Orthopaedic Foot and Ankle Society Ankle-Hindfoot scale, and visual analog scale for pain). To analyze discriminant validity, we evaluated the difference between the experimental and control groups using the Student t test. Of the 152 patients in the experimental group, 29 revisited our clinic postoperatively and repeated the Korean AFAQ. To analyze responsiveness, we used paired t tests to evaluate postoperative changes. In terms of test–retest reliability, the intraclass correlation coefficient ranged from 0.979 to 0.999. In terms of internal reliability, Cronbach’s a was 0.528 for the stiffness and swelling subscale and greater than 0.7 for all other subscales. In terms of criterion validity, Pearson’s correlation coefficient ranged from 0.492 to 0.699. The probability of the null hypothesis for discriminant validity and responsiveness was statistically significant (p < .001 and p ¼ .021, respectively). These results showed that the Korean version of the AFAQ had the same concept and intention as the original version and is reliable, valid, and responsive. Ó 2015 by the American College of Foot and Ankle Surgeons. All rights reserved.
Keywords: AAOS discriminant validity outcome assessment test-retest reliability transcultural adaptation translation VAS pain
Disorders involving the foot and ankle increase with age owing to degenerative changes involving bone, joint, neuromuscular tissues, and tendon (1). Active lifestyles, shoe gear, and an increased body mass can influence the prevalence of foot and ankle disorders (2). In fact, foot and ankle disorders are usually not correlated directly to age; rather, they correlate with quality of life. However, assessing the effect of foot and ankle disorders on each individual can be quite difficult, because, even in cases of similar severity, they could have different effects, depending on the lifestyle of the patient (3). Financial Disclosure: This study was supported by Basic Science Research Program through the National Research Foundation of Korea funded by the Ministry of Science, ICT and Future Planning (grant NRF-2012R1A1A2044067). Conflict of Interest: None reported. Address correspondence to: Dong Yeon Lee, MD, Associate Professor (Clinical), Department of Orthopedic Surgery, Seoul National University Hospital, Yeongeondong, Jongno-gu, Seoul, Korea, 110-744. E-mail address: [email protected] (D.Y. Lee).
To assess foot and ankle problems, determining the subjective malfunction and severity of pain is as important as finding the anatomic malformation (4). Various questionnaires have been used as an instrument to measure the subjective discomfort due to the disorders and/or the improvement after treatment (5–10). However, no reference standard instrument has yet been determined for assessment of the foot and ankle. For example, the RAND Medical Outcomes Study Short Form-36 questionnaire was supposed to represent the subjective discomfort of patients well, but it does not focus on foot and ankle problems (11) but, instead, is a general health measurement. The American Orthopaedic Foot and Ankle Society (AOFAS) scales focus on foot and ankle problems and seem to be the most widely used scales in foot and ankle research in recent years (6). The AOFAS scale studies reported the scales could evaluate the outcomes of surgery (12), and their subjective component had acceptable validity (13). However, other studies have reported that the AOFAS scales are difficult to perform and lack reliability (14), validity, and precision (15–17).
1067-2516/$ - see front matter Ó 2015 by the American College of Foot and Ankle Surgeons. All rights reserved. http://dx.doi.org/10.1053/j.jfas.2014.08.011
J.-B. Kim et al. / The Journal of Foot & Ankle Surgery 54 (2015) 46–50
The American Academy of Orthopaedic Surgeons (AAOS) (9) developed an instrument for subjective patient assessment (i.e., the AAOS Lower Limb Instrument), and the AAOS Foot and Ankle Questionnaire (AFAQ) is one of the major parts of the AAOS Lower Limb Instrument, focusing on foot and ankle problems. The AFAQ is a self-administered questionnaire designed for efficient collection of patient-oriented data. It consists of 25 items and combines the Global Foot and Ankle scale (GFAS) and the Shoe Comfort scale (SCS). The GFAS is formatted into 4 subscales that address pain, function, stiffness and swelling, and “giving way” (9). “Giving way” of an ankle means the ankle is unable to support the person’s weight unexpectedly and suddenly. The SCS is formatted into 5 questions about the type of shoes that are comfortable to the patient. An assessment instrument should be examined for reliability, validity, and responsiveness. After an assessed instrument has been determined to be reliable, valid, and responsive, physicians and surgeons can trust the instrument and use it. The AFAQ was expected to reflect a patient’s subjective discomfort well and to produce meaningful data. Although AAOS group insisted that the AFAQ showed high reliability and validity (9), similar support for the AFAQ in the published data, to our knowledge, is nonexistent. The original AFAQ was developed in the English language and was intended for use in English-speaking countries (9). To validate the AFAQ in Korea, the original AFAQ had to be translated into the Korean language. Although it is not easy to translate an English-language questionnaire into another language without losing some of the meaning in translation, we created the Korean version of the AFAQ with adaptation of the cultural nuances in accordance with internationally published recommendations (18,19). The purpose of the present report was to demonstrate the reliability, validity, and responsiveness of the Korean version of the AFAQ. Patients and Methods Our hospital’s institutional review board approved the present study, and all enrolled patients provided written informed consent. Before beginning the present study, we ensured that the original AFAQ had not already been translated into the Korean language. In addition, we contacted Johanson, the author of the AAOS Lower Limb Instrument (9), and he granted us permission to produce the Korean version of the AFAQ. We performed 2 steps in the present study. The first step was to create the Korean version of the AFAQ by translation and transcultural adaptation. The second step was to examine the reliability, validity, and responsiveness of the Korean version of the AFAQ. The original AFAQ was translated into Korean by 2 Koreans, an orthopedic surgeon and a nonmedical person. Both translators were native Koreans and fluent in English. They created the Korean version of the AFAQ separately. To reconcile these 2 Korean versions of the AFAQ, the 2 translators and 1 Korean nurse met and discussed the translations. After producing a reconciled Korean version of the AFAQ, 2 bilingual speakers (Korean–English) translated the reconciled Korean version of the AFAQ into English together, without any background information pertaining to the original AFAQ. A committee consisting of 2 back translators and 2 forward translators met to compare the back-translated version of the AFAQ with the original AFAQ. The committee determined that some expressions in the back-translated version of the AFAQ were not equivalent to those of the original AFAQ. To improve these expressions, the reconciled Korean version of the AFAQ was revised. Finally, 20 Korean outpatients from our clinic reviewed the revised Korean version of the AFAQ and were asked whether they had any problems in understanding and answering the revised Korean version. After the pilot study, several ambiguous expressions were modified further. Errors in the revised Korean version of the AFAQ were then corrected, and the Korean version was finalized. To conduct the second step of our investigation, we recruited patients who had undergone an operation at our foot and ankle clinic from January 2012 to March 2013. The patients who had had a chronic foot and ankle problem for more than 1 year were enrolled in the experimental group, and those with an acute foot and ankle problem that had been present for less than 1 month were enrolled in the control group. After admission, the patients enrolled in our study completed the Korean version of the AFAQ for the first time and the visual analog scale for pain (VAS pain). Twenty-four hours later, they completed the Korean version of the AFAQ for the second time. An orthopedic surgeon investigator then visited the patients in the experimental group and completed the other, previously validated, foot-related quality-of-life scales. If patients had a forefoot problem, the AOFAS Hallux-Metatarsophalangeal-Interphalangeal (Hallux-MTP-IP) scale was completed. If patients had a hindfoot or ankle problem,
47
the AOFAS Ankle-Hindfoot scale was completed. After leaving the hospital, the patients in the experimental group visited our clinics and again completed the Korean version of the AFAQ. The patients in the control group completed the Korean AFAQ 1 time at admission. The patients in the experimental group completed the Korean AFAQ 3 times, at admission, 24 hours after completing the Korean AFAQ at admission, and after the operation. We assessed the test–retest reliability and internal reliability to understand the reliability of the Korean version of the AFAQ. Test–retest reliability was evaluated for 2 of the questionnaires, which were completed at a 24-hour interval by the patients in the experimental group before surgery. We calculated the intraclass correlation coefficient as a measure of test–retest reliability. Internal reliability was evaluated for the different items in each of the subscales. We calculated Cronbach’s a coefficient to understand the internal reliability of the Korean version of the AFAQ. The criterion validity and discriminant validity were also assessed to understand the validity of the Korean version of the AFAQ. Criterion validity was analyzed using the correlation between the mean standardized score of the Korean version of the GFAS and other validated scales, including the AOFAS Hallux-MTP-IP scale, AOFAS Ankle-Hindfoot scale, and VAS pain scale. We calculated the Pearson correlation coefficients to assess criterion validity. To assess discriminant validity, we used Student’s t test to determine the difference in the mean standardized score of the Korean version of the AFAQ between the control and experimental groups. Responsiveness indicates the sensitiveness of an instrument to detect clinically meaningful changes in the state of patients. Postoperatively, the patients visited our clinic again to check their status. They also completed the Korean version of the AFAQ a second time. We used paired t tests to compare the differences between the preoperative mean standardized score of the Korean version of the GFAS and the postoperative mean standardized score of the Korean version of the GFAS. The Statistical Package for Social Sciences, version 18.0 (SPSS, Chicago, IL), was used for statistical analysis.
Results A total of 206 patients were included in our study, of whom 152 (73.79%) were enrolled in the experimental group and 54 (26.21%) in the control group. In the experimental group, 152 patients (73.79%) provided us with useable questionnaire information from the first test after enrollment. After a 24-hour interval, of the 152 patients in the experimental group, 144 (94.73% of the experimental group) provided useable questionnaire information from the second test. All 54 patients (26.21%) in the control group provided useable information. All 152 patients in the experimental group provided VAS pain scale data, 38 (25.00% of the experimental group) provided chronic forefoot information by completing the AOFAS Hallux-MTP-IP scale, and 46 (30.26% of the experimental group) provided chronic ankle and/or hindfoot information by completing the AOFAS Ankle-Hindfoot scale. In the experimental group, 29 patients (19.07% of experimental group) visited our clinic again after the operation and completed the Korean version of the AFAQ. The mean average interval from surgery to the follow-up visit when the Korean AFAQ was completed again was 8.2 (range 5.1 to 13.9) months. Of the 152 patients in the experimental group, 61 were male (40.13% of the experimental group) and 91 were female (59.87% of the experimental group), and their mean age was 49 (range 14 to 81) years. Of the 54 patients in the control group, 31 were male (57.41% of the control group) and 23 were female (42.59% of the control group), and their mean age was 46 (range 17 to 76) years. All the patients enrolled in our study were literate and appeared to understand the questionnaires. A statistical description of the prevalence of the diagnoses by patient group is given in Table 1. Test–retest reliability was calculated for the 144 patients (69.9%) in the experimental group who had provided us useable information after completing the questionnaire twice. Internal reliability was calculated for the 152 patients (73.79%) in the experimental group who had provided us with useable questionnaire results after completing the questionnaire the first time after enrollment into the study. The mean standardized scores of the subscales and statistical reliability coefficients are presented in Table 2. The test–retest reliability was considered excellent, because the intraclass correlation coefficient ranged from 0.979 to 0.999. Internal reliability was considered moderate to high, because Cronbach’s a ranged from 0.528
48
J.-B. Kim et al. / The Journal of Foot & Ankle Surgery 54 (2015) 46–50
Discussion
Table 1 Distribution of diagnoses (N ¼ 206 patients) Diagnosis
n (%)
Experimental group (n ¼ 152) Hallux valgus Osteochondral lesion of the talus Mass* Osteoarthritis Ankle instability Foot deformity Nonunion Chronic osteomyelitis Miscellaneousy Control group (n ¼ 54) Fracture Metal removal Miscellaneousy * y
32 23 22 19 11 10 6 6 23
(15.53) (11.17) (10.68) (9.22) (5.34) (4.85) (2.91) (2.91) (11.17)
37 (17.96) 11 (5.34) 6 (2.91)
Bone or soft tissue mass. Diagnoses that included fewer than 5 patients.
to 0.926. The stiffness and swelling subscale showed the lowest Cronbach’s a coefficient, which was 0.528. To assess criterion validity, the results of the Korean version of the AFAQ were compared with those of the AOFAS Hallux-MTP-IP scale, AOFAS Ankle-Hindfoot scale, and the VAS pain scale. To evaluate the correlation between the Korean version of the AFAQ and the other validated scales, we used the Korean version of the GFAS; the measures of criterion validity are presented in Table 3. The correlation between the Korean version of the GFAS and the 2 AOFAS scales were considered moderate, with a Pearson’s correlation coefficient of 0.492 and 0.699, respectively. The correlation between the Korean version of the AFAQ and the VAS pain scale was considered moderate, with a Pearson’s correlation coefficient of 0.687. The higher score of the Koran version of the AFAQ indicated a less severe problem; in contrast, a higher VAS pain score indicated more severe pain. Hence, a moderate inverse correlation coefficient (0.687) was calculated in the present study. To assess discriminant validity, we compared the results of the Korean version of the GFAS between the experimental and control groups using unpaired Student’s t test. The mean standardized score of the Korean version of the GFAS in the experimental group was 67 (range 15 to 100) and that in the control group was 83 (range 41 to 100); this difference was statistically significant (p < .001). To assess the responsiveness of the Korean version of the AFAQ, we compared the results of the Korean version of the GFAS from preoperatively and postoperatively, using paired Student’s t tests for 29 patients (19.07% of the experimental group). The mean standardized score for the preoperative Korean version of the GFAS was 65 (range 15 to 93) and that postoperatively was 77 (range 50 to 100); this difference was also statistically significant (p ¼ .021).
We have demonstrated that the Korean version of the AFAQ, after translation and transcultural adaptation from its original English version, is a reliable instrument for evaluation of the subjective functional status in patients with disorders involving the foot and ankle. Diverse clinical outcome questionnaires for measuring foot and ankle problems have been invented and refined. For these to be useful in the clinical setting, they must demonstrate reliability, validity, and responsiveness. Several groups have reported that the AOFAS scales are reliable, valid, and responsive to change (6,9,13). Thus, the AOFAS scales have become widely used (5). Experienced foot and ankle surgeons developed the AOFAS scales, and their purpose was to provide a reliable measurement of the influence that foot and ankle problems have on an individual (6). The AOFAS scales include 4 different scales that focus on different anatomic regions: the Ankle-Hindfoot, Midfoot, Hallux-MTP-IP, and MTP-IP-Lesser Toe scale. Each scale can be used independently and has 3 subscales (i.e., pain, function, and alignment subscales). The subscales are weighted differently in the AOFAS Ankle-Hindfoot scale than in the other scales. The pain subscale has 1 subjective question in each scale. However, the function subscale has a different number of questions in each scale. The Ankle-Hindfoot scale has 7 questions, the Midfoot scale has 6, and the Hallux-MTP-IP and MTP-IP-Lesser Toe scales both have 5 questions in the function subscale. The alignment subscale consists of 1 question. Patients can complete the pain subscale alone; however, patients cannot complete the function subscale and the alignment subscale without the help of a physician. Each of the AOFAS scales consists of subjective questions related to discomfort and objective elements related to function. Although the AOFAS scales have been widely used, the objective symptom elements have some limitations. First, they require the presence of a physician. Second, the data obtained with these questions might not accurately represent the patient’s status, because the physician’s expectations could influence the data. Third, if the physician completing these objective elements has not been properly trained, the reliability of the results could be threatened (14). In contrast, the AFAQ consists entirely of questions related to subjective discomfort; thus, it is advantageous regarding the procurement of data that effectively measures the patient’s status. Although many foot and ankle surgeons expected the AFAQ to be an alternative health measurement instrument to the AOFAS scales, and although the AFAQ has been shown to be reliable, valid, and responsive to change (9), follow-up studies reinforcing the validity of the AFAQ are nonexistent to our knowledge. Thus, we believe the AFAQ has not been used as widely as the AOFAS scales. In the present report, we have described our findings related to the reliability, validity, and responsiveness of the Korean version of the AFAQ, and we compared our results with those of the AAOS group’s study.
Table 2 Internal reliability and test–retest reliability of the Korean version of the AAOS foot and ankle questionnaire subscales Scale or Subscale
Questions (n)
First AFAQ Score (n ¼ 152)
Second AFAQ Score (n ¼ 144)
Test–Retest Reliability (n ¼ 144) (ICC)
Internal Reliability (n ¼ 152) (Cronbach’s a)
Global Foot and Ankle scale* Pain Function Stiffness and swelling Giving way Shoe Comfort scale
20 9 6 2 3 5
67 64 6 71 55 46
66 64 6 72 56 46
0.998y 0.999y 0.999y 0.995y 0.979y 0.999y
0.926 0.867 0.868 0.528 0.709 NA
(15 to 100) (15 to 100) (9 to 100) (12 to 100) (0 to 100) (0 to 100)
(15 to 100) (15 to 100) (9 to 100) (12 to 100) (0 to 100) (0 to 100)
Abbreviations: AAOS, American Academy of Orthopaedic Surgeons; AFAQ, AAOS Foot and Ankle Questionnaire; ICC, intraclass correlation coefficient; NA, not applicable. Data presented as standardized mean (range), unless otherwise noted. * The Global Foot and Ankle Scale consists of 4 subscales (pain, function, stiffness and swelling, and giving way). y p < .001.
J.-B. Kim et al. / The Journal of Foot & Ankle Surgery 54 (2015) 46–50
Table 3 Criterion validity (measured using Pearson’s correlation coefficient) between the Global Foot and Ankle scale score and other selected scales Scale Correlation With GFAS
Pearson’s Correlation Coefficient*
AOFAS Hallux-MTP-IP scale (n ¼ 38 patients) AOFAS Ankle-Hindfoot scale (n ¼ 46 patients) VAS Pain (n ¼ 152 patients)
0.492 0.699 0.610
Abbreviations: AOFAS, American Orthopaedic Foot and Ankle Society; GFAS, Global Foot and Ankle Scale; IP, interphalangeal; MTP, metatarsophalangeal; VAS, visual analog scale. * All correlations were statistically significant (p < .01).
We translated the original AFAQ into the Korean language and performed transcultural adaptation using published guidelines to validate the AFAQ in Korea (18,19). Because the AFAQ is a patientassessed form, patients cannot request a professional explanation from the physician when completing the AFAQ. Thus, it is easy for patients to misunderstand an item in the questionnaire. To prevent misunderstandings, we made an effort to use easy-to-understand expressions when we translated the original AFAQ into the Korean language. Despite this effort, patients in the pilot study had difficulty interpreting the term “give way” and confused it with “muscle weakness” or “pain.” This possible misunderstanding by the patients was addressed and discussed in an effort to achieve a consensus. Subsequently, we finalized the Korean version of the AFAQ. Before discussing the results of our study, the study limitations should be mentioned. First, to evaluate the test–retest reliability, we used 2 questionnaires, the second completed 24 hours after the first, although a 1- to 2-week interval (20) and a minimum interval of 48 hours (21) has been recommended to limit remembrance of the previous questionnaire. Although the 24-hour interval decreased the likelihood that the patient’s status would change, it might not have been long enough to reduce the effect of recent memory. We believe, however, that this methodologic limitation could not have been avoided without imparting excessive inconvenience to the study participants. Furthermore, although the AFAQ was designed to reflect the influence that a patient’s foot and ankle has on their status during the previous week, not the previous 1 to 2 days, the control group was composed of patients with acute foot and ankle problems. Hence, our findings could have been subject to biases related to insufficient discriminant validity. To reduce this problem, we highlighted and underlined the words “During the past one week” in the Korean version of the AFAQ. Still further, we used the AOFAS Hallux-MTP-IP scale and AOFAS Ankle-Hindfoot scale to evaluate criterion validity. We did this, with appreciation of the differences between the AOFAS scales and the AFAQ. In particular, the function and alignment subscales of the AOFAS scales are physician-assessed, but the AFAQ is entirely patient-assessed. This difference might have caused a low correlation between the AFAQ and AOFAS scales. We also appreciate the potential biases associated with the comparison of one’s results with those of previously published studies as historical controls and that the comparison of our results with those from the AAOS study group could be limited in this regard. In the AAOS group’s study, the mean standardized score of the original GFAS was 69 (9). In our study, the mean standardized score of the Korean version of the GFAS was 67. It seemed that patients in our study had more severe foot and ankle problems than did those in the AAOS study, and this difference in patient characteristics could have contributed to this observation. First, osteochondral lesions of talus (n ¼ 34) and ankle instability (n ¼ 11) were included in our study but were excluded from the AAOS group’s study. Because patients with these diagnoses presented mainly with symptoms of the ankle “giving way,” the results of our study revealed lower mean standardized scores on the “giving way” subscale (55 in our study, and 89 in the
49
AAOS group’s study). Furthermore, just as with the AAOS group’s study, we thought that the SCS was not an appropriate measure of internal reliability, because the scoring of some items in the SCS (Shoe Comfort scale component of the AFAQ) was contingent on biases inherent in the questionnaire (9). Second, patients who had had chronic foot and ankle problems for at least 1 year were included in the AAOS group’s study. However, in our study, the patients who required surgery to solve their chronic foot and ankle problem were included. Therefore, we believe, the patients in our study might have had more severe foot and ankle problems than those in the AAOS group’s study. These differences in the diagnosis and chronicity of symptoms likely contributed to the statistically significant differences we observed between our study group and those in the AAOS group’s study in the GFAS questionnaire. The test–retest reliability value in our study was more than 0.9 on all subscales. However, the short interval between the first and second times the participants completed the questionnaire might have influenced this high test–retest reliability value. The results of the internal reliability in our study were comparable to those observed in the AAOS group’s study. It is known that Cronbach’s a coefficient should be greater than 0.7 and less than 0.9, because high Cronbach’s a coefficients could mean that the correlated items are redundant (colinear or the same) (22). In our study, the Cronbach’s a coefficient for the Korean version of the GFAS was 0.926, and those for all the other subscales were less than 0.9. In the AAOS group’s study, Cronbach’s a coefficient for the original GFAS was greater than 0.9, similar to the findings in our study. Also, Cronbach’s a coefficient for the stiffness and swelling subscale was the lowest and the only value that was less than 0.7 in both studies. It was 0.528 in our study and 0.61 in the AAOS group’s study. The stiffness and swelling subscale consists of 2 items, 1 for stiffness and 1 for swelling. Because the correlation is low between stiffness and swelling, a low Cronbach’s a coefficient of stiffness and swelling subscale is inevitable. In terms of internal reliability, the results of our study were similar to those of the AAOS group’s study. To assess criterion validity, we calculated Pearson’s correlation coefficients between the Korean version of the GFAS and other previously validated scales (studies know to produce valid information). Because the AOFAS scales have been widely use and previously validated by several groups (5,13), we used 2 of the AOFAS scales to assess the criterion validity of the Korean version of the AFAQ. It is known that moderate correlation is evident when Pearson’s correlation coefficient ranges from 0.3 to 0.7 (23). The physician-assessed function in the AAOS group’s study (i.e., the function subscale and alignment subscales of the AOFAS scales) include physician-assessed elements that do not depend solely on a patient’s subjective assessment of their status. In the AAOS group’s study, the original GFAS had moderate correlation with physician-assessed function (r ¼ .43). In our study, the Korean version of the GFAS also showed a correlation with the AOFAS Hallux-MTP-IP scale (r ¼ .492) and the AOFAS Ankle-Hindfoot scale (r ¼ .699). The VAS pain scale and Short Form-36 questionnaire are patient-assessed forms. In the AAOS group’s study, the original GFAS had moderate correlation with the Short Form-36 questionnaire (r ¼ .65). In our study, the Korean version of the GFAS displayed a moderate inverse correlation with the VAS pain scale (r ¼ .610; again, inverse because better foot function is associated with a higher GFAS score and lower VAS pain score). The Korean version of the AFAQ showed moderate correlation with the other validated scales, and this finding was comparable to that of the AAOS group’s study in connection with both the physician-assessed form and the patientassessed form. To assess discriminant validity, we compared the results of the Korean version of the GFAS between the experimental and control groups. The experimental group had a lower mean standardized score in the Korean version of the GFAS than did the control group, and this difference was statistically significant (p < .0001). This
50
J.-B. Kim et al. / The Journal of Foot & Ankle Surgery 54 (2015) 46–50
result suggested that the patients in the experimental group had more severe foot and ankle conditions than did those in the control group and that the Korean version of the AFAQ could distinguish these differences. Also, 29 patients who revisited our foot and ankle clinic postoperatively showed higher postoperative results with the Korean version of the AFAQ when the postoperative and preoperative results were compared. This difference was also statistically significant (p ¼ .021). This result showed the responsiveness of the Korean version of the AFAQ. The AAOS group did not study the discriminant validity and the responsiveness of the original AFAQ, and we believe that these 2 findings were a strong point of our study. From these observations, we determined that the Korean version of the AFAQ has satisfactory validity and responsiveness. We translated and transculturally adapted the original AFAQ into the Korean language using strict international guidelines. After finalizing the Korean version of the AFAQ, we accepted that each item in the Korean version of the AFAQ conveyed the same concept and intention as the original English-language version. The test–retest reliability of each subscale was satisfactory, and so too was the internal reliability of each subscale. In terms of criterion validity, our study showed moderate correlations between the Korean version of the AFAQ and other validated scales. Discriminant validity and responsiveness were evident in our study. The results of our study showed that the Korean version of the AFAQ had satisfactory reliability, validity, and responsiveness for patients with foot and ankle problems. In conclusion, after translation and transcultural adaptation, we considered that each item of the Korean version of the AFAQ had the same concept and intention as the original version. We have also demonstrated that the Korean version of the AFAQ has satisfactory reliability, validity, and responsiveness. We believe that the Korean version of the AFAQ can be used as an instrument to assess subjective function in patients with foot and ankle disorders. Acknowledgments We thank Hyo Ji Ma for collecting data from the study participants. References 1. Thomas MJ, Roddy E, Zhang W, Menz HB, Hannan MT, Peat GM. The population prevalence of foot and ankle pain in middle and old age: a systematic review. Pain 152:2870–2880, 2011. 2. Balint GP, Korda J, Hangody L, Balint PV. Regional musculoskeletal conditions: foot and ankle disorders. Best Pract Res Clin Rheumatol 17:87–111, 2003. 3. Janwantanakul P, Pensri P, Jiamjarasrangsi W, Sinsongsook T. Biopsychosocial factors are associated with high prevalence of self-reported musculoskeletal
4.
5.
6.
7. 8.
9.
10.
11. 12.
13.
14.
15. 16. 17.
18.
19. 20.
21. 22. 23.
symptoms in the lower extremities among office workers. Arch Med Res 40: 216–222, 2009. Sangeorzan BJ, Beskin JL, Britt SM, Brage M, Holt S, Kadel NJ, Legro MW, Sands AK, SooHoo NF, Stroud CC. Issues of importance to patients seeking care from members of the AOFAS: a preliminary report of the outcomes committee of the AOFAS. Foot Ankle Int 26:638–644, 2005. Button G, Pinney S. A meta-analysis of outcome rating scales in foot and ankle surgery: is there a valid, reliable, and responsive system? Foot Ankle Int 25: 521–525, 2004. Kitaoka HB, Alexander IJ, Adelaar RS, Nunley JA, Myerson MS, Sanders M. Clinical rating systems for the ankle-hindfoot, midfoot, hallux, and lesser toes. Foot Ankle Int 15:349–353, 1994. Budiman-Mak E, Conrad KJ, Roach KE. The Foot Function Index: a measure of foot pain and disability. J Clin Epidemiol 44:561–570, 1991. Martin RL, Irrgang JJ, Burdett RG, Conti SF, Van Swearingen JM. Evidence of validity for the Foot and Ankle Ability Measure (FAAM). Foot Ankle Int 26: 968–983, 2005. Johanson NA, Liang MH, Daltroy L, Rudicel S, Richmond J. American Academy of Orthopaedic Surgeons Lower Limb outcomes assessment instruments: reliability, validity, and sensitivity to change. J Bone Joint Surg Am 86: 902–909, 2004. Morris C, Liabo K, Wright P, Fitzpatrick R. Development of the Oxford ankle foot questionnaire: finding out how children are affected by foot and ankle problems. Child Care Health Dev 33:559–568, 2007. Ware JE Jr, Sherbourne CD. The MOS 36-item short-form health survey (SF-36). I: conceptual framework and item selection. Med Care 30:473–483, 1992. Toolan BC, Wright Quinones VJ, Cunningham BJ, Brage ME. An evaluation of the use of retrospectively acquired preoperative AOFAS clinical rating scores to assess surgical outcome after elective foot and ankle surgery. Foot Ankle Int 22:775–778, 2001. Ibrahim T, Beiri A, Azzabi M, Best AJ, Taylor GJ, Menon DK. Reliability and validity of the subjective component of the American Orthopaedic Foot and Ankle Society clinical rating scales. J Foot Ankle Surg 46:65–74, 2007. Madeley NJ, Wing KJ, Topliss C, Penner MJ, Glazebrook MA, Younger AS. Responsiveness and validity of the SF-36, Ankle Osteoarthritis scale, AOFAS Ankle Hindfoot score, and Foot Function Index in end stage ankle arthritis. Foot Ankle Int 33:57–63, 2012. SooHoo NF, Shuler M, Fleming LL. Evaluation of the validity of the AOFAS clinical rating systems by correlation to the SF-36. Foot Ankle Int 24:50–55, 2003. Guyton GP. Theoretical limitations of the AOFAS scoring systems: an analysis using Monte Carlo modeling. Foot Ankle Int 22:779–787, 2001. Baumhauer JF, Nawoczenski DA, DiGiovanni BF, Wilding GE. Reliability and validity of the American Orthopaedic Foot and Ankle Society clinical rating scale: a pilot study for the hallux and lesser toes. Foot Ankle Int 27:1014–1019, 2006. Guillemin F, Bombardier C, Beaton D. Cross-cultural adaptation of health-related quality of life measures: literature review and proposed guidelines. J Clin Epidemiol 46:1417–1432, 1993. Beaton DE, Bombardier C, Guillemin F, Ferraz MB. Guidelines for the process of cross-cultural adaptation of self-report measures. Spine 25:3186–3191, 2000. Deyo RA, Andersson G, Bombardier C, Cherkin DC, Keller RB, Lee CK, Liang MH, Lipscomb B, Shekelle P, Spratt KF. Outcome measures for studying patients with low back pain. Spine 19:2032S–2036S, 1994. Dawson J, Fitzpatrick R, Carr A. Questionnaire on the perceptions of patients about shoulder surgery. J Bone Joint Surg Br 78:593–600, 1996. Fitzpatrick R, Davey C, Buxton MJ, Jones DR. Evaluating patient-based outcome measures for use in clinical trials. Health Technol Assess 2, 1998. i–iv, 1–74. Engelberg R, Martin DP, Agel J, Obremsky W, Coronado G, Swiontkowski MF. Musculoskeletal function assessment instrument: criterion and construct validity. J Orthop Res 14:182–192, 1996.
