Rheumatol Int (2014) 34:171–173 DOI 10.1007/s00296-013-2891-0
LETTER TO THE EDITOR
Searching for optimal rating scales in the Bath Ankylosing Spondylitis Functional Index (BASFI) and Bath Ankylosing Spondylitis Disease Activity Index (BASDAI) Franco Franchignoni · Fausto Salaffi · Alessandro Ciapetti · Andrea Giordano
Received: 10 October 2013 / Accepted: 23 October 2013 / Published online: 9 November 2013 © Springer-Verlag Berlin Heidelberg 2013
Abstract The comparison of the performance of the numerical rating scale (NRS) versus visual analog scale (VAS) in the Bath Ankylosing Spondylitis Functional Index (BASFI) and Bath Ankylosing Spondylitis Disease Activity Index (BASDAI) showed that the 11-point NRS is psychometrically superior to the 10-cm VAS. This finding is in agreement with previous studies and the recommendation by the Assessment of SpondyloArthritis international Society. To illustrate the functioning of the response categories of both BASFI and BASDAI, we analyzed the rating scales (using the Rasch rating scale model) in patients with ankylosing spondylitis. Our results have shown that the 11 categories available in the 0–10 NRS version of both BASFI and BASDAI exceed the number of levels of a construct that participants can discriminate. This indicates the need for improving the metric quality of both rating scales by appropriately reducing the number of categories.
F. Franchignoni Unit of Occupational Rehabilitation and Ergonomics, Salvatore Maugeri Foundation, Clinica del Lavoro e della Riabilitazione, IRCCS, Veruno, Novara, Italy F. Salaffi · A. Ciapetti Department of Rheumatology, Polytechnic University of the Marche, Jesi, Ancona, Italy F. Salaffi (*) Clinica Reumatologica, Ospedale “C. Urbani”, Università Politecnica delle Marche, Via dei Colli 52, 60035 Jesi, Ancona, Italy e-mail: [email protected] A. Giordano Unit of Bioengineering, Salvatore Maugeri Foundation, Clinica del Lavoro e della Riabilitazione, IRCCS, Veruno, Novara, Italy
Keywords Bath Ankylosing Spondylitis Functional Index · Bath Ankylosing Spondylitis Disease Activity Index · Rasch analysis
To the Editor We read with interest the paper by Akad et al. [1] comparing the performance of the numerical rating scale (NRS) versus visual analog scale (VAS) in the Bath Ankylosing Spondylitis Functional Index (BASFI) and Bath Ankylosing Spondylitis Disease Activity Index (BASDAI). We agree with their conclusion that the 11-point NRS is psychometrically superior to the 10-cm VAS. Their findings are in agreement with previous studies [2] and with the recommendation by the Assessment of SpondyloArthritis international Society (ASAS) [3]. Along the same lines, we recently reported some practical and metric limitations linked to use of VAS [4], namely that it shows a linearity and accuracy which are illusory. However, also the 11-point NRS is not an ideal response scale, as Streiner and Norman [5] wrote the following: “It is reasonable to presume that the upper practical limit of useful levels on a scale can be set at seven… the ‘one in a hundred’ precision of the VAS is illusory,” and “when a large number of individual items are designed to be summed to create a scale score, it is likely that reducing the number of levels to five or three will not result in significant loss of information.” Indeed, the 7-option rule of thumb is true for healthy adults, but item-level analyses—such as those present in Rasch analysis—have shown that some people with special needs are unable to appreciably discern between more than five categories as indicating different levels of a variable [6, 7]. In order to ascertain whether a rating scale is being used in the intended manner, Rasch analysis applies the
13
172
Rheumatol Int (2014) 34:171–173
Fig. 1 Category probability curves: (a) curves of the original 11 categories (0–10) of BASFI; and (b) of the 5 revised categories obtained by combining the original categories as follows: 0–1 = 0; 2–3 = 1; 4–6 = 2; 7–8 = 3; 9–10 = 4. The y axis represents the probability (0–1) of responding to one of the rating categories, and the x axis the different performance values (patient ability minus item difficulty) in
logits. Graphs (c) and (d) show the same process as in (a) and (b) concerning BASDAI. The ideal plot should look like an ordered even succession of hills—as in (b) and (d)—with an ‘emerging’ crest where each category is modal over a certain range. Conversely, in (a) and (c), the probability of using some categories is never higher than that of using an adjacent category
procedure of ‘rating scale diagnostics’ that evaluates the performance of the response categories according to a set of criteria (adequate number of responses per category, even use of the categories, monotonic increase in the difficulty of each category, etc.). To illustrate the functioning of the response categories of both BASFI and BASDAI, we analyzed the rating scales (using the Rasch rating scale model) in 126 consecutive patients with ankylosing spondylitis (91 % males; mean age 48 years, SD 11). In Fig. 1, category probability plots (a) and (c) show the probability of responding to each of the rating categories in relation to patients’ actual performance values. The ‘0’ curve (‘easy’ in BASFI, ‘no problem’ in BASDAI) declines as the subject’s disability increases; the crossing point (where options 0 and 1 are equally probable) is the first ‘threshold.’ The same applies to the other curves. Both rating scales display disordered thresholds: this can occur when there are too many response options. We then collapsed the categories in an attempt to maximize statistical performance and clinical meaningfulness. Graphs (b) and (d) show the result: now, the probability of selecting each of the five revised rating categories is a clear function of the level of disability shown by the x axis. This procedure improved the measurement quality of the scale, minimizing
irrelevant construct variance without decreasing the scale’s reliability. Similar problems with rating scale functioning (indicating the presence of excessive category levels) have been reported: (1) by two previous studies analyzing BASFI with Rasch methods: the former using the 0–100 VAS scale [8], the latter adopting the 0-10 NRS [9]; (2) when the 0–10 NRS is used in the assessment of other rheumatic diseases, such as fibromyalgia [10]. Overall, these observations suggest that the 11 categories available in the 0–10 NRS version of both BASFI and BASDAI exceed the number of levels of a construct that participants can discriminate. This indicates the need for improving the metric quality of both rating scales by appropriately reducing the number of categories.
13
Conflict of interest The authors declare that they have no conflict of interest.
References 1. Akad K, Solmaz D, Sari I, Onen F, Akkoc N, Akar S (2013) Performance of response scales of activity and functional measures of ankylosing spondylitis: numerical rating scale versus visual analog scale. Rheumatol Int 33:2617–2623
Rheumatol Int (2014) 34:171–173 2. Van Tubergen A, Debats I, Ryser L, Londono J, Burgos-Vargas R, Cardiel MH et al (2002) Use of a numerical rating scale as an answer modality in ankylosing spondylitis-specific questionnaires. Arthritis Rheum 47:242–248 3. Sieper J, Rudwaleit M, Baraliakos X, Brandt J, Braun J, BurgosVargas R et al (2009) The Assessment of SpondyloArthritis international Society (ASAS) handbook: a guide to assess spondyloarthritis. Ann Rheum Dis 68(Suppl 2):ii1–ii44 4. Franchignoni F, Salaffi F, Tesio L (2012) How should we use the visual analogue scale (VAS) in rehabilitation outcomes? I: how much of what? The seductive VAS numbers are not true measures. J Rehabil Med 44:798–799 5. Streiner DL, Norman GR (2008) Health measurement scales: a practical guide to their development and use, 4th edn. Oxford University Press, Oxford, pp 41–55 6. Wolfe EW, Smith EV Jr (2007) Instrument development tools and activities for measure validation using Rasch models: part I. J Appl Meas 8:97–123
173 7. Khadka J, Gothwal VK, McAlinden C, Lamoureux EL, Pesudovs K (2012) The importance of rating scales in measuring patientreported outcomes. Health Qual Life Outcomes 10:80 8. Eyres S, Tennant A, Kay L, Waxman R, Helliwell PS (2002) Measuring disability in ankylosing spondylitis: comparison of Bath Ankylosing Spondylitis Functional Index with revised Leeds Disability Questionnaire. J Rheumatol 29:979–986 9. Leung YY, Tam LS, Kun EW, Ho KW, Li EK (2008) Comparison of 4 functional indexes in psoriatic arthritis with axial or peripheral disease subgroups using Rasch analyses. J Rheumatol 35:1613–1621 10. Salaffi F, Franchignoni F, Giordano A, Ciapetti A, Sarzi Puttini P, Ottonello M (2013) Psychometric characteristics of the Italian version of the revised Fibromyalgia Impact Questionnaire using classical test theory and Rasch analysis. Clin Exp Rheumatol Jun 26 (Epub ahead of print)
13
LETTER TO THE EDITOR
Searching for optimal rating scales in the Bath Ankylosing Spondylitis Functional Index (BASFI) and Bath Ankylosing Spondylitis Disease Activity Index (BASDAI) Franco Franchignoni · Fausto Salaffi · Alessandro Ciapetti · Andrea Giordano
Received: 10 October 2013 / Accepted: 23 October 2013 / Published online: 9 November 2013 © Springer-Verlag Berlin Heidelberg 2013
Abstract The comparison of the performance of the numerical rating scale (NRS) versus visual analog scale (VAS) in the Bath Ankylosing Spondylitis Functional Index (BASFI) and Bath Ankylosing Spondylitis Disease Activity Index (BASDAI) showed that the 11-point NRS is psychometrically superior to the 10-cm VAS. This finding is in agreement with previous studies and the recommendation by the Assessment of SpondyloArthritis international Society. To illustrate the functioning of the response categories of both BASFI and BASDAI, we analyzed the rating scales (using the Rasch rating scale model) in patients with ankylosing spondylitis. Our results have shown that the 11 categories available in the 0–10 NRS version of both BASFI and BASDAI exceed the number of levels of a construct that participants can discriminate. This indicates the need for improving the metric quality of both rating scales by appropriately reducing the number of categories.
F. Franchignoni Unit of Occupational Rehabilitation and Ergonomics, Salvatore Maugeri Foundation, Clinica del Lavoro e della Riabilitazione, IRCCS, Veruno, Novara, Italy F. Salaffi · A. Ciapetti Department of Rheumatology, Polytechnic University of the Marche, Jesi, Ancona, Italy F. Salaffi (*) Clinica Reumatologica, Ospedale “C. Urbani”, Università Politecnica delle Marche, Via dei Colli 52, 60035 Jesi, Ancona, Italy e-mail: [email protected] A. Giordano Unit of Bioengineering, Salvatore Maugeri Foundation, Clinica del Lavoro e della Riabilitazione, IRCCS, Veruno, Novara, Italy
Keywords Bath Ankylosing Spondylitis Functional Index · Bath Ankylosing Spondylitis Disease Activity Index · Rasch analysis
To the Editor We read with interest the paper by Akad et al. [1] comparing the performance of the numerical rating scale (NRS) versus visual analog scale (VAS) in the Bath Ankylosing Spondylitis Functional Index (BASFI) and Bath Ankylosing Spondylitis Disease Activity Index (BASDAI). We agree with their conclusion that the 11-point NRS is psychometrically superior to the 10-cm VAS. Their findings are in agreement with previous studies [2] and with the recommendation by the Assessment of SpondyloArthritis international Society (ASAS) [3]. Along the same lines, we recently reported some practical and metric limitations linked to use of VAS [4], namely that it shows a linearity and accuracy which are illusory. However, also the 11-point NRS is not an ideal response scale, as Streiner and Norman [5] wrote the following: “It is reasonable to presume that the upper practical limit of useful levels on a scale can be set at seven… the ‘one in a hundred’ precision of the VAS is illusory,” and “when a large number of individual items are designed to be summed to create a scale score, it is likely that reducing the number of levels to five or three will not result in significant loss of information.” Indeed, the 7-option rule of thumb is true for healthy adults, but item-level analyses—such as those present in Rasch analysis—have shown that some people with special needs are unable to appreciably discern between more than five categories as indicating different levels of a variable [6, 7]. In order to ascertain whether a rating scale is being used in the intended manner, Rasch analysis applies the
13
172
Rheumatol Int (2014) 34:171–173
Fig. 1 Category probability curves: (a) curves of the original 11 categories (0–10) of BASFI; and (b) of the 5 revised categories obtained by combining the original categories as follows: 0–1 = 0; 2–3 = 1; 4–6 = 2; 7–8 = 3; 9–10 = 4. The y axis represents the probability (0–1) of responding to one of the rating categories, and the x axis the different performance values (patient ability minus item difficulty) in
logits. Graphs (c) and (d) show the same process as in (a) and (b) concerning BASDAI. The ideal plot should look like an ordered even succession of hills—as in (b) and (d)—with an ‘emerging’ crest where each category is modal over a certain range. Conversely, in (a) and (c), the probability of using some categories is never higher than that of using an adjacent category
procedure of ‘rating scale diagnostics’ that evaluates the performance of the response categories according to a set of criteria (adequate number of responses per category, even use of the categories, monotonic increase in the difficulty of each category, etc.). To illustrate the functioning of the response categories of both BASFI and BASDAI, we analyzed the rating scales (using the Rasch rating scale model) in 126 consecutive patients with ankylosing spondylitis (91 % males; mean age 48 years, SD 11). In Fig. 1, category probability plots (a) and (c) show the probability of responding to each of the rating categories in relation to patients’ actual performance values. The ‘0’ curve (‘easy’ in BASFI, ‘no problem’ in BASDAI) declines as the subject’s disability increases; the crossing point (where options 0 and 1 are equally probable) is the first ‘threshold.’ The same applies to the other curves. Both rating scales display disordered thresholds: this can occur when there are too many response options. We then collapsed the categories in an attempt to maximize statistical performance and clinical meaningfulness. Graphs (b) and (d) show the result: now, the probability of selecting each of the five revised rating categories is a clear function of the level of disability shown by the x axis. This procedure improved the measurement quality of the scale, minimizing
irrelevant construct variance without decreasing the scale’s reliability. Similar problems with rating scale functioning (indicating the presence of excessive category levels) have been reported: (1) by two previous studies analyzing BASFI with Rasch methods: the former using the 0–100 VAS scale [8], the latter adopting the 0-10 NRS [9]; (2) when the 0–10 NRS is used in the assessment of other rheumatic diseases, such as fibromyalgia [10]. Overall, these observations suggest that the 11 categories available in the 0–10 NRS version of both BASFI and BASDAI exceed the number of levels of a construct that participants can discriminate. This indicates the need for improving the metric quality of both rating scales by appropriately reducing the number of categories.
13
Conflict of interest The authors declare that they have no conflict of interest.
References 1. Akad K, Solmaz D, Sari I, Onen F, Akkoc N, Akar S (2013) Performance of response scales of activity and functional measures of ankylosing spondylitis: numerical rating scale versus visual analog scale. Rheumatol Int 33:2617–2623
Rheumatol Int (2014) 34:171–173 2. Van Tubergen A, Debats I, Ryser L, Londono J, Burgos-Vargas R, Cardiel MH et al (2002) Use of a numerical rating scale as an answer modality in ankylosing spondylitis-specific questionnaires. Arthritis Rheum 47:242–248 3. Sieper J, Rudwaleit M, Baraliakos X, Brandt J, Braun J, BurgosVargas R et al (2009) The Assessment of SpondyloArthritis international Society (ASAS) handbook: a guide to assess spondyloarthritis. Ann Rheum Dis 68(Suppl 2):ii1–ii44 4. Franchignoni F, Salaffi F, Tesio L (2012) How should we use the visual analogue scale (VAS) in rehabilitation outcomes? I: how much of what? The seductive VAS numbers are not true measures. J Rehabil Med 44:798–799 5. Streiner DL, Norman GR (2008) Health measurement scales: a practical guide to their development and use, 4th edn. Oxford University Press, Oxford, pp 41–55 6. Wolfe EW, Smith EV Jr (2007) Instrument development tools and activities for measure validation using Rasch models: part I. J Appl Meas 8:97–123
173 7. Khadka J, Gothwal VK, McAlinden C, Lamoureux EL, Pesudovs K (2012) The importance of rating scales in measuring patientreported outcomes. Health Qual Life Outcomes 10:80 8. Eyres S, Tennant A, Kay L, Waxman R, Helliwell PS (2002) Measuring disability in ankylosing spondylitis: comparison of Bath Ankylosing Spondylitis Functional Index with revised Leeds Disability Questionnaire. J Rheumatol 29:979–986 9. Leung YY, Tam LS, Kun EW, Ho KW, Li EK (2008) Comparison of 4 functional indexes in psoriatic arthritis with axial or peripheral disease subgroups using Rasch analyses. J Rheumatol 35:1613–1621 10. Salaffi F, Franchignoni F, Giordano A, Ciapetti A, Sarzi Puttini P, Ottonello M (2013) Psychometric characteristics of the Italian version of the revised Fibromyalgia Impact Questionnaire using classical test theory and Rasch analysis. Clin Exp Rheumatol Jun 26 (Epub ahead of print)
13
