EDITORIAL For reprint orders, please contact: [email protected]
Clinically meaningful changes in pain ratings: why we need special cut points in children and adolescents
“...optimal cut points overestimate the diagnostic utility...”
Gerrit Hirschfeld* Recurrent pain in children and adolescents is a pervasive healthcare issue that can only be addressed when pain and changes therein can be reliably measured. Of special importance is the assessment of clinically meaningful changes in pain since these changes provide feedback for clinicians about the trajectory of an individual patient, and guide researchers trials into the effectiveness of novel interventions. It seems self evident that the measures we use to collect pain ratings have to be validated in the sample in which they are used. Accordingly, there is a vast literature validating various pain scales (faces-pain scale, numerical rating scale [NRS], visual analog scale [VAS]), in children and adolescents [1]. Overall, these studies agree that collecting pain ratings is feasible in children 5 years and older from the age of 5 years onwards [2]. However, most of these validation studies focus on reliability (e.g., test-retest reliability), validity (e.g., correlations to parental ratings [3]), and responsitivity (e.g., increase after painful procedures, and decrease during recovery [4]). While these are important aspects of validation, they do not offer any standards by which one might interpret changes in
individuals as clinically meaningful. In other words, knowing that larger changes in parental pain ratings are associated with larger changes in children’s ratings does not help to decide whether or not the treatment is successful or needs to be intensified. Cut points (e.g., decision thresholds) are the most widely used method to aid clinical decisions and classify study participant as responders or nonresponders. Before turning to the question why we need special cut points in children and adolescents, I will first summarize why we need cut points to determine clinically meaningful change and why we need special cut points for different populations. Why we need cut points to determine clinically meaningful change For practitioners the availability of rating scales that yield continuous test-results (e.g., 0–10 NRS, 0–100 VAS) from which differences can be calculated, immediately raises the question of how to turn such continuous results into categorical decisions (start, stop, continue or intensify treatment). This problem pertains both to raw differences (e.g., a change of 2 points) and percentage differences (e.g.,
“...cut points for changes in pain intensity are extremely useful in clinical practice and a central aspect of trials into the effectiveness of interventions.”
*German Paediatric Pain Centre, Children’s & Adolescents’ Hospital, Datteln, Witten/Herdecke University, Faculty of Health, School of Medicine, Germany; Tel.: +49 236 397 5183; Fax: +49 236 397 5181; [email protected]
10.2217/PMT.14.2 © 2014 Future Medicine Ltd
Pain Manage. (2014) 4(2), 81–83
part of
ISSN 1758-1869
81
Editorial Hirschfeld
“Taking into account the chance-variability, the question of why we need special cut points in children and adolescents largely becomes an empirical issue.”
50% reduction) as well as changes in acute and recurrent pain. Even though practitioners should take into account various pieces of information in order to develop treatment decisions [2], changes in pain intensity are obviously among the most relevant pieces of information. It is intuitively plausible that improvements in perceived pain intensity need to be larger than a specific threshold to be of practical relevance to patients. Among researchers the use of cut points for clinically meaningful change is much more debated. On the one hand dichotomizing end points in clinical trials (clinically improved vs not) severely reduces statistical power [5]. Since all recommended pain scales yield continuous pain intensity scores, it seems counter-productive to dichotomize them for the analysis. On the other hand, dichotomizing end points allows classifying individual patients as responders or nonresponders, which is important to calculate the number needed to treat and other metrics that are employed in meta-analysis. Furthermore, if for a group the pain-intensity decreased on average by 2 points on the NRS, this may result from different scenarios. In the first scenario all patients uniformly show a decrease of 2 points. In the second scenario half of the patients reduce their pain by 4 points while the other half does not change at all. While the first scenario highlights the need to improve the intervention, the second scenario highlights the need to identifying nonresponders before the intervention. Even though these two analysis strategies can complement each other [6], the international guidelines for good clinical practice that govern the design of clinical trials in general and pain-specific guidelines, such as IMMPACT [7] and PedIMMPACT [8] require the use of dichotomous end points and by extension cut points. Why we need special cut points for different populations Given the practical relevance and scientific importance of cut points there is a rich tradition to determine ‘optimal’ cut points for clinically meaningful changes in pain intensity in different populations [7]. Most of the studies in pain research focus on defining minimally clinically significant differences (MCSD) – that is, the smallest change that is considered relevant to an individual patient. Following Farrar and colleagues [9] several studies have used a receiveroperating characteristic-based method to identify
82
Pain Manage. (2014) 4(2)
MCSDs. This uses patients’ global impressions of change (“Overall my pain has improved”) as anchors against which alternative cut points are tested. For each of the possible cut points the sensitivity and specificity with regard to the global impression of change is calculated. The optimal cut point is defined as the cut point that yields the best balance between sensitivity and specificity. There are several pitfalls establishing MCSD using such an optimal cut point method. First, optimal cut points overestimate the diagnostic utility [10] if the same sample is both used to define the optimal cut point and assess its diagnostic utility so that optimal cut points need to be crossvalidated [11]. Second, optimal cut points are inherently variable, for example, chance fluctuations within a sample may result in different optimal cut points. If this variability is not accounted for, it may tempt researchers to interpret differences as reflecting meaningful differences between the groups. This has resulted in a series of high-profile studies claiming that specific cut points for mild, moderate, and severe pain are necessary in different etiologic groups [12]. On the upside it has recently been shown that estimating this variability is useful in identifying cut points that are feasible in different groups [13]. Why we need special cut points in children & adolescents Taking into account the chance-variability, the question of why we need special cut points in children and adolescents largely becomes an empirical issue. Namely, do the estimated optimal cut points in children and adolescents overlap with optimal cut points estimated in adults? So far only a handful of studies have tried to develop meaningful cut points for minimally clinically significant differences [14–18]. Overall, these suggest using smaller decreases as clinically meaningful changes in childrens’ pain ratings than studies in adults. For example Voepel-Lewis and colleagues [18] found that a change of one point on the NRS can be considered MCSD for postoperative pain. So far only one study also took into account the variability of such findings [16]. This study found that even though the cut points for minimally clinically significant change were variable they did not overlap with adolescent cut points, while different etiologies result in overlapping optimal cut points [17]. Specifically this study showed that a raw change of 1 point or a percentage-change of 12.5% on the 0–11 NRS should be considered minimally clinically
future science group
Changes in pain ratings: why we need special cut points in children & adolescents significant. Furthermore, this study showed that the cut points developed for adults perform very poorly when applied to pediatric samples [17]. While much future research is needed to extend these findings to other pain-scales and pain etiologies, the existing data clearly supports the need for cut points specifically determined in children and adolescents. To conclude, cut points for changes in pain intensity are extremely useful in clinical practice and a central aspect of trials into the effectiveness of interventions. The question of whether or not specific cut points are needed for children and adolescents is an empirical issue that needs to take into account the chance variability of their estimation methods. With regard to changes in the NRS following intensive inpatient therapies References 1
Stinson JN, Kavanagh T, Yamada J, Gill N, Stevens B. Systematic review of the psychometric properties, interpretability and feasibility of self-report pain intensity measures for use in clinical trials in children and adolescents. Pain 125(1–2), 143–157 (2006).
2
von Baeyer CL. Children’s self-reports of pain intensity: Scale selection, limitations and interpretation. Pain Res. Manag. 11(3), 157–162 (2006).
3
Hicks CL, von Baeyer CL, Spafford PA, van Korlaar I, Goodenough B. The Faces Pain Scale – revised: toward a common metric in pediatric pain measurement. Pain 93(2), 173–183 (2001).
4
Perrott DA, Goodenough B, Champion GD. Children’s ratings of the intensity and unpleasantness of post-operative pain using facial expression scales. Eur. J. Pain 8(2), 119–127 (2004).
5
Altman DG, Royston P. The cost of dichotomising continuous variables. BMJ 332(7549), 1080 (2006).
6
Hirschfeld G, Hechler T, Dobe M et al. Maintaining lasting improvements: one-year follow-up of children with severe chronic pain undergoing multimodal inpatient
future science group
in children the data strongly support the notion of specialized cut points for children and adolescents. More research into other pain scales and in different therapeutic settings (inpatient vs outpatient) is warranted to corroborate and extend these findings. Financial & competing interests disclosure The author has no relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript. This includes employment, consultancies, honoraria, stock ownership or options, expert testimony, grants or patents received or pending, or royalties. No writing assistance was utilized in the production of this manuscript. treatment. J. Pediatr. Psychol. 38(2), 224–236 (2013).
7
8
9
Editorial
comparing groups. Pain 154(1), 154–159 (2013).
Dworkin RH, Turk DC, Wyrwich KW et al. Interpreting the clinical importance of treatment outcomes in chronic pain clinical trials: IMMPACT recommendations. J. Pain 9(2), 105–121 (2008).
13 Hirschfeld G, Zernikow B. Cutpoints for
McGrath PJ, Walco GA, Turk DC et al. Core outcome domains and measures for pediatric acute and chronic/recurrent pain clinical trials: PedIMMPACT recommendations. J. Pain 9(9), 771–783 (2008).
14 Powell CV, Kelly AM, Williams A.
Farrar JT, Young Jr JP, LaMoreaux L, Werth JL, Poole RM. Clinical importance of changes in chronic pain intensity measured on an 11-point numerical pain rating scale. Pain 94(2), 149–158 (2001).
10 Ewald B. Post hoc choice of cut points
introduced bias to diagnostic research. J. Clin. Epidemiol. 59(8), 798–801 (2006). 11 Hirschfeld G, do Brasil P. A simulation study
into the performance of ’optimal’ diagnostic thresholds in the population – ‘large’ effect sizes are not enough. J. Clin. Epidemiol. (2014) (In press). 12 Hirschfeld G, Zernikow B. Variability of
‘optimal’ cut points for mild, moderate, and severe pain: neglected problems when
mild, moderate, and severe pain on the VAS for children and adolescents – what can be learned from 10 million ANOVAs? Pain 154(12), 2626–2632 (2014). Determining the minimum clinically significant difference in visual analog pain score for children. Ann. Emerg. Med. 37(1), 28–31 (2001). 15 Bulloch B, Tenenbein M. Assessment of
clinically significant changes in acute pain in children. Acad. Emerg. Med. 9(3), 199–202 (2002). 16 McConahay T, Bryson M, Bulloch B.
Clinically significant changes in acute pain in a pediatric ED using the Color Analog Scale. Am. J. Emerg. Med. 25(7), 739–742 (2007). 17 Hirschfeld G, Wager J, Schmidt P, Zernikow
B. Minimally clinically significant differences for adolescents with chronic pain-variability of ROC-based cutpoints. J. Pain 15(1). 32–39 (2014). 18 Voepel-Lewis T, Burke CN, Jeffreys N,
Malviya S, Tait AR. Do 0–10 numeric rating scores translate into clinically meaningful pain measures for children? Anesth. Analg. 112(2), 415–421 (2011).
www.futuremedicine.com
83
Clinically meaningful changes in pain ratings: why we need special cut points in children and adolescents
“...optimal cut points overestimate the diagnostic utility...”
Gerrit Hirschfeld* Recurrent pain in children and adolescents is a pervasive healthcare issue that can only be addressed when pain and changes therein can be reliably measured. Of special importance is the assessment of clinically meaningful changes in pain since these changes provide feedback for clinicians about the trajectory of an individual patient, and guide researchers trials into the effectiveness of novel interventions. It seems self evident that the measures we use to collect pain ratings have to be validated in the sample in which they are used. Accordingly, there is a vast literature validating various pain scales (faces-pain scale, numerical rating scale [NRS], visual analog scale [VAS]), in children and adolescents [1]. Overall, these studies agree that collecting pain ratings is feasible in children 5 years and older from the age of 5 years onwards [2]. However, most of these validation studies focus on reliability (e.g., test-retest reliability), validity (e.g., correlations to parental ratings [3]), and responsitivity (e.g., increase after painful procedures, and decrease during recovery [4]). While these are important aspects of validation, they do not offer any standards by which one might interpret changes in
individuals as clinically meaningful. In other words, knowing that larger changes in parental pain ratings are associated with larger changes in children’s ratings does not help to decide whether or not the treatment is successful or needs to be intensified. Cut points (e.g., decision thresholds) are the most widely used method to aid clinical decisions and classify study participant as responders or nonresponders. Before turning to the question why we need special cut points in children and adolescents, I will first summarize why we need cut points to determine clinically meaningful change and why we need special cut points for different populations. Why we need cut points to determine clinically meaningful change For practitioners the availability of rating scales that yield continuous test-results (e.g., 0–10 NRS, 0–100 VAS) from which differences can be calculated, immediately raises the question of how to turn such continuous results into categorical decisions (start, stop, continue or intensify treatment). This problem pertains both to raw differences (e.g., a change of 2 points) and percentage differences (e.g.,
“...cut points for changes in pain intensity are extremely useful in clinical practice and a central aspect of trials into the effectiveness of interventions.”
*German Paediatric Pain Centre, Children’s & Adolescents’ Hospital, Datteln, Witten/Herdecke University, Faculty of Health, School of Medicine, Germany; Tel.: +49 236 397 5183; Fax: +49 236 397 5181; [email protected]
10.2217/PMT.14.2 © 2014 Future Medicine Ltd
Pain Manage. (2014) 4(2), 81–83
part of
ISSN 1758-1869
81
Editorial Hirschfeld
“Taking into account the chance-variability, the question of why we need special cut points in children and adolescents largely becomes an empirical issue.”
50% reduction) as well as changes in acute and recurrent pain. Even though practitioners should take into account various pieces of information in order to develop treatment decisions [2], changes in pain intensity are obviously among the most relevant pieces of information. It is intuitively plausible that improvements in perceived pain intensity need to be larger than a specific threshold to be of practical relevance to patients. Among researchers the use of cut points for clinically meaningful change is much more debated. On the one hand dichotomizing end points in clinical trials (clinically improved vs not) severely reduces statistical power [5]. Since all recommended pain scales yield continuous pain intensity scores, it seems counter-productive to dichotomize them for the analysis. On the other hand, dichotomizing end points allows classifying individual patients as responders or nonresponders, which is important to calculate the number needed to treat and other metrics that are employed in meta-analysis. Furthermore, if for a group the pain-intensity decreased on average by 2 points on the NRS, this may result from different scenarios. In the first scenario all patients uniformly show a decrease of 2 points. In the second scenario half of the patients reduce their pain by 4 points while the other half does not change at all. While the first scenario highlights the need to improve the intervention, the second scenario highlights the need to identifying nonresponders before the intervention. Even though these two analysis strategies can complement each other [6], the international guidelines for good clinical practice that govern the design of clinical trials in general and pain-specific guidelines, such as IMMPACT [7] and PedIMMPACT [8] require the use of dichotomous end points and by extension cut points. Why we need special cut points for different populations Given the practical relevance and scientific importance of cut points there is a rich tradition to determine ‘optimal’ cut points for clinically meaningful changes in pain intensity in different populations [7]. Most of the studies in pain research focus on defining minimally clinically significant differences (MCSD) – that is, the smallest change that is considered relevant to an individual patient. Following Farrar and colleagues [9] several studies have used a receiveroperating characteristic-based method to identify
82
Pain Manage. (2014) 4(2)
MCSDs. This uses patients’ global impressions of change (“Overall my pain has improved”) as anchors against which alternative cut points are tested. For each of the possible cut points the sensitivity and specificity with regard to the global impression of change is calculated. The optimal cut point is defined as the cut point that yields the best balance between sensitivity and specificity. There are several pitfalls establishing MCSD using such an optimal cut point method. First, optimal cut points overestimate the diagnostic utility [10] if the same sample is both used to define the optimal cut point and assess its diagnostic utility so that optimal cut points need to be crossvalidated [11]. Second, optimal cut points are inherently variable, for example, chance fluctuations within a sample may result in different optimal cut points. If this variability is not accounted for, it may tempt researchers to interpret differences as reflecting meaningful differences between the groups. This has resulted in a series of high-profile studies claiming that specific cut points for mild, moderate, and severe pain are necessary in different etiologic groups [12]. On the upside it has recently been shown that estimating this variability is useful in identifying cut points that are feasible in different groups [13]. Why we need special cut points in children & adolescents Taking into account the chance-variability, the question of why we need special cut points in children and adolescents largely becomes an empirical issue. Namely, do the estimated optimal cut points in children and adolescents overlap with optimal cut points estimated in adults? So far only a handful of studies have tried to develop meaningful cut points for minimally clinically significant differences [14–18]. Overall, these suggest using smaller decreases as clinically meaningful changes in childrens’ pain ratings than studies in adults. For example Voepel-Lewis and colleagues [18] found that a change of one point on the NRS can be considered MCSD for postoperative pain. So far only one study also took into account the variability of such findings [16]. This study found that even though the cut points for minimally clinically significant change were variable they did not overlap with adolescent cut points, while different etiologies result in overlapping optimal cut points [17]. Specifically this study showed that a raw change of 1 point or a percentage-change of 12.5% on the 0–11 NRS should be considered minimally clinically
future science group
Changes in pain ratings: why we need special cut points in children & adolescents significant. Furthermore, this study showed that the cut points developed for adults perform very poorly when applied to pediatric samples [17]. While much future research is needed to extend these findings to other pain-scales and pain etiologies, the existing data clearly supports the need for cut points specifically determined in children and adolescents. To conclude, cut points for changes in pain intensity are extremely useful in clinical practice and a central aspect of trials into the effectiveness of interventions. The question of whether or not specific cut points are needed for children and adolescents is an empirical issue that needs to take into account the chance variability of their estimation methods. With regard to changes in the NRS following intensive inpatient therapies References 1
Stinson JN, Kavanagh T, Yamada J, Gill N, Stevens B. Systematic review of the psychometric properties, interpretability and feasibility of self-report pain intensity measures for use in clinical trials in children and adolescents. Pain 125(1–2), 143–157 (2006).
2
von Baeyer CL. Children’s self-reports of pain intensity: Scale selection, limitations and interpretation. Pain Res. Manag. 11(3), 157–162 (2006).
3
Hicks CL, von Baeyer CL, Spafford PA, van Korlaar I, Goodenough B. The Faces Pain Scale – revised: toward a common metric in pediatric pain measurement. Pain 93(2), 173–183 (2001).
4
Perrott DA, Goodenough B, Champion GD. Children’s ratings of the intensity and unpleasantness of post-operative pain using facial expression scales. Eur. J. Pain 8(2), 119–127 (2004).
5
Altman DG, Royston P. The cost of dichotomising continuous variables. BMJ 332(7549), 1080 (2006).
6
Hirschfeld G, Hechler T, Dobe M et al. Maintaining lasting improvements: one-year follow-up of children with severe chronic pain undergoing multimodal inpatient
future science group
in children the data strongly support the notion of specialized cut points for children and adolescents. More research into other pain scales and in different therapeutic settings (inpatient vs outpatient) is warranted to corroborate and extend these findings. Financial & competing interests disclosure The author has no relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript. This includes employment, consultancies, honoraria, stock ownership or options, expert testimony, grants or patents received or pending, or royalties. No writing assistance was utilized in the production of this manuscript. treatment. J. Pediatr. Psychol. 38(2), 224–236 (2013).
7
8
9
Editorial
comparing groups. Pain 154(1), 154–159 (2013).
Dworkin RH, Turk DC, Wyrwich KW et al. Interpreting the clinical importance of treatment outcomes in chronic pain clinical trials: IMMPACT recommendations. J. Pain 9(2), 105–121 (2008).
13 Hirschfeld G, Zernikow B. Cutpoints for
McGrath PJ, Walco GA, Turk DC et al. Core outcome domains and measures for pediatric acute and chronic/recurrent pain clinical trials: PedIMMPACT recommendations. J. Pain 9(9), 771–783 (2008).
14 Powell CV, Kelly AM, Williams A.
Farrar JT, Young Jr JP, LaMoreaux L, Werth JL, Poole RM. Clinical importance of changes in chronic pain intensity measured on an 11-point numerical pain rating scale. Pain 94(2), 149–158 (2001).
10 Ewald B. Post hoc choice of cut points
introduced bias to diagnostic research. J. Clin. Epidemiol. 59(8), 798–801 (2006). 11 Hirschfeld G, do Brasil P. A simulation study
into the performance of ’optimal’ diagnostic thresholds in the population – ‘large’ effect sizes are not enough. J. Clin. Epidemiol. (2014) (In press). 12 Hirschfeld G, Zernikow B. Variability of
‘optimal’ cut points for mild, moderate, and severe pain: neglected problems when
mild, moderate, and severe pain on the VAS for children and adolescents – what can be learned from 10 million ANOVAs? Pain 154(12), 2626–2632 (2014). Determining the minimum clinically significant difference in visual analog pain score for children. Ann. Emerg. Med. 37(1), 28–31 (2001). 15 Bulloch B, Tenenbein M. Assessment of
clinically significant changes in acute pain in children. Acad. Emerg. Med. 9(3), 199–202 (2002). 16 McConahay T, Bryson M, Bulloch B.
Clinically significant changes in acute pain in a pediatric ED using the Color Analog Scale. Am. J. Emerg. Med. 25(7), 739–742 (2007). 17 Hirschfeld G, Wager J, Schmidt P, Zernikow
B. Minimally clinically significant differences for adolescents with chronic pain-variability of ROC-based cutpoints. J. Pain 15(1). 32–39 (2014). 18 Voepel-Lewis T, Burke CN, Jeffreys N,
Malviya S, Tait AR. Do 0–10 numeric rating scores translate into clinically meaningful pain measures for children? Anesth. Analg. 112(2), 415–421 (2011).
www.futuremedicine.com
83
