Acta Pædiatrica ISSN 0803-5253
SHORT COMMUNICATION
Validity and reliability of a new short verbal rating scale for stress for use in clinical practice G€osta Alfven ([email protected])1, Stefan Nilsson2,3 1.CLINTEC Departement, Karolinska Institute, Stockholm, Sweden 2.School of Health Sciences, University of Bor as, Bor as, Sweden 3.Institute of Health and Care Sciences, University of Gothenburg, Gothenburg, Sweden
Correspondence €sta Alfve n, CLINTEC Departement, Karolinska Go Institute, Stockholm, Sweden. Tel: +46707660835 | Email: [email protected] Received 13 May 2013; revised 18 November 2013; accepted 13 January 2014. DOI:10.1111/apa.12558
Stress is a common experience that affects the body and the brain in many ways. It can sometimes be of interest in research and clinical practice to measure the intensity of single stress reactions. For example, data on intensity can be important for understanding factors such as stress-related behaviour and pain and for a better understanding of physiological stress reactions in hormones, muscles and the circulatory system. Stress can be measured in a number of ways, including shifts in the production of cortisol and adrenaline and in skin conductance. But performing such tests is not without its difficulties, and the results can be hard to evaluate. An alternative to physiological measurements is verbal instruments. According to a search at PubMed in November 2013 and discussions with colleagues, those that have been developed so far involve many issues and, as a result, take time to answer. Two examples are the State-Trait Anxiety Inventory for Children (STAIC) with 20 items (1) and the Multidimensional Anxiety Scale for Children (MASC) with 37 items (2). However, there is a need for an instrument that assesses the intensity of single stress attacks in a simple, valid and reliable way, and we have not been able to find one. A short verbal rating scale for stress (VRSS), reporting recent or present experience of stress and easy to use in clinical routine settings and research, would meet this demand. An important issue in the development of such an instrument is how it operates, including the theoretical framework, the context of the application and the selection of optimal responses. These are also the key factors that define how we measure the quality of an instrument (3). Instruments capturing stress should be valid and reliable, easy to understand and use and have good compliance. The aim of this study was therefore to test a new VRSS developed by one of the authors.
The material consisted of two samples. The first included 28 children, seven boys and 21 girls, with a mean age of 12.6 years (range: seven to 16) visiting a clinic for recurrent pain. The second sample consisted of 34 children, 15 boys and 19 girls, aged 11.3 years (range: eight to 16). Fifteen of them were recruited from the same clinic as the first sample, and 19 were recruited from a clinic for children with needle phobias. All the children indicated that they knew what stress was and all of them were able to describe their own feeling of stress. After a brief talk about the child’s stress reaction, the child was asked to describe their latest stressful experience in a few words. For the children attending the pain clinic, this incident had happened during the last few days and not longer than a week ago. The children with needle phobias described the stress they felt when they came in for a practice injection. The researchers used a new verbal rating scale for stress (VRSS), based on a scale of zero to five, that had been developed, tested and retested in school age children, in collaboration with two psychologists. The VRSS provides an easy to understand scale that offers the children alternative responses when it comes to describing an increase in their stress levels (see Appendix). Validity of a scale is best evaluated by comparing it with a validated scale measuring the same parameter. But such a scale could not be found, so we measured the agreement between the VRSS and a visual analogue scale (VAS) measuring the same phenomenon. This was done in both groups. The child was asked to choose one of six alternatives on the VRSS to report how strong the latest experienced stress (VRSS1) was and to report the same experience on a visual analogue scale (VAS1) with the written anchors ‘no stress’ and ‘the worst known stress.’
©2014 Foundation Acta Pædiatrica. Published by John Wiley & Sons Ltd 2014 103, pp. e173–e175
e173
n and Nilsson Alfve
A short verbal rating test scale
Verbal rating scale (VRS) second occassion
The reliability was measured by retesting the second sample (VRSS2 and VAS2) 3–5 days later via a web server (es-maker, Sahlgrenska University Hospital). Five children dropped out at this point. The local ethics committee in Stockholm, 2012/581 -315, approved the study. The statistics were calculated using STATISTICA 7.0; Statsoft Inc., Tulsa, OK, USA, and SYSRAN 1.0 for Matlab 6. The construct validity for the VRSS was evaluated using a statistical method that measures the concordance with VAS (3). The concordance indicates the level of interchangeability between two scales. The number of disorder pairs, of all possible different pairs, was calculated and defined as the measure of disorder, D. The level of order consistency is defined by the coefficient of monotonic agreement, MA, which was calculated with MA = 1 2D and ranges from 1 to 1 (3). The reliability of the VRS for stress was estimated from test–retest pairs of data as the relative change of position (RP), relative individual changes (RV) and relative change in concentration (RC). RC and RP indicate systematic disagreement, whereas RV indicates random disagreement between repeated observations. Low values indicate good reliability, whereas when RC, RP and RV were equal to 0 it indicates no disagreement and excellent reliability. Estimation of validity of the VRSS (VRSS1 versus VAS1) in samples 1 and 2: sample 1 showed D = 0.1227 and MA = 0.75, and sample 2 showed D = 0.083 and MA = 0.834. Estimation of reliability between replicates/paired observations of VRSS (VRSS1 and VRSS2): we calculated (i) the relative change of position (RP), which was low, 0.06 (SE = 0.10, 95% CI 0.26 to 0.14; (ii) the relative individual changes (RV), which also was low, 0.17 (SE = 0.10, 95% CI 0.03 to 0.36)); and (iii) the relative change in concentration (RC), which was low, 0.07 (SE = 0.11, 95% CI 0.28 to 0.15). See Figure 1.
CONCLUSION A short verbal stress rating (VRSS), ranging from zero to five, was tested and shown to have good validity and acceptable reliability. It can be recommended as an instrument for measuring present or recent stress experiences, for example, by staff providing everyday care in hospitals and primary care and by researchers.
References
5
1. Spielberger CD, editor. Manual for the state-trait anxiety inventory for children. Palo Alto, CA: Consulting Psychologists Press, 1973. 2. March JS, Parker JD, Sullivan K, Stallings P, Conners CK. The Multidimensional Anxiety Scale for Children (MASC): factor structure, reliability, and validity. J Am Acad Child Adolesc Psychiatry 1997; 36: 554–65. 3. Svensson E. Concordance between ratings using different scales for the same variables. Stat Med 2000; 19: 3483–96.
4
3
2 1 2 3 4 5 6 7
1
0 0
1
2
3
4
Verbal rating scale (VRS) first occassion
Figure 1 Scatterplot of VRS2 against VRS1.
e174
We also tested VAS1 versus VAS2, which showed statistically significant systematic disagreement in measure replicates: RC = 0.30 (SE = 0.10, 95% CI 0.50 to 0.10); and RP = 0.16 (SE = 0.16, 95% CI 0.03 to 0.35). There was a substantial, but not significant, random disagreement, RV = 0.29 (SE = 0.13, 95% CI 0.03 to 0.54). There was no statistical evidence for systematic disagreement (RP) or random disagreement (RV) for VRSS. This indicates that it is a valid and reliable scale. However, the sample we used was relatively small, which reduces the chance of detecting small effects on disagreement. The rather short interval of three to 5 days between test and retest increases the likelihood of an information bias, but increases compliance. A long interval increases the likelihood of not remembering the stress experience. For VAS, we found both systematic disagreement as a measure of change in concentration (RC) and a sign of random disagreement in individual changes (RV). The systematic disagreement for VAS explains some of the disagreement in the estimation of validity where VRSS was compared with VAS. A weakness of the VRSS is the subjective quality of the response. Another weakness could be that the reliability test was carried out with two groups of children reporting different stress experiences. However, the experience of stress as such was similar in both situations. A strength is that it is easy to respond to a short verbal scale, and current or recent data are more reliable than retrospective data.
5
APPENDIX: VERBAL STRESS RATING SCALE IN ENGLISH AND SWEDISH English translation 0. 1. 2. 3.
I I I I
didn′t feel any stress at all felt a little stress clearly felt stress. felt too much stress.
©2014 Foundation Acta Pædiatrica. Published by John Wiley & Sons Ltd 2014 103, pp. e173–e175
n and Nilsson Alfve
4. I felt a lot of stress. 5. I felt the worst stress I can think of. Swedish version
A short verbal rating test scale
2. 3. 4. 5.
Jag Jag Jag Jag
€nde ka €nde ka €nde ka €nde ka
mig mig mig den
klart stressad ganska mycket stressad mycket stressad €rsta stress som jag kan ta €nka mig va
€nde mig inte alls stressad 0. Jag ka €nde mig lite stressad 1. Jag ka
©2014 Foundation Acta Pædiatrica. Published by John Wiley & Sons Ltd 2014 103, pp. e173–e175
e175
SHORT COMMUNICATION
Validity and reliability of a new short verbal rating scale for stress for use in clinical practice G€osta Alfven ([email protected])1, Stefan Nilsson2,3 1.CLINTEC Departement, Karolinska Institute, Stockholm, Sweden 2.School of Health Sciences, University of Bor as, Bor as, Sweden 3.Institute of Health and Care Sciences, University of Gothenburg, Gothenburg, Sweden
Correspondence €sta Alfve n, CLINTEC Departement, Karolinska Go Institute, Stockholm, Sweden. Tel: +46707660835 | Email: [email protected] Received 13 May 2013; revised 18 November 2013; accepted 13 January 2014. DOI:10.1111/apa.12558
Stress is a common experience that affects the body and the brain in many ways. It can sometimes be of interest in research and clinical practice to measure the intensity of single stress reactions. For example, data on intensity can be important for understanding factors such as stress-related behaviour and pain and for a better understanding of physiological stress reactions in hormones, muscles and the circulatory system. Stress can be measured in a number of ways, including shifts in the production of cortisol and adrenaline and in skin conductance. But performing such tests is not without its difficulties, and the results can be hard to evaluate. An alternative to physiological measurements is verbal instruments. According to a search at PubMed in November 2013 and discussions with colleagues, those that have been developed so far involve many issues and, as a result, take time to answer. Two examples are the State-Trait Anxiety Inventory for Children (STAIC) with 20 items (1) and the Multidimensional Anxiety Scale for Children (MASC) with 37 items (2). However, there is a need for an instrument that assesses the intensity of single stress attacks in a simple, valid and reliable way, and we have not been able to find one. A short verbal rating scale for stress (VRSS), reporting recent or present experience of stress and easy to use in clinical routine settings and research, would meet this demand. An important issue in the development of such an instrument is how it operates, including the theoretical framework, the context of the application and the selection of optimal responses. These are also the key factors that define how we measure the quality of an instrument (3). Instruments capturing stress should be valid and reliable, easy to understand and use and have good compliance. The aim of this study was therefore to test a new VRSS developed by one of the authors.
The material consisted of two samples. The first included 28 children, seven boys and 21 girls, with a mean age of 12.6 years (range: seven to 16) visiting a clinic for recurrent pain. The second sample consisted of 34 children, 15 boys and 19 girls, aged 11.3 years (range: eight to 16). Fifteen of them were recruited from the same clinic as the first sample, and 19 were recruited from a clinic for children with needle phobias. All the children indicated that they knew what stress was and all of them were able to describe their own feeling of stress. After a brief talk about the child’s stress reaction, the child was asked to describe their latest stressful experience in a few words. For the children attending the pain clinic, this incident had happened during the last few days and not longer than a week ago. The children with needle phobias described the stress they felt when they came in for a practice injection. The researchers used a new verbal rating scale for stress (VRSS), based on a scale of zero to five, that had been developed, tested and retested in school age children, in collaboration with two psychologists. The VRSS provides an easy to understand scale that offers the children alternative responses when it comes to describing an increase in their stress levels (see Appendix). Validity of a scale is best evaluated by comparing it with a validated scale measuring the same parameter. But such a scale could not be found, so we measured the agreement between the VRSS and a visual analogue scale (VAS) measuring the same phenomenon. This was done in both groups. The child was asked to choose one of six alternatives on the VRSS to report how strong the latest experienced stress (VRSS1) was and to report the same experience on a visual analogue scale (VAS1) with the written anchors ‘no stress’ and ‘the worst known stress.’
©2014 Foundation Acta Pædiatrica. Published by John Wiley & Sons Ltd 2014 103, pp. e173–e175
e173
n and Nilsson Alfve
A short verbal rating test scale
Verbal rating scale (VRS) second occassion
The reliability was measured by retesting the second sample (VRSS2 and VAS2) 3–5 days later via a web server (es-maker, Sahlgrenska University Hospital). Five children dropped out at this point. The local ethics committee in Stockholm, 2012/581 -315, approved the study. The statistics were calculated using STATISTICA 7.0; Statsoft Inc., Tulsa, OK, USA, and SYSRAN 1.0 for Matlab 6. The construct validity for the VRSS was evaluated using a statistical method that measures the concordance with VAS (3). The concordance indicates the level of interchangeability between two scales. The number of disorder pairs, of all possible different pairs, was calculated and defined as the measure of disorder, D. The level of order consistency is defined by the coefficient of monotonic agreement, MA, which was calculated with MA = 1 2D and ranges from 1 to 1 (3). The reliability of the VRS for stress was estimated from test–retest pairs of data as the relative change of position (RP), relative individual changes (RV) and relative change in concentration (RC). RC and RP indicate systematic disagreement, whereas RV indicates random disagreement between repeated observations. Low values indicate good reliability, whereas when RC, RP and RV were equal to 0 it indicates no disagreement and excellent reliability. Estimation of validity of the VRSS (VRSS1 versus VAS1) in samples 1 and 2: sample 1 showed D = 0.1227 and MA = 0.75, and sample 2 showed D = 0.083 and MA = 0.834. Estimation of reliability between replicates/paired observations of VRSS (VRSS1 and VRSS2): we calculated (i) the relative change of position (RP), which was low, 0.06 (SE = 0.10, 95% CI 0.26 to 0.14; (ii) the relative individual changes (RV), which also was low, 0.17 (SE = 0.10, 95% CI 0.03 to 0.36)); and (iii) the relative change in concentration (RC), which was low, 0.07 (SE = 0.11, 95% CI 0.28 to 0.15). See Figure 1.
CONCLUSION A short verbal stress rating (VRSS), ranging from zero to five, was tested and shown to have good validity and acceptable reliability. It can be recommended as an instrument for measuring present or recent stress experiences, for example, by staff providing everyday care in hospitals and primary care and by researchers.
References
5
1. Spielberger CD, editor. Manual for the state-trait anxiety inventory for children. Palo Alto, CA: Consulting Psychologists Press, 1973. 2. March JS, Parker JD, Sullivan K, Stallings P, Conners CK. The Multidimensional Anxiety Scale for Children (MASC): factor structure, reliability, and validity. J Am Acad Child Adolesc Psychiatry 1997; 36: 554–65. 3. Svensson E. Concordance between ratings using different scales for the same variables. Stat Med 2000; 19: 3483–96.
4
3
2 1 2 3 4 5 6 7
1
0 0
1
2
3
4
Verbal rating scale (VRS) first occassion
Figure 1 Scatterplot of VRS2 against VRS1.
e174
We also tested VAS1 versus VAS2, which showed statistically significant systematic disagreement in measure replicates: RC = 0.30 (SE = 0.10, 95% CI 0.50 to 0.10); and RP = 0.16 (SE = 0.16, 95% CI 0.03 to 0.35). There was a substantial, but not significant, random disagreement, RV = 0.29 (SE = 0.13, 95% CI 0.03 to 0.54). There was no statistical evidence for systematic disagreement (RP) or random disagreement (RV) for VRSS. This indicates that it is a valid and reliable scale. However, the sample we used was relatively small, which reduces the chance of detecting small effects on disagreement. The rather short interval of three to 5 days between test and retest increases the likelihood of an information bias, but increases compliance. A long interval increases the likelihood of not remembering the stress experience. For VAS, we found both systematic disagreement as a measure of change in concentration (RC) and a sign of random disagreement in individual changes (RV). The systematic disagreement for VAS explains some of the disagreement in the estimation of validity where VRSS was compared with VAS. A weakness of the VRSS is the subjective quality of the response. Another weakness could be that the reliability test was carried out with two groups of children reporting different stress experiences. However, the experience of stress as such was similar in both situations. A strength is that it is easy to respond to a short verbal scale, and current or recent data are more reliable than retrospective data.
5
APPENDIX: VERBAL STRESS RATING SCALE IN ENGLISH AND SWEDISH English translation 0. 1. 2. 3.
I I I I
didn′t feel any stress at all felt a little stress clearly felt stress. felt too much stress.
©2014 Foundation Acta Pædiatrica. Published by John Wiley & Sons Ltd 2014 103, pp. e173–e175
n and Nilsson Alfve
4. I felt a lot of stress. 5. I felt the worst stress I can think of. Swedish version
A short verbal rating test scale
2. 3. 4. 5.
Jag Jag Jag Jag
€nde ka €nde ka €nde ka €nde ka
mig mig mig den
klart stressad ganska mycket stressad mycket stressad €rsta stress som jag kan ta €nka mig va
€nde mig inte alls stressad 0. Jag ka €nde mig lite stressad 1. Jag ka
©2014 Foundation Acta Pædiatrica. Published by John Wiley & Sons Ltd 2014 103, pp. e173–e175
e175
