Childs Nerv Syst (2015) 31:693–697 DOI 10.1007/s00381-015-2650-5

ORIGINAL PAPER

Inter-rater reliability and validity of two ataxia rating scales in children with brain tumours H. Hartley & B. Pizer & S. Lane & C. Sneade & R. Pratt & A. Bishop & R. Kumar

Received: 20 November 2014 / Accepted: 6 February 2015 / Published online: 4 March 2015 # Springer-Verlag Berlin Heidelberg 2015

Abstract Objectives This study aimed to investigate the inter-rater reliability and construct validity of the Scale for the Assessment and Rating of Ataxia (SARA) and Brief Ataxia Rating Scale (BARS) in children with posterior fossa tumours. These scales have been developed for adults with genetic ataxias, and the performance of these scales in children with brain tumours has not previously been described. Methods The participants, who had undergone surgical resection for a posterior fossa tumour (inclusion criteria age 4– 18 years), were recruited from the neuro-oncology service at a tertiary children’s hospital. Children were assessed using the

Statistical analysis was completed by Dr. S. Lane of University of Liverpool. Electronic supplementary material The online version of this article (doi:10.1007/s00381-015-2650-5) contains supplementary material, which is available to authorized users. H. Hartley (*) : C. Sneade : R. Pratt Physiotherapy Department, Alder Hey Children’s NHS Foundation Trust, Eaton Road, Liverpool L12 2AP, UK e-mail: [email protected] B. Pizer Oncology Department, Alder Hey Children’s NHS Foundation Trust, Eaton Road, Liverpool L12 2AP, UK S. Lane Department of Biostatistics, University of Liverpool, Liverpool, UK A. Bishop Institute of Primary Care and Health Sciences, Keele University, Staffordshire, UK R. Kumar Department of Paediatric Neurology, Alder Hey Children’s NHS Foundation Trust, Eaton Road, Liverpool L12 2AP, UK

SARA, BARS and Paediatric Evaluation of Disability Index (PEDI) mobility domain, a measure of function. Children were independently rated by two therapists to determine the inter-rater reliability of the SARA and BARS. The construct validity was determined by assessing the correlation between the two scales with the PEDI. Results Forty-four children were recruited. Inter-rater reliability was good for both scales, demonstrating the strong correlations (SARA, r=0.94; BARS, r=0.91) and the good consistency (93 % of SARA and 90 % of BARS paired scores differing by less than 2 points) between two raters. Both ataxia scales demonstrated a strong negative correlation with the mobility domain of the PEDI (SARA, r=−0.77; BARS, r= −0.76), indicating that more severe ataxia was associated with worse mobility. The mean time for completion of the SARA was 4.5 and 2.7 min for the BARS. Conclusions The SARA and BARS are reliable and valid measures and appear to be of equal value in determining the severity of ataxia in children with posterior fossa tumours. Keywords Ataxia . Brain tumours . Paediatric . Scales

Introduction Outcome measures examining quality of survival are important in the management of children with brain tumours, providing a pre-operative baseline, determining an impact of interventions, and identifying effectiveness of rehabilitation. Ataxia is the most frequently described motor deficit in children with brain tumours, occurring in 58–90 % of children with posterior fossa tumours [2, 6]. There is minimal evidence in the literature to determine how ataxia is recorded clinically which makes it difficult to objectively establish if ataxia has

694

changed after intervention. Therefore, there is a particular need for a validated outcome measure to assess the severity of ataxia in this patient group. There is, however, little previously published work in this area. The Scale for the Assessment and Rating of Ataxia (SARA) and the Brief Ataxia Rating Scale (BARS) have recently been developed based on the standard examination of coordination, initially evaluated in adults with genetic ataxias. They have both demonstrated high inter-rater reliability and significant correlation with the International Cooperative Ataxia Rating Scale (ICARS) [3, 4]. It is reported the BARS is quicker to carry out [3]. There is no longitudinal data on the BARS, whereas a recent study has demonstrated that the SARA is sensitive to change [5]. The SARA has also been examined in different population groups and has been demonstrated to correlate with the Barthel scale (a well-recognized adult activity of daily life (ADL) scale) [4, 7]. Children with brain tumours have a distinct set of issues, e.g. quick onset of ataxia, potential for gross change pre/post-operatively, hydrocephalus and increased intracranial pressure adding deficits distinct from ataxia, in contrast from the slow progression of genetic ataxias. It is therefore useful to evaluate if these ataxia scales are of value in this population group which has not been previously reported. This study aimed to investigate the inter-rater reliability and construct validity of the SARA and BARS. Feasibility was a secondary aim, exploring time taken to complete the scales.

Childs Nerv Syst (2015) 31:693–697

whole domain or as a stand-alone domain. The second rater subsequently assessed the child completing the SARA and BARS and a GCI of ataxia severity. If the child was an inpatient, all assessments were completed within a 24-h period. If the child attended a follow-up clinic, assessments were completed on the same day. The order of completion of rating scales was randomized to minimize fatigue and practice effects. The time taken to complete each scale was recorded. The raters underwent one joint training session with the outcome measures prior to commencement of the study.

Results Participants Forty-four patients were recruited following the surgical resection of a posterior fossa brain tumour. Characteristics are detailed in Table 1.tgroup Descriptive statistics The median SARA score was 5 (range 1–21; a higher score representing worse ataxia), and the median BARS score was 4 (range 0–20; a higher score representing worse ataxia). The median PEDI mobility domain score was 55 (range 18–59; a higher score representing better mobility). Table 1

Participant characteristics

Methods

Characteristic

The study was approved by the relevant local ethics and research and development committees. Informed and written consent was obtained from all children and/or their parents. Inclusion criteria included children with a posterior fossa tumour aged between 4 and 18 years during the study period (June 2012–September 2013). Patients newly diagnosed within the study period were evaluated as inpatients in the postoperative phase, prior to any adjuvant therapy. Patients who were previously treated for a posterior fossa tumour were assessed during outpatient follow-up. The children were rated independently by two raters (from a pool of three) who were experienced paediatric neuro-physiotherapists. The first rater completed the SARA, BARS and Paediatric Evaluation of Disability Index (PEDI) mobility domain and recorded a Global Clinical Impression (GCI) of ataxia severity (categorized as no, mild, moderate or severe). The PEDI was chosen as it is the only validated functional measure in paediatric-acquired brain injury and is also similar in scope to the ADL scales that have been used in the validation of the ataxia scales in the adult population. It has self-care, mobility and social function sections which can be used as a

Age at diagnosis (years) Median (range) Age at entering study Median (range) Time (years) post diagnosis Median (range) Histology Low-grade glioma Medulloblastoma Ependymoma Schwannoma Patient group Inpatient Outpatient Adjuvant treatment (at assessment) Surgery Surgery and radiotherapy Surgery and chemotherapy Surgery, radiotherapy and chemotherapy

Females N=23

6 (1–15)

Males N=21

7 (1–18)

Total N=44

6 (1–18)

13 (7–16) 12 (8–18) 12 (7–18) 3 (0–11)

6 (0–13)

4 (0–13)

12 4 5 2

7 12 2 0

19 16 7 2

3 20

6 15

9 35

13 6

8 2

21 8

0 4

7 4

7 8

Childs Nerv Syst (2015) 31:693–697

695

Inter-rater reliability

Construct validity

The Bland-Altman comparison plots (Fig. 1) illustrate no systematic difference between the two raters on both SARA and BARS. In both plots, only two observations lie outside the limits of agreement, which is within statistical expectation. The correlation coefficients between the two raters were high, 0.94 for the SARA and 0.91 for the BARS. There was minimal disagreement between raters; 37 out of 40 (93 %) paired SARA total scores and 36 out of 40 (90 %) paired BARS total scores were within 2 points of each other.

The correlation of the SARA and the BARS (for rater 1) against the PEDI was determined using Spearman’s rank correlation coefficient to quantify the relationship between the ataxia scale score and the functional ability (Fig. 2). The SARA demonstrated a strong correlation with the PEDI (rater 1, r=−0.77); lower SARA scores were associated with higher PEDI scores. The BARS also demonstrated a good association with the PEDI (rater 1, r=−0.76). Feasibility The mean time to complete the SARA was 4.5 min (SD 1.49; range 2.0–9.0), and the mean time for the BARS was 2.7 min (SD 1.36; range 1.0–8.0).

a

SARA difference rater 1 - rater 2

4.00

2.00

.00

-2.00 .00

5.00

10.00

15.00

20.00

25.00

SARA average score rater 1 & rater 2

b

BARS difference rater 1 - rater 2

7.50

5.00

2.50

.00

-2.50

.00

5.00

10.00

15.00

20.00

BARS average score rater 1 & rater 2

Fig. 1 Bland-Altman method comparison plots. a Comparison plot of SARA rater 1/rater 2. Dashed lines represent limits of agreement, mean difference±1.96×standard deviation of differences. b Comparison plot of BARS rater 1/rater 2. Dashed lines represent limits of agreement, mean difference±1.96×standard deviation of differences

Fig. 2 Scatter plot of participants’ scores of ataxia scales against their PEDI mobility domain scores. a SARA score (rater 1) against PEDI mobility domain scores. b BARS score (rater 1) against PEDI mobility domain scores

696

Threshold values of ataxia scales with GCI grades of severity The web supplement details the sensitivity and specificity of varying thresholds of SARA and BARS scores for classifying the GCI severity of ataxia. Cut-off values are detailed in Supplemental Tables 1, 2 and 3 which indicate that it would be appropriate to use a score of 2 on both the SARA and the BARS as a threshold for distinguishing no ataxia from mild ataxia: a score of 7 on the SARA and a score of 5 on the BARS as the threshold for determining mild from moderate ataxia and a score of 14 on the SARA and a score of 12 on the BARS as the threshold for distinguishing moderate from severe ataxia.

Discussion Both the SARA and the BARS have good inter-rater reliability in children with posterior fossa tumours. This reflects the high inter-rater reliability reported for the SARA (intraclass correlation coefficient (ICC) 0.98) and the BARS (ICC 0.91) in adult populations [3, 4, 7]. A pilot study in healthy children by Brandsma and colleagues also reported good reliability for the SARA (ICC 0.81) and the BARS (ICC 0.70) [6]. Our study suggests a higher inter-rater reliability than the latter paediatric pilot study, which may have been influenced by the fact that despite opening recruitment to children aged 4–18 years during our study time frame, only children aged 7–18 were identified who met all our inclusion criteria. The pilot study of Brandsma and colleagues included children from the age of 4. They identified increased variance in rater scores for the SARA and the BARS under the age of 6, impacting on the overall reliability. For future research, it is important to validate the scales in children with posterior fossa tumours under the age of 7. We found strong associations of both the SARA (r=−0.77) and the BARS (r=−0.76) with the PEDI (mobility domain). Low SARA and BARS scores (indicating less ataxia) were correlated with higher PEDI (better mobility) scores. This provides evidence for the construct validity of both ataxia scales. It also demonstrates the impact of ataxia on functional ability. This is comparable to results in adult studies examining the correlation of the SARA with the Barthel scale (an adult ADL measure). The correlation of r=−0.8 between the SARA and the Barthel scale was reported in the original SARA development study [4], with subsequent studies reporting correlations of r=−0.63 and r=−0.71, respectively [1, 8]. The mean time to complete either scale was under 5 min, making either feasible to use in routine clinical care. A similar time for the SARA assessment (mean 4 min) is presented in an adult study [8]. All items on both scales could be assessed in

Childs Nerv Syst (2015) 31:693–697

all participants, with no item unrecorded in children across varying ages and different functional levels providing further evidence of the clinical feasibility of the scales. The youngest participant in our study as mentioned was 7 years old, although the minimum age for our inclusion criteria was 4 years of age. A lower age limit of 4 years was used in published normative data for the ICARS, another wellknown ataxia scale with many more items. The ICARS was not chosen to be examined in this study as it is a lengthy tool to use (100-point scale) which can be difficult in paediatrics especially in the acute phase where engagement and compliance can sometimes be a challenge. The SARA has been demonstrated to take a third of the time of the ICARS [8], and both the SARA and the BARS have been shown to highly correlate with the ICARS, again proving their value [3, 4]. The majority (n=25) of the participants had ‘mild’ ataxia. The range of scores for both scales indicates that no child scored at the upper (worse) end of the scales. As expected, there were few children with severe ataxia, given that most children were under long-term surveillance, who are likely to have improved over time. Our participant sample was thus not fully representative of the entire spectrum of ataxia severity. However, it is encouraging that there is a high correlation between raters within this smaller range of scores. To further validate the SARA and BARS in children with brain tumours, additional psychometric properties should be assessed, e.g. internal consistency and testretest reliability. Although it is not possible to determine the criterion validity as there is no current gold standard to compare the scales with, other elements of construct validity could be explored. The sensitivity of the scales to change in ataxia over time is another key property, in order to determine the impact of treatments and measure progress guiding rehabilitation. Longitudinal studies should be carefully planned considering the pilot data in healthy children reports that both the SARA and BARS are age dependent in healthy children (up to age of 14 where adult optimum values are reached) [1]. Further work is being undertaken to establish complete European paediatric normative values which would be useful to distinguish if a change over time is due to recovery or age. To our knowledge, this is the first study examining the SARA and the BARS in children with posterior fossa tumours. It complements the pilot study in healthy children and further work underway by the Childhood Ataxia and Cerebellar Group providing normative data for the scales in healthy children [1, D. Sival 2014, personal communication]. Ataxia is the most common motor impairment in children with posterior fossa tumours, and it can significantly affect functional ability. It has been reported as a long-term problem [6] and is under-recognized as a persistent disability in survivors of childhood

Childs Nerv Syst (2015) 31:693–697

posterior fossa tumours; therefore, the importance of accurately assessing this is clear.

Conclusion We have provided initial evidence that the SARA and the BARS are reliable and valid measures of ataxia in children with posterior fossa tumours and are feasible to use in routine clinical practice. Compliance with ethical standards Conflict of interest H. Hartley reports no conflicts of interest. Prof. B. Pizer reports no conflicts of interest. Dr. S. Lane reports no conflicts of interest. C. Sneade reports no conflicts of interest. R. Pratt reports no conflicts of interest. Dr. A. Bishop is funded by a NIHR Research Professorship for Professor NE Foster (NIHR-RP-011-015) and reports no conflicts of interest. Dr. R. Kumar reports no conflicts of interest. Research involving human participants All procedures performed in studies involving human participants were in accordance with the ethical standards of the local research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards Informed consent Informed consent was obtained from all individual participants (or parent/guardian) included in the study.

697

References 1. Brandsma R, Spits A, Kuiper M et al (2014) Ataxia rating scales are age-dependent in healthy children. Developmental Medicine and Child. Neurology 56:556–563 2. Di Rocco C, Chieffo D, Pettorini B, Massimi L, Calderelli M, Tamburrini G (2010) Preoperative and postoperative neurological, neuropsychological and behavioural impairment in children with posterior cranial fossa astrocytomas and medulloblastomas: the role of the tumour and the impact of the surgical treatment. Childs Nerv Syst 26: 1173–1188 3. Schmahmann J, Gardner R, MacMore J, Vangel M (2009) Development of a Brief Ataxia Scale (BARS) based on a modified form of the ICARS. Mov Disord 24(12):1820–1828 4. Schmitz-Hubsch T, Tezenas du Montcel S, Baliko L et al (2006) Scale for the assessment and rating of ataxia. Neurology 66(11): 1717–1720 5. Schmitz-Hubsch T, Fimmers R, Rackowicz M, Rola R et al (2010) Responsiveness of different rating instruments in spinocerebellar ataxia patients. Neurology 24(8):678–684 6. Sonderkaer S, Schmiegelow M, Carstensen H, Neilsen L, Muller J, Schmiegelow K (2003) Long term neurological outcome of childhood brain tumours treated by surgery only. J Clin Oncol 21:1347–1351 7. Weyer A, Abele M, Schmitz-Hubsch T, Schoch B, Frings M, Timman D, Klockgether T (2007) Reliability and validity of the scale for assessment and rating of ataxia: a study in 64 ataxia patients. Mov Disord 22(11):1633–1637 8. Yabe I, Matsushima M, Soma H, Basri R, Sasaki H (2008) Usefulness of the scale for assessment and rating of ataxia (SARA). J Neurol Sci 266(1):164–166

Inter-rater reliability and validity of two ataxia rating scales in children with brain tumours.

This study aimed to investigate the inter-rater reliability and construct validity of the Scale for the Assessment and Rating of Ataxia (SARA) and Bri...
205KB Sizes 0 Downloads 7 Views