J Head Trauma Rehabil Vol. 31, No. 1, pp. 23–29 c 2016 Wolters Kluwer Health, Inc. All rights reserved. Copyright
Three Scoring Approaches to the Neurobehavioral Symptom Inventory for Measuring Clinical Change in Service Members Receiving Intensive Treatment for Combat-Related mTBI Michael Dretsch, PhD; Joseph Bleiberg, PhD; Kathy Williams, MA; Jesus Caban, PhD; James Kelly, MD; Geoffrey Grammer, MD; Thomas DeGraba, MD Objective: To examine the use of the Neurobehavioral Symptom Inventory to measure clinical changes over time in a population of US service members undergoing treatment of mild traumatic brain injury and comorbid psychological health conditions. Setting: A 4-week, 8-hour per day, intensive, outpatient, interdisciplinary, comprehensive treatment program at the National Intrepid Center of Excellence in Bethesda, Maryland. Participants: Three hundred fourteen active-duty service members being treated for combat-related comorbid mild traumatic brain injury and psychological health conditions. Design: Repeated-measures, retrospective analysis of a single-group using a pretest-posttest treatment design. Main Measures: Three Neurobehavioral Symptom Inventory scoring methods: (1) a total summated score, (2) the 3-factor method, and (3) the 4-factor method (with and without orphan items). Results: All 3 scoring methods yielded statistically significant within-subject changes between admission and discharge. The evaluation of effect sizes indicated that the 3 different Neurobehavioral Symptom Inventory scoring methods were comparable. Conclusion: Findings indicate that the different scoring methods all have potential for assessing clinical changes in symptoms for groups of patients undergoing treatment, with no clear advantage with any one method. Key words: mild traumatic brain injury, neurobehavioral symptom inventory, postconcussive syndrome, neurorehabilitation, rehabilitation outcome assessment
I
N THE PAST 12 YEARS, the Department of Defense has recorded more than 290 000 service members who have sustained combat and non–combat-related traumatic brain injuries (TBIs), with 18% to 44% screen-
Author Affiliations: Department of Research, National Intrepid Center of Excellence, Walter Reed National Military Medical Center, Bethesda, Maryland. The authors are grateful for the writing and editorial assistance from Ms Shalini Mehta, MPH, of Booz Allen Hamilton. The authors thank CDR Petit for her support in ensuring patients completed the Neurobehavioral Symptom Inventory at admission and discharge. The views expressed in this article are those of the authors and do not reflect the official policy of the Department of Army/Navy/Air Force, Department of Defense, or US Government. The authors declare no conflicts of interest. Corresponding Author: Michael Dretsch, PhD, Department of Research, National Intrepid Center of Excellence, Walter Reed National Military Medical Center, 4860 South Palmer Rd, Bethesda, MD 20889 ([email protected]). DOI: 10.1097/HTR.0000000000000109
ing positive for posttraumatic stress.1 This complex population represents a challenge with regard to reliable characterization and tracking recovery in treatment programs. Recently, the Department of Defense chose the Neurobehavioral Symptom Inventory (NSI)2 as the assessment tool to be used as a unifying outcome metric in all Military Health System TBI clinics. The scale has been found to be a reliable tool to characterize service members with postconcussive injury.3,4,5 However, information regarding its utility as a direct measure of recovery related to treatment has not been completely explored. Within the research literature and clinical practice, valid but varied scoring approaches to the NSI are common.3–8 A recent information paper from the Defense and Veterans Brain Injury Center (http:// dvbic.dcoe.mil/information-papers)9 recommends several NSI scoring approaches for standardization of the assessment of postconcussive symptoms. However, no study has compared the different scoring systems on the 23
Copyright © 2016 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
24
JOURNAL OF HEAD TRAUMA REHABILITATION/JANUARY–FEBRUARY 2016
same population of patients and only one small sample size study10 has applied these scoring systems to preand posttest treatment data. This is especially important because the vast majority of NSI publications have used data from a single NSI administration, as opposed to serial assessment as would be typical in clinical use. In light of a lack of clear consensus, varied methods of scoring have been used in the clinical setting. One of the most familiar scoring techniques is the total score,5 which gives equal weight, making no distinction among the 22 items. In this method, a higher score is associated with greater severity of symptom reporting. Another approach is the 3-factor scoring technique. In this method, the individual NSI items are consolidated into 3 subcategories: Somatosensory, Cognitive, and Affective.6 Again, higher scores indicate more severe responses; however, this approach allows for the observation of severity levels among specific groupings of symptoms. Another commonly used scoring technique is the 4-factor technique; here the individual NSI items are grouped into 4 subcategories: Vestibular, Somatosensory, Cognitive, and Affective.4,8,11 Within this scoring method, the exclusion of 2 NSI items (hearing difficulties and loss of appetite) deemed “orphan items” is sometimes seen because they do not group well into any of the 4 subcategories mentioned earlier (ie, poor fit in the factor structure model),3,11 although they may be clinically important. The intent of this article was to compare 3 different NSI scoring methods for measuring changes in postconcussive symptoms in a large sample of service members with mild traumatic brain injury (mTBI) who received treatment in a 4-week intensive, multidisciplinary, outpatient program. METHODS Subjects Three hundred fourteen patients who provided consent at admission to the National Intrepid Center of Excellence (NICoE) program, completed the program, and had both pre- and posttest treatment NSI data were included in this retrospective analysis. Patients were referred by a primary clinician at their home station. Patients admitted to the NICoE treatment program have medically documented mTBI(s) and co-occurring psychological health conditions, who suffer from persistent symptoms, and have not responded to conventional treatment at their home duty station. The NICoE program is 4 weeks in duration, and patients attend 5 days per week for 8 hours daily. This intensive, interdisciplinary, comprehensive treatment program was designed for service members with a medically documented mTBI(s) and concurrent psychological health conditions. Patients enrolled at the NICoE
receive a comprehensive conventional medical evaluation, advanced neuroimaging, skills-based education and training, complementary and alternative medicine interventions, and self-management skill training. As part of standard care, service members complete the NSI upon admission and again at discharge. Only data from patients who consented to have their treatment data used for research purposes were included in this study. The study was performed at the NICoE, Bethesda, Maryland, following review and approval by the Walter Reed National Military Medical Center Institutional Review Board. The study was conducted in accordance with all federal laws, regulations, and standards of practice, as well as those of the Department of Defense and the Department of Army/Navy/Air Force. Materials The NSI7 is a 22-item self-report questionnaire designed to assess postconcussive symptoms. Participants rate the severity of each symptom within the past month on a 5-point Likert scale ranging from 0 (none) to 4 (very severe). Study design The study is a repeated-measures, retrospective analysis of a single-group using a pretest-posttest treatment design. Data analysis Demographic and NSI data on 314 patients were analyzed. Demographic data were based on patients’ self-report at the time of admission. The NSI scoring techniques included the following: (1) a total summated score; (2) the 3-factor method,6 which included Cognitive, Affective, and Somatosensory factors; and (3) the 4-factor method,1,4,7 which included Vestibular, Cognitive, Affective, and Somatosensory factors (without orphan items). Because of violations in normality (P < .01; ShapiroWilk test) for data based on the specific NSI scoring techniques mentioned earlier, analysis of the pre- and posttest treatment NSI scores (admission vs discharge) was conducted using the Wilcoxon signed-rank test (z) with Bonferroni corrections for multiple comparisons. In addition, a delta change score () was calculated by subtracting discharge scores from admission scores. This was transformed to a percent (% ) change using preand posttest treatment data. Cohen d12 was used to assess the effect sizes of the various scores for comparing preand postdischarge NSI score changes for the different scoring techniques. Statistically significant changes and effect sizes were used as indicators of reliable change for each of the different NSI scoring techniques.
Copyright © 2016 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
NSI Scoring Approaches for Measuring Clinical Change in Combat-Related mTBI RESULTS Subjects’ demographics are displayed in Table 1. The majority of the participants were male (97.1%), Caucasian (72.3%), and had a history of multiple deployments and multiple mTBIs. The mTBI scores reflect a self-report of a lifetime history of mTBIs. The majority of the participants (72.3%) reported that their most recent mTBI had been more than 12 months from the admission date at the NICoE. Itemized scores There were significant improvements across all 22 NSI items between admission and discharge (P ≤ .002; Bonferroni adjusted for 22 comparisons). Table 2 shows the mean scores for each item at admission and discharge, as well as quantifying patients’ change. Hearing difficulty, Poor concentration, forgetfulness, difficulty falling asleep, feeling anxious, irritability, and poor frusTABLE 1 Demographic characteristics of the sample (N = 314) Age, M (SD), y Education, M (SD), y Gender Male Female Ethnicity Caucasian Hispanic African American Other Time in service, M (SD), y Branch of service US Army US Navy US Marine Corps US Air Force Rank Junior level (E1-E3) Midlevel (E4-E6) Senior NCO (E7-E9) Officer Number of deployments 1 2-3 4+ Injury, M (SD) Self-reported lifetime mTBIs Number of blast-related mTBIs Number of LOCs Time since last mTBI 12 mo
34.7 (8.0) 13.7 (1.9) 97.1% 2.8% 72.3% 5.4% 3.5% 2.5% 13.2 (7.4) 40.6% 28.9% 21.6% 8.9% 3.2% 50.0% 34.7% 12.1% 15.9% 37.6% 41.7% 4.8 (5.2) 2.9 (2.3) 1.9 (1.1) 1.6% 8.5% 17.6% 72.3%
Abbreviations: LOC, loss of consciousness; mTBI, mild traumatic brain injury; NCO, noncommissioned officer.
25
tration showed the biggest changes as assessed by effect sizes ranging from 0.50 to 0.76. Total score There was a significant difference between admission (M = 43.7; SD = 16.8) and discharge (M = 31.7; SD = 17.8) for the total score (z = −12.08; P < .001). There was a 28% reduction in overall symptoms, with a medium effect size (d = 0.69), suggesting that the total score can identify pretest-posttest treatment changes within the context of a 4-week duration treatment program. Figure 1 characterizes the response to treatment by illustrating the percentage of patients who have no change to a 15-point change or greater on the NSI in either direction. Individual columns in Figure 1 illustrate the percentage of patients with NSI scores who get better (decrease), stay the same (no change), or get worse (increase) based on the change of points. 3-Factor scoring Results indicate that the differences between admission and discharge were significant for each of the 3 factors (P < .016; Bonferroni corrected for 3 comparisons). As can be seen in Table 3, the Somatosensory, Cognitive, and Affective factors all had medium effect sizes (d = 0.55, 0.64, and 0.72, respectively), comparable with the total summated score method. Figure 2 illustrates the items for the 3-factor scoring technique, which are categorized as improving by 1 point (Better +1), improving by 2 points or more (Better +2), no change (Same), worsening by 1 point (Worse +1), and worsening by 2 points or more (Worse +2). 4-Factor scoring Figure 3 illustrates the results for items in the 4-factor approach. Results indicate that the difference between admission and discharge was significant for each of the 4 factors (P < .013; Bonferroni corrected for 4 comparisons). As seen in Table 3, 2 of the factors, Cognitive and Affective, had medium effect sizes, with a Cohen d of 0.64 and 0.71, respectively, whereas the Somatosensory and Vestibular factors were smaller, with a Cohen d of 0.44 and 0.49, respectively. Comparison of effect sizes indicated that the NSI total score, the Cognitive factor, and the Affective factor, in both the 3-factor and 4-factor approaches, were comparable in yielding medium effect sizes (0.50). In contrast, the Somatosensory factor in the 3-factor, but not the 4-factor, approach had a medium effect size. The Somatosensory and Vestibular factors in the 4-factor approach had effect sizes smaller than 0.50. www.headtraumarehab.com
Copyright © 2016 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
26
JOURNAL OF HEAD TRAUMA REHABILITATION/JANUARY–FEBRUARY 2016
TABLE 2
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
NSI itemized scores at admission and discharge (N = 314)a NSI Items
Admission, M (SD)
Discharge, M (SD)
z
%
d
Dizzy Balance Poor coordination Headaches Nausea Vision problems Sensitivity to light Hearing difficulties Sensitivity to noise Numbness Change in taste or smell Loss of appetite Poor concentration Forgetfulness Difficulty making decisions Slowed thinking Fatigue Difficulty falling asleep Feeling anxious Feeling depressed Irritability Poor frustration
1.37 (1.10) 1.38 (1.10) 1.46 (1.17) 2.47 (1.21) 0.86 (1.10) 1.55 (1.16) 1.94 (1.33) 2.00 (1.22) 1.89 (1.39) 1.71 (1.29) 0.70 (1.11) 1.43 (1.34) 2.78 (1.05) 2.97 (1.00) 2.16 (1.31) 2.44 (1.20) 2.42 (1.23) 2.84 (1.18) 2.39 (1.21) 1.79 (1.31) 2.68 (1.16) 2.49 (1.25)
1.03 (0.95) 0.95 (0.95) 1.01 (1.05) 1.96 (1.17) 0.66 (0.93) 1.17 (1.07) 1.60 (1.29) 1.40 (1.17) 1.39 (1.26) 1.21 (1.16) 0.38 (0.83) 0.88 (1.14) 2.10 (1.20) 2.20 (1.14) 1.54 (1.29) 1.86 (1.25) 1.90 (1.21) 2.06 (1.29) 1.74 (1.23) 1.30 (1.25) 1.76 (1.26) 1.64 (1.30)
− 6.36 − 7.36 − 7.00 − 7.35 − 3.22 − 5.99 − 5.22 − 7.71 − 7.34 − 8.03 − 6.18 − 7.43 − 8.61 − 10.14 − 8.32 − 7.73 − 6.90 − 9.38 − 8.64 − 7.16 − 10.19 − 9.70
0.34 0.43 0.45 0.51 0.20 0.38 0.34 0.60 0.50 0.50 0.32 0.55 0.68 0.77 0.62 0.58 0.52 0.78 0.65 0.49 0.92 0.85
24.8 31.2 30.8 20.7 23.3 24.5 17.5 30.0 26.5 29.2 45.7 38.5 24.5 25.9 28.7 23.8 21.5 27.5 27.2 27.4 34.3 34.1
0.34 0.42 0.41 0.43 0.20 0.35 0.26 0.50b 0.37 0.41 0.34 0.44 0.60b 0.72b 0.48 0.47 0.43 0.64b 0.54b 0.39 0.76b 0.66b
Abbreviation: NSI, Neurobehavioral Symptom Inventory. a Means and standard deviations, Wilcoxon signed-rank test (z), delta change score (), percentage change (% ), and effect sizes (d) of the NSI. b Cohen d ≥0.50, medium effect size (bold values indicate medium to large effect sizes). All items significant at P ≤ .002 (Bonferroni corrected; 0.05/22).
Figure 1. Treatment response in patients assessed using Neurobehavioral Symptom Inventory total score.
Copyright © 2016 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
Dizzy, poor balance, poor coordination, nausea, vision problems, sensitivity to light, hearing difficulties, sensitivity to noise, numbness, change in taste, loss of appetite, poor concentration, forgetfulness, difficulty making decisions, slowed thinking, headaches, fatigue, difficulty falling asleep, feeling anxious, feeling depressed, irritability, poor frustration tolerance
Dizzy, poor balance, poor coordination Headaches, nausea, vision, sensitivity to light, sensitivity to noise, numbness, change in taste Poor concentration, forgetfulness, difficulty making decisions, slowed thinking Fatigue, difficulty falling asleep, feeling anxious, feeling depressed, irritability, poor frustration
Dizzy, poor balance, poor coordination, nausea, vision problems, sensitivity to light, hearing difficulties, sensitivity to noise, numbness, change in taste, loss of appetite Poor concentration, forgetfulness, difficulty making decisions, slowed thinking Headaches, fatigue, difficulty falling asleep, feeling anxious, feeling depressed, irritability, poor frustration tolerance
Item content
0-88
0-18
0-12
0-7 0-15
0-28
0-16
0-44
Score range
43.73 (16.81)
14.61 (5.63)
10.35 (3.89)
4.21 (2.93) 11.13 (5.85)
17.09 (6.25)
10.35 (3.89)
16.29 (8.73)
Admission, M (SD)
31.71 (17.80)
10.39 (6.26)
7.70 (4.37)
3.00 (2.61) 8.36 (5.44)
12.35 (6.93)
7.70 (4.37)
11.66 (8.01)
Discharge, M (SD)
Abbreviation: NSI, Neurobehavioral Symptom Inventory. a NSI 4-factor approach omits the following items: hearing difficulties and loss of appetite. b Cohen d ≥0.50, medium effect size (bold values indicate medium to large effect sizes). P ≤ .013 (Bonferroni corrected; 0.05/4).
22
6
Affective
Summated score Total score
4
Cognitive
3 7
7
Affective
4-Factor Vestibular Somatosensory
4
11
Items per factor
Cognitive
3-Factor Somatosensory
Factors
12.02
4.22
− 11.30
− 12.08
2.65
− 10.32
4.74
− 11.45
1.21 2.77
2.65
− 10.32
− 8.80 − 10.00
4.63
− 11.26
z
27.49
28.9
25.6
28.7 24.9
27.7
25.6
28.4
%
0.70b
0.71b
0.64b
0.44 0.49
0.72b
0.64b
0.55b
d
Admission and discharge 3-factor and 4-factor means and standard deviations, Wilcoxon signed-rank test (z), delta change score (), percentage change (% ), and effect sizes (d) of the NSI (N = 314)a
TABLE 3
NSI Scoring Approaches for Measuring Clinical Change in Combat-Related mTBI 27
www.headtraumarehab.com
Copyright © 2016 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
28
JOURNAL OF HEAD TRAUMA REHABILITATION/JANUARY–FEBRUARY 2016
Figure 2. Treatment response in patients assessed using Neurobehavioral Symptom Inventory 3-factor scoring.
Figure 3. Treatment response in patients assessed using Neurobehavioral Symptom Inventory 4-factor scoring.
DISCUSSION The results of this article showed that the NSI is a reliable indicator of response to a 4-week intensive, multidisciplinary, outpatient program. All 3 scoring methods yielded statistically significant within-subject changes during the 4-week period between admission and discharge. Thus, the various scoring techniques of the NSI have the ability to document clinical improvement in a military population suffering from persistent symptoms related to combat and mission-related TBIs and comorbid psychological health symptoms. There were minor differences in effect sizes between the different scoring techniques. Although the total
score had a robust effect size, the 3-factor and 4factor approaches provide more granularity that could be informative in both clinical and research settings for individualizing treatments based on symptom configuration. It is important to highlight that these differences illustrate different approaches to measuring clinical change and are not ranked on superiority. Small changes in effect sizes between the factor scoring approaches reflect differences in the categorical association of the various items for each approach. The Somatosensory and Vestibular factors had less robust effect sizes than the Cognitive and Affective factors. This reflects the lower mean scores at admission, which
Copyright © 2016 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
NSI Scoring Approaches for Measuring Clinical Change in Combat-Related mTBI would result in a smaller range for potential change between admission and discharge. However, these data should not be interpreted as smaller effect sizes being less significant in terms of clinical improvement. Although medium to large effects are the preferred size of a difference, small effects may have large clinical implications in the long-term trajectory of patients’ outcomes. As such, it is possible that the accumulative small change in somatosensory symptoms such as dizziness, balance, vision, and hearing, among others, contributed to improvements in both affective and cognitive symptoms. Similar to differences in effect sizes, our figures show various ways to illustrate degrees of improvement, lack thereof, and worsening (see Figure 1 vs Figures 2 and 3). These differences merely reflect preferences on how to illustrate patients’ responses to the treatment. Hence, patients’ change is not dictated by the arbitrary method of scoring (eg, 5-point change), but rather there are multiple approaches for quantifying patients’ change in symptoms. This article is the first to illustrate the use of 3 scoring methods to detect pretest-posttest treatment effects in a large group of comorbid patients,4,5,7,8 as would be done to compare different treatment groups in a clinical setting.
29
Limitations and future directions This article has several limitations. First, although the article illustrates symptom changes between 2 time points (ie, admission and discharge), it does not assess the psychometrics of the instrument such as test-retest reliability. The population in this study was generally older and held higher rank than other NSI studies. Because of the subspecialized and tertiary care nature of the NICoE, one cannot exclude the potential confounding influence of an unintentional selection bias. Future studies should apply the varying NSI scoring techniques to different populations within the Military Health System, Veterans Affairs Medical System, and civilian population to explore the contributions of both intrinsic (eg, comorbid psychological health conditions) and extrinsic factors (eg, treatment modality). CONCLUSION Overall, the findings indicate that the different NSI scoring techniques all have potential utility for assessing changes in postconcussive symptoms in patients undergoing treatment for comorbid mTBI and psychological health conditions, with no clear advantage with any one method.
REFERENCES 1. Invisible Wounds of War: Psychological and Cognitive Injuries, Their Consequences, and Services to Assist recovery. Santa Monica, CA: RAND Corporation; 2008. 2. Vanderploeg RD, Silva MA, Soble JR, et al. The structure of postconcussion symptoms on the Neurobehavioral Symptom Inventory: a comparison of alternative models [published online ahead of print November 20, 2013]. J Head Trauma Rehabil. doi:10.1097/HTR.0000000000000009. 3. Wilde EA, Whiteneck GG, Bogner J, et al. Recommendations for the use of common outcome measures in traumatic brain injury research. Arch Phys Med Rehabil. 2010;91(11):1650–1660. e1617. 4. Soble JR, Silva MA, Vanderploeg RD, et al. Normative data for the Neurobehavioral Symptom Inventory (NSI) and postconcussion symptom profiles among TBI, PTSD, and nonclinical samples. Clin Neuropsychol. 2014;28:614–632. 5. King PR, Donnelly KT, Donnelly JP, et al. Psychometric study of the Neurobehavioral Symptom Inventory. J Rehabil Res Dev. 2012;49(6):879–888. 6. Caplan LJ, Ivins B, Poole JH, Vanderploeg RD, Jaffee MS, Schwab K. The structure of postconcussive symptoms in three military samples. J Head Trauma Rehabil. 2010;25:447–460.
7. Cicerone KD, Kalmar K. Persistent postconcussion syndrome: the structure of subjective complaints after mild traumatic brain injury. J Head Trauma Rehabil. 1995;10(3):1–17. 8. Meterko M, Baker E, Stolzmann KL, Hendricks AM, Cicerone KD, Lew HL. Psychometric assessment of the Neurobehavioral Symptom Inventory-22: the structure of persistent postconcussive symptoms following deployment-related mild traumatic brain injury among veterans. J Head Trauma Rehabil. 2012;27(1):55–62. 9. Neurobehavioral Symptom Inventory (NSI): Recommendations for Scoring and Serial Administration for Concussion Health Care Outcomes Standardization. Silver Spring, MD: Defense and Veterans Brain Injury Center. http://dvbic.dcoe.mil/information-papers. Accessed September 25, 2014. 10. Sullivan KW, Quinn JE, Pramuka M, Sharkey LA, French LM. Outcomes from a pilot study using computer-based rehabilitative tools in a military population. Stud Health Technol Inform. 2012;181:71–77. 11. Vanderploeg RD, Cooper DB, Belanger HG, et al. Screening for postdeployment conditions: development and cross-validation of an embedded validity scale in the Neurobehavioral Symptom Inventory. J Head Trauma Rehabil. 2014;29(1):1–10. 12. Cohen J. A power primer. Psychol Bull. 1992;112:155–159.
www.headtraumarehab.com Copyright © 2016 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
Three Scoring Approaches to the Neurobehavioral Symptom Inventory for Measuring Clinical Change in Service Members Receiving Intensive Treatment for Combat-Related mTBI Michael Dretsch, PhD; Joseph Bleiberg, PhD; Kathy Williams, MA; Jesus Caban, PhD; James Kelly, MD; Geoffrey Grammer, MD; Thomas DeGraba, MD Objective: To examine the use of the Neurobehavioral Symptom Inventory to measure clinical changes over time in a population of US service members undergoing treatment of mild traumatic brain injury and comorbid psychological health conditions. Setting: A 4-week, 8-hour per day, intensive, outpatient, interdisciplinary, comprehensive treatment program at the National Intrepid Center of Excellence in Bethesda, Maryland. Participants: Three hundred fourteen active-duty service members being treated for combat-related comorbid mild traumatic brain injury and psychological health conditions. Design: Repeated-measures, retrospective analysis of a single-group using a pretest-posttest treatment design. Main Measures: Three Neurobehavioral Symptom Inventory scoring methods: (1) a total summated score, (2) the 3-factor method, and (3) the 4-factor method (with and without orphan items). Results: All 3 scoring methods yielded statistically significant within-subject changes between admission and discharge. The evaluation of effect sizes indicated that the 3 different Neurobehavioral Symptom Inventory scoring methods were comparable. Conclusion: Findings indicate that the different scoring methods all have potential for assessing clinical changes in symptoms for groups of patients undergoing treatment, with no clear advantage with any one method. Key words: mild traumatic brain injury, neurobehavioral symptom inventory, postconcussive syndrome, neurorehabilitation, rehabilitation outcome assessment
I
N THE PAST 12 YEARS, the Department of Defense has recorded more than 290 000 service members who have sustained combat and non–combat-related traumatic brain injuries (TBIs), with 18% to 44% screen-
Author Affiliations: Department of Research, National Intrepid Center of Excellence, Walter Reed National Military Medical Center, Bethesda, Maryland. The authors are grateful for the writing and editorial assistance from Ms Shalini Mehta, MPH, of Booz Allen Hamilton. The authors thank CDR Petit for her support in ensuring patients completed the Neurobehavioral Symptom Inventory at admission and discharge. The views expressed in this article are those of the authors and do not reflect the official policy of the Department of Army/Navy/Air Force, Department of Defense, or US Government. The authors declare no conflicts of interest. Corresponding Author: Michael Dretsch, PhD, Department of Research, National Intrepid Center of Excellence, Walter Reed National Military Medical Center, 4860 South Palmer Rd, Bethesda, MD 20889 ([email protected]). DOI: 10.1097/HTR.0000000000000109
ing positive for posttraumatic stress.1 This complex population represents a challenge with regard to reliable characterization and tracking recovery in treatment programs. Recently, the Department of Defense chose the Neurobehavioral Symptom Inventory (NSI)2 as the assessment tool to be used as a unifying outcome metric in all Military Health System TBI clinics. The scale has been found to be a reliable tool to characterize service members with postconcussive injury.3,4,5 However, information regarding its utility as a direct measure of recovery related to treatment has not been completely explored. Within the research literature and clinical practice, valid but varied scoring approaches to the NSI are common.3–8 A recent information paper from the Defense and Veterans Brain Injury Center (http:// dvbic.dcoe.mil/information-papers)9 recommends several NSI scoring approaches for standardization of the assessment of postconcussive symptoms. However, no study has compared the different scoring systems on the 23
Copyright © 2016 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
24
JOURNAL OF HEAD TRAUMA REHABILITATION/JANUARY–FEBRUARY 2016
same population of patients and only one small sample size study10 has applied these scoring systems to preand posttest treatment data. This is especially important because the vast majority of NSI publications have used data from a single NSI administration, as opposed to serial assessment as would be typical in clinical use. In light of a lack of clear consensus, varied methods of scoring have been used in the clinical setting. One of the most familiar scoring techniques is the total score,5 which gives equal weight, making no distinction among the 22 items. In this method, a higher score is associated with greater severity of symptom reporting. Another approach is the 3-factor scoring technique. In this method, the individual NSI items are consolidated into 3 subcategories: Somatosensory, Cognitive, and Affective.6 Again, higher scores indicate more severe responses; however, this approach allows for the observation of severity levels among specific groupings of symptoms. Another commonly used scoring technique is the 4-factor technique; here the individual NSI items are grouped into 4 subcategories: Vestibular, Somatosensory, Cognitive, and Affective.4,8,11 Within this scoring method, the exclusion of 2 NSI items (hearing difficulties and loss of appetite) deemed “orphan items” is sometimes seen because they do not group well into any of the 4 subcategories mentioned earlier (ie, poor fit in the factor structure model),3,11 although they may be clinically important. The intent of this article was to compare 3 different NSI scoring methods for measuring changes in postconcussive symptoms in a large sample of service members with mild traumatic brain injury (mTBI) who received treatment in a 4-week intensive, multidisciplinary, outpatient program. METHODS Subjects Three hundred fourteen patients who provided consent at admission to the National Intrepid Center of Excellence (NICoE) program, completed the program, and had both pre- and posttest treatment NSI data were included in this retrospective analysis. Patients were referred by a primary clinician at their home station. Patients admitted to the NICoE treatment program have medically documented mTBI(s) and co-occurring psychological health conditions, who suffer from persistent symptoms, and have not responded to conventional treatment at their home duty station. The NICoE program is 4 weeks in duration, and patients attend 5 days per week for 8 hours daily. This intensive, interdisciplinary, comprehensive treatment program was designed for service members with a medically documented mTBI(s) and concurrent psychological health conditions. Patients enrolled at the NICoE
receive a comprehensive conventional medical evaluation, advanced neuroimaging, skills-based education and training, complementary and alternative medicine interventions, and self-management skill training. As part of standard care, service members complete the NSI upon admission and again at discharge. Only data from patients who consented to have their treatment data used for research purposes were included in this study. The study was performed at the NICoE, Bethesda, Maryland, following review and approval by the Walter Reed National Military Medical Center Institutional Review Board. The study was conducted in accordance with all federal laws, regulations, and standards of practice, as well as those of the Department of Defense and the Department of Army/Navy/Air Force. Materials The NSI7 is a 22-item self-report questionnaire designed to assess postconcussive symptoms. Participants rate the severity of each symptom within the past month on a 5-point Likert scale ranging from 0 (none) to 4 (very severe). Study design The study is a repeated-measures, retrospective analysis of a single-group using a pretest-posttest treatment design. Data analysis Demographic and NSI data on 314 patients were analyzed. Demographic data were based on patients’ self-report at the time of admission. The NSI scoring techniques included the following: (1) a total summated score; (2) the 3-factor method,6 which included Cognitive, Affective, and Somatosensory factors; and (3) the 4-factor method,1,4,7 which included Vestibular, Cognitive, Affective, and Somatosensory factors (without orphan items). Because of violations in normality (P < .01; ShapiroWilk test) for data based on the specific NSI scoring techniques mentioned earlier, analysis of the pre- and posttest treatment NSI scores (admission vs discharge) was conducted using the Wilcoxon signed-rank test (z) with Bonferroni corrections for multiple comparisons. In addition, a delta change score () was calculated by subtracting discharge scores from admission scores. This was transformed to a percent (% ) change using preand posttest treatment data. Cohen d12 was used to assess the effect sizes of the various scores for comparing preand postdischarge NSI score changes for the different scoring techniques. Statistically significant changes and effect sizes were used as indicators of reliable change for each of the different NSI scoring techniques.
Copyright © 2016 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
NSI Scoring Approaches for Measuring Clinical Change in Combat-Related mTBI RESULTS Subjects’ demographics are displayed in Table 1. The majority of the participants were male (97.1%), Caucasian (72.3%), and had a history of multiple deployments and multiple mTBIs. The mTBI scores reflect a self-report of a lifetime history of mTBIs. The majority of the participants (72.3%) reported that their most recent mTBI had been more than 12 months from the admission date at the NICoE. Itemized scores There were significant improvements across all 22 NSI items between admission and discharge (P ≤ .002; Bonferroni adjusted for 22 comparisons). Table 2 shows the mean scores for each item at admission and discharge, as well as quantifying patients’ change. Hearing difficulty, Poor concentration, forgetfulness, difficulty falling asleep, feeling anxious, irritability, and poor frusTABLE 1 Demographic characteristics of the sample (N = 314) Age, M (SD), y Education, M (SD), y Gender Male Female Ethnicity Caucasian Hispanic African American Other Time in service, M (SD), y Branch of service US Army US Navy US Marine Corps US Air Force Rank Junior level (E1-E3) Midlevel (E4-E6) Senior NCO (E7-E9) Officer Number of deployments 1 2-3 4+ Injury, M (SD) Self-reported lifetime mTBIs Number of blast-related mTBIs Number of LOCs Time since last mTBI 12 mo
34.7 (8.0) 13.7 (1.9) 97.1% 2.8% 72.3% 5.4% 3.5% 2.5% 13.2 (7.4) 40.6% 28.9% 21.6% 8.9% 3.2% 50.0% 34.7% 12.1% 15.9% 37.6% 41.7% 4.8 (5.2) 2.9 (2.3) 1.9 (1.1) 1.6% 8.5% 17.6% 72.3%
Abbreviations: LOC, loss of consciousness; mTBI, mild traumatic brain injury; NCO, noncommissioned officer.
25
tration showed the biggest changes as assessed by effect sizes ranging from 0.50 to 0.76. Total score There was a significant difference between admission (M = 43.7; SD = 16.8) and discharge (M = 31.7; SD = 17.8) for the total score (z = −12.08; P < .001). There was a 28% reduction in overall symptoms, with a medium effect size (d = 0.69), suggesting that the total score can identify pretest-posttest treatment changes within the context of a 4-week duration treatment program. Figure 1 characterizes the response to treatment by illustrating the percentage of patients who have no change to a 15-point change or greater on the NSI in either direction. Individual columns in Figure 1 illustrate the percentage of patients with NSI scores who get better (decrease), stay the same (no change), or get worse (increase) based on the change of points. 3-Factor scoring Results indicate that the differences between admission and discharge were significant for each of the 3 factors (P < .016; Bonferroni corrected for 3 comparisons). As can be seen in Table 3, the Somatosensory, Cognitive, and Affective factors all had medium effect sizes (d = 0.55, 0.64, and 0.72, respectively), comparable with the total summated score method. Figure 2 illustrates the items for the 3-factor scoring technique, which are categorized as improving by 1 point (Better +1), improving by 2 points or more (Better +2), no change (Same), worsening by 1 point (Worse +1), and worsening by 2 points or more (Worse +2). 4-Factor scoring Figure 3 illustrates the results for items in the 4-factor approach. Results indicate that the difference between admission and discharge was significant for each of the 4 factors (P < .013; Bonferroni corrected for 4 comparisons). As seen in Table 3, 2 of the factors, Cognitive and Affective, had medium effect sizes, with a Cohen d of 0.64 and 0.71, respectively, whereas the Somatosensory and Vestibular factors were smaller, with a Cohen d of 0.44 and 0.49, respectively. Comparison of effect sizes indicated that the NSI total score, the Cognitive factor, and the Affective factor, in both the 3-factor and 4-factor approaches, were comparable in yielding medium effect sizes (0.50). In contrast, the Somatosensory factor in the 3-factor, but not the 4-factor, approach had a medium effect size. The Somatosensory and Vestibular factors in the 4-factor approach had effect sizes smaller than 0.50. www.headtraumarehab.com
Copyright © 2016 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
26
JOURNAL OF HEAD TRAUMA REHABILITATION/JANUARY–FEBRUARY 2016
TABLE 2
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
NSI itemized scores at admission and discharge (N = 314)a NSI Items
Admission, M (SD)
Discharge, M (SD)
z
%
d
Dizzy Balance Poor coordination Headaches Nausea Vision problems Sensitivity to light Hearing difficulties Sensitivity to noise Numbness Change in taste or smell Loss of appetite Poor concentration Forgetfulness Difficulty making decisions Slowed thinking Fatigue Difficulty falling asleep Feeling anxious Feeling depressed Irritability Poor frustration
1.37 (1.10) 1.38 (1.10) 1.46 (1.17) 2.47 (1.21) 0.86 (1.10) 1.55 (1.16) 1.94 (1.33) 2.00 (1.22) 1.89 (1.39) 1.71 (1.29) 0.70 (1.11) 1.43 (1.34) 2.78 (1.05) 2.97 (1.00) 2.16 (1.31) 2.44 (1.20) 2.42 (1.23) 2.84 (1.18) 2.39 (1.21) 1.79 (1.31) 2.68 (1.16) 2.49 (1.25)
1.03 (0.95) 0.95 (0.95) 1.01 (1.05) 1.96 (1.17) 0.66 (0.93) 1.17 (1.07) 1.60 (1.29) 1.40 (1.17) 1.39 (1.26) 1.21 (1.16) 0.38 (0.83) 0.88 (1.14) 2.10 (1.20) 2.20 (1.14) 1.54 (1.29) 1.86 (1.25) 1.90 (1.21) 2.06 (1.29) 1.74 (1.23) 1.30 (1.25) 1.76 (1.26) 1.64 (1.30)
− 6.36 − 7.36 − 7.00 − 7.35 − 3.22 − 5.99 − 5.22 − 7.71 − 7.34 − 8.03 − 6.18 − 7.43 − 8.61 − 10.14 − 8.32 − 7.73 − 6.90 − 9.38 − 8.64 − 7.16 − 10.19 − 9.70
0.34 0.43 0.45 0.51 0.20 0.38 0.34 0.60 0.50 0.50 0.32 0.55 0.68 0.77 0.62 0.58 0.52 0.78 0.65 0.49 0.92 0.85
24.8 31.2 30.8 20.7 23.3 24.5 17.5 30.0 26.5 29.2 45.7 38.5 24.5 25.9 28.7 23.8 21.5 27.5 27.2 27.4 34.3 34.1
0.34 0.42 0.41 0.43 0.20 0.35 0.26 0.50b 0.37 0.41 0.34 0.44 0.60b 0.72b 0.48 0.47 0.43 0.64b 0.54b 0.39 0.76b 0.66b
Abbreviation: NSI, Neurobehavioral Symptom Inventory. a Means and standard deviations, Wilcoxon signed-rank test (z), delta change score (), percentage change (% ), and effect sizes (d) of the NSI. b Cohen d ≥0.50, medium effect size (bold values indicate medium to large effect sizes). All items significant at P ≤ .002 (Bonferroni corrected; 0.05/22).
Figure 1. Treatment response in patients assessed using Neurobehavioral Symptom Inventory total score.
Copyright © 2016 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
Dizzy, poor balance, poor coordination, nausea, vision problems, sensitivity to light, hearing difficulties, sensitivity to noise, numbness, change in taste, loss of appetite, poor concentration, forgetfulness, difficulty making decisions, slowed thinking, headaches, fatigue, difficulty falling asleep, feeling anxious, feeling depressed, irritability, poor frustration tolerance
Dizzy, poor balance, poor coordination Headaches, nausea, vision, sensitivity to light, sensitivity to noise, numbness, change in taste Poor concentration, forgetfulness, difficulty making decisions, slowed thinking Fatigue, difficulty falling asleep, feeling anxious, feeling depressed, irritability, poor frustration
Dizzy, poor balance, poor coordination, nausea, vision problems, sensitivity to light, hearing difficulties, sensitivity to noise, numbness, change in taste, loss of appetite Poor concentration, forgetfulness, difficulty making decisions, slowed thinking Headaches, fatigue, difficulty falling asleep, feeling anxious, feeling depressed, irritability, poor frustration tolerance
Item content
0-88
0-18
0-12
0-7 0-15
0-28
0-16
0-44
Score range
43.73 (16.81)
14.61 (5.63)
10.35 (3.89)
4.21 (2.93) 11.13 (5.85)
17.09 (6.25)
10.35 (3.89)
16.29 (8.73)
Admission, M (SD)
31.71 (17.80)
10.39 (6.26)
7.70 (4.37)
3.00 (2.61) 8.36 (5.44)
12.35 (6.93)
7.70 (4.37)
11.66 (8.01)
Discharge, M (SD)
Abbreviation: NSI, Neurobehavioral Symptom Inventory. a NSI 4-factor approach omits the following items: hearing difficulties and loss of appetite. b Cohen d ≥0.50, medium effect size (bold values indicate medium to large effect sizes). P ≤ .013 (Bonferroni corrected; 0.05/4).
22
6
Affective
Summated score Total score
4
Cognitive
3 7
7
Affective
4-Factor Vestibular Somatosensory
4
11
Items per factor
Cognitive
3-Factor Somatosensory
Factors
12.02
4.22
− 11.30
− 12.08
2.65
− 10.32
4.74
− 11.45
1.21 2.77
2.65
− 10.32
− 8.80 − 10.00
4.63
− 11.26
z
27.49
28.9
25.6
28.7 24.9
27.7
25.6
28.4
%
0.70b
0.71b
0.64b
0.44 0.49
0.72b
0.64b
0.55b
d
Admission and discharge 3-factor and 4-factor means and standard deviations, Wilcoxon signed-rank test (z), delta change score (), percentage change (% ), and effect sizes (d) of the NSI (N = 314)a
TABLE 3
NSI Scoring Approaches for Measuring Clinical Change in Combat-Related mTBI 27
www.headtraumarehab.com
Copyright © 2016 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
28
JOURNAL OF HEAD TRAUMA REHABILITATION/JANUARY–FEBRUARY 2016
Figure 2. Treatment response in patients assessed using Neurobehavioral Symptom Inventory 3-factor scoring.
Figure 3. Treatment response in patients assessed using Neurobehavioral Symptom Inventory 4-factor scoring.
DISCUSSION The results of this article showed that the NSI is a reliable indicator of response to a 4-week intensive, multidisciplinary, outpatient program. All 3 scoring methods yielded statistically significant within-subject changes during the 4-week period between admission and discharge. Thus, the various scoring techniques of the NSI have the ability to document clinical improvement in a military population suffering from persistent symptoms related to combat and mission-related TBIs and comorbid psychological health symptoms. There were minor differences in effect sizes between the different scoring techniques. Although the total
score had a robust effect size, the 3-factor and 4factor approaches provide more granularity that could be informative in both clinical and research settings for individualizing treatments based on symptom configuration. It is important to highlight that these differences illustrate different approaches to measuring clinical change and are not ranked on superiority. Small changes in effect sizes between the factor scoring approaches reflect differences in the categorical association of the various items for each approach. The Somatosensory and Vestibular factors had less robust effect sizes than the Cognitive and Affective factors. This reflects the lower mean scores at admission, which
Copyright © 2016 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
NSI Scoring Approaches for Measuring Clinical Change in Combat-Related mTBI would result in a smaller range for potential change between admission and discharge. However, these data should not be interpreted as smaller effect sizes being less significant in terms of clinical improvement. Although medium to large effects are the preferred size of a difference, small effects may have large clinical implications in the long-term trajectory of patients’ outcomes. As such, it is possible that the accumulative small change in somatosensory symptoms such as dizziness, balance, vision, and hearing, among others, contributed to improvements in both affective and cognitive symptoms. Similar to differences in effect sizes, our figures show various ways to illustrate degrees of improvement, lack thereof, and worsening (see Figure 1 vs Figures 2 and 3). These differences merely reflect preferences on how to illustrate patients’ responses to the treatment. Hence, patients’ change is not dictated by the arbitrary method of scoring (eg, 5-point change), but rather there are multiple approaches for quantifying patients’ change in symptoms. This article is the first to illustrate the use of 3 scoring methods to detect pretest-posttest treatment effects in a large group of comorbid patients,4,5,7,8 as would be done to compare different treatment groups in a clinical setting.
29
Limitations and future directions This article has several limitations. First, although the article illustrates symptom changes between 2 time points (ie, admission and discharge), it does not assess the psychometrics of the instrument such as test-retest reliability. The population in this study was generally older and held higher rank than other NSI studies. Because of the subspecialized and tertiary care nature of the NICoE, one cannot exclude the potential confounding influence of an unintentional selection bias. Future studies should apply the varying NSI scoring techniques to different populations within the Military Health System, Veterans Affairs Medical System, and civilian population to explore the contributions of both intrinsic (eg, comorbid psychological health conditions) and extrinsic factors (eg, treatment modality). CONCLUSION Overall, the findings indicate that the different NSI scoring techniques all have potential utility for assessing changes in postconcussive symptoms in patients undergoing treatment for comorbid mTBI and psychological health conditions, with no clear advantage with any one method.
REFERENCES 1. Invisible Wounds of War: Psychological and Cognitive Injuries, Their Consequences, and Services to Assist recovery. Santa Monica, CA: RAND Corporation; 2008. 2. Vanderploeg RD, Silva MA, Soble JR, et al. The structure of postconcussion symptoms on the Neurobehavioral Symptom Inventory: a comparison of alternative models [published online ahead of print November 20, 2013]. J Head Trauma Rehabil. doi:10.1097/HTR.0000000000000009. 3. Wilde EA, Whiteneck GG, Bogner J, et al. Recommendations for the use of common outcome measures in traumatic brain injury research. Arch Phys Med Rehabil. 2010;91(11):1650–1660. e1617. 4. Soble JR, Silva MA, Vanderploeg RD, et al. Normative data for the Neurobehavioral Symptom Inventory (NSI) and postconcussion symptom profiles among TBI, PTSD, and nonclinical samples. Clin Neuropsychol. 2014;28:614–632. 5. King PR, Donnelly KT, Donnelly JP, et al. Psychometric study of the Neurobehavioral Symptom Inventory. J Rehabil Res Dev. 2012;49(6):879–888. 6. Caplan LJ, Ivins B, Poole JH, Vanderploeg RD, Jaffee MS, Schwab K. The structure of postconcussive symptoms in three military samples. J Head Trauma Rehabil. 2010;25:447–460.
7. Cicerone KD, Kalmar K. Persistent postconcussion syndrome: the structure of subjective complaints after mild traumatic brain injury. J Head Trauma Rehabil. 1995;10(3):1–17. 8. Meterko M, Baker E, Stolzmann KL, Hendricks AM, Cicerone KD, Lew HL. Psychometric assessment of the Neurobehavioral Symptom Inventory-22: the structure of persistent postconcussive symptoms following deployment-related mild traumatic brain injury among veterans. J Head Trauma Rehabil. 2012;27(1):55–62. 9. Neurobehavioral Symptom Inventory (NSI): Recommendations for Scoring and Serial Administration for Concussion Health Care Outcomes Standardization. Silver Spring, MD: Defense and Veterans Brain Injury Center. http://dvbic.dcoe.mil/information-papers. Accessed September 25, 2014. 10. Sullivan KW, Quinn JE, Pramuka M, Sharkey LA, French LM. Outcomes from a pilot study using computer-based rehabilitative tools in a military population. Stud Health Technol Inform. 2012;181:71–77. 11. Vanderploeg RD, Cooper DB, Belanger HG, et al. Screening for postdeployment conditions: development and cross-validation of an embedded validity scale in the Neurobehavioral Symptom Inventory. J Head Trauma Rehabil. 2014;29(1):1–10. 12. Cohen J. A power primer. Psychol Bull. 1992;112:155–159.
www.headtraumarehab.com Copyright © 2016 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
