Support Care Cancer DOI 10.1007/s00520-015-2732-7
REVIEW ARTICLE
The psychometric properties of cancer multisymptom assessment instruments: a clinical review Aynur Aktas 1 & Declan Walsh 1,2,3 & Jordanka Kirkova 1
Received: 27 January 2015 / Accepted: 29 March 2015 # Springer-Verlag Berlin Heidelberg 2015
Abstract Purpose Various instruments are used to assess both individual and multiple cancer symptoms. We evaluated the psychometric properties of cancer multisymptom assessment instruments. Methods An Ovid MEDLINE search was done. All searches were limited to adults and in English. All instruments published from 2005 to 2014 (and with at least one validity test) were included. We excluded those who only reported content validity. Instruments were categorized by the three major types of symptom measurement scales employed as follows: visual analogue (VAS), verbal rating (VRS), and numerical rating (NRS) scales. They were then examined in two areas: (1) psychometric thoroughness (number of tests) and (2) Presented at the 20th International Symposium of the Multinational Association of Supportive Care in Cancer (MASCC) Houston, Texas, USA, June 26–28, 2008 Electronic supplementary material The online version of this article (doi:10.1007/s00520-015-2732-7) contains supplementary material, which is available to authorized users. Harry R. Horvitz Center is a World Health Organization Demonstration Project in Palliative Medicine and an ESMO Designated Center of Integrated Oncology and Palliative Care. * Declan Walsh [email protected]
psychometric strength of evidence (validity, reliability, generalizability). We also assigned an empirical global psychometric quality score (which combined the concepts of thoroughness and strength of evidence) to rank the instruments. Results We analyzed 57 instruments (17 original, 40 modifications). They varied in types of scales used, symptom dimensions measured, and time frames evaluated. Of the 57, 10 used VAS, 28 VRS, and 19 NRS. The Edmonton Symptom Assessment System (ESAS), ESAS-Spanish, Hospital Anxiety and Depression Scale (HADS), Profile of Mood States (POMS), Symptom Distress Scale (SDS), M.D. Anderson Symptom Inventory (MDASI)-Russian, and MDASI-Taiwanese were the most comprehensively tested for validity and reliability. The ESAS, ESAS-Spanish, ASDS-2, Memorial Symptom Assessment Scale (MSAS)-SF, POMS, SDS, MDASI (and some translations), and MDASI-Heart Failure all showed good validity and reliability. Conclusions The MDASI appeared to be the best overall from a psychometric perspective. This was followed by the ESAS, ESAS-Spanish, POMS, SDS, and some MDASI translations. VRS-based instruments were most common. There was a wide range of psychometric rigor in validation. Consequently, meta-analysis was not possible. Most cancer multisymptom assessment instruments need further extensive validation to establish the excellent reliability and validity required for clinical utility and meaningful research. Keywords Instruments . Patient-reported outcomes . Psychometrics . Reliability . Validity
1
Section of Palliative Medicine and Supportive Oncology, Department of Solid Tumor Oncology, Cleveland Clinic, Taussig Cancer Institute, Cleveland, OH, USA
2
The Harry R. Horvitz Center Chair in Palliative Medicine and Supportive Oncology, Cleveland Clinic, Cleveland, OH, USA
Introduction
Faculty of Health Sciences, Trinity College Dublin, University College Dublin, Old Chemistry Building Extension Trinity College Dublin 2, Dublin, Ireland
Patient-reported outcomes (PRO) include subjective assessments of symptoms, function, health status, and health-
3
Support Care Cancer
related quality of life (HRQOL) [1]. PRO may improve evaluation of disease and treatment outcomes [2–4]. The Center for Medical Technology Policy recommended inclusion of PRO into adult oncology research [5]. To conduct rigorous PRO research, symptom data should ideally be gathered by valid, reliable, and responsive instruments. For symptom PRO to be successful in clinical practice, any instrument should meet adequate psychometric criteria but also have good generalizability. Several guidelines exist to help establish evidence of reliability and validity [6, 7]. Assessment burden and instrument responsiveness to change are also a challenge [8]. In a previous systematic review, we identified 44 cancer symptom assessment instruments [9]. We evaluated them for content, time frame, validity, reliability, and clinical utility. Heterogeneity was evident in design, symptom number and type, methodology, and assessment timeframe [9]. We noted there appeared to be no ideal instrument with excellent psychometric properties although did not address the issue in detail [9]. In this clinical review, we extended this work to specifically examine the psychometric properties of cancer multisymptom assessment instruments, in particular, those which employ patient self-reported rating scales and where symptom evaluation is their primary purpose.
from each study that met inclusion criteria. Data included patient and disease characteristics, study design (original or modifications), and clinical setting. We only evaluated instruments whose primary purpose is symptom assessment. Quality of life instruments were not included, although some contain symptom items. Instruments were excluded which reported: (1) No reliability or validity tests (2) Only content validity. Each author separately performed a detailed instrument assessment. All psychometric tests were reported as categorized by the original authors. For convenience, instruments were grouped by the three major types of symptom measurement scales they used as follow: visual analogue (VAS), verbal rating (VRS), and 11-point numerical rating scale (NRS). They were then examined in three ways [10]: (1) Thoroughness: the number of validity and reliability tests reported for each instrument (2) Strength of evidence: high instrument correlation scores for both good validity and reliability (3) Global quality score: psychometric value and also generalizability.
Methods Thoroughness Data sources and search strategy A MEDLINE search using Ovid MEDLINE (Ovid Technologies Inc., New York, NY) retrieved abstracts (2005–2014) with the following key words: (1) BSeverity of Illness Index^ or BKarnofsky Performance Status^ and Bany Neoplasms^ or BPalliative Care,^ (2) BSymptom^ within two words of Bassessment^ (in either order, in article title, abstract, or indexing) and Bany Neoplasms^ or BPalliative Care^ (3) BQuestionnaires^ and Bany Neoplasms^ or BPalliative Care^. All searches were limited to adults and in English. An electronic search of the Web of Science identified journal articles cited from our previous report [9] from 2006 to 2014. Finally, a reference hand search was done of the papers identified through MEDLINE. Duplicate citations were eliminated using EndNote Version X3 for Windows (Thomson Reuters, NY, USA), reference organization software. Data analysis Published recommendations [10–15] to evaluate instrument validity and reliability were reviewed and outlined by an interdisciplinary research team with both clinical expertise and scientific background. Two authors (J.K., A.A.) independently reviewed studies to identify eligible instruments. Final inclusion was by mutual agreement. Information was then abstracted
Thoroughness (Appendix 1) was ranked in four levels (none, poor, moderate, good). This was done solely by test number and variety, i.e., a larger number of published validity and reliability tests was deemed to reflect greater thoroughness. This was easier to evaluate than strength of evidence. For example, known-group (discriminant) validity was tested for some instruments between different antitumor treatments, and at various levels of Karnofsky performance status (KPS). In this case, we ranked both tests as Bpoor^ for thoroughness (BX;^ Appendix 1), because although they represented one type of validity, but in the same population. However, if consistently high correlations were obtained, then strength of evidence was rated as Bgood^ (BXXX;^ Appendix 1). Strength of evidence To rank strength of evidence (none, poor, moderate, good) consensus criteria were established (Appendix 2). For most, high (r≥0.7) correlation coefficient scores (0=worst and 1= best) were deemed to reflect good validity and reliability. Longitudinal design, sample size >100, and multiple confirmatory statistical methods gave additional weight to strength of evidence scores (Appendix 1). In validation studies by factor analysis, a sample size of >100 subjects is recommended [15]. In smaller validation study, we reduced the strength of evidence
Support Care Cancer
level accordingly [16]. We also gave lower strength of evidence scores to convenience samples (as non-consecutive patient enrollment might cause selection bias). We considered that psychometric tests performed in addition to Delphi techniques (expert opinion) improved instrument credibility, and scored them as good for content validity. Similarly, forward and backward translations of instrument modifications were considered to ensure better semantic and content equivalence. Validity was assessed separately: (1) Known-group validity (measured at a single point in time) (2) Response to change (repeated measures) (3) Instrument sensitivity and specificity [area under the curve (AUC)] [17]. Known-group validity and response to change needed at least one statistically significant test (p0.5 on the receiver operator characteristic curve (ROC). Outcome measurement also requires good response to change [10]. These characteristics assist objective analysis of symptom associations. Except for the HADS, instrument goals (symptom screen or response measurement) were never specified. The absence of any specific intervention in instrument validation studies complicates response differences [28]. Although different treatments for fatigue were tested in some clinical trials, single ESAS severity items [72, 73] did not improve. Despite its importance in outcomes research, predictive validity was tested infrequently [77, 79]. Given the subjective nature of symptoms and their variability, the most appropriate criterion in validation of an instrument modification should be the original instrument [77]. This also supported the need for multidimensional symptom assessment. Internal consistency of subscales (or factors from factor analysis) could be tested separately, because they might represent separate symptom constructs within the total instrument [56, 58]. Intra-rater reliability tests were less common than those for instrument consistency. This was possibly due to symptom variability over time. Intra-rater reliability was preferred by the WCQ [52] because each item in the multi-item instrument was viewed as an equal but separate measure of an underlying construct [12]. Inter-rater reliability is particularly important when instruments target the very ill [22, 24, 25, 83], and needs extensive testing. Agreement between instruments was rarely tested. Both good agreement (small mean differences) and moderate to high correlations (strong relationships and consistent item rank) may support better reliability [96]. Other challenges included the use of different time frames during instrument validation. For example, the Functional Living Index Cancer (FLIC) [97] which assesses pain and nausea during the Bpast 2 weeks,^ and depression Bat times, ^ was compared with the Memorial Symptom Assessment Scale (MSAS) which has a Bpast week^ time frame [49].
Similarly, the scales compared were never of the same type, and generally not interconverted to a similar format. All VASbased instruments were compared to VRS instruments. These included the MSAS [58], HADS [50], or the European Organization for Research and Treatment of Cancer Quality of Life Questionnaire Version 3 (EORTC-QLQ C-30) [98]. All VRS instruments were validated against VAS; all NRS-based instruments, except one [57], had been validated only against VAS instruments [99]. Symptom scores from a single-item scale were sometimes correlated to combined multi-item scores that allegedly measured the same construct [45]. For example, single anxiety and depression distress items were correlated with aggregated subscale scores for the same symptoms in the HADS [57]. Another was the MDASI-Japanese [45] interference scales correlated with single items from EORTC-QLQ C-30. The moderate correlations observed suggested the single-item scales actually measured different aspects of more complex constructs. Both single-item and composite scores may theoretically provide useful clinical information. Composite scores, e.g., the Distress Index of the MSAS [58], may also speed follow-up assessments. However, separate symptoms or symptom subscales might be more clinically relevant [87, 100]. The effect size of any intervention on each subscale, and the level of correlation between subscales may also determine the efficiency of composite scores [101]. Psychometrically sound instruments should also have wide generalizability. This requires large study sample sizes, so variability (measurement error) is reduced [10]. Overall generalizability was limited. Population heterogeneity was common, and likely influenced convergent, divergent, and factorial validity. For instance, the RSCL Black of energy^ and Bheadache^ items loaded on the psychological scale factor, when the sample was heterogeneous in disease stage and treatment. In contrast, it was on the physical scale during antitumor treatment [56]. Attrition and loss of data challenge generalizability in palliative medicine symptom research in particular [66, 82]. We excluded quality of life (QOL) instruments from the review. QOL is a broader construct. Some include symptoms, but that is not their primary purpose. Our focus was on symptom measures. While QOL may serve as a proxy measure for symptom burden, it does not suffice for clinical practice in which the focus is on physical symptom control. In this review, we only assessed the psychometric properties of cancer multisymptom instruments. We did not look at clinical utility, i.e., time to complete of individual instruments. The psychometric properties of an instrument are not the only determinant of its value in a clinical or research setting. Psychometric rigor must be accompanied by clinical utility. The global psychometric score was constructed to help us consolidate and rank the instruments for the purpose of this review. It is not intended to be used for future studies or development of new instruments.
Support Care Cancer
Heterogeneity in study design, symptom content, and the extent of psychometric validation affected the final global evaluation. We believe the choice of a valid and reliable instrument should be based upon instrument purpose, target population, psychometric thoroughness, and strength of evidence. The ability of a PRO measure to improve decision making in research relies on its psychometric strength to capture symptom burden [6]. Comprehensive instruments with good psychometric properties and generalizability are still needed.
References 1.
2.
3.
Conclusions Fifty-seven cancer multisymptom assessment instruments (17 original; 40 modifications) were identified in this review. Most used VRS to measure symptoms. Psychometric thoroughness for the ESAS, ESAS-Spanish, HADS, Profile of Mood States (POMS), SDS, MDASI-Russian, and MDASI-Taiwanese was good. The ESAS, ESASSpanish, MSAS-SF, POMS, SDS, some MDASI translations (Chinese, Greek, Taiwanese), and MDASI-Heart Failure all had good strength of evidence. Only the original MDASI had both very good validity and good reliability. Studies that examined the validity and reliability of original ESAS and ESAS-Spanish consistently revealed good thoroughness and strength of evidence. Individual instruments had particular advantages for specific populations, or certain clinical or research goals. There was a wide range of psychometric rigor in validation. As a result, meta-analysis was not possible. In most, important psychometric properties (Thoroughness and strength of evidence) of VAS, VRS, and NRS instruments ranged from moderate to good for validity. Similarly, reliability varied from poor to good. Most available cancer multisymptom assessment instruments need further extensive validation to establish the excellent reliability and validity required for clinical utility and meaningful research.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13. Acknowledgments This paper was completed in part while one of the authors (D.W.) was a Lord Harris Visiting Fellow at the Tseu Medical Institute, Manchester Harris College, University of Oxford, UK, May– June 2011. We are grateful to Gretchen Hallerberg for help with the review of the literature. We thank Matthew Karafa, PhD; Wael Lasheen, MD; and Curtis Tatsuoka, PhD for their comments on the manuscript and Ellen Schleckman and Aditya Nair for their administrative assistance. Funding This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors. Jordanka Kirkova was supported in part by a fellowship grant in End-of-Life Care from the Mt. Sinai Health Care Foundation. Conflict of interest The authors declare that there is no conflict of interest.
14.
15.
16. 17.
(FDA) FaDA. Food and Drug Administration (FDA) (2011) Guidance for industry. Patient-reported outcome measures: use in medical product development to support labeling claims. In: Drugs/GuidanceComplianceRegulatoryInformation/Guidances/ UCM193282.pdf. hwfgd, editor. Accessed 24 May 20132011 Sloan JA, Berk L, Roscoe J, Fisch MJ, Shaw EG, Wyatt G et al (2007) Integrating patient-reported outcomes into cancer symptom management clinical trials supported by the National Cancer Institute-sponsored clinical trials networks. J Clin Oncol 25(32): 5070–5077 Cleeland CS (2007) Symptom burden: multiple symptoms and their impact as patient-reported outcomes. J Natl Cancer Inst Monogr 37:16–21 Greenhalgh J, Meadows K (1999) The effectiveness of the use of patient-based measures of health in routine practice in improving the process and outcomes of patient care: a literature review. J Eval Clin Pract 5(4):401–416 Policy CfMT. Center for Medical Technology Policy. Recommendations for incorporating patient-reported outcomes (PROs) into clinical comparative effectiveness research (CER) in adult oncology. 2013. http://www.cmtp.net.org/wp-content/ uploads/downloads/2012/05/PRO-EGDpdf [Internet]. 2013 Frost MH, Reeve BB, Liepa AM, Stauffer JW, Hays RD (2007) What is sufficient evidence for the reliability and validity of patient-reported outcome measures? Value Health 10(Suppl 2): S94–S105 DeVon HA, Block ME, Moyle-Wright P, Ernst DM, Hayden SJ, Lazzara DJ et al (2007) A psychometric toolbox for testing validity and reliability. J Nurs Scholarsh 39(2):155–164 Swartz RJ, Schwartz C, Basch E, Cai L, Fairclough DL, McLeod L et al (2011) The king's foot of patient-reported outcomes: current practices and new developments for the measurement of change. Qual Life Res 20(8):1159–1167 Kirkova J, Davis MP, Walsh D, Tiernan E, O'Leary N, LeGrand SB et al (2006) Cancer symptom assessment instruments: a systematic review. J Clin Oncol 24(9):1459–1473 McDowel I (2006) Measuring health: a guide to rating scales and questionnaires. the theoretical and technical foundations of health measurement, 3rd edn. Oxford University Press, New York, pp 10–54 Lipscomb J, Snyder CF, Gotay CC (2007) Cancer outcomes measurement: through the lens of the Medical Outcomes Trust framework. Qual Life Res 16(1):143–164 Bowling A (2005) Theory of measurement. measuring health: a review of quality of life measurement scales, 3rd edn. Open University Press, Berkshire, pp 10–18 Streiner DL, GN (2003) Validity. Health measurement scales a practical guide to their development and use, New York, United States, Oxford University Press, pp. 172-93 Bourbonnais FF, Perreault A, Bouvette M (2004) Introduction of a pain and symptom assessment tool in the clinical setting–lessons learned. J Nurs Manag 12(3):194–200 de Vet HC, Ader HJ, Terwee CB, Pouwer F (2005) Are factor analytical techniques used appropriately in the validation of health status questionnaires? A systematic review on the quality of factor analysis of the SF-36. Qual Life Res 14(5):1203–1218, dicussion 19-21, 23-4 Mundfrom DSD, Ke TL (2005) Minimum sample size recommendations for conducting factor analysis. Int J Test 5:159–168 Le Pechoux C, Laplanche A, Faivre-Finn C, Ciuleanu T, Wanders R, Lerouge D et al (2011) Clinical neurological outcome and quality of life among patients with limited small-cell cancer treated with two different doses of prophylactic cranial irradiation in the
Support Care Cancer
18. 19.
20.
21.
22.
23.
24.
25.
26.
27.
28.
29.
30.
31.
32.
33.
34.
35.
intergroup phase III trial (PCI99-01, EORTC 22003-08004, RTOG 0212 and IFCT 99-01). Ann Oncol 22(5):1154–1163 Tucci RA, Bartels KL (1998) Patient use of the symptom reporting tool. Clin J Oncol Nurs 2(3):97–99 Kutner JS, Kassner CT, Nowels DE (2001) Symptom burden at the end of life: hospice providers' perceptions. J Pain Symptom Manage 21(6):473–480 Edmonds PM, Stuttaford JM, Penny J, Lynch AM, Chamberlain J (1998) Do hospital palliative care teams improve symptom control? Use of a modified STAS as an evaluation tool. Palliat Med 12(5):345–351 Kuebler KK, KO (1998) Psychometric evaluation of an objective assessment instrument to measure pain, dyspnea and restlessness. J Palliat Care Report from 12th International Congress on Care of the Terminally ill 14:125 McMillan SC, Moody LE (2003) Hospice patient and caregiver congruence in reporting patients' symptom intensity. Cancer Nurs 26(2):113–118 Given B, Given CW, McCorkle R, Kozachik S, Cimprich B, Rahbar MH et al (2002) Pain and fatigue management: results of a nursing randomized clinical trial. Oncol Nurs Forum 29(6):949– 956 Nekolaichuk CL, Maguire TO, Suarez-Almazor M, Rogers WT, Bruera E (1999) Assessing the reliability of patient, nurse, and family caregiver symptom ratings in hospitalized advanced cancer patients. J Clin Oncol 17(11):3621–3630 Nekolaichuk CL, Bruera E, Spachynski K, MacEachern T, Hanson J, Maguire TO (1999) A comparison of patient and proxy symptom assessments in advanced cancer patients. Palliat Med 13(4):311–323 Ivanova MO, Ionova TI, Kalyadina SA, Uspenskaya OS, Kishtovich AV, Guo H et al (2005) Cancer-related symptom assessment in Russia: validation and utility of the Russian M. D. Anderson Symptom Inventory. J Pain Symptom Manage 30(5): 443–453 Lin CC, Chang AP, Cleeland CS, Mendoza TR, Wang XS (2007) Taiwanese version of the M. D. Anderson symptom inventory: symptom assessment in cancer patients. J Pain Symptom Manage 33(2):180–188 Moro C, Brunelli C, Miccinesi G, Fallai M, Morino P, Piazza M et al (2006) Edmonton symptom assessment scale: Italian validation in two palliative care settings. Support Care Cancer 14(1):30– 37 Rhodes VA, McDaniel RW (1999) The index of nausea, vomiting, and Retching: a new format of the lndex of Nausea and Vomiting. Oncol Nurs Forum 26(5):889–894 Wang XS, Laudico AV, Guo H, Mendoza TR, Matsuda ML, Yosuico VD et al (2006) Filipino version of the M. D. Anderson Symptom Inventory: validation and multisymptom measurement in cancer patients. J Pain Symptom Manage 31(6):542–552 Yun YH, Mendoza TR, Kang IO, You CH, Roh JW, Lee CG et al (2006) Validation study of the Korean version of the M. D. Anderson Symptom Inventory. J Pain Symptom Manage 31(4): 345–352 Fu MR, Rhodes V, Xu B (2002) The Chinese translation of the index of nausea, vomiting, and retching. Cancer Nurs 25(2):134– 140 Baba K, Fransson P, Lindh J (2007) Use of a modified ESAS in cancer patients: a pilot study of patient and staff experiences. Int J Palliat Nurs 13(12):610–616 Fadol A, Mendoza T, Gning I, Kernicki J, Symes L, Cleeland CS et al (2008) Psychometric testing of the MDASI-HF: a symptom assessment instrument for patients with cancer and concurrent heart failure. J Card Fail 14(6):497–507 Armstrong TS, Gning I, Mendoza TR, Weinberg JS, Gilbert MR, Tortorice ML et al (2009) Clinical utility of the MDASI-BT in
36.
37.
38.
39.
40.
41.
42.
43.
44.
45.
46.
47.
48.
49.
50. 51.
52.
patients with brain metastases. J Pain Symptom Manage 37(3): 331–340 Claessens P, Menten J, Schotsmans P, Broeckaert B (2011) Development and validation of a modified version of the Edmonton Symptom Assessment Scale in a Flemish palliative care population. Am J Hosp Palliat Care 28(7):475–482 Chinda M, Jaturapatporn D, Kirshen AJ, Udomsubpayakul U (2011) Reliability and validity of a Thai version of the edmonton symptom assessment scale (ESAS-Thai). J Pain Symptom Manage 42(6):954–960 Carvajal A, Centeno C, Watson R, Bruera E (2011) A comprehensive study of psychometric properties of the Edmonton Symptom Assessment System (ESAS) in Spanish advanced cancer patients. Eur J Cancer 47(12):1863–1872 Guirimand F, Buyck J-F, Lauwers-Allot E, Revnik J, Kerguen T, Aegerter P et al (2010) Cancer-related symptom assessment in France: validation of the French M. D. Anderson Symptom Inventory. J Pain Symptom Manage 39(4):721–733 Yildirim Y, Tokem Y, Bozkurt N, Fadiloglu C, Uyar M, Uslu R (2011) Reliability and validity of the Turkish version of the Memorial Symptom Assessment Scale in cancer patients. Asian Pac J Cancer Prev 12(12):3389–3396 Badger TA, Segrin C, Meek P (2011) Development and validation of an instrument for rapidly assessing symptoms: the general symptom distress scale. J Pain Symptom Manage 41(3):535–548 Aoun SM, Monterosso L, Kristjanson LJ, McConigley R (2011) Measuring symptom distress in palliative care: psychometric properties of the Symptom Assessment Scale (SAS). J Palliat Med 14(3):315–321 Wang XS, Wang Y, Guo H, Mendoza TR, Hao XS, Cleeland CS (2004) Chinese version of the M. D. Anderson Symptom Inventory: validation and application of symptom measurement in cancer patients. Cancer 101(8):1890–1901 Mystakidou K, Cleeland C, Tsilika E, Katsouda E, Primikiri A, Parpa E et al (2004) Greek M.D. Anderson Symptom Inventory: validation and utility in cancer patients. Oncology 67(3-4):203– 210 Okuyama T, Wang XS, Akechi T, Mendoza TR, Hosaka T, Cleeland CS et al (2003) Japanese version of the MD Anderson Symptom Inventory: a validation study. J Pain Symptom Manage 26(6):1093–1104 Abu-Saad Huijer H, Sagherian K, Tamim H (2014) Validation of the Arabic version of the memorial symptom assessment scale among Lebanese cancer patients. J Pain Symptom Manage Kwon JH, Nam SH, Koh S, Hong YS, Lee KH, Shin SW et al (2013) Validation of the edmonton symptom assessment system in Korean patients with cancer. J Pain Symptom Manage 46(6):947– 956 Harding G, Cella D, Robinson D Jr, Mahadevia PJ, Clark J, Revicki DA (2007) Symptom burden among patients with renal cell carcinoma (RCC): content for a symptom index. Health Qual Life Outcomes 5:34 Hickman SE, Tilden VP, Tolle SW (2001) Family reports of dying patients' distress: the adaptation of a research tool to assess global symptom distress in the last week of life. J Pain Symptom Manage 22(1):565–574 Zigmond AS, Snaith RP (1983) The hospital anxiety and depression scale. Acta Psychiatr Scand 67(6):361–370 Morrow GR (1984) Methodology in behavioral and psychosocial cancer research. The assessment of nausea and vomiting. Past problems, current issues and suggestions for future research. Cancer 53(10 Suppl):2267–2280 Sitzia J, Dikken C, Hughes J (1997) Psychometric evaluation of a questionnaire to document side-effects of chemotherapy. J Adv Nurs 25(5):999–1007
Support Care Cancer 53.
54.
55.
56.
57.
58.
59.
60.
61.
62.
63.
64.
65.
66.
67.
68. 69.
70.
71.
Rhodes VA, Watson PM, Johnson MH (1984) Development of reliable and valid measures of nausea and vomiting. Cancer Nurs 7(1):33–41 Barresi MJ, Shadbolt B, Byrne D, Stuart-Harris R (2003) The development of the Canberra symptom scorecard: a tool to monitor the physical symptoms of patients with advanced tumours. BMC Cancer 3:32 Chang VT, Hwang SS, Feuerman M, Kasimis BS, Thaler HT (2000) The memorial symptom assessment scale short form (MSAS-SF). Cancer 89(5):1162–1171 de Haes JC, van Knippenberg FC, Neijt JP (1990) Measuring psychological and physical distress in cancer patients: structure and application of the Rotterdam Symptom Checklist. Br J Cancer 62(6):1034–1038 Munro AJ, Potter S (1996) A quantitative approach to the distress caused by symptoms in patients treated with radical radiotherapy. Br J Cancer 74(4):640–647 Portenoy RK, Thaler HT, Kornblith AB, Lepore JM, FriedlanderKlar H, Kiyasu E et al (1994) The Memorial Symptom Assessment Scale: an instrument for the evaluation of symptom prevalence, characteristics and distress. Eur J Cancer 30A(9): 1326–1336 Rhodes VA, McDaniel RW, Homan SS, Johnson M, Madsen R (2000) An instrument to measure symptom experience. Symptom occurrence and symptom distress. Cancer Nurs 23(1):49–54 Stein KD, Denniston M, Baker F, Dent M, Hann DM, Bushhouse S et al (2003) Validation of a modified Rotterdam Symptom Checklist for use with cancer patients in the United States. J Pain Symptom Manage 26(5):975–989 Watson M, ML, Maguire GP (1992) Further development of a quality of life measure for cancer patients: The Rotterdam Symptom Checklist (revised). Psychooncology 1:35-44 Cleeland CS, Mendoza TR, Wang XS, Chou C, Harle MT, Morrissey M et al (2000) Assessing symptom distress in cancer patients: the M.D. Anderson Symptom Inventory. Cancer 89(7): 1634–1646 Chow E, Wong R, Connolly R, Hruby G, Franzcr, Franssen E et al (2001) Prospective assessment of symptom palliation for patients attending a rapid response radiotherapy program. feasibility of telephone follow-up. J Pain Symptom Manage 22(2):649–656 McCorkle R, Quint-Benoliel J (1983) Symptom distress, current concerns and mood disturbance after diagnosis of life-threatening disease. Soc Sci Med 17(7):431–438 Holmes S (1991) Preliminary investigations of symptom distress in two cancer patient populations: evaluation of a measurement instrument. J Adv Nurs 16(4):439–446 Bruera E, MacDonal S (1993) Audit methods: the edmonton symptom assessment system. In: Higgenson I (ed) Clinical audit in palliative care, 1st edn. Radcliffe Medical Press, Oxford, pp 61– 77 Morrow GR (1992) A patient report measure for the quantification of chemotherapy induced nausea and emesis: psychometric properties of the Morrow assessment of nausea and emesis (MANE). Br J Cancer Suppl 19:S72–S74 Slocumb EM, Cole FL (1991) A practical approach to content validation. Appl Nurs Res 4(4):192–195 Brown V, Sitzia J, Richardson A, Hughes J, Hannon H, Oakley C (2001) The development of the Chemotherapy Symptom Assessment Scale (C-SAS): a scale for the routine clinical assessment of the symptom experiences of patients receiving cytotoxic chemotherapy. Int J Nurs Stud 38(5):497–510 Youngblood M, Williams PD, Eyles H, Waring J, Runyon S (1994) A comparison of two methods of assessing cancer therapy-related symptoms. Cancer Nurs 17(1):37–44 Peruselli C, Camporesi E, Colombo AM, Cucci M, Mazzoni G, Paci E (1993) Quality-of-life assessment in a home care program
for advanced cancer patients: a study using the Symptom Distress Scale. J Pain Symptom Manage 8(5):306–311 72. Bruera E, El Osta B, Valero V, Driver LC, Pei BL, Shen L et al (2007) Donepezil for cancer fatigue: a double-blind, randomized, placebo-controlled trial. J Clin Oncol 25(23):3475–3481 73. Bruera E, Valero V, Driver L, Shen L, Willey J, Zhang T et al (2006) Patient-controlled methylphenidate for cancer fatigue: a double-blind, randomized, placebo-controlled trial. J Clin Oncol 24(13):2073–2078 74. Ibbotson T, Maguire P, Selby P, Priestman T, Wallace L (1994) Screening for anxiety and depression in cancer patients: the effects of disease and treatment. Eur J Cancer 30A(1):37–40 75. Razavi D, Delvaux N, Farvacques C, Robaye E (1990) Screening for adjustment disorders and major depressive disorders in cancer in-patients. Br J Psychiatry 156:79–83 76. Lobchuk MM (2003) The memorial symptom assessment scale: modified for use in understanding family caregivers' perceptions of cancer patients' symptom experiences. J Pain Symptom Manage 26(1):644–654 77. Chang VT, Hwang SS, Kasimis B, Thaler HT (2004) Shorter symptom assessment instruments: the Condensed Memorial Symptom Assessment Scale (CMSAS). Cancer Invest 22(4): 526–536 78. Nelson JE, Meier DE, Oei EJ, Nierman DM, Senzel RS, Manfredi PL et al (2001) Self-reported symptom experience of critically ill cancer patients receiving intensive care. Crit Care Med 29(2):277– 282 79. Ewing G, Todd C, Rogers M, Barclay S, McCabe J, Martin A (2004) Validation of a symptom measure suitable for use among palliative care patients in the community: CAMPAS-R. J Pain Symptom Manage 27(4):287–299 80. Nunnally J, I. B (1994) Psychometric theory. Psychometric theory, McGraw-Hill, New York, p. 256 81. Chang VT, Hwang SS, Feuerman M (2000) Validation of the Edmonton symptom assessment scale. Cancer 88(9):2164–2171 82. Pautex S, Berger A, Chatelain C, Herrmann F, Zulian GB (2003) Symptom assessment in elderly cancer patients receiving palliative care. Crit Rev Oncol Hematol 47(3):281–286 83. Ewing G, Rogers M, Barclay S, McCabe J, Martin A, Campbell M et al (2006) Palliative care in primary care: a study to determine whether patients and professionals agree on symptoms. Br J Gen Pract 56(522):27–34 84. Hoekstra J, de Vos R, van Duijn NP, Schade E, Bindels PJ (2006) Using the symptom monitor in a randomized controlled trial: the effect on symptom prevalence and severity. J Pain Symptom Manage 31(1):22–30 85. Hoekstra J, Bindels PJ, van Duijn NP, Schade E (2004) The symptom monitor. A diary for monitoring physical symptoms for cancer patients in palliative care: feasibility, reliability and compliance. J Pain Symptom Manage 27(1):24–35 86. Richardson LA, Jones GW (2009) A review of the reliability and validity of the Edmonton Symptom Assessment System. Curr Oncol 16(1):55 87. Smith AB, Selby PJ, Velikova G, Stark D, Wright EP, Gould A et al (2002) Factor analysis of the Hospital Anxiety and Depression Scale from a large cancer population. Psychol Psychother 75(Pt 2):165–176 88. Moorey S, Greer S, Watson M, Gorman C, Rowden L, Tunmore R et al (1991) The factor structure and factor stability of the hospital anxiety and depression scale in patients with cancer. Br J Psychiatry 158:255–259 89. Vignaroli E, Pace EA, Willey J, Palmer JL, Zhang T, Bruera E (2006) The Edmonton Symptom Assessment System as a screening tool for depression and anxiety. J Palliat Med 9(2):296–303 90. Williams PD, Ducey KA, Sears AM, Williams AR, TobinRumelhart SE, Bunde P (2001) Treatment type and symptom
Support Care Cancer severity among oncology patients by self-report. Int J Nurs Stud 38(3):359–367 91. Baker F, Denniston M, Zabora J, Polland A, Dudley WN (2002) A POMS short form for cancer patients: psychometric and structural evaluation. Psychooncology 11(4):273–281 92. Reddy S, Bruera E, Pace E, Zhang K, Reyes-Gibby CC (2007) Clinically important improvement in the intensity of fatigue in patients with advanced cancer. J Palliat Med 10(5):1068–1075 93. Philip J, Smith WB, Craft P, Lickiss N (1998) Concurrent validity of the modified Edmonton Symptom Assessment System with the Rotterdam Symptom Checklist and the Brief Pain Inventory. Support Care Cancer 6(6):539–541 94. Lund I, Lundeberg T, Sandberg L, Budh CN, Kowalski J, Svensson E (2005) Lack of interchangeability between visual analogue and verbal rating pain scales: a cross sectional description of pain etiology groups. BMC Med Res Methodol 5:31 95. Huschka MM, Mandrekar SJ, Schaefer PL, Jett JR, Sloan JA (2007) A pooled analysis of quality of life measures and adverse events data in north central cancer treatment group lung cancer clinical trials. Cancer 109(4):787–795 96. Bland JM, Altman DG (1999) Measuring agreement in method comparison studies. Stat Methods Med Res 8(2):135–160 97. Schipper H, Clinch J, McMurray A, Levitt M (1984) Measuring the quality of life of cancer patients: the Functional Living IndexCancer: development and validation. J Clin Oncol 2(5):472–483 98. Aaronson NK, Ahmedzai S, Bergman B, Bullinger M, Cull A, Duez NJ et al (1993) The European Organization for Research and Treatment of Cancer QLQ-C30: a quality-of-life instrument for use in international clinical trials in oncology. J Natl Cancer Inst 85(5):365–376 99. Huskisson E (1983) Visual analogue scales. In: Melzack R (ed) Pain management and assessment. Raven, New York, pp 33–37 100. Hinds PS, Schum L, Srivastava DK (2002) Is clinical relevance sometimes lost in summative scores? West J Nurs Res 24(4):345– 353 101. Vickers AJ (2004) Statistical considerations for use of composite health-related quality-of-life scores in randomized trials. Qual Life Res 13(4):717–723 102. Watanabe S, Nekolaichuk C, Beaumont C, Mawani A (2009) The Edmonton symptom assessment system—what do patients think? Support Care Cancer 17(6):675–683
103. 104.
105.
106.
107.
108. 109.
110.
111.
112.
113.
114. 115.
Holmes S (1989) Use of a modified symptom distress scale in assessment of the cancer patient. Int J Nurs Stud 26(1):69–79 Sutherland HJ, Walker P, Till JE (1988) The development of a method for determining oncology patients' emotional distress using linear analogue scales. Cancer Nurs 11(5):303–308 Morrow GR, Chiarello RJ, Derogatis LR (1978) A new scale for assessing patients' psychosocial adjustment to medical illness. Psychol Med 8(4):605–610 Pud D (2014) The psychometric properties of the Hebrew version of the Memorial Symptom Assessment Scale (MSAS-Heb) in breast cancer patients. J Pain Symptom Manage Browall M, Kenne Sarenmalm E, Nasic S, Wengstrom Y, GastonJohansson F (2013) Validity and reliability of the Swedish version of the Memorial Symptom Assessment Scale (MSAS): an instrument for the evaluation of symptom prevalence, characteristics, and distress. J Pain Symptom Manage 46(1):131–141 McCorkle R, Young K (1978) Development of a symptom distress scale. Cancer Nurs 1(5):373–378 Chow E, Davis L, Holden L, Tsao M, Danjoux C (2005) Prospective assessment of patient-rated symptoms following whole brain radiotherapy for brain metastases. J Pain Symptom Manage 30(1):18–23 Chow E, Fan G, Hadi S, Filipczak L (2007) Symptom clusters in cancer patients with bone metastases. Support Care Cancer 15(9): 1035–1043 Stromgren AS, Groenvold M, Pedersen L, Olsen AK, Sjogren P (2002) Symptomatology of cancer patients in palliative care: content validation of self-assessment questionnaires against medical records. Eur J Cancer 38(6):788–794 Gabrilove JL, Perez EA, Tomita DK, Rossi G, Cleeland CS (2007) Assessing symptom burden using the M. D. Anderson symptom inventory in patients with chemotherapy-induced anemia: results of a multicenter, open-label study (SURPASS) of patients treated with darbepoetin-alpha at a dose of 200 microg every 2 weeks. Cancer 110(7):1629–1640 Yamagishi A, Morita T, Miyashita M, Kimura F (2009) Symptom prevalence and longitudinal follow-up in cancer outpatients receiving chemotherapy. J Pain Symptom Manage 37(5):823–830 Chen ML, Tseng HC (2006) Symptom clusters in cancer patients. Support Care Cancer 14(8):825–830 Chen ML, Lin CC (2007) Cancer symptom clusters: a validation study. J Pain Symptom Manage 34(6):590–599
REVIEW ARTICLE
The psychometric properties of cancer multisymptom assessment instruments: a clinical review Aynur Aktas 1 & Declan Walsh 1,2,3 & Jordanka Kirkova 1
Received: 27 January 2015 / Accepted: 29 March 2015 # Springer-Verlag Berlin Heidelberg 2015
Abstract Purpose Various instruments are used to assess both individual and multiple cancer symptoms. We evaluated the psychometric properties of cancer multisymptom assessment instruments. Methods An Ovid MEDLINE search was done. All searches were limited to adults and in English. All instruments published from 2005 to 2014 (and with at least one validity test) were included. We excluded those who only reported content validity. Instruments were categorized by the three major types of symptom measurement scales employed as follows: visual analogue (VAS), verbal rating (VRS), and numerical rating (NRS) scales. They were then examined in two areas: (1) psychometric thoroughness (number of tests) and (2) Presented at the 20th International Symposium of the Multinational Association of Supportive Care in Cancer (MASCC) Houston, Texas, USA, June 26–28, 2008 Electronic supplementary material The online version of this article (doi:10.1007/s00520-015-2732-7) contains supplementary material, which is available to authorized users. Harry R. Horvitz Center is a World Health Organization Demonstration Project in Palliative Medicine and an ESMO Designated Center of Integrated Oncology and Palliative Care. * Declan Walsh [email protected]
psychometric strength of evidence (validity, reliability, generalizability). We also assigned an empirical global psychometric quality score (which combined the concepts of thoroughness and strength of evidence) to rank the instruments. Results We analyzed 57 instruments (17 original, 40 modifications). They varied in types of scales used, symptom dimensions measured, and time frames evaluated. Of the 57, 10 used VAS, 28 VRS, and 19 NRS. The Edmonton Symptom Assessment System (ESAS), ESAS-Spanish, Hospital Anxiety and Depression Scale (HADS), Profile of Mood States (POMS), Symptom Distress Scale (SDS), M.D. Anderson Symptom Inventory (MDASI)-Russian, and MDASI-Taiwanese were the most comprehensively tested for validity and reliability. The ESAS, ESAS-Spanish, ASDS-2, Memorial Symptom Assessment Scale (MSAS)-SF, POMS, SDS, MDASI (and some translations), and MDASI-Heart Failure all showed good validity and reliability. Conclusions The MDASI appeared to be the best overall from a psychometric perspective. This was followed by the ESAS, ESAS-Spanish, POMS, SDS, and some MDASI translations. VRS-based instruments were most common. There was a wide range of psychometric rigor in validation. Consequently, meta-analysis was not possible. Most cancer multisymptom assessment instruments need further extensive validation to establish the excellent reliability and validity required for clinical utility and meaningful research. Keywords Instruments . Patient-reported outcomes . Psychometrics . Reliability . Validity
1
Section of Palliative Medicine and Supportive Oncology, Department of Solid Tumor Oncology, Cleveland Clinic, Taussig Cancer Institute, Cleveland, OH, USA
2
The Harry R. Horvitz Center Chair in Palliative Medicine and Supportive Oncology, Cleveland Clinic, Cleveland, OH, USA
Introduction
Faculty of Health Sciences, Trinity College Dublin, University College Dublin, Old Chemistry Building Extension Trinity College Dublin 2, Dublin, Ireland
Patient-reported outcomes (PRO) include subjective assessments of symptoms, function, health status, and health-
3
Support Care Cancer
related quality of life (HRQOL) [1]. PRO may improve evaluation of disease and treatment outcomes [2–4]. The Center for Medical Technology Policy recommended inclusion of PRO into adult oncology research [5]. To conduct rigorous PRO research, symptom data should ideally be gathered by valid, reliable, and responsive instruments. For symptom PRO to be successful in clinical practice, any instrument should meet adequate psychometric criteria but also have good generalizability. Several guidelines exist to help establish evidence of reliability and validity [6, 7]. Assessment burden and instrument responsiveness to change are also a challenge [8]. In a previous systematic review, we identified 44 cancer symptom assessment instruments [9]. We evaluated them for content, time frame, validity, reliability, and clinical utility. Heterogeneity was evident in design, symptom number and type, methodology, and assessment timeframe [9]. We noted there appeared to be no ideal instrument with excellent psychometric properties although did not address the issue in detail [9]. In this clinical review, we extended this work to specifically examine the psychometric properties of cancer multisymptom assessment instruments, in particular, those which employ patient self-reported rating scales and where symptom evaluation is their primary purpose.
from each study that met inclusion criteria. Data included patient and disease characteristics, study design (original or modifications), and clinical setting. We only evaluated instruments whose primary purpose is symptom assessment. Quality of life instruments were not included, although some contain symptom items. Instruments were excluded which reported: (1) No reliability or validity tests (2) Only content validity. Each author separately performed a detailed instrument assessment. All psychometric tests were reported as categorized by the original authors. For convenience, instruments were grouped by the three major types of symptom measurement scales they used as follow: visual analogue (VAS), verbal rating (VRS), and 11-point numerical rating scale (NRS). They were then examined in three ways [10]: (1) Thoroughness: the number of validity and reliability tests reported for each instrument (2) Strength of evidence: high instrument correlation scores for both good validity and reliability (3) Global quality score: psychometric value and also generalizability.
Methods Thoroughness Data sources and search strategy A MEDLINE search using Ovid MEDLINE (Ovid Technologies Inc., New York, NY) retrieved abstracts (2005–2014) with the following key words: (1) BSeverity of Illness Index^ or BKarnofsky Performance Status^ and Bany Neoplasms^ or BPalliative Care,^ (2) BSymptom^ within two words of Bassessment^ (in either order, in article title, abstract, or indexing) and Bany Neoplasms^ or BPalliative Care^ (3) BQuestionnaires^ and Bany Neoplasms^ or BPalliative Care^. All searches were limited to adults and in English. An electronic search of the Web of Science identified journal articles cited from our previous report [9] from 2006 to 2014. Finally, a reference hand search was done of the papers identified through MEDLINE. Duplicate citations were eliminated using EndNote Version X3 for Windows (Thomson Reuters, NY, USA), reference organization software. Data analysis Published recommendations [10–15] to evaluate instrument validity and reliability were reviewed and outlined by an interdisciplinary research team with both clinical expertise and scientific background. Two authors (J.K., A.A.) independently reviewed studies to identify eligible instruments. Final inclusion was by mutual agreement. Information was then abstracted
Thoroughness (Appendix 1) was ranked in four levels (none, poor, moderate, good). This was done solely by test number and variety, i.e., a larger number of published validity and reliability tests was deemed to reflect greater thoroughness. This was easier to evaluate than strength of evidence. For example, known-group (discriminant) validity was tested for some instruments between different antitumor treatments, and at various levels of Karnofsky performance status (KPS). In this case, we ranked both tests as Bpoor^ for thoroughness (BX;^ Appendix 1), because although they represented one type of validity, but in the same population. However, if consistently high correlations were obtained, then strength of evidence was rated as Bgood^ (BXXX;^ Appendix 1). Strength of evidence To rank strength of evidence (none, poor, moderate, good) consensus criteria were established (Appendix 2). For most, high (r≥0.7) correlation coefficient scores (0=worst and 1= best) were deemed to reflect good validity and reliability. Longitudinal design, sample size >100, and multiple confirmatory statistical methods gave additional weight to strength of evidence scores (Appendix 1). In validation studies by factor analysis, a sample size of >100 subjects is recommended [15]. In smaller validation study, we reduced the strength of evidence
Support Care Cancer
level accordingly [16]. We also gave lower strength of evidence scores to convenience samples (as non-consecutive patient enrollment might cause selection bias). We considered that psychometric tests performed in addition to Delphi techniques (expert opinion) improved instrument credibility, and scored them as good for content validity. Similarly, forward and backward translations of instrument modifications were considered to ensure better semantic and content equivalence. Validity was assessed separately: (1) Known-group validity (measured at a single point in time) (2) Response to change (repeated measures) (3) Instrument sensitivity and specificity [area under the curve (AUC)] [17]. Known-group validity and response to change needed at least one statistically significant test (p0.5 on the receiver operator characteristic curve (ROC). Outcome measurement also requires good response to change [10]. These characteristics assist objective analysis of symptom associations. Except for the HADS, instrument goals (symptom screen or response measurement) were never specified. The absence of any specific intervention in instrument validation studies complicates response differences [28]. Although different treatments for fatigue were tested in some clinical trials, single ESAS severity items [72, 73] did not improve. Despite its importance in outcomes research, predictive validity was tested infrequently [77, 79]. Given the subjective nature of symptoms and their variability, the most appropriate criterion in validation of an instrument modification should be the original instrument [77]. This also supported the need for multidimensional symptom assessment. Internal consistency of subscales (or factors from factor analysis) could be tested separately, because they might represent separate symptom constructs within the total instrument [56, 58]. Intra-rater reliability tests were less common than those for instrument consistency. This was possibly due to symptom variability over time. Intra-rater reliability was preferred by the WCQ [52] because each item in the multi-item instrument was viewed as an equal but separate measure of an underlying construct [12]. Inter-rater reliability is particularly important when instruments target the very ill [22, 24, 25, 83], and needs extensive testing. Agreement between instruments was rarely tested. Both good agreement (small mean differences) and moderate to high correlations (strong relationships and consistent item rank) may support better reliability [96]. Other challenges included the use of different time frames during instrument validation. For example, the Functional Living Index Cancer (FLIC) [97] which assesses pain and nausea during the Bpast 2 weeks,^ and depression Bat times, ^ was compared with the Memorial Symptom Assessment Scale (MSAS) which has a Bpast week^ time frame [49].
Similarly, the scales compared were never of the same type, and generally not interconverted to a similar format. All VASbased instruments were compared to VRS instruments. These included the MSAS [58], HADS [50], or the European Organization for Research and Treatment of Cancer Quality of Life Questionnaire Version 3 (EORTC-QLQ C-30) [98]. All VRS instruments were validated against VAS; all NRS-based instruments, except one [57], had been validated only against VAS instruments [99]. Symptom scores from a single-item scale were sometimes correlated to combined multi-item scores that allegedly measured the same construct [45]. For example, single anxiety and depression distress items were correlated with aggregated subscale scores for the same symptoms in the HADS [57]. Another was the MDASI-Japanese [45] interference scales correlated with single items from EORTC-QLQ C-30. The moderate correlations observed suggested the single-item scales actually measured different aspects of more complex constructs. Both single-item and composite scores may theoretically provide useful clinical information. Composite scores, e.g., the Distress Index of the MSAS [58], may also speed follow-up assessments. However, separate symptoms or symptom subscales might be more clinically relevant [87, 100]. The effect size of any intervention on each subscale, and the level of correlation between subscales may also determine the efficiency of composite scores [101]. Psychometrically sound instruments should also have wide generalizability. This requires large study sample sizes, so variability (measurement error) is reduced [10]. Overall generalizability was limited. Population heterogeneity was common, and likely influenced convergent, divergent, and factorial validity. For instance, the RSCL Black of energy^ and Bheadache^ items loaded on the psychological scale factor, when the sample was heterogeneous in disease stage and treatment. In contrast, it was on the physical scale during antitumor treatment [56]. Attrition and loss of data challenge generalizability in palliative medicine symptom research in particular [66, 82]. We excluded quality of life (QOL) instruments from the review. QOL is a broader construct. Some include symptoms, but that is not their primary purpose. Our focus was on symptom measures. While QOL may serve as a proxy measure for symptom burden, it does not suffice for clinical practice in which the focus is on physical symptom control. In this review, we only assessed the psychometric properties of cancer multisymptom instruments. We did not look at clinical utility, i.e., time to complete of individual instruments. The psychometric properties of an instrument are not the only determinant of its value in a clinical or research setting. Psychometric rigor must be accompanied by clinical utility. The global psychometric score was constructed to help us consolidate and rank the instruments for the purpose of this review. It is not intended to be used for future studies or development of new instruments.
Support Care Cancer
Heterogeneity in study design, symptom content, and the extent of psychometric validation affected the final global evaluation. We believe the choice of a valid and reliable instrument should be based upon instrument purpose, target population, psychometric thoroughness, and strength of evidence. The ability of a PRO measure to improve decision making in research relies on its psychometric strength to capture symptom burden [6]. Comprehensive instruments with good psychometric properties and generalizability are still needed.
References 1.
2.
3.
Conclusions Fifty-seven cancer multisymptom assessment instruments (17 original; 40 modifications) were identified in this review. Most used VRS to measure symptoms. Psychometric thoroughness for the ESAS, ESAS-Spanish, HADS, Profile of Mood States (POMS), SDS, MDASI-Russian, and MDASI-Taiwanese was good. The ESAS, ESASSpanish, MSAS-SF, POMS, SDS, some MDASI translations (Chinese, Greek, Taiwanese), and MDASI-Heart Failure all had good strength of evidence. Only the original MDASI had both very good validity and good reliability. Studies that examined the validity and reliability of original ESAS and ESAS-Spanish consistently revealed good thoroughness and strength of evidence. Individual instruments had particular advantages for specific populations, or certain clinical or research goals. There was a wide range of psychometric rigor in validation. As a result, meta-analysis was not possible. In most, important psychometric properties (Thoroughness and strength of evidence) of VAS, VRS, and NRS instruments ranged from moderate to good for validity. Similarly, reliability varied from poor to good. Most available cancer multisymptom assessment instruments need further extensive validation to establish the excellent reliability and validity required for clinical utility and meaningful research.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13. Acknowledgments This paper was completed in part while one of the authors (D.W.) was a Lord Harris Visiting Fellow at the Tseu Medical Institute, Manchester Harris College, University of Oxford, UK, May– June 2011. We are grateful to Gretchen Hallerberg for help with the review of the literature. We thank Matthew Karafa, PhD; Wael Lasheen, MD; and Curtis Tatsuoka, PhD for their comments on the manuscript and Ellen Schleckman and Aditya Nair for their administrative assistance. Funding This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors. Jordanka Kirkova was supported in part by a fellowship grant in End-of-Life Care from the Mt. Sinai Health Care Foundation. Conflict of interest The authors declare that there is no conflict of interest.
14.
15.
16. 17.
(FDA) FaDA. Food and Drug Administration (FDA) (2011) Guidance for industry. Patient-reported outcome measures: use in medical product development to support labeling claims. In: Drugs/GuidanceComplianceRegulatoryInformation/Guidances/ UCM193282.pdf. hwfgd, editor. Accessed 24 May 20132011 Sloan JA, Berk L, Roscoe J, Fisch MJ, Shaw EG, Wyatt G et al (2007) Integrating patient-reported outcomes into cancer symptom management clinical trials supported by the National Cancer Institute-sponsored clinical trials networks. J Clin Oncol 25(32): 5070–5077 Cleeland CS (2007) Symptom burden: multiple symptoms and their impact as patient-reported outcomes. J Natl Cancer Inst Monogr 37:16–21 Greenhalgh J, Meadows K (1999) The effectiveness of the use of patient-based measures of health in routine practice in improving the process and outcomes of patient care: a literature review. J Eval Clin Pract 5(4):401–416 Policy CfMT. Center for Medical Technology Policy. Recommendations for incorporating patient-reported outcomes (PROs) into clinical comparative effectiveness research (CER) in adult oncology. 2013. http://www.cmtp.net.org/wp-content/ uploads/downloads/2012/05/PRO-EGDpdf [Internet]. 2013 Frost MH, Reeve BB, Liepa AM, Stauffer JW, Hays RD (2007) What is sufficient evidence for the reliability and validity of patient-reported outcome measures? Value Health 10(Suppl 2): S94–S105 DeVon HA, Block ME, Moyle-Wright P, Ernst DM, Hayden SJ, Lazzara DJ et al (2007) A psychometric toolbox for testing validity and reliability. J Nurs Scholarsh 39(2):155–164 Swartz RJ, Schwartz C, Basch E, Cai L, Fairclough DL, McLeod L et al (2011) The king's foot of patient-reported outcomes: current practices and new developments for the measurement of change. Qual Life Res 20(8):1159–1167 Kirkova J, Davis MP, Walsh D, Tiernan E, O'Leary N, LeGrand SB et al (2006) Cancer symptom assessment instruments: a systematic review. J Clin Oncol 24(9):1459–1473 McDowel I (2006) Measuring health: a guide to rating scales and questionnaires. the theoretical and technical foundations of health measurement, 3rd edn. Oxford University Press, New York, pp 10–54 Lipscomb J, Snyder CF, Gotay CC (2007) Cancer outcomes measurement: through the lens of the Medical Outcomes Trust framework. Qual Life Res 16(1):143–164 Bowling A (2005) Theory of measurement. measuring health: a review of quality of life measurement scales, 3rd edn. Open University Press, Berkshire, pp 10–18 Streiner DL, GN (2003) Validity. Health measurement scales a practical guide to their development and use, New York, United States, Oxford University Press, pp. 172-93 Bourbonnais FF, Perreault A, Bouvette M (2004) Introduction of a pain and symptom assessment tool in the clinical setting–lessons learned. J Nurs Manag 12(3):194–200 de Vet HC, Ader HJ, Terwee CB, Pouwer F (2005) Are factor analytical techniques used appropriately in the validation of health status questionnaires? A systematic review on the quality of factor analysis of the SF-36. Qual Life Res 14(5):1203–1218, dicussion 19-21, 23-4 Mundfrom DSD, Ke TL (2005) Minimum sample size recommendations for conducting factor analysis. Int J Test 5:159–168 Le Pechoux C, Laplanche A, Faivre-Finn C, Ciuleanu T, Wanders R, Lerouge D et al (2011) Clinical neurological outcome and quality of life among patients with limited small-cell cancer treated with two different doses of prophylactic cranial irradiation in the
Support Care Cancer
18. 19.
20.
21.
22.
23.
24.
25.
26.
27.
28.
29.
30.
31.
32.
33.
34.
35.
intergroup phase III trial (PCI99-01, EORTC 22003-08004, RTOG 0212 and IFCT 99-01). Ann Oncol 22(5):1154–1163 Tucci RA, Bartels KL (1998) Patient use of the symptom reporting tool. Clin J Oncol Nurs 2(3):97–99 Kutner JS, Kassner CT, Nowels DE (2001) Symptom burden at the end of life: hospice providers' perceptions. J Pain Symptom Manage 21(6):473–480 Edmonds PM, Stuttaford JM, Penny J, Lynch AM, Chamberlain J (1998) Do hospital palliative care teams improve symptom control? Use of a modified STAS as an evaluation tool. Palliat Med 12(5):345–351 Kuebler KK, KO (1998) Psychometric evaluation of an objective assessment instrument to measure pain, dyspnea and restlessness. J Palliat Care Report from 12th International Congress on Care of the Terminally ill 14:125 McMillan SC, Moody LE (2003) Hospice patient and caregiver congruence in reporting patients' symptom intensity. Cancer Nurs 26(2):113–118 Given B, Given CW, McCorkle R, Kozachik S, Cimprich B, Rahbar MH et al (2002) Pain and fatigue management: results of a nursing randomized clinical trial. Oncol Nurs Forum 29(6):949– 956 Nekolaichuk CL, Maguire TO, Suarez-Almazor M, Rogers WT, Bruera E (1999) Assessing the reliability of patient, nurse, and family caregiver symptom ratings in hospitalized advanced cancer patients. J Clin Oncol 17(11):3621–3630 Nekolaichuk CL, Bruera E, Spachynski K, MacEachern T, Hanson J, Maguire TO (1999) A comparison of patient and proxy symptom assessments in advanced cancer patients. Palliat Med 13(4):311–323 Ivanova MO, Ionova TI, Kalyadina SA, Uspenskaya OS, Kishtovich AV, Guo H et al (2005) Cancer-related symptom assessment in Russia: validation and utility of the Russian M. D. Anderson Symptom Inventory. J Pain Symptom Manage 30(5): 443–453 Lin CC, Chang AP, Cleeland CS, Mendoza TR, Wang XS (2007) Taiwanese version of the M. D. Anderson symptom inventory: symptom assessment in cancer patients. J Pain Symptom Manage 33(2):180–188 Moro C, Brunelli C, Miccinesi G, Fallai M, Morino P, Piazza M et al (2006) Edmonton symptom assessment scale: Italian validation in two palliative care settings. Support Care Cancer 14(1):30– 37 Rhodes VA, McDaniel RW (1999) The index of nausea, vomiting, and Retching: a new format of the lndex of Nausea and Vomiting. Oncol Nurs Forum 26(5):889–894 Wang XS, Laudico AV, Guo H, Mendoza TR, Matsuda ML, Yosuico VD et al (2006) Filipino version of the M. D. Anderson Symptom Inventory: validation and multisymptom measurement in cancer patients. J Pain Symptom Manage 31(6):542–552 Yun YH, Mendoza TR, Kang IO, You CH, Roh JW, Lee CG et al (2006) Validation study of the Korean version of the M. D. Anderson Symptom Inventory. J Pain Symptom Manage 31(4): 345–352 Fu MR, Rhodes V, Xu B (2002) The Chinese translation of the index of nausea, vomiting, and retching. Cancer Nurs 25(2):134– 140 Baba K, Fransson P, Lindh J (2007) Use of a modified ESAS in cancer patients: a pilot study of patient and staff experiences. Int J Palliat Nurs 13(12):610–616 Fadol A, Mendoza T, Gning I, Kernicki J, Symes L, Cleeland CS et al (2008) Psychometric testing of the MDASI-HF: a symptom assessment instrument for patients with cancer and concurrent heart failure. J Card Fail 14(6):497–507 Armstrong TS, Gning I, Mendoza TR, Weinberg JS, Gilbert MR, Tortorice ML et al (2009) Clinical utility of the MDASI-BT in
36.
37.
38.
39.
40.
41.
42.
43.
44.
45.
46.
47.
48.
49.
50. 51.
52.
patients with brain metastases. J Pain Symptom Manage 37(3): 331–340 Claessens P, Menten J, Schotsmans P, Broeckaert B (2011) Development and validation of a modified version of the Edmonton Symptom Assessment Scale in a Flemish palliative care population. Am J Hosp Palliat Care 28(7):475–482 Chinda M, Jaturapatporn D, Kirshen AJ, Udomsubpayakul U (2011) Reliability and validity of a Thai version of the edmonton symptom assessment scale (ESAS-Thai). J Pain Symptom Manage 42(6):954–960 Carvajal A, Centeno C, Watson R, Bruera E (2011) A comprehensive study of psychometric properties of the Edmonton Symptom Assessment System (ESAS) in Spanish advanced cancer patients. Eur J Cancer 47(12):1863–1872 Guirimand F, Buyck J-F, Lauwers-Allot E, Revnik J, Kerguen T, Aegerter P et al (2010) Cancer-related symptom assessment in France: validation of the French M. D. Anderson Symptom Inventory. J Pain Symptom Manage 39(4):721–733 Yildirim Y, Tokem Y, Bozkurt N, Fadiloglu C, Uyar M, Uslu R (2011) Reliability and validity of the Turkish version of the Memorial Symptom Assessment Scale in cancer patients. Asian Pac J Cancer Prev 12(12):3389–3396 Badger TA, Segrin C, Meek P (2011) Development and validation of an instrument for rapidly assessing symptoms: the general symptom distress scale. J Pain Symptom Manage 41(3):535–548 Aoun SM, Monterosso L, Kristjanson LJ, McConigley R (2011) Measuring symptom distress in palliative care: psychometric properties of the Symptom Assessment Scale (SAS). J Palliat Med 14(3):315–321 Wang XS, Wang Y, Guo H, Mendoza TR, Hao XS, Cleeland CS (2004) Chinese version of the M. D. Anderson Symptom Inventory: validation and application of symptom measurement in cancer patients. Cancer 101(8):1890–1901 Mystakidou K, Cleeland C, Tsilika E, Katsouda E, Primikiri A, Parpa E et al (2004) Greek M.D. Anderson Symptom Inventory: validation and utility in cancer patients. Oncology 67(3-4):203– 210 Okuyama T, Wang XS, Akechi T, Mendoza TR, Hosaka T, Cleeland CS et al (2003) Japanese version of the MD Anderson Symptom Inventory: a validation study. J Pain Symptom Manage 26(6):1093–1104 Abu-Saad Huijer H, Sagherian K, Tamim H (2014) Validation of the Arabic version of the memorial symptom assessment scale among Lebanese cancer patients. J Pain Symptom Manage Kwon JH, Nam SH, Koh S, Hong YS, Lee KH, Shin SW et al (2013) Validation of the edmonton symptom assessment system in Korean patients with cancer. J Pain Symptom Manage 46(6):947– 956 Harding G, Cella D, Robinson D Jr, Mahadevia PJ, Clark J, Revicki DA (2007) Symptom burden among patients with renal cell carcinoma (RCC): content for a symptom index. Health Qual Life Outcomes 5:34 Hickman SE, Tilden VP, Tolle SW (2001) Family reports of dying patients' distress: the adaptation of a research tool to assess global symptom distress in the last week of life. J Pain Symptom Manage 22(1):565–574 Zigmond AS, Snaith RP (1983) The hospital anxiety and depression scale. Acta Psychiatr Scand 67(6):361–370 Morrow GR (1984) Methodology in behavioral and psychosocial cancer research. The assessment of nausea and vomiting. Past problems, current issues and suggestions for future research. Cancer 53(10 Suppl):2267–2280 Sitzia J, Dikken C, Hughes J (1997) Psychometric evaluation of a questionnaire to document side-effects of chemotherapy. J Adv Nurs 25(5):999–1007
Support Care Cancer 53.
54.
55.
56.
57.
58.
59.
60.
61.
62.
63.
64.
65.
66.
67.
68. 69.
70.
71.
Rhodes VA, Watson PM, Johnson MH (1984) Development of reliable and valid measures of nausea and vomiting. Cancer Nurs 7(1):33–41 Barresi MJ, Shadbolt B, Byrne D, Stuart-Harris R (2003) The development of the Canberra symptom scorecard: a tool to monitor the physical symptoms of patients with advanced tumours. BMC Cancer 3:32 Chang VT, Hwang SS, Feuerman M, Kasimis BS, Thaler HT (2000) The memorial symptom assessment scale short form (MSAS-SF). Cancer 89(5):1162–1171 de Haes JC, van Knippenberg FC, Neijt JP (1990) Measuring psychological and physical distress in cancer patients: structure and application of the Rotterdam Symptom Checklist. Br J Cancer 62(6):1034–1038 Munro AJ, Potter S (1996) A quantitative approach to the distress caused by symptoms in patients treated with radical radiotherapy. Br J Cancer 74(4):640–647 Portenoy RK, Thaler HT, Kornblith AB, Lepore JM, FriedlanderKlar H, Kiyasu E et al (1994) The Memorial Symptom Assessment Scale: an instrument for the evaluation of symptom prevalence, characteristics and distress. Eur J Cancer 30A(9): 1326–1336 Rhodes VA, McDaniel RW, Homan SS, Johnson M, Madsen R (2000) An instrument to measure symptom experience. Symptom occurrence and symptom distress. Cancer Nurs 23(1):49–54 Stein KD, Denniston M, Baker F, Dent M, Hann DM, Bushhouse S et al (2003) Validation of a modified Rotterdam Symptom Checklist for use with cancer patients in the United States. J Pain Symptom Manage 26(5):975–989 Watson M, ML, Maguire GP (1992) Further development of a quality of life measure for cancer patients: The Rotterdam Symptom Checklist (revised). Psychooncology 1:35-44 Cleeland CS, Mendoza TR, Wang XS, Chou C, Harle MT, Morrissey M et al (2000) Assessing symptom distress in cancer patients: the M.D. Anderson Symptom Inventory. Cancer 89(7): 1634–1646 Chow E, Wong R, Connolly R, Hruby G, Franzcr, Franssen E et al (2001) Prospective assessment of symptom palliation for patients attending a rapid response radiotherapy program. feasibility of telephone follow-up. J Pain Symptom Manage 22(2):649–656 McCorkle R, Quint-Benoliel J (1983) Symptom distress, current concerns and mood disturbance after diagnosis of life-threatening disease. Soc Sci Med 17(7):431–438 Holmes S (1991) Preliminary investigations of symptom distress in two cancer patient populations: evaluation of a measurement instrument. J Adv Nurs 16(4):439–446 Bruera E, MacDonal S (1993) Audit methods: the edmonton symptom assessment system. In: Higgenson I (ed) Clinical audit in palliative care, 1st edn. Radcliffe Medical Press, Oxford, pp 61– 77 Morrow GR (1992) A patient report measure for the quantification of chemotherapy induced nausea and emesis: psychometric properties of the Morrow assessment of nausea and emesis (MANE). Br J Cancer Suppl 19:S72–S74 Slocumb EM, Cole FL (1991) A practical approach to content validation. Appl Nurs Res 4(4):192–195 Brown V, Sitzia J, Richardson A, Hughes J, Hannon H, Oakley C (2001) The development of the Chemotherapy Symptom Assessment Scale (C-SAS): a scale for the routine clinical assessment of the symptom experiences of patients receiving cytotoxic chemotherapy. Int J Nurs Stud 38(5):497–510 Youngblood M, Williams PD, Eyles H, Waring J, Runyon S (1994) A comparison of two methods of assessing cancer therapy-related symptoms. Cancer Nurs 17(1):37–44 Peruselli C, Camporesi E, Colombo AM, Cucci M, Mazzoni G, Paci E (1993) Quality-of-life assessment in a home care program
for advanced cancer patients: a study using the Symptom Distress Scale. J Pain Symptom Manage 8(5):306–311 72. Bruera E, El Osta B, Valero V, Driver LC, Pei BL, Shen L et al (2007) Donepezil for cancer fatigue: a double-blind, randomized, placebo-controlled trial. J Clin Oncol 25(23):3475–3481 73. Bruera E, Valero V, Driver L, Shen L, Willey J, Zhang T et al (2006) Patient-controlled methylphenidate for cancer fatigue: a double-blind, randomized, placebo-controlled trial. J Clin Oncol 24(13):2073–2078 74. Ibbotson T, Maguire P, Selby P, Priestman T, Wallace L (1994) Screening for anxiety and depression in cancer patients: the effects of disease and treatment. Eur J Cancer 30A(1):37–40 75. Razavi D, Delvaux N, Farvacques C, Robaye E (1990) Screening for adjustment disorders and major depressive disorders in cancer in-patients. Br J Psychiatry 156:79–83 76. Lobchuk MM (2003) The memorial symptom assessment scale: modified for use in understanding family caregivers' perceptions of cancer patients' symptom experiences. J Pain Symptom Manage 26(1):644–654 77. Chang VT, Hwang SS, Kasimis B, Thaler HT (2004) Shorter symptom assessment instruments: the Condensed Memorial Symptom Assessment Scale (CMSAS). Cancer Invest 22(4): 526–536 78. Nelson JE, Meier DE, Oei EJ, Nierman DM, Senzel RS, Manfredi PL et al (2001) Self-reported symptom experience of critically ill cancer patients receiving intensive care. Crit Care Med 29(2):277– 282 79. Ewing G, Todd C, Rogers M, Barclay S, McCabe J, Martin A (2004) Validation of a symptom measure suitable for use among palliative care patients in the community: CAMPAS-R. J Pain Symptom Manage 27(4):287–299 80. Nunnally J, I. B (1994) Psychometric theory. Psychometric theory, McGraw-Hill, New York, p. 256 81. Chang VT, Hwang SS, Feuerman M (2000) Validation of the Edmonton symptom assessment scale. Cancer 88(9):2164–2171 82. Pautex S, Berger A, Chatelain C, Herrmann F, Zulian GB (2003) Symptom assessment in elderly cancer patients receiving palliative care. Crit Rev Oncol Hematol 47(3):281–286 83. Ewing G, Rogers M, Barclay S, McCabe J, Martin A, Campbell M et al (2006) Palliative care in primary care: a study to determine whether patients and professionals agree on symptoms. Br J Gen Pract 56(522):27–34 84. Hoekstra J, de Vos R, van Duijn NP, Schade E, Bindels PJ (2006) Using the symptom monitor in a randomized controlled trial: the effect on symptom prevalence and severity. J Pain Symptom Manage 31(1):22–30 85. Hoekstra J, Bindels PJ, van Duijn NP, Schade E (2004) The symptom monitor. A diary for monitoring physical symptoms for cancer patients in palliative care: feasibility, reliability and compliance. J Pain Symptom Manage 27(1):24–35 86. Richardson LA, Jones GW (2009) A review of the reliability and validity of the Edmonton Symptom Assessment System. Curr Oncol 16(1):55 87. Smith AB, Selby PJ, Velikova G, Stark D, Wright EP, Gould A et al (2002) Factor analysis of the Hospital Anxiety and Depression Scale from a large cancer population. Psychol Psychother 75(Pt 2):165–176 88. Moorey S, Greer S, Watson M, Gorman C, Rowden L, Tunmore R et al (1991) The factor structure and factor stability of the hospital anxiety and depression scale in patients with cancer. Br J Psychiatry 158:255–259 89. Vignaroli E, Pace EA, Willey J, Palmer JL, Zhang T, Bruera E (2006) The Edmonton Symptom Assessment System as a screening tool for depression and anxiety. J Palliat Med 9(2):296–303 90. Williams PD, Ducey KA, Sears AM, Williams AR, TobinRumelhart SE, Bunde P (2001) Treatment type and symptom
Support Care Cancer severity among oncology patients by self-report. Int J Nurs Stud 38(3):359–367 91. Baker F, Denniston M, Zabora J, Polland A, Dudley WN (2002) A POMS short form for cancer patients: psychometric and structural evaluation. Psychooncology 11(4):273–281 92. Reddy S, Bruera E, Pace E, Zhang K, Reyes-Gibby CC (2007) Clinically important improvement in the intensity of fatigue in patients with advanced cancer. J Palliat Med 10(5):1068–1075 93. Philip J, Smith WB, Craft P, Lickiss N (1998) Concurrent validity of the modified Edmonton Symptom Assessment System with the Rotterdam Symptom Checklist and the Brief Pain Inventory. Support Care Cancer 6(6):539–541 94. Lund I, Lundeberg T, Sandberg L, Budh CN, Kowalski J, Svensson E (2005) Lack of interchangeability between visual analogue and verbal rating pain scales: a cross sectional description of pain etiology groups. BMC Med Res Methodol 5:31 95. Huschka MM, Mandrekar SJ, Schaefer PL, Jett JR, Sloan JA (2007) A pooled analysis of quality of life measures and adverse events data in north central cancer treatment group lung cancer clinical trials. Cancer 109(4):787–795 96. Bland JM, Altman DG (1999) Measuring agreement in method comparison studies. Stat Methods Med Res 8(2):135–160 97. Schipper H, Clinch J, McMurray A, Levitt M (1984) Measuring the quality of life of cancer patients: the Functional Living IndexCancer: development and validation. J Clin Oncol 2(5):472–483 98. Aaronson NK, Ahmedzai S, Bergman B, Bullinger M, Cull A, Duez NJ et al (1993) The European Organization for Research and Treatment of Cancer QLQ-C30: a quality-of-life instrument for use in international clinical trials in oncology. J Natl Cancer Inst 85(5):365–376 99. Huskisson E (1983) Visual analogue scales. In: Melzack R (ed) Pain management and assessment. Raven, New York, pp 33–37 100. Hinds PS, Schum L, Srivastava DK (2002) Is clinical relevance sometimes lost in summative scores? West J Nurs Res 24(4):345– 353 101. Vickers AJ (2004) Statistical considerations for use of composite health-related quality-of-life scores in randomized trials. Qual Life Res 13(4):717–723 102. Watanabe S, Nekolaichuk C, Beaumont C, Mawani A (2009) The Edmonton symptom assessment system—what do patients think? Support Care Cancer 17(6):675–683
103. 104.
105.
106.
107.
108. 109.
110.
111.
112.
113.
114. 115.
Holmes S (1989) Use of a modified symptom distress scale in assessment of the cancer patient. Int J Nurs Stud 26(1):69–79 Sutherland HJ, Walker P, Till JE (1988) The development of a method for determining oncology patients' emotional distress using linear analogue scales. Cancer Nurs 11(5):303–308 Morrow GR, Chiarello RJ, Derogatis LR (1978) A new scale for assessing patients' psychosocial adjustment to medical illness. Psychol Med 8(4):605–610 Pud D (2014) The psychometric properties of the Hebrew version of the Memorial Symptom Assessment Scale (MSAS-Heb) in breast cancer patients. J Pain Symptom Manage Browall M, Kenne Sarenmalm E, Nasic S, Wengstrom Y, GastonJohansson F (2013) Validity and reliability of the Swedish version of the Memorial Symptom Assessment Scale (MSAS): an instrument for the evaluation of symptom prevalence, characteristics, and distress. J Pain Symptom Manage 46(1):131–141 McCorkle R, Young K (1978) Development of a symptom distress scale. Cancer Nurs 1(5):373–378 Chow E, Davis L, Holden L, Tsao M, Danjoux C (2005) Prospective assessment of patient-rated symptoms following whole brain radiotherapy for brain metastases. J Pain Symptom Manage 30(1):18–23 Chow E, Fan G, Hadi S, Filipczak L (2007) Symptom clusters in cancer patients with bone metastases. Support Care Cancer 15(9): 1035–1043 Stromgren AS, Groenvold M, Pedersen L, Olsen AK, Sjogren P (2002) Symptomatology of cancer patients in palliative care: content validation of self-assessment questionnaires against medical records. Eur J Cancer 38(6):788–794 Gabrilove JL, Perez EA, Tomita DK, Rossi G, Cleeland CS (2007) Assessing symptom burden using the M. D. Anderson symptom inventory in patients with chemotherapy-induced anemia: results of a multicenter, open-label study (SURPASS) of patients treated with darbepoetin-alpha at a dose of 200 microg every 2 weeks. Cancer 110(7):1629–1640 Yamagishi A, Morita T, Miyashita M, Kimura F (2009) Symptom prevalence and longitudinal follow-up in cancer outpatients receiving chemotherapy. J Pain Symptom Manage 37(5):823–830 Chen ML, Tseng HC (2006) Symptom clusters in cancer patients. Support Care Cancer 14(8):825–830 Chen ML, Lin CC (2007) Cancer symptom clusters: a validation study. J Pain Symptom Manage 34(6):590–599
