CRE0010.1177/0269215515579286Clinical RehabilitationBouffard et al.
Psychometric properties of the Musculoskeletal Function Assessment and the Short Musculoskeletal Function Assessment: A systematic review
Clinical Rehabilitation 1–17 © The Author(s) 2015 Reprints and permissions: sagepub.co.uk/journalsPermissions.nav DOI: 10.1177/0269215515579286 cre.sagepub.com
Jason Bouffard1,2, Michaël Bertrand-Charette1,2 and Jean-Sébastien Roy1,2
Abstract Objectives: To investigate the psychometric properties of the Musculoskeletal Function Assessment (MFA) and Short Musculoskeletal Function Assessment (SMFA). Data sources: A systematic search of the following databases was undertaken concerning psychometric evidence of the MFA and SMFA: PubMed, Embase, Scopus and Cinahl. References of retrieved articles were inspected for additional data. Review method: Articles evaluating the validity, reliability or responsiveness of the MFA or SMFA in patients with musculoskeletal disorders were included in this systematic review. The methodological quality of included articles was critically appraised and the psychometric data were extracted using standardized forms. An established set of criteria were used to synthetize the evidence in order to highlight the strengths and weaknesses of included questionnaires and the gaps in the literature. Results: Nine articles on MFA and 24 articles on SMFA met the inclusion criteria. The SMFA fulfilled 75% of the psychometric criteria analyzed, while the MFA fulfilled only 50%. MFA and SMFA have excellent content validity and relative reliability (weighted average intraclass correlation coefficient ⩾ 0.87), and are moderately to highly responsive (standardized response mean between 0.65 and 1.13). Absolute reliability and clinically important difference of both questionnaires need to be defined, while the construct validity of MFA still needs to be established. Conclusion: MFA and SMFA are reliable and responsive tools for monitoring the function of patients with various musculoskeletal disorders. Still, research is needed to justify their usage in a clinical setting. Keywords Systematic review, patient-centred outcome measure, psychometry, musculoskeletal disorders, musculoskeletal function assessment Received: 8 August 2014; accepted: 7 March 2015 1Université
Laval, Rehabilitation Department, Quebec City, QC, Canada 2Center for Interdisciplinary Research on Rehabilitation and Social Integration (CIRRIS), Quebec City, QC, Canada
Corresponding author: Jean-Sébastien Roy, Université Laval, Rehabilitation Department, 525, Boul. Wilfrid-Hamel, office H-1710, Quebec City, G1M 2S8, Canada. Email: [email protected]
Introduction The Global Burden of Disease 2010 study showed that musculoskeletal disorders are the largest contributor of years lived with disability in the United States and that their burden have increased in the last 20 years.1 Optimal care for this clinical population starts with a holistic evaluation of the patient’s condition. The usage of patient-reported outcome measures (PROs) is important for this evaluation as it allows the assessment of the patients’ symptoms and functional limitation in their daily life as well as their perception about their condition in a standardized way. A great number of PROs have been developed in the last two decades, particularly in the field of musculoskeletal disorders. Some PROs have been developed for specific conditions, while other PROs can be used with heterogenic populations.2,3 General questionnaires enable the comparison of the functional limitations of different clinical populations, while condition specific questionnaires are believed to be more responsive to change.4 The Musculoskeletal Function Assessment (MFA) and its abbreviate adaptation, the Short Musculoskeletal Function Assessment (SMFA), have been developed to be used with populations presenting a variety of musculoskeletal disorders.5,6 Since their development, these questionnaires have been used in multiple studies as the primary outcome to document functional limitations in populations with various musculoskeletal disorders.7 In fact, the American Academy of Orthopedic Surgeons recommends the SMFA for documenting the outcome of interventions in people with musculoskeletal disorders.8 Since these questionnaires are used to orientate treatment decision-making, researchers and clinicians must make sure that they present adequate psychometric properties (validity, reliability and responsiveness) in order to justify their use. One good way to establish if a given questionnaire possesses these psychometric properties is to systematically and critically review the literature. By performing such review, it is possible to highlight the advantages and drawbacks of the PROs under study, as well as the gaps in the literature. MFA or SMFA have been considered in several narrative or systematic reviews concerning its
Clinical Rehabilitation usage with different clinical populations.7,9–14 These reviews were mainly aimed at reporting on the different PROs available for the evaluation of patient’s with musculoskeletal disorders without systematically analyzing their psychometric properties.9,11,12 Thus, no systematic review has ever been published concerning the psychometric properties of the MFA and SMFA for populations with musculoskeletal disorders. Therefore, the aim of the present study is to conduct a systematic review of the quality and content of the psychometric evidence relating to the MFA and SMFA.
MFA and SMFA The MFA is a self-reported questionnaire designed to evaluate the perception of patients with various musculoskeletal disorders about their function.5 It contains 100 yes/no questions divided in 10 categories: self-care, sleep and rest, hand/fine motor, mobility, housework, employment/work, leisure/ recreation, family relationship, cognition/thinking and emotional adjustment coping/adaptation. Each question responded positively get a score of 1 while those responded negatively get a score of 0, with the exception of thirteen reverse questions. MFA has been developed using a rigorous process involving the review of existing questionnaires, open-ended interviews with clinicians working in the field of musculoskeletal disorders and with patients with a variety of musculoskeletal disorders, pre-testing of the questionnaire and revision by experts.5 The SMFA is the abbreviate version of MFA.6 It has been adapted from the original questionnaire in order to render its usage more practical in a clinical setting. It contains a 34-item dysfunction index and a 12-item bother index. The questions of the dysfunction index are grouped in 4 categories: daily activities, emotional status, function of the arm and hand and mobility. The questions of the bother index refer to the affective feelings patients have toward their dysfunctions in different functional areas such as housework or leisure and recreation, and each question begins by asking “How much are you bothered by …”. The authors selected items that were judged clinically and conceptually
Bouffard et al. important, stable and moderately endorsed from the original version of the MFA. All questions of the SMFA are scored using a 5 points Likert scale. The MFA total score and SMFA bother and dysfunction indexes give total scores on 100. Higher scores refer to greater disability. The questionnaires and more information about the scoring process are freely available on the website of the Orthopaedic Surgery Department of University of Minnesota (http://www.ortho.umn.edu/research/mfa-smfaresources/index.htm).15
appraisal tool.17 This tool has an excellent interrater reliability.18–20 After the independent evaluation of all included articles, raters met to discuss their scores. Each specific item on the quality appraisal tool was discussed to reach consensus. If consensus was not reached, a third evaluator (JSR) was involved in the discussion to solve the disagreement. An intraclass correlation coefficient (ICC) was calculated to evaluate pre-consensus inter-rater reliability of the total score on the critical appraisal tool.
Literature search and study selection
Data extraction and analysis
An electronic literature search on Pubmed, Embase, Scopus and Cinahl databases was conducted to find articles concerning the psychometric properties of the MFA or SMFA. The sensitive search filter suggested by Terwee et al.16 when conducting a systematic review on psychometric properties of health measures was used in association with the expressions “musculoskeletal function assessment” OR “musculoskeletal functional assessment”. All databases were searched from their date of inception to March 2015. Reference lists of all retrieved studies were searched for further relevant studies. Two evaluators reviewed the title and abstract of each article to determine eligibility. Pair of raters then independently reviewed each article to determine whether it met one of the following inclusion criteria: 1- original research designed to evaluate the psychometric properties of the dysfunction or bother indexes of SMFA or the total score of MFA; 2) original article designed to evaluate the construct validity of another health measure using the dysfunction or bother indexes of SMFA or the total score of MFA as a comparator. Articles were excluded if they were not written in English or French, they presented literature reviews or they did not include subjects with musculoskeletal disorders.
Two raters extracted data from each article using a standardized data collection form. Descriptive information of the population such as age, sex and diagnosis was collected. Data on content validity (floor-ceiling effect, missing values), construct validity (convergent-divergent, known group and structural validity), reliability (relative and absolute reliability, minimal detectable change [MDC], internal consistency), responsiveness (standardized response mean [SRM], effect-size [ES], clinically important difference [CID]) and readability/administration burden were extracted. Data concerning each of the psychometric quality cited above were then synthesized for the total score of MFA, SMFA dysfunction index and SMFA bother index. Weighted averages were calculated for reliability indices (ICC) and for correlation coefficients used for evaluating convergent-divergent validity. When the same data were analyzed in several studies included in the review, the first published study was considered for the weighted average. Considering the important heterogeneity of the studies evaluating responsiveness, no weighted average were performed for this psychometric quality. When the responsiveness of the questionnaire was evaluated at multiple follow-up intervals for one group of subjects, only the interval showing the best responsiveness was considered. When the construct validity (convergent, divergent, known group) was analyzed at different follow-up intervals in a study, the data collected in the most acute phase were included in the review. Data from the different translations/cultural adaptations were pooled together.
Critical appraisal All studies included were independently appraised for their methodological qualities by two of the authors (JB and MBC) using a structured critical
4 Some frequently utilized benchmarks were used to characterize the construct validity, relative reliability and responsiveness of the questionnaires. Pearson or Spearman correlations were categorized as high ⩾ 0.70, moderate between 0.50 and 0.70, and low between 0.26 and 0.50. For the relative reliability, ICC ⩾ 0.7 are judged as adequate for group comparisons while ICC ⩾ 0.9 are adequate to monitor single subjects.21 For the responsiveness, ES and SRM were considered large ⩾ 0.8, moderate between 0.50 and 0.80, and small between 0.20 and 0.50.22 In order to summarize the results of the systematic review, the summary table developed by Bot and collaborators was used.23 This table allows to evaluate and compare the psychometric qualities of the included questionnaires based on 12 specific criteria: time to administer, ease of scoring, readability and comprehension, content validity, internal consistency, construct validity, floor/ceiling effect, reliability, agreement, responsiveness, interpretability, CID.
Results After removal of duplicates, a total of 242 titles and abstracts were reviewed (Figure 1). Fourty-two articles were fully reviewed and 33 (MFA: n=9, SMFA: n=24) met the inclusion criteria and were included. The results of the critical appraisal process are presented in Supplementary Table 1. The mean methodological score of the reviewed articles was 70 ± 13%. Overall pre-consensus inter-rater reliability for the critical appraisal tool was good (ICC=0.73 95% confidence interval [0.51-0.86]).
Content validity Floor or ceiling effects are observed when 15% or more of the participants achieve the highest or the lowest possible scores, respectively.24 According to this criterion, MFA and both indexes of SMFA do not have any significant floor or ceiling effects when used to evaluate the current function of people with actual disability.5,6,25–35 As expected, some ceiling effect can however be observed when the SMFA is used in time points when participants do
Clinical Rehabilitation not have much disability such as before or a long time after an injury.29–31,36 The number of missing items or incomplete questionnaires in the included studies appears adequately low for the MFA and SMFA.6,27–29,34 However, it is worth noting that a relatively high proportion of participants (3% to 25%) omit answering to the question related to sexual activities in the SMFA.6,28,29 This problematic had also been reported in MFA’s development study.5 Agel and collaborators evaluated the clinical usefulness of SMFA.37 They showed that both patients and physicians perceived that SMFA has no to moderate effects on verbal interactions between them. Moreover, physicians found that the results of SMFA have no to some effects on their clinical decisions. Problems were experienced when completing or scoring the questionnaire in less than 5% of the participants. It was observed that 88% of the patients found that it was worthwhile completing the SMFA, and 94% would complete it again.37 More than 75% of the participants in Pinsker’s study reported that SMFA’s dysfunction index had an adequate length and “asked enough” to highlight the variety of difficulties experienced by people with ankle arthritis.35 A study highlighted some difficulties with the usage of the MFA in older population. The MFA was found to be too long, to contain questions difficult to understand or too personal such as questions on sexual activities.38 In summary, MFA’s total score and SMFA’s both indexes do not have any significant floor or ceiling effects and have an adequate proportion of missing items supporting their content validity. Moreover, patients and clinicians perceive the SMFA positively. Some issues have however been found with the use of MFA with older people.
Construct validity Convergent/divergent validity. The convergent-divergent validity of SMFA has been evaluated with a large variety of validated measures.6,28,30,32,34,39–45,67 As expected, the SMFA’s dysfunction index correlates highly with 14 out of 17 scales or subscales of validated measures evaluating solely patients’
Bouffard et al.
'musculoskeletal function assessment' OR ‘musculoskeletal functional assessment’ AND Expressions refering to psychometric properties (Terwee et al.)
• • • •
Hand searching: n=1
Pubmed : n= 196 Embase : n=205 Cinahl : n= 131 Scopus : n=190 Rejected after screening the title and abstract : 242 titles and abstracts screened after suppression of the duplicates
• • •
Not meeting the inclusion criteria : n=176 Review : n=12 Language : n=12
Rejected after reading the full text : • • 42 full-text assessed for eligibility •
Abstract only : n=2 No patients with musculoskeletal disorder : n=6 Not evaluating the MFA total score, or the SMFA dysfonction or bother index : n=1 ;
33 articles included in the study
Figure 1. Flow diagram of included articles.
perceived ability to perform functional tasks. SMFA’s bother index correlates highly with seven out of fifteen of such measures and moderately with five of them.6,28,34,35,39–43 The convergentdivergent validity of MFA’s total score has been less evaluated.25,26,46–48 MFA’s total score correlates lowly (r=0.4) with the physical function subscale of SF-36 and moderately with its physical role subscale even though those SF-36 subscales
are supposed to evaluate constructs closely related to MFA.26 MFA’s total score and both indexes of SMFA showed low or no correlations with clinical measures (for instance, strength, range of motion or walking speed).6,26,28,39,41,48 Both SMFA indexes showed correlations ranging from 0.42 to 0.81 with different pain subscales of validated questionnai res. 6,28,34,35,39–42 In summary, the convergentdivergent validity of both indexes of the SMFA is
Table 1. Subgroups successfully differentiate by the MFA or SMFA. Hypothesis
Mean subgroup 1
Mean subgroup 2
Rheumatoid arthritis (RA) > Traumatic injuries
RA: MFA: 45.15; 35.146
Presence of comorbidities Change in handedness
1 or more: MFA: 32.246 Yes: MFA: 40.046 Yes: MFA: 36.846 ISS ⩾ 10: MFA: 35.346
Lower extremity injury: MFA: 28.45; 28.246 Upper extremity injury: MFA: 27.45 None: MFA: 28.346 No: MFA: 28.046 No: MFA: 27.546 ISS = 9: MFA: 27.146 ISS = 1-8: MFA: 25.946 >High school: MFA: 27.3446 SMFA-D: 24.9;6 SMFA-B : 28.46
Presence of complications Injury severity scale score Education
Health habits/smoking Health insurance Disability compensation Income
Change in income Legal action Injured at work Current employment
Good day vs. bad day
⩽8th grade: MFA: 44.046 SMFA-D: 30.7;6 SMFA-B : 41.76 > 8th grade & ⩽ High school: MFA: 33.0546 SMFA-D: 31.7;6 SMFA-B : 35.96 Smoker: MFA: 32.646 Public/no insurance: MFA: 32.646 SMFA-D: 33.56; SMFA-B: 38.16 Yes: MFA: 37.2 46 SMFA-D: 41.26; SMFA-B: 47.36 ⩽ 20 000$: MFA: 31.946 SMFA-D: 34.36; SMFA-B: 40.96 Decrease: MFA: 36.346 Yes: MFA: 37.846 SMFA-D: 38.76; SMFA-B: 47.46 Yes: MFA: 32.046 Not working: MFA: 34.246 Not working (health related): SMFA-D: 40.06; SMFA-B : 46.16 Average day: MFA: 38.046 Bad day: MFA: 38.646
Nonsmoker: MFA: 28.346 Private: MFA: 26.746 SMFA-D: 22.76; SMFA-B: 26.76 No: MFA: 26.9346 SMFA-D: 23.66; SMFA-B: 27.06 ⩾ 50 000$: MFA: 25.9746 SMFA-D: 22.36; SMFA-B: 25.16 >20 000$ & 30°: SMFA-D: 3929; SMFA-B: 3929 10-30°: SMFA-D: 3229; SMFA-B: 3229 Grade 1: Extreme difficulty to walk, inability to walk > 1 m/s SMFA: 39.8050 Grade 2: Normal or antalgic walking < 1m/sec & obvious antalgia when walking > 1 m/sec SMFA: 16.46 50 OTA 44C: SMFA-D: 21.734 OTA 44B: SMFA-D: 15.734
Good: SMFA-D: 30.36; SMFA-B: 33.46 Excellent / very good: SMFA-D: 20.26; SMFA-B: 23.06 No to moderate pain: SMFA-D: 40.836; SMFA-B: 28.136 Yes: SMFA-D: 39.936; SMFA-B: 28.136