Accepted Manuscript Environmental Factors Item Development for Persons with Stroke, Traumatic Brain Injury and Spinal Cord Injury Allen W. Heinemann, Susan Magasi, Joy Hammel, Noelle E. Carlozzi, Sofia F. Garcia, Elizabeth A. Hahn, Jin-Shei Lai, David Tulsky, David B. Gray, Holly Hollingsworth, Sara Jerousek PII:
S0003-9993(13)01308-7
DOI:
10.1016/j.apmr.2013.11.024
Reference:
YAPMR 55692
To appear in:
ARCHIVES OF PHYSICAL MEDICINE AND REHABILITATION
Received Date: 1 August 2013 Revised Date:
30 October 2013
Accepted Date: 10 November 2013
Please cite this article as: Heinemann AW, Magasi S, Hammel J, Carlozzi NE, Garcia SF, Hahn EA, Lai J-S, Tulsky D, Gray DB, Hollingsworth H, Jerousek S, Environmental Factors Item Development for Persons with Stroke, Traumatic Brain Injury and Spinal Cord Injury, ARCHIVES OF PHYSICAL MEDICINE AND REHABILITATION (2014), doi: 10.1016/j.apmr.2013.11.024. This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
ACCEPTED MANUSCRIPT
Environmental Factors Item Development for Persons with Stroke, Traumatic Brain
2
Injury and Spinal Cord Injury
3
Allen W. Heinemann1
4
Susan Magasi2
5
Joy Hammel3
6
Noelle E. Carlozzi4
7
Sofia F. Garcia5
8
Elizabeth A. Hahn6
9
Jin-Shei Lai7
10
David Tulsky8
11
David B. Gray9
12
Holly Hollingsworth10
13
Sara Jerousek11
TE D
M AN U
SC
RI PT
1
14
Running head: Environmental Factors Item Development
15 16 17 18
1
19 20
2
21 22 23
3
24 25 26 27
4
28 29 30 31
5
EP
Allen Heinemann, PhD, Director, Center for Rehabilitation Outcomes Research, Rehabilitation Institute of Chicago and Professor, Department of Physical Medicine and Rehabilitation, Feinberg School of Medicine, Northwestern University, 345 E. Superior St. Chicago, IL 60611, 312-238-2802, [email protected].
AC C
Susan Magasi, PhD, Assistant Professor, University of Illinois at Chicago, Occupational Therapy, 1919 W. Taylor St., Room 327, Chicago, IL 60613, 312-996-4306 Joy Hammel, PhD, OTR/L, Professor: University of Illinois at Chicago, Depts. of Occupational Therapy and Disability & Human Development 1919 W. Taylor St., Room 311, Chicago, IL 60612; [email protected]; 312-996-3513 Noelle E. Carlozzi, PhD, University of Michigan, Rehabilitation Outcomes and Assessment Research, Department of Physical Medicine & Rehabilitation, Assistant Professor, North Campus Research Complex, 2800 Plymouth Road, Building NCRC B520, Room 3211, Ann Arbor, MI 48109-2800, 734-763-8917 [email protected] Sofia F. Garcia, PhD, Assistant Professor, Department of Medical Social Sciences, Department of Psychiatry and Behavioral Sciences, Feinberg School of Medicine, Northwestern University, 625 N. Michigan Ave., Suite 2700, Chicago, IL 60611, 312-503-3449, [email protected]
ACCEPTED MANUSCRIPT RRTC on Improving Measurement of Medical Rehabilitation Outcomes
R3. Environmental Factor Measures
32 33
6
34 35 36
7
37 38 39 40
8
41 42
9
43 44
10
45 46
11
47 48 49 50 51 52 53 54 55 56 57 58
National Institute on Disability and Rehabilitation Research provided funding through a Rehabilitation Research and Training Center on Improving Measurement of Medical Rehabilitation Outcomes grant (H133B090024) awarded to the Rehabilitation Institute of Chicago. A portion of Dr. Garcia’s time toward this publication was supported by the National Center for Complementary and Alternative Medicine and the National Institute of Arthritis and Musculoskeletal and Skin Diseases of the National Institutes of Health under award number U54AR057951. The content of this publication is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. Research staff assisting with the literature review and cognitive interviews includes Ana Miskovic, Marybeth Winingham, Allison Todd, Nicholas Formanski, and Azra Cikmirovic. We are indebted to the persons living with stroke, SCI and TBI who participated in the cognitive interviews.
59
We certify that no party having a direct interest in the results of the research supporting this article has or will confer a benefit on us or on any organization with which we are associated AND, if applicable, I certify that all financial and material support for this research and work are clearly identified in the title page of the manuscript.
63
Jin-Shei Lai, Ph.D., OTR/L, Research Associate Professor, Medical Social Sciences, Northwestern University Feinberg School of Medicine, 710 N Lake Shore Drive, #724, Chicago, IL 60611, 312-503-3370, [email protected]
RI PT
David Tulsky, PhD, Director, Assessment Research and Translation Rusk Institute, Department of Rehabilitation Medicine Department of Orthopedic Surgery Department of General Medicine New York University Langone Medical Center Ambulatory Care Center, 240 E. 38th Street, 17th Floor, New York, NY 10013, 212-263-6164, [email protected]
SC
David B. Gray, PhD, Professor, Program in Occupational Therapy, St. Louis, MO., [email protected], 314-286-1658.
Holly Hollingsworth, Ph.D., Associate Research Professor, Program in Occupational Therapy, St. Louis, MO. [email protected], 314-286-1630
EP
TE D
M AN U
Sara Jerousek, Project Manager, Center for Rehabilitation Outcomes Research, Rehabilitation Institute of Chicago, 345 E Superior St, Chicago IL 60611, [email protected], 312-238-4856.
AC C
60 61 62
Elizabeth A. Hahn, MA, Associate Professor, Department of Medical Social Sciences, Northwestern University Feinberg School of Medicine, [email protected]
ACCEPTED MANUSCRIPT
Environmental Factors Item Development for Persons with Stroke, Traumatic Brain
2
Injury and Spinal Cord Injury
AC C
EP
TE D
M AN U
SC
RI PT
1
1
ACCEPTED MANUSCRIPT
Abstract
3
Objective: To describe methods used in operationalizing environmental factors, to describe the
5
results of a research project to develop measures of environmental factors that affect
6
participation, and to define an initial item set of facilitators and barriers to participation
7
following stroke, traumatic brain injury, and spinal cord injury.
8
Design: Instrument development included an extensive literature review, item classification and
9
selection, item writing, and cognitive testing following the approach of the Patient-Reported
SC
RI PT
4
Outcomes Measurement Information System (PROMIS).
11
Setting: General community.
12
Participants: Ten content-area and outcome measurement experts contributed to instrument
13
development; 200 individuals with the target conditions participated in focus groups and 15
14
participated in cognitive testing.
15
Interventions: None.
16
Main Outcome Measures: Environmental factor items were categorized in six domains:
17
assistive technology; built and natural environment; social environment; services, systems, and
18
policies; access to information and technology; and economic quality of life.
19
Results: We binned 2,273 items across the six domains, winnowed this pool to 291 items for
20
cognitive testing, and recommended 274 items for pilot data collection.
21
Conclusions: Five of the six domains correspond closely to the ICF taxonomy of environmental
22
factors; the sixth domain, economic quality of life, reflects an important construct that reflects
23
financial resources that affect participation. Testing with a new and larger sample is underway to
24
evaluate reliability, validity, and sensitivity.
AC C
EP
TE D
M AN U
10
2
ACCEPTED MANUSCRIPT Environmental Factors Item Development Keywords: Environment, patient-reported outcomes, cognitive interviews, qualitative research,
26
questionnaires.
AC C
EP
TE D
M AN U
SC
RI PT
25
3
ACCEPTED MANUSCRIPT Environmental Factors Item Development 27
The United Nations’ General Assembly has identified participation as a primary and highly valued goal of rehabilitation, health care, and social services for people with disabilities.1
29
People with disabilities and their advocates emphasize that the physical, social, political, and
30
economic aspects of the environment are important influences on health and participation.2 The
31
World Health Organization’s International Classification of Functioning, Disability and Health
32
(ICF)3 has become the de facto taxonomy by which to describe the precursors and consequences
33
of disability. The ICF identifies environmental factors in five categories: 1) Products and
34
Technology; 2) Natural Environment; 3) Support and Relationships; 4) Attitudes; and 5)
35
Services, Systems and Policies. In spite of its ascendency, the ICF has been criticized for failing
36
to provide a model of the disablement/enablement process.4 The ICF provides no less than three
37
different coding conventions for its five environmental factor chapters, leaving instrument
38
developers with considerable latitude in how to operationalize environmental factors.
SC
M AN U
Many measures of the physical and social environment lack a strong theoretical
TE D
39
RI PT
28
foundation5,6 and fail to clarify the dynamic interaction between participation and the
41
environment.7 Mallinson and colleagues8 emphasize the need for a transactional measurement
42
approach that describes activity and participation in the context of the environment including
43
physical, social, and attitudinal supports and barriers to task and role performance, as well as
44
social and community participation.
AC C
EP
40
45
In spite of these conceptual and measurement challenges, the past decade has seen
46
increased interest in the development of measures of the environment, and there are several high
47
quality reviews of these instruments in the rehabilitation literature.5,9_ENREF_9 For example,
48
Whiteneck and Dikjers5 and Noreau and Boschen9 describe methodological challenges in
49
operationalizing environmental factors. There is, however, no consensus on which instruments
4
ACCEPTED MANUSCRIPT Environmental Factors Item Development 50
are best for describing participation-environment interactions. Empirical evidence of the impact
51
on the environment on participation is inconsistent.9,10
52
Many contemporary measures of the environment use classical test theory and combine multiple aspects of the environment into a single scale or report descriptive, item-level
54
information only; these issues are reviewed in the first paper of this special issue.11_ENREF_11
55
Given that participation occurs across multifaceted environments, instrument development must
56
strike a balance between measurement precision and respondent burden. 5 Rather than try and
57
capture the environment in its entirety (which would be very difficult to do), we should focus on
58
assessing aspects of the environment that are either amenable to change or that are expected to
59
affect specific outcomes in targeted communities. Whereas it remains unclear what the most
60
appropriate measurement models are for assessing environmental factors and their impact on
61
participation, it is evident that we need additional research and approaches to refine the construct
62
of environmental facilitators and barriers to participation.
SC
M AN U
TE D
63
RI PT
53
Contemporary psychometric methods and the application of item response theory provide a viable approach to measurement of environmental factors. In this study, we sought to apply a
65
fresh approach to the measurement of environmental factors using the mixed methods instrument
66
development process championed by the Patient-Reported Outcomes Measurement Information
67
System (PROMIS) initiative.12 We sought to focus on environmental factors that people with
68
disabilities perceive as facilitating participation or acting as barriers. We purposely focused on
69
the subjective experiences of environmental factors, which are best captured by self-report,
70
rather than objective environmental descriptions. These issues are summarized in the second
71
paper of this special issue.13_ENREF_13 Thus, the objectives of this manuscript are to describe: (1)
72
methods used to operationalize environmental factors; (2) the results of a research project to
AC C
EP
64
5
ACCEPTED MANUSCRIPT Environmental Factors Item Development develop measures of environmental factors that affect participation; and (3) an initial item set of
74
facilitators and barriers to participation. The goal of the larger project of which this study is a
75
component is to develop, test, and evaluate measures of environmental factors and their
76
influence on participation for persons with stroke, TBI, and spinal cord injury (SCI). A validated
77
set of environmental factor instruments will allow us to determine to what extent they influence
78
social health and participation.
80
SC
Methods
79
RI PT
73
We used a mixed-methods approach to constructing and evaluating items. The major tasks, described below, entailed concept elicitation, item classification and creation, and item
82
refinement. These tasks and procedures follow the standards established by the PROMIS
83
Network.12 A domain chair and co-chair oversaw the development of each item set. We obtained
84
Institutional Review Board approval for this project prior to the start of this project.
85
Phase I: Concept Elicitation, Qualitative Focus Group Analysis, and Conceptual Modeling
86
As part of our ongoing research on participation, we had access to verbatim transcripts
87
from 38 focus groups involving over 200 people with diverse disabilities across different sites
88
and regions.14-16 The samples included people with stroke, TBI, SCI, and other disabilities. We
89
analyzed these data to create participation and quality of life assessments such as the Community
90
Participation Indicators, and the SCI-QOL/TBI-QOL measurement systems.16-18 In the focus
91
groups, participants stressed the importance of environmental factors influencing participation;
92
the depth and detail of environmental factor information allowed us to reanalyze the data for the
93
current project. We used a grounded theory approach and constant comparative approach to
94
coding and interpreting._ENREF_1719 This analysis yielded rich environmental factor themes,
95
detailed examples, and relevant quotes to highlight specific factors, across the groups. We
AC C
EP
TE D
M AN U
81
6
ACCEPTED MANUSCRIPT Environmental Factors Item Development categorized the feedback in eight distinct environmental factor domains identified as important
97
by people with disabilities. The second column of Table 1 lists the domains. Some of these
98
domains directly correspond to ICF typology shown in column 1 (e.g., systems and services);
99
others focus on specific components of ICF environmental factor chapters (e.g., transportation,
RI PT
96
assistive technology), and another represents an issue that is an infrequent focus of research,
101
economic quality of life. We grouped social supports and societal attitudes together as a social
102
environmental domain.
103
Phase II: Item Classification and Creation
We binned items assigning them to domains, using expert opinion. Thus, we winnowed
M AN U
104
SC
100
items, removing items with similar content, using expert opinion. We created a working
106
definition of each thematic code based on focus group data, review of legacy assessments, and
107
environmental factor research. We used this guide to categorize and generate sample items
108
within each domain.
109
Phase III: Item Refinement
110
TE D
105
Cognitive interviews. Following PROMIS’ qualitative item review process,12 we conducted cognitive interviews with community-dwelling people with disabilities to ensure
112
comprehension of items. Participants completed a subset of up to 50 items. We asked
113
participants to explain what the question meant in their own words and to describe how they
114
arrived at the answer. Interviewers probed to clarify the participants’ interpretation of specific
115
words and phrases. Each interview lasted 20 to 40 minutes. Most participants volunteered to
116
complete 2 to 3 interviews, requiring 1 to 4 hours in total. Each item was reviewed by at least 4
117
and up to 9 participants who had different disabilities. The project team worked to ensure that
118
members of each diagnostic group reviewed items in each subset to minimize sampling bias.
AC C
EP
111
7
ACCEPTED MANUSCRIPT Environmental Factors Item Development 119
Participants received an honorarium and accommodations as necessary. Accommodations
120
included increasing time, discussing confusion regarding item content, and providing breaks, as
121
needed. Interviewers took verbatim notes during the interviews and reviewed them with the team.
123
We coded responses and domain chair and co-chairs reviewed them. The research team reviewed
124
and discussed problematic items. Revised items underwent a second round of cognitive testing
125
with an additional 3 participants. Once the research team reached consensus on the suitability of
126
the revised items, we deemed the items ready for pilot testing.
SC
Reading Level Analysis: Patient literacy is a well-documented challenge for use of self-
M AN U
127
RI PT
122
administered questionnaires.20-23 Although we designed the items to be self-administered, it is
129
essential to verify that respondents comprehend their wording and meanings are clear. High
130
literacy expectations, complex terms, and ambiguous items can create comprehension problems
131
and impair test validity. We assessed the reading level of each item using the Flesch-Kinkaid and
132
Lexile Framework24 and reworded items that were above a fifth grade level. Maintaining a fifth
133
grade reading level maximizes the scale’s accessibility to participants with low reading skills. Translatability Review: In anticipation of future translation of items to Spanish, we
EP
134
TE D
128
followed PROMIS Guidelines for Translation and Cultural Adaptation.25 This process involves
136
forward and backward translations of items, response options, and instructions. We followed an
137
iterative process of forward and back-translation with bilingual expert review.
138
AC C
135
Results
139
Literature Review
140
We searched PsychLit, PubMed, and CINAHL using the terms “environment,” “social
141
environment,” and “environmental factors” as a keyword and identified articles that also
8
ACCEPTED MANUSCRIPT Environmental Factors Item Development contained the term “social participation” or the keyword “participation.” We reviewed the
143
matching articles and focused on those that reported development of instruments for medical,
144
vocational, or mental health rehabilitation.
145
Domain Description
146
RI PT
142
Table 1 lists the initial eight domains, the number of items we pooled and binned from the literature review and the qualitative review, the count of winnowed items and the final set of
148
items that underwent cognitive testing. After binning and winnowing items, we revised the
149
conceptual framework. From an initial set of 2,273 items across domains, we winnowed this set
150
to 291 items for cognitive testing. Table 2 provides examples of item content. We phrased each
151
item as a simple, declarative statement that is either positively- or negatively-stated. Given the
152
several ways in which we characterized environmental features, we used five types of rating
153
scales as shown in Table 3, including two variations of a frequency rating scale, one indicating
154
capability using a dichotomous environmental attribution response option and the other
155
consisting of a five-point magnitude rating scale.
TE D
M AN U
SC
147
Domain 1, Access to Information and Technology, pertains to knowledge and
157
information technologies. Information technology includes conventional devices and technology
158
to transmit and receive information such as mobile or landline phones, computers, email and
159
internet services. Usability of information includes factors that influence the ability to access and
160
understand information, and includes considerations of reading level, literacy, usability,
161
transparency, and information finding. While this domain does not include specialized
162
communication or devices, it does include built-in, commercially available access features such
163
as Microsoft control panel accessibility features.
AC C
EP
156
9
ACCEPTED MANUSCRIPT Environmental Factors Item Development 164
Domain 2, Assistive Technology (AT), includes the use and availability of specialized devices to facilitate participation in daily activities, such as mobility, personal care, work,
166
leisure, and home participation. These devices are typically specialized and not part of standard
167
built environments. AT belongs to an individual and may be portable. Our definition of AT does
168
not include features of the built environment in public spaces, universal design features like
169
ramps on public buildings and Braille signage. People use AT to assist their performance across
170
a variety of functional tasks. We developed parallel items for mobility, personal care, vision and
171
hearing, and communication AT. Participants answer a filter question, “Do you use any mobility
172
devices, such as a cane, walker, scooter, or wheelchair?” If the answer is affirmative, they name
173
the device they use most often and report on the availability and reliability of the device and its
174
effect on participation. Participants who report that they need a device, but do not currently have
175
it, answer a follow-up question on barriers to accessing AT. Participants who report that they do
176
not need a device skip to the next section.
SC
M AN U
TE D
177
RI PT
165
Domain 3, Built and Natural Environment, pertains to the natural and built factors in the environment, including architectural features of buildings; land development such as
179
sidewalks and roadways; environmental features such as noise, crowds, indoor air quality; and
180
cognitive and sensory cues. Built factors include aspects of neighborhood and community
181
livability such as safety, lighting, and access to community resources like stores, and banks.
182
Natural features of the environment include climate and weather conditions, preparedness for
183
natural disasters, and the means for addressing weather, climate, and emergency conditions. This
184
domain includes geographical and topographical features of the environment such as outdoor air
185
quality. We wrote items in pairs; the first item assesses “how much difficulty do you have”
186
performing an activity and the second item asks respondent if they attribute barriers to the
AC C
EP
178
10
ACCEPTED MANUSCRIPT Environmental Factors Item Development environment such as precipitation or noise. For example, the first item of one pair asks “How
188
much difficulty do you have feeling safe due to crime in your community?” The second item
189
asks respondents to indicate “yes” or “no” whether “The difficulty I have feeling safe is due to a
190
high crime rate, inadequate number of police, or no emergency call boxes.
191
RI PT
187
Domain 4, Systems, Services, and Policies, pertains to social services, employment, educational, housing, independent living, health care policies, and systems of delivery. It
193
includes services provided by non-profit, voluntary and community agencies. Two subdomains
194
emerged pertaining to economics and transportation. The economic subdomain assesses access to
195
coverage of work and educational systems and resources, as well as to alternative disability and
196
income subsidy programs, workers’ compensation, unemployment, and others systems such as
197
food stamps, child care, and affordable housing. The transportation subdomain pertains to public
198
and private travel, including paratransit and adapted transit systems, and access, availability,
199
affordability, and quality of these services. The items are introduced with the statement: “Many
200
systems and services can affect participation at home and in the community. For each category
201
below, tell us how these services and systems affect your participation.”
TE D
M AN U
SC
192
Domain 5, Social Environment, includes content from the ICF Support and
203
Relationships and Attitudes domains. Following the ICF model, we sought to measure both
204
Social Support and Attitudes. To measure Social Support, we adopted as legacy instruments the
205
PROMIS Social Support item pools, which assess the availability of companionship,
206
informational, instrumental, and emotional support. The PROMIS Social Support item pools
207
have been tested in large samples of English- and Spanish-speaking adults and are intended for
208
use across chronic illness, disability, and general population samples. For the current project we
209
created an Attitudes item set specific to disability by selecting and, when necessary, modifying
AC C
EP
202
11
ACCEPTED MANUSCRIPT Environmental Factors Item Development items from existing instruments (e.g., FABS/M Section 6-Community Environment: Services &
211
Attitudes; Stigma Scale for Chronic Illness26) and writing new items covering content identified
212
by our qualitative data and input from content area experts. In keeping with the ICF, we included
213
both positive and negative attitudes about disability. Therefore, item content includes
214
marginalization, stigmatization, oppression, and discrimination as well as inclusion, acceptance,
215
respect, and fairness. Also like the ICF, items cover both the attitudes of individuals such as
216
health care providers, as well as society as a whole. We excluded the attitudes of respondents
217
themselves, reflecting self-stigma, because it does not constitute an environmental factor. The
218
fourth article of this special issue describe the psychometric properties of this item pool in
219
greater detail.27
M AN U
SC
RI PT
210
Domain 6, Economic Quality of Life, is based on the empirical results of consistent
221
feedback from focus group participants regarding the importance of economic assets, and the
222
effects of these factors on participation. The ICF taxonomy code for “financial aspects” (e1650)
223
is related to this construct. This domain is critical to individuals’ quality of life as well as serving
224
as a barrier to participation; it reflects a distinctive construct that few environmental instruments
225
measure.28,29 This domain focuses on how financial resources influence satisfaction with one’s
226
living situation, adequacy and affordability of health services, adequate and affordable food,
227
affordable community recreational activities, and family and friend financial assistance. The fifth
228
article of this special issue describe the psychometric properties of this item pool in greater
229
detail.30
230
Cognitive Interviews
231 232
AC C
EP
TE D
220
We recruited 15 people with stroke (n=4), TBI (n=5), or SCI (n=6) to complete cognitive interviews by telephone. The mean age was 46 years; women comprised 33% of the sample.
12
ACCEPTED MANUSCRIPT Environmental Factors Item Development Participants included African-Americans (54%), Non-Hispanic Caucasians (33%) and Hispanics
234
(13%). While 80% had completed some post-secondary education, only 13% was employed;
235
others were retired (40%), seeking employment (20%) or unemployed and not seeking
236
employment (27%). The majority was ambulatory (53%) though 47% used a walking device and
237
47% used a wheelchair. Participants reviewed 50 of the 291 items; 4 to 12 participants examined
238
each item. For each newly developed item, we asked participants follow-up questions to
239
ascertain comprehension, opinions about the suitability of the items, and revisions. We used this
240
feedback to delete 17 items and revise 57 others. Revisions included narrowing and specifying
241
the focus of concepts that were too broad; reducing the length of items; clarifying the meaning of
242
ambiguous items; and reducing potential ceiling or floor effects. Table 2 provides examples of
243
the revisions.
M AN U
SC
RI PT
233
Discussion
245
We began this project by reviewing the social sciences, disability and rehabilitation
TE D
244
literature and identifying a large pool of items from a diverse literature describing environmental
247
factors for medical, vocational, developmental, and psychiatric rehabilitation applications. It
248
quickly became evident that this topic has generated considerable measurement interest across
249
varied human service arenas, but that existing instruments such as FABS/M were narrow in
250
scope while others pertain to nearly everyone. None of the instruments that we reviewed used
251
contemporary psychometric methods of instrument development; all relied on classical test
252
theory methods or reported descriptive, item-level information. Therefore, we proceeded to
253
develop a new measurement system that would describe and assess environmental factors. As
254
such, an immediate challenge was to reduce the multiplicity of approaches and items to a
255
conceptually concise and clinically practical set of items. We began with the ICF’s taxonomy of
AC C
EP
246
13
ACCEPTED MANUSCRIPT Environmental Factors Item Development environmental factors and distinguished subcomponents of some ICF domains. In the case of the
257
Built and Natural Environment, we realized that these are inseparable aspects of the physical
258
world. Consequently, five of the six domains correspond closely to the ICF taxonomy of
259
environmental factors; the sixth domain, economic quality of life, reflects an important construct
260
that reflects the resources individuals have that affect their participation.
RI PT
256
Cognitive interviewing provided us with feedback that allowed us to reduce the 291 draft
262
items to 274. The number of items per domain ranges from 14 (AT) to 82 (Social Attitudes). The
263
AT items are used as a set to evaluate the use of mobility devices, equipment to assist with
264
activities of daily living, and devices to interact with people and environments. Depending on an
265
individual’s use of technology, a respondent would likely complete no more than 45 of the total
266
274 items. Feedback from persons living with the consequences of disability was critically
267
important to reduce ambiguity and wordiness, and enhance clarity and improve response
268
distributions. The next step in transforming these item sets into functional item pools is to
269
administer the items to a diverse group of individuals living with stroke, SCI and TBI. We will
270
use methods developed by the PROMIS project to explore the dimensionality of each domain
271
using confirmatory factor analysis and item fit to models based on item response
272
theory._ENREF_2431 We envision developing computer adaptive tests and short form versions to
273
measure each domain to evaluate the influence of environmental factors on participation.
274
Study Limitations
M AN U
TE D
EP
AC C
275
SC
261
Results from this study are limited by the number of participants in the focus groups, and
276
the number and diversity of persons who participated in the cognitive interviews. While we
277
searched for literature in various aspects of rehabilitation, including medical, vocational
278
intellectual and psychiatric, we may have overlooked some sources. Because of time constraints,
14
ACCEPTED MANUSCRIPT Environmental Factors Item Development we relied on focus groups that we conducted in earlier projects; these groups addressed multiple
280
aspects of participation and environmental factors. The geographic sampling was limited to
281
Colorado, Illinois, and New Jersey and drew participants primarily from urban and suburban
282
communities. The consequences of under representation of persons from rural settings are
283
unknown. The persons completing cognitive interviews were from metropolitan Chicago; the
284
extent to which an urban, Midwestern city limits perspectives on environmental barriers and
285
facilitators is unknown. Future studies should recruit participants from a broader sample of
286
communities. Conclusions
288
M AN U
287
SC
RI PT
279
A fully developed set of environmental factor items will support a variety of research and clinical applications. Clinicians are interested in identifying aspects of the environment that are
290
potentially modifiable; their interventions could focus on problematic environmental factors that
291
limit participation and life satisfaction. Clinicians and researchers are interested in acquiring a
292
fuller knowledge of how environmental factors affect participation and quality of life. Disability
293
advocates could use more specific and detailed information about environmental factors to lobby
294
for elimination of barriers. The domain definitions and items that operationalize the domains will
295
allow us to evaluate these environmental factor influences in a variety of applications.
AC C
EP
TE D
289
15
ACCEPTED MANUSCRIPT RRTC on Improving Measurement of Medical Rehabilitation Outcomes
R3. Environmental Factor Measures
References
7.
8.
9.
10.
11.
12. 13.
14.
15.
16.
17.
RI PT
SC
6.
M AN U
4. 5.
TE D
3.
EP
2.
Resolution adopted by the General Assembly of the United Nations. The Standard Rules on the Equalization of Opportunities for persons with Disabilities. 48th Session, agenda item 109. 1994. National Disability Rights Network. National Disability Rights Network. 2012; http://www.ndrn.org/index.php. Accessed April 13, 2013. World Health Organization. International Classification of Functioning, Disability, and Health (ICF) 2001:234-237. Field MJ, Jette A. The future of disability in America. National Academies Press; 2007. Whiteneck G, Dijkers MP. Difficult to measure constructs: conceptual and methodological issues concerning participation and environmental factors. Archives of physical medicine and rehabilitation. Nov 2009;90(11 Suppl):S22-35. Alvarelhao J, Silva A, Martins A, et al. Comparing the content of instruments assessing environmental factors using the International Classification of Functioning, Disability and Health. Journal of rehabilitation medicine. Jan 2012;44(1):1-6. Badley EM. Enhancing the conceptual clarity of the activity and participation components of the International Classification of Functioning, Disability, and Health. Social science & medicine. Jun 2008;66(11):2335-2345. Mallinson T, Hammel J. Measurement of participation: intersecting person, task, and environment. Archives of physical medicine and rehabilitation. Sep 2010;91(9 Suppl):S29-33. Noreau L, Boschen K. Intersection of participation and environmental factors: a complex interactive process. Archives of physical medicine and rehabilitation. Sep 2010;91(9 Suppl):S44-53. Hollingsworth H, Gray DB. Structural equation modeling of the relationships between participation in leisure activities and community environments by people with mobility impairments. Archives of physical medicine and rehabilitation. Aug 2010;91(8):11741181. Magasi S, Wong A, Wang C, et al. Theoretical FFoundations for the Measurement of Enviromental Factors and Participation among People with Disabilities. Archives of physical medicine and rehabilitation. Unver review. DeWalt DA, Rothrock N, Yount S, Stone AA, Group PC. Evaluation of item candidates: the PROMIS qualitative item review. Medical care. May 2007;45(5 Suppl 1):S12-21. Hammel J, Magasi S, Heinemann AW, et al. Environmental Barriers & Supports to Participation: An Insider Perspective from People with Disabilities. Archives of physical medicine and rehabilitation. Under review. Gray DB, Hollingsworth HH, Stark SL, Morgan KA. Participation survey/mobility: psychometric properties of a measure of participation for people with mobility impairments and limitations. Archives of physical medicine and rehabilitation. Feb 2006;87(2):189-197. Hammel J, Magasi S, Heinemann A, Whiteneck G, Bogner J, Rodriguez E. What does participation mean? An insider perspective from people with disabilities. Disability and rehabilitation. 2008;30(19):1445-1460. Carlozzi NE, Tulsky DS, Kisala PA. Traumatic brain injury patient-reported outcome measure: identification of health-related quality-of-life issues relevant to individuals with traumatic brain injury. Archives of physical medicine and rehabilitation. Oct 2011;92(10 Suppl):S52-60. Heinemann AW, Magasi S, Bode RK, et al. Measuring enfranchisement: Importance and control of participation by people with disabilities. Archives of physical medicine and rehabilitation. in press.
AC C
1.
16
ACCEPTED MANUSCRIPT RRTC on Improving Measurement of Medical Rehabilitation Outcomes
23.
24. 25.
26.
27.
28.
29.
30.
31.
RI PT
SC
22.
M AN U
21.
TE D
20.
EP
19.
Tulsky DS, Kisala PA, Victorson D, et al. Developing a contemporary patient-reported outcomes measure for spinal cord injury. Archives of physical medicine and rehabilitation. Oct 2011;92(10 Suppl):S44-51. Strauss AL, Corbin JM. Basics of qualitative research : techniques and procedures for developing grounded theory. 2nd ed. Thousand Oaks: Sage Publications; 1998. Cella D, Clauser SB, Flynn KE, et al. Standardizing patient-reported outcomes assessment in cancer clinical trials: A PROMIS initiative. Journal of Clinical Oncology. 2007;25:5106-5112. Hammel J, Magasi S, Heinemann A, Whiteneck G, Bogner J, Rodriguez E. What does participation mean? An insider perspective from people with disabilities. Disability and Rehabilitation. 2008;30(19):1445-1460. Magasi S, Hammel, J., Heinemann, A.W., Whiteneck, G., Corrigan, J., Bogner, J. Participation: Multiple stakeholder's constructions, priorities and comparisons. Under review. Hahn EA, Cella D. Health outcomes assessment in vulnerable populations: measurement challenges and recommendations. Archives of physical medicine and rehabilitation. Apr 2003;84(4 Suppl 2):S35-42. Stenner A, Horabin, I., Smith, D., Smith, M. The Lexile framework. The Lexile framework. Durham, NC: Metametrics; 1998. PROMIS® Instrument Development and Psychometric Evaluation Scientific Standards. 2012; http://www.nihpromis.org/Documents/PROMIS_Standards_050212.pdf. Accessed February 11, 2013. Rao D, Choi SW, Victorson D, et al. Measuring stigma across neurological conditions: the development of the stigma scale for chronic illness (SSCI). Quality of life research : an international journal of quality of life aspects of treatment, care and rehabilitation. Jun 2009;18(5):585-595. Garcia SF, Hahn EA, Magasi S, et al. Development of Self-Report Measures of Social Attitudes that Act as Environmental Barriers and Facilitators for People with Disabilities. Archives of physical medicine and rehabilitation. Under review. Reinhardt JD, Post MW. Measurement and evidence of environmental determinants of participation in spinal cord injury: A systematic review of the literature. Topics in Spinal Cord Injury Rehabilitation. 2010;15(4):26-48. Escorpizo R, Graf S, Marti A, et al. Domain sets and measurement instruments on participation and environmental factors in spinal cord injury research. American Journal of Physical Medicine & Rehabilitation. 2011;90(11):S66-S78. Tulsky DS, Kisala PA, Lai JS, Carlozzi N, Hammel J, Heinemann A. Developing an Item Bank to Measure Economic Quality of Life for Individuals with Disabilities. Archives of physical medicine and rehabilitation. Under review. Reeve BB, Hays RD, Bjorner JB, et al. Psychometric evaluation and calibration of health-related quality of life item banks: plans for the Patient-Reported Outcomes Measurement Information System (PROMIS). Medical care. May 2007;45(5 Suppl 1):S22-31.
AC C
18.
R3. Environmental Factor Measures
17
ACCEPTED MANUSCRIPT
Table 1. Comparison of ICF and Project-Specific Conceptual Domains Cognitive Interviews Item Count
Pilot Test Item Count
15 (5)*
14 (5)*
36
36
90
82
76
75
32
25
37
37
Total 2,273 392 291 *AT items are repeated for mobility device, ADL equipment, and devices used to interact with people and environments. There was also 1 item developed to assess barriers to these devices.
274
Winnowed Items
178
104
79
37
Assistive technology
Natural Natural environment Environment and human Built made changes environment
Services, Systems and Policies
Social supports and attitudes Services, systems, policies Transportation Access to information and technology
710
91
411 136 112
42
Assistive technology
Built and natural environment
Social environment
35
28
37
35
Services, systems, policies Access to information and technology Economic Quality of Life
AC C
EP
Economicfinancial
51
TE D
Support and Relationships Attitudes
605
Revised
RI PT
Binned Items
SC
Products and Technology
Initial Framework
M AN U
ICF Taxonomy
1
ACCEPTED MANUSCRIPT
Table 2. Examples of Item Revisions Item Stem Domain
Access to information and technology
Excessive wordiness
EP
AC C
SC
Assistive Technology
New Item Broken into two items: In case of a health emergency, I can get the information I need easily. In case of a natural disaster, I can get the information I need easily.
RI PT
‘Emergency’ is too broad of a concept
Infrequently used device category
Websites are available in a format I can understand if I need them.
Do you use any equipment or devices to help you communicate and interact with people; to see, hear, or remember things; or to control your environment? I can use my in a variety of places.
Assistive Technology
Confusion with “settings”
Built and Natural Environment
Suggested rephrase to specify crime
How much difficulty do you have feeling safe due to crime in your community?
Built and Natural Environment
Suggested rephrase, edit: “buildings” and add ‘emergency’ to be more specific
How much difficulty do you have feeling safe in stores during an emergency?
Suggested rephrase
My community offers support groups I can use.
Suggested rephrase
Help to make my home accessible i available if I need it.
Confusion over the word: “sensitive”
The people in my life understand the challenges I face because of my disability.
TE D
Websites with information are available in a format I can understand if I need them. Do you use any communication devices, such as a voice synthesizer or communication board? I can use my in a variety of settings. How much difficulty do you have feeling safe outdoors in your community? How much difficulty do you have feeling safe in buildings in your community? My community offers self-help and social support groups. Help modifying my home to make it accessible is available if I need it. The people in my life are sensitive to the challenges I face because of my disability. Health care providers understand the needs of people with disabilities. How do you believe your financial situation is today as compared to other people your age? I have access to emergency funds.
Access to information and technology
M AN U
In case of a health emergency, I can get the information I need easily
Reason for Revision
Services, Systems and Policies Services, Systems and Policies
Social Environmental
Social Environmental Economic Quality of Life Economic Quality of Life
Ambiguity about ”health care providers” Difficult comparison; idiosyncratic rating scale (worse vs. same vs. better) Confusing
Health care professionals understand the needs of people with disabilities. Item eventually dropped
I have access to extra money in case of an emergency.
2
ACCEPTED MANUSCRIPT
Economic quality of life
Capability (difficulty)
Built and natural environment
M AN U
Yes – No
SC
Frequency
Environmental Attribution Built and natural environment Assistive technology
AC C
EP
TE D
Magnitude
Response Options Never Rarely Sometimes Usually Always Never Rarely Sometimes Often Always None A little Somewhat A lot Complete difficulty Yes No Not at all A little bit Somewhat Quite a bit Very much
RI PT
Table 3. Item Response Options Category Domain Frequency Access to Information and Technology Social Environment Systems, Services, Policies
3
S0003-9993(13)01308-7
DOI:
10.1016/j.apmr.2013.11.024
Reference:
YAPMR 55692
To appear in:
ARCHIVES OF PHYSICAL MEDICINE AND REHABILITATION
Received Date: 1 August 2013 Revised Date:
30 October 2013
Accepted Date: 10 November 2013
Please cite this article as: Heinemann AW, Magasi S, Hammel J, Carlozzi NE, Garcia SF, Hahn EA, Lai J-S, Tulsky D, Gray DB, Hollingsworth H, Jerousek S, Environmental Factors Item Development for Persons with Stroke, Traumatic Brain Injury and Spinal Cord Injury, ARCHIVES OF PHYSICAL MEDICINE AND REHABILITATION (2014), doi: 10.1016/j.apmr.2013.11.024. This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
ACCEPTED MANUSCRIPT
Environmental Factors Item Development for Persons with Stroke, Traumatic Brain
2
Injury and Spinal Cord Injury
3
Allen W. Heinemann1
4
Susan Magasi2
5
Joy Hammel3
6
Noelle E. Carlozzi4
7
Sofia F. Garcia5
8
Elizabeth A. Hahn6
9
Jin-Shei Lai7
10
David Tulsky8
11
David B. Gray9
12
Holly Hollingsworth10
13
Sara Jerousek11
TE D
M AN U
SC
RI PT
1
14
Running head: Environmental Factors Item Development
15 16 17 18
1
19 20
2
21 22 23
3
24 25 26 27
4
28 29 30 31
5
EP
Allen Heinemann, PhD, Director, Center for Rehabilitation Outcomes Research, Rehabilitation Institute of Chicago and Professor, Department of Physical Medicine and Rehabilitation, Feinberg School of Medicine, Northwestern University, 345 E. Superior St. Chicago, IL 60611, 312-238-2802, [email protected].
AC C
Susan Magasi, PhD, Assistant Professor, University of Illinois at Chicago, Occupational Therapy, 1919 W. Taylor St., Room 327, Chicago, IL 60613, 312-996-4306 Joy Hammel, PhD, OTR/L, Professor: University of Illinois at Chicago, Depts. of Occupational Therapy and Disability & Human Development 1919 W. Taylor St., Room 311, Chicago, IL 60612; [email protected]; 312-996-3513 Noelle E. Carlozzi, PhD, University of Michigan, Rehabilitation Outcomes and Assessment Research, Department of Physical Medicine & Rehabilitation, Assistant Professor, North Campus Research Complex, 2800 Plymouth Road, Building NCRC B520, Room 3211, Ann Arbor, MI 48109-2800, 734-763-8917 [email protected] Sofia F. Garcia, PhD, Assistant Professor, Department of Medical Social Sciences, Department of Psychiatry and Behavioral Sciences, Feinberg School of Medicine, Northwestern University, 625 N. Michigan Ave., Suite 2700, Chicago, IL 60611, 312-503-3449, [email protected]
ACCEPTED MANUSCRIPT RRTC on Improving Measurement of Medical Rehabilitation Outcomes
R3. Environmental Factor Measures
32 33
6
34 35 36
7
37 38 39 40
8
41 42
9
43 44
10
45 46
11
47 48 49 50 51 52 53 54 55 56 57 58
National Institute on Disability and Rehabilitation Research provided funding through a Rehabilitation Research and Training Center on Improving Measurement of Medical Rehabilitation Outcomes grant (H133B090024) awarded to the Rehabilitation Institute of Chicago. A portion of Dr. Garcia’s time toward this publication was supported by the National Center for Complementary and Alternative Medicine and the National Institute of Arthritis and Musculoskeletal and Skin Diseases of the National Institutes of Health under award number U54AR057951. The content of this publication is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. Research staff assisting with the literature review and cognitive interviews includes Ana Miskovic, Marybeth Winingham, Allison Todd, Nicholas Formanski, and Azra Cikmirovic. We are indebted to the persons living with stroke, SCI and TBI who participated in the cognitive interviews.
59
We certify that no party having a direct interest in the results of the research supporting this article has or will confer a benefit on us or on any organization with which we are associated AND, if applicable, I certify that all financial and material support for this research and work are clearly identified in the title page of the manuscript.
63
Jin-Shei Lai, Ph.D., OTR/L, Research Associate Professor, Medical Social Sciences, Northwestern University Feinberg School of Medicine, 710 N Lake Shore Drive, #724, Chicago, IL 60611, 312-503-3370, [email protected]
RI PT
David Tulsky, PhD, Director, Assessment Research and Translation Rusk Institute, Department of Rehabilitation Medicine Department of Orthopedic Surgery Department of General Medicine New York University Langone Medical Center Ambulatory Care Center, 240 E. 38th Street, 17th Floor, New York, NY 10013, 212-263-6164, [email protected]
SC
David B. Gray, PhD, Professor, Program in Occupational Therapy, St. Louis, MO., [email protected], 314-286-1658.
Holly Hollingsworth, Ph.D., Associate Research Professor, Program in Occupational Therapy, St. Louis, MO. [email protected], 314-286-1630
EP
TE D
M AN U
Sara Jerousek, Project Manager, Center for Rehabilitation Outcomes Research, Rehabilitation Institute of Chicago, 345 E Superior St, Chicago IL 60611, [email protected], 312-238-4856.
AC C
60 61 62
Elizabeth A. Hahn, MA, Associate Professor, Department of Medical Social Sciences, Northwestern University Feinberg School of Medicine, [email protected]
ACCEPTED MANUSCRIPT
Environmental Factors Item Development for Persons with Stroke, Traumatic Brain
2
Injury and Spinal Cord Injury
AC C
EP
TE D
M AN U
SC
RI PT
1
1
ACCEPTED MANUSCRIPT
Abstract
3
Objective: To describe methods used in operationalizing environmental factors, to describe the
5
results of a research project to develop measures of environmental factors that affect
6
participation, and to define an initial item set of facilitators and barriers to participation
7
following stroke, traumatic brain injury, and spinal cord injury.
8
Design: Instrument development included an extensive literature review, item classification and
9
selection, item writing, and cognitive testing following the approach of the Patient-Reported
SC
RI PT
4
Outcomes Measurement Information System (PROMIS).
11
Setting: General community.
12
Participants: Ten content-area and outcome measurement experts contributed to instrument
13
development; 200 individuals with the target conditions participated in focus groups and 15
14
participated in cognitive testing.
15
Interventions: None.
16
Main Outcome Measures: Environmental factor items were categorized in six domains:
17
assistive technology; built and natural environment; social environment; services, systems, and
18
policies; access to information and technology; and economic quality of life.
19
Results: We binned 2,273 items across the six domains, winnowed this pool to 291 items for
20
cognitive testing, and recommended 274 items for pilot data collection.
21
Conclusions: Five of the six domains correspond closely to the ICF taxonomy of environmental
22
factors; the sixth domain, economic quality of life, reflects an important construct that reflects
23
financial resources that affect participation. Testing with a new and larger sample is underway to
24
evaluate reliability, validity, and sensitivity.
AC C
EP
TE D
M AN U
10
2
ACCEPTED MANUSCRIPT Environmental Factors Item Development Keywords: Environment, patient-reported outcomes, cognitive interviews, qualitative research,
26
questionnaires.
AC C
EP
TE D
M AN U
SC
RI PT
25
3
ACCEPTED MANUSCRIPT Environmental Factors Item Development 27
The United Nations’ General Assembly has identified participation as a primary and highly valued goal of rehabilitation, health care, and social services for people with disabilities.1
29
People with disabilities and their advocates emphasize that the physical, social, political, and
30
economic aspects of the environment are important influences on health and participation.2 The
31
World Health Organization’s International Classification of Functioning, Disability and Health
32
(ICF)3 has become the de facto taxonomy by which to describe the precursors and consequences
33
of disability. The ICF identifies environmental factors in five categories: 1) Products and
34
Technology; 2) Natural Environment; 3) Support and Relationships; 4) Attitudes; and 5)
35
Services, Systems and Policies. In spite of its ascendency, the ICF has been criticized for failing
36
to provide a model of the disablement/enablement process.4 The ICF provides no less than three
37
different coding conventions for its five environmental factor chapters, leaving instrument
38
developers with considerable latitude in how to operationalize environmental factors.
SC
M AN U
Many measures of the physical and social environment lack a strong theoretical
TE D
39
RI PT
28
foundation5,6 and fail to clarify the dynamic interaction between participation and the
41
environment.7 Mallinson and colleagues8 emphasize the need for a transactional measurement
42
approach that describes activity and participation in the context of the environment including
43
physical, social, and attitudinal supports and barriers to task and role performance, as well as
44
social and community participation.
AC C
EP
40
45
In spite of these conceptual and measurement challenges, the past decade has seen
46
increased interest in the development of measures of the environment, and there are several high
47
quality reviews of these instruments in the rehabilitation literature.5,9_ENREF_9 For example,
48
Whiteneck and Dikjers5 and Noreau and Boschen9 describe methodological challenges in
49
operationalizing environmental factors. There is, however, no consensus on which instruments
4
ACCEPTED MANUSCRIPT Environmental Factors Item Development 50
are best for describing participation-environment interactions. Empirical evidence of the impact
51
on the environment on participation is inconsistent.9,10
52
Many contemporary measures of the environment use classical test theory and combine multiple aspects of the environment into a single scale or report descriptive, item-level
54
information only; these issues are reviewed in the first paper of this special issue.11_ENREF_11
55
Given that participation occurs across multifaceted environments, instrument development must
56
strike a balance between measurement precision and respondent burden. 5 Rather than try and
57
capture the environment in its entirety (which would be very difficult to do), we should focus on
58
assessing aspects of the environment that are either amenable to change or that are expected to
59
affect specific outcomes in targeted communities. Whereas it remains unclear what the most
60
appropriate measurement models are for assessing environmental factors and their impact on
61
participation, it is evident that we need additional research and approaches to refine the construct
62
of environmental facilitators and barriers to participation.
SC
M AN U
TE D
63
RI PT
53
Contemporary psychometric methods and the application of item response theory provide a viable approach to measurement of environmental factors. In this study, we sought to apply a
65
fresh approach to the measurement of environmental factors using the mixed methods instrument
66
development process championed by the Patient-Reported Outcomes Measurement Information
67
System (PROMIS) initiative.12 We sought to focus on environmental factors that people with
68
disabilities perceive as facilitating participation or acting as barriers. We purposely focused on
69
the subjective experiences of environmental factors, which are best captured by self-report,
70
rather than objective environmental descriptions. These issues are summarized in the second
71
paper of this special issue.13_ENREF_13 Thus, the objectives of this manuscript are to describe: (1)
72
methods used to operationalize environmental factors; (2) the results of a research project to
AC C
EP
64
5
ACCEPTED MANUSCRIPT Environmental Factors Item Development develop measures of environmental factors that affect participation; and (3) an initial item set of
74
facilitators and barriers to participation. The goal of the larger project of which this study is a
75
component is to develop, test, and evaluate measures of environmental factors and their
76
influence on participation for persons with stroke, TBI, and spinal cord injury (SCI). A validated
77
set of environmental factor instruments will allow us to determine to what extent they influence
78
social health and participation.
80
SC
Methods
79
RI PT
73
We used a mixed-methods approach to constructing and evaluating items. The major tasks, described below, entailed concept elicitation, item classification and creation, and item
82
refinement. These tasks and procedures follow the standards established by the PROMIS
83
Network.12 A domain chair and co-chair oversaw the development of each item set. We obtained
84
Institutional Review Board approval for this project prior to the start of this project.
85
Phase I: Concept Elicitation, Qualitative Focus Group Analysis, and Conceptual Modeling
86
As part of our ongoing research on participation, we had access to verbatim transcripts
87
from 38 focus groups involving over 200 people with diverse disabilities across different sites
88
and regions.14-16 The samples included people with stroke, TBI, SCI, and other disabilities. We
89
analyzed these data to create participation and quality of life assessments such as the Community
90
Participation Indicators, and the SCI-QOL/TBI-QOL measurement systems.16-18 In the focus
91
groups, participants stressed the importance of environmental factors influencing participation;
92
the depth and detail of environmental factor information allowed us to reanalyze the data for the
93
current project. We used a grounded theory approach and constant comparative approach to
94
coding and interpreting._ENREF_1719 This analysis yielded rich environmental factor themes,
95
detailed examples, and relevant quotes to highlight specific factors, across the groups. We
AC C
EP
TE D
M AN U
81
6
ACCEPTED MANUSCRIPT Environmental Factors Item Development categorized the feedback in eight distinct environmental factor domains identified as important
97
by people with disabilities. The second column of Table 1 lists the domains. Some of these
98
domains directly correspond to ICF typology shown in column 1 (e.g., systems and services);
99
others focus on specific components of ICF environmental factor chapters (e.g., transportation,
RI PT
96
assistive technology), and another represents an issue that is an infrequent focus of research,
101
economic quality of life. We grouped social supports and societal attitudes together as a social
102
environmental domain.
103
Phase II: Item Classification and Creation
We binned items assigning them to domains, using expert opinion. Thus, we winnowed
M AN U
104
SC
100
items, removing items with similar content, using expert opinion. We created a working
106
definition of each thematic code based on focus group data, review of legacy assessments, and
107
environmental factor research. We used this guide to categorize and generate sample items
108
within each domain.
109
Phase III: Item Refinement
110
TE D
105
Cognitive interviews. Following PROMIS’ qualitative item review process,12 we conducted cognitive interviews with community-dwelling people with disabilities to ensure
112
comprehension of items. Participants completed a subset of up to 50 items. We asked
113
participants to explain what the question meant in their own words and to describe how they
114
arrived at the answer. Interviewers probed to clarify the participants’ interpretation of specific
115
words and phrases. Each interview lasted 20 to 40 minutes. Most participants volunteered to
116
complete 2 to 3 interviews, requiring 1 to 4 hours in total. Each item was reviewed by at least 4
117
and up to 9 participants who had different disabilities. The project team worked to ensure that
118
members of each diagnostic group reviewed items in each subset to minimize sampling bias.
AC C
EP
111
7
ACCEPTED MANUSCRIPT Environmental Factors Item Development 119
Participants received an honorarium and accommodations as necessary. Accommodations
120
included increasing time, discussing confusion regarding item content, and providing breaks, as
121
needed. Interviewers took verbatim notes during the interviews and reviewed them with the team.
123
We coded responses and domain chair and co-chairs reviewed them. The research team reviewed
124
and discussed problematic items. Revised items underwent a second round of cognitive testing
125
with an additional 3 participants. Once the research team reached consensus on the suitability of
126
the revised items, we deemed the items ready for pilot testing.
SC
Reading Level Analysis: Patient literacy is a well-documented challenge for use of self-
M AN U
127
RI PT
122
administered questionnaires.20-23 Although we designed the items to be self-administered, it is
129
essential to verify that respondents comprehend their wording and meanings are clear. High
130
literacy expectations, complex terms, and ambiguous items can create comprehension problems
131
and impair test validity. We assessed the reading level of each item using the Flesch-Kinkaid and
132
Lexile Framework24 and reworded items that were above a fifth grade level. Maintaining a fifth
133
grade reading level maximizes the scale’s accessibility to participants with low reading skills. Translatability Review: In anticipation of future translation of items to Spanish, we
EP
134
TE D
128
followed PROMIS Guidelines for Translation and Cultural Adaptation.25 This process involves
136
forward and backward translations of items, response options, and instructions. We followed an
137
iterative process of forward and back-translation with bilingual expert review.
138
AC C
135
Results
139
Literature Review
140
We searched PsychLit, PubMed, and CINAHL using the terms “environment,” “social
141
environment,” and “environmental factors” as a keyword and identified articles that also
8
ACCEPTED MANUSCRIPT Environmental Factors Item Development contained the term “social participation” or the keyword “participation.” We reviewed the
143
matching articles and focused on those that reported development of instruments for medical,
144
vocational, or mental health rehabilitation.
145
Domain Description
146
RI PT
142
Table 1 lists the initial eight domains, the number of items we pooled and binned from the literature review and the qualitative review, the count of winnowed items and the final set of
148
items that underwent cognitive testing. After binning and winnowing items, we revised the
149
conceptual framework. From an initial set of 2,273 items across domains, we winnowed this set
150
to 291 items for cognitive testing. Table 2 provides examples of item content. We phrased each
151
item as a simple, declarative statement that is either positively- or negatively-stated. Given the
152
several ways in which we characterized environmental features, we used five types of rating
153
scales as shown in Table 3, including two variations of a frequency rating scale, one indicating
154
capability using a dichotomous environmental attribution response option and the other
155
consisting of a five-point magnitude rating scale.
TE D
M AN U
SC
147
Domain 1, Access to Information and Technology, pertains to knowledge and
157
information technologies. Information technology includes conventional devices and technology
158
to transmit and receive information such as mobile or landline phones, computers, email and
159
internet services. Usability of information includes factors that influence the ability to access and
160
understand information, and includes considerations of reading level, literacy, usability,
161
transparency, and information finding. While this domain does not include specialized
162
communication or devices, it does include built-in, commercially available access features such
163
as Microsoft control panel accessibility features.
AC C
EP
156
9
ACCEPTED MANUSCRIPT Environmental Factors Item Development 164
Domain 2, Assistive Technology (AT), includes the use and availability of specialized devices to facilitate participation in daily activities, such as mobility, personal care, work,
166
leisure, and home participation. These devices are typically specialized and not part of standard
167
built environments. AT belongs to an individual and may be portable. Our definition of AT does
168
not include features of the built environment in public spaces, universal design features like
169
ramps on public buildings and Braille signage. People use AT to assist their performance across
170
a variety of functional tasks. We developed parallel items for mobility, personal care, vision and
171
hearing, and communication AT. Participants answer a filter question, “Do you use any mobility
172
devices, such as a cane, walker, scooter, or wheelchair?” If the answer is affirmative, they name
173
the device they use most often and report on the availability and reliability of the device and its
174
effect on participation. Participants who report that they need a device, but do not currently have
175
it, answer a follow-up question on barriers to accessing AT. Participants who report that they do
176
not need a device skip to the next section.
SC
M AN U
TE D
177
RI PT
165
Domain 3, Built and Natural Environment, pertains to the natural and built factors in the environment, including architectural features of buildings; land development such as
179
sidewalks and roadways; environmental features such as noise, crowds, indoor air quality; and
180
cognitive and sensory cues. Built factors include aspects of neighborhood and community
181
livability such as safety, lighting, and access to community resources like stores, and banks.
182
Natural features of the environment include climate and weather conditions, preparedness for
183
natural disasters, and the means for addressing weather, climate, and emergency conditions. This
184
domain includes geographical and topographical features of the environment such as outdoor air
185
quality. We wrote items in pairs; the first item assesses “how much difficulty do you have”
186
performing an activity and the second item asks respondent if they attribute barriers to the
AC C
EP
178
10
ACCEPTED MANUSCRIPT Environmental Factors Item Development environment such as precipitation or noise. For example, the first item of one pair asks “How
188
much difficulty do you have feeling safe due to crime in your community?” The second item
189
asks respondents to indicate “yes” or “no” whether “The difficulty I have feeling safe is due to a
190
high crime rate, inadequate number of police, or no emergency call boxes.
191
RI PT
187
Domain 4, Systems, Services, and Policies, pertains to social services, employment, educational, housing, independent living, health care policies, and systems of delivery. It
193
includes services provided by non-profit, voluntary and community agencies. Two subdomains
194
emerged pertaining to economics and transportation. The economic subdomain assesses access to
195
coverage of work and educational systems and resources, as well as to alternative disability and
196
income subsidy programs, workers’ compensation, unemployment, and others systems such as
197
food stamps, child care, and affordable housing. The transportation subdomain pertains to public
198
and private travel, including paratransit and adapted transit systems, and access, availability,
199
affordability, and quality of these services. The items are introduced with the statement: “Many
200
systems and services can affect participation at home and in the community. For each category
201
below, tell us how these services and systems affect your participation.”
TE D
M AN U
SC
192
Domain 5, Social Environment, includes content from the ICF Support and
203
Relationships and Attitudes domains. Following the ICF model, we sought to measure both
204
Social Support and Attitudes. To measure Social Support, we adopted as legacy instruments the
205
PROMIS Social Support item pools, which assess the availability of companionship,
206
informational, instrumental, and emotional support. The PROMIS Social Support item pools
207
have been tested in large samples of English- and Spanish-speaking adults and are intended for
208
use across chronic illness, disability, and general population samples. For the current project we
209
created an Attitudes item set specific to disability by selecting and, when necessary, modifying
AC C
EP
202
11
ACCEPTED MANUSCRIPT Environmental Factors Item Development items from existing instruments (e.g., FABS/M Section 6-Community Environment: Services &
211
Attitudes; Stigma Scale for Chronic Illness26) and writing new items covering content identified
212
by our qualitative data and input from content area experts. In keeping with the ICF, we included
213
both positive and negative attitudes about disability. Therefore, item content includes
214
marginalization, stigmatization, oppression, and discrimination as well as inclusion, acceptance,
215
respect, and fairness. Also like the ICF, items cover both the attitudes of individuals such as
216
health care providers, as well as society as a whole. We excluded the attitudes of respondents
217
themselves, reflecting self-stigma, because it does not constitute an environmental factor. The
218
fourth article of this special issue describe the psychometric properties of this item pool in
219
greater detail.27
M AN U
SC
RI PT
210
Domain 6, Economic Quality of Life, is based on the empirical results of consistent
221
feedback from focus group participants regarding the importance of economic assets, and the
222
effects of these factors on participation. The ICF taxonomy code for “financial aspects” (e1650)
223
is related to this construct. This domain is critical to individuals’ quality of life as well as serving
224
as a barrier to participation; it reflects a distinctive construct that few environmental instruments
225
measure.28,29 This domain focuses on how financial resources influence satisfaction with one’s
226
living situation, adequacy and affordability of health services, adequate and affordable food,
227
affordable community recreational activities, and family and friend financial assistance. The fifth
228
article of this special issue describe the psychometric properties of this item pool in greater
229
detail.30
230
Cognitive Interviews
231 232
AC C
EP
TE D
220
We recruited 15 people with stroke (n=4), TBI (n=5), or SCI (n=6) to complete cognitive interviews by telephone. The mean age was 46 years; women comprised 33% of the sample.
12
ACCEPTED MANUSCRIPT Environmental Factors Item Development Participants included African-Americans (54%), Non-Hispanic Caucasians (33%) and Hispanics
234
(13%). While 80% had completed some post-secondary education, only 13% was employed;
235
others were retired (40%), seeking employment (20%) or unemployed and not seeking
236
employment (27%). The majority was ambulatory (53%) though 47% used a walking device and
237
47% used a wheelchair. Participants reviewed 50 of the 291 items; 4 to 12 participants examined
238
each item. For each newly developed item, we asked participants follow-up questions to
239
ascertain comprehension, opinions about the suitability of the items, and revisions. We used this
240
feedback to delete 17 items and revise 57 others. Revisions included narrowing and specifying
241
the focus of concepts that were too broad; reducing the length of items; clarifying the meaning of
242
ambiguous items; and reducing potential ceiling or floor effects. Table 2 provides examples of
243
the revisions.
M AN U
SC
RI PT
233
Discussion
245
We began this project by reviewing the social sciences, disability and rehabilitation
TE D
244
literature and identifying a large pool of items from a diverse literature describing environmental
247
factors for medical, vocational, developmental, and psychiatric rehabilitation applications. It
248
quickly became evident that this topic has generated considerable measurement interest across
249
varied human service arenas, but that existing instruments such as FABS/M were narrow in
250
scope while others pertain to nearly everyone. None of the instruments that we reviewed used
251
contemporary psychometric methods of instrument development; all relied on classical test
252
theory methods or reported descriptive, item-level information. Therefore, we proceeded to
253
develop a new measurement system that would describe and assess environmental factors. As
254
such, an immediate challenge was to reduce the multiplicity of approaches and items to a
255
conceptually concise and clinically practical set of items. We began with the ICF’s taxonomy of
AC C
EP
246
13
ACCEPTED MANUSCRIPT Environmental Factors Item Development environmental factors and distinguished subcomponents of some ICF domains. In the case of the
257
Built and Natural Environment, we realized that these are inseparable aspects of the physical
258
world. Consequently, five of the six domains correspond closely to the ICF taxonomy of
259
environmental factors; the sixth domain, economic quality of life, reflects an important construct
260
that reflects the resources individuals have that affect their participation.
RI PT
256
Cognitive interviewing provided us with feedback that allowed us to reduce the 291 draft
262
items to 274. The number of items per domain ranges from 14 (AT) to 82 (Social Attitudes). The
263
AT items are used as a set to evaluate the use of mobility devices, equipment to assist with
264
activities of daily living, and devices to interact with people and environments. Depending on an
265
individual’s use of technology, a respondent would likely complete no more than 45 of the total
266
274 items. Feedback from persons living with the consequences of disability was critically
267
important to reduce ambiguity and wordiness, and enhance clarity and improve response
268
distributions. The next step in transforming these item sets into functional item pools is to
269
administer the items to a diverse group of individuals living with stroke, SCI and TBI. We will
270
use methods developed by the PROMIS project to explore the dimensionality of each domain
271
using confirmatory factor analysis and item fit to models based on item response
272
theory._ENREF_2431 We envision developing computer adaptive tests and short form versions to
273
measure each domain to evaluate the influence of environmental factors on participation.
274
Study Limitations
M AN U
TE D
EP
AC C
275
SC
261
Results from this study are limited by the number of participants in the focus groups, and
276
the number and diversity of persons who participated in the cognitive interviews. While we
277
searched for literature in various aspects of rehabilitation, including medical, vocational
278
intellectual and psychiatric, we may have overlooked some sources. Because of time constraints,
14
ACCEPTED MANUSCRIPT Environmental Factors Item Development we relied on focus groups that we conducted in earlier projects; these groups addressed multiple
280
aspects of participation and environmental factors. The geographic sampling was limited to
281
Colorado, Illinois, and New Jersey and drew participants primarily from urban and suburban
282
communities. The consequences of under representation of persons from rural settings are
283
unknown. The persons completing cognitive interviews were from metropolitan Chicago; the
284
extent to which an urban, Midwestern city limits perspectives on environmental barriers and
285
facilitators is unknown. Future studies should recruit participants from a broader sample of
286
communities. Conclusions
288
M AN U
287
SC
RI PT
279
A fully developed set of environmental factor items will support a variety of research and clinical applications. Clinicians are interested in identifying aspects of the environment that are
290
potentially modifiable; their interventions could focus on problematic environmental factors that
291
limit participation and life satisfaction. Clinicians and researchers are interested in acquiring a
292
fuller knowledge of how environmental factors affect participation and quality of life. Disability
293
advocates could use more specific and detailed information about environmental factors to lobby
294
for elimination of barriers. The domain definitions and items that operationalize the domains will
295
allow us to evaluate these environmental factor influences in a variety of applications.
AC C
EP
TE D
289
15
ACCEPTED MANUSCRIPT RRTC on Improving Measurement of Medical Rehabilitation Outcomes
R3. Environmental Factor Measures
References
7.
8.
9.
10.
11.
12. 13.
14.
15.
16.
17.
RI PT
SC
6.
M AN U
4. 5.
TE D
3.
EP
2.
Resolution adopted by the General Assembly of the United Nations. The Standard Rules on the Equalization of Opportunities for persons with Disabilities. 48th Session, agenda item 109. 1994. National Disability Rights Network. National Disability Rights Network. 2012; http://www.ndrn.org/index.php. Accessed April 13, 2013. World Health Organization. International Classification of Functioning, Disability, and Health (ICF) 2001:234-237. Field MJ, Jette A. The future of disability in America. National Academies Press; 2007. Whiteneck G, Dijkers MP. Difficult to measure constructs: conceptual and methodological issues concerning participation and environmental factors. Archives of physical medicine and rehabilitation. Nov 2009;90(11 Suppl):S22-35. Alvarelhao J, Silva A, Martins A, et al. Comparing the content of instruments assessing environmental factors using the International Classification of Functioning, Disability and Health. Journal of rehabilitation medicine. Jan 2012;44(1):1-6. Badley EM. Enhancing the conceptual clarity of the activity and participation components of the International Classification of Functioning, Disability, and Health. Social science & medicine. Jun 2008;66(11):2335-2345. Mallinson T, Hammel J. Measurement of participation: intersecting person, task, and environment. Archives of physical medicine and rehabilitation. Sep 2010;91(9 Suppl):S29-33. Noreau L, Boschen K. Intersection of participation and environmental factors: a complex interactive process. Archives of physical medicine and rehabilitation. Sep 2010;91(9 Suppl):S44-53. Hollingsworth H, Gray DB. Structural equation modeling of the relationships between participation in leisure activities and community environments by people with mobility impairments. Archives of physical medicine and rehabilitation. Aug 2010;91(8):11741181. Magasi S, Wong A, Wang C, et al. Theoretical FFoundations for the Measurement of Enviromental Factors and Participation among People with Disabilities. Archives of physical medicine and rehabilitation. Unver review. DeWalt DA, Rothrock N, Yount S, Stone AA, Group PC. Evaluation of item candidates: the PROMIS qualitative item review. Medical care. May 2007;45(5 Suppl 1):S12-21. Hammel J, Magasi S, Heinemann AW, et al. Environmental Barriers & Supports to Participation: An Insider Perspective from People with Disabilities. Archives of physical medicine and rehabilitation. Under review. Gray DB, Hollingsworth HH, Stark SL, Morgan KA. Participation survey/mobility: psychometric properties of a measure of participation for people with mobility impairments and limitations. Archives of physical medicine and rehabilitation. Feb 2006;87(2):189-197. Hammel J, Magasi S, Heinemann A, Whiteneck G, Bogner J, Rodriguez E. What does participation mean? An insider perspective from people with disabilities. Disability and rehabilitation. 2008;30(19):1445-1460. Carlozzi NE, Tulsky DS, Kisala PA. Traumatic brain injury patient-reported outcome measure: identification of health-related quality-of-life issues relevant to individuals with traumatic brain injury. Archives of physical medicine and rehabilitation. Oct 2011;92(10 Suppl):S52-60. Heinemann AW, Magasi S, Bode RK, et al. Measuring enfranchisement: Importance and control of participation by people with disabilities. Archives of physical medicine and rehabilitation. in press.
AC C
1.
16
ACCEPTED MANUSCRIPT RRTC on Improving Measurement of Medical Rehabilitation Outcomes
23.
24. 25.
26.
27.
28.
29.
30.
31.
RI PT
SC
22.
M AN U
21.
TE D
20.
EP
19.
Tulsky DS, Kisala PA, Victorson D, et al. Developing a contemporary patient-reported outcomes measure for spinal cord injury. Archives of physical medicine and rehabilitation. Oct 2011;92(10 Suppl):S44-51. Strauss AL, Corbin JM. Basics of qualitative research : techniques and procedures for developing grounded theory. 2nd ed. Thousand Oaks: Sage Publications; 1998. Cella D, Clauser SB, Flynn KE, et al. Standardizing patient-reported outcomes assessment in cancer clinical trials: A PROMIS initiative. Journal of Clinical Oncology. 2007;25:5106-5112. Hammel J, Magasi S, Heinemann A, Whiteneck G, Bogner J, Rodriguez E. What does participation mean? An insider perspective from people with disabilities. Disability and Rehabilitation. 2008;30(19):1445-1460. Magasi S, Hammel, J., Heinemann, A.W., Whiteneck, G., Corrigan, J., Bogner, J. Participation: Multiple stakeholder's constructions, priorities and comparisons. Under review. Hahn EA, Cella D. Health outcomes assessment in vulnerable populations: measurement challenges and recommendations. Archives of physical medicine and rehabilitation. Apr 2003;84(4 Suppl 2):S35-42. Stenner A, Horabin, I., Smith, D., Smith, M. The Lexile framework. The Lexile framework. Durham, NC: Metametrics; 1998. PROMIS® Instrument Development and Psychometric Evaluation Scientific Standards. 2012; http://www.nihpromis.org/Documents/PROMIS_Standards_050212.pdf. Accessed February 11, 2013. Rao D, Choi SW, Victorson D, et al. Measuring stigma across neurological conditions: the development of the stigma scale for chronic illness (SSCI). Quality of life research : an international journal of quality of life aspects of treatment, care and rehabilitation. Jun 2009;18(5):585-595. Garcia SF, Hahn EA, Magasi S, et al. Development of Self-Report Measures of Social Attitudes that Act as Environmental Barriers and Facilitators for People with Disabilities. Archives of physical medicine and rehabilitation. Under review. Reinhardt JD, Post MW. Measurement and evidence of environmental determinants of participation in spinal cord injury: A systematic review of the literature. Topics in Spinal Cord Injury Rehabilitation. 2010;15(4):26-48. Escorpizo R, Graf S, Marti A, et al. Domain sets and measurement instruments on participation and environmental factors in spinal cord injury research. American Journal of Physical Medicine & Rehabilitation. 2011;90(11):S66-S78. Tulsky DS, Kisala PA, Lai JS, Carlozzi N, Hammel J, Heinemann A. Developing an Item Bank to Measure Economic Quality of Life for Individuals with Disabilities. Archives of physical medicine and rehabilitation. Under review. Reeve BB, Hays RD, Bjorner JB, et al. Psychometric evaluation and calibration of health-related quality of life item banks: plans for the Patient-Reported Outcomes Measurement Information System (PROMIS). Medical care. May 2007;45(5 Suppl 1):S22-31.
AC C
18.
R3. Environmental Factor Measures
17
ACCEPTED MANUSCRIPT
Table 1. Comparison of ICF and Project-Specific Conceptual Domains Cognitive Interviews Item Count
Pilot Test Item Count
15 (5)*
14 (5)*
36
36
90
82
76
75
32
25
37
37
Total 2,273 392 291 *AT items are repeated for mobility device, ADL equipment, and devices used to interact with people and environments. There was also 1 item developed to assess barriers to these devices.
274
Winnowed Items
178
104
79
37
Assistive technology
Natural Natural environment Environment and human Built made changes environment
Services, Systems and Policies
Social supports and attitudes Services, systems, policies Transportation Access to information and technology
710
91
411 136 112
42
Assistive technology
Built and natural environment
Social environment
35
28
37
35
Services, systems, policies Access to information and technology Economic Quality of Life
AC C
EP
Economicfinancial
51
TE D
Support and Relationships Attitudes
605
Revised
RI PT
Binned Items
SC
Products and Technology
Initial Framework
M AN U
ICF Taxonomy
1
ACCEPTED MANUSCRIPT
Table 2. Examples of Item Revisions Item Stem Domain
Access to information and technology
Excessive wordiness
EP
AC C
SC
Assistive Technology
New Item Broken into two items: In case of a health emergency, I can get the information I need easily. In case of a natural disaster, I can get the information I need easily.
RI PT
‘Emergency’ is too broad of a concept
Infrequently used device category
Websites are available in a format I can understand if I need them.
Do you use any equipment or devices to help you communicate and interact with people; to see, hear, or remember things; or to control your environment? I can use my in a variety of places.
Assistive Technology
Confusion with “settings”
Built and Natural Environment
Suggested rephrase to specify crime
How much difficulty do you have feeling safe due to crime in your community?
Built and Natural Environment
Suggested rephrase, edit: “buildings” and add ‘emergency’ to be more specific
How much difficulty do you have feeling safe in stores during an emergency?
Suggested rephrase
My community offers support groups I can use.
Suggested rephrase
Help to make my home accessible i available if I need it.
Confusion over the word: “sensitive”
The people in my life understand the challenges I face because of my disability.
TE D
Websites with information are available in a format I can understand if I need them. Do you use any communication devices, such as a voice synthesizer or communication board? I can use my in a variety of settings. How much difficulty do you have feeling safe outdoors in your community? How much difficulty do you have feeling safe in buildings in your community? My community offers self-help and social support groups. Help modifying my home to make it accessible is available if I need it. The people in my life are sensitive to the challenges I face because of my disability. Health care providers understand the needs of people with disabilities. How do you believe your financial situation is today as compared to other people your age? I have access to emergency funds.
Access to information and technology
M AN U
In case of a health emergency, I can get the information I need easily
Reason for Revision
Services, Systems and Policies Services, Systems and Policies
Social Environmental
Social Environmental Economic Quality of Life Economic Quality of Life
Ambiguity about ”health care providers” Difficult comparison; idiosyncratic rating scale (worse vs. same vs. better) Confusing
Health care professionals understand the needs of people with disabilities. Item eventually dropped
I have access to extra money in case of an emergency.
2
ACCEPTED MANUSCRIPT
Economic quality of life
Capability (difficulty)
Built and natural environment
M AN U
Yes – No
SC
Frequency
Environmental Attribution Built and natural environment Assistive technology
AC C
EP
TE D
Magnitude
Response Options Never Rarely Sometimes Usually Always Never Rarely Sometimes Often Always None A little Somewhat A lot Complete difficulty Yes No Not at all A little bit Somewhat Quite a bit Very much
RI PT
Table 3. Item Response Options Category Domain Frequency Access to Information and Technology Social Environment Systems, Services, Policies
3
