Authors: Thomas N. Bryce, MD Marcel P. Dijkers, PhD Allan J. Kozlowski, PhD, BSc (PT)

Usability Framework

Affiliations: From the Icahn School of Medicine at Mount Sinai, New York, New York.

ANALYSIS

Correspondence: All correspondence and requests for reprints should be addressed to: Thomas N. Bryce, MD, Department of Rehabilitation Medicine, Icahn School of Medicine at Mount Sinai, 5 E 98th St, Box 1240B, New York, NY 10029-6574.

Framework for Assessment of the Usability of Lower-Extremity Robotic Exoskeletal Orthoses

Disclosures: There is no funding for this analysis and no financial benefits to the authors related to the material presented. This material has not been presented or published in any form previously. Financial disclosure statements have been obtained, and no conflicts of interest have been reported by the authors or by any individuals in control of the content of this article.

0894-9115/15/9411-1000 American Journal of Physical Medicine & Rehabilitation Copyright * 2015 Wolters Kluwer Health, Inc. All rights reserved. DOI: 10.1097/PHM.0000000000000321

ABSTRACT Bryce TN, Dijkers MP, Kozlowski AJ: Framework for assessment of the usability of lower-extremity robotic exoskeletal orthoses. Am J Phys Med Rehabil 2015;94:1000Y1014. Persons with neurologic conditions such as spinal cord injury, stroke, and multiple sclerosis often lose the ability to stand and walk. Robotic hip-knee-ankle-foot exoskeletal orthoses have become commercially available and may allow some of these people to stand and walk again. These devices may also have applications beyond mobility, such as exercise, amelioration of secondary complications related to lack of ambulation, and the promotion of neuroplasticity. The authors present a framework for assessment of the usability of robotic exoskeletal orthoses available now or in the near future. The framework contains six modules: Functional applications, Personal factors, Device factors, External factors, Activities, and Health outcomes. Metrics and measures are suggested for each framework factor. Key Words: Conditions

Exoskeletons, Spinal Cord Injury, Robotics, Orthoses, Plasticity, Secondary

T

here are millions of individuals living with spinal cord injury (SCI), stroke, multiple sclerosis, and other neurologic conditions worldwide who are unable to stand or walk, even with the assistance of conventional walking aids and traditional orthoses, compelling them to be full-time wheelchair users. Robotic hip-knee-ankle-foot orthoses (HKAFOs), variously also called (powered) exoskeletons or exoskeletal devices, which are essentially lower-extremity orthoses with computer-controlled motors or actuators, have developed rapidly over the last decade because of the advancement in various technologies, including robotics, sensors, actuators, miniature computers, and control system software. Several models have now reached the market, offering people with neurologic conditions opportunities to walk. Powered exoskeletons have potential for a number of applications: in early rehabilitation to support gait training and facilitate neuroplasticity1; in late rehabilitation and community living as an exercise modality to promote physical, mental, and social wellness; and for mobility as a wheelchair alternative. Early reports describe use for locomotor training with persons with SCI2 and stroke.3 Safety and feasibility of walking have been reported for persons with SCI by multiple investigators,4Y9 including for exercise.9Y11 Regardless of these reported benefits, persons

1000

Am. J. Phys. Med. Rehabil. & Vol. 94, No. 11, November 2015 Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.

with complete or incomplete neurologic impairment of the legs may want to use an exoskeleton to achieve household and community mobility, just for the experience of walking. Usability of assistive technology has been described as the interaction between the user, the device, the context or environment, and nature of the activities in which the user engages with the device.12,13 Different stakeholdersVincluding the developers and manufacturers of robotic HKAFOs; the (potential) users and their families, friends, and co-workers; individual service providers; vendors; payers; prescribing clinicians; researchers; and policy makersVmay have different perspectives on the utility (as well as cost-benefits) of robotic HKAFOs. Conceptual models that sufficiently describe the scope of such interactions for prospective device users provide a basis for measurement of usability.12,13 Such frameworks have been described for wheelchair usability14,15 but, to the authors_ knowledge, have not been developed for lowerextremity orthoses of any kind. Given the rate at with HKAFO technology is entering the market, formulation of a usability framework for robotic HKAFOs specific to current or near future technology is warranted. Such a framework could guide efficient information collection and thoughtful decisions to match individuals to robotic HKAFOs, allowing people to ambulate and perform tasks upright. A model described by Fuhrer et al.13 to guide thinking about optimizing use of assistive technology devices is relevant here. It includes five major components: (1) the functional problem a device is intended to impact; (2) critical device features responsible for addressing the problem; (3) characteristics of individuals (personal factors) that make them successful device users; (4) other elements not intrinsic to the device but closely linked, for example, access to the technology; and (5) expected changes in user status, especially those alterations related to continued and regular use of the device. These components have been incorporated into a usability assessment framework for robotic HKAFOs, the Framework of Usability for Robotic Exoskeletal Orthoses (FUREO), which was designed to incorporate information that is relevant to all stakeholders. FUREO is organized into six modules: Functional applications, Personal factors, Device factors, External factors, Activities, and Health outcomes. In the following, metrics are suggested for each FUREO factor and outcome to facilitate and standardize the collection of data that can be used to compare robotic devices and to make decisions rewww.ajpmr.com

garding (1) who fits a device physically; (2) who can learn to use the device, for what purposes, and how fast; (3) what are the short-term and medium-term effects (psychologic, physiologic, social, other) of continued use; (4) what are the monetary and other costs of use, short-term and long-term; and (5) for whom does purchase and training make a good investment, given the balance of long-term costs vs. functional and other outcomes. Because most of exoskeletal walking research to date has been done with individuals with SCI, and the authors_ own experience is limited to this group, the particular metrics used in some instances are specific to this groupVfor instance, the use of the SCI-QoL Depression measure16 to quantify depressive symptoms. In applications of FUREO to persons with other conditions such as stroke or multiple sclerosis, measures specific to those groups or generic measures (e.g., the Neuro-QoL Depression measure17,18) could be substituted.

Potential Functional Applications of Use of Robotic HKAFOs In assessing assistive technology devices, including robotic HKAFOs, it is essential to identify the functional problems they are intended to impact and their potential solutions, which may be characterized as the functional applications of the device (Table 1). This can vary by individual stakeholder perspective as well as by phase of recovery of individuals with new-onset impairment; for example, neural plasticity may be more active in persons with new neurologic injuries. Moreover, an individual device may have multiple functional applications. As might be expected, different devices have different characteristics and manufacturer specifications of intended use. It should be noted also that governmental regulation of medical devices, by the Food and Drug Administration in the United States and CE Mark in Europe, which include these robotic units, can influence the manufacturer plan for introducing devices by limiting the indication permitted for marketing. If a particular design is approved by a TABLE 1 Functional applications of robotic HKAFOs Functional Application Community-level mobility Household-level mobility Exercise Ambulation training Other use:

Usability of Robotic Exoskeletal Orthoses Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.

1001

regulatory agency to be used for a particular indication, for example, use with persons with SCI who have a neurologic level of injury between, for instance, T12 and C8 or individuals with stroke, competitors may limit their applications for approval of their devices using this precedent to improve their chances of approval. After approval is obtained for this particular indication, regulatory approvals for additional indications may be sought. As of 2014, in the United States or Europe, there were five robotic HKAFOs already on the market or near to being on the market as approved medical devices: the Ekso19 by Ekso Bionics, the ReWalk20 by ReWalk Robotics, the Rex21 by Rex Bionics, the Hybrid Assistive Limb (HAL)22 by Cyberdyne, and the Indego23 by Parker Hannifin. These robotic HKAFOs fall into two main categories: not self-supporting devices, which require use of upper-extremity supports for balance (crutches, walker, or similar simple technologies) by the user (ReWalk, Ekso, HAL, and Indego), and fully self-supporting devices, which require no upperextremity support (Rex). The Ekso is primarily designed by the manufacturer to be used in a rehabilitation setting under supervision of specially trained physical therapists, as is the HAL. The others have both institutional use and personal use versions, the latter presumably for use in the home and community. Nevertheless, stakeholders can change the intended use of the manufacturer (and regulatory agency) by disregarding the manufacturer_s advice. These may include the clinicians and researchers who prescribe the device or study the equipment in clinical settings. If a specific device, although intended for home use, is found to promote, in a cost-effective manner, gait improvements in those who are nonambulatory or near ambulatory, it may be used in this fashion in a rehabilitation setting despite it being marketed for home use as a wheelchair alternative, that is, off-label use. Ultimately, within the constraints of one_s ability to use a device, the end user will determine the functional problems a robotic HKAFO will impact, especially if it is used in an unregulated setting. Just as motorized wheelchairs are commonly used in rough environments that they were not designed to operate

in, once in the possession of an individual, a robotic HKAFO may be used in environments for which it was neither designed nor approved, such as outdoors. In addition to potentially facilitating personal mobility, HKAFOs may also function as exercise devices either to promote neurologic recovery with patterned walking or to provide fitness training. If a physiologic training effect is desired, it is likely that a device that incorporates features such as variable motor assist and/or functional electrical stimulation (FES) would be more effective, purportedly by allowing titration of exercise intensity, than an exoskeleton without these features. The Ekso, Indego, and HAL all have variable-assist features, differing in individual design, that allow users who have some motor function (in one or both legs) to use their own residual strength to walk, supplementing the device motors. In contrast, a self-stabilized robotic HKAFO such as Rex is unlikely to provide any significant physiologic training benefit. Similarly, devices that incorporate FES would likely provide the maximum benefit for maintaining or even improving bone health, as the greatest known effect on bone density arises from weight bearing plus the physiologic stresses caused by muscle contraction. There is an Indego prototype that incorporates FES.

Personal Factors Fit Within Device The ability to fit into a robotic HKAFO is of course critical to its use. Fit, being a component of the interaction between the individual and the device, can be seen as a characteristic either of the device or of the individual who wishes to use it. Basic comparisons of device fit, grouped here under personal factors, are shown in Table 2, whereas a more comprehensive list of personal factors contained within FUREO is shown in Table 3.

Other Body Characteristics Spasticity can provide resistance to the motors and can inhibit the function of the devices and volitional control. Severe spasticity can be a contraindication to use. Conversely, spasticity may become less pronounced with ongoing exoskeletonassisted walking.

TABLE 2 Personal factors for different robotic HKAFO models Device Height limits, cm Weight limit, kg

1002

Bryce et al.

Ekso19

ReWalk20

Rex21

HAL22

Indego23

150Y190 G100

160Y190 G100

146Y193 G100

145Y185 G85

155Y190 G125

Am. J. Phys. Med. Rehabil. & Vol. 94, No. 11, November 2015 Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.

TABLE 3 FUREO personal factors module Factor

Metric

Fit within deviceVmodifiable and unmodifiable factors

Muscle excitability (as it relates to applicability of functional electrical stimulation)

Trunk stability

Neurologic level of injury and severity

Spasticity

1. Pelvic width, cm 2. L&R thigh length, cm 3. L&R shank length, cm 4. Leg length discrepancy, cm 5. L&R ROM hip extension, degrees to degrees 6. L&R ROM hip flexion, degrees to degrees 7. L&R ROM knee flexion, degrees to degrees 8. L&R ROM plantar flexion ankle, degrees to degrees 9. L&R ROM dorsiflexion ankle, degrees to degrees 10. Body weight, kg 11. Height, cm 12. Other:___________________________ 1. Trunk (present, absent) 2. L&R hip extensors (present, absent) 3. L&R hip flexors (present, absent) 4. L&R knee extensors (present, absent) 5. L&R knee flexors (present, absent) 6. L&R ankle dorsiflexors (present, absent) 7. L&R ankle plantar flexors (present, absent) 8. Other:_____________________________________ 1. Seated Berg Balance Scale24Y27 Modified Functional Reach Test24,28 2. Computerized posturography29 3. Other:________________________________ International Standards for the Neurological Classification of Spinal Cord Injury30 1. ASIA Impairment Scale 2. Neurologic level of injury 1. L&R Modified Ashworth31,32 Scale for hip flexion 2. L&R Modified Ashworth Scale for hip extension 3. L&R Modified Ashworth Scale for knee flexion 4. L&R Modified Ashworth Scale for knee extension 5. L&R Modified Ashworth Scale for ankle plantar flexion33 6. Other:____________________________________

L&R, left and right; ROM, range of motion.

Trunk stability and volitional movement seem to be determinants of whether someone can use robotic HKAFO designs that are dependent on specific upper-extremity trunk movements to trigger stepping. The designs that require active trunk flexion/extension and lateral bending, such as the ReWalk and Ekso, are more difficult or even impossible to use by those who lack this motor control. Devices that can be triggered by trunk flexion in the sagittal plane alone to initiate stepping, such as the Indego, or require no trunk motion at all, such as the Rex, have been used successfully by persons with higher levels of SCI.

Device Factors Device Components Robotic HKAFOs all include external support skeletons, electric motors for the lower-extremity joints, and batteries incorporated into various segments to supply electricity to the motors and the computer that controls the whole. A comparison www.ajpmr.com

of various device factors for the different robotic HKAFOs is provided in Table 4, whereas a more comprehensive list of proposed device factors with recommended metrics is presented in Table 5. Currently, all the devices have exoskeletal components that extend laterally from the pelvis and hip of the user. This lateral bulk, which varies among the devices, practically limits where one can sit in them; most hip/pelvic components preclude sitting in one_s own wheelchair. The exoskeletons also can be bulky, as they contain various electrical components, batteries, and motors, and may or may not fit under clothing. Currently, all five are designed to be worn on top of clothing. Ankle-foot orthoses are incorporated directly into all five designs, externally for Ekso and Rex, within-shoe for ReWalk and Indego, and integrated into a shoe for HAL and as another option for ReWalk. In addition to the standard externally powered motors, the Indego in the future may have capability for constant FES of the trunk, combined with Usability of Robotic Exoskeletal Orthoses

Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.

1003

TABLE 4 Device factors for different robotic HKAFO models Ekso19

Device

ReWalk20

Rex21

HAL22

Indego23

Yes Yes No Yes Yes Requires upper-extremity support for balance (walker or crutches) Weight of device, kg 23 20 38 15 12.5 Battery life per charge, mins 240 240 120 90 240 Motors at joints Hip, knee Hip, knee Hip, knee, Hip, knee Hip, knee (externally powered) ankle 1 1 1 1 5 HKAFO modularity (breaks down into separate pieces easily) Designed with adequate power No Yes No No Yes and a control algorithm to ascend/descend stairsa Remote Joystick Remote Remote Types of step trigger modes Remote control control control control control initiated, initiated, initiated, initiated, body body body body movement movement movement movement initiated initiated initiated initiated Forward Movement Hand Forward trunk Body movements required for Lateral and trunk of leg in movement excursion to forward trunk stepping (with body excursion stepping of joystick initiate stepping excursion to movement triggering pattern and lateral reach preset modes) shift to clear targets in the swing foot both planes to initiate stepping FES capabilitya No No No No Yes Variable-assist capability Yes No No Yes Yes a

Devices are not Food and Drug Administration approved for stair climbing or for FES.

gait-cycled stimulation of knee and hip extensors. HAL, in contrast, uses electromyographic biofeedback coupled to the external motors at the hip and knee joints to facilitate gait.

Ability to Reconfigure Device Components The ability to reconfigure a device can affect feasibility of use in a clinical or other setting that relies on multiple-user access. The Ekso can be reconfigured for all fit characteristics (leg length, thigh length, hip width, crutch length, etc.) in approximately 5 mins. The ReWalk, in contrast, requires 5Y10 mins to reset the leg and thigh lengths and can require up to 30 mins if the pelvic band (adjustment for hip width) and the foot plates (adjustment for shoe size) also have to be reconfigured.

Body Device Interfaces The way robotic HKAFOs are physically attached to the user can vary as well. At present, all of the devices use straps, but perhaps future designs will incorporate custom shells that can better optimize interface pressure distribution. This is important for preventing pressure ulceration in

1004

Bryce et al.

persons with sensory dysfunction at surface interfaces, especially if the devices are worn for extended periods of time, as would be the case if they are used as wheelchair alternatives. Molded shells might be custom-fabricated by orthotists similar to what has evolved for nonmotorized orthoses. Alone among the four devices that require upper-extremity support, the Ekso incorporates a rigid back component (containing the batteries and computer) firmly attached to the pelvic section and connected to and stabilizing the upper thorax, with shoulder straps that effectively provide increased trunk stability. Such stability is not provided by the ReWalk, which has a backpack containing batteries and the computer, which is not rigidly attached, or the Indego, which has the batteries and computer contained within the pelvic component. The Indego does, however, include posterolateral supports of different heights, the most supportive of which can extend just below the axilla, which can increase trunk stability. Both Hartigan et al.8 and Kozlowski et al.9 have shown that having this increased trunk support extends the range of individuals with SCI who can use an exoskeleton to include those with cervical levels of injury.

Am. J. Phys. Med. Rehabil. & Vol. 94, No. 11, November 2015 Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.

TABLE 5 FUREO device factors module Factor

Metric

Noise level of device when in use Type of upper-extremity assistive device required for level surface ambulation

Decibels 1. Platform walker (yes/no) 2. Standard walker (yes/no) 3. Standard wheeled walker (yes/no) 4. Forearm crutches (bilateral) (yes/no) 5. Forearm crutches (unilateral) (yes/no) 6. Four-point cane (unilateral) (yes/no) 7. Four-point cane (unilateral) (yes/no) 8. Straight cane (bilateral) (yes/no) 9. Straight cane (bilateral) (yes/no) 10. Straight cane (unilateral) (yes/no) 11. None (yes/no) 12. Other: ____________________ Type of upper-extremity support 1. Forearm crutches (bilateral) (yes/no) required for using stairs 2. Forearm crutches (unilateral) (yes/no) 3. Four-point cane (unilateral) (yes/no) 4. Four-point cane (unilateral) (yes/no) 5. Straight cane (bilateral) (yes/no) 6. Straight cane (bilateral) (yes/no) 7. Straight cane (unilateral) (yes/no) 8. Railing (bilateral) (yes/no) 9. Railing (unilateral) (yes/no) 10. None (yes/no) 11. Other: ____________________ Step/stair climbing capability 1. Single (yes/no) 2. Multiple in succession (yes/no) 3. None (yes/no) 4. Other: ____________________ Types of controllers for triggering movement 1. Motion sensing (yes/no) 2. Joystick (yes/no) 3. Instrument panel (yes/no) 1. Wireless (yes/no) 2. Touch screen (yes/no) 3. Internet connected (yes/no) 4. Other:_________________ 4. Other:___________________________ Control system autonomy 1. None (operator has complete control of the movements of the actuators) 2. Supervisory (operator specifies general position changes and device decides specific movements of actuators) 3. Task level (operator specifies task only, e.g., walk, step, etc. and device performs task) 4. Other:_____________________________________________ Control system operation 1. User controlled (yes/no) 2. Clinician controlled (yes/no) 3. Both user and clinician controlled (yes/no) 4. Other:______________________________________ Motor torque (peak) 1. Hip, Nm 2. Knee, Nm 3. Ankle, Nm Maximal achievable speed for ambulation m/sec (device dependent) Maximal achievable speed for sit to stand secs (device dependent) Maximum curbed ramp slope degrees (device dependent) Maximum linear ramp slope degrees (device dependent) Maximum walking surface camber degrees (device dependent) Maximum step height (device dependent) cm Minimum step depth (from edge to riser) cm (device dependent) (continued on next page)

www.ajpmr.com

Usability of Robotic Exoskeletal Orthoses Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.

1005

TABLE 5 (Continued) Factor

Metric

User device interfaces

HKAFO modularity Battery

1. Straps 1. Shoulder (yes/no) 2. Chest (yes/no) 3. Abdomen (yes/no) 4. Pelvis (yes/no) 5. Upper thigh (yes/no) 6. Lower thigh (yes/no) 7. Upper shank (yes/no) 8. Lower shank (yes/no) 9. Other: _____________________ 2. Custom-molded interface components 1. Trunk (yes/no) 2. Pelvis (yes/no) 3. Thigh (yes/no) 4. Shank (yes/no) 5. Foot (yes/no) 6. Other: ___________________________ 3. Other_______________ 1. Modular components (yes/no) 2. Number of components:________________________ 1. Battery life per charge for intermittent use, hrs 2. Battery life per charge for continuous walking at comfortable speed, hrs 3. Battery life per charge for continuous walking at maximal speed, hrs 4. Time needed for charging from empty to fully charged, hrs 5. Size of batteryVlength  width  height, cm 6. Time needed to switch in a new battery, if replaceable, mins 7. Other:______________________ Length  width  height, cm

Minimum size of device for storage and/or transport (when not in use) Weight of device Weight of device, kg Durability 1. Expected service calls per year, n 2. Expected episodes where device must be left with vendor for repair per year, n Ease of maintenance Repairable by local vendor (yes/no) Need for trained assistance 1. Physical therapist (yes/no) 2. Trained companion (yes/no) 3. Other:_____________________

Exertion Required for Use The Ekso and ReWalk can be used with only a mild to moderate effort, the degree of exertion varying somewhat by the nature of one_s neurologic impairment. Using the Borg scale,34,35 Kozlowski et al.9 reported ratings of perceived exertion during walking ranging from 8 to 15 (where 6 represents no exertion and 20 represents maximal exertion) for seven individuals with SCI using the Ekso, whereas Spungen et al.6 reported average ratings of perceived exertion of 15 during the first five sessions for individuals using ReWalk, which decreased to 8 after 40 sessions. This is in contrast to the effort and energy consumption required to ambulate with conventional nonmotorized knee-ankle-foot orthoses, which are significant for persons with SCI and often lead to abandonment. As a self-balancing robotic HKAFO, the Rex likely does not have any significant exercise

1006

Bryce et al.

effect. Devices with a variable-assist feature, which would allow titration of user effort, and devices that incorporate FES likely require intermediate levels of exercise effort while these features are being used.

Aesthetic Design Aesthetic design and form are also critical factors in long-term acceptance and use of assistive technology. If a device is pleasing to the eye, it is more likely to be accepted and used. On the other hand, if an appliance is noisy, it is more liable to be discarded than one that is quiet, unless the noise provides a clearly recognized control or safety functionVlike the beeping of a car in reverse gear.

Component Characteristics Battery characteristics such as life, size, weight, duration and ease of recharging, and ease of swapping will impact where one can go with the device.

Am. J. Phys. Med. Rehabil. & Vol. 94, No. 11, November 2015 Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.

Battery life will vary depending on the demand placed on the motors, for example, more demand if an individual has significant lower-extremity spasticity and less if the person assists with movements. Although weight is not a concern for walking, as all of these devices bear their own weight through their footplates, the weight, dimensions, and whether the robotic HKAFO can be broken down into components (Indego) or not (all others) and easily reassembled presumably factor in transporting the device when not in use.

Variability in Power The various models also differ in the power provided by the motors. Devices with less powerful motors may require additional user upperextremity assistance to go from sit to stand and may be more difficult to use independently for those with limited upper-extremity strength. This is also the case when a device is slower in going from sit to stand. Anecdotally, the Ekso requires more upperextremity assistance in coming from sit to stand than do the ReWalk and Indego, whereas the Rex requires no upper-extremity assist. To allow stair climbing, in addition to having the appropriate software control system, the motors also need to be able to generate and react to large forces when the user is descending and ascending steps, as the power required for these tasks significantly exceeds what is needed for ambulating on level surfaces. Devices that have adequate power for level walking may not generate the forces required for stair ascent and descent. The power needed for running and jumping would be substantially greater yet, and none of the current devices have this capability.

Speed Speed of movement is likely one of the most significant contributors to device use and acceptance; it varies widely among the models. For those robotic HKAFOs that depend on the users contin-

uously changing the position of their body in space, for example, Ekso, ReWalk, and Indego, speed can vary significantly with the skill of the user, but it is also limited by device design. The speed of the Rex is fully device dependent. At the current time, the maximal achievable speed of all of the devices does not exceed 0.8 m/sec and is usually much less, especially during the training period. Kozlowski et al.9 reported a mean speed of 0.15 m/sec for seven individuals using the Ekso, whereas Spungen et al.6 described a mean speed of 0.28 m/sec for seven individuals using the ReWalk; both were reported as the highest speed the participants produced. Talaty et al.,36 in reporting the walking speeds of 12 individuals with SCI using the ReWalk, noted a wide variation in walking speeds from 0.1 m/sec to 0.5 m/sec, not explained by injury level. This large variability was also seen by Spungen et al.6

Accommodation of Environments The environments for which robotic HKAFOs have been designed also vary widely. For instance, the devices can differ in the step heights that can be accommodated for stair and curb ascent and descent, as well as in the surface camber on which they can safely be used.

External Factors Other factors that affect the use of robotic HKAFOs for mobility and/or exercise and need to be addressed in purchase decisions include financial resources, access, vendors for the devices, rehabilitation facilities that have experience with robotic HKAFOs and their capacity to provide training, and trainers skilled in instructing individuals to use the devices. A comparison, for the available devices, of a few basic external factors is given in Table 6, whereas a more comprehensive list of proposed external factors is provided in Table 7, along with recommended metrics.

TABLE 6 External factors for different robotic HKAFO models Device Regulatory approvalV institutional usea Regulatory approvalV personal useb Approximate cost, $

Ekso19

ReWalk20

FDA, CE Mark

FDA, CE Mark

None

FDA, CE Mark

120,000Y150,000

70,000Y85,000

Rex21

HAL22

Indego23

FDA, CE Mark CE Mark

CE Mark

None

None

None

Not available in the United States

Not available in the United States

None

a

Adjustable to fit multiple users. Custom fit to the individual. FDA, Food and Drug Administration. b

www.ajpmr.com

Usability of Robotic Exoskeletal Orthoses Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.

1007

TABLE 7 FUREO external factors module Factor

Assessment Tool/Measurement

Cost

Availability of device

Availability of training

Availability of repair service

1. 2. 3. 4. 5. 6. 7. 8. 9. 1. 2. 3. 4. 5. 1. 2. 3. 4. 5. 1. 2. 3. 4.

Cost of device to purchase, $ Cost of device to rent per day/month/year, $ Cost of training, $/hr Minimal training sessions for basic mobility Minimal training sessions for advanced mobility Typical number of training sessions for basic mobility Typical number of training sessions for advanced mobility Cost of ongoing personal assistance after training is complete, $/hr Cost (other):___________________ Covered by insurance with justification to purchase (yes/no) Covered by insurance with justification to rent (yes/no) Available through philanthropic funding (yes/no) Available only under research protocol (yes/no) AvailableVother:_______________________ Covered by insurance (yes/no) Covered by philanthropic funding (yes/no) Available only under research protocol (yes/no) Not available (yes/no) AvailableVother:_____________________ Covered by insurance (yes/no) Covered by philanthropic funding (yes/no) Not available (yes/no) AvailableVother:_____________________

Financial resources are necessary to procure a device for personal use, whereas devices must be widely available to those who are potential candidates, both for trial use and training and for ongoing use. Repair services must also be available to fix the devices should they break down. Perhaps similar to the model of wheelchair clinics staffed by individuals who specialize in seating and positioning and maintain Assistive Technology Professional certification, robotic orthosis clinics may be a viable model in the future, staffed by rehabilitation professionals with an equivalent expertise. An environment in which training of candidate users takes place must be available and affordable as well.

Activities Training There is a certain amount of time that is needed for a user to learn to use a robotic exoskeleton safely and effectively. The length of this period, labeled introductory use by Fuhrer and referred to as training period here, is dependent on the functional outcome expected, for example, ambulation on level surfaces vs. stair climbing, as well as the target level of assistance to be received. It will take less time to be able to ambulate with minimal assistance than to ambulate independently, and less time to ambulate on level surfaces with independence than to traverse uneven ground, or to climb stairs. The duration of the training period is dependent on the learning

1008

Bryce et al.

capacity of an individual, as some persons are able to pick up new skills faster than others, and (in SCI) on completeness and neurologic level of injury. Other factors, such as time since injury, may even be more significant than the level of injury in SCI, with time since injury perhaps being a proxy for learned shoulder girdle and trunk motor control patterns. (In other diagnostic groups such as multiple sclerosis and stroke, parallel severity measures are available to operationalize physiologic capability.) As of now, the authors have no measures of a trainer_s competence, but as with learning of all motor skills, the quality of instruction and coaching can be expected to make a difference in the level of outcomes and the speed with which they are attained. Lastly and probably most significantly, the length of the training period is dependent on the ease of use of each robotic HKAFO_s design. Devices that are more intuitive and simpler to use have shorter training periods. Spungen et al.6 reported that it took users of the ReWalk an average of 10Y15 sessions to walk independently and 15Y25 sessions to be able to ascend stairs with moderate assist, whereas Kozlowski et al.9 reported that most users required 5Y11 sessions to use the Ekso to walk with minimal assistance, but that they required 8Y28 sessions to walk with contact guard only or close supervision. These differences may be influenced by the recommended training protocols from the two companies (e.g., Ekso encourages

Am. J. Phys. Med. Rehabil. & Vol. 94, No. 11, November 2015 Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.

www.ajpmr.com

Usability of Robotic Exoskeletal Orthoses

Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.

1009

Ambulate on carpet Assistive device used for ambulation on carpet Speed of ambulating on carpet Ambulate on rough surface such as asphalt or cement Assistive device used for ambulation on rough surface Speed of ambulating on rough surface Ambulate on grass Assistive device used for ambulation on grass Speed of ambulating on grass Ascend and descend 6µ curb Speed for ascending and descending 6µ curb Ascend and descend standard ADA specification 4.8-degree grade ramp Speed to ascend and descend standard ADA specification 4.8-degree grade ramp Stair ascent and descent

Ambulation on smooth tile surface Assistive device used for ambulation on smooth tile Speed of ambulation on smooth tile

TransfersVsit to stand Time for sit to stand transfer

Ease of donning/doffing deviceVon floor Time for donning/doffing deviceVon floor

Ease of donning/doffing deviceVin chair Time for donning/doffing deviceVin chair

Activity

TABLE 8 FUREO activities module

90.3 m/sec Supervision

90.6 m/sec Independent

10-m walk, secs Lowest level assistance neededa

Walker 90.3 m/sec Supervision Walker 90.3 m/sec Dependent Any Dependent Any Dependent

Walker 90.6 m/sec Independent Walker 90.6 m/sec Independent G30 secs Independent G10 secs Independent

10-m walk, secs Lowest level of assistance neededa Type of devicec 10-m walk, secs Lowest level of assistance neededa Speed for ascending and descending a 6µ curb Lowest level of assistance neededa Speed to ascend and descend a standard ADA 4.8-degree grade ramp of 3 m Lowest level of assistance needed to ascend and descend 10 stepsa

Type of devicec

Supervision Walker

Independent Walker

1. 90.3 m/sec

1. 90.6 m/sec

1. 10-m walk, secs 2. Distance covered during 6-min walk, m 3. Timed up and go, secs Lowest level of assistance neededa Type of devicec

Supervision Walker

supervision 1. G15 secs

Independent 1. G5 secs Independent Walker

Dependent Any

Dependent G5 mins

Dependent

Any

Any Dependent Any Dependent

Any Dependent Any

Any

Any Dependent

Dependent Any

Any

Maximal Any

Dependent Any

Dependent Any

Dependent Any

Exercise Use

(continued on next page)

Minimum Threshold for Household Use as a Mobility Device

Independent Any

Independent G5 mins

Minimum Threshold for Community Use as a Mobility Device

Lowest level assistance needed to completea Time to don/doff at lowest level of assistance needed, mins Lowest level of assistance needed to completea Time to don/doff at lowest level of assistance needed, mins Lowest level of assistance neededa 1. Time for sit to stand transfer, secs 2. Timed up and go, secs Lowest level of assistance neededa Type of deviceb

Metric37

1010

Bryce et al.

Am. J. Phys. Med. Rehabil. & Vol. 94, No. 11, November 2015

Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.

Metric37

Any Any Any

Any Independent Independent

Any

Any

Any

Any Any

Any Any

Any

Independent

Independent

G13 G6 mets

Any

G5 mins

G13 G4 mets

Any

b

Minimum Threshold for Household Use as a Mobility Device

Any

Minimum Threshold for Community Use as a Mobility Device

Independent, supervision, contact guard assist, minimal assist, moderate assist, maximal assist, dependent. Parallel bars, platform walker, standard walker, bilateral forearm crutches/canes, single forearm crutch/cane, no device, other. c Platform walker, standard walker, bilateral forearm crutches/canes, single forearm crutch/cane, no device, other. ADA, Americans with Disabilities Act.

a

(Single rail, bilateral rail, single forearm crutch/cane, bilateral forearm crutch/cane, no device, other) Speed of stair ascent and descent Time it takes to ascend and descend 10 steps at lowest level of assistance, secs Unilateral arm reach for object from Lowest level of assistance neededa stand (other hand can be used for support) Bilateral arm reach (simultaneous) Lowest level of assistance neededa Carry object with one hand Lowest level of assistance neededa while ambulating Carry object with both hands Lowest level of assistance neededa while ambulating Energy expenditure For 6-min walk: 1. Ratio Perceived Exertion (6Y22) 2. Metabolic equivalent (HR prediction or metabolic cart measurement if available) Ease of transport while not being Lowest level of assistance neededa worn by user Running Lowest level of assistance neededa Floor/ground-to-chair transfer Lowest level of assistance neededa wearing orthoses Floor/ground-to-stand transfer Lowest level of assistance neededa wearing orthoses

Assistive device used for stairs

Activity

TABLE 8 (Continued)

Any

Any Any

Any

1. Any 2. Any

Any

Any Any

Dependent

Any

Any

Exercise Use

taking steps promptly after standing, whereas ReWalk promotes balance and coordination training in standing before attempting to take steps) and by the skill level of the instructors. A device that does not require any trunk movements or upperextremity support, such as the Rex, theoretically requires less time for users to be able to ambulate independently.

Differences in Operation Most robotic HKAFOs have multiple modes of triggering successive steps, with different levels of complexity and need for control by the trainer or another assistant. For new, less skilled users, the trainers or the walkers themselves use a remote controller to trigger individual steps. In the most advanced programming mode, the devices vary in how body movements are used to automatically trigger successive steps. To advance a limb, the ReWalk user moves forward in the sagittal plane to trigger the next step but also must shift laterally to the stance side to clear the swing foot. The Ekso user must make a lateral and forward shift and reach preset targets in both planes to trigger a step. Persons with poor lateral trunk stability, for which in SCI neurologic level of injury is a fairly good proxy measure, may have difficulty with this and require more assistance than those with good lateral trunk control and stability in bending. The Indego, however, requires only a forward lean to initiate stepping.

Functional Activities Possible In addition to walking, there are many other functional activities that need to be mastered during the training period. The length of time needed to learn these other tasks, which is dependent on the speed of learning and the terminal skill level targeted, also depends on personal characteristics. These functional activities are listed in Table 8, along with recommended metrics and suggested thresholds for describing levels of functioning, that is, community, household, or for exercise only. There are assessments of ability as well as of performance (time required to perform the task), if appropriate. Donning and doffing a device is one such activity that can greatly influence its use. Its modular nature with quick-release connectors allows the donning/ doffing of the Indego in less than 5 mins, whereas the same person may need more time, more assistance, or both to don another brand without these features. Robotic HKAFOs that incorporate FES or biofeedback applied through surface electrodes require additional donning and doffing time, owing to the need to place electrodes underneath clothing, as the current generation of devices is worn over the clothes. www.ajpmr.com

The ability to climb stairs and to step up onto curbs can significantly increase the likelihood of the user participating in a range of activities and accessing different environments. At the present time, the Indego prototype, ReWalk, and Rex have stair and curb climbing algorithms, but to date these features have not been approved for use in the United States. For those using the device as a wheelchair alternative, the ability to get up from the ground, for example after a fall, is an important skill. Rising can be made easier by using a chair as an intermediate stepVif a chair is at hand. Doffing the device while on the ground to allow a floor-to-chair transfer without the device can perhaps facilitate this task, depending on device design. In using robotic HKAFOs, certain thresholds of safe performance of activities need to be met, which are specific to each potential functional application. At the higher levels of function, for example, community ambulation, achieving speeds approaching those of individuals without neurologic impairment is the goal. Although users may have their own cutoff for Bfast enough,[ there are certain tasks that need to be achieved with a minimal speed, such as crossing a street before the traffic light changes, typically requiring a walking speed of 0.6 m/sec. The thresholds for gait speed suggested in the FUREO were derived from the European Multicenter Study about Spinal Cord Injury database, which includes people with SCI who are community ambulators as well as householdonly ambulators, and may not be truly representative of a population using a robotic HKAFO.38 Despite the less-than-community-ambulator speeds currently achievable with robotic HKAFOs, the devices could potentially reduce or eliminate other current barriers to community ambulation, such as user endurance.

Health Outcomes Physical Effects Changes in body function are expected with longer-term use of a robotic HKAFO. These changes may be expected to vary by frequency and duration of powered exoskeleton use as well as in relation to the circumstances of and appropriateness of use (e.g., robotic HKAFO used in environments for which it was not designed or approved). Given that much of the use of robotic HKAFOs takes place in standing with weight bearing through the limbs, changes in bone density may be expected, particularly if device use is combined with FES. Changes in joint range of motion, posture, and spasticity can also be expected with prolonged upright positioning. As ambulating with a robotic Usability of Robotic Exoskeletal Orthoses

Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.

1011

orthosis provides mild to moderate exercise (as measured by heart rate, Borg rating of perceived exertion level, or VO2 uptake), if walking is done routinely with sufficient duration and frequency, the negative consequences of SCI that are seen more commonly in persons who lose the ability to walk than those who do not (such as body composition alterations, increased insulin resistance, diabetes, cardiovascular disease, and difficulty with bowel evacuation) can be expected to be impacted as well. The exercise level could be titrated if variableassist options are used (available for Ekso, Indego, and HAL) as well as FES-assisted walking (Indego). In several investigations, walking in exoskeletons has anecdotally improved bowel function, spasticity, sleep, fatigue, subjective well-being, and pain, even during the training period.4Y7,9 It is likely that, in addition to the purported benefits, greater walking durations and frequencies will lead to an increased risk of pressure ulcerations, abrasions, fractures, and falls. At this time, there is insufficient information available to estimate likelihood and severity of these adverse events, overall or by user characteristics, brand, and/or device applications.

Psychologic Effects Psychologic effects also cannot be underestimated. Learning how to use a robotic exoskeleton and the

actual use of a device during everyday activities require psychologic accommodation. This is typically feedback dependent; a positive experience will generally promote use whereas a negative one will result in the opposite. Short-term and long-term effects on psychologic and physiologic health resulting from continued use are anticipated, and potential outcomes are listed in Table 9, along with suggested metrics.

Use of FUREO Answering the five queries posed in the introduction is straightforward for some of them and less so for others. Some of these questions can be answered at the individual level by simply measuringVfor example, height and weight of the user. However, for most, the field of rehabilitation needs to create an evidence base that incorporates information from many individuals, because outcomes are multiple determined by complex interactions of factors. Most of the answers to these questions require the systematic acquisition and sophisticated analysis of data from numerous subjects, each using one or more exoskeleton models, using metrics that are part of FUREO. At the present time, the minimal amount of time (minimal training period) required for any individual to become capable of using a device safely and effectively is a key unknown in estimating the costs. The final

TABLE 9 FUREO health outcomes module Outcome37

Metric39

Bone density at supracondylar femur Cardiovascular fitness Cholesterol Body composition Glucose intolerance Pressure ulcers Bowel function Depression Mood

Pain Fatigue Spasticity Autonomic function Sleep

1. DEXA 2. qCT Oxygen consumption (VO2) Chol, HDL, LDL, TG DEXA HgA1C International SCI Pressure Ulcer Data Set International SCI Bowel Function Data Set 1. PHQ-9 2. SCI-QoL depression SCI-QoL: Self-esteem Resilience Positive affect International SCI Pain Data Set Neuro-QoLVFatigue 1. Modified Ashworth Scale 2. Penn Spasm Frequency Scale 1. Sit up orthostatic challenge test 2. Blood pressure and heart rate variability 1. PROMIS Sleep DisturbanceVShort Form 2. Neuro-QoLVSleep Disturbance

DEXA, dual-energy x-ray absorptiometry; qCT, quantitative computed tomography; Chol, cholesterol; HDL, high density lipoprotein; LDL, low density lipoprotein; TG, triglycerides; HgA1C, hemoglobin A1C; PHQ-9, patient health questionnaire-9; SCIQoL, Spinal Cord Injury multidimensional Quality of Life; Neuro-QoL, quality of life in neurological disorders measurement system; PROMIS, Patient Reported Outcomes Measurement Information System.

1012

Bryce et al.

Am. J. Phys. Med. Rehabil. & Vol. 94, No. 11, November 2015 Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.

question on cost-benefit ratios is the hardest, but the systematic longitudinal acquisition of data for multiple persons with varied characteristics using a framework such as FUREO is one way of maximizing the chances of proving that robotic HKAFOs are effective in allowing a certain segment of the millions who currently are full-time wheelchair users to ambulate.

Conclusion The FUREO modules may be used at an individual level to help guide the clinical prescription of robotic HKAFOs and to predict the functional applications of a given robotic exoskeleton for a particular person. Additionally, the collection and analysis of FUREO data for groups of individuals can provide insight into the resources associated with training and the level of independence that persons with a range of impairments can attain with robotic HKAFOs. Within the Activities module, FUREO includes suggested thresholds for describing levels of functioning and mobility (community, household, or exercise only) yet to be validated, which can help predict functional outcome. Such data on varied samples of users also can provide insight as to what features should be modified in future generations of robotic exoskeletons. REFERENCES 1. Ferris DP, Sawicki GS, Domingo R: Powered lower limb orthoses for gait rehabilitation. Top Spinal Cord Inj Rehabil 2005;11:34Y49 2. Kubota S, Nakata Y, Eguchi K, et al: Feasibility of rehabilitation training with a newly developed wearable robot for patients with limited mobility. Arch Phys Med Rehabil 2013;94:1080Y7 3. Watanabe H, Tanaka N, Inuta T, et al: Locomotion improvement using a hybrid assistive limb in recovery phase stroke patients: A randomized controlled pilot study. Arch Phys Med Rehabil 2014;95:2006Y12 4. Zeilig G, Weingarden H, Zwecker M, et al: Safety and tolerance of the ReWalk exoskeleton suit for ambulation by people with complete spinal cord injury: A pilot study. J Spinal Cord Med 2012;35:96Y101

8. Hartigan C, Kandilakis C, Dalley S, et al: Mobility outcomes following five training sessions with a powered exoskeleton. Top Spinal Cord Inj Rehabil 2015;21:93Y9 9. Kozlowski AJ, Bryce T, Dijkers M: Time and effort required by persons with spinal cord injury to learn to use a powered exoskeleton for assisted walking. Top Spinal Cord Inj Rehabil 2015;21:110Y21 10. Spungen A, Asselin P, Fineberg D, et al: Beneficial changes in body composition after exoskeletal-assisted walking: Implications for improved metabolic function. Top Spinal Cord Inj Rehabil 2013;19:8Y9 11. Evans N, Hartigan C, Kandilakis C, et al: Acute cardiorespiratory and metabolic responses during exoskeleton-assisted walking overground among persons with chronic spinal cord injury. Top Spinal Cord Inj Rehabil 2015;21:122Y32 12. Arthanat S, Bauer SM, Lenker JA, et al: Conceptualization and measurement of assistive technology usability. Disabil Rehabil Assist Technol 2007;2: 235Y48 13. Fuhrer MJ, Jutai JW, Scherer MJ, et al: A framework for the conceptual modelling of assistive technology device outcomes. Disabil Rehabil 2003;25:1243Y51 14. Arthanat S, Nochajski SM, Lenker JA, et al: Measuring usability of assistive technology from a multicontextual perspective: The case of power wheelchairs. Am J Occup Ther 2009;63:751Y64 15. Routhier F, Vincent C, Desrosiers J, et al: Mobility of wheelchair users: A proposed performance assessment framework. Disabil Rehabil 2003;25:19Y34 16. Tulsky D, Kisala P, Heinemann A, et al: Validation of the SCI-QOL and SCI-FI Measurement Systems for Persons with SCI. Top Spinal Cord Inj Rehabil 2013;19:33 17. Cella D, Lai JS, Nowinski CJ, et al: Neuro-QOL: Brief measures of health-related quality of life for clinical research in neurology. Neurology 2012;78:1860Y7 18. Cella D, Moy C: Neuro-QOL: Quality of Life in Neurological Disorders. Available at: http://www.neuroqol.org/Pages/ default.aspx. Accessed October 22, 2014 19. Ekso Bionics. Available at: http://www.eksobionics.com/. Accessed September 28, 2014 20. ReWalk Robotics. Available at: http://rewalk.com/. Accessed September 28, 2014 21. Rex Bionics. Available at: http://www.rexbionics.com/. Accessed September 28, 2014

5. Esquenazi A, Talaty M, Packel A, et al: The ReWalk powered exoskeleton to restore ambulatory function to individuals with thoracic-level motor-complete spinal cord injury. Am J Phys Med Rehabil 2012;91:911Y21

22. Cyberdyne. Available at: http://www.cyberdyne.jp/ english/. Accessed September 28, 2014

6. Spungen A, Asselin P, Fineberg D, et al: Exoskeletalassisted walking for persons with motor-complete paraplegia. STO-MP-HFM-228 2013;6-1

24. Forrest GF, Lorenz DJ, Hutchinson K, et al: Ambulation and balance outcomes measure different aspects of recovery in individuals with chronic, incomplete spinal cord injury. Arch Phys Med Rehabil 2012;93:1553Y64

7. Kolakowsky-Hayner S, Crew J, Moran S, et al: Safety and feasibility of using the EksoTM bionic exoskeleton to aid ambulation after spinal cord injury. J Spine 2013;S4 www.ajpmr.com

23. Parker Hannifin. Available at: http://www.indego.com/ indego/en/home. Accessed September 28, 2014

25. Wirz M, Muller R, Bastiaenen C: Falls in persons with spinal cord injury: Validity and reliability of the Berg Balance Scale. Neurorehabil Neural Repair 2010;24:70Y7

Usability of Robotic Exoskeletal Orthoses Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.

1013

26. Lemay JF, Nadeau S: Standing balance assessment in ASIA D paraplegic and tetraplegic participants: Concurrent validity of the Berg Balance Scale. Spinal Cord 2010;48:245Y50

33. Mishra C, Ganesh GS: Inter-rater reliability of modified Ashworth scale in the assessment of plantar flexor muscle spasticity in patients with spinal cord injury. Physiother Res Int 2014;19:231Y7

27. Ditunno JF Jr, Barbeau H, Dobkin BH, et al: Validity of the walking scale for spinal cord injury and other domains of function in a multicenter clinical trial. Neurorehabil Neural Repair 2007;21:539Y50

34. Al-Rahamneh HQ, Eston RG: Prediction of peak oxygen consumption from the ratings of perceived exertion during a graded exercise test and ramp exercise test in able-bodied participants and paraplegic persons. Arch Phys Med Rehabil 2011;92:277Y83

28. Lynch SM, Leahy P, Barker SP: Reliability of measurements obtained with a modified functional reach test in subjects with spinal cord injury. Phys Ther 1998;78:128Y33 29. Harel NY, Asselin PK, Fineberg DB, et al: Adaptation of computerized posturography to assess seated balance in persons with spinal cord injury. J Spinal Cord Med 2013;36:127Y33 30. Kirshblum SC, Waring W, Biering-Sorensen F, et al: Reference for the 2011 revision of the International Standards for Neurological Classification of Spinal Cord Injury. J Spinal Cord Med 2011;34:547Y54 31. Craven BC, Morris AR: Modified Ashworth scale reliability for measurement of lower extremity spasticity among patients with SCI. Spinal Cord 2010; 48:207Y13 32. Bohannon RW, Smith MB: Interrater reliability of a modified Ashworth scale of muscle spasticity. Phys Ther 1987;67:206Y7

1014

Bryce et al.

35. Borg GA: Psychophysical bases of perceived exertion. Med Sci Sports Exerc 1982;14:377Y81 36. Talaty M, Esquenazi A, Briceno JE: Differentiating ability in users of the ReWalk(TM) powered exoskeleton: An analysis of walking kinematics. IEEE Int Conf Rehabil Robot 2013;2013:6650469 37. Alexander MS, Anderson KD, Biering-Sorensen F, et al: Outcome measures in spinal cord injury: Recent assessments and recommendations for future directions. Spinal Cord 2009;47:582Y91 38. van Hedel HJ, EMSCI Study Group. Gait speed in relation to categories of functional ambulation after spinal cord injury. Neurorehabil Neural Repair 2009;23:343Y50 39. NINDS CDE Team. NINDS Common Data Elements. Available at: http://www.commondataelements. ninds.nih.gov/SCI.aspx#tab=Data_Standards. Accessed October 4, 2014

Am. J. Phys. Med. Rehabil. & Vol. 94, No. 11, November 2015 Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.

No title

No title - PDF Download Free
170KB Sizes 1 Downloads 5 Views