171
NeuroRehabilitation 35 (2014) 171–179 DOI:10.3233/NRE-141114 IOS Press
Optimising resource management in neurorehabilitation Richard M. Wooda , Jeff D. Griffithsa,∗ , Janet E. Williamsa and Jakko Brouwersb a School
of Mathematics, Cardiff University, Cardiff, UK Hospital, Cardiff, UK
b Rookwood
Abstract. BACKGROUND: To date, little research has been published regarding the effective and efficient management of resources (beds and staff) in neurorehabilitation, despite being an expensive service in limited supply. OBJECTIVE: To demonstrate how mathematical modelling can be used to optimise service delivery, by way of a case study at a major 21 bed neurorehabilitation unit in the UK. METHODS: An automated computer program for assigning weekly treatment sessions is developed. Queue modelling is used to construct a mathematical model of the hospital in terms of referral submissions to a waiting list, admission and treatment, and ultimately discharge. This is used to analyse the impact of hypothetical strategic decisions on a variety of performance measures and costs. The project culminates in a hybridised model of these two approaches, since a relationship is found between the number of therapy hours received each week (scheduling output) and length of stay (queuing model input). RESULTS: The introduction of the treatment scheduling program has substantially improved timetable quality (meaning a better and fairer service to patients) and has reduced employee time expended in its creation by approximately six hours each week (freeing up time for clinical work). The queuing model has been used to assess the effect of potential strategies, such as increasing the number of beds or employing more therapists. CONCLUSIONS: The use of mathematical modelling has not only optimised resources in the short term, but has allowed the optimality of longer term strategic decisions to be assessed. Keywords: Health resources, personnel management, systems theory, operations research, personnel staffing and scheduling, database management systems
1. Introduction The rehabilitation of patients with acquired brain injury is an expensive and time-consuming process. Studies undertaken in the UK suggest a bed-cost per day of approximately £500 (Turner-Stokes, Bill, & Dredge, 2012) and an average length of stay in excess of 3 months (Turner-Stokes, Poppleton, Williams, Schoewenaars, & Badwan, 2012). Not only this, the ser∗ Address
for correspondence: Jeff Griffiths, School of Mathematics, Cardiff University, Senghennydd Road, Cardiff CF24 4AG, UK. E-mail: [email protected].
vices are in high demand, with about 15,000 people each year in the UK requiring specialist rehabilitation having received moderate to severe acquired brain injury (Turner-Strokes, 2003). Owing to these factors, efficient and effective resource management is necessary in order to ensure that an appropriate service is delivered at an acceptable cost. In this paper we discuss how mathematical modelling (operational research) can be used to help achieve this aim, by way of a case study at a major nuerorehabilitation unit in the UK. Operational research is defined as ‘a scientific approach to the solution of problems in the management of complex systems’ (Association of European
1053-8135/14/$27.50 © 2014 – IOS Press and the authors. All rights reserved
172
R.M. Wood et al. / Optimising resource management
Operational Research Societies, 2013). Typically this involves creating a model that is representative of a real-life system and then asking questions of it that cannot be easily answered in the real world. For example, what is the effect of opening more checkout lanes in a supermarket? Of course it is possible to simply build more checkout lanes in order to find out, but this could be a costly error if no improvements are found in practice. Producing a model, however, allows potential scenarios to be evaluated on the grounds of expected performance measures, such as the throughput of customers, the revenue earned, and the cost incurred (more checkout lanes, more employees). Having weighed up the projected costs and benefits, a decision can be made as to what action is taken. In this paper we describe two operational research tools that are relevant to neurorehabilitation, and report on their application at Rookwood hospital (a 21 bed facility in Cardiff, UK). The first – personnel scheduling – involves the use of a computer in producing a treatment timetable by fitting patient demand for therapy sessions to staff supply (availability), for a forthcoming period of time (such as a week or month). This automated approach was first applied to an inpatient rehabilitation setting in (Beggs, Vallbona, Spencer, Jacobs, & Baker, 1971). However, whilst their computer program takes into account patient demand in creating personalised schedules for patients, it fails to acknowledge the availability of therapists and thus requires subsequent alterations by staff. Since this early effort, and despite major advances in personnel scheduling in other areas of healthcare (such as nurse rostering (Burke, De Causmaecker, Berghe, & Van Landeghem, 2004) and ER staffing (Centeno, Giachetti, Linn, & Ismail, 2003)), there has been little advancement regarding rehabilitation. At Rookwood hospital an automated program could have a profound effect in freeing up resources, with approximately 8 hours each week spent producing physiotherapy timetables by hand. The second tool of operational research that is pertinent to neurorehabiltation is queue modelling. A queuing process involves the arrival of something at a queuing facility, the service of that something by one of a number of servers, and the subsequent departure of that something from the system. That something is customers in the earlier example of the supermarket, whereby the servers are represented by assistants at checkout lanes. In an inpatient healthcare setting that something is patients, whose referrals to a unit are submitted to a waiting list until a bed (server)
becomes available, at which point the patient is transferred. Although no research of this kind has been undertaken in the field of neurorehabilitation, queuing systems have been studied for other healthcare facilities, such as intensive care units (Griffiths, Price-Lloyd, Smithies, & Williams, 2006; McManus, Long, Cooper, & Litvak, 2004) and geriatric wards (McClean & Millard, 2006; Taylor, McClean, & Millard, 1998). Studies regarding the latter are particularly relevant due to the similarity with ABI patients in pathway, i.e. accident and emergency followed by specialist hospital ward, rehabilitation and then social services. Rookwood hospital could benefit particularly from queue modelling due to an excess demand for its services; all beds are typically occupied and there always exists a queue (waiting list).
2. Methods 2.1. Personnel scheduling Treatment at an inpatient neurorehabiliation unit is typically provided by a multidisciplinary team, involving physiotherapy, occupational therapy, and speech and language therapy departments amongst others (Greenwood, 1997). Due to substantial temporal variability in patient condition and workforce availability (due to annual leave, sickness etc.), many units employ an approach in which demand for treatment sessions (determined by therapists) is somehow fitted to the availability of staff on a short-term routine basis, such as each week. This is not an easy task since there are typically a range of competing objectives and constraints in making therapy session assignments, particularly if demand outstrips supply. Thus creating treatment timetables by hand can not only be time-consuming but can lead to the production of poor quality timetables. Each week at Rookwood hospital the various departments manually produce (i.e. by hand) treatment timetables for the proceeding week consecutively and in order of size. Physiotherapy is the largest department (approximately nine full-time equivalents) and thus acts after all of the other smaller departments (which have less flexibility) have made their assignments. Obtaining an optimal treatment schedule for this department is most difficult since there is more patient demand and employee availability meaning a greater number of potential session assignments. The problem is compounded by reduced patient availability due to the prior arrangements of the other departments. In order
R.M. Wood et al. / Optimising resource management
to demonstrate how operational research can improve personnel scheduling in this setting, we develop an automated program for timetabling physiotherapy treatment – a process that takes eight hours each week if conducted by hand. But first it is necessary to define how the quality of a schedule can be assessed. For this, consider a standard treatment schedule that details which patient(s) are being seen by which employee(s) at what times and for what type of therapy session. If a patient has two hydrotherapy sessions scheduled for a week and these are both scheduled to occur on Monday morning then clearly this is not ideal in terms of attaining an even spread throughout the week. It would also be of concern if the patient in question had a demand of five hydrotherapy sessions for that week and had a preference for afternoons. These problems can be quantified by assigning a penalty to an objective function in each instance. The magnitude of these penalties reflects the type (e.g. having sessions without the patient’s primary/secondary therapist in attendance is less of a problem than having no sessions at all) and severity (e.g. the more sessions demanded but not scheduled – the larger the penalty) of the problem and the priority of the patient (larger penalties are accrued for patients of higher priority ceteris paribus). This is modelled by applying appropriate weights to these variables, such that the value of the objective function is an accurate proxy for schedule quality. The value of these weights has been quantified following discussion with clinicians as well as observing and critiquing the thought process in creating schedules by-hand. These initial values were fine-tuned in the weeks following the replacement of the by-hand approach with the automated program (January, 2011). The scheduling of physiotherapy treatment is modelled here as a hierarchical multi-objective combinatorial optimisation problem. Basically this means that the aim is to find an optimal arrangement, or combination, of treatment sessions subject to many competing objectives of varying importance (some mentioned above). Due to the size of the problem an exact solution (in which all possible schedules are evaluated to find the optimal) is not possible on a desktop computer in the required timeframe of about 6 hours. Instead, heuristic methods (which are able to find ‘good quality solutions in a limited time’ (Puchinger & Raidl, 2005)) are used. The process is firstly to construct an initial allocation of sessions on a ‘blank’ timetable (containing only current patient and therapist availability) by making assignments one-by-one. This takes the computer only
173
a couple of seconds and once complete, the solution is further optimised by making random moves and swaps of treatment sessions around the timetable, and with demanded sessions that are at the time unscheduled. If the alteration reduces the value of the objective function (i.e. increases schedule quality) then the move/swap is accepted and another is performed, otherwise it is rejected and an alternative move/swap is made. At the time specified by the user, the automated program (coded in VBA due to user familiarity with Excel) will stop working and output the most current and thus best schedule produced so far. For more information on this process, and on the construction of the automated scheduling program, see (Griffiths, Williams, & Wood, 2012b). 2.2. Queue modelling As mentioned earlier, the movement of patients through a neurorehabilitation unit can be modelled as a queuing process. But before we describe the queuing model deduced for Rookwood hospital, it is necessary to understand a bit more about the theory. As laid out in (Kendall, 1953) there are three major determinants of a simple queuing system: the (probability) distribution of inter-arrival times, the distribution of service completion times, and the number of servers. If arrivals at the queuing system are random then an exponential distribution represents the variability in times between successive arrivals. The exponential distribution is also the simplest choice in representing the variability of service completion times. With a specification of the number of servers and the rates of arrival and service completion, results can be conveniently output for performance measures such as queue length, waiting time, and throughput. At Rookwood hospital, however, the queuing process is rather more complex. Whilst the arrival of patient referrals from other units (with rate 2.55 referrals/week) has been shown to be random (using data obtained over a 16 month period), not all referred patients are admitted. Many referrals (61%) are withdrawn from the queue before a bed becomes available (due to transfer, death, etc) in a process called reneging. Also, prospective referrals may be dissuaded by the size of the waiting list and simply not join the queue in the first place (balking), the extent of which is unknown. Once admitted, expected length of stay (service completion time) is not found to be exponentially distributed (from an 8 year dataset available), but is better represented by a more complicated phase-type distribution resembling
174
R.M. Wood et al. / Optimising resource management
Fig. 1. Abstraction of the queuing process at Rookwood hospital.
a right-skewed bell-shape (i.e. similar to a log-normal distribution). To further complicate matters, the shape and magnitude of this distribution is found to be very different for broadly speaking four groups of patients (see Fig. 1). Through inspection of records, patients of these groups can typically expect to occupy 6, 3, 3, and 9 of the 21 beds respectively. As a final complexity, discharge is often delayed due to extraneous factors relating to the discharge destination. The duration of this delay is found to be exponentially distributed with a mean time depending on the patient group. The queuing process for this system is depicted in Fig. 1. This queuing process is modelled here using analytical queuing theory – a rigorous mathematical approach that guarantees the accuracy of the results. But how is this model parameterised? Firstly, the probability distributions pertaining to length of stay are fitted (by maximum likelihood estimation) to the data available for the four patient groups. This involves fitting the phase-type distributions to time from admission to date ready for discharge (Fig. 2 A-D), and the exponential distributions to the further time until ultimate discharge (Fig. 2 E-H), yielding means similar to those of the data (Fig. 1). Since the aggregate arrival rate of referrals is unknown due to balking, this is varied (alongside three parameters relating to the reneging and balking functions) such that the resulting performance measures for the observed arrival rate and reneging probability are
effectively equal to their empirically available counterparts. For the model corresponding to the optimal values, there is only a 0.2% and 0.5% difference in the model and empirical results for these measures. See (Griffiths, Williams, & Wood, 2012a) for a more detailed description of the construction of the queuing system.
3. Results 3.1. Personnel scheduling Before its introduction, the automated scheduling program for physiotherapy treatment at Rookwood hospital was trialled in the latter quarter of 2010. During this time it was used alongside the manual approach, and performance results of both efforts were evaluated (Table 1). It can be seen from these results that there are significant gains in schedule quality (as assessed by objective function value) through using the automated program. Whilst there is no obvious difference in unmet patient demand (this is essentially bounded by staffing levels), other constituents of the objective function have improved substantially, e.g. the assignment of sessions to primary or secondary therapist. Note that such performance measures can be affected by altering the weights of the constituent, for example, if it were
Fig. 2. (A-D). Empirical probability densities for time from admission until ready for discharge (solid line), with phase-type distributions fitted by maximum likelihood (dashed line). Distribution means are 72, 137, 121, and 178 days respectively. (E-H). Empirical probability densities for time from date ready for discharge until ultimate discharge (solid line), with phase-type distributions fitted by maximum likelihood (dashed line). Distribution means are 14, 26, 55, and 77 days respectively.
R.M. Wood et al. / Optimising resource management 175
176
R.M. Wood et al. / Optimising resource management
Table 1 A comparison of performance measures relating to the scheduling of physiotherapy treatment using the manual ‘by-hand’ approach and the automated scheduling program. Results are given for objective function value (lower is better), percent of demanded sessions scheduled (higher is better), and percent of sessions with neither primary/secondary therapist in attendance (lower is better) Week start
20 September 2010
Method Objective function value (normalised) Average percent of demanded sessions scheduled (%) Average percent of sessions with neither primary/secondary therapist (%)
4 October 2010
1 November 2010
By-hand
Program
By-hand
Program
By-hand
Program
1 85 48
0.53 95 28
1 93 37
0.6 90 16
1 76 37
0.48 74 18
Fig. 3. Probability density of the total number of referrals and patients in the system. Table 2 Responsiveness of performance measures and costs to a number of hypothetical scenarios Measure Reneging probability Mean bed occupancy (patients) Annual throughput (patients/year) Mean queue length (referrals) Mean waiting time (days) Annual cost
Original model
Reduce delays to discharge (50% / 100%)
One-third increase in older patients
Three additional therapists
More group sessions
0.62 20.8 51 10 29 £3.64m
0.58 / 0.45 20.7 / 20.4 57 / 60 9/8 26 / 22 £3.62 m / £3.57m
0.65 21 51 11 27 £3.68m
0.6 20.7 53 10 28 £3.78m*
0.59 20.7 54 10 27 £3.62m
*indicates an amendment in how cost is calculated to include a change in workforce composition.
desired that less patients received sessions with neither the primary or secondary therapist in attendance then the corresponding weight is increased (this would however make it harder for sessions to be scheduled and so would unfavourably affect unmet demand). As mentioned in the Introduction, modelling is often used to ask questions that cannot be easily answered in real-life. To this end, we use the automated program to evaluate the effect of changes to physiotherapy treatment assignment on the amount of therapy received by patients. This involves the use of an initial refer-
ence schedule which details typical levels of demand and priorities for the 6, 3, 3, and 9 patients of each group, and typical levels of staff availability. A rather trivial example is an expansion in the number of fulltime equivalents from 9 to 12. In order to deduce the effect of this on the average treatment intensity (which is an important measure that can affect length of stay (Blackerby, 1990)), the automated program is run with typical patient demand levels (for the four patient groups) but a modified staff roster. As would be expected, the treatment intensity increases by one-third
R.M. Wood et al. / Optimising resource management
177
Fig. 4. Responsiveness in performance measures and costs to a change in the capacity of the unit. The left-hand axis represents the probability of reneging, and the right-hand axis represents the annual throughput (in patients), the mean waiting time (in days) and the mean queue length (in referrals).
on average. Another example is a change in session composition to increase the provision of group sessions (which involve a greater patient-therapist ratio than standard sessions). In this case, the staff availability is unchanged but patient demand is modified to include three more group sessions in place of one of the standard sessions for each of the patient groups. Results indicate a 62% increase in average treatment intensity. 3.2. Queue modelling Results of the parameterised queuing model for Rookwood hospital are now presented. The probability density of the total number of referrals and patients in the system is provided in Fig. 3. It can be seen from this that the bed occupancy is nearly always at capacity and that the waiting list can reach up to 20 referrals at times. Interestingly the aggregate demand for the unit is found to be 2.751 referrals/week; suggesting that the length of the waiting list is sufficient to dissuade 8% of prospective referrals (since the actual arrival rate of referrals at the queue is 2.541 per week). The performance measures of the system are detailed in the first column of Table 2. The annual running cost is calculated as the product of mean bed occupancy, bed-cost per day (£480 with current workforce), and 365. From time to time, hospital management may seek to make alterations to aspects of service delivery at a hospital unit. This may be as a result of new guidelines, efficiency drives or budgetary issues. The queuing
model created here has been used to evaluate the effect of a number of relevant scenarios on performances measures and costs. A simple instrument is hospital capacity (i.e. number of beds), which when varied by even a small amount can have a noticeable consequence (Fig. 4). As previously mentioned, the discharge of patients is typically delayed – on average by 14, 26, 55, and 77 days for the four patient groups (Fig. 3). If however this can be reduced by half or cut altogether then the effect is a much more fluid flow of patients through the unit (Table 2). This can be achieved by informing social services of expected patient readiness dates (using onadmission predictors of length of stay, e.g. age, sex, diagnosis, admission source, etc.); such that efforts can be placed in ensuring the discharge destination is ready on time. A particularly relevant scenario to an ageing population is also analysed, in which the arrival rate of referrals from patients over the age of 50 years is increased by one-third. Since patient length of stay is dependent on treatment intensity (as alluded to earlier in Results), this permits the analysis of scenarios involving changes to the variables of the automated scheduling program, e.g. increasing the number of therapists and providing more group sessions (Table 2). But this requires knowledge of how responsive length of stay is to a change in treatment intensity. To quantify this we fit a simple inverse relation (representing a convex curve) between these two variables to the available data from Rookwood hospital for each patient group. Average length
178
R.M. Wood et al. / Optimising resource management
of stay in each group can thus be calculated from the output of the scheduling program (treatment intensity) through these relations. This is used to appropriately scale the probability distributions relating to length of stay for each patient group in the queuing model. This hybrid queuing-scheduling model has also been used to evaluate scenarios relating to redundancy strategies (in light of UK budget cuts) at the behest of senior management. 4. Discussion In this article we describe how mathematical modelling can be used to improve resource management in neurorehabilitation. To this end, we introduce two well-known tools of operational research - personnel scheduling and queue modelling - and exemplify the benefits they can bring to service delivery through a case study at a major unit in the UK. The purpose of this paper is not to provide a detailed account of how these tools are implemented (for this see (Griffiths et al., 2012a, 2012b)), but to provide an overview of how and why they are used and the kind of improvements they can bring. It must also be acknowledged that, whilst many units share similar features, the model created here is designed specifically for Rookwood hospital, and that other units will likely require bespoke approaches. A major component of this investigation has been the construction and introduction of an automated scheduling program for assigning physiotherapy treatment. Not only has this led to considerable increases in timetable quality, but it has also reduced employee time expended in the creation of timetables from eight hours each week to fewer than two (thus freeing up substantial time for clinical work). In addition, data required for audit purposes is automatically output (removing the need for paper records) and there is no longer a burden on the employee tasked with timetabling to cater to the personal requests of therapists. The program has been successfully in use at Rookwood hospital since January 2011 and has enjoyed much praise from its users. For units seeking to introduce a similar scheduling program, note that timetable quality as judged by the user is critically dependent on the objectives provided and the corresponding weights. Queue modelling has also proved to be a valuable asset to Rookwood hospital. This has been through the ability to analyse the impact of relevant hypothetical scenarios on performance measures and costs without having to commit to long-term changes, such as
changing the number of beds. The model has also been used to evaluate scenarios that are, to some extent, outside the control of the hospital, such as an increase in older patients and a reduction in delays to discharge. If, for example, the delay to discharge can be cut by half then the probability of reneging is reduced by 6% and throughput increased by 12% (meaning more patients get the service they require). The accuracy of such predictions is ensured by developing a model that is sufficiently representative of activities on the ground. But with greater representation comes greater complexity, and this increases the difficulty in modelling. In this project we use analytical queuing theory in order to rigidly ensure accuracy, but a far more practical approach, especially for non-mathematicians, is through simulation (using off-the-shelf software like Witness and Simul8). As well as promoting accuracy, a complex model also allows the consideration of a greater assortment of relevant scenarios. In a simple queuing model, the only ‘handles’ are the probability distributions of arrival and service and the number of servers. Even if the model predicts substantial improvements by, say, changing the service (length of stay) distribution from exponential to log-normal, this is useless information for policy-makers without the ability to say how this can be achieved. But if such a change could be shown to be brought about by changing the composition of the workforce (through the scheduling program) then the model predictions would be useful to hospital management who are able to influence such decisions. Thus appropriate model complexity is necessary to give users relevant ‘handles’ on the kind of scenarios that can be evaluated. An example of this that has been considered is using more group sessions than one-to-one therapy. But in this particular example there lies a problem that the model lacks the complexity to comprehend. Whilst results show that under this scenario treatment intensity increases and thus length of stay reduces, the model fails to take account of the quality of treatment and thus patient outcome. Future research in this field should investigate the trilateral relationship between treatment intensity, length of stay and outcome at discharge.
Acknowledgments The authors would like to acknowledge the help and support of all staff at Rookwood hospital involved in this project, with particular thanks to Jenny Thomas and Angelo Lamberti. This study would not have
R.M. Wood et al. / Optimising resource management
been possible without their sustained efforts and contributions.
Declaration of interest None. References Association of European Operational Research Societies (2013). What is OR? Retrieved September 9, 2013, from http://www.euroonline.org/web/pages/197/what-is-or Beggs, S., Vallbona, C., Spencer, W. A., Jacobs, F. M., & Baker, R. L. (1971). Evaluation of a system for on-line computer scheduling of patient care activities. Computers and Biomedical Research, 4(6), 634-654. Blackerby, W. F. (1990). Intensity of rehabilitation and length of stay. Brain Injury, 4(2), 167-173. Burke, E. K., De Causmaecker, P., Berghe, G. V., & Van Landeghem, H. (2004). The state of the art of nurse rostering. Journal of scheduling, 7(6), 441-499. Centeno, M. A., Giachetti, R., Linn, R., & Ismail, A. M. (2003). Emergency departments II: A simulation-ilp based tool for scheduling ER staff. In Proceedings of the 35th conference on Winter simulation: Driving innovation (pp. 1930-1938). Retrieved from http://dl.acm.org/citation.cfm?id=1031086 Greenwood, R. (1997). Neurological rehabilitation. Psychology Press. Griffiths, J. D., Price-Lloyd, N., Smithies, M., & Williams, J. (2006). A queueing model of activities in an intensive care unit. IMA Journal of Management Mathematics, 17(3), 277-288. Griffiths, J. D., Williams, J. E., & Wood, R. M. (2012a). Modelling activities at a neurological rehabilitation unit. European Journal of Operational Research. Retrieved from http://www. sciencedirect.com/science/article/pii/S0377221712008028
179
Griffiths, J. D., Williams, J. E., & Wood, R. M. (2012b). Scheduling physiotherapy treatment in an inpatient setting. Operations Research for Health Care. Retrieved from http://www. sciencedirect.com/science/article/pii/S2211692312000379 Kendall, D. G. (1953). Stochastic processes occurring in the theory of queues and their analysis by the method of the imbedded Markov chain. The Annals of Mathematical Statistics, 338-354. McClean, S., & Millard, P. (2006). Where to treat the older patient? Can Markov models help us better understand the relationship between hospital and community care? Journal of the Operational Research Society, 58(2), 255-261. McManus, M. L., Long, M. C., Cooper, A., & Litvak, E. (2004). Queuing theory accurately models the need for critical care resources. Anesthesiology, 100(5), 1271-1276. Puchinger, J., & Raidl, G. R. (2005). Combining metaheuristics and exact algorithms in combinatorial optimization: A survey and classification. In Artificial intelligence and knowledge engineering applications: A bioinspired approach (pp. 41-53). Springer. Retrieved from http://link.springer.com/chapter/ 10.1007/11499305 5 Taylor, G. J., McClean, S. I., & Millard, P. H. (1998). Using a continuous-time Markov model with Poisson arrivals to describe the movements of geriatric patients. Applied Stochastic Models and Data Analysis, 14(2), 165-174. Turner-Stokes, L., Bill, A., & Dredge, R. (2012). A cost analysis of specialist inpatient neurorehabilitation services in the UK. Clinical Rehabilitation, 26(3), 256-263. Turner-Stokes, L., Poppleton, R., Williams, H., Schoewenaars, K., & Badwan, D. (2012). Using the UKROC dataset to make the case for resources to improve cost-efficiency in neurological rehabilitation. Disability and Rehabilitation, 34(22), 1900-1906. Turner-Strokes, L. (2003). Rehabilitation following acquired brain injury: National clinical guidelines. Royal College of Physicians. Retrieved from http://www.rcplondon.ac.uk/sites /default/files/documents/rehabilitation-followingacquired-bra in-injury.pdf
Copyright of NeuroRehabilitation is the property of IOS Press and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.
NeuroRehabilitation 35 (2014) 171–179 DOI:10.3233/NRE-141114 IOS Press
Optimising resource management in neurorehabilitation Richard M. Wooda , Jeff D. Griffithsa,∗ , Janet E. Williamsa and Jakko Brouwersb a School
of Mathematics, Cardiff University, Cardiff, UK Hospital, Cardiff, UK
b Rookwood
Abstract. BACKGROUND: To date, little research has been published regarding the effective and efficient management of resources (beds and staff) in neurorehabilitation, despite being an expensive service in limited supply. OBJECTIVE: To demonstrate how mathematical modelling can be used to optimise service delivery, by way of a case study at a major 21 bed neurorehabilitation unit in the UK. METHODS: An automated computer program for assigning weekly treatment sessions is developed. Queue modelling is used to construct a mathematical model of the hospital in terms of referral submissions to a waiting list, admission and treatment, and ultimately discharge. This is used to analyse the impact of hypothetical strategic decisions on a variety of performance measures and costs. The project culminates in a hybridised model of these two approaches, since a relationship is found between the number of therapy hours received each week (scheduling output) and length of stay (queuing model input). RESULTS: The introduction of the treatment scheduling program has substantially improved timetable quality (meaning a better and fairer service to patients) and has reduced employee time expended in its creation by approximately six hours each week (freeing up time for clinical work). The queuing model has been used to assess the effect of potential strategies, such as increasing the number of beds or employing more therapists. CONCLUSIONS: The use of mathematical modelling has not only optimised resources in the short term, but has allowed the optimality of longer term strategic decisions to be assessed. Keywords: Health resources, personnel management, systems theory, operations research, personnel staffing and scheduling, database management systems
1. Introduction The rehabilitation of patients with acquired brain injury is an expensive and time-consuming process. Studies undertaken in the UK suggest a bed-cost per day of approximately £500 (Turner-Stokes, Bill, & Dredge, 2012) and an average length of stay in excess of 3 months (Turner-Stokes, Poppleton, Williams, Schoewenaars, & Badwan, 2012). Not only this, the ser∗ Address
for correspondence: Jeff Griffiths, School of Mathematics, Cardiff University, Senghennydd Road, Cardiff CF24 4AG, UK. E-mail: [email protected].
vices are in high demand, with about 15,000 people each year in the UK requiring specialist rehabilitation having received moderate to severe acquired brain injury (Turner-Strokes, 2003). Owing to these factors, efficient and effective resource management is necessary in order to ensure that an appropriate service is delivered at an acceptable cost. In this paper we discuss how mathematical modelling (operational research) can be used to help achieve this aim, by way of a case study at a major nuerorehabilitation unit in the UK. Operational research is defined as ‘a scientific approach to the solution of problems in the management of complex systems’ (Association of European
1053-8135/14/$27.50 © 2014 – IOS Press and the authors. All rights reserved
172
R.M. Wood et al. / Optimising resource management
Operational Research Societies, 2013). Typically this involves creating a model that is representative of a real-life system and then asking questions of it that cannot be easily answered in the real world. For example, what is the effect of opening more checkout lanes in a supermarket? Of course it is possible to simply build more checkout lanes in order to find out, but this could be a costly error if no improvements are found in practice. Producing a model, however, allows potential scenarios to be evaluated on the grounds of expected performance measures, such as the throughput of customers, the revenue earned, and the cost incurred (more checkout lanes, more employees). Having weighed up the projected costs and benefits, a decision can be made as to what action is taken. In this paper we describe two operational research tools that are relevant to neurorehabilitation, and report on their application at Rookwood hospital (a 21 bed facility in Cardiff, UK). The first – personnel scheduling – involves the use of a computer in producing a treatment timetable by fitting patient demand for therapy sessions to staff supply (availability), for a forthcoming period of time (such as a week or month). This automated approach was first applied to an inpatient rehabilitation setting in (Beggs, Vallbona, Spencer, Jacobs, & Baker, 1971). However, whilst their computer program takes into account patient demand in creating personalised schedules for patients, it fails to acknowledge the availability of therapists and thus requires subsequent alterations by staff. Since this early effort, and despite major advances in personnel scheduling in other areas of healthcare (such as nurse rostering (Burke, De Causmaecker, Berghe, & Van Landeghem, 2004) and ER staffing (Centeno, Giachetti, Linn, & Ismail, 2003)), there has been little advancement regarding rehabilitation. At Rookwood hospital an automated program could have a profound effect in freeing up resources, with approximately 8 hours each week spent producing physiotherapy timetables by hand. The second tool of operational research that is pertinent to neurorehabiltation is queue modelling. A queuing process involves the arrival of something at a queuing facility, the service of that something by one of a number of servers, and the subsequent departure of that something from the system. That something is customers in the earlier example of the supermarket, whereby the servers are represented by assistants at checkout lanes. In an inpatient healthcare setting that something is patients, whose referrals to a unit are submitted to a waiting list until a bed (server)
becomes available, at which point the patient is transferred. Although no research of this kind has been undertaken in the field of neurorehabilitation, queuing systems have been studied for other healthcare facilities, such as intensive care units (Griffiths, Price-Lloyd, Smithies, & Williams, 2006; McManus, Long, Cooper, & Litvak, 2004) and geriatric wards (McClean & Millard, 2006; Taylor, McClean, & Millard, 1998). Studies regarding the latter are particularly relevant due to the similarity with ABI patients in pathway, i.e. accident and emergency followed by specialist hospital ward, rehabilitation and then social services. Rookwood hospital could benefit particularly from queue modelling due to an excess demand for its services; all beds are typically occupied and there always exists a queue (waiting list).
2. Methods 2.1. Personnel scheduling Treatment at an inpatient neurorehabiliation unit is typically provided by a multidisciplinary team, involving physiotherapy, occupational therapy, and speech and language therapy departments amongst others (Greenwood, 1997). Due to substantial temporal variability in patient condition and workforce availability (due to annual leave, sickness etc.), many units employ an approach in which demand for treatment sessions (determined by therapists) is somehow fitted to the availability of staff on a short-term routine basis, such as each week. This is not an easy task since there are typically a range of competing objectives and constraints in making therapy session assignments, particularly if demand outstrips supply. Thus creating treatment timetables by hand can not only be time-consuming but can lead to the production of poor quality timetables. Each week at Rookwood hospital the various departments manually produce (i.e. by hand) treatment timetables for the proceeding week consecutively and in order of size. Physiotherapy is the largest department (approximately nine full-time equivalents) and thus acts after all of the other smaller departments (which have less flexibility) have made their assignments. Obtaining an optimal treatment schedule for this department is most difficult since there is more patient demand and employee availability meaning a greater number of potential session assignments. The problem is compounded by reduced patient availability due to the prior arrangements of the other departments. In order
R.M. Wood et al. / Optimising resource management
to demonstrate how operational research can improve personnel scheduling in this setting, we develop an automated program for timetabling physiotherapy treatment – a process that takes eight hours each week if conducted by hand. But first it is necessary to define how the quality of a schedule can be assessed. For this, consider a standard treatment schedule that details which patient(s) are being seen by which employee(s) at what times and for what type of therapy session. If a patient has two hydrotherapy sessions scheduled for a week and these are both scheduled to occur on Monday morning then clearly this is not ideal in terms of attaining an even spread throughout the week. It would also be of concern if the patient in question had a demand of five hydrotherapy sessions for that week and had a preference for afternoons. These problems can be quantified by assigning a penalty to an objective function in each instance. The magnitude of these penalties reflects the type (e.g. having sessions without the patient’s primary/secondary therapist in attendance is less of a problem than having no sessions at all) and severity (e.g. the more sessions demanded but not scheduled – the larger the penalty) of the problem and the priority of the patient (larger penalties are accrued for patients of higher priority ceteris paribus). This is modelled by applying appropriate weights to these variables, such that the value of the objective function is an accurate proxy for schedule quality. The value of these weights has been quantified following discussion with clinicians as well as observing and critiquing the thought process in creating schedules by-hand. These initial values were fine-tuned in the weeks following the replacement of the by-hand approach with the automated program (January, 2011). The scheduling of physiotherapy treatment is modelled here as a hierarchical multi-objective combinatorial optimisation problem. Basically this means that the aim is to find an optimal arrangement, or combination, of treatment sessions subject to many competing objectives of varying importance (some mentioned above). Due to the size of the problem an exact solution (in which all possible schedules are evaluated to find the optimal) is not possible on a desktop computer in the required timeframe of about 6 hours. Instead, heuristic methods (which are able to find ‘good quality solutions in a limited time’ (Puchinger & Raidl, 2005)) are used. The process is firstly to construct an initial allocation of sessions on a ‘blank’ timetable (containing only current patient and therapist availability) by making assignments one-by-one. This takes the computer only
173
a couple of seconds and once complete, the solution is further optimised by making random moves and swaps of treatment sessions around the timetable, and with demanded sessions that are at the time unscheduled. If the alteration reduces the value of the objective function (i.e. increases schedule quality) then the move/swap is accepted and another is performed, otherwise it is rejected and an alternative move/swap is made. At the time specified by the user, the automated program (coded in VBA due to user familiarity with Excel) will stop working and output the most current and thus best schedule produced so far. For more information on this process, and on the construction of the automated scheduling program, see (Griffiths, Williams, & Wood, 2012b). 2.2. Queue modelling As mentioned earlier, the movement of patients through a neurorehabilitation unit can be modelled as a queuing process. But before we describe the queuing model deduced for Rookwood hospital, it is necessary to understand a bit more about the theory. As laid out in (Kendall, 1953) there are three major determinants of a simple queuing system: the (probability) distribution of inter-arrival times, the distribution of service completion times, and the number of servers. If arrivals at the queuing system are random then an exponential distribution represents the variability in times between successive arrivals. The exponential distribution is also the simplest choice in representing the variability of service completion times. With a specification of the number of servers and the rates of arrival and service completion, results can be conveniently output for performance measures such as queue length, waiting time, and throughput. At Rookwood hospital, however, the queuing process is rather more complex. Whilst the arrival of patient referrals from other units (with rate 2.55 referrals/week) has been shown to be random (using data obtained over a 16 month period), not all referred patients are admitted. Many referrals (61%) are withdrawn from the queue before a bed becomes available (due to transfer, death, etc) in a process called reneging. Also, prospective referrals may be dissuaded by the size of the waiting list and simply not join the queue in the first place (balking), the extent of which is unknown. Once admitted, expected length of stay (service completion time) is not found to be exponentially distributed (from an 8 year dataset available), but is better represented by a more complicated phase-type distribution resembling
174
R.M. Wood et al. / Optimising resource management
Fig. 1. Abstraction of the queuing process at Rookwood hospital.
a right-skewed bell-shape (i.e. similar to a log-normal distribution). To further complicate matters, the shape and magnitude of this distribution is found to be very different for broadly speaking four groups of patients (see Fig. 1). Through inspection of records, patients of these groups can typically expect to occupy 6, 3, 3, and 9 of the 21 beds respectively. As a final complexity, discharge is often delayed due to extraneous factors relating to the discharge destination. The duration of this delay is found to be exponentially distributed with a mean time depending on the patient group. The queuing process for this system is depicted in Fig. 1. This queuing process is modelled here using analytical queuing theory – a rigorous mathematical approach that guarantees the accuracy of the results. But how is this model parameterised? Firstly, the probability distributions pertaining to length of stay are fitted (by maximum likelihood estimation) to the data available for the four patient groups. This involves fitting the phase-type distributions to time from admission to date ready for discharge (Fig. 2 A-D), and the exponential distributions to the further time until ultimate discharge (Fig. 2 E-H), yielding means similar to those of the data (Fig. 1). Since the aggregate arrival rate of referrals is unknown due to balking, this is varied (alongside three parameters relating to the reneging and balking functions) such that the resulting performance measures for the observed arrival rate and reneging probability are
effectively equal to their empirically available counterparts. For the model corresponding to the optimal values, there is only a 0.2% and 0.5% difference in the model and empirical results for these measures. See (Griffiths, Williams, & Wood, 2012a) for a more detailed description of the construction of the queuing system.
3. Results 3.1. Personnel scheduling Before its introduction, the automated scheduling program for physiotherapy treatment at Rookwood hospital was trialled in the latter quarter of 2010. During this time it was used alongside the manual approach, and performance results of both efforts were evaluated (Table 1). It can be seen from these results that there are significant gains in schedule quality (as assessed by objective function value) through using the automated program. Whilst there is no obvious difference in unmet patient demand (this is essentially bounded by staffing levels), other constituents of the objective function have improved substantially, e.g. the assignment of sessions to primary or secondary therapist. Note that such performance measures can be affected by altering the weights of the constituent, for example, if it were
Fig. 2. (A-D). Empirical probability densities for time from admission until ready for discharge (solid line), with phase-type distributions fitted by maximum likelihood (dashed line). Distribution means are 72, 137, 121, and 178 days respectively. (E-H). Empirical probability densities for time from date ready for discharge until ultimate discharge (solid line), with phase-type distributions fitted by maximum likelihood (dashed line). Distribution means are 14, 26, 55, and 77 days respectively.
R.M. Wood et al. / Optimising resource management 175
176
R.M. Wood et al. / Optimising resource management
Table 1 A comparison of performance measures relating to the scheduling of physiotherapy treatment using the manual ‘by-hand’ approach and the automated scheduling program. Results are given for objective function value (lower is better), percent of demanded sessions scheduled (higher is better), and percent of sessions with neither primary/secondary therapist in attendance (lower is better) Week start
20 September 2010
Method Objective function value (normalised) Average percent of demanded sessions scheduled (%) Average percent of sessions with neither primary/secondary therapist (%)
4 October 2010
1 November 2010
By-hand
Program
By-hand
Program
By-hand
Program
1 85 48
0.53 95 28
1 93 37
0.6 90 16
1 76 37
0.48 74 18
Fig. 3. Probability density of the total number of referrals and patients in the system. Table 2 Responsiveness of performance measures and costs to a number of hypothetical scenarios Measure Reneging probability Mean bed occupancy (patients) Annual throughput (patients/year) Mean queue length (referrals) Mean waiting time (days) Annual cost
Original model
Reduce delays to discharge (50% / 100%)
One-third increase in older patients
Three additional therapists
More group sessions
0.62 20.8 51 10 29 £3.64m
0.58 / 0.45 20.7 / 20.4 57 / 60 9/8 26 / 22 £3.62 m / £3.57m
0.65 21 51 11 27 £3.68m
0.6 20.7 53 10 28 £3.78m*
0.59 20.7 54 10 27 £3.62m
*indicates an amendment in how cost is calculated to include a change in workforce composition.
desired that less patients received sessions with neither the primary or secondary therapist in attendance then the corresponding weight is increased (this would however make it harder for sessions to be scheduled and so would unfavourably affect unmet demand). As mentioned in the Introduction, modelling is often used to ask questions that cannot be easily answered in real-life. To this end, we use the automated program to evaluate the effect of changes to physiotherapy treatment assignment on the amount of therapy received by patients. This involves the use of an initial refer-
ence schedule which details typical levels of demand and priorities for the 6, 3, 3, and 9 patients of each group, and typical levels of staff availability. A rather trivial example is an expansion in the number of fulltime equivalents from 9 to 12. In order to deduce the effect of this on the average treatment intensity (which is an important measure that can affect length of stay (Blackerby, 1990)), the automated program is run with typical patient demand levels (for the four patient groups) but a modified staff roster. As would be expected, the treatment intensity increases by one-third
R.M. Wood et al. / Optimising resource management
177
Fig. 4. Responsiveness in performance measures and costs to a change in the capacity of the unit. The left-hand axis represents the probability of reneging, and the right-hand axis represents the annual throughput (in patients), the mean waiting time (in days) and the mean queue length (in referrals).
on average. Another example is a change in session composition to increase the provision of group sessions (which involve a greater patient-therapist ratio than standard sessions). In this case, the staff availability is unchanged but patient demand is modified to include three more group sessions in place of one of the standard sessions for each of the patient groups. Results indicate a 62% increase in average treatment intensity. 3.2. Queue modelling Results of the parameterised queuing model for Rookwood hospital are now presented. The probability density of the total number of referrals and patients in the system is provided in Fig. 3. It can be seen from this that the bed occupancy is nearly always at capacity and that the waiting list can reach up to 20 referrals at times. Interestingly the aggregate demand for the unit is found to be 2.751 referrals/week; suggesting that the length of the waiting list is sufficient to dissuade 8% of prospective referrals (since the actual arrival rate of referrals at the queue is 2.541 per week). The performance measures of the system are detailed in the first column of Table 2. The annual running cost is calculated as the product of mean bed occupancy, bed-cost per day (£480 with current workforce), and 365. From time to time, hospital management may seek to make alterations to aspects of service delivery at a hospital unit. This may be as a result of new guidelines, efficiency drives or budgetary issues. The queuing
model created here has been used to evaluate the effect of a number of relevant scenarios on performances measures and costs. A simple instrument is hospital capacity (i.e. number of beds), which when varied by even a small amount can have a noticeable consequence (Fig. 4). As previously mentioned, the discharge of patients is typically delayed – on average by 14, 26, 55, and 77 days for the four patient groups (Fig. 3). If however this can be reduced by half or cut altogether then the effect is a much more fluid flow of patients through the unit (Table 2). This can be achieved by informing social services of expected patient readiness dates (using onadmission predictors of length of stay, e.g. age, sex, diagnosis, admission source, etc.); such that efforts can be placed in ensuring the discharge destination is ready on time. A particularly relevant scenario to an ageing population is also analysed, in which the arrival rate of referrals from patients over the age of 50 years is increased by one-third. Since patient length of stay is dependent on treatment intensity (as alluded to earlier in Results), this permits the analysis of scenarios involving changes to the variables of the automated scheduling program, e.g. increasing the number of therapists and providing more group sessions (Table 2). But this requires knowledge of how responsive length of stay is to a change in treatment intensity. To quantify this we fit a simple inverse relation (representing a convex curve) between these two variables to the available data from Rookwood hospital for each patient group. Average length
178
R.M. Wood et al. / Optimising resource management
of stay in each group can thus be calculated from the output of the scheduling program (treatment intensity) through these relations. This is used to appropriately scale the probability distributions relating to length of stay for each patient group in the queuing model. This hybrid queuing-scheduling model has also been used to evaluate scenarios relating to redundancy strategies (in light of UK budget cuts) at the behest of senior management. 4. Discussion In this article we describe how mathematical modelling can be used to improve resource management in neurorehabilitation. To this end, we introduce two well-known tools of operational research - personnel scheduling and queue modelling - and exemplify the benefits they can bring to service delivery through a case study at a major unit in the UK. The purpose of this paper is not to provide a detailed account of how these tools are implemented (for this see (Griffiths et al., 2012a, 2012b)), but to provide an overview of how and why they are used and the kind of improvements they can bring. It must also be acknowledged that, whilst many units share similar features, the model created here is designed specifically for Rookwood hospital, and that other units will likely require bespoke approaches. A major component of this investigation has been the construction and introduction of an automated scheduling program for assigning physiotherapy treatment. Not only has this led to considerable increases in timetable quality, but it has also reduced employee time expended in the creation of timetables from eight hours each week to fewer than two (thus freeing up substantial time for clinical work). In addition, data required for audit purposes is automatically output (removing the need for paper records) and there is no longer a burden on the employee tasked with timetabling to cater to the personal requests of therapists. The program has been successfully in use at Rookwood hospital since January 2011 and has enjoyed much praise from its users. For units seeking to introduce a similar scheduling program, note that timetable quality as judged by the user is critically dependent on the objectives provided and the corresponding weights. Queue modelling has also proved to be a valuable asset to Rookwood hospital. This has been through the ability to analyse the impact of relevant hypothetical scenarios on performance measures and costs without having to commit to long-term changes, such as
changing the number of beds. The model has also been used to evaluate scenarios that are, to some extent, outside the control of the hospital, such as an increase in older patients and a reduction in delays to discharge. If, for example, the delay to discharge can be cut by half then the probability of reneging is reduced by 6% and throughput increased by 12% (meaning more patients get the service they require). The accuracy of such predictions is ensured by developing a model that is sufficiently representative of activities on the ground. But with greater representation comes greater complexity, and this increases the difficulty in modelling. In this project we use analytical queuing theory in order to rigidly ensure accuracy, but a far more practical approach, especially for non-mathematicians, is through simulation (using off-the-shelf software like Witness and Simul8). As well as promoting accuracy, a complex model also allows the consideration of a greater assortment of relevant scenarios. In a simple queuing model, the only ‘handles’ are the probability distributions of arrival and service and the number of servers. Even if the model predicts substantial improvements by, say, changing the service (length of stay) distribution from exponential to log-normal, this is useless information for policy-makers without the ability to say how this can be achieved. But if such a change could be shown to be brought about by changing the composition of the workforce (through the scheduling program) then the model predictions would be useful to hospital management who are able to influence such decisions. Thus appropriate model complexity is necessary to give users relevant ‘handles’ on the kind of scenarios that can be evaluated. An example of this that has been considered is using more group sessions than one-to-one therapy. But in this particular example there lies a problem that the model lacks the complexity to comprehend. Whilst results show that under this scenario treatment intensity increases and thus length of stay reduces, the model fails to take account of the quality of treatment and thus patient outcome. Future research in this field should investigate the trilateral relationship between treatment intensity, length of stay and outcome at discharge.
Acknowledgments The authors would like to acknowledge the help and support of all staff at Rookwood hospital involved in this project, with particular thanks to Jenny Thomas and Angelo Lamberti. This study would not have
R.M. Wood et al. / Optimising resource management
been possible without their sustained efforts and contributions.
Declaration of interest None. References Association of European Operational Research Societies (2013). What is OR? Retrieved September 9, 2013, from http://www.euroonline.org/web/pages/197/what-is-or Beggs, S., Vallbona, C., Spencer, W. A., Jacobs, F. M., & Baker, R. L. (1971). Evaluation of a system for on-line computer scheduling of patient care activities. Computers and Biomedical Research, 4(6), 634-654. Blackerby, W. F. (1990). Intensity of rehabilitation and length of stay. Brain Injury, 4(2), 167-173. Burke, E. K., De Causmaecker, P., Berghe, G. V., & Van Landeghem, H. (2004). The state of the art of nurse rostering. Journal of scheduling, 7(6), 441-499. Centeno, M. A., Giachetti, R., Linn, R., & Ismail, A. M. (2003). Emergency departments II: A simulation-ilp based tool for scheduling ER staff. In Proceedings of the 35th conference on Winter simulation: Driving innovation (pp. 1930-1938). Retrieved from http://dl.acm.org/citation.cfm?id=1031086 Greenwood, R. (1997). Neurological rehabilitation. Psychology Press. Griffiths, J. D., Price-Lloyd, N., Smithies, M., & Williams, J. (2006). A queueing model of activities in an intensive care unit. IMA Journal of Management Mathematics, 17(3), 277-288. Griffiths, J. D., Williams, J. E., & Wood, R. M. (2012a). Modelling activities at a neurological rehabilitation unit. European Journal of Operational Research. Retrieved from http://www. sciencedirect.com/science/article/pii/S0377221712008028
179
Griffiths, J. D., Williams, J. E., & Wood, R. M. (2012b). Scheduling physiotherapy treatment in an inpatient setting. Operations Research for Health Care. Retrieved from http://www. sciencedirect.com/science/article/pii/S2211692312000379 Kendall, D. G. (1953). Stochastic processes occurring in the theory of queues and their analysis by the method of the imbedded Markov chain. The Annals of Mathematical Statistics, 338-354. McClean, S., & Millard, P. (2006). Where to treat the older patient? Can Markov models help us better understand the relationship between hospital and community care? Journal of the Operational Research Society, 58(2), 255-261. McManus, M. L., Long, M. C., Cooper, A., & Litvak, E. (2004). Queuing theory accurately models the need for critical care resources. Anesthesiology, 100(5), 1271-1276. Puchinger, J., & Raidl, G. R. (2005). Combining metaheuristics and exact algorithms in combinatorial optimization: A survey and classification. In Artificial intelligence and knowledge engineering applications: A bioinspired approach (pp. 41-53). Springer. Retrieved from http://link.springer.com/chapter/ 10.1007/11499305 5 Taylor, G. J., McClean, S. I., & Millard, P. H. (1998). Using a continuous-time Markov model with Poisson arrivals to describe the movements of geriatric patients. Applied Stochastic Models and Data Analysis, 14(2), 165-174. Turner-Stokes, L., Bill, A., & Dredge, R. (2012). A cost analysis of specialist inpatient neurorehabilitation services in the UK. Clinical Rehabilitation, 26(3), 256-263. Turner-Stokes, L., Poppleton, R., Williams, H., Schoewenaars, K., & Badwan, D. (2012). Using the UKROC dataset to make the case for resources to improve cost-efficiency in neurological rehabilitation. Disability and Rehabilitation, 34(22), 1900-1906. Turner-Strokes, L. (2003). Rehabilitation following acquired brain injury: National clinical guidelines. Royal College of Physicians. Retrieved from http://www.rcplondon.ac.uk/sites /default/files/documents/rehabilitation-followingacquired-bra in-injury.pdf
Copyright of NeuroRehabilitation is the property of IOS Press and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.
