Health Care Manag Sci DOI 10.1007/s10729-015-9320-8

Optimal control of ICU patient discharge: from theory to implementation Fermín Mallor & Cristina Azcárate & Julio Barado

Received: 21 November 2013 / Accepted: 23 February 2015 # Springer Science+Business Media New York 2015

Abstract This paper deals with the management of scarce health care resources. We consider a control problem in which the objective is to minimize the rate of patient rejection due to service saturation. The scope of decisions is limited, in terms both of the amount of resources to be used, which are supposed to be fixed, and of the patient arrival pattern, which is assumed to be uncontrollable. This means that the only potential areas of control are speed or completeness of service. By means of queuing theory and optimization techniques, we provide a theoretical solution expressed in terms of service rates. In order to make this theoretical analysis useful for the effective control of the healthcare system, however, further steps in the analysis of the solution are required: physicians need flexible and medically-meaningful operative rules for shortening patient length of service to the degree needed to give the service rates dictated by the theoretical analysis. The main contribution of this paper is to discuss how the theoretical solutions can be transformed into effective management rules to guide doctors’ decisions. The study examines three types of rules based on intuitive interpretations of the theoretical solution. Rules are evaluated through implementation in a simulation model. We compare the service rates provided by the different policies with those dictated by the theoretical solution. Probabilistic analysis is also included to support rule validity. An Intensive Care Unit is used to illustrate this F. Mallor (*) : C. Azcárate Department Statistics and Operational Research, ISC-Institute of Smart Cities, Public University of Navarre, Campus Arrosadía, 31006 Pamplona, Spain e-mail: [email protected] C. Azcárate e-mail: [email protected] J. Barado Hospital of Navarre, C/ Irunlarrea 3, 31008 Pamplona, Spain e-mail: [email protected]

control problem. The study focuses on the Markovian case before moving on to consider more realistic LoS distributions (Weibull, Lognormal and Phase-type distribution). Keywords Queuing theory . Healthcare . Optimization . Control problem . Simulation . Intensive care unit Length of stay MSC 60K25 . 80M50 . 93E20 . 93C65

1 Introduction Healthcare resources, both material and human, are limited; demand is variable and sometimes exceeds what is available. In these cases physicians have to strike a balance between the number of patients served and the Bindividual quality^ of the service they receive. This paper discusses this healthcare delivery control problem which physicians face almost every day. This study first examines the theoretical aspects of the problem using queuing theory and optimization techniques and then analyzes the structure of the theoretical solution and the practical implementation issues. It concludes by discussing several ways of transforming the solution into a helpful decision tool for health care system management. Healthcare delivery can be viewed as a queuing system in which patients arrive, wait to be served, get served, and depart. The resources (or servers) include both the medical and other personnel, and the specialized equipment and infrastructure required by the healthcare process. Queuing theory mathematically models queuing systems and uses probability to calculate various performance measures, such as waiting time in queue, length of queue, server occupancy rates, probability of patient rejection, etc.

F. Mallor et al.

Specifically, queuing theory has been applied in healthcare to study a range of management problems including: &

&

&

Patient waiting time and resource utilization: the effects on performance measures of queue discipline, reneging and blocking have all been considered. See, for example, the case of transplant waiting lists [1] and studies of waiting time in emergency departments [2, 3] using queuing models with blocking and reneging. System design and dimensioning: the objective is to determine the levels of resources (beds, nurses, etc.) required to meet some quality of service level. Determining the capacity by minimizing a cost function is a special case of the sizing problem. Bed sizing problems are analyzed in [4], which considered 24 wards in a medical centre and in [5], where a 400-bed obstetric hospital is studied. Several studies focus on the optimal number of nurses (see, for example, [6] for an intensive care unit). Scheduling and appointment rules: the purpose is to reduce patient waiting times without increasing resource idle time. As an example, the reader is referred to the scheduling admissions study in [7] and the simulations models to analyze the effect of different appointment schedules in an Ear, Nose and Throat outpatient department in [8] and in surgical care in [9].

An exhaustive review of the literature on the application of queuing theory in healthcare management problems can be found in [10]. Control problems [11] try to determine the best operative rules for optimizing some measure of interest. This type of control problem can arise, for example, when developing a policy for switching workers between several departments within a healthcare system in order to cope with changing workload ([12] and references therein) or when attempting to control the queue via the application of optimal admission policies incorporating customer behavior characteristics, leading to the idea of controlling arrivals by pricing [13]. We consider a type of control problem in which the objective is to minimize the rate of patient rejection due to service saturation. The scope of decisions is limited, in terms of both the amount of resources to be used, which are supposed to be fixed, and the patient arrival pattern, which is assumed to be uncontrollable. This means that the only potential areas of control are speed and completeness of service (that is, allowing early discharges of patients). The motivation for the study of this control problem arose from a simulation study carried out to analyze a capacity problem in an intensive care unit (ICU) [14]. Intensive care units (ICUs) are key components in the management of critically but not terminally ill patients, whose care involves the use of highly complex technological equipment and the work of a large number of specialized ICU staff. Since the ICU is

one of the most complex and costly medical resources in a hospital, the ideal approach to its management is a matter of debate: ICU admission and discharge policies are important not only in resource management terms but also in terms of critical patient care outcome. Several studies show that, in the event of a bed shortage, admissions and discharges are triaged (see, for example, [15–17]). These patient discharge decisions are already implicit in much of the medical literature analyzing the consequences of premature discharge, the aim of which is often to create free beds (see for example [18–22]). Patient discharge is a process that is influenced by factors relating to the patient’s state of health and environment and certain organizational issues [23]. A very busy ICU increases the number of rejected admission requests and the severity threshold for ICU admission, and shortens length of stay [24, 25]. Other consequences of an excessive bed occupancy rate are scheduled surgery cancellations and transfers to other centres. The validation process of the ICU simulation model of the Hospital of Navarre showed that physicians controlled the bed occupancy level by decreasing the length of stay (LoS) at times of high bed occupancy. Early discharge of patients is reported in the simulation and modeling literature. For example, in [26] it is pointed out that the early discharge of the more recovered patients for transfer to other wards is a common solution to ICU bed shortage. Authors in [27] observed changes in ICU management dynamics when units become full and physicians attempt to limit admissions or discharge recovering patients. Other medical studies have reported [28, 29] that admissions and discharges are triaged when not enough beds are available. However, while all these studies suggest early discharge as a bed management tool, they do not include it in their mathematical models. The control problem can be approached either from a normative or a descriptive standpoint. An example of a descriptive approach to the control problem can be found in [14] which models doctors’ patient discharge decisions and defines a set of rules to determine the conditions for earlier or delayed discharge of certain patients, according to the bed occupancy level. The parameter values for these rules were calibrated by solving a nonlinear stochastic optimization problem aimed at matching the model outputs with the real system outputs: a procedure which was later improved in [30] by including the principle of Bminimum medical intervention^ as a second objective function. From the normative perspective, the aim of the problem is to define the control rules to optimize goals for physicians regardless of their current approach or any they have used in the past. It is necessary to specify the objectives of the control problem, identify the decision variables and any constraints upon them, and finally obtain a mathematical model of all these elements, which, together, constitute the optimization problem. The optimal solution of this optimization problem

Optimal control of ICU patient discharge

will identify the best way of controlling the health system, assuming the truth of the hypotheses included in the mathematical model. Although this normative approach to solving a control problem is considered in this paper to lead to a control policy, the practical application of such a policy by a physician is not straightforward and requires some adjustment. The main contribution of this paper is to discuss how the theoretical solutions provided by queuing and optimization procedures can be transformed into effective management rules to guide doctors’ patient discharge decisions. In Section 2, we formulate an optimization-based queuing control problem and interpret the solution. The problem formulation presented here leads to a solution in terms of the service rates identified by the physicians of an ICU as a representative means of dealing with the pressure of a near-full ICU in the real setting. A discussion of other possible formulations of the control problem and the analysis of their solutions can be found in [31]. The practical implementation of the theoretical solution for the Markovian case is discussed in section 3, while the general case is analysed in Section 4. The paper ends with a discussion of the results and some concluding remarks.

2 Mathematical modelling of the bed occupancy control problem 2.1 Optimization problem definition Healthcare resources need to be managed in such a way that their benefits reach the largest population possible. The inevitable randomness of healthcare demand, however, can sometimes result in a shortage of available resources, which are fixed and cannot be increased on an ad hoc basis. Physicians therefore have to balance the number of patients served and the Bindividual quality^ of this service. The degree to which these two objectives are met is what determines the quality of service (QoS) level. Thus, healthcare resources are controlled by modifying service rates, and making them dependent on the current state of the system. The busier a healthcare system, the higher the service rate needs to be. To illustrate this control problem, we consider an intensive care unit where patients arriving to find a full ICU are rejected (eligible patients are usually referred to another ward or hospital and scheduled surgeries may be cancelled). The capacity of this health service system is measured in terms of the number of available beds, which is fixed (and determines the number of physicians, nurses and medical equipment). The ICU serves non-elective patients, whose arrival is random and impossible to schedule. Medical staff making decisions relating to patient admission and discharge issues must consider the following two QoS criteria: the first is the percentage of population

that can benefit from it when needed, and the second is the degree of recovery reached by a patient when discharged from the ICU. Thus, high service quality means both a low percentage of patient rejections and ICU length of stays as long as necessary. The two objectives of the control problem are to minimize the probability of patient rejection and the shortening of patients’ LoS (see Fig. 1). Such shortening implies a decrease in the LoS average and an increase in the service rate. Furthermore, because increased service rates are intended to reduce the probability of patient rejection it seems natural and reasonable to suppose non-decreasing service-rate values to reflect the fact that service capacity does not decrease as the ICU becomes busier. A mathematical formulation of this control problem is: ½CP1 min PR min ∑ci¼1 ðμi −μÞ sub ject to μ ≤ μ1 ≤ μ2 ≤⋯ ≤ μc

where c is the total number of beds available, μ is the individual service rate when no control is applied, and μi is the individual service rate when the number of occupied beds is i and control is applied. Other formulations of the control problem objective function related with the shortening of patients’ LoS can be considered accounting for different aspects of the service process. Each of the different formulations of the control problem provides different optimal solutions. A full discussion of the solution structures for the different mathematical formulations of the control problem as well as the practical implications to their derived management policies is included in [31]. Nevertheless, the control rules studied in this paper are neither linked to nor dependent on the optimization problem formulation. The rules proposed in next section are designed to mimic in practice any set of previously fixed service rates regardless of how these rates are found. Assuming the above mathematical formulation, we estimate the Pareto frontier by using the ε-constraint method and considering different bounds for the probability of patient rejection (εj-values). Each bound gives a nonlinear optimization problem whose solution is an efficient solution for the biobjective problem: ½CP2 min ∑ci¼1 ðμ
i −μÞ sub ject to

PR ≤ ε j μ ≤ μ1 ≤ μ2 ≤ ⋯ ≤ μc

As suggested in the introduction, a healthcare system can be mathematically modelled as a queuing system. In the case of the ICU under consideration, the clients are the patients, the servers are the beds and there is no waiting room. Adopting Markovian assumptions on

F. Mallor et al. Fig. 1 Goals for the control problem with an illustration of the objective functions

arrivals and service times, the queuing model representing the system is a Bmulti-server Birth-and-Death queue with state-dependent service rates^, where the arrival rate is a constant λand the service rates μ=(μ1, ⋯, μi,⋯, μc) can be dependent on the state of the system i, that is, on the number i of occupied beds. P=(P0, P1, ⋯, Pc) denotes the stationary probability distribution of the system state (number of occupied beds) when the service is being provided at service rates μ. By the PASTA property (Poisson Arrivals See Time Averages), PR is the probability of an arriving patient finding a full ICU, Pc [32]. Queuing theory provides an explicit expression for Pc: Pc ¼

λc þ

Xc  i¼1

λc λc−i i!ðci Þ ∏ j¼c−iþ1 μ j c

Þ

Thus, the optimization problem has c decision variables (μ1,μ2, …, μc) and is nonlinear. This nonlinear problem can be equivalently formulated as the maximization of a convex function over a convex set as follows. Let us consider the minimization of the probability of patient rejection and include the other objective function in the constraint set. An efficient solution of the bi-objective problem is obtained by fixing a value for the threshold εj and solving the following problem 8 X < μi ≤ ε j min Pc subject to : μ i≤ μ ≤ … ≤ μ 1

2

c

Observe that minimizing Pc is equivalent to maximizc λc ing g (μ), where Pc ¼ λc þg ðμÞ and g ðμÞ ¼ ∑i αi ∏ j¼c−iþ1 μ j

is a posynomial, where αi are constant values depending on λ and c. A convex representation of the posynomial g

(μ) can be obtained by using new variables yj =log(μj) and taking the logarithm of function g (μ). max log

X

c

α ∏ j¼c−iþ1 e i i

yj

(

 subject to

X

e yi ≤ ε j i y1 ≤ y2 ≤ … ≤ yc

Such problems are usually hard to solve and need to be solved by numerical nonlinear programming algorithms. We use LINGO solver for the case in hand. 2.2 Interpretation of theoretical solutions to the control problem The optimal solution to this optimization problem has a clear structure for the values of the decision variables. As the healthcare service becomes busier, and closer to full occupancy, the service rate needs to increase in a gradual and sustained manner. We illustrate this solution structure with two examples. In the first, we consider an ICU with five beds, λ=4 and μ=1 (λ/cμ=80%). Table 1 presents the service rates needed to obtain various percentages of patient rejections. Without control, that is, without bringing forward the discharge of any patient, patient rejection is 19.9 %. When this percentage is decreased slightly to 19 %, the full occupancy service rate only needs to be increased slightly, from 1 to 1.0596. When the percentage of patient rejection decreases, while the full occupancy service rate increases, an increase in the service rate in the neighbor states is also required. In the second example, we consider the simplified Markovian version of a real 20-bed-ICU. Without loss of generality, we set average LoS at 1/μ=1. Table 2 includes the results for the case that λ=16 and μ=1 (λ/cμ=80%). The percentage of patient rejection is 0.064411. Service rate values follow the same pattern when the proportion of patients rejected is decreased from 6 to 1 % (εj =6, 5, 4, 3, 2, 1 %). The mathematical solution is plausible from the medical perspective, because it mirrors standard ICU management

Optimal control of ICU patient discharge Table 1 Optimal solutions to the optimization problem [CP2] for a Markovian 5-bed-ICU, with λ=4 and μ=1 Rejected patients

μ1

μ2

μ3

μ4

μ5

19 % 16 % 13 % 10 % 7% 4% 1%

1 1 1 1 1 1 1

1 1 1 1 1 1 1.3959

1 1 1 1 1 1.4516 2.9959

1 1 1.1027 1.3507 1.7380 2.2721 3.7080

1.0596 1.3049 1.5534 1.8014 2.1887 2.6332 3.9000

practices. A medical interpretation of this solution is included in the discussion section. The theoretical solution to the control problem is provided in terms of the service rates that need to be attained at each system state. The service rate value indicates the achievement of an overall LoS target for all patients leaving the system in the same state. Obviously, the service rate value is open to fuzzy interpretation by physicians as a measure of pressure to discharge patients early, that is, before full recovery (by full recovery, we mean that the patient has been stabilized and can be discharged because he cannot benefit from any further time in the ICU). The higher the ratio μi/μ, the higher the pressure to discharge patients early. For example, a value of μi/μ=1.12 means that a 12 % increase is required in the number of patients served per unit of time. An understanding of these service rate values can, of course, heighten physicians’ awareness of the pressure to discharge current patients in order to avoid rejecting new ones. This alone, however, does not help physicians to make individual discharge decisions. Further steps in the analysis of the solution are required to make the queuing and optimization analysis useful for the effective control of the ICU. Physicians need flexible and medically-meaningful operative rules for shortening patient length of service to the degree needed to give the service rates dictated by the theoretical analysis. Although there are existing guides for ICU admission criteria [22, 29], the medical literature contains few references

to guides for ICU patient discharge criteria, and therefore few Units possess them [33]. Thus, the premise remains valid that discharge from the ICU to a lower level of care is appropriate when the patient's physiological status has stabilized and ICU monitoring and care are no longer required, or when it has deteriorated to the point that active interventions are no longer planned. From the moment of admission, the state of health of recovering patients would ideally improve gradually until the moment of discharge, as determined by the above mentioned guidelines. However, in reality the course of events from admission to discharge (see Fig. 2) varies considerably depending on the comorbidity of the patient and any complications that may arise following admission, often as a result of serious nosocomial (hospital-acquired) infections. The patient’s evolution will often follow a more or less upwardly sloping linear trend, as is the case with most patients who undergo planned surgery involving few complications. In other cases, the patient’s evolution is a more tortuous series of improvements and deteriorations due to concurrent complications. However, the last stage in the recovery of all ICU patients will be marked by an upward trend, whatever the slope. In the event of a patient’s LoS being shortened, this will occur in the final stage of the recovery process. Thus, early discharge will correspond to a health state prior (but hopefully close) to full recovery. In the next section we analyze the suitability of different rules to guide discharge depending on the health status of the patient (indirectly measured as the time spent in the ICU) and the bed occupancy level. 3 Practical implementation of the theoretical solution: the Markovian case A set of rules constitutes a management policy for control of the healthcare system when it provides the physician with a Health Status Full recovery

Infection Table 2 Optimal solutions to [CP2] for the Markovian version of the 20-bed-ICU

6 5 4 3 2 1

% % % % % %

μ1

…

μ15

μ16

μ17

μ18

μ19

μ20

1 1 1 1 1 1

1 1 1 1 1 1

1 1 1 1 1 1

1 1 1 1 1 1.2616

1 1 1 1.0452 1.2096 1.4742

1 1 1.0837 1.2102 1.3746 1.6099

1 1.1029 1.2096 1.3314 1.4813 1.6848

1.0786 1.1963 1.2972 1.4081 1.5419 1.7223

Time Ideal discharge time

Fig. 2 4 potential ICU patient-recovery processes from admission to discharge. Solid lines represent progressive and almost linear health improvements, while dotted lines represent recovery processes accompanied by health complications

F. Mallor et al.

guide for deciding which patient to discharge and when, for any potential system state. This section explores three types of management policies for shortening LoS. The three rules are based on the time the patient has already spent in the healthcare system in relation to the total time required for full recovery, which can be interpreted as an estimation of the patient’s health status, as discussed in a previous section. A valid set of rules should give the service rates obtained in the theoretical solution to the control problem, and also be coherent with the medical practice. The three types of management policies analyzed are the following: Type 1. Discharge rules are based on the total percentage of time already spent in the healthcare system. Type 2. Discharge rules are based on the remaining time required for full recovery. Type 3. Patient discharge follows a probabilistic rule combined with the time spent in the healthcare system. These management policies are evaluated through implementation in a simulation model. We compare the service rates provided by the different policies with those dictated by a theoretical solution. Probabilistic analysis is also included to support rule validity. For illustrative purposes, Figs. 3, 4, 5, 6, 7 and 8 are obtained with the following set of service rates:

recovery. The value of the second variable is exactly known in the simulation setting but not in a real situation, where the time remaining for complete recovery needs to be estimated by the physician from the patient’s current health status. Both type 1 and type 3 rules are directly dependent upon the knowledge of these two time variables, whereas type 2 rules are based on shortening the patient’s length of stay, that is, bringing forward the planned date of discharge. Type 1. Rule based on total percentage time already spent in healthcare system We begin by analyzing what could be seen as a natural individual discharge rule for reaching the desired collective service rate. If the normal service rate is μ but has to be increased to a value of μi when the system enters state i, then average LoS in this state needs to be reduced from 1/μ to 1/μi. That is, average LoS is reduced by a factor of μ/μi. This reduction factor can be fairly applied to all patients when the health system is in state i. This means that, with a healthcare system in state i, a patient will be discharged on reaching a completed LoS ratio higher than μ/μi. That is, a LoS T distributed as Exp(μ) rescaled by a factor μ/μi, leads to a LoS (μ/μi)T distributed as Exp (μi). Operative rule T1: when the health system enters state i, discharge a patient as soon as

μ1 ¼ μ2 ¼ … ¼ μ14 ¼ 1; μ15 ¼ 1:05; μ16 ¼ 1:1; μ17

Time already spent in system μ ≥ Total time required f or full recovery μi

¼ 1:15; μ18 ¼ 1:2; μ19 ¼ 1:25; μ20 ¼ 1:3 Patient LoS is individually generated (simulated) the moment the patient is assigned a bed. Thus, at any point in time, there will be two known variables for each patient: accumulated time in the ICU and time remaining for complete Fig. 3 Theoretical rates compared with simulated rates obtained from the application of patient discharge rule T1

The simulation results for this control rule clearly show that it does not achieve the intended goal (see Fig. 3). In other words, it fails to achieve the target service rates, sometimes

Goal Rates vs Control Rules Rates Shortening above a minimum LoS% threshold 2.5 Goal Rates Control Rules T1

Service Rates

2.0

1.5

1.0

0.5 10

12

14 16 Bed Occupancy

18

20

Optimal control of ICU patient discharge Fig. 4 Theoretical rates compared with simulated rates obtained from the application of discharge rule T2a

Goal Rates vs Control Rule Rates Shortening minimum remaining LoS Individual shortening 1.3 Goal Rates Control Rules T2a

Service Rates

1.2

1.1

1.0

0.9 8

through overestimation and sometimes through underestimation. If all service rates were to be modified from μ to the same μ*, then the rule would work. Nevertheless, this rule fails because the time reduction is applied to the total time required for full recovery, that is, from admission onwards, whereas the need to accelerate the process or shorten the LoS only arises when the system enters state i. Type 2. Rule based on time remaining for full recovery. Shortening of one patient’s LoS. We now consider the remaining time in the healthcare system required to achieve full recovery as our patient discharge criterion. That is, when the healthcare system enters state i, the time remaining for full recovery is estimated for all patients,

Fig. 5 Theoretical rates compared with simulated rates obtained by applying discharge rule T2b

10

12

14 16 Bed Occupancy

18

20

and the one with the lowest estimate is selected for early discharge. This patient’s remaining time in the healthcare system is reduced by a factor of μ/μi. We aim to address the problem encountered by rule 1 by applying the time reduction factor only to that part of the patient’s LoS, during which the system is in state i. Operative rule T2a: when the system enters state i, the patient with the least remaining LoS becomes a candidate for early discharge after a period equal to the remaining LoS reduced by a factor of μ/μi. If, during the time interval between the system entering state i and the patient’s moment of discharge, a new patient arrives, the system enters state i+1 and

Goal Rates vs Control Rule Rates Shortening minimum remaining LoS Collective shortening 1.35 1.30

Goal Rates Control Rules T2b

Service Rates

1.25 1.20 1.15 1.10 1.05 1.00 8

10

12

14 16 Bed Occupancy

18

20

F. Mallor et al. Fig. 6 Theoretical rates compared with simulated rates obtained by applying discharge rule T3a

Goal Rates vs Control Rule Rates Probability of immediate discharge Minimum remaining LoS 1.30 Goal Rates Control Rules T3a

Service Rates

1.25 1.20 1.15 1.10 1.05 1.00

8

the rule corresponding to this new state is applied. The simulation results for this policy show that this rule also fails to achieve the intended goal, as can be seen in Fig. 4. Observe that, by the lack of memory property of the exponential distribution, the time remaining for a patient’s full recovery is exponentially distributed with rate μ. Thus, the change to a new rate of service μi >μ can be viewed as the change of the time scale by a factor of μ/μi xg ¼ P fT > x þ s = T > sg ¼ F ðx þ sÞ= F ðsÞ; x ≥ 0 and expected

Optimal control of ICU patient discharge

 ∞ ∞ v a l u e ∫0 F γk ðxÞdx ¼ ∫s F ðxÞ=F ðsÞ dx≤ 1=μ ∀s > 0 b e cause F is NBUE. Then the substitution of patient k by a new patient at time t implies an increase in the expected remaining time of all patients in the ICU at time t in Z 1=μ−

∞

 F ðxÞ= F ðsÞ dx≥ 0

s

which implies a decrease in the service rate. ■ In like fashion, the following result can be established. Corollary 2. Let F be the LoS distribution function belonging to the ageing class NWUE with expected value 1/μ. Then the removal of a patient immediate to the admittance of a new one with positive probability provides a higher state service rate than that given by an exponentially-distributed LoS with rate μ. We analyze rule performance with the following distribution families: Weibull, Lognormal, and Phase-type distributions. The Weibull distribution family has two parameters: shape β and scale σ. The behaviour of the hazard function depends on the values of β. When β>1 (β0) and Phase (q= 0), respectively, and they consider the discharge of patients only in phase 3.

The parameter defining Rules T3b and T3c are dictated by the optimal solution for the simplified Markovian version of the 20-bed-ICU introduced in section 2.1, with a 3 % patientrejection (see Table 2): μ1 ¼ μ2 ¼ … ¼ μ16 ¼ 1; μ17 ¼ 1:0452; μ18 ¼ 1:2102; μ19 ¼ 1:3314; μ20 ¼ 1:4081

Simulation experiment design. Each simulation scenario defined by one type of LoS distribution and one discharge rule was run with a warm-up period of 1 year and duration determined by sequential sampling to achieve half-width confidence intervals of less than 0.01 for the percentage of rejected patients and less than 0.005 for the service rates. The time span of the simulations was around 100–150 years and a few minutes were needed to run each scenario. Tables 3 and 4 and Figs. 11, 12, 13 and 14 illustrate the correspondence between the theoretical analysis and the

F. Mallor et al. 1.0

1.0

0.8

0.8

0.6

0.6

0.4

0.4

0.4

0.2

0.2

0.2

0.8

0.6

0.5

1.0

1.5

2.0

2.5

3.0

0.5

1.0

1.5

2.0

2.5

0.5

3.0

0.70

1.0

0.65

0.8

1.0

1.5

2.0

2.5

3.0

2.0

1.8

1.6

0.6

0.60 1.4

0.4

0.55 1.2

0.2

0.50

1.0 0.5

1.0

1.5

2.0

2.5

3.0

0.5

1.0

1.5

2.0

2.5

3.0

0.5

1.0

1.5

2.0

2.5

3.0

Fig. 9 Distribution function (upper row) and Mean residual life (lower row) for Lognormal distributions with mean 1 and σ=1 (left), 0.5 (centre), 0.2 (right)

simulation results. Specifically, we point out the following findings from the comparison of a general LoS distribution with the exponential assumption:

DFR (as Weibull distributions with β1) leads to lower service rates, longer expected LoS and a higher patient-rejection probability. Similar behaviour is observed for Lognormal distributions with σ=0.2 and σ=0.5 and for phase-type distributions. Application of rules T3b and T3c to LoS with DFR (as Weibull distributions with β1), Lognormal distributions with σ=0.2 and σ=0.5 and the phase-type distributions are considered. The opposite behaviour is observed for LoS with

Fig. 10 Phase-type distribution diagram

In this paper, we have used mathematical and probabilistic tools to analyze a healthcare service system control problem, where the objective is to balance the number of patients served and the quality of service provided. The purpose of this theoretical analysis was to obtain optimal control strategies. The mathematical solution is convincing from the medical perspective, because it reflects standard ICU management practices. Extra effort to speed up patient discharge is reserved for times of high bed occupancy and demand pressure. In this way, staff are not continually required to maintain an abovenormal pace of activity, which is commonly associated with a higher frequency of medical error [35]. The control problem admits other mathematical formulations, leading to different optimal solution patterns (see [31]). Nevertheless, the control rules proposed in this paper are designed to approximate any set of target service rates, regardless of how these rates are found, and they are assessed according to their ability to reproduce them when applied to control bed occupancy level.

Entry

S1

S2

S3

S4

Optimal control of ICU patient discharge Table 3

Application of rule T3b to general LoS distributions E[LoS] 0.937

Stnd. dev. 0.937

% reject 3

μ10 1

μ14 1

μ15 1

μ16 1

μ17 1.045

μ18 1.210

μ19 1.331

μ20 1.408

Weib (β=1) Weib (β=0.5) Weib (β=0.9) Weib (β=1.1)

0.937 0.882 0.932 0.940

0.941 1.825 1.034 0.865

2.997 1.500 2.855 3.112

1.000 1.026 1.000 0.999

1.000 1.053 1.004 0.996

1.001 1.069 1.006 0.996

1.000 1.100 1.008 0.994

1.044 1.197 1.056 1.038

1.214 1.426 1.227 1.200

1.333 1.591 1.351 1.318

1.407 1.621 1.419 1.395

Weib (β=2) Weib (β=5) LogN(σ=0.2) LogN(σ=0.5) LogN(σ=1) LogN(σ=2) Phase (q>0) Phase (q=0)

0.954 0.960 0.961 0.954 0.924 0.796 0.969 0.971

0.525 0.273 0.251 0.527 1.129 2.884 0.633 0.582

3.552 3.734 3.745 3.554 2.735 0.458 4.226 4.258

1.000 0.992 0.992 0.993 1.013 1.148 0.999 0.996

0.986 0.980 0.979 0.988 1.020 1.216 0.986 0.983

0.981 0.973 0.973 0.983 1.021 1.249 0.976 0.974

0.973 0.965 0.962 0.974 1.021 1.302 0.959 0.957

1.008 0.996 0.998 1.008 1.061 1.431 0.976 0.975

1.161 1.146 1.146 1.157 1.226 1.733 1.106 1.105

1.273 1.261 1.261 1.274 1.351 1.949 1.202 1.200

1.356 1.343 1.344 1.355 1.438 2.027 1.288 1.283

The theoretical solution does not provide a healthcare system management guideline in terms of which patient to discharge and when, in high occupancy scenarios. We studied three types of rules for the practical implementation of the theoretical solution. These rules, based on intuitive interpretations of the theoretical solution, are applicable to each patient individually. The first type of rules implies the discharge of the patient with largest ratio of LoS consumed in the healthcare system. The rule makes some sense medically but is technically unsound because it fails to reproduce the service rates set by the theoretical solution. Decision-making based on time remaining to full recovery leads to the type 2 set of rules. The first rule of this type, which we call rule T2a, also makes some sense medically, but it also fails to reproduce the target service rates. A probabilistic analysis of this rule suggested a modification, in the form of rule T2b, which is able to attain

Table 4

the collective service target from the individual decisions but makes little sense medically. Finally, two of the three typethree rules are found both medically coherent and able to achieve target service rates. These rules are based on the immediate discharge, with a specified probability, of one patient upon the arrival of a new ICU patient. In particular, the rule that discharges the patient that has spent the longest time in the ICU (rule T3c) makes sense medically when LoS follows a NBUE-type distribution. This rule describes the real workings of ICUs under full occupancy. When a free bed is required, it is necessary to evaluate all the patients in the ICU and select the one whose health status would best allow safe transfer to a less critical care unit (this could correspond to state 3 of the phase-type distribution considered in section 4). This decision, however, tends to be taken when the need for a free bed is imminent and occupancy is high. The medical literature

Application of rule T3c to general LoS distributions E[LoS] 0.937

Stnd. dev. 0.937

% reject 3

μ10 1

μ14 1

μ15 1

μ16 1

μ17 1.045

μ18 1.210

μ19 1.331

μ20 1.408

Weib (β=1) Weib (β=0.5) Weib (β=0.9) Weib (β=1.1) Weib (β=2)

0.937 0.852 0.928 0.944 0.973

0.840 1.455 0.912 0.780 0.483

3.001 1.054 2.733 3.212 4.296

1.000 1.065 1.001 0.997 0.998

1.001 1.104 1.007 0.993 0.981

0.999 1.124 1.012 0.992 0.971

0.999 1.163 1.015 0.990 0.952

1.046 1.274 1.065 1.030 0.971

1.212 1.524 1.236 1.189 1.101

1.334 1.698 1.365 1.305 1.202

1.408 1.712 1.436 1.388 1.284

Weib (β=5) LogN(σ=0.2) LogN(σ=0.5) LogN(σ=1) LogN(σ=2) Phase (q>0) Phase (q=0)

0.988 0.987 0.964 0.901 0.746 0.969 0.971

0.220 0.190 0.459 0.883 1.976 0.620 0.564

5.219 5.165 3.997 2.211 0.221 4.246 4.256

0.999 1.000 0.995 1.036 1.251 0.998 0.997

0.993 0.992 0.988 1.050 1.332 0.987 0.983

0.980 0.979 0.978 1.054 1.368 0.976 0.974

0.953 0.952 0.964 1.056 1.428 0.958 0.957

0.943 0.947 0.987 1.104 1.576 0.975 0.974

1.042 1.046 1.120 1.279 1.906 1.106 1.105

1.110 1.117 1.226 1.410 2.150 1.203 1.200

1.189 1.191 1.307 1.499 2.243 1.284 1.284

F. Mallor et al. Fig. 11 Service rates with rule T3b applied to general LoS distributions

Sevice rates with Rule T3b applied to general LoS distributions LoS distribs Exponential Weib (b=0.5) Weib (b=0.9) Weib (b =1.1) Weib (b =2) Weib (b=5) LogN (s=0.2) LogN (s=0.5) LogN (s=1) LogN (s=2) Phase (q>0) Phase (q=0)

2,0

Service rate values

1,8

1,6

1,4

1,2

1,0 13

does in fact describe this kind of scenario and its consequences in the form of untimely patient discharge, as an expression of the imminent need for beds. Several papers document a poorer outcome for patients discharged under such circumstances (see for example [21, 28, 36, 37]). As noted in section 3, in a real healthcare system, individual discharge decisions are based on the medical condition of the patient, which is given by various indicators. It can be assumed, however, that recovery is not an immediate event but a gradual process that takes time, and the percentage of time remaining for full recovery can therefore be used as an indicator of patient health status. Analysis of the mathematical solution and its implementation through the set of rules for individual discharge shows that intervention to accelerate patient discharge occurs, not just at times of maximum occupancy, but as soon as occupancy exceeds moderate levels, thereby leading to closer patient supervision, which could help to prevent untimely Fig. 12 Service rates with rule T3c applied to general LoS distributions

14

15

16 17 Bed occupancy

18

19

20

discharge and allow more time for preparation of the receiving ward. According to the relative frequency of this type of discharge, the hospital organization could benefit from alternative types of care provision, such as intermediate care units or the deployment of ICU professionals in normal wards [38]. The control problem addressed in this paper could be extended to include systems such as these intermediate care facilities in order to assess their potential contribution to the quality of the health service. These new intermediate wards are viewed by physicians [24, 38, 39] as a way to reduce the stress under which medical staff have to work and as a potentially necessary change in patient care practices (an increase in the number of beds to lower the service rate) aimed at preventing medical errors due to work overload. Moreover, although the control problem has been contextualized in an ICU environment, it could be adapted to the analysis of other health care resource and service management issues.

Sevice rates with Rule T3c applied to general LoS distributions 2,4

LoS distribs Exponential Weib (b=0.5) Weib (b=0.9) Weib (b =1.1) Weib (b =2) Weib (b=5) LogN (s=0.2) LogN (s=0.5) LogN (s=1) LogN (s=2) Phase (q>0) Phase (q=0)

2,2

Service rate values

2,0 1,8 1,6 1,4 1,2 1,0 13

14

15

16 17 Bed occupancy

18

19

20

Optimal control of ICU patient discharge Fig. 13 Rejected patients (%) with rules T3b and T3c applied to general LoS distributions

Rejected patients (%) with Rules T3b and T3c Weib(b=5) LogN(s=0.2)

5

% rejected

4

3

LogN(s=0.2) Weib(b=5) LogN(s=0.5) Weib(b =2) Phase(q=0) Weib(b =1.1) Expo Weib(b=0.9) LogN(s=1)

Weib(b =2) Phase(q=0) LogN(s=0.5)

Phase(q>0)

Phase(q>0)

Weib(b =1.1) Expo Weib(b=0.9) LogN(s=1)

2 Weib(b=0.5) Weib(b=0.5)

1 LogN(s =2)

LogN(s =2)

0

Ru le

T3b

literature reports that these Markovian hypotheses often fail to hold in real healthcare systems (see for example [40, 41]). The extension of the control problem to non-Markovian settings has therefore been analyzed through the consideration of more realistic LoS distributions. Although queuing theory does not provide theoretical solutions to all queuing models, we have assessed the validity of the rules based on the mimic of the service rates provided by the Markovian analysis in the nonMarkovian case. A theoretical analysis enabled us to explain under which conditions they provide a good approximation to the optimal control. We have performed tests by using LoS distributed as a Weibull, Lognormal and Phase-type distributions. In the Weibull case the shape parameter is used as a measure of the skew from the Markov assumption. Simulation results show that slight deviations from the Markovian assumption, that is, values of the shape parameter close to one, produce slight deviations from the target service

Mathematical modelling of individual health-status would enable individual discharge decisions based on health indicators, rather than on expected time remaining and time already spent in the health system. A naïf approximation to this type of modelling is depicted in Fig. 2. In section 4 we have considered the phase-type distribution as a suitable stochastic model of the underlying dynamics of the recovery process. In particular, future adaptations of this study might use more detailed patient discharge assessment scales, such as the SOFA (Sequential Organ Failure Assessment) or workload assessment scales such as the NEMS (Nine Equivalents of Nursing Manpower Use Score) or the TISS (Therapeutic Interventions Scoring System), which the medical literature documents as having been used to monitor mortality after discharge from intensive care. The theoretical queuing model assumes the Markovian property for the LoS and for the patient arrival pattern. The

Fig. 14 Expected LoS with rules T3c amd T3b applied to general LoS distributions

T3 c

E[LoS] with Rules T3c and T3b 1,05 1,00

Weib( b =1.1) Expo

0,95

E[LoS]

Weib(b=0 .9 )

0,90

P hase( q>0 ) LogN( s=0 .2 ) Weib(b=5 ) L ogN( s=0 .5 ) Weib( b =2) P hase( q=0 ) Weib( b =1.1) Weib( b=0 .9 ) Expo L ogN( s=1 )

Weib( b=5 ) L ogN( s=0 .2 ) Weib( b =2) P hase( q=0 ) LogN( s=0 .5 ) P hase( q>0 )

LogN(s=1 ) Weib( b=0 .5 )

Weib( b=0 .5 )

0,85

LogN( s=2)

0,80 LogN( s=2)

0,75 0,70

Rule

T3c

T3 b

F. Mallor et al.

rates and also from the target percentage of rejected patients. Service rates are underestimated and rejection probability overestimated for the shape parameter above one (cases where the hazard function is increasing) while service rates are overestimated and rejection probability underestimated for values of the shape parameter below one (decreasing hazard function). More conclusions are obtained in section 4 for Lognormal and Phase-type distributions. More research is necessary to investigate the control problem in cases where patient arrivals follow a general distribution, thus enabling the modelling of different types of patients. This issue constitutes one of the current author’s research topics. Acknowledgments This paper has been partially funded by grant MTM2012-36025. The authors would like to extend their gratitude to the reviewers and guest editor for their insightful comments, which have increased the quality of this paper.

References 1. Zenios SA (2004) Patient choice in kidney allocation: the role of the queueing discipline. Manuf Serv Oper Manag 6:280–301 2. Seshaiah CV, Thiagaraj HB (2011) A queueing network congestion model in hospitals. Eur J Oper Res 63:419–427 3. Solberg B, Asplin B, Weinick R, Magid D (2003) Emergency department crowding: consensus development of potential measures. Ann Emerg Med 42:824–834 4. de Bruin AM, Bekker R, van Zanten L, Koole GM (2010) Dimensioning hospital wards using the Erlang loss model. Ann Oper Res 178:23–43 5. Cochran JK, Barthi A (2006) Stochastic bed balancing of an obstetrics hospital. Health Care Manag Sci 9:31–45 6. Griffiths JD, Price-Lloyd N, Smithies M, Williams JE (2005) Modelling the requirement for supplementary nurses in an intensive care unit. J Oper Res Soc 56:126–133 7. Bekker R, Koeleman PM (2011) Scheduling admissions and reducing variability in bed demand. Health Care Manag Sci 14:237–249 8. Harper P, Gamlin H (2003) Reduced outpatient waiting times with improved appointment scheduling: a simulation modelling approach. OR-Spectrum 25:207–222 9. Vasilakis C, Sobolev BG, Kuramoto L, Levy AR (2007) A simulation study of scheduling clinic appointments in surgical care: individual surgeon versus pooled lists. J Oper Res Soc 58:202– 211 10. Lakshmi C, Sivakumar A (2013) Application of queueing theory in health care: a literature review. Oper Res Health Care 2:25–39 11. Gross D, Harris CM (2008) Fundamentals of queueing theory. Wiley 12. Terekhov D, Beck JC (2008) A constraint programming approach for solving a queueing control problem. J Art Int Res 32:123–167 13. Stidham S (2002) Analysis, design and control of queueing systems. Oper Res 50:197–216 14. Mallor F, Azcárate C (2014) Combining optimization with simulation to obtain credible models for intensive care units. Ann Oper Res 221: 255–271 15. Anderson D, Price C, Golden B, Jank G, Wasil E (2011) Examining the discharge practices of surgeons at a large medical center. Health Care Manag Sci 14:338–347 16. Dobson G, Lee H, Pinker E (2010) A model of ICU bumping. Oper Res 58:1564–1576

17. Shmueli A, Sprug CL, Kaplan E (2003) Optimizing admissions to an intensive care unit. Health Care Manag Sci 6:131–136 18. Capuzzo M, Moreno RP, Alvisi R (2010) Admission and discharge of critically ill patients. Curr Opin Crit Care 16:499–504 19. Kramer AA, Higgins TL, Zimmerman JE (2012) Intensive care unit readmissions in U.S. hospitals: patient characteristics, risk factors, and outcomes. Crit Care Med 40:3–10 20. Marmor YN, Rohleder TR, Cook DJ, Huschka TD, Thompson JE (2013) Recovery bed planning in cardiovascular surgery: a simulation case study. Health Care Manag Sci. doi:10.1007/s10729-013-9231-5 21. Robert R, Reignier J, Tournoux-Facon C et al (2012) Refusal of intensive care unit admission due to a full unit: impact on mortality. Am J Respir Crit Care Med 185:1081–1087 22. Sprung CL, Baras M, Iapichino G et al (2012) The Eldicus prospective, observational study of triage decision making in European intensive care units: part I–European intensive care admission triage scores. Crit Care Med 40:125–131 23. Lin F, Chaboyer W, Wallis M (2009) A literature review of organisational, individual and teamwork factors contributing to the ICU discharge process. Aust Crit Care 22:29–43 24. Sinuff T, Kahnamoui K, Cook DJ et al (2004) Rationing critical care beds: a systematic review. Crit Care Med 32:1588–1597 25. Walter KL, Siegler M, Hall JB (2008) How decisions are made to admit patients to medical intensive care units (MICUs): a survey of MICU directors at academic medical centers across the United States. Crit Care Med 36:414–420 26. Ridge JC, Jones SK, Nielsen MS, Shahani AK (1998) Capacity planning for intensive care units. Eur J Oper Res 105:346–355 27. Costa AX, Ridley SA, Shahani AK, Harper PR, De Senna V, Nielsen MS (2003) Mathematical modelling and simulation for planning critical care capacity. Anaesthesia 58:320–327 28. Sprung CL, Danis M, Iapichino G et al (2013) Triage of intensive care patients: identifying agreement and controversy. Int Care Med 39:1916–1924 29. Task Force of the American College of Critical Care Medicine, Society of Critical Care Medicine (1999) Guidelines for intensive care unit admission, discharge, and triage. Crit Care Med 27:633–638 30. Azcárate C, Mallor F, Barado J (2012) Calibration of a decisionmaking process in a simulation model by a bicriteria optimization problem, In Proceedings of the 2012 Winter Simulation Conference, 782–791 31. Mallor F, Azcárate C, Barado J (2014) Control problems and management policies in health systems. Application to intensive care units. Flex Serv Manuf J. doi:10.1007/s10696-014-9209-8 32. Wolf RW (1982) Poisson arrivals see time average. Oper Res 30: 223–231 33. Heidegger CP, Treggiari MM, Romand JA (2005) A nationwide survey of intensive care unit discharge practices. Intensive Care Med 31: 1676–1682 34. Barlow RE, Proschan F (1981) Statistical theory of reliability and life testing. To begin with, silver spring, MD 35. Steyrer J, Schiffinger M, Huber C et al (2013) Attitude is everything? The impact of workload, safety climate, and safety tools on medical errors: a study of intensive care units. Health Care Manage Rev 38: 306–316 36. Metcalfe MA, Sloggett A, McPherson K (1997) Mortality among appropriately referred patients refused admission to intensive-care units. Lancet 350:7–11 37. Simchen E, Sprung CL, Galai N et al (2007) Survival of critically ill patients hospitalized in and out of intensive care. Crit Care Med 35: 449–457 38. Niven DJ, Bastos JF, Stelfox HT (2014) Critical care transition programs and the risk of readmission or death after discharge from an ICU: a systematic review and meta-analysis. Crit Care Med 42:179–187

Optimal control of ICU patient discharge 39. Lyons RA, Wareham K, Hutchings HA et al (2000) Population requirement for adult critical-care beds: a prospective quantitative and qualitative study. Lancet 355:595–598 40. Rauner MS, Zeiles A, Schaffhauser-Linzattti MM, Hornik K (2003) Modelling the effects of the Austrian inpatient

reimbursement system on length-of-stay distributions. ORSpectrum 25:183–206 41. Vasilakis C, Marshall AH (2005) Modelling nationwide hospital length of stay: opening the black box. J Oper Res Soc 56:862–869