Behav. Res. & Therap).
1975. Vol. 13. pp X7- 236. Pcrgamon Press Prmted m Great Brttam
MEASURING
THE QUALITY
OF RESIDENTIAL
CARE*
RON WHATMORE, LYN DURWARD and ALBERT KUSHLICK wessex Regional Health Authority, Romsey Road, Winchester, (Rrceiaed
Hampshire.
England
19 July 1974)
Summary-Administrative personnel require objective, reliable and valid information in order to monitor the effects of allocating resources to different components of the service that they are providing. The initial problem is to define and measure the dependent variable-the ‘quality of care’ -against which the effects of changing various independent variables can be evaluated. The paper describes an attempt to use the operant conditioning model as a basis for constructing such a measure of the quality ofcare provided in residential settings for severely mentally handicapped people. The rationale for adopting this model is described. against a background of other studies in which the same problem has been addressed. The hypothesis. the tasks generated, and the methodological problems encountered. are outlined and discussed. A subsequent paper will present the resuhs obtained when using the method to measure the quality of care, so defined, in two residential units for severely mentally handicapped children.
personnel of medical and social welfare organisations (e.g. hospital officers, local authority officers) are responsible for the allocation and distribution of vast financial, material and manpower resources to large populations ‘at risk’. Government policy statements require that they execute these functions in a manner which maintains or raises the ‘quality’ of each of the services provided. Thus, the administrator faces two immediate questions: (1) How do I ‘know’ whether the quality of care is high or low? (2) Of the many factors determining the quality of care which ones have the most significant effect?? These two general questions define the general research task. In this case, it is applied to residential settings for severely mentally handicapped people. Administrative
The concept of ‘quality of residential care’
The first task is to specify what the ultimate benefit to the client should be, given that the residential unit is providing ‘high’ quality care. We decided that the major goal should be that the clients make ‘progress’, i.e. (i) acquire new and/or maintain existing defined ‘appropriate’ behaviour (e.g. social skills like feeding/washing/dressing self). (ii) not lose such appropriate social skills. (iii) lose defined ‘inappropriate’ behaviour (e.g. anti-social skills like pulling hair, biting people, destroying furniture). Given these as the top priority goals, the second task is to focus on aspects of the daily living environment, in order to identify, and then quantify, those factors which appear to determine the probability that clients will make progress. Recent attempts to quantify the quality of residential care have avoided the use of such traditional criteria as cleanliness (absence of dirt or germs), tidiness (absence of physical disorder), and social order (e.g. absence, at any time, of noise), and have concentrated on factors in the social environment believed intuitively to be related to client welfare. Moos (1972) for example, has attempted to quantify the ‘environmental climate’ in residential and other settings. Starting from Goffman’s (1961) descriptions of ‘total institutions’, King, Raynes and Tizard (1971) have developed a scale which operationally defines and measures staff practices relating to the management of children in residential settings. This *This paper covers one section of an ongoing research project aimed at evaluating services delivered to severely mentally handicapped children and their families. For detailed accounts of other sections of the research, see Kushhck (1972) and Kushlick and Cox (1973). t The paper mainly covers the first of these questions. The second question, necessarily dependent upon an answer to the first. is the subject of subsequent research. 227
228
R. WHATMORE. L. DURWARD and A. KUSHLICIG
‘child management scale’ measures the extent to which residential units ‘depersonalise’ and ‘block treat’ the children, and to which they manifest ‘rigidity of routine’ and ‘social distance’. A unit in which these negative practices are avoided is judged to be providing ‘high quality care; ‘low’ quality care is provided in a unit in which these practices occur. In our early approacheswe followed King, Raynes and Tizard’s formulations. During our detailed observations of daily routines on living units (i.e. the sequences of staff and client activities throughout the waking day), however, it became clear that residential units in which these negative management practices are avoided do not necessarily provide conditions under which their clients make ‘progress’ or fail to ‘regress’. This arises from the types of management practices which are measured in the scale. Some practices appear to bear little relation to factors which are likely to influence progress. for example. whether ‘there are set times when visitors can come to the unit’. Other practices are more closely related, for example, ‘block treatment’. The relevant scale questions probe the extent to which the children ‘wait around’ before and after important social activities like mealtimes, bathtimes, etc; ‘social distance’ questions probe the extent to which staff are accessible to the children, e.g. at mealtimes, during TV times. etc. However, one cannot infer from lack of ‘waiting around’ and ‘presence of staff that (a) staff and children are interacting, or (b) that the effect of the interaction will be to increase the children’s social skills. Also. the scale questions do not probe long periods of the day when interactions between children and staff may or may not take place. A unit in which children are left alone acquiring a large range of ‘inappropriate’ behaviours (e.g. faeces smearing during recreation periods) could still obtain a high rating on the management scale. In brief, then, although the management practices scale measures important features of the residential environment (i.e. the extent to which a unit might appear ‘homely’ to an observer) it appears to lack sensitivity to staff practices directly affecting individual clients. The model We have therefore attempted to develop a more sensitive measure in which the components are directly related to client progress. We have found that the conceptual framework of operant conditioning (Skinner, 1953) has many advantages. The framework clarifies the dependent variable-frequencies of specific responses classes-as well as the independent variables (conditions) under which these occur; it allows the study of directly observable behaviour measured in frequencies (Bijou rt al.. 1969); the efficacy of procedures developed from the framework has been demonstrated experimentally in many practical settings (Ulrich, Stachnik and Mabry. 1966, 1970); and a sophisticated technology for collecting, analysing and interpreting data has been developed (Bijou and Baer, 1966). The hypothesis The central premise of operant conditioning theory is that much behaviour is operant behaviour. The characteristic of such behaviour is that it is controlled by stimulus consequences. The hypothesis is that most client behaviour exhibited in the residential setting is operant in nature, that is, its frequency of occurrence is determined by stimulus consequences. Furthermore, we hypothesise that of all such possible stimuli. the class most likely to determine the frequency of specific classes of client behaviour is staff attention. Thus. classes of client behaviour followed by staff attention are likely to increase or be maintained in frequency; classes not followed by such attention are likely to be extinguished. This further defines the research tasks. Firstly. classes of client behaviour must be classified into those which (we judge) clients should acquire, and those which they should not.* Secondly, staff attention must be clearly differentiated from its opposite-absence of staff * The researcher must make this judgement where the client cannot verbalise and where staff and parents or others have not done so. There are now many individual studies where clients and their relatives have chosen the behavioural target.
Measuring the quality of residential care
229
attention. These are discussed in the following sections, in the context of the methodological problems arising from the use of the operant model. The ~et~odo~ogicul problems There are important differences between the approach of the behaviour modification experimentalist investigating changes in behaviour and other approaches to the evaluation of services (e.g. that of King, Raynes and Tizard). Since we are attempting a sort of ‘marriage’ of these two models, specification of the differences between them will help to ciarify the problems that arise from this union. They can be summarised as follows: Table I. Differences between the operant conditioning approach and that used in this research Operant approach Aims To effect a change in the frequency of specific classes of behaviour exhibited by one, or a small number of, individuals, and to demonstrate a functional relationship between specific dependent variables (classes of behaviour) and specific independent variables (stimulus consequences). Design (a) Individual or small homogenous groups of subjects. (b) Relatively short observation periods. (c) Limited number of ‘target’ classes of behaviour specified. (d) Use of subject as his own control (reversal or multiple baseline design) in most studies. (e) A priori functional relationship demonstrated.
Our approach
To derive a score, or series of scores which are inferred as representing the abstract concept ‘quality of care’ provided to a large number of individuals in residential care, and to investigate the relationship between this variable and general environmental variables (e.g. staffing ratio, type of training of caring staff, etc.) (a) Large heterogeneous groups of subjects. (b) Observations undertaken over the whole of the waking day. (c) All behaviours of all subjects must be classified. (d) Use of matched group of subjects (control group design) (e) A posteriori relationship suggested by means of statistical manipulations.
Assumptions No assumptions made about what is ‘reinforcing’ or ‘punishing’ to the subject.
Assumptions made about the reinforcing nature of different types of staff attention.
Degree of control High degree of control over independent variables.
No control over inde~ndent variables.
Research aim and questions Senior administrators (i.e. those having little day-to-day contact with the residential unit) require an answer to the following question ‘How can we manipulate resource and other variables in order to create settings for both staff and residents such that optimal progress of clients will take place? An answer in the form that manipulating variable X (e.g. staff attention) produces a change in the dependent variable Y (decrease in hitting behaviour) for child F with teacher A in situation B can be of limited relevance in occasioning their responses. The information, i.e. the measures that they require, must relate to the behaviour of groups of clients, of different levels of ability, and to the distribution of staff attention between different classes of client behaviour, throughout the whole of a waking day. Classijcation of client behauiour I. The D.Z.N.A. categories. Initially a two-category classification was devised-inappropriate client behaviour (e.g. hitting other children, destroying furniture, etc.) and appropriate client behaviour (all other behaviour), This is a very similar classification to those
230
R. WHATMORE.L. DURWARD and A. KUSHLICE
used in many behaviour modification studies (e.g. Peterson. Cox and Bijou, 1971; Herbert and Baer, 1972). In these two studies, the authors were able to pre-define the specific target behaviour of the individual subjects. In both studies, a task. defined in behavioural terms, was specified. By definition. their subjects would always behave either appropriately or inappropriately (‘on task’ or ‘off task’). Since the observation periods were short (4 hr or 1 hr), it was reasonably easy to set up a task-orientated setting with the undivided attention of one adult. The situation in a residential setting is very different. There are few defined tasks. particularly during certain activity-periods like recreation. Even at mealtimes. the clients may spend 15-20 min ‘waiting’ while staff prepare and distribute the food. During such no task periods it is inaccurate to describe the behaviour of a client as appropriate (‘on task’) or inappropriate (‘off task’) if he is, for example, simply sitting in a chair. This system, therefore, did not adequately describe all client behaviour in such a setting. In addition, the scores obtained did not differentiate between residential units in which, on the one hand. a client undertook or was cued to undertake a task by himself (e.g. using a spoon to feed himself) and, on the other. one in which the same client was passively ‘processed’ (e.g. fed by a member of stall). Hence. the former setting in which the client was beginning to learn new skills, would obtain the same score on the measure of ‘quality of residential care’ as the latter setting, in which a comparable client was denied the opportunity to learn such skills. Furthermore, a setting in which clients had more independent-behaviour could score lower (poorer quality) on the measure than a setting in which clients emitted fewer such skills, e.g. a client who fed himself might receive little staff attention during a mealtime, yet a client who had to be fed received some attention (by definition) whilst being fed. This arose because the category of appropriate behaviour included passive behaviour as well as that requiring active participation on the client’s part. It was thus unlikely to provide a valid index of ‘progress’. Another category was therefore introduced into the classification-‘neutral’ behaviour. This category includes passive behaviour (sitting on a chair, being fed by a member of staff), well-established skills (e.g. walking for a completely mobile person) and active behaviour having very little impact on the physical environment (e.g. rocking, bizarre hand movements). The category of appropriate behaviour is reserved for ‘adaptive’ behaviour in which it is likely that new skills will be acquired and/or maintained. This ensures that a setting in which a client is encouraged to undertake an activity by himself and is attended to during this time, will always obtain a higher score on the measure than a setting in which he has things done for him by a member of staff. A further problem was encountered because we had included a variety of behaviour in the ‘inappropriate’ category. For example, it included behaviour of relatively small impact on the environment such as ‘throwing a cloth doll’ through to more disruptive behaviour like ‘biting other children’. The same score on the measure might therefore be obtained from staff responses to very different types of ‘inappropriate’ child behaviour. To differentiate between these scores, we divided the inappropriate category into two separate categories-disruptive, which included behaviour that had a major disruptive impact on the social and physical environment, e.g. hitting other persons, playing with faeces, breaking glass; and inappropriate, which included behaviour that had a minor impact on the social and physical environment, for example, sucking or chewing toys. There are therefore four classes in the final system: ‘Disruptive’ behauiour is any behaviour
which is likely to, or does, cause physical damage, to the child or other persons (biting, hitting self or others), or which causes physical damage to furniture and fittings (pulling down curtains, pictures, etc.). ‘Inappropriate behaviour is any behaviour which is socially disruptive (screaming) or which is less physically damaging to the physical environment (sucking, chewing toys, destroying dolls, etc.). ‘Neutral’ behauiour is any behaviour which is totally passive (sitting on chair, etc.); which has very little impact on physical environment (bizarre hand movements, rock-
Measuring
the quality
of residential
care
231
ing); which is a well-established skill (e.g. walking for a completely mobile child). ‘Appropriate’ bekaviour is any active ‘adaptive’ behaviour in which it is likely that new skills will be acquired and existing skills maintained. These are abbreviated as D, I, N and A respectively. An example of the coding system is given in Appendix A. 2. Mobility categories. It has also been necessary to categorise degrees of mobility of the client. The degree of independent mobility of a client sets limits on the extent to which the client can contact staff members. There are three categories: (a) Completely mobile (CM) persons-those who walk unaided at least 10 yards on a level surface. (b) Partly mobile (PM) persons-those who walk unaided for less than 10 yards on a level surface: those who propel themselves along in some way on a level surface, e.g. walk with help, crawl, pull self along in wheelchair; those who sit up unaided for at least 10 minutes. (c) Non-mobile (NM) persons-those who sit up unaided for less than 1Omin; those who do not sit up unaided; those who do not propel themselves along a level surface at all. The categories for ‘completely mobile’ and ‘partly mobile’ are very similar. The only difference is that classes of behaviour which are prerequisite to walking (e.g. standing, sitting up) plus walking itself. are defined as appropriate for the ‘partly mobile’ client, and neutral for the ‘completely mobile’ client. For the ‘partly mobile’ client, mobility skills constitute an essential basis for the development of other skills and hence should be reinforced. For the ‘completely mobile’ client, however, such skills have long been well developed and are unlikely to be extinguished if not reinforced, as long as the opportunity for emitting them is not restricted (i.e. residents are not restrained in chairs or confined to bed). Non-mobile clients present more difficult problems for behaviour classification. By definition, they emit very little behaviour. Hence, for these clients, small movements (for example, of hands, arms. legs, etc.) are defined as appropriate. These would be the classes of behaviour which a physiotherapist. for example, would identify and reinforce in such a client. Clearly, one classification cannot cover the whole range of behaviour exhibited by the clients. If walking was defined as appropriate for the completely mobile client many would be behaving appropriately 90-100 per cent of the time. The same effect would arise if any minor movement was defined as appropriate for the ‘partly’ and ‘completely mobile’ client. The resulting quality of care scores would be most unlikely to predict progress sensitively. The classzjicatiorz of stafShehariou~
As described earlier. staff behaviour is classified into two categories-attending to or not attending to the client. Staff attending behaviour is further defined as any verbal and/ or physical contact with the client. The rules for specifying verbal and/or physical contact under several sets of conditions are defined. The codes obtained by using the method have proved highly reliable. The general hypothesis (p. 228) can now be stated in more precise terms: that the quality of residential care is ‘high’ if. among a group of clients, a high proportion of their appropriate, and a low proportion of their neutral, inappropriate and disruptive behaviour is followed by staff contact. The converse follows in units providing ‘low’ quality of care. ClassifJGng spec$c
clierlt hehaciour us D.I.N.A.
Difficulties have arisen in coding a given behaviour in the same class (D, I, N or A) on each occasion that it occurs. An important requirement of the codes is that they should be relatively easy to use, that is, it should be possible for a relatively naive observer, after a short period of training, to code client behaviour in the natural setting with a high degree of reliability. The reliability is likely to be higher the simpler the discriminations required of the coders. Thus, it would be an advantage if a specific topography of a behaviour was always
232
R. WHATMORE,L. DURWARD and A. KUSHLICK
coded in the same category. e.g. if ‘screaming’ were always an inappropriate behaviour, irrespective of the situation in which the client screams. If this is not the case, and ‘screaming’ were classified as, say, neutral during recreation times, but inappropriate during mealtimes, the observer would have to code that behaviour differently in the different activity periods. Failure to code differentially according to the activity period might be detected in reliability checks. If not, this will lead to erroneous conclusions from the analysed data. For most behaviour. this presents no problem. However. there are important exceptions, e.g. sitting. For all clients, ‘sitting’ is a neutral behaviour-it is neither active, nor does it constitute acquiring a new skill. However. during a mealtime ‘sitting’ is clearly one precurrent behaviour necessary to the task in hand--eating at the table. It could therefore justifiably be classified as appropriate during this activity-period; and ‘being out of place’ classified as inappropriate. At this point, we were faced with two alternatives. Either to code ‘sitting’ during a mealtime differently to ‘sitting’ during other activity-periods, and risk the possibility of coding errors, or to code ‘sitting’ the same irrespective of the activity-period, and risk the possibility of lowering the sensitivity of our measure. The latter possibility arises because if ‘sitting’ is classified as neutral behaviour during mealtime. the measure would not differentiate between a unit in which all the clients sat at the table throughout the meal, and one in which all the clients got up from the table and wandered around the dining room. We decided to take the latter alternative, for two reasons. Firstly, there are many appropriate types of behaviour at the mealtable other than ‘sitting’, e.g. feeding self, picking up salt/pepper, chatting, etc. If units differ on the extent to which these types of behaviour are exhibited and responded to by members of staff, the measure will be sensitive to these differences. Secondly, from unsystematic observation in residential settings, it appears that most clients readily sit down and remain seated for the whole of the mealtime. Therefore, this may not be an area in which the measure is required to be sensitive. (The validity of this impression will be tested by examination of the verbal description of client behaviour during mealtimes. The description is written down by observers at the same time as they code client behaviour and staff-client contact). Having stated that ‘sitting’ behaviour is classified as neutral at all times, there is one type of ‘sitting’ which creates another, quite separate, coding problem. This is ‘sitting’ on the toilet/pot. The problem arises because the two, central, appropriate aspects of behaviour in this task-urinating and defaecating-are unobservable, without the aid of special, technical apparatus which is impractical to use in our research setting. The closest observable approximation to these two classes of behaviour is the ‘sitting on the toilet’. Therefore, unlike other types of ‘sitting’. this is classified as appropriate behaviour. Similar difficulties have arisen over the classification of behaviour which is, in itself, appropriate, but is emitted at inappropriate times, e.g. getting into a bed or bath in the middle of the morning without staff instruction or consent. Again, the use of ‘time of day’ or ‘instruction’ as a stimulus event (i.e. discriminative stimulus) to occasion client responses results in the same behaviour being classified as appropriate during some activityperiods, and neutral or inappropriate during others-with the same implications for the reliability of the codes and possibly erroneous interpretation of data. We decided that stimulus events associated with ‘place’ (e.g. the room or area in the building) are as, or more, likely to control client response than those associated with ‘time of day’. Furthermore, it may also be more important for the client to learn to get into a bath in the bathroom, to get into a bed in the bedroom, rather than to learn to emit these behaviours only at particular times of the day. As can be seen from the two examples given, this has the additional advantage of ensuring that such behaviour is always classified the same, since the material equipment necessary for the emission of the behaviour is always in the same place. One can only get into a bath in a bathroom, and this behaviour in this room is, by definition, appropriate, no matter what the time of day. There is only one class of behaviour in which this is not the case-‘undressing’. Since a client can undress in any room, this behaviour is appropriate only in the bedroom/bath-
Measuring
the quality
of residential
room/toilet area. In any other room, it is (in most circumstances) disruptive to be partly or completely undressed. Dejning
233
care
inappropriate
or even
target hehavioul
The behaviour modifier is in a position to define the target behaviour in consultation with the staff/teacher, etc. Thus, the experimentalist and the practitioner agree on what behaviour is ‘appropriate’ and ‘inappropriate’. To obtain such agreement in our situation would require discussion with hundreds of members of staff, and it is doubtful if such agreement could be obtained. There are two alternatives. Either to construct a different set of codes for each residential unit (assuming a consensus could be arrived at on each unit), or to decide on our own codes after consulting certain members of staff who care for the clients on the living units. We have taken the latter alternative since without using the same codes between units, it becomes impossible to make inter-unit comparisons. However, the latter course of action gives rise to some peculiarities. For example, if a client stands on a table, this is classifiedas inappropriate. Yet, in order to dry a client with a towel, a member of staff may stand him on a table, which is clearly, in the staffs classification, not an inappropriate client behaviour. Exceptions could be made where it is clear that staff have instructed a client to behave in a specific way, but this would increase the complexity of an already complex set of codes. It would also raise further problems in interpreting the quality of care score between residential units. Activity periods
Within any one living unit, the quality of care may well vary at different times of the day. This is especially likely to occur where living unit staff concentrate on particular classes of behaviour, like a comprehensive mealtime or toiletting programme (Foxx and Azrin, 1973) and ignore other behaviour. It is therefore important before deciding to concentrate observations on a short period of the day to establish whether and to what extent variation in staff practices occurs. Since practices must vary at certain key periods of the day, a method is required for dividing the day into such meaningful periods. ‘Natural’ divisions occur on the basis of the common activity in which most clients or sub-groups of clients engage at the same time. These we have called ‘activity-periods’. There are three such activity-periods (each occurring more than once throughout the day) -mealtimes, recreation, and toiletting/washing/dressing. The identification of the period with respect to an individual being observed is sometimes difficult within any one group of clients. They may not all be engaged in the same activity at any given time (e.g. a resident may be waiting to be fed); they may not all be in the same room (residents who have been fed may be taken through to the playroom whilst the remainder of the group are fed); some may be engaged in a different individual activity (e.g. being given physiotherapy whilst the remainder of the group are being washed). Assumptions
The behaviour modifier makes no assumptions about the reinforcing or punishing effects of consequences to the clients’ behaviour. These are carefully tested by quantifying the change in frequencies of the specific consequated behaviour. Furthermore, by simultaneously recording the frequency of two classes of behaviour-one of which is consequated and the other ignored-the specificity of a reinforcer can be checked. In our research setting, it is not possible to test which particular stimuli are reinforcing and which are punishing to each of the 250 clients-indeed, these may well change over a period of time for any given individual. It is therefore necessary to make certain assumptions about which stimuli are likely to reinforce and which to punish the behaviour of clients in general. Our hypothesis focuses on contact with a staff member as the stimulus consequence most likely to have a major effect in determining the frequency of the clients’ responses.
234
R. WHATMORE. L. DURWARD and A. KUSHLIC~
There are many ways of classifying such contact. For example, we could differentiate between topographies of contact, and predict that some (e.g. shouting, smacking) will be punishing, and that others (e.g. praising. cuddling) will be reinforcing (for examples of this approach see the work of Buehler. Patterson and Furniss (1966). and Gefland. Gefland and Dobson (1967)); or we could assume that all contact is equally reinforcing (or punishing). From observations of staff-client contact in the residential settings. it appears that the frequency of such contact is very low for the majority of the clients. The exceptions are clients who exhibit very severely disruptive behaviours (pulling hair. throwing chairs. spitting. etc.). Their contact frequency is high because staff must intervene to prevent damage to persons and property. Furthermore. a/?~ staff contact--even severe verbal and/or physical reprimands which are normally’ assumed to have punishing effects-appears to maintain the behaviour of these clients. (There is also some research evidence for this. e.g. Allen and Harris, 1966.) Given this type of reinforcement history-either very little staff contact. or contact contingent upon inappropriate and/or disruptive behaviour-it seems reasonable to assume that one topography of contact is as likely to be reinforcing to clients, in general. as any other. Time sampling method
We have to observe many clients (250 approximately in 10 different living-units) in order to obtain a score of quality of care that living-units provide for those clients. This involves sampling clients’ behaviour. and the time sample of any one individual is necessarily small because of the large numbers involved. We have therefore to determine which particular methods of time sampling will give a representative picture of a group of individual clients. At present a maximum of four clients are observed by one observer. Client A is observed for 15 set, with a further 15 set used for writing down rough descriptions of the classes. then client B, and so on. Analysis has shown that, provided that the proportion of staffclient contacts and of one class of client behaviour (D.I.N.A.) is not extremely low (not less than 5 per cent) and provided that the observation period is reasonably long (approximately 4 hr or more), this method yields a representative sample. However. there are practical problems with this method. In switching observation from one client to another (even if the clients are in the same mobility category) every 30 set, the observer may have to start recording a different class of client behaviour and/or of staff-client contact. This rapid changing can be a source of unreliability. More important, in order to obtain the same number of observations on each client, those four clients would have to remain together in the same vicinity. This rarely happens; often one or two leave the vicinity, in which case, observations on them will be lost for as long as they are away. This raises problems in the analysis, since. without considerable re-arrangement of the data, the scores will be weighted towards those of clients observed more frequently. One way of overcoming these difficulties would be to use a 5 min-in-20 min time sample method. Thus, an observer would only have to switch observations every 5 min, rather than every 30 set, and she could also follow an individual client who moved to another part of the building, returning to the group for the 5-min observation on the next client. The one limitation may be that such a method, which samples larger ‘blocks’ of observations on each client may not be representative. A pilot study is being designed to test this. Co~ltrol over independent
variuhles
In a paper on the methodology of behaviour modification studies, Bijou et al. (1969) state that in a nursery school setting, “the absence of the head teacher. or the presence of the child’s mother would probably vary the situation enough to preclude data collection. Marked reduction in class attendance due to illness. inclement weather, or holiday preparations at home would also be considered a cardinal alteration in the field situation”. In preliminary field studies that we have undertaken, it seemed impossible to define a standard or normal situation, as described by Bi_jouet al. This is obviously partly because our observations extend over the whole of the waking day. At different times and on differ-
Measuring
the quality
of residential
care
335
ent days, different staff would be on duty: staff members would be absent; volunteers, relatives, the physiotherapist, senior nursing personnel, would visit. etc. However, even if some days happen to be more ‘standard’ than others. it would be of little help because the basic question that we are attempting to answer is different to that of Bijou et al. We want to know what learning opportunities are available to a client throughout his waking day on a particular living-unit. ‘Non-standard’ days (depending on their frequency) may therefore be just as important as ‘standard’ days. For an NM client. what happens on a day when the physiotherapist is absent is as important as what happens when she is present. Quality of care is an amalgam of what happens throughout a series of days during which the independent variables may change considerably. In this sense. there are no ‘standard’ or ‘non-standard’ days. Our measurement of ‘quality of care’ must yet ignore other sophistications and subtleties which are part of the operant model (e.g. Lindsley. 1964). Skinner (1953) emphasises the complex differential effects of various reinforcement schedules (continuous or intermittent). Wahler (1972) describes the influence of ‘setting events’, and Bijou and Baer (1961) discuss the function of ‘discriminative stimuli’. In the classical intervention design. the behaviour modifier can specify beforehand what type of reinforcement schedule and what type of discrimination (if any) are to be used. He can also control such setting events as the degree of deprivation and satiation of reinforcers. In our research setting. these controls are absent. Not only are we in no position to intervene, but having to deal with large numbers of subjects prevents us from being able to specify such factors for individual subjects. As research techniques in the area of quality of care improve, it is likely that the full power of this method of analysis will be more subtly explored. Conclusion
This paper has attempted to clarify the variables involved and to discuss some of the difficulties in adapting the operant model to a group evaluation model. The difficulties arise out of two differences-the lack of control over independent variables and the nature of the questions asked. The former prevents us from determining, by actual experiment, the precise functional relationships between the independent and dependent variablecit necessitates the manipulation of variables by statistical procedures. However, even if we could control such variables, many of the problems discussed in this paper would still arise. because questions are asked about groups of subjects; about whole days in their lives. The implications of this have been listed-the need to develop general categories of client behaviour; the difficulty of specifying target behaviour, etc. These are problems which will need solutions if senior management personnel are to be provided with reliable and comparable feedback on the standards of care they are providing to large numbers of clients.
REFERENCES ALLEN K. E. and HARRIS F. R. (1966) Elimination of a child’s excessive scratching by training the mother in reinforcement procedures. Brllur. Res. Thrrapy 4, 79-84. BIJOU S. W. and BAER D. M. (1961 I Child DcwloprmwrI. A Syst~~natic and Empirical Theory. Appleton-CenturyCroft. New York. BIJOV S. W. and BAER D. M. (1966) Operant methods in child behaviour and development. In Opcrarlt BrhuI;iour: Areas of Resrarchand Applicutim (Ed. W. K. HONIG). Appleton. New York. BIJOL S. W.. PETERSON R. F. HARKIS F. R.. ALLEN K. E. and JOHNSTON M. S. (1969) Methodology for experimental studies of young children in natural settings. Psychol. Rec. 19. 177-210. BUEHLERR. E.. PATTERSONG. R. and FURNISS J. M. (1966) The reinforcement of behaviour in institutional settings. Behm.
Res. Therap>’ 4, 157- 167.
FOXX R. M. and AZRIN N. H. (1973) Toiler Truinrq rhr Retarded: A rapid programme /or day and night-tinw indrpwdozt roilrttlq. Research Press. Illinois. GEFLAND D. M.. GEFLAND S. and DOBSOX W. R. (1967) Unprogrammed reinforcement of patients’ behaviour m a mental hospital. Brkrr-. RCL T/wrrrp~ 5, 201-207. GOFFMAN E. (1961) A.s~/urm: .Es.suy.\ m the social siruariorl qf‘menral patiems and other inmotrs. Pelican. HERBERT E. W. and BAER D. M. (1972) Training parents as behaviour modifiers: self recording of contingent attention. J. appl. Bcimr. Amd. 5, I39- 149.
236
R. WHATMORE. L. DURWARD and A. KUSHLICK
KING R. D., RAYNES N. V. and TIZARD J. (1971) Patterns ofResidentialCare-Sociological studies irl institutions for handicapped children. Routledge & Kegan Paul. London. KUSHLICK A. (1972) Evaluating residential services for the mentally handicapped. In Approaches to Acrion: A symposium on servicesfor the mentally ill and handicapped. Oxford University Press. London. KUSHLICK A. and Cox G. R. (1973) Epidemiology of mental handicap. Deu. Med. Child Neural. 15, 748-759. LINDSLEY 0. R. (1964) Direct measurement and prosthesis of retarded behaviour. J. Educ. 147. 62-81. Moos R. (1972) Assessment of the psychosocial environments of community-oriented psychiatric treatment programmes. 1. ahnorm. Psychol. 79, 9-l 8. PETERSON R. F., COX M. A and BIJOU S. W. (1971) Training children to work productively in classroom groups. Except. Child. March 1971. SKINNER B. F. (1953) Science and Humm Eehaoiour. Macmillan. London. ULRICH R.. STACHNIK T. and MABRY J. (1966, 1970) Control of Human Behaniour. Vols. I and 11. Scott, Foresman. Illinois. WAHLER R. G. (1972) Some ecological problems in child behaviour modification. In Behnriour Modification: Issues and Extensions (Eds. S. W. BIJOU and E. RIBES-INESTA).Academic Press, New York.
Appendix
A. Classification
of ‘playing’
behaviour
for a completely
Disruptive
Inappropriate
Pushing large toy into child/adult, e.g. bike, truck.
Pushing large toy into furniture/door/wall, etc.
State of touching/not touching/not moving toys.
Throwing hard/heavy/ sharp toys, throwing them off table/shelf.
Throwing light/soft toys, throwing them off table/shelf, e.g. cloth doll (except balls, balloons).
Holding
Tearing/pulling to pieces hard/heavy/ sharp toys which as a result are dangerous to others and self. Tipping/pushing over large toy equipment, e.g. slide, bike, rocking horse.
Tearing/pulling pieces light/soft
Neutral
to toys.
Tearing/pulling to pieces hard/heavy/sharp toys which as a result are not dangerous to others and self. Hitting/kicking any toys (except balls, etc.). Sucking/ chewing any toys. Banging toys on each other or on any other object surface.
toy still.
Sitting on large toy. e.g. slide, bike. Letting toy fall. dropping toy (even large heavy toys). Putting toy down. Being given a toy which the child drops/lets fall immediately it is put into his hand, i.e. the child does not hold the toy even for a second or two. Pushing/knocking over small toys. Pushing toy away. Dropping toy off table/shelf. Making any movement with the toy which is repetitive and is not its intended use. Standing on large toy with or without help from person or object.
mobile client Appropriate Act of touching with hand/ picking up/playing with toys -a toy being an object with a generalised use-‘playing with’, e.g. banging drum, throwing ball, doing puzzle, going down slide, turning pages of book. painting on paper. climbing up frame, riding bike. Using the toy for its intended purpose even in a repetitive way. Teasing/catching/spinning any small toy even in a repetitive way. Putting toy in its proper place, e.g. book on shelf, pegs in box, doll in cupboard.
1975. Vol. 13. pp X7- 236. Pcrgamon Press Prmted m Great Brttam
MEASURING
THE QUALITY
OF RESIDENTIAL
CARE*
RON WHATMORE, LYN DURWARD and ALBERT KUSHLICK wessex Regional Health Authority, Romsey Road, Winchester, (Rrceiaed
Hampshire.
England
19 July 1974)
Summary-Administrative personnel require objective, reliable and valid information in order to monitor the effects of allocating resources to different components of the service that they are providing. The initial problem is to define and measure the dependent variable-the ‘quality of care’ -against which the effects of changing various independent variables can be evaluated. The paper describes an attempt to use the operant conditioning model as a basis for constructing such a measure of the quality ofcare provided in residential settings for severely mentally handicapped people. The rationale for adopting this model is described. against a background of other studies in which the same problem has been addressed. The hypothesis. the tasks generated, and the methodological problems encountered. are outlined and discussed. A subsequent paper will present the resuhs obtained when using the method to measure the quality of care, so defined, in two residential units for severely mentally handicapped children.
personnel of medical and social welfare organisations (e.g. hospital officers, local authority officers) are responsible for the allocation and distribution of vast financial, material and manpower resources to large populations ‘at risk’. Government policy statements require that they execute these functions in a manner which maintains or raises the ‘quality’ of each of the services provided. Thus, the administrator faces two immediate questions: (1) How do I ‘know’ whether the quality of care is high or low? (2) Of the many factors determining the quality of care which ones have the most significant effect?? These two general questions define the general research task. In this case, it is applied to residential settings for severely mentally handicapped people. Administrative
The concept of ‘quality of residential care’
The first task is to specify what the ultimate benefit to the client should be, given that the residential unit is providing ‘high’ quality care. We decided that the major goal should be that the clients make ‘progress’, i.e. (i) acquire new and/or maintain existing defined ‘appropriate’ behaviour (e.g. social skills like feeding/washing/dressing self). (ii) not lose such appropriate social skills. (iii) lose defined ‘inappropriate’ behaviour (e.g. anti-social skills like pulling hair, biting people, destroying furniture). Given these as the top priority goals, the second task is to focus on aspects of the daily living environment, in order to identify, and then quantify, those factors which appear to determine the probability that clients will make progress. Recent attempts to quantify the quality of residential care have avoided the use of such traditional criteria as cleanliness (absence of dirt or germs), tidiness (absence of physical disorder), and social order (e.g. absence, at any time, of noise), and have concentrated on factors in the social environment believed intuitively to be related to client welfare. Moos (1972) for example, has attempted to quantify the ‘environmental climate’ in residential and other settings. Starting from Goffman’s (1961) descriptions of ‘total institutions’, King, Raynes and Tizard (1971) have developed a scale which operationally defines and measures staff practices relating to the management of children in residential settings. This *This paper covers one section of an ongoing research project aimed at evaluating services delivered to severely mentally handicapped children and their families. For detailed accounts of other sections of the research, see Kushhck (1972) and Kushlick and Cox (1973). t The paper mainly covers the first of these questions. The second question, necessarily dependent upon an answer to the first. is the subject of subsequent research. 227
228
R. WHATMORE. L. DURWARD and A. KUSHLICIG
‘child management scale’ measures the extent to which residential units ‘depersonalise’ and ‘block treat’ the children, and to which they manifest ‘rigidity of routine’ and ‘social distance’. A unit in which these negative practices are avoided is judged to be providing ‘high quality care; ‘low’ quality care is provided in a unit in which these practices occur. In our early approacheswe followed King, Raynes and Tizard’s formulations. During our detailed observations of daily routines on living units (i.e. the sequences of staff and client activities throughout the waking day), however, it became clear that residential units in which these negative management practices are avoided do not necessarily provide conditions under which their clients make ‘progress’ or fail to ‘regress’. This arises from the types of management practices which are measured in the scale. Some practices appear to bear little relation to factors which are likely to influence progress. for example. whether ‘there are set times when visitors can come to the unit’. Other practices are more closely related, for example, ‘block treatment’. The relevant scale questions probe the extent to which the children ‘wait around’ before and after important social activities like mealtimes, bathtimes, etc; ‘social distance’ questions probe the extent to which staff are accessible to the children, e.g. at mealtimes, during TV times. etc. However, one cannot infer from lack of ‘waiting around’ and ‘presence of staff that (a) staff and children are interacting, or (b) that the effect of the interaction will be to increase the children’s social skills. Also. the scale questions do not probe long periods of the day when interactions between children and staff may or may not take place. A unit in which children are left alone acquiring a large range of ‘inappropriate’ behaviours (e.g. faeces smearing during recreation periods) could still obtain a high rating on the management scale. In brief, then, although the management practices scale measures important features of the residential environment (i.e. the extent to which a unit might appear ‘homely’ to an observer) it appears to lack sensitivity to staff practices directly affecting individual clients. The model We have therefore attempted to develop a more sensitive measure in which the components are directly related to client progress. We have found that the conceptual framework of operant conditioning (Skinner, 1953) has many advantages. The framework clarifies the dependent variable-frequencies of specific responses classes-as well as the independent variables (conditions) under which these occur; it allows the study of directly observable behaviour measured in frequencies (Bijou rt al.. 1969); the efficacy of procedures developed from the framework has been demonstrated experimentally in many practical settings (Ulrich, Stachnik and Mabry. 1966, 1970); and a sophisticated technology for collecting, analysing and interpreting data has been developed (Bijou and Baer, 1966). The hypothesis The central premise of operant conditioning theory is that much behaviour is operant behaviour. The characteristic of such behaviour is that it is controlled by stimulus consequences. The hypothesis is that most client behaviour exhibited in the residential setting is operant in nature, that is, its frequency of occurrence is determined by stimulus consequences. Furthermore, we hypothesise that of all such possible stimuli. the class most likely to determine the frequency of specific classes of client behaviour is staff attention. Thus. classes of client behaviour followed by staff attention are likely to increase or be maintained in frequency; classes not followed by such attention are likely to be extinguished. This further defines the research tasks. Firstly. classes of client behaviour must be classified into those which (we judge) clients should acquire, and those which they should not.* Secondly, staff attention must be clearly differentiated from its opposite-absence of staff * The researcher must make this judgement where the client cannot verbalise and where staff and parents or others have not done so. There are now many individual studies where clients and their relatives have chosen the behavioural target.
Measuring the quality of residential care
229
attention. These are discussed in the following sections, in the context of the methodological problems arising from the use of the operant model. The ~et~odo~ogicul problems There are important differences between the approach of the behaviour modification experimentalist investigating changes in behaviour and other approaches to the evaluation of services (e.g. that of King, Raynes and Tizard). Since we are attempting a sort of ‘marriage’ of these two models, specification of the differences between them will help to ciarify the problems that arise from this union. They can be summarised as follows: Table I. Differences between the operant conditioning approach and that used in this research Operant approach Aims To effect a change in the frequency of specific classes of behaviour exhibited by one, or a small number of, individuals, and to demonstrate a functional relationship between specific dependent variables (classes of behaviour) and specific independent variables (stimulus consequences). Design (a) Individual or small homogenous groups of subjects. (b) Relatively short observation periods. (c) Limited number of ‘target’ classes of behaviour specified. (d) Use of subject as his own control (reversal or multiple baseline design) in most studies. (e) A priori functional relationship demonstrated.
Our approach
To derive a score, or series of scores which are inferred as representing the abstract concept ‘quality of care’ provided to a large number of individuals in residential care, and to investigate the relationship between this variable and general environmental variables (e.g. staffing ratio, type of training of caring staff, etc.) (a) Large heterogeneous groups of subjects. (b) Observations undertaken over the whole of the waking day. (c) All behaviours of all subjects must be classified. (d) Use of matched group of subjects (control group design) (e) A posteriori relationship suggested by means of statistical manipulations.
Assumptions No assumptions made about what is ‘reinforcing’ or ‘punishing’ to the subject.
Assumptions made about the reinforcing nature of different types of staff attention.
Degree of control High degree of control over independent variables.
No control over inde~ndent variables.
Research aim and questions Senior administrators (i.e. those having little day-to-day contact with the residential unit) require an answer to the following question ‘How can we manipulate resource and other variables in order to create settings for both staff and residents such that optimal progress of clients will take place? An answer in the form that manipulating variable X (e.g. staff attention) produces a change in the dependent variable Y (decrease in hitting behaviour) for child F with teacher A in situation B can be of limited relevance in occasioning their responses. The information, i.e. the measures that they require, must relate to the behaviour of groups of clients, of different levels of ability, and to the distribution of staff attention between different classes of client behaviour, throughout the whole of a waking day. Classijcation of client behauiour I. The D.Z.N.A. categories. Initially a two-category classification was devised-inappropriate client behaviour (e.g. hitting other children, destroying furniture, etc.) and appropriate client behaviour (all other behaviour), This is a very similar classification to those
230
R. WHATMORE.L. DURWARD and A. KUSHLICE
used in many behaviour modification studies (e.g. Peterson. Cox and Bijou, 1971; Herbert and Baer, 1972). In these two studies, the authors were able to pre-define the specific target behaviour of the individual subjects. In both studies, a task. defined in behavioural terms, was specified. By definition. their subjects would always behave either appropriately or inappropriately (‘on task’ or ‘off task’). Since the observation periods were short (4 hr or 1 hr), it was reasonably easy to set up a task-orientated setting with the undivided attention of one adult. The situation in a residential setting is very different. There are few defined tasks. particularly during certain activity-periods like recreation. Even at mealtimes. the clients may spend 15-20 min ‘waiting’ while staff prepare and distribute the food. During such no task periods it is inaccurate to describe the behaviour of a client as appropriate (‘on task’) or inappropriate (‘off task’) if he is, for example, simply sitting in a chair. This system, therefore, did not adequately describe all client behaviour in such a setting. In addition, the scores obtained did not differentiate between residential units in which, on the one hand. a client undertook or was cued to undertake a task by himself (e.g. using a spoon to feed himself) and, on the other. one in which the same client was passively ‘processed’ (e.g. fed by a member of stall). Hence. the former setting in which the client was beginning to learn new skills, would obtain the same score on the measure of ‘quality of residential care’ as the latter setting, in which a comparable client was denied the opportunity to learn such skills. Furthermore, a setting in which clients had more independent-behaviour could score lower (poorer quality) on the measure than a setting in which clients emitted fewer such skills, e.g. a client who fed himself might receive little staff attention during a mealtime, yet a client who had to be fed received some attention (by definition) whilst being fed. This arose because the category of appropriate behaviour included passive behaviour as well as that requiring active participation on the client’s part. It was thus unlikely to provide a valid index of ‘progress’. Another category was therefore introduced into the classification-‘neutral’ behaviour. This category includes passive behaviour (sitting on a chair, being fed by a member of staff), well-established skills (e.g. walking for a completely mobile person) and active behaviour having very little impact on the physical environment (e.g. rocking, bizarre hand movements). The category of appropriate behaviour is reserved for ‘adaptive’ behaviour in which it is likely that new skills will be acquired and/or maintained. This ensures that a setting in which a client is encouraged to undertake an activity by himself and is attended to during this time, will always obtain a higher score on the measure than a setting in which he has things done for him by a member of staff. A further problem was encountered because we had included a variety of behaviour in the ‘inappropriate’ category. For example, it included behaviour of relatively small impact on the environment such as ‘throwing a cloth doll’ through to more disruptive behaviour like ‘biting other children’. The same score on the measure might therefore be obtained from staff responses to very different types of ‘inappropriate’ child behaviour. To differentiate between these scores, we divided the inappropriate category into two separate categories-disruptive, which included behaviour that had a major disruptive impact on the social and physical environment, e.g. hitting other persons, playing with faeces, breaking glass; and inappropriate, which included behaviour that had a minor impact on the social and physical environment, for example, sucking or chewing toys. There are therefore four classes in the final system: ‘Disruptive’ behauiour is any behaviour
which is likely to, or does, cause physical damage, to the child or other persons (biting, hitting self or others), or which causes physical damage to furniture and fittings (pulling down curtains, pictures, etc.). ‘Inappropriate behaviour is any behaviour which is socially disruptive (screaming) or which is less physically damaging to the physical environment (sucking, chewing toys, destroying dolls, etc.). ‘Neutral’ behauiour is any behaviour which is totally passive (sitting on chair, etc.); which has very little impact on physical environment (bizarre hand movements, rock-
Measuring
the quality
of residential
care
231
ing); which is a well-established skill (e.g. walking for a completely mobile child). ‘Appropriate’ bekaviour is any active ‘adaptive’ behaviour in which it is likely that new skills will be acquired and existing skills maintained. These are abbreviated as D, I, N and A respectively. An example of the coding system is given in Appendix A. 2. Mobility categories. It has also been necessary to categorise degrees of mobility of the client. The degree of independent mobility of a client sets limits on the extent to which the client can contact staff members. There are three categories: (a) Completely mobile (CM) persons-those who walk unaided at least 10 yards on a level surface. (b) Partly mobile (PM) persons-those who walk unaided for less than 10 yards on a level surface: those who propel themselves along in some way on a level surface, e.g. walk with help, crawl, pull self along in wheelchair; those who sit up unaided for at least 10 minutes. (c) Non-mobile (NM) persons-those who sit up unaided for less than 1Omin; those who do not sit up unaided; those who do not propel themselves along a level surface at all. The categories for ‘completely mobile’ and ‘partly mobile’ are very similar. The only difference is that classes of behaviour which are prerequisite to walking (e.g. standing, sitting up) plus walking itself. are defined as appropriate for the ‘partly mobile’ client, and neutral for the ‘completely mobile’ client. For the ‘partly mobile’ client, mobility skills constitute an essential basis for the development of other skills and hence should be reinforced. For the ‘completely mobile’ client, however, such skills have long been well developed and are unlikely to be extinguished if not reinforced, as long as the opportunity for emitting them is not restricted (i.e. residents are not restrained in chairs or confined to bed). Non-mobile clients present more difficult problems for behaviour classification. By definition, they emit very little behaviour. Hence, for these clients, small movements (for example, of hands, arms. legs, etc.) are defined as appropriate. These would be the classes of behaviour which a physiotherapist. for example, would identify and reinforce in such a client. Clearly, one classification cannot cover the whole range of behaviour exhibited by the clients. If walking was defined as appropriate for the completely mobile client many would be behaving appropriately 90-100 per cent of the time. The same effect would arise if any minor movement was defined as appropriate for the ‘partly’ and ‘completely mobile’ client. The resulting quality of care scores would be most unlikely to predict progress sensitively. The classzjicatiorz of stafShehariou~
As described earlier. staff behaviour is classified into two categories-attending to or not attending to the client. Staff attending behaviour is further defined as any verbal and/ or physical contact with the client. The rules for specifying verbal and/or physical contact under several sets of conditions are defined. The codes obtained by using the method have proved highly reliable. The general hypothesis (p. 228) can now be stated in more precise terms: that the quality of residential care is ‘high’ if. among a group of clients, a high proportion of their appropriate, and a low proportion of their neutral, inappropriate and disruptive behaviour is followed by staff contact. The converse follows in units providing ‘low’ quality of care. ClassifJGng spec$c
clierlt hehaciour us D.I.N.A.
Difficulties have arisen in coding a given behaviour in the same class (D, I, N or A) on each occasion that it occurs. An important requirement of the codes is that they should be relatively easy to use, that is, it should be possible for a relatively naive observer, after a short period of training, to code client behaviour in the natural setting with a high degree of reliability. The reliability is likely to be higher the simpler the discriminations required of the coders. Thus, it would be an advantage if a specific topography of a behaviour was always
232
R. WHATMORE,L. DURWARD and A. KUSHLICK
coded in the same category. e.g. if ‘screaming’ were always an inappropriate behaviour, irrespective of the situation in which the client screams. If this is not the case, and ‘screaming’ were classified as, say, neutral during recreation times, but inappropriate during mealtimes, the observer would have to code that behaviour differently in the different activity periods. Failure to code differentially according to the activity period might be detected in reliability checks. If not, this will lead to erroneous conclusions from the analysed data. For most behaviour. this presents no problem. However. there are important exceptions, e.g. sitting. For all clients, ‘sitting’ is a neutral behaviour-it is neither active, nor does it constitute acquiring a new skill. However. during a mealtime ‘sitting’ is clearly one precurrent behaviour necessary to the task in hand--eating at the table. It could therefore justifiably be classified as appropriate during this activity-period; and ‘being out of place’ classified as inappropriate. At this point, we were faced with two alternatives. Either to code ‘sitting’ during a mealtime differently to ‘sitting’ during other activity-periods, and risk the possibility of coding errors, or to code ‘sitting’ the same irrespective of the activity-period, and risk the possibility of lowering the sensitivity of our measure. The latter possibility arises because if ‘sitting’ is classified as neutral behaviour during mealtime. the measure would not differentiate between a unit in which all the clients sat at the table throughout the meal, and one in which all the clients got up from the table and wandered around the dining room. We decided to take the latter alternative, for two reasons. Firstly, there are many appropriate types of behaviour at the mealtable other than ‘sitting’, e.g. feeding self, picking up salt/pepper, chatting, etc. If units differ on the extent to which these types of behaviour are exhibited and responded to by members of staff, the measure will be sensitive to these differences. Secondly, from unsystematic observation in residential settings, it appears that most clients readily sit down and remain seated for the whole of the mealtime. Therefore, this may not be an area in which the measure is required to be sensitive. (The validity of this impression will be tested by examination of the verbal description of client behaviour during mealtimes. The description is written down by observers at the same time as they code client behaviour and staff-client contact). Having stated that ‘sitting’ behaviour is classified as neutral at all times, there is one type of ‘sitting’ which creates another, quite separate, coding problem. This is ‘sitting’ on the toilet/pot. The problem arises because the two, central, appropriate aspects of behaviour in this task-urinating and defaecating-are unobservable, without the aid of special, technical apparatus which is impractical to use in our research setting. The closest observable approximation to these two classes of behaviour is the ‘sitting on the toilet’. Therefore, unlike other types of ‘sitting’. this is classified as appropriate behaviour. Similar difficulties have arisen over the classification of behaviour which is, in itself, appropriate, but is emitted at inappropriate times, e.g. getting into a bed or bath in the middle of the morning without staff instruction or consent. Again, the use of ‘time of day’ or ‘instruction’ as a stimulus event (i.e. discriminative stimulus) to occasion client responses results in the same behaviour being classified as appropriate during some activityperiods, and neutral or inappropriate during others-with the same implications for the reliability of the codes and possibly erroneous interpretation of data. We decided that stimulus events associated with ‘place’ (e.g. the room or area in the building) are as, or more, likely to control client response than those associated with ‘time of day’. Furthermore, it may also be more important for the client to learn to get into a bath in the bathroom, to get into a bed in the bedroom, rather than to learn to emit these behaviours only at particular times of the day. As can be seen from the two examples given, this has the additional advantage of ensuring that such behaviour is always classified the same, since the material equipment necessary for the emission of the behaviour is always in the same place. One can only get into a bath in a bathroom, and this behaviour in this room is, by definition, appropriate, no matter what the time of day. There is only one class of behaviour in which this is not the case-‘undressing’. Since a client can undress in any room, this behaviour is appropriate only in the bedroom/bath-
Measuring
the quality
of residential
room/toilet area. In any other room, it is (in most circumstances) disruptive to be partly or completely undressed. Dejning
233
care
inappropriate
or even
target hehavioul
The behaviour modifier is in a position to define the target behaviour in consultation with the staff/teacher, etc. Thus, the experimentalist and the practitioner agree on what behaviour is ‘appropriate’ and ‘inappropriate’. To obtain such agreement in our situation would require discussion with hundreds of members of staff, and it is doubtful if such agreement could be obtained. There are two alternatives. Either to construct a different set of codes for each residential unit (assuming a consensus could be arrived at on each unit), or to decide on our own codes after consulting certain members of staff who care for the clients on the living units. We have taken the latter alternative since without using the same codes between units, it becomes impossible to make inter-unit comparisons. However, the latter course of action gives rise to some peculiarities. For example, if a client stands on a table, this is classifiedas inappropriate. Yet, in order to dry a client with a towel, a member of staff may stand him on a table, which is clearly, in the staffs classification, not an inappropriate client behaviour. Exceptions could be made where it is clear that staff have instructed a client to behave in a specific way, but this would increase the complexity of an already complex set of codes. It would also raise further problems in interpreting the quality of care score between residential units. Activity periods
Within any one living unit, the quality of care may well vary at different times of the day. This is especially likely to occur where living unit staff concentrate on particular classes of behaviour, like a comprehensive mealtime or toiletting programme (Foxx and Azrin, 1973) and ignore other behaviour. It is therefore important before deciding to concentrate observations on a short period of the day to establish whether and to what extent variation in staff practices occurs. Since practices must vary at certain key periods of the day, a method is required for dividing the day into such meaningful periods. ‘Natural’ divisions occur on the basis of the common activity in which most clients or sub-groups of clients engage at the same time. These we have called ‘activity-periods’. There are three such activity-periods (each occurring more than once throughout the day) -mealtimes, recreation, and toiletting/washing/dressing. The identification of the period with respect to an individual being observed is sometimes difficult within any one group of clients. They may not all be engaged in the same activity at any given time (e.g. a resident may be waiting to be fed); they may not all be in the same room (residents who have been fed may be taken through to the playroom whilst the remainder of the group are fed); some may be engaged in a different individual activity (e.g. being given physiotherapy whilst the remainder of the group are being washed). Assumptions
The behaviour modifier makes no assumptions about the reinforcing or punishing effects of consequences to the clients’ behaviour. These are carefully tested by quantifying the change in frequencies of the specific consequated behaviour. Furthermore, by simultaneously recording the frequency of two classes of behaviour-one of which is consequated and the other ignored-the specificity of a reinforcer can be checked. In our research setting, it is not possible to test which particular stimuli are reinforcing and which are punishing to each of the 250 clients-indeed, these may well change over a period of time for any given individual. It is therefore necessary to make certain assumptions about which stimuli are likely to reinforce and which to punish the behaviour of clients in general. Our hypothesis focuses on contact with a staff member as the stimulus consequence most likely to have a major effect in determining the frequency of the clients’ responses.
234
R. WHATMORE. L. DURWARD and A. KUSHLIC~
There are many ways of classifying such contact. For example, we could differentiate between topographies of contact, and predict that some (e.g. shouting, smacking) will be punishing, and that others (e.g. praising. cuddling) will be reinforcing (for examples of this approach see the work of Buehler. Patterson and Furniss (1966). and Gefland. Gefland and Dobson (1967)); or we could assume that all contact is equally reinforcing (or punishing). From observations of staff-client contact in the residential settings. it appears that the frequency of such contact is very low for the majority of the clients. The exceptions are clients who exhibit very severely disruptive behaviours (pulling hair. throwing chairs. spitting. etc.). Their contact frequency is high because staff must intervene to prevent damage to persons and property. Furthermore. a/?~ staff contact--even severe verbal and/or physical reprimands which are normally’ assumed to have punishing effects-appears to maintain the behaviour of these clients. (There is also some research evidence for this. e.g. Allen and Harris, 1966.) Given this type of reinforcement history-either very little staff contact. or contact contingent upon inappropriate and/or disruptive behaviour-it seems reasonable to assume that one topography of contact is as likely to be reinforcing to clients, in general. as any other. Time sampling method
We have to observe many clients (250 approximately in 10 different living-units) in order to obtain a score of quality of care that living-units provide for those clients. This involves sampling clients’ behaviour. and the time sample of any one individual is necessarily small because of the large numbers involved. We have therefore to determine which particular methods of time sampling will give a representative picture of a group of individual clients. At present a maximum of four clients are observed by one observer. Client A is observed for 15 set, with a further 15 set used for writing down rough descriptions of the classes. then client B, and so on. Analysis has shown that, provided that the proportion of staffclient contacts and of one class of client behaviour (D.I.N.A.) is not extremely low (not less than 5 per cent) and provided that the observation period is reasonably long (approximately 4 hr or more), this method yields a representative sample. However. there are practical problems with this method. In switching observation from one client to another (even if the clients are in the same mobility category) every 30 set, the observer may have to start recording a different class of client behaviour and/or of staff-client contact. This rapid changing can be a source of unreliability. More important, in order to obtain the same number of observations on each client, those four clients would have to remain together in the same vicinity. This rarely happens; often one or two leave the vicinity, in which case, observations on them will be lost for as long as they are away. This raises problems in the analysis, since. without considerable re-arrangement of the data, the scores will be weighted towards those of clients observed more frequently. One way of overcoming these difficulties would be to use a 5 min-in-20 min time sample method. Thus, an observer would only have to switch observations every 5 min, rather than every 30 set, and she could also follow an individual client who moved to another part of the building, returning to the group for the 5-min observation on the next client. The one limitation may be that such a method, which samples larger ‘blocks’ of observations on each client may not be representative. A pilot study is being designed to test this. Co~ltrol over independent
variuhles
In a paper on the methodology of behaviour modification studies, Bijou et al. (1969) state that in a nursery school setting, “the absence of the head teacher. or the presence of the child’s mother would probably vary the situation enough to preclude data collection. Marked reduction in class attendance due to illness. inclement weather, or holiday preparations at home would also be considered a cardinal alteration in the field situation”. In preliminary field studies that we have undertaken, it seemed impossible to define a standard or normal situation, as described by Bi_jouet al. This is obviously partly because our observations extend over the whole of the waking day. At different times and on differ-
Measuring
the quality
of residential
care
335
ent days, different staff would be on duty: staff members would be absent; volunteers, relatives, the physiotherapist, senior nursing personnel, would visit. etc. However, even if some days happen to be more ‘standard’ than others. it would be of little help because the basic question that we are attempting to answer is different to that of Bijou et al. We want to know what learning opportunities are available to a client throughout his waking day on a particular living-unit. ‘Non-standard’ days (depending on their frequency) may therefore be just as important as ‘standard’ days. For an NM client. what happens on a day when the physiotherapist is absent is as important as what happens when she is present. Quality of care is an amalgam of what happens throughout a series of days during which the independent variables may change considerably. In this sense. there are no ‘standard’ or ‘non-standard’ days. Our measurement of ‘quality of care’ must yet ignore other sophistications and subtleties which are part of the operant model (e.g. Lindsley. 1964). Skinner (1953) emphasises the complex differential effects of various reinforcement schedules (continuous or intermittent). Wahler (1972) describes the influence of ‘setting events’, and Bijou and Baer (1961) discuss the function of ‘discriminative stimuli’. In the classical intervention design. the behaviour modifier can specify beforehand what type of reinforcement schedule and what type of discrimination (if any) are to be used. He can also control such setting events as the degree of deprivation and satiation of reinforcers. In our research setting. these controls are absent. Not only are we in no position to intervene, but having to deal with large numbers of subjects prevents us from being able to specify such factors for individual subjects. As research techniques in the area of quality of care improve, it is likely that the full power of this method of analysis will be more subtly explored. Conclusion
This paper has attempted to clarify the variables involved and to discuss some of the difficulties in adapting the operant model to a group evaluation model. The difficulties arise out of two differences-the lack of control over independent variables and the nature of the questions asked. The former prevents us from determining, by actual experiment, the precise functional relationships between the independent and dependent variablecit necessitates the manipulation of variables by statistical procedures. However, even if we could control such variables, many of the problems discussed in this paper would still arise. because questions are asked about groups of subjects; about whole days in their lives. The implications of this have been listed-the need to develop general categories of client behaviour; the difficulty of specifying target behaviour, etc. These are problems which will need solutions if senior management personnel are to be provided with reliable and comparable feedback on the standards of care they are providing to large numbers of clients.
REFERENCES ALLEN K. E. and HARRIS F. R. (1966) Elimination of a child’s excessive scratching by training the mother in reinforcement procedures. Brllur. Res. Thrrapy 4, 79-84. BIJOU S. W. and BAER D. M. (1961 I Child DcwloprmwrI. A Syst~~natic and Empirical Theory. Appleton-CenturyCroft. New York. BIJOV S. W. and BAER D. M. (1966) Operant methods in child behaviour and development. In Opcrarlt BrhuI;iour: Areas of Resrarchand Applicutim (Ed. W. K. HONIG). Appleton. New York. BIJOL S. W.. PETERSON R. F. HARKIS F. R.. ALLEN K. E. and JOHNSTON M. S. (1969) Methodology for experimental studies of young children in natural settings. Psychol. Rec. 19. 177-210. BUEHLERR. E.. PATTERSONG. R. and FURNISS J. M. (1966) The reinforcement of behaviour in institutional settings. Behm.
Res. Therap>’ 4, 157- 167.
FOXX R. M. and AZRIN N. H. (1973) Toiler Truinrq rhr Retarded: A rapid programme /or day and night-tinw indrpwdozt roilrttlq. Research Press. Illinois. GEFLAND D. M.. GEFLAND S. and DOBSOX W. R. (1967) Unprogrammed reinforcement of patients’ behaviour m a mental hospital. Brkrr-. RCL T/wrrrp~ 5, 201-207. GOFFMAN E. (1961) A.s~/urm: .Es.suy.\ m the social siruariorl qf‘menral patiems and other inmotrs. Pelican. HERBERT E. W. and BAER D. M. (1972) Training parents as behaviour modifiers: self recording of contingent attention. J. appl. Bcimr. Amd. 5, I39- 149.
236
R. WHATMORE. L. DURWARD and A. KUSHLICK
KING R. D., RAYNES N. V. and TIZARD J. (1971) Patterns ofResidentialCare-Sociological studies irl institutions for handicapped children. Routledge & Kegan Paul. London. KUSHLICK A. (1972) Evaluating residential services for the mentally handicapped. In Approaches to Acrion: A symposium on servicesfor the mentally ill and handicapped. Oxford University Press. London. KUSHLICK A. and Cox G. R. (1973) Epidemiology of mental handicap. Deu. Med. Child Neural. 15, 748-759. LINDSLEY 0. R. (1964) Direct measurement and prosthesis of retarded behaviour. J. Educ. 147. 62-81. Moos R. (1972) Assessment of the psychosocial environments of community-oriented psychiatric treatment programmes. 1. ahnorm. Psychol. 79, 9-l 8. PETERSON R. F., COX M. A and BIJOU S. W. (1971) Training children to work productively in classroom groups. Except. Child. March 1971. SKINNER B. F. (1953) Science and Humm Eehaoiour. Macmillan. London. ULRICH R.. STACHNIK T. and MABRY J. (1966, 1970) Control of Human Behaniour. Vols. I and 11. Scott, Foresman. Illinois. WAHLER R. G. (1972) Some ecological problems in child behaviour modification. In Behnriour Modification: Issues and Extensions (Eds. S. W. BIJOU and E. RIBES-INESTA).Academic Press, New York.
Appendix
A. Classification
of ‘playing’
behaviour
for a completely
Disruptive
Inappropriate
Pushing large toy into child/adult, e.g. bike, truck.
Pushing large toy into furniture/door/wall, etc.
State of touching/not touching/not moving toys.
Throwing hard/heavy/ sharp toys, throwing them off table/shelf.
Throwing light/soft toys, throwing them off table/shelf, e.g. cloth doll (except balls, balloons).
Holding
Tearing/pulling to pieces hard/heavy/ sharp toys which as a result are dangerous to others and self. Tipping/pushing over large toy equipment, e.g. slide, bike, rocking horse.
Tearing/pulling pieces light/soft
Neutral
to toys.
Tearing/pulling to pieces hard/heavy/sharp toys which as a result are not dangerous to others and self. Hitting/kicking any toys (except balls, etc.). Sucking/ chewing any toys. Banging toys on each other or on any other object surface.
toy still.
Sitting on large toy. e.g. slide, bike. Letting toy fall. dropping toy (even large heavy toys). Putting toy down. Being given a toy which the child drops/lets fall immediately it is put into his hand, i.e. the child does not hold the toy even for a second or two. Pushing/knocking over small toys. Pushing toy away. Dropping toy off table/shelf. Making any movement with the toy which is repetitive and is not its intended use. Standing on large toy with or without help from person or object.
mobile client Appropriate Act of touching with hand/ picking up/playing with toys -a toy being an object with a generalised use-‘playing with’, e.g. banging drum, throwing ball, doing puzzle, going down slide, turning pages of book. painting on paper. climbing up frame, riding bike. Using the toy for its intended purpose even in a repetitive way. Teasing/catching/spinning any small toy even in a repetitive way. Putting toy in its proper place, e.g. book on shelf, pegs in box, doll in cupboard.
