This article was downloaded by: [University of Tennessee, Knoxville] On: 30 December 2014, At: 07:57 Publisher: Routledge Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

Social Work in Health Care Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/wshc20

SEVEN STEPS TO AUDIT a

Barbara Berkman DSW & Helen Rehr DSW

b

a

Adjunct Assistant Professor of Community Medicine (Social Work), Mount Sinai School of Medicine of the City University of New York, New York, NY 10029 b

Professor of Community Medicine (Social Work), Mount Sinai School of Medicine of the City University of New York, New York, NY 10029 Published online: 26 Oct 2008.

To cite this article: Barbara Berkman DSW & Helen Rehr DSW (1977) SEVEN STEPS TO AUDIT, Social Work in Health Care, 2:3, 295-303, DOI: 10.1300/J010v02n03_05 To link to this article: http://dx.doi.org/10.1300/J010v02n03_05

PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content. This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://www.tandfonline.com/page/termsand-conditions

SEVEN STEPS TO AUDIT

Downloaded by [University of Tennessee, Knoxville] at 07:57 30 December 2014

Barbara Berkman, DSW Helen Rehr, DSW

ABSTRACT. One hospital$ approach to social service audit is discussed through the delineation o f seuen steps. Close collaboration between the researcher and staff is necessary for the implementation o f any audit system. The problems in conceptualization of the variables for study, sample selection, sample attrition, data collection, and analyses o f findings are treated. The demands for accountability require the systematic examination o f practice in order to answer the questions of relevance, efficiency, and effectiuenessposed by consumers, other health professionals, and reimbursing agencies.

The paucity of social work research in hospital settings prior to the 1970s was often attributed t o lack of funds t o support it. Currently, the public demand for hospital accountability includes hospital social work programs, with an expectation of implementation of professional standards review systems. Serviceevaluation issues are raised by government mandate, as established in the PSRO legislation, by program directors who must account for the value of their services, by funding agencies, professional groups, clientele, a more sophisticated general public, and our social work profession. Systematic research studies, by devising methods t o clarify concepts of performance and by delineating the role of the social worker, can be one means of audit. Research, in this role, can be useful in meeting accountability demands of public and private funding agencies, hospital administrators, and consumers. Traditionally, health-related work has been housed in institutions where "financial" accountability has been a primary concern. Statistics have more often been gathered by hospital administrators for cost analyses than t o assess and evaluate performance. In addition t o "financial" accountability, "performance" accountability is now raised as an issue.' Is social work reaching the needs of the people it serves by the treatment provided? Are the services provided comprehensive enough? In the past ten years, the Social Service Department of the Mount ~ t r ~ e r k m aisn Adjunct Assistant Professor o f Community Medicine (Social Work), and Dr. Rehr is Professor o f Community Medicine (Social Work), Mount Sinai School o f Medicine o f the City University o f New York, Fifth Avenue a t 100th Street, New York, NY 10029. 295 Social Work in Health Care, Vol. 2(3),Spring 1977

Downloaded by [University of Tennessee, Knoxville] at 07:57 30 December 2014

SOCIAL WORK IN HEALTH CARE

Sinai Hospital in New York City has completed numerous studies, similar in approach and purpose, but varied in terms of the division of social work involved and the populations studied. Many research questions were pursued. Based on these experiences in intradepartmental research, an approach t o conducting necessary audit studies has been developed. It is being used effectively with minimal cost expenditure, frequently without benefit of outside funding. This paper discusses the steps involved in implementing intradepartmental audit studies to document the social worker's contribution t o the management of the social component in illness. The approach involves: (1)the study of what psychosocial needs or problems social workers say they deal with (their objectives) on a case-bycase basis; (2) the social workers' judgment of "outcome" for each need they identify; and (3) the study of clients' perceptions of needs and the role of social workers in meeting those needs. The specific delineation of clients' social needs and outcome of intervention as directly related t o these needs are essential components of this audit system. In this approach, consumer evaluation is assumed to be one of the primary sources of validation of the social worker's role with clients, and, therefore, some indication of client perception and satisfaction should be secured. Such a classification makes review and assessment of goals and outcomes feasible. The process of conducting an audit based on this approach is described in seven steps.

STEP 1 Determining the Audit Questions The first step in the design of an audit study is t o determine the question or questions t o be answered. Some common inter-related issues which can be studied t o meet accountability demands are: (1)who was served? (for example, in sociodemographic terms); (2) how does the client get into your system of care? (who referred him?) (3) what does the referrer request? (4) what services were given? over what period of time? (5) what was the outcome of the intervention as perceived by the social worker? (6) what are the client's social needs as perceived by the referrer and social worker? (7) was the patient and/or family satisfied? (8) did the recipient and referrer understand the benefit from the service? In addition, more complex questions can be asked, such as: do social workers' perceptions of outcome vary according t o the types of social needs involved? Do social workers and clients agree on outcome? Do social needs and outcome vary according t o whether the patient is an inpatient or an outpatient?

Barbara Berkman and Helen Rehr

Are there certain normative ranges of intervention patterns related to particular problems? In designing inexpensive audits, an important principle is not t o expect to answk all possible questions in one study but rather to focus on answering a small segment. Then, in the next study, add pieces onto the original. Audits should be designed to build onto each other.

Downloaded by [University of Tennessee, Knoxville] at 07:57 30 December 2014

STEP 2 Administrative lssues2 The first administrative issue is the need for close collaboration between the persons conducting the audit and departmental social work staff. If the study is to be viable, the department's development of the audit design must be done with the cooperation of staff members. While the persons heading the research should have a plan (design) of how to do it, and a good grasp of the variables for study, the staff must be engaged in the research process to assure continued cooperation in implementation and in the collection of data. Another administrative issue concerns clarification of any ethical problems related to research involving human subjects. This is of particular concern if consumer follow-up is planned. The hospital's research administration committee (or whatever the responsible administrative body is called) should provide any existing institutional guidelines relevant to patient interviewing. Many hospitals have a standard form which patients must sign before they can be involved in research. The integration of data collection into treatment effort may be valid and. effective so long as the ethic of responsibility to client need is exercised. In the Adolescent Unwed Mother Clinic and in the Pregnancy Interruption Clinic at Mount Sinai, studies have been done over the last few years using a research questionnaire which the social workers follow in the first and last interviews with their clients. Aside from that, the information collected was data that the social worker normally obtained during the course of her interviews with patients and their families. Such studies must guarantee the interviewee confidentiality and anonymity with respect to data collection and reporting. The third administrative issue involves avoiding overuse of subjects for research purposes. This hazard is probably greater in psychiatric settings, where there are many different professions doing similar research and asking for the same types of data, than in the medical setting where social service might be the only department collecting psychosocial data.

SOCIAL WORK IN HEALTH CARE

Downloaded by [University of Tennessee, Knoxville] at 07:57 30 December 2014

STEP 3

Determining the Variables One of the major tasks in implementing an audit is to specify the concepts which are involved, and to identify and take into account independent and dependent variables. Independent variables are those which are thought to be independent of social service intervention and which do not vary over the time of the study. Examples of independent variables are the sex of the patient, the patient's age at the time of the study, and marital status. There are a number of such client social indicators which can be considered independent variables. The dependent variables, on the other hand, are those concepts, such as psychosocial needs, which may vary according to different conditions. For example, do the elderly (independent variable) have different psychosocial needs (dependent variable) than a younger age population? Psychosocial needs are conceived as variable related to the age of the patient. There can be complex independent variables. For example, does an elderly black man living alone (complex independent variable) have different psychosocial needs than an elderly white man living alone? The independent variables are usually not that difficult to understand or t o incorporate within the study design. Measuring the dependent variables, those concepts which you wish to study, is the more difficult measurement problem. For example, without an acceptable, valid classification system of "psychosocial needs," the dependent variable, it is not possible to collect the necessary data to answer audit questions such as: what are the psychosocial needs of patients referred to social service? or, what are the psychosocial needs as perceived by social workers? or, what are the needs as perceived by clients? Thus, a major factor in the type of audit study suggested is a classification system which clearly delineates the psychosocial needs social workers deal with. Much work has been done in this area and the use of available valid classification systems is not only acceptable but probably the most advantageous approach to take. For example, a social-need classification system, first published in 1972,3 has been replicated and validated for different patient groups, and, in fact, was recently piloted in 35 hospital social work departments across the state of Massachusetts with 237 social work cases.4 This system has been proven applicable to most adult medical and surgical social service cases in the general acutecare hospital, and is currently being revised for pediatric and psychiatric services. This, of course, does not mean that an existing classification system cannot be modified to fit a particular social work clientele more perfectly if indicated.

Barbara Berkman and Helen Rehr

Downloaded by [University of Tennessee, Knoxville] at 07:57 30 December 2014

STEP 4 Sample Selection A major research issue, after determining the question for study and the operationalizing of the variables, involves the definition of a "case." What should be considered a case? Is a situation involving one interview, or two interviews or more a case? What is a social work "consultation"? Often, these concepts are not well defined, with the result that, in a set of cases selected for study, some have involved client interviews and some are without interviews. Who is the primary client of social work intervention-the patient, or is it the patient and family? This question is particularly relevant for an audit which involves follow-up interviews, and the definition must be determined prior t o data collection. It would not be valid t o do a follow-up interview with a patient who had seen the social worker only once, if it had been the patient's spouse who had been the primary focus of intervention. Closely related here is the problem of deciding whether or not t o conduct follow-up interviews with primary client relatives of deceased patients. A great deal of resistance may be encountered from social workers in doing follow-up interviewing of any clients, but particularly of grieving relatives. If the social worker believes a primary client should not be interviewed, the researcher gives the practitioner opportunity t o explain why the client should not be contacted but determines in advance what types of reasons will be acceptable for omitting someone from follow-up and how such omission, if allowed, will be handled within the study. Another sample-selection problem involved in interviewing clients is simply getting accurate names and addresses which are necessary t o find them. Hospital and social work records are frequently incomplete, and records can be lost or outdated with respect t o addresses,and telephone numbers. Thus, if retrospective data (closed case records) are to be used, there is usually a real problem in finding current addresses or telephone numbers needed for a follow-up study. A prospective study alleviates this problem, although not being able t o locate some clients who move away or die after hospitalization must always be anticipated.

STEP 5 Accounting for Sample Attrition Taking these steps in stride, let us assume a sample has been selected and a prospective audit of one hundred social service

Downloaded by [University of Tennessee, Knoxville] at 07:57 30 December 2014

SOCIAL WORK IN HEALTH CARE

clients is t o be conducted, gathering data from social workers through precoded questionnaires, t o be followed by telephone interviews on a random sample of these clients. There is always a percentage of clients who cannot be reached, who have either died, have moved away, or just won't answer your questions. This problem in research is called "sample attrition." It must be remembered, however, that on clients that you can't reach, there are always some data available and these data can be analyzed. In the audit model suggested, patient social-indicator data and social worker judgments of social needs and outcome are available. I t is necessary t o determine if the clients not interviewed are significantly different from the interviewed. For example, in one study at Mount Sinai we found that we were able to interview significantly fewer family members of patients who had died during the hospitalization than family members of other patients. This definitely put a different light on the meaning of our data. The findings had t o be interpreted in view of the fact that certain data were missing and that the findings could not be generalized t o family members of the dying patient. STEP 6

Data Collection In determining the methodological design for data collection, the actual operation of the hospital and the department's system of practice must be considered. Each hospital has certain particulars and unique aspects t o its system which must be taken into consideration when designing a research study. For example, in determining the viability of content analysis of previously recorded records, those records must be examined t o be sure that they contain information on which the study is dependent. If retrospective data from case records are to be used, the issue of missing data must be allowed for in the study design. Thus, the method of data collection requires a research decision as t o the use of closed or open cases from which t o obtain data. There are many problems in retrospective closedcase data. Closed cases have been recorded for purposes other than the current research and may not have the potential t o provide the kind of information that is sought. Lately, at Mount Sinai we have been doing prospective studies, which means that the audit is conducted on cases, for example, which are open during a one-month period and closed over the following two t o three months. There are also problems in prospective data collection. If you are going t o conduct a prospective study which has social workers completing a special questionnaire, the question of the reliability and validity of

Downloaded by [University of Tennessee, Knoxville] at 07:57 30 December 2014

Barbara Berkman and Helen Rehr

their answers must be considered. While there may be no absolute means to alleviate this problem, there are research methods to account for most problems of this sort. For example, the use of multiple questions to get the answers to one issue can be used to ascertain the validity or "truth" of the findings. A prospective audit method utilizing telephone interviews with clients assumes that client judgment has some validity and will reflect certain problems in the program. One methodological argument which has been raised against consumer evaluation is that the clients do not have the necessary qualifications to evaluate professional services, and the findings will be biased or superficial. This type of evaluative approach, using client judgments, is methodologically valid, especially in looking at such issues as the clients' views of accessibility of service, acceptability of service, continuity and comprehensiveness of care. Around these issues the consumer can and does make an assessment of the services received, deciding whether to return, and whether to recommend the services to others. Within this particular frame of reference, the client is probably the only valid source or criterion of whether service is achieving its goals. But what of the clients who are going t o continue t o come to the hospital because of their outpatient status, for instance? Will these clients commit the errors of omission or commission because they are fearful of saying what they really feel? We must consider this as a possible validity problem and analyze our data with the possible implications of this in mind. Another issue is whether to use questionnaires which the respondent fills out himself or whether to use a more expensive interview approach. What about telephone versus direct interviews? What about questions in which the full answer given to us by the client is written out versus questions that have "closed" answers. (In other words, open-ended questions versus precoded answers that have been determined to cover all possible answers, perhaps with a category for "other. ") Exploratory interviews with clients which permit open-ended responses to in-depth questions may be more cogent and valid than other approaches using quantitative measurements. Through direct questioning of clients we are able to explore areas which at this time in our research practice sophistication would be limited and invalid if we used standard codifiable measurements. These types of research issues can be answered based on the system in which you operate, and there are many standard research texts which give the various pros and cons to each approach.' What must be kept in mind is that there is never one, superior, "only" method to do a study, but rather the research design must be developed within the context and limitations of the setting in which the study is being conducted.

SOCIAL WORK IN HEALTH CARE

Downloaded by [University of Tennessee, Knoxville] at 07:57 30 December 2014

STEP 7 Analyses In the analyses and interpretation of findings, the data should be treated as exploratory with the awareness of the many explanatory variables which have not been available so that in explaining your findings you do not get fooled by the weaknesses of your own design. The meaning of the data should be discussed with the staff in order to get the full breadth of the analyses. In each step of the audit analyses, there must be an awareness of the pros and cons of why the study was designed in the way chosen. There may have been only one way to design the study that was feasible or economical at a certain point in time. The researcher should be aware of this and in the write-up, the limitations of the methodology, as it may have effected the data obtained, should be stated clearly. At Mount Sinai, we have done a number of studies literally on a shoestring because we have had the encouragement of the administration and workers in the department who were willing to examine their case load and who were willing to follow certain structured, often precoded, questionnaires which may have been somewhat timeconsuming. The studies we are proposing can be done in a minimum amount of time. Data can be collected for one month and the analysis done in staff meetings where the implications of the findings can be openly discussed. The time to ask the question "Should we do audit studies?" has passed. The demands for accountability require that we examine our practice systematically and account for our contributions t o the care of patients and their families. To be accountable as a profession, we need to answer: what is unique about social work? This question can be answered through the use of an appropriate social-need data system. We can, indeed, show what it is that we work on that other health professionals do not. It is our premise at Mount Sinai that, through some such social need/outcome approach, we will eventually be able t o answer consistently the questions of relevance, efficiency, and effectiveness posed to us by consumers, other health professionals, and reimbursing agencies.

REFERENCES 1 . Eveline Burns, "Health Insurance: Not If, or When, but What Kind?" American Journal ofpublic Health 6 1 , no. 11 (November 1 9 7 1 ) : 2164-75. 2. The authors are indebted to Molly Groh's excellent paper "Teaching Graduate Students Accountability t o the Agency Through Evaluative Studies" (presented at the annual Program Meeting, Council o n Social Work Education, Chicago, Ill., March 3 , 1 9 7 5 ) .

Barbam Berkman and Helen Rehr

3. Barbara Berkman and Helen Rebr, "Social Needs of the Hospitalized Elderly: A Classification," Social Work 17, no. 4 (July 1972): 80-88.

Downloaded by [University of Tennessee, Knoxville] at 07:57 30 December 2014

4. Barbara Berkman, "Are Social Service Audit Systems Feasible? Experiences with a Hospital-Based and Regional Approach" (paper presented a t the Society for Hospital Social Work Directors PSRO-UR and Social Work Working Conference, Fort Worth, Texas, January 18-21.1976). 5. Two excellent research texts are: CIair Selltiz e t al., Research Methods in Social Relations (New York: Holt, Rinehart and Winston, 1962); and William Goode and Paul Holt. Methods in Social Research (New York: McGraw-Hill Book Co.. 1952). 6. William Hall and Gerald St. Denis, "Accountability: A Critical Issue in Social Services" (report of meeting, Webster Hall Hotel, Pittsburg, Pa., June 19-22, 1972).

Seven steps to audit.

This article was downloaded by: [University of Tennessee, Knoxville] On: 30 December 2014, At: 07:57 Publisher: Routledge Informa Ltd Registered in En...
339KB Sizes 0 Downloads 0 Views