Quality Assurance in Health Care, Vol. 4, No. 3, pp. 205-215, 1992 Printed in Great Britain

1040-6166/92 $5.00 + 0.00 © 1992 Pergamon Prea Ltd

EFFECTIVENESS OF THE DAILY RECORD REVIEW AT THE MEDICAL EMERGENCY ROOM DEPARTMENT IN A FRENCH TEACHING CHILDREN'S HOSPITAL Pierre Lombrail, Ariane Leroyer, Christine Vitoux-Brot,* Catherine Dolfuss,* Antoine Bourrillon,t Francois Beaufils,t Max Hassan§ and Marc Brodin Service de Santd Publique 'Service des Urgences tService de Pidiatrie Gdnirale ^Service de Reanimation Polyvalente 5 Service de Radiologie Hflpital Robert Debrf 48 bd Senirier 75019 Paris France (First submitted 12 September 1991; accepted after revision 5 January 1992)

We evaluate the effectiveness of the daily record review (DRR) of 4393 ambulatory medical patients seen at the emergency room department of a teaching pediatric hospital in Paris between 8th January and 11 March 1991. For these patients, 137 potential quality problems were found, 117 of them remaining after discussion with the junior. For 80 of the 117 confirmed problems (68.4%), the reviewing senior estimated that sufficient advice had been given to the child's usual caretaker or health care provider and (or) the children returned to seek medical advice when indicated. Experts proved the procedure to be reliable: they detected a problem for 36% of the cases with potential quality problems and for only 2% of a sample of control records considered without potential quality problems after the DRR. Key words: Quality assurance, pediatrics, emergency care, hospital, medical record.

INTRODUCTION

Adverse events occurring in the emergency room department (ERD) are rated as infrequent as 3% of all the adverse events faced by hospitalized patients in New York state's hospitals in 1984 [1]. However, if the relative risk of adverse event incidence is low, the attributable risk is potentially high because of the high turn-over of patients cared for in a short time by a junior staff. First presented to the 8th Congress of ISQA, Washington, 29-31 May 1991.

205

206

P. Lombrail et al.

The goal of quality assurance programs is to achieve the best outcome for the patients. Thefirststep is to avoid adverse events by considering structure and process attributes leading to such outcomes. Quality assurance programs are developed in the ERD setting [2]. Managing negative outcomes at this level means reducing or eliminating departures from expectations along the chain of processes leading to the outcome [3]: triage, physician evaluation, ancillary services work-up, disposition and follow-up. One of the goals of a QA program is to screen on-time for potential quality problems and to take corrective actions when necessary. Neither the diagnostic accuracy of such a program (i.e. its predictive value) nor its ability to allow effective corrective actions have been evaluated to our knowledge in the ERD setting. This is the aim of our study.

MATERIAL AND METHODS

The Setting Robert Debre" Hospital is a 500-bed children and mothers' hospital open since May 1988 in the north-eastern part of Paris. The pediatric emergency room department (ERD) is supervised by the chiefs of three departments: general surgery, general pediatrics and intensive care medicine. It consists of a medical and a surgical unit. Medical staffing of the medical unit is: between 8 a.m. and 6 p.m.: three full-time senior pediatricians and six juniors (residents and interns according to the French system); between 6 and 12 p.m.: one senior and two juniors; between 0 and 8 a.m.: two juniors. The ERD is located close to the intensive care unit and the juniors may seek help immediately from the seniors on duty 24 hr per day in this unit. Juniors stay 6 months at the ERD and shift in November and May. They are trained during the 15 days following their arrival at this hospital on the essentials of pediatric emergency care (11 2-hr sessions for refreshing academic knowledge and learning about the local functioning). At any time a typical patient is seen by a junior. A patient is looked after by the senior only if the junior feels unsecure with the case, or if the senior is alerted by a nurse or in the case of overcrowding. A medical sheet is recorded for each patient mentioning reason for visit, essential findings from history and physical examination, disposition and follow-up. If necessary a prescription sheet is made and the duplicate kept with the medical sheet. The Daily Record Review (DRR) Since the opening of the hospital, each morning, including weekends, one of the seniors reviews the records of medical ERD patients seen in the previous 24 hr and discharged home, i.e. the medical sheet and the prescription sheet, if any. The review encompasses also the laboratory reports not available at the time of discharge. All the X-Ray films taken in the same period for surgical and medical patients and read by a junior radiologist are also reviewed by the head of the radiology department. If a potential quality problem is detected, the radiological record is transmitted for action to the senior pediatrician. The review of medical and

Effectiveness of DRR

207

radiological records is based on implicit criteria. Definitions of potential quality problems are listed below. Once a potential quality problem is detected, the senior may seek further explanation from the doctor who cared for the patient. If a doubt about a quality problem persists, or if there has been no discussion, the senior attempts to reach the child's parents or the child's usual health care provider. This is done by phone if urgent action is needed or otherwise by mail. Reassurance concerning the child's health status may be obtained after the phone contact or sufficient counselling given. Counselling concerns mainly how to look after the child or a treatment modification. If there is evidence of an unresolved health problem, the patient may be called back to the ERD immediately, told to receive a revaluation in a physician's office or offered an appointment at the hospital's general pediatrics department.

The DRR Evaluation The study was conducted during 9 weeks in 1991, between 8 January and 11 March. Of the 5115 children who.consulted at the medical ERD during the 63 days of the study, 4393 were not admitted and constitute the study's population. A treatment sheet was established for 80% of them; 499 had an X-Ray examination. For each potential quality problem the seniors documented the type of problem and the action undertaken. The effectiveness was evaluated for the potential quality problems identified by the DRR. Was a corrective action necessary? If it was, could it be implemented? To answer these questions, we asked (by phone call or by letter) the parents told to seek medical advice if they did and we looked at the computerized administrative system to check for patients returning to the general pediatrics department when it was planned. For these patients, we returned to the medical record to check if the reason for visit fitted the follow-up planned after the ERD visit. Finally we considered the DRR effective on two occasions: (1) if the potential quality problem could be ruled out or corrected after contacting the parents or the child's principal health care provider: (2) if the parents complied with the recommendation of seeking further medical advice. To test the reliability and the validity of the DRR, we provided two independent experts with all the records identified as a "potential quality problem" by the DRR and an equal number of "controls". Experts were senior pediatricians from the general pediatrics department who were not involved in the routine work of the ERD. Control records were selected randomly from all the records considered as being without quality problems by the DRR. Each medical record (with the exception of the X-Ray films) was blindly rereviewed in the same way as the DRR was conducted, i.e. without the expert knowing whether a particular record was that of a case or of a control. The conclusion was: presence or absence of a definite quality problem. If the experts disagreed, we considered as a gold standard the conclusion of a third re-review done by the head of the general pediatrics department who ran this activity for a long time in another hospital. A definite quality problem was defined as a situation in which the experts considered that a corrective action had to be taken. The problem could be an inadequate work-up or a harmful therapy as defined above with implicit criteria.

208

P. Lombrail et al. Quality Problem Definitions

A record with a potential quality problem is a record presenting any of these findings: doubt concerning the work-up of a case, harmful therapy or abnormal radiological findings. A doubt concerning the work-up of a case may arise on several occasions: — The history or physical examination documented on the medical sheet doesn't allow for the review to rule out a potential quality problem (Example: the reason for visit is a cold and there is no mention of an ear examination concluding the presence or absence of otitis media). — Potential discrepancy between the chief complaint and the diagnosis at discharge (Example: 3-month-old child with fever discharged with a diagnosis of viral infection). — Cases where a potentially severe condition does not seem to have been ruled out (Examples: a child with blood in the feces returning home with a diagnosis of gastro-enteritis; diagnosis of bronchitis without eliminating the possibility of whooping-cough for a 2-month-old child with fit of coughing). — Due paraclinic examination apparently not done or not worked-up (Example: test for urine nitrites not documented for a child consulting with persisting fever). Harmful therapy is documented on several occasions: — Inappropriate drug dosage or duration (Example: overdosage of antipyretic drugs or underdosage of antibiotics). — Undue prescription (Example: corticotherapy for a bronchiolitis) or lack of prescribing of a due prescription (Example: lack of nutritional counselling for a child with diarrhea). — Inadequate disposition or follow-up (Examples: child with diarrhea and 9% dehydration not admitted; child with diagnosed gastro-oesophageal regurgitation returning home without follow-up appointment. — Lack of follow-up for children having paraclinic examinations at the ERD (microbiological, for example). For the drug prescription, the goal of the review is to pick up only treatments potentially endangering a child. Risk evaluation is left to the appreciation of the senior. The doubts concerning the work-up and a harmful therapy are categorized as clinical problems. An abnormal radiological finding is defined by the senior radiologist detecting an inadequate reading of X-Rays.

Statistical Analysis

Comparisons are done with Chi-squared for proportions and non-parametric tests (Willcoxon) for quantitative data. Proportions are expressed with their 95% confidence interval.

Effectiveness of DRR

209 RESULTS

The DRR The exhaustivity of the DRR is 96% for the medical sheets, 82% for the treatment sheets and 100% for the X-Ray films. The daily mean time required from the seniors was 1 hr and 10 min for reviewing the medical sheets (range: 20 min to 2 hr 30 min) and 1 hr for reviewing the X-Ray films (range: 30 min to 2 hr 30 min). One hundred and thirty seven records with a potential quality problem were identified (95% CI: 3.1 ± 5.1%). Problems identified are: 40 doubts concerning the workup of a case (29.2 ± 7.6% of the problems), 77 harmful treatments (56.2 ± 8.3%) and 31 abnormal radiologicalfindings(22.6 ± 7.0%). Twelve records with a doubt concerning the workup and a harmful treatment were counted twice. Complete data are available for 136 records. Fifty five of the 105 records with a potential clinical problem were discussed with the junior; 38 were confirmed. Reasons for discussion were: inadequate documentation of the case (30), and teaching (19) or precision needed concerning the followup (7). The process continued for 88 records with a potential clinical problem (doubt concerning the work-up and/or harmful treatment). Two abnormal radiological findings were for health problems already cared for by the pediatricians; the process continued for 29 radiological records with an abnormal finding. A contact with the family or the usual health care provider was attempted for 108 of the 117 cases with the probability of a problem remaining after discussion: 90 by phone and 18 by letter. The contact failed (6) or was not attempted (8) for 14 cases. Three times the parents returned to the ERD before the contact was achieved and twice a letter was sent back to the hospital. The potential problem could be eliminated after the contact with reasonable confidence for nine children. Sufficient counselling could be given for the procedure to stop for 22 children. Twenty six children were called back to the hospital (at the ERD or at the general pediatrics department) and 11 were told to seek advice from their usual health care provider. The whole trajectory is detailed separately for the three types of potential quality problem in Figs 1-3. Effectiveness of the DRR Twenty-one of the 31 problems concerning a doubt about the work-up of the case were solved as well as 42 of the 66 treatment problems and 20 of the 29 abnormal radiological findings. Taking into consideration the children, whatever the number of problems they could have, the global effectiveness of the DRR was 68.4% (80/ 117), i.e. the physician estimated to have given sufficient advice and (or) the children returned to seek medical advice when indicated. Reliability and Validity of the DRR The reliability and the validity of the DRR were evaluated blindly by senior pediatricians for 102 records with a potential clinical quality problem and 102

210

P. Lombrail et al. 5115 Patients attending the medical ERD 722 "Patients admitted 4393 Patients not admitted (Study population) 40 Doubt concerning the worX-up Doubt eliminated after discussion with the intern 31 Doubt remaining after discussion Action cancelled or failed 27 Corrective action

_8 'Reassurance after phone call 19 Counselling (parents, usual care provider) 6 Ineffective action 13 Effective action FIGURE 1.

Trajectory of the patients with a doubt concerning the work-op.

controls. The children of both groups were quite similar for age, sex, weight, temperature and arrival time (Table 1). Thirty nine records were considered as presenting a definite quality problem (39/ 204 = 19%- Table 2). We can therefore estimate that the positive predictive value of the DRR is 36.3 ± 9.5% and its negative predictive value 98.0 ± 2.8%. DISCUSSION The validity of our observations may be challenged. First, concerning the diagnostic accuracy of the review, did we measure what was worthwhile? The DRR is based on process assessment and it is possible that we missed some adverse events only detectable by outcome assessment. The ability to assess final outcome for every patient is a major shortcoming of most QA systems where continued patient followup is lacking. The quality of the process assessment itself depends on the quality of the documentation of the cases on the medical record. If underdocumentation usually leads to false positive screening (like in this study), it may also cause false

Effectiveness of DRR

211

5115 Patients attending the medical ERD 722 Patients admitted 4393 Patients not admitted (Study population) 77 Potentially harmful therapy _11 Problem eliminated after discussion with the intern 66 Problem remaining after discussion 12 Action cancelled or failed 54 Corrective action Reassurance after phone call 52 Counselling (parents, usual care provider) 12 Ineffective action 40 Effective action FIGURE 2.

Trajectory of the patients with a potentially harmful therapy.

negative screening. Finally we were able to review 80% only of the treatment sheets. For all these reasons we may probably have overestimated slightly the negative predictive value of the DRR. However, our data are in keeping with the 4.4% total error rate observed by Harchelroad et al. [2] for surgical and medical cases. The predictive values of the procedure are those of a good screening test [4]. The ability of the medical record to be a good screening tool for quality problems has been challenged. While some studies documented a reliable detection of severe adverse events suffered by hospitalized patients [4], others exhibited an underreporting of mild perioperative complications [5]. Our medical sheet is structured to guide the recording of history, details of physical examination, disposition and follow-up. However, like every hand-written narrative record, it is often ambiguous, over-abbreviated or illegible. This is a limitation of the DRR. A possible way to enhance the quality of recording is to think of a different way to structure the sheet. One approach is to build the sheet for supporting a tool providing an aid to the decision. Ideally this would enable clinicians to manage risk and quality on a real-time basis. Research has to be done in this direction. Another less ambitious way is at least to record the information necessary to screen for

212

P. Lombrail et al. 4393 Study population 439 X-Ray examination 31 Abnormal radiological finding 2 "Health problem already known 29 New finding 5 "Action cancelled or failed 24 Corrective action 3 "Reassurance after phone call 21 Counselling (parents, usual care provider) 4 Ineffective action 17 Effective action

FIGURE 3. Trajectory of the patients with an abnormal radiological Ending.

potential quality problems. In the later approach, to establish a list of at-risk situations may contribute to selection of the core information set needed for quality assurance purposes. We agree with Murphy and Jacobson that "researchers must work toward record-based measures that are valid and reliable and contain elements that can be expected to affect health status" [6]. The recognition of at-risk situations may be achieved by establishing a list of "trigger diagnosis". Holbrook and Aghabadian defined a trigger diagnosis as the diagnosis recorded by the physician who "failed to diagnose" a more serious TABLE 1.

Evaluation of the reliability and the validity of the DRR

Age (months)* Sex (males %) Weight (kg)* Temperature (°C)* Arrival time (% between 8 a.m. and 8 p.m.) 'Mean (range).

Potential quality problem (N=102)

Control (N = 102)

38.5 (0-168) 52.9 14.6 (3.6-58.0) 37.9(36.0-41.1) 67.5

38.0 (0-180) 55.9 14.2 (3.0-48.2) 37.7 (36.2-40.4) 66.7

213

Effectiveness ofDRR TABLE 2.

Reliability of the DRR (204 records) Definite quality problem for experts Yes

No

Potential quality problem (DRR, N = 102) Controls (N = 102)

37 2

loot

Total

39

165

65*

•Positive predictive value: 36.3 ± 9.5%. tNegatrve predictive value: 98.0 ± 2.8%.

condition [7]. Limiting the search to diagnosis would be insufficient in a pediatric setting. At least age and temperature have also to be taken into consideration to characterized at-risk situations. The definition of what would therefore be a list of "trigger situations" may have several benefits. First, it could produce a specific list of itemized deficiencies that could be used to improve the training of the juniors on their arrival at this institution. Second, it may help to optimize the review by identifying explicit criteria. This is disputable since implicit review has been shown to have a higher rate of problem detection for example in the case of blunt trauma patients [8]. Finally, it could lead to automatization of the quality assurance procedure in the future. The automatic detection of an at-risk situation "should alert the clinician that the presence or absence of certain keyfindingsmust be collected and recorded on the chart to exclude the more serious conditions" [8]. We failed to identify such situations (data not shown). The way we have defined the effectiveness of the review has to be discussed too. Counselling given to the parents does not equal efficacy for at least two reasons. First, we have no information about their compliance with the recommendations. Second, some recommendations were done by letter and we have no information about the reception rate nor the timeliness of the counselling. However, the majority of the letters (26/33) were for X-Ray findings allowing time to be taken for a corrective action. Second, return to the hospital of a child who has received counselling doesn't guarantee that the child's problem has been handled correctly by the physician. If we consider as effective only the actions for which a direct contact has been established with the family, a corrective action was implemented for 67 of the 117 children with a problem identified by the DRR (57%). With such definitions, physicians can implement a corrective action in no more than a half of the cases for which they think by implicit judgement it would have been necessary. The effectiveness is slightly better for definite problems identified by the experts: an action could be taken for 27 (73%) or 25 (67.5%) of the 37 medical cases if direct contact only was taken into consideration. The difference with the potential problems did not reach the level of statistical significance. This relatively poor result has to be tempered. Some potential problems were judged a posteriori by the reviewing pediatricians as not so serious as to necessitate absolutely establishment of a contact. We have further information for 13 of the 16 children for which a corrective action was cancelled or failed. Five didn't undergo the planned visit because the child recovered; three returned to the ERD before a

214

P. LombraU et al.

corrective action could be implemented; two were seen by another physician; three didn't undergo the planned visit and we have no information concerning the child's health status, but the initial condition was not really serious. On the contrary, all serious conditions but one could be handled effectively. The exception is for a child with gastroenteritis and dehydration. The phone call could not reach the family on the way to a remote city where the child was hospitalized. Finally, all the staff remember the case of a 5-year-old boy called back for abdominal pain and cured of intestinal intusception and the case of a child with fever and some purpuric spots on the face who was shown to have meningitis when called back to the hospital a few hours later. Such cases are enough for the clinicians to justify the procedure. Other justifications are its pedagogic effect, which is part of the role of a teaching hospital, and the sense of confidence induced in the public as attested by positive comments during the phone calls. To avoid adverse events is a part of delivering high quality care.. Limiting quality assurance to this goal would emphasize disproportionately the technical component of care and the physical-physiological aspects of health. Evaluative attributes for quality assessment in a hospital emergency unit should be broader [9]. Physicians have to consider that portion of an episode of care for which the emergency unit can reasonably assume responsibility. At the ERD level this means at least to consider the appropriateness of the referral as well as the maintenance of continuity of care through successful linkage with a more stable source of care. This is a critical issue not dealt with at present [10]. It means also patient education and motivation with a view to prevention. Finally, evaluation of quality of care at the hospital level would be incomplete without taking into consideration the dimension of accessibility. For some parts of the population of western countries as for the majority of people living outside these countries the problem of access to the health system is a greater problem than lack of quality within it [11]. Acknowledgement: This study was supported in part by a grant of the Agence pour le Developpement de l'E valuation MeViicale.

REFERENCES 1. Leape L L, Brennan T A, Laird N M et al.. The nature of adverse events in hospitalized patients. Results of the Harvard Medical Practice Study II. N EnglJ Med 324: 377, 1991. 2. Harchelroad F P, Martin M L, Kreman R M and Murray K W, Emergency department daily record review: a quality assurance system in a teaching hospital. QRB February: 45,1988. 3. Dagher M and Lloyd R J, Managing negative outcome by reducing variances in the emergency department. QRB January: 15,1991. 4. Troyen A, Brennan A, Localio R and Laird N M, Reliability and validity of judgments about severe events suffered by hospitalized patients. Med Care 27: 1148, 1989. 5. Svarsudd K, Arvidsson S, Ouchterlony J, Nilsson S and Sjostedt L, Perioperative risk assessment. The project perioperative risk (PROPER). Quality Assurance Health Care 2: 243, 1990. 6. Murphy J G and Jacobson S, Assessing the quality of emergency care: the medical record versus patients outcome. Ann Emerg Med 13: 158,1984. 7. Holbrook J and Aghabadian R, A computerized audit of 15,009 emergency department records. Ann Emerg Med 19: 139-144,1990. 8. Schriger D L, Baraff L J and Fink A, A comparison of implicit and explicit methods of process quality assurance for blunt trauma patients. Ann Emerg Med 19: 736,1990.

Effectiveness of DRR

215

9. Rhee K J, Donabedian A and Bumey R E, Assessing the quality of care in a hospital emergency unit: a framework and its application. QRB January: 4, 1987. 10. Pierre Lombrail, Cristina Alfaro, Max Bensadon and Marc Brodin for the Parisian study group for pediatric care, Continuity of care for children consulting at an emergency room department. Poster presentation at the 8th conference of the ISQA, Washington, 28-31 May 1991. 11. Link J A, The quality of health care: the practical clinical view. Quality Assurance Health Care 2:219, 1990.

Effectiveness of the daily record review at the medical emergency room department in a French teaching children's hospital.

We evaluate the effectiveness of the daily record review (DRR) of 4393 ambulatory medical patients seen at the emergency room department of a teaching...
500KB Sizes 0 Downloads 0 Views