HEALTH SERVICES RESEARCH 0
RECHERCHE EN SERVICES DE- SOINS DE SANTE'
Qual'ity of care: 2. Quali'ty of care studi'es and
their consequences
Health Services Research Group A study assessing the process of care in a chain of ambulatory health care centres reported substandard care in 25% to 75% of cases of common ambulatory problems; the rate of provision of "at least adequate care" by individual physicians ranged from 44% to 90%.1 In another study the rate of unnecessary surgical procedures was shown to be about 1 in 6.2 Are these findings the rule rather than the exception? Do they represent global "truths" about the quality of care in the health care system? Are they generalizable to all settings? All practitioners would be deeply disturbed at such results in their own practice setting and would wish to rectify matters. How can the quality of care be assessed, and can it be improved? As we noted previously3 the determinants of practice patterns and outcomes as well as the consequences of changes to the health care system remain largely unexplored. Studying the existing quality of care is thus a necessary beginning.
Design and analysis of studies measuring quality Quality assessment studies range in size and sophistication - from the $100 million Rand Health Insurance Experiment4 (a controlled randomized trial of different payment plans) to retrospective chart audits of a single diagnosis within a single organization. Although a few controlled experiments
have been undertaken, the difficulty in manipulating health care delivery to individual patients means that observational (i.e., cohort, case-control and cross-sectional) or quasi-experimental studies are more usual. A common analytic strategy is to describe the variations in health care outcomes and in the structures or processes of care. Studies of quality of care that aim at effecting improvement or maintaining high levels of care may be directed most efficiently at areas of high risk, high volume, high cost and high rates of problems. Each health care organization must look within its own delivery system to identify the extent of potential problems. Studies are challenging to perform and interpret. Consistency across several independently performed investigations, however, is invaluable. For example, there are numerous articles reporting an association between a high volume of surgery and good outcome (when outcome measures are the length of stay and the rate of death in hospital).5 A further understanding of the mechanisms of the outcome-volume relation may be reached: Hughes and colleagues6 demonstrated that the mechanisms "practice makes perfect" (i.e., higher volume leads to better outcome) and "selective referral" (i.e., better outcome leads to higher volume) were operating in the case of hip replacement. Analyses such as these may be conducted with the use of hospital abstract databases, administrative
Members: Drs. Antoni S.H. Basinski (principal author), Department of Family and Community Medicine, Marsha M. Cohen, Department ofHealth Administration, C. David Naylor, Department of Medicine, Lorraine E. Ferris, Department of Behavioural Science, J. Ivan Williams, Department of Preventive Medicine and Biostatistics, and Hilary A. Llewellyn-Thomas, Faculty of Nursing, University of Toronto, Toronto, Ont.
The Health Services Research Group is part of the Clinical Epidemiology Unit, Sunnybrook Health Science Centre, Toronto, Ont. Reprint requests to: Health Services Research Group, Clinical Epidemiology Unit, Sunnybrook Health Science Centre, 2075 Bayview Ave., Toronto, ON M4N 3M5 JULY 15, 1992
CAN MED ASSOC J 1992; 147 (2)
163
hospital admissions adverse events occurred that were due to negligence; one quarter of these events led to death or permanent disability. Despite the lack of agreement about the level of the quality of care in individual cases it is clear that quality problems are not uncommon. Processes and outcomes of care provided by individual providers, institutions or within small areas must be assessed with caution: case-mix severity, variations in demographic and risk factors, social and economic circumstances and available resources will all affect the results. Even if these are taken into account there may be "unexplained variation," the result of unidentified and unmeasured factors and random fluctuations rather than of faulty practice. Assessment by implicit review can be unreliable. Explicit review may fail to capture the effects of important factors or unduly weight others. Shortterm studies may show changes in process or outcome measures, but in the long term these effects may be diminished and difficult to demonstrate. Observation of the health care process may lead to the Hawthorne effect - that is, knowing that they are being assessed, providers act differently than they would normally. This effect cannot be accounted for in uncontrolled observational studies lacking adequate comparison groups. As with all epidcare. Certain studies evaluate the impact of changes emiologic, observational studies, sampling and in the system of delivering care. In a recent study8 measurement biases and confounding variables may explicit process criteria and scales were applied to create problems in imputing causation. medical records of randomly selected patients admitted to hospital with congestive heart failure, acute Response to information myocardial infarction, cerebrovascular accident, pneumonia or hip fracture. For all these conditions a Once the level of quality has been identified the better process of care, whether measured explicitly next step is to respond to the information. There are by process criteria8 or by a structured, implicit two approaches. The "macroresponse" to quality judgement system,9 was associated with lower rates research may entail the modification of systems of of death 30 days after admission. In addition, payment or of reimbursement levels for procedures improvements in the process of care after the intro- or the promulgation of broad guidelines duction of a prospective payment system were asso- for aspects of medical care in an attempt to influciated with a 1% reduction in the rate of death 30 ence the overall system of health care. For examdays after admission.8 Yet the improvements were ple, a comparison between Florida and Baltimore of accompanied by an increase in the number of the use of magnetic resonance imaging services patients considered to be unstable or discharged too led to the suggestion that "a significant causative soon,9 and by 180 days after admission the 30-day factor for the higher Florida utilization and cost improvement in the rate of death was halved. '0 experience is the large number of imaging joint These results underscore the importance of measur- ventures in which physicians have financial incening a wide range of end points over a period long tives to refer patients and increase the number of enough to include other important consequences of referrals."'3 One response to these findings would be changes to the system. legislation to limit joint ownership of diagnostic Despite the observed improvements in care facilities by physicians who are not radiologists in these studies also revealed that 12% of patients were active practice. deemed to have received poor or very poor care.91'0 The "microresponse" involves a focus on specifThis is in accord with the findings of the Harvard ic local problems. The approach of continuous qualMedical Practice Study,"'12 in which a quarter of all ity improvement of process or outcome (e.g., by adverse events and half of the serious adverse events decreasing reporting time for radiologic results or were attributed to negligence. Overall, in 1% of increasing patient satisfaction with services) within
databases or registries, but they are limited in their ability to provide more detailed information on the quality of care. Data from medical records, however, may provide such necessary additional information. For instance, in a study of the abstracts from all 91 hospitals in a chain across the United States Dubois and associates7 found marked variations in rates of death from cerebrovascular accident, pneumonia and acute myocardial infarction. "Outlier" hospitals (i.e., those with high and low rates) were identified and studied further. A review of the medical records enabled adjustments to be made for the complexity and severity of illness. The adjusted death rate in the high-outlier hospitals was only 3% to 10% more and in the low-outlier hospitals only 10% to 15% less than that predicted from the severity of illness. Moreover, no significant differences in the quality of the process of care were found with the use of 125 explicit review criteria. An expert "implicit" review revealed that greater numbers of preventable deaths occurred in the high-outlier hospitals. However, without explicit criteria to guide them the experts tended to disagree as to which deaths were indeed preventable. Hence, a review of even the best current information may lead to an unacceptable lack of reliability in judgements of good and poor quality of
164
CAN MED ASSOC J 1992; 147 (2)
LE 15 JUILLET 1992
specific settings may pertain. Strategies to improve the quality or efficiency of care have evolved along a number of lines: education, peer review with feedback, administrative changes, participation, and penalties and rewards.'4 Many of these have been tested, albeit in teaching institutions over short periods, with hospitalized patients and physicians in training. Feedback may take many forms; for instance, an institution may attempt to reduce the rate of unnecessary tests. A simple intervention might provide feedback of the cumulative costs of patient services.'5 '6 More intensive feedback involving peer review of the necessity of services ordered is labourintensive and requires a sustained effort.'7 Although these forms of intervention can lead to decreases in rates of use with no sacrifice in apparent quality of care, it is not clear that sustained improvements in the delivery of care will result. For instance, although educational programs can decrease the rate of ordering of laboratory tests during the period of intervention the rate has been found to return to baseline levels after withdrawal of the intervention. 16 Macroresponses and microresponses may be combined. For instance, in Saskatchewan a review committee was formed to establish a list of indications for hysterectomy that were generally accepted as conforming to good practice.'8 Peer feedback to individual hospitals in which there had been a large number of unjustified hysterectomies was associated with decreases in the proportion of such operations from 24% to 8%. The decrease coincided with publicity about the excess rates and began before the review committee met. It was concluded that the excess hysterectomy rates "resulted from a breakdown of adequate review at the hospital level."'8 Alternatively, changes in the approach to patients may influence process and outcome. For example, clinical trials have tested interventions aimed at increasing patient involvement in care by reviewing with each patient his or her medical record and encouraging the patient to negotiate medical decisions with the physician.'9'20 These interventions led to improvements in outcomes, such as the control of diabetes'9 and reduced functional limitations.'9'20 Similarly, a randomized controlled trial of the effect of a self-care health education program on the use of ambulatory care demonstrated a decrease of 15% in the total number of medical visits and in direct costs and no negative impact on the quality of health.2'
necessary for clinical research studies to be successful and valid. Although not always sufficient to ensure success these conditions provide a foundation for inference from study outcomes. Factors that may promote or impede quality of care assessments and influence the ability to respond to their results have yet to be formalized. However, advocates of continuous quality improvement emphasize that a number of factors may influence success.22,23 The prescriptions for the formulation of quality initiatives are rooted in industrial management theories rather than in clinical science. Although unsupported by experimental experience they appear to be a reasonable starting point. Quality initiatives are more likely to succeed if health care managers - physicians and administrators - provide leadership and demonstrate a commitment to evaluate and, if necessary, change the existing system while providing the resources to do so. The goals and conduct of quality research and the response to it must be acceptable to providers, payers and patients. Providers are more likely to contribute toward changes in the system if they feel a sense of ownership and involvement in the quality improvement initiative. Providers furnished with information on their own performance are more likely to make changes that they see are necessary. Payers for health care services in Canada have traditionally observed the processes and outcomes of health care from a distance. With increased pressure on resources and increased competition for funding of programs, payers will demand a more active role in the management of health care. The demonstration of quality of care will become increasingly important in the choice between competing demands. Patients must feel that they are represented, their interests safeguarded and their views meaningfully taken into account. Education of consumers reinforces improved health care;20'2' in turn, informed consumers provide direction to the system through feedback and demand. Assessing and changing existing systems is often difficult. Many factors may consistently militate against sustained or substantial changes. For instance, payment systems have inherent biases. Although the fee-for-service method may encourage inappropriate overuse of services, prepaid reimbursement may foster inappropriate underuse. Resistance to evaluation and change may arise from many sources, including a fear of blame and adverse consequences. Moreover, policies and directives for Factors affecting initiatives to assess and change may be viewed as arbitrary rather than improve the quality of care necessary. Often the data cannot answer the questions Various prerequisites (e.g., avoidance of bias and the reliability of measurements) are considered posed, and the links between structure or process JULY 15, 1992
CAN MED ASSOC J 1992; 147 (2)
165
and outcomes are tenuous and inconsistent. The consequences of some processes (e.g., preventive care) may not be evident for a long time. Outcome rates are often low and may be subject to wide random variation. Variations among patients and among providers may have a much greater influence on the desired outcome than do the processes of care. Many social, economic and psychologic factors come into play, and these remain largely unaccounted for in assessments of quality. Difficulties in defining and obtaining valid and reliable measures of health care quality impede assessment. Interrater and intrarater reliability of implicit measurements is often low, and yet explicit standards or guidelines may be controversial and amount to little more than the formalization of guesswork.24
Future directions Further research and developments can be expected on a number of fronts. As we noted in an earlier paper3 the release of data on death rates in hospitals led to multiple attempts to explain the data. However, these analyses used different measures of outcome, case mix, severity, process quality and hospital factors.25 The focus of health services research is moving from exploratory and explanatory studies to applications, policy development and more generalizable research. Because the pressure to compare the quality of care across jurisdictions is increasing, reliable and valid measures, methods and comparisons are needed, as are standardization of the assessment and improvement of quality. Progress is being made on these fronts.26 Population-based analyses become more feasible when information is readily available and standardized. In these analyses the population "cared for" by a particular system of health care is tracked. Studies that track patients through different components of care (from primary care to specialist referral to hospital admission and then discharge or transfer to long-term care or back to primary care) are not generally feasible. Yet factors that compromise the quality of care may be common at the interfaces of these phases of care. The use of unique patient identifiers throughout the health care system will allow improved appraisal of these interfaces.
Conclusions Provision of the best care possible is implicit in the fiduciary relationship between provider and patient. Providers want to do well for their patients, yet the care provided is sometimes below the stan166
CAN MED ASSOC J 1992; 147 (2)
dards set by their peers.""l Rectifying the shortcomings of the health care system as well as maintaining its strengths will require a re-examination of the way in which health care is monitored and managed. This will be especially important as the complexity of medical care continues to increase dramatically. Historically, health care systems have not been geared to monitor practice patterns and patient outcomes routinely and systematically or to respond to the results. In the evolving health care environment, providers and institutions will increasingly be challenged to evaluate the quality of care they deliver, to demonstrate an ability to improve the structures and processes of care and to show the benefits of care to patients' health and satisfaction.
References 1. Parks CL, Cashman S, Winickoff RN et al: Quality of acute episodic care in investor-owned ambulatory health centers.
Med Care 1991; 29: 72-86 2. Chassin M, Kosecoff J, Park RE et al: Does inappropriate use explain geographic variations in the use of health care services? JAMA 1987; 258: 2533-2537 3. Health Services Research Group: Quality of care: 1. What is quality and how can it be measured? Can Med Assoc J 1992; 146: 2153-2158 4. Ware JE Jr, Brook RH, Rogers WH et al: Comparison of health outcomes at a health maintenance organization with those of fee-for-service care. Lancet 1986; 1: 1017-1022 5. Hughes RG, Hunt SS, Luft HS: Effects of surgeon volume and hospital volume on quality of care in hospitals. Med Care 1987; 25: 489-503 6. Hughes RG, Garnick DW, Luft HS et al: Hospital volume and patient outcomes: the case of hip fracture patients. Med Care 1988; 26: 1057-1067 7. Dubois RW, Rogers WH, Moxley JH et al: Hospital inpatient mortality: Is it a predictor of quality? N Engl J Med 1987; 317: 1674-1680 8. Kahn KL, Rogers WH, Rubenstein LV et al: Measuring quality of care with explicit process criteria before and after implementation of the DRG-based prospective payment system. JAMA 1990; 264: 1969-1973 9. Rubenstein LV, Kahn KL, Reinisch EJ et al: Changes in quality of care for five diseases measured by implicit review, 1981 to 1986. Ibid: 1974-1979 10. Kahn KL, Keeler EB, Sherwood MJ et al: Comparing outcomes of care before and after implementation of the DRGbased prospective payment system. Ibid: 1984-1988 11. Brennan TA, Leape LL, Laird NM et al: Incidence of adverse events and negligence in hospitalized patients: results of the Harvard Medical Practice Study I. N Engl J Med 1991; 324: 370-376 12. Leape LL, Brennan TA, Laird N et al: The nature of adverse events in hospitalized patients: results of the Harvard Medical Practice Study II. Ibid: 377-384 13. Dyckman A: Joint Ventures among Health Care Providers in Florida: Impact on MRI Services and Costs, Center for Health Policy Studies, Columbia, Md, 1991: 3 14. Eisenberg JM, Williams SV: Cost containment and changing physicians' practice behavior: Can the fox learn to guard the chicken coop? JAMA 1981; 246: 2195-2201 15. Kroenke K, Hanley JF, Copley JB et al: Improving house staff ordering of three common laboratory tests. Reductions in test ordering need not result in underutilization. Med Care 1987; 25: 928-935 LE 15 JUILLET 1992
16. Tierney WM, Miller ME, McDonald CJ: The effect on test ordering of informing physicians of the charges for outpatient diagnostic tests. N Engl J Med 1990; 322: 1499-1504 17. Pop P, Winkens RAG: A diagnostic centre for general practitioners: results of individual feedback on diagnostic actions. JR Coll Gen Pract 1989; 39: 507-508 18. Dyck FJ, Murphy FA, Murphy JK et al: Effect of surveillance on the number of hysterectomies in the province of Saskatchewan. N Engi J Med 1977; 296: 1326-1328 19. Greenfield S, Kaplan SH, Ware JE Jr et al: Patients' participation in medical care: effects on blood sugar control and quality of life in diabetes. J Gen Intern Med 1988; 3: 448-457 20. Greenfield S, Kaplan S, Ware JE Jr: Expanding patient involvement in care: effects on patient outcomes. Ann Intern Med 1985; 102: 520-528
21. Vickery DM, Golaszewski TJ, Wright EC et al: The effect of self-care interventions on the use of medical service within a Medicare population. Med Care 1988; 26: 580-588 22. Deming WE: Out ofthe Crisis, MIT-CAES, Cambridge, Mass, 1986: 23-24 23. Berwick DM, Godfrey AB, Roessner J: Curing Health Care, Jossey-Bass, San Francisco, 1990: 144-158 24. Health Services Research Group: Standards, guidelines and clinical policies. Can Med Assoc J 1992; 146: 833-837 25. Park RE, Brook RH, KosecoffJ et al: Explaining variations in hospital death rates: randomness, severity of illness, quality of care. JAMA 1990; 264: 484-490 26. Siu AL, McGlynn EA, Morgenstern H et al: A fair approach to comparing quality of care. Health Aff (Millwood) 1991; 10: 62-75
Conferences
Sept. 2-4, 1992: International Conference on Self-Help/Mutual Aid Government Conference Centre, Ottawa Canadian Council on Social Development, 55 Parkdale Ave., PO Box 3505, Stn. C, Ottawa, ON KIY 4G1; (613) 728-1865, fax (613) 728-9387
continuedfrom page 160 Aug. 20-25, 1992: Canadian Society of Forensic Science Annual Conference - Truth Through Science and Integrity Citadel Inn, Halifax Fredricka Monti, executive secretary, Canadian Society of Forensic Science, 215-2660 Southvale Cres., Ottawa, ON KIB 4W5; (613) 731-2096 Aug. 30-Sept. 2, 1992: 9th International Congress on Child Abuse and Neglect Chicago, Ill. Congress headquarters, Moorevents, Inc., 1765-676 N St. Clair St., Chicago, IL 60611; (312) 951-9600, fax (312) 951-9854 Aug. 30-Sept. 3, 1992: European Federation of Immunological Societies John Humphrey Course on Tumour Immunology (a satellite conference to the 8th International Congress of Immunology organized by the Romanian Society for Immunology and with the sponsorship of the German Society of Immunology) Iasi, Romania Dr. Eugen Carasevici, Clinica de Oncologie, Laboratorul de Imunologie Tumorala, Spitalul Universitar "Sf. Spiridon," b-dul Independentei nr. 1, 6600 Iasi, Romania
Aug. 31-Sept. 2, 1992: IgA Nephropathy: the 25th Year International Symposium Nancy, France Meeting Secretariat, Laboratoire d'Immunologie, BP 184, Avenue de la Foret de Haye, 54500 Vandoeuvre-Ies-Nancy, France; telephone 011-33-8359-28-56, fax 011-33-83-44-60-22 Du 2 au 4 sept. 1992: Conference internationale sur l'entraide Centre des conferences du gouvernement, Ottawa Conseil canadien de developpement social, 55, av. Parkdale, CP 3505, succ. C, Ottawa, ON K1Y 4G1; (613) 728-1865, fax (613) 728-9387 JULY 15, 1992
Sept. 3-5, 1992: Physical Medicine Research Foundation 5th International Multidisciplinary Conference Oxford University, Oxford, England Physical Medicine Research Foundation, 510-207 W Hastings St., Vancouver, BC V6B 1H7 Sept. 6-10, 1992: Medinfo '92 - 7th World Congress on Medical Informatics Palexpo Congress Center, Geneva Official language: English Medinfo '92, Administrative Office, Symporg SA, 108, route de Frontenex, CH- 1208 Geneva, Switzerland; telephone 011-41-22-786-37-44, fax 011-41-22-78640-80
Sept. 8-11, 1992: Prevcon '92 - An International Conference on Hearing Impairment Prevention and the Strategies and Methods Designed for Developing Communities Johannesburg, South Africa Mrs. Henna Opperman, national director, South African National Council for the Deaf, Private Bag X04, Westhoven 2142, South Africa; telephone 011 -27-11 482-16 10, fax 011-27-11-726-5873 Sept. 11-12, 1992: Canadian Society for Aesthetic (Cosmetic) Plastic Surgery Annual Meeting Vancouver Mrs. Pat Hewitt, executive secretary, Canadian Society for Aesthetic (Cosmetic) Plastic Surgery, 4650 Highway 7, Woodbridge, ON L4L 1S7; (416) 831-7750 or 1-800-263-4429, fax (416) 831-7248
continued on page 188 CAN MED ASSOC J 1992; 147 (2)
167
RECHERCHE EN SERVICES DE- SOINS DE SANTE'
Qual'ity of care: 2. Quali'ty of care studi'es and
their consequences
Health Services Research Group A study assessing the process of care in a chain of ambulatory health care centres reported substandard care in 25% to 75% of cases of common ambulatory problems; the rate of provision of "at least adequate care" by individual physicians ranged from 44% to 90%.1 In another study the rate of unnecessary surgical procedures was shown to be about 1 in 6.2 Are these findings the rule rather than the exception? Do they represent global "truths" about the quality of care in the health care system? Are they generalizable to all settings? All practitioners would be deeply disturbed at such results in their own practice setting and would wish to rectify matters. How can the quality of care be assessed, and can it be improved? As we noted previously3 the determinants of practice patterns and outcomes as well as the consequences of changes to the health care system remain largely unexplored. Studying the existing quality of care is thus a necessary beginning.
Design and analysis of studies measuring quality Quality assessment studies range in size and sophistication - from the $100 million Rand Health Insurance Experiment4 (a controlled randomized trial of different payment plans) to retrospective chart audits of a single diagnosis within a single organization. Although a few controlled experiments
have been undertaken, the difficulty in manipulating health care delivery to individual patients means that observational (i.e., cohort, case-control and cross-sectional) or quasi-experimental studies are more usual. A common analytic strategy is to describe the variations in health care outcomes and in the structures or processes of care. Studies of quality of care that aim at effecting improvement or maintaining high levels of care may be directed most efficiently at areas of high risk, high volume, high cost and high rates of problems. Each health care organization must look within its own delivery system to identify the extent of potential problems. Studies are challenging to perform and interpret. Consistency across several independently performed investigations, however, is invaluable. For example, there are numerous articles reporting an association between a high volume of surgery and good outcome (when outcome measures are the length of stay and the rate of death in hospital).5 A further understanding of the mechanisms of the outcome-volume relation may be reached: Hughes and colleagues6 demonstrated that the mechanisms "practice makes perfect" (i.e., higher volume leads to better outcome) and "selective referral" (i.e., better outcome leads to higher volume) were operating in the case of hip replacement. Analyses such as these may be conducted with the use of hospital abstract databases, administrative
Members: Drs. Antoni S.H. Basinski (principal author), Department of Family and Community Medicine, Marsha M. Cohen, Department ofHealth Administration, C. David Naylor, Department of Medicine, Lorraine E. Ferris, Department of Behavioural Science, J. Ivan Williams, Department of Preventive Medicine and Biostatistics, and Hilary A. Llewellyn-Thomas, Faculty of Nursing, University of Toronto, Toronto, Ont.
The Health Services Research Group is part of the Clinical Epidemiology Unit, Sunnybrook Health Science Centre, Toronto, Ont. Reprint requests to: Health Services Research Group, Clinical Epidemiology Unit, Sunnybrook Health Science Centre, 2075 Bayview Ave., Toronto, ON M4N 3M5 JULY 15, 1992
CAN MED ASSOC J 1992; 147 (2)
163
hospital admissions adverse events occurred that were due to negligence; one quarter of these events led to death or permanent disability. Despite the lack of agreement about the level of the quality of care in individual cases it is clear that quality problems are not uncommon. Processes and outcomes of care provided by individual providers, institutions or within small areas must be assessed with caution: case-mix severity, variations in demographic and risk factors, social and economic circumstances and available resources will all affect the results. Even if these are taken into account there may be "unexplained variation," the result of unidentified and unmeasured factors and random fluctuations rather than of faulty practice. Assessment by implicit review can be unreliable. Explicit review may fail to capture the effects of important factors or unduly weight others. Shortterm studies may show changes in process or outcome measures, but in the long term these effects may be diminished and difficult to demonstrate. Observation of the health care process may lead to the Hawthorne effect - that is, knowing that they are being assessed, providers act differently than they would normally. This effect cannot be accounted for in uncontrolled observational studies lacking adequate comparison groups. As with all epidcare. Certain studies evaluate the impact of changes emiologic, observational studies, sampling and in the system of delivering care. In a recent study8 measurement biases and confounding variables may explicit process criteria and scales were applied to create problems in imputing causation. medical records of randomly selected patients admitted to hospital with congestive heart failure, acute Response to information myocardial infarction, cerebrovascular accident, pneumonia or hip fracture. For all these conditions a Once the level of quality has been identified the better process of care, whether measured explicitly next step is to respond to the information. There are by process criteria8 or by a structured, implicit two approaches. The "macroresponse" to quality judgement system,9 was associated with lower rates research may entail the modification of systems of of death 30 days after admission. In addition, payment or of reimbursement levels for procedures improvements in the process of care after the intro- or the promulgation of broad guidelines duction of a prospective payment system were asso- for aspects of medical care in an attempt to influciated with a 1% reduction in the rate of death 30 ence the overall system of health care. For examdays after admission.8 Yet the improvements were ple, a comparison between Florida and Baltimore of accompanied by an increase in the number of the use of magnetic resonance imaging services patients considered to be unstable or discharged too led to the suggestion that "a significant causative soon,9 and by 180 days after admission the 30-day factor for the higher Florida utilization and cost improvement in the rate of death was halved. '0 experience is the large number of imaging joint These results underscore the importance of measur- ventures in which physicians have financial incening a wide range of end points over a period long tives to refer patients and increase the number of enough to include other important consequences of referrals."'3 One response to these findings would be changes to the system. legislation to limit joint ownership of diagnostic Despite the observed improvements in care facilities by physicians who are not radiologists in these studies also revealed that 12% of patients were active practice. deemed to have received poor or very poor care.91'0 The "microresponse" involves a focus on specifThis is in accord with the findings of the Harvard ic local problems. The approach of continuous qualMedical Practice Study,"'12 in which a quarter of all ity improvement of process or outcome (e.g., by adverse events and half of the serious adverse events decreasing reporting time for radiologic results or were attributed to negligence. Overall, in 1% of increasing patient satisfaction with services) within
databases or registries, but they are limited in their ability to provide more detailed information on the quality of care. Data from medical records, however, may provide such necessary additional information. For instance, in a study of the abstracts from all 91 hospitals in a chain across the United States Dubois and associates7 found marked variations in rates of death from cerebrovascular accident, pneumonia and acute myocardial infarction. "Outlier" hospitals (i.e., those with high and low rates) were identified and studied further. A review of the medical records enabled adjustments to be made for the complexity and severity of illness. The adjusted death rate in the high-outlier hospitals was only 3% to 10% more and in the low-outlier hospitals only 10% to 15% less than that predicted from the severity of illness. Moreover, no significant differences in the quality of the process of care were found with the use of 125 explicit review criteria. An expert "implicit" review revealed that greater numbers of preventable deaths occurred in the high-outlier hospitals. However, without explicit criteria to guide them the experts tended to disagree as to which deaths were indeed preventable. Hence, a review of even the best current information may lead to an unacceptable lack of reliability in judgements of good and poor quality of
164
CAN MED ASSOC J 1992; 147 (2)
LE 15 JUILLET 1992
specific settings may pertain. Strategies to improve the quality or efficiency of care have evolved along a number of lines: education, peer review with feedback, administrative changes, participation, and penalties and rewards.'4 Many of these have been tested, albeit in teaching institutions over short periods, with hospitalized patients and physicians in training. Feedback may take many forms; for instance, an institution may attempt to reduce the rate of unnecessary tests. A simple intervention might provide feedback of the cumulative costs of patient services.'5 '6 More intensive feedback involving peer review of the necessity of services ordered is labourintensive and requires a sustained effort.'7 Although these forms of intervention can lead to decreases in rates of use with no sacrifice in apparent quality of care, it is not clear that sustained improvements in the delivery of care will result. For instance, although educational programs can decrease the rate of ordering of laboratory tests during the period of intervention the rate has been found to return to baseline levels after withdrawal of the intervention. 16 Macroresponses and microresponses may be combined. For instance, in Saskatchewan a review committee was formed to establish a list of indications for hysterectomy that were generally accepted as conforming to good practice.'8 Peer feedback to individual hospitals in which there had been a large number of unjustified hysterectomies was associated with decreases in the proportion of such operations from 24% to 8%. The decrease coincided with publicity about the excess rates and began before the review committee met. It was concluded that the excess hysterectomy rates "resulted from a breakdown of adequate review at the hospital level."'8 Alternatively, changes in the approach to patients may influence process and outcome. For example, clinical trials have tested interventions aimed at increasing patient involvement in care by reviewing with each patient his or her medical record and encouraging the patient to negotiate medical decisions with the physician.'9'20 These interventions led to improvements in outcomes, such as the control of diabetes'9 and reduced functional limitations.'9'20 Similarly, a randomized controlled trial of the effect of a self-care health education program on the use of ambulatory care demonstrated a decrease of 15% in the total number of medical visits and in direct costs and no negative impact on the quality of health.2'
necessary for clinical research studies to be successful and valid. Although not always sufficient to ensure success these conditions provide a foundation for inference from study outcomes. Factors that may promote or impede quality of care assessments and influence the ability to respond to their results have yet to be formalized. However, advocates of continuous quality improvement emphasize that a number of factors may influence success.22,23 The prescriptions for the formulation of quality initiatives are rooted in industrial management theories rather than in clinical science. Although unsupported by experimental experience they appear to be a reasonable starting point. Quality initiatives are more likely to succeed if health care managers - physicians and administrators - provide leadership and demonstrate a commitment to evaluate and, if necessary, change the existing system while providing the resources to do so. The goals and conduct of quality research and the response to it must be acceptable to providers, payers and patients. Providers are more likely to contribute toward changes in the system if they feel a sense of ownership and involvement in the quality improvement initiative. Providers furnished with information on their own performance are more likely to make changes that they see are necessary. Payers for health care services in Canada have traditionally observed the processes and outcomes of health care from a distance. With increased pressure on resources and increased competition for funding of programs, payers will demand a more active role in the management of health care. The demonstration of quality of care will become increasingly important in the choice between competing demands. Patients must feel that they are represented, their interests safeguarded and their views meaningfully taken into account. Education of consumers reinforces improved health care;20'2' in turn, informed consumers provide direction to the system through feedback and demand. Assessing and changing existing systems is often difficult. Many factors may consistently militate against sustained or substantial changes. For instance, payment systems have inherent biases. Although the fee-for-service method may encourage inappropriate overuse of services, prepaid reimbursement may foster inappropriate underuse. Resistance to evaluation and change may arise from many sources, including a fear of blame and adverse consequences. Moreover, policies and directives for Factors affecting initiatives to assess and change may be viewed as arbitrary rather than improve the quality of care necessary. Often the data cannot answer the questions Various prerequisites (e.g., avoidance of bias and the reliability of measurements) are considered posed, and the links between structure or process JULY 15, 1992
CAN MED ASSOC J 1992; 147 (2)
165
and outcomes are tenuous and inconsistent. The consequences of some processes (e.g., preventive care) may not be evident for a long time. Outcome rates are often low and may be subject to wide random variation. Variations among patients and among providers may have a much greater influence on the desired outcome than do the processes of care. Many social, economic and psychologic factors come into play, and these remain largely unaccounted for in assessments of quality. Difficulties in defining and obtaining valid and reliable measures of health care quality impede assessment. Interrater and intrarater reliability of implicit measurements is often low, and yet explicit standards or guidelines may be controversial and amount to little more than the formalization of guesswork.24
Future directions Further research and developments can be expected on a number of fronts. As we noted in an earlier paper3 the release of data on death rates in hospitals led to multiple attempts to explain the data. However, these analyses used different measures of outcome, case mix, severity, process quality and hospital factors.25 The focus of health services research is moving from exploratory and explanatory studies to applications, policy development and more generalizable research. Because the pressure to compare the quality of care across jurisdictions is increasing, reliable and valid measures, methods and comparisons are needed, as are standardization of the assessment and improvement of quality. Progress is being made on these fronts.26 Population-based analyses become more feasible when information is readily available and standardized. In these analyses the population "cared for" by a particular system of health care is tracked. Studies that track patients through different components of care (from primary care to specialist referral to hospital admission and then discharge or transfer to long-term care or back to primary care) are not generally feasible. Yet factors that compromise the quality of care may be common at the interfaces of these phases of care. The use of unique patient identifiers throughout the health care system will allow improved appraisal of these interfaces.
Conclusions Provision of the best care possible is implicit in the fiduciary relationship between provider and patient. Providers want to do well for their patients, yet the care provided is sometimes below the stan166
CAN MED ASSOC J 1992; 147 (2)
dards set by their peers.""l Rectifying the shortcomings of the health care system as well as maintaining its strengths will require a re-examination of the way in which health care is monitored and managed. This will be especially important as the complexity of medical care continues to increase dramatically. Historically, health care systems have not been geared to monitor practice patterns and patient outcomes routinely and systematically or to respond to the results. In the evolving health care environment, providers and institutions will increasingly be challenged to evaluate the quality of care they deliver, to demonstrate an ability to improve the structures and processes of care and to show the benefits of care to patients' health and satisfaction.
References 1. Parks CL, Cashman S, Winickoff RN et al: Quality of acute episodic care in investor-owned ambulatory health centers.
Med Care 1991; 29: 72-86 2. Chassin M, Kosecoff J, Park RE et al: Does inappropriate use explain geographic variations in the use of health care services? JAMA 1987; 258: 2533-2537 3. Health Services Research Group: Quality of care: 1. What is quality and how can it be measured? Can Med Assoc J 1992; 146: 2153-2158 4. Ware JE Jr, Brook RH, Rogers WH et al: Comparison of health outcomes at a health maintenance organization with those of fee-for-service care. Lancet 1986; 1: 1017-1022 5. Hughes RG, Hunt SS, Luft HS: Effects of surgeon volume and hospital volume on quality of care in hospitals. Med Care 1987; 25: 489-503 6. Hughes RG, Garnick DW, Luft HS et al: Hospital volume and patient outcomes: the case of hip fracture patients. Med Care 1988; 26: 1057-1067 7. Dubois RW, Rogers WH, Moxley JH et al: Hospital inpatient mortality: Is it a predictor of quality? N Engl J Med 1987; 317: 1674-1680 8. Kahn KL, Rogers WH, Rubenstein LV et al: Measuring quality of care with explicit process criteria before and after implementation of the DRG-based prospective payment system. JAMA 1990; 264: 1969-1973 9. Rubenstein LV, Kahn KL, Reinisch EJ et al: Changes in quality of care for five diseases measured by implicit review, 1981 to 1986. Ibid: 1974-1979 10. Kahn KL, Keeler EB, Sherwood MJ et al: Comparing outcomes of care before and after implementation of the DRGbased prospective payment system. Ibid: 1984-1988 11. Brennan TA, Leape LL, Laird NM et al: Incidence of adverse events and negligence in hospitalized patients: results of the Harvard Medical Practice Study I. N Engl J Med 1991; 324: 370-376 12. Leape LL, Brennan TA, Laird N et al: The nature of adverse events in hospitalized patients: results of the Harvard Medical Practice Study II. Ibid: 377-384 13. Dyckman A: Joint Ventures among Health Care Providers in Florida: Impact on MRI Services and Costs, Center for Health Policy Studies, Columbia, Md, 1991: 3 14. Eisenberg JM, Williams SV: Cost containment and changing physicians' practice behavior: Can the fox learn to guard the chicken coop? JAMA 1981; 246: 2195-2201 15. Kroenke K, Hanley JF, Copley JB et al: Improving house staff ordering of three common laboratory tests. Reductions in test ordering need not result in underutilization. Med Care 1987; 25: 928-935 LE 15 JUILLET 1992
16. Tierney WM, Miller ME, McDonald CJ: The effect on test ordering of informing physicians of the charges for outpatient diagnostic tests. N Engl J Med 1990; 322: 1499-1504 17. Pop P, Winkens RAG: A diagnostic centre for general practitioners: results of individual feedback on diagnostic actions. JR Coll Gen Pract 1989; 39: 507-508 18. Dyck FJ, Murphy FA, Murphy JK et al: Effect of surveillance on the number of hysterectomies in the province of Saskatchewan. N Engi J Med 1977; 296: 1326-1328 19. Greenfield S, Kaplan SH, Ware JE Jr et al: Patients' participation in medical care: effects on blood sugar control and quality of life in diabetes. J Gen Intern Med 1988; 3: 448-457 20. Greenfield S, Kaplan S, Ware JE Jr: Expanding patient involvement in care: effects on patient outcomes. Ann Intern Med 1985; 102: 520-528
21. Vickery DM, Golaszewski TJ, Wright EC et al: The effect of self-care interventions on the use of medical service within a Medicare population. Med Care 1988; 26: 580-588 22. Deming WE: Out ofthe Crisis, MIT-CAES, Cambridge, Mass, 1986: 23-24 23. Berwick DM, Godfrey AB, Roessner J: Curing Health Care, Jossey-Bass, San Francisco, 1990: 144-158 24. Health Services Research Group: Standards, guidelines and clinical policies. Can Med Assoc J 1992; 146: 833-837 25. Park RE, Brook RH, KosecoffJ et al: Explaining variations in hospital death rates: randomness, severity of illness, quality of care. JAMA 1990; 264: 484-490 26. Siu AL, McGlynn EA, Morgenstern H et al: A fair approach to comparing quality of care. Health Aff (Millwood) 1991; 10: 62-75
Conferences
Sept. 2-4, 1992: International Conference on Self-Help/Mutual Aid Government Conference Centre, Ottawa Canadian Council on Social Development, 55 Parkdale Ave., PO Box 3505, Stn. C, Ottawa, ON KIY 4G1; (613) 728-1865, fax (613) 728-9387
continuedfrom page 160 Aug. 20-25, 1992: Canadian Society of Forensic Science Annual Conference - Truth Through Science and Integrity Citadel Inn, Halifax Fredricka Monti, executive secretary, Canadian Society of Forensic Science, 215-2660 Southvale Cres., Ottawa, ON KIB 4W5; (613) 731-2096 Aug. 30-Sept. 2, 1992: 9th International Congress on Child Abuse and Neglect Chicago, Ill. Congress headquarters, Moorevents, Inc., 1765-676 N St. Clair St., Chicago, IL 60611; (312) 951-9600, fax (312) 951-9854 Aug. 30-Sept. 3, 1992: European Federation of Immunological Societies John Humphrey Course on Tumour Immunology (a satellite conference to the 8th International Congress of Immunology organized by the Romanian Society for Immunology and with the sponsorship of the German Society of Immunology) Iasi, Romania Dr. Eugen Carasevici, Clinica de Oncologie, Laboratorul de Imunologie Tumorala, Spitalul Universitar "Sf. Spiridon," b-dul Independentei nr. 1, 6600 Iasi, Romania
Aug. 31-Sept. 2, 1992: IgA Nephropathy: the 25th Year International Symposium Nancy, France Meeting Secretariat, Laboratoire d'Immunologie, BP 184, Avenue de la Foret de Haye, 54500 Vandoeuvre-Ies-Nancy, France; telephone 011-33-8359-28-56, fax 011-33-83-44-60-22 Du 2 au 4 sept. 1992: Conference internationale sur l'entraide Centre des conferences du gouvernement, Ottawa Conseil canadien de developpement social, 55, av. Parkdale, CP 3505, succ. C, Ottawa, ON K1Y 4G1; (613) 728-1865, fax (613) 728-9387 JULY 15, 1992
Sept. 3-5, 1992: Physical Medicine Research Foundation 5th International Multidisciplinary Conference Oxford University, Oxford, England Physical Medicine Research Foundation, 510-207 W Hastings St., Vancouver, BC V6B 1H7 Sept. 6-10, 1992: Medinfo '92 - 7th World Congress on Medical Informatics Palexpo Congress Center, Geneva Official language: English Medinfo '92, Administrative Office, Symporg SA, 108, route de Frontenex, CH- 1208 Geneva, Switzerland; telephone 011-41-22-786-37-44, fax 011-41-22-78640-80
Sept. 8-11, 1992: Prevcon '92 - An International Conference on Hearing Impairment Prevention and the Strategies and Methods Designed for Developing Communities Johannesburg, South Africa Mrs. Henna Opperman, national director, South African National Council for the Deaf, Private Bag X04, Westhoven 2142, South Africa; telephone 011 -27-11 482-16 10, fax 011-27-11-726-5873 Sept. 11-12, 1992: Canadian Society for Aesthetic (Cosmetic) Plastic Surgery Annual Meeting Vancouver Mrs. Pat Hewitt, executive secretary, Canadian Society for Aesthetic (Cosmetic) Plastic Surgery, 4650 Highway 7, Woodbridge, ON L4L 1S7; (416) 831-7750 or 1-800-263-4429, fax (416) 831-7248
continued on page 188 CAN MED ASSOC J 1992; 147 (2)
167
