EVALUATING MEDICAL TECHNOLOGY IN THE 1990's* GEORGE E. THIBAULT** BOSTON

The explosive growth in medical technology in the past two decades has created a tension for both clinicians and policy makers. On the one hand, advances in technology are credited with improving patient survival and quality of life. On the other hand, technology is held responsible for unacceptable cost escalation and for practices that have been questioned on clinical, economic, and ethical grounds. Traditionally, technology assessment has dealt principally with the issues of the safety and efficacy of devices and drugs. It has focused on a single device or drug in an idealized, controlled setting. The question has been does the technology do what it is supposed to at an acceptable risk, and the answer has been either "yes" or "no". Once the question was answered, the technology has generally not been reexamined. A broader and more relevant concept of technology assessment for the 1990's involves a study of the patterns of use of technology in dealing with clinical problems. Viewed in this way technology assessment becomes synonymous with the evaluation of medical practices. This expanded form of medical technology assessment deals with the issues of appropriateness of use, possible alternative strategies, and outcomes of care. It is clinically based and in the real world. It is not the evaluation of individual technologies, but rather of technologies in aggregate, and it compares alternative technologies and approaches. The answers to this form of technology assessment are not the binary "yes" or "no", but rather deal with "when", "how", "under what circumstances", "for what indication", "to achieve what end", and "at what cost". The clinical condition, rather than the technology itself, is the ceterpiece of this evaluation. We began evaluating practice patterns in the medical intensive care unit (MICU) in the late 1970's with the idea that this aggregate technology lent itself to this concept of medical technology assessment as medical practice evaluation (1). We chose the MICU as our laboratory for several reasons. We were intimately involved in the training of house staff and fellows, and it was clear that the MICU had a significant hold on their time and attention. We had explicit educational goals to try to improve * From the Department of Medicine, Brockton/West Roxbury Veterans Affairs Medical Center and Brigham and Women's Hospital, Harvard Medical School. ** Address for Reprints: George E. Thibault, M.D., Brockton/West Roxbury Veterans Affairs Medical Center, 1400 VFW Parkway, West Roxbury, MA 02132. 255

256

GEORGE E. THIBAULT

our ability to teach appropriate decision making in the MICU. We also felt that MICU decision making could be a model for medical decision making in general. It is an area in which there is a high degree of uncertainty and therefore a great deal of variation in practice patterns exist. Finally, we were concerned about our ability to justify new innovations and our ability to protect access, given the rising costs of medical intensive care. We felt that by learning more about what works and what doesn't work we could best achieve the goals of continued innovation and equity of access. We began by posing some very simple questions: Who needs MICU care? How long should patients stay in the MICU? Following MICU transfer, how long should the patient remain hospitalized? Which ancillary tests and treatments should be used in caring for the patient and in which sequence? We looked at a variety of intensive care outcome measures including resources consumed, MICU mortality, hospital mortality, long term mortality, functional status, and quality of life. Early in our studies, it became clear to us that we could not evaluate the intensive care unit by staying within the MICU walls. Since most of the patients treated in the MICU are being treated for acute exacerbations of chronic medical problems, we could only evaluate the appropriateness and efficacy of MICU care by taking a longer term view. Estimating the true costs of intensive care proved to be a particularly difficult task. We had to deal with the traditional distinction between charges and costs in the hospital billing structure and the degree to which cross-subsidization occurs when two patients receiving very different intensities of care are charged the same fixed day rate. We also had to deal with the issue of induced costs; that is, the degree to which the patient's presence in the MICU led to more tests, procedures, or drugs than if the same illness were treated outside of the MICU. In addition, we had to consider the degree to which iatrogenic complications related to interventions in the MICU added to costs. Finally, we had to deal with the complicated issues of other medical and non-medical costs to society that were either incurred and avoided as a consequence of the MICU interventions. We prospectively collected data on all patients admitted to an 18 bed MICU at the Massachusetts General Hospital. All patients admitted over a five year period were entered into a follow up study (2). During that period there were 7,114 admissions involving 5,424 patients. The age and sex distributions of the patients are shown in Table 1, and the distributions of primary diagnoses are shown in Table 2. The outcomes of those patients with nearly a one year follow-up period are shown in Figure 1. Both in-hospital and follow-up mortality rates varied strikingly with patient age and with the principle reason for intensive care (Tables

EVALUATING MEDICAL TECHNOLOGY

257

TABLE 1 Age and Sex Distribution of MICU Population Age

Total

Male (%)

Female (%)

0-19 20-29 30-39 40-49 50-59 60-69 70-79 80-89 90+ Totals

52 354 375 687 1298 1705 1443 622 104 6680

33 (63.5) 187 (52.8) 262 (69.9) 487 (70.9) 854 (65.8) 982 (57.6) 751 (52.0) 286 (43.2) 44 (42.3) 3387

19 (36.5) 167 (47.2) 113 (30.1) 200 (29.1) 444 (34.2) 722 (42.2) 692 (48.0) 376 (56.8) 60 (57.7) 2793

3-6). This confirmed our hypothesis that one could not speak in aggregate of MICU decision-making and outcomes. One-third of the hospital mortality occurred after patients were transferred from the MICU. This important observation raised other questions about MICU decisionmaking. Are some patients prematurely discharged from the MICU and would they benefit from more intensive care? (3) Mortality in the year TABLE 2 ICU Primary Diagnoses Diagnosis

Number

% of MICU Admissions

Myocardial infarction 1279 19.1 887 13.8 Coronary insufficiency 651 9.7 Arrhythmias Acute respiratory diseasea 562 8.4 506 7.6 Congestive heart failure 501 7.5 Chest painb 350 5.2 Drug overdose 310 Gastrointestinal bleeding 4.6 Acute neurologic diseasec 225 3.4 141 2.1 S/P cardiac arrestd 136 2.0 Syncope 133 2.0 Sepsis Diabetic syndromes 120 1.8 Renal failure 90 1.3 Adverse effect of drugs 84 1.3 aIncludes pneumonia, asthma, decompensated chronic obstructive lung disease, and acute respiratory failure. b Patients admitted with suspected ischemic chest pain who were discharged from the unit without the diagnosis of myocardial infarction, coronary insufficiency, or a specific noncardiac cause of pain. c Includes strokes, seizures, and coma of unknown cause. d Out-of-hospital cardiac arrest without another primary diagnosis.

GEORGE E. THIBAULT

258

MICU Outcome: Follow-Up 5424 Patients Admitted to ICU 1 730 Died in Hospital 4694 Eligible for Follow-Up 570 No Consent or No Follow-Up l 4124 Follow-Up Status Known 530 Dead at 1 st Follow-Up (Mean 273 Days)

3594 Alive at 1 st Follow-Up (Mean 241 Days)

FIG. 1. MICU outcome: Follow-UP TABLE 3 Outcome of Medical Intensive Care Mortality by Diagnosis MICU Mortality Hospital Mortality Number Diagnosis 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15.

Myocardial infarction Coronary insufficiency Arrhythmias Respiratory disease Congestive heart failure Chest pain Drug overdose Gastrointestinal bleeding Neurologic disease S/P cardiac arrest Syncope Sepsis Diabetic syndromes Renal failure Adverse effects of drugs

15.4 2.1 6.3 24.4 10.5 0.2 0.6 20.0 19.6 70.9 1.5 47.4 2.5 23.3 1.2

10.0 0.1 3.2 16.4 5.3 0.0 0.6 8.4 11.1 54.0 0.0 32.3 0.8 16.7 0.0

1279 887 651 562 506 501 350 310 225 141 136 133 120 90 84

TABLE 4 Outcome of Medical Intensive Care Mortality by Decade MICU

Hospital

Age

Number

Mortality

Mortality

0-19 20-29 30-39 40-49 50-59 60-69 70-79 80-89 90+

52 354 375 687 1298 1705 1443 662 104

5.8 4.4 5.9 3.6 6.3 8.4 9.7 13.0 12.5

7.7 4.8 8.5 5.8 9.6 14.0 16.8 21.8 26.9

259

EVALUATING MEDICAL TECHNOLOGY

TABLE 5 Outcomes of MICU: Follow-Up Mortality by Diagnosis Diagnosis

Myocardial infarction Coronary insufficiency Arrhythmias Chest pain Congestive heart failure Respiratory disease Drug overdose Gastrointestinal bleeding Neurologic disease Syncope Diabetic syndrome Adverse effects of drugs Sepsis Renal failure S/P cardiac arrest

Number

828 650 441 337 305 286 241 181 92 91 80 62 54 54 31

Post-hospital

Hospital

Mortality

Mortality

12.2 6.2 12.7 4.7

15.4 2.1 6.3 0.2 10.5 24.4 0.6 20.0 19.6 1.5 2.5 1.2 47.4 23.3 70.9

26.6 20.6 5.8 17.1 20.6 12.1 10.0 9.7 18.5 20.4 12.9

following MICU care was particularly high in the elderly and in patients with congestive heart failure, chronic obstructive pulmonary disease, renal failure, and stroke. In these patient groups, physicians who only see patients in the MICU may overestimate the impact of their interventions. As we analyzed this complex and heterogenous patient population, we found three simple decision models were useful as paradigms for the types of patients for whom physicians need to make a decision about whether to admit to an intensive care unit. Case 1 (Figure 2) represents a critically ill patient in whom the probability of survival is low without intensive care. But there is known efficacious therapy for the patient's critical illness, so the probability of survival is high if intensive care is TABLE 6 MICU Outcomes: Follow- Up Mortality by Age Posthospital Age AgeNumber Mortality Number

0-19 20-29 30-39 40-49 50-59 60-69 70-79 80-89 90+

38 236 237 474 842 1015 831 389 62

5.3 4.2 7.6 7.0 7.8 12.7 19.5 23.6 29.0

GEORGE E. THIBAULT

260 Case 1.

Good Prognosis Wi th MICU Intervention

Critically Ill Patient

P(Survival NoMICU)

LowJ

P(Surviva1 MICU)

Hi qh

FIG. 2. Case 1. Good prognosis with MICU intervention

employed. An example of this paradigm would be a patient with acute respiratory failure from a drug overdose, where the basic pathophysiologic abnormality is known to be reversible and the patient will survive with ventilatory support. The same might be said of a patient with acute reversible renal failure requiring short term dialysis or a patient with complete heart block or a life threatening arrythmia responsive to medical intervention. If we can improve our understanding of pathophysiologic mechanisms and develop specific, effective interventions for life threatening illnesses, we will have no problems in justifying intensive care in these patients. Case 2 (Figure 3) represents a critically ill patient who also has a low probability of survival without intensive care. But this patient has a low probability of survival with intensive care, because the patient's underlying pathophysiologic abnormalities cannot be entirely corrected. The prototypes for this paradigm may be many patients with cardiogenic shock, end stage cirrhosis, or multi-organ system failure. These patients present a much more difficult decision-making dilemma. One needs to estimate how great a difference in these two low probabilities there needs to be to justify intensive care, and at what point one would reconsider that decision. In aggregate in our MICU these represent a small percentage of the patients but they consume a disproportionate amount of resources (4). Case 3 (Figure 4) represents a patient who is not critically ill at the time of presentation but has the potential for critical illness. This

EVALUATING MEDICAL TECHNOLOGY Case 2.

261

Poor Progno iis Despite MICU Intervention

Critically III Patient

P(Survival P(Survival

NOMICU) MICU)

Low Lcm

FIG. 3. Case 2. Poor prognosis despite MICU intervention

Case 3.

Good Prognosis With or WI\thout MICU

InterventIon

Patient at Risk of Critical Illness

P(SurvivallNo MICU) High P(SurvivalI MICU) FIG. 4. Case 3. Good prognosis with or without MICU intervention

High

262

GEORGE E. THIBAULT

patient's probability of survival is high with or without intensive care. Whether this patient will benefit from intensive care or not depends upon the product of the probability of having a complication and the probability that the outcome of that complication would be beneficially altered by the patient's presence in the MICU. We found this paradigm to be particularly common in the MICU, comprising 50 to 75% of the patients. (1) These patients may be admitted for prophylactic interventions to prevent complications; they may be admitted for monitoring in order to identify complications as soon as they occur; or they may be admitted purely for diagnostic purposes to "rule out" a diagnosis before it is deemed safe to care for them in a less intensively monitored area. Since these patients were so prevalent in the MICU, in aggregate they consume the majority of the resources. We decided that the third paradigm was a particularly fruitful area for the examination of medical decision-making, because the number of patients was large and more efficient decision-making could possibly lead to a saving of resources without having adverse effects on patient outcome. We, therefore, studied a number of patient groups who were representative of this paradigm. These include patients with chest pain and suspected myocardial infarction (5), uncomplicated pulmonary edema (6), syncope (7), atrial arrhythmias (8), and tricyclic drug overdose (9). In each of these instances we were able to identify clinical and laboratory variables that were present at the time of or soon after MICU admission that would enable the clinician to stratify these patient groups into those who were at high or low risk for the feared complications. Using clinical data, we could then begin to triage patients either prior to MICU admission or shortly after MICU admission based on their probability of having life threatening complications that would benefit from MICU care. An example of this risk stratification is shown in Figure 5 for a group of patients with uncomplicated chest pain admitted to the MICU with the suspicion of myocardial infarction. Based on simple clinical variables (EKG, CK, and clinical course) over the first 24 hours of MICU care, these patients were able to be stratified into high, intermediate, and low risk groups. The low risk group could be triaged from the MICU with less than half the previous length of stay with only a small increased risk. This is but one example of how this type of medical technology assessment can lead to a more efficient use of technology. A recently initiated and ongoing research project looks at another set of practice patterns of aggregate technology use. This is a study being led by Dr. Barbara McNeil, Chairman, Department of Health Care Policy at Harvard Medical School, under one of the first Patient Outcomes Research Team grants from the Agency for Health Care Policy & Research. This study involves evaluating regional variations in practice

263

EVALUATING MEDICAL TECHNOLOGY

STUDY POPULATION (360) ECG ZOINAG nSTIC (292)

Cm

50"dj SONY

F iLOW

CPK

CPK

CPI

SOmU

5OmU (9)

5Orn0U

(tO0)

(185)

No mato( complication in fist 24 hours (168)

ECG POSSIBLE or DEFINITE (68)

Malor

I death

conplication in first 24 hours (17)

during

I death

during

forst 24 hours

tiEst

(108)

7DSIT|ERNEDIAE A

(59)

SK

24 hours (58)

M | RISK7]

FIG. 5. Stratification possible at the end of the first 24 hours in the MICU/CCU on basis of electrocardiographic changes, maximum serum creatine phosphokinase (CPK) determination, and major complications.

patterns in Medicare patients in the three months following an acute myocardial infarction. A parallel study has recently been funded by the Department of Veterans Affairs, and this will be carried out at the West Roxbury VA Medical Center. We will be examining regional variations in practice patterns following acute MI in patients hospitalized in the 170 Department of Veterans Affairs Hospitals. Our research team at West Roxbury will be collaborating Dr. McNeil's research team so that we can compare findings in the VA system with those in the private sector. Preliminary findings from the Medicare database (not yet published) indicate that there are, in fact, striking regional variations in the use of invasive interventions in the 90 day period following acute MI in patients over 65. There are over 200,000 acute MI yearly among Medicare beneficiaries. The 90 day mortality in these patients is 26%, and the one year mortality is 40%. Within 90 days of the index MI, 23% of the Medicare patients underwent cardiac catheterization, 8% underwent coronary bypass surgery, and 5% underwent percutaneous coronary angioplasty. The rates of these procedures declined with age. Of greatest interest was the regional variation in the use of these technologies. In a state by state analysis, the rates of cardiac catheterization ranged from

264

GEORGE E. THIBAULT

a low of 13.4% to a high of 36.6%, and the coronary bypass surgery rates ranged from a low of 4.8% to a high of 11.9%. There were no apparent differences in demographic or patient characteristics that would explain these striking threefold differences, nor were there corresponding differences in 30 day or one year mortality rates. These provocative findings indicate that we must explore the reasons for these regional variations and look at far more detailed patient outcome measures (particularly functional status, symptom level, and patient satisfaction). We do not yet know whether these differences can be explained on clinical grounds, on access grounds, or on other structural or systemic bases. Nor do we yet know whether these figures represent underutilization in some areas or overutilization in others. It is these clinically relevant answers that must be sought in our technology assessment efforts in the 1990's if we are to both improve the quality of care and control medical costs. A number of organizations have begun to participate in this process with this new concept of technology assessment. For several years the Institute of Medicine sponsored a Council of Healthcare Technology that was funded by a joint effort of the National Center for Health Services Research and the private sector. One of the activities of this council was a pilot project for the establishment of national priorities for the assessment of medical technologies (10). The goal of this effort was to establish a national approach for assessing medical technology by setting out explicit and well accepted criteria that are applicable at a national level. It was also hoped that it could establish a conceptual framework that would accommodate both clinical conditions and medical technologies as assessment priorities. Finally, it hoped to establish an accountable process for priority setting that involves a broad range of assessment interests. Explicit criteria were arrived at by a panel of experts. The primary criteria were that the assessment should: 1. 2. 3. 4.

Have a high probability of improving individual patient outcome. Affect a large patient population. Have a potential for reducing unit or aggregate cost. Have the potential of reducing unexplained variations in medical practice. The secondary criteria were that the assessment might:

1. 2. 3. 4. 5.

Address social and ethical implications. Advance medical knowledge. Affect policy decisions. Enhance the national capacity for assessment. Be readily conducted.

EVALUATING MEDICAL TECHNOLOGY

265

Using these criteria and reviewing all the data available from both federal and private data sources, a group of experts arrived at a list of technology assessment priorities by a modified Delphi method. These priorities included fourteen clinical conditions and six individual or aggregate technologies and are shown in Figure 6. It was hoped that these priorities would assist federal research programs, private foundations, and other major funding sources in selecting assessments of national importance and that it would encourage assessment organizations to undertake evaluations in areas of national need consistent with their respective program missions and capabilities. It was hoped further that this pilot project would lead to an ongoing process in which all interested parties would participate to have a continuous revision and updating of these priorities for "new technology assessment". A second phase of this project is currently underway at the Institute of Medicine. A number of professional societies have now become active in this process of medical practice evaluation and technology assessment which is now evolving into the newly developing area of guideline formation. The American College of Physicians has been a leader in this effort through its Clinical Efficacy Assessment Program. This project which has been in existence for over ten years has now produced over 100 assessments which have been models for clinically based technology assessment and practice evaluation. This leadership role of the professional organizations is going to be essential if the field is going to move forward. A final example of current technology assessment is a recently comTWENTY ASSESSMENT PRIORITIES Clinical Conditions: Breast cancer Cataracts Chronic obstructive pulmonary

disease Coronary artery disease Gallbladder disease Gastrointestinal bleeding Human immunodeficiency virus infection Joint disease & injury Low back pain

Technologies: Diagnostic imaging technologies Diagnostic laboratory testing Erythropoietin Implantable devices Intensive care units Organ transplantation & replacement

Osteoporosis Pregnancy Prostatism Psychiatric disorders Substance abuse FIG. 6. Twenty assessment priorities.

266

GEORGE E. THIBAULT

pleted Institute of Medicine evaluation of the Artificial Heart Program (11), commissioned by the National Heart Lung Blood Institute. This year-long effort was, in part, "traditional" technology assessment, focusing on the engineering, materials, power sources, and other crucial technical issues that have been potential barriers to the development of this technology. This assessment, however, went beyond traditional technology assessment to look at this technology in the context of other treatments that are available for end stage heart failure, and to look ahead to the possible implications of its introduction into clinical use. It called for an effort to prospectively develop clinical guidelines for the appropriate use of all circulatory support systems, and for a specific plan to monitor the use of these systems after they are introduced. This would involve commissioning technology assessments to evaluate the cost effectiveness of the devices and to determine patient-specific outcome measures. It was recommended that a registry be established to monitor long term outcomes, and that guidelines be developed for hospitals, physicians, and third party payors to assure uniformity in professional standards, clinical criteria, and coverage decisions. It was hoped that this approach to technology assessment would be a model for the prospective evaluation of other new technologies. If technology assessment is to aid in our efforts to improve the quality of medical care and control costs in the 1990's, we are going to have to approach it with a broader definition. This means a broad definition not only of medical technology, but of what the assessment process involves. We are going to need to use many different methodologies in this pursuit, including randomized clinical trials, epidemiologic studies, registries, administrative data bases, meta-analysis, group judgement, decision analysis, cost effectiveness analysis, and mathematical modeling. We are going to have to have many participants in the process, including drug and device makers, payors, employers, healthcare organizations, government, healthcare researchers, physicians, and patients. While we may think that what we are embarking on is a new endeavor, we should be reminded that there have been many before us who saw the need for this. One of these was E.A. Codman, a surgeon at the Massachusetts General Hospital in the first part of this century. In 1916 he wrote a seminal

book entitled, A Study in Hospital Efficiency in which he foresaw the need for this new technology assessment. "That the Trustees of Hospitals should see to it that an effort is made to follow up each patient they treat, long enough to determine whether the treatment given has permanently relieved the condition or symptoms complained of. That they should give the members of the Staff credit for taking the responsibility of successful treatment and promote them accordingly.

EVALUATING MEDICAL TECHNOLOGY

267

Likewise they should see that all cases in which the treatment is found to have been unsuccessful or unsatisfactory are carefully analyzed, in order to fix the responsibility for failure on: 1. The physician or surgeon responsible for the treatment. 2. The organization carrying out the detail of the treatment. 3. The disease or condition of the patient. 4. The personal or social conditions preventing the cooperation of the patient. This will give a definite basis on which to make effort at improvement." We all owe a debt to Dr. Codman, and he should be an inspiration to us as we move ahead in this very important and absolutely essential activity of technology assessment and medical practice evaluation in the '90's.

ACKNOWLEDGMENTS The work summarized in this presentation involved the collaboration of many people. The author wishes to particularly acknowledge Dr. Albert Mulley's central role as a collaborator for all the MICU studies done at the Massachusetts General Hospital. The author also wishes to acknowledge the invaluable assistance of Ms. Dorothy Baldwin in the preparation of this manuscript. REFERENCES 1. Thibault GE, Mulley AG, Barnett GO, et al. Medical intensive care: indications, interventions, and outcomes. N Eng J Med 1980; 302: 938. 2. Thibault GE. The medical intensive care unit: a five-year perspective. NIH Consensus Development Conference on Critical Care Medicine, Bethesda, MD. In: Parillo JE, Ayres SM, eds. Major Issues in Critical Care Medicine. Baltimore: Williams & Wilkins; 1984: 2. 3. Singer DE, Mulley AG, Thibault GE, et al. Unexpected readmissions to the coronary care unit during recovery from acute myocardial infarction. N Eng J Med 1981; 304: 625. 4. Detsky AS, Stricker SC, Mulley AG, et al. Prognosis, survival and the expenditure of hospital resources for patients in an intensive care unit. N Eng J Med 1981; 305: 667. 5. Mulley AG, Thibault GE, Hughes RA, et al. The course of patients with suspected myocardial infarction: the identification of low risk patients for early transfer from intensive care. N Eng J Med 1980; 302: 943. 6. Katz MH, Nicholson BW, Singer DE, et al. The triage decision in pulmonary edema. J Gen Intern Med 1988; 3: 6: 533. 7. Silverstein MD, Singer DE, Mulley AG, et al. Patients with syncope admitted to medical intensive care units. JAm Med Assoc 1982; 248: 1185. 8. Herrmann HC, Thibault GE. The clinical course of patients with atrial fibrillation and flutter admitted to medical intensive care units. J Intensive Care Med 1989; 4: (3): 112. 9. Stern TA, O'Gara PT, Mulley AG, et al. Complications following overdose with tricyclic antidepressants. Critical Care Medicine 1985; 13: 8: 672.

268

GEORGE E. THIBAULT

10. Institute of Medicine. National priorities for the assessment of clinical conditions and medical technologies: Report of a pilot study. Washington, DC: National Academy Press; 1990. 11. Institute of Medicine. The artificial heart-prototypes, policies, and patients. Washington, DC: National Academy Press; 1991.

Evaluating medical technology in the 1990's.

EVALUATING MEDICAL TECHNOLOGY IN THE 1990's* GEORGE E. THIBAULT** BOSTON The explosive growth in medical technology in the past two decades has creat...
1MB Sizes 0 Downloads 0 Views