Changing Practice Patterns in the Management of Primary Breast Cancer: Consensus Development Program Jacqueline Kosecoff, David E. Kanouse, and Robert H. Brook, M.D. In the last decade, new knowkdge has emerged concerning the efficacy of treatment for breast cancer. For that reason, the National Institutes of Health devoted a consensus conference to this topic. To determine whether the consensus conference had influenced practice patterns, and to evaluate the level of quality of care given to women with breast cancer, the medical records of 573 patients treated in ten hospitals throughout the state of Washington were abstracted and analyzed. Results showed no changes with respect to the consensus conference's recommendations for use of a total mastectomy with axillary dissection or the use of a two-step procedure in which the biopsy is performedfirst and therapeutic options are discussed before a definitive surgery is undertiken. Analyses of quality of care issues not addressed by the consensus conference revealed that 4 percent of the sampk were explicitly staged preoperatively and 29 percent postoperatively and that littk changed over time in the use of sentinel laboratory tests. These results also show that consensus recommendations will not necessarily change physicians'behavior even where change is possibkl, This research was supported in part by contract NO1-OD-2-2128 from the Office of Medical Applications of Research, National Institutes of Health. The views expressed in this article are the authors' own and are not necessarily shared by The RAND Corporation, the University of California, Los Angeles, or the National Institutes of Health. Address correspondence and requests for reprints to David E. Kanouse, Ph.D., Head, Behavioral Sciences Department, The RAND Corporation, 1700 Main Street, P.O. Box 2138, Santa Monica, CA 90406-2138. Jacqueline Kosecoff, Ph.D. is Executive Vice-President of Value Health Sciences, and Robert H. Brook, M.D., Sc.D is Deputy Health Program Director, The RAND Corporation, and Chief of the Division of Geriatrics and Professor in the Departments of Medicine and of Public Health, University of California at Los Angeles.
810
HSR: Health Services Research 25:5 (December 1990)
and that quality of care in diagnosis and treatment of breast cancer still needs to be addressed.
Breast cancer is a serious disease that in 1989 affected an estimated 142,900 newly diagnosed women and caused 43,300 deaths (The World Almanac and Book of Facts 1990 1989). In the last decade, new knowledge has been gained concerning the efficacy of its treatment. For that reason, in June 1979, the National Institutes of Health (NIH) held a consensus conference on the treatment of primary breast cancer (National Institutes of Health 1979). Among the major recommendations resulting from the conference were that (1) a total mastectomy with axillary dissection should be the treatment standard for women with Stage I and early Stage II primary breast cancer, and (2) a twostep procedure should be followed in which a diagnostic biopsy is studied by permanent histologic sections and therapeutic options are discussed before definitive surgery is performed. The NIH consensus conferences are designed to facilitate the appropriate and timely application of biomedical research findings in medical practice. They seek to provide practicing physicians with timely and useful information, to facilitate informed decision making by physicians and patients regarding complex medical therapies and to foster, where they are needed, appropriate changes in medical practices that will improve patients' health care outcomes. None of these aims is easily achieved, especially that of changing medical practices. Nevertheless, success in achieving any of these aims depends on the relevance of conference recommendations to clinical practice. Thus, we sought to determine, as part of out evaluation of the NIH Consensus Development Program, whether the conferences were addressing physician practices where change was needed and, if so, whether there were changes in physician behavior that might be attributable to the program. We abstracted hospital medical records to determine, first, whether these two consensus conference recommendations had influenced physicians' surgical approaches to primary breast cancer. At the same time, and as the second major purpose of the study, we collected extensive data about the services provided to women with breast cancer and the quality of those services. This article draws on those data to describe how primary breast cancer is managed during the woman's initial hospitalization for surgical therapy. Although the data collected
Changing Practice Patterns
811
represent the early part of the 1980s, the issues they raise are probably still relevant today.
METHODS STUDY DESIGN
We identified all nonspecialty, nonfederal acute care hospitals in the state of Washington with more than 150 beds and stratified them by teaching status. We then randomly selected ten hospitals (likelihood of being accepted proportional to number in strata) such that one institution was a major teaching hospital, two had close ties with teaching hospitals, one was a nonaffiliated hospital with a residency program, and the remaining six were community hospitals without university affiliation. Of the ten hospitals approached to participate in the study, two refused and were replaced with other hospitals having similar characteristics. We studied the medical records from two one-year periods before the conference Uuly 1977 through June 1978 and July 1978 through June 1979) and one period after the conference (July 1980 through June 1981). The after period began 12 months after the conference to allow for dissemination of the conference results. The inclusion of a second preconference time period allowed for estimates of pre- and post-consensus conference differences in the rate of change in medical practice as well as the absolute amount of change. Hospital logs or hospital abstracting services' reports were used to identify patients who had both an ICD-9-CM diagnosis of breast cancer (codes 174.0-174.9) and a procedure of mastectomy (codes 85.20-85.23, 85.41-85.48, 85.99). We employed a two-stage sampling procedure, first sampling physicians within hospitals, then patients within physicians in such a way that within a given hospital, each patient had an equal chance of being selected, and so that the sampled physicians would be representative of the physicians in the state who were providing care to breast cancer patients. DATA COLLECTION
The basic data source was a medical-record review of patients hospitalized during the relevant time periods. A medical record abstraction form targeted to the first surgical treatment of primary breast cancer was specifically designed and pilot tested for the study. The instrument consisted of items measuring (1) patient demographics; (b) stage of
812
HSR: Health Services Research 25:5 (December 1990)
breast cancer; (c) aspects of care addressed in the NIH consensus conference, such as type of surgical therapy; and (d) other aspects of the process of care that could be used to describe or evaluate the care provided to the patient during hospitalization. The abstraction form took about 30 minutes to complete. Examples of specific information collected for each patient included: affected breast, preoperative results of alkaline phosphatase and serum calcium tests, whole-body scans, mammography, chest x-rays, pre- and postoperative stage, preoperative examination of both breasts, type of surgery, length of stay for biopsy and definitive surgery, pathology results, and performance of estrogen receptor assays. Twenty-eight persons with a medical record abstraction background were trained to collect the data. All had prior medical record review experience, passed an initial test of their abstraction skills, were given four days of intensive training, and successfully completed a further test at the end of training. Data collectors were provided with a list of records to pull and abstract and a list of backup records to pull if cases in the initial list were unavailable or met the study's exdusion criteria. Work was supervised by a chief data collector who regularly visited each hospital and reabstracted records to maintain quality control. Completed abstraction forms were reviewed by both a physician and nonphysician, who assessed internal consistency and whether coding decisions were consistent with the descriptive clinical data copied from medical records to justify decisions. Discrepancies that could not be resolved during the review process were returned for reabstraction. To assure the confidentiality of information, we assigned coded identifiers to patients, physicians, and hospitals. Once the data collection process had been completed, all files linking these identifiers to patients, physicians, or institutions were destroyed. In all, 573 charts were selected to be abstracted. To achieve this level, 84 records that failed the exclusion criteria were systematically replaced with substitute records. Reasons for exclusion included: (1) missing records (N = 8); (2) duplicate records (N = 2); (3) wrong time periods (N = 2); or (4) meeting one or more of the following exdusion criteria: the admission was not the first surgical treatment (mastectomy) for a primary unilateral breast cancer, the patient was a nursing home resident, or the patient had obvious metastatic disease at the time of admission (N = 72). Five records were lost during the keypunch process. Thus, the final sample was 568 records (101 in time period 1, 219 in time 2, and 248 in time 3). To identify women with Stage I and early Stage II breast cancer, we drew upon information abstracted from the preoperative physical
Changing Practice Pattns
813
examination and history recorded in the medical record. To interpret this information, we used the Tumor Node Metastatic (TNM) system, which uses data about the suspected size of the tumor, whether lymph nodes in the axilla are thought to contain cancer, and whether the tumor has spread beyond the breast to distant parts of the body (Stockdale 1986). Because the NIH consensus conference primarily addressed surgical approaches to the treatment of Stage I and early Stage II breast cancer, we took pains to distinguish women in these stages from those with more advanced disease. Specifically, a woman was classified as having Stage I disease if and only if six conditions were met: (1) the tumor was not palpable or was less than or equal to 2 centimeters in size; (2) the tumor was described as not fixed or fixed only to muscle or pectoral fascia; (3) no skin changes over the breast (skin edema, infiltration, or peau d'orange) were noted; (4) axillary nodes were not thought to contain tumor; (5) clavicular nodes were not palpable; and (6) there was no evidence of metastatic disease beyond the regional lymph nodes. We classified a woman as having early Stage II breast cancer if six criteria were met: (1) the tumor was greater than 2 but less than 5 centimeters in size; (2) the tumor was described as not fixed; (3) no skin changes (skin edema, peu d'orange, or infiltration) were noted; (4) axillary nodes were not thought to contain tumor; (5) clavicular nodes were not palpable; and (6) no evidence of metastatic disease presented beyond the regional lymph nodes. In addition, women with a tumor that was less than or equal to 2 centimeters, who met criteria 2, 3, 5, and 6 above, but who had suspicious axillary nodes that still were freely movable (not fixed) were classified as having early Stage II disease. We also excluded from Stage I or early Stage II anyone who had evidence of advanced disease based on results of laboratory tests or other data known prior to the definitive surgery. Accordingly, a woman who met any of the following criteria was excluded: (1) a chest x-ray that was suspicious of cancer; (2) a mammogram or xeromammogram that showed evidence of cancer in the presumably unaffected breast; (3) a preoperative statement by the physician that assigned the patient to Stage III or IV; (4) a whole-body scan that was suspicious of cancer; or (5) both breasts described as affected by cancer. It should be noted that in defining stages, we interpreted the absence of certain data in the medical record as evidence that a significant event had not occurred. Thus, no mention whatsoever of clavicular node involvement was interpreted as negative clavicular node involvement. Further details about the study methods have been pub-
814
HSR: Health Services Research 25:5 (Decernber 1990)
lished elsewhere (Kanouse et al. 1989; Kosecoff, Kanouse, Rogers, et al. 1987). ANALYSIS
Changes over time were tested with repeated-measures analysis of covariance in which the covariate was the patient's age at the time of definitive cancer surgery. Two orthogonal contrasts were used to test different patterns of change. The first tested whether there was a linear trend from time 1 to time 3; the second (quadratic) contrast tested whether the change from time 1 to time 2 (two contiguous years immediately prior to the conference) was different from the change from time 2 to time 3 (one year before the conference to the 12-24 month period after the conference). The contrast of greatest interest from the standpoint of testing possible consensus conference effects is the second (quadratic) contrast, which captures any acceleration in the rate of change from the preconference to postconference periods. In addition to these two orthogonal contrasts, we also report results for a third contrast, testing whether significant change occurred during the preconference period (time 1 to time 2). The statistical significance of other study findings was examined with chi-square analyses.
RESULTS RELIABILITY
Fourteen breast cancer records were randomly reabstracted by a different data collector. Kappa was used to assess reliability for ten critical variables that were used to define compliance with a recommendation. Across the 14 records, the mean Kappa was 1.0. Two records were also abstracted by a physician who was blind to the prior abstraction. Agreement between the physician and the original abstractor was 100 percent. DEMOGRAPHICS
Of the women studied across all three time periods, 60 percent had private insurance, and about one-third had Medicare coverage. Only 5 percent had Medicaid coverage or were self-pay. The average age at the time of the definitive cancer surgery was 58 years, with a range of 25 to 92 years. Cancer was equally distributed in the right and left breasts, with 49.7 percent having disease in the left breast, 49.7 percent in the right breast, and the remainder in both breasts. One hundred
Changing Practice Patterns
815
eighty-seven women (33 percent) met the definition of Stage I breast cancer and 214 (38 percent) met the definition for early Stage II. PREOPERATIVE PHYSICAL EXAMINATION AND STAGING
In terms of quality of care, 4 percent of the sample were explicitly staged preoperatively. Descriptions in the medical record or examinations of both breasts, size of the tumor, and whether the axillary nodes were fixed, not fixed, or thought to be tumor-free were contained in over 80 percent of the medical records. Other components of staging systems -fixedness of tumor, presence of clavicular nodes, and presence of skin changes over the affected breast -were recorded about 50 percent of the time, however (see Table 1). TYPE OF SURGERY
Among women with Stage I or early Stage II breast cancer, we defined strict compliance with the NIH consensus conference's recommendation on type of surgery as performance of a total mastectomy with axillary dissection, weak compliance as performance of either a total mastectomy or lumpectomy with axillary dissection (the latter is a more tissue-conserving operation than the NIH panel in 1979 was willing to recommend). Seventy-three women were classified as having Stage I or early Stage II breast cancer during time period 1, 12-24 months prior to the consensus conference. Of these, 74 percent had surgeries that were in strict compliance and 77 percent had surgeries that were in weak compliance with the NIH recommendation. Compliance increased at a fairly steady rate thereafter, with the broader meaTable 1: Completeness of Physical Examination of Patients Undergoing Mastectomy as Recorded in Hospital Medical Record Time I
N Component of Examination Both breasts examined Size of tumor Tumor fixed/not fixed Axillary nodes fixed/not fixed Skin changes present/absent Clavicular nodes present/absent Preoperative staging
101 (%) -
83 83 49 83 60 51 2
Percent of Time Recorded Time 2 Time 3 N - 219 N - 248
Time 4 - 568
N
(%)
(%)
(%)
78 91 54 88 52 53 3
81 90 47 90 50 55 7
80 89 50 88 53 53 4
816
HSR: Health Services Research 25:5 (December 1990)
sure ("weak compliance") reaching 93 percent by two years after the conference. Radical mastectomy, a procedure that the panel declared inappropriate and that we defined as noncompliant, was performed on 14 percent of patients in time 1, 7 percent in time 2, and 2 percent in time 3. Also, from time period 1 to 3, the percentage of women receiving a total mastectomy or lumpectomy without axillary dissection declined from 10 percent to 6 percent (Table 2). Although not specifically disapproved by the panel, this procedure clearly did not represent its therapeutic choice. No matter which definition was used to describe compliance (weak or strict), no acceleration in the rate of change could be detected by means of analysis of covariance (ANCOVA) between the pre- and postconference periods, but there was a significant preconference trend toward the disuse of radical mastectomy (p < .05). The high initial compliance with the recommendation at preconference time 1 and the still higher rate at preconference time 2 indicate that by the time the conference was held, most physicians had already changed their practices to those deemed appropriate by the NIH panel. By either definition, compliance varied significantly among hospitals (p < .05), and the patterns of compliance among hospitals further varied by time (shown by a hospital-by-time interaction, p < .05). The patient's age was not a significant predictor of compliance (see Table 3). ONE-STEP VERSUS TWO-STEP PROCEDURES
Even though some physicians may disagree, the NIH consensus conference recommended against the use of a one-step procedure, in which a biopsy to determine if cancer is present and a definitive surgery to remove the cancer are performed under one anesthesia without waking the patient up to explain the biopsy results. At all three time periods, one-step procedures were performed more frequently than two-step procedures (Table 4). Discussions between the patient receiving a two-step procedure and her physician regarding therapeutic options were infrequently recorded in the medical record. Discussions occurred between biopsy and definitive surgery 3, 14, and 16 percent of the time in periods 1, 2, and 3, respectively. Discussions any time before the definitive therapy were noted 14, 28, and 30 percent of the time. (These percentages are based on samples of 36, 80, and 114 women, respectively, in each time period.) These results provide little evidence of change following the conference. We defined weak compliance with this recommendation as per-
Changing Practice Patrns
817
Table 2: Type of Breast Cancer Surgery Performed on Women with Stage I or Early Stage II Breast Cancer Percent of Time Recorded Time 1 Time 2 Time 3 N - 73 N- 158 N = 170 (%) (%) (%) 74 78 84 3 6 9 14 7 2 8 10 6
Type of Surgery Total mastectomy with axillary dissection* Lumpectomy with axillary dissectiont Radical mastectomy Total mastectomies or lumpectomies without axillary dissection or type of surgery not known Totalt 100 100 100 *This category represents strict compliance with the NIH panel's recommendation. tThis category when combined with * represents weak compliance. tPercentages were rounded to the nearest number.
Table 3: Compliance with Two Recommendations by Patient Age 3a. Patients Receiving Particular Types of Surgeries Percent
Total Mastectomy with Axillary Dissection (%) Age* Number Radical 25-49 5 124 81 50-65 167 7 83 66+ 110 5 74 All ages 401t 6 80 3b. Procedures Performed in One Step or Two Steps
Other or
Lumpectomy
Unknown
(%)
(%)
7 7 6 7
6 4 15 7
Percent One Step Two Step Number (%) (%) 157 25-49 47 53 50-65 236 61 39 66+ 175 65 35 All ages 568 59 42 'The relationship between age and type of procedure is not significant. IThe total N = 401 includes all women who were Stage I or early Stage II. tThe relationship between age and type of procedure is significant (p < .001).
Aget
818
HSR: Health Services Research 25:5 (December 1990)
Table 4: Number and Percent of Breast Cancer Surgeries Performed as Two-Step Procedures, by Disease Stage and Time Period Early
Other
All
N
(%)
Stages (N = 167)
Stages
187)
Stage II (N = 214)
(61) (51) (62) (57)
8 29 40 77
(21) (34) (44) (36)
Stage I
Time Period 1 (N = 101) 2 (N = 219) 3 (N = 248) All times
(N N
21 37 49 107
=
(%)
N
(%)
7 20 25 52
(25) (33) (32) (31)
(N N 36 86 114 236
=
568) (%) (36) (39) (46) (42)
forming a two-step procedure, and strict compliance as performing a two-step procedure with a recorded discussion of therapeutic options. The population studied included 401 women identified as having Stage I or early Stage II breast cancer. Once again, we performed an ANCOVA analysis with the major focus on changes in practice over time. The only significant trend to emerge from this analysis was a general linear trend toward increasing compliance over time; this was true regardless of our use of strong compliance or weak compliance as the dependent variable. The rate of change was, if anything, greatest between time 1 and 2 prior to the conference. When the whole population was analyzed, rather than just patients in Stage I and early Stage II, significant linear and preconference changes were found. Once again, the pattern of results does not suggest conference-induced changes in behavior. With this recommendation, unlike the first, compliance remained low even in time period 3, especially for older women. Younger women were much more likely to receive a two-step procedure (see Table 3). SELECTED MEASURES OF QUALITY OF CARE
In addition to measuring the adequacy of preoperative staging and compliance with the consensus conference recommendations, we examined how often sentinel laboratory tests or procedures were performed in each time period and the frequency of postoperative staging and examination of nodes. This allowed us to determine whether changes in practice were occurring in areas other than those addressed by the conference. As Table 5 shows, very little change in practice occurred in the six areas we examined concerning the use of laboratory tests. There was, however, a significant increase in the presence of explicit postoperative staging, from 22 percent in time 1 to 34 percent
Changing Practice Pattens
819
Table 5: Use of Laboratory Tests or Procedures Prior to Mastectomy and Postoperative Pathologic Staging in Patients Undergoing Surgery for Breast Cancer Time 1 N = 101 Preoperative laboratory test or procedure Alkaline phosphatase Serum calcium Whole-body scan Chest x-ray Mammography Postoperative pathologic exam Patient staged Nodes examined
Percent of Patients on Whom Test Was Performed Time 2 Time 3 N = 219 N = 248
Time 4 N = 568
(%)
(%)
(%)
(%)
75 73 20 78 100
73 72 30 81 100
74 75 25 80 100
74 73 26 80 100
22 87
27 89
34 97
29 92
in time 3, and an increase in examination of nodes, from 87-97 percent. The information from pathologic examination of nodes could be used for formal patient staging, but these data are not sufficient for a complete TNM staging.
DISCUSSION The results reported here present a mixed picture of the care given to women with new, primary breast cancers. On the one hand, pectoral muscle-conserving operations were used for the majority of women, even before the NIH's consensus conference adopted a recommendation supporting such action. Any further changes in practice observed after the conference can probably best be interpreted as a continuation of an established trend. On the other hand, despite the urging of the panel, fewer than half of the patients received a two-step procedure in the 13 to 24 months after the conference. Fewer than 5 percent of the women were formally staged preoperatively in 1981, and fewer than half of the medical records for these women contained all of the information that would be needed for adequate staging. Because even patients with early disease have substantial recurrence rates, prompt staging is required for proper management and therapy. Thus, the infrequency with which staging was performed and documented repre-
820
HSR: Health Services Research 25:5 (December 1990)
sents a noteworthy deficiency in quality of care, a deficiency not addressed by the NIH consensus conference. Perhaps other studies should be performed to see if such deficiencies in care exist for patients with other cancers. Physicians sometimes differed in their treatment between younger and older women, with older women receiving care that was less likely to conform to the NIH's recommendations. It is not clear why this should have been happening, although differences in treatment according to the patient's age have been observed in other research (Greenfield et al. 1987). A two-step procedure may be considered more important by younger women (or their surgeons). Neither the NIH panel's recommendations nor the literature on which they were based, however, gave any basis for performing two-step procedures on younger women and one-step procedures on older women. It is possible that some of the one-step procedures represented cases in which the mammogram was highly suspicious of cancer and the axillary nodes were palpable. In such cases, the physician and patient might have discussed the likelihood of cancer and possible treatment options, with the woman agreeing to a one-step procedure if the frozen section was positive. In our sample, 100 percent of women had a mammogram or a zeromammogram on the operated side, but just 26 percent had results that were highly suspicious of cancer and 46 percent had results that were suspicious. In addition, although 88 percent of women had an axillary node examination by a physician preoperatively, only 8 percent of these had palpable and fixed nodes or nodes that were suspicious of cancer. Taken together, these data suggest that the certainty of cancer being present prior to surgery was less than might otherwise be expected and that therefore a one-step procedure could not be justified on these grounds. Because only 6 percent of the medical records contained some explanation for the use of a one-step procedure, we were not able to determine how often this approach might have been taken because of a highly suspicious presentation. We note, however, that in 1979, the NIH panel did not consider the presence of a highly suspicious result on mammography to be a justification for a one-step approach. The percentage of two-step procedures changes from 36 percent in time 1 to 39 percent in time 2 to 46 percent in time 3. These results are consistent with those from a study of the use of two-step procedures among 993 women diagnosed between 1974 and 1981 in western Washington state (Polissar and Finley 1985). That research also
ChLanging Practice Paterns
821
reported an increase in the use of two-step procedures, from 28 to 57 percent over the eight-year study period. Discussions between physicians and patients concerning surgical options, including the issue of one-step versus two-step procedures, were infrequently documented in medical records. We recognize that discussions between patients and physicians about treatment options may not always be recorded and that the quality of recording may vary among physicians. To minimize the effects of recording quality in our data, we accepted as evidence of a discussion any note about surgical options appearing anywhere in the medical record. If documentation reflects the importance physicians assign to these interactions, then our data suggest that they are not considered an essential aspect of care. Our findings are limited in that (1) they come from ten hospitals in one region of the country and (2) the study does not provide statistical power to detect anything less than moderate changes. Nevertheless, they demonstrate the failure of the NIH's Consensus Development Program to significantly affect the quality of care given to breast cancer patients, even though improvement for at least some recommendations was possible. For the panel's main recommendation, on the elimination of radical mastectomies, preconference compliance was so high that the NIH panel's recommendation could have had very little influence. At the same time, there were areas of deficient care that were not addressed at all by the conference. Better selection of issues to be covered by the conference might have increased its effectiveness. For conference planners and panelists to do a better job of addressing relevant issues, however, they probably need better epidemiological data on practice patterns. Without information of the sort provided by this study, for example, it is difficult to see how conference planners could have known how often breast cancer patients were being staged or how many were receiving radical mastectomies. For those topics where change could have occurred but did not, it is possible that physicians did not know about the NIH consensus conference, were unaware of its specific findings, or disagreed with the panel's recommendations. We investigated these issues with data from a national physician sample. Our findings suggest that the NIH Consensus Development Program achieved at least moderate success in making itself known to physicians (Kanouse et al. 1989). Further, results of this particular conference were widely published in both the lay and professional press. Beginning the day after the conference on June 6, 1979, the study was reported by the Associated Press and the Washington Post, and was frequently mentioned for many months thereafter in popular press articles on breast cancer and breast cancer treat-
822
HSR: Health Services Research 25:5 (December 1990)
ment (Winkler et al. 1986). On August 9, the New EnglandJournal of Medicine published the entire consensus statement (National Institutes of Health 1979). A brief report covering the major recommendations appeared in Science (Holden 1979), and several articles discussing the panel recommendations appeared the following year (Fisher 1979; Moxley et al. 1990; Urban 1980). The postconference measurement period defined in our evaluation began in July 1980, by which time physicians had had at least a fulll year to learn about the conference and to incorporate its recommendations into their practice. We should not be overly pessimistic about evaluation results based on a single conference, especially since it is notoriously difficult to change physicians' (or other people's) behavior simply by providing information (Kanouse and Jacoby 1988; Lloyd and Abrahamson 1979; McGuire 1985). One strategy for enhancing change is to engage the active participation of the target audience in defining and disseminating the message. For consensus conferences, this might be done by seeking the active collaboration and even cosponsorship of conferences by professional societies and relevant lay organizations. These associations, through polls of their members, could screen potential conference topics and help identify those that are timely and important. Further, associations that have helped organize a conference may be more enthusiastic about endorsing and disseminating its recommendations. Finally, the opportunity we had to evaluate the NIH's consensus conference on breast cancer allowed us to collect data about quality of care. We identified major deficiencies in the process of care, especially in the areas of staging the cancer and communicating to women about their disease. These findings, although a decade old, are most likely relevant today and lead us to call for a system that monitors the quality of care given to cancer patients, especially when the cancer is amenable to therapy.
ACKNOWLEDGMENTS We are grateful to Itzhak Jacoby of OMAR for his cooperation and to Carol Weiss and other members of our project oversight committee for their constructive advice. We are also indebted to Armando Guiliano, George Goldberg, and Mark Chassin for their clinical insight, to Lois McCloskey for supervising data collection, and to Marian Oshiro for computer programming.
Changing Practice Patterns
823
REFERENCES Fisher, B. "Breast Cancer Management: Alternatives to Radical Mastectomy." New EnglandJournal of Medicine 301, no. 6 (1979):326-28. Greenfield, S., D. M. Blanco, R. M. Elashoff, and P. A. Ganz. "Patterns of Care Related to Age of Breast Cancer Patients." Journal of the American Medical Association 257, no. 20 (1987):2766-70. Holden, C. "Modified Early Surgery for Breast Cancer." Science 204, no. 4399 (1979): 1284-85. Kanouse, D. E., et al. Changing Medical Practice Through Technology Assessment: An Evaluation of the NIH Consensus Development Program. Ann Arbor, MI: Health Administration Press, 1989. Kanouse, D. E., and I. Jacoby. "When Does Information Change Practitioners' Behavior?" InternationalJournal of Technology Assessment in Health Care 4, no. 1 (1988):27-33. Kosecoff, J., D. E. Kanouse, W. H. Rogers, L. McCloskey, C. M. Winslow, and R. H. Brook. "Effects of the NIH Consensus Development Program on Physician Practice." Journal of the American Medical Association 258, no. 19 (1987):2708-13. Lloyd, J. S., and S. Abrahamson. "Effectiveness of Continuing Medical Education." Evaluation and the Health Professions 2, no. 3 (1979):251-80. McGuire, W. J. "Attitudes and Attitude Change." In Handbook of Social Psycholoy. 3rd ed. Vol. 2. Edited by G. Lindzey and E. Aronson. New York: Random House, 1985, pp. 232-346. Moxley, J. H., III, J. C. Allegra, J. Henney, and F. Muggia. "Treatment of Primary Breast Cancer." Journal of the American Medical Association 244, no. 8 (1980):797-800. National Institutes of Health. "Special Report: Treatment of Primary Breast Cancer." New EnglandJournal of Medicine 301, no. 6 (1979):340-41. Polissar, L., and M. L. Finley. "Time Trends and Key Factors in the Choice of One-Step or Two-Step Biopsy and Surgery for Breast Cancer." Social Science and Medicine 21, no. 7 (1985):733-40. Stockdale, F. E. "Breast Cancer." In Scientific American Medicine. Edited by F. E. Rubinstein and D. Federman. December 1986, IX, no. 12, Section 12, no. 7. The World Almanac and Book of Facts 1990. New York: Pharos Books, 1989. Urban, J. A. "Seeking Consensus on Breast Cancer." Journal of the American Medical Association 244, no. 8 (1980):800-803. Winkler, J. D., D. E. Kanouse, L. Brodsley, and R. H. Brook. "Popular Press Coverage of Eight National Institutes of Health Consensus Development Conferences."Joumal of the American Medical Association 255, no. 10 (1986):1323-27.
810
HSR: Health Services Research 25:5 (December 1990)
and that quality of care in diagnosis and treatment of breast cancer still needs to be addressed.
Breast cancer is a serious disease that in 1989 affected an estimated 142,900 newly diagnosed women and caused 43,300 deaths (The World Almanac and Book of Facts 1990 1989). In the last decade, new knowledge has been gained concerning the efficacy of its treatment. For that reason, in June 1979, the National Institutes of Health (NIH) held a consensus conference on the treatment of primary breast cancer (National Institutes of Health 1979). Among the major recommendations resulting from the conference were that (1) a total mastectomy with axillary dissection should be the treatment standard for women with Stage I and early Stage II primary breast cancer, and (2) a twostep procedure should be followed in which a diagnostic biopsy is studied by permanent histologic sections and therapeutic options are discussed before definitive surgery is performed. The NIH consensus conferences are designed to facilitate the appropriate and timely application of biomedical research findings in medical practice. They seek to provide practicing physicians with timely and useful information, to facilitate informed decision making by physicians and patients regarding complex medical therapies and to foster, where they are needed, appropriate changes in medical practices that will improve patients' health care outcomes. None of these aims is easily achieved, especially that of changing medical practices. Nevertheless, success in achieving any of these aims depends on the relevance of conference recommendations to clinical practice. Thus, we sought to determine, as part of out evaluation of the NIH Consensus Development Program, whether the conferences were addressing physician practices where change was needed and, if so, whether there were changes in physician behavior that might be attributable to the program. We abstracted hospital medical records to determine, first, whether these two consensus conference recommendations had influenced physicians' surgical approaches to primary breast cancer. At the same time, and as the second major purpose of the study, we collected extensive data about the services provided to women with breast cancer and the quality of those services. This article draws on those data to describe how primary breast cancer is managed during the woman's initial hospitalization for surgical therapy. Although the data collected
Changing Practice Patterns
811
represent the early part of the 1980s, the issues they raise are probably still relevant today.
METHODS STUDY DESIGN
We identified all nonspecialty, nonfederal acute care hospitals in the state of Washington with more than 150 beds and stratified them by teaching status. We then randomly selected ten hospitals (likelihood of being accepted proportional to number in strata) such that one institution was a major teaching hospital, two had close ties with teaching hospitals, one was a nonaffiliated hospital with a residency program, and the remaining six were community hospitals without university affiliation. Of the ten hospitals approached to participate in the study, two refused and were replaced with other hospitals having similar characteristics. We studied the medical records from two one-year periods before the conference Uuly 1977 through June 1978 and July 1978 through June 1979) and one period after the conference (July 1980 through June 1981). The after period began 12 months after the conference to allow for dissemination of the conference results. The inclusion of a second preconference time period allowed for estimates of pre- and post-consensus conference differences in the rate of change in medical practice as well as the absolute amount of change. Hospital logs or hospital abstracting services' reports were used to identify patients who had both an ICD-9-CM diagnosis of breast cancer (codes 174.0-174.9) and a procedure of mastectomy (codes 85.20-85.23, 85.41-85.48, 85.99). We employed a two-stage sampling procedure, first sampling physicians within hospitals, then patients within physicians in such a way that within a given hospital, each patient had an equal chance of being selected, and so that the sampled physicians would be representative of the physicians in the state who were providing care to breast cancer patients. DATA COLLECTION
The basic data source was a medical-record review of patients hospitalized during the relevant time periods. A medical record abstraction form targeted to the first surgical treatment of primary breast cancer was specifically designed and pilot tested for the study. The instrument consisted of items measuring (1) patient demographics; (b) stage of
812
HSR: Health Services Research 25:5 (December 1990)
breast cancer; (c) aspects of care addressed in the NIH consensus conference, such as type of surgical therapy; and (d) other aspects of the process of care that could be used to describe or evaluate the care provided to the patient during hospitalization. The abstraction form took about 30 minutes to complete. Examples of specific information collected for each patient included: affected breast, preoperative results of alkaline phosphatase and serum calcium tests, whole-body scans, mammography, chest x-rays, pre- and postoperative stage, preoperative examination of both breasts, type of surgery, length of stay for biopsy and definitive surgery, pathology results, and performance of estrogen receptor assays. Twenty-eight persons with a medical record abstraction background were trained to collect the data. All had prior medical record review experience, passed an initial test of their abstraction skills, were given four days of intensive training, and successfully completed a further test at the end of training. Data collectors were provided with a list of records to pull and abstract and a list of backup records to pull if cases in the initial list were unavailable or met the study's exdusion criteria. Work was supervised by a chief data collector who regularly visited each hospital and reabstracted records to maintain quality control. Completed abstraction forms were reviewed by both a physician and nonphysician, who assessed internal consistency and whether coding decisions were consistent with the descriptive clinical data copied from medical records to justify decisions. Discrepancies that could not be resolved during the review process were returned for reabstraction. To assure the confidentiality of information, we assigned coded identifiers to patients, physicians, and hospitals. Once the data collection process had been completed, all files linking these identifiers to patients, physicians, or institutions were destroyed. In all, 573 charts were selected to be abstracted. To achieve this level, 84 records that failed the exclusion criteria were systematically replaced with substitute records. Reasons for exclusion included: (1) missing records (N = 8); (2) duplicate records (N = 2); (3) wrong time periods (N = 2); or (4) meeting one or more of the following exdusion criteria: the admission was not the first surgical treatment (mastectomy) for a primary unilateral breast cancer, the patient was a nursing home resident, or the patient had obvious metastatic disease at the time of admission (N = 72). Five records were lost during the keypunch process. Thus, the final sample was 568 records (101 in time period 1, 219 in time 2, and 248 in time 3). To identify women with Stage I and early Stage II breast cancer, we drew upon information abstracted from the preoperative physical
Changing Practice Pattns
813
examination and history recorded in the medical record. To interpret this information, we used the Tumor Node Metastatic (TNM) system, which uses data about the suspected size of the tumor, whether lymph nodes in the axilla are thought to contain cancer, and whether the tumor has spread beyond the breast to distant parts of the body (Stockdale 1986). Because the NIH consensus conference primarily addressed surgical approaches to the treatment of Stage I and early Stage II breast cancer, we took pains to distinguish women in these stages from those with more advanced disease. Specifically, a woman was classified as having Stage I disease if and only if six conditions were met: (1) the tumor was not palpable or was less than or equal to 2 centimeters in size; (2) the tumor was described as not fixed or fixed only to muscle or pectoral fascia; (3) no skin changes over the breast (skin edema, infiltration, or peau d'orange) were noted; (4) axillary nodes were not thought to contain tumor; (5) clavicular nodes were not palpable; and (6) there was no evidence of metastatic disease beyond the regional lymph nodes. We classified a woman as having early Stage II breast cancer if six criteria were met: (1) the tumor was greater than 2 but less than 5 centimeters in size; (2) the tumor was described as not fixed; (3) no skin changes (skin edema, peu d'orange, or infiltration) were noted; (4) axillary nodes were not thought to contain tumor; (5) clavicular nodes were not palpable; and (6) no evidence of metastatic disease presented beyond the regional lymph nodes. In addition, women with a tumor that was less than or equal to 2 centimeters, who met criteria 2, 3, 5, and 6 above, but who had suspicious axillary nodes that still were freely movable (not fixed) were classified as having early Stage II disease. We also excluded from Stage I or early Stage II anyone who had evidence of advanced disease based on results of laboratory tests or other data known prior to the definitive surgery. Accordingly, a woman who met any of the following criteria was excluded: (1) a chest x-ray that was suspicious of cancer; (2) a mammogram or xeromammogram that showed evidence of cancer in the presumably unaffected breast; (3) a preoperative statement by the physician that assigned the patient to Stage III or IV; (4) a whole-body scan that was suspicious of cancer; or (5) both breasts described as affected by cancer. It should be noted that in defining stages, we interpreted the absence of certain data in the medical record as evidence that a significant event had not occurred. Thus, no mention whatsoever of clavicular node involvement was interpreted as negative clavicular node involvement. Further details about the study methods have been pub-
814
HSR: Health Services Research 25:5 (Decernber 1990)
lished elsewhere (Kanouse et al. 1989; Kosecoff, Kanouse, Rogers, et al. 1987). ANALYSIS
Changes over time were tested with repeated-measures analysis of covariance in which the covariate was the patient's age at the time of definitive cancer surgery. Two orthogonal contrasts were used to test different patterns of change. The first tested whether there was a linear trend from time 1 to time 3; the second (quadratic) contrast tested whether the change from time 1 to time 2 (two contiguous years immediately prior to the conference) was different from the change from time 2 to time 3 (one year before the conference to the 12-24 month period after the conference). The contrast of greatest interest from the standpoint of testing possible consensus conference effects is the second (quadratic) contrast, which captures any acceleration in the rate of change from the preconference to postconference periods. In addition to these two orthogonal contrasts, we also report results for a third contrast, testing whether significant change occurred during the preconference period (time 1 to time 2). The statistical significance of other study findings was examined with chi-square analyses.
RESULTS RELIABILITY
Fourteen breast cancer records were randomly reabstracted by a different data collector. Kappa was used to assess reliability for ten critical variables that were used to define compliance with a recommendation. Across the 14 records, the mean Kappa was 1.0. Two records were also abstracted by a physician who was blind to the prior abstraction. Agreement between the physician and the original abstractor was 100 percent. DEMOGRAPHICS
Of the women studied across all three time periods, 60 percent had private insurance, and about one-third had Medicare coverage. Only 5 percent had Medicaid coverage or were self-pay. The average age at the time of the definitive cancer surgery was 58 years, with a range of 25 to 92 years. Cancer was equally distributed in the right and left breasts, with 49.7 percent having disease in the left breast, 49.7 percent in the right breast, and the remainder in both breasts. One hundred
Changing Practice Patterns
815
eighty-seven women (33 percent) met the definition of Stage I breast cancer and 214 (38 percent) met the definition for early Stage II. PREOPERATIVE PHYSICAL EXAMINATION AND STAGING
In terms of quality of care, 4 percent of the sample were explicitly staged preoperatively. Descriptions in the medical record or examinations of both breasts, size of the tumor, and whether the axillary nodes were fixed, not fixed, or thought to be tumor-free were contained in over 80 percent of the medical records. Other components of staging systems -fixedness of tumor, presence of clavicular nodes, and presence of skin changes over the affected breast -were recorded about 50 percent of the time, however (see Table 1). TYPE OF SURGERY
Among women with Stage I or early Stage II breast cancer, we defined strict compliance with the NIH consensus conference's recommendation on type of surgery as performance of a total mastectomy with axillary dissection, weak compliance as performance of either a total mastectomy or lumpectomy with axillary dissection (the latter is a more tissue-conserving operation than the NIH panel in 1979 was willing to recommend). Seventy-three women were classified as having Stage I or early Stage II breast cancer during time period 1, 12-24 months prior to the consensus conference. Of these, 74 percent had surgeries that were in strict compliance and 77 percent had surgeries that were in weak compliance with the NIH recommendation. Compliance increased at a fairly steady rate thereafter, with the broader meaTable 1: Completeness of Physical Examination of Patients Undergoing Mastectomy as Recorded in Hospital Medical Record Time I
N Component of Examination Both breasts examined Size of tumor Tumor fixed/not fixed Axillary nodes fixed/not fixed Skin changes present/absent Clavicular nodes present/absent Preoperative staging
101 (%) -
83 83 49 83 60 51 2
Percent of Time Recorded Time 2 Time 3 N - 219 N - 248
Time 4 - 568
N
(%)
(%)
(%)
78 91 54 88 52 53 3
81 90 47 90 50 55 7
80 89 50 88 53 53 4
816
HSR: Health Services Research 25:5 (December 1990)
sure ("weak compliance") reaching 93 percent by two years after the conference. Radical mastectomy, a procedure that the panel declared inappropriate and that we defined as noncompliant, was performed on 14 percent of patients in time 1, 7 percent in time 2, and 2 percent in time 3. Also, from time period 1 to 3, the percentage of women receiving a total mastectomy or lumpectomy without axillary dissection declined from 10 percent to 6 percent (Table 2). Although not specifically disapproved by the panel, this procedure clearly did not represent its therapeutic choice. No matter which definition was used to describe compliance (weak or strict), no acceleration in the rate of change could be detected by means of analysis of covariance (ANCOVA) between the pre- and postconference periods, but there was a significant preconference trend toward the disuse of radical mastectomy (p < .05). The high initial compliance with the recommendation at preconference time 1 and the still higher rate at preconference time 2 indicate that by the time the conference was held, most physicians had already changed their practices to those deemed appropriate by the NIH panel. By either definition, compliance varied significantly among hospitals (p < .05), and the patterns of compliance among hospitals further varied by time (shown by a hospital-by-time interaction, p < .05). The patient's age was not a significant predictor of compliance (see Table 3). ONE-STEP VERSUS TWO-STEP PROCEDURES
Even though some physicians may disagree, the NIH consensus conference recommended against the use of a one-step procedure, in which a biopsy to determine if cancer is present and a definitive surgery to remove the cancer are performed under one anesthesia without waking the patient up to explain the biopsy results. At all three time periods, one-step procedures were performed more frequently than two-step procedures (Table 4). Discussions between the patient receiving a two-step procedure and her physician regarding therapeutic options were infrequently recorded in the medical record. Discussions occurred between biopsy and definitive surgery 3, 14, and 16 percent of the time in periods 1, 2, and 3, respectively. Discussions any time before the definitive therapy were noted 14, 28, and 30 percent of the time. (These percentages are based on samples of 36, 80, and 114 women, respectively, in each time period.) These results provide little evidence of change following the conference. We defined weak compliance with this recommendation as per-
Changing Practice Patrns
817
Table 2: Type of Breast Cancer Surgery Performed on Women with Stage I or Early Stage II Breast Cancer Percent of Time Recorded Time 1 Time 2 Time 3 N - 73 N- 158 N = 170 (%) (%) (%) 74 78 84 3 6 9 14 7 2 8 10 6
Type of Surgery Total mastectomy with axillary dissection* Lumpectomy with axillary dissectiont Radical mastectomy Total mastectomies or lumpectomies without axillary dissection or type of surgery not known Totalt 100 100 100 *This category represents strict compliance with the NIH panel's recommendation. tThis category when combined with * represents weak compliance. tPercentages were rounded to the nearest number.
Table 3: Compliance with Two Recommendations by Patient Age 3a. Patients Receiving Particular Types of Surgeries Percent
Total Mastectomy with Axillary Dissection (%) Age* Number Radical 25-49 5 124 81 50-65 167 7 83 66+ 110 5 74 All ages 401t 6 80 3b. Procedures Performed in One Step or Two Steps
Other or
Lumpectomy
Unknown
(%)
(%)
7 7 6 7
6 4 15 7
Percent One Step Two Step Number (%) (%) 157 25-49 47 53 50-65 236 61 39 66+ 175 65 35 All ages 568 59 42 'The relationship between age and type of procedure is not significant. IThe total N = 401 includes all women who were Stage I or early Stage II. tThe relationship between age and type of procedure is significant (p < .001).
Aget
818
HSR: Health Services Research 25:5 (December 1990)
Table 4: Number and Percent of Breast Cancer Surgeries Performed as Two-Step Procedures, by Disease Stage and Time Period Early
Other
All
N
(%)
Stages (N = 167)
Stages
187)
Stage II (N = 214)
(61) (51) (62) (57)
8 29 40 77
(21) (34) (44) (36)
Stage I
Time Period 1 (N = 101) 2 (N = 219) 3 (N = 248) All times
(N N
21 37 49 107
=
(%)
N
(%)
7 20 25 52
(25) (33) (32) (31)
(N N 36 86 114 236
=
568) (%) (36) (39) (46) (42)
forming a two-step procedure, and strict compliance as performing a two-step procedure with a recorded discussion of therapeutic options. The population studied included 401 women identified as having Stage I or early Stage II breast cancer. Once again, we performed an ANCOVA analysis with the major focus on changes in practice over time. The only significant trend to emerge from this analysis was a general linear trend toward increasing compliance over time; this was true regardless of our use of strong compliance or weak compliance as the dependent variable. The rate of change was, if anything, greatest between time 1 and 2 prior to the conference. When the whole population was analyzed, rather than just patients in Stage I and early Stage II, significant linear and preconference changes were found. Once again, the pattern of results does not suggest conference-induced changes in behavior. With this recommendation, unlike the first, compliance remained low even in time period 3, especially for older women. Younger women were much more likely to receive a two-step procedure (see Table 3). SELECTED MEASURES OF QUALITY OF CARE
In addition to measuring the adequacy of preoperative staging and compliance with the consensus conference recommendations, we examined how often sentinel laboratory tests or procedures were performed in each time period and the frequency of postoperative staging and examination of nodes. This allowed us to determine whether changes in practice were occurring in areas other than those addressed by the conference. As Table 5 shows, very little change in practice occurred in the six areas we examined concerning the use of laboratory tests. There was, however, a significant increase in the presence of explicit postoperative staging, from 22 percent in time 1 to 34 percent
Changing Practice Pattens
819
Table 5: Use of Laboratory Tests or Procedures Prior to Mastectomy and Postoperative Pathologic Staging in Patients Undergoing Surgery for Breast Cancer Time 1 N = 101 Preoperative laboratory test or procedure Alkaline phosphatase Serum calcium Whole-body scan Chest x-ray Mammography Postoperative pathologic exam Patient staged Nodes examined
Percent of Patients on Whom Test Was Performed Time 2 Time 3 N = 219 N = 248
Time 4 N = 568
(%)
(%)
(%)
(%)
75 73 20 78 100
73 72 30 81 100
74 75 25 80 100
74 73 26 80 100
22 87
27 89
34 97
29 92
in time 3, and an increase in examination of nodes, from 87-97 percent. The information from pathologic examination of nodes could be used for formal patient staging, but these data are not sufficient for a complete TNM staging.
DISCUSSION The results reported here present a mixed picture of the care given to women with new, primary breast cancers. On the one hand, pectoral muscle-conserving operations were used for the majority of women, even before the NIH's consensus conference adopted a recommendation supporting such action. Any further changes in practice observed after the conference can probably best be interpreted as a continuation of an established trend. On the other hand, despite the urging of the panel, fewer than half of the patients received a two-step procedure in the 13 to 24 months after the conference. Fewer than 5 percent of the women were formally staged preoperatively in 1981, and fewer than half of the medical records for these women contained all of the information that would be needed for adequate staging. Because even patients with early disease have substantial recurrence rates, prompt staging is required for proper management and therapy. Thus, the infrequency with which staging was performed and documented repre-
820
HSR: Health Services Research 25:5 (December 1990)
sents a noteworthy deficiency in quality of care, a deficiency not addressed by the NIH consensus conference. Perhaps other studies should be performed to see if such deficiencies in care exist for patients with other cancers. Physicians sometimes differed in their treatment between younger and older women, with older women receiving care that was less likely to conform to the NIH's recommendations. It is not clear why this should have been happening, although differences in treatment according to the patient's age have been observed in other research (Greenfield et al. 1987). A two-step procedure may be considered more important by younger women (or their surgeons). Neither the NIH panel's recommendations nor the literature on which they were based, however, gave any basis for performing two-step procedures on younger women and one-step procedures on older women. It is possible that some of the one-step procedures represented cases in which the mammogram was highly suspicious of cancer and the axillary nodes were palpable. In such cases, the physician and patient might have discussed the likelihood of cancer and possible treatment options, with the woman agreeing to a one-step procedure if the frozen section was positive. In our sample, 100 percent of women had a mammogram or a zeromammogram on the operated side, but just 26 percent had results that were highly suspicious of cancer and 46 percent had results that were suspicious. In addition, although 88 percent of women had an axillary node examination by a physician preoperatively, only 8 percent of these had palpable and fixed nodes or nodes that were suspicious of cancer. Taken together, these data suggest that the certainty of cancer being present prior to surgery was less than might otherwise be expected and that therefore a one-step procedure could not be justified on these grounds. Because only 6 percent of the medical records contained some explanation for the use of a one-step procedure, we were not able to determine how often this approach might have been taken because of a highly suspicious presentation. We note, however, that in 1979, the NIH panel did not consider the presence of a highly suspicious result on mammography to be a justification for a one-step approach. The percentage of two-step procedures changes from 36 percent in time 1 to 39 percent in time 2 to 46 percent in time 3. These results are consistent with those from a study of the use of two-step procedures among 993 women diagnosed between 1974 and 1981 in western Washington state (Polissar and Finley 1985). That research also
ChLanging Practice Paterns
821
reported an increase in the use of two-step procedures, from 28 to 57 percent over the eight-year study period. Discussions between physicians and patients concerning surgical options, including the issue of one-step versus two-step procedures, were infrequently documented in medical records. We recognize that discussions between patients and physicians about treatment options may not always be recorded and that the quality of recording may vary among physicians. To minimize the effects of recording quality in our data, we accepted as evidence of a discussion any note about surgical options appearing anywhere in the medical record. If documentation reflects the importance physicians assign to these interactions, then our data suggest that they are not considered an essential aspect of care. Our findings are limited in that (1) they come from ten hospitals in one region of the country and (2) the study does not provide statistical power to detect anything less than moderate changes. Nevertheless, they demonstrate the failure of the NIH's Consensus Development Program to significantly affect the quality of care given to breast cancer patients, even though improvement for at least some recommendations was possible. For the panel's main recommendation, on the elimination of radical mastectomies, preconference compliance was so high that the NIH panel's recommendation could have had very little influence. At the same time, there were areas of deficient care that were not addressed at all by the conference. Better selection of issues to be covered by the conference might have increased its effectiveness. For conference planners and panelists to do a better job of addressing relevant issues, however, they probably need better epidemiological data on practice patterns. Without information of the sort provided by this study, for example, it is difficult to see how conference planners could have known how often breast cancer patients were being staged or how many were receiving radical mastectomies. For those topics where change could have occurred but did not, it is possible that physicians did not know about the NIH consensus conference, were unaware of its specific findings, or disagreed with the panel's recommendations. We investigated these issues with data from a national physician sample. Our findings suggest that the NIH Consensus Development Program achieved at least moderate success in making itself known to physicians (Kanouse et al. 1989). Further, results of this particular conference were widely published in both the lay and professional press. Beginning the day after the conference on June 6, 1979, the study was reported by the Associated Press and the Washington Post, and was frequently mentioned for many months thereafter in popular press articles on breast cancer and breast cancer treat-
822
HSR: Health Services Research 25:5 (December 1990)
ment (Winkler et al. 1986). On August 9, the New EnglandJournal of Medicine published the entire consensus statement (National Institutes of Health 1979). A brief report covering the major recommendations appeared in Science (Holden 1979), and several articles discussing the panel recommendations appeared the following year (Fisher 1979; Moxley et al. 1990; Urban 1980). The postconference measurement period defined in our evaluation began in July 1980, by which time physicians had had at least a fulll year to learn about the conference and to incorporate its recommendations into their practice. We should not be overly pessimistic about evaluation results based on a single conference, especially since it is notoriously difficult to change physicians' (or other people's) behavior simply by providing information (Kanouse and Jacoby 1988; Lloyd and Abrahamson 1979; McGuire 1985). One strategy for enhancing change is to engage the active participation of the target audience in defining and disseminating the message. For consensus conferences, this might be done by seeking the active collaboration and even cosponsorship of conferences by professional societies and relevant lay organizations. These associations, through polls of their members, could screen potential conference topics and help identify those that are timely and important. Further, associations that have helped organize a conference may be more enthusiastic about endorsing and disseminating its recommendations. Finally, the opportunity we had to evaluate the NIH's consensus conference on breast cancer allowed us to collect data about quality of care. We identified major deficiencies in the process of care, especially in the areas of staging the cancer and communicating to women about their disease. These findings, although a decade old, are most likely relevant today and lead us to call for a system that monitors the quality of care given to cancer patients, especially when the cancer is amenable to therapy.
ACKNOWLEDGMENTS We are grateful to Itzhak Jacoby of OMAR for his cooperation and to Carol Weiss and other members of our project oversight committee for their constructive advice. We are also indebted to Armando Guiliano, George Goldberg, and Mark Chassin for their clinical insight, to Lois McCloskey for supervising data collection, and to Marian Oshiro for computer programming.
Changing Practice Patterns
823
REFERENCES Fisher, B. "Breast Cancer Management: Alternatives to Radical Mastectomy." New EnglandJournal of Medicine 301, no. 6 (1979):326-28. Greenfield, S., D. M. Blanco, R. M. Elashoff, and P. A. Ganz. "Patterns of Care Related to Age of Breast Cancer Patients." Journal of the American Medical Association 257, no. 20 (1987):2766-70. Holden, C. "Modified Early Surgery for Breast Cancer." Science 204, no. 4399 (1979): 1284-85. Kanouse, D. E., et al. Changing Medical Practice Through Technology Assessment: An Evaluation of the NIH Consensus Development Program. Ann Arbor, MI: Health Administration Press, 1989. Kanouse, D. E., and I. Jacoby. "When Does Information Change Practitioners' Behavior?" InternationalJournal of Technology Assessment in Health Care 4, no. 1 (1988):27-33. Kosecoff, J., D. E. Kanouse, W. H. Rogers, L. McCloskey, C. M. Winslow, and R. H. Brook. "Effects of the NIH Consensus Development Program on Physician Practice." Journal of the American Medical Association 258, no. 19 (1987):2708-13. Lloyd, J. S., and S. Abrahamson. "Effectiveness of Continuing Medical Education." Evaluation and the Health Professions 2, no. 3 (1979):251-80. McGuire, W. J. "Attitudes and Attitude Change." In Handbook of Social Psycholoy. 3rd ed. Vol. 2. Edited by G. Lindzey and E. Aronson. New York: Random House, 1985, pp. 232-346. Moxley, J. H., III, J. C. Allegra, J. Henney, and F. Muggia. "Treatment of Primary Breast Cancer." Journal of the American Medical Association 244, no. 8 (1980):797-800. National Institutes of Health. "Special Report: Treatment of Primary Breast Cancer." New EnglandJournal of Medicine 301, no. 6 (1979):340-41. Polissar, L., and M. L. Finley. "Time Trends and Key Factors in the Choice of One-Step or Two-Step Biopsy and Surgery for Breast Cancer." Social Science and Medicine 21, no. 7 (1985):733-40. Stockdale, F. E. "Breast Cancer." In Scientific American Medicine. Edited by F. E. Rubinstein and D. Federman. December 1986, IX, no. 12, Section 12, no. 7. The World Almanac and Book of Facts 1990. New York: Pharos Books, 1989. Urban, J. A. "Seeking Consensus on Breast Cancer." Journal of the American Medical Association 244, no. 8 (1980):800-803. Winkler, J. D., D. E. Kanouse, L. Brodsley, and R. H. Brook. "Popular Press Coverage of Eight National Institutes of Health Consensus Development Conferences."Joumal of the American Medical Association 255, no. 10 (1986):1323-27.
