Diagnostic Radiology
Manitoba Barium Enema Efficacy Study 1 Douglas W. MacEwan, M.D., Steven Kavanagh, B. A., Peter Chow, M.Sc., and Jack M. Tishler, M.D. Claims analysis and efficacy study are two new methods of evaluating medical services. In Manitoba, with a 1974 population of 1,053,382, approximately 100,000 patient-physician interfaces resulted in 16,594 barium enema examinations; 1,000 significant abnormalities were discovered, and the referring physician was unusually perceptive in predicting the likelihood of these abnormalities. It is estimated that a 10% reduction in the use of barium enemas could be achieved by referring physicians with no deterioration in the care of patients. INDEX TERMS: 7[5].1281)
Barium-enema examination • Colon, neoplasms. (Colon, routine exam,
Radiology 126:39-44, January 1978 TABLE
w o METHODS of gaining insight into physician performance are analysis of physicians' claims (1) and radiology efficacy studies (2). In Manitoba, the barium enema examination was studied in a closed population, where in-depth medical information and complete financial data were available. The referring physician emerged as essential to optimal cost-effective patient care.
T
I: OVERALL HEALTH PLAN DATA 1975
Total Medicare and hospital expenditures (8) Manitoba Department of Health and Social Development Manitoba Pharmicare plan Budget of the Faculty of Medicine Manitoba per capita medical costs" United States per capita medical costs (9) (estimate for 1976) Population Total practitioners Number of claims Number of services to patients Number of separations Value of professional services to patients
METHOD OF STUDY
Analysis of physicians' claims is particularly appropriate in Manitoba, as the government health plan covers all hospital and ambulatory care for the entire population. Physicians are paid a fee based on claims for patient service. Extensive reliable medical information can be assembled by analysis of the complete computer records used for all physicians' claims and all hospital separations
$330.812,659 $285,094,000 $32.000,000 $6,958,019 $679 $547 $638 1,063,644 1,249 5,457,050 9,679,066 194,301 $70.200.576
• Includes 10% added for estimated federal expenditures on all aspects of health.
were compared to the actual findings indicated by the barium enema examination. To date 1,014 such records have been analyzed in detail, and some information is available on all of the 16,594 patients who had barium enemas in Manitoba during 1974.
(TABLE I).
The study is based upon the principles of the American College of Radiology Efficacy Study (2) with modifications to utilize the data available in Manitoba. A diagnostic radiological examination is considered efficacious if it alters the physician's diagnostic thinking, if patient management is influenced by the radiological information, and if the patient is better-off following the performance of the study. Colonic disease offered an opportunity to study the use of the barium enema. The clinical use of the examination is well understood and there is very slight risk or discomfort for the patient. Medical practitioners, through the Manitoba Medical Association and the College of Physicians and Surgeons of Manitoba, supported the study and made available their records on patients who had barium enema studies in 1974. An efficacy analysis was done in two private clinics and one hospital; predictions about the expected findings
Data from the Manitoba Health Services Commission
Residents of Manitoba use the provincial medical and hospitalization services that are provided at no direct cost. The physicians all participate in the plan with a government-approved, physician-negotiated tariff. Each claim for ambulatory service includes the reason for the patient's visit; the separation report contains detailed medical and surgical information. In 1974, 10,024 barium enemas were administered in private radiology offices, 3,305 in urban hospitals, and 3,265 in rural health units-95 % were standard, 4 % were air-contrast combined with standard, and 1% were air-contrast studies only. The use of barium enemas in Manitoba has been constant since 1969 (TABLE II). No accurate information about
1 From the Department of Radiology (D.W.M., J.M.T.) and the Computer Department for Health Sciences (P.C.), The University of Manitoba, and the Manitoba Health Services Commission (S.K.), Winnipeg, Manitoba, Canada. Presented at the Sixty-Second Scientific Assembly and Annual Meeting of the Radiological Society of North America, Chicago, III., Nov. 14-19, 1976. Supported in part by grants of the Manitoba Medical Service Foundation, Inc., Health and Welfare Canada grant 607 -1013-21, and the Efficacy Committee of the American College of Radiology under grant 01546. wjw
39
DOUGLAS W. MACEwAN AND OTHERS
40
Year
Barium Enemas
1969 1970 1971 1972 1973 1974 1975
15.486 16.672 16.777 16.706 16,717 16,594 17,074
Population
Per 1,OOO/Year 16 17 16 16 16 16 16
971.000 971.947 1,018,535 1,017,666 1,027,866 1,053,382 1,063,644
January 1978
alization only; 6 % underwent biopsy and 2 % had a polyp or papilloma removed. Endoscopy was performed more frequently in two teaching hospitals and two specialty clinics. 54 % of the patients examined in hospitals and 63 % of those examined in private offices were women-a function of the increased use of medical services by middle-aged women (6,7). Patient-Physician Interface TABLE III correlates the 1975 patient claims and hospital separation data (coded by the International Classification of Diseases) with that information calling for a barium enema examination. Medical private practice (3 digit coding) lists the actual number of patients, while the hospital in-patient separations (4 digit coding) cite separate admissions, which were multiple for some patients. The International Classification of Diseases often combines the small and large intestines, while categories such as abdominal pain involve many diseases. The information is useful for problems the physician faces in establishing a working diagnosis for the care of his patient.
20
16 CI)
I-
Z
w
j::: ~
a. u.
0 I-
10
Z
w 0 a: w a.
Colonic Disease in Manitoba
6
o 10
20
30
40
50
60
70
80
90
100
YEARS
Fig. 1.
Age distribution of radiology office patients.
use of the barium enema in other countries was found (5), but discussion with knowledgeable individuals suggests similar or lower rates than the Manitoba experience. Although the use of the barium enema is stable, the overall use of radiology services has grown steadily in Manitoba over the past decade (3). In 1965, there were 520 x-ray examinations per 1,000 population per year; this increased to 890 by 1975. In 1975, the United States averaged 700 radiology examinations per 1,000 population (4). The age distribution of the 10,024 patients undergoing barium enema examinations in private offices is shown in Figure 1. The relationship of proctosigmoidoscopy and barium enema studies was obtained by an analysis of the computer printout for the last two months of 1974. The overlap of barium enema and proctosigmoidoscopy was 10%; 3 % of the patients in each group required repeat studies. About 10% more proctosigmoidoscopies were performed than barium enema examinations, and 92 % of proctosigmoidoscopic studies were performed for visu-
A measure of the actual incidence of colonic disease is obtained from three sources: (a) specific colonic disease identification in the claims and separation data of the Manitoba Health Services Commission (TABLE IV); (b) the 1974 report of the Manitoba Cancer Treatment and Research Foundation, which recorded 272 tumors of the large intestine (excluding the rectum) and 186 cases of colon cancer as the primary cause of death (10); and (c) a previous clinical study suggesting that 300 new cases of colon cancer are discovered each year in Manitoba (11). The three sources are similar, and the incidence and mortality rates from cancer of the colon are similar to those reported in other parts of Canada. The hospitals and health units report primary, secondary, and tertiary discharge diagnoses on all patient separations to the Commission, usually with pathological and operative proof. The information in TABLE IV is based upon search of claims data for patients who underwent total colectomy, partial colectomy, surgical reduction of volvulus, surgical treatment of intussusception, and surgical intervention for diverticular disease. All of the information is accurate, with pathological proof, except for the total number of polyps and papillomata, which has been closely estimated by pathologists who received specimens from patients in hospitals and in private facilities. The polyp(s) or papilloma(s) are arbitrarily divided into two groups based on size. It is reasonable to expect radiologists to discover polyps over 0.5 cm in size. The information is not complete as some of the unanalyzed surgical codes contain information on both IClrge and small intestines. In all, approximately 1,000 barium enema examinations are performed each year that reveal abnormalities important for the referring physician.
Vol. 126
Diagnostic Radiology
41
MANITOBA BARIUM ENEMA EFFICACY STUDY
TABLE III: PRIVATE PRACTICE. MEDICAL HOSPITAL CARE. AND SURGICAL HOSPITAL CARE
International Classification of Diseases 006 008 009 014 153 211 280 285 285.9 444.2 455 560.0 560.2 560.3 560.4 560 561 562 562.1 563 563.0 563.1 563.9 564 564.0 564.1 565 566 567 568 569 569.3 751 751.1 751.2 785 785.5 785.6 785.7
Amebiasis Enteritis due to other specified organism Diarrheal disease Tuberculosis of the intestines, peritoneum, and mesenteric glands Malignant neoplasm of the large intestine, except the rectum Benign neoplasm of other parts of the digestive system Iron-deficiency anemia Other unspecified anemia Anemia, unspecified Arterial embolism and thrombosis of the mesenteric artery Hemorrhoids Intussusception Volvulus Impaction of the intestine Adhesions with obstruction Intestinal obstruction without mention of hernia Gastroenteritis and colitis, except ulcerative, of noninfectious origin Diverticula of the intestine Colon diverticula Chronic enteritis and ulcerative colitis Regional enteritis Ulcerative colitis Other Functional disorder of the intestines Constipation Irritable colon Anal fissure and fistula Abscess of the anal and rectal regions Peritonitis Peritoneal adhesions Other disease of the peritoneum and intestines Fistula of the intestine Other congenital anomalies of the digestive system Anomalies of intestinal fixation Hirschsprung's disease Symptoms referable to the abdomen and lower gastrointestinal tract Abdominal pain Incontinence of feces Melena (not of newborn)
Male
12 275 14,420 1 611 393 575 4,622
24 297 16,444 14 529 399 1,791 12,573
36 572 30,864 15 1,140 792 2,366 17,195
$921 6,156 497,483 496 109,427 42,836 40,312 349,304
3,911
3,980
7,891
209,607
735 75 927
880 61 1,409
1,615 136 2,336
95,980 2,653 75,110
667
529
1,140
109,427
4,084
6,801
10,885
1,324 213 108 116 5,789
2,582 676 194 166 10,787
68,147 20,866 11,243 6,072 460,532
106
63
169
11,022
18,816
31,761
50,577
1,070,156
TABLE IV:
1 123 1,663 0 73 0 57
0 129 1,732
56
82
30 9 3 30
25 2 1 35
3
3
119
188
32 39 28
37 34 18
88 16 12 28 7 0 158 2 17 4 1 831 762 9 16
75 46
2 89 0 61
0 3 60 1 155 0 10
0 4 74 1 185 0 29
13 9 334 7 18 4 34
21 11 326 3 19 7 63 2
89 64 27 34
79 82 43 36
3
3
8 9 151 131 19 12 119 6 42 1 6 131 111 1 6
9 11 96 45 10 39 127 10 20 0
169,153
1,258 463 86 50 4,998
The Diagnostic Process (Efficacy Study) Insight into the individual patient problem and the physician's method of solution was sought through a questionnaire (TABLE V). Symptoms, signs, correlation with endoscopy, the physician's expected diagnosis, the x-ray examination result, the physican's new diagnostic impression, the final diagnosis, and the exhibitors' evaluation of the x-ray study were detailed. Analysis was performed by a computer at the Faculty of Medicine. The purpose of collecting the clinical attributes and the physician's predictions was to study the efficacy of patient referral for the barium enema examination. Of the 1,014 patients studied in detail, 119 pediatric and 458 adult patients were from teaching hospital I; 84 were from teaching hospital II; 258 were from private clinic I (40 M.D.s); and 95 were from private clinic II (60 M.D.s). All patients were selected at random except at teaching hospital II, where those with proved colonic abnormalities were selected to increase the data base of abnormal patients. Efficacy predictions were not made in teaching hospital II nor in the pediatric cases. The final diagnosis was established 6-15 months after the barium enema by chart review and an occasional physician interview. In evaluating the efficacy of radiological procedures, the symptoms and signs were defined
Medical Private Practice Total Female Fees
Hospital In-Patient Separations (i.e., Admissions) Without Surgery With Surgery Male Female Male Female
8 10 7 10 120 3 10 1 0 1,292 1,241 5 11
a
189 174 1 2
1974 DISEASE PROVED BY SURGICAL PROCEDURE
Carcinoma of the colon and rectosigmoid junction Diverticular disease Regional enteritis Volvulus Ulcerative colitis Intussusception Adhesions Embolus or thrombus Fistula Hirschsprung's disease Perforation Polyp(s) or papilloma(s) 0.5 cm or greater
223 44 21 19 10 9 7 5 3 1 1 1 500'
Total Polyp(s) or papilloma(s) less than 0.5 cm
844 600'
O~M
• These figures were estimated by our consultants in pathology, based on specimens from both hospitals and private facilities.
as clinical attributes. These were weighted for the 1,014 patients (Table VI). However, these risk factors cannot be used to accurately predict the likelihood of an abnormal barium enema examination. Evaluation of the performance of the physician in anticipating the likelihood of disease is much more revealing. Carcinoma of the colon was selected for analysis of
42
DOUGLAS
Table V:
W.
January 1978
MACEwAN AND OTHERS
Sample Data Analysis for Patient with Carcinoma of the Colon
[1] [2] [41 [9] [6] [7]
Registration Number (Manitoba Health Services Commission) Year of Birth Sex Facility (M.H.S.C. number) Clinical attributes: Was Reported Yes No -weight loss X weakness X abdominal pain X vomiting ~diarrhea X change bowel habit X previous colonic disease X blood loss X mucus per rectum X tenesmus X X__ constipation anorexia X Xother Yes
No
[11 [2]
[ 2] [21 [11 [21
Was Found
3311 3313 3315 3317
proctosigm 0 i d oscopy sigmoidoscopy with biopsy polyp or papilloma removal multiple removal
X X X X
Date of service Sigmoidoscopy:
[2] [21 [21 [ 11 [21
7077 7078 7095
X X X__ X__
Date of service Endoscopy
[ 1] [2]
palpable mass distention abnormal bowel sounds blood per rectum hemorrhoids anemia fever tender other barium enema combined with air contrast air contrast only
X
Barium enema
[2] [1] [ 1] [1] [2] [31 [1]
[ 11
[2] [2]
[21 [2] [ 11 [ 2] [ 11 [21 [7] [0] [7\ (7]
[1] [0] [01 [4] [7] [4]
1. 2. 3. 4. 5. 6.
normal tumor polyp colitis other not done Most likely diagnosis: before and percentage X-ray results Most likely diagnosis: after percentage Final diagnosis (obtained 6-15 months after enema) X-ray contribution significant (two radiologists' opinion) 1. Yes 2. No
relative risk, on patients over 30 years of age. In all, 312 out of 1,014 patients were suspected of having carcinoma of the colon at the time the barium enema was requested, and 91 out of the 1,014 patients were found to have carcinoma of the colon, with pathological proof in all but 2. Predictions of the likelihood of colonic disease before and after the performance of the barium enema was made on 138 of these patients, as they were in the hospital; clinic and office patients had prior predictions only, as they could not be followed so closely before the physician would take other medical actions (TABLE VII). 38/1,014 barium enemas were performed for exclusion of carcinoma of the colon when the physician did not believe that a carcinoma would be found; on follow up all 38 patients were normal.
[3] [3] [11 [1]
[0] [9] [0] [4] [7] [4] [ I]
[8] [I] [4] [0] [21 [5] [8] [1] [4] [0]
[8] [1] [41 [0] [9] [9] [8] [I] [41 [0] [1 ]
The Pediatric Patient 119/1,014 patients were 16 years of age or younger. Prior predictions were obtained on few of them, and a retrospective analysis was performed (TABLE VIII). Nineteen of the patients were referred from outside physicians and have been omitted from the retrospective study, as detailed hospital records were not available. In all, an estimated 400 pediatric barium enema studies were performed in 1974 in Manitoba. To reduce the radiation dose to children, approximately 25 % of the barium enema examinations could be postponed, and probably avoided. The fact that carcinoma of the colon does not exist in this age group allows the referring physician and
consultant radiologist to be conservative in the use of the barium enema; for example, in the Hospital for Sick Children in Toronto, 722 barium enema examinations were performed over a five-year period in children with abdominal pain. Dr. W. A. Cumming found that all were normal. ANAL YSIS OF THE EFFICACY STUDY
At present there is no precise mechanism for predicting whether barium enema examinations should be performed; nevertheless, some interesting observations can be made: (a) Whenever the physician performed a barium enema examination to rule out carcinoma of the colon, which was believed to be nonexistent, he was always correct and no carcinoma was found; this was also true in 14 patients suspected of other abnormalities. (b) The greater the physician's concern that a lesion was present, the greater the likelihood that it would be found. (c) The physician had a high regard for the accuracy of the barium enema, and upon receipt of a normal report, he usually stated that there was no longer any likelihood of the suspected colonic lesion being present. His subsequent medical decisions, on chart review, confirmed this impression. (d) Review of the charts and discussion with physicians suggests that a precise differential diagnosis is not formulated early, but the barium enema and laboratory investigations are used to discover clues that will eventually establish the final diagnosis. (e) When there are no symptoms or signs related to colonic disease, there is little likelihood that colonic disease will be discovered. The only important exception was one patient who had no symptoms or signs related to the colon, who was not suspected of having colonic disease, but serendipitously had a barium enema and was discovered to have carcinoma of the colon. (f) The physician's expertise is very important, and clinical skill predicted the discovery of rare colonic abnormalities or colonic disease in unusual disguises. (g) Carcinoma of the colon emerged as the major clinical concern in the adult. Without the need to discover this abnormality early, it would be possible to delay, or not even perform, barium enema examinations on many patients. DISCUSSION
This investigation was prompted by an interest in the quality of medical care and the effective use of costly medical resources. Brook outlines the complexities of such efforts, but emphasizes the need to identify problems and provide mechanisms to overcome them (12). The study revealed approximately 100,000 patient-physician interactions that might have been the result of colonic disease, 16,594 barium enema consultations, approximately 1,000 colonic abnormalities discovered by the barium enema examination that are important to the referring physician, and an efficacy analysis of the physician's performance showing that he is unusually perceptive and reliable in his clinical assessment. The education of
Diagnostic Radiology
43
MANITOBA BARIUM ENEMA EFFICACY STUDY
Vol. 126
TABLE
VI:
EFFICACY PATIENTS
A %
Patients
Final Diagnosis
1
71.4 9.5 4.5 2.5 2.2 1.9 1.4 1.2 1.0 0.6 0.4 0.4 0.4 0.4 0.3 0.2 0.2 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1
1,014
100.0
Normal Adenocarcinoma Diverticulosis (significant)' Polyp Diverticulitis Ulcerative colitis Intussusception Regional enteritis Fistula Unknown Agangliosis Malrotation Stricture Necrotizing inflammation Abnormal Appendiceal abscess Imperforate anus Strangulation due to hernia Radiation injury Perforating wound Developmental malformation Abnormal shortening Cholecystitis Fecal impaction Cyst of colon Obstruction Appendicitis-acute Adhesion Necrosis Squamous-cell carcinoma Lymphosarcoma Lymphoma
726 96 46 25 22 19 14 12 10 6 4 4 4 4 3 2 2
1 1 1 1 1 1 1
1 1 1 1 1 1
1
• Diverticulosis was not listed as an abnormality unless the physician considered it a cause of symptoms. 37% of the adult patients were found to have diverticula on x-ray study.
TABLE
VII:
DETAILED DATA ON
138
PATIENTS SUSPECTED OF CARCINOMA
Normal Colon
Patients
1-25% 26-50 % 51-75% 76-99 %
likelihood of likelihood of likelihood of likelihood of
tumor tumor tumor tumor
TABLE
Group I:
VIII:
95 25 3 1
3 5 2 4
138
124
14
PEDIATRIC PATIENTS
Important Medical Abnormalities Requiring Barium Enema Normal Abnormal
Intussusception Hirschsprung's disease Necrotizing enterocolitis Other forms of colitis Polyp Purpura Group II:
Carcinoma
98 30 5 5
14
10
7
2
3 3 2 2
4 4
o o
Patients with Conditions that are Known to Yield Poor Information with Barium Enema in Childhood
Abdominal pain Constipation Blood loss (with or without anemia) Abdominal mass Rectal polyp or fissure Anxiety Failure to thrive Alcoholic child Influenza "Worried mother" No symptoms or signs
14 9 10 5 3 2
2 1 1 1 1
o o o o o o o o o o o
44
DOUGLAS W. MACEwAN AND OTHERS
medical students and the attitude of physicians is parallel to that prevailing in other parts of Canada and the United States with the following exceptions: there are no executive health programs or systematic searches for colonic disease in presumably healthy patients. In Canada, severe restraints on the budgets for health, education, and social services are directly affecting hospitals, payments for medical services, and university operating grants. Changes in the legislation concerning federal contributions to post-secondary education, hospital insurance, and medicare emerged in April, 1977 under a program entitled "Established Programs Finance." The result is a tax transfer of smaller sums of money to the provinces, which are responsible for education and health care. In this atmosphere of restraint it appears that some barium enema examinations need not be performed. Improved preparation of patients would result in fewer repeat barium enema examinations and proctosigmoidoscopic studies. At least 25 % of the pediatric examinations are not indicated. During the study, the radiologists estimated that 30 % of the adult barium enema examinations were unnecessary. The possibility of identifying and eliminating potentially unnecessary barium enema examinations was considered. Discussions with clinical colleagues from the facilities where the questionnaires were used and with medical leaders in the Manitoba Medical Association and the College of Physicians and Surgeons of Manitoba, who supported the study, led to the suggestion that approximately 10% of the barium enema examinations could safely be omitted by the referring physician. The cost of barium enema examinations performed in Manitoba in 1975 was $338,000. A reduction of 10% would result in a potential saving of $33,800. The data and observations suggest that a balance between patient care and other priorities of society can be obtained by the well-trained physician, aware of the broad patterns of health in his community. The claims data provide insight into the realities of present-day practice, and the efficacy study highlights the physicians' remarkable ability to direct the patient to the appropriate investigation. Addendum: A description of the American College of Radiology Efficacy Study was published after this paper was prepared. ACKNOWLEDGMENTS:
We are very grateful for many hours of help
January 1978
by R. Harvey, J. Lock, and F. Toll of the Manitoba Health Services Commission; R. l. Macpherson of the Manitoba Medical Association; Dr. J. Morison, Registrar of the College of Physicians and Surgeons of Manitoba, for securing the support of the medical profession; C. Olson, R. J. Walton, and O. Gundrum, radiology technologists; Kathleen Holmes, Edith Fairchuk, Eleanor Morton, Charlotte Stankiewich, Audrey Fehr, and Carol Rose, Medical Record Librarians; Ken McGregor and his photography staff; and to Donna Hutchinson, secretary. Dr. D. E. Gerson and Dr. A. Lewicki provided information on barium enema utilization that assisted in defining the risk factors. The Executive Committees of the Winnipeg Clinic and the Manitoba Clinic gave generous support and encouragement. Department of Radiology Health Sciences Centre 700 William Ave. Winnipeg, Manitoba Canada R3E OZ3
REFERENCES 1. Roos NP, Henteleff PD, Roos LL: A new audit procedure applied to an old question: Is the frequency of T & A justified? Med Care 15:1-18, Jan 1977 2. Lusted LB, Bell RS, Edwards W, et al: Evaluating the efficacy of radiologic procedures by Bayesian methods: a progress report. [In] Snapper K, ed: Models in Metrics For Decision Makers. Washington DC, Information Resources Press, 1977 (in press) 3. MacEwan DW, Lock J, Walton RJ: Radiology services for one million people (1974 diagnostic radiology services in the Province of Manitoba). J Canad Assoc. Radiol 28:3-11, Mar 1977 4. U.S. Department of Health, Education, and Welfare. Population exposure to x-rays U.S. 1970. A report on the Public Health Service x-ray exposure study. Washington DC, DHEW PHSPublication 73-8047, Nov 1973 5. Puijlaert CBA: The expansion of radiodiagnostics. Medicamundi 14:137-149, 1969 6. Statistical Supplement to the Annual Report of the Manitoba Health Services Commission, R3G 3H2, 1SSN03833933. Winnipeg, Manitoba, 1973 7. Cooperstock R: Psychotropic drug use among women. Can Med Assoc J 115:760-763, Oct 1976 8. Manitoba Health Services Commission, Department of Health and Social Development: Annual Report, 1975 9. Cooper BS, Worthington NL, McGee MF: Compendium of national health expenditures data. U.S. DHEW Publ. (SSA) 76-11927, Jan 1976 10. Manitoba Cancer Treatment and Research Foundation: Annual Report, 1974. 700 Bannatyne Avenue, Winnipeg, Manitoba, R3E
OV9 11. Saunders CG, MacEwan DW: Delay in diagnosis of colonic cancer-a continuing challenge. Editorial. Radiology 101:207-208, Oct 1971 12. Brook RH: Quality-can we measure it? Editorial N Engl J Med 296:170-172, Jan 1977 13. Manheimer A: ACR efficacy study: a profession examines itself. Applied RadioI6:97-102, May-Jun 1977
Manitoba Barium Enema Efficacy Study 1 Douglas W. MacEwan, M.D., Steven Kavanagh, B. A., Peter Chow, M.Sc., and Jack M. Tishler, M.D. Claims analysis and efficacy study are two new methods of evaluating medical services. In Manitoba, with a 1974 population of 1,053,382, approximately 100,000 patient-physician interfaces resulted in 16,594 barium enema examinations; 1,000 significant abnormalities were discovered, and the referring physician was unusually perceptive in predicting the likelihood of these abnormalities. It is estimated that a 10% reduction in the use of barium enemas could be achieved by referring physicians with no deterioration in the care of patients. INDEX TERMS: 7[5].1281)
Barium-enema examination • Colon, neoplasms. (Colon, routine exam,
Radiology 126:39-44, January 1978 TABLE
w o METHODS of gaining insight into physician performance are analysis of physicians' claims (1) and radiology efficacy studies (2). In Manitoba, the barium enema examination was studied in a closed population, where in-depth medical information and complete financial data were available. The referring physician emerged as essential to optimal cost-effective patient care.
T
I: OVERALL HEALTH PLAN DATA 1975
Total Medicare and hospital expenditures (8) Manitoba Department of Health and Social Development Manitoba Pharmicare plan Budget of the Faculty of Medicine Manitoba per capita medical costs" United States per capita medical costs (9) (estimate for 1976) Population Total practitioners Number of claims Number of services to patients Number of separations Value of professional services to patients
METHOD OF STUDY
Analysis of physicians' claims is particularly appropriate in Manitoba, as the government health plan covers all hospital and ambulatory care for the entire population. Physicians are paid a fee based on claims for patient service. Extensive reliable medical information can be assembled by analysis of the complete computer records used for all physicians' claims and all hospital separations
$330.812,659 $285,094,000 $32.000,000 $6,958,019 $679 $547 $638 1,063,644 1,249 5,457,050 9,679,066 194,301 $70.200.576
• Includes 10% added for estimated federal expenditures on all aspects of health.
were compared to the actual findings indicated by the barium enema examination. To date 1,014 such records have been analyzed in detail, and some information is available on all of the 16,594 patients who had barium enemas in Manitoba during 1974.
(TABLE I).
The study is based upon the principles of the American College of Radiology Efficacy Study (2) with modifications to utilize the data available in Manitoba. A diagnostic radiological examination is considered efficacious if it alters the physician's diagnostic thinking, if patient management is influenced by the radiological information, and if the patient is better-off following the performance of the study. Colonic disease offered an opportunity to study the use of the barium enema. The clinical use of the examination is well understood and there is very slight risk or discomfort for the patient. Medical practitioners, through the Manitoba Medical Association and the College of Physicians and Surgeons of Manitoba, supported the study and made available their records on patients who had barium enema studies in 1974. An efficacy analysis was done in two private clinics and one hospital; predictions about the expected findings
Data from the Manitoba Health Services Commission
Residents of Manitoba use the provincial medical and hospitalization services that are provided at no direct cost. The physicians all participate in the plan with a government-approved, physician-negotiated tariff. Each claim for ambulatory service includes the reason for the patient's visit; the separation report contains detailed medical and surgical information. In 1974, 10,024 barium enemas were administered in private radiology offices, 3,305 in urban hospitals, and 3,265 in rural health units-95 % were standard, 4 % were air-contrast combined with standard, and 1% were air-contrast studies only. The use of barium enemas in Manitoba has been constant since 1969 (TABLE II). No accurate information about
1 From the Department of Radiology (D.W.M., J.M.T.) and the Computer Department for Health Sciences (P.C.), The University of Manitoba, and the Manitoba Health Services Commission (S.K.), Winnipeg, Manitoba, Canada. Presented at the Sixty-Second Scientific Assembly and Annual Meeting of the Radiological Society of North America, Chicago, III., Nov. 14-19, 1976. Supported in part by grants of the Manitoba Medical Service Foundation, Inc., Health and Welfare Canada grant 607 -1013-21, and the Efficacy Committee of the American College of Radiology under grant 01546. wjw
39
DOUGLAS W. MACEwAN AND OTHERS
40
Year
Barium Enemas
1969 1970 1971 1972 1973 1974 1975
15.486 16.672 16.777 16.706 16,717 16,594 17,074
Population
Per 1,OOO/Year 16 17 16 16 16 16 16
971.000 971.947 1,018,535 1,017,666 1,027,866 1,053,382 1,063,644
January 1978
alization only; 6 % underwent biopsy and 2 % had a polyp or papilloma removed. Endoscopy was performed more frequently in two teaching hospitals and two specialty clinics. 54 % of the patients examined in hospitals and 63 % of those examined in private offices were women-a function of the increased use of medical services by middle-aged women (6,7). Patient-Physician Interface TABLE III correlates the 1975 patient claims and hospital separation data (coded by the International Classification of Diseases) with that information calling for a barium enema examination. Medical private practice (3 digit coding) lists the actual number of patients, while the hospital in-patient separations (4 digit coding) cite separate admissions, which were multiple for some patients. The International Classification of Diseases often combines the small and large intestines, while categories such as abdominal pain involve many diseases. The information is useful for problems the physician faces in establishing a working diagnosis for the care of his patient.
20
16 CI)
I-
Z
w
j::: ~
a. u.
0 I-
10
Z
w 0 a: w a.
Colonic Disease in Manitoba
6
o 10
20
30
40
50
60
70
80
90
100
YEARS
Fig. 1.
Age distribution of radiology office patients.
use of the barium enema in other countries was found (5), but discussion with knowledgeable individuals suggests similar or lower rates than the Manitoba experience. Although the use of the barium enema is stable, the overall use of radiology services has grown steadily in Manitoba over the past decade (3). In 1965, there were 520 x-ray examinations per 1,000 population per year; this increased to 890 by 1975. In 1975, the United States averaged 700 radiology examinations per 1,000 population (4). The age distribution of the 10,024 patients undergoing barium enema examinations in private offices is shown in Figure 1. The relationship of proctosigmoidoscopy and barium enema studies was obtained by an analysis of the computer printout for the last two months of 1974. The overlap of barium enema and proctosigmoidoscopy was 10%; 3 % of the patients in each group required repeat studies. About 10% more proctosigmoidoscopies were performed than barium enema examinations, and 92 % of proctosigmoidoscopic studies were performed for visu-
A measure of the actual incidence of colonic disease is obtained from three sources: (a) specific colonic disease identification in the claims and separation data of the Manitoba Health Services Commission (TABLE IV); (b) the 1974 report of the Manitoba Cancer Treatment and Research Foundation, which recorded 272 tumors of the large intestine (excluding the rectum) and 186 cases of colon cancer as the primary cause of death (10); and (c) a previous clinical study suggesting that 300 new cases of colon cancer are discovered each year in Manitoba (11). The three sources are similar, and the incidence and mortality rates from cancer of the colon are similar to those reported in other parts of Canada. The hospitals and health units report primary, secondary, and tertiary discharge diagnoses on all patient separations to the Commission, usually with pathological and operative proof. The information in TABLE IV is based upon search of claims data for patients who underwent total colectomy, partial colectomy, surgical reduction of volvulus, surgical treatment of intussusception, and surgical intervention for diverticular disease. All of the information is accurate, with pathological proof, except for the total number of polyps and papillomata, which has been closely estimated by pathologists who received specimens from patients in hospitals and in private facilities. The polyp(s) or papilloma(s) are arbitrarily divided into two groups based on size. It is reasonable to expect radiologists to discover polyps over 0.5 cm in size. The information is not complete as some of the unanalyzed surgical codes contain information on both IClrge and small intestines. In all, approximately 1,000 barium enema examinations are performed each year that reveal abnormalities important for the referring physician.
Vol. 126
Diagnostic Radiology
41
MANITOBA BARIUM ENEMA EFFICACY STUDY
TABLE III: PRIVATE PRACTICE. MEDICAL HOSPITAL CARE. AND SURGICAL HOSPITAL CARE
International Classification of Diseases 006 008 009 014 153 211 280 285 285.9 444.2 455 560.0 560.2 560.3 560.4 560 561 562 562.1 563 563.0 563.1 563.9 564 564.0 564.1 565 566 567 568 569 569.3 751 751.1 751.2 785 785.5 785.6 785.7
Amebiasis Enteritis due to other specified organism Diarrheal disease Tuberculosis of the intestines, peritoneum, and mesenteric glands Malignant neoplasm of the large intestine, except the rectum Benign neoplasm of other parts of the digestive system Iron-deficiency anemia Other unspecified anemia Anemia, unspecified Arterial embolism and thrombosis of the mesenteric artery Hemorrhoids Intussusception Volvulus Impaction of the intestine Adhesions with obstruction Intestinal obstruction without mention of hernia Gastroenteritis and colitis, except ulcerative, of noninfectious origin Diverticula of the intestine Colon diverticula Chronic enteritis and ulcerative colitis Regional enteritis Ulcerative colitis Other Functional disorder of the intestines Constipation Irritable colon Anal fissure and fistula Abscess of the anal and rectal regions Peritonitis Peritoneal adhesions Other disease of the peritoneum and intestines Fistula of the intestine Other congenital anomalies of the digestive system Anomalies of intestinal fixation Hirschsprung's disease Symptoms referable to the abdomen and lower gastrointestinal tract Abdominal pain Incontinence of feces Melena (not of newborn)
Male
12 275 14,420 1 611 393 575 4,622
24 297 16,444 14 529 399 1,791 12,573
36 572 30,864 15 1,140 792 2,366 17,195
$921 6,156 497,483 496 109,427 42,836 40,312 349,304
3,911
3,980
7,891
209,607
735 75 927
880 61 1,409
1,615 136 2,336
95,980 2,653 75,110
667
529
1,140
109,427
4,084
6,801
10,885
1,324 213 108 116 5,789
2,582 676 194 166 10,787
68,147 20,866 11,243 6,072 460,532
106
63
169
11,022
18,816
31,761
50,577
1,070,156
TABLE IV:
1 123 1,663 0 73 0 57
0 129 1,732
56
82
30 9 3 30
25 2 1 35
3
3
119
188
32 39 28
37 34 18
88 16 12 28 7 0 158 2 17 4 1 831 762 9 16
75 46
2 89 0 61
0 3 60 1 155 0 10
0 4 74 1 185 0 29
13 9 334 7 18 4 34
21 11 326 3 19 7 63 2
89 64 27 34
79 82 43 36
3
3
8 9 151 131 19 12 119 6 42 1 6 131 111 1 6
9 11 96 45 10 39 127 10 20 0
169,153
1,258 463 86 50 4,998
The Diagnostic Process (Efficacy Study) Insight into the individual patient problem and the physician's method of solution was sought through a questionnaire (TABLE V). Symptoms, signs, correlation with endoscopy, the physician's expected diagnosis, the x-ray examination result, the physican's new diagnostic impression, the final diagnosis, and the exhibitors' evaluation of the x-ray study were detailed. Analysis was performed by a computer at the Faculty of Medicine. The purpose of collecting the clinical attributes and the physician's predictions was to study the efficacy of patient referral for the barium enema examination. Of the 1,014 patients studied in detail, 119 pediatric and 458 adult patients were from teaching hospital I; 84 were from teaching hospital II; 258 were from private clinic I (40 M.D.s); and 95 were from private clinic II (60 M.D.s). All patients were selected at random except at teaching hospital II, where those with proved colonic abnormalities were selected to increase the data base of abnormal patients. Efficacy predictions were not made in teaching hospital II nor in the pediatric cases. The final diagnosis was established 6-15 months after the barium enema by chart review and an occasional physician interview. In evaluating the efficacy of radiological procedures, the symptoms and signs were defined
Medical Private Practice Total Female Fees
Hospital In-Patient Separations (i.e., Admissions) Without Surgery With Surgery Male Female Male Female
8 10 7 10 120 3 10 1 0 1,292 1,241 5 11
a
189 174 1 2
1974 DISEASE PROVED BY SURGICAL PROCEDURE
Carcinoma of the colon and rectosigmoid junction Diverticular disease Regional enteritis Volvulus Ulcerative colitis Intussusception Adhesions Embolus or thrombus Fistula Hirschsprung's disease Perforation Polyp(s) or papilloma(s) 0.5 cm or greater
223 44 21 19 10 9 7 5 3 1 1 1 500'
Total Polyp(s) or papilloma(s) less than 0.5 cm
844 600'
O~M
• These figures were estimated by our consultants in pathology, based on specimens from both hospitals and private facilities.
as clinical attributes. These were weighted for the 1,014 patients (Table VI). However, these risk factors cannot be used to accurately predict the likelihood of an abnormal barium enema examination. Evaluation of the performance of the physician in anticipating the likelihood of disease is much more revealing. Carcinoma of the colon was selected for analysis of
42
DOUGLAS
Table V:
W.
January 1978
MACEwAN AND OTHERS
Sample Data Analysis for Patient with Carcinoma of the Colon
[1] [2] [41 [9] [6] [7]
Registration Number (Manitoba Health Services Commission) Year of Birth Sex Facility (M.H.S.C. number) Clinical attributes: Was Reported Yes No -weight loss X weakness X abdominal pain X vomiting ~diarrhea X change bowel habit X previous colonic disease X blood loss X mucus per rectum X tenesmus X X__ constipation anorexia X Xother Yes
No
[11 [2]
[ 2] [21 [11 [21
Was Found
3311 3313 3315 3317
proctosigm 0 i d oscopy sigmoidoscopy with biopsy polyp or papilloma removal multiple removal
X X X X
Date of service Sigmoidoscopy:
[2] [21 [21 [ 11 [21
7077 7078 7095
X X X__ X__
Date of service Endoscopy
[ 1] [2]
palpable mass distention abnormal bowel sounds blood per rectum hemorrhoids anemia fever tender other barium enema combined with air contrast air contrast only
X
Barium enema
[2] [1] [ 1] [1] [2] [31 [1]
[ 11
[2] [2]
[21 [2] [ 11 [ 2] [ 11 [21 [7] [0] [7\ (7]
[1] [0] [01 [4] [7] [4]
1. 2. 3. 4. 5. 6.
normal tumor polyp colitis other not done Most likely diagnosis: before and percentage X-ray results Most likely diagnosis: after percentage Final diagnosis (obtained 6-15 months after enema) X-ray contribution significant (two radiologists' opinion) 1. Yes 2. No
relative risk, on patients over 30 years of age. In all, 312 out of 1,014 patients were suspected of having carcinoma of the colon at the time the barium enema was requested, and 91 out of the 1,014 patients were found to have carcinoma of the colon, with pathological proof in all but 2. Predictions of the likelihood of colonic disease before and after the performance of the barium enema was made on 138 of these patients, as they were in the hospital; clinic and office patients had prior predictions only, as they could not be followed so closely before the physician would take other medical actions (TABLE VII). 38/1,014 barium enemas were performed for exclusion of carcinoma of the colon when the physician did not believe that a carcinoma would be found; on follow up all 38 patients were normal.
[3] [3] [11 [1]
[0] [9] [0] [4] [7] [4] [ I]
[8] [I] [4] [0] [21 [5] [8] [1] [4] [0]
[8] [1] [41 [0] [9] [9] [8] [I] [41 [0] [1 ]
The Pediatric Patient 119/1,014 patients were 16 years of age or younger. Prior predictions were obtained on few of them, and a retrospective analysis was performed (TABLE VIII). Nineteen of the patients were referred from outside physicians and have been omitted from the retrospective study, as detailed hospital records were not available. In all, an estimated 400 pediatric barium enema studies were performed in 1974 in Manitoba. To reduce the radiation dose to children, approximately 25 % of the barium enema examinations could be postponed, and probably avoided. The fact that carcinoma of the colon does not exist in this age group allows the referring physician and
consultant radiologist to be conservative in the use of the barium enema; for example, in the Hospital for Sick Children in Toronto, 722 barium enema examinations were performed over a five-year period in children with abdominal pain. Dr. W. A. Cumming found that all were normal. ANAL YSIS OF THE EFFICACY STUDY
At present there is no precise mechanism for predicting whether barium enema examinations should be performed; nevertheless, some interesting observations can be made: (a) Whenever the physician performed a barium enema examination to rule out carcinoma of the colon, which was believed to be nonexistent, he was always correct and no carcinoma was found; this was also true in 14 patients suspected of other abnormalities. (b) The greater the physician's concern that a lesion was present, the greater the likelihood that it would be found. (c) The physician had a high regard for the accuracy of the barium enema, and upon receipt of a normal report, he usually stated that there was no longer any likelihood of the suspected colonic lesion being present. His subsequent medical decisions, on chart review, confirmed this impression. (d) Review of the charts and discussion with physicians suggests that a precise differential diagnosis is not formulated early, but the barium enema and laboratory investigations are used to discover clues that will eventually establish the final diagnosis. (e) When there are no symptoms or signs related to colonic disease, there is little likelihood that colonic disease will be discovered. The only important exception was one patient who had no symptoms or signs related to the colon, who was not suspected of having colonic disease, but serendipitously had a barium enema and was discovered to have carcinoma of the colon. (f) The physician's expertise is very important, and clinical skill predicted the discovery of rare colonic abnormalities or colonic disease in unusual disguises. (g) Carcinoma of the colon emerged as the major clinical concern in the adult. Without the need to discover this abnormality early, it would be possible to delay, or not even perform, barium enema examinations on many patients. DISCUSSION
This investigation was prompted by an interest in the quality of medical care and the effective use of costly medical resources. Brook outlines the complexities of such efforts, but emphasizes the need to identify problems and provide mechanisms to overcome them (12). The study revealed approximately 100,000 patient-physician interactions that might have been the result of colonic disease, 16,594 barium enema consultations, approximately 1,000 colonic abnormalities discovered by the barium enema examination that are important to the referring physician, and an efficacy analysis of the physician's performance showing that he is unusually perceptive and reliable in his clinical assessment. The education of
Diagnostic Radiology
43
MANITOBA BARIUM ENEMA EFFICACY STUDY
Vol. 126
TABLE
VI:
EFFICACY PATIENTS
A %
Patients
Final Diagnosis
1
71.4 9.5 4.5 2.5 2.2 1.9 1.4 1.2 1.0 0.6 0.4 0.4 0.4 0.4 0.3 0.2 0.2 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1
1,014
100.0
Normal Adenocarcinoma Diverticulosis (significant)' Polyp Diverticulitis Ulcerative colitis Intussusception Regional enteritis Fistula Unknown Agangliosis Malrotation Stricture Necrotizing inflammation Abnormal Appendiceal abscess Imperforate anus Strangulation due to hernia Radiation injury Perforating wound Developmental malformation Abnormal shortening Cholecystitis Fecal impaction Cyst of colon Obstruction Appendicitis-acute Adhesion Necrosis Squamous-cell carcinoma Lymphosarcoma Lymphoma
726 96 46 25 22 19 14 12 10 6 4 4 4 4 3 2 2
1 1 1 1 1 1 1
1 1 1 1 1 1
1
• Diverticulosis was not listed as an abnormality unless the physician considered it a cause of symptoms. 37% of the adult patients were found to have diverticula on x-ray study.
TABLE
VII:
DETAILED DATA ON
138
PATIENTS SUSPECTED OF CARCINOMA
Normal Colon
Patients
1-25% 26-50 % 51-75% 76-99 %
likelihood of likelihood of likelihood of likelihood of
tumor tumor tumor tumor
TABLE
Group I:
VIII:
95 25 3 1
3 5 2 4
138
124
14
PEDIATRIC PATIENTS
Important Medical Abnormalities Requiring Barium Enema Normal Abnormal
Intussusception Hirschsprung's disease Necrotizing enterocolitis Other forms of colitis Polyp Purpura Group II:
Carcinoma
98 30 5 5
14
10
7
2
3 3 2 2
4 4
o o
Patients with Conditions that are Known to Yield Poor Information with Barium Enema in Childhood
Abdominal pain Constipation Blood loss (with or without anemia) Abdominal mass Rectal polyp or fissure Anxiety Failure to thrive Alcoholic child Influenza "Worried mother" No symptoms or signs
14 9 10 5 3 2
2 1 1 1 1
o o o o o o o o o o o
44
DOUGLAS W. MACEwAN AND OTHERS
medical students and the attitude of physicians is parallel to that prevailing in other parts of Canada and the United States with the following exceptions: there are no executive health programs or systematic searches for colonic disease in presumably healthy patients. In Canada, severe restraints on the budgets for health, education, and social services are directly affecting hospitals, payments for medical services, and university operating grants. Changes in the legislation concerning federal contributions to post-secondary education, hospital insurance, and medicare emerged in April, 1977 under a program entitled "Established Programs Finance." The result is a tax transfer of smaller sums of money to the provinces, which are responsible for education and health care. In this atmosphere of restraint it appears that some barium enema examinations need not be performed. Improved preparation of patients would result in fewer repeat barium enema examinations and proctosigmoidoscopic studies. At least 25 % of the pediatric examinations are not indicated. During the study, the radiologists estimated that 30 % of the adult barium enema examinations were unnecessary. The possibility of identifying and eliminating potentially unnecessary barium enema examinations was considered. Discussions with clinical colleagues from the facilities where the questionnaires were used and with medical leaders in the Manitoba Medical Association and the College of Physicians and Surgeons of Manitoba, who supported the study, led to the suggestion that approximately 10% of the barium enema examinations could safely be omitted by the referring physician. The cost of barium enema examinations performed in Manitoba in 1975 was $338,000. A reduction of 10% would result in a potential saving of $33,800. The data and observations suggest that a balance between patient care and other priorities of society can be obtained by the well-trained physician, aware of the broad patterns of health in his community. The claims data provide insight into the realities of present-day practice, and the efficacy study highlights the physicians' remarkable ability to direct the patient to the appropriate investigation. Addendum: A description of the American College of Radiology Efficacy Study was published after this paper was prepared. ACKNOWLEDGMENTS:
We are very grateful for many hours of help
January 1978
by R. Harvey, J. Lock, and F. Toll of the Manitoba Health Services Commission; R. l. Macpherson of the Manitoba Medical Association; Dr. J. Morison, Registrar of the College of Physicians and Surgeons of Manitoba, for securing the support of the medical profession; C. Olson, R. J. Walton, and O. Gundrum, radiology technologists; Kathleen Holmes, Edith Fairchuk, Eleanor Morton, Charlotte Stankiewich, Audrey Fehr, and Carol Rose, Medical Record Librarians; Ken McGregor and his photography staff; and to Donna Hutchinson, secretary. Dr. D. E. Gerson and Dr. A. Lewicki provided information on barium enema utilization that assisted in defining the risk factors. The Executive Committees of the Winnipeg Clinic and the Manitoba Clinic gave generous support and encouragement. Department of Radiology Health Sciences Centre 700 William Ave. Winnipeg, Manitoba Canada R3E OZ3
REFERENCES 1. Roos NP, Henteleff PD, Roos LL: A new audit procedure applied to an old question: Is the frequency of T & A justified? Med Care 15:1-18, Jan 1977 2. Lusted LB, Bell RS, Edwards W, et al: Evaluating the efficacy of radiologic procedures by Bayesian methods: a progress report. [In] Snapper K, ed: Models in Metrics For Decision Makers. Washington DC, Information Resources Press, 1977 (in press) 3. MacEwan DW, Lock J, Walton RJ: Radiology services for one million people (1974 diagnostic radiology services in the Province of Manitoba). J Canad Assoc. Radiol 28:3-11, Mar 1977 4. U.S. Department of Health, Education, and Welfare. Population exposure to x-rays U.S. 1970. A report on the Public Health Service x-ray exposure study. Washington DC, DHEW PHSPublication 73-8047, Nov 1973 5. Puijlaert CBA: The expansion of radiodiagnostics. Medicamundi 14:137-149, 1969 6. Statistical Supplement to the Annual Report of the Manitoba Health Services Commission, R3G 3H2, 1SSN03833933. Winnipeg, Manitoba, 1973 7. Cooperstock R: Psychotropic drug use among women. Can Med Assoc J 115:760-763, Oct 1976 8. Manitoba Health Services Commission, Department of Health and Social Development: Annual Report, 1975 9. Cooper BS, Worthington NL, McGee MF: Compendium of national health expenditures data. U.S. DHEW Publ. (SSA) 76-11927, Jan 1976 10. Manitoba Cancer Treatment and Research Foundation: Annual Report, 1974. 700 Bannatyne Avenue, Winnipeg, Manitoba, R3E
OV9 11. Saunders CG, MacEwan DW: Delay in diagnosis of colonic cancer-a continuing challenge. Editorial. Radiology 101:207-208, Oct 1971 12. Brook RH: Quality-can we measure it? Editorial N Engl J Med 296:170-172, Jan 1977 13. Manheimer A: ACR efficacy study: a profession examines itself. Applied RadioI6:97-102, May-Jun 1977
