FROM THE DEPARTMENT OF RADIOLOGY (DIRECTOR: PROF. P. VIRTAMA), UNIVERSITY HOSPITAL, SF-20520 TURKU, FINLAND.
COMPUTER AIDED DIAGNOSIS OF BONE TUMORS P. VIRTAMA, K. KATEVUO, P. MAKELA and E. O. MAKINEN The adoption of computer technology in clinical decision making has been relatively slow as compared to the increased use of computers in hospital administration. Computer aided diagnostic programs have been used in the analysis of bone tumors (LODWICK et coIl. 1963), gastric ulcers (WILSON et coIl. 1965) and heart disease (HALL et coIl. 1971), demonstrating that Bayes' formula of inverse probability is suitable for analysing data derived from radiographic films. Computer aided diagnosis is based on the assumption that systematic analysis of fine radiographic detail results in a more reliable diagnosis than does a more or less subjective impression. Computer aided programs can be designed to solve simple differential diagnostic problems by applying statistically significant differences and using the Bayes' formula (LUSTED 1968). Such a program may improve the diagnostic result when sufficient expertise with the clinical problem is not available (FRYBACK & THORNBURY 1976). The clinical value of a large computer aided program is still open to question. The bone tumor program (LODWICK et coli. 1963, LODWICK 1965, LODWICK & REICHERTZ 1968) offers an interesting model for exploration of the effectiveness of such a program in clinical diagnostic decision making. Material and Methods
In computer aided diagnosis, descriptive radiologic terms are transferred in digital form and analysed in the computer program; the statistical probability is computed for each possible diagnosis. Supported by a grant from the Finnish Cancer Association. Submitted for publication 9 January 1978. Acta Radiologica Diagnosis 20 (1979) Fasc. 1 A
70 Downloaded from acr.sagepub.com by guest on July 27, 2015
71
COMPUTER AIDED DIAGNOSIS OF BONE TUMORS
Table 1
True positive (TP) and false positive (FP) rates (in per cent) of observers A, E, C and D in different diagnostic groups at 90 and 60 per cent confidence levels A
B
C
Confidence level
D
TP
FP
TP
FP
TP
FP
TP
FP
Osteosarcoma (33 cases)
88 89
15 24
85 91
18 24
67 76
24 30
70 79
36 48
90 60
Ewing's sarcoma (23 cases)
87 91
13 17
91 91
17 22
70 74
26 30
74 78
35 43
90
Chondrosarcoma (26 cases)
77 85
12 19
81 88
14 15
46 58
23 31
46 50
23 31
90 60
Fibrosarcoma (44 cases)
76 81
19 24
79 86
14 19
62 67
19 24
50 62
24 29
90 60
Benign tumors (51 cases)
69 73
11
13
78 89
18 22
62 67
22 27
69 78
27 31
90 60
Mean
82
17
86
18
65
28
66
33
60
The material comprised 126 malignant and 51 benign solitary bone tumors. A short clinical history and pertinent laboratory data were available. The radiographic quality of the films varied but films with a low diagnostic information were not included. The distribution of malignant tumors appears in Table 1. The benign tumors consisted of 4 osteoid osteomas, 6 non-ossifying fibromas, 9 benign cartilaginous tumors, 3 aneurysmal bone cysts, 3 chondroblastomas, 4 chondromyxoid fibromas, 6 cysts, 8 benign giant cell tumors and 8 osteochondromas. Four radiologists (A, B, C, D) analysed the films. A and C were staff members of a university hospital, qualified specialists in radiology, but with no special expertise in bone radiology. D was a third year resident in radiology, not yet qualified specialist. B in addition to being qualified specialist in radiology had special experience in bone radiology and bone tumor diagnosis from working one year at a bone tumor center. Radiologist A used the computer aided bone tumor program. Two confidence levels of diagnostic decision making were employed, one corresponding to 90 per cent of the computer response and the other to 60 per cent. Diagnoses on a high confidence level (90 %) are usually associated with low true positive and low false positive ratios, while more liberal criteria of diagnosis lead to a greater percentage of false positive and false negative diagnoses (McNEIL et colI. 1977). Results The percentage of true positive and false positive diagnoses of each observer was calculated for each diagnosis. The benign tumors were analysed as one group (Table
Downloaded from acr.sagepub.com by guest on July 27, 2015
72
P. VIRTAMA, K. KATEVUO, P. MAKELA AND E. 0. MAKINEN
Table 2 Differences between observers A, B, C and D in true positive diagnoses of bone tumours at 90 and 60 per cent confidence levels
Observers
Confidence level
AlB
AIC
AID
BIC
BID
C/D
N N
S HS
S HS
S HS
HS HS
N N
N
~
90
60
no difference, S ~ significant difference, HS = highly significant difference
1). Inter-observer differences in diagnostic accuracy were calculated using the true positive and false positive rates (Table 2). Discussion
Osteosarcoma and Ewing's sarcoma turned out to be relatively easy diagnostic problems for the computer aided radiologist (A) and the one with experience in bone radiology (B); chondrosarcoma and benign bone tumors appeared to be more difficult for A than for B. The diagnostic accuracy in chondrosarcoma was particularly low for C and D, and in benign bone tumors for C, while D did not succeed well in diagnosing fibrosarcoma. In general, no significant difference was found between the performance of the computer aided general radiologist (A) and that of the radiologist with experience in bone radiology (B). Nor was there any significant difference between the resident radiologist (D) and general radiologist (C), both of whom performed significantly worse at the 90 per cent level than A or B and still worse at the 60 per cent level. No difference was known to exist between A and C in experience of bone tumor radiology. However, in this experiment A gained an 82 per cent true positive rate at the 90 per cent confidence level while the corresponding rate of C was only 65 per cent (Table 1). The difference in performance between A and C at both the 90 and 60 per cent confidence levels was highly significant (Table 2). The only reasonable explanation seems to be the computer aided bone tumor program used by A. The level of diagnostic accuracy in the present series is lower than that obtained by LODWICK et call. (1963). This may depend on the material selected for the analysis. The present, unselected series comprised consecutive cases from different hospitals including several difficult and atypical lesions. The present computer aided program is based on the recognition of numerous fine radiographic details which are not always easily observed on ordinary films. An improved quality of the films may raise the diagnostic accuracy. The use of special soft tissue radiographic technique, additional projections and tomography may also
Downloaded from acr.sagepub.com by guest on July 27, 2015
COMPUTER AIDED DIAGNOSIS OF BONE TUMORS
73
improve the result. On the other hand, the effective size of the data base could be increased by reducing the number of criteria processed by the present model. The diagnostic accuracy probably does not change by assuming conditional independence between discriminating criteria, to satisfy Bayes' theorem, although the criteria do exhibit a considerable amount of interdependence. The experience of those who created the present computer aided bone tumor program appears to be filed in a form that is readily understandable and can be used by relatively unexperienced radiologists. The use of the program improves diagnostic accuracy significantly and results in improved patient management and cost-saving. However, some work is still needed to improve the present computer program and to develop a generally acceptable clinical strategy in the diagnosis of bone tumors.
SUMMARY Four radiologists, three of whom having no special expertise in bone tumor radiology, analysed 177 bone tumors. One of the radiologists, using a computer aided bone tumor program, performed significantly better than the other two at a comparable level of training and was able to compete successfully with the fourth radiologist experienced in bone diagnosis. The results validate the assumption that computer aided diagnostic programs may improve the diagnostic accuracy of radiologists having limited experience with the problem at hand.
ZUSAMMENFASSUNG Vier Rontgendiagnostiker, von denen drei keine besondere Erfahrung in der Diagnostik von Knochentumoren hatten, analysierten 177 Knochentumoren. Einer von diesen Rontgendiagnostikern, f'uhrte unter Verwendung eines Computer-unterstutzten Knochentumorprogramms signifikant bessere Untersuchungen durch als die zwei anderen mit einem vergleichbaren Niveau von Training. Seine Resultate waren gleich denen des Rontgendiagnostikers, der besondere Erfahrung in der Knochendiagnostik hatte. Die Ergebnisse stiitzen die Annahme, dass Computer-unterstiitzte diagnostische Programme die Diagnostik des Rontgenologens mit begrenzter Erfahrung verbessern.
RESUME Quatre radiologistes dont trois n'avaient pas d'experience speciale dans la radiologie des tumeurs osseuses, ont etudie 177 tumeurs osseuses. L'un des radiologistes utilisant un programme de diagnostic des tumeurs osseuses par ordinateur a eu des resultats significativement meilleurs que les deux autres radiologistes qui avaient un niveau comparable de formation et a ete capable de concurrencer avec succes Ie quatrieme radiologiste qui avait une experience dans Ie diagnostic osseux. Ces resultats confirment l'hypothese que les programmes d'aide au diagnostic par ordinateur peuvent ameliorer I'exactitude du diagnostic de radiologistes ayant une experience limitee du probleme en question.
REFERENCES FRYBACK D. G. and THORNBURY J. R.: Evaluation of a computerized Bayesian model for diagnosis of renal cyst vs. tumor vs. normal variant from excretory urogram information. Invest. Radiol. II (1976), 102.
Downloaded from acr.sagepub.com by guest on July 27, 2015
74
P. VIRTAMA, K. KATEVUO, P. MAKELA AND E. O. MAKINEN
HALL D. L., LODWICK G. S., KRUGER R. P., DWYER S. J. and TOWNES J. R.: Direct computer diagnosis of rheumatic heart disease. Radiology 101 (1971),497. LODWICK G. S.: A probabilistic approach to the diagnosis of bone tumors. Radiol. Clin. N. Amer. 3 (1965), 487. - and REICHERTZ P.: Computer assisted diagnosis of tumors and tumor-like lesions of bone. Proceedings of Symposium Ossium, London 1968. - TURNER A. H. JR, LUSTED L. B. and TEMPLETON W. W.: Computer-aided analysis of radiographic images. J. chron. Dis. 19 (1966), 485. - HAUN C. L., SMITH W. E. S., KELLER R. F. and ROBERTSON E. D.: Computer diagnosis of primary bone tumors. Radiology 80 (1963), 273. LUSTED L. B.: Introduction to medical decision making. Charles C. Thomas, Springfield, Ill. 1968. McNEIL B. J., WEBER E., HARRISON D. and HELLMAN S.: Use of signal detection theory in examining the results of a contrast examination. A case study using the lymphangiogram. Radiology 123 (1977), 613. WILSON W. J., TEMPLETON A. W., TURNER A. H. JR and LODWICK G. S.: The computer analysis and diagnosis of gastric ulcers. Radiology 85 (1965), 1064.
Downloaded from acr.sagepub.com by guest on July 27, 2015
COMPUTER AIDED DIAGNOSIS OF BONE TUMORS P. VIRTAMA, K. KATEVUO, P. MAKELA and E. O. MAKINEN The adoption of computer technology in clinical decision making has been relatively slow as compared to the increased use of computers in hospital administration. Computer aided diagnostic programs have been used in the analysis of bone tumors (LODWICK et coIl. 1963), gastric ulcers (WILSON et coIl. 1965) and heart disease (HALL et coIl. 1971), demonstrating that Bayes' formula of inverse probability is suitable for analysing data derived from radiographic films. Computer aided diagnosis is based on the assumption that systematic analysis of fine radiographic detail results in a more reliable diagnosis than does a more or less subjective impression. Computer aided programs can be designed to solve simple differential diagnostic problems by applying statistically significant differences and using the Bayes' formula (LUSTED 1968). Such a program may improve the diagnostic result when sufficient expertise with the clinical problem is not available (FRYBACK & THORNBURY 1976). The clinical value of a large computer aided program is still open to question. The bone tumor program (LODWICK et coli. 1963, LODWICK 1965, LODWICK & REICHERTZ 1968) offers an interesting model for exploration of the effectiveness of such a program in clinical diagnostic decision making. Material and Methods
In computer aided diagnosis, descriptive radiologic terms are transferred in digital form and analysed in the computer program; the statistical probability is computed for each possible diagnosis. Supported by a grant from the Finnish Cancer Association. Submitted for publication 9 January 1978. Acta Radiologica Diagnosis 20 (1979) Fasc. 1 A
70 Downloaded from acr.sagepub.com by guest on July 27, 2015
71
COMPUTER AIDED DIAGNOSIS OF BONE TUMORS
Table 1
True positive (TP) and false positive (FP) rates (in per cent) of observers A, E, C and D in different diagnostic groups at 90 and 60 per cent confidence levels A
B
C
Confidence level
D
TP
FP
TP
FP
TP
FP
TP
FP
Osteosarcoma (33 cases)
88 89
15 24
85 91
18 24
67 76
24 30
70 79
36 48
90 60
Ewing's sarcoma (23 cases)
87 91
13 17
91 91
17 22
70 74
26 30
74 78
35 43
90
Chondrosarcoma (26 cases)
77 85
12 19
81 88
14 15
46 58
23 31
46 50
23 31
90 60
Fibrosarcoma (44 cases)
76 81
19 24
79 86
14 19
62 67
19 24
50 62
24 29
90 60
Benign tumors (51 cases)
69 73
11
13
78 89
18 22
62 67
22 27
69 78
27 31
90 60
Mean
82
17
86
18
65
28
66
33
60
The material comprised 126 malignant and 51 benign solitary bone tumors. A short clinical history and pertinent laboratory data were available. The radiographic quality of the films varied but films with a low diagnostic information were not included. The distribution of malignant tumors appears in Table 1. The benign tumors consisted of 4 osteoid osteomas, 6 non-ossifying fibromas, 9 benign cartilaginous tumors, 3 aneurysmal bone cysts, 3 chondroblastomas, 4 chondromyxoid fibromas, 6 cysts, 8 benign giant cell tumors and 8 osteochondromas. Four radiologists (A, B, C, D) analysed the films. A and C were staff members of a university hospital, qualified specialists in radiology, but with no special expertise in bone radiology. D was a third year resident in radiology, not yet qualified specialist. B in addition to being qualified specialist in radiology had special experience in bone radiology and bone tumor diagnosis from working one year at a bone tumor center. Radiologist A used the computer aided bone tumor program. Two confidence levels of diagnostic decision making were employed, one corresponding to 90 per cent of the computer response and the other to 60 per cent. Diagnoses on a high confidence level (90 %) are usually associated with low true positive and low false positive ratios, while more liberal criteria of diagnosis lead to a greater percentage of false positive and false negative diagnoses (McNEIL et colI. 1977). Results The percentage of true positive and false positive diagnoses of each observer was calculated for each diagnosis. The benign tumors were analysed as one group (Table
Downloaded from acr.sagepub.com by guest on July 27, 2015
72
P. VIRTAMA, K. KATEVUO, P. MAKELA AND E. 0. MAKINEN
Table 2 Differences between observers A, B, C and D in true positive diagnoses of bone tumours at 90 and 60 per cent confidence levels
Observers
Confidence level
AlB
AIC
AID
BIC
BID
C/D
N N
S HS
S HS
S HS
HS HS
N N
N
~
90
60
no difference, S ~ significant difference, HS = highly significant difference
1). Inter-observer differences in diagnostic accuracy were calculated using the true positive and false positive rates (Table 2). Discussion
Osteosarcoma and Ewing's sarcoma turned out to be relatively easy diagnostic problems for the computer aided radiologist (A) and the one with experience in bone radiology (B); chondrosarcoma and benign bone tumors appeared to be more difficult for A than for B. The diagnostic accuracy in chondrosarcoma was particularly low for C and D, and in benign bone tumors for C, while D did not succeed well in diagnosing fibrosarcoma. In general, no significant difference was found between the performance of the computer aided general radiologist (A) and that of the radiologist with experience in bone radiology (B). Nor was there any significant difference between the resident radiologist (D) and general radiologist (C), both of whom performed significantly worse at the 90 per cent level than A or B and still worse at the 60 per cent level. No difference was known to exist between A and C in experience of bone tumor radiology. However, in this experiment A gained an 82 per cent true positive rate at the 90 per cent confidence level while the corresponding rate of C was only 65 per cent (Table 1). The difference in performance between A and C at both the 90 and 60 per cent confidence levels was highly significant (Table 2). The only reasonable explanation seems to be the computer aided bone tumor program used by A. The level of diagnostic accuracy in the present series is lower than that obtained by LODWICK et call. (1963). This may depend on the material selected for the analysis. The present, unselected series comprised consecutive cases from different hospitals including several difficult and atypical lesions. The present computer aided program is based on the recognition of numerous fine radiographic details which are not always easily observed on ordinary films. An improved quality of the films may raise the diagnostic accuracy. The use of special soft tissue radiographic technique, additional projections and tomography may also
Downloaded from acr.sagepub.com by guest on July 27, 2015
COMPUTER AIDED DIAGNOSIS OF BONE TUMORS
73
improve the result. On the other hand, the effective size of the data base could be increased by reducing the number of criteria processed by the present model. The diagnostic accuracy probably does not change by assuming conditional independence between discriminating criteria, to satisfy Bayes' theorem, although the criteria do exhibit a considerable amount of interdependence. The experience of those who created the present computer aided bone tumor program appears to be filed in a form that is readily understandable and can be used by relatively unexperienced radiologists. The use of the program improves diagnostic accuracy significantly and results in improved patient management and cost-saving. However, some work is still needed to improve the present computer program and to develop a generally acceptable clinical strategy in the diagnosis of bone tumors.
SUMMARY Four radiologists, three of whom having no special expertise in bone tumor radiology, analysed 177 bone tumors. One of the radiologists, using a computer aided bone tumor program, performed significantly better than the other two at a comparable level of training and was able to compete successfully with the fourth radiologist experienced in bone diagnosis. The results validate the assumption that computer aided diagnostic programs may improve the diagnostic accuracy of radiologists having limited experience with the problem at hand.
ZUSAMMENFASSUNG Vier Rontgendiagnostiker, von denen drei keine besondere Erfahrung in der Diagnostik von Knochentumoren hatten, analysierten 177 Knochentumoren. Einer von diesen Rontgendiagnostikern, f'uhrte unter Verwendung eines Computer-unterstutzten Knochentumorprogramms signifikant bessere Untersuchungen durch als die zwei anderen mit einem vergleichbaren Niveau von Training. Seine Resultate waren gleich denen des Rontgendiagnostikers, der besondere Erfahrung in der Knochendiagnostik hatte. Die Ergebnisse stiitzen die Annahme, dass Computer-unterstiitzte diagnostische Programme die Diagnostik des Rontgenologens mit begrenzter Erfahrung verbessern.
RESUME Quatre radiologistes dont trois n'avaient pas d'experience speciale dans la radiologie des tumeurs osseuses, ont etudie 177 tumeurs osseuses. L'un des radiologistes utilisant un programme de diagnostic des tumeurs osseuses par ordinateur a eu des resultats significativement meilleurs que les deux autres radiologistes qui avaient un niveau comparable de formation et a ete capable de concurrencer avec succes Ie quatrieme radiologiste qui avait une experience dans Ie diagnostic osseux. Ces resultats confirment l'hypothese que les programmes d'aide au diagnostic par ordinateur peuvent ameliorer I'exactitude du diagnostic de radiologistes ayant une experience limitee du probleme en question.
REFERENCES FRYBACK D. G. and THORNBURY J. R.: Evaluation of a computerized Bayesian model for diagnosis of renal cyst vs. tumor vs. normal variant from excretory urogram information. Invest. Radiol. II (1976), 102.
Downloaded from acr.sagepub.com by guest on July 27, 2015
74
P. VIRTAMA, K. KATEVUO, P. MAKELA AND E. O. MAKINEN
HALL D. L., LODWICK G. S., KRUGER R. P., DWYER S. J. and TOWNES J. R.: Direct computer diagnosis of rheumatic heart disease. Radiology 101 (1971),497. LODWICK G. S.: A probabilistic approach to the diagnosis of bone tumors. Radiol. Clin. N. Amer. 3 (1965), 487. - and REICHERTZ P.: Computer assisted diagnosis of tumors and tumor-like lesions of bone. Proceedings of Symposium Ossium, London 1968. - TURNER A. H. JR, LUSTED L. B. and TEMPLETON W. W.: Computer-aided analysis of radiographic images. J. chron. Dis. 19 (1966), 485. - HAUN C. L., SMITH W. E. S., KELLER R. F. and ROBERTSON E. D.: Computer diagnosis of primary bone tumors. Radiology 80 (1963), 273. LUSTED L. B.: Introduction to medical decision making. Charles C. Thomas, Springfield, Ill. 1968. McNEIL B. J., WEBER E., HARRISON D. and HELLMAN S.: Use of signal detection theory in examining the results of a contrast examination. A case study using the lymphangiogram. Radiology 123 (1977), 613. WILSON W. J., TEMPLETON A. W., TURNER A. H. JR and LODWICK G. S.: The computer analysis and diagnosis of gastric ulcers. Radiology 85 (1965), 1064.
Downloaded from acr.sagepub.com by guest on July 27, 2015
