M e d i c a l P hy s i c s a n d I n f o r m a t i c s • O r i g i n a l R e s e a r c h
Downloaded from www.ajronline.org by NYU Langone Med Ctr-Sch of Med on 05/31/15 from IP address 128.122.253.228. Copyright ARRS. For personal use only; all rights reserved
Kovacs et al. Use of Lossy Data Compression in Full-Field Digital Mammography Medical Physics and Informatics Original Research
Evaluation of Lossy Data Compression in Primary Interpretation for Full-Field Digital Mammography Mark D. Kovacs1, 2 Joshua J. Reicher 3 Jonathan F. Grotts 4 Murray A. Reicher 5 Michael A. Trambert 1 Kovacs MD, Reicher JJ, Grotts JF, Reicher MA, Trambert MA
Keywords: clinical accuracy, full-field digital mammography, lossy data compression, Mammography Quality Standards Act, telemammography DOI:10.2214/AJR.14.12912 Received March 26, 2014; accepted after revision May 24, 2014. Presented at the ARRS 2014 Annual Meeting, San Diego, CA. This project was supported by DR Systems. M. A. Reicher is the Chairman of DR Systems. M. A. Trambert is an advisor to DR Systems. 1
Department of Radiology, Santa Barbara Cottage Hospital, Santa Barbara, CA. Address correspondence to M. D. Kovacs ([email protected]). 2 Present address: Department of Radiology and Biomedical Imaging, University of California, San Francisco, Room M-391, Box 0628, San Francisco, CA 94143-0628. 3 Department of Radiology, Stanford Hospital and Clinics, Stanford, CA. 4 Department of Research Compliance, Santa Barbara Cottage Hospital, Santa Barbara, CA. 5
DR Systems, San Diego, CA.
AJR 2015; 204:570–575 0361–803X/15/2043–570 © American Roentgen Ray Society
570
OBJECTIVE. For full-field digital mammography (FFDM), federal regulations prohibit lossy data compression for primary reading and archiving, unlike all other medical images, where reading physicians can apply their professional judgment in implementing lossy compression. Faster image transfer, lower costs, and greater access to expert mammographers would result from development of a safe standard for primary interpretation and archive of lossy-compressed FFDM images. This investigation explores whether JPEG 2000 80:1 lossy data compression affects clinical accuracy in digital mammography. MATERIALS AND METHODS. Randomized FFDM cases (n = 194) were interpreted by six experienced mammographers with and without JPEG 2000 80:1 lossy compression applied. A cancer-enriched population was used, with just less than half of the cases (42%) containing subtle (< 1 cm) biopsy-proven cancerous lesions, and the remaining cases were negative as proven by 2-year follow-up. Data were analyzed using the jackknife alternative free-response ROC (JAFROC) method. RESULTS. The differences in reader performance between lossy-compressed and nonlossy-compressed images using lesion localization (0.660 vs 0.671), true-positive fraction (0.879 vs 0.879), and false-positive fraction (0.283 vs 0.271) were not statistically significant. There was no difference in the JAFROC figure of merit between lossy-compressed and non-lossy-compressed images, with a mean difference of −0.01 (95% CI, −0.03 to 0.01; F1,5 = 2.30; p = 0.189). CONCLUSION. These results suggest that primary interpretation of JPEG 2000 80:1 lossy-compressed FFDM images may be viable without degradation of clinical quality. Benefits would include lower storage costs, faster telemammography, and enhanced access to expert mammographers.
F
or all imaging modalities except full-field digital mammography (FFDM), lossy data compression can be used at the discretion of the reading physician for both archive and primary interpretation [1]. In contrast to lossless data compression, lossy compression results in an irreversible change to the image and, typically, a much smaller file. Digital mammograms represent high-spatial-resolution images with file sizes ranging from 8–50 MB of space for a single view [2, 3] to 32–200 MB for a four-view screening examination. Despite multiple potential benefits from the use of lossy data compression in mammography, the U.S. Food and Drug Administration (FDA) interprets the Mammography Quality Standards Act (MQSA) as prohibiting lossy compression of both digital and digitized mammographic images, specifically in regard to primary image interpretation and archive
[4]. Lossy compression is permitted for secondary viewing [4]. If it can be shown sufficiently that lossy data compression can be safely applied to digital mammography for primary interpretation, many benefits could result, including markedly faster access across the network for image viewing, reduced storage costs, faster telemammography, and greater access to expert mammographers. The resulting efficient and cost-effective access to expert mammographers could significantly increase breast cancer detection rates and potentially decrease callback rates and radiation exposure to patients. Despite advances in computer-aided detection and FFDM techniques, the greatest factor in determining breast cancer detection in mammography is still the interpreting radiologist. One review showed that positive predictive values for diagnostic mammography vary from 9.1% at the 10th percentile to
AJR:204, March 2015
Downloaded from www.ajronline.org by NYU Langone Med Ctr-Sch of Med on 05/31/15 from IP address 128.122.253.228. Copyright ARRS. For personal use only; all rights reserved
Use of Lossy Data Compression in Full-Field Digital Mammography 40.0% at the 90th percentile [5]. Another review of radiologists reported a cancer detection rate of 6% among high-volume mammographers versus a cancer detection rate of 3.4% among general radiologists, a difference of 76% [6]. In comparison, reported increases in cancer detection rates by computer-aided detection are 7.4%, 7.6%, and 19.5% [7–9]. How and to what degree mammography interpretation skill is dependent on case volume, specialization, or training may be questioned, but the fact remains that some mammographers detect breast cancer with more accuracy than others [10–13]. Allowing better access to experts via inexpensive telemammography of lossy data compressed images could significantly improve cancer detection rates. Previous investigations have found compression levels of anywhere from 14:1 to 80:1 to be potentially acceptable levels of compression [2, 14–21]. However, to date, there have been insufficient data to prove that lossy data compression can be used in a clinical setting for primary interpretation of the mammographic images. We are aware of only limited previous research into the variability of effects of image compression across different digital mammography vendors, which is important if indeed a universal standard for mammography image compression is to be established [22]. To our knowledge, this is the first large-scale clinical study performed in the United States comparing lossy compression versus no lossy compression in the diagnostic accuracy of FFDM images. Materials and Methods This study was approved by an institutional review board before initiation. A waiver of authorization for the HIPAA Privacy Rule was deemed unnecessary because the study involved deidentified data.
Study Design Many different techniques have been used to evaluate the effect of lossy compression on medical imaging data, including numeric analysis of pixel values before and after compression, subjective observer evaluation with a focus on aesthetic acceptability, and objective measurement of diagnostic accuracy using blinded evaluation methods. The study described here represents a multireader multicase framework using diagnostic accuracy as the objective metric for comparing lossy-compressed to non-lossy-compressed mammography images. Readers were asked to treat each provided FFDM examination as a diagnostic study and to
Right
Left
Right
Left
Mediolateral Oblique
Craniocaudal
1 Upper
2 Upper
3 Lower
4 Lower
5 Outer
6 Outer
7 Inner
8 Inner
Fig. 1—Quadrant system used for localizing suspicious findings. (Drawing by Reicher MA) Fig. 2—Patient study flowchart.
Eligible Patients n = 205 Excluded Patients (n = 11):
• Inconclusive (n = 6) • Used for training (n = 5)
Cancer Cases n = 81
assign each examination a probability of malignancy, using a rating scale of 1, 2, 3, 4A, 4B, 4C, or 5, which paralleled the modified BI-RADS rating system (the 0 rating was not allowed because this was an experimental study). Readers were asked to mark the quadrant of suspicious lesions on a form using a 1–8 numeric system, corresponding to the four quadrants of each breast (Fig. 1). If more than one suspicious lesion was perceived as being present, responses were to be provided for only the most suspicious lesion. If a suspicious lesion was noted, readers were asked to document whether the lesion contained a mass, calcifications, or both. This design has been applied in previous research involving mammography images on a smaller scale as part of the Pan-Canadian Evaluation of Irreversible Compression Ratios [23]. We considered a lesion successfully localized if the reader correctly identified the lesion and its type and localized it to the correct quadrant.
Patient Population A total of 205 deidentified FFDM cases were obtained from four different radiology practices. Two East Coast and two West Coast radiology practices were included. Examinations used in the study were performed between July 2003 and March 2007. Two different manufacturers of digital mammography systems were used to pro-
Normal or Benign Cases n = 113
duce the mammographic images. All mammograms were bilateral (right-to-left craniocaudal and right-to-left mediolateral oblique), with two views of each breast. Several examinations were included for patients with breast implants, which contained additional standard implant-displaced views. The mean age of the participants was 59.5 years, with a range of 31–89 years. The collection of mammograms represented an enriched cancer population, to enhance the methods proposed for statistical analysis. Of the 205 cases, five cases (two of which contained a cancer and three of which were normal) were removed and used for reader training. Six additional cases (all of which contained a cancer) were removed at the time of determination of ground truth because of poor correlation between the mammographic findings and supporting documents (e.g., mammogram and pathology reports). Of the 194 remaining cases, 113 were normal or contained only benign findings (proven by 2-year negative follow-up) and 81 contained a subtle (< 1 cm) cancerous lesion as confirmed with biopsy (Fig. 2). Documentation regarding 2-year negative followup for negative examinations or biopsy results for positive examinations spanned the period between May 2004 and March 2007. Cases were selected for the study on the basis of indication by pathology reports, lack of identifying marks, lack of significant artifact such as that due to motion, posi-
AJR:204, March 2015 571
Downloaded from www.ajronline.org by NYU Langone Med Ctr-Sch of Med on 05/31/15 from IP address 128.122.253.228. Copyright ARRS. For personal use only; all rights reserved
Kovacs et al. tioning of the modality, and availability of ground truth for the diagnosis of the case. The images were anonymized by the mammography sites so that the identities of the subjects were unknown to the investigators. The cases were accompanied by copies of the associated clinical information used to establish the diagnosis for each image. This information did not make the source of the image identifiable to the reading radiologists. A unique identifier was assigned to each subject, and that identifier was used to replace patient-identifying information contained in the copies of such records.
Ground Truth Localization The ground truth was established on the basis of pathology reports for biopsy-proven cancers or subsequent normal radiology reports for negative examinations. Positive examinations contained a single breast cancer with a lesion of 1 cm or smaller that was verified by a breast biopsy. Normal examinations showed normal follow-up at least 2 years after the images presented. Prior examinations were not included for comparison in the study.
Viewing Environment Environmental conditions similar to the typical clinical environment were established, including temperature, ambient light, light sources (< 50 lux), level of comfort, level of furnishings, and ambient noise. Display of digital mammograms was conducted using industry standard monitors (DR Systems) of 5-megapixel resolution, which meets current FDA requirements for mammography.
Data Compression Algorithm The generic JPEG 2000 80:1 compression algorithm (Joint Photographic Experts Group) was used for creation of lossy-compressed images. Advantages of JPEG 2000 over standard JPEG include control of the compression ratio, progressive lossy-to-lossless coding and decoding, multiresolution representation, ROI encoding, and error resilience [2]. In general, JPEG 2000 is preferred over other options given its nonproprietary status and wide availability.
Readers The readers of the study consisted of six radiologists on staff at a single institution, where FFDM has been in use since 2003. Each reader had at least 4 years’ experience and MQSA certification. The six readers had 4–29 years of experience (mean, 15.8 years). The readers were blinded to the truth status (positive or negative) and to whether the images were lossy compressed. To minimize recall bias, readings of lossy-compressed and non-lossy-compressed cases from the
572
same patient were separated by a minimum of 2 months. This was done by separating readings into two phases that were at least 2 months apart for each reader. Each radiologist reader had his or her own set of images randomized into two blocks with varying sets of cases having undergone lossy data compression. The first block contained all 194 patients and the second block contained all 194 patients. Readers were scheduled to do readings of approximately 50 cases per session within each phase of the study. Before beginning the study, each reader completed a training session during which the reading session instructions, reader response form, and overall study design were reviewed. Each reader completed a five-examination practice reading session that contained a mixture of positive (cancer) and normal cases that were not contained within the study set.
Statistical Analysis The study was designed in the multireader multicase framework with each reader examining all cases in both modalities (lossy-compressed and non-lossy-compressed mammography examinations). Reader performance was summarized using the lesion localization fraction (number of lesions localized divided by number of lesions), true-positive fraction (number of correctly identified cancer cases divided by the total number of cancer cases), and false-positive fraction (number of normal or benign cases marked as cancerous divided by the number of normal or benign cases). A comparison of readers’ performance between modalities was done using the jackknife alternative free-response ROC (JAFROC) method [24]. Lesion localization and nonlocalization was used to create a figure of merit (FOM) in the JAFROC paradigm, defined as the probability that a cancer on an abnormal image is scored higher than a falsely marked location on a normal image. The JAFROC FOM between modalities, along with the other reader performance summary statistics, were compared using an ANOVA and results are
presented as the F statistic, with numerator and denominator degrees of freedom, along with the p value. A p value below 0.05 was considered statistically significant. A random effect was used within the JAFROC for reader and cases to infer results to the general population. The JAFROC analysis was implemented in JAFROC version 4.2 [25]. Two subgroup analyses were performed according to cancer type. Reader performance was tested with a cancer population composed of only calcification and a cancer population composed of only cancer cases with a mass or mass with a calcification. Interrater reliability was computed using the Fleiss kappa statistic and by converting reader responses from the modified BI-RADS to the presence of cancer or normal or benign findings. A formal sample size was not conducted as part of this project because we were unable to identify the variability between readers and modalities and within readers. A chart on sample size for multireader multicase studies using the AUC paradigm was consulted to determine the approximate sample size needed to test for a difference in diagnostic accuracy between the two imaging modalities [26]. Moderate accuracy, moderate difference between modalities, and moderate variability among observers suggested that each reader would have to read 78 cases. The final case number of 194 was thought to adequately account for any unseen variability in the study.
Results Pathologic Findings There were 194 cases included in the study with 81 patients having biopsy-proven cancer (42%). Of the 81 positive (cancer) cases, 45 (55.6%) were invasive ductal carcinoma, 22 (27.2%) were ductal carcinoma in situ, 3 (3.7%) were invasive lobular carcinoma, 3 (3.7%) were invasive cancer with ductal and lobular components, 2 (2.5%) were tubular carcinoma (subtype of invasive ductal carcinoma), 2 (2.5%) were infiltrating carcinoma
TABLE 1: Lesion Localization Fraction, True-Positive Fraction (TPF), and False-Positive Fraction (FPF), by Reader and Modality Reader
Lesion Localization Fraction
TPF
1
0.704/0.691
0.864/0.852
0.212/0.150
2
0.741/0.704
0.951/0.914
0.363/0.398
3
0.704/0.716
0.938/0.901
0.425/0.381
4
0.543/0.593
0.827/0.864
0.257/0.274
5
0.556/0.580
0.815/0.840
0.239/0.265
6
0.716/0.741
0.877/0.901
0.204/0.159
Mean
0.660/0.671
0.879/0.879
0.283/0.271
FPF
Note—Data are compressed/uncompressed fractions.
AJR:204, March 2015
Downloaded from www.ajronline.org by NYU Langone Med Ctr-Sch of Med on 05/31/15 from IP address 128.122.253.228. Copyright ARRS. For personal use only; all rights reserved
Use of Lossy Data Compression in Full-Field Digital Mammography not otherwise specified, 1 (1.2%) was lobular carcinoma in situ or ductal carcinoma in situ not otherwise specified, 1 (1.2%) was intraductal papillary carcinoma, 1 (1.2%) was pleomorphic lobular carcinoma (subtype of invasive lobular carcinoma), and 1 (1.2%) was adenoid cystic carcinoma. Of the 81 positive cases, 38 (47%) contained a suspicious mass lesion, 30 (37%) contained suspicious calcifications, and 13 (16%) cases contained a suspicious mass lesions with calcifications. All lesions measured 1 cm or smaller. Observer Performance Lesion localization fraction, true-positive fraction, and false-positive fraction between modalities are presented in Table 1. The differences in reader performance between imaging modalities using lesion localization (F1,5 = 0.68; p = 0.449), true-positive fraction (F1,5 = 0; p = 1.000), and false-positive fraction (F1,5 = 0.44; p = 0.536) were not statistically significant. There was no evidence of a difference in the JAFROC FOM between lossycompressed and uncompressed imaging modalities with a mean difference of −0.01 (95% CI, −0.03 to 0.01; F1,5 = 2.30; p = 0.189). Individual reader performance using the JAFROC FOM is presented in Table 2. There were no observed differences between the JAFROC
FOM of the imaging modalities in the cancer population subgroup containing just calcifications (F1,5 = 0.36; p = 0.569) or the cancer population consisting of masses and masses with calcifications (F1,5 = 2.83; p = 0.153). Reader performance for the subgroups is presented in Table 3 and Table 4. The interrater reliability was similar in the lossy-compressed and nonlossy-compressed modalities with kappa statistics of 0.529 and 0.504, respectively. Discussion For FFDM, the FDA currently permits lossy data compression of images only for secondary viewing. Many benefits, including much faster access to images for viewing, reduced storage costs, faster telemammography, and greater access to expert mammographer consultants, would result if a standard could be developed to safely enable lossy data compression for primary viewing and storage. We hypothesized that mammography image data compression with nonproprietary JPEG 2000 80:1 compression algorithm might represent a diagnostically lossless solution that would not degrade clinical accuracy. Previous studies in mammography have evaluated irreversible compression methods by looking at their effects on radiologist performance. Good et al. [27] applied the orig-
TABLE 2: Comparison of F igures of Merit (FOM) Between Compressed and Uncompressed Images for Jackknife A lternative Free-Response ROC (JAFROC) Reader
JAFROC FOM
1
0.791/0.804
2
0.713/0.699
3
0.739/0.766
4
0.681/0.69
5
0.705/0.704
6
0.806/0.830
Mean
0.739/0.749
Note—Data are compressed/uncompressed FOM values.
inal JPEG algorithm to 60 digitized (not FFDM) mammograms and evaluated the performance of eight readers in the detection of clusters of microcalcifications and masses with ROC analysis. This investigation found no statistically significant difference in detecting masses, but a significant difference was found for the detection of microcalcification clusters when very high levels of compression were used (about 100:1).
TABLE 3: Reader Performance in Both Modalities for Mass Cancer Types (Study Types Mass or Both) False-Positive Fraction
JAFROC FOMa
Reader
Lesion Localization Fraction
True-Positive Fraction
1
0.627/0.706
0.843/0.863
0.212/0.150
0.745/0.809
2
0.765/0.745
1.000/0.980
0.363/0.398
0.733/0.737
3
0.647/0.686
0.922/0.902
0.425/0.381
0.704/0.752
4
0.529/0.569
0.863/0.863
0.257/0.274
0.671/0.679
5
0.627/0.627
0.863/0.863
0.239/0.265
0.748/0.725
6
0.706/0.745
0.902/0.902
0.204/0.159
0.799/0.831
Mean
0.650/0.680
0.899/0.895
0.283/0.271
0.734/0.755
Note—Data are compressed/uncompressed values. JAFROC = jackknife alternative free-response ROC, FOM = figures of merit. ap = 0.153.
TABLE 4: Reader Performance in Both Modalities for Calcification Cancer Types (Study Type Calcification) Reader
Lesion Localization Fraction
True-Positive Fraction
False-Positive Fraction
JAFROC FOMa
1
0.833/0.667
0.900/0.833
0.212/0.150
0.868/0.795
2
0.700/0.633
0.867/0.800
0.363/0.398
0.680/0.634
3
0.800/0.767
0.967/0.900
0.425/0.381
0.799/0.790
4
0.567/0.633
0.767/0.867
0.257/0.274
0.698/0.708
5
0.433/0.500
0.733/0.800
0.239/0.265
0.631/0.670
6
0.733/0.733
0.833/0.900
0.204/0.159
0.816/0.829
Mean
0.678/0.656
0.844/0.850
0.283/0.271
0.749/0.738
Note—Data are compressed/uncompressed values. JAFROC = jackknife alternative free-response ROC, FOM = figures of merit. ap = 0.569.
AJR:204, March 2015 573
Downloaded from www.ajronline.org by NYU Langone Med Ctr-Sch of Med on 05/31/15 from IP address 128.122.253.228. Copyright ARRS. For personal use only; all rights reserved
Kovacs et al. Additional performance-based studies have used the newer wavelet-transform methods of image compression, such as the JPEG 2000 standard or set partitioning in hierarchic trees (SPIHT) algorithm. Perlmutter et al. [28] used 57 digitized mammograms compressed using SPIHT to determine whether the use of lossy compression resulted in patient care decisions that were different from those that resulted from the use of uncompressed images. Their study found no statistically significant difference between original analog or digitized mammography examinations and their compressed counterparts at an 80:1 compression ratio. Penedo et al. [2] digitized 112 mammographic images (64 abnormal and 48 normal) and applied compression at 40:1 and 80:1 using the JPEG 2000 and SPIHT methods, and five experienced radiologists were asked to locate and rate clusters of microcalcifications and masses on the freeresponse ROC data acquisition paradigm. They determined that lossy compression of digitized mammographic data at 80:1 with JPEG 2000 or SPIHT could be performed without decreasing the rate of detection of clusters of microcalcifications and masses. To our knowledge, this work represents the first large clinical study to show that no degradation of diagnostic accuracy resulted when implementing JPEG 2000 80:1 lossy compression on FFDM images. Of note, this study also found that, in grouping the lesions as masses (Table 3) versus calcifications (Table 4), there was still no statistically significant difference between the lossy-compressed and non-lossy-compressed image sets. This is in contradistinction from previous work, such as by Suryanarayanan et al. [15], which has suggested that lossy compression can adversely affect the detection of calcifications. The implications of this study are considerable. With validation of diagnostically lossless data compression at the level of 80:1 for FFDM, a new era of highly practical telemammography could emerge, with potentially dramatic improvement in cancer detection rates if images are funneled to expert readers. Of note, documented variations in reader accuracy and sensitivity rates far exceed the measurable benefit provided by technologies such as FFDM, computer-aided detection, and breast tomography. That is not to say that these beneficial technologies should be ignored, but there is nevertheless a huge benefit to be obtained by shifting mammography to the more accurate and sensitive readers, which may actually lower costs,
574
whereas the FFDM, computer-aided detection, and tomography require capital investments. The net result could be higher cancer detection rates with lower-cost mammography. Data compression will become even more important with breast tomography because the file sizes are significantly larger. Data compression also provides benefits to other important constituents. The dramatic reduction in file sizes not only increases the transportability of FFDM examinations for the purpose of primary reading but also improves access for nonradiologists who require remote viewing, such as oncologists or breast surgeons. Finally, there is a growing demand for patient access to medical images. Data compression makes it more feasible for patients to control, own, and view their own personal images. Limitations of the study included the following: there was a limited sample size chosen for the study of 194 cases, of which 81 were positive for cancer, and a limited number of readers (six in total). These numbers were chosen to provide sufficient statistical power. Although the case number is limited, it does represent an increase over previous studies (of different design) involving evaluation of lossy compression of mammographic images. Recall bias is a potential limitation of the study but is expected to be minimal because we instituted a minimum 2-month interval between sessions for each reader. The likelihood of recall bias was also decreased because the cases were displayed in a random order. It has been shown previously that some degree of memory washout occurs within days [29]. An additional potential limitation of the study is that although the cases did come from four different radiology practices, these practices were limited to two practices on the West Coast and two on the East Coast, which does represent some limitation in geographic variation in cases. Prior examinations were not available for comparison purposes. This may have made it challenging at times for the reader to decide on an appropriate rating for lesions that were borderline for having benign characteristics. This may have affected overall sensitivity and specificity numbers but was not expected to (and did not) significantly vary overall on the basis of the compressed or uncompressed status of the case. Of note, one reader (reader 3) consistently rated a higher proportion of perceived low probability (likely benign finding) cases as a “3” rather than a “2” which did lead to more false-positives (lowering specificity), but this also did
not significantly vary according to the compressed or uncompressed status of the case. The method of localization is also a potential limitation of the study design. Because we chose to localize via quadrant, it is conceivable that a reader misinterpreted a benign finding as malignant, but it was counted as a true-positive or localization if there was a separate true malignant finding of the same type (i.e., mass, calcifications, or both) within the same quadrant. We think, however, that this scenario would have occurred extremely infrequently if at all. Another way in which the quadrant system represented a potential limitation of the study design is for lesions that straddled the boundary between two or more quadrants. There were a number of such lesions in the dataset (23 of 194 cases), and for these, at the time of determination of ground truth, two or more quadrants were listed as possible correct choices. Conclusion This investigation shows no degradation of clinical accuracy when implementing JPEG 2000 80:1 lossy compression for primary reading of FFDM, further suggesting that the FDA should consider changing its policy of prohibiting primary reading of lossy-compressed FFDM images. If such lossy-compressed images are considered equivalent to the original and therefore are permissible under MQSA, costs would be lower and access to mammographer experts would be enhanced. Primary reading and archive of lossy-compressed FFDM images should be permitted, given no difference in clinical accuracy in interpretation of lossycompressed versus lossless or uncompressed versions of images. Acknowledgments We thank all the readers from Santa Barbara Radiology Medical Group who volunteered their time for this study. We also thank Elizabeth Krupinski for her insight regarding questions on study design and Matthew Borzage for his work in initial development of the study protocol and design. References 1. American College of Radiology. ACR-AAPM-SIIM technical standard for electronic practice of medical imaging (resolution 35). American College of Radiology website. www.acr.org/~/media/ACR/ Documents/PGTS/standards/ElectronicPractice MedImg.pdf. Published 2007. Revised 2012. Accessed April 2, 2014
AJR:204, March 2015
Downloaded from www.ajronline.org by NYU Langone Med Ctr-Sch of Med on 05/31/15 from IP address 128.122.253.228. Copyright ARRS. For personal use only; all rights reserved
Use of Lossy Data Compression in Full-Field Digital Mammography 2. Penedo M, Souto M, Tahoces PG, et al. Free-response receiver operating characteristic evaluation of lossy JPEG2000 and object-based set partitioning in hierarchical tree compression of digitized mammograms. Radiology 2005; 237:450–457 3. Brown DE. Archiving: fundamentals of storage technology. Society for Imaging Informatics in Medicine website. siim.org/books/archiving/TOC. Accessed April 2, 2014 4. U.S. Food and Drug Administration. The Mammography Quality Standards Act final regulations: modifications and additions to policy guidance help system #9. U.S. Food and Drug Administration website. www.fda.gov/Radiation-EmittingProducts/ MammographyQualityStandardsActandProgram/ DocumentArchives/ucm114207.htm. Published April 19, 2006. Accessed June 6, 2013 5. Sickles EA, Miglioretti DL, Ballard-Barbash R, et al. Performance benchmarks for diagnostic mammography. Radiology 2005; 235:775–790 6. Sickles EA, Wolverton DE, Dee KE. Performance parameters for screening and diagnostic mammography specialist and general radiologists. Radiology 2002; 224:861–869 7. Birdwell RL, Bandodkar P, Ikeda DM. Computeraided detection with screening mammography in a university hospital setting. Radiology 2005; 236:451–457 8. Morton MJ, Whaley DH, Brandt KR, Amrami KK. Screening mammograms: interpretation with computer-aided detection—prospective evaluation. Radiology 2006; 239:375–383 9. Freer TW, Ulissey MJ. Screening mammography with computer-aided detection: prospective study of 12,860 patients in a community breast center. Radiology 2001; 220:781–786 10. Beam CA, Conant EF, Sickles EA. Association of volume and volume-independent factors with ac-
curacy in screening mammogram interpretation. J Natl Cancer Inst 2003; 95:282–290 11. Elmore JG, Miglioretti DL, Carney PA. Does practice make perfect when interpreting mammography? Part II. J Natl Cancer Inst 2003; 95:250–252 12. Esserman L, Cowley H, Eberle C, et al. Improving the accuracy of mammography: volume and outcome relationships. J Natl Cancer Inst 2002; 94:369–375 13. Linver MN, Paster SP, Rosenberg RB, et al. Improvement in mammography interpretation skills in a community radiology practice after dedicated teaching courses: a two-year audit of 38,633 cases. Radiology 1992; 184:39–43 14. Kallergi M, Lucier BJ, Berman CG, et al. Highperformance wavelet compression for mammography localization response operating characteristic evaluation. Radiology 2006; 238:62–73 15. Suryanarayanan S, Karellas A, Vedantham S, et al. Detection of simulated lesions on data-compressed digital mammograms. Radiology 2005; 236:31–36 16. Betts BJ, Aiyer A, Li J, et al. Management and lesion detection effects of lossy image compression on digitized mammograms. In: Karssemeijer N, Thijssen M, Hendriks J, van Ernin L, eds. Proceedings of the 4th International Workshop on Digital Mammography. Nijmegen, The Netherlands: Springer Netherlands, 1998:449–456 17. Destounis S, Somerville P, Murphy P, et al. Perceived sufficiency of full-field digital mammograms with and without irreversible image data compression for comparison with next-year mammograms. J Digit Imaging 2011; 24:66–74 18. Kang BJ, Kim HS, Park CS, et al. Acceptable compression ratio of full-field digital mammography using JPEG 2000. Clin Radiol 2011; 66:609–613 19. Liang Z, Du X, Yang Y, et al. Effects of different compression techniques on diagnostic accuracies
of breast masses on digitized mammograms. Acta Radiol 2008; 49:747–751 20. Visser R, Oostveen L, Karssemeijer N. Lossless compression of digital mammograms. In: Astley SM, Brady M, Rose C, Zwiggelaar R, eds. Proceedings of the 8th International Workshop on Digital Mammography. Manchester, UK: IWDM, 2006:533–540 21. Lehmann TM, Abel J, Weiss C. The impact of lossless image compression to radiographs. Proc SPIE 2003; 6145:290–297 22. Slone RM. Assessment of visually lossless irreversible image compression: comparison of three methods by using an image-comparison workstation. Radiology 2000; 215:543–553 23. Koff D, Bak P, Brownrigg P, et al. Pan-Canadian evaluation of irreversible compression ratios (“lossy” compression) for development of national guidelines. J Digit Imaging 2009; 22:569–578 24. Chakraborty DP, Berbaum KS. Observer studies involving detection and localization: modeling, analysis and validation. Med Phys 2004; 31:2313–2330 25. Chakraborty DP. Dev Chakraborty’s FROC website. www.devchakraborty.com. Published 2005. Accessed November 12, 2014 26. Obuchowski NA. Sample size tables for receiver operating characteristic studies. AJR 2000; 175:603–608 27. Good WF, Sumkin JH, Ganott M, et al. Detection of masses and clustered microcalcifications on data compressed mammograms: an observer performance study. AJR 2000; 175:1573–1576 28. Perlmutter S, Cosman P, Gray R, et al. Image quality in lossy compressed digital mammograms. Signal Process 1997; 59:189–210 29. Hardesty LA, Ganott MA, Hakim CM, et al. “Memory effect” in observer performance studies of mammograms. Acad Radiol 2005; 12:286–290
AJR:204, March 2015 575
Downloaded from www.ajronline.org by NYU Langone Med Ctr-Sch of Med on 05/31/15 from IP address 128.122.253.228. Copyright ARRS. For personal use only; all rights reserved
Kovacs et al. Use of Lossy Data Compression in Full-Field Digital Mammography Medical Physics and Informatics Original Research
Evaluation of Lossy Data Compression in Primary Interpretation for Full-Field Digital Mammography Mark D. Kovacs1, 2 Joshua J. Reicher 3 Jonathan F. Grotts 4 Murray A. Reicher 5 Michael A. Trambert 1 Kovacs MD, Reicher JJ, Grotts JF, Reicher MA, Trambert MA
Keywords: clinical accuracy, full-field digital mammography, lossy data compression, Mammography Quality Standards Act, telemammography DOI:10.2214/AJR.14.12912 Received March 26, 2014; accepted after revision May 24, 2014. Presented at the ARRS 2014 Annual Meeting, San Diego, CA. This project was supported by DR Systems. M. A. Reicher is the Chairman of DR Systems. M. A. Trambert is an advisor to DR Systems. 1
Department of Radiology, Santa Barbara Cottage Hospital, Santa Barbara, CA. Address correspondence to M. D. Kovacs ([email protected]). 2 Present address: Department of Radiology and Biomedical Imaging, University of California, San Francisco, Room M-391, Box 0628, San Francisco, CA 94143-0628. 3 Department of Radiology, Stanford Hospital and Clinics, Stanford, CA. 4 Department of Research Compliance, Santa Barbara Cottage Hospital, Santa Barbara, CA. 5
DR Systems, San Diego, CA.
AJR 2015; 204:570–575 0361–803X/15/2043–570 © American Roentgen Ray Society
570
OBJECTIVE. For full-field digital mammography (FFDM), federal regulations prohibit lossy data compression for primary reading and archiving, unlike all other medical images, where reading physicians can apply their professional judgment in implementing lossy compression. Faster image transfer, lower costs, and greater access to expert mammographers would result from development of a safe standard for primary interpretation and archive of lossy-compressed FFDM images. This investigation explores whether JPEG 2000 80:1 lossy data compression affects clinical accuracy in digital mammography. MATERIALS AND METHODS. Randomized FFDM cases (n = 194) were interpreted by six experienced mammographers with and without JPEG 2000 80:1 lossy compression applied. A cancer-enriched population was used, with just less than half of the cases (42%) containing subtle (< 1 cm) biopsy-proven cancerous lesions, and the remaining cases were negative as proven by 2-year follow-up. Data were analyzed using the jackknife alternative free-response ROC (JAFROC) method. RESULTS. The differences in reader performance between lossy-compressed and nonlossy-compressed images using lesion localization (0.660 vs 0.671), true-positive fraction (0.879 vs 0.879), and false-positive fraction (0.283 vs 0.271) were not statistically significant. There was no difference in the JAFROC figure of merit between lossy-compressed and non-lossy-compressed images, with a mean difference of −0.01 (95% CI, −0.03 to 0.01; F1,5 = 2.30; p = 0.189). CONCLUSION. These results suggest that primary interpretation of JPEG 2000 80:1 lossy-compressed FFDM images may be viable without degradation of clinical quality. Benefits would include lower storage costs, faster telemammography, and enhanced access to expert mammographers.
F
or all imaging modalities except full-field digital mammography (FFDM), lossy data compression can be used at the discretion of the reading physician for both archive and primary interpretation [1]. In contrast to lossless data compression, lossy compression results in an irreversible change to the image and, typically, a much smaller file. Digital mammograms represent high-spatial-resolution images with file sizes ranging from 8–50 MB of space for a single view [2, 3] to 32–200 MB for a four-view screening examination. Despite multiple potential benefits from the use of lossy data compression in mammography, the U.S. Food and Drug Administration (FDA) interprets the Mammography Quality Standards Act (MQSA) as prohibiting lossy compression of both digital and digitized mammographic images, specifically in regard to primary image interpretation and archive
[4]. Lossy compression is permitted for secondary viewing [4]. If it can be shown sufficiently that lossy data compression can be safely applied to digital mammography for primary interpretation, many benefits could result, including markedly faster access across the network for image viewing, reduced storage costs, faster telemammography, and greater access to expert mammographers. The resulting efficient and cost-effective access to expert mammographers could significantly increase breast cancer detection rates and potentially decrease callback rates and radiation exposure to patients. Despite advances in computer-aided detection and FFDM techniques, the greatest factor in determining breast cancer detection in mammography is still the interpreting radiologist. One review showed that positive predictive values for diagnostic mammography vary from 9.1% at the 10th percentile to
AJR:204, March 2015
Downloaded from www.ajronline.org by NYU Langone Med Ctr-Sch of Med on 05/31/15 from IP address 128.122.253.228. Copyright ARRS. For personal use only; all rights reserved
Use of Lossy Data Compression in Full-Field Digital Mammography 40.0% at the 90th percentile [5]. Another review of radiologists reported a cancer detection rate of 6% among high-volume mammographers versus a cancer detection rate of 3.4% among general radiologists, a difference of 76% [6]. In comparison, reported increases in cancer detection rates by computer-aided detection are 7.4%, 7.6%, and 19.5% [7–9]. How and to what degree mammography interpretation skill is dependent on case volume, specialization, or training may be questioned, but the fact remains that some mammographers detect breast cancer with more accuracy than others [10–13]. Allowing better access to experts via inexpensive telemammography of lossy data compressed images could significantly improve cancer detection rates. Previous investigations have found compression levels of anywhere from 14:1 to 80:1 to be potentially acceptable levels of compression [2, 14–21]. However, to date, there have been insufficient data to prove that lossy data compression can be used in a clinical setting for primary interpretation of the mammographic images. We are aware of only limited previous research into the variability of effects of image compression across different digital mammography vendors, which is important if indeed a universal standard for mammography image compression is to be established [22]. To our knowledge, this is the first large-scale clinical study performed in the United States comparing lossy compression versus no lossy compression in the diagnostic accuracy of FFDM images. Materials and Methods This study was approved by an institutional review board before initiation. A waiver of authorization for the HIPAA Privacy Rule was deemed unnecessary because the study involved deidentified data.
Study Design Many different techniques have been used to evaluate the effect of lossy compression on medical imaging data, including numeric analysis of pixel values before and after compression, subjective observer evaluation with a focus on aesthetic acceptability, and objective measurement of diagnostic accuracy using blinded evaluation methods. The study described here represents a multireader multicase framework using diagnostic accuracy as the objective metric for comparing lossy-compressed to non-lossy-compressed mammography images. Readers were asked to treat each provided FFDM examination as a diagnostic study and to
Right
Left
Right
Left
Mediolateral Oblique
Craniocaudal
1 Upper
2 Upper
3 Lower
4 Lower
5 Outer
6 Outer
7 Inner
8 Inner
Fig. 1—Quadrant system used for localizing suspicious findings. (Drawing by Reicher MA) Fig. 2—Patient study flowchart.
Eligible Patients n = 205 Excluded Patients (n = 11):
• Inconclusive (n = 6) • Used for training (n = 5)
Cancer Cases n = 81
assign each examination a probability of malignancy, using a rating scale of 1, 2, 3, 4A, 4B, 4C, or 5, which paralleled the modified BI-RADS rating system (the 0 rating was not allowed because this was an experimental study). Readers were asked to mark the quadrant of suspicious lesions on a form using a 1–8 numeric system, corresponding to the four quadrants of each breast (Fig. 1). If more than one suspicious lesion was perceived as being present, responses were to be provided for only the most suspicious lesion. If a suspicious lesion was noted, readers were asked to document whether the lesion contained a mass, calcifications, or both. This design has been applied in previous research involving mammography images on a smaller scale as part of the Pan-Canadian Evaluation of Irreversible Compression Ratios [23]. We considered a lesion successfully localized if the reader correctly identified the lesion and its type and localized it to the correct quadrant.
Patient Population A total of 205 deidentified FFDM cases were obtained from four different radiology practices. Two East Coast and two West Coast radiology practices were included. Examinations used in the study were performed between July 2003 and March 2007. Two different manufacturers of digital mammography systems were used to pro-
Normal or Benign Cases n = 113
duce the mammographic images. All mammograms were bilateral (right-to-left craniocaudal and right-to-left mediolateral oblique), with two views of each breast. Several examinations were included for patients with breast implants, which contained additional standard implant-displaced views. The mean age of the participants was 59.5 years, with a range of 31–89 years. The collection of mammograms represented an enriched cancer population, to enhance the methods proposed for statistical analysis. Of the 205 cases, five cases (two of which contained a cancer and three of which were normal) were removed and used for reader training. Six additional cases (all of which contained a cancer) were removed at the time of determination of ground truth because of poor correlation between the mammographic findings and supporting documents (e.g., mammogram and pathology reports). Of the 194 remaining cases, 113 were normal or contained only benign findings (proven by 2-year negative follow-up) and 81 contained a subtle (< 1 cm) cancerous lesion as confirmed with biopsy (Fig. 2). Documentation regarding 2-year negative followup for negative examinations or biopsy results for positive examinations spanned the period between May 2004 and March 2007. Cases were selected for the study on the basis of indication by pathology reports, lack of identifying marks, lack of significant artifact such as that due to motion, posi-
AJR:204, March 2015 571
Downloaded from www.ajronline.org by NYU Langone Med Ctr-Sch of Med on 05/31/15 from IP address 128.122.253.228. Copyright ARRS. For personal use only; all rights reserved
Kovacs et al. tioning of the modality, and availability of ground truth for the diagnosis of the case. The images were anonymized by the mammography sites so that the identities of the subjects were unknown to the investigators. The cases were accompanied by copies of the associated clinical information used to establish the diagnosis for each image. This information did not make the source of the image identifiable to the reading radiologists. A unique identifier was assigned to each subject, and that identifier was used to replace patient-identifying information contained in the copies of such records.
Ground Truth Localization The ground truth was established on the basis of pathology reports for biopsy-proven cancers or subsequent normal radiology reports for negative examinations. Positive examinations contained a single breast cancer with a lesion of 1 cm or smaller that was verified by a breast biopsy. Normal examinations showed normal follow-up at least 2 years after the images presented. Prior examinations were not included for comparison in the study.
Viewing Environment Environmental conditions similar to the typical clinical environment were established, including temperature, ambient light, light sources (< 50 lux), level of comfort, level of furnishings, and ambient noise. Display of digital mammograms was conducted using industry standard monitors (DR Systems) of 5-megapixel resolution, which meets current FDA requirements for mammography.
Data Compression Algorithm The generic JPEG 2000 80:1 compression algorithm (Joint Photographic Experts Group) was used for creation of lossy-compressed images. Advantages of JPEG 2000 over standard JPEG include control of the compression ratio, progressive lossy-to-lossless coding and decoding, multiresolution representation, ROI encoding, and error resilience [2]. In general, JPEG 2000 is preferred over other options given its nonproprietary status and wide availability.
Readers The readers of the study consisted of six radiologists on staff at a single institution, where FFDM has been in use since 2003. Each reader had at least 4 years’ experience and MQSA certification. The six readers had 4–29 years of experience (mean, 15.8 years). The readers were blinded to the truth status (positive or negative) and to whether the images were lossy compressed. To minimize recall bias, readings of lossy-compressed and non-lossy-compressed cases from the
572
same patient were separated by a minimum of 2 months. This was done by separating readings into two phases that were at least 2 months apart for each reader. Each radiologist reader had his or her own set of images randomized into two blocks with varying sets of cases having undergone lossy data compression. The first block contained all 194 patients and the second block contained all 194 patients. Readers were scheduled to do readings of approximately 50 cases per session within each phase of the study. Before beginning the study, each reader completed a training session during which the reading session instructions, reader response form, and overall study design were reviewed. Each reader completed a five-examination practice reading session that contained a mixture of positive (cancer) and normal cases that were not contained within the study set.
Statistical Analysis The study was designed in the multireader multicase framework with each reader examining all cases in both modalities (lossy-compressed and non-lossy-compressed mammography examinations). Reader performance was summarized using the lesion localization fraction (number of lesions localized divided by number of lesions), true-positive fraction (number of correctly identified cancer cases divided by the total number of cancer cases), and false-positive fraction (number of normal or benign cases marked as cancerous divided by the number of normal or benign cases). A comparison of readers’ performance between modalities was done using the jackknife alternative free-response ROC (JAFROC) method [24]. Lesion localization and nonlocalization was used to create a figure of merit (FOM) in the JAFROC paradigm, defined as the probability that a cancer on an abnormal image is scored higher than a falsely marked location on a normal image. The JAFROC FOM between modalities, along with the other reader performance summary statistics, were compared using an ANOVA and results are
presented as the F statistic, with numerator and denominator degrees of freedom, along with the p value. A p value below 0.05 was considered statistically significant. A random effect was used within the JAFROC for reader and cases to infer results to the general population. The JAFROC analysis was implemented in JAFROC version 4.2 [25]. Two subgroup analyses were performed according to cancer type. Reader performance was tested with a cancer population composed of only calcification and a cancer population composed of only cancer cases with a mass or mass with a calcification. Interrater reliability was computed using the Fleiss kappa statistic and by converting reader responses from the modified BI-RADS to the presence of cancer or normal or benign findings. A formal sample size was not conducted as part of this project because we were unable to identify the variability between readers and modalities and within readers. A chart on sample size for multireader multicase studies using the AUC paradigm was consulted to determine the approximate sample size needed to test for a difference in diagnostic accuracy between the two imaging modalities [26]. Moderate accuracy, moderate difference between modalities, and moderate variability among observers suggested that each reader would have to read 78 cases. The final case number of 194 was thought to adequately account for any unseen variability in the study.
Results Pathologic Findings There were 194 cases included in the study with 81 patients having biopsy-proven cancer (42%). Of the 81 positive (cancer) cases, 45 (55.6%) were invasive ductal carcinoma, 22 (27.2%) were ductal carcinoma in situ, 3 (3.7%) were invasive lobular carcinoma, 3 (3.7%) were invasive cancer with ductal and lobular components, 2 (2.5%) were tubular carcinoma (subtype of invasive ductal carcinoma), 2 (2.5%) were infiltrating carcinoma
TABLE 1: Lesion Localization Fraction, True-Positive Fraction (TPF), and False-Positive Fraction (FPF), by Reader and Modality Reader
Lesion Localization Fraction
TPF
1
0.704/0.691
0.864/0.852
0.212/0.150
2
0.741/0.704
0.951/0.914
0.363/0.398
3
0.704/0.716
0.938/0.901
0.425/0.381
4
0.543/0.593
0.827/0.864
0.257/0.274
5
0.556/0.580
0.815/0.840
0.239/0.265
6
0.716/0.741
0.877/0.901
0.204/0.159
Mean
0.660/0.671
0.879/0.879
0.283/0.271
FPF
Note—Data are compressed/uncompressed fractions.
AJR:204, March 2015
Downloaded from www.ajronline.org by NYU Langone Med Ctr-Sch of Med on 05/31/15 from IP address 128.122.253.228. Copyright ARRS. For personal use only; all rights reserved
Use of Lossy Data Compression in Full-Field Digital Mammography not otherwise specified, 1 (1.2%) was lobular carcinoma in situ or ductal carcinoma in situ not otherwise specified, 1 (1.2%) was intraductal papillary carcinoma, 1 (1.2%) was pleomorphic lobular carcinoma (subtype of invasive lobular carcinoma), and 1 (1.2%) was adenoid cystic carcinoma. Of the 81 positive cases, 38 (47%) contained a suspicious mass lesion, 30 (37%) contained suspicious calcifications, and 13 (16%) cases contained a suspicious mass lesions with calcifications. All lesions measured 1 cm or smaller. Observer Performance Lesion localization fraction, true-positive fraction, and false-positive fraction between modalities are presented in Table 1. The differences in reader performance between imaging modalities using lesion localization (F1,5 = 0.68; p = 0.449), true-positive fraction (F1,5 = 0; p = 1.000), and false-positive fraction (F1,5 = 0.44; p = 0.536) were not statistically significant. There was no evidence of a difference in the JAFROC FOM between lossycompressed and uncompressed imaging modalities with a mean difference of −0.01 (95% CI, −0.03 to 0.01; F1,5 = 2.30; p = 0.189). Individual reader performance using the JAFROC FOM is presented in Table 2. There were no observed differences between the JAFROC
FOM of the imaging modalities in the cancer population subgroup containing just calcifications (F1,5 = 0.36; p = 0.569) or the cancer population consisting of masses and masses with calcifications (F1,5 = 2.83; p = 0.153). Reader performance for the subgroups is presented in Table 3 and Table 4. The interrater reliability was similar in the lossy-compressed and nonlossy-compressed modalities with kappa statistics of 0.529 and 0.504, respectively. Discussion For FFDM, the FDA currently permits lossy data compression of images only for secondary viewing. Many benefits, including much faster access to images for viewing, reduced storage costs, faster telemammography, and greater access to expert mammographer consultants, would result if a standard could be developed to safely enable lossy data compression for primary viewing and storage. We hypothesized that mammography image data compression with nonproprietary JPEG 2000 80:1 compression algorithm might represent a diagnostically lossless solution that would not degrade clinical accuracy. Previous studies in mammography have evaluated irreversible compression methods by looking at their effects on radiologist performance. Good et al. [27] applied the orig-
TABLE 2: Comparison of F igures of Merit (FOM) Between Compressed and Uncompressed Images for Jackknife A lternative Free-Response ROC (JAFROC) Reader
JAFROC FOM
1
0.791/0.804
2
0.713/0.699
3
0.739/0.766
4
0.681/0.69
5
0.705/0.704
6
0.806/0.830
Mean
0.739/0.749
Note—Data are compressed/uncompressed FOM values.
inal JPEG algorithm to 60 digitized (not FFDM) mammograms and evaluated the performance of eight readers in the detection of clusters of microcalcifications and masses with ROC analysis. This investigation found no statistically significant difference in detecting masses, but a significant difference was found for the detection of microcalcification clusters when very high levels of compression were used (about 100:1).
TABLE 3: Reader Performance in Both Modalities for Mass Cancer Types (Study Types Mass or Both) False-Positive Fraction
JAFROC FOMa
Reader
Lesion Localization Fraction
True-Positive Fraction
1
0.627/0.706
0.843/0.863
0.212/0.150
0.745/0.809
2
0.765/0.745
1.000/0.980
0.363/0.398
0.733/0.737
3
0.647/0.686
0.922/0.902
0.425/0.381
0.704/0.752
4
0.529/0.569
0.863/0.863
0.257/0.274
0.671/0.679
5
0.627/0.627
0.863/0.863
0.239/0.265
0.748/0.725
6
0.706/0.745
0.902/0.902
0.204/0.159
0.799/0.831
Mean
0.650/0.680
0.899/0.895
0.283/0.271
0.734/0.755
Note—Data are compressed/uncompressed values. JAFROC = jackknife alternative free-response ROC, FOM = figures of merit. ap = 0.153.
TABLE 4: Reader Performance in Both Modalities for Calcification Cancer Types (Study Type Calcification) Reader
Lesion Localization Fraction
True-Positive Fraction
False-Positive Fraction
JAFROC FOMa
1
0.833/0.667
0.900/0.833
0.212/0.150
0.868/0.795
2
0.700/0.633
0.867/0.800
0.363/0.398
0.680/0.634
3
0.800/0.767
0.967/0.900
0.425/0.381
0.799/0.790
4
0.567/0.633
0.767/0.867
0.257/0.274
0.698/0.708
5
0.433/0.500
0.733/0.800
0.239/0.265
0.631/0.670
6
0.733/0.733
0.833/0.900
0.204/0.159
0.816/0.829
Mean
0.678/0.656
0.844/0.850
0.283/0.271
0.749/0.738
Note—Data are compressed/uncompressed values. JAFROC = jackknife alternative free-response ROC, FOM = figures of merit. ap = 0.569.
AJR:204, March 2015 573
Downloaded from www.ajronline.org by NYU Langone Med Ctr-Sch of Med on 05/31/15 from IP address 128.122.253.228. Copyright ARRS. For personal use only; all rights reserved
Kovacs et al. Additional performance-based studies have used the newer wavelet-transform methods of image compression, such as the JPEG 2000 standard or set partitioning in hierarchic trees (SPIHT) algorithm. Perlmutter et al. [28] used 57 digitized mammograms compressed using SPIHT to determine whether the use of lossy compression resulted in patient care decisions that were different from those that resulted from the use of uncompressed images. Their study found no statistically significant difference between original analog or digitized mammography examinations and their compressed counterparts at an 80:1 compression ratio. Penedo et al. [2] digitized 112 mammographic images (64 abnormal and 48 normal) and applied compression at 40:1 and 80:1 using the JPEG 2000 and SPIHT methods, and five experienced radiologists were asked to locate and rate clusters of microcalcifications and masses on the freeresponse ROC data acquisition paradigm. They determined that lossy compression of digitized mammographic data at 80:1 with JPEG 2000 or SPIHT could be performed without decreasing the rate of detection of clusters of microcalcifications and masses. To our knowledge, this work represents the first large clinical study to show that no degradation of diagnostic accuracy resulted when implementing JPEG 2000 80:1 lossy compression on FFDM images. Of note, this study also found that, in grouping the lesions as masses (Table 3) versus calcifications (Table 4), there was still no statistically significant difference between the lossy-compressed and non-lossy-compressed image sets. This is in contradistinction from previous work, such as by Suryanarayanan et al. [15], which has suggested that lossy compression can adversely affect the detection of calcifications. The implications of this study are considerable. With validation of diagnostically lossless data compression at the level of 80:1 for FFDM, a new era of highly practical telemammography could emerge, with potentially dramatic improvement in cancer detection rates if images are funneled to expert readers. Of note, documented variations in reader accuracy and sensitivity rates far exceed the measurable benefit provided by technologies such as FFDM, computer-aided detection, and breast tomography. That is not to say that these beneficial technologies should be ignored, but there is nevertheless a huge benefit to be obtained by shifting mammography to the more accurate and sensitive readers, which may actually lower costs,
574
whereas the FFDM, computer-aided detection, and tomography require capital investments. The net result could be higher cancer detection rates with lower-cost mammography. Data compression will become even more important with breast tomography because the file sizes are significantly larger. Data compression also provides benefits to other important constituents. The dramatic reduction in file sizes not only increases the transportability of FFDM examinations for the purpose of primary reading but also improves access for nonradiologists who require remote viewing, such as oncologists or breast surgeons. Finally, there is a growing demand for patient access to medical images. Data compression makes it more feasible for patients to control, own, and view their own personal images. Limitations of the study included the following: there was a limited sample size chosen for the study of 194 cases, of which 81 were positive for cancer, and a limited number of readers (six in total). These numbers were chosen to provide sufficient statistical power. Although the case number is limited, it does represent an increase over previous studies (of different design) involving evaluation of lossy compression of mammographic images. Recall bias is a potential limitation of the study but is expected to be minimal because we instituted a minimum 2-month interval between sessions for each reader. The likelihood of recall bias was also decreased because the cases were displayed in a random order. It has been shown previously that some degree of memory washout occurs within days [29]. An additional potential limitation of the study is that although the cases did come from four different radiology practices, these practices were limited to two practices on the West Coast and two on the East Coast, which does represent some limitation in geographic variation in cases. Prior examinations were not available for comparison purposes. This may have made it challenging at times for the reader to decide on an appropriate rating for lesions that were borderline for having benign characteristics. This may have affected overall sensitivity and specificity numbers but was not expected to (and did not) significantly vary overall on the basis of the compressed or uncompressed status of the case. Of note, one reader (reader 3) consistently rated a higher proportion of perceived low probability (likely benign finding) cases as a “3” rather than a “2” which did lead to more false-positives (lowering specificity), but this also did
not significantly vary according to the compressed or uncompressed status of the case. The method of localization is also a potential limitation of the study design. Because we chose to localize via quadrant, it is conceivable that a reader misinterpreted a benign finding as malignant, but it was counted as a true-positive or localization if there was a separate true malignant finding of the same type (i.e., mass, calcifications, or both) within the same quadrant. We think, however, that this scenario would have occurred extremely infrequently if at all. Another way in which the quadrant system represented a potential limitation of the study design is for lesions that straddled the boundary between two or more quadrants. There were a number of such lesions in the dataset (23 of 194 cases), and for these, at the time of determination of ground truth, two or more quadrants were listed as possible correct choices. Conclusion This investigation shows no degradation of clinical accuracy when implementing JPEG 2000 80:1 lossy compression for primary reading of FFDM, further suggesting that the FDA should consider changing its policy of prohibiting primary reading of lossy-compressed FFDM images. If such lossy-compressed images are considered equivalent to the original and therefore are permissible under MQSA, costs would be lower and access to mammographer experts would be enhanced. Primary reading and archive of lossy-compressed FFDM images should be permitted, given no difference in clinical accuracy in interpretation of lossycompressed versus lossless or uncompressed versions of images. Acknowledgments We thank all the readers from Santa Barbara Radiology Medical Group who volunteered their time for this study. We also thank Elizabeth Krupinski for her insight regarding questions on study design and Matthew Borzage for his work in initial development of the study protocol and design. References 1. American College of Radiology. ACR-AAPM-SIIM technical standard for electronic practice of medical imaging (resolution 35). American College of Radiology website. www.acr.org/~/media/ACR/ Documents/PGTS/standards/ElectronicPractice MedImg.pdf. Published 2007. Revised 2012. Accessed April 2, 2014
AJR:204, March 2015
Downloaded from www.ajronline.org by NYU Langone Med Ctr-Sch of Med on 05/31/15 from IP address 128.122.253.228. Copyright ARRS. For personal use only; all rights reserved
Use of Lossy Data Compression in Full-Field Digital Mammography 2. Penedo M, Souto M, Tahoces PG, et al. Free-response receiver operating characteristic evaluation of lossy JPEG2000 and object-based set partitioning in hierarchical tree compression of digitized mammograms. Radiology 2005; 237:450–457 3. Brown DE. Archiving: fundamentals of storage technology. Society for Imaging Informatics in Medicine website. siim.org/books/archiving/TOC. Accessed April 2, 2014 4. U.S. Food and Drug Administration. The Mammography Quality Standards Act final regulations: modifications and additions to policy guidance help system #9. U.S. Food and Drug Administration website. www.fda.gov/Radiation-EmittingProducts/ MammographyQualityStandardsActandProgram/ DocumentArchives/ucm114207.htm. Published April 19, 2006. Accessed June 6, 2013 5. Sickles EA, Miglioretti DL, Ballard-Barbash R, et al. Performance benchmarks for diagnostic mammography. Radiology 2005; 235:775–790 6. Sickles EA, Wolverton DE, Dee KE. Performance parameters for screening and diagnostic mammography specialist and general radiologists. Radiology 2002; 224:861–869 7. Birdwell RL, Bandodkar P, Ikeda DM. Computeraided detection with screening mammography in a university hospital setting. Radiology 2005; 236:451–457 8. Morton MJ, Whaley DH, Brandt KR, Amrami KK. Screening mammograms: interpretation with computer-aided detection—prospective evaluation. Radiology 2006; 239:375–383 9. Freer TW, Ulissey MJ. Screening mammography with computer-aided detection: prospective study of 12,860 patients in a community breast center. Radiology 2001; 220:781–786 10. Beam CA, Conant EF, Sickles EA. Association of volume and volume-independent factors with ac-
curacy in screening mammogram interpretation. J Natl Cancer Inst 2003; 95:282–290 11. Elmore JG, Miglioretti DL, Carney PA. Does practice make perfect when interpreting mammography? Part II. J Natl Cancer Inst 2003; 95:250–252 12. Esserman L, Cowley H, Eberle C, et al. Improving the accuracy of mammography: volume and outcome relationships. J Natl Cancer Inst 2002; 94:369–375 13. Linver MN, Paster SP, Rosenberg RB, et al. Improvement in mammography interpretation skills in a community radiology practice after dedicated teaching courses: a two-year audit of 38,633 cases. Radiology 1992; 184:39–43 14. Kallergi M, Lucier BJ, Berman CG, et al. Highperformance wavelet compression for mammography localization response operating characteristic evaluation. Radiology 2006; 238:62–73 15. Suryanarayanan S, Karellas A, Vedantham S, et al. Detection of simulated lesions on data-compressed digital mammograms. Radiology 2005; 236:31–36 16. Betts BJ, Aiyer A, Li J, et al. Management and lesion detection effects of lossy image compression on digitized mammograms. In: Karssemeijer N, Thijssen M, Hendriks J, van Ernin L, eds. Proceedings of the 4th International Workshop on Digital Mammography. Nijmegen, The Netherlands: Springer Netherlands, 1998:449–456 17. Destounis S, Somerville P, Murphy P, et al. Perceived sufficiency of full-field digital mammograms with and without irreversible image data compression for comparison with next-year mammograms. J Digit Imaging 2011; 24:66–74 18. Kang BJ, Kim HS, Park CS, et al. Acceptable compression ratio of full-field digital mammography using JPEG 2000. Clin Radiol 2011; 66:609–613 19. Liang Z, Du X, Yang Y, et al. Effects of different compression techniques on diagnostic accuracies
of breast masses on digitized mammograms. Acta Radiol 2008; 49:747–751 20. Visser R, Oostveen L, Karssemeijer N. Lossless compression of digital mammograms. In: Astley SM, Brady M, Rose C, Zwiggelaar R, eds. Proceedings of the 8th International Workshop on Digital Mammography. Manchester, UK: IWDM, 2006:533–540 21. Lehmann TM, Abel J, Weiss C. The impact of lossless image compression to radiographs. Proc SPIE 2003; 6145:290–297 22. Slone RM. Assessment of visually lossless irreversible image compression: comparison of three methods by using an image-comparison workstation. Radiology 2000; 215:543–553 23. Koff D, Bak P, Brownrigg P, et al. Pan-Canadian evaluation of irreversible compression ratios (“lossy” compression) for development of national guidelines. J Digit Imaging 2009; 22:569–578 24. Chakraborty DP, Berbaum KS. Observer studies involving detection and localization: modeling, analysis and validation. Med Phys 2004; 31:2313–2330 25. Chakraborty DP. Dev Chakraborty’s FROC website. www.devchakraborty.com. Published 2005. Accessed November 12, 2014 26. Obuchowski NA. Sample size tables for receiver operating characteristic studies. AJR 2000; 175:603–608 27. Good WF, Sumkin JH, Ganott M, et al. Detection of masses and clustered microcalcifications on data compressed mammograms: an observer performance study. AJR 2000; 175:1573–1576 28. Perlmutter S, Cosman P, Gray R, et al. Image quality in lossy compressed digital mammograms. Signal Process 1997; 59:189–210 29. Hardesty LA, Ganott MA, Hakim CM, et al. “Memory effect” in observer performance studies of mammograms. Acad Radiol 2005; 12:286–290
AJR:204, March 2015 575
