The Journal of Emergency Medicine, Vol. -, No. -, pp. 1–8, 2014 Copyright Ó 2014 Elsevier Inc. Printed in the USA. All rights reserved 0736-4679/$ - see front matter

http://dx.doi.org/10.1016/j.jemermed.2013.12.002

Selected Topics: Emergency Radiology

EMERGENCY DEPARTMENT VARIATION IN UTILIZATION AND DIAGNOSTIC YIELD OF ADVANCED RADIOGRAPHY IN DIAGNOSIS OF PULMONARY EMBOLUS Dana R. Kindermann, MD, MPH,* Melissa L. McCarthy, SCD,† Ru Ding, MS,† William J. Frohna, MD,‡ Jonathan Hansen, MD MBA,§ Kevin Maloy, MD,‡ David P. Milzman, MD,‡ and Jesse M. Pines, MD, MBA, MSCE† *Department of Emergency Medicine, George Washington University Hospital, Washington, DC, †Departments of Emergency Medicine and Health Policy, George Washington University, Washington, DC, ‡Department of Emergency Medicine, Medstar Washington Hospital Center, Washington, DC, and §Department of Emergency Medicine, Medstar Franklin Square Hospital Center, Baltimore, Maryland Reprint Address: Dana R. Kindermann, MD, MPH, 1047 Curtis Street, Albany, CA 94706

, Abstract—Background: There is growing pressure to measure and reduce unnecessary imaging in the emergency department. Objective: We study provider and hospital variation in utilization and diagnostic yield for advanced radiography in diagnosis of pulmonary embolism (PE) and to assess patient- and provider-level factors associated with diagnostic yield. Methods: Retrospective chart review of all adult patients presenting to four hospitals from January 2006 through December 2009 who had a computed tomography or ventilation/perfusion scan to evaluate for PE. Demographic data on the providers ordering the scans were collected. Diagnostic yield (positive scans/total scans ordered) was calculated at the hospital and provider level. The study was not designed to assess appropriateness of imaging. Results: There was significant variation in utilization and diagnostic yield at the hospital level (chi-squared, p < 0.05). Diagnostic yield ranged from 4.2% to 8.2%; after adjusting for patient- and provider-level factors; the two hospitals with an emergency medicine residency training program had higher diagnostic yields (odds ratio [OR] 2.0, 95% confidence interval [CI] 1.6–2.5 and OR 1.9, 95% CI 1.5–2.4). There was no significant variation in diagnostic yield among the 90 providers after adjusting for patient, hospital, and provider characteristics. Providers with < 10 years of experience had lower odds of diagnosing a PE than more experienced graduates (OR 0.8, 95% CI 0.6– 0.9). Conclusions: Although we found significant variation in utilization of advanced radiography for PE and diagnostic yield at the hospital level, there was no significant variation

at the provider level after adjusting for patient-, hospital-, and provider-level factors. Ó 2014 Elsevier Inc. , Keywords—imaging; resource utilization; diagnostic yield

INTRODUCTION Background Pulmonary embolism (PE) is often considered among patients presenting with chest pain and shortness of breath. Early studies estimated the mortality rate of untreated PE as high as 26%, whereas newer studies have suggested that it may be substantially less (1,2). Clinical and laboratory pathways can often be used to rule out PE without imaging, but many patients get a computed tomography (CT) scan or a ventilation-perfusion (V/Q) scan to definitively rule out PE. Significance Studies have shown increasing use of advanced radiography in the emergency department (ED), as well as geographic and hospital-level variation in utilization (3–8). Overutilization of CT scans is associated with higher costs, radiation exposure, time in the ED, and risks of both contrast-induced nephropathy and allergic

RECEIVED: 25 February 2013; FINAL SUBMISSION RECEIVED: 2 October 2013; ACCEPTED: 3 December 2013 1

2

D. R. Kindermann et al.

reactions to contrast dye (6,9–11). The increasing use of CT scans has led to decreasing diagnostic yields, defined as the proportion of positive tests among all tests ordered (8,12–14). Multiple validated clinical decision rules (CDRs) can help guide physicians’ decision of who needs a CT scan (15–18). However, studies have shown variable physician use of these rules to risk-stratify patients, along with high use of CT in low-risk patients with negative D-dimer results (19,20). No studies, to our knowledge, have examined the variability of approach to ruling out PE within groups of emergency physicians. Goals of this Investigation We studied variation in utilization and diagnostic yield— defined as the number of positive tests divided by the total tests ordered—for advanced radiography in diagnosis of pulmonary embolism at the hospital and provider level, as well as the factors associated with these differences in diagnostic yield. Our study focuses on utilization and diagnostic yield. We were not able to assess appropriateness of imaging, which may be based on CDRs, laboratory tests, and clinical gestalt. MATERIALS AND METHODS Study Design/Setting This was a retrospective chart review of all patients age 18 years and older, presenting to four EDs from January 1, 2006 to December 31, 2009 in one health system, who had a CT chest or V/Q scan specifically to diagnose or rule out PE. Each provider had to specifically order a PE-protocol chest CT scan (timing of intravenous contrast is different for PE-protocol scans compared to other CT scans of the chest). V/Q scans are ordered only to diagnose PE. Only scans ordered by physicians who had ordered at least 50 CT or V/Q scans to evaluate for PE during the study period were included to capture a sufficient number of scans to estimate the diagnostic yield rate per provider. We also excluded repeat scans on the same patient. We felt the decision to order multiple scans on patients would likely involve different clinical scenarios (sicker or more high-risk patients, worsening symptoms in patients who have already received a diagnosis of PE, etc.). The health system’s institutional review board approved the research protocol, including waiver of informed consent. Of the four EDs, two were urban, academic tertiary care centers with an emergency medicine residency training program. One was an urban community hospital, and another was a suburban community hospital; both of these hospitals had nonemergency medicine residents at

times rotating through the ED. All hospitals were located within an approximate 60-mile radius. Specific information on the hospitals is included in Table 1. Data Processing An information technology specialist extracted charts from the electronic medical record (EMR) where a PEprotocol CT or V/Q scan was ordered in the ED. For each patient, we also automatically extracted data on the date and time of visit, patient age, sex, race, chief complaint as noted by the triage nurse, radiology report, and resident or student involvement recorded in the medical chart. For each patient, a focused chart review, including a review of the full radiology report, was performed to assess whether a PE was identified. Five trained chart abstractors did this review, which focused on reading each radiology report. The indication for the CT was listed at the beginning of the radiology report. Patients with CTs for alternative reasons (such as to rule out pneumonia or trauma) were excluded. Technically difficult studies were identified by abstractors using predetermined criteria (i.e., the segmental or sub-segmental arteries were not visualized). These CT scans were labeled as indeterminate. CT scans were classified as positive if the radiologist specifically stated a clot was visualized in the report. V/Q scans were classified as positive if the scan was read as ‘‘intermediate,’’ ‘‘high,’’ or ‘‘very high’’ probability. ‘‘Low’’ and ‘‘very low’’ probability V/Q scans were classified as negative. Including the intermediate scans as positive was a conservative approach, though consistent with prior publications on V/Q scans (21). Chart reviews were conducted in concordance with standard chart-review methods in emergency medicine (22). In a 5% random sample of the charts abstracted by a second, blinded reviewer, the kappa agreement on classification was 0.96 for CT scans, and 0.97 for V/Q scans. We also collected demographic information on the providers ordering the scans. These data were provided by leadership at each hospital and included age, gender, residency training type, and year of completion. The ordering provider was defined as the physician listed on the radiology report as having ordered the scan, and was the attending physician taking care of the patient. There were no mid-level providers who ordered enough scans (>50) to be included in our analysis. At two hospitals, residents were involved in cases, but the overseeing attending physician was listed as the ordering physician. In these cases, we recorded whether a resident was also involved in the case. In one of the hospitals (Hospital 2), during the last year of the study, there was sometimes a triage physician who ordered the scan, and his/her

Advanced Radiography in Diagnosis of Pulmonary Embolus

3

Table 1. Description of Study Setting and Study Population by Hospital (1/2006–12/2009)

Study setting Average annual ED volume Average admission rate Hospital setting Academic/community Trauma level Average patient age, years % African American patients Critical care admissions Study population Total adult ED visits Total scans ordered Scans/1000 ED visits % V/Q scans ordered§ % Subjects seen by residentk Positive diagnostic yield rate CT scan positive diagnostic yield rate (excluding V/Q scans)

Hospital 1

Hospital 2

Hospital 3

Hospital 4

103,000 21% Suburban Community Level III 38 26% 1.2%†

78,800 25% Urban Academic Level I 47* 80% 2.8%†

35,000 18% Urban Academic Level II 40 43% 1.1%‡

57,250 20% Urban Community Level III 37 74% 1.3%†

412,371 5385 13.2 4.1% 0% 4.2% 4.2%

315,460 3850 12.5 17.5% 22% 8.2% 7.9%

139,950 1692 12.6 4.6% 32.3% 7.5% 7.5%

228,790 1956 8.9 11.8% 0% 4.7% 4.5%

ED = emergency department; V/Q = ventilation-perfusion; CT = computed tomography. * Hospital 2 does not see pediatric patients in the ED; the average age of the remaining hospitals includes all comers, including pediatric patients. † Includes both intermediate care and intensive care admissions. ‡ Includes only intensive care admissions (no intermediate care unit in hospital–patients who would need intermediate care status are admitted to the intensive care unit at this hospital). § Percentages for the following five rows use the total scans ordered at each hospital as the denominator. k These data were not accessible from Azyxxi, but < 1% of patients are seen by residents (nonemergency medicine) in the ED at Hospital 1 and 4.

name was listed on the report as the ordering provider. That physician may have performed a more limited history and physical examination and likely did not ultimately take care of the patient in the ED. Data were extracted into a Microsoft Excel (Microsoft Corporation, Redmond, WA) worksheet, and Stata 11 (Stata Corporation, College Station, TX) was used for analysis. Data Analysis The population was described using standard descriptive statistics. The main outcome for the study was the presence of a PE. Each patient visit was classified as being either PE-positive, PE-negative, or indeterminate based on the radiology report. We calculated a positive diagnostic yield rate (PE-positive scans/total scans ordered) by provider and hospital. A positive scan was defined as a positive CT scan or an intermediate-, high-, or very high-probability V/Q scan. We conducted a bivariate analysis to examine the relationship between PE and patient (i.e., age, gender, race, chief complaint, and time of visit), provider (type of training, years of experience, gender, and whether a resident was involved in care), and site characteristics using a chi-squared test statistic. We had a limited number of patient-level variables that were available from the ED information system to use for risk adjustment and we included all of them in the analyses. As a result, we were not able to assess ‘‘appropri-

ateness’’ of imaging or use of CDRs to determine which patients should have a scan. To determine if the PE diagnostic yield varied significantly across hospitals and among providers within each hospital, we modeled the presence of a PE as a function of site, provider, and patient characteristics. We used a hierarchical logistic regression model that accounted for the clustering of providers working at the same hospital as well as the clustering of patients seen by the same provider (23). The site and patient characteristics were included in the multilevel model as fixed effects, and the providers were entered as a random effect. For each independent variable included in the model, the adjusted odds ratio (OR) and 95% confidence interval (CI) are reported. To determine if there was significant variation in the diagnostic PE rate between the providers, we computed the log likelihood ratio test based on models that included and excluded the providers (we used a p value of 0.1 for the likelihood ratio test). We also conducted a sensitivity analysis in which we only included CT scans (V/Q scans were dropped from this analysis). RESULTS Hospital-level Data A total of 16,283 scans were ordered during the study period; 2025 (12.4%) were excluded because they were ordered for reasons other than to rule out PE. Another 1057

4

D. R. Kindermann et al.

Table 2. Characterization of Study Population With Positive Scans by Patient Groups Patient Characteristics

Patients in Group (Overall N = 12,883)

Age† < 50 years 6744 $ 50 years 6139 Gender† Male (ref) 4490 Female 8383 Race† White (ref) 5791 African 6675 American Other 417 Chief complaint† Chest pain (ref) 4767 Shortness of 3091 breath Leg pain/leg 309 swelling Other‡ 4716 Time of visit 4973 AM (12:00 AM– 11:59 AM) 7910 PM (12:00 PM– 11:59 PM) Year of visit 2006 3504 2007 3169 2008 3427 2009 2283 Patient seen by resident† Yes 1404 No 11,479

PE+ (CT and V/Q Scans) n = 779

PE+ (CT only) n = 682*

5.2% 6.6%

5.0% 6.5%

7.1% 5.3%

6.8% 5.1%

5.1% 6.7%

5.1% 6.4%

5.9%

5.0%

3.7% 7.6%

3.5% 7.2%

16.7%

16.1%

6.1%

6.0%

5.9%

5.8%

5.8%

5.8%

5.9% 6.0% 5.6% 6.1%

5.8% 5.6% 5.4% 6.1%

7.8% 5.6%

7.4% 5.5%

CT = computed tomography; V/Q = ventilation/perfusion; PE = pulmonary embolism. * The overall number for CT scans ordered in this study was 11,951. † Signifies significant differences were found between the observed and expected percentage of patients with PE for this variable at the p < 0.05 level using chi-squared testing. ‡ Other included complaints such as back pain, abdominal pain, weakness, syncope, etc.

(6.5%) were excluded because they were ordered by providers who had ordered < 50 scans, and 318 (1.8%) were excluded because they were repeat scans on the same patient. Table 1 describes the four study sites and their patient populations. The hospital scan rate varied from 8.9 to 13.2/ 1000 patients (p = 0.02) across hospitals. Hospitals 2 and 4 used a higher percentage of V/Q scans compared to the other two hospitals. The overall diagnostic yield was 5.9%. However, it ranged from 4.2% to 8.2% (p < 0.01) across the four hospitals. When the positive PE rate was based only on CT scan results, the diagnostic yield rates changed very little across the four hospitals. Patient and Provider-level Data Table 2 displays the PE results by patient characteristics. Older age, male gender, African-American race, and a

chief complaint of shortness of breath or leg pain were associated with higher PE diagnostic yield rates. There were 90 providers in this study who ordered > 50 CT or V/Q scans to evaluate for PE during this study period. The median number of scans ordered by a provider in this group was 124 (interquartile range 93–174). Table 3 displays the PE diagnostic yield rate by provider characteristics. The majority of the providers were male, < 10 years out of residency, and emergency medicine trained. As a group, more-experienced providers (those > 10 years out of residency) had a higher PE diagnostic yield rate compared to less-experienced providers. Positive Diagnostic Yield Rate by Patient, Provider, and Hospital Characteristics Table 4 shows the adjusted OR and 95% CI for patient, provider, and hospital factors that were associated with PE diagnostic yield. The adjusted OR for PE was significantly lower among younger patients and female patients, and higher among those presenting with shortness of breath (compared to other chief complaints). The odds of diagnosing a PE were significantly lower among providers with less experience. There were no significant differences in the PE diagnostic yield rate among the providers within each site (p value for the log likelihood ratio test > 0.10). The difference in the adjusted positive diagnostic yield rate among the providers within each site ranged from 0.9% at Hospital 4 (lowest provider rate 5.5% and highest provider rate 6.4%) to 1.6% at Hospital 1 (lowest provider rate 5.0% and highest provider rate 6.6%). Finally, the adjusted OR of a positive scan was twice as high in the two hospitals with the emergency medicine residency training program (Hospitals 2 and 3), but whether a resident was involved in the case had no significant effect. When the definition of a positive diagnostic yield rate was based only on a positive CT scan (excludes all V/Q scans), the results are similar to the main results. DISCUSSION The overall mean positive diagnostic yield in our study was 5.9%. This was significantly lower than the diagnostic yield reported in studies in the late 1990s and early 2000s, which ranged from 15–20%, but comparable to yields found in more recent studies (8,12–14). The diagnostic yield did not vary significantly from 2006 through 2009. The number of scans ordered per 1000 ED visits also did not change significantly over time. This may suggest that we’ve reached a new, lower set point in utilization and diagnostic yield since the widespread availability of multi-detector CT scans in EDs. This is in contrast to a recent study documenting

Advanced Radiography in Diagnosis of Pulmonary Embolus Table 3. Percent Distribution of Provider Characteristics by Positive Diagnostic Yield Rate for Pulmonary Embolism (1/2006–12/2009) Provider Characteristics

Total Providers n = 90

Gender Male 56 Female 34 Provider experience† < 10 years 62 $ 10 years 28 Specialty training Emergency 89 medicine Other 1

Total Scans Completed n = 12,883

PE + n = 779*

PE+ (CT only) n = 682*

8472 4729

5.5% 6.1%

5.9% 5.3%

9515 3664

5.4%‡ 7.2%‡

5.2%‡ 6.9%‡

12,884

5.9%

5.7%

317

5.9%

5.8%

PE = pulmonary embolism; CT = computed tomography. * Each cell represents the diagnostic yield within the listed category. † For provider age and experience, we used the provider’s age at the beginning of the study (2006  provider’s year of birth). ‡ Signifies that significant differences were found between the observed and expected percentage of patients with PE for this variable at the p < 0.05 level using chi-squared test statistic.

increasing use of advanced radiography nationally for patients presenting to the ED with chest pain (24). This difference may be due to local practice patterns in our study or because we used more recent data, suggesting we’re reaching a plateau on utilization rates. This study looked at patient- and provider-level characteristics associated with diagnostic yield. At the patient level, our results were consistent with previous studies (25–27). Older and male patients had higher odds of having a PE in our study. Patients presenting with a chief complaint of shortness of breath also had higher odds, and those with chest pain had lower odds. This is similar to previous studies showing a low prevalence of the diagnosis of pulmonary embolism among all comers presenting with chest pain, and higher specificity of shortness of breath for the diagnosis of pulmonary embolism (26,27). At the hospital level, there was significant variation in both the utilization and diagnostic yield, as has been shown in other studies (6,7,28). Variation in utilization of CT has been found up to 1.5 times across regions in a national level study and up to fourfold among hospitals in a study looking at CT use for appendicitis (7,28). We found similar, though slightly smaller, differences in our study. Utilization varied 1.5 times between the highest ordering hospital (Hospital 1) and the lowest ordering hospital (Hospital 4), whereas diagnostic yield varied nearly twofold between Hospitals 2 and 1. Hospitals 2 and 3 had both similar utilization (12.5 and 12.6 per 1000 patient visits, respectively), and the highest diagnostic yields (8.2% and 7.5%, respectively). These trends in utilization and

5 Table 4. Adjusted Odds of a Positive CT or V/Q Scan (OR, 95% CI), Hospitals 1–4, Using a Hierarchical Logistic Regression Model that Accounted for Clustering of Physicians Within Sites and Patients Treated by the Same Provider (1/2006–12/2009)

Patient characteristics Age < 50 years Female gender African American Chief complaint Chest pain Shortness of breath Other chief complaint Patient seen in PM (12:00 PM–11:59 PM) Hospital Hospital 1 Hospital 2 Hospital 3 Hospital 4 Year 2006 2007 2008 2009 Provider characteristics Provider with < 10 years experience Female EM trained Patient seen by resident

OR (PE+) n = 12,883

CT Scans only n = 12,036

0.84 (0.73, 0.97) 0.74 (0.64, 0.84) 1.08 (0.96, 1.22)

0.84 (0.74, 0.96) 0.71 (0.60, 0.85) 1.08 (0.99, 1.19)

0.71 (0.52, 0.98) 1.44 (1.25, 1.65) Ref. 0.95 (0.82, 1.11)

0.72 (0.51, 1.01) 1.44 (1.22, 1.70) Ref. 0.97 (0.84, 1.11)

Ref. 2.00 (1.79, 2.22) 1.88 (1.77, 1.98) 1.20 (1.06, 1.35)

Ref. 1.91 (1.74, 2.09) 1.88 (1.78, 1.98) 1.21 (1.10, 1.34)

Ref 1.10 (0.95, 1.27) 0.96 (0.79, 1.15) 1.13 (0.98, 1.32)

Ref. 1.08 (0.91, 1.28) 0.96 (0.78, 1.17) 1.14 (0.93, 1.40)

0.75 (0.67, 0.84)

0.77 (0.65, 0.91)

0.95 (0.88, 1.04) 0.96 (0.62, 1.50) 0.96 (0.85, 1.07)

0.94 (0.86, 1.03) 0.97 (0.58, 1.61) 0.93 (0.81, 1.06)

CT = computed tomography; V/Q = ventilation/perfusion; OR = odds ratio; PE = pulmonary embolism; EM = emergency medicine. Bolded items indicate statistical significance.

diagnostic yield at Hospitals 2 and 3 seem to be independent of acuity level in the ED; Hospital 3 had the lowest acuity rates (as measured by admission and intensive care unit admission), and Hospital 2 had the highest. The two hospitals are located approximately 5 miles apart, and several physicians work at both facilities. Local practice patterns may have been the driving force for the higher positive diagnostic yield in these two hospitals. Hospitals 2 and 3 are the main teaching sites of a shared emergency medicine residency training program. The impact of teaching may have affected attending physician use of clinical decision rules, leading to less testing. One previous study found higher utilization of advanced radiography in university compared to community hospitals, but as far as we could see, none have assessed diagnostic yield (29). At the provider level, the only significant predictor of diagnostic yield was years in practice. Providers who had been in practice < 10 years had significantly lower diagnostic yields compared to their colleagues with more experience. This may be because younger, less-

6

experienced physicians may have a more conservative approach when ordering tests (i.e., they may be less willing to take risks that might lead to missing a PE), or they may be less likely to follow clinical decision rules to help guide testing. However, this effect was not a dramatic one. It is likely that other factors such as differences in practice style, risk tolerance, malpractice fear, or clinical judgment may explain some of the variation. Previous studies have found that individual risk tolerance and malpractice fear impact emergency physicians’ decision to admit patients or order abdominal CT scans (30–32). Future work should focus on identifying factors associated with lower yield and interventions to improve yield, such as directed feedback or electronically based integration of decision rules. There was substantial unadjusted variation across the four sites between the provider with the highest diagnostic yield (20%) and that with the lowest (1%). Interestingly, when we examined the provider variation in the multilevel regression model, the adjusted variation by provider was markedly reduced and not significant. This may have been a sample-size issue. Based on the provider variation we observed and the number of scans ordered by each physician, we would have needed a minimum of 360 scans per provider to detect significant differences in diagnostic yield rate across providers. It would take most providers at least 8 years to order this number of scans; 75% of providers ordered < 124 scans in the 4-year study period. Another perspective is that after adjusting for multiple patient- and hospital-level characteristics and clustering by site, there is actually little variation in practice. Individual patient risk factors, the typical acuity of an ED’s patient population, and ED- or hospital-level utilization patterns and risk tolerance may be more significant drivers of utilization and diagnostic yield than individual physician differences. This suggests that looking at unadjusted provider diagnostic yield as a measure of appropriateness of imaging use may be inaccurate and misleading. In our study, there was a 20-fold difference in unadjusted diagnostic yield between providers. Looking only at unadjusted provider diagnostic yield without considering ED-, hospital-, and patient-level characteristics could lead to misguided conclusions about provider practice patterns. Another group found similar findings of large unadjusted provider diagnostic yield for PE over a 4-year study period (5% to 25%), with no significant difference after adjustment for patient- and provider-level characteristics (33). Their findings also suggest that using unadjusted administrative databases to assess provider utilization could lead to inaccurate conclusions. There is a growing consensus on the need for decreasing unnecessary imaging in the ED. The Centers for Medicare & Medicaid Services has sought to decrease

D. R. Kindermann et al.

‘‘inappropriate’’ imaging in the ED through their outpatient quality measurement initiative. One proposed measure would look at ‘‘appropriateness’’ of head CT utilization in the ED using the metric of unadjusted diagnostic yield (positive scans/total scans ordered) (34). Hospital-level data on diagnostic yield would be collected and published, and there would likely be financial incentives tied to performance on this measure. The collection and publishing of individual providers’ diagnostic yields on various imaging modalities may be the next step. The results of this study suggest that understanding variation in utilization at the provider level is more complex. Medicare claims data that relies on retrospective data that includes only positive scans and total scans ordered could lead to erroneous conclusions about true variation in provider diagnostic yield. The National Quality Forum’s Imaging Efficiency Measure would specifically measure potentially unavoidable use of CT for the evaluation of low-risk PE patients through use of clinical decision rules and D-dimers (35). A recent prospective study using the measure found that one-third of scans were potentially avoidable, but found few patient-level predictors of avoidable imaging (36). Our study suggests that although individual physician variation may play some role in imaging efficiency, hospital-level utilization patterns, local practice, and patient preference may be more predictive of avoidable imaging. Further research would help to address several unanswered questions from this study. Ideally, a study would be able to assess a provider’s ability to balance unnecessary ordering of CT or V/Q scans with a very low miss rate. To do this, a study would likely need to assess the outcomes of patients who are presumed to be low risk for PE but who don’t get a CT or V/Q scan in the ED to try to capture each provider’s ‘‘miss rate.’’ It would also have been useful to study providers’ use of clinical decision rules to risk-stratify patients and the impact of this use with positive diagnostic yield. Finally, future studies should look at the role of patient preference and hospital characteristics as predictors of imaging utilization. Limitations This study was limited because we did not extract commonly reported risk factors for PE in the chart reviews; for example, a history of prior PE, hypoxia, or leg swelling. The ability to determine patient-level risk outside of demographic factors may have explained some of the variation between physicians and hospitals. We also did not study the use of risk-stratification tools by providers in this study; this may have impacted their decision-making. Availability of patient clinical data as well as physician use of CDRs would allow for much greater understanding of

Advanced Radiography in Diagnosis of Pulmonary Embolus

7

appropriate use of imaging, which is the actual metric of interest; utilization and diagnostic yield are surrogates that can’t fully address this issue but are more readily calculated using retrospective data. There is a gray area between over-utilization (likely including some ‘‘inappropriately’’ ordered scans) and under-testing, leading to missed diagnoses; this study did not assess this careful balance. There may have been potentially missed PEs by under-ordering on the one hand, or unnecessary ordering in low-risk patients on the other. Our study population was patients who were scanned to rule out PE; we did not include all patients at risk of PE. We were unable to draw any conclusions on whether more or less ordering of CT and V/Q scans led to better or worse care. The hospitals and providers with higher utilization and lower diagnostic yield may have missed fewer PEs, but this study was not designed to evaluate for this. In addition, although the results of this study suggest that the odds of a positive test vary significantly between hospitals and are higher among more experienced physicians, the findings may not be generalizable to other settings. The physicians included in the study may not be representative of ED physicians nationally or internationally. The results also do not account for regional variation in advanced radiography utilization or patient preferences in use and type of scan ordered. Although we tried to capture every CT or V/Q scan ordered from the ED during the study period, it is possible that our methods of chart extraction may have missed some scans or patients, for example, during EMR downtimes. Also, although we tried to include only those CT scans ordered specifically to evaluate for PE, it is possible that we erroneously included CTs ordered for other reasons, such as to evaluate for infection or aortic dissection. Each radiology report was read, and scans for reasons other than PE were excluded from the study, but it is possible that we may have included some non-PE scans. There may have been some selection bias in which CT scans were included (i.e., some non-PE-specific scans included, not all PE-specific scans included), but this bias was nondifferential across physicians and hospitals because the order entry system is the same across sites. This would be expected to underestimate diagnostic yield. Also, there may have been some information bias with regard to who ordered the scans. For example, at one of the hospitals, from 12 PM to 9 PM on weekdays, a triage physician often placed initial orders during the last year of study (this would have accounted for approximately 5% of total scans ordered in this study). This physician’s name is therefore listed as the ordering physician instead of the ‘‘team’’ physician, who likely manages the patient once the initial orders are placed. The triage physician

often performs a more limited history and physical examination. This may affect their risk-stratification of patients, their utilization of CT or V/Q scans to rule out PE, and their resulting positive diagnostic yield. The ‘‘team’’ physician may evaluate the patient before the scan is actually performed, and may cancel the scan, which might also affect the ordering physician’s (i.e., the triage physician) overall diagnostic yield. On the other hand, the scan may have been completed by the time the team physician has time to assess the patient. This would likely decrease diagnostic yield. It is possible that some CT scans may have been misclassified as positive or negative. Our classification of the V/Q scans as being ‘‘positive’’ or ‘‘negative’’ was limited by the lack of a documented pretest probability of PE prior to V/Q scan and our inability to collect these data retrospectively. Previous research has shown that intermediate-probability V/Q scans have a prevalence of PE of approximately 30–40% (37). Most of these patients go on to have further imaging. By including all intermediate scans as positive, we likely falsely included many scans in our positive category. We felt this was an appropriate, although conservative, approach given our inability to track further confirmatory imaging and lack of pretest probability. CONCLUSIONS We found significant variation in resource utilization and positive diagnostic yield among hospitals in this study. Studies ordered by more experienced providers (i.e., those with more than 10 years of postresidency experience), and those ordered at the two academic centers were more likely to be positive for PE than those ordered by more recent graduates and those ordered at the two community centers. We found no significant variation in provider diagnostic yield after controlling for multiple patient-, hospital-, and provider-level characteristics.

REFERENCES 1. Barrit DW, Jordan SC. Anticoagulant drugs in the treatment of pulmonary embolism: a controlled trial. Lancet 1960;1:1309–12. 2. Calder KK, Herbert M, Henderson SO. The mortality of untreated pulmonary embolism in emergency department patients. Ann Emerg Med 2005;45:302–10. 3. Arnold RW, Graham DA, Melvin PR, Taylor GA. Variability in imaging utilization in U.S. pediatric hospitals. Pediatr Radiol 2011;41: 867–74. 4. Bhargavan M, Sunshine JH. Utilization of radiology services in the United States: levels and trends in modalities, regions, and populations. Radiology 2005;234:824–32. 5. Broder J, Warshauer DM. Increasing utilization of computed tomography in the adult emergency department, 2000–2005. Emerg Radiol 2006;13:25–30. 6. Jain S, Elon LK, Johnson BA, Frank G, DeGuzman M. Physician practice variation in the pediatric emergency department and its

8

7. 8.

9. 10. 11. 12. 13.

14. 15. 16.

17.

18.

19. 20.

21. 22.

D. R. Kindermann et al. impact on resource use and quality of care. Pediatr Emerg Care 2010;12:902–8. Parker L, Levin DC, Frangos A, Rao VM. Geographic variation in the utilization of noninvasive diagnostic imaging: national Medicare data, 1998–2007. AJR Am J Roentgenol 2010;194:1034–9. Prologo JD, Gilkerson RC, Diaz M, Asaad J. CT pulmonary angiography: a comparative analysis of the utilization patterns in emergency department and hospitalized patients between 1998 and 2003. AJR Am J Roentgenol 2004;183:1093–6. Boland GW, Guimareaes AS, Mueller PR. The radiologist’s conundrum: benefits and costs of increasing CT capacity and utilization. Eur Radiol 2009;19:9–11. Winslow JE, Hinshaw JW, Highes MJ, Williams RC, Bozeman WP. Quantitative assessment of diagnostic radiation doses in adult blunt trauma patients. Ann Emerg Med 2008;52:93–7. Hasebrook KM, Serkova JN. Toxicity of MRI and CT contrast agents. Expert Opin Drug Metab Toxicol 2009;5:403–16. Weir ID, Drescher F, Cousin D, et al. Trends in use and yield of chest CT with angiography for diagnosis of PE in a Connecticut hospital emergency department. Conn Med 2010;74:5–9. Donohoo JH, Mayo-Smith WW, Pezzullo JA, Egglin TK. Utilization patterns and diagnostic yield of 3421 consecutive multidetector row computed tomography pulmonary angiograms in a busy emergency department. J Comput Assist Tomogr 2008;32:421–5. Ost D, Khanna D, Shah R, et al. Impact of spiral computed tomography on the diagnosis of pulmonary embolism in a community hospital setting. Respiration 2004;72:450–8. Le Gal G, Righini M, Roy PM, et al. Prediction of PE in the ED: the revised Geneva Score. Ann Intern Med 2006;144:165–71. Pasha SM, Klok FA, Snoep JD, et al. Safety of excluding acute pulmonary embolism based on an unlikely clinical probability by the Wells rule and normal D-dimer concentration: a meta-analysis. Thromb Res 2010;125:123–7. Wells PS, Anderson DR, Rodger M, et al. Derivation of a simple clinical model to categorize patients’ probability of pulmonary embolism: increasing the models utility with the SimpliRED D-Dimer. Thromb Haemost 2000;83:416–20. Kline JA, Webb WB, Jones AE, Hernandez-Nino J. Impact of a rapid rule-out protocol for pulmonary embolism on the rate of screening, missed cases, and pulmonary vascular imaging in an urban US emergency department. Ann Emerg Med 2004;44:490–502. Weiss CR, Haponik EF, Diette GB, Marriman B, Scartarige JC, Fishman EK. Pretest risk assessment in suspected acute PE. Acute Radiol 2008;15:3–14. Corwin MT, Donohoo JH, Partridge R, Egglin TK, MayoSmith WW. Do emergency physicians use serum D-dimer effectively to determine the need for CT when evaluating patients for PE? AJR Am J Roentgenol 2009;192:1319–23. Hull RD, Hirsh J, Carter CJ, et al. Diagnostic value of ventilationperfusion lung scanning in patients with suspected pulmonary embolism. Chest 1985;88:819–28. Gilbert EH, Lowenstein SR, Koziol-McLain J, Barta DC, Steiner J. Chart reviews in emergency medicine research: where are the methods? Ann Emerg Med 1996;27:305–8.

23. Snijders TA, Bosker RJ. Multilevel analysis: an introduction to basic and advance multilevel modeling. London: Sage Publications; 1999. 24. Bhuiya FA, Pitts SR, McCaig LF. Emergency department visits for chest pain and abdominal pain: United States, 1999–2008. NCHS Data Brief 2010;(43):1–8. 25. Kline JA, Courney DM, Moore J, et al. Prospective multicenter evaluation of the pulmonary embolism rule-out criteria. J Thromb Haemost 2008;6:772–80. 26. Holst AG, Jensen G, Prescott E. Risk factors for venous thromboembolism: results from the Copenhagen City Heart Study. Circulation 2010;4:1896–903. 27. Kohn M, Kwan E, Gupta M, Tabas J. Prevalence of acute myocardial infarction and other serious diagnoses in patients presenting to an urban emergency department with chest pain. J Emerg Med 2005;29:383–90. 28. Newman K, Ponsky T, Kittle K, et al. Appendicitis 2000: variability in practice, outcomes, and resource utilization at thirty pediatric hospitals. J Pediatr Surg 2003;38:372–9. 29. Caveney AF, Silbergleit R, Frederiksen S, et al. Resource utilization and outcome at a university versus a community teaching hospital in tPA treated stroke patients: a retrospective cohort study. BMC Health Serv Res 2010;10:44. 30. Pines JM, Hollander JE, Isserman JA, et al. The association between physician risk tolerance and imaging use in abdominal pain. Am J Emerg Med 2009;27:552–7. 31. Pines JM, Isserman JA, Szyld D, Dean AJ, McCusker CM, Hollander JE. The effect of physician risk tolerance and the presence of an observation unit on decision making for ED patients with chest pain. Am J Emerg Med 2010;28:771–9. 32. Katz DA, Williams GC, Brown RL, et al. Emergency physicians’ fear of malpractice in evaluating patients with possible acute cardiac ischemia. Ann Emerg Med 2005;4:525–33. 33. Andruchow JE, Raja AS, Prevedello LM, Zane RD, Khorasani R. Physician risk tolerance and diagnostic yield of CT pulmonary angiography: are they related? Ann Emerg Med 2012; 60(Suppl):S4. 34. Centers for Medicare and Medicaid Services. Hospital outpatient quality data reporting program. Outpatient imaging efficiency measures OP-13, OP-14, OP-15; April 20, 2011. Available at: http:// www.hopqdrponline.com/media/OutpatientImagingEfficiencyMea sures508Checked041211_1perslide.pdf. Accessed February 21, 2012. 35. National Quality Forum. NQF IEP-005–010: candidate measure for pulmonary CT imaging for patients at low risk for pulmonary embolism. Available at: http://www.brighamandwomens.org/Departments_ and_Services/emergencymedicine/Quality_Improvement.aspx?sub=0. Accessed October 17, 2012. 36. Venkatesh AK, Kline JA, Courtney DM, et al. Evaluation of pulmonary embolism in the emergency department and consistency with a national quality measure: quantifying the opportunity for improvement. Arch Intern Med 2012;172:1028–32. 37. Worsley DF, Alavi A. Comprehensive analysis of the results of the PIOPED Study. J Nucl Med 1995;36:2380–7.

Emergency department variation in utilization and diagnostic yield of advanced radiography in diagnosis of pulmonary embolus.

There is growing pressure to measure and reduce unnecessary imaging in the emergency department...
220KB Sizes 3 Downloads 3 Views