THE VOICE OF EXPERIENCE
Optimizing Emergency Department Imaging Utilization Through Advanced Health Record Technology Arun Krishnaraj, MD, MPH, Sayon Dutta, MD, Andrew T. Reisner, MD, Adam B. Landman, MD, Garry Choy, MD, MBA, MSc, Paul Biddinger, MD, Abraham Lin, AB, Neeraj Joshi, MS BACKGROUND AND SIGNIFICANCE
According to the Centers for Disease Control and Prevention, approximately 136.1 million people visited emergency departments (EDs) in the United States in 2009 [1]. The percentage of these patients who undergo imaging as part of their diagnostic workup has grown dramatically over the past few decades [2-5]. More important, imaging utilization rates in EDs are growing more rapidly than imaging performed in other settings, including outpatient facilities, inpatient facilities, and private offices [6]. Much of the care provided in the ED setting is driven by the difficulties of evaluating complex patients under time pressure and is exacerbated by incomplete awareness of medical history. Imaging and laboratory tests often provide more objective information than a history and physical examination and are hence utilized to a greater extent in this challenging environment compared with a traditional outpatient setting. Although prior studies have demonstrated improved outcomes for certain diseases due to imaging, there is growing concern that imaging is being overutilized in the ED setting [7]. Electronic health record (EHR) systems capture a broad range of information and events in a health care enterprise, including laboratory tests, pathology reports, diagnostic studies, clinical notes, and demographic information. However, much of the data contained in EHRs
are stored as free-text documents and are difficult to access and review efficiently by care providers, which may lead to redundant or inappropriate utilization of health care resources. In response to this difficulty, a team of clinicians and software developers within our Department of Radiology’s informatics division has leveraged advanced EHR technology to develop a search engine for the EHR known as Queriable Patient Inference Dossier (QPID) [8]. QPID
QPID is a programmable health record intelligence system, adding semantic search and knowledge management layers to an EHR system. If one considers an essential component of an EHR to be its data repository, then QPID is a programmable system that facilitates the extraction of information from that repository. Many EHR systems include databases that can handle basic data retrieval functions. Laying on top of a data repository, and unlike standard database systems, QPID permits users to specify topicrelated “packages” of data and concepts in saved, operable queries that retrieve information across both structured and unstructured data sources. The platform exposes to the user powerful programming tools to leverage the structure of medical English, including ontologic knowledge representation and powerful natural language processing tools, yet the syntax requirements for the end user are light, making it easy for the clinical end user to specify packages
ª 2013 American College of Radiology 1546-1440/13/$36.00 http://dx.doi.org/10.1016/j.jacr.2013.07.011
of information on arbitrary topics relevant to the user’s workflow. For example, a clinician may wish to associate the concept “abscess” with text and other EHR data such as “fever,” “white blood cell count,” or “antibiotic administration” to automate information retrieval on those topics for management decision making. On the QPID platform, the user can author an “abscess” query or use an existing query (if it exists) from the system’s query library. Execution of each query can be automated against a service schedule or care unit census. Moreover, queries can be tied together to form information-rich applications whose output can be integrated into clinical workflows. Accumulated input from users and from system staff members contributes to a rapidly growing library of clinician-directed queries that combine knowledge representation and query content. Systems that provide advanced integration of EHR data such as QPID offer 4 potentially significant advantages over manual searching of the EHR: (1) near instantaneous searches of the entire EHR on the basis of a clinical question, such as “I wonder if the patient has a history of pulmonary embolism mentioned in the EHR”; (2) the automation of complex, structured queries on the basis of clinical service schedule or care unit census (such as searching the entire medical intensive care unit for the administration of proton pump inhibitors); (3) the ability to easily integrate the results of a search into a web browser or other software 1
2 The Voice of Experience
application; and (4) the ability to measure reproducible context-specific stimulus-response patterns of clinicians responding to presentation of EHR information. OBJECTIVE
The present work represents the first in a series of investigations whose overall aim is to analyze and optimize care in the ED environment and potentially influence resource utilization in the ED environment. To that end, the QPID informatics team worked jointly with clinicians from the ED to develop a customized QPID application for ED care that expedites summarization of medical history relevant to initial screening and management decisions, including the ordering of imaging and other diagnostic examinations. Our initial aim in this multiphase work was to validate the accuracy of the queries that constitute this information retrieval and workflow improvement tool. APPLICATION DESIGN
A custom QPID ED application was developed, which included a set of 74 query topics deemed important for screening and management by ED physicians (Online Table 1). The selection of the query topics was performed by informal survey of the ED staff physicians at our institution, collated and organized by two of us (S.D. and A.T.R.) and presented to the QPID development group as a list. The search queries included both structured data (laboratory values) and unstructured data (eg, recent reports, medication alerts, and text searches, such as “Does this patient have evidence in the EHR indicating dementia?”). The application’s interface, which we refer to as a dashboard, was designed to allow ED physicians to rapidly view a summary of relevant history contained in the EHR (Online Fig. 1) Online Table 2.
VALIDATION
Five hundred consecutive adult patients, who presented to our urban academic medical center in January 2011, constituted the study cohort, which was approved by our institutional review board. All clinical documents containing both structured data and unstructured data in the EHR were reviewed for each patient. We validated the performance of each of the 74 queries by designing a tool that performed an automated QPID search on all 74 queries included in the dashboard for each of the 500 patients (Online Fig. 2). The output of this search was subsequently reviewed by two physicians (A.K. and G.C.), who analyzed the automated query results from 30 randomly selected patients (15 positive hits, when present, and 15 negative hits, when present) from the cohort of 500 for each of the 74 queries. The reference standard for performance for each query was manual review by the physicians using our hospital’s standard clinical interface to the EHR (Clinical Application Suite; Partners HealthCare Information Systems, Boston, Massachusetts), not the saved QPID queries. Using the validation software, we were able to keep and record the performance characteristics for each query, including the positive predictive value (PPV) and negative predictive value (NPV), with manual review of the EHR serving as the reference standard. To better estimate real-world PPVs and NPVs given that certain concepts searched for by the QPID queries were of low prevalence in the cohort tested, the modified Wald method was applied to calculate average PPV and NPV for the searches. The F1 score (2 [precision recall]/[precision þ recall]) of each search was also calculated. In addition, Cohen’s k coefficient was calculated to determine interreviewer agreement between the two human reviewers on the primary data.
Finally, average search time for the application to complete all queries included in the dashboard was calculated by pooling data on 20 randomly selected patients from the cohort of 500. RESULTS AND OUTCOMES
The mean search time for the application to complete all 74 searches on each patient was 15 5 seconds. Notable searches included among the 74 that have the potential to directly influence imaging utilization in the ED included deep vein thrombosis in the past 5 years, pulmonary embolus in the past 10 years, evidence of prior ectopic pregnancy, low ejection fraction, mechanical valve, coagulation parameters, and the presence of an automatic implantable cardioverter-defibrillator. For structured data, the QPID queries demonstrated a pooled calculated PPV of 87% (range, 67%e94%) and an NPV of 86% (range, 50%e94%). For unstructured data, the QPID queries demonstrated a pooled calculated PPV of 75% (range, 25%e94%) and an NPV of 88% (range, 67%e94%) Online Table 3. The F1 score performance of each individual query (when positive data were available) is presented in Online Table 4. The calculated free marginal k coefficient for interrater reliability between the two clinicians whose search results served as the gold standard was 0.87. DISCUSSION
Our study demonstrates that a large set of predefined, automated EHR queries using the QPID platform can be executed rapidly, and these queries demonstrate a high level of accuracy for identification of information deemed salient to ED physicians.
The Voice of Experience 3
Advanced heath ITs, specifically EHR systems, are poised to undergo rapid dissemination and widespread adoption spurred by initiatives in the American Recovery and Reinvestment Act of 2009 [9]. When properly integrated into clinical workflow, an advanced EHR system has the potential to improve the quality of care delivery [10]. However, even when available, the amount of information contained in an EHR can be overwhelming for a busy ED physician to sort through in the care of patients with complex medical histories. In addition, recent studies have shown that current implementations of EHRs have not lived up to their initial billing with regard to reducing health care costs and improving health outcomes [11]. Inappropriate imaging utilization represents a potentially modifiable expenditure and may expose patients to unnecessary ionizing radiation [12]. Because of the growing contribution of imaging costs to overall health expenditures, predominately during the early part of the past decade, health policymakers have enacted legislation to curb imaging growth [13]. According to the National Quality Forum, the use of informatics by health professionals may help reduce the inappropriate utilization of medical technologies, including medical imaging [14]. To date, two major approaches have been used to reduce inappropriate imaging utilization rates: (1) the incorporation of clinical decision support tools into computerized physician order entry systems and (2) the use of external authorization bodies such as radiology benefit managers. Both approaches have been shown to reduce imaging costs and growth rates, but neither incorporates advanced EHR technology to better inform referring clinicians of all available history contained in the EHR, a critical factor dictating why an imaging
study is ordered [15-18]. The present work is a necessary, preliminary step toward demonstrating that the combination of concept specification by clinical users, and the capability to handle heterogeneous (structured and unstructured) input EHR data may facilitate review of salient clinical data on the often complex patients who present to the ED. We hope to demonstrate in future studies that this tool will influence the rate and appropriateness of imaging utilization as well as other health care resources. Limitations of our study include using an untimed manual review as the reference standard. However, we believe that our choice of an untimed manual review by experienced clinicians approximates the best current standard of care that exists for evaluation of medical history contained in the EHR in the time-constrained setting. Additionally, our ability to estimate the accuracy of the QPID queries for very low prevalence items (such as “bowel ischemia”) may be limited. Also, patients may have medical histories not included in the EHR because care was received at other institutions not accessible to QPID. Finally, although the intent of the QPID ED application is to reduce resource utilization in this setting, more information may paradoxically lead to increased resource utilization. The creation and validation of each search query is an ongoing process. As we progress with the development of this application we will continue to develop and validate new queries on the basis of the needs of ED physicians and determine the optimal configuration of alerts, being mindful of the feedback we continually receive from its users. CONCLUSIONS
Targeted summaries of data contained in the EHR, generated
by queries written for the QPID health record intelligence platform, are a rapid and accurate means by which clinicians can gather relevant medical histories on patients. Incorporating this tool into the busy ED environment may help reduce the overutilization of health resources, including imaging, in this challenging setting. Furthermore, as health systems EHRs are connected by health information exchanges, more electronic patient information will be made available creating an even greater need for tools such as QPID. REFERENCES 1. Centers for Disease Control and Prevention. Fast stats: emergency department visits. Available at: http://www.cdc.gov/nchs/fastats/ ervisits.htm. Accessed October 18, 2012. 2. Raja AS, Mortele KJ, Hanson R, Sodickson AD, Zane R, Khorasani R. Abdominal imaging utilization in the emergency department: trends over two decades. Int J Emerg Med 2011;4:19. 3. Rao VM, Levin DC, Parker L, Frangos AJ, Sunshine JH. Trends in utilization rates of the various imaging modalities in emergency departments: nationwide Medicare data from 2000 to 2008. J Am Coll Radiol 2011;8:706-9. 4. Larson DB, Johnson LW, Schnell BM, Salisbury SR, Forman HP. National trends in CT use in the emergency department: 1995-2007. Radiology 2011;258:164-73. 5. Korley FK, Pham JC, Kirsch TD. Use of advanced radiology during visits to US emergency departments for injury-related conditions 1998-2007. JAMA 2010;304: 1465-71. 6. Levin DC, Rao VM, Parker L, Frangos AJ, Sunshine JH. Recent shifts in place of service for noninvasive diagnostic imaging: have hospitals missed an opportunity? J Am Coll Radiol 2009;6:96-9. 7. Raja AS, Wright C, Sodickson AD, et al. Negative appendectomy rate in the era of CT: an 18-year perspective. Radiology 2010;256:460-5. 8. Zalis M, Harris M. Advanced search of the electronic medical record: augmenting safety and efficiency in radiology. J Am Coll Radiol 2010;7:625-33. 9. Blumenthal D, Tavenner M. The “meaningful use” regulation for electronic health records. N Engl J Med 2010;363:501-4. 10. Chaudhry B, Wang J, Wu S, et al. Systematic review: impact of health information
4 The Voice of Experience
technology on quality, efficiency, and costs of medical care. Ann Intern Med 2006;144: 742-52. 11. Kellermann AL, Jones SS. What it will take to achieve the as-yet-unfulfilled promises of health information technology. Health Aff (Millwood) 2013;32:63-8.
14. National Priorities Partnership. National priorities and goals: aligning our efforts to transform America’s healthcare. Washington: District of Columbia: National Quality Forum; 2008.
12. Brenner DJ, Hall EJ. Computed tomography—an increasing source of radiation exposure. N Engl J Med 2007;357:2277-84.
15. Mitchell JM, Lagalia RR. Controlling the escalating use of advanced imaging: the role of radiology benefit management programs. Med Care Res Rev 2009;66: 339-51.
13. Iglehart JK. Health insurers and medicalimaging policy—a work in progress. N Engl J Med 2009;360:1030-7.
16. Levin DC, Bree RL, Rao VM, Johnson J. A prior authorization program of a radiology benefits management company and how it
has affected utilization of advanced diagnostic imaging. J Am Coll Radiol 2010;7:33-8. 17. Sistrom CL, Dang PA, Weilburg JB, Dreyer KJ, Rosenthal DI, Thrall JH. Effect of computerized order entry with integrated decision support on the growth of outpatient procedure volumes: sevenyear time series analysis. Radiology 2009;251:147-55. 18. Blackmore CC, Mecklenburg RS, Kaplan GS. Effectiveness of clinical decision support in controlling inappropriate imaging. J Am Coll Radiol 2011;8:19-25.
Arun Krishnaraj, MD, MPH, is from University of Virginia, Charlottesville, Virginia. Sayon Dutta, MD, Andrew T. Reisner, MD, Paul Biddinger, MD, are from Department of Emergency Medicine, Massachusetts General Hospital. Gary Choy, MD, MBA, MSc, Abraham Lin, AB, and Neeraj Joshi, MS, are from the Department of Radiology, Massachusetts General Hospital, Boston, Massachusetts. Adam Landman, MD, is from Brigham and Women’s Hospital, Boston, Massachusetts. Arun Krishnaraj, MD, MPH, University of Virginia, Department of Radiology and Medical Imaging, 1215 Lee Street, PO Box 800170, Charlottesville, VA 22902; e-mail: [email protected].
The Voice of Experience 4.e1
Online Fig. 1. Screenshot of QPID emergency department summary screen (HIPAA-compliant test patient displayed).
Online Fig. 2. Sample screen shot of the validation tool (validation of laboratory value for calcium).
4.e2 The Voice of Experience
Online Table 1. List of search queries Structured Data Absolute neutrophils Arterial PCO2 Arterial pH Arterial PO2 BNP Calcium Creatinine Direct bilirubin HCO3 HCT Lactate Lipase Mg Na Plasma CO2 Platelets Potassium PT/INR PTT Qualitative bHCG Quantitative bHCG SGOT SGPT Troponin T Venous PCO2 WBC Alkaline phosphatase
Unstructured Data
All medications: searches for all medications listed in any data repository in the past 30 days Antiarrhythmic drugs Antibiotics Anticoagulants Anticonvulsants Antipsychotics Any note (past 1,825 days) Any note (past 28 days) Aortic aneurysm Aortic stenosis Coronary stent Dementia DVT in the past 5 years Ectopic pregnancy ED discharge note (past 1,825 days) ED discharge note (past 28 days) Encounter note (past 1,825 days) Encounter note (past 28 days) Evidence of pelvic inflammatory disease HIV infection Immunosuppressants IV drug abuse Last admission note Last catheterization note Last discharge note Last echocardiogram note Last ED note Last exercise treadmill test note Last intensive care unit note Last referral note Last surgical note Low ejection fraction (
Optimizing Emergency Department Imaging Utilization Through Advanced Health Record Technology Arun Krishnaraj, MD, MPH, Sayon Dutta, MD, Andrew T. Reisner, MD, Adam B. Landman, MD, Garry Choy, MD, MBA, MSc, Paul Biddinger, MD, Abraham Lin, AB, Neeraj Joshi, MS BACKGROUND AND SIGNIFICANCE
According to the Centers for Disease Control and Prevention, approximately 136.1 million people visited emergency departments (EDs) in the United States in 2009 [1]. The percentage of these patients who undergo imaging as part of their diagnostic workup has grown dramatically over the past few decades [2-5]. More important, imaging utilization rates in EDs are growing more rapidly than imaging performed in other settings, including outpatient facilities, inpatient facilities, and private offices [6]. Much of the care provided in the ED setting is driven by the difficulties of evaluating complex patients under time pressure and is exacerbated by incomplete awareness of medical history. Imaging and laboratory tests often provide more objective information than a history and physical examination and are hence utilized to a greater extent in this challenging environment compared with a traditional outpatient setting. Although prior studies have demonstrated improved outcomes for certain diseases due to imaging, there is growing concern that imaging is being overutilized in the ED setting [7]. Electronic health record (EHR) systems capture a broad range of information and events in a health care enterprise, including laboratory tests, pathology reports, diagnostic studies, clinical notes, and demographic information. However, much of the data contained in EHRs
are stored as free-text documents and are difficult to access and review efficiently by care providers, which may lead to redundant or inappropriate utilization of health care resources. In response to this difficulty, a team of clinicians and software developers within our Department of Radiology’s informatics division has leveraged advanced EHR technology to develop a search engine for the EHR known as Queriable Patient Inference Dossier (QPID) [8]. QPID
QPID is a programmable health record intelligence system, adding semantic search and knowledge management layers to an EHR system. If one considers an essential component of an EHR to be its data repository, then QPID is a programmable system that facilitates the extraction of information from that repository. Many EHR systems include databases that can handle basic data retrieval functions. Laying on top of a data repository, and unlike standard database systems, QPID permits users to specify topicrelated “packages” of data and concepts in saved, operable queries that retrieve information across both structured and unstructured data sources. The platform exposes to the user powerful programming tools to leverage the structure of medical English, including ontologic knowledge representation and powerful natural language processing tools, yet the syntax requirements for the end user are light, making it easy for the clinical end user to specify packages
ª 2013 American College of Radiology 1546-1440/13/$36.00 http://dx.doi.org/10.1016/j.jacr.2013.07.011
of information on arbitrary topics relevant to the user’s workflow. For example, a clinician may wish to associate the concept “abscess” with text and other EHR data such as “fever,” “white blood cell count,” or “antibiotic administration” to automate information retrieval on those topics for management decision making. On the QPID platform, the user can author an “abscess” query or use an existing query (if it exists) from the system’s query library. Execution of each query can be automated against a service schedule or care unit census. Moreover, queries can be tied together to form information-rich applications whose output can be integrated into clinical workflows. Accumulated input from users and from system staff members contributes to a rapidly growing library of clinician-directed queries that combine knowledge representation and query content. Systems that provide advanced integration of EHR data such as QPID offer 4 potentially significant advantages over manual searching of the EHR: (1) near instantaneous searches of the entire EHR on the basis of a clinical question, such as “I wonder if the patient has a history of pulmonary embolism mentioned in the EHR”; (2) the automation of complex, structured queries on the basis of clinical service schedule or care unit census (such as searching the entire medical intensive care unit for the administration of proton pump inhibitors); (3) the ability to easily integrate the results of a search into a web browser or other software 1
2 The Voice of Experience
application; and (4) the ability to measure reproducible context-specific stimulus-response patterns of clinicians responding to presentation of EHR information. OBJECTIVE
The present work represents the first in a series of investigations whose overall aim is to analyze and optimize care in the ED environment and potentially influence resource utilization in the ED environment. To that end, the QPID informatics team worked jointly with clinicians from the ED to develop a customized QPID application for ED care that expedites summarization of medical history relevant to initial screening and management decisions, including the ordering of imaging and other diagnostic examinations. Our initial aim in this multiphase work was to validate the accuracy of the queries that constitute this information retrieval and workflow improvement tool. APPLICATION DESIGN
A custom QPID ED application was developed, which included a set of 74 query topics deemed important for screening and management by ED physicians (Online Table 1). The selection of the query topics was performed by informal survey of the ED staff physicians at our institution, collated and organized by two of us (S.D. and A.T.R.) and presented to the QPID development group as a list. The search queries included both structured data (laboratory values) and unstructured data (eg, recent reports, medication alerts, and text searches, such as “Does this patient have evidence in the EHR indicating dementia?”). The application’s interface, which we refer to as a dashboard, was designed to allow ED physicians to rapidly view a summary of relevant history contained in the EHR (Online Fig. 1) Online Table 2.
VALIDATION
Five hundred consecutive adult patients, who presented to our urban academic medical center in January 2011, constituted the study cohort, which was approved by our institutional review board. All clinical documents containing both structured data and unstructured data in the EHR were reviewed for each patient. We validated the performance of each of the 74 queries by designing a tool that performed an automated QPID search on all 74 queries included in the dashboard for each of the 500 patients (Online Fig. 2). The output of this search was subsequently reviewed by two physicians (A.K. and G.C.), who analyzed the automated query results from 30 randomly selected patients (15 positive hits, when present, and 15 negative hits, when present) from the cohort of 500 for each of the 74 queries. The reference standard for performance for each query was manual review by the physicians using our hospital’s standard clinical interface to the EHR (Clinical Application Suite; Partners HealthCare Information Systems, Boston, Massachusetts), not the saved QPID queries. Using the validation software, we were able to keep and record the performance characteristics for each query, including the positive predictive value (PPV) and negative predictive value (NPV), with manual review of the EHR serving as the reference standard. To better estimate real-world PPVs and NPVs given that certain concepts searched for by the QPID queries were of low prevalence in the cohort tested, the modified Wald method was applied to calculate average PPV and NPV for the searches. The F1 score (2 [precision recall]/[precision þ recall]) of each search was also calculated. In addition, Cohen’s k coefficient was calculated to determine interreviewer agreement between the two human reviewers on the primary data.
Finally, average search time for the application to complete all queries included in the dashboard was calculated by pooling data on 20 randomly selected patients from the cohort of 500. RESULTS AND OUTCOMES
The mean search time for the application to complete all 74 searches on each patient was 15 5 seconds. Notable searches included among the 74 that have the potential to directly influence imaging utilization in the ED included deep vein thrombosis in the past 5 years, pulmonary embolus in the past 10 years, evidence of prior ectopic pregnancy, low ejection fraction, mechanical valve, coagulation parameters, and the presence of an automatic implantable cardioverter-defibrillator. For structured data, the QPID queries demonstrated a pooled calculated PPV of 87% (range, 67%e94%) and an NPV of 86% (range, 50%e94%). For unstructured data, the QPID queries demonstrated a pooled calculated PPV of 75% (range, 25%e94%) and an NPV of 88% (range, 67%e94%) Online Table 3. The F1 score performance of each individual query (when positive data were available) is presented in Online Table 4. The calculated free marginal k coefficient for interrater reliability between the two clinicians whose search results served as the gold standard was 0.87. DISCUSSION
Our study demonstrates that a large set of predefined, automated EHR queries using the QPID platform can be executed rapidly, and these queries demonstrate a high level of accuracy for identification of information deemed salient to ED physicians.
The Voice of Experience 3
Advanced heath ITs, specifically EHR systems, are poised to undergo rapid dissemination and widespread adoption spurred by initiatives in the American Recovery and Reinvestment Act of 2009 [9]. When properly integrated into clinical workflow, an advanced EHR system has the potential to improve the quality of care delivery [10]. However, even when available, the amount of information contained in an EHR can be overwhelming for a busy ED physician to sort through in the care of patients with complex medical histories. In addition, recent studies have shown that current implementations of EHRs have not lived up to their initial billing with regard to reducing health care costs and improving health outcomes [11]. Inappropriate imaging utilization represents a potentially modifiable expenditure and may expose patients to unnecessary ionizing radiation [12]. Because of the growing contribution of imaging costs to overall health expenditures, predominately during the early part of the past decade, health policymakers have enacted legislation to curb imaging growth [13]. According to the National Quality Forum, the use of informatics by health professionals may help reduce the inappropriate utilization of medical technologies, including medical imaging [14]. To date, two major approaches have been used to reduce inappropriate imaging utilization rates: (1) the incorporation of clinical decision support tools into computerized physician order entry systems and (2) the use of external authorization bodies such as radiology benefit managers. Both approaches have been shown to reduce imaging costs and growth rates, but neither incorporates advanced EHR technology to better inform referring clinicians of all available history contained in the EHR, a critical factor dictating why an imaging
study is ordered [15-18]. The present work is a necessary, preliminary step toward demonstrating that the combination of concept specification by clinical users, and the capability to handle heterogeneous (structured and unstructured) input EHR data may facilitate review of salient clinical data on the often complex patients who present to the ED. We hope to demonstrate in future studies that this tool will influence the rate and appropriateness of imaging utilization as well as other health care resources. Limitations of our study include using an untimed manual review as the reference standard. However, we believe that our choice of an untimed manual review by experienced clinicians approximates the best current standard of care that exists for evaluation of medical history contained in the EHR in the time-constrained setting. Additionally, our ability to estimate the accuracy of the QPID queries for very low prevalence items (such as “bowel ischemia”) may be limited. Also, patients may have medical histories not included in the EHR because care was received at other institutions not accessible to QPID. Finally, although the intent of the QPID ED application is to reduce resource utilization in this setting, more information may paradoxically lead to increased resource utilization. The creation and validation of each search query is an ongoing process. As we progress with the development of this application we will continue to develop and validate new queries on the basis of the needs of ED physicians and determine the optimal configuration of alerts, being mindful of the feedback we continually receive from its users. CONCLUSIONS
Targeted summaries of data contained in the EHR, generated
by queries written for the QPID health record intelligence platform, are a rapid and accurate means by which clinicians can gather relevant medical histories on patients. Incorporating this tool into the busy ED environment may help reduce the overutilization of health resources, including imaging, in this challenging setting. Furthermore, as health systems EHRs are connected by health information exchanges, more electronic patient information will be made available creating an even greater need for tools such as QPID. REFERENCES 1. Centers for Disease Control and Prevention. Fast stats: emergency department visits. Available at: http://www.cdc.gov/nchs/fastats/ ervisits.htm. Accessed October 18, 2012. 2. Raja AS, Mortele KJ, Hanson R, Sodickson AD, Zane R, Khorasani R. Abdominal imaging utilization in the emergency department: trends over two decades. Int J Emerg Med 2011;4:19. 3. Rao VM, Levin DC, Parker L, Frangos AJ, Sunshine JH. Trends in utilization rates of the various imaging modalities in emergency departments: nationwide Medicare data from 2000 to 2008. J Am Coll Radiol 2011;8:706-9. 4. Larson DB, Johnson LW, Schnell BM, Salisbury SR, Forman HP. National trends in CT use in the emergency department: 1995-2007. Radiology 2011;258:164-73. 5. Korley FK, Pham JC, Kirsch TD. Use of advanced radiology during visits to US emergency departments for injury-related conditions 1998-2007. JAMA 2010;304: 1465-71. 6. Levin DC, Rao VM, Parker L, Frangos AJ, Sunshine JH. Recent shifts in place of service for noninvasive diagnostic imaging: have hospitals missed an opportunity? J Am Coll Radiol 2009;6:96-9. 7. Raja AS, Wright C, Sodickson AD, et al. Negative appendectomy rate in the era of CT: an 18-year perspective. Radiology 2010;256:460-5. 8. Zalis M, Harris M. Advanced search of the electronic medical record: augmenting safety and efficiency in radiology. J Am Coll Radiol 2010;7:625-33. 9. Blumenthal D, Tavenner M. The “meaningful use” regulation for electronic health records. N Engl J Med 2010;363:501-4. 10. Chaudhry B, Wang J, Wu S, et al. Systematic review: impact of health information
4 The Voice of Experience
technology on quality, efficiency, and costs of medical care. Ann Intern Med 2006;144: 742-52. 11. Kellermann AL, Jones SS. What it will take to achieve the as-yet-unfulfilled promises of health information technology. Health Aff (Millwood) 2013;32:63-8.
14. National Priorities Partnership. National priorities and goals: aligning our efforts to transform America’s healthcare. Washington: District of Columbia: National Quality Forum; 2008.
12. Brenner DJ, Hall EJ. Computed tomography—an increasing source of radiation exposure. N Engl J Med 2007;357:2277-84.
15. Mitchell JM, Lagalia RR. Controlling the escalating use of advanced imaging: the role of radiology benefit management programs. Med Care Res Rev 2009;66: 339-51.
13. Iglehart JK. Health insurers and medicalimaging policy—a work in progress. N Engl J Med 2009;360:1030-7.
16. Levin DC, Bree RL, Rao VM, Johnson J. A prior authorization program of a radiology benefits management company and how it
has affected utilization of advanced diagnostic imaging. J Am Coll Radiol 2010;7:33-8. 17. Sistrom CL, Dang PA, Weilburg JB, Dreyer KJ, Rosenthal DI, Thrall JH. Effect of computerized order entry with integrated decision support on the growth of outpatient procedure volumes: sevenyear time series analysis. Radiology 2009;251:147-55. 18. Blackmore CC, Mecklenburg RS, Kaplan GS. Effectiveness of clinical decision support in controlling inappropriate imaging. J Am Coll Radiol 2011;8:19-25.
Arun Krishnaraj, MD, MPH, is from University of Virginia, Charlottesville, Virginia. Sayon Dutta, MD, Andrew T. Reisner, MD, Paul Biddinger, MD, are from Department of Emergency Medicine, Massachusetts General Hospital. Gary Choy, MD, MBA, MSc, Abraham Lin, AB, and Neeraj Joshi, MS, are from the Department of Radiology, Massachusetts General Hospital, Boston, Massachusetts. Adam Landman, MD, is from Brigham and Women’s Hospital, Boston, Massachusetts. Arun Krishnaraj, MD, MPH, University of Virginia, Department of Radiology and Medical Imaging, 1215 Lee Street, PO Box 800170, Charlottesville, VA 22902; e-mail: [email protected].
The Voice of Experience 4.e1
Online Fig. 1. Screenshot of QPID emergency department summary screen (HIPAA-compliant test patient displayed).
Online Fig. 2. Sample screen shot of the validation tool (validation of laboratory value for calcium).
4.e2 The Voice of Experience
Online Table 1. List of search queries Structured Data Absolute neutrophils Arterial PCO2 Arterial pH Arterial PO2 BNP Calcium Creatinine Direct bilirubin HCO3 HCT Lactate Lipase Mg Na Plasma CO2 Platelets Potassium PT/INR PTT Qualitative bHCG Quantitative bHCG SGOT SGPT Troponin T Venous PCO2 WBC Alkaline phosphatase
Unstructured Data
All medications: searches for all medications listed in any data repository in the past 30 days Antiarrhythmic drugs Antibiotics Anticoagulants Anticonvulsants Antipsychotics Any note (past 1,825 days) Any note (past 28 days) Aortic aneurysm Aortic stenosis Coronary stent Dementia DVT in the past 5 years Ectopic pregnancy ED discharge note (past 1,825 days) ED discharge note (past 28 days) Encounter note (past 1,825 days) Encounter note (past 28 days) Evidence of pelvic inflammatory disease HIV infection Immunosuppressants IV drug abuse Last admission note Last catheterization note Last discharge note Last echocardiogram note Last ED note Last exercise treadmill test note Last intensive care unit note Last referral note Last surgical note Low ejection fraction (
