CIN: Computers, Informatics, Nursing
& Vol. 32, No. 11, 545–549 & Copyright B 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins
F E A T U R E A R T I C L E
A Review of Dashboards for Data Analytics in Nursing BRYAN A. WILBANKS, DNP, CRNA PATSY A. LANGFORD, MNA, CRNA
Increased adoption of electronic health records in recent years has substantially increased the amount of readily accessible digital data for use in clinical and business decision making; however, the current methodologies used for database analysis in healthcare have been inefficient for guiding evidence-based decision making.1 Evidence-based practice is characterized by the use of the current published research and available evidence to guide the delivery of patient care with the ultimate goal of improving clinical outcomes.2 A quick and easy method to leverage the massive amount of data in electronic health records to generate actionable knowledge is needed to improve patient outcomes and clinical practices. A dashboard is defined in the literature as a data-driven clinical decision support tool capable of querying multiple databases and providing a visual representation of key performance indicators in a single report,3 often in a format visually similar to a car’s dashboard.4–6 The purpose of a dashboard is to provide a concise overview of a large volume of data; consequently, there are many different types of dashboards that are intended for different applications. A key performance indicator is a specific quantified metric (eg, surgical-site infection rates or hospital-acquired decubitus ulcers) that represents an organizational goal or a type of performance evaluation measured for the purpose of guiding managerial or clinical decisions.4,6 One type of dashboard many nurses are familiar with is the physiological vital sign monitor screen(eg, the electrocardiogram, noninvasive blood pressure cuff reading, respiratory rate, and pulse oximetry)7 used to monitor patients. The combined use of clinical dashboards with electronic health records can provide near– real-time surveillance to improve patient outcomes when appropriately configured.4 Using an inductive content anal-
Dashboards are data-driven clinical decision support tools used to analyze data from multiple databases using easy-to-read, color-coded graphical displays, much like the dashboards of automobiles. Dashboards can be used to promote data-driven decision making and improve adherence to evidencebased practice guidelines. The purpose of this article was to provide a review of dashboards used to query electronic health records for the purpose of guiding clinical practice and research. An inductive content analysis approach was used to identify emerging themes directly from the literature. Five basic dashboard properties identified include the type of database integration, visual properties, purpose, time focus (ie, retrospective, real time, or predictive), and type of process monitored. These dashboard properties are determined by the characteristics of the specific organization, user, and purpose of data analysis. Using dashboards to perform automated analytical reviews of clinical data will prove more efficient when data elements stored in electronic health records become standardized. Other limitations of dashboard use include user anxiety, information overload, and technology overload. The increased use of electronic documentation in healthcare settings will provide a wealth of data, and dashboards will play a pivotal role in converting these data into actionable knowledge. KEY WORDS Clinical analytics & Dashboard & Data analytics & Decision support & Nursing informatics
ysis approach, this article reviews the use of dashboards to perform data analytics in the hospital setting for the purpose of guiding nursing practice and research.
Author Affiliations: School of Nursing, The University of Alabama at Birmingham (Dr Wilbanks); and Huntsville Hospital (Ms Langford), AL. The authors received no funding for this study. The authors have disclosed that they have no significant relationship with, or financial interest in, any commercial companies pertaining to this article. Corresponding author: Bryan A. Wilbanks, DNP, CRNA, 105 Shady Spring Dr, Harvest, AL 35749 ([email protected]). DOI: 10.1097/CIN.0000000000000106
CIN: Computers, Informatics, Nursing & November 2014 Copyright © 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
545
SEARCH METHODOLOGY AND RESULTS In November 2013, the electronic databases PubMed and CINAHL were searched for peer-reviewed articles using the single keyword dashboard, returning a total of 398 articles. No other search terms were included because this review focuses exclusively on data analytics using dashboards. No time limitations were used in the search because of the limited number of published articles pertaining to dashboards. Inclusion criteria consisted of articles that discussed the use of dashboards to query electronic health records in hospital settings. Exclusion criteria consisted of non–Englishbased articles, articles requiring fee for access, or articles that reported the use of dashboards that required manual data collection in addition to the use of electronic health records. The initial evaluation was based on article titles and abstracts, with a total of 81 articles identified that met the specified criteria. The articles were electronically retrieved for further evaluation, and 23 articles, published between the years 2008 and 2013, met the search criteria for final data analysis. Excluded articles did not involve hospital settings or used dashboards requiring a manual chart review and data entry process. The reviewed articles consisted of literature reviews (n = 12), descriptive quantitative studies (n = 5), randomized controlled trials using mixed-methods study designs (n = 2), mixed-methods studies (n = 2), a descriptive comparative outcomes study (n = 1), and a descriptive qualitative study (n = 1). The literature review was conducted using NVivo qualitative data analysis software, version 10 (QSR International Pty Ltd, Doncaster, Victoria, Australia).
ANALYSIS AND DISCUSSION An inductive content analysis using a constant comparative technique was used to identify the major themes and descriptions of dashboards in the literature. A constant comparative technique is used to generate categorical themes directly from the data and is characterized by the continuous comparisons of the newer data with the previously collected data for the purpose of identifying similarities and differences in the emerging themes.8 This technique was chosen because of its ability to generate comprehensive and descriptive categories directly from the literature.9 The major identified themes from the analysis include: definition and properties of dashboards, applications of dashboards, benefits and limitations of dashboard use, evaluation of dashboards, and future clinical and research implications of dashboards. The reviewed articles presented very diverse descriptions of dashboards, with no contradictions, possibly reflecting that the authors of the literature in this review collectively agree on the basic properties and uses of dashboards. 546
Definition and Properties of Dashboards A dashboard is a performance measurement tool used to monitor structure, process, or outcome variables6 by querying an electronic data set and presenting the results in an easy-to-interpret graphical format.4,10 The type of organization utilizing a dashboard, the type of system user (eg, direct patient care provider or managerial staff), and the specific purpose for data analysis determine the specific properties of a dashboard.4,11 The five basic types of properties of dashboards identified include database integration, visual properties, purpose, time focus, and type of process monitored. 1. Database integration for dashboards. A dashboard integrates data from multiple databases,4,12,13 which use different types of data formatting (eg, time stamps for documented events can use the symbol ‘‘/,’’ ‘‘-,’’ or ‘‘:’’ to separate the month, day, year, and time).14 Standardized definitions for data elements need to be established to effectively use dashboards to aggregate and analyze data across multiple data sets.15 Also, the degree of database integration (ie, the ability to electronically query multiple databases without a manual chart review) will determine the degree of automation for data analysis4; it is crucial to have electronic access to all of the data needed for analysis to maximize the benefits of dashboards. 2. Visual properties of dashboards. Dashboards are typically presented in simple graphical formats providing an intuitive at-a-glance assessment of key performance indicators4,10,15 in a manner that promotes evidence-based decision making. Color-coding the graphical displays facilitates rapid interpretation of performance status6— commonly using green for ‘‘acceptable,’’ yellow for ‘‘marginal,’’ and red for ‘‘unacceptable.’’ Dashboard styles can include stacked column graphs, scatterplots, pie graphs, column graphs, area graphs, radar graphs, or a gauge style closely resembling an automobile speedometer.4 Dashboards incorporating physiological monitoring of patients use different types of waveforms (eg, electrocardiogram, electroencephalogram, or specific hemoglobin plethysmography) or numerical number presentations.7 The intricate nature of data sets and the diverse information needs of administrators make it necessary for informaticists to work closely with management personnel to optimally design the visual properties of a dashboard.15 3. Intended purpose of dashboards. The possible functions of dashboards include (1) benchmarking performance, (2) providing notifications and warnings, (3) predicting performance to guide managerial decision making,4,12 and (4) summarizing and analyzing data to provide performance feedback to guide clinical decision making. Dashboards can automate the collection of key performance indicators to prove compliance with national standards (eg, compliance with standards of The Joint Commission)10,14,16 or benchmarking the performance of
CIN: Computers, Informatics, Nursing & November 2014 Copyright © 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
healthcare providers, hospital departments, or entire hospitals for the purpose of objectively comparing the respective groups.17 This allows rapid identification of underperformers for the purpose of generating targeted interventions for performance improvement. Using dashboards for benchmarking can also assist with quality assurance and quality improvement programs.4,12,18,19 Appropriately configured dashboards can provide user notifications when a specific metric deviates from acceptable values, prompting quick corrective measures to limit or prevent adverse events (eg, identification of an excessive medication dose being ordered for a patient or an increase in surgical-site infection rates).4,6,19 4. Time focus of dashboards. Dashboards can be used for retrospective data collection in electronic health records,11,15 provide real-time monitoring of current patient care (eg, vital sign monitors or ventilator setting control panels),7,13 or analyze databases for the purpose of predicting future events.15 Even though the literature mentions that the most successful dashboards will have the ability to help predict the outcomes of decisions based on current information,15 there are no reported examples of predictive dashboards used in healthcare settings. 5. Types of processes monitored by dashboards. The Donabedian model of patient safety management is the only theoretical model mentioned in the reviewed literature,20,21 and it is used to classify the key performance indicators in dashboards as structure, process, or outcome oriented. Structure-based key performance indicators measure aspects of the organization and can include staffing levels of different provider types or the number of patients who received healthcare services at a given point in time.22 Process-based key performance indicators are used to quantify patient care activities (eg, nursing diagnosis and treatment).20 Outcomebased key performance indicators are used to assess if predetermined goals were met and can include quality indicators such as mortality rates,20 patient satisfaction,20 or hospital-acquired infection rates.21
Benefits and Limitations of Dashboards BENEFITS OF DASHBOARDS In the reviewed literature, the most mentioned benefits of dashboards were the ability to evaluate large amounts of data and presenting the results in an easy-to-interpret visual format,4,15–18,23 to provide notifications of metrics that deviate from predefined acceptable levels to minimize adverse events,4,5,10,13,14,17,24,25 to provide decision support to improve efficiency and quality,5,12–14,23–26 and to provide decision support that promotes data-driven decision making for executive-level management.11,15,21 Other benefits mentioned in the literature include the high degree of customizability of dashboards to specific purposes18,21 and
workflows17; the ability of dashboards to coordinate patient care among different providers12,16; and the use of adequately defined key performance indicators to provide reliable and valid measurements used to evaluate performance over time.6 LIMITATIONS OF DASHBOARD UTILIZATION The identified limitations of using dashboards can be divided into sociocultural factors and technical limitations. The identified social factors include decreased acceptance of dashboards secondary to (1) clinician anxiety about electronic surveillance of their performance,26 (2) information overload caused by excessive monitoring of key performance indicators,10 and (3) use of key performance indicators that clinicians believe cannot be directly changed (eg, an RN in a surgical ICU might believe that a surgical-site infection was acquired intraoperatively, and nothing could have been done postoperatively to prevent the infection).14,26 The limitations related to organizational culture include the following: (1) dashboard metrics can have different work environment contexts (ie, work environments can be different and require slightly different metrics for evaluation)26; (2) the organizational culture has to value objective, data-driven decision making to gain the full benefits of dashboards (ie, the organization has to be willing to actually use the information gained by dashboards to make decisions)1,12; and (3) dashboard utilization requires financial and human resources.21 The technical limitations that can impair the functionality of dashboards are related to database management issues and properties of the data. The primary technical limitations that limit the usefulness of dashboards are the lack of standardized terminology used in electronic health records1 and the lack of standardized definitions for key performance indicators.12,15,18,20 The specific processes for data management (ie, the collection, entry, storage, and retrieval of data) currently used in clinical settings do not always support dashboard utilization because data are not always aggregated in a meaningful format4; data are distributed over multiple systems that do not communicate10; dashboards based on real-time data are dependent on timely documentation by healthcare providers to maintain accuracy13; and outcome measures of patient care are often not entered into the electronic health record in an easy-to-retrieve format.27 Dashboards are less efficacious for small data sets or measuring rare events because the increased variability in the key performance indicators makes it unreliable for trending purposes6,20; consequently, small organizations would not benefit as much from dashboards as larger organizations.
Evaluation of Dashboards The evaluation methods for dashboards identified in the literature include the use of predetermined criteria and desired characteristics,4,12,18,20,21 field-testing dashboards in realworld settings,7,19,21,23 obtaining user feedback on dashboard performance,4 and establishing the usefulness of
CIN: Computers, Informatics, Nursing & November 2014 Copyright © 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
547
dashboards to evaluate the effectiveness of clinical interventions.19 The predetermined criteria and characteristics for evaluating dashboards are often determined by literature reviews21 or consultation with content experts (eg, establishing a committee for using a Delphi approach of identifying important criteria).15,20,21 The evaluation criteria focus primarily on establishing a theoretical rationale to justify the criteria and assess dashboard performance in real-world conditions.
Future Clinical and Research Implications FUTURE CLINICAL IMPLICATIONS Historically, obtaining information from healthcare records was a time-consuming process that limited the practicality of retrospective chart reviews.1 As more hospitals convert to electronic documentation, the amount of digital information available for analysis is going to increase exponentially. This will provide a wealth of data to better inform evidence-based practice guidelines1 and data-driven managerial decision making.19 The role of nursing informatics specialists will be crucial in maximizing the benefits of big data and minimizing unintended consequences. Data analytics skills are recognized by hospital administrators as an extremely important ability to support quality improvement and assurance programs,1 but the majority of hospitals do not have adequately trained personnel to fill this role. Nursing informatics specialists will need to be familiar with the many methods for data analytics in healthcare. Dashboards are an effective tool to help informaticists leverage the data in electronic documentation to produce actionable knowledge to improve patient outcomes. The ethical concerns created by ‘‘big data’’ have not been adequately addressed in the literature. The only ethical issue discussed in the reviewed articles27 suggested that obtaining routine informed consent to use patient data collected from regular medical care be allowed so that real-time analysis of patient care be achieved. Currently, retrospective chart reviews do not usually require patient consent. However, realtime collection of clinical information from day-to-day patient care might blur the lines between clinical practice and research. If electronic access to medical records greatly increases the availability of patient information to researchers, will patients demand the right to decide how researchers will use their personal health information? This is just one possible concern of many. FUTURE RESEARCH IMPLICATIONS OF DASHBOARDS The future research implications of dashboards involve design issues, evaluation methodologies, and use of dashboards in outcomes research. Research is needed to identify the optimal design of the graphical user interface for dashboards (eg, placement on screen, format, and color of in548
formation to improve usability).26 Research into the optimal alert design to prevent alert fatigue is needed to ensure warnings issued by dashboards result in prompt corrective actions instead of being ignored. There is very little published research for optimal dashboard design. Because failure to assess how information technology is actually used in clinical practice is the leading cause of failed technology implementations,28 using real-world settings to evaluate the use of dashboards would be beneficial. This would allow an assessment of how the dashboard will actually be utilized by the end-users. Methods used to evaluate dashboards are not standardized, and development of a systematic approach would be very beneficial in guiding future research and clinical practice.24 Evaluation of dashboards should include17 the accuracy of data, timeliness of data, and the specific criteria used to trigger alerts. Dashboards should also be evaluated on their ability to collect data in real-world settings (ie, this criteria would address the reliability and validity of the key performance indicators measured with the dashboard).20 The largest potential research application for dashboards is outcomes-based clinical research.1 Electronic health records can be queried in real time, and the results can be rapidly translated into new evidence-based practices. More research can be conducted in a shorter time frame using dashboards to automate the data collection and analysis process.
LIMITATIONS OF THE LITERATURE REVIEW Dashboards are generic tools that are used to visually summarize large volumes of data and can be used with many different types of databases. This review is limited to describing how nurses use automated dashboards to access information in the electronic health record to facilitate decision making. This review excluded articles that described dashboards that required manual data entry or did not involve electronic health records.
SUMMARY Well-designed dashboards that are customized to specific organizational and provider needs can provide real-time feedback to healthcare providers for management of immediate problems and forewarn of potential future problems before they can endanger patient safety. Data dashboards are much like the dashboard of a car warning of immediate problems and a need for future maintenance. Research to identify standardized evaluation criteria for dashboards is needed to promote optimal design of dashboards for real-time monitoring. The use of standardized terminologies in electronic health records would allow the creation of dashboards using comparable key performance indicators to allow for benchmarking
CIN: Computers, Informatics, Nursing & November 2014 Copyright © 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
between and within organizations. This could also facilitate the documentation of compliance with national standards. The benefits of dashboards identified in literature include the ability to evaluate large amounts of data in an intuitive visual format, provide alerts for key performance metrics deviating from acceptable ranges to prompt rapid corrective interventions, provide data-driven decision support to improve the efficiency and quality of healthcare, coordinate patient care among different providers, and evaluate healthcare provider performance over time. The increased adoption of electronic documentation in healthcare settings will provide a wealth of data, and dashboards will play a pivotal role in converting these data into actionable knowledge for guiding nursing practice.
11. 12. 13. 14. 15. 16. 17. 18.
REFERENCES 1. Womack DM, Kennedy R, Bria B. Current practices in clinical analytics: a hospital survey report. In: Nursing Informatics. Proceedings of the International Congress on Nursing Informatics. Bethesda, MD: American Medical Informatics Association; 2012;2012:458. 2. Zaccagnini ME, White KW. The Doctor of Nursing Practice Essentials: A New Model for Advanced Practice Nursing. Sudbury, MA: Jones and Bartlett Publishers; 2011. 3. Mick J. Data-driven decision making: a nursing research and evidencebased practice dashboard. J Nurs Adm. 2011;41(10):391–393. 4. Baskett L, Le Rouge C, Tremblay MC. Using the dashboard technology properly. Health Prog (St Louis, MO). 2008;89(5):16–23. 5. Batley NJ, Osman HO, Kazzi AA, Musallam KM. Implementation of an emergency department computer system: design features that users value. J Emerg Med. 2011;41(6):693–700. 6. Sprague AE, Dunn SI, Fell DB, et al. Measuring quality in maternalnewborn care: developing a clinical dashboard. J Obstet Gynaecol Canada. 2013;35(1):29–38. 7. Sebastian K, Sari V, Loy LY, Zhang F, Zhang Z, Feng M. Multi-signal Visualization of Physiology (MVP): a novel visualization dashboard for physiological monitoring of Traumatic Brain Injury patients. In: Conference proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Chicago, IL: Institute of Electrical and Electronics Engineers (IEEE); 2012;2012:2000–2003. 8. Polit DF, Beck CT. Nursing Research: Generating and Assessing Evidence for Nursing Practice. 9th ed. Philadelphia, PA: Wolters Kluwer/Lippincott Williams & Wilkins; 2012. 9. Creswell JW. Qualitative Inquiry and Research Design: Choosing Among Five Approaches. 3rd ed. Los Angeles, CA: Sage Publications; 2013. 10. Simms RA, Ping H, Yelland A, Beringer AJ, Fox R, Draycott TJ. Development of maternity dashboards across a UK health region;
19. 20. 21. 22. 23. 24. 25.
26.
27. 28.
current practice, continuing problems. Eur J Obstet Gynecol Reprod Biol. 2013;170(1):119–124. Battie RN. PNDS dashboard helps showcase Magnet accomplishments. AORN J. 2009;90(2):273–277. Park KW, Smaltz D, McFadden D, Souba W. The operating room dashboard. J Surg Res. 2010;164(2):294–300. Starmer J, Giuse D. A real-time ventilator management dashboard: toward hardwiring compliance with evidence-based guidelines. AMIA Annu Symp Proc. 2008:702–706. Salazar A, Tyroch AH, Smead DG. Electronic trauma patient outcomes assessment tool: performance improvement in the trauma intensive care unit. J Trauma Nurs. 2011;18(4):197–201. Frith KH, Anderson F, Sewell JP. Assessing and selecting data for a nursing services dashboard. J Nurs Adm. 2010;40(1):10–16. Zhang J, Lu X, Nie H, Huang Z, van der Aalst WM. Radiology information system: a workflow-based approach. Int J Comput Assist Radiol Surg. 2009;4(5):509–516. Sidebottom AC, Collins B, Winden TJ, Knutson A, Britt HR. Reactions of nurses to the use of electronic health record alert features in an inpatient setting. Comput Inform Nurs. 2012;30(4):218–226; quiz 227–218. Stone-Griffith S, Englebright JD, Cheung D, Korwek KM, Perlin JB. Data-driven process and operational improvement in the emergency department: the ED Dashboard and Reporting Application. J Healthc Manag. 2012;57(3):167–180; discussion 161–180. Waitman LR, Phillips IE, McCoy AB, et al. Adopting real-time surveillance dashboards as a component of an enterprisewide medication safety strategy. Jt Comm J Qual Patient Saf. 2011;37(7):326–332. Boulkedid R, Alberti C, Sibony O. Quality indicator development and implementation in maternity units. Best Pract Res Clin Obstet Gynaecol. 2013;27(4):609–619. Blais R, Champagne F, Rousseau L. TOCSIN: a proposed dashboard of indicators to control healthcare-associated infections. Healthc Q (Toronto, ON, Canada). 2009;12 spec. no. Patient:161–167. Donabedian A. Quality assurance. Structure, process and outcome. Nurs Stand. 1992;7(11 suppl QA):4–5. Koopman RJ, Kochendorfer KM, Moore JL, et al. A diabetes dashboard and physician efficiency and accuracy in accessing data needed for high-quality diabetes care. Ann Fam Med. 2011;9(5):398–405. Harrison AM, Yadav H, Pickering BW, Cartin-Ceba R, Herasevich V. Validation of computerized automatic calculation of the sequential organ failure assessment score. Crit Care Res Pract. 2013;2013:975672. Zaydfudim V, Dossett LA, Starmer JM, et al. Implementation of a realtime compliance dashboard to help reduce SICU ventilator-associated pneumonia with the ventilator bundle. Archives of surgery (Chicago, Ill.: 1960). 2009;144(7):656–662. Redwood S, Ngwenya NB, Hodson J, Ferner RE, Coleman JJ. Effects of a computerized feedback intervention on safety performance by junior doctors: results from a randomized mixed method study. BMC Med Inform Decis Mak. 2013;13(1):63. Vrieze SI, Docherty A, Thuras P, et al. Best practices: the electronic medical record is an invaluable clinical tool: let’s start using it. Psychiatr Serv (Washington, DC). 2013;64(10):946–949. Wilbanks BA. An integrative literature review of contextual factors in perioperative information management systems. Comput Inform Nurs. 2013;31(12):622–628.
CIN: Computers, Informatics, Nursing & November 2014 Copyright © 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
549
& Vol. 32, No. 11, 545–549 & Copyright B 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins
F E A T U R E A R T I C L E
A Review of Dashboards for Data Analytics in Nursing BRYAN A. WILBANKS, DNP, CRNA PATSY A. LANGFORD, MNA, CRNA
Increased adoption of electronic health records in recent years has substantially increased the amount of readily accessible digital data for use in clinical and business decision making; however, the current methodologies used for database analysis in healthcare have been inefficient for guiding evidence-based decision making.1 Evidence-based practice is characterized by the use of the current published research and available evidence to guide the delivery of patient care with the ultimate goal of improving clinical outcomes.2 A quick and easy method to leverage the massive amount of data in electronic health records to generate actionable knowledge is needed to improve patient outcomes and clinical practices. A dashboard is defined in the literature as a data-driven clinical decision support tool capable of querying multiple databases and providing a visual representation of key performance indicators in a single report,3 often in a format visually similar to a car’s dashboard.4–6 The purpose of a dashboard is to provide a concise overview of a large volume of data; consequently, there are many different types of dashboards that are intended for different applications. A key performance indicator is a specific quantified metric (eg, surgical-site infection rates or hospital-acquired decubitus ulcers) that represents an organizational goal or a type of performance evaluation measured for the purpose of guiding managerial or clinical decisions.4,6 One type of dashboard many nurses are familiar with is the physiological vital sign monitor screen(eg, the electrocardiogram, noninvasive blood pressure cuff reading, respiratory rate, and pulse oximetry)7 used to monitor patients. The combined use of clinical dashboards with electronic health records can provide near– real-time surveillance to improve patient outcomes when appropriately configured.4 Using an inductive content anal-
Dashboards are data-driven clinical decision support tools used to analyze data from multiple databases using easy-to-read, color-coded graphical displays, much like the dashboards of automobiles. Dashboards can be used to promote data-driven decision making and improve adherence to evidencebased practice guidelines. The purpose of this article was to provide a review of dashboards used to query electronic health records for the purpose of guiding clinical practice and research. An inductive content analysis approach was used to identify emerging themes directly from the literature. Five basic dashboard properties identified include the type of database integration, visual properties, purpose, time focus (ie, retrospective, real time, or predictive), and type of process monitored. These dashboard properties are determined by the characteristics of the specific organization, user, and purpose of data analysis. Using dashboards to perform automated analytical reviews of clinical data will prove more efficient when data elements stored in electronic health records become standardized. Other limitations of dashboard use include user anxiety, information overload, and technology overload. The increased use of electronic documentation in healthcare settings will provide a wealth of data, and dashboards will play a pivotal role in converting these data into actionable knowledge. KEY WORDS Clinical analytics & Dashboard & Data analytics & Decision support & Nursing informatics
ysis approach, this article reviews the use of dashboards to perform data analytics in the hospital setting for the purpose of guiding nursing practice and research.
Author Affiliations: School of Nursing, The University of Alabama at Birmingham (Dr Wilbanks); and Huntsville Hospital (Ms Langford), AL. The authors received no funding for this study. The authors have disclosed that they have no significant relationship with, or financial interest in, any commercial companies pertaining to this article. Corresponding author: Bryan A. Wilbanks, DNP, CRNA, 105 Shady Spring Dr, Harvest, AL 35749 ([email protected]). DOI: 10.1097/CIN.0000000000000106
CIN: Computers, Informatics, Nursing & November 2014 Copyright © 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
545
SEARCH METHODOLOGY AND RESULTS In November 2013, the electronic databases PubMed and CINAHL were searched for peer-reviewed articles using the single keyword dashboard, returning a total of 398 articles. No other search terms were included because this review focuses exclusively on data analytics using dashboards. No time limitations were used in the search because of the limited number of published articles pertaining to dashboards. Inclusion criteria consisted of articles that discussed the use of dashboards to query electronic health records in hospital settings. Exclusion criteria consisted of non–Englishbased articles, articles requiring fee for access, or articles that reported the use of dashboards that required manual data collection in addition to the use of electronic health records. The initial evaluation was based on article titles and abstracts, with a total of 81 articles identified that met the specified criteria. The articles were electronically retrieved for further evaluation, and 23 articles, published between the years 2008 and 2013, met the search criteria for final data analysis. Excluded articles did not involve hospital settings or used dashboards requiring a manual chart review and data entry process. The reviewed articles consisted of literature reviews (n = 12), descriptive quantitative studies (n = 5), randomized controlled trials using mixed-methods study designs (n = 2), mixed-methods studies (n = 2), a descriptive comparative outcomes study (n = 1), and a descriptive qualitative study (n = 1). The literature review was conducted using NVivo qualitative data analysis software, version 10 (QSR International Pty Ltd, Doncaster, Victoria, Australia).
ANALYSIS AND DISCUSSION An inductive content analysis using a constant comparative technique was used to identify the major themes and descriptions of dashboards in the literature. A constant comparative technique is used to generate categorical themes directly from the data and is characterized by the continuous comparisons of the newer data with the previously collected data for the purpose of identifying similarities and differences in the emerging themes.8 This technique was chosen because of its ability to generate comprehensive and descriptive categories directly from the literature.9 The major identified themes from the analysis include: definition and properties of dashboards, applications of dashboards, benefits and limitations of dashboard use, evaluation of dashboards, and future clinical and research implications of dashboards. The reviewed articles presented very diverse descriptions of dashboards, with no contradictions, possibly reflecting that the authors of the literature in this review collectively agree on the basic properties and uses of dashboards. 546
Definition and Properties of Dashboards A dashboard is a performance measurement tool used to monitor structure, process, or outcome variables6 by querying an electronic data set and presenting the results in an easy-to-interpret graphical format.4,10 The type of organization utilizing a dashboard, the type of system user (eg, direct patient care provider or managerial staff), and the specific purpose for data analysis determine the specific properties of a dashboard.4,11 The five basic types of properties of dashboards identified include database integration, visual properties, purpose, time focus, and type of process monitored. 1. Database integration for dashboards. A dashboard integrates data from multiple databases,4,12,13 which use different types of data formatting (eg, time stamps for documented events can use the symbol ‘‘/,’’ ‘‘-,’’ or ‘‘:’’ to separate the month, day, year, and time).14 Standardized definitions for data elements need to be established to effectively use dashboards to aggregate and analyze data across multiple data sets.15 Also, the degree of database integration (ie, the ability to electronically query multiple databases without a manual chart review) will determine the degree of automation for data analysis4; it is crucial to have electronic access to all of the data needed for analysis to maximize the benefits of dashboards. 2. Visual properties of dashboards. Dashboards are typically presented in simple graphical formats providing an intuitive at-a-glance assessment of key performance indicators4,10,15 in a manner that promotes evidence-based decision making. Color-coding the graphical displays facilitates rapid interpretation of performance status6— commonly using green for ‘‘acceptable,’’ yellow for ‘‘marginal,’’ and red for ‘‘unacceptable.’’ Dashboard styles can include stacked column graphs, scatterplots, pie graphs, column graphs, area graphs, radar graphs, or a gauge style closely resembling an automobile speedometer.4 Dashboards incorporating physiological monitoring of patients use different types of waveforms (eg, electrocardiogram, electroencephalogram, or specific hemoglobin plethysmography) or numerical number presentations.7 The intricate nature of data sets and the diverse information needs of administrators make it necessary for informaticists to work closely with management personnel to optimally design the visual properties of a dashboard.15 3. Intended purpose of dashboards. The possible functions of dashboards include (1) benchmarking performance, (2) providing notifications and warnings, (3) predicting performance to guide managerial decision making,4,12 and (4) summarizing and analyzing data to provide performance feedback to guide clinical decision making. Dashboards can automate the collection of key performance indicators to prove compliance with national standards (eg, compliance with standards of The Joint Commission)10,14,16 or benchmarking the performance of
CIN: Computers, Informatics, Nursing & November 2014 Copyright © 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
healthcare providers, hospital departments, or entire hospitals for the purpose of objectively comparing the respective groups.17 This allows rapid identification of underperformers for the purpose of generating targeted interventions for performance improvement. Using dashboards for benchmarking can also assist with quality assurance and quality improvement programs.4,12,18,19 Appropriately configured dashboards can provide user notifications when a specific metric deviates from acceptable values, prompting quick corrective measures to limit or prevent adverse events (eg, identification of an excessive medication dose being ordered for a patient or an increase in surgical-site infection rates).4,6,19 4. Time focus of dashboards. Dashboards can be used for retrospective data collection in electronic health records,11,15 provide real-time monitoring of current patient care (eg, vital sign monitors or ventilator setting control panels),7,13 or analyze databases for the purpose of predicting future events.15 Even though the literature mentions that the most successful dashboards will have the ability to help predict the outcomes of decisions based on current information,15 there are no reported examples of predictive dashboards used in healthcare settings. 5. Types of processes monitored by dashboards. The Donabedian model of patient safety management is the only theoretical model mentioned in the reviewed literature,20,21 and it is used to classify the key performance indicators in dashboards as structure, process, or outcome oriented. Structure-based key performance indicators measure aspects of the organization and can include staffing levels of different provider types or the number of patients who received healthcare services at a given point in time.22 Process-based key performance indicators are used to quantify patient care activities (eg, nursing diagnosis and treatment).20 Outcomebased key performance indicators are used to assess if predetermined goals were met and can include quality indicators such as mortality rates,20 patient satisfaction,20 or hospital-acquired infection rates.21
Benefits and Limitations of Dashboards BENEFITS OF DASHBOARDS In the reviewed literature, the most mentioned benefits of dashboards were the ability to evaluate large amounts of data and presenting the results in an easy-to-interpret visual format,4,15–18,23 to provide notifications of metrics that deviate from predefined acceptable levels to minimize adverse events,4,5,10,13,14,17,24,25 to provide decision support to improve efficiency and quality,5,12–14,23–26 and to provide decision support that promotes data-driven decision making for executive-level management.11,15,21 Other benefits mentioned in the literature include the high degree of customizability of dashboards to specific purposes18,21 and
workflows17; the ability of dashboards to coordinate patient care among different providers12,16; and the use of adequately defined key performance indicators to provide reliable and valid measurements used to evaluate performance over time.6 LIMITATIONS OF DASHBOARD UTILIZATION The identified limitations of using dashboards can be divided into sociocultural factors and technical limitations. The identified social factors include decreased acceptance of dashboards secondary to (1) clinician anxiety about electronic surveillance of their performance,26 (2) information overload caused by excessive monitoring of key performance indicators,10 and (3) use of key performance indicators that clinicians believe cannot be directly changed (eg, an RN in a surgical ICU might believe that a surgical-site infection was acquired intraoperatively, and nothing could have been done postoperatively to prevent the infection).14,26 The limitations related to organizational culture include the following: (1) dashboard metrics can have different work environment contexts (ie, work environments can be different and require slightly different metrics for evaluation)26; (2) the organizational culture has to value objective, data-driven decision making to gain the full benefits of dashboards (ie, the organization has to be willing to actually use the information gained by dashboards to make decisions)1,12; and (3) dashboard utilization requires financial and human resources.21 The technical limitations that can impair the functionality of dashboards are related to database management issues and properties of the data. The primary technical limitations that limit the usefulness of dashboards are the lack of standardized terminology used in electronic health records1 and the lack of standardized definitions for key performance indicators.12,15,18,20 The specific processes for data management (ie, the collection, entry, storage, and retrieval of data) currently used in clinical settings do not always support dashboard utilization because data are not always aggregated in a meaningful format4; data are distributed over multiple systems that do not communicate10; dashboards based on real-time data are dependent on timely documentation by healthcare providers to maintain accuracy13; and outcome measures of patient care are often not entered into the electronic health record in an easy-to-retrieve format.27 Dashboards are less efficacious for small data sets or measuring rare events because the increased variability in the key performance indicators makes it unreliable for trending purposes6,20; consequently, small organizations would not benefit as much from dashboards as larger organizations.
Evaluation of Dashboards The evaluation methods for dashboards identified in the literature include the use of predetermined criteria and desired characteristics,4,12,18,20,21 field-testing dashboards in realworld settings,7,19,21,23 obtaining user feedback on dashboard performance,4 and establishing the usefulness of
CIN: Computers, Informatics, Nursing & November 2014 Copyright © 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
547
dashboards to evaluate the effectiveness of clinical interventions.19 The predetermined criteria and characteristics for evaluating dashboards are often determined by literature reviews21 or consultation with content experts (eg, establishing a committee for using a Delphi approach of identifying important criteria).15,20,21 The evaluation criteria focus primarily on establishing a theoretical rationale to justify the criteria and assess dashboard performance in real-world conditions.
Future Clinical and Research Implications FUTURE CLINICAL IMPLICATIONS Historically, obtaining information from healthcare records was a time-consuming process that limited the practicality of retrospective chart reviews.1 As more hospitals convert to electronic documentation, the amount of digital information available for analysis is going to increase exponentially. This will provide a wealth of data to better inform evidence-based practice guidelines1 and data-driven managerial decision making.19 The role of nursing informatics specialists will be crucial in maximizing the benefits of big data and minimizing unintended consequences. Data analytics skills are recognized by hospital administrators as an extremely important ability to support quality improvement and assurance programs,1 but the majority of hospitals do not have adequately trained personnel to fill this role. Nursing informatics specialists will need to be familiar with the many methods for data analytics in healthcare. Dashboards are an effective tool to help informaticists leverage the data in electronic documentation to produce actionable knowledge to improve patient outcomes. The ethical concerns created by ‘‘big data’’ have not been adequately addressed in the literature. The only ethical issue discussed in the reviewed articles27 suggested that obtaining routine informed consent to use patient data collected from regular medical care be allowed so that real-time analysis of patient care be achieved. Currently, retrospective chart reviews do not usually require patient consent. However, realtime collection of clinical information from day-to-day patient care might blur the lines between clinical practice and research. If electronic access to medical records greatly increases the availability of patient information to researchers, will patients demand the right to decide how researchers will use their personal health information? This is just one possible concern of many. FUTURE RESEARCH IMPLICATIONS OF DASHBOARDS The future research implications of dashboards involve design issues, evaluation methodologies, and use of dashboards in outcomes research. Research is needed to identify the optimal design of the graphical user interface for dashboards (eg, placement on screen, format, and color of in548
formation to improve usability).26 Research into the optimal alert design to prevent alert fatigue is needed to ensure warnings issued by dashboards result in prompt corrective actions instead of being ignored. There is very little published research for optimal dashboard design. Because failure to assess how information technology is actually used in clinical practice is the leading cause of failed technology implementations,28 using real-world settings to evaluate the use of dashboards would be beneficial. This would allow an assessment of how the dashboard will actually be utilized by the end-users. Methods used to evaluate dashboards are not standardized, and development of a systematic approach would be very beneficial in guiding future research and clinical practice.24 Evaluation of dashboards should include17 the accuracy of data, timeliness of data, and the specific criteria used to trigger alerts. Dashboards should also be evaluated on their ability to collect data in real-world settings (ie, this criteria would address the reliability and validity of the key performance indicators measured with the dashboard).20 The largest potential research application for dashboards is outcomes-based clinical research.1 Electronic health records can be queried in real time, and the results can be rapidly translated into new evidence-based practices. More research can be conducted in a shorter time frame using dashboards to automate the data collection and analysis process.
LIMITATIONS OF THE LITERATURE REVIEW Dashboards are generic tools that are used to visually summarize large volumes of data and can be used with many different types of databases. This review is limited to describing how nurses use automated dashboards to access information in the electronic health record to facilitate decision making. This review excluded articles that described dashboards that required manual data entry or did not involve electronic health records.
SUMMARY Well-designed dashboards that are customized to specific organizational and provider needs can provide real-time feedback to healthcare providers for management of immediate problems and forewarn of potential future problems before they can endanger patient safety. Data dashboards are much like the dashboard of a car warning of immediate problems and a need for future maintenance. Research to identify standardized evaluation criteria for dashboards is needed to promote optimal design of dashboards for real-time monitoring. The use of standardized terminologies in electronic health records would allow the creation of dashboards using comparable key performance indicators to allow for benchmarking
CIN: Computers, Informatics, Nursing & November 2014 Copyright © 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
between and within organizations. This could also facilitate the documentation of compliance with national standards. The benefits of dashboards identified in literature include the ability to evaluate large amounts of data in an intuitive visual format, provide alerts for key performance metrics deviating from acceptable ranges to prompt rapid corrective interventions, provide data-driven decision support to improve the efficiency and quality of healthcare, coordinate patient care among different providers, and evaluate healthcare provider performance over time. The increased adoption of electronic documentation in healthcare settings will provide a wealth of data, and dashboards will play a pivotal role in converting these data into actionable knowledge for guiding nursing practice.
11. 12. 13. 14. 15. 16. 17. 18.
REFERENCES 1. Womack DM, Kennedy R, Bria B. Current practices in clinical analytics: a hospital survey report. In: Nursing Informatics. Proceedings of the International Congress on Nursing Informatics. Bethesda, MD: American Medical Informatics Association; 2012;2012:458. 2. Zaccagnini ME, White KW. The Doctor of Nursing Practice Essentials: A New Model for Advanced Practice Nursing. Sudbury, MA: Jones and Bartlett Publishers; 2011. 3. Mick J. Data-driven decision making: a nursing research and evidencebased practice dashboard. J Nurs Adm. 2011;41(10):391–393. 4. Baskett L, Le Rouge C, Tremblay MC. Using the dashboard technology properly. Health Prog (St Louis, MO). 2008;89(5):16–23. 5. Batley NJ, Osman HO, Kazzi AA, Musallam KM. Implementation of an emergency department computer system: design features that users value. J Emerg Med. 2011;41(6):693–700. 6. Sprague AE, Dunn SI, Fell DB, et al. Measuring quality in maternalnewborn care: developing a clinical dashboard. J Obstet Gynaecol Canada. 2013;35(1):29–38. 7. Sebastian K, Sari V, Loy LY, Zhang F, Zhang Z, Feng M. Multi-signal Visualization of Physiology (MVP): a novel visualization dashboard for physiological monitoring of Traumatic Brain Injury patients. In: Conference proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Chicago, IL: Institute of Electrical and Electronics Engineers (IEEE); 2012;2012:2000–2003. 8. Polit DF, Beck CT. Nursing Research: Generating and Assessing Evidence for Nursing Practice. 9th ed. Philadelphia, PA: Wolters Kluwer/Lippincott Williams & Wilkins; 2012. 9. Creswell JW. Qualitative Inquiry and Research Design: Choosing Among Five Approaches. 3rd ed. Los Angeles, CA: Sage Publications; 2013. 10. Simms RA, Ping H, Yelland A, Beringer AJ, Fox R, Draycott TJ. Development of maternity dashboards across a UK health region;
19. 20. 21. 22. 23. 24. 25.
26.
27. 28.
current practice, continuing problems. Eur J Obstet Gynecol Reprod Biol. 2013;170(1):119–124. Battie RN. PNDS dashboard helps showcase Magnet accomplishments. AORN J. 2009;90(2):273–277. Park KW, Smaltz D, McFadden D, Souba W. The operating room dashboard. J Surg Res. 2010;164(2):294–300. Starmer J, Giuse D. A real-time ventilator management dashboard: toward hardwiring compliance with evidence-based guidelines. AMIA Annu Symp Proc. 2008:702–706. Salazar A, Tyroch AH, Smead DG. Electronic trauma patient outcomes assessment tool: performance improvement in the trauma intensive care unit. J Trauma Nurs. 2011;18(4):197–201. Frith KH, Anderson F, Sewell JP. Assessing and selecting data for a nursing services dashboard. J Nurs Adm. 2010;40(1):10–16. Zhang J, Lu X, Nie H, Huang Z, van der Aalst WM. Radiology information system: a workflow-based approach. Int J Comput Assist Radiol Surg. 2009;4(5):509–516. Sidebottom AC, Collins B, Winden TJ, Knutson A, Britt HR. Reactions of nurses to the use of electronic health record alert features in an inpatient setting. Comput Inform Nurs. 2012;30(4):218–226; quiz 227–218. Stone-Griffith S, Englebright JD, Cheung D, Korwek KM, Perlin JB. Data-driven process and operational improvement in the emergency department: the ED Dashboard and Reporting Application. J Healthc Manag. 2012;57(3):167–180; discussion 161–180. Waitman LR, Phillips IE, McCoy AB, et al. Adopting real-time surveillance dashboards as a component of an enterprisewide medication safety strategy. Jt Comm J Qual Patient Saf. 2011;37(7):326–332. Boulkedid R, Alberti C, Sibony O. Quality indicator development and implementation in maternity units. Best Pract Res Clin Obstet Gynaecol. 2013;27(4):609–619. Blais R, Champagne F, Rousseau L. TOCSIN: a proposed dashboard of indicators to control healthcare-associated infections. Healthc Q (Toronto, ON, Canada). 2009;12 spec. no. Patient:161–167. Donabedian A. Quality assurance. Structure, process and outcome. Nurs Stand. 1992;7(11 suppl QA):4–5. Koopman RJ, Kochendorfer KM, Moore JL, et al. A diabetes dashboard and physician efficiency and accuracy in accessing data needed for high-quality diabetes care. Ann Fam Med. 2011;9(5):398–405. Harrison AM, Yadav H, Pickering BW, Cartin-Ceba R, Herasevich V. Validation of computerized automatic calculation of the sequential organ failure assessment score. Crit Care Res Pract. 2013;2013:975672. Zaydfudim V, Dossett LA, Starmer JM, et al. Implementation of a realtime compliance dashboard to help reduce SICU ventilator-associated pneumonia with the ventilator bundle. Archives of surgery (Chicago, Ill.: 1960). 2009;144(7):656–662. Redwood S, Ngwenya NB, Hodson J, Ferner RE, Coleman JJ. Effects of a computerized feedback intervention on safety performance by junior doctors: results from a randomized mixed method study. BMC Med Inform Decis Mak. 2013;13(1):63. Vrieze SI, Docherty A, Thuras P, et al. Best practices: the electronic medical record is an invaluable clinical tool: let’s start using it. Psychiatr Serv (Washington, DC). 2013;64(10):946–949. Wilbanks BA. An integrative literature review of contextual factors in perioperative information management systems. Comput Inform Nurs. 2013;31(12):622–628.
CIN: Computers, Informatics, Nursing & November 2014 Copyright © 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
549
