G Model
ARTICLE IN PRESS
IJB-3215; No. of Pages 9
International Journal of Medical Informatics xxx (2015) xxx–xxx
Contents lists available at ScienceDirect
International Journal of Medical Informatics journal homepage: www.ijmijournal.com
Public health nurse perceptions of Omaha System data visualization Seonah Lee a,∗ , Era Kim b , Karen A. Monsen c a
College of Nursing, University of Missouri-St. Louis, St. Louis, MO, USA Medica Research Institute, Minnetonka, MN and Institute for Health Informatics, University of Minnesota, Minneapolis, MN, USA c School of Nursing, University of Minnesota, Minneapolis, MN, USA b
a r t i c l e
i n f o
Article history: Received 12 January 2015 Received in revised form 11 May 2015 Accepted 30 June 2015 Available online xxx Keywords: Visualization Electronic health records Omaha System Public health nurse Care plan
a b s t r a c t Background: Electronic health records (EHRs) provide many benefits related to the storage, deployment, and retrieval of large amounts of patient data. However, EHRs have not fully met the need to reuse data for decision making on follow-up care plans. Visualization offers new ways to present health data, especially in EHRs. Well-designed data visualization allows clinicians to communicate information efficiently and effectively, contributing to improved interpretation of clinical data and better patient care monitoring and decision making. Public health nurse (PHN) perceptions of Omaha System data visualization prototypes for use in EHRs have not been evaluated. Purpose: To visualize PHN-generated Omaha System data and assess PHN perceptions regarding the visual validity, helpfulness, usefulness, and importance of the visualizations, including interactive functionality. Methods: Time-oriented visualization for problems and outcomes and Matrix visualization for problems and interventions were developed using PHN-generated Omaha System data to help PHNs consume data and plan care at the point of care. Eleven PHNs evaluated prototype visualizations. Results: Overall PHNs response to visualizations was positive, and feedback for improvement was provided. Conclusion: This study demonstrated the potential for using visualization techniques within EHRs to summarize Omaha System patient data for clinicians. Further research is needed to improve and refine these visualizations and assess the potential to incorporate visualizations within clinical EHRs. © 2015 Published by Elsevier Ireland Ltd.
1. Introduction Electronic health records (EHRs) have a capacity for the storage, deployment, and retrieval of patient care-related data that surpasses that of paper-based systems. However, even with their capacity, EHRs have not fully met the need to use data in an efficient and effective way to provide decision support for patient care. Despite the rapid accumulation of healthcare data within the EHR, such data are seldom used as a resource to support decision making for current or follow-up care [1]. It is critical to develop methods to analyze and present multi-dimensional and time-oriented healthcare data within EHRs in a user-friendly or task-specific manner. Problem oriented documentation in EHRs is facilitated by the use of standardized terminologies, such as the Omaha System [2]. EHRs designed for Omaha System data contain records of care episodes that consist of identified health problems, interventions provided to solve the problems, and outcome ratings specific to the problems. Clients may have multiple care episodes during which
∗ Corresponding author. Fax: 314 516 6730. E-mail address: [email protected] (S. Lee).
public health nurses (PHNs) address the identified health problems. However, EHR data are generally presented in a linear manner that increases cognitive load and contributes to poor ergonomics of information displays. Therefore it is very time-consuming for a clinician to extract and classify meaningful information from lengthy and diverse pages [3–5]. The inability to retrieve information easily prevents PHNs from effectively and efficiently reviewing and utilizing care episodes for decision making. Thus, the data from all care episodes need to be integrated and presented in a manner conducive to understanding the interplay of multiple variables [6]. Visualization is the “communication of information using graphical representation” [7], with the goal of providing the viewer with a qualitative understanding of the information contents [7,8]. Visualization provides a powerful means to understand and monitor data when visualization techniques extract key information from large quantities of data [9]. A line graph, for example, presents discernible differences among lines up and down by giving different colors to each line. A column chart presents clear differences among columns by giving different lengths to each column. Displayed visual cues can help PHNs better interpret data. A well-designed visualization lets clinicians see or communicate meaningful patterns and exceptions in data, by taking advantage of
http://dx.doi.org/10.1016/j.ijmedinf.2015.06.010 1386-5056/© 2015 Published by Elsevier Ireland Ltd.
Please cite this article in press as: S. Lee, et al., Public health nurse perceptions of Omaha System data visualization, Int. J. Med. Inform. (2015), http://dx.doi.org/10.1016/j.ijmedinf.2015.06.010
G Model
ARTICLE IN PRESS
IJB-3215; No. of Pages 9
S. Lee et al. / International Journal of Medical Informatics xxx (2015) xxx–xxx
2 Table 1 The structure of the Omaha System.
Standardized terms
Non-standardized descriptions
Problem Classification Scheme
Problem Rating Scale for Outcomes (Problem-specific)
Intervention Scheme (Problem specific)
42 Problems
Client’s
Level 1 – Four Categories:
• signs/symptoms
• Knowledge (K) • Behavior (B) • Status (S)
• • • •
Using a 5-point Likert scale for each above from 1 (worst) to 5 (best).
Level 2 – 76 Targets
• Other signs/symptoms (specified by text)
• 75 targets (defined) • 1 other (specified by text) Level 3 – client-specific information or care description • Care description (may be structured according to Martin, 2005 examples) • Other (specified by text) • Text
visual perception that is our most powerful sense [10]. Thus, bulky Omaha System data need to be converted to graphics to provide feedback based on given care and patient data. Visualizing Omaha System data will help a PHN better understand a patient’s progress over time and make better decisions for follow-up actions [11]. 2. Literature review A number of studies have been conducted employing visualization for meaningful use of the nursing data stored in an EHR. Mamykina et al. [12] decomposed nursing narratives from two nursing home facilities into multiple-choice entries and then visualized them. Chittaro [8] displayed vital signs, symptoms, and medications on a single screen of a mobile phone. Koch et al. [3] designed an integrated information display that provides the information needed by PHNs in an intensive care unit and compared it with existing information displays. Kim et al. [13] discovered and compared nursing intervention patterns by visualizing Omaha System intervention data using stream graphs. There were several software applications integrating medical records with visual navigation tools that allowed timeline access to particular events or sets of events in a patient’s medical history [6,14–17]. These studies suggest the clinical usefulness of data visualization. None of the studies, however, have tried to visualize the combination of patient assessment, relevant nursing interventions, and patient outcomes over time. These three things are key components of nursing care [4]. Visualizations integrating these three key components have the potential to provide a meaningful overview of data for decision making based on the context of a patient’s clinical history in real time. 3. Omaha System data The Omaha System [2] is a standardized interface terminology and multidisciplinary ontology for health that is recognized by the American PHNs Association [18]. It consists of three valid, reliable instruments, the Problem Classification Scheme, the Problem Rating Scale for Outcomes, and the Intervention Scheme. There are 42 health problems in the Problem Classification Scheme, each with a definition and a unique set of sign/symptoms. There are three Likert-type ordinal measures of problem-specific client Knowledge (K), Behavior (B), and Status (S) in the Problem Rating Scale for Outcomes. There are four Categories and 76 Targets in the Intervention Scheme. Both the Problem Rating Scale for Outcomes and the Inter-
General
Teaching, Guidance, and Counseling (TGC) Treatments and Procedures (TP) Case Management (CM) Surveillance (S)
Specific
vention Scheme relate to the 42 concepts (Problems) in the Problem Classification Scheme in that each measure and intervention relates to a specific problem. Further, the Intervention Scheme hierarchy of terms describes interventions at three levels (Category, Target, and Care description). At the first, most general level, four broad categories appear: Teaching, guidance, and counseling; Treatments and procedures; Case management; and Surveillance. At the second level, there are 76 targets that provide information on care activities about the selected category and can be used with each of the four categories. However, Care description at the third, most specific level is not standardized. Instead, examples are offered in the Omaha System [2]. In addition the Care description can be customized for any practice, population, or client condition. Further, a narrative note consisting of free text relative to any ProblemCategory-Target-Care description intervention may be enabled in various software applications (Table 1) [2,19]. Little is known about PHN perceptions of visualizing Omaha System data of an individual patient. The purpose of this study was to visualize PHN-generated Omaha System data and assess PHN perceptions regarding the visual validity, helpfulness, usefulness, and importance of the visualizations, including interactive functionality. 4. Methods Approval for this mixed-methods survey research was obtained from the University of Missouri-St.Louis Institutional Review Board. This study used an existing dataset to develop true-to-life prototype visualizations for a typical client record. The data were abstracted from clinical documentation for 932 pregnant women who received family home visiting services from PHNs between 2006 and 2008. All clients were represented by a unique fictitious identifier. Omaha System Problem Classification Scheme, Intervention Scheme, and Problem Rating Scale for Outcomes data were used in the study. 4.1. Visualization The researchers (authors) developed two dynamic and interactive data visualizations designed for concurrent use in real time in a clinical EHR. Preliminary visualizations were developed and refined until prototypes were considered to be testable by consensus of the researchers. To ensure that the prototypes accurately reflected PHN services to real clients, a large dataset was explored and a
Please cite this article in press as: S. Lee, et al., Public health nurse perceptions of Omaha System data visualization, Int. J. Med. Inform. (2015), http://dx.doi.org/10.1016/j.ijmedinf.2015.06.010
G Model IJB-3215; No. of Pages 9
ARTICLE IN PRESS S. Lee et al. / International Journal of Medical Informatics xxx (2015) xxx–xxx
3
Fig. 1. Time-oriented visualization for problems and outcomes using a multi-series line graph with interactive functionality. There are eight Problems depicted with beginning Status ratings ranging from 2 (Pregnancy, Family planning, Substance use, Residence, and Income) to 4 (Postpartum). Final Status ratings range from 3 (Income) to 5 (Postpartum, Family planning, and Substance use).
Fig. 2. Interactive functionality shows enlarged circle and additional information provided when hovering over a small circle.
representative client was selected. Initially, data were extracted with a Hypertext Preprocessor (PHP)-based web application developed for this study using Eclipse [20]. The program running on Apache web server [21] received the parameter of client identification (ID) from a web browser via Hypertext Transfer Protocol (HTTP), accessed and queried the MySQL [22] database where the intervention data were loaded, processed the query results into JavaScript Object Notation (JSON) format [23], and sent the results back to the web browser. The JSON object was fed to DataDriven Documents, commonly called D3 [24]. D3 is a JavaScript library that enables manipulation of data and creation of interactive and effective visual data representations in a web browser. Data were visualized in D3 from two different orientations: (1) Time-oriented visualization for problems and outcomes (Timeoutcomes Visualization or TOV); and (2) Matrix visualization for problems and interventions (Matrix-interventions Visualization or MIV). The goal of the TOV was to present the changes in Problemspecific status rating scores during the care episode. Hence, to handle time-oriented data for multiple categorical variables, a multi-series line graph was selected (Fig. 1). The X axis indicated time from beginning to end of the care episode. The TOV consisted of a set of lines arranged in a vertical hierarchy to depict each problem as it was identified and resolved over time. The length of the Problem lines indicated the time between its first and last Status rating. Small circles were used to depict client outcomes for each problem based on Problemspecific Status ratings. Varying colors indicated the numeric value of the Status rating. For the purposes of this demonstration the Status rating was assumed to be a more conclusive rating in presenting the outcome of client care vs. Knowledge and Behavior
ratings. Interactive functionality of the visualization included hovering to show detailed KBS ratings for all KBS circles and problem lines (Figs. 2 and 3). The goal of the Matrix visualization for problems and interventions (MIV) was to depict patterns of care delivered to a client over the entire care episode. Hence, to handle high-dimensional data for multiple categorical variables, a heat map or matrix technique was selected (Fig. 4). A heat map is the most popular matrix visualization technique [25]. It consists of shaded tiles representing the content of a data matrix. The shading of each tile on a color scale represents a corresponding frequency of a Problem-Category-Target triplet in the data matrix. Patterns may be found by sorting rows and/or columns of the matrix [25]. This visualization technique is particularly effective for high-dimensional data visualization as it can display hundreds of rows and columns in a relatively compact area. High dimensional data visualization is needed to depict complex nursing care [13]. The Omaha System enables the description of such care using the four intervention Categories and 76 intervention Targets. Considering N problems were diagnosed for a client, the number of interventions is 4 × 76 × N. The goal of the MIV was to show all possible interventions for a particular client based on the client’s problems, providing a comprehensive view of the possible approaches to care. For this study, MIV shows care patterns that consist of problem, category, target, and care description, and number of care descriptions. In contrast to the TOV, time is not included as a component of MIV. Instead, the X axis displayed all Target terms, and the Y axis displayed all Problem terms repeated four times to show Category terms, which were represented by colors of the Problem terms. The shading of the color indicated the frequency of the intervention (darker = more). Thus, any Problem-Category-Target intervention and frequencies of Problem-Category-Target interventions were observable in the visualization. Interactive functionality of the visualization included sorting and hovering. Sorting options included by Category (as in Fig. 4), Problem (Fig. 5), and frequency (Fig. 6). Additionally, Care descriptions and number of interventions by Care description were shown when hovering over any cell in the MIV (Fig. 7). 4.2. Survey 4.2.1. Eligibility and recruitment of subjects Eligible subjects were PHNs who have used the Omaha System. Approval was obtained from the University of Missouri-St.Louis Institutional Review Board for the survey of healthcare professionals and was determined to be exempt from IRB regulations. The study proposal was presented and questions were answered via webinar for 10–15 min during a meeting of the Minnesota Omaha System Users group (an international grass roots organi-
Please cite this article in press as: S. Lee, et al., Public health nurse perceptions of Omaha System data visualization, Int. J. Med. Inform. (2015), http://dx.doi.org/10.1016/j.ijmedinf.2015.06.010
G Model IJB-3215; No. of Pages 9 4
ARTICLE IN PRESS S. Lee et al. / International Journal of Medical Informatics xxx (2015) xxx–xxx
Fig. 3. Interactive functionality shows additional information provided when hovering over a line.
zation promoting quality use of terminology and data) [Minnesota Omaha System Users Group, 2015]. Public health departments were contacted to seek their cooperation for study participation. After completion of the survey upon return of the documents, $25 gift cards were mailed either to the participating PHNs or their agencies (based on their preference). 4.2.2. Survey development The researchers (authors) developed survey materials to test PHN perceptions of TOV and MIV, including a letter of invitation, a PowerPoint presentation, and a document containing corresponding survey questions. The letter of invitation explained the exempt study and the fact that PHN responses and completion of the survey indicated consent to the participation in the study. The presentation described the purpose of the study and how to participate in the survey. Detailed instructions were included for viewing and interpreting each visualization as shown in the Figs presented herein (Figs. 1–7). As PHNs read through the presentation slides and observed each Fig, they were guided to respond to visualizationspecific questions. Methods for evaluating clinicians’ perceptions of visualization were not found in the literature. Therefore the researchers devel-
oped a survey on aspects of the interactive visualizations that included visual validity of the image (depicting intended content), and clinical usefulness, importance, and helpfulness. These concepts were measured using Likert-type ordinal responses to 14 items. All questions were rated on a Likert-type ordinal scale from lowest to highest score: Strongly agree (1), Agree (2), Neither agree nor disagree (3), Disagree (4), and Strongly disagree (5). These 14 questions are shown in Table 2. One question evaluated PHN preference of the sorting options for TIV (by Problem, by Category, by intervention frequency, and no preference). There were five open-ended questions soliciting rationale for the PHN responses and suggestions for improvements in the visualizations. For example, “Please share any comments regarding the usefulness of the proposed visual displays within the EHR for PHNs understanding and navigating data about patient problems over time”, and “Please share any insights about how the data visualization could be improved”. In addition, there were two questions asking the number of years working as a PHN and as a user of the Omaha System. A PHN/doctoral student who had extensive experience using the Omaha System practice evaluated the study documents and gave suggestions for improvement. After revision by the researchers and
Fig. 4. Matrix visualization for problems and interventions for one client having eight Problems shown on the Y axis, including all Categories represented by color, and Targets shown on the X axis. Shading indicates number of interventions (darker = more).
Please cite this article in press as: S. Lee, et al., Public health nurse perceptions of Omaha System data visualization, Int. J. Med. Inform. (2015), http://dx.doi.org/10.1016/j.ijmedinf.2015.06.010
G Model IJB-3215; No. of Pages 9
ARTICLE IN PRESS S. Lee et al. / International Journal of Medical Informatics xxx (2015) xxx–xxx
5
Fig. 5. Matrix visualization for problems and interventions sorted by problem.
Fig. 6. Matrix visualization for problems and interventions sorted by total intervention frequency per row (top left = highest frequency).
re-evaluation by the PHN/doctoral student, the researchers reached consensus on the survey and the documents were emailed to PHNs who volunteered to participate. 4.2.3. Data analysis For survey items with numeric responses, descriptive and nonparametric statistics were used to analyze the data. New variables were computed to summarize PHN responses for the concepts of visual validity, helpfulness, usefulness, and importance of the visualizations, including interactive functionality by averaging each
PHN’s responses as shown in Table 2. To evaluate responses to the open-ended questions, the researchers organized phrases and quotes according to visual validity, helpfulness, usefulness, importance of the visualizations, and suggestions for improvement as expressed in PHN narratives. 5. Results and discussion Eleven PHNs participated in the study, with nine completing questions regarding number of years worked as PHN and using
Please cite this article in press as: S. Lee, et al., Public health nurse perceptions of Omaha System data visualization, Int. J. Med. Inform. (2015), http://dx.doi.org/10.1016/j.ijmedinf.2015.06.010
G Model
ARTICLE IN PRESS
IJB-3215; No. of Pages 9
S. Lee et al. / International Journal of Medical Informatics xxx (2015) xxx–xxx
6
Fig. 7. Interactive functionality shows additional information provided when hovering over a cell. Problem (Income) – Category (Teaching, guidance, and counseling) – Target (Finances) – has a total of nine instances, including two interventions for Care description Community Resources, and seven interventions for Long-Range Planning/Decision Making.
the Omaha System. The majority worked at least 10 years as a PHN (average years = 10.2, S.D. = 6.5), and had used the Omaha System at least 5 years (average years = 6.1, S.D. = 4.6). A subgroup of seven PHNs had used the Omaha System for at least two years. This subgroup could be considered expert Omaha System users, versus early Omaha System users. To examine differences between the responses between expert Omaha System users and early Omaha System users, responses of the two groups were compared using a Mann-Whitney U test. There were no significant differences in perceptions of the visualizations between the expert Omaha System users and the early Omaha System users based on PHN ratings.
5.1. PHN Perceptions of time-oriented visualization for problems and outcomes (TOV) PHNs agreed that TOV had visual validity (depicted the intended concepts) (average rating = 1.91 [1 = strongly agree, 2 = agree], S.D. = .73) and there were several comments supporting these ratings such as “This is a quick overview that summarizes well the progress made”, and “I can see outcomes fairly well”. About half of PHNs agreed with the statement that “The image helps me interpret client documentation over time” (average rating = 2.55 [2 = agree, 3 = neither agree nor disagree], S.D. = .85). Comments included the following: “It is helpful to visualize client progress over time”. In
Table 2 Survey of PHN perceptions of visual validity, helpfulness, usefulness, and importance of the visualizations with results grouped by agreement (Strongly agree and agree, neither agree nor disagree, and disagree and strongly disagree.) Visualization
Aspect
Item
Strongly agree/ agree
TOV TOV TOV TOV TOV TOV TOV TOV
Visual validity Visual validity Visual validity Helpfulness Usefulness Usefulness Importance Importance
The image depicts the client’s problems The image depicts the time of services The image depicts client status at beginning and end of services The image helps me interpret client documentation over time The KBS interactivity is useful to me as a PHN The time interactivity is useful to me as a PHN The KBS interactivity is important The time interactivity is important
10(91%) 9(82%) 8(73%) 5(45%) 10(91%) 8(73%) 10(91%) 8(73%)
MIV MIV MIV MIV MIV MIV
Visual validity Visual validity Usefulness Usefulness Importance Importance
The image depicts the client’s problems The image depicts most frequent vs. least frequent interventions The sorting functionality is useful to me The intervention detail is useful to me as a PHN The sorting functionality is important The intervention detail interactivity is important
9(82%) 6 (55%) 9(82%) 10(91%) 8(73%) 10(91%)
Neither agree nor disagree
Strongly disagree/ disagree
1(9%) 2(18%) 5(45%)
1(9%) 1(9%) 1(9%) 1(9%) 1(9%)
3(27%) 1(9%) 3(27%) 1(9%) 2(18%)
2(18%)
1(9%) 3(27%) 2(18%) 1(9%) 1(9%) 1(9%)
TOV= Time-oriented visualization for problems and outcomes (Time-outcomes Visualization); MIV= Matrix visualization for problems and interventions (Matrix-interventions Visualization)
Please cite this article in press as: S. Lee, et al., Public health nurse perceptions of Omaha System data visualization, Int. J. Med. Inform. (2015), http://dx.doi.org/10.1016/j.ijmedinf.2015.06.010
G Model IJB-3215; No. of Pages 9
ARTICLE IN PRESS S. Lee et al. / International Journal of Medical Informatics xxx (2015) xxx–xxx
the responses to open-ended questions asking about the usefulness of the proposed visual displays, six (55%) of the PHNs mentioned the usefulness of the TOV. “These visualizations are an easy way to show the problems of the client. It is useful for PHNs to be able to look at a visual in order to get a quick glimpse of what’s been going on with the client or what problems the PHN/client were working on”. PHNs agreed that the interactive functionality of the TOV was important (average rating = 1.73, S.D = .79) and useful (average rating = 1.86, S.D. = .78). Comments included the following: “The KBS interactivity portion would be helpful”. PHNs gave numerous suggestions for improving the TOV, specifically in regard to time representation, color and size of lines and circles, and numeric values for outcomes. For example, PHNs suggested adding beginning and ending dates for the care episode and adding actual visit dates on the X axis to help identify patients needing more frequent attention to attain positive outcomes. In addition, one PHN suggested that current problem lines could be bolded, and solved problem lines could be faded out. Four PHNs (36%) wanted to see a visual display of KBS scores with more circles on each line at the dates of additional KBS assessments, and one suggested adding KBS numbers to the circles. Two suggested having the lines slope up and down to depict the change. One suggested enlarging circles to make them easier to see and interpret. Based on the PHNs’ comments and suggestions, a revised TOV prototype should include the following: first, KBS-assessment dates and visit dates could be presented on the X axis, presented by month, day, and year; second, interactive hovering should include the letters K, B, and S and their corresponding scores within enlarged circles. 5.2. PHN perceptions of the matrix visualization for problems and interventions (MIV) PHNs agreed that the MIV had visual validity (average rating = 2.13 [2 = agree, 3 = neither agree nor disagree], S.D. = .95). However, they differed in their preferences regarding how the visualization should be displayed (i.e. sorted by Category as in Fig. 4, Problem as in Fig. 5, or frequency as in Fig. 6). Four PHNs (36%) preferred MIV sorted by category because problems listed on the left and sorted by category were easier to follow and understand. Three PHNs (27%) preferred MIV sorted by problem. According to them, “It was easiest to see what all has been addressed in problem area at a time. It enabled PHNs to see client’s problems as a main axis to provide interventions” and “By having the problems grouped together, it would let me quickly assess all the different categories of intervening that had already been attempted. This image helps me in recognizing how much of a focus was given to various problems, coordinating within a team, and organizing the next steps in care.” Three PHNs (27%) preferred MIV sorted by intervention frequency (one of them liked MIV by problem and by intervention frequency). They commented, “This is a great way of easily determining where the PHN has been spending most of her time with the client. It gives a general sense of what’s going on with the client and which problems the PHN/client are focusing on”, and “it was easier to visualize major problems.” Two PHNs (18%) had no preference. PHNs agreed that the interactivity functionality of the MIV was important (average rating = 1.82 [1 = strongly agree, 2 = agree], S.D = 84) and useful (average rating = 1.82, S.D. = .92). Comments included the following: five PHNs (45%) stated the usefulness of the MIV by mentioning “[I] Really like the hover tool”, and “I like the interactive feature.” “Availability to sort [to know] what is working for the client and what interventions are most effective”. Nearly all (90%) of PHNs identified the importance and usefulness of the interactivity functionality to show intervention details. The interactivity may help PHNs more easily understand and interpret the high dimensional information presented in the matrix that was noted by
7
six PHNs (55%), who commented that the MIV had too much information in a small space; and in particular, that the target list on across the top of the visualization was overwhelming. PHNs gave numerous suggestions for improving the MIV, specifically in regard to limiting the number of Target terms to those in the client data (as with Problems), adding a date range, and providing the link to the original chart. Based on the PHNs’ comments and suggestions, a revised MIV prototype should include the following: a view that depicts all intervention-related data in a single screen including narrative, incorporating visit dates, and linking to original charting. 5.3. Comparison across visualizations Overall, the PHN scores across all aspects of each visualization were very similar. The average rating for all questions related to the TOV was 1.93 [1 = strongly agree, 2 = agree], (S.D. = .48) and for all questions related to the MIV was 1.98 (S.D. = .83). Thus, in general there is considerable support to pursue use of such visualizations to summarize and present clinical data in real time for nursing care planning. This conclusion is supported by PHN comments: “Any time you delete an extra step in obtaining client information, it makes follow up much more easy.”, “In research or practice, it is helpful to know and understand the customized details that can’t be provided elsewhere in the graph. The visualization of the data becomes more meaningful in my opinion. It brings the data to life, so to speak”. 5.4. Limitations and suggestions for future research Findings of this study are limited by the small sample size of 11 PHNs; however the majority had extensive experience as a PHNs and Omaha System users. The expertise of the sample may increase confidence in the findings. Nevertheless, future research should include revised versions of TOV and MIV that should be tested with a larger PHN sample. Furthermore, the static presentation of visualizations in a presentation was necessary due to technology limitations. Future research should provide actual interactive visualizations in an electronic platform to assess PHN perceptions of actual interactive functionality. There is potential that visualizations may have a positive impact on PHN workflow if they reduce time reviewing data in the EHR. Future research should evaluate the impact of such visualizations on PHN workflow and efficiency. Given the promising findings about these visualizations for information synthesis and consumption in EHRs, future research should also give attention to enhancing EHR capacity to display information and display data using programming available in the public domain such as D3, which can automatically generate TOV and MIV for individual patients. In addition, this research suggests the potential for using visualization with data from clinical data repositories, and thus there is a need for developing technologies to support such efforts. Furthermore, as the use of visualization increases in clinical EHRs, it will be important to evaluate and refine tools for assessing clinician perceptions of visualizations such as the survey used in this study. This study is potentially important in four ways. First, the research produced prototypes of individual patient-focused visualization, rather than a summarized visualization merging all patients’ data. Data visualization for individual patients is crucial to practice because it enables a PHN to see and understand an individual patient’s progress over time, which leads to better decision making for individual follow-up care plans. Second, the prototype visualizations represented all of an individual clients’ health problems, nursing interventions, and outcomes—all key factors that inform the nursing process. Third, the use of the Omaha System in practice was well suited for development of intelligent datadriven clinical decision support using visualization techniques. As
Please cite this article in press as: S. Lee, et al., Public health nurse perceptions of Omaha System data visualization, Int. J. Med. Inform. (2015), http://dx.doi.org/10.1016/j.ijmedinf.2015.06.010
G Model
ARTICLE IN PRESS
IJB-3215; No. of Pages 9
S. Lee et al. / International Journal of Medical Informatics xxx (2015) xxx–xxx
8
Summary points What was already known on the topic? • EHRs provide clinicians with the advantages for storage, deployment, and retrieval of large amounts of patient carerelated data. However, EHRs do not usually provide an efficient view of the data accumulated in a linear manner. It is very time-consuming for clinicians to extract meaningful information from the lengthy records. • Standardized nursing terminology used in EHRs provides the benefits of efficient recording and better communication among clinicians. However, it is still time-consuming to review the data typed using standardized terms to grasp a patient’ progress and monitor patient care. What this study added to our knowledge? • Nursing data including text and numbers could be visualized for improved interpretation of patient data and better patient care monitoring and decision making. It increases the benefits of the standardized Omaha System terminology for the use in nursing practice. • There was considerable support by public health nurses to pursue use of visualizations to summarize and present clinical data at the point of care for patient care monitoring and planning. • This study indicated the potential for using visualization techniques within EHRs to summarize nurse-generated patient data. It allows visualized data to be easily integrated into nurses’ workflow and enables nurses to review the visualizations in real time at the point care.
an extension of these three key points, the use of standardized nursing language has potential to improve communication among PHNs and other health care providers and increased visibility of nursing practice [26]. Thus visualizations such as TOV and MIV may maximize the benefits of using Omaha System data by expanding the capacity of PHNs to see results and inform future care decisions. 6. Conclusion Clinical data visualizations were developed using PHNgenerated Omaha System data. This study examined PHN perceptions of the visualizations, and demonstrated the potential for using visualization techniques within EHRs to summarize patient data for clinicians. Further research is needed to improve and refine these visualizations and assess the potential to incorporate visualizations within clinical EHRs for improved patient-centered decision-making by PHNs. Funding This study was supported by the research funding of University of Missouri-St. Louis. Author contributions Seonah Lee: Developed the concept and design of the study; developed the initial visualization; conducted the analysis and interpretation of data; wrote and revised the paper from first to final for important intellectual content; gave the final approval of the version to be submitted. Era Kim: Conducted data cleaning; developed and refined the visualization; wrote the content about visualizations in the sections of Methods.
Karen A. Monsen: Assisted access to Omaha System data; assisted nurse survey including recruitment of public health nurses; revised the paper for improvement.
Conflicts of interest None of authors has no conflict of interest.
References [1] D.A. Keim, F. Mansmann, J. Schneidewind, J. Thomas, H. Ziegler, Visual analytics: scope and challenges, in: S.J. Simoff, M.H. Bohlen, A. Mazeika (Eds.), Visual Data Mining: Theory, Techniques and Tools for Visual Analytics, LNCS 4404, Springer-Verlag, Berlin Heidelberg, Germany, 2008, pp. 76–90, Available at http://link.springer.com/chapte/10.1007%2F978-3-540-71080-6 6#page-1 [2] K.S. Martin, The Omaha System: A key to Practice, Documentation, and Information Management, 2nd ed., Health Connections Press, Omaha, NE, 2005. [3] S.H. Koch, N. Staggers, C. Weir, J. Agutter, D. Liu, D.R. Westenskow, Integrated information displays for ICU PHNs: field observations, display design, and display evaluation, in: Proceedings from the 54th Annual Meeting of the Human Factors and Ergonomics Society, San Francisco, California, September 27–October 1, 2010, Available at http://igva2012wikispaces.asu.edu/file/ view/Integrated+Information+Displays+for+ICU+PHNs.pdf/315703330/ Integrated+Information+Displays+for+ICU+PHNs.pdf [4] W. Blair, B. Smith, Nursing documentation: frameworks and barriers, Contemp. PHN 41 (2012) 160–168. [5] B. Shneiderman, C. Plaisant, B.W. Hesse, Improving health and healthcare with interactive visualization methods, 2013. Avaialble at http://hcil2.cs.umd. edu/trs/2013-01/2013-01.pdf [6] A.A.T. Bui, D.R. Aberle, H. Kangarloo, Timeline: visualizing integrated patient records, IEEE T. Inf. Technol. Biomed. 11 (2007) 462–473, http://dx.doi.org/10. 1109/TITB.2006.884365 [7] M. Ward, Overview of data visualization, Available at http://web.cs.wpi.edu/ ?matt/courses/cs563/talks/datavis.html [8] L. Chittaro, Visualization of patient data at different temporal granularities on mobile devices, in: Proceedings from AVI ’06: the Working Conference on Advanced Visual Interfaces, Venezia, Italy, May 23–26, 2006, Available at http://dl.acm.org/citation.cfm?id=1133364 [9] O. Farri, A. Rahman, K.A. Monsen, R. Zhang, S.V. Pakhomov, D.S. Pieczkiewicz, S.M. Speedie, G.B. Melton, Impact of a prototype visualization tool for new information in EHR clinical documents, Appl. Clin. Inf. 3 (2012) 404–418, http://dx.doi.org/10.4338/ACI-2012-05-RA-0017 [10] S. Fang, M. Lwin, P. Ebright, Visualization of unstructured text sequences of nursing narratives, in: Proceedings from SAC’06: the 2006 ACM Symposium on Applied Computing, Dijon, France, April 23–27, 2006, Available at http://dl. acm.org/citation.cfm?id=1141331 [11] P. Hanrahan, C. Stolte, J. Mackinlay, Tableau software selecting a visual analytics application, 2009 Available at http://mkt.tableausoftware.com/files/ selecting-visual-analytics-app.pdf [12] L. Mamykina, S. Goose, D. Hedqvist, D.V. Beard, Careview: analyzing nursing narratives for temporal trends, in: Proceedings from CHI EA ’04: Extended Abstracts of Conference on Human Factors in Computing Systems, Vienna, Austria, April 24–29, 2004, Available at http://dl.acm.org/citation. cfm?doid=985921.986010 [13] E. Kim, K.A. Monsen, D.S. Pieczkiewicz, Visualization of Omaha System data enables data-driven analysis of Outcomes, American Medical Informatics Association, 2013. [14] T.D. Wang, C. Plaisant, A.J. Quinn, R. Stanchak, B. Shneiderman, S. Murphy, Aligning temporal data by sentinel events: discovering patterns in electronic health records, in: Proceedings from CHI ’08: the SIGCHI Conference on Human Factors in Computing Systems, Florence, Italy, April 5–10, 2008, Available at https://www.cs.umd.edu/∼ben/papers/Wang2008Aligning.pdf [15] C. Plaisant, B. Milash, A. Rose, S. Widoff, B. Shneiderman, Lifelines: visualizing personal histories, in: Proceedings from CHI’96: the SIGCHI Conference on Human Factors in Computing Systems, Vancouver, BC Canada, April 13–18, 1996, Available at http://www.cs.umd.edu/?ben/papers/ Plaisant1996LifeLines.pdf [16] C. Hallett, Multi-modal presentation of medical histories, in: Proceedings from IUI ‘08: the 13th International Conference on Intelligent User Interfaces, Canary Islands, Spain, January 13–16, 2008, Available at http://www.sussex. ac.uk/Users/drs22/projects/CLEF/publications/hallett-IUI.pdf [17] W. Aigner, S. Miksch, Carevis: integrated visualization of computerized protocols and temporal patient data, Artif. Intell. Med. 37 (2006) 203–218. [18] American PHNs Association, ANA. recognized terminologies and data element sets, 2006 May 11. Available at http://nursingworld.org/npii/terminologies. htm [19] The Omaha System. Available at http://www.omahasystem.org/overview. html [20] Eclipse Available at https://www.eclipse.org/
Please cite this article in press as: S. Lee, et al., Public health nurse perceptions of Omaha System data visualization, Int. J. Med. Inform. (2015), http://dx.doi.org/10.1016/j.ijmedinf.2015.06.010
G Model IJB-3215; No. of Pages 9
ARTICLE IN PRESS S. Lee et al. / International Journal of Medical Informatics xxx (2015) xxx–xxx
[21] [22] [23] [24] [25]
Apache Available at http://htt.apache.org/ MySQL. Available at http://www.mysql.com/ JSON, Available at http://www.json.org/ D3js, Available at http://d3js.org/ L. Wilkinson, M. Friendly, The History of the Cluster Heat Map, The American Statistician, 2015.
[26] M.A. Rutherford, Standardized nursing language: what does it mean for nursing practice? Online J. Issues Nurs. 13 (2008), Available at http://www. nursingworld.org/MainMenuCategories/ThePracticeofProfessionalNursing/ Health-IT/StandardizedNursingLanguage.html
Please cite this article in press as: S. Lee, et al., Public health nurse perceptions of Omaha System data visualization, Int. J. Med. Inform. (2015), http://dx.doi.org/10.1016/j.ijmedinf.2015.06.010
9
ARTICLE IN PRESS
IJB-3215; No. of Pages 9
International Journal of Medical Informatics xxx (2015) xxx–xxx
Contents lists available at ScienceDirect
International Journal of Medical Informatics journal homepage: www.ijmijournal.com
Public health nurse perceptions of Omaha System data visualization Seonah Lee a,∗ , Era Kim b , Karen A. Monsen c a
College of Nursing, University of Missouri-St. Louis, St. Louis, MO, USA Medica Research Institute, Minnetonka, MN and Institute for Health Informatics, University of Minnesota, Minneapolis, MN, USA c School of Nursing, University of Minnesota, Minneapolis, MN, USA b
a r t i c l e
i n f o
Article history: Received 12 January 2015 Received in revised form 11 May 2015 Accepted 30 June 2015 Available online xxx Keywords: Visualization Electronic health records Omaha System Public health nurse Care plan
a b s t r a c t Background: Electronic health records (EHRs) provide many benefits related to the storage, deployment, and retrieval of large amounts of patient data. However, EHRs have not fully met the need to reuse data for decision making on follow-up care plans. Visualization offers new ways to present health data, especially in EHRs. Well-designed data visualization allows clinicians to communicate information efficiently and effectively, contributing to improved interpretation of clinical data and better patient care monitoring and decision making. Public health nurse (PHN) perceptions of Omaha System data visualization prototypes for use in EHRs have not been evaluated. Purpose: To visualize PHN-generated Omaha System data and assess PHN perceptions regarding the visual validity, helpfulness, usefulness, and importance of the visualizations, including interactive functionality. Methods: Time-oriented visualization for problems and outcomes and Matrix visualization for problems and interventions were developed using PHN-generated Omaha System data to help PHNs consume data and plan care at the point of care. Eleven PHNs evaluated prototype visualizations. Results: Overall PHNs response to visualizations was positive, and feedback for improvement was provided. Conclusion: This study demonstrated the potential for using visualization techniques within EHRs to summarize Omaha System patient data for clinicians. Further research is needed to improve and refine these visualizations and assess the potential to incorporate visualizations within clinical EHRs. © 2015 Published by Elsevier Ireland Ltd.
1. Introduction Electronic health records (EHRs) have a capacity for the storage, deployment, and retrieval of patient care-related data that surpasses that of paper-based systems. However, even with their capacity, EHRs have not fully met the need to use data in an efficient and effective way to provide decision support for patient care. Despite the rapid accumulation of healthcare data within the EHR, such data are seldom used as a resource to support decision making for current or follow-up care [1]. It is critical to develop methods to analyze and present multi-dimensional and time-oriented healthcare data within EHRs in a user-friendly or task-specific manner. Problem oriented documentation in EHRs is facilitated by the use of standardized terminologies, such as the Omaha System [2]. EHRs designed for Omaha System data contain records of care episodes that consist of identified health problems, interventions provided to solve the problems, and outcome ratings specific to the problems. Clients may have multiple care episodes during which
∗ Corresponding author. Fax: 314 516 6730. E-mail address: [email protected] (S. Lee).
public health nurses (PHNs) address the identified health problems. However, EHR data are generally presented in a linear manner that increases cognitive load and contributes to poor ergonomics of information displays. Therefore it is very time-consuming for a clinician to extract and classify meaningful information from lengthy and diverse pages [3–5]. The inability to retrieve information easily prevents PHNs from effectively and efficiently reviewing and utilizing care episodes for decision making. Thus, the data from all care episodes need to be integrated and presented in a manner conducive to understanding the interplay of multiple variables [6]. Visualization is the “communication of information using graphical representation” [7], with the goal of providing the viewer with a qualitative understanding of the information contents [7,8]. Visualization provides a powerful means to understand and monitor data when visualization techniques extract key information from large quantities of data [9]. A line graph, for example, presents discernible differences among lines up and down by giving different colors to each line. A column chart presents clear differences among columns by giving different lengths to each column. Displayed visual cues can help PHNs better interpret data. A well-designed visualization lets clinicians see or communicate meaningful patterns and exceptions in data, by taking advantage of
http://dx.doi.org/10.1016/j.ijmedinf.2015.06.010 1386-5056/© 2015 Published by Elsevier Ireland Ltd.
Please cite this article in press as: S. Lee, et al., Public health nurse perceptions of Omaha System data visualization, Int. J. Med. Inform. (2015), http://dx.doi.org/10.1016/j.ijmedinf.2015.06.010
G Model
ARTICLE IN PRESS
IJB-3215; No. of Pages 9
S. Lee et al. / International Journal of Medical Informatics xxx (2015) xxx–xxx
2 Table 1 The structure of the Omaha System.
Standardized terms
Non-standardized descriptions
Problem Classification Scheme
Problem Rating Scale for Outcomes (Problem-specific)
Intervention Scheme (Problem specific)
42 Problems
Client’s
Level 1 – Four Categories:
• signs/symptoms
• Knowledge (K) • Behavior (B) • Status (S)
• • • •
Using a 5-point Likert scale for each above from 1 (worst) to 5 (best).
Level 2 – 76 Targets
• Other signs/symptoms (specified by text)
• 75 targets (defined) • 1 other (specified by text) Level 3 – client-specific information or care description • Care description (may be structured according to Martin, 2005 examples) • Other (specified by text) • Text
visual perception that is our most powerful sense [10]. Thus, bulky Omaha System data need to be converted to graphics to provide feedback based on given care and patient data. Visualizing Omaha System data will help a PHN better understand a patient’s progress over time and make better decisions for follow-up actions [11]. 2. Literature review A number of studies have been conducted employing visualization for meaningful use of the nursing data stored in an EHR. Mamykina et al. [12] decomposed nursing narratives from two nursing home facilities into multiple-choice entries and then visualized them. Chittaro [8] displayed vital signs, symptoms, and medications on a single screen of a mobile phone. Koch et al. [3] designed an integrated information display that provides the information needed by PHNs in an intensive care unit and compared it with existing information displays. Kim et al. [13] discovered and compared nursing intervention patterns by visualizing Omaha System intervention data using stream graphs. There were several software applications integrating medical records with visual navigation tools that allowed timeline access to particular events or sets of events in a patient’s medical history [6,14–17]. These studies suggest the clinical usefulness of data visualization. None of the studies, however, have tried to visualize the combination of patient assessment, relevant nursing interventions, and patient outcomes over time. These three things are key components of nursing care [4]. Visualizations integrating these three key components have the potential to provide a meaningful overview of data for decision making based on the context of a patient’s clinical history in real time. 3. Omaha System data The Omaha System [2] is a standardized interface terminology and multidisciplinary ontology for health that is recognized by the American PHNs Association [18]. It consists of three valid, reliable instruments, the Problem Classification Scheme, the Problem Rating Scale for Outcomes, and the Intervention Scheme. There are 42 health problems in the Problem Classification Scheme, each with a definition and a unique set of sign/symptoms. There are three Likert-type ordinal measures of problem-specific client Knowledge (K), Behavior (B), and Status (S) in the Problem Rating Scale for Outcomes. There are four Categories and 76 Targets in the Intervention Scheme. Both the Problem Rating Scale for Outcomes and the Inter-
General
Teaching, Guidance, and Counseling (TGC) Treatments and Procedures (TP) Case Management (CM) Surveillance (S)
Specific
vention Scheme relate to the 42 concepts (Problems) in the Problem Classification Scheme in that each measure and intervention relates to a specific problem. Further, the Intervention Scheme hierarchy of terms describes interventions at three levels (Category, Target, and Care description). At the first, most general level, four broad categories appear: Teaching, guidance, and counseling; Treatments and procedures; Case management; and Surveillance. At the second level, there are 76 targets that provide information on care activities about the selected category and can be used with each of the four categories. However, Care description at the third, most specific level is not standardized. Instead, examples are offered in the Omaha System [2]. In addition the Care description can be customized for any practice, population, or client condition. Further, a narrative note consisting of free text relative to any ProblemCategory-Target-Care description intervention may be enabled in various software applications (Table 1) [2,19]. Little is known about PHN perceptions of visualizing Omaha System data of an individual patient. The purpose of this study was to visualize PHN-generated Omaha System data and assess PHN perceptions regarding the visual validity, helpfulness, usefulness, and importance of the visualizations, including interactive functionality. 4. Methods Approval for this mixed-methods survey research was obtained from the University of Missouri-St.Louis Institutional Review Board. This study used an existing dataset to develop true-to-life prototype visualizations for a typical client record. The data were abstracted from clinical documentation for 932 pregnant women who received family home visiting services from PHNs between 2006 and 2008. All clients were represented by a unique fictitious identifier. Omaha System Problem Classification Scheme, Intervention Scheme, and Problem Rating Scale for Outcomes data were used in the study. 4.1. Visualization The researchers (authors) developed two dynamic and interactive data visualizations designed for concurrent use in real time in a clinical EHR. Preliminary visualizations were developed and refined until prototypes were considered to be testable by consensus of the researchers. To ensure that the prototypes accurately reflected PHN services to real clients, a large dataset was explored and a
Please cite this article in press as: S. Lee, et al., Public health nurse perceptions of Omaha System data visualization, Int. J. Med. Inform. (2015), http://dx.doi.org/10.1016/j.ijmedinf.2015.06.010
G Model IJB-3215; No. of Pages 9
ARTICLE IN PRESS S. Lee et al. / International Journal of Medical Informatics xxx (2015) xxx–xxx
3
Fig. 1. Time-oriented visualization for problems and outcomes using a multi-series line graph with interactive functionality. There are eight Problems depicted with beginning Status ratings ranging from 2 (Pregnancy, Family planning, Substance use, Residence, and Income) to 4 (Postpartum). Final Status ratings range from 3 (Income) to 5 (Postpartum, Family planning, and Substance use).
Fig. 2. Interactive functionality shows enlarged circle and additional information provided when hovering over a small circle.
representative client was selected. Initially, data were extracted with a Hypertext Preprocessor (PHP)-based web application developed for this study using Eclipse [20]. The program running on Apache web server [21] received the parameter of client identification (ID) from a web browser via Hypertext Transfer Protocol (HTTP), accessed and queried the MySQL [22] database where the intervention data were loaded, processed the query results into JavaScript Object Notation (JSON) format [23], and sent the results back to the web browser. The JSON object was fed to DataDriven Documents, commonly called D3 [24]. D3 is a JavaScript library that enables manipulation of data and creation of interactive and effective visual data representations in a web browser. Data were visualized in D3 from two different orientations: (1) Time-oriented visualization for problems and outcomes (Timeoutcomes Visualization or TOV); and (2) Matrix visualization for problems and interventions (Matrix-interventions Visualization or MIV). The goal of the TOV was to present the changes in Problemspecific status rating scores during the care episode. Hence, to handle time-oriented data for multiple categorical variables, a multi-series line graph was selected (Fig. 1). The X axis indicated time from beginning to end of the care episode. The TOV consisted of a set of lines arranged in a vertical hierarchy to depict each problem as it was identified and resolved over time. The length of the Problem lines indicated the time between its first and last Status rating. Small circles were used to depict client outcomes for each problem based on Problemspecific Status ratings. Varying colors indicated the numeric value of the Status rating. For the purposes of this demonstration the Status rating was assumed to be a more conclusive rating in presenting the outcome of client care vs. Knowledge and Behavior
ratings. Interactive functionality of the visualization included hovering to show detailed KBS ratings for all KBS circles and problem lines (Figs. 2 and 3). The goal of the Matrix visualization for problems and interventions (MIV) was to depict patterns of care delivered to a client over the entire care episode. Hence, to handle high-dimensional data for multiple categorical variables, a heat map or matrix technique was selected (Fig. 4). A heat map is the most popular matrix visualization technique [25]. It consists of shaded tiles representing the content of a data matrix. The shading of each tile on a color scale represents a corresponding frequency of a Problem-Category-Target triplet in the data matrix. Patterns may be found by sorting rows and/or columns of the matrix [25]. This visualization technique is particularly effective for high-dimensional data visualization as it can display hundreds of rows and columns in a relatively compact area. High dimensional data visualization is needed to depict complex nursing care [13]. The Omaha System enables the description of such care using the four intervention Categories and 76 intervention Targets. Considering N problems were diagnosed for a client, the number of interventions is 4 × 76 × N. The goal of the MIV was to show all possible interventions for a particular client based on the client’s problems, providing a comprehensive view of the possible approaches to care. For this study, MIV shows care patterns that consist of problem, category, target, and care description, and number of care descriptions. In contrast to the TOV, time is not included as a component of MIV. Instead, the X axis displayed all Target terms, and the Y axis displayed all Problem terms repeated four times to show Category terms, which were represented by colors of the Problem terms. The shading of the color indicated the frequency of the intervention (darker = more). Thus, any Problem-Category-Target intervention and frequencies of Problem-Category-Target interventions were observable in the visualization. Interactive functionality of the visualization included sorting and hovering. Sorting options included by Category (as in Fig. 4), Problem (Fig. 5), and frequency (Fig. 6). Additionally, Care descriptions and number of interventions by Care description were shown when hovering over any cell in the MIV (Fig. 7). 4.2. Survey 4.2.1. Eligibility and recruitment of subjects Eligible subjects were PHNs who have used the Omaha System. Approval was obtained from the University of Missouri-St.Louis Institutional Review Board for the survey of healthcare professionals and was determined to be exempt from IRB regulations. The study proposal was presented and questions were answered via webinar for 10–15 min during a meeting of the Minnesota Omaha System Users group (an international grass roots organi-
Please cite this article in press as: S. Lee, et al., Public health nurse perceptions of Omaha System data visualization, Int. J. Med. Inform. (2015), http://dx.doi.org/10.1016/j.ijmedinf.2015.06.010
G Model IJB-3215; No. of Pages 9 4
ARTICLE IN PRESS S. Lee et al. / International Journal of Medical Informatics xxx (2015) xxx–xxx
Fig. 3. Interactive functionality shows additional information provided when hovering over a line.
zation promoting quality use of terminology and data) [Minnesota Omaha System Users Group, 2015]. Public health departments were contacted to seek their cooperation for study participation. After completion of the survey upon return of the documents, $25 gift cards were mailed either to the participating PHNs or their agencies (based on their preference). 4.2.2. Survey development The researchers (authors) developed survey materials to test PHN perceptions of TOV and MIV, including a letter of invitation, a PowerPoint presentation, and a document containing corresponding survey questions. The letter of invitation explained the exempt study and the fact that PHN responses and completion of the survey indicated consent to the participation in the study. The presentation described the purpose of the study and how to participate in the survey. Detailed instructions were included for viewing and interpreting each visualization as shown in the Figs presented herein (Figs. 1–7). As PHNs read through the presentation slides and observed each Fig, they were guided to respond to visualizationspecific questions. Methods for evaluating clinicians’ perceptions of visualization were not found in the literature. Therefore the researchers devel-
oped a survey on aspects of the interactive visualizations that included visual validity of the image (depicting intended content), and clinical usefulness, importance, and helpfulness. These concepts were measured using Likert-type ordinal responses to 14 items. All questions were rated on a Likert-type ordinal scale from lowest to highest score: Strongly agree (1), Agree (2), Neither agree nor disagree (3), Disagree (4), and Strongly disagree (5). These 14 questions are shown in Table 2. One question evaluated PHN preference of the sorting options for TIV (by Problem, by Category, by intervention frequency, and no preference). There were five open-ended questions soliciting rationale for the PHN responses and suggestions for improvements in the visualizations. For example, “Please share any comments regarding the usefulness of the proposed visual displays within the EHR for PHNs understanding and navigating data about patient problems over time”, and “Please share any insights about how the data visualization could be improved”. In addition, there were two questions asking the number of years working as a PHN and as a user of the Omaha System. A PHN/doctoral student who had extensive experience using the Omaha System practice evaluated the study documents and gave suggestions for improvement. After revision by the researchers and
Fig. 4. Matrix visualization for problems and interventions for one client having eight Problems shown on the Y axis, including all Categories represented by color, and Targets shown on the X axis. Shading indicates number of interventions (darker = more).
Please cite this article in press as: S. Lee, et al., Public health nurse perceptions of Omaha System data visualization, Int. J. Med. Inform. (2015), http://dx.doi.org/10.1016/j.ijmedinf.2015.06.010
G Model IJB-3215; No. of Pages 9
ARTICLE IN PRESS S. Lee et al. / International Journal of Medical Informatics xxx (2015) xxx–xxx
5
Fig. 5. Matrix visualization for problems and interventions sorted by problem.
Fig. 6. Matrix visualization for problems and interventions sorted by total intervention frequency per row (top left = highest frequency).
re-evaluation by the PHN/doctoral student, the researchers reached consensus on the survey and the documents were emailed to PHNs who volunteered to participate. 4.2.3. Data analysis For survey items with numeric responses, descriptive and nonparametric statistics were used to analyze the data. New variables were computed to summarize PHN responses for the concepts of visual validity, helpfulness, usefulness, and importance of the visualizations, including interactive functionality by averaging each
PHN’s responses as shown in Table 2. To evaluate responses to the open-ended questions, the researchers organized phrases and quotes according to visual validity, helpfulness, usefulness, importance of the visualizations, and suggestions for improvement as expressed in PHN narratives. 5. Results and discussion Eleven PHNs participated in the study, with nine completing questions regarding number of years worked as PHN and using
Please cite this article in press as: S. Lee, et al., Public health nurse perceptions of Omaha System data visualization, Int. J. Med. Inform. (2015), http://dx.doi.org/10.1016/j.ijmedinf.2015.06.010
G Model
ARTICLE IN PRESS
IJB-3215; No. of Pages 9
S. Lee et al. / International Journal of Medical Informatics xxx (2015) xxx–xxx
6
Fig. 7. Interactive functionality shows additional information provided when hovering over a cell. Problem (Income) – Category (Teaching, guidance, and counseling) – Target (Finances) – has a total of nine instances, including two interventions for Care description Community Resources, and seven interventions for Long-Range Planning/Decision Making.
the Omaha System. The majority worked at least 10 years as a PHN (average years = 10.2, S.D. = 6.5), and had used the Omaha System at least 5 years (average years = 6.1, S.D. = 4.6). A subgroup of seven PHNs had used the Omaha System for at least two years. This subgroup could be considered expert Omaha System users, versus early Omaha System users. To examine differences between the responses between expert Omaha System users and early Omaha System users, responses of the two groups were compared using a Mann-Whitney U test. There were no significant differences in perceptions of the visualizations between the expert Omaha System users and the early Omaha System users based on PHN ratings.
5.1. PHN Perceptions of time-oriented visualization for problems and outcomes (TOV) PHNs agreed that TOV had visual validity (depicted the intended concepts) (average rating = 1.91 [1 = strongly agree, 2 = agree], S.D. = .73) and there were several comments supporting these ratings such as “This is a quick overview that summarizes well the progress made”, and “I can see outcomes fairly well”. About half of PHNs agreed with the statement that “The image helps me interpret client documentation over time” (average rating = 2.55 [2 = agree, 3 = neither agree nor disagree], S.D. = .85). Comments included the following: “It is helpful to visualize client progress over time”. In
Table 2 Survey of PHN perceptions of visual validity, helpfulness, usefulness, and importance of the visualizations with results grouped by agreement (Strongly agree and agree, neither agree nor disagree, and disagree and strongly disagree.) Visualization
Aspect
Item
Strongly agree/ agree
TOV TOV TOV TOV TOV TOV TOV TOV
Visual validity Visual validity Visual validity Helpfulness Usefulness Usefulness Importance Importance
The image depicts the client’s problems The image depicts the time of services The image depicts client status at beginning and end of services The image helps me interpret client documentation over time The KBS interactivity is useful to me as a PHN The time interactivity is useful to me as a PHN The KBS interactivity is important The time interactivity is important
10(91%) 9(82%) 8(73%) 5(45%) 10(91%) 8(73%) 10(91%) 8(73%)
MIV MIV MIV MIV MIV MIV
Visual validity Visual validity Usefulness Usefulness Importance Importance
The image depicts the client’s problems The image depicts most frequent vs. least frequent interventions The sorting functionality is useful to me The intervention detail is useful to me as a PHN The sorting functionality is important The intervention detail interactivity is important
9(82%) 6 (55%) 9(82%) 10(91%) 8(73%) 10(91%)
Neither agree nor disagree
Strongly disagree/ disagree
1(9%) 2(18%) 5(45%)
1(9%) 1(9%) 1(9%) 1(9%) 1(9%)
3(27%) 1(9%) 3(27%) 1(9%) 2(18%)
2(18%)
1(9%) 3(27%) 2(18%) 1(9%) 1(9%) 1(9%)
TOV= Time-oriented visualization for problems and outcomes (Time-outcomes Visualization); MIV= Matrix visualization for problems and interventions (Matrix-interventions Visualization)
Please cite this article in press as: S. Lee, et al., Public health nurse perceptions of Omaha System data visualization, Int. J. Med. Inform. (2015), http://dx.doi.org/10.1016/j.ijmedinf.2015.06.010
G Model IJB-3215; No. of Pages 9
ARTICLE IN PRESS S. Lee et al. / International Journal of Medical Informatics xxx (2015) xxx–xxx
the responses to open-ended questions asking about the usefulness of the proposed visual displays, six (55%) of the PHNs mentioned the usefulness of the TOV. “These visualizations are an easy way to show the problems of the client. It is useful for PHNs to be able to look at a visual in order to get a quick glimpse of what’s been going on with the client or what problems the PHN/client were working on”. PHNs agreed that the interactive functionality of the TOV was important (average rating = 1.73, S.D = .79) and useful (average rating = 1.86, S.D. = .78). Comments included the following: “The KBS interactivity portion would be helpful”. PHNs gave numerous suggestions for improving the TOV, specifically in regard to time representation, color and size of lines and circles, and numeric values for outcomes. For example, PHNs suggested adding beginning and ending dates for the care episode and adding actual visit dates on the X axis to help identify patients needing more frequent attention to attain positive outcomes. In addition, one PHN suggested that current problem lines could be bolded, and solved problem lines could be faded out. Four PHNs (36%) wanted to see a visual display of KBS scores with more circles on each line at the dates of additional KBS assessments, and one suggested adding KBS numbers to the circles. Two suggested having the lines slope up and down to depict the change. One suggested enlarging circles to make them easier to see and interpret. Based on the PHNs’ comments and suggestions, a revised TOV prototype should include the following: first, KBS-assessment dates and visit dates could be presented on the X axis, presented by month, day, and year; second, interactive hovering should include the letters K, B, and S and their corresponding scores within enlarged circles. 5.2. PHN perceptions of the matrix visualization for problems and interventions (MIV) PHNs agreed that the MIV had visual validity (average rating = 2.13 [2 = agree, 3 = neither agree nor disagree], S.D. = .95). However, they differed in their preferences regarding how the visualization should be displayed (i.e. sorted by Category as in Fig. 4, Problem as in Fig. 5, or frequency as in Fig. 6). Four PHNs (36%) preferred MIV sorted by category because problems listed on the left and sorted by category were easier to follow and understand. Three PHNs (27%) preferred MIV sorted by problem. According to them, “It was easiest to see what all has been addressed in problem area at a time. It enabled PHNs to see client’s problems as a main axis to provide interventions” and “By having the problems grouped together, it would let me quickly assess all the different categories of intervening that had already been attempted. This image helps me in recognizing how much of a focus was given to various problems, coordinating within a team, and organizing the next steps in care.” Three PHNs (27%) preferred MIV sorted by intervention frequency (one of them liked MIV by problem and by intervention frequency). They commented, “This is a great way of easily determining where the PHN has been spending most of her time with the client. It gives a general sense of what’s going on with the client and which problems the PHN/client are focusing on”, and “it was easier to visualize major problems.” Two PHNs (18%) had no preference. PHNs agreed that the interactivity functionality of the MIV was important (average rating = 1.82 [1 = strongly agree, 2 = agree], S.D = 84) and useful (average rating = 1.82, S.D. = .92). Comments included the following: five PHNs (45%) stated the usefulness of the MIV by mentioning “[I] Really like the hover tool”, and “I like the interactive feature.” “Availability to sort [to know] what is working for the client and what interventions are most effective”. Nearly all (90%) of PHNs identified the importance and usefulness of the interactivity functionality to show intervention details. The interactivity may help PHNs more easily understand and interpret the high dimensional information presented in the matrix that was noted by
7
six PHNs (55%), who commented that the MIV had too much information in a small space; and in particular, that the target list on across the top of the visualization was overwhelming. PHNs gave numerous suggestions for improving the MIV, specifically in regard to limiting the number of Target terms to those in the client data (as with Problems), adding a date range, and providing the link to the original chart. Based on the PHNs’ comments and suggestions, a revised MIV prototype should include the following: a view that depicts all intervention-related data in a single screen including narrative, incorporating visit dates, and linking to original charting. 5.3. Comparison across visualizations Overall, the PHN scores across all aspects of each visualization were very similar. The average rating for all questions related to the TOV was 1.93 [1 = strongly agree, 2 = agree], (S.D. = .48) and for all questions related to the MIV was 1.98 (S.D. = .83). Thus, in general there is considerable support to pursue use of such visualizations to summarize and present clinical data in real time for nursing care planning. This conclusion is supported by PHN comments: “Any time you delete an extra step in obtaining client information, it makes follow up much more easy.”, “In research or practice, it is helpful to know and understand the customized details that can’t be provided elsewhere in the graph. The visualization of the data becomes more meaningful in my opinion. It brings the data to life, so to speak”. 5.4. Limitations and suggestions for future research Findings of this study are limited by the small sample size of 11 PHNs; however the majority had extensive experience as a PHNs and Omaha System users. The expertise of the sample may increase confidence in the findings. Nevertheless, future research should include revised versions of TOV and MIV that should be tested with a larger PHN sample. Furthermore, the static presentation of visualizations in a presentation was necessary due to technology limitations. Future research should provide actual interactive visualizations in an electronic platform to assess PHN perceptions of actual interactive functionality. There is potential that visualizations may have a positive impact on PHN workflow if they reduce time reviewing data in the EHR. Future research should evaluate the impact of such visualizations on PHN workflow and efficiency. Given the promising findings about these visualizations for information synthesis and consumption in EHRs, future research should also give attention to enhancing EHR capacity to display information and display data using programming available in the public domain such as D3, which can automatically generate TOV and MIV for individual patients. In addition, this research suggests the potential for using visualization with data from clinical data repositories, and thus there is a need for developing technologies to support such efforts. Furthermore, as the use of visualization increases in clinical EHRs, it will be important to evaluate and refine tools for assessing clinician perceptions of visualizations such as the survey used in this study. This study is potentially important in four ways. First, the research produced prototypes of individual patient-focused visualization, rather than a summarized visualization merging all patients’ data. Data visualization for individual patients is crucial to practice because it enables a PHN to see and understand an individual patient’s progress over time, which leads to better decision making for individual follow-up care plans. Second, the prototype visualizations represented all of an individual clients’ health problems, nursing interventions, and outcomes—all key factors that inform the nursing process. Third, the use of the Omaha System in practice was well suited for development of intelligent datadriven clinical decision support using visualization techniques. As
Please cite this article in press as: S. Lee, et al., Public health nurse perceptions of Omaha System data visualization, Int. J. Med. Inform. (2015), http://dx.doi.org/10.1016/j.ijmedinf.2015.06.010
G Model
ARTICLE IN PRESS
IJB-3215; No. of Pages 9
S. Lee et al. / International Journal of Medical Informatics xxx (2015) xxx–xxx
8
Summary points What was already known on the topic? • EHRs provide clinicians with the advantages for storage, deployment, and retrieval of large amounts of patient carerelated data. However, EHRs do not usually provide an efficient view of the data accumulated in a linear manner. It is very time-consuming for clinicians to extract meaningful information from the lengthy records. • Standardized nursing terminology used in EHRs provides the benefits of efficient recording and better communication among clinicians. However, it is still time-consuming to review the data typed using standardized terms to grasp a patient’ progress and monitor patient care. What this study added to our knowledge? • Nursing data including text and numbers could be visualized for improved interpretation of patient data and better patient care monitoring and decision making. It increases the benefits of the standardized Omaha System terminology for the use in nursing practice. • There was considerable support by public health nurses to pursue use of visualizations to summarize and present clinical data at the point of care for patient care monitoring and planning. • This study indicated the potential for using visualization techniques within EHRs to summarize nurse-generated patient data. It allows visualized data to be easily integrated into nurses’ workflow and enables nurses to review the visualizations in real time at the point care.
an extension of these three key points, the use of standardized nursing language has potential to improve communication among PHNs and other health care providers and increased visibility of nursing practice [26]. Thus visualizations such as TOV and MIV may maximize the benefits of using Omaha System data by expanding the capacity of PHNs to see results and inform future care decisions. 6. Conclusion Clinical data visualizations were developed using PHNgenerated Omaha System data. This study examined PHN perceptions of the visualizations, and demonstrated the potential for using visualization techniques within EHRs to summarize patient data for clinicians. Further research is needed to improve and refine these visualizations and assess the potential to incorporate visualizations within clinical EHRs for improved patient-centered decision-making by PHNs. Funding This study was supported by the research funding of University of Missouri-St. Louis. Author contributions Seonah Lee: Developed the concept and design of the study; developed the initial visualization; conducted the analysis and interpretation of data; wrote and revised the paper from first to final for important intellectual content; gave the final approval of the version to be submitted. Era Kim: Conducted data cleaning; developed and refined the visualization; wrote the content about visualizations in the sections of Methods.
Karen A. Monsen: Assisted access to Omaha System data; assisted nurse survey including recruitment of public health nurses; revised the paper for improvement.
Conflicts of interest None of authors has no conflict of interest.
References [1] D.A. Keim, F. Mansmann, J. Schneidewind, J. Thomas, H. Ziegler, Visual analytics: scope and challenges, in: S.J. Simoff, M.H. Bohlen, A. Mazeika (Eds.), Visual Data Mining: Theory, Techniques and Tools for Visual Analytics, LNCS 4404, Springer-Verlag, Berlin Heidelberg, Germany, 2008, pp. 76–90, Available at http://link.springer.com/chapte/10.1007%2F978-3-540-71080-6 6#page-1 [2] K.S. Martin, The Omaha System: A key to Practice, Documentation, and Information Management, 2nd ed., Health Connections Press, Omaha, NE, 2005. [3] S.H. Koch, N. Staggers, C. Weir, J. Agutter, D. Liu, D.R. Westenskow, Integrated information displays for ICU PHNs: field observations, display design, and display evaluation, in: Proceedings from the 54th Annual Meeting of the Human Factors and Ergonomics Society, San Francisco, California, September 27–October 1, 2010, Available at http://igva2012wikispaces.asu.edu/file/ view/Integrated+Information+Displays+for+ICU+PHNs.pdf/315703330/ Integrated+Information+Displays+for+ICU+PHNs.pdf [4] W. Blair, B. Smith, Nursing documentation: frameworks and barriers, Contemp. PHN 41 (2012) 160–168. [5] B. Shneiderman, C. Plaisant, B.W. Hesse, Improving health and healthcare with interactive visualization methods, 2013. Avaialble at http://hcil2.cs.umd. edu/trs/2013-01/2013-01.pdf [6] A.A.T. Bui, D.R. Aberle, H. Kangarloo, Timeline: visualizing integrated patient records, IEEE T. Inf. Technol. Biomed. 11 (2007) 462–473, http://dx.doi.org/10. 1109/TITB.2006.884365 [7] M. Ward, Overview of data visualization, Available at http://web.cs.wpi.edu/ ?matt/courses/cs563/talks/datavis.html [8] L. Chittaro, Visualization of patient data at different temporal granularities on mobile devices, in: Proceedings from AVI ’06: the Working Conference on Advanced Visual Interfaces, Venezia, Italy, May 23–26, 2006, Available at http://dl.acm.org/citation.cfm?id=1133364 [9] O. Farri, A. Rahman, K.A. Monsen, R. Zhang, S.V. Pakhomov, D.S. Pieczkiewicz, S.M. Speedie, G.B. Melton, Impact of a prototype visualization tool for new information in EHR clinical documents, Appl. Clin. Inf. 3 (2012) 404–418, http://dx.doi.org/10.4338/ACI-2012-05-RA-0017 [10] S. Fang, M. Lwin, P. Ebright, Visualization of unstructured text sequences of nursing narratives, in: Proceedings from SAC’06: the 2006 ACM Symposium on Applied Computing, Dijon, France, April 23–27, 2006, Available at http://dl. acm.org/citation.cfm?id=1141331 [11] P. Hanrahan, C. Stolte, J. Mackinlay, Tableau software selecting a visual analytics application, 2009 Available at http://mkt.tableausoftware.com/files/ selecting-visual-analytics-app.pdf [12] L. Mamykina, S. Goose, D. Hedqvist, D.V. Beard, Careview: analyzing nursing narratives for temporal trends, in: Proceedings from CHI EA ’04: Extended Abstracts of Conference on Human Factors in Computing Systems, Vienna, Austria, April 24–29, 2004, Available at http://dl.acm.org/citation. cfm?doid=985921.986010 [13] E. Kim, K.A. Monsen, D.S. Pieczkiewicz, Visualization of Omaha System data enables data-driven analysis of Outcomes, American Medical Informatics Association, 2013. [14] T.D. Wang, C. Plaisant, A.J. Quinn, R. Stanchak, B. Shneiderman, S. Murphy, Aligning temporal data by sentinel events: discovering patterns in electronic health records, in: Proceedings from CHI ’08: the SIGCHI Conference on Human Factors in Computing Systems, Florence, Italy, April 5–10, 2008, Available at https://www.cs.umd.edu/∼ben/papers/Wang2008Aligning.pdf [15] C. Plaisant, B. Milash, A. Rose, S. Widoff, B. Shneiderman, Lifelines: visualizing personal histories, in: Proceedings from CHI’96: the SIGCHI Conference on Human Factors in Computing Systems, Vancouver, BC Canada, April 13–18, 1996, Available at http://www.cs.umd.edu/?ben/papers/ Plaisant1996LifeLines.pdf [16] C. Hallett, Multi-modal presentation of medical histories, in: Proceedings from IUI ‘08: the 13th International Conference on Intelligent User Interfaces, Canary Islands, Spain, January 13–16, 2008, Available at http://www.sussex. ac.uk/Users/drs22/projects/CLEF/publications/hallett-IUI.pdf [17] W. Aigner, S. Miksch, Carevis: integrated visualization of computerized protocols and temporal patient data, Artif. Intell. Med. 37 (2006) 203–218. [18] American PHNs Association, ANA. recognized terminologies and data element sets, 2006 May 11. Available at http://nursingworld.org/npii/terminologies. htm [19] The Omaha System. Available at http://www.omahasystem.org/overview. html [20] Eclipse Available at https://www.eclipse.org/
Please cite this article in press as: S. Lee, et al., Public health nurse perceptions of Omaha System data visualization, Int. J. Med. Inform. (2015), http://dx.doi.org/10.1016/j.ijmedinf.2015.06.010
G Model IJB-3215; No. of Pages 9
ARTICLE IN PRESS S. Lee et al. / International Journal of Medical Informatics xxx (2015) xxx–xxx
[21] [22] [23] [24] [25]
Apache Available at http://htt.apache.org/ MySQL. Available at http://www.mysql.com/ JSON, Available at http://www.json.org/ D3js, Available at http://d3js.org/ L. Wilkinson, M. Friendly, The History of the Cluster Heat Map, The American Statistician, 2015.
[26] M.A. Rutherford, Standardized nursing language: what does it mean for nursing practice? Online J. Issues Nurs. 13 (2008), Available at http://www. nursingworld.org/MainMenuCategories/ThePracticeofProfessionalNursing/ Health-IT/StandardizedNursingLanguage.html
Please cite this article in press as: S. Lee, et al., Public health nurse perceptions of Omaha System data visualization, Int. J. Med. Inform. (2015), http://dx.doi.org/10.1016/j.ijmedinf.2015.06.010
9
