Characterization of a Handoff Documentation Tool Through Usage Log Data Silis Y. Jiang1, Alexandrea Murphy1 MPH, David Vawdrey1 PhD, R. Stanley Hum2 MD, Lena Mamykina1 PhD 1 Department of Biomedical Informatics, Columbia University 2 Department of Pediatrics, Columbia University Abstract Handoffs are a critical component of coordinated patient care; however, poor handoffs have been associated with near misses and adverse events. To address this, national agencies have recommended standardizing handoffs, for example through the use of handoff documentation tools. Recent research suggests that handoff tools, typically designed for physicians, are often used by non-physician providers as information sources. In this study, we investigated patterns of edits of an electronic handoff tool in a large teaching hospital through examination of its usage log data. Qualitative interviews with clinicians were used to triangulate log data findings. The analysis showed that despite its primary focus on facilitating transitions of care, information in the handoff documentation tool was updated throughout the day. Interviews with residents confirmed that they purposefully updated information to make it available for other members of their patient care teams. This further reiterates the view of electronic handoff tools as facilitators of team communication and coordination. However, the study also showed considerable variability in the frequency of updates between different units and across different patients. Further research is required to understand what factors drive such diversity in the use of electronic handoff tool and whether this diversity can be used to make inferences about patients’ conditions. Introduction Handoffs are a critical aspect of providing continuous care for patients in inpatient services1. Due to recent changes towards more restrictive resident work hours, handoffs have become commonplace in the hospital2. However, poor handoffs have been associated with near misses and adverse events3,4. Institutions such as the Joint Commission and the Accreditation Counsel of Graduate Medical Education (ACGME) have made recommendations towards improving handoffs5 and require hospitals to ensure resident competency in handoffs6. These identified limitations of the current handoff practices suggest the need for new tools for facilitating handoff. With few nationally published or commercially available handoff tools, many healthcare institutions developed and optimized handoff tools to their own needs and workflows7. While many of these tools are paper based, there is a growing trend towards developing electronic and often EHR-integrated handoff documentation systems8-10. There is emerging evidence that interventions that include handoff documentation tools reduce errors in patient care and improve the quality and structure of handoffs11. Previous research on handoff tools has primarily focused on the impact of these tools on handoff standardization or patient care12. Furthermore, these previous studies usually adopted a singular user perspective, typically focusing on physicians12,13. At the same time, an emerging theme from recent handoff tool research suggests that electronic resident handoff documentation tools are adopted beyond their original user group. For example, Vawdrey et al. showed that non-physician clinicians (nurses, pharmacists, social workers, etc.) accounted for 60% of active users in a resident handoff documentation tool7. Similarly, Schuster et al. reported that the physician handoff tool was integrated into the daily workflow of non-physicians14. However, these studies have focused on handoff tools from a user viewing or information retrieval perspective. The process of creating or updating handoff documents is much less understood. This study focuses on investigating patterns of use of a resident oriented handoff documentation tool, with a particular attention to creation of the handoff document, including edits and updates. The study was conducted at Columbia University Medical Center, where clinicians use a handoff documentation tool known as Handoff Tab. The tool’s interface includes a series of labeled free-text boxes that can be directly edited by authorized users. Additionally, clinicians have the option to print the user-contributed data along with recent structured data, such as lab values and medication list. The tool is primarily edited by residents but available to all clinicians to view. This study sought to identify: 1) whether there are any systematic temporal patterns in the edits to the different fields of

749

the Handoff Tab; 2) whether there are any differences in the frequency and patterns of updates between different clinical units; and 3) whether there were any differences in how frequently Handoff Tab was updates across different patients. Materials and Methods Handoff Tab Handoff Tab is a custom-designed module included in the commercial EHR system (Sunrise Eclypsis). It has 9 freetext boxes: Active Issues, Consult Notes, Contact Info, Coverage To Do List, Discharge Planning, Hospital Course, Notes and Comments, Patient Summary, and Primary To Do List. The tool also provided the functionality to produce a printable report that included structured data, such as labs and medication. Furthermore, users could choose to print a cover sheet that summarized select patients. Dataset To analyze patterns in Handoff Tab documentation, we used the document-editing event logs for all of October 2013 at the Columbia University Medical Center campuses of New York Presbyterian Hospital. The date and time of each edit event, which free-text box was edited, patient medical record number, patient bed location, clinician user ID, and care provider role were extracted from the audit log. All analyses on the unit level were normalized based on the number of unique patients in that unit. It is important to note, however, that not all patients in a unit may have Handoff Tab documents associated with them. Therefore, there may actually be more patients in the unit at a given time than can be estimated by the dataset. Analyses on the patient level were normalized based on the number of handoff days. The number of handoff days was calculated based on the first and last day a clinician edited the Handoff Tab document of each patient. Additionally, four interviews were conducted with the medical staff in the pediatric intensive care unit at Columbia University Medical Center (CUMC). Three residents and one attending were interviewed. The interviewees were selected by convenience. The three residents were in their final year of residency, while the attending physician had previously used a handoff documentation tool during the final year of critical care fellowship training. Quantitative Analysis To identify editing patterns throughout the workday, all edit events were aggregated into a single 24-hour period and visualized, regardless of the day of the month. To understand the influence of unit practices on editing patterns, the dataset was divided by units and then clustered. First, the total number of edits and the number of edits per Handoff Tab Field was calculated for each unit. These values were then normalized based on the patient count for individual units. Euclidean distance was used to measure the difference of each unit based on the edit frequency of individual fields. Hierarchical clustering based on the complete method was then used to group the units. A k-value was selected based on the outcome. This k-value was then applied to a k-means clustering algorithm (Hartigan and Wong method) of each analysis dataset. To understand the influence of patients on editing patterns, the dataset was split by unique patients and then clustered. A similar method similar to the unit cluster analysis was used. Briefly, edits per Handoff Tab field was calculated on a patient basis and then normalized based on the handoff days. A distance matrix was used to calculate the Euclidean distance between each patient in the dataset. Hierarchical clustering based on the complete method was used to group the patients based on that matrix. A k-value was extracted from the hierarchical clustering analysis and applied to the k-means clustering algorithm (Hartigan and Wong method). The statistical tool R was used for cluster data analysis and visualization. All clustering related methods were performed using the stats package, and all visualizations were created using the ggplot2 package. All statistics are presented as means along with the standard error of the mean. Qualitative Analysis

750

To supplement the quantitative data, interviews with physicians were conducted to provide insight and reasoning for unit differences in documentation behaviors. Four interviews were conducted with the medical staff in the pediatric intensive care unit at Columbia University Medical Center (CUMC). Three residents and one attending were interviewed. The interviewees were selected by convenience. The clinicians were consented prior to the start of the interview. The interviews were recorded and transcribed for analysis. Transcripts were reviewed for common themes. This study was approved by the Columbia University Institutional Review Board. Results Temporal Distribution of Handoff Tab Updates On average, each unit updated the Handoff Tab 13.67 times per patient during the study period with a standard error of the mean (SEM) score of 0.21 edits. The patients in this dataset had an average stay of 5.12 days (SEM = 0.007 days) and an average of 22.42 edits to their handoff document during their stay (SEM = 0.003 edits). Table 1. Dataset Characteristics Characteristic Total Number of Edit Entries During Study Period Total Number of Units Total Number of Patients

Sample size 145,872 132 6515

After aggregating the dataset into a single 24-hour time frame, the data showed that Handoff Tab was updated throughout the day (Figure 1). However, not all fields were utilized equally. In general, the field “Consult Notes” were updated the least and the “Primary To Do List” field was updated the most. Updates were most frequently made during two timeframes corresponding to the periods right before handoff. However, the data also shows that Handoff Tab was updated throughout the workday.

Figure 1. Hourly distributions of edit frequencies per Handoff Tab field

751

During interviews clinicians expressed awareness of other clinicians utilizing the tool and their notes as information sources. For this reason, residents reported updating the Handoff Tab up to three or four times per shift to make sure the information remain up to date. “If we have down time and I have time to update during the day, I do that as well just so it is up to date for any one who refers to it and sometimes I know our nurses refer to it and the attendings refer to it.” – Resident 1 “Yeah. I’m usually – I usually keep track of things on my paper printout but then probably like three or four times a day I would do that [updating Handoff Tab throughout the day].” – Resident 3 In these examples, the residents indicate a conscious effort to regularly maintain the information found in Handoff Tab. The desire to maintain an up-to-date handoff document provides one explanation for continuous updates throughout the day. Unit Cluster Analysis To understand the update pattern of the different units, a cluster analysis was performed based on individual unit update frequencies. Hierarchical clustering was used to identify possible cluster numbers for k-means clustering. In this case, a k-value of 5 was selected based on the hierarchical clustering (Figure 2A and 2B). The units in each cluster are differentiated by two factors: update frequency magnitude and field update distribution. Cluster 1 is characterized by having well above median update frequency magnitude. Cluster 1 has a unique distribution in its frequency peak for the “Notes and Comments” field, but the distribution also shows the “Patient Summary” field being edited more frequently than the “Hospital Course” field. Cluster 2 has well below median update frequency magnitude. Units in this cluster made less than 0.5 updates per patient in all of the fields. Furthermore, the distribution of updates per patient is relatively flat. Cluster 3 has the median frequency magnitude. For this cluster, the distribution of edits is appears to be roughly equal between the “Hospital Course”, “Notes and Comments”, and “Primary To Do List” fields. Cluster 4 has below median magnitude and roughly equal distributions between the “Coverage To Do List”, “Hospital Course”, and “Primary To Do List” fields. Cluster 5 has above median update frequency magnitude. The update distribution for this cluster is primarily focused on the “Notes and Comments” section and higher update frequency for “”Hospital Course” compared to “Patient Summary” (the opposite trend from cluster 1).

Figure 2A and 2B. Mean and standard error of the mean (SEM) of field frequencies per unit clustering. 1 = Active Issues, 2 = Consult Notes, 3 = Contact Info, 4 = Coverage To Do List, 5 = Discharge Planning, 6 = Hospital Course, 7 = Notes and Comments, 8 = Patient Summary, 9 = Primary To Do List

752

To test whether clusters could characterize unit properties, the patient count of each cluster was compared. Analysis comparing patient counts in each cluster showed no difference between clusters. Some units that had many patients during the study period update the handoff tab very frequently for each patient, while others with similar patient counts did not. Conversely, some units that did not have very many patients updated the Handoff Tab very frequently. Therefore, considering only the patient count was not predictive of that unit’s cluster membership. Patient Cluster Analysis To identify whether different patients had differences in the update pattern of their associated handoff document, clustering was used to group patients. A hierarchical cluster analysis showed that a k-value of 8 was optimal for a kmeans cluster analysis. Based on Figure 3, each cluster of patients has a different distribution of update frequencies per Handoff Tab field. Unlike the unit cluster analysis, each cluster of the patient cluster analysis is differentiated by the distribution of updates between the Handoff Tab fields. For the majority of the clusters (clusters 1 to 6), the magnitude of update frequencies is roughly equal. The distinctive feature of patients in cluster 1 is the low update frequency in the “Consult Notes” field compared to other fields. Cluster 2 patients have relatively more “Consult Notes” and “Discharge Planning” edits compared to the other fields. Cluster 3 patients have a roughly equal number of edits in each field. While cluster 4 patients are similar to those in cluster 3, the edit frequency to “Patient Summary” and “Primary To Do List” appear to be slightly more relative to other fields in cluster 4. Cluster 5 patients have a distinctly high edit frequency in the “Active Issues” field. Cluster 6 patients have relative edit frequency troughs in three fields. Patients in cluster 7 have very distinct peaks in edit frequency in the “Active Issues” and “Contact Information” fields. The final cluster is characterized by having a very low edit frequency in every field, with the exception of the “Patient Summary” field.

Figure 3. Mean and standard error of the mean (SEM) of field frequencies per patient cluster. 1 = Active Issues, 2 = Consult Notes, 3 = Contact Info, 4 = Coverage To Do List, 5 = Discharge Planning, 6 = Hospital Course, 7 = Notes and Comments, 8 = Patient Summary, 9 = Primary To Do List Following cluster analysis, a comparison of the handoff days for each cluster was made. Based on Table 2, the mean handoff days for each cluster is similar. This suggests that handoff days is not a good indicator for predicting cluster membership. However, it may be possible for more complex patient characteristics, such as patient severity score, to be indicative of being a member of a certain cluster.

753

Table 2. Means and deviations of patient handoff days (Days) by patient clusters Stay Standard Cluster Patient Count Mean Stays Deviation 1 821 4.790499 6.513398 2 2025 5.568395 6.99678 3 589 5.198642 6.517084 4 697 4.731707 6.043999 5 292 6.130137 8.733759 6 927 4.738943 6.235724 7 886 4.715576 6.430535 8 278 5.309353 6.192868

Stay Standard Error of the Mean 0.007933 0.003455 0.011065 0.008671 0.02991 0.006727 0.007258 0.022277

Discussion There is a considerable body of research suggesting that communication is a key component of teamwork and team coordination15-18. Particularly in inpatient settings, where care is provided by patient care teams, communication, both verbal and non-verbal, is a crucial part of a team’s ability to coordinate their efforts15. Due to common delays in written documentation, clinicians continue to rely on verbal communication to stay up to date19. At the same time, there is a growing awareness that verbal communication can be disruptive for work, requires both parties to be available simultaneously, and leaves no record20. Computer-mediated communication among members of patient care team can provide an alternative or complementary channel, less disruptive to the flow of clinical work. Yet, the patterns of communication through EHRs are not well understood21. Recently, handoff documentation tools were shown to be useful as a source of information for many user groups, beyond physicians14. The results of our study suggest that residents, who are primarily responsible for editing information in the Handoff Tab, purposefully use it as an improvised team communication tool, rather than strictly resident handoff tool. The analysis of the temporal patterns of Handoff Tool updates show that residents enter new information throughout the day, not only in preparation for transitions of care. Our interviews with residents further confirmed that often they updated the information to make it available to other members of their teams. Moreover, the residents indicated that Handoff Tab can be used as a collaborative writing platform, with other members of their teams suggesting updates and changes, even if they do not edit the fields directly. At the same time, the results of our cluster analysis suggested that Handoff Tool is used differently by different departments, and even for different patients. Previous evaluations of handoff tools that included multiple departments were generally taken from a homogenous perspective12,13. In contrast to these studies, our analysis found substantial differences in the frequency of updates to Handoff Tool between different departments. Some of these differences are easy to account for. For example, departments with more critically ill patients, such as Intensive Care Units, had a higher frequency of updates than many other hospital units. Yet other differences are not as straightforward. For example, we found that neither the number of patients in the unit nor the average lengths of stay were indicative of the frequency of Handoff Tool updates in that unit. Moreover, interviews with residents suggested that different units developed varying practices in regards to what information is captured in different text boxes. Further research is needed to further explicate local practices in handoff tool use and the different factors that lead to the diversity in its adoption. Finally, the analysis of differences in frequencies between different patients suggests that the use of Handoff Tab may reflect differences in patients’ conditions and care. For example, patients with complex conditions who require ongoing care from different clinical specialties may have more updates in their Consult Notes and Contact Info fields. In contrast, patients with rapidly changing conditions might have more updates in the Active Issues list. Finally, patients who are nearing their discharge from the hospital may have more updates in the Discharge Planning area. While our dataset did not allow us to examine these relationships in greater detail, our interviews with the residents confirmed these intuitions. Recently, nursing documentation patterns were shown to be predictive of patients who experience a cardiac arrest or death22. Our study further suggests that there is great utility in understanding the documentation patterns of individual units and patients. For instance deviations in established handoff update frequency patterns for patients in a cardiology ward could prove to be indicative of future complications or deteriorations, such as cardiac arrest.

754

Limitations A key limitation of the dataset is that it only provides the fields clinicians edited but not the information that was input. Since Handoff Tab consists of free-text fields, clinicians are free to write information unrelated to that field in the text box. This limitation partially hinders the ability to draw conclusions about what the cluster membership means for either units or patients. In other words, while cluster 1 in the unit clustering analysis suggests that the “Notes and Comments” field is edited frequently, it is impossible to determine what content the edits were focused on. Another limitation regards the correlation between handoff days and length of stay for patients in this dataset. Patients may have been admitted to the hospital before a Handoff Tab document was created or discharged to a unit that did not incorporate Handoff Tab into the unit workflow. Therefore the length of stay may be longer than estimated by handoff days. Furthermore approximately 1.5% of the patient population had length of stays that equaled or exceeded the study period. Additionally, this dataset provided little information characterizing either the unit or the patient. It was impossible to determine the service (cardiology, nephrology, surgical intensive care, etc.) that patient was being treated by. The dataset contained little information about patient severity or any patient outcomes. This limited the abilities to draw conclusions about the patients in each cluster. Lastly, it should be recognized that this dataset represents a small time slice within one academic medical center. With handoff workflows constantly changing, it must be acknowledged these findings may not capture all variations currently in clinical practice. Conclusion Handoff Tab was frequently edited throughout the day. This suggests that handoff documentation tools may be a valuable source of up to date information. Furthermore, clusters analysis indicates that different update patterns exist based on units and patients. Acknowledgements The research described was supported by the T15LM007079 grant from the National Library of Medicine.

References 1.

Ahmed J, Mehmood S, Rehman S, Ilyas C, Khan LUR. Impact of a structured template and staff training on compliance and quality of clinical handover. Int J Surg. 2012;10(9):571–4.

2.

Choma NN, Vasilevskis EE, Sponsler KC, Hathaway J, Kripalani S. Effect of the ACGME 16-Hour Rule on Efficiency and Quality of Care: Duty Hours 2.0. JAMA Intern Med. 2013 Apr 1;:1–2.

3.

Horwitz LI, Meredith T, Schuur JD, Shah NR, Kulkarni RG, Jenq GY. Dropping the baton: a qualitative analysis of failures during the transition from emergency department to inpatient care. Annals of Emergency Medicine. 2009 Jun;53(6):701–4.

4.

Cohen MD, Hilligoss PB. The published literature on handoffs in hospitals: deficiencies identified in an extensive review. Qual Saf Health Care. 2010;19(6):493–7.

5.

Agency for Healthcare Research and Quality (AHRQ). AHRQ Patient Safety Network - Handoffs and Signouts [Internet]. psnet.ahrq.gov. [cited 2014 Mar 13]. Available from: http://psnet.ahrq.gov/primer.aspx?primerID=9

6.

Education ACFGM. Graduate Medical Education > Duty Hours [Internet]. www.acgme.org. [cited 2014 Mar 13]. Available from: https://www.acgme.org/acgmeweb/tabid/271/GraduateMedicalEducation/DutyHours.aspx

7.

Vawdrey DK, Stein DM, Fred MR, Bostwick SB, Stetson PD. Implementation of a computerized patient handoff application. AMIA Annu Symp Proc. 2013;2013:1395–400.

755

8.

Van Eaton EG, Horvath KD, Lober WB, Rossini AJ, Pellegrini CA. A randomized, controlled trial evaluating the impact of a computerized rounding and sign-out system on continuity of care and resident work hours. Journal of the American College of Surgeons. 2005 Apr;200(4):538–45.

9.

Bernstein JA, Imler DL, Sharek P, Longhurst CA. Improved physician work flow after integrating sign-out notes into the electronic medical record. Jt Comm J Qual Patient Saf. 2010 Feb 1;36(2):72–8.

10.

Abraham J, Kannampallil TG, Patel VL. Bridging gaps in handoffs: A continuity of care based approach. Journal of Biomedical Informatics. Elsevier Inc; 2012 Apr 1;45(2):240–54.

11.

Horwitz LI. Does improving handoffs reduce medical error rates? JAMA. 2013 Dec 4;310(21):2255–6.

12.

Riesenberg LA, Leitzsch J, Massucci JL, Jaeger J, Rosenfeld JC, Patow C, et al. Residents“ and attending physicians” handoffs: a systematic review of the literature. Acad Med. 2009 Dec;84(12):1775–87.

13.

Riesenberg LA, Leitzsch J, Cunningham JM. Nursing handoffs: a systematic review of the literature. Am J Nurs. 2010 Apr;110(4):24–34–quiz35–6.

14.

Schuster KM, Jenq GY, Thung SF, Hersh DC, Nunes J, Silverman DG, et al. Electronic handoff instruments: a truly multidisciplinary tool? Journal of the American Medical Informatics Association. 2014 Feb 19.

15.

Sarcevic A, Marsic I, Burd RS. Teamwork Errors in Trauma Resuscitation. ACM Trans Comput-Hum Interact. 19(2):13:1–13:30.

16.

Manser T. Teamwork and patient safety in dynamic domains of healthcare: a review of the literature. Acta Anaesthesiol Scand. 53(2):143–51.

17.

Leonard M. The human factor: the critical importance of effective teamwork and communication in providing safe care. Qual Saf Health Care. 2004 Oct 1;13(suppl_1):i85–i90.

18.

Thomas EJ, Sexton JB, Lasky RE, Helmreich RL, Crandell DS, Tyson J. Teamwork and quality during neonatal care in the delivery room. J Perinatol. 2006;26(3):163–9.

19.

Collins SA, Bakken S, Vawdrey DK, Coiera E, Currie L. Clinician preferences for verbal communication compared to EHR documentation in the ICU. Appl Clin Inform. 2011;2(2):190–201.

20.

Coiera E. The science of interruption. BMJ Quality & Safety. 2012 May;21(5):357–60.

21.

Walsh C, Siegler EL, Cheston E, O'Donnell H, Collins S, Stein D, et al. Provider-to-provider electronic communication in the era of meaningful use: A review of the evidence. Journal of Hospital Medicine. 2013;8(10):589–97.

22.

Collins SA, Cato K, Albers D, Scott K, Stetson PD, Bakken S, et al. Relationship Between Nursing Documentation and Patients' Mortality. Am J Crit Care. 2013 Jul 1;22(4):306–13.

756

Characterization of a handoff documentation tool through usage log data.

Handoffs are a critical component of coordinated patient care; however, poor handoffs have been associated with near misses and adverse events. To add...
918KB Sizes 2 Downloads 18 Views