Accepted Manuscript Title: Heuristic Evaluation of a Telehealth System from the Danish TeleCare North Trial Author: Pernille Heyckendorff Lilholt Morten Hasselstrøm Jensen Ole K. Hejlesen PII: DOI: Reference:

S1386-5056(15)00015-5 http://dx.doi.org/doi:10.1016/j.ijmedinf.2015.01.012 IJB 3163

To appear in:

International Journal of Medical Informatics

Received date: Revised date: Accepted date:

13-2-2014 15-1-2015 17-1-2015

Please cite this article as: P.H. Lilholt, M.H. Jensen, O.K. Hejlesen, Heuristic Evaluation of a Telehealth System from the Danish TeleCare North Trial, International Journal of Medical Informatics (2015), http://dx.doi.org/10.1016/j.ijmedinf.2015.01.012 This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Heuristic Evaluation of a Telehealth System from the Danish TeleCare North Trial

ip t

Pernille Heyckendorff Lilholta,*, Morten Hasselstrøm Jensena, Ole K. Hejlesena,b

b

cr

Department of Health Science and Technology, Aalborg University, Aalborg 9220, Denmark Department of Computer Science, University of Tromsø, Tromsø 9019, Norway

us

a

*Corresponding author:

an

Pernille Heyckendorff Lilholt, PhD fellow Department of Health Science and Technology, Aalborg University Address: Fredrik Bajers Vej 7C, C1-223

M

PO BOX 9220, Aalborg East, Denmark

te

Research Highlights

d

Email address / Tel.: [email protected] / (+45) 26914860

A heuristic evaluation helps identify problems in the user interface design



Every usability problem can be linked to a generally established usability principle



The method has identified factors that are crucial for the implementation of Telekit



Insight into COPD patients’ use of Telekit is recommended

Ac ce p



Keywords: Public health informatics; user computer interface; usability; handheld computer; telemedicine; COPD; home monitoring.

1

Page 1 of 20

ip t cr

Abstract

us

Objective: The aim was to evaluate the usability of the design of the telehealth system, named Telekit, developed for the Danish TeleCare North Trial, early into the design process in order to assess potential problems and limitations which could hinder its successful implementation.

an

Methods: Five experts, including one who pilot-tested the Telekit system, individually evaluated its usability and its compliance with Jakob Nielsen’s ten usability heuristics for interaction design. Usability problems were categorised

M

according to Rolf Molich’s severity classification.

Results: The five experts identified a total of 152 problems in the Telekit system, each identifying 22-40 problems. 86 (57%) out of the 152 problems were identified only once. All heuristics were used, but the three most frequently used

d

were: “Match between system and the real world” (32%), “Consistency and standards” (13%) and “Aesthetic and

(43%).

te

minimalist design” (13%). The most widely used classifications were: “Improvement” (40%) and “Minor problem”

Ac ce p

Conclusion: Heuristic evaluation was an effective method for uncovering and identifying problems with the system. The consistent finding of particular usability problems confirms that the development of a telehealth system should pay particular attention to user aspects. The most serious problem was the inability of the system to inform users of how to perform measurements correctly and to “speak the users’ language”. The problems found in the heuristic evaluation have led to several significant changes in the telehealth system. We suggest that heuristic evaluation always be followed by user tests to evaluate the design of telehealth systems.

2

Page 2 of 20

ip t

1. Introduction The use of information and communication technology, like telehealth, has rapidly increased worldwide

cr

and has facilitated management of chronic diseases in the clinical and health delivery sectors [1]. However, a maximum of user friendliness is required to reap the full benefit from telehealth systems in

us

terms of clinical outcomes and patient satisfaction. User friendliness may be enhanced by paying close attention to users during the development of technologies even if this is time-consuming and resource

an

demanding [2].

Up until the present time, pilot studies have found that the use of telehealth care can be an effective

M

method to improve the quality of care, reduce comorbidity and rehospitalisation and thereby reduce overall healthcare-related costs. Telehealth appears ideally suited to enable real-time remote monitoring and management of patients from their own homes. Telehealth thereby has the potential to transform the

d

delivery of the care for the growing number of people with chronic obstructive pulmonary disease

te

(COPD). Telehealth can provide care to patients who live far away from hospitals and simultaneously guarantee continuity of care [3]. It is also known to improve patients’ knowledge and motivation to

Ac ce p

change behaviours that can help them manage their disease [4]. COPD poses increasing challenges and burdens on patients, providers, and healthcare systems. Despite well-established pharmacological therapy, patients are affected by severe symptoms that restrict their normal daily activities and affect their quality of life. The symptoms of COPD (breathlessness, tiredness, cough, and sputum production) indicate an abnormality in lung function. The number of exacerbations is a major problem because they induce frequent hospital admissions and have a negative impact on quality of life, prognosis, and costs. Denmark has made many advances within the telehealth care area; but in most cases, the fascinating visions have never passed the stage of promising, one-off pilot trials. As a response, a large-scale, clusterrandomised Danish project, TeleCare North has been established to develop a telehealth system (named 3

Page 3 of 20

Telekit) for all COPD patients in the North Denmark Region. All eleven North Jutland municipalities, the North Denmark Region, general practitioners, and Aalborg University are taking part in the project [5]. Approximately 1,225 patients are randomised to control- or intervention group. COPD patients will have

ip t

equipment at home that supports monitoring and treatment of their disease. There are two main objectives of the TeleCare North study. One objective is to evaluate whether the

cr

Telekit system, in addition to conventional treatment and care, leads to a significant decrease in the

mortality and an increase in health status measured with the Short Form-36 Health Survey. The second

us

objective is to examine whether the telehealth system is cost-effective in the way of care for COPD patients [6]. The study raises the hypothesis that telehealth will result in no significant changes in

an

mortality and a significantly better health status for the included patients. Secondary effectiveness outcomes will consist of changes in vital signs (blood pressure, pulse, oxygen saturation, and weight) and patient satisfaction within a 12-month period. The clinical impact of telehealth is expected to be

M

beneficial if pulse and blood pressure are lowered, and oxygen saturation and weight are raised. Furthermore, the hope is that telehealth can reduce the number of admissions and readmissions to

d

hospitals, and can reduce the number of outpatient visits. Potential savings could be spent on other areas

te

that may benefit the patients. The outcomes of the study will inform clinical and political decisionmaking as to whether the telehealth system should be implemented as a standard treatment for COPD

Ac ce p

patients in Denmark [5,7,8].

Usability is defined by the International Organization for Standardization (ISO 9241-11): ”The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use” [9].

Two recommended usability assessment methods can be selected to evaluate a telehealth solution like Telekit. Inspection methods are based on experts’ evaluations, while user testing methods are based on the users’ interaction with the solution [10]. In this study, the inspection method heuristic evaluation was used to evaluate the usability of the design. Using heuristic evaluation, the usability of the design of the telehealth system Telekit was assessed to identify factors critical to its implementation and operation [11]. The Danish usability expert Jakob Nielsen and the Danish software engineer Rolf Molich originally 4

Page 4 of 20

developed the method in 1990. It is a method for finding problems in a user interface design by inviting experts to evaluate the interface based on established usability principles. In several studies, the method has been used to support the choice of clinical information systems, medical devices, telemedicine

ip t

programs, and other information systems [12–17]. The aim of the present paper is to present the procedure for a heuristic evaluation of the Telekit system,

cr

and to present the results of its application to assess the user-friendliness of the Telekit system.

us

2. Methods and Materials 2.1. Telekit

an

Telekit is an internet-based monitoring and treatment system that consists of a tablet (Galaxy TAB 2, 10.1, Samsung Electronics, Seoul, South Korea), a fingertip pulse oximeter (Nonin, Onyx II% SpO2, A

M

and D Medical, Tokyo, Japan), a blood pressure monitor (Model UA-767, plus BT-C, Nonin Medical, Minnesota, USA), and a scale (Precision Health Scale, UC-321PBT-C, A & D Medical, Tokyo, Japan). The scale has not yet been implemented in the trial.

d

Telekit uses an open source platform, OpenTele, which collects health data from citizens at home via an

te

application. The system consists of a web-based and mobile healthcare platform. The web portal is the

Ac ce p

clinicians’ tool for interacting with the patient. Clinicians receive measurements, view, and analyse the patient’s health data. The application guides the patients through measurements using questionnaires. Initially, OpenTele is used in three out of five Danish regions and is expected to be rolled out nationally within a year or two. The platform is mainly used for chronic patients with COPD, diabetes, and heart failure [18]. Figure 1 below illustrates the Telekit system.

5

Page 5 of 20

ip t cr us

Figure 1: The telehealth system, Telekit consists of a Samsung Galaxy tablet, a fingertip pulse oximeter, a blood pressure monitor,

an

and a scale.

Telekit can measure four vital signs, which are transferred wirelessly to the municipalities or the

M

hospitals: blood pressure, pulse, blood oxygen saturation, and weight. A home care nurse educates the patients in how to use the Telekit system, including taking measurements, and answer questions about

d

their medical condition using the tablet. The patient will have a follow-up visit 3-4 weeks after the first

te

training session to check whether the patient uses the Telekit appropriately. Data are monitored by healthor community care personnel, and the personnel responds if data show deviations from the expected

Ac ce p

values [19].

The heuristic evaluation of the Telekit system version 1.5.0 was performed based on the Region North “COPD Questionnaire” (version 7). The COPD patients perform measurements and ask questions related to their disease through the “COPD Questionnaire”. 2.2. Participants

Telekit was pilot-tested first by a single expert and subsequently by four other experts. The five experts (three men, two women) were biomedical engineers and considered to be usability as well as interface experts [20]. They were recruited from the same workplace and worked as PhD students (n = 3), assistant professor (n = 1) or associate professor (n = 1). Data will be published anonymously. The debriefing took place as an open discussion among experts and investigators. The average age of the five experts was 29 years (min 25, max 36). All experts received verbal and written information about the purpose and the 6

Page 6 of 20

method of the study, and were also informed that they had to take into account that the Telekit system had been developed for COPD patients. COPD patients’ health-literacy level is reported to be low and they

Table I: Illustrates the experts’ characteristics.

ip t

might therefore have poor technology proficiency [21]. Table I below features the experts’ characteristics.

Expert 2

Expert 3

Expert 4

Sex

Male

Male

Male

Female

Female

Age

25

25

28

36

33

MSc in Engineering (Biomedical Engineering and Informatics)

MSc in Engineering (Biomedical Engineering and Informatics)

MSc in Engineering (Biomedical Engineering and Informatics)

PhD fellow

PhD fellow

PhD fellow

2.3. Study design and data collection

us

an

Profession

MSc in Engineering (Biomedical Engineering and Informatics), PhD Assistant professor

MSc in Engineering (Biomedical Engineering and Informatics), PhD Associate professor

M

Education

Expert 5

cr

Expert 1 (pilot)

Usability may be measured by a number of methods [10]. In the present study, the inspection method

d

called heuristic evaluation was chosen. This method is a discount usability testing method in which a

te

small group of experts evaluate (by means of general heuristics – rules of thumb) whether the interface of

Ac ce p

a system is in accordance with recognised usability principles [22]. The experts use their experience and knowledge from usability to evaluate a system under development (mock-ups) or an existing system [22]. A heuristic evaluation can hardly be performed by one individual only, as a single person will not be able to detect all usability problems in a particular interface. It is typically enough to have the system tested by five users to identify a system’s most important usability problems. The participation of five experts in a properly conducted usability test will usually ensure detection of approximately 80% of all usability problems. It simply is not possible to detect all problems super users may encounter [23,24]. The heuristic evaluation procedure was pilot-tested. The pilot caused us to chance the original procedure according to which each expert had to go through all elements of the system twice before describing the problems. Yet, the pilot showed that during the second round of testing, the tester found it difficult to remember the problems identified in the first round. In the revised procedure, there was only one round,

7

Page 7 of 20

and it was therefore decided that the tester should immediately write down any usability problems encountered. Before each evaluation, the heuristics and the severity classification of problems were briefly discussed,

ip t

to ensure that they were understandable. The experts evaluated the Telekit system individually in order to avoid bystander bias and to ensure an independent assessment. The experts evaluated the system through constructed scenarios and judged its compliance with Jakob Nielsen’s ten general usability principles for

cr

interaction design [25]. They continuously evaluated the various dialogue elements of the system and

severity scale [26]. Problem severity was graded as follows:

us

compared them with the heuristics. Usability problems were categorized according to Rolf Molich’s

an

1 = Improvement - which does not substantially disturb the user’s experience 2 = Minor problem – the user will be somewhat delayed (few minutes) 3 = Severe problem – the user will be much delayed (several minutes)

M

4 = Critical problem – the user cannot carry out the task 5 = Malfunction – the system does not work properly

d

One of Molich’s categories “Good comments” was not included: as positive comments were not

te

considered as usability problems. A table was made which contained the ten usability principles and columns with problems, severity, and solutions that experts could easily fill out. Some of the problems

Ac ce p

from the list were considered more important than others; and by using the severity scale, it would be possible to prioritize problems for a future redesign process. Each of the five experts used two hours for testing. A one-hour debriefing was held after the heuristic evaluation. This conversation was structured by the experts’ statements and the individual evaluations were compared. At the end of the debriefing, the experts were asked to name the three most positive aspects of the Telekit system and the three worst problems. Table II below lists the four steps of the heuristic evaluation procedure.

Table II: Test procedure.

Test procedure 1. Introduction and clarification of questions 2. Expert completes questionnaire about characteristics

8

Page 8 of 20

3. Heuristic evaluation and classification of problems through scenarios 4. Debriefing

2.4. Data analysis

ip t

For the evaluation, each expert ticked off the relevant columns and cells in a table featuring the ten

heuristics, described the problems encountered, and graded the severity of the problems. The problems

M

an

us

chart to compare the number of problems among experts and their severity.

cr

and notes from the evaluations were structured and analysed. The data were described in a simple bar

3. Results 3.1. Heuristic evaluation

d

The experts identified a total of 152 problems in the Telekit system of which 86 (57%) were unique

te

problems. Each expert identified 22-40 problems.

Ac ce p

Three problems (3.5%) were found by all experts. These problems were related to spelling errors, inconsistencies (spelling, contents, buttons) and doubts about how measurements were performed using the system. Three problems (3.5%) were detected by four experts. These problems were related to doubts about how to perform a “Chair Stand” test. Another problem was that the input field of the number of repetitions was not limited, which could result in human errors. Seven problems (8%) were identified by three experts, 31 problems (36%) by two and, finally, 42 problems (49%) were identified by a single expert. The problems identified by one to three experts were distributed throughout all ten heuristics. Table III below provides an overview of the experts' classification of the severity of the problems found. As the most experienced experts, experts No. 4 and 5 identified the largest number of problems. Experts No. 1 and 5 used all degrees of severity.

9

Page 9 of 20

Table III: Number of problems and their severity classification according to expert.

Expert 1

Expert 2

Expert 3

Expert 4

Expert 5

Total

Improvement

12

11

10

14

14

61

Minor problem

8

12

12

19

15

66

Severe problem

3

0

0

7

7

17

Critical problem

2

0

0

0

2

Malfunction

3

0

0

0

1

Total

28

23

22

40

cr

ip t

Classification

4

152

us

39

4

Table IV illustrates the relationship between the heuristics and severity. The most frequent classifications

an

were: “Improvement” (40%) with 61 of the problems and “Minor problem” (43%) with 66 of the problems. Seventeen problems were classified as a “Severe problem” (11%) and four problems were

Ac ce p

te

d

M

found in each of the categories “Critical problem” (3%) and “Malfunction” (3%).

10

Page 10 of 20

Table IV: Problems identified on the basis of Nielsen´s ten usability principles are classified according to severity.

Problems and severity Improvement

Mino r problem

Severe problem

Critical problem

1 22

3

Help and documentation

M

1

te

d

11

Help users, recognise, diagnose, and recover from errors

1

us

1

5

an

1

Error prevention

1

32

Aesthetic and minimalist design

5

Consistency and standards

43

User control and freedom

3

Match between system and the real world

Visibility og system status

23

10 6

Flexibility and efficiency of use

8

14

11

Recognition rather than recall

12

cr

17

ip t

29

Malfunction

Table V illustrates the distribution of problems into heuristics. All heuristics were used, but the three most

Ac ce p

frequently used heuristics were: “Match between system and the real world” (32%) with 49 of the problems, “Consistency and standards” (13%) with 20 of the problems, and “Aesthetic and minimalist design” (13%) with 20 of the problems. The problems identified in “Match between system and the real world” were related in particular to the inability of the Telekit system to speak the users’ language, i.e. its inability to use words and phrases with which users are familiar. In the Telekit system, information should appear in a natural and logical order and consist of terms that potential users were already using and able to understand. Within the heuristic, “Match between system and the real world”, there were doubts about how the measurements should be performed and how to move forward after completing the measurements. There were also some unnecessary text that should be rephrased or text that was not adequately explained. Some questions were difficult to answer on the tablet because the experts had

11

Page 11 of 20

doubts about the functionality of different buttons. Not all heuristics were applied by all experts. Only one of the most experienced experts, No. 5, used all ten heuristics to classify the problems encountered.

ip t

Table V: The distribution of problems into heuristics.

5

Expert 2

8

Expert 3

1

7

Expert 4

3

14

Expert 5

1

15

Flexibil ity and efficie ncy of use

Aesthe tic and minim alist design

an

Recog nition rather than recall

1 1

7

2

2

3

5

2

3

3

3

2

3

1

2

5

1

6

6

5

1

4

2

2

5

2

3

1

5 1 2

1

Ac ce p

6

Help users, recogn Help ize, and diagno docum se, entati and on recove r from errors

M

Expert 1

User Consis contro tency Error l and and preven freedo standa tion m rds

d

Visibili ty of system status

Match betwe en system and the real world

us

cr

16 14 12 10 8 6 4 2 0

te

No. of problems

Heuristics

3.2. Debriefing

During debriefing, the three most positive aspects and the three worst problems were discussed between the experts. The first of the positive aspects were that the Telekit system provided guidance. The experts hinted that especially the menu and navigation system were easy to use for the target audience. The second positive aspect was that because of its simple design, it was found to be useful for people who are not familiar with technology in general. The third positive aspect was that the Telekit system provided feedback, in the sense that it was obvious when the data had been transmitted. The three worst problems were, first, that no help function was integrated into the system, which meant that a user could not get help. The second negative feature was that there was no consistency in the contents and design, which caused confusion. Third, standards unique to the tablet platform were not consistently met and therefore, the system could be used on several technology platforms, i.e. 12

Page 12 of 20

smartphones, tablets, laptops, etc. The features of the system did not fully live up to standard tablet conventions such as the availability of several options through the user interface (less information,

ip t

zooming, navigation, buttons, etc.).

4. Discussion

cr

The present paper found that the heuristic method is feasible for evaluating the usability of Telekit. The Telekit system is now fully implemented, and it will be regularly updated in the future. The goal of the

us

present heuristic evaluation was to discover usability problems in Telekit so that these problems can be solved and COPD patients easily can assess the system. In order for TeleCare North to be successful, it is

an

highly important to develop and test systems, which are both easy to use by the patients in their daily lives, and which increase patients’ ability to manage their own disease and empower them to make their daily lives more rewarding.

M

The heuristic evaluation was performed using a structured table in which each expert organised the problems identified according to their severity and offered suggestions for solutions. As expected, the

d

method identified problems of the interface’s nature and it was found to be an effective method for

te

detecting problems in Telekit. The identified problems will be easy to understand and resolve for technical personnel and healthcare professionals and also readily applicable for the redesign process.

Ac ce p

The efficiency of this method makes it particularly well suited for iterative design processes. The prepared list of usability problems allowed the experts to quickly identify a large number of problems (N=152). The most experienced experts identified the largest number of problems and one of them used all classifications of severity. Even though most of the problems were classified as “Improvement” and “Minor problem”, is it still relevant to find and fix these problems. During the debriefing, the experts estimated that the problems could be solved in approximately one week. The experts’ use of heuristics and their severity classification gave rise to no problems, but they found it difficult to consistently write down the problems they noticed, among others because they would often observe several problems at the same time. To ensure that all problems were duly noted, both the individual experts and the investigator documented the full range of problems in writing during the evaluation process. The experts felt that the system was quite unfinished, and they were unanimous that 13

Page 13 of 20

general tablet conventions had not been met. The use of small buttons and the lack of touchscreen feedback on a small screen raised concerns. Many people with severe COPD are old and have physical limitations like poor eyesight, which makes it important to have large buttons in order to prevent errors.

ip t

Moreover, elderly people are often not used to new portable devices, so it would have been relevant to consider other audio or visual cues. In spite of these problems, the Telekit system had a quite simple

design, and the experts found that navigation was amenable to people unfamiliar with technology in

cr

general.

has to be improved or redesigned to be sufficiently user-friendly. 4.1. Limitations

us

The findings indicate that the Telekit system needs improvement. Especially the interface of the Telekit

an

Although use of the heuristic evaluation identified many usability problems, it is possible that important problems were missed, and that such problems could have been detected in a user test. The heuristic

M

method relies heavily on the experts’ expertise. All experts in this study had a broad knowledge of health care and COPD, but they could have missed some problems that only users might be able to find. Also,

problems.

d

the experts could have identified several problems that COPD patients would not have defined as

te

Another limitation of the method is that its focus is entirely on problems. However, the debriefing made it

Ac ce p

possible also to recognise some positive features of the system and to discuss these features. 4.2. Future work

It could be interesting to follow up with several tests to see if the changes actually work. Usability inspection methods identify potential, not actual usability problems; and Nielsen recommends a combination of both inspection methods and user testing [10]. Therefore, when Telekit has been improved by removing as many usability problems as possible, the system will undergo a user test. Because technologies become more advanced and productivity depends on patients’ use, usability methods should have a high priority in the development of telehealth systems.

5. Conclusion

14

Page 14 of 20

In this study, we evaluated the telehealth system Telekit using an inspection method called heuristic evaluation, and we identified 86 different problems that may hinder its use. The evaluation was shown to be an effective method for discovering problems of varying severity.

ip t

The system has a simple layout, which suggests that it can easily be navigated even by a non-technical person. However, the Telekit system should be improved to eliminate spelling errors, inconsistencies (spelling, content, and buttons) and to remove any doubts about how measurements should be performed.

cr

The heuristic evaluation is an important tool for developing effective telehealth systems for COPD

us

patients and for ensuring optimal user-friendliness. Importantly, areas such as communication and user testing with patients were not performed in this study. Improvements have already been made, and the system is ready for a user test planned to involve six COPD patients [27].

an

In conclusion, usability testing approaches like the heuristic evaluation give software designers and developers an opportunity to uncover severe usability issues which in a worst-case scenario could cause

M

information technology tools to remain unused. Pending successful implementation of the Telekit system in the management of COPD patients, the hope is that the present study may be used for the management

Ac ce p

Author contributions

te

Denmark.

d

of other chronic diseases in the North Denmark Region and for the management of COPD throughout

The paper was drafted by PHL and revised by MHJ and OKH. The final paper was approved by all authors.

Acknowledgements

The authors will like to thank the experts who participated in the heuristic evaluation. Furthermore, a special thanks to Associate Professor, Morten Pilegaard at the Department of Business Communication at Aarhus University for making language revisions of the manuscript.

Competing interests 15

Page 15 of 20

The authors have no conflicting interests to report.

Summary table

ip t

What was already known before this heuristic evaluation? A heuristic evaluation helps identify problems in the user interface design



Every usability problem can be linked to a generally established usability principle

us

What has the heuristic evaluation added to the body of knowledge?

cr



The method has identified factors that are crucial for the implementation of Telekit



Insight into COPD patients’ use of Telekit is recommended

an



M

References

Ho K, Cordeiro J, Hoggan B, N. Lauscher H, Grajales F, Oliveira L, et al. Telemedicine: Opportunities and developments in Member States: Report on the second global survey on eHealth. 2010 p. 1–96.

2.

Tang Z, Johnson TR, Tindall RD, Zhang J. Applying heuristic evaluation to improve the usability of a telemedicine system. Telemed J e-Health. Tang, Z., University of Texas Health Science Center at Houston, School of Health Information Sciences, UCT 600, Houston, TX 77030, United States; 2006;12(1):24–34.

3.

Bernocchi P, Scalvini S, Tridico C, Borghi G, Zanaboni P, Masella C, et al. Healthcare continuity from hospital to territory in Lombardy: TELEMACO project. Am J Manag Care. 2012 Mar;18(3):e101–8.

4.

Annandale J, Lewis KE. Can telehealth help patients with COPD? Nurs Times. 2005;107(1516):12–4.

5.

Udsen FW, Lilholt PH, Hejlesen O, Ehlers LH. Effectiveness and cost-effectiveness of telehealthcare for chronic obstructive pulmonary disease : study protocol for a cluster randomized controlled trial. 2014;1–7.

6.

Hejlesen OK. Telemedicine for Patients Suffering From COPD (Danish Telecare North Trial) Full Text View - ClinicalTrials.gov [Internet]. Clin. Trials.gov. 2014 [cited 2014 Apr 1]. p. 1. Available from: https://clinicaltrials.gov/ct2/show/NCT01984840

7.

North T. Research design in TeleCare Nord [Internet]. 2013 [cited 2014 Apr 27]. p. 1–8. Available from: http://www.rn.dk/Sundhed/Til-sundhedsfaglige-og-samarbejdspartnere/TeleCareNord/Evaluering-og-forskning/~/media/Rn_dk/Sundhed/Til sundhedsfaglige og samarbejdspartnere/TeleCare Nord/Evaluering og forskning/Research design.ashx

Ac ce p

te

d

1.

16

Page 16 of 20

TeleCare Nord. Business case TeleCare Nord [Internet]. Reg. Nord. Aalborg; 2012 [cited 2013 Mar 13]. p. 1–10. Available from: http://www.rn.dk/NR/rdonlyres/01F1E186-E075-4D94-A7BE581B9BDFC540/0/TeleCare_Nord_Business_Case_v1.pdf

9.

Bevan N. International Standards for HCI. Encycl Hum Comput Interact Idea Gr Publ. 2006. p. 1–15.

10.

Nielsen J. Characteristics of Usability Problems Found by Heuristic Evaluation [Internet]. 1995 [cited 2013 Sep 19]. Available from: http://www.nngroup.com/articles/usability-problems-foundby-heuristic-evaluation/

11.

Nielsen J. Usability 101: Introduction to Usability [Internet]. 2012 [cited 2013 Sep 15]. Available from: http://www.nngroup.com/articles/usability-101-introduction-to-usability/

12.

Lathan CE, Sebrechts MM, Newman DJ, Doarn CR. Heuristic evaluation of a web-based interface for internet telemedicine. Telemed J. Biomedical Engineering, The Catholic University of America, Washington, DC 20064, USA.; 1999;5(2):177–85.

13.

Beuscart-Zephir MC, Watbled L, Carpentier AM, Degroisse M, Alao O. A rapid usability assessment methodology to support the choice of clinical information systems: a case study. AMIA Annu Symp Preceedings. EVALAB- CERIM, Faculte de Medecine, 59045 Lille Cedex, France. [email protected]; 2002;46–50.

14.

Choi J, Bakken S. Web-based education for low-literate parents in Neonatal Intensive Care Unit: development of a website and heuristic evaluation and usability testing. Int J Med Inform. Elsevier Ireland Ltd; 2010 Aug;79(8):565–75.

15.

Graham MJ, Kubose TK, Jordan D, Zhang J, Johnson TR, Patel VL. Heuristic evaluation of infusion pumps: implications for patient safety in Intensive Care Units. Int J Med Inform. 2004 Nov;73(11-12):771–9.

16.

Kushniruk AW, Triola MM, Borycki EM, Stein B, Kannry JL. Technology induced error and usability: the relationship between usability problems and prescription errors when using a handheld application. Int J Med Inform. 2005 Aug;74(7-8):519–26.

Ac ce p

te

d

M

an

us

cr

ip t

8.

17.

Croll PR, Croll J. Investigating risk exposure in e-health systems. Int J Med Inform. 2006;76(56):460–5.

18.

Silverbullet. OpenTele | Architecture [Internet]. Opentele.Silverbullet.dk. 2013 [cited 2014 Apr 7]. Available from: http://opentele.silverbullet.dk/architecture/

19.

Dinesen M. TeleCare Nord - FAQ [Internet]. Aalborg; 2013 [cited 2014 Jun 14]. Available from: http://www.rn.dk/SundhedOgSygehuse/TeleCareNord/FAQ.htm

20.

Nielsen J. How to conduct a heuristic evaluation [Internet]. 2001 [cited 2013 Aug 12]. Available from: http://www.nngroup.com/articles/how-to-conduct-a-heuristic-evaluation/

21.

Roberts NJ, Ghiassi R, Partridge MR. Health literacy in COPD. Int J Chron Obstruct Pulmon Dis. 2008 Jan;3(4):499–507.

22.

Nielsen J. 10 Heuristics for User Interface Design: Article by Jakob Nielsen [Internet]. 1995 [cited 2013 Sep 19]. Available from: http://www.nngroup.com/articles/ten-usability-heuristics/

17

Page 17 of 20

Munk TB, Mørk K. Brugervenlighed på internettet: en introduktion. Frederiksberg: Samfundslitteratur; 2002.

24.

Virzi RA. Refining the Test Phase of Usability Evaluation : How Many Subjects Is Enough ? Hum Factors. 1992;34(4):457–68.

25.

Mack RL, Nielsen J. Usability inspection methods. Wiley & Sons; 1994. p. 1–414.

26.

Molich R. Usable web design. Copenhagen: Nyt teknisk Forlag; 2007.

27.

Heyckendorff P, Heiden S, Hejlesen OK. User Satisfaction and Experience with a Telehealth System for the Danish TeleCare North Trial : A Think - Aloud Study. 2014;900–4.

Ac ce p

te

d

M

an

us

cr

ip t

23.

18

Page 18 of 20

AUTHOR CONTRIBUTIONS FORM Pernille Heyckendorff Lilholt conceptualised the study, was the main investigator and participated in all

ip t

phases of the heuristic evaluation. Morten Hasselstrøm Jensen and Ole K. Hejlesen assisted and supervised the design of the study, its choice of methodology and the data analysis. Furthermore, the manuscript was reviewed for the English language by Associate Professor, Morten Pilegaard, Department

Ac ce p

te

d

M

an

us

cr

of Business Communication, Aarhus University.

19

Page 19 of 20

Summary Points What was already known before this heuristic evaluation? 

The practicability of performing a heuristic evaluation, namely to identify problems quickly



ip t

make this method particularly well to an iterative software design process The method does not point to systematic ways of solving problems, but because every usability

cr

problem can be linked to a general established usability principle, it is easy to make a revised

us

design of the system

What has the Heuristic Evaluation added to the body of knowledge?

The method has identified factors that are crucial for the implementation of the Telekit system



Insight into COPD patients use of the Telekit system is recommended after removing as many

an



Ac ce p

te

d

M

problems as possible in order to make sure that the system is user-friendly and satisfying

20

Page 20 of 20

Heuristic evaluation of a telehealth system from the Danish TeleCare North Trial.

The aim was to evaluate the usability of the design of the telehealth system, named Telekit, developed for the Danish TeleCare North Trial, early into...
282KB Sizes 1 Downloads 8 Views