Informatics for Health and Social Care, Early Online: 1–22, 2013 ! Informa UK Ltd. ISSN: 1753-8157 print / 1753-8165 online DOI: 10.3109/17538157.2013.872114

A novel multimodal tool for telemonitoring patients with COPD

Inform Health Soc Care Downloaded from informahealthcare.com by Nanyang Technological University on 06/05/14 For personal use only.

Daniel Sa ´ nchez-Morillo,1 Mario Crespo,1 Antonio Leo ´ n,2 and Luis F. Crespo Foix1 1

Biomedical Engineering and Telemedicine Lab, University of Ca´diz, Ca´diz, Spain and 2 Pulmonology and Allergy Unit, University Hospital Puerta del Mar, Ca´diz, Spain Introduction: Among factors that underlie high rates of non-participation reported in telehealth interventions are the low older users’ acceptance of information technologies and the low levels of non-compliance with therapy of chronic patients. Therefore, inclusion of potential users into design stages of assistive technologies is challenging. In this paper, the design, implementation and evaluation of a multimodal mobile application for telemonitoring chronic obstructive pulmonary disease (COPD) is presented. The goal of the study was to assess the usability and feasibility of the designed tool. Methods: An iterative user-centered design methodology was applied to implement a prototype that satisfied users’ requirements. Feasibility (compliance, COPD knowledge and satisfaction) of the application was assessed in a 6-month field trial with COPD patients. Results: A usable, effective and efficient prototype was released after the development process. A high compliance (86.1%) and an increasing in COPD knowledge were achieved in the field trial. Conclusions: The findings reveal the importance of integrating usability in the design development processes to improve adherence to routine tasks and to reduce the high rates of non-participation reported in recent evaluation studies of telehealth interventions. The presented tool can help to recognize early symptoms of deterioration and to support patients in COPD self-management. Keywords COPD, home monitoring, multimodal, telemonitoring, usability

INTRODUCTION Chronic obstructive pulmonary disease (COPD) is an important cause of morbidity and mortality worldwide (1). COPD adversely affects general health conditions and patient’s quality of life and has a huge socioeconomic burden (2). Factors related to COPD that influence disease progression and increase healthcare costs are well-known. Among the aforementioned problems are inadequate patient education, lack of patient compliance with therapy guidelines and failure to early detect symptoms of acute exacerbations (AECOPD) before emergency treatment and hospitalization. Therefore, there is a growing need for new COPD management tools, not only to control and alleviate symptoms and complications, but also to teach the Correspondence: Daniel Sa´nchez-Morillo, Universidad de Ca´diz, Escuela Superior de Ingenierı´a, C/Chile, 1 CP 11003 Ca´diz, Spain. Tel: +34 956015709. E-mail: [email protected]

Inform Health Soc Care Downloaded from informahealthcare.com by Nanyang Technological University on 06/05/14 For personal use only.

2

D. Sa ´ nchez-Morillo et al.

patients how to adapt their daily activities to their physiological impairments and to manage the symptoms, treatment and lifestyle changes inherent in living with a chronic condition (3). Home telemonitoring presents a sustainable alternative for the close follow-up of patients with chronic conditions. In the particular case of COPD, several studies have been recently reported (4–14) and a significant reduction in mortality and a lesser extent in hospital admissions has been reported in telehealth experiences (11). Telemonitoring of patients with COPD presents two key challenges: the advanced age of the patients and the high dropout rate reported in recent studies of home telemonitoring (15). The prevalence of COPD in patients aged over 65 years is up to 4 times more than that among those aged 45–64 years old (14). Since mean age at diagnosis is over 60 years (16), remote monitoring tools have to be usable and accessible. Otherwise they will be ineffective, function poorly and will result in medical errors (17). In this regard, computing systems and new interaction paradigms such as multimodal user interfaces can help the aging society to improve patient education, increase therapy and enhance diary keeping compliance beyond the users’ communicative limitations (18). Multimodal interfaces increase usability in case of sensory decline in elderly, increase human working memory and improve general performance by mitigating age-related attention difficulties in complex multi-tasking environments (19). Concerning the low compliance rates reported in telehealth experiences (15), operation of the devices has been identified as one of the most relevant concerns of users. Therefore, for the solutions to be effective and useful, information about patients’ needs and requirements have to be elicited through the design and development process (20). According to these challenges, this work has focused on the development of a multimodal mobile application adapted to the characteristics and needs of this target population. User-centered design (UCD) methodology, with three iterative cycles of re-development, was applied in order to deploy a solution able to overcome the high rates of non-participation in telehealth interventions reported lately. The result is a mobile application whose main objective is to allow COPD patients to report, in an autonomous way, with little training and no support figure, their symptoms and prodromes daily. The overall telemedical system has been given the name AMICA (Autonomy Motivation & Individual Self-Management for COPD patients). The aim of the present work is to report the overall experience of implementing the AMICA mobile application. We discuss the early stages when the system requirements were analyzed. A final feasibility study with 15 COPD patients that included a 6-month-long deployment was conducted to determine if patients could use this newly developed intervention accurately and in a timely manner in real-world settings. All the intermediate stages are examined when describing the development process technologies applied. The remainder of the paper is structured as follows: Section titled ‘‘Background’’ provides a detailed overview of the overall system, ‘‘Methods’’ describes the methodologies used, ‘‘Results’’ presents the results achieved through the several development and evaluation phases, ‘‘Discussion’’ discusses the results achieved and ‘‘Conclusions’’ summarizes the conclusions.

Multimodal tool for telemonitoring COPD patients

Inform Health Soc Care Downloaded from informahealthcare.com by Nanyang Technological University on 06/05/14 For personal use only.

BACKGROUND AMICA was conceived to address challenges of COPD disease management by ensuring timely transmission of clinical data and by supporting prompt medical intervention before deteriorations in patients’ conditions occur. Predicted long term health outcomes are reduced hospitalizations, improved self-management and increased patients’ quality of life. To achieve this goal, a conceptual model (Figure 1) for the remote monitoring of COPD patients was developed. AMICA consist of a client application running on a patient’s dedicated mobile device (DmD) as the main component. Questionnaire data are collected from the patient on the DmD on a daily basis. A central server with an electronic patient record module allows collecting data and provides data access for health care personnel. Automatic algorithms detect changes in basal line of symptoms. If an abnormal situation is detected, a warning is sent to the physicians who are in charge of the patient. Medical questionnaire and respiratory sounds recording A critical step in the disease management is to obtain valid information on the impact of COPD on patients’ health status. The need for a validated and simple instrument to quantify COPD and how it affects to quality of life has motivated the development of questionnaires of health status measures. Some of them like the COPD Assessment Test (21) and the clinical COPD questionnaire (22) are currently being evaluated to be used for the detection of AECOPD. In this regards, we have developed the Automated Questionnaire for the early

Figure 1. Conceptual model for the remote monitoring of patients with COPD.

3

Inform Health Soc Care Downloaded from informahealthcare.com by Nanyang Technological University on 06/05/14 For personal use only.

4

D. Sa ´ nchez-Morillo et al.

detection of COPD Exacerbations (AQCE) (23). The AQCE is specifically based on symptoms and prodromes associated to AECOPD and it has been designed to be applied at home through electronic implementation. The AQCE includes 15 items: general health status (1), cough (1), phlegm (2), dyspnea (3), sleep conditions (2), cold-like symptoms (4), lung sounds (1) and coordination test (1). Items are related to the assessment of major, minor (24) and other complementary symptoms. The answers are based on a combination of Likert scales, yes/no questions and a hand-tremor test. Symptoms and prodromes recorded through the AQCE were complemented by respiratory signals recorded through a respiratory sensor were planned to be collected daily. Hardware of the DmD and multimodal interaction characteristics An ad hoc designed tablet PC was used as DmD. DmD was equipped with a touch-screen, speakers, microphone, HSDPA capabilities, webcam, a USB connection for connecting the respiratory sensor and Windows 2007 (Microsoft Corp., Redmond, WA) operating system. The interface application for supporting the patients to perform the required daily tasks of reporting their symptoms (AQCE) included two interaction modalities. First, a visual and haptic mode allowed the patient to interact with the device through the haptic interface or mouse. Second, a voice mode was developed. Automatic speech recognition (ASR) service allowed the users to give commands or answers. A text to speech (TTS) service gave feedback to the user about the state of the interaction and the actions executed and also allowed the patients to listen the questions or options out loudly. Speech recognition and synthesis processes were accorded to international linguistic standards like SRGS and SSML grammars that permit an efficient control of conversation (25,26).

METHODS Development process: user centred design The long-term nature of COPD requires that a system deployed in a home-care setting considers not only efficiency but also usability as important factor in the design (10). Needs and requirements of COPD patients have to be included through the design and development process (20). UCD is widely considered the key to product usefulness and usability (27). The UCD approach places emphasis on the needs of users. In this study, phases of the ISO 9241-210 UCD methodology for the design and development of the multimodal mobile application were followed (28). The standard emphasizes that the process has to be iterative; that users should be actively involved in all phases of design and development for a clear understanding of user and task requirements, and that the characteristics of the potential users should be identified. Figure 2 shows the software development model applied. It is important to highlight the iterative and incremental procedure that involved the multidisciplinary team and end users. The re-development process was composed of five stages: (1) plan; (2) requirement analysis; (3) design; (4) implementation and (5) release. Phases 3 and 4 were iteratively conducted. This strategy allowed designers to obtain new releases of the application after an entire cycle was completed (29).

Inform Health Soc Care Downloaded from informahealthcare.com by Nanyang Technological University on 06/05/14 For personal use only.

Multimodal tool for telemonitoring COPD patients

Figure 2. AMICA mobile application development process.

The input of each of the performed iterations consisted of both a version of software for improvement and new requirements from users. The output of each of these iterations was a new high fidelity prototype that became the input version of the next iteration. In this study, three iterations were carried out until users were satisfied with the prototype. Phases 1 and 2: planning and initial requirements analysis The research team was made up with experts of all relevant areas for the application: pulmonology, usability, software engineer, graphical design and computational linguistic. The goal of the plan phase was to develop an overall plan to implement software to meet user’s requirements. Then, as e-health tools to predict and detect AECOPD by identifying its symptoms do not exist in clinical practice, a field-study to gain domain knowledge had to be conducted. Data from the field study and input from physicians, designers and engineers was analyzed and used as input to gather initial requirements. Phase 3: prototype analysis and design A set of concrete and detailed systems requirements was formulated as an outcome of the previous phases. Based on these requirements, a first prototype of the AMICA mobile application (V1) was designed. In the analysis phase, the performance and usability problems of the given software, resulted from a previous development cycle, were gathered and analyzed. Then, in the design

5

6

D. Sa ´ nchez-Morillo et al.

Inform Health Soc Care Downloaded from informahealthcare.com by Nanyang Technological University on 06/05/14 For personal use only.

stage, conclusions about performance and usability were used to redesign AMICA mobile application. Software engineers were mainly involved in this phase and paper-based prototyping was used to sketch redesigns. Phase 4: implementation Two activities were included in the implementation phase. First, a high fidelity prototype based on the results of the analysis and design phase was developed (30). High-fidelity prototypes were implemented using Microsoft Visual Studio.NET development environment with C# programming language. Second, usability tests were conducted with potential end-users to gather usability problems and recommendations. Participants. Tests in a lab-environment were carried out to conduct the usability evaluation of the mobile application. We decided that the sample of subjects to be studied should consist of representative target users of the system, i.e. senior users with a limited relationship with information and communication technologies (ICT). The tests took place in two elderly homes with 25 users. The experiments were carried out with portable equipment in a controlled laboratory environment and the sessions were conducted in the presence of both usability experts and the facility’s psychologist and therapist. Procedure. Usability testing was addressed according to the methodology for testing usability in the medical context in a naturalist setting proposed by Kushniruk & Patel (17). The evaluation objectives were the assessment of system functionality and usability, the generation of inputs into refinement of prototypes and the identification of problems in human–computer interaction. Subjects were divided in groups of 4 and 5 and two types of sessions (focus group and individual) were conducted. During the focus group sessions the participants were briefed on the overall project and on how to operate the application. Touch screen computers were used. During individual sessions, different tasks related to the use of the interface under development were proposed to users (for example, answer a question, choose a color for sputum, return to a previous question or check the warnings). The final task consisted of completing the medical questionnaire and the respiratory sounds recording. Subjects were asked to think aloud as they interacted with the application. This talk out loud methodology facilitates a systematic assessment of usability by capturing the individual’s ongoing thought processes during task performance (17). Following the observation, two questionnaires were presented to subjects and a semi-structured interview to elucidate the users’ opinions of using the application was conducted. One questionnaire was designed to explore participant’s attitude towards the tool. It used a 5-point Likert scales and had also open-ended questions. The second questionnaire contained scales from a standard usability questionnaire: the System Usability Scale (SUS) (31). SUS provides a global measure of system satisfaction and two correlated sub-scales of usability and learnability (32). Think-aloud reports (audio recordings) and live video recordings including the computer screen, participant’s face and physical actions were captured for a later thorough evaluation.

Multimodal tool for telemonitoring COPD patients

Inform Health Soc Care Downloaded from informahealthcare.com by Nanyang Technological University on 06/05/14 For personal use only.

As a result of this evaluation, a list of suggestions and user problems encountered during the testing sessions were forwarded to system developers in order to refine the prototype (phase 3). The interface was iteratively refined according to this procedure. Measures of usability. Measures of usability must themselves be dependent on the way in which usability is defined. In this study, three dimensions of usability were evaluated according to ISO 9241-11 (28): effectiveness, efficiency and satisfaction. Effectiveness is referred as the ability of users to complete tasks using the system. Efficiency is used to indicate the level of resource consumed in performing tasks while satisfaction is regarded to users’ subjective reactions when using the system. Among quantitative usability metrics used were: (1) the time required for completing the task, as a measure of efficiency; (2) task completion rate and the frequency and classes of incidents and data entry errors to quantify effectiveness and (3) the SUS global score as a measure of users’ satisfaction. Qualitative data were obtained through the observation of the participants and the analysis of the think aloud recordings and used to identify problems that subjects experienced in using the interface. Phase 5: release These results of the usability tests were communicated to system designers to refine the prototype. Three incremental versions that resulted from the three redevelopment iterations were developed. After the refinement of the high fidelity prototype in the third iteration, the software version was released because all design and user’s requirements were found satisfied. Feasibility evaluation field trial Best results in usability testing of mobile applications are achieved when usability tests in laboratory are combined with a field pilot (33), especially before expensive randomized clinical trials (34). In this study, feasibility was evaluated in a natural setting in a 6-month field trial, mainly to check users’ ability to self-manage the technical device to complete the remote monitoring tasks at home (unsupervised) on a daily basis. Feasibility evaluation is a goal directed activity (35). As such, three dimensions of feasibility were explored: compliance with data entry, COPD knowledge and users’ satisfaction. Participants A convenience sample of 16 patients with COPD was recruited from the Pneumology and Allergy Department in the University Hospital Puerta del Mar of Ca´diz (Spain). Field trial test was approved by the ethics boards of the University Hospital of Ca´diz. Procedure Before the field trial, individual sessions with functional demonstrations were carried out in order to explain the objectives and goals of the experiment. Finally, a DmD with the post-usability prototype AMICA R4 that resulted from the laboratory usability evaluation was delivered to each one of the COPD

7

8

D. Sa ´ nchez-Morillo et al.

Inform Health Soc Care Downloaded from informahealthcare.com by Nanyang Technological University on 06/05/14 For personal use only.

patients. Patients were contacted every 2 months by the specialists. Occasional contacts were kept in order to solve minor technical problems. Measures of feasibility Compliance was measured by analyzing data entry on the DmD that was sent to the central server. During the field trial, a diary with the users’ data and an automatic log with time and date in which tests were completed were kept. Compliance rate was calculated for each subject as the ratio of completed tasks (number of days on which a valid questionnaire and respiratory recording was received) and the number of effective days. First day was defined as the day on which the portable device was delivered and last day was considered as the day on which the device was returned. Available days were calculated as the number of days between the first and last one. Effective days were calculated excluding hospitalization days and days associated to technical problems with the user device from the available days. Overall compliance was calculated as average of individual compliance rates. Diary compliance was compared across demographic and clinical characteristics using Student’s t tests and oneway analysis of variance. Statistical significance was set at the 5% level. Statistical analysis was performed using the statistical software SPSS v19 (Chicago, IL). Knowledge of COPD was assessed at the beginning and at the end of the trial. A 20 items survey using a questionnaire derived from (36) was conducted. After the conclusion of the field trial, satisfaction was evaluated using a questionnaire developed by our usability experts. A semi-structured interview was then conducted to assess participant’s experiences on the technical operation of the device.

RESULTS Development process Phases 1 and 2: planning and initial system requirements The field study allowed to understand the health-care processes involved in the clinical management of COPD patients. Based on current chronic care and COPD guidelines, domain knowledge in COPD management was gathered. Domain knowledge included risk factors, medical requirements about monitoring, identification of stakeholders and main boarded research projects. Measurable variables of therapy success and adherence were also analyzed. Pulmonologists defined the required elements of the system and developed the content for the clinical functions. Clinical specifications that took into account context and the users’ profile were used by usability experts and engineers as input for defining initial system requirements, namely the design goals, the planning of the overall development lifecycle process and the proposal with initial recommendations for the graphic design (colors, fonts, layout and voice). Recommendations from the existing vast experience in the development of web interfaces for elderly people were stressed (Table 1). Phase 3: prototype analysis and design A first paper-prototype was designed from initial system requirements. It was designed to accept inputs via keyboard, mouse and touch-screen.

Clear and familiar Left justified text Reduce inferences Use the active voice Simple language Provide explanation of technical terms Break lengthy documents into short sections

Sans serif: Helvetica, Arial, News Gothic 12 or 14 point size Medium or bold face Capital letters and italics in headlines only

Underlining only for links

Avoid yellow, blue and green in proximity

Use dark type or graphics against a light background or white lettering on a black-colored background

Presentation of information

Font design

Graphical displays Use text-relevant images only Use icons with text as links

Table 1. Initial recommendations for the visual design of the mobile application. Navigation Simple and straightforward Explicit step-by-step navigation Single mouse clicks Static page design: everything must be found in the same places Large buttons to avoid precise mouse movements Pull down menus sparingly Avoid automatically scrolling text Incorporate buttons like previous page and next page Site map to show how the site is organized

Inform Health Soc Care Downloaded from informahealthcare.com by Nanyang Technological University on 06/05/14 For personal use only.

Multimodal tool for telemonitoring COPD patients

9

Inform Health Soc Care Downloaded from informahealthcare.com by Nanyang Technological University on 06/05/14 For personal use only.

10

D. Sa ´ nchez-Morillo et al.

Graphical icons, visual indicators and text displays were included to fully represent the information and actions available to users. Therefore users’ actions with the aim to achieve certain goals were planned to be performed through direct manipulation of the graphical elements. The application was designed considering that end-users are older than 60 years and may suffer from some deficits in their sensor capabilities like deficits in vision, hearing, attention and memory (19). Possible shortage of sensory-motor capabilities was compensated by simplifying the design, increasing the number and intensity of stimulus and by minimizing the use of attention and memory. Four main modules were included in the design: (1) daily test; (2) medication; (3) tools and (4) warnings. The daily test module included the AMICA medical questionnaire (15 items). Content layout and visual aspects. Widgets on the graphical user interface with similar functions (buttons, questions and pop-up windows) evolved to share shape and color. Their location was static with the aim of minimizing the stimulus clutter. Each screen presented a new piece of information with a limited list of stimuli. Microsoft Sans Serif black font was finally proposed for the first design. Size was 28 pt. for questions and 16 pt. for answers to improve readability. For Likert scales, five buttons as maximum were placed horizontally trying to represent a value scale. Gold color was used for the background and tomato color for navigation buttons. Questionnaire and options buttons were green, yellow-green, yellow, orange and red colored. Interface control. The content of screens presented to the patient was determined by the physicians. Each screen was designed as simple conversation with a threefold structure: (1) question or presentation of the information; (2) different options to response and (3) feedback about the selected option. This conversational structure was designed to be presented to users using graphical displays and speech recognition and synthesis technologies. The system read out the question, the answers and provided feedback to the user. In addition to that, the user could reply the system using natural voice. Conversation was modeled using finite state automata as a 5-tuple (S,,T,s0,f) where S is a set of states,  is an alphabet, T is a transition function T:Sx ! S, s0 is the initial state and f the set of final states (37). As a design requirement, data collected from conversation should be reported to the electronic patient record to be accessible health care authorized personnel. Phase 4: implementation First design: AMICA R0. A first high fidelity prototype based on the results of the analysis and design phase was implemented. It included the daily test module and the user control interface. AMICA V1 was not developed using UCD and consequently a number of usability violations were detected. This motivated the researchers to go through three iterations of re-development processes for addressing these problems and other improvements proposed by users. Usability tests were conducted with the first (V1) and subsequent high fidelity prototypes (V2 and V3).

Multimodal tool for telemonitoring COPD patients Table 2. Characteristics of the users that participated in the laboratory tests (N ¼ 25).

Inform Health Soc Care Downloaded from informahealthcare.com by Nanyang Technological University on 06/05/14 For personal use only.

Characteristic All Gender Female Male Computer use Yes No Education 8 years or less High school University

N (%) 25

Age (years), mean (SD) 75.6  9.6

19 (76) 6 (24)

76.3  9.1 73.7  10.9

7 (28) 18 (72)

74.6  9.4 76.7  8.9

13 (52) 10 (40) 2 (8)

73.6  8.8 75.5  9.1 89.5  3.5

Figure 3. Testing AMICA prototypes with elderly users.

Participants. Demographic characteristics of the users that participated in the laboratory tests are shown in Table 2. The mean (SD) age of users was 75.6 (9.6) years. Five users (20%) had some previous experience with computers. Only two users (8%) had higher education. Figure 3 shows two snapshots during usability tests. Re-development iterations. Three cycles of iteration were needed to meet the needs of users about stimulation, ease of use, expectations, quality of interaction, comfort and overall satisfaction. During the first cycle, the high fidelity prototype V1, created in the early testing process, was tested by users in a lab-setting. The content layout changed as a result of the first design cycle and remained stable in the following iterations. The design of a simple pathway that could be understood by participants with sensor-motor impairments was a priority. Fonts with only primary line traces and a simpler shape were preferred. Users were inclined to select solid background colors. Black for text and different tints of blue for backgrounds were the preferred options. Results from the first iteration were used to build an improved prototype V2. As a result of the second design cycle, the interface control was modified. Aesthetics aspects were adjusted. A more intuitive scale of colors was used for grading answers of the questionnaire. It ranged from green, for positive health values, to red for a worsening of patient conditions. The visual interface was redesigned to get a final high quality design. Taking into account these

11

12

D. Sa ´ nchez-Morillo et al.

Inform Health Soc Care Downloaded from informahealthcare.com by Nanyang Technological University on 06/05/14 For personal use only.

improvements, the prototype V3 was implemented and evaluated in the last re-development iteration. Some grammars were re-elaborated to enhance recognition performance. Dialectal phonetic features were added manually to the speech recognition grammar by a linguist to enable interface to understand nonstandard pronunciations. Software stability issues were fixed in order to improve general performance and efficiency and speed of transitions was increased. There were no further changes in content layout. Finally, AMICA R4 was released. Qualitative analysis. Through the analysis of the think aloud recordings were used to identify problems that subjects experienced in using the interface. Problems detected through the qualitative analysis have been described in the section titled ‘‘Phase 3: Prototype analysis and design’’ since there were referred as outputs to enhance high fidelity prototypes in the iterative design process. The global SUS scale ranges from 0 to 100. Item values range from 0 to 4, with four being the most positive response. Data are expressed as mean (SD). Quantitative analysis. User satisfaction was evaluated with the SUS global score that ranges from 0 to 100 (38). The SUS questionnaire was completed after the first usability test in the implementation phase and the results were compared along iterations to assess whether the each new prototype was more usable than the previous one. An analysis of variance (ANOVA) was performed in order to assess whether the differences in distribution were significant. A significance ¼ 0.05 was used. A Levene’s test for homogeneity of variance was performed. The results showed that the homogeneity of variance existed in all cases. The F ratios and the associated significance values were calculated. The detailed results of conducting SUS and the F-ratios of the differences between means after every development iteration are presented in Table 3. Significant differences among subjectively perceived usability during iterative evaluations were found in: (a) the global scale; (b) the user’s perception of the predisposition to daily use the application; (c) integration of functions; (d) comfortability; (e) confidence and (f) ease of use. Overall user satisfaction, estimated from the SUS global scale after the final iteration, outperformed the results obtained in the first and second design cycles. The final prototype was perceived as less difficult and tedious for its use on a daily basis. In the tests with the final prototype, the touch modality proved to be easy to use, but the voice mode was preferred by 68% of users. Effectiveness was assessed through the task completion rate and the frequency and classes of incidents and data entry errors. About 22 out of the original 25 subjects were able to successfully complete the required tasks using the high fidelity AMICA V3 prototype. Adjusted Wald confidence interval and best point estimate for the observed completion rate were calculated. The observed completion rate was 88%, 95% CI (69–97). The number of errors made by the system, voice recognition problems and errors made though not perceived by the user were annotated. Table 4 displays a summary of these results in the first and last iteration.

AMICA V1 72.5 (5.5) 2.5 (0.7) 3.11 (0.5) 3.1 (0.2) 2.9 (0.6) 2.0 (0.4) 2.8 (0.8) 3.0 (0.5) 3.6 (0.5) 3.0 (0.2) 3.1 (0.2)

Prototype

Global Scale 1. Would to use the system frequently 2. System unnecessarily complex 3. System easy to use 4. Need support to use this system 5. Functions in this system well integrated 6. Too much inconsistency in this system 7. Most people would learn to use this system quickly 8. Found the system very cumbersome to use 9. Felt very confident using the system 10. Need to learn a lot before going with this system

Table 3. SUS converted scores for each of the iterative cycles in laboratory tests.

72.9 2.9 2.8 3.0 3.0 1.9 2.8 3.1 3.5 3.1 3.1

(8.3) (0.5) (1.0) (0.6) (0.6) (0.3) (0.8) (0.6) (0.6) (0.5) (0.3)

AMICA V2 80.6 3.0 3.2 3.1 3.1 2.9 3.0 3.2 4.0 3.5 3.3

(5.0) (0.5) (0.7) (0.4) (0.5) (0.5) (0.5) (0.7) (0.2) (0.5) (0.5)

AMICA V3

Inform Health Soc Care Downloaded from informahealthcare.com by Nanyang Technological University on 06/05/14 For personal use only.

7.7 3.0 1.7 0.3 0.4 3.0 1.5 0.4 8.5 7.6 3.4

F-ratio

50.001 50.05 ns ns ns 50.05 ns ns 50.001 50.005 50.05

p

Multimodal tool for telemonitoring COPD patients

13

14

D. Sa ´ nchez-Morillo et al. Table 4. Errors made during the performance of tasks throughout the iterative design cycle.

Inform Health Soc Care Downloaded from informahealthcare.com by Nanyang Technological University on 06/05/14 For personal use only.

System errors (%) Voice misrecognition (%) User errors (%) Total errors (%)

First iteration (%)

Third iteration (%)

2.6 1.6 2.1 6.3

0.4 0.9 0.4 1.7

Figure 4. Times for the completion of the daily test using the final prototype.

Efficiency was estimated by measuring the time required for completing a task: the completion of the questionnaire. The average time needed by participants to complete the test using the prototype AMICA V3 was 332 seconds, 95% CI (31.5). Figure 4 shows the values displayed in a cumulativefrequency graph. Phase 5: release The process that started with an initial conceptual design concluded with a prototype that was satisfactory in terms of the user’s perceptions. Figure 5 illustrates some examples of transition from the first conceptual design to the released version R4 to be used in the field trial. Feasibility evaluation field trial Participants About 16 consenting patients were screened for telemonitoring and 15 were included in the final study. Table 5 includes the demographic details of the users that participated in the telemedical pilot. All users were patients with

Inform Health Soc Care Downloaded from informahealthcare.com by Nanyang Technological University on 06/05/14 For personal use only.

Multimodal tool for telemonitoring COPD patients

Figure 5. From the conceptual design to the functional prototype. The first design (left) was carried out according to recommendations and objectives. The final design (right) was the result of the iterative development process.

COPD (FEV1/FVC50.7) aged 60–81 years. COPD severity was ranged from II to IV according to GOLD criteria. About 93.8% of patients did not have higher education. Cognitive impairments were not detected as Pfeiffer test’s score was normal in all cases. Procedure The patients were monitored during 2672 days. About six patients were hospitalized during 195 days and two patients had technical problems during 68 days. A period of 30 days could not be analyzed due to a fault in a patient’s device. The remaining 38 days affected a patient that moved to an area with a poor high-speed downlink packet access (HSDPA) network coverage that prevented effective and timely transmission of data to the remote management center. The number of effective days was therefore 2409. Measures of feasibility Feasibility data were based on assessment of compliance and satisfaction (Section ‘‘Feasibility evaluation field trial’’).

15

16

D. Sa ´ nchez-Morillo et al. Table 5. Characteristics of the users that participated in the field trial (N ¼ 16) and compliance rates.

Inform Health Soc Care Downloaded from informahealthcare.com by Nanyang Technological University on 06/05/14 For personal use only.

Variable

N (%)

All 15 Gender Female 1 (7%) Male 14 (93%) Marital status Married/de facto 9 (60%) Other 6 (40%) Education 8 years or less 11 (73%) High school 3 (20%) College 1 (7%)

Increase Age (years), on COPD p Overall p mean (SD) knowledge* Value compliance Value 70.2  6.6

10.4  6.9

86.1  11.9%

68.0 70.4  6.9

10.0 10.4  7.2

63.9% 87.7  10.5%

67.3  5.8 74.5  5.8

11.9  7.0 8.4  6.8

70.8  7.0 70.7  5.5 62.0

10.00  7.1 12.5  10.6 10.0

0.36

0.55 87.7  11.0% 83.8  13.8%

0.91

0.63 84.3  13.0% 90.4  7.9% 93.5%

Overall compliance is presented as percentages expressed as mean  SD. *Results for the final COPD Knowledge Questionnaire were not available in one patient.

Compliance. During the field trial, 2104 daily tests were received correctly in the telemedical server. Mean for individual compliance rates was 86.1% and the ratio of the total number of days completed (2014) and the number of available days for completion (2409) was 87.3%. About 51 non-programmed medical interventions were registered for 10 patients. About 33 out of the 51 events corresponded to AECOPD and the resting 18 events were associated to non-recovered exacerbations. Compliance rates were compared across demographic and clinical features. D’Agostino-Pearson test for normal distribution accepted normality (p ¼ 0.1351) for compliance rates distribution. As can be appreciated in Table 5, there were no significant differences in rates across demographic variables. Participants with primary education seemed to present a lower compliance rate than those with high school or university education. Single subjects and those living alone had a lower average compliance. But bigger sample would be required to verify the statistical significance of those relationships. Table 6 summarizes compliance with diary keeping by clinical variables. Participants with different disease severity or number of comorbidities did not show significant differences in their compliance rates. Similarly, compliance seemed not to be related to the number of reported acute exacerbations or to the number of days hospitalized during the study period. Satisfaction and COPD knowledge. An increase of 10.4 points (out of 100) in the demonstrated knowledge of COPD along the trial was quantified. Overall satisfaction of field trial users was 4.9 on a scale of 5. Overall compliance is presented as percentages expressed as mean  SD.

DISCUSSION A mobile application with multimodal interaction for the remote monitoring of patients with COPD was developed in a laboratory setting and tested during a long-term field trial.

Multimodal tool for telemonitoring COPD patients Table 6. Compliance with diary keeping by clinical features.

Inform Health Soc Care Downloaded from informahealthcare.com by Nanyang Technological University on 06/05/14 For personal use only.

Variable COPD stage Stage I (480%) Stage II (50–79%) Stage III (30–49%) Stage IV (530%) Comorbidities None One More than one Number of AECOPD during the field trial None 1–2 3 or more Days hospitalized 0 1–10 11 or more

N (%)

Overall compliance

p Value 0.50

0 (0%) 2 (13.3%) 7 (46.7%) 6 (40.0%)

93.7  3.6% 82.9  13.0% 87.3  12.2%

4 (26.7%) 3 (20.0%) 8 (53.3%)

88.9  4.7% 86.8  %15.8 84.4  13.9%

5 (33.3%) 2 (13.3%) 8 (53.4%)

88.7  4.2% 88.1  7.4% 84.0  15.9%

9 (60.0%) 2 (13.3%) 4 (26.7%)

86.5  10.3% 83.0  20.0% 86.8  15.3%

0.84

0.79

0.93

The mail goal of the application was to overcome the limitations of paper for self-monitoring, namely low compliance, labor-intensiveness and possible recall-bias (39). Strengthening compliance of elderly chronic patients with daily computerized tasks is a major challenge. Cognitive decline, motor and sensor disabilities affect older patient’s ability to effectively and comfortably interact with health related technology systems (40). Therefore, older people targeted by assistive technologies need more easy-to-use methods to interact with inherently complex supporting technology (41). In particular, patients with COPD have demonstrated low levels of intentional non-compliance with therapy (42). To this purpose, needs and requirements of older participants were first included through the design and development process using a user centred design approach to obtain early feedback from users. Beyond laboratory tests, the refined prototype was evaluated in a long-term field trial aimed to evaluate the feasibility of the system by assessing whether a multimodal telemonitoring tool could be daily used by patients with moderate to severe COPD. Increasing COPD knowledge and overall user satisfaction were also assessed in the field trial. To date, a monitoring tool of symptoms for COPD, especially designed for older users, has not been reported. In the last years, a few studies have presented telemedical systems based on commercial off-the-shelf devices for remote monitoring of patients with COPD (4–11). However, these devices are not generally well accepted in medical applications (43), mainly due to that human-machine interface mechanisms of mobile devices are not adapted to the needs and requirements of older users. Moreover, evaluation studies of telehealth interventions are reporting high rates of non-participation that are not well understood. Barriers to participation and adoption of telehealth solutions have been recently explored in the WSD trial (15), where the requirements for technical competence and operation

17

Inform Health Soc Care Downloaded from informahealthcare.com by Nanyang Technological University on 06/05/14 For personal use only.

18

D. Sa ´ nchez-Morillo et al.

of the equipment were identified as a prime concern of users. Designers are not taking into account either acceptance of older users or their needs and priorities although there is scientific consensus on that the usability of health technologies is critical to improve healthcare (44). However, only 19% of the studies actually apply usability methods within the development phase (45). In our study, an iterative UCD methodology was applied. Design guidelines and input from domain experts and target users allowed for an easy-to-use multimodal application which may overcome compliance and satisfaction issues. The results of usability tests showed that the post-usability prototype was usable and able to span both age and computer experience. Qualitative analysis of collected data during the usability tests enabled the release of a final prototype that met users’ requirements. Along the UCD processes, usability was evaluated in three dimensions: effectiveness, efficiency and satisfaction. In terms of efficiency, average completion time for the daily test in the final iteration of the re-development cycle was 332 seconds. The average response rate for the 15 items questionnaire was over 4 items per minute. Completion time depends on many factors like the type of questions, age, reading skills, education and cognitive status among others. A rule of thumb is about three to five items can be administered per minute as average (46). Whereas the target users are older people and that the interface provides an acoustic feedback on every item to avoid mistakes, the average completion time is considered appropriate. Effectiveness was characterized by a completion rate of 88%. Regarding with the dimension of usability related to satisfaction, all study participants felt that the computer application would help them feel more engaged in the process. A high scoring for the overall satisfaction was achieved. The SUS global scale in the third iteration (80.6) outperformed the results of the previous stages. Furthermore, users considered that they would use the tool frequently (3 on a scale of 4). But the presence of the physician during the survey after the field trial may have affected the behavior of users. Although the participants were encouraged to mention both strong and weak features of the application, they might have felt reluctant to be critical. Incorporating input from patients in the early stages of application development may improve the construct validity of the telemonitoring tools and enhance their practical application in healthcare. This practical application was evaluated in a 6-month field trial. The results of the field trial, in terms of compliance, appear to support this strategy. Compliance rates in COPD telemonitoring are barely reported. Rates ranged from 62% to 82% have been recently published for trials over 6 months (4,7,9). Many of these studies contacted patients regularly to improve compliance. In our study, the daily compliance achieved (86.1%) in the field trial without any mentoring program outperformed other published results. Significant differences in compliance rates across demographic variables as well as across disease severity, number of comorbidities, days hospitalized and number of AECOPD were not found. However, participants with primary education seemed to present a lower compliance rate than those with high school or university education. These differences has been previously reported (7) and worthy further research. Similarly, single subjects and those living alone showed a lower average rate.

Inform Health Soc Care Downloaded from informahealthcare.com by Nanyang Technological University on 06/05/14 For personal use only.

Multimodal tool for telemonitoring COPD patients

A high increase in COPD knowledge along the trial was also quantified. This fact must not be overlooked given that success in self-management approaches lies in the ability of patients to early recognize AECOPD symptoms in order to promptly start on treatment (47). Finally, a remarkable overall satisfaction after the field trial was 4.9 on a scale of 5. In summary, this study has demonstrated that a high compliance with selfmonitoring tasks in COPD can be achieved when the tool is designed according to users’ preferences and needs. As far as the authors know, the developed mobile application is the first multimodal application reported for supporting the collection of symptoms for the remote monitoring of COPD patients. The resulting clinical benefits are of great importance. Telemonitoring enhanced patients’ disease knowledge what support the early recognition of deterioration signs. Patients reported that the tool helped them to increase their awareness of their symptoms and how AECOPD are impacting their life. Furthermore, since the medical questionnaires and respiratory sounds are available on a daily basis in the remote management center, decision algorithms to manage COPD can be easily updated to adapt the changes in clinical guidelines. Finally, certain limitations in our study should be considered. First, a larger field trial sample is needed. Possible correlation between compliance rate and demographic features like education or marital status requires further investigation that could lead to the design of specific support actions for people in lower educational layers.

CONCLUSIONS Design, implementation and evaluation of a multimodal health interface for the remote monitoring of patients with COPD have been presented. UCD methodology was applied to implement a prototype that satisfied users’ requirements. Feasibility (compliance, increased COPD knowledge and satisfaction) of the developed application was assessed in a 6-month field trial where a group of patients with COPD used the post-usability prototype at home on a daily basis. The designed mobile application achieved a high daily compliance in the field trial, outperforming other reported results. Furthermore, the telemonitoring process enhanced patients’ disease knowledge, which is crucial in selftreatment approaches. The findings reveal the importance of integrating usability in the design development processes to improve adherence to routine tasks and to reduce the high rates of non-participation that have been reported in recent evaluation studies of telehealth interventions. The results confirm that the developed multimodal mobile application fits the older users’ needs regardless most of their common limitations. Future work will have to include an investigation over a larger stratified sample of patients of the increase in COPD knowledge and the possible influence in the compliance rate of socioeconomic factors.

DECLARATION OF INTEREST The authors report no conflicts of interest. This work was supported in part by the Ambient Assisted Living (AAL) E.U. Joint Programme, by grants from

19

20

D. Sa ´ nchez-Morillo et al.

Ministerio de Educacio´n y Ciencia of Spain and Instituto de Salud Carlos III under Projects PI08/90946 and PI08/90947.

Inform Health Soc Care Downloaded from informahealthcare.com by Nanyang Technological University on 06/05/14 For personal use only.

REFERENCES 1. Divo M, Cote C, de Torres JP, et al. Comorbidities and risk of mortality in patients with chronic obstructive pulmonary disease. Am J Respir Crit Care Med 2012;186: 155–61. 2. Vestbo J, Hurd SS, Agustı´ AG, et al. Global strategy for the diagnosis, management, and prevention of chronic obstructive pulmonary disease. Am J Respir Crit Care Med 2013;187:347–65. 3. Barlow J, Wright C, Sheasby J, et al. Self-management approaches for people with chronic conditions: a review. Patient Educ Couns 2013;48:177–87. 4. Sund ZM, Powell T, Greenwood R, Jarad NA. Remote daily real-time monitoring in patients with COPD – a feasibility study using a novel device. Respir Med 2009; 103:1320–8. 5. Ure J, Pinnock H, Hanley J. Piloting tele-monitoring in COPD: a mixed methods exploration of issues in design and implementation. Care Respir J 2012;21:57–64. 6. Singh D, Kampschulte J, Wedzicha JA, et al. A trial of beclomethasone/formoterol in COPD using EXACT-PRO to measure exacerbations. Eur Respir J 2013;41: 12–17. 7. Walters EH, Walters J, Wills KE, et al. Clinical diaries in COPD: compliance and utility in predicting acute exacerbations. Int J Chron Obstruct Pulmon Dis 2012;7: 427–35. 8. Trappenburg JCA, Touwen I, de Weert-van Oene GH, et al. Detecting exacerbations using the Clinical COPD Questionnaire. Health Qual Life Outcomes 2010;8:102. 9. Bennett AV, Amtmann D, Diehr P, Patrick DL. Comparison of 7-day recall and daily diary reports of COPD symptoms and impacts. Value Health 2012;15:466–74. 10. Van der Heijden M, Lucas PJF, Lijnse B, et al. An autonomous mobile system for the management of COPD. J Biomed Inform 2013;46:458–69. 11. Steventon A, Bardsley M. Effect of telehealth on use of secondary care and mortality: findings from the Whole System Demonstrator cluster randomised trial. BMJ 2012;21:344:e3874. 12. Bolton CE, Waters CS, Peirce S, Elwyn G. Insufficient evidence of benefit: a systematic review of home telemonitoring for COPD. J Eval Clin Pract 2011;17: 1216–22. 13. Polisena J, Tran K, Cimon K, et al. Home telehealth for chronic obstructive pulmonary disease: a systematic review and meta-analysis. J Telemed Telecare 2010;16:120–7. 14. Akinbami L, Liu X. Chronic obstructive pulmonary disease among adults aged 18 and over in the United States, 1998–2009. NCHS Data Brief 2011;63:1–8. 15. Sanders C, Rogers A, Bowen R, et al. Exploring barriers to participation and adoption of telehealth and telecare within the Whole System Demonstrator trial: a qualitative study. BMC Health Serv Res 2012;12:220. 16. Devereux G. ABC of chronic obstructive pulmonary disease: definition, epidemiology, and risk factors. BMJ 2006;332:1142–4. 17. Kushniruk AW, Patel VL. Cognitive and usability engineering methods for the evaluation of clinical information systems. J Biomed Inform 2004;37:56–76. 18. Naumann A, Wechsung I, Hurtienne J. Multimodal interaction: a suitable strategy for including older users? Interact Comput 2010;22:465–74. 19. Schieber F. Human factors and aging: identifying and compensating for age-related deficits in sensory and cognitive function. In: Charness N, Schaie KW, eds. Impact of technology on successful aging. New York: Springer; 2003:42–84. 20. Das A, Svanæs D. Human-centred methods in the design of an e-health solution for patients undergoing weight loss treatment. Int J Med Inform 2013;80:1075–91. 21. Jones PW, Harding G, Berry P, et al. Development and first validation of the COPD Assessment Test. Eur Respir J 2009;34:648–54.

Inform Health Soc Care Downloaded from informahealthcare.com by Nanyang Technological University on 06/05/14 For personal use only.

Multimodal tool for telemonitoring COPD patients

22. van der Molen T. Development, validity and responsiveness of the Clinical COPD Questionnaire. Health Qual Life Outcomes 2003;1:13. 23. Leo´n A, Astorga S, Crespo M, et al. Development of an automated questionnaire for the early detection of COPD exacerbations. Eur Respir J 2011;38:82s. 24. Anthonisen NR, Manfreda J, Warren CP, et al. Antibiotic therapy in exacerbations of chronic obstructive pulmonary disease. Ann Intern Med 1987;106:196–204. 25. Dahl DA. The W3C multimodal architecture and interfaces standard. J Multim User Interfaces 2013;1–12. Available from: http://link.springer.com/10.1007/ s12193-013-0120-5 [last accessed 13 Sept 2013]. 26. Larson JA. VoiceXML and the W3C speech interface framework. IEEE Multimedia 2003;10:91–3. 27. Mao J-Y, Vredenburg K, Smith PW, Carey T. The state of user-centered design practice. Commun ACM 2005;48:105–9. 28. ISO FDIS 9241-210. Ergonomics of human–system interaction – Part 210: Humancentred design for interactive systems. International Organization for Standardization (ISO) 2010. 29. Lu C-C, Kang S-C, Hsieh S-H, Shiu R-S. Improvement of a computer-based surveyor-training tool using a user-centered approach. Adv Eng Inform 2009;23: 81–92. 30. Robertson S, Robertson J. Investigating the work. In: Mastering the requirements process: getting requirements right. 3rd ed. Upper Saddle River, NJ: AddisonWesley Professional; 2012:109–16. 31. Bangor A, Kortum P, Miller J. Determining what individual SUS scores mean: adding an adjective rating scale. JUS 2009;4:114–23. 32. Borsci S, Federici S, Lauriola M. On the dimensionality of the System Usability Scale: a test of alternative measurement models. Cogn Process 2009;10: 193–7. 33. Kaikkonen A, Keka¨la¨inen A, Cankar M, et al. Usability testing of mobile applications: a comparison between laboratory and field testing. JUS 2005;1:23–8. 34. Farzanfar R, Finkelstein J, Friedman R. Testing the usability of two automated home-based patient-management systems. J Med Syst 2004;28:143–53. 35. Ryan P, Pumilia NJ, Henak B, Chang T. Development and performance usability testing of a theory-based, computerized, tailored intervention. Comput Inform Nurs 2009;27:288–300. 36. Leo´n Jime´nez A, Casas Maldonado F, Oliva´n Martı´nez E, et al. Conozca co´mo vivir mejor con la EPOC: Respuestas sencillas a las preguntas ma´s frecuentes sobre la enfermedad pulmonar obstructiva cro´nica (EPOC). Conserjerı´a de Salud, Junta Andalucı´a; 2009. Available from: http://www.repositoriosalud.es/handle/10668/220 [last accessed 13 Sept 2013]. 37. Crespo M, Sa´nchez D, Crespo F. Collaborative dialogue agent for COPD selfmanagement in AMICA: a first insight. In: Demazeau Y, Dignum F, Corchado JM, Bajo Pe´rez J, eds. Advances in practical applications of agents and multiagent systems. Vol. 70. Berlin Heidelberg: Springer; 2010:75–80. 38. Brooke J. SUS-A quick and dirty usability scale. In: Jordan PW, Thomas B, Weerdmeester BA, McClelland AL, eds. Usability evaluation in industry. London: Taylor and Francis; 1996. 39. Tsai CC, Lee G, Raab F, et al. Usability and feasibility of PmEB: a mobile phone application for monitoring real time caloric balance. Mobile Netw Appl 2007;12: 173–84. 40. Piper AM, Hollan JD. Supporting medical communication for older patients with a shared touch-screen computer. Int j Med Inform 2011:1–9. Available from: http:// www.ijmijournal.com/article/S1386-5056(11)00071-2/ [last accessed 13 Sept 2013]. 41. Goetze S, Moritz N, Appell J. Acoustic user interfaces for ambient-assisted living technologies. Inform Health Soc Care 2010;35:125–43. 42. Jones R, Hyland M, Hanney K, Erwin J. A qualitative study of compliance with medication and lifestyle modification in Chronic Obstructive Pulmonary Disease (COPD). Prim Care Respir J 2004;13:149–54.

21

Inform Health Soc Care Downloaded from informahealthcare.com by Nanyang Technological University on 06/05/14 For personal use only.

22

D. Sa ´ nchez-Morillo et al.

43. Pare´ G, Jaana M, Sicotte C. Systematic review of home telemonitoring for chronic diseases: the evidence base. J Am Med Inform Assoc 2007;14:269–77. 44. Yen P, Bakken S. Review of health information technology usability study methodologies. J Am Med Inform Assoc 2012;19:413–22. 45. Peute LWP, Spithoven R, Bakker PJM, Jaspers MWM. Usability studies on interactive health information systems; where do we stand? Stud Health Technol Inform 2008;136:327–32. 46. Hays R, Reeve B. Measurement and modeling of health-related quality of life. In: Heggenhougen K, Quah S, eds. International encyclopedia of public health. San Diego: Academic Press; 2008:241–52. 47. Seemungal TA, Donaldson GC, Bhowmik A, et al. Time course and recovery of exacerbations in patients with chronic obstructive pulmonary disease. Am J Respir Crit Care Med 2000;161:1608–13.

A novel multimodal tool for telemonitoring patients with COPD.

Introduction: Among factors that underlie high rates of non-participation reported in telehealth interventions are the low older users' acceptance of ...
483KB Sizes 0 Downloads 0 Views