The design and evaluation of a peripheral device for use with a computer game intended for children with motor disabilities.

c o m p u t e r m e t h o d s a n d p r o g r a m s i n b i o m e d i c i n e 1 1 8 ( 2 0 1 5 ) 44–58

journal homepage: www.intl.elsevierhealth.com/journals/cmpb

The design and evaluation of a peripheral device for use with a computer game intended for children with motor disabilities Terigi Augusto Scardovelli ∗ , Annie France Frère University of Mogi das Cruzes, Technology Research Centre, Av. Dr. Cândido Xavier de Almeida e Souza 200, Mogi das Cruzes, São Paulo, Brazil

a r t i c l e

i n f o

a b s t r a c t

Article history:

Many children with motor impairments cannot participate in games and jokes that con-

Received 18 June 2014

tribute to their formation. Currently, commercial computer games there are few options of

Received in revised form

software and sufficiently flexible access devices to meet the needs of this group of children.

1 October 2014

In this study, a peripheral access device and a 3D computerized game that do not require the

Accepted 2 October 2014

actions of dragging, clicking, or activating various keys at the same time were developed. The peripheral access device consists of a webcam and a supervisory system that processes

Keywords:

the images. This method provides a field of action that can be adjusted to various types

Adapted peripheral device

of motor impairments. To analyze the sensitivity of the commands, a virtual course was

3D game

developed using the scenario of a path of straight lines and curves. A volunteer with good

Motor impairment

ability in virtual games performed a short training with the virtual course and, after 15 min of training, obtained similar results with a standard keyboard and the adapted peripheral

Leisure

device. A 3D game in the Amazon forest was developed using the Blender 3D tool. This free software was used to model the characters and scenarios. To evaluate the usability of the 3D game, the game was tested by 20 volunteers without motor impairments (group A) and 13 volunteers with severe motor limitations of the upper limbs (group B). All the volunteers (group A and B) could easily execute all the actions of the game using the adapted peripheral device. The majority positively evaluated the questions of usability and expressed their satisfaction. The computerized game coupled to the adapted device will offer the option of leisure and learning to people with severe motor impairments who previously lacked this possibility. It also provided equality in this activity to all the users. © 2014 Published by Elsevier Ireland Ltd.

1.

Introduction

For children with motor deficiencies, participation in daily life activities, especially playing, guarantees that the brain and the body continue to be stimulated and active. Playing is the

∗

first productive activity of a child. The future competence of children is constructed by simulating the roles that they will later hold as a student, father/mother, doctor, police officer, driver, etc. [1]. Through games, children learn to act; their curiosity is stimulated; and they acquire initiative and selfconfidence [1].

Corresponding author. Tel.: +55 11 47987112. E-mail addresses: [email protected] (T.A. Scardovelli), [email protected] (A.F. Frère).

http://dx.doi.org/10.1016/j.cmpb.2014.10.002 0169-2607/© 2014 Published by Elsevier Ireland Ltd.


However, when physical conditions limit exploration and manipulation activities, which are basic requirements for play, they can impede a special-needs child from developing the skills and attitudes that allow them to discover the world around them. These children in addition to being limited within their body, they are often not able to develop a healthy relationship with themselves, their family or society; therefore, they are confronted not only by their own limitations but also by the limitations in their interpersonal interactions, primarily in the pursuit of pleasure. It is difficult to find a child who is not fascinated by contact with a computer [2]. Therefore, the computer can be a very useful tool in supporting the global development of children with severe motor disorders when used as an instrument for stimulation and communication. The computer can potentially serve as a therapeutic tool because computer use can combine functionality, pleasure, discovery, and autonomy. In addition, computers have been used in combination with virtual environments for the rehabilitation of patients who have experienced cerebrovascular accidents [3]; simulated training [4,5]; the cognitive rehabilitation of patients with different types of mental health disorders [6,7]; providing leisure activities for the visually impaired [8]; and analyzing movements [9,10]. Another study [11] has demonstrated that virtual environments can play an important role in the treatment and training of individuals with disabilities. However, accessibility and usability are essential for allowing users to completely realize the benefits of informatics. Usability is related to efficiency and user satisfaction [12]. Accessibility refers to not only the ability to reach a resource but also the potential for using a resource in a satisfactory manner [13]. Therefore, individuals with physical disabilities can only fully enjoy the benefits of informatics if peripheral devices are adapted to account for these individuals’ limitations. Certain authors have used webcams [14–17], Kinect [18], Wii [19] or Leap Motion [20] as tools for developing these types of peripheral devices. Webcams have been used as access devices that allow children with severe physical limitations to play 2D computer games [14]. Other authors [15,16] have sought to aid the rehabilitation of stroke victims by utilizing a webcam or augmented reality [21] in the development of a system and a 3D environment with increased levels of realism and immersion for players. Several cameras were used for the recognition of all the movements of the hand [22], but computer access was only possible for children who had preserved hand movements. The Leap Motion is a revolution in the field of human–computer interaction, but due to irregular sampling frequency, it is not suitable as motion tracking system [20,23]. A depth-sensitive camera and a virtual environment were used to analyze the involvement of the pre-frontal cortex in equilibrium tasks [24,25] and for neuro-rehabilitation [26]. In another study [17], a webcam was also used for navigation of a 3D environment; however, the system was not developed for children with fine motor limitations, as it required precision in the movements. In another study [18], Kinect was used as a tool for rehabilitation; however, it did not permit interaction by people with fine motor limitations. Wii was used as a

45

computer access tool [19]; however, the system requires precision in the activation of the console buttons. To enable all individuals with physical limitations to enjoy computer-based entertainment, interfaces adapted to the requirements of each individual are required. In this sense, the need for interfaces that suit individuals’ needs is evident [3]. Relevant requirements for achieving this objective include the ability to use an interface in real time and with precision. In 3D environments, characters’ movements must be triggered by rapid sequences of commands or keystrokes or by the use of complex, multi-level menus. These requirements require dexterity and precise movements. Consequently, children with severe upper-body motor limitations cannot effectively play certain computer games because the available systems associated with these games are insufficiently accessible to and usable by these children. Therefore, in this work, a 3D game triggered by a peripheral device adaptable to every need was developed. The peripheral device is composed of a webcam that allows players to control their characters’ actions in a 3D environment, enabling children with upper-body motor limitations and children without these limitations to derive equal pleasure from the game in question.

2.

Method

2.1.

System design – peripheral device – game

The system was composed of a 3D game and a supervisory system that controls the character based on the images captured by a webcam. The 3D game was executed in a computer screen window (game window), and the supervisory system was executed on a second plane in a second window (control window). The user image, captured by the webcam, was shown in the control window (Fig. 1). In control window three action regions were selected. Only the pixels within these regions were analyzed by the supervisory program. The other regions of this image served only as a reference. When a movement was filmed in one of these three regions, there were significant changes in the pixels of the region. The supervisory system analyzed which region exhibited a change and sends a command to the 3D game. The three action regions selected to control the movements and actions of the game character were bounded by three squares. These squares could be positioned in any part of the image captured by the webcam. This system permitted adaptation as a function of the type and degree of motor impairment of the user. For example, for a user with only the articular amplitude of the fingers preserved, the squares could be placed side-by-side in the control window (Fig. 2A). In this case, if the user was positioned at the standard distance for computer use (45 cm from the webcam); he/she just needed to position the squares next to each other to control the character. For a user with tremors who was also 45 cm from the webcam, the squares could be positioned such that one was in the middle at the top of the control window, one was in the lower left corner, and one was in the lower right corner (Fig. 2B). Thus, the squares would not be activated by involuntary movements. In this manner, even a user with severe

46


Fig. 1 – Computer screen with the windows of the system. Windows of the 3D game on the left side and the control window in the lower right corner (supervisory system).

motor impairments could activate the game using their preserved movements. To aid in the recognition of actions that the user executes in the 3D game, each square was identified by a letter: R to turn to the right, L to turn to the left, and F to walk forward or select the items necessary to overcome the challenges.

2.1.1.

2.1.1.1. Model layer. A database was used to store users’ data as well as the position of the action areas adjusted to their needs. Thus, pre-registered users did not need to perform the calibration again. The aid of a companion was therefore necessary only for the initial user recognition by the system. The information requested from the volunteer was as follows: name, age, login, and password.

Architecture of the supervisory system

The architecture of the supervisory system was divided into three layers: Objects that collected information from the external environment and allowed interaction between the user and the system were shown in the vision layer. In the control layer, objects that specify the domain of control were defined, in other words, objects of capture, treatment, and rules for comparison of the images to detect the movement. The objects that perform the communication between the supervisory system and the 3D game were also defined in this layer. In the model layer, the information to be stored in the database for later analysis was defined.

2.1.1.2. Vision layer. The supervisory system was divided into two working modes: editing mode and execution mode. In editing mode, the positions of the action areas could be altered. When the supervisory system was in editing mode, the squares had red edges and could be moved within the control window. To move the squares, the xy position of the upper left corner of each square could be redefined. Therefore, the companion must select a square, click, and drag it to the desired position. After positioning, the user must perform some movements to allow the software to analyze whether there were differences between the frames. If there were differences, the

Fig. 2 – Examples of positioning of the squares in the control window. Positioning of the squares in the control window according to the type of movement possible for the user: (A) users capable of precise small-amplitude movements; or (B) users with tremors.


47

Fig. 3 – Diagram of activities of the supervisory system. Execution sequence of the supervisory system before entering execution mode.

corresponding square was entirely filled with the red color, signaling that it was in the position desired by the user. If the user could activate each of the three squares, the program showed another message asking if the calibration was complete. If not, the system returned to allow the companion to alter the positioning. If the calibration was complete, the user data and position of each square were recorded in the database. Thus, the positions selected during calibration were available when the same user returns to the game. Fig. 3 is a diagram of the activities of the supervisory system in editing mode. When the software finishes recording in the database, the square edges become green, indicating that the software was in execution mode. In execution mode, when the software identified a movement in one of the squares, the square was entirely filled with green (Fig. 4), signaling that the command was executed in the game. In execution mode, a movement in one of the action areas was identified and activates, through the API of Windows®, the corresponding command in the 3D game. This movement was interpreted as the pressing of an arrow on the keyboard. Therefore, each detected alteration in the action areas generates an action of the primary character in the virtual environment of the 3D game. First, the supervisory system

Fig. 4 – Action field of the user in execution mode. The squares with green edges indicate that they were ready to control the game. The letter identified the command: L – turn to the left, F – go forward, and R – turn to the right. The green square indicated that it detects movement indicating the command “turn to the left”.

48


Fig. 5 – Diagram of activities of communication between the supervisory system and the 3D game. Execution sequence of the supervisory system in execution mode and communication with the 3D game.


2.2.1.

Fig. 6 – Use case diagram. This diagram describes the functional requirements of the system.

analyzed the square with the letter L, then the square with the letter F, and finally the square with the letter R to verify whether there was an alteration. For the user to move the character and execute actions, he/she must use the up arrow key (go forward), right arrow key (turn to the right), and left arrow key (turn to the left) or the squares of the supervisory system that contain the same commands. To move the character, all that was required to activate the desired direction. Fig. 5 shows the diagram of activities in execution mode of the supervisory system and the communication with the 3D game.

2.1.1.3. Control layer. This layer was very important for the development of the system, because it interacts with the vision layer. In this layer was processed the image captured by the webcam, convert a detected movement into a command (Fig. 6), and record each user configuration.

2.1.2.

3D game

Among the various classes of games, the adventure style was selected for the present study. In adventure games, the player must solve problems to overcome obstacles in a world that was explored by making decisions and observing their consequences. The game followed the story of Beto, a boy who was lost in the Amazon forest after a plane crash; additionally, animals from the Brazilian fauna have been placed in the game. To return to his home, Beto must find a firefighter helicopter; however, to accomplish this, he must follow a trail and face various challenges related to the type of food that each animal of the forest normally eats. If the player could not provide the food that was requested after three attempts, the animal cries and the game continues. If the animal receives food, it performs an animation showing satisfaction and helps the character to overcome a challenge, permitting the character to continue its journey.

2.2.

System implementation

The computer used in this research was an Intel (R) Core (TM) i3 M330 2.13 GHz, 4GB memory and operating system Microsoft Windows 7 64-bit.

49

Supervisory system

In model layer was used in the database Microsoft Access database to store users’ data and the position of the action areas adjusted to their needs. In vision layer were specified three virtual squares (40 × 40 pixels) in the image captured by the webcam (320 × 240 pixel) from the Region and Square classes that bound the action areas of the system. In initiating the supervisory system, the coordinates of each square were loaded with pre-established values, positioning all of the squares in the image captured by the webcam. When a square was dragged to a new position, the supervisory system recalculated the xy coordinates of the four corners of the square and again selected all of the pixels contained in the new action area. This procedure was performed using the inside() method of the Region class, which identified a left mouse click while the pointer was inside one of the action regions. The move() method updated the position data to where the region was dragged. In control layer, the supervisory system was developed in Python. Certain extensions and libraries were used to generate and process the image captured by the webcam, converted a detected movement into a command and record each user configuration. First, to permit integration with the Windows platform, the PyWin32 extension was used. To capture the webcam images, VideoCapture, a Win32 extension of Python, was used. To add processing capacity from the Python interpreter, the PIL library (Python Imaging Library) was used. For graphical programming, the Pygame module provided the API of the SDL library. For communication with the Microsoft Access database, the PyPyODBC module provided the ODBC interface function. Fig. 7 shows the component diagram of the supervisory system and the 3D game. To prevent the processing of the supervisory system from interfering with the performance of the 3D game, its scanning system was limited using the tick() method of the clock function of Pygame to capture 30 frames per second from the webcam. The getdata() method of the PIL library stored two sequential frames (frame1 and frame2) of the webcam image and then compared the action regions in these two images. For this purpose, a sum of the RBG components of one pixel from imagem1 was performed, from which the sum of the RBG components of the pixel in the corresponding position of imagem2 was subtracted (absolute value). If the result was greater than 50, an alteration was considered to have occurred in this pixel. If an alteration was detected a significant in this action area (Test of determination of the movement area and its precision), the supervisory system then sent the ASCII code of the keyboard through the API, using the windll.user32.SendInput command of Pygame. Therefore, the operational system interpreted this information as the activation of a keyboard key. The flowchart of Fig. 8 shows the sequence of events in the supervisory system for movement recognition (alteration in the image of the square) and its actions. Because the Blender software allowed access to the API of Windows, the GetKeyState() method of the win32api library of Blender was used to identify whether any key used in the

50


Fig. 7 – Component diagram. Components used for image capture and processing; components for communication between applications.

game was activated. Thus, the 3D game identified the ASCII code of the keyboard and performs the predetermined action. Therefore, the user can use either the standard keyboard or the adapted peripheral device to control the 3D game.

2.2.2.

Development of the 3D game

To develop the scenarios, characters and objects of the game, the software Blender from Blender Foundation was used because it is free and provides resources for modeling, animation, rendering and post-production. In Blender, the game engine allowed scenes and interactive objects to be created without the need for programming; for example, the detection of a collision between objects was automatic [27]. The interactions in the scenario can be performed using the keyboard or mouse or with other objects and events from the same scene. Fig. 9 shows an example of a challenge in the 3D game.

3.

Evaluation and tests

3.1.

Research subjects

was adopted so that the type and degree of impairment of the final user did not influence the results. The volunteers of both genders, with ages between 23 and 45 years old, were divided into two groups: group A1 was composed of individuals who did not have experience with games, and group A2 included individuals with experience. The groups were divided in this manner to verify whether experience with computer games was an important factor with regard to using the adapted peripheral device as an access device. Thus, it was considered that people with motor impairments are not accustomed to playing. One volunteer (T1) was also selected from the graduate program of the University of Mogi das Cruzes to test the adapted peripheral device. This volunteer was selected due to their large experience and ability in computerized games. Thus, it was evaluated whether, with training, the development of handling for the adapted peripheral device was similar to the development of handling for the standard keyboard. To verify the accessibility of the device, 13 volunteers were also selected (group B) from both genders, with ages less than

For evaluation, 20 volunteers without motor impairments (group A), who were students from the graduate program of the University of Mogi das Cruzes, were selected. This criterion

Fig. 8 – Flowchart. Flowchart indicating the sequence of operations for the detection of movement.

Fig. 9 – 3D game challenge. The Beto character encountering the challenge of associating a requested word with the food available for selection to pass the challenge.


51

Table 1 – The data related to the volunteers from group B. Volunteers

Gender

Age

BV1

Female

10

BV2

Male

9

BV3

Male

7

BV4 BV5

Female Female

8 7

BV6

Male

9

BV7

Female

7

BV8 BV9 BV10

Female Male Male

8 15 10

BV11 BV12 BV13

Male Female Male

15 8 7

Average Standard deviation

– –

9.23 2.77

Diagnosis (ICD-10) Spastic quadriplegic cerebral palsy (G80) and mild mental retardation (F70) Monoplegia of upper limb (G83.2) Burn Neuropatia intercostal (G58.0) and brachial plexus disorders (G54.0) Spastic hemiplegic cerebral palsy (G80.2) Monoplegia of upper limb (G83.2) e retardo mental não especificado (F79) Monoplegia of upper limb (G83.2) and mild mental retardation (F70) Acquired clawhand, clubhand, clawfoot and clubfoot (M21.5) and mild mental retardation (F70) Spastic hemiplegic cerebral palsy (G80.2) Spastic quadriplegic cerebral palsy (G80) Spastic quadriplegic cerebral palsy (G80) and mild mental retardation (F70) Spastic quadriplegic cerebral palsy (G80) Spastic hemiplegic cerebral palsy (G80.2) Spastic quadriplegic cerebral palsy (G80) and moderate mental retardation (F71) – –

16 years, with motor impairments of the upper limbs due to spastic quadriplegia and/or muscular dystrophy, but with movement of the head, elbow, wrist, or sufficient extremities to act voluntarily in a minimum area of 5.5 cm2 . The following exclusion criteria were adopted: non-preserved cognitive function, quadriplegics with a total lack of movement of the upper limbs, or hemiplegics (who could control the game with the preserved movements of one side of the body). (Favorable assent from the ethics committee CAAE – 0140.0.237.000-09.) The specific data from group B are in Table 1.

3.2.

Tests of the supervisory system

To evaluate the performance of the supervisory system to process the images captured by the webcam and transform these images into the desired commands in the 3D game, four tests were performed to measure the following quantities: • The time interval between detection of a movement and the recognition of a command in 3D game To perform this first test, a routine was implemented both in the supervisory system and in the 3D game. This routine’s role was to write into a log file the time of the operating system (date, hour, minute, second and microsecond). At the supervisory system, this routine was called after detecting a movement in one of the regions of operation. The 3D game performed this routine when receiving command. In this routine, the method datetime.now() in datetime library was used to capture the current time. A volunteer TI was asked to push the three regions individually adapted with the peripheral. This test was repeated 10 times. • Determination of the movement area and its precision

Fig. 10 – Image grid. Image grid used to calculate the area of action region area and movement accuracy.

To perform this test an image grid (Fig. 10) was used. With divisions 3.75 mm × 3.75 mm, the drawing was placed on the table. A webcam positioned at a distance of 45 cm recorded the squared image in a supervisory system. To determine the area required to cause a change in the action region was asked to volunteer T1 to make horizontal and vertical movements on the image grid. To determine the detection accuracy was asked to volunteer T1 repeat the same move 50 times. To verify if all movements were detected by supervisory system, a routine count how many changes was detected in the action area. • The time interval required to execute two successive commands To accomplish this test, a routine was added to the code of the 3D game that measured the time required for Blender

52


Fig. 11 – The pathway for the virtual path. The pathway designed to test the sensitivity of the adapted peripheral device. The user attempted to traverse the pathway in the shortest possible time, with minimal collisions with the sides of the path.

to perform two successive commands. The volunteer T1 was asked to keep a keyboard key triggered for 5 s. Subsequently, the volunteer T1 used the adapted peripheral device and kept his/her finger in the action region for 5 s. This test was repeated 10 times. • The sensitivity of the peripheral device Among the parameters used to evaluate the performance and comfort of users of pointing devices, the most important consideration is sensitivity. Devices with greater sensitivity will result in fewer errors in the performance of a task. To evaluate the sensitivity of the peripheral device developed in this study and determine whether this device allows a user to execute all of the actions available in the 3D game examined in this work, we developed a virtual path (Fig. 11) consisting of straight and curved segments bounded by walls. This path must be wandered with minimal collisions, using the commands that cause the in-game character to move forward, turn left, or turn right. In this test, a volunteer with good virtual gaming skills (volunteer T1) was asked to navigate the virtual path using a standard keyboard and the developed peripheral device. The volunteer T1 trained until he believed that he had adapted to the peripheral device. Then the volunteer used each of the two input devices (the developed device and the standard keyboard), and a software was used to record the number of collisions with the walls and the time required to navigate from the beginning to the end of the path.

3.3.

Evaluation of the 3D game

The 20 volunteers from group A tested the accessibility and usability of the 3D game using two peripherals (the adapted developed device and the standard keyboard). After the presentation of the storyline of the game and its goals, each volunteer played the game twice, using a different peripheral device each time. The order of use of the peripherals was

selected randomly. None of the volunteers were familiar with the game, and no pre-test was allowed. After each test, each volunteer was asked to fill out a questionnaire, giving their opinion regarding accessibility and usability. The 13 volunteers with motor limitations (group B) also tested the accessibility and usability of the game; however, they only used the adapted peripheral device for computer access. The tests with group B were performed in a room at the AVAPE (Associação para Valorização e Promoção de Excepcionais) clinical unit, with modified furniture and with an absence of external stimuli and no auditory and/or visual interference. Before each test, the preserved movements of the upper limbs that could be used for the tests were evaluated by a therapist. Each volunteer was positioned, seated at a 90◦ angle at the hips with the knees maintained at an average distance of 50 cm from the webcam. When necessary, lateral supports were used for greater stability. The assessment of each volunteer was provided by the AVAPE professionals themselves. Before each test, a specific calibration of the webcam was performed for each volunteer. The calibrations consist of asking the volunteer to activate the three squares available on the control window one at a time. The squares are moved in the window according to the preserved range of movements (without effort) of the volunteer, with the possibility of different arrangements in the window. The distance from the webcam was also noted: if the volunteer had a large range of motion, the webcam was moved away from the user to increase the visualization area. Alternatively, if the volunteer had a small range of motion, the webcam was placed closer to the user to reduce the visualization area and focus on the preserved movements. An explanation of the 3D game was presented, and each volunteer was asked to access it (the volunteers were also asked whether they wanted to play). During the tests, the occupational therapist that had been following the development of the clinical status of each volunteer was present and involved. After each test, the professional was asked to evaluate the adapted peripheral device accessibility and the satisfaction of each volunteer with the 3D game.

3.4.

Data analysis

For analysis of the sensitivity test of the adapted peripheral device, the D’Agostino statistical test [28] was used to analyze whether the data have a normal distribution, and the Wilcoxon test [29] was used to analyze whether there was a significant difference between the times of the volunteers using the standard keyboard and the adapted peripheral. The statistical tests were also used to analyze the collisions in the virtual course and determine whether the volunteer with training time could achieve the same precision of movements with the peripherals. For the test of the 3D game with group A, we used the D’Agostino test again as well as the ANOVA test – a criterion [29] to analyze whether there was a significant difference between the times of volunteers using the standard keyboard and the adapted peripheral device and whether there was a difference between groups A1 and A2. These tests were also used to compare the time of use between groups A and B to

53


Table 2 – The lengths of time that volunteer T1 required to completely traverse the virtual pathway.

Table 3 – The time required by the volunteers from group A1 to finish the 3D game with two peripherals.

Trials

Volunteers from group A1 who are not accustomed to play

Keyboard Time (min)

Collisions

Adapted peripheral device Time (min)

Collisions

Usage time (min) Keyboard


1 2 3 4 5 6 7 8 9 10

01:55 02:00 02:12 01:52 01:49 01:48 01:42 01:44 01:50 01:45

12 15 9 11 17 15 19 16 14 13

04:15 02:19 03:32 02:08 01:54 01:53 02:00 01:35 01:56 01:40

50 51 29 42 46 49 57 58 47 46

A1V1 A1V2 A1V3 A1V4 A1V5 A1V6 A1V7 A1V8 A1V9 A1V10

03:06 03:47 06:13 03:15 04:15 03:00 06:35 03:31 03:31 02:36

05:00 04:54 07:43 06:07 06:55 05:24 07:40 04:46 06:45 05:25

Average

01:52

14

02:19

47

Average Standard deviation (SD)

03:59 01:21

06:04 01:08

analyze whether the adapted peripheral device is influenced by the ability or deficiency of the volunteer.

4.

Results

The results are based on the game records, the responses of the volunteers to the questionnaire and the observations of the children.

4.1.

The supervisory system

The average interval between movement detection and command recognition in the 3D game was 27 ms, with a standard deviation of 8 ms. For the system to detect a change in the action region it was necessary to change the area in 37.24%, i.e.; 10 cm2 . In the precision test with the volunteer repeating the same move 50 times, the supervisory system detected all the user’s movements. For commands triggered by either the keyboard or the adapted peripheral device, the average response time between two successive commands was 16 ms, without significant differences in this response time for the tested input methods. This time corresponds to the scan cycle for the Blender software; that is, the time that the supervisory system required to process an image captured by the webcam was less than or equal to the length of the software’s scan cycle. Therefore, the processing procedure did not influence the response times for commands in the 3D game. In tests of the sensitivity of the adapted peripheral device, volunteer T1 underwent five iterations of training with the virtual pathway before he deemed himself to be adapted to the peripheral developed for this study. The results of ten trials, including the average times required to traverse the pathway and the numbers of collisions with the sides of the path, are indicated in Table 2. The evolution of the volunteer’s ability to utilize the adapted peripheral device can be observed; in particular, his performance with the device became increasingly similar to his performance with a standard keyboard.

4.2. Tests of the 3D game with volunteers from groups A1 and A2 The 3D game developed for the present study was used by the 20 volunteers without motor limitations (A1 and A2), based on the use of a standard keyboard and adapted peripheral device. In Table 3 and Fig. 12, the usage times for each peripheral device are presented for the volunteers from group A1 (the volunteers who were not accustomed to playing games). The times required by the 10 volunteers from group A2 to finish the 3D game using the two different peripherals are shown in Table 4 and Fig. 13. The D’Agostino test found that the samples from the A1 and A2 groups showed a normal data distribution (significance level of ˛ = 0.05, p = ns). The one-way ANOVA and the Tukey’s post hoc test applied to the time averages in the 3D game showed that the A1 and A2 groups were not significantly different when they used the same peripheral device (significance level of ˛ = 0.01, p = ns); however, there was a significant difference between the times spent using each peripheral device (significance level of ˛ = 0.01, p < 0.01).

Table 4 – The times of the group A2 in the 3D game. Volunteers from group A2 who are accustomed to play

Usage time (min) Keyboard


A2V1 A2V2 A2V3 A2V4 A2V5 A2V6 A2V7 A2V8 A2V9 A2V10

03:11 03:43 02:56 03:43 03:04 02:34 02:21 02:47 02:40 04:04

07:33 05:45 06:10 06:00 05:46 03:54 04:18 04:41 04:22 05:05

Average Standard deviation (SD)

03:06 00:34

05:21 01:06

54


Fig. 12 – The lengths of time that volunteers from group A1 required to complete the 3D game with each input device. This figure indicates the length of time that each volunteer from group A1 (which consisted of individuals who do not habitually play computer games) required to finish the 3D game and the order in which each volunteer used the two access devices (orange for a keyboard; green for the adapted peripheral device). (For interpretation of the references to color in this figure legend, the reader is referred to the web version of the article.)

After playing the game twice, the volunteers filled out a questionnaire with questions referring to playability, usability and satisfaction. The analyses of the responses from the questionnaire show that the majority gave a positive score to all aspects. The most significant difference was in enjoyment: 50% of the people who are not accustomed to playing games considered the game to be “fun,” while 70% of those that play games considered the game to be “extremely fun.” This difference likely occurs because the former group does not find computer games to be fun in general. Regarding accessibility, the responses from groups A1 and A2 showed that the adapted peripheral device was compatible with the study aim, in other words, to provide the children with motor limitations equality in leisure, allowing them to control the actions of the character in the 3D environment through a webcam.

4.3. Accessibility tests with volunteers from group B using the 3D game and peripheral device The times required by these volunteers to finish the 3D game using the adapted peripheral device are shown in Table 5. Fig. 14 shows a volunteer from group B accessing the 3D game through the adapted peripheral device, moving a hand placed on the table, which was his preferred situation. Applying the ANOVA test: one criterion for comparing the times of group B using the adapted peripheral device with the times of groups A1 and A2 also using the adapted peripheral

device. There was no significant difference between groups A1 and B (significance level of ˛ = 0.01, p = ns), proving that motor impairment did not affect the performance of the volunteers when they used the system developed in this study. However, there was a significant difference between groups A2 and B (significance level of ˛ = 0.01, p < 0.05), demonstrating that the ability of volunteers with computerized games influenced the result.

Table 5 – The length of time required to finish the 3D game with the adaptive peripheral device. Volunteers

Adaptive peripheral device Time (min)

BV1 BV2 BV3 BV4 BV5 BV6 BV7 BV8 BV9 BV10 BV11 BV12 BV13

05:24 04:46 05:14 08:39 10:03 04:36 05:24 06:16 08:54 07:59 06:52 09:24 07:46

Average

07:01

55


Fig. 13 – The lengths of time that volunteers from group A2 required to complete the 3D game with each input device. This figure indicates the length of time that each volunteer from group A2 (which consisted of individuals who habitually play computer games) required to finish the 3D game and the order in which each volunteer used the two access devices (sequence of colors, orange for a keyboard; green for the adaptive peripheral device). (For interpretation of the references to color in this figure legend, the reader is referred to the web version of the article.)

Table 6 shows the responses of professionals in relation to the accessibility of the adapted peripheral device and the satisfaction of the volunteers with the 3D game.

4.4. Qualitative evaluation of the 3D game and the adapted peripheral device The professionals considered the game to be satisfactory (average score of 9 for satisfaction) and fully accessible for

all users, noting that the adapted peripheral device enabled the efficient use of the remaining movements of the volunteers. The interest of the volunteers in the computer game and the adapted peripheral device also was apparent. All of the volunteers went to the test room excited by the opportunity to use the computer. Before performing the test, four volunteers asked how they would access the 3D game, because their limitations. After the test, five volunteers wanted to know if

Table 6 – The evaluations of volunteer accessibility and satisfaction, performed by the healthcare professionals. Volunteers

Fig. 14 – Volunteer from group B accessing the 3D game with the adapted peripheral device. Volunteer with cerebral palsy moving his hand on the table to access the 3D game with the adapted peripheral device.

BV1 BV2 BV3 BV4 BV5 BV6 BV7 BV8 BV9 BV10 BV11 BV12 BV13 Average

Adapted peripheral device Accessibility

Satisfaction (0–10)

Complete Complete Complete Complete Complete Complete Complete Complete Complete Complete Complete Complete Complete

10 9 8 7 10 10 10 9 9 8 9 10 8

–

9

3D game

56


they could play again, and three showed interest in having the game and the adapted peripheral device at home. Eight volunteers asked if the tests would be performed for more days, showing interest in continuing to access the 3D game.

5.

Discussion

All volunteers from groups A1 and A2 easily performed all ingame actions with both a standard keyboard and the adapted peripheral device. All volunteers in group B were able to trigger all in-game actions with the adapted peripheral device, indicating the accessibility of the system developed in this study. The tests of the supervisory system showed that the action times between the two commands, through the keyboard or adapted peripheral device, are 16 ms. Therefore, whether the command is executed using the standard keyboard or using the adapted peripheral device, the interval between one action and another is the same. This time interval corresponds to the scanning cycle of the Blender software, which is the sampling interval. Because the processing time of the supervisory system is less than this interval, there is no delay in the performance of the adapted peripheral device. In using the 3D game, volunteers from group A1 spent 52.26% more time with the adapted peripheral device than with the keyboard. For volunteers from group A2, the time used was 72.8% greater with the adapted peripheral device. The time spent with the adapted peripheral device, however, was similar for the two groups, with the difference between the two groups being primarily in the keyboard time. The volunteers who are accustomed to playing (A2) were faster than the others (A1) when using the keyboard. Therefore, it is suspected that the lack of familiarity with the equipment is a relevant factor in the time spent with the adapted peripheral. Therefore, we requested that one volunteer T1 be previously trained with the adapted peripheral device. After a few minutes of training, he/she achieved similar performance with the two control systems. But the delay in finishing the game as well as the limitations of the adapted peripheral did not affect the interest of the users. The responses of the volunteers to the questionnaire showed that 70% considered the story quality to be good and the game to be extremely fun or very fun. In fact, no user complained about the time required to use the adapted peripheral device. The motion-based control of the in-game character was regarded as a recreational activity by all users. This perception primarily arose because the in-game character could be controlled by performing any movement within the action area. This versatility allowed the in-game character’s actions to be triggered by the preserved movements of users with severe motor limitations. Even users whose preserved movements included only a few motions of the fingers or head could easily perform all of the commands required to move the character and finish the game. In addition, the developed peripheral device does not cause discomfort or embarrassment because it does not require the placement of an apparatus on the user. An analysis of currently available commercial computer games reveals that in these games, characters’ movements and actions must be performed by entering sequences or

combinations of commands. Moreover, in most of these games, commands must be rapidly executed to ensure that the desired in-game actions occur. In addition, in certain strategy games or role-playing games (RPGs), a complex system of menus and submenus is utilized to perform various movements of in-game characters. These commands must be input using standard devices, such as keyboards or joysticks. In the developed system, click and drag actions, used to grab and drag objects, were replaced by predetermined animations from the selection (pressed key) of the object. The proximity of the keys on a standard keyboard can prevent the precise triggering of a specific key. As a result, the sequences of commands required for a game may not be input correctly. Standard joysticks are also unsuitable for individuals with physical limitations because these joysticks must be firmly gripped with both hands to ensure that directional input can be manipulated and triggered with the necessary skill and precision. Prior scientific work has described other adapted peripheral devices with webcams [9,15,16] that allowed individuals with limited motion to access virtual environments. However, no plots or scenarios were developed to help users immerse themselves in these environments because researchers’ objectives involved the rehabilitation of individuals who had experienced cerebrovascular accidents or the analysis of users’ movements. In addition, previously developed access systems were designed to capture users’ broad motions but did not consider the issue of precision. The new motion detection techniques used by Wii and Kinect devices have recently been incorporated into virtual environments and games. Certain authors [18] have developed a Kinect-based system for physical rehabilitation. Another group of researchers [19] has used a Wii as an input device for recreational activities. However, the system developed by these researchers was intended for users without physical limitations and requires precision in the activation of console keys; therefore, this system is not suited for use by children with special needs. Other researchers [3], in an effort to aid the therapist in adapting rehabilitation exercises to the characteristics of the patients, developed a customized interface. The game commands, however, are performed using broad movements, captured by the Kinect, which are not appropriate for the leisure of children with severe motor impairments. The adapted peripheral device developed in our study was designed to be adjustable to users with various types of motor impairment. This device was designed to be triggered by small movements of the fingers, wrists, or head, but it can be adapted to poorly coordinated movements by simply increasing the distance between the action field squares. This increased distance ensures that users with little control of their movements can trigger a single action area at a time when inputting a command. Therefore, an adapted peripheral device that incorporates the concepts of universal design has been created. Relevantly, the game-playing performance of volunteers with severe motor limitations was similar to the performance of volunteers from group A. Thus, relative to the volunteers in group A, these individuals with motor limitations could use a peripheral device adapted to any motor impairment


to derive equal game-playing pleasure without experiencing discomfort or embarrassment. Therefore, in future studies, this device will permit the development of virtual games in which children with motor limitations and children without these limitations can equitably share enjoyable and educational experiences. The initial period of the game is critical, as it is the time in which the player decides whether he/she will continue to play [30]. Therefore, the game begins with an impactful scene of Beto in the forest next to a burning plane. The developed game has a simple menu to avoid users being lost or confused as can occur with complex menus with various levels. To move the character in the scene, the user has the options of walking forward or turning to the right or left, making the character control system simple.

6.

Conclusion

The technological advances of recent years, with the invention of the computer, Internet and other multimedia features, provide users with fun and knowledge. However, children with motor difficulties still need access to resources with similar quality to that given to the child without movement difficulty. The system developed in this research provided an activity without challenging the time axis as occurred in most commercial games, but which users consider fun. With some adjustments to the developed system, it may provide, in addition to leisure, access for people with mobility limitations a serious game. So it enables people to learn diverse contents, contributing to social and educational inclusion. The developed system provided access to a computer game accessible through easy-to-handle, financially affordable appropriate hardware and software, available to anyone.

Authors’ contributions TAS and AFF participated in the concept and development of the game. TAS and AFF also participated in the acquisition, analysis, and interpretation of the data. All authors revised and approved the current version of the manuscript.

Acknowledgements We are grateful to the FAEP (Fundação de Amparo a Ensino e Pesquisa) from University of Mogi das Cruzes and CAPES (Coordination for the Improvement of Higher Education Personnel) for the financial support.

references

[1] L.S. Vygotsky, Mind in Society: The Development of Higher Psychological Processes, 14th edition, Harvard University Press, Cambridge, Massachusetts., 1980. [2] P. Barr, J. Noble, R. Biddle, Video game values: human–computer interaction and games, Interact. Comput. 19 (2007) 180–195.

57

[3] D. Avola, M. Spezialetti, G. Placidi, Design of an efficient framework for fast prototyping of customized human–computer interfaces and virtual environments for rehabilitation, Comput. Methods Programs Biomed. 110 (2013) 490–502. [4] D. Koro, A. Holobar, M. Divjak, D. Zazula, Building interactive virtual environments for simulated training in medicine using VRML and Java/JavaScript, Comput. Methods Programs Biomed. 80 (2005) S61–S70. [5] L.T. Kawamoto Júnior, A.F.F. Slaets, Software to Train Scuba Dive Procedures. Advanced Materials Research, Trans Tech Publications Ltd., Zurich, 2013. [6] R.M.E.M. Costa, L.A.V. Carvalho, The acceptance of virtual reality devices for cognitive rehabilitation: a report of positive results with schizophrenia, Comput. Methods Programs Biomed. 73 (2004) 173–182. [7] A.S. Silva, A.F. Frère, Virtual environment to quantify the influence of colour stimuli on the performance of tasks requiring attention, BioMed. Eng. Online 74 (2011), n. 10. [8] R. Raisamo, S. Patomäki, M. Hasu, V. Pasto, Design and evaluation of a tactile memory game for visually impaired children, Interact. Comput. 19 (2007) 196–205. [9] T. Chuang, W. Huang, S. Chiang, Y. Tsai, J. Doong, H. Cheng, A virtual reality-based system for hand function analysis, Comput. Methods Programs Biomed. 69 (2002) 189–196. [10] S.R.M.S. Boschi, A.F. Frère, Grip and pinch capability assessment system for children, Med. Eng. Phys. 35 (2013) 626–635. [11] B.S. Lange, P. Requejo, S.M. Flynn, A.A. Rizzo, F.J. Valero-Cuevas, L. Baker, C. Winstein, The potential of virtual reality and gaming to assist successful aging with disability, Phys. Med. Rehabil. Clin. N. Am. 21 (2) (2010) 339–356. [12] W. Barendregt, M.M. Bekker, D.G. Bouwhuis, E. Baauw, Identifying usability and fun problems in a computer game during first use and after some practice, Int. J. Hum-Comput. Stud. 64 (2006) 830–846. [13] M. Virvou, G. Katsionis, On the usability and likeability of virtual reality games for education: the case of VR-ENGAGE, Comput. Educ. 50 (2006) 154–178. [14] M. Betke, J. Gips, P. Fleming, The camera mouse: visual tracking of body features to provide computer access for people whit severe disabilities, IEEE Trans. Rehabil. Eng. 10 (1) (2002). [15] K. Eng, E. Siekierka, P. Pyk, E. Chevrier, Y. Hauser, M. Cameirao, L. Holper, K. Hägni, L. Zimmerli, A. Duff, C. Schuster, C. Bassetti, P. Verschure, D. Kiper, Interactive visuo-motor therapy system for stroke rehabilitation, Med. Biol. Eng. Comput. 45 (9) (2007) 901–907. [16] M.S. Cameirão, S.B. Badia, E.D. Oller, P.F.M.J. Verschure, Neurorehabilitation using the virtual reality based Rehabilitation Gaming System: methodology, design, psychometrics, usability and validation, J. NeuroEng. Rehabil. 48 (7) (2010). [17] H. Sun, W. Cheng, The input-interface of Webcam applied in 3D virtual reality systems, Comput. Educ. 53 (2009) 1231–1240. [18] Y. Chang, S. Chen, J.A. Huang, Kinect-based system for physical rehabilitation: a pilot study for young adults with motor disabilities, Res. Dev. Disabilities 32 (6) (2011) 2566–2570. [19] J.C. Lee, Hacking the Nintendo Wii remote, IEEE Pervasive Comput. 7 (3) (2008) 39–45. [20] L.C. Ebert, P.M. Flach, M.J. Thali, S. Ross, Out of touch – a plugin for controlling Osiri X with gestures using the leap controller, J. Forensic Radiol. Imag. 2 (3) (2014) 126–128. [21] H. Mousavi Hondori, M. Khademi, l. Dodakian, S.C. Cramer, C.V. Lopes, A spatial augmented reality rehab system for

58

[22]

[23]

[24]

[25]


post-stroke hand rehabilitation, Med. Meets Virtual Reality 20 (2013). G. Placidi, D. Avola, D. Iacoviello, L. Cinque, Overall design and implementation of the virtual glove, Comput. Biol. Med. 43 (2013) 1927–1940. ˇ c, ˇ J. Sodnik, Evaluation of leap G. Jakus, J. Guna, S. Tomazi motion controller with a high precision optical tracking system, human–computer interaction, Adv. Interaction Modalities Tech. 8511 (2014) 254–263. S.B. Moro, S. Bisconti, M. Muthalib, M. Spezialetti, S. Cutini, M. Ferrari, G. Placidi, V. Quaresima, A semi-immersive virtual reality incremental swing balance task activates prefrontal cortex: a functional near-infrared spectroscopy study, NeuroImage 85 (2014) 451–460. M. Ferrari, S. Bisconti, M. Spezialetti, S.B. Moro, C. Di Palo, G. Placidi, V. Quaresima, Prefrontal cortex activated bilaterally by a tilt board balance task: a functional near-infrared

[26]

[27] [28] [29] [30]

spectroscopy study in a semi-immersive virtual reality environment, Brain Topogr. 27 (3) (2013) 353–365. L. Holper, T. Muehlemann, F. Scholkmann, K. Eng, D. Kiper, M. Wolf, Testing the potential of a virtual reality neurorehabilitation system during performance of observation, imagery and imitation of motor actions recorded by wireless functional nearinfrared spectroscopy (fNIRS), J. NeuroEng. Rehabil. 7 (57) (2010). BLENDER, in http://www.blender.org (2013). T. Colton, Statistic in Medicine, Little, Brown and Company, New York, 1974. B. Dawson, R.G. Trapp, Basic & Clinical Biostatistics, 4th edition, Lange Medical Books-McGraw-Hill, New York., 2004. J.P. Davis, K. Steury, R. Pagulayan, A survey method for assessing perceptions of a game: the consumer playtest in game design, Int. J. Comput. Game Res. 5 (1) (2005).

Computer access for people with motor disabilities.

Evaluation of an automated blood pressure measuring device intended for general public use.

Qualitative evaluation of a pain intensity screen for children with severe neurodevelopmental disabilities.

Computer-based reading and spelling practice for children with learning disabilities.

Hearing-impaired children with associated disabilities: a team evaluation.

Evaluating 'FREDA Challenge': a coproduced human rights board game in services for people with intellectual disabilities.

Obesity Prevention for Children with Developmental Disabilities.

Care needs of children with disabilities - Use of the Pediatric Evaluation of Disability Inventory.

Integrational deficits in children with visual-perceptual-motor disabilities.

An Evaluation of the Design and Usability of a Novel Robotic Bilateral Arm Rehabilitation Device for Patients with Stroke.

A randomized controlled trial of the computer-based cognitive rehabilitation program for children (CoTras-C) to examine cognitive function and visual perception in children with developmental disabilities.

Out-of-home placement for children and adolescents with disabilities.

Are individuals with disabilities and their families "resilient"? Deconstructing and recasting a well-intended concept.

The meaning of leisure for children and young people with physical disabilities: a systematic evidence synthesis.

Design and Implementation of Foot-Mounted Inertial Sensor Based Wearable Electronic Device for Game Play Application.

A qualitative study: Barriers and support for participation for children with disabilities.

Preoperative, computer simulation-based, three-dimensional corrective osteotomy for cubitus varus deformity with use of a custom-designed surgical device.

Design and implementation of Metta, a metasearch engine for biomedical literature retrieval intended for systematic reviewers.

Design and evaluation of a device for fast multispectral time-resolved fluorescence spectroscopy and imaging.

Preclinical evaluation of investigational radiopharmaceutical RISAD-P intended for use as a diagnostic and molecular radiotherapy agent for prostate cancer.

Design and preliminary evaluation of the FINGER rehabilitation robot: controlling challenge and quantifying finger individuation during musical computer game play.

Financial counseling for families of children with chronic disabilities.

Developing a Novel Speech Intervention iPad Game for Children With Cleft Palate: A Pilot Study.

A detachable electronic device for use with a long white cane to assist with mobility.