Research in Developmental Disabilities 37 (2015) 1–8
Contents lists available at ScienceDirect
Research in Developmental Disabilities
Interactive augmented reality using Scratch 2.0 to improve physical activities for children with developmental disabilities Chien-Yu Lin a,*, Yu-Ming Chang b a
Department of Special Education, National University of Tainan, 33, Section 2, Shu-Lin Street, Tainan 700, Taiwan College of Digital Design, Southern Taiwan University of Science and Technology, No. 1, Nantai Street, Yongkang Dist., Tainan City 710, Taiwan
b
A R T I C L E I N F O
A B S T R A C T
Article history: Received 9 October 2014 Accepted 10 October 2014 Available online
This study uses a body motion interactive game developed in Scratch 2.0 to enhance the body strength of children with disabilities. Scratch 2.0, using an augmented-reality function on a program platform, creates real world and virtual reality displays at the same time. This study uses a webcam integration that tracks movements and allows participants to interact physically with the project, to enhance the motivation of children with developmental disabilities to perform physical activities. This study follows a single-case research using an ABAB structure, in which A is the baseline and B is the intervention. The experimental period was 2 months. The experimental results demonstrated that the scores for 3 children with developmental disabilities increased considerably during the intervention phrases. The developmental applications of these results are also discussed. ß 2014 Elsevier Ltd. All rights reserved.
Keywords: Disabilities Scratch 2.0 Augmented-reality Webcam
1. Introduction These recent changes in Information and Communication Technologies (ICTs) have changed society, influencing the method people relate, communicate, work and learn (Simo˜es, Redondo, & Vilas, 2013), real-time interactive multimedia is more popular and affordable (Hwang, 2012), thus, allowing for the use of technology for special needs (Kagohara, Sigafoos, Achmadi, O’Reilly, & Lancioni, 2012). However, one of the challenges in human–computer interactions is to design systems that are not only usable but also appealing to users (Bonnardel, Piolat, & Bigot, 2011). Augmented reality (AR), as an emerging interactive technology, has increasingly attracted public interest during the last few years (Olsson, Ka¨rkka¨inen, Lagerstam, & Venta¨-Olkkonen, 2012) (Radu & MacIntyre, 2009). AR technology enables the merging of virtual objects with real objects, resulting in augmented reality environments (Trojan et al., 2013), and can be used for a live direct or indirect view of a physical real-world environment, the elements of which are augmented by computer-generated sensory inputs, such as sound or graphics (Chang, Kang, & Huang, 2013). Because in augmented reality environments both virtual and real objects can co-exist and interact in real time, the augmented reality applications are broad and the number of applications is increasing rapidly (Solari, Chessa, Garibotti, & Sabatini, 2012). The main advantages of AR applications used across different fields have been widely discussed in the literature, such as in education (Wojciechowski & Cellary, 2013), textiles (Harris,
* Corresponding author at: 33, Section 2, Shu-Lin Street, Tainan 700, Taiwan. Tel.: +886 62133111x744. E-mail address: [email protected] (C.-Y. Lin). http://dx.doi.org/10.1016/j.ridd.2014.10.016 0891-4222/ß 2014 Elsevier Ltd. All rights reserved.
2
C.-Y. Lin, Y.-M. Chang / Research in Developmental Disabilities 37 (2015) 1–8
2013) surgical interventions (Volonte´ et al., 2011), games, home-training system, online teaching (Andujar, Mejı´as, & Marquez, 2011),and learning disabilities(Chang et al., 2013). Augmented reality can lower the barrier to entry for students engaging in virtual content. The ease of interaction within AR-based experiences has already been shown by the use of virtual experiences in early school years (Bujak et al., 2013). People with developmental disabilities experience limitations in fine motor control, strength, and range of motion, which can reduce their participation in community and leisure activities (Chang, Chen, & Huang, 2011). Custom made alternative devices for those with special needs are expensive and the low unit turnover makes the prospect unattractive to potential manufacturers (Standen, Camm, Battersby, Brown, & Harrison, 2011). Most leisure pastimes available for people with severe physical limitations are often extremely limited (Lotan, Yalon-Chamovitz, & Weiss, 2009), so are not useful for people with developmental disabilities. Art games on console systems such as the Microsoft Xbox, Nintendo Wii or Sony PlayStation are not only for fun, but have also been applied in rehabilitation treatments, and therapeutic systems (Ding et al., 2013). Recently, many open source activities have become available, which share their programs, so users can design their own custom interfaces. This study uses Scratch 2.0, which is a visual programming environment designed at the MIT Media Lab (Resnick, 2012; Resnick & Rosenbaum, 2013) and which includes costumes and scripts. Scratch Web site (http://scratch.mit.edu) create a free online interactive community, with people sharing, discussing their programs, most important that could be designing, creating, and remixing one another’s projects (Resnick et al., 2009a,b). Programming is done by inserting command blocks which form scripts that control the interactive interface. Various researchers and designers have imagined that Augmented Reality technology could be well suited for children (Radu & MacIntyre, 2009). In Scratch 2.0, video sensing has been added, and there are new camera programing blocks that allow users to create projects that react to movements in the physical world through the use of a webcam (Carini, 2012). In other words, augmented reality could be applied to this interface (Massachusetts Institute of Technology, 2013). Using the webcam effect design in Scratch 2.0 (Radu & MacIntyre, 2009), the virtual objects and related sounds appear when the webcam detects movement, thus allowing for augmented reality. Users are able to now use their webcam to interact with projects and can also create their own programming blocks in Scratch 2.0. The advantage of this study is that is it low-cost, as only a webcam is needed for children with different needs to extend their activities. 2. Materials and methods 2.1. Participants There were 3 participants in this study, all of them children with different developmental difficulties. Prior to the study, we obtained formal consent from their parents. This study designed individual physical activities for the different needs of the participants to enhance their body motion motivation. Cindy, a 4-year 1-month-old female has developmental disabilities. She is able to walk by herself, but she walks slowly and unevenly. She is mildly intellectually retarded and seldom talks, she could understands basic oral instructions, the oral ability belongs to basic expression with tardy. In this case, we wanted to train Cindy’s body movement abilities by stimulating her feet using an AR interactive game, which was similar to step training. Kitty, the second participant, is a 6-year-old girl, who has severe cerebral palsy. She belongs to no oral expression. Because of her condition she is unable to stand by herself and all her limbs involuntarily twitch, so she has to sit on a customized Kinder chair and be strapped into her H-harness to remain stable. As she has weak leg movements, she is always sitting and lacks the motivation to lift her legs. In this case, we wanted to train her leg muscles and give her the motivation to move her legs and feet. John, our third participant, is a 3-year 11-month-old boy, who has moderate multiple disabilities, weak legs and low vision. He is able to climb but is not able to stand or walk by himself, so he is always sitting, he could understands basic oral instructions and he could express what he want to do. He is able to lift his legs and feet, but lacks the initiative to strengthen them. 2.2. Apparatus, material and setting The interactive feedback was designed using Scratch 2.0 software. The original design was a video pop balloon game, from an original program by Christine Garrity. With the Scratch 2.0 added webcam support, she designed an intuitive interactive game. Using a webcam, users are able to see themselves and the balloons on the screen. As the balloon changes position, the user uses their hands (or any part of their body) to move in the air. When the body movement is in the sensor area (in the original design it is in balloon area), the screen shows a punctured balloon. The game’s design is similar to the Xbox, and is fit for normal children. However, in this study pretest, as the participants all had developmental disabilities, the knowledge required exceeded their cognitive load, so at the beginning, the participants always approached the screen with the intention of touching the balloon. Therefore, this study revised and remix the program, that make the virtual stimulus object always appear in the same position, which overlap the real object on the screen, when the participant touch the real object, it’s means also touch the virtual stimulus object, and then the participant could got the feedback, so it could be used for special needs participants. To do this, we put a real brick on the floor, and focused the external webcam toward the real brick, making
C.-Y. Lin, Y.-M. Chang / Research in Developmental Disabilities 37 (2015) 1–8
3
sure that the sensor area overlapped the brick. When the participant touched the sensor area, dynamic pictures and the related sounds were shown. Then, after asking for the parents and teachers’ suggestions, we remixed the original design, and used cartoon characters as the basic images to design the dynamic images and related sounds as feedback. The concept of the study is shown in Fig. 1. We used the webcam to detect the body movement on the screen. The goals for each participant were adjusted for each of their different disabilities. Cindy can walk but is not stable, so this study trained her to step up onto the brick. Kitty and John have physical disabilities, so are unable to stand by themselves, so the study required them to sit and lift their feet up 7 cm onto the brick’s surface. 2.3. General procedure This study focused on how the effect of the webcam and the interactive computer game can assist people with physical disabilities. The computer screen was arranged in front of the participants, and two speakers were put on the floor. When the participants did the correct foot lifting and movement, they received the dynamic pictures and sounds as feedback. Fig. 2 shows the experimental setup. In the interactive effect design, this study used Scratch 2.0 from the Media Laboratory at MIT as the software, which allows for the development of new technologies to engage people in creative learning experiences (Brennan & Resnick, 2013). Using this Scratch software, the designer can create their own interactive stories, games, and animations (MonroyHerna´ndez & Resnick, 2008). This study examined how to use an external webcam to detect movement and give real-time feedback. At the beginning, a sensor area is shown on the screen, and, in the original design, when the participant moves their body; the webcam detects the movement when they move into the sensor area. If this is done correctly, feedback, which includes dynamic pictures and the related sound, is shown on the screen. Because our participants were children with physical disabilities, their response time was slower than for normal people, so the operating processing time was set to 90 s. The input interface is a low-cost device with a computer. This research adopted Scratch software technology (Brennan, Resnick, & Monroy-Hernandez, 2010; Resnick et al., 2009a,b). The experimental design adopted an ABAB reversal design for single-case research is a tool in special education and assistive technology (Neely, Rispoli, Camargo, Davis, & Boles, 2013; Shih, Shih, & Luo, 2013), in which A (baseline phase) was followed by B (intervention phase), a return to baseline phase, and then a final intervention phase. The A represented baseline phases while the B represented intervention phases with the webcam and Scratch 2.0 system. The experiment was divided into four phases: Baseline 1, Intervention 1, Baseline 2, and Intervention 2. Single-case design is a research method involving deliberate assignment of different conditions to the same individual and measurement of one or more outcomes over time (Hedges, Pustejovsky, & Shadish, 2013). Cohen (1988) offered a large (>0.35), medium (0.15–0.35), or small effect size (0.02–0.15), an effect size for predictive regression equations that could indicate the actual effect. The data was collected over 2 months, with 2–3 sessions per day, 3–5 days a week, with each session lasting 90 s, during which we collected the data points. In the first phase, Baseline 1 (A1), we collected 9 data points. In the second phase, Intervention 1 (B1), the intervention used an external webcam and the Scratch 2.0 system, from which we collected 18 data points. In the third phrase, Baseline 2 (A2), from which we collected 9 data points, the laptop, external webcam and AR contents were withdrawn. In the fourth phase, intervention 2 (B2), the intervention setup was the same as for B1, from which we collected 18 data points.
[(Fig._1)TD$IG]
Fig. 1. the concept of the study.
[(Fig._2)TD$IG]
4
C.-Y. Lin, Y.-M. Chang / Research in Developmental Disabilities 37 (2015) 1–8
Fig. 2. the experimental setup.
3. Results The results of this study are based primarily on a descriptive and qualitative analyses of the data. The data collected from all 4 phases were used to create a graph, in which the x-axis indicated the 4 different phases and points scored, while the y-axis showed the number of times the participant did the correct movement. 3.1. Cindy’s result Cindy is mildly retarded, walks slowly and she also has weak balance. In her case, a brick was put on the floor, and the laptop screen was in front of her. She was asked to do step training using the 7 cm high brick. When she stepped up on the brick, dynamic pictures and sounds were displayed on the screen. She was then asked to step down, and wait for 5 s while at the same time seeing the stimulus on the screen. Cindy needed to step up onto the brick and then got feedback from the screen. Fig. 3 shows Cindy’s data.
[(Fig._3)TD$IG]
Fig. 3. Cindy’s result.
C.-Y. Lin, Y.-M. Chang / Research in Developmental Disabilities 37 (2015) 1–8
5
In Baseline 1 (A1: 9 sessions), the mean score for each session was low, at only 3.78, with a range from 3 to 5 over the 90 s. The results for Baseline 1 (A1) indicated that Cindy lacked motivation for the step training because such traditional training was too boring. When the experiment proceeded to Intervention 1 (B1:18 sessions), the mean score was 10.72, with a range for 9–12 in 90 s, indicating that she had a different stability in B1. Cindy achieved higher scores in B1 than A1. In B1, Cindy understood that when she did the step training, she could see the dynamic pictures and related sound. If she stepped up again, she could see different dynamic pictures and sound. In Baseline 2 (A2: 9 session), the intervention was withdrawn, and the data indicated that the mean in A2 reverted to same as in A1, with a mean score of 3.89, with a range 2–5 over 90 s. The results of A2 indicated that upon withdrawing the intervention, Cindy again lacked the motivation to do the physical activities, indicating that without the attraction, she did not wish to perform step the training. For Intervention 2(B2: 18 sessions) there was a significant difference, with a mean score of 10.89, and a range 9–13 over 90 s. Cindy found that with the intervention materials, she already understood that when she stepped up, she could get the feedback, and because she loved the cartoon figures very much, this was an attractive motivation for Cindy. In the effect measurement, for A1 and B1, the xt’s p = .795 > .05, and the intercept effect size f_square = 6.2969. For A2 and B2, the xt’s p = .721 > .05, there is not significant on slope’s difference, the intercept effect size f_square = 4.7286. The intervention phase demonstrated significantly higher effectiveness than the baseline; A1 and B1 produced a large effect (6.2969 > 0.35), as did A2 and B2 produced a large effect (4.7286 > 0.35), with the interventions showing immediate effects for Cindy’s step training activities. From a visual analysis, the score significantly increased in the B1 phase, and the B2 score results were higher than those for A2. The effect sizes were both large. The results demonstrated that the improvement between baseline phases and intervention phases was significant (p = .00 < .05), from the Kolmogorov–Smirnov statistical test, the results demonstrated that the improvement between the baseline phases and the intervention phases was significant, thus, this intervention had a significant effect on Cindy’s physical activity. 3.2. Kitty’s result Kitty finds taking and operating items with her feet difficult. When she stretched her feet and touched the brick, the dynamic cartoon pictures and relative sounds showed, and after playing for 5 s, the feedback disappeared. From this interaction, Kitty understood that when she lifted and stretched her feet again, she could get different dynamic pictures and sounds. Fig. 4 shows Kitty ‘s data. In Baseline 1 (A1: 9 sessions), the mean for each session was low, with the mean score being 0.78, with a range 0–2 over 90 s. The results of Baseline 1 (A1) indicated that Kitty lacked motivation for physical activities as she maintained a static position sitting on her customization Kinder chair. When the experiment proceeded to Intervention 1 (B1:18 sessions), the mean score was 4.78, with a range 3–7 over 90 s, indicating a different response. Kitty achieved higher scores in B1 than in A1. In B1, Kitty understood that when she stretched her feet to touch the brick, she could watch the multimedia and listen to the sound from the screen; and when 5 s later, the dynamic pictures and sounds disappear, if she lift her feet again to put on the surface of the brick, she could watch the multimedia continue. In Baseline 2 (A2: 9 session) the intervention was withdrawn, and the mean for A2 reverted to the same as that for A1. The mean score for A2 was 1.11, with a range 0–2 over 90 s. The results from A2 indicated that when the intervention was withdrawn, Kitty again lacked the motivation to do the physical activities as there was no attraction to motivate her to perform the physical activities, so she sat immobile on her Kinder chair just as before. In intervention 2(B2: 18 sessions) the study indicated a difference from A2, with the mean score
[(Fig._4)TD$IG]
Fig. 4. Kitty’s result.
6
C.-Y. Lin, Y.-M. Chang / Research in Developmental Disabilities 37 (2015) 1–8
for B2 being 5.33, with a range 4–6 over 90 s. With the interventions materials, Kitty lifted and stretched her feet to repeatedly touch the brick, and understood that when she performed this action repeatedly, she could watch the dynamic pictures and audio feedback continue, thus representing an attractive motivation for her. In the effect measurement, for A1 and B1, the xt’s p = .179 > .05, and the intercept effect size f_square = 6973. For A2 and B2, the xt’s p = .627 > .05, and the intercept effect size f_square = 3.5298. The intervention phase demonstrated a significantly higher effectiveness than the baseline; The movement from A1 and B1 produced a large effect (.6973 > .35), as did that from A2 and B2 (3.5298 > .35), with the interventions producing an immediate effect on Kitty’s physical activities. From a visual analysis, the score significantly increased at the B1 phase, the B2 score results were higher than for A2. The effect sizes were both large. The results demonstrated that the improvement between baseline phases and intervention phases was significant (p = .00 < .05), from the Kolmogorov–Smirnov statistical test, the results demonstrated that the improvement between the baseline phases and the intervention phases was significant, thus, this intervention had a significant effect on Kitty’s physical activity. 3.3. John’s result Because John has cerebral palsy and has low vision, this study sought to train him to lift his legs and move by himself. He sat on the chair, and we arranged a brick on the floor, so that he was required to lift his feet up 7 cm and move his legs to touch the brick. Because he has low vision, we focused more on audio feedback. The audio contents were made up of recordings of many sentences from his teacher’s teaching materials and their dialogs, which were then remixed using Scratch 2.0. After 5 s, the sentences would stop, motivating him to listen to another audio feedback by lifting his feet up 7 cm to the brick surface. Fig. 5 shows John’s data. For Baseline 1 (A1: 9 sessions), the mean for each session was low, with a mean score of 1.44, with a range 0–2 over 90 s. These results indicated that John lacked the motivation for physical activities. When the experiment proceeded to Intervention 1 (B1: 18 sessions), the mean score was 5.89, with a range 5–7 over 90 s, indicating different stability in B1. John understood that when he lifted his legs 7 cm and moved his feet to touch the brick, he could hear the sentences; then, 5 s later, when the audio feedback stopped, he had a strong motivation to lift his feet again as he was expecting to hear another audio feedback. In Baseline 2 (A2: 9 session), the intervention was withdrawn, and the mean for A2 reverted to that of A1, with the mean score being 1.56, with a range 0–3 over 90 s. The results of A2 indicated that John also lacked motivation to exercise his legs. For Intervention 2 (B2: 18 session), there was a difference from A2, with the mean score being 6.22, with a range 5–7 over 90 s. When John found that he could control the feedback again, he appeared excited and lifted his feet to touch the brick repeatedly. He understood that when he repeated this action, he could obtain the audio feedback, which was a big achievement for him. In the effect measurement, for A1 and B1, the xt’s p = .982 > .05, and the intercept effect size f_square = 3.0263. For A2 and B2, the xt’s p = .618 > .05, and the intercept effect size f_square = 2.8328. The intervention phase showed significant higher effectiveness than the baseline as there was a big difference between A1 and B1 and A2 and B2, with the interventions producing an immediate effect on John’s physical activities. A1 and B1 produced a large effect (3.0263 > 0.35), as did A2 and B2 produced a large effect (2.8328 > 0.35), with the interventions producing an immediate effects on John’s physical activities. The intervention phase demonstrated significantly higher effectiveness than the baseline; thus, the interventions produced an immediate effect on John’s physical activity. B1 and B2 score results were higher than for A1 and A2. The effect sizes both are both large effect. The results demonstrated that the improvement between baseline phases and intervention
[(Fig._5)TD$IG]
Fig. 5. John’s result.
C.-Y. Lin, Y.-M. Chang / Research in Developmental Disabilities 37 (2015) 1–8
7
phases was significant (p = .00 < .05). From the Kolmogorov–Smirnov statistical test, the results indicated that the improvement between the baseline and intervention was significant. Therefore, this study suggests that a good intervention could persuade John to engage in physical activity training.
4. Discussion In this study, the use of technology from a real-time feedback concept via external webcam and Scratch 2.0 to make a flexible low-cost interactive basic program was investigated for an individual diagnosed with developmental disabilities. The results indicated a significant and positive effect on the physical activity of children (Fedewa & Ahn, 2011). In Cindy’s report, for the baseline 1 and 2, she only lacked motivation to do the simple step training. From the results of intervention 1 and 2, when dynamic cartoon pictures and sounds were used as stimulus, the duration of the interactive effect increased significantly. This study also made another observation. The purpose of this study was to use dynamic pictures and sounds to focus on physical activities, but after the experiments, Cindy looked at the pictures and also repeated the sounds out loud (she seldom talked before the experiment). This project revealed not only changes in physical activity but also in cognitive learning. For baseline 1 and 2, Kitty only sat in her Kinder chair. From the results of intervention 1 and 2, she moved and lifted her feet to the brick’s surface. The research asked her mother what cartoon character she liked, then designed the interactive contents specifically for her. The results showed that using the cartoon character had a positive reaction, as she knew when she lifted her feet 7 cm on to the brick, she could get the image and audio feedback, so this study supported her motivation to do the lift training. In the baseline 1 and 2, John sat on his chair, or moved on the floor. This study asked him to not only move his feet, but also to lift his feet 7 cm which was quite difficult as he has low vision. In this case, the study used audio feedback made up of recordings of sentences that he knew, his teachers’ talk, or their dialogs. If he lifted his feet onto the brick, he could hear the sentences. For interventions 1 and 2 he sat on a chair, moved and lifted his feet onto the brick’s surface. He also showed the cognition to know that when he lifted his feet onto the brick, he could hear the sound from the speakers. He liked to hold the speaker near to his ear to get the feedback. Because he wanted to hear the sentences again, he was motivated to do the feet lifting training. While the Xbox, Wii, or Kinect may be suitable for normal people, the original design does not suit developmental or physical disabilities, especially children with developmental disabilities as this game type exceeds their cognitive load. Because they are unable to understand body movement, their cognitive abilities focus on touching real objects then receiving feedback, so they do not understand how to control AR games. This study used an external webcam and Scratch 2.0 interactive technology to provide a variety of content. It was only necessary to change the pictures and sounds, so that there could be specific contents for specific participants. Because Scratch 2.0 software is free and has off-line software support, this application could be used flexibly. A new feature of Scratch 2.0 is the webcam integration that can track movements when the participants move their body. This function is similar to Kinect, but where Kinect tracks the depth of the video, Scratch tracks movements. Participants in this novel program are not restricted by age: elderly individuals with normal cognitive function or those with severe disabilities may also benefit, because only a laptop and an external webcam is required. Other augmented reality applications, such as X-box and Kinect, require specific equipment. Therefore, from the interactive design, body motion interactive games can provide feedback and stimulate children’s motivation. Therefore, it is an advantage to design in Scratch 2.0, because the software is free and open source and the contents are easy to revise, so it could be a flexible interface for designing different contents to fit different children. Content could be designed for special individual needs, and because it is free, teachers and parents would not need to consider the price, unlike other assistive technology. For this study, we only bought an external webcam, which cost less then $10 USD, to execute this body movement project. In related research, Shih and Chiu (2014) play participant’s favorite video and Lin and Chang (2014) used multimedia as a solution to improve motivation and enhance physical activities. This study also focused on different feedback according to which video and audio could attract the participants. The results showed that the interface interaction is significant for people with different disabilities, such as Cerebral Palsy and developmental disabilities. From the research results, this methodology could be applied to custom-made interactive augmented reality games. After this study, the researchers designed some activities in a resource class in an elementary school and junior high school with the content revised to fit the different ages. Patients with partial paralysis, muscle and nerve damage, paralysis of the brain, spinal cord injury, the elderly, and those who can still move some parts of their limb, could benefit from this methodology and the results of this study show this could be a flexible design. The same concept can be used but only the contents need to be resigned, allowing for custom-made designs for developmental disabilities. Acknowledgements This work was financially supported by the National Science Council, Taiwan, under the Grant No. 100-2410-H-024-028MY2. The authors wish to thanks for Scratch team at the MIT Media Lab for their wonderful open-source projects and website (http://scratch.mit.edu).
8
C.-Y. Lin, Y.-M. Chang / Research in Developmental Disabilities 37 (2015) 1–8
References Andujar, J. M., Mejı´as, A., & Marquez, M. A. (2011). Augmented reality for the improvement of remote laboratories: An augmented remote laboratory. IEEE Transactions on Education, 54(3), 492–500. Bonnardel, N., Piolat, A., & Bigot, L. L. (2011). The impact of colour on Website appeal and users’ cognitive processes. Displays, 32, 69–80. Brennan, K., & Resnick, M. (2013). Stories from the Scratch community: Connecting with ideas, interests, and people. SIGCSE 2013. Brennan, K., Resnick, M., & Monroy-Hernandez, A. (2010). Making projects, making friends: Online community as catalyst for interactive media creation. New Directions for Youth Development, 128, 75–83. Bujak, K. R., Radu, I., Catrambone, R., MacIntyre, B., Zheng, R., & Golubski, G. (2013). A psychological perspective on augmented reality in the mathematics classroom. Computers & Education, 68, 536–544. Carini, G. (2012). Finding a needle in a haystack: New ways to search and browse on Scratch (doctoral dissertation) Massachusetts Institute of Technology. Chang, Y. J., Chen, S. F., & Huang, J. D. (2011). A Kinect-based system for physical rehabilitation: A pilot study for young adults with motor disabilities. Research in Developmental Disabilities, 32, 2566–2570. Chang, Y. J., Kang, Y. S., & Huang, P. C. (2013). An augmented reality (AR)-based vocational task prompting system for people with cognitive impairments. Research in Developmental Disabilities, 34(10), 3049–3056. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum. Ding, Q., Stevenson, I. H., Wang, N., Li, W., Sun, Y., Wang, Q., et al. (2013). Motion games improve balance control in stroke survivors: A preliminary study based on the principle of constraint-induced movement therapy. Displays, 34, 125–131. Fedewa, A. L., & Ahn, S. (2011). The effects of physical activity and physical fitness on children’s achievement and cognitive outcomes: A meta-analysis. Research Quarterly for Exercise and Sport, 82(3), 521–535. Harris, J. (2013). Digital skin: How developments in digital imaging techniques and culture are informing the design of futuristic surface and fabrication concepts. Textile: The Journal of Cloth and Culture, 11(3), 242–261. Hedges, L. V., Pustejovsky, J. E., & Shadish, W. R. (2013). A standardized mean difference effect size for multiple baseline designs across individuals. Research Synthesis Methods, 4(4), 324–341. Hwang, T. H. T. (2012). Exploring real-time video interactivity with Scratch (doctoral dissertation) Massachusetts Institute of Technology. Kagohara, D. M., Sigafoos, J., Achmadi, D., O’Reilly, M., & Lancioni, G. (2012). Teaching children with autism spectrum disorders to check the spelling of words. Research in Autism Spectrum Disorders, 6, 304–310. Lin, C. Y., & Chang, Y. M. (2014). Increase in physical activities in kindergarten children with cerebral palsy by employing MaKey–MaKey-based task systems. Research in Developmental Disabilities, 35(9), 1963–1969. Lotan, M., Yalon-Chamovitz, S., & Weiss, P. L. (2009). Lessons learned towards a best practices model of virtual reality intervention for individuals with intellectual and developmental disability. Virtual Rehabilitation International Conference (pp. 70–77). Massachusetts Institute of Technology (MIT) (2013). What’s new in Scratch 2.0?. Retrieved from http://scratch.mit.edu/overview/ Monroy-Herna´ndez, A., & Resnick, M. (2008). Feature: Empowering kids to create and share programmable media. Interactions, 15(2), 50–53. Neely, L., Rispoli, M., Camargo, S., Davis, H., & Boles, M. (2013). The effect of instructional use of an iPad on challenging behavior and academic engagement for two students with autism. Research in Autism Spectrum Disorders, 7, 509–516. Olsson, T., Ka¨rkka¨inen, T., Lagerstam, E., & Venta¨-Olkkonen, L. (2012). User evaluation of mobile augmented reality scenarios. Journal of Ambient Intelligence and Smart Environments, 4(1), 29–47. Radu, I., & MacIntyre, B. (2009). Augmented-reality Scratch: A tangible programming environment for children. Proceedings of conference on interaction design for children. Resnick, M. (2012). Mother’s day, warrior cats, and digital fluency: Stories from the Scratch online community. Proceedings of the constructionism 2012 conference: Theory, practice and impact (pp. 52–58). Resnick, M., Maloney, J., Monroy-Herna´ndez, A., Rusk, N., Eastmond, E., Brennan, K., et al. (2009). Scratch: Programming for all. Communications of the ACM, 52(11), 60–67. Resnick, M., & Rosenbaum, E. (2013). Designing for tinkerability. In M. Honey & D. Kanter (Eds.), Design, make, play: Growing the next generation of STEM innovators (pp. 163–181). New York: Routledge. Resnick, M., Maloney, J., Monroy-Herna´ndez, A., Rusk, N., Eastmond, E., Brennan, K., et al. (2009). Scratch: Programming for all. Communications of the ACM, 52, 60– 67. Simo˜es, J., Redondo, R. D., & Vilas, A. F. (2013). A social gamification framework for a K-6 learning platform. Computers in Human Behavior, 29(2), 345–353. Shih, C. H., & Chiu, Y. C. (2014). Assisting obese students with intellectual disabilities to actively perform the activity of walking in place using a dance pad to control their preferred environmental stimulation. Research in Developmental Disabilities, 35(10), 2394–2402. Shih, C. T., Shih, C. H., & Luo, C. H. (2013). Assisting people with disabilities in actively performing physical activities by controlling the preferred environmental stimulation with a gyration air mouse. Research in Developmental Disabilities, 34(12), 4328–4333. Solari, F., Chessa, M., Garibotti, M., & Sabatini, S. P. (2012). Natural perception in dynamic stereoscopic augmented reality environments. Displays, 34(2), 142–152. Standen, P. J., Camm, C., Battersby, S., Brown, D. J., & Harrison, M. (2011). An evaluation of the Wii Nunchuk as an alternative assistive device for people with intellectual and physical disabilities using switch controlled software. Computers & Education, 56(1), 2–10. Trojan, J., Diers, M., Fuchs, X., Bach, F., Bekrater-Bodmann, R., Foell, J., et al. (2013). An augmented reality home-training system based on the mirror training and imagery approach. Behavior Research Methods, 46, 1–7. Volonte´, F., Pugin, F., Bucher, P., Sugimoto, M., Ratib, O., & Morel, P. (2011). Augmented reality and image overlay navigation with OsiriX in laparoscopic and robotic surgery: Not only a matter of fashion. Journal of Hepato-Biliary-Pancreatic Sciences, 18(4), 506–509. Wojciechowski, R., & Cellary, W. (2013). Evaluation of learners’ attitude toward learning in ARIES augmented reality environments. Computers & Education, 68, 570–585.
Contents lists available at ScienceDirect
Research in Developmental Disabilities
Interactive augmented reality using Scratch 2.0 to improve physical activities for children with developmental disabilities Chien-Yu Lin a,*, Yu-Ming Chang b a
Department of Special Education, National University of Tainan, 33, Section 2, Shu-Lin Street, Tainan 700, Taiwan College of Digital Design, Southern Taiwan University of Science and Technology, No. 1, Nantai Street, Yongkang Dist., Tainan City 710, Taiwan
b
A R T I C L E I N F O
A B S T R A C T
Article history: Received 9 October 2014 Accepted 10 October 2014 Available online
This study uses a body motion interactive game developed in Scratch 2.0 to enhance the body strength of children with disabilities. Scratch 2.0, using an augmented-reality function on a program platform, creates real world and virtual reality displays at the same time. This study uses a webcam integration that tracks movements and allows participants to interact physically with the project, to enhance the motivation of children with developmental disabilities to perform physical activities. This study follows a single-case research using an ABAB structure, in which A is the baseline and B is the intervention. The experimental period was 2 months. The experimental results demonstrated that the scores for 3 children with developmental disabilities increased considerably during the intervention phrases. The developmental applications of these results are also discussed. ß 2014 Elsevier Ltd. All rights reserved.
Keywords: Disabilities Scratch 2.0 Augmented-reality Webcam
1. Introduction These recent changes in Information and Communication Technologies (ICTs) have changed society, influencing the method people relate, communicate, work and learn (Simo˜es, Redondo, & Vilas, 2013), real-time interactive multimedia is more popular and affordable (Hwang, 2012), thus, allowing for the use of technology for special needs (Kagohara, Sigafoos, Achmadi, O’Reilly, & Lancioni, 2012). However, one of the challenges in human–computer interactions is to design systems that are not only usable but also appealing to users (Bonnardel, Piolat, & Bigot, 2011). Augmented reality (AR), as an emerging interactive technology, has increasingly attracted public interest during the last few years (Olsson, Ka¨rkka¨inen, Lagerstam, & Venta¨-Olkkonen, 2012) (Radu & MacIntyre, 2009). AR technology enables the merging of virtual objects with real objects, resulting in augmented reality environments (Trojan et al., 2013), and can be used for a live direct or indirect view of a physical real-world environment, the elements of which are augmented by computer-generated sensory inputs, such as sound or graphics (Chang, Kang, & Huang, 2013). Because in augmented reality environments both virtual and real objects can co-exist and interact in real time, the augmented reality applications are broad and the number of applications is increasing rapidly (Solari, Chessa, Garibotti, & Sabatini, 2012). The main advantages of AR applications used across different fields have been widely discussed in the literature, such as in education (Wojciechowski & Cellary, 2013), textiles (Harris,
* Corresponding author at: 33, Section 2, Shu-Lin Street, Tainan 700, Taiwan. Tel.: +886 62133111x744. E-mail address: [email protected] (C.-Y. Lin). http://dx.doi.org/10.1016/j.ridd.2014.10.016 0891-4222/ß 2014 Elsevier Ltd. All rights reserved.
2
C.-Y. Lin, Y.-M. Chang / Research in Developmental Disabilities 37 (2015) 1–8
2013) surgical interventions (Volonte´ et al., 2011), games, home-training system, online teaching (Andujar, Mejı´as, & Marquez, 2011),and learning disabilities(Chang et al., 2013). Augmented reality can lower the barrier to entry for students engaging in virtual content. The ease of interaction within AR-based experiences has already been shown by the use of virtual experiences in early school years (Bujak et al., 2013). People with developmental disabilities experience limitations in fine motor control, strength, and range of motion, which can reduce their participation in community and leisure activities (Chang, Chen, & Huang, 2011). Custom made alternative devices for those with special needs are expensive and the low unit turnover makes the prospect unattractive to potential manufacturers (Standen, Camm, Battersby, Brown, & Harrison, 2011). Most leisure pastimes available for people with severe physical limitations are often extremely limited (Lotan, Yalon-Chamovitz, & Weiss, 2009), so are not useful for people with developmental disabilities. Art games on console systems such as the Microsoft Xbox, Nintendo Wii or Sony PlayStation are not only for fun, but have also been applied in rehabilitation treatments, and therapeutic systems (Ding et al., 2013). Recently, many open source activities have become available, which share their programs, so users can design their own custom interfaces. This study uses Scratch 2.0, which is a visual programming environment designed at the MIT Media Lab (Resnick, 2012; Resnick & Rosenbaum, 2013) and which includes costumes and scripts. Scratch Web site (http://scratch.mit.edu) create a free online interactive community, with people sharing, discussing their programs, most important that could be designing, creating, and remixing one another’s projects (Resnick et al., 2009a,b). Programming is done by inserting command blocks which form scripts that control the interactive interface. Various researchers and designers have imagined that Augmented Reality technology could be well suited for children (Radu & MacIntyre, 2009). In Scratch 2.0, video sensing has been added, and there are new camera programing blocks that allow users to create projects that react to movements in the physical world through the use of a webcam (Carini, 2012). In other words, augmented reality could be applied to this interface (Massachusetts Institute of Technology, 2013). Using the webcam effect design in Scratch 2.0 (Radu & MacIntyre, 2009), the virtual objects and related sounds appear when the webcam detects movement, thus allowing for augmented reality. Users are able to now use their webcam to interact with projects and can also create their own programming blocks in Scratch 2.0. The advantage of this study is that is it low-cost, as only a webcam is needed for children with different needs to extend their activities. 2. Materials and methods 2.1. Participants There were 3 participants in this study, all of them children with different developmental difficulties. Prior to the study, we obtained formal consent from their parents. This study designed individual physical activities for the different needs of the participants to enhance their body motion motivation. Cindy, a 4-year 1-month-old female has developmental disabilities. She is able to walk by herself, but she walks slowly and unevenly. She is mildly intellectually retarded and seldom talks, she could understands basic oral instructions, the oral ability belongs to basic expression with tardy. In this case, we wanted to train Cindy’s body movement abilities by stimulating her feet using an AR interactive game, which was similar to step training. Kitty, the second participant, is a 6-year-old girl, who has severe cerebral palsy. She belongs to no oral expression. Because of her condition she is unable to stand by herself and all her limbs involuntarily twitch, so she has to sit on a customized Kinder chair and be strapped into her H-harness to remain stable. As she has weak leg movements, she is always sitting and lacks the motivation to lift her legs. In this case, we wanted to train her leg muscles and give her the motivation to move her legs and feet. John, our third participant, is a 3-year 11-month-old boy, who has moderate multiple disabilities, weak legs and low vision. He is able to climb but is not able to stand or walk by himself, so he is always sitting, he could understands basic oral instructions and he could express what he want to do. He is able to lift his legs and feet, but lacks the initiative to strengthen them. 2.2. Apparatus, material and setting The interactive feedback was designed using Scratch 2.0 software. The original design was a video pop balloon game, from an original program by Christine Garrity. With the Scratch 2.0 added webcam support, she designed an intuitive interactive game. Using a webcam, users are able to see themselves and the balloons on the screen. As the balloon changes position, the user uses their hands (or any part of their body) to move in the air. When the body movement is in the sensor area (in the original design it is in balloon area), the screen shows a punctured balloon. The game’s design is similar to the Xbox, and is fit for normal children. However, in this study pretest, as the participants all had developmental disabilities, the knowledge required exceeded their cognitive load, so at the beginning, the participants always approached the screen with the intention of touching the balloon. Therefore, this study revised and remix the program, that make the virtual stimulus object always appear in the same position, which overlap the real object on the screen, when the participant touch the real object, it’s means also touch the virtual stimulus object, and then the participant could got the feedback, so it could be used for special needs participants. To do this, we put a real brick on the floor, and focused the external webcam toward the real brick, making
C.-Y. Lin, Y.-M. Chang / Research in Developmental Disabilities 37 (2015) 1–8
3
sure that the sensor area overlapped the brick. When the participant touched the sensor area, dynamic pictures and the related sounds were shown. Then, after asking for the parents and teachers’ suggestions, we remixed the original design, and used cartoon characters as the basic images to design the dynamic images and related sounds as feedback. The concept of the study is shown in Fig. 1. We used the webcam to detect the body movement on the screen. The goals for each participant were adjusted for each of their different disabilities. Cindy can walk but is not stable, so this study trained her to step up onto the brick. Kitty and John have physical disabilities, so are unable to stand by themselves, so the study required them to sit and lift their feet up 7 cm onto the brick’s surface. 2.3. General procedure This study focused on how the effect of the webcam and the interactive computer game can assist people with physical disabilities. The computer screen was arranged in front of the participants, and two speakers were put on the floor. When the participants did the correct foot lifting and movement, they received the dynamic pictures and sounds as feedback. Fig. 2 shows the experimental setup. In the interactive effect design, this study used Scratch 2.0 from the Media Laboratory at MIT as the software, which allows for the development of new technologies to engage people in creative learning experiences (Brennan & Resnick, 2013). Using this Scratch software, the designer can create their own interactive stories, games, and animations (MonroyHerna´ndez & Resnick, 2008). This study examined how to use an external webcam to detect movement and give real-time feedback. At the beginning, a sensor area is shown on the screen, and, in the original design, when the participant moves their body; the webcam detects the movement when they move into the sensor area. If this is done correctly, feedback, which includes dynamic pictures and the related sound, is shown on the screen. Because our participants were children with physical disabilities, their response time was slower than for normal people, so the operating processing time was set to 90 s. The input interface is a low-cost device with a computer. This research adopted Scratch software technology (Brennan, Resnick, & Monroy-Hernandez, 2010; Resnick et al., 2009a,b). The experimental design adopted an ABAB reversal design for single-case research is a tool in special education and assistive technology (Neely, Rispoli, Camargo, Davis, & Boles, 2013; Shih, Shih, & Luo, 2013), in which A (baseline phase) was followed by B (intervention phase), a return to baseline phase, and then a final intervention phase. The A represented baseline phases while the B represented intervention phases with the webcam and Scratch 2.0 system. The experiment was divided into four phases: Baseline 1, Intervention 1, Baseline 2, and Intervention 2. Single-case design is a research method involving deliberate assignment of different conditions to the same individual and measurement of one or more outcomes over time (Hedges, Pustejovsky, & Shadish, 2013). Cohen (1988) offered a large (>0.35), medium (0.15–0.35), or small effect size (0.02–0.15), an effect size for predictive regression equations that could indicate the actual effect. The data was collected over 2 months, with 2–3 sessions per day, 3–5 days a week, with each session lasting 90 s, during which we collected the data points. In the first phase, Baseline 1 (A1), we collected 9 data points. In the second phase, Intervention 1 (B1), the intervention used an external webcam and the Scratch 2.0 system, from which we collected 18 data points. In the third phrase, Baseline 2 (A2), from which we collected 9 data points, the laptop, external webcam and AR contents were withdrawn. In the fourth phase, intervention 2 (B2), the intervention setup was the same as for B1, from which we collected 18 data points.
[(Fig._1)TD$IG]
Fig. 1. the concept of the study.
[(Fig._2)TD$IG]
4
C.-Y. Lin, Y.-M. Chang / Research in Developmental Disabilities 37 (2015) 1–8
Fig. 2. the experimental setup.
3. Results The results of this study are based primarily on a descriptive and qualitative analyses of the data. The data collected from all 4 phases were used to create a graph, in which the x-axis indicated the 4 different phases and points scored, while the y-axis showed the number of times the participant did the correct movement. 3.1. Cindy’s result Cindy is mildly retarded, walks slowly and she also has weak balance. In her case, a brick was put on the floor, and the laptop screen was in front of her. She was asked to do step training using the 7 cm high brick. When she stepped up on the brick, dynamic pictures and sounds were displayed on the screen. She was then asked to step down, and wait for 5 s while at the same time seeing the stimulus on the screen. Cindy needed to step up onto the brick and then got feedback from the screen. Fig. 3 shows Cindy’s data.
[(Fig._3)TD$IG]
Fig. 3. Cindy’s result.
C.-Y. Lin, Y.-M. Chang / Research in Developmental Disabilities 37 (2015) 1–8
5
In Baseline 1 (A1: 9 sessions), the mean score for each session was low, at only 3.78, with a range from 3 to 5 over the 90 s. The results for Baseline 1 (A1) indicated that Cindy lacked motivation for the step training because such traditional training was too boring. When the experiment proceeded to Intervention 1 (B1:18 sessions), the mean score was 10.72, with a range for 9–12 in 90 s, indicating that she had a different stability in B1. Cindy achieved higher scores in B1 than A1. In B1, Cindy understood that when she did the step training, she could see the dynamic pictures and related sound. If she stepped up again, she could see different dynamic pictures and sound. In Baseline 2 (A2: 9 session), the intervention was withdrawn, and the data indicated that the mean in A2 reverted to same as in A1, with a mean score of 3.89, with a range 2–5 over 90 s. The results of A2 indicated that upon withdrawing the intervention, Cindy again lacked the motivation to do the physical activities, indicating that without the attraction, she did not wish to perform step the training. For Intervention 2(B2: 18 sessions) there was a significant difference, with a mean score of 10.89, and a range 9–13 over 90 s. Cindy found that with the intervention materials, she already understood that when she stepped up, she could get the feedback, and because she loved the cartoon figures very much, this was an attractive motivation for Cindy. In the effect measurement, for A1 and B1, the xt’s p = .795 > .05, and the intercept effect size f_square = 6.2969. For A2 and B2, the xt’s p = .721 > .05, there is not significant on slope’s difference, the intercept effect size f_square = 4.7286. The intervention phase demonstrated significantly higher effectiveness than the baseline; A1 and B1 produced a large effect (6.2969 > 0.35), as did A2 and B2 produced a large effect (4.7286 > 0.35), with the interventions showing immediate effects for Cindy’s step training activities. From a visual analysis, the score significantly increased in the B1 phase, and the B2 score results were higher than those for A2. The effect sizes were both large. The results demonstrated that the improvement between baseline phases and intervention phases was significant (p = .00 < .05), from the Kolmogorov–Smirnov statistical test, the results demonstrated that the improvement between the baseline phases and the intervention phases was significant, thus, this intervention had a significant effect on Cindy’s physical activity. 3.2. Kitty’s result Kitty finds taking and operating items with her feet difficult. When she stretched her feet and touched the brick, the dynamic cartoon pictures and relative sounds showed, and after playing for 5 s, the feedback disappeared. From this interaction, Kitty understood that when she lifted and stretched her feet again, she could get different dynamic pictures and sounds. Fig. 4 shows Kitty ‘s data. In Baseline 1 (A1: 9 sessions), the mean for each session was low, with the mean score being 0.78, with a range 0–2 over 90 s. The results of Baseline 1 (A1) indicated that Kitty lacked motivation for physical activities as she maintained a static position sitting on her customization Kinder chair. When the experiment proceeded to Intervention 1 (B1:18 sessions), the mean score was 4.78, with a range 3–7 over 90 s, indicating a different response. Kitty achieved higher scores in B1 than in A1. In B1, Kitty understood that when she stretched her feet to touch the brick, she could watch the multimedia and listen to the sound from the screen; and when 5 s later, the dynamic pictures and sounds disappear, if she lift her feet again to put on the surface of the brick, she could watch the multimedia continue. In Baseline 2 (A2: 9 session) the intervention was withdrawn, and the mean for A2 reverted to the same as that for A1. The mean score for A2 was 1.11, with a range 0–2 over 90 s. The results from A2 indicated that when the intervention was withdrawn, Kitty again lacked the motivation to do the physical activities as there was no attraction to motivate her to perform the physical activities, so she sat immobile on her Kinder chair just as before. In intervention 2(B2: 18 sessions) the study indicated a difference from A2, with the mean score
[(Fig._4)TD$IG]
Fig. 4. Kitty’s result.
6
C.-Y. Lin, Y.-M. Chang / Research in Developmental Disabilities 37 (2015) 1–8
for B2 being 5.33, with a range 4–6 over 90 s. With the interventions materials, Kitty lifted and stretched her feet to repeatedly touch the brick, and understood that when she performed this action repeatedly, she could watch the dynamic pictures and audio feedback continue, thus representing an attractive motivation for her. In the effect measurement, for A1 and B1, the xt’s p = .179 > .05, and the intercept effect size f_square = 6973. For A2 and B2, the xt’s p = .627 > .05, and the intercept effect size f_square = 3.5298. The intervention phase demonstrated a significantly higher effectiveness than the baseline; The movement from A1 and B1 produced a large effect (.6973 > .35), as did that from A2 and B2 (3.5298 > .35), with the interventions producing an immediate effect on Kitty’s physical activities. From a visual analysis, the score significantly increased at the B1 phase, the B2 score results were higher than for A2. The effect sizes were both large. The results demonstrated that the improvement between baseline phases and intervention phases was significant (p = .00 < .05), from the Kolmogorov–Smirnov statistical test, the results demonstrated that the improvement between the baseline phases and the intervention phases was significant, thus, this intervention had a significant effect on Kitty’s physical activity. 3.3. John’s result Because John has cerebral palsy and has low vision, this study sought to train him to lift his legs and move by himself. He sat on the chair, and we arranged a brick on the floor, so that he was required to lift his feet up 7 cm and move his legs to touch the brick. Because he has low vision, we focused more on audio feedback. The audio contents were made up of recordings of many sentences from his teacher’s teaching materials and their dialogs, which were then remixed using Scratch 2.0. After 5 s, the sentences would stop, motivating him to listen to another audio feedback by lifting his feet up 7 cm to the brick surface. Fig. 5 shows John’s data. For Baseline 1 (A1: 9 sessions), the mean for each session was low, with a mean score of 1.44, with a range 0–2 over 90 s. These results indicated that John lacked the motivation for physical activities. When the experiment proceeded to Intervention 1 (B1: 18 sessions), the mean score was 5.89, with a range 5–7 over 90 s, indicating different stability in B1. John understood that when he lifted his legs 7 cm and moved his feet to touch the brick, he could hear the sentences; then, 5 s later, when the audio feedback stopped, he had a strong motivation to lift his feet again as he was expecting to hear another audio feedback. In Baseline 2 (A2: 9 session), the intervention was withdrawn, and the mean for A2 reverted to that of A1, with the mean score being 1.56, with a range 0–3 over 90 s. The results of A2 indicated that John also lacked motivation to exercise his legs. For Intervention 2 (B2: 18 session), there was a difference from A2, with the mean score being 6.22, with a range 5–7 over 90 s. When John found that he could control the feedback again, he appeared excited and lifted his feet to touch the brick repeatedly. He understood that when he repeated this action, he could obtain the audio feedback, which was a big achievement for him. In the effect measurement, for A1 and B1, the xt’s p = .982 > .05, and the intercept effect size f_square = 3.0263. For A2 and B2, the xt’s p = .618 > .05, and the intercept effect size f_square = 2.8328. The intervention phase showed significant higher effectiveness than the baseline as there was a big difference between A1 and B1 and A2 and B2, with the interventions producing an immediate effect on John’s physical activities. A1 and B1 produced a large effect (3.0263 > 0.35), as did A2 and B2 produced a large effect (2.8328 > 0.35), with the interventions producing an immediate effects on John’s physical activities. The intervention phase demonstrated significantly higher effectiveness than the baseline; thus, the interventions produced an immediate effect on John’s physical activity. B1 and B2 score results were higher than for A1 and A2. The effect sizes both are both large effect. The results demonstrated that the improvement between baseline phases and intervention
[(Fig._5)TD$IG]
Fig. 5. John’s result.
C.-Y. Lin, Y.-M. Chang / Research in Developmental Disabilities 37 (2015) 1–8
7
phases was significant (p = .00 < .05). From the Kolmogorov–Smirnov statistical test, the results indicated that the improvement between the baseline and intervention was significant. Therefore, this study suggests that a good intervention could persuade John to engage in physical activity training.
4. Discussion In this study, the use of technology from a real-time feedback concept via external webcam and Scratch 2.0 to make a flexible low-cost interactive basic program was investigated for an individual diagnosed with developmental disabilities. The results indicated a significant and positive effect on the physical activity of children (Fedewa & Ahn, 2011). In Cindy’s report, for the baseline 1 and 2, she only lacked motivation to do the simple step training. From the results of intervention 1 and 2, when dynamic cartoon pictures and sounds were used as stimulus, the duration of the interactive effect increased significantly. This study also made another observation. The purpose of this study was to use dynamic pictures and sounds to focus on physical activities, but after the experiments, Cindy looked at the pictures and also repeated the sounds out loud (she seldom talked before the experiment). This project revealed not only changes in physical activity but also in cognitive learning. For baseline 1 and 2, Kitty only sat in her Kinder chair. From the results of intervention 1 and 2, she moved and lifted her feet to the brick’s surface. The research asked her mother what cartoon character she liked, then designed the interactive contents specifically for her. The results showed that using the cartoon character had a positive reaction, as she knew when she lifted her feet 7 cm on to the brick, she could get the image and audio feedback, so this study supported her motivation to do the lift training. In the baseline 1 and 2, John sat on his chair, or moved on the floor. This study asked him to not only move his feet, but also to lift his feet 7 cm which was quite difficult as he has low vision. In this case, the study used audio feedback made up of recordings of sentences that he knew, his teachers’ talk, or their dialogs. If he lifted his feet onto the brick, he could hear the sentences. For interventions 1 and 2 he sat on a chair, moved and lifted his feet onto the brick’s surface. He also showed the cognition to know that when he lifted his feet onto the brick, he could hear the sound from the speakers. He liked to hold the speaker near to his ear to get the feedback. Because he wanted to hear the sentences again, he was motivated to do the feet lifting training. While the Xbox, Wii, or Kinect may be suitable for normal people, the original design does not suit developmental or physical disabilities, especially children with developmental disabilities as this game type exceeds their cognitive load. Because they are unable to understand body movement, their cognitive abilities focus on touching real objects then receiving feedback, so they do not understand how to control AR games. This study used an external webcam and Scratch 2.0 interactive technology to provide a variety of content. It was only necessary to change the pictures and sounds, so that there could be specific contents for specific participants. Because Scratch 2.0 software is free and has off-line software support, this application could be used flexibly. A new feature of Scratch 2.0 is the webcam integration that can track movements when the participants move their body. This function is similar to Kinect, but where Kinect tracks the depth of the video, Scratch tracks movements. Participants in this novel program are not restricted by age: elderly individuals with normal cognitive function or those with severe disabilities may also benefit, because only a laptop and an external webcam is required. Other augmented reality applications, such as X-box and Kinect, require specific equipment. Therefore, from the interactive design, body motion interactive games can provide feedback and stimulate children’s motivation. Therefore, it is an advantage to design in Scratch 2.0, because the software is free and open source and the contents are easy to revise, so it could be a flexible interface for designing different contents to fit different children. Content could be designed for special individual needs, and because it is free, teachers and parents would not need to consider the price, unlike other assistive technology. For this study, we only bought an external webcam, which cost less then $10 USD, to execute this body movement project. In related research, Shih and Chiu (2014) play participant’s favorite video and Lin and Chang (2014) used multimedia as a solution to improve motivation and enhance physical activities. This study also focused on different feedback according to which video and audio could attract the participants. The results showed that the interface interaction is significant for people with different disabilities, such as Cerebral Palsy and developmental disabilities. From the research results, this methodology could be applied to custom-made interactive augmented reality games. After this study, the researchers designed some activities in a resource class in an elementary school and junior high school with the content revised to fit the different ages. Patients with partial paralysis, muscle and nerve damage, paralysis of the brain, spinal cord injury, the elderly, and those who can still move some parts of their limb, could benefit from this methodology and the results of this study show this could be a flexible design. The same concept can be used but only the contents need to be resigned, allowing for custom-made designs for developmental disabilities. Acknowledgements This work was financially supported by the National Science Council, Taiwan, under the Grant No. 100-2410-H-024-028MY2. The authors wish to thanks for Scratch team at the MIT Media Lab for their wonderful open-source projects and website (http://scratch.mit.edu).
8
C.-Y. Lin, Y.-M. Chang / Research in Developmental Disabilities 37 (2015) 1–8
References Andujar, J. M., Mejı´as, A., & Marquez, M. A. (2011). Augmented reality for the improvement of remote laboratories: An augmented remote laboratory. IEEE Transactions on Education, 54(3), 492–500. Bonnardel, N., Piolat, A., & Bigot, L. L. (2011). The impact of colour on Website appeal and users’ cognitive processes. Displays, 32, 69–80. Brennan, K., & Resnick, M. (2013). Stories from the Scratch community: Connecting with ideas, interests, and people. SIGCSE 2013. Brennan, K., Resnick, M., & Monroy-Hernandez, A. (2010). Making projects, making friends: Online community as catalyst for interactive media creation. New Directions for Youth Development, 128, 75–83. Bujak, K. R., Radu, I., Catrambone, R., MacIntyre, B., Zheng, R., & Golubski, G. (2013). A psychological perspective on augmented reality in the mathematics classroom. Computers & Education, 68, 536–544. Carini, G. (2012). Finding a needle in a haystack: New ways to search and browse on Scratch (doctoral dissertation) Massachusetts Institute of Technology. Chang, Y. J., Chen, S. F., & Huang, J. D. (2011). A Kinect-based system for physical rehabilitation: A pilot study for young adults with motor disabilities. Research in Developmental Disabilities, 32, 2566–2570. Chang, Y. J., Kang, Y. S., & Huang, P. C. (2013). An augmented reality (AR)-based vocational task prompting system for people with cognitive impairments. Research in Developmental Disabilities, 34(10), 3049–3056. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum. Ding, Q., Stevenson, I. H., Wang, N., Li, W., Sun, Y., Wang, Q., et al. (2013). Motion games improve balance control in stroke survivors: A preliminary study based on the principle of constraint-induced movement therapy. Displays, 34, 125–131. Fedewa, A. L., & Ahn, S. (2011). The effects of physical activity and physical fitness on children’s achievement and cognitive outcomes: A meta-analysis. Research Quarterly for Exercise and Sport, 82(3), 521–535. Harris, J. (2013). Digital skin: How developments in digital imaging techniques and culture are informing the design of futuristic surface and fabrication concepts. Textile: The Journal of Cloth and Culture, 11(3), 242–261. Hedges, L. V., Pustejovsky, J. E., & Shadish, W. R. (2013). A standardized mean difference effect size for multiple baseline designs across individuals. Research Synthesis Methods, 4(4), 324–341. Hwang, T. H. T. (2012). Exploring real-time video interactivity with Scratch (doctoral dissertation) Massachusetts Institute of Technology. Kagohara, D. M., Sigafoos, J., Achmadi, D., O’Reilly, M., & Lancioni, G. (2012). Teaching children with autism spectrum disorders to check the spelling of words. Research in Autism Spectrum Disorders, 6, 304–310. Lin, C. Y., & Chang, Y. M. (2014). Increase in physical activities in kindergarten children with cerebral palsy by employing MaKey–MaKey-based task systems. Research in Developmental Disabilities, 35(9), 1963–1969. Lotan, M., Yalon-Chamovitz, S., & Weiss, P. L. (2009). Lessons learned towards a best practices model of virtual reality intervention for individuals with intellectual and developmental disability. Virtual Rehabilitation International Conference (pp. 70–77). Massachusetts Institute of Technology (MIT) (2013). What’s new in Scratch 2.0?. Retrieved from http://scratch.mit.edu/overview/ Monroy-Herna´ndez, A., & Resnick, M. (2008). Feature: Empowering kids to create and share programmable media. Interactions, 15(2), 50–53. Neely, L., Rispoli, M., Camargo, S., Davis, H., & Boles, M. (2013). The effect of instructional use of an iPad on challenging behavior and academic engagement for two students with autism. Research in Autism Spectrum Disorders, 7, 509–516. Olsson, T., Ka¨rkka¨inen, T., Lagerstam, E., & Venta¨-Olkkonen, L. (2012). User evaluation of mobile augmented reality scenarios. Journal of Ambient Intelligence and Smart Environments, 4(1), 29–47. Radu, I., & MacIntyre, B. (2009). Augmented-reality Scratch: A tangible programming environment for children. Proceedings of conference on interaction design for children. Resnick, M. (2012). Mother’s day, warrior cats, and digital fluency: Stories from the Scratch online community. Proceedings of the constructionism 2012 conference: Theory, practice and impact (pp. 52–58). Resnick, M., Maloney, J., Monroy-Herna´ndez, A., Rusk, N., Eastmond, E., Brennan, K., et al. (2009). Scratch: Programming for all. Communications of the ACM, 52(11), 60–67. Resnick, M., & Rosenbaum, E. (2013). Designing for tinkerability. In M. Honey & D. Kanter (Eds.), Design, make, play: Growing the next generation of STEM innovators (pp. 163–181). New York: Routledge. Resnick, M., Maloney, J., Monroy-Herna´ndez, A., Rusk, N., Eastmond, E., Brennan, K., et al. (2009). Scratch: Programming for all. Communications of the ACM, 52, 60– 67. Simo˜es, J., Redondo, R. D., & Vilas, A. F. (2013). A social gamification framework for a K-6 learning platform. Computers in Human Behavior, 29(2), 345–353. Shih, C. H., & Chiu, Y. C. (2014). Assisting obese students with intellectual disabilities to actively perform the activity of walking in place using a dance pad to control their preferred environmental stimulation. Research in Developmental Disabilities, 35(10), 2394–2402. Shih, C. T., Shih, C. H., & Luo, C. H. (2013). Assisting people with disabilities in actively performing physical activities by controlling the preferred environmental stimulation with a gyration air mouse. Research in Developmental Disabilities, 34(12), 4328–4333. Solari, F., Chessa, M., Garibotti, M., & Sabatini, S. P. (2012). Natural perception in dynamic stereoscopic augmented reality environments. Displays, 34(2), 142–152. Standen, P. J., Camm, C., Battersby, S., Brown, D. J., & Harrison, M. (2011). An evaluation of the Wii Nunchuk as an alternative assistive device for people with intellectual and physical disabilities using switch controlled software. Computers & Education, 56(1), 2–10. Trojan, J., Diers, M., Fuchs, X., Bach, F., Bekrater-Bodmann, R., Foell, J., et al. (2013). An augmented reality home-training system based on the mirror training and imagery approach. Behavior Research Methods, 46, 1–7. Volonte´, F., Pugin, F., Bucher, P., Sugimoto, M., Ratib, O., & Morel, P. (2011). Augmented reality and image overlay navigation with OsiriX in laparoscopic and robotic surgery: Not only a matter of fashion. Journal of Hepato-Biliary-Pancreatic Sciences, 18(4), 506–509. Wojciechowski, R., & Cellary, W. (2013). Evaluation of learners’ attitude toward learning in ARIES augmented reality environments. Computers & Education, 68, 570–585.
