376
IEEE TRANSACTIONS ON HAPTICS,
Effect of Dynamic Vibrotactile Feedback on the Control of Isometric Finger Force Teemu Ahmaniemi Abstract—This study investigates how controllability of force is influenced by concurrent vibrotactile feedback that is generated proportionally to the applied force. Three different models to provide tactile feedback are introduced: amplitude and frequency modulation and granular synthesis. Then, an experiment investigating the effect of the feedback models on force control is reported. The tactile feedback conditions were compared to each other and to a condition with no feedback in a force repetition and a force hold tasks. In the force repetition task, all the feedback conditions yielded significantly better accuracy compared to no feedback condition. In the force hold task, there was no difference in the accuracy between the conditions including the no feedback condition. The results suggest that dynamic vibrotactile feedback assists the force control in force repetition tasks. Index Terms—Haptic I/O, input devices and strategies, augmented feedback, force sensing
Ç 1
INTRODUCTION
CONTROLLING applied force is very important in every day interactions. Holding and manipulating objects with hands necessitates accurate control of fingers in contact with the object. In human-computer interaction (HCI), touch- and force-based interfaces are becoming more popular. In touch screens or touch pads, the interaction is based on touch sensitive elements such as virtual buttons that can be touched with a finger or a stylus. Corresponding functions are triggered when a suitable amount of force is applied to them. The setup is rather different from a system with a real keyboard where the deformation or movement of the buttons informs the user about the triggering. In addition, button size, material, and stiffness can be felt only through the sense of touch. These features play an important role in the interaction efficiency and overall user experience. Beyond virtual buttons, applied force has been proposed to be used as an input method of the user interface. Ramos et al. [1] introduced force-controlled user-interface widgets such as expanding pie and pressure grid. In their experiments, force was applied with a pressure sensitive stylus on a touch pad. Force was proposed to be used in menu selection by Stewart et al. [2]. In their work, force was applied with a finger on mobile phone touch screen and measured with force sensitive resistors. A similar menu selection method was proposed by Rekimoto and Schwesig [3]. Force control in a pen-based interface was studied by Mizobuchi et al. [4]. They found out that users were able to discriminate five to seven levels of force within the range of 0 to 10 Newtons. Cechanowicz et al. [5] assembled force sensors on a desktop mouse and studied the use of force as an input method in mouse interaction. They found out that the number of controllable force levels was significantly higher when there was more than one sensor used, attached to different sides of the mouse. In the above studies, feedback played an important role in the force control accuracy. Whereas visual feedback had significant influence on the accuracy [4], [5], tactile feedback turned out to be . The author is with the Nokia Research Center, Nokia Corporation, PO Box 226, FI-00045 Nokia Group, Otaniementie 19, 02150 Espoo, Finland. E-mail: [email protected]. Manuscript received 8 Feb. 2012; revised 1 Nov. 2012; accepted 13 Nov. 2012; published online 5 Dec. 2012. Recommended for acceptance by S. Choi. For information on obtaining reprints of this article, please send e-mail to: [email protected], and reference IEEECS Log Number TH-2012-02-0014. Digital Object Identifier no. 10.1109/TOH.2012.72. 1939-1412/13/$31.00 ß 2013 IEEE
Published by the IEEE CS, RAS, & CES
VOL. 6,
NO. 3,
JULY-SEPTEMBER 2013
useful as well. In force-controlled menu selection studies [2], [3], a vibrotactile burst was provided when the applied force increased, and the next item in the menu was activated. In a target selection experiment, the users performed equally well or even better with tactile feedback than with audio feedback [2]. This study aims to take more detail look into the methods to provide tactile feedback: The feedback is provided proportionally to the applied force using amplitude or frequency modulation or granular synthesis. The main goal of this approach is to increase the controllability of force. An improvement in the control would improve the interaction efficiency and ergonomics of virtual button and menu selection examples above. In addition to the increased force control, the approach may provide more degrees of freedom for user-interface design. Continuous feedback could make the feeling of user-interface elements unique. This aspect was dealt in a study by Kildal [6]. He showed that dynamic tactile feedback reacting to applied force on a touch pad could generate illusions of elasticity, stiffness, and flexibility. The force was applied to the touch pad with a stylus that was equipped with a vibrotactile actuator. In the current study, different methods to provide the tactile feedback were compared in a force control experiment. The tasks in the experiment were derived from typical use cases in touch interfaces: 1) repeating a force level a certain number of times, as in a number or text entry task, and 2) holding the obtained force level for a certain period of time as in a long press of a virtual button or drag-and-drop interaction on a touch screen.
2
FORCE CONTROL AND FEEDBACK
The sense of touch and haptic feedback has shown to contribute to human force control. Jones and Piateski [7] showed that perception of force when produced by finger flexors is significantly overestimated when tactile feedback from the finger is attenuated. The attenuation was produced with splints molded around subjects’ fingertips. Henningsen et al. [8] indicated that when tactile feedback from fingers is damped with a finger nail or it is interfered with vibration, force control becomes coarser. Monzee et al. [9] used digital anesthesia for blocking tactile feedback from fingers and found that it significantly degrades the controllability of force. The above studies provide clear evidence for the importance of tactile feedback in force control. However, they solely concentrate on the natural “intrinsic” feedback, while the current study investigates the role of tactile feedback that is provided in addition to the natural feedback. When vibrotactile feedback is provided proportionally to the applied force, it basically augments the applied input. Augmented feedback has shown to have significant influence on human motor control when provided visually [10], [11], aurally [11], [12], or via sense of touch [13]. Jiang et al. [13] studied the effect of vibrotactile feedback on the control of grasping force with multiple sclerosis patients. The haptic feedback was either proportional to the absolute grasping force or indicated whether the force was within the desired range. Both of these conditions yielded significantly better results in terms of control accuracy than the condition without haptic feedback. The current study investigated the control of isometric force that was applied to a rigid surface with a finger. Augmented feedback was provided with a vibrotactile actuator that was colocated with the force sensor. The study attempted to identify an optimal feedback model for control accuracy in short-term uses. The setup was similar to the study by Jiang et al. [13], but the tasks in our experiment were derived from typical use scenarios in HCI.
3
EXPERIMENT
The experiment had two parts: force repetition and force hold. The aim in the experiment was similar to the study by Jiang et al. [13]
IEEE TRANSACTIONS ON HAPTICS,
VOL. 6,
NO. 3,
JULY-SEPTEMBER 2013
377
on amplitude-based feedback: to study how different models of augmented tactile feedback influence the repeatability of different force levels and maintaining a certain force level. The tactile feedback was proportional to the applied force, whereas the target itself (or target range) was presented visually. The participants were assumed to learn what the target level felt like, especially with dynamic tactile feedback as in the study by Jiang et al. [13]. The experiments were conducted by following a similar method as in recent force or grip control studies [7], [8], [9], [14]; the target force level was first presented visually, and the task was to repeat or sustain the force level without the visual guidance. This approach resembled the text- or number entry use cases introduced above: User can first learn the system behavior by looking at the interface while pressing the virtual buttons. Then, after a period of time, the pressing can be done in eyes free manner solely relying on the haptic sensation of the interaction.
3.1
Feedback Design
The tactile feedback design was based on our previous studies about tactile textures and continuous feedback [15], [16]. The feedback was generated with Pure Data software (http://puredata. info/). The main principle was to control the drive signal within the limits of the selected actuator. The first approach was to modify either the amplitude or the frequency of the feedback signal based on the applied force. The second approach was to provide feedback as discrete bursts (grains) whenever the applied force changed similarly to the study by Kildal [6]. The experiment consisted of four feedback conditions: three different tactile feedback designs and one without any feedback (control condition, N). The three feedback designs were the following: .
A. Amplitude-modulated vibration of 250 Hz. The base frequency was the nominal frequency of the actuator, and the amplitude range was chosen so that the actual vibration amplitude (A) of the device increased linearly with force: A ¼ kA F ;
ð1Þ
g N,
.
and F is applied force. where kA ¼ 0:58 AF . Amplitude- and frequency-modulated “textured” vibration increasing in amplitude and envelope frequency along the applied force. The design was similar to our previous study [16] with vibrotactile textures where the nominal frequency of the actuator was modulated with an envelope signal. The amplitude response was similar to the A condition (1) and the envelope frequency (f) was increased by following a linear equation: f ¼ kf F ;
ð2Þ
where kf ¼ 11:1 Hz N , and F is applied force. AG. Grained feedback. The whole range of force was divided into 26 levels, and a discrete vibration burst (grain) was provided when the force level changed. The frequency of the vibration was 250 Hz and the duration 40 ms. The amplitude of the burst increased linearly with force following the same equation as in the A and AF conditions (1). The coefficients kA and kf were chosen so that the amplitude and frequency ranges of the tactile actuator were covered optimally. The fundamental difference between the feedback models was that in A and AF, the vibration was continuous as long as force was applied but in AG vibration bursts were provided only when the force changed. An illustration of the feedback conditions is shown in Fig. 1. The signals correspond to the applied force that first increases linearly, stays constant for 1 second, and then starts increasing again. The figure shows the acceleration of the vibration on the top of the device. .
Fig. 1. The feedback models in the experiments. The simulated force signal first increases linearly, then remains constant for 1 second, and then increases again. The difference between the grained vibration (AG) and the other models (A and AF ) was that in the grain model there was no feedback if the force level remained constant, whereas in other models constant force was responded with continuous vibration of constant amplitude or frequency.
Since the amplitude- and frequency-modulated feedback (AF) provided two dimensions of feedback related to the applied force level, the amplitude and the envelope frequency, it was expected to yield the best results in force control. The grain feedback (AG), on the other hand, indicated only how to get to a certain level, i.e., how many grains needed to be felt to reach it. Although the grains’ amplitude increased with applied force, there was no continuous feedback reflecting the absolute force level. Therefore, the grained feedback was not expected to assist the force control.
3.2
Experiment Setup
The experiment was arranged in a quiet office room equipped with a chair, a table, a PC with monitor (2200 ), and sensor-actuator hardware. Twenty-four subjects (four women, 20 men, mean age 37) participated in the experiment. The experiment took approximately 45 minutes and was compensated with a cinema ticket worth of 10 euros.
3.2.1
Hardware
A capacitive force sensor (Loadstar LoadVUE Lite) was firmly packed into a custom made box (38 38 25 mm) with a tactile actuator (C2). The sensor provided 0-10 lbs (0-40 N) force readings with 500-Hz sampling rate within a resolution of 0.004 N. The sensor was connected to a laptop PC (Lenovo W510) through an interface module (Loadstar DQ-1000U) provided by the sensor manufacturer. The tactile actuator was connected to the PC’s soundcard through a stereo amplifier (NAD 312). An illustration of this setup is presented in Fig. 2. The sensor values were captured by a USB serial interface and routed to Pure Data synthesis software. The frequency response of C2 tactile actuator was rather narrow, 200-300 Hz. This limitation caused saturation of the signal amplitude with higher frequencies. This impacted the amplitude response of the AF feedback condition as shown in Fig. 1. The endto-end latency of the system was measured with a multimodal
378
IEEE TRANSACTIONS ON HAPTICS,
VOL. 6,
NO. 3,
JULY-SEPTEMBER 2013
Fig. 2. The diagram of the system (top left) consisted of a laptop computer running Pure Data for feedback synthesis, stereo amplifier, and sensor-actuator device. The sensor box (exploded view on top right) was placed on a rack that stood on an iron frame. The participant applied force with her/his index finger of the dominant hand.
latency measurement tool by Kaaresoja and Brewster [17] yielding 54 ms. The latency was introduced by the sensor sampling, serial interface, feedback synthesis, audio hardware buffering, and the actuator rise time.
3.2.2
Visual Guidance and Force Levels
All the visuals in the experiment were rendered with Graphic Environment for Multimedia plug-in for Pure Data software. To minimize the effect of visual guidance in task performance, only a white solid circle appeared on the screen when the applied force on the sensor was within a 20 percent range from the target. In the repetition test, the circle turned green when the force had been within the target zone for 300 ms (see Fig. 2). This ensured that participants had intentionally reached the zone. A press was considered complete when the circle had turned green and the force was released. In the hold test, the white circle disappeared after being within the zone for 1 second. After this, the task was to maintain the reached force level for 3 seconds. Ten different target force levels equidistantly distributed between 0.7 and 4.0 N were used, but 20 percent error in both directions was allowed, i.e., the visual guidance appeared on the screen when the applied force was 20 percent from the target level. This made the task easier but ensured that the whole range of force was covered.
3.2.3
Procedure
In the repetition test, the participant had to first reach the target force level by increasing the force as long as the white circle appeared on the screen, hold it for 300 ms and then release it. This was repeated five times, after which the participant was asked to repeat the same level another five times without the visual indicator. Each force level was presented once in each block resulting in 100 presses per block. Each participant went through four blocks one in each condition (N, A, AF, and AG). The order of the force levels was randomized, and the feedback conditions were counterbalanced.
Fig. 3. Examples of the raw force sensor data in the repetition (top) and hold (bottom) tests of the main experiment. The dashed line indicates the targets force level and the dotted lines the borders of the accepted range of the force. The section with gray background reflects the period when the visual guidance was not available.
The hold test was conducted after the repetition test with a short break in between. The procedure was similar to that of the repetition test, but instead of repeating the level several times the participant was asked to maintain the reached force level for a period of time. The target force level was indicated by the white circle, but this time participant had to stay on the target zone for 1 second. Then, the white circle disappeared and a 3-second hold period started. It was indicated by a short beep sound (800-Hz sine tone) and a “Hold...” text on the screen. The participants were instructed to maintain the reached level as accurately as possible until the hold period ends. The end of the hold period was indicated by “Release” text on the screen and another beep (400-Hz sine tone). Again, each of the 10 force levels was presented once in a block in a random order, and each of the four blocks was conducted with a different feedback condition.
3.3
Results
Because there was 20 percent tolerance in reaching the target force level, the obtained data were processed in the following way: In the repetition test, the reached peak values during the first five presses with the visual guidance (circle) were averaged and used as a reference point for the performance during the five presses without the visual guidance. The main measure of performance was the average absolute difference from the reference point. Similarly, in the hold test, the reference point was calculated as the average of the 1-second period with the visual guidance just before the hold period started. The measure of performance was the average absolute difference from the reference point. Fig. 3 illustrates examples of raw data from both repetition and hold tests.
IEEE TRANSACTIONS ON HAPTICS,
VOL. 6,
NO. 3,
JULY-SEPTEMBER 2013
379
Fig. 5. The effect of the repetition number and block order on the error of the repetition test.
Fig. 4. The median errors of the press force in all feedback conditions in the repetition test. The central marks in the boxes are the medians, the edges are the 25th and 75th percentiles, and the “+” signs outside of error bars are outliers.
The obtained data were analyzed with Friedman nonparametric analysis of variance. The pairwise comparisons were done with Wilcoxon rank sum test with Bonferroni correction. All the statistical analysis was done in Matlab. The tactile feedback condition had a significant effect on the average error, i.e., the difference of the force level from the reference point (2 ¼ 125:7; p < 0:01). The condition without any tactile feedback (N) yielded significantly the highest error, whereas the grained feedback (AG) yielded the lowest. The median errors are presented in Fig. 4 and the significance levels of the pairwise comparison test in Table 1. Feedback condition had also a significant effect on the duration of the presses (2 ¼ 233; p < 0:01). In the N condition, the durations were significantly shorter than in the other conditions. The number of repetitions and block number had significant influence on the performance. The error increased systematically when the visual guidance had been switched off. However, the effect of the repetition was independent of the feedback condition; there was no interaction effect between the condition and number of repetition revealed by three-way ANOVA (F ¼ 0:55; p ¼ 0:88). The effect of the block number was not systematic; first, the error slightly decreased, but in the final block, the performance was significantly lower than in the previous blocks. The average errors with respect of the repetition number and the block order are presented in Fig. 5. In the hold test, the condition did not have any influence on the average error (2 ¼ 0:59; p ¼ 0:90). Also, variance of the force during the hold period was not influenced by the feedback condition (2 ¼ 1:9; p ¼ 0:60). The average errors are presented in Fig. 6.
4
cues improved the accuracy of the force control. However, there are some concerns that need to be taken into account before generalizing these findings. A single press was very short in duration. Hence, the participants most probably did not have time to interpret the dynamic feedback during the press but rather afterward before the next press. During the visual guidance, they carefully explored the correct force range, but after the guidance had disappeared, they paid more attention to the tactile sensation and repeated the movement pattern as accurately as possible. Thus, the task can be considered a repetitive open-loop task, where each trial is executed based on the sensation of the previous trial. Furthermore, although the median press duration (1.06 seconds) was shorter than that of the hold test (1 þ 3 seconds), it was much longer than typical button presses on a mobile device (540-620 ms) [18] or a mouse click (100-120 ms) [19]. In these applications, the number of repetitions in long term use is very high, and an optimized press is achieved by recalling the sensory experience of the previous presses. The clear finding of the hold test can also be explained with this division between open- and closed-loop interactions. The force hold task, due to the longer duration of the trials, was more clearly a closed-loop interaction task. However, the participants were not able to use the feedback to improve the accuracy of the force control during the trial. These results suggest that in the reference use cases, such as long press or drag-and-drop interactions, dynamic feedback is not useful; a simple binary feedback reflecting the needed force level is enough. Our hypothesis was that the amplitude- and frequencymodulated feedback (AF) would lead into the best result in terms of error from the target force in the repetition test because both the amplitude and frequency of the vibration referred to the applied force level. However, the grained feedback (AG) yielded the lowest error in the repetition test. This result may be explained by a physical metaphor of applying force to an object. There are not many examples in the real world where surface vibrates more
DISCUSSION
The repetition test showed the potential of dynamic tactile feedback in force control. The results showed that if the task is to repeat a certain force level as accurately as possible, the dynamic
TABLE 1 Significance Levels of the Pairwise Comparison (Bolded Figures Indicate Significant Difference after Bonferroni Correction (p < 0:0083))
Fig. 6. The median error during the hold period in the hold test. None of the differences between the conditions are significant.
380
IEEE TRANSACTIONS ON HAPTICS,
intensively or frequently when it is pressed harder. Instead, such an object that discretely responses to increasing force is quite common. For example, pressing a button or pedal equipped with a spring often produces crackling sounds due to mechanical friction. This kind of familiar metaphor may have helped the participants to control the applied force although the grained feedback did not provide any cue for the absolute force level. Adaptation of the mechanoreceptors may also have played a role in the experiment. In the repetition and hold tests, the required press durations were 300 and 4,000 ms, respectively. Thus, in the hold test, the mechanoreceptors responsible for pressure sensation (Meissner Corpuscles and Merkel nerve endings) had much more time to adapt to the force compared to the repetition test. As indicated by Birznieks et al. [20], most of the adaptation occurs within 200 ms after the stimulus onset, but the slowly adapting receptors continue responding as long as the stimulation is active. The impact of this adaptation may also have influenced the perception of the feedback. Alternatively, the above-mentioned receptors may have been influenced by the vibration; even if the tactile feedback could have helped the force control, the pressure sensation itself could have been interfered by the vibration. This could explain the difference between the findings of the repetition and hold tests. The bandwidth limitations of the C2 actuator may have influenced the results. At least, the amplitude- and frequencymodulated feedback (AF) suffered from saturation of the amplitude at higher frequencies (higher force) of the feedback (Fig. 1). Also, the overall latency in the experiment setup may have played a role, especially in the difference between the findings of the repetition and hold experiments. This paper argues that in this context, a closed-loop task is more sensitive to latency than an open-loop task. The latency may also explain the duration of the presses in the repetition test. The duration of the presses in the tactile feedback conditions were, on average, 68 ms longer than in the control condition. The participants may have adapted the press durations according to the latency of the feedback (54 ms).
5
CONCLUSIONS AND FURTHER WORK
The current study proved that augmenting the applied force with dynamic tactile feedback improves the control of force in a force repetition task. Furthermore, the grain feedback model in which short tactile bursts were provided when the applied force changed yielded the best performance in the repetition test. These findings are in line with other studies regarding augmented feedback and motor control [11], [12], [13]. The feedback does help the user to understand the action better and repeat the movement pattern more accurately. Kildal [6] showed that a similar grain model can be used to create illusion of elasticity in touch experience. Now, together with results of the current study, the model forms an attractive tool to generate tactile feedback in touch- and forcebased interfaces. In contrast, none of the feedback models assisted the performance in the force hold task. This finding is different from the results of Jiang et al. [13] dealing with a force hold task. This conflict could be explained by the system latency and the sensory adaptation in the current experiment setup. These facts should be taken into focus in the further studies with more advanced hardware and stimulus design. The current study concentrated only on tactile feedback. Interaction with touch screens and touch pads, however, usually involves visual and audio feedback as well. Like in the real world, visual and auditory responses are congruent with the tactile feedback. This would form another opportunity for dynamic feedback design. Each of the modalities should be responding dynamically and concurrently, and they should be in accurate synchrony. It would be worth studying the role of each modality in interaction in terms of both the impression they evoke and the motor control accuracy.
VOL. 6,
NO. 3,
JULY-SEPTEMBER 2013
REFERENCES [1] [2]
[3]
[4]
[5]
[6] [7]
[8]
[9]
[10]
[11]
[12] [13]
[14] [15]
[16] [17]
[18]
[19]
[20]
G. Ramos, M. Boulos, and R. Balakrishnan, “Pressure Widgets,” Proc. SIGCHI Conf. Human Factors in Computing Systems, pp. 487-494, 2004. C. Stewart, M. Rohs, S. Kratz, and G. Essl, “Characteristics of PressureBased Input for Mobile Devices,” Proc. 28th Int’l Conf. Human Factors in Computing Systems, pp. 801-810, 2010. J. Rekimoto and C. Schwesig, “PreSenseII: Bi-Directional Touch and Pressure Sensing Interactions with Tactile Feedback,” Proc. CHI ’06 Extended Abstracts on Human Factors in Computing Systems, 2006. S. Mizobuchi, S. Terasaki, T. Keski-Jaskari, J. Nousiainen, M. Ryynanen, and M. Silfverberg, “Making an Impression: Force-Controlled Pen Input for Handheld Devices,” Proc. CHI ’05 Extended Abstracts on Human Factors in Computing Systems, pp. 1661-1664, 2005. J. Cechanowicz, P. Irani, and S. Subramanian, “Augmenting the Mouse with Pressure Sensitive Input,” Proc. SIGCHI Conf. Human Factors in Computing Systems, 2007. J. Kildal, “3D-Press: Haptic Illusion of Compliance When Pressing on a Rigid Surface,” Proc. 12th Int’l Conf. Multimodal Interfaces, 2010. L. Jones and E. Piateski, “Contribution of Tactile Feedback from the Hand to the Perception of Force,” Experimental Brain Research, vol. 168, pp. 298302, 2006. H. Henningsen, S. Knecht, and B. Ende-Henningsen, “Influence of Afferent Feedback on Isometric Fine Force Resolution in Humans,” Experimental Brain Research, vol. 113, pp. 207-213, 1997. J. Monzee, Y. Lamarre, and A.M. Smith, “The Effects of Digital Anesthesia on Force Control Using a Precision Grip,” J. Neurophysiology, vol. 89, pp. 672-683, 2001. S.R. Hurley and T.D. Lee, “The Influence of Augmented Feedback and Prior Learning on the Acquisition of a New Bimanual Coordination Pattern,” Human Movement Science, vol. 25, pp. 3394-348, 2006. K. Mononen, “The Effects of Augmented Feedback on Motor Skill Learning in Shooting,” PhD dissertation, Physical Education and Health, Univ. of Jyva¨skyla¨, Finland, 2007. M. Rath and D. Rocchesso, “Continuous Sonic Feedback from a Rolling Ball,” IEEE Multimedia, vol. 12, no. 2, pp. 60-69, Apr.-June 2005. L. Jiang, M.R. Cutkosky, J. Ruutiainen, and R. Raisamo, “Using Haptic Feedback to Improve Grasp Force Control in Multiple Sclerosis Patients,” IEEE Trans. Robotics, vol. 25, no. 3, pp. 593-601, June 2009. L.A. Jones, “Visual and Haptic Feedback in the Control of Force,” Experimental Brain Research, vol. 130, pp. 269-272, 2000. T. Ahmaniemi, “Gesture Controlled Virtual Instrument with Dynamic Vibrotactile Feedback,” Proc. Int’l Conf. New Interfaces for Musical Expression (NIME ’10), pp. 485-488, 2010. T. Ahmaniemi, J. Marila, and V. Lantz, “Design of Dynamic Vibrotactile Textures,” IEEE Trans. Haptics, vol. 3, no. 4, pp. 245-256, Oct.-Dec. 2010. T. Kaaresoja and S.A. Brewster, “Feedback Is. . . Late: Measuring Multimodal Delays in Mobile Device Touchscreen Interaction,” Proc. Int’l Conf. Multimodal Interfaces, 2010. R.S. Amant, T.E. Horton, and F.E. Ritter, “Model-Based Evaluation of Expert Cell Phone Menu Interaction,” ACM Trans. Computer-Human Interaction, vol. 14, article 1, 2007. S. Komandur, P.W. Johnson, and R.L. Storch, “Relation between Mouse Button Click Duration and Muscle Contraction Time,” Proc. 30th Ann. Int’l Conf. IEEE Eng. in Medicine and Biology Soc., pp. 2299-2301, 2008. I. Birznieks, P. Jenmalm, A.W. Goodwin, and R.S. Johansson, “Encoding of Direction of Fingertip Forces by Human Tactile Afferents,” J. Neuroscience, vol. 15, pp. 8222-8237, 2001.
. For more information on this or any other computing topic, please visit our Digital Library at www.computer.org/publications/dlib.
IEEE TRANSACTIONS ON HAPTICS,
Effect of Dynamic Vibrotactile Feedback on the Control of Isometric Finger Force Teemu Ahmaniemi Abstract—This study investigates how controllability of force is influenced by concurrent vibrotactile feedback that is generated proportionally to the applied force. Three different models to provide tactile feedback are introduced: amplitude and frequency modulation and granular synthesis. Then, an experiment investigating the effect of the feedback models on force control is reported. The tactile feedback conditions were compared to each other and to a condition with no feedback in a force repetition and a force hold tasks. In the force repetition task, all the feedback conditions yielded significantly better accuracy compared to no feedback condition. In the force hold task, there was no difference in the accuracy between the conditions including the no feedback condition. The results suggest that dynamic vibrotactile feedback assists the force control in force repetition tasks. Index Terms—Haptic I/O, input devices and strategies, augmented feedback, force sensing
Ç 1
INTRODUCTION
CONTROLLING applied force is very important in every day interactions. Holding and manipulating objects with hands necessitates accurate control of fingers in contact with the object. In human-computer interaction (HCI), touch- and force-based interfaces are becoming more popular. In touch screens or touch pads, the interaction is based on touch sensitive elements such as virtual buttons that can be touched with a finger or a stylus. Corresponding functions are triggered when a suitable amount of force is applied to them. The setup is rather different from a system with a real keyboard where the deformation or movement of the buttons informs the user about the triggering. In addition, button size, material, and stiffness can be felt only through the sense of touch. These features play an important role in the interaction efficiency and overall user experience. Beyond virtual buttons, applied force has been proposed to be used as an input method of the user interface. Ramos et al. [1] introduced force-controlled user-interface widgets such as expanding pie and pressure grid. In their experiments, force was applied with a pressure sensitive stylus on a touch pad. Force was proposed to be used in menu selection by Stewart et al. [2]. In their work, force was applied with a finger on mobile phone touch screen and measured with force sensitive resistors. A similar menu selection method was proposed by Rekimoto and Schwesig [3]. Force control in a pen-based interface was studied by Mizobuchi et al. [4]. They found out that users were able to discriminate five to seven levels of force within the range of 0 to 10 Newtons. Cechanowicz et al. [5] assembled force sensors on a desktop mouse and studied the use of force as an input method in mouse interaction. They found out that the number of controllable force levels was significantly higher when there was more than one sensor used, attached to different sides of the mouse. In the above studies, feedback played an important role in the force control accuracy. Whereas visual feedback had significant influence on the accuracy [4], [5], tactile feedback turned out to be . The author is with the Nokia Research Center, Nokia Corporation, PO Box 226, FI-00045 Nokia Group, Otaniementie 19, 02150 Espoo, Finland. E-mail: [email protected]. Manuscript received 8 Feb. 2012; revised 1 Nov. 2012; accepted 13 Nov. 2012; published online 5 Dec. 2012. Recommended for acceptance by S. Choi. For information on obtaining reprints of this article, please send e-mail to: [email protected], and reference IEEECS Log Number TH-2012-02-0014. Digital Object Identifier no. 10.1109/TOH.2012.72. 1939-1412/13/$31.00 ß 2013 IEEE
Published by the IEEE CS, RAS, & CES
VOL. 6,
NO. 3,
JULY-SEPTEMBER 2013
useful as well. In force-controlled menu selection studies [2], [3], a vibrotactile burst was provided when the applied force increased, and the next item in the menu was activated. In a target selection experiment, the users performed equally well or even better with tactile feedback than with audio feedback [2]. This study aims to take more detail look into the methods to provide tactile feedback: The feedback is provided proportionally to the applied force using amplitude or frequency modulation or granular synthesis. The main goal of this approach is to increase the controllability of force. An improvement in the control would improve the interaction efficiency and ergonomics of virtual button and menu selection examples above. In addition to the increased force control, the approach may provide more degrees of freedom for user-interface design. Continuous feedback could make the feeling of user-interface elements unique. This aspect was dealt in a study by Kildal [6]. He showed that dynamic tactile feedback reacting to applied force on a touch pad could generate illusions of elasticity, stiffness, and flexibility. The force was applied to the touch pad with a stylus that was equipped with a vibrotactile actuator. In the current study, different methods to provide the tactile feedback were compared in a force control experiment. The tasks in the experiment were derived from typical use cases in touch interfaces: 1) repeating a force level a certain number of times, as in a number or text entry task, and 2) holding the obtained force level for a certain period of time as in a long press of a virtual button or drag-and-drop interaction on a touch screen.
2
FORCE CONTROL AND FEEDBACK
The sense of touch and haptic feedback has shown to contribute to human force control. Jones and Piateski [7] showed that perception of force when produced by finger flexors is significantly overestimated when tactile feedback from the finger is attenuated. The attenuation was produced with splints molded around subjects’ fingertips. Henningsen et al. [8] indicated that when tactile feedback from fingers is damped with a finger nail or it is interfered with vibration, force control becomes coarser. Monzee et al. [9] used digital anesthesia for blocking tactile feedback from fingers and found that it significantly degrades the controllability of force. The above studies provide clear evidence for the importance of tactile feedback in force control. However, they solely concentrate on the natural “intrinsic” feedback, while the current study investigates the role of tactile feedback that is provided in addition to the natural feedback. When vibrotactile feedback is provided proportionally to the applied force, it basically augments the applied input. Augmented feedback has shown to have significant influence on human motor control when provided visually [10], [11], aurally [11], [12], or via sense of touch [13]. Jiang et al. [13] studied the effect of vibrotactile feedback on the control of grasping force with multiple sclerosis patients. The haptic feedback was either proportional to the absolute grasping force or indicated whether the force was within the desired range. Both of these conditions yielded significantly better results in terms of control accuracy than the condition without haptic feedback. The current study investigated the control of isometric force that was applied to a rigid surface with a finger. Augmented feedback was provided with a vibrotactile actuator that was colocated with the force sensor. The study attempted to identify an optimal feedback model for control accuracy in short-term uses. The setup was similar to the study by Jiang et al. [13], but the tasks in our experiment were derived from typical use scenarios in HCI.
3
EXPERIMENT
The experiment had two parts: force repetition and force hold. The aim in the experiment was similar to the study by Jiang et al. [13]
IEEE TRANSACTIONS ON HAPTICS,
VOL. 6,
NO. 3,
JULY-SEPTEMBER 2013
377
on amplitude-based feedback: to study how different models of augmented tactile feedback influence the repeatability of different force levels and maintaining a certain force level. The tactile feedback was proportional to the applied force, whereas the target itself (or target range) was presented visually. The participants were assumed to learn what the target level felt like, especially with dynamic tactile feedback as in the study by Jiang et al. [13]. The experiments were conducted by following a similar method as in recent force or grip control studies [7], [8], [9], [14]; the target force level was first presented visually, and the task was to repeat or sustain the force level without the visual guidance. This approach resembled the text- or number entry use cases introduced above: User can first learn the system behavior by looking at the interface while pressing the virtual buttons. Then, after a period of time, the pressing can be done in eyes free manner solely relying on the haptic sensation of the interaction.
3.1
Feedback Design
The tactile feedback design was based on our previous studies about tactile textures and continuous feedback [15], [16]. The feedback was generated with Pure Data software (http://puredata. info/). The main principle was to control the drive signal within the limits of the selected actuator. The first approach was to modify either the amplitude or the frequency of the feedback signal based on the applied force. The second approach was to provide feedback as discrete bursts (grains) whenever the applied force changed similarly to the study by Kildal [6]. The experiment consisted of four feedback conditions: three different tactile feedback designs and one without any feedback (control condition, N). The three feedback designs were the following: .
A. Amplitude-modulated vibration of 250 Hz. The base frequency was the nominal frequency of the actuator, and the amplitude range was chosen so that the actual vibration amplitude (A) of the device increased linearly with force: A ¼ kA F ;
ð1Þ
g N,
.
and F is applied force. where kA ¼ 0:58 AF . Amplitude- and frequency-modulated “textured” vibration increasing in amplitude and envelope frequency along the applied force. The design was similar to our previous study [16] with vibrotactile textures where the nominal frequency of the actuator was modulated with an envelope signal. The amplitude response was similar to the A condition (1) and the envelope frequency (f) was increased by following a linear equation: f ¼ kf F ;
ð2Þ
where kf ¼ 11:1 Hz N , and F is applied force. AG. Grained feedback. The whole range of force was divided into 26 levels, and a discrete vibration burst (grain) was provided when the force level changed. The frequency of the vibration was 250 Hz and the duration 40 ms. The amplitude of the burst increased linearly with force following the same equation as in the A and AF conditions (1). The coefficients kA and kf were chosen so that the amplitude and frequency ranges of the tactile actuator were covered optimally. The fundamental difference between the feedback models was that in A and AF, the vibration was continuous as long as force was applied but in AG vibration bursts were provided only when the force changed. An illustration of the feedback conditions is shown in Fig. 1. The signals correspond to the applied force that first increases linearly, stays constant for 1 second, and then starts increasing again. The figure shows the acceleration of the vibration on the top of the device. .
Fig. 1. The feedback models in the experiments. The simulated force signal first increases linearly, then remains constant for 1 second, and then increases again. The difference between the grained vibration (AG) and the other models (A and AF ) was that in the grain model there was no feedback if the force level remained constant, whereas in other models constant force was responded with continuous vibration of constant amplitude or frequency.
Since the amplitude- and frequency-modulated feedback (AF) provided two dimensions of feedback related to the applied force level, the amplitude and the envelope frequency, it was expected to yield the best results in force control. The grain feedback (AG), on the other hand, indicated only how to get to a certain level, i.e., how many grains needed to be felt to reach it. Although the grains’ amplitude increased with applied force, there was no continuous feedback reflecting the absolute force level. Therefore, the grained feedback was not expected to assist the force control.
3.2
Experiment Setup
The experiment was arranged in a quiet office room equipped with a chair, a table, a PC with monitor (2200 ), and sensor-actuator hardware. Twenty-four subjects (four women, 20 men, mean age 37) participated in the experiment. The experiment took approximately 45 minutes and was compensated with a cinema ticket worth of 10 euros.
3.2.1
Hardware
A capacitive force sensor (Loadstar LoadVUE Lite) was firmly packed into a custom made box (38 38 25 mm) with a tactile actuator (C2). The sensor provided 0-10 lbs (0-40 N) force readings with 500-Hz sampling rate within a resolution of 0.004 N. The sensor was connected to a laptop PC (Lenovo W510) through an interface module (Loadstar DQ-1000U) provided by the sensor manufacturer. The tactile actuator was connected to the PC’s soundcard through a stereo amplifier (NAD 312). An illustration of this setup is presented in Fig. 2. The sensor values were captured by a USB serial interface and routed to Pure Data synthesis software. The frequency response of C2 tactile actuator was rather narrow, 200-300 Hz. This limitation caused saturation of the signal amplitude with higher frequencies. This impacted the amplitude response of the AF feedback condition as shown in Fig. 1. The endto-end latency of the system was measured with a multimodal
378
IEEE TRANSACTIONS ON HAPTICS,
VOL. 6,
NO. 3,
JULY-SEPTEMBER 2013
Fig. 2. The diagram of the system (top left) consisted of a laptop computer running Pure Data for feedback synthesis, stereo amplifier, and sensor-actuator device. The sensor box (exploded view on top right) was placed on a rack that stood on an iron frame. The participant applied force with her/his index finger of the dominant hand.
latency measurement tool by Kaaresoja and Brewster [17] yielding 54 ms. The latency was introduced by the sensor sampling, serial interface, feedback synthesis, audio hardware buffering, and the actuator rise time.
3.2.2
Visual Guidance and Force Levels
All the visuals in the experiment were rendered with Graphic Environment for Multimedia plug-in for Pure Data software. To minimize the effect of visual guidance in task performance, only a white solid circle appeared on the screen when the applied force on the sensor was within a 20 percent range from the target. In the repetition test, the circle turned green when the force had been within the target zone for 300 ms (see Fig. 2). This ensured that participants had intentionally reached the zone. A press was considered complete when the circle had turned green and the force was released. In the hold test, the white circle disappeared after being within the zone for 1 second. After this, the task was to maintain the reached force level for 3 seconds. Ten different target force levels equidistantly distributed between 0.7 and 4.0 N were used, but 20 percent error in both directions was allowed, i.e., the visual guidance appeared on the screen when the applied force was 20 percent from the target level. This made the task easier but ensured that the whole range of force was covered.
3.2.3
Procedure
In the repetition test, the participant had to first reach the target force level by increasing the force as long as the white circle appeared on the screen, hold it for 300 ms and then release it. This was repeated five times, after which the participant was asked to repeat the same level another five times without the visual indicator. Each force level was presented once in each block resulting in 100 presses per block. Each participant went through four blocks one in each condition (N, A, AF, and AG). The order of the force levels was randomized, and the feedback conditions were counterbalanced.
Fig. 3. Examples of the raw force sensor data in the repetition (top) and hold (bottom) tests of the main experiment. The dashed line indicates the targets force level and the dotted lines the borders of the accepted range of the force. The section with gray background reflects the period when the visual guidance was not available.
The hold test was conducted after the repetition test with a short break in between. The procedure was similar to that of the repetition test, but instead of repeating the level several times the participant was asked to maintain the reached force level for a period of time. The target force level was indicated by the white circle, but this time participant had to stay on the target zone for 1 second. Then, the white circle disappeared and a 3-second hold period started. It was indicated by a short beep sound (800-Hz sine tone) and a “Hold...” text on the screen. The participants were instructed to maintain the reached level as accurately as possible until the hold period ends. The end of the hold period was indicated by “Release” text on the screen and another beep (400-Hz sine tone). Again, each of the 10 force levels was presented once in a block in a random order, and each of the four blocks was conducted with a different feedback condition.
3.3
Results
Because there was 20 percent tolerance in reaching the target force level, the obtained data were processed in the following way: In the repetition test, the reached peak values during the first five presses with the visual guidance (circle) were averaged and used as a reference point for the performance during the five presses without the visual guidance. The main measure of performance was the average absolute difference from the reference point. Similarly, in the hold test, the reference point was calculated as the average of the 1-second period with the visual guidance just before the hold period started. The measure of performance was the average absolute difference from the reference point. Fig. 3 illustrates examples of raw data from both repetition and hold tests.
IEEE TRANSACTIONS ON HAPTICS,
VOL. 6,
NO. 3,
JULY-SEPTEMBER 2013
379
Fig. 5. The effect of the repetition number and block order on the error of the repetition test.
Fig. 4. The median errors of the press force in all feedback conditions in the repetition test. The central marks in the boxes are the medians, the edges are the 25th and 75th percentiles, and the “+” signs outside of error bars are outliers.
The obtained data were analyzed with Friedman nonparametric analysis of variance. The pairwise comparisons were done with Wilcoxon rank sum test with Bonferroni correction. All the statistical analysis was done in Matlab. The tactile feedback condition had a significant effect on the average error, i.e., the difference of the force level from the reference point (2 ¼ 125:7; p < 0:01). The condition without any tactile feedback (N) yielded significantly the highest error, whereas the grained feedback (AG) yielded the lowest. The median errors are presented in Fig. 4 and the significance levels of the pairwise comparison test in Table 1. Feedback condition had also a significant effect on the duration of the presses (2 ¼ 233; p < 0:01). In the N condition, the durations were significantly shorter than in the other conditions. The number of repetitions and block number had significant influence on the performance. The error increased systematically when the visual guidance had been switched off. However, the effect of the repetition was independent of the feedback condition; there was no interaction effect between the condition and number of repetition revealed by three-way ANOVA (F ¼ 0:55; p ¼ 0:88). The effect of the block number was not systematic; first, the error slightly decreased, but in the final block, the performance was significantly lower than in the previous blocks. The average errors with respect of the repetition number and the block order are presented in Fig. 5. In the hold test, the condition did not have any influence on the average error (2 ¼ 0:59; p ¼ 0:90). Also, variance of the force during the hold period was not influenced by the feedback condition (2 ¼ 1:9; p ¼ 0:60). The average errors are presented in Fig. 6.
4
cues improved the accuracy of the force control. However, there are some concerns that need to be taken into account before generalizing these findings. A single press was very short in duration. Hence, the participants most probably did not have time to interpret the dynamic feedback during the press but rather afterward before the next press. During the visual guidance, they carefully explored the correct force range, but after the guidance had disappeared, they paid more attention to the tactile sensation and repeated the movement pattern as accurately as possible. Thus, the task can be considered a repetitive open-loop task, where each trial is executed based on the sensation of the previous trial. Furthermore, although the median press duration (1.06 seconds) was shorter than that of the hold test (1 þ 3 seconds), it was much longer than typical button presses on a mobile device (540-620 ms) [18] or a mouse click (100-120 ms) [19]. In these applications, the number of repetitions in long term use is very high, and an optimized press is achieved by recalling the sensory experience of the previous presses. The clear finding of the hold test can also be explained with this division between open- and closed-loop interactions. The force hold task, due to the longer duration of the trials, was more clearly a closed-loop interaction task. However, the participants were not able to use the feedback to improve the accuracy of the force control during the trial. These results suggest that in the reference use cases, such as long press or drag-and-drop interactions, dynamic feedback is not useful; a simple binary feedback reflecting the needed force level is enough. Our hypothesis was that the amplitude- and frequencymodulated feedback (AF) would lead into the best result in terms of error from the target force in the repetition test because both the amplitude and frequency of the vibration referred to the applied force level. However, the grained feedback (AG) yielded the lowest error in the repetition test. This result may be explained by a physical metaphor of applying force to an object. There are not many examples in the real world where surface vibrates more
DISCUSSION
The repetition test showed the potential of dynamic tactile feedback in force control. The results showed that if the task is to repeat a certain force level as accurately as possible, the dynamic
TABLE 1 Significance Levels of the Pairwise Comparison (Bolded Figures Indicate Significant Difference after Bonferroni Correction (p < 0:0083))
Fig. 6. The median error during the hold period in the hold test. None of the differences between the conditions are significant.
380
IEEE TRANSACTIONS ON HAPTICS,
intensively or frequently when it is pressed harder. Instead, such an object that discretely responses to increasing force is quite common. For example, pressing a button or pedal equipped with a spring often produces crackling sounds due to mechanical friction. This kind of familiar metaphor may have helped the participants to control the applied force although the grained feedback did not provide any cue for the absolute force level. Adaptation of the mechanoreceptors may also have played a role in the experiment. In the repetition and hold tests, the required press durations were 300 and 4,000 ms, respectively. Thus, in the hold test, the mechanoreceptors responsible for pressure sensation (Meissner Corpuscles and Merkel nerve endings) had much more time to adapt to the force compared to the repetition test. As indicated by Birznieks et al. [20], most of the adaptation occurs within 200 ms after the stimulus onset, but the slowly adapting receptors continue responding as long as the stimulation is active. The impact of this adaptation may also have influenced the perception of the feedback. Alternatively, the above-mentioned receptors may have been influenced by the vibration; even if the tactile feedback could have helped the force control, the pressure sensation itself could have been interfered by the vibration. This could explain the difference between the findings of the repetition and hold tests. The bandwidth limitations of the C2 actuator may have influenced the results. At least, the amplitude- and frequencymodulated feedback (AF) suffered from saturation of the amplitude at higher frequencies (higher force) of the feedback (Fig. 1). Also, the overall latency in the experiment setup may have played a role, especially in the difference between the findings of the repetition and hold experiments. This paper argues that in this context, a closed-loop task is more sensitive to latency than an open-loop task. The latency may also explain the duration of the presses in the repetition test. The duration of the presses in the tactile feedback conditions were, on average, 68 ms longer than in the control condition. The participants may have adapted the press durations according to the latency of the feedback (54 ms).
5
CONCLUSIONS AND FURTHER WORK
The current study proved that augmenting the applied force with dynamic tactile feedback improves the control of force in a force repetition task. Furthermore, the grain feedback model in which short tactile bursts were provided when the applied force changed yielded the best performance in the repetition test. These findings are in line with other studies regarding augmented feedback and motor control [11], [12], [13]. The feedback does help the user to understand the action better and repeat the movement pattern more accurately. Kildal [6] showed that a similar grain model can be used to create illusion of elasticity in touch experience. Now, together with results of the current study, the model forms an attractive tool to generate tactile feedback in touch- and forcebased interfaces. In contrast, none of the feedback models assisted the performance in the force hold task. This finding is different from the results of Jiang et al. [13] dealing with a force hold task. This conflict could be explained by the system latency and the sensory adaptation in the current experiment setup. These facts should be taken into focus in the further studies with more advanced hardware and stimulus design. The current study concentrated only on tactile feedback. Interaction with touch screens and touch pads, however, usually involves visual and audio feedback as well. Like in the real world, visual and auditory responses are congruent with the tactile feedback. This would form another opportunity for dynamic feedback design. Each of the modalities should be responding dynamically and concurrently, and they should be in accurate synchrony. It would be worth studying the role of each modality in interaction in terms of both the impression they evoke and the motor control accuracy.
VOL. 6,
NO. 3,
JULY-SEPTEMBER 2013
REFERENCES [1] [2]
[3]
[4]
[5]
[6] [7]
[8]
[9]
[10]
[11]
[12] [13]
[14] [15]
[16] [17]
[18]
[19]
[20]
G. Ramos, M. Boulos, and R. Balakrishnan, “Pressure Widgets,” Proc. SIGCHI Conf. Human Factors in Computing Systems, pp. 487-494, 2004. C. Stewart, M. Rohs, S. Kratz, and G. Essl, “Characteristics of PressureBased Input for Mobile Devices,” Proc. 28th Int’l Conf. Human Factors in Computing Systems, pp. 801-810, 2010. J. Rekimoto and C. Schwesig, “PreSenseII: Bi-Directional Touch and Pressure Sensing Interactions with Tactile Feedback,” Proc. CHI ’06 Extended Abstracts on Human Factors in Computing Systems, 2006. S. Mizobuchi, S. Terasaki, T. Keski-Jaskari, J. Nousiainen, M. Ryynanen, and M. Silfverberg, “Making an Impression: Force-Controlled Pen Input for Handheld Devices,” Proc. CHI ’05 Extended Abstracts on Human Factors in Computing Systems, pp. 1661-1664, 2005. J. Cechanowicz, P. Irani, and S. Subramanian, “Augmenting the Mouse with Pressure Sensitive Input,” Proc. SIGCHI Conf. Human Factors in Computing Systems, 2007. J. Kildal, “3D-Press: Haptic Illusion of Compliance When Pressing on a Rigid Surface,” Proc. 12th Int’l Conf. Multimodal Interfaces, 2010. L. Jones and E. Piateski, “Contribution of Tactile Feedback from the Hand to the Perception of Force,” Experimental Brain Research, vol. 168, pp. 298302, 2006. H. Henningsen, S. Knecht, and B. Ende-Henningsen, “Influence of Afferent Feedback on Isometric Fine Force Resolution in Humans,” Experimental Brain Research, vol. 113, pp. 207-213, 1997. J. Monzee, Y. Lamarre, and A.M. Smith, “The Effects of Digital Anesthesia on Force Control Using a Precision Grip,” J. Neurophysiology, vol. 89, pp. 672-683, 2001. S.R. Hurley and T.D. Lee, “The Influence of Augmented Feedback and Prior Learning on the Acquisition of a New Bimanual Coordination Pattern,” Human Movement Science, vol. 25, pp. 3394-348, 2006. K. Mononen, “The Effects of Augmented Feedback on Motor Skill Learning in Shooting,” PhD dissertation, Physical Education and Health, Univ. of Jyva¨skyla¨, Finland, 2007. M. Rath and D. Rocchesso, “Continuous Sonic Feedback from a Rolling Ball,” IEEE Multimedia, vol. 12, no. 2, pp. 60-69, Apr.-June 2005. L. Jiang, M.R. Cutkosky, J. Ruutiainen, and R. Raisamo, “Using Haptic Feedback to Improve Grasp Force Control in Multiple Sclerosis Patients,” IEEE Trans. Robotics, vol. 25, no. 3, pp. 593-601, June 2009. L.A. Jones, “Visual and Haptic Feedback in the Control of Force,” Experimental Brain Research, vol. 130, pp. 269-272, 2000. T. Ahmaniemi, “Gesture Controlled Virtual Instrument with Dynamic Vibrotactile Feedback,” Proc. Int’l Conf. New Interfaces for Musical Expression (NIME ’10), pp. 485-488, 2010. T. Ahmaniemi, J. Marila, and V. Lantz, “Design of Dynamic Vibrotactile Textures,” IEEE Trans. Haptics, vol. 3, no. 4, pp. 245-256, Oct.-Dec. 2010. T. Kaaresoja and S.A. Brewster, “Feedback Is. . . Late: Measuring Multimodal Delays in Mobile Device Touchscreen Interaction,” Proc. Int’l Conf. Multimodal Interfaces, 2010. R.S. Amant, T.E. Horton, and F.E. Ritter, “Model-Based Evaluation of Expert Cell Phone Menu Interaction,” ACM Trans. Computer-Human Interaction, vol. 14, article 1, 2007. S. Komandur, P.W. Johnson, and R.L. Storch, “Relation between Mouse Button Click Duration and Muscle Contraction Time,” Proc. 30th Ann. Int’l Conf. IEEE Eng. in Medicine and Biology Soc., pp. 2299-2301, 2008. I. Birznieks, P. Jenmalm, A.W. Goodwin, and R.S. Johansson, “Encoding of Direction of Fingertip Forces by Human Tactile Afferents,” J. Neuroscience, vol. 15, pp. 8222-8237, 2001.
. For more information on this or any other computing topic, please visit our Digital Library at www.computer.org/publications/dlib.
