Exp Brain Res DOI 10.1007/s00221-015-4403-9

RESEARCH ARTICLE

Relative errors can cue absolute visuomotor mappings Loes C. J. van Dam1,2 · Marc O. Ernst1,2,3 

Received: 26 April 2015 / Accepted: 3 August 2015 © Springer-Verlag Berlin Heidelberg 2015

Abstract  When repeatedly switching between two visuomotor mappings, e.g. in a reaching or pointing task, adaptation tends to speed up over time. That is, when the error in the feedback corresponds to a mapping switch, fast adaptation occurs. Yet, what is learned, the relative error or the absolute mappings? When switching between mappings, errors with a size corresponding to the relative difference between the mappings will occur more often than other large errors. Thus, we could learn to correct more for errors with this familiar size (Error Learning). On the other hand, it has been shown that the human visuomotor system can store several absolute visuomotor mappings (Mapping Learning) and can use associated contextual cues to retrieve them. Thus, when contextual information is present, no error feedback is needed to switch between mappings. Using a rapid pointing task, we investigated how these two types of learning may each contribute when repeatedly switching between mappings in the absence of task-irrelevant contextual cues. After training, we examined how participants changed their behaviour when a single error probe indicated either the often-experienced error (Error Learning) or one of the previously experienced absolute mappings (Mapping Learning). Results were consistent with Mapping Learning despite the relative nature of the error information in the feedback. This shows that

* Loes C. J. van Dam Loes.van_Dam@uni‑bielefeld.de 1

Department of Cognitive Neuroscience, Universität Bielefeld, Universitätsstraße 25, 33615 Bielefeld, Germany

2

Cognitive Interaction Technology (CITEC) Center of Excellence, Universität Bielefeld, Bielefeld, Germany

3

Bernstein Center for Computational Neuroscience, Tübingen, Germany



errors in the feedback can have a double role in visuomotor behaviour: they drive the general adaptation process by making corrections possible on subsequent movements, as well as serve as contextual cues that can signal a learned absolute mapping. Keywords  Perception and action · Visuomotor learning · Dual adaptation · Error Learning · Mapping Learning

Introduction In daily life, we frequently use different sensorimotor mappings which require us to be able to rapidly switch between them. For instance, when a person puts on a pair glasses in the morning, the optical correction in the glasses do not only allow him to see better, but it also creates distortions in the geometry of the visual input. This change in the sensory input means that the interactions between the senses and the motor system are changed and thus behaviour has to be updated accordingly. This discrepancy between the senses is what causes us to be dizzy the first time a new pair of glasses is worn. But after a few days of wearing them, repeatedly putting them on and taking them off, humans are able to switch apparently instantaneously to the corresponding mapping when the glasses are put on. That is, humans learn the association between the glasses and the sensorimotor mappings that go with it, and putting on the glasses serves as a cue to adopt these mappings (Kravitz 1972; Kravitz and Yaffe 1974; Martin et al. 1996; Welch 1971). It has been well established that we can learn such associative cues to help us switch between different visuomotor mappings when the mapping change consists of a simple shift in behaviour, e.g. when wearing prism glasses

13



(Donderi et al. 1985; Fernandez-Ruiz et al. 2000, 2006; Kravitz 1972; Kravitz and Yaffe 1972, 1974; Martin et al. 1996; Pick et al. 1969; Seidler et al. 2001; van Dam et al. 2013; van Dam and Ernst 2015; Welch 1971). Such associative cues help us to identify the correct mapping that should be used even before a movement is initiated. However, there are also situations in which a change in the required visuomotor mapping is not a priori evident by an associated cue. For instance, when rapidly lifting a milk jug, there is often no a priori information whether the jug will be full or empty, but depending on the content of the jug different forces have to be applied. If too much force is applied (e.g. when it is empty) we may send it flying. Too little force when it is full, would scarcely result in a successful lift. The question is what will happen if a person continues to lift similar jugs that are either full or empty (and e.g. are never half full) in a rapid fashion? Without a priori information about the current situation (full or empty), would he continue to treat each lift like the very first one, or would he start using the information that certain states occur more often than others (i.e. the jug is either full or empty and never half full)? The literature does not provide clear evidence whether separate visuomotor mappings can be learned simultaneously, in the absence of a priori contextual (task-irrelevant) cues. For instance, when two mappings are alternated in quick succession, either by presenting them in a random order or simply alternating the mappings on each next trial, learning the separate mappings does not seem to occur in the absence of context cues (e.g. Karniel and Mussa-Ivaldi 2002; Hinder et al. 2008; Ernst 2001). In this case, participants often adopt a single average mapping rather than learning the two separate mappings. On the other hand, when two separate visuomotor mappings are presented in a blocked fashion using the classical prism adaptation paradigm, participants do learn to switch more rapidly between two mappings without the apparent need for additional context cues (e.g. Bingham and Romack 1999; Flook and McGonigle 1977; McGonigle and Flook 1978; Welch et al. 1993). This supports the idea that allowing participants to fully adapt to and experience the separate mappings is the key for learning the mappings and to switch between them without external cues. However, since these studies were performed using physical prisms it is not clear to what extent unintended contextual cues from either the prisms themselves or from having to physically exchange the prisms when switching mappings (e.g. introducing pauses in the paradigm etc.) may have been informative in an a priori contextual cue manner for switching between the mappings. For prism adaptation, it can even be argued (see e.g. Bingham and Romack 1999; Redding and Wallace 2003) that additional associative cues may arise from the visual colour and shape distortions caused by the prisms.

13

Exp Brain Res

For this and other reasons, it is still very much debated whether separate mappings can be learned at all in the absence of task-irrelevant associative cues (Bingham and Romack 1999; Redding and Wallace 2003; Woolley et al. 2007). Here, we address this question using a rapid target pointing task in which we manipulated the feedback that participants received about their movement endpoints. To ensure that no task-irrelevant associative cues were available, we used a virtual reality interface, in which visuomotor mappings can be changed without physically modifying the set-up or adding physical visual distortions that would otherwise be informative of the current mapping through associative learning. To introduce the separate mappings, we simply shifted the visual feedback that participants received about their movement endpoints. Using this feedback manipulation, participants were exposed to two different mappings (Figs. 1a, 2c) and they could not know when a switch between them had occurred until after making the first movement in the changed mapping. However, the feedback received after that first movement, that is the error between movement endpoint and target, does provide information about how the current behavioural response mapping should be updated in order to be more successful on the next movement. Do participants learn to use this information more effectively over time and if so in what way? Here, there are two possibilities: Error Learning and Mapping Learning. First, participants can learn to adjust for certain large errors, relating to the switches between the two mappings, more quickly (Error Learning), thus speeding up the error correction process for specific sizes of the error (Fig. 1a, b). The initial error observed in the feedback after a mapping switch directly depends on the relative difference between the two mappings before and after the switch. Thus, when repeatedly switching between the two mappings, large errors directly related to a switch will be experienced more often than other large errors. The visuomotor system could therefore learn to correct for certain sizes of errors more quickly than differently sized errors and in such a way adapt more and more quickly each time a mapping switch occurs (Error Learning). In this case, the size of a correction on a next movement would be dependent on the experienced error only, and for instance be independent of the actual mapping currently used. This type of learning would be consistent with learning a prior distribution of occurring errors (see e.g. Körding and Wolpert 2004) with a peak at the error corresponding to the relative difference between the two mappings. Alternatively, participants could learn the two different absolute mappings (Fig. 1c). When information is present indicating which motor program is currently appropriate, they can simply apply it without needing to go through the iterative correction process to correct for the error

Exp Brain Res mapping 1 mapping 2 learned error

time

a

Pointing Location mapping switch

Feedback Error Learning Mapping Learning

novel adaptation

time

b

mapping switch

learned error

?

time

c mapping switch

Unfamiliar error

? Fig.  1  a During training, participants repeatedly switch between mapping 1 and mapping 2. This means that they are repeatedly exposed to the same size of error in the feedback of their pointing behaviour. In principle, both Mapping Learning and Error Learning could occur. Fingertips indicate pointing behaviour; circles indicate feedback. b Error Learning prediction when the familiar error in the feedback is presented after adapting to a novel mapping. Since the error does not indicate one of the learned mappings, a singular Mapping Learning prediction cannot be made, but shifts in behaviour are expected to be small. c Mapping Learning prediction when the feedback indicates a switch to one of the previously learned mappings. Here, no singular prediction for Error Learning is possible, but again shifts in behaviour are expected to be small

(Absolute Mapping Learning). The initial feedback of the first trial after a mapping change could in principle be used to identify which of the two learned mappings is currently appropriate. Provided that participants are able to distinguish errors related to a mapping switch, from errors due to the participant’s own motor noise, the observed error could thus serve as a cue, in a similar fashion as contextual cues, to switch to one of the previously experienced mappings. This would mean that two absolute mappings are learned and the error signal, despite its relative nature, is used as a “contextual cue” to identify the currently correct one. Such

Mapping Learning has been shown previously, for instance, when using contextual target colour cues associated to each mapping (van Dam et al. 2013). To distinguish between the two possibilities of Error and Mapping Learning, we investigated how the learning generalizes to similar, but novel situations. That is, after training (a 5-day period in which participants repeatedly switched between the two trained mappings), we introduced a third novel mapping that had not been previously experienced (Figs.  1b, c, 2e, f). Participants adapted to this novel mapping, after which we presented them with what we will call the Probe and Catch Trial method (Fig. 2g). The probe trial is a single trial in which the feedback corresponds to a different mapping compared to the adapted mapping, and is followed by several catch trials without feedback to investigate the effectiveness of the probe in terms of evoking a shift in behaviour. To probe for the possible types of learning, probe trials were either consistent with one of the previously learned mappings (to probe Mapping Learning) or represented the familiar error as observed during training (to probe Error Learning). If the probe trial is consistent with the way in which learning occurred (Error Learning or Mapping Learning), the probe trial should be very effective in eliciting the corresponding change in pointing behaviour. However, if the probe trial is inconsistent with learning, the effectiveness of the single probe can be expected to be much smaller. Thus, using the Probe and Catch Trial method, we can expect different results depending on whether Error Learning or Mapping Learning occurred. Error Learning only depends on the size of the error presented and thus on the difference between the current mapping (before the probe) and the probed mapping (represented by the probe), but not their absolute values. That is, regardless of the absolute value of the novel mapping that was adapted to, a quick change in response is expected whenever the size of the error corresponds to the familiar error size as observed during training (Fig. 1b). In the case of Mapping Learning, the effectiveness of a probe for producing a change in behaviour should depend only on the current mapping being probed, i.e. its absolute value, and should be independent of the difference with respect to the current mapping (Fig. 1c). Our results showed, first of all, that learning two visuomotor mappings is possible in the absence of task-irrelevant contextual cues. Second, learning was most consistent with Mapping Learning as was previously found for taskirrelevant contextual cues (van Dam et al. 2013). Together the results indicate that visuomotor feedback, besides being used for making relative corrections in an adaptive manner, can serve as a “contextual cue” for switching between previously learned absolute mappings.

13



Exp Brain Res

b

53.5 cm

c

e Tim

e

trial with feedback (novel mapping)

...

Relative Error

... mapping 1 (0 deg)

...

... Relative Error

Mapping

...

f (12.8 deg)

catch trial (no feedback)

feedback displace -6.4 deg required mapping: tapped - target location = 6.4 deg

starting point tablet stylus

until participant starts trial

trial with feedback (trained mapping)

mapping 2 (6.4 deg)

-6.4 deg

Relative Error

d

mapping

until movement is complete

0.5 sec Tablet

Tablet

veridical feedback.

0.5 sec

Screen

Screen

mapping 2

mapping 2 mapping 1 Relative Error

a

catch trials (-6.4 deg)

...

Trial number

xp Mapping

h

probe

g

Trial number trial block

training

0 deg

6.4 deg

test 12.8 deg

probe effectiveness = xr/xp Trial

6.4 deg

19.2

xr

12.8

test -6.4 deg

-6.4

12.8

-6.4 6.4

Fig.  2  a Experimental set-up and dimensions used. b Time course of a single trial. Participants started the trial by tapping at the start location after which a target was shown. The trial was ended when the participant tapped on the tablet again to indicate the target location and the visual feedback was shown. c Two mappings were being trained. One had veridical feedback for a 0◦ mapping (top). For the second mapping (bottom), the visual feedback was horizontally displaced by −6.4◦ (6 cm) leading to a required behavioural mapping of 6.4◦. d Training schematic. Participants repeatedly switched between mapping 1 (0◦ mapping) and mapping 2 (6.4◦). Potentially they could learn both the mappings directly (Mapping Learning) and/or their relative difference (Error Learning). e, f Test conditions on Day 5. After training, participants adapted to either a 12.8◦ mapping (e) or a −6.4◦ mapping (f). Next, we measured the shifts in behaviour for Error Learning, Mapping Learning and Combined probes. Probes in e represent Combined and Error Learning probes, respectively. Probes in f indicate Combined and Mapping Learning probes, respectively. Arrows and circles indicate possible contributions of Error Learning

13

0 deg

6.4 deg

probe trials 0 deg

19.2

6.4

0

Trial number

break 0 deg

12.8

12.8

-6.4

-6.4 6.4

6.4 deg

0 deg

6.4

}

probe learned error alone

}

probe learned error & learned mapping combined

}

probe learned mapping alone

0 deg

catch trials (no feedback)

0

6.4

0

...

0

and Mapping Learning, respectively. g The probe effectiveness indicates to which extent the learned error or mapping information was being used. The effectiveness was measured by dividing the behavioural shift xr (thick arrow showing the difference in mean behaviour across five trials before and after the probe) by the experimentally probed change in the mapping xp (thin arrow). h Schematic of training and test days for the cue conditions. On training days (top), there were five blocks that each start with mapping 1 (between 20 and 40 trials) then switch to mapping 2 (40 trials) and back again (100 trials). The last block on each training day contained probe and catch trials. On the probe trial, feedback corresponded to the change in required mapping. Catch trials in which no feedback was provided directly followed the probe. On Day 5, the 3rd and 5th blocks were test blocks in which the participant adapted to either a 12.8◦ (middle) or −6.4◦ mapping (bottom) before probe and catch trials were introduced (see also e and f). Initial adaptation took between 70 and 90 trials; top-up adaptation between probe and catch trials periods 35–45 trials and washout to 0◦ at the end of the test block was 50 trials

Exp Brain Res

General methods Apparatus Stimuli were displayed on a large back-projection screen (220 by 176 cm) in an otherwise dark room. Participants were seated behind a custom-made rack (see Fig. 2a). The pointing behaviour of the participants was recorded using a graphics tablet (WACOM Intuos 3 A3-wide; active area 48.8 by 30.5 cm and a grip pen) placed on the first level of the rack. A second level, draped in black cloth, prevented the participants from seeing their own arm or the graphics tablet. The head movements of the participants were restricted by a chin rest and the viewing distance was 53.5 cm. The visual stimuli were implemented in C using OpenGL libraries and using an intermediate grey background colour. Distances for targets and feedback on the screen were scaled one-to-one to distances on the tablet, meaning that for target placement only a small portion at the centre of the screen was used. Stimulus, task, and trial procedure Participants used their preferred hand for pointing. They held the grip pen in a full grasp to ensure that the posture of the hand relative to the tablet would be more or less constant across training days. Individual trials were initiated by tapping with the pen within the start zone which was a semi-circular area (radius of 25 mm) centred on the lower edge of the tablet (see Fig. 2b). A small bump on the graphics tablet haptically indicated this starting position. Participants were told to position their non-preferred hand on this bump so as to provide an additional reference for finding this starting position with the preferred hand used for pointing, without the need to see their hand or the bump. Before each trial start, a white horizontal line on the screen provided a visual reference for the vertical height of the starting position. This horizontal line was displayed along the whole width of the screen, such that it did not provide a visual reference for the horizontal direction (in which we manipulated the visual feedback). Upon trial initiation, a target, which was a circular disc with radius 0.75°, was displayed for 0.5 s. After this, both the target and the white reference line disappeared. Participants’ task was to tap on the location of the graphics tablet corresponding to the visual target location as accurately and as quickly as possible. When the movement was complete (i.e. the participant landed on the tablet), feedback about the tapped location was displayed in form of a high contrast Gaussian blob (standard deviation of 3.0◦) at the corresponding visual location on the screen (Fig. 2b). By manipulating the horizontal location of the feedback, we

introduced different visuomotor mappings between visual target location and the required motor response. That is, to induce visuomotor adaptation, we manipulated the visual feedback by horizontally displacing it from the actual tap locations (see Fig. 2c). For simplicity, we chose to manipulate both the target and the feedback locations only in the horizontal direction. Target locations were chosen randomly from within a horizontal range of 16.0° visual angle centred around 0 (straight ahead). The vertical position was always 15.7° above the horizontal white reference line. Procedure The first part of the experiment was to train participants on two separate visuomotor mappings and to learn to switch between them. Training took place in 5 one-hour sessions, one each on 5 consecutive days. To increase the chances of the two mappings being learned, participants were exposed to each mapping for multiple trials before a switch to the other mapping occurred. For each session, there were only ten mapping switches, that is five times back and forth between the two mappings. The detailed procedure for each training session was as follows. To familiarize the participants every day with the pointing task, each session started with a short block of 50 trials with non-manipulated feedback, i.e. corresponding to the 0° mapping (on the very first day, this block consisted of 200 trials for training of the mapping between the vertical screen and the horizontal tablet). The data of this block was not used for analysis. After a 2-min break, 5 experimental training blocks followed. Each block consisted of three stages (see Fig. 2h): (1) a random number of trials (between 20 and 40 to prevent counting) for which the visual feedback was not shifted (0° mapping); (2) 40 trials for which the feedback was displaced by 6.4° visual angle (half the participants were trained with a positive 6.4° mapping, i.e. participants needed to point right-of-target to hit it, and the other half with a negative 6.4° mapping, i.e. left of target pointing was required. For the analysis, we combined both groups by mirroring their response at 0); (3) 100 washout trials for which the feedback was not shifted (0° mapping). The difference between the 0° and 6.4° mappings was chosen such that it was well above both the motor noise (which from pilot experiments was estimated to be about 1.3° on average) and the uncertainty in the feedback. This meant that the error between feedback and target position after a mapping change in principle provided a reliable cue to promote learning (of either the error or the mappings). It is important to emphasize that besides the visual feedback, there were no other cues that could indicate a change in the required behavioural mapping.

13



To track the progress of learning the mappings, the fifth and last block in each training session included probe and catch trials. The probe trial was a single trial for which the required mapping first changed and visual feedback was provided corresponding to the changed mapping. This probe trial was immediately followed by three catch trials without visual feedback in order to test the level of adjustment in the behaviour (Fig. 2g). To obtain a measure of the extent of learning on each day, the behavioural mapping on these catch trials was compared with the behavioural response on the three trials immediately preceding the catch trials (this included the probe trial for which the feedback for the new mapping was given only after the movement was complete). To motivate participants to improve their performance, we gave them additional feedback in terms of a score (based on both accuracy and response time for individual movements). The score was calculated for each trial individually and was 50 points when the error was below 1.0 cm, 20 points when the error was below 2.0 cm, 10 points when the error was below 3.0 cm and zero otherwise. The score was doubled when the response time (from trial start to landing on the tablet) was below 1 s, and negated when the response time was longer than 2 s. Note, however, that participants did not immediately receive these scores after each trial, but were only presented with the cumulative scores for each block at the end of each session. That is, the scores did not provide any information about the relative distance or the switches between the two trained mappings. For further motivation a high-score list was kept and presented to the participants at the end of each session. Procedure Day 5 On Day 5, we tested whether learning was more consistent with Error Learning or with Mapping Learning. That is, participants adapted to a new mapping (in separate test blocks either 12.8° or −6.4◦ in counterbalanced order across participants) before probe and catch trials were introduced. Probe trials were single trials with visual feedback corresponding to the probed mapping, i.e. either one of the previously trained absolute mappings (to test if Mapping Learning occurred) or a shift by 6.4° relative to the current one (to test for Error Learning). If the probe trial corresponded to how learning occurred, we expected the probe trial to result in a relatively large shift towards the probed mapping. If the feedback on a probe trial, however, did not correspond to the previously learned relationship, the effectiveness of the probe for shifting behaviour was expected to be much less (Fig. 2g). To include these tests, Day 5 differed from the previous days. Participants started with two training blocks as described in the training procedure above (the second one

13

Exp Brain Res

of which contained probe and catch trials to identify the learning extent on this day). The third block on Day 5, however, was a test block in which participants adapted to a third novel mapping before probe and catch trials were interleaved to test the type of learning. Block four was again a regular training block, but the fifth and final block of this last session was again a test block in which participants adapted to a novel mapping (different from the one in the third block) to further test the type of learning. Using the Probe and Catch Trial method, we tested three different conditions corresponding to the Mapping Learning and Error Learning predictions. First of all, we probed for Error Learning using the familiar size of large errors corresponding to the relative difference between the mappings (Fig. 2e, right). To do so, after initially adapting to the 12.8° mapping in the test block, the 19.2° mapping was probed using the Probe and Catch Trial method. The 19.2° mapping differs from the 12.8° mapping by 6.4°, which is the relative difference between the two trained mappings, and thus the 19.2° probe should result in a familiar size of the error. Therefore, if Error Learning occurred the probe effectiveness of the 19.2° probe after adaptation to the 12.8° mapping should be very high. Since the 19.2° mapping was not one of the two trained mappings, Mapping Learning should not effect the response for this probe, and in this way the contribution of Error Learning can be tested in isolation. Second, we probed if Mapping Learning had occurred (Fig. 2f right). Participants adapted to a −6.4◦ mapping and the mapping probed was the 6.4° mapping which is one of the explicitly trained mappings. Thus if Mapping Learning occurred, the probe effectiveness should be high in this case. Note that the required behavioural shift between the −6.4◦ and the +6.4° mappings is 12.8° which is twice as large as the relative difference between the two trained mappings. Thus, in this case the error signal on the probe trial should be much larger than the errors observed during training and Error Learning should not effect the behavioural response in this condition. Third, during training both Mapping Learning and Error Learning could potentially have occurred at the same time. To test to which extent there was a possible combination of the two learning types, we probed conditions in which both types can contribute without directly switching between the two trained mappings. This was done by probing the 6.4° mapping after having adapted to the 12.8° mapping (Fig. 2e, left), and by probing the 0° mapping after having adapted to the −6.4◦ mapping (Fig. 2f, left). In both these cases, one of the explicitly trained mappings was probed and the required behavioural shift to adopt the corresponding behaviour was 6.4°, i.e. the same as the relative difference between the two trained mappings. Yet, in both cases, the mapping before the probes were presented was not one of the two mappings

Exp Brain Res

Participants In total, eight naive participants took part in the experiment and gave informed consent. Participants received instructions about the task, but they were not informed about the fact that they would be required to repeatedly switch between the same two visuomotor mappings. All participants had normal or corrected-to-normal vision and were right handed. Ages ranged from 21 to 29.

Results As mentioned above, we used probe and catch trials (trials without feedback) in the last training block of each training session to track learning (Fig. 2h, top right). That is, on each day there was one probe for the 6.4° mapping followed by catch trials and one probe back to the 0° mapping followed by catch trials. Average probe effectiveness, i.e. the behavioural shift on these catch trials with respect to the immediately preceding trials significantly increased across days (Repeated Measures ANOVA on probe effectiveness across days, F(4,28) = 3.85; p = 0.013, t test on slope of the learning trend, t(7) = 3.93; p = 0.0057; Fig. 3) indicating that indeed the participants learned to switch more rapidly between the mappings. Note, however, that even after five days of training the learning was not complete, indicating that learning is slow. The slow learning

1.2 1.0

Probe effectiveness

that were extensively trained during the training procedure. These conditions should be informative if the two possible types of learning were in some way combined or added up also for conditions that were not explicitly trained. If the two types of learning add up, the probe trials in these conditions should be more effective than either Mapping Learning and Error Learning probes alone. For each of the probed mappings in this design, there were two such probe and catch trial periods (Fig. 2h). Each probe trial was followed by five catch trials. To ensure that all probe and catch trial periods started more or less from the same initial mapping, they were interleaved with a random number of trials (between 35 and 45) in which the feedback corresponded to the initial mapping (either 12.8 or −6.4◦) for that block (top-up adaptation). Initial adaptation occurred in 70–90 trials (the number of initial adaptation trials was again randomized across test blocks). At the end of each test block, participants adapted back to the 0° mapping in 50 trials. Probe effectiveness was evaluated as the difference in behavioural mapping during the five catch trials following the probe and the five trials directly preceding the probe trial, divided by the difference in mapping as indicated by the feedback (Fig. 2g).

full learning

0.8 0.6 0.4 0.2 0.0 -0.2

no learning 1

2

3

4

5

training day Fig. 3  Probe effectiveness across days (see Fig. 2g). 0 means the probe was ineffective in evoking a behavioural mapping change, 1 means that behaviour fully shifted to the probed mapping. Error bars represent standard deviations across participants. The dotted line indicates the average linear regression of probe effectiveness across days. The results indicate that learning accumulates across days

is consistent with the finding that learning to switch to the mapping of one’s own glasses may never be fully complete, even after more than a year of exposure to such switches (Schot et al. 2012). On the last day of training (Day 5) we tested whether the mappings were learned in an absolute sense (Mapping Learning), or if only the relative difference between the mappings had been learned, corresponding to certain sizes of large errors occurring more often than others (Error Learning). The latter form of learning would be consistent with the relative nature of the error signal in the feedback that indicates a mapping change, which was the only cue available to the participants. To investigate which type of learning had occurred, participants first adapted to a new visuomotor mapping that had not been used during training, after which probe and catch trials were introduced. We used three different test cases (see Fig. 2e, f). First, the probes could be towards a mapping that was the relative difference between the two trained mappings away from the newly adapted mapping, and if Error Learning had occurred, the probe should therefore trigger a relative large response in behaviour (Fig. 2e right). Second, the probe could be towards one of the two mappings that were actually used during training. This should result in a large behavioural shift if Mapping Learning had occurred (Fig. 2f right). Third, to test the possible additivity of the two types of learning, we included test conditions in which the probe was towards one of the two trained mappings as well as being the learned difference between the mappings away from the newly adapted one (Fig.  2e, f, left). This was possible because we chose the new mappings, for adapting participants before introducing

13



Exp Brain Res

b 1.5

** ** **

1.0

0.5

0.0

−0.5

6.4 deg

12.8 deg

6.4 deg

Error Learning

Mapping Combined Learning

6.4 deg Switch during training day 5

Probed type of learning

Probe effectiveness (∆behavior / ∆mapping)

Probe effectiveness (∆behavior / ∆mapping)

a

1.5

1.0

Error Learning Mapping Learning Combined

0.5

0.0

−0.5

1

2 3 4 Catch trial in the sequence

5

Fig.  4  a Probe effectiveness for the Error Learning, Mapping Learning and Combined probe conditions (light grey bars) as well as the probe results for training on Day 5 (dark grey bar). 0 means the probe was ineffective in evoking a behavioural mapping change, 1 means that behaviour fully shifted to the probed mapping. The grey numbers

beneath each bar indicate the necessary shift in behaviour indicated by the probe for each condition (i.e. xp Fig. 2g). b The trend across consecutive catch trials in the probe and catch trial sequences. Error bars represent standard deviation across participants. The results suggest that learning occurred exclusively in terms of Mapping Learning

the probe and catch trials, at the trained distances (6.4°) away from either of the two directly trained mappings. If both Mapping Learning and Error Learning contribute, then the effectiveness of the probe in this last condition should be larger than either of the conditions in which Mapping Learning and Error Learning are probed in isolation. The results of these test conditions are shown in Fig. 4a in which the effectiveness of the probe trials is expressed as the shift in behavioural mapping divided by the necessary shift in mapping as indicated by the probe. The probe effectiveness is shown for probes testing Error Learning, Mapping Learning and the two types of learning combined. To compare these test results with the results from the training, we added the observed probe effectiveness for the probes in the last training block before testing on Day 5. The results show that the probe was not at all effective when Error Learning was probed. However, the probes were equally effective when probing Mapping Learning and when probing the effect of Mapping Learning and Error Learning combined. Moreover, there was no difference between probe effectiveness for probing Mapping Learning in the test conditions, and the overall learning extent as observed for the last training block before testing on Day 5. This suggests that learning of the mappings in the absolute sense, i.e. Mapping Learning, can fully account for our results in both training and test conditions. This is very surprising given the relative nature of the error signal in the feedback through which the mappings have been learned.

It is striking that the probe effectiveness for Error Learning seems to be indistinguishable from zero. This is surprising because, in normal circumstances, the adaptation process of correcting for observed errors should have led to at least a small effect, even without learning. Furthermore, it is important to note that for Error Learning the familiar size of the error is usually experienced only once per mapping change. On the next trial, the error would already have been (partially) corrected, and thus also be smaller. It is possible that for this reason, Error Learning only led to relatively short-lived effects, aiding a first adjustment after which the normal adaptation process could take over. To investigate if there may be such a short-lived contribution of Error Learning or the normal adaptation process, we had a closer look at the sequence of catch trials in the test conditions. Figure 4b shows the probe effectiveness as a function of the sequence of catch trials for the three different probe conditions. For the Mapping Learning probe and the Combined Learning probe, the results are quite constant across consecutive catch trials. For the Error Learning probe, however, it is evident that a behavioural shift is present for the first catch trial only (t test on first catch trial compared to zero: t(7) = 3.62; p = 0.0085). In the absence of further feedback, the effect rapidly decays and no longterm behavioural shift remains. To investigate if this effect is due to Error Learning or simply reflects normal adaptive behaviour (i.e. correcting for observed errors), we compared the probe effectiveness for the first catch trial after the Error

13

Exp Brain Res

Learning probe to the first catch trials during training on Day 1 when little or no learning could still have occurred. The result of this analysis showed that there was no significant difference (t test, t(7) = 1.69; p = 0.13) between the first Error Learning catch trial and the first training catch trial on Day 1, suggesting that the observed trend in catch trials after the Error Learning probe is simply due to the normal adaptation process. In short, together our results suggest that, despite the nature of the feedback, Mapping Learning accounts for our results and any possible effects of Error Learning are very small and fail to evoke a longterm shift in behavioural mapping. The result that probing for Error Learning did not induce long-lasting behavioural mapping changes when no further feedback was provided, is consistent with previous findings that the effects of visual feedback are not necessarily long lasting (see e.g. Smeets et al. 2006) and can act on multiple timescales (Smith et al. 2006). That is, after the initial response to the single probe, pointing behaviour reverts back to the previously reinforced mapping (in this case the 12.8° mapping) in the absence of feedback (i.e. with consecutive catch trials).

Discussion We investigated the learning of separate visuomotor mappings when the only relevant information available was the relative error signal between target location and endpoint location in the visual feedback. That is, there were no additional associative cues indicating the required mappings in an a priori fashion as was the case in many previous studies (e.g. Kravitz 1972; Kravitz and Yaffe 1972, 1974; Martin et al. 1996; Seidler et al. 2001; Hay and Pick 1966; Pick et al. 1969; Donderi et al. 1985; van Dam et al. 2013; van Dam and Ernst 2015; Welch 1971). Without associative contexts, there is in principle no need for any long-term learning to occur, since the feedback is only provided after the movement is already complete and thus cannot provide a priori knowledge about the currently correct mapping. If long-lasting learning should occur, it would make sense to learn to respond more quickly to certain specific sizes of the errors, corresponding to the relative difference between the two trained mappings (Error Learning). This would still be in line with the nature of the error signal, which in principle indicates how to shift behaviour with respect to our current mapping. However, our results show that instead the specifically trained absolute mappings had been stored (Mapping Learning) and little evidence for Error Learning was found. In the literature, it has been argued that learning separate visuomotor mappings may not be possible without additional associative cues (Bingham and Romack 1999; Redding and Wallace 2003; Woolley et al. 2007). Indeed,

results for learning multiple mappings are very mixed when there are no or very minimal additional cues. In our case, we did find learning but several others did not (e.g. Karniel and Mussa-Ivaldi 2002; Hinder et al. 2008). There are several differences in the experimental design that may account for the different results. Generally, studies differ in the type of visuomotor transformation (rotation vs a shift of the feedback); the type of visual feedback (continuous vs endpoint feedback); and the temporal order in which mappings are presented (random order, quick alternation or longer phases of the same mapping). Particularly, the temporal order in which the mappings are presented likely has a major influence on whether separate mappings are learned when there are no associative cues present. If presented in random order or in quick alternation, it is likely that the mappings are never fully experienced, since there is no time to adapt to them. All that is learned in this case could be that the feedback is changing all the time and thus is very unreliable and adaptation should be suppressed. Moreover, if only terminal visual feedback is provided, fully adapting to the mappings would even become impossible. In that case, the relevant feedback is provided only after each movement is complete and any behavioural shift on the next trial would be too late if the mapping immediately changes back again. In short, if fully experiencing the mappings is a prerequisite for Mapping Learning to occur, longer phases of the same mapping would be needed, as was the case in the current study. Another possibility is that explicit knowledge about the mappings is necessary for Mapping Learning to occur. For learning associations between colour cues and rotated visuomotor mappings, there is evidence that learning is dependent on whether the participant had explicit knowledge about this relationship (Hegele and Heuer 2010). Also for learning visuomotor shifts, there is evidence that awareness of the cues plays a dominant role for whether or not learning occurs (van Dam and Ernst 2015). The same could be true when there are no associative cues, as in the present study. Then, Mapping Learning should only occur if the discrepancy between the mappings is large enough, such that explicit knowledge about the discrepancy becomes available. Presenting the mappings in random order would make it harder to separate the two mappings. Also continuous visual feedback, the type of feedback often used for visuomotor feedback rotations or force-field manipulations, might hamper learning the mappings in this case. The continuous feedback would be driving corrections for the ongoing movement directly, such that perceived errors between target and visual feedback of the hand remain relatively small. Without associative cues, the observed error is the only available information through which it is possible to learn that separate mappings are involved. Reduced errors by correcting the movement online would

13



therefore be less likely to result in Mapping Learning, since the errors are less obvious. Note that in the current study in which only endpoint feedback was presented, the observed errors resulting from a mapping switch were sufficiently large in order to be distinguishable from regular motor noise. After the experiment, most of our participants also indicated that they had become aware that there were certain large errors they had to adapt to. Thus, explicit knowledge about the mapping changes was available during training, supporting the possibility that awareness was important. Nevertheless, the result that Mapping Learning occurred instead of Error Learning is somewhat surprising given the relative nature of the error signal through which the mappings needed to be learned. Note, however, that in the current study each mapping was experienced for longer periods of time and actual mapping switches—and thus the corresponding large errors—did not occur very often. This may have pushed the balance towards Mapping Learning and it is possible that Error Learning will become more apparent if switches between the mappings occur more frequently. That is, based on the current results we cannot rule out Error Learning as a possible learning mechanism in general, only that it does not appear to have a measurable influence when mappings are experienced for longer periods of time and switches between mappings are relatively infrequent. Interestingly, the present result that Mapping Learning occurs is strikingly similar to the results from a study in which two separate colour cues were used to signal two separate mappings (van Dam et al. 2013). In that study, participants were not provided with the task-relevant error signal in the feedback when they had to switch mappings after training. Instead, unique context colour cues indicated the current required mapping through learned associations even before a movement was initiated. The results of van Dam et al. (2013) indicated that in association to the colour cues, absolute mappings were stored (Mapping Learning) rather than the relative difference between mappings (Error Learning). The similarity between that study and our current results indicates that the task-relevant visual feedback can have a double role: on top of driving normal adaptation, it can act as a contextual cue in a similar fashion as was found previously for task-irrelevant associative cues. Thus, the feedback can cue absolute mappings in spite of the relative nature of the error signal in the feedback indicating the mapping change. Acknowledgments  This research was supported by the Human Frontier Science Program, the DFG Cluster of Excellence: Cognitive Interaction Technology “CITEC” (EXC 277) and EU Grant No. 601165 “WEARHAP”. The experiments were conducted at the Max Planck Institute for Biological Cybernetics in Tübingen Germany. The authors thank David Hawellek for his help with data collection,

13

Exp Brain Res and Cesare Parise for helpful comments on an earlier version of the manuscript.

References Bingham GP, Romack JL (1999) The rate of adaptation to displacement prisms remains constant despite acquisition of rapid calibration. J Exp Psychol Hum Percept Perform 25(5):1331–1346 Donderi DC, Jolicoeur P, Berg I, Grimes R (1985) A color-contingent prism displacement aftereffect. Perception 14(6):691–709. doi:10.1068/p140691 Ernst MO (2001) Psychophysikalische Untersuchungen zur visuomotorischen Integration beim Menschen: visuelle und haptische Wahrnehmung virtueller und realer Objekte. MVK Medien Verlag Köhler, Tübingen Fernandez-Ruiz J, Hall-Haro C, Diaz R, Mischner J, Vergara P, Lopez-Garcia J (2000) Learning motor synergies makes use of information on muscular load. Learn Mem 7(4):193–198. doi:10.1101/lm.7.4.193 Fernandez-Ruiz J, Diaz R, Moreno-Briseno P, Campos-Romo A, Ojeda R (2006) Rapid topographical plasticity of the visuomotor spatial transformation. J Neurosci 26(7):1986–1990. doi:10.1523/JNEUROSCI.4023-05.2006 Flook JP, McGonigle BO (1977) Serial adaptation to conflicting prismatic rearrangement effects in monkey and man. Perception 6(1):15–29. doi:10.1068/p060015 Hay JC, Pick HLJ (1966) Gaze-contingent prism adaptation: optical and motor factors. J Exp Psychol 72(5):640–648. doi:10.1037/ h0023737 Hegele M, Heuer H (2010) Implicit and explicit components of dual adaptation to visuomotor rotations. Conscious Cogn 19(4):906– 917. doi:10.1016/j.concog.2010.05.005 Hinder MR, Woolley DG, Tresilian JR, Riek S, Carson RG (2008) The efficacy of colour cues in facilitating adaptation to opposing visuomotor rotations. Exp Brain Res 191(2):143–155. doi:10.1007/s00221-008-1513-7 Karniel A, Mussa-Ivaldi F (2002) Does the motor control system use multiple models and context switching to cope with a variable environment? Exp Brain Res 143(4):520–524. doi:10.1007/ s00221-002-1054-4 Körding KP, Wolpert DM (2004) Bayesian integration in sensorimotor learning. Nature 427(6971):244–247. doi:10.1038/nature02169 Kravitz JH (1972) Conditioned adaptation to prismatic displacement. Percept Psychophys 11(1A):38–42. doi:10.3758/BF03212680 Kravitz JH, Yaffe FL (1972) Conditioned adaptation to prismatic displacement with a tone as the conditional stimulus. Percept Psychophys 12(3):305–308. doi:10.3758/BF03207210 Kravitz JH, Yaffe FL (1974) Conditioned adaptation to prismatic displacement: training trials and decay. J Exp Psychol 102(2):194– 198. doi:10.1037/h0035863 Martin TA, Keating JG, Goodkin HP, Bastian AJ, Thach WT (1996) Throwing while looking through prisms. II. Specificity and storage of multiple gaze-throw calibrations. Brain 119(4):1199– 1211. doi:10.1093/brain/119.4.1199 McGonigle BO, Flook J (1978) Long-term retention of single and multistate prismatic adaptation by humans. Nature 272(5651):364–366. doi:10.1038/272364a0 Pick HLJ, Hay JC, Martin R (1969) Adaptation to split-field wedge prism spectacles. J Exp Psychol 80(1):125–132. doi:10.1037/ h0027111 Redding GM, Wallace B (2003) Dual prism adaptation: calibration or alignment? J Mot Behav 35(4):399–408

Exp Brain Res Schot WD, Brenner E, Sousa R, Smeets JBJ (2012) Are people adapted to their own glasses? Perception 41(8):991–993. doi:10.1068/p7261 Seidler RD, Bloomberg JJ, Stelmach GE (2001) Context-dependent arm pointing adaptation. Behav Brain Res 119(2):155–166. doi:10.1016/S0166-4328(00)00347-8 Smeets JBJ, van den Dobbelsteen JJ, de Grave DDJ, van Beers RJ, Brenner E (2006) Sensory integration does not lead to sensory calibration. Proc Natl Acad Sci USA 103(49):18,781–18,786. doi:10.1073/pnas.0607687103 Smith MA, Ghazizadeh A, Shadmehr R (2006) Interacting adaptive processes with different timescales underlie short-term motor learning. PLoS Biol 4(6):1035–1043. doi:10.1371/journal. pbio.0040179 van Dam LCJ, Ernst MO (2015) Mapping shape to visuomotor mapping: learning and generalisation of sensorimotor

behaviour based on contextual information. PLoS Comput Biol. doi:10.1371/journal.pcbi.1004172 van Dam LCJ, Hawellek DJ, Ernst MO (2013) Switching between visuomotor mappings: learning absolute mappings or relative shifts. J Vis. doi:10.1167/13.2.26 Welch RB (1971) Discriminative conditioning of prism adaptation. Percept Psychophys 10(2):90–92. doi:10.3758/BF03214321 Welch RB, Bridgeman B, Anand S, Browman KE (1993) Alternating prism exposure causes dual adaptation and generalization to a novel displacement. Percept Psychophys 54(2):195–204. doi:10.3758/BF03211756 Woolley DG, Tresilian JR, Carson RG, Riek S (2007) Dual adaptation to two opposing visuomotor rotations when each is associated with different regions of workspace. Exp Brain Res 179(2):155– 165. doi:10.1007/s00221-006-0778-y

13

Relative errors can cue absolute visuomotor mappings.

When repeatedly switching between two visuomotor mappings, e.g. in a reaching or pointing task, adaptation tends to speed up over time. That is, when ...
1KB Sizes 1 Downloads 7 Views