Perception, 1992, volume 21, pages 7-19
Attentional modulation of a figural aftereffect
Gordon L Shulman Washington University School of Medicine, Department of Neurology and Neurological Surgery, 660 S Euclid, St Louis, MO 63110, USA Received 22 September 1990, in revised form 24 June 1991
Abstract. Evidence is reported that indicates that adaptation of the Schroder staircase is affected by attention. In previous work it has been shown that if subjects adapt to an unambiguous staircase, responses to an ambiguous test figure are biased towards the opposing perspective. In the current work, subjects adapted to superimposed upright and inverted Schroder staircases. Both staircases were centered on a common fixation point and were of different sizes and colors. Attention to each staircase was controlled by asking subjects to detect color changes in the line segments that defined one or the other staircase. Responses to an ambiguous test figure depended on which of the adapting staircases was attended.
1 Introduction Adapting to a stimulus produces profound effects on the perception of subsequent stimuli. These adaptation effects are specific to spatial frequency (Blakemore and Campbell 1969; Georgeson and Harris 1984), orientation (Blakemore and Nachmias 1971), direction of motion (Sekuler and Ganz 1963), and other visual parameters of the adapting stimulus. This specificity suggests that adaptation affects particular mechanisms in the visual system. By studying the manner in which adaptation effects depend on how the adapting stimulus is attended, the role of attention in activating particular visual mechanisms can be studied. Effects of attention on adaptation have recently been reported both for twodimensional and for three-dimensional motion aftereffects (Chaudhuri 1990; Shulman 1991). Chaudhuri presented a moving-texture field which contained a central window framing an alphanumeric character that changed identity at 4 Hz. He measured the duration of the motion aftereffect induced by this stimulus on a subsequent stationary-texture field under two different attentional conditions. During the adaptation phase, subjects either fixated the central window (passive condition), or fixated and pressed a key whenever they detected a digit in the window (distractor condition). Chaudhuri reported that the duration of the motion aftereffect was much less when subjects attended to the central window (distractor condition) during the adaptation phase. He concluded that attention modulates the mechanisms underlying the motion aftereffect. Shulman (1991) reached a similar conclusion concerning the three-dimensional motion aftereffect. Previous work (Petersik et al 1984; Nawrot and Blake 1989) had shown that after adapting to a single stimulus rotating in depth in one direction, a test stimulus in parallel projection, whose direction of rotation is normally ambiguous, was more likely to be perceived as rotating in the opposite direction. Shulman (1991) presented two adapting squares rotating in depth in opposite directions and centered on a common fixation point. The squares were of different colors and sizes so as to make them readily discriminable. During the adaptation phase of the experiment, attention to the two stimuli was controlled by having subjects detect small perturbations in either the large or the small square. In the subsequent test phase, subjects judged the direction of motion of a test square in parallel projection. The perceived
8
G L Shulman
direction of rotation of the test square was determined by the rotation direction of the attended adapting square, indicating that attention controlled the degree to which the adapted mechanism was activated by the unambiguous rotating stimulus. Shulman (1991) noted an important difference between the method used in his study and in Chaudhuri's (1990). Chaudhuri used a 'distractor' design that determines whether the adapting effect produced by a stimulus is decreased if the subject concurrently performs another task on an irrelevant stimulus. The idea is that the distracting stimulus prevents the adapting stimulus from causing adaptation of those mechanisms that respond to the test. If one of the two adapting squares in the experiment of Shulman (1991) had been a 'neutral' stimulus that did not produce adaptation, then that experiment would also have involved a distractor technique. The use of two adapting squares, however, introduces an additional factor since both the small and large square activate the same mechanism as the test (ie they both can produce adaptation effects on the subsequent test stimulus). Therefore when activity is shifted from the large square to the small square, not only is the activity of the mechanism responding to the large square decreased, but the activity of that mechanism responding to the small square, which signals the opposite direction of motion, is increased. This increased activation of the opposing mechanism will further weaken the adaptation effect produced by the large square. Shulman (1991) argued that this opponent design would permit larger attention effects than a comparable distractor design and confirmed this prediction in an experiment that compared the two methods. This increased power, however, is only evident if two conditions are compared in which attention is directed to one or the other adapting component. (For example, under conditions in which the large square rotates to the right and the small square to the left, the condition in which one attends to the large square is compared with that in which one attends to the small square.) This is equivalent to comparing the negative aftereffect (the effect of the adapting condition on test responses relative to an unadapted baseline) produced when attending to one component with the negative aftereffect produced when attending to the other(1). A single adapting condition will produce a greater effect on test responses relative to the baseline in a distractor design than in an opponent design. This is because the aftereffect produced by a single component in an opponent paradigm is decreased by whatever adaptation is produced by the other unattended component, whereas no such opposing adaptation occurs in a distractor design. However, the change in the observer's response to the test figure as attention is shifted from one stimulus to another (from one adapting stimulus to the other in an opponent design, or from the adapting stimulus to the irrelevant stimulus in a distractor design) will be larger in an opponent design than a distractor design (Shulman 1991). The advantage of the opponent technique is therefore only exploited by analyses that directly compare different adaptation conditions, rather than by the more standard analyses in which a single condition is compared to a baseline. Since these ^ A n example will make this point clear. Suppose that when the large square rotating to the right is attended, the subject responds 'left' 75% of the time; when the small square rotating to the right is attended, the subject responds 'left' 30% of the time; and the unadapted or baseline response rate is 'left' 55% of the time. The difference between the adapting conditions is 45%, ie the sum of the negative aftereffect produced by the small square (25%) and the large square (20%). If one directly compares the negative aftereffects, since they are opposite in sign, the difference is again 45%. A statistical analysis directly comparing adapting conditions in which the large square rotated to the left and the small square rotated to the right will give identical results to an analysis that compares the negative aftereffects produced by each adapting condition.
Attentional modulation of a figural aftereffect
9
comparisons always involve the same physical stimulus (only the distribution of attention is changed), however, they allow a clean assessment of attentional effects. The mechanisms that underlie the three-dimensional motion aftereffect pool information over stereopsis and structure-from-motion. Nawrot and Blake (1989) have reported that adaptation to a rotating three-dimensional globe whose direction of motion is specified stereoscopically induces the opposing perception in an otherwise ambiguous globe produced by a parallel projection. Virsu (1975) has studied a different adaptation effect that also appears to pool different sources of depth information. He showed that adapting subjects to an unambiguous version of the Schrpder staircase, whose figure perspective was specified by stereopsis, created a bias towards the opposing perception in the traditional ambiguous test figure. This adaptation effect could also be produced by adapting subjects to an actual model of an unambiguous staircase. This aftereffect is also similar to that studied by Shulman (1991) since it involves a bistable test figure. Shulman (1991) has developed a simple model for describing the influence of attention on adaptation effects involving bistable test figures. In the current experiments the attentional dependence of adaptation of the Schroder staircase was therefore examined by using the opponent technique of Shulman (1991). During the adaptation phase, two unambiguous staircases were superimposed and centered on a common fixation point. On different adaptation blocks, subjects attended to one or the other staircase. The experiment determined if shifting attention from one adapting component to the other affected the subsequent perception of the test. Such a result would imply that attention can modulate the degree to which the adapting staircases activate the mechanisms that underlie the aftereffect. 2 Methods 2.1 Experiment la 2.1.1 Subjects. Ten naive observers participated in experiment la. 2.1.2 Apparatus and stimuli. Stimuli were displayed on a color monitor controlled by an Amiga 1000 microcomputer. A fixation cross was displayed at all times. The adapting and test stimuli are shown in figure 1. The adapting staircase could be either large or small, and when two adapting staircases were presented simultaneously, one was large and the other was small. The size of each step of the small staircase was 0.5 deg and the entire stimulus subtended 3.0 deg, while the step size of the large staircase was 1.25 deg and
Figure 1. Adaptation and test stimuli used in experiments la and lb. The left panel displays an unambiguous upright staircase, the middle panel an ambiguous staircase, and the right panel an unambiguous inverted staircase. The relative luminance of a line segment is indicated by the thickness of the line. The left and right unambiguous stimuli were displayed in adaptation blocks and the center ambiguous stimulus in test blocks.
10
G L Shulman
the entire stimulus subtended 7.5 deg. One size of adapting staircase was colored red and the other green, and the assignment of colors to sizes was counterbalanced across observers. In order to decrease the ambiguity of the adapting staircases, the luminance of the line segments depended on the distance of the segment from the observers (Dosher et al 1986). For this purpose, line segments were grouped into three categories: the segments defining the front face of the staircase, those transverse segments defining the steps of the staircase, and those defining the back face of the staircase. For the red staircase, the respective luminances were 39.7, 8.6, and 2.7 cd m~2, and for the green staircase, the luminances were 37.3, 12.7, and 3.42 cd m" 2 . During the adaptation sequence, color changes were occasionally introduced in one of the transverse segments that defined the protruding edges of the steps of each of the adapting staircases. (The identity of these line segments depend on whether the staircase is upright or inverted. In figure 1 numbers 1-3 are the relevant segments for the upright staircase, and numbers 4 - 7 are for the inverted staircase.) This was accomplished by increasing the blue color in one (randomly selected) of these line segments, which were positioned symmetrically with respect to the fixation point. The degree of color change was adjusted for each subject to yield a hit rate of 60%-70%. The color change occurred in each staircase on average once every 10 s with a duration of 100 ms. Since the position of the color change varied randomly about the fixation point, the brief duration ensured that subjects would maintain fixation during the block. The test staircase was the traditional ambiguous figure. The step size was 0.75 deg, the entire figure subtended 4.5 deg, and all line segments were white with a luminance of 31.5 cd m" 2 . 2.1.3 Procedure. Observers were instructed to maintain fixation on the fixation cross at all times. Observers participated in two types of blocks, opponent and alone. In opponent blocks, two superimposed staircases, one small and one large, were presented during the adaptation phase and these staircases were always in opposing perspective (ie one upright and one inverted). In alone blocks, only one staircase was present in the adaptation phase. On half the opponent blocks, subjects pressed a key when they detected color changes in the small adapting staircase, disregarding those that occurred in the large one, while in the other half they detected changes in the large staircase, disregarding those that occurred in the small staircase. The same adapting stimuli were therefore presented on the two types of opponent block, which only differed in terms of the response that was required to the adapting stimuli. In alone blocks, subjects detected color changes in the single adapting staircase. The basic sequence in a block of trials was as follows: The adapting staircases were presented for 30 s, during which the subject's only task was to detect color changes in the appropriate staircase. Following a 0.5 s interstimulus interval, the test stimulus was presented for 4 s. Subjects pressed one key if the staircase was currently seen as upright and a second key if inverted. If their perception of the test staircase reversed, they were instructed to press the corresponding key to indicate the new perspective. Following an intertrial interval of 2 s, a new adapt-test sequence was begun. One block consisted of ten trials or adapt-test sequences. During each block, the perspective of the adapting staircase or staircases and the designation of the taskrelevant staircase remained constant. At the beginning of each session, subjects participated in a baseline block of ten trials in which only the test phase was presented.
Attentional modulation of a figural aftereffect
11
2.1.4 Design. There were four different opponent blocks (subjects could attend to either the small or to the large staircase, both of which could be in upright or inverted perspective) and four different alone blocks (the factorial combination of the two staircase sizes and perspectives). The order of these eight blocks was randomized, and subjects observed two different orders, a total of sixteen blocks. Subjects also received a total of five baseline blocks. 2.2 Experiment lb After collecting data from the ten subjects in experiment la, it appeared that the large staircase produced less adaptation than the small staircase. A second group of ten subjects was therefore tested with the step size of the test staircase increased to 1.1 deg. By making the test size more similar to the large staircase, than to the small staircase, it was thought that the adapting effect produced by the large staircase would be increased relative to the small staircase, since adaptation effects often depend on the similarity of the adapting and test stimulus (Blakemore and Campbell 1969). These subjects also participated in 'perception' blocks to provide information on the effectiveness of the figural and proximity-luminance manipulation in producing unambiguous percepts of the adapting staircases. In these blocks, only the adaptation phase was presented. Subjects detected color changes in the designated staircase and indicated whether it was perceived as upright or inverted. Perception blocks were conducted both for alone and for opponent configurations. A perception block consisted of ten adaptation phases, each of 30 s duration, with an intertrial interval of 2 s. Eight perception blocks (four opponent and four alone) were conducted in a randomized order following the completion of the sixteen standard blocks (eight opponent and eight alone, as in experiment la). Subjects also served in four baseline blocks, rather than five as in experiment la. 3 Results In the analyses reported here, data from experiments la and lb are collapsed. As noted, the two experiments were the same except for the step size of the test staircase, which was increased in experiment l b to increase the adaptation effect produced by the large staircase. Statistical analyses explicitly comparing the two experiments are reported at the end of the results section. During the test phase, reversals were fairly common, producing an average of 2.2 responses during the 4 s test phase. Two subjects, KB and KN, showed particularly large response rates of 7.4 and 4.4. The average response rate of the remaining eighteen subjects was 1.8. On each test trial, the subject's response was defined as that perspective which was seen for the longest duration (Shulman 1991). Figures 2a and 2b show the percentage of upright responses, averaged across subjects, for both alone and opponent blocks(2). 3.1 Alone blocks The data from the alone blocks replicated the effect reported by Virsu (1975). For both the large and the small adapting staircase, subjects tended to perceive the test staircase in the perspective opposite to that of the adapting perspective. An analysis of variance was conducted with adapting staircase size (small, large) and perspective (2)
Since the figures show the percentage of upright test responses in each condition, the standard errors in the figures include variability due to differences in response bias across subjects. Although overall there was a preference for upright responses, the extent of this bias differed between subjects, and this variability will increase the standard errors. In all the statistical analyses reported here, however, one adapting condition is compared to another. Since the response bias is constant in both conditions, this variability will not be reflected in the standard error of the difference between the two conditions.
12
G L Shulman
(upright, inverted) as factors and the percentage of upright test responses as the dependent measure. The analysis yielded a significant effect of adapting perspective (^1,19 = 60.13, p < 0.001), indicating that the perception of the test figure changed reliably depending on whether the adapting perspective was upright or inverted. In this analysis, the effect of adapting to one perspective is directly compared with that of adapting to the other. If there were no adaptation effects, these two conditions would yield the same results. The difference in the test response to adaptation with an upright versus an inverted staircase is the sum of the negative aftereffects specified separately for each adapting staircase (see footnote 1). By summing the effects, however, one cannot determine if adapting with one perspective produces a larger effect than adapting with the other. For this reason, negative aftereffects were computed separately (table 1) for adaptation to upright and to inverted staircases by subtracting the appropriate quantity from the baseline response rate (65% upright responses). For example, when the adapting staircase was small and upright, the negative aftereffect was 33.6% (65%-31.4%), but when the adapting staircase was small and inverted, the negative aftereffect was 19.5% (84.5% -65%). The results of Mests, performed to determine whether the negative aftereffects for the four conditions differed from zero, indicated significant effects for the small staircase in upright (t19 = 5.42, p < 0.001) and inverted perspective {tl9 = 5.70, p < 0.001), and the large staircase in upright (tl9 = 5.74, p < 0.001) and inverted perspective {t19 = 3.39, p < 0.005). To determine whether the magnitude of these negative aftereffects depended upon condition, an analysis of variance was conducted with the negative aftereffect as the dependent measure (always expressed in positive sign) and adapting perspective (upright, inverted) and size (small, large) as factors. The analysis yielded a significant effect of 100
• inverted staircase s upright staircase
84.5%
Small staircase
71.7%
Small staircase attended
Large staircase
• small staircase inverted (large upright) ^ small staircase 65.1% ^Pright 56.4% T ( lar 8 e inverted)
Large staircase attended
(b) (a) Figure 2. Mean percentage of upright test responses in (a) the alone blocks and (b) the opponent blocks, The dotted line gives the baseline percentage of upright responses which was 65%. Table 1. Negative aftereffects for the small and large staircase in the alone and in the opponent condition. Standard errors are given in parentheses. Perspective
Upright staircase Inverted staircase
Alone
Opponent
small
large
small
large
33.6 (6.2) « 19.5 (3.4)
31.6 (5.5) 14 (4.1)
25 (4.8) 6.7 (3.4)
8.6 (4.2) 0.1 (5.2)
Attentional modulation of a figural aftereffect
perspective (F1A9 = 6.19, p < 0.05), indicating a larger negative aftereffect adaptation with an upright staircase.
13
for
3.2 Opponent blocks Figure 2b shows the percentage of upright test responses as a function of the perspective of the small staircase (upright or inverted) and whether it was attended during the adaptation phase. The data indicate that the magnitude of the adaptation effect produced by a staircase of a particular perspective depended on whether that staircase was attended. When the small staircase was attended, a large adaptation effect appropriate to the perspective of that staircase was found: the test staircase was seen in the perspective opposite to that of the small staircase (40% upright when the small staircase was upright, 71.7% upright when the small staircase was inverted). When the same adapting stimulus was presented and the large staircase was attended, however, the effect of staircase perspective now corresponded to that of the large staircase (56.4% upright responses when the large staircase was upright, 65.1% when the large staircase was inverted). This reversal, however, was quite small, a point that will be discussed later when a quantitative model of the results is introduced. Note that the left and right panels of figure 2b refer to the same physical conditions. All that differs between panels is the distribution of attention. If there were no effect of attention, the left and right panels should be equivalent. An analysis of variance with perspective of the small staircase (upright or inverted) and whether the small staircase was attended as factors and with the percentage of upright test responses as the dependent measure yielded a main effect of staircase perspective (Flt 19 = 8.40, p < 0.01) and an interaction of attention with staircase perspective {F1 19 = 38.6, p < 0.001). The latter interaction indicates that the effect of the perspective of an adapting staircase on test responses (ie its adaptation effect) depended on whether it was attended. The interaction term in the ANOVA is the most sensitive way to test for effects of attention since it considers the data from all four cells of the opponent condition. As noted in the introduction, the 'push-pull' concept underlying the opponent technique requires that different adaptation conditions be compared directly. In line with this argument, the attention effect can also be seen by considering how the test responses changed for a particular opponent configuration as attention was shifted from one adapting staircase to the other. In these comparisons, the physical stimulus is again held constant; only the direction of attention changes. Significant effects on test responses can therefore be unambiguously attributed to attention. For example, consider opponent adaptation blocks in which the small staircase was upright and the large staircase was inverted; when attention was directed to the small upright staircase, subjects perceived the test as upright on 40% of the trials, but when attention was directed to the large inverted staircase, the percentage of upright test responses was 65.1%, a significant shift (tl9 = 4.92, p < 0.001). Conversely, consider configurations in which the small staircase was inverted and the large staircase was upright; attending to the small inverted staircase produced 71.7% upright test responses, while attending to the large upright staircase produced 56.4% upright test responses. This difference was also significant (*19 = 3.67, p < 0.005). As noted in the introduction, these r-tests are equivalent to comparing the negative aftereffect produced when attending to one component with the negative aftereffect produced when attending to the other component. These negative aftereffects are shown in table 1. The baseline test response was 65% upright. A bias towards upright responses was also evident in Virsu's (1975) data. The preceding analyses were based on a measure in which the response on a trial was assigned to the percept that was present for the longest duration. The data were
14
G L Shulman
also analyzed with respect to the first response that the subject made. This procedure is more equivalent to that of Virsu (1975), who presented the test stimulus for 1 s and did not find any test reversals. KB and KN, the two subjects who showed large reversal rates, also gave 'upright' as their first response on virtually every trial (95.6% forKN, 98.7% for KB). This made their first response data uninformative and their data has been excluded from the analysis. The results from the remaining subjects look quite similar to those obtained by using the dominant response measure (figures 3a and 3b). The interaction of figure perspective and attention in the opponent blocks was again significant (Ful7 = 22.5, p < 0.001).
80.7%
• inverted staircase s upright staircase
76.3%
small staircase inverted (large upright) small staircase upright (large inverted)
Small staircase Large staircase attended (b) attended Figure 3. Mean percentage of 'first' upright test responses in (a) the alone blocks and (b) the opponent blocks. The dotted line gives the baseline percentage of upright responses which was 69%. Small staircase Large staircase
3.3 Perception blocks The overall percentages of time that the designated staircases were seen as upright or inverted were computed both for opponent and for alone configurations and are displayed in table 2. The upright staircase was virtually always seen as upright, but the inverted staircase showed some tendency to reverse. Table 2. Perception blocks. Percentage of time that the adaptation stimulus was seen in the indicated perspective. Standard errors are given in parentheses. Perspective
Inverted staircase Upright staircase
Small staircase
Large staircase
alone
opponent
alone
opponent
75.9 (10.1) 99.1 (0.64)
83.9 (8.2) 97.5 (1.2)
83.3 (5.9) 98.5 (0.86)
84.6 (6.3) 97.9 (1.4)
3.4 Comparisons of experiments la and lb The data from the alone block in experiment la suggested that the overall adaptation effect produced by the small staircase (this effect, the difference between the percentage of upright responses after adapting to inverted and upright staircases, was 87.5%-28.5% = 59%) was larger than that produced by the large staircase (the corresponding score was 79.8% - 3 8 . 3 % = 41.5%). This suggestion was supported somewhat by an ANOVA on the alone blocks, with staircase size and perspective as factors. The analysis yielded a marginal interaction of staircase size by perspective (Fl9 = 4.9, p < 0.052), indicating that the effect of staircase perspective (the overall
Attentional modulation of a figural aftereffect
15
adaptation effect) depended on staircase size. The analogous difference scores in experiment lb are 81.5% -34.4% = 47.1% for the small staircase, and 78.2%-28.5% = 49.7% for the large staircase. No interaction of size and perspective was observed in the ANOVA on this experiment. The two experiments were explicitly compared in an ANOVA with experiment as a between-subjects factor and staircase size and perspective as within-subject factors. The analysis yielded a marginal experiment by staircase size by perspective interaction [Flfl8 = 3.90, p = 0.061). These analyses are suggestive but not conclusive with respect to whether the increase in the size of the test staircase in experiment l b increased the size of the adaptation effect produced by the large staircase relative to the small one. The opponent blocks from the two experiments were also analyzed with experiment as a between-subjects factor and attention and staircase perspective as withinsubject factors (ie the same within-subject factors as reported in the opponent analyses of the pooled data). The experimental factor did not interact with any term involving the attention factor, supporting the observation that similar attentional effects occurred in the two experiments. Since these effects are the main focus of this report, it seems reasonable to collapse the two experiments. 4 Discussion The results from the alone block replicate those of Virsu (1975), indicating that adapting to an unambiguous staircase produces a tendency to perceive the opposing perspective in an ambiguous figure. One difference, however, is that the unambiguous staircases in the present experiment were defined by figural manipulations and proximity-luminance covariance rather than binocular disparity as in Virsu's (1975) study. The adaptation effect produced by a staircase of a particular perspective was influenced by whether the adapting staircase was attended. The assertion here is not that attention can produce adaptation by itself, nor that it can reverse the effect of staircase perspective (ie make a single upright staircase bias test responses towards upright), but just that it can modulate the adapting effect produced by a staircase of a particular perspective. This assertion is directly supported by the attention-bystaircase perspective interaction in the opponent blocks, as well as the separate analyses of each adapting configuration. Shifting attention from one adapting component to the other significantly affected responses to the test stimulus. Although the results of the experiment suggest some role of attention in modulating the aftereffect, the data indicated an unexpected asymmetry. Analyses of the negative aftereffects in the alone blocks indicated that adaptation was significantly more effective in biasing test responses when the adapting staircase was upright. This might reflect a 'ceiling' effect since test responses was biased towards upright. However, data from the perception blocks also indicated that the 'unambiguous' inverted staircase was sometimes prone to reversals. These reverals could weaken the adapting effectiveness of inverted staircases, resulting in the observed asymmetry. 5 Model A simple descriptive model developed by Shulman (1991) to analyze the effects of attention on the three-dimensional motion aftereffect was applied to the present data. The advantage of the model is that it includes a single parameter that describes the degree to which attention modulates the aftereffect. In the model, strength values represent the extent to which shape mechanisms coding the test stimulus signal an upright or inverted staircase. The strength values are distributed according to a standard normal, and on any test trial, the subject samples from the distribution (figure 4). If the sample strength value is positive, the
G L Shulman
16
subject responds 'upright', and if negative, the subject responds 'inverted'. The probability of an 'upright' response during the test phase is therefore the area under the curve to the right of zero, and the mean strength value can be determined by converting the probability of an 'upright' judgment to a z -score. Adapting to an upright perspective shifts the strength distribution towards negative values; adapting to inverted perspective shifts it towards positive values. During an opponent condition, therefore, each adapting component shifts the strength distribution in an opposing direction, and the net adaptation effect is the sum of the component effects. The degree to which an adapting stimulus changes the strength value elicited by a subsequent test stimulus is affected by whether the adapted stimulus is attended. Equations (1) and (2) give the strength values for conditions in which the observer attends to the small adapting stimulus (in the following exposition, the perspective of the adapting stimulus is always described with respect to the small staircase). Equation (1) describes the strength values for an 'upright' response when the small adapting stimulus was attended and was inverted, and equation (2) is for when the adapting stimulus was small and upright: adapt inverted:
z-x = As - ksAx + BU,
(1)
adapt upright:
zu = ~AS + ksAx +Bu.
(2)
Z\ is the z -score for the probability of making an 'upright' response when the adapting staircase is inverted. zn is the corresponding score when the adapting staircase is upright. As is the contribution to the total strength from the small adapting stimulus when it is attended.(3) It is negative in equation (2) since adapting with an upright stimulus shifts the strength distribution towards negative values. ksA{ represents the contribution to the total strength from the large staircase, where ks is the attention coefficient, ie the degree to which the adaptation effect is attenuated when the subject does not attend to the adapting stimulus, and Ax is the contribution from the large stimulus. Bu represents the observer's bias to respond 'upright'. By subtracting equations (1) and (2), ks can be determined:
adapt upright
-
adapt inverted
respond inverted 0 respond upright + strength values Figure 4. Three strength distributions for the mechanism that controls the perception of the test stimulus. The center distribution describes the mechanisms in an unadapted state, and the left and right distributions describe the mechanisms in adapted states produced by upright and inverted staircases. (3) In an earlier paper (Shulman 1991), the parameters As and Ax were broken down into two components, the adaptation strength A when the adaptation and test stimulus were identical, and transfer conditions Ts and Tu reflecting the degree to which the adaptation effect was transferred when the adapting and test stimuli were different. As and A} in the present paper therefore equal ATS and ATX in the earlier paper. The adaptation and transfer parameters have been collapsed in the present paper for simplicity.
Attentional modulation of a figural aftereffect
17
The parameters As and Ax can be computed from the alone conditions. Equations (4) and (5) give the strength values for the conditions in which a single small staircase was presented: adapt inverted:
zx=As
+ Bu,
(4)
adapt upright:
zu = -As + Bu,
(5)
A = iUi-zJ-
(6)
The corresponding conditions for the large staircase yield Ax. One can also compute k from those opponent conditions in which the observer attends to the large staircase. The strength equations for upright and inverted adapting perspectives are given in equations (7) and (8): adapt inverted:
Z\ = -Ax+kxAs
adapt upright:
zu = Ax - kxAs + Bn,
fc, =
A
'
+
+ Bu,
^ " ^ .
(7) (8)
(9)
The quantity ?(z\ —zu) is added in equation (9), rather than subtracted as in equation (3), simply because of the convention that the perspective of the adapting stimulus in opponent blocks is specified with respect to the small staircase. The following values for these parameters were obtained: As = 0.75, Ax = 0.62, fcs = 0.55, and kx = 0.67. The reciprocal of k indicates the multiplicative factor by which the adaptation effect of the unattended stimulus has been attenuated. Therefore k = 0 indicates that the unattended input has been completely blocked, while k = 1 means that the attended and unattended stimuli have both produced their 'normal' adaptation effects (ie the adaptation effects in the alone block). The value of k reflects two factors: the degree to which the aftereffect can be attentionally modulated, and the degree to which the two opponent stimuli can be separately attended. (Even if a particular aftereffect depended on attention, the value of k would be 1 if the spatial configuration of the two adapting stimuli did not permit them to be separately selected.) The value of k is higher for the large staircase. This result suggests that subjects were less able to attend selectively to the large staircase, perhaps due to the spatial distribution of the two staircases. When the small staircase is attended, the line segments from surrounding regions that define the large staircase are outside the attentional focus. When the large staircase is attended, however, the entire spatial region defining the small staircase falls within the focus. This may prevent subjects from attending to the large staircase exclusively, thus resulting in a larger value of k. To the extent, therefore, that selective attention to each staircase is mediated partly by spatial location, and not simply by the color or grouping given by figural factors such as good continuity, selection of the small staircase will be more efficient. In the results section, it was noted that when subjects attended to the small staircase, the subsequent adaptation effect was appropriate to the perspective of that staircase. When subjects attended to the large staircase, however, the adaptation effect appropriate to that staircase was much smaller. This asymmetry appears to be a consequence both of the smaller adaptation effect for the large staircase in the alone blocks (ie lower values of Ax) and of less efficient selection of the large staircase (ie higher values of kx). The attention coefficient, k, was 0.55 or greater, whereas in previous work on the effect of attention on the three-dimensional motion aftereffect (Shulman 1991), the attention coefficients ranged approximately from 0 to 0.6. This difference may reflect
18
G L Shulman
the different spatial configurations of the adapting stimuli in the two studies. In the rotation studies, the two adaptation stimuli differed in size and color and were both centered on fixation, as in the present work. However, they did not overlap spatially, unlike the opponent staircases in the present work. This spatial segregation in the earlier studies may have increased the ability of subjects to select efficiently from the task-relevant stimulus, producing lower values of k. 6 General discussion The degree of adaptation produced by a staircase of a particular perspective is influenced by whether it is attended. Shifting attention from one member of the adapting-stimulus pair to the other produced reliable changes in the reported perspective of the test stimulus. The results of several studies have indicated that subjects have some control over the interpretation of ambiguous perspective figures (Pelton and Solley 1968; Hochberg and Peterson 1987), but the relationship between those results and the present results is unclear. The present work shows that the degree to which an unambiguous figure activates mechanisms involved in analyzing the test figure depends on attention. This does not necessarily imply that attention can alter in a specific manner the interpretation assigned to an ambiguous test figure. Reisberg and O'Shaughnessy (1984) have shown that the frequency of reversal of an ambiguous figure decreases when subjects are given a secondary task, suggesting that attention is involved in test-figure reversals. The attention coefficient was well above zero, suggesting some analysis of the 'unattended' figure. This analysis might have occurred because that figure was partly attended and/or because unattended stimuli automatically activate figural mechanisms to some extent. The current technique is not intended to determine whether any adaptation effect can be produced automatically in the complete absence of attention; it is very difficult to ensure that a stimulus presented for 30 s is totally unattended. Rather, the goal of these experiments is to determine if changes in the degree to which each adaptation component is attended shifts the subject's response to the test figure. A positive result indicates that the aftereffect can be attentionally modulated, but does not deny that some adaptation might be automatic. Therefore, nonzero values of k may reflect difficulties in attending exclusively to one or the other adapting figure, but are also consistent with automatic activation of the adapting mechanisms. The results of the present experiments confirm those of Chaudhuri (1990) and Shulman (1991) showing that adaptable representations can be attentionally modulated. Both in the current experiments and in those of Shulman (1991), representations pool information from different sources of depth information, while Chaudhuri's results implicate processes involved in two-dimensional motion analysis. The physiological correlates of these results are unclear. The results of single-cell studies in monkeys indicate that neurons in the MST respond to rotational stimuli in the frontal plane and in depth (Saito et al 1986), whereas other work indicates that MST cells respond to extraretinal influences (Newsome et al 1988). These cells may mediate the effect of attention on the three-dimensional motion aftereffect (Shulman 1991). Attentional modulations have also been reported in V4 and in inferotemporal cortex (Moran and Desimone 1985; Spitzer et al 1988). The inferotemporal cortex has been implicated in object recognition (Gross 1973) and could be involved in the representation of figural perspective. Studies of V4 suggest it is involved in the analysis of shape and color (Desimone et al 1985), but it is unclear if cell responses code figural perspective. Finally, a recent study involving measurement of cerebral blood-flow change in humans by using positron emission tomography shows that there
Attentional modulation of a figural aftereffect
19
are a variety of extrastriate areas that process shape, motion, and color and can be modulated by attention (Corbetta et al 1990). None of the attributes studied in those experiments, however, involved three-dimensional structure. Acknowledgment. This work was supported by the Office of Naval Research, grant N-0001489-J-1426, and by the McDonnell Center for Higher Brain Function. I am grateful to Maurizio Corbetta for comments on a draft of the manuscript. References Blakemore C, Campbell F, 1969 "On the existence of neurones in the human visual system selectively sensitive to the orientation and size of retinal images" Journal of Physiology 203 237-260 Blakemore C, Nachmias J, 1971 "The orientation specificity of two visual aftereffects" Journal of Physiology 213 157 -174 Chaudhuri A, 1990 "Modulation of the motion aftereffect by selective attention" Nature (London) 344 60 - 6 2 Corbetta M, MiezinF, Dobmeyer S, Shulman G, Petersen S, 1990 "Attentional modulation of neural processing of shape, color, and velocity in humans" Science 248 1556-1559 DesimoneR, Schein S, Moran J, Ungerleider L, 1985 "Contour, color and shape analysis beyond the striate cortex" Vision Research 25 441 -452 DosherB, Sperling G, Wurst S, 1986 "Tradeoffs between stereopsis and proximity luminance covariance as determinants of perceived 3D structures" Vision Research 26 973 - 991 GeorgesonM, Harris M, 1984 "Spatial selectivity of contrast adaptation: Models and data" Vision Research 24 729 - 742 Gross C, 1973 "Visual functions of inferotemporal cortex" in Handbook of Sensory Physiology volume VII(3B) Ed. R Jung (New York: Springer) pp 451 - 482 HochbergJ, Peterson M, 1987 "Piecemeal organization and cognitive components in object perception: perceptually coupled responses to moving objects" Journal of Experimental Psychology: General 116 370-380 Moran J, DesimoneR, 1985 "Selective attention gates visual processing in extrastriate cortex" Science 229 782-784 NawrotM, Blake R, 1989 "Neural integration of information specifying structure from stereopsis and motion" Science 244 716-718 NewsomeW, Wurtz R, Komatsu H, 1988 "Relation of cortical areas MT and MST to pursuit eye movements. II. Differentiation of retinal from extraretinal inputs" Journal of Neurophysiology 5 825 - 840 PeltonL, Solley C, 1968 "Acceleration of reversals of a Necker cube" American Journal of Psychology 81 585-589 Petersik A, Shepard A, MalschR, 1984 "A three-dimensional motion aftereffect produced by prolonged adaptation to a rotation simulation" Perception 13 488-497 ReisbergD, O'Shaughnessy M, 1984 "Diverting subjects' concentration slows figural reversals" Perception 1 3 4 6 1 - 4 6 8 SaitoH, YukieM, Tanaka K, Hikosaka K, Fukada Y, Iwai E, 1986 "Integration of direction signals of image motion in the superior temporal sulcus of the macaque monkey" Journal of Neuroscience 6 1 4 5 - 1 5 7 Sekuler R, Ganz L, 1963 "Aftereffect of seen motion with a stabilized retinal image" Science 139 419-420 Shulman G, 1991 "Attentional modulation of mechanisms that analyze rotation in depth" Journal of Experimental Psychology: Human Perception and Performance 17 726-737 Virsu V, 1975 "Determination of perspective reversals" Nature (London) 257 786 - 787
Attentional modulation of a figural aftereffect
Gordon L Shulman Washington University School of Medicine, Department of Neurology and Neurological Surgery, 660 S Euclid, St Louis, MO 63110, USA Received 22 September 1990, in revised form 24 June 1991
Abstract. Evidence is reported that indicates that adaptation of the Schroder staircase is affected by attention. In previous work it has been shown that if subjects adapt to an unambiguous staircase, responses to an ambiguous test figure are biased towards the opposing perspective. In the current work, subjects adapted to superimposed upright and inverted Schroder staircases. Both staircases were centered on a common fixation point and were of different sizes and colors. Attention to each staircase was controlled by asking subjects to detect color changes in the line segments that defined one or the other staircase. Responses to an ambiguous test figure depended on which of the adapting staircases was attended.
1 Introduction Adapting to a stimulus produces profound effects on the perception of subsequent stimuli. These adaptation effects are specific to spatial frequency (Blakemore and Campbell 1969; Georgeson and Harris 1984), orientation (Blakemore and Nachmias 1971), direction of motion (Sekuler and Ganz 1963), and other visual parameters of the adapting stimulus. This specificity suggests that adaptation affects particular mechanisms in the visual system. By studying the manner in which adaptation effects depend on how the adapting stimulus is attended, the role of attention in activating particular visual mechanisms can be studied. Effects of attention on adaptation have recently been reported both for twodimensional and for three-dimensional motion aftereffects (Chaudhuri 1990; Shulman 1991). Chaudhuri presented a moving-texture field which contained a central window framing an alphanumeric character that changed identity at 4 Hz. He measured the duration of the motion aftereffect induced by this stimulus on a subsequent stationary-texture field under two different attentional conditions. During the adaptation phase, subjects either fixated the central window (passive condition), or fixated and pressed a key whenever they detected a digit in the window (distractor condition). Chaudhuri reported that the duration of the motion aftereffect was much less when subjects attended to the central window (distractor condition) during the adaptation phase. He concluded that attention modulates the mechanisms underlying the motion aftereffect. Shulman (1991) reached a similar conclusion concerning the three-dimensional motion aftereffect. Previous work (Petersik et al 1984; Nawrot and Blake 1989) had shown that after adapting to a single stimulus rotating in depth in one direction, a test stimulus in parallel projection, whose direction of rotation is normally ambiguous, was more likely to be perceived as rotating in the opposite direction. Shulman (1991) presented two adapting squares rotating in depth in opposite directions and centered on a common fixation point. The squares were of different colors and sizes so as to make them readily discriminable. During the adaptation phase of the experiment, attention to the two stimuli was controlled by having subjects detect small perturbations in either the large or the small square. In the subsequent test phase, subjects judged the direction of motion of a test square in parallel projection. The perceived
8
G L Shulman
direction of rotation of the test square was determined by the rotation direction of the attended adapting square, indicating that attention controlled the degree to which the adapted mechanism was activated by the unambiguous rotating stimulus. Shulman (1991) noted an important difference between the method used in his study and in Chaudhuri's (1990). Chaudhuri used a 'distractor' design that determines whether the adapting effect produced by a stimulus is decreased if the subject concurrently performs another task on an irrelevant stimulus. The idea is that the distracting stimulus prevents the adapting stimulus from causing adaptation of those mechanisms that respond to the test. If one of the two adapting squares in the experiment of Shulman (1991) had been a 'neutral' stimulus that did not produce adaptation, then that experiment would also have involved a distractor technique. The use of two adapting squares, however, introduces an additional factor since both the small and large square activate the same mechanism as the test (ie they both can produce adaptation effects on the subsequent test stimulus). Therefore when activity is shifted from the large square to the small square, not only is the activity of the mechanism responding to the large square decreased, but the activity of that mechanism responding to the small square, which signals the opposite direction of motion, is increased. This increased activation of the opposing mechanism will further weaken the adaptation effect produced by the large square. Shulman (1991) argued that this opponent design would permit larger attention effects than a comparable distractor design and confirmed this prediction in an experiment that compared the two methods. This increased power, however, is only evident if two conditions are compared in which attention is directed to one or the other adapting component. (For example, under conditions in which the large square rotates to the right and the small square to the left, the condition in which one attends to the large square is compared with that in which one attends to the small square.) This is equivalent to comparing the negative aftereffect (the effect of the adapting condition on test responses relative to an unadapted baseline) produced when attending to one component with the negative aftereffect produced when attending to the other(1). A single adapting condition will produce a greater effect on test responses relative to the baseline in a distractor design than in an opponent design. This is because the aftereffect produced by a single component in an opponent paradigm is decreased by whatever adaptation is produced by the other unattended component, whereas no such opposing adaptation occurs in a distractor design. However, the change in the observer's response to the test figure as attention is shifted from one stimulus to another (from one adapting stimulus to the other in an opponent design, or from the adapting stimulus to the irrelevant stimulus in a distractor design) will be larger in an opponent design than a distractor design (Shulman 1991). The advantage of the opponent technique is therefore only exploited by analyses that directly compare different adaptation conditions, rather than by the more standard analyses in which a single condition is compared to a baseline. Since these ^ A n example will make this point clear. Suppose that when the large square rotating to the right is attended, the subject responds 'left' 75% of the time; when the small square rotating to the right is attended, the subject responds 'left' 30% of the time; and the unadapted or baseline response rate is 'left' 55% of the time. The difference between the adapting conditions is 45%, ie the sum of the negative aftereffect produced by the small square (25%) and the large square (20%). If one directly compares the negative aftereffects, since they are opposite in sign, the difference is again 45%. A statistical analysis directly comparing adapting conditions in which the large square rotated to the left and the small square rotated to the right will give identical results to an analysis that compares the negative aftereffects produced by each adapting condition.
Attentional modulation of a figural aftereffect
9
comparisons always involve the same physical stimulus (only the distribution of attention is changed), however, they allow a clean assessment of attentional effects. The mechanisms that underlie the three-dimensional motion aftereffect pool information over stereopsis and structure-from-motion. Nawrot and Blake (1989) have reported that adaptation to a rotating three-dimensional globe whose direction of motion is specified stereoscopically induces the opposing perception in an otherwise ambiguous globe produced by a parallel projection. Virsu (1975) has studied a different adaptation effect that also appears to pool different sources of depth information. He showed that adapting subjects to an unambiguous version of the Schrpder staircase, whose figure perspective was specified by stereopsis, created a bias towards the opposing perception in the traditional ambiguous test figure. This adaptation effect could also be produced by adapting subjects to an actual model of an unambiguous staircase. This aftereffect is also similar to that studied by Shulman (1991) since it involves a bistable test figure. Shulman (1991) has developed a simple model for describing the influence of attention on adaptation effects involving bistable test figures. In the current experiments the attentional dependence of adaptation of the Schroder staircase was therefore examined by using the opponent technique of Shulman (1991). During the adaptation phase, two unambiguous staircases were superimposed and centered on a common fixation point. On different adaptation blocks, subjects attended to one or the other staircase. The experiment determined if shifting attention from one adapting component to the other affected the subsequent perception of the test. Such a result would imply that attention can modulate the degree to which the adapting staircases activate the mechanisms that underlie the aftereffect. 2 Methods 2.1 Experiment la 2.1.1 Subjects. Ten naive observers participated in experiment la. 2.1.2 Apparatus and stimuli. Stimuli were displayed on a color monitor controlled by an Amiga 1000 microcomputer. A fixation cross was displayed at all times. The adapting and test stimuli are shown in figure 1. The adapting staircase could be either large or small, and when two adapting staircases were presented simultaneously, one was large and the other was small. The size of each step of the small staircase was 0.5 deg and the entire stimulus subtended 3.0 deg, while the step size of the large staircase was 1.25 deg and
Figure 1. Adaptation and test stimuli used in experiments la and lb. The left panel displays an unambiguous upright staircase, the middle panel an ambiguous staircase, and the right panel an unambiguous inverted staircase. The relative luminance of a line segment is indicated by the thickness of the line. The left and right unambiguous stimuli were displayed in adaptation blocks and the center ambiguous stimulus in test blocks.
10
G L Shulman
the entire stimulus subtended 7.5 deg. One size of adapting staircase was colored red and the other green, and the assignment of colors to sizes was counterbalanced across observers. In order to decrease the ambiguity of the adapting staircases, the luminance of the line segments depended on the distance of the segment from the observers (Dosher et al 1986). For this purpose, line segments were grouped into three categories: the segments defining the front face of the staircase, those transverse segments defining the steps of the staircase, and those defining the back face of the staircase. For the red staircase, the respective luminances were 39.7, 8.6, and 2.7 cd m~2, and for the green staircase, the luminances were 37.3, 12.7, and 3.42 cd m" 2 . During the adaptation sequence, color changes were occasionally introduced in one of the transverse segments that defined the protruding edges of the steps of each of the adapting staircases. (The identity of these line segments depend on whether the staircase is upright or inverted. In figure 1 numbers 1-3 are the relevant segments for the upright staircase, and numbers 4 - 7 are for the inverted staircase.) This was accomplished by increasing the blue color in one (randomly selected) of these line segments, which were positioned symmetrically with respect to the fixation point. The degree of color change was adjusted for each subject to yield a hit rate of 60%-70%. The color change occurred in each staircase on average once every 10 s with a duration of 100 ms. Since the position of the color change varied randomly about the fixation point, the brief duration ensured that subjects would maintain fixation during the block. The test staircase was the traditional ambiguous figure. The step size was 0.75 deg, the entire figure subtended 4.5 deg, and all line segments were white with a luminance of 31.5 cd m" 2 . 2.1.3 Procedure. Observers were instructed to maintain fixation on the fixation cross at all times. Observers participated in two types of blocks, opponent and alone. In opponent blocks, two superimposed staircases, one small and one large, were presented during the adaptation phase and these staircases were always in opposing perspective (ie one upright and one inverted). In alone blocks, only one staircase was present in the adaptation phase. On half the opponent blocks, subjects pressed a key when they detected color changes in the small adapting staircase, disregarding those that occurred in the large one, while in the other half they detected changes in the large staircase, disregarding those that occurred in the small staircase. The same adapting stimuli were therefore presented on the two types of opponent block, which only differed in terms of the response that was required to the adapting stimuli. In alone blocks, subjects detected color changes in the single adapting staircase. The basic sequence in a block of trials was as follows: The adapting staircases were presented for 30 s, during which the subject's only task was to detect color changes in the appropriate staircase. Following a 0.5 s interstimulus interval, the test stimulus was presented for 4 s. Subjects pressed one key if the staircase was currently seen as upright and a second key if inverted. If their perception of the test staircase reversed, they were instructed to press the corresponding key to indicate the new perspective. Following an intertrial interval of 2 s, a new adapt-test sequence was begun. One block consisted of ten trials or adapt-test sequences. During each block, the perspective of the adapting staircase or staircases and the designation of the taskrelevant staircase remained constant. At the beginning of each session, subjects participated in a baseline block of ten trials in which only the test phase was presented.
Attentional modulation of a figural aftereffect
11
2.1.4 Design. There were four different opponent blocks (subjects could attend to either the small or to the large staircase, both of which could be in upright or inverted perspective) and four different alone blocks (the factorial combination of the two staircase sizes and perspectives). The order of these eight blocks was randomized, and subjects observed two different orders, a total of sixteen blocks. Subjects also received a total of five baseline blocks. 2.2 Experiment lb After collecting data from the ten subjects in experiment la, it appeared that the large staircase produced less adaptation than the small staircase. A second group of ten subjects was therefore tested with the step size of the test staircase increased to 1.1 deg. By making the test size more similar to the large staircase, than to the small staircase, it was thought that the adapting effect produced by the large staircase would be increased relative to the small staircase, since adaptation effects often depend on the similarity of the adapting and test stimulus (Blakemore and Campbell 1969). These subjects also participated in 'perception' blocks to provide information on the effectiveness of the figural and proximity-luminance manipulation in producing unambiguous percepts of the adapting staircases. In these blocks, only the adaptation phase was presented. Subjects detected color changes in the designated staircase and indicated whether it was perceived as upright or inverted. Perception blocks were conducted both for alone and for opponent configurations. A perception block consisted of ten adaptation phases, each of 30 s duration, with an intertrial interval of 2 s. Eight perception blocks (four opponent and four alone) were conducted in a randomized order following the completion of the sixteen standard blocks (eight opponent and eight alone, as in experiment la). Subjects also served in four baseline blocks, rather than five as in experiment la. 3 Results In the analyses reported here, data from experiments la and lb are collapsed. As noted, the two experiments were the same except for the step size of the test staircase, which was increased in experiment l b to increase the adaptation effect produced by the large staircase. Statistical analyses explicitly comparing the two experiments are reported at the end of the results section. During the test phase, reversals were fairly common, producing an average of 2.2 responses during the 4 s test phase. Two subjects, KB and KN, showed particularly large response rates of 7.4 and 4.4. The average response rate of the remaining eighteen subjects was 1.8. On each test trial, the subject's response was defined as that perspective which was seen for the longest duration (Shulman 1991). Figures 2a and 2b show the percentage of upright responses, averaged across subjects, for both alone and opponent blocks(2). 3.1 Alone blocks The data from the alone blocks replicated the effect reported by Virsu (1975). For both the large and the small adapting staircase, subjects tended to perceive the test staircase in the perspective opposite to that of the adapting perspective. An analysis of variance was conducted with adapting staircase size (small, large) and perspective (2)
Since the figures show the percentage of upright test responses in each condition, the standard errors in the figures include variability due to differences in response bias across subjects. Although overall there was a preference for upright responses, the extent of this bias differed between subjects, and this variability will increase the standard errors. In all the statistical analyses reported here, however, one adapting condition is compared to another. Since the response bias is constant in both conditions, this variability will not be reflected in the standard error of the difference between the two conditions.
12
G L Shulman
(upright, inverted) as factors and the percentage of upright test responses as the dependent measure. The analysis yielded a significant effect of adapting perspective (^1,19 = 60.13, p < 0.001), indicating that the perception of the test figure changed reliably depending on whether the adapting perspective was upright or inverted. In this analysis, the effect of adapting to one perspective is directly compared with that of adapting to the other. If there were no adaptation effects, these two conditions would yield the same results. The difference in the test response to adaptation with an upright versus an inverted staircase is the sum of the negative aftereffects specified separately for each adapting staircase (see footnote 1). By summing the effects, however, one cannot determine if adapting with one perspective produces a larger effect than adapting with the other. For this reason, negative aftereffects were computed separately (table 1) for adaptation to upright and to inverted staircases by subtracting the appropriate quantity from the baseline response rate (65% upright responses). For example, when the adapting staircase was small and upright, the negative aftereffect was 33.6% (65%-31.4%), but when the adapting staircase was small and inverted, the negative aftereffect was 19.5% (84.5% -65%). The results of Mests, performed to determine whether the negative aftereffects for the four conditions differed from zero, indicated significant effects for the small staircase in upright (t19 = 5.42, p < 0.001) and inverted perspective {tl9 = 5.70, p < 0.001), and the large staircase in upright (tl9 = 5.74, p < 0.001) and inverted perspective {t19 = 3.39, p < 0.005). To determine whether the magnitude of these negative aftereffects depended upon condition, an analysis of variance was conducted with the negative aftereffect as the dependent measure (always expressed in positive sign) and adapting perspective (upright, inverted) and size (small, large) as factors. The analysis yielded a significant effect of 100
• inverted staircase s upright staircase
84.5%
Small staircase
71.7%
Small staircase attended
Large staircase
• small staircase inverted (large upright) ^ small staircase 65.1% ^Pright 56.4% T ( lar 8 e inverted)
Large staircase attended
(b) (a) Figure 2. Mean percentage of upright test responses in (a) the alone blocks and (b) the opponent blocks, The dotted line gives the baseline percentage of upright responses which was 65%. Table 1. Negative aftereffects for the small and large staircase in the alone and in the opponent condition. Standard errors are given in parentheses. Perspective
Upright staircase Inverted staircase
Alone
Opponent
small
large
small
large
33.6 (6.2) « 19.5 (3.4)
31.6 (5.5) 14 (4.1)
25 (4.8) 6.7 (3.4)
8.6 (4.2) 0.1 (5.2)
Attentional modulation of a figural aftereffect
perspective (F1A9 = 6.19, p < 0.05), indicating a larger negative aftereffect adaptation with an upright staircase.
13
for
3.2 Opponent blocks Figure 2b shows the percentage of upright test responses as a function of the perspective of the small staircase (upright or inverted) and whether it was attended during the adaptation phase. The data indicate that the magnitude of the adaptation effect produced by a staircase of a particular perspective depended on whether that staircase was attended. When the small staircase was attended, a large adaptation effect appropriate to the perspective of that staircase was found: the test staircase was seen in the perspective opposite to that of the small staircase (40% upright when the small staircase was upright, 71.7% upright when the small staircase was inverted). When the same adapting stimulus was presented and the large staircase was attended, however, the effect of staircase perspective now corresponded to that of the large staircase (56.4% upright responses when the large staircase was upright, 65.1% when the large staircase was inverted). This reversal, however, was quite small, a point that will be discussed later when a quantitative model of the results is introduced. Note that the left and right panels of figure 2b refer to the same physical conditions. All that differs between panels is the distribution of attention. If there were no effect of attention, the left and right panels should be equivalent. An analysis of variance with perspective of the small staircase (upright or inverted) and whether the small staircase was attended as factors and with the percentage of upright test responses as the dependent measure yielded a main effect of staircase perspective (Flt 19 = 8.40, p < 0.01) and an interaction of attention with staircase perspective {F1 19 = 38.6, p < 0.001). The latter interaction indicates that the effect of the perspective of an adapting staircase on test responses (ie its adaptation effect) depended on whether it was attended. The interaction term in the ANOVA is the most sensitive way to test for effects of attention since it considers the data from all four cells of the opponent condition. As noted in the introduction, the 'push-pull' concept underlying the opponent technique requires that different adaptation conditions be compared directly. In line with this argument, the attention effect can also be seen by considering how the test responses changed for a particular opponent configuration as attention was shifted from one adapting staircase to the other. In these comparisons, the physical stimulus is again held constant; only the direction of attention changes. Significant effects on test responses can therefore be unambiguously attributed to attention. For example, consider opponent adaptation blocks in which the small staircase was upright and the large staircase was inverted; when attention was directed to the small upright staircase, subjects perceived the test as upright on 40% of the trials, but when attention was directed to the large inverted staircase, the percentage of upright test responses was 65.1%, a significant shift (tl9 = 4.92, p < 0.001). Conversely, consider configurations in which the small staircase was inverted and the large staircase was upright; attending to the small inverted staircase produced 71.7% upright test responses, while attending to the large upright staircase produced 56.4% upright test responses. This difference was also significant (*19 = 3.67, p < 0.005). As noted in the introduction, these r-tests are equivalent to comparing the negative aftereffect produced when attending to one component with the negative aftereffect produced when attending to the other component. These negative aftereffects are shown in table 1. The baseline test response was 65% upright. A bias towards upright responses was also evident in Virsu's (1975) data. The preceding analyses were based on a measure in which the response on a trial was assigned to the percept that was present for the longest duration. The data were
14
G L Shulman
also analyzed with respect to the first response that the subject made. This procedure is more equivalent to that of Virsu (1975), who presented the test stimulus for 1 s and did not find any test reversals. KB and KN, the two subjects who showed large reversal rates, also gave 'upright' as their first response on virtually every trial (95.6% forKN, 98.7% for KB). This made their first response data uninformative and their data has been excluded from the analysis. The results from the remaining subjects look quite similar to those obtained by using the dominant response measure (figures 3a and 3b). The interaction of figure perspective and attention in the opponent blocks was again significant (Ful7 = 22.5, p < 0.001).
80.7%
• inverted staircase s upright staircase
76.3%
small staircase inverted (large upright) small staircase upright (large inverted)
Small staircase Large staircase attended (b) attended Figure 3. Mean percentage of 'first' upright test responses in (a) the alone blocks and (b) the opponent blocks. The dotted line gives the baseline percentage of upright responses which was 69%. Small staircase Large staircase
3.3 Perception blocks The overall percentages of time that the designated staircases were seen as upright or inverted were computed both for opponent and for alone configurations and are displayed in table 2. The upright staircase was virtually always seen as upright, but the inverted staircase showed some tendency to reverse. Table 2. Perception blocks. Percentage of time that the adaptation stimulus was seen in the indicated perspective. Standard errors are given in parentheses. Perspective
Inverted staircase Upright staircase
Small staircase
Large staircase
alone
opponent
alone
opponent
75.9 (10.1) 99.1 (0.64)
83.9 (8.2) 97.5 (1.2)
83.3 (5.9) 98.5 (0.86)
84.6 (6.3) 97.9 (1.4)
3.4 Comparisons of experiments la and lb The data from the alone block in experiment la suggested that the overall adaptation effect produced by the small staircase (this effect, the difference between the percentage of upright responses after adapting to inverted and upright staircases, was 87.5%-28.5% = 59%) was larger than that produced by the large staircase (the corresponding score was 79.8% - 3 8 . 3 % = 41.5%). This suggestion was supported somewhat by an ANOVA on the alone blocks, with staircase size and perspective as factors. The analysis yielded a marginal interaction of staircase size by perspective (Fl9 = 4.9, p < 0.052), indicating that the effect of staircase perspective (the overall
Attentional modulation of a figural aftereffect
15
adaptation effect) depended on staircase size. The analogous difference scores in experiment lb are 81.5% -34.4% = 47.1% for the small staircase, and 78.2%-28.5% = 49.7% for the large staircase. No interaction of size and perspective was observed in the ANOVA on this experiment. The two experiments were explicitly compared in an ANOVA with experiment as a between-subjects factor and staircase size and perspective as within-subject factors. The analysis yielded a marginal experiment by staircase size by perspective interaction [Flfl8 = 3.90, p = 0.061). These analyses are suggestive but not conclusive with respect to whether the increase in the size of the test staircase in experiment l b increased the size of the adaptation effect produced by the large staircase relative to the small one. The opponent blocks from the two experiments were also analyzed with experiment as a between-subjects factor and attention and staircase perspective as withinsubject factors (ie the same within-subject factors as reported in the opponent analyses of the pooled data). The experimental factor did not interact with any term involving the attention factor, supporting the observation that similar attentional effects occurred in the two experiments. Since these effects are the main focus of this report, it seems reasonable to collapse the two experiments. 4 Discussion The results from the alone block replicate those of Virsu (1975), indicating that adapting to an unambiguous staircase produces a tendency to perceive the opposing perspective in an ambiguous figure. One difference, however, is that the unambiguous staircases in the present experiment were defined by figural manipulations and proximity-luminance covariance rather than binocular disparity as in Virsu's (1975) study. The adaptation effect produced by a staircase of a particular perspective was influenced by whether the adapting staircase was attended. The assertion here is not that attention can produce adaptation by itself, nor that it can reverse the effect of staircase perspective (ie make a single upright staircase bias test responses towards upright), but just that it can modulate the adapting effect produced by a staircase of a particular perspective. This assertion is directly supported by the attention-bystaircase perspective interaction in the opponent blocks, as well as the separate analyses of each adapting configuration. Shifting attention from one adapting component to the other significantly affected responses to the test stimulus. Although the results of the experiment suggest some role of attention in modulating the aftereffect, the data indicated an unexpected asymmetry. Analyses of the negative aftereffects in the alone blocks indicated that adaptation was significantly more effective in biasing test responses when the adapting staircase was upright. This might reflect a 'ceiling' effect since test responses was biased towards upright. However, data from the perception blocks also indicated that the 'unambiguous' inverted staircase was sometimes prone to reversals. These reverals could weaken the adapting effectiveness of inverted staircases, resulting in the observed asymmetry. 5 Model A simple descriptive model developed by Shulman (1991) to analyze the effects of attention on the three-dimensional motion aftereffect was applied to the present data. The advantage of the model is that it includes a single parameter that describes the degree to which attention modulates the aftereffect. In the model, strength values represent the extent to which shape mechanisms coding the test stimulus signal an upright or inverted staircase. The strength values are distributed according to a standard normal, and on any test trial, the subject samples from the distribution (figure 4). If the sample strength value is positive, the
G L Shulman
16
subject responds 'upright', and if negative, the subject responds 'inverted'. The probability of an 'upright' response during the test phase is therefore the area under the curve to the right of zero, and the mean strength value can be determined by converting the probability of an 'upright' judgment to a z -score. Adapting to an upright perspective shifts the strength distribution towards negative values; adapting to inverted perspective shifts it towards positive values. During an opponent condition, therefore, each adapting component shifts the strength distribution in an opposing direction, and the net adaptation effect is the sum of the component effects. The degree to which an adapting stimulus changes the strength value elicited by a subsequent test stimulus is affected by whether the adapted stimulus is attended. Equations (1) and (2) give the strength values for conditions in which the observer attends to the small adapting stimulus (in the following exposition, the perspective of the adapting stimulus is always described with respect to the small staircase). Equation (1) describes the strength values for an 'upright' response when the small adapting stimulus was attended and was inverted, and equation (2) is for when the adapting stimulus was small and upright: adapt inverted:
z-x = As - ksAx + BU,
(1)
adapt upright:
zu = ~AS + ksAx +Bu.
(2)
Z\ is the z -score for the probability of making an 'upright' response when the adapting staircase is inverted. zn is the corresponding score when the adapting staircase is upright. As is the contribution to the total strength from the small adapting stimulus when it is attended.(3) It is negative in equation (2) since adapting with an upright stimulus shifts the strength distribution towards negative values. ksA{ represents the contribution to the total strength from the large staircase, where ks is the attention coefficient, ie the degree to which the adaptation effect is attenuated when the subject does not attend to the adapting stimulus, and Ax is the contribution from the large stimulus. Bu represents the observer's bias to respond 'upright'. By subtracting equations (1) and (2), ks can be determined:
adapt upright
-
adapt inverted
respond inverted 0 respond upright + strength values Figure 4. Three strength distributions for the mechanism that controls the perception of the test stimulus. The center distribution describes the mechanisms in an unadapted state, and the left and right distributions describe the mechanisms in adapted states produced by upright and inverted staircases. (3) In an earlier paper (Shulman 1991), the parameters As and Ax were broken down into two components, the adaptation strength A when the adaptation and test stimulus were identical, and transfer conditions Ts and Tu reflecting the degree to which the adaptation effect was transferred when the adapting and test stimuli were different. As and A} in the present paper therefore equal ATS and ATX in the earlier paper. The adaptation and transfer parameters have been collapsed in the present paper for simplicity.
Attentional modulation of a figural aftereffect
17
The parameters As and Ax can be computed from the alone conditions. Equations (4) and (5) give the strength values for the conditions in which a single small staircase was presented: adapt inverted:
zx=As
+ Bu,
(4)
adapt upright:
zu = -As + Bu,
(5)
A = iUi-zJ-
(6)
The corresponding conditions for the large staircase yield Ax. One can also compute k from those opponent conditions in which the observer attends to the large staircase. The strength equations for upright and inverted adapting perspectives are given in equations (7) and (8): adapt inverted:
Z\ = -Ax+kxAs
adapt upright:
zu = Ax - kxAs + Bn,
fc, =
A
'
+
+ Bu,
^ " ^ .
(7) (8)
(9)
The quantity ?(z\ —zu) is added in equation (9), rather than subtracted as in equation (3), simply because of the convention that the perspective of the adapting stimulus in opponent blocks is specified with respect to the small staircase. The following values for these parameters were obtained: As = 0.75, Ax = 0.62, fcs = 0.55, and kx = 0.67. The reciprocal of k indicates the multiplicative factor by which the adaptation effect of the unattended stimulus has been attenuated. Therefore k = 0 indicates that the unattended input has been completely blocked, while k = 1 means that the attended and unattended stimuli have both produced their 'normal' adaptation effects (ie the adaptation effects in the alone block). The value of k reflects two factors: the degree to which the aftereffect can be attentionally modulated, and the degree to which the two opponent stimuli can be separately attended. (Even if a particular aftereffect depended on attention, the value of k would be 1 if the spatial configuration of the two adapting stimuli did not permit them to be separately selected.) The value of k is higher for the large staircase. This result suggests that subjects were less able to attend selectively to the large staircase, perhaps due to the spatial distribution of the two staircases. When the small staircase is attended, the line segments from surrounding regions that define the large staircase are outside the attentional focus. When the large staircase is attended, however, the entire spatial region defining the small staircase falls within the focus. This may prevent subjects from attending to the large staircase exclusively, thus resulting in a larger value of k. To the extent, therefore, that selective attention to each staircase is mediated partly by spatial location, and not simply by the color or grouping given by figural factors such as good continuity, selection of the small staircase will be more efficient. In the results section, it was noted that when subjects attended to the small staircase, the subsequent adaptation effect was appropriate to the perspective of that staircase. When subjects attended to the large staircase, however, the adaptation effect appropriate to that staircase was much smaller. This asymmetry appears to be a consequence both of the smaller adaptation effect for the large staircase in the alone blocks (ie lower values of Ax) and of less efficient selection of the large staircase (ie higher values of kx). The attention coefficient, k, was 0.55 or greater, whereas in previous work on the effect of attention on the three-dimensional motion aftereffect (Shulman 1991), the attention coefficients ranged approximately from 0 to 0.6. This difference may reflect
18
G L Shulman
the different spatial configurations of the adapting stimuli in the two studies. In the rotation studies, the two adaptation stimuli differed in size and color and were both centered on fixation, as in the present work. However, they did not overlap spatially, unlike the opponent staircases in the present work. This spatial segregation in the earlier studies may have increased the ability of subjects to select efficiently from the task-relevant stimulus, producing lower values of k. 6 General discussion The degree of adaptation produced by a staircase of a particular perspective is influenced by whether it is attended. Shifting attention from one member of the adapting-stimulus pair to the other produced reliable changes in the reported perspective of the test stimulus. The results of several studies have indicated that subjects have some control over the interpretation of ambiguous perspective figures (Pelton and Solley 1968; Hochberg and Peterson 1987), but the relationship between those results and the present results is unclear. The present work shows that the degree to which an unambiguous figure activates mechanisms involved in analyzing the test figure depends on attention. This does not necessarily imply that attention can alter in a specific manner the interpretation assigned to an ambiguous test figure. Reisberg and O'Shaughnessy (1984) have shown that the frequency of reversal of an ambiguous figure decreases when subjects are given a secondary task, suggesting that attention is involved in test-figure reversals. The attention coefficient was well above zero, suggesting some analysis of the 'unattended' figure. This analysis might have occurred because that figure was partly attended and/or because unattended stimuli automatically activate figural mechanisms to some extent. The current technique is not intended to determine whether any adaptation effect can be produced automatically in the complete absence of attention; it is very difficult to ensure that a stimulus presented for 30 s is totally unattended. Rather, the goal of these experiments is to determine if changes in the degree to which each adaptation component is attended shifts the subject's response to the test figure. A positive result indicates that the aftereffect can be attentionally modulated, but does not deny that some adaptation might be automatic. Therefore, nonzero values of k may reflect difficulties in attending exclusively to one or the other adapting figure, but are also consistent with automatic activation of the adapting mechanisms. The results of the present experiments confirm those of Chaudhuri (1990) and Shulman (1991) showing that adaptable representations can be attentionally modulated. Both in the current experiments and in those of Shulman (1991), representations pool information from different sources of depth information, while Chaudhuri's results implicate processes involved in two-dimensional motion analysis. The physiological correlates of these results are unclear. The results of single-cell studies in monkeys indicate that neurons in the MST respond to rotational stimuli in the frontal plane and in depth (Saito et al 1986), whereas other work indicates that MST cells respond to extraretinal influences (Newsome et al 1988). These cells may mediate the effect of attention on the three-dimensional motion aftereffect (Shulman 1991). Attentional modulations have also been reported in V4 and in inferotemporal cortex (Moran and Desimone 1985; Spitzer et al 1988). The inferotemporal cortex has been implicated in object recognition (Gross 1973) and could be involved in the representation of figural perspective. Studies of V4 suggest it is involved in the analysis of shape and color (Desimone et al 1985), but it is unclear if cell responses code figural perspective. Finally, a recent study involving measurement of cerebral blood-flow change in humans by using positron emission tomography shows that there
Attentional modulation of a figural aftereffect
19
are a variety of extrastriate areas that process shape, motion, and color and can be modulated by attention (Corbetta et al 1990). None of the attributes studied in those experiments, however, involved three-dimensional structure. Acknowledgment. This work was supported by the Office of Naval Research, grant N-0001489-J-1426, and by the McDonnell Center for Higher Brain Function. I am grateful to Maurizio Corbetta for comments on a draft of the manuscript. References Blakemore C, Campbell F, 1969 "On the existence of neurones in the human visual system selectively sensitive to the orientation and size of retinal images" Journal of Physiology 203 237-260 Blakemore C, Nachmias J, 1971 "The orientation specificity of two visual aftereffects" Journal of Physiology 213 157 -174 Chaudhuri A, 1990 "Modulation of the motion aftereffect by selective attention" Nature (London) 344 60 - 6 2 Corbetta M, MiezinF, Dobmeyer S, Shulman G, Petersen S, 1990 "Attentional modulation of neural processing of shape, color, and velocity in humans" Science 248 1556-1559 DesimoneR, Schein S, Moran J, Ungerleider L, 1985 "Contour, color and shape analysis beyond the striate cortex" Vision Research 25 441 -452 DosherB, Sperling G, Wurst S, 1986 "Tradeoffs between stereopsis and proximity luminance covariance as determinants of perceived 3D structures" Vision Research 26 973 - 991 GeorgesonM, Harris M, 1984 "Spatial selectivity of contrast adaptation: Models and data" Vision Research 24 729 - 742 Gross C, 1973 "Visual functions of inferotemporal cortex" in Handbook of Sensory Physiology volume VII(3B) Ed. R Jung (New York: Springer) pp 451 - 482 HochbergJ, Peterson M, 1987 "Piecemeal organization and cognitive components in object perception: perceptually coupled responses to moving objects" Journal of Experimental Psychology: General 116 370-380 Moran J, DesimoneR, 1985 "Selective attention gates visual processing in extrastriate cortex" Science 229 782-784 NawrotM, Blake R, 1989 "Neural integration of information specifying structure from stereopsis and motion" Science 244 716-718 NewsomeW, Wurtz R, Komatsu H, 1988 "Relation of cortical areas MT and MST to pursuit eye movements. II. Differentiation of retinal from extraretinal inputs" Journal of Neurophysiology 5 825 - 840 PeltonL, Solley C, 1968 "Acceleration of reversals of a Necker cube" American Journal of Psychology 81 585-589 Petersik A, Shepard A, MalschR, 1984 "A three-dimensional motion aftereffect produced by prolonged adaptation to a rotation simulation" Perception 13 488-497 ReisbergD, O'Shaughnessy M, 1984 "Diverting subjects' concentration slows figural reversals" Perception 1 3 4 6 1 - 4 6 8 SaitoH, YukieM, Tanaka K, Hikosaka K, Fukada Y, Iwai E, 1986 "Integration of direction signals of image motion in the superior temporal sulcus of the macaque monkey" Journal of Neuroscience 6 1 4 5 - 1 5 7 Sekuler R, Ganz L, 1963 "Aftereffect of seen motion with a stabilized retinal image" Science 139 419-420 Shulman G, 1991 "Attentional modulation of mechanisms that analyze rotation in depth" Journal of Experimental Psychology: Human Perception and Performance 17 726-737 Virsu V, 1975 "Determination of perspective reversals" Nature (London) 257 786 - 787
