Psychonomic Bulletin & Review 1994, 1 (1), 115-118
Divided attention: Storing and classifying briefly presented objects HAROLD PASHLER University of California, San Diego, California Although in many studies divided attention has been examined by having people perform the same task (e.g., report or search) with a large number of objects, in few studies have people had to perform two logically independent tasks involving the same brief display. In two experiments, subjects saw 200-msec arrays of characters. In dual-task blocks, they classified the color of some or all of the items (making an immediate response) and stored the shape of some of the items for a later recognition test. There was not much mutual interference between classifying and storing per se. However, the tasks were by no means independent: there was substantial interference when different objects from the array had to be stored for one task and classified for the other. The results confirm that wholly unrelated visual tasks depend on the same input-attention system and suggest that attending to an object for any purpose may entail storing a representation of it in visual short-term memory. Our visual systems are ordinarily confronted with input originating from different objects, and this input conveys information that could be useful for many different purposes. When a person who is sitting at a bar reaches for a beer, for example, some of the visual input confirms that it is indeed a beer mug that is about to be grasped, while other input provides the location of potential obstacles to the reaching movement, features of the beer mug that may be important for planning how to grasp it, and so on. At the same time, still other visual input might be useful for other activities, such as making postural adjustments (to avoid falling over) or detecting dangers (such as any possible beer mugs that might be flying across the room). Many researchers have explored people's ability to process different visual inputs at the same time (e.g., Duncan, 1980; Eriksen & Spencer, 1969), or to divide attention, as it is commonly termed. In just about all these studies, however, what the subjects had to do with one of the inputs was the same as what they had to do with all the others. For example, in a whole-report task, subjects try to store and report all the items in a display (e.g., Sperling, 1960), and in a visual search task, theyevaluate each item with respect to whether or not it is the target that they are looking for (e.g., Estes & Taylor, 1964). Since the same kinds of operations are applied to each of the inputs, such studies may say little about control and
This work was supported by grants from ONR (NOOOI4-91-J-1735) and NIMH (l-R29-MH45584-oI). The author is grateful to Alann Lopes for technical assistance, to Andrea Alfonso for assistance in running subjects, and to James C. Johnston and Allen Osman for useful discussions. Richard Schweickert and Addie Dutta provided useful comments on an earlier version. Address correspondence to the author at the Department of Psychology 0109, DC San Diego, La Jolla, CA, 92093 (e-mail: [email protected]).
capacity limits in the more ordinary situation where some inputs are used for one purpose and others are used for another. The ways in which the terms visual attention or spatial attention are often used (both by psychologists and by laymen) seem to presuppose that we have a single processing limitation and/or a single input system that governs all visual input-regardless of what use the input may be put to. (Otherwise, the concept of visual attention would be too crude to be of much use.) Oddly, though, this presupposition has seldom been tested, and there would seem to be no firm basis for it. The two experiments reported here represent a preliminary examination of this issue; they provide a look at what happens when people try to carry out two different visual tasks simultaneously and both tasks involve the same brief array (although not necessarily the same elements within the array). Studies of visual divided attention (involving many stimuli and just one task) confirm the common sense idea that visual processing is subject to capacity limits. Several fairly easy discriminations can often be performed in parallel without any apparent capacity limits (see, e.g., Shiffrin & Gardner, 1972; Treisman & Gelade, 1980). However, when several different difficult visual discriminations must be performed, limits on capacity become evident (see, e.g., Duncan, 1987; Kleiss & Lane, 1986). Detecting a target in a search task also tends to interfere with detecting another target (see, e.g., Duncan, 1980). Interestingly, though, judging or detecting two different features of the same object (e.g., the color and orientation of aline) can often be accomplished without evident capacity limits (Allport, 1971; Duncan, 1984). As mentioned above, in very few divided attention studies have subjects been required to perform two completely unrelated visual tasks simultaneously. A study performed by Hoffman, Nelson, and Houck (1983) was among the few in which this was done. Their subjects searched
115
Copyright 1994 Psychonomic Society, Inc.
116
PASHLER
successive frames of letters for a digit target and determined which direction a figure shaped like a letter "C" pointed. There was substantial interference between these two tasks, and the interference was greater when the two figures were spatially separated. The present studies involved two logically independent tasks performed on a single brief display, each involving a different (separable) dimension. Task 1 required subjects to classify and respond immediately to the color of some letters; in Task 2, they were to retain in short-term memory the form of letters from the same array. One could envision various ways in which performance in the two tasks might be related. First, processing might proceed completely independently. Second, getting information into short-term memory (Task 2) might draw heavily on the same processing resources that are needed for perceptual or cognitive aspects of the color classification task (Task 1). This possibility would fit the suggestions of Kahneman (1973) and Broadbent (1982), who hypothesized extremely general mental processing limitations. A third possibility (which would not exclude the second) is that the two tasks might be constrained to use information from the same locations (or objects) but that the tasks could otherwise proceed independently. More specifically, people might be able to classify the letters in Task 1 while they store letters for Task 2, with no mutual interference between these processes, but only as long as the letters they classify in Task 1 are the very same letters as the ones they classify in Task 2. This will be referred to as yoking of the two tasks at the input stage. Other possibilities could also be considered. In the experiments reported below, subjects' short-term memory for the letter forms was assessed with the use of a matching task. The matching task required them to decide whether or not a second test display of letters that was presented slightly later matched the original target display. Pashler (1988) found that subjects make many errors in this task even when the interval between the displays is only 67 msec. Subjects could not hold on to more than about four or five characters, roughly the "span of apprehension" found in whole-report tasks (Sperling, 1960). This is surprising in some ways, since one might have supposed that momentary awareness of what is in a visual scene would include the ability to detect changes if the display flashes off and on again rapidly. In any case, performance was almost as good with reflected-letter forms as with letters, suggesting that people accomplish the task by storing information in a visual short-term memory (cf. Frick, 1984; Phillips, 1974, 1983), rather than by storing more abstract information.
TARGET
TEST
DISPLAY
DISPLAY QTV NXE
Time
~.
r ms o o Reaction
( RESPOND "DIFFERENT" AT LEISURE)
Time
Figure 1. The basic sequence of events in the two experiments. Sometimes the subjects respond to the color of elements in the target display (see text); in all cases, they try to remember elements in the target display and determine whether the test display has changed or not. Responses to the target display are speeded; the same/different response to the test display is unspeeded, and the test display remains present until this response.
250-msec blank interval and then the target display. The target display consisted of six letters (selected randomly without replacement from a set of 20 letters of relatively low confusability) presented against the black background for 200 msec. (Each character measured .6 0 x 1.1 0 of visual angle, based on a typical viewing distance of 60 cm. The dimensions of the whole display were 4.0 0 [width] x 3.4 0 [height].) There were 12 blocks of 42 trials in the experiment, with a rest period between blocks. Twelve subjects (mostly University of California, San Diego, undergraduates) participated in Experiment 1, which was run on ffiM PCs with NEC Multisync 2.2 displays. The first task required a response to the target display. On singletask blocks, the letters in the target display were gray (of approximately the same contrast as the colored letters to be described next); the subject made no response to them. On one-item-colored blocks, a single, randomly selected letter in the target display was either red or green (color selected randomly with replacement). The subject responded as quickly as possible by pressing the "Zoo key if it was red and the "X" key if it was green, using the index or the middle finger of the left hand. On all-items-colored blocks, all six letters were red or all six were green; the subject responded as in the one-item-colored blocks. There were 4 blocks of each type, with block type counterbalanced over the sequence of 12 blocks, and random ordering of trials within a block. (The luminances of the red, green, and gray were roughly comparable.) Two hundred milliseconds after the target display appeared, it disappeared; after another 800 msec, a test display of six letters appeared in the position formerly occupied by the target display. (On a typical trial, this occurred several hundred milliseconds after the response to the target display had been made.) On half the trials, this test display consisted of the same six letters as did the target display. On the other half of the trials, one of the six letters was randomly switched to another letter from the stimulus set. The test display did not disappear until the subject had made an unspeeded same/different judgment, responding with the right hand, and the next trial began after a further 250-msec delay. (The color of the item in any given position in the test display was always the same as the color of the item in the corresponding position of the target display, whatever the condition; thus, color was completely irrelevant in the same/different task.)
Results and Discussion EXPERIMENT 1
The subjects' responses to the color of the target displays were rapid, but, not surprisingly, it took a little longer to respond when only a single item in the target display was colored (499 msec) rather than all of them Method Figure 1 shows the method used in the present experiments. Subjects (445 msec). An analysis of variance with two factorsfirst saw a target display, and in some conditions they had to respond color condition and same/different-showed that color very rapidly to the color of some letter or letters in this display. On each trial, the target display was preceded by a plus sign (fixation point) condition made a significant difference [F( 1,11) = 36.7, that was exposed in the center of the screen for 1 sec, followed by a p < .001]. Response times on same and different trials
Storing and Discriminating
VISUAL DIVIDED ATTENTION did not differ. There were more errors for color responses in the one-item-eolored(9.6%) than in the all-items-eolored condition (6.4%) [F(I,l1) = 40.7, p < .001], but no other significant effects in error rates. The question of primary interest was how well subjects could store and retain the form of the letters in visual short-term memory, as indicated by their performance on the same/different judgment. Differences among the conditions on this task were fairly modest: overall error rates were 26.4 %, 35.1 %, and 32.0% in the single-task, oneitem-eolored, and all-items-colored conditions [F(2,22) = 16.9, p < .001]. The subjects failed to find a difference more often (45.2%) than they falsely reported one (17.1%) [F(l,ll) = 26.5,p < .001], and there was an interaction between same/different and condition [F(2,22) = 6.1, p < .01], since performing the color tasks increased error rates on different displays more than on same displays. The difference in same/different task performance between single-task and all-items-eoloredconditions was significant [F(1, 11) = 13.0, p < .01] but relatively modest (about 5%). Earlier findings have indicated that response selection and short-term memory storage can be functionally separate (see Pashler, 1993). The dual-task cost observed here could well be due to preparing, rather than carrying out, the concurrent task. The modest dual-task decrement here suggests that the tasks do not generate much mutual interference. However, a further analysis revealed that the tasks were by no means independent of each other. In the one-colored-item condition, when a single letter was changed, it might be the letter that was colored, or it might be one of the other five letters. Although switches were no more likely to occur in the colored position, subjects detected 56.3 % of these; they detected only 29.4% of switches of the uncolored letters [F(l,l1) = 34.0, p < .001]. Thus, the input of visual information from the different objects for the two tasks may be so constrained that the object providing color information is much more likely to be stored for the memory task than are the other objects. A strong interpretation of this result is that the color task required visual attention to be allocated to the colored item(s), and that visual attention is both necessary and sufficient to ensure that items are stored in short-term memory (subject to data and time limitations). If this is the case, the linkage between which items are targets for color classification and which items are stored may be obligatory, and not merely a matter of optional strategy. If so, the difference in the likelihood of storing items bearing color information and items not bearing color information should persist even if subjects are given an incentive to classify some objects and store information from different objects. One might ask why people do as well as they do in storing information in the one-item-colored condition if the color task forces them to attend to just one item. Subjects plainly store more than just the single colored letter-in fact, they retain almost as many total items as in the singletask condition. This could be reconciled with the obliga-
117
tory linkage hypothesis, however, if one supposes that subjects have sufficient time to shift their attention from the colored item to the remaining items in these (unmasked) displays. On this view, same/different performance would be limited primarily by the storage capacity of short-term memory (as in a whole-report task, which is scarcely affected by exposure duration; Sperling, 1960).
EXPERIMENT 2 Store Some Items While Classifying Others If the linkage between selection of items for one task and for the other is truly obligatory, it ought to be extremely difficult for people to classify some objects from an array while storing others if the time for which a display remains available is abbreviated, as by a mask. This idea was tested in Experiment 2.
Method Experiment 2 differed from Experiment I only as follows: In each target display, there were always three colored letters (all red or all green) and three gray letters (in Positions 2,4, and 6, numbered along the upper, then lower, row), and the subject always responded immediately to the color. In half the blocks of trials, when a letter in the target display was switched in the test display, it was always one of the colored letters (colored-switch condition). In the other half of the blocks, a switched letter was always one of the gray letters (gray-switch condition). The subjects were told about each block type, so in the gray-switch condition they understood (through a series of consecutive blocks) that they needed to remember the form information only about the gray items, not the colored items. In the masked version of the experiment, a mask (consisting of gray sharp signs in the six letter positions) was present throughout the 800 msec between the offset of the target display and the onset of the test display, whereas in the unmasked version of the experiment, there was no such mask. The first seven blocks were in one condition (colored-switch or gray-switch), and the last seven blocks were in the other condition; order was counterbalanced, and the first block of each condition was considered practice.
Results and Discussion Sixteen subjects participated in the masked version. Mean response times for the color classification responses were 411 msec for the colored-switch condition, and 421 msec for the gray-switch condition (not a significant difference). Errors on the color task did not differ by condition. In the same/different judgment task, error rates were as follows: For the colored-switch condition, errors were 19.7 % and 22. 1% for same and different trials, respectively. For the gray-switch condition, errors were 42.3% and 44.9% for same and different trials, respectively. The difference between color-switch and gray conditions was significant [F(1,15) = 117, p < .001]; differences between same and different trials were not. Twelve subjects participated in the unmasked version. For the colored-switch condition, same/different errors were 11.8% and 17.5% for same and different trials, respectively; for the gray-switch condition, errors were 12.6% and 35.1 % for same and different trials, respectively. Thus, the results were similar to the masked version, but the effect of switch condition on accuracy was much smaller.
118
PASHLER GENERAL DISCUSSION
Several conclusions can be drawn. First, people do not have much trouble in responding to the color of some letters and storing the forms of these items in short-term memory. Memory storage is not quite as good as it is for a single-task control, but the interference is still quite modest. What causes people much greater difficulty, however, is the storing of some items in short-term memory while they are responding to the color of different items in the same display. These results are consistent with the view that visual attention is allocated to objects or their locations (the present data do not distinguish among these possibilities), and that this results in information from these locations being made available for multiple purposes-"awareness," memory, or voluntary responses in any task. Although many investigators have assumed a single input-selection process governing the intake of information regardless of the purpose for which the information is used "downstream," evidence for this has been sparse. The results are also consistent with Duncan's (1985) suggestion that any access to even the simplest visual attributes requires allocation of visual attention (see also Kahneman & Treisman, 1984). Logically speaking, the color task could have been performed without subjects' selectively orienting to the locations of colored objects; indeed, Treisman and Gelade (1980) have suggested that the presence of simple features can be detected without even registering their locations (but see Johnston & Pashler, 1990). The results fit nicely with recent findings of Tsal and Lavie (1993). They had subjects report an array of letters, but only on those trials on which a single letter that preceded the display had certain specified properties. In general, people were much more likely to report an item in the array that occupied the same position as had the single letter that preceded it, even when the position of that letter was irrelevant for the subjects' task. Tsal and Lavie's findings show that people tend to pick up information in a display from the same position as that at which they have just picked up color or identity information in a preceding display; the results reported here show that people have great trouble using color information from some items and storing form information from other items in the very same display, even when it would serve their purposes to do so. Finally, the results would appear to suggest that storage in visual shortterm memory may be an operation that occurs as a "contingently automatic" consequence of having visual attention allocated to a given location. Yet while the results suggest this, they do not prove it beyond a doubt, because the subjects did not have any incentive not to store the forms, even in the colored-switch condition. This interesting issue could bear further examination, because Phillips and Christie (1977)-on the basis of a very different kind of evidence-inferred that subjects could attend to a new pattern without writing over information just stored in visual short-term memory. And of course the present investigation has only scratched the surface of the broader issue mentioned at the beginning of this article-namely, what happens when people must use visual information in the same display for quite different purposes. Obviously, there are many other kinds of tasks-including ones of practical importance-for which such questions could be addressed. REFERENCES ALLPORT, D. A. (1971). Parallel encoding within and between elementary stimulus dimensions. Perception & Psychophysics, 10, 104-108. BROADBENT, D. E. (1982). Task combination and the selective intake of information. Acta Psychologica, 50, 253-290. DUNCAN, J. (1980). The locus of interference in the perception ofsimultaneous stimuli. Psychological Review, 87, 272-300.
DUNCAN, J. (1984). Selective attention and the organization of visual information. Journal of Experimental Psychology: General, 113, 501-517. DUNCAN, J. (1985). Visual search and visual attention. In M. Posner and O. S. M. Marin (Eds.), Attention and performance XI (pp. 85105). Hillsdale, NJ: Erlbaum. DUNCAN, J. (1987). Attention and reading: Wholes and parts in shape recognition-A tutorial review. In M. Coltheart (Ed.), Attention and performance XII: The psychology of reading (pp. 39-61). Hillsdale, NJ: Erlbaum. ERIKSEN, C. W., & SPENCER, T. (1969). Rate of information processing in visual perception: Some results and methodological considerations. Journal ofExperimental Psychology Monographs, 79(Pt. 2), 1-16. ESTES, W. K., & TAYLOR, H. A. (1964). A detection method and probabilistic models for assessing information processing from brief visual displays. Proceedings of the National Academy of Sciences, 52, 446-454. FRICK, R. W. (1984). Using both an auditory and a visual short-term store to increase digit span. Memory & Cognition, 12, 507-514. HOFFMAN, J. E., NELSON, B., & HOUCK, M. R. (1983). The role of attentional resources in automatic detection. Cognitive Psychology, 51, 379-410. JOHNSTON, J. C., & PASHLER, H. (1990). Close binding of identity and location in visual feature perception. Journal of Experimental Psychology: Human Perception & Performance, 16, 843-846. KAHNEMAN, D. (1973). Attention and effort. Cambridge, MA: Harvard University Press. KAHNEMAN, D., & TREISMAN, A. (1984). Changing views of attention and automaticity. In R. Parasurarnan & D. R. Davies (Eds.), Varieties of attention (pp. 29-62). New York: Academic Press. KLEISS, J. A., & LANE, D. M. (1986). Locus and persistence of capacity limitations in visual information processing. Journal ofExperimental Psychology: Human Perception & Performance, 12, 200-210. PASHLER, H. (1988). Familiarity and visual change detection. Perception & Psychophysics, 44, 369-378. PASHLER, H. (1993). Dual-task interference and elementary mental mechanisms. In D. Meyer & S. Kornblum (Eds.), Attention and performance XIV (pp. 245-264). Cambridge, MA: MIT Press. PHILLIPS, W. A. (1974). On the distinction between sensory storage and short-term visual memory. Perception & Psychophysics, 16, 283-290. PHILLIPS, W. A. (1983). Short-term visual memory. Philosophical Transactions ofthe Royal Society ofLondon: Series B, 302, 295-309. PHILLIPS, W. A., & CHRISTIE, D. F. M. (1977). Interference with visualization. Quarterly Journal of Experimental Psychology, 29, 637-650. SHIFFRlN, R. M., & GARDNER, G. T. (1972). Visual processing capacity and attentional control. Journal ofExperimental Psychology, 93, 78-82. SPERLING, G. (1960). The information available in brief visual presentations. Psychological Monographs: General & Applied, 74 (Whole No. 498), 1-29. TREISMAN, A., & GELADE, G. (1980). A feature integration theory of attention. Cognitive Psychology, 12, 97-136. TSAL, Y., & LAVIE, N. (1993). Location dominance in attending to color and shape. Journal of Experimental Psychology: Human Perception & Performance, 19, 131-139. (Manuscript received March 13, 1993; revision accepted for publication August 30, 1993.)
Divided attention: Storing and classifying briefly presented objects HAROLD PASHLER University of California, San Diego, California Although in many studies divided attention has been examined by having people perform the same task (e.g., report or search) with a large number of objects, in few studies have people had to perform two logically independent tasks involving the same brief display. In two experiments, subjects saw 200-msec arrays of characters. In dual-task blocks, they classified the color of some or all of the items (making an immediate response) and stored the shape of some of the items for a later recognition test. There was not much mutual interference between classifying and storing per se. However, the tasks were by no means independent: there was substantial interference when different objects from the array had to be stored for one task and classified for the other. The results confirm that wholly unrelated visual tasks depend on the same input-attention system and suggest that attending to an object for any purpose may entail storing a representation of it in visual short-term memory. Our visual systems are ordinarily confronted with input originating from different objects, and this input conveys information that could be useful for many different purposes. When a person who is sitting at a bar reaches for a beer, for example, some of the visual input confirms that it is indeed a beer mug that is about to be grasped, while other input provides the location of potential obstacles to the reaching movement, features of the beer mug that may be important for planning how to grasp it, and so on. At the same time, still other visual input might be useful for other activities, such as making postural adjustments (to avoid falling over) or detecting dangers (such as any possible beer mugs that might be flying across the room). Many researchers have explored people's ability to process different visual inputs at the same time (e.g., Duncan, 1980; Eriksen & Spencer, 1969), or to divide attention, as it is commonly termed. In just about all these studies, however, what the subjects had to do with one of the inputs was the same as what they had to do with all the others. For example, in a whole-report task, subjects try to store and report all the items in a display (e.g., Sperling, 1960), and in a visual search task, theyevaluate each item with respect to whether or not it is the target that they are looking for (e.g., Estes & Taylor, 1964). Since the same kinds of operations are applied to each of the inputs, such studies may say little about control and
This work was supported by grants from ONR (NOOOI4-91-J-1735) and NIMH (l-R29-MH45584-oI). The author is grateful to Alann Lopes for technical assistance, to Andrea Alfonso for assistance in running subjects, and to James C. Johnston and Allen Osman for useful discussions. Richard Schweickert and Addie Dutta provided useful comments on an earlier version. Address correspondence to the author at the Department of Psychology 0109, DC San Diego, La Jolla, CA, 92093 (e-mail: [email protected]).
capacity limits in the more ordinary situation where some inputs are used for one purpose and others are used for another. The ways in which the terms visual attention or spatial attention are often used (both by psychologists and by laymen) seem to presuppose that we have a single processing limitation and/or a single input system that governs all visual input-regardless of what use the input may be put to. (Otherwise, the concept of visual attention would be too crude to be of much use.) Oddly, though, this presupposition has seldom been tested, and there would seem to be no firm basis for it. The two experiments reported here represent a preliminary examination of this issue; they provide a look at what happens when people try to carry out two different visual tasks simultaneously and both tasks involve the same brief array (although not necessarily the same elements within the array). Studies of visual divided attention (involving many stimuli and just one task) confirm the common sense idea that visual processing is subject to capacity limits. Several fairly easy discriminations can often be performed in parallel without any apparent capacity limits (see, e.g., Shiffrin & Gardner, 1972; Treisman & Gelade, 1980). However, when several different difficult visual discriminations must be performed, limits on capacity become evident (see, e.g., Duncan, 1987; Kleiss & Lane, 1986). Detecting a target in a search task also tends to interfere with detecting another target (see, e.g., Duncan, 1980). Interestingly, though, judging or detecting two different features of the same object (e.g., the color and orientation of aline) can often be accomplished without evident capacity limits (Allport, 1971; Duncan, 1984). As mentioned above, in very few divided attention studies have subjects been required to perform two completely unrelated visual tasks simultaneously. A study performed by Hoffman, Nelson, and Houck (1983) was among the few in which this was done. Their subjects searched
115
Copyright 1994 Psychonomic Society, Inc.
116
PASHLER
successive frames of letters for a digit target and determined which direction a figure shaped like a letter "C" pointed. There was substantial interference between these two tasks, and the interference was greater when the two figures were spatially separated. The present studies involved two logically independent tasks performed on a single brief display, each involving a different (separable) dimension. Task 1 required subjects to classify and respond immediately to the color of some letters; in Task 2, they were to retain in short-term memory the form of letters from the same array. One could envision various ways in which performance in the two tasks might be related. First, processing might proceed completely independently. Second, getting information into short-term memory (Task 2) might draw heavily on the same processing resources that are needed for perceptual or cognitive aspects of the color classification task (Task 1). This possibility would fit the suggestions of Kahneman (1973) and Broadbent (1982), who hypothesized extremely general mental processing limitations. A third possibility (which would not exclude the second) is that the two tasks might be constrained to use information from the same locations (or objects) but that the tasks could otherwise proceed independently. More specifically, people might be able to classify the letters in Task 1 while they store letters for Task 2, with no mutual interference between these processes, but only as long as the letters they classify in Task 1 are the very same letters as the ones they classify in Task 2. This will be referred to as yoking of the two tasks at the input stage. Other possibilities could also be considered. In the experiments reported below, subjects' short-term memory for the letter forms was assessed with the use of a matching task. The matching task required them to decide whether or not a second test display of letters that was presented slightly later matched the original target display. Pashler (1988) found that subjects make many errors in this task even when the interval between the displays is only 67 msec. Subjects could not hold on to more than about four or five characters, roughly the "span of apprehension" found in whole-report tasks (Sperling, 1960). This is surprising in some ways, since one might have supposed that momentary awareness of what is in a visual scene would include the ability to detect changes if the display flashes off and on again rapidly. In any case, performance was almost as good with reflected-letter forms as with letters, suggesting that people accomplish the task by storing information in a visual short-term memory (cf. Frick, 1984; Phillips, 1974, 1983), rather than by storing more abstract information.
TARGET
TEST
DISPLAY
DISPLAY QTV NXE
Time
~.
r ms o o Reaction
( RESPOND "DIFFERENT" AT LEISURE)
Time
Figure 1. The basic sequence of events in the two experiments. Sometimes the subjects respond to the color of elements in the target display (see text); in all cases, they try to remember elements in the target display and determine whether the test display has changed or not. Responses to the target display are speeded; the same/different response to the test display is unspeeded, and the test display remains present until this response.
250-msec blank interval and then the target display. The target display consisted of six letters (selected randomly without replacement from a set of 20 letters of relatively low confusability) presented against the black background for 200 msec. (Each character measured .6 0 x 1.1 0 of visual angle, based on a typical viewing distance of 60 cm. The dimensions of the whole display were 4.0 0 [width] x 3.4 0 [height].) There were 12 blocks of 42 trials in the experiment, with a rest period between blocks. Twelve subjects (mostly University of California, San Diego, undergraduates) participated in Experiment 1, which was run on ffiM PCs with NEC Multisync 2.2 displays. The first task required a response to the target display. On singletask blocks, the letters in the target display were gray (of approximately the same contrast as the colored letters to be described next); the subject made no response to them. On one-item-colored blocks, a single, randomly selected letter in the target display was either red or green (color selected randomly with replacement). The subject responded as quickly as possible by pressing the "Zoo key if it was red and the "X" key if it was green, using the index or the middle finger of the left hand. On all-items-colored blocks, all six letters were red or all six were green; the subject responded as in the one-item-colored blocks. There were 4 blocks of each type, with block type counterbalanced over the sequence of 12 blocks, and random ordering of trials within a block. (The luminances of the red, green, and gray were roughly comparable.) Two hundred milliseconds after the target display appeared, it disappeared; after another 800 msec, a test display of six letters appeared in the position formerly occupied by the target display. (On a typical trial, this occurred several hundred milliseconds after the response to the target display had been made.) On half the trials, this test display consisted of the same six letters as did the target display. On the other half of the trials, one of the six letters was randomly switched to another letter from the stimulus set. The test display did not disappear until the subject had made an unspeeded same/different judgment, responding with the right hand, and the next trial began after a further 250-msec delay. (The color of the item in any given position in the test display was always the same as the color of the item in the corresponding position of the target display, whatever the condition; thus, color was completely irrelevant in the same/different task.)
Results and Discussion EXPERIMENT 1
The subjects' responses to the color of the target displays were rapid, but, not surprisingly, it took a little longer to respond when only a single item in the target display was colored (499 msec) rather than all of them Method Figure 1 shows the method used in the present experiments. Subjects (445 msec). An analysis of variance with two factorsfirst saw a target display, and in some conditions they had to respond color condition and same/different-showed that color very rapidly to the color of some letter or letters in this display. On each trial, the target display was preceded by a plus sign (fixation point) condition made a significant difference [F( 1,11) = 36.7, that was exposed in the center of the screen for 1 sec, followed by a p < .001]. Response times on same and different trials
Storing and Discriminating
VISUAL DIVIDED ATTENTION did not differ. There were more errors for color responses in the one-item-eolored(9.6%) than in the all-items-eolored condition (6.4%) [F(I,l1) = 40.7, p < .001], but no other significant effects in error rates. The question of primary interest was how well subjects could store and retain the form of the letters in visual short-term memory, as indicated by their performance on the same/different judgment. Differences among the conditions on this task were fairly modest: overall error rates were 26.4 %, 35.1 %, and 32.0% in the single-task, oneitem-eolored, and all-items-colored conditions [F(2,22) = 16.9, p < .001]. The subjects failed to find a difference more often (45.2%) than they falsely reported one (17.1%) [F(l,ll) = 26.5,p < .001], and there was an interaction between same/different and condition [F(2,22) = 6.1, p < .01], since performing the color tasks increased error rates on different displays more than on same displays. The difference in same/different task performance between single-task and all-items-eoloredconditions was significant [F(1, 11) = 13.0, p < .01] but relatively modest (about 5%). Earlier findings have indicated that response selection and short-term memory storage can be functionally separate (see Pashler, 1993). The dual-task cost observed here could well be due to preparing, rather than carrying out, the concurrent task. The modest dual-task decrement here suggests that the tasks do not generate much mutual interference. However, a further analysis revealed that the tasks were by no means independent of each other. In the one-colored-item condition, when a single letter was changed, it might be the letter that was colored, or it might be one of the other five letters. Although switches were no more likely to occur in the colored position, subjects detected 56.3 % of these; they detected only 29.4% of switches of the uncolored letters [F(l,l1) = 34.0, p < .001]. Thus, the input of visual information from the different objects for the two tasks may be so constrained that the object providing color information is much more likely to be stored for the memory task than are the other objects. A strong interpretation of this result is that the color task required visual attention to be allocated to the colored item(s), and that visual attention is both necessary and sufficient to ensure that items are stored in short-term memory (subject to data and time limitations). If this is the case, the linkage between which items are targets for color classification and which items are stored may be obligatory, and not merely a matter of optional strategy. If so, the difference in the likelihood of storing items bearing color information and items not bearing color information should persist even if subjects are given an incentive to classify some objects and store information from different objects. One might ask why people do as well as they do in storing information in the one-item-colored condition if the color task forces them to attend to just one item. Subjects plainly store more than just the single colored letter-in fact, they retain almost as many total items as in the singletask condition. This could be reconciled with the obliga-
117
tory linkage hypothesis, however, if one supposes that subjects have sufficient time to shift their attention from the colored item to the remaining items in these (unmasked) displays. On this view, same/different performance would be limited primarily by the storage capacity of short-term memory (as in a whole-report task, which is scarcely affected by exposure duration; Sperling, 1960).
EXPERIMENT 2 Store Some Items While Classifying Others If the linkage between selection of items for one task and for the other is truly obligatory, it ought to be extremely difficult for people to classify some objects from an array while storing others if the time for which a display remains available is abbreviated, as by a mask. This idea was tested in Experiment 2.
Method Experiment 2 differed from Experiment I only as follows: In each target display, there were always three colored letters (all red or all green) and three gray letters (in Positions 2,4, and 6, numbered along the upper, then lower, row), and the subject always responded immediately to the color. In half the blocks of trials, when a letter in the target display was switched in the test display, it was always one of the colored letters (colored-switch condition). In the other half of the blocks, a switched letter was always one of the gray letters (gray-switch condition). The subjects were told about each block type, so in the gray-switch condition they understood (through a series of consecutive blocks) that they needed to remember the form information only about the gray items, not the colored items. In the masked version of the experiment, a mask (consisting of gray sharp signs in the six letter positions) was present throughout the 800 msec between the offset of the target display and the onset of the test display, whereas in the unmasked version of the experiment, there was no such mask. The first seven blocks were in one condition (colored-switch or gray-switch), and the last seven blocks were in the other condition; order was counterbalanced, and the first block of each condition was considered practice.
Results and Discussion Sixteen subjects participated in the masked version. Mean response times for the color classification responses were 411 msec for the colored-switch condition, and 421 msec for the gray-switch condition (not a significant difference). Errors on the color task did not differ by condition. In the same/different judgment task, error rates were as follows: For the colored-switch condition, errors were 19.7 % and 22. 1% for same and different trials, respectively. For the gray-switch condition, errors were 42.3% and 44.9% for same and different trials, respectively. The difference between color-switch and gray conditions was significant [F(1,15) = 117, p < .001]; differences between same and different trials were not. Twelve subjects participated in the unmasked version. For the colored-switch condition, same/different errors were 11.8% and 17.5% for same and different trials, respectively; for the gray-switch condition, errors were 12.6% and 35.1 % for same and different trials, respectively. Thus, the results were similar to the masked version, but the effect of switch condition on accuracy was much smaller.
118
PASHLER GENERAL DISCUSSION
Several conclusions can be drawn. First, people do not have much trouble in responding to the color of some letters and storing the forms of these items in short-term memory. Memory storage is not quite as good as it is for a single-task control, but the interference is still quite modest. What causes people much greater difficulty, however, is the storing of some items in short-term memory while they are responding to the color of different items in the same display. These results are consistent with the view that visual attention is allocated to objects or their locations (the present data do not distinguish among these possibilities), and that this results in information from these locations being made available for multiple purposes-"awareness," memory, or voluntary responses in any task. Although many investigators have assumed a single input-selection process governing the intake of information regardless of the purpose for which the information is used "downstream," evidence for this has been sparse. The results are also consistent with Duncan's (1985) suggestion that any access to even the simplest visual attributes requires allocation of visual attention (see also Kahneman & Treisman, 1984). Logically speaking, the color task could have been performed without subjects' selectively orienting to the locations of colored objects; indeed, Treisman and Gelade (1980) have suggested that the presence of simple features can be detected without even registering their locations (but see Johnston & Pashler, 1990). The results fit nicely with recent findings of Tsal and Lavie (1993). They had subjects report an array of letters, but only on those trials on which a single letter that preceded the display had certain specified properties. In general, people were much more likely to report an item in the array that occupied the same position as had the single letter that preceded it, even when the position of that letter was irrelevant for the subjects' task. Tsal and Lavie's findings show that people tend to pick up information in a display from the same position as that at which they have just picked up color or identity information in a preceding display; the results reported here show that people have great trouble using color information from some items and storing form information from other items in the very same display, even when it would serve their purposes to do so. Finally, the results would appear to suggest that storage in visual shortterm memory may be an operation that occurs as a "contingently automatic" consequence of having visual attention allocated to a given location. Yet while the results suggest this, they do not prove it beyond a doubt, because the subjects did not have any incentive not to store the forms, even in the colored-switch condition. This interesting issue could bear further examination, because Phillips and Christie (1977)-on the basis of a very different kind of evidence-inferred that subjects could attend to a new pattern without writing over information just stored in visual short-term memory. And of course the present investigation has only scratched the surface of the broader issue mentioned at the beginning of this article-namely, what happens when people must use visual information in the same display for quite different purposes. Obviously, there are many other kinds of tasks-including ones of practical importance-for which such questions could be addressed. REFERENCES ALLPORT, D. A. (1971). Parallel encoding within and between elementary stimulus dimensions. Perception & Psychophysics, 10, 104-108. BROADBENT, D. E. (1982). Task combination and the selective intake of information. Acta Psychologica, 50, 253-290. DUNCAN, J. (1980). The locus of interference in the perception ofsimultaneous stimuli. Psychological Review, 87, 272-300.
DUNCAN, J. (1984). Selective attention and the organization of visual information. Journal of Experimental Psychology: General, 113, 501-517. DUNCAN, J. (1985). Visual search and visual attention. In M. Posner and O. S. M. Marin (Eds.), Attention and performance XI (pp. 85105). Hillsdale, NJ: Erlbaum. DUNCAN, J. (1987). Attention and reading: Wholes and parts in shape recognition-A tutorial review. In M. Coltheart (Ed.), Attention and performance XII: The psychology of reading (pp. 39-61). Hillsdale, NJ: Erlbaum. ERIKSEN, C. W., & SPENCER, T. (1969). Rate of information processing in visual perception: Some results and methodological considerations. Journal ofExperimental Psychology Monographs, 79(Pt. 2), 1-16. ESTES, W. K., & TAYLOR, H. A. (1964). A detection method and probabilistic models for assessing information processing from brief visual displays. Proceedings of the National Academy of Sciences, 52, 446-454. FRICK, R. W. (1984). Using both an auditory and a visual short-term store to increase digit span. Memory & Cognition, 12, 507-514. HOFFMAN, J. E., NELSON, B., & HOUCK, M. R. (1983). The role of attentional resources in automatic detection. Cognitive Psychology, 51, 379-410. JOHNSTON, J. C., & PASHLER, H. (1990). Close binding of identity and location in visual feature perception. Journal of Experimental Psychology: Human Perception & Performance, 16, 843-846. KAHNEMAN, D. (1973). Attention and effort. Cambridge, MA: Harvard University Press. KAHNEMAN, D., & TREISMAN, A. (1984). Changing views of attention and automaticity. In R. Parasurarnan & D. R. Davies (Eds.), Varieties of attention (pp. 29-62). New York: Academic Press. KLEISS, J. A., & LANE, D. M. (1986). Locus and persistence of capacity limitations in visual information processing. Journal ofExperimental Psychology: Human Perception & Performance, 12, 200-210. PASHLER, H. (1988). Familiarity and visual change detection. Perception & Psychophysics, 44, 369-378. PASHLER, H. (1993). Dual-task interference and elementary mental mechanisms. In D. Meyer & S. Kornblum (Eds.), Attention and performance XIV (pp. 245-264). Cambridge, MA: MIT Press. PHILLIPS, W. A. (1974). On the distinction between sensory storage and short-term visual memory. Perception & Psychophysics, 16, 283-290. PHILLIPS, W. A. (1983). Short-term visual memory. Philosophical Transactions ofthe Royal Society ofLondon: Series B, 302, 295-309. PHILLIPS, W. A., & CHRISTIE, D. F. M. (1977). Interference with visualization. Quarterly Journal of Experimental Psychology, 29, 637-650. SHIFFRlN, R. M., & GARDNER, G. T. (1972). Visual processing capacity and attentional control. Journal ofExperimental Psychology, 93, 78-82. SPERLING, G. (1960). The information available in brief visual presentations. Psychological Monographs: General & Applied, 74 (Whole No. 498), 1-29. TREISMAN, A., & GELADE, G. (1980). A feature integration theory of attention. Cognitive Psychology, 12, 97-136. TSAL, Y., & LAVIE, N. (1993). Location dominance in attending to color and shape. Journal of Experimental Psychology: Human Perception & Performance, 19, 131-139. (Manuscript received March 13, 1993; revision accepted for publication August 30, 1993.)
