Losing the trees for the forest in dynamic visual search.

HHS Public Access Author manuscript Author Manuscript

J Exp Psychol Hum Percept Perform. Author manuscript; available in PMC 2017 May 01. Published in final edited form as:

J Exp Psychol Hum Percept Perform. 2016 May ; 42(5): 617–630. doi:10.1037/xhp0000186.

Losing the trees for the forest in dynamic visual search Nicole L. Jardine and Cathleen M. Moore Department of Psychological and Brain Sciences, University of Iowa, Iowa City, United States

Abstract Author Manuscript Author Manuscript

Representing temporally continuous objects across change (e.g., in position) requires integration of newly sampled visual information with existing object representations. We asked what consequences representational updating has for visual search. In this dynamic visual search task, bars rotated around their central axis. Observers searched for a single episodic target state (oblique bar among vertical and horizontal bars). Search was efficient when the target display was presented as an isolated static display. Performance declined to near chance, however, when the same display was a single state of a dynamically changing scene (Experiment 1), as though temporal selection of the target display from the stream of stimulation failed entirely (Experiment 3). The deficit is due neither to masking (Experiment 2), nor to a lack of temporal marker for the target display (Experiment 4). The deficit was partially reduced by visually marking the target display with unique feature information (Experiment 5). We suggest that representational updating causes a loss of access to instantaneous state information in search. Similar to spatially crowded displays that are perceived as textures (Parkes, Lund, Angelucci, & Solomon, 2001), we propose a temporal version of the trees (instantaneous orientation information) being lost for the forest (rotating bars).

Keywords visual search; updating; context; dynamic scenes; temporal selection

Author Manuscript

Vision enables animals to search the environment for significant objects and events. Decades of research using simple, static displays like that shown in Figure 1 has examined how basic visual features, such as orientation, size, and color, are encoded, represented, and used to guide search processes. Target objects that are defined by simple visual features tend to “pop out” and can sometimes be represented and used for search in an unlimited capacity manner (Foster & Ward, 1991; Julesz & Bergen, 1983; Palmer, 1994; Scharff, Palmer, & Moore, 2011b; Treisman & Gelade, 1980; Treisman & Gormican, 1988). Other targets, however, yield less efficient search reflecting clear processing capacity limitations. Examples include object representations of conjunctions of features (Eckstein, Thomas, Palmer, & Shimozaki, 2000; Pashler, 1987; Treisman & Gelade, 1980; Zhaoping & Guyader, 2007), complex shapes (Cunningham & Wolfe, 2014; Scharff, Palmer, & Moore, 2011a; Yang & Zelinsky, 2009), and objects within complex scenes (Asher, Tolhurst,

Corresponding Author: Nicole L. Jardine, Department of Psychological and Brain Sciences, 308 Iowa Ave., University of Iowa, Iowa City, IA 52240, [email protected], (319) 335-2427.

Jardine and Moore

Page 2

Author Manuscript

Troscianko, & Gilchrist, 2013; Castelhano & Heaven, 2010; Theeuwes & Kooi, 1994; Torralba, Oliva, Castelhano, & Henderson, 2006). Although questions remain regarding the specific nature of these capacity limitations (Eckstein, 2011; Palmer, 1995; Wolfe, 2003), it is established that there is a distinction between the processing necessary for simple-feature search and that for complex-object search. These and other findings concerning search for targets with cues to 3-dimensional structure and other perceptually derived information (Enns & Rensink, 1990; Kleffner & Ramachandran, 1992; Rensink & Cavanagh, 2004; Rensink & Enns, 1998) indicate that search processes at least sometimes act on representations that have received some degree of high-level processing (i.e., beyond basic feature extraction). Research has focused on identifying what specific higher-level processing occurs prior to the engagement of search processes (Aks & Enns, 1996; Moore & Brown, 2001).

Author Manuscript Author Manuscript

There exists, however, another source of complexity in visual search that has received less attention than structural complexity: scene dynamics. Visual processing typically occurs in a world within which objects move and change over time but are perceived as temporally continuous entities. We may, for example, search for a street sign while driving, a child on a busy playground, or a safe path in a game of Pac-Man. Scenes like these could be studied as a series of snapshot-like episodic representations within which search processes unfold (e.g., Neider & Zelinsky, 2011), but how observers integrate and update visual representations of the world over time – thereby forming representations of temporally continuous objects within which search occurs – requires a different approach. One example of a popular task using dynamic displays is the multiple object-tracking task (Pylyshyn & Storm, 1988). Many experiments using this task have led to the conclusion that when objects move, observers use moment-to-moment samples of locations (and, to a lesser extent, features) to update a limited number of representations of target objects (e.g., Flombaum, Scholl, & Santos, 2009; Jardine & Seiffert, 2011). This process, referred to as object-based updating, allows observers to perceive a coherent and up-to-date world of consistent objects. Updating mechanisms can, however, also result in the loss of an object’s previous features, such as would occur through an overwriting process (Enns, Lleras, & Moore, 2009; Lleras & Moore, 2003; Moore & Lleras, 2005; Moore, Mordkoff, & Enns, 2007; see also Enns & Di Lollo, 2000).

Author Manuscript

Putting it all together, visual search studies using static displays have shown that search for a target defined by a single salient feature can be highly efficient, perhaps unfolding independently across items in the display without regard to the structure of the scene as a whole. Other studies, also using static displays in which targets were complex objects, indicate that visual search often occurs within object-based representations. In this study we asked whether object-based updating processes, revealed in the context of dynamic nonsearch tasks, have specific consequences in visual search. Our method was to explore the effects of dynamics on a simple visual search task that would yield highly efficient search performance in a non-dynamic task. Note that dynamics have been considered in visual search to ask, for example, whether search is efficient for a target that is defined on the basis of various types of motion (Horowitz & Treisman, 1994; McLeod, Driver, & Crisp, 1988) and whether motion can

J Exp Psychol Hum Percept Perform. Author manuscript; available in PMC 2017 May 01.

Jardine and Moore

Page 3

Author Manuscript

capture attention (Franconeri & Simons, 2003; Hillstrom & Yantis, 1994; Mühlenen, Müller, & Müller, 2008). These questions are different from that addressed here. Here we ask what impact scene dynamics has on what, under static conditions, would be an efficient “pop out” search. Previous work has shown that scene-irrelevant motion (e.g., translation of an entire search array across the screen) has little impact on search performance (Alvarez, Konkle, & Oliva, 2007; Hulleman, 2009). Introducing target-irrelevant motion separately to individual items within displays, however, does impair search, with the greatest impairment occurring when there was uncertainty regarding the specific features that would define the target (Kunar & Watson, 2011). Here, we imbedded a highly efficient search task (search for an oriented bar among vertical and horizontal bars) and asked what consequences adding simple dynamics to the displays, as might occur if they were objects out in the world, has on search performance.


We started with a standard efficient orientation visual-search task (search for an oblique bar among vertical and horizontal bars, see Figure 1) and added minimal dynamic context to it. Figure 2 illustrates the basic strategy for adding dynamic context. The critical frame (surrounded by black brackets in Figure 2) was a standard simple orientation-search display. Observers searched for the oblique bar (±45°) among horizontal (0°) and vertical (90°) bars, and reported its orientation (left or right). When the critical frame is displayed without temporal context (i.e., as a static display), the task is identical to that illustrated in Figure 1, and can be expected to yield highly efficient search performance (e.g., Wolfe’s 1992, Experiment 6; Treisman & Gelade’s 1988, Experiment 5). When the same image is presented as one frame of a 3-frame movie (blue brackets), the scene appears as a set of bars that rotate (see Movie 1). Notice that objects in these dynamic scenes do not change locations and the rotational motion in the scene is task-irrelevant. We started with a basic question: can information in a pop-out display be selected when it comprises one episodic moment of a coherent, dynamic scene? Or does the task-relevant pop-out display become inaccessible in search?

Overview of findings

Author Manuscript

The dynamic context illustrated in Figure 2 impaired search severely (Experiment 1). Performance dropped from near ceiling in the static condition to near chance in the dynamic condition. This suggests that individual states of the objects were inaccessible to search mechanisms, perhaps due to low-level mechanisms like masking, or to high-level updating processes. The remaining experiments tested different explanations for this impairment. The impairment from dynamics persisted even when each frame was displayed for 800 ms (Experiment 2), thus ruling out masking and the attentional blink (Raymond, Shapiro, & Arnell, 1992) as potential explanations. The impairment was equivalent in magnitude to that caused by presenting all of the stimuli from a dynamic display in a single static display, resulting in a highly heterogeneous, large-set size, difficult search (Experiment 3). This suggests that temporal selection of the critical frame from within its temporal context fails completely, despite the fact that the perceptual experience of the two situations—a relatively small number of rotating bars (dynamic) versus a large number of heterogeneously oriented bars (static)—is extremely different (compare Movies 1 and 4). Making the critical frame more distinct at a featural level (e.g., presenting its objects in a different color or length than J Exp Psychol Hum Percept Perform. Author manuscript; available in PMC 2017 May 01.

Jardine and Moore

Page 4

Author Manuscript

the surrounding frames) reduced the deficit, but none of the manipulations eliminated (Experiment 5). We conclude that temporal selection of a pop-out orientation target is severely limited in situations where dynamic stimuli are represented as objects changing orientation over time. Conscious access to specific episodic information may be lost in favor of up-to-date representations of the scene as functional objects.

EXPERIMENT 1: The Basic Contrast of Dynamic vs. Static Search Methods Participants—Participants were 12 University of Iowa undergraduate students (18–20 years old) who were naïve to the purpose of the experiment. All reported normal or corrected-to-normal visual acuity and color vision, gave informed consent, and received course credit for participating.

Author Manuscript

Apparatus—Experiments were developed in MATLAB (R2010a, 32-bit) using the Psychophysics Toolbox (Version 3.0.8 beta; Brainard, 1997; Pelli, 1997); and were run on an Apple Mac Pro (OSX 10.6.7) with an NVIDIA Quadro FX 4500 graphics card driving a 17inch CRT monitor set to 1024 × 768 resolution at a refresh rate of 100 Hz. Participants completed the study in individual, sound attenuated, partially darkened rooms. Viewing distance was fixed at 60 cm using a chinrest. A sheet of paper showing images of the targets and the response for each was visible throughout the experiment.

Author Manuscript

Stimuli—Stimuli were black (0.5 cd/m2) bars of 0.2° × 1.0° of visual angle (VA) centred within a randomly chosen cell of an imaginary centred 6 × 6 grid (each cell 3.1° × 3.1° VA) of potential positions on a grey (37 cd/m2) background. The centre-to-centre distance between cells of the grid was 2.3° VA both vertically and horizontally. A centred 0.4° × 0.4° VA black fixation cross indicated the beginning of each trial. The critical frame consisted of one oblique target bar (±45°) and a variable number of horizontal (0°) and vertical (90°) distractor bars. Static displays consisted of the critical frame only. Dynamic displays consisted of a sequence of 3 or 11 frames in which the bars changed orientation 3 or 11 times in 12.5° steps of orientation such that they rotated through the positions of the critical frame, with an equal number of frames preceding and following the critical frame. The onset of the critical frame was visually ‘marked’ by a centred doubleline black rectangular border1 surrounding the search array. The border’s lines were 0.2° VA thick, with innermost edges 10.25° VA away from the middle of the search grid and at least 3° VA from target and distractor bars.

Author Manuscript

Task—Participants searched for an oblique (±45°) bar (target) among horizontal (0°) or vertical (90°) bars (distractors). There was a target on every trial, and the task was to report its orientation (left/right) by pressing the F/J key with the index finger of their left/right hand, respectively. Participants were asked to take as much time as they needed and to strive for accuracy rather than speed. They were given feedback on each incorrect trial.

1Another version of this experiment without the border yielded nearly identical results.


Jardine and Moore

Page 5

Author Manuscript

Procedure—Following informed consent, participants were shown printed instructions and images that described both static and dynamic displays in the task. Researchers guided participants through slowed demonstrations of the static and dynamic conditions, emphasizing that observers were to ignore the direction in which the target was rotating and to respond only to the target’s orientation at the critical frame. Participants completed three blocks of practice trials. The first included only static conditions. The second included both static and dynamic conditions, with frame durations slowed by a factor of two (e.g., 200 ms became 400 ms). The third block of practice was identical to those of the experiment.

Author Manuscript

Figure 3 illustrates a typical trial sequence. Trials began with a 1000 ms fixation cross, followed by a 200-ms blank display, followed by the onset of the search display. Search displays consisted of 1, 3, or 11 frames of a “movie.” Single-frame static displays consisted of just the critical frame. Multi-frame displays were movies in which each bar rotated around its midpoint clockwise or counter-clockwise throughout the trial, with the critical frame always occurring in the middle of the movie sequence. Each bar’s direction of rotation was selected randomly at the beginning of the trial. Each frame duration was 200 ms with an inter-stimulus interval (ISI) of 0 ms. Trial displays were thus 200, 600, or 2200 ms long for the 1-, 3- and 11-frame displays respectively (see Movies 1 and 2 for the 3- and 11-frame dynamic displays). Trial feedback following an incorrect response consisted of a 150 ms 400 Hz tone. The next trial began 1000 ms after a correct response or after feedback. Design—A fully factorial 3 Display (Static: 1 vs. Dynamic: 3 vs. Dynamic: 11 frames) × 2 Set size (16 vs. 32), within-subjects design was used. These 6 conditions were mixed within experimental blocks of trials. Data were collected from 5 blocks of 72 trials each, yielding a total of 60 observations per condition.

Author Manuscript

Measure & Analysis—The dependent measure was accuracy in percent of correct responses (50% is chance). Subjects’ arc-sin transformed mean proportions correct were analysed with within-subjects ANOVAs with α = .05 (Greenhouse-Geisser corrected when necessary). Effect sizes are reported with partial eta-squared ηp2 as output by SPSS, and effect sizes for t-tests are reported with standardized effect size Cohen’s d of untransformed percent correct, using Morris and DeShon’s (2002) correction for correlations between within-subjects means. Results—Data were excluded from one participant whose mean percent correct across all conditions was below 55%. Figure 4 shows mean percent correct as a function of set size and number of frames. Accuracy was high in the static condition (1 frame), and unaffected by set size. In contrast, accuracy was near chance in dynamic conditions.

Author Manuscript

A 2 (Set size) × 3 (Display) repeated-measures ANOVA confirmed that Set size had no reliable effect on performance, F(1,10) < 1, and did not reliably interact with Display, F(2,20) = 2.02, p = .159, ηp2 = .168. Display, however, had a significant effect, F(1.27,12.68) = 123.03, p < .001, ηp2 = .925. Sidak-corrected pairwise comparisons revealed no significant difference in accuracy between the 3- and 11-frame dynamic displays, p = .91. Search was highly accurate for static displays but impaired for both the 3- and 11-frame dynamic displays.


Jardine and Moore

Page 6

Author Manuscript

It is possible that the target in the critical arrays of the dynamic scenes in Experiment 1 actually did “pop out”, such that observers detected the presence of the target but failed to extract the additional information needed to identify its orientation. To assess this we ran a variation of the experiment in which only half of trials contained a left- or right-tilted target, and new observers (N=12) reported the presence or absence of a target in the display. Results are shown in Figure 5 and exhibit the same pattern as in the main Experiment 1: Display significantly affected performance, F(2,22) = 93.47, p < .001, ηp2 = .895. There was no main effect of Set Size nor did Set Size interact with Display (both Fs < 1). Target presence or absence did not have a significant main effect on accuracy, F(1,11) = 2.06, p = . 18, ηp2 = 158. A significant 3-way interaction between Display, Set Size, and Target Presence, F(2,22) = 11.37, p < .001, ηp2 = .508, was driven by observers’ increased tendency at greater set sizes to miss targets in dynamic displays.

Author Manuscript

Discussion Despite the fact that the critical frame in which the simple orientation-search display appeared was available for the same duration (200 ms) in static and dynamic conditions, performance fell to near chance when the critical frame appeared within dynamic context. This was true with even a single frame flanking the critical frame (Dynamic: 3 frame), and even though the critical frame was marked with a border that cued the critical frame1. Moreover, simplifying the report task from identification to detection did not alleviate the deficit induced by dynamic context. What “popped out” in a static display failed to “pop out” when the display was one of multiple episodic states of a changing scene.

Author Manuscript

There are multiple possible explanations for the poor performance in the dynamic conditions in Experiment 1. In the following experiments we sought to isolate what specific aspect of processing was made difficult by the dynamic context. We begin, in Experiment 2, by testing whether the poor performance in the dynamic conditions can be accounted for by basic masking effects.

EXPERIMENT 2: Dynamic deficit at extended frame durations

Author Manuscript

The dynamic search conditions in Experiment 1 are ones under which masking can occur. Specifically preceding or following targets with other stimuli (masks) can severely impair target visibility (Breitmeyer, 1984). Masking occurs in many different forms and is probably due to multiple mechanisms (for reviews see Breitmeyer & Ogmen, 2000; Breitmeyer & Ogmen, 2006; Enns & Di Lollo, 2001) depending on the details of the target, the mask, and their relative timing. Specifically, the spatiotemporal parameters of our dynamic displays are consistent with the conditions that induce para-contrast and meta-contrast masking, in which the target contours are made less visible by nearby, but not overlapping, contours of the mask or masks (e.g., Francis, 2002; Ogmen, Breitmeyer, & Melvin, 2003). In order to assess whether the deficit observed in dynamic search displays can be accounted for by stimulus-level masking, we manipulated frame durations within a range of 100 to 800 ms. This range extends well beyond that at which basic masking would contribute significantly to performance.


Jardine and Moore

Page 7

Author Manuscript

Manipulating frame duration also allowed us to assess another possible explanation of the dynamic deficit. The structure of dynamic search scenes shares characteristics with displays in which a deficit known as the attentional blink occurs (e.g., Raymond et al., 1992; for a review see Shih, 2008). When stimuli are presented in rapid serial temporal streams at the same location, and observers must report two targets within the stream, a severe impairment in detecting the second target can occur. Although there is only one target to be detected in the present task, it is possible that the need to selectively process information in a temporal stream of stimuli elicited similar processing limitations that give rise to the attentional blink (see also Anderson, 2014). Manipulating the time course allowed us to broadly examine the role of masking and attentional blink-like processes in dynamic search displays. Methods

Author Manuscript

Participants—Thirteen new participants (aged 18–19), all who reported normal or corrected-to-normal visual acuity and color vision, completed this experiment. Design—A 2 Display (Static: 1 vs. Dynamic: 5 frames) × 2 Set size (16 vs. 32) × 6 Frame duration (100 vs. 200 vs. 300 vs. 400 vs. 500 vs. 800 ms) design was used. All 24 conditions were mixed within each block. Data were collected from 4 blocks of 144 trials, yielding a total of 24 observations per condition. The time between the offset of fixation and the start of the trial display was equal to that trial’s frame duration. See Movie 3 for a demonstration of the 500 ms frame duration.

Author Manuscript

Results—Mean percent correct is shown as a function of Display and Frame duration in Figure 6. Data are collapsed across Set Size for clarity. Accuracy was high in all static displays. In dynamic displays, accuracy was near chance for short durations and increased steadily with frame duration. Importantly, at durations of 400 and 500 ms, accuracy remained 20% worse in the dynamic condition than the static condition. Even at 800 ms, accuracy remained impaired by 10% worse from dynamics. These durations are well beyond the range at which that amount of masking would occur and beyond the range at which a performance impairment could be attributed to the attentional blink.

Author Manuscript

A repeated-measures ANOVA confirmed reliable main effects of both Display, F(1,12) = 169.42, p < .001, ηp2 = .934, and Frame duration, F(5,60) = 19.62, p < .001, ηp2 = .620 as well as a reliable interaction between them, F(5,60) = 9.07, p < .001, ηp2 = .431. Set Size had no reliable main effect or interaction (all Fs < 2, all ps > .2, all ηp2 < .07). Specific comparisons confirmed that performance in the dynamic condition remained worse than that in the static condition even at the frame durations of 400 ms, t(12) = 7.04, p < .001 d = 2.26, 500 ms, t(12) = 6.88, p < .001, d = 1.81, and 800 ms, t(12) = 3.50, p = .004, d = 1.14. It is striking that even when each frame in a dynamic display was displayed for 800 ms, accuracy in dynamic displays was on average 10 percentage points worse than in static search2. To establish that there exists some frame duration that yields performance as high

2Two observers frequently (∼20% of trials) made responses before the onset of the critical frame in 5-frame trials. Excluding all data from these observers did not change the significances of any of the inferential tests. Critically, there remained evidence for a difference in performance between static and dynamic trials with 800 ms frame durations, t(10) = 2.53, p = .03, d = 1.14.


Jardine and Moore

Page 8

Author Manuscript

as that in the static condition, we tested 8 new participants with just two frame durations (Experiment 2B: 200 and 2500 ms). Data are shown as separate points in Figure 6. At a frame duration of 2500 ms, performance was near ceiling for all conditions and there was no reliable dynamic search impairment, F(1,7) = 5.32, p = .54, ηp2 = .43. Discussion

Author Manuscript

Experiment 2 delineated the temporal boundary conditions within which the performance deficit for dynamic search displays occurs. A performance impairment of 20% in dynamic conditions relative to static conditions persisted when the critical frame and surrounding context frames were displayed for 500 ms and an impairment of 10% persisted at 800 ms. Notice that these durations are longer than mean response time in many visual search experiments using static displays and speeded responses for this kind of orientation task (e.g., 393 ms in Treisman & Sato, 1990, Experiment 2; and approximately 450 ms in Wolfe, Friedman-Hill, Stewart, & O’Connell, 1992). Whatever causes the severe impairment under dynamic conditions cannot be attributed entirely to stimulus-level masking effects. The temporal unfolding of stimulus events is outside of the range in which masking occurs. Likewise, the deficit cannot be attributed to whatever impairment in processing gives rise to the attentional blink. We do not suggest that there are no masking or blink-like effects in the dynamic conditions – rather, we conclude that something beyond any present masking and attentional-blink effects is impairing performance in these dynamic conditions.

EXPERIMENT 3: Spatiotemporal Heterogeneity Author Manuscript

Here we considered the possibility that the impairment in dynamic search is caused by a failure of temporal selection. This would prevent the isolation of the target-containing (critical) frame from temporally flanking frames. As a consequence, search is performed not only on the stimuli presented in the homogenous critical frame, but on the stimuli presented in all frames. Under this view, dynamic displays have both a larger effective set size and increased stimulus heterogeneity. Search is less efficient with greater stimulus heterogeneity (Duncan & Humphreys, 1989; Rosenholtz, Li, & Nakano, 2007; Treisman & Gelade, 1980). Thus, one account of the dynamic-search deficit is that observers were unable to isolate the content of one frame from another over the course of an unfolding movie, and the resulting increased effective set size and heterogeneity yielded poor search performance.

Author Manuscript

In Experiment 3, we created a temporally “collapsed” condition by arranging the content of all frames of a dynamic display within a single static search array. As illustrated in Figure 7, it is clear that this is a difficult search task. Experiment 3 compared performance in this “collapsed” condition to that in the dynamic condition to ask whether stimulus heterogeneity is sufficient to account for the severe deficit observed under dynamic, “temporally segregated” conditions. To prevent configural effects of overalpping stimuli, “collapsed” display stimuli were spatially separated from the center point of the original bar. This was done in two different ways. In Experiment 3A, stimuli from the 3 frames of a corresponding dynamic-search trial


Jardine and Moore

Page 9

Author Manuscript

were drawn at random locations within an imaginary grid (“distal clutter”; see Figure 7). In Experiment 3B, stimuli were drawn in a clumped fashion, such that the 3 states of a given bar from the dynamic display were presented near each other with some spatial jitter (“proximal clutter”; see Figure 7). To preview some of the results, the two versions of the experiment yielded the same pattern of effects. Search performance might be better in the dynamic conditions than in the collapsed-static conditions. This would indicate that observers benefitted from the temporal isolation of subsets of stimuli, similar to advantages that have been observed in other temporally segregated search situations that involved the asynchronous presentation of subsets of items (e.g., Watson & Humphreys, 1997). Alternatively, search might be equally bad or worse in the dynamic condition compared to the collapsed condition, indicating a failure to isolate critical-frame information in the dynamic condition.

Author Manuscript

Method Participants—Thirteen participants (aged 18 – 20) and 12 participants (aged 18 – 20), all who reported normal or corrected-to-normal visual acuity and color vision and who had not participated in other experiments, completed Experiments 3A and 3B, respectively.

Author Manuscript

Design—Both experiments used a 3 Display (Static: 1 vs. Collapsed: 3 frames vs. Dynamic: 3 frames) × 2 Set size (6 vs. 18), within-subjects design. The collapsed condition was created by generating all stimuli in a 3-frame dynamic display but pseudo-randomly positioning them in one static array (3A) or clumping the three states of each object near each other (3B). We chose set sizes of 6 and 18 because a 3-frame dynamic trial with set size 6 will produce a total of 18 stimuli for the collapsed condition. The 18 stimuli collapsed condition has the same heterogeneity but more stimuli in its frame compared to each frame of the 6-object 3-frame dynamic display, and it has more heterogeneity (more possible stimulus orientations) but the same number of stimuli as the 1-frame, relatively homogenous array with 18 stimuli. Frame duration was always 200 ms and stimuli centers were selected from an 8 × 8 array. Data were collected from 8 blocks of 84 trials, yielding a total of 112 observations per condition.

Author Manuscript

Results—One participant’s data were excluded for mean performance lower than 55%. Mean percent correct as a function of Display and Set size is shown in Figure 8. Accuracy remained, as in prior experiments, highest for 1-frame static displays and impaired for 3frame dynamic displays. Critically, there was no evidence of different levels of impairment between the two heterogeneous displays – one of them static (collapsed), the other dynamic. These observations were verified with a mixed, between 2 (Experiment) × 3 (Display) × 2 (Set size) ANOVA. There were significant main effects of Display, F(2,22) = 113.93, p < . 001, ηp2 = .912, and Set size, F(1,11) = 48.27, p < .001, ηp2 = .814, with marginal evidence for their interaction, F(2,22) = 3.43, p = .051, ηp2 = .238. There was no evidence for a difference between Experiments, nor did Experiment interact with any of the within-subjects factors (all Fs < 2.23, all ps > .15). Sidak-corrected pairwise comparisons of the


Jardine and Moore

Page 10

Author Manuscript

heterogeneous dynamic and collapsed displays found no evidence that they differed in accuracy, p = .58. Discussion

Author Manuscript

The deficit in performance due to dynamic context was equivalent to that due to presenting all of the stimuli from all of the frames at the same time in a single frame. This implies that the effective set size and featural heterogeneity were the same when stimuli were presented sequentially (6, then 6, then 6) as all together (18). This implies a complete failure of temporal selection in this task. More specifically, no information was extracted from the critical frame. This complete failure of temporal selection is striking when one considers that the two versions of the display are perceived very differently (compare Movies 1 and 4). Dynamic displays appear as sparse displays of six rotating bars. Collapsed displays appear as cluttered displays of many differently oriented bars. These are very different scenes in terms of the organization of the visual information into functional units, yet they yielded essentially identical search performance. In the next two experiments, we further pursued the hypothesis that the dynamic-search deficit is due to limitations of temporal selection by adding cues to the critical frame with the goal of facilitating temporal selection.

EXPERIMENT 4: Auditory cueing of the critical display

Author Manuscript

In Experiment 4, the critical display was accompanied by an auditory stimulus (a pure tone). The logic was that given the superior temporal resolution of audition over vision, auditory cues might facilitate temporal selection (Posner, Nissen, & Klein, 1976). Consistent with this, visual search within a display of items with randomly changing features improves when a synchronous but spatially non-informative tone is played with the target change (Van der Burg, Cass, Olivers, Theeuwes, & Alais, 2010; Van der Burg, Olivers, Bronkhorst, & Theeuwes, 2008). In Experiment 4A, a single cycle of the display was presented and a tone occurred simultaneously with the critical frame. In Experiment 4B, a cycling version of the display was presented in which the bars rotated back and forth, passing repeatedly through the critical frame. Every time the critical frame was presented, a tone occurred with it. Methods

Author Manuscript

Participants—Groups of 10 and 11 (plus 2 who were eliminated due to extremely low performance) new participants were tested in Experiments 4A and 4B, respectively. All reported normal or corrected-to-normal visual acuity and color vision, their ages ranged between 18 and 26, and none had participated in Experiments 1 – 3. Procedure—Feedback was changed in both of these experiments. Incorrect responses were followed by a red ‘X’ displayed in the center of the screen for 1000 ms. 4A: Single tone and display: A single 600 Hz tone lasting 100 ms occurred during each trial at an ISI of either −200, 0, or 200 ms relative to the onset of the critical frame. (On single-frame static trials, tones with ISIs of −200 or 200 ms occurred during a blank screen, before the onset or after the offset of the critical frame.) If the cue occurs before or at the J Exp Psychol Hum Percept Perform. Author manuscript; available in PMC 2017 May 01.

Jardine and Moore

Page 11

Author Manuscript

onset of the critical frame in dynamic context, it should facilitate temporal selection, and accuracy should be improved relative to when the tone occurs after the critical frame. After practice, there were 8 experimental blocks of 96 trials: each with 8 trials per condition for factorial manipulations of Display (Static: 1 vs. Dynamic: 11 frames) × Tone onset (−200 vs. 0 vs. 200 ms before, during, or after the onset of the CF) × Set Size (16 vs. 32).

Author Manuscript

4B: Cycling tone and display: Instead of displaying the static (1-frame) or dynamic (5frame) displays only once per trial, here they cycled back and forth. Each cycle, both static and dynamic trials lasted 5*200 ms = 1000 ms. That is, in the static display condition the critical frame was presented for 1000 ms. In the dynamic display condition, the dynamic cycle consecutively showed five frame of 200 ms duration for a total of 1000 ms, and on the next cycle the same five frames were presented in a reversed order (i.e., playing reverse; see Movie 5). The displays cycled until the participant responded or until 10 cycles had completed with no response. On half of trials, a 600 Hz tone lasting 50 ms was presented at each onset of the critical frame. On the other half of trials, no tone was played. Participants were to report the orientation of the target as quickly as possible while maintaining > 80% accuracy. In this experiment, response time (RT) was the primary dependent measure, because the displays repeated (cycled) instead of terminating as in other experiments. We report both RT and accuracy. After practice, there were 6 experimental blocks of 48 trials, with each block containing 6 trials per condition for factorial manipulations of Display (Static: 1 vs. Dynamic: 5 frames) × Tone (present vs. absent) × Set Size (16 vs. 32). Results

Author Manuscript

4A—Figure 9A shows mean accuracy as a function of condition in Experiment 4A. As in other experiments, accuracy was higher for static displays than for dynamic displays, F(1,9) = 160.27, p < .001, ηp2 = .947. There were no main effects of tone ISI or Set Size (both Fs < 1) and there were no significant interactions, all Fs < 1.5, ps > .1. Search was impaired by dynamics, but remained unaffected by the auditory cue.

Author Manuscript

4B—Figure 9B shows mean correct reaction times (RTs) as a function of condition for Experiment 4B. RTs that were shorter than 300 ms or longer than 10,000 ms were excluded from analyses. There were main effects of Display, F(1,10) = 68.93, p < .001, ηp2 = .873, and Set Size, F(1,10) = 16.74, p = .002, ηp2 = .6, but not of Tone, F < 1.5, p = .3. Set Size did interact with display type such that the impairment from dynamic displays was more pronounced for larger set sizes, F(1,10) = 16.74, p = .002, ηp2 = .626. Tone, however, did not interact with either Display or Set Size, nor was the 3-way interaction reliable, all Fs < 1.5, ps > .1. Thus, the slowing of RT in dynamic conditions relative to static conditions was the same, regardless of whether or not a tone was played during the critical frame. Table 1 shows error rates for each condition. An ANOVA revealed effects that mirrored the RT findings, with one exception: Errors were overall slightly but significantly reduced by the presence of the tone, F(1,10) = 11.05, p = .008, ηp2 = .53.


Jardine and Moore

Page 12

Discussion


Even though audition has better temporal resolution than vision (Correa, Sanabria, Spence, Tudela, & Lupiáñez, 2006; Welch, DuttonHurt, & Warren, 1986), auditory information could not be leveraged to facilitate visual temporal selection in this context. The addition of a tone that coincided with the critical frame had no impact on the poor search performance in dynamic displays (Experiment 4A). This was true even when participants could observe the critical frame repeatedly within a trial as the motion cycled back and forth through the critical frame and the tone occurred each time the critical frame was displayed (Experiment 4B). The presence of a tone slightly reduced error rates overall in Experiment 4B. This was probably a consequence of a general multisensory enhancement (Stein & Stanford, 2008) and, critically, the tone did not improve performance more in dynamic than static displays. Whatever advantage the auditory system may have over the visual system in terms of temporal resolution, it is not an advantage that observers were able to leverage in support of the temporal selection of information in the context of dynamically changing displays.

EXPERIMENT 5: Visual consistency of the critical frame Experiments 5A and 5B used visual cues to distinguish the critical frame from its surrounding temporal context, such that stimuli in the critical frame appeared to be the same as or different from stimuli in the context frames. In Experiment 5A, the critical frame was presented in a different color and luminance than the temporally surrounding frames (see Figure 10A). In Experiment 5B, the context frames were comprised of stimuli that had the same or different length as the critical frame stimuli (see Figure 10B). By way of preview, these manipulations similarly reduced the deficit due to dynamic context, but failed to eliminate it.

Author Manuscript

Methods Participants—Experiments 5A and 5B each had different sets of participants who all reported normal or corrected-to-normal visual acuity and color vision, whose ages ranged between 18 and 26, and who had not participated in Experiments 1 – 4. For each experiment, 12 observers participated and none were excluded.

Author Manuscript

5A: Color: Stimuli were presented on a dark gray (RGB 50, 50, 50; 6.6 cd/m2) background. Each subject saw two colors throughout the experiment: either dark red and bright green, or dark blue and bright yellow (color assignment was counterbalanced between subjects). All stimuli in the critical frame of a given trial were drawn in one of four colors: dark red (RGB 190, 0, 0; CIE coordinates x = 0.626, y = 0.330; luminance 20.4 cd/m2), dark blue (RGB 0, 0, 250; CIE coordinates x = 0.151, y = 0.069; luminance 20.4 cd/m2), bright green (RGB 0, 250, 0; CIE coordinates x = 0.281, y = 0.597; luminance 111.1 cd/m2), or bright yellow (RGB 220, 220, 0; CIE coordinates x = 0.448 y = 0.481; luminance 138.6 cd/m2). Notice that the two “dark” colors were lower in luminance than the other two. We treated brightness level (dark versus light) as a variable in the design. The color of stimuli in surrounding context frames was either the same as or different from the stimulus color of items in the critical frame. When they were different, they were of the opposite luminance level. Thus the sequence of frame luminances could be either same (all low-luminance frames, all high-


Jardine and Moore

Page 13

Author Manuscript

luminance frames) or different (a low-luminance critical frame surrounded by highluminance context frames or vice versa; see Movie 6). Dynamic displays were 11 frames in duration. Data were collected from 7 experimental blocks of 96 trials. The design was Display (Static vs. Dynamic-Same vs. Dynamic-Different) × Critical Frame Color/Luminance (dark vs. bright) × Set Size (12 vs. 20). Half of trials were static, and others were dynamic.

Author Manuscript

5B: Length: Stimuli were black on a gray background as in Experiments 1 – 4. Stimuli in all critical frames and stimuli in the context frames of the same length condition were identical to those in Experiments 1 – 4. In the different condition, the stimuli in the context frames were versions of the oriented bars that were extended in length beyond the borders of the search display (see Figure 10B). The idea was that locally, these were the same orientation stimuli as in the standard condition, excepting line terminators. The extreme difference in length and lack of terminators made the stimuli in the context frames appear very different from those in the critical frame. Experientially, the displays appear to be dynamic jumble of lines that suddenly clears to reveal the critical frame of distinct bars, followed by the dynamic jumble of lines again (see Movie 7). Data were collected from 10 experimental blocks of 48 trials, with 4 trials each block for factorial manipulations of Display (Static: 1 vs. Dynamic: 3 vs. Dynamic: 11 frames) × Length (Same vs. Different) × Set Size (16 vs. 32). Here, there was a small difference between Static-Same and Static-Different: the former display was comprised only of search stimuli, and the latter display added a two-frame border. (The presence or absence of border in static scenes had no effect on search.)

Author Manuscript

Results The results from both experiments are summarized in Figure 11. The main dynamic context deficit was replicated in both experiments, though it was reduced. In Experiments 5A (color) and 5B (length), performance was worse in Dynamic-Same conditions (59.6% and 57.9% in 5A and 5B, respectively) than in Static conditions (93.8% and 92.9% in 5A and 5B, respectively). Accuracy was also reliably worse in Dynamic-Different conditions (68.4% and 77.2%3 in 5A and 5B, respectively) than in Static conditions, indicating that marking the stimuli in the critical frame by a different color or different length did not eliminate the effect.

Author Manuscript

A mixed ANOVA, including the within-subjects factors of Display (Static, Dynamic-Same, Dynamic-Different) × Set Size (small, large) and the between-subjects factor of Experiment (5A, 5B) revealed a reliable main effect of Display F(1.6,35.7) = 548.12, p < .001, ηp2 = .96. Sidak-corrected pairwise comparisons revealed that Static, Dynamic-Same, and DynamicDifferent search were all significantly different from each other, all ps < .001. Specifically, performance in the Dynamic-Different condition was better than in the Dynamic-Same

3The two dynamic conditions in Experiments 5B (3 and 11 frames) were not reliably different from each other. For ease of comparison, we collapsed across those two dynamic conditions only for these critical statistical tests.


Jardine and Moore

Page 14

Author Manuscript

condition, but performance in Dynamic-Different condition was worse than in the Static condition. There was no significant difference in search between Experiments 5A and 5B overall, F < 1, p > .3. However, there was an interaction between Experiment and Display, F(1.6,35.71) = 11.88, p < .001, ηp2 = .35, confirming greater improvement for the length difference than the color/luminance difference. Finally, there was a marginal main effect of Set Size, F(1,22) = 3.06, p = .09, ηp2 = .12, and a marginal interaction between Experiment and Set Size, F(1,22) = 3.85, p = .06, ηp2 = .15. The interaction could be due to the fact that set sizes were 12 and 20 in Experiment 5A and 16 and 32 in Experiment 5B. The full ANOVAs for Experiments 5A and 5B are reproduced in Supplemental Tables 1 and 2.

Author Manuscript

Discussion The critical conclusion from these experiments is that search performance for dynamic displays improved when the critical frame contained features that differed from the context frames (color and luminance in Experiment 5A and length in Experiment 5B). Despite this improvement, performance for dynamic search displays remained impaired relative to static search.

Author Manuscript

Although these manipulations failed to eliminate the deficit for dynamic search displays, it did succeed in reducing it, unlike the use of an auditory signal to mark the critical frame (Experiment 4), The abrupt visual-feature changes may have disrupted the continuity of the visual representations of the bars as temporally continuous objects. This in turn, could have protected existing representations of those bars from being updated on the basis of later stimulus information because they were no longer represented as the same object subject to updating (Enns et al., 2009; Lleras & Moore, 2003; Moore et al., 2007; Moore & Lleras, 2005; Moore, Stephens, & Hein, 2010).

General Discussion

Author Manuscript

Observers may need to search for information that is only briefly available at one moment in time, such as the identity of the soccer player currently closest to the soccer ball on a field. In such dynamic scenes, the visual system must continually sample new information and integrate it into previously established representations. Previous work has suggested that this updating process is mediated through object representations (Enns et al., 2009; Lleras & Moore, 2003; Moore et al., 2007; Moore & Lleras, 2005; see also Enns & Di Lollo, 2000), and we therefore refer to it as object-based updating. These observations motivated two questions. What consequences does such a process have for visual search through dynamic scenes? Is episodic feature information ‘lost’ in the updated representation of the target, or can selective search processes access this episodic feature information? We examined the impact on performance of adding minimal dynamic context to a simple search display. We started with an otherwise easy visual search display, in which observers searched for an oblique bar among horizontal and vertical bars. To create dynamic displays,


Jardine and Moore

Page 15

Author Manuscript

we embedded this “critical frame” within the context of temporally flanking frames, such that the bars were presented at different orientations in step across frames and appeared to rotate through the critical search display (see Figure 3 and Movies 1 and 2).

Author Manuscript

Our general finding is that there was a massive performance deficit due to dynamic context compared to static search. Figure 12 summarizes this general finding as well as the followup experiments designed to identify the cause of this deficit by asking what changes in the task and display can reduce or eliminate it. The figure allows a comparison of the magnitude of the deficit across many conditions. Data are collapsed across secondary factors like set size, and results are grouped into categories based on the manipulations that were designed to reduce the effect. Performance was consistently good in static single-frame conditions (black triangles), ranging from 84% to 95% correct, confirming that the basic task was a standard efficient feature-search situation. In contrast, performance was consistently poor in dynamic conditions (black circles). These constitute boundary conditions where performance was nearly at ceiling in one case and nearly at chance in the other. Grey squares show data from conditions in which some manipulation was introduced to the dynamic displays to assess whether it would reduce or eliminate the deficit. The nature of the manipulation is labelled on the x-axis.

Author Manuscript

Figure 12 highlights three general findings. First, dynamic context induces a large performance deficit in a simple visual search task. Second, although that deficit can be reduced by modifying whether the critical frame is visually consistent with the dynamic context or not, it could not be eliminated. Third, the only manipulations that succeeded in reducing the deficit at all were ones that disrupted the perceptual continuity of the display, either with spatiotemporal manipulations (i.e., longer temporal separations) or with feature manipulations of the critical display. A tone, for example, did not disrupt perceived continuity even though it did temporally mark the critical frame, and it had no effect on the deficit due to dynamic context. In general, the results from this study indicate that the fidelity of temporal selection is insufficient to isolate information from the critical frame when stimuli systematically change state, at least under the parameters tested here. The question is what caused the limitation on temporal selection. The pattern of data across experiments suggests that the deficit was caused by the loss of access (by the search system) to instantaneous visual information. This constitutes a specific failure of temporal selection for representational objects in which prior information about single states (feature values) of dynamic objects become inaccessible after the representation is overwritten or otherwise updated on the basis of newly sampled information.

Author Manuscript

The failure of temporal selection in these experiments seems to be one of selecting information from instantaneous states of changing objects, rather than an absolute limit on the resolution of temporal selection. Estimates of attentional dwell time – the amount of time it takes to switch the locus of selection from one stimulus to another – range from 500 ms (Duncan, Ward, & Shapiro, 1994) to 200 ms (Moore, Egeth, Berglan, & Luck, 1996), to less than 100 ms (Wolfe, 1994). Although “attentional dwell time” and “the resolution of temporal selection” are different limitations, the former must be limited by the latter. Under


Jardine and Moore

Page 16

Author Manuscript

the right conditions, temporal selection can resolve stimuli separated by 200 ms or less. Yet, the dynamic deficit observed in this study persisted when the frames themselves lasted 800 ms (Experiment 2). We suggest that is because the stimuli in the current study were perceived as individual objects changing states over time. Thus, we suggest that it was not simply that the resolution of temporal selection was insufficient to select the target display, but rather that the target display was not represented separately within any representation to which the search system had access.

Author Manuscript

An implication of the conclusion offered here is that representing stimuli as objects has a consequence: susceptibility to the loss of component instantaneous state information. This characterization is a temporal version to the spatial metaphor of losing the trees (instantaneous states of bars) for the forest (representations of rotating bars). A related account has been offered with regard to visual spatial crowding, where it becomes difficult to identify a single stimulus that is surrounded spatially by other stimuli. (Parkes, Lund, Angelucci, & Solomon, 2001), for example, argued that a compulsory averaging process yields an ensemble representation of the set of stimuli within a given spatial region, rendering the specific values of the individual stimuli lost to the ensemble. They suggest that visual crowding across space is more like texture perception than masking. We suggest that there is a similar information integration across time.

Author Manuscript

Our findings add to other work demonstrating that object representations can effectively trump instantaneous-state information. In the attentional blink paradigm, Raymond (2003) showed that the decline in performance for a second target was more pronounced when it was an instantaneous state of a rotating object than when it was a separate object. Similarly, Goddard & Clifford (2013) demonstrated that a scene of dynamically changing colors impaired observers’ ability to select a region of objects in the scene. More generally, the idea of losing the trees for the forest has an extensive history in psychology (Digirolamo & Hintzman, 1997; see also Simons, 2000). The precedence of global information in global/ local stimuli (Navon, 1977) is a specific example, but more general is the emphasis on holistic representations that was highlighted by Gestalt Psychologists such as Wertheimer, Koffka, and Köhler (for review, see Wagemans et al., 2012). Although discussions of holistic processing do not always state this explicitly, a consequence of holistic processing could be that representations of feature-level details, accessible under some conditions, are lost under other conditions 1.

Author Manuscript

With this hypothesis, we can revisit previous findings concerning dynamic search. Two studies reported no dynamic impairment when objects changed their locations but maintained their features over time (Alvarez et al., 2007; Hulleman, 2009), but another study found a dynamic impairment at higher set sizes when scenes contained objects that changed locations and features (Kunar & Watson, 2011). We suggest that the difference is that in the latter study, individual objects within the scene changed state and required updating, whereas in the former, the entire “scene” (i.e., the search display) changed position, while individual objects maintained their identities and relative positions within the scene. In the tasks used here, although objects maintained their positions, they changed state (orientation) independently of each other and therefore required updating. Thus, a dynamic deficit was present for those studies in which objects required updating (the current study and Kunar &


Jardine and Moore

Page 17

Author Manuscript

Watson, 2011), and was absent for those in which they did not (Alvarez et al., 2007; Hulleman, 2009). Generalization

Author Manuscript

The exploration of the dynamic search deficit reported in this study has been limited to conditions of changing orientation, and to the task of searching for an oblique bar among horizontal and vertical bars. It cannot be ascertained at this point, therefore, whether this reflects a general deficit due to dynamic context, or something specific to the conditions under which we have studied it. Future studies exploring other dimensions and tasks will be valuable in allowing more general conclusions. A consideration, however, must be that whatever the change in the scene is, it has to be one that gives rise to a strong perception of temporally continuous objects undergoing change. An initial exploratory study in our lab, for example, substituted color change (step-wise through color space) for orientation. The resulting displays were clearly changing (dynamic), but they were not as compellingly organized as continuous objects changing state. We believe that how changing displays are represented in regard to object continuity is not an all-or-none quality. Generalization of this work, we argue, is going to require converging measures of object status, which is a challenge beyond the scope of this initial study. Here we have appealed to phenomenological impressions of the displays (see Movies 1 and 4) and we acknowledge that, without measures of object status, the current study also awaits confirmation that object status was critical to the observed deficit. At this point, we offer it as our best working hypothesis. Conclusions

Author Manuscript

We have documented a large effect of dynamic context on simple orientation search that cannot be attributed to masking or attentional-blink like deficits. We believe it reflects a specific failure of temporal selection that is due to observers losing access to instantaneous state information (specific orientations) through the process of organizing, and representing, the scene in terms of higher-order units (rotating bars). Assuming that account is correct, it remains to be determined whether that instantaneous state information is truly lost to the system, or whether it is represented but inaccessible within the context of the sort of task used in this study.

Supplementary Material Refer to Web version on PubMed Central for supplementary material.

Acknowledgments Author Manuscript

Thanks to Michael Dodd, Jim Enns, Jeremy Wolfe, and an anonymous reviewer for thoughtful comments on an earlier draft of this manuscript. We thank our research assistants – Hayley Nelson, Tanner Downey, and Olga Shakhnovich – for their many hours running participants in these experiments. Parts of this work have been presented as posters and received invaluable feedback at meetings of the Vision Sciences Society and the Object Perception, Attention, and Memory conference. This work was supported by NSF GRF-1048597 and the APAGS/Psi Chi Junior Scientist Fellowship to NLJ, and by NIH R21 EY023750 and NSF BCS-0818536 to CMM.


Jardine and Moore

Page 18

Author Manuscript

References

Author Manuscript Author Manuscript Author Manuscript

Aks DJ, Enns JT. Visual search for size is influenced by a background texture gradient. Journal of Experimental Psychology: Human Perception and Performance. 1996; 22(6):1467–1481. [PubMed: 8953230] Alvarez GA, Konkle T, Oliva A. Searching in dynamic displays: Effects of configural predictability and spatiotemporal continuity. Journal of Vision. 2007; 7(14):12–12. http://doi.org/10.1167/7.14.12. [PubMed: 18217807] Anderson BA. On the Precision of Goal-Directed Attentional Selection. Journal of Experimental Psychology: Human Perception and Performance. 2014 http://doi.org/10.1037/a0037685. Asher MF, Tolhurst DJ, Troscianko T, Gilchrist ID. Regional effects of clutter on human target detection performance. Journal of Vision. 2013; 13(5):25–25. http://doi.org/10.1167/13.5.25. [PubMed: 23620535] Brainard DH. The psychophysics toolbox. Spatial Vision. 1997; 10(4):433–436. http://doi.org/ 10.1163/156856897x00357. [PubMed: 9176952] Breitmeyer, BG. Visual Masking. USA: Oxford University Press; 1984. Breitmeyer BG, Ogmen H. Recent models and findings in visual backward masking: A comparison, review, and update. Perception & Psychophysics. 2000; 62(8):1572–1595. http://doi.org/10.3758/ BF03212157. [PubMed: 11140180] Breitmeyer, BG.; Ogmen, H. Visual masking: Time slices through conscious and unconscious vision. 2nd ed.. New York: Oxford University Press; 2006. Castelhano MS, Heaven C. The relative contribution of scene context and target features to visual search in scenes. Attention, Perception, & Psychophysics. 2010; 72(5):1283–1297. http://doi.org/ 10.3758/APP.72.5.1283. Correa Á, Sanabria D, Spence C, Tudela P, Lupiáñez J. Selective temporal attention enhances the temporal resolution of visual perception: Evidence from a temporal order judgment task. Brain Research. 2006; 1070(1):202–205. http://doi.org/10.1016/j.brainres.2005.11.094. [PubMed: 16403468] Cunningham CA, Wolfe JM. The Role of Object Categories in Hybrid Visual and Memory Search. 2014 http://doi.org/10.1037/a0036313.supp. Digirolamo GJ, Hintzman DL. First impressions are lasting impressions: A primacy effect in memory for repetitions. Psychonomic Bulletin & Review. 1997; 4(1):121–124. http://doi.org/10.3758/ BF03210784. Duncan J, Humphreys GW. Visual search and stimulus similarity. Psychological Review. 1989; 96:433–458. [PubMed: 2756067] Duncan J, Ward R, Shapiro K. Direct measurement of attentional dwell time in human vision. Nature. 1994; 369(6478):313–315. http://doi.org/10.1038/369313a0. [PubMed: 8183369] Eckstein MP. Visual search: A retrospective. Journal of Vision. 2011; 11(5):14–14. http://doi.org/ 10.1167/11.5.14. [PubMed: 22209816] Eckstein MP, Thomas JP, Palmer J, Shimozaki SS. A signal detection model predicts the effects of set size on visual search accuracy for feature, conjunction, triple conjunction, and disjunction displays. Perception & Psychophysics. 2000; 62(3):425–451. http://doi.org/10.3758/BF03212096. [PubMed: 10909235] Enns JT, Di Lollo V. What’s new in visual masking? Trends in Cognitive Sciences. 2000; 4(9):345– 352. [PubMed: 10962616] Enns JT, Di Lollo V. Origins of substitution. Trends in Cognitive Sciences. 2001; 5(2):53. [PubMed: 11166634] Enns JT, Rensink RA. Influence of Scene-Based Properties on Visual Search. Science. 1990; 247:721– 723. [PubMed: 2300824] Enns, JT.; Lleras, A.; Moore, CM. Object updating: a force for perceptual continuity and scene stability in human vision; Problems of space and time in perception and action. 2009. p. 503-520.Retrieved from http://aegean.psychology.uiowa.edu/labs/iapl/iAPL_people/ cathleen_moore/pdfs/enns_et_al_in_press_chapter.pdf


Jardine and Moore

Page 19

Author Manuscript Author Manuscript Author Manuscript Author Manuscript

Flombaum, JI.; Scholl, BJ.; Santos, LR. Spatiotemporal priority as a fundamental principle of object persistence. In: Hood, BM.; Santos, LR., editors. The Origins of Object Knowledge. 2009. p. 135-164. Foster DH, Ward PA. Horizontal--Vertical Filters in Early Vision Predict Anomalous Line-Orientation Identification Frequencies. Proceedings of the Royal Society of London B: Biological Sciences. 1991; 243(1306):83–86. Francis G. Quantitative theories of metacontrast masking. Psychological Review. 2002; 107(4):1–20. Franconeri SL, Simons DJ. Moving and looming stimuli capture attention. Perception & Psychophysics. 2003; 65(7):999–1010. [PubMed: 14674628] Goddard E, Clifford CWG. A new type of change blindness: Smooth, isoluminant color changes are monitored on a coarse spatial scale. Journal of Vision. 2013; 13(5):20–20. http://doi.org/ 10.1167/13.5.20. [PubMed: 23608342] Hillstrom AP, Yantis S. Visual motion and attentional capture. Perception & Psychophysics. 1994; 55(4):399–411. http://doi.org/10.3758/BF03205298. [PubMed: 8036120] Horowitz T, Treisman A. Attention and apparent motion. Spatial Vision. 1994; 8(2):193–219. http:// doi.org/10.1163/156856894x00332. [PubMed: 7993876] Hulleman J. No need for inhibitory tagging of locations in visual search. Psychonomic Bulletin & Review. 2009; 16(1):116–120. http://doi.org/10.3758/PBR.16.1.116. [PubMed: 19145020] Jardine NL, Seiffert AE. Tracking objects that move where they are headed. Attention, Perception, & Psychophysics. Jul 16.2011 http://doi.org/10.3758/s13414-011-0169-8. Julesz B, Bergen JR. Textons, the fundamental elements in preattentive vision and perception of textures. Bell System Technical Journal, the. 1983; 62(6):1619–1645. http://doi.org/10.1002/j. 1538-7305.1983.tb03502.x. Kleffner DA, Ramachandran VS. On the perception of shape from shading. Perception & Psychophysics. 1992; 52(1):18–36. [PubMed: 1635855] Kunar MA, Watson DG. Visual search in a multi-element asynchronous dynamic (MAD) world. Journal of Experimental Psychology: Human Perception and Performance. 2011; 37(4):1017– 1031. http://doi.org/10.1037/a0023093. [PubMed: 21500946] Lleras A, Moore CM. When the target becomes the mask: using apparent motion to isolate the objectlevel component of object substitution masking. Journal of Experimental Psychology: Human …. 2003 http://doi.org/10.1037/0096-1523.29.1.106. McLeod P, Driver J, Crisp J. Visual search for a conjunction of movement and form is parallel. Nature. 1988; 332(10):154–155. [PubMed: 3347252] Moore CM, Brown LE. Preconstancy information can influence visual search: The case of lightness constancy. Journal of Experimental Psychology: Human Perception and Performance. 2001; 27(1): 178–194. http://doi.org/10.1037//0096-1523.27.1.178. [PubMed: 11248932] Moore CM, Lleras A. On the role of object representations in substitution masking. Journal of Experimental Psychology: Human …. 2005 http://doi.org/10.1037/0096-1523.31.6.1171. Moore CM, Egeth H, Berglan LR, Luck SJ. Are attentional dwell times inconsistent with serial visual search? Psychonomic Bulletin & Review. 1996; 3(3):360–365. http://doi.org/10.3758/ BF03210761. [PubMed: 24213938] Moore CM, Mordkoff JT, Enns JT. The path of least persistence: Object status mediates visual updating. Vision Research. 2007; 47(12):1624–1630. http://doi.org/10.1016/j.visres.2007.01.030. [PubMed: 17451777] Moore CM, Stephens T, Hein E. Features, as well as space and time, guide object persistence. Psychonomic Bulletin & Review. 2010; 17(5):731–736. http://doi.org/10.3758/PBR.17.5.731. [PubMed: 21037174] Morris SB, DeShon RP. Combining effect size estimates in meta-analysis with repeated measures and independent-groups designs. Psychological Methods. 2002; 7(1):105–125. http://doi.org/ 10.1037//1082-989X.7.1.105. [PubMed: 11928886] Mühlenen von A, Müller HJ, Müller D. Sit-and-Wait Strategies in Dynamic Visual Search. Psychological Science. 2008; 14(4):309–314. Navon D. Forest before trees: The precedence of global features in visual perception. Cognitive Psychology. 1977; 9(3):353–383. http://doi.org/10.1016/0010-0285(77)90012-3. J Exp Psychol Hum Percept Perform. Author manuscript; available in PMC 2017 May 01.

Jardine and Moore

Page 20


Neider MB, Zelinsky GJ. Cutting through the clutter: Searching for targets in evolving complex scenes. Journal of Vision. 2011; 11(14):7–7. http://doi.org/10.1167/11.14.7. [PubMed: 22159628] Ogmen H, Breitmeyer BG, Melvin R. The what and where in visual masking. Vision Research. 2003; 43(12):1337–1350. http://doi.org/10.1016/S0042-6989(03)00138-X. [PubMed: 12742104] Palmer J. Set-Size Effects In Visual Search: the Effect of Attention is Independent of the Stimulus for Simple Tasks. Vision Research. 1994; 34(13):1703–1721. [PubMed: 7941377] Palmer J. Attention in visual search: Distinguishing four causes of a set-size effect. Current Directions in Psychological Science. 1995 Parkes L, Lund J, Angelucci A, Solomon JA. Compulsory averaging of crowded orientation signals in human vision. Nature. 2001 Pashler H. Detecting conjunctions of color and form: Reassessing the serial search hypothesis. Perception & Psychophysics. 1987; 41(3):191–201. http://doi.org/10.3758/BF03208218. [PubMed: 3575078] Pelli DG. The VideoToolbox software for visual psychophysics: Transforming numbers into movies. Spatial Vision. 1997 Posner MI, Nissen MJ, Klein RM. Visual dominance: An information-processing account of its origins and significance. Psychological Review. 1976; 83(2):157–171. http://doi.org/10.1037/0033-295X. 83.2.157. [PubMed: 769017] Pylyshyn ZW, Storm RW. Tracking multiple independent targets: Evidence for a parallel tracking mechanism. Spatial Vision. 1988; 3(3):179–197. [PubMed: 3153671] Raymond JE, Shapiro KL, Arnell KM. Temporary suppression of visual processing in an RSVP task: An attentional blink? Journal of Experimental Psychology: Human Perception and Performance. 1992; 18(3):849. [PubMed: 1500880] Rensink RA, Cavanagh P. The influence of cast shadows on visual search. Perception. 2004; 33(11): 1339–1358. http://doi.org/10.1068/p5322. [PubMed: 15693675] Rensink RA, Enns JT. Early completion of occluded objects. Vision Research. 1998; 38:2489–2505. [PubMed: 9798011] Rosenholtz R, Li Y, Nakano L. Measuring visual clutter. Journal of Vision. 2007; 7(2):17–17. http:// doi.org/10.1167/7.2.17. [PubMed: 18217832] Scharff A, Palmer J, Moore CM. Evidence of fixed capacity in visual object categorization. Psychonomic Bulletin & Review. 2011a; 18(4):713–721. http://doi.org/10.3758/ s13423-011-0101-1. [PubMed: 21538202] Scharff A, Palmer J, Moore CM. Extending the simultaneous-sequential paradigm to measure perceptual capacity for features and words. Journal of Experimental Psychology: Human Perception and Performance. 2011b; 37(3):813–833. http://doi.org/10.1037/a0021440. [PubMed: 21443383] Shih S-I. The attention cascade model and attentional blink. Cognitive Psychology. 2008; 56(3):210– 236. http://doi.org/10.1016/j.cogpsych.2007.06.001. [PubMed: 17624321] Simons DJ. Current approaches to change blindness. Visual Cognition. 2000 Stein BE, Stanford TR. Multisensory integration: current issues from the perspective of the single neuron. Nature Reviews Neuroscience. 2008; 9(4):255–266. http://doi.org/10.1038/nrn2331. [PubMed: 18354398] Theeuwes J, Kooi FL. Parallel search for a conjunction of contrast polarity and shape. Vision Research. 1994; 34(22):3013–3016. http://doi.org/10.1016/0042-6989(94)90274-7. [PubMed: 7975336] Torralba A, Oliva A, Castelhano MS, Henderson JM. Contextual guidance of eye movements and attention in real-world scenes: The role of global features in object search. Psychological Review. 2006; 113(4):766–786. http://doi.org/10.1037/0033-295X.113.4.766. [PubMed: 17014302] Treisman AM, Gelade G. A feature-integration theory of attention. 1980 Treisman A, Gormican S. Feature analysis in early vision: Evidence from search asymmetries. Psychological Review. 1988; 95(1):15–48. [PubMed: 3353475] Treisman A, Sato S. Conjunction search revisited. Journal of Experimental Psychology: Human …. 1990; 16(3):459–478.


Jardine and Moore

Page 21


Van der Burg E, Cass J, Olivers CNL, Theeuwes J, Alais D. Efficient Visual Search from Synchronized Auditory Signals Requires Transient Audiovisual Events. PLoS ONE. 2010; 5(5):e10664. http:// doi.org/10.1371/journal.pone.0010664. [PubMed: 20498844] Van der Burg E, Olivers CNL, Bronkhorst AW, Theeuwes J. Pip and pop: Nonspatial auditory signals improve spatial visual search. Journal of Experimental Psychology: Human Perception and Performance. 2008; 34(5):1053–1065. http://doi.org/10.1037/0096-1523.34.5.1053. [PubMed: 18823194] Wagemans J, Elder JH, Kubovy M, Palmer SE, Peterson MA, Singh M, Heydt von der R. A century of Gestalt psychology in visual perception: I. Perceptual grouping and figure-ground organization. Psychological Bulletin. 2012; 138(6):1172–1217. http://doi.org/10.1037/a0029333. [PubMed: 22845751] Watson DG, Humphreys GW. Visual marking: Prioritizing selection for new objects by top-down attentional inhibition of old objects. Psychological Review. 1997; 104(1):90–122. http://doi.org/ 10.1037/0033-295X.104.1.90. [PubMed: 9009881] Welch RB, DuttonHurt LD, Warren DH. Contributions of audition and vision to temporal rate perception. Perception & Psychophysics. 1986; 39(4):294–300. http://doi.org/10.3758/ BF03204939. [PubMed: 3737359] Wolfe JM. Guided search 2.0 A revised model of visual search. Psychonomic Bulletin & Review. 1994; 1(2):202–238. [PubMed: 24203471] Wolfe JM. Moving towards solutions to some enduring controversies in visual search. Trends in Cognitive Sciences. 2003; 7(2):70–76. http://doi.org/10.1016/S1364-6613(02)00024-4. [PubMed: 12584025] Wolfe JM, Friedman-Hill SR, Stewart MI, O’Connell KM. The role of categorization in visual search for orientation. Journal of Experimental Psychology: Human Perception and Performance. 1992; 18(1):34. [PubMed: 1532193] Yang H, Zelinsky GJ. Visual search is guided to categorically-defined targets. Vision Research. 2009; 49(16):2095–2103. http://doi.org/10.1016/j.visres.2009.05.017. [PubMed: 19500615] Zhaoping L, Guyader N. Interference with Bottom-Up Feature Detection by Higher-Level Object Recognition. Current Biology. 2007; 17(1):26–31. http://doi.org/10.1016/j.cub.2006.10.050. [PubMed: 17208182]

Author Manuscript Author Manuscript J Exp Psychol Hum Percept Perform. Author manuscript; available in PMC 2017 May 01.

Jardine and Moore

Page 22


Figure 1.

The oblique target “pops out”: it is quickly and accurately localized and identified regardless of the number of 0° and 90° distractors, even with a brief 50 ms presentation (e.g., Wolfe, 1992).


Jardine and Moore

Page 23


Figure 2.

When a search array from which a target “pops out” is embedded in dynamic context, can visual search proceed encapsulated within the critical array (black brackets), or does search operate on a representation that includes some information about the context (blue brackets)?


Jardine and Moore

Page 24

Author Manuscript Figure 3.

Author Manuscript

Illustration of trial events for a 3-frame dynamic display. The critical frame, in which distractors are horizontal or vertical and only the target is tilted, occurred in the middle of the sequence. Each bar rotated consistently CW or CCW throughout the trial, generating coherent if not smooth motion. Observers reported whether the target was tilted left or right (unspeeded 2AFC).


Jardine and Moore

Page 25

Author Manuscript Author Manuscript Author Manuscript Figure 4.

Author Manuscript

Results of Experiment 1. Mean percent of correctly identified targets (2AFC orientation). 50% is chance. Error bars show standard error of the mean.


Jardine and Moore

Page 26


Figure 5.

Results of Experiment 1B. Percent of correctly identified targets, averaged across whether the target was present or absent. 50% is chance. Error bars show standard error of the mean.


Jardine and Moore

Page 27


Figure 6.

Results of Experiment 2A (hollow circles) and 2B (filled circles) as a function of display (static, dynamic) and frame duration, averaged across set size. Dynamic displays lasted 5 frames.

Author Manuscript J Exp Psychol Hum Percept Perform. Author manuscript; available in PMC 2017 May 01.

Jardine and Moore

Page 28

Author Manuscript Author Manuscript Figure 7.

Author Manuscript

The Collapsed conditions in Experiments 3A and 3B were created by taking all of the orientations that would be presented in a 3-frame dynamic trial and spatially rearranging them within a single static array, either fully randomly within the 8×8 cell grid used in all trials (3A) or near each other (3B).


Jardine and Moore

Page 29


Figure 8.

Results, as a function of Display and Set Size, of Experiments 3A and 3B. The collapsed and dynamic conditions have the same numerosity and heterogeneity of stimuli, such that these display conditions differ only as to whether stimuli are presented overlapping with temporal dynamics (dynamic) or non-overlapping and without them (collapsed).


Jardine and Moore

Page 30


Author Manuscript

Results of Experiments 4A&B. In 4A, a tone occurred before, during, or after the onset of the Critical Frame in Static and Dynamic trials. In 4B, the display cycled back and forth up to 10 times until observers responded. A tone was present or absent at each onset of the Critical Frame.


Jardine and Moore

Page 31


Figure 10.

Examples of Dynamic displays in Experiments 5A and B, in which objects in the critical frame were the Same or Different color (5A) or length (5B) as objects in the context frames.


Jardine and Moore

Page 32


Author Manuscript

Results of Experiments 5A&B. Solid lines mark displays in which objects in the critical frame had the Same appearance with those in context frames; dashed lines indicate inconsistency. A, B: The color (A) or size (B) of all critical frame objects was consistent or inconsistent with context. In 5B, data are averaged between the two types of Dynamic displays (3 or 11 frames).


Jardine and Moore

Page 33

Author Manuscript Author Manuscript Author Manuscript Figure 12.

Author Manuscript

Figure summarizing findings in this manuscript. The “Baseline Impairment” group contains data from from all experiment conditions with consistent context (N=11), including frame durations 400 ms and shorter from Exp. 2. It also includes Experiment 4A, in which a tone cued the onset of the critical frame. The “Frame manipulations” set includes the longer frame durations of Exp. 2 (500–2500 ms) and the frame consistency manipulations of Exp. 5. Error bars, when present, show standard error of the mean for across each condition.


Jardine and Moore

Page 34

Table 1

Author Manuscript

Error rates (S.E.M.) for experiment 5B, in which the display cycled back and forth, allowing observers up to 10 onsets of the critical frame to identify the target’s orientation. Error Rates (in Percent) for Experiment 5B Tone Absent

Tone Present

Static

3.5 (0.9)

2.3 (0.8

Dynamic

21.6 (4.3)

20.0 (3.7)

Set Size 16

Set Size 32 Static

2.8 (0.5)

1.3 (0.4)

Dynamic

27.0 (3.7)

25.8 (4.5)

Author Manuscript Author Manuscript Author Manuscript J Exp Psychol Hum Percept Perform. Author manuscript; available in PMC 2017 May 01.

Losing the forest for the trees.

Nanotoxicology: Seeing the trees for the forest.

Pediatric Nephrology in Primary Care: The Forest for the Trees.

Seeing the forest through the trees.

You can detect the trees as well as the forest when adding the leaves: evidence from visual search tasks containing three-level hierarchical stimuli.

Forest Plots and Prediction of Postoperative Delirium: Missing the Forest for the Trees?

Sickle Cardiomyopathy: The Missing Forest in the Trees.

On pluralism in ecology: seeing the forest and the trees.

Specific versus relational factors in addiction treatment: the forest and the trees, or just more trees?

Not seeing the forest for the trees: size of the minimum spanning trees (MSTs) forest and branch significance in MST-based phylogenetic analysis.

Vulnerability of dynamic genetic conservation units of forest trees in Europe to climate change.

Beyond Bertin: seeing the forest despite the trees.

The trees and the forest: Characterization of complex brain networks with minimum spanning trees.

Ruptured intracranial aneurysms in young adults: seeing the forest for the trees.

Seeing the forest for the trees: kidney oncogenomes in relation to therapeutic outcomes.

Cue search and comparison processes in visual search for letters.

The role of categorization in visual search for orientation.

The role of memory for visual search in scenes.

Statistical templates for visual search.

Visual search and the N2pc in children.

Searching for inefficiency in visual search.

Parallel mechanisms for visual search in zebrafish.

Attention in visual search: multiple search classes.

Parvalbumin Interneurons: All Forest, No Trees.