0042-6989/92 $5.00 + 0.00 Copyright 0 1992 Pergamon Press plc
Vision Res. Vol. 32, No. 3, pp. 417424, 1992 Printed in Great Britain. All rights reserved
Electrophysiological Correlates of Texture Segregation in the Human Visual Evoked Potential* MICHAEL
BACH,t$
THOMAS
MEIGENt
Received 5 November 1990; in revisedform 5 April 1991
We investigated whether the visual evoked potential (VEP) reflects cortical processing associated with preattentive texture segregation. On a visual display unit we presented stimuli with various arrangements of oriented line segments that either led to the appearance of a ‘preattentive” checkerboard or did not. Two presentation modes were used (pattern onset at 1 Hz and rapid pattern change at 4.3 Hz), while luminance (57cd/m2) and contrast (92%) of the line segments remained constant. VEPs were recorded in 7 human subjects. The VEP was analyzed as a linear combination of putative components, which are evoked by either local pattern, quasi-local orientation contrast or global preattentive structure. In the transient VEP, we found a negativity over the posterior pole at a latency between 161 and 225msec (FWHM) in the linear combination designed to extract segregationspecljic components. Peak amplitude reached 3.1 f 0.8 p V (mean _+ SEM) at 199 msec. This negative peak appeared only for textures containing orientation contrast. Steady-state analysis of the rapid presentation also revealed a sign$cant component (P = 0.002) associated with texture segregation. These potentials either represent processing of orientation contrast or global processing of texture segregation. The results suggest that specljic surface potentials, diflering from cognitive potentials, can be derived which are associated with preattentive processing. Texture segregation
Visual evoked potentials
Orientation
contrast
Texton
Feature analysis
Human
appeared or did not appear. We hypothesized that the activity of any component related to texture segregation would be superimposed on the “normal” VEP. Using appropriate linear combinations of the responses to the various stimuli, these different responses could then be isolated.
INTRODUCTION Preattentive processing of textures can lead to perceptual “pop out” or “segregation” of global elements (Beck, 1972, 1983; Julesz & Bergen, 1983; Treisman, 1985; Julesz, 1986). Segregation occurs if gradients of certain local features, sometimes called textons, are present. We investigated whether cortical processing, associated with texture segregation, can be studied using the visual evoked potential (VEP) in human subjects. Orientation gradients provide very robust texture segregation (Nothdurft, 1990). For this reason we chose orientation contrast among the many possible local features. Presentation of any visual suprathreshold stimulus (e.g. checkerboards, gratings, or line drawings of faces) evokes an associated, though very small, scalp potential. By means of multiple presentation and averaging, this component can be separated from ongoing brain activity. We presented various stimuli in which the number of line segments stayed constant, but orientation was arranged such that a “preattentive checkerboard”
METHODS Subjects
Seven visually normal observers served as subjects. They wore appropriate refraction if necessary. Acuity was 2 1.2; age ranged from 23 to 69 years. The subjects gave their informed consent to participate in the experiment. Four of the 7 subjects were naive as to the specific aim of the experiment. Stimuli
Stimuli were presented on a visual display unit (AEG-HCM38) with a resolution of 832 x 832 pixels. The graphics controller (Miro-510) was programmed for pixels of square size, resulting in a line frequency df 64 kHz and a frame rate of 69 Hz. By tilting both the VDU and the line segments by 45”, giving the overall appearance of horizontal and vertical lines, we eliminated artifactal luminance differences. Such luminance
*A preliminary report was presented at the ARVO 1990. tElektrophysiologisches Labor, UniversiWs-Augenklinik, Killianstr. 5, D-7800 Freiburg, Fed. Rep. Germany. STo whom all correspondence should be addressed. 417
418
MICHAEL
0’
0
BACH
and THOMAS
---Radius of stimulus area [“I
FIGURE 1. Effect of stimulus extent. The ordinate plots the steadystate response to alternation between differently oriented line segments [(b) and (c) of Fig. 2 were displayed with a variable aperture]. On the abscissa the radius of the pattern varies between 0.7 and 4.3’. Individual data and mean are shown for 3 subjects. At eccentricities above z4”, response to these stimuli does not increase appreciably.
artifacts occur with raster scan VDUs, as lines along the scan direction are brighter than lines perpendicular to the scan direction. Two temporal presentation modes were used: in experiments (1) and (2), patterns repetitively appeared for 300 msec, followed by a blank screen for 700 msec. This evoked one pattern-onset response per second. In experiment (3) steady-state “pattern change” (alternation between two different patterns) was presented at a rate of 8.6 pattern changes per second. The sweep time of 465 msec contained 4 steady-state responses. The luminance of the line segments was 57 cd/m*, that of the background 2.4 cd/m*, resulting in a contrast of 92%. Space averaged luminance stayed constant at 3.0 cd/m2 in experiments (1) and (3) and at 2.7cd/m2 in experiment (2). Control measurements to determine appropriate$eld
size
To determine the maximal useful eccentricity where orientation contrast would evoke appreciable responses, we performed the following initial experiment: steadystate responses to homogenous texture stimuli were recorded. The radius of the circular stimulus field varied from 1.4” up to 4.2”. The orientation of the line segments changed by 90” 8.6 times per second. Figure 1 shows that the responses saturated at ~4” of eccentricity. Based on these results, we placed the display at a distance of 114 cm and surrounded it by a circular mask, leaving a stimulus radius of 5”. We then chose subjectively the specific values for the final stimulus patterns as a tradeoff between the aim to introduce many
MEIGEN
orientation contrast borders (assuming that many such borders increase associated VEP components) and sufficient definition of each individual check. The final stimuli (Fig. 2) consisted of line segments of 0.2” length (0.1 o in experiment 2). The line segments were placed in a regular lattice, with a distance of 0.4” between the lines. Their individual position varied randomly up to + 0.05”, introducing a spatial “‘jitter” to avoid regular structures at borders. For the “preattentive checkerboard” [Fig. 2(a)], orientation differed by 90”, subdividing the lattice into 4 x 4 cells. Recording
VEP was recorded from an Oz-Fz derivation using gold-cup electrodes. Signals were amplified and filtered (first-order bandpass, 0.3-70 Hz, Toennies “Physiologic Amplifier”) and digitized to a resolution of 12 bits at a sampling interval of 2.92 msec with a small laboratory computer (386 AT-compatible). The computer averaged the sweeps if their amplitude did not exceed f 100 pV, displayed them on-line and simultaneously generated the stimuli. Procedure
At the beginning of each session we measured VEP responses to luminance-contrast checkerboard patterns as used in the clinical routine. Experiments (1) and (2) were performed in the same session. We presented the various stimuli in an interleaved block design: each stimulus appeared 10 times, then the next stimulus followed. This sequence, comprising the 3 stimuli for experiment (l), was repeated twice, followed by the stimuli for experiment (2). This cycle was repeated 5 times, resulting in a total of 100 sweeps for each condition. The entire recording session lasted about two hours. A similar procedure was used for experiment (3). As behavioral task, the subjects fixated the center of the screen and reported random digits which appeared there for 300 msec in random intervals between 2 and 10 sec. Data analysis
We based our data analysis on the assumption that the VEP is an aggregate consisting of the local patternonset response plus hypothetical responses related to preattentive texture segregation. The latter may either
VEP CORRELATES
OF TEXTURE
419
SEGREGATION
Subject DB
“Segregation-specific”
Amplitude
Time [ms] FIGURE 3. Experiment (I), transient pattern-onset stimulation, subject DB. On the left, the stimuli are symbolized. In the center, raw responses {a}, {b} and {c} to these stimuli are depicted. On the right, a linear combination of the raw responses is plotted, the “segregation-specific” response: (d) = {a) - (fb) + {c))/2. For interpretation see Fig. 4.
reflect “quasi-1ocai” processing of orientation contrast between neighboring line elements, or more global and complex mechanisms. To separate the local responses from those specific to texture segregation, certain linear combinations of the responses were calculated (see Results). Experiments (1) and (2) used transient stimulation. To estimate the amplitudes of peak responses,
7 subjects
,$$ ;,,.
- , “&‘“:.* .;fl
texture
-gray+ ‘.9t c
baseline was defined as the mean between 0 and 60 msec. For analysis of differences in shape between tracings, d.c.-offsets were removed by subtracting baseline offset from each individual trace. The steady-state responses in experiment (3) were subjected to Fourier analysis. As a sweep contained exactly four responses, spill-over artifacts could not occur and no data-windowing was
“Segregation-specific” I
,
I ‘3t-----_i
, Amplitude
_..
bl
{cl} = {a} - z
I-
{c)
Jime [ms]
-lOti
I....
100
. .
. ..I.........I.........
200
300
I
400
Time [ms] FIGURE 4. Experiment (I), grand mean of 7 subjects, arrangement as in Fig. 3. The raw traces seem to differ little whether preattentive segregation is present {a] or not ({b), (cl}. Th e Iinear combination (d), however, shows a marked negativity between Nil-225 msec (FWHM), the “segregation-specific” response.
MICHAEL
420
BACH
and THOMAS
Figure 4 presents the grand mean from all 7 subjects. Traces are arranged as in Fig. 3. The “segregationspecific” response is a negativity which appears between 161 and 225 msec (FWHM). With reference to baseline, it had a peak amplitude of 3.1 + 0.8 PV (mean f SEM) at 199 msec.
necessary. Thus, the spectrum consists of discrete frequency components. Both amplitude and phase of the response at the presentation frequency (If) were extracted for further analysis.
RESULTS
Experiment (2): variation in line -segment length
Experiment (1): transient pattern -onset stimulation
Texton size and distance are known to influence strength of texture segregation (Nothdurft, 1985; Sagi & Julesz, 1987). In experiment (2), we reduced the length of the line segments by a factor of 2, leaving the inter-line distances constant. All subjects described the resulting “preattentive checkerboard” as being less pronounced. In Fig. 5, the responses to these stimuli are compared to the ones from experiment (1) (thick line, long line segments; thin line, short line segments; thin stepwise line, P-values of the difference). The patternonset responses to short and to long lines are very similar. The t-test for difference showed only spurious peaks, all with P Z 0.01 without Bonferroni correction (Fig. 5, top). The linear combination to extract the segregation-specific component shows a negativity around 190msec (Fig. 5, bottom). It was present for both short and long lines; for short line segments, however, the latency increased by 20 msec. The t-test showed that the difference between the two traces
Figure 3 presents VEP traces to pattern-onset stimulation of three different patterns {a}, {b) and {c} from a single subject, and a linear combination {d) of these responses. Stimuli were “preattentive orientation checkerboard” {a], a pattern consisting of vertical line segments {b}, and a pattern consisting of horizontal line segments {cl. Responses differed little between conditions {b} and {c}. As half of the area in stimulus (a} consisted of stimulus (b} and the other half of stimulus {cl, we sought to extract the “segregation-specific” responses by calculating the linear combination: (d}
=
{a} _
(b)_ (c), 2
MEIGEN
2
The resulting trace (d} shows a negativity between 161 and 206 msec (full width at half magnitude, FWHM), with a peak amplitude of 7.7 f 1.1 PV (mean + SEM) at a latency of 187 msec.
I
\
/
Amplitude 5-
-5 0
!PY
[kV]
,““,‘,‘,“‘.,.‘..,,.......,(.....,... 100
-P=.OOl -P=.Ol -P=.05
200 Time [ms]
300
41 0
FIGURE 5. Experiment (2), variation in line-segment length, grand mean of 7 subjects. On the left, the stimuli and the linear combination to extract the “segregation-specific” response are shown in pictorial form. Two line lengths differing by a factor of two were used. The resulting responses appear on the right. Thick trace, long line segments; thin trace, short line segments; thin stepwise line, P-value of their difference. The normal pattern responses to the two line lengths are very similar (top right). The “segregation-specific” response using long line segments (with strong perceptual segregation) has 20 msec shorter latency as compared to short line segments (with weak segregation), but roughly equal amplitude.
VEP CORRELATES
OF TEXTURE
was significant between 160 and 170 msec after pattern onset. Experiment (3): steady-state stimulation Steady-state responses to rapid “pattern-change” stimulation were also measured. Each stimulus consisted of a pair of two patterns, which alternated at 4.3 Hz. Fourier analysis separated the responses occurring on every pattern change (at 2f = 8.6 Hz) from the responses specific for the difference of patterns (at lf = 4.3 Hz) in the frequency domain. The specific stimuli used are depicted symbolically in Fig. 6, their properties, predicted responses and actual responses were as follows:
14 Stimulus
composition: all local features change, both patterns have identical preattentive segregation. Prediction: local pattern response at 2f, “segregation reversal” at 2f (which was not further
q 1
I--
l
I
--
--
--I
-
--
I
1
l
I
SEGREGATION
analyzed), no response at lf (Control’). Result: strong response at 2f, low Control’ response at lf Stimulus composition: all local features change, both patterns have no preattentive segregation. Prediction: mixture, used for linear combination {f} below. Stimulus composition: half of the local features change (from vertical to horizontal), segregation appears on the first of the two patterns. Prediction: mixture, used for linear combinations below. Partial inversion of {c}, segregation appears on the second of the two patterns. Linear combination, algebraic difference of {c} and {d}. Stimulus composition: local features cancel, segregation onset is inversely combined with segregation offset. Prediction: “segregation-specific” component should appear at lf Result: strong response at lj
I
-I
I I I l--
421
I
-
Linear combination {el = {cl - (4
Linear combination (9 = {al + {bl - {cl - (4
Time [ms]
Frequency [Hz]
FIGURE 6. Experiment (3). Rapid “pattern-change” stimulation, subject TM. On the left are shown the stimuli {a}-(d), or the linear combination rules. The resulting traces appear in the center. Their frequency spectrum is depicted on the right. Three types of responses are used for further analysis: two control conditions (Control’ and Control”, O), and the “segregationspecific” component (*). For explanation see text.
422
MICHAEL
0 Control’
BACH
and THOMAS
v Control”
6 Subjects
Po_*
= ,002
Grand Mean
FIGURE 7. Responses from experiment (3), plotted in the complex plane. The distance from the origin indicates the amplitude, the angular position indicates the phase of the response. The circles (0) represent the Control’ and Control” responses ({a} and {f} in Fig. 6), and the asterisks (*) represent the “segregation-specific” component ({e} in Fig. 6). (a) Data from Fig. 6, subject TM; (b) 6 subjects; (c) grand mean of 6 subjects. Both control conditions (0) cluster around zero. The “segregation-specific” component (*) differs significantly from the control conditions (P = 0.002).
f} Linear combination {a> + {b} - {c} - {d} to yield a further control condition. Stimulus composition: on the average, all local features and segregation cues cancel (note that the same four patterns making up (a} and {b} also appear in (c} and {d}, though in a different combination). Prediction: no response should appear at If (Control”). Result: very low response Control” at 1s The magnitude spectrum in Fig. 6 omits the phase. A polar plot, or Bode diagram [Fig. 7(a)], shows both
MEIGEN
amplitude and phase of the responses (Victor, 1985). The circles (0) represent Control’ and Control” responses ((a} and {f} in Fig. 6), and the asterisk (*) represents the segregation-specific component ((e} in Fig. 6). Figure 7(b) compiles the results from all 6 subjects, while Fig. 7(c) shows their grand mean. In Fig. 7(b), it can be seen that the Control’ and Control” responses cluster around zero, the mean amplitude was 0.07 PV resp. 0.11 pV. The segregation-specific component has a mean amplitude of 1.56 pV. For determination of statistical significance, we performed a multiple analysis of variance (MANOVA) on these results. Initially, the two control conditions (Control’ {a), Control” {f}) and the sine- and cosinecomponents of the complex response were entered as factors (2 x 2 MANOVA). No significant difference resulted between Control’ and Control” (P = 0.88). Consequently, these two conditions were collapsed into one (Control = Control’ u Control”). Against this control we then tested the “segregation-specific” response (e> and found a highly significant difference (P = 0.002). Our subjects reported an interesting additional observation: if one of the two rapidly interchanging patterns contained global segregation, and the other did not ({c) or {d) in Fig. 6), global segregation was perceived as continuously present. Varying the presentation rate, we found that at 2 Hz and below segregation onset and offset was perceived, while at 3 Hz and above it appeared stationary, although the 90”-rotation of the individual line segments was still clearly visible.
DISCUSSION
If VEP responses to stimuli with and without preattentive segregation are subtracted such that only putative segregation-specific components remain, we find a significant negativity in the occipital evoked potential between 161 and 225 msec after stimulus onset. Extraction of these components assumes linearity of the system. The specific results found in experiment (2) suggest that this assumption holds sufficiently to extract components associated with segregation. The latency of the “segregation-specific” component suggests that it occurs after the normal pattern onset- or pattern-reversal components (which peak around 140 msec with these stimuli) and earlier than endogenous cognitive potentials (which typically occur around 300 msec). An additional requirement for our analysis is sufficient stationarity, which is questionable in human subjects under extended recording conditions. The low responses on the control conditions suggest that either fluctuations of arousal and attention did not appreciably modulate responses associated with preattentive processing, or that interleaving of the various recording conditions has been sufficient to distribute such effects equally among the conditions. Nothdurft and Li (1985) have shown that texture segregation often arises simply from luminance differences. We used “orientation” as texton, which is not affected by such effects if sufficient spatial jitter is
VEP CORRELATES
OF TEXTURE
employed (Nothdurft, 1990). Our subjects could not detect any global features when they viewed the stimulus with strong blur. Thus we assume that the luminance distribution did not play a significant role in detecting texture borders in our experiments. The most parsimonious interpretation would then relate the “segregationspecific” response to the processing of “quasi-local” orientation contrast, subserving preattentive segregation of the stimulus patterns. As possible neural substrate for processing of orientation contrast, the neurons reported by van Essen et al. (1989) might be likely candidates. Some of the cells recorded by these authors responded well to orientation contrast, but not to homogenous textures. This suggests that orientation contrast is processed directly and not extracted after detecting orientation per se. In our experiments, the “segregationspecific” component had a higher latency (Z 190 msec) compared to the “normal” pattern response (x 140 msec with these stimuli). It would be interesting to analyze latencies in van Essen et al.‘s (1989) data. Can the negativity at x 190 msec be related to the “mismatch negativity” (Naatanen, 1990), which occurs on any different stimulus, regardless of its task relevance (Paavilainen et al., 1987)? The “segregation-specific” component was extracted by calculating differences between the responses to various stimuli. As all of the three stimulus types in experiments (1) and (2) were “different” to their predecessor, a mismatch negativity would occur in all responses likewise and be cancelled when taking differences. We believe that the “segregation-specific” component does not reflect cognitive processing for the following reasons: first, the behavioral task directed the subjects’ attention towards the fixation point. Our subjects reported that after a number of trials they often no longer consciously registered the segregation content of the screen. Second, segregation-specific responses were also found with rapid stimulation ({cl and {d}, Fig. 6), during which the “preattentive checkerboard” was perceived as continuously present. The results of experiment (2) support the interpretation of the negativity at z 190 msec as “segregationspecific” or, more specifically, as representative for processing of orientation contrast: it is well known that shortening of the line segments (effectively lowering the rate of orientation-change over distance) decreases the subjective strength of “pop-out” (Nothdurft, 1985; Sagi & Julesz, 1987). While the normal pattern response differed very little using long or short line segments (Fig. 5, top right), the linear combination to extract the “segregation-specific” component shows a marked increase in latency. The finding that at presentation rates of 3 Hz and above the segregation is perceived as stationary, while a segregation-specific VEP could be recorded at 4.3 Hz, seems to argue against our interpretation. However, Skrandies (1987) reported a closely related finding: if dynamic random-dot stereo-stimuli are presented as rapid pattern-onset, the stereo target appears perceptually fused above 3.15 Hz, while associated VEP com-
SEGREGATION
423
ponents are recordable up to 10 Hz. We hypothesize that some low-pass mechanism, possibly related to visual persistence (Coltheart, 1980), is placed after the mechanisms which process orientation contrast in the visual processing chain. We are not aware of other experiments concerned with preattentive texture segregation in visually evoked potentials. Closely related, however, are Victor’s (1985) experiments using textures with odd-even statistics. The current results suggest that the VEP contains not only “global aspects of form” but also the activity of mechanisms related to the processing of texture segregation induced by orientation contrast. Jeffreys (1989) reported another interesting activity of higher-order processing in the VEP: a midline central and a parietal positivity in the latency range of 15&200 msec after presentation of face stimuli. As we have only used a single bipolar derivation so far, we will have to take multichannel topographic recordings to study relations to such responses. The results suggest that the VEP may be a useful tool to further our understanding of preattentive processing in vision by linking psychophysical and electrophysiological observations in human subjects.
REFERENCES Beck, J. (1972). Similarity grouping and peripheral discriminability under uncertainty. American Journal of Psychology, 85, 1-19. Beck, J. (1983). Textural segmentation, second-order statistics, and textural elements. Biological Cybernetics, 48, 125-130. Coltheart, M. (1980). The persistences of vision. Philosophical Transactions of the Royal Society B, 290, 57-69.
van Essen, D. C., de Yoe, E. A., Olavarria, J. F., Knierim, J. J., Fox, J. M., Sagi, D. & Julesz, B. (1989). Neural responses to static and moving texture patterns in visual cortex of the macaque monkey. In Lom, D. M. K. & Gilbert, C. D. (Eds), Neural mechanisms of visual perception (pp. 137-154). Woodlands, Tex.: Portfolio. Jeffreys, D. A. (1989). A face-responsive potential recorded from the human scalp. Experimental Brain Research, 78, 193-202. Julesz, B. (1986). Texton gradients: The texton theory revisited. Biological Cybernetics, 54, 245-25 1. Julesz, B. & Bergen, J. R. (1983). Textons, the fundamental elements in preattentive vision and perception of textures. The Bell System Technical Journal, 62, 1619-1645.
NZitlnen, R. (1990). The role of attention in auditory information processing as revealed by event-related potentials and other brain measures of cognitive function. Behavioral Brain Sciences, 13, 201-288. Nothdurft, H. C. (1985). Sensitivity for structure gradient in texture discrimination tasks. Vision Research, 25, 1957-1968. Nothdurft, H. C. (1990). Texton segregation by associated differences in global and local luminance distribution. Proceedings of the Royal Society London B, 239, 295-320.
Nothdurft, H. C. & Li, C. Y. (1985). Texture discrimination: Representation of orientation and luminance differences in cells of the cat striate cortex. Vision Research, 25, 99-113. Paavilainen, P., Alho, K., Reinikainen, K., Sams, M. & Niilmnen, R. (1987). Attention independence of the mismatch negativity of the human event-related potential (ERP). Neuroscience (Suppl.), 22, S759 (2271P).
Sagi, D. & Julesz, B. (1987). Short-range limitation on detection of feature differences. Spatial Vision, 2, 39-49. Skrandies, W. (1987). Visual persistence of stereoscopic stimuli: Electric brain activity without perceptual correlate. Vision Research, 27, 2109-2118.
MICHAEL BACH and THOMAS MEIGEN Treisman, A. (1985). Preattentive
processing in vision. Computer
Vision, Graphics, and Image Processing, 31, 156-177.
Victor, J. D. (1985). Complex visual textures as a tool for studying the VEP. Vision Research, 25, 1811-1827.
Acknowledgements-This
work was supported by the Deutsche Forschungsgemeinschaft (SFB 325, B3). We thank St. Waltenspiel for help in programming, H. Ch. Nothdurft and M. W. Greenlee for critical comments on the manuscript, our subjects for their patience and Ch. Schlier for supporting interdisciplinary research.
Vision Res. Vol. 32, No. 3, pp. 417424, 1992 Printed in Great Britain. All rights reserved
Electrophysiological Correlates of Texture Segregation in the Human Visual Evoked Potential* MICHAEL
BACH,t$
THOMAS
MEIGENt
Received 5 November 1990; in revisedform 5 April 1991
We investigated whether the visual evoked potential (VEP) reflects cortical processing associated with preattentive texture segregation. On a visual display unit we presented stimuli with various arrangements of oriented line segments that either led to the appearance of a ‘preattentive” checkerboard or did not. Two presentation modes were used (pattern onset at 1 Hz and rapid pattern change at 4.3 Hz), while luminance (57cd/m2) and contrast (92%) of the line segments remained constant. VEPs were recorded in 7 human subjects. The VEP was analyzed as a linear combination of putative components, which are evoked by either local pattern, quasi-local orientation contrast or global preattentive structure. In the transient VEP, we found a negativity over the posterior pole at a latency between 161 and 225msec (FWHM) in the linear combination designed to extract segregationspecljic components. Peak amplitude reached 3.1 f 0.8 p V (mean _+ SEM) at 199 msec. This negative peak appeared only for textures containing orientation contrast. Steady-state analysis of the rapid presentation also revealed a sign$cant component (P = 0.002) associated with texture segregation. These potentials either represent processing of orientation contrast or global processing of texture segregation. The results suggest that specljic surface potentials, diflering from cognitive potentials, can be derived which are associated with preattentive processing. Texture segregation
Visual evoked potentials
Orientation
contrast
Texton
Feature analysis
Human
appeared or did not appear. We hypothesized that the activity of any component related to texture segregation would be superimposed on the “normal” VEP. Using appropriate linear combinations of the responses to the various stimuli, these different responses could then be isolated.
INTRODUCTION Preattentive processing of textures can lead to perceptual “pop out” or “segregation” of global elements (Beck, 1972, 1983; Julesz & Bergen, 1983; Treisman, 1985; Julesz, 1986). Segregation occurs if gradients of certain local features, sometimes called textons, are present. We investigated whether cortical processing, associated with texture segregation, can be studied using the visual evoked potential (VEP) in human subjects. Orientation gradients provide very robust texture segregation (Nothdurft, 1990). For this reason we chose orientation contrast among the many possible local features. Presentation of any visual suprathreshold stimulus (e.g. checkerboards, gratings, or line drawings of faces) evokes an associated, though very small, scalp potential. By means of multiple presentation and averaging, this component can be separated from ongoing brain activity. We presented various stimuli in which the number of line segments stayed constant, but orientation was arranged such that a “preattentive checkerboard”
METHODS Subjects
Seven visually normal observers served as subjects. They wore appropriate refraction if necessary. Acuity was 2 1.2; age ranged from 23 to 69 years. The subjects gave their informed consent to participate in the experiment. Four of the 7 subjects were naive as to the specific aim of the experiment. Stimuli
Stimuli were presented on a visual display unit (AEG-HCM38) with a resolution of 832 x 832 pixels. The graphics controller (Miro-510) was programmed for pixels of square size, resulting in a line frequency df 64 kHz and a frame rate of 69 Hz. By tilting both the VDU and the line segments by 45”, giving the overall appearance of horizontal and vertical lines, we eliminated artifactal luminance differences. Such luminance
*A preliminary report was presented at the ARVO 1990. tElektrophysiologisches Labor, UniversiWs-Augenklinik, Killianstr. 5, D-7800 Freiburg, Fed. Rep. Germany. STo whom all correspondence should be addressed. 417
418
MICHAEL
0’
0
BACH
and THOMAS
---Radius of stimulus area [“I
FIGURE 1. Effect of stimulus extent. The ordinate plots the steadystate response to alternation between differently oriented line segments [(b) and (c) of Fig. 2 were displayed with a variable aperture]. On the abscissa the radius of the pattern varies between 0.7 and 4.3’. Individual data and mean are shown for 3 subjects. At eccentricities above z4”, response to these stimuli does not increase appreciably.
artifacts occur with raster scan VDUs, as lines along the scan direction are brighter than lines perpendicular to the scan direction. Two temporal presentation modes were used: in experiments (1) and (2), patterns repetitively appeared for 300 msec, followed by a blank screen for 700 msec. This evoked one pattern-onset response per second. In experiment (3) steady-state “pattern change” (alternation between two different patterns) was presented at a rate of 8.6 pattern changes per second. The sweep time of 465 msec contained 4 steady-state responses. The luminance of the line segments was 57 cd/m*, that of the background 2.4 cd/m*, resulting in a contrast of 92%. Space averaged luminance stayed constant at 3.0 cd/m2 in experiments (1) and (3) and at 2.7cd/m2 in experiment (2). Control measurements to determine appropriate$eld
size
To determine the maximal useful eccentricity where orientation contrast would evoke appreciable responses, we performed the following initial experiment: steadystate responses to homogenous texture stimuli were recorded. The radius of the circular stimulus field varied from 1.4” up to 4.2”. The orientation of the line segments changed by 90” 8.6 times per second. Figure 1 shows that the responses saturated at ~4” of eccentricity. Based on these results, we placed the display at a distance of 114 cm and surrounded it by a circular mask, leaving a stimulus radius of 5”. We then chose subjectively the specific values for the final stimulus patterns as a tradeoff between the aim to introduce many
MEIGEN
orientation contrast borders (assuming that many such borders increase associated VEP components) and sufficient definition of each individual check. The final stimuli (Fig. 2) consisted of line segments of 0.2” length (0.1 o in experiment 2). The line segments were placed in a regular lattice, with a distance of 0.4” between the lines. Their individual position varied randomly up to + 0.05”, introducing a spatial “‘jitter” to avoid regular structures at borders. For the “preattentive checkerboard” [Fig. 2(a)], orientation differed by 90”, subdividing the lattice into 4 x 4 cells. Recording
VEP was recorded from an Oz-Fz derivation using gold-cup electrodes. Signals were amplified and filtered (first-order bandpass, 0.3-70 Hz, Toennies “Physiologic Amplifier”) and digitized to a resolution of 12 bits at a sampling interval of 2.92 msec with a small laboratory computer (386 AT-compatible). The computer averaged the sweeps if their amplitude did not exceed f 100 pV, displayed them on-line and simultaneously generated the stimuli. Procedure
At the beginning of each session we measured VEP responses to luminance-contrast checkerboard patterns as used in the clinical routine. Experiments (1) and (2) were performed in the same session. We presented the various stimuli in an interleaved block design: each stimulus appeared 10 times, then the next stimulus followed. This sequence, comprising the 3 stimuli for experiment (l), was repeated twice, followed by the stimuli for experiment (2). This cycle was repeated 5 times, resulting in a total of 100 sweeps for each condition. The entire recording session lasted about two hours. A similar procedure was used for experiment (3). As behavioral task, the subjects fixated the center of the screen and reported random digits which appeared there for 300 msec in random intervals between 2 and 10 sec. Data analysis
We based our data analysis on the assumption that the VEP is an aggregate consisting of the local patternonset response plus hypothetical responses related to preattentive texture segregation. The latter may either
VEP CORRELATES
OF TEXTURE
419
SEGREGATION
Subject DB
“Segregation-specific”
Amplitude
Time [ms] FIGURE 3. Experiment (I), transient pattern-onset stimulation, subject DB. On the left, the stimuli are symbolized. In the center, raw responses {a}, {b} and {c} to these stimuli are depicted. On the right, a linear combination of the raw responses is plotted, the “segregation-specific” response: (d) = {a) - (fb) + {c))/2. For interpretation see Fig. 4.
reflect “quasi-1ocai” processing of orientation contrast between neighboring line elements, or more global and complex mechanisms. To separate the local responses from those specific to texture segregation, certain linear combinations of the responses were calculated (see Results). Experiments (1) and (2) used transient stimulation. To estimate the amplitudes of peak responses,
7 subjects
,$$ ;,,.
- , “&‘“:.* .;fl
texture
-gray+ ‘.9t c
baseline was defined as the mean between 0 and 60 msec. For analysis of differences in shape between tracings, d.c.-offsets were removed by subtracting baseline offset from each individual trace. The steady-state responses in experiment (3) were subjected to Fourier analysis. As a sweep contained exactly four responses, spill-over artifacts could not occur and no data-windowing was
“Segregation-specific” I
,
I ‘3t-----_i
, Amplitude
_..
bl
{cl} = {a} - z
I-
{c)
Jime [ms]
-lOti
I....
100
. .
. ..I.........I.........
200
300
I
400
Time [ms] FIGURE 4. Experiment (I), grand mean of 7 subjects, arrangement as in Fig. 3. The raw traces seem to differ little whether preattentive segregation is present {a] or not ({b), (cl}. Th e Iinear combination (d), however, shows a marked negativity between Nil-225 msec (FWHM), the “segregation-specific” response.
MICHAEL
420
BACH
and THOMAS
Figure 4 presents the grand mean from all 7 subjects. Traces are arranged as in Fig. 3. The “segregationspecific” response is a negativity which appears between 161 and 225 msec (FWHM). With reference to baseline, it had a peak amplitude of 3.1 + 0.8 PV (mean f SEM) at 199 msec.
necessary. Thus, the spectrum consists of discrete frequency components. Both amplitude and phase of the response at the presentation frequency (If) were extracted for further analysis.
RESULTS
Experiment (2): variation in line -segment length
Experiment (1): transient pattern -onset stimulation
Texton size and distance are known to influence strength of texture segregation (Nothdurft, 1985; Sagi & Julesz, 1987). In experiment (2), we reduced the length of the line segments by a factor of 2, leaving the inter-line distances constant. All subjects described the resulting “preattentive checkerboard” as being less pronounced. In Fig. 5, the responses to these stimuli are compared to the ones from experiment (1) (thick line, long line segments; thin line, short line segments; thin stepwise line, P-values of the difference). The patternonset responses to short and to long lines are very similar. The t-test for difference showed only spurious peaks, all with P Z 0.01 without Bonferroni correction (Fig. 5, top). The linear combination to extract the segregation-specific component shows a negativity around 190msec (Fig. 5, bottom). It was present for both short and long lines; for short line segments, however, the latency increased by 20 msec. The t-test showed that the difference between the two traces
Figure 3 presents VEP traces to pattern-onset stimulation of three different patterns {a}, {b) and {c} from a single subject, and a linear combination {d) of these responses. Stimuli were “preattentive orientation checkerboard” {a], a pattern consisting of vertical line segments {b}, and a pattern consisting of horizontal line segments {cl. Responses differed little between conditions {b} and {c}. As half of the area in stimulus (a} consisted of stimulus (b} and the other half of stimulus {cl, we sought to extract the “segregation-specific” responses by calculating the linear combination: (d}
=
{a} _
(b)_ (c), 2
MEIGEN
2
The resulting trace (d} shows a negativity between 161 and 206 msec (full width at half magnitude, FWHM), with a peak amplitude of 7.7 f 1.1 PV (mean + SEM) at a latency of 187 msec.
I
\
/
Amplitude 5-
-5 0
!PY
[kV]
,““,‘,‘,“‘.,.‘..,,.......,(.....,... 100
-P=.OOl -P=.Ol -P=.05
200 Time [ms]
300
41 0
FIGURE 5. Experiment (2), variation in line-segment length, grand mean of 7 subjects. On the left, the stimuli and the linear combination to extract the “segregation-specific” response are shown in pictorial form. Two line lengths differing by a factor of two were used. The resulting responses appear on the right. Thick trace, long line segments; thin trace, short line segments; thin stepwise line, P-value of their difference. The normal pattern responses to the two line lengths are very similar (top right). The “segregation-specific” response using long line segments (with strong perceptual segregation) has 20 msec shorter latency as compared to short line segments (with weak segregation), but roughly equal amplitude.
VEP CORRELATES
OF TEXTURE
was significant between 160 and 170 msec after pattern onset. Experiment (3): steady-state stimulation Steady-state responses to rapid “pattern-change” stimulation were also measured. Each stimulus consisted of a pair of two patterns, which alternated at 4.3 Hz. Fourier analysis separated the responses occurring on every pattern change (at 2f = 8.6 Hz) from the responses specific for the difference of patterns (at lf = 4.3 Hz) in the frequency domain. The specific stimuli used are depicted symbolically in Fig. 6, their properties, predicted responses and actual responses were as follows:
14 Stimulus
composition: all local features change, both patterns have identical preattentive segregation. Prediction: local pattern response at 2f, “segregation reversal” at 2f (which was not further
q 1
I--
l
I
--
--
--I
-
--
I
1
l
I
SEGREGATION
analyzed), no response at lf (Control’). Result: strong response at 2f, low Control’ response at lf Stimulus composition: all local features change, both patterns have no preattentive segregation. Prediction: mixture, used for linear combination {f} below. Stimulus composition: half of the local features change (from vertical to horizontal), segregation appears on the first of the two patterns. Prediction: mixture, used for linear combinations below. Partial inversion of {c}, segregation appears on the second of the two patterns. Linear combination, algebraic difference of {c} and {d}. Stimulus composition: local features cancel, segregation onset is inversely combined with segregation offset. Prediction: “segregation-specific” component should appear at lf Result: strong response at lj
I
-I
I I I l--
421
I
-
Linear combination {el = {cl - (4
Linear combination (9 = {al + {bl - {cl - (4
Time [ms]
Frequency [Hz]
FIGURE 6. Experiment (3). Rapid “pattern-change” stimulation, subject TM. On the left are shown the stimuli {a}-(d), or the linear combination rules. The resulting traces appear in the center. Their frequency spectrum is depicted on the right. Three types of responses are used for further analysis: two control conditions (Control’ and Control”, O), and the “segregationspecific” component (*). For explanation see text.
422
MICHAEL
0 Control’
BACH
and THOMAS
v Control”
6 Subjects
Po_*
= ,002
Grand Mean
FIGURE 7. Responses from experiment (3), plotted in the complex plane. The distance from the origin indicates the amplitude, the angular position indicates the phase of the response. The circles (0) represent the Control’ and Control” responses ({a} and {f} in Fig. 6), and the asterisks (*) represent the “segregation-specific” component ({e} in Fig. 6). (a) Data from Fig. 6, subject TM; (b) 6 subjects; (c) grand mean of 6 subjects. Both control conditions (0) cluster around zero. The “segregation-specific” component (*) differs significantly from the control conditions (P = 0.002).
f} Linear combination {a> + {b} - {c} - {d} to yield a further control condition. Stimulus composition: on the average, all local features and segregation cues cancel (note that the same four patterns making up (a} and {b} also appear in (c} and {d}, though in a different combination). Prediction: no response should appear at If (Control”). Result: very low response Control” at 1s The magnitude spectrum in Fig. 6 omits the phase. A polar plot, or Bode diagram [Fig. 7(a)], shows both
MEIGEN
amplitude and phase of the responses (Victor, 1985). The circles (0) represent Control’ and Control” responses ((a} and {f} in Fig. 6), and the asterisk (*) represents the segregation-specific component ((e} in Fig. 6). Figure 7(b) compiles the results from all 6 subjects, while Fig. 7(c) shows their grand mean. In Fig. 7(b), it can be seen that the Control’ and Control” responses cluster around zero, the mean amplitude was 0.07 PV resp. 0.11 pV. The segregation-specific component has a mean amplitude of 1.56 pV. For determination of statistical significance, we performed a multiple analysis of variance (MANOVA) on these results. Initially, the two control conditions (Control’ {a), Control” {f}) and the sine- and cosinecomponents of the complex response were entered as factors (2 x 2 MANOVA). No significant difference resulted between Control’ and Control” (P = 0.88). Consequently, these two conditions were collapsed into one (Control = Control’ u Control”). Against this control we then tested the “segregation-specific” response (e> and found a highly significant difference (P = 0.002). Our subjects reported an interesting additional observation: if one of the two rapidly interchanging patterns contained global segregation, and the other did not ({c) or {d) in Fig. 6), global segregation was perceived as continuously present. Varying the presentation rate, we found that at 2 Hz and below segregation onset and offset was perceived, while at 3 Hz and above it appeared stationary, although the 90”-rotation of the individual line segments was still clearly visible.
DISCUSSION
If VEP responses to stimuli with and without preattentive segregation are subtracted such that only putative segregation-specific components remain, we find a significant negativity in the occipital evoked potential between 161 and 225 msec after stimulus onset. Extraction of these components assumes linearity of the system. The specific results found in experiment (2) suggest that this assumption holds sufficiently to extract components associated with segregation. The latency of the “segregation-specific” component suggests that it occurs after the normal pattern onset- or pattern-reversal components (which peak around 140 msec with these stimuli) and earlier than endogenous cognitive potentials (which typically occur around 300 msec). An additional requirement for our analysis is sufficient stationarity, which is questionable in human subjects under extended recording conditions. The low responses on the control conditions suggest that either fluctuations of arousal and attention did not appreciably modulate responses associated with preattentive processing, or that interleaving of the various recording conditions has been sufficient to distribute such effects equally among the conditions. Nothdurft and Li (1985) have shown that texture segregation often arises simply from luminance differences. We used “orientation” as texton, which is not affected by such effects if sufficient spatial jitter is
VEP CORRELATES
OF TEXTURE
employed (Nothdurft, 1990). Our subjects could not detect any global features when they viewed the stimulus with strong blur. Thus we assume that the luminance distribution did not play a significant role in detecting texture borders in our experiments. The most parsimonious interpretation would then relate the “segregationspecific” response to the processing of “quasi-local” orientation contrast, subserving preattentive segregation of the stimulus patterns. As possible neural substrate for processing of orientation contrast, the neurons reported by van Essen et al. (1989) might be likely candidates. Some of the cells recorded by these authors responded well to orientation contrast, but not to homogenous textures. This suggests that orientation contrast is processed directly and not extracted after detecting orientation per se. In our experiments, the “segregationspecific” component had a higher latency (Z 190 msec) compared to the “normal” pattern response (x 140 msec with these stimuli). It would be interesting to analyze latencies in van Essen et al.‘s (1989) data. Can the negativity at x 190 msec be related to the “mismatch negativity” (Naatanen, 1990), which occurs on any different stimulus, regardless of its task relevance (Paavilainen et al., 1987)? The “segregation-specific” component was extracted by calculating differences between the responses to various stimuli. As all of the three stimulus types in experiments (1) and (2) were “different” to their predecessor, a mismatch negativity would occur in all responses likewise and be cancelled when taking differences. We believe that the “segregation-specific” component does not reflect cognitive processing for the following reasons: first, the behavioral task directed the subjects’ attention towards the fixation point. Our subjects reported that after a number of trials they often no longer consciously registered the segregation content of the screen. Second, segregation-specific responses were also found with rapid stimulation ({cl and {d}, Fig. 6), during which the “preattentive checkerboard” was perceived as continuously present. The results of experiment (2) support the interpretation of the negativity at z 190 msec as “segregationspecific” or, more specifically, as representative for processing of orientation contrast: it is well known that shortening of the line segments (effectively lowering the rate of orientation-change over distance) decreases the subjective strength of “pop-out” (Nothdurft, 1985; Sagi & Julesz, 1987). While the normal pattern response differed very little using long or short line segments (Fig. 5, top right), the linear combination to extract the “segregation-specific” component shows a marked increase in latency. The finding that at presentation rates of 3 Hz and above the segregation is perceived as stationary, while a segregation-specific VEP could be recorded at 4.3 Hz, seems to argue against our interpretation. However, Skrandies (1987) reported a closely related finding: if dynamic random-dot stereo-stimuli are presented as rapid pattern-onset, the stereo target appears perceptually fused above 3.15 Hz, while associated VEP com-
SEGREGATION
423
ponents are recordable up to 10 Hz. We hypothesize that some low-pass mechanism, possibly related to visual persistence (Coltheart, 1980), is placed after the mechanisms which process orientation contrast in the visual processing chain. We are not aware of other experiments concerned with preattentive texture segregation in visually evoked potentials. Closely related, however, are Victor’s (1985) experiments using textures with odd-even statistics. The current results suggest that the VEP contains not only “global aspects of form” but also the activity of mechanisms related to the processing of texture segregation induced by orientation contrast. Jeffreys (1989) reported another interesting activity of higher-order processing in the VEP: a midline central and a parietal positivity in the latency range of 15&200 msec after presentation of face stimuli. As we have only used a single bipolar derivation so far, we will have to take multichannel topographic recordings to study relations to such responses. The results suggest that the VEP may be a useful tool to further our understanding of preattentive processing in vision by linking psychophysical and electrophysiological observations in human subjects.
REFERENCES Beck, J. (1972). Similarity grouping and peripheral discriminability under uncertainty. American Journal of Psychology, 85, 1-19. Beck, J. (1983). Textural segmentation, second-order statistics, and textural elements. Biological Cybernetics, 48, 125-130. Coltheart, M. (1980). The persistences of vision. Philosophical Transactions of the Royal Society B, 290, 57-69.
van Essen, D. C., de Yoe, E. A., Olavarria, J. F., Knierim, J. J., Fox, J. M., Sagi, D. & Julesz, B. (1989). Neural responses to static and moving texture patterns in visual cortex of the macaque monkey. In Lom, D. M. K. & Gilbert, C. D. (Eds), Neural mechanisms of visual perception (pp. 137-154). Woodlands, Tex.: Portfolio. Jeffreys, D. A. (1989). A face-responsive potential recorded from the human scalp. Experimental Brain Research, 78, 193-202. Julesz, B. (1986). Texton gradients: The texton theory revisited. Biological Cybernetics, 54, 245-25 1. Julesz, B. & Bergen, J. R. (1983). Textons, the fundamental elements in preattentive vision and perception of textures. The Bell System Technical Journal, 62, 1619-1645.
NZitlnen, R. (1990). The role of attention in auditory information processing as revealed by event-related potentials and other brain measures of cognitive function. Behavioral Brain Sciences, 13, 201-288. Nothdurft, H. C. (1985). Sensitivity for structure gradient in texture discrimination tasks. Vision Research, 25, 1957-1968. Nothdurft, H. C. (1990). Texton segregation by associated differences in global and local luminance distribution. Proceedings of the Royal Society London B, 239, 295-320.
Nothdurft, H. C. & Li, C. Y. (1985). Texture discrimination: Representation of orientation and luminance differences in cells of the cat striate cortex. Vision Research, 25, 99-113. Paavilainen, P., Alho, K., Reinikainen, K., Sams, M. & Niilmnen, R. (1987). Attention independence of the mismatch negativity of the human event-related potential (ERP). Neuroscience (Suppl.), 22, S759 (2271P).
Sagi, D. & Julesz, B. (1987). Short-range limitation on detection of feature differences. Spatial Vision, 2, 39-49. Skrandies, W. (1987). Visual persistence of stereoscopic stimuli: Electric brain activity without perceptual correlate. Vision Research, 27, 2109-2118.
MICHAEL BACH and THOMAS MEIGEN Treisman, A. (1985). Preattentive
processing in vision. Computer
Vision, Graphics, and Image Processing, 31, 156-177.
Victor, J. D. (1985). Complex visual textures as a tool for studying the VEP. Vision Research, 25, 1811-1827.
Acknowledgements-This
work was supported by the Deutsche Forschungsgemeinschaft (SFB 325, B3). We thank St. Waltenspiel for help in programming, H. Ch. Nothdurft and M. W. Greenlee for critical comments on the manuscript, our subjects for their patience and Ch. Schlier for supporting interdisciplinary research.
