Vol. 9, No. 10/October 1992/J. Opt. Soc. Am. A
H. R. Wilson and W A. Richards
1653
Curvature and separation discrimination at texture boundaries Hugh R. Wilson Visual Sciences Center, University of Chicago, 939 East 57th Street, Chicago, Illinois 60637 Whitman A. Richards Department of Brain and Cognitive Science, Massachusetts Institute of Technology, Cambridge,Massachusetts 02139 Received February 28, 1991; revised manuscript received May 19, 1992; accepted May 19, 1992 Visual discrimination of contour curvature was investigated by using contours defined by the locus of points at which the phase of a square-wave grating was shifted by 180° (a texture boundary). Curvature-increment thresholds were measured for contour curvatures from 0.31 to 10.65 deg-', for grating spatial frequencies of 4.0 and 16.0 cycles per degree (cpd), and for gratings in either sine or cosine phase at the point of maximum curvature. Thresholds for these conditions were compared with curvature discrimination at black-white edges. Grating phase had no effect on performance at any curvature or grating frequency, but 16.0-cpd gratings produced a threshold elevation at all curvatures by an average factor of 2.4. Two-line separation discrimination was also measured for lines defined by texture boundaries. These data can be predicted by a model incorporating end-stopped complex cells of a type reported physiologically in primate area V2.
INTRODUCTION The retinal images of most visual objects are bounded by simple closed contours. Because these contours usually vary in curvature, curvature encoding must play an important role in the visual representation of these objects. Research on curvature processing has focused almost exclusively on discrimination of the curvature of simple line stimuli,"' 0 and it has been shown that these discrimination data can be predicted by the differential responses of orientation-selective visual mechanisms. 4' 5 However, curved contours can be generated by complex texture boundaries as well as by luminance edges. For example, contours are clearly visible between regions having different contrasts, spatial-frequency spectra, orientation ranges, or phases. Texture differences giving rise to effortless visual texture segregation have been intensively studied by Julesz and Bergen." However, no attempt has been made to determine the accuracy with which the shape of texture boundaries is represented in the visual system. This paper focuses on three key questions concerning the encoding of texture boundaries. First, can the visual system discriminate texture-boundary curvature as accurately as it can discriminate simple line or edge curvature? Second, is two-line separation discrimination as accurate for texture lines as it is for contrast-defined lines? Third, can the same orientation-selective mechanisms predict curvature and separation discrimination for both contrast and texture boundaries? To answer these questions it was necessary to choose a class of stimuli in which the texture boundary always contained the same amount of information as a curved black-white edge; this ensured that any deficits in performance would be due to limitations of visual processing rather than to 0740-3232/92/101653-10$05.00
reduced stimulus information content. We have chosen to use patterns in which the texture boundary is defined by the locus of a 1800 phase shift in a square-wave grating (see Figs. 1 and 2). In the limiting case of a low grating frequency the stimulus becomes a black-white edge. At higher spatial frequencies the 1800 phase-shift locus becomes a boundary between one-dimensional, periodic textures. In all cases, however, the same amount of information defines the contour, because there are the same number of black-white transitions along the curved contour regardless of the spatial frequency of the squarewave grating. Curvature-increment thresholds were measured as a function of the spatial frequency of the square-wave grating. Relative to discrimination thresholds obtained with black-white edges, curved contours defined by a 1800 phase shift in a square-wave grating of 16.0 cycles per degree (cpd) caused threshold elevations of a factor averaging 2.4 at all curvatures tested. A quantitative model for curvature discrimination that has been successfully applied to curved lines4 5 accurately predicted data obtained with curved edges, but it predicted far too great a threshold elevation for the phase-shift contours in 16.0-cpd gratings. However, this model employed only mechanisms with spatial filters similar to those of cortical simple cells. Because recent work has suggested that more complex, nonlinear filtering is necessary to explain aspects of texture perception, 2 3 mechanisms of this type were employed to model our data. A satisfactory fit was achieved by using mechanisms with properties similar to those of the end-stopped complex cells found in area 18 of macaque visual cortex. 4 5 Thus our results indicate that texture boundaries cannot be encoded with as much accuracy as can simple luminance edges, even when the same amount of information is present in the stimulus, and that differ© 1992 Optical Society of America
1654
J. Opt. Soc. Am. A/Vol. 9, No. 10/October 1992
H. R. Wilson and W A. Richards
two different contour curvatures and two different grating spatial frequencies, whereas Fig. 2 illustrates pairs of texture bars used for two-line separation discrimination. In all cases the vertical square-wave grating was at 100% contrast. The curved contours used in this study consisted of a parabola grafted to a pair of straight lines, usually oriented at ±45.00, so that the first derivative would be continuous. In terms of the horizontal (x) and vertical (y) coordinates, the locus of the contour was thus defined by the equation
(a)
-AX2 +
A
I+-Ixl+A
for Ix I2 forIxl>j2
y
(b)
(c)
Fig. 1. Stimulus patterns having textures with (a), (b) equal spatial frequencies and (b), (c) equal contour curvatures.
(a)
(b)
(d)
(c)
III I I I I I CI
Fig. 2. Pairs of (a), (b) luminance bars and (c), (d) texture bars used in two-line separation discrimination experiments.
ent mechanisms are involved in processing these two types of contour.
METHODS The texture boundaries used in this study were defined by the locus of a 180° phase shift in a vertical square-wave grating. Figure 1 illustrates such texture boundaries for
(1)
12
As discussed below, these stimuli have a unique curvature extremum at x = 0, where the curvature is equal to 2A. This extreme curvature will be used to describe each stimulus. In addition, each contour has an identical range of orientations of ±45.00, so that only curvature can provide a cue for discrimination. To prevent the absolute position of the pattern on the screen from providing a spurious cue for discrimination, the vertical location of each pattern was varied by adding a random displacement A from trial to trial. Values of A were uniformly distributed through ±10% of the display height. Because of limitations of display size, experiments with low curvatures used an orientation range of ±26.6 or 14.00, but all stimuli in a given experiment had the same orientation range. The general formula for the curvature K at any point along a contour defined by the equation y = f(x) may be shown to be
K =
d2y/dx 2
[1 + (dy/dx)2 ]3 2
(2)
Along the parabolic contour used in these experiments, therefore, the curvature becomes K-[1
2A 2 2 + (2Ax) ]3/
(3)
From this it follows that the maximum curvature occurs at x = 0, where K = 2A. Stimuli were presented on the face of an Apple Macintosh computer, which the subject viewed by using a front surface mirror located across the room; this placed the display at an optical distance of 10.0 m. At this distance screen dimensions were 1.01° by 0.67° and individual pixels subtended 7.09 arcsec. The mirror was surrounded by a cardboard mask that was illuminated at the same mean intensity as was the display (30.0 cd/M2). The subject initiated each trial by pressing the button on the Macintosh mouse. The two-interval forced-choice presentation that followed comprised two 0.75-s stimulus presentations separated by a 0.50-s interval at the mean luminance. The subject rolled the mouse left or right to indicate whether the first or the second interval contained the more sharply curved contour. As illustrated in Fig. 1, the height of the curves on the display was randomized so that distance to the top or bottom could not be used as a cue for discrimination.
In separate experiments increment thresholds were measured for base curvatures of 0.31, 0.61, 1.42, 3.55, and 10.65 deg-'. Square-wave grating frequencies of 1.0, 4.0, and 16.0 cpd were used to define the curved contours. In addition, each grating was set at either cosine or sine phase relative to the point of maximum curvature. In cosine phase the point of maximum curvature occurred at the middle of a bar, whereas the curvature maximum fell at the edge of a grating bar when sine phase was used [see Fig. 1(c)]. In separate experiments discrimination thresholds were measured for all combinations of curvature, grating spatial frequency, and relative grating phase. Two-line separation discrimination was measured by using horizontal bars 11.33 arcmin long by 1.9 arcmin wide. Different experiments measured discrimination thresholds for base separations of 2.36, 3.78, 7.56, and 11.34 arcmin. As shown in Fig. 2, the patterns were either simple black bars on a white background (contrast bars) or bars defined by the locus of a 1800 phase shift in a 16.0-cpd square-wave grating (texture bars). The same bar dimensions and separations were used for both contrast and texture bars. Stimuli for a single experiment consisted of a contour of standard curvature plus four test contours with curvatures progressively greater than the standard. A method of constant stimuli was used in which the Macintosh randomly selected the interval containing the standard curvature and also randomly selected the test curvature for the other interval. Each test curvature was presented in 25 trials (a total of 100 trials for each experiment), and the resulting data giving percentage correct discrimination versus curvature increment were fit with a Quick' psychometric function by using a maximum-likelihood estimation procedure. Reported discrimination thresholds correspond to the 75% point on the fitted function. The two authors (HRW and WAR) served as subjects.
RESULTS Curvature Discrimination The goal of these experiments was to determine the accuracy of curvature discrimination at texture boundaries. Therefore curvature discrimination was first measured by using a black-white curved edge to provide a baseline for subsequent texture measurements. Thus the stimuli were the same as those in Fig. 1, except that they were black below the curved contour and white above the contour. Data for two subjects for a curvature range of 0.31-10.65 deg-' are shown by filled circles in Fig. 3. Thresholds are roughly constant at approximately 0.05 deg-' for curvatures below 1.0 deg-'. At higher curvatures thresholds rise roughly linearly on double logarithmic coordinates. A least-mean-squares curve fit determined the slope of this straight-line portion to be 0.93 for HRW and 1.05 for WAR, thus indicating that curvature discrimination approximately obeys Weber's law in this region. The Weber fraction averaged 6.6%, which further supports the finding that curvature discrimination falls into the hyperacuity range.6 The general features of black-white edge-curvature discrimination are the same as those for line-curvature discrimination, and the increment thresholds are quantitatively similar.4' 5
1655
Vol. 9, No. 10/October 1992/J. Opt. Soc. Am. A
H. R. Wilson and W A. Richards
Curvature discrimination at texture boundaries was next measured by using curves defined by the locus of a 1800 phase shift in a vertical square-wave grating (see Fig. 1). The open circles in Fig. 3 show curvature thresholds for a contour in sine phase on a 1.0-cpd squarewave grating. Note that these thresholds did not differ significantly from thresholds for a black-white edge, which is in cosine phase relative to the point of maximum curvature. Similarly, data for grating spatial frequencies of 4.0 and 16.0 cpd (see Fig. 4) showed no significant difference between sine and cosine phase thresholds for either subject. A three-factor analysis of variance was run 10 0)
.*
'a)
O .
HRW Edges HRW sin Theoryi
I
~0 U)
.1
'op~~c
E C)
. I .
... . . . ..... I
.
........
0.01 0. .1
IC10
10
1
Curvature (deg-') 10 * O 0
-
U)
WAR Edges WAR sin Theory
.1
I-
a)
E 0.01 L 0.1
.
.
.
. . .1 .
.
.
.
1 ..
.
.
1
.
.
.
10
1 00
1
0) a)
Curvature (deg- ) Fig. 3. Edge-curvature discrimination measurements.
10 O *
la a)
a) C,
Sine phase Cosine phase
o
1
0S Edge Thresholds
0.1
C I
0.01
grating. curves HRW, ,6cpd I .. . . .I
'
.
100 10 1 Curvature (deg-') Fig. 4. Sine and cosine phase threshold versus curvature for 16-cpd grating curves. The subject is HRW 0.1
1656
J. Opt. Soc. Am. A/Vol. 9, No. 10/October 1992
H. R. Wilson and W A. Richards
Table 1. Effect of Square-Wave Grating Phase on Curvature Discrimination at Texture Boundaries Subject
Frequency (cpd)
t (df = 4)a
Significance p
WAR
1.0
0.220
>0.83
WAR
4.0
1.325
>0.24
WAR
16.0
1.298
>0.25
HRW
1.0
0.538
>0.61
HRW
4.0
0.983
>0.38
HRW
16.0
1.028
>0.36
'For each subject at each grating spatial frequency, a two-tailed t test was performed to compare discrimination thresholds across curvatures for sine and cosine phase conditions (paired measures). As is evident from the table, thresholds did not differ significantly between sine and cosine phase conditions.
on the data for each subject by using the variables of curvature, spatial frequency, and phase. Although curvature and spatial frequency were found to be significant at the 0.001 level, phase (along with all pairwise and three-way interactions involving phase) was insignificant (p > 0.35 for each subject). For each subject we also performed a t test for each grating frequency to assess the significance of any difference between curvature thresholds for gratings in sine and cosine phase (paired measures). As is indicated in Table 1, there was no significant effect of relative grating phase on curvature discrimination for either subject at any spatial frequency. This phase independence has important theoretical implications, as discussed below. Curvature-discrimination thresholds measured at texture boundaries in 16.0-cpd square-wave gratings [see Figs. 1(a) and 1(b)] were elevated significantly relative to thresholds at contrast edges, as is apparent from the data for HRW plotted in Fig. 4. Relative to the solid curve, which shows edge thresholds from Fig. 3, curvature thresholds at the 16.0-cpd texture boundary (circles) were higher at all base curvatures. As noted above, thresholds obtained with the grating in sine phase (open circles) did not differ significantly from those for the cosine phase condition (filled circles). Therefore sine and cosine phase data are averaged in all subsequent plots. Threshold elevations are defined as the ratio of curvature-increment threshold at the 16-cpd texture boundary to the threshold for a contrast edge. A threshold elevation of unity would indicate that curvature discrimination at the texture boundary was as accurate as discrimination at a contrast boundary. As is shown in Fig. 5, however, all threshold elevations were greater than unity, thus indicating a degradation of performance with the curved texture boundary. A power-law fit to the data for each subject revealed a very slight trend toward increasing threshold elevations with increasing curvature (the power-law exponent averaged 0.11 for both subjects). Because this is virtually flat, however, threshold elevations were averaged across curvatures, producing mean elevations of 2.13 for HRW and 2.72 for WAR. Although threshold elevations for curvature discrimination at texture boundaries were certainly expected, the fact that they were relatively small has important theoretical implications (see below). Curvature threshold elevations were also measured at texture boundaries defined by the locus of a 1800 phase shift in 1.0- and 4.0-cpd square waves. The pattern of
threshold elevations for 4.0-cpd gratings was qualitatively similar to that of the 16.0-cpd data in Fig. 4, except that threshold elevations were significantly smaller, averaging 1.63 for HRW and 1.44 for WAR. As is shown by the open circles in Fig. 3, thresholds were not elevated for the 1.0-cpd gratings. Two-Bar Separation Discrimination To characterize further the encoding of spatial information at texture boundaries, we also measured thresholds for a more common hyperacuity task. This was two-line separation discrimination measured as a function of separation (see stimuli in Fig. 2). The texture bars in this experiment were defined by the locus of a 1800 phase shift in a vertical 16.0-cpd square-wave grating. The bars subtended a height of 1.9 arcmin and a length of 11.33 arcmin. Psychometric functions for separation discrimination at a base separation of 2.36 arcmin are plotted for one subject in Fig. 6. It is evident that the 75%correct point occurs at a much larger separation for the texture-defined bars (solid circles and solid curve) than for the contrast bars (open circles and dashed curve). However, this graph also reveals a problem in estimating the 75%-correct value for the contrast lines: the smallest available separation increment in our experiments was 30 |
HRW|
0O-WARI
C 0
10 a)
Simple Cel Model
Texture Model
P
co
:* .
0.1,:2
.
-r
.
.
.
.
.
.
1
,,,,,..ll
.
.
.
.
.
.
.
.
.
10
.
20
Curvature (deg- 1 ) Fig. 5. Threshold elevation versus curvature for the 16-cpd texture curve.
1.0 4-
a L-. 0
0
0.9 0.8
C 0
0.7
U-.
0.6 0.5
0
5 10 15 20 25 Increment Separation (arcsec)
30
Fig. 6. Fraction correct versus increment separation for the two-bar separation test. The subject is WAR.
RdVol. 9, No. 10/October 1992/J. Opt. Soc. Am. A
H. R. Wilson and W A. Richards
THEORY
1 00 Texture Bars
S
X
0
1657
1
-
Contrast Bars F
1
HRW Texture *+~~~~~~~~~~~4~- HRW Bars -W l WAR Texture -13- WAR Bars 10
30
Separation (arcmin) Fig. 7. Thresholds for contrast bars and texture bars for the two-bar separation discrimination test.
7.09 arcsec, yet the data in Fig. 6 show 92%-correct discrimination at this point. Clearly, the threshold must be within the range 0.0-7.09 arcsec in this case. To provide a more constrained range for the estimate, we adopted the 6 following procedure. Four separate Quick functions 1, of value fixed were fitted, with each fit using a different 2, 3, or 4 for the exponent. This covers the range of exponents reported in both detection and increment threshold experiments. The four 75%-correct points thus obtained were then averaged. The dashed curve in Fig. 6 shows the Quick function obtained with an exponent of 2, which produced the threshold estimate closest to the average, and the solid horizontal line crossing this function at the 75% point shows the range for all four estimates. This threshold-estimation procedure was necessary for both subjects only at the smallest base separation, because thresholds were significantly higher for larger separations (see Fig. 7). This problem did not arise for curvature discrimination, because every psychometric function had at least one point falling below the 75% correct level. Thresholds for the simple black bars on a white background (contrast bars) are plotted for two subjects as open symbols in Fig. 7. At the smallest separation of 2.36 arcmin, descrimination thresholds averaged 3.9 arcsec, which is well within the hyperacuity range. As separation increased, thresholds rose linearly on double logarithmic coordinates in accordance with Weber's law. These data are in agreement with those from previous studies of separation discrimination. 17 2- ' As is shown by the filled symbols in Fig. 7, separation discrimination thresholds for the texture bars were higher than those for the contrast bars at all separations tested. These data are replotted in Fig. 8 as threshold elevations for the two subjects. Threshold elevation was greatest at the smallest separation of 2.36 arcmin and decreased as separation increased. The threshold elevation at the smallest separation was 8.14 for HRW and 3.42 for WAR, and these elevations decreased to 1.63 for HRW and 1.44 for WAR at the largest separations used. Note, however, that at the smallest separation the threshold for texture bars was 13.0 arcsec for WAR and 28.5 arcsec for HRW (see Fig. 7). These thresholds are still somewhat less than the spacing of foveal cones. 22
Two key results have emerged from a comparison of curvature and separation discrimination at texture boundaries with discrimination at simple contrast boundaries. Because these differences are found primarily at the high grating frequencies, we will focus on the results obtained with 16.0-cpd square-wave gratings. First, curvature discrimination for all curvatures is poorer at texture boundaries by an average factor of 2.4. Second, separation discrimination for texture bars is poorer than for contrast bars, but the greatest degradation of performance occurs at small separations. This section raises the question of whether the texture results can be predicted by models that have been shown to predict curvature 4'5 and hyperacuity 23 data accurately for simple contrast patterns. In a study of curvature discrimination using bandpassfiltered lines as stimuli, Wilson and Richards 5 showed that an extension of a previous model for pattern discrimination 4'2 3 could accurately predict thresholds for these simple contrast-defined curves. This model was based on the responses of orientation-selective simple cells whose spatial filtering properties were derived from psychophysical masking data. 24'25 A vertically oriented model filter is described by the equation
F(x,y) = A[exp(- 2 /o-12 ) - B exp(-x 2 /0a2 2 ) + C exp(-x 2 /03 2)]exp(-y 2 /ry 2 ),
(4)
and other orientations may be obtained by simple rotation of coordinates. The curved stimulus is first convolved with these filters, and the result is then passed through a contrast nonlinearity. As determined from masking studies,2 42 62 7 this nolinearity is accelerated at low contrasts but exhibits a power-law compression at high contrasts. Thus the filter response R to a two-dimensional pattern P(x, y) is given by the equations F(x
S(x, ) =
-
x'y - y')P(x'y')dx'dy',
(5)
f.
R(SC) =
(SC)2 + K(SC)3-8 K + (SC)2
(6)
10
C 0 co
a) 0 CD
1
1
10
20
Separation (arcmin) Fig. 8. Threshold elevation versus curvature for the two-bar separation discrimination test.
1658
J. Opt. Soc. Am. A/Vol. 9, No. 10/October 1992
where C is the pattern contrast and K and sEare constants measured by masking.24 The difference in response to pairs of curved lines is then computed by pooling response differences among mechanisms tuned to all orientations and spatial frequencies, with the pooling operation defined by the Euclidean distance. Line curvature was increased until the difference in pooled response between two curves reached a threshold value of unity, which corresponds to 75% correct in the model. The pooling operation computed curvature by using one of two procedures, and the lower of the two thresholds was chosen. This first procedure, which was most sensitive to curvatures above 3.0 deg-', was entirely local, being based on the responses of units centered at the point of maximum curvature. The second, which was most sensitive at lower curvatures, was bilocal, being based on pooled responses of high-spatial-frequency filters displaced symmetrically at a fixed distance from the point of maximum curvature along the tangent to the curve. The distance of ±8.2 arcmin for the displaced units was found to provide the best fit to line curvature data.5 The use of only highspatial-frequency filters in the bilocal procedure resulted from the experimental discovery that high-frequency, bandpass filtering had no effect on discrimination at low curvatures. Details of this model were published previously. 5 The solid curves in Fig. 3 show edge-curvaturediscrimination thresholds predicted by this model, and the arrows show the point of transition between discrimination based on displaced units (bilocal discrimination) and discrimination based on units localized at the curvature maximum. The theory is obviously in good quantitative agreement with the experimental data for both subjects. Can this model predict curvature discrimination at texture boundaries in a 16.0-cpd grating? The thin line (Simple Cell Model) in Fig. 5 shows that the answer is no: this model predicts that thresholds will be elevated by an average factor of 10, whereas the data show an elevation averaging 2.4. The reason for this failure is illustrated in Fig. 9, which depicts vertically and horizontally oriented visual filters superimposed upon a 16.0-cpd grating. Because the filters are tuned to 16.0 cpd, vertical filters above and below the texture boundary will respond strongly to the stimulus. A horizontal filter tangent to the texture boundary, however, will integrate over several cycles of the grating and will therefore respond poorly if at all. Thus texture boundaries in high-spatial-frequency gratings are not encoded effectively by orientationselective units resembling cortical simple cells. How are texture boundaries processed by the visual system? An answer has been suggested by recent studies of texture discrimination. 21 3 As is shown in Fig. 9, vertically oriented simple-cell filters will respond well to the grating bars above and below the discontinuity. As the polarity of the bars switches from black to white at the discontinuity, however, a vertical filter will give zero response when centered on the discontinuity. This is illustrated in Fig. 10(a), which shows a contour plot of the vertical filter responses to a curved texture boundary. Note that there is no response in a band along the curve. If the pointwise responses are now either rectified or squared (squaring is the lowest-order term in the Taylor series for rectification), filters tuned to the horizontal
H. R. Wilson and W A. Richards
should respond well along the curve. This is illustrated in Fig. 10(b). This second-stage filter must be tuned to lower spatial frequencies to avoid responses to the individual bars of the original grating. Based on the results of an analogous study of two-dimensional motion processing, the second-stage filter has been set lower in frequency by a factor of 2.0.25 The model for texture-boundary analysis is shown schematically in Fig. 10(c). First, high-frequency filters process the stimulus, then the response is squared, and finally a second stage of linear filtering at lower spatial frequencies occurs. In the current model the second stage incorporates filters tuned to spatial frequencies 2.0, 4.0, and 8.0 times lower in frequency than are the firststage filters. These filters are also defined by Eq. (4) and thus have the same aspect ratio as that of the first-stage filters. These filters essentially perform an analysis of the texture boundary across several coarser spatial scales. Although the second filter was chosen perpendicular to the first in Fig. 10(c), the second filtering stage has a range of orientations that differ from those of the firststage filters by as much as ±450 in 15° increments. This range of second-stage orientations is necessary for pro-
Fig. 9. Filter response to texture boundaries. 0.2'
(a)U0 UU j QQ 0 V0 L C L
39.
U
I
Position
-0.2' L-
0.2 o
-0.2'
Position
0.2'
(C) +
First Filter
- 0-*
IL Response Squaring
1
+
Second Filter
Fig. 10. Contour plots of responses to curved texture boundaries by (a) vertical first-stage filter and (b) horizontal second-stage filter. (c) Schematic diagram of the model used for textureboundary analysis.
Vol. 9,No. 10/October 1992/J. Opt. Soc. Am. A
H. R. Wilson and W A. Richards
cessing texture-boundary curvatures over the range studied here. Because of this rough independence between the orientation of the texture boundary and the orientation of the texture elements defining this boundary, highorder operations that attempt to measure directly the curvature of the texture boundary, such as those proposed for contour curvature, 8 do not seem plausible. In summary, the model for pattern discrimination at texture boundaries consists of oriented, simple-cell-type filtering followed by rectification or squaring and a second stage of oriented filtering at lower spatial frequencies. Because the presence of thresholds in simple cells produces a half-wave rectification of their inputs, full-wave rectification may be easily accomplished in the visual system by pooling responses of on-center and off-center simple cells. The second stage performs pattern discrimination in essentially the same manner as does the model for curvature discrimination at luminance contours that has been described by Eqs. (4)-(6) and previously.4'5 '23
The only difference is that it operates on a
filtered and rectified neural representation rather than on the luminance distribution itself. For the stimulus used in our experiments, the computation engendered in this model can be greatly simplified. Because the first-stage visual filter in Eq. (4) is separable in Cartesian coordinates, the response of vertical filters to a horizontal texture boundary in a vertical grating (e.g., Fig. 9) is just S(x, y) -
MTF(wo) 2 f exp(-y' 2 /o.Y2 )dy'
-
uyrV7cos(2.wox). (7)
MTF(w) is the Fourier transform of the difference of the three Gaussian functions of x in Eq. (4). Note that the response involves a cosine term, because the filter in Eq. (4) is insensitive to the third and higher harmonics of the square wave. The term in brackets is just an error function minus a constant, and it ranges from -cy - 7r for y > 0. When Eq. (7) is squared, the variation in the y direction will be given by the square of the terms in brackets. Computations have shown that this expression may be well approximated by a constant minus a single Gaussian function: 2f
exp(-y'2/o.y2)dyP
=
-
VI
a2[1 - exp(-y2/0.81cry2)].
(8)
The Gaussian on the right-hand side of relation (8) was found to provide the best least-mean-squares approximation when the space constant was 0.9oy. This approximation is so good that plots of the left- and right-hand sides of relation (8) differ by less than the width of the plotting line, and thus a graphical comparison is pointless. Calculations show that the difference between the two functions is always less than 0.6% of the maximum value, 7roy . When the filtered stimulus in Eq. (7) is squared, in addition to the y variation in relation (8) there is also spatial
1659
variation resulting from the term [cos(2wrwx)] 2, which gives a constant term plus a term at frequency 2w. Because the second-stage filters are tuned to frequencies of co/2.0 or lower, however, they give no significant response to frequency 2w. Therefore predictions of the response of this model to various patterns can be calculated by treating the locus of the phase shift in the stimulus grating as a Gaussian-blurred line described by the right-hand side of relation (8). That is to say, the initial filtering and response squaring generate, to a high degree of accuracy, the Gaussian-blurred contour described by relation (8). This blurred contour is then processed by the larger (lower-frequency) oriented filters of the second stage. One final point remains in the specification of this model: the gain of the second filter stage relative to that of the first stage. To state it otherwise, the squared response of the first stage must be normalized before it is filtered by the lower-frequency filters of the second stage. This normalization was determined empirically by measuring the contrast threshold for detecting the presence of a horizontal texture boundary in a vertical 16.0-cpd square-wave grating. Average data for two subjects showed that the contrast threhold for detecting the texture boundary was 2.6%, whereas the threshold for detecting the 16.0-cpd vertical grating itself was 1.4%. The normalization of the second stage was therefore adjusted to reflect the ratio of these thresholds. We derived predictions of curvature discrimination at texture boundaries from this model and compared them with the predictions for discrimination at luminance edges (see Fig. 3) to obtain model threshold elevations. The result is shown by the heavy solid curve in Fig. 5, which is in reasonable agreement with data for the two subjects. Note that this two-stage discrimination model predicts much smaller threshold elevations than does a model based on responses of the first stage alone (the thin curve labeled Simple Cell Model in Fig. 5). One further aspect of the curvature data is readily explained by this model. As illustrated in Figs. 3 and 4 and discussed above, the relative phase of the vertical grating had no effect on curvature discrimination at texture boundaries (see Table 1). Because of the squaring of the first-stage response plus second-stage filtering at lower spatial frequencies, however, all phase information from the 16.0-cpd square-wave grating is effectively discarded, and only the locus of the texture boundary is extracted. In contrast, any model based directly on the simple-celltype filters in the first stage would incorrectly predict a phase dependence for discrimination. This model for encoding texture boundaries was also applied to the two-bar separation discrimination data of Fig. 7. In that figure the lower heavy curve is a plot of threshold predictions for simple contrast bars. These are predictions from the simple model that was also used for discrimination of edge curvature (solid curves in Fig. 3). ' and summarized above in As reported previously5 2 3 29 Eqs. (4)-(6), this model is based on the responses of only the first filtering stage. Although they lie slightly below the data of the two subjects (open symbols in Fig. 7) on average, the two-bar separation predictions of this model have previously been shown to provide an accurate fit to a wide range of data from a variety of other studies. 3 0 However, this model does not predict separation data for
1660
J. Opt. Soc. Am. A/Vol. 9, No. 10/October 1992
texture bars for the same reason that it does not predict curvature discrimination at texture boundaries. Namely, filters oriented along the texture boundaries respond poorly to the image, as is illustrated on the right-hand side of Fig. 9. As is shown by the upper solid line in Fig. 7, however, separation discrimination for texture bars is accurately predicted by the two-stage texture model developed to fit the curvature data. The ratio of the two theoretical curves gives the predicted threshold elevation, which is compared with data in Fig. 8. The fact that the model produces large threshold elevations at small separations and virtually no elevation at the largest separations is easy to explain. Relation (8) indicates that the effective stimulus for the second-stage filters is a line blurred by a Gaussian function with 0.9 times the space constant of the 16.0-cpd visual filters in the first processing stage. For these filters 0.93, = 3.28 arcmin, which is a relatively large blur at small separations but is a relatively small blur at large separations. At the second processing stage it is mainly the smaller filters that analyze small separations and larger filters that analyze larger separations. Because the blur is constant, it thus has a negligible effect at large separations. It may be noted that Levi and Westheimer 3 1 employed a variant of the two-line separation discrimination paradigm that produced results similar to those obtained with texture bars in the present study. In their study, thresholds for discriminating the separation between two dark bars were compared with separation thresholds obtained when one of the bars was dark and the other was white. The white-dark combination raised thresholds at a 2.0arcmin separation relative to the two dark bars, but this threshold elevation vanished before the separation reached 8.0 arcmin. Their data thus show the same quantitative pattern as the data in Fig. 5. Because of the rectification stage, the present model is also capable of predicting the Levi-Westheimer data. 3
DISCUSSION The experiments reported above were designed to determine how accurately the visual system can encode information at texture boundaries. In the cases of greatest interest these boundaries were defined by the locus of a 1800 phase shift in a vertical 16.0-cpd square-wave grating. Curvature discrimination at these texture boundaries was degraded relative to discrimination at contrast edges, but the degradation was only by a modest factor averaging 2.4 over all conditions. Two-bar separation discrimination thresholds were also greater for texture bars than for contrast bars, but here the elevation was large at separations below 4.0 aremin and small at greater separations (see Fig. 8). Because the texture boundaries used in this study contained precisely the same amount of information as is present in a simple contrast edge or bar, the performance degradation between texture and contrast cases is ipso facto a reflection of limitations in the visual analysis of texture. A previously published model incorporating visual mechanisms similar to cortical simple cells was able to predict discrimination thresholds accurately for curves and bars defined by contrast edges, but it failed to predict discrimination at texture boundaries. In order to predict
H. R. Wilson and W A. Richards
discrimination at texture boundaries, it was necessary to develop a two-stage model consisting of a first filtering stage followed by response squaring and a second stage of filtering at lower spatial frequencies. This model is similar to others developed for predicting visual responses to texture. 12,13 The model for texture boundary extraction developed here raises an interesting question concerning the relationship between the first- and second-stage filters (see Fig. 10). In particular, is the output at each first-stage orientation analyzed by a wide range of oriented secondstage filters? The problem here, of course, is one of proliferation of units. A possible resolution would be to pool the responses of the first-stage units across orientations and use this as the input to a single set of second-stage filters, as suggested by Morgan et al. to explain geometrical illusions.3 2 First-stage pooling across orientations should suffice for the patterns used in this study, which contain only one square-wave grating orientation (vertical). However, such pooling would fail to extract texture boundaries defined by an orientation change in the underlying grating, such as the stimuli employed by Bergen and Landy. 2 The resolution of this issue thus remains a challenging topic for further research. Is there any physiological evidence for cells responding to stimuli in ways similar to the units in this two-stage texture boundary model? In some respects the 'model texture units resemble the end-stopped complex cells in monkey cortex described by Hubel and Wiesel. 33 However, there is a greater similarity to units discovered in area V2 of monkeys by von der Heydt et al. 4 and von der Heydt and Peterhans. 5 They found cells that responded to discontinuities in simple line gratings, such as that illustrated in Fig. 11(a). These cells were found to be selective for the orientation of the discontinuity rather than for the orientation of the grating bars. Furthermore, these cells exhibited length summation along the discontinuity, because the response increased with the number of bars in the grating defining the discontinuity. Finally, cells sensitive to discontinuity were found in V2 but not in VI. Because the stimulus in Fig. 11(a) may be thought of as a rectangular wave grating with a 1800 phase shift across the middle, it is closely related to the texture boundaries used in the present study. To demonstrate that the twostage texture model developed above responds in a manner similar to that of the V2 discontinuity cells, it was applied to the stimulus in Fig. 11(a). As is shown in Fig. 11(b), the two-stage texture model does indeed extract the discontinuity locus from the image. Because of the elongation of the second-stage filter [see Fig. 10(c)], the texture model will also mimic the V2 cells in showing response summation along the discontinuity. This suggests the hypothesis that the first filtering stage of the current texture model might be identified with simple cells in Vi and that the squaring stage plus the second filtering stage might be identified with the discontinuity cells in V2. Further psychophysical evidence that perception of discontinuities in simple line gratings is consistent with V2 physiology has been provided by others.3 4 Experimental and modeling studies have recently shown that end-stopped simple cells will generate responses that vary with the curvature of a line or edge. 0 However, there are several problems with this model for
Vol. 9, No. 10/October 1992/J. Opt. Soc. Am. A
H. R. Wilson and W A. Richards
1661
V2. If we think of Vi as encoding simple contrast edges and boundaries and V2 as encoding texture boundaries, then it is plausible that V3 contains a common processing apparatus for the extraction of shape information fromboth types of stimuli. In a like manner, MT receives input directly from Vi and V2, and a new model for twodimensional motion perception suggests that these pathways contribute to a common motion analysis in MT.28 It should be acknowledged here that these suggestions do not incorporate the feedback pathways known to lead from V3 and MT back to Vi and V2. Although obviously they are speculative, these hypotheses provide a testable explanation for the parallel input to several higher cortical areas from Vi and V2. 0.20
Position
-0.20
(a)
ACKNOWLEDGMENTS
0.20
This research was conducted while H. R. Wilson was a visiting professor in the Department of Brain and Cognitive Science at the Massachusetts Institute of Technology. Results were first reported at the 1988 annual meeting of the Association for Research in Vision and Ophthalmology. The work was supported in part by National Institutes of Health grant EY02158 to H. R. Wilson and U.S. Air Force Office of Scientific Research grant F49620-83C-0135; 89-0504 to W A. Richards.
(b)j
REFERENCES -0.2°0_ -0.2°
.
.
.
0.2 Position (b) Fig. 11. (a) Simple line grating with discontinuities and (b) discontinuity locus extracted by the two-stage texture model. curvature encoding. First, responses of both the cells and the model also vary as a function of the length of straight lines and edges, and they therefore confound curvature and length. As Koenderink and Richards demonstrated, it is necessary to compare the responses of
end-stopped cells with responses of orientation-selective simple cells to extract curvature information unambiguously.8 Second, the end-stopped simple cells in these models will not extract curvature information at texture boundaries, because their response to curvature will be disrupted by the presence of the square-wave grating bars. The units responding to texture boundaries in our model might be viewed as being end stopped in the sense that they fail to respond when there is no texture discontinuity in their receptive field. However, their properties resemble those of end-stopped complex cells rather than endstopped simple cells. Comparison with primate physiology 4 "5 has suggested that the first and second stages of the texture boundary model may be associated with simple cells in VI and discontinuity cells in V2, respectively. The actual curvaturediscrimination calculation for texture is then identical to that for simple lines and edges,4'5 except that it employs the responses of the model V2 units rather than the model simple cells of Vi. This leads to a conjecture concerning the anatomical interconnections of higher cortical areas. V3, for example, receives input directly from both Vi and
1. F. Attneave, "Some informational aspects of visual perception," Psychol. Rev. 61, 183-193 (1954). 2. D. D. Hoffman and W Richards, "Parts of recognition," Cognition 18, 65-96 (1984).
3. W Richards, B. Dawson, and D. Whittington, "Encoding 4. 5. 6. 7. 8. 9.
contour shape by curvature extrema," J. Opt. Soc. Am A 3, 1483-1491 (1986). H. R. Wilson, "Discrimination of contour curvature: data and theory," J. Opt. Soc. Am. A 2, 1191-1199 (1985). H. R. Wilson and W A. Richards, "Mechanisms of contour curvature discrimination," J. Opt. Soc. Am. A 6, 106-115 (1989). R. J. Watt and D. P. Andrews, "Contour curvature analysis: hyperacuities in the discrimination of detailed shape," Vision Res. 22, 449-460 (1982). M. Ferraro and D. H. Foster, "Discrete and continuous modes of curved line discrimination controlled by effective stimulus duration," Spatial Vision 1, 219-230 (1986). J. J. Koenderink and W Richards, "Two-dimensional curvature operators," J. Opt. Soc. Am. A 5, 1136-1141 (1988). A. Dobbins, S. W Zucker, and M. S. Cynader, "Endstopped neurons in the visual cortex as a substrate for calculating curvature," Nature (London) 329, 438-441 (1987).
10. A. Dobbins, S. W Zucker, and M. S. Cynader, "Endstopping
and curvature," Vision Res. 29, 1371-1387 (1989). 11. B. Julesz and J. R. Bergen, "Textons, the fundamental elements in preattentive vision and perception of textures," Bell Syst. Tech. J. 62, 1619-1645 (1983). 12. J. R. Bergen and M. S. Landy, "Computational modeling of visual texture segregation," in Computational Models of Visual Processing, M. S. Landy and J. A. Movshon, eds.
(MIT Press, Cambridge, Mass., 1991).
13. N. Graham, "Complex channels, early local nonlinearities, and normalization in texture segregation," in Computational Models of Visual Processing, M. S. Landy and J. A.
Movshon, eds. (MIT Press, Cambridge, Mass., 1991).
14. R. von der Heydt, E. Peterhans, and G. Baumgartner, "Illusory contours and cortical neuron responses," Science 224, 1260-1262 (1984). 15. R. von der Heydt and E. Peterhans, "Mechanisms of contour
1662
J. Opt. Soc. Am. A/Vol. 9, No. 10/October 1992
H. R. Wilson and W A. Richards
perception in monkey visual cortex. I. Lines of pattern discontinuity," J. Neurosci. 9, 1731-1748 (1989). 16. R. F. Quick, 'A vector-magnitude model of contrast detection," Kybernetik 16, 1299-1302 (1974).
26. J. Nachmias and R. V Sansbury, "Grating contrast: discrimination may be better than detection," Vision Res. 14,
17. G. Westheimer and S. P. McKee, "Spatial configurations for
vision," J. Opt. Soc. Am. 70, 1458-1470 (1980). 28. H. R. Wilson, C. Yo, and V P. Ferrera, 'A psychophysically motivated model for two-dimensional motion perception," Visual Neurosci. 9, 79-97 (1992). 29. H. R. Wilson and D. J. Gelb, "Modified line element theory for spatial frequency and width discrimination," J. Opt. Soc. Am. A 1, 124-131 (1984). 30. H. R. Wilson, "Pattern discrimination, visual filters, and spatial sampling irregularity," in Computational Models of Visual Processing, M. S. Landy and J. A. Movshon, eds.
18. 19. 20. 21.
visual hyperacuity," Vision Res. 17, 941-947 (1977). R. J. Watt and M. J. Morgan, "The recognition and representation of edge blur: evidence for spatial primitives in human vision," Vision Res. 23, 1465-1477 (1983). G. Westheimer, "Line-separation discrimination curve in the human fovea: smooth or segmented?" J. Opt. Soc. Am. A 1, 683-684 (1984). J. Hirsch and R. Hylton, "Limits of spatial frequency discrimination as evidence of neural interpolation," J. Opt. Soc. Am. 72, 1367-1374 (1982). D. M. Levi, S. A. Klein, and Y. L. Yap, "'Weber's law' for position: unconfounding the role of separation and eccentricity," Vision Res. 28, 597-603 (1988).
22. C. A. Curcio, K. R. Sloan, 0. Packer, A. E. Hendrickson, and R. E. Kalina, "Distribution of cones in human and monkey retina:
individual variability and radial asymmetry,"
Science 236, 579-582 (1987). 23. H. R. Wilson, "Responses of spatial mechanisms can explain hyperacuity," Vision Res. 26, 453-469 (1986). 24. H. R. Wilson, D. K. McFarlane, and G. C. Phillips, "Spatial frequency tuning of orientation selective units estimated by oblique masking," Vision Res. 23, 873-882 (1983). 25. G. C. Phillips and H. R. Wilson, "Orientation bandwidths of spatial mechanisms measured by masking," J. Opt. Soc. Am.
A 1, 226-232 (1984).
1039-1042 (1974).
27. G. E. Legge and J. M. Foley, "Contrast masking in human
(MIT Press, Cambridge, Mass., 1991). 31. D. M. Levi and G. Westheimer, "Spatial interval discrimination in the human fovea: what delimits the interval?" J.
Opt. Soc. Am. A 4, 1304-1313 (1987). 32. M. J. Morgan, G. J. Hole, and A. Glennerster, "Biases and sensitivities in geometrical illusions," Vision Res. 30, 17931810 (1990).
33. D. H. Hubel and T. N. Wiesel, "Receptive fields and functional architecture of monkey striate cortex," J. Physiol. 195, 215-243 (1968). 34. R. Gurnsey, G. K. Humphrey, and P. Kapitan, "Parallel discrimination of subjective contours defined by offset gratings," Percept. Psychophys. (to be published).
H. R. Wilson and W A. Richards
1653
Curvature and separation discrimination at texture boundaries Hugh R. Wilson Visual Sciences Center, University of Chicago, 939 East 57th Street, Chicago, Illinois 60637 Whitman A. Richards Department of Brain and Cognitive Science, Massachusetts Institute of Technology, Cambridge,Massachusetts 02139 Received February 28, 1991; revised manuscript received May 19, 1992; accepted May 19, 1992 Visual discrimination of contour curvature was investigated by using contours defined by the locus of points at which the phase of a square-wave grating was shifted by 180° (a texture boundary). Curvature-increment thresholds were measured for contour curvatures from 0.31 to 10.65 deg-', for grating spatial frequencies of 4.0 and 16.0 cycles per degree (cpd), and for gratings in either sine or cosine phase at the point of maximum curvature. Thresholds for these conditions were compared with curvature discrimination at black-white edges. Grating phase had no effect on performance at any curvature or grating frequency, but 16.0-cpd gratings produced a threshold elevation at all curvatures by an average factor of 2.4. Two-line separation discrimination was also measured for lines defined by texture boundaries. These data can be predicted by a model incorporating end-stopped complex cells of a type reported physiologically in primate area V2.
INTRODUCTION The retinal images of most visual objects are bounded by simple closed contours. Because these contours usually vary in curvature, curvature encoding must play an important role in the visual representation of these objects. Research on curvature processing has focused almost exclusively on discrimination of the curvature of simple line stimuli,"' 0 and it has been shown that these discrimination data can be predicted by the differential responses of orientation-selective visual mechanisms. 4' 5 However, curved contours can be generated by complex texture boundaries as well as by luminance edges. For example, contours are clearly visible between regions having different contrasts, spatial-frequency spectra, orientation ranges, or phases. Texture differences giving rise to effortless visual texture segregation have been intensively studied by Julesz and Bergen." However, no attempt has been made to determine the accuracy with which the shape of texture boundaries is represented in the visual system. This paper focuses on three key questions concerning the encoding of texture boundaries. First, can the visual system discriminate texture-boundary curvature as accurately as it can discriminate simple line or edge curvature? Second, is two-line separation discrimination as accurate for texture lines as it is for contrast-defined lines? Third, can the same orientation-selective mechanisms predict curvature and separation discrimination for both contrast and texture boundaries? To answer these questions it was necessary to choose a class of stimuli in which the texture boundary always contained the same amount of information as a curved black-white edge; this ensured that any deficits in performance would be due to limitations of visual processing rather than to 0740-3232/92/101653-10$05.00
reduced stimulus information content. We have chosen to use patterns in which the texture boundary is defined by the locus of a 1800 phase shift in a square-wave grating (see Figs. 1 and 2). In the limiting case of a low grating frequency the stimulus becomes a black-white edge. At higher spatial frequencies the 1800 phase-shift locus becomes a boundary between one-dimensional, periodic textures. In all cases, however, the same amount of information defines the contour, because there are the same number of black-white transitions along the curved contour regardless of the spatial frequency of the squarewave grating. Curvature-increment thresholds were measured as a function of the spatial frequency of the square-wave grating. Relative to discrimination thresholds obtained with black-white edges, curved contours defined by a 1800 phase shift in a square-wave grating of 16.0 cycles per degree (cpd) caused threshold elevations of a factor averaging 2.4 at all curvatures tested. A quantitative model for curvature discrimination that has been successfully applied to curved lines4 5 accurately predicted data obtained with curved edges, but it predicted far too great a threshold elevation for the phase-shift contours in 16.0-cpd gratings. However, this model employed only mechanisms with spatial filters similar to those of cortical simple cells. Because recent work has suggested that more complex, nonlinear filtering is necessary to explain aspects of texture perception, 2 3 mechanisms of this type were employed to model our data. A satisfactory fit was achieved by using mechanisms with properties similar to those of the end-stopped complex cells found in area 18 of macaque visual cortex. 4 5 Thus our results indicate that texture boundaries cannot be encoded with as much accuracy as can simple luminance edges, even when the same amount of information is present in the stimulus, and that differ© 1992 Optical Society of America
1654
J. Opt. Soc. Am. A/Vol. 9, No. 10/October 1992
H. R. Wilson and W A. Richards
two different contour curvatures and two different grating spatial frequencies, whereas Fig. 2 illustrates pairs of texture bars used for two-line separation discrimination. In all cases the vertical square-wave grating was at 100% contrast. The curved contours used in this study consisted of a parabola grafted to a pair of straight lines, usually oriented at ±45.00, so that the first derivative would be continuous. In terms of the horizontal (x) and vertical (y) coordinates, the locus of the contour was thus defined by the equation
(a)
-AX2 +
A
I+-Ixl+A
for Ix I2 forIxl>j2
y
(b)
(c)
Fig. 1. Stimulus patterns having textures with (a), (b) equal spatial frequencies and (b), (c) equal contour curvatures.
(a)
(b)
(d)
(c)
III I I I I I CI
Fig. 2. Pairs of (a), (b) luminance bars and (c), (d) texture bars used in two-line separation discrimination experiments.
ent mechanisms are involved in processing these two types of contour.
METHODS The texture boundaries used in this study were defined by the locus of a 180° phase shift in a vertical square-wave grating. Figure 1 illustrates such texture boundaries for
(1)
12
As discussed below, these stimuli have a unique curvature extremum at x = 0, where the curvature is equal to 2A. This extreme curvature will be used to describe each stimulus. In addition, each contour has an identical range of orientations of ±45.00, so that only curvature can provide a cue for discrimination. To prevent the absolute position of the pattern on the screen from providing a spurious cue for discrimination, the vertical location of each pattern was varied by adding a random displacement A from trial to trial. Values of A were uniformly distributed through ±10% of the display height. Because of limitations of display size, experiments with low curvatures used an orientation range of ±26.6 or 14.00, but all stimuli in a given experiment had the same orientation range. The general formula for the curvature K at any point along a contour defined by the equation y = f(x) may be shown to be
K =
d2y/dx 2
[1 + (dy/dx)2 ]3 2
(2)
Along the parabolic contour used in these experiments, therefore, the curvature becomes K-[1
2A 2 2 + (2Ax) ]3/
(3)
From this it follows that the maximum curvature occurs at x = 0, where K = 2A. Stimuli were presented on the face of an Apple Macintosh computer, which the subject viewed by using a front surface mirror located across the room; this placed the display at an optical distance of 10.0 m. At this distance screen dimensions were 1.01° by 0.67° and individual pixels subtended 7.09 arcsec. The mirror was surrounded by a cardboard mask that was illuminated at the same mean intensity as was the display (30.0 cd/M2). The subject initiated each trial by pressing the button on the Macintosh mouse. The two-interval forced-choice presentation that followed comprised two 0.75-s stimulus presentations separated by a 0.50-s interval at the mean luminance. The subject rolled the mouse left or right to indicate whether the first or the second interval contained the more sharply curved contour. As illustrated in Fig. 1, the height of the curves on the display was randomized so that distance to the top or bottom could not be used as a cue for discrimination.
In separate experiments increment thresholds were measured for base curvatures of 0.31, 0.61, 1.42, 3.55, and 10.65 deg-'. Square-wave grating frequencies of 1.0, 4.0, and 16.0 cpd were used to define the curved contours. In addition, each grating was set at either cosine or sine phase relative to the point of maximum curvature. In cosine phase the point of maximum curvature occurred at the middle of a bar, whereas the curvature maximum fell at the edge of a grating bar when sine phase was used [see Fig. 1(c)]. In separate experiments discrimination thresholds were measured for all combinations of curvature, grating spatial frequency, and relative grating phase. Two-line separation discrimination was measured by using horizontal bars 11.33 arcmin long by 1.9 arcmin wide. Different experiments measured discrimination thresholds for base separations of 2.36, 3.78, 7.56, and 11.34 arcmin. As shown in Fig. 2, the patterns were either simple black bars on a white background (contrast bars) or bars defined by the locus of a 1800 phase shift in a 16.0-cpd square-wave grating (texture bars). The same bar dimensions and separations were used for both contrast and texture bars. Stimuli for a single experiment consisted of a contour of standard curvature plus four test contours with curvatures progressively greater than the standard. A method of constant stimuli was used in which the Macintosh randomly selected the interval containing the standard curvature and also randomly selected the test curvature for the other interval. Each test curvature was presented in 25 trials (a total of 100 trials for each experiment), and the resulting data giving percentage correct discrimination versus curvature increment were fit with a Quick' psychometric function by using a maximum-likelihood estimation procedure. Reported discrimination thresholds correspond to the 75% point on the fitted function. The two authors (HRW and WAR) served as subjects.
RESULTS Curvature Discrimination The goal of these experiments was to determine the accuracy of curvature discrimination at texture boundaries. Therefore curvature discrimination was first measured by using a black-white curved edge to provide a baseline for subsequent texture measurements. Thus the stimuli were the same as those in Fig. 1, except that they were black below the curved contour and white above the contour. Data for two subjects for a curvature range of 0.31-10.65 deg-' are shown by filled circles in Fig. 3. Thresholds are roughly constant at approximately 0.05 deg-' for curvatures below 1.0 deg-'. At higher curvatures thresholds rise roughly linearly on double logarithmic coordinates. A least-mean-squares curve fit determined the slope of this straight-line portion to be 0.93 for HRW and 1.05 for WAR, thus indicating that curvature discrimination approximately obeys Weber's law in this region. The Weber fraction averaged 6.6%, which further supports the finding that curvature discrimination falls into the hyperacuity range.6 The general features of black-white edge-curvature discrimination are the same as those for line-curvature discrimination, and the increment thresholds are quantitatively similar.4' 5
1655
Vol. 9, No. 10/October 1992/J. Opt. Soc. Am. A
H. R. Wilson and W A. Richards
Curvature discrimination at texture boundaries was next measured by using curves defined by the locus of a 1800 phase shift in a vertical square-wave grating (see Fig. 1). The open circles in Fig. 3 show curvature thresholds for a contour in sine phase on a 1.0-cpd squarewave grating. Note that these thresholds did not differ significantly from thresholds for a black-white edge, which is in cosine phase relative to the point of maximum curvature. Similarly, data for grating spatial frequencies of 4.0 and 16.0 cpd (see Fig. 4) showed no significant difference between sine and cosine phase thresholds for either subject. A three-factor analysis of variance was run 10 0)
.*
'a)
O .
HRW Edges HRW sin Theoryi
I
~0 U)
.1
'op~~c
E C)
. I .
... . . . ..... I
.
........
0.01 0. .1
IC10
10
1
Curvature (deg-') 10 * O 0
-
U)
WAR Edges WAR sin Theory
.1
I-
a)
E 0.01 L 0.1
.
.
.
. . .1 .
.
.
.
1 ..
.
.
1
.
.
.
10
1 00
1
0) a)
Curvature (deg- ) Fig. 3. Edge-curvature discrimination measurements.
10 O *
la a)
a) C,
Sine phase Cosine phase
o
1
0S Edge Thresholds
0.1
C I
0.01
grating. curves HRW, ,6cpd I .. . . .I
'
.
100 10 1 Curvature (deg-') Fig. 4. Sine and cosine phase threshold versus curvature for 16-cpd grating curves. The subject is HRW 0.1
1656
J. Opt. Soc. Am. A/Vol. 9, No. 10/October 1992
H. R. Wilson and W A. Richards
Table 1. Effect of Square-Wave Grating Phase on Curvature Discrimination at Texture Boundaries Subject
Frequency (cpd)
t (df = 4)a
Significance p
WAR
1.0
0.220
>0.83
WAR
4.0
1.325
>0.24
WAR
16.0
1.298
>0.25
HRW
1.0
0.538
>0.61
HRW
4.0
0.983
>0.38
HRW
16.0
1.028
>0.36
'For each subject at each grating spatial frequency, a two-tailed t test was performed to compare discrimination thresholds across curvatures for sine and cosine phase conditions (paired measures). As is evident from the table, thresholds did not differ significantly between sine and cosine phase conditions.
on the data for each subject by using the variables of curvature, spatial frequency, and phase. Although curvature and spatial frequency were found to be significant at the 0.001 level, phase (along with all pairwise and three-way interactions involving phase) was insignificant (p > 0.35 for each subject). For each subject we also performed a t test for each grating frequency to assess the significance of any difference between curvature thresholds for gratings in sine and cosine phase (paired measures). As is indicated in Table 1, there was no significant effect of relative grating phase on curvature discrimination for either subject at any spatial frequency. This phase independence has important theoretical implications, as discussed below. Curvature-discrimination thresholds measured at texture boundaries in 16.0-cpd square-wave gratings [see Figs. 1(a) and 1(b)] were elevated significantly relative to thresholds at contrast edges, as is apparent from the data for HRW plotted in Fig. 4. Relative to the solid curve, which shows edge thresholds from Fig. 3, curvature thresholds at the 16.0-cpd texture boundary (circles) were higher at all base curvatures. As noted above, thresholds obtained with the grating in sine phase (open circles) did not differ significantly from those for the cosine phase condition (filled circles). Therefore sine and cosine phase data are averaged in all subsequent plots. Threshold elevations are defined as the ratio of curvature-increment threshold at the 16-cpd texture boundary to the threshold for a contrast edge. A threshold elevation of unity would indicate that curvature discrimination at the texture boundary was as accurate as discrimination at a contrast boundary. As is shown in Fig. 5, however, all threshold elevations were greater than unity, thus indicating a degradation of performance with the curved texture boundary. A power-law fit to the data for each subject revealed a very slight trend toward increasing threshold elevations with increasing curvature (the power-law exponent averaged 0.11 for both subjects). Because this is virtually flat, however, threshold elevations were averaged across curvatures, producing mean elevations of 2.13 for HRW and 2.72 for WAR. Although threshold elevations for curvature discrimination at texture boundaries were certainly expected, the fact that they were relatively small has important theoretical implications (see below). Curvature threshold elevations were also measured at texture boundaries defined by the locus of a 1800 phase shift in 1.0- and 4.0-cpd square waves. The pattern of
threshold elevations for 4.0-cpd gratings was qualitatively similar to that of the 16.0-cpd data in Fig. 4, except that threshold elevations were significantly smaller, averaging 1.63 for HRW and 1.44 for WAR. As is shown by the open circles in Fig. 3, thresholds were not elevated for the 1.0-cpd gratings. Two-Bar Separation Discrimination To characterize further the encoding of spatial information at texture boundaries, we also measured thresholds for a more common hyperacuity task. This was two-line separation discrimination measured as a function of separation (see stimuli in Fig. 2). The texture bars in this experiment were defined by the locus of a 1800 phase shift in a vertical 16.0-cpd square-wave grating. The bars subtended a height of 1.9 arcmin and a length of 11.33 arcmin. Psychometric functions for separation discrimination at a base separation of 2.36 arcmin are plotted for one subject in Fig. 6. It is evident that the 75%correct point occurs at a much larger separation for the texture-defined bars (solid circles and solid curve) than for the contrast bars (open circles and dashed curve). However, this graph also reveals a problem in estimating the 75%-correct value for the contrast lines: the smallest available separation increment in our experiments was 30 |
HRW|
0O-WARI
C 0
10 a)
Simple Cel Model
Texture Model
P
co
:* .
0.1,:2
.
-r
.
.
.
.
.
.
1
,,,,,..ll
.
.
.
.
.
.
.
.
.
10
.
20
Curvature (deg- 1 ) Fig. 5. Threshold elevation versus curvature for the 16-cpd texture curve.
1.0 4-
a L-. 0
0
0.9 0.8
C 0
0.7
U-.
0.6 0.5
0
5 10 15 20 25 Increment Separation (arcsec)
30
Fig. 6. Fraction correct versus increment separation for the two-bar separation test. The subject is WAR.
RdVol. 9, No. 10/October 1992/J. Opt. Soc. Am. A
H. R. Wilson and W A. Richards
THEORY
1 00 Texture Bars
S
X
0
1657
1
-
Contrast Bars F
1
HRW Texture *+~~~~~~~~~~~4~- HRW Bars -W l WAR Texture -13- WAR Bars 10
30
Separation (arcmin) Fig. 7. Thresholds for contrast bars and texture bars for the two-bar separation discrimination test.
7.09 arcsec, yet the data in Fig. 6 show 92%-correct discrimination at this point. Clearly, the threshold must be within the range 0.0-7.09 arcsec in this case. To provide a more constrained range for the estimate, we adopted the 6 following procedure. Four separate Quick functions 1, of value fixed were fitted, with each fit using a different 2, 3, or 4 for the exponent. This covers the range of exponents reported in both detection and increment threshold experiments. The four 75%-correct points thus obtained were then averaged. The dashed curve in Fig. 6 shows the Quick function obtained with an exponent of 2, which produced the threshold estimate closest to the average, and the solid horizontal line crossing this function at the 75% point shows the range for all four estimates. This threshold-estimation procedure was necessary for both subjects only at the smallest base separation, because thresholds were significantly higher for larger separations (see Fig. 7). This problem did not arise for curvature discrimination, because every psychometric function had at least one point falling below the 75% correct level. Thresholds for the simple black bars on a white background (contrast bars) are plotted for two subjects as open symbols in Fig. 7. At the smallest separation of 2.36 arcmin, descrimination thresholds averaged 3.9 arcsec, which is well within the hyperacuity range. As separation increased, thresholds rose linearly on double logarithmic coordinates in accordance with Weber's law. These data are in agreement with those from previous studies of separation discrimination. 17 2- ' As is shown by the filled symbols in Fig. 7, separation discrimination thresholds for the texture bars were higher than those for the contrast bars at all separations tested. These data are replotted in Fig. 8 as threshold elevations for the two subjects. Threshold elevation was greatest at the smallest separation of 2.36 arcmin and decreased as separation increased. The threshold elevation at the smallest separation was 8.14 for HRW and 3.42 for WAR, and these elevations decreased to 1.63 for HRW and 1.44 for WAR at the largest separations used. Note, however, that at the smallest separation the threshold for texture bars was 13.0 arcsec for WAR and 28.5 arcsec for HRW (see Fig. 7). These thresholds are still somewhat less than the spacing of foveal cones. 22
Two key results have emerged from a comparison of curvature and separation discrimination at texture boundaries with discrimination at simple contrast boundaries. Because these differences are found primarily at the high grating frequencies, we will focus on the results obtained with 16.0-cpd square-wave gratings. First, curvature discrimination for all curvatures is poorer at texture boundaries by an average factor of 2.4. Second, separation discrimination for texture bars is poorer than for contrast bars, but the greatest degradation of performance occurs at small separations. This section raises the question of whether the texture results can be predicted by models that have been shown to predict curvature 4'5 and hyperacuity 23 data accurately for simple contrast patterns. In a study of curvature discrimination using bandpassfiltered lines as stimuli, Wilson and Richards 5 showed that an extension of a previous model for pattern discrimination 4'2 3 could accurately predict thresholds for these simple contrast-defined curves. This model was based on the responses of orientation-selective simple cells whose spatial filtering properties were derived from psychophysical masking data. 24'25 A vertically oriented model filter is described by the equation
F(x,y) = A[exp(- 2 /o-12 ) - B exp(-x 2 /0a2 2 ) + C exp(-x 2 /03 2)]exp(-y 2 /ry 2 ),
(4)
and other orientations may be obtained by simple rotation of coordinates. The curved stimulus is first convolved with these filters, and the result is then passed through a contrast nonlinearity. As determined from masking studies,2 42 62 7 this nolinearity is accelerated at low contrasts but exhibits a power-law compression at high contrasts. Thus the filter response R to a two-dimensional pattern P(x, y) is given by the equations F(x
S(x, ) =
-
x'y - y')P(x'y')dx'dy',
(5)
f.
R(SC) =
(SC)2 + K(SC)3-8 K + (SC)2
(6)
10
C 0 co
a) 0 CD
1
1
10
20
Separation (arcmin) Fig. 8. Threshold elevation versus curvature for the two-bar separation discrimination test.
1658
J. Opt. Soc. Am. A/Vol. 9, No. 10/October 1992
where C is the pattern contrast and K and sEare constants measured by masking.24 The difference in response to pairs of curved lines is then computed by pooling response differences among mechanisms tuned to all orientations and spatial frequencies, with the pooling operation defined by the Euclidean distance. Line curvature was increased until the difference in pooled response between two curves reached a threshold value of unity, which corresponds to 75% correct in the model. The pooling operation computed curvature by using one of two procedures, and the lower of the two thresholds was chosen. This first procedure, which was most sensitive to curvatures above 3.0 deg-', was entirely local, being based on the responses of units centered at the point of maximum curvature. The second, which was most sensitive at lower curvatures, was bilocal, being based on pooled responses of high-spatial-frequency filters displaced symmetrically at a fixed distance from the point of maximum curvature along the tangent to the curve. The distance of ±8.2 arcmin for the displaced units was found to provide the best fit to line curvature data.5 The use of only highspatial-frequency filters in the bilocal procedure resulted from the experimental discovery that high-frequency, bandpass filtering had no effect on discrimination at low curvatures. Details of this model were published previously. 5 The solid curves in Fig. 3 show edge-curvaturediscrimination thresholds predicted by this model, and the arrows show the point of transition between discrimination based on displaced units (bilocal discrimination) and discrimination based on units localized at the curvature maximum. The theory is obviously in good quantitative agreement with the experimental data for both subjects. Can this model predict curvature discrimination at texture boundaries in a 16.0-cpd grating? The thin line (Simple Cell Model) in Fig. 5 shows that the answer is no: this model predicts that thresholds will be elevated by an average factor of 10, whereas the data show an elevation averaging 2.4. The reason for this failure is illustrated in Fig. 9, which depicts vertically and horizontally oriented visual filters superimposed upon a 16.0-cpd grating. Because the filters are tuned to 16.0 cpd, vertical filters above and below the texture boundary will respond strongly to the stimulus. A horizontal filter tangent to the texture boundary, however, will integrate over several cycles of the grating and will therefore respond poorly if at all. Thus texture boundaries in high-spatial-frequency gratings are not encoded effectively by orientationselective units resembling cortical simple cells. How are texture boundaries processed by the visual system? An answer has been suggested by recent studies of texture discrimination. 21 3 As is shown in Fig. 9, vertically oriented simple-cell filters will respond well to the grating bars above and below the discontinuity. As the polarity of the bars switches from black to white at the discontinuity, however, a vertical filter will give zero response when centered on the discontinuity. This is illustrated in Fig. 10(a), which shows a contour plot of the vertical filter responses to a curved texture boundary. Note that there is no response in a band along the curve. If the pointwise responses are now either rectified or squared (squaring is the lowest-order term in the Taylor series for rectification), filters tuned to the horizontal
H. R. Wilson and W A. Richards
should respond well along the curve. This is illustrated in Fig. 10(b). This second-stage filter must be tuned to lower spatial frequencies to avoid responses to the individual bars of the original grating. Based on the results of an analogous study of two-dimensional motion processing, the second-stage filter has been set lower in frequency by a factor of 2.0.25 The model for texture-boundary analysis is shown schematically in Fig. 10(c). First, high-frequency filters process the stimulus, then the response is squared, and finally a second stage of linear filtering at lower spatial frequencies occurs. In the current model the second stage incorporates filters tuned to spatial frequencies 2.0, 4.0, and 8.0 times lower in frequency than are the firststage filters. These filters are also defined by Eq. (4) and thus have the same aspect ratio as that of the first-stage filters. These filters essentially perform an analysis of the texture boundary across several coarser spatial scales. Although the second filter was chosen perpendicular to the first in Fig. 10(c), the second filtering stage has a range of orientations that differ from those of the firststage filters by as much as ±450 in 15° increments. This range of second-stage orientations is necessary for pro-
Fig. 9. Filter response to texture boundaries. 0.2'
(a)U0 UU j QQ 0 V0 L C L
39.
U
I
Position
-0.2' L-
0.2 o
-0.2'
Position
0.2'
(C) +
First Filter
- 0-*
IL Response Squaring
1
+
Second Filter
Fig. 10. Contour plots of responses to curved texture boundaries by (a) vertical first-stage filter and (b) horizontal second-stage filter. (c) Schematic diagram of the model used for textureboundary analysis.
Vol. 9,No. 10/October 1992/J. Opt. Soc. Am. A
H. R. Wilson and W A. Richards
cessing texture-boundary curvatures over the range studied here. Because of this rough independence between the orientation of the texture boundary and the orientation of the texture elements defining this boundary, highorder operations that attempt to measure directly the curvature of the texture boundary, such as those proposed for contour curvature, 8 do not seem plausible. In summary, the model for pattern discrimination at texture boundaries consists of oriented, simple-cell-type filtering followed by rectification or squaring and a second stage of oriented filtering at lower spatial frequencies. Because the presence of thresholds in simple cells produces a half-wave rectification of their inputs, full-wave rectification may be easily accomplished in the visual system by pooling responses of on-center and off-center simple cells. The second stage performs pattern discrimination in essentially the same manner as does the model for curvature discrimination at luminance contours that has been described by Eqs. (4)-(6) and previously.4'5 '23
The only difference is that it operates on a
filtered and rectified neural representation rather than on the luminance distribution itself. For the stimulus used in our experiments, the computation engendered in this model can be greatly simplified. Because the first-stage visual filter in Eq. (4) is separable in Cartesian coordinates, the response of vertical filters to a horizontal texture boundary in a vertical grating (e.g., Fig. 9) is just S(x, y) -
MTF(wo) 2 f exp(-y' 2 /o.Y2 )dy'
-
uyrV7cos(2.wox). (7)
MTF(w) is the Fourier transform of the difference of the three Gaussian functions of x in Eq. (4). Note that the response involves a cosine term, because the filter in Eq. (4) is insensitive to the third and higher harmonics of the square wave. The term in brackets is just an error function minus a constant, and it ranges from -cy - 7r for y > 0. When Eq. (7) is squared, the variation in the y direction will be given by the square of the terms in brackets. Computations have shown that this expression may be well approximated by a constant minus a single Gaussian function: 2f
exp(-y'2/o.y2)dyP
=
-
VI
a2[1 - exp(-y2/0.81cry2)].
(8)
The Gaussian on the right-hand side of relation (8) was found to provide the best least-mean-squares approximation when the space constant was 0.9oy. This approximation is so good that plots of the left- and right-hand sides of relation (8) differ by less than the width of the plotting line, and thus a graphical comparison is pointless. Calculations show that the difference between the two functions is always less than 0.6% of the maximum value, 7roy . When the filtered stimulus in Eq. (7) is squared, in addition to the y variation in relation (8) there is also spatial
1659
variation resulting from the term [cos(2wrwx)] 2, which gives a constant term plus a term at frequency 2w. Because the second-stage filters are tuned to frequencies of co/2.0 or lower, however, they give no significant response to frequency 2w. Therefore predictions of the response of this model to various patterns can be calculated by treating the locus of the phase shift in the stimulus grating as a Gaussian-blurred line described by the right-hand side of relation (8). That is to say, the initial filtering and response squaring generate, to a high degree of accuracy, the Gaussian-blurred contour described by relation (8). This blurred contour is then processed by the larger (lower-frequency) oriented filters of the second stage. One final point remains in the specification of this model: the gain of the second filter stage relative to that of the first stage. To state it otherwise, the squared response of the first stage must be normalized before it is filtered by the lower-frequency filters of the second stage. This normalization was determined empirically by measuring the contrast threshold for detecting the presence of a horizontal texture boundary in a vertical 16.0-cpd square-wave grating. Average data for two subjects showed that the contrast threhold for detecting the texture boundary was 2.6%, whereas the threshold for detecting the 16.0-cpd vertical grating itself was 1.4%. The normalization of the second stage was therefore adjusted to reflect the ratio of these thresholds. We derived predictions of curvature discrimination at texture boundaries from this model and compared them with the predictions for discrimination at luminance edges (see Fig. 3) to obtain model threshold elevations. The result is shown by the heavy solid curve in Fig. 5, which is in reasonable agreement with data for the two subjects. Note that this two-stage discrimination model predicts much smaller threshold elevations than does a model based on responses of the first stage alone (the thin curve labeled Simple Cell Model in Fig. 5). One further aspect of the curvature data is readily explained by this model. As illustrated in Figs. 3 and 4 and discussed above, the relative phase of the vertical grating had no effect on curvature discrimination at texture boundaries (see Table 1). Because of the squaring of the first-stage response plus second-stage filtering at lower spatial frequencies, however, all phase information from the 16.0-cpd square-wave grating is effectively discarded, and only the locus of the texture boundary is extracted. In contrast, any model based directly on the simple-celltype filters in the first stage would incorrectly predict a phase dependence for discrimination. This model for encoding texture boundaries was also applied to the two-bar separation discrimination data of Fig. 7. In that figure the lower heavy curve is a plot of threshold predictions for simple contrast bars. These are predictions from the simple model that was also used for discrimination of edge curvature (solid curves in Fig. 3). ' and summarized above in As reported previously5 2 3 29 Eqs. (4)-(6), this model is based on the responses of only the first filtering stage. Although they lie slightly below the data of the two subjects (open symbols in Fig. 7) on average, the two-bar separation predictions of this model have previously been shown to provide an accurate fit to a wide range of data from a variety of other studies. 3 0 However, this model does not predict separation data for
1660
J. Opt. Soc. Am. A/Vol. 9, No. 10/October 1992
texture bars for the same reason that it does not predict curvature discrimination at texture boundaries. Namely, filters oriented along the texture boundaries respond poorly to the image, as is illustrated on the right-hand side of Fig. 9. As is shown by the upper solid line in Fig. 7, however, separation discrimination for texture bars is accurately predicted by the two-stage texture model developed to fit the curvature data. The ratio of the two theoretical curves gives the predicted threshold elevation, which is compared with data in Fig. 8. The fact that the model produces large threshold elevations at small separations and virtually no elevation at the largest separations is easy to explain. Relation (8) indicates that the effective stimulus for the second-stage filters is a line blurred by a Gaussian function with 0.9 times the space constant of the 16.0-cpd visual filters in the first processing stage. For these filters 0.93, = 3.28 arcmin, which is a relatively large blur at small separations but is a relatively small blur at large separations. At the second processing stage it is mainly the smaller filters that analyze small separations and larger filters that analyze larger separations. Because the blur is constant, it thus has a negligible effect at large separations. It may be noted that Levi and Westheimer 3 1 employed a variant of the two-line separation discrimination paradigm that produced results similar to those obtained with texture bars in the present study. In their study, thresholds for discriminating the separation between two dark bars were compared with separation thresholds obtained when one of the bars was dark and the other was white. The white-dark combination raised thresholds at a 2.0arcmin separation relative to the two dark bars, but this threshold elevation vanished before the separation reached 8.0 arcmin. Their data thus show the same quantitative pattern as the data in Fig. 5. Because of the rectification stage, the present model is also capable of predicting the Levi-Westheimer data. 3
DISCUSSION The experiments reported above were designed to determine how accurately the visual system can encode information at texture boundaries. In the cases of greatest interest these boundaries were defined by the locus of a 1800 phase shift in a vertical 16.0-cpd square-wave grating. Curvature discrimination at these texture boundaries was degraded relative to discrimination at contrast edges, but the degradation was only by a modest factor averaging 2.4 over all conditions. Two-bar separation discrimination thresholds were also greater for texture bars than for contrast bars, but here the elevation was large at separations below 4.0 aremin and small at greater separations (see Fig. 8). Because the texture boundaries used in this study contained precisely the same amount of information as is present in a simple contrast edge or bar, the performance degradation between texture and contrast cases is ipso facto a reflection of limitations in the visual analysis of texture. A previously published model incorporating visual mechanisms similar to cortical simple cells was able to predict discrimination thresholds accurately for curves and bars defined by contrast edges, but it failed to predict discrimination at texture boundaries. In order to predict
H. R. Wilson and W A. Richards
discrimination at texture boundaries, it was necessary to develop a two-stage model consisting of a first filtering stage followed by response squaring and a second stage of filtering at lower spatial frequencies. This model is similar to others developed for predicting visual responses to texture. 12,13 The model for texture boundary extraction developed here raises an interesting question concerning the relationship between the first- and second-stage filters (see Fig. 10). In particular, is the output at each first-stage orientation analyzed by a wide range of oriented secondstage filters? The problem here, of course, is one of proliferation of units. A possible resolution would be to pool the responses of the first-stage units across orientations and use this as the input to a single set of second-stage filters, as suggested by Morgan et al. to explain geometrical illusions.3 2 First-stage pooling across orientations should suffice for the patterns used in this study, which contain only one square-wave grating orientation (vertical). However, such pooling would fail to extract texture boundaries defined by an orientation change in the underlying grating, such as the stimuli employed by Bergen and Landy. 2 The resolution of this issue thus remains a challenging topic for further research. Is there any physiological evidence for cells responding to stimuli in ways similar to the units in this two-stage texture boundary model? In some respects the 'model texture units resemble the end-stopped complex cells in monkey cortex described by Hubel and Wiesel. 33 However, there is a greater similarity to units discovered in area V2 of monkeys by von der Heydt et al. 4 and von der Heydt and Peterhans. 5 They found cells that responded to discontinuities in simple line gratings, such as that illustrated in Fig. 11(a). These cells were found to be selective for the orientation of the discontinuity rather than for the orientation of the grating bars. Furthermore, these cells exhibited length summation along the discontinuity, because the response increased with the number of bars in the grating defining the discontinuity. Finally, cells sensitive to discontinuity were found in V2 but not in VI. Because the stimulus in Fig. 11(a) may be thought of as a rectangular wave grating with a 1800 phase shift across the middle, it is closely related to the texture boundaries used in the present study. To demonstrate that the twostage texture model developed above responds in a manner similar to that of the V2 discontinuity cells, it was applied to the stimulus in Fig. 11(a). As is shown in Fig. 11(b), the two-stage texture model does indeed extract the discontinuity locus from the image. Because of the elongation of the second-stage filter [see Fig. 10(c)], the texture model will also mimic the V2 cells in showing response summation along the discontinuity. This suggests the hypothesis that the first filtering stage of the current texture model might be identified with simple cells in Vi and that the squaring stage plus the second filtering stage might be identified with the discontinuity cells in V2. Further psychophysical evidence that perception of discontinuities in simple line gratings is consistent with V2 physiology has been provided by others.3 4 Experimental and modeling studies have recently shown that end-stopped simple cells will generate responses that vary with the curvature of a line or edge. 0 However, there are several problems with this model for
Vol. 9, No. 10/October 1992/J. Opt. Soc. Am. A
H. R. Wilson and W A. Richards
1661
V2. If we think of Vi as encoding simple contrast edges and boundaries and V2 as encoding texture boundaries, then it is plausible that V3 contains a common processing apparatus for the extraction of shape information fromboth types of stimuli. In a like manner, MT receives input directly from Vi and V2, and a new model for twodimensional motion perception suggests that these pathways contribute to a common motion analysis in MT.28 It should be acknowledged here that these suggestions do not incorporate the feedback pathways known to lead from V3 and MT back to Vi and V2. Although obviously they are speculative, these hypotheses provide a testable explanation for the parallel input to several higher cortical areas from Vi and V2. 0.20
Position
-0.20
(a)
ACKNOWLEDGMENTS
0.20
This research was conducted while H. R. Wilson was a visiting professor in the Department of Brain and Cognitive Science at the Massachusetts Institute of Technology. Results were first reported at the 1988 annual meeting of the Association for Research in Vision and Ophthalmology. The work was supported in part by National Institutes of Health grant EY02158 to H. R. Wilson and U.S. Air Force Office of Scientific Research grant F49620-83C-0135; 89-0504 to W A. Richards.
(b)j
REFERENCES -0.2°0_ -0.2°
.
.
.
0.2 Position (b) Fig. 11. (a) Simple line grating with discontinuities and (b) discontinuity locus extracted by the two-stage texture model. curvature encoding. First, responses of both the cells and the model also vary as a function of the length of straight lines and edges, and they therefore confound curvature and length. As Koenderink and Richards demonstrated, it is necessary to compare the responses of
end-stopped cells with responses of orientation-selective simple cells to extract curvature information unambiguously.8 Second, the end-stopped simple cells in these models will not extract curvature information at texture boundaries, because their response to curvature will be disrupted by the presence of the square-wave grating bars. The units responding to texture boundaries in our model might be viewed as being end stopped in the sense that they fail to respond when there is no texture discontinuity in their receptive field. However, their properties resemble those of end-stopped complex cells rather than endstopped simple cells. Comparison with primate physiology 4 "5 has suggested that the first and second stages of the texture boundary model may be associated with simple cells in VI and discontinuity cells in V2, respectively. The actual curvaturediscrimination calculation for texture is then identical to that for simple lines and edges,4'5 except that it employs the responses of the model V2 units rather than the model simple cells of Vi. This leads to a conjecture concerning the anatomical interconnections of higher cortical areas. V3, for example, receives input directly from both Vi and
1. F. Attneave, "Some informational aspects of visual perception," Psychol. Rev. 61, 183-193 (1954). 2. D. D. Hoffman and W Richards, "Parts of recognition," Cognition 18, 65-96 (1984).
3. W Richards, B. Dawson, and D. Whittington, "Encoding 4. 5. 6. 7. 8. 9.
contour shape by curvature extrema," J. Opt. Soc. Am A 3, 1483-1491 (1986). H. R. Wilson, "Discrimination of contour curvature: data and theory," J. Opt. Soc. Am. A 2, 1191-1199 (1985). H. R. Wilson and W A. Richards, "Mechanisms of contour curvature discrimination," J. Opt. Soc. Am. A 6, 106-115 (1989). R. J. Watt and D. P. Andrews, "Contour curvature analysis: hyperacuities in the discrimination of detailed shape," Vision Res. 22, 449-460 (1982). M. Ferraro and D. H. Foster, "Discrete and continuous modes of curved line discrimination controlled by effective stimulus duration," Spatial Vision 1, 219-230 (1986). J. J. Koenderink and W Richards, "Two-dimensional curvature operators," J. Opt. Soc. Am. A 5, 1136-1141 (1988). A. Dobbins, S. W Zucker, and M. S. Cynader, "Endstopped neurons in the visual cortex as a substrate for calculating curvature," Nature (London) 329, 438-441 (1987).
10. A. Dobbins, S. W Zucker, and M. S. Cynader, "Endstopping
and curvature," Vision Res. 29, 1371-1387 (1989). 11. B. Julesz and J. R. Bergen, "Textons, the fundamental elements in preattentive vision and perception of textures," Bell Syst. Tech. J. 62, 1619-1645 (1983). 12. J. R. Bergen and M. S. Landy, "Computational modeling of visual texture segregation," in Computational Models of Visual Processing, M. S. Landy and J. A. Movshon, eds.
(MIT Press, Cambridge, Mass., 1991).
13. N. Graham, "Complex channels, early local nonlinearities, and normalization in texture segregation," in Computational Models of Visual Processing, M. S. Landy and J. A.
Movshon, eds. (MIT Press, Cambridge, Mass., 1991).
14. R. von der Heydt, E. Peterhans, and G. Baumgartner, "Illusory contours and cortical neuron responses," Science 224, 1260-1262 (1984). 15. R. von der Heydt and E. Peterhans, "Mechanisms of contour
1662
J. Opt. Soc. Am. A/Vol. 9, No. 10/October 1992
H. R. Wilson and W A. Richards
perception in monkey visual cortex. I. Lines of pattern discontinuity," J. Neurosci. 9, 1731-1748 (1989). 16. R. F. Quick, 'A vector-magnitude model of contrast detection," Kybernetik 16, 1299-1302 (1974).
26. J. Nachmias and R. V Sansbury, "Grating contrast: discrimination may be better than detection," Vision Res. 14,
17. G. Westheimer and S. P. McKee, "Spatial configurations for
vision," J. Opt. Soc. Am. 70, 1458-1470 (1980). 28. H. R. Wilson, C. Yo, and V P. Ferrera, 'A psychophysically motivated model for two-dimensional motion perception," Visual Neurosci. 9, 79-97 (1992). 29. H. R. Wilson and D. J. Gelb, "Modified line element theory for spatial frequency and width discrimination," J. Opt. Soc. Am. A 1, 124-131 (1984). 30. H. R. Wilson, "Pattern discrimination, visual filters, and spatial sampling irregularity," in Computational Models of Visual Processing, M. S. Landy and J. A. Movshon, eds.
18. 19. 20. 21.
visual hyperacuity," Vision Res. 17, 941-947 (1977). R. J. Watt and M. J. Morgan, "The recognition and representation of edge blur: evidence for spatial primitives in human vision," Vision Res. 23, 1465-1477 (1983). G. Westheimer, "Line-separation discrimination curve in the human fovea: smooth or segmented?" J. Opt. Soc. Am. A 1, 683-684 (1984). J. Hirsch and R. Hylton, "Limits of spatial frequency discrimination as evidence of neural interpolation," J. Opt. Soc. Am. 72, 1367-1374 (1982). D. M. Levi, S. A. Klein, and Y. L. Yap, "'Weber's law' for position: unconfounding the role of separation and eccentricity," Vision Res. 28, 597-603 (1988).
22. C. A. Curcio, K. R. Sloan, 0. Packer, A. E. Hendrickson, and R. E. Kalina, "Distribution of cones in human and monkey retina:
individual variability and radial asymmetry,"
Science 236, 579-582 (1987). 23. H. R. Wilson, "Responses of spatial mechanisms can explain hyperacuity," Vision Res. 26, 453-469 (1986). 24. H. R. Wilson, D. K. McFarlane, and G. C. Phillips, "Spatial frequency tuning of orientation selective units estimated by oblique masking," Vision Res. 23, 873-882 (1983). 25. G. C. Phillips and H. R. Wilson, "Orientation bandwidths of spatial mechanisms measured by masking," J. Opt. Soc. Am.
A 1, 226-232 (1984).
1039-1042 (1974).
27. G. E. Legge and J. M. Foley, "Contrast masking in human
(MIT Press, Cambridge, Mass., 1991). 31. D. M. Levi and G. Westheimer, "Spatial interval discrimination in the human fovea: what delimits the interval?" J.
Opt. Soc. Am. A 4, 1304-1313 (1987). 32. M. J. Morgan, G. J. Hole, and A. Glennerster, "Biases and sensitivities in geometrical illusions," Vision Res. 30, 17931810 (1990).
33. D. H. Hubel and T. N. Wiesel, "Receptive fields and functional architecture of monkey striate cortex," J. Physiol. 195, 215-243 (1968). 34. R. Gurnsey, G. K. Humphrey, and P. Kapitan, "Parallel discrimination of subjective contours defined by offset gratings," Percept. Psychophys. (to be published).
