Vision Res. Vol. 32, No. 5, pp. 963-981, 1992 Printed in Great Britain. All rights resewed

Copyright

0

0042-6989/92 $5.00 + 0.00 1992 Pergamon Press plc

Simulation of Neural Contour Mechanisms: from Simple to End-stopped Cells FRIEDRICH HEITGER,* ESTHER PETERHANS,? Received 22 May

LUKAS ROSENTHALER,* OLAF KtllBLER*

1991; in revised form 21 November

RUDIGER

VON DER HEYDT,?

1991

Early stages of visual form prucessing were modelled by simulating cortical simple, complex and end-stopped cells. The computation involves (1) convolution of the image witb even and odd symmetrical orientation selective filters (S-operators), (2) combination of even and odd filter outputs to a local energy measure (C-operator), (3) “differentiation” of the C-operator maps along the respective orientation (single and double end-stopped operators) and (4) determination of local maxima (“key-points”) of the combined end-stopped operator activity. While S- aud C-operators are optimised for the representation of 1-D features such as edges and lines, the end-stopped operator responses at the key-points make explicit 2-D signal variations such as line ends, corners and segments of strong curvature. The theoretical need for this complementary representation is discussed. The model was tested on grey-valued images. Simple cells Complex cells End-stopped cells Computational model Contour

INTRODUCTION

Early vision

orthogonal direction. This is apparently the function of many of the orientation selective cells in the primary Biological visual systems appear universal compared visual cortex, and similar operations are used in techto the specialised technical systems used for automatic nical systems (e.g. Canny, 1986). However, with this object recognition, but they also rely on constraints of kind of edge operator, even in its optimised versions, a visual stimulation. “Knowledge” about the statistical conflict arises between the goals of contour definition. While lengthwise integration improves the definition of regularities of the stimulus is a basic property of sensory systems (Marr, 1982; Barlow, 1989). In general, this orientation and position of contours if they are straight, knowledge is implicit, reflected in the design of the it leads to imprecisions at curved segments, corners and system. Many characteristic features of biological visual terminations. In general, two-dimensional signal varisystems can be interpreted in this way by analysing ations are not adequately represented by edge operators. the statistics of images of natural scenes (cf. Switkes, The search for a complementary representation is the Mayer & Sloan, 1978; Srinivasan, Laughlin & Dubs, theme of this paper. 1982; Field, 1987; Derrico & Buchsbaum, 1990). Neurophysiological evidence suggests that, in the The abundance in the visual cortex of orientation visual cortex, simple and complex cells represent oneselective cells which can be activated by straight dimensional variations (Hubel & Wiesel, 1962), while light-dark borders, for example, can be considered as end-stopped cells take care of two-dimensional features a consequence of the facts that (1) object surfaces are such as blobs, line-ends, corners or curved segments usually smooth (due to the cohesiveness of matter), so (Hubel & Wiesel, 1965, 1968; Dobbins, Zucker & that the contours of objects in images can be approxiCynader, 1987; Versavel, Orban & Lagae, 1990). mated by mostly smooth curves and (2) reflectance in Complementation of the edge-detector scheme with general varies at a lesser rate across the surface of objects “blob-detectors” has also been proposed in computer than between objects; thus, abrupt intensity changes are vision (Marr, 1977). indicators of object edges. For detecting object contours Talking about representation of blobs, line-ends, it is therefore a good strategy to analyse images with corners, curved segments etc. one tends to think of these operators that are sensitive to the local intensity vari- signal variations as corresponding to certain object ation in one direction and integrate (average) along the features (just as lines and edges are often associated with the contours of objects). However, two-dimensional variations have yet another significance. Similarly as *CommunicationTechnologyLaboratory, SwissFederal Institute of sketched above for contrast borders, statistical correTechnologyETH, CH-8092Zurich, Switzerland. TDepartmentof Neurology, University Hospital Zurich, CH-8091 lations with object contours exist also for the twoZurich, Switzerland. dimensional variations. These correlations are again 963

964

FRIEDRICH

related to the high incidence of smooth contrast borders, and also seem to have a parallel in receptive fields of the visual cortex. When an object is seen against a background, the structure of the background is usually not related to the foreground object. The result is a discontinuity of pattern statistics at the contour. Obvious examples of such discontinuity are terminations of the contrast borders of the background at the occluding contour. Contrast borders can be considered as firstorder discontinuities, terminations of such borders as second-order discontinuities. Such terminations can be detected by relatively simple mechanisms. This might be the function of the cortical “end-stopped” cells. These cells are usually activated selectively by oriented contrast patterns like edges or lines, but only if the pattern does not extend into the “inhibitory end-zones” of the receptive field. Typically, such cells respond to short segments of lines, but also to line-ends and corners (Hubel & Wiesel, 1965, 1968; Peterhans & von der Heydt, 1987). In general there is no way of discriminating object features from occlusion features on a local basis; a corner could be the image of the corner of a cube (for example) or of an edge terminated by occlusion. If, however, several corners and/or line-ends are grouped in a linear fashion, they have a high probability of being generated by occlusion. Thus the detection of such aggregations of “occlusion features” is a way of identifying occluding contours. Based on a study of the neural representation of “illusory contours”, von der Heydt and Peterhans have argued that the mechanism of contour perception uses a combination of two operations, the detection of linear contrast borders and the detection of linear aggregations of occlusion features, and that these features are represented by the signals of end-stopped cells in the visual cortex (von der Heydt, Peterhans & Baumgartner, 1984; Peterhans, von der Heydt & Baumgartner, 1986; von der Heydt & Peterhans, 1989; Peterhans & von der Heydt, 1989). Thus endstopped cells play an important role in this theory; besides representing object features, they may serve the detection of local configurations that typically occur in situations of occlusion (von der Heydt & Peterhans, 1986; Peterhans & von der Heydt, 1987). This principle can be used to resolve situations of occlusion and to find “illusory contours”, at least in simple binary images (Finkel & Edelman, 1989). In this paper we present a computational scheme in which the two types of image information are represented equivalently: borders are given by operators akin to simple and complex cells, and terminations are provided by operators akin to end-stopped cells. We show that these operators complement each other and thus provide a solution to principle shortcomings of edge detectors. The resulting representation appears as a suitable basis for grouping and image segmentation operations of subsequent stages. By applying this scheme to grey-level images we show that this representation is very compact and at the same time precise in the localisation of essential image features. The process of

HEITGER

et al.

grouping and image segmentation will be treated in a subsequent paper (Heitger, Rosenthaler, von der Heydt. Peterhans & Kiibler, 1992). METHODS Simulations were done in Fortran on a VAX-780 computer equipped with a IP-8500 Gould DeAnza image processing system and more recently on DEC3100 workstations using standard C. Image dimensions were 512 x 512 pixels with 8-bit grey-level resolution. Convolutions were carried out by multiplication in the Fourier domain. All other calculations were performed in the space domain using single precision float format. For imaging purposes, results were transformed back to a 8-bit range. The spatial filtering (even and odd masks in 6 orientations) took most of the computing time while the calculation of end-stopped operator responses and key-points took only a fraction of it. All algorithms are suited for parallel processing. RESULTS Spatial filtering and local energy operators The first stage in processing intensity discontinuities consists of a convolution with orientation selective, linear filters. We have constructed odd- and even-symmetrical basis function pairs that are similar to 2-D Gabor filters (Kulikowski, Marcelja & Bishop, 1982), but with the important differences that our filters have polar-separable Fourier spectra, and both, the odd and the even variants integrate to zero in the space domain. The 2-D Gabor filters are not polar separable (Daugman, 1983) and only the odd type has zero integral. The non-zero integral of the even (cosine) Gabor titer means that it responds to uniform fields and is sensitive to changes of the absolute intensity. This disadvantage becomes particularly relevant if the titers are not narrowly tuned to spatial frequency because the Gaussian envelope then concentrates the weight in the centre lobe of the function. Polar-separability of the Fourier spectrum means that the spatial-frequency tuning of the filter does not change shape if measured at non-optimal orientation and, vice versa, its orientation tuning is the same for edges, lines or any one-dimensional variation in the image. Odd and even pairs zero mean

with

Gaussian

envelope

and

Starting from odd and even Gabor functions we obtain functions with zero means by introducing a gradual decrease in frequency of the sine and cosine functions with increasing distance from the centre of the envelope: Ix2 Godd(x) = e 20~.sin[2nv,xt(x)] (1) for the odd-symmetrical G_(x)

function, and

= e 2~~.cos[27cv,x~(x)]

(2)

SIMULATION

OF NEURAL

CONTOUR

MECHANISMS

FIGURE 1. “S-Gabor” functions. For the simulation of cortical receptive fields a family of functions was defined that are similar to Gabor functions in spatial filtering properties, but have zero integral. The figure shows a comparison of S-Gabor (solid lines) and Gabor functions (dashed lines) for even and odd types with bandwidths of 1.8 (A) and 1.0 (B) octaves. Space domain functions are shown on the left, Fourier spectra on the right (real part for even, imaginary part for odd fiction). The difference between the two types of function is most pronounced for the larger bandwidth, where the d.c. term in the spectrum of the even Gabor is obvious (A). With smaller bandwidth, the S-Gabors approach Gabor functions (B).

965

966

FRIEDRICH HEITGER et al.

for the even-symmetrical function, where c determines the width of the Gaussian envelope, v,, is the frequency at the origin and t(x) specifies the frequency sweep. We define t(x) = k .e-i’(X/a)2+ (1 _ k)

(3)

where k is the relative amplitude of the frequency sweep (0 < k < 1) (we used k = 0.5), and Iz is chosen so that (4) m G,,,,=O. s -m The effect of the frequency sweep is to increase the relative weight of the negative side-lobes of the even function. The parameter L determines how fast the wavefunction will sweep to lower frequencies. If the

product D . v. is not too small (> 0.38 with k = 0.5, which means that the original Gabor function must not be concentrated too much on the centre lobe) there exists at least one value of I that satisfies equation (4). We choose the lowest value of 2 with this property. In equation (3) the sweep rate is normalised with respect to (T so that 1 becomes independent of the filter size. Since simple cells are seldom strictly odd or even, but have been shown to occur in pairs with 90“ phase difference (Pollen & Ronner, 1981, in cat; we are not aware of comparable data from the monkey), which is also theoretically the ideal relationship (Morrone & Burr, 1988), we point out that, if 1 satisfies equation (4), the zero-mean condition is satisfied for all phases of the trigonometric wave function.

(4

(B)

/ -\ I

I

I I

:

1-1

I

II fl

I

1

I

J :

I

I-

’ ’



\

I

\



\

.-

I

FIGURE 2. Simple eeU receptive @is were rimulated by two-dim sqmtialfflters with po&ar-wparabk Fourier rpactra. (A) even, (B) odd symetrical filters. Contour plots of space functions are shown on the left, Fourier spectra on the right. Dashed lines indicate negative values. The filters are extensions of S-Gabor functions with 1.5 octaves spatial frequency bandwidth and k23.5” orientation bandwidth, corresponding approximately to the median cell of the monkey striate cortex.

SIMULATION OF NEURAL CONTOUR MECHANISMS

The new set of functions, equations (1) and (2), shares important properties with the Gabor functions: the envelope, defined by the root of the sum of squares of even- and odd-symmetrical filters, is a Gaussian and the combination of filter outputs in the form of a quadrature pair summation produces monomodal response profiles for lines, edges and their combinations. For small values of L the odd- and even-symmetric filters, equations (1) and (2) approximate sine and cosine Gabor functions. Because of this proximity we call them “stretched Gabor” or “S-Gabor” functions. Mathematically, S-Gabors are not as easy to handle as Gabor functions (cf. Daugman, 1985) because spectral peak and bandwidth depend not only on v. and 6, but also on 1: for non-zero 2, the spectral peak is shifted towards lower frequencies and the bandwidth is broad-

967

ened. We have used numerical methods to obtain filters with the desired peak frequencies and bandwidths. Figure l(A) and (B) shows examples of even and odd S-Gabor functions with spectral bandwidths of 1.8 and 1.O octaves (full width at half height) on the left, and the spectra on the right. The corresponding Gabor pairs (2 = 0) and their spectra are also shown (dashed lines). One can see that the spectrum of the Gabor function is positive at zero (d.c. component) but vanishes for the S-Gabor. The difference is conspicuous for the broader bandwidth [Fig. l(A)]. S-operators: two -dimensional jilters with polar separable spectrum We have constructed a set of two-dimensional operators by defining the radial variation in the Fourier plane

FIGURE 3. The representation of a test image (A) by S- (“simple”) and C- (“complex”) operators, as defined in text. The channels of horizontal orientation are shown. (R) odd, (C) even-symmetrical S-operator, (D) C-operator. While the S-operators show different bands of activity, depending on the underlying pattern, the C-operator represents edges and lines uniformly and irrespective of their polarity. The response to the large disk illustrates the orientation selectivity. Image dimensions, 512 x 512 pixels; 0 of filter envelopes, 2.5 pixels.

968

FRIEDRICH HEITGER ef al.

by the (complex valued) spectra of S-Gabor functions and the angular variation by a power of a cosine function, 9(r)

F(r, cp) = W(r). cos2m(cpo - cp1

(5)

where ‘pOdetermines the orientation of the filter and m the sharpness of orientation tuning. Since the mean value in space is represented by the value of the spectrum at the origin, which is zero for S-Gabors, the 2-D filters have again zero means. We used a discrete Fourier transform of the one-dimensional filters, equations (1) and (2). The polar representation was transferred to a Cartesian sampling raster using conventional interpolation techniques. Given this 2-D Fourier domain representation of the S-Gabor filters, the convolutions with images were done by multiplying filter and image spectra and transforming the result back to the space domain. The results presented below were obtained with 6 orientation channels spaced 30” with a width at half height of k23.5” (m = 4) and spectral bandwidths of 1.5 octaves. Figure 2 shows, for vertical orientation, the corresponding even and odd filter masks [Fig. 2(A) and (B), left] and their spectra (right, real plane for the even and imaginary plane for the odd S-Gabor). In analogy to simple receptive fields we call these filters “Soperators”. C-operator

Following Adelson and Bergen (1985) and Morrone and Burr (1988), we then combine the responses of S-operator pairs to a “local energy” representation defined by

L

r D

FIGURE 4. Localisation

of edges and lines. (A) shows the profile of a one-dimensional intensity variation consisting of up- and downgoing edges and bright and dark lines. (B) and (C) are the convolutions with odd and even S-operators, (D) is the output of the C-operator. Note the similarity of the C-operator resm to edp and lines. Its peaks indicate exactly the positions of~thes futures.

either a maximum or a minimum of response occurs at

(6) the position of the edge or line, and this only in the where Oi,oddand Olevenare the convolutions of the image with the odd and even S-operators of orientation cpi (i=l,... ,6). We use the square root in order to obtain the same (linear) contrast response as for the S-operators. We call this the “C-operator” in analogy to complex receptive fields. Figure 3 shows the result of these operations on an image of 3 grey-levels [Fig. 3(A)], Fig. 3(B) and (C) show the convolutions with the odd and even S-operators of horizontal orientation, Fig. 3(D) the representation produced by the C-operator. Similarly to most simple cells in the visual cortex, the S-operators show ridges and valleys of activity depending on the type of pattern (edge or line) and contrast polarity (light or dark), while the C-operator produces similar monomodal responses to any of these features, which is typical for complex cells. Figure 4 illustrates the responses of S- and Coperators for an intensity profile that consists of up- and down-going edges and dark and light lines, as shown in Fig. 4(A). The profiles below represent the convolution with the odd and even S-operators [Fig. 4(B) and (C)l, and the output of the C-operator [Fig. 4(D)]. It can be seen that each of the S-operators provides only ambiguous information about the localisation of features since

responses of the operator of corresponding symmetry. On the other hand, both features are localised unambiguously and accurately by the maxima of the C-operator. In fact, the line response of this operator is simply the envelope of the S-operators, i.e. a Gaussian, and the edge response is a similar monomodal symmetrical function. Thus, the C-operator provides a unified representation of edges and lines (and combinations of both). While the C-operator allows to localise intensity discontinuities easily, its output carries no information about their type and polarity. We suggest that this information can be retrieved to a large extent from the responses of the S-operators at the point of localisation (cf. Morrone 8c Burr, 1988). The localisation is the prerequisite for this analysis. End-stopped operators

The S- and C-operators defined in the previous section were designed to represent essentially one-dimensional signals, i.e. signals that are constant along one orientation and can therefore be completely character&d by their variation in the orthogonal direction. These operators represent accurately the orientations of straight edges, lines etc. as well as their locations (in the orthogonal

SIMULATION

OF NEURAL

CONTOUR

MECHANISMS

969

(A)

FIGURE 5. “End-stopped” operators are constructed by taking the rectified differences of two or three identical, displaced C-operators, as symbolised by the ellipses (major axes indicate optimum orientation). We call these operators “single-stopped” (A) and “double-stopped” (B). The positive and negative inputs have balanced weights. Axes indicate the coordinate origin.

direction), but do not provide an explicit representation of terminations and deviations from straightness. Thus, the maps of S- and C-operators have to be complemented. This is exactly the purpose of the “end-stopped operators” to be defined in this section. In principle, these new operators produce the first and second derivatives (with rectification) of the C-operator representations in the direction of their orientation. The physiological parallel suggested by the term “end-stopped” will be discussed below. We define anti-symmetrical and symmetrical endstopped operators, as shown schematically in Fig. 5, and refer to these as single-stopped and double-stopped operators. For horizontal orientation, we define &(x, Y) = [C(X - d, Y) - C(X + 4

YII’

(7)

for the single-stopped and &(x, Y) = (C(x, Y) - +[C(x - 24 v) + C(x + 24

Y)I}+

(8)

for the double-stopped operator, where C is the Coperator of horizontal orientation, d is a constant and ( }+ denotes clipping of negative values. Thus each operator consists of two [Fig. 5(A)] or three [Fig. 5(B)] “subunits” of the same size and orientation preference, displaced along the preferred orientation by the distance 2d. Other orientations were obtained by rotating these operators about the origin. Steps of 30” were implemented yielding 12 orientations of single- and 6 orientations of double-stopped operators (because the latter have 180” rotational symmetry). The separation constant d was chosen according to the “length” of the C-operator (which in turn depended on the space constant and width of angular function of the S-operators). It was chosen so that the response of the single-stopped operator to a line-end was three-quarters that of the C-operator to a

line. Figure 6 schematically illustrates the single- and double-stopped operator responses to simple figures as shown in Fig. 6(A). Figure 6(B) shows the horizontal channel of the C-operator representation, Fig. 6(C) is the profile of Fig. 6(B) along the dashed line. Figure 6(D) shows the corresponding response profile of the horizontal single-stopped operator sensitive to left terminations, and Fig. 6(E) the response of the double-stopped operator. The clipped negative responses are indicated by dotted lines. One can see that the activity profile of the single-stopped operator is monophasic for line-ends and biphasic for a dot or a disc, while the reverse is true for the double-stopped operator. Note that the maximum activity of the single-stopped operator accurately coincides with the line-ends, but not with the centre of the dot or the tangent point on the disc. The reverse holds for the double-stopped operator. Since single-stopped receptive fields in cortical cells are not well documented in the literature and their existence may not be generally recognised we include here an example of such a cell recorded in area V2 of the monkey (Fig. 7). Responses to edges of different lengths are shown in Fig. 7(A); the optimum length is depicted in the inset on the right, together with a map of the receptive field. Lengthening the edge further, reduced the responses. Figure 7(B) shows responses to a long edge in different positions of the receptive field. This edge evoked the strongest response when its left corner just covered the excitatory part of the receptive field (b), while the right corner in the corresponding position produced no response (a), indicating asymmetrical endstopping. The curve of Fig. 7(B) may be compared with the response profile of the single-stopped operator in Fig. 6(D). Figure 10 illustrates the performance of these endstopped operators on a test pattern composed of 3 grey

I I

I I I

. . . . .

. .

I

1

. .

... .+

. .................

.................

I

I

. ..__....___...._..

. . . . . . . . . . . . . . . .._.........

I I I ,

971

SIMULATION OF NEURAL CONTOUR MECHANISMS

levels [Fig. 10(A)]. Figure 10(B) and (C) show the output of the single- and double-stopped operators, summed over the 12 or 6 orientations, respectively. One can see strong activity at end-points, corners and strongly curved edges. However, both operators show also traces of activity at straight lines and edges; the single-stopped variant tends to produce a double trace, the doublestopped variant a single trace. This “false” activity deserves some attention and will be the subject of the next section. False responses

The activity of the end-stopped operators at straight edges and lines is a consequence of the use of displaced subunits with identical characteristics. This is illustrated in Fig. 8. When a line or edge is centred on the positive subunit and rotated, starting from the optimum orientation, the positive subunit is affected only by the change of orientation, but the activity of the negative subunit(s) is reduced also by the non-optimal position. Thus the negative signal(s) will not cancel the positive signal. While equal selectivity for spatial frequency and orientation and balanced weights of the displaced subunits are desirable for the purpose of detecting terminations and curvature, the “false responses” at straight edges and lines are disturbing for subsequent stages of analysis. Surround inhibition

Cortical neurons with strong end-inhibition do not respond at all to long edges or lines, and this feature is apparently independent of the stimulus orientation. Thus “false” responses do not seem to occur, although, to our knowledge, this question has not been studied systematically. Apparently the design of end-stopped receptive fields avoids this problem; either excitatory

(A)

and inhibitory subunits are composed differently, or an additional inhibitory input eliminates the false responses. There are indeed many reports on “inhibitory side bands” or “side stopping”, and “cross-orientation inhibition”, that make the latter possibility a plausible one (Blakemore & Tobin, 1972; Bishop, Coombs & Henry, 1973; Maffei BEFiorentini, 1976; Nelson & Frost, 1978; Orban, Kato & Bishop, 1979b; Morrone, Burr 8z Maffei, 1982; De Valois, Thorell & Albrecht, 1985; Allman, Miezin & McGuinness, 1985), but the exact mechanism has not been identified. We have solved the problem by adding to the above scheme of end-stopped operators [equations (8) and (7)] a mechanism of “surround inhibition”:

Ek,= Psi - g *(4 + 4)1+

(9)

where i is the index of the orientation channel, Zt and Z, are the surround inhibition operators defined below, and g is a gain factor set to 0.5 for the present results. Zt and Z, were defined by Zt(x,Y)=

f [Ci(x,Y)-wW,.c,(xl,Yj)l+

(11)

zr(x, Y) = F [ci(x, Y) - wr~c(i+N/2)(xl~YI)1+

Cl21

i=l

and

i=l

with x]=x

+r*cos(i-A.a)

yi=y

+r*sin(i*Aa).

(B)

FIGURE 8. The problem of “false. responses”. End-stopped operators of the type of Fig. 5 respond also to one-dimensional image structures. With a straight bar of non-optimum orientation, for example, the negative input component is smaller than the positive for the positions illustrated. Thus the single+stopped operator produces false responses flanking the bar (A), the double-stopped operator right on the bar (B).

FRIEDRICH HEITGER et al.

972

N is the number of discrete orientation channels [channel Cci+N,zjis orthogonal to C,], Aa the orientation sampling interval, w, and w, are weighting constants and I is the radius of the inhibitory surround. r was set equal to d, i.e. half the separation of the subunits [cf. equations (7) and (S)]. The mechanism is illustrated in Fig. 9. For Z,, differences between peripheral and central activities are calculated within each orientation channel (tangential inhibition), for Z,, between orthogonal channels (radial inhibition). The clipped differences are summed; thus, a signal on the periphery contributes to the inhibition only if, after attenuation with w, it exeeds the corresponding signal in the centre. This has the effect that surround inhibition acts in the vicinity of continual lines and edges, but does not much affect the responses at line-ends and corners. The inhibitory signals Z,and Z,are shown in Fig. 10(D) and (E). One can see that both are strong at the straight lines and edges, but weak at the line-ends and corners, and that this pattern of activity corresponds to the “false activity” of the representation above: Ztis in register with the activity produced by the single-stopped operator [Fig. 10(B)], Z, with the activity of the double-stopped operator [Fig. 10(C)]. At end-points and corners the inhibitory signals are weak or absent. Key -points Figure 10(F) shows the performance of the “corrected” end-stopped operators E; and EL [equations (9) and (lo)]. Note that Z, and Z, are shown separately in Fig. 10(D) and (E) only for illustration, but a combination of both is used for the correction of either type of endstopped operator. The sum of EL and Eb, summed over the orientations is shown in Fig. 10(F). One can see that this representation selectively highlights the endpoints, corners and segments of strong curvature, without “false” activity along the straight segments. The correction by surround inhibition had the additional

effect of reducing the spread of the signal near end-points. We define the “key-points” of an image as the peaks (local maxima in a 3 x 3 neighbourhood) in the summed end-stopped representation. The idea is that the peaks of this (scalar) representation provide an accurate localisation of the relevant 2-D features of the image, so that the further stages of processing can rely on the values of the end-stopped operators at these points. This is analogous to the role of the maxima (ridges) of the C-operator representation in the analysis of contrast borders discussed above. The knowledge of the keypoints of an image reduces the computational burden of the further analysis enormously because their number is generally low, even in complex images. The competence of the key-point map for grey-valued images of complex scenes will be demonstrated below (Fig. 12). Another important aspect of the use of key-points concerns the evaluation of the different end-stopped operator signals. The pattern of activity across the different channels contains important information about the 2-D signal variation. On end-points and corners, for example, we find stronger responses of single- than double-stopped operators, whereas on curved segments the reverse holds. The activity also varies characteristically across the orientations and directions. This information can be used to classify the local configuration. We found, however, that valid patterns of endstopped operator responses are generally obtained only at the singular points of comers, line-ends, etc. Therefore, these points have to be localised correctly first. The map of key-points provides this localisation. Figure 1l(A) and (B) shows the responses of the individual channels of the single- and double-stopped operators, respectively, at the key-points. Responses are shown in the form of a local polar plot, where the length of the rays originating from the key-points is proportional to the strength of the end-stopped signal in a given

(A) FIGURE 9. Scheme of “surround inhibition’* ursd to compensate false responses. It consists of a tangential component (A) and a radial component (B). Each is a sum of rectified differences between two C-operators, one on the periphery and one in the centre. The tangential component suppmases like orientations at offset positions. The radial component produces “cross-orientation inhibition”. It is strong on I-D image structures, whereas in the presence of multipk orkntations the antrt fields are able to compensate

the surround signals. The surround fields are weighted with a constant (w, = operator and w, = 0.25 for the radial operator).

I for the tangmtial

SIMULATION

OF NEURAL

direct ion. The double-stopped signals have been plotted in b(>th directions for each orientation, the singlestopp ed signals in the direction of termination, i.e. the direct ion of decreasing C-signal. It can be seen that each 2-D feature generates a characteristic response pattern.

CONTOUR

MECHANISMS

At corners, for example, we find two major componer 1ts pointing in the directions opposite to the constitue :nt edges, in the single-stopped representation. On kc:ypoints where a line ends or joins another line or surfac=, one component prevails. The single-stopped responses at

FIGURE 10. Illustration of the corrected end-stopped operator scheme. (A) Test image, (B and C) activity of single- and double-stopped operators, summed over orientations, (D) and (E) tangential and radial components of surround inhibition. The inhibition patterns selectively match the false responses of the end-stopped operators at straight contour segments. There is no inhibition at end-points, comers and terminations. Thus the end-stopped signals are left unattenuated. Note that the radial inhibition is reduced for strong curvature, as on the small disk, and entirely absent on the dot (E). (F) Shows the combined responses of the corrected end-stopped operators. Terminations, comers, junctions and segments of strong curvature are highlighted selectively. “R

32/s

4

913

974

FRIEDRICH

HEITGER

et al.

FI

Simulation of neural contour mechanisms: from simple to end-stopped cells.

Early stages of visual form processing were modelled by simulating cortical simple, complex and end-stopped cells. The computation involves (1) convol...
3MB Sizes 0 Downloads 0 Views