J. theor. Biol. (1975) 49, 355-376

The Informon in Classical Conditioning A. M. UTTLEY Laboratory of Experimental Psychology, University of Sussex, Falmer, Brighton, Sussex, BN1 9QG, England (Received 21 January 1974) An informon network of interconnected units with the property of automatic pattern recognition has been described earlier in this journal; the connection between any two units has a variable conductivity proportional to the mutual information between the output signals of the two units. It has been found that a single unit, with its set of converging

input pathways, reproduces much of the known behaviour of an animal under classical conditioning to compound stimuli. There are five corre-

spondences between psychological and network concepts. The explanation here offered of animal learning in terms of a physical network is a first step towards a physiological explanation. There is already some evidence that the informon law of variable conductivity obtains in pathways of cerebral cortex. 1. Introduction

A study has been made of networks of interconnected units whose input and output signals are binary; the pathway from the output of one unit to the input of another has a conductivity which varies as the mutual information function between the signals at either end; the output signal of a unit has a frequency which is proportional to the total excitation reaching it from all the pathways converging on it. A set of such pathways which converge on a single unit, as in Fig. 1, has been called an informon; the pattern recognition properties of such a network have already been described (Uttley, 1970). The concept of an informon can have different levels of representation in different sciences. Mathematically it is a set of equations relating the frequency of output signals to input signals. As a psychological model of learning the input signals correspond to stimuli and output signals to responses; the informon theory predicts the relation between them. By simulation studies it has been found that a single informon can exhibit many of the behavioural properties of classical conditioning. In the first part of this paper the mathematical ideas of an informon are summarized, and equations are derived for the steady state of an informon 355

356

A. M. UTTLEY

when the statistics of the input signals are stationary. In the second part an informon network is considered under conditions of classical conditioning to compound stimuli, with inputs from an unconditioned stimulus, US, and from sets of conditioned stimuli, CS. Simulation studies show that conductivities of pathways behave very much like associative strengths of CS, during conditioning, conditioned inhibition, Kamin blocking and when contingencies between CS and US are varied. If the basic equation of the informon is expressed as a difference equation, it is found to be very similar to the difference equation developed by Rescorla & Wagner (1972) to describe the changing associative strength of a conditioned stimulus. By comparing corresponding terms in the two equations, correspondences are drawn between some psychological concepts and the physical concepts of a network. 2. Informon Principles An informon is a set of pathways converging on a single unit as is shown in Fig. 1. The output of the unit is a binary signal Y; the inputs to the pathways are also binary signals. For each input X, the corresponding pathway has a variable conductivity yi; the total signal reaching the unit is

The probability

of the output Qy)

Y is given by =

YO+

TxiYi

where y. is a constant.

FIG. 1. The informon: a linear separation unit with pathways of variable conductivity.

(1)

THE

INFORMON

IN

CLASSICAL

CONDITIONING

357

An informon is therefore a linear separation unit (Highleyman, 1962) with variable parameters and a probabilistic output. The right-hand side of equation (1) is a linear discriminant function which ranks input sets according to their probability of possessing the property Y. The conductivity of each pathway is made proportional to the mutual information function (Shannon, 1949; Fano, 1961) between the signals at its input and output; this function is defined as

It follows that y will be zero if P(X & Y) = P(x)P( Y), i.e. if X is statistically independent of Y; in general, y may be positive or negative. Probabilities can be used only if the ensemble of events is defined in some way; for events occurring serially in time it has been decided to substitute for probability the frequency of events relative to an arbitrary maximum frequency F,,,. In this paper time has been made quantal, so that the maximum frequency F, is not arbitrary but is the reciprocal of the time quantum. By making this quantum the unit for measuring time, F,,, becomes unity, and relative frequency has the same numerical value as probability. Equation (1) is therefore rewritten as

also

F(Y) = YO+ CxiYi; I

(2)

(3)

From equation (2), if an input Xi occurs, the relative frequency of the output increases by a quantity yi, the information given by Xi about Y. Because yr may be positive or negative the output frequency may increase or decrease; this is possible only if the output has some basic frequency from which to deviate; this is one reason why the constant y. is introducted in equation (2). The actual value of y. is unimportant in many cases; in a network with the output from one unit forming the input to another, it is mutual information, signalled by changes in frequency, which is communicated. It is necessary to consider frequencies averaged over short and long periods. Frequency in equation (2) changes rapidly with every change in the inputs X,; on the other hand frequencies in equation (3) are averaged over a past of long period, so as to give a significant value for y, which will therefore vary only slowly with time. Exponential weighting functions have so far proved adequate for simulation of behavioural data; accordingly in equation

358

A.

M.

UTTLEY

(3) the term F(Y) is replaced by G(Y) where r=O K exp T/T G(Y),=+~ c r=--a3

Similarly for I;(X,), and I;(Xi & Y),. The suffix t will be omitted in future; equation (3) is therefore rewritten in the form (4)

The system so far described is shown in Fig. 2. Two feedback pathways can be seen in the system; from equation (2) F(Y) depends on yi; from

FIG. 2. Functional diagram of informon with frequency generator in feedback loop. G, smoothing function; fg, frequency generator; In, logarithmic function.

equation (4) yi depends on G(Y) and on G(Xi & Y). The system is stable only if k in equation (4) is made negative, as the following qualitative argument shows. If yi is positive, F(Y) will increase in the presence of Xi hence the mutual information between Xi and Y will increase; since k is negative y1 will decrease. Conversely, if yi is negative it will increase. (A)

THE

INCLUSION

OF INFORMATION

FROM

AN

ABSENT

INPUT

From equation (2), if Xi is present, F(Y) is increased by ye the information gain about Y. But if Xi is absent there is also a change in information about Y. For example, in a system to recognize handwritten numbers, if a horizontal line is known to be absent, there will be a change in the probability of each number; the probability of a 4 will decrease, the probability of a 6 will

THE

increase. Formally, of X,, is

INFORMON

IN

CLASSICAL

the mutual information

359

CONDITIONING

between Y and Xi, the absence

IQ{ ;lf;);(;;}* Letting ri = k

G(Xi & Y)

In,

i

--Z-G(Xi)

(5)

* G( Y) 1

equation (2) becomes extended to FtY)

=

YO +

TCxiYi

(6)

+8t7i)-

It can be shown (e.g. Uttley, 1970) that, in the steady state, the average information given by the absence of an event is equal and opposite to the average information given by the presence of that event; i.e. G(Xj)yj + { 1 - G(Xj)}7j = 0. (7) It follows from equation (6) that, in the steady state, the average value of F(Y) is yo; i.e. (8) G(Y) = YO. This extension to a symmetrical informon is optional; its most obvious consequence is an increase in feedback gain. The behaviour of the two forms is, in many cases, very similar. However, the equations for the steady state of a symmetrical informon often have simple informative solutions; in this paper the unsymmetrical equations are not treated.

FIQ.

3. Functional

diagram of informon

with frequency generator outside feedback

loop. T.B.

24

360

A.

M.

UTTLEY

There is less fluctuation in y and, hence better discrimination by the informon, if the frequency generator of Fig. 2 is placed outside the feedback loop, as in Fig. 3. Equations (4) and (5) then become (9) and

This is because the ensemble average of G(Y), written as G(Y) is given by Gt Y) = G{Ft Y)) ; similarly G(X, & Y) = G{Xi +F(Y)}. Hence

--___ G(Xi & Y) G(Xi * J’t Y>> ti(X3G(Y) = G(X,)G(F( Y)> ’ (B)

STEADY

STATE

EQUATIONS

(i) One input

From equation (6) the frequency of Y whenever A’, occurs is yo+ yI; i.e. Xi - F( Y)/F(Xi) = y. +yi; so, in the steady state G{Xr * F( Y)}/G(XJ = y. + yi also. Combining this equation with equation (8), equation (9) becomes yi = k In,

{

I+?‘1

roJ

which has only one real solution if y. is non-zero, i.e. yi = 0. (ii) Two inputs Xi and Xj, with high feedback gain k

In this case the frequency of Y when Xis present will depend on whether Xj is present or absent, being increased by yj or Yj respectively. It follows from equation (6) that, in the steady state,

G{Xi ----.- *J’(Y)} G(XJ

= yo+yi+yj

G(Xi & Xi) _ G(Xi & Xj) qii>-~ ~-e yj __-~. (Xxi)

From equation (7) this simplifies to

G(X#lY)) ___-‘3X,)

= Yo+Yi+Yj~-G(xj)

(11)

THE

INFORMON

Remembering

IN

CLASSICAL

361

CONDITIONING

that GF( Y) = y0 equation (9) becomes G(Xi _-_.-

‘yi

kX.1 .._ !- -l

k G(XJGF(Xj) Now let k, the feedback gain, be high so that the left-hand side of this equation, and hence the right-hand side, is approximately zero. Then since In 1 = 0,

Combining

equations (I 1) and (12)

G{Xi *F(Y)} G(XJ

z ‘O’

From equation (11) this would also be true if yt and yj were both zero. In other words, in the steady state the average effects of the inputs Xi and Xj on G(Xi . F(Y)} cancel approximately. (iii) Many inputs

For n inputs, two of which are Xi and Xj, equation ( 12) becomes Yi+ i

jtj

Y’-

G(Xj)

G(Xi & Xj)

‘l-G(Xj)

GtXiMXj)

-

1

>

ZO

(13)

where i j#i is summation over all inputs but i; there are n such equations for 1 I i I n. If all y are variable, there are n simultaneous linear equations in the n variables, with no constant terms; hence, in the steady state, alI y are zero. If, however, one of the inputs, X, say, has a pathway with fixed conductivity yt, then one of the n equations reduces to yt = a constant. The other (n - 1) equations in the (n - 1) variables all possess constant terms of the form

so in general, all variable y tend to finite values. The remainder of the paper is concerned only with this use of an informon.

362

A.

M.

UTTLEY

3. Classical Conditioning In this section it will be shown that a single informon reproduces many of the phenomena of classical conditioning, the changing conductivity of pathways corresponding to the changing associative strengths of conditioned stimuli. Before considering particular examples some general points must be discussed. In developing an informon theory of the adaptive behaviour of a nervous system, one must consider whether an input signal to an informon is to be regarded as corresponding to a single nerve impulse in a neurone, or to a psychological stimulus such as a light flash or the sound of a bell; the intensity of stimuli will be discussed later. Under certain simplifying conditions, the informon can be regarded either way, for the following reasons. Suppose that a particular stimulus always gives rise to a constant number of nerve impulses. Then, in a psychological model, each stimulus will be represented by a single binary input signal, and in a neurophysiological model each stimulus will be represented by a train of binary signals. Whichever system of representation is used the mutual information between the signals at a pair of such inputs will be the same. For example in Table 1, where every stimulus corresponds to four nerve impulses, the mutual information between the two stimuli X and Y is the same as the mutual information between the corresponding nerve impulses, namely In 2 9/8. TABLE

1

Psychological stimuli and nerve impulse trains Stimllhsx Stimulus Y Impulse train X Impulse train Y

1 1 1 0 1 0 1 0 1 1 1 0 1111 1111 1111 0000 1111 0000 1111 0000 1111 1111 1111 0000

Mutual information depends on relative frequency so that, in simulation studies, conductivities will change according to the same equations and reach the same average steady state values, whichever form of representation is used. It follows that informon theory can form a bridge between psychology and neurophysiology because its basic equations can be used in both fields; they can give rise to predictions about animal responses when learning, and about changing conductivities in neural pathways. One further point must be made. Because the informon takes account of the co-occurrence of an input with the common output, it is sensitive to the co-occurrence of inputs. The informon can take account of the cooccurrence of conditioned and unconditioned stimuli, of the co-occurrence

THE

INFORMON

IN

CLASSICAL

CONDITIONING

363

of motor responses and reinforcing stimuli, and above all of the cooccurrence of different parts of a spatial pattern. But the informon alone cannot discriminate temporal pattern; for example, it cannot distinguish different time intervals between conditioned and unconditioned stimuli. However, if a short-term storage system is placed between input channels and an informon network, spatio-temporal pattern can be discriminated (Uttley, 1956, 1970). This extension is not fully treated in this paper, but it is discussed in one or two places. (A)

ONE CONDITIONED

STIMULUS

It has been known since the days of Pavlov that if a conditioned stimulus (CS) is repeatedly presented with an unconditioned stimulus (US), then the former will tend to evoke a response similar to that evoked by the unconditioned stimulus; US is said to reinforce CS. It has recently been found (Rescorla, 1968) that if CS is made statistically independent of US, i.e. if reinforcement is as likely when CS occurs as when it is absent, then CS develops no associative strength; furthermore CS develops an associative strength which is a monotonically increasing function of the mutual information between CS and US, becoming negative if US is less likely when CS occurs than when it is absent (Rescorla, 1969). Kamin & Schaub (1963) have shown that the rate at which a CS develops strength is monotonically related to the intensity of the CS, although the asymptotic strength of a CS is independent of its intensity; a more intense CS is said to have a greater salience. (B)

CONDITIONING

TO COMPOUND

STIMULI

Pavlov (1927) considered the combination of two CS A and X; he found that if the two CS were jointly presented with US, one might develop greater asymptotic strength than the other. Pavlov also showed that if A is reinforced by an unconditioned stimulus U until it develops a positive associative strength and then, alternately, A is reinforced and A and X combined are non-reinforced, X will develop an inhibitory effect; this is called conditioned inhibition, and is tested by presenting X with some stimulus B which has been previously reinforced to have a positive associative strength. More recently Kamin (1967) by a conditioned suppression paradigm, has shown a “blocking effect” which may be summarized as follows. If a CS A is reinforced to its asymptotic strength, and then A and X combined are reinforced, CS X develops no strength, while that of A remains unchanged. (C)

THE THEORY

OF RESCORLA

& WAGNER

Rescorla & Wagner (1972) have proposed an equation which predicts all the above facts; they suggest, firstly that associative strengths are additive,

364

A. M. UTTLEY

and secondly that the increment in strength at a trial is proportional to the difference between the asymptotic strength which US can support and the sum of the associative strengths of the stimuli being presented at that trial. Introducing a number of parameters, their equations for increments in strength of two CS take the form AV,= A.a,/!?(U.I-A-I/,-B*V,) (14) At’,= B*cr&J*I-A*V,-B-V,) (1% where A, B and U are binary signals representing the presence or absence of the two CS and the US, and VA = Associative strength of A, V, = Associative strength of B, tlA = Salience of A, ctB = Salience of B, j3 = Learning rate parameter of U, A = Asymptotic level of associative strength which U will support. Note that there is no increment in the associative strength of a CS when it is absent.

0.6

08-0.2

0.4

0.8-O-4

0.2 l.*

0.8-0.8

0 -0.2 -

0.4-0.E

-0.4 0.2-0.E

-0.6 -0.8

O-O.8 Trials

FIG. 4. Predicted strength of association of X as a function of different US probabilities in the presence and absence of X. The first number by each curve indicates the probability of US during CS; the second, the probability in the absence of the CS. (After Rescorla & Wagner, 1972.)

THE

INFORMON

IN

CLASSICAL

365

CONDITIONING

x

‘

x I

FIG. 5. Conditioned inhibition. CS A is reinforced to asymptote; then, alternately, A is reinforced and A and B combined are non-reinforced. Simulation of the equations of Rescorla & Wagner and of informon; time scales 300 and 80 chosen to give best fit. For this and all other simulations, Rescorla & Wagner+xosses Informon-dots Ordinate Y, scale 0.05. Ordinate y/yO; scale 0.05. aA = 0.1. nA= 1. aB = 0.1. n)j = 1. rz= o-4. n” = 1; y” = -0.2. j?= 0.2. T= 3200;k = 32; yo = 0.5.

Some examples will now be given of simulating the above equations. First of all suppose that a single CS A is always reinforced by U; then U and A being always 1 and B always zero, equation (14) states that, in the steady state when AV, is zero, VA = I; and V, approaches 1 exponentially. The example of variable contingency is given in Fig. 4, which shows that the final value of V, is monotonically related to the mutual information between A and U. Figure 5 shows the simulation of conditioned inhibition; Table 2 gives the cycles of events which are repeated in the first and second phases of conTABLE

Conditioned inhibition Phase 1

A B

00000000

c u

11111111 11000000

11000000

2 with background Phase 2 11000000 01000000 1 1 1 1 1 1 1 1 10000000

366

A. M. UTTLEY

ditioning. A background stimulus C is introduced which is always present; this step permits the system to respond indirectly to the absenceof CS A, although equation (14) does not allow a direct response. The blocking effect of Kamin (1967) is shown in Fig. 6(a), for which the cycle of events is as in Table 3. In all the examples so far, the salience of all CS has been taken as the same; to account for the conditioning of compound stimuli with different salience, Rescorla & Wagner introduce the factor a. Figure 7 shows the effect of

I

.

.,,

(b) FIG. 6. (a) Kamin’s blocking effect. CS A is reinforcedto asymptote;then A and B combined are reinforced. Time scale 120 W.R., 40 informon. (b) Plot of (a) for informon with time scale 400, showinggradual unblocking.

THE

INFORMON

IN CLASSICAL TABLE

CONDITIONING

367

3

Kamin blocking with background

A B c u

Phase 1

Phase 2

1 1 0 0. 0000 1111 1100

1100 1100 1 1 1 1 1100

increasing the salience of a single CS; this leads to a greater rate of conditioning, but has no effect on the asymptotic strength. Figure 8 shows the effect of conditioning a compound stimulus A and B, when A has twice the salience of B. A is said to overshadow B.

. . .x . . x’ . . :+, : :s. . ,x. .x.-.x.

. ..X.‘.X x.*x’ :x ,x” .x” * .x’ - .‘x” .:.; 9, . I

I

FIG. 7. ming curves for CS of diierent no = 1. Time scale, 240 W.R., 40 informon.

salience (a) ,, = 0.2; nA = 2. (b) A = 0.1;

?(’ xK - x. -k,

,x *x-K’

- .x Y FIG. 8. Overshadowing. Conditioning a compound stimulus A and B, where A has twice salience of B. Tie scale, 200 W.R., 40 informon.

368

M.

A. (D)

THE

UTTLEY

INFORMON

THEORY

Suppose that a CS excites a pathway of variable conductivity in an informon, and that US excites a pathway of fixed conductivity in the same informon; then the steady state equation (12) can describe the asymptotic effect of reinforcing a single CS. The network is shown in Fig. 9; A is the CS and U is the US; the output of the informon determines a conditioned response CR. U, via some innate pathway, gives rise to an unconditioned response UR. Equation (12) takes the form __G(U) ..-

YA+Yul-G(U)

(16)

FIG. 9. Classical conditioning in an informon. U unconditional stimulus. A conditioned stimulus. CR conditioned response. UR unconditioned response. U excites the informon via a pathway of fixed inhibitory conductivity yu, A via a pathway of variable conductivity YA.

First suppose that A is always reinforced; then G(A & U) = G(A), and equation (16) reduces to YA+Yu = 0. It follows that, to fit the psychological fact that A develops a positive associative strength, yu must be negative; the pathway from U to CR must be inhibitory. Next suppose that A and U are statistically independent, then G(A & U) = G(A)G(U)

and equation (16) reduces to YA = 0, which is in accord with the facts. Finally, equation (16) shows that, in general, yA develops an asymptotic strength proportional to W 8~ W -1 > { WWW which is approximately equal to the mutual information between A and U.

THE INFORMON

IN CLASSICAL

369

CONDITIONING

Again, this is in accord with the facts. Figure 10 shows the simulation of the last example, for the informon and for Rescorla & Wagner under the same conditions. The cycle of events is as in Table 4. TABLE

4

Negative information A

c u

from which P(Ul.4)

11011010 11111111 10110110

= $, P(U/A)

= 3 and the mutual information,

is -0.06. The initial transient response is different for the two systems; for the informon it is affected by the a priori probability of the background stimulus.

FIG. 10. Partial reinforcement with negative mutual Time scale 200 W.R., 80 informon.

information

between U and A.

Turning now to compound stimuli, the steady state informon equation (13) for many inputs can be applied to the case of conditioned inhibition. Inputs A and B represent two CS, and input U the US which excites a pathway of fixed conductivity yu. Table 5 shows the two cycles of events which are repeated in the first and second phases respectively of conditioned inhibition; because the informon here discussed has an exponentially decaying storage system, the intertrial intervals must be represented.

370

A. M. UTTLEY

5 Conditioned inhibition TABLE

In phase 1 the following cycle is repeated; there are N events per cycle AllO---O BOOO---0 UllO---0 In phase 2 the following cycle is repeated; there are N events per cycle AllO---O BOlO---0 UlOO---0

To determine the final steady state of yA and yB, equation (13) is applied, for phase two only, to the inputs A and B; hence yA+& J&

(Z-l)+&

(E-1)=0;

(S-l)+y,+&(-l)=O.

whence YA

N-2 = -YU . N-l

and ye = yu.

Remembering that yu is negative, CS B becomes inhibitory with the same strength as US; CS A retains a positive strength slightly less than before, negligibly so if the intertrial interval is large. Simulation of such a three input informon network for N = 8 is shown in Fig. 5, together with a simulation of the model of Rescorla & Wagner under the same conditions; the transient behaviour of the two models is very similar. Although the steady state equations (12) are useful, there are cases where they are necessary but not sufficient, because some of them are identical; this occurs when one input is a logical function of others. It is then necessary to simulate the fundamental equations (6), (9) and (10) from which equation (12) is derived. As an example, Kamin’s blocking effect is simulated in Fig. 6(a) for the informon and the model of Rescorla & Wagner; the cycle of events is given in Table 6. For the informon Fig. 6(b) shows that CS B slowly becomes unblocked. This is due to the exponentially decaying storage system; the informon eventually “forgets” phase 1. In the model of Wagner & Rescorla there is no such decay.

‘THE

INFORMON

IN

CLASSICAL

CONDITIONING

371

TABLE 6 Kamin blocking A B u

1100 1100 1100

1100 0000 1100

To allow for the effects of different CS having different salience, Rescorla & Wagner introduce the factor a. Exactly the same effects are produced in the informon by introducing a corresponding factor n, the number of parallel identical pathways from an input to the informon; an example of this is shown in Fig. 11; the CS A, with nA parallel pathways each of conductivity yA, will have an overall pathway conductivity n,y,.

A {

B I

FIG. 11. To account for salience stimuli excite different numbers of parallel pathways in an informon model.

Figure 7 shows the effect of reinforcing two single stimuli of different salience, for the informon and for the model of Rescorla & Wagner. Figure 8 shows the effect of reinforcing a combined pair of stimuli of different salience. (E)

DIFFERENCE

EQUATIONS

FOR

THE TWO

THJZORIES

The similarity in behaviour predicted by the two theories arises from the similarity of their difference equations; that of Rescorla & Wagner is

372

A.

M.

LJYTLEY

equation (14). In the appendix the difference equation for the informon has been derived from its basic equations (6), (9) and (10). For the symmetrical informon

approximately. Equation (14) of Rescorla & Wagner does not include terms like w,jri which allow for information from absent conditioned stimuli; it corresponds more closely to the equation Ayi = 2:, --

{ $$j

-l}LxiYi,

(17)

which approximates to that of the unsymmetrical informon. Equation (17) will now be extended to include salience, i.e. to the example of Fig. 12; an unconditioned stimulus X, excites nLI parallel pathways each of fixed conductivity yu,* two conditioned stimuli X, and X, excite corresponding numbers of pathways nA and nB, with variable conductivities yA and yB. Equation (17) becomes nAyA

=

2

{ &

--I}

{X”n,y”+X,n,yA+XBnByB);

(I81

similarly for Aye. For comparison equation (14) of Rescorla & Wagner is here repeated. * VA-B* V,>. AVA = AaJ(U*I--A (14) Firstly, since k and yu are both negative, corresponding terms in the two equations have the same sign; US makes a positive contribution and CS a negative contribution to the right-hand side of each equation. Corresponding terms in the two equations suggest a set of correspondences between psychological concepts and those of a physical network; they are set out in Table 7. There is an important difference between the two equations (14) and (18) regarding absent stimuli, even though the information from absent stimuli has been omitted from equation (18); i.e. terms such as X,n,vA. The first term of the right-hand side of equation (14) is the term A which is 1 for the presence of CS A and 0 for its absence; this ensures that in the absence of a CS its associative strength cannot change. On the other hand, for the informon the corresponding term is

Again X, is either 1 or 0, so its average value G(X,) lies between 1 and 0.

THE INFORMON

IN CLASSICAL TABLE

CONDITIONING

373

7

Correspondencesbetween psychological concepts and those of a physical network Psychological concepts of behaviour. Rescorla & Wagner

Functional concepts of a network. Informon Theory

U, A, B; stimuli V,; the associative strength of CS A.

X,, X,, X,; input signals. nrya; the overall conductivity of nl parallel pathways, each of conductivity ya, from CS A, to the informon. nA; the number of parallel pathways from CS A to the informon. k/Ty,; the rate of decay of an exponentially weighted storage system. nuyu; the overall conductivity of all the pathways from US to the informon. (A negative quantity.)

aA; the salience of CS A. 8; a learning rate associated with US. 3.: the asymptotic level of associative strength which US will support.

Consequently the term

is positive when X, = 1 and negative when X, = 0; there is an increment in 7” when X, is absent which is of the opposite sign to that when X’ is present. It can be shown that, in the steady state, the positive and negative increments cancel on average. (i) The time interval between CS and US

It has been shown that after conditioning, yA+ yrr = 0; consequently it can be seen from Fig. 9 that if US occurs alone UR occurs, if CS occurs alone CR occurs, but if both CS and US occur only UR occurs. Classical conditioning differs in different animals and different conditions but in general, (i) CS must occur before US for optimal conditioning; (ii) CR occurs immediately after CS and often before US; (iii) CR is not then suppressed when US occurs.

However, in many cases, it is not possible to distinguish CS from US. The effect of varying the time interval between CS and US is sketched in Fig. 12; there is a small effect, known as backward conditioning, for negative values of the interval. The phenomenon of forward conditioning is reproduced

374

A. M. UTTLEY

PIO. 12. Maximum associativestrength, V,, as a function of the time interval between and CS.

US

FIG. 13. The effect of time interval betweenUS and CS introduced into an informon by means of prior short-term storage.- - - -, additional inhibitory pathways.

in an informon if CS gives rise to an input which decays in a comparable period; the phenomenon of backward conditioning is reproduced if US gives rise to an input which decays in the much shorter corresponding period. The degree of conditioning depends on the amount of overlap between the two decaying input functions; formally, the function in Fig. 13 is the mathematical convolution of the two decay functions. The extended model is shown in Fig. 13; CR correctly occurs immediately after CS and before the inhibitory effect of US has begun. Two possible additional inhibitory pathways have been shown; the first would ensure that once CR had started US could not inhibit it; the second would ensure that CR inhibited UR. The phenomenon of trace conditioning, when there is a long time interval between CS and US, will not be discussed; it would require a full treatment of temporal pattern recognition. The author has had valuable discussions with Professor R. A. Rescorla, Dr M. S. Halliday and Dr R. Morris; Miss P. Pickbourne has written the computer simulation programs. The research has been supported by a grant from the Science Research Council of Great Britain.

THE

INFORMON

IN

CLASSICAL

375

CONDITIONING

REFERENCES M. (1961). Transmission of Information anda Statistical Theory of Communication. London : Wiley. HIGHLEYMAN, W. H. (1962). Proc. Znstn Radio Eitgrs 6, 1501. KAMIN, L. J. (1967). In Miami Symposium on the prediction of behaviour: Aversive stimrdation (M. R. Jones, ed.) pp. 9-31. Miami: University of Miami Press. KAMIN, L. J. & SCHAUB, R. E. (1963). J. camp. physiol. Psychol. 56, 502. PAVLOV, I. P. (1927). Conditioned Reflexes (translated by G. V. Anrep). London: Oxford University Press. RESCORLA, R. A. (1968). J. camp. physiol. Psychol. 66, 1. RESCORLA. R. A. (1969). J. camp. physiol. Psychol. 67,504. RESCORLA, R. A. & WAGNER, A. R. (1972). A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and non reinforcement. In Classical Conditioning II (A. Black & W. F. Prokasy eds). New York: Appleton-Century-Crofts. SHANNON, C. E. &WEAVER, W. (1949). The mathematical theory of communication. Urbana: University of Illinois Press. UTTLEY, A. M. (1956). Ann. math. Stud. 34, 1. U~LEY, A. M. (1970). J. theor. Biol. 27, 31. WAGNER, A. R. & RESCORLA, R. A. (1972). Inhibition in Pavlovian Conditioning. In Inhibition and Learning (R. Boakes & M. S. Halliday, eds). London: Academic Press. FANO,

R.

APPENDIX An approximate incremental equation, for small values of y,. Equation

(9) states that G{Xi * f’(Y)} yi = k In2 G(Xi)G(F( Y)] ’

The time quantum is taken as the unit of measurement,

Ayi = ‘+

(Al) so At = 1 and hence

AG{Xi . F(Y)} ~_-~AG(Xi) AG{F( Y)} G(Xi.F(Y)) G(XJ --iqFjy

Exponential weighting of past events is approximately values of T, by the difference equations

1

(0

*

achieved, for large

AG(Xi) = {Xi-G(Xi)}/T,

AW’( Y>>= F’( Y)- WC V)IIT

and

AG{ Xi . F( Y)} = [Xi . F( Y) - G{ Xi .F( y)}]/T. Substituting

in equation (A2)

Ay,= ;; T.B.

-. Xi’F(Y) .-_~-.G{Xi*F(Y)}

Xi.~.. _ _F(YL + 1 . _ __... G(Xi) G(F(Y)) I

643) 2.5

376

A.

M.

UTTLEY

If yi is small equation (Al) can be written in the approximate

G{X,*W)) G(Xi)G{F(Y)}

Substituting

for G{Xi - F(Y)> from (A4) in (A3)

- ’

1 ’

form (A41

i.e.

but k, the feedback gain, is Iarge so the last term can be neglected; therefore @5) For the symmetrical

informon, GW’))

= YO,

and F( y, = yO+Z(Xiyi+Xiyi); whence equation (5) becomes