Behavioural Elsevier

Processes,

CLASSICAL Nestor ment

14 (1987) 277-289

CONDITIONING,

A.

SIGNAL

Schmajuk,

Center for 111 Cummington

of Mathematics,

(Accepted

277

10 February

DETECTION,

AND EVOLUTION

Adaptive Systems, Boston University, DepartStreet, Boston, Massachusetts 02215 USA

1987)

ABSTRACT N.A. Schmajuk. Behav.

Classical

Processes

conditioning,

14:

signal

detection,

and evolution.

277-289.

Strength of classical conditioning is increased either by increasing discriminability of the conditioned stimulus (CS) from the background, or by increasing contingency between conditioned and unconditioned stimuli (US). Classical conditioning can be regarded as a decision process in which the subject has to decide whether or not to respond with a conditioned response in the presence or absence of the CS. According to modern evolutionary theories, it might be assumed that this decision process maximizes the trade-off between costs and benefits. By assuming that the decision rule maximizes expected benefit, the empiric,al relationship between contingency and the strength of classical conditioning is theoretically derived. In addition, when the decision rule is incorporated to a signal detection paradigm, theoretical results describing the relationship between C’S discriminability and CS - US contingency

with the strength

of classical

conditioning

are in agreement

with experimental

data.

Introduction Discriminability gency

bet,ween

of the major Strength

conditioned

variables

and, except

Conditioning rather

than

with

0376-6357/87/$03.50

as a function

was a function

simple

temporal

of the noise

CS

independent,

of CS CS

- ITS ront,iguity.

as some

discriminability

intensity.

from

Using

either

(1965)

found

from the background

noise.

Kamin

of the change in background from the absolute

with

Q 1987 Elsevier Science Publishers

have been identified

as a CS,

discriminability

also to increase CS

intensity

and contin-

conditioning.

absolute

of the magnitude

in intensity,

seems

stimuli

to increase

as a function

strength

(US)

from the background,

of classical

by increasing

for small changes

(CS)

the strength

in background

increased

of conditioning

seems

than

or an increase

that, conditioning St,rength

rather

stimulus

and unconditioned

determining

of conditioning

t,he background. a decrease

of the conditioned

inrreasing Rcsrorla

CS

noise

intensity.

- 1i.S contingency,

(1068)

B.V. (Biomedical

CS

suggested

Division)

that

278

conditioning

is attained

when the CS

is a reliable

predictor

of the IJS,

receiving

the US in the presence

of the CS

(P(USlCS))

is greater

receiving

the US in the absence

of the CS

(P(USlCS)).

R escorla

conditioning

is obtained

conditioning

from negative

from

non-contingency

(1974)

proposed

CS

and US,

CS.

Gibbon

from positive

contingencies

(P(USlCS)

et al. (1974)

that

detect

learning

terms

of costs

Smith,

1978)

about

mechanisms,

propose

that

excitatory inhibitory

and no conditioning

Berryman,

a statistical

of the reliability

and Thompson

correlation

of the prediction of conditioning

natural

between

of the i/S

by the

was proportional

theories

to

rules can be theoretically In addition,

the probability

and Davies, maximally

trade-off

animals

increasing

values

analyzed

efficient

in

Maynard behavioral

costs and benefits

theories, of CS

to

The survival

1981;

between

to optimization

the present

study

derived by assuming

when the decision operating

of emitting

1972; Testa,

to enable

the strategy

of

discriminability

and

for survival.

predict,

benefit.

and Kalat,

can be quantitatively (Krebs

has chosen

According

value with

is optimal

receiver

behaviors, theories

selection

Rozin

evolution

in their environment.

governed by a particular

tingency

paradigm,

other

net benefit.

- US associative

As optimization

1971;

through

causal relationships

among

i.e., those strategies

CS

1980; Revusky,

Optimization

that

CS - US contingency

tection

< P(USlCS)),

contingency,

have been shaped

and benefits.

gives the maximum

increasing

argued

> P(USlCS)),

Gibbon,

that the strength

(Dickinson,

mechanisms

and store information

strategies;

(P(US(CS)

of of

of the contingency.

value of learning

that

measure

suggested

It has been suggested 1974)

(P(USlCS)

= P(US(CS)).

use of the root mean square

aa a quantitative

the magnitude

contingencies

t,he probability

than the probability

shows that

a CR under different

(1968)

that these rules maximize

rule maximizing

characteristic

Rescorla’s

(ROC)

benefit

is applied

con-

expected

to a signal

de-

curves can be used to determine

CS - US contingencies

and CS

discrim-

inabilities.

Contingency

and derision

Classical the animal

conditioning decides

theory

can be regarded

whether

the presence

of the CS superimposed

noise alone.

This decision

CS

is present

by the animal, world,

the US

or absent

process

(C’s)

or absent

detection

with a conditioned

to background is characterized

or not emmited

(US) (Egan,

problem

response

(CR)

noise or in the presence by two different

on the noisy background;

the CR is emitted is present

as a fundamental

or not to respond

(cri);

either

responses

and two different and Swets,

in

of background

values of evidence,

two alternative

1975; Green

in which

states

1966).

the

selected of the

279

In t.hcLfundamental hit, (the animal a CR

with

detection

but

no

is delivered),

US

t,he IJ.5’ is delivered),

and correct

the TJS is not delivered). different

pay-off

Table

values should

1 shows

value:

r/(US,

the CR),

conditioning

that

pay-off

(1982)

deal with

the forthcoming

rival male.

and l/S,

lJS

the US

V(US,CR)

with

Hollis

the prrsent,ation

Hollis’( 1982)

the CS

the lJS

proposed

plus the cost of

(the cost of eliciting

interactions

Hollis (1984)

a

when the CS

view is applied,

on the basis the animal

with predators,

enjoy an adaptive

precedes

it is V(US,CR)

< V(IIS,CR); be less than

alone.

Payoff

matrix

for a classical

I conditioning

experiment

DE

_-~_y(US> CR) _-

CR, minus the cost of receiving

the benefit

a CR whenever

If

V(US,CR)

sign) of producing

of Equation

rule:

> V(U_S, CR)

to the CS.

large number

the CR,

rule to maximize

+ P(USICS)V(US,CR)

in the decision

P(USlC.5) ~_ P(USlCS)

is to produce

produces

A decision

+ P(~~f$‘S)V(US,CR)

P(USlCS)V(US,CR)

results

the animal

the CR.

can be rewritten

P(USICS)V(US,CR)

>

when

does not produce

when the CS

CR) (Egan,

E(CS,

matrix:

when the animal

the CR.

than

Increasing

of t,he t,he

required

for responding.

of P(IrSICR)

required

for

When

the

responding. So far, a decision CS

is absent,

when

rule has been derived for the case when the CS

two expected

the animal

produces

the CR.

A decision

produce

CR whenever

values can be computed the CR,

and E(CRICS),

rule to maximize

E(CS,(,‘R)

the expected

terms

results

in the decision

P(USlCS) m~Lw P(USlCS)

E(CRICS).

does not produce

when the CS

is absent

is t,o

as

+ P(USICS)V(US,CR)

> P(USICS)V(TJS.CR) Reorganizing

when the animal benefit

This can be written

> E(CS,CR).

r(r’slcs)V(r!s,CR)

is present.

upon the pay-off matrix:

’

+ P(UslCS)V(US,CR) rule:

[31

If

V(US,CR)

- V(TJS, CR)

V(US,CR)

- V(TJS, CR) ’

PI

281

respond

with a C’R during

The animal

should

the absence

not respond

of the CS.

in the absence

of the C’S when

P(US(CS) < V(US,CR) - V(USYCR) _-__ P(US(CS) C’onceptually,

the decision

the rat,io between absence

of receiving the ratio

and those of not receiving

with

the CR if

the US

in the

between

for not responding.

Excitatory CS

than

as: Do not respond

the cost and the benefit of the CR. Increasing the benefit produced by the CR decreases the value of P(USlCS) required for not responding. Increasing the cost of producing the CR increases the value of P(USlCS) required

is smaller

- V(U?’

rule can be expressed

the probabilities

of the CS

V(US,CR)

conditioning

is attained

when

but not in its absence.

Therefore,

the conditions

should

be simultaneously

satisfied.

Equations

P(US~CS)

P(USlCS)

7 is identical

Therefore

the criteria

the absence

of the CS

Inhibitory

for obtaining is equivalent

rondit,ioning

but not, in its presence. 2 and 5 should

to Rescorla’s

be simultaneously

P(IJSjCS)

9 is identical

= 1 - P(USlCS),

we have:

for effective

opposite

excitatory

benefit

for establishing

when the animal

conditions

171

net behavioral

to the criterion

responds

conditioning.

in the presence

classical

in the absence

to those represented

and

conditioning. of the CS

by Equations

satisfied:

= 1 - P(US(CS),

< P(USlC.9) ~., P(USlCS) --and P(USlCS)

P(USlCS) Equation

condition

the maximum

P(US(CS) ~_____ P(USlCS) Replacing

161

> P(USlCS).

(1968)

is attained

Therefore,

of the 2 and 5

P(U.qCS)

P(US(CS) Equation

in the presence by Equations

2 and 5 imply:

and P(US(CS)

= 1 - P(fiSlCS),

responds

represented

> P(USlCS) ~.__._.

__.._~

P(US(CS) Replacing

the animal

to Rescorla’s

(1968)

= 1 - P(USlcS),

we have:

I91

< P(USlCS). condition

for effective

inhibitory

conditioning.

282

Contingency,

discrirninebility,

In the preceding

section

baaed has only two discrete evidence

has a continuous

z constitutes

and signal detection

we have assumed values:

CS

on which the decision

Pyz)

Bayes’

P(xlUS)

the posterior

-P(zpJS)P(US)

Equation

12 gives the decision

likelihood

ratio,

P(xlUS)

If the right-hand written

probabilities

P(USls)

and P(-iislx),

we obtain

- V(US,CR) -~ - V(US,CR)

I111

P(US)V(tiS,CR) _~ ~.. .~~.

-~~_~~_ V(vS,CR) ~~.~ -. CR) - V(US, CR)

> ._~

rule for responding

= P(xIUS)/P(xlUS), and P(xlUS)

(121

based on the evidence

of the observation.

is discussed

term of Equation

z, in terms of the

The meaning

of the prior

the decision

rule can be

below.

12 is designated

L(zo),

as

Equation

13 divides the L(x)

each x that belongs the second

> L(x0).

[I31

axis into two intervals:

L(x)

to the first interval no CR is emitted,

< L(xo)

and L(z)

2 L(zo).

For

and for each z that belongs

to

interval a CR is emitted.

and signal-plus-noise

distributions

under

different

contingencies

We assume that the CS signal is added to noise with a normal intensity density function along a physical

continuum.

4 =

always

1, the CS

density

a stimulus function

of intensity

of intensity

with mean rncs

signal detection

in the presence

= 1. Under these conditions

P(CSlUS) receiving

In a standard

appears

two density functions

x when US is produced,

p(zlTf~),

u,.

procedure

of the 1JS;

x when US is produced, and variance

’

WI

V(US,CR)

L(x)

Noise

When

2 can be rewritt,en as

-. CR)

> V(US,CR) -

P(US)V(US,

probabilities

such as light or sound.

Equation

is

that the

terms: L(x)

L(z)

the decision

to assume

- V(US,CR)

and P(zlUS),

P(x~US)P(US)

and after rearranging

is based,

V(US, CR) -I/(US,

law for replacing

on which

It is also possible

> V(US,CR)

WJS I4 Applying

that the evidence

and CS.

value, z, along a physical dimension

the evidence

by the prior probabilities

theory

i.e.,

appear.

p(zll/S),

The probability

is given by a normal

with contingency

P(CSlUS)

= 1 and

The probability

of

is given by a normal of receiving

a stimulus

density function

with

283

mean mcs

and variance

physical intensity When

u,.

The distance between

is the discriminability

contingency

is different

is produced,

p(z]US),

combination

of two density functions,

in proportions

given by P(CS]US)

P(C.YS]US)~O&

of receiving

a stimulus of intensity

is given by a normal density function

has mean .st = P(CSIUS)mcs

z when US

that, results from the linear

one with mean rnCS and the other with mean rnz, respectively.

and P(CS]US),

+ P(CS(US)mcs,

For the reasons presented

The resulting function

and variance UT = P(CS]US)2a&s

= (2auf)):

exp[-(s

+

in the preceding

- sr)‘/2uf].

paragraph,

stimulus of intensity z when US 1s produced, p(zlUS), with mean .s2 = P(CSIUS)mcs

I141

the probability

of receiving

IS . g iven by a normal density function and variance ui = P(CSIi!???)2u&

+ P(CSIUS)mm,

a

+

Therefore,

P(CsIUS)2u$,s.

p(zlUS)

and p(zlUS)

p(zlUS)

Assuming

rnCS =

P(CSIUS))d’/2,

z)

P(CSjUS)

=

- s2)2/2ul].

I151

in the expression of the likelihood

=

ratio,

e3?!~rk2drL2~~ exp[-(z

d’/2 and rnc:g =

and s2 = (P(CSlU.5’)

For the case P(C5’lCJS)

exp[-(2

= (2~4~~

q

placing

in

(Hays, 1973, page 314). Therefore, p(zlUS)

Replacing

i.e., the difference

than 4 = 1, the CS may imply either the US or the

We will assume that the probability

US.

rn~s and rnz;

of the CS.

-d’/2,

mean values are sr =

(P(CSlUS)

-

- P(CSIUS))d’/Z.

= P(cSl~s),

1 - P(CSIIIS),

I161

- 42/2u;1

= (I#+ 1)/2 (see Appendix).

P(CSlUS) P(CSlUS)

=

1 - P(CS(US),

and (I,

=

Re1, we

obtain L(z) where

= exp[@‘d’z],

4’ = 2$/$2 + 1. 4’ is an increasingly

4 5 1. The effect of changing contingency

monotonic

4 is equivalent

function

of 4 in the range 0 5

to varying t,he effective

signal detection

paradigm.

disrriminability

d’. c$ smaller than 1 implies a decrease in the effective

Figure

4 equal to 1 implies that L(z)

1171

1 shows likelihood

4 = 1, the likelihood

ratio L(z)

ratio is L(s)

is determined

by the stimulus

d’.

as a function of 2 for different values of 4. When

= cd”, which is the value obtained

theory for two normal density functions

d’ in a

in signal detection

(Green and Swets, 1966, page 60). For 4 = 0, the

284

0

’

_d;2

Figure 1. Two normal -

density

functions,

‘0

d)2

one for the CS

(mean

d/2), and other

for the

CS (mean -d/2) are shown. When different CS - US contingencies are applied to these density functions, different likelihood ratios L(z) as a function of CS intensity (z) and contingency (4) are obtained. When L(r) equals L(zO) at zO, z, divides the 2 axis in

two intervals. The animal responds with a CR t,o z values in one interval and with CR to z values in the other interval. The shaded area under the P(CS) function represents P(CRICS). For this illustration Q = .2. Higher values of 4 imply lower values of zc,, and therefore greater values of P(CRIC.5’). likelihood

ratio

L(z)

for I$ with values

is one for every value of z. Intermediate

.8,.6,.4,

and .2. In all cases

values

of L(z)

is an exponentially

L(X)

are obtained

growing

function

of z. The CS CS,

response

density

P(CRlCS),

Figure

in the presence

for I > z,.

is the integral

1 shows

decreasing

rate

function

that

under

for a given

the areas under P(CS)

of the CS,

Correspondingly, the CS

L(z,),

z,

P(CR(CS), the response

density

increases

and the P(CS)

is the integral

under

rate in the absence

function

within

as contingency

and consequently

the

of the

the same

interval.

decreases,

thereby

decreasing

P(CRjCS)

and P(CRICS). Integrals normal

of normal

integral

functions

cannot

be evaluated

with mean zero and unit variance

in closed

can be obtained

form,

but

the st,andard

from tables.

Therefore,

285

P((.‘RICS)

and P(CRICS)

and determining Application

compared

Equation dat,a from

tioning during CS

the CS

onset.

is assumed between

to have

the baseline

Figure between

period

2 shows

theoretical

derived

(1968)

values

for cases

and Stein,

equals

L(s,),

ratios,

A/(A

as the difference

probability

one)

rate of responding

and

(1958)

ex-

have been

d’ was selected

was set equal to 20. Strength

equal

of condi-

+ B), where A is the rate of responding

between

during a comparable the baseline

P(CRICS).

period

prior to

rate of responding

(which

B is computed

as the

difference

and P(CRIcs).

and experimental

and experimental

= P(CSlUX),

and Brady

D’lscriminability

of P(CRICS).

and B is the rate of responding

theoretical

L(.z)

with P(CS1U.S)

Sidman,

to 2.5, and L(s,)

as suppression

A is computed

t at which

functions.

data

on was set equal

is expressed

by finding

the density

17 has been Rescorla

to t,he theoretical

to 10, variance

be obtained

under

to cxprrirnrnt~al

Although perimenhal

might

the integrals

values

values is significant,

as a function

of 4.

Correlation

rzY = .97, L(5) = 6.86,

p < 0.01.

theoretical experimental

.2

.4

.6

.8

1.

contingency Figure 2. Suppression rat,ios as a function of 6. Experimental values were obtained from Rrscorla (1968) (d = 0 to .6), and from and Stein. Sidman, and Brady’s (1958) study recalculated by Gibbon ct ~1. (1974) m . suppression rat,io form (4 = 1).

286

ROC

St/u&d Standard different

curves

ROC

values

of P(CRIUS) ext,reme

curves

of 4.

might

In signal

and P(CRIUS)

values.

When

curves

contingency

detection

between

L(z,).

2,

by entering

theory

obtained

q5 = 1, ROC

P(C.5’) curves for different P(CS)

be used to determine

when

curves

Since

and infinity,

P(CRICS)

a ROC

and

is the set, of different

IS varied continuously

L(r,)

provide

the area

ROC

curves

provide

ROC

curves

can

under

for

P(CRIC.7)

values

between

the P(CS)

the area under

its

and the and

P(CS)

be used for different

values

of

a R,OC curve with D = q5’d’.

Discussion In this subject

paper,

classical

condit,ioning

has to decide whether

or absence assumed By

or not to respond

of the conditioned that

this decision

assuming

that

corla’s

(1968)

values

of contingency

maximum

process

that

benefit

for establishing

have been

signal detection the model linear limited

to the

description

Therefore

theories,

it is

costas and benefits. expected

increases

the criteria

and the absence

and CS-IJS

experimental

lower range

in the presence

benefit. with

Res-

increasing

for obtaining

of the CS

the

is equivalent

conditioning.

derived

contingency

contingency

by applying

on the strength

the optimal

data.

As Gibbon

values.

et al.

(1974)

experimental

The

present

of classical

decision

I erent contingencies f or d’ff

and Rescorla’s

of contingency

also provides

of conditioning,

a good description

as described

rules to a

computed

with

pointed

out, a

suppression model

in CS discriminability

intensities

of t,he (,‘,S distribution,

in t,hr presence Several

ratios

provides

is

a good

suggested

between

paradigms

and Yarensky,

of the effect of CS (1965).

discriminability

A.5 in a signal detection

thereby

increasing

on the paradigm,

ratio for the lower

the t,ot,al probability

of responding

of the CS.

authors

int,eractions

by Kamin

(d ’) increase the value of the likelihood

increments

tioning

of conditioning

the

for all values of contingency.

The model strength

between

response

in which

evolutionary

maximize

strength

Values of P(CRICS)

tit. well Rescorla’s

relationship

process

between

derived.

classical

theoretically

paradigm.

strategies

in the presence

The effects of CS discriminability conditioning

to modern

the trade-off

condit,ioning

postulate

as a decision

with a conditioned

According

maximizes

is theoretically

net behavioral

to the criterion

stimulus.

classical

empirical

is regarded

that

signal

rate of reinforcement (Davison

1977).

Nevin

and Tusting, (1981)

detection

theory

can be applied

and stimulus

discriminability

1978:

1981;

Nevin.

Nevin,

to describe

the

in opf’rant, rondiJenkins,

b,h owed t,hat,, whrn changc>s in disrriminability

Whitt,akrr. arr rqj-

287

resented

as changes

likelihood

ratio

in d’, and changes

inabilit,y

and contingency

ing case,

the present

theory

in contzingency

R.OC-l’k1 e curves describe

L(r),

in operant

paper

conditioning

between

As in the operant

ROC curves

CR probability

as changes

interactions

paradigms.

shows that standard

can be used for describing

are represented

adequately

in the discrim-

rondition-

derived from signal detection

as a function

of discriminability

and

contingency. As ment,ioned maximally

above,

efficient

optimization

behavioral

theory

t,o operant

derived

from optimal

conditioning foraging

In the case of classical has chosen to the

behavioral

CS.

This

discriminabilit,y the principles

rules.

The version

value,

that

optimization

is confirmed

is an account

an explanation

law can

that natural

be

that

selection

when to respond the effects

of C’S

can be derived

(1983)

of the rules used in classical

for vrhy animals not,ed,

(1983)

has pointed

was supported

in deciding

showing

Shettlcworth

is grateful

predict

efficient

conditioning

a description

author

has chosen optimization

matching

theories

on classical

learn,

of this manuscript,.

select,ion applied

from

t,heory.

approach

animals

(1970)

by our results

contingency

and Hinson

hew

natural

successfully

Herrnstein’s

of hour animals

luhy and

that

(1980)

that, are maximally

derision

optimality

.4s Staddon

Furthermore, hrtwcrn

conditioning,

of optimal

of survival

description

showing

strategies

prediction

assume

Staddon

models.

and C’S - US

The present in terms

theories

strategies.

learn

opt,imality

according

approaches

of the mechanisms out that

there

conditioning to particular

do not

involved

is no simple

provide

a

in learning. relationship

learn.

to Dr.

Dan

Bullock

Also to Cynthia

for his vahiable

Suchta

for typing

comments

on an early

the manuscript.

This paper

in part by NSF grant, IST-8417756.

REFERENCES Dickinson, A., 1980. Ilniversity Press. Egan.

.J.P., Press.

1975.

Condemporary

Signal

det,ection

animal theory