Dynamic Spike Threshold and Nonlinear Dendritic Computation for Coincidence Detection in Neuromorphic Circuits Chih-Chieh Hsu and Alice C. Parker, Life Fellow, IEEE Ming Hsieh Department of Electrical Engineering University of Southern California Abstract— We present an electronic cortical neuron incorporating dynamic spike threshold and active dendritic properties. The circuit is simulated using a carbon nanotube field-effect transistor SPICE model. We demonstrate that our neuron has lower spike threshold for coincident synaptic inputs; however when the synaptic inputs are not in synchrony, it requires larger depolarization to evoke the neuron to fire. We also demonstrate that a dendritic spike is key to precisely-timed input-output transformation, produces reliable firing and results in more resilience to input jitter within an individual neuron.
and efficient computations are achieved in the brain opens up opportunities for neuromorphic computing. In the next section, we discuss the neuroscience research that shows a dynamic spike threshold mechanism in cortical neurons. In Section III, we describe our biomimetic circuit implementation. Then we present our HSPICE simulation results that demonstrate the impacts of dynamic spike threshold and dendritic spikes on AP firing precision and reliability in Section IV.
I. I NTRODUCTION Recent studies have suggested that in neocortex and hippocampus, the neural information is reliably conveyed by the precisely-timed action potentials (APs) at the neuron’s output [1]. In addition, research has shown that cortical neurons are highly sensitive to synchronous inputs and have millisecond temporal precision under rapid depolarization [2]. Some evidence has supported the theory that the threshold to fire APs of cortical neurons in vivo is sensitive to both the amplitude and the rate of membrane potential depolarization [3]. This mechanism suggests that the rapid depolarization caused by coincident synaptic inputs can enhance reliable and precisely-timed firing in the cortical neurons. Biological cortical neurons are complex computational engines, each performing nonlinear computations in the dendrites. By amplifying the post-synaptic potentials (PSPs) a local dendritic spike can be evoked [4]. The propagated dendritic spikes result in rapid depolarization at the somatic membrane, hence they can reduce the threshold to fire APs and induce reliable and precisely-timed spiking. In this paper, we present an axon hillock neuromorphic circuit that embodies dynamic AP spike threshold using carbon nanotube field-effect transistors (CNFETs). Our neuron has lower spike threshold for coincident synaptic inputs and can transform spatiotemporal input information into a preciselytimed response by incorporating nonlinear dendritic computation with dynamic spike threshold. Neuromorphic circuits can be used as testbed system for comprehensive noninvasive brain study. Developing more bio-realistic neuromorphic chips incorporating the fine-grained mechanisms is crucial to the in-depth understanding of the brain’s working mechanism. Deeper understanding of how massively-parallel This material is based upon work supported by the USC Viterbi School of Engineering, the USC WiSE Program, and the National Science Foundation under Grant No. 0726815. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.
978-1-4244-7929-0/14/$26.00 ©2014 IEEE
II. BACKGROUND AND R ELATED W ORK Azouz et al. have shown that higher input synchronization (spikes from different presynaptic neurons arriving at the same time) induces lower spike threshold in their neuron model in Fig. 1 compared to the constant threshold in the integrate-and-fire neuron model [3]. The inverse correlation between the spike threshold and the synaptic input slope rate can be fitted with the equation shown in Eq. (1).
Fig. 1: The normalized dV/dt slope preceding a spike vs. spike threshold. The dashed line indicates the threshold of the integrate-and-fire neuron model. Figure from [3]. y = a + b · e−τ /C , τ = dV /dt
(1)
Several researchers in neuromorphic engineering have implemented spiking silicon neurons. A network of integrateand-fire neurons was used to demonstrate temporal coding in silico [5]. A silicon neuron with active dendrites to reduce somatic jitter was presented [6]. However, in these research efforts, rate-sensitive dynamic spike threshold was not implemented. In our circuit, the dendritic excitability is plastic and the spike threshold is dynamic which leads to precisely-timed AP spikes and reliable firing. We have chosen to simulate our circuits using a CNFET SPICE model [7] for the following reasons: the carbon nanotube transistor is a promising candidate for low-power circuit [8] and has
461
III. R ATE -S ENSITIVE A XON H ILLOCK C IRCUIT The axon hillock is responsible for triggering AP firing in a neuron. Our biomimetic axon hillock circuit design is shown in Fig. 2. In this paper, we augment our original axon hillock module with a differentiation circuit component at the input stage. The rate-sensitive axon hillock transistor circuit is shown in Fig. 2. The idea is to emphasize the temporal changes at the input and use a differentiator to filter out slowly varying input patterns. In Eq. (2), if the slope (dV/dt) on the left side of the equation is small, it results a small V2 . Hence it makes the axon hillock less likely to fire spikes. Therefore, the rate-sensitive axon hillock circuit has a dynamic threshold while the amplitude-sensitive axon hillock circuit has a constant threshold. V2 d(V1 − V2 ) = (2) dt RC The color-shaded segments mimic the activation and inactivation phases of the sodium and potassium channels as well as the delay time for the sequence of events that define the dynamics of an AP. First, the Na+ -activation segment (modeled by X1, X2) turns on Na+ channels (X8) rapidly. After some delay (X4) Na+ -inactivation (X3) then turns off Na+ channels. This portrays the gradual inactivation of Na+ channels when the action potential reaches a depolarizing state. After some more delay (X4), K+ -activation (X10) turns on K+ channels (X7). K+ -inactivation (X9) turns off K+ channels after a longer delay (X5, X6 in series). The delay is controlled by adjusting the strengths of transistors. We use the resistive and capacitive properties of the transistors to achieve the desired time constants.
performed on some of the circuits used, and delaying the circuits to match biological timing will be straightforward, since the fanin and fanout on order of magnitude 104 along with dendritic and axonal attenuation models, can slow the circuits appreciably to achieve real-time emulation. However, accelerated emulation has a speed advantage if the application is to use the circuits as a neuromorphic computing system. Parameter AP half-width AP amplitude Resting potential AP initiation threshold
A. Dynamic Spike Threshold Characterization First, we investigate the scenario when the spike threshold changes with synaptic input slope (dV/dt = VPSP /tr ). Fig. 3 demonstrates the simulation results by testing the ratesensitive axon hillock circuit with different synaptic input amplitudes (VPSP ) and rise times (tr ). It is observed that when the PSP rise time increases, the amplitude of the PSP required to evoke AP spikes increases as well. For example, when tr is 100 ps, it fails to evoke spikes when VPSP is 160mV, but it evokes spikes when VPSP rises to 180 mV. Then, we evaluate the effects of input synchrony on the spike threshold and find a similar inverse correlation of spike threshold and input synchronization to the computational model Azouz et al. proposed (Fig. 1). Fig. 4 demonstrates the dynamic spike threshold and suggests when the the synaptic inputs are synchronous, fewer synapses are needed to evoke an AP spike.
05/16/2014
12:06:08 0
v(psp_in):x_t_r v(psp_in):x_t_r v(psp_in):x_t_r
volt (lin)
X5 v(ah_out):x_t_r
Voltage (V)
Na+
X3
tr = 100ps! tr = 20ps!
C R
Output AP @! VPSP = 140mV!
0.8 0.6
1
v(ah_out):x_t_r
X1
0 1 0
Output AP @! VPSP = 180mV!
1
0.8 0.6 0.4 0.2
0 1 0
X10
v(ah_out):x_t_r v(ah_out):x_t_r
X9
0.6
0.2
AP output spike
X7
v(ah_out):x_t_r
Output AP @! VPSP = 200mV!
1
volt (lin)
X2
Output AP @! VPSP = 160mV!
0.8
0.4
X8
0.8 0.6 0.4 0.2
0 1 0
v(ah_out):x_t_r
K+
v(ah_out):x_t_r v(ah_out):x_t_r
1
volt (lin)
Somatic PSP
tr = 200ps!
0 1
v(ah_out):x_t_r
v(ah_out):x_t_r
V2
700p
0
v(ah_out):x_t_r
X4
V1
waveview 5
0.2
v(ah_out):x_t_r
+
Amplifier
600p
0.4
volt (lin)
delay K inactivation
500p
1
volt (lin)
K inactivation
delay K activation
400p
VPSP!
0 1
+
K activation +
v(ah_out):x_t_r
300p
150m 0.1
v(ah_out):x_t_r
X6
200p
0.2
0
Na+ inactivation
100p
0.2
v(ah_out):x_t_r
+
Electrical 5 ps 900 mV 0 V (Ground) 158 mV
TABLE I: Biological and electrical AP spike parameters
50m
Na+ activation
Biological 1 ms 110 mV -75 mV -56 mV
volt (lin)
superior biocompatibility compared to conventional CMOS technology [9].
Output AP @! VPSP = 220mV!
0.8 0.6 0.4
Fig. 2: Rate-sensitive axon hillock circuit at transistor level.
0.2
0 0
0 0
0.1 100p
printed Fri May 16 2014 13:23:05 by hsuchihc on aludra.usc.edu
IV. HSPICE S IMULATION R ESULTS In this section, we present the simulation results to demonstrate our neuron’s capability to fire AP spikes with dynamic threshold. The biological AP parameters such as half-width, amplitude, resting potential, and initiation threshold and their scaled electrical counterparts are shown in Table I. Although the circuits reported on here have voltages (hundreds of mV) and timings (pico second) scaled commensurate with the technology used, scaling to subthreshold voltages has been
0.2 200p
0.3 300p
0.4 400p
(lin)
Time (ns)
0.5 500p
0.6 600p
0.7
700p
Synopsys, Inc. (c) 2000-2009
Fig. 3: Spike firing threshold at different PSP rise time (tr ) and amplitudes (VPSP ). The top panel shows one of the test input PSP with different rise times (tr = 20, 100, 200 ps). The rest of the panels show output AP characterized under different tr (in the same panel) and VPSP (in separate panels). B. Dynamic and Constant Spike Threshold Comparison In this section, we study a neuron’s firing behavior over dynamic spike threshold and constant spike threshold.
462
0.5
PSP input tr = 20ps
Voltage (V)
Fig. 4: The inverse correlation between the spike threshold and the synaptic input synchronization (represented as the normalized slope.)
0 1
Dynamic Threshold AP output
0 1
Constant Threshold AP output
0
We implement two neurons, one configured with the ratesensitive axon hillock and the other configured with the amplitude-sensitive axon hillock. In Fig. 5, the neuron with constant spike threshold fires when the threshold (158 mV in this configuration) is reached. However the neuron with dynamic spike threshold fires at a lower threshold when the synaptic input slope dV/dt is higher and fires at higher threshold when dV/dt is lower. The synaptic input rise time (tr ) represents the synchronization level of the input, for instance, the smaller tr is, the higher the input synchrony is.
0
0.2
0.4
0.6 0.8 Time (ns)
0.5
Voltage (V)
1.2
Dynamic Threshold AP output
0 1
Constant Threshold AP output
0
0.2
0.4
0.6 0.8 Time (ns)
0.5
1.0
1.2
1.4
PSP input tr = 150ps 200mV
180mV
160mV 0 1
Dynamic Threshold AP output
0 1
Constant Threshold AP output
0
1.4
200mV
180mV
0 1
0
Voltage (V)
1.0
PSP input tr = 100ps 160mV
C. Effect of Dendritic Spike on Precise Spike Timing In this section, we demonstrate that a dendritic spike (dspike) can induce rapid depolarization (dV/dt) of the somatic potential and therefore evoke reliable spiking and more precise spike timing. To study the aforementioned effects, the same input temporal and intensity profile has been applied to two neurons, one with an active dendrite and the other without an active dendrite. The active dendritic branch that models Calcium Influx and Sodium Spike is shown in Fig. 6. The dendritic excitability is modifiable through Threshold Adjustment and the oneshot timing circuitry [10]. Because input arrival may not be in perfectly synchrony, we test our circuits with different levels of input jitter, eg. 2X, and 4X AP half-width. For each test, a consecutive 40 trials were carried out with varying stimulus intensity, such that the somatic potential ranges from approximately -5% to +20% of the threshold to initiate APs. A range of synaptic intensity is chosen because of the stochastic nature of synaptic transmission, ion channel gating and background synaptic noise [11]. We vary the Neurotransmitter Release variable in our synapse circuit to model the stochastic behavior of the neurotransmitter release [12]. The neuron circuit at the transistor level is simulated using HSPICE and then extrapolated into repetitive trials stimulated at 20 GHz using MATLAB to demonstrate the spiking pattern in raster plots shown in Fig. 7 for different inter-stimuli intervals (ISIs) among the synaptic inputs. It is observed that with a dendritic spike, the neuron has enhanced spike timing precision and spike reliability compared to the neuron without an active dendrite. The raster plots also suggest that dendritic spikes can reliably
200mV
180mV
160mV
0
0.2
0.4
0.6 0.8 Time (ns)
1.0
1.2
1.4
Fig. 5: Comparison between dynamic and constant spike threshold at different synaptic input (PSP) rise time (tr ). transform the synchronous input into precise spiking output. The simulation results are shown in Fig. 8 and Fig. 9 for ISI = 2 ps and 4 ps respectively. V. C ONCLUSION In this paper, we demonstrate that our neuron can detect and process coincident inputs with a depolarization ratesensitive axon hillock circuit. We also demonstrate that with dynamic spike threshold, the neuron is more likely to fire when synaptic inputs are synchronous. We further demonstrate that the rapid depolarization induced by the active dendrite (dendritic spike) can enhance the transformation
463
Threshold Adjustment
bAP
bAP** Monostable dspike* Timing (One-shot) Circuit bAP* VDD
EPSP1
UpR
I0 O
Dend Exc
VCa X12
PSPSUM
Na+ X8
X14
CaLVL
X11
X13
VSS1
VSS2
EPSP4 EPSP5
VNa
X3
Calcium Influx
EPSP2 6X
0 1
X4
Down AverageAmplify
EPSP6
X2
X10
X1
X9
Branchi
0 1
VDEND
57.5 % fail to initiate AP 0 40
Down-regulation Up-regulation Fig. 6: Active dendrite circuit at transistor level.
0 DendExc
VREFD
VDD
dspike* CaLVL
A
20
60 80 100 0
40 VREFU
60 80
50
100
150
200
250
100 0
50
Time (ps)
0
0
20
20
40
40
80
100
150
200
250
100 0
77.5% fail to initiate AP
20
20
40
40
Trial #
0
80
50
100
150
200
250
200
250
60 80
100
150
200
250
50
60
70 Time (ps)
80
90
100
Fig. 9: Dendritic spike impact on spike timing (ISI=4 ps).
(d) without dspike
0
50
0 1 AP output without dspike
250
60
(c) with dspike
Trial #
200
Time (ps)
60
100
0 1 AP output with dspike
0 40
Time (ps)
100 0
150
80
50
90
20ps
(b) without dspike
Trial #
Trial #
(a) with dspike
100 0
100
Time (ps)
60
80
UpR
DownR
Voltage (V)
Trial #
Trial #
VSS2
VSS1
GND
70 Time (ps)
bAP**
PSP
40
60
1 AP input ISI = 4ps
0 bAP*
20
50
Fig. 8: Dendritic spike impact on spike timing (ISI=2 ps).
Threshold Adjustment + Calcium Influx
VPP
AP output without dspike
K+
VDD
VPP
AP output with dspike
X7
X0
VDD
10ps
X5
I1 S
DownR
EPSP3
1 AP input ISI = 2ps
Sodium Spike X6
Voltage (V)
dspike
100 0
Time (ps)
50
100
150
Time (ps)
(e) with dspike
(f) without dspike
Fig. 7: Raster plots for different levels of input jitter. (a, b): synchronous input, ISI = 0 ps, no input jitter. (c, d): ISI = 2 ps. Input jitter, 10 ps, 2X AP spike half-width. (e, f): ISI = 4 ps. Input jitter, 20 ps, 4X AP spike half-width. of spatiotemporal neural information into precisely-timed responses. In addition, the neuron with active dendrites is also more resilient to input jitter. R EFERENCES [1] R. C. deCharms, and A. Zador, “Neural representation and the cortical code,” Annu. Review of Neuroscience, vol. 23, no. 1, pp. 613-647, 2000.
[2] G. Ariav, A. Polsky, and J. Schiller, “Submillisecond precision of the input-output transformation function mediated by fast sodium dendritic spikes in basal dendrites of CA1 pyramidal neurons,” J. Neurosci., vol. 23, no. 21, pp. 7750-7758, 2003. [3] R. Azouz, and C. M. Gray, “Dynamic spike threshold reveals a mechanism for synaptic coincidence detection in cortical neurons in vivo,” Proceedings of the National Academy of Sciences, vol. 97, no. 14, pp. 8110-8115, July 2000. [4] M. E. Larkum and T. Nevian, “Synaptic clustering by dendritic signaling mechanisms,” Current opinion in neurobiology, vol. 18, no. 3, pp. 321-331, June 2008. [5] S-.C. Liu, and R. Douglas, “Temporal coding in a silicon network of integrate-and-fire neurons,” IEEE Trans. Neural Networks, vol. 15, no. 5, pp.1305-1314, Sep 2004. [6] Y. Wang and S.-C. Liu, “Active processing of spatio-temporal input patterns in silicon dendrites,” in IEEE Trans. Biomedical Circuits and Systems, vol. 7, no. 3, pp. 307-318, 2013. [7] J. Deng and H.-S. P. Wong, “A Compact SPICE Model for CarbonNanotube Field-Effect Transistors Including Nonidealities and Its Application-Part I: Model of the Intrinsic Channel Region,” IEEE Trans. Electron Devices, vol. 54, pp. 3186-3194, Dec 2007. [8] H.-S. P. Wong and D. Akinwande,“Carbon nanotube and graphene device physics,” Cambridge University Press, 2011. [9] A. Mazzatenta, et al., “Interfacing neurons with carbon nanotubes: electrical signal transfer and synaptic stimulation in cultured brain circuits,” J Neurosci., vol. 27, no. 26, pp. 6931-6936, Jun 2007. [10] C-.C. Hsu, A. C. Parker, and J. Joshi , “Dendritic computations, dendritic spiking and dendritic plasticity in nanoelectronic neurons,” Midwest Symposium on Circuits and Systems, pp. 89-92, Aug 2010. [11] C. F. Stevens, “Neurotransmitter release at central synapses,” Neuron, vol. 40, no .2, pp. 381-388, 2003. [12] A. C. Parker, J. Joshi, C.-C. Hsu, and N. A. D. Singh, “A carbon nanotube implementation of temporal and spatial dendritic computations,” in Midwest Symposium on Circuits and Systems, pp. 818-821, Aug 2008.
464
and efficient computations are achieved in the brain opens up opportunities for neuromorphic computing. In the next section, we discuss the neuroscience research that shows a dynamic spike threshold mechanism in cortical neurons. In Section III, we describe our biomimetic circuit implementation. Then we present our HSPICE simulation results that demonstrate the impacts of dynamic spike threshold and dendritic spikes on AP firing precision and reliability in Section IV.
I. I NTRODUCTION Recent studies have suggested that in neocortex and hippocampus, the neural information is reliably conveyed by the precisely-timed action potentials (APs) at the neuron’s output [1]. In addition, research has shown that cortical neurons are highly sensitive to synchronous inputs and have millisecond temporal precision under rapid depolarization [2]. Some evidence has supported the theory that the threshold to fire APs of cortical neurons in vivo is sensitive to both the amplitude and the rate of membrane potential depolarization [3]. This mechanism suggests that the rapid depolarization caused by coincident synaptic inputs can enhance reliable and precisely-timed firing in the cortical neurons. Biological cortical neurons are complex computational engines, each performing nonlinear computations in the dendrites. By amplifying the post-synaptic potentials (PSPs) a local dendritic spike can be evoked [4]. The propagated dendritic spikes result in rapid depolarization at the somatic membrane, hence they can reduce the threshold to fire APs and induce reliable and precisely-timed spiking. In this paper, we present an axon hillock neuromorphic circuit that embodies dynamic AP spike threshold using carbon nanotube field-effect transistors (CNFETs). Our neuron has lower spike threshold for coincident synaptic inputs and can transform spatiotemporal input information into a preciselytimed response by incorporating nonlinear dendritic computation with dynamic spike threshold. Neuromorphic circuits can be used as testbed system for comprehensive noninvasive brain study. Developing more bio-realistic neuromorphic chips incorporating the fine-grained mechanisms is crucial to the in-depth understanding of the brain’s working mechanism. Deeper understanding of how massively-parallel This material is based upon work supported by the USC Viterbi School of Engineering, the USC WiSE Program, and the National Science Foundation under Grant No. 0726815. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.
978-1-4244-7929-0/14/$26.00 ©2014 IEEE
II. BACKGROUND AND R ELATED W ORK Azouz et al. have shown that higher input synchronization (spikes from different presynaptic neurons arriving at the same time) induces lower spike threshold in their neuron model in Fig. 1 compared to the constant threshold in the integrate-and-fire neuron model [3]. The inverse correlation between the spike threshold and the synaptic input slope rate can be fitted with the equation shown in Eq. (1).
Fig. 1: The normalized dV/dt slope preceding a spike vs. spike threshold. The dashed line indicates the threshold of the integrate-and-fire neuron model. Figure from [3]. y = a + b · e−τ /C , τ = dV /dt
(1)
Several researchers in neuromorphic engineering have implemented spiking silicon neurons. A network of integrateand-fire neurons was used to demonstrate temporal coding in silico [5]. A silicon neuron with active dendrites to reduce somatic jitter was presented [6]. However, in these research efforts, rate-sensitive dynamic spike threshold was not implemented. In our circuit, the dendritic excitability is plastic and the spike threshold is dynamic which leads to precisely-timed AP spikes and reliable firing. We have chosen to simulate our circuits using a CNFET SPICE model [7] for the following reasons: the carbon nanotube transistor is a promising candidate for low-power circuit [8] and has
461
III. R ATE -S ENSITIVE A XON H ILLOCK C IRCUIT The axon hillock is responsible for triggering AP firing in a neuron. Our biomimetic axon hillock circuit design is shown in Fig. 2. In this paper, we augment our original axon hillock module with a differentiation circuit component at the input stage. The rate-sensitive axon hillock transistor circuit is shown in Fig. 2. The idea is to emphasize the temporal changes at the input and use a differentiator to filter out slowly varying input patterns. In Eq. (2), if the slope (dV/dt) on the left side of the equation is small, it results a small V2 . Hence it makes the axon hillock less likely to fire spikes. Therefore, the rate-sensitive axon hillock circuit has a dynamic threshold while the amplitude-sensitive axon hillock circuit has a constant threshold. V2 d(V1 − V2 ) = (2) dt RC The color-shaded segments mimic the activation and inactivation phases of the sodium and potassium channels as well as the delay time for the sequence of events that define the dynamics of an AP. First, the Na+ -activation segment (modeled by X1, X2) turns on Na+ channels (X8) rapidly. After some delay (X4) Na+ -inactivation (X3) then turns off Na+ channels. This portrays the gradual inactivation of Na+ channels when the action potential reaches a depolarizing state. After some more delay (X4), K+ -activation (X10) turns on K+ channels (X7). K+ -inactivation (X9) turns off K+ channels after a longer delay (X5, X6 in series). The delay is controlled by adjusting the strengths of transistors. We use the resistive and capacitive properties of the transistors to achieve the desired time constants.
performed on some of the circuits used, and delaying the circuits to match biological timing will be straightforward, since the fanin and fanout on order of magnitude 104 along with dendritic and axonal attenuation models, can slow the circuits appreciably to achieve real-time emulation. However, accelerated emulation has a speed advantage if the application is to use the circuits as a neuromorphic computing system. Parameter AP half-width AP amplitude Resting potential AP initiation threshold
A. Dynamic Spike Threshold Characterization First, we investigate the scenario when the spike threshold changes with synaptic input slope (dV/dt = VPSP /tr ). Fig. 3 demonstrates the simulation results by testing the ratesensitive axon hillock circuit with different synaptic input amplitudes (VPSP ) and rise times (tr ). It is observed that when the PSP rise time increases, the amplitude of the PSP required to evoke AP spikes increases as well. For example, when tr is 100 ps, it fails to evoke spikes when VPSP is 160mV, but it evokes spikes when VPSP rises to 180 mV. Then, we evaluate the effects of input synchrony on the spike threshold and find a similar inverse correlation of spike threshold and input synchronization to the computational model Azouz et al. proposed (Fig. 1). Fig. 4 demonstrates the dynamic spike threshold and suggests when the the synaptic inputs are synchronous, fewer synapses are needed to evoke an AP spike.
05/16/2014
12:06:08 0
v(psp_in):x_t_r v(psp_in):x_t_r v(psp_in):x_t_r
volt (lin)
X5 v(ah_out):x_t_r
Voltage (V)
Na+
X3
tr = 100ps! tr = 20ps!
C R
Output AP @! VPSP = 140mV!
0.8 0.6
1
v(ah_out):x_t_r
X1
0 1 0
Output AP @! VPSP = 180mV!
1
0.8 0.6 0.4 0.2
0 1 0
X10
v(ah_out):x_t_r v(ah_out):x_t_r
X9
0.6
0.2
AP output spike
X7
v(ah_out):x_t_r
Output AP @! VPSP = 200mV!
1
volt (lin)
X2
Output AP @! VPSP = 160mV!
0.8
0.4
X8
0.8 0.6 0.4 0.2
0 1 0
v(ah_out):x_t_r
K+
v(ah_out):x_t_r v(ah_out):x_t_r
1
volt (lin)
Somatic PSP
tr = 200ps!
0 1
v(ah_out):x_t_r
v(ah_out):x_t_r
V2
700p
0
v(ah_out):x_t_r
X4
V1
waveview 5
0.2
v(ah_out):x_t_r
+
Amplifier
600p
0.4
volt (lin)
delay K inactivation
500p
1
volt (lin)
K inactivation
delay K activation
400p
VPSP!
0 1
+
K activation +
v(ah_out):x_t_r
300p
150m 0.1
v(ah_out):x_t_r
X6
200p
0.2
0
Na+ inactivation
100p
0.2
v(ah_out):x_t_r
+
Electrical 5 ps 900 mV 0 V (Ground) 158 mV
TABLE I: Biological and electrical AP spike parameters
50m
Na+ activation
Biological 1 ms 110 mV -75 mV -56 mV
volt (lin)
superior biocompatibility compared to conventional CMOS technology [9].
Output AP @! VPSP = 220mV!
0.8 0.6 0.4
Fig. 2: Rate-sensitive axon hillock circuit at transistor level.
0.2
0 0
0 0
0.1 100p
printed Fri May 16 2014 13:23:05 by hsuchihc on aludra.usc.edu
IV. HSPICE S IMULATION R ESULTS In this section, we present the simulation results to demonstrate our neuron’s capability to fire AP spikes with dynamic threshold. The biological AP parameters such as half-width, amplitude, resting potential, and initiation threshold and their scaled electrical counterparts are shown in Table I. Although the circuits reported on here have voltages (hundreds of mV) and timings (pico second) scaled commensurate with the technology used, scaling to subthreshold voltages has been
0.2 200p
0.3 300p
0.4 400p
(lin)
Time (ns)
0.5 500p
0.6 600p
0.7
700p
Synopsys, Inc. (c) 2000-2009
Fig. 3: Spike firing threshold at different PSP rise time (tr ) and amplitudes (VPSP ). The top panel shows one of the test input PSP with different rise times (tr = 20, 100, 200 ps). The rest of the panels show output AP characterized under different tr (in the same panel) and VPSP (in separate panels). B. Dynamic and Constant Spike Threshold Comparison In this section, we study a neuron’s firing behavior over dynamic spike threshold and constant spike threshold.
462
0.5
PSP input tr = 20ps
Voltage (V)
Fig. 4: The inverse correlation between the spike threshold and the synaptic input synchronization (represented as the normalized slope.)
0 1
Dynamic Threshold AP output
0 1
Constant Threshold AP output
0
We implement two neurons, one configured with the ratesensitive axon hillock and the other configured with the amplitude-sensitive axon hillock. In Fig. 5, the neuron with constant spike threshold fires when the threshold (158 mV in this configuration) is reached. However the neuron with dynamic spike threshold fires at a lower threshold when the synaptic input slope dV/dt is higher and fires at higher threshold when dV/dt is lower. The synaptic input rise time (tr ) represents the synchronization level of the input, for instance, the smaller tr is, the higher the input synchrony is.
0
0.2
0.4
0.6 0.8 Time (ns)
0.5
Voltage (V)
1.2
Dynamic Threshold AP output
0 1
Constant Threshold AP output
0
0.2
0.4
0.6 0.8 Time (ns)
0.5
1.0
1.2
1.4
PSP input tr = 150ps 200mV
180mV
160mV 0 1
Dynamic Threshold AP output
0 1
Constant Threshold AP output
0
1.4
200mV
180mV
0 1
0
Voltage (V)
1.0
PSP input tr = 100ps 160mV
C. Effect of Dendritic Spike on Precise Spike Timing In this section, we demonstrate that a dendritic spike (dspike) can induce rapid depolarization (dV/dt) of the somatic potential and therefore evoke reliable spiking and more precise spike timing. To study the aforementioned effects, the same input temporal and intensity profile has been applied to two neurons, one with an active dendrite and the other without an active dendrite. The active dendritic branch that models Calcium Influx and Sodium Spike is shown in Fig. 6. The dendritic excitability is modifiable through Threshold Adjustment and the oneshot timing circuitry [10]. Because input arrival may not be in perfectly synchrony, we test our circuits with different levels of input jitter, eg. 2X, and 4X AP half-width. For each test, a consecutive 40 trials were carried out with varying stimulus intensity, such that the somatic potential ranges from approximately -5% to +20% of the threshold to initiate APs. A range of synaptic intensity is chosen because of the stochastic nature of synaptic transmission, ion channel gating and background synaptic noise [11]. We vary the Neurotransmitter Release variable in our synapse circuit to model the stochastic behavior of the neurotransmitter release [12]. The neuron circuit at the transistor level is simulated using HSPICE and then extrapolated into repetitive trials stimulated at 20 GHz using MATLAB to demonstrate the spiking pattern in raster plots shown in Fig. 7 for different inter-stimuli intervals (ISIs) among the synaptic inputs. It is observed that with a dendritic spike, the neuron has enhanced spike timing precision and spike reliability compared to the neuron without an active dendrite. The raster plots also suggest that dendritic spikes can reliably
200mV
180mV
160mV
0
0.2
0.4
0.6 0.8 Time (ns)
1.0
1.2
1.4
Fig. 5: Comparison between dynamic and constant spike threshold at different synaptic input (PSP) rise time (tr ). transform the synchronous input into precise spiking output. The simulation results are shown in Fig. 8 and Fig. 9 for ISI = 2 ps and 4 ps respectively. V. C ONCLUSION In this paper, we demonstrate that our neuron can detect and process coincident inputs with a depolarization ratesensitive axon hillock circuit. We also demonstrate that with dynamic spike threshold, the neuron is more likely to fire when synaptic inputs are synchronous. We further demonstrate that the rapid depolarization induced by the active dendrite (dendritic spike) can enhance the transformation
463
Threshold Adjustment
bAP
bAP** Monostable dspike* Timing (One-shot) Circuit bAP* VDD
EPSP1
UpR
I0 O
Dend Exc
VCa X12
PSPSUM
Na+ X8
X14
CaLVL
X11
X13
VSS1
VSS2
EPSP4 EPSP5
VNa
X3
Calcium Influx
EPSP2 6X
0 1
X4
Down AverageAmplify
EPSP6
X2
X10
X1
X9
Branchi
0 1
VDEND
57.5 % fail to initiate AP 0 40
Down-regulation Up-regulation Fig. 6: Active dendrite circuit at transistor level.
0 DendExc
VREFD
VDD
dspike* CaLVL
A
20
60 80 100 0
40 VREFU
60 80
50
100
150
200
250
100 0
50
Time (ps)
0
0
20
20
40
40
80
100
150
200
250
100 0
77.5% fail to initiate AP
20
20
40
40
Trial #
0
80
50
100
150
200
250
200
250
60 80
100
150
200
250
50
60
70 Time (ps)
80
90
100
Fig. 9: Dendritic spike impact on spike timing (ISI=4 ps).
(d) without dspike
0
50
0 1 AP output without dspike
250
60
(c) with dspike
Trial #
200
Time (ps)
60
100
0 1 AP output with dspike
0 40
Time (ps)
100 0
150
80
50
90
20ps
(b) without dspike
Trial #
Trial #
(a) with dspike
100 0
100
Time (ps)
60
80
UpR
DownR
Voltage (V)
Trial #
Trial #
VSS2
VSS1
GND
70 Time (ps)
bAP**
PSP
40
60
1 AP input ISI = 4ps
0 bAP*
20
50
Fig. 8: Dendritic spike impact on spike timing (ISI=2 ps).
Threshold Adjustment + Calcium Influx
VPP
AP output without dspike
K+
VDD
VPP
AP output with dspike
X7
X0
VDD
10ps
X5
I1 S
DownR
EPSP3
1 AP input ISI = 2ps
Sodium Spike X6
Voltage (V)
dspike
100 0
Time (ps)
50
100
150
Time (ps)
(e) with dspike
(f) without dspike
Fig. 7: Raster plots for different levels of input jitter. (a, b): synchronous input, ISI = 0 ps, no input jitter. (c, d): ISI = 2 ps. Input jitter, 10 ps, 2X AP spike half-width. (e, f): ISI = 4 ps. Input jitter, 20 ps, 4X AP spike half-width. of spatiotemporal neural information into precisely-timed responses. In addition, the neuron with active dendrites is also more resilient to input jitter. R EFERENCES [1] R. C. deCharms, and A. Zador, “Neural representation and the cortical code,” Annu. Review of Neuroscience, vol. 23, no. 1, pp. 613-647, 2000.
[2] G. Ariav, A. Polsky, and J. Schiller, “Submillisecond precision of the input-output transformation function mediated by fast sodium dendritic spikes in basal dendrites of CA1 pyramidal neurons,” J. Neurosci., vol. 23, no. 21, pp. 7750-7758, 2003. [3] R. Azouz, and C. M. Gray, “Dynamic spike threshold reveals a mechanism for synaptic coincidence detection in cortical neurons in vivo,” Proceedings of the National Academy of Sciences, vol. 97, no. 14, pp. 8110-8115, July 2000. [4] M. E. Larkum and T. Nevian, “Synaptic clustering by dendritic signaling mechanisms,” Current opinion in neurobiology, vol. 18, no. 3, pp. 321-331, June 2008. [5] S-.C. Liu, and R. Douglas, “Temporal coding in a silicon network of integrate-and-fire neurons,” IEEE Trans. Neural Networks, vol. 15, no. 5, pp.1305-1314, Sep 2004. [6] Y. Wang and S.-C. Liu, “Active processing of spatio-temporal input patterns in silicon dendrites,” in IEEE Trans. Biomedical Circuits and Systems, vol. 7, no. 3, pp. 307-318, 2013. [7] J. Deng and H.-S. P. Wong, “A Compact SPICE Model for CarbonNanotube Field-Effect Transistors Including Nonidealities and Its Application-Part I: Model of the Intrinsic Channel Region,” IEEE Trans. Electron Devices, vol. 54, pp. 3186-3194, Dec 2007. [8] H.-S. P. Wong and D. Akinwande,“Carbon nanotube and graphene device physics,” Cambridge University Press, 2011. [9] A. Mazzatenta, et al., “Interfacing neurons with carbon nanotubes: electrical signal transfer and synaptic stimulation in cultured brain circuits,” J Neurosci., vol. 27, no. 26, pp. 6931-6936, Jun 2007. [10] C-.C. Hsu, A. C. Parker, and J. Joshi , “Dendritic computations, dendritic spiking and dendritic plasticity in nanoelectronic neurons,” Midwest Symposium on Circuits and Systems, pp. 89-92, Aug 2010. [11] C. F. Stevens, “Neurotransmitter release at central synapses,” Neuron, vol. 40, no .2, pp. 381-388, 2003. [12] A. C. Parker, J. Joshi, C.-C. Hsu, and N. A. D. Singh, “A carbon nanotube implementation of temporal and spatial dendritic computations,” in Midwest Symposium on Circuits and Systems, pp. 818-821, Aug 2008.
464
