1466
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, VOL. 24, NO. 9, SEPTEMBER 2013
[24] Q. X. Zhu and J. D. Cao, “Stability analysis of markovian jump stochastic BAM neural networks with impulse control and mixed time delays,” IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 3, pp. 467–479, Mar. 2012. [25] H. G. Zhang, Z. S. Wang, and D. R. Liu, “Global asymptotic stability and robust stability of a class of Cohen-Grossberg neural networks with mixed delays,” IEEE Trans. Circuits Syst. Part I, Regular Papers, vol. 56, no. 3, pp. 616–629, Mar. 2009. [26] M. S. Mahmoud, S. Z. Selim, and P. Shi, “Global exponential stability criteria for neural networks with probabilistic delays,” IET Control Theory Appl., vol. 4, no. 11, pp. 2405–2415, Nov. 2010. [27] G. B. Zhang, T. Wang, T. Li, and S. M. Fei, “Delay-derivativedependent stability criterion for neural networks with probabilistic time-varying delay,” Int. J. Syst. Sci., May 2012, DOI: 10.1080/00207721.2012.685198. [28] J. Fu, H. G. Zhang, and T. D. Ma, “Delay-probability-distributiondependent robust stability analysis for stochastic neural networks with time-varying delay,” Progr. Natural Sci., vol. 19, no. 10, pp. 1333–1340, Oct. 2009. [29] Y. J. Zhang, D. Yue, and E. G. Tian, “New stability criteria of neural networks with interval time-varying delay: A piecewise delay method,” Appl. Math. Comput., vol. 208, no. 1, pp. 249–259, Feb. 2009. [30] S. S. Mou, H. J. Gao, and J. Lam, “A new criterion of delay-dependent asymptotic stability for Hopfield neural networks with time delay,” IEEE Trans. Neural Netw., vol. 19, no. 3, pp. 532–535, Mar. 2008. [31] H. G. Zhang, Z. W. Liu, G. B. Huang, and Z. S. Wang, “Novel weighting-delay-based stability criteria for recurrent neural networks with time-varying delay,” IEEE Trans. Neural Netw., vol. 21, no. 1, pp. 91–106, Jan. 2010. [32] W. H. Chen and W. X. Zheng, “Improved delay-dependent asymptotical stability criteria for delayed neural networks,” IEEE Trans. Neural Netw., vol. 19, no. 12, pp. 2154–2161, Dec. 2008. [33] H. Y. Shao and Q. L. Han, “New delay-dependent stability criteria for neural networks with two additive time-varying delay components,” IEEE Trans. Neural Netw., vol. 22, no. 5, pp. 812–818, Mar. 2011. [34] J. K. Tian and S. M. Zhong, “Improved delay-dependent stability criteria for neural networks with two additive time-varying delay components,” Neurocomputing, vol. 77, no. 3, pp. 114–119, Feb. 2012. [35] E. Fridman, U. Shaked, and K. Liu, “New conditions for delay-derivative-dependent stability,” Automatica, vol. 45, no. 11, pp. 2723–2727, Sep. 2009. [36] P. Park, J. W. Ko, and C. Jeong, “Reciprocally convex approach to stability of systems with time-varying delays,” Automatica, vol. 47, no. 1, pp. 235–238, Jan. 2011. [37] J. Liu and J. Zhang, “Note on stability of discrete-time time-varying delay systems,” IET Control Theory Appl., vol. 6, no. 2, pp. 335–339, Jan. 2012. [38] J. Sun, G. P. Liu, J. Chen, and D. Rees, “Improved delayrange-dependent stability criteria for linear systems with timevarying delays,” Automatica, vol. 46, no. 2, pp. 466–470, Feb. 2010. [39] D. Yue, E. G. Tian, Y. J. Zhang, and C. Peng, “Delay-distributiondependent stability and stabilization of T-S fuzzy systems with probabilistic interval delay,” IEEE Trans. Syst., Man, Cybern. Part B, Cybern., vol. 39, no. 2, pp. 503–516, Apr. 2009.
Low-Temperature Fabrication of Spiking Soma Circuits Using Nanocrystalline-Silicon TFTs Anand Subramaniam, Student Member, IEEE, Kurtis D. Cantley, Member, IEEE, Harvey J. Stiegler, Member, IEEE, Richard A. Chapman, Fellow, IEEE, and Eric M. Vogel, Senior Member, IEEE Abstract— Spiking neuron circuits consisting of ambipolar nanocrystalline-silicon (nc-Si) thin-film transistors (TFTs) have been fabricated using low temperature processing conditions (maximum of 250 °C) that allow the use of flexible substrates. These circuits display behaviors commonly observed in biological neurons such as millisecond spike duration, nonlinear frequency– current relationship, and spike frequency adaptation. The maximum drive capacity of a simple soma circuit was estimated to be approximately 9200 synapses. The effect of bias stressinduced threshold voltage degradation of component nc-Si TFTs on the spike frequency of soma circuits is explored. The measured power consumption of the circuit when spiking at 100 Hz was approximately 12 nW. Finally, the power consumption of the soma circuits at different spiking conditions and its implications on a large-scale system are discussed. The fabricated circuits can be employed as part of a compact multilayer learning network. Index Terms— Frequency–current curve, nanocrystalline silicon (nc-Si), soma circuits, thin-film transistor (TFT).
I. I NTRODUCTION Neuromorphic spiking networks and the associated largely parallel architectures have been the object of much interest in recent years. These systems are expected to complement CMOS-based digital architectures in applications where substantial amounts of imprecise information have to be processed. Some of the applications where neuromorphic systems could outperform binary architectures include adaptive learning, pattern classification, and outcome prediction. Deriving inspiration from the operation of the human brain, neuromorphic systems are expected to operate at low speeds of a few hundred Hz maximum [1]. Furthermore, the entire human brain consumes approximately 10 W of power [2]. This low power operation stems from the way the human brain integrates memory and logic. Neuromorphic circuits would also require fewer operations to make a complex decision than digital logic, and would therefore consume less energy Manuscript received February 16, 2012; revised December 18, 2012; accepted March 30, 2013. Date of publication April 30, 2013; date of current version August 16, 2013. This work was supported in part by the SRC/NRI SWAN, the Erik Jonsson School of Engineering at the University of Texas at Dallas, and the Texas Analog Center of Excellence. A. Subramaniam and R. A. Chapman are with the Department of Electrical Engineering, University of Texas at Dallas, Richardson, TX 75080 USA (e-mail: [email protected]; [email protected]). E. M. Vogel is with the School of Materials Science and Engineering, Georgia Institute of Technology, Atlanta, GA 30332 USA, and also with the Department of Materials Science and Engineering, University of Texas at Dallas, Richardson, TX 75080 USA (e-mail: [email protected]). K. D. Cantley and H. J. Stiegler are with the Department of Materials Science and Engineering, University of Texas at Dallas, Richardson, TX 75080 USA (e-mail: [email protected]; [email protected]). Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/TNNLS.2013.2256926
2162-237X © 2013 IEEE
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, VOL. 24, NO. 9, SEPTEMBER 2013
in performing the same operation. Unlike in a typical binary system, each neuron in the brain is connected to as many as 104 other neurons through synapses [3]. Thus, the foremost challenge in building neuromorphic networks arises from the need for a large number of devices with a high degree of interconnection, motivating the search for appropriate device technology. One of the most promising options is nanocrystalline-silicon (nc-Si) thin-film transistors (TFTs) with sub-micrometer dimensions [4]–[6]. Unlike crystalline silicon, nc-Si can be deposited over large areas at low temperatures (< 250 °C). This allows for the possibility of 3-D networks with a high degree of interconnection, as well as for the use of alternate substrates such as plastic [7] or metal foil [8]. A large area, flexible artificial cortex could be folded to create a much lower volume, leading to a structure physically similar to the human neocortex. The fabrication process of nc-Si TFTs is compatible with the back-end of a CMOS process, which facilitates applications where the neuromorphic circuitry is integrated 3-D in multiple layers around a CMOS core [9]. The neurons would then be linked using arbitrary connections, complementing an address event representation (AER) scheme [10]. However, unlike the CMOS fabrication process, nc-Si TFTs avoid complex processes such as doping, ion implantation, or diffusion. Mobilities as high as 150 cm2 /V·s for electrons have been achieved by depositing nc-Si layers with high crystalline volume fraction [11]. Potential drawbacks such as lack of p-channel operation and threshold voltage shift upon the application of voltage bias stress have been addressed by reducing the impurity-oxygen concentration in the channel layer [5]–[6], [12]. Separate processing steps for n- and p-channel transistors are not required due to the achievement of ambipolar operation. The performance of these devices has been considerably improved by utilizing thin high-κ gate dielectrics that reduce the subthreshold swing and threshold voltages (VT ) of these devices [12]. Lastly, neuromorphic systems are defect-tolerant in nature, so that the performance degradation or even failure of individual transistors might not affect the circuit operation to a large degree. Spiking soma circuits are one of the basic building blocks of a neuromorphic system. Soma circuits linked through synapses would act as information-transfer pathways in the system. Most previous implementations of soma circuits have been CMOS-based [13]–[14]. These circuits have been used in neuromorphic systems performing associative learning [15], pattern recognition [16], and frequency detection [17]. In this brief, we demonstrate the fabrication of spiking soma circuits using nc-Si TFTs and capacitors alone. Section II provides an overview of the nc-Si TFT technology including fabrication and I–V characteristics. In Section III, the operation of spiking soma circuits is analyzed in detail and compared with that of biological neurons. Section IV deals with synaptic load capacity and the effect of bias stress degradation on spike frequencies. Finally, in Section V the power consumption at different spiking modes is examined, and is demonstrated to be at acceptable levels by comparing with a CMOS-based implementation.
Fig. 1.
1467
Schematic cross-section of an ambipolar nc-Si TFT.
II. N C -S I TFT T ECHNOLOGY The fabrication process employs a set of four lithography steps with a maximum temperature of 250 °C (523 K). A p-type Si wafer with moderate doping (1017 cm−3 ) is used as the substrate on which a 300 nm SiO2 was grown with dry thermal oxidation. First, an 80 nm Cr or Ti metal is deposited and patterned. This layer acts as source/drain of the individual TFTs as well as the bottom level of metal lines. All metal lines are 6 μm wide. Next, an 80 nm thick nc-Si layer is deposited using plasmaenhanced chemical vapor deposition (PECVD) at 13.56 MHz, 125 W RF power, 120 Pa (0.9 Torr), and 250 °C of deposition temperature in a 1:100 mixture of SiH4 and H2 . The deposition conditions are described in further detail in [12]. The channel layer is then patterned by inductively coupled plasma etch (ICP) with Cl-based chemistry. The next step was deposition of the gate dielectric. SiO2 or HfO2 with equivalent oxide thicknesses (EOTs) ranging from 200 nm to 4.5 nm is used on different samples. SiO2 was deposited using PECVD, whereas HfO2 was deposited by atomic layer deposition (ALD). EOTs were calculated assuming dielectric constant of 17 for HfO2 . The third level of lithography was then used to pattern contact holes through the gate dielectric, and 100 nm Au was used as a contact metal. The last layer was a 100 nm Al, used as a gate metal as well as the top-level of metal lines. A final 250 °C anneal in forming gas was performed for 2 h. Individual transistors, inverters, as well as diverse soma circuits were fabricated with the same mask set. Fig. 1 displays the cross-section of an nc-Si TFT. The low-impurity oxygen concentration in the channel layer (∼3 × 1017 cm−3 ) permits ambipolar operation of the individual TFTs [5], [11]. Current–voltage (I–V) characteristics of a device with 20-nm HfO2 gate dielectric (EOT 4.5 nm) are shown in Fig. 2. The electron and hole threshold voltages of this device are 2.1 V and −2.3 V, respectively. The subthreshold swing is also relatively low (400 mV/dec. for n-channel operation). The gate current is lower than 10−8 A/cm2 throughout the measurements. These transistors have been observed to provide stable operation with threshold voltage shift of less than 0.4 V after operation at VG = 7 V, V D = 5 V for 104 s [5], [12]. The threshold voltage variation between fifteen measured devices was also low, falling within a 0.2 V range. Fig. 3(a) shows the voltage transfer characteristics at different operating voltages of an inverter (Wpull−up/ Wpull−down = 4) that consists of two nc-Si TFTs. The inverter exhibits nearly peak-to-peak operation, and high voltage gain
1468
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, VOL. 24, NO. 9, SEPTEMBER 2013 VDD
VDD=6 V
(a) C1=C2=1 pF L=10 µm Iin
W=20 M3 µm
M15
M5
M9
M16 M10
Vadap
M1
V1
M2
C2 M4
C1
Vout
M6
M4 M7
Vmem
Iin
Vout
C1
W=5 µm
M3
M5
Ireset
M2
Vlk
W=20 µm
M1
Vrfr
M17 M12
M8
M14
M6
M13 M11
Vb=1.5 V W=5 µm
Fig. 2. (a) Transfer characteristics of an ambipolar nc-Si TFT and (b) Output characteristics of the same device.
Fig. 4. Circuit diagrams of (a) a simple soma circuit proposed by Mead [21] and (b) a complex soma circuit proposed by Indiveri et al. [22]. Individual transistor channel widths are indicated for either circuit while the channel lengths remain constant.
Fig. 3. (a) Voltage transfer characteristics of a fabricated nc-Si TFT-based inverter and (b) corresponding inverter “crowbar” current.
(in the range 10–15). This high voltage gain can be ascribed to the good saturation characteristics of the component TFTs. Fig. 3(b) shows the corresponding inverter currents. The inverter “crowbar” current is a critical factor in the power consumption of neuromorphic circuits. To minimize the inverter current, the pullup/pulldown ratio of the transistors can be optimized. For instance, the maximum current in Fig. 2(b) is 550 nA, and can be reduced by a factor of three by changing the ratio of transistor widths from 4 to 8. In the case of the fabricated inverter described above, operation up to 2 MHz was possible without significant distortion of the output [9]. This value is among the highest for noncrystalline silicon TFT-based inverters [18], [19]. For neuromorphic circuit applications, operation up to a few kHz is sufficient [20]. The frequency response of the fabricated inverter is most likely to be limited by gate to source/drain overlap capacitance. In these inverters, the overlap between the gate and the source/drain is 5 μm each, which results in an overlap capacitance of 0.86 nF/m (normalized with respect to the device width). Reducing this overlap is likely to allow the operation at even higher frequencies. III. S PIKING S OMA C IRCUITS Two different soma circuits were fabricated: a simple circuit proposed by Mead [21] consisting of six transistors and a more complex one based on a circuit developed by Indiveri et al. [22] that consists of more than 12 transistors (Fig. 4). These circuits were selected chiefly for their ease of analysis. While there have been many other designs of CMOS-based soma circuits, our focus in this brief is not to compare numerous designs but to demonstrate that neuromorphic circuits are feasible with nc-silicon technology. Each soma circuit uses three metal layers in addition to the nc-Si layer. The indi-
Fig. 5. Optical images of fabricated circuits (a) a simple soma circuit and (b) a complex soma circuit.
vidual transistor dimensions are indicated in the figure for either circuit. In addition, both circuits employ capacitors. Ambipolar nc-Si TFTs were used throughout the circuits. Planar capacitors were implemented by overlapping metal pads separated by the gate dielectric material. Optical images of fabricated circuits are shown in Fig. 5. The circuit layout was not optimized for area, but rather for ease of probing. The Mead soma circuit operates by integrating an input current (modeled after current injection from different synapses in a biological soma) across capacitor C1 (membrane capacitance) [21]. When the voltage at the input node crosses the inverter threshold, a voltage pulse is generated at the output, which is analogous to generating an action potential. The feedback path from the output to the gate of transistor M2 controls the leakage current from the membrane capacitor. The more complex soma circuit proposed by Indiveri [Fig. 4(b)] consists of additional circuitry for spike frequency adaptation, reduced power consumption, and setting arbitrary refractory periods. The operation of a fabricated simple (Mead) soma circuit at VDD = 6 V is shown in detail in Fig. 6(a). An input current of linearly increasing magnitude is applied, which results in output spikes at increased frequencies. The spikes (Vout in the figure) are roughly rectangular in shape, and have an average duration of a few milliseconds, which is close to biological action potential widths. The relationship between the input current and the spike frequency is called the discharge curve [shown in Fig. 6(b)], and its shape for the fabricated circuit is similar to the biological data [20]. The spike frequency
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, VOL. 24, NO. 9, SEPTEMBER 2013
Fig. 6. (a) Injected current, “membrane” capacitance voltage, output voltage, and spike frequency of a simple soma circuit as a function of time. Capacitances used are: C1 = C2 = 1.1 pF. (b) Variation of spike frequency with membrane capacitance. The discharge curve saturates at high input currents depending on the capacitance values.
is limited at high current inputs by the refractory period of the neuron, i.e., the time following an output spike that the neuron cannot spike again. This leads to a saturation effect in the discharge curve at high-current injections [observed in the low-capacitance traces in Fig. 6(b)]. The operation of this simple soma circuit proves that low-performance nc-Si TFTs are adequate for spiking soma circuits. In fact, the circuit capacitances play as much of a role in determining the output frequency as device mobilities. In addition to transistors and capacitors, large (100 × 100 μm) probe pads were included in each circuit to measure voltages/currents. The total size of a typical neuron circuit was approximately 800 × 400 μm. However, the active area of the circuitry in both cases was less than 15000 μm2 , including two 5000 μm2 capacitors. The use of large capacitances (∼1 pF) leads to the capacitors dominating the active circuit area. However, smaller capacitors can also be used without affecting the circuit operation. For example, the change in spike frequency when different values of membrane capacitances (obtained by changing dielectric thickness and area) were used in the simple soma circuit is shown in Fig. 6(b). For the same input current range, the spike rate increases for smaller capacitances, down to 110 fF. This suggests that even parasitic gate to S/D capacitances of individual TFTs could be used as the membrane capacitance in the circuit. Since reducing C2 increases the spike frequency values by a larger percentage than reducing C1 does, scaling of the capacitor values has to be performed with care. The performance of the circuit is also increased by this scaling since lower input current is sufficient for the same output frequencies. In addition, the TFTs can be fabricated at the nanoscale [5], leading to scaled soma circuits that occupy much less area. Fig. 7(a) displays the output spikes and instantaneous spike frequency of a fabricated complex (Indiveri) circuit when injected with a constant current of 500 pA for a time period of 0.8 s. Output spikes are rectangular and are approximately 10 ms wide. Fig. 7(b) explores the effect of the spike frequency adaptation in a complex soma circuit. This effect models biological neurons that display less sensitivity to a constant current than a variable one. An additional discharge path that is controlled by an external voltage (Vadap in this case) subtracts a current Iadap from the input current, thus reducing the spike
1469
Fig. 7. (a) Output voltage and spike frequency of a complex neuron circuit when injected with a constant input current. (b) Effect of spike frequency adaptation on the complex neuron circuit when a constant current is injected. The spike frequency decreases with time when Vadap is set to 5 V, thus proving the operation of the adaptation circuitry.
frequency. When there is no spiking activity, the voltage at the controlling node leaks away, thereby resetting the circuit to the original state. The adaptation was repeated after a 10-s delay to prove that TFT degradation is not responsible for the result. The complex soma circuit was proposed to be implemented using CMOS transistors; the achievement of spike frequency adaptation with nc-Si TFTs demonstrates that higher order functionalities can be achieved in circuits using low-temperature, ambipolar nc-Si devices. IV. H ARDWARE I MPLEMENTATION I SSUES Soma circuits in a neuromorphic system will not be operated in isolation, but connected to numerous synapses at its output. Different implementations of the synapse circuit have been proposed, including a floating-gate transistor [23], a memristor [24] or a combination of the two [25]. In our calculations, we utilize the last implementation because it achieves a biological synaptic weight change rule using rectangular voltage pulses. This implementation requires each neuron to directly drive another nc-Si TFT. From Fig. 2, it can be assumed that a spike voltage of 3 V is required for changing the synaptic weight (since the output spike is fed to the gate of another TFT). For nc-Si TFTs with 20 × 10 μm channels (as used in the fabricated soma), the individual synaptic load can be calculated to be R0 = 50 G and C0 = 3.45 aF. Since all synapses are connected in parallel, the capacitive load goes up with the number of synapses. In addition, considering interconnect lines with 10-nm width and pitch, and length of 200 nm per synapse, the interconnect capacitance is 14 aF per synapse. To measure the synaptic load capacity of a soma circuit, the variation of spike voltage with different capacitive and resistive loads was measured. Fig. 8(a) displays the spike voltage as a function of the number of synapses modeled at the output. Approximately 9200 synapses can be driven with this soma circuit at VDD = 5 V. The drive capacity increases with supply voltage, and this relation is shown in Fig. 8(b). The capability of the soma circuit to drive numerous synapses renders them of interest for use in learning systems [26], or large-scale networks [9]. For neuromorphic applications that necessitate integration with CMOS logic for high-speed computation, the soma circuits could be built around the CMOS core in a 3-D structure.
1470
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, VOL. 24, NO. 9, SEPTEMBER 2013
Fig. 8. Effect of synaptic load on the spike voltage of a simple soma circuit. (a) Calculation of the load capacity of the circuit at VDD = 5 V. (b) Variation of the load capacity with operating voltage, assuming 3 V voltage pulses are required to change the synaptic weight.
Noncrystalline silicon TFTs suffer from threshold voltage degradation under voltage bias stress [27], [28]. By using higher quality nc-Si as the channel layer, and by lowering impurity oxygen concentration [5], the amount of VT shift in our TFTs has been lowered significantly. However, under typical bias conditions encountered in spiking neuron circuits (VG = 7 V and V D = 5 V), the VT shift is still approximately 0.4 V after a stress period of 104 seconds [6], and could play a detrimental role in circuit operation. To quantify the effect of VT shift on a simple soma circuit, the circuit was stressed for different time periods by operating at a supply voltage of 5 V and input current of 0.8 nA. Subsequently, the discharge curves were measured by varying the input current. The results [Fig. 9(a)] show a gradual decrease in spike frequency upon increasing the stress time. After a stress period of 104 seconds, the spike frequency at an input current of 1 nA decreased by 22%. A similar experiment was also performed on the complex soma circuit, where the spike frequency after different periods of stress were measured, the injected current being held constant at 0.5 nA [Fig. 9(b)]. After 104 seconds, the spike frequency reduced by 7%. However, any VT shift resulting from charge trapping in the gate dielectric can recover during periods of rest when the neuron is inactive. A partial recovery was observed by letting the device rest for a period of 24 h after stress [also Fig. 9(b)]. Furthermore, the VT is observed to saturate at prolonged stress times (>5 h) in accordance with earlier observations [29]. Other approaches to reduce the degradation include performing the nc-Si deposition at a higher temperature and avoiding residual moisture [30]. In a large-scale system, soma circuits would form discrete units connected through synapses. A mismatch of I–V characteristics from individual TFTs could change the spike frequencies of a soma circuit by a small percentage. However, the error from a particular unit should not have a large effect on the entire system because neuromorphic circuits will be designed with a certain level of defect tolerance [31]. For example, if a neuron has a slightly higher threshold of membrane capacitance voltage for firing, it may fire with a slight delay from the expected time instance. However, since the next neuron in line will also rely on the integrated inputs from large number of neurons to fire, the delay from the first neuron may not affect its firing. Also, multiple parallel pathways for
Fig. 9. (a) Discharge curves from a simple soma circuit after different stress periods. (b) Variation of spike frequency in the complex soma circuit after different stress periods when the injected current is held constant. The VT shift saturates after long time periods, and recovers partially after a rest period.
information transfer impart an additional measure of stability to the complete circuit. V. P ERFORMANCE A NALYSIS OF S OMA C IRCUIT It is vital to ensure low power consumption in spiking soma circuits since a large number of these circuits would be used in a neuromorphic system. The instantaneous power consumption can be calculated by multiplying the supply voltage by the current sourced out of the supply node. Integrating the power when the circuit spike frequency is constant provides the total energy consumption at that frequency, including the contribution of standby power. Fig. 10(a) shows an expanded view of the calculation for a simple soma circuit when spiking at 100 Hz using VDD = 6 V. The sharp rises, or ripples, in the energy transient correspond to spikes and much slower rises occur during the stand-by period. At this frequency, the standby power is 5 nW and the total power is 12 nW (approximately 90 pJ/spike). The corresponding values for the complex soma circuit are 23 nW and 90 nW. For a large network of 1010 simple neurons as present in the human brain neocortex, these values translate to 120 W. A CMOS model based on the BSIM3v3 AIM-SPICE built-in model [32] and 180 nm node was also developed for comparing the average power consumption in a simple soma circuit. Transistors with identical device dimension ratios to the TFTs in the fabricated circuit were used in the simulation. Different supply voltages were used in the CMOS simulation. The results [shown in Fig. 10(b)] indicate that when spiking at the same frequency, the power consumption of the fabricated circuit compares well with a CMOS implementation when operated at the same supply voltage. This is because the higher leakage values in nc-Si transistors are compensated by the much lower ONcurrents. The supply voltage can be scaled lower in the CMOS case resulting in lower power I–V characteristics of a TFT can be improved by performing nc-Si deposition at slightly higher temperatures [33], as well as by scaling the gate dielectric thickness [12]; therefore it should also be possible to further scale the supply voltage of the nc-Si TFT-based circuits. The spike durations do not affect the power calculations because they primarily are a function of the circuit capacitances. The power dissipation of fabricated CMOS-based neurons varies from 9 pJ/spike (at VDD = 3.3 V) [13] to 3 nJ/spike (at VDD = 5 V) [14] depending on design complexity and supply voltage.
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, VOL. 24, NO. 9, SEPTEMBER 2013
Fig. 10. (a) Energy dissipation of a fabricated nc-Si simple soma circuit when spiking at a constant frequency of 100 Hz. The ripples correspond to increased power consumption at output spikes. (b) Comparison between the power consumption of a fabricated nc-Si simple soma circuit and corresponding CMOS simulations at different supply voltages.
1471
complex behaviors including spike-frequency adaptation were also realized. The circuits were tested by including different synaptic loads at the output, and their maximum drive capacity was measured to be approximately 9200 synapses. The effect of threshold voltage degradation on the operation of soma circuits was studied. The degradation in output frequency due to bias stress was shown to recover partly after rest. The power consumption of a simple soma circuit was measured under various conditions, and operated at only 12 nW at 100Hz output, translating to 12 W for a large system with 109 neurons. Previous work has demonstrated nanoscale nc-Si TFTs which could be incorporated into these spiking soma circuits. Together with electronic synapses using the low-temperature approach, highly integrated neuromorphic learning systems are certainly feasible. R EFERENCES
Fig. 11. (a) Variation of stand-by and total power consumption with spike frequency in a simple soma circuit with VDD = 6 V. (b) Dependence of power on the supply voltage in the same circuit.
Variation of power consumption of the fabricated Mead circuit with spike frequency and supply voltage is shown in Fig. 11(a) and (b), respectively. Similar trends were also observed in the case of the complex soma circuit. At low spike frequencies (< 30 Hz), the standby power dominates. The total power consumption increases with supply voltage, as can be expected. Employing thin-gate dielectrics in the nc-Si TFTs increases the device performance [12], allowing the use of lower supply voltage in neuromorphic circuits. The spike frequency of soma circuits in neuromorphic applications is expected to be very low (< 200 Hz) [1]. Achieving low stand-by power is crucial since it will dominate at these frequencies. Optimizing the geometry ratio of constituent inverters can reduce it to a large extent. Another method could be to apply a back-gate voltage that shifts the I D -VG curve such that the lowest drain current occurs at VG = 0. A third technique would be to use different S/D metal contacts to prevent the injection of carriers at low gate voltages, thus lowering the off-current at high drain voltages. Efforts to implement these methods are in progress. VI. C ONCLUSION Spiking soma circuits have been fabricated with ambipolar nc-Si TFTs using low-temperature (250 °C maximum) processing. These conditions enable utilization of flexible substrates, as well as 3-D integration with CMOS structures. The soma circuits output discrete rectangular voltage pulses. The spike width and discharge curves associated with these circuits matched closely with that of biological neurons, and
[1] C. Koch, Biophysics of Computation. New York, NY, USA: Oxford Univ. Press, 1999. [2] D. Attwell and S. B. Laughlin, “An energy budget for signaling in the grey matter of the brain,” J. Cerebral Blood Flow Metabolism, vol. 21, no. 10, pp. 1133–1145, Oct. 2001. [3] J. Hawkins and S. Blakeslee, On Intelligence. New York, NY, USA: Henry Holt & Company, 2004. [4] K. D. Cantley, A. Subramaniam, H. J. Stiegler, R. A. Chapman, and E. M. Vogel, “SPICE simulation of nanoscale non-crystalline silicon TFTs in spiking neuron circuits,” in Proc. 53rd IEEE Int. Midwest Symp. Circuits Syst., Aug. 2010, pp. 1202–1205. [5] A. Subramaniam, K. D. Cantley, R. A. Chapman, B. Chakrabarty, and E. M. Vogel, “Ambipolar non-crystalline-silicon TFTs with submicron dimensions and reduced threshold voltage shift,” in Proc. 69th Device Res. Conf., Jun. 2011, pp. 99–100. [6] A. Subramaniam, K. D. Cantley, R. A. Chapman, H. Stiegler, and E. M. Vogel, “Submicron ambipolar nanocrystalline-silicon thin-film transistors and inverters,” IEEE Trans. Electron Devices, vol. 59, no. 2, pp. 359–366, Feb. 2012. [7] H. Gleskova, S. Wagner, V. Gasparik, and P. Kovac, “150 °C amorphous silicon thin-film transistor technology for polyimide substrates,” J. Electrochem. Soc., vol. 148, no. 7, pp. G370–G374, May 2001. [8] E. Y. Ma and S. Wagner, “Amorphous silicon transistors on ultrathin steel substrates,” Appl. Phys. Lett., vol. 74, no. 18, pp. 2661–2662, May 1999. [9] O. Turel, J. H. Lee, X. Ma, and K. K. Likharev, “Neuromorphic architectures for nanoelectronic circuits,” Int. J. Circuit Theory Appl., vol. 32, no. 5, pp. 277–302, Sep. 2004. [10] K. A. Boahen, “Point-to-point connectivity between Neuromorphic Chips using Address Events,” IEEE Trans. Circuits Syst. II, Analog Digital Signal Process., vol. 47, no. 5, pp. 416–433, May 2000. [11] C.-H. Lee, A. Sazonov, M. R. E. Rad, G. R. Chaji, and A. Nathan, “Ambipolar thin-film transistors fabricated by PECVD nanocrystalline silicon,” in Proc. Mater. Res. Soc. Symp., vol. 910. 2006, pp. A05–A22. [12] A. Subramaniam, K. D. Cantley, R. A. Chapman, B. Chakrabarty, and E. M. Vogel, “Submicron ambipolar nanocrystalline-silicon TFTs with high-K gate dielectrics,” in Proc. Int. Semicond. Device Res. Symp., Dec. 2011, pp. 1–2. [13] J. H. B. Wijekoon and P. Dudek, “Integrated circuit implementation of a cortical neuron,” in Proc. IEEE Int. Symp. Circuits Syst., May 2008, pp. 1784–1787. [14] G. Indiveri, “A low-power adaptive integrate-and-fire neuron circuit,” in Proc. IEEE Int. Symp. Circuits Syst., May 2003, pp. 820–823. [15] Y. V. Pershin and M. Di Ventra, “Experimental demonstration of associative memory with memristive neural networks,” Neural Netw., vol. 23, no. 7, pp. 881–886, Sep. 2010. [16] T. Masquelier, R. Guyonneau, and S. J. Thorpe, “Spike timing dependent plasticity finds the start of repeating patterns in continuous spike trains,” PLoS One, vol. 3, no. 1, p. e1377, Jan. 2008. [17] K. D. Cantley, A. Subramaniam, H. J. Stiegler, R. A. Chapman, and E. M. Vogel, “Neural learning circuits utilizing nano-crystalline silicon transistors and memristors,” IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 4, pp. 565–573, Apr. 2012.
1472
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, VOL. 24, NO. 9, SEPTEMBER 2013
[18] A. J. Snell, W. E. Spear, P. G. LeComber, and K. Mackenzie, “Application of amorphous silicon thin film transistors in integrated circuits,” Appl. Phys., vol. A26, pp. 83–86, Jul. 1981. [19] Y. Nara, Y. Kudou, and M. Matsumura, “Application of amorphoussilicon thin-film transistors in three-dimensional integrated circuits,” Jap. J. Appl. Phys., vol. 22, no. 6, pp. L370–L372, Jun. 1983. [20] C. E. Stafstrom, P. C. Schwindt, and W. E. Crill, “Repetitive firing in layer V neurons from cat neocortex in vitro,” J. Neurophysiol., vol. 52, no. 2, pp. 264–277, Aug. 1984. [21] C. Mead, “Neuromorphic electronic systems,” Proc IEEE, vol. 78, no. 10, pp. 1629–1636, Oct. 1990. [22] G. Indiveri, E. Chicca, and R. Douglas, “A VLSI array of low-power spiking neurons and bistable synapses with spike-timing dependent plasticity,” IEEE Trans. Neural Netw., vol. 17, no. 1, pp. 211–221, Jan. 2006. [23] C. Diorio, P. Hasler, B. A. Minch, and C. A. Mead, “A single transistor silicon synapse,” IEEE Trans. Electron Devices, vol. 43, no. 11, pp. 1972–1980, Nov. 1996. [24] B. Linares-Barranco and T. Terrano-Gotarredona, “Exploiting memristance in adaptive asynchronous spiking neurmorphic nanotechnology systems,” in Proc. 9th IEEE Conf. Nanotechnol., Jul. 2009, pp. 978–981. [25] K. D. Cantley, A. Subramaniam, H. Stiegler, R. Chapman, and E. Vogel, “Spike timing-dependent synaptic plasticity using memristors and nanocrystalline silicon TFT memories,” in Proc. 11th IEEE Conf. Nanotechnol., Aug. 2011, pp. 421–425. [26] K. D. Cantley, A. Subramaniam, H. J. Stiegler, R. A. Chapman, and E. M. Vogel, “Hebbian Learning in Spiking Neural Networks with Nanocrystalline Silicon TFTs and Memristive Synapses,” IEEE Trans. Nanotechnol., vol. 10, no. 5, pp. 1066–1073, Sep. 2011. [27] M. J. Powell, C. van Berkel, I. D. French, and D. H. Nicholls, “Bias dependence of instability mechanisms in amorphous silicon thinfilm transistors,” Appl. Phys. Lett., vol. 51, no. 16, pp. 1242–1244, Oct. 1987. [28] Y.-S. Tai, J.-W. Tsai, H.-C. Chung, and F.-C. Su, “Instability mechanisms for the hydrogenated amorphous silicon thin-film transistors with negative and positive bias stresses on the gate electrodes,” Appl. Phys. Lett., vol. 67, no. 1, pp. 76–78, Jul. 1995. [29] M. R. Esmaeili-Rad, A. Sazonov, and A. Nathan, “Absence of defect state creation in nanocrystalline silicon thin film transistors deduced from constant current stress measurements,” Appl. Phys. Lett., vol. 91, no. 11, pp. 113511-1–113511-3, Sep. 2007. [30] D. R. Allee, L. T. Clark, B. D. Vogt, R. Shringarpure, S. M. Venugopal, S. G. Uppili, K. Kaftanoglu, H. Shivalingaiah, Z. P. Li, J. J. Ravindra Fernando, E. J. Bawolek, and S. M. O’Rourke, “Circuit-level impact of a-Si:H thin-film-transistor degradation effects,” IEEE Trans. Electron Devices, vol. 56, no. 6, pp. 1166–1176, Jun. 2009. [31] N. Joye, A. Schmid, Y. Leblebici, T. Asai, and Y. Amemiya, “Fault-tolerant logic gates using neuromorphic CMOS circuits,” in Proc. Res. Microelectron. Electron. Conf., Jul. 2007, pp. 249–252. [32] W. Liu, MOSFET Models for SPICE Simulation: Including BSIM3v3 and BSIM4, 1st ed. New York, NY, USA: Wiley, Feb. 2001. [33] C.-H. Lee, A. Sazonov, and A. Nathan, “High hole and electron mobilities in nanocrystalline silicon thin-film transistors,” J. Non Crystalline Solids, vol. 352, nos. 9–20, pp. 1732–1736, Jun. 2006.
Effect of Input Noise and Output Node Stochastic on Wang’s kWTA John Sum, Senior Member, IEEE, Chi-Sing Leung, Member, IEEE, and Kevin Ho
Abstract— Recently, an analog neural network model, namely Wang’s kWTA, was proposed. In this model, the output nodes are defined as the Heaviside function. Subsequently, its finite time convergence property and the exact convergence time are analyzed. However, the discovered characteristics of this model are based on the assumption that there are no physical defects during the operation. In this brief, we analyze the convergence behavior of the Wang’s kWTA model when defects exist during the operation. Two defect conditions are considered. The first one is that there is input noise. The second one is that there is stochastic behavior in the output nodes. The convergence of the Wang’s kWTA under these two defects is analyzed and the corresponding energy function is revealed. Index Terms— Convergence analysis, energy function, input noise, kWTA, output node stochastic.
I. I NTRODUCTION In conventional design, a traditional kWTA network consists of n nodes and n 2 connections [2], [10], [13]. By reformulating the k winners selection problem as a linear program, Jun Wang and his coworkers succeeded a much simpler structure to realize a kWTA network based on the idea of dual neural network (DNN) [5], [17], [18]. Here, we call it the Wang’s kWTA model. This model consists of n output nodes (which are defined as Heaviside activation functions), 2n connections, and a hidden node (which behaves as a recurrent state variable).1 Because of the Heaviside activation function, Wang in [18] proved the finite time convergence of this kWTA and later Xiao et al. in [20] derived the exact convergence time. Because of its simple structure, this model is particularly suitable for hardware implementation. However, as known in the studies of fault tolerant neural network. Hardware implementation can never be perfect [8], [19], [21], [22]. For example, the electrical noise [21] from the power supplies or thermal effect may be injected into the input signals. In addition, the offset voltage of comparators may drift randomly [22]. In this regard, it is valuable to analyze the effects of input noise and output node stochastic on the dynamic behavior of the Wang’s kWTA model. Manuscript received May 26, 2012; accepted March 24, 2013. Date of publication May 7, 2013; date of current version August 16, 2013. This work was supported in part by Research Grants from Taiwan National Science Council under Grant 100-2221-E-126-015 and Grant 101-2221-E-126-016, and a Research Grant under Grant CityU 115612 from the Research Grants Council of the Government of the Hong Kong Special Administrative Region. J. Sum is with the Institute of Technology Management, National Chung Hsing University, Taichung 40227, Taiwan (e-mail: [email protected]). C.-S. Leung is with the Department of Electronic Engineering, City University of Hong Kong, Kowloon, Hong Kong (e-mail: [email protected]). K. Ho is with the Department of Computer Science and Communication Engineering Providence University, Taichung 43301, Taiwan (e-mail: [email protected]). Digital Object Identifier 10.1109/TNNLS.2013.2257182 Note that an identical model has been proposed independently in [9].
2162-237X © 2013 IEEE
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, VOL. 24, NO. 9, SEPTEMBER 2013
[24] Q. X. Zhu and J. D. Cao, “Stability analysis of markovian jump stochastic BAM neural networks with impulse control and mixed time delays,” IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 3, pp. 467–479, Mar. 2012. [25] H. G. Zhang, Z. S. Wang, and D. R. Liu, “Global asymptotic stability and robust stability of a class of Cohen-Grossberg neural networks with mixed delays,” IEEE Trans. Circuits Syst. Part I, Regular Papers, vol. 56, no. 3, pp. 616–629, Mar. 2009. [26] M. S. Mahmoud, S. Z. Selim, and P. Shi, “Global exponential stability criteria for neural networks with probabilistic delays,” IET Control Theory Appl., vol. 4, no. 11, pp. 2405–2415, Nov. 2010. [27] G. B. Zhang, T. Wang, T. Li, and S. M. Fei, “Delay-derivativedependent stability criterion for neural networks with probabilistic time-varying delay,” Int. J. Syst. Sci., May 2012, DOI: 10.1080/00207721.2012.685198. [28] J. Fu, H. G. Zhang, and T. D. Ma, “Delay-probability-distributiondependent robust stability analysis for stochastic neural networks with time-varying delay,” Progr. Natural Sci., vol. 19, no. 10, pp. 1333–1340, Oct. 2009. [29] Y. J. Zhang, D. Yue, and E. G. Tian, “New stability criteria of neural networks with interval time-varying delay: A piecewise delay method,” Appl. Math. Comput., vol. 208, no. 1, pp. 249–259, Feb. 2009. [30] S. S. Mou, H. J. Gao, and J. Lam, “A new criterion of delay-dependent asymptotic stability for Hopfield neural networks with time delay,” IEEE Trans. Neural Netw., vol. 19, no. 3, pp. 532–535, Mar. 2008. [31] H. G. Zhang, Z. W. Liu, G. B. Huang, and Z. S. Wang, “Novel weighting-delay-based stability criteria for recurrent neural networks with time-varying delay,” IEEE Trans. Neural Netw., vol. 21, no. 1, pp. 91–106, Jan. 2010. [32] W. H. Chen and W. X. Zheng, “Improved delay-dependent asymptotical stability criteria for delayed neural networks,” IEEE Trans. Neural Netw., vol. 19, no. 12, pp. 2154–2161, Dec. 2008. [33] H. Y. Shao and Q. L. Han, “New delay-dependent stability criteria for neural networks with two additive time-varying delay components,” IEEE Trans. Neural Netw., vol. 22, no. 5, pp. 812–818, Mar. 2011. [34] J. K. Tian and S. M. Zhong, “Improved delay-dependent stability criteria for neural networks with two additive time-varying delay components,” Neurocomputing, vol. 77, no. 3, pp. 114–119, Feb. 2012. [35] E. Fridman, U. Shaked, and K. Liu, “New conditions for delay-derivative-dependent stability,” Automatica, vol. 45, no. 11, pp. 2723–2727, Sep. 2009. [36] P. Park, J. W. Ko, and C. Jeong, “Reciprocally convex approach to stability of systems with time-varying delays,” Automatica, vol. 47, no. 1, pp. 235–238, Jan. 2011. [37] J. Liu and J. Zhang, “Note on stability of discrete-time time-varying delay systems,” IET Control Theory Appl., vol. 6, no. 2, pp. 335–339, Jan. 2012. [38] J. Sun, G. P. Liu, J. Chen, and D. Rees, “Improved delayrange-dependent stability criteria for linear systems with timevarying delays,” Automatica, vol. 46, no. 2, pp. 466–470, Feb. 2010. [39] D. Yue, E. G. Tian, Y. J. Zhang, and C. Peng, “Delay-distributiondependent stability and stabilization of T-S fuzzy systems with probabilistic interval delay,” IEEE Trans. Syst., Man, Cybern. Part B, Cybern., vol. 39, no. 2, pp. 503–516, Apr. 2009.
Low-Temperature Fabrication of Spiking Soma Circuits Using Nanocrystalline-Silicon TFTs Anand Subramaniam, Student Member, IEEE, Kurtis D. Cantley, Member, IEEE, Harvey J. Stiegler, Member, IEEE, Richard A. Chapman, Fellow, IEEE, and Eric M. Vogel, Senior Member, IEEE Abstract— Spiking neuron circuits consisting of ambipolar nanocrystalline-silicon (nc-Si) thin-film transistors (TFTs) have been fabricated using low temperature processing conditions (maximum of 250 °C) that allow the use of flexible substrates. These circuits display behaviors commonly observed in biological neurons such as millisecond spike duration, nonlinear frequency– current relationship, and spike frequency adaptation. The maximum drive capacity of a simple soma circuit was estimated to be approximately 9200 synapses. The effect of bias stressinduced threshold voltage degradation of component nc-Si TFTs on the spike frequency of soma circuits is explored. The measured power consumption of the circuit when spiking at 100 Hz was approximately 12 nW. Finally, the power consumption of the soma circuits at different spiking conditions and its implications on a large-scale system are discussed. The fabricated circuits can be employed as part of a compact multilayer learning network. Index Terms— Frequency–current curve, nanocrystalline silicon (nc-Si), soma circuits, thin-film transistor (TFT).
I. I NTRODUCTION Neuromorphic spiking networks and the associated largely parallel architectures have been the object of much interest in recent years. These systems are expected to complement CMOS-based digital architectures in applications where substantial amounts of imprecise information have to be processed. Some of the applications where neuromorphic systems could outperform binary architectures include adaptive learning, pattern classification, and outcome prediction. Deriving inspiration from the operation of the human brain, neuromorphic systems are expected to operate at low speeds of a few hundred Hz maximum [1]. Furthermore, the entire human brain consumes approximately 10 W of power [2]. This low power operation stems from the way the human brain integrates memory and logic. Neuromorphic circuits would also require fewer operations to make a complex decision than digital logic, and would therefore consume less energy Manuscript received February 16, 2012; revised December 18, 2012; accepted March 30, 2013. Date of publication April 30, 2013; date of current version August 16, 2013. This work was supported in part by the SRC/NRI SWAN, the Erik Jonsson School of Engineering at the University of Texas at Dallas, and the Texas Analog Center of Excellence. A. Subramaniam and R. A. Chapman are with the Department of Electrical Engineering, University of Texas at Dallas, Richardson, TX 75080 USA (e-mail: [email protected]; [email protected]). E. M. Vogel is with the School of Materials Science and Engineering, Georgia Institute of Technology, Atlanta, GA 30332 USA, and also with the Department of Materials Science and Engineering, University of Texas at Dallas, Richardson, TX 75080 USA (e-mail: [email protected]). K. D. Cantley and H. J. Stiegler are with the Department of Materials Science and Engineering, University of Texas at Dallas, Richardson, TX 75080 USA (e-mail: [email protected]; [email protected]). Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/TNNLS.2013.2256926
2162-237X © 2013 IEEE
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, VOL. 24, NO. 9, SEPTEMBER 2013
in performing the same operation. Unlike in a typical binary system, each neuron in the brain is connected to as many as 104 other neurons through synapses [3]. Thus, the foremost challenge in building neuromorphic networks arises from the need for a large number of devices with a high degree of interconnection, motivating the search for appropriate device technology. One of the most promising options is nanocrystalline-silicon (nc-Si) thin-film transistors (TFTs) with sub-micrometer dimensions [4]–[6]. Unlike crystalline silicon, nc-Si can be deposited over large areas at low temperatures (< 250 °C). This allows for the possibility of 3-D networks with a high degree of interconnection, as well as for the use of alternate substrates such as plastic [7] or metal foil [8]. A large area, flexible artificial cortex could be folded to create a much lower volume, leading to a structure physically similar to the human neocortex. The fabrication process of nc-Si TFTs is compatible with the back-end of a CMOS process, which facilitates applications where the neuromorphic circuitry is integrated 3-D in multiple layers around a CMOS core [9]. The neurons would then be linked using arbitrary connections, complementing an address event representation (AER) scheme [10]. However, unlike the CMOS fabrication process, nc-Si TFTs avoid complex processes such as doping, ion implantation, or diffusion. Mobilities as high as 150 cm2 /V·s for electrons have been achieved by depositing nc-Si layers with high crystalline volume fraction [11]. Potential drawbacks such as lack of p-channel operation and threshold voltage shift upon the application of voltage bias stress have been addressed by reducing the impurity-oxygen concentration in the channel layer [5]–[6], [12]. Separate processing steps for n- and p-channel transistors are not required due to the achievement of ambipolar operation. The performance of these devices has been considerably improved by utilizing thin high-κ gate dielectrics that reduce the subthreshold swing and threshold voltages (VT ) of these devices [12]. Lastly, neuromorphic systems are defect-tolerant in nature, so that the performance degradation or even failure of individual transistors might not affect the circuit operation to a large degree. Spiking soma circuits are one of the basic building blocks of a neuromorphic system. Soma circuits linked through synapses would act as information-transfer pathways in the system. Most previous implementations of soma circuits have been CMOS-based [13]–[14]. These circuits have been used in neuromorphic systems performing associative learning [15], pattern recognition [16], and frequency detection [17]. In this brief, we demonstrate the fabrication of spiking soma circuits using nc-Si TFTs and capacitors alone. Section II provides an overview of the nc-Si TFT technology including fabrication and I–V characteristics. In Section III, the operation of spiking soma circuits is analyzed in detail and compared with that of biological neurons. Section IV deals with synaptic load capacity and the effect of bias stress degradation on spike frequencies. Finally, in Section V the power consumption at different spiking modes is examined, and is demonstrated to be at acceptable levels by comparing with a CMOS-based implementation.
Fig. 1.
1467
Schematic cross-section of an ambipolar nc-Si TFT.
II. N C -S I TFT T ECHNOLOGY The fabrication process employs a set of four lithography steps with a maximum temperature of 250 °C (523 K). A p-type Si wafer with moderate doping (1017 cm−3 ) is used as the substrate on which a 300 nm SiO2 was grown with dry thermal oxidation. First, an 80 nm Cr or Ti metal is deposited and patterned. This layer acts as source/drain of the individual TFTs as well as the bottom level of metal lines. All metal lines are 6 μm wide. Next, an 80 nm thick nc-Si layer is deposited using plasmaenhanced chemical vapor deposition (PECVD) at 13.56 MHz, 125 W RF power, 120 Pa (0.9 Torr), and 250 °C of deposition temperature in a 1:100 mixture of SiH4 and H2 . The deposition conditions are described in further detail in [12]. The channel layer is then patterned by inductively coupled plasma etch (ICP) with Cl-based chemistry. The next step was deposition of the gate dielectric. SiO2 or HfO2 with equivalent oxide thicknesses (EOTs) ranging from 200 nm to 4.5 nm is used on different samples. SiO2 was deposited using PECVD, whereas HfO2 was deposited by atomic layer deposition (ALD). EOTs were calculated assuming dielectric constant of 17 for HfO2 . The third level of lithography was then used to pattern contact holes through the gate dielectric, and 100 nm Au was used as a contact metal. The last layer was a 100 nm Al, used as a gate metal as well as the top-level of metal lines. A final 250 °C anneal in forming gas was performed for 2 h. Individual transistors, inverters, as well as diverse soma circuits were fabricated with the same mask set. Fig. 1 displays the cross-section of an nc-Si TFT. The low-impurity oxygen concentration in the channel layer (∼3 × 1017 cm−3 ) permits ambipolar operation of the individual TFTs [5], [11]. Current–voltage (I–V) characteristics of a device with 20-nm HfO2 gate dielectric (EOT 4.5 nm) are shown in Fig. 2. The electron and hole threshold voltages of this device are 2.1 V and −2.3 V, respectively. The subthreshold swing is also relatively low (400 mV/dec. for n-channel operation). The gate current is lower than 10−8 A/cm2 throughout the measurements. These transistors have been observed to provide stable operation with threshold voltage shift of less than 0.4 V after operation at VG = 7 V, V D = 5 V for 104 s [5], [12]. The threshold voltage variation between fifteen measured devices was also low, falling within a 0.2 V range. Fig. 3(a) shows the voltage transfer characteristics at different operating voltages of an inverter (Wpull−up/ Wpull−down = 4) that consists of two nc-Si TFTs. The inverter exhibits nearly peak-to-peak operation, and high voltage gain
1468
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, VOL. 24, NO. 9, SEPTEMBER 2013 VDD
VDD=6 V
(a) C1=C2=1 pF L=10 µm Iin
W=20 M3 µm
M15
M5
M9
M16 M10
Vadap
M1
V1
M2
C2 M4
C1
Vout
M6
M4 M7
Vmem
Iin
Vout
C1
W=5 µm
M3
M5
Ireset
M2
Vlk
W=20 µm
M1
Vrfr
M17 M12
M8
M14
M6
M13 M11
Vb=1.5 V W=5 µm
Fig. 2. (a) Transfer characteristics of an ambipolar nc-Si TFT and (b) Output characteristics of the same device.
Fig. 4. Circuit diagrams of (a) a simple soma circuit proposed by Mead [21] and (b) a complex soma circuit proposed by Indiveri et al. [22]. Individual transistor channel widths are indicated for either circuit while the channel lengths remain constant.
Fig. 3. (a) Voltage transfer characteristics of a fabricated nc-Si TFT-based inverter and (b) corresponding inverter “crowbar” current.
(in the range 10–15). This high voltage gain can be ascribed to the good saturation characteristics of the component TFTs. Fig. 3(b) shows the corresponding inverter currents. The inverter “crowbar” current is a critical factor in the power consumption of neuromorphic circuits. To minimize the inverter current, the pullup/pulldown ratio of the transistors can be optimized. For instance, the maximum current in Fig. 2(b) is 550 nA, and can be reduced by a factor of three by changing the ratio of transistor widths from 4 to 8. In the case of the fabricated inverter described above, operation up to 2 MHz was possible without significant distortion of the output [9]. This value is among the highest for noncrystalline silicon TFT-based inverters [18], [19]. For neuromorphic circuit applications, operation up to a few kHz is sufficient [20]. The frequency response of the fabricated inverter is most likely to be limited by gate to source/drain overlap capacitance. In these inverters, the overlap between the gate and the source/drain is 5 μm each, which results in an overlap capacitance of 0.86 nF/m (normalized with respect to the device width). Reducing this overlap is likely to allow the operation at even higher frequencies. III. S PIKING S OMA C IRCUITS Two different soma circuits were fabricated: a simple circuit proposed by Mead [21] consisting of six transistors and a more complex one based on a circuit developed by Indiveri et al. [22] that consists of more than 12 transistors (Fig. 4). These circuits were selected chiefly for their ease of analysis. While there have been many other designs of CMOS-based soma circuits, our focus in this brief is not to compare numerous designs but to demonstrate that neuromorphic circuits are feasible with nc-silicon technology. Each soma circuit uses three metal layers in addition to the nc-Si layer. The indi-
Fig. 5. Optical images of fabricated circuits (a) a simple soma circuit and (b) a complex soma circuit.
vidual transistor dimensions are indicated in the figure for either circuit. In addition, both circuits employ capacitors. Ambipolar nc-Si TFTs were used throughout the circuits. Planar capacitors were implemented by overlapping metal pads separated by the gate dielectric material. Optical images of fabricated circuits are shown in Fig. 5. The circuit layout was not optimized for area, but rather for ease of probing. The Mead soma circuit operates by integrating an input current (modeled after current injection from different synapses in a biological soma) across capacitor C1 (membrane capacitance) [21]. When the voltage at the input node crosses the inverter threshold, a voltage pulse is generated at the output, which is analogous to generating an action potential. The feedback path from the output to the gate of transistor M2 controls the leakage current from the membrane capacitor. The more complex soma circuit proposed by Indiveri [Fig. 4(b)] consists of additional circuitry for spike frequency adaptation, reduced power consumption, and setting arbitrary refractory periods. The operation of a fabricated simple (Mead) soma circuit at VDD = 6 V is shown in detail in Fig. 6(a). An input current of linearly increasing magnitude is applied, which results in output spikes at increased frequencies. The spikes (Vout in the figure) are roughly rectangular in shape, and have an average duration of a few milliseconds, which is close to biological action potential widths. The relationship between the input current and the spike frequency is called the discharge curve [shown in Fig. 6(b)], and its shape for the fabricated circuit is similar to the biological data [20]. The spike frequency
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, VOL. 24, NO. 9, SEPTEMBER 2013
Fig. 6. (a) Injected current, “membrane” capacitance voltage, output voltage, and spike frequency of a simple soma circuit as a function of time. Capacitances used are: C1 = C2 = 1.1 pF. (b) Variation of spike frequency with membrane capacitance. The discharge curve saturates at high input currents depending on the capacitance values.
is limited at high current inputs by the refractory period of the neuron, i.e., the time following an output spike that the neuron cannot spike again. This leads to a saturation effect in the discharge curve at high-current injections [observed in the low-capacitance traces in Fig. 6(b)]. The operation of this simple soma circuit proves that low-performance nc-Si TFTs are adequate for spiking soma circuits. In fact, the circuit capacitances play as much of a role in determining the output frequency as device mobilities. In addition to transistors and capacitors, large (100 × 100 μm) probe pads were included in each circuit to measure voltages/currents. The total size of a typical neuron circuit was approximately 800 × 400 μm. However, the active area of the circuitry in both cases was less than 15000 μm2 , including two 5000 μm2 capacitors. The use of large capacitances (∼1 pF) leads to the capacitors dominating the active circuit area. However, smaller capacitors can also be used without affecting the circuit operation. For example, the change in spike frequency when different values of membrane capacitances (obtained by changing dielectric thickness and area) were used in the simple soma circuit is shown in Fig. 6(b). For the same input current range, the spike rate increases for smaller capacitances, down to 110 fF. This suggests that even parasitic gate to S/D capacitances of individual TFTs could be used as the membrane capacitance in the circuit. Since reducing C2 increases the spike frequency values by a larger percentage than reducing C1 does, scaling of the capacitor values has to be performed with care. The performance of the circuit is also increased by this scaling since lower input current is sufficient for the same output frequencies. In addition, the TFTs can be fabricated at the nanoscale [5], leading to scaled soma circuits that occupy much less area. Fig. 7(a) displays the output spikes and instantaneous spike frequency of a fabricated complex (Indiveri) circuit when injected with a constant current of 500 pA for a time period of 0.8 s. Output spikes are rectangular and are approximately 10 ms wide. Fig. 7(b) explores the effect of the spike frequency adaptation in a complex soma circuit. This effect models biological neurons that display less sensitivity to a constant current than a variable one. An additional discharge path that is controlled by an external voltage (Vadap in this case) subtracts a current Iadap from the input current, thus reducing the spike
1469
Fig. 7. (a) Output voltage and spike frequency of a complex neuron circuit when injected with a constant input current. (b) Effect of spike frequency adaptation on the complex neuron circuit when a constant current is injected. The spike frequency decreases with time when Vadap is set to 5 V, thus proving the operation of the adaptation circuitry.
frequency. When there is no spiking activity, the voltage at the controlling node leaks away, thereby resetting the circuit to the original state. The adaptation was repeated after a 10-s delay to prove that TFT degradation is not responsible for the result. The complex soma circuit was proposed to be implemented using CMOS transistors; the achievement of spike frequency adaptation with nc-Si TFTs demonstrates that higher order functionalities can be achieved in circuits using low-temperature, ambipolar nc-Si devices. IV. H ARDWARE I MPLEMENTATION I SSUES Soma circuits in a neuromorphic system will not be operated in isolation, but connected to numerous synapses at its output. Different implementations of the synapse circuit have been proposed, including a floating-gate transistor [23], a memristor [24] or a combination of the two [25]. In our calculations, we utilize the last implementation because it achieves a biological synaptic weight change rule using rectangular voltage pulses. This implementation requires each neuron to directly drive another nc-Si TFT. From Fig. 2, it can be assumed that a spike voltage of 3 V is required for changing the synaptic weight (since the output spike is fed to the gate of another TFT). For nc-Si TFTs with 20 × 10 μm channels (as used in the fabricated soma), the individual synaptic load can be calculated to be R0 = 50 G and C0 = 3.45 aF. Since all synapses are connected in parallel, the capacitive load goes up with the number of synapses. In addition, considering interconnect lines with 10-nm width and pitch, and length of 200 nm per synapse, the interconnect capacitance is 14 aF per synapse. To measure the synaptic load capacity of a soma circuit, the variation of spike voltage with different capacitive and resistive loads was measured. Fig. 8(a) displays the spike voltage as a function of the number of synapses modeled at the output. Approximately 9200 synapses can be driven with this soma circuit at VDD = 5 V. The drive capacity increases with supply voltage, and this relation is shown in Fig. 8(b). The capability of the soma circuit to drive numerous synapses renders them of interest for use in learning systems [26], or large-scale networks [9]. For neuromorphic applications that necessitate integration with CMOS logic for high-speed computation, the soma circuits could be built around the CMOS core in a 3-D structure.
1470
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, VOL. 24, NO. 9, SEPTEMBER 2013
Fig. 8. Effect of synaptic load on the spike voltage of a simple soma circuit. (a) Calculation of the load capacity of the circuit at VDD = 5 V. (b) Variation of the load capacity with operating voltage, assuming 3 V voltage pulses are required to change the synaptic weight.
Noncrystalline silicon TFTs suffer from threshold voltage degradation under voltage bias stress [27], [28]. By using higher quality nc-Si as the channel layer, and by lowering impurity oxygen concentration [5], the amount of VT shift in our TFTs has been lowered significantly. However, under typical bias conditions encountered in spiking neuron circuits (VG = 7 V and V D = 5 V), the VT shift is still approximately 0.4 V after a stress period of 104 seconds [6], and could play a detrimental role in circuit operation. To quantify the effect of VT shift on a simple soma circuit, the circuit was stressed for different time periods by operating at a supply voltage of 5 V and input current of 0.8 nA. Subsequently, the discharge curves were measured by varying the input current. The results [Fig. 9(a)] show a gradual decrease in spike frequency upon increasing the stress time. After a stress period of 104 seconds, the spike frequency at an input current of 1 nA decreased by 22%. A similar experiment was also performed on the complex soma circuit, where the spike frequency after different periods of stress were measured, the injected current being held constant at 0.5 nA [Fig. 9(b)]. After 104 seconds, the spike frequency reduced by 7%. However, any VT shift resulting from charge trapping in the gate dielectric can recover during periods of rest when the neuron is inactive. A partial recovery was observed by letting the device rest for a period of 24 h after stress [also Fig. 9(b)]. Furthermore, the VT is observed to saturate at prolonged stress times (>5 h) in accordance with earlier observations [29]. Other approaches to reduce the degradation include performing the nc-Si deposition at a higher temperature and avoiding residual moisture [30]. In a large-scale system, soma circuits would form discrete units connected through synapses. A mismatch of I–V characteristics from individual TFTs could change the spike frequencies of a soma circuit by a small percentage. However, the error from a particular unit should not have a large effect on the entire system because neuromorphic circuits will be designed with a certain level of defect tolerance [31]. For example, if a neuron has a slightly higher threshold of membrane capacitance voltage for firing, it may fire with a slight delay from the expected time instance. However, since the next neuron in line will also rely on the integrated inputs from large number of neurons to fire, the delay from the first neuron may not affect its firing. Also, multiple parallel pathways for
Fig. 9. (a) Discharge curves from a simple soma circuit after different stress periods. (b) Variation of spike frequency in the complex soma circuit after different stress periods when the injected current is held constant. The VT shift saturates after long time periods, and recovers partially after a rest period.
information transfer impart an additional measure of stability to the complete circuit. V. P ERFORMANCE A NALYSIS OF S OMA C IRCUIT It is vital to ensure low power consumption in spiking soma circuits since a large number of these circuits would be used in a neuromorphic system. The instantaneous power consumption can be calculated by multiplying the supply voltage by the current sourced out of the supply node. Integrating the power when the circuit spike frequency is constant provides the total energy consumption at that frequency, including the contribution of standby power. Fig. 10(a) shows an expanded view of the calculation for a simple soma circuit when spiking at 100 Hz using VDD = 6 V. The sharp rises, or ripples, in the energy transient correspond to spikes and much slower rises occur during the stand-by period. At this frequency, the standby power is 5 nW and the total power is 12 nW (approximately 90 pJ/spike). The corresponding values for the complex soma circuit are 23 nW and 90 nW. For a large network of 1010 simple neurons as present in the human brain neocortex, these values translate to 120 W. A CMOS model based on the BSIM3v3 AIM-SPICE built-in model [32] and 180 nm node was also developed for comparing the average power consumption in a simple soma circuit. Transistors with identical device dimension ratios to the TFTs in the fabricated circuit were used in the simulation. Different supply voltages were used in the CMOS simulation. The results [shown in Fig. 10(b)] indicate that when spiking at the same frequency, the power consumption of the fabricated circuit compares well with a CMOS implementation when operated at the same supply voltage. This is because the higher leakage values in nc-Si transistors are compensated by the much lower ONcurrents. The supply voltage can be scaled lower in the CMOS case resulting in lower power I–V characteristics of a TFT can be improved by performing nc-Si deposition at slightly higher temperatures [33], as well as by scaling the gate dielectric thickness [12]; therefore it should also be possible to further scale the supply voltage of the nc-Si TFT-based circuits. The spike durations do not affect the power calculations because they primarily are a function of the circuit capacitances. The power dissipation of fabricated CMOS-based neurons varies from 9 pJ/spike (at VDD = 3.3 V) [13] to 3 nJ/spike (at VDD = 5 V) [14] depending on design complexity and supply voltage.
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, VOL. 24, NO. 9, SEPTEMBER 2013
Fig. 10. (a) Energy dissipation of a fabricated nc-Si simple soma circuit when spiking at a constant frequency of 100 Hz. The ripples correspond to increased power consumption at output spikes. (b) Comparison between the power consumption of a fabricated nc-Si simple soma circuit and corresponding CMOS simulations at different supply voltages.
1471
complex behaviors including spike-frequency adaptation were also realized. The circuits were tested by including different synaptic loads at the output, and their maximum drive capacity was measured to be approximately 9200 synapses. The effect of threshold voltage degradation on the operation of soma circuits was studied. The degradation in output frequency due to bias stress was shown to recover partly after rest. The power consumption of a simple soma circuit was measured under various conditions, and operated at only 12 nW at 100Hz output, translating to 12 W for a large system with 109 neurons. Previous work has demonstrated nanoscale nc-Si TFTs which could be incorporated into these spiking soma circuits. Together with electronic synapses using the low-temperature approach, highly integrated neuromorphic learning systems are certainly feasible. R EFERENCES
Fig. 11. (a) Variation of stand-by and total power consumption with spike frequency in a simple soma circuit with VDD = 6 V. (b) Dependence of power on the supply voltage in the same circuit.
Variation of power consumption of the fabricated Mead circuit with spike frequency and supply voltage is shown in Fig. 11(a) and (b), respectively. Similar trends were also observed in the case of the complex soma circuit. At low spike frequencies (< 30 Hz), the standby power dominates. The total power consumption increases with supply voltage, as can be expected. Employing thin-gate dielectrics in the nc-Si TFTs increases the device performance [12], allowing the use of lower supply voltage in neuromorphic circuits. The spike frequency of soma circuits in neuromorphic applications is expected to be very low (< 200 Hz) [1]. Achieving low stand-by power is crucial since it will dominate at these frequencies. Optimizing the geometry ratio of constituent inverters can reduce it to a large extent. Another method could be to apply a back-gate voltage that shifts the I D -VG curve such that the lowest drain current occurs at VG = 0. A third technique would be to use different S/D metal contacts to prevent the injection of carriers at low gate voltages, thus lowering the off-current at high drain voltages. Efforts to implement these methods are in progress. VI. C ONCLUSION Spiking soma circuits have been fabricated with ambipolar nc-Si TFTs using low-temperature (250 °C maximum) processing. These conditions enable utilization of flexible substrates, as well as 3-D integration with CMOS structures. The soma circuits output discrete rectangular voltage pulses. The spike width and discharge curves associated with these circuits matched closely with that of biological neurons, and
[1] C. Koch, Biophysics of Computation. New York, NY, USA: Oxford Univ. Press, 1999. [2] D. Attwell and S. B. Laughlin, “An energy budget for signaling in the grey matter of the brain,” J. Cerebral Blood Flow Metabolism, vol. 21, no. 10, pp. 1133–1145, Oct. 2001. [3] J. Hawkins and S. Blakeslee, On Intelligence. New York, NY, USA: Henry Holt & Company, 2004. [4] K. D. Cantley, A. Subramaniam, H. J. Stiegler, R. A. Chapman, and E. M. Vogel, “SPICE simulation of nanoscale non-crystalline silicon TFTs in spiking neuron circuits,” in Proc. 53rd IEEE Int. Midwest Symp. Circuits Syst., Aug. 2010, pp. 1202–1205. [5] A. Subramaniam, K. D. Cantley, R. A. Chapman, B. Chakrabarty, and E. M. Vogel, “Ambipolar non-crystalline-silicon TFTs with submicron dimensions and reduced threshold voltage shift,” in Proc. 69th Device Res. Conf., Jun. 2011, pp. 99–100. [6] A. Subramaniam, K. D. Cantley, R. A. Chapman, H. Stiegler, and E. M. Vogel, “Submicron ambipolar nanocrystalline-silicon thin-film transistors and inverters,” IEEE Trans. Electron Devices, vol. 59, no. 2, pp. 359–366, Feb. 2012. [7] H. Gleskova, S. Wagner, V. Gasparik, and P. Kovac, “150 °C amorphous silicon thin-film transistor technology for polyimide substrates,” J. Electrochem. Soc., vol. 148, no. 7, pp. G370–G374, May 2001. [8] E. Y. Ma and S. Wagner, “Amorphous silicon transistors on ultrathin steel substrates,” Appl. Phys. Lett., vol. 74, no. 18, pp. 2661–2662, May 1999. [9] O. Turel, J. H. Lee, X. Ma, and K. K. Likharev, “Neuromorphic architectures for nanoelectronic circuits,” Int. J. Circuit Theory Appl., vol. 32, no. 5, pp. 277–302, Sep. 2004. [10] K. A. Boahen, “Point-to-point connectivity between Neuromorphic Chips using Address Events,” IEEE Trans. Circuits Syst. II, Analog Digital Signal Process., vol. 47, no. 5, pp. 416–433, May 2000. [11] C.-H. Lee, A. Sazonov, M. R. E. Rad, G. R. Chaji, and A. Nathan, “Ambipolar thin-film transistors fabricated by PECVD nanocrystalline silicon,” in Proc. Mater. Res. Soc. Symp., vol. 910. 2006, pp. A05–A22. [12] A. Subramaniam, K. D. Cantley, R. A. Chapman, B. Chakrabarty, and E. M. Vogel, “Submicron ambipolar nanocrystalline-silicon TFTs with high-K gate dielectrics,” in Proc. Int. Semicond. Device Res. Symp., Dec. 2011, pp. 1–2. [13] J. H. B. Wijekoon and P. Dudek, “Integrated circuit implementation of a cortical neuron,” in Proc. IEEE Int. Symp. Circuits Syst., May 2008, pp. 1784–1787. [14] G. Indiveri, “A low-power adaptive integrate-and-fire neuron circuit,” in Proc. IEEE Int. Symp. Circuits Syst., May 2003, pp. 820–823. [15] Y. V. Pershin and M. Di Ventra, “Experimental demonstration of associative memory with memristive neural networks,” Neural Netw., vol. 23, no. 7, pp. 881–886, Sep. 2010. [16] T. Masquelier, R. Guyonneau, and S. J. Thorpe, “Spike timing dependent plasticity finds the start of repeating patterns in continuous spike trains,” PLoS One, vol. 3, no. 1, p. e1377, Jan. 2008. [17] K. D. Cantley, A. Subramaniam, H. J. Stiegler, R. A. Chapman, and E. M. Vogel, “Neural learning circuits utilizing nano-crystalline silicon transistors and memristors,” IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 4, pp. 565–573, Apr. 2012.
1472
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, VOL. 24, NO. 9, SEPTEMBER 2013
[18] A. J. Snell, W. E. Spear, P. G. LeComber, and K. Mackenzie, “Application of amorphous silicon thin film transistors in integrated circuits,” Appl. Phys., vol. A26, pp. 83–86, Jul. 1981. [19] Y. Nara, Y. Kudou, and M. Matsumura, “Application of amorphoussilicon thin-film transistors in three-dimensional integrated circuits,” Jap. J. Appl. Phys., vol. 22, no. 6, pp. L370–L372, Jun. 1983. [20] C. E. Stafstrom, P. C. Schwindt, and W. E. Crill, “Repetitive firing in layer V neurons from cat neocortex in vitro,” J. Neurophysiol., vol. 52, no. 2, pp. 264–277, Aug. 1984. [21] C. Mead, “Neuromorphic electronic systems,” Proc IEEE, vol. 78, no. 10, pp. 1629–1636, Oct. 1990. [22] G. Indiveri, E. Chicca, and R. Douglas, “A VLSI array of low-power spiking neurons and bistable synapses with spike-timing dependent plasticity,” IEEE Trans. Neural Netw., vol. 17, no. 1, pp. 211–221, Jan. 2006. [23] C. Diorio, P. Hasler, B. A. Minch, and C. A. Mead, “A single transistor silicon synapse,” IEEE Trans. Electron Devices, vol. 43, no. 11, pp. 1972–1980, Nov. 1996. [24] B. Linares-Barranco and T. Terrano-Gotarredona, “Exploiting memristance in adaptive asynchronous spiking neurmorphic nanotechnology systems,” in Proc. 9th IEEE Conf. Nanotechnol., Jul. 2009, pp. 978–981. [25] K. D. Cantley, A. Subramaniam, H. Stiegler, R. Chapman, and E. Vogel, “Spike timing-dependent synaptic plasticity using memristors and nanocrystalline silicon TFT memories,” in Proc. 11th IEEE Conf. Nanotechnol., Aug. 2011, pp. 421–425. [26] K. D. Cantley, A. Subramaniam, H. J. Stiegler, R. A. Chapman, and E. M. Vogel, “Hebbian Learning in Spiking Neural Networks with Nanocrystalline Silicon TFTs and Memristive Synapses,” IEEE Trans. Nanotechnol., vol. 10, no. 5, pp. 1066–1073, Sep. 2011. [27] M. J. Powell, C. van Berkel, I. D. French, and D. H. Nicholls, “Bias dependence of instability mechanisms in amorphous silicon thinfilm transistors,” Appl. Phys. Lett., vol. 51, no. 16, pp. 1242–1244, Oct. 1987. [28] Y.-S. Tai, J.-W. Tsai, H.-C. Chung, and F.-C. Su, “Instability mechanisms for the hydrogenated amorphous silicon thin-film transistors with negative and positive bias stresses on the gate electrodes,” Appl. Phys. Lett., vol. 67, no. 1, pp. 76–78, Jul. 1995. [29] M. R. Esmaeili-Rad, A. Sazonov, and A. Nathan, “Absence of defect state creation in nanocrystalline silicon thin film transistors deduced from constant current stress measurements,” Appl. Phys. Lett., vol. 91, no. 11, pp. 113511-1–113511-3, Sep. 2007. [30] D. R. Allee, L. T. Clark, B. D. Vogt, R. Shringarpure, S. M. Venugopal, S. G. Uppili, K. Kaftanoglu, H. Shivalingaiah, Z. P. Li, J. J. Ravindra Fernando, E. J. Bawolek, and S. M. O’Rourke, “Circuit-level impact of a-Si:H thin-film-transistor degradation effects,” IEEE Trans. Electron Devices, vol. 56, no. 6, pp. 1166–1176, Jun. 2009. [31] N. Joye, A. Schmid, Y. Leblebici, T. Asai, and Y. Amemiya, “Fault-tolerant logic gates using neuromorphic CMOS circuits,” in Proc. Res. Microelectron. Electron. Conf., Jul. 2007, pp. 249–252. [32] W. Liu, MOSFET Models for SPICE Simulation: Including BSIM3v3 and BSIM4, 1st ed. New York, NY, USA: Wiley, Feb. 2001. [33] C.-H. Lee, A. Sazonov, and A. Nathan, “High hole and electron mobilities in nanocrystalline silicon thin-film transistors,” J. Non Crystalline Solids, vol. 352, nos. 9–20, pp. 1732–1736, Jun. 2006.
Effect of Input Noise and Output Node Stochastic on Wang’s kWTA John Sum, Senior Member, IEEE, Chi-Sing Leung, Member, IEEE, and Kevin Ho
Abstract— Recently, an analog neural network model, namely Wang’s kWTA, was proposed. In this model, the output nodes are defined as the Heaviside function. Subsequently, its finite time convergence property and the exact convergence time are analyzed. However, the discovered characteristics of this model are based on the assumption that there are no physical defects during the operation. In this brief, we analyze the convergence behavior of the Wang’s kWTA model when defects exist during the operation. Two defect conditions are considered. The first one is that there is input noise. The second one is that there is stochastic behavior in the output nodes. The convergence of the Wang’s kWTA under these two defects is analyzed and the corresponding energy function is revealed. Index Terms— Convergence analysis, energy function, input noise, kWTA, output node stochastic.
I. I NTRODUCTION In conventional design, a traditional kWTA network consists of n nodes and n 2 connections [2], [10], [13]. By reformulating the k winners selection problem as a linear program, Jun Wang and his coworkers succeeded a much simpler structure to realize a kWTA network based on the idea of dual neural network (DNN) [5], [17], [18]. Here, we call it the Wang’s kWTA model. This model consists of n output nodes (which are defined as Heaviside activation functions), 2n connections, and a hidden node (which behaves as a recurrent state variable).1 Because of the Heaviside activation function, Wang in [18] proved the finite time convergence of this kWTA and later Xiao et al. in [20] derived the exact convergence time. Because of its simple structure, this model is particularly suitable for hardware implementation. However, as known in the studies of fault tolerant neural network. Hardware implementation can never be perfect [8], [19], [21], [22]. For example, the electrical noise [21] from the power supplies or thermal effect may be injected into the input signals. In addition, the offset voltage of comparators may drift randomly [22]. In this regard, it is valuable to analyze the effects of input noise and output node stochastic on the dynamic behavior of the Wang’s kWTA model. Manuscript received May 26, 2012; accepted March 24, 2013. Date of publication May 7, 2013; date of current version August 16, 2013. This work was supported in part by Research Grants from Taiwan National Science Council under Grant 100-2221-E-126-015 and Grant 101-2221-E-126-016, and a Research Grant under Grant CityU 115612 from the Research Grants Council of the Government of the Hong Kong Special Administrative Region. J. Sum is with the Institute of Technology Management, National Chung Hsing University, Taichung 40227, Taiwan (e-mail: [email protected]). C.-S. Leung is with the Department of Electronic Engineering, City University of Hong Kong, Kowloon, Hong Kong (e-mail: [email protected]). K. Ho is with the Department of Computer Science and Communication Engineering Providence University, Taichung 43301, Taiwan (e-mail: [email protected]). Digital Object Identifier 10.1109/TNNLS.2013.2257182 Note that an identical model has been proposed independently in [9].
2162-237X © 2013 IEEE
