1188

IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, VOL. 26, NO. 6, JUNE 2015

Output-Feedback Adaptive Neural Control for Stochastic Nonlinear Time-Varying Delay Systems With Unknown Control Directions Tieshan Li, Senior Member, IEEE, Zifu Li, Dan Wang, and C. L. Philip Chen, Fellow, IEEE

Abstract— This paper presents an adaptive output-feedback neural network (NN) control scheme for a class of stochastic nonlinear time-varying delay systems with unknown control directions. To make the controller design feasible, the unknown control coefficients are grouped together and the original system is transformed into a new system using a linear state transformation technique. Then, the Nussbaum function technique is incorporated into the backstepping recursive design technique to solve the problem of unknown control directions. Furthermore, under the assumption that the time-varying delays exist in the system output, only one NN is employed to compensate for all unknown nonlinear terms depending on the delayed output. Moreover, by estimating the maximum of NN parameters instead of the parameters themselves, the NN parameters to be estimated are greatly decreased and the online learning time is also dramatically decreased. It is shown that all the signals of the closed-loop system are bounded in probability. The effectiveness of the proposed scheme is demonstrated by the simulation results. Index Terms— Adaptive output feedback control, neural network (NN), stochastic nonlinear systems, time-varying delays, unknown control directions.

I. I NTRODUCTION ECENTLY, stochastic nonlinear control has received more and more attention due to the existence of stochastic disturbances in many practical systems. With the proposition of many important concepts of stochastic stability theory [1], quite a number of research results on deterministic nonlinear systems have been extended to stochastic nonlinear

R

Manuscript received July 2, 2013; revised June 13, 2014; accepted June 23, 2014. Date of publication July 21, 2014; date of current version May 15, 2015. This work was supported in part by the National Natural Science Foundation of China under Grant 51179019, Grant 61374114, and Grant 61273137, in part by the Program for Liaoning Excellent Talents in University under Grant LR2012016, in part by the Applied Basic Research Program, Ministry of Transport of China, under Grant 2011-329-225-390 and Grant 2013-329-225270, in part by the Fundamental Research Funds for the Central Universities under Grant 313-2014-321, in part by the National Fundamental Research 973 Program of China under Grant 2011CB302801, in part by the Macau Science and Technology Development Foundation, Macau, China, under Grant 008/2010/A1, and in part by the Multiyear Research Grants. (Corresponding author: Zifu Li.) T. Li is with Navigation College, Dalian Maritime University, Dalian 116026, China (e-mail: [email protected]). Z. Li is with Navigation College, Dalian Maritime University, Dalian 116026, China, and also with Navigation College, Jimei University, Xiamen 361021, China (e-mail: [email protected]). D. Wang is with the Marine Engineering College, Dalian Maritime University, Dalian 116026, China (e-mail: [email protected]). C. L. P. Chen is with the Faculty of Science and Technology, University of Macau, Macau 99999, China (e-mail: [email protected]). Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/TNNLS.2014.2334638

systems, for example, Sontags stabilization formula, nonlinear optimality, and backstepping technique [2], [3], where the backstepping approach was developed to the point of a step-by-step design procedure in [4]. The main technical obstacle in the Lyapunov design for stochastic systems is that the I t oˆ stochastic differentiation involves not only the gradient but also the higher order Hessian term. Pan and Basar [5] were the first to solve the stochastic stabilization problem for a class of strict-feedback systems based on a risk-sensitive cost criterion. By employing the quadratic Lyapunov functions, Deng et al. [2], [3] extended the backstepping design method to output-feedback stabilization and state-feedback stabilization of stochastic nonlinear systems, respectively. However, the combined problem of the control of stochastic nonlinear systems with nonlinear uncertainties simultaneously is still a cumbersome issue. As well known, both neural network (NN) and fuzzy logic system (FLS) have been proved to be a useful tool for solving the control problem of uncertain systems with unknown nonlinear functions in practical applications [6]. By combining NNs or FLSs with the backstepping design technique, several adaptive NN or adaptive fuzzy backstepping control schemes have been developed for several classes of stochastic nonlinear systems with mismatched conditions [7]–[11]. For example, a novel adaptive fuzzy backstepping control scheme was proposed based on the observer design in [8] for a class of stochastic nonlinear strict-feedback systems. In [9] and [10], two constructive output-feedback control approaches were developed based on adaptive NNs for two classes of stochastic nonlinear systems, respectively. In addition, by combining backstepping technique with stochastic small-gain approach, a novel robust adaptive fuzzy output feedback controller was presented for a class of stochastic nonlinear systems in [11]. Recently, [12] and [13] extended the above results to a class of stochastic nonlinear large-scale systems, respectively. More recently, a novel adaptive control scheme was proposed for a class of uncertain nonlinear stochastic systems based on fuzzy NN approximation in [14]. However, when approximating the unknown smooth functions using either NNs or FLSs, the number of parameters to be tuned online, i.e., the neural weights of hidden units in a NN, or the fuzzy weights in a FLS, will grow rapidly with the dimension of the argument vector of the function to be approximated [15], which causes the explosion of learning parameters. Consequently, it makes the complex NNs or FLSs

2162-237X © 2014 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

LI et al.: OUTPUT-FEEDBACK ADAPTIVE NEURAL CONTROL FOR STOCHASTIC NONLINEAR TIME-VARYING DELAY SYSTEMS

unsuitable for the real-time systems that are sensitive to time delays, and the time-consuming process is unavoidable during the implementation of these control algorithms. This problem was pointed out by Fischle and Schroder [16] and first solved by Yang and Ren [17] in their pioneering work, where the so-called minimal learning parameter (MLP) algorithm containing much less online adaptive parameters were constructed by fusion of traditional backstepping technique and radialbasis-function (RBF) NNs. Recently, by combining dynamic surface control and MLP techniques, Li et al. [18] first proposed an algorithm that can simultaneously solve both problems of the explosion of learning parameters and the explosion of computation complexity. Then, a novel adaptive neural output feedback controller based on reduced-order observer was explored for a class of uncertain nonlinear Single Input Single Output systems in [19]. More recently, by estimating the maximum of NN parameters, Yu et al. [20] designed an adaptive neural controller containing only one adaptive parameter. Nevertheless, little work has been done to investigate the adaptive NN or fuzzy output-feedback control problem for stochastic nonlinear systems with unknown control directions. It is well known that the control directions, defined as signs of the control gains, are normally required to be known a priori in adaptive control literature. However, the unknown control directions often exist in many practical nonlinear systems. It is also a major source of resulting in instability of the control systems. When the signs of control gains are unknown, the adaptive control problem becomes much more difficult, since we cannot decide the direction along which the control operates. This problem has been remained open till the Nussbaum-type gain was first introduced in [21] for adaptive control of a class of first-order linear systems. Later on, the Nussbaum gain was extended to adaptive control of nonlinear systems with unknown control directions incorporated with different techniques. For example, Liu [22] investigated the output-feedback adaptive stabilization for a class of nonlinear systems with unknown control directions using the linear state transformation technique. Wen and Ren [23] proposed a state observer-based adaptive neural control scheme for the systems with unknown control directions and unmeasurable states. To handle the unknown control directions in stochastic nonlinear systems, Wang et al. [24] and Yu et al. [25] proposed adaptive fuzzy or neural backstepping control methods for stochastic nonlinear systems using the Nussbaum function technique under state feedback framework, respectively. Recently, Wang et al. [26] developed an adaptive fuzzy outputfeedback control scheme for a class of stochastic nonlinear systems with unknown control direction, in which only one control coefficient was assumed unknown. However, little work was dedicated to the unknown control direction problem for stochastic nonlinear systems along with time delays. In many practical control systems, such as biological system, microwave oscillators, nuclear reactors, network system and so on, there often exists time delay, which is frequently a source of instability, and the control performance of these systems is often degraded. Over the past years, the problem of adaptive control design along with the stability analysis

1189

for the time-delay systems has been a popular topic and significant progress has been achieved [27]–[40]. For example, Ge et al. [33] and Hong et al. [34] successfully constructed state feedback controllers for nonlinear time-delay systems, respectively. Hua et al. [35] and Tong et al. [36] investigated the output-feedback control problems for nonlinear time-delay systems, respectively. Then, Chen et al. [38] developed an adaptive consensus control scheme for a class of nonlinear multiagent time-delay systems with the help of NNs approximation. Recently, Chen and Jiao [39] addressed the problem of adaptive NN output-feedback control for a class of uncertain stochastic nonlinear strict-feedback systems with time-varying delays, where the problem of nonlinear observer design was solved by introducing the circle criterion. More recently, without controllable linearization and by first employing the adding-a-power-integer technique to solve the stabilization problem of stochastic time-delay systems, Chen et al. [41] investigated the state-feedback stabilization problem for a class of lower-triangular stochastic time-delay nonlinear systems. However, to the best of the authors knowledge, there is no result reported on the output-feedback control problem of stochastic nonlinear systems with both time-varying delays and unknown control directions. Motivated by the above observations, incorporating the linear state transformation with MLP techniques, an adaptive backstepping output-feedback neural control scheme is proposed for a class of stochastic nonlinear systems with both time-varying delays and unknown control directions. The unknown control coefficients are grouped together and the original system is transformed into a new system by employing the linear state transformation technique. Then, the Nussbaum-type gain function is used to deal with the unknown parameters caused by the unknown control directions in the new system. In addition, an RBF NN is used to approximate all unknown nonlinear terms depending on the delayed output. Still, the MLP technique is used to alleviate the computational burden by estimating the maximum of NN parameters. Compared with the existing results, the main contributions in this paper lie in the following: 1) for the first time, the linear state transformation technique is introduced to the stochastic nonlinear time-varying delay systems with unknown control directions. Moreover, unlike [23] and [39], the assumption on the value of the unknown control coefficients is not needed, both control singularity and unknown control direction can be tackled in this paper; 2) all the unknown output-dependent functions are grouped into a suitable unknown function that is compensated only by one NN. This simplifies the design procedure and reduces the computation loads dramatically; and 3) by estimating the maximum of NN parameters instead of the parameters themselves, the parameters to be estimated are greatly decreased. Hence, the explosion of learning parameters is solved efficiently. The rest of this paper is organized as follows. Section II provides some notations and preliminary knowledge. The problem formulation is presented in Section III. In Section IV, the adaptive output-feedback control design and stability analysis are presented for the stochastic nonlinear time-varying delay systems with unknown directions. Section V gives

1190

IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, VOL. 26, NO. 6, JUNE 2015

some simulation examples to illustrate the effectiveness of the method proposed in this paper, followed by the conclusion presented in Section VI.

random variable, M(t) be a real valued continuous local martingale with M(0) = 0 such that t −ct (δ N(ξ )ξ˙ + ξ˙ )ecτ dτ + M(t). (4) V (x, t) ≤ χ + e

II. N OTATIONS AND P RELIMINARY K NOWLEDGE

t Then, the functions V (x, t), ξ(t) and 0 (δ N(ξ )ξ˙ + ξ˙ )dτ must be bounded in probability. In this paper, the RBF NN will be used to approximate any unknown continuous function F(Z ), namely Fnn (Z ) = W T S(Z ), where Z ∈  Z ⊂ R q is the input vector with q being the input dimension of NNs, W = [w1 , w2 , . . . , wl ]T ∈ Rl is the weight vector with l > 1 being the node number of a NN, and S(Z ) means the basis function vector with si (Z ) being chosen as a Gaussian function si (Z ) = exp[−(Z − μi )T (Z − μi )/ς 2 ], i = 1, 2, . . . ,l, where μi = [μi; , μi2 , . . . , μiq ]T is the center of the receptive field, and ς > 0 is the width of the basis function si (Z ). It has been proven that an RBF NN can approximate any continuous function over a compact set  Z ⊂ R q to an arbitrary accuracy as

0

The following notations will be used throughout this paper. R + denotes the set of all nonnegative real numbers. R n denotes the real n-dimensional space. R n×r denotes the real n × r matrix space. For a given vector or matrix X, X T denotes its transpose. T r (X) denotes its trace when X is square. |X| denotes the Euclidean norm of a vector X, and the corresponding induced norm for matrix X is denoted by X. XF denotes the Frobenius norm of X defined by X F = T r (X T X) with properties X ≤ X F and XY  F ≤ X F Y  F . λmin (X) and λmax (X) denote the minimal eigenvalue and maximal eigenvalue of symmetric real matrix X, respectively. C i denotes the set of all functions with continuous i th partial derivatives. C 2,1 (R n × [−d, ∞); R + ) denotes the family of all nonnegative functions V (x, t) on R n × [−d, ∞); R + with C 2 in x and C 1 in t. C 2,1 denotes the family of all functions with C 2 in the first argument and C 1 in t. Consider an n-dimensional stochastic time-delay system d x = f (t, x(t), x(t − d(t)))dt + g(t, x(t), x(t − d(t)))dw (∀t ≥ 0)

(1)

with initial condition {x(s) : −d ≤ s ≤ 0} = ξ ∈ C Fb 0 × [(−d, 0); R n ], where d(t) : R+ → [0, d] is a Borel measurable function. f : R + × R n × R n → R n and g : R + × R n × R n → R n×r are locally Lipschitz, and w is r -dimensional standard Brownian motion defined on the complete probability space (, F, {Ft }t ≥0 , P) with  being a sample space, with F being a σ field, {Ft }t ≥0 being a filtration, and P being the probability measure. Define a differential operator L known as infinitesimal generator for twice continuously differentiable function V (x, t) ∈ C 2,1 as follows:   ∂V 1 ∂2V ∂V + f (x(t), x(t − d(t)), t) + T r g T . LV = ∂t ∂x 2 ∂x2 (2) Definition 2.1: N(·) is an even smooth Nussbaum-type function, if it satisfies   s  limt →∞ sup 1s 0 N(v)dv  s  (3) limt →∞ inf 1s 0 N(v)dv . From the definition, we know that Nussbaum functions should have infinite gains and switching frequencies. There are many continuous function satisfying these conditions, √ 2 such as ev cos(v), ln(v + 1) cos ln(v + 1), and v 2 cos(v). In this paper, the even Nussbaum function v 2 cos(v) is used. Lemma 2.1 [24]: Consider the stochastic nonlinear system (1), and assume that there exists a smooth function ξ : R + → R, a function V (x, t) ∈ C 2,1 (R n × [−d, ∞); R + ), and a Nussbaum type even function N(·). Let χ be a nonnegative

F(Z ) = W ∗T S(Z ) + δ(Z ).

(5)

The ideal weight vector W ∗ is an artificial quantity required for analytical purposes. It is defined as  





∗ T ˆ sup h nn (Z ) − W S(Z ) . W := arg min Wˆ ∈R q

Z ∈ Z

Assumption 2.1 [25]: For ∀Z ∈  Z , there exists an ideal constant weight vector W ∗ such that W ∗ ∞ ≤ wmax and |δ(Z )| ≤ δmax with bounds wmax and δmax > 0. It is obvious that





W ∗T S(Z ) + δ(Z ) ≤ W ∗T S(Z ) + |δ(Z )| ≤

l

|si (Z )| wmax + δmax

i=1

≤ θβ (Z ) where

(6)

  l  

2 



si (Z ) + 1 , β (Z ) = (1 + l) i=1

θ = max{δmax , wmax }. Assumption 2.2: For 1 ≤ i ≤ n, according to the mean value theorem, the following equalities hold: f i (y, y(t − d(t))) = y(t − d(t)) f¯i (y, y(t − d(t))) (7) ¯ h i (y, y(t − d(t))) = y(t − d(t))h i (y, y(t − d(t))) (8) where f i (y, 0) = 0, h i (y, 0) = 0, for 0 < ϑ fi , ϑh i < 1, the unknown functions satisfy

∂ fi (y, s)

¯ f i (y, y(t − d(t))) = ∂s s=ϑ f y(t −d(t )) i



(y, s) ∂h i

h¯ i (y, y(t − d(t))) =

∂s s=ϑh i y(t −d(t )).

Remark 2.1: These unknown nonlinear functions depending on the delayed output will be grouped into a suitable unknown function that will be compensated by only one NN.

LI et al.: OUTPUT-FEEDBACK ADAPTIVE NEURAL CONTROL FOR STOCHASTIC NONLINEAR TIME-VARYING DELAY SYSTEMS

III. P ROBLEM F ORMULATION Consider the following stochastic nonlinear time-varying delay system described by: ⎧   ⎪ d x i = gi x i+1 + φi (y) + f i‘ (y, y(t − d(t))) dt ⎪ ⎪ ⎪ ⎪ + h i (y, y(t − d(t)))dw ⎪ ⎪ ⎪ ⎪ ⎨ .. . (9) ⎪ ⎪ d x n = [gn u + φn (y) + f n (y, y(t − d(t)))] dt ⎪ ⎪ ⎪ ⎪ + h n (y, y(t − d(t)))dw ⎪ ⎪ ⎪ ⎩ y = x1 where x i ∈ R(i = 1, . . . n), u ∈ R, and y ∈ R are the unmeasured system states, the control input, and the system output, respectively. gi = 0, (i = 1, 2, . . . , n) called control coefficients, are unknown constants with unknown directions. φi (y) are known smooth nonlinear functions with φi (0) = 0. fi (·) : R 2 → R and h i (·) : R 2 → Rr are unknown locally Lipschitz smooth functions with f i (y, 0) = 0 and h i (y, 0) = 0. The uncertain time-varying delay d(t) : R + →[0, d] sat˙ isfies d(t) ≤ ς < 1, with ς being an unknown constant. The initial condition {x(s) : −d ≤ s ≤ 0} = ξ ∈ C Fb 0 × [(−d, 0); R n ] is unknown. w has been defined in (1). Only the system output y can be available for measurement. Remark 3.1: It is known that nonlinear systems with unknown directions exist in a wide range of physical and engineering practices, such as in ship control problem and air space control problem. When all the control directions of the nonlinear systems are unknown, the control design for such systems will become more challenging. It has been shown that the Nussbaum function technique is an effective tool to deal with the unknown control coefficients, and various results have been obtained, but most of them are for state feedback, or output feedback with only one unknown control coefficient. Therefore, (9) is more general. Remark 3.2: In this paper, we only consider that the smooth functions φi (·) are the function of the output y, but not the function of the system state vector x¯i . φi (0) = 0 implies that the origin is the equilibrium point of (9). gi are positive or negative but not zero, which satisfies the controllability condition of (9). Remark 3.3: The time-varying delays of (9) only exist in the system output. Similar to [10], [39], and [41], the timevarying delay problem will be solved by constructing an appropriate Lyapunov–Krasovskii function. However, they did not consider the cases with all unknown control directions along with time-varying delays in the stochastic nonlinear systems. Since the values and signs of the unknown control coefficients are unknown, the output-feedback control design becomes very hard. To make the output-feedback control design feasible, (9) is transformed into a new system using the linear state transformation technique. Therefore, the original unknown control coefficients are grouped together in the new states.

1191

 Let ζi = x i / nj =i g j , for i = 1, . . . , n, then, the new state variables ζ = [ζ1 , . . . , ζn ] are given by   ⎧ ⎪ n 1 ⎪ dζ = ζ + (φ (y) + f (y, y(t − d(t)))) dt i i+1 i i ⎪ ⎪ j =i g j ⎪ ⎪ ⎪ ⎪ + n 1 g h i (y, y(t − d(t)))dw ⎪ ⎨ j =i j .. (10) ⎪ . ⎪   ⎪ ⎪ ⎪ ⎪ dζn = u + g1n (φn (y) + f n (y, y(t − d(t)))) dt ⎪ ⎪ ⎪ ⎩ + g1n h n (y, y(t − d(t)))dw. It is obvious that all the control coefficients are known in (10). If all the states of (10) are available, it is easy to design a controller. But, because of the existence of unknown control coefficients in the linear state transformation, all the states of (10) are unavailable. Therefore, a full-order state observer must be established first to estimate the unmeasured states. Then, a novel NN adaptive output-feedback control scheme will be explored based on the designed observers. Remark 3.4: After the linear state transformation, (9) becomes a strict-feedback uncertain nonlinear system with  both unknown parameters 1/ nj =i g j and unknown timevarying delays. Therefore, it is much feasible to design a better controller. Consider the observers for (10) as follows: ⎧ ⎪ ζ˙ˆ = ζˆi+1 − ki ζˆ1 ⎪ ⎨ i .. (11) . ⎪ ⎪ ⎩ ˙ˆ ζ n = u − kn ζˆ1 where k1 , . . . , kn are optional positive constants such that the following matrix: ⎤ ⎡ −k1 I n−1 ⎥ ⎢ A = ⎣ ... ⎦ −kn 0 · · · 0 is asymptotically stable. Denote ζˆ = [ζˆ1 , . . . , ζˆn ] and let ζ˜ = ζ − ζˆ be the observe error. For convenience, define ζˆn+1 = u and ζ˜n+1 = 0. By combining (10) and (11), the observer error dynamics of ζ˜ can be obtained as follows: d ζ˜ = [ Aζ˜ + (y) + F(y, y(t − d(t))) + K ζ1 ]dt + H (y, y(t − d(t)))dw (12) where

# (·) =

φn n ,..., gn j =1 g j φ1

$T

K = [k1 , . . . , kn ]T # $T f1 fn F(·) = n ,..., gn j =1 gi #  h1 hn T H (·) = n ,..., . gn j =1 g j Hence, there exists a positive definite matrix P = P T such that A T P + P A = −I .

1192

IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, VOL. 26, NO. 6, JUNE 2015

Then, the complete system can be expressed as ⎧   ˜ ˜ ⎪ ⎪d ζ = Aζ + (y) + F(y, y(t − d(t))) + K ζ1 dt ⎪ ⎪ ⎪ + H (y, y(t − d(t)))dw ⎪ ⎪ ⎪ ⎪ ⎪ d y = [gζ ¯ 2 + φ1 (y) + f i (y, y(t − d(t))] dt ⎪ ⎨ + h 1 (y, y(t − d(t)))dw   ⎪ ⎪ ˆ d ζi = ζˆi+1 − ki ζi dt ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ... ⎪ ⎪ ⎪ ⎩ ˆ d ζn = [u − kn ζn ] dt  where g¯ = nj =1 g j .

(13)

IV. A DAPTIVE O UTPUT-F EEDBACK C ONTROL D ESIGN AND S TABILITY A NALYSIS In this section, the adaptive backstepping technique will be used to design an adaptive NN output-feedback controller. To simplify the design procedure, some derivations are omitted, and only the main design procedures are given. Step 1: Following the adaptive backstepping design idea, we define the error variables as follows: ˆ z 1 = y, z 2 = ζˆ2 − α1 (y, ξ, ζˆ1 , θˆ , l) where α1 (·) is a smooth function to be determined later, and ξ is the variable of Nussbaum-type function N(ξ ). θˆ denotes the estimate of θ and the error θ˜ = θ −θˆ . lˆ denotes the estimate ˆ l is an unknown constant defined as of l and the error l˜ = l − l,  4/3  3 g¯ 3 g¯ 4/3 3 g¯ 4/3 l = max , , 4δi1 4δi2 4δ j 3 where i = 1, . . . , n, j = 2, . . . , n, δi1 , and δi2 are known parameters to be designed later. For simplicity, here and hereafter, f i = f i (y, y(t − d(t))), h i = h i (y, y(t − d(t))), F = F(y, y(t − d(t))), and H = H (y, y(t − d(t))). Then, the differential of z 1 is   dz 1 = g( ¯ ζ˜2 + ζˆ2 ) + φ1 (y) + f 1 dt + h 1 dw. (14) Remark 4.1: By defining the unknown constant l, rather than limiting the value of unknown parameters g¯ i in (10), the difficulty caused by the unknown control directions can be overcome using integrator backstepping approach together with tuning function technique and a Nussbaum-type function. Consider the following Lyapunov function candidate: b % ˜ T ˜ &2 1 4 1 ˜T −1 ˜ V1 = (15) ζ P ζ + z 1 + l  l. 2 4 2 It follows from (2), (13), and (14) that: LV1 = −b ζ˜ T P ζ˜ |ζ˜ |2 + 2b ζ˜ T P ζ˜ (T (y)P ζ˜ ) ' ( 3 + bT r (2P ζ˜ ζ˜ T P + ζ˜ T P ζ˜ P)H H T + z 12 h 1 h 1T 2 + 2bζ˜ T P ζ˜ ((K ζ1 )T P ζ˜ ) + 2b ζ˜ T P ζ˜ (F T P ζ˜ )   ˜ + z 13 g( ¯ ζ˜2 + ζˆ2 ) + φ1 (y) + f1 − l˜T  −1 l. (16) Then, using (16) and (A.1)–(A.8) in Appendix A, one has  ) * 3 3 ¯ 1 + φ1 (y) + z 1 2lˆ + + LV1 ≤ z 13 gα 4λ11 4λ12 1 + δ12 z 24 − bζ˜ T P ζ˜ |ζ˜ |2 + 1 − l˜T  −1 (l˙ˆ − τ1 ) (17) 4

where τ1 = 2z 14 . According to the well-known mean value theorem, we can get φi (y) = y φ¯ i (y). ˆ with Step 2: Define z 3 = ζˆ3 − α2 (y, ξ, ζˆ1 , ζˆ2 , θˆ , l) α2 (·) being a smooth function to be determined later. Then from (13), the differential of z 2 is  ∂α1 dz 2 = ζˆ3 − k2 ζˆ1 − (g( ¯ ζ˜2 + ζˆ2 ) + φ1 (y) + f 1 ) ∂y ∂α1 ∂α1 ˙ ∂α1 ˙ˆ − (ζˆ2 − k1 ζˆ1 ) − θˆ − l ∂ ζˆ1 ∂ θˆ ∂ lˆ  ∂α1 1 ∂ 2 α1 T ∂α1 ˙ ξ dt − h 1 dw. (18) h h − − 1 1 2 2 ∂y ∂ξ ∂y Consider the following Lyapunov function candidate: 1 V2 = V1 + z 24 . 4

(19)

The derivative of V2 satisfies LV2

 & & ∂α1 % % ˜ g¯ ζ2 + ζˆ + φ1 (y) + f 1 ≤ LV1 +z 23 ζˆ3 − k2 ζˆ1 − ∂y ∂α1 ∂α1 ˙ ∂α1 ˙ˆ 1 ∂ 2 α1 (ζˆ2 −k1 ζˆ1 )− h 1 h 1T l− θˆ − 2 ∂ y2 ∂ ζˆ1 ∂ θˆ ∂ lˆ  * ) 3 3 ∂α1 2 ∂α1 ˙ ξ + z2 − h 1 h 1T . (20) ∂ξ 2 ∂y

−

Then, following (17), (20), and (A.9)–(A.14), where i = 2, and (A.15) in Appendix A, one has:  ) ¯ 1 + φ1 (y) + z 1 2lˆ + LV2 ≤ z 13 gα

3 3 + 4λ11 4λ12

*

1 − b ζ˜ T P ζ˜ |ζ˜ |2 + 2 + δ23 y 4 + z 23 (α2 − 2 ) 4 1 (21) + λ24 z 34 − l˜T  −1 [lˆ˙ − τ2 ] 4 4/3

where τ2 = τ1 + z 24 (∂α1 /∂y)4/3 (2 + α¯ 1 ). Step i: (3 ≤ i ≤ n − 1) Define the error variable z i = ζˆi − αi−1 , z i+1 = ζˆi+1 − αi , where αi (·) is a smooth function to be determined later. Then, from (13), the differential of z i is  ∂αi−1 dz i = ζˆi+1 − ki ζˆ1 − (g( ¯ ζ˜2 + ζˆ2 ) + φ1 (y) + f 1 ) ∂y −

i−1 ∂αi−1 j =1

∂ ζˆ j

(ζˆ j +1 − k j ζˆ1 ) −

∂αi−1 ˙ ∂αi−1 ˙ˆ θˆ − l ∂ θˆ ∂ lˆ

 ∂αi−1 ˙ 1 ∂ 2 αi−1 ∂αi−1 T h 1 dw. h1h1 − − ξ dt − 2 ∂ y2 ∂ξ ∂y (22) Consider the following Lyapunov function candidate: 1 Vi = Vi−1 + z i4 . 4

(23)

LI et al.: OUTPUT-FEEDBACK ADAPTIVE NEURAL CONTROL FOR STOCHASTIC NONLINEAR TIME-VARYING DELAY SYSTEMS

Similar to the Step 2, and noting (A.9)–(A.14), (A.16), and (A.18) in Appendix A, the differential of Vi satisfies  ) * 3 3 LVi ≤ z 13 gα + ¯ 1 + φ1 (y) + z 1 2lˆ + 4λ11 4λ12 − b ζ˜ T P ζ˜ |ζ˜ |2 + z 23 [α2 − 2 ] +

i

z 3j [α j − i ]

j =3

1 λ j 4 z 4j +1 − l˜T  −1 [l˙ˆ − τi ] + i + 4 i

(24)

j =3

4/3 α¯ i−1 ).

where τi = Step n: Define the error variable z n = ζˆn − αn−1 . Then, from (13), the differential of z n is  ∂αn−1 (g( ¯ ζ˜2 + ζˆ2 ) + φ1 (y) + f 1 ) dz n = u − kn ζˆ1 − ∂y n−1 ∂αn−1 ∂αn−1 ˙ ∂αn−1 ˆ˙ (ζˆi+1 − ki ζˆ1 ) − − θˆ − l ˆ ˆ ∂ ζ ∂ θ ∂ lˆ i i=1  ∂αn−1 ∂αn−1 1 ∂ 2 αn−1 T ˙ ξ dt − h 1 dw. h1h1 − − 2 2 ∂y ∂ξ ∂y (25) τi−1 + z i4 (∂αi−1 /∂y)4/3 (2 +

Consider the following Lyapunov function candidate: 1 Vn = Vn−1 + z n4 . (26) 4 Repeating the similar operation in the former steps, and with the help of Itˆo formula and (A.17) in Appendix A, the differential of Vn satisfies  ) * 3 3 3 ˆ ¯ 1 + φ1 (y) + z 1 2l + + LVn ≤ z 1 gα 4λ11 4λ12 # n 4 1 3 δ j 1 + bP2 ε13 − b ζ˜ T P ζ˜ |ζ˜ |2 + 4 2 i=1

4 3 + ε4 (2bP + bPλmax ( p)) + bP2 ε23 2  4 3 + bP2 ε33 |ζ˜ |4 + z 23 [α2 − 2 ] 2 n−1 z i3 [αi − i ] + [u − n ] + y 4 θβ(Z ) +

i=3 4

¯

+ y (t − d(t))e−r d ψ(y, y(t − d(t))) 1 y 4 ψ(y, y(t)) − l˜T  −1 [l˙ˆ − τn ] − 1−ς

where r is a known positive scalar, d¯ = max{d(t)}, ψ(y, y(t − d(t))) = ψ1 (y, y(t −d(t)))+ψ2 (y, y(t −d(t))), which satisfies  4 n 1 1 r d¯ n ψ1 (y, y(t − d(t))) = 4 e bP2 f¯i gj 2ε2 j =i i=1 1 r d¯ e (2bP + bPλmax ( p)) ε4  4 n 1 ¯ n (28) × hi j =i g j

+

i=1

1 r d¯ 3 ¯ λi1 f¯14 + er d λ j 2 h¯ 41 e 4 4 n

ψ2 (y, y(t − d(t))) =

n

i=1

1 ¯ + er d 4

n

i=1

λ j 3 h¯ 41 .

(29)

i=2

Now, define a new smooth function as follows: 4  n n 1 1 1 2 ¯ n f = δi3 + 4 bP φi (y) 4 2ε1 j =i g j i=2 i=1

 4 n 1 1 1 2 ki n ψ(y, y(t)). + 4 bP + 1−ς 2ε3 j =1 g j i=1

Since gi are unknown control coefficients, f cannot be directly implemented to construct the virtual controller α1 . Thus, the RBF NN approximation property will be employed here so that f can be approximated by an RBF NN in the form of (6) on the compact set z as follows: f = θβ(Z ).

(30)

Now, consider the following positive definite function for the whole closed-loop system: ¯ t 1 e−r d V = Vn + λ−1 θ˜ 2 + erυ ψ(y, y(υ))dυ (31) 2 1 − ς t −d(t ) where λ is a design parameter and ψ(·) is a positive continuous ˙ function to be determined. Notice that d(t) ≤ ς < 1, the differential of V satisfies  ) * 3 3 LV ≤ z 13 gα ¯ 1 + φ1 (y) + z 1 2lˆ + + 4λ11 4λ12   $ z 4 β(Z ) + θˆ β(Z ) tanh 1 σ n  4 4 3 1 3 δ j 1 − bP2 ε23 − bλmin ( p) − bP2 ε13 − 2 4 2 i=1

−

 3 bP2 ε3 − ε4 (2bP + bPλmax ( p)) |ζ |4 2 4 3

+ z 23 [α2

− 2 ] +

n−1

z i3 [αi − i ] + [u − n ]

i=3

**$ ) ) 4 z 1 β(Z ) ˙

3 −λ θ θˆ − λ λθˆ + λz 1 β(Z ) tanh σ   ¯ t e−r d 1 −r erυ ψ(y, y(υ))dυ − ωl˜2 1 − ς t −d(t ) 2 #

(27)

1193

−1 ˜

  1 − λ θ˜ 2 + D − l˜T  −1 l˙ˆ − (−ωlˆ + τn ) 2

(32)

where D = 1/2ωl 2 + 1/2λ θ 2 + 0.2785σ θ . From (32), we can design the virtual control law, the actual control law and the adaptive laws as follows: α1 = N(ξ )η

(33)

1194

IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, VOL. 26, NO. 6, JUNE 2015

∂α1 ∂α1 (ζˆ2 − k1 ζˆ1 ) φ1 (y) + ∂y ∂ ζˆ1 ∂α1 ˙ ∂α1 ˙ˆ ∂α1 ξ˙ − z 2 + θˆ + l+ ∂ξ ∂ θˆ ∂ lˆ ⎡ ) *4 1 ∂α1 3 1 3 3 ⎣ δ12 + δ22 + × + 4 4 4λ24 4λ21 ∂y

where c = min{2c0 , 4ci , λ λ, ω}, i = 1, 2, . . . , n, and

α2 = −c2 z 2 + k2 ζˆ1 +

)

*2 ∂ 2 α1 2 z2 ∂ y2

)

3 ∂α1 4λ22 ∂y ⎤ * 43 % ) 4& ∂α 1 2 + α¯ 13 ⎦ + lˆ ∂y

+

1 4λ23

n

i=1

3 − bP2 ε3 − ε4 (2bP + bPλmax ( p)). 2 4 3

*4

+

(34)

∂αi−1 ∂αi−1 αi = −ci z i + ki ζˆ1 + (ζˆ j +1 − k j ζˆ1 ) φ1 (y) + ˆj ∂y ∂ ζ j =1  ∂αi−1 ˙ˆ ∂αi−1 ˙ˆ ∂αi−1 ˙ 1 λi−1,4 + l+ ξ − zi θ+ ˆ ˆ ∂ξ 4 ∂θ ∂l ) *4 3 1 3 3 ∂αi−1 3 + + δi2 + + 4λi4 4 4λi4 4λi1 ∂y *4 ) 2 *2 ) ∂ αi−1 ∂αi−1 1 3 2 + zi + 4λi3 ∂ y2 4λi2 ∂y ⎤ 4 ) * 4& ∂αi−1 3 % 2 + α¯ i3 ⎦ + lˆ (35) ∂y i−1

∂αn−1 ∂αn−1 ˙ˆ ∂αn−1 ˙ˆ φ1 (y) + u = −cn z n + kn ζˆ1 + l θ+ ∂y ∂ θˆ ∂ lˆ +

n−1 ∂αn−1

∂ ζˆ j

j =1

⎡

(ζˆ j +1 − k j ζˆ1 ) +

∂αn−1 ξ˙ ∂ξ

) *4 1 3 1 ∂αn−1 3 ⎣ −z n λn−1,4 + δn2 + 4 4 4λn1 ∂y ) 2 *2 ) *4 1 3 ∂ αn−1 ∂αn−1 2 z + n 4λn3 ∂ y2 4λn2 ∂y ⎤ *4 ) 4& ∂αn−1 3 % + lˆ 2 + α¯ n3 ⎦ ∂y +

(36)

) * * 43 ) n 4 ˙ˆ 4 ∂αi−1 3 ˆ z j 2 + α¯ 1 l = −ωl + τ1 + ∂y

(37)

* ) 4 z 1 β(Z ) ˙

ˆ 4 ˆ θ = −λ λθ + λz 1 β(Z ) tanh σ

(38)

j =2

where ξ˙ = z 13 η, τ1 = 2z 14 , η = z 1 [c1 +2lˆ+3/4λ11 +3/4λ12 + θˆ β(Z ) tanh(z 14 β(Z )/σ )] + φ1 (y). Substituting (33)–(38) into (32) yields LV ≤ −cV + [g¯ N(ξ ) + 1]ξ˙ + D

4 4 1 3 3 δ j 1 − bP2 ε13 − bP2 ε23 4 2 2

c0 = bλmin (P) −

(39)

(40)

Theorem 4.1: Consider the closed-loop system consisting of (9), the observer (11), the control law (36), and the adaptive laws (37) and (38). Under the Assumptions 2.1 and 2.2, the following properties can be obtained: 1) all the involved signals are bounded in probability and 2) the error signal can + converge to the compact set defined as z := {z i ∈ R|E( ni=1 z i4 ) ≤ 4}. Proof: See Appendix B. V. S IMULATION E XAMPLE In this section, two simulation examples will be given to illustrate the effectiveness of the proposed control method. Example 5.1: Consider the following second-order stochastic nonlinear system: ⎧ ⎪ d x (t) = [g1 x 2 (t) + y 2 (t) + y(t)y(t − d(t))]dt ⎪ ⎪ 1 ⎪ ⎪ ⎪ ⎪ + sin(y(t − d(t)))dw ⎪ ⎨ (41) d x 2 (t) = [g2 u + y(t) + sin(y(t)y(t − d(t))]dt ⎪ ⎪ 2 ⎪ ⎪ + y(t − d(t))e−y (t −d(t ))dw ⎪ ⎪ ⎪ ⎪ ⎩ y(t) = x (t) 1 where g1 and g2 represent the unknown control direction coefficients, d(t) = 2 + 0.5 cos(t). Manipulating the linear state transformations with ζ1 = x 1 /(g1 g2 ) and ζ2 = x 2 /g2 , the following system can be obtained: ⎧   1 2 1 ⎪ ⎪ ⎪ dζ (t) = ζ (t) + y (t) + y(t)y(t − d(t)) dt 1 2 ⎪ ⎪ g1 g2 g1 g2 ⎪ ⎪ ⎪ 1 ⎪ ⎪ ⎪ sin(y(t − d(t)))dw + ⎪ ⎪ g1 g2 ⎪   ⎪ ⎪ ⎪ ⎨dζ (t) = u + 1 y(t) + 1 sin(y(t)y(t − d(t)) dt 2 g2 g2 (42) ⎪ ⎪ 1 2 (t −d(t )) ⎪ −y ⎪ + y(t − d(t))e dw ⎪ ⎪ g2 ⎪   ⎪ ⎪ ⎪ ⎪ 2 (t) + y(t)y(t − d(t)) dt ⎪ d y(t) = g g ζ (t) + y ⎪ 1 2 2 ⎪ ⎪ ⎪ ⎪ ⎩ + sin(y(t − d(t)))dw. Design the observer for (42) as follows: ⎧ ⎨ζ˙ˆ1 = ζˆ2 − ζˆ1 ⎩ζ˙ˆ = u − ζˆ . 2

(43)

1

Define the error variables z 1 = y and z 2 = ζˆ2 − α1 (y, ξ, ˆ By virtual of the design procedures and results given ζˆ1 , θˆ , l). in Section IV, the signal ξ , the adaptive output feedback virtual

LI et al.: OUTPUT-FEEDBACK ADAPTIVE NEURAL CONTROL FOR STOCHASTIC NONLINEAR TIME-VARYING DELAY SYSTEMS

Fig. 1.

Response of x1 , xˆ1 , and the estimate error x˜1 .

Fig. 2.

Response of x2 , xˆ2 , and the estimate error x˜2 .

Fig. 3.

Nussbaum function signal ξ .

Fig. 4.

Control input u.

Fig. 5.

Adaptive parameter θ .

1195

control law α1 , the actual control law u, and the adaptive laws are given by ξ˙ = z 13 η, α1 = N(ξ )η, α¯ 1 = α1 /y ) 4  * z β(Z ) 3 3 ˆ + + θβ(Z ) tanh 1 η = z 1 c1 + 2lˆ + 4λ11 4λ12 σ + y 2 (t) u = −c2 z 2 + ζˆ1 +

∂α1 ∂α1 ∂α1 ˙ φ1 (y) + (ζˆ2 − ζˆ1 ) + θˆ ˆ ∂y ∂ ζ1 ∂ θˆ

∂α1 ˙ˆ ∂α1 ξ˙ l+ ∂ξ ∂⎡ lˆ * 43 ) ) 2 *2 ∂α ∂ α1 2 1 3 1 1 1 −z 2 ⎣ δ22 + λ24 + + z2 4 4 4λ21 ∂y 4λ23 ∂ y 2 ⎤ ) ) * *4 * 43 ) 4 3 ∂α ∂α1 1 + + lˆ 2 + α¯ 13 ⎦ 4λ22 ∂y ∂y +

) * * 43 ) 4 ∂α 1 ˙ˆ 4 4 l = −ωlˆ + 2z 1 + z 2 2 + α¯ 13 ∂y ) 4 * z β(Z ) θ˙ˆ = −λ λθˆ + λz 14 β(Z ) tanh 1 . σ In this simulation, one RBF NN is used to approximate the unknown function, which contains 100 nodes and the width of the basis function is chosen as five. The Nussbaum function is chosen as N(ξ ) = ξ 2 cos(ξ ). The following initial conditions and suitable parameters can be chosen as x 1 (0) = 0.5, ζ2 (0) = 0.5, ζˆ1 (0) = 0.7, ζˆ2 (0) = 1, ξ(0) = 0.5, θˆ (0) = 0.8, ˆ = 0.5, c1 = c2 = 1, λ11 = λ12 = 5, σ = 0.05, δ22 = 0.05, l(0) λ21 = λ22 = λ23 = 200, λ24 = 1, ω = 400,  = 0.01, λ = 0.02, and λ = 200. If the unknown control coefficients are chosen as g1 = −0.8 and g2 = −0.5, the simulation results are shown in Figs. 1–6. Then, changing the unknown control coefficients as g1 = +0.8 and g2 = +0.5, the simulation results, with the same initial conditions, the parameters, the controller and the adaptive laws, are shown in Figs. 7–12. From the simulation results, it can be seen that the proposed controller in this paper is effective. Example 5.2: To further illustrate the effectiveness of our results, we consider the following practical example of a pendulum system with stochastic disturbances [42]: 1 ml q¨ = −mg sin q − klq + u l

(44)

where u is the torque applied to the pendulum, q is the anticlockwise angle between the vertical axis through the pivot point and the rod, g is the gravity acceleration, and the constants k, l, and m denote a coefficient of friction,

1196

IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, VOL. 26, NO. 6, JUNE 2015

Fig. 6.

Adaptive parameter l.

Fig. 10.

Control input u.

Fig. 7.

Response of x1 , xˆ1 , and the estimate error x˜1 .

Fig. 11.

Adaptive parameter θ .

Fig. 8.

Response of x2 , xˆ2 , and the estimate error x˜2 .

Fig. 12.

Adaptive parameter l.

time-delay coefficient s, which satisfy s ∈ [0, 1]. Let x 1 = q/k, x 2 = ml q˙ + klq. The nonlinear stochastic system with time delays can be expressed as follows:

Fig. 9.

Nussbaum function signal ξ .

the length of the rod, and the mass of the bob, respectively. It is assumed that the constant l is unknown. Since there are not stochastic disturbances and time delays in the pendulum systems, we introduce the stochastic disturbances and the

⎧ ⎪ d x 1 = (g1 x 2 − s mk x 1 − (1 − s)x 1 (t − d(t)))dt ⎪ ⎪ ⎪ ⎪ ⎪ + 0.5(x 1(t − d(t)))2 dw ⎨ d x 2 = (g2 u −smg sin kx 1 +(1−s) sin(x 1 (t −d(t))))dt (45) ⎪ ⎪ ⎪ + 0.5(x 1(t − d(t)))2 dw ⎪ ⎪ ⎪ ⎩ y=x 1 where the time-delay coefficient is chosen as s = 0.5, g1 = 1/mkl and g2 = 1/l represent the unknown control direction coefficients. d(t) = 0.5(1 + sin t (t)) is the time delay. After the same linear state transformation as the Example 5.1, (45) can be rewritten in the form of (10).

LI et al.: OUTPUT-FEEDBACK ADAPTIVE NEURAL CONTROL FOR STOCHASTIC NONLINEAR TIME-VARYING DELAY SYSTEMS

Then, the following system can be obtained:   ⎧ 1 k 1 ⎪ ⎪ dζ x (t) = ζ (t)− s − (1−s)x (t − d(t)) dt 2 1 1 ⎪ 1 ⎪ g1 g2 m g1 g2 ⎪ ⎪ ⎪ 1 ⎪ ⎪ ⎪ 0.5(x 1 (t − d(t)))2 dw + ⎪ ⎪ g g ⎪ 1 2   ⎪ ⎨ 1 1 dζ2 (t) = u − smg sin kx + (1−s) sin(x 1 (t −d(t))) dt g2 g2 ⎪ ⎪ ⎪ 1 ⎪ ⎪ + 0.5(x 1 (t − d(t)))2 dw ⎪ ⎪ ⎪ g2 ⎪   ⎪ k ⎪ ⎪ g d y(t) = g 1 2 ζ2 (t) − s m x 1 − (1 − s)x 1 (t − d(t)) dt ⎪ ⎪ ⎩ + 0.5(x 1 (t − d(t)))2 dw. (46)

Fig. 13.

Response of x1 , xˆ1 , and the estimate error x˜1 .

Fig. 14.

Response of x2 , xˆ2 , and the estimate error x˜2 .

Fig. 15.

Control input u.

1197

Based on the proposed control scheme, the foregoing control problem is easily solved. First, the observer is designed as follows:  ζ˙ˆ1 = ζˆ2 − ζˆ1 (47) ζ˙ˆ2 = u − ζˆ1 . Then, define the error variables z 1 = y, z 2 = ζˆ2 − α1 , and follow the same manipulations in Section IV, the signal ξ , the virtual control law α1 , the actual control law u, and the adaptive laws can be given as follows ξ˙ = z 13 η, α1 = N(ξ )η, α¯ 1 = α1 /y: ) 4 *  z β(Z ) 3 3 + + θˆ β(Z ) tanh 1 η = z 1 c1 + 2lˆ + 4λ11 4λ12 σ k − s x1 m ∂α1 ∂α1 ∂α1 ˙ φ1 (y) + u = −c2 z 2 + ζˆ1 + (ζˆ2 − ζˆ1 ) + θˆ ˆ ∂y ∂ ζ1 ∂ θˆ ∂α1 ˙ˆ ∂α1 ξ˙ + l+ ∂ξ ∂ lˆ *4  ) 1 ∂α1 3 1 3 −z 2 δ22 + λ24 + 4 4 4λ21 ∂y ) 2 *2 ) *4 1 3 ∂ α1 ∂α1 2 + z + 2 4λ23 ∂ y 2 4λ22 ∂y * 43 ) * ) 4 ∂α1 2 + α¯ 13 + lˆ ∂y ) * * 43 ) 4 ˙ˆ 4 4 ∂α1 3 ˆ 2 + α¯ 1 l = −ωl + 2z 1 + z 2 ∂y * ) 4 z 1 β(Z ) ˙

ˆ 4 ˆ . θ = −λ λθ + λz 1 β(Z ) tanh σ Similar to the Example 5.1, in this simulation, one RBF NN is taken for the unknown function, which contains 100 nodes and the width of the basis function is chosen as five. The Nussbaum function is chosen as N(ξ ) = ξ 2 cos(ξ ). The following initial conditions and suitable parameters can be chosen as m = 2,k = 0.98, l = 1, g = 9.8, x 1 (0) = 1, ζ2 (0) = 0,ζˆ1 (0) = 0.5, ζˆ2 (0) = 0, ζ (0) = 0, θˆ (0) = 0, ˆ = 0, c1 = c2 = 2, λ11 = λ12 = 10, σ = 0.1, δ22 = 0.01, l(0) λ21 = λ22 = λ23 = 12, λ24 = 26, ω = 0.4,  = 0.005, λ = 0.1, and λ = 10. The simulation results are shown in

Figs. 13–15, from which we can see that the control performance is still very well. To demonstrate the superiority of the algorithm proposed in this paper, a comparative simulation on (45) will be also given using the algorithm proposed in [42]. The initial conditions and suitable design parameters are chosen as x 1 (0) = 0.1, xˆ1 (0) = 0.1, x 2 (0) = xˆ2 (0) = 0, m = 2, k = 0.98, l = 1, g = 9.8, k1 = 0.001, k2 = 0.002, r1 = r2 = 1, r¯1 = 5, r¯2 = 6, σ1 = 0.1, σ2 = 0.12, σ¯ 1 = 0.01, σ¯ 1 = 0.01, c1 = 0.5, c2 = 0.01, v 1 = 0.01, v 2 = 0.05, k = 0.005, η = 0.1, b = 1, and n = 2. All the meaning of the design parameters can be founding [42], which will not be explained in this paper. The simulation results are shown in Figs. 16–18. Comparing the results in Figs. 16–18 with the results in Figs. 13–15, it is obvious that it takes more time to keep (45) stable using the algorithm in [42] than the algorithm proposed in this paper. This indicates that the algorithm proposed in this paper has a better transient performance than the algorithm proposed in [42].

1198

IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, VOL. 26, NO. 6, JUNE 2015

where ε > 0, the constants p > 1 and q > 1 satisfy ( p − 1)(q − 1) = 1, and (x, y) ∈ R 2 . Applying these inequalities leads to  4 n  T  1 1 T 2 n φi (y) 2b ζ˜ P ζ˜  (y)P ζ˜ ≤ 4 bP 2ε1 j =i g j i=1

Fig. 16.

Fig. 17.

Fig. 18.

4 3 + bP2 ε13 |ζ˜ |4 (A.1) 2  4 n  T  1 1 T 2 n 2b ζ˜ P ζ˜ F P ζ˜ ≤ 4 bP fi 2ε2 j =i g j i=1

Response of x1 and xˆ1 in [42].

Response of x2 and xˆ2 in [42].

Control input u in [42].

VI. C ONCLUSION In this paper, the output-feedback adaptive neural control has been investigated for a class of stochastic nonlinear systems with time-varying delays and unknown control directions. Through a linear state transformation, the unknown control coefficients are grouped together and the original system is transformed into a new system that makes the control design become feasible. The proposed adaptive neural controller contains only one adaptive parameter that is relevant to the given NNs such that the online learning time is dramatically decreased. Notably, only one NN is employed to compensate for all the unknown functions. The stability analysis guarantees that all the signals in the closed-loop system are bounded in probability. The simulation examples demonstrate the performance of the proposed approach. Future works will focus on the virtual control gain function and the function φ(x¯i ) rather than φ(y), which will make the design more challenging. A PPENDIX A In this appendix, we use Young’s inequality [43] xy ≤

1 εp p |x| + p |y|q p qε

4 3 + bP2 ε23 |ζ˜ |4 (A.2) 2 4   3 2b ζ˜ T P ζ˜ (K ζ1 )T P ζ˜ ≤ bP2 ε33 |ζ˜ |4 2 n 3 + 4 bP2 (ki ζ1 )4 (A.3) 2ε3 i=1 ( ' bT r (2P ζ˜ ζ˜ T P + ζ˜ T P ζ˜ P)H H T  4 n 1 1 n ≤ [2bP + bPλmax ( p)] hi ε4 j =i g j

i=1

+ ε4 [2bP + bPλmax ( p)]|ζ˜ |4 1 3 4 4 g¯ 3 z 1 ≤ δ11 |ζ˜ |4 gz ¯ 13 ζ˜1 ≤ δ11 |ζ˜2 |4 + 4 4δ11 3 4 4 + g¯ 3 z 1 4δ11 1 3 4 4 g¯ 3 z 1 gz ¯ 13 z 2 ≤ δ12 z 24 + 4 4δ12 3 4 1 z 13 f 1 ≤ z + λ11 y 4 (t − d(t)) 4λ11 1 4 × f¯14 (y, y(t − d(t)))

(A.4)

(A.5) (A.6)

(A.7)

3 3 4 3 2 z 1 h 1 h 1T ≤ λ12 y 4 (t − d(t))h¯ 41 (y, y(t − d(t))) + z 2 4 4λ12 1 (A.8) ) * 43 ∂αi−1 3 ˜ 1 3 4 ∂αi−1 gz ¯ i ζi ≤ δi1 |ζ˜i |4 + − g¯ 3 z i4 ∂y 4 4δi1 ∂y *4 ) 1 3 4 ∂αi−1 3 4 4 ˜ 3 ≤ δi1 |ζ | + g¯ z i (A.9) 4 4δi1 ∂y ) *4 1 3 4 ∂αi−1 3 4 ∂αi−1 3 ˜ 4 ˜ 3 gz ¯ i ζi ≤ δi1 |ζi | + g¯ zi − ∂y 4 4δi1 ∂y *4 ) 1 3 4 ∂αi−1 3 4 4 ˜ 3 ≤ δi1 |ζ | + g¯ z i (A.10) 4 4δi1 ∂y −

∂αi−1 3 ∂αi−1 3 gz ¯ i ζˆi = − gz ¯ i (z i + y α¯ i−1 ) ∂y ∂y *4 ) 1 3 4 ∂αi−1 3 4 4 ≤ δi2 z i + g¯ 3 zi 4 4δi2 ∂y ) *4 3 4 ∂αi−1 3 43 4 1 + g¯ 3 α¯ i−1 z i + δi3 y 4 4δi3 ∂y 4 (A.11)

LI et al.: OUTPUT-FEEDBACK ADAPTIVE NEURAL CONTROL FOR STOCHASTIC NONLINEAR TIME-VARYING DELAY SYSTEMS

) *4 ∂αi−1 3 3 ∂αi−1 3 4 z f i−1 ≤ − zi ∂y i 4λi1 ∂y 1 4 + λi1 y 4 (t − d(t)) f¯i−1 4 ) 2 *2 1 ∂ 2 αi−1 3 ∂ αi−1 1 T − z h h ≤ z i6 i−1 i−1 2 ∂ y2 i 4λi3 ∂ y2 1 + λi3 y 4 (t − d(t))h¯ 4i−1 4 ) ) *2 *4 3 2 ∂αi−1 3 ∂αi−1 T z h i−1 h i−1 ≤ z i4 2 i ∂y 4λi2 ∂y 3 + λi2 y 4 (t − d(t))h¯ 4i−1 . 4

⎡ i = y bP (A.12)

# (A.13)

(A.14)

∂α1 ∂α1 ˆ ∂α1 ˙ˆ φ1 (y) + 2 = k2 ζˆ1 + (ζ2 − k1 ζˆ1 ) + θ ˆ ∂y ∂ ζ1 ∂ θˆ  ) *4 1 3 1 ∂α1 3 ∂α1 ˙ˆ l − z 2 δ12 + δ22 + + 4 4 4λ21 ∂y ∂ lˆ *4 ) 2 *2 ) ∂ α1 ∂α1 1 3 2 + z2 + 4λ22 ∂ y 2 4λ23 ∂y ) *4 ) * 4 3 ∂α1 3 3 ˆ + +l 2 + α¯ 1 (A.15) 4λ24 ∂y ∂αi−1 ∂αi−1 φ1 (y) + i = ki ζˆ1 + (ζˆ j +1 − k j ζˆ1 ) ∂y ∂ ζˆ j i−1

j =1

∂αi−1 ˙ ∂αi−1 ˙ˆ ∂αi−1 ξ˙ θˆ + l+ ∂ξ ∂ θˆ ∂ lˆ  ) *4 3 1 3 3 1 ∂αi−1 3 −z i λi−1,4 + + δi2 + + 4 4λi4 4 4λi4 4λi1 ∂y *4 ) 2 *2 ) ∂ αi−1 ∂αi−1 1 3 + z i2 + 4λi3 ∂ y2 4λi2 ∂y *4 ) * ) 4 ∂αi−1 3 2 + α¯ i3 (A.16) + lˆ ∂y +

∂αn−1 ∂αn−1 φ1 (y) + (ζˆ j +1 − k j ζˆ1 ) ∂y ∂ ζˆ j n−1

j =1

∂αn−1 ˙ ∂αn−1 ˙ˆ ∂αn−1 ξ˙ θˆ + l+ ∂ξ ∂ θˆ ∂ lˆ  ) *4 1 3 1 ∂αn−1 3 −z n λn−1,4 + δn2 + 4 4 4λn1 ∂y ) 2 *2 ∂ αn−1 1 + z n2 4λn3 ∂ y2 ) ) * *4 *4 ) 4 3 ∂αn−1 3 ∂αn−1 + + lˆ 2 + α¯ n3 4λn2 ∂y ∂y (A.17) +

 4 n 1 1 n φ¯m (y) 2ε14 m=1 j =m g j  4 ⎤ n 1 1 ⎦ km n + 2ε34 j =1 g j

2⎣

4

Then, we can define

n = kn ζˆ1 +

1199

+

m=1

1 4

i i=1

4 3 δ j 1 + bP2 ε13 + ε4 (2bP 2

4 3 + bPλmax ( p)) + bP2 ε23 2 $ 4 3 + bP2 ε33 |ζ˜ |4 + y 4 (t − d(t)) 2 ⎡  4 i n 1 1 1 2 ¯ ⎣  f bP + λ j 1 f¯14 × m n 4 gj 2ε24 j =m m=1 j =1 ⎤ i i 3 1 + λ j 2 h¯ 41 + λ j 3 h¯ 41 ⎦ . (A.18) 4 4

j =1

j =2

A PPENDIX B P ROOF OF T HEOREM 4.1 1) Define W (t, x) = V (t, x)ect . Applying [Itoˆ s formula to the function W (t, x) yields d(V (t, x)ect ) = ect (V (t, x)+ LV )dt   n ∂ V ∂α ∂ V ∂ V i−1 h1 + + ect h1 − H dw ∂z 1 ∂z i ∂y ∂ ζ˜ i=2

≤ ect ([g¯ N(ξ ) + 1]ξ˙ + D)dt   n ∂V ∂ V ∂αi−1 ct ∂ V h1 + +e h1 − H dw. ∂z 1 ∂z i ∂y ∂ ζ˜ i=2

(B.1) Integrating (B.1) over (0, t), we obtain D V (t, x) ≤ e−ct V (0, x(0)) + c t −c(t −s) + e ([g¯ N(ξ ) + 1]ξ˙ )ds 0  t n ∂ V ∂αi−1 −c(t −s) ∂ V h1 + e h1 − ∂z 1 ∂z i ∂y 0 i=2 * ∂V + H dw(s). (B.2) ∂ ζ˜ The stochastic integral (second integral) in (B.2) is local martingale. Then, according to Lemma 1, the conclusion can be obtained that the functions V (t, x), ξ(t), and  t −c(t −s) ([ g¯ N(ξ ) + 1]ξ˙ )ds must be bounded in probability. e 0 2) Taking expectations on (B.2) and applying the Founds lemma, we have E V (t, x) ≤ E V (0, x(0)) +

D + σ¯ c

(B.3)

1200

IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, VOL. 26, NO. 6, JUNE 2015

where σ¯ = sup

t 0

([g¯ N(ξ ) + 1]ξ˙ )ds. Then

1 4 D z i ≤ E V (t, x) ≤ E V (0, x(0)) + E + σ¯ . 4 c 4

i=1

Denote  = E V (0, x(0)) +

D c

+ σ¯ , then E

4 + i=1

z i4 ≤ 4.

Therefore, there exists a compact set z , such that z i ∈ z . Thus, the proof is completed. R EFERENCES [1] P. Florchinger, “Lyapunov-like techniques for stochastic stability,” SIAM J. Control Optim., vol. 33, no. 4, pp. 1151–1169, 1995. [2] H. Deng and M. Krsti´c, “Output-feedback stabilization of stochastic nonlinear systems driven by noise of unknown covariance,” Syst. Control Lett., vol. 39, no. 3, pp. 173–182, 2000. [3] H. Deng, M. Krsti´c, and R. J. Williams, “Stabilization of stochastic nonlinear systems driven by noise of unknown covariance,” IEEE Trans. Autom. Control, vol. 46, no. 8, pp. 1237–1253, Aug. 2001. [4] M. Krstic, I. Kanellakopoulos, and P. V. Kokotovic, Nonlinear and Adaptive Control Design. New York, NY, USA: Wiley, 1995. [5] Z. Pan and T. Basar, “Backstepping controller design for nonlinear stochastic systems under a risk-sensitive cost criterion,” SIAM J. Control Optim., vol. 37, no. 3, pp. 957–995, 1999. [6] S. S. Ge and C. Wang, “Direct adaptive NN control of a class of nonlinear systems,” IEEE Trans. Neural Netw., vol. 13, no. 1, pp. 214–221, Jan. 2002. [7] W. S. Chen, L. C. Jiao, and Z. B. Du, “Brief paper: Output-feedback adaptive dynamic surface control of stochastic non-linear systems using neural network,” IET Control Theory Appl., vol. 4, no. 12, pp. 3012–3021, Dec. 2010. [8] S. Tong, Y. Li, Y. Li, and Y. Liu, “Observer-based adaptive fuzzy backstepping control for a class of stochastic nonlinear strict-feedback systems,” IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 41, no. 6, pp. 1693–1704, Dec. 2011. [9] J. Li, W. Chen, J.-M. Li, and Y. Fang, “Adaptive NN output-feedback stabilization for a class of stochastic nonlinear strict-feedback systems,” ISA Trans., vol. 48, no. 4, pp. 468–475, 2009. [10] W. Chen, L. C. Jiao, and J. Wu, “Decentralized backstepping output-feedback control for stochastic interconnected systems with time-varying delays using neural networks,” Neural Comput. Appl., vol. 21, no. 6, pp. 1375–1390, Sep. 2011. [11] S. Tong, T. Wang, Y. Li, and B. Chen, “A combined backstepping and stochastic small-gain approach to robust adaptive fuzzy output feedback control,” IEEE Trans. Fuzzy Syst., vol. 21, no. 2, pp. 314–327, Apr. 2013. [12] J. Li, W. Chen, and J.-M. Li, “Adaptive NN output-feedback decentralized stabilization for a class of large-scale stochastic nonlinear strict-feedback systems,” Int. J. Robust Nonlinear Control, vol. 21, no. 3, pp. 452–472, Mar. 2011. [13] Y. Li, Y. Li, and S.-C. Tong, “Adaptive fuzzy decentralized output feedback control for stochastic nonlinear large-scale systems,” Neurocomputing, vol. 83, pp. 38–46, Apr. 2012. [14] C. L. P. Chen, Y.-J. Liu, and G.-X. Wen, “Fuzzy neural network-based adaptive control for a class of uncertain nonlinear stochastic systems,” IEEE Trans. Cybern., vol. 44, no. 5, pp. 583–593, May 2014. [15] A. Papadimitropoulos, G. A. Rovithakis, and T. Parisini, “Fault detection in mechanical systems with friction phenomena: An online neural approximation approach,” IEEE Trans. Neural Netw., vol. 18, no. 4, pp. 1067–1082, Jul. 2007. [16] K. Fischle and D. Schroder, “An improved stable adaptive fuzzy control method,” IEEE Trans. Fuzzy Syst., vol. 7, no. 1, pp. 27–40, Feb. 1999. [17] Y. Yang and J. Ren, “Adaptive fuzzy robust tracking controller design via small gain approach and its application,” IEEE Trans. Fuzzy Syst., vol. 11, no. 6, pp. 783–795, Dec. 2003. [18] T.-S. Li, D. Wang, G. Feng, and S.-C. Tong, “A DSC approach to robust adaptive NN tracking control for strict-feedback nonlinear systems,” IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 40, no. 3, pp. 915–927, Jun. 2010. [19] Y.-J. Liu, S.-C. Tong, D. Wang, T.-S. Li, and C. L. P. Chen, “Adaptive neural output feedback controller design with reduced-order observer for a class of uncertain nonlinear SISO systems,” IEEE Trans. Neural Netw., vol. 22, no. 8, pp. 1328–1334, Aug. 2011.

[20] Z. Yu, S. Li, and H. Du, “Razumikhin–Nussbaum-lemma-based adaptive neural control for uncertain stochastic pure-feedback nonlinear systems with time-varying delays,” Int. J. Robust Nonlinear Control, vol. 23, no. 11, pp. 1214–1239, 2012. [21] R. D. Nussbaum, “Some remarks on a conjecture in parameter adaptive control,” Syst. Control Lett., vol. 3, no. 5, pp. 243–246, Nov. 1983. [22] Y.-G. Liu, “Output-feedback adaptive control for a class of nonlinear systems with unknown control directions,” Acta Autom. Sinica, vol. 33, no. 12, pp. 1306–1312, 2007. [23] Y. Wen and X. Ren, “Neural networks-based adaptive control for nonlinear time-varying delays systems with unknown control direction,” IEEE Trans. Neural Netw., vol. 22, no. 10, pp. 1599–1612, Oct. 2011. [24] W. Ying-Chun, Z. Hua-Guang, and W. Yi-Zhong, “Fuzzy adaptive control of stochastic nonlinear systems with unknown virtual control gain function,” Acta Autom. Sinica, vol. 32, no. 2, pp. 170–178, Mar. 2006. [25] Z. Yu, Z. Jin, and H. Du, “Adaptive neural control for a class of non-affine stochastic non-linear systems with time-varying delay: A Razumikhin–Nussbaum method,” IET Control theory Appl., vol. 6, no. 1, pp. 14–23, Jan. 2012. [26] T. Wang, S. Tong, and Y. Li, “Robust adaptive fuzzy output feedback control for stochastic nonlinear systems with unknown control direction,” Neurocomputing, vol. 106, pp. 31–41, Apr. 2013. [27] Y. Fu, Z. Tian, and S. Shi, “Output feedback stabilization for a class of stochastic time-delay nonlinear systems,” IEEE Trans. Autom. Control, vol. 50, no. 6, pp. 847–851, Jun. 2005. [28] H. Zhang, Y. Wang, and D. Liu, “Delay-dependent guaranteed cost control for uncertain stochastic fuzzy systems with multiple time delays,” IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 38, no. 1, pp. 126–140, Feb. 2008. [29] J. Fu, H. Zhang, T. Ma, and Q. Zhang, “On passivity analysis for stochastic neural networks with interval time-varying delay,” Neurocomputing, vol. 73, nos. 4–6, pp. 795–801, 2010. [30] Z. Li, X. Cao, and N. Ding, “Adaptive fuzzy control for synchronization of nonlinear teleoperators with stochastic time-varying communication delays,” IEEE Trans. Fuzzy Syst., vol. 19, no. 4, pp. 745–757, Apr. 2011. [31] L. Wu and W. X. Zheng, “L 2 –L ∞ control of nonlinear fuzzy ItÔ stochastic delay systems via dynamic output feedback,” IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 39, no. 5, pp. 1308–1315, Oct. 2009. [32] Y. Xia, M. Fu, P. Shi, Z. Wu, and J. Zhang, “Adaptive backstepping controller design for stochastic jump systems,” IEEE Trans. Autom. Control, vol. 54, no. 12, pp. 2853–2859, Dec. 2009. [33] S. S. Ge, F. Hong, and T. H. Lee, “Adaptive neural network control of nonlinear systems with unknown time delays,” IEEE Trans. Autom. Control, vol. 48, no. 11, pp. 2004–2010, Nov. 2003. [34] F. Hong, S. S. Ge, and T. H. Lee, “Practical adaptive neural control of nonlinear systems with unknown time delays,” IEEE Trans. Syst., Man Cybern. B, Cybern., vol. 35, no. 4, pp. 849–854, Aug. 2005. [35] C. Hua, X. Guan, and P. Shi, “Robust backstepping control for a class of time delayed systems,” IEEE Trans. Autom. Control, vol. 50, no. 6, pp. 894–899, Jun. 2005. [36] S.-C. Tong, Y. M. Li, and H.-G. Zhang, “Adaptive neural network decentralized backstepping output-feedback control for nonlinear large-scale systems with time delays,” IEEE Trans. Neural Netw., vol. 22, no. 7, pp. 1073–1086, Jul. 2011. [37] Z. Liu, G. Lai, Y. Zhang, X. Chen, and C. L. P. Chen, “Adaptive neural control for a class of nonlinear time-varying delay systems with unknown hysteresis,” IEEE Trans. Neural Netw. Learn. Syst., 2014, doi: 10.1109/TNNLS.2014.2305717. [38] C. L. P. Chen, G. Wen, Y. Liu, and F. Wang, “Adaptive consensus control for a class of nonlinear multiagent time-delay systems using neural networks,” IEEE Trans. Neural Netw. Learn. Syst., vol. 25, no. 6, pp. 1217–1226, Jun. 2014. [39] W. Chen, L. Jiao, J. Li, and R. Li, “Adaptive NN backstepping output-feedback control for stochastic nonlinear strict-feedback systems with time-varying delays,” IEEE Trans. Syst. Man Cybern. B, Cybern., vol. 40, no. 3, pp. 939–950, Jun. 2010. [40] S.-J. Liu, S. S. Ge, and J.-F. Zhang, “Adaptive output-feedback control for a class of uncertain stochastic non-linear systems with time delays,” Int. J. Control, vol. 81, no. 8, pp. 1210–1220, 2008. [41] W. Chen, J. Wu, and L. C. Jiao, “State-feedback stabilization for a class of stochastic time-delay nonlinear systems,” Int. J. Robust Nonlinear Control, vol. 22, no. 17, pp. 1921–1937, Nov. 2012.

LI et al.: OUTPUT-FEEDBACK ADAPTIVE NEURAL CONTROL FOR STOCHASTIC NONLINEAR TIME-VARYING DELAY SYSTEMS

[42] Y. Li, S. Tong, and Y. Li, “Observer-based adaptive fuzzy backstepping control for strict-feedback stochastic nonlinear systems with time delays,” Int. J. Innovative Comput., Inform., Control, vol. 8, no. 12, pp. 8103–8114, Dec. 2012. [43] G. Hardy, J. E. Littlewood, and G. Polya, Inequalities, 2nd ed. Cambridge, U.K.: Cambridge Univ. Press, 1989.

Tieshan Li (M’08–SM’12) received the B.S. degree from the Ocean University of China, Qingdao, China, in 1992, and the Ph.D. degree in transportation from Dalian Maritime University (DMU), Dalian, China, in 2005. He was a Post-Doctoral Scholar with Shanghai Jiao Tong University, Shanghai, China, from 2007 to 2010. He also visited the City University of Hong Kong, Hong Kong, from 2008 to 2009, as a Senior Research Associate, and the University of Macau, Macau, China, in 2013, as a Visiting Scholar. He is currently a Professor at DMU. His current research interests include decentralized adaptive control, fuzzy control and neural-network control for nonlinear systems, and their applications to marine control.

Zifu Li received the B.S. degree in marine technology and the M.S. degree in transportation information engineering and control from Dalian Maritime University (DMU), Dalian, China, in 1998 and 2004, respectively, where he is currently pursuing the Ph.D. degree in transportation information engineering and control. His current research interests include adaptive control, robust control, and neural-network control for stochastic nonlinear systems.

1201

Dan Wang was born in Dalian, China. He received the B.Eng. degree in industrial automation engineering from the Dalian University of Technology, Dalian, in 1982, the M.Eng. degree in marine automation engineering from Dalian Maritime University, Dalian, in 1987, and the Ph.D. degree in mechanical and automation engineering from the Chinese University of Hong Kong, Hong Kong, in 2001. He was with Dalian Maritime University as a Lecturer from 1987 to 1998, and an Associate Professor in 1992. From 2001 to 2005, he was a Research Scientist with Temasek Laboratories, National University of Singapore, Singapore. Since 2006, he has been with Dalian Maritime University, where he is a Professor with the Department of Marine Electrical Engineering, Marine Engineering College. His current research interests include nonlinear control theory and applications, neural networks, adaptive control, robust control, fault detection and isolation, and system identification.

C. L. Philip Chen (S’88–M’88–SM’94–F’07) received the M.S. degree in electrical engineering from the University of Michigan, Ann Arbor, MI, USA, in 1985, and the Ph.D. degree in electrical engineering from Purdue University, West Lafayette, IN, USA, in 1988. He is currently the Chair Professor with the Department of Computer and Information Science and the Dean of the Faculty of Science and Technology at the University of Macau, Macau, China. Dr. Chen is a fellow of the American Association for the Advancement of Science. He is currently the President of the IEEE Systems, Man, and Cybernetics Society. In addition, he is an Accreditation Board of Engineering and Technology Education Program Evaluator for Computer Engineering, Electrical Engineering, and Software Engineering programs.