1660
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, VOL. 24, NO. 10, OCTOBER 2013
Tracking Algorithms for Multiagent Systems Deyuan Meng, Member, IEEE, Yingmin Jia, Member, IEEE, Junping Du, and Fashan Yu Abstract— This paper is devoted to the consensus tracking issue on multiagent systems. Instead of enabling the networked agents to reach an agreement asymptotically as the time tends to infinity, the consensus tracking between agents is considered to be derived on a finite time interval as accurately as possible. We thus propose a learning algorithm with a gain operator to be determined. If the gain operator is designed in the form of a polynomial expression, a necessary and sufficient condition is obtained for the networked agents to accomplish the consensus tracking objective, regardless of the relative degree of the system model of agents. Moreover, the H∞ analysis approach is introduced to help establish conditions in terms of linear matrix inequalities (LMIs) such that the resulting processes of the presented learning algorithm can be guaranteed to monotonically converge in an iterative manner. The established LMI conditions can also enable the iterative learning processes to converge with an exponentially fast speed. In addition, we extend the learning algorithm to address the relative formation problem for multiagent systems. Numerical simulations are performed to demonstrate the effectiveness of learning algorithms in achieving both consensus tracking and relative formation objectives for the networked agents. Index Terms— Consensus tracking, H∞ analysis approach, learning algorithms, multiagent systems, relative formation.
I. I NTRODUCTION
A
MULTIAGENT system generally consists of a group of agents, which are required to carry out cooperative tasks for the group. The coordination control for multiagent systems has attracted considerable research interest owing to its wide applications in many practical areas. As shown in [1]–[4], the consensus issue plays a fundamental role in the multiagent coordination control, which can also be extended to include as special cases other coordination control problems, such as relative formation and reference tracking. In the literature, there have been proposed promising classes of distributed algorithms to guarantee the consensus objectives; see [5]–[7] for stochastic algorithms, [8]–[11] for pinning algorithms, and [12]–[17] for leader-following algorithms. Such consensus techniques aim to asymptotically
Manuscript received August 14, 2012; revised March 14, 2013; accepted May 5, 2013. Date of publication July 4, 2013; date of current version September 27, 2013. This work was supported in part by the National 973 Program under Grant 2012CB821200 and Grant 2012CB821201, the NSFC under Grant 61104011, Grant 61134005, and Grant 61203044, the MOE under Grant 2011110212003, and the Beijing Natural Science Foundation under Grant 4122046. D. Meng and Y. Jia are with the Seventh Research Division and the Department of Systems and Control, Beihang University, Beijing 100191, China (e-mail: [email protected]; [email protected]). J. Du is with the Beijing Key Laboratory of Intelligent Telecommunications Software and Multimedia, School of Computer Science and Technology, Beijing University of Posts and Telecommunications, Beijing 100876, China (e-mail: [email protected]). F. Yu is with the School of Electrical Engineering and Automation, Henan Polytechnic University, Jiaozuo 454000, China (e-mail: [email protected]). Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/TNNLS.2013.2262234
vi1 vi2
tend to be the same as t
O
t
vi3 vin
vi1 vi2 O vi3 vin
t
t r
r N
t t N
Fig. 1. Two classes of consensus tracking tasks of multiagent systems. Upper: consensus tracking achieved asymptotically as t → ∞. Lower: consensus tracking achieved perfectly over a finite time interval t ∈ [r,N]. Here, t is the time variable, r and N are two constants satisfying 0 ≤ r ≤ N < ∞, and vi j ∈ {v1 , v2 , v3 , ..., vn } for j = 1, 2, ..., n, where {v1 , v2 , v3 , ..., vn } denotes a group of n agents.
achieve agreement in the stationary case (i.e., as time goes to infinity) regarding a certain quantity of interest that depends on the states/outputs of agents [3]. As a demonstration, Fig. 1 (upper) gives a sketch of this class of consensus tracking tasks for multiagent systems. In contrast, Fig. 1 (lower) describes another class of consensus tracking tasks, which are required to be achieved perfectly over a finite time interval. This shows essentially a class of “desired” consensus tracking tasks. A good alternative way to practically accomplish such a desirable cooperative objective is to develop consensus tracking tasks with arbitrary high precision (in other words, to make consensus tracking tasks on the specified finite time interval be derived with a tolerance that can be arbitrarily prescribed). Since arbitrary high precision consensus tracking is one of the most desirable coordination control objectives in applications, this creates a new challenging consensus problem that is rather different from those typical ones considered in [1]–[17]. It has been shown in [18]–[20] that the iterative schemes can be applied to develop a type of learning algorithm such that the control inputs for agents can be learned using the disagreements between agents to obtain the consensus tracking tasks with arbitrary high precision. This idea has been applied also to the issues about relative formation between agents [20]–[23]. For practical instances requiring arbitrary high precision, see [24] for trajectory-keeping in satellite flying and [25] for stereo vision-based formation of mobile robots.
2162-237X © 2013 IEEE
MENG et al.: TRACKING ALGORITHMS FOR MULTIAGENT SYSTEMS
The main motivation behind the learning algorithms of [18]–[25] is to take full advantage of the salient property of iterative learning control (ILC) approaches, which are able to determine control input signals to accomplish the arbitrary high precision tracking through an iterative learning process. However, the learning algorithms of the multiagent systems are constructed by using the disagreements between agents to improve the control input signals for all the agents from one iteration to the next, which are different from typical ILC that works based on the tracking error generated by the prior knowledge of the desired reference trajectory for the controlled systems. Until now, there have been only limited results on studying the learning algorithms for arbitrary high precision consensus tracking tasks, and the corresponding theory is far from complete. In particular, it is worth noting that only the asymptotic convergence of the learning algorithms for multiagent systems has been considered in the existing results [18]–[25]; however, the issue for their monotonic convergence (or nonovershoot) has not been addressed, which is practically important in all kinds of learning algorithms, as demonstrated in the ILC surveys [26]–[28] and references therein. Motivated by the aforementioned observations, we deal with the consensus tracking tasks shown in Fig. 1 (lower) for multiagent systems in directed networks. By instead considering the arbitrary high precision consensus tracking tasks, we apply the iterative schemes to construct a class of learning algorithms. In contrast to the existing results [18]–[20], we design the learning algorithms without any prescribed reference as a priori and aim to achieve their monotonic convergence through the H∞ -based analysis approach to ILC [29]–[31]. The plant model for every agent is considered to possess not only multiple-input multiple-output (MIMO) dynamics but also a common relative degree, which can be higher order. This renders our considered consensus problem more challenging, because the analysis and design of our learning algorithms should take both the generic relative degree plant of agents and the network topology of the multiagent systems into account. Thus, it is required to develop new approaches such that the agents can be enabled to achieve the consensus tracking with each other. In this paper, we use the relative output knowledge between agents to construct a learning algorithm, which is in an update form and has a gain operator to be designed. If this operator is taken in terms of a polynomial expression, the agents can be enabled to achieve the consensus tracking with each other as soon as the input signals can take effect in their corresponding outputs, and a necessary and sufficient condition is established simultaneously to guarantee the asymptotic convergence of the multiagent learning system. It is verified that this convergence condition needs only the basic structure information of agents, under which an alternative method can be derived to induce the gain operator. Furthermore, the monotonic convergence for our learning algorithm is investigated, for which the H∞ analysis approach is applied to give convergence conditions in terms of linear matrix inequalities (LMIs), as well as directly providing formulas for the gain operator. It is also demonstrated that the LMI conditions can guarantee the multiagent learning systems
1661
to converge exponentially fast. In view of this observation, the exponentially fast convergence is further considered, and more relaxed conditions for our proposed learning algorithm can be derived. It is worthwhile to mention that the proposed learning algorithm is also extended to accomplish the relative formation between agents. In addition, some numerical simulation results are presented to demonstrate the effectiveness of our learning algorithms, which consider a six-agent system with the relative degree of two under directed graphs. This paper is organized as follows. In Section II, preliminaries in the graph theory are introduced. The problem statement is proposed for the consensus tracking of multiagent systems in Section III. The asymptotic convergence results are given in Section IV, and the monotonic and/or exponential convergence results are established in Section V. The consensus results are made an extension in Section VI to the relative formation issue for agents. Simulation results, and then conclusions, are posed in Sections VII and VIII, respectively. Notations: In = {1, 2, . . ., n}, ZN = {0, 1, . . . , N}, 1n = [1, 1, . . . , 1]T ∈ Rn , I and 0 denote the identity and null matrices with required dimensions, respectively, diag{·} denotes a block diagonal matrix with zero off-diagonal elements, and a bigstar () used in symmetric block matrices denotes a term induced by the symmetry. For matrices or vectors A and B, ρ (A) is the spectral radius of a matrix A, A2 is the spectral norm of a matrix A, A is the Euclidean norm of a vector A, A ≥ 0 if its elements are all nonnegative, A ≺ 0 (respectively, A 0) is a negative-definite (respectively, positive-definite) matrix, A ⊗ B is Kronecker product of A and B, and A ∈ Rn×n is a stochastic matrix1 where A ≥ 0 and A1n = 1n . For any discrete-time domain vector function z(t), q is a shift operator, i.e., qz(t) = z(t + 1) and q−1 z(t) = z(t − 1), and z(t)2 is the L2 -norm of z(t). II. P RELIMINARIES In this section, some preliminaries in graph theory are given. Let G denote an nth order directed graph consisting of a vertex set V and an edge set E. Here, we adopt vi to denote the vertex of G , and hence we have V = {vi : i ∈ In }. The corresponding pair (vi , v j ) is used to denote the edge of G . If (vi , v j ) ∈ E, then it means that there exists a communication channel between vi and v j , where the information flows from v j to vi . This implies also that v j is a neighbor of vi . Hence, Nvi = {v j : (vi , v j ) ∈ E} is defined as the neighbor set of the vertex vi . Let the index set of neighbors of the vertex vi be given as Ni = { j : (vi , v j ) ∈ E}. In addition, it can be easily seen that E ⊆ {(vi , v j ) : vi , v j ∈ V}. Example 1: Let us consider the directed graph shown in Fig. 2. If we take the vertex v8 for instance, there exists information that flows from the vertex v7 to it. Hence, we know that (v8 , v7 ) is an edge, and v7 is a neighbor of v8 . Moreover, we can derive Nv8 = {v5 , v6 , v7 } and N8 = {5, 6, 7}. 1 If A ≥ 0 and A1 = 1 , it can be seen that every row of A may be considered n n as a discrete probability distribution over a sample space with n points. Hence, n×n satisfying A ≥ 0 and A1n = 1n is said to be a stochastic matrix. here A ∈ R For more details, see also [32].
1662
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, VOL. 24, NO. 10, OCTOBER 2013
v6
v1
v8 v8
v2
v5
v 8 v7
v3 v7
v4
Fig. 2. Small example of a directed graph with eight vertices and ten edges. Fig. 3.
Remark 1: It can be seen that there are only communication channels between each vertex and its neighbor vertices (rather than all vertices). This implies that the information of neighbor vertices of a vertex is available to its controller design, whereas the information of other non-neighbor vertices is not available. Such a way to the controller design is the nearest neighbor rule considered in the literature [1]–[3]. Take for instance, the vertex v8 of the directed graph shown in Fig. 2. According to the nearest neighbor rule, the controller design can only use the information of its neighbor vertices v5 , v6 , and v7 . In addition, we also introduce the following definitions with respect to the directed graphs. A path in G is a finite sequence vi1 , vi2 , . . . , vi j of vertices such that (vil , vil+1 ) ∈ E holds for l = 1, 2, . . . , j − 1. If there is a special vertex that can be connected to all the other vertices through paths, then G is said to have a spanning tree, and this special vertex is called the root vertex. A nonnegative weighted adjacency matrix A = [ai j ] associated with G is defined to model the information exchange between agents, where ai j > 0 ⇔ (vi , v j ) ∈ E and ai j = 0 otherwise. Here, the weighted directed graph is denoted by G (A ), and aii = 0 is assumed for i ∈ In . The Laplacian matrix of G (A ) is defined as LA = Δ − A , where Δ = diag{Δ11 , Δ22 , . . . , Δnn } and Δii = ∑nj=1, j =i ai j for all i ∈ In . Example 2: We again take for instance the directed graph of Fig. 2. It can be seen that the sequence of vertices given by v6 , v8 , v5 , v3 , v2 , v7 , and v4 is a path in this directed graph, but there are no paths such that these seven vertices can be connected to the vertex v1 . Hence, there are no spanning trees in the directed graph of Fig. 2. If we remove the vertex v1 from this directed graph, the remaining part has spanning trees (particularly, v4 is connected to v2 , v3 , v5 , v6 , v7 , and v8 through paths). Moreover, a weighted graph for the directed graph of Fig. 2 is shown in Fig. 3. It can be easily seen from this figure that the adjacency matrix A is given by ⎡ ⎤ 0 0 0 0 0 0 0 0 ⎢0 0 0 0 0 0 a27 0 ⎥ ⎢ ⎥ ⎢0 a32 0 0 0 0 0 0 ⎥ ⎢ ⎥ ⎢0 0 0 0 0 0 a47 0 ⎥ ⎢ ⎥ A =⎢ ⎥ ⎢0 0 a53 0 0 0 0 0 ⎥ ⎢0 0 0 0 0 0 0 a68 ⎥ ⎢ ⎥ ⎣0 0 0 a74 0 a76 0 0 ⎦ 0 0 0 0 a85 a86 a87 0 where the nonzero elements ai j are positive weights associated with the edges of the directed graph shown in Fig. 3.
Weighted directed graph corresponding to the one shown in Fig. 2.
III. P ROBLEM S TATEMENT In this paper, we consider networked systems with n agents, which are labeled 1 through n. For all i ∈ In , the ith agent is regarded as the vertex vi in the weighted directed graph G (A ), and is assumed to have the following MIMO dynamics: xi,k (t + 1) = Axi,k (t) + Bui,k (t) (1) yi,k (t) = Cxi,k (t), xi,k (0) = xi0 where t ∈ ZN and k ∈ Z+ denote the time and iteration indices, respectively, xi,k (t) ∈ Rnx , ui,k (t) ∈ Rnu and yi,k (t) ∈ Rny denote the state, protocol (or input) and output, respectively, and A, B, and C denote any matrices of appropriate dimensions. Without loss of generality, CAi B = 0 (i < r − 1) and CAr−1 B of full-row rank are assumed to meet a basic requirement of the following learning algorithms on the system structure of the agents [that is, the relative degree of (1) is r, where r ≥ 1]. It can be easily verified that yi,k (t) = gi (t) holds for all t = 0, . . ., r − 1, where gi (t) = CAt xi0 . For any t ≥ r, the input–output relationship of (1) is given by yi,k (t) = P(q)ui,k (t) + gi(t)
(2)
P(q) = C(qI − A)−1B.
where As shown above, the output of every agent described by (1) is only dependent on the initial state over the first r time steps, which cannot be controlled by the protocol. This in fact arises from the effects of r since the relative degree for discrete-time systems means exactly the steps of time delay in the output, in order to have the control input appearing. By this observation, we address the output consensus tracking problem over t ∈ {r, r + 1, . . ., N} for the multiagent system (1). Let us consider the disagreement between the agents vi and v j as their relative output yi,k (t) − y j,k (t) for ∀i, j ∈ In , and if
lim yi,k (t) − y j,k (t) = 0, t = r, r + 1, . . . , N (3) k→∞
then the agents vi and v j are said to accomplish the consensus tracking with each other. It can be seen that the disagreement yi,k (t)−y j,k (t) can be guaranteed to be arbitrarily small (that is, yi,k (t) − y j,k (t)2 < ε for any arbitrarily prescribed tolerance ε > 0) after a certain number of iterations if the objective (3) is derived. This means that (3) shows the arbitrary high precision consensus tracking tasks, which is different from the consensus problems considered in [1]–[17] and also can include as special cases the consensus problems studied in [18]–[20].
MENG et al.: TRACKING ALGORITHMS FOR MULTIAGENT SYSTEMS
1663
Remark 2: In practice, the multiagent system (1) or (2) can be considered as the linearized model of practical plants, such as flying satellites [24] and mobile robots [25]. The consensus tracking problem (3) can be extended to the relative formation problem considered in [24] and [25] (see the following discussions of Section VI for details). In particular, the individual plant of such applications usually suffers the relative degree problem, which is taken into account by (1). A. Problem Representation It can be easily seen that (3) holds if there exists a common (unknown) trajectory c(t) ∈ Rny on ZN−r satisfying lim yk (t + r) = 1n ⊗ c(t), t ∈ ZN−r
k→∞
where yk (t) = [yT1,k (t), yT2,k (t), . . . , yTn,k (t)]T . By noting this fact, let us denote zk (t) = (D ⊗ I)yk (t + r), t ∈ ZN−r
(4)
where D ∈ R(n−1)×n represents a full-row matrix with the null space spanned by 1n . For the sake of computational simplicity, here we adopt
(5) D = −1n−1 I . 0 E = ∈ Rn×(n−1) I
If we denote
then we can obtain with (4) that
F yk (t + r) = 1n E ⊗ I yk (t + r) D = (1n F ⊗ I)yk (t + r) + (E ⊗ I)zk (t)
k→∞
= yi,k (t + r)
+ ∑ ai j P(q)K(q) y j,k (t+r)−yi,k (t+r) j∈Ni
(6)
which can be formulated in a compact form of yk+1 (t + r) = [I − LA ⊗ P(q)K(q)]yk (t + r). This, together with (4) and (7), further leads to
(7)
where F = [1, 0, . . . , 0] ∈ R1×n . Furthermore, we can develop (4) to reveal the equivalent relationship between zk (t) and the disagreement error for agents, i.e., (3) holds if and only if2 lim zk (t) = 0, t ∈ ZN−r
where ui,0 (t) is a bounded initial input, which can be arbitrarily prescribed, K(q) is an nu × ny gain operator to be designed, and ai j is the weight of edge (vi , v j ) in the directed graph G (A ). Remark 3: Although the iterative schemes have been introduced to design coordination control algorithms of multiagent systems [18]–[25], (9) is presented as a new general consensus tracking algorithm that can take the relative degree of multiagent systems into account and does not require any references as the prior knowledge. In particular, if we set yn,k (t) ≡ yd (t) (i.e., assume that vn is a leader agent without loss of generality to specify the reference trajectory yd (t) for agents), (9) gives a learning algorithm to derive the consensus tracking with yd (t). This implies that (9) can include as special cases the consensus tracking algorithms in [18]–[20], which can be further extended to include as special cases the learning algorithms for relative formation issues of multiagent systems presented in [20]–[25] (for more details, see the following extensions in Section VI). By combining (9) with (2), it can be derived that
yi,k+1 (t + r) = yi,k (t+r)+P(q) ui,k+1 (t+r)−ui,k (t+r)
(8)
which is a stability problem. Thus, we can achieve the consensus tracking objective (3) through solving the stability problem (8) in the following discussions. B. Algorithm Design The consensus tracking algorithm considered in this paper is a type of learning algorithm that is designed using the iterative schemes. For each agent vi , i ∈ In , the next learning algorithm is constructed based on the relative output information between it and its neighbors
ui,k+1 (t) = ui,k (t)+ ∑ ai j K(q) y j,k (t)−yi,k (t) , t ∈ ZN−r j∈Ni
(9) 2 If (3) holds, we can easily obtain (8) based on (4) and (5). On the contrary,
if (8) holds, we can deduce from (4) and (5) that limk→∞ yl,k (t) − y1,k (t) = 0, where t = r, r +1, ..., N and l ∈ In . This leads to that, have
for ∀i, j ∈ In , we limk→∞ yi,k (t) − y j,k (t) = limk→∞ yi,k (t) − y1,k (t) − y j,k (t) − y1,k (t) = limk→∞ yi,k (t) − y1,k (t) − limk→∞ y j,k (t) − y1,k (t) = 0, where t = r, r + 1, ..., N. That is, (3) can be derived based on the satisfaction of (8).
zk+1 (t) = (D ⊗ I)yk+1 (t + r) = (D ⊗ I) [I − LA ⊗ P(q)K(q)]yk (t + r) = (D ⊗ I) [I − LA ⊗ P(q)K(q)](1n F ⊗ I)yk (t + r) +(D ⊗ I) [I − LA ⊗ P(q)K(q)](E ⊗ I)zk (t) = [I − DLA E ⊗ P(q)K(q)]zk (t) for t ∈ ZN−r (10) where the facts D1n = 0, LA 1n = 0, and DE = I are inserted. It can be obtained from (10) that the stability of zk (t) as k → ∞ depends on the mapping operator I − DLA E ⊗ P(q)K(q). However, it is worth noticing that this stability problem is different from that of classical ILC, in which not only the plant for agents but also the network topology of multiagent system plays a significant role (see also the discussion in [20]). In spite of the challenging issue, we attempt to establish some consensus tracking results based on the development of (10) by making (3) be approached both asymptotically and monotonically (or, exponentially fast). IV. A SYMPTOTIC C ONVERGENCE The following lemma is helpful in achieving the asymptotic convergence for consensus learning systems, which is adopted directly from the literature [20]. Lemma 1: For zk (t), let appropriately dimensioned matrices Az , Bz , Cz , and Dz make zk+1 (t) = [Cz (qI − Az )−1 Bz + Dz ]zk (t) hold for zk (t) over time steps t ∈ ZN−r . Then (8) can be derived if and only if ρ (Dz ) < 1.
1664
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, VOL. 24, NO. 10, OCTOBER 2013
Now, with Lemma 1, the following theorem can be proposed by considering the consensus learning algorithm (9) via a gain operator in the polynomial form. Theorem 1: Consider the multiagent system (1) associated with the directed graph G (A ). Let the algorithm (9) be applied with its gain operator K(q) in the following polynomial form: K(q) = K0 + K1 q + · · · + Kr qr
(11)
where Kl , ∀l ∈ Zr is an nu ×ny gain matrix. Then the consensus tracking objective (3) can be achieved if and only if the gain matrix Kr can be designed to satisfy (12) ρ I − DLA E ⊗ CAr−1 BKr < 1. Proof: By incorporating P(q) = C(qI − A)−1 B and K(q) = K0 + K1 q + · · · + Kr qr , we have
P(q)K(q) =
∞
∑ CAs−1Bq−s
s=r
= CA
r−1
r
∑ Kl ql
l=0
∞
BKr + ∑ CA s=1
s−1
r
∑ A BKl l
q−i
l=0 r −1
= CAr−1 BKr + C(qI − A)
∑ Al BKl
(13)
l=0
where the fact that CAs B = 0, s ∈ Zr−2 holds due to the relative degree r is inserted. Using the property of Kronecker product, we can get from (13) immediately that I − DLA E ⊗ P(q)K(q) = I − DLA E ⊗ CAr−1 BKr
r
− DLA E ⊗ C(qI − A)−1 ∑ Al BKl l=0
= I − DLA E ⊗ CAr−1 BKr −1
+ (I ⊗ C)(qI − I ⊗ A)
r
−DLA E ⊗ ∑ A BKl . (14) l
l=0
By combining (10) and (14), we can obtain from Lemma 1 that (12) provides a necessary and sufficient condition to guarantee (8). Then the equivalence between (3) and (8) further gives that (3) can be established if and only if (12) can be satisfied. This completes the proof. Remark 4: It can be clearly seen from Theorem 1 that there exist learning algorithms in terms of (9) and (11) to accomplish the consensus tracking objective (3) for the multiagent system (1) associated with G (A ). In practice, it means that the control input signals can be determined through the presented learning algorithms such that the consensus tracking tasks for agents are achieved with arbitrary high precision. This development can be further guaranteed with a necessary and sufficient condition in parallel with the convergence results for classic ILC, which is only dependent on Kr , regardless of the gain operator of (11) containing r + 1 gain matrices. However, unlike convergence conditions of classic ILC requiring only the information on the relative degree of the plant for agents, the consensus condition (12) also needs the information about the network topology of multiagent system since the plant of agents and the interaction network between agents both play important roles in the multiagent ILC. In addition, Theorem 1 also shows
that the learning algorithms can be performed to accomplish the tracking tasks for the agents even without prescribing any reference trajectory globally reachable as prior knowledge, in comparison with the existing consensus results of e.g., [20]. Motivated by the discussion remarked above, we induce the following result by considering a simple gain operator through particularly taking Kl = 0, l ∈ Zr−1 in (11). Corollary 1: For the multiagent system (1) associated with the directed graph G (A ), let the algorithm (9) be applied with K(q) = Kr qr . Then the consensus tracking objective (3) can be achieved if and only if the gain matrix Kr can be designed to satisfy (12). Proof: A direct consequence of Theorem 1. For the consensus tracking results presented above, it is left to determine whether, and under which conditions, there exists a gain matrix Kr that can fulfill (12). This is a more challenging issue to deal with in comparison with the design for classic ILC, which should simultaneously deal with the effects arising from both the plant for agents and the interaction network between agents. Toward this end, the next result is helpful, which is adopted directly from the literature [2]. Lemma 2: For a stochastic (respectively, Laplacian) matrix H, the algebraic multiplicity of its eigenvalue λ = 1 (respectively, λ = 0) is equal to one if and only if the graph associated with H has a spanning tree. Moreover, if the stochastic matrix H has positive diagonal elements and its associated graph has a spanning tree, then H has the property that |λ | < 1 holds for every eigenvalue not equal to one. With the aid of Lemma 2, we present the following theorem to give a necessary and sufficient design condition to guarantee the feasibility of (12). Theorem 2: There exists a gain matrix Kr satisfying (12) if and only if the directed graph G (A ) has a spanning tree, and the matrix CAr−1 B has a full-row rank. Proof: The proofs for the necessary and sufficient condition of (12) are established separately as follows. Necessity (Proof by Contradiction): Assume that G (A ) does not have a spanning tree. Thus, we can get from Lemma 2 that LA has at least two zero eigenvalues. This can further imply rank(DLA E ) ≤ rank(LA ) ≤ n − 2. That is, the square matrix DLA E of (n − 1)th order has at least one zero eigenvalue and, consequently, the matrix I − DLA E ⊗ CAr−1 BKr has at least one eigenvalue equal to one, due to the property of Kronecker product. Then ρ I − DLA E ⊗ CAr−1 BKr ≥ 1 follows, which contradicts with (12). Analogously, if CAr−1 B is not a full-row rank matrix, then we have rank(CAr−1 BKr ) ≤ rank(CAr−1 B) ≤ ny − 1. This will make the ny th-order square matrix CAr−1 BKr have zero eigenvalues. Hence, using the property of Kronecker product can also yield that I − DLA E ⊗CAr−1 BKr has at least one eigenvalue equal to one, and then it follows immediately that ρ I − DLA E ⊗ CAr−1BKr ≥ 1. That is, the contradiction with (12) is also generated. On the contrary, it is necessary for (12) to require that G (A ) has a spanning tree and CAr−1 B has a full-row rank. Sufficiency (Proof by Construction): If G (A ) has a spanning tree, and the matrix CAr−1 B is of full-row rank, then we will prove that a gain matrix Kr can be constructed to satisfy (12).
MENG et al.: TRACKING ALGORITHMS FOR MULTIAGENT SYSTEMS
As an alternative, we simply take
T r−1 r−1 T −1 CA B CA B Kr = ε CAr−1 B where ε is a scalar gain satisfying ε0 ε= max ∑ ai j
1665
(15)
(16)
i∈In j∈N i
that the scalar ε0 ∈ (0, 1) can be arbitrarily chosen. However, if we apply the learning gain matrix given in terms of (15), the selection of ε0 in (16) has impacts on the convergence speed of the learning algorithm (9). This is mainly because ε0 has influences on the matrix I − DLA E ⊗CAr−1 BKr , which, by (10) and (14), plays a significant role in achieving the convergence of the learning algorithm (9). V. M ONOTONIC AND / OR E XPONENTIALLY FAST C ONVERGENCE
with any prescribed positive scalar ε0 ∈ (0, 1). It can be verified that the condition (16) together with LA 1n = 0 guarantees I − ε LA ≥ (1 − ε0 )I,
(I − ε LA )1n = 1n
which clearly implies that I − ε LA is a stochastic matrix with positive diagonal elements. Since G (A ) has a spanning tree, using Lemma 2 gives that I − ε LA has exactly one eigenvalue equal to one and |λ | < 1 holds for every eigenvalue not equal
T to one. Moreover, noting the fact that [1n E ]−1 = F T D T , we can perform a similarity transformation on the matrix I − ε LA to further obtain
−1
1n E (I − ε LA ) 1n E F (I − ε LA ]1n F (I − ε LA )E = D(I − ε LA )1n D(I − ε LA )E 1 −ε F LA E = (17) 0 I − ε DLA E where F 1n = 1, F E = 0, D1n = 0, DE = I, and LA 1n = 0 are also inserted. Based on (17), we can then conclude that all the eigenvalues of I − ε DLA E satisfy |λ | < 1, i.e.,
ρ (I − ε DLA E ) < 1 which, together with (15) as well as the property of Kronecker product, yields ρ I − DLA E ⊗ CAr−1 BKr = ρ ((I − ε DLA E ) ⊗ I) = ρ (I − ε DLA E )
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, VOL. 24, NO. 10, OCTOBER 2013
Tracking Algorithms for Multiagent Systems Deyuan Meng, Member, IEEE, Yingmin Jia, Member, IEEE, Junping Du, and Fashan Yu Abstract— This paper is devoted to the consensus tracking issue on multiagent systems. Instead of enabling the networked agents to reach an agreement asymptotically as the time tends to infinity, the consensus tracking between agents is considered to be derived on a finite time interval as accurately as possible. We thus propose a learning algorithm with a gain operator to be determined. If the gain operator is designed in the form of a polynomial expression, a necessary and sufficient condition is obtained for the networked agents to accomplish the consensus tracking objective, regardless of the relative degree of the system model of agents. Moreover, the H∞ analysis approach is introduced to help establish conditions in terms of linear matrix inequalities (LMIs) such that the resulting processes of the presented learning algorithm can be guaranteed to monotonically converge in an iterative manner. The established LMI conditions can also enable the iterative learning processes to converge with an exponentially fast speed. In addition, we extend the learning algorithm to address the relative formation problem for multiagent systems. Numerical simulations are performed to demonstrate the effectiveness of learning algorithms in achieving both consensus tracking and relative formation objectives for the networked agents. Index Terms— Consensus tracking, H∞ analysis approach, learning algorithms, multiagent systems, relative formation.
I. I NTRODUCTION
A
MULTIAGENT system generally consists of a group of agents, which are required to carry out cooperative tasks for the group. The coordination control for multiagent systems has attracted considerable research interest owing to its wide applications in many practical areas. As shown in [1]–[4], the consensus issue plays a fundamental role in the multiagent coordination control, which can also be extended to include as special cases other coordination control problems, such as relative formation and reference tracking. In the literature, there have been proposed promising classes of distributed algorithms to guarantee the consensus objectives; see [5]–[7] for stochastic algorithms, [8]–[11] for pinning algorithms, and [12]–[17] for leader-following algorithms. Such consensus techniques aim to asymptotically
Manuscript received August 14, 2012; revised March 14, 2013; accepted May 5, 2013. Date of publication July 4, 2013; date of current version September 27, 2013. This work was supported in part by the National 973 Program under Grant 2012CB821200 and Grant 2012CB821201, the NSFC under Grant 61104011, Grant 61134005, and Grant 61203044, the MOE under Grant 2011110212003, and the Beijing Natural Science Foundation under Grant 4122046. D. Meng and Y. Jia are with the Seventh Research Division and the Department of Systems and Control, Beihang University, Beijing 100191, China (e-mail: [email protected]; [email protected]). J. Du is with the Beijing Key Laboratory of Intelligent Telecommunications Software and Multimedia, School of Computer Science and Technology, Beijing University of Posts and Telecommunications, Beijing 100876, China (e-mail: [email protected]). F. Yu is with the School of Electrical Engineering and Automation, Henan Polytechnic University, Jiaozuo 454000, China (e-mail: [email protected]). Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/TNNLS.2013.2262234
vi1 vi2
tend to be the same as t
O
t
vi3 vin
vi1 vi2 O vi3 vin
t
t r
r N
t t N
Fig. 1. Two classes of consensus tracking tasks of multiagent systems. Upper: consensus tracking achieved asymptotically as t → ∞. Lower: consensus tracking achieved perfectly over a finite time interval t ∈ [r,N]. Here, t is the time variable, r and N are two constants satisfying 0 ≤ r ≤ N < ∞, and vi j ∈ {v1 , v2 , v3 , ..., vn } for j = 1, 2, ..., n, where {v1 , v2 , v3 , ..., vn } denotes a group of n agents.
achieve agreement in the stationary case (i.e., as time goes to infinity) regarding a certain quantity of interest that depends on the states/outputs of agents [3]. As a demonstration, Fig. 1 (upper) gives a sketch of this class of consensus tracking tasks for multiagent systems. In contrast, Fig. 1 (lower) describes another class of consensus tracking tasks, which are required to be achieved perfectly over a finite time interval. This shows essentially a class of “desired” consensus tracking tasks. A good alternative way to practically accomplish such a desirable cooperative objective is to develop consensus tracking tasks with arbitrary high precision (in other words, to make consensus tracking tasks on the specified finite time interval be derived with a tolerance that can be arbitrarily prescribed). Since arbitrary high precision consensus tracking is one of the most desirable coordination control objectives in applications, this creates a new challenging consensus problem that is rather different from those typical ones considered in [1]–[17]. It has been shown in [18]–[20] that the iterative schemes can be applied to develop a type of learning algorithm such that the control inputs for agents can be learned using the disagreements between agents to obtain the consensus tracking tasks with arbitrary high precision. This idea has been applied also to the issues about relative formation between agents [20]–[23]. For practical instances requiring arbitrary high precision, see [24] for trajectory-keeping in satellite flying and [25] for stereo vision-based formation of mobile robots.
2162-237X © 2013 IEEE
MENG et al.: TRACKING ALGORITHMS FOR MULTIAGENT SYSTEMS
The main motivation behind the learning algorithms of [18]–[25] is to take full advantage of the salient property of iterative learning control (ILC) approaches, which are able to determine control input signals to accomplish the arbitrary high precision tracking through an iterative learning process. However, the learning algorithms of the multiagent systems are constructed by using the disagreements between agents to improve the control input signals for all the agents from one iteration to the next, which are different from typical ILC that works based on the tracking error generated by the prior knowledge of the desired reference trajectory for the controlled systems. Until now, there have been only limited results on studying the learning algorithms for arbitrary high precision consensus tracking tasks, and the corresponding theory is far from complete. In particular, it is worth noting that only the asymptotic convergence of the learning algorithms for multiagent systems has been considered in the existing results [18]–[25]; however, the issue for their monotonic convergence (or nonovershoot) has not been addressed, which is practically important in all kinds of learning algorithms, as demonstrated in the ILC surveys [26]–[28] and references therein. Motivated by the aforementioned observations, we deal with the consensus tracking tasks shown in Fig. 1 (lower) for multiagent systems in directed networks. By instead considering the arbitrary high precision consensus tracking tasks, we apply the iterative schemes to construct a class of learning algorithms. In contrast to the existing results [18]–[20], we design the learning algorithms without any prescribed reference as a priori and aim to achieve their monotonic convergence through the H∞ -based analysis approach to ILC [29]–[31]. The plant model for every agent is considered to possess not only multiple-input multiple-output (MIMO) dynamics but also a common relative degree, which can be higher order. This renders our considered consensus problem more challenging, because the analysis and design of our learning algorithms should take both the generic relative degree plant of agents and the network topology of the multiagent systems into account. Thus, it is required to develop new approaches such that the agents can be enabled to achieve the consensus tracking with each other. In this paper, we use the relative output knowledge between agents to construct a learning algorithm, which is in an update form and has a gain operator to be designed. If this operator is taken in terms of a polynomial expression, the agents can be enabled to achieve the consensus tracking with each other as soon as the input signals can take effect in their corresponding outputs, and a necessary and sufficient condition is established simultaneously to guarantee the asymptotic convergence of the multiagent learning system. It is verified that this convergence condition needs only the basic structure information of agents, under which an alternative method can be derived to induce the gain operator. Furthermore, the monotonic convergence for our learning algorithm is investigated, for which the H∞ analysis approach is applied to give convergence conditions in terms of linear matrix inequalities (LMIs), as well as directly providing formulas for the gain operator. It is also demonstrated that the LMI conditions can guarantee the multiagent learning systems
1661
to converge exponentially fast. In view of this observation, the exponentially fast convergence is further considered, and more relaxed conditions for our proposed learning algorithm can be derived. It is worthwhile to mention that the proposed learning algorithm is also extended to accomplish the relative formation between agents. In addition, some numerical simulation results are presented to demonstrate the effectiveness of our learning algorithms, which consider a six-agent system with the relative degree of two under directed graphs. This paper is organized as follows. In Section II, preliminaries in the graph theory are introduced. The problem statement is proposed for the consensus tracking of multiagent systems in Section III. The asymptotic convergence results are given in Section IV, and the monotonic and/or exponential convergence results are established in Section V. The consensus results are made an extension in Section VI to the relative formation issue for agents. Simulation results, and then conclusions, are posed in Sections VII and VIII, respectively. Notations: In = {1, 2, . . ., n}, ZN = {0, 1, . . . , N}, 1n = [1, 1, . . . , 1]T ∈ Rn , I and 0 denote the identity and null matrices with required dimensions, respectively, diag{·} denotes a block diagonal matrix with zero off-diagonal elements, and a bigstar () used in symmetric block matrices denotes a term induced by the symmetry. For matrices or vectors A and B, ρ (A) is the spectral radius of a matrix A, A2 is the spectral norm of a matrix A, A is the Euclidean norm of a vector A, A ≥ 0 if its elements are all nonnegative, A ≺ 0 (respectively, A 0) is a negative-definite (respectively, positive-definite) matrix, A ⊗ B is Kronecker product of A and B, and A ∈ Rn×n is a stochastic matrix1 where A ≥ 0 and A1n = 1n . For any discrete-time domain vector function z(t), q is a shift operator, i.e., qz(t) = z(t + 1) and q−1 z(t) = z(t − 1), and z(t)2 is the L2 -norm of z(t). II. P RELIMINARIES In this section, some preliminaries in graph theory are given. Let G denote an nth order directed graph consisting of a vertex set V and an edge set E. Here, we adopt vi to denote the vertex of G , and hence we have V = {vi : i ∈ In }. The corresponding pair (vi , v j ) is used to denote the edge of G . If (vi , v j ) ∈ E, then it means that there exists a communication channel between vi and v j , where the information flows from v j to vi . This implies also that v j is a neighbor of vi . Hence, Nvi = {v j : (vi , v j ) ∈ E} is defined as the neighbor set of the vertex vi . Let the index set of neighbors of the vertex vi be given as Ni = { j : (vi , v j ) ∈ E}. In addition, it can be easily seen that E ⊆ {(vi , v j ) : vi , v j ∈ V}. Example 1: Let us consider the directed graph shown in Fig. 2. If we take the vertex v8 for instance, there exists information that flows from the vertex v7 to it. Hence, we know that (v8 , v7 ) is an edge, and v7 is a neighbor of v8 . Moreover, we can derive Nv8 = {v5 , v6 , v7 } and N8 = {5, 6, 7}. 1 If A ≥ 0 and A1 = 1 , it can be seen that every row of A may be considered n n as a discrete probability distribution over a sample space with n points. Hence, n×n satisfying A ≥ 0 and A1n = 1n is said to be a stochastic matrix. here A ∈ R For more details, see also [32].
1662
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, VOL. 24, NO. 10, OCTOBER 2013
v6
v1
v8 v8
v2
v5
v 8 v7
v3 v7
v4
Fig. 2. Small example of a directed graph with eight vertices and ten edges. Fig. 3.
Remark 1: It can be seen that there are only communication channels between each vertex and its neighbor vertices (rather than all vertices). This implies that the information of neighbor vertices of a vertex is available to its controller design, whereas the information of other non-neighbor vertices is not available. Such a way to the controller design is the nearest neighbor rule considered in the literature [1]–[3]. Take for instance, the vertex v8 of the directed graph shown in Fig. 2. According to the nearest neighbor rule, the controller design can only use the information of its neighbor vertices v5 , v6 , and v7 . In addition, we also introduce the following definitions with respect to the directed graphs. A path in G is a finite sequence vi1 , vi2 , . . . , vi j of vertices such that (vil , vil+1 ) ∈ E holds for l = 1, 2, . . . , j − 1. If there is a special vertex that can be connected to all the other vertices through paths, then G is said to have a spanning tree, and this special vertex is called the root vertex. A nonnegative weighted adjacency matrix A = [ai j ] associated with G is defined to model the information exchange between agents, where ai j > 0 ⇔ (vi , v j ) ∈ E and ai j = 0 otherwise. Here, the weighted directed graph is denoted by G (A ), and aii = 0 is assumed for i ∈ In . The Laplacian matrix of G (A ) is defined as LA = Δ − A , where Δ = diag{Δ11 , Δ22 , . . . , Δnn } and Δii = ∑nj=1, j =i ai j for all i ∈ In . Example 2: We again take for instance the directed graph of Fig. 2. It can be seen that the sequence of vertices given by v6 , v8 , v5 , v3 , v2 , v7 , and v4 is a path in this directed graph, but there are no paths such that these seven vertices can be connected to the vertex v1 . Hence, there are no spanning trees in the directed graph of Fig. 2. If we remove the vertex v1 from this directed graph, the remaining part has spanning trees (particularly, v4 is connected to v2 , v3 , v5 , v6 , v7 , and v8 through paths). Moreover, a weighted graph for the directed graph of Fig. 2 is shown in Fig. 3. It can be easily seen from this figure that the adjacency matrix A is given by ⎡ ⎤ 0 0 0 0 0 0 0 0 ⎢0 0 0 0 0 0 a27 0 ⎥ ⎢ ⎥ ⎢0 a32 0 0 0 0 0 0 ⎥ ⎢ ⎥ ⎢0 0 0 0 0 0 a47 0 ⎥ ⎢ ⎥ A =⎢ ⎥ ⎢0 0 a53 0 0 0 0 0 ⎥ ⎢0 0 0 0 0 0 0 a68 ⎥ ⎢ ⎥ ⎣0 0 0 a74 0 a76 0 0 ⎦ 0 0 0 0 a85 a86 a87 0 where the nonzero elements ai j are positive weights associated with the edges of the directed graph shown in Fig. 3.
Weighted directed graph corresponding to the one shown in Fig. 2.
III. P ROBLEM S TATEMENT In this paper, we consider networked systems with n agents, which are labeled 1 through n. For all i ∈ In , the ith agent is regarded as the vertex vi in the weighted directed graph G (A ), and is assumed to have the following MIMO dynamics: xi,k (t + 1) = Axi,k (t) + Bui,k (t) (1) yi,k (t) = Cxi,k (t), xi,k (0) = xi0 where t ∈ ZN and k ∈ Z+ denote the time and iteration indices, respectively, xi,k (t) ∈ Rnx , ui,k (t) ∈ Rnu and yi,k (t) ∈ Rny denote the state, protocol (or input) and output, respectively, and A, B, and C denote any matrices of appropriate dimensions. Without loss of generality, CAi B = 0 (i < r − 1) and CAr−1 B of full-row rank are assumed to meet a basic requirement of the following learning algorithms on the system structure of the agents [that is, the relative degree of (1) is r, where r ≥ 1]. It can be easily verified that yi,k (t) = gi (t) holds for all t = 0, . . ., r − 1, where gi (t) = CAt xi0 . For any t ≥ r, the input–output relationship of (1) is given by yi,k (t) = P(q)ui,k (t) + gi(t)
(2)
P(q) = C(qI − A)−1B.
where As shown above, the output of every agent described by (1) is only dependent on the initial state over the first r time steps, which cannot be controlled by the protocol. This in fact arises from the effects of r since the relative degree for discrete-time systems means exactly the steps of time delay in the output, in order to have the control input appearing. By this observation, we address the output consensus tracking problem over t ∈ {r, r + 1, . . ., N} for the multiagent system (1). Let us consider the disagreement between the agents vi and v j as their relative output yi,k (t) − y j,k (t) for ∀i, j ∈ In , and if
lim yi,k (t) − y j,k (t) = 0, t = r, r + 1, . . . , N (3) k→∞
then the agents vi and v j are said to accomplish the consensus tracking with each other. It can be seen that the disagreement yi,k (t)−y j,k (t) can be guaranteed to be arbitrarily small (that is, yi,k (t) − y j,k (t)2 < ε for any arbitrarily prescribed tolerance ε > 0) after a certain number of iterations if the objective (3) is derived. This means that (3) shows the arbitrary high precision consensus tracking tasks, which is different from the consensus problems considered in [1]–[17] and also can include as special cases the consensus problems studied in [18]–[20].
MENG et al.: TRACKING ALGORITHMS FOR MULTIAGENT SYSTEMS
1663
Remark 2: In practice, the multiagent system (1) or (2) can be considered as the linearized model of practical plants, such as flying satellites [24] and mobile robots [25]. The consensus tracking problem (3) can be extended to the relative formation problem considered in [24] and [25] (see the following discussions of Section VI for details). In particular, the individual plant of such applications usually suffers the relative degree problem, which is taken into account by (1). A. Problem Representation It can be easily seen that (3) holds if there exists a common (unknown) trajectory c(t) ∈ Rny on ZN−r satisfying lim yk (t + r) = 1n ⊗ c(t), t ∈ ZN−r
k→∞
where yk (t) = [yT1,k (t), yT2,k (t), . . . , yTn,k (t)]T . By noting this fact, let us denote zk (t) = (D ⊗ I)yk (t + r), t ∈ ZN−r
(4)
where D ∈ R(n−1)×n represents a full-row matrix with the null space spanned by 1n . For the sake of computational simplicity, here we adopt
(5) D = −1n−1 I . 0 E = ∈ Rn×(n−1) I
If we denote
then we can obtain with (4) that
F yk (t + r) = 1n E ⊗ I yk (t + r) D = (1n F ⊗ I)yk (t + r) + (E ⊗ I)zk (t)
k→∞
= yi,k (t + r)
+ ∑ ai j P(q)K(q) y j,k (t+r)−yi,k (t+r) j∈Ni
(6)
which can be formulated in a compact form of yk+1 (t + r) = [I − LA ⊗ P(q)K(q)]yk (t + r). This, together with (4) and (7), further leads to
(7)
where F = [1, 0, . . . , 0] ∈ R1×n . Furthermore, we can develop (4) to reveal the equivalent relationship between zk (t) and the disagreement error for agents, i.e., (3) holds if and only if2 lim zk (t) = 0, t ∈ ZN−r
where ui,0 (t) is a bounded initial input, which can be arbitrarily prescribed, K(q) is an nu × ny gain operator to be designed, and ai j is the weight of edge (vi , v j ) in the directed graph G (A ). Remark 3: Although the iterative schemes have been introduced to design coordination control algorithms of multiagent systems [18]–[25], (9) is presented as a new general consensus tracking algorithm that can take the relative degree of multiagent systems into account and does not require any references as the prior knowledge. In particular, if we set yn,k (t) ≡ yd (t) (i.e., assume that vn is a leader agent without loss of generality to specify the reference trajectory yd (t) for agents), (9) gives a learning algorithm to derive the consensus tracking with yd (t). This implies that (9) can include as special cases the consensus tracking algorithms in [18]–[20], which can be further extended to include as special cases the learning algorithms for relative formation issues of multiagent systems presented in [20]–[25] (for more details, see the following extensions in Section VI). By combining (9) with (2), it can be derived that
yi,k+1 (t + r) = yi,k (t+r)+P(q) ui,k+1 (t+r)−ui,k (t+r)
(8)
which is a stability problem. Thus, we can achieve the consensus tracking objective (3) through solving the stability problem (8) in the following discussions. B. Algorithm Design The consensus tracking algorithm considered in this paper is a type of learning algorithm that is designed using the iterative schemes. For each agent vi , i ∈ In , the next learning algorithm is constructed based on the relative output information between it and its neighbors
ui,k+1 (t) = ui,k (t)+ ∑ ai j K(q) y j,k (t)−yi,k (t) , t ∈ ZN−r j∈Ni
(9) 2 If (3) holds, we can easily obtain (8) based on (4) and (5). On the contrary,
if (8) holds, we can deduce from (4) and (5) that limk→∞ yl,k (t) − y1,k (t) = 0, where t = r, r +1, ..., N and l ∈ In . This leads to that, have
for ∀i, j ∈ In , we limk→∞ yi,k (t) − y j,k (t) = limk→∞ yi,k (t) − y1,k (t) − y j,k (t) − y1,k (t) = limk→∞ yi,k (t) − y1,k (t) − limk→∞ y j,k (t) − y1,k (t) = 0, where t = r, r + 1, ..., N. That is, (3) can be derived based on the satisfaction of (8).
zk+1 (t) = (D ⊗ I)yk+1 (t + r) = (D ⊗ I) [I − LA ⊗ P(q)K(q)]yk (t + r) = (D ⊗ I) [I − LA ⊗ P(q)K(q)](1n F ⊗ I)yk (t + r) +(D ⊗ I) [I − LA ⊗ P(q)K(q)](E ⊗ I)zk (t) = [I − DLA E ⊗ P(q)K(q)]zk (t) for t ∈ ZN−r (10) where the facts D1n = 0, LA 1n = 0, and DE = I are inserted. It can be obtained from (10) that the stability of zk (t) as k → ∞ depends on the mapping operator I − DLA E ⊗ P(q)K(q). However, it is worth noticing that this stability problem is different from that of classical ILC, in which not only the plant for agents but also the network topology of multiagent system plays a significant role (see also the discussion in [20]). In spite of the challenging issue, we attempt to establish some consensus tracking results based on the development of (10) by making (3) be approached both asymptotically and monotonically (or, exponentially fast). IV. A SYMPTOTIC C ONVERGENCE The following lemma is helpful in achieving the asymptotic convergence for consensus learning systems, which is adopted directly from the literature [20]. Lemma 1: For zk (t), let appropriately dimensioned matrices Az , Bz , Cz , and Dz make zk+1 (t) = [Cz (qI − Az )−1 Bz + Dz ]zk (t) hold for zk (t) over time steps t ∈ ZN−r . Then (8) can be derived if and only if ρ (Dz ) < 1.
1664
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, VOL. 24, NO. 10, OCTOBER 2013
Now, with Lemma 1, the following theorem can be proposed by considering the consensus learning algorithm (9) via a gain operator in the polynomial form. Theorem 1: Consider the multiagent system (1) associated with the directed graph G (A ). Let the algorithm (9) be applied with its gain operator K(q) in the following polynomial form: K(q) = K0 + K1 q + · · · + Kr qr
(11)
where Kl , ∀l ∈ Zr is an nu ×ny gain matrix. Then the consensus tracking objective (3) can be achieved if and only if the gain matrix Kr can be designed to satisfy (12) ρ I − DLA E ⊗ CAr−1 BKr < 1. Proof: By incorporating P(q) = C(qI − A)−1 B and K(q) = K0 + K1 q + · · · + Kr qr , we have
P(q)K(q) =
∞
∑ CAs−1Bq−s
s=r
= CA
r−1
r
∑ Kl ql
l=0
∞
BKr + ∑ CA s=1
s−1
r
∑ A BKl l
q−i
l=0 r −1
= CAr−1 BKr + C(qI − A)
∑ Al BKl
(13)
l=0
where the fact that CAs B = 0, s ∈ Zr−2 holds due to the relative degree r is inserted. Using the property of Kronecker product, we can get from (13) immediately that I − DLA E ⊗ P(q)K(q) = I − DLA E ⊗ CAr−1 BKr
r
− DLA E ⊗ C(qI − A)−1 ∑ Al BKl l=0
= I − DLA E ⊗ CAr−1 BKr −1
+ (I ⊗ C)(qI − I ⊗ A)
r
−DLA E ⊗ ∑ A BKl . (14) l
l=0
By combining (10) and (14), we can obtain from Lemma 1 that (12) provides a necessary and sufficient condition to guarantee (8). Then the equivalence between (3) and (8) further gives that (3) can be established if and only if (12) can be satisfied. This completes the proof. Remark 4: It can be clearly seen from Theorem 1 that there exist learning algorithms in terms of (9) and (11) to accomplish the consensus tracking objective (3) for the multiagent system (1) associated with G (A ). In practice, it means that the control input signals can be determined through the presented learning algorithms such that the consensus tracking tasks for agents are achieved with arbitrary high precision. This development can be further guaranteed with a necessary and sufficient condition in parallel with the convergence results for classic ILC, which is only dependent on Kr , regardless of the gain operator of (11) containing r + 1 gain matrices. However, unlike convergence conditions of classic ILC requiring only the information on the relative degree of the plant for agents, the consensus condition (12) also needs the information about the network topology of multiagent system since the plant of agents and the interaction network between agents both play important roles in the multiagent ILC. In addition, Theorem 1 also shows
that the learning algorithms can be performed to accomplish the tracking tasks for the agents even without prescribing any reference trajectory globally reachable as prior knowledge, in comparison with the existing consensus results of e.g., [20]. Motivated by the discussion remarked above, we induce the following result by considering a simple gain operator through particularly taking Kl = 0, l ∈ Zr−1 in (11). Corollary 1: For the multiagent system (1) associated with the directed graph G (A ), let the algorithm (9) be applied with K(q) = Kr qr . Then the consensus tracking objective (3) can be achieved if and only if the gain matrix Kr can be designed to satisfy (12). Proof: A direct consequence of Theorem 1. For the consensus tracking results presented above, it is left to determine whether, and under which conditions, there exists a gain matrix Kr that can fulfill (12). This is a more challenging issue to deal with in comparison with the design for classic ILC, which should simultaneously deal with the effects arising from both the plant for agents and the interaction network between agents. Toward this end, the next result is helpful, which is adopted directly from the literature [2]. Lemma 2: For a stochastic (respectively, Laplacian) matrix H, the algebraic multiplicity of its eigenvalue λ = 1 (respectively, λ = 0) is equal to one if and only if the graph associated with H has a spanning tree. Moreover, if the stochastic matrix H has positive diagonal elements and its associated graph has a spanning tree, then H has the property that |λ | < 1 holds for every eigenvalue not equal to one. With the aid of Lemma 2, we present the following theorem to give a necessary and sufficient design condition to guarantee the feasibility of (12). Theorem 2: There exists a gain matrix Kr satisfying (12) if and only if the directed graph G (A ) has a spanning tree, and the matrix CAr−1 B has a full-row rank. Proof: The proofs for the necessary and sufficient condition of (12) are established separately as follows. Necessity (Proof by Contradiction): Assume that G (A ) does not have a spanning tree. Thus, we can get from Lemma 2 that LA has at least two zero eigenvalues. This can further imply rank(DLA E ) ≤ rank(LA ) ≤ n − 2. That is, the square matrix DLA E of (n − 1)th order has at least one zero eigenvalue and, consequently, the matrix I − DLA E ⊗ CAr−1 BKr has at least one eigenvalue equal to one, due to the property of Kronecker product. Then ρ I − DLA E ⊗ CAr−1 BKr ≥ 1 follows, which contradicts with (12). Analogously, if CAr−1 B is not a full-row rank matrix, then we have rank(CAr−1 BKr ) ≤ rank(CAr−1 B) ≤ ny − 1. This will make the ny th-order square matrix CAr−1 BKr have zero eigenvalues. Hence, using the property of Kronecker product can also yield that I − DLA E ⊗CAr−1 BKr has at least one eigenvalue equal to one, and then it follows immediately that ρ I − DLA E ⊗ CAr−1BKr ≥ 1. That is, the contradiction with (12) is also generated. On the contrary, it is necessary for (12) to require that G (A ) has a spanning tree and CAr−1 B has a full-row rank. Sufficiency (Proof by Construction): If G (A ) has a spanning tree, and the matrix CAr−1 B is of full-row rank, then we will prove that a gain matrix Kr can be constructed to satisfy (12).
MENG et al.: TRACKING ALGORITHMS FOR MULTIAGENT SYSTEMS
As an alternative, we simply take
T r−1 r−1 T −1 CA B CA B Kr = ε CAr−1 B where ε is a scalar gain satisfying ε0 ε= max ∑ ai j
1665
(15)
(16)
i∈In j∈N i
that the scalar ε0 ∈ (0, 1) can be arbitrarily chosen. However, if we apply the learning gain matrix given in terms of (15), the selection of ε0 in (16) has impacts on the convergence speed of the learning algorithm (9). This is mainly because ε0 has influences on the matrix I − DLA E ⊗CAr−1 BKr , which, by (10) and (14), plays a significant role in achieving the convergence of the learning algorithm (9). V. M ONOTONIC AND / OR E XPONENTIALLY FAST C ONVERGENCE
with any prescribed positive scalar ε0 ∈ (0, 1). It can be verified that the condition (16) together with LA 1n = 0 guarantees I − ε LA ≥ (1 − ε0 )I,
(I − ε LA )1n = 1n
which clearly implies that I − ε LA is a stochastic matrix with positive diagonal elements. Since G (A ) has a spanning tree, using Lemma 2 gives that I − ε LA has exactly one eigenvalue equal to one and |λ | < 1 holds for every eigenvalue not equal
T to one. Moreover, noting the fact that [1n E ]−1 = F T D T , we can perform a similarity transformation on the matrix I − ε LA to further obtain
−1
1n E (I − ε LA ) 1n E F (I − ε LA ]1n F (I − ε LA )E = D(I − ε LA )1n D(I − ε LA )E 1 −ε F LA E = (17) 0 I − ε DLA E where F 1n = 1, F E = 0, D1n = 0, DE = I, and LA 1n = 0 are also inserted. Based on (17), we can then conclude that all the eigenvalues of I − ε DLA E satisfy |λ | < 1, i.e.,
ρ (I − ε DLA E ) < 1 which, together with (15) as well as the property of Kronecker product, yields ρ I − DLA E ⊗ CAr−1 BKr = ρ ((I − ε DLA E ) ⊗ I) = ρ (I − ε DLA E )
