Yan Zhang

Member, IEEE Zhengrong Xiang
email: xiangzr@njust.edu.cn

Mohammed Chadli
email: mchadli20@gmail.com

Prescribed-Time Formation Control for a Class of Multiagent Systems via Fuzzy Reinforcement Learning

Keywords: Formation control, fuzzy logic systems, Hamilton-Jacobi-Bellman, nonlinear multiagent systems (MASs), prescribed-time control, reinforcement learning (RL)

This article concerns optimal prescribed-time formation control for a class of nonlinear multiagent systems (MASs). Optimal control depends on the solution of the Hamilton-Jacobi-Bellman equation, which is hard to be calculated directly due to its inherent nonlinearity. To overcome this difficulty, the reinforcement learning strategy with fuzzy logic systems is proposed, in which identifier, actor, and critic are used to estimate unknown nonlinear dynamics, implement control behavior, and evaluate system performance, respectively. Different from the existing optimal control algorithms, a new performance index function considering formation error cost and control input energy cost is constructed to achieve optimal formation control of MASs within a prescribed time. The presented control strategy can ensure that the formation error converges to the desired accuracy within a prescribed time. Finally, the validity of the presented strategy is verified via a simulation example.

I. INTRODUCTION

C OOPERATIVE control of multiagent systems (MASs) has drawn widespread concern from scholars because of its extensive application. Cooperative control includes consensus control, containment control, formation control, etc. Formation control refers to constructing control protocols to stabilize agents' positions with respect to each other and to achieve specified geometric shapes. The formation control of MASs has been applied in the fields of mobile robots [START_REF] Sharma | Tracking control of mobile robots in formation in the presence of disturbances[END_REF], underactuated marine surface vessels [START_REF] Børhaug | Straight line path following for formations of underactuated marine surface vessels[END_REF], and unmanned aerial vehicles [START_REF] Kang | Robust leaderless time-varying formation control for nonlinear unmanned aerial vehicle swarm system with communication delays[END_REF]. Zhang et al. [START_REF] Zhang | Adaptive time-varying formation tracking control for multiagent systems with nonzero leader input by intermittent communications[END_REF] presented a formation-tracking control scheme for linear MASs. Then, an event-triggered formation containment control strategy was developed in [START_REF] Zhang | Adaptive event-triggered time-varying output bipartite formation containment of multiagent systems under directed graphs[END_REF] for linear MASs. Furthermore, the formation control issues for nonlinear MASs were further investigated in [START_REF] Tian | Distributed timevarying group formation tracking for multiagent systems with switching interaction topologies via adaptive control protocols[END_REF], [START_REF] Yu | Neural adaptive distributed formation control of nonlinear multi-UAVs with unmodeled dynamics[END_REF], [START_REF] Yu | Distributed adaptive fuzzy formation control of uncertain multiple unmanned aerial vehicles with actuator faults and switching topologies[END_REF], and [START_REF] Tang | Event-based formation control for nonlinear multiagent systems under DoS attacks[END_REF]. The results obtained in [START_REF] Zhang | Adaptive time-varying formation tracking control for multiagent systems with nonzero leader input by intermittent communications[END_REF], [START_REF] Zhang | Adaptive event-triggered time-varying output bipartite formation containment of multiagent systems under directed graphs[END_REF], [START_REF] Tian | Distributed timevarying group formation tracking for multiagent systems with switching interaction topologies via adaptive control protocols[END_REF], [START_REF] Yu | Neural adaptive distributed formation control of nonlinear multi-UAVs with unmodeled dynamics[END_REF], [START_REF] Yu | Distributed adaptive fuzzy formation control of uncertain multiple unmanned aerial vehicles with actuator faults and switching topologies[END_REF], and [START_REF] Tang | Event-based formation control for nonlinear multiagent systems under DoS attacks[END_REF] only realized the expected formation; however, the optimal performance was not considered.

Optimal control is a research topic with important academic value and broad application potential [START_REF] Wu | Optimal partial feedback attacks in cyber-physical power systems[END_REF], [START_REF] Zhang | Optimal DoS attack scheduling in wireless networked control system[END_REF].T om a k et h e controlled system complete the tasks with the least resources, Bellman in [START_REF] Bellman | Dynamic Programming[END_REF] and Pontryagin in [START_REF] Pontryagin | Robust process control[END_REF] put forward the optimal control. Usually, it is expected that the optimal function can be got by solving the Hamilton-Jacobi-Bellman (HJB) equation, and then, the optimal controller can be constructed [START_REF] Lewis | Optimal Control,3 r de d[END_REF].A s a partial differential equation with multiple nonlinear terms, it is hard to find an analytical solution to the HJB equation. As a machine learning method, reinforcement learning (RL) was inspired by animal behavior and gradually became a powerful tool. Under the actor-critic architecture, the optimal consensus and formation control of MASs concerned in [START_REF] Wen | Optimized leader-follower consensus control using reinforcement learning for a class of second-order nonlinear multiagent systems[END_REF], [START_REF] Wen | Optimized backstepping consensus control using reinforcement learning for a class of nonlinear strict-feedbackdynamic multi-agent systems[END_REF], and [START_REF] Li | Fuzzy adaptive optimized leader-following formation control for second-order stochastic multi-agent systems[END_REF], [START_REF] Lan | Time-varying optimal formation control for second-order multiagent systems based on neural network observer and reinforcement learning[END_REF], [START_REF] Wen | Optimized adaptive nonlinear tracking control using actor-critic reinforcement learning strategy[END_REF], respectively. It should be pointed out that most RL algorithms [START_REF] Wen | Optimized leader-follower consensus control using reinforcement learning for a class of second-order nonlinear multiagent systems[END_REF], [START_REF] Wen | Optimized backstepping consensus control using reinforcement learning for a class of nonlinear strict-feedbackdynamic multi-agent systems[END_REF], [START_REF] Li | Fuzzy adaptive optimized leader-following formation control for second-order stochastic multi-agent systems[END_REF], [START_REF] Lan | Time-varying optimal formation control for second-order multiagent systems based on neural network observer and reinforcement learning[END_REF], [START_REF] Wen | Optimized adaptive nonlinear tracking control using actor-critic reinforcement learning strategy[END_REF] need prior knowledge of system dynamics while the nonlinear terms of most actual dynamic systems may be unknown. By applying RL under the actor-critic-identifier (ACI) architecture, Wen et al. [START_REF] Wen | Optimized multi-agent formation control based on an identifier-actor-critic reinforcement learning algorithm[END_REF], [START_REF] Wen | Optimized formation control using simplified reinforcement learning for a class of multiagent systems with unknown dynamics[END_REF] developed the optimal control strategies for MASs. However, the optimal control results [START_REF] Wu | Optimal partial feedback attacks in cyber-physical power systems[END_REF], [START_REF] Zhang | Optimal DoS attack scheduling in wireless networked control system[END_REF], [START_REF] Bellman | Dynamic Programming[END_REF], [START_REF] Pontryagin | Robust process control[END_REF], [START_REF] Lewis | Optimal Control,3 r de d[END_REF], [START_REF] Wen | Optimized leader-follower consensus control using reinforcement learning for a class of second-order nonlinear multiagent systems[END_REF], [START_REF] Wen | Optimized backstepping consensus control using reinforcement learning for a class of nonlinear strict-feedbackdynamic multi-agent systems[END_REF], [START_REF] Li | Fuzzy adaptive optimized leader-following formation control for second-order stochastic multi-agent systems[END_REF], [START_REF] Lan | Time-varying optimal formation control for second-order multiagent systems based on neural network observer and reinforcement learning[END_REF], [START_REF] Wen | Optimized adaptive nonlinear tracking control using actor-critic reinforcement learning strategy[END_REF], [START_REF] Wen | Optimized multi-agent formation control based on an identifier-actor-critic reinforcement learning algorithm[END_REF], [START_REF] Wen | Optimized formation control using simplified reinforcement learning for a class of multiagent systems with unknown dynamics[END_REF] can only ensure the expected performance when the time variable goes to infinity.

Convergence rate, as a crucial performance criterion, is one of the hot topics in the control field [START_REF] Zhang | Finite-time adaptive neural control for a class of nonlinear systems with asymmetric time-varying full-state constraints[END_REF], [START_REF] Zhang | Predefined-time adaptive fuzzy control for a class of nonlinear systems with output hysteresis[END_REF]. To improve the convergence rate, some finite-time formation control results of MASs were reported in [START_REF] Zhou | Finite-time adaptive fuzzy prescribed performance formation control for high-order nonlinear multi-agent systems based on event-triggered mechanism[END_REF], [START_REF] Xiao | Finite-time formation control for multi-agent systems[END_REF], [START_REF] Liu | Finite-time formation control for linear multiagent systems: A motion planning approach[END_REF], and [START_REF] Hua | Finite-time time-varying formation tracking for high-order multiagent systems with mismatched disturbances[END_REF]. The settling time in [START_REF] Zhou | Finite-time adaptive fuzzy prescribed performance formation control for high-order nonlinear multi-agent systems based on event-triggered mechanism[END_REF], [START_REF] Xiao | Finite-time formation control for multi-agent systems[END_REF], [START_REF] Liu | Finite-time formation control for linear multiagent systems: A motion planning approach[END_REF], and [START_REF] Hua | Finite-time time-varying formation tracking for high-order multiagent systems with mismatched disturbances[END_REF] is dependent on initial conditions, which limits the application of finite-time control. To address this issue, some fixed-time formation control issues were investigated in [START_REF] Cheng | Fixed-time fault-tolerant formation control for a cooperative heterogeneous multiagent system with prescribed performance[END_REF], [START_REF] Chang | Fixed-time formation-containment control for uncertain multi-agent systems with varying gain extended state observer[END_REF], [START_REF] Hou | Adaptive fuzzy fixed time timevarying formation control for heterogeneous multi-agent systems with full state constraints[END_REF], and [START_REF] Zheng | NN-based fixed-time attitude tracking control for multiple unmanned aerial vehicles with nonlinear faults[END_REF]. It should be mentioned that the settling time in [START_REF] Cheng | Fixed-time fault-tolerant formation control for a cooperative heterogeneous multiagent system with prescribed performance[END_REF], [START_REF] Chang | Fixed-time formation-containment control for uncertain multi-agent systems with varying gain extended state observer[END_REF], [START_REF] Hou | Adaptive fuzzy fixed time timevarying formation control for heterogeneous multi-agent systems with full state constraints[END_REF], and [START_REF] Zheng | NN-based fixed-time attitude tracking control for multiple unmanned aerial vehicles with nonlinear faults[END_REF] depends on the design parameters, which makes it difficult to adjust the design parameters according to actual control requirements. For this reason, some prescribed-time consensus control issues for MASs were investigated in [START_REF] Mao | Predefined-time bounded consensus of multiagent systems with unknown nonlinearity via distributed adaptive fuzzy control[END_REF], [START_REF] Chen | Prescribed-time event-triggered bipartite consensus of multiagent systems[END_REF], [START_REF] Liang | Prescribed performance cooperative control for multiagent systems with input quantization[END_REF], [START_REF] Ren | Prescribed-time consensus tracking of multiagent systems with nonlinear dynamics satisfying time-varying Lipschitz growth rates[END_REF], [START_REF] Li | Output feedback predefined-time bipartite consensus control for high-order nonlinear multiagent systems[END_REF], [START_REF] Ni | Predefined-time consensus tracking of second-order multiagent systems[END_REF], and [START_REF] Wang | Prescribed-time consensus and containment control of networked multiagent systems[END_REF]. However, the schemes developed in [START_REF] Zhang | Finite-time adaptive neural control for a class of nonlinear systems with asymmetric time-varying full-state constraints[END_REF], [START_REF] Zhang | Predefined-time adaptive fuzzy control for a class of nonlinear systems with output hysteresis[END_REF], [START_REF] Zhou | Finite-time adaptive fuzzy prescribed performance formation control for high-order nonlinear multi-agent systems based on event-triggered mechanism[END_REF], [START_REF] Xiao | Finite-time formation control for multi-agent systems[END_REF], [START_REF] Liu | Finite-time formation control for linear multiagent systems: A motion planning approach[END_REF], [START_REF] Hua | Finite-time time-varying formation tracking for high-order multiagent systems with mismatched disturbances[END_REF], [START_REF] Cheng | Fixed-time fault-tolerant formation control for a cooperative heterogeneous multiagent system with prescribed performance[END_REF], [START_REF] Chang | Fixed-time formation-containment control for uncertain multi-agent systems with varying gain extended state observer[END_REF], [START_REF] Hou | Adaptive fuzzy fixed time timevarying formation control for heterogeneous multi-agent systems with full state constraints[END_REF], [START_REF] Zheng | NN-based fixed-time attitude tracking control for multiple unmanned aerial vehicles with nonlinear faults[END_REF], [START_REF] Mao | Predefined-time bounded consensus of multiagent systems with unknown nonlinearity via distributed adaptive fuzzy control[END_REF], [START_REF] Chen | Prescribed-time event-triggered bipartite consensus of multiagent systems[END_REF], [START_REF] Liang | Prescribed performance cooperative control for multiagent systems with input quantization[END_REF], [START_REF] Ren | Prescribed-time consensus tracking of multiagent systems with nonlinear dynamics satisfying time-varying Lipschitz growth rates[END_REF], [START_REF] Li | Output feedback predefined-time bipartite consensus control for high-order nonlinear multiagent systems[END_REF], [START_REF] Ni | Predefined-time consensus tracking of second-order multiagent systems[END_REF], and [START_REF] Wang | Prescribed-time consensus and containment control of networked multiagent systems[END_REF] did not take into account optimal performances. To the author's knowledge, for a class of uncertain nonlinear MASs, the prescribed-time formation control via RL is still an interesting and challenging topic, which is rarely reported.

Motivated by the above statements, a prescribed-time formation control strategy is presented for a class of MASs, which can guarantee that the formation error converges to the desired accuracy within a prescribed time. The main innovations are as follows.

1) Different from the existing prescribed-time control [START_REF] Mao | Predefined-time bounded consensus of multiagent systems with unknown nonlinearity via distributed adaptive fuzzy control[END_REF], [START_REF] Chen | Prescribed-time event-triggered bipartite consensus of multiagent systems[END_REF], [START_REF] Liang | Prescribed performance cooperative control for multiagent systems with input quantization[END_REF], [START_REF] Ren | Prescribed-time consensus tracking of multiagent systems with nonlinear dynamics satisfying time-varying Lipschitz growth rates[END_REF], [START_REF] Li | Output feedback predefined-time bipartite consensus control for high-order nonlinear multiagent systems[END_REF], [START_REF] Ni | Predefined-time consensus tracking of second-order multiagent systems[END_REF], [START_REF] Wang | Prescribed-time consensus and containment control of networked multiagent systems[END_REF], the optimal control problem is considered. By constructing an auxiliary function and an error transformation function, the original performance index function is rewritten, which lays the foundation for realizing the prescribed-time formation control of MASs with the least resources.

2) The proposed strategy can eliminate the assumption of persistence excitation, which is required by most optimal control algorithms (such as [START_REF] Wen | Optimized adaptive nonlinear tracking control using actor-critic reinforcement learning strategy[END_REF] and [START_REF] Wen | Optimized multi-agent formation control based on an identifier-actor-critic reinforcement learning algorithm[END_REF]). In addition, new actor updating laws are designed to implement training sufficiently. 3) Compared with the existing optimal control results for MASs (such as [START_REF] Bellman | Dynamic Programming[END_REF], [START_REF] Pontryagin | Robust process control[END_REF], [START_REF] Lewis | Optimal Control,3 r de d[END_REF], [START_REF] Wen | Optimized leader-follower consensus control using reinforcement learning for a class of second-order nonlinear multiagent systems[END_REF], [START_REF] Wen | Optimized backstepping consensus control using reinforcement learning for a class of nonlinear strict-feedbackdynamic multi-agent systems[END_REF], [START_REF] Li | Fuzzy adaptive optimized leader-following formation control for second-order stochastic multi-agent systems[END_REF], [START_REF] Lan | Time-varying optimal formation control for second-order multiagent systems based on neural network observer and reinforcement learning[END_REF], [START_REF] Wen | Optimized adaptive nonlinear tracking control using actor-critic reinforcement learning strategy[END_REF], [START_REF] Wen | Optimized multi-agent formation control based on an identifier-actor-critic reinforcement learning algorithm[END_REF], [START_REF] Wen | Optimized formation control using simplified reinforcement learning for a class of multiagent systems with unknown dynamics[END_REF]), the settling time is not influenced by initial conditions and design parameters and can be prescribed. The rest is organized as follows. Section II gives the problem description and some preliminaries. The prescribed-time control scheme is developed in Section III. Thereafter, the validity of the strategy is verified in Section IV via a simulation example. Finally Section V concludes the article.

II. PROBLEM DESCRIPTION AND SOME PRELIMINARIES

A. Graph Theory

The undirected connected graph G =(V,χ,A) is modeled to describe the communication network among agents, where V = {v 1 ,v 2 ,...,v n }, χ ⊆V×V, and A =[a ij ] represent vertex set, edge set, and adjacency matrix, respectively. If there is an information flow from v j to v i , then vij =(v i ,v j ) ∈ χ and a ij =1, and v j is a neighbor of v i ; otherwise, vij / ∈ χ and a ij = 0. The neighbor node set is denoted by

N i = {v j | (v i ,v j ) ∈ χ}.
If and only if a ij = a ji , G is an undirected graph. If any pair of distinct vertices has a path, the undirected graph is connected, where a path from v i to v j is an edge sequence in the form of

(v i ,v i1 ), (v i1 ,v i2 ),...,(v il ,v j ).
L = D-A∈R n×n and B =diag{b 1 ,b 2 ,...,b n } denote the Laplacian matrix and the communication matrix between agent and leader, respectively, where D =diag{ n j=1 a 1j ,..., n j=1 a nj }. Assume that at least one agent is connected to the leader, which means n i=1 b i > 0.

B. Fuzzy Logic Systems

IF-THEN rules:

Rule l: IF x 1 is A l 1 , x 2 is A l 2 ,..., x k is A l k THEN: y B is B l .
where x =[x 1 ,...,x k ] T ∈ R k and y B denote the input and output of l represents the number of fuzzy rules. μ A l i (x i ) and μ B l (y B ) denote the membership functions of fuzzy sets A l i and B l , respectively.

Combining the singleton fuzzifier, product inference, and center average defuzzifier, y B can be expressed as

y B (x)= L l=1 Φ l k i=1 μ A l i (x i ) L l=1 k i=1 μ A (l) i (x i )
where L denotes the total number of fuzzy rules, Φ l satisfies μ B l (Φ l )=max{μ B l (y B )|y B ∈ R}.

Let

ϕ l = k i=1 μ A l i (x i ) L l=1 k i=1 μ A (l) i (x i ) and ϕ =[ϕ 1 ,...,ϕ L ] T and Φ=[Φ 1 ,...,Φ L ] T . Then, y B (x)
is represented as

y B (x)=Φ T ϕ(x).
Lemma 1 (see [START_REF] Li | Fuzzy adaptive optimized leader-following formation control for second-order stochastic multi-agent systems[END_REF]): For ∀ε>0, there exists FLS Φ T ϕ(x) such that sup x∈Ξ |f (x) -Φ T ϕ(x)|≤ε, where f (x) denotes a continuous function defined on a compact set Ξ.

C. Problem Description

Consider the following multiagent system (MAS):

ẋi = u i + f i (x i ),i=1,...,n (1)
where Define the coordinate transformations as

x i =[x i1 ,...,x im ] T ∈ R m , u i =[u i1 ,...,u im ] T ∈ R m and f i (•) ∈ R m represent
z i = x i -x d -ǫ i (2)
where z i =[z i1 ,...,z im ] T ∈ R m denotes the tracking error; ǫ i =[ǫ i1 ,...,ǫ im ] T ∈ R m denotes the relative position between the leader and the agent i, which describes the formation mode.

Remark 1: x d and ẋd represent the desired formation position and speed, respectively. In actual formation control, the given formation position and speed must be bounded. Otherwise, the formation would not be feasible. Assumption 1 is usually used in formation control, such as [START_REF] Wen | Optimized multi-agent formation control based on an identifier-actor-critic reinforcement learning algorithm[END_REF] and [START_REF] Wen | Optimized formation control using simplified reinforcement learning for a class of multiagent systems with unknown dynamics[END_REF].

The formation error is defined as

e i = j∈N i a ij (x i -ǫ i -x j + ǫ j )+b i (x i -x d -ǫ i ) = j∈N i a ij (z i -z j )+b i z i (3)
where e i =[e i1 ,...,e im ] T ∈ R m . Definition 1: The practical prescribed-time formation control of MAS (1) can be realized if there exists protocol u i such that e i < ε i for ∀t>T, where T>0 is a settling time and

ε i =[ε i1 ,ε i2 ,...,ε im ] T ∈ R m denotes the desired accuracy.
Define the performance index function as follows:

J i (e i )= ∞ t e -α i (τ -t) (e T i Q i e i + u T i R i u i )dτ (4)
where α i > 0 represents the discount factor, and Q i and R i denote the symmetric positive-definite matrices. Definition 2 (see [START_REF] Beard | Galerkin approximations of the generalized Hamilton-Jacobi-Bellman equation[END_REF]): For MAS (1),if1) u i is continuous; 2) u i (0) = 0; 3) u i stabilizes (1) on the set Ω; 4) J i (e i ) is finite, then the control protocol u i is admissible on Ω, which is represented as

u i ∈ Ψ(Ω).
Remark 2: To obtain the optimal formation control protocol u * so that MAS (1) can achieve the desired formation, the performance index function includes the formation error cost and the control input cost is considered in [START_REF] Zhang | Adaptive time-varying formation tracking control for multiagent systems with nonzero leader input by intermittent communications[END_REF].

The tasks of this article are to present a prescribed-time control algorithm for MAS (1) such that 1) the formation error (3) converges to the desired accuracy within a prescribed time; 2) the performance index function ( 4) can be minimized.

D. Necessary Lemmas

Lemma 2 (see [START_REF] Wen | Neural networkbased adaptive leader-following consensus control for a class of nonlinear multiagent state-delay systems[END_REF]): The Laplacian matrix of the undirected connected graph is irreducible.

Lemma 3 (see [START_REF] Wen | Optimized formation control using simplified reinforcement learning for a class of multiagent systems with unknown dynamics[END_REF]): If the Laplacian matrix is irreducible, then L + B is a positive-definite matrix.

Lemma 4 (see [START_REF] Krstic | Nonlinear and Adaptive Control Design (Adaptive and Learning Systems for Signal Processing, Communications and Control Series)[END_REF]): For ı, j ∈ R, one has

ıj ≤ m p p ı p + 1 qm q j q (5)
where m>0, and p, q > 1 with (p -1)(q -1) = 1.

III. PRESCRIBED-TIME FORMATION CONTROL

A. Error Transformation Function

To realize the prescribed-time control performance of MAS (1), an auxiliary function is given as follows:

ζ = (1 -τ ) T -t T 2 + τ, 0 ≤ t<T τ, t ≥ T (6)
where 0 <τ <1 represents the design parameter, and T>0 denotes the prescribed time. The following properties are obtained from ( 6

ξ i = [ξ i1 ,...,ξ im ] T ∈ R m . ξ iℓ is constructed as follows: ξ iℓ =tan π 2 e iℓ ℘ζ ,ℓ=1,...,m (7)
where ℘>0 is a constant, and |e iℓ (0)| <℘ζ(0). Remark 3: For ∀t ≥ 0,ifξ iℓ is bounded, it can be observed that -℘ζ < e iℓ <℘ζ from [START_REF] Yu | Neural adaptive distributed formation control of nonlinear multi-UAVs with unmodeled dynamics[END_REF]. From the definition of ζ,i t further indicates that as long as ξ iℓ is made bounded, -℘τ < e iℓ <℘τ for ∀t ≥ T , that is, the prescribed performance of e i can be realized.

B. Optimal Formation Control

From [START_REF] Yu | Neural adaptive distributed formation control of nonlinear multi-UAVs with unmodeled dynamics[END_REF], we can obtain that e iℓ = 2℘ζ π arctan(ξ iℓ ),i=1,...,n,ℓ =1,...,m.

Then, e i can be further expressed as e i =[ 2℘ζ π arctan(ξ i1 ), ..., 2℘ζ π arctan(ξ im )] T . Let e i = 2℘ζ π υ i with υ i =[υ i1 ,...,υ im ] = [arctan(ξ i1 ), ...,arctan(ξ im )] T . The performance index function (4) can be represented as

J i = ∞ t e -α i (τ -t) 2℘ζ π υ i T Q i 2℘ζ π υ i + u T i R i u i dτ. ( 9
)
Remark 4: Through the transformation function [START_REF] Yu | Neural adaptive distributed formation control of nonlinear multi-UAVs with unmodeled dynamics[END_REF], the error e iℓ is rewritten as [START_REF] Yu | Distributed adaptive fuzzy formation control of uncertain multiple unmanned aerial vehicles with actuator faults and switching topologies[END_REF] and embedded into the performance index function (4). This operation can ensure that the performance index function includes formation error cost and control input energy cost while achieving the prescribed-time formation control performance of MAS [START_REF] Sharma | Tracking control of mobile robots in formation in the presence of disturbances[END_REF].

Taking the optimal formation control u * =[u * T 1 ,...,u * T n ] T ∈ R nm into (9), we can get the optimal performance index function as follows:

J * i =m i n u i ∈Ψ(Ω) ∞ t e -α i (τ -t) 2℘ζ π υ i T Q i 2℘ζ π υ i + u T i R i u i dτ = ∞ t e -α i (τ -t) 2℘ζ π υ i T Q i 2℘ζ π υ i + u * T i R i u * i dτ. (10)
From ( 2) and (3), taking the time derivative of e i yields

ėi = c i f i + c i u i -b i ẋd - j∈N i a ij ẋj (11)
where

c i = j∈N i a ij + b i .
Then, taking the time derivative of υ iℓ yields

υiℓ = ξiℓ 1+ξ 2 iℓ = 1 1+ξ 2 iℓ K iℓ (ė iℓ -γ iℓ ) (12)
where

K iℓ = π 2℘ζ cos 2 ( πe iℓ 2℘ζ ) and γ iℓ = e iℓ ζ ζ .
From [START_REF] Zhang | Optimal DoS attack scheduling in wireless networked control system[END_REF] and [START_REF] Bellman | Dynamic Programming[END_REF],wehave

υi = Ki (ė i -γ i ) = Ki ⎛ ⎝ c i f i + c i u i -b i ẋd - j∈N i a ij ẋj -γ i ⎞ ⎠ ( 13
)
where Ki =diag

K i1 1+ξ 2 1ℓ ,..., K im 1+ξ 2 mℓ
, and

γ i =[γ i1 ,..., γ im ] T .
By calculating the time derivative of (10), we can get HJB equation as follows:

H i υ i ,ζ,u * i , ∂J * i ∂υ i , ∂J * i ∂ζ = 2℘ζ π υ i T Q i 2℘ζ π υ i + u * T i R i u * i -α i J * i + ∂J * i ∂υ T i × ⎡ ⎣ Ki ⎛ ⎝ c i f i + c i u * i -b i ẋd - j∈N i a ij ẋj -γ i ⎞ ⎠ ⎤ ⎦ + ∂J * i ∂ζ dζ dt = 2℘ζ π υ i T Q i 2℘ζ π υ i + u * T i R i u * i -α i J * i + ∂J * i ∂ξ T i ⎡ ⎣ K i × ⎛ ⎝ c i f i + c i u * i -b i ẋd - j∈N i a ij ẋj -γ i ⎞ ⎠ ⎤ ⎦ + ∂J * i ∂ζ dζ dt =0 (14)
where K i =diag{K i1 ,...,K im }. The right-hand side of ( 10) is existent and unique, it can be known that u * i is the unique solution of [START_REF] Lewis | Optimal Control,3 r de d[END_REF]. By solving

∂H i υ i ,ζ,u * i , ∂J * i ∂υ i , ∂J * i ∂ζ ∂u * i =0
, one can get the optimal control input as follows:

u * i = - c i 2 R -1 i K T i ∂J * i ∂ξ i . (15)
Substituting ( 15) into (14) yields

2℘ζ π υ i T Q i 2℘ζ π υ i -α i J * i + ∂J * i ∂ζ dζ dt + ∂J * i ∂ξ T i ⎡ ⎣ K i ⎛ ⎝ c i f i -b i ẋd - j∈N i a ij ẋj -γ i ⎞ ⎠ ⎤ ⎦ - c 2 i 4 ∂J * i ∂ξ T i K i R -1 i K T i ∂J * i ∂ξ i =0. ( 16
)
Remark 5: It is expected to get the term ∂J * i ∂ξ i by solving [START_REF] Wen | Optimized backstepping consensus control using reinforcement learning for a class of nonlinear strict-feedbackdynamic multi-agent systems[END_REF]. Due to that the HJB equation ( 16) contains completely unknown system dynamics, it is difficult or even impossible to calculate its solution.

RL methods, as a powerful tool, will be used to address the optimal formation issue.

C. ACI Design

To achieve the optimal formation control,

∂J * i ∂ξ i is segmented as ∂J * i ∂ξ i = 2 i c i K -2 i ξ i + 2 c i K -2 i fi (X i )+ 1 c i K -2 i J 0 i (X i ) ( 17
)
where i > 3 4 is a design parameter,

J 0 i (X i )=-2 i ξ i -2 fi + c 2 i K 2 i dJ * i dξ i , and fi (X i )=R i K i (f i (x i ) -ẋd -μ -1 γ i ) with μ = λ max ( L), L = L + B, X i =[x i ,ξ i ,x d , ẋd ,ζ, ζ] T ,X i = [x i ,x d , ẋd ,ζ, ζ] T .
By substituting ( 17) into [START_REF] Wen | Optimized leader-follower consensus control using reinforcement learning for a class of second-order nonlinear multiagent systems[END_REF],wehave

u * i = -i R -1 i K -1 i ξ i -R -1 i K -1 i fi (X i ) - 1 2 R -1 i K -1 i J 0 i (X i ). (18)
The unknown terms fi (X i ) and J 0 i (X i ) are continuous, by Lemma 1,f o r∀ε fi ∈ R m and ∀ε i ∈ R m , there exist FLSs

Ψ * T fi ϕ fi (X i ) and Ψ * T i ϕ i (X i ) such that fi (X i )=Ψ * T fi ϕ fi (X i )+ε fi (X i ) (19)
J 0 i (X i )=Ψ * T i ϕ i (X i )+ε i (X i ) (20)
where Ψ * fi ∈ R p 1 ×m and Ψ * i ∈ R p 2 ×m denote the optimal parameter matrices; p 1 and p 2 denote the fuzzy rule numbers; ϕ fi ∈ R p 1 and ϕ i ∈ R p 2 denote the fuzzy basis function vectors; the approximation errors ε fi ∈ R m and ε i ∈ R m satisfy ε fi ≤δ fi and ε i ≤δ i ; δ fi and δ i are constants. Substituting ( 19) and ( 20) into ( 17) and ( 18), we get

∂J * i ∂ξ i = 2 i c i K -2 i ξ i + 2 c i K -2 i Ψ * T fi ϕ fi (X i ) + 2 c i K -2 i ε fi (X i )+ 1 c i K -2 i Ψ * T i ϕ i (X i ) + 1 c i K -2 i ε i (X i ) (21)
u * i = -i R -1 i K -1 i ξ i -R -1 i K -1 i Ψ * T fi ϕ fi (X i ) -R -1 i K -1 i ε fi (X i ) - 1 2 R -1 i K -1 i Ψ * T i ϕ i (X i ) - 1 2 R -1 i K -1 i ε i (X i ). (22)
Because Ψ * fi and Ψ * i are unknown, ( 21) and ( 22) are not available. To this end, we design the following identifier, critic, and actor.

First, to identify the unknown dynamic function, we design the identifier as follows:

fi (X i )= ΨT fi ϕ fi (X i ) (23)
where fi (X i ) and Ψfi ∈ R p 1 ×m denote the output of FLS and identifier parameter matrices, respectively. In addition, we design the identifier updating law as follows:

Ψfi = P i (μϕ fi (X i )ξ T i R -1 i -ω i Ψfi ) (24)
where ω i > 0 denotes a design parameter, and P i ∈ R p 1 ×p 1 denotes a positive-definite matrix. To evaluate the control performance, we design the following critic:

∂ Ĵ * i ∂ξ i = 2 i c i K -2 i ξ i + 2 c i K -2 i ΨT fi ϕ fi (X i ) + 1 c i K -2 i ΨT ci ϕ i (X i ) (25)
where ∂ Ĵ * i ∂ξ i and Ψci ∈ R p 2 ×m denotes the critic parameter matrix. Then, we design the following updating law for [START_REF] Xiao | Finite-time formation control for multi-agent systems[END_REF]:

Ψci = -η ci (ϕ i (X i )ϕ T i (X i )+jI p 2 ) Ψci (26)
where η ci > 0 and j>0 represent design parameters. To implement the control behavior, the actor is designed as follows:

u i = -i R -1 i K -1 i ξ i -R -1 i K -1 i ΨT fi ϕ fi (X i ) - 1 2 R -1 i K -1 i ΨT ai ϕ i (X i ) (27)
where i > 3 4 ; Ψai ∈ R p 2 ×m represents the actor parameter matrix. We construct the following actor updating law:

Ψai = -(ϕ i (X i )ϕ T i (X i )+jI p 2 )(η ai ( Ψai -Ψci )+η ci Ψci ) (28)
where η ai > 0 represents the actor design parameter. Remark 6: The actor updating laws in [START_REF] Wen | Optimized formation control using simplified reinforcement learning for a class of multiagent systems with unknown dynamics[END_REF] was constructed as Ψai = -ϕ i (X i )ϕ T i (X i )(η ai ( Ψai -Ψci )+η ci Ψci ).I ft h e term Ψai -Ψci falls on the value of zero eigenvectors of ϕ i (X i )ϕ T i (X i ), the training may be terminated. To this end, jI p 2 is introduced in (28) for sufficient training.

Remark 7: The actor u i is expected to satisfy 25) and ( 27) into ( 14), we can get the approximated HJB equation as follows:

H i (υ i ,ζ,u i , ∂ Ĵ * i ∂υ i , ∂ Ĵ * i ∂ζ ) -→ 0. Inserting (
H i (υ i ,ζ,u i , ∂ Ĵ * i ∂υ i , ∂ Ĵ * i ∂ζ ) = 2℘ζ π υ i T Q i 2℘ζ π υ i + i R -1 i K -1 i ξ i + 1 2 R -1 i K -1 i ΨT ai ϕ i (X i )+R -1 i K -1 i ΨT fi × ϕ fi (X i ) T R i i R -1 i K -1 i ξ i + 1 2 R -1 i K -1 i × ΨT ai ϕ i (X i )+R -1 i K -1 i ΨT fi ϕ fi (X i ) -α i J * i - 1 c i 2 i K -2 i ξ i + K -2 i ΨT ci ϕ i (X i )+2K -2 i ΨT fi × ϕ fi (X i ) ⎛ ⎝ K i ⎛ ⎝ c i i R -1 i K -1 i ξ i + c i R -1 i K -1 i × ΨT fi ϕ fi (X i )+ c i 2 R -1 i K -1 i ΨT ai ϕ i (X i ) -c i f i + b i ẋd + γ i + j∈N i a ij ẋj ⎞ ⎠ ⎞ ⎠ . If H i (υ i ,ζ,u i , ∂ Ĵ * i ∂υ i , ∂ Ĵ * i ∂ζ )=0
holds, since the HJB equation has the unique solution, then we have

∂H i υ i ,ζ,u i , ∂ Ĵ * i ∂υ i , ∂ Ĵ * i ∂ζ ∂ Ψai = 1 2 ϕ i (X i )ϕ T i (X i )( Ψai -Ψci )=0. Define Φ=Tr(( Ψai - Ψci ) T ( Ψai -Ψci ))
.From( 26) and ( 28), we can get

Φ=Tr ∂Φ ∂ Ψai Ψai + ∂Φ ∂ Ψci Ψci = Tr - ∂Φ ∂ Ψai (ϕ i (X i )ϕ T i (X i )+jI p 2 ) × (η ai ( Ψai -Ψci )+η ci Ψci )+η ci × ∂Φ ∂ Ψai (ϕ i (X i )ϕ T i (X i )+jI p 2 ) Ψci = - η ai 2 Tr ∂Φ ∂ Ψai ϕ i (X i )ϕ T i (X i ) ∂Φ ∂ Ψai ≤ 0. ( 29
)
The inequality ( 29) means that

∂H i υ i ,ζ,u i , ∂ Ĵ * i ∂υ i , ∂ Ĵ * i ∂ζ ∂ Ψai =0holds
by the designed RL updating laws ( 26) and [START_REF] Cheng | Fixed-time fault-tolerant formation control for a cooperative heterogeneous multiagent system with prescribed performance[END_REF]. Furthermore,

we get that H i (υ i ,ζ,u i , ∂ Ĵ * i ∂υ i , ∂ Ĵ * i ∂ζ ) -→ 0.

D. Main Results

Theorem 1: Consider MAS (1) under Assumption 1,i ft h e identifier in [START_REF] Zhang | Predefined-time adaptive fuzzy control for a class of nonlinear systems with output hysteresis[END_REF], the critic in [START_REF] Xiao | Finite-time formation control for multi-agent systems[END_REF], and the actor in [START_REF] Hua | Finite-time time-varying formation tracking for high-order multiagent systems with mismatched disturbances[END_REF] with the updating laws ( 24), [START_REF] Liu | Finite-time formation control for linear multiagent systems: A motion planning approach[END_REF], and ( 28) are employed, then the formation error converges to the desired accuracy within a prescribed time, where the design parameters i , η ai , and η ci satisfy i > 3 4 and η ai >η ci > η ai 2 >

µR -1 i 4 . Proof: Construct the Lyapunov function as follows:

V = 1 2 ξ T ξ + 1 2 n i=1 (Tr{ ΨT fi P -1 i Ψfi } + Tr{ ΨT ci Ψci } + Tr{ ΨT ai Ψai }) (30)
where 3), ( 7), ( 24), [START_REF] Liu | Finite-time formation control for linear multiagent systems: A motion planning approach[END_REF], and (28),wehave

ξ =[ξ T 1 ,...,ξ T n ] T ∈ R nm , Ψfi = Ψfi -Ψ * fi , Ψai = Ψai -Ψ * i , Ψci = Ψci -Ψ * i . From (
V = ξ T [K( L⊗I m )ż -Kγ] + n i=1 Tr{ ΨT fi (μR -1 i ϕ fi (X i )ξ T i -ω i Ψfi )} - n i=1 Tr{η ci ΨT ci (ϕ i (X i )ϕ T i (X i )+jI p 2 ) Ψci } - n i=1 Tr{ ΨT ai (ϕ i (X i )ϕ T i (X i )+jI p 2 )) × (η ai ( Ψai -Ψci )+η ci Ψci )} ≤ μ n i=1 ξ T i K i (f i + u i -ẋd ) - n i=1 ξ T i K i γ i + n i=1 Tr{ ΨT fi (μR -1 i ϕ fi (X i )ξ T i -ω i Ψfi )} - n i=1 Tr{η ci ΨT ci (ϕ i (X i )ϕ T i (X i )+jI p 2 ) Ψci } - n i=1 Tr{ ΨT ai (ϕ i (X i )ϕ T i (X i )+jI p 2 ) × (η ai ( Ψai -Ψci )+η ci Ψci )} (31)
where

K =diag{K 1 ,...,K n }∈R nm×nm ,z =[z T 1 ,..., z T n ] T ∈ R nm ,γ =[γ T 1 ,...,γ T n ]
T ∈ R nm and μ = λ max ( L). Substituting ( 19) and ( 27) into [START_REF] Zheng | NN-based fixed-time attitude tracking control for multiple unmanned aerial vehicles with nonlinear faults[END_REF],wehave

V ≤ n i=1 ξ T i (-μ i R -1 i ξ i -μR -1 i ΨT fi ϕ fi (X i ) + μR -1 i ε fi (X i ) - μ 2 R -1 i ΨT ai ϕ i (X i )) + n i=1 Tr{ ΨT fi (μR -1 i ϕ fi (X i )ξ T i -ω i Ψfi )} - n i=1 Tr{η ci ΨT ci (ϕ i (X i )ϕ T i (X i )+jI p 2 ) Ψci } - n i=1 Tr{ ΨT ai (ϕ i (X i )ϕ T i (X i )+jI p 2 ) × (η ai ( Ψai -Ψci )+η ci Ψci )} ≤ n i=1 ξ T i (-μ i R -1 i ξ i - μ 2 R -1 i ΨT ai ϕ i (X i ) + μR -1 i ε fi (X i )) - n i=1 (Tr{ω i ΨT fi Ψfi } - n i=1 Tr{η ci ΨT ci (ϕ i (X i )ϕ T i (X i )+jI p 2 ) Ψci } - n i=1 Tr{η ai ΨT ai (ϕ i (X i )ϕ T i (X i )+jI p 2 ) Ψai } + n i=1 (η ai -η ci )Tr{ ΨT ai (ϕ i (X i )ϕ T i (X i )+jI p 2 ) × Ψci }. (32)
By applying Cauchy-Schwartz inequality and Young's inequality, we have

μR -1 i ξ T i ε fi ≤ μ 2 R -1 i ξ i 2 + μ 2 R -1 i δ fi 2 (33)
- μ 2 R -1 i ξ T i ΨT ai ϕ i (X i ) ≤ μR -1 i 4 ξ i 2 + μ 4 R -1 i × Tr{ ΨT ai ϕ i (X i )ϕ T i (X i ) Ψai }. (34)
From Ψfi,ai,ci = Ψfi,ai,ci -Ψ * f i,i,i ,wehave

Tr{ ΨT fi Ψfi } = 1 2 Tr{ ΨT fi Ψfi + ΨT fi Ψfi -Ψ * T fi Ψ * fi } (35) Tr{ ΨT ci,ai (ϕ i (X i )ϕ T i (X i )+jI p 2 ) Ψci,ai } = 1 2 Tr{ ΨT ci,ai (ϕ i (X i )ϕ T i (X i )+jI p 2 ) Ψci,ai } + 1 2 Tr{ ΨT ci,ai (ϕ i (X i )ϕ T i (X i )+jI p 2 ) Ψci,ai } - 1 2 Tr{Ψ * T i (ϕ i (X i )ϕ T i (X i )+jI p 2 )Ψ * i } (36)
(η ai -η ci )Tr{ ΨT ai (ϕ i (X i )ϕ T i (X i )+jI p 2 ) Ψci } ≤ η ai -η ci 2 [Tr{ ΨT ai (ϕ i (X i )ϕ T i (X i )+jI p 2 ) Ψai } + Tr{ ΨT ci (ϕ i (X i )ϕ T i (X i )+jI p 2 ) Ψci }]. (37)
Substituting ( 33)-( 37) into (32), we can get

V ≤- n i=1 μR -1 i i - 3 4 ξ i 2 - n i=1 η ci 2 Tr{ ΨT ci × (ϕ i (X i )ϕ T i (X i )+jI p 2 ) Ψci }- n i=1 η ci 2 × Tr{ ΨT ai (ϕ i (X i )ϕ T i (X i )+jI p 2 ) Ψai } - n i=1 Tr ω i 2λ max (P -1 i ) ΨT fi P -1 i Ψfi - n i=1 η ci - η ai 2 Tr{ ΨT ci (ϕ i (X i )ϕ T i (X i ) + jI p 2 ) Ψci }- n i=1 η ai 2 - μR -1 i 4 Tr{ ΨT ai × ϕ i (X i )ϕ T i (X i ) Ψai }- n i=1 η ai 2 Tr{ ΨT ai × jI p 2 Ψai }- n i=1 ω i 2 Tr{ ΨT fi Ψfi } + ̺ (38)
where λ max (P -1 i ) represents the maximal eigenvalue of the matrix

P -1 i , ̺ = n i=1 µ 2 R -1 i δ fi 2 + n i=1 ω i 2 Tr{Ψ * T fi Ψ * fi } + n i=1 η ai +η ci 2 Tr{Ψ * T i (ϕ i (X i )ϕ T i (X i )+jI p 2 )Ψ * i }. From µR -1 i 4 < η ai 2 <η ci <η ai , we can obtain that -n i=1 (η ci -η ai 2 )Tr{ ΨT ci (ϕ i (X i )ϕ T i (X i )+jI p 2 ) Ψci }-n i=1 ( η ai 2 - µR -1 i 4 )Tr{ ΨT ai ϕ i (X i )ϕ T i (X i ) Ψai }-n i=1 η ai 2 Tr { ΨT ai jI p 2 Ψai }-n i=1 ω i 2 Tr{ ΨT fi Ψfi } < 0.
Then, the inequality (38) can be rewritten as

V ≤- n i=1 μR -1 i ( i - 3 4
)

ξ i 2 - n i=1 η ci 2 Tr{ ΨT ci × (ϕ i (X i )ϕ T i (X i )+jI p 2 ) Ψci }- n i=1 η ci 2 × Tr{ ΨT ai (ϕ i (X i )ϕ T i (X i )+jI p 2 ) Ψai } - n i=1 Tr ω i 2λ max (P -1 i ) ΨT fi P -1 i Ψfi + ̺. (39)
There exists a constant ̺ such that ̺ ≤ ̺. Then, we can get that

V ≤- ν 1 2 n i=1 ξ i 2 - ν 2 2 n i=1 Tr{ ΨT fi P -1 i Ψfi } - ν 3 2 n i=1
Tr{ ΨT ci Ψci }-

ν 3 2 n i=1 Tr{ ΨT ai Ψai } +̺ ≤-νV +̺ (40)
where

ν 1 =min i=1,...,n 2 μR -1 i i - 3µR -1 i 4 ,ν 2 = min i=1,...,n ω i λ max (P -1 i ) ,ν 3 =min i=1,...,n {η ci λ min (ϕ i )}, ν =min{ν 1 ,ν 2 ,ν 3 }. λ min (ϕ i ) represents the minimal eigenvalue of matrix ϕ i (X i )ϕ T i (X i ). From (40), we can get that V ∈ L ∞ . Then, ξ i ∈ L ∞ , Ψfi ∈ L ∞ , ΨT
ci ∈ L ∞ , and ΨT ai ∈ L ∞ can be obtained. As ξ i is bounded, it follows from (7) that -℘ζ < e iℓ <℘ζ.Duetoζ = τ for ∀t ≥ T , we get that j∈N i a ij (z iz j )+b i z i < ℘ for ∀t ≥ T with ℘ =[℘τ,...,℘τ] T ∈ R m , and then, the expected formation can be realized. It can be further obtained that the formation error can converge to the specified accuracy within a prescribed time.

Remark 8: The performance index functions specified in [START_REF] Wen | Optimized leader-follower consensus control using reinforcement learning for a class of second-order nonlinear multiagent systems[END_REF], [START_REF] Wen | Optimized backstepping consensus control using reinforcement learning for a class of nonlinear strict-feedbackdynamic multi-agent systems[END_REF], [START_REF] Li | Fuzzy adaptive optimized leader-following formation control for second-order stochastic multi-agent systems[END_REF], [START_REF] Lan | Time-varying optimal formation control for second-order multiagent systems based on neural network observer and reinforcement learning[END_REF], [START_REF] Wen | Optimized adaptive nonlinear tracking control using actor-critic reinforcement learning strategy[END_REF], [START_REF] Wen | Optimized multi-agent formation control based on an identifier-actor-critic reinforcement learning algorithm[END_REF], and [START_REF] Wen | Optimized formation control using simplified reinforcement learning for a class of multiagent systems with unknown dynamics[END_REF] did not involve performance weights while the results presented in this article can make a tradeoff between the formation error cost and control input energy cost by choosing appropriate performance weights. By choosing R i , which is smaller than Q i , the formation error can converge to the desired accuracy with a faster convergence speed. Compared with the results in [START_REF] Wen | Optimized leader-follower consensus control using reinforcement learning for a class of second-order nonlinear multiagent systems[END_REF], [START_REF] Wen | Optimized backstepping consensus control using reinforcement learning for a class of nonlinear strict-feedbackdynamic multi-agent systems[END_REF], [START_REF] Li | Fuzzy adaptive optimized leader-following formation control for second-order stochastic multi-agent systems[END_REF], [START_REF] Lan | Time-varying optimal formation control for second-order multiagent systems based on neural network observer and reinforcement learning[END_REF], [START_REF] Wen | Optimized adaptive nonlinear tracking control using actor-critic reinforcement learning strategy[END_REF], [START_REF] Wen | Optimized multi-agent formation control based on an identifier-actor-critic reinforcement learning algorithm[END_REF], and [START_REF] Wen | Optimized formation control using simplified reinforcement learning for a class of multiagent systems with unknown dynamics[END_REF], the presented control scheme is more conducive to engineering application.

Remark 9: It should be mentioned that the following knowledge on MAS (1) is required: the number of agents; the general connection between agents; the formation information loaded in MAS [START_REF] Sharma | Tracking control of mobile robots in formation in the presence of disturbances[END_REF].

Remark 10: It is obvious that the controller parameter i should be selected under the condition that i > 3 4 . In practical engineering, the Laplacian matrix L, the communication matrix B, and the weight matrix R i can be determined in advance, that is, the value of 2 . The block diagram of MAS is given in Fig. 1. Moreover, the proposed design procedure is given in Algorithm 1.

IV. SIMULATION EXAMPLE

In this section, to verify the feasibility of the scheme, the following MAS is considered: Algorithm 1: Prescribed-Time Formation Control via Fuzzy RL.

ẋi = -ι i x i - 0.3x i1 cos 2 (κ i x i1 ) x i2 -0.5sin 2 (κ i x i2 ) + u i i =1, 2, 3, 4
Input: x d ,ǫ i , A, D, B, L,T,0 <τ <1,R i > 0,μ= λ max ( L) Initialization: x i (0),x d (0), Ψfi (0), Ψci (0), Ψai (0) Parameters: ℘>0,P i > 0,ω i > 0,η ci > 0,η ai > 0,j>0, i >

[ι 1 ,ι 2 ,ι 3 ,ι 4 ] T =[-0.3, 0.1, -0.3, 0.1] T , [κ 1 ,κ 2 ,κ 3 , κ 4 ] T =[0.5, 0.4, -5.5, -1.5] T .
The initial positions and the desired trajectories are set as

x 1 (0) = [5, 5] T ,x 2 (0) = [-5, 4] T ,x 3 (0) = [5, -3] T ,x 4 (0) = [-4, - 5
] T , and x d =[6sin(0.5t), 20 7 sin(0.7t)] T with x d (0) = [0, 0] T , respectively. The formation pattern is described by ǫ

1 =[4, 4] T ,ǫ 2 =[-4, 4] T ,ǫ 3 =[4, -4] T ,ǫ 4 =[-4, -4] T .
The settling time is specified as T =3, and parameters are selected as τ =0.06 and ℘ =10.

The adjacency matrix is selected as The fuzzy sets of FLS Ψ * T fi ϕ fi (X i ) are defined over the interval [- 23 2 , 23 2 ]. Define the vector X i as X i =[x i ,x d , ẋd ,ζ, ζ] T and for l =1, 2,...,24, define Thus, the fuzzy membership functions of FLS Ψ * T fi ϕ fi (X i ) are given as

A = ⎡ ⎢ ⎢ ⎣ 0100
X 0 i = ⎡ ⎢ ⎢ ⎣ - 25 2 + l, - 25
μ i F l (X i )=exp - (X i -X 0 i ) T (X i -X 0 i ) 2 . ( 41
)
Then, one can obtain the fuzzy basis function vector as ϕ fi (

X i )=[ϕ 1 fi (X i ),ϕ 2 fi (X i ),...,ϕ 24 fi (X i )],
where

ϕ l fi (X i )= μ i F l 24 l=1 μ i F l .
The fuzzy sets of FLS Ψ * T i ϕ i (X i ) are defined over the interval [- 11 2 , 11 2 ]. Define the vector X i as

X i =[x i ,ξ i ,x d , ẋd ,ζ, ζ] T and for ℓ =1, 2,...,12, define X 0 i = ⎡ ⎢ ⎢ ⎣ - 13 2 + ℓ, - 13 2 + ℓ
T ,..., - 13 2 + ℓ, - 13 2 + ℓ
T 6 ⎤ ⎥ ⎥ ⎦ T .
Thus, the fuzzy membership functions of FLS Ψ * T i ϕ i (X i ) are given as

μ i F ℓ (X i )=exp - (X i -X 0 i ) T (X i -X 0 i ) 2 .
Then, one can get the fuzzy basis function vector as

ϕ i (X i )=[ϕ 1 i (X i ),ϕ 2 i (X i ),...,ϕ 12 i (X i )], where ϕ ℓ i (X i )= μ i F ℓ 12 ℓ=1 μ i F ℓ .
For identifier updating laws [START_REF] Zhou | Finite-time adaptive fuzzy prescribed performance formation control for high-order nonlinear multi-agent systems based on event-triggered mechanism[END_REF], critic updating laws [START_REF] Liu | Finite-time formation control for linear multiagent systems: A motion planning approach[END_REF], and actor updating laws [START_REF] Cheng | Fixed-time fault-tolerant formation control for a cooperative heterogeneous multiagent system with prescribed performance[END_REF] strategy. Fig. 3 shows the curves of errors. As observed from Fig. 3 that the errors z i converge to the expected accuracy within 3(s).F i g s .4-6 show the norm of the identifier, critic, and actor parameter matrices. From Figs. 2 to 6, we can see that the presented method can realize the control goal. The comparison results between this article and the scheme in [START_REF] Wen | Optimized multi-agent formation control based on an identifier-actor-critic reinforcement learning algorithm[END_REF] are displayed in Figs. 789.Fig. 7(a) and (b) shows the formation of agents under the proposed scheme and formation control [START_REF] Wen | Optimized multi-agent formation control based on an identifier-actor-critic reinforcement learning algorithm[END_REF], respectively. Fig. 8(a) and (b) shows formation errors z i1 under the proposed scheme and formation control [START_REF] Wen | Optimized multi-agent formation control based on an identifier-actor-critic reinforcement learning algorithm[END_REF], respectively. Fig. 9(a) and (b) shows formation errors z i2 under the proposed scheme and formation control [START_REF] Wen | Optimized multi-agent formation control based on an identifier-actor-critic reinforcement learning algorithm[END_REF], respectively. The proposed scheme shows higher convergence accuracy than that in [START_REF] Wen | Optimized multi-agent formation control based on an identifier-actor-critic reinforcement learning algorithm[END_REF].

V. C ONCLUSION

In this article, the prescribed-time formation control problem was investigated for a class of MASs. First, a new auxiliary function was given, and then, a new performance index function considering formation error cost and control input energy cost was constructed. Then, by developing the RL algorithm with ACI structure, an optimal prescribed-time control approach was presented, which can minimize the constructed performance index function. In addition, the formation can be achieved, and the settling time can be prespecified as required. The validity of the presented control strategy was verified by an example. Future work will focus on the prescribed-time formation control for MASs subjected to DoS attacks via RL.

  ):1)ζ is strictly decreasing over [0,T), with ζ(0) = 1, and ζ = τ for ∀t ≥ T ;2)ζ is smooth, and ζ and ζ are bounded for all t ≥ 0.Define the error transformation function as

µR - 1 i 2 can 1 i 2 .

 1212 be obtained in advance. Then, the selection of η ai should satisfy η ai > µR -Finally, the selection of η ci should satisfy η ai >η ci > η ai

Fig. 1 .

 1 Fig. 1. Block diagram of MAS via RL.

  matrix is set as B =diag{1, 0, 0, 0}.

24 }.

 24 Fig.2that the formation can be achieved by the presented

Fig. 2 .

 2 Fig. 2. Formation trajectories of agents.

Fig. 3 .

 3 Fig. 3. Trajectories of formation errors z i1 and z i2 .

Fig. 4 .

 4 Fig. 4. Trajectories of Ψfi ,i =1, 2, 3, 4.

Fig. 5 .

 5 Fig. 5. Trajectories of Ψai ,i =1, 2, 3, 4.

Fig. 6 .

 6 Fig. 6. Trajectories of Ψci ,i =1, 2, 3, 4.

Fig. 7 .

 7 Fig. 7. Formation trajectories of agents.

Fig. 8 .

 8 Fig. 8. Trajectories of formation errors z i1 , i =1, 2, 3, 4.

Fig. 9 .

 9 Fig. 9. Trajectories of formation errors z i2 , i =1, 2, 3, 4.

  The desired formation trajectory x d ∈ R m and its derivative ẋd ∈ R m are bounded.

	the system state, control input,
	and unknown continuous nonlinear function, respectively.
	Assumption 1:

  3 4 , where η ci and η ai should satisfy η ai >η ci > η ai 2 > T , define X 0 i , determine fuzzy membership functions μ i F l (X i ), and then determine fuzzy basis function vector ϕ fi (X i ) Define fuzzy IF-THEN rules, inputX i =[x i ,ξ i ,x d , ẋd ,ζ, ζ] T , define X 0

		µR -1 i
	for follower agent i do	4
	Compute e i with (3)
	Compute ζ with (6)
	Compute ξ i with (7)
	Define fuzzy IF-THEN rules, input
	X i , determine fuzzy membership functions μ i F ℓ (X i ), and then determine fuzzy basis function vector ϕ i (X i )
	Compute u i with (27) Update Ψfi with (24)
	Update Ψci with (26) Update Ψai with (28)
	end for
	where

i =[x i ,x d , ẋd ,ζ, ζ]

This work was supported in part by the National Natural Science Foundation of China under Grant 61873128 and Grant 62273210, in part by the Postgraduate Research and Practice Innovation Program of Jiangsu Province, and in part by the Shandong Provincial Natural Science Foundation under Grant ZR2021MF052 and Grant ZR2020MF062.


published or not. The documents may come   L'archive ouverte pluridisciplinaire