Antonio Franchi

Decision-Making for Human-Robot Collaboration Systems Human-Collaborative Schemes in the Motion Control of Single and Multiple Mobile Robots

In this chapter we show and compare several representative examples of human-collaborative schemes in the control of mobile robots, with a particular emphasis on the aerial robot case. We first provide a simplified yet descriptive model of the robot and its interactions. We then use this model to define a taxonomy that highlights the main aspects of these collaboration schemes, such as: the physical domain of the robots, the degree of autonomy, the force interaction with the operator (e.g., the unilateral versus the bilateral haptic shared control), the near-operation versus the teleoperation, the contact-free versus the physically interactive situation, the use of onboard sensors, and the presence of a time-horizon in the operator reference. We then specialize the proposed taxonomy to the multi-robot case in which we further distinguish the methods depending on their level of centralization, the presence of leader-follower schemes, of formation control schemes, the ability to preserve graph theoretical properties, and to perform cooperative physical interaction. The common denominator of all the examples presented in this chapter is the presence of a human operator in the control loop. The main goal of the chapter is to introduce the reader and provide a firstlevel analysis on the several ways to effectively include human operators in the control of both single and multiple aerial robots and, by extension, of more generic mobile robots.

I. INTRODUCTION

A mobile robot is primarily a machine that is able to perceive the surrounding environment and move in it in a safe and effective way for the humans, for itself, and for the given objective. The ability to automatically control its own motion, at both the higher level (guidance, planning) and the lower one (navigation and servoing), constitutes one of the fundamental building blocks of the sought robot autonomy. However, robots, which in general do not exist and operate isolated from humans, can actually improve their effectiveness in achieving the given task by means of a welldesigned human-collaborative control schemes. Analyzing and properly designing the way to achieve an optimal human collaboration is therefore a crucial aspect in order to attain a fully mature robotic system that is able to operate in a complex and real world.

In this chapter we show and compare several ways to effectively design the collaborative control. We do so by first introducing a simple yet descriptive model for the robots and the human interfaces. We then propose a list of what we consider fundamental axes for a proper classification of the collaborative scheme, namely the

• physical domain of the robots as, e.g,, whether they are operating and moving on the ground or in the air;

1 LAAS-CNRS, Université de Toulouse, CNRS, Toulouse, France, afranchi@laas.fr

• degree of autonomy from the human operator, i.e., at which level and with which frequency the human operator is involved in influencing the robot motion, in particular, in this chapter we mainly focus on the so called 'shared control' case. Within this case we further specialize our taxonomy depending on the presence of a time-horizon in the reference that is sent by the operator to the robots. • kind force interaction with the operator, i.e., whether the collaboration involves or not mechanical interaction and haptics and in which direction (forward-force, feedbackforce (bilateral), unilateral); • near-operation versus the teleoperation;

• amount of physical interaction with the environment; • use of onboard sensors;

The proposed taxonomy is then further specialized in the case of the use of multiple robots, for which we introduce an additional axes, namely the • level of centralization; • use of a scheme of the kind master-leader-followers; • use of a scheme of the kind formation-orthogonal; • use of a scheme of the kind global property preservation; • ability to perform cooperative physical maneuvers.

The common denominator of all the axes presented in this chapter is the presence of a human operator in the control loop. In fact, fully autonomous control algorithms are not in general easy to extend in order to cope with (and to exploit) the presence of a human operator. The main goal of the chapter is therefore to introduce the reader to the several ways to effectively include human operators in the control of both single and multiple aerial mobile robots, and to provide a first-level analysis on these systems. A particular emphasis is set on the aerial robots case, which is at date not yet fully explored and understood, and is among the most challenging cases. Nevertheless, all the presented aerial robotics schemes are easily transferable to many other mobile robot as well.

II. MODELING OF THE ROBOT AND THE INTERACTIONS

In this section we provide a formal description of the robot and its interactions. The main goal is to introduce a standard model and a nomenclature that is convenient for the description of human-collaborative schemes like the ones that we illustrate in this chapter. For the sake of completeness, the notation introduced goes beyond the examples included in the taxonomy proposed in the next sections. x e w i w j , w e , w h Fig. 1: High-level model of the i-th mobile robot.

A. Mobile Robot

A mobile robot can be viewed as a mechanical system possessing some degrees of freedom (DoFs) that is equipped with actuators, sensors, a processing unit, a communication module, and a mathematical model of itself and the surrounding environment. The combination of all these components are used by the robot to control its own DoFs and, as a byproduct, to modify the state of the surrounding environment, in presence of external disturbances and uncertainties. We refer to Figure 1 for a high-level model of a mobile robot physically interacting and communicating with the surrounding environment.

The configuration of the n q ∈ N DoFs of the robot is represented by a vector q ∈ R n q . The physical model of the robots is implicitly represented by a differential equation

ẋ = f r (x, u, d) (1)
where x ∈ R n x is the physical state, u ∈ R n u is the control input (a signal that the processing unit can change arbitrarily and drives the robot actuators), and d ∈ R n d is the external input (a signal that the is out of control to the robot and can represent both the interaction with the environment and the inaccuracies of the model). The vector x includes the configuration q and may include also its derivative, i.e., x = (q T qT ) T . In presence of more than one robot we assume an index i ∈ {1, . . . , N} being associated to each robot, where N is the total number of robots. We shall refer to the robot associated to i as the 'i-th robot' or the 'robot i'. The quantities corresponding to the i-th robot are added the subscript i, e.g., x becomes x i , u becomes u i , and so on so forth.

The sensors of a robot typically provide a partial information about the physical state, the external input, and the surrounding environment. This information is in general represented by the measurement function

y = h(x, u, d, x e ) (2)
where y ∈ R n y is the measurement and x e ∈ R n xe is the environment state, which may include external objects, other robots as seen by the robot i, and a nearby human collaborator.

The communication module provides the robots additional signals that can be used by its processing unit to achieve their mission. A signal that comes from another robot j is denoted with w j ∈ R n w j , where j ∈ N i , being N i ⊆ {1, . . . , N} the set of robots that communicate with the robot i (communication neighbors). A signal that comes from the human-robot interface is instead denoted with w h ∈ R n w h . A signal could come also from other sources (like, e.g., a smart environment) in that case we denote it with w e ∈ R n we . The communication module is also in charge of disseminating to each robot j, with j ∈ N i , the signal w i coming from the robot processing unit.

The processing unit implements a robot control algorithm that chooses u in order to, typically, minimize an objective function

J(x, u, d, x e , {w j | j ∈ N i }, w h , w e ). (3)
In order to do so, the robot control algorithm can rely on the following information:

• a model of the robot (e.g., ( 1)) and a model of the environment • the measurement y • the communicated signals {w j | j ∈ N i }, w h , and w e , which constitute the input to the control algorithm. The output of the control algorithm is u, i.e., the control input to the actuators.

B. Communication Infrastructure

A network of robots is a group of robots whose processing units communicate through a communication infrastructure. The simplest way to model the overall communication state is to consider a graph G = {I , E }, where the set of vertexes I = {1, . . . , N} represents the robots in the network, and E ⊂ I × I is the set of edges for which (i, j) ∈ E if and only if j ∈ N i , where N i has been previously defined as the set of communication neighbors of the robot i.

A graph has an algebraic representation given by the adjacency and the incidence matrixes, denoted with A and E, respectively (see, e.g., [START_REF] Giordano | A passivity-based decentralized strategy for generalized connectivity maintenance[END_REF], [START_REF] Zelazo | Decentralized rigidity maintenance control with range measurements for multi-robot systems[END_REF] for the formal definitions of these and all the algebraic graph concepts mentioned in this chapter). Another matrix that plays an important role in the control of networks of robots is the Laplacian matrix L = E T E ∈ R N×N , which is always positive semidefinite, i.e., with all real and non-negative eigenvalues. A fundamental result in algebraic graph theory is that the multiplicity of the eigenvalue λ 1 = 0 of L is equal to number of connected components of G [START_REF] Fiedler | Algebraic connectivity of graphs[END_REF]. For this reason, the second smallest eigenvalue of L, typically denoted with λ 2 , is called algebraic connectivity, and λ 2 > 0 if and only if the graph is connected. Those definitions extend seamlessly to the case of weighted graphs, i.e., graphs in which a weight number is associated to each edge, representing, e.g., the quality of the communication. Algebraic graph theory plays a fundamental role in the control of networked robots, see, e.g., [START_REF] Mesbahi | Graph Theoretic Methods in Multiagent Networks[END_REF] for an introduction to this vast topic.

ẋd = f d (x d , u d , u h ) z d = h d (x d , u d , u h ) Human Communication module Processing Unit Physical part u d x d , u h z d w h w j , w e u h
Fig. 2: High-level model of the human-robot interface.

C. Human-robot Interface

A human-robot interface is an input-output device that allows a human operator to retrieve/send information from/to one or more robots at the same time. The interface is constituted by a physical part, which is able to record the human actions and to provide a sensorial feedback to the human, a processing unit, and a communication module. Examples of interfaces are joysticks, control pads, button pads, displays, movement recognition units, and haptic interfaces, just to mention a few. We refer the reader to Figure 2 for a highlevel model of a human-robot interface robot interacting with the human and communicating with the robots.

The physical part of the interface is implicitly modeled by a differential equation

ẋd = f d (x d , u d , u h ) (4)
where

x d ∈ R n x d , is the state of the interface, u d ∈ R n u d
is the control input of the interface (a signal that can be changed arbitrarily by the processing unit), and u h ∈ R n u h is the external input from the human operator (a signal that is out of control of the interface and represents the human input to the system). We do not include inaccuracies and disturbances in the interface model since human-robot interfaces are typically well-calibrated instruments that operate in a structured environment. For the same reason we assume that the internal state of the interface x d is fully measurable. The communication module feeds the interface with signals coming from a set of robots denoted with N l ⊆ {1, . . . , N} and called interface neighbors. We denote the signal coming from the robot l ∈ N d with w dl ∈ R n w d l . As in the previous case, additional signals coming from other sources (like, e.g., a smart environment) are denoted with w e ∈ R n we . Finally, the communication module is also in charge of sending the signal w h to the robots in N d .

The processing unit of the interface has two roles. The first role is to compute w h on the basis of x d so that the human can influence the motion control of the robots. The second role is to provide a feedback to the human operator in the form of, e.g., an image, a change of color, a beep sound, a force provided back to the operator in a haptic interface, and so on. Generically speaking, any kind of feedback can be modeled by a static function

z d = h d (x d , u d , u h ). (5)
The input u d is computed with the purpose of letting z d be informative of the communicated signals {w j | j ∈ N i }, w h , and w e .

III. A TAXONOMY OF COLLABORATIVE HUMAN-ROBOT CONTROL

In this section we present a taxonomy1 for the situation in which a human collaborates in the motion control of one or multiple mobile robots. We shall refer when needed to the terminology introduced in Section II. A particular emphasis will be given to the case of aerial robots, even though many results and considerations hold for any kind of mobile robot.

A. Physical Domain of the Robots

First of all, the collaborative human-robot control can be divided in several categories, corresponding to the physical domain of the robotic system, i.e., terrestrial (ground), aerial, space, marine, and underwater, just to mention the main ones. In the following we provide a quick review on the models used for the first two domains.

The models of ground robots stop typically at the kinematic level. An example is to consider x = (p T ψ) T ∈ SE(2) where p T and ψ are the position and orientation of the mobile robot, respectively. The simplest model can be represented as ẋ = u if the robot is omnidirectional and holonomic or as ẋ = G(x)u with rank(G) < 3 if the platform is underactuated and non-holonomic.

Aerial robots cannot be modeled just kinematically because the dynamic effects are non-negligible, due mainly to: 1) the larger speed of aerial robots when compared to ground robots, and 2) the impossibility to aerodynamically generate input forces that are as strong as the one generated using the motor-wheel actuators available in the ground robot case. Furthermore, in order to increase the energy efficiency, the mechanical simplicity and robustness, and the lightness of the platform, a low number of actuators are used, leading to underactuated robots. The most used aerial robots nowadays are multi-rotor platforms, such as, e.g., the quadrotor configuration [START_REF] Franchi | Shared control: Balancing autonomy and human assistance with a group of quadrotor UAVs[END_REF]. In this case the robot state is the one of a rigid body floating in space x = ((p, R), v, ω) ∈ SE(3) × R 3 × R 3 , where p, R, v, and ω represent the position, attitude, velocity, and angular velocity of the robot, respectively. The dynamical model is then the

following ṗ = v (6) v = f 1 (x) + G 1 (x)u (7) Ṙ = Rω (8) ω = f 2 (x) + G 2 (x)u. (9)
In most of the cases, e.g., for coplanar multirototors, the attitude dynamics (the last two equations) is fully actuated (i.e., rank(G 2 ) = 3), while the position dynamics (the first two equations) is under-actuated (typically rank(G 1 ) = 1). Furthermore there is a cascaded structure in which f 1 , G 1 , f 2 , and G 2 depend only on R and ω.

The increased range applications enabled by the use of aerial robots with respect to the exclusive use of ground robots, together with the larger complexity of aerial robots platforms made the design of collaborative human-aerialrobot control a particularly interest research domain. In this chapter we will therefore focus especially on the aerial robot case. However the proposed taxonomy and the large majority of presented methods apply for all the other domains as well.

B. Degree of Autonomy from the Human Operator

The collaborative human-robot control can be then roughly divided in three categories, corresponding to the degree of autonomy of the robotic system. This kind of classification, recalled in Table I, is similar to the one used in the teleoperation literature, see e.g., [START_REF] Niemeyer | Telerobotics[END_REF].

The separation between the three categories cannot be sharp. Typically, real situations show intermediate aspects between the three categories. In this chapter we will present works that belong mainly to shared control category, with some supervisory control aspects as well.

1) Direct Control: Direct control refers to the case in which the robot motion is completely (i.e., in all its DoFs) decided by the human operator. The role of the robot is only to reproduce the human operator motion with as much fidelity as possible. In this case the human-robot interface has the same number of DoFs of the robot (n x = n x d ), w h = x d , and the goal of the robot control u is to have

J = x -w h < ε (10)
with ε as small as possible.

2) Supervisory control: Supervisory control refers to the case in which the robot motion is mainly controlled by autonomous decisions of the robot, while the human operator is in charge of providing only high-level directives to be fulfilled. In this also case the human-robot interface has typically a much lower number of DoFs of the robot (n x > n x d ), and the goal of the robot control u is to minimize

J = h(x(t ), x e (t )) -w h (11)
where in this case h(x, x e (t )) is a projection map for both the robot and environment state and t are some particular time instants in which the sub-task should be achieved.

3) Shared Control: Shared control refers to the case in which the robot motion is determined by both the human operator and robot decisions in a mostly balanced fashion. In this case typically the human-robot interface has a lower number of DoFs of the robot (n x > n x d ), and the goal of the robot control u is to minimiize

J = h(x) -w h ( 12
)
where h(x) is a map that projects x on the lower-dimensional space controlled by the human operator. a) Time Horizon in the Operator Reference: Another important aspect in the collaborative control schemes that we are analyzing is the length of the time horizon associated to the reference signal (e.g., desired position or velocity) given by the operator to the robot. In the direct control case the length of time-horizon is intrinsically zero since the operator reference is instantaneous and refers to the 'now'. On the contrary in supervisory control schemes the timehorizon is typically very large and the reference command pertains to some time instant in the relative long-term future, as explained before in this chapter.

In the shared control scenario the presence of intermediate cases is possible. However, in the literature, the majority of the shared control works considers that the human operator is in charge of guiding the aerial robot during the task by specifying the current reference position or velocity, as e.g., in [START_REF] Schill | Admittance mode framework for haptic teleoperation of hovering vehicles with unlimited workspace[END_REF], [START_REF] Mersha | Switching-based mapping and control for haptic teleoperation of aerial robots[END_REF], [START_REF] Stegagno | A semiautonomous UAV platform for indoor remote operation with visual and haptic feedback[END_REF]. This persistent commitment can result rather demanding for the human operator. Furthermore instantaneous and persistent collaboration may be unnecessary in some parts of the task execution, such as in all the applications in which the robot has to follow a predefined path and the purpose of the operator is only to provide sporadic modification to the planned path in response to unpredicted changes of the external situation.

In the work presented in [START_REF] Masone | Semi-autonomous trajectory generation for mobile robots with integral haptic shared control[END_REF] (see Figure 3 for an illustration of this scheme) this consideration is taken into account and the collaboration is shifted directly at the planning level by letting the human operator control the planned path for a given future (and non-negligible) time horizon. First, the operator is in charge of modifying online the shape of a planned path with a haptic interface (e.g., by changing the location of some control points). Secondly the robot autonomously correct the modified path in order to meet additional requirements such as collision avoidance, dynamic limits etc. Finally, consistently with the spirit of feedback-force schemes, the haptic cue is computed based on the mismatch between the path modification requests asked by the operator and the autonomous modifications of the robot using an impedance-like algorithm. In our formalism, this kind of paradigm can be summarized by the following equations:

J = h(x [t,t+T ] ) -x d (13) u d = K(h(x [t,t+T ] ) -x d ) + f(x d ), . (14)
where x [t,t+T ] ) represents the future state of the robot in the time horizon T > 0, and x[t,t+T] is state re-planned by the robot. A interesting future direction in this sense would be to study task-dependent automatic adaptation of the length of the time-horizon in the operator reference.

4) Outside Collaborative Human-robot Control: Outside the two extremes of collaborative control there are two cases which do not involve collaboration, see Table I.

A robot is fully autonomous if the human operator has no influence on it, i.e., the signal w h is empty.

The antipodal case is when the human is in charge of steering all the DoFs by means of low-level actions, like while driving a (non-autonomous) car or operating a mechanical tool. In this case the machine is not a robotic system, because it lacks of any relevant autonomous aspect. The machine is said fully human-operated. In this case the human acts directly on u.

C. Force Interaction with the Operator

Another distinction can be made depending on the information value given to the mechanical exchange between the human operator and the interface.

A collaboration scheme is forward-force if the force exerted by the human operator is informative for the collaborative control. In other words, in a forward-force collaboration, the human input u h includes a measure (or an indirect estimation) of the force exerted by the human operator on the human-robot interface and this measure influences w h as well.

A collaboration scheme is feedback-force (or bilateral) if the human-robot interface exerts, towards the human operator, a force that conveys some information related to the collaborative control. In other words, in a feedbackforce collaboration the feedback to the human operator z d includes a force exerted by the interface toward the human operator. An interface that is not feedback-force (bilateral) is said unilateral.

Notice that:

• in this terminology, 'unilateral' means that there is no force feedback. Other kind of feedback signals (e.g., visual) may be still present in a unilateral scheme; • a feedback-force scheme does not necessarily imply that the scheme is also forward-force and, vice versa, a forward-force scheme could be unilateral; • the previous definitions can be extended by replacing force with mechanical quantities related to forces and, more in general, mechanical power such as, e.g., moments, stiffness/compliance, and friction.

Forward-force schemes include the so-called 'forceposition' or admittance-based schemes, in which the force provided by the human operator u h is recorded by a force sensor or estimated by an observer on the human interface and it is used to generate online a reference position and velocity that the robot tracks using a local position control loop. In our formalism this kind of scheme can be represented by

ẋref = f(x ref , u h ) (15) J = h(x) -x ref . ( 16
)
This kind of approach has been used e.g., in [START_REF] Schill | Admittance mode framework for haptic teleoperation of hovering vehicles with unlimited workspace[END_REF] where the authors propose an admittance control modality for controlling an aerial vehicle. The operator applies a force to a haptic interface and the interface responds with a corresponding displacement, at the same time the force applied by the operator is used to generate a reference for the vehicle controller.

The simplest feedback-force schemes are the so-called 'position-force' or impedance-based schemes. In these schemes the configuration in which a haptic device is set by the operator is used a reference position or velocity for the robot. The force feedback is then proportional to the mismatch between the actual position/velocity and the reference. In our formalism this kind of scheme can be represented by

J = h(x) -x d (17) u d = K(h(x) -x d ) + f(x d ), (18)
where K and f(x d ) are a gain matrix and a stabilizing term for the device, respectively. A seminal work in this field is [START_REF] Diolaiti | Teleoperation of a mobile robot through haptic feedback[END_REF], where a force feedback scheme based on virtual viscoelastic couplings is designed for controlling a ground mobile robot in an obstacle populated environment. Extension to the multiple groundrobot case has been considered, e.g., in [START_REF] Lee | Bilateral teleoperation of multiple cooperative robots over delayed communication network: application[END_REF]. Finally many applications and extensions of this concept to the aerial robot case can be found in the very recent literature, see, e.g., [START_REF] Stramigioli | A novel approach to haptic tele-operation of aerial robot vehicles[END_REF], [START_REF] Rodríguez-Seda | Bilateral teleoperation of multiple mobile agents: Coordinated motion and collision avoidance[END_REF], [START_REF] Franchi | A passivity-based decentralized approach for the bilateral teleoperation of a group of UAVs with switching topology[END_REF], [START_REF] Lee | Haptic teleoperation of multiple unmanned aerial vehicles over the internet[END_REF], [START_REF] Mersha | Switching-based mapping and control for haptic teleoperation of aerial robots[END_REF].

More advanced feedback force schemes include in the feedback the measurement of the force exerted by the robot on the environment or vice versa. In [START_REF] Hou | Dynamic kinesthetic boundary for haptic teleoperation of aerial robotic vehicles[END_REF] the authors propose a scheme that is both forward-force and feedback-force to control an aerial robot. The forward scheme is similar to an admittance-like one, while the feedback force is used to display obstacles using the concept of dynamic kinesthetic boundary. In [START_REF] Farkhatdinov | Improving mobile robot bilateral teleoperation by introducing variable force feedback gain[END_REF] a variable force-feedback gain is instead proposed.

Researcher efforts have also been directed toward the critical comparison between different schemes. In [START_REF] Hou | Comparative study of haptic interfaces for bilateral teleoperation of VTOL aerial robot[END_REF] a theoretical and experimental comparison between the admittanceand impedance-like schemes is proposed both from the perceptual and performance point of view. In [START_REF] Son | Human-centered design and evaluation of haptic cueing for teleoperation of multiple mobile robots[END_REF] the authors compare two different haptic cueing, namely the aforementioned reference mismatch versus the obstacle force cueing.

The main advantage of forward-force schemes is the availability of an additional intuitive information channel for the human operator in the forward channel. The main disadvantage of forward-force is the necessity to measure the human force, which is usually very noisy and also difficult to give a unique interpretation among the different human operators.

The main advantages of feedback-force schemes are:

• the additional channel for providing information to the operator, which hopefully results in a more immersive experience; • the fact that haptic feedback may result more immediate to interpret than visual feedback due to the longer cognitive pipeline related to the latter; • the fact that haptic feedback requires less bandwidth than vision (e.g., to feedback the distance from an obstacle), which in turns generates less delay than vision in long-distance internet based communication, see, e.g., [START_REF] Riedel | Experiments on intercontinental haptic control of multiple UAVs[END_REF].

The main disadvantages of feedback-force are:

• the fact that the force applied to the interface influences back its the motion in a way that might result involuntarily for the operator; • the increased difficult to ensure overall stability;

• the sensibility to delays; • the need to take into account ergonomics issues, which may increase operator fatigue. Stability issues in the bilateral human-collaborative control of mobile robots have been addressed in several ways, many of which related to passivity based control such as, e.g., the passivity observer/passivity controller (PO/PC) approach [START_REF] Hannaford | Time-domain passivity control of haptic interfaces[END_REF] the passive-set-position-modulation (PSPM) approach [START_REF] Lee | Passive-set-position-modulation framework for interactive robotic systems[END_REF], and the two-layer approach [START_REF] Franken | Bilateral telemanipulation with time delays: A two-layer approach combining passivity and transparency[END_REF].

D. Near-operation vs. Teleoperation

A human-robot collaborative control is a near-operation if the human operator perceives the scene related to the control task by direct sensing, e.g., by line-of-sight, sense of touch, etc. Conversely, the human-robot collaborative control is a teleoperation when the perception is made through indirect sensing, e.g., tele-vision, haptic interfaces, etc.

A teleoperation is delayed when the delays of the communication are non-negligible (due to, e.g., long distances or long processing times). As it is well known, the presence of delays in the teleoperation may have a dramatic influence on the stability of feedback-force schemes if not properly addressed, see, e.g., [START_REF] Niemeyer | Telerobotics[END_REF] and reference therein.

Large delays (usually greater than 0.15 s) between the operator and the robots are unavoidable in intercontinental collaborative control settings. In addition to stability, those delays generate also practical problems in the quality of video streaming, the safety and the ability to maneuver the robot. In fact, in long-distances communication, the video transmission has usually a larger delay than the reception of state of the remote robots because of the larger size of the packets of the video streaming.

Examples of intercontinental shared control of aerial robots are still very few at date. In [START_REF] Riedel | Experiments on intercontinental haptic control of multiple UAVs[END_REF] the authors show experiments of collaborative control for aerial robots with German-Korean intercontinental communication using a standard internet channel with average delay of 0.35 s (see Figure 4). In this work it is shown that the feedbackforce scheme can be made stable in presence of such large delay and packet losses by resorting to the PSPM passivity based technique [START_REF] Lee | Passive-set-position-modulation framework for interactive robotic systems[END_REF]. The benefit of using force feedback to detect obstacles in advance with respect to the more delayed visual feedback is also made clear through the experiments. Following the results provided in [START_REF] Riedel | Experiments on intercontinental haptic control of multiple UAVs[END_REF], other intercontinental schemes for human-collaborative control with aerial vehicles have been studied and experimentally validated, such as e.g., in [START_REF] Mersha | Intercontinental haptic teleoperation of a flying vehicle: A step towards real-time applications[END_REF].

A different, but still passivity based, technique is used in [START_REF] Secchi | Bilateral teleoperation of a group of UAVs with communication delays and switching topology[END_REF] to cope with delays in the collaborative force interaction control of aerial robots. In this case the control scheme resorts to the two-layer architecture, in which a passivity layer (whose goal is to preserve passivity, hence stability, of the interconnection) is built on top of the transparency layer, whose goal is instead to deteriorate the quality of the references and the force feedback as less as possible [START_REF] Franken | Bilateral telemanipulation with time delays: A two-layer approach combining passivity and transparency[END_REF].

E. Physical Interaction with the Environment

Another important distinction concerns the nature of physical interaction of the robot(s) with the environment. This represents a sort of dual aspect with respect to the force interaction on the human-operator side. Table II summarizes the tree cases described in the following.

When the robot interacts with a solid/impenetrable environment which can significantly constrain the robot DoFs then the physical interaction is with contact.

The contact-free physical interaction case is instead when the robot is subject to external physical disturbances that are not constraining its DoFs, as in the previous case, but are mostly acting as an external disturbance, such as e.g., in the presence of wind, magnetic field, etc.

We finally refer to the negligible physical interaction case when physical interaction is only part of the actuation mechanism and plays almost no role in constraining or disturbing the robot motion. This is the case, e.g., of a ground mobile robot moving freely on a flat surface with perfect wheel traction, and of a multi-rotor robot flying in absence of wind and away from other surfaces (walls, floors, ceilings, etc.) thus not experiencing any undesired aerodynamical effect. Note that a sort of 'physical interaction' is actually present in both the examples and essential for the functioning of the robot (i.e., the wheel traction needed to produce the moving force, and the aerodynamic flow needed to produce the drag and the thrust forces). Nevertheless, these interactions do not constitute an impediment to the robot motion or an external disturbance that has to be compensated by the controller.

The cases of both fixed-base manipulators and grounded mobile manipulators have been deeply studied in the past, see e.g., [START_REF] Niemeyer | Telerobotics[END_REF], [START_REF] Lee | Bilateral teleoperation of multiple cooperative robots over delayed communication network: theory[END_REF], [START_REF] Lee | Bilateral teleoperation of multiple cooperative robots over delayed communication network: application[END_REF] and references therein for an overview of the state of the art.

A mostly open problem is instead the case of collaborative control of aerial robots in contact with the environment. In fact, while the study of control strategies for aerial robots in physical interaction with contact has received a lot of attention in the recent years, a few works exist at date addressing the more challenging problem of having a human in the loop with, e.g., a feedback-force scheme. An example of these few works is [START_REF] Gioioso | A forcebased bilateral teleoperation framework for aerial robots in contact with the environment[END_REF] where a teleoperation scheme is presented that allows to convert a contact-free physical interaction controller into a controller for physical interaction with contact, see Figure 5. The approach is based on the aerial physical interaction control algorithm presented in [START_REF] Gioioso | Turning a near-hovering controlled quadrotor into a 3D force effector[END_REF], where using a rigid tool attached to the aerial vehicle the robot is able to exert a 3D force on the environment. Following the feedback-force paradigm, the algorithm in [START_REF] Gioioso | A forcebased bilateral teleoperation framework for aerial robots in contact with the environment[END_REF] provides the user with a force feedback that is proportional to the force measured by a force sensor at the tool-tip,

z d = d. ( 19
)
During the contact-free approaching phases the algorithm automatically slows down the speed of the robot in order to achieve a smooth passage from free flight to contact constrained motion as shown in the block diagram of Figure 6. This kind of variable/tunable autonomy scheme is a typical example of collaborative shared control with physical interaction extended to the more challenging aerial robotics case.

F. Use of Onboard Sensors Only

Of fundamental importance in robotics is the ability of the robot to be fully autonomous from the sensing point of view. With respect to this issue we can roughly distinguish TABLE II: Amount of physical interaction of the robot(s) with the environment Amount of physical interaction of the robot with the environment With contact Contact-free Negligible The external environment constrains the DoFs of the robot The forces produced by the external environment do not constrain the DoFs of the robot The disturbing/constraining external forces are negligible Fig. 7: Experimental validation of a feedback-force collaboration scheme that uses onboard only sensors, namely an RGB-D camera and an IMU [START_REF] Stegagno | A semiautonomous UAV platform for indoor remote operation with visual and haptic feedback[END_REF]. between two different classes of approaches: the approaches that use external sensorial infrastructures and the approaches that make use of onboard only sensorial equipment. The large majority of works in collaborative control of aerial robots belong to the first category. Motion capture systems, global positioning systems (GPS), and wireless based localization are some example of the infrastructures used in these works [START_REF] Rodríguez-Seda | Bilateral teleoperation of multiple mobile agents: Coordinated motion and collision avoidance[END_REF], [START_REF] Franchi | Bilateral teleoperation of groups of mobile robots with time-varying topology[END_REF], [START_REF] Lee | Semi-autonomous haptic teleoperation control architecture of multiple unmanned aerial vehicles[END_REF], [START_REF] Mersha | Switching-based mapping and control for haptic teleoperation of aerial robots[END_REF], [START_REF] Grabe | The TeleKyb framework for a modular and extendible ROS-based quadrotor control[END_REF]. Some recent efforts have been directed toward the second kind of approach, which presents clearly more challenges, especially in the aerial robotics case. The work in [START_REF] Omari | Bilateral haptic teleoperation of VTOL UAVs[END_REF], where a bilateral teleoperation scheme for aerial robots is presented, represents a first step in the direction of the second class of approaches. In this work the state of the robot is retrieved using a motion capture system, however the obstacle detection is performed using a laser range finder which then feeds the obstacle avoidance algorithm with onboard data.

In the experiments presented in [START_REF] Franchi | Modeling and control of UAV bearingformations with bilateral high-level steering[END_REF] a human operator controls a group of aerial robots that use onboard cameras to measure the angles between themselves while the velocity is still estimated using an external motion capture system.

One of the first work addressing the problem of fullonboard sensorial equipment in the collaborative feedbackforce control of aerial robots is presented in [START_REF] Stegagno | A semiautonomous UAV platform for indoor remote operation with visual and haptic feedback[END_REF]. The human operates with velocity control (and with the feedback-force paradigm) the aerial robot in an unstructured environment, see Figure 7. The state of the robot is estimated using only the onboard IMU and a depth camera (RGB-D). Thanks to the availability of the depth information the obstacle safety is ensured. During the collaboration the aerial robot conducts autonomously some extra-tasks that increase the ergonomics of the operation and its safety. The main of these tasks is a continuous pan scanning that temporarily enlarges the field of view of the robot exploiting the yaw DoF that is left free by the human operator collaboration. This movement is compensated in real time using an adaptive filter thus allowing the operator to experience a yaw-rate-free operation. The haptic feedback is based on the impedance paradigm where the actual velocity is estimated using a photometric error approach.

The development of human-collaborative control schemes that can cope with the use of onboard sensors only is crucial for the employment in the real world. However this aspect has been often overlooked by the recent literature, while it should probably deserve a higher attention.

IV. A TAXONOMY OF COLLABORATIVE

HUMAN-MULTI-ROBOT CONTROL The collaborative scheme may include just one robot (single-robot case) or several distinct robots (multi-robot case). The use a group of coordinating and cooperating robots represents an enabling technology for a large number of applications. As a matter of fact multi-robot systems can show an improved robustness with respect to single point system failures and the capacity of adaptation to environments that are both uncertain and dynamic. Multi-robot systems may present also economical advantages compared to monolithic complex robots due to the lower price of small size systems. Range of applications span from data retrieval and collection, distributed sensing on large-scale areas, deep space interferometry, and search and rescue missions. Among multi-robot systems aerial robots play an essential role for their ability to cover large distances due to their speed and large areas thanks to the presence of the third (altitude) dimension.

Thanks to the appealing features and the great potentials of multiple aerial robot systems, the recent years have seen a sharp increasing in the study of collaborative humanrobot schemes for such kind of robotic systems. The goal of this section is to revise and classify some of these approaches from a multi-robot perspective. In [START_REF] Franchi | Shared control: Balancing autonomy and human assistance with a group of quadrotor UAVs[END_REF] the authors provide an overview that summarizes generic collaborative scheme for the bilateral shared control of groups of aerial robots. The proposed approach is able to integrate three main components, i.e., the human operator assistance, the force feedback and the control of the group topological and metrical properties. Regarding the last component the work suggests to consider three main cases, the unconstrained-, the connected-, and the constant-topology cases. In this section we aim at generalizing this classification taking into account additional aspects related to the collaborative human/multirobot interaction.

A. Level of Centralization

A first classification when dealing with human collaborative schemes for multi-robot systems is about the level of centralization. Several definitions of decentralization has been given in the literature, which often overlap with the concept of a distributed algorithm. Here we adopt a simple definition that is the following: a scheme is decentralized (or distributed) if the amount of communication (packets exchanged) and computation (complexity) for unit of time, and the size of memory needed by each robot and by the human-robot interface, are constant when the total number of robots is increased. A scheme that is not decentralized is centralized. Centralized schemes are the one that need for example the communication graph G to be complete, i.e., (i, j) ∈ E ∀i = j. Centralized schemes are also the one assuming that the human-robot interface sends the message w h to all the robots in the team, i.e., N l = N.

In general, decentralized algorithms are more scalable, i.e., applicable to higher number of robots, but are less efficient than centralized algorithm in the case of a smaller number of robot and when the communication bandwidth and the computation capabilities of each robot allow centralized implementations.

B. Master-leader-followers Schemes

A possible approach to collaboratively control a group of robots is the master-leader-followers scheme. In this scheme all the robots implement any decentralized control strategy resorting to local communication in order to carry on some task while the human operator (the master) communicates with only one robot of the group at time (the leader). All the non-leader robots are called the followers. For example the robotic team task could be to monitor a given area while keeping a certain formation, and the master, that is supervising the overall task providing, e.g., velocity references, can be in communication only with one robot at time due to long distance bandwidth limitations. This scenario is abstractly depicted in Figure 8(left).

Several challenging problems can arise even from this fairly basic situation when collaborative human-robot control is needed. A first example is the difficulty of maintaining the stability of the feedback-force interconnection between the human operator and such multi-robot system, especially in the case in which the multi-robot team interaction topology can arbitrarily switch even independently of the robot state. In fact, in [START_REF] Franchi | Bilateral teleoperation of groups of mobile robots with time-varying topology[END_REF] it is shown that this kind of switching interaction can generate a non-passive and potentially unstable behaviors. The authors then present a passivity-based approach to guarantee a stable behavior of the leader and the followers regardless of the autonomous motion, the switching topology and the interaction with the environment. At the same time, making the multi-robot system passive turns instrumental for the feedback-force collaborative scheme The human operator (master agent) can communicate only with one robot at the time (the leader) using a low channel bandwidth. The leader robot and the followers communicated through a time-varying distributed communication topology. Right: a human operator collaborates with a group of robots using as feedback-force scheme. Passivity of the heterogeneous team of robots is guaranteed despite the switching nature of the communication [START_REF] Franchi | Bilateral teleoperation of groups of mobile robots with time-varying topology[END_REF].

with the human operator (the master). Experiments using this approach are show in Figure 8(right).

Another challenge arising in this context is called the online leader selection problem, i.e., the ability of the system to choose online the best leader that the master has to communicate with. This problem is addressed, e.g., in [START_REF] Franchi | Distributed online leader selection in the bilateral teleoperation of multiple UAVs[END_REF], [START_REF] Franchi | Online leader selection for improved collective tracking and formation maintenance[END_REF] for what concerns the maximization of the effectiveness of the velocity reference provided by the human operator. In this work the authors show that this effectiveness is upperbounded by an index that depends on the current state of the robots and by a graph theoretical index, namely the algebraic connectivity of a special leader-dependent digraph.

C. Formation-orthogonal Control Schemes

Another approach to collaboratively control a group of robots is the formation-orthogonal control scheme. In this scheme the robots autonomously stabilize the relative geometry of the formation using relative or absolute measurements, while the human operator sends references to steer the remaining DoFs of the multi-robot system.

In [START_REF] Lee | Bilateral teleoperation of multiple cooperative robots over delayed communication network: theory[END_REF], [START_REF] Lee | Bilateral teleoperation of multiple cooperative robots over delayed communication network: application[END_REF] the authors propose a method to bilaterally teleoperate a group robots such as, e.g., a group of ground mobile manipulators grasping an object. The main idea of the method relies on the concept of passive decomposition, with which it is possible to decompose the multi-robot dynamics in two independent systems while preserving passivity from an energetic point of view. The locked system abstracts the overall team motion and is controlled by the human operator using a feedback-force paradigm. The shape system defines instead the cooperative grasping aspect and is autonomously controlled by the team of robots. This paradigm allows the human to indirectly control the motion of the grasped object without influencing the reliability of the grasping.

Regarding the aerial robot case, in [START_REF] Lee | Semi-autonomous haptic teleoperation control architecture of multiple unmanned aerial vehicles[END_REF] the authors propose a feedback-force collaboration scheme in which the human operator teleoperates the motion of the multiple aerial robots that act as a virtual deformable object. The scheme is based on three layers: the low-level robot control, the online velocity reference generator, and the bilateral teleoperation layer. The low-level robot control is used in order to abstract from the particular kind of dynamics of the robot. The velocity reference generator mixes together the velocity reference provided by the operator, the obstacle avoidance action, and the distance preservation term. The last two terms are based on an artificial potential approach. The haptic cue provided to the operator is proportional to the average of the velocity reference mismatch in a way that is similar to an impedancelike approach, but extended to the whole multi-robot system considered as a virtual large deformable object. In order to implement this formation control scheme the position measurements of each robot (e.g., from GPS) are needed.

An alternative (especially from the sensing point of view) formation-orthogonal control scheme is presented [START_REF] Franchi | Modeling and control of UAV bearingformations with bilateral high-level steering[END_REF], see Figure 9. This case differs from [START_REF] Lee | Semi-autonomous haptic teleoperation control architecture of multiple unmanned aerial vehicles[END_REF] for three main aspects. The first is that it uses only measurements of the relative angles between the robots (that can be, e.g., retrieved from onboard cameras). The second is that the formation control approach used is almost globally convergent and it is not prone to local minima like the artificial potential schemes typically are. The third is that the human operator velocity reference acts on the tangent bundle of the manifold of configurations that keep the relative angles constant, which includes collective rotations, translations, and dilations. In this way the formation control specifications and the human control result always orthogonal and do not interfere with each others thus establishing a clear separation between the fully autonomous and the human-operated domains of intervention.

D. Group-property Preservation Schemes

Formation-orthogonal control schemes may excessively limit the relative mobility between the robots in the group by, e.g., blocking the relative distances and angles to some fixed or overly constrained values. On the other side, for some tasks it is enough to preserve some high-level structural properties of the group rather than determining the exact shape of it. We refer to such approaches as group-property preservation schemes.

A first example of high-level structural property of a group of robots is the connectivity of its topology, intended either as the communication network or the sensing graph. Connectivity is a very important property since it ensures that the data flow between the different robots cannot be broken and therefore the information can be successfully distributed.

In [START_REF] Giordano | A passivity-based decentralized strategy for generalized connectivity maintenance[END_REF] the authors present a decentralized method that maintains group the connectivity in a flexible way, i.e., allowing the topology to change over time as long as connectivity is preserved, see Figure 10. Connectivity preservation is ensured by keeping the algebraic connectivity λ 2 (see Section II-B) positive during all the robot maneuvers. The algebraic connectivity concept is then extended to take into account complex robot-to-robot interaction going beyond simple max-distance models, which brings to the definition of the generalized connectivity eigenvalue. This is obtained by introducing a state-dependent adjacency matrix A(x 1 , . . . , x N ) and consequently, a state-dependent Laplacian L(x 1 , . . . , x N ). The resulting generalized connectivity is also able to embed mild formation control constrains and obstacle avoidance if needed. Exogenous control actions coming from human operators or from a distributed target visiting algorithm, like the one presented in [START_REF] Nestmeyer | Decentralized simultaneous multi-target exploration using a connected network of multiple robots[END_REF], can be also included. Using a feedback-force collaborative scheme one or more human operators can interact with the group of robots and immersively feel, through haptics, the connectivity force generated by the group.

Connectivity alone might be not enough to achieve certain tasks in which the robots can rely on relative sensing only. In these cases it is more appropriate to consider the concept of rigidity instead, which defines the property of being able to instantaneously reconstruct the group shape using local relative measurements, such as relative distances or relative angles. The rigidity concept has been originally introduced in the structure theory in order to characterize the notion of flexibility and stiffness of rigid-body structures.

The main algebraic object of what is called infinitesimal rigidity [START_REF] Zelazo | Decentralized rigidity maintenance control with range measurements for multi-robot systems[END_REF] is the so called rigidity matrix R(G , p 1 , . . . , p N ) which is defined as

R(G , p 1 , . . . , p N ) = [E l (G 1 ) T • • • E l (G N ) T ](I N ⊗ [p T 1 . . . p T 1 ] T ))
where E l (G i ) is the local incidence matrix relative to the i-th robot, I N is the N × N identity matrix, and ⊗ denotes the Kronecker product between matrices.

In the context of multi-robot systems, the authors of [2] develop a theoretical machinery to express the degree of rigidity using a single non-negative number, namely the where W is the diagonal matrix of the state-dependent weights defined on the edges of E . The authors in [START_REF] Zelazo | Decentralized rigidity maintenance control with range measurements for multi-robot systems[END_REF] present a distributed algorithm that ensures the preservation of the rigidity resorting to local relative distance measurements only. Experiments are shown in which multiple human operators can interact with the group of robots providing velocity references while the group of aerial robots preserves the required degree of rigidity in an autonomous way. As in the connectivity case, haptic cues can be used to perceive the group rigidity using the feedback-force scheme. Another advanced way of establishing a collaboration between human and the robotics team in the group-property preservation schemes is to let the human finely control the index that defines the group properties as, e.g., the connectivity and the rigidity eigenvalues, using one of the DoFs of the interface. A work that explores this idea in a feedback-force fashion for the connectivity index is presented in [START_REF] Secchi | Bilateral control of the degree of connectivity in multiple mobile-robot teleoperation[END_REF].

E. Physical Interaction with Contact

We have already shown examples of coordination of ground robots physical interaction with contact, such as, e.g., [START_REF] Lee | Bilateral teleoperation of multiple cooperative robots over delayed communication network: theory[END_REF], [START_REF] Lee | Bilateral teleoperation of multiple cooperative robots over delayed communication network: application[END_REF]. Many other works are present in the literature for the ground mobile robot case which are not mentioned here. On the other side, human collaborative control of multiple aerial robots performing physical interaction with contact is a very young research field and not many works have been considering this scenario at date.

In [START_REF] Gioioso | The Flying Hand: a formation of uavs for cooperative aerial telemanipulation[END_REF] it is proposed a control framework to let a group of aerial robots grasp an object in a way that each robot uses an attached rigid tool to establish a single contact, see Figure 11. Each robot acts as a flying finger that collaborates with the others to establish a N-fingered hand. In this way the aerial robots realize a unique hand-like system that can grasp and transport an object. The whole system is operated by a human hand using a two-layer approach. First the operator hand is tracked with an RGB-D camera and some features are extracted and mapped to virtual attraction points. Secondly the aerial robots use an impedance force control in order to track the virtual attraction points with some compliance thus enabling grasping and transportation. The approach is validated in simulation only.

Another approach that solves a similar problem from the robot cooperative side but it lacks of the human collaborative part, is presented in [START_REF] Nguyen | Aerial tool operation system using quadrotors as rotating thrust generators[END_REF]. In this work multiple aerial robots are connected to an object using spherical joints and are used as thrust generators to cooperative manipulate the object in a physical interaction with contact scenario. The method, which is validated with experiments using a team of quadrotors as aerial robots, represents a promising building block for a possible extension to the human collaborative case.

V. CONCLUSIONS

In this chapter we have shown and compared several different ways available in the literature to design the human collaborative control of the motion of mobile robots, with particular emphasis on the aerial robot case. We have introduced a simple yet descriptive model for the robots and the human interfaces and we have proposed a list of fundamental axes for a proper classification of the collaborative scheme, including but not limited to the degree of autonomy, the presence of interaction force with the human or with the environment, the kind multi-robot schemes and many others.

The common denominator of all the classification axes proposed in this chapter is the presence of a human operator in the control loop. In fact, fully autonomous control algorithms are not in general easily extendable in order to cope with the presence of a human operator and at the same time to take the best out of this collaboration. The main goal of the chapter has been to introduce the reader and provide a first-level analysis on the several ways to effectively include human operators in the control of both single and multiple aerial mobile robots. A particular emphasis has been set on the aerial robots case, which is at date not yet fully explored and understood, and is among the most challenging cases in robotics.

A lot of problems remain still unsolved in this field, among of which is worth to mention the possibility to perform effective bilateral teleoperation of aerial manipulators, the possibility of relying on onboard sensing in any environmental situation, the ability to cope with the large uncertainty in the perception and control of bilaterally teleoperated aerial robots thus enabling their use in the real world for helping humans in the industry and in their everyday life.

  ẋ = f r (x, u, d) y = h(x, u, d, x e ) min u J(x, u, d, x e )

Fig. 3 :

 3 Fig. 3: Top: block diagram of a feedback-force human-robot collaboration scheme with relatively long time horizon in the operator reference. Bottom: Examples of canonical path transformations. Green arrows represent the DoFs. Depending on the transformation the device motion is artificially restricted (continuous green arrows): 2 DoFs for translation, 1 DoFs for scaling, 1 DoFs for rotation. Blue arrows represent the commands and corresponding motion of the control points.

Fig. 4 :

 4 Fig. 4: Experiments of intercontinental (Korean-German) feedback-force collaborative human-robot control. Top; scenario presented in [21]. Bottom: demo at the IEEE International Conference on Systems Man and Cybernetics 2012, in Seoul, South Korea.

Fig. 5 :

 5 Fig.5: The scenarios of aerial teleoperation for physical interaction with contact considered in[START_REF] Gioioso | A forcebased bilateral teleoperation framework for aerial robots in contact with the environment[END_REF]. Top: docking to a surface. Bottom: pushing a cart.

Fig. 6 :

 6 Fig.6: Block diagram of a control algorithm for aerial teleoperation in physical interaction with contact[START_REF] Gioioso | A forcebased bilateral teleoperation framework for aerial robots in contact with the environment[END_REF].

Fig. 8 :

 8 Fig.8: Left: abstraction of master-leader-followers scheme. The human operator (master agent) can communicate only with one robot at the time (the leader) using a low channel bandwidth. The leader robot and the followers communicated through a time-varying distributed communication topology. Right: a human operator collaborates with a group of robots using as feedback-force scheme. Passivity of the heterogeneous team of robots is guaranteed despite the switching nature of the communication[START_REF] Franchi | Bilateral teleoperation of groups of mobile robots with time-varying topology[END_REF].

Fig. 9 :

 9 Fig.9: Top: block diagram of a formation-orthogonal collaboration scheme that uses relative bearing (angle) measurements[START_REF] Franchi | Modeling and control of UAV bearingformations with bilateral high-level steering[END_REF]. Bottom: implementation with aerial robots using onboard cameras to retrieve the bearing measurements.

Fig. 10 :Fig. 11 :

 1011 Fig. 10: Experiments and simulations involving two operators controlling groups of mobile robots in a generalized connectivity preservation scheme [1].

TABLE I :

 I Degrees of autonomy in the human-robot collaborative control

		Non-robotic systems		Robotic systems
		Fully human-operated	Direct	Collaborative human-robot control Shared	Supervisory	Fully autonomous
	Human	Input Device
		Haptic
		Feedback

Preprint version, final version at http://www.springer.com/ in the book: Trends in Control and Decision-Making for Human-Robot Collaboration Systems page: 2

A taxonomy that is comprehensive of all the possible cases and shows all the most important works in collaborative human-robot control is clearly out of the scope of this chapter, which does not claim to be a complete survey of this huge topic.Preprint version, final version at http://www.springer.com/ in the book: Trends in Control and Decision-Making for Human-Robot Collaboration Systems page: 3

Preprint version, final version at http://www.springer.com/ in the book: Trends in Control and Decision-Making for Human-Robot Collaboration Systems page: 4

Preprint version, final version at http://www.springer.com/ in the book: Trends in Control and Decision-Making for Human-Robot Collaboration Systems page: 5

Preprint version, final version at http://www.springer.com/ in the book: Trends in Control and Decision-Making for Human-Robot Collaboration Systems page: 6

Preprint version, final version at http://www.springer.com/ in the book: Trends in Control and Decision-Making for Human-Robot Collaboration Systems page: 7

Preprint version, final version at http://www.springer.com/ in the book: Trends in Control and Decision-Making for Human-Robot Collaboration Systems page: 8

Preprint version, final version at http://www.springer.com/ in the book: Trends in Control and Decision-Making for Human-Robot Collaboration Systems page: 9

Preprint version, final version at http://www.springer.com/ in the book: Trends in Control and Decision-Making for Human-Robot Collaboration Systems page: 10

Preprint version, final version at http://www.springer.com/ in the book: Trends in Control and Decision-Making for Human-Robot Collaboration Systems page: 11

Preprint version, final version at http://www.springer.com/ in the book: Trends in Control and Decision-Making for Human-Robot Collaboration Systems page: 12

Preprint version, final version at http://www.springer.com/ in the book: Trends in Control and Decision-Making for Human-Robot Collaboration Systems page: 13

ACKNOWLEDGEMENTS

The author wants to thank all the co-authors that collaborated with him in the works cited in this chapter. A special acknowledgment goes to Paolo Robuffo Giordano, with whom many of the algorithms and ideas that are present in those works have been conceived and developed.


T Levi

M Ambroise

F Grassia

S Saïghi

T Kohno

H Kinoshita

T Fujii

Biomimetic CPGs for robotic applications

Keywords: Central Patter Generators, Spiking neural networks, biomimetic, robotic

Locomotion is one of the most basic abilities of animals. Neurobiologists have established that locomotion results from the activity of half-center oscillators that provides alternating bursts. Most rhythmic movements are programmed by central pattern-generating (CPG) networks consisting of neural oscillators. A biomimetic system mimics the nature for simulating the living behavior and/or replaces it (rebuilding the living part). We propose a network of hundreds biomimetic CPGs using biomimetic neuron model and synapses. This network is implemented in an FPGA (Field Programmable Gate Array). This article is decomposed into two parts: the description, the implementation and results of the biomimetic network of CPGs, and the architecture and advantages of robots including our biomimetic CPGs.

INTRODUCTION

Locomotion is one of the most basic abilities of animals. Neurobiologists have established that locomotion results from the activity of half-center oscillators that provides alternating bursts. The first half-center oscillator was proposed by Brown [START_REF] Brown | On the nature of the fundamental activity of the nervous centres; together with an analysis of the conditioning of rhythmic activity in progression and a theory of the evolution of function in the nervous system[END_REF] in 1914. Pools of interneurons control flexor and extensor motor neurons with reciprocal inhibitory connections. Most rhythmic movements are programmed by central pattern-generating networks consisting of neural oscillators (Marder [2]). Central Pattern Generators (CPGs) are neural networks capable of producing rhythmic patterned outputs without rhythmic sensory or central input. CPGs underlie the production of most rhythmic motor patterns and have been extensively studied as models of neural network function (Hooper [3]). Half-center oscillators control various animal locomotion like swimming in xenopus, salamander (Ijspeert [4]), and lamprey (Cohen [START_REF] Cohen | Modelling of intersegmental coordination in the lamprey central pattern generator for locomotion[END_REF]), as well as leech heartbeat system (Cymbalyuk [START_REF] Cymbalyuk | Bursting in Leech Heart Interneurons: Cell-Autonomous and Network-Based Mechanisms[END_REF], Hill [START_REF] Hill | A model of a segmental oscillator in the leech heartbeat neuronal network[END_REF]).

Usually, in robotic field, the CPGs are made using simple neuron-models, are not in biological time scale and were more bio-inspired than biomimetic. A bio-inspired system is inspired by nature for dealing with engineering task (neural network for pattern recognition, data mining, etc.). A biomimetic system mimics the nature for simulating the living behavior and/or replaces it (rebuilding the living part. While multi-legged robots need CPG to move or coordinate their movements, researchers implement an Amari-Hopfield CPG (Amari [START_REF] Amari | Characteristic of the random nets of analog neuron-like elements[END_REF]) or basic CPGs like Van der Pol ones (Van der Pol [START_REF] Van Der Pol | The heartbeat considered as a relaxation oscillation, and an electrical model of the heart[END_REF]), modeled as non-linear oscillators. Those models provide sinusoidal oscillations that are not biomimetic. To be closer to the nature, we developed biomimetic CPGs.

This article is decomposed into two parts: the description and results of the biomimetic network of CPGs, and the architecture and advantages of robots including our biomimetic CPGs.

DESIGN OF BIOMIMETIC CPGs

We propose a network of hundreds biomimetic CPGs using biomimetic neuron and synapse models. This network is implemented in an FPGA (Field Programmable Gate Array). The network implementation architecture operates on a single computation core. This digital system works in real-time, requires few resources and is low power consumption. It is important points for robotic applications. The implementation of this network of CPGs is validated by comparing it with biological data of leech heartbeat neural network. These CPGs could reproduce biological activity of CPG neural networks in terms of period but also in terms of variation of duty cycle and period of bursting.

Model of the Leech Heartbeat system

Our CPG is based on the Leech heartbeat neural network system (Hill [7]). This network is simple; just 8 excitatory neurons with inhibitive synapses are needed, making it an ideal candidate system for elucidating the various biomechanisms governing CPG behaviors and for robotic applications.

Neuron model

In designing a Spiking Neural Network (SNN), the first step is the choice of a biologically realistic model. Indeed, a mathematical model based differential equations is capable of reproducing a behavior quite similar to that of a biological cell. The choice of model was based on two criteria: the family of neurons able to be reproduced and the number of equations. These criteria were used to compare several models, including the Leaky Integrate and Fire model (LIF) (Indiveri [START_REF] Indiveri | Synaptic plasticity and spikebased computation in VLSI networks of integrate-and-fire neurons[END_REF]), the Hodgkin-Huxley model (HH) (Hodkin-Huxley [START_REF] Hodgkin | A quantitative description of membrane current and its applications to conduction and excitation in nerve[END_REF]), and the Izhikevich model (IZH) (Izhikevic [START_REF] Izhikevich | Simple model of spiking neurons[END_REF]).

(Hill [START_REF] Hill | A model of a segmental oscillator in the leech heartbeat neuronal network[END_REF]) used the HH to reproduce the leech heartbeat system with eight neurons (Fig. 1). The HH model reproduces all types of neurons with good accuracy (spike timing and shape). Its main drawbacks are the large number of parameters and the equations required. In the heartbeat network, the main focus is on excitatory neurons, like regular-spiking neurons (RS). The HH model required 32 parameters for an RS and 26 for a fast-spiking neuron (FS) (Grassia [START_REF] Grassia | Tunable neuromimetic integrated system for emulating cortical neuron models[END_REF]). Furthermore, simulating an RS neuron required four ionic channels (dynamics of potassium and sodium ions, leak current, and slow potassium). In contrast, LIF only involves two equations but is only capable of simulating a few types of neurons.

The IZH represents a good solution, as it is based on two equations and is capable of reproducing many different families of neurons by changing four parameters. Furthermore, according to (Izhikevic [START_REF] Izhikevich | Simple model of spiking neurons[END_REF]), this model is resource-frugal, a key advantage when the aim is to design a large CPG network embedded in the same board. The IZH model depends on four parameters, a,b,c and d which make it possible to reproduce the spiking and bursting behavior of specific types of cortical neurons. From a mathematical standpoint, the model is described by a two-dimensional system of ordinary differential equations (Izhikevic [START_REF] Izhikevich | Simple model of spiking neurons[END_REF]):

Izh I u v v dt dv      140 5 04 . 0 2 (1) ) ( u bv a dt du   (2)
with the after-spike resetting conditions:

        d u u c v mV v if 30 (3)

Synapse model

A network is defined by a group of neurons and a group of synapses. Once the neuron model had been chosen, it was obviously necessary to choose a synapse model. Like the neuron model, this model had to be biomimetic but frugal in its use of resources. As the synaptic behavior described in (Hill [START_REF] Hill | A model of a segmental oscillator in the leech heartbeat neuronal network[END_REF]) requires too many resources to be implemented on FPGA, the method chosen to fit overall biological behavior was activity-dependent depression (Tabak [START_REF] Tabak | Modeling of Spontaneous Activity in Developing Spinal Cord Using Activity-Dependent Depression in an Excitatory Network[END_REF]). Activity-dependent depression of synapses is another biological phenomenon consisting of reducing a synaptic weight after a spike. The synaptic current is included into I Izh in (1).

Network topology

The architecture was based on three main blocks: the neuron implemented (or neuron computation core), a synapse, and the RAM. The connectivity between those blocks is shown in Fig. 2. Global architecture of the spiking neural network So far, the neuron computation core can update the state ('u' and 'v' variables) of each neuron. In the digital network, the role of the synapse is to update all synaptic currents and weights related to the activity of all neurons, so the synapse block exhibits two behaviors (spiking or not). The I Izh current is divided into 3 currents: I stat , I exc and I inh . I stat is a constant current that have originally the neuron, I exc and I inh are respectively the excitatory synaptic current and the inhibitory one. These 2 currents describe if there is an excitation or inhibition to apply at the neuron.

Implementation and results

We implement a network of 240 CPGs into a Spartan 6 digital board. This number will allow us several robotic applications like the simulation of Salamander which requires 40 CPGs which we will describe later. To validate our CPG into FPGA, we compare it first with the model described in (Hill [7]). We compare the period, the duty cycle of the burst and also the frequency of the 8 neurons. Fig. 3. Comparison between CPG bursting activity in the complex model simulated by scilab, as described in (Hill [START_REF] Hill | A model of a segmental oscillator in the leech heartbeat neuronal network[END_REF]) and our digital CPG presented above thanks to a logic analyzer. L3/R3 is the couple of neurons (elemental oscillator) described in Fig. 1. The time scale is the same.

In this case (Fig. 3), the activity of one neuron inhibits the second neuron. Due to activity-dependent depression and the GABA effect, the inhibition ends and lets the second neuron fire again. In both cases (biological modeling system and digital system), the bursting activity was similar in terms of period (12s) and duty cycle (55%), thus validating our digital CPG.

Modifying the I stat included into I Izh (1), we could modify the period of the bursting activity. This point is really important because we could in real-time modifying the frequency of the locomotion activity tuning only one parameter.

We described the network of biomimetic CPGs that we designed. Now we will describe the possible applications of these networks and why the biomimetic part is important.

ROBOTIC APPLICATIONS

In the second part, as this CPG network is likely to be useful for mimicking the locomotion activity of various animals, we propose some possible robotic architecture using this network of CPGs.

Biomimetic animal robots

CPGs are really important for designing biomimetic animal locomotion. Usually, the CPGs are simple oscillators or bio-inspired ones. In our case, the biomimetic advantage of our network of CPGs is that the behavior is really the same as biological neural networks, with the variation of duty cycle and periods. The animal robots will then behave more like in the nature. For designing a salamander robot, it needs 20 CPGs with a frequency of walking from 0.6 to 1.2 Hz, and of swimming from 1.6 to 2.9 Hz [START_REF] Ijspeert | From swimming to walking with a salamander robot driven by a spinal cord model[END_REF]. Our system fits with these specifications, and we could also improve it using more CPGs as our network contains 240 CPGs. As the period of our CPG could be modified depending just one parameter, we could make a closed-loop between a sensor and the CPG.

Hybrid robots

One main advantage of our system is that it allows hybrid experiments. Then we could use our CPG as an artificial control unit, and the different actuators and sensors will be biological ones. We already made a hybrid experiment where our biomimetic CPG controls the locomotion through a rat spinal cord. Fig. 4. The biomimetic CPG send its bursting activity to a rat spinal cord. The ventral root output of the spinal cord, which is directly sent to muscles, is the same than the CPG bursting activity.

Microfluidic robots

Currently, we are designing a microfluidic robot controlled by our CPG (Fig. 5). This robot is based on biomimetic control, actuator and sensor architecture that features highly modularized components and used few energy. It can conduct autonomous investigation and is less subject at pressure conditions. This main part of the robot is based on PDMS (Polydimethylsiloxane) microfluidic devices. The advantage of using our biomimetic CPG as control unit is that we could easily change the period and the duty cycle of the bursting activity. Furthermore, the FPGA power consumption is less than microprocessor one, that is important for the autonomy of the robot. As our control unit is biomimetic, the microfluidic robot will reproduce better the nature locomotion (variation of duty cycle and period) than simple electronic oscillators.

CONCLUSION

A network of biologically realistic CPGs was implemented with a minimum resource cost, while maintaining its biomimetic activity, as shown in the results section. The biomimetic part is important for robotic applications especially for animal robots and hybrid robots. The variation of period and duty cycle that we could observe in nature is possible with these CPGs. Future robotic applications using these CPGs are described.

Fig. 1

 1 describes the diagram of the CPG.

Fig. 1 .

 1 Fig. 1. A diagram of the CPG in the leech heartbeat system, including two elemental oscillators, L3/R3 and L4/R4, and two pairs of coordination neurons, L1/R1 and L2/R2

  Fig. 2.Global architecture of the spiking neural network So far, the neuron computation core can update the state ('u' and 'v' variables) of each neuron. In the digital network, the role of the synapse is to update all synaptic currents and weights related to the activity of all neurons, so the synapse block exhibits two behaviors (spiking or not). The I Izh current is divided into 3 currents: I stat , I exc and I inh . I stat is a constant current that have originally the neuron, I exc and I inh are respectively the excitatory synaptic current and the inhibitory one. These 2 currents describe if there is an excitation or inhibition to apply at the neuron.

Fig. 5 .

 5 Fig. 5. Microfluidic robot using biomimetic CPGs as control unit

Table 1

 1 describes the resources needed for the implementation of the network into a Spartan 6 digital board.

Table 1 .

 1 Resources needed for our CPG network on a

	Spartan 6 digital board
	Resources	Total available Used for	Used for
			one CPG	240 CPGs
	Slice FF's	184304	1,093	1,459
			(0.6%)	(0.8%)
	Slice LUT	92152	1,037	1,756
			(1.2%)	(1.9%)
	Multiplier	180	10 (5.6%)	10 (5.6%)
	Total RAM	4824 kb	9 kb	765 kb
			(0.2%)	(16%)

ACKNOWLEDGMENT

This work was supported by the European Union's Seventh Framework Programme (ICT-FET FP7/2007-2013, FET Young Explorers scheme) under grant agreement n° 284772 BRAIN BOW (www.brainbowproject.eu).


F Grassia

T Levi

S Saighi

T Kohno

Bifurcation analysis in a silicon neuron

Keywords: Silicon Neuron, Hopf bifurcation, Hodgkin-Huxley equations, neuromorphic engineering

In this paper, we describe an analysis of the nonlinear dynamical phenomenon associated with a silicon neuron. Our silicon neuron integrates Hodgkin-Huxley (HH) model formalism, including the membrane voltage dependency of temporal dynamics. Analysis of the bifurcation conditions allow us to identify different regimes in the parameter space that are desirable for biasing our silicon neuron. This approach of studying bifurcations is useful because it is believed that computational properties of neurons are based on the bifurcations exhibited by these dynamical systems in response to some changing stimulus. We describe numerical simulations and measurements of the Hopf bifurcation which is characteristic of class 2 excitability in the HH model. We also show a phenomenon observed in biological neurons and termed excitation block. Hence, by showing that this silicon neuron has similar bifurcations to a certain class of biological neurons, we can claim that the silicon neuron can also perform similar computations.

INTRODUCTION

Neuromorphic engineering is a research field where microelectronics meets biology. The link between both is done by computational neurosciences which model and emulate a part of brain activity. Neuromorphic systems emulate biological structures in the hope of retaining its remarkable computational efficiency. Analysis of the bifurcation is useful because it is believed that computational properties of neurons are based on the bifurcations exhibited by these dynamical systems in response to some changing stimulus [START_REF] Izhikevich | Neural excitability, spiking and bursting[END_REF] [START_REF] Rinzel | Analysis of Neural excitability and oscillations[END_REF]. These artificial systems would be useful to neuroscientists for exploring neural computation. Contrarily to computer simulation, analog circuits are inherently suitable for simulating differential equations in real time and in parallel [START_REF] Chua | CNN universal chips crank up the computing power[END_REF]. The hardware simulation further facilitates the building of a hybrid network connecting artificial and biological neurons to create a real-time loop or can be even used to replace nonfunctional neurons as a prosthesis.

Pioneering work in modeling the dynamics of a biological neuron was done by Hodgkin and Huxley (HH) [START_REF] Hodgkin | A quantitative description of membrane current and its application to conduction and excitation nerve[END_REF]. In the most detailed family of neuron models, known as conductance-based models, ionic and synaptic currents charge and discharge a capacitor representing the neuron membrane. All of these models find their origins in the HH formalism. Each ionic channel (sodium, potassium, calcium, etc.) is represented by a time and voltage dependent conductance: this electrophysiological description makes these models particularly well-suited to an implementation involving analog electronics [START_REF] Simoni | A multiconductance silicon neuron with biologically matched dynamics[END_REF]. The main advantage of this formalism is that it relies on biophysically realistic parameters and describes individual ionic and synaptic conductances for each neuron in accordance with the dynamics of ionic channels.

Hodgkin classified biological neurons mainly into two classes according to their response properties to a sustained current stimulus [START_REF] Hodgkin | The local electric changes associated with repetitive action in a non medullated axon[END_REF]. The cells that show Class 1 neural excitability can fire with an arbitrarily low frequency by applying a sufficiently close-to-threshold current. The frequency increases monotonically as the current increase. The cells that show Class 2 neural excitability transitioned from silence to firing at an arbitrary nonzero frequency. Furthermore, the Class 1 characteristic is can be associated with a saddle-node bifurcation at the transition from silence to spiking. As shown by bifurcation analysis, the Class 2 excitability is observed when the resting state loses its stability via an Hopf-bifurcation.

The Hodgkin-Huxley model belongs to Class 2. Its dynamical properties with a constant stimulus have been studied extensively [START_REF] Hassard | Bifurcation of periodic soluti-ons of the Hodgkin-Huxley model of the squid giant axon[END_REF] [START_REF] Hassard | Theory and Applications of Hopf Bifurcation[END_REF], and these have shown that the Class 2 neural excitability of the HH model is generated by Hopf bifurcations.

Innovative work in modeling of a simplified silicon neuron that exhibits HH type dynamics was done in [START_REF] Kohno | A MOSFET -basedmodel of a class 2 nerve membrane[END_REF] and in [START_REF] Basu | Dynamics and Bifurcations in a Silicon Neuron[END_REF]. On the other hand, some designs have faithfully replicated full Hodgkin-Huxley dynamics, resulting in large footprints for each neuron [START_REF] Simoni | A multiconductance silicon neuron with biologically matched dynamics[END_REF]. We implement in our chip the HH model with an approximation: we use a fixed time constant to reduce the silicon area required by the neuron implementation in the chip. In this paper we present an analysis of the nonlinear dynamical phenomenon associated with our silicon neuron. We show a Hopf bifurcation which is characteristic of class 2 excitability in the HH model in our approximated model. We also show a phenomenon observed in real neurons and termed excitation block.

HH MODEL IN VLSI

The Hodgkin-Huxley Formalism

We chose to implement neuron model following the Hodgkin-Huxley formalism. Electrical activity of a neuron is the consequence of the diffusion of different ionic species through its membrane. The HH formalism provides a set of equations and an equivalent electrical circuit which describes conductance phenomena.

The current flowing across the membrane is integrated on the membrane capacitance, according to expression (1):

S I M M I + I - = dt dV C  (1)
where VM is the membrane potential, CM is the membrane capacitance, and IS is a stimulation or synaptic current. II is the current for a given channel type and its associated equation is:

) E .(V .h .m g I I M q p I I   ( 2
)
where gI is the maximum conductance; m and h represent the activation and inactivation terms, respectively. They are dynamic functions, which describe the permeability of membrane channels to its specific ion. EI is the ion-specific reverse potential and p and q are integers.

According to the first order differential equation (3), m relaxes back towards its associated steady-state value m, which is a sigmoid function of VM (4). The time constant for convergence is τm which is also voltage dependent of the membrane voltage VM.

  m m dt dm V M m     (3)              m , Slope m , Offset M V V V exp 1 1 m (4)
In ( 4), VOffset,m and VSlope,m are the offset and the slope of the activation sigmoid respectively. The inactivation parameter h follows identical equations, except for the sign inside the brackets, which is positive.

The HH model describes sodium, potassium and leakage channels, with p = 3 and q = 1; p = 4 and q = 0; p = 0 and q = 0 respectively, in expression (2). These channels are responsible for action potential generation. For more complex activity patterns, such as bursting or the discharge of action potentials with adaptation phenomena, additional channels such as slow potassium (p=1 and q=0) and L-Type calcium for bursting (p=2 and q=1) have to be taken into account. Our chip has a large range of validity domains for the parameter to reproduce different kind of neurons like: Fast Spiking (FS), Regular Spiking (RS), Intrinsically Bursting (IB), and Low Threshold Spiking (LTS) [START_REF] Grassia | Tuna-ble neuromimetic integrated system for emulating corti-al neuron models[END_REF]. Although these HH-type models sometimes exhibit much more complex dynamics than the original HH model, they share common nonlinear characteristics and dynamics with the HH in many aspects.

The model dedicated to our integrated circuit

Our system is composed of our most recent chip called Galway and the dedicated board named Ekerö. This chip includes analog operators for the computation of the HH formalism, and for the construction of neural networks, multi-synapses that consist of gathering all synaptic inputs in one electronic input. We implement in our chip the HH model with an approximation: we use a fixed time constant in [START_REF] Chua | CNN universal chips crank up the computing power[END_REF]. We made this choice to reduce the size of the chip. So the only difference between the model for the VLSI and the HH model presented in [START_REF] Hansel | Phase dynamics for weakly coupled Hodgkin-Huxley neurons[END_REF] is the approximation used for the gating variable.

Hopf bifurcation

The Hopf bifurcation theory [START_REF] Hassard | Theory and Applications of Hopf Bifurcation[END_REF] asserts that if a parameterized system possesses an equilibrium point and two of the eigenvalues of the Jacobian matrix of the system linearized around the equilibrium point are conjugate pure imaginary numbers and the others have negative real parts, one of the following bifurcations takes place as the parameter changes:

1) A bifurcation from a stable equilibrium point to an unstable equilibrium point with a stable limit cycle around it; or a bifurcation with the opposite direction. We refer to this bifurcation as supercritical.

2) A bifurcation from an unstable equilibrium point to a stable equilibrium point with an unstable limit cycle around it; or a bifurcation with the opposite direction. We refer to this bifurcation as subcritical.

Simulation

To validate our approximation we used continuation of solutions in AUTO, an open source mathematical package that can produce bifurcation curves for equilibria as well as for periodic orbits, to get the complete bifurcation diagram of the system. We compare the simulations between HH based model and our implemented HH model. 1 shows the resulting bifurcation diagram of our HH approximated model where solid thick and solid thin curves represent stable and unstable equilibria respectively while solid and dashed circles denote stable and unstable limit cycles. It can be seen that the limit cycles are born initially through a fold bifurcation of cycles. Both the Hopf bifurcations are subcritical as they involve an unstable limit cycle. The amplitude of the stable limit cycle keeps on reducing until it coalesces with the unstable limit cycle in another fold bifurcation. The resting state loses stability via the subcritical Hopf bifurcation at the first point HB in Fig. 1. At the second point HB in Fig. 2, a stable limit cycle and an unstable one arise via the fold bifurcation. Bifurcation diagram in Fig. 1 illustrate how the repetitive firing emerges when we apply sustained stimulus current. Stable limit cycle corresponds to repetitive firing.

For a comparison with HH model, its bifurcation is also shown in Fig. 2. The Hopf bifurcation for smaller current is subcritical. The limit cycle appears by a fold bifurcation and disappears by a supercritical Hopf bifurcation. The reduction in amplitude before the subcritical Hopf bifurcation is similar to the Fig. 1. It can be seen in Fig. 2 that the limit cycle is born by a fold bifurcation but terminates in a supercritical Hopf bifurcation. The bifurcation diagrams shown in Fig. 1 and Fig. 2 share the topology in a biologically meaningful range of stimulus current. The qualitative nature of the plots (Fig. 1 and Fig. 2) is similar as the reduction in amplitude of the limit cycle before its disappearance is present in both pictures.

RESULTS IN VLSI

Firing rate

In Fig. 1, the system state jumps to the stable limit cycle when the stimulus current exceeds the Hopf bifurcation point. In this case the firing frequency cannot be zero at the bifurcation point for the Class 2. Steps of input current are important type of stimulus typically used to characterize the behavior of neuron models. The resulting spike frequencies for different values of stimulation current are plotted in Fig. 3. This figure shows a behavior typical of class 2 neurons where the spiking frequency does not tend zero upon reduction of the input current. In other word, the disappearance of the limit cycle is not associated with its frequency reducing to zero like in class 1 neurons.

Effect of current impulses

Fig. 4 shows the effect of current impulses of different values in our silicon neuron. Larger current pulses shifts the initial condition beyond the unstable limit cycle leading to spontaneous oscillations. We used four different current pulses of increasing amplitude to stimulate our silicon neuron. It can be seen in Fig. 4 that for the first two case the neuron come back to resting state and in the third and fourth case it start the oscillations. This demonstrates the existence of the threshold that in our silicon neuron is 0.5 nA.

Fig. 4.

Hardware neuron membrane voltage depending on current impulses.

Excitation block

To prove that the silicon neuron reproduces similar dynamical properties of biological neuron, we also explored the phenomenon known in biology and termed excitation block. In this case, there is the cessation of repetitive spiking as the amplitude of the stimulus current increases. The equilibrium regains stability through a Hopf bifurcation which may be subcritical or supercritical. To observe this phenomenon a slowly increasing ramp of current was injected. In Fig. 5 is showed that for much larger value of the stimulus the equilibrium becomes stables again.

CONCLUSION

In this paper, we described an analysis of the nonlinear dynamical phenomenon associated with a silicon neuron. We implemented an approximated HH model using fixed time constant. We showed numerical simulations of the Hopf bifurcation which is characteristic of Class 2 excitability in the HH model. We compared the results with the HH model showing that our model shares the dynamics with the full HH model. We also showed the firing rate and then the phenomenon observed in real neurons and termed excitation block. Hence, by showing that this silicon neuron has nonlinear dynamical phenomenon similar to a certain class of biological neurons, we can claim that the silicon neuron can also perform similar computations.

Fig. 1 .

 1 Fig. 1. Bifurcation diagram for our approximated model.

Fig. 2 .

 2 Fig. 2. Bifurcation diagram for HH model.

Fig.

  Fig.1shows the resulting bifurcation diagram of our HH approximated model where solid thick and solid thin curves represent stable and unstable equilibria respectively while solid and dashed circles denote stable and unstable limit cycles. It can be seen that the limit cycles are born initially through a fold bifurcation of cycles. Both the Hopf bifurcations are subcritical as they involve an unstable limit cycle. The amplitude of the stable limit cycle keeps on reducing until it coalesces with the unstable limit cycle in another fold bifurcation. The resting state loses stability via the subcritical Hopf bifurcation at the first point HB in Fig.1. At the second point HB in Fig.2, a stable limit cycle and an unstable one arise via the fold bifurcation. Bifurcation diagram in Fig.1illustrate how the repetitive firing emerges when we apply sustained stimulus current. Stable limit cycle corresponds to repetitive firing.For a comparison with HH model, its bifurcation is also shown in Fig.2. The Hopf bifurcation for smaller current is subcritical. The limit cycle appears by a fold bifurcation and disappears by a supercritical Hopf bifurcation. The reduction in amplitude before the subcritical Hopf

Fig. 3 .

 3 Fig. 3. Hardware measurements of the frequency versus stimulation current curves of our silicon neurons.

Fig. 5 .

 5 Fig. 5. Measured data of the excitation block where the input current is slowly increased.

ACKNOWLEDGMENTS

This project was partly supported by funding under the Seventh Research Framework Program of the European Union (FP7-PEOPLE-ITN-2008) under the Grant n°237955 (FACETS-ITN).