Ajay Kattepur
email: ajay.kattepur@tcs.com

Balamuralidhar

RoboPlanner: Towards an Autonomous Robotic Action Planning Framework for Industry 4.0

Keywords: Ajay Kattepur, Balamuralidhar Picker Robot (e, g, KUKA KMR Quantec ) Industry 4, 0 Warehouse

whether they are published or not. The documents may come   L'archive ouverte pluridisciplinaire

Introduction

Advances in robotics, cyber-physical systems and industrial automation has come to the forefront with Industry 4.0 [START_REF] Lasi | Industry 4.0. Business & Info[END_REF], with the following key requirements:

1. Interoperability: Machines, Internet of Things (IoT) [Greengard, 2015] enabled devices and humans connected and coordinating with each other. 2. Information transparency: Physical systems enhanced with sensor data to create added value information systems. 3. Technical Assistance: Use of intelligent devices to aid in informed decision making. Robotic automation may be identified to perform repetitive, unsafe or precise tasks. 4. Decentralized Decisions: The ability of such systems to make autonomous decisions; only critical cases will involve human intervention.

A fundamental characteristic required in Industry 4.0 deployments is the ability of autonomous robotic devices to self-configure in dynamic goal and deployment conditions. Autonomic computing [Huebscher and McCann, 2008] models have been proposed to create self-aware robotic systems that respond to both high level goals as well as external stimuli [START_REF] Faniyi | Architecting self-aware software systems[END_REF]. This has led to the development of Cognitive Robotic Architectures [START_REF] Levesque | [END_REF]Lakemeyer, 2010][Beetz et al., 2010], that are at the intersection of robotics, IoT and Artificial Intelligence [START_REF] Russell | Artificial Intelligence: A Modern Approach[END_REF].

Cognitive robots are able to intelligently execute tasks based on high level goals, dependent on world model knowledge and sensory perceptions to generate efficient actions [START_REF] Levesque | Cognitive robotics[END_REF]. In order to be deployed in dynamic Industry 4.0 environments, the robots must be autonomous and adaptive to runtime changes. Given a high level task such as "pick ball from warehouse rack", the autonomous robot must identify appropriate action plans to perform this task. As the robots are intended to be learning world models, knowledge bases are needed to populate information about the world, object, perception and action sequences needed. Any runtime anomalies are dealt with through further queries and eventual exception handling.

Distilling these high level requirements, an autonomous planning module for robots should include: (i) Knowledge Bases that efficiently capture relationships between world models, objects, robot actions and tasks (ii) Action Plans that are efficiently decomposed from a high level goal task; this involves querying the knowledge base as well as triggering perceptions in case of knowledge mismatch (iii) Techniques to Reconfigure actions at runtime, when plans cannot be executed due to constraints (iv) Rules for consistent Updates to the world model, which allows multiple robots to coordinate or analyze exceptions during execution. While individual modules may have been developed in the robotic and embedded software communities, integrating these features into a common framework for industrial deployments remains a challenge.

In this paper, we propose RoboPlanner, a structured technique to generate design time action plans for autonomous robots. In order to enable autonomy in de-ployments, we integrate knowledge bases, design time action planning and runtime adaptation modules. Knowledge representation and queries are enabled using efficient graph database technologies [START_REF] Angles | Survey of graph database models[END_REF]. Design time action plans as provided using the formal concurrent programming knowledge Orc [START_REF] Kitchin | The orc programming language[END_REF], that allows structured composition of action plans. To take care of runtime adaptation, we provide general rules for triggering perception and exception handling. An integrity check is also provided to update the graph database with runtime knowledge. This framework is implemented over a realistic industrial use case involving autonomous picking robots employed in Industry 4.0 warehouses [START_REF] Wurman | Coordinating hundreds of cooperative, autonomous vehicles in warehouses[END_REF].

Principal contributions of this paper:

1. RoboPlanner Knowledge Base module that formally models robotic world models, capabilities, object descriptions and task templates. 2. RoboPlanner Action Planner that uses design-time queries/updates to knowledge graph databases, including exception handling. 3. RoboPlanner Runtime simulation, adaptation and performance analysis of action plans using graph queries. This may be used to generate executable task templates for physical robots. 4. RoboPlanner integrity checks for runtime updates to the knowledge base. 5. Demonstration of the framework over an Industry 4.0 warehouse automation task. The rest of this paper is organized as follows: Section 2 provides an overview of Industry 4.0 warehouse automation and the autonomous robots deployed in them. The RoboPlanner modules are described in Section 3. Details of knowledge base representation using graph databases are covered in Section 4. Section 5 describes the techniques used for action plan generation. Simulation, performance analysis and knowledge updates in autonomous robot deployments are presented in Section 6. The paper ends with related work and conclusions.

Warehouse Automation

In this section, we introduce Industry 4.0 warehouse automation tasks that may be fulfilled by autonomous robots. A high level description of autonomous robots is also introduced, which is used to build the RoboPlanner framework in proceeding sections.

Industry 4.0 Warehouses

Industrial warehouses are employed as buffers in supplychains to maintain excess product, when there are variations in procurement/customer demand [START_REF] Bartholdi | Warehouse and Distribution Science[END_REF]. Considerable effort has gone in reducing the stowing and procurement times in such warehouses, with automated picking robots [Zhang and et al., 2016] being throughput of pick & place tasks.

Fig. 1 presents a high level view of operations taking place in automated warehouses. Once a delivery order is received, the products are procured from the warehouse. As shown in Fig. 1, autonomous Picker robots (such as KUKA KMR Quantec1 ) are being proposed for Industry 4.0 automating pick & place tasks. The robots are intended to be autonomous, with adaptation seen for varying pick-up locations, product dimensions and rates of procurement. When the required products are procured, they are collated and checked for final packing and product shipment.

In order to successfully integrate robotic entities into complex industrial deployments, it is crucial to develop a unified modeling framework for autonomous robots.

Autonomous Robots

To model the robotic components in warehouses, we make use of the Autonomous Robot abstraction, inspired by intelligent agents [START_REF] Russell | Artificial Intelligence: A Modern Approach[END_REF]. Typical activities, for instance with a pick & place robot in a smart warehouse, include:

1. Goals: Understanding goals of each task and subtask, such as, placing correct parts into correct bins within the given time constraints.

2. Perception: Object identification and obstacle detection using camera and odometry sensors that sense the environment. This aids the robot in object detection and identification. Robot location, view and environment may also be perceived.

Actions:

Identifying granular actionable subtasks, such as, moving to particular location, picking up parts of orders or sorting objects. Constraints may be placed on the robot capabilities, motion plans and accuracy in performing such actions.

4. Knowledge Base: Using domain models of the world for goal completion, such as warehouse environment maps, rack type and product features. The robot capabilities and necessary algorithms should enable completion of goals.

Algorithm 1 presents an overview of an intelligent robot's perception and action via a Knowledge Base [START_REF] Russell | Artificial Intelligence: A Modern Approach[END_REF]. The knowledge base coordinates the appropriate action in relation to an individual robot's perception. The knowledge base should also include descriptions of domain ontology, task templates, algorithmic implementations and resource descriptions. To integrate the above elements into robotic interactions for Industry 4.0, we propose the RoboPlanner autonomous architecture framework.

RoboPlanner Modules

In this section, we provide details about the various modules to be integrated within RoboPlanner. These modules cover the principal requirements of cognitive robotic architectures [START_REF] Levesque | [END_REF]Lakemeyer, 2010][Beetz et al., 2010], including knowledge representation, action planning, reconfiguration and knowledge updates. Fig. 2 provides an overview of the modules that are integrated within RoboPlanner:

• Design Time Action Planning Module: This module is responsible for generating efficient action plans, when input with a high level goal. The module decomposes the goal into atomic tasks, and applies workflow specification languages (such as Orc [START_REF] Kitchin | The orc programming language[END_REF]) to complete the goal task. Action planning involves querying the Knowledge Graph Database Module to ascertain requirements for goal completion. Robot perception may also be triggered to acquire further information for action planning.

• Knowledge Graph Database Module: An integral part of all autonomous/cognitive robotic architectures is the knowledge base. We model this using graph databases [START_REF] Angles | Survey of graph database models[END_REF], that maintain relationships between data in a graphical form. Entities such as the world model, robotic algorithms and task templates are stored in the database. The knowledge database is queried both at design time for action generation and at runtime for knowledge updates.

• Runtime Execution Module: The action plans are executed by one or multiple autonomous robots to complete the task. Translation of the action plan to a robot specific middleware language such as ROS 2 may be done. The execution module may be aided by robotic perception. Knowledge that is gained during the execution is to be updated to the graph knowledge database, after satisfying some integrity constraints.

• Adaptation Monitoring Module: This modules monitors runtime deployments of intelligent robots 2 http://www.ros.org/ to estimate plan completion. While robotic perception may be used to aid in unforeseen circumstances, more severe exceptions may require replanning. Performance degradation (leading to noncompletion of plans), may also trigger re-planning. Knowledge of instances that trigger re-planning are learnt and updated.

The following sections dive further into the modeling and implementation of these modules.

Robotic Knowledge Base

The robotic knowledge base is modeled using property graphs, with data stored in graph databases. Queries using the Gremlin graph query language are also studied.

Knowledge Graphs

In order to model knowledge bases inherent in intelligent automation, we make use of property graphs [START_REF] Angles | Survey of graph database models[END_REF]. Property graphs are attributed, labeled, directed graphs. This is an alternative to semantic ontologies [START_REF] Grimm | Knowledge representation and ontologies[END_REF] and tuple datastores that are use in implementations such as Knowrob [START_REF] Tenorth | Knowrob -a knowledge processing infrastructure for cognition-enabled robots[END_REF] and CRAM [START_REF] Beetz | Cram: A cognitive robot abstract machine for everyday manipulation in human environments[END_REF]. Our knowledge base has the following knowledge graphs included:

• World Models: Describes the environment map and layout, including object locations.

• Object Templates: Describes the target objects of interest, including shape, size, colour and location.

• Robot Capabilities: Provides robot models, capabilities, sensors and actuators that are integrated to perform tasks.

• Robotic Algorithms: Navigation, manipulation and task allocation algorithms that are used within robotic actions.

• Task Templates: High level task requirements and corresponding outputs are provided.

Fig. 3 provides the property graph models for world models, task templates, object templates, robot capabilities and robot algorithms. To describe properties between edges, we limit ourselves to four relations: isOfType, hasProperty, requires and produces. isOfType provides hierarchical sub-class relationships; hasProperty extends property descriptions using keyvalue pairs; requires provides pre-conditions to extract knowledge from the graph; requires provides post condition effects of executing the node. These relationships may be queried to extract information from the knowledge base. Fig. 3a provides the capabilities of a Pick Robot that Robot Model, Capabilities, Perception; it requires Target, World Model, Algorithms and produces the Pick, Place Actions.

Algorithms necessary for the robotic executions are provided in Fig. 3d, with path planning, image template matching and grasp manipulation algorithms included. Explicit definitions of each task is provided in Fig. 3b, for instance with the Place task, which requires World Model, Target Object, Picker Robot and produces Placed Object. Note that the property graph modeling approach provides extensibility and reuse of information across multiple autonomous robotic deployments.

Graph Database Queries

Semantic ontologies typically store data in tuple datastores that reduce expressivity provided in graph representations [START_REF] Angles | Survey of graph database models[END_REF]. Scalability is another hindrance in representation, update and query of large ontologies. Graph databases are emerging as an appropriate tool to model interconnectivity and topology of relationships among large knowledge data sets [START_REF] Angles | Survey of graph database models[END_REF]. Principal advantages include: (i) Being able to keep all the information about an entity in a single node and show related information by arcs connected to it; (ii) Queries can refer directly to this graph structure, such as finding shortest paths or determining certain subgraphs; (iii) Graph databases provide efficient storage structures for graphs, thus reducing computational complexity in operations. Graph databases are also emerging as high-performance back end stores when making use of complex dialogue and chatbot engines [M. Maro and Origlia, 2017].

To implement the property graphs in Section 4.1, we make use of the multi-modal OrientDB database3 . Ori-entDB uses a generic vertex persistent class V and a class for edges E. Unlike ontologies that store data using triple stores, graph databases maintain the graphical structure with vertices and edges. In the graph data model, nodes are physically connected to each other via pointers, thus enabling complex queries to be executed faster and more effectively than in a relational data model [START_REF] Angles | Survey of graph database models[END_REF]. Properties are represented as Key-Value pairs that may be queried.

An example graph database (of the World Model in Fig. 3e) with vertices, edges and properties in OrientDB is presented below: In order to query this graph, we use Gremlin4 , a domain-specific (DSL) open source programming language focusing on graph traversal and manipulation. The following types of queries may be made:

1. Filtering: Filter out vertices or edges according to given property labels. For instance, the query may g.v().Name be used to filter out properties such as Name of a vertex. 2. Complex Queries: Queries can combine multiple vertices, edges and properties. Queries can also provide range or equality constraints to numeric property values.

For instance, the complex query g.V.has('Name', 'Warehouse').out('hasProperty').map matches the vertex with property key-value pair (Name, Warehouse), output edge with property hasProperty and produces an output of the vertices. gremlin> g.V.has('Name', 'Warehouse').

out('hasProperty').map ==>{Rack=RackConfig, Layout=WarehouseLayout, Aisles=AislesConfig, Name=Map} ==>{Properties=ObjectProperties, Name=Objects, Location=ObjectLocation}

Graph Traversal:

Another advantage of storing data using graph databases is the ability to traverse graphs.

For instance, the query g.v().outE.inV.name.path traverses the output edges (outE) of a vertex, and provides the path traversed. While we have made use of Gremlin as the language for explicit graph database querying, this can also be a backend for an efficient dialogue/chatbot implementation [M. Maro and Origlia, 2017]. Questions such as "Where is the target?" or "What are the target's properties?" or "Can the robot lift this?" can be translated into efficient knowledge base queries as defined above. It is of interest to translate this knowledge to efficient action plans for the robot to act upon, which is explored next.

Action Plan Generation

In order to study the design time action planning module, we formalize the interaction between the planner and knowledge base. An overview of the concurrent programming language Orc is also provided, that is later used to simulate action planning.

Orc Language

In order to implement robotic action plans, we make use of the formal specification language Orc. The Orc concurrent programming language is grounded on formal process-calculi to specify complex distributed computing patterns [START_REF] Kitchin | The orc programming language[END_REF]. The execution of programs in Orc makes use of Expressions, with the atomic abstraction being a site. To create complex expressions based on site invocations, Orc employs the following Combinators:

• Parallel Combinator (|): Given two sites s 1 and s 2 , the expression s 1 | s 2 invokes both sites in parallel.

• Sequential Combinator (>x>, ): In the expression s 1 >x> s 2 (shorthand s 1 s 2 ), site s 1 is evaluated initially, with every value published by s 1 initiating a separate execution of site s 2 .

• Pruning Combinator (<x<, ): In the expression s 1 <x< s 2 (shorthand s 1 s 2 ), both sites s 1 and s 2 execute in parallel. If s 2 publishes a value, that value is bound to x and the execution of s 2 is terminated.

• Otherwise Combinator (;): The expression s 1 ; s 2 first executes site s 1 . If s 1 publishes no value and halts, then s 2 is executed instead.

The val declaration in Orc binds variables to values.

The def declaration defines a function. Orc further contains built-in sites incorporating distributed computing paradigms such as channels, semaphores and synchronization primitives (further details available in the Orc website 5 ).

Action Planning Module

As specified in Fig. 2, action planning involves interacting with the knowledge base to efficiently plan manipulation, navigation and task planning actions. However, perception and exception handling must also be built in to take care of insufficient knowledge.

To formalize the process of generating action plans required to satisfy goals, we present Algorithm 2. Given an input goal (e.g. pick ball from rack using picker robot), the first step (lines 3, 4 in Algorithm 2) is to verify and subdivide goals from the task descriptions available (pick target, being an atomic subgoal). For each of these subgoals, there are pre-conditions to be satisfied (lines 6-8 in Algorithm 2): subgoals require (actions, targets), actions require (targets, object attributes, capabilities), targets require (object attributes). The object attributes of interest (environment rack, locations) can either be derived from the world model knowledge base or by querying robot perception (robot sensor observation and interpretation, environment point cloud). The target of interest (ball) can either be identified from the object templates knowledge base or by querying robot perception (robot sensor observation and interpretation, perception algorithms). The capability to complete goal (robot model, arm length, battery state) is also extracted from the robot capability knowledge base. Finally, the action (pick ball) needed to satisfy the subgoal is derived, dependent on the specified target, object attributes and capability (line 12 in Algorithm 2). The actions consist of both navigation (path planning) and manipulation (grasping, lifting) procedures. This process is used iteratively for each subgoal to derive the action plan needed to enact the goal (line 13, 14 in Algorithm 2). In case there are Exceptions observed within the subgoal planning, re-planning is triggered.

An example of such an action plan in presented in Fig. 4, wherein the high level input task of: information to identify targets (target?) or atomic actions (action?). Perception triggering and re-planning in case of exceptions are also provided. Such a process of decomposing high level expressions to actionable tasks has also been employed in the automated planning community with Hierarchical Task Networks [START_REF] Erol | Htn planning: Complexity and expressivity[END_REF].

Note that though we have represented this via graph queries, another view would be request-responses with a dialogue agent [A. Bordes and Weston, 2017], representing the knowledge base. The dialogue agent could be used to further clarify queries that may be ambiguous. A typical conversation instance could be: User: yes Such a system unifies the modeling of both knowledge acquisition from a central repository, robotic updates and queries that may be made to human participants. The action plans that are formulated result in valid goal fulfillment, due to varied knowledge sources incorporated.

To further generalize this action planning framework, we provide instances of multiple artifacts in Table 1. We emphasize that the procedure outlined in Algorithm 2 and Fig. 4 is structured to be generic, allowing action plans to be generated with various world models, robotic capabilities and task templates. Such a structured way of planning actions will prove valuable across multiple deployments.

Simulation and Analysis

In this section, we provide an end-to-end simulation of the design time planning and runtime adaptation process, with further analysis on performance aspects. Further constructs to ensure graph database integrity with knowledge base updates are provided.

Action Planning Simulation

Given a high level goal task such as "Pick Ball from Rack using Picker Robot", the first step is to decompose goals into appropriate sub-tasks. The tasks are mapped to appropriate Knowledge Bases (using the member function in Orc), depending on whether they represent actions, targets, robotic components or properties. The following code provides a map of the atomic terms to individual knowledge base elements (Line 12 in Knowledge Resolution). For instance, the term rack is located as a member of the World model knowledge base (Line 15 in Knowledge Resolution). --Input Goals 12 map ( plan, [ " pick " , " ball " , " rack " , " picker " ]) 13 14 Once the appropriate knowledge base elements are recognized, Gremlin queries are used to obtain dependencies from the Graph database. We assume that the knowledge base is pre-populated with property graphs as described in Section 4. Terms used in Fig. 3, such as hasProperty and requires are used in conjunction with Gremlin graph database filtering and complex queries, to populate local robotic knowledge bases (Lines 9-20 in Knowledge Query). While we represent this as explicit queries, alternate implementations may use dialogue engines to extract necessary information from the knowledge base via question-answers [A. Bordes and Weston, 2017][M. Maro and Origlia, 2017]. We make use of the def class declaration that allows us to implement sites within Orc [START_REF] Kitchin | The orc programming language[END_REF], which provides encapsulation similar to classes in object-oriented programming. we make use of the Ref site in Orc, that creates a rewritable storage location. The following Orc code presents these aspects:

--Orc Output ---------------------------------------

--S e a r c h i n g D e p e n d e n c i e s for " rack "

23 val gremlin = gremlin () 24 gremlin.find ( " Name " , " rack " ) | g r e m l i n . h a s P r o p er t y ( " Name " , " Objects " ) | g r e m l i n . r e q ui r e s ( " Name " , "

Map " ) | g r e m l i n . i s O f T yp e ( " Name " , " WorldModel " ) >> World_model?

25 26

--Output -----------------------------------------

27

[ " WorldModel " , ( " Map " , " W ar eh o us eL ay o ut " ) ,

28

( " Objects " , " Obje ctLocati on " , " O b j e c t P r o p e r ti e s " ) , " rack " ]

Action planning with procedures outlined in Section 5 can now be performed, with the high level goals being enacted through decomposition. Queries are made to the knowledge base to determine if the query terms are located in the world or target models, the absence of which triggers perception (Lines 9-11 in Action Planner). Similarly, queries for the robot and action models are triggered, which can trigger runtime exceptions such as lack of robot capabilities (Lines 14-17 in Action Planner). We also introduce a function to trigger re-planning replan action, that looks for exceptions and may add capabilities such as a new robot model or action template (Lines 20-21 in Action Planner). The following Orc code presents these aspects: The output of a typical action plan is now presented, which is input goals that are similar to those planned at design time. Once the query results from various knowledge models are obtained, the action can be performed that includes navigation, manipulation and task completion (Lines 14-15 in Design Time Simulation). Such an execution is straightforward as neither external perception or exceptions are triggered. The following Orc code presents these aspects:

1 +++ Action Planner +++ 2 3

--K n o w l e d g e Base, P e r c e p t i o n and E x c e p t i o n P o i n t e r s

1 +++ Design Time Simulation +++ ( " ball " , [ " ball " , " cube " ])

--Output -------------------------------------------

12

( " picker " , [ " picker " ])

13

( " rack " , [ " warehouse " , " rack " ]) 14 (( " navigation " , [ " navigation " , " manipulation " , " task " ]) , ( " task " , [ " navigation " , " manipulation " , " task " ]) , ( " manipulation " , [ " navigation " , " manipulation " , " task " ]) )

15

Action Completed pick

Runtime Adaptation Simulation

An important aspect of autonomous robotic deployment is runtime adaptation to changes. The goals are modified with the robot type replaced by mover, action collect and the target object replaced by cylinder. As these requirements are not pre-populated into the graph knowledge base, adaptation and exception handling procedures are triggered in Algorithm 2. We notice that perception is triggered to identify the target cylinder (Lines 12-13 in Runtime Adaptation Simulation). Exceptions are also triggered for the lack of collect actions and mover robot capabilities, that are further added into the knowledge base (Lines 15-21 in Runtime Adaptation Simulation). Post this adaptation, the action execution is completed. The following Orc code presents these aspects:

1 +++ Runtime Adaptation Simulation +++ Updated KB ["collect", "pick", "drop", "assign"] 22 (( " navigation " , [ " navigation " , " manipulation " , " task " ]) , ( " task " , [ " navigation " , " manipulation " , " task " ]) , ( " manipulation " , [ " navigation " , " manipulation " , " task " ]) )

--Output -------------------------------------------

23

Action Completed collect

ROS Smach Code Generation

To deploy the action plans to physical/virtual robots, we make use of the open source ROS Smach6 framework. This is a finite state machine where states and transition of the robot may be described with respect to complex tasks. We auto-generate this from the Orc task list, by referencing robot capabilities, world model and task templates seen in Fig. 3. An example of the ROS Smach code generated is presented below, that produces the output of each state transition as succeeded, aborted or preempted. The PERCEPTION task, if successful is followed by ROBOT ARM MOVEMENT; else an abort or preemption is triggered: Integrating RoboPlanner with physical robotic simulator for action planning, such as that shown in Fig. 5, is then done using ROS API calls mapped to each ROS Smach task. ROS also provides ROS bridge interfaces to call physical robot sensor-actuator APIs via the task planning framework. This presents an end-to-end system for autonomous robot action planning (refer to Fig. 2), with knowledge integration, design time action planning, runtime execution and adaptation.

Performance Analysis

Given that we propose the use of knowledge bases and Gremlin graph queries to retrieve the language, performance impact of the queries must be analyzed. This is specially important in the case of Industry 4.0 deployments, where automation is intended to improve throughput. To estimate query and update times in OrientDB graph databases, we run the following stress test on a Linux workstation with 4 core i5-6200U CPU @ 2.30GHz, 4 GB RAM, which simulates the hyperconnected graph traversal over 50 nodes: The average graph traversal latency is seen to be around 211 milliseconds, that outperforms conventional perception and object recognition algorithms (2300 milliseconds in [Zhang and et al., 2016]). Using these mean values for exponentially distributed latency outputs, Monte-Carlo runs are performed over 20, 000 runs. Fig. 6 demonstrates outputs for various cases with the Knowledge Base having 100%, 90% and 70% of the action planning information (triggering perception and exception handling in case of missing knowledge). For instance, over the base case of 70% plan information in the Knowledge Base, the 95% percentile latency improves by 56.5% (90% queries answered by knowledge base) and by 73.9% (90% queries answered by knowledge base). This demonstrates that continuous learning and runtime updates have a significant impact on autonomous robotic performance. Thus, it is crucial to maintain an updated knowledge base within the RoboPlanner framework.

Graph Database Integrity

While the multi-modal OrientDB satisfies ACID (Atomicity, Consistency, Isolation, Durability) properties for databases, integrity checks are to be maintained when updating the databases. Integrity constraints are rules which define the set of consistent database states or changes of state. Typically, three types of checks are performed [START_REF] Rabuzin | Implementing unique integrity constraint in graph databases[END_REF]:

1. Schema instance: Entity types and type checking integrity.

Referential integrity:

This checks that the nodes and edges are uniquely named and that the edges are provided with labels and start/end vertices.

3. Functional dependencies: Value restrictions on particular attributes. Defining minimum and maximum property value.

These checks are incorporated into the below Orc code for knowledge base updates. We notice that type checking (Line 4 in Database Update Integrity), redundancy [START_REF] Tenorth | Knowrob -a knowledge processing infrastructure for cognition-enabled robots[END_REF] ACT-R/E [START_REF] Trafton | Actr/e: An embodied cognitive architecture for human-robot interaction[END_REF] SOAR [START_REF] Laird | Cognitive robotics using the soar cognitive architecture[END_REF] OpenRobots Ontology (ORO) [START_REF] Lemaignan | Oro, a knowledge management platform for cognitive architectures in robotics[END_REF] Such integrity checks and superior performance aspects can prove useful in other applications such as intelligent chatbots and dialogue engines, where updated knowledge bases and real time responses are crucial.

RoboPlanner

Application

Domains

Cognitive

In summary, our work demonstrates the following:

1. RoboPlanner Knowledge Base module that formally models robotic world models, capabilities, object descriptions and task templates -Fig. 3 and inputs to Knowledge Resolution/Knowledge Query examples in Section 6. 2. RoboPlanner Action Planner that uses design-time queries/updates to knowledge graph databases, including exception handling -Algorithm 2, Fig. 4 and Action Planner/Design Time Simulation examples in Section 6. 3. RoboPlanner Runtime simulation, adaptation and performance analysis of action plans using graph queries -Runtime Adaptation Simulation example in Section 6 and Fig. 6. Executable task templates as ROS Smach codes as presented in Section 6.3. 4. RoboPlanner integrity checks for runtime updates to the knowledge base -Database Update Integrity example in Section 6. Such modules will prove useful across a host of Industry 4.0 deployments invoking autonomous robots.

Related Work

Industry 4.0 deployments [START_REF] Lasi | Industry 4.0. Business & Info[END_REF] propose the use of autonomous robotic entities to complete complex tasks. Commercial deployments have been used in warehouses [Bartholdi andHackman, 2016][Zhang andet al., 2016] to improve throughput of automated tasks. Amazon7 has deployed hundreds of autonomous robots to aid in reducing costs of warehouse logistics [START_REF] Wurman | Coordinating hundreds of cooperative, autonomous vehicles in warehouses[END_REF]. Inspiration is drawn from the use of autonomic computing technologies [Huebscher and McCann, 2008], that allow robotic runtime reconfiguration and adaptation. Architectures with self-aware, self-configuring and self-optimizing capabilities have also been proposed [START_REF] Faniyi | Architecting self-aware software systems[END_REF], that may be applied to such automation frameworks.

This has led to recent research on cognitive robotic systems [START_REF] Levesque | Cognitive robotics[END_REF], with architectures such as RoboEarth [START_REF] Tenorth | Knowrob -a knowledge processing infrastructure for cognition-enabled robots[END_REF] and CRAM [START_REF] Beetz | Cram: A cognitive robot abstract machine for everyday manipulation in human environments[END_REF] being proposed. While a few of these make use of semantic ontologies to represent knowledge, others make use of biological memory models to cache information. A review of cognitive architectures applied in multiple domains such as vision, learning, memory models and robotics have been provided in [START_REF] Kotseruba | 40 years of cognitive architectures: core cognitive abilities and practical applications[END_REF]. Table 2 provides a detailed comparison between RoboPlanner and other cognitive/autonomous robotic architectures. We notice that OWL based ontologies [START_REF] Grimm | Knowledge representation and ontologies[END_REF] and queries using SPARQL/Prolog are heavily used, which suffer from performance deterioration when the knowledge base is large. Automated planners such as ROSPlan [Cashmore and et al., 2015] make use of logical PDDL transitions at task design time, rather than runtime executions. In particular, runtime exception handling and consistent model updates have not been fully considered in these frameworks.

In RoboPlanner, we propose the use of graph databases [START_REF] Angles | Survey of graph database models[END_REF] for knowledge representation, which maintain graph relationships within the database. Efficient graph queries are useful in dialogue and chatbot engines as presented in [M. Maro and Origlia, 2017]. We also propose using the Orc concurrent programming language, that may be use in conjunction with industrial workflow specifications (redacted for double blind review). Aspects of the Orc framework are similar to Hierarchical Task Networks [START_REF] Erol | Htn planning: Complexity and expressivity[END_REF], with complex expressions being sub-divided into atomic tasks. Orc further provides granularity in controlling concurrency, temporal actions and runtime behavior, that is more suited for action planning in robotics. A related programming approach is the GOAL agent programming language [START_REF] Hindriks | Goal: A multi-agent programming language applied to an exploration game[END_REF], that makes use of belief-desire-intention approaches to programming intelligent agents. Aspects of knowledge modeling, action templates and goal functions may be mapped to similar axioms provided in our framework. Such an approach may be extended to multiple autonomous robotic deployments.

Conclusions

Autonomous robots are being increasingly used in Industry 4.0 deployments to solve problems via intelligent adaptive mechanisms. A central tenet in such deployments is eliciting efficient action plans that may be executed at runtime. In this paper, we generate action plans through graph knowledge base queries via the RoboPlanner framework. Knowledge about robotic world models and capabilities are encoded in efficient graph database models, that may be efficiently queried to extract information for task completion. Using the concurrent programming language Orc, action plans are generated that can handle robotic runtime exceptions and perception information. End-to-end design/runtime simulations and performance analysis demonstrate the advantages of maintaining the robotic knowledge base.

Figure 1 :

 1 Figure 1: Automation for Warehouse Pick & Place Tasks.

Figure 2 :

 2 Figure 2: RoboPlanner Design/Runtime Execution Modules.

Figure 3 :

 3 Figure 3: Knowledge Property Graphs for Autonomous Robots.

  Fig. 3e provides an example of the Warehouse world model, which hasProperty Map and Object. In order to extract the property of Object Location requires a Map of the area. Fig. 3c provides properties of objects in the world model, including their Location, Shape and Contour Map.

  gremlin> g.V.map ==>{Name=WorldModel} ==>{Name=Objects, Properties=ObjectProperties, Location=ObjectLocation} ==>{Name=Warehouse} ==>{Name=Map, Rack=RackConfig, Layout=WarehouseLayout, Aisles=AislesConfig} gremlin> g.E ==>e[#26:0][#10:0-isOfType->#9:0] ==>e[#29:0][#10:0-hasProperty->#11:0] ==>e[#30:0][#10:0-hasProperty->#9:1] ==>e[#33:0][#9:1-requires->#11:0]

  gremlin> g.v('10:0').Name ==>Warehouse gremlin> g.v('10:0').bothE ==>e[#26:0][#10:0-isOfType->#9:0] ==>e[#29:0][#10:0-hasProperty->#11:0] ==>e[#30:0][#10:0-hasProperty->#9:1]

  gremlin> g.v('10:0').outE.inV.name.path ==>[v[#10:0], e[#26:0][#10:0-isOfType->#9:0], v[#9:0], null] ==>[v[#10:0], e[#29:0][#10:0-hasProperty->#11:0], v[#11:0], null]

  a,b ) = Ift member ( a,head ( b ) ) >> Println ( a + " in " + head ( head ( b ) ) ) | Iff member ( a,head ( b ) ) >> search ( a,tail ( b ) ) 9 def plan ( a ) = search ( a,b ) 10 11

  e f e r e n c e store for r e t r i e v e d data v,D ) = g.V.has ( v,D ) .map >v> 11 append_model ( v ) 12 def hasProperty ( v,D ) = g.V.has ( v,D ) .outE 13 ( ' hasProperty ') .inV.map >v> append_model ( v ) 14 def requires ( v,D ) = g.V.has ( v,D ) .outE 15 ( ' requires ') .inV.map >v> append_model ( v ) 16 def isOfType ( v,D ) = g.V.has ( v,D ) .outE 17 ( ' isofType ') .inV.map >v> append_model ( v ) 18 def produces ( v,D ) = g.V.has ( v,D ) .outE 19 ( ' produces ') .inV.map >v> append_model ( v )

  for world and object t e m p l a t e s , with p e r c e p t i o n 9 def query1 ( v,db ) = Ift member ( v,db ) >> ( v,db ) | Iff member ( v,db ) >> perception.p := v >> perception.p? 10 def world ( w ) = query1 ( w, world_model ) 11 def target ( o ) = query1 ( o, o bj ec t_ t em pl at e ) 12 13 ---Queries for robot c a p a b i l i t i e s and a c t i o n s , with e x c e p t i o n s 14 def query2 ( v,db ) = Ift member ( v,db ) >> ( v,db ) | 15 Iff member ( v,db ) >> exception.ex := v >> add_c a p a b i l it i e s ( v,db ) 16 def robot ( r ) = query2 ( r, robot_cap ) 17 def action ( a ) = query2 ( a, task_template ) >> ( query2 (( " navigation " , " task " , " manipulation " ) ,robot_algo ) ) | replan_action ( a ) ) 18 19 --R e p l a n n i n g p r o c e d u r e s for runtime e x c e p t i o n s 20 def add_capabi l i t i es ( v,db ) = merge ( db, [ v ]) 21 def replan_action ( a ) = Ift ( exception.ex? = null ) >> stop | Iff ( exception.ex? = null ) >> exception.ex := null >> action ( a )

  picker " ) | action ( " pick " ) | object ( " ball " ) | world ( " rack " ) 5 6

  mover " ) | action ( " collect " ) | target ( " cylinder " ) | world ( " rack " ) 5 6

Figure 5 :

 5 Figure 5: KUKA Picker Robot API Call Integration via ROS Smach.

Figure 6 :

 6 Figure 6: Latency Measurements dependent on Knowledge Base Queries.

  Starting workload GSP (concurrencyLevel=4)... -Workload in progress 100% [Shortest paths blocks (block size=50) executed: 50/50] -Total execution time: 2.768 secs -Executed 50 shortest paths in 2.762 secs -Path depth: maximum 8, average 5.286, not connected 0 -Throughput: 18.103/sec (Avg 55.240ms/op) -Latency Avg: 211.996ms/op (58th percentile) -Min: 55.838ms -99th Perc: 576.653ms -99.9th Perc: 576.653ms -Max: 576.653ms -Conflicts: 0

  i n f o r m a t i o n 4 type world_model = { . Name :: String, Colour :: String, Location :: ( N u m b e r , N u m b e r , N u m b e r ) . } 5 val new_world_ mo de l = Dictionary () a n c y _ ch e c k ( key,value ) = Ift ( key = value ) >> false | Iff ( key = value ) >> true 11 def value_check ( key,range ) = Ift ( member ( key,range ) ) >> true | Iff ( member ( key,range ) ) key,value ) = Read ( key ) >aa> ( i n t e g r i t y . r e d u n d a n c y _ c h e c k ( key,value ) , i n t e g r i t y . v a l u e _ c h e c k ( key,value ) ) >> n ew _ w o r l d _ m o d e l . a a := value 16 stop

Algorithm 1 :

 1 Stateful Intelligent Robotic Agent. Knowledge Base ← Update(Knowledge Base, Perception); 5 Action ← Choose-Best-Action(Knowledge Base); 6 Robot State ← Update(State, Action); 7 Knowledge Base ← Update(Knowledge Base, Action);

1 Input: Robot Perception; Knowledge Base; Robot State; 2 Output: Robot Action; 3 Robot State ← Interpret(Perception); 4

  picker | pick | ball | rack is decomposed iteratively to complete the task. Queries to the knowledge base enable generating 5 https://orc.csres.utexas.edu/ Input Goal; Knowledge Base[World Model, Object Templates, Task Descriptions, Robot Capability, Algorithms]; 2 Output: Action Plan; 3 Goal ← Verify(Input Goal, Knowledge Base[Task Descriptions]);

	Algorithm 2: Generating Action Plans for Goals via Knowledge Bases.
	4 Subgoals ← Decompose(Goal, Knowledge Base[Task
		Descriptions]);
	5 for each Subgoal do
	6	(Action?, Target?) ← Requirements(Subgoal);
	7	(Target?, Object Attributes?, Capability?) ←
		Requirements(Action);
	8	Object Attributes? ← Requirements(Target);
	9	if Object Attributes? is a member of Knowledge
		Base[World Model] then
		Object Attributes ← Query(Object Attributes?,
		Knowledge Base[World Model]);
		else
		if Object Attributes? can be obtained by Perception
		then
		Object Attributes ← Perception(Object
		Attributes?, Knowledge Base[World Model, Robot
		Capability, Perception Algorithms]);
		else
		Exception ← Object Attributes?
	10	if Target? is a member of Knowledge Base[Object
		Templates] then
		Target ← Query(Target?, Knowledge Base[Object
		Templates]);
		else
		if Target? can be obtained by Perception then
		Target ← Perception(Target?, Knowledge
		Base[World Model, Robot Capability, Perception
		Algorithms]);
		else
		Exception ← Target?
		Attributes,
		Capability, Knowledge Base[Navigation/Manipulation
		Algorithms]);
		else
		Exception ← Action?

1 Input: 11 Capability ← Query(Capability?, Knowledge Base[Robot Capability]); 12 if Capability satisfies Action then Action ← Query(Action?, Target, Object 13 if Exception is null then Action Plan ← Update(Action); else Trigger Re-planning of Subgoal 14 return Action Plan satisfying Input Goal;

Table 2 :

 2 Autonomous / Cognitive Robotic Architectures.

		CRAM	[Beetz
		et	al.,	2010]
	Feature Modules	RoboEarth

  Line 0 in Database Update Integrity) and valid range of properties (Line 11 in Database Update Integrity) are included. When a robot produces a runtime update, the site update node(key,value) checks for integrity before pushing it to the knowledge base (Line 15 in Database Update Integrity).

		Service
		Robots, Knowledge	Human-Robot	Autonomous Mobile	Cognitive	Service	Autonomous
		Sharing	among	Coordination		Robots	Robots	Robots
		Robots
				Declarative knowl-			Graph Databases
	Knowledge Base	KnowRob [Tenorth and Beetz, 2013] OWL Ontologies	edge memories); (fact-based Proce-dural knowledge (rule-based memo-	Semantic Models -Symbolic Memory and Episodic	ORO OWL and RDF Triplestore	with Model, Capabilities, World Robot Al-gorithms and
				ries)				Task Templates
	Knowledge Queries	Knowrob queries, that can be (Prolog) extended to other ontology queries	High level model in-teractions	STRIPS [Russell and Norvig, 2015] like decision procedures	SPARQL Queries	Gremlin edge Queries	Knowl-Graph
		CRAM Plan Lan-					Orc Specifications
	Action Planning	guage (CPL) allow-ing concurrent, par-	Intentional module	(Goal)	Procedural memory module	CRAM integration with logical rules	with Base queries; ROS Knowledge
		allel processes						Smach Code
	Runtime tion Handling Excep-	COGNITO reason-ing system that processes failure traces	Utility based re-wards; Visual and Aural modules	Reenforcement Learning	Human expert inter-vention	Adaptation exception dling modules han-and
	Runtime Knowl-edge Base Up-dates	No Explicit Mention	Knowledge updated	chunks	Chunks of memory data updated	RDF Triple updates with consistency checks	Graph update with in-database tegrity checks
	Performance Eval-uation	No Explicit Mention	Accuracy tions with respect of Ac-to World Model changes	Cognitive Reactivity measured	ORO Server perfor-mance evaluation (updation, queries)	Graph performance, ex-database ception handling delay
	of input data (

1

+++ Database Update Integrity +++

https://www.kuka.com

https://orientdb.com/

http://tinkerpop.apache.org/

http://wiki.ros.org/smach

https://www.amazonrobotics.com/


A H Clément

A Babarit

Discrete control of resonant wave energy devices

Keywords: wave energy, wave energy converter, wave energy device, discrete control

Aiming at amplifying the energy productive motion of wave energy converters (WECs) in response to irregular sea waves, the strategies of discrete control presented here feature some major advantages over continuous control, which is known to require, for optimal operation, a bidirectional power take-off able to re-inject energy into the WEC system during parts of the oscillation cycles. Three different discrete control strategies are described: latching control, declutching control and the combination of both, which we term latched-operating-declutched control. It is shown that any of these methods can be applied with great benefit, not only to mono-resonant WEC oscillators, but also to bi-resonant and multi-resonant systems. For some of these applications, it is shown how these three discrete control strategies can be optimally defined, either by analytical solution for regular waves, or numerically, by applying the optimal command theory in irregular waves. Applied to a model of a seven degree-of-freedom system (the SEAREV WEC) to estimate its annual production on several production sites, the most efficient of these discrete control strategies was shown to double the energy production, regardless of the resource level of the site, which may be considered as a real breakthrough, rather than a marginal improvement.

Introduction

Many different principles have been conceived to harvest the energy of water waves [START_REF] Clément | Wave energy in Europe: current status and perspectives[END_REF][START_REF] Falcão | Wave energy utilization: a review of the technologies[END_REF]. Among them, wave energy converters (WECs), which exploit the motion of a body in response to the waves, generally exploit one or several mechanical resonances of the system to enhance their performances. Basically, they are mechanical oscillators, thus performing better as the wave frequency approaches their natural frequency. However, because random seas have a distributed frequency spectrum, the efficiency of these devices may fall rapidly in narrow-banded sea states. One of the means to counteract this loss is to equip the device with an active controller, able to adapt the response of the mechanical system to the continuously varying wave properties.

In the early 1980s, Budal & Falnes [START_REF] Budal | Interacting point absorbers with controlled motion[END_REF] showed that, for point absorbers in long waves, one condition for maximizing the energy production is to keep the velocity in phase with the excitation force. As a result, they introduced a special kind of phase control, called latching control, in order to achieve this phase condition on a heaving buoy WEC. Latching consists of locking the motion of the body at the instant when its velocity vanishes at the end of one oscillation, and waiting for the most favourable instant to release the body. The determination of the duration during which the system remains locked is the problem to be solved. The optimal solution is non-causal relative to the wave excitation, but can be found in the frequency domain where one knows the future, which, by definition, is the same as the past. In the real world, which is in the time domain, however, sub-optimal solutions must be sought to overcome this difficulty. The main advantage of latching control is that it is passive, which means that it does not need reactive energy to be fed into the system. Latching has been the subject of many studies in the last few years, showing the capability of this discrete control to substantially increase the amount of absorbed energy in regular and irregular waves. Most of these studies dealt with a one degree of freedom (1 d.f.) heaving buoy WEC [START_REF] Falnes | Laboratory experiment on heaving body with hydraulic power take-off and latching control[END_REF][START_REF] Valerio | Optimization of wave energy extraction with archimedes wave swing[END_REF][START_REF] Falcão | Phase control trough load control of oscillating-body wave energy converters with hydraulic PTO systems[END_REF], but some of them also considered several d.f. WECs [START_REF] Korde | Latching control of deep water wave energy devices using an active reference[END_REF][START_REF] Babarit | Latching control of a power take off oscillator carried by a wave activated body[END_REF][START_REF] Babarit | Optimal latching control of a wave energy device in regular and irregular waves[END_REF]. Following a review of some general results obtained with latching, we focus here on the possibility to extend its applicability throughout the whole frequency range, beyond the first natural frequency of the system.

Declutching control (also called unlatching in Salter et al. [START_REF] Salter | Power conversion mechanisms for wave energy[END_REF]) consists of uncoupling the power take-off (PTO), which means that the PTO force is set equal to 0 in the equations, during some portion of the power cycle. As with latching control, it allows the generation of parametric resonance in the response of the mechanical system, and hence an improvement in energy production. It is even simpler than latching control to implement practically, since it requires only valves in the case of a hydraulic PTO [START_REF] Josset | A wave to wire model of the power take off unit of the SEAREV wave energy converter[END_REF] or a switch in the case of an electric PTO, instead of mechanical brakes or locks as required for latching control. Declutching has already been considered by Justino & Falcao [START_REF] Justino | Active relief valve for an OWC wave energy device[END_REF], using relief valves, for the Portuguese oscillating water column in Pico. However, declutching control was used in the Pico study in order to prevent damage in the system rather than to improve the energy production. In Babarit et al. [START_REF] Babarit | Simulation of the SEAREV wave energy converter with a by-pass control of its hydraulic power-take-off[END_REF], declutching was applied to the SEAREV WEC, and it was shown that such control can lead to a large increase in energy production.

In the present study, we additionally consider the combination of latching control and declutching control in a new discrete control mode called latchedoperating-declutched (LOD). We will apply it successively to (i) a mono-oscillator WEC and (ii) a bi-oscillator WEC, both in regular and irregular wave conditions. Even in regular waves, solutions and simulations are performed in the time domain, which allows us to find non-harmonic solutions. Practically, the switching sequences are derived by the optimal command method, which was first applied to the wave energy area by Hoskin & Nichols [START_REF] Hoskin | Optimal strategies for phase control of wave energy devices[END_REF]. Simulations in regular and irregular waves show that this new three-state mode of control enhances dramatically the performance of both two-state control philosophies (latching or declutching).

A mono-oscillator system (a) Mathematical model

A linear approach will be adopted here for modelling the hydrodynamic forces, under the usual assumptions of a perfect fluid, small wave steepness and small body motions. We will start by considering the simplest generic wave energy device, a heaving cylinder of mass m, as described in figure 1. The body is able to heave in response to the excitation of the waves, all the other motions being perfectly restrained. The vertical motion of the cylinder is measured by the variable x. All the restoring forces (hydrostatic and mooring) are gathered and represented by a simple linear spring of stiffness k. The PTO unit is represented as a simple linear damper of coefficient b. Under the above assumptions, the behaviour of the body in waves is governed by the following integro-differential equation:

(m + m ∞ ) ẍ + t 0 ẋ(t)K (t -t) dt + b ẋ + kx = F ex (t), (2.1)
in which we recognize the so-called Cummins decomposition [START_REF] Cummins | The impulse response function and ship motions[END_REF] of the radiation forces into an instantaneous added mass term m ∞ ẍ, and a memory term expressed by a convolution product t 0 ẋ(t)K (tt) dt. The kernel K of this integral, generally named the impulse response function or sometimes the retardation function, depends fundamentally on the shape of the body. It has been computed directly in the time domain by the seakeeping boundary-element method computational codes ACHIL3D [START_REF] Clément | Using differential properties of the Green function in seakeeping computational codes[END_REF]. In the present time-domain linear formulation, the free motion x(t) of the device may be calculated by integrating (2.1) for a given history of the wave excitation forcing term F ex (t), and any given initial conditions x(0) and ẋ(0). But the convolution integral prevents the use of compact methods developed in systems theory to work with the state equation representation of linear systems. In order to get such a form for the system in (2.1), the impulse response function K is first approximated by a sum of complex exponential functions using Prony's method [START_REF] Prony | Essai expérimental et analytique sur les lois de la dilatabilité des fluides élastiques et sur celles de la force expansive de la vapeur de l'eau et de la vapeur de l'alcool à différentes températures[END_REF] as detailed by Duclos et al. [START_REF] Duclos | Absorption of outgoing waves in a numerical wave tank using a self-adaptive boundary condition[END_REF],

K (t) N j=0 a j e b j t , ( 2.2)
where (a j , b j ) j=0,N are a set of complex coefficients. Since the radiative impulse response function for such a simple floating body has a regular behaviour, five to six pairs of complex conjugate coefficients generally give an excellent approximation. Given this approximation, the convolution product t 0 K (tt) ẋ(t) dt appearing in (2.1) is itself approximated by a sum of additional complex radiative states I j , where each I j (t),

I (t) = N j=0 I j = t 0 K (t -t) ẋ(t) dt, (2.3
I j (t) = t 0 ẋ(t)a i e b i (t-t) dt, ( 2.4)
is the solution of a simple ordinary differential equation

İ j = b j I j + a j ẋ. (2.5)
Denoting the real and imaginary parts of the complex state I j as I R i and I I i , respectively, equation (2.1) can then be replaced by a system of ordinary differential equations,

(m + m ∞ ) ẍ(t) + N i=0 I i (t) + b ẋ(t) + kx(t) = F ex (t), İ R i (t) = b R i I R i (t) -b I i I I i (t) + a R i ẋ(t) and İ I i (t) = b R i I I i (t) + b I i I R i (t) + a I i ẋ(t). ⎫ ⎪ ⎪ ⎪ ⎪ ⎪ ⎬ ⎪ ⎪ ⎪ ⎪ ⎪ ⎭ (2.6)
Defining an extended state vector as

X = [ x ẋ I R 1 I I 1 ... I R N I I N ]
T incorporating the additional wave radiation states I i , the system of equations in (2.6) can now be written under the classical state equation matrix form Ẋ(t) = AX(t) + B(t).

(2.7)

The fully developed form of the matrices A and B are found in Babarit & Clément ([9], eqns [START_REF] Evans | Some analytic results for two and three dimensional wave-energy absorbers[END_REF][START_REF] Falnes | Ocean waves and oscillating systems[END_REF]). Given the known excitation force in a time interval [0, T ], and the initial conditions X(0), the linear system (2.7) is then solved numerically by any standard integration algorithm, e.g. Runge-Kutta as used here. The mean power P extracted by the PTO over [0, T ] is then given by The ultimate goal of the control strategy discussed here is to maximize the power extracted through the PTO of the system in a realistic situation, namely in irregular waves. As described above, the device is a linear mechanical oscillator, set into motion by the external forcing term F ex (t). It has a single natural resonant frequency u 0 .

P = 1 T T 0 b ẋ2 (t) dt. ( 2
The optimization of such a device in regular waves (i.e. the frequency domain) has been performed previously [START_REF] Evans | A theory for wave-power absorption by oscillating bodies[END_REF][START_REF] Evans | Some analytic results for two and three dimensional wave-energy absorbers[END_REF][START_REF] Falnes | Ocean waves and oscillating systems[END_REF]. It consists of determining the PTO characteristics (k(u), b(u)) that maximize the mean extracted power when the incident waves are plane monochromatic waves at a frequency u. In the present case of a 1 d.f. heaving device, with a damping coefficient CA 33 (u) and added mass coefficient CM 33 (u), it is easy to establish that -if only the PTO coefficient b can be tuned, then its optimal value is b opt,k = CA Since, the coefficients b and CA are essentially real, this last relation (2.12) indicates that, in the optimal device configuration, the velocity of the buoy and the excitation force are exactly in phase. This is the basis of the so-called phasecontrol proposed by Budal & Falnes [START_REF] Budal | Optimum operation of improved wave-power converter[END_REF] in the late 1970s. The above relations have been derived in the frequency domain, and are valid when and only when the waves are regular and monochromatic, and the system is linear.

From these results, we can derive a basic principle for the control we seek to develop and ultimately use in the real world (i.e. in the time domain), by trying to continuously vary the PTO parameters k and b as the wave frequency u varies continuously in the time domain, in order to maintain this state of resonance (phase condition) and to satisfy the condition on amplitude (2.12). In fact, this continuous optimal control cannot be transposed straightforwardly in the current form, in the time domain, for several reasons.

-In the time domain and irregular waves, F ex will be a random signal and the 'phase' concept is meaningless with such a signal. One can possibly require the two signals to have their peaks at the same instant, but not to be 'in phase', which should mean that they are proportional to each other. -An 'instantaneous frequency' needs to be defined if one wants to directly apply the above rules in the time domain; this has been attempted with limited success by some authors [START_REF] Chatry | Self-adaptive control of a piston wave absorber[END_REF], using Kalman filtering or other signal processing tools. Otherwise, one must revert to the time domain using the inverse Fourier transform of the frequency domain results, which is the only mathematically justifiable way to proceed. -When proceeding via the inverse Fourier transform route, we are led, at last, to the fundamental property that the optimal controller derived in the time domain is not causal [START_REF] Naito | Wave energy absorption in irregular waves by feedforward control systems[END_REF], which means that one must know the future wave excitation force to adapt the PTO in real time.

Another drawback of a continuous optimal control is the fact, established rigorously in the frequency domain, that the total power flowing out of the system through the PTO is not always positive during a cycle, meaning that one must be able to feed reactive power into the device in order to achieve optimal control. This is not really a great problem as long as we consider the problem in the mathematical space, but it could become a serious drawback when we have to devise the technology of the PTO to transfer energy in both directions, inward and outward. The required level of reactive power can become very large, sometimes 10 times larger than the mean output power. If the efficiency of the converter is not sufficient, then all the energy production could potentially be dissipated via energy losses.

In contrast to this continuous control approach, the discrete control modes that we are presenting in the following sections circumvent this implementation inconvenience.

(c) Latching control

Basically, we consider that we have added a technological component (e.g. a brake) to the original system, which is able to lock the moving body in its current position x 0 at the exact instant t 0 when its velocity vanishes ( ẋ(t 0 ) = 0), and is able to release it instantaneously on demand (see the lock on figure 2). Proposed in 1980 by Budal & Falnes [START_REF] Budal | Interacting point absorbers with controlled motion[END_REF], this control strategy was introduced primarily to satisfy the phase condition between excitation force and body velocity implied by (2.12), in the cases where the wave frequency does not match the natural frequency of the system. It was called phase-control for this reason. More precisely the principle was to 'slow down' the natural response of the heaving buoy when the wave frequency was lower than the natural frequency, in order to have the velocity and the force reaching their maxima at the same time. This principle was validated by experiments in the Norwegian University of Science and Technology in the early 1980s. A significant amplification of the WEC motion was obtained (by parametric resonance) compared with the uncontrolled system, without feeding any energy into the system, and expending only a tiny fraction of the available energy in the locking system.

(i) Optimized latching control of a 1 degree-of-freedom system in regular waves

There are two important instants in this control mode, one when the system is switched from operating mode to latched mode, the other when the body is released ('unlatched') from the latched mode to return to the operating mode. This is why this control mode is termed discrete. The latching instant is determined by the system itself when the velocity vanishes, the remaining practical problem one has to solve to implement latching control in a real system is to determine the release time (or equivalently the duration of the latched state).

If we assume a regular monochromatic plane wave, then the problem can be solved quasi-analytically by a time-domain approach, as shown in Babarit et al. [START_REF] Babarit | Comparison of latching control strategies for a heaving wave energy device in random sea[END_REF]. Under this assumption, with the wave excitation force given by F ex (t) = |F ex | cos(ut + 4 0 + 4 ex ), the system is simply switched between two state equations, namely,

(m + m ∞ ) ẍ(t) + N i=0 l i I i (t) + b ẋ(t) + kx(t) = |F ex | cos(ut + 4 0 + 4 ex ), İ R i (t) = b R i I R i (t) -b I i I I i (t) + a R i ẋ(t) and İ I i (t) = b R i I I i (t) + b I i I R i (t) + a I i ẋ(t), ⎫ ⎪ ⎪ ⎪ ⎪ ⎪ ⎬ ⎪ ⎪ ⎪ ⎪ ⎪ ⎭ (2.13)
or in matrix form

Ẋ = AX + (B e i(ut+4 0 ) ) (2.14)
in operating mode, and the second system

ẍ(t) = 0, ẋ(t) = 0, İ R i (t) = b R i I R i (t) -b I i I I i (t) and İ I i (t) = b R i I I i (t) + b I i I R i (t), ⎫ ⎪ ⎪ ⎪ ⎪ ⎬ ⎪ ⎪ ⎪ ⎪ ⎭ (2.15) or Ẋ = A X + (B e i(ut+4 0 ) )
when the system is latched. The analytical solution method for the determination of the optimal latching delay d, for various wave frequencies [START_REF] Babarit | Comparison of latching control strategies for a heaving wave energy device in random sea[END_REF], is based on the general solution of (2.14) in the time domain under a sinusoidal forcing, which can be expressed as

X(t) = exp(A(t -t i ))X i + ((Ie iu(t-t i ) -exp(A(t -t i ))) × (iuI -A) -1 B e i(ut i +4 0 ) ), (2.16)
with exp(At) the notation for the matrix exponential, and X i = X(t i ) the initial condition. Requiring the solution to be periodic with a period as a multiple of the wave period 2p/u when a steady-state regime is established, we found two sets of optimal solutions. The first one corresponds to these responses where the latching occurs at alternating positions x max , -x max , x max , . . .; we call it alternating maxima modes. In these modes, the period of the response, given by T out = (2k + 1)2p/u, k ∈ [0, 1, 2, . . .], is successively T , 3T , 5T , . . . (figure 3). Therefore, using the time-domain formulation and representation of the solution, we have found latching modes with periods greater than the excitation period. These sub-harmonic modes are optimal solutions when the waves have a frequency higher than the natural frequency u 0 of the device. More precisely, the device period T out is three times the wave period T if u 0 < u < 3u 0 , five times when 3u 0 < u < 5u 0 , and so on. Other sets of optimal solutions can be found when we require the body to be always latched at the same position x max , in a mode we have called equal ending ramps (figure 4). The period of the optimal response is now T out = 2lp/u, l ∈ [0, 1, 2, . . .] in a series T , 2T , 3T , . . . These new sub-harmonic modes allow us to extend the application of latching control beyond the natural frequency of the system, with a real gain in the heaving amplitude of the body. This can be observed in figure 5 showing the absorbed power curve versus wave frequency for a 1 d.f. heaving cylinder of 5 m radius, 10 m draft, m = 360 T, k = 240 kN m -1 and b = 27 kN s -1 m -1 .

In figure 5, we have also plotted the maximum power absorbed by an axisymmetric body, P = rg 3 /4u 3 , as established by Evans [START_REF] Evans | Some analytic results for two and three dimensional wave-energy absorbers[END_REF] and Falnes [START_REF] Falnes | Ocean waves and oscillating systems[END_REF]. One can clearly see the benefit of latching control even beyond the natural frequency (0.62 rad s -1 ), provided one switch to the first sub-harmonic mode with T out = 3T . One must also realize that when the system runs in such a sub-harmonic mode, the situation is drastically different from the optimal harmonic response described in §2b, based on a frequency-domain approach and, in particular, that the optimal condition consisting of phasing the body velocity with the excitation force is no longer valid.

(ii) Optimized latching control of a 1 degree-of-freedom system in irregular waves

Since the ultimate goal is to apply such discrete control strategies in real sea conditions, let us now turn to the case of irregular waves for this simple 1 d.f. system. As usual in such a linear approach, we shall consider irregular wave trains as the sum of regular waves. The free-surface elevation, h(t) is a sum of N c elementary sinus functions whose amplitudes (a j ) j=1,N c are derived from the standard Pierson-Moskowitz energy spectrum [START_REF] Molin | Hydrodynamique des structures offshore[END_REF] and whose phases (4 j ) j=1,N c are chosen randomly. The problem to be solved remains the same as above, namely finding the optimal latching delay d that determines the instant when the body is released from the latched mode (2.15), and the system is switched to the operating mode (2.13). As stated above, it is now impossible, in this context of irregular waves, to rigorously define a phase for the signals involved, and therefore to base the release criterion on a phase condition.

The solution of the problem can be found using optimal command theory and Pontryagin's principle, as first suggested in this context by Hoskin & Nichols [START_REF] Hoskin | Optimal strategies for phase control of wave energy devices[END_REF].

A time series of the irregular wave excitation force F ex (t) is supposed to be known over an interval of time [0, T ]. The latching force that is necessary to cancel the acceleration is represented here by an infinite damper added in parallel to the PTO damper at the instant when one wants to lock the system. A control variable u ∈ [0, 1] is used to command this force, cancelling it when u = 0. Therefore, F latch = -uG ẋ. When G is infinite and u = 0, we recover an absolute latching control with this additional term [START_REF] Kamenski | Optimal switching control of an absorber ocean energy device[END_REF]. In simulation, we have rather implemented a weaker version of latching with G large compared with the PTO damper, but not infinite. When G is large enough, the correct behaviour of the latching sequences, as described above, is obtained numerically. With this implementation of latching, systems (2.6) and (2.15) can be merged into a single system,

(m + m ∞ ) ẍ(t) + N i=0 I i (t) + (b + Gu(t)) ẋ(t) + kx(t) = F ex (t), İ R i (t) = b R i I R i (t) -b I i I I i (t) + a R i ẋ(t) and İ I i (t) = b R i I I i (t) + b I i I R i (t) + a I i ẋ(t). ⎫ ⎪ ⎪ ⎪ ⎪ ⎪ ⎬ ⎪ ⎪ ⎪ ⎪ ⎪ ⎭ (2.17)
The problem is now to find u(t), which maximizes the energy recovered through the PTO damper during the complete time range [0, T ],

max u E(u) = T 0 b ẋ2 dt. (2.18)
Let X(t) be the generalized state vector of the system in (2.17) when written in the compact form Ẋ(t) = f(t, X, u). A Hamiltonian H can be defined by

H = b ẋ2 + l t f, (2.19)
where l t is an adjoint vector solution of the adjoint system

li = - vH vX i (t, X, u), ( 2.20)
with final conditions l i (T ) = 0. From (2.17) and (2.19), we have that

H = b ẋ2 + l 1 ẋ + l 2 F ex - 1 m + m ∞ (b + Gu) ẋ + kx + N i=1 I R i (2.21) + N i=0 l R i (b R i I R i -b I i I I i + a R i ẋ) + l I i (b R i I I i + b I i I R i + a I i ẋ), (2.22)
and hence the adjoint system is given as

l1 = k m + m ∞ l, l2 = -2b ẋ -l 1 + l 2 (b + Gu) - N i=1 (l R i a R i + l I i a I i ), lR i = -l R i b R i -l I i b I i + l 2 m + m ∞ and lI i = l R i b I i -l I i b R i . ⎫ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎬ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎭ (2.23)
The solution principle is based on a double time-stepping integration of the two dynamical systems (2.17) from 0 forward to T , and (2.23) backwards, from T (with initial conditions l i (T ) = 0) to 0. During this backward integration, we determine u(t) by applying Pontryagin's principle, requiring that the optimal command should maximize the Hamiltonian at each time t ∈ [0, T ]. Since the Hamiltonian is a linear function of the command variable u (see equation (2.21)), its maximization will require u to be either 0 or 1 depending on the sign of its coefficient, so

u = 1, if (-l 2 G ẋ) < 0, 0, otherwise. (2.24)
This procedure of forward and backward integration is initialized by a given sequence for u(t), and then iterated up to final convergence of the absorbed energy over [0, T ] [START_REF] Babarit | Optimal latching control of a wave energy device in regular and irregular waves[END_REF].

Two noteworthy properties follow from (2.24).

-The optimal control is a 'bang-bang' control, u being either 0 or 1, but never an intermediate value in [0, 1]. Thus, in that case, discrete control is superior to a continuous control. -The system switches every time ẋ vanishes and changes its sign. This (re)establishes the latching control principle mathematically, by applying the Hamilton-Pontryagin principle, as proposed heuristically by Budal & Falnes [START_REF] Budal | Optimum operation of improved wave-power converter[END_REF] in the late 1970s.

A simulation of latching control in irregular waves is illustrated by figure 6. One can easily see the magnification of the response, especially for t > 260 s. It seems that after this time value, a regime of parametric resonance is reached by the system. Since the absorbed energy is a quadratic function of the motion amplitude, one can understand the interest in getting such an amplified response.

(d) Declutching control

Latching control is not the only way of getting such parametric resonance by switching, at discrete times, between sub-models of the system. Latching was implemented here as switching between a finite and an 'infinite' PTO damping coefficient; namely between b and b + G. We will now instead consider that we have the possibility of uncoupling the PTO at chosen discrete instants. In a real device, this could be done by a by-pass valve, if the PTO is hydraulic [START_REF] Babarit | Declutching control of a wave energy converter[END_REF], or using a power-electronic switch in the case of a direct electrical generator, or via a mechanical clutch. We named this action declutching, as opposed to unlatching, which was used in Salter et al. [START_REF] Salter | Power conversion mechanisms for wave energy[END_REF], but which could also be mistakenly interpreted as the release action during latching control. In declutching control, the system will now switch between a finite and a null PTO damping coefficient, namely between b and 0. Since the switching occurs at discrete instants (to be determined), this control mode also belongs to the class of what we call discrete control. Retaining the heaving cylinder as an example, the system of differential equations will differ from the previous one (2.6) only with respect to the damping term,

(m + m ∞ ) ẍ(t) + N i=0 I i (t) + vb ẋ(t) + kx(t) = F ex (t), İ R i (t) = b R i I R i (t) -b I i I I i (t) + a R i ẋ(t) and İ I i (t) = b R i I I i (t) + b I i I R i (t) + a I i ẋ(t), ⎫ ⎪ ⎪ ⎪ ⎪ ⎪ ⎬ ⎪ ⎪ ⎪ ⎪ ⎪ ⎭ (2.25)
where the command variable v ∈ [0, 1] must be set to 0 to model declutching.

The complete study for latching control was repeated with this new discrete mode of control, following the same steps.

We found that declutching can also be profitable, based on the same principle of modifying (online) the natural frequency of the system to best fit the incoming wave properties. In regular waves, figure 7 shows a simulation of the heaving response of a cylinder computed by integrating (2.25) by a standard (Runge-Kutta) iterative method. In irregular waves, the same methodology was applied, using the Hamiltonian formulation as exposed above, but now applied to (2.25). Again, owing to the linearity of the Hamiltonian with respect to the control variable v, a bang-bang control is shown to be optimal.

Figure 8 shows the response of the cylinder to a sequence of irregular waves. The amplification of the response with declutching observed here is smaller than with latching control. Nevertheless, declutching control alone is still beneficial. As with latching control, declutching has the property that it does not require any energy to be fed into the system. Furthermore, we will see in §2e that the combination of latching and declutching can further enhance the performance of such oscillating wave energy devices.

(e) Latching and declutching applied to a bi-oscillator system

We consider now a more complex wave energy device featuring 2 d.f. as described in figure 9. It is composed of the following.

-A positively buoyant free-surface piercing cylindrical hull of mass m 1 , moored to the sea bottom and restrained to move only in heave motion (measured by x 1 ). We will assume all its other motion to be perfectly restricted. -An internal moving mass m 2 , which can slide without friction along the vertical axis x inside the floating cylinder. Let x 2 be the relative displacement of m 2 from its equilibrium position.

When the system is excited by the wave forces, both the cylinder and the internal mass are set into motion, in such a way that the relative motion between the two parts can be converted into energy by means of an appropriate PTO. We assume, as before, that the PTO can be modelled by a linear spring and damper with coefficients, k 2 and b 2 , respectively. The equations of motion for this 2 d.f. system in the time domain are

m 1 ẍ1 + m 2 ( ẍ1 + ẍ2 ) = F ex -m ∞ ẍ1 - t 0 K (t -t) ẋ1 (t) dt -b 1 ẋ1 -k 1 x 1 and m 2 ( ẍ1 + ẍ2 ) = -b 2 ẋ2 -k 2 x 2 , ⎫ ⎪ ⎬ ⎪ ⎭ (2.26)
where -b 1 is a damping force coefficient associated with an external linearized viscous force; it can be set to zero (b 1 = 0) here without essentially changing the conclusions of the paper; -k 1 is the model for the restoring force applied to the buoy (mooring + hydrostatics), we can further neglect the contribution from the moorings, and k 1 is simply the hydrostatic force given by k 1 = rgS WP ; and -F ex , m ∞ and K have been defined in preceding sections. Using King's approach [START_REF] King | Time domain analysis of wave exciting forces on ships and bodies[END_REF], let K ex (t) be the force response associated with an impulsive elevation on the free surface at a given reference point propagating along the x-axis. Using the superposition principle, the excitation force can be expressed as

F ex (t) = t 0 K ex (t -t)h(t) dt, (2.27)
where h(t) is the free-surface elevation. For simulation, we have considered a 5 m radius and 10 m depth vertical cylinder as the hull of the WEC, figure 9, the mechanical characteristics of which are summarized in table 1.

The equations of motion now become

(m 1 + m ∞ ) ẍ1 + m 2 ( ẍ1 + ẍ2 ) = t 0 K ex (t -t)h(t) dt - N j=0 I j -rgS WP x 1 , (2.28) m 2 ( ẍ1 + ẍ2 ) = -b 2 ẋ2 -k 2 x 2
and İ j = b j I j + a j ẋ1 .

(2.29) Fundamentally, this system comprises two oscillators with two distinct natural frequencies: one for the heaving buoy and the other for the moving mass. These oscillators are coupled through equation (2.28). A Runge-Kutta second-order scheme was used to numerically integrate the equations of motion in order to perform time-domain simulations of this 2 d.f. WEC, with a time step of 0.01 s.

(i) Latching control

In this 2 d.f. device, latching control consists of locking the relative motion x 2 when the velocity ẋ2 vanishes, and releasing it after an optimal time, to be determined. As above, latching control is implemented in the model by adding in the equation of motion a very large damping force on the relative motion x 2 into the equations of motion, as

f latching = -u 1 G(m 2 + m ∞ ) ẋ2 , (2.30)
where u 1 ∈ [0, 1] is the control variable. This weak modelling of latching control may have a lower efficiency than an absolute formulation, but it allows the more convenient implementation of optimal command theory in order to compute the control law in a time-step numerical procedure, as described in previous sections.

It is also more realistic when one wants to model real mechanical components, e.g. brakes, which do not necessarily have instantaneous action. Practically, a delay or lag will exist between the time when the controller decides to apply the control and the time when brakes will hold the body. In order to take this delay into account, we refine the PTO model and we now consider the coefficient G to be time-varying, instead of a constant, solution to the differential equation

G(t) + t latching Ġ(t) = u 1 G 0 .
(

When the controller switches to the latching mode, the control variable u 1 is set to 1, and G(t) grows exponentially to its final value G 0 with a time constant t latching . When the controller switches the system back to the normal operating mode, u 1 is set to 0 and G(t) decays exponentially to 0.

(ii) Declutching control

For the 2 d.f. device, we also consider declutching control, which effectively disconnects the PTO between the buoy and the moving mass at some instants of the motion, which have also to be determined. Again, to take into account possible delays or lags in the response of the real actuators, with a time constant t PTO , we consider the b 2 coefficient as a function of time, as

b 2 (t) + t PTO ḃ2(t) = u 2 B 0 .
(2.32)

For declutching, the control variable is u 2 .

(iii) Latched-operating-declutched: a three-state discrete control mode

As for the single degree of freedom device, and using the same methodology, we have shown that latching control or declutching control are also efficient with the 2 d.f. device. Now, we want to go further, to assess the result that may be obtained by combining these two modes of control. We shall therefore implement a threestate discrete control, where the PTO can be alternatively latched, operating or declutched (we use LOD control as an abbreviation).

Hence, we have defined a three-stage controller, depending on the combinations of the value of the control variables u 1 and u 2 :

-when u 1 = 1, irrespective of the value of u 2 , the system is latched; -when u 1 = 0 and u 2 = 0, the system is operating; and -when u 1 = 0 and u 2 = 1, the system is declutched.

The optimal command method is again used to solve for the determination of the switching times in the time-domain simulation.

Let X = ( x 1 x 2 x 3 x 4 G b 2 I 1 ... I N ) T be the new state vector, with x 3 = ẋ1 and x 4 = ẋ2 . One can rewrite the equation of motion as a standard state equation,

Ẋ = f(t, X, u), (2.33)
with

f(t, X, u) = ⎛ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎝ x 3 x 4 M -1 F e x -k 1 x 1 -N j=0 I j -(b 2 + G(m 2 + m ∞ )x 4 -k 2 x 2 1 t latching (u 1 G 0 -G), 1 t PTO (u 2 B 0 -b 2 ) b 1 I 1 + a 1 x 3 . . . b N I N + a N x 3 ⎞ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎠ , (2.34)
M -1 = 1 (m 1 + m inf )m 2 m 2 -m 2 -m 2 m 1 + m 2 + m inf (2.35)
and the initial conditions X(0) = X 0 .

(2.36)

The optimization problem is to maximize the energy extracted,

max u E = T 0 b 2 ẋ2 4 dt. (2.37)
Now define the Hamiltonian

H = b 2 ẋ2 4 + l t f, (2.38)
where l ∈ R N +6 is the adjoint state vector. l is the solution of the set of adjoint differential equations

li = - vH vX i (X, u), (2.39)
with the final condition l(T ) = 0.

Using equations (2.34) and (2.40), one can show that the Hamiltonian is linear in u, H(t) = h(X, l) + u 1 l 5 + u 2 l 6 .

(2.40)

Hence, the switching instants will be determined by the signs of the adjoint states l 5 and l 6 , and we again have a bang-bang control defined by the protocol

u 1 = 1, if l 5 ≥ 0, 0, if l 5 < 0, (2.41)
and

u 2 = 1, if l 6 ≥ 0, 0, if l 6 < 0. (2.42)
Since the iterative process includes forward and backward integration of (2.34) and (2.39), which are used to determine the optimal latching control law, knowledge of the excitation force during the complete duration of the simulation, up to the final instant T , is required. The decision to latch or declutch the body at a current time t then depends on the future of the system beyond t, which means that this process is anti-causal. Its practical implementation in the real world requires some wave prediction, or necessitates the adoption of sub-optimal causal algorithms for calculating the control law online. These issues are not addressed here, since the aim of this study is simply to assess the ability of the discrete LOD control to increase energy production.

(f ) Latched-operating-declutched control in regular waves

A comparison of the results of time-domain simulations of the motion without control, with latching control, with declutching control and finally, with LOD control is plotted in figure 10. The period of the incident wave is 10 s and its height is 1 m, with the damping coefficient b 2 set to 10000 N m -1 s -1 . The latching coefficient is G 0 = 80 and the time constants for the PTO and the latching are t PTO = t latching = 0.1 s. The control is activated at t = 40 s.

In this example, both latching and LOD control increase the energy production in comparison with the case without control (2.5 kW), whereas the declutching control does not bring any improvement. When the steady state is reached, the mean power produced with LOD control (61.7 kW) is considerably higher than with latching control (23.5 kW). The power is measured here as the slope of the bottom curve: energy versus time. The increase in the amplitude of the relative motion x 2 (the 'working' d.f.) with LOD control is also spectacular in comparison with the other control strategies. This is one of the major findings of this study: the combination of latching and declutching control together can give far better results than each one used separately. The price to pay is that the duration of the transient period between the activation of the control (40 s) and the establishment of the steady state (approx. 220 s) is longer with LOD control than with latching control.

It was shown in Babarit & Clément [START_REF] Babarit | Optimal latching control of a wave energy device in regular and irregular waves[END_REF] that the weak modelling of latching control we use here (i.e. a large but not infinite damping force) has more influence on the computed amplitude of the system response, than on the mean absorbed power. Figure 11 shows that the closer we are to absolute latching (the highest value of G 0 ), the larger is the computed power absorbed. However, as the dynamical system becomes stiffer and stiffer, it requires a very short time step in order to keep the numerical time integration scheme stable. Hence, a reasonable value of 80 was assigned to G 0 in the following.

To take into account the delay between the time the controller decides to apply a control and the time the actuators physically apply the control, we introduce the time constants t latching and t PTO on the latching coefficient G and on the PTO damping coefficient b 2 , respectively. Figure 12 shows the motion and the energy production we get under LOD control, using different values for the time constants. Again, better results are obtained as we tend towards absolute discrete control: the shorter the time constant, the better the energy production.

Time-domain simulations were performed for a set of periods of the regular incident waves in the range (5-15 s) in order to establish the power performance curves of the 2 d.f. system under the three discrete control modes: latching; declutching; and LOD (here G 0 was set to 80, B 0 to 10 000, and t latching and t PTO to 0.05). Figure 13 shows the power produced in each mode, measured in the time-domain simulation, when the steady-state regime is established. For the sake of comparison, the theoretical maximum achievable with such an axisymmetric device is also plotted for each wave period, as a reference.

As could be expected, the response without control features two peaks associated with each resonance mode of this bi-oscillator device. One can see that all three controls considered are able to increase the energy absorption for certain wave periods. With the chosen set of coefficients, declutching control seems beneficial mostly around the two natural frequencies of the system, amplifying the peak responses. Latching gives good results in the lowfrequency range, as expected, but also between the peaks. The most interesting results on figure 13 are those associated with LOD control; it gives better energy absorption than the sum of the energy absorbed with latching control and the energy absorbed with declutching control, giving energy absorption at the theoretical maximum over a large range of wave periods. As shown by the detailed example of figure 10, combining the declutching and latching strategies seems to boost the performance of latching, while declutching alone brings no improvement, as for T > 9 s in figure 13. These conclusions can be extended for other values of the system parameters (G 0 , B 0 ), not shown here for brevity.

(g) Latched-operating-declutched control in irregular waves

Finally, a set of time-domain simulations were performed in random wave conditions, in sequences of 600 s, each one defined by its peak period T p and its wave height H s , according to a spectral density following the Pierson-Moskowitz law. The three discrete control strategies were tested for each run. The control sequence was determined by the optimal command methodology exposed above, which means that the control is optimal but not causal, and needs the whole sequence (600 s) of the wave excitation to be computed at each time step.

Figure 14 shows the results of a run with peak period T p = 10 s and significant wave height H s = 2 m. The PTO damping coefficient was set to B 0 = 50 kN m -1 s -1 .

In this example, the absorbed power is multiplied by a factor 5.2, from 15.5 kW without control to 78 kW with LOD control. One can see that, in addition to the increase of energy absorption due to LOD control, there is also a large increase in the response of the system, with an amplification of the maximum amplitude of the motion by a factor 2 for x 1 and by a factor of 10 for x 2 . The amplitude of the inner moving mass reaches almost 20 m, and would indeed be meaningless within the present context of linear theory, the total draught of the buoy considered here being only 10 m. Constraints on the amplitude of the motion should be taken into account in any serious application study.

Figure 15 shows the power absorbed by the system with and without control in random waves. Each dot on this figure corresponds to the mean power absorbed by the system during a 600 s time-domain simulation of the motion. The significant wave height was 2 m in all runs. Time constants for the controller were set equal to 0.05 s, and G 0 was set equal to G 0 = 80.

Results are noisy owing to the fact that, for each sequence of irregular incident waves, the phases of the components are set randomly, and each sample is of finite duration. According to standard statistical theory implicitly assumed here, this scattering could be avoided by repeating each run a large number of times and averaging, or by extending the simulation time to infinity. However, the results given in this figure are globally similar and confirm the behaviour observed in the case of regular waves:

-whatever the peak period of the spectrum, the three discrete control strategies are able to increase the energy production in comparison with the reference case without control; -declutching control works better with large values of the PTO damping coefficient B 0 , whereas latching control works better with small values; and -LOD control is the control strategy that leads to the largest increase in energy absorption, with gains up to 500 per cent. Moreover, LOD control seems to be relatively insensitive to the value of the PTO damping coefficient, even if large values of it seem to marginally improve the amount of absorbed energy.

Conclusion

In this study, three strategies of control, applied at the PTO level to oscillating wave energy devices, were presented. The most famous of them, latching control introduced by Budal & Falnes [START_REF] Budal | Optimum operation of improved wave-power converter[END_REF] in the early 1980s, was shown to be beneficial not only for long wave periods beyond the natural period of the floating body, but also for shorter waves in sub-harmonic modes, and in irregular waves. Declutching control, which was proposed more recently, mainly for hydraulic PTOs, is also efficient, especially with large values of the PTO damping coefficient, which is the opposite trend compared with latching.

The three-state control strategy (LOD), consisting of combining these two individual modes, appeared to give far better results then each one individually. It seems that declutching control enhances the power of latching in this combination, especially for long waves, and its performance is less sensitive to the PTO damping coefficient.

The results have been shown to be always better when the discrete control is absolute, switching instantaneously from one model to the other. However, modelling of the transients by suitable time constants have shown that the performances are only slightly affected, with reasonable values of them.

These conclusions were established not only for a single oscillator system, but also for a bi-oscillator harnessing its relative motion; this was not so evident when we began the study. We have shown, in recent unpublished extensions of this work, that LOD control is also superior when applied to the SEAREV device which is a 7 d.f. wave energy device, with four of them being resonant.

Figure 1 .

 1 Figure 1. The heaving cylinder: a generic 1 d.f. wave energy converter.

. 8 )

 8 (b) Which control strategy optimizes power extraction?

Figure 2 .

 2 Figure 2. The generic 1 d.f. wave energy converter in latching mode.

Figure 3 .

 3 Figure 3. Optimal solutions for latching with alternated maxima in regular waves; harmonic (k = 0)and sub-harmonic (k = 1, 2) solutions (dashed lines, no control; solid lines, latching control).

Figure 4 . 2 )Figure 5 .

 425 Figure 4. Optimal solutions for latching with equal ending ramps in regular waves; harmonic (l = 0)and sub-harmonic (l = 1, 2) solutions (dashed lines, no control; solid lines, latching control).

Figure 6 .

 6 Figure 6. 1 d.f. heaving device in irregular waves. Natural-and latching-controlled heave response versus excitation force (solid lines, latching control; dashed lines, no control).

Figure 7 .

 7 Figure 7. Response of a heaving cylinder in regular waves with (solid lines) and without (dashed line) declutching control.

Figure 8 .

 8 Figure 8. Response of a heaving cylinder in irregular waves with (dashed lines) and without (solid lines) declutching control.

2 Figure 9 .

 29 Figure 9. 2 d.f. heaving wave energy converter: definition sketch.

Figure 10 .

 10 Figure 10. (a-d) Simulations of the response of the 2 d.f. wave energy device in regular waves; under latching control, declutching control or latched-operating-declutched control. Wave period T = 10 s; wave height H = 1 m. (e) evolution of the total extracted energy versus time.

Figure 11 .

 11 Figure 11. (a-d) Simulations of the response of the 2 d.f. wave energy device under latchedoperating-declutched control, with different values of the latching damping G 0 . Incident wave period T = 10 s; wave height H = 1 m.

Figure 13 .

 13 Figure 13. Power curves of the 2 d.f. wave energy device in regular waves under the three discrete control strategies (thick line, without control; right-facing triangles, latching; thin line, theoretical maximum; diamonds, declutching; squares, latched-operating-declutched).

Figure 14 .

 14 Figure 14. (a-d) Response of the 2 d.f. wave energy device with latched-operating-declutched in random waves. Peak period of the wavetrain T p = 10 s; significant wave height H s = 2 m. Power take-off damping coefficient B 0 = 50 kN m -1 s -1 (dashed lines, without control; solid lines, with latched-operating-declutched control).

Figure 15 .

 15 Figure 15. (a-d) Power function of the 2 d.f. wave energy converter versus peak period T p in random waves, under latching, declutching or latched-operating-declutched control with four values of the power take-off damping B 0 . Significant wave height H s = 2 m (solid lines, no control; diamonds, declutching; right-facing triangles, latching; squares, latched-operating-declutched).

Table 1 .

 1 Mechanical parameters of the wave energy converter.

	m 1	705 032	kg
	m 2 k 1 k 2	100 000 789 736 700 000	kg N m -1 N m -1

  Figure 12. 2 d.f. wave energy device under latched-operating-declutched control with varying time constants. Wave period T = 10 s; wave height H = 2 m (squares, t = 0.2 s; triangles, t = 0.1 s; inverted triangles, t = 0.05 s; circles, t = 0.02 s).

		3E + 07
	energy (J)	2E + 07
		1E + 07
		0	200	time (s)	400	600

Nevertheless, one should keep in mind that, owing to the methods applied here, we have demonstrated only the potential of these discrete control strategies, but not the way to implement them in the real world. The method of Hamilton-Pontryagin, as used here, requires knowledge of the future of the excitation, and is therefore not causal. The development of causal control algorithms, necessarily sub-optimal, is still an open challenge in which we are engaged.

The authors wish to express their warm thanks to Prof. F. Farley for his enthusiastic support and invitation, and Prof. J. Ringwood for his helpful comments at the editorial stage.