Ying Siu Liang
email: liangyi@univ-grenoble-alpes.fr

Damien Pellier
email: pellierd@univ-grenoble-alpes.fr

Humbert Fiorino
email: fiorinhu@univ-grenoble-alpes.fr

Sylvie Pesty
email: pestys@univ-grenoble-alpes.fr

iRoPro: An interactive Robot Programming Framework

Keywords: User Robot Programming, Programming by Demonstration, Automated Planning, Human-Robot Interaction

niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

Despite the ongoing advances in Robotics and A.I., it is extremely challenging to pre-program robots for specific end-user applications. Instead of developing robots for domain-specific tasks, a more flexible solution is to have them learn new actions directly from end-users who can customise the robot for their specific application. Programming by Demonstration (PbD) [START_REF] Billard | Robot programming by demonstration[END_REF] is a popular approach for end-users to teach robots actions in an intuitive way by taking demonstrations as input and inferring a policy for the task. However, PbD solutions usually require users to teach robots an action sequence to achieve a certain goal. If the goal changes, the user has to teach the robot a new sequence.

Consider the Tower of Hanoi, a puzzle consisting of three pegs and a number of differently-sized disks, stacked on one peg in descending order, with the largest peg at the bottom. The goal is to move the entire stack from one peg to another, by moving one disk at a time and only to a larger disk or an empty peg. The solution is different depending on the given number of disks. If we want to teach a robot to solve this problem, it would be infeasible to demonstrate the solution each time. A more efficient approach would be to teach the robot the primitive action of moving a disk, associate rules or conditions to this action (e.g., smaller disks can only be placed on top of larger ones), and have the robot generate an optimal solution. Different approaches to generate robot actions have been proposed [START_REF] Calzado | Sami: Interactive, multi-sense robot architecture[END_REF]. The reactive approach is characterised by independent and concurrent basic behaviours that generate the robot global behaviour. The robot sensors are the input of the basic behaviours that, in turn, activate/inhibit the robot effectors. This approach is effective but predicting the robot global behaviour is difficult since no explicit action model is used. The deliberative approach consists of three steps: perception of the world state, plan making and action execution in order to change the world state. Plan making also known as Automated Planning [START_REF] Ghallab | Automated planning: Theory & Practice[END_REF] is based on an explicit action model encoded in a symbolic planning language. This action model is used by a task planner to compute action sequences achieving the expected robot behaviours.

Our research argues for teaching robots primitive actions, instead of entire action sequences, and delegating the logical reasoning process of finding a solution to a task planner. To this end, we present iRoPro, an interactive Robot Programming framework that allows efficient programming of both how an action is performed (low-level action representation) and when it can be applied (high-level), while generalising both aspects to new scenarios. The low-level representation allows the robot to execute the motion trajectory, while the high-level representation allows it to be used with a task planner.

We implement the framework on a Baxter robot and develop an intuitive graphical interface that allows users to teach new actions by demonstration and customise them to be reused for new problems that can be solved with a task planner. The developed end-to-end system involves solutions in perception (e.g., object identification), motion planning (e.g., manipulation, navigation, safety), cognitive robotics (e.g., action learning, task planning) and human-robot interaction (e.g., multi-modal interaction and teacher feedback). Even though task planners are generally used by domain-experts, we show that users with little to no programming experience can easily learn and use their main concepts. We conduct pre-design experiments, where we simulate the framework with the Wizard-of-Oz technique to evaluate the usability of the programming process. We compare these results with post-design experiments and empirically investigate our system's usability with a user study (N=21) where real end-users programmed a Baxter robot directly with our end-to-end system.

In Sec. 2 we give a brief overview of PbD and Automated Planning, the two underlying techniques of the framework. In Sec. [START_REF] Alexandrova | Robot Programming by Demonstration with Interactive Action Visualizations[END_REF] we present iRoPro, the interative Robot Programming framework and its main components. Then in Sec. [START_REF] Argall | A survey of robot learning from demonstration[END_REF] we provide details of the system implemented on a Baxter robot and the user programming process. Sec. 6 presents the experimental evaluation of our approach, where we compare pre-design experiments with post-design experiments. Finally, in Sec. 8, we conclude by discussing limitations and possible extensions to further increase the system's generalisability.

Background

Traditional robot programming processes have task-specific definitions, are generally robot-dependent and require pro-gramming expertise. In the past few decades, different techniques have been developed to facilitate the robot programming process. Biggs et al. [START_REF] Biggs | A Survey of Robot Programming Systems[END_REF] defined two main categories, manual and automatic programming, distinguishing between systems where users can or cannot directly control the robot's executed behaviour.

In manual programming, the user encodes the robot's behaviour via text-based systems using procedural languages such as python, C++, java or graphical systems that use a graph, flow-chart or diagram [24,[START_REF] Fraser | Blockly: A visual programming editor[END_REF][START_REF] Majed | Learn to program with Scratch: a visual introduction to programming with games[END_REF]. For automatic programming techniques, robots generate their behaviour from data provided as input to the system. We differentiate between Deep Learning (DL) [START_REF] Schmidhuber | Deep learning in neural networks: An overview[END_REF], where the robot is provided a large amount of labelled or unlabelled data, Reinforcement Learning [START_REF] Kaelbling | Reinforcement learning: A survey[END_REF][START_REF] Gosavi | Reinforcement learning: A tutorial survey and recent advances[END_REF], where the robot gathers the data by exploring the environment, and Programming by Demonstration (PbD) [START_REF] Billard | Robot programming by demonstration[END_REF][START_REF] Argall | A survey of robot learning from demonstration[END_REF], where the robot learns from example demonstrations provided by a human teacher. While DL approaches allow the robot to learn skills autonomously, they often require programming and domain experts to prepare the system input (e.g., label or preprocess data, define policy and reward functions). Similarly, reinforcement learning generally requires long training times to gather enough data to learn a skill.

In contrast, PbD provides a more intuitive low-effort solution, where the teacher's main task involves providing demonstrations to the robot. Since PbD solutions allow the robot to learn from a sparse set of examples, the data and time required to learn a skill is moderately low. Thus, our work uses PbD to allow end-users to program robots. In the following sections we will give a brief overview of PbD and Automated Planning, the two main concepts used in our proposed framework.

Programming by Demonstration

PbD, also referred to as Learning from Demonstration, is an end-user programming technique for teaching a robot new skills by demonstrating them, without writing code [START_REF] Billard | Robot programming by demonstration[END_REF]. It has become a central topic in research areas, with the aim to move from purely pre-programmed robots to flexible userbased interfaces for training robots. PbD is traditionally used to learn low-level actions from trajectory demonstrations using Gaussian Mixture Models [START_REF] Billard | Robot programming by demonstration[END_REF][START_REF] Calinon | Incremental learning of gestures by imitation in a humanoid robot[END_REF] or Dynamic Movement Primitives [START_REF] Pastor | Learning and generalization of motor skills by learning from demonstration[END_REF]. They can also be learned from keyframebased demonstrations (kfPbD), where the user kinesthetically manipulates the robot's arm to record a series of endeffector poses, referred to as keyframes [START_REF] Akgun | Keyframebased learning from demonstration[END_REF]. While demonstrations can be provided in different ways (e.g., by observating a human teacher), users prefer to control the robot directly [START_REF] Suay | A practical comparison of three robot learning from demonstration algorithm[END_REF]. In kfPbD, actions are represented as a sparse sequence of gripper states (open/close) and end-effector poses relative to perceived objects or to the robot's coordinate Effects:

(at ?obj ?posB) (empty ?posA)

Fig. 2 Action model representation to move a cube from position A to B in terms of preconditions and effects [START_REF] Liang | Evaluation of a robot programming framework for non-experts using symbolic planning representations[END_REF].

frame. Alexandrova et al. [START_REF] Alexandrova | Robot Programming by Demonstration with Interactive Action Visualizations[END_REF] implemented an end-user robot programming system to teach generalisable actions from a single demonstration where keyframes are automatically inferred and actions can be modified retrospectively via a graphical interface.

Automated Planning

Automated Planning, also known as AI Planning, is a research field that focuses on the development of task planners consisting of efficient search algorithms to generate solutions to problems [START_REF] Ghallab | Automated planning: Theory & Practice[END_REF]. Given a planning domain, i.e., a description of the state of the world and a set of actions, and a planning problem, i.e., an initial state (Fig. 1 (a)) and a goal (Fig. 1 (d)), the task planner generates a sequence of actions (Fig. 1 (c)), which guarantees the transition from initial states to goal. To allow a correct transition between different world states, high-level actions are defined in terms of preconditions and effects, which represent states before and after the action execution respectively (Fig. 2). An action is represented as a tuple a = (param(a), pre(a), eff(a)), whose elements are:

param(a): set of parameters that a applies to pre(a): set of predicates that must be true to apply a eff(a) -: set of predicates that are false after applying a eff(a) + : set of predicates that are true after applying a where eff(a) = eff(a) -∪ eff(a) + . Action parameters are associated with a type (Fig. Fig. 3 Experimental setup of the user study. Users programmed the Baxter robot via a graphical interface in order to manipulate objects (shown with their type hierarchies) in the task domain [START_REF] Liang | End-User Programming of Low-and High-Level Actions for Robotic Task Planning[END_REF].

(Fig. 3). Predicates are defined in first-order logic and used to describe world states and relations between their elements (e.g., (at obj1 A)). Planning algorithms use a symbolic planning language as their standard encoding language, such as STRIPS [START_REF] Fikes | STRIPS: A new approach to the application of theorem proving to problem solving[END_REF] or PDDL [START_REF] Ghallab | Automated planning: Theory & Practice[END_REF]. An example of a planning domain in PDDL can be seen in Fig. 4.

The Tower of Hanoi problem could be defined in terms of a planning problem, where the domain consists of 3 pegs and a number of disks, and the action is defined as moving a disk from one peg to another, with associated rules as preconditions and effects. A planner can then be used to generate a solution to the problem for any number of disks.

From now on, the word "action" will refer to an action as defined in automated planning. It is worth noting that the predicates used in the planning domains are freely defined by the persons encoding those domains. However, creating a standard for knowledge representation and reasoning in autonomous robotics is a basic requirement to allow the interoperability of the robotic systems, the communication between robots and humans etc. based on a common vocabulary along with clear and concise definitions. The IEEE Standard Association's Robotics and Automation Society recognized this need, and a set of ontologies have been developped [START_REF] Olszewska | Ontology for autonomous robotics[END_REF].

Related Work

End-user robot programming has been addressed previously for industrial robots to be programmed by non-robotics domain experts, where users specify and modify existing plans for robots to adapt to new scenarios [START_REF] Paxton | CoSTAR: Instructing collaborative robots with behavior trees and vision[END_REF][START_REF] Perzylo | Intuitive instruction of industrial robots: Semantic process descriptions for small lot production[END_REF][START_REF] Stenmark | Simplified programming of re-usable skills on a safe industrial robot: Prototype and evaluation[END_REF]. For example, Paxton et al. [START_REF] Paxton | CoSTAR: Instructing collaborative robots with behavior trees and vision[END_REF] use Behaviour Trees to represent task plans that are explicitly defined by the user and can be modified to adapt to new tasks. In our work we argue for the use of task planners to automatically generate plans for new scenarios, rather than have the user manually modify them. Previous work has addressed knowledge engineering tools for constructing planning domains but usually require PDDL experts (PDDL Studio [START_REF] Plch | Inspect, edit and debug PDDL documents: Simply and efficiently with PDDL studio[END_REF]), or common knowledge in software engineering (GIPO [START_REF] Simpson | Planning domain definition using GIPO[END_REF], itSIMPLE [START_REF] Vaquero | itSIMPLE: towards an integrated design system for real planning applications[END_REF]). There has been previous work on integrating task planning with robotic systems [START_REF] Cashmore | ROSPlan: Planning in the Robot Operating System[END_REF][START_REF] Kuhner | Closed-loop robot task planning based on referring expressions[END_REF], learning high-level actions through natural language instructions [START_REF] She | Teaching robots new actions through natural language instructions[END_REF] or learning preconditions and effects of actions to be used in planning [START_REF] Jetchev | Learning grounded relational symbols from continuous data for abstract reasoning[END_REF][START_REF] Konidaris | From skills to symbols: Learning symbolic representations for abstract highlevel planning[END_REF][START_REF] Ugur | Bottom-up learning of object categories, action effects and logical rules: From continuous manipulative exploration to symbolic planning[END_REF]. However, in all of these cases, the robot is provided with a fixed set of low-level motor skills. In our approach, we do not provide the robot with any predefined actions but enable users to teach both low-and high-level actions from scratch.

PbD has been commonly applied to allow end-users to teach robots new actions by demonstration. Alexandrova et al. [START_REF] Alexandrova | Robot Programming by Demonstration with Interactive Action Visualizations[END_REF] created an end-user programming framework with an interactive action visualisation allowing the user to teach new actions from single demonstrations but do not reuse them with a task planner. Most closely related to our approach is the work by Abdo et al. [START_REF] Abdo | Learning manipulation actions from a few demonstrations[END_REF] where manipulation actions are learned from kinesthetic demonstrations and reused with task planners. However, the approach requires 5-10 demonstrations to learn action conditions which becomes tedious and impractical if several actions need to be taught. In this work we argue for having the user act as the expert by letting them correct inferred action conditions, thus allowing a new action to be learned from a single demonstration. We further provide a graphical interface that allows users to create new actions and address previously unseen problems that can be solved with task planners.

iRoPro -interactive Robot Programming

In our previous work [START_REF] Liang | A framework for robot programming in cobotic environments: First user experiments[END_REF] we proposed iRoPro, an interactive Robot Programming framework that allows end-users to teach robots new actions that can be reused with task planners. The framework consists of the following three aspects (Fig. The user is provided with a GUI that abstracts from the underlying modeling language used for Automated Planning. For each step, the user interacts with the GUI to navigate between the components to teach new actions by demonstration, modify inferred action conditions, define new planning problems for the robot to solve and execute generated plans.

In the following sections, we give a brief description of each component. We refer the reader to our previous work [START_REF] Liang | A framework for robot programming in cobotic environments: First user experiments[END_REF][START_REF] Liang | Evaluation of a robot programming framework for non-experts using symbolic planning representations[END_REF] for more details.

Programming by Demonstration: teach actions

Teaching primitive actions consists of learning both how and when an action should be applied, i.e., learning the low-level action trajectory as in PbD and the high-level representation with preconditions and effects as used in Automated Planning. We consider an action that consists of both lowand high-level representations an action model. The highlevel representation can be entered directly by the user or inferred from observing the world state before and after the action demonstration. In our work, we first infer the preconditions and effects which the user can subsequently modify on the GUI. We rely on the user's logical reasoning and understanding of what they want to teach the robot and allow them to directly program and correct inferred action models. Thus, the robot can learn a new action from a single demonstration with the user acting as the expert to correct inferred conditions. The user validates the learned action model or provides additional demonstrations to refine the low-or high-level representations. The user repeats this process and creates an action model for each primitive action (see Fig. 5A).

Automated Planning: reuse actions

In the PbD step, the robot learned action models that include the high-level representation used in Automated Planning. The Automated Planning step consists of creating a planning problem and defining a goal for which the integrated task planner can generate a solution (Fig. 5B). Given a description of a planning domain, we can define a planning problem with an initial state and a desired goal state to achieve. Depending on the robot architecture and perception system, a partial PDDL domain can be predefined in the system that includes a set of object types and predicates that the robot can recognise. The action models created in the PbD step complete the partial PDDL domain. Similar as before, the robot can infer initial world states for the planning problem and the user can modify and correct them via the GUI. Given the user-defined goal, the task planner generates a plan consisting of an ordered action sequence for the robot to execute. The user can verify the generated plan and have the robot execute it in real life. If no plan is generated or if the plan seems incorrect, the user can modify the taught action models, as well as the initial and goal states and relaunch the planner.

Retro-active Loop: refine actions

The retro-active loop allows the user to revisit and correct created action models (see Fig. 5C). It is likely that the initially generated plan would not achieve the specified goal, especially if the context of the planning problem is different to that of the initial demonstration (e.g., different object types or positions). Programmed action models can be generalised and reused, especially if the low-level action remains the same. Instead of creating new action models for each new problem, the user can simply modify the action parameters, preconditions or effects. This minimises the user's programming effort and the number of demonstrations required. Thus, the application to a new context is an important step to test the generalisability of action models.

There are several possible causes why the planner might generate incorrect or non-existent solutions:

-Action parameters: this restricts or generalises the application of the action as they dictate what types an action can be applied to. An action is not considered by the planner, if the types do not match those in the initial state of the planning problem (e.g., pick-and-place was only defined for cube objects but not for other types). -Preconditions: similar to action parameters, they define when an action can be applied, but in terms of predicates describing the initial world state. All stated preconditions must hold in a world state in order to apply the action (e.g., pick-and-place of an object only if it is clear). -Effects: they define how the world state is updated after the action execution and help the robot to keep track of changes. If they are not defined correctly, there can be a mismatch of the robot's assumed world state and the actual world state (e.g., a position is still considered free when it is occupied). -Initial states: they describe the existing world state to the robot. If the initial states are incorrect or missing the planner may consider certain actions as invalid or not find a plan to the goal (e.g., an object is not mentioned in the initial states at all). -Goal: the user defines the set of predicates for the robot to achieve. This should not include intermediate steps to achieve the goal nor contradicting states (e.g., 'object is on A' and 'A is clear' are both stated as goal states).

Knowledge engineering tools can facilitate this process of modifying action models. They often provide useful functionalities for dynamic testing, model checking and visualisation [START_REF] Simpson | Planning domain definition using GIPO[END_REF], but most tools require expertise in Automated Planning or Software Engineering. In our work we argue that the proposed robot programming process does not require this expertise and can be learned easily by users with different educational backgrounds. Fig. 6 Overview of iRoPro that allows users to teach low-and high-level actions by demonstration. The user interacts with the GUI to run the demonstration, modify inferred action conditions, create new planning problems for the robot to solve and execute.

System Implementation

We implemented iRoPro on a Baxter robot with two arms (one claw and one suction gripper), both with 7-DoF and a load capacity of 2.2kg. For the object perception we mounted a Kinect Xbox 360 depth camera on the robot. We developed a user interface as a web application that can be accessed via a browser on a PC, tablet or smartphone. The source code for iRoPro is developed in ROS [START_REF] Quigley | ROS: an open-source Robot Operating System[END_REF] and available online 1 . The action is learned by demonstration using the open-source system Rapid PbD2 . The integration of the task planner is implemented using the ROS package PDDL planner 3 . In our implementation, we define landmarks as either predefined table positions or objects that are detected from Kinect point cloud clusters using an open-source tabletop segmentation library 4 . An object o = (x, y, z, width, length, height) is represented by its detected location and bounding box, which are used to infer its type and related predicates. The user completes the partial PDDL domain via the GUI by creating new action models by demonstration. Then they create planning problems that can be solved with the integrated task planner. Figure 6 shows an overview of the programming process. 5 In the following sections we will give a brief overview of the system implementation. Further details can be found in our previous work [START_REF] Liang | End-User Programming of Low-and High-Level Actions for Robotic Task Planning[END_REF].

Low-level Action Representation

We represent low-level actions as proposed in previous work using keyframe-based PbD [START_REF] Alexandrova | Robot Programming by Demonstration with Interactive Action Visualizations[END_REF], where the action is represented as a sparse sequence of gripper states (open/close) and end-effector poses relative to perceived objects or to the robot's coordinate frame. During the action demonstration, the user guides the robot arm using kinesthetic manipulation and saves poses that they find relevant for the action. For example, the pick-and-place action of an object to a marked position could be represented as poses relative to the object (for the pick action), poses relative to the target position (for the place action), and corresponding open/close gripper states. Action executions are performed by first detecting the landmarks in the environment, calculating the end-effector poses relative to the observed landmarks, and interpolating between the poses. While these actions can be learned from multiple demonstrations [START_REF] Niekum | Learning and generalization of complex tasks from unstructured demonstrations[END_REF], we take the approach that only requires a single demonstration by heuristically assigning poses and letting the user correct them if needed [START_REF] Alexandrova | Robot Programming by Demonstration with Interactive Action Visualizations[END_REF]. Thus, the first demonstrated action is already an executable action.

The user can teach multiple manipulation actions and discriminate between them by associating different conditions that specify when the robot should use them (e.g., different conditions for actions using claw or suction grippers).

High-level Action Representation

We implemented a partial PDDL domain with predefined types and predicates that the robot can automatically detect using its sensors. We defined five predicates commonly used for object manipulation tasks and included two (flat and thin) to describe further object properties:

-ELEMENT is clear: an element has nothing on top of it -OBJECT is on ELEMENT: an object is on an element -OBJECT is stackable on ELEMENT: an object can be placed on an element -OBJECT is flat: an object has a flat top -OBJECT is thin: an object can be grasped with the claw gripper

In our definition, CUBE and BASE objects are flat, while CUBE and ROOF objects are thin enough for the robot to grasp. The set of inferred types and predicates could be extended for more complex tasks (e.g., object colour or orientation [START_REF] Li | Learning object orientation constraints and guiding constraints for narrow passages from one demonstration[END_REF]) but was beyond the scope of this work.

Action Inference from Demonstration

Instead of manually defining action parameters, preconditions and effects, we accelerate the programming process

O 1 ∩ O 2 ) = {φ i |φ i ∈ O 1 ∧ φ i / ∈ O 2 }, eff(a) = (O 2 -O 1 ∩ O 2 ) = {ψ i |ψ i / ∈ O 1 ∧ ψ i ∈ O 2 },
where eff(a) includes positive and negative effects (Fig. 7).

A predicate φ has variables var(φ ) = {v 1 , v 2 , . . . }, where each v i has a type. Therefore, action parameters are the set of variables that appear in either preconditions or effects, i.e.,

param(a) = {v i | ∃φ ∈ pre(a) s.t. v i ∈ var(φ ) ∨ ∃ψ ∈ eff(a) s.t. v i ∈ var(ψ)}.
Note that conditions could be learned from multiple demonstrations [START_REF] Abdo | Learning manipulation actions from a few demonstrations[END_REF][START_REF] Konidaris | From skills to symbols: Learning symbolic representations for abstract highlevel planning[END_REF]. Our work argues for accelerating the teaching phase by learning from a single demonstration and letting the user act as the expert to correct wrongly inferred conditions.

Action Generalisation

The low-level action representation (Sec. 5.1) generalises motion trajectories by re-calculating poses based on detected landmarks from the demonstrated to the new environment. The high-level representation (Sec. 5.2) specifies when an action can be applied, therefore allows taught low-level motion trajectories to be reused for other objects (e.g., use suction gripper for all objects, regardless of their type) or to be restricted for certain types (e.g., only BASE objects). By combining these two representation levels, taught actions can be generalised for new environments that are different to the demonstrated one, allowing the user to customise them for their specific use case.

Interactive Robot Programming

The interacts with the GUI that visualises the robot and detected objects (Fig. 8). During the programming process, the GUI allows them to create new actions, run the kinesthetic move(OBJECT, POSITION, POSITION) Before (O1) teaching by demonstration, modify inferred types and predicates and to create and solve new problems with the task planner. The interactive robot programming cycle consists of creating and modifying actions and problems:

obj is on A A is not clear obj is not on B B is clear C is clear obj is clear obj is flat ____ ____ ____ A B C After (O2) obj is on B B is not clear obj is not on A A is clear C is clear obj is clear obj is flat ____ ____ ____ A B C _______________ A B Goal Preconditions obj is on A A is not clear obj is not on B B is clear
Actions. New actions are taught by kinesthetically moving the robot's arms using kfPbD [START_REF] Akgun | Keyframebased learning from demonstration[END_REF], where both low-level and high-level actions are learned and generalised (Sec. 5.4).

The user can have the robot re-execute the taught action immediately in order to validate it. The user can modify the action properties if the inference was not correct. To teach more actions, the user can either create a new one or copy a previously taught action and modify it.

Problems. New planning problems can be generated if at least one action exists. To create a problem, the robot first detects the existing landmarks and infers their types and initial states. The user can modify them if the inference was not correct. Then, the user enters predicates that describe the goal states to achieve. The complete planning domain and problem are translated into PDDL and sent to the Fast-Forward planner [START_REF] Hoffmann | The FF planning system: Fast plan generation through heuristic search[END_REF]. If a solution is found that reaches the goal, it is displayed on the GUI for the user to verify and execute on the robot (Fig. 9). If no solution is found or if the generated plan is wrong, the user can open a debug menu to review actions. It provides a summary of the entire planning domain with hints described in natural language to troubleshoot (e.g., 'make sure the action effects can achieve the goal states'). In our post-design study (Sec. 6.4) we found that this helped users understand how the system worked and why the generated plan was wrong. Once the user modified actions, initial or goal states, they can relaunch the planner to see if a correct plan is generated. For any subsequent tasks, the user can create a new problem or modify existing ones by re-detecting the objects.

Plan Execution

The generated plan is a sequence of actions with parameters that correspond to detected objects. For each action, the sequence of end-effector poses are calculated relative to the landmarks that the action is being applied to (Sec. 5.1). To accelerate the execution, we only detect landmarks once at the start, then save their new positions in memory for quick reference, which we refer to as a mental model. After each action execution, the mental model is updated with the latest positions of the landmarks according to the action's effects (e.g., obj moved from position A to B). As this assumes that actions are always executed successfully, successful executions need to be checked separately but were beyond the scope of this The mental model is also used as a workaround for our limited perception system as it does not detect stacked objects in their initial states (as discussed in Sec. 7).

Experimental Evaluation

We first conducted two pre-design experiments to evaluate the usability of the proposed framework. They consisted of qualitative user studies to respond to the following questions:

Q1 How do non-expert users adopt the Automated Planning language with its action model representation? (Sec. 6.1) Q2 Can users teach a robot action model for Automated Planning using the proposed framework? (Sec. 6.2)

In both experiments we particularly focused on elements to assess the user's understanding of action models such as defining their preconditions and effects. Understanding this symbolic representation is a key requirement to use iRoPro.

Based on these results, we implemented iRoPro on the Baxter robot (Sec. 5) and subsequently conducted postdesign experiments to evaluate the working system with real end-users (Sec. 6.4). Furthermore, we compared results obtained from pre-design with post-design experiments and validate the usability of our proposed framework (Sec. 6.5). In the following sections we give a brief overview of the experimental setup, design, measurements and results. Further details on all experiments can be found in previous work [START_REF] Liang | Evaluation of a robot programming framework for non-experts using symbolic planning representations[END_REF][START_REF] Liang | End-User Programming of Low-and High-Level Actions for Robotic Task Planning[END_REF].

Acceptance of Automated Planning concepts

In this experiment, we addressed the following question: Q1 How do non-expert users adopt the Automated Planning language with its action model representation?

Users (N=10) with little to no programming experience were introduced to the symbolic language and syntax with type structures used in Automated Planning. Users were instructed to describe world state configurations to the robot. The goal was to assess the user's adoption of the planning concepts (e.g., object types, action models) and to verify that the symbolic planning language was appropriate for nonexpert users.

Experimental Design & Measurements

The experimental setup consisted of a 2x2 board (with positions A1, A2, B1, B2), 2 cubes, 1 ball, and 1 ball recipient in the form of a bowl. The participants were given sheets with empty tables to complete for each task. Each participant was allocated 1 hour, but the average duration of the experiment was 49 minutes. Users were told that they needed

(c) (stackable ball cube) (b) (move ball A1 B2)

PreC: (at ball A1) Eff: (at ball B2)

(d) (empty B2) A1 1b B2 A2 A1 1b B2 A2
(a) Initial state:

(at ball A1) (at cube B2) (empty A2) (empty B1) Fig. 10 Users were instructed to provide a description of (a) the initial state of the world and (b) an initial move action model. Then they derived additional preconditions for moving the ball from position A1 to B2: (c) (stackable ball cube): the ball can be stacked onto cube, and (d) (empty B2): if the ball cannot be stacked, the target position should be empty [START_REF] Liang | Evaluation of a robot programming framework for non-experts using symbolic planning representations[END_REF].

to use a symbolic planning language to describe the state of the world and the semantic meaning of actions to the robot (Fig. 10). At the course of the experiment, users were faced with three different scenarios of increasing complexity. The participants' behaviour was observed by the experimenter and the experiment was recorded on camera. We evaluated their capability to learn the presented planning language and apply it to different problem statements. At the end, participants were given a questionnaire related to their experience and their understanding of the learned planning language and concepts.

Acceptance of the Robot Programming Process

In this experiment, we addressed the following question:

Q2 Can users teach a robot action model for Automated Planning using the proposed framework?

Users (N=11) 6 with different programming experience were presented a simulated implementation of iRoPro and had to teach action models by kinesthetically manipulating a Baxter robot. Users were instructed to teach a primitive action by demonstration and assign preconditions and effects. The goal was to assess the framework's usability and the user's difficulties encountered during the programming process. At the end, participants were given a questionnaire related to their experience, their perceived understanding of the presented concepts and the usability of the framework. In the following sections we briefly outline the experimental setup, measurements and results of the experiment. The experiments were conducted using a Baxter robot, mounted with a partial implementation of the framework. The implemented functionalities included:

-'learn new action': record the kinesthetic action demonstration, -'find a coloured object': apply the recorded action to an object of the specified colour, 6 All participants were different from the first experiment -'execute an action sequence': execute a sequence of previously taught actions.

We used the Wizard-of-Oz technique to simulate the remaining functionalities (e.g., 'infer action preconditions and effects', 'generate solution using a planner'). Each participant was allocated 1 hour, but the average duration was 29.5 minutes. The participants' behaviour was observed by the experimenter and the experiment was recorded on camera.

Experimental Design & Measurements

The experiment scenario was set in a simulated assembly line, where objects of the same shape, but different colour arrived consecutively at a departure position. Users had to teach the Baxter robot the action for moving an object from the departure position to an arrival position, where another maintenance task would be performed later. Users first demonstrated the action on the robot by guiding its arm through the desired trajectory, then they were presented with an action model that the robot had 'learned'. At the course of the experiment, users were faced with two different scenarios, where they had to suggest logical conditions for the action model in order for the robot to apply them successfully (Fig. 11). We evaluated the user's capability to improve action models and associate conditions when faced with different situations, and assessed the framework's overall usability.

Pre-Design Experiments Findings

In both experiments, we did not observe a significant difference in the performance between users with different programming experience. In the first experiment, the majority (9 or 90%) of the participants managed to describe the complete world state using the correct syntax. All participants gave correct explanations for preconditions and effects of action models, and provided correct examples. Figure 13a) shows the user responses to the questionnaire in the second experiment. All 11 users were satisfied with the programming process and Baxter's ability to learn and reproduce the demonstrated move action.

The majority of the users had issues formulating the logical properties used for preconditions and effects. In the first experiment (Sec. 6.1), users had difficulties formulating certain conditions in the planning language (e.g., (stackable ball cube)), but stated equivalent ones (e.g., 'only place the ball, if it is stackable on the cube'). Similarly, in the second experiment (Sec. 6.2), users formulated missing preconditions (e.g., 'position B is empty') with other equivalent conditions (e.g., 'do not place the object on position B, if it is occupied').

Some of the users made wide assumptions about the robot's capabilities. In the second experiment, when both arrival and departure positions were occupied, 5 (or 50%) of the users expected Baxter to consider the occupied position, even though the condition was not mentioned in its action model. This is a common problem in PbD solutions as there is a difference between the robot's intelligence and the one perceived by its teacher [START_REF] Suay | A practical comparison of three robot learning from demonstration algorithm[END_REF]. This can be addressed by reproducing the learned action in a new context and verifying the robot's knowledge base, as we did throughout the experiment.

With these two qualitative experiments, we showed that the Automated Planning language and its main concepts can easily be learned by users without any programming background. The action model representation, in terms of preconditions and effects, seems to be intuitive for non-expert users. These initial experiments provided us with an initial idea of how the users might perceive the proposed robot programming framework. We intentionally limited the set of Automated Planning concepts (i.e., object types, predicates, and actions with preconditions and effects) that are necessary to use the framework to the bare minimum to assess the potential usability of such a framework. Further experiments should test scalability, address more Automated Planning concepts (e.g., object-type hierarchy, more predicates, planning problem definition and resolution) and potentially compare separate control groups (e.g., experts vs non-experts) in less structured environments.

Post-Design Experiment

The second part of the evaluation was conducted using THE-DRE [START_REF] Mandran | THEDRE: a Traceable Process for High Quality in Human Centred Computer Science Research[END_REF], a human experiment design method that combines qualitative and quantitative approaches to continuously improve and evaluate the developed system from the experimental ground. The aim was to evaluate our approach with real end-users and we were also interested in the user's programming strategy for using the system. We split participants into two control groups, with and without condition inference (Sec. 5.3) and evaluated user performance in terms of programming times for completing a set of benchmark tasks. We set the following hypotheses for our experiments: H1 Action creation: users can teach new low-and high-level actions by demonstration H2 Problem solving: users can solve new problems by defining the goal states and executing the plan on Baxter H3 Autonomous system navigation: users understand the system and can navigate and troubleshoot on their own H4 Condition inference (CI) -Group 1 vs 2: users without CI will understand the system better H5 Pre-study test (PT): users that score higher in the PT have shorter programming times

In the following we will give a brief overview of the experiment setup, design, measurements and results. Further details can be found in our previous work [START_REF] Liang | End-User Programming of Low-and High-Level Actions for Robotic Task Planning[END_REF].

Participants

The study was conducted with 21 participants (10M, 11F) in the range of 18-39 years (M=24.67, SD=6.1). We recruited participants with different educational background and programming levels: 6 'CS' (either completed a degree in computer science or were currently pursuing one), 7 'non-CS' (have previously taken a programming course before), and 8 'no experience' (only had experience with office productivity software). Furthermore, 3 participants (in 'CS') have programmed a robot before, out of which 1 had intermediate experience with symbolic planning languages while the remaining participants had no experience in either. One participant in the category 'non-CS' failed to complete the majority of tasks and was excluded from the result evaluation. The two control groups included equal number of participants in all three categories.

Protocol

Users were first given a brief introduction to task planning concepts, the Baxter robot and the experimental set up (Fig. 3). They were then asked to complete a pre-study test to capture the participant's understanding of the presented concepts. Users were given 8 tasks to complete (Table 1), where the first two were practice tasks to introduce them to the system. The tasks were designed to address different aspects to familiarise them with the system: create new actions (Task 6), modify parameter types (Tasks 4&7), modify action conditions (Tasks 3,5,8). For each task they needed to create a new problem, define the goal states, and launch the planner to generate an action sequence. When the generated plan was correct, they were executed on the robot (Fig. 12). Otherwise, the user had to modify the existing input until the plan was correctly generated. Tasks 6-8 were similar to the previous tasks (1-5) but use both robot grippers.

Metrics

We captured the following data during the experiments: Table 1 Benchmark tasks for the user study where the first two tasks were used to introduce participants to the system [START_REF] Liang | End-User Programming of Low-and High-Level Actions for Robotic Task Planning[END_REF]. The pre-study test included seven questions related to their understanding of the concepts presented at the start of the experiment, e.g., syntax ('If move(CUBE) describes a move action, tick all statements that are true.'), logical reasoning ('Which two conditions can never be true at the same time?'), and other concepts ('Tick all predicates that are required as preconditions for the given action'). The questions were multiple choice and the highest achievable score was 7.

#
In the post-study survey we used the System Usability Scale (SUS) [START_REF] Brooke | SUS: A retrospective[END_REF] where participants had to give a rating on a 5-Point Likert scale ranging from 'Strongly agree' to 'Strongly disagree'. It enabled us to measure the perceived usability of the system with a small sample of users. As a benchmark, we compared overall responses to the second user study, where users were simulated a similar robot programming experience using the Wizard-of-Oz technique but had no direct interaction with a working system (Sec. 6.2). Finally, participants were asked which aspects they found most useful, most difficult, and which they liked the best and the least.

Results

20 participants completed all tasks, while one 'non-CS' user failed to complete the majority of tasks and did not seem to understand the presented concepts. This participant was excluded in the results presented below (Table 2):

H1)-H3) User performance

Users took between 22-60 minutes to complete the main tasks [START_REF] Akgun | Keyframebased learning from demonstration[END_REF][START_REF] Alexandrova | Robot Programming by Demonstration with Interactive Action Visualizations[END_REF][START_REF] Argall | A survey of robot learning from demonstration[END_REF][START_REF] Bangor | An empirical evaluation of the system usability scale[END_REF][START_REF] Biggs | A Survey of Robot Programming Systems[END_REF][START_REF] Billard | Robot programming by demonstration[END_REF], with an average of 41.2 minutes. 'non-CS' users completed the tasks the fastest, followed by users with no programming experience. 'CS' users took on average longer as they were often interested in testing the system's functionalities that were beyond the given tasks.

Users initially had problems with different concepts that were presented at the start of the study, in particular they confused action parameters, preconditions and goal states. For example, in Task 3, 6 (or 30%) users tried to add intermediate action steps to achieve the goal state, instead of simply letting the planner generate the solution. In Task 4, 14 (or 70%) wanted to create a new action, even though they could reuse the existing action by modifying the parameter types. However, by Task 6, all users were able to use the system autonomously to create new actions and problems and navigated the system with little to no guidance. By the end of the experiment, users programmed two manipulation actions (one for each gripper) that were reused to complete all 8 benchmark tasks.

H4) Condition inference (CI)

We noticed a discrepancy in the programming strategies between the two control groups (Group 1 with CI vs. Group 2 without CI). Participants in Group 1 had the tendency to leave the inferred conditions unmodified without adding conditions that would improve the action's generalisability to different use cases. As participants in Group 2 had to add action conditions manually, they considered all predicates they deemed necessary for the action and added additional ones that were required for later tasks. Thus, Group 2 took on average longer to complete tasks where a new action had to be created (Tasks 1&6), but was faster than Group 1 for subsequent tasks, where conditions had to be modified (Tasks 3,5,7). Overall both groups had similar completion times for all tasks.

System usability and learnability

There are several ways to interpret the System Usability Scale (SUS) scores [START_REF] Brooke | SUS: A retrospective[END_REF] obtained from the post-study survey. Using Bangor et al.'s categories [START_REF] Bangor | An empirical evaluation of the system usability scale[END_REF], 14 (70%) users ranked iRoPro as 'acceptable', 6 (30%) rated it 'marginally acceptable', and no one ranked it 'not acceptable'. Correlating this with the Net Promoter Score [START_REF] Sauro | Predicting net promoter scores from system usability scale scores[END_REF], this corresponds to 10 (50%) participants being 'promoters' (most likely to recommend the system), 5 (25%) 'passive', and 5 (25%) 'detractors' (likely to discourage). Overall, iRoPro was rated with a good system usability and learnability. 9 (45%) users stated 'generate solutions to defined goals automatically' as the most useful feature, followed by 'robot learns action from my demonstration' (4 or 20%) -two main aspects of our approach. 4 (20%) stated the most difficult part as 'finding out why Baxter didn't solve a problem correctly', similarly 8 (40%) stated difficulties related to 'understanding predicates and defining conditions'. 11 (55%) disliked 'assigning action conditions' the most, while the rest stated different aspects. A common feedback was 'it takes time to understand how the system works at the start'. The most liked parts were 'executing the generated plan' (8 or 40%) and 'demonstrating an action on Baxter' (7 or 35%).

Pre-vs Post-Design Experiments

We compare post-study questionnaire responses between pre-and post-design experiments (Fig. 13). In the first experiment (Sec. 6.2), users had no direct interaction with the robot programming system as it was simulated using the Wizard-of-Oz technique, while in the last experiment (Sec. 6.4), users programmed the robot using the end-to-end system implementation. The main differences were noted regarding difficulties encountered during the experiment: In the first study 11 (or 100%) users agreed that they encountered no difficulties, while in the last study only 7 (or 35%) users stated the same. However, all users in the last study claimed to have a good understanding of the action representation and how the robot learned new actions from their demonstrations, while an average of 2 (18%) disagreed in the first study. Both differences can be explained by the fact that in the last study, users had to use an end-to-end system to program the robot, while in the first study, users had no direct interaction with a working system. Even though users encountered more difficulties in the last study, they got a better understanding of the functionalities due to getting handson experience. This also correlates with negative responses in our survey to the question if 'no programming experience was required' where 13/20 (65%) agreed and 4 disagreed. Overall, both user studies received positive responses.

Continuous Improvement of the System

The system underwent four phases of improvement, allowing us to refine the system functionalities, GUI and user instruction methods:

1. Initial prototype: Based on the feedback received on our initial prototype by domain experts, we changed the flow for introducing the system to novice users as it included a lot of new information (e.g., PbD and Automated planning concepts) that they were not familiar with. 2. Pre-tests with 3 users: We ran pre-tests with 3 users who have never seen the system before and further improved the experiment flow (e.g., create an action for one object at a time). We also made the user interface more friendly to include Baxter icons (Fig. 14) and eyes that followed the robot's moving joint [START_REF] Gaisne | Baxter eyes[END_REF]) on the robot's screen so that it seemed more human-like. We noticed that troubleshooting incorrect actions seemed difficult for all users, so we included an option to review actions and goals which provides a summary of the created input and hints to guide the debugging steps. 3. First experimental tests with 5 users with condition inference: After running the experiments with 5 users, we noticed that the majority did not modify the inferred action conditions. This raised an interesting HRI question of whether users who were given automatically generated conditions would 'blindly trust' them. Hence, we decided to create two experiment groups: with and without condition inference, where the first 5 participants belonged to the former. Participants in the latter group would need to manually enter all conditions via the interface. 4. Final experimental tests with 16 users: The remaining users were divided into the two control groups so that we had an equal number of participants in both control groups, while also maintaining an even distribution of programming levels. The results of the 21 participants in the experimental tests were used as the final evaluation of the system as discussed in the previous sections.

Discussion

We demonstrated that iRoPro can be used to generalise primitive actions to a range of complex manipulation tasks and that it is easy to learn for users with or without programming experience. In our system evaluation we could have programmed other actions, such as turning or pushing for packaging tasks [START_REF] Liang | Simultaneous End-User Programming of Goals and Actions for Robotic Shelf Organization[END_REF]. As the purpose of our evaluation was to show the generalisability of primitive actions with the use of a task planner, we decided to stick to pick-and-place actions with two different grippers. Limitations and interesting extensions of our work are as follows:

1. Our object perception is limited as it does not detect objects that are too close together (e.g., stacked objects). An improved perception system would allow the detection of initial states with stacked objects, automatically detecting goal states, or verifying action executions. 2. Due to the different grippers, we did not program actions that use both arms (e.g., carrying a tray). A possible extension would be to include a better motion and task planning system in order to allow executing both arms simultaneously while avoiding self-collision. 3. We only included a minimal set of predicates (Sec. 5.2) that we deemed intuitive and useful for object manipulation tasks. It could be interesting to include and learn predicates to capture more complex domains such as object orientation [START_REF] Li | Learning object orientation constraints and guiding constraints for narrow passages from one demonstration[END_REF]. 4. iRoPro can easily be adapted to different kinds of robots: the source code is available online (Sec. 5) and developed in ROS, which is a popular framework facilitating the interoperability of a wide variety of robotic platforms. Moreover, the PDDL language used to encode planning domains is platform-independent, and a standard used to benchmark task planners. However, we need to push our standardization effort further by bridging the gap between PDDL and IEEE Standard Ontologies for Robotics and Automation [START_REF] Olszewska | Ontology for autonomous robotics[END_REF].

Conclusion

We presented iRoPro, an interactive Robot Programming system that allows simultaneous teaching of low-and highlevel actions by demonstration. The robot reuses the actions with a task planner to generate solutions to tasks that go beyond the demonstrated action. We conducted pre-design experiments to evaluate the feasibility of the proposed framework and involved concepts in PbD and Automated Planning. Then, we implemented the system on a Baxter robot and showed its generalisability on six benchmark tasks by teaching a minimal set of primitive actions that were reused for all tasks. We demonstrated its usability with a user study where participants with diverse educational backgrounds and programming levels learned how to use the system in less than an hour. Furthermore, we compared user responses from our pre-design experiments with our post-design evaluation and investigate discrepancies. Both user performance and feedback confirmed iRoPro's usability, with the majority ranking it as 'acceptable' and half being promoters. Overall, we demonstrated that our approach allows users with any programming level to efficiently teach robots new actions that can be reused for complex tasks. Thus, iRoPro enables end-users to program robots from scratch, without writing code, therefore maximising the generalisability of taught actions with minimum programming effort.

9 Future Work Future work will focus on exploring more challenging domains to extend the system to other platforms by including a wider range of predicates and probabilistic techniques to improve the condition inference. The next step is to focus on less controlled environments involving factory workers who may ultimately use this technology as well as mobile robots that move around between workspaces. This will require more complex planning domains to consider planning between robots and humans, human-robot collaborative tasks as well as solutions for multi-modal communication involving natural cues [START_REF] Pais | Assessing interaction dynamics in the context of robot programming by demonstration[END_REF].

Fig. 1

 1 Fig. 1 Definition of a planning problem (a) properties describing the initial world state (b) object names and their types (c) instantiated actions (d) properties describing the goal [26].

Fig. 4

 4 Fig. 4 Example of a planning domain in PDDL.

  5): A. Programming by Demonstration: The user teaches the robot primitive actions by demonstration. The robot creates an action model that the user can modify and validate. B. Automated Planning: The user defines a new planning problem with a goal to achieve. The robot reuses the taught actions with a planner to generate solutions for new problems. C. Retro-active Loop: The user observes the robot execution and refines taught actions via the graphical interface.

Fig. 5

 5 Fig.5Overview of the main components of iRoPro: A. the user teaches primitive actions by demonstration B. the robot reuses these with a task planner to generate an action sequence to achieve a goal C. After observing the robot execution, the user can refine the taught action models[START_REF] Liang | Evaluation of a robot programming framework for non-experts using symbolic planning representations[END_REF].

  by inferring them from the observed sensor data during the teaching phase. Object types are inferred based on their detected bounding boxes. Object positions are determined by the proximity of the object to given positions. If the nearest position p to the object o is within a certain threshold d, then the predicates 'o is on p' and 'p is not clear' are added to the detected world state. To infer action conditions, the robot perceives the initial world state before and after the action demonstration as seen in similar work for learning object manipulation tasks[START_REF] Ahmadzadeh | Learning symbolic representations of actions from human demonstrations[END_REF]. Let O 1 = {φ 1 , φ 2 , ...} be the set of predicates observed before the action demonstration and O 2 = {ψ 1 , ψ 2 , ...} after. The action inference is the heuristic deduction of predicates that have changed between O 1 and O 2 , i.e., pre(a) = (O 1 -

Fig. 7

 7 Fig. 7 Example of a high-level action for moving an object from A to B. Conditions are inferred from the observed predicates before (O 1 ) and after (O 2 ) the demonstration [28].

Fig. 8

 8 Fig.8The iRoPro interface showing the action condition menu and an interactive visualisation of the Baxter robot and detected objects.

Fig. 9

 9 Fig.9The iRoPro interface showing the problems menu and the action sequence generated by the task planner.

Fig. 11

 11 Fig. 11 Continuous refinement of the move action model: (a) initial action model learned by demonstration, (b) action model for all cubes of any colour, (c) action model with an additional condition, if the target position is occupied and cubes can not be stacked [26].

Fig. 12

 12 Fig. 12 Snapshots of an example task execution for stacking CUBE objects with the claw gripper.

Fig. 13

 13 Fig. 13 User responses from the post-study survey comparing a) the pre-design study (Sec. 6.2) with b) the post-design study (Sec. 6.4).

Fig. 14

 14 Fig.14Baxter icons used for the graphical interface[START_REF] Freedman | The Rise of the Robotic Work Force[END_REF].

ELEMENT OBJECT BASE POSITION CUBE ROOF A M C B

	1 (b)) and a potential type hier-
	archy. For example, a type hierarchy, consisting of a gen-
	eral type ELEMENT, divided into POSITION and OB-
	JECT, which further divides into BASE, CUBE, and ROOF

  Qualitative data: video recording of the experiment, observations during the experimental protocol. 2. Quantitative data: task duration and UI activity log, pre-study test, post-study survey.

	Task description	Main solution
	(1) move BASE object (suction grip) create new action (+demo)
	(2) move BASE object to any position create new problem
	3 swap two BASE objects	add condition ('is clear')
	4 stack CUBE on BASE	modify types ('OBJECT')
	5 do not stack CUBE on ROOF	add condition ('is stackable')
	6 move ROOF object (claw grip)	create new action (+demo)
	7 stack ROOF on a CUBE	modify types ('ELEMENT')
	8 build a house (BASE, CUBE, ROOF) navigate autonomously
	1.

Table 2

 2 User performance comparing task completion times with prestudy test scores[START_REF] Liang | End-User Programming of Low-and High-Level Actions for Robotic Task Planning[END_REF].

	Main tasks (in min) Pre-score (out of 7)
	AVG	STD AVG	STD
	no experience	43.6	5.37	5.8	0.95
	non-CS	36.6	7.46	6.2	0.55
	CS	43.8	14.13	5.3	1.11
	Overall	41.2	9.08	5.8	0.91
	Group 1	41.0	7.89	6.1	0.77
	Group 2	41.4	10.56	5.5	0.98
	H5) Pre-study test
	As expected, participants who demonstrated a better under-
	standing of the introduced concepts in the pre-study test
	completed the main tasks (Tasks 3-8) faster on average (p-
	value< 0.05). Users scored between 4.3-6.93 out of 7 points.
	'non-CS' users scored above average points and completed
	the fastest. As an outlier we observed that the fastest par-
	ticipant scored only 4.7, but easily learned how to use the
	system and completed the tasks in 22 minutes. Even though
	Group 1 performed slightly better in the pre-study test than
	Group 2, both completion times were on average similar.

https://github.com/ysl208/iRoPro/tree/cond

https://github.com/jstnhuang/rapid_pbd

http://docs.ros.org/indigo/api/pddl_planner

https://github.com/jstnhuang/surface_perception

Video can be seen at https://youtu.be/NgaTPG8dZwg

Acknowledgements The authors would like to thank Nadine Mandran, research engineer at the Laboratoire d'Informatique de Grenoble, for her support and guidance during for the experimental studies.


Maxence Grand
email: maxence.grand@univ-grenoble-alpes.fr

Damien Pellier
email: damien.pellier@univ-grenoble-alpes.fr

Humbert Fiorino
email: humbert.fiorino@univ-grenoble-alpes.fr

TempAMLSI : Temporal Action Model Learning based on Grammar Induction

Hand-encoding PDDL domains is generally accepted as difficult, tedious and error-prone. The difficulty is even greater when temporal domains have to be encoded. Indeed, actions have a duration and their effects are not instantaneous. In this paper, we present TempAMLSI, an algorithm based on the AMLSI approach able to learn temporal domains. Tem-pAMLSI is based on the classical assumption done in temporal planning that it is possible to convert a non-temporal domain into a temporal domain. TempAMLSI is the first approach able to learn temporal domain with single hard envelope and Cushing's intervals. We show experimentally that TempAMLSI is able to learn accurate temporal domains, i.e., temporal domain that can be used directly to solve new planning problem, with different forms of action concurrency.

Introduction

Thanks to description languages like PDDL [START_REF] Mcdermott | PDDL-the planning domain definition language[END_REF], AI planning has become more and more important in many application fields. One reason is the versatility of PDDL to represent durative actions [START_REF] Coles | PDDL2.1: an extension to PDDL for expressing temporal planning domains[END_REF], i.e. actions that have a duration, and whose preconditions and effects must be satisfied and applied at different times.

Temporal PDDL domains have different levels of required action concurrency (Cushing et al. 2007). Some of them are sequential, which means that all the plan parts containing overlapping durative actions can be rescheduled into a completely sequential succession of durative actions: each durative action starts after the previous durative action is terminated. One important property of sequential temporal domains is that they can be rewritten as classical domains, and therefore used by classical non-temporal planners. Some temporal domains require different forms of action concurrence such as Single Hard Envelope (SHE) (Coles et al. 2009). SHE is a form of action concurrency where a durative action can be executed only if another durative action called the envelope, is executed simultaneously. This is due to the fact that the enveloped durative action needs a resource, during all its execution, added at the start of the execution of the envelope and deleted at the end of the execution of the envelope. One important property of SHE temporal domains is that they cannot be sequentially rescheduled.

Hand-encoding PDDL domains is generally considered difficult, tedious and error-prone by experts, and this is even more harder with action concurrency. It is therefore essential to develop tools allowing to acquire temporal domains.

To facilitate PDDL domain acquisition, different machine learning algorithms have been proposed. First, for classical domains as for instance, ARMS [START_REF] Yang | Learning action models from plan examples using weighted MAX-SAT[END_REF], SLAF [START_REF] Shahaf | Learning partially observable action schemas[END_REF], Louga [START_REF] Kucera | LOUGA: learning planning operators using genetic algorithms[END_REF], LSONIO [START_REF] Mourão | Learning STRIPS operators from noisy and incomplete observations[END_REF], LOCM (Cresswell, McCluskey, and West 2009), IRale [START_REF] Rodrigues | Incremental learning of relational action models in noisy environments[END_REF], PlanMilner [START_REF] Segura-Muros | Learning numerical action models from noisy and partially observable states by means of inductive rule learning techniques[END_REF]. In these approaches, training data are either (possibly noisy and partial) intermediate states and plans previously generated by a planner, or randomly generated action sequences. These learning techniques are promising, but they cannot be used to learn temporal domains. [START_REF] Garrido | Learning temporal action models via constraint programming[END_REF] have proposed an algorithm to learn temporal domains using CSP techniques, however their approach is limited to sequential temporal domains. To our best knowledge, there is no learning approach for both SHE and sequential temporal domains.

Several temporal planners (Fox and Long 2002;[START_REF] Halsey | Crikey-a temporal planner looking at the integration of scheduling and planning[END_REF][START_REF] Celorrio | Temporal planning with required concurrency using classical planning[END_REF][START_REF] Blanco | Forward-search temporal planning with simultaneous events[END_REF] attempt to exploit classical planning techniques for temporal planning. These planners convert a temporal domain into a classical domain, i.e. a domain containing non-durative action, generate a plan using this classical domain, and use rescheduling techniques to make the plan compatible with durative actions. Our contribution is to propose an approach exploiting the conversion of temporal domains into classical domains, initially proposed to solve temporal planning problems, for the temporal domain learning task. More precisely, in this work we assume that is possible to reduce the temporal domain learning task into the classical domain learning task.

In this paper, we present TempAMLSI, a learning algorithm for temporal domains including different levels of required action concurrency. TempAMLSI is built on AMLSI (Grand, Fiorino, and Pellier 2020a), a PDDL domain learner based on grammar induction. Like AMLSI, TempAMLSI takes as input feasible and infeasible action sequences to frame what is allowed by the targeted domain. More precisely, TempAMLSI consists of three steps:

(1) TempAMLSI converts temporal sequences into non-temporal sequences, (2) TempAMLSI learns a classical domain containing non-durative action using AMLSI, and (3) converts it into a temporal domain containing durative actions.

The rest of the paper is organized as follows. In section 2 we present a problem statement. In section 3 we give some backgrounds on AMLSI approach and, in section 4, we detail TempAMLSI steps. Finally, section 5 evaluates the performance of TempAMLSI on IPC temporal benchmarks.

Problem Statement

This section introduces a formalization of planning domain learning which consisting in learning a transition function of a grounded planning domain, and in expressing it as PDDL operators.

In classical planning, world states s are modeled as sets of logical propositions, and actions change the world states. Formally, let S be the set of all the propositions modeling properties of world, and A the set of all the possible actions in this world. A state s is a subset of S and each action a ∈ A is a triple of proposition sets (ρ a , + a , - a ), where ρ a , + a , - a ⊆ S, and + a ∩ - a = ∅. ρ a are the preconditions of action a, that is, the propositions that must be in the state before the execution of action a. + a and - a are respectively the positive (add list) and the negative (del list) effects of action a, that is, the propositions that must be added or deleted in s after the execution of the action a. Therefore, learning a classical planning domain consists in learning the deterministic state transition function γ : S × A → S defined as: γ(s, a) = (s ∪ + a ) \ - a where γ(s, a) exists if a is applicable in s, i.e., if and only if ρ a ⊆ s.

In temporal planning [START_REF] Coles | PDDL2.1: an extension to PDDL for expressing temporal planning domains[END_REF], states are defined as in classical planning. However, the action set A is a set of durative actions. A durative action a is composed of: • d a ; the duration • ρ a (s), ρ a (e), ρ a (o): preconditions of a at start, over all, and at end, respectively. • + a (s), + a (e): positive effects of a at start and at end, respectively.

• - a (s), - a (e): negative effects of a at start and at end, respectively. The semantics of durative actions is defined in terms of two discrete events start a and end a , each of which is naturally expressed as a classical action. Starting a durative action a in state s is equivalent to applying the classical action start a in s, first verifying that ρ starta holds in s. Ending a in state s is equivalent to applying end a in s , first by verifying that ρ enda holds in s . start a and end a are defined as follows:

start a : ρ a (s) = ρ starta + a (s) = + starta - a (s) = - starta end a : ρ a (e) = ρ enda + a (e) = + enda - a (e) = - enda
This process is restricted by the duration of a, denoted d a and the over all precondition. Event end a has to occur exactly d a time units after start a and the over all precondition has to hold in all states between start a and end a . Although a has a duration, its effects apply instantaneously at the start and end of a, respectively. The preconditions ρ a (s) and ρ a (e) are also checked instantaneously, but ρ a (o) has to hold for the entire duration of a. The structure of a durative action is summarized in the Figure 1.

A temporal action sequence is a set of action-time pairs π = {(a 1 , t 1 ), . . . , (a n , t n )}. Each action-time pair (a, t) ∈ π is composed of a durative action a ∈ A and a scheduled start time t of a, and induces two events start a and end a with associated timestamps t and t + d a , respectively. Events start a (resp. end a ) is applied in the state s t (resp. s t+da ), s t (resp. s t+da ) being a state time-stamped with t (resp. t + d a ). Then, the temporal transition function γ to learn can be rewritten as: γ(s, a, t) = (γ(s t , start a ), γ(s t+da , end a )). The transition function γ(s, a, t) is defined if and only if: ρ a (s) ∈ s t , ρ a (e) ∈ s t+da and ∀t such that t ≤ t ≤ t + d a ρ a (o) ∈ s t .

To learn the state transition function γ and to express it as a PDDL temporal domain, we assume that:

• we are able to observe temporal sequences of state/action defined recursively as follows:

Γ(s, π) =    [s] if π = ∅ [s] if γ(s, a 0 , t 0 ) undef [s] + Γ(γ(s, a 0 , t 0 ), [(a 1 , t 1 ), .., (a n , t n )]) otherwise
where observed states s can be possibly partial. A partial state is a state where some propositions are missing.

• for all action a = (η a , d a , ρ a (s), ρ a (e), ρ a (o), + a (s), + a (e), - a (s), - a (e)) in the sequences of state/action, η a , the name of a is known, d a is a known constant, and ρ a (s), ρ a (e), ρ a (o), + a (s), + a (e), - a (s), and - a (e) are unknown.

• learned temporal domains can required different forms of action concurrency (see Figure -2) such as Single Hard Envelope (SHE) (Coles et al. 2009). SHE is a form of action concurrency where the execution of a durative action a is required for the execution of a second durative action a . Formally, a SHE is a durative action a that adds a proposition p at start and deletes it at end while p is an over all precondition of a durative action a. Contrary to sequential temporal domains, for temporal domains containing SHE there exists temporal action sequences that cannot be sequentially rescheduled. For instance, the Match domain and the the following actions:

-M EN D(f m) such that (light m) ∈ ρ M EN D(f m) (o) -LIGHT (m) such that (light m) ∈ + LIGHT (m) (s) and (light m) ∈ - LIGHT (m) (e)
The durative action M EN D(f m) cannot start before the start of the durative action LIGHT (m) and M EN D(f m) cannot end after the end of LIGHT (m), so M EN D(f m) have to start after the start of LIGHT (m) and M EN D(f m) have to end before the end of LIGHT (m), it is therefore impossible to sequentially reschedule any temporal action sequences containing these actions. Finally, note that there is other forms of required action concurrency than SHE (Cushing et al. 2007). The AMLSI algorithm consists of 3 steps: (1) AMLSI learns a Deterministic Finite Automaton (DFA) corresponding to the regular grammar generating the action sequences in I + and forbidding those in I -; (2) AMLSI induces the PDDL operators from the learned DFA; (3) finally, AMLSI refines these operators to deal with noisy and partial state observations.

ρ a (s) + a (s) - a (s) ρ a (o) ρ a (e) + a (e) - a (e) d a
The first step consisting in learning the DFA is carried out by using a variant of the classic algorithm for learning regular grammar RPNI [START_REF] Oncina | Inferring regular languages in polynomial update time[END_REF]. The RPNI algorithm used by AMLSI deals with PDDL features and encodes the links between preconditions and effects of actions to speedup the learning process (see (Grand, Fiorino, and Pellier 2020b) for a complete description). Formally, the learned DFA is a quintuple (A, N, n 0 , γ, F ), where A is the set of actions, N is the set of nodes, n 0 ∈ N is the initial node, γ is the node transition function, and F ⊆ N is the set of final nodes.

The second step consists in generating the PDDL operators of the planning domain to learn. To carry out this step, AMLSI must first know which node of the DFA corresponds to which observed state. Thus, AMLSI maps the pairs "node, action" in the DFA with the pairs "state, action" of all π ∈ I + (see Figure 3) and labels the propositions of The last step consists in refining the PDDL operators induced at step 2 to deal with noisy and partial state observations. First of all, AMLSI starts by refining the operator effects to ensure that the generated operators allow to regenerate the induced DFA. To that end, AMLSI adds all effects allowing to ensure that each transition in the automaton are feasible. Then, AMLSI refines the preconditions of the operators. AMLSI makes the following assumptions as in [START_REF] Yang | Learning action models from plan examples using weighted MAX-SAT[END_REF]: The negative effects of an operator must be used in its preconditions. Thus, for each negative effect of an operator, AMLSI adds the corresponding proposition in the preconditions. Since effect refinements depend on preconditions and precondition refinements depend on effects, AMLSI repeats these two refinements steps until convergence, i.e., no more precondition or effect is added. Finally, AMLSI performs a Tabu Search to improve the PDDL operators independently of the induced DFA, on which operator generation is based. The fitness score used to evaluate a candidate set D of PDDL operators is:

J(D|I + , I -) = J ρ (D|I + ) + J (D|I + )+ J + (D|I + ) + J -(D|I -)
where :

• J ρ (D|I + ) = π∈I+ (s,a)∈Γ(s0,π) Accept(ρ a , s) -Reject(ρ a , s)
computes the fitness score for the preconditions of the actions a. Accept(ρ a , s) counts the number of positive and negative preconditions in the observed state s, and Reject(ρ a , s) counts the number of positive and negative preconditions that are not in s.

• J (D|I + ) = π∈I+ (s,a,s )∈Γ(s0,π)

Equal(s , γ(s, a)) -Different(s , γ(s, a))

computes the fitness score for the effects of the action a. s (resp. s ) is the observed states before (resp. after) the execution of the action a. Equal(s , γ(s, a)) counts the number of similar propositions in s and γ(s, a), and Dif f erent(s , γ(s, a)) counts the differences.

• J + (D|I + ) = Once the Tabu Search reaches a local optimum, AMLSI repeats all the three refinement steps until convergence.

Temporal AMLSI

The TempAMLSI approach is summarized by the Figure -4. After having built the samples containing temporal sequences (including both feasible and infeasible sequences), TempAMLSI converts the temporal samples into non-temporal sequences (see Section -4.1). We use the AMLSI algorithm to learn an intermediate classical PDDL domain, and then convert it into a temporal PDDL domain (see Section -4.2). Finally TempAMLSI performs a temporal refinement step (see ). This step allows to take into account the temporal constraints which are not present in the non-temporal sequences.

Sample conversion

Let us take a sample containing π T , π T being a positive temporal sequence such that:

π T = {(0, LIGHT (m)), (0.5, M EN D(f 1 , m)), (2.6, M EN D(f 2 , m))}
We can convert each durative action in the following way: Each durative action a is converted into two event actions start(a) and end(a). After conversion, we have the following sample:

π = {start(LIGHT (m)), start(M EN D(f 1 , m)), end(M EN D(f 1 , m)), start(M EN D(f 2 , m)), end(M EN D(f 2 , m)), end(LIGHT (m))}
Negative temporal sequences are converted in the same way.

Domain conversion

After having learned the classical PDDL domain with AMLSI, TempAMLSI converts it PDDL operators into a temporal one in the following way:

ρ a (s) = ρ start(a) \ ρ end(a) , + a (s) = + start(a) , - a (s) = - start(a) ρ a (e) = ρ end(a) \ ρ start(a) , + e (s) = + end(a) , - e (s) = - end(a) ρ a (o) = ρ start(a) ∩ ρ end(a)
First of all, At start (resp. at end) effects are the effects of start (resp. end) classical operators. Then, Overall preconditions are the intersection of preconditions of start and end classical operators. Finally, At start (resp. at end) preconditions are preconditions of the start (resp. end) classical operator excluding end (resp. start) preconditions. The Figure -5 gives an example of action conversion for the MEND operator of the Match domain.

temporal refinement

For domains that are not sequential (see Section -2) it is possible that some non-temporal actions sequences are both feasible and infeasible, since the feasibility of an action sequence depends on temporal constraints which are not present in non-temporal sequences. For instance, the action sequence:

π 1 = {start(LIGHT (m)), start(M EN D(f 1 , m)}
could be both a negative example and the prefix of a positive example. Indeed, π 1 is the non-temporal action sequence returned after the conversion of the following negative temporal action sequence:

π T 1 = {(0, LIGHT (m)), (4, M EN D(f 1 , m))} However, π 1 is also the prefix of the action sequence π 2 = {start(LIGHT (m)), start(M EN D(f 1 , m) end(M EN D(f 1 , m)), end(LIGHT (m))}
which is the sequence returned by the conversion of the following positive example:

π T 1 = {(0, LIGHT (m)), (2, M EN D(f 1 , m
))} For this type of domain, it is therefore impossible for the Tabu search of the AMLSI algorithm to reach a global maximum (see Section -3). Let us take the equation -3 and suppose that π 2 ∈ I + and π 1 ∈ I -, if J + (D|I + ) is maximized then J + (D|I + ) cannot be maximized and vice versa. Indeed, a classical domain cannot both accept π 2 and reject π 1 . However, it is necessary that the temporal domain accepts π T 2 and rejects π T 2 to ensure that (light m) ∈ ρ(o) M EN D . The objective of the temporal refinement step is to take into account the temporal constraints which are not present in the non-temporal sequences.. The temporal refinement step is a Tabu search. The fitness score for this Tabu search is defined as follows:

J T (D|I T + , I T -) = J T + (D|I T + ) + J T -(D|I T -) 2 where J T + (D|I T + ) = π T ∈I T + 1 Accept(D,π T ) |I T + | J T -(D|I T -) = π T ∈I T - 1 Reject(D,π T )
|I T -| Then the neighborhood of a candidate domain D is the set of domains where a precondition or an effect is added or removed in an operator of D. And the search space of the Tabu Search is the set of all possible domains compatible with the following syntax constraints: • ρ a (o) ∩ { + a (s) ∪ + a (e)} = ∅: At start and at end positive effect cannot add propositions present in the the over all precondition.

• ρ a (s) ∩ + a (s) = ∅: At start positive effect cannot add propositions present in the at start precondition.

• ρ a (e) ∩ + a (e) = ∅: At end positive effect cannot add propositions present in the at end precondition.

•a (s) ∩ + a (s) = ∅: An at start effect cannot be both positive and negative.

•a (e) ∩ + a (e) = ∅:An at end effect cannot be both positive and negative.

•a (s) ∩ {ρ a (o) ∪ ρ a (e)} = ∅: An at start effect cannot delete a proposition present in the at end and over all precondition.

• + a (s) ∩ + a (e) = ∅: An at start effect and an at end effect cannot add the same proposition.

•a (s) ∩a (e) = ∅: An at start effect and an at end effect cannot delete the same proposition.

These syntactical constraints are base on the syntactical constraints proposed by [START_REF] Yang | Learning action models from plan examples using weighted MAX-SAT[END_REF] We deliberately choose the size of the test sets larger than the training sets to show TempAMLSI ability to learn accurate domains with small datasets. The training and test sets are generated as follows: at a given state s, we randomly choose a durative action a in A. If a is feasible, the current state is observed, and we add a to the current π. This random walk is iterated until π reaches an arbitrary length (randomly chosen between 5 and 15), and added to I + . If a is infeasible in the current state, the concatenation of π and a is added to I -. In the test sets E + and E -, we generate action sequences with a length randomly chosen between 1 and 30.

We test each domains with three different initial states over five runs, and we use five seeds randomly generated for each run. All tests were performed on an Ubuntu 14.04 server with a multi-core Intel Xeon CPU E5-2630 clocked at 2.30 GHz with 16GB of memory.

Evaluation Metrics

We evaluate TempAMLSI with four different metrics. The first metric, the syntactical error, is the most used metric in the literature. The other metrics, the FScore, the accuracy and the IPC score are more specific to our approach.

• Syntactical Error : The syntactical error error(a) for an action a is defined as the number of extra or missing predicates in the preconditions ρ a (s, e, o), the positive effects + a (s, e) and the negative effects - a (s, e) divided by the total number of possible predicates [START_REF] Zhuo | Learning complex action models with quantifiers and logical implications[END_REF]. The syntactical error for a domain with a set of actions A is:

E σ = 1 |A| a∈A error(a).
• FScore: This metric is initially used for pattern recognition and binary classification [START_REF] Rijsbergen | Information Retrieval[END_REF]). Nevertheless, it can be used to evaluate the quality of a learned grammar. Indeed, a grammar is equivalent to a binary classification system labeled with {1, 0}. For grammars we can assume that the sequences belonging to the grammar are data labeled with 1, and non-grammar sequences are data labeled with 0. This metric is therefore able to test to what extent the learned domain D can regenerate the grammar. A domain D can regenerate a grammar if D accept, i.e. can generate, all positive test sequences e ∈ E + and reject, i.e. cannot generate, test negative sequences e ∈ E -. Formally, the FScore is computed as follows: FScore = 2.P.R P +R where R is the recall, i.e. the rate of sequences e accepted by the ground truth domain that are successfully accepted by the learned domain, computed as follows: R = |{e∈E + | accept(D,e)}| |E| and P is the precision, i.e. the rate of sequences e accepted by the learned domain that are sequences accepted by the ground truth domain, computed as follow: P = are able to solve new planning problems [START_REF] Zhuo | Refining incomplete planning domain models through plan traces[END_REF]. Most of the works addressing the problem of learning planning domains uses the syntactical error to quantify the performance of the learning algorithm. However, domains are learned to be used for planning, and it often happens that one missing precondition or effect, which amounts to a small syntactical error, makes them unable to solve planning problems. Formally, the accuracy Acc = N N * is the ratio between N , the number of correctly solved problems with the learned domain, N * , the total number of problems to solve. The accuracy is computed over 20 problems. We also report in our results the ratio of (possibly incorrectly) solved problems.

In the experiments, we solve the test problems with different planners. Instances of Sequential domains and SHE domains are solved with the TP-SHE [START_REF] Celorrio | Temporal planning with required concurrency using classical planning[END_REF] planner and instances of the Cushing domain are solved with the Tempo planner (Celorrio, Jonsson, and Palacios 2015). We use different planners because some planners have good results with only some forms of action concurrency. For instance, TP-SHE is the domain with the best performances for instances with Single Hard Envelope but has bad results for Cushing. Plan validation is realized with the automatic validation tool used in the IPC competition VAL [START_REF] Howey | Val's progress: The automatic validation tool for pddl2. 1 used in the international planning competition[END_REF].

• IPC quality score: This metric is initially used to compared different planners. We can use to test a learned domain. The score of a solved problem is the ratio C * /C where C is the cost of the plan found by the learned domain and C * is the cost of the plan founded by the reference domain. The cost of a plan is the sum of all durations of all actions of the plan. The score on an unsolved prob-lem is 0. The score of a learned domain is the sum of its scores for all problems.

Results

In this section we perform an ablation study and compare two variants: (1) TempAMLSI: TempAMLSI only uses the AMLSI algorithm to learn temporal domain and (2) Tem-pAMLSI + TR: TempAMLSI uses both AMLSI algorithm and temporal refinement to learn temporal domains.

Learning with complete observations Table -2 shows TempAMLSI's performance when observations are complete.

First of all, Table -2a shows results for Sequential domains. Firstly, we can note that for each domains Tem-pAMLSI and TempAMLSI+TR variants have same performances, whatever the metric. Then, we observe that learning runtimes is generally higher for TempAMLSI+TR variants. For the Peg Solitaire domain we can observe that learned domains are accurate. More precisely we observe that both classical and temporal F Score and accuracy are optimal, also we observe that IP C = 20 that implies that plans generated with learned domains are identical to the plans generated with the ground truth domain. Also, the syntactical distance is not optimal (E σ = 2.8%), this is due to the fact that some implicit preconditions which are not encoded in the reference domain and encoded in the learned domain. Then, for the Sokoban and Parking domain results are similar. However, we can note that the IPC score is not optimal (IP C = 19.9 for Sokoban and Parking), this is due to the fact that some plans found by the learned domains has a higher cost than the plan found by the reference domain. Then, for the Zenotravel domain we can observe that Temporal Fscore and accuracy are optimal. Also, we observe that the classical FScore is not optimal, this is due to the fact that some preconditions encoded as at start preconditions in the reference domain are encoded as overall preconditions in the learned domains.

Then, Table -2b gives results for SHE domains. For the Turn and Open domain we can observe that, whatever the variant used, only the temporal FScore is optimal. This is due to the fact that the Turn and Open domain contains SHE. Also we can note that the learned domains are able to solve the majority of new problems (Acc = 55%). For this domain, we observe that the temporal refinement has no impact, this is due to the fact that the Turn and Open domain contains both sequential operators (move, pick, drop) and SHE operators (open -door, turndoorknob). Then, for the Match domain we can observe that the domain learned with temporal refinement has better results for syntactical distance and both classical and temporal FScore. A better temporal Score means that the domains learned with the TempAMLSI+TR variant better respect the forms of required action concurrency of the ground truth domain. However, TempAMLSI and TempAMLSI+TR variants have the same accuracy (Acc = 93.3%). Finally, we observe that learning runtimes are higher for the TempAMLSI+TR variant for all SHE domains.

Finally, for the Cushing domain (see observe that the temporal refinement step only increases the temporal FScore. This implies that the forms of concurrency present in the domains learned with the Tem-pAMLSI+TR variant are closer to the forms of concurrency in the ground truth domain than the domains learned with the TempAMLSI variant. Also, the TempAMLSI variant gives better syntactical distance and classical FScore than the TempAMLSI+TR variant. Also, we observe that the learned domains cannot solve problem whatever the variant used. Finally, we note that the temporal refinement strongly increases the learning runtimes.

Learning with partial observations Figures -6, 7 and 8 show how temporal FScore varies when the level of observed proposition in intermediate states varies. First, we can observe that for all sequential domains (see Figure -6) the temporal refinement step has no effect. Also, Temporal FScore is always optimal whatever the level of observability. Then, Table -7 shows that, for SHE domains, the TempAMLSI+TR variant gives better results. We observe that FScore is optimal when at least 40% of propositions are observed for the Turn and Open and when at least 60% of propositions are observed for the Match domain. Finally, Table -8 shows that for the Cushing domain the TempAMLSI+TR variant gives better. However, Tem-pAMLSI+TR never gives the optimal FScore.

Conclusion

In this paper we presented TempAMLSI, a novel algorithm to learn temporal PDDL domains. TempAMLSI is based on the AMLSI approach. In this paper we reused the idea to use classical PDDL domain proposed by several tem-poral planners. More precisely, TempAMLSI converts the temporal sample into a sample containing non-temporal sequences, then TempAMLSI uses the AMLSI algorithm to learn a classical PDDL domain and convert it into a temporal PDDL domain, Also, TempAMLSI has a temporal refinement step allowing to deal with different forms of required action concurrency. Finally, we show experimentally that the TempAMLSI approach was able to learn accurate domains with sequential sequences and single hard envelopes. In future works, TempAMLSI will be extended to learn temporal PDDL domain with noisy observations and temporal PDDL domain with other form of required action concurrency than Single Hard Envelopes. Also, TempAMLSI will be extended to be able to deal with different ways for the durative action conversion, such as the LGP translation (Fox and Long 2002) for instance. Observability

Figure 1 :

 1 Figure 1: Structure of a durative action a

Figure 2 :

 2 Figure 2: Different form of required action concurrency

Figure 3 :

 3 Figure 3: Mapping overview

π∈I+ |π| × 1 1

 1 Accept(D,π) where 1 Accept(D,π) = 1 if and only if D can generate the positive sample π. |π| is the length of π. J + (D|I + ) is weighted by the length of π ∈ I + because I + is smaller than I - • J -(D|I -) = π∈I-Reject(D,π) where 1 Reject(D,π) = 1 if and only if D cannot generate the negative samples π.

Figure 4 :

 4 Figure 4: Overview of the TempAMLSI approach

  |{e∈E + | accept(D,e)}| |{e∈E + | accept(D,e)}∪{e∈E -| accept(D,e)}| . • Accuracy: It quantifies to what extent learned domains

Figure 6 :

 6 Figure 6: Learned domains Syntactical distance with different levels of observability

Figure 7 :Figure 8 :

 78 Figure 7: Learned domains Syntactical distance with different levels of observability

Table -

 -

	Domain		Algorithm	E σ		FScore (classical) FScore (temp) Solved	Acc	IP C Time (sec)
	Peg	TempAMLSI TempAMLSI+TR 2.8% 2.8%		100% 100%	100% 100%	100% 100%	100% 100%	20 20	4.2 3.2
	Sokoban	TempAMLSI TempAMLSI+TR 0.1% 0.1%		100% 100%	100% 100%	100% 100%	100% 19.9 100% 19.9	23.4 27
	Parking	TempAMLSI TempAMLSI+TR 5.6% 5.6%		100% 100%	100% 100%	100% 100%	100% 19.9 100% 19.9	62.3 74.2
	Zenotravel	TempAMLSI TempAMLSI+TR 0.7% 0.7%		72.1% 72.1%	100% 100%	100% 100%	100% 100%	20 20	20.1 20
				(a) Domain learning results on sequential temporal domains.
	Domain		Algorithm	E σ		FScore (classical) FScore (temp) Solved	Acc	IP C Time (sec)
	Turn and Open	TempAMLSI TempAMLSI+TR 3.2% 3.2%	82.4% 82.4%	100% 100%	55% 55%	55% 55%	10.7 10.7	36.6 37.3
	Match		TempAMLSI TempAMLSI+TR 2.7% 5.3%	86% 87.4%	82.3% 100%	93.3% 93.3% 18.7 93.3% 93.3% 18.7	1.8 4.3
				(b) Domain learning results on SHE temporal domains.
	Domain		Algorithm	E σ	FScore (classical) FScore (temp) Solved Acc IP C Time (sec)
	Cushing	TempAMLSI TempAMLSI+TR 8.2% 7.9%		83.6% 90.7%	47.4% 90.6%	0% 0%	0% 0%	0 0	23.9 451.3
									2c) we

(c) Domain learning results on Cushing temporal domain.

Table 2 :

 2 Domain learning results on 7 domains when observations are complete. TempAMLSI performance is measured in terms of, syntactical error E σ , FScore for temporal and classical samples, accuracy Acc, IPC Score and runtimes.

Acknowledgments

This research is supported by the French National Research Agency under the "Investissements d'avenir" program (ANR-15-IDEX-02) on behalf of the Cross Disciplinary Program CIRCULAR.