Jose Sanchez

Juan-Antonio Corrales

Belhassen-Chedli Bouzgarrou

Youcef Mezouar

Robotic Manipulation and Sensing of Deformable Objects in Domestic and Industrial Applications: A Survey

Keywords: Deformable objects, Robotic manipulation, Cloth manipulation, Deformation control

We present a survey of recent work on robot manipulation and sensing of deformable objects, a field with relevant applications in diverse industries such as medical (e.g. surgery assistance), food handling, manufacturing, and domestic chores (e.g. folding clothes). We classify the reviewed approaches into four categories based on the type of object they manipulate. Furthermore, within this object classification we divide the approaches based on the particular task they perform on the deformable object. Finally, we conclude this survey with a discussion of the current state of the art and propose future directions within the proposed classification.

Introduction

Endowing robots with the ability to manipulate deformable objects spawns diverse applications with tremendous economical benefit. For instance, using robots to handle fragile products in the food industry could reduce labor cost [START_REF] Tokumoto | Deformation modeling of viscoelastic objects for their shape control[END_REF][START_REF] Buckingham | Robotics for de-heading fish -a case study[END_REF], manufacturing plants could use robots to manipulate flexible objects in order to lessen physically burdens on workers [START_REF] Acker | Manipulating deformable linear objects: characteristic features for vision-based detection of contact state transitions[END_REF][START_REF] Rambow | Autonomous manipulation of deformable objects based on teleoperated demonstrations[END_REF] or robots becoming more involved in caregiving activities (e.g. dressing, feeding) for the elder and disabled [START_REF] Chen | Robots for humanity: using assistive robotics to empower people with disabilities[END_REF][START_REF] Yamazaki | Bottom dressing by a life-sized humanoid robot provided failure detection and recovery functions[END_REF].

Although manipulation and grasping of rigid objects is a mature field in robotics, with over three decades of work, the study of deformable objects has not been as extensive in the robotics community. Another important issue is that many of the techniques and strategies devised for the manipulation of rigid objects cannot be directly applied to deformable objects. For instance, two of the main conditions applied to grasping rigid objects are form closure and force closure.

The former consists in applying kinematic constraints on an object such that the object cannot perform any relative motion [START_REF] Nguyen | Constructing Force-Closure Grasps[END_REF], and the latter considers a set of contact points such that contact forces can balance an arbitrary external wrench [START_REF] Bicchi | On the Closure Properties of Robotic Grasping[END_REF]. When it comes to deformable objects, form closure fails to immobilize every degree of freedom of the object since deformable objects have infinite degrees of freedom [START_REF] Guo | Squeeze grasping of deformable planar objects with segment contacts and stick/slip transitions[END_REF]. Similarly, force closure relies on applying a set of forces to restrain the object's movement based on its undeformed shape. However, the required contact forces applied to the deformable object rely on its deformation [START_REF] Jia | On two-finger grasping of deformable planar objects[END_REF]. This entails that the force closure calculation needs to be performed continuously in order to consider the object's new shape caused by the grasping action. Furthermore, rigid object manipulation considers mostly the control of the grasped object's pose, as noted by Khalil et al. (2010). However, the manipulation of a deformable object also needs to address its deformation. This consideration, as well as the inapplicability of the aforementioned conditions, has led to many diverse approaches to handle deformable objects with robots.

In this survey, we review the latest advances on the topic of robotic manipulation of deformable objects. Previous surveys have covered topics such as manipulation of deformable objects in industrial applications [START_REF] Henrich | Robot Manipulation of Deformable Objects[END_REF][START_REF] Saadat | Industrial applications of automatic manipulation of flexible materials[END_REF], or planning the 1 Universit é Clermont Auvergne, SI GMA Clermont, Institut Pascal 2 CNRS manipulation of deformable objects [START_REF] Jimenez | Survey on model-based manipulation planning of deformable objects[END_REF]. [START_REF] Henrich | Robot Manipulation of Deformable Objects[END_REF] reported the first attempts of the robotic community to manipulate deformable objects. This was followed by the survey of [START_REF] Saadat | Industrial applications of automatic manipulation of flexible materials[END_REF], where the objects are classified based on their shape, material and industrial application. More recently, [START_REF] Jimenez | Survey on model-based manipulation planning of deformable objects[END_REF] presented a survey that focuses on object modeling and on manipulation planning of deformable objects. Khalil and Payeur (2010) present a very comprehensive survey, where most of the surveyed papers focus on modeling and simulation, and approaches that have been applied to industrial tasks such as assembly and food handling.

However, since these surveys, the field of deformable object manipulation has seen significant contributions.

Our survey focuses on recent advances in the robotics community to address the sensing and manipulation of deformable objects, particularly in domestic and industrial applications. We propose a new classification of deformable objects that not only considers their geometric shape (as it has been previously suggested in [START_REF] Jimenez | Survey on model-based manipulation planning of deformable objects[END_REF][START_REF] Saadat | Industrial applications of automatic manipulation of flexible materials[END_REF]), but also their physical properties. We consider as deformable the objects that either 1) have no compression strength or 2) have a large strain 1 or present a large displacement. Objects that have no compression strength are those which do not present any resistance when two opposite endpoints are pushed towards each other. Ropes and clothes are examples of objects with no compression strength.

As noted before, objects can also be categorized based on their geometry. Objects having one dimension significantly larger than the other two, for instance the length of a cable is much larger than its width or height, are considered uniparametric. Biparametric objects are those having one dimension considerably smaller than its other two, for instance the thickness of a paper is negligible compared to its width and height. Finally, triparametric objects represent solid objects. Using these two criteria we classify the reviewed approaches into the following four main categories, as shown in Figure 1:

Type I: Uniparametric objects that either have no compression strength such as cables, strings and ropes; or they have a large strain such as elastic tubes and beams. This type of object are widely referred to as linear in the robotics community.

Type II: Biparametric objects that present a large strain, or a large displacement, such as paper, cards and foam sheets. Also, thin-shell objects such as empty plastic bottles and hollow rubber balls are considered in this category. In the literature, these type of objects are commonly referred to as planar objects.

Type III: Biparametric objects not possessing any compression strength. Shirts, pants, curtains and fabric sheets are examples of this type of object. These objects are mostly known as cloth-like objects.

Type IV: Triparametric objects such as a sponges, plush toys and food products fall in this object category.

Common names used to describe these objects are solid or volumetric.

Deformable objects

L arge strain

T riparam B iparam U niparam N o compression strength U niparam B iparam

Physical property

Object's shape Cloth-like "Linear" "Planar" Solid To further divide the approaches, we group them based on their applications, and three main groups are considered:

1) sensing, 2) manipulation and 3) task-specific. The latter group deals with tasks inherent to the object types. For instance, folding a shirt or tying a knot on a rope.

This proposed categorization, based on both object and application type, is succinctly presented in tables that group the approaches for each type of object.

Following this introduction, frequent concepts and terms used in the field of deformable objects are presented in Section 2. Then we review the approaches that focus on type I objects in Section 3, followed by type II objects in Section 4. Section 5 presents approaches handling type III objects and Section 6 focuses on objects of type IV. A occurs when an external force 2 applied to an object results in the object changing its shape. Moreover, depending on the response of the object once the external force is removed the deformation can be either plastic, elastic or elasto-plastic.

A plastic deformation entails a permanent deformation, that is, an object maintains the shape caused by a deforming force even when that force is removed. On the contrary, an elastic deformation results on the object returning to its undeformed shape once the deforming force is removed [START_REF] Callister | Mechanical Properties of Metals[END_REF]. Lastly, an elasto-plastic deformation is a combination of both, elastic and plastic, deformations; where the object does not return to its original shape, but it does not hold the deformation entirely. The deformation types are shown in Figure 2. Figure 3 shows an object being deformed by a tensile load.

The amount of deformation induced by stress is referred to as strain (ε); where stress (σ ) is the ratio between the applied force F and the cross-section area A 0 [START_REF] Callister | Mechanical Properties of Metals[END_REF]. For linear elastic deformations 3 , stress and strain are related by Hooke's law [START_REF] Callister | Mechanical Properties of Metals[END_REF]:

σ = Eε
Where E is the modulus of elasticity, also called Young's modulus, and is measured in pressure units such as Pascal (N/m 2 ) [START_REF] Askeland | Mechanical Properties and Behavior[END_REF].

Another important elasticity parameter is the Poisson's ratio (ν), an adimensional number that relates the ratio between axial and lateral strains [START_REF] Callister | Mechanical Properties of Metals[END_REF]. In Figure 3 The blue dashed lines represent the original, undeformed, shape and the red solid lines represent the deformed shape [START_REF] Callister | Mechanical Properties of Metals[END_REF]).

represented by a set of partial differential equations solved through discretization techniques in order to approximate the displacement field. However, these parameters are only valid for linear elasticity. Linear deformation can refer either to a geometric or a material linearity. Geometrical linearity is not appropriate for large deformations, since only small deformations can be modeled accurately [START_REF] Nealen | Physically based deformable models in computer graphics[END_REF]).

On the other hand, material linearity refers to a deformation that retains a linear stress-strain relation [START_REF] Callister | Mechanical Properties of Metals[END_REF]).

Modeling a deformation can be done with a variety of techniques. Usually, these techniques required a deformation model and a representation of the object's shape, usually by a set of particles or a mesh. A mesh represents an object as set of points (vertices), edges and faces or elements for a two dimensional or a three dimensional object, respectively. The faces are usually triangles or quadrilaterals, and the elements are commonly represented as tetrahedra or hexahedra. The deformation models provide a function to compute the position of every vertex based on their current position and an input force [START_REF] Nealen | Physically based deformable models in computer graphics[END_REF].

Deformation models 4 that do not require a mesh are termed mesh-free (or meshless), and particle based models [START_REF] Tonnesen | Dynamically Coupled Particle Systems for Geometric Modeling, Reconstruction, and Animation[END_REF] with finite element methods (FEM), where the object is split into a set of discrete geometric parts called finite elements in order to approximate the object's shape [START_REF] Moore | A Survey of Computer-Based Deformable Models[END_REF]. A comparison between different physically-based models is shown in Figure 4.

MSD models are more intuitive and simpler to implement than FEM-based models, however FEM-based models are able to produce more physically realistic simulation [START_REF] Moore | A Survey of Computer-Based Deformable Models[END_REF][START_REF] Nealen | Physically based deformable models in computer graphics[END_REF]. Furthermore, MSD models present drawbacks such as inability to preserve volume and inverting easily [START_REF] Moore | A Survey of Computer-Based Deformable Models[END_REF][START_REF] Sin | Vega: Non-linear FEM deformable object simulator[END_REF].

Particlebased

Physically more realistic [START_REF] Sin | Vega: Non-linear FEM deformable object simulator[END_REF]) and the classification presented in [START_REF] Nealen | Physically based deformable models in computer graphics[END_REF].

Model types

Elastic Hyperelastic

Material types

M

This section provided merely a brief summary of interdisciplinary topics that are involved with the simulation of deformations. The interested reader is referred to [START_REF] Nealen | Physically based deformable models in computer graphics[END_REF][START_REF] Moore | A Survey of Computer-Based Deformable Models[END_REF] for more comprehensive surveys of deformable models and modeling techniques in computer graphics. The following references are recommended [START_REF] Callister | Mechanical Properties of Metals[END_REF][START_REF] Askeland | Mechanical Properties and Behavior[END_REF] for a technical review of mechanical properties and elastic behavior.

Linear objects

The first approaches to manipulate deformable objects concentrated mostly on deformable linear objects (DLOs), also called deformable one-dimensional objects (DOOs); such as ropes, elastic rods, beams, cables, etc. One of the reasons for these initial research efforts might be the simplicity of DLOs when compared to planar or solid objects. Also, their simulation is not as computational expensive and assumptions such as modeling them as chains of links allow for simpler algebraic solutions. Some examples of the tasks involving DLOs are displayed in Figure 5, and Table 1 shows a classification of the reviewed approaches.

Tying knots

Tying knots tasks require development of dedicated trajectory planning methods and force control strategies.

They also require perception abilities to detect, localize and track specific elements in ropes (e.g. intersection points) to monitor the task's progress. To meet these requirements, sensors able to accurately measure forces, joint velocities and the state of the rope; as well as actuators able to achieve the necessary robot actions, should be carefully chosen. [START_REF] Yamakawa | Knotting manipulation of a flexible rope by a multifingered hand system based on skill synthesis[END_REF] proposed a simple description of a rope in order to apply manipulation skills to tie a knot. This description consisted on identifying the intersections created by the rope crossing itself. Furthermore, they utilized a highspeed robotic hand with tactile sensing and a vision system in order to apply the manipulation skills required.

Manipulation primitives are used in [START_REF] Vinh | A new strategy for making a knot with a general-purpose arm[END_REF] to perform the task of knot tying. These primitives are identified by observing how a human knots a rope using one hand. Specifically, three primitives were identified: 1) grasping the rope, 2) producing a loop and 3) pulling the rope to create the knot. The robot then followed a set of points, previously extracted from trajectories demonstrated by a human teacher, to perform these primitives. However, the object's behavior is not modeled and thus, the success of the approach depends only on a open loop execution of these primitives by the robot. [START_REF] Kudoh | In-air Knotting of Rope by a Dual-arm Multi-finger Robot[END_REF] extended these manipulation primitives to tie a knot in the air with two dexterous robotic hands (i.e. the fingers are actively controlled). However, this approach is also not robust against disturbance, since no model of the object is derived and no sensor feedback is considered.

Also focusing on tying a knot in the air without a physical model of the object is the work described in [START_REF] Yamakawa | Motion planning for dynamic knotting of a flexible rope with a high-speed robot arm[END_REF]. Their approach differs however, on a dynamic manipulation of the rope that relies on high-speed sensors and actuators (1 KHz). The rope was represented as a chain of joints which is algebraically related to the robot's motion.

Prepared using sagej.cls (a) Knot tying [START_REF] Huang | Leveraging appearance priors in non-rigid registration, with application to manipulation of deformable objects[END_REF].

(b) String insertion (Weifu [START_REF] Wang | An online method for tight-tolerance insertion tasks for string and rope[END_REF].

(c) Rope untangling [START_REF] Lui | Tangled: Learning to untangle ropes with RGB-D perception[END_REF].

(d) Shape manipulation [START_REF] Rambow | Autonomous manipulation of deformable objects based on teleoperated demonstrations[END_REF]. Table 1. Classification of approaches to manipulate deformable linear objects using a robot. The state estimation approaches are divided into discrete and continuous depending on whether they represent the object as a finite set of points/segments or not. LfD [START_REF] Lee | Unifying scene registration and trajectory optimization for learning from demonstrations with application to manipulation of deformable objects[END_REF][START_REF] Lee | Learning force-based manipulation of deformable objects from multiple demonstrations[END_REF]; [START_REF] Huang | Leveraging appearance priors in non-rigid registration, with application to manipulation of deformable objects[END_REF] Moreover, the position of each joint of the rope is affected by a time delay factor that is proportional to the distance from the joint to the grasping point; that is, the time delay at the grasped joint is zero, while the joints that are farther away get an increasing time delay. By determining this time delay parameter they were able to estimate the configuration of the rope in real time allowing the robot to tie the knot.

State estimation

Most recently, learning techniques have been applied in order to solve the problem of tying a knot using a dualarm robot. Specifically, in [START_REF] Lee | Unifying scene registration and trajectory optimization for learning from demonstrations with application to manipulation of deformable objects[END_REF]) learning from demonstration (LfD) was used to learn a function that maps a pairs of correspondence points (i.e. from a demonstrated and a test scene), while minimizing a bending cost (warping cost). The correspondence points consist of a point cloud representing the state of the rope as a point cloud (obtained through an RGB-D sensor) and the gripper's trajectories, which were extracted from kinesthetic demonstrations. The bending cost was computed based on the Thin Plate Spline Robust Point Matching (TPS-RPM) 5 . Also relying on the TPS-RPM to learn a warping function, the approach presented in [START_REF] Huang | Leveraging appearance priors in non-rigid registration, with application to manipulation of deformable objects[END_REF]) used a convolutional neural network to label parts of the rope such as end-points and crossings.

Another approach, based on a geometric formulation, was proposed by [START_REF] Takizawa | Method for placing a rope in a target shape and its application to a clove hitch[END_REF] where a dual-arm robot manipulated the shape of a rope in order to tie it around a pipe. However, their approach did not consider the tightness of the tied knot. This was shown by [START_REF] Lee | Unifying scene registration and trajectory optimization for learning from demonstrations with application to manipulation of deformable objects[END_REF] to be a necessary condition to ensure the knot stays tied to the pipe.

In order to solve this issue, [START_REF] Lee | Learning force-based manipulation of deformable objects from multiple demonstrations[END_REF] extended their previous work to incorporate force information into their learning algorithm and thus achieve a sufficiently tight knot.

Grasping and manipulation

Several tasks that need to manipulate DLOs, such as insertion and shape control, require an accurate estimate of the object's shape as it is being deformed. This is challenging since modeling and perceiving deformation can be extremely complicated in certain configurations, e.g. when the object is deformed by an unexpected obstacle or when self-occlusion occurs. Several works attempt to overcome this issue.

Based on their previous work [START_REF] Yamakawa | Motion planning for dynamic knotting of a flexible rope with a high-speed robot arm[END_REF], which assumes the object follows the end-effector's trajectory if the robot's motion is fast enough, [START_REF] Yamakawa | Simple Model and Deformation Control of a Flexible Rope using Constant, High-Speed Motion of a Robot Arm[END_REF] were able to deform a rope into different shapes such as a rectangular corner and a semi-circle.

The inverse problem of tying ropes, namely untangling ropes, is tackled by [START_REF] Lui | Tangled: Learning to untangle ropes with RGB-D perception[END_REF]. The untangling of a rope was performed by a PR-2 robot using RGB-D data to build a graph that represents the object. To build this graph, the points obtained by the RGB-D sensor were grouped in segments by using a region-growing algorithm which reduced the representation of the rope from thousands of points to hundreds of segments. With this simpler representation, a particle filter was applied to find the best rope configuration for a set of segments. Subsequently a manipulation action was chosen so that the current configuration could be transformed into a desired configuration (i.e. an action that leads to an untangled rope).

Another desired robotic task with ropes is that of insertion, which has applications such as needle threading and assembly tasks. Weifu [START_REF] Wang | An online method for tight-tolerance insertion tasks for string and rope[END_REF] tackled the problem of inserting a rope, and a string, through a series of holes with tight-tolerance diameters with respect to the rope's diameter. To achieve this, vector fields were set in the center of each hole to drive the tip of the rope through the hole.

A similar task to inserting a DLO was presented in [START_REF] Yoshida | Simulation-based optimal motion planning for deformable object[END_REF], where an elastic band (O-ring) was extended such that it could be wrapped around a peg. Unlike, the approaches described previously in this section, this approach relied on FEM to simulate the deformation of an O-ring. A motion planning algorithm 6 , which combined different objective functions such as collision avoidance and minimum deformation, was then used to generate a plan that inserts the O-ring into a peg with a larger diameter, thus requiring the O-ring to be deformed in order to insert it.

Other approaches have focused on deforming a linear object such that it reaches a desired configuration. [START_REF] Rambow | Autonomous manipulation of deformable objects based on teleoperated demonstrations[END_REF] used a two-arm robot to mount a tube in a desired configuration based on a single teleoperated demonstration. The task consisted on manipulating the tube to pass by two constraining walls, thus keeping the tube in the required shape. The demonstration recorded the grippers' poses and the forces produced by the contacts with the rigid environment. Since a mere repetition of the grippers trajectory does not account for the position uncertainties caused by the physically interaction with the environment, a variable admittance control was applied that allowed for a good tracking of the position when there were no contacts but decreased the tracking's accuracy to achieve a more compliant behavior, thus maintaining low contact forces while interacting with the environment. To monitor the contact forces, the robot was equipped with a force-torque sensor attached to its end-effector.

A common action among these manipulation tasks is that of reaching intermediate desired configurations in order to accomplish a final goal such as tying, or untangling, a knot. One way to solve this is to use motion planning algorithms. The seminal work of Kavraki with Lamiraux and, subsequently with Moll [START_REF] Lamiraux | Planning Paths for Elastic Objects under Manipulation Constraints[END_REF]Moll andKavraki 2004, 2006), focused on path planning for DLOs and planar objects; where sampling-based roadmap algorithms (e.g. Rapidly exploring Random Tree) were applied to find intermediate configurations of the object to build a plan that reached a specified configuration. The object was described as a parameterized curve and the sampling algorithm chose curves with minimal energy, since these were assumed to represent the most likely configuration of the object. These, and older works focusing on manipulation planning of deformable objects, have been covered in more depth in [START_REF] Jimenez | Survey on model-based manipulation planning of deformable objects[END_REF].

More recently, [START_REF] Roussel | Deformable Linear Object manipulation planning with contacts[END_REF] proposed an approach that incorporated a physics engine to their motion planning algorithm. Here, the DLO was described as a connection of nodes, where the links were modeled using FEM, and the state space was defined by a vector of all the nodes and their positions and velocities. As the physics engine is able to determine the next state of the object based on its current state and an applied wrench, they sampled different control commands (i.e. wrenches applied by a gripper) to find the one that moved the object closer to their goal state. Additionaly, they exploited the enviroment to apply additional external wrenches to the object, and thus reach configurations that would not be possible using only a gripper. In fact, based on the test scenarios presented, allowing the object to contact the environment was necessary to solve the path planning problem. Similarly, [START_REF] Alvarez | An interactive simulator for deformable linear objects manipulation planning[END_REF] also used a physics engine to simulate the behavior of the deformable object, in combination with a planning algorithm, while allowing a user to modify the simulated environment in an interactive manner. They showed that their planning algorithm was robust against unexpected changes to the environment. An extension of these approaches was recently described in [START_REF] Shah | Towards manipulation planning for multiple interlinked deformable linear objects[END_REF], where a set of interlinked cables is manipulated into an array of clamps. This problem requires the motion planning algorithm to handle an extra constraint, namely that the interlinks between cables do not become taut. They defined a series of actions for two available manipulators that were only allowed to grasp a finite set of points on each cable. Then their planning algorithm searched for the combination of actions that solved the clamping task without violating the constraint of overstretching the interlinks.

Although these approaches might be helpful to other tasks, as they solve the motion planning required to attain desired configurations; they have, so far, not been shown to work in real time as finding an adequate plan usually requires long times (e.g. minutes). Furthermore, these works have only been tested on simulation environments where the object's Prepared using sagej.cls behavior, and its interaction with the environment, are greatly simplified.

Deformation sensing

Estimating the necessary parameters of a linear deformable object to manipulate it is a crucial issue. Several approaches have been proposed with different objectives. For some of them, the objective was to perceive the configuration of the object (e.g. are there knots or intersections? and, if so, where are their locations?); other works have focused on estimating the global shape by either learning parameters that describe the object's deformation or by describing the object in geometric terms. Depending on the approach, different assumptions are required as it is noted below. [START_REF] Matsuno | Manipulation of deformable linear objects using knot invariants to classify the object condition based on image sensor information[END_REF] were able to estimate the state of a rope using a topological description, that is, the rope was described as a node graph. The nodes of the graph represented either the ends of the rope or intersections. In order to distinguish whether the side of the rope was passing over, or under, they used the luminance variance on an image obtained by a stereo vision system.

A more recent approach that is able to perceive the deformation of a one-dimensional object, such as surgical suture, was presented in [START_REF] Javdani | Modeling and perception of deformable one-dimensional objects[END_REF]. Using a stereo vision system together with a simulator, they were able to predict the configuration of the object. Specifically, they learned a set of parameters that minimize an energy function that considers the bending, twist and gravity effects on the object. However, this approach assumes the object does not stretch, that is, its geodesic distance remains unchanged while being deformed. Also relying on a simulation, [START_REF] Caldwell | Optimal parameter identification for discrete mechanical systems with application to flexible object manipulation[END_REF] estimated the stiffness parameters of a flexible loop.

Rather than incorporating visual information, proprioception sensing (i.e. joint values and forward kinematics) was used instead to compare with the simulated values of the object. The object was represented as a series of rigid links connected by springs, and the optimal values for the springs' stiffness were found by optimization of an error function given by the difference between measured and simulated values.

An analytic formulation to estimate the shape of a flexible tube was proposed by [START_REF] Bretl | Quasi-static manipulation of a Kirchhoff elastic rod based on a geometric analysis of equilibrium configurations[END_REF]. They assume, however, that the tube cannot be stretched and is held at both ends. The object was described as a Kirchoff elastic rod, where the elastic energy is computed based on three scalar functions that measure strain. When the object is in static equilibrium, its shape can be described by geometrically solving an optimal control problem. With this formulation they were able to describe the possible equilibrium configurations of the object, based on the poses of the end-effectors holding each end, using a single global chart 7 ; and thus, estimate the configuration of the object. This theoretical framework was later evaluated in simulation and experimentally on a two-dimensional case, that is, the deformation was restricted only to remain on a plane [START_REF] Borum | State estimation and tracking of deforming planar elastic rods[END_REF].

Planar objects

In this section we review approaches used on planar objects, such as printing paper, cards, tennis balls to name a few. Most of the research, especially at the beginning of this sub-field, has focused on these type of objects as a simplified version of three-dimensional objects, e.g. in a simulated environment.

The two most studied manipulation tasks on a planar object are grasping and controlling its deformation. The former consists of positioning the fingers of a robotic manipulator on the object such that the manipulator is able to lift it and hold it in the air, while the latter consists in changing the shape of the object into a desired configuration.

Other tasks have concentrated on more specific tasks such as paper folding, manipulating the pose of the object by relying on its deformation (e.g. rotating a pizza dough-like object and reorienting a bill) and sensing the state of a deformed object. Table 2 summarizes the approaches reviewed in this section.

Manipulation

The works on manipulation of planar objects can be classified into three groups, namely, paper folding, grasping and rotating an object. Each group raises distinct challenges due to the different types of objects they manipulate (e.g. a sheet of paper vs a thin sponge) or the specific task they focus on. Paper folding requires the robot to perform precise motions to ensure the paper is being held firmly and bent in an appropriate manner. To grasp and pick up an object, the robot must have the ability to decide where to grasp the object and how tight it should hold it. This would require different grasp forces depending on the material type, whose properties might not be known in advance.

Finally, approaches that focused on rotating objects have either exploited the environment or made use of the object's own dynamics. In the following, a review of works that address this type of manipulation are presented. Shape control Single point using a custom made robotic system that is able to perform a sequence of folds resulting in an origami pattern (e.g. a paper plane). The system consisted on a four degrees of freedom arm, a suction pump to hold and move the paper, and a base with a clamp that holds the paper while the arm folds it. As no sensors were used by the system, the arm motions were

performed in an open loop as selected by a simple breadfirst search algorithm. Since this approach relied on applying a sequence of predefined folds to the paper, no dexterous manipulation of the paper was performed.

On the contrary, [START_REF] Elbrechter | Bi-manual robotic paper manipulation based on real-time marker tracking and physical modelling[END_REF] presented an approach to first grasp a paper resting on a table with a bimanual robot equipped with a five-fingered hand, and they later extended their approach to fold the paper [START_REF] Elbrechter | Folding paper with anthropomorphic robot hands using real-time physics-based modeling[END_REF]). Both approaches rely on a vision system with 5 calibrated cameras that tracked the position of a set of fiducial markers on both sides of the paper. Furthermore, they simulated the paper's deformation with a physicsengine, where the paper is modeled as a 2D grid of nodes.

The extension of the latter approach consisted on adding a connecting link between all neighboring nodes. Each link in the physics-engine was represented by a distance and a stiffness coefficient. The stiffness coefficient is decreased if a crease line, detected by the vision system, passes through any given link. Thus, the links which cross a crease present a higher deformation and can render the folding of the paper.

A similar approach, also using a bi-manual robot system, based on integrating visual information into a physics-engine simulation was presented in [START_REF] Namiki | Robotic origami folding with dynamic motion primitives[END_REF].

However, the paper was modeled with a mass-spring-damper model. Furthermore a set of dynamic primitives was used to apply folds to the paper. Unlike the approaches proposed in [START_REF] Elbrechter | Bi-manual robotic paper manipulation based on real-time marker tracking and physical modelling[END_REF][START_REF] Elbrechter | Folding paper with anthropomorphic robot hands using real-time physics-based modeling[END_REF], the paper is not covered with fiducial markers. Instead the corners are marked with different colors to allow the vision system to detect them.

Grasp and pick up

We consider a grasp as the action of holding an object through contact points, while a pick up is a subsequent action that lifts the grasped object from a supporting surface. One of the first works on grasping flat deformable objects was done by Gopalakrishnan and Goldberg, where they present an approach that extended the concept of form closure for rigid objects to deformable planar objects [START_REF] Gopalakrishnan | D-space and deform closure: a framework for holding deformable parts[END_REF][START_REF] Gopalakrishnan | D-space and Deform Closure Grasps of Deformable Parts[END_REF]. They assumed that a set of contacts, e.g. the grippers of a robot, holding a rigid object in form closure would also hold a deformable object in deform closure. Deform closure was defined as a set of contacts holding an object such that an increment of potential energy is required to release the object. Here, the potential energy is proportional to the amount that the gripper deforms the object. In order to determine the potential energy, an FEM model of the object and its stiffness matrix were used to compute the internal forces based on the object's deformation.

Also relying on an energy-based formulation and using an FEM model Guo and Jia, in a series of papers, focused on grasping planar deformable objects using only two fingers.

Their formulation of potential energy, which later they relate to work, is based on the distance traveled by the fingers times the force they apply to the object. The translation of the fingers is described as squeeze grasping, where one finger moves towards the other while both maintain contact with the object. In [START_REF] Jia | On two-finger grasping of deformable planar objects[END_REF], they applied this approach only to hollow objects (e.g. ring shaped objects), thus considerably reducing the computation complexity as the FEM was only simulated on segments describing the object's boundary rather than on elements describing a solid object. This was extended to solid planar objects in [START_REF] Guo | Squeeze grasping of deformable planar objects with segment contacts and stick/slip transitions[END_REF] where they also defined the finger contacts as a set of points that either stick or slip, in which each of the points coincides Prepared using sagej.cls with a node of the object's mesh. In order to determine whether a contact is sticking or slipping, they check if its friction cone rotates after a change in the squeeze depth 8 (see Figure 6). Based on the state of each contact point (i.e. either sticking or slipping) they were able to infer, for instance, if the object was slipping out of the grasp. Based on the nodes that were in contact, as well as their contact type (stick/slip), they computed a reduced stiffness matrix which significantly reduced the computational burden.

Later on, they introduced grasp metrics that can be applied to deformable objects, namely they defined the concept of pure and stable squeezes [START_REF] Jia | Optimal two-finger squeezing of deformable objects[END_REF]. A pure squeeze is defined as a grasp that generates only deformation (i.e.

no translation or rotation are generated while squeezing); a stable squeeze is a squeeze along a vector such that minimizes the potential energy on the object (i.e. for all the grasps with the same squeeze depth, the stable squeeze is the grasp that generates least deformation). Additionally, they proposed an optimal method to resist a third finger that tries to push the object out of the grasp by moving the two grasping fingers such that the object remains grasped and the work they apply is the minimum required. Figure 6 shows an example of a squeeze grasp. It also shows a peculiarity of deformable objects, namely, the contact point grows into a contact area as the grasp progresses. Finally, Jia et al.

revised the works described in [START_REF] Jia | On two-finger grasping of deformable planar objects[END_REF][START_REF] Guo | Squeeze grasping of deformable planar objects with segment contacts and stick/slip transitions[END_REF][START_REF] Jia | Optimal two-finger squeezing of deformable objects[END_REF] and presented an extensive and selfcontained manuscript detailing their approach to grasp a planar deformable object using only two fingers [START_REF] Jia | Grasping deformable planar objects: Squeeze, stick/slip analysis, and energy-based optimalities[END_REF]). However, their work does not considers dynamics, nor gravity, while computing the FEM simulation. Furthermore, because the analysis is based on the assumption of linear elasticity, large deformations cannot be accurately described. presented an approach that is able to rotate a deformable planar object, resting on a plate attached to a robot's end-effector, using dynamic motions consisting in rotating around and translating along a single axis of the plate as shown in Figure 7. They represented the object as a mass-spring-damper model able to describe bending and compression. The elasticity parameters were estimated by experimentation. Although their approach exploits the deformation of the object in order to rotate it on the plate, it is not able to control the deformation, e.g. to reach a desired configuration of the object. Similarly, [START_REF] Kristek | Orienting deformable polygonal parts without sensors[END_REF] also proposed a method to rotate planar deformable objects using two grippers, however the applied gripper motions consisted of a set of actions that are executed in an open-loop manner as they do not use any sensor. They evaluated their approach using planar objects such as a dollar bill, a card and various shapes of a foam material. This approach used two grippers that exploit the environment to facilitate the task of rotating the object. The first gripper controlled only one degree of freedom that slides a board so as to sweep the object against a wall in order to bend it enough thus, allowing the second gripper to grasp the object and subsequently rotate it.

Deformation control

This section focuses on approaches that address the problem of controlling the deformation of planar objects in order to achieve a desired shape. These approaches define this deformation control problem with the objective to move a target point, or a set of target points, located on the object such that once the target point reaches a specified position the overall shape of the object approximates the desired deformed shape. Thus, they indirectly control the deformation of the object through a target point, or a set of target points.

This type of problems are sometimes referred to as indirect simultaneous positioning (ISP) problems, a term proposed by [START_REF] Wada | Indirect simultaneous positioning operations of extensionally deformable objects[END_REF]. They were one of the first researchers to address the problem of not only grasping a deformable object, but also actively controlling its deformation [START_REF] Wada | Indirect simultaneous positioning operations of extensionally deformable objects[END_REF][START_REF] Hirai | Robust grasping manipulation of deformable objects[END_REF]). Here, the object was modeled as a mesh where the nodes are connected with springs and two sets of points (coincident with the nodes of the mesh) are defined, namely, controlled and manipulated points. The controlled points were located inside the object and were tracked by a vision system to represent the deformation of the object. The manipulated points were the contact points between the fingers of a manipulator and the object. Thus, they were able to indirectly control the deformation of the object by moving the manipulated points such that the controlled points reach a desired configuration. Recently, there have been several works attempting to solve the ISP problem. These approaches control the position of several points (multi-point control) as first proposed in [START_REF] Wada | Indirect simultaneous positioning operations of extensionally deformable objects[END_REF]), or they focus on controlling one internal point on the object (single-point control). Das andSarkar (2010, 2013) used three robotic fingers to deform an object such that a single internal point on the object reaches a desired position.

Single-point control

They solved this problem using a control system that consists of a trajectory generator, a controller for each finger and the object. The finger controller is a PI controller that computes forces based on a positional error and the trajectory generator computes the desired velocities of the fingers to reach the desired position of the target point, both of which are user defined. In order to guarantee stability, a passivity observer monitored the energy applied by each finger controller to the object and two passivity controllers were placed at the input (trajectory generator side) and the output of the finger controller to dissipate any excess of energy. They only presented simulation results were the object's deformation was modeled using a Mass-Spring-Damper system with known coefficients. [START_REF] Fanson | Model based deformable object manipulation using linear robust output regulation[END_REF] proposed a method, based on the concepts of manipulated and controlled points as described by [START_REF] Wada | Indirect simultaneous positioning operations of extensionally deformable objects[END_REF], that linearizes the deformation model. The model was linearized around the origin, i.e. the object's undeformed state, by using one state variable that describes both the position of the points along with their velocities. Having this formulation they designed an output regulator that drives the control points to a desired configuration. In order to compute the position of the control points based on the force applied at the manipulation points, the object was modeled as a mesh-free particle system using the reproducing kernel particle method (RKPM) [START_REF] Chen | Reproducing Kernel Particle Methods for large deformation analysis of non-linear structures[END_REF]. Although their approach was formulated to manipulate and control several points simultaneously, in their evaluation they used only a single control point and a single manipulation point.

Multi-point control

Afterwards, [START_REF] Kinio | A comparative study of H∞ and PID control for indirect deformable object manipulation[END_REF] used an H ∞ controller instead and modeled the object using an FEM based model.

In their evaluations they used two control points and two manipulation points.

In [START_REF] Das | Autonomous Shape Control of a Deformable Object by Multiple Manipulators[END_REF], a Mass-Spring-Damper model was used to simulate a deformable object where the links between nodes are modeled as a Kelvin-Voigt model in series with a damper. This model was used to deform a planar object such that its shape, as described by a curve, can be changed into another desired shape. This approach, however, requires a significant amount of manipulation points 9 to achieve acceptable results.

Instead of relying on a model, [START_REF] Berenson | Manipulation of deformable objects without modeling and simulating deformation[END_REF] proposed a model-free approach to manipulate an object, grasped by a set of grippers, into a desired configuration. The object was described as a set of points. Additionally, it is assumed that the points of the object that are grasped by a gripper do not move with respect to that gripper. Thus, the position of the points farther away from a gripper are affected less by the gripper's motion. This assumption, termed diminishing rigidity, was then used to compute an approximate Jacobian that relates the motion of the points with respect to the grippers' motions.

This approach was recently extended in [START_REF] Mcconachie | Bandit-Based Model Selection for Deformable Object Manipulation[END_REF]) by choosing among a set of models for this Jacobian using a multi-arm bandit formulation. Under this formulation, the "arms" were defined as models based on their diminishing rigidity and an adaptive Jacobian (as proposed by Navarro-Alarcón et al., see Subsection 6.3) with various sets of parameter. Although they presented simulation results on a rope and a cloth-like object this approach is better classified based on the task it performed, namely multi-point control, since the task considered by these approaches focused on reducing the overall distance between a set of target points T and a set of points P defined on the object.

Deformation sensing

These approaches focus on either determining the object's physical parameters during a deformation or estimating the state of the deformed object. In the former case, the Prepared using sagej.cls parameters are required as input for the object's deformation to be simulated; while the latter might make use of these parameters, they can also rely solely on the output of devices like cameras or force-torque sensors. An example of an approach to identify deformation parameters was proposed by [START_REF] Boonvisut | Estimation of Soft Tissue Mechanical Parameters From Robotic Manipulation Data[END_REF], where the elasticity parameters of a soft tissue phantom are estimated. In order to estimate the elasticity parameters, they minimized an objective function that reduce the difference between an observed deformation and an FEM simulation. The observed deformation was obtained by a stereo-vision system that tracked a set of markers on the object, while the inputs to the FEM simulation consisted of force and positions measurements obtained while manipulating the object.

Instead of only identifying parameters, other approaches used these parameters in physical models to track deformations of an object. One such example was reported in [START_REF] Tian | Modeling Deformations of General Parametric Shells Grasped by a Robot Hand[END_REF], where the deformation of thin shell objects, such as a tennis ball, is tracked by an FEM simulation that is based on shell theory [START_REF] Timoshenko | Theory of plates and shells[END_REF]. [START_REF] Schulman | Tracking deformable objects with point clouds[END_REF] instead used a probabilistic model in combination with a physics engine simulator to account for parts of an object that might not be visible to a depth range sensor. The output of the depth range sensor consists of a set of observable points which were used to infer the set of points that conform to the physical model of the object in simulation. Although the proposed algorithm was able to track objects in real time, between 20 and 50 Hz; there is no chance to recover if the algorithm diverges.

Another application on estimating the state of a deformable object was presented in [START_REF] Boonvisut | Identification and active exploration of deformable object boundary constraints through robotic manipulation[END_REF], where the constrained boundaries of an object were predicted by manipulating the object with a robot gripper. The object's boundary was separated into three sections; namely, a fixed one, a second one that is free to move and a third one that is manipulated by the gripper. Stereo vision, together with an FEM model, were used to track a set of points (nodes) on the object. The nodes were then grouped into patches to reduce dimensionality.

With the object represented as a collection of patches, a hill climbing algorithm was applied similarly to an occupancy grid problem in order to estimate which patches are the fixed ones.

Cloth-like objects

In this section we review clothes and fabric materials such as towels, sheets, rags, etc. Most of the approaches that deal with cloth-like objects have focused on solving one or more tasks from the pipeline described in Figure 8. The first step requires the robot to locate the clothing item, possibly identify its configuration, and decide where to grasp (e.g. a shirt's collar). Next, the robot needs to grasp and pick up the item and, depending on the desired task, perform a preparatory action such as extending, or untangling, the garment or laying it flat on a surface. Finally, the end-task might consist on folding or hanging the item in order to store it or putting the item on a person (i.e. clothing assistance).

Moreover, these approaches can be grouped into three major types of tasks, namely, sensing, grasp/manipulate and cloth-specific tasks. The sensing tasks are mainly concerned with recognizing the state (e.g. the shape) of the garment, or detecting ideal grasp points for further manipulation. The grasp/manipulate tasks handle issues such as picking up clothes and bringing them into a desired configuration (e.g. lay them flat on a table). Finally, cloth-specific tasks refer to tasks such as folding and hanging clothes items as well as aiding disable people with their dressing. A classification of the reviewed approaches is summarized in Table 3. Table 3. Classification of approaches to manipulate cloth-like objects using a robot. Another application for estimating the state of garment can be found in [START_REF] Bergstrom | On-line learning of temporal state models for flexible objects[END_REF], where the focus is on estimating the folding of an article (e.g. a T-shirt and a piece of cloth). In order to determine the folding state of the article, they estimated its contour using a Hidden Markov Model (HMM) based on a single feature. This feature is extracted from a sequence of images and consists of an histogram of distance between each point on the object's contour to the rest of the contour points which is referred to as Shape context. This representation is much more compact and thus it allows for a fast estimation of the article's shape.

Sensing

Detection/ estimation

Although a lot of local information is lost with this sparse representation, the use of temporal information proved to be more informative for correctly estimating whether or not an article was fold correctly.

Besides the configuration, other interesting characteristics to understand the behavior of a garment are its material and fabric pattern. For instance, a laundry robot would decide either to use warm water for jeans or cool water for woolen items. This problem was tackled by Kampouris et al. ( 2016) using a multi-modal approach that combined RGB-D, photometric and tactile data. They applied a variety of machine learning algorithms such as Support Vector Machines, random forests and HMMs; and then fused the results using a majority voting. Although their results

showed that the combination of modalities was the most accurate for the majority of the test cases, there were particular cases where either using only photometric or tactile data was clearly more accurate; showing the need for a better fusion strategy.

Grasp point detection

The previous approaches rely on holding the garment in the air in order to estimate its configuration. However, before the robot holds the garment, it is necessary to grasp the garment from a possible crumpled configuration (a more likely state for clothes that have been taking out of the drier or dropped on a table). Thus, in order to grasp a crumpled garment a robot must first decide where to grasp.

Prepared using sagej.cls [START_REF] Ramisa | Using depth and appearance features for informed robot grasping of highly wrinkled clothes[END_REF] tried to solve the issue of grasping polo shirts by detecting their collar. They used a Bag of Features (BoF) that is generated with SIFT features from a 2D image and a histogram based on depth data. They achieved decent results where only one polo shirt was present, although when other types of garments (such as pants and t-shirts) were in the field of view of the camera, the accuracy rate reduced drastically (from ∼70% to ∼30%).

Also focusing on selecting grasp points, [START_REF] Yamazaki | Grasping point selection on an item of crumpled clothing based on relational shape description[END_REF] proposed an approach based solely on depth data to pick

Grasping and manipulation

Depending on the manipulation task and the particular state of a clothing article, the robot is required to handle the article in a particular fashion. For instance, if the article needs to be flatten or stretched in order to remove wrinkles, multiple regrasps might be required such that the article is held by opposite corners. The following works focused on manipulation tasks that consider different starting configurations for the articles (e.g. crumpled or lying flat).

Grasping for garment picking-up A geometric

approach to picking up a cloth is described in [START_REF] Shibata | Wiping motion for deformable object handling[END_REF], where a robot generates a wrinkle by deforming the cloth so that the robot can grasp the cloth. They defined the concept of wiping deformation, which consists in moving the jaws of the robot's gripper towards each other while maintaining contact with the cloth. In this way, the cloth moves with respect to the surface but there is no relative motion between the cloth and the gripper at the contact points. However, this approach assumes the cloth is lying flat on a surface, a scenario that is rather uncommon since cloth-like objects tend to easily crumple.

Not relying on the assumption of the object lying flat, [START_REF] Monsó | POMDP approach to robotized clothes separation[END_REF] where the gripper is grasping the garment. The robot then repeatedly regrasps the garment using the planning algorithm, which relies on the garment state output by the HMM, until the cloth is brought into a known configuration.

Once the configuration of the piece of cloth is known,

although not the desired one, the robot regrasps the object to bring it to the desired configuration (e.g. holding a shirt by its shoulders). The magnitude of this velocity is based on the amount of normals that are within a user defined threshold, e.g. the normals should point upwards to keep the cloth stretched, and the direction is chosen as the mode of the normals that are not within the threshold. The force information, obtained through the estimation of joint torques, is used by a force controller with a damping coefficient to generate a velocity that maintains a desired tension for the cloth. To obtain the velocity of the hybrid controller, both of these velocities are combined by a simple addition.

Cloth-specific tasks

As previously mentioned, works on cloth-specific tasks have focused on folding, hanging clothes or assisting in dressing people. In order to hang clothes, or put them on people, the robot is required to identify and track openings on the clothes so they can be put through a hanger or someone's legs, arms or head; whereas folding might not require any feedback as planned actions can be executed with open-loop motions. Another approach that focuses on detecting grasp points to fold a piece of cloth is found in [START_REF] Bersch | Bimanual robotic cloth manipulation for laundry folding[END_REF]. Two different type of g-folds were proposed, where one folds the whole stack 10 and the other folds only the previously folded part by not releasing the grasped points (e.g. after folding a long-sleeve by pivoting on the shoulder, the sleeve is then folded such that the end of the sleeve is aligned with the bottom of the article). Also relying on a polygonal description of a clothing article, the work presented in [START_REF] Stria | Garment perception and its folding using a dual-arm robot[END_REF]) was able to fold clothes much faster, requiring seconds instead of minutes (as in [START_REF] Miller | A geometric approach to robotic laundry folding[END_REF]). This major speed-up is achieved during the perception phase and is due to the novel model they propose. This polygonal model uses the vertices on the clothing article to define angles at the vertices and also the lengths between vertices.

Folding

Unlike in (Maitin

Thus, having a compact description of the clothing article as a set of angles and lengths, which they encode with probability distributions for a collection of articles of the same class (e.g. pants would be a class and a short-sleeved shirt another). Both of these approaches require however that only a single garment is resting completely spread on a supporting surface. Recently, Li et al. (2015b) proposed an approach for a bi-manual robot to fold different garments such as shirts, pants and towels. Their approach first detects key points for the robot to grasp (e.g collar, sleeve ends) and then finds the optimal folding trajectory of the robot's arms.

The trajectories are parameterized as Bezier curves, and their points are found by minimizing the cost between the desired fold and a simulated fold which is computed using a physics engine.

Machine learning approaches have also been utilized to solve the task of folding a garment using a robot. For instance, [START_REF] Balaguer | Combining imitation and reinforcement learning to fold deformable planar objects[END_REF] applied reinforcement learning (RL) to fold a towel using a two-armed robot. In order to reduce the search space, demonstrations were taught to the robot by a human teacher. The RL problem was formulated as finding a map between the trajectory of both end-effectors and a set of points describing the object. This set of points was obtained through eight cameras tracking the position of 28 markers attached to the object's boundary.

Other approaches that rely on learning techniques to fold towels can be found in [START_REF] Lee | Unifying scene registration and trajectory optimization for learning from demonstrations with application to manipulation of deformable objects[END_REF][START_REF] Huang | Leveraging appearance priors in non-rigid registration, with application to manipulation of deformable objects[END_REF] as reviewed in Section 3. person, is an important issue to consider when using robots to assist people with their dressing. [START_REF] Erickson | What does the person feel? Learning to infer applied forces during robot-assisted dressing[END_REF] recently addressed this problem by using a recurrent neural network (RNN), a type of network architecture that is ideal for sequential data, to map variables at the end-effector of a robot to a force distribution on a person's simulated limb.

The input consists of nine variables describing the force, torque and velocity measurements of the end-effector that is grasping the garment being pulled into a simulated limb; the output is a force map with hundreds of predefined points that have being uniformly distributed along the simulated limb.

This approach was able to obtain visually similar results

to the ground truth, and the authors proposed to extend it to real robots in the future to cope with previously unseen scenarios (e.g. higher forces and velocities not available in the simulated data).

As mentioned previously, hanging garments is another task that requires the manipulation of clothes. An application of this task is showcased by [START_REF] Twardon | Interaction skills for a coat-check robot: identifying and handling the boundary components of clothes[END_REF],

where two robotic hands work in combination to hang a knit cap on a coat rack. The approach proposed by Twardon and Ritter begins by generating a graph representation of the cap from depth data. In order to generate the graph, it first processed a point cloud to detect edges based on the points' normals (e.g. by removing smooth curves). Once the edges were found, a thinning algorithm was applied to reveal the intersections of the edges. These intersections were selected as nodes if they had at least three neighbors. Finally, it used this graph to compute simple cycles to subsequently extract boundaries that represent, in this case, the opening of the cap.

Solid objects

Solid objects such as sponges, plush toys and food, remain the least researched type of deformable objects. Perhaps the main reason for this are the computational costs imposed when simulating three-dimensional objects which are much higher than those of simulating planar, or even clothlike, objects. However, recent advances in computing have increased the processing power significantly such that real time simulation of realistic deformation (e.g. using FEM) of solid objects is now possible. This progress has led to an increment of research of solid deformable objects in the robotics community.

Another kind of solid deformable object is soft tissue. It has been extensively studied for medical robotic applications. Since medical applications are subject to constraints, not shared by domestic and industrial applications, such as dealing with moving organs in real time and limited space when performing minimally invasive surgeries; we cover only those works where the proposed methodologies could be applied to task in industrial (e.g. cutting meat) or domestic settings. For a review on medical robots and their applications to surgery, the reader is referred to the survey by [START_REF] Taylor | Medical robotics and computer-integrated surgery[END_REF]. The manipulation tasks that have been researched for solid objects can be grouped into three main categories: grasp/manipulate, deformation sensing and deformation control.

Grasping and manipulation

Similar to deformable planar objects, grasping and picking up a solid object was one of the first tasks studied in this sub-field and it remains an active research topic. Other manipulation tasks have been investigated for solid objects as well; for instance, cutting meat, splitting food items and handling soft tissue to assist in surgery. Depending on the task, the robot might be required to manipulate the object Prepared using sagej.cls (a) Grasp and lift [START_REF] Lin | Picking up a soft 3D object by "feeling" the grip[END_REF].

(b) Cutting with a knife [START_REF] Gemici | Learning haptic representation for manipulating deformable food objects[END_REF]. between the distances of the opposing fingers and the measured forces from the tactile sensors, which they termed deformability ratio. This deformability ratio is then used to compute the maximum force allowed to be exerted on the object, thus reducing the deformation of the object. This approach was later extended in [START_REF] Delgado | Control of Robot Fingers with Adaptable Tactile Servoing to Manipulate Deformable Objects[END_REF]) by coupling it with a grasp planner to perform tactile servoing, which consists in moving each finger until a desired force and a desired contact location on each sensor are reached.

In contrast to these approaches, other approaches require a physically accurate model 13 in order to grasp a deformable object. For instance, [START_REF] Lin | Picking up soft 3D objects with two fingers[END_REF][START_REF] Lin | Picking up a soft 3D object by "feeling" the grip[END_REF] proposed an approach where a deformable object is grasped and lifted by a robotic gripper that relies only on a mesh model of the object and the positions of each finger, thus no exteroceptive sensors (such as tactile or force sensors) are required. To do so, they used an FEM formulation to solve the deformation simulation based on the displacements caused by the fingers.

Each finger contact is modeled as a set of points that are either sliding or sticking. The approach consists in moving the fingers towards each other in order to squeeze the object and subsequently lift it, after a lift test is passed. This Other research works have focused on more complex manipulation tasks such as cutting deformable objects.

In [START_REF] Hu | Intelligent automation of bird deboning[END_REF], an approach to debone a chicken using a cutting robot is presented, where cutting through ligamets, tendons and skin is required. The cutting robot had only two degrees of freedom and followed a path plan that was derived based on key points, obtained from a stereo-vision system, of the chicken such that they form a cutting plane. As cutting through the bone is undesired due to health hazard, the robot was also equipped with a force-torque sensor to detect contact with the bone. In case of contact, the robot switched to force control and modified, in real time, the initial path to maintain the force within a specified threshold. Other efforts by the same group have focused on cutting other food products, such as potatoes (Zhou et al. 2006a,b). By using a cutting robot arm with seven degrees of freedom, Long et al. (2014b) were able to follow a curve (as opposed to a straight line as presented in [START_REF] Hu | Intelligent automation of bird deboning[END_REF], to separate a beef muscle. Furthermore, they made use of a second robot that was rigidly attached to the meat, with a special gripper, to pull the meat in order to reduce the friction for the cutting robot. The pulling robot used impedance control to maintain a desired pulling force; while the cutting robot used position control to follow the cutting path with a visual feedback that updated changes caused by the meat deformation. However, the cutting robot was not able to follow the path completely due to its inability to overcome the required friction to cut the meat. This was later addressed in (Long et al. 2014a) by adding a force-torque sensor to the cutting robot. The force control applied a slicing motion that reduced the required cutting force. Note that the meat's deformation was not explicitly modeled in these works.

Gemici and Saxena (2014) used a PR2 robot to prepare a salad, in which the robot actions required cutting and splitting food objects such as bread, cheese and lettuce. The approach used haptic data, such as force and tactile data, to learn the objects' properties (e.g. brittleness and elasticity) by performing manipulation actions that stretch, bend and cut the objects. The properties were assigned a value from zero to one by human experts, which then the robot learned using a regression model. The different learned properties were used to classify the objects and serve to decide a proper action for a given task. For instance, the task of cutting a cheese depends on its properties. Since cream cheese is much brittler than Cheddar cheese, the robot is required to hold the cream cheese with a fork, instead of using its gripper so as to not damage the cheese.

Although the works presented in the following paragraphs do not directly manipulate deformable objects, as they are concerned with automated surgical suturing, they highlight intrinsic problems arising when interacting with soft tissue.

For instance, inserting a needle into soft tissue causes deformation and thus, it might be necessary to either estimate the contact forces or track the deformations in order to adapt the suturing trajectory to reduce damage to the tissue.

Since suturing is one of the most common surgical tasks, automating it would result in considerable reduction times as well as in a consistency increment. Robotically assisted surgeries (RAS) have not been, so far, truly autonomous as a surgeon is still required during the procedure. The following works have proposed approaches that attempt to automate the suturing process. [START_REF] Iyer | A single arm, single camera system for automated suturing[END_REF] required only a set of entry and exit points defined by a surgeon. They used a six degrees of freedom manipulator with a suturing tool attached to its end-effector to give an extra degree of freedom of rotation.

This tool held a circular needle to go through the set of points to suture the incision. Using a monocular camera, they estimated the location of the needle by fitting an ellipse. Once the needle was located, they computed an ideal center point for the needle to go in and out of the entry and exit points, respectively. Their results were fairly consistent, although the approach was highly sensitive to poor illumination conditions.

A further step towards full automation was achieved by Leonard et al. (2014b), as they proposed an approach capable of automatically determine the suturing points given an incision contour. They used the Smart Tissue Anastomosis Robot (SMART), which is a robotic system developed for suturing equipped with a seven degrees of freedom arm, a monocular camera and a custom made tool that rotates a circular needle to pass a thread through soft tissue. Based on the incision contour, the stitches locations required to perform the suture where computed such that they would be evenly distributed. Once the locations where the needle was required to pass were known, a tracking algorithm (based on Kanade-Lucas-Tomasi) was used to deal with deformations of the soft tissue caused by the needle. The features used for the tracking algorithm were based on the distance to the closest pixel in the target contour, the color of the foreground and a bounding box of the incision. To perform the suturing, the robot first tied a knot on one edge of the incision and Prepared using sagej.cls then continued to the next stitch location. Additionally, force sensing was used to tighten the knot without exceeding a threshold that was experimentally determined to avoid damaging the tissue. Their results were four times faster than surgeons using the same custom made tool and they also achieved higher accuracy. They later extended their approach in (Leonard et al. 2014a) by adding a multi-spectral vision system to detect near-infrared fluorescent (NIRF)

markers. The addition of NIRF markers allowed their system to deal with occlusions, since they can be detected through tissue and blood. However, as their approaches relied on a homography, they were limited to dealing with only flat surfaces on the phantom.

Focusing on a different aspect of automated suturing, [START_REF] Jackson | Needle-tissue interaction force state estimation for robotic surgical suturing[END_REF] presented an approach to estimate the deformation forces of the tissue caused by the needle.

Since tissues can be damaged by excessive force, their approach aims to minimize the internal forces produced by the needling while suturing. Instead of using a vision system, their approach relied solely on position and force sensing, in combination with an Unscented Kalman Filter (UKF), to estimate the forces caused by the needle. Although the UKF produce accurate results, it was not run online in their experimental validation.

Deformation sensing

The approaches presented in this section address either parameter identification, to later use those parameters in a deformation model; or shape estimation, where sensors are used to directly measure the deformation (or shape) of the object without the need of identifying parameters, as they are known in advance or not required at all. While the former approaches require to actively deform the object in order to obtain sensor measurements (e.g. through force/tactile or vision sensors) to infer certain deformation parameters; the latter approaches mostly focus on detecting changes on the object's shape as it is being deformed by another agent (e.g. a surgeon manipulating soft-tissue) and using those changes to provide a 3D visualization of the deformed object. object in order to describe its deformation. In this approach, the object was divided into sections that have a specific curvature produced by the deformation. These curvatures, along with the object's pose obtained through RGB-D data, described the deformation state of the object. As well as the previously mentioned approaches, this approach also minimized a function that considers both the sensed error and the simulated error.

Other types of models that do not require a mesh have also being explored in the field of robotics. For instance, Güler et al. ( 2015) used a position-based 14 physics simulation called Meshless Shape Matching (MSM). This simulation only requires position information, which is obtained using an optical flow algorithm on a sequence of images. Since the images are obtained from a single view, only one face of the object was considered for estimating the deformation.

Instead of relying on the usual elasticity parameters (e.g.

Young's modulus and Poisson's ratio), their approach is based on the computation of a deformability parameter, used in the simulation, that minimizes the error between the sensed positions and the simulated positions.

Shape estimation/tracking

One of the first works on deformation estimation used a stereo vision system to track the surface of an object being deformed by a robotic gripper (Khalil et al. 2010). It integrated the estimation in GraspIt! [START_REF] Miller | GraspIt![END_REF] to have a real time representation of the deformation. Cretu et al. (2010a) were able to track the contour of an object using a growing neural gas network while the object was deformed by a robotic hand. To track the contour, they first identified the foreground from the background using the HSV value of each pixel and later applied a Sobel filter to detect the object' contour.

Lastly, the neural gas network was used to determine the minimum amount of points that described the contour. Later on, in (Cretu et al. 2010b), force measurements obtained with strain gauges at the fingertips of the robotic hand, along with their positions, were used to learn the elastic behavior with a neural network that mapped the deformed contour, obtained through visual data, to the force and position measurements. Thus, the neural network input consisted of a six-dimensional vector containing the force and position information and its output consisted of the two-dimensional position of a set of points representing the object's contour.

A further improvement was then made in [START_REF] Cretu | Soft object deformation monitoring and learning for model-based robotic hand manipulation[END_REF], by also tracking lines that formed a grid on the object. These lines were however artificially marked on the object.

Similarly, an approach to track deformation of objects was presented in [START_REF] Leizea | Real-time deformation, registration and tracking of solids based on physical simulation[END_REF], with a focus on augmented reality applications. The approach consisted as well in

Deformation control

The methods presented in this section extend the technique firstly proposed by [START_REF] Wada | Indirect simultaneous positioning operations of extensionally deformable objects[END_REF], as reviewed in subsection 4.2, to three-dimensional objects. These approaches have been applied to two different types of objects, namely elastic objects and elasto-plastic objects.

Elastic objects return to their original shape once the external force is removed, and elasto-plastic objects are objects that present both plastic and elastic deformations (e. g. a loaf of bread). A commonality among these works is the use of a set of target points that are located inside the object which the robot cannot directly manipulate; thus requiring an indirect manipulation of the points by deforming the object's surface.

Elasto-plastic objects

In [START_REF] Higashimori | Active shaping of an unknown rheological object based on deformation decomposition into elasticity and plasticity[END_REF] a spring-mass-damper model was used to predict the deformation of a clay-like object and modify its shape to a desired configuration. The approach consisted of two phases, namely, a sensing phase and a shaping phase. In the first phase, the object was pushed by an robotic actuator equipped with a force sensor to estimate the elasticity parameters of the object. Afterwards, the force required to reach the desired shape was computed using the obtained parameters and the integrated force applied during the sensing phase. The force was then applied in the shaping phase with a simple force feedback controller. In this approach, the deformation was only controlled in one dimension, i.e. it was deformed by a desired distance. [START_REF] Yoshimoto | Active outline shaping of a rheological object based on plastic deformation distribution[END_REF] extended this approach to deform the object in two dimensions by modeling the object as a connection of nodes in 3D space.

Elastic objects

One of the first works concerning elastic, specifically hyperelastic, objects was reported in [START_REF] Smolen | Deformation planning for robotic soft tissue manipulation[END_REF]. Based on a meshless model of the object they controlled a set of control points on the surface of the object through another set of points termed manipulation points. The control points were arbitrarily assigned to deform the object into a desired shape, while the manipulation points were located on the object's surface, i.e.

where the manipulators could be positioned. Using these two set of points a Jacobian was computed in order to actuate the manipulators such that their motion drove the control points to a required configuration. Figure 12 shows a graphic representation of these deformation features. Furthermore, these approaches are based on estimating a Jacobian matrix representing the relation between the manipulator's motion and the feature vector. This Jacobian is termed deformation Jacobian, and is estimated online using the Broyden method 16 in (Navarro-Alarcon et al. 2013b,a).

In (Navarro-Alarcon et al. 2014), they proposed a new online Jacobian estimator that uses views from multiple cameras. Since these previous approaches only controlled three degrees of freedom of the manipulator's gripper, namely its position; in (Navarro-Alarcon and Liu 2014), they extended their approach to incorporate the control of the gripper's orientation in order to have a 6 DoF controller.

Another limitation of these approaches is that the control is performed on a two-dimensional projection based on the cameras' images, since the control points are described in pixel coordinates. In other words, this approach only controls the deformation features on a plane. This limitation was addressed in [START_REF] Navarro-Alarcon | Automatic 3-D Manipulation of Soft Objects by Robotic Arms With an Adaptive Deformation Model[END_REF], by using stereovision to track the points in 3D and subsequently define the deformation feature vectors also in 3D.

The latest approach by Navarro-Alarcon and Liu (2017)

proposed a different way of describing deformations by using a truncated Fourier series to approximate a contour shape, rather than a set of points. This novel representation is able to describe the contour of the object in a 2D plane and it is used as a feedback signal to control the deformation of the object.

As the method depends on the number of coefficients used to describe the object's 2D shape, a trade-off between speed and accuracy can be adjusted depending on the application (e.g. a large number of harmonics can better describe the shape, but becomes computationally expensive to be used in real time applications). One limitation of the approach, as it does not use a physical model, is its inability to guarantee if a desired shape can be reached in advance. Nevertheless, it would be interesting to research the viability of extending this innovative approach to 3D shapes.

Discussion and future directions

Discussion

In the previous sections we reviewed recent approaches used in robot applications that manipulate deformable objects.

These approaches were categorized according to the type of objects they manipulate, based on the categories described in Section 1, and subsequently depending on the manipulation task they perform upon the objects. The top-level classification grouped the approaches into four categories of objects. Approaches used across these categories differ substantially. Greatly, due to the assumptions employed by the approaches. For instance, cloth-like objects can be represented as a polygonal structure, an assumption that has not been proven effective for solid objects. Another common assumption made by approaches used in the manipulation of deformable linear objects is to describe them as a chain of links, which requires them to be extended to a grid in order to represent planar objects, as shown, for instance, in [START_REF] Elbrechter | Bi-manual robotic paper manipulation based on real-time marker tracking and physical modelling[END_REF][START_REF] Elbrechter | Folding paper with anthropomorphic robot hands using real-time physics-based modeling[END_REF][START_REF] Namiki | Robotic origami folding with dynamic motion primitives[END_REF].

A further classification of the approaches was applied in order to distinguish them based on the manipulation task they focused on since the approach's success, and applicability, highly depends on the particular task at hand. Some tasks require high accuracy estimation of the deformation (e.g. deformation control), while other do not have this constraint (e.g. picking up an object). Based on these differences, three main types of tasks were identified, namely:

(a) Object grasping is unarguably the most common manipulation task. Therefore, a vast amount of approaches have focused on grasping, picking up and holding a object as a first attempt to manipulate deformable objects. This is a trend that has been consistent across all the object types, specially in the earliest approaches. Also, grasping is usually a necessary first step in order to execute other manipulation actions such as lifting the object, folding a garment, or deforming the shape of an object. Another reason for the popularity of deformable object grasping might be related to the fact that rigid body grasping is a very well studied subject in robotics. So far, some approaches have tried to extend grasping techniques developed for rigid objects to deformable objects. For instance, the concept of form closure was extended to deform closure in [START_REF] Gopalakrishnan | D-space and deform closure: a framework for holding deformable parts[END_REF][START_REF] Gopalakrishnan | D-space and Deform Closure Grasps of Deformable Parts[END_REF] to securely hold a deformable object; and force closure, in combination with an FEM simulation, was utilized in [START_REF] Lin | Picking up soft 3D objects with two fingers[END_REF][START_REF] Zaidi | Model-based strategy for grasping 3D deformable objects using a multi-fingered robotic hand[END_REF]). Other approaches have developed new strategies, such as, bulging the object enough for a manipulator to grasp it [START_REF] Elbrechter | Bi-manual robotic paper manipulation based on real-time marker tracking and physical modelling[END_REF][START_REF] Kristek | Orienting deformable polygonal parts without sensors[END_REF] or using machine learning to learn the force require to lift the object [START_REF] Howard | Intelligent learning for deformable object manipulation[END_REF].

(b) Specific manipulation tasks arise inherently due to the kind of object being manipulated. One specific task to deformable linear object is to tie a knot, whereas for a cloth-like object, folding a garment is a particular task. Other specific tasks can be found in more than one category, as is the case of controlling the object deformation. Deformation control is a common task for both planar and solid objects.

(c) Sensing the state of a deformable object is a necessary requirement for most manipulation tasks. For instance, a manipulator deforming an object must be able to sense the object's shape in order to reach a desired shape. Sensing tasks can be classified in either parameter identification or shape tracking. Approaches focusing on parameter identification tasks use vision, and sometimes force data, to estimate either elasticity parameters (e.g.

Young's modulus and Poisson's ration) that are required by deformation models, such as, Mass-Spring models or FEM-based models; or approach-specific parameters, for instance, a degree of deformation required by a Meshless Shape Matching simulation [START_REF] Güler | Estimating the deformability of elastic materials using optical flow and position-based dynamics[END_REF], the twisting strength and bending angles of a beam that are used in a Euler-Bernoulli model by [START_REF] Fugl | Simultaneous Estimation of Material Properties and Pose for Deformable Objects from Depth and Color Images[END_REF],

or the links' stiffness in a physics-based simulation of an object modeled as a series of rigid links [START_REF] Caldwell | Optimal parameter identification for discrete mechanical systems with application to flexible object manipulation[END_REF]. For shape tracking, there are approaches that rely only on vision systems to estimate the contour of the deformed object (Cretu et al. 2010a,b); while others, combine deformation models with vision to estimate the object's shape [START_REF] Leizea | Real-time deformation, registration and tracking of solids based on physical simulation[END_REF][START_REF] Petit | Real-time tracking of 3D elastic objects with an RGB-D sensor[END_REF].

Figure 13 shows the different tasks the reviewed approaches have focused for each deformable object type.

As shown in the figure, a larger number of approaches have been applied to on cloth-like objects. This disparity might be due to the increment of recent projects focusing on solving laundry-related tasks with robots. Most notably, the UC Berkeley group 17 and the CloPeMa project 18 have produced a lot of results to perform many of the tasks required to enable a robot to autonomously do the laundry. The interest on solutions of laundry oriented tasks, within clothlike objects, is also apparent in the amount of approaches that solve a specific task (19) like folding and hanging; in comparison to approaches focusing on manipulation (10) or sensing tasks (14).

Another remark from Figure 13 is the limited numbers of approaches focusing on sensing tasks for planar objects.

A contrast to this, is the amount of approaches that do focus on sensing task for solid objects. This is not a surprising fact, since our survey has mostly concentrated on reviewing recent approaches and as previously noted, sensing tasks tend to precede manipulation tasks. Therefore, the amount of approaches focusing sensing deformation on planar objects (which themselves have preceded approaches on solid objects) is significantly less than those on solid objects, which were not as researched until the last few years.

Future directions

Finally, we discuss open problems within each category and suggest possible directions to be further researched. 7.2.1 Linear objects A particular issue that has not been addressed substantially is that of estimating the state of linear deformable objects when three-dimensional deformations occur. This might be due to the fact that the majority of reviewed approaches either making assumptions on the object's behavior (e.g. assuming a rope's shape follows the trajectory of the gripper grasping it as in [START_REF] Vinh | A new strategy for making a knot with a general-purpose arm[END_REF][START_REF] Yamakawa | Motion planning for dynamic knotting of a flexible rope with a high-speed robot arm[END_REF][START_REF] Takizawa | Method for placing a rope in a target shape and its application to a clove hitch[END_REF][START_REF] Yamakawa | Simple Model and Deformation Control of a Flexible Rope using Constant, High-Speed Motion of a Robot Arm[END_REF])) or simplifying the object's configuration to, for instance, a topological description 19 as described in [START_REF] Yamakawa | Knotting manipulation of a flexible rope by a multifingered hand system based on skill synthesis[END_REF][START_REF] Matsuno | Manipulation of deformable linear objects using knot invariants to classify the object condition based on image sensor information[END_REF]. Nonetheless, there have been approaches attempting to estimate the shape of a linear deformable object. These approaches either use a modeling framework [START_REF] Bretl | Quasi-static manipulation of a Kirchhoff elastic rod based on a geometric analysis of equilibrium configurations[END_REF], sensing [START_REF] Matsuno | Manipulation of deformable linear objects using knot invariants to classify the object condition based on image sensor information[END_REF], or more commonly, a combination of both [START_REF] Javdani | Modeling and perception of deformable one-dimensional objects[END_REF][START_REF] Caldwell | Optimal parameter identification for discrete mechanical systems with application to flexible object manipulation[END_REF][START_REF] Borum | State estimation and tracking of deforming planar elastic rods[END_REF].

However, the current state of the art on the state estimation of deformable objects is limited. On the one hand, modeling approaches require a priori knowledge of the object (e.g. elasticity parameters) and tend to be computational expensive, rendering too slow for real time manipulation tasks. On the other hand, sensing approaches thus far have mainly relied on visual sensors; a sensing modality that is affected greatly by occlusion, which occurs more often when manipulating the object. Other sensors, such as forcetorque sensors, are currently limited by their sensitivity and might not be able to detect a significant measurement on objects that present large strains. Tactile sensing could be an interesting complementary modality to visual sensing due to its ability to extract relevant information on the parts of the objects that are being occluded by the manipulation process, although the resolution of a given tactile sensor restricts the sensing of sufficiently thin linear objects.

An accurate estimation of an object's state could allow more complex manipulation actions such as routing cables (e.g. in a car manufacturing facility) and helping disable people tie their shoe laces. Also, improved sensing capabilities could detect unexpected events to trigger a replanning strategy. This sensing ability will benefit path planning approaches as it would allow them to adapt to dynamic environments; and understanding, either by modeling or sensing, how an object behaves while being manipulated, could be incorporated into planning strategies to generate more robust motions.

Planar objects

Most planar objects that have been studied so far are a simplified case of a solid object, where only one side of the object is considered. For instance, in [START_REF] Guo | Squeeze grasping of deformable planar objects with segment contacts and stick/slip transitions[END_REF]) a sponge-like object is manipulated while considering only its deformation on a two-dimensional plane. This was a necessary simplification at the beginning of the field, since the computational resources were not as powerful as they are today. By considering one dimension less of the object, simulation approaches could be applied that might not have been possible on a solid object as it was demonstrated in [START_REF] Jia | On two-finger grasping of deformable planar objects[END_REF][START_REF] Gopalakrishnan | D-space and Deform Closure Grasps of Deformable Parts[END_REF].

The state-of-the art computing is now, however, capable of simulating deformation of solid objects at an acceptable speed. Thus, the field has focused its attention in other types of objects such as paper. Paper manipulation remains an under-researched field, but there have been notable efforts to improve robots' abilities to handle paper [START_REF] Elbrechter | Bi-manual robotic paper manipulation based on real-time marker tracking and physical modelling[END_REF][START_REF] Elbrechter | Folding paper with anthropomorphic robot hands using real-time physics-based modeling[END_REF][START_REF] Namiki | Robotic origami folding with dynamic motion primitives[END_REF]. Similar objects, e.g. cards and bills, have received even less attention and the current approaches focus on simple manipulation actions such as flipping them or rotating them [START_REF] Kristek | Orienting deformable polygonal parts without sensors[END_REF]. Developing control algorithms that can be coupled with recent vision-based approaches, able to reconstruct deformations of planar surfaces in an online manner (see for instance [START_REF] Famouri | Fast shape-from-template using local features[END_REF])), would provide robots with capabilities to manipulate paper-like objects in real time. This would allow robots, for example, to take over tedious office tasks such as sorting documents, making copies, opening mail and handling bills.

Figure 13 indicates a clear trending on approaches focusing more on manipulating planar objects rather than on sensing their deformation. Most recent approaches have relied on estimating the object's deformation through modeling (either using Mass-Spring or FEM based models)

and apply control theory algorithms to manipulate them. A potential improvement to these techniques would be to add sensing information (e.g. force and vision) in the loop to increase the accuracy of both the deformation estimation and its control.

Cloth-like objects

Most of the focus so far has been on approaches to solve the so-called laundry problem;

that is, picking up a cloth, place it into a washing machine/drier, taking it out of the washing machine/drier, stretching it out, folding it, and putting it away. However, more complicated tasks involving handling of clothes have received less attention. For instance, hanging a cap on a standing hanger was covered in [START_REF] Twardon | Interaction skills for a coat-check robot: identifying and handling the boundary components of clothes[END_REF] and an approach to iron a shirt with a robot was recently proposed by [START_REF] Li | Multi-sensor surface analysis for robotic ironing[END_REF]. Machine learning algorithms, already used for solving folding tasks, such as learning from demonstration [START_REF] Balaguer | Combining imitation and reinforcement learning to fold deformable planar objects[END_REF][START_REF] Huang | Leveraging appearance priors in non-rigid registration, with application to manipulation of deformable objects[END_REF][START_REF] Lee | Learning force-based manipulation of deformable objects from multiple demonstrations[END_REF]) could be applied to some of these complex tasks. Another example is that of aiding a person getting dressed; an initial attempt was presented in [START_REF] Yamazaki | Bottom dressing by a life-sized humanoid robot provided failure detection and recovery functions[END_REF][START_REF] Tamei | Reinforcement learning of clothing assistance with a dual-arm robot[END_REF]. However, helping a person button (and unbutton) a shirt proves to be a much more difficult task for a robot. Another desired capability for a robot is that of drying a disable person using a towel, a task that does not only require the manipulation of the cloth itself, but also the interaction with the person's body which should be performed in a safe manner. Handling safety concerns opens the possibility of incorporating techniques currently being developed in the field of physical humanrobot interaction.

7.2.4 Solid objects This type of deformable objects remain the least researched. A reason for this is the computation complexity required to model a threedimensional object in a realistic manner. However, due to recent computing progress that has allowed complex modeling and more realistic simulations to be performed at a reasonable rate; solid objects are now receiving more attention from the robotics community.

In so far, the research of solid deformable objects has focused mostly on grasping and holding an object [START_REF] Lin | Picking up soft 3D objects with two fingers[END_REF][START_REF] Lin | Picking up a soft 3D object by "feeling" the grip[END_REF][START_REF] Zaidi | Model-based strategy for grasping 3D deformable objects using a multi-fingered robotic hand[END_REF], and on estimating its deformation [START_REF] Leizea | Real-time deformation, registration and tracking of solids based on physical simulation[END_REF][START_REF] Petit | Real-time tracking of 3D elastic objects with an RGB-D sensor[END_REF]) (see Figure 13). Another direction that remains under-researched is that of shape-servoing, that is, to deform an object into a desired shape [START_REF] Smolen | Deformation planning for robotic soft tissue manipulation[END_REF]Navarro-Alarcon et al. 2013b[START_REF] Yamazaki | Grasping point selection on an item of crumpled clothing based on relational shape description[END_REF]. However, the current approaches that have focused on shape-servoing rely on visual sensing; which, similarly to the linear objects, they are widely affected by occlusion and thus, adding force and tactile sensing could address the shortcomings of vision systems.

Unlike vision sensing, force and tactile sensing require the robot to physically interact with the object which in turns require a manipulation strategy to perform different exploratory actions such as tapping, rubbing, shaking or grasping an object. Furthermore, these sensing modalities have demonstrated their ability to extract material properties to improve the object's deformation model.

An additional task with a lot of potential is that of dynamic manipulation of soft objects. For instance, a robot able to manipulate soft tissue in real time could assist a surgeon in a medical procedure; or help out in a domestic setting by handling food items to prepare a meal. As an example, one recent attempt of dynamic food manipulation was described in [START_REF] Satici | A coordinate-free framework for robotic pizza tossing and catching[END_REF], where they proposed a theoretical framework to dynamically manipulate a pizza dough, specifically; to toss and catch the dough with a humanoid robot. However, their results are yet to be validated on a real platform. Improvements on sensing, modeling and actuation are still required to allow robots to handle deformable objects in a human-like manner. Accurate, and real time, sensing would allow the robot to know the state of the object to better decide how to manipulate it to achieve a given task. Dexterous and fast manipulators are also necessary to carry on movements that adapt to an object constantly changing its shape.

Soft robots, unlike traditional robots, are made of materials exhibiting large strains (e.g. low Young Modulus)

and could very well be considered deformable objects.

The field of soft robotics shares several issues with deformable objects. For instance, soft robotics is concerned with estimating the robot's deformation and controlling its configuration by actions that deform the robot. Additionally, methods such as FEM simulation are common, on both fields, to model the object/robot's behavior and, due to their intrinsic safety; soft robots are also ideal for manipulation tasks where safety is paramount in handling fragile objects (e.g. muscles and organs in a surgical operation or humans themselves while providing elderly care). As soft robotics covers a wide diversity of topics such as design, fabrication, modeling and control; we refer the reader to previous surveys [START_REF] Trivedi | Soft robotics: Biological inspiration, state of the art, and future research[END_REF][START_REF] Kim | Soft robotics: A bioinspired evolution in robotics[END_REF][START_REF] Rus | Design, fabrication and control of soft robots[END_REF], where these are discussed in depth.

Conclusion

This survey presented a review of recent approaches that focus on robot applications that manipulate deformable objects in domestic and industrial environments. A Prepared using sagej.cls categorization of the approaches based on the type of objects they manipulate was defined, as well as a further classification that considers the type of task being executed.

We identified four object types: 1) Linear, 2) Planar, 3) Cloth-like and 4) Solid; and three types of tasks, namely, sensing, manipulation and tasks particular to each object type (e.g. folding a shirt). Using this classification, we arranged the reviewed approaches in tables to offer a concise overview of the proposed categories.

We also discussed current limitations based on this

Notes

1. For linear elasticity, large strain implies a low Young's modulus, e.g. less than 10 MPa (see Section 2 for a definition of Young's modulus).

2. We do not considered deformations produced by other physical phenomena such as temperature.

3. Linear elastic deformation occurs when the stress and strain are proportional [START_REF] Callister | Mechanical Properties of Metals[END_REF].

4. We only consider physically-based deformation models.

5. An image registration algorithm that can be used for non-rigid registration (i.e. to track a deformable object) [START_REF] Yang | The thin plate spline robust point matching (TPS-RPM) algorithm: A revisit[END_REF]).

The planning algorithm is based on the Covariant Hamiltonian

Optimization and Motion Planning (CHOMP) method [START_REF] Ratliff | CHOMP: Gradient optimization techniques for efficient motion planning[END_REF].

7. A chart is an invertible map between a simple space (e.g. a 3D

Euclidean space) and a subset of a manifold [START_REF] Morita | Manifolds[END_REF].

8. The squeeze depth is the distance traveled by the fingers along a direction that produces either, a stable or a pure, squeeze. 9. Their simulated evaluation used 12 to 36 manipulation points.

10. A stack here refers to the stack created by a previous fold. For instance, a towel folded by the middle would result in a stack of two.

11. In this context, topological coordinates refer to relationships between segments as introduced in (Ho and Komura 2009).

12. A slip was considered if a vision system detects that the object was not longer being grasped 13. Volumetric mesh with known elasticity parameters such as Young's modulus and Poisson's ratio. 15. Shape-from-Motion is a technique that first generates a 3D template of the target shape. Then, based on a set of points, it finds the correspondence between images, as the camera moves, and the previously generated template.

16. The Broyden method computes the Jacobian once at the beginning and then approximates it at each iteration using the previous Jacobian and the changes of the feature vectors and the end-effector's pose. In this way, the computed Jacobian relates the changes of the end-effector's pose with the deformation feature vector. 17. https://people.eecs.berkeley.edu/ ˜pabbeel/personal_robotics.html 18. http://www.clopema.eu/ 19. One example of a topological description is to represent the object as a graph; where each node represents either an end of a rope or an intersection, and links represent the rope segments between the nodes.

Prepared using sagej.cls

Figure 1 .

 1 Figure 1. Proposed classification of deformable objects.

  Elasto-plastic. (f) Plastic.

Figure 2 .

 2 Figure 2. Top row: a soft object being deformed by an external force. Bottom row: the resulting types of deformation once the external force is removed.

Figure 3 .

 3 Figure 3. A tensile load (F) producing axial and lateral strains.The blue dashed lines represent the original, undeformed, shape and the red solid lines represent the deformed shape[START_REF] Callister | Mechanical Properties of Metals[END_REF]).

  are an example of a meshless model. The mesh-based models are categorized either as continuum or lumped (discrete) variable models, according to the consistency of the mass and stiffness parameters with the approximated displacement fields in the elements of the mesh. The discrete based models are mainly represented as Mass-Spring-Damper (MSD) systems, where the vertices are treated as mass particles and the edges are considered as springs. Continuum based methods are commonly modeled

Figure 4 .

 4 Figure 4. Comparison of physically-based deformation models based on the evaluation results from[START_REF] Sin | Vega: Non-linear FEM deformable object simulator[END_REF]) and the classification presented in[START_REF] Nealen | Physically based deformable models in computer graphics[END_REF]).

Figure 5 .

 5 Figure 5. Examples of different manipulation tasks performed on linear deformable objects.

Discrete

  [START_REF] Lui | Tangled: Learning to untangle ropes with RGB-D perception[END_REF];[START_REF] Matsuno | Manipulation of deformable linear objects using knot invariants to classify the object condition based on image sensor information[END_REF]; Caldwell et al. (2014) Continuous Javdani et al. (2011); Bretl and McCarthy (2014); Borum et al. (2014) Manipulation Shape control Rambow et al. (2012); Bretl and McCarthy (2014); Yamakawa et al. (2012) Untangle Lui and Saxena (2013) Insertion Weifu Wang et al. (2015); Yoshida et al. (2015) Knot tying Motion primitives Yamakawa et al. (2008); Vinh et al. (2012); Kudoh et al. (2015) Algebraic formulation Yamakawa et al. (2010); Takizawa et al. (2015)

  4.1.1 Paper folding A clear example of paper manipulation is origami, which Balkcom and Mason (2008) addressed

  [START_REF] Wada | Indirect simultaneous positioning operations of extensionally deformable objects[END_REF];[START_REF] Hirai | Robust grasping manipulation of deformable objects[END_REF]; Fanson and Patriciu (2010)[START_REF] Das | Autonomous Shape Control of a Deformable Object by Multiple Manipulators[END_REF];[START_REF] Kinio | A comparative study of H∞ and PID control for indirect deformable object manipulation[END_REF];[START_REF] Berenson | Manipulation of deformable objects without modeling and simulating deformation[END_REF]Berenson ( ) (2010));[START_REF] Schulman | Tracking deformable objects with point clouds[END_REF];[START_REF] Boonvisut | Identification and active exploration of deformable object boundary constraints through robotic manipulation[END_REF]

Figure 6 .

 6 Figure 6. A two-finger grasp (a) before and (b) after a squeeze grasp. The points represent contact points, white ones are in slip mode and black ones in stick mode (Jia et al. 2014).

Figure 7 .

 7 Figure 7. The X arrow shows the translational movement and the Θ arrow the rotational motion. Image extracted from (Ramirez-Alpizar et al. 2012).

Figure 8 .

 8 Figure 8. Pipeline describing the common tasks of robotic cloth manipulation.

  State estimation[START_REF] Lee | Learning force-based manipulation of deformable objects from multiple demonstrations[END_REF];[START_REF] Huang | Leveraging appearance priors in non-rigid registration, with application to manipulation of deformable objects[END_REF];[START_REF] Kita | Clothes handling based on recognition by strategic observation[END_REF]; Yamazaki (2014)[START_REF] Bergstrom | On-line learning of temporal state models for flexible objects[END_REF];[START_REF] Li | Real-time pose estimation of deformable objects using a volumetric approach[END_REF]; Cusumano-Towner et al. (2011) Maitin-Shepard et al. (2010); Bersch et al. (2011); Twardon and Ritter (2015) Grasp point detection Kita et al. (2011); Ramisa et al. (2012); Yamazaki (2014) Monsó et al. (2012); Maitin-Shepard et al. (2010); Bersch et al. (2011) Grasp Grasp / lift Lee et al. (2015); Huang et al. (2015); Kita et al. (2011); Cusumano-Towner et al. (2011) Maitin-Shepard et al. (2010); Bersch et al. (2011); Twardon and Ritter (2015) Ramisa et al. (2012); Monsó et al. (2012); Shibata et al. (2009) Manipulation Separate Maitin-Shepard et al. (2010); Monsó et al. (2012) Untangle Lee et al. (2015); Huang et al. (2015); Cusumano-Towner et al. (2011) Maitin-Shepard et al. (2010); Bersch et al. (2011); Twardon and Ritter (2015) Doumanoglou et al. (2014); Li et al. (2015a) Lay / flatten Lee et al. (2015); Huang et al. (2015); Maitin-Shepard et al. (2010) Li et al. (2015a); Sun et al. (2015); Kruse et al. (2015) Final task Fold Lee et al. (2015); Huang et al. (2015); Maitin-Shepard et al. (2010) Bersch et al. (2011); Yamakawa et al. (2011); Miller et al. (2012) Stria et al. (2014); Balaguer and Carpin (2011); Li et al. (2015b) Hang Twardon and Ritter (2015) Clothing assistance Yamazaki et al. (2014); Tamei et al. (2011)the sections, a binary vector describing the three-dimensional garment is formed.

  up a crumpled cloth. The object was represented by the edge of the cloth, along with creases created by wrinkles, as perceived by the depth data. Thus, this contour (edges and creases of the cloth) is described by points obtained from a depth sensor. Then, each point on the contour is compared to grasp points from a training dataset in order to select the most similar point to any grasp point in the dataset. This training dataset was generated with manually assigned grasps from a set of sample images.

  [START_REF] Doumanoglou | Autonomous active recognition and unfolding of clothes using random decision forests and probabilistic planning[END_REF] were able to solve this problem much faster by reducing the number of manipulation actions required to reach the desired configuration of a garment.Using a robot platform with two arms they begin by grasping the garment at a random point with one arm and let it hang to allow the other arm to grasp the lowest hanging point of the garment, e.g. at the end of a sweater's sleeve. Then, the robot releases the first grasp to leave the cloth is hanging by one of its ends. Once hanging, the robot uses a POMDP to continuously rotate the garment until a desired grasp point is detected and it can be grasped by the free arm. This procedure is repeated for the second arm. Thus, allowing the robot to unfold the article by extending it by the two grasped points. To determine the two desired grasp points they used Hough forests, which are random forests using a generalized Hough transform (i.e. able to detect arbitrary shapes). To reduce the number of locations of possible grasp points, they divided the object's bounding box in an 8 × 8 grid.With help of a thin-shell simulation,Li et al. (2015a) also proposed an approach to unfold clothes. They first simulate mesh models of the garments while they are grasped at different points (e.g. neck, sleeves, etc.) and let them hang loose under only the effects of gravity in a physics engine.Next, they built a database out of these models and use it to compare it against the sensed data, which consists of depthdata from multiple-views acquired by rotating the object back and forth. To reconstruct the model based on these data, they use Iterative Closest Point (ICP) to first apply a rigid registration and then a non-rigid registration; where the rigid registration considers only translation and rotation between the sensed data and the modeled data, and the non-rigid registration accounts for the deformation of the garment.Once the model has been reconstructed they are able to grasp the required points on the garments, for instance the elbows of sweater, to unfold them.Besides unfolding a piece of cloth, or bringing it to a desired configuration, some approaches rely on the garment to be laying flat on a surface.[START_REF] Sun | Accurate garment surface analysis using an active stereo robot head with application to dual-arm flattening[END_REF] attempted to solve this task, where a two arm robot stretches a piece of cloth to flatten out the wrinkles. This approach uses RGB-D data to detect and locate wrinkles on the cloth and subsequently plan the required motions for the robot's arms to remove the wrinkles. The wrinkles are described as fifth order polynomial curves and they are assigned a weight based on their volume. The largest wrinkle is then selected to be flatten by a predefined motion of the arms, which pulls the cloth in order to remove the wrinkle.Using a different approach, namely hybrid force-vision control,[START_REF] Kruse | Collaborative human-robot manipulation of highly deformable materials[END_REF] also used a two-armed manipulator to stretch a cloth. However, they include a human in the control loop which holds one edge of the cloth while the robot holds the other. This approach uses the output of an RGB-D sensor and torque sensors to maintain the cloth taut. They used a vision controller relying on the normals of each point on the cloth to compute a velocity vector.

  One of the most popular tasks regarding cloth manipulation is that of folding. A fold can be performed either on a supporting surface, as depicted in Figure 9, or in the air as shown in Figure 10. The approaches that have tackled the problem of robotic cloth-folding have relied on a variety of techniques such as detecting grasp points, learning by demonstration, geometric representation (e.g. describing clothes as polygons), etc. Maitin-Shepard et al. (2010) were the first to enable a robot manipulator to solve the complete task of folding clothes starting from an unorganized pile of clothes. The main focus of their work concentrated on detecting grasp points to facilitate the folding, since predefined manipulation actions execute the folding maneuver. Their algorithm assumes the cloth is of rectangular shape (e.g. a towel) and they used stereo correspondence to detect the garment's corners. Specifically, they applied random sample consensus (RANSAC) to a border classification algorithm which detects the edge of the garment based on depth discontinuities.

  -Shepard et al. 2010), where only towels are considered due to their rectangular shape, this work enables a robot to fold a T-shirt. However, the approach relies on the T-shirt being completely covered with fiducial markers. These markers are used to infer the position where a robot's gripper is holding the T-shirt by rotating it to generate a 3D model that includes a point cloud representation along with the markers' positions. Once the robot knows the location of the grasp it uses its free arm to grasp one of the predefined key points, in this case the shoulders of the T-shirt. This sequence is repeated until the robot is holding the T-shirt by both shoulders and then it executes a motion to fold the Tshirt in an open-loop manner.Besides detecting grasp points to subsequently fold a garment, other approaches assume this step has been performed and focus only on the folding action. For instance,[START_REF] Yamakawa | Motion planning for dynamic folding of a cloth with two high-speed robot hands and two high-speed sliders[END_REF] proposed an approach where two robotic hands holding a garment by its corners fold it in midair. The garment was modeled as grid of nodes, where the nodes that are grasped by the robotic grippers are assumed to have the same position as the grippers (e.g. if the hand moves, the grasped node follows the same trajectory); and the position of the remaining nodes is determined proportionally based on the distance to the grasp nodes. This

Figure 9 .

 9 Figure9. A geometry based approach to fold a clothing item laying on a table[START_REF] Stria | Garment perception and its folding using a dual-arm robot[END_REF]).

Figure 10 .

 10 Figure10. Sequences of a dynamic in-air fold executed in less than half a second[START_REF] Bergstrom | On-line learning of temporal state models for flexible objects[END_REF]).

  (Clothing assistance, hanging) Other applications for clothing manipulation found in the literature concern clothing assistance and hanging of clothes. One case of clothing assistance, such as helping a person put on a shirt, was tackled in[START_REF] Tamei | Reinforcement learning of clothing assistance with a dual-arm robot[END_REF]). Here, reinforcement learning was used on a two-armed robot to put a shirt on a mannequin. In order to reduce the task's dimensionality, they define topological coordinates 11 , on both the shirt and the mannequin; and only control one joint on each arm. A motion capturing system tracked relevant points on the Prepared using sagej.cls mannequin and the T-shirt, which later were used to compute the topological coordinates. They start by teaching the robot how to solve the task through kinesthetic training to extract an initial trajectory, then reinforcement learning modifies the way-points of the trajectory in order to achieve the task.Another case of clothing assistance can be seen in the work of[START_REF] Yamazaki | Bottom dressing by a life-sized humanoid robot provided failure detection and recovery functions[END_REF], where a humanoid robot helps a person putting on a pair of pants. This approach computes an optical flow on images to detect the state of the task by comparing the flow with a previously generated database of flows, e.g. it can identify if a leg is being introduced in an opening of a pair of pants. It also incorporates depth data to estimate the location of the person's legs and force sensing is used to detect a possible failure such as having a leg stuck in the pants. If a failure is detected, the approach is able to recover by modifying a predefined trajectory. One recent approach, proposed by[START_REF] Yu | Haptic Simulation for Robot-Assisted Dressing[END_REF], focused on predicting the outcome of dressing a person with a robot using HMMs. The outcomes, for a task of pulling a medical gown onto a person's arm, were classified as either successful dressing, missing the arm or getting the arm caught by the gown. Due to the complexity to gather haptic data that involves a robot to generate multiple contacts with persons and as this process is unsafe and time consuming, the authors propose to use a physics simulation to generate simulated haptic data. By relying on simulation, they were able to reduce risk, since the number of data gathered from real interactions with people was kept at a minimum while still achieving high accuracy results. Estimating the forces exerted by a garment on a

( c )

 c Shape control (Navarro-Alarcon et al. 2013b).

Figure 11 .

 11 Figure 11. Examples of different manipulation tasks performed on solid deformable objects.Table 4. Classification of approaches to manipulate solid objects using a robot.

  lift test is based on the simulated deformation generated by the set of contact points of each finger. Using this simulated deformation, they checked whether the majority of the contact points are not sliding (i.e. are sticking) to determine if the lift test is passed. Similarly, citeZaidi2017 applied an FEM simulation with a Mass-Spring-Damper model to estimate the object's deformation caused by a three-finger grasp. As opposed to[START_REF] Lin | Picking up soft 3D objects with two fingers[END_REF][START_REF] Lin | Picking up a soft 3D object by "feeling" the grip[END_REF], this approach computes the contact forces using a spring-damper contact model based on the fingers' positions and velocities. The computed forces are the input to the FEM simulation which outputs the new positions of the mesh's nodes, based on the Mass-Spring-Damper model, to describe the object's deformation. However, only simulation results were presented in this work.

6. 2 . 1

 21 Parameter identification Parameters (either physically meaningful such as the Young's Modulus or taskspecific, as a coefficient of deformation) can be estimated through sensor measurements with or without a model. An approach that requires a model was presented by Frank et al. (2010, 2014), where the Young's modulus and the Poisson ratio of different flexible objects were estimated. The estimation was based on force sensor measurements, obtained by probing the objects, a volumetric model of the object which was generated using depth data from multiple views and a co-rotational FEM model. Having the simulation of the deformation from the FEM model and the observed deformation, a search for the best values of these parameters was performed such that the error between the observed and simulated deformation was minimized. Instead of an FEM model, Fugl et al. (2012) used a more limited deformation model, namely the Euler-Bernoulli beam model, to estimate the Young's modulus of a flexible

  incorporating depth data into a physical model. However, this approach used a mass-spring model to estimate the deformations caused by an external force. The chosen model resulted in a much faster simulation, between 80-120 Hz; with the drawback of not being able to handle large deformations. Recently, a real time simulation using a FEM model was presented in[START_REF] Petit | Real-time tracking of 3D elastic objects with an RGB-D sensor[END_REF]. A known mesh of the object is assumed and its pose and deformation are tracked with an RGB-D sensor. They demonstrated accurate results with a reasonable computational cost, at around 35 Hz. Another task that requires real time estimation of deformable objects is soft tissue tracking, which is essential to automate surgical procedures like minimally invasive surgeries. As minimally invasive surgeries, such as laparoscopic procedures, are preferred over open surgery as they have been shown to reduce recovery times for patients; there is an increasing need to improve the visualization of internal anatomy during surgery. One of the most successful approaches to address this issue has been the geometric reconstruction of soft tissue in 3D. For example, Malti et al. (2011) used Shape-from-Motion 15 to first construct a 3D model (template) of a soft tissue before the surgeon caused any deformation. Once the surgeon started deforming the tissue, the 3D shape was reconstructed based on monocular images as obtained by a laparoscopic camera. In order to account for the deformations caused by the surgeon, their approach considered shearing and anisotropy scaling while computing the correspondance between an image and the template. Their approach was able to produce decent results on in-vivo experiments. Similarly, dos Santos et al. (2014) proposed an algorithm to track soft tissue deformations where, instead of relying on scaling parameters, they used surface matching to perform the registration. Their method proved to be accurate and fast enough to be used in clinical trials. Since the body of works focused on 3D registration for surgery assistance is beyond the scope of this survey, the interested reader is referred to (Maier-Hein et al. 2013) for a thorough review of vision techniques dedicated to sensing soft-tissue.

A

  series of reports by Navarro-Alarcón et al. used a similar approach, namely defining control and manipulation points and then commanding the motion of a robotic manipulator, through visual servoing, to reach a desired configuration for Prepared using sagej.cls an elastic object. However, these works do not rely on a model of the object; instead a vision system was used to track the control points. A distinctive characteristic for these works, was the use of so-called deformation feature vectors based on a set of control points to describe the different types of deformation, such as: (a) Point-based deformation: The object is deformed in order to drive one point on the object to a desired target point. (b) Distance-based deformation: For instance, moving the midpoint between two points a specified distance. (c) Angle-based deformation: A line between two points on the object can be rotated as desired. (d) Curvature-based deformation: An arc of three points on the object can be manipulated to achieve a specific curvature.

Figure 12 .

 12 Figure 12. Deformation features proposed by Navarro-Alarcon et al. (2013a).

Figure 13 .

 13 Figure13. Classification of the 106 reviewed approaches based on their task(s) and object type(s). Note that some approaches appear multiple times across task and object types.

  classification and provided prospective research lines for each type of object. We recognize that deformation sensing, mostly based on vision systems, has seen important advances recently. In comparison, manipulation skills still remain underdeveloped, in particular, dynamic manipulation of deformable objects. Control algorithms, and perhaps improved hardware, need to be developed to improve endeffectors' dexterity such that they can appropriately react to the object's deformation in real time. As manipulating the object might cause occlusions, vision-based sensing could overcome this by including the output of force and tactile sensors. This, in turn, requires deformation and contact models that transform force and tactile signals into a compatible representation (e.g. geometric information) obtained by visual systems. Another insight, obtained by reviewing and classifying the state of the art on this field, is a clear tendency of approaches to develop techniques focused on specific tasks, or by exploiting assumptions that are unique to a given object type. Thus, a lack of approaches that provide general solutions for different object types and tasks remains a major open issue. Finally, we intend this survey helps researchers new to the field get an overview of the state-of-the-art techniques and their applications; and to researchers already in the field, as a quick guide to relevant approaches on their particular interests. of Excellence (ANR-10-EQPX-44) and the IMobS3 Laboratory of Excellence (ANR-10-LABX-16-01), by the European Union through the program Regional competitiveness and employment 2007-2013 (ERDF -Auvergne region) and by the Auvergne region.

  14. Unlike FEM and MSD models, position-based simulation are geometrically-based rather than physics-based. They trade off accuracy for speed in simulation.

Table 2 .

 2 Classification of approaches that focus on planar objects.

Manipulation Grasp Guo et al. (2013); Jia et al. (2011); Gopalakrishnan and Goldberg (2004) Gopalakrishnan (2005); Jia et al. (2013, 2014) Pick up Jia et al. (2014); Elbrechter et al. (2011); Kristek and Shell (2012) Rotation Kristek and Shell (2012); Ramirez-Alpizar et al. (2012) Paper folding Balkcom and Mason (2008); Elbrechter et al. (2012) Namiki and Yokosawa (2015)

  the table to the other, thus reducing uncertainty, and removing a piece cloth from the table once the uncertainty is below a threshold. Picking more than one piece of cloth is considered a failure. Observation and transition models were obtained experimentally. Lastly, the goal was defined as removing all the clothes from the table with the least amount of manipulation actions.

	5.2.2 Manipulation	for	garment	reconfiguration
	Cusumano-Towner et al. (2011) sought to bring a clothing
	article into a desired configuration using a PR2 robot. To
	do so, they used a HMM, a cloth simulator and a planning
	algorithm. The HMM is used to estimate the garment's
	configuration, where the hidden state only includes the
	article's category and size, and the grasp points for both
	grippers. These grasp points represent the nodes of the
	garment's mesh model, according to the cloth simulator,

used a robot manipulator to separate clothes. They considered the inherent sensing uncertainty to solve this problem using a partially observable Markov decision problem (POMDP). Here, the state is represented by the number of clothes on two sides of a table. The actions consist of either moving a piece of cloth from one side of

Prepared using sagej.cls

Journal Title XX(X)

Funding

This work has been sponsored by the French government research program Investissements d'Avenir through the RobotEx Equipment