Ivan Laptev

Emmanuel Cottanceau

Gil Boye De Sousa

Pierre Besset

Ainsi Que

Tiago Moraes

Franck Hernoux

David Busson

Arthur Givois

Guillaume Baille

Carine Charles

Juli- Ette Morin

Thomas Pidancier

Nadim El

Hayek Je

Mathias Guérin

Keywords: Robotic sorting, Image clustering, Transfer learning, Optimal viewselection, Trajectory learning, Autonomous dataset generation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Byron Boots, assistant professor à Georgia Tech, pour m'avoir fait beaucoup avancer grâce à ses précieux conseils. J'adresse des remerciements à l'équipe d'ingénieurs du LISPEN, Jorge Palos, Adrien Florit et Vincent Deschodt qui m'ont épaulé dans chacune des expériences pratiques que j'ai dû mettre en place. Je remercie l'ensemble de mes amis d'Atlanta, Yash Chitalia, Marina Alaricheva, Shray Bansal et Sasha Lambert qui ont fait de mes mois américains une période enrichissante, de laquelle je garde d'excellents souvenirs.

we propose to rethink the standard robotic sorting application within this context. Usually, to decide where to sort objects, a robotic sorting system needs to solve either an instance retrieval (known object) or a supervised classification (predefined set of classes) problem. In this chapter, we introduce a new decision making module, where the robotic system chooses how to sort the objects in an unsupervised way. This approach generalizes common robotic sorting to any new set of previously unseen objects. This application is called Unsupervised Robotic Sorting (URS) and entails most of our contributions. Furthermore, other robotic skills for autonomous trajectory learning and 3D stereo localization are developed with similar motivations.

1.1 Machine learning for adaptive robotic skills 1.1.1 History of industrial robotics

In 1962, General Motors installed serial robots to carry out spot welding and diecastings extraction in their New Jersey's plant. These machines, developed by Unimation, were the first robots to successfully perform industrial tasks and mark the beginning of a new era in industrial production. Industrial robots are multifunctional machines which can be programmed to carry out different tasks. They differ from more standard specialized machines in that they can be reprogrammed for a new task without physical modification, and indeed, during three decades, many Unimation robots were sold worldwide to execute a variety of tasks. This reprogrammability presents the huge advantage of reducing the costs and times of mechanical design for new industrial applications, which rapidly lead to the emergence of many new robot manufacturer. The main tasks robots were dealing with in the end of the 20 th century were assembly (( [START_REF] Chen | Determining task optimal modular robot assembly configurations[END_REF]), [START_REF] Grunes | Robot assembly[END_REF]), [START_REF] Bonert | Motion planning for multi-robot assembly systems[END_REF])), welding (( [START_REF] Sicard | An approach to an expert robot welding system[END_REF]), [START_REF] Kim | Robot arc welding operations planning with a rotating/tilting positioner[END_REF])) and sorting (see Section 1.2.1.1). A more complete study about the history of industrial robotics can be found in both ( [Westerlund, 2000]) and ( [Wallén, 2008]).

This important development of robotics in industrial factories has brought many new interesting problems to the research community (( [START_REF] Sciavicco | Modelling and control of robot manipulators[END_REF]), [START_REF] Groover | Industrial robotics: technology, programming, and applications[END_REF]), ( [Nof, 1999]), ( [Mair, 1988])). During 40 years, research in robotics was mostly driven by two problems of utmost importance for industry:

• How to define an optimal trajectory to accomplish a given task? (mechanics)

• How to ensure that the given trajectories are correctly followed? (control)

All these years of research in automation lead to impressive production plants where robots perform complex tasks at high velocity and with high precision (Figure 1.1).

However, this way of approaching robotics also has its limits as the robots can only perform highly repeatable tasks. In addition, such high velocity and non-adaptive applications constrain the robots to be isolated in secured environments, preventing interaction with humans. Hence, implementing a new application requires to physically build the robotic cell as well as programming the robot, which is a complex task requiring highly qualified engineers. These limitations mitigate the advantage of industrial robots over special machines since a huge amount of work is still required for a new implementation, thus limiting the use of robots to tasks dealing with large production volumes in order to be profitable.

A new robotics context

In the past 20 years, robotics research has undergone important transformations driven by an increasing industrial demand for more flexible robots. Indeed, in order to diminish drudgery of work while improving precision, and to some extent increasing productivity, the diversity of tasks involving robots in industry has increased significantly. Some examples of these new tasks include bin picking and sorting (( [START_REF] Liu | Fast object localization and pose estimation in heavy clutter for robotic bin picking[END_REF]), [START_REF] Lukka | Zenrobotics recycler-robotic sorting using machine learning[END_REF])), visual inspection and metrology (([de Sousa et al., 2017]), [START_REF] Jayaweera | Metrology-assisted robotic processing of aerospace applications[END_REF])), machining [START_REF] Olabi | Feedrate planning for machining with industrial six-axis robots[END_REF]), ( [START_REF] Chen | Robot machining: recent development and future research issues[END_REF])), kitting [START_REF] Balakirsky | Knowledge driven robotics for kitting applications[END_REF]), [START_REF] Banerjee | An ontology to enable optimized task partitioning in human-robot collaboration for warehouse kitting operations[END_REF])), etc.

Together with the number of different tasks, the flexibility required within the context of each task has also increased. The growing demand for mass customization [START_REF] Zawadzki | Smart product design and production control for effective mass customization in the industry 4.0 concept[END_REF]) together with the apparition of new industries in the robotics market (aerospace, energy, etc.) has lead to a new industrial context called Industry 4.0 [START_REF] Bahrin | Industry 4.0: A review on industrial automation and robotic[END_REF]) or smart factory ( [START_REF] Lucke | Smart factory-a step towards the next generation of manufacturing[END_REF]). This new context is defined by, among other things, smaller production volumes and higher responsiveness of the production plants. To illustrate the need for more adaptive robotics applications within this context, we can consider the simple example of a screwing operation. On the one hand, in a large automotive robotics assembly line, a robot needs to drive a given number of screws that are always at the same location, and repeat this operation an important number of times. This operation is programmed by finding an optimal trajectory and a control strategy to follow such trajectory. In some sense, implementing such a robotic screwing operation does not really differ from implementing a welding or painting operation. On the other hand, we can consider the context of satellites production in the aerospace industry, which also requires many screwing operations but are only produced in "batches" of one.

The task thus becomes "screw what you encounter on the structure", which is much more challenging as cinematic alone is not sufficient and needs to be coupled with sharp perception skills. This thesis is part of the Colrobot european project 1 , which also illustrates well the new context of industrial robotics.

Machine learning and robotics

Because of these new expectations, instead of directly dealing with applications, current research focuses on providing robots with diverse skills, which can be combined to achieve various goals. To be functional, such skills must either be robust to a variety of conditions, such as different poses of manipulated objects and different environments, or be easy to reconfigure by unqualified operators. Learning methods 2 are well suited to develop such robotic bricks and are being widely adopted in recent research.

Some examples of robust robotic skills which were successfully learned are scene understanding ( [START_REF] Shi | Datadriven contextual modeling for 3d scene understanding[END_REF]), grasping [START_REF] Bohg | Datadriven grasp synthesis-a survey[END_REF]), [START_REF] Lenz | Deep learning for detecting robotic grasps[END_REF])), object recognition ( [Eitel et al., 2015a]), motion planning ( [START_REF] Dong | Motion planning as probabilistic inference using gaussian processes and factor graphs[END_REF]), localization and mapping ( [START_REF] Kim | Active visual slam for robotic area coverage: Theory and experiment[END_REF]), trajectory generation ( [START_REF] Levine | Learning contact-rich manipulation skills with guided policy search[END_REF]), etc. To obtain the desired robust behaviors, a robot must be highly equipped with sophisticated sensors. Processing such complex information and link it to robot movements is a difficult task that in many cases can only be solved with learning methods. In the meantime, the machine learning framework is also well adapted to design applications that are easily reconfigurable by unqualified human workers, as suggested by the path followed by the field of human-robot collaboration (( [START_REF] Tsarouchi | On a human-robot collaboration in an assembly cell[END_REF]), [START_REF] Munzer | Efficient behavior learning in human-robot collaboration[END_REF])).

This thesis fits within this adaptable and customizable robotics context. The different contributions that are presented in this manuscript aim at:

• providing the community with new algorithmic tools and datasets to develop such kind of robust robotics skills,

• demonstrating feasibility of the proposed approaches on real robotic systems.

The main focus of this work, which constitutes most of this thesis, is on the development of object understanding and sorting applications that are context independent 1 https://www.youtube.com/watch?v=8zpYzVEw-Io 2 A beginner's introduction to what is machine learning can be found in Appendix A.

and robust. These topics are studied within the context of Unsupervised Robotic Sorting, which is introduced in the next section. Along the same line, we also propose a more generic method for trajectory learning, which is easier to set up by non experts and to embed into more complex learning frameworks. Our last contribution consists in defining a methodology for autonomous dataset construction for object stereo localization. These contributions as well as the thesis organization are better detailed in Section 1.3 and more specific bibliographical elements are discussed in each chapter.

Unsupervised Robotic Sorting

In this section, we propose a broader introduction of the unsupervised robotic sorting application, which motivates many of the contributions of this thesis.

Context 1.2.1.1 Autonomous sorting

The problem of automatic sorting has a long history in industry, with the first tomatoes sorting system dating back to the 70's ( [START_REF] Husome | Color sorting system[END_REF]). Since then, it has received a lot of attention, with important focus on combining computer vision and robotic manipulators to solve the pick-and-place task ( [START_REF] Mouli | A review on wireless embedded system for vision guided robot arm for object sorting[END_REF]).

Although it was among the first robotics tasks, designing a pick-and-place application is challenging and requires solving multiple subtasks. If the objects to be sorted are cluttered, the system first needs to segment the scene and identify the different objects ( [START_REF] Ecins | Cluttered scene segmentation using the symmetry constraint[END_REF]). Then, the system must use various sensors, such as 2D or 3D cameras, color sensors or bar code readers, to gather data about the different objects. These data are used to find a grasping strategy for the object ( [START_REF] Bohg | Datadriven grasp synthesis-a survey[END_REF]) and to decide how to classify it. Naturally, all along this process, smart motion planning and control is also required ( [START_REF] Siciliano | Springer handbook of robotics[END_REF]).

Although every bricks of the pick-and-place pipeline are interesting and challenging problems, this thesis focuses more on the decision making subproblem, i.e. how to decide where to place an object after grasping it? Other skills that can also be useful for robotic sorting are studied in the final part of the manuscript.

Decision making in autonomous sorting

Previous implementations of autonomous sorting dealt with objects which are either known or belong to some predefined classes. In machine learning terminology, one could say that the robot intelligence usually resides in solving either an instance retrieval or a supervised classification problem. Different approaches in solving such decision-making problems differ in the kind of sensing devices used, as well as the algorithmic choices.

Instance retrieval Several implementations have considered the problem of recognizing each object as a member of a known database, and sorting it according to an associated predefined rule. In ( [START_REF] Giannoccaro | A smart robotic arm for automatic sorting of objects with different tags[END_REF]), RFID markers are used to recognize and sort objects. An implementation of a surgical tools sorting application using barcode reading and template matching to find and sort objects from a cluttered scene is proposed in [START_REF] Tan | An integrated vision-based robotic manipulation system for sorting surgical tools[END_REF]). Finally, in the application presented in [START_REF] Dragusu | Practical applications for robotic arms using image processing[END_REF]), real-world objects (usb, glue-stick, etc.) are sorted with a serial manipulator using template matching.

Supervised classification Many robotics supervised classification applications sort objects based on simple rules such as thresholding, applied to simple features. For example, [START_REF] Szabo | Automated colored object sorting application for robotic arms[END_REF]), [START_REF] Shum | Design, build and remote control of a miniature automated robotic sorting system[END_REF]) and ( [START_REF] Nkomo | A color-sorting scara robotic arm[END_REF])) all use color features extracted from either images or color sensors, ( [START_REF] Pereira | Low cost object sorting robotic arm using raspberry pi[END_REF]) extend these approaches by adding shape features. In ( [START_REF] Gupta | Using manipulation primitives for brick sorting in clutter[END_REF]), robot manipulation is used to unclutter Duplo bricks, which are then sorted according to their length and color. In ( [START_REF] Singh | Object sorting by robotic arm using image processing[END_REF]), a classical computer vision framework is implemented to carry out faulty parts removal. A robotic system able to recycle construction wastes using features provided by metal detectors and sensors sensitive to visual wavelengths is presented in [START_REF] Lukka | Zenrobotics recycler-robotic sorting using machine learning[END_REF]). Recent papers have started to use Convolutional Neural Networks (CNN) to carry out supervised classification in robotic sorting. The authors of ( [START_REF] Eitel | Multimodal deep learning for robust rgb-d object recognition[END_REF]) have used a CNN on RGBD images to find the class of an object and sort it. In ( [START_REF] Zhihong | A vision-based robotic grasping system using deep learning for garbage sorting[END_REF]), a fast R-CNN proposal network is combined with a pretrained VGG16 CNN architecture to jointly localize and sort objects with a robot manipulator.

Unsupervised

Robot Sorting

Problem definition

Chapters 2 to 6 of this manuscript are about a new kind of pick-and-place application, consisting in sorting unknown objects, which do not belong to any predefined class.

Given a set of previously unseen objects, the robotic system needs to sort them such that objects stored together are similar to each other and different from other objects.

Such problem definition is the definition of a clustering problem (see Chapter 2 for more details) in which the outputs consist in physically sorted objects, hence, we call this problem Unsupervised Robotic Sorting (URS).

The above statement of the URS problem neither defines the notion of similarity nor the number of groups to be made. Different definitions of these concepts defines different instances of the URS problem. This thesis mainly addresses the URS problem in which the number of groups is imposed by the number of storage spaces available and the notion of similarity is defined by "how a human would solve the problem". Figure 1.2 shows the inputs and associated expected outputs for two practical examples of URS. It is important to note that the notion of reproducing human decision is highly subjective as the optimal classification can vary between individuals. More specifically, in this thesis we aim at reproducing the decision which would be mostly adopted by a group of human experts at the classification task at hand. For example, specialist doctors for unsupervised classification of medical images, qualified workers for industrial classification tasks, etc. In practice, we validate our algorithms by trying to reproduce the classifications of several standard supervised datasets without using the labels. These datasets are created by human experts and can thus be used as proxies for the expert populations described above. More details on the validation methods can be found in the relevant chapters.

Contribution and thesis organization

The rest of this thesis is organized as follows. Chapters 2 to 6 deal with the URS problem introduced previously. Chapter 2 complements the introduction of URS by proposing a first implementation. This unsupervised sorting application is based on color and shape features and leads to the development of a new K-means clustering algorithm to handle such kind of data when they are noisy. The contributions of Chapter 2 were published as a conference paper at CCSEIT 2017 ([Guérin et al., 2017]).

In Part II of this thesis, we propose to study the problem of Image Clustering (IC), which is a required skill to implement URS with a higher level of abstraction. Hence, Chapter 3 proposes a benchmark study on solving IC by transferring knowledge from deep Convolutional Neural Networks (CNN) pretrained on large and versatile datasets. Then, Chapter 4 builds on the results from Chapter 3 to propose an ensemble clustering approach to leverage information from several such CNNs. This is shown to improve state-of-the-art results at image clustering. The studies in Chapters 3 and 4 were published in the proceedings of two conferences, respectively AIFU 2018 ([Guérin et al., 2018c]) and BMVC 2018 ( [START_REF] Guérin | Improving image clustering with multiple pretrained cnn feature extractors[END_REF]).

After studying image clustering on standard datasets, we propose to study practical URS implementations in Part III. In Chapter 5, we introduce a first implementation and evaluate its robustness to various external factors. This study suggests that the results are highly dependant on the point of view under which the objects are observed, which motivates the development of an optimal view selection method in Chapter 6. This contribution consists in creating a large multi-view dataset and use to train a neural network to choose optimal camera poses. This approach is shown to work better than fixed camera poses. The physical implementation of URS proposed in Chapter 5 has been published in the International Journal of Artificial Intelligence and Applications ( [START_REF] Guérin | Unsupervised robotic sorting: Towards autonomous decision making robots[END_REF]), and the developments on optimal view selection from Chapter 6 can be found in the proceedings of IROS 2018 ([Guérin et al., 2018a]).

The contributions about object understanding proposed in Part II and III are important skill for many autonomous robotic tasks. However, many other abilities are required to reach full autonomy, as discussed in Section 1.2.1.1 in the context of URS. Hence, in Part IV, we propose to explore two of such skills. On the one hand, Chapter 7 presents an adaptation of a popular trajectory learning method. By learning a quadratic model of the cost function instead of computing it analytically, this method is made independent of both the system's model and the cost function's definition, thus making the system easier to program in more situations. This contribution resulted in the publication of two conference papers: ( [Guérin et al., 2017]) and [START_REF] Guérin | Learning local trajectories for high precision robotic tasks: application to kuka lbr iiwa cartesian positioning[END_REF]), which were presented respectively at ACD 2016 and IECON 2016. On the other hand, Chapter 8 proposes a methodology to autonomously build datasets for 3D stereo localization using a robot manipulator. One such dataset is built as a proof of concept and made publicly available. A short paper on this thematic can be found in the proceedings of IECON 2018[START_REF] Guérin | Automatic construction of real-world datasets for 3d object localization using two cameras[END_REF]).

Finally, Part V proposes a summary of our contributions and draws perspectives for future work. All the different research items introduced in this thesis share the same objective of developing industrial robotic application with a broader validity range. We believe that it can potentially lead to easier and more accessible robot programming, and by extension help the democratization of robotics.

Chapter 2

First example of URS: clustering from color and shape features -the Gap-Ratio K-means algorithm

Abstract

The most standard robotics sorting setting consists in using color and shape features to carry out supervised classification. To illustrate the Unsupervised Robotics Sorting (URS) problem, we propose to extend this use case to the unsupervised setting. This application presents various challenges: 1. Color and shape features are, in general, of a different order of magnitude. 2. In robotics sorting, data are measured from real world objects in shop floor conditions (i.e., uncontrolled lighting), which involves a large spread in the data (i.e., noisy data). 3. Data dimensions are on different levels of measurements: RGB color features are interval type variables while length features are ratio type variables. To overcome these difficulties, a new weighted K-means algorithm, called Gap Ratio K-means, is introduced. It consists in defining weights which capture information about the relative differences between consecutive data points for each feature dimension. Gap Ratio K-means is evaluated on a real world example consisting in physically clustering Lego bricks of different colors and shapes. It is compared with two other variants of K-means and demonstrates more robust clustering results.

Problem description and challenges

As mentioned in the previous chapter, the standard approach to robot sorting consists in solving an instance retrieval or supervised classification problem on objects characterized by color and shape features. In this chapter, we propose to adapt such kind of sorting application to the unsupervised case (Figure 2.1).

Figure 2.1: KUKA LBR iiwa sorting objects from color and shape features using the Gap-Ratio K-means algorithm. Video at : https://www.youtube.com/watch?v=korkcYs1EHM

Unsupervised sorting of real world objects represented by color and shape features presents various challenges: Firstly, in the general case, length and color features are of a different order of magnitude, which implies the need for data normalization.

Secondly, because such application is meant for ordinary environments, the clustering algorithm needs to be robust to unmastered light conditions, which is synonymous with widely spread datasets. For such noisy data, the loss of information involved by normalization can impact negatively the clustering results. A way to solve this issue is to use weighted K-means algorithms to re-inject the information in the normalized data. Finally, the features chosen are on different levels of measurement ( [Stevens, 1946]): RGB-color features are interval variables whereas lengths are on a ratio scale.

Statistics using ratios or means (e.g., Coefficient of Variation (CV)) cannot be used on interval type data. Hence, all the weighted K-means algorithms using this kind of statistics cannot be used for the proposed problem. These three specificities of the clustering problem motivate the development of a new weighted K-means algorithm.

This clustering method, called Gap-Ration K-means (GRKM), is the main contribution of this chapter.

Preliminaries

Clustering

The sorting problem described above is a clustering problem (( [START_REF] Theodoridis | Chapter 11-clustering: Basic concepts[END_REF]), [START_REF] Duda | Unsupervised learning and clustering[END_REF])), also called unsupervised classification.

Given an unlabelled dataset, the goal of clustering is to define groups (called clusters) among the entities. Elements in one cluster should be as similar as possible to each other and as different as possible to other clusters' members. In this chapter, only clustering with predefined number of classes (imposed by the number of bins in which objects can be stored) is studied. There are many possible definitions of similarity between data points. The choice of such definition, together with the choice of the metric to optimize, defines a clustering algorithms. The two surveys ( [START_REF] Xu | Survey of clustering algorithms[END_REF]) and ( [Berkhin, 2006]), give two slightly different taxonomies of the various clustering algorithms.

For robotics sorting applications, the number of objects to cluster might be small (≈ 10), which makes some deep clustering ( [START_REF] Aljalbout | Clustering with deep learning: Taxonomy and new methods[END_REF]) and density based ( [START_REF] Ester | A density-based algorithm for discovering clusters in large spatial databases with noise[END_REF]) methods irrelevant. After trying several clustering algorithms on simulated color and shape datasets using scikit-learn ( [START_REF] Pedregosa | Scikit-learn: Machine learning in Python[END_REF]), K-means ([Theodoridis and Koutroumbas, 2006c]), a partitioning method, appeared efficient for our problem. Therefore, among all clustering methods, this chapter focuses on K-means.

K-means

Notations

Throughout this chapter, the following notations are used. Letter i represents indexing on data objects whereas letter j designates the features. Thus, X = {x 1 , ..., x M } represents the dataset to cluster, composed of M data points. Each data point is represented by N features and x ij stands for the j th component of the feature vector of object x i ∈ R N . Likewise, the use of letter k represents the different clusters and

C = {C 1 , ..., C k , ..., C K } is a set of K clusters. Each cluster C k ,
is represented by a cluster center, or centroid, denoted c k , which is simply a point in the feature space. We also introduce d, the function used to measure dissimilarity between a data object and a centroid. For K-means, such dissimilarity is quantified with Euclidean distance

d(x i , c k ) = N j=1 (x ij -c kj ) 2 .
(2.1)

Derivation

Given a set of cluster centers c = {c 1 , ..., c k , ..., c K }, cluster membership is defined by

x i ∈ C l ⇐⇒ d(x i , c l ) ≤ d(x i , c k ), for all k ∈ {1, ..., K}. (2.
2)

The goal of K-means is to find the set of cluster centers c * which minimizes the sum of dissimilarities between each data object and its closest cluster center. Introducing the binary variable a ik , which is 1 if x i belongs to C k and 0 else, and the membership

matrix A = (a ik ) i∈{1, ... M } k∈{1, ... K}
. K-means can be written as an optimization problem:

Minimize A, c M i=1 K k=1 a ik × d(x i , c k ), subject to K k=1 a ik = 1, for all i ∈ {1, ..., M }, a ik ∈ {0, 1}
, for all i, k.

(2.3)

In practice, (2.3) is optimized by solving iteratively two subproblems, one where the set c is fixed and one where A is fixed. The most widely used algorithm to implement K-means clustering is the Lloyd's algorithm ( [Lloyd, 1982]). It is based on the method of Alternating Optimization ( [START_REF] Bezdek | Some notes on alternating optimization[END_REF]), also used in the Expectation-Maximization algorithm ( [START_REF] Dempster | Maximum likelihood from incomplete data via the em algorithm[END_REF]). The Expectation step (E-step) consists in associating each data point to it's closest cluster center following (2.2), i.e. computing A. The Maximization step (M-step) consists in computing the centroids minimizing the total dissimilarity within clusters, i.e., computing c. When the L 2 norm is used for dissimilarity, which is the case for K-means, it can be shown that the M-step optimization is equivalent to computing the cluster mean [START_REF] Theodoridis | Chapter 14-clustering algorithms iii: Schemes based on function optimization[END_REF]).

Data normalization

In most cases, running K-means on raw data does not work well. This is particularly true when data dimensions are of a different order of magnitude: features with largest scales are given more importance during dissimilarity calculation and clustering results are biased (Figure 2.2). The kind of features studied in this chapter are impacted by this problem, colors are expressed in RGB (between 0 and 255) while lengths depend on the size of the object and the unit chosen to express it. For objects sorted with a standard manipulator, lengths usually measure a few centimeters. To deal with this issue, a common practice is to normalize the data before running the clustering algorithm:

∀i, ∀j, x ij ← x ij -µ j σ j , (2.4)
where µ j and σ j represent respectively the empirical mean and standard deviation over {x ij , for all i ∈ {1, ..., M }}.

Weighted K-means

As explained above, data normalization is often necessary to obtain satisfactory clustering results. However, reducing each feature distribution to a Gaussian of variance 1 can involve a loss of valuable information for clustering. This is particularly true for data with a large spread as the span of each dimension is lost during normalization.

For example, Figure 2.3 shows an dataset where three groups are to be found along the same axis. We can see that, by normalizing the data, the important information is lost and K-means fails. Hence, for the color and shape clustering problem, simple normalization cannot be satisfying and other methods need to be employed.

Weighted K-means ([Chen et al., 2009]) is based on the idea that information about the data can be captured before normalization and reinjected in the normalized dataset. In this way, the most relevant features for clustering can be enlarged and the others curtailed. More precisely, in a weighted K-means algorithm, weights are attributed to each feature, giving them different importance. Let us call w j the weight associated with the j th feature. Then, the norm used in the E-step of weighted Kmeans is

d(x i , c k ) = N j=1 w j (x ij -c kj ) 2 .
(2.5)

The distinction between different versions of weighted K-means algorithms lies in the choice of the weights.

A particular example of weighted K-means algorithm is weighted K-means based on coefficient of variation (CV K-means) ( [START_REF] Ren | K-means clustering algorithm based on coefficient of variation[END_REF]). It relies on the idea that the variance of a feature is an good indicator of its importance for clustering.

Hence, the CV weights are derived based on coefficient of variation, also called relative standard deviation:

w j = cv j N j =1 cv j
, where cv j = σ j µ j .

(2.6)

In the example of Figure 2.3, CV weighted K-means enables to overcome the problem involved by data normalization.

Exponentiated weighted K-means

In this chapter, we also propose an extension of weighted K-means that consists in raising the weights to the power of an integer p in the norm formula:

d(x i , c k ) = N j=1 w p j (x ij -c kj ) 2 .
(2.7)

By doing so, we emphasize even more the importance of features with large weights, which makes sense if the information captured by the weights is relevant.

In practice, as the weights are between 0 and 1, p should not be too large to avoid considering only one feature. In the rest of this chapter, CV p denotes exponentiated CV K-means with exponent p. The influence of p in the clustering results is studied in Section 2.4.3.

Levels of measurements

Using CV weights to solve the unsupervised robotics sorting problem using color and length features cannot be robust. The reason for this statement lies in the concept of levels of measurement ( [Stevens, 1946]). More specifically, it comes from the difference between ratio scale and interval scale.

A quantity measured on a ratio scale is one for which ratios are meaningful. For instance, lengths are measured on a ratio scale and one can assert that "10 cm is twice as large as 5 cm". A ratio scale possesses a unique and non-arbitrary zero value, and if a quantity gets further from this zero, the absolute precision to measure it decreases and thus the variance increases proportionally. Hence, coefficient of variation makes sense on a ratio scale.

On the other hand, interval scale is a measurement scale where all possible values are spread between two defined points, for instance the freezing and boiling point for temperatures in Celsius. From a precision perspective, measurement errors are not larger for high values than for low ones. On an interval scale, it is not relevant to use coefficient of variation because when the mean decreases, the variance does not change accordingly. Therefore, at equal variance, features closer to zero have higher coefficients of variation, which biases the clustering process. RGB color features, which are spread between 0 and 255, are interval variables. Hence, CV weights are not relevant for the problem studied in this chapter, which motivates the development of a new weighted K-means algorithm in the next section.

Gap Ratio K-means

For interval-type variables, ratios are not meaningful and one cannot use moments about the origin (e.g., the mean) as its choice is arbitrary. Hence CV weights are not appropriate for interval variables. However, ratios of differences are meaningful and standardized moments can be defined. Thus, we propose an approach that only considers relative values between the data points. Such choice leads to the definition of weights that are relevant for interval type data.

The Gap Ratio K-means algorithm

When doing clustering, we want to distinguish if different feature values between two objects come from noise or from the fact that objects are of different nature.

This observation is the main motivation behind the development of Gap Ratio Kmeans (GR K-means). If we consider that the distribution of a certain feature differs between classes, this feature's values should be more different between objects of different classes than between objects within a class. The goal of GR weights is to capture such information about the features.

To formulate the concept of Gap Ratio mathematically, the different values x ij for each feature are first sorted according to the j th component. For every j, we define the permutation σ j such that ∀i, i ∈ {1, ..., M }, x σ j (i),j ≤ x σ j (i ),j ⇔ σ j (i) ≤ σ j (i ).

(2.8) Then, we define the i th gap of the j th feature by:

g ij = x σ j (i+1),j -x σ j (i),j .
(2.9)

If there are M data objects, there are M -1 gaps for each feature.

After computing all the gaps for feature j, we define I j , the index corresponding to the biggest gap:

I j = argmax i∈{1,...,M -1} g ij .
(2.10)

Then, the biggest gap G j and the mean gap ḡj are defined as follows:

G j = g I j ,j , (2.11) ḡj = 1 M -2 M -1 i=1 i =I j g ij .
(2.12)

Finally, we define the Gap Ratio for the j th feature by:

gr j = G j ḡj .
(2.13)

In other words, for a given feature, the Gap Ratio is the ratio between the highest gap and the mean of all other gaps. Then the GRs are used to compute scaled weights:

w j = gr j N j =1
gr j .

(2.14)

The dissimilarity measure for GR K-means is obtained by using weights (2.14) in (2.5). Likewise, we call exponentiated GR K-means the algorithm using dissimilarity measure (2.7) with weights (2.14). Exponentiated GR K-means with exponent p is denoted GR p K-means.

Intuition behind GR K-means

For datasets that are noisy in dimensions that are not important, CV K-means fails because it focuses on these dimensions with high variance. This problem is illustrated on Figure 2.4a, which shows a 2-dimensional toy example where the variance is high in a dimension which is not important. On the other hand, weights and groups obtained with GR K-means indicate that the right information is stored in GR weights for such problem. The biggest gap along the y-axis is much bigger than average gaps whereas these two quantities are similar along the x-axis. In this case, using GR weights is more appropriate than CV weights.

CV weights also fail for interval variables with means close to zero. Indeed, because the variance does not decrease for low values on an interval scale, the coefficient of variation for these dimensions goes to very high values. This problem is illustrated in Figure 2.4b, where we can see that CV K-means completely fails while GR Kmeans is not affected and still succeeds to cluster the data correctly. This example is particularly relevant for the clustering case studied in this chapter. Indeed, RGB features are on an interval scale and data are noisy, which can lead to such kind of misclassifications.

Experimental validation

Experiment description

In this section, we validate the intuitive reasoning about GR K-means. To do so, we compare the different weighted K-means algorithms (including regular K-means, with weights w j = 1, for all j, and exponentiated weights) on a first implementation of unsupervised robotic sorting. This implementation consists in sorting a set of Lego bricks of different sizes and colors using a Kuka LBR iiwa collaborative robot equipped with a camera. The application is illustrated in Figure 2.1, which gives a link to a demonstration video.

The dataset used to compare different algorithms is one composed of nine Lego bricks of different sizes and colors, as shown in Figure 2.5. Each Lego brick is represented by its length and width (in cm), as well as its three RGB components (between 0 and 255). The natural classes in this dataset are straightforward to define, the clustering algorithm needs to place the big green, small green and small red bricks in different bins. Hence, the desired number of classes is set to three. Furthermore, on Figure 2.5, we can see that among the four pictures, lighting varies significantly and that data are very noisy. Color features observed are really different between two runs of the application. The algorithm needs to be robust to poor lighting conditions and to be able to distinguish between red and green even when colors tend to be saturated (see bottom right image). Feature extraction is implemented with open CV ( [START_REF] Itseez | Open source computer vision library[END_REF]), using background segmentation to obtain the contours of each object and pixels averaging for color features.

Weighted K-means implementation

To implement both GR K-means and CV K-means, we use the K-means implementation of the open-source library scikit-learn ( [START_REF] Pedregosa | Scikit-learn: Machine learning in Python[END_REF]). This way, our results can be checked and further improvements can be tested easily. To implement weighted K-means algorithms, we also use scikit-learn implementation but on a modified dataset. After normalization, the initial data are transformed using the following feature map:

Φ : x ij → √ ω j x ij .
(2.15)

As the dissimilarity computation appears only in the E-step, the dataset modifications are equivalent to changing the norm. Indeed, Equation (2.5) is equivalent to

d(x i , c k ) = N j=1 ( √ w j x ij - √ w j c kj ) 2 .
(2.16)

By doing this, results obtained can be compared more reliably. Differences in the results are less likely to come from poor implementation as the K-means implementation is always the same. Following these steps, implementation is straightforward.

Results

For our experimental comparison of the different algorithms, the experiment is repeated 98 times with different arrangements of the Lego bricks presented on Figure 2.5, and with different lighting conditions. For each trial, if the algorithm misclassifies one or more bricks, it is counted as a failure, else, it is a success. Figure 2.6 presents results obtained on the 98 trials for different weighted K-means algorithms.

The "Original dataset" contains the real measured data and the "Slightly modified dataset" is the same dataset with a slight color modification. To test the robustness of the algorithms, we removed 50 to the blue component of the bricks, which corresponds to using bricks of slightly different colors. The notation p * represents the optimal exponent for the weights for each algorithm (its values can be found in Section 2.4.4). All the results presented in this section are computed on the scaled datasets. Without scaling, success rates are very low (around 5%) because lengths (≈ 5cm) and colors (≈ 150) are of a totally different order of magnitude.

The first observation from these results is that regular K-means always results in very poor classification. One possible explanation for such bad behavior is the relatively high spread in the data (due to lighting conditions), which involve high variance in the features values and makes it difficult to differentiate between noise and true difference of nature. For this reason, emphasizing certain features is required, which justifies the use of weighted K-means. components too close to zero. On the other hand, GR K-means performs reasonably well (≈ 20% error rates), and is stable to dataset modifications.

Finally, looking at the optimal versions of the exponentiated weighted K-means algorithms, we see that exponentiated GR K-means can reach perfect classification on both datasets while CV weights cannot produce good classification on the modified dataset. A study of the exponents is proposed in the next section.

Influence of the weights exponents

Another way to determine the relevance of the information stored in the weights is to look at different values of the exponent for exponentiated weighted K-means. If the weights computed by a given algorithm are properly balanced, then increasing their importance by increasing p should improve the clustering result. On the other, if the clustering results decrease when p increases, then the weights are probably not very well balanced and too much importance is given to certain features. In this section, different values of p are tried to evaluate the "stability" of the weights found by both GR and CV K-means. Figure 2.7 represents the evolution of the clustering results

(averaged on the 98 runs) when the value of the weights exponent increases, for both the original and the modified datasets.

The first thing to be noticed from these plots is that the two curves for GR weights are superimposable. This shows that GR K-means algorithm is insensitive to average values of the interval scaled features. As for CV K-means, it performs good under certain conditions (Original dataset) but is not robust to decreasing the mean value of one feature (Modified dataset).

The left plot of Figure 2.7 shows another interesting thing. For low value of p, CV K-means performs better than GR K-means (0% error rate for p = 3). Even if the most important thing is to have one exponent for which error is low, it is interesting to note that high exponents generate poor clustering results with CV weights. Such behavior shows that the weights are not so relevant because if they are given too much importance, clustering gets worse. On the other hand, with exponentiated GR Kmeans error rate tends to decrease when the exponent increases. Information carried by GR weights is good for such clustering problem and should be given more importance. Error rate falls to zero at p = 9 and remains stable to exponent increases until relatively high values of p (> 20); the balance between important components is well respected within the weights. For this kind of datasets, characterized by large spread, mixed scales of measurement and relatively independent features, exponentiated GR K-means with relatively high exponent seems to be a good clustering method.

Extension to other data sets

On the one hand, Gap-Ratio weighted K-means was developed with a specific problem (URS from color and shape features) in mind. Hence, it is not surprising that it performs good on such datasets. On the other hand, it is also interesting to test this algorithm on other classification datasets of different nature to see how well it generalizes. Different weighted K-means methods are compared on two famous supervised learning datasets: the Fisher Iris dataset ( [Fisher, 1936]) and the Wine dataset, both taken from the UCI Machine Learning Repository ( [Lichman, 2013]). Table 2.1 gives some important characteristics of both datasets. Supervised dataset are chosen in order to have labels to evaluate the clustering output with external metrics ( [START_REF] Pfitzner | Characterization and evaluation of similarity measures for pairs of clusterings[END_REF]). Hence clustering results are evaluated using both cluster purity (PUR) and Normalized Mutual Information (NMI). These two metrics produce outputs between 0 and 1, with 1 representing perfect match between the cluster assignments and the true labels. For the Iris dataset, both GR and CV K-means implementations are better than regular K-means. Moreover, increasing the weights exponent improves the quality of the clustering. This means that both gap-ratio and coefficient of variation weights are able to capture the important information for clustering. However, for the Wine dataset, the best option is regular K-means, which might mean that the data dimensions are highly correlated and that one cannot isolate certain dimensions to improve clustering.

Conclusion

Key findings

In this chapter, we have introduced a first example of unsupervised robotic sorting consisting in clustering real objects from color and shape information. The datasets manipulated for this applications appeared to present various challenges for clustering.

Hence, to overcome the fact that the data are noisy and that some dimensions of the feature space are interval variables, we have proposed a new weighted K-means algorithm. This method, called Gap-Ratio K-means, leads to obtain better and more robust clustering results on a physical implementation of URS. Perspectives and future work regarding GR K-means are discussed in Chapter 9.

Limitations of hand-designed features for URS

Although it motivated the development of GR K-means, the data representation used in this chapter is very specific to certain kind of objects. Any classes of objects cannot be clustered properly from simple color, length and width features. For example, there is very little chances that different models of screw drivers would be clustered together using this kind of features (see Figure 1.2). Instead, such generic URS problem requires features with higher level of abstraction, able to understand the semantic nature of the objects at hand. The last two statements motivate the use of ensemble methods in the next chapter.

Introduction

From URS to Image Clustering

The Unsupervised Robotic Sorting (URS) problem studied in this thesis consists in physically sorting objects in an unsupervised way. The objective is to endow robots with the ability to sort and clean in a way similar to humans. Therefore, the data representation used in such an application should contain enough information to infer semantic content of the different objects and to produce a human-level unsupervised sorting. Stemming from the recent progresses in image classification [START_REF] Krizhevsky | Imagenet classification with deep convolutional neural networks[END_REF]), [START_REF] Simonyan | Very deep convolutional networks for large-scale image recognition[END_REF]), [START_REF] Szegedy | Rethinking the inception architecture for computer vision[END_REF]), [START_REF] He | Deep residual learning for image recognition[END_REF])), representing the various objects by images seem a smart choice for such clustering application. Then, the decision making module of URS boils down to an Image Clustering (IC) problem. Given a set of unlabeled images, the IC (or image-set clustering) problem consists in finding subsets of images based on their content: two images representing the same object should be clustered together and separated from images representing other objects. This part of the manuscript addresses the humanlevel image clustering problem, which expected outputs are illustrated in Figure 3.1.

The datasets presented in Figure 3.1, as well as all the other datasets studied in this chapter, are originally supervised image datasets. Although in all our experiments the labels are not used at clustering time, having supervision enables to characterize the "human-level" image clustering and gives a way to evaluate our algorithms.

Remark : This problem should not be confused with image segmentation ( [START_REF] Dhanachandra | A survey on image segmentation methods using clustering techniques[END_REF]), which is also sometimes called image clustering.

Previous work

Image clustering has received a lot of attention over the last two decades. It has applications for searching large image databases [START_REF] Flickner | Query by image and video content: The qbic system[END_REF]), [START_REF] Gong | Web scale photo hash clustering on a single machine[END_REF]), [START_REF] Avrithis | Web-scale image clustering revisited[END_REF])), concept discovery in images ( [START_REF] Lee | Shape discovery from unlabeled image collections[END_REF]), storyline reconstruction ( [START_REF] Kim | Joint summarization of large-scale collections of web images and videos for storyline reconstruction[END_REF]), medical images classification ( [START_REF] Wang | Unsupervised joint mining of deep features and image labels for large-scale radiology image categorization and scene recognition[END_REF]), etc. The first successful methods focused on feature selection and used sophisticated algorithms to deal with complex features. For instance, [START_REF] Goldberger | Unsupervised image-set clustering using an information theoretic framework[END_REF]) represents images by Gaussian Mixture Models fitted to the pixels and clusters the set using the Information Bottleneck (IB) method ( [START_REF] Tishby | The information bottleneck method[END_REF]). ( [START_REF] Seldin | Unsupervised clustering of images using their joint segmentation[END_REF]) uses features resulting from image joint segmentation and sequential IB for clustering. [START_REF] Fukui | Commonality preserving image-set clustering based on diverse density[END_REF] Recently, IC algorithms have shifted towards using features extracted from Convolutional Neural Networks (CNN) pretrained on ImageNet ( [START_REF] Russakovsky | Imagenet large scale visual recognition challenge[END_REF]).

( [START_REF] Liu | Infinite ensemble for image clustering[END_REF]) uses deep auto-encoders combined with ensemble clustering to generate feature representations suitable for clustering. [START_REF] Wang | Unsupervised joint mining of deep features and image labels for large-scale radiology image categorization and scene recognition[END_REF]) learns jointly the clusters and the representation using alternating optimization [START_REF] Bezdek | Some notes on alternating optimization[END_REF]). For complex IC problems, the two papers cited above, as well as [START_REF] Gong | Web scale photo hash clustering on a single machine[END_REF]) and [START_REF] Hu | Learning discrete representations via information maximizing self augmented training[END_REF]), use pretrained CNN feature extractors to generate a new data representation of the images before clustering.

Limitations

In recent research, using pretrained CNN features has enabled to obtain good clustering results on complex IC problems. However, there exists a variety of publicly available pretrained CNN architectures and, to the best of our knowledge, choosing which one to use has not been studied yet. Indeed, in all the literature mentioned in Section 3.1.2, the choice of the feature extractor architecture is never the same, and never justified. There might be several explanations for such absence of research in this direction. First, because IC is unsupervised, it is not possible to cross-validate design choices for a specific problem, making the CNN selection process very hard.

Moreover, using any CNN feature extractor usually results in a huge boost in performance compared to standard computer vision features. These excellent results might feel satisfactory enough and hide the fact that a good architecture choice might improve even more the clustering performance.

Contributions

In this chapter, we propose to study the use of deep pretrained CNN features for unsupervised classification tasks. By carrying out an extensive set of experiments over 8 datasets representing 4 different IC tasks (see Section 3.2.1), we investigate the interrelations between the different: 1. IC datasets; 2. CNN architectures; 3. feature extraction layers; and 4. kinds of clustering algorithms. This study is intended to provide better insight on the use of pretrained CNN features for unsupervised tasks.

More precisely, we want to know if • using different architectures, although pretrained on the same dataset (Im-ageNet), can change the clustering results,

• some layers in these networks extract features which are better suited for unsupervised classification,

• the feature representations from different CNNs combine better with a certain type of clustering algorithm.

Our experimental results reveal that overall, the last layer before softmax is better than all the other layers for feature extraction, and this is true for every architecture and dataset. The rest of our results can be summarized in one simple sentence: properly choosing the CNN architecture is important to have good clustering results but it is a hard task. This conclusion motivates the introduction of a new ensemble method to generate state-of-the-art results at IC in Chapter 4.

Experiments design

This chapter aims at answering several questions about the use of deep features for image clustering. We want to know if different CNN architectures, although pretrained on the same dataset (ImageNet), behave differently when presented an unsupervised dataset. We also want to know if a CNN architecture should be "cut" in the early or late layers for feature extraction. The ideal would be to come up with generic rules such as: "when facing a particular dataset DS, and to optimize a

given metric M, one should choose the architecture NN, extract features from layer L and cluster the new feature set with algorithm C". To do so, we implement the straightforward pipeline presented in Figure 3.2 for several datasets. The choices made for studying the different elements in the pipeline in Figure 3.2 are described in details in the coming sections.

Datasets

To obtain results about CNN feature extractors which are generalizable, experiments need to be carried out on many datasets, belonging to different subtasks of IC. Hence, the proposed pipeline, with different feature extractors and clustering algorithms, is applied to the unsupervised versions of the following tasks:

• Natural object recognition: We call natural object recognition the task of classifying images based on a single object it contains. Classes are defined in the most generic way possible (cat, dog, car, etc.). This task is the most similar to ImageNet. Hence, although the precise task (categories) and domain (backgrounds) are different, pretrained deep features are expected to generate good clustering results on this task. Moreover, we also expect that final layers will be better suited for this task because this is the kind of objects they are separating for ImageNet classification.

• Scene recognition: This task is different from what the pretrained network was trained to do. Indeed, in scene recognition, a category is defined by the simultaneous presence of multiple objects on the image. For example, a dining room needs to contain chairs and a table on the image (Figure 3.1b). We still expect the last layers to perform better at this task as they are supposed to contain higher level information.

• Fine-grained recognition: This task is also very challenging for pretrained deep features because classes are defined within what usually defines a single category for ImageNet. For example, a fine-grained recognition task might consist in recognizing different species of birds. It is difficult because the pretrained network might have learned to produce features which are too generic for this task without additional supervision.

• Face recognition: This task is also a fine-grained recognition task, however, it is of such importance today that we study it as its own class of problems.

For each unsupervised classification task listed above, we pick two datasets: one small and one large. The datasets studied, together with their statistics, are listed in Table 3.1.

Hence, we study eight datasets in total. More details on these datasets can be found on the papers in which they were introduced: VOC2007 ( [START_REF] Everingham | The pascal visual object classes challenge 2007[END_REF]), COIL100 [START_REF] Nayar | Columbia object image library (coil 100)[END_REF]), Archi [START_REF] Xu | Architectural style classification using multinomial latent logistic regression[END_REF]), MIT ([Quattoni and Torralba, 2009]), Flowers [START_REF] Nilsback | Delving deeper into the whorl of flower segmentation[END_REF]), Birds [START_REF] Welinder | Caltech-UCSD Birds 200[END_REF]), UMist [START_REF] Wechsler | Face recognition: From theory to applications[END_REF]), FEI [START_REF] Thomaz | A new ranking method for principal components analysis and its application to face image analysis[END_REF]).

Architectures

To ease and speed up development, we compare the Keras ( [Chollet, 2015]) implementations of five popular CNN architectures:

• Two VGG architectures: VGG16 & VGG19 ( [START_REF] Simonyan | Very deep convolutional networks for large-scale image recognition[END_REF]),

• One ResNet architecture: ResNet50 ( [START_REF] He | Deep residual learning for image recognition[END_REF]),

• Two Inception-like architectures: InceptionV3 ( [START_REF] Szegedy | Rethinking the inception architecture for computer vision[END_REF]), Xception ( [Chollet, 2016]).

We also use the ImageNet pretrained weights provided by Keras.

As of today, ImageNet is the only very large labelled public dataset which has enough variety in its classes to be a good feature extractor for a variety of tasks. Moreover, there are plenty of CNNs pretrained on ImageNet already available online. For these reasons, we use CNNs pretrained on ImageNet. However, the results presented here would probably apply to other databases, when larger and more diverse datasets will be created.

Layers

The IC problem studied in this thesis consists in discovering classes represented by "objects" present on the pictures. Thus, the feature extractor needs to gather semantic level information about the data to make such clustering possible. Such high level information is present in the final layers of the pretrained networks. Thus, to study the impact of the layer chosen, we pick three layers among the last ones for each network :

• One layer in the end of the convolutional block (L1),

• The second layer before the ImageNet softmax layer (L2),

• The last layer before softmax (L3).

Picking layers from earlier stages of the network is both not very relevant and not practical. Indeed, the closer to the beginning of the network the layer is, the bigger the feature space is and the longer the clustering is. It is probably not relevant because the features are too low level to be informative without additional supervision.

On the one hand, the last one or two layers might provide better results as their goal is to separate the data (at least for the fully-connected layers). On the other hand, the opposite intuition is also relevant as we can imagine that these layers are too specialized to be transferable. The names (as given in the keras implementation) of the layers retained for this study are reported in Table 3.2. In this chapter, the goal is not to find a good algorithm to solve a specific problem but to study the influence of the chosen CNN feature extractor (architecture + layer) on the IC results. Hence, in this chapter, we only consider standard clustering algorithms in order to isolate the influence of the features. For now, we leave aside the most recent deep methods, which are state-of-the-art at clustering. They will be the purpose of the next chapter. To keep our experiments simple and interpretable, we pick the most widely used algorithms from each of these two main families of algorithms:

• K-means (KM) ( [START_REF] Arthur | k-means++: The advantages of careful seeding[END_REF]),

• Agglomerative Hierarchical Clustering (Agg) ( [Murtagh, 1983]).

For both algorithms, we use the default configuration of the scikit-learn implementations ( [START_REF] Pedregosa | Scikit-learn: Machine learning in Python[END_REF]). This avoids fine-tuning our clustering algorithms specifically for deep pretrained features.

There exist a variety of other simple and very popular clustering methods (( [START_REF] Xu | Survey of clustering algorithms[END_REF]), ( [Berkhin, 2006])). The ones available in scikit-learn have been tried ([Guérin et al., 2018c]) and did not present a major interest with respect to the conclusions drawn from these experiments. However, keeping connectivity based and graph based algorithms enable us to analyze if different architectures represent data differently.

Metrics

Although we do not use the labels, our experiments are carried out on datasets that are inherently supervised. Hence, we can use external validation metrics [START_REF] Pfitzner | Characterization and evaluation of similarity measures for pairs of clusterings[END_REF]) to evaluate the quality of the clustering for the different combinations. Thus, we use two popular external clustering metrics in our study.

• Normalized Mutual Information (NMI), which is a metric based on information theory and defined as:

N M I(Y, C) = 2 × I(Y, C) H(Y ) + H(C) , (3.1)
where Y is the list of ground truth labels, C the cluster assignments, H(.) represents the entropy and I(Y, C) the mutual information between Y and C.

NMI ranges between 0 and 1, with 1 representing perfect accuracy.

• Cluster purity (PUR) is defined by

P U R = 1 N c∈C max y∈Y | c ∩ y |, (3.2)
were N is the number of elements in the dataset. Purity measures how much each cluster contains a single class, it also varies between 0 and 1, and a good algorithm has a purity close to 1.

Results

This We also represent the different NMI scores in the form of histograms in Appendix C to visualize better the influence of the features on the clustering results. The body of the chapter only presents a summary of these results in order to highlight the important information.

For completeness, Appendix B also includes results using bag of sifts features (BoF) representations ( [START_REF] Fukui | Commonality preserving image-set clustering based on diverse density[END_REF]). This enables to compare CNN features with standard computer vision features and we can see that although BoF features produce decent purity scores, the NMI scores are much below deep features.

We note that BoF results only appear for the smallest dataset of each task because BoFs are expensive to compute and of limited interest for our study.

To evaluate the influence of specific components of the clustering pipeline, we consider our experiments as a 4-dimensional design space which dimensions are: architecture, layer, task, clustering algorithm. The correlation between two factors is then studied by computing the mean and standard deviation (std) over all the results containing them. These two statistics enable us to evaluate both the overall performance and the stability of a combination. Although our experiments are relatively small to draw general conclusion, they enable to give a general trend.

Influence of the layer

This results section begins by studying the impact of the choice of the layer on the clustering results. We want to know how different architectures perform under different layers. The relation between the clustering task and the position of the layer in the network is also studied. The relation results under the different architecture-layer pairs can be visualized in Figure 3.3a. For all architecture, NMI scores for later layers have higher mean and lower standard deviation. This reveals that, for our experiments, final layers perform better overall and are more consistent. The high standard deviation of earlier layers shows that in some cases, features extracted from early layers can perform well, however, there is more variability and the results can drop to much lower scores in other cases. Such statement can be analyzed in more detailed looking at the plots in Appendix C. For example, for face recognition, some L1 layers present slightly better results than other layers, on the other hand, for fine-grained recognition, choosing L1 can results in NMI scores lower than L3 by about 0.35.

The influence of the layer on different image clustering tasks is represented in Figure 3.3b. Before conducting these experiments, our intuition was that early layers would be better suited for face recognition and fine-grained classification tasks while late layers would be better at natural object and scene recognition. Indeed, high level information about an image are contained in the last layers while early layers represent lower level information (Gabor filters, color blobs, etc.) ( [START_REF] Yosinski | How transferable are features in deep neural networks? In Advances in neural information processing systems[END_REF]). However, our experimental results show that whatever the task is, later layers perform better. This effect is damped for faces but it is still true. Moreover, for all tasks, std is higher for early layers.

These results suggest that only the last layer before softmax (L3) should be considered for clustering. Although in some cases other layers slightly outperform L3 (e.g., L2 for Xception on FEI), the profit is small and the risk is high (high std). Such finding implies that some "low-level" information is contained in the last layers of deep CNNs pretrained on ImageNet. This may be explained by the presence of fine-grained recognition classes in the ImageNet dataset (e.g., different breeds of dogs). Hence, only L3 layers are considered in the rest of this results section. Dropping the first layers is also motivated by the fact that their feature spaces are of higher dimensions, which means higher clustering time. For example, on average, clustering L1 layers take about one hour while L3 layers only take one minute. This difference is proportional to the size of the dataset.

Influence of the architecture

The next analyses focus on the choice of the CNN architecture. These results can be found in Figure 3.4. We want to know if an architecture is better suited for clustering than the others in general (Figure 3.4a) and depending on the task at hand (Figure 3.4b).

Besides the fact that, in our experiments, Xception presents slightly better results than the other architectures (higher mean and lower std), it is difficult to come up with relevant comments about these histograms. The results for each subtask contain too much variability, which prevents any kind of conclusion like: "for task T, use architecture A". However, we underline that an absence of strong pattern does not mean that any choice is equivalent. Indeed, there can be a huge difference in the results between a good and a bad architecture choice (Figure 3.5). Such absence of trend, together with the criticality of this choice, motivates the development of a new IC algorithm in Chapter 4, which leverages ensemble methods to remove the need for architecture selection.

Influence of the clustering algorithm

For completeness, correlations between the architecture used and the type of clustering algorithm are also checked. This analysis is intended to investigate if the features extracted by different architecture are better suited for partitioning or graph-based

V G G 1 6 V G G 1 9 I n c e p t i

Conclusion

Key findings

Using pretrained CNN architecture to extract features is now a common practice to solve image clustering. However, the choice of the architecture and layer for feature extraction is often arbitrary. In this chapter, we conducted extended experiments on 8 standard computer vision datasets from four different IC subtasks to investigate the behavior of these features. Our first key finding is that for all architectures and tasks, the last layer before softmax seems to produce the most discriminative features for clustering. Our experiments also demonstrate that the selected deep feature extractor has a major impact on the results, however, they do not give any insight about how to select it.

Towards ensemble methods for feature extraction

The initial purpose of this benchmark study was to lead to a set of simple rules to help designing better IC algorithms by optimizing feature extraction. For example, we expected that for fine-grained clustering, earlier layers in the network might perform better as they are supposed to contain lower level information. We also expected to find some clear correlations between the tasks to solve and the architecture to use. However, the experiments carried out in this chapter rather demonstrate that such simple rules do not exist. In addition, when facing an IC problem, the feature extractor selection process cannot be cross-validated because of the absence of labels (unsupervised task). The importance of architecture selection, together with the absence of method to solve this problem, motivates the introduction of a new ensemble method to generate state-of-the-art results at IC in Chapter 4.

Chapter 4

Improving Image Clustering using Multiple Pretrained CNN Feature Extractors Abstract

To improve image clustering results, the current practice consists in replacing raw image data with features extracted by a pretrained convolutional neural network (CNN). However, in the previous chapter, we have shown that the specific features extracted, and, by extension, the selected CNN architecture, can have a major impact on the clustering results. In addition, we have seen that this crucial design choice is hard and is often decided arbitrarily due to the impossibility of using cross-validation with unsupervised learning problems. However, information contained in the different CNN architectures may be complementary, even when pretrained on the same data. In this chapter, we propose to improve clustering performance, by rephrasing the image clustering problem as a multi-view clustering (MVC) problem that considers multiple different pretrained feature extractors as different "views" of the same data. We then propose a multi-input neural network architecture that is trained endto-end to solve the MVC problem effectively. Our experimental results, conducted on three different natural image datasets, show that: 1. using multiple pretrained CNNs jointly as feature extractors improves image clustering; 2. using an end-to-end approach improves MVC; and 3. combining both produces state-of-the-art results for the problem of image clustering.

Introduction

Overview

Image Clustering (IC) is a major research topic in machine learning which attempts to partition unlabeled images based on their content. In this chapter, we still consider the IC setting where the number of clusters is a user defined parameter. As explained earlier, research in IC has recently shifted towards using features extracted from Convolutional Neural Networks (CNN) pretrained on ImageNet, which has lead to substantial improvements on complex IC datasets. In the previous chapter, it was shown that choosing a proper architecture is a very hard task when designing an image clustering pipeline. This difficulty, together with the impossibility to cross-validate algorithmic choices for unsupervised tasks, lead to pretrained feature extractor choices which are often arbitrary [START_REF] Liu | Infinite ensemble for image clustering[END_REF]), [START_REF] Wang | Unsupervised joint mining of deep features and image labels for large-scale radiology image categorization and scene recognition[END_REF]), [START_REF] Gong | Web scale photo hash clustering on a single machine[END_REF]), [START_REF] Hu | Learning discrete representations via information maximizing self augmented training[END_REF])). This is potentially problematic as we also showed in Chapter 3 that the choice of the architecture used for feature extraction has a major impact on the clustering results.

In this chapter, we aim to remove the need for this design choice. Following the intuition that different pretrained deep networks may contain complementary information (see Section 4.2.2), we propose to use multiple pretrained networks to generate multiple feature representations. Such representations are treated as different "views" of the data, thus casting the initial IC problem into Multi-View Clustering (MVC).

The success of ensemble methods for clustering [START_REF] Vega-Pons | A survey of clustering ensemble algorithms[END_REF]) suggests that such an approach can improve overall clustering results.

Finally, building on the recent success of end-to-end clustering methods [START_REF] Aljalbout | Clustering with deep learning: Taxonomy and new methods[END_REF]), we also propose to leverage JULE ( [START_REF] Yang | Joint unsupervised learning of deep representations and image clusters[END_REF]), a deep clustering method, to solve the MVC problem. By adapting JULE to optimize the weights of a parallel neural network architecture we demonstrate state-of-the-art IC results on several public image datasets. This approach to MVC also has the advantage of producing a unified representation of the initial dataset which is lowdimensional and compact. An overview of the proposed method to solve IC can be seen in Figure 4.1.

Contributions

We propose to transform the IC problem into MVC by extracting features from several different pretrained CNNs. This removes the crucial design choice of feature extractor selection. We also propose to adapt a deep clustering approach to address MVC. Our experimental results, carried out across 8 standard computer vision datasets, suggest that:

F z . . . Z 1 Z 2 Z M Multi-view generator = = = Multi-layer perceptrons Φ . . . X 1 X 2 X M Z 1 Z 2 Z M + Concatenate ϕ out
• Image clustering can be improved by using features extracted from several pretrained CNN architectures, eliminating the need to select one.

• Multi-view clustering can be improved by adopting an end-to-end clustering approach.

• These two ideas can be combined to obtain state-of-the-art results at image clustering on several standard IC datasets.

• This methodology produces a unified compact low-dimensional representation of the original dataset. [START_REF] Kumar | [END_REF]), the authors propose different loss functions applied on the concatenated views, in [START_REF] Zhao | Multi-view clustering via deep matrix factorization[END_REF]) and ( [START_REF] Wang | Iterative views agreement: An iterative low-rank based structured optimization method to multi-view spectral clustering[END_REF]) lower dimensional subspaces are learned before clustering with standard methods.

MVC and EC are closely related and have already been combined in previous work.

In [START_REF] Tao | From ensemble clustering to multi-view clustering[END_REF]), good MVC results are attained by embedding MVC within the EC framework. The authors leveraged the different views to generate independent partitions and then used a co-association-based method to obtain the consensus. In both [START_REF] Gao | Multi-view subspace clustering[END_REF]) and [START_REF] Ceci | Semi-supervised multi-view learning for gene network reconstruction[END_REF]), generation mechanisms borrowed from EC are used to generate artificial multiple views of the data. In this chapter, we propose to use multiple pretrained CNNs to generate different feature representations of an image dataset. Hence, we generate a MVC problem from an ensemble of pretrained CNN feature extractors.

Why combining CNN architectures may succeed?

Combining CNN architectures that were pretrained on the same dataset might seem counter-intuitive as one can expect that all networks have learned the same information. This section aims at explaining the intuition behind trying this idea. This intuition is then validated experimentally in the rest of this chapter. Let I = {0, ...255} ν 1 ×ν 2 ×3 be the space of ν 1 by ν 2 colored images considered for IC. Then, a classification task T=(L, f * ) is defined by:

• A set of possible labels L = {0, 1, ...K}, where 0 represents "none of the defined labels".

• An oracle labelling function f * : I → L, which associates a label to every image.

For example, for the task of classifying images of cats and dogs, L would be {0, 1, 2}, and for an image x, the oracle f * (x) would output 1 if there is a cat on the image, 2 if there is a dog and 0 if there is either none or both. This definition of a classification task is valid for supervised classification or unsupervised classification with known number of classes, which is the case studied in this chapter. Although in practice we only study datasets composed of images which possess at least one of the labels, adding the "zero" label, allows us to define T on all I.

This definition of a classification task is abstract and f * is unknown and exists independently of any dataset. In practice, to solve T, one first need to materialize it in the form of a dataset DS = (X, y * ), where X ⊂ I is a set of images and y * = f * (X) are the corresponding labels in L, which are inferred by human experts. For certain problems, such as medical image annotation, human experts might be scarce, thus making labelling very costly. The classification problem (T, DS) is supervised if y * is known and unsupervised else. In the supervised setting, solving T for DS means finding a function f : I → L for which there exists a domain on which it is equal to

f * : D f = {x ∈ I | f (x) = f * (x)}.
We call D f the domain of validity of f . Then, we have:

• D f ⊂ X ⇒ f does not fit the training set, • D f = X ⇒ f overfits the dataset DS, • D f ⊃ X ⇒ f generalizes to some extent.
For many image classification problems, it is very hard to learn f from scratch.

Instead, it is more common to use a pretrained CNN feature extractor f z to project the initial dataset X to a latent feature space of lower dimension d:

f z : I → R d X → Z. (4.1)
From now on, X will denote a clustering dataset. Now, let A be a clustering algorithm. Unlike in the supervised case, a clustering algorithm solves an unsupervised classification problem by looking at the whole set at once. In other words, if X contains N data points and A is applied to the outputs of f z , we have:

A : (R d ) N → L N Z → y, (4.2)
where y are the cluster assignments produced by A. From these definitions, we can introduce D T fz,A ⊂ I, the domain on which applying classification algorithm A to the outputs of f z enables to solve task T . In the previous chapter, our experimental results suggested that, for a given dataset, the ranking of the different feature extractors is little dependent on the chosen clustering algorithm. Hence, the subscript A is dropped to define D T fz , the domain on which f z produces a "clustering friendly" latent space for task T.

Finally, let f 1 z and f 2 z be two pretrained CNN feature extractors, and let T be the task that we aim to solve on the clustering dataset X. In the previous chapter, it was shown that f 1 z and f 2 z perform differently on the different datasets. For (T, X), let's assume that, according to some clustering validation metric (e.g. NMI), f 1 z outperforms f 2 z . Then, we can conclude that either

• D T f 2 z D T f 1 z (Figure 4.2a) or, • ∀j ∈ {1, 2}, D T f j z (D T f 1 z ∪ D T f 2 z ) (Figure 4.2b),
where the symbol represents strict inclusions.

D T f 1 z D T f 2 z D X (a) Case 1: D T f 2 z D T f 1 z D T f 1 z D T f 2 z D X (b) Case 2: ∀j ∈ {1, 2}, D T f j z (D T f 1 z ∪ D T f 2 z ) Figure 4.2: Schematic representation of the two possible implications of f 1 z being better than f 2 z .
In both situation, leveraging both networks can have positive implications on the clustering results:

• Case 1: f 1
z alone contains all the information to cluster X. However, in Chapter 3, we showed that for unsupervised datasets, it is not possible to know which network performs best. Hence, using the two networks allows to make sure that all the information available for clustering are provided to the final clustering algorithm.

• Case 2: The combination of the two networks contains more information than each of the networks separately. In other words, even if f 2 z performs worse than f 1 z , it still contains information that f 1 z does not. This kind of situation defines a typical setting where ensemble learning would be beneficial.

The domain of X, called D X , is represented in green on both sketches of Figure 4.2 and illustrates the two potential benefits of combining CNN feature extractors. Obviously, this intuition can be generalized to more than two networks. Although all this development is just an intuition and does not have theoretical evidences, we intend to validate it experimentally in the rest of the chapter.

The potential improvement from using multiple pretrained CNN feature extractors can also be understood through the following contrived example. To recognize a car, one network might learn a wheel detector while another one might detect wing mirrors. Both sets of discriminative features would enable to solve the ImageNet classification task, on which both networks were trained, but would also carry very different information that might be useful in solving a new IC task.

Finally, it is also important to note that the opposite intuition may also be valid.

Indeed, introducing redundancy of information might hide the important information and decrease the clustering results. This will be investigated in the rest of the chapter.

Visualisation

To visualize this intuition on real data, we leverage the Fowlkes-Mallows Index (FMI)

( [START_REF] Fowlkes | A method for comparing two hierarchical clusterings[END_REF]), another external clustering validation metric, which has the advantage of having a local form. For a dataset (X, y * ) and cluster assignments y, which associates a predicted label y i to every point x i in X, we note FM i the local Fowlkes-Mallows score of datapoint x i . FM i ranges between 0 and 1 and is high if x i is well clustered with respect to y * . The FMI is better explained in Chapter 6 (Section 6.4.1). From this definition of FM i , we introduce the concept of FM score per class:

F M C k = 1 N C k N C k p=1 F M p , (4.3)
where

C k represents true class k and N C k is the number of elements of C k in the dataset.
Then, we demonstrate the complementarity of deep feature extractors by carrying out experiments on the UMist dataset (see Section 3.2.1). We apply agglomerative clustering to the best performing network on this dataset (InceptionResnet) as well as the two worst performing networks (VGG16 and Densenet121). These networks are introduced in Section 4.2.4. Then, the NMI, purity, FM and FM C 4 scores are computed. As shown in the

IC problem reformulation

Let X = {x 1 , ..., x N } ⊂ I be an unlabeled set of N natural images, and let F z = {f 1 z , ...f M z } be a set of M feature extractors. In theory, F z can be composed of any function mapping raw pixel representations to lower-dimensional vectors, but in practice, we use pretrained deep CNNs. The first step in our approach is to generate a set of feature vectors from each element of F z . For all i ∈ [1, ...M ], we denote Z j the matrix of features representing X such that, its row Z j i is the feature vector representing x i and extracted by f j z :

Z j i = f j z (x i ). (4.4)
In other words, Z = {Z 1 , ..., Z M } can be interpreted as a set of views representing X.

Thus, Z is a multiview dataset representing X and the problem of clustering Z is a MVC problem, which can be solved using any MVC algorithm ( [START_REF] Chao | A survey on multi-view clustering[END_REF]).

A visual representation of the multiview generation mechanism can be seen in the blue frame of Figure 4.1, which is repeated in Figure 4.4 to ease readability.

Unsupervised image set

X

Deep feature extractors

F z . . . Z 1 Z 2 Z M Figure 4
.4: Proposed approach to generate multiple views from a unique unsupervised set images.

Remark: All along this chapter, we use letter i for indexing across data samples, letter j for indexing across feature extractors and letter k for indexing across clusters.

Similarly, N , M and K respectively stand for the number of data samples, feature extractors and clusters.

Experimental evidences

To conclude this section on artificial multiview data generation, we give first experimental evidences and study the optimal number of networks to use. To do so, experiments are carried out on 4 of the standard datasets introduced in Chapter 3: VOC2007, Archi, Flowers and UMist. In these experiments, as well as in the rest of the chapter, we use 10 pretrained architectures: VGG16, VGG19, Inception, Xception, Resnet50, Densenet121, Densenet169, Densenet201 ( [START_REF] Huang | Densely connected convolutional networks[END_REF]), Nasnet ( [START_REF] Zoph | G.2 Partitionnement de données à partir de caractéristiques de couleur et de forme[END_REF]) and InceptionResnet ( [START_REF] Szegedy | Inception-v4, inception-resnet and the impact of residual connections on learning[END_REF]). In this section, MVC problems are solved using the Multi-View Ensemble Clustering method ( [START_REF] Tao | From ensemble clustering to multi-view clustering[END_REF]) with agglomerative clustering (MVEC agg ). This method consists in clustering each view separately using agglomerative clustering and generating a consensus partition using an ensemble clustering method based on co-association matrix.

This method is better detailed in Chapter 5. We choose agglomerative clustering as our base algorithm because its simplicity enables us to study straightforwardly the clusterability of the different feature spaces. Agglomerative clustering is also preferred over K-means because it does not depend on random initialization and thus avoids having random effects in our results.

Then, to study the clustering quality of multiview data from m (≤ 10) CNNs, we generate all the multiview problems from all the possible combinations among the different architectures. Each of these problems is then solved with MVEC agg and the NMI scores are computed. In C 10 m = 1023 MVC problems. These results suggest that combining more networks both increases the clustering accuracy on average and decreases the variability, which can be seen as the risk to obtain poor results. These results are in line with the intuitions from Section 4.2.2. From these simple experiments, we decide to use all ten networks in the rest of the chapter, which is likely to give the most robust clustering results. It is also interesting to note that the two networks case performs worst than the single network case for all four datasets. This probably comes from the absence of a "majority" to distinguish which information is relevant. can be run in parallel on a dedicated GPU. Thus, for a given dataset, if we note • t j 1 the time for feature extraction with f j z ,

• t j 2 the time for running agglomerative clustering on Z j and

• t 3 the time for merging the different partitions into a consensus partition, the total clustering time is max j (t j 1 + t j 2 ) + t 3 . For this reason, the time for running MVEC agg does not increase linearly with the number of feature extractors. Using 10 networks instead of one only increases the clustering time by a factor of around 2.5, As illustrated on Figure 4.5. The increase in the total time comes from the fact that the slowest networks are selected more often when more networks are used and that the partitions grouping becomes slower. In practice the slowest step is feature extraction. If only m (< m) GPUs are available, then the total time increases by a factor of ∼ m m , where . denotes the ceiling function. Making the method scalable to a large number of CNNs when few GPUs are available is a potential area of improvement for the method.

Deep Multi-View Clustering

Preliminaries: Deep end-to-end clustering

In this chapter, we also propose to leverage recent end-to-end deep clustering methods to solve the MVC problem. End-to-end clustering methods based on neural networks have produced excellent results over the past two years. A complete literature review of the topic is outside the scope of this paper, for that we refer the reader to the following recent survey ( [START_REF] Aljalbout | Clustering with deep learning: Taxonomy and new methods[END_REF]). However, to better understand how deep clustering works we describe the first successful method, Deep Embedding for Clustering analysis (DEC) ( [START_REF] Xie | Unsupervised deep embedding for clustering analysis[END_REF]), which is a centroid-based clustering method. This approach begins by pretraining a multi-layer perceptron (MLP) using an auto-encoder input reconstruction loss function. Then, the MLP is finetuned to output a set of cluster centers which define cluster assignments. The joint optimization of the network and the centroids are based on the minimization of the Kullback-Leibler (KL) divergence between the current distribution of the features and an auxiliary target distribution, derived from high-confidence predictions. Improved Deep Embedding Clustering (IDEC) is an improved version of DEC introduced in ([ [START_REF] Guo | Improved deep embedded clustering with local structure preservation[END_REF]). It modifies DEC by replacing the loss function by a combined loss, taking into account the auto-encoder reconstruction loss during fine tuning. Such an approach preserves the local structure of the data and appears to improve the DEC clustering results.

Deep multi-view clustering (DMVC)

In this section, we define our approach for solving MVC. Let C ee be any deep endto-end clustering framework. C ee is defined by a loss function L and a procedure P to optimize the loss function. Multiple approaches have already been adopted to define the clustering-oriented loss L and the optimization procedure P. Two examples, IDEC and JULE, are discussed respectively in Section 4.3.1 and 4.3.3.1.

To apply C ee to an unsupervised dataset X, one first needs to specify a neural network architecture ϕ θ , parameterized by θ, which projects X into a lower dimensional feature space: Z θ = ϕ θ ( X). Then, P is applied to minimize L(θ, X), producing both a good representation Z θ final and a set of cluster assignments y final .

Remark: The proposed solution to Image clustering (IC) is in two steps: feature extraction and deep clustering. The "tilde" notation is introduced to avoid confusion between the different input spaces and latent spaces. Hence, X and Z refer respectively to the input and latent spaces of the deep feature extractor f z , which weights are fixed, whereas X and Z refer to the deep clustering network ϕ θ , parameterized by θ.

Obviously, if features from a single feature extractor are used to train a single deep clustering network, we have Z = X.

The choice of the architecture of ϕ θ usually depends on the kind of dataset to solve. For example, when dealing with large images, ϕ θ can be a CNN and when X is composed of smaller vectors, ϕ θ can be a multi-layer perceptron (MLP). In the case of MVC, each element of X is a collection of vectors. For example, the i th element of X is written as

X i = {Z j i ; ∀j ∈ [1, ...M ]}.
For this reason, to embed MVC into a deep clustering framework, we need to define a different neural network architecture for

ϕ θ , which we call MVnet. MVnet consists of a set of M independent MLPs, denoted Φ = {ϕ θ 1 , ...ϕ θ M }, such that, ∀j ∈ [1, ...M ],
the dimension of the input layer of ϕ θ j is equal to the dimension of the output layer of the associated f j z . We also define ϕ θout , another MLP with input layer dimension equal to the sum of the dimensions of the output layers over the elements of Φ. Thus, an MVnet is composed of three layers: a parallel layer containing all the elements of Φ, followed by a concatenating layer which feeds into ϕ θout . A visual representation of the MVnet architecture can be seen in Figure 4.6, which is a duplicate of the red box in Figure 4.1. We note that all the elements of Φ are independent and do not share any weights. DMVC is a generic framework and MVnet can be optimized using most deep end-to-end clustering approaches. In practice, we have tried to implement DMVC within both the IDEC and the JULE ( [START_REF] Yang | Joint unsupervised learning of deep representations and image clusters[END_REF]) frameworks. After carrying out some simple experiments on standard datasets, we noticed that JULE performs significantly better. In addition, implementing IDEC on a new dataset requires additional parameters tuning to pretrain the autoencoders, which is time consuming, potentially error-prone and less generic. For these two reasons, in the rest of this chapter, we adopt JULE to solve the DMVC problem. The proposed JULE-DMVC implementation is explained in more details in the next section.

Multi-layer perceptrons

Φ . . . X 1 X 2 X M Z 1 Z 2 Z M + Concatenate ϕ out Z out y Clusters

DMVC with JULE

Joint Unsupervised Learning of Deep Representations and Image Clusters, or JULE, is an iterative end-to-end clustering process that has demonstrated excellent experimental results on several natural image datasets. In this section, we propose to leverage JULE to train an MVnet to solve the MVC problem. We start by explaining the standard JULE framework and then propose an extention to adapt it to MVnet.

We note that Section 4.3.3.1 is a summary of JULE, for a more complete description, we refer the reader to the original paper [START_REF] Yang | Joint unsupervised learning of deep representations and image clusters[END_REF]).

Joint Unsupervised Learning of Deep Representations and Image Clusters

Notations and Definitions

Let X be an unsupervised dataset containing N samples that we aim to cluster into K * groups, where the target number of clusters K * ∈ N, is a user defined parameter. Let ϕ θ denote a neural network parameterized by θ, which produces a lower dimensional representation of the initial dataset Z θ = ϕ θ ( X). JULE is an iterative process, hence we introduce θ[t] and y[t] to talk about the values of the weights and cluster assignments at iteration t. At step t, the cluster assignment y[t] defines a set of

K[t] ≤ K * clusters: {C k [t], k ∈ {1, ..., K[t]}}. Likewise, θ[t] defines a latent space Z θ[t] = { z i [t] = ϕ θ[t] ( x i ), x i ∈ X}.
Then, we define N κ i ⊂ Z θ , the set of the κ nearest neighbors of z i . This definition is used to introduce the matrix W , which defines the similarity between data samples:

W (i 1 , i 2 ) = exp(- || z i 1 -z i 2 || 2 2 σ 2 ), if z i 2 ∈ N κ i 1 0, else. (4.5) In this equation, σ 2 = 1 N ×κ z i 1 ∈ Z θ z i 2 ∈N κ i 1 || z i 1 -z i 2 || 2
2 is used to normalize W across all the data and κ is a user defined parameter. In practice, we follow the recommendations of the authors of the original paper ( [START_REF] Yang | Joint unsupervised learning of deep representations and image clusters[END_REF]) and use κ = 20. The definition of similarity in Equation 4.5, which can be found in ( [START_REF] Zhang | Graph degree linkage: Agglomerative clustering on a directed graph[END_REF]), considers that a sample is similar to another sample only if it belongs to its neighborhood. If this condition is verified, the similarity varies inversely to the distance between these points. We note that W is not symmetric.

Finally, W is used to define the notion of clusters affinity which represents the similarity between two clusters C k 1 and C k 2 : .

A(C k 1 , C k 2 ) = 1 |C k 1 | 2 1 T |C k 1 | W C k 1 ,C k 2 W C k 2 ,C k 1 1 T |C k 2 | + 1 |C k 2 | 2 1 T |C k 2 | W C k 2 ,C k 1 W C k 1 ,C k 2 1 T |C k 1 | . (4.6) In Equation 4.6, |C k | is the number of samples in cluster C k , 1 |C k | is a vector of length |C k | composed
These two steps are explained in more details in the rest of this section. Because clusters are being merged, the total number of clusters decreases when we progress through the optimization:

t < t ⇒ K[t] < K[t ]. JULE stops at iteration t f such that K[t f ] = K * . As a final step, the network ϕ θ[t f ] can optionally be trained on y[t f ]
to fine-tune the representation. All the steps composing JULE can be visualized on a two dimensional toy example in Appendix D.

Initialization

The initial set of clusters y[t 0 ] is computed using the initialization method proposed in ( [START_REF] Zhang | Agglomerative clustering via maximum incremental path integral[END_REF]). At first, a cluster is created for each sample, containing the sample and its nearest neighbor in the input space X, thus creating N clusters.

Then, the number of clusters is reduced by merging clusters which contains duplicated samples. This heuristic process usually lead to clusters which contain between 3 to 5 samples. The weights of the neural network θ[t 0 ] are initialized using Xavier initialization ( [START_REF] Glorot | Understanding the difficulty of training deep feedforward neural networks[END_REF]), which consists in drawing them from a normal distribution with zero mean and a variance which is inversely proportional to the number of neurons.

Representation learning

To go from θ[t] to θ[t + 1], JULE creates triplets of samples (i a , i p , i n ), where

• i a is the anchor sample,

• (i a , i p ) is a "positive pair", i.e. y[t] ia = y[t] ip , • (i a , i n ) is a "negative pair", i.e. y[t] ia = y[t] in .
Then ϕ θ , which has initial weights θ[t], is trained to minimize a triplet loss ( [START_REF] Schroff | Facenet: A unified embedding for face recognition and clustering[END_REF]):

L(θ, (i a , i p , i n )) = γW t (i a , i p ) -W t (i a , i n ) + α, (4.7)
where γ and α are user defined parameters. This kind of loss has become very common in representation learning ( [START_REF] Sermanet | Time-contrastive networks: Self-supervised learning from video[END_REF]). It consists in bringing closer points with the same labels while increasing their distance to other points. The parameter γ weights the importance between the positive and negative pairs. A large γ fosters ϕ θ to produce compact clusters while a small one emphasizes well separated clusters. The parameter α is usually called the margin, it defines a minimum distance between clusters. In ( [START_REF] Yang | Joint unsupervised learning of deep representations and image clusters[END_REF]), the authors suggest to use γ = 2 and α = 0.2.

Clusters merging

The cluster merging process is driven by a single parameter η, which is called the unfolding rate.

(C a , C b ) = argmax k 1 =k 2 A(C k 1 , C k 2 ). (4.8)
When merging two clusters, the total number of clusters goes from

K[t] to K[t] -1.
During the "cluster merging" step of iteration t, this operation is repeated until the total number of clusters reaches

K[t + 1] = max{ (1 -η)K[t] , K * }. Recommend-
ations for the unfolding rate are to use η = 0.2 for face recognition datasets and η = 0.9 for all other cases.

Remark: After each merging, the set of clusters is modified and A must be recomputed.

A few tricks to fasten this operation are introduced in the original paper.

JULE with multiview data

In this section, we propose to use JULE to train an MVnet to solve the MVC problem. The initialization step for JULE requires to merge the first clusters based on distances in the initial feature space of the data. To avoid having to define a distance in a multiview space, a different approach is adopted. First, each ϕ θ j is pretrained separately on Z j . After training all the ϕ θ j 's, ϕ θout is trained on the concatenation of the Z j = ϕ θ j (Z j ). Once MVnet has been properly initialized, it is used to produce a meaningful initial unified latent representation of the multiview data. This representation serves as the initial space in which the first cluster labels are assigned. Once the first clusters are initialized, JULE can be carried out normally on the MV data.

Another straightforward way to use JULE to solve MVC is to concatenate the different views and apply JULE to the concatenated features. This method is taken as a baseline for comparison in order to evaluate the MVnet approach and is referred to as the Concatenate and Cluster approach (CC).

Experimental validation 4.4.1 Experimental setup

Our experiments are conducted on the same 8 datasets presented in Section 3.2.1 from Chapter 3. For multiview generation, we use the Keras ( [Chollet, 2015]) implementations and pretrained weights of the ten CNN architectures introduced in Section 4.2.4. For each network, the chosen layer is the last before softmax, as suggested by the experimental results of Chapter 3.

To solve the generated MVC problem, we compare the two proposed DMVC methods (CC and MVnet) against MVEC. These three methods are implemented using JULE, which has been implemented in Keras to ease integration within the full IC pipeline (Figure 4.1). Other versions of these algorithms, using IDEC instead of JULE, have been tested in ( [START_REF] Guérin | Improving image clustering with multiple pretrained cnn feature extractors[END_REF]) but do not present a major interest. For MVnet, we also report the results without fine tuning (MVnet fix ), i.e. just after the initialization of each MLP. The results are also compared to MVEC with agglomerative clustering (MVEC agg ) so as to have a standard baseline for comparison.

DMVC is a framework for unsupervised classification, hence, hyperparameter tuning should be avoided. In all of our experiments, we use default parameters for every sub-algorithm used. For agglomerative clustering, we use the default configuration of the scikit-learn implementation ( [START_REF] Pedregosa | Scikit-learn: Machine learning in Python[END_REF]). For MVEC, the coassociation matrix is clustered with agglomerative clustering with average linkage. Finally, for JULE, we use the hyperparameters recommended in the original paper [START_REF] Yang | Joint unsupervised learning of deep representations and image clusters[END_REF]), which are given in Section 4.3.3.1. We also use the same kind of neural network architecture used in the original paper:

• all the MLPs constituting the MVnet architecture used in our experiments have dimensions d -160 -160, where d is the input dimension,

• the activation functions for the hidden layer are rectified linear unit,

• and we also use l2-regularization during training.

.

The clustering results are evaluated using both normalized mutual information (NMI) and purity (PUR), which are commonly used in unsupervised classification.

They both range between 0 and 1, with 1 representing perfect accuracy.

Experimental results

All the results of our experiments can be found in Appendix E. They report NMI and PUR scores for every pretrained CNN independently as well as for the different MVC methods applied to the MVC problems generated with the ten feature extractors. It is difficult to draw conclusions from the large number of results reported in Appendix E. For this reason, this section presents a condensed version of these results.

First of all, an algorithm is good if it produces cluster assignments with both high NMI and high purity. To simplify the analysis of the results, we introduce a mixed clustering evaluation metric to analyze jointly NMI and PUR results:

MIX β = β NMI + (1 -β) PUR, (4.9)
where β is a coefficient between 0 and 1 which weights the relative importance between NMI and PUR. Figure 4.7 represents the MIX 0.5 scores for the different methods and datasets.

To evaluate the interest of the proposed multi-view generation approach, both DMVC approches and MVEC need to be compared with each feature extractor taken independently. Comparing results with each of the ten networks is both cumbersome and difficult to analyze. Instead, we prefer to report results for the Best network (BNet) and the Worst one (WNet). BNet (respectively WNet) represents the network which demonstrates the best (respectively worst) results on the precise dataset where it appears. In practice, BNet is impossible to choose in advance because on a true unsupervised dataset, external evaluation metrics (NMI, PUR, etc.) cannot be computed. The only possible strategy for selecting a feature extractor f j * z among F z = {f j z , j ∈ {1, ..., M }} is one we call the Leading Network (LNet) strategy. Given a clustering problem X and a clustering algorithm A, the LNet strategy consists in using a set of P supervised datasets {(X 1 , y * 1 ), ..., (X P , y * P )} and choose f j * z such that

j * = argmax j∈{1,...,M } 1 P P p=1 MIX 0.5 (A(f j z (X p )), y * p ) . (4.10)
In other words, the leading network is the one which presents the best clustering results on average across the P supervised datasets, with respect to algorithm A. Using the online optimization formalism ( [Hazan et al., 2016]), different feature extractors can be considered as experts and the obtained MIX 0.5 scores as rewards from previous trials. Then, the LNet strategy simply becomes a Follow-the-Leader strategy.

In our case, we use 8 datasets to evaluate the proposed image clustering methods.

Hence, for each dataset, f j * z is computed by applying the LNet strategy on the P = 7 other datasets. The results obtained by f j * z on each dataset are reported under the name LNet. In practice, it appears that the optimal feature extractor f j * z is Densenet169 for all eight datasets 1 .

The condensed version of the results defined above can be found in Figure 4.7. We now propose to analyze these experimental results to explain our three key findings.

Results interpretations

4.4.3.1 IC can benefit from the use of several CNN feature extractors When using multiple pretrained CNNs instead of one, the ideal scenario is when the MV approach outperforms every independent network, i.e. it outperforms BNet. When this occurs, we can conclude that the different feature extractors contain complementary information which should be leveraged when possible (see Figure 4.2b and Section 4.2.2). In Figure 4.7, we can see that for all datasets but Birds and COIL100, the best of all methods is a MV method. In the specific case of COIL100, the tie between BNet and all the MV methods might mean that the domain of validity of BNet includes the domains of all the other networks (Figure 4.2a).

We remind that, in practice, it is impossible to predict which feature extractor will be the BNet. When facing an unsupervised dataset, without additional knowledge, the only way to obtain the BNet results is to choose a CNN at random and to get lucky. The risk of using random selection is to fall in the worst case scenario (WNet). This risk can be measured by the margin separating the MV methods from WNet.

Likewise, the potential benefit of random selection is measured by the difference between MV and BNet results. In Figure 4.7, we see that random selection is not worth considering because the risk is much higher than the potential benefit, which most of the time is even negative.

1

The fact that Densenet169 is the LNet for all the datasets means that it is consistently good across datasets. It does not mean that it is the BNet for each of the datasets. Our results actually show that Densenet169 is the BNet only for Flowers. The second possible benefit of leveraging our MV generation approach is to improve results from LNet. To the best of our knowledge, the LNet strategy introduced above is the only feature extractor selection method which is better than random.

B N e t a g g M V E C a g g M V E C C C M V n e t fi x M V n e t 0.7 0.75 0.8 0.85 BNet LNet WNet (a) VOC2007 B N e t a g g M V E C a g g M V E C C C M V n e t fi x M V n e t 0.92 0.94 0.96 0.98 1 BNet LNet WNet (b) COIL100 B N e t a g g M V E C a g g M V E C C C M V n e t fi x M V n e t 0.35 0.4 0.45 0.5 BNet LNet WNet (c) Archi B N e t a g g M V E C a g g M V E C C C M V n e t fi x M V n e t 0.4 0.5 0.6 BNet LNet WNet (d) MIT B N e t a g g M V E C a g g M V E C C C M V n e t fi x M V n e t 0.6 0.8 BNet LNet WNet (e) Flowers B N e t a g g M V E C a g g M V E C C C M V n e t fi x M V n e t 0.3 0.4 0.5 BNet LNet WNet (f ) Birds B N e t a g g M V E C a g g M V E C C C M V n e t fi x M V n e t 0.7 0.8 0.9 1 BNet LNet WNet (g) UMist B N e t a g g M V E C a g g M V E C C C M V n e t fi x M V n
In Figure 4.7, we can see that both MVnet and MVEC are above LNet for a large majority of cases. We also point out that for all 8 datasets, there is at least one ensemble method that outperforms LNet. One possible way to improve the LNet strategy would be to increase the number of trials, i.e. the number of dataset on which LNet is computed. However, we doubt that the results would vary much with a larger P . These experimental results suggest that when facing an unsupervised dataset, using multiple feature extractors should be preferred over selecting a single one.

MVC can be improved by adopting an end-to-end approach

We first note that methods implemented with JULE outperform agglomerative clustering for most of the datasets. The three exceptions are the two scene recognition datasets (Archi and MIT) and Birds. One possible reason for the failure of JULE in these cases might be that the hyperparameters are not appropriate. Indeed, on the one hand, in the original paper, authors give hyperparameters recommendations for natural recognition and face recognition datasets. For these two tasks, we note that the results with JULE are very good. On the other hand, for other IC problems, such as scene and fine-grained recognition, different parameters may work better. The scalability of JULE to different kind of IC datasets would be worth investigating. We also believe that other deep clustering methods might work better on these datasets. For exemple, the results reported in ( [START_REF] Wang | Unsupervised joint mining of deep features and image labels for large-scale radiology image categorization and scene recognition[END_REF]) appear to be good for unsupervised scene recognition tasks. It might be worth trying to adapt their method to multiview data and thus improve results on Archi and MIT.

To evaluate the interest of using MVnet independently from other considerations, such as the problems involved by JULE on certain datasets, we now only look at the 4 bars on the right of each diagram. The first thing we note is that the MVnet architecture is better suited for multiview data than the CC approach. We also underline that in most cases, CC presents limited interest compared to LNet. These results suggests that, for a new IC dataset, data extracted from multiple CNN feature extractors should be preprocessed independently before being considered jointly. We now compare the MVnet approach against MVEC. Overall, MVnet seems to perform better and in cases where it does not, results are similar. Finally, fine-tuning MVnet fix end-to-end seem to be a good idea. Indeed, except for Flowers, it always seem to perform either better or similarly.

Combining multi-view generation from multiple architectures and DMVC produces state-of-the-art results at IC

We conclude this results interpretation section by stating that, to the best of our knowledge, the results reported in this thesis for VOC2007, COIL100, Flowers, UMist and FEI are the new state-of-the-art for classifying these datasets without labels.

Learned representations

The quality of a deep clustering algorithm can also be assessed by studying the new feature representation it generates.

Evaluation with K-means

The features extracted with MVnet are first evaluated by reclustering them using

Conclusion

Key results

In this chapter, we propose a two-step approach to solving the image clustering problem. First, we generate multiple representations of each image using pretrained CNN feature extractors, and reformulate the problem as a multi-view clustering problem.

Second, we define a multi-input neural network architecture, MVnet, which is used to solve MVC in an end-to-end manner. In theory, any deep clustering framework can be adapted to train an MVnet on unsupervised data. In practice we propose to implement this approach within the JULE framework and demonstrate state-of-theart results for image clustering on several natural images datasets. This approach also has the advantage of removing the design choice of selecting a single feature extractor. Perspectives and future work regarding Image Clustering from multiple pretrained CNN feature extractors are discussed in Chapter 9.

Back to unsupervised robotic sorting

This part of the manuscript focuses on image clustering, which is a necessary skill for a robot to sort objects in an unsupervised way. The proposed methods for solving image clustering sets a new benchmark for this problem. However, one of the objectives of this thesis is to propose methods for unsupervised robotic sorting, which is a broader problem than image clustering. Indeed, in the IC setting studied in this part, images are given as inputs to our algorithms whereas in a typical robotics setting, the robot needs to collect its own data, for example by moving a hand-mounted camera. In this setting, the points of view under which the objects are observed can have a significant impact on the final results. For this reason, the next part of this manuscript focuses on view selection for semantic content maximization.

Part III

Image Acquisition in Unsupervised Robotic Sorting

Chapter 5

Unsupervised Robotic Sorting from Fixed Camera Poses Abstract

To provide a robot with the ability to sort objects based on their high level semantic nature, it is relevant to represent the objects by images. In the previous chapter, we studied Image Clustering (IC), which is a required skill for such an implementation of Unsupervised Robotic Sorting (URS). However, IC alone is not sufficient and the quality of the sorting also depends on the image acquisition process. This chapter presents an implementation of URS using fixed, vertical camera poses to gather the images, which is a common approach in robotic sorting. This methodology is first shown to work well on an industrial use case, consisting in unsupervised sorting of tools in a shopfloor environment. Then, a challenging dataset is built to further evaluate the robustness of this image acquisition approach to lighting conditions, background changes and objects poses. By using different poses of each objects jointly in a multi-view clustering framework, we show that the objects poses are a very important factor regarding the success of URS from fixed camera poses. Although the poses of the objects cannot be changed in a practical robotic sorting situation, these experiments show that the views under which the objects are observed are paramount for URS and motivate the development of a feasible optimal view selection method in Chapter 6.

Introduction

Real world classification system

The standard Machine Learning (ML) formulation of a classification problem is concerned with building pipelines that map numerical data to predictions. For example, in the case of image classification, the goal is to build models mapping tensors of pixels (images) to categories. According to ([Theodoridis and Koutroumbas, 2006a]), to design such a pipeline, one must solve four major subproblems: feature generation, feature selection, classifier design and system evaluation. Some classification methods perform some of these steps jointly, for example, deep learning models perform feature generation, feature selection and classifier design simultaneously. This general definition of a classification pipeline applies to both supervised and unsupervised tasks. However, when implementing a real world classification system, the inputs to the classification pipeline do not exist a priori and first need to be measured from physical objects. A schematic representation of a real world implementation of a classification system can be seen in Figure 5. To illustrate this problem, the very common toy example of a classification system to distinguish between apples and oranges can be considered. To be able to assign a category to a fruit, i.e. a physical object, one first needs to represent it by a mathematical object, i.e. extract a data representation. For example, a fruit can be represented by three real numbers: its weight, its diameter and its rugosity. Then, the measurement process consists in weighting the fruit, as well as measuring its diameter and its rugosity. After choosing what quantities are measured, the values obtained for a given fruit can still vary from two classes of factors:

• Measurement settings: These are the design choices that are made for measuring the objects, e.g. the tools used, the protocol, etc.

• Unmastered disturbances: These are all the variations that cannot be controlled but still influence the values of the measures, e.g. non-circularity of the fruits, thermal expansion, etc.

Then, building a good real world classification system consists in choosing measurement settings and a classification pipeline that are robust to the variations of the unmastered disturbances.

Real world URS implementation

The Unsupervised Robotic Sorting (URS) problem studied in this thesis consists in physically sorting objects based on their semantic nature. With the recent advances in deep learning, many hard image understanding problems have now been solved. For this reason, the feature representations chosen to represent the objects to be sorted are images. Therefore, the final predictions are obtained by solving an Image Clustering (IC) problem. The important problem of IC was studied in Part II of this thesis, in which it was proposed to leverage multiple pretrained feature extractors to improve state-of-the-art results. However, to implement URS on a real robot, the data acquisition process also needs to be defined and can be of utmost importance.

Indeed, depending on the scene in which the robot evolves, the objects poses, the camera poses, and the lighting conditions, the images passed as input to the IC pipeline vary a lot. A schematic representation of the decision making module of a URS implementation is proposed in Figure 5.2. The parameters in purple change among the different runs of the application and cannot be controlled, while the ones in orange need to be defined when setting up the application. Then, the objective of a URS application is to find an IC pipeline and to define rules for camera poses such that they jointly produce unsupervised object classifications which are consistently good for many kind of objects and in the range of variation of the disturbances.

Chapter organization

This chapter begins by describing a real world implementation of URS. It consists in observing objects restricted to a given region from a fixed camera pose, which is fairly common in robotics sorting [START_REF] Zhihong | A vision-based robotic grasping system using deep learning for garbage sorting[END_REF]), [START_REF] Eitel | Multimodal deep learning for robust rgb-d object recognition[END_REF])). Then, a challenging dataset is created to test the robustness to background and lighting conditions of such an approach for image acquisition. Finally, some experiments using multiple poses of each objects jointly are carried out, and appear to have a very positive impact on the robustness to unmastered disturbances. These results suggest that the view under which the objects are observed is paramount for the clustering results and motivate the development of a semantic view selection model in Chapter 6.

URS implementation with fixed camera poses

A first implementation of semantic URS was proposed for the demonstration sessions of the "TechDay Robotique Arts et Métiers1 " in Lille, France. This demonstrator is implemented with a KUKA LBR iiwa robot with both a camera and a parallel gripper mounted on the end-effector. The environment, i.e. the workspace in which the objects are placed, is an empty wooden table that is split into disjoint areas.

Then, the URS application goes as follows:

• Setting up the scene: Different previously unseen objects are placed in the environment such that there is no more than one object per area.

• Scene scanning: For each of the environment subdivisions, the robot moves the camera to the perpendicular top pose where it can see the whole area and takes a picture.

• Clustering: Once an image dataset of all the objects to sort has been gathered, the IC pipeline with one feature extractor presented in Chapter 3 is applied and returns a bin number for each object. The number of bins available determines the number of clusters that the IC pipeline produces.

• Grasping and sorting: Finally, the robot physically sorts the objects according to the clustering results.

Figure 5.3 shows a schematic view of the system, and a video of the implementation can be found at https://youtu.be/NpZIwY3H-gE.

The demonstrations were carried out using sets of different tools (screw drivers, allen keys, flat keys, clamps, etc.) and the goal was to correctly group together the different models of the same tool. The demonstration sessions happened in a shopfloor with unmastered lighting conditions (glass roof), and in about 100 runs, only one misclassification was recorded, which was due to very high brightness (white image). In practice, the vision module was composed of an Xception feature extractor combined with agglomerative clustering, which is the best performing pipeline on the robustness evaluation dataset (see Section 5.3).

Robustness to background/lighting conditions

The proposed implementation, with fixed top down camera poses, demonstrates the feasibility of a URS decision making module in a real world application. In the perspective of a real application in industrial workstations, it is essential to further investigate its robustness to different environments and lighting conditions. In this section, we propose to build a dataset to evaluate such robustness.

{C(i)} i∈Im

Vision Module

Dataset

To test the robustness to unmastered disturbances of the URS application, a challenging dataset for image clustering is created, it is composed of pictures of objects which can be found in industrial workstations. All pictures constituting the dataset are taken with a fixed camera, which looks perpendicularly at a planar surface. To create the dataset, we vary both the background on which the objects are placed and the location and intensity of the light source. Such a pair defines a background/lighting condition (BLC). Objects are chosen from seven classes, and pictures of each object are taken under five different BLC. For each object and each BLC, the dataset contains four pictures under different position/orientation within the field of view of the top down camera. Using multiple poses of each object enables to reduce the influence of the object pose, and thus isolate the impact of background and lighting conditions.

The different poses are also used in Section 5.4 to study how the point of view under which the objects are observed influences the sorting results. This dataset, which is illustrated in Figure 5.4, appears to be challenging for image clustering because of the BLC variations but also because some classes have low intra-cluster similarity (e.g.

USB drives) and extra-cluster similarity between some classes is relatively high (e.g. pens and screws). The dataset statistics are summarized in Table 5.

Results

The objective of this section is to evaluate the robustness of the image acquisition method to background and lighting conditions. However, in Chapter 3, it was shown that a bad choice of CNN feature extractor and/or clustering algorithm can lead to poor clustering results for certain datasets. Hence, to isolate the influence of the image acquisition scheme as much as possible, a good clustering pipeline is needed.

Therefore Averaging over all the positions enables us to reduce the dependence of the results on the objects poses, and thus better study the robustness to BLC. From these experimental results, three important conclusions are drawn:

• In Tables 5.2 and 5.3, we can see that agglomerative clustering always present better results than KMeans. Likewise, Inception-like architectures tend to outperform their competitors, particularly Xception, which outperforms all the other CNN architectures for every clustering algorithm and every BLC. Based on these results, from now on, we only study the Xception + Agg vision pipeline.

• Table 5.3 shows that the chosen pipeline seems to be robust to reasonable changes in lighting conditions (Dark and Bright) but its performances start to decrease when the brightness is really high or low. This makes sens as it also becomes difficult for a human to identify these objects.

• Background changes seem to have a more important impact on the clustering results. Indeed, in Table 5.2, BLC 4 shows much lower results than other BLC.

This might come from the fact that the background in BLC4 contains geometric shapes (lines, circles) which are distractors for the network.

Robustness to objects poses

Methodology

In Section 5.1, it was shown that given a set of objects to sort, the clustering results depend on the lighting conditions, the scene layout, the objects poses and the camera poses to observe each object. The dependence on background and lighting conditions was studied experimentally in Section 5.3, and we now want to evaluate the influence of the objects poses on the results of the proposed implementation. To do so, we propose to vary the poses of the objects under a fixed camera view, which boils down to leveraging the different poses in the dataset introduced in Section 5.3.1.

In the unsupervised image classification setting, the clustering results depend on the entire set of images jointly, thus making it hard to evaluate the contribution of an individual image. For this reason, a Multi-View Clustering (MVC) approach is adopted to evaluate the influence of the object poses. Indeed, if using multiple poses jointly enables to improve the clustering results substantially, we can conclude that either some views are better than the others, or that the views are complementary (see Section 4.2.2). In both cases, this would motivate the development of better view selection methods, which is the purpose of the next chapter. The MVC problems are solved using the Multi View Ensemble Clustering (MVEC) method that was used for comparison in Chapter 4, and which is described in more depth in Section 5.4.2.

Remark: The Deep Multi-View Clustering method proposed in Chapter 4 was not used because our implementation of URS consists in sorting a small number of objects (< 50), which leads to IC problems containing too few instances. Deep end-to-end clustering methods are usually bad at solving small scale tasks because, with too few data, the desired number of clusters is reached too fast (after the initialization step) and the representation learning networks cannot converge. However, such method is not to be excluded for larger scale industrial URS problems leveraging multiple views.

Multi-view clustering using Ensemble-clustering

Let Ω = {Ω 1 , ...Ω M } be the set of M objects to cluster. For each object Ω i , let ω i = {ω i,1 , ...ω i,m i } be the set of m i views representing it. The clustering pipeline chosen from the experiments in Section 5.3.2 (Xception + Agg) is denoted C. The proposed MVEC method, inspired by ( [START_REF] Tao | From ensemble clustering to multi-view clustering[END_REF]), goes as follows. For each object Ω i , one image ω i,j is selected randomly. Then, the new image set is clustered using C, thus producing a partition (cluster assignment for each object) denoted P k . This procedure is repeated N times and the set of N partitions generated is noted

P = {P 1 , ..., P N }.
Once the N partitions have been generated by randomly sampling images in each ω i , MVEC must find a consensus partition P * , maximizing the agreement between the different partitions. In this thesis, an approach based on objects co-occurrence is adopted to compute P * . Since this work is about unsupervised classification, there is no correspondence between the class assignments of the different partitions of P . To avoid this problem, we use intermediate representation of P , called the co-association matrix. This matrix, denoted A, is a M × M symetric matrix which entries are defined by

A pq = 1 N N t=1
δ(P t (Ω p ), P t (Ω q )), (5.1) where P t (Ω i ) is the label associated with object Ω i and generated by partition P t , and δ(a, b) is the Kronecker symbol, which is 1 if a = b and 0 otherwise. Entry (i, j)

of CA measures how many times objects Ω i and Ω j have been classified together by the different partitions.

Finally, let C * be any connectivity-based clustering algorithm. For instance, C * can be variants of agglomerative clustering or spectral clustering. The consensus partition P * is obtained by applying C * using A as the precomputed similarity measure between objects:

P * = C * (A).
(5.2)

A schematic representation for the MVEC pipeline can be found in Figure 5.5. In our implementation, image sampling for partition generation is uniform, C * is agglomerative clustering and the number of partitions generated is N = 1000.

Multiview Input images

. . .

Ω 1 Ω 2 Ω M Random view sampling C P 1 C P 2 C P N . . .

Results

The results using MVEC with the clustering pipeline chosen above are reported in Table 5.4. MVEC consistently performs significantly better than the corresponding single view approach, which are written in parenthesis in Table 5.4 as a reminder. It is also more robust to poor background (BLC4) and lighting (Very bright) conditions. Regarding computation time, although MVEC takes longer, this time can be drastically reduced by parallelizing both the partition generation process and the co-association matrix computation.

The excellent results obtained by MVEC suggest that the proposed image acquisition method that consists in using fixed top down camera views is very sensitive to objects poses. Indeed, the fact that the robustness can be increased significantly by duplicating the objects poses suggests that in the single view approach, some of the poses were very poor for clustering, thus decreasing the average results presented in Tables 5.2 and 5.3. Another possible cause of having poor results with a single view approach is that the individual images may not be compatible to use jointly. For example, it may not be easy to group together two mugs if one is seen from the top and the other from the side. Hence, by increasing the number of views, the event of having similar images for similar objects becomes more likely. Robustness to poor lighting conditions also makes sense as light comes from a certain direction and there are always better angles to observe the objects.

Conclusion

Key results

Unsupervised Robotic Sorting consists in physically grouping together previously unseen objects in a way that makes sense at a human level. In this chapter, a first working implementation is presented. In this application, objects are placed in constrained regions of the environment and the image representations are acquired from predefined fixed camera poses. The unsupervised image classification module is com-posed of a pretrained Xception feature extractor and agglomerative clustering. This approach is shown to work for a practical use case, in a real-world industrial environment. To further test the robustness of this image acquisition approach to lighting conditions, background changes and objects poses, a robustness testing dataset was specifically created. This dataset, which is challenging for image clustering, is made publicly available to help other researchers to test their image clustering algorithms.

On the one hand, the proposed URS pipeline appears to be fairly robust to reasonable lighting changes and to different uniform backgrounds. On the other, the results decrease drastically when the background presents strong patterns and for very bright/dark images. The influence of the poses of the objects is evaluated by carrying out multi-view clustering experiments. We show that sorting results can be improved by using jointly images from several objects poses, which suggests that under a fixed camera pose, the poses of the observed objects are of major importance. Perspectives and future work regarding practical implementations of URS are discussed in Chapter 9.

Towards adaptive camera poses

The first approach implemented in this chapter to solve URS demonstrates rather good results, which is very encouraging. However, it was also shown that using fixed camera poses for images collection is sensitive to objects poses and would benefit from using several of these poses for each object. This makes the fixed camera approach problematic because, in practical situations, the pose of an object in the environment is an unmastered input to URS and cannot be changed. Hence, the fixed camera approach is risky as it cannot account for all possible variations of the objects poses. On another note, the goodness of a pose of an object is not intrinsic to the pose itself but rather relative to the camera pose. What really matters is the relative pose between the object and the camera. In addition, the camera pose is a user defined setting that can be modified. Consequently, a different image acquisition module, where the camera poses are adapted to the unmastered objects poses, could help improving URS. This statement motivates the development of an autonomous optimal view selection pipeline in Chapter 6.

Chapter 6

Semantically Meaningful View Selection Abstract

An understanding of the nature of objects could help robots to solve both highlevel abstract tasks and improve performance at lower-level concrete tasks. Although deep learning has facilitated progress in image understanding, a robot's performance in problems like object recognition often depends on the angle from which the object is observed. Traditionally, robot sorting tasks rely on fixed top-down views of the objects. By changing its viewing angle, a robot can select a more semantically informative view leading to better performance for object recognition. In this chapter, we introduce the problem of semantic view selection, which consists in finding good camera poses to gain semantic knowledge about observed objects. We propose a conceptual generic formulation of the problem, together with a relaxation based on clustering, to make it solvable. We then present a new image dataset consisting of around 10k images representing various views of 144 objects under different poses.

Finally we use this dataset to propose a first solution to the problem by training a neural network to predict a custom "semantic score" from a top view image and camera pose. The views predicted to have higher scores are then showed to provide better clustering results than fixed top-down views.

6.1 Introduction tions. The link back to URS is done through the experimental results section, where we demonstrate that our approach for adapting camera poses enables to improve unsupervised sorting. The following section proposes a richer introduction about SVS, thus extending the context of URS.

Introduction and context for Semantic View Selection

Recent advances in machine learning have increased robot autonomy, allowing them to better understand their own state and environment, and to perform more complex tasks. An important research direction is to improve semantic understanding of objects, which can aid in tasks such as manipulation. For example, better semantic understanding can be directly used to solve tasks such as supervised [START_REF] Eitel | Multimodal deep learning for robust rgb-d object recognition[END_REF]), [START_REF] Zhihong | A vision-based robotic grasping system using deep learning for garbage sorting[END_REF])) and unsupervised (Chapter 5) sorting. It can also impact indirectly other important tasks, such as robotic grasping (( [START_REF] Bohg | Datadriven grasp synthesis-a survey[END_REF]), [START_REF] Lenz | Deep learning for detecting robotic grasps[END_REF])). Indeed, ( [START_REF] Chua | Grasp representations depend on knowledge and attention[END_REF]) showed that the way people grasp objects not only depends on their forms and shapes, but also on our semantic understanding of the object. Knowledge of manipulated objects is also important for human robot collaboration ( [START_REF] Tsarouchi | On a human-robot collaboration in an assembly cell[END_REF]), where significant efforts have been made to make robot behavior safe and adaptive to humans ( [START_REF] Munzer | Efficient behavior learning in human-robot collaboration[END_REF]). For example, a robot should not hand a human a knife by the blade.

Vision-based methods are a natural choice to acquire knowledge about manipulated objects, as supported by the recent advances in deep learning for both supervised (( [Chollet, 2016]), [START_REF] He | Deep residual learning for image recognition[END_REF])) and unsupervised [START_REF] Yang | Joint unsupervised learning of deep representations and image clusters[END_REF]), [START_REF] Aljalbout | Clustering with deep learning: Taxonomy and new methods[END_REF])) image classification. However, a robot can act in the real world to change the view under which the object is observed. This can have a huge impact on understanding what the object is. For example, in Figure 6.1, only the middle image enables the robot to understand that it is looking at a comb. The robot's ability to act has not been fully exploited in previous research. For example, prior work on robotic sorting of objects relies on a fixed, perpendicular top pose for the robot camera (see Chapter 5). Some previous work for best view selection ( [START_REF] Dutagaci | A benchmark for best view selection of 3d objects[END_REF]) has focused on producing representative views of 3d mesh models. Although this is a promising approach, it is not applicable for many robotics tasks, especially when complete 3d models are not available for all of the manipulated objects.

A possible reason why view selection was not studied before may be that previous work mostly deals with instance retrieval or supervised classification. In these two applications, the models to recognize the object are trained to be robust to view changes by overfitting the problem to solve. Indeed, instance retrieval deals with recognizing a precise object from a bank of objects. In this context, the view is not crucial as each face of an object is likely to contain characteristics which are specific enough to distinguish this object from the rest of the set. In other words, an instance retrieval model is overfitting on the set of objects. Likewise, supervised learning can be viewed as a form of overfitting on the given classes. As a matter of fact, when training a supervised model, we just need to be able to separate the object from classes which are present in the supervised problem. These characteristics do not have to be very specific to the semantic class of the object. For example, detecting a wheel might be sufficient to distinguish a car from a horse, but it is not generic enough to know that we are not looking at a bus.

In this chapter, we aim to find a method to optimize the poses of a robot with a hand-mounted camera, to maximize the semantic content of the images and understand the nature of objects being observed (Figure 6.1). In Section 6. proposed camera pose to a semantic score. Our experimental results, in Section 6.5, demonstrate that the proposed network can predict camera poses which outperform fixed poses at unsupervised sorting tasks.

The Semantic View Selection (SVS) problem

In this section, we formally introduce the problem of selecting optimal views for semantic understanding and introduce notations.

Generic formulation: the semantic function

We begin this section by introducing some definitions. A scene is defined by a set of objects and their relative poses. For example, the scene in Figure 6.2 is composed of a table, a mug, a bottle and other objects around. In this chapter, the scenes are considered static, which means that in Figure 6.2, changing the pose of the mug would S ω (I) is high if I is highly informative about the nature of ω (second column of Figure 6.1) and low if it's not (third column of Figure 6.1). A natural choice for the output space of S ω is [0, 1]. A semantic function is defined relatively to the object considered, which can be understood by considering the example in Figure 6.2.

Indeed, for the image in Figure 6.2b, S bottle should be high while S mug should be low. We also note that in practice, the semantic function is defined with respect to a subjective label definition. In this thesis, we use the most generic and simple possible label definition (e.g. spoon, mug, toothbrush, etc.), without adding any specific description (e.g. silver spoon, blue mug, etc.). More concretely, semantic meaningfulness can be viewed as the information contained in the output of a high level feature extractor, e.g. last layer of a pretrained deep CNN, which can be used to infer the general category of the object represented in the image.

For a given scene ζ and a given object ω in this scene, the Semantic View Selection (SVS) problem is defined as follows:

Maximize

pc∈D ζ S ω • v ζ (p c ). (6.2)
In other words, we aim to find p * c such that the view v ζ (p c ) maximizes S ω . The function S ω • v ζ (.) depends on the object being study ω, its pose, the other objects in the scene that can act as distractors, and the pose of the camera.

First relaxation: clusterability functions

The semantic function defines the general form of the SVS problem, but in practice, it cannot be evaluated. Therefore, to approximate S, we introduce a new family of clusterability functions: {S a,m , a ∈ A, m ∈ M}, where A represents the space of all possible image clustering algorithms and M the space of all clustering evaluation metrics. In other words, an element of A is a function mapping any set of images to a corresponding set of labels: (6.3) where L = {1, ..., K} is the set of possible labels, n is the number of images in the dataset and K is the expected number of clusters. In practice, K does not have to be fixed but in this thesis we only deal with clustering algorithms for which K is known. We also assume that for each image in I, there exists an associated ground-truth label, which can be seen as an underlying subjective optimal classification (see Section 6.2.1). For example, in the context of unsupervised robotic sorting, although no information is known by the robot a priori, we can suppose that a qualified worker should be able to judge the quality of the sorting afterwards. Following these notations, an elements of M is a function which takes two sets of integers as inputs, the predicted labels and the ground truth labels, and outputs a real valued score, usually

a ∈ A ⇐⇒ a : I n → L n ,
in [0, 1]: m ∈ M ⇐⇒ m : L n × L n → [0, 1]. (6.4)
Now, let Ω ∞ be the conceptual infinite set of all possible objects. Similarly, let Z ω ∞ be the conceptual infinite set of all possible scenes containing object ω. A natural image clustering problem with n images is defined by

cp = {v ζ ω[i] (p c [i]) | for all i ∈ {1, ..., n} : ω[i] ∈ Ω ∞ , ζ ω[i] ∈ Z ω[i] ∞ , p c [i] ∈ D ζ ω[i] }.
In other words, any set of natural images containing an underlying label (defined by the ω[i]'s) can be viewed as a clustering problem. Let L cp be the ground truth labels associated with cp and a(cp) be the cluster assignments (predictions) for cp with algorithm a. Finally, for a given natural image I, generated with the hand-mounted camera c, we define CP I ∞ , the infinite set of all possible natural image clustering problems containing I. Then, S a,m is defined by We assume that a and m are respectively a good image clustering routine and a good clustering metric, i.e. they have been shown to work well in practice. In the rest of the chapter, we also assume that S a,m and S are highly positively correlated. This assumption is based on the intuition that a semantically meaningful image should be properly clustered with similar objects by a good clustering pipeline. Indeed, as detailed later, the common clustering pipeline used in this paper consists in extracting features from a deep feature extractor and clustering the new set of features using a standard clustering algorithm. This choice for a is in line with the definition of semantic meaningfulness proposed in Section 6.2.1, as the final representation of a view, passed to the clustering algorithm, is a vector a features extracted from a pretrained CNN. Another motivation for choosing a clustering-based estimate for the semantic function is that supervised classification or object detection methods might not be adapted. Indeed, to compute the Monte-Carlo estimate of such function (see Section 6.2.3), the selected algorithm needs to be run many times on relatively small datasets. Doing this in a supervised way has high chances to result in overfitting, in which case all views would have high semantic scores.

S a,m (I) = E cp∈CP I ∞ [m(a(cp), L cp )], ( 6

Second relaxation: clusterability on a finite dataset

As it is not feasible to consider all possible scenes containing all possible objects, we further relax the above definitions to consider a finite dataset. Let Ω N be a finite set of objects containing N elements. For a given element ω of Ω N , we also define Z ω Nω , a set of N ω scenes containing ω, and P ζω N ζω a set of N ζω camera poses observing object ω in scene ζ ω . In other words, the set

X = {v ζω (p c ) | ω ∈ Ω N , ζ ω ∈ Z ω Nω , p c ∈ P ζω N ζω } is an natural image dataset containing ω∈Ω N ζω∈Z ω Nω (N ζω ) images.
For a given image I ∈ X, if the dataset X is large and diverse enough, an estimate of S a,m (I) can be computed by

S a,m X (I) = E cp∈CP I X [m(a(cp), L cp )], (6.6)
where CP I X is defined like CP I ∞ with cp's sampled from X. For large datasets, it might be computationally intractable to compute S a,m X (I) as the number of possible combinations of images grows exponentially with the number of views. Thus, we propose to compute the Monte-Carlo estimate Ŝa,m

X (I) = E cp∈CP I X,M C
[m(a(cp), L cp )], (6.7)

where CP I X,M C is a subset of N M C elements of CP I X , and N M C is a large natural integer (N M C ≥ 2×10 5 in our experiments). The method used for sampling clustering problems from X and thus creating CP I X,M C is explained in Section 6.4.2.

Partially-observable Semantic View Selection

Given an object ω in a scene ζ, the relaxed SVS problem, aims to find a camera pose

p c such that Ŝa,m X (v ζ (p c
)) is high. In a generic robotic pipeline, the exact setup of a scene is generally unknown and needs to be estimated from partial observations. Let ψ ζ be the observation from which we want to estimate the elements composing ζ and their relative poses. For example, ψ ζ can be a top-view image, taken from an initial predefined camera pose. Our approximation of the clusterability function score is then dependent on ψ ζ as a surrogate for the exact scene setup. More concretely, we want to optimize the parameters α of a function f α : {ψ ζ , p c } → s ∈ D m , where D m is the output domain of the metric m (usually [0, 1]), such that s is an estimate of Ŝa,m X (v ζ (p c )). A typical practical choice for f α would be a convolutional neural network, where α represents its trainable parameters.

Dataset Construction

To tackle the proposed relaxed SVS problem, we have built an image dataset representing various everyday objects under different poses, and observed under multiple views with a camera mounted on the end-effector of a UR10 robot manipulator (see Figure 6.1). The dataset can be downloaded at https://github.com/jorisguerin/ SemanticViewSelection_dataset and its statistics can be found in Table 6.1. In this dataset, for a given object, the scenes are composed of the object alone on a wooden table, under different poses. For example, Figure 6.1 images taken from different camera poses for one of such scenes for a comb.

Estimating object location and size

The dataset was collected using an Asus Xtion RGBD sensor, hand-mounted on a UR10 robot manipulator. For a given object ω in a given pose, we gather images corresponding to several camera poses, with ω centered in the image. The first step is to estimate the location of the Geometrical Center of the object (GC o ). To do so, we place the robot in an initial pose such that the camera can see the entire workspace in which objects can be placed. We store a background image of this pose, corresponding to what the camera sees when there is no object. Then, using RGB background subtraction, the xy-contour of the object is obtained, the z axis being vertical. From this contour, we estimate the x and y components of GC o , the width and the length of ω. Finally, we compare the minimum values of the point cloud inside and outside the xy-contour to estimate both the z component of GC o and the height.

Parameterization of camera poses

To parameterize camera poses, we define a reference frame at GC o . We then compute d = length 2 + width 2 + height 2 , the diagonal of the object's bounding box, and define the radius R such that d takes 70% of the smallest dimension of the image if the optical center of the camera (OC cam ) is at a distance R of GC o and z cam is pointing towards GC o . The camera poses are sampled on the half-sphere of radius R, centered at GC o , such that z o is positive. For each position of OC cam on the sphere, the camera is positioned such that z cam is pointing towards GC o , x cam is in the xy o plane and y cam is pointing "upwards".

View sampling and data collection

On the sphere, the location of OC cam is localized by two angles, θ and ϕ, which are defined as in Figure 6.3. Hence, a camera pose is simply represented by a (θ, ϕ) pair.

In our implementation, θ is sampled every 45 • between 0 • and 315 • but excluding 270 • , ϕ is sampled every 15 • between 45 • and 75 • . The views for θ = 270 • correspond to configurations where the camera is oriented towards the robot base. They were not collected in the dataset to avoid seeing the robot on the images. The other missing values come from unreachability of the camera poses with the robot manipulator, which occurs when the RRT connect ( [START_REF] Kuffner | Rrt-connect: An efficient approach to single-query path planning[END_REF]) planner fails to generate a valid plan. Furthermore, while it could be interesting to sample angles lower than ϕ = 45 • , these configurations are often unreachable because the robot would collide the table. A subset of the views gathered for one object in a particular pose can be seen in Figure 6.4. individual indexes are framed in green (resp. red). See Sections 6.4 and 6.5 for more details.

6.4 Proposed approach

Clustering pipeline and metric

Given an image I, the clusterability function Ŝa,m X (I), used to represent the semantic function, is defined by both a good clustering pipeline a and a clustering evaluation metric m.

90

In this work, we use the image clustering pipeline described in Chapter 3, which consists in getting a new representation of each image from the last layer of a deep CNN feature extractor, pretrained on ImageNet ( [START_REF] Russakovsky | Imagenet large scale visual recognition challenge[END_REF]), and clustering the new set of features using a standard clustering algorithm. Although some variants of this algorithm are tested in Section 6.5, the standard pipeline in this paper uses Xception ( [Chollet, 2016]) to extract features, and agglomerative clustering ( [Murtagh, 1983]) to cluster the deep features set. We use the implementation and weights of Xception proposed by the Keras library ( [Chollet, 2015]).

The clustering metric chosen to represent the clusterability is the Fowlkes-Mallows (FM) index ( [START_REF] Fowlkes | A method for comparing two hierarchical clusterings[END_REF]), which was already used in Chapter 4 and is defined by

F M I cp,a = T P (T P + F P )(T P + F N ) , (6.8)
where T P , F P and F N respectively represent the number of true positive, false positive and false negative pairs after clustering cp using a. The FM index ranges between 0 and 1. We choose this index because it can be converted straightforwardly to a local form

F M I i cp,a = T P i (T P i + F P i )(T P i + F N i ) , (6.9)
where F M I i represents the individual FM score of image I i ∈ cp, T P i , F P i and F N i respectively represent the number of true positive, false positive and true negative pairs containing I i . This individual form is used in the next section to reduce the sample complexity for computing { Ŝa,m X (I), I ∈ X}. The clustering pipeline and metric being chosen, we drop the a and m superscipts in the coming sections.

Training Set

From the 29 categories composing the dataset, five are chosen at random to constitute a validation set which is neither used for fitting the clusterability scores nor for training the view selection network. In this section, X refers to all the views composing the 24 remaining categories. From X, a clustering problem is created by sampling randomly the number of categories, the selected categories, the number of objects per category, the selected objects, the pose for each object (scene), and one view (camera pose) for each scene. As every estimated quantity is computed on the training set from now on, we drop the subscript X. We build CP M C such that min({N I cp , I ∈ X}) is at least 2 × 10 5 . In practice, this requires to solve N cp ≈ 3×10 7 clustering problems. Because of the high computational expenses of this process, we cannot get a much higher number of samples for the Monte Carlo estimate. Hence, ŝ(I) seems more appropriate than Ŝ(I) to estimate the semantic content of I because it evaluates the individual contribution of each view to the global clustering results. Hence, our training set for the next section is composed of {{I I top , ϕ I , θ I }, ŝ(I)} input/output pairs, where I I top represents the top view image associated with I, ϕ I and θ I are the angles parameterizing I. We note that, for each pose, the scores among all the views are scaled to the [0, 1] interval to help training, as some objects are harder to cluster than others. In such case the best views of a difficult object might have lower ŝ values than the worst views of an easy object. This makes the view selection problem harder because predicting the intrinsic "clusterability" of an object from a poor view can be very challenging.

Learn to predict semantic scores

After computing semantic view scores for each view in our training set, we aim to solve the SVS problem introduced in Section 6.2.4. To do so, we train a multi-input neural network architecture to predict ŝI from a triplet (I I top , θ I , ϕ I ). The top image is first passed through a VGG convolutional block ( [START_REF] Simonyan | Very deep convolutional networks for large-scale image recognition[END_REF]) with 19 layers initialized with weights pretrained on ImageNet.

Then, the outputs of the convolution block are fed into a first multi-layer perceptron (MLP). The outputs of this first MLP are concatenated with the angular inputs and fed into a second MLP, which outputs the semantic estimate. The architecture is summarized in Figure 6 The Conv Block is the convolutional part of VGG19 (until "block4 pool" layer).

To train this network, we use the Adam optimizer ( [START_REF] Kingma | Adam: A method for stochastic optimization[END_REF]), with an initial learning rate of 10 -3 . The choice of the architecture was cross-validated by removing randomly two categories from the training set. At test time, all the dropout rates are set to 0.

The choice of treating SVS problem by predicting a score from proposed camera poses instead of regressing directly on these poses has two main motivations. First, when dealing with robots, it might be impossible to plan a trajectory to some views, hence, it seems more relevant to only consider reachable views. Then, the SVS problem may contain many possible solutions, in which case there is no unique mapping between a top view and a good camera pose. Indeed, a network would likely converge to the average of all good poses, which might not be a good pose.

Experiments

Baseline for comparison

Given an object clustering problem, we define a view selector as any process to select the camera pose to observe the objects. To evaluate the quality of a view selector on a clustering problem, we compare its results under a certain (a, m) pair against two baseline view selectors. The first method is usually the one implemented when dealing with autonomous robot sorting, it consists in observing the object from the top view. This view selector will be noted TOP in our experiments. Another baseline view selector, denoted RAND, consists in choosing the views uniformly at random among the possible views.

We consider a view selector successful if it can outperform these two baselines. Indeed, borrowing from the reinforcement learning literature, comparing against the best fixed view can be assimilated to some form of regret analysis. Computing the best fixed view for our problem would be too computationally expensive. Instead, we compare against one arbitrary fixed view (TOP) and also against randomly selected views to ensure that the chosen fixed view was not particularly weak.

Evaluation of the individual semantic view score

The individual semantic view scores are fit using a particular (a, m) pair. Therefore, to evaluate it, we must test it on additional clustering pipelines and metrics. We vary pipelines by changing both the deep feature extractor and the clustering algorithm.

The three pipelines tested are denoted XCE AGG, VGG AGG and XCE KM, where XCE stands for Xception, VGG for VGG19, AGG for agglomerative clustering and KM for KMeans. As for the clustering metrics, we use the Fowlkes-Mallows index (FM), normalized mutual information (NMI) and cluster purity (PUR), which are three commonly used metrics to evaluate clustering algorithms when the ground truth is known.

We compare two view selectors against RAND and TOP. The first one, denoted OPT ind , consists of choosing the image with the highest individual semantic view score. Following the notations from Section 6.2.3, for a given object ω in a given scene ζ ω , the camera poses are sampled from P ζω N ζω . The OPT ind view selector chooses the pose p ind c = argmax

pc∈P ζω N ζω (ŝ(v ζω (p c ))
). (6.12)

Similarly, the second view selector, denoted OPT glob , chooses the pose that produces the image with the highest global semantic view score:

p glob c = argmax pc∈P ζω N ζω ( Ŝ(v ζω (p c ))). (6.13)
The results are averaged over 10 4 clustering problems and reported in Table 6.2. The first thing to note is that both semantic estimators, although fit with a = XCE AGG and m = FM, seem to pick views which are much better than TOP and RAND. This is not surprising as these results were computed on the dataset used to compute the estimators. However, it strengthens the belief that ŝ and Ŝ are good semantic function estimators as they generalize to other feature extractors, clustering algorithms, and metrics. Surprisingly, we also note that Ŝ and ŝ performances are very similar. This might mean that the number of samples in the MC computation is sufficient for Ŝ to be a good estimator of S. However, in our experiments, we also acknowledge that for a given object ω in a given scene ζ, the values of the set

{ Ŝ(v ζ (p c )) | p c ∈ P ζω
N ζω } are much closer to each other, which reveals that information about individual data is lost when considering the global estimator. The slightly better results of the ŝ estimator, as well as its better separability, justifies its use for training SV-net.

Finally, we refer the reader back to Figure 6.4, where images with both high and low ŝ(I) values have been outlined. This gives a qualitative validation of the index relevance for estimating the semantic function. Indeed, it is easier to tell that the robot is looking at sun glasses from the green-outlined images than from the redoutlined ones.

Evaluation of the learned semantic view selector

To evaluate our semantic view selection network (SV-net), we adopt a similar approach to the one in the previous section. The SV-net view selector is compared against RAND and TOP under various configurations on the validation set, which was neither seen for ŝ(I) computation, nor for training SV-net. The complete list of the 29 categories can be found in the dataset folder (https://github.com/jorisguerin/ SemanticViewSelection_dataset). Results are averaged over 10 4 clustering problems randomly sampled from the validation set and are reported in Table 6.3. In the results presented, the five randomly chosen categories composing the validation set were: comb, hammer, knife, toothbrush and wrench. We note that SV-net was able to predict views which are better than TOP and RAND, which is a more remarkable result than in the previous section as these kind of objects where never seen by the network before. SV-net is able to extract sufficient information from a single top view image to predict if a camera pose will provide good high-level features about the object being observed. As a qualitative validation, four samples of predicted images can be seen on Figure 6.6. We also underline that the absolute values of the clustering scores cannot be compared between tables. Indeed, the object considered are different and there is no guarantees that there exist views able to reach similar clustering accuracy when different classes of objects are considered.

Conclusion 6.6.1 Key results

In this chapter, we have introduced a new problem called semantic view selection. The SVS problem consists of finding a good camera pose to improve semantic knowledge about an object from a partial observation of the object. We created an image dataset and proposed an approach based on deep learning to solve a relaxed version of SVS problem.

By fitting an index based on averaged view clustering quality and training a neural network to predict this index from a top view image, we show that it is possible to infer which view results in good semantic features. This has many practical applications including autonomous robot sorting, which is generally solved from top view images only. Indeed, one can use the SV-net to enhance any sorting robot with the ability to select better views to reduce sorting errors. Results reported in Section 6.5.3 demonstrate that SV-net predictions outperform the approach proposed in Chapter 5. The views obtained with SVS are better than fixed views and have the potential to improve URS. Perspectives and future work regarding semantic view selection are discussed in Chapter 9.

Towards fully autonomous unsupervised robotic sorting

Parts II and III of the thesis focus on the decision making module of Unsupervised Robotic Sorting (URS). It was shown that by properly choosing the views under which to observe the objects and with a good image clustering pipeline, the results

for objects classification can be improved. However, a complete sorting pipeline is composed of many other steps than decision making. To reach complete autonomy and robustness to different kinds of objects and environments, a sorting robot must have various other skills such as scene segmentation, objects localization, grasping, trajectory generation, control, etc. Hence, the next part focuses on two of such skills: trajectory learning and object localization, and thus constitutes a step towards context independent robotic sorting.

Part IV Further developments

Introduction

Chapters 2 to 6 focus on the unsupervised robotic sorting application. This problem is closely related to unsupervised object understanding, an important skill for many autonomous robotic tasks. However, to implement industrial robotic applications that work under a broad range of conditions, many other skills need to be mastered.

In this part of the manuscript, we study two of such skills: trajectory learning and 3D object localization. For example, in the robotic sorting context, these two skills are essential to build a functional implementation that works in various environments, and for any kind of objects. Indeed, object localization is a required skill to be able to observe the different objects (view selection) and to grasp them. Likewise, to navigate the robot between different configurations, the robotic system may benefit from smart trajectory optimization methods, which is the focus of this chapter. We propose to adapt a recent method based on optimal control, to accept any kind of loss function based on sensor signals. The remainder of this section gives a specific introduction about trajectory learning.

Literature overview

Optimal feedback control theory provides an efficient framework for robot manipulators movement generation as it enables to compute both an optimal open-loop trajectory and a feedback controller at once. To carry out such tasks, classical optimal control uses a model of the dynamics and selects the solution that minimizes a properly designed cost function. Under linear dynamics and quadratic cost, an optimal solution can be found analytically thanks to the widely studied theory of Linear-Quadratic-Regulators (LQR) ( [START_REF] Lewis | Optimal Control[END_REF]). When the dynamics is a more complex, non-linear function, the problem becomes more difficult. However, the iterative Linear-Quadratic-Regulator algorithm (iLQR) ( [START_REF] Li | Iterative linear quadratic regulator design for nonlinear biological movement systems[END_REF]) still enables to converge towards a locally optimal solution by iteratively fitting local linear approximations to the dynamics. On the other hand, autonomous learning of manipulation skills have received much interest over the past decades. Thanks to different Reinforcement Learning (RL)

techniques [START_REF] Kober | Reinforcement learning in robotics: A survey[END_REF]), various problems involving robot manipulators have been explored and successfully applied (( [Lin, 2009]), [START_REF] Deisenroth | Learning to control a low-cost manipulator using data-efficient reinforcement learning[END_REF]), [START_REF] Park | Path planning for a robot manipulator based on probabilistic roadmap and reinforcement learning[END_REF])). Among RL techniques, policy search methods seem to be the most appropriate for high dimensional robotic control problems ( [START_REF] Deisenroth | A survey on policy search for robotics[END_REF]). Recently, several researchers have proposed methods to link these two fields by using iterative Linear-Quadratic-Gaussian control (iLQG) (( [START_REF] Todorov | A generalized iterative lqg method for locally-optimal feedback control of constrained nonlinear stochastic systems[END_REF]), [START_REF] Tassa | Synthesis and stabilization of complex behaviors through online trajectory optimization[END_REF])) to compute control laws under unknown dynamics (( [START_REF] Mitrovic | Adaptive optimal feedback control with learned internal dynamics models[END_REF]), [START_REF] Levine | Learning neural network policies with guided policy search under unknown dynamics[END_REF])). This approach is interesting as it removes the need to model the dynamics, which can be difficult for complex systems and environments.

Limitations

Besides the dynamics, an optimal control problem is also defined by its cost function.

In order to find a good trajectory it is required to design a proper cost function in relation to the task to be carried out. Nevertheless, in most applications, we acknowledge that the cost function is quadratic and expressed explicitly in terms of the state and control variables. Such form of the cost function makes it easy to run the iLQG algorithm and appears to be well suited for many applications. However, in some cases, it can be useful to formulate the cost function in terms of variables that are not explicitly the state and control vectors. For example, ( [START_REF] Levine | Learning contact-rich manipulation skills with guided policy search[END_REF])

propose a cost function defined in the Cartesian space whereas the robot is controlled in joint space. In such cases, a model of the system (i.e. a direct Denavit-Hartenberg (DH) model ( [START_REF] Dombre | Robot manipulators: modeling, performance analysis and control[END_REF]) of the manipulator) is required to express analytically a quadratic expansion of the cost function, which is necessary to run iLQG. Such dependence on a robot model can be problematic since calibration is a complex and time consuming process [START_REF] Majarena | An overview of kinematic and calibration models using internal/external sensors or constraints to improve the behavior of spatial parallel mechanisms[END_REF]), [START_REF] Elatta | An overview of robot calibration[END_REF]))

and because the method is very specific to a certain system (( [START_REF] Nubiola | Absolute calibration of an abb irb 1600 robot using a laser tracker[END_REF]), [START_REF] Jubien | Dynamic identification of the kuka lwr robot using motor torques and joint torque sensors data[END_REF])). It is impossible to derive a model taking into account every dynamic phenomena. For example, if the payload on the robot is changed, the model calibration might not be valid anymore as strains might appear. In an autonomous learning framework, it does not seem appropriate to have to redesign a model after any change on the system.

Contributions and chapter organization

In this chapter, we propose a method to overcome the model dependence issue by regressing the quadratic approximation of the cost function from samples, in the same way as for the dynamics. In this way, the cost function can be defined from any measurable quantity, which allows to have a more appropriate cost in some cases. Moreover, the cost function can be defined in a more intuitive way and robot programming can become more accessible to non-specialists.

The task chosen to validate the method is a Cartesian positioning task. The robot is trained to learn how to reach a Cartesian point with its end-effector using only position sensors information and without any DH model. The cost function is the distance measured between the end effector and the target Cartesian position while the robot is controlled in joint space. To emphasize the interest of the method, we also propose to learn trajectories for target reaching with a laser pointer. For this task, the DH model cannot be computed in general because the transform to the end effector is unknown.

Two different methods are proposed to update the controller. The first one, which is a first order optimization method, is less elaborated but has less parameters to tune and is faster to implement. Therefore, the cost function learning scheme described above is validated using first-order controller updates. Using the V-REP software [START_REF] Rohmer | [END_REF]), robots with different specifications (number of joints, length of links, ...) learn the same positioning task, which illustrates that the method is independent of the robot model. We compare our results with another method where the Cartesian distance is included in the state vector and the cost is thus a function of the dynamics. Our method is shown to be more stable and to converge faster to high precision positioning. Once the method is shown to work, a second order optimization technique is implemented and its parameters are tuned using V-REP. Finally, the parameters found are tested on a real Kuka LBR iiwa robot on both the standard Cartesian positioning problem and a laser pointer target reaching task. This work received an IEEE IES Student Travel Paper Award ( [START_REF] Guérin | Learning local trajectories for high precision robotic tasks: application to kuka lbr iiwa cartesian positioning[END_REF]). A video that summarizes the key findings from this chapter is available at: https://www.youtube.com/watch?v=M9RK2XznHEE&index=4&list=

PLO5umi31c_83SMDdkk26shJZbOnbqbSpl.

The chapter is organized as follows. The derivation of the iLQG algorithm is written in Section 7.2, where both first order and second order methods are proposed to update the controller. In Section 7.3, the regression method for cost function estimation is validated through simulation. We tune the parameters for second order controller updates with V-REP simulation in Section 7.4 and in Section 7.5 we validate them experimentally on a real robot.

Trajectory optimization using ILQG

Overview of Reinforcement Learning and Trajectory Optimization

Reinforcement Learning (RL) is a subfield of Machine Learning that studies the behavior of an agent taking actions in an environment. The environment responds to each action by rewarding (or penalizing) the agent. The goal of the agent is to maximize such reward (or minimize the cost). In this chapter, we consider the episodic setting. For a better overview of the field of RL, the reader can refer to the following good survey ( [START_REF] Kober | Reinforcement learning in robotics: A survey[END_REF]). To go further, we also suggest the following textbook ( [START_REF] Sutton | Introduction to reinforcement learning[END_REF]).

At a given time t, the RL framework is composed of three main elements:

• x t , the state of the system, is a vector that contains all relevant information about the configuration of the system (or agent). For example, in our case, the agent is the serial robot and its state vector is composed of the angular joint positions.

• u t , the action taken by the agent. This control vector represents commands sent to change the state of the system. For us, u t corresponds to joint target positions.

• l t , the cost resulting from sending command u t when in state x t . In a robotic

Cartesian positioning task, the cost at time t is proportional to the distance to the target point.

With these elementary notations, we can introduce the following concepts:

• The cost and dynamics functions at time t are defined as follows:

l t = L t (x t , u t ), (7.1) x t+1 = F t (x t , u t ). (7.2)
L t outputs the cost and F t the next state, both with respect to current state and action. We note that environment and dynamics refer to the same thing.

• The controller is the function we want to optimize. For a given state, it needs to output the action with smallest cost that follows the dynamics. In our case, at time step t, it is denoted by Π t and has the special form of a time-varying linear controller:

u t = Π t (x t ) = K t x t + k t . (7.3)
Then, a trajectory τ , of fixed length T , is defined by the repetition T times of the pattern shown in Figure 7.1. Mathematically, it can be denoted by The guiding principle of iLQG is the following:

τ = {x 0 , u 0 , x 1 , u 1 , ..., u T -1 , x T }, ( 7
• From a nominal trajectory, denoted τ = {x 0 , ū0 , . . . , xT }, a new improved controller is derived by:

approximating the Taylor expansions of the cost and dynamics (Section 7.2.2),

updating the controller according to this approximations (Section 7.2.3).

• From this new controller, a new nominal trajectory is computed.

This process is repeated until a good enough controller is reached. In most application, this iterative process turns out to converge rather rapidly towards a locally optimal trajectory.

Local approximations of cost and dynamics

As explained earlier, from a given nominal trajectory τ , the goal is to update the controller such that the cost is minimized in its neighborhood. In this process, the first step is to compute local approximations of the cost function and dynamics around the nominal trajectory:

F t (x t + δx t , ūt + δu t ) ≈ xt+1 + [F xt , F ut ] δx t δu t , (7.5) L t (x t + δx t , ūt + δu t ) ≈ lt + [L xt , L ut ] δx t δu t + 1 2 δx T t , δu T t L x,xt L x,ut L u,xt L u,ut δx t δu t , (7.6)
where δx t and δu t represent variations from the nominal trajectory and capital letters indexed by one (respectively two) letters represent sub-vectors (respectively submatrices) of the appropriate Jacobian (respectively Hessian) matrices.

Different approaches

In the original paper for iLQG [START_REF] Todorov | A generalized iterative lqg method for locally-optimal feedback control of constrained nonlinear stochastic systems[END_REF]), equations (7.1) and ( 7.2) are considered known and both Taylor expansions are computed analytically. Later on, it was proposed to compute the linear dynamics through regression on observed values [START_REF] Levine | Learning neural network policies with guided policy search under unknown dynamics[END_REF]), [START_REF] Mitrovic | Adaptive optimal feedback control with learned internal dynamics models[END_REF])). This way, trajectories can be found without a model of the environment, which is difficult to have in most situations.

However, the cost function still needs to be expressed directly in terms of the state and action variables. If one wants to design a cost function from sensor measurements, it might not be straightforward to obtain the partial derivatives with respect to the state and control vectors. A good example is the task of robot positioning, where the cost function is a Cartesian distance and the variables controlled are joint positions (Section 7.3). In this case, different methods, illustrated in Figure 7.2, can be used to compute the second order Taylor expansion of the cost.

The first method (Figure 7.2b) consists in using a model of the robot to convert the angular state variables into Cartesian end-effector positions. This robot model is then used to derive the cost function with respect to the angular positions. In some practical situations, such model can be hard to obtain with enough precision, e.g.

humanoid robots with a lot of degrees of freedom. Indeed, coming up with a precise model of a robot manipulator is a hard and tedious task ( [START_REF] Majarena | An overview of kinematic and calibration models using internal/external sensors or constraints to improve the behavior of spatial parallel mechanisms[END_REF]). Moreover, the need for a model of the robot reduces the generality of the algorithm and its precision is dependent on the model, which varies from one robot to another and even with the same robot loaded differently.

As stated in the introduction, the framework for this chapter requires a modelfree method. One possibility to avoid the model is to increase the state vector with a measurement directly proportional to the cost function (Figure 7.2c). For example, for the positioning task in Section 7.3, the Cartesian distance can be considered to be part of the state. By doing this, the cost function can be expressed directly from the state and the dynamics and the derivation can be done. This trick enables to define the cost function from any measurable quantity. However, as shown in Figure 7.2c, the cost function approximation is not really quadratic as it is computed from a first order approximation of the distance.

In this chapter, we propose a different model independent approach, illustrated in Figure 7.2d, in which the cost function is directly approximated quadratically. This method is developed in the next section.

x x t , u t

Learnt linear dynamics

Learnt quadratic cost function

x t+1 l t (d)
Learning the quadratic approximation of the cost.

Quadratic cost approximation through exploration and regression

The proposed method consists in computing both approximations following an exploration and regression scheme. The first stage generates a certain number N of random trajectories around the nominal. These trajectories are normally distributed around τ with a certain time-varying covariance, denoted Σ t for time step t. Therefore, during sample generation, the controller is stochastic and follows:

Π t (u t |x t ) = N (K t x t + k t , Σ t ), ∀t ∈ {0, ..., T -1}, (7.7)
where N (µ, Σ) is the normal distribution of mean µ and standard deviation Σ. From these samples, we can make two regressions, a linear one ( [Freedman, 2009]) to get the dynamics and a quadratic one ([De Brabanter et al., 2013]) to approximate the cost function.

A quadratic regression consists in transforming the data with a second order polynomial kernel and applying a linear regression on the resulting higher dimensional data. In the general case, a quadratic expansion of the cost w.r.

] T .
This corresponds to a regression with (1

+ m + n + n(n+1) 2 + m(m+1)

2

+ m × n) learnable parameters, which is more than the other approaches where only the linear dynamics is learned ((1 + m + n) learnable parameters).

The higher dimensionality of the cost regression requires a higher number of samples for equal approximation precision, which can be an issue when data generation is expansive. However, all the samples found during the dynamics exploration phase can also be used for the quadratic regression if the cost is recorded at the same time than the new state observed. The efficiency of the methods in Figures 7.2c and 7.2d are compared in Section 7.3.

Update the controller 7.2.3.1 Compute the cost-to-go

Once the Taylor expansions of the cost and dynamics functions have been estimated, the next step is to update the controller to get lower cost over the whole trajectory.

To do so, we need to leverage the state-value (V ) and action-value (Q) functions, which are defined as follows: (7.9) where Π = {Π t , t ∈ {1, ..., T -1}} denotes the time-varying linear controller. In other words, Q Π t represents the expected cost until the end of the trajectory if following Π after being in state x t and selecting action u t . The state-value function V Π t is the same but conditioned only on x t . If Π is deterministic, these two functions are strictly equivalent. To simplify the notations, from now on, we remove the Π exponents of Q and V . Under the assumption that the trajectories are Markov Decision Process (MDP), (7.8) and (7.9) can be reformulated as:

Q Π t (x t , u t ) = E Π T j=t l j = L t (x t , u t ) + E Π T j=t+1 l j , (7.8) V Π t (x t ) = E Π T j=t l j = E Π L t (x t , Π t (x t )) + T j=t+1 l j ,
Q t (x t , u t ) = l t + V t+1 (x t+1 ), (7.10) V t (x t ) = Q t (x t , Π t (x t )). (7.11)
To lighten the derivation of the updated controller, we also define an intermediate controller that enables us to compute improvements on the time-varying linear controller Π during each improvement step:

δu t = π(δx t ) = P t δx t + p t .
(7.12)

The P t term improves the controller response to state variations and the p t term is a shift in the nominal trajectory.

To compute an improved controller, we first compute quadratic Taylor expansions of both value functions:

Q t (x t + δx t , ūt + δu t ) ≈ Q 0t + [Q xt , Q ut ] δx t δu t + 1 2 δx T t , δu T t Q x,xt Q x,ut Q u,xt Q u,ut δx t δu t
, (7.13)

V Π t (x t + δx t ) ≈ V 0t + V xt δx t + 1 2 δx T t V x,xt δx t . (7.14)
These quadratic approximation are computed with a backward recursive method and Q t can be expressed in terms of V t+1 by plugging (7.5), (7.6) and (7.14) into (7.10). This leads to the following expressions for the coefficients of (7.13):

Q 0t = lt + V 0 t+1 + V x t+1 xt+1 + 1 2 xT t+1 V x,x t+1 xt+1 , Q xt = L xt + V x t+1 F xt + xT t+1 V x,x t+1 F xt , Q x,xt = L x,xt + F T xt V x,x t+1 F xt , Q ut = L ut + V x t+1 F ut + xT t+1 V x,x t+1 F ut , Q u,ut = L u,ut + F T ut V x,x t+1 F ut , Q x,ut = L x,ut + F T xt V x,x t+1 F ut .
(7.15)

The coefficients of (7.14) are then obtained using (7.12), (7.13) and (7.11):

V 0t = Q 0t + Q ut p t + 1 2 p T t Q u,ut p t , V xt = Q xt + Q ut P t + p T t Q u,ut P t + p T t Q T x,ut , V x,xt = Q x,xt + P T t Q u,ut P t + Q x,ut P t .
(7.16) Once all the value functions have been computed backward with initial condition

V T = L T (x T
), the controller with lowest cost-to-go at each time step needs to be computed.

First order controller update

In order to improve the controller, δu t that minimizes Q t needs to be computed at each time step t. Hence, we start by isolating the terms of Q t depending on δu t in (7.13), which gives the following quadratic function to minimize:

f (δx t , δu t ) = (Q ut + δx T t Q x,ut )δu t + 1 2 δu T t Q u,ut δu t .
(7.17)

Then, we derive this function with respect to δu t :

∂f ∂δu t (.) = Q ut + δx T t Q x,ut + δu T t Q u,ut . (7.18)
Note that this partial derivative is in the Jacobian form (row wise).

To optimize the controller, several methods (first and second order) have been applied [START_REF] Todorov | A generalized iterative lqg method for locally-optimal feedback control of constrained nonlinear stochastic systems[END_REF]). In Section 7.2.3, we present an efficient second order technique, validated in Sections 7.4 and 7.5. However, to ease the characterization of the method, this section introduces a first order method, which is more intuitive. The gradient of f with respect to δu t is

∇ δut f (δx t , 0) = Q T ut + Q T x,ut δx t . (7.19)
And steps towards better controllers are made in the opposite direction of the gradient. Hence, for the same input δx t , the new chosen δu t should be

δu t = -(Q T ut + Q T x,ut δx t ). (7.20)
Obviously, the model being only valid locally, the step should be chosen small enough not to violate a "proximity to nominal" constraint.

In practice, we have implemented a slightly different method, similar to super-SAB ( [START_REF] Allard | Neural networks: different problems require different learning rate adaptive methods[END_REF]). The variance for exploration (see Section 7.3) is modified according to the evolution of the partial derivatives signs, i.e. it is increased if the sign remains the same and decreased otherwise. Then, is tuned to stay inside the variance boundaries. In this way, as we get closer to the optimal solution, the exploration gets tighter and the improvements are more accurate. This controller improvement method appear to converge within a reasonable number of steps for positioning tasks (Section 7.3).

Then, the controller can be updated to produce new commands: (7.21) where u t is the action chosen if following the current controller Π = ( K t , k t ). By definition of the current controller, we have (7.22) where x t is a slight deviation from the nominal trajectory:

u t = u t + δu t ,
u t = K t x t + k t ,
x t = xt + δx t . (7.23)
By plugging (7.12), (7.22) and ( 7.23) into (7.21), we obtain: (7.24) Finally, by adding and subtracting P t xt to the equation above, the global controller is updated as follows: (7.26) where the values of P t and p t can be found by identification between (7.20) and (7.12): (7.27)

u t = K t x t + k t + P t δx t + p t ,
K t = K t + P t , (7.25)
k t = k t + p t -P t xt ,
P t = -Q T x,ut ,
p t = -Q T ut . (7.28)
The new average trajectory is obtained by applying this new controller to the initial state.

Second order controller update

Another method, more efficient but more complex, can be implemented to update the controller. Under the quadratic value functions (7.13) and (7.14), it can be shown, by cancelling the derivative (7.18), that the optimal controller under such dynamics and cost is defined by

K t = -Q -1 u,ut Q u,xt , k t = ūt -Q -1 u,ut Q ut -K t xt . (7.29)
A criterion to compute the new covariance is also needed. The goal being to explore the environment, we follow ( [START_REF] Levine | Guided policy search[END_REF]) and choose the covariance with highest statistical entropy in order to maximize information gained during exploration. Such covariance matrix is:

Σ t = Q -1 u,ut . (7.30)
However, (7.13) and (7.14) being only valid locally, we need to limit the deviation from the nominal trajectory or the environment might respond completely differently and we might fall into bad behaviors. Such issue is solved in the next section by introducing a proximity constraints on the trajectories, thus making the choice of the initial covariance matrix of major importance. Indeed, it influences the exploration range for all the future steps: if it has large values, next iteration also needs to have large covariance. In our implementation, we start with diagonal covariance matrices (for all t ∈ {0, ...T -1}) where all the diagonal entries are the same. The choice of such diagonal entries, denoted σ ini , is studied experimentally for the task of Cartesian positioning in Section 7.4.

Limit the deviation from nominal trajectory

The controller derived above is optimal only if the dynamics and cost are respectively linear and quadratic everywhere. The approximations being only valid locally, the controller needs to stay close from the nominal trajectory after update. This problem can be solved by adding a constraint to the cost minimization problem: (7.31) where D KL is the statistical Kullback-Leibler divergence. ρ(τ ) and ρ(τ ) are the trajectories probability distributions under the current controller and the updated one, and is a user defined parameter.

D KL (ρ(τ )|| ρ(τ )) ≤ ,
In [START_REF] Levine | Learning contact-rich manipulation skills with guided policy search[END_REF]), it is shown that such constrained optimization problem can be solved rather easily by introducing the modified cost function: (7.32) where Π is the current controller (before update) and η is also user defined. Indeed, using dual gradient descent, we can find a solution to the constrained problem by alternating between the two following steps:

L modt (x t , u t ) = 1 η L t (x t , u t ) -log( Π t (x t , u t )),
• Compute the optimal unconstrained controller under L mod for a given η

• If the controller does not satisfy (7.31), increase η.

112

A large η has the effect of increasing the importance on constraint satisfaction, so the larger η is, the closer the new trajectory distribution will be from the previous one.

To evaluate (7.31), one first needs to compute D KL (ρ(τ )|| ρ(τ )). In the appendix of ( [START_REF] Montgomery | Guided policy search via approximate mirror descent[END_REF]), the authors show that

D KL (ρ(τ )|| ρ(τ )) = T t=0 E Πt(xt) D KL (Π t (u t |x t )|| Π t (u t |x t )) .
(7.33)

In the context of the time-varying linear-Gaussian policies, we show that this formula can be written as follows:

D KL (ρ(τ )|| ρ(τ )) = 1 2 T t=0 Tr(A t Σ xt ) + µ T xt A t µ xt + 2µ T xt b t + c t , (7.34)
where A t , b t , c t , µ xt and Σ xt (the mean and covariance of the state distribution at time step t) are computed recursively from the initial state distribution, the Taylor expansions of the dynamics and the expression of the controller. A complete proof as well as the exact recursive formulas for A t , b t , c t , µ xt and Σ xt are given in Appendix F.

Initialize η and choose

The way Q is defined from approximation does not guarantee positive definiteness for Q u,ut , i.e. it might not be eligible to be a covariance matrix. This issue is addressed by increasing η such that the distribution is close enough from the previous one. As the previous trajectory has a positive definite covariance, there exists a η that enforces positive definiteness. This gives a good way to initialize η for a given pass. Finally, the choice of is also paramount. If it is too small, the controller sequence will not progress towards optimality and if it is too large, it might be unstable. The idea is to start with a certain ini and decrease it if the new accepted controller is worse than the previous one. The choice of ini is studied experimentally for the task of Cartesian positioning in Section 7.4.

Simulation validation of quadratic regression for cost computation

In order to validate the quadratic regression method to compute the cost, we start by implementing the simple gradient descent update on the V-REP software.

Problem definition

In order to show the effectiveness of the method, a robot manipulator positioning task has been implemented using the V-REP simulation software. The robot is controlled in angular position, the state vector is the vector of angular joint coordinates and the control vector is the desired angular positions for the next step. The goal is to reach a target Cartesian point with the end-effector of the robot. The cost function is expressed naturally as the Cartesian distance between the connector and the target. If one want to solve such control problem with classical methods, he first needs to build a model of the robot (e.g. DH), giving the end-effector position w.r.t. joint coordinates. Indeed, to be able to get the quadratic approximation of the cost, one needs to map analytically the joint space to the Cartesian space in order to differentiate it. Obviously, no such model has been derived for the simulation and we implemented the algorithm on four industrial robots with different specifications (number of links, lengths of links, ...) to illustrate that the method is model-free (Figure 7.3). Testing any additional new robot would take only a few additional minutes of work as there is no need to explicit a DH-model. Thanks to this method, we are free of modelling imprecision and only sensor precision is involved in the learning process.

It is also worth underlining that the task would be similar if fixing a reference frame to the end-effector and adding orientation constraint. All that is needed is to take distances between three points instead of only one.

Trajectory found on different robots

The nominal trajectory used to initialize the iLQG algorithm was chosen randomly. We chose to set all desired joint positions to 60 • and explore around this initial command. Figure 7.3a represents the initial nominal trajectory. Note that the angular point-to-point (PTP) movement between two joints configuration is considered as a black box, part of the V-REP environment. In this way, the dynamics can really be considered unknown. On the other hand, Figure 7.4 also shows that if the number of samples gathered is too small (e.g. 5), the cost and dynamics will not be approximated well and the algorithm might not converge towards the desired solution.

Number of samples needed

Comparison with state modification method

As mentioned in Section 7.2, another method to avoid robot modeling consists in including the Cartesian distance in the state vector. This technique corresponds to diagram 7.2c of Figure 7.2. In this section, the proposed quadratic regression method is compared with this common technique. Figure 7.5 illustrates the learning curves for both methods under different learning sample sizes. We acknowledge that state modification is less stable. Indeed, in this simulation, results with 30 samples are better than the ones with 40, which does not happen using quadratic approximation. Moreover, the state modification does not show the nice property of being independent of the number of samples when this number becomes high (< 40). Finally, the quadratic regression appears to be more efficient in the final stages of the algorithm, which might come from the fact that a better model of the cost is required to reach high precision in the positioning.

Parameters tuning for second order methods

Now that we have seen that the method is working and overall better than the other model free method, we try to implement a more complex and elaborated controller update. We use the second order scheme defined in Section 7.2.3.3 and a more complex cost function in order to boost the learning of the positioning task. Such inverse kinematics task is only carried out with the KUKA LBR iiwa robot Figure 7.6.

Cost function

For this problem, the cost function needs to be expressed in terms of the Cartesian distance between the end-effector and the target point. We chose the more elaborated cost function proposed in ( [START_REF] Levine | Learning contact-rich manipulation skills with guided policy search[END_REF]):

l(d) = d 2 + v log(d 2 + α), (7.35)
where v and α are both real user defined parameters. The squared distance term encourages fast convergence to the neighborhoods of the target and the log term encourages precise positioning. Hence, v is a trade-off parameter between the two penalties and α is a small positive value that ensures numerical stability. As we do not consider any geometric parameter of the robot, the distance cannot be obtained with direct model considerations and needs to be measured from sensors.

Tune the algorithm parameters

Section 7.3 shows that a number of samples around 40 is a good balance between accurate quadratic regression and exploration time for 7 d.o.f. robots. So we carry out our experiments with N = 40. Then, the learning process depends on four parameters, which are studied experimentally in this section:

• the initial covariance, σ ini , which corresponds to the diagonal entries of the covariance matrices of the initial time-varying controller (Section 7.2.3.3),

• the v and α terms from the cost function (Equation 7.35),

• the initial upper bound for the KL divergence between the current and updated trajectories, ini (Section 7.2.3.5).

Different values of these parameters are tested in simulation using V-REP on the following positioning task:

• Initial position : All 7 angles at 0 (straight position on Figure 7.6)

• Target position : Cartesian vector [500, 500, 500] T in mm, in the robot frame (red sphere on Figure 7.6)

• Initial mean command : target angular positions = initial positions (no move command).

Figure 7.6 shows a trajectory found by the algorithm.

Results and analysis

After a preliminary study, we choose three values for each parameter and all the 81 possible combinations are tried to choose a good set of parameters for positioning tasks. Results obtained are summarized in Table 7.1. In our simulation, the robot was allowed only 16 trials to reach a precision of 0.1 mm. Thus, we insist that in Table 7.1, an underlined number represents the number of iLQG iterations before convergence whereas other numbers are the remaining distance to objective after 16 iterations.

Table 7.1: Influence of the four user defined parameters on the convergence of the method. An underlined number represents the number of iLQG iterations to reach 0.1mm precision, Other numbers represent the distance remaining after 16 iterations.

σ ini = 1 v ini α 10 -3 10 -5 10 -7 0.
Results in Table 7.1 suggest that the best set of parameters for the positioning task is σ ini = 1, v = 0.1, α = 10 -7 and ini = 10000. Indeed, these parameters have the smallest number of iterations and are thus chosen for the experimental validation in section 7.5.

Together with the raw data in Table 7.1, we also plot the evolution of the distance within the iterations of a simulation for several sets of parameters. Looking at Table 7.1, it seems that the most critical parameter is σ ini . Figure 7.7 shows three learning curves where only σ ini varies, all the other parameters are set to their median value. From here it appears that the initial covariance is not crucial in the early stages of the learning process. However, the right plot, which is a zoom on the final steps, reminds that if the covariance is too large, the algorithm will not converge towards the desired accuracy behavior. Hence, as already suggested by Table 7.1, keep σ ini around 1 seems to be a good choice to obtain the desired accurate behavior. After setting σ ini to 1, we draw the same plots for the other parameters in Figure 7.8. These reveal that v and α do not appear to influence the behavior in this range of values. However, looking at Figure 7.8c, we can see that ini needs to be kept large enough such that an iLQG iteration can make enough progress towards optimality. For small ini , convergence is slower near the initial configuration. • Initial pose : [140, 0, 0, 0, 0, 0, 0] T , angular positions in • (Figure 7.9a),

Experimental validation for the method

• Target position : [-600, 400, 750] T , Cartesian position in mm expressed in the robot frame (Position of the end effector on Figure 7.9b),

• Initial mean command : target angular pose = initial pose (no move).

The choice of changing the initial configuration is motivated by two reasons:

• It shows that the parameters found in Section 7.4 are not case dependent,

• This setup reduces the risk of collision with the working environment.

Results

The learning process defined above resulted in the learning curve on Figure 7.10. We note that it takes as many steps to go from the initial configuration to 1 mm accuracy than from 1 mm to 0.1 mm. The final command, i.e. target angular configuration, provided by the algorithm is [144.266, 25.351, 2.328, -56.812, 5.385, 24.984, 4.754] T .

Regarding the learning time, the overall process took approximately 9 minutes, 6 for exploration and 3 for calculations. 122 On this experimental validation, the distance is computed from the end-effector position read from the robot internal sensors. Even if it was probably computed thanks to a direct DH model, our algorithm uses it as a black box sensor measurement. Thus, similar results would have been obtained using any other distance measurement sensor (e.g. laser tracker) instead of using the internal robot variables. We just note that, the precision reached is relative to the precision of the measurement tool.

Qualitative validation on a case where no DH model is available

A qualitative validation on a case for which the direct geometric model of the system cannot be computed is proposed in this section. The robot, equipped with a twofinger gripper, is positioned in a room at a random and unknown location. Then, a laser pointer is placed in the gripper in a random orientation and the end-effector is defined by the projection of the laser on a given white wall of the room. A target is stuck on this wall and the objective is to find a robot configuration to reach its center. A camera continuously streams the distance (in pixels) between the red dot and the center of the target thanks to a simple computer vision pipeline. This distance information is used to compute the cost function and provide feedback to the robot.

The different elements described above and the initial setup for this tasks can be seen in Figure G.29a. For this task, the mapping from joint positions to end effector positions cannot be computed because both the location of the robot and the orientation of the laser pointer are unknown. Hence, this example is a good case study to test iLQG with learned dynamics and cost function. The proposed approach converges towards the center of the target in approximately two minutes at a precision of around 2 mm, which is the precision of our simple external measurement system. The final configuration reached can be seen in figure G.29b and a video of the robot solving the task is available at: https://www.youtube.com/watch?v=Ekda9q3vv6Y. This implementation demonstrates qualitatively the interest of the method by solving a task where obtaining a model is really hard and we can see that our approach is not impacted by such absence of model.

123

Conclusion

In this chapter, we have introduced a modified version of iLQG to solve the locally optimal trajectory learning problem. This approach consists in regressing the second order Taylor expansion of the cost function from data measurements. Compared to standard ways of computing the cost function, this method has the advantage of being independent from the model of the system and avoids nested approximations. To validate the proposed method, we have studied experimentally the task of Cartesian positioning for serial robots, without using the forward model. We assume that during learning, the robot can only access its angular joint positions and the Cartesian distance between its end-effector and its target, measured by an external sensor.

The simulated experiments conducted in Section 7.3 demonstrate that learning the quadratic cost function is more stable and converges faster to high precision trajectories than including distance as a state variable. Simulation is also leveraged to learn a good set of hyperparameters for second order controller improvements.

These parameters are then shown to work for a real robot for a standard positioning task. The high precision reached for this simple positioning task let us hope that such methods will be suitable for more complex industrial tasks. Finally, we demonstrate the model independence of the approach by solving a target reaching task with a laser pointer, for which a direct model cannot be computed in general. Perspectives and future work regarding trajectory learning are discussed in Chapter 9.

Chapter 8

Automatic Construction of Real-World Datasets for 3D Object Localization using Two Cameras Abstract Similarly to object understanding and trajectory learning, object localization is an elementary robotic skill that is use in the conceptual definition of many robotic applications, including robotic sorting. Unlike classification, position labels cannot be assigned manually by humans. For this reason, generating supervision for precise object localization is a hard task. This chapter details a method to create large datasets for 3D object localization, with real world images, using an industrial robot to generate position labels. By knowledge of the geometry of the robot, we are able to automatically synchronize the images of the two cameras and the object 3D position.

We applied it to generate a screw-driver localization dataset with stereo images, using an industrial robot. This dataset could then serve to train a CNN regressor to learn end-to-end stereo object localization from a set of two standard uncalibrated cameras.

Introduction

Like object understanding and trajectory learning, object localization is an elementary brick which can serve in building many robotic applications and can be made more autonomous and easier to "program" by unqualified workers. Hence, this chapter addresses the problem of 3D object localization and introduces a method for automatic generation of large stereo localization datasets. By leveraging the precision and repeatability of industrial robots, the proposed methodology enables to save labor time and increase precision for datasets construction. This chapter is an ongoing research and these datasets should be used for training end-to-end stereo localization networks to validate the methodology. However, we believe that this chapter introduces some interesting concepts about dataset construction that can be useful to the community. We now propose a more specific introduction to the topic of 3D localization using a system of stereo cameras.

In the context of autonomous manipulation, 3D object localization plays an essential role. Stereo vision is one of the most efficient methods for 3D reconstruction and thus is a very useful tool when dealing with robotic manipulation. Indeed, stereo vision has received many attention in research over the past decade in different subfields of autonomous manipulation such as grasping [START_REF] Azad | Stereo-based 6d object localization for grasping with humanoid robot systems[END_REF]), [START_REF] Morales | Integrated grasp planning and visual object localization for a humanoid robot with five-fingered hands[END_REF])), contact-reach tasks ( [START_REF] Hudson | End-to-end dexterous manipulation with deliberate interactive estimation[END_REF]), and even playing soccer ( [START_REF] Käppeler | 3d object localization via stereo vision using an omnidirectional and a perspective camera[END_REF]).

However, classical methods of stereo vision are hard to design and require a lot of tuning. They can be inaccurate as they are composed of many steps, which are all potential sources of errors (see Section 8.2). For this reason, we strongly believe that stereo localization would benefit from an end-to-end learning approach, which would simplify the design process and hopefully improve the system accuracy. Recently, several end-to-end approaches have been successful in achieving complex tasks, such as robot manipulation ([Levine et al., 2015a]), self-driving cars ( [START_REF] Bojarski | End to end learning for self-driving cars[END_REF]), speech recognition ( [START_REF] Amodei | Deep speech 2: End-to-end speech recognition in english and mandarin[END_REF]) or obstacle avoidance ( [START_REF] Muller | Off-road obstacle avoidance through end-to-end learning[END_REF]).

Even for end-to-end learning of manipulation tasks, which includes object localization, the vision part must be pretrained to locate objects if we want reinforcement learning to be scalable to real life applications ([Levine et al., 2015a]). Hence, this work, which mainly focuses on end-to-end learning of 3D stereo object localization, also has great significance within the wider context of robotic autonomous learning of manipulation tasks.

To train a regression model (e.g. convolutional neural network) to learn stereo localization, we need to generate labeled data for localization, i.e., stereo images of the object to be located together with its position in space. Gathering labels for localization is a hard task. Position labels are hard to get as they cannot be written manually without spending precious time doing very precise measurements for each sample. In this chapter, we introduce an approach for building such a dataset, by using an industrial robot to generate labeled data for 3D stereo localization. The main contribution of this chapter is to describe a procedure to gather a lot of labeled stereo data for automatic object localization. This procedure is applied to generate a dataset for screw drivers 3D localization, composed of more than three thousand samples. To provide a baseline for future research to test regression models for end-toend stereo localization, this dataset is made publicly available and can be downloaded at https://goo.gl/stu5UE.

Motivations for end-to-end stereo localization 8.2.1 Classical stereo vision methods for 3D localization

The use of stereo vision has a long history in robotics manipulation [START_REF] Azad | Stereo-based 6d object localization for grasping with humanoid robot systems[END_REF]), [START_REF] Hudson | End-to-end dexterous manipulation with deliberate interactive estimation[END_REF])). However, currently, stereo localization consists in stacking different methods that can each be inaccurate. As we can see in Figure 8.1, to implement 3D object localization, it is needed to calibrate both cameras, compute accurate stereo matching, build a computer vision pipeline to identify pixels of interest, triangulate, and measure accurate transformation of frames to project the result in the frame of interest.

More recently, researchers have started to use deep learning for stereo vision (( [START_REF] Luo | Efficient deep learning for stereo matching[END_REF]), [START_REF] Knöbelreiter | End-to-end training of hybrid cnn-crf models for stereo[END_REF]), [START_REF] Zbontar | Stereo matching by training a convolutional neural network to compare image patches[END_REF])). However, they mainly focus on stereo matching, which is only solving part of the problem. This trend can also be seen by looking at the problem proposed with the most famous datasets for stereo vision [START_REF] Geiger | Are we ready for autonomous driving? the kitti vision benchmark suite[END_REF]), [START_REF] Scharstein | High-resolution stereo datasets with subpixel-accurate ground truth[END_REF])). To locate an object in space, one also needs to get the precise pixels in one of the image, which can cause errors, even with perfect stereo matching. Indeed, this problem is close to instance segmentation [START_REF] Romera-Paredes | Recurrent instance segmentation[END_REF]) and is not easy. In general, end-to-end learning approaches tend to be better than stacking subsystems as it can correct internal errors. For this reason we introduce such a framework for stereo localization in the next section. This chapter is a first step towards implementing it as right image left image disparity map depth map

x i y i image   x i y i z i   cam   x i y i z i   robot stereo matching triangulation computer

vision algorithm Change of frame

To encode in the Network it proposes a generic way to produce real-world datasets for stereo object localization using a robot to generate position labels.

End-to-end pipeline for stereo localization

An end-to-end pipeline for stereo object localization would consist in mapping pixels from two images to 3D points in space representing an object pose, using a regression function approximator. Such a pipeline could then be used for grasping, or any robotic manipulation task.

In this chapter, we propose an approach to build datasets for stereo localization learning. Showing the feasibility of gathering large datasets is a first step towards investigating the feasibility of end-to-end object stereo localization. The purple dotted line in Figure 8.1 illustrates what the network is supposed to encode, the relative camera calibration could be added to this frame. After seeing the excellent results produced by CNNs o, various tasks dealing with images, it seems that only the difficulty to gather enough labelled data prevents them to carry out end-to-end stereo localization. The idea implemented in this chapter consists in defining a method, using a precise and accurate industrial robot, to generate labelled data for object localization. We also apply it to the get data for screw-drivers localization. Section 8.3 contains more details about the dataset generation.

Dataset generation

When building a dataset for supervised learning, it is crucial to make sure that our input data are sufficient to know everything about our outputs. For the case of stereo object localization, all that is required is to have two cameras with fixed focal length and relative positions. We also need to keep the position of the two cameras fixed with respect to the robot and to make sure that both cameras are oriented such that they share partially common fields of view, i.e. the object must be seen by both cameras.

Generate diversity to avoid overfitting

In order to avoid overfitting, the dataset should have as much diversity as possible.

To do so, we try to change the following parameters:

• Lighting conditions, by gathering the data in a shop-floor with a glass roof,

• Background, by placing different other objects in the scene.

We also add images where distractors (other tools) are placed in the background. This way, the CNN must learn to locate screwdrivers and not any object. The dataset contains different screwdrivers to help discover the concept of what makes a screwdriver.

Finally, we make sure that the random configurations of the robot explore the full range of positions and orientations allowed within the common view range of the cameras. Figure 8.2 shows a representative subset of the images present in the dataset. Note that each image shown has a corresponding image from the other camera.

Avoid learning the wrong thing by removing the robot

The idea of using a robot to generate supervision opens a broad range of possibilities for object localization. However, the robot, or at least the tool holder, will appear on every image, which is a drawback of the approach. Indeed, we can fear that the regressor learns to locate the robot and just applies some kind of shift to find the object to be located. To avoid such problem, we need to remove the robot from some of the images. We proceed in two different ways:

• Physically, we hide the robot by wrapping some cloth around it (Figure 8.2)

• Computationally, using a computer vision pipeline to remove the robot from images. To do so, we increase the dataset and take four images per sample: robot + tool, robot alone, and the backgrounds corresponding to the two situations.

The full pipeline is described in Figure 8. The underlying idea behind the robot removal computer vision algorithm is to use the robot only image and the background to compute a mask of the robot and replace the corresponding pixels by background pixels. A similar approach has been proposed by [START_REF] Levine | Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection[END_REF]). The necessity to get two background images comes from the changes in lighting conditions. This algorithm is then fine tuned to remove only the good pixels and get smooth edges.

To ensure that all the images in our dataset do not contain any systemic patterns coming from our robot removal algorithm, it is also important to keep original images (with the robot).

Technical details on data generation

For tool calibration, we assume that the screw driver is orthogonal to the tool holder and we use the very accurate torque sensors of the Kuka LBR iiwa robot to detect when contact is reached with a plaque in several directions. In this way, we can get all the shifts and compute the transformation of frames. This procedure is fully automated and the operator just needs to place the screw driver in the clamp.

When we generate data, we make sure that the object stands within a certain cuboid (the biggest inside the common views), that the robot is not hiding the object in the image, and obviously, that the robot is not colliding with itself or the environment.

We also check, for the cases where we take several images, that the robot is not behind the screw driver so that there is no mask superposition and we do not remove part of the tool with our computer vision algorithm.

Conclusion

In this chapter, we described a procedure to build a dataset for 3D object localization using stereo vision. By using an industrial robot, we have shown that it is possible to generate data with accurate position labels. We propose an approach to generate data with enough variability, to avoid overfitting, as well as a method to remove the robot from some of the images to prevent our regressor from learning the wrong thing. This

Key contributions

Modern industrial robots are flexible machines that can be reprogrammed without mechanical modification and are highly instrumented. These characteristics enable them to successfully achieve a large variety of tasks. However, their current use in industry is still far from the level of autonomy and adaptability that we can expect from such complex systems. In this manuscript, we have proposed several contributions in machine learning to start bridging this gap and to increase robustness of robotic applications to different setups. These contributions are mostly centered around the unsupervised sorting task but have applications in many other domains of robotics.

After introducing the new application of Unsupervised Robotic Sorting (URS), we proposed a first implementation consisting in sorting objects based on their shapes and colors in an unsupervised fashion. This adaptation of a common robotic sorting pipeline to the unsupervised setting lead to the development of a new clustering algorithm that we called Gap-Ratio K-means and that appears useful for noisy datasets containing interval scale data. Then, to enable our sorting robot to cluster objects with a higher level of abstraction, the problem of image clustering has been studied. The current state-of-the-art methods for complex datasets are using Convolutional Neural Networks (CNN) pretrained on Imagenet to extract features from complex images. For this reason, we first carried out a benchmark study to evaluate how well the different CNN architectures and layers transfer to new unsupervised image datasets. The absence of strong trend in the results motivated the development of an ensemble clustering approach to leverage information from several of such CNNs. This new approach was shown to be state-of-the-art at image clustering on several public datasets.

After developing good image clustering pipelines on standard datasets, we proposed to embed them in practical URS implementations. A first implementation has been created in a shopfloor environment and its robustness to various external factors has been tested. The results from this study suggested that the quality of the sorting is highly dependent on the point of view under which the objects are observed. This finding motivated the development of an optimal view selection method, consisting in creating a large multi-view dataset and use it to train a neural network to choose optimal camera poses. This approach was shown to perform better than fixed and random camera poses on classes of objects that were not seen at training time, which suggests that the network has learned to use geometric features for view selection.

Besides the crucial skill of object understanding, we introduced contributions in two other areas of robotics, which are also essential to improve robustness and adaptability of industrial applications. On the one hand, a standard trajectory learning method was adapted to learn a quadratic model of the cost function instead of computing it analytically. This way, the method becomes independent of the robot's model, thus making the system easier to program in more situations. On the other hand, a method using a robot manipulator to autonomously produce datasets for 3D stereo localization was developed. This technique was used to create one such dataset for screwdriver localization, which was released to the community.

In summary, the different contributions presented in this manuscript address different tasks, which are all important skills for industrial robotics. Furthermore, we would like to emphasize that all the proposed methods have the same underlying purpose: increasing the validity range of the different skills without reprogramming.

Indeed,

• unsupervised robotic sorting generalizes standard sorting applications to any set of objects, removing the need to reconfigure the application when the classes change,

• optimal view selection has the potential to make robotic vision systems independent of the observed objects poses,

• the proposed trajectory learning method is generic and does not need to be adapted to the learning agent or its configuration,

• autonomous dataset generation can possibly be an interesting method for developing new specific industrial vision applications with minimal human efforts.

9.2 Directions for future work

Gap-Ratio K-means algorithm

The clustering algorithm proposed in Chapter 2 demonstrated promising results on the unsupervised robotic sorting from color and shape features. Furthermore, the additional experiments carried out in Section 2.4.5 suggest that Gap-Ratio (GR)

K-means can also work for datasets such as Iris, that do not present the kind of features specific to the URS implementation. Based on this observation, it would be interesting to investigate further the scope of validity of GR K-means and to try to find a statistical test to choose if GR K-means should be used over other variants.

Another interesting extension of GR K-means would be to combine it with data orthogonalization methods (such as ICA ( [START_REF] Oja | A fast fixed-point algorithm for independent component analysis[END_REF])). Indeed, GR weights are computed along different dimensions of the feature space and if features have strong correlation, gaps might disappear and variance might be spread along several dimensions. For this reason, it seems appealing to try to decorrelate data using orthogonalization methods.

Finally, it could also be interesting to consider not only the largest gap along one dimension but also the next ones, according to the number of different classes desired. Indeed, if within three classes the two separations come from the same features, even more importance should be given to this set of features. Some modifications of the equations in Section 2.3 should enable to try such approach.

Image clustering

Experimental results from Chapter 4 illustrate that different CNNs, pretrained on the same task, often contain different and complementary information about a target dataset. Differences may arise from a number of sources including the architecture (number of layers, layer shape, presence of skip connections, etc.), the regularization method, or the loss functions used for training. Investigating which parameters influence knowledge transfer to unsupervised tasks is an interesting axis of research for future work and such knowledge may help to design better CNN architectures.

We also note that pretrained CNNs are used as feature extractors for many applications, not just clustering. Using multiple pretrained CNNs to define a multi-view learning problem may be appealing for other tasks where complementary information present in pretrained feature extractors can improve performance. Although JULE [START_REF] Yang | Joint unsupervised learning of deep representations and image clusters[END_REF]) appears to be a good algorithm to solve MVC, we acknowledged that it fails for some datasets. Hence, it would be interesting to try different deep clustering methods or to adapt the parameters of JULE for different IC tasks. Finally, as the number of available pretrained CNNs will keep increasing and many researchers have limited resources, the case where the number of available feature extractors is much larger than the number of available GPUs needs to be considered. Hence, investigating an optimal strategy for architectures selection and resources allocation to maximize the available information is a promising research direction.

Unsupervised robotic sorting

The experiments carried out in Chapter 6 demonstrated the positive impact of semantic view selection on the URS results. Likewise, in Chapter 5, we showed that using multiple viewpoints of an object can help to understand its nature. Hence, a natural perspective would be to extend the semantic view selection problem to the one of multi-view selection. Implementing such a multi-view approach in the formalism of the "next best view selection" literature ( [START_REF] Krainin | Autonomous generation of complete 3d object models using next best view manipulation planning[END_REF]) would also allow to detect when the top view is already good, thus avoiding unnecessary computation.

The dataset presented in Chapter 6 could be used to train a recurrent neural network to address this problem.

Now that the decision making module of the unsupervised sorting application has been shown to work, it will also be interesting to include it in a more complete pickand-place pipeline, which handles automatic scene segmentation, objects localization and robotic grasp detection.

Trajectory learning

To further validate the quadratic cost regression approach, several research perspectives can be considered. On the one hand, it would be interesting to use an external measurement tool for the basic positioning task in order to compare our positioning method precision with other techniques. Indeed, the precision of the inverse kinematics of the robot cannot be defined with internal robot measurements. On the other hand, it would also be interesting to study the behavior of the method for more complex manipulation tasks involving contact.

Another directions of improvement of the proposed trajectory learning method is to try to decrease sample complexity. Indeed, the cost quadratic regression has more learnable parameters than the dynamics linear regression and thus requires more examples to produce a valid estimate. This involves additional exploration time. To solve this issue, we could leverage the approach proposed in ( [START_REF] Levine | Learning neural network policies with guided policy search under unknown dynamics[END_REF]), which consists in building a prior on the regressed variables in the form of a Gaussian mixture model. We conducted preliminary experiments with this method and the first results suggest that the overhead exploration cost due to quadratic regression can be highly attenuated with such Bayesian regression approach.

We finally note that the local trajectory learning problem studied in Chapter 7

is an important step in the Guided Policy Search (GPS) algorithm [START_REF] Levine | Learning contact-rich manipulation skills with guided policy search[END_REF]), [START_REF] Levine | Learning neural network policies with guided policy search under unknown dynamics[END_REF])). GPS is a very promising method that can learn successful visuomotor policies for complex robotic tasks by using learned timevarying linear-Gaussian trajectories to guide the training of a complex neural policy.

Hence, embedding our method in the framework of GPS is an interesting direction for future research because the improvements proposed on the trajectory optimization method could improve and accelerate GPS.

Stereo localization

The logical continuation of this work on autonomous datasets generation is to exploit the screwdriver localization dataset that was created and train a CNN architecture to encode the 3D stereo localization pipeline. Another perspective is to apply this process as a preprocessing step for reinforcement learning of manipulation tasks, likewise in ( [Levine et al., 2015a]). By doing this, we can expect to get better results with a policy using stereo images instead of a single camera.

Part VI Appendix

wrote the first self-learning program in 1959, defines ML as the "field of study that gives computers the ability to learn without being explicitly programmed". In other words, a ML program is not written for a specific task but for a class of problems; it can modify itself to solve different problems of the same nature. This very generic definition contains all the subfield of ML which we are about to define.

There exists many different ways of grouping different ML algorithms. One possible classification is to consider the kind of inputs received by the algorithm and the outputs it produces. From there, we can define three main categories:

• Supervised Learning: From a training set, composed of pairs of data objects and desired outputs, a supervised learning algorithm learns general rules to map inputs to outputs. Such rules should generalize to predict outputs for previously unseen data. It is called supervised learning because it generalizes from known examples. The classification of apples and oranges described above is an example of supervised learning.

• Unsupervised learning: The goal of unsupervised learning is mainly to group data based on their similarity. Inputs do not have labels, the idea is to maximize similarity between objects in the same groups. For instance, finding two groups among a set of fruits features, without initial labelled examples is an unsupervised learning problem.

• Reinforcement learning: The last major area of ML is Reinforcement Learning (RL). It serves as a tool to solve control problems and games. The RL framework is composed of an agent, who evolves by taking actions in an environment. The environment responds by sending a score to reward (or punish) the agent according to a user defined criteria. The rewards measure how good the agent's action are with respect to the task at hand. Then, by interacting with the environment, the agent should change its behavior to choose actions which maximize reward over its lifetime. In other words, RL consists in learning through interaction. A good text book to better understand RL concepts is ( [START_REF] Sutton | Introduction to reinforcement learning[END_REF]).

Although less frequent, we also find the field of semi-supervised learning, which mixes labelled and unlabelled data, as well as active learning, where the algorithm can request specific data from the user.

When defining ML, it is also interesting to consider the kind of problems it can solve. Apart from the RL problems, which were already introduced, two other main problems are solved using ML:

• Classification: Given a data object (vector, matrix, pixels, etc.), the goal of classification is to identify the group it belongs to. A good definition of the classification problem as well as tools to solve it can be found in ( [START_REF] Theodoridis | Introduction to pattern recognition: a matlab approach[END_REF]). Classification can be either supervised or unsupervised, in which case it is also called clustering.

• Regression: Regression, or function estimation, consists in finding the output of a function for a given input. More details can be found in ( [START_REF] Vapnik | Statistical learning theory[END_REF]). If the outputs are finite and discrete, a regression problem becomes a classification problem, however, the special structure of classification problems is widely used and should be kept as a separate problem. Most of the time, regressions with continuous outputs are supervised learning problems.

We conclude this brief ML introduction with a word about deep learning, a subfield of supervised learning which has received much attention lately. As shown in the initial example of apples and oranges, ML is an interesting tool to automate model creation and parameters tuning, it is a nice way to enable people with little knowledge to create usable, case specific programs. In addition to this nice application, some ML algorithms are also able to learn things humans cannot program. For example, learning functions to associate raw images pixel representations to categories. The most widely used tool for such application are artificial Neural Networks (NN) ( [Yegnanarayana, 2009]) and more particularly deep neural networks [START_REF] Lecun | Deep learning[END_REF]). Such NN models can be arbitrarily complex and are able to discover highly complex models and structures in the data if provided with enough examples.

A good application to illustrate deep learning is image classification. For example, using deep NN with image pixels as inputs [START_REF] Szegedy | Rethinking the inception architecture for computer vision[END_REF]), [START_REF] Huang | Densely connected convolutional networks[END_REF])), very low error rates can be reached for the problem of ImageNet classification ( [START_REF] Russakovsky | Imagenet large scale visual recognition challenge[END_REF]). ImageNet is a dataset composed of over one million high-resolution images, which needs to be classified within 1000 classes. Such deep NN predictors are composed of tens of millions of parameters, and thus are impossible to code manually. With these notations, the trajectory distributions can be written

ρ(τ ) = ρ(x 0 ) T -1 t=0 Π(u t |x t )ρ(x t+1 |x t , u t ), (F.5) ρ(τ ) = ρ(x 0 ) T -1 t=0 Π(u t |x t )ρ(x t+1 |x t , u t ). (F.6)

F.2 Trajectories divergence in terms of conditional action distributions divergences

In ( [START_REF] Montgomery | Guided policy search via approximate mirror descent[END_REF]), the authors show that KL divergence between the two trajectory distributions can be written as

D KL (ρ(τ )|| ρ(τ )) = T -1 t=0 E ρ(xt) D KL (Π(u t |x t )|| Π(u t |x t )) , (F.7)
where ρ(x t ) is the probability distribution of the state at time step t under the controller Π. Their proof is the following:

Proof of Formula (F.7). By definition of the KL divergence, we can write

D KL (ρ(τ )|| ρ(τ )) = E ρ(τ ) [ln(ρ(τ )) -ln( ρ(τ ))] . (F.8)
Then, by expanding the trajectories (F.5) and (F.6) in (F.8) and by cancelling the initial state and dynamics, we get

D KL (ρ(τ )|| ρ(τ )) = E ρ(τ ) T -1 t=0 ln(Π(u t |x t )) -ln( Π(u t |x t )) . (F.9)
By linearity of the expectation, (F.9) becomes

D KL (ρ(τ )|| ρ(τ )) = T -1 t=0 E ρ(xt,ut) ln(Π(u t |x t )) -ln( Π(u t |x t )) . (F.10)
Since ρ(x t , u t ) denotes the joint distribution of states and actions at time step t for trajectory distribution ρ, it can be written as ρ(x t , u t ) = ρ(x t )Π(u t |x t ). Hence, (F.10) becomes

D KL (ρ(τ )|| ρ(τ )) = T -1 t=0 E ρ(xt) E Π(ut|xt) ln(Π(u t |x t )) -ln( Π(u t |x t )) . (F.11)
Finally, we note that the definition of the KL divergence between Π and Π at time step t is defined by

D KL (Π(u t |x t )|| Π(u t |x t )) = E Π(ut|xt) ln(Π(u t |x t )) -ln( Π(u t |x t )) .
(F.12) Substituting (F.12) into (F.11) leads to Formula (F.7).

F.3 Expected conditional actions distributions divergence in terms of state distributions

For all t in {0, ..., T }, the expected value of the KL divergence between the conditional action distributions in (F.7) can be expressed by

E ρ(xt) D KL (Π(u t |x t )|| Π(u t |x t )) = 1 2 Tr(A t Σ xt ) + µ T xt A t µ xt + 2µ T xt b t + c t , (F.13)
where µ xt and Σ xt are respectively the mean and variance of the state distribution at time step t (see Section F.4), and where

A t = ( K t -K t ) T Σ -1 ut ( K t -K t ), b t = ( K t -K t ) T Σ -1 ut ( k t -k t ), c t = α t + β t , α t = ln | Σ ut | |Σ ut | + Tr( Σ -1 ut Σ ut ) -dim(u t ), β t = ( k t -k t ) T Σ -1 ut ( k t -k t ). (F.14)
And where Tr(.) represents the trace of a matrix, |.| its determinant and dim(.) the dimension of a vector.

Proof of Formula (F.13). Using the formula of KL divergence between two multivariate Gaussian distributions we have

E ρ(xt) D KL (Π(u t |x t )|| Π(u t |x t )) = E ρ(xt) 1 2 ln | Σ ut | |Σ ut | + Tr( Σ -1 ut Σ ut ) -dim(u t ) + ( K t -K t )x t + ( k t -k t ) T Σ -1 ut ( K t -K t )x t + ( k t -k t ) . (F.15)
By linearity of the expectation and because α t , defined in (F.14), does not depend on

x t , we can write

E ρ(xt) D KL (Π(u t |x t )|| Π(u t |x t )) = 1 2 (α t + Θ t ), (F.16) where Θ t = E ρ(xt) ( K t -K t )x t + ( k t -k t ) T Σ -1 ut ( K t -K t )x t + ( k t -k t ) . (F.17)
By expending (F.17), we can leverage the linearity of expectation and the symmetry of covariance matrices to obtain

Θ t = E ρ(xt) x T t ( K t -K t ) T Σ -1 ut ( K t -K t )x t + 2E ρ(xt) x T t ( K t -K t ) T Σ -1 ut ( k t -k t ) + ( k t -k t ) T Σ -1 ut ( k t -k t ) (F.18)
Then, we use the formula of the expectation of a quadratic form, and with the notations (F.14), we get

Θ t = Tr(A t Σ xt ) + µ T xt A t µ xt + 2µ T xt b t + β t (F.19)
The final formula (F.13) is obtained by injecting (F.19) into (F.16).

F.4 Recursive computation of state distributions

With Equation (F.13), we now only need the expressions of µ xt and Σ xt for all t in {0, ..., T } to be able to compute D KL (ρ(τ )|| ρ(τ )). Starting from the known moments of ρ(x 0 ), these can be computed recursively:

µ x t+1 = F xut µ xut + f t , (F.20) Σ x t+1 = F xut Σ xut (F xut ) T + Σ dyn t , (F.21)
where f t is the constant term in the dynamics regression, which in practice corresponds to the value of state of the current nominal trajectory, and where

µ xut = µ xt K t µ xt + k t , (F.22) Σ xut = Σ xt Σ xt K T t K t Σ xt K t Σ xt K T t + Σ ut . (F.23)
Proof of Formulas (F.20) and (F.21). Let's assume that for time step t, µ xt and Σ xt , the mean vector and covariance matrix of the state distribution, are known. Then, we have

E ρ(xt,ut) [u t ] = E ρ(xt,ut) [K t x t + k t + ω t ] = K t µ xt + k t . (F.24)
Where ω t = N (0, Σ ut ) is the noise of the stochastic controller Π. Likewise, because

x t and ω t are independent, we also have

Var ρ(xt,ut) [u t ] = Var ρ(xt,ut) [K t x t + k t ] + Var ρ(xt,ut) [ω t ] = K t Σ xt K T t + Σ ut , (F.25) and Cov ρ(xt,ut) [x t , u t ] = E ρ(xt,ut) [(x t -E[x t ])(u t -E[u t ])] = E [(x t -E[x t ])(K t (x t -E[x t ]) + ω t -E[ω t ])] = E (x t -E[x t ])(x t -E[x t ]) T K T t + E (x t -E[x t ])(ω t -E[ω t ]) T = Σ xt K T t . (F.26)
Where the last line comes from the independence between x t and ω t (Cov[x t , ω t ] = 0). Finally, from (F.24), (F.25) and (F.26), we can write µ xut and Σ xut in the form of (F.22) and (F.23). Then, if Ω t denotes a Gaussian noise drawn from N (0, Σ dyn t ) we can compute

µ x t+1 = E ρ(xt,ut) [x t+1 ] = E ρ(xt,ut) [F xut xu t + f t + Ω t ] = F xut µ xut + f t , (F.27)
and, by independence of xu t and Ω t ,

Σ x t+1 = Var[F xut xu t + f t + Ω t ] = F xut Σ xut (F xut ) T + Σ dyn t .
(F.28)

G.1.2 Tri robotique non supervisé

Malgré que le tri automatisé soit parmi les plus anciennes applications de robotique industrielle, il reste une tâche complexe. En effet, le schéma complet d'une application de tri est composé de nombreuses sous-tâches :

• segmentation de scène,

• identification et localisation d'objets,

• collecte de données sur les différents objets,

• préhension,

• prise de décision sur la façon de trier les objets. • 2 algorithmes de partitionnement,

• 2 métriques de validation externes.

Le descriptif exhaustif de tous les éléments de l'étude peut être trouvé dans le corps du texte. Les résultats obtenus par cette approche, présentés sur la

G.3.3 Conclusions

Figure 1 . 1 :

 11 Figure 1.1: Robotic assembly cell in a Volkswagen plant in Slovakia.

  Figure 1.2: Two different instances of the unsupervised robotic sorting problem.

Figure 2 . 2 :

 22 Figure 2.2: Comparison of K-means with and without normalization on a two dimensional made up dataset with dimensions of a different order of magnitude.

Figure 2 . 3 :

 23 Figure 2.3: Comparison of normalization + K-means against CV K-means on a two dimensional made up dataset where several separations occur along the same axis.

  Noisy data on a dimension that is not important. Interval type data with low mean.

Figure 2 . 4 :

 24 Figure 2.4: Made up two dimensional toy examples demonstrating the intuition behind GR K-means.

Figure 2 . 5 :

 25 Figure 2.5: Example real-world URS dataset under different light conditions.

Figure 2 . 7 :

 27 Figure 2.7: Exponent influence for Lego bricks clustering with exponentiated weighted K-means algorithms.

For

  this reason, in the next chapters, the URS decision making problem is viewed as an image clustering problem, where each object is represented by a raw picture of the object. The recent successes in transferring knowledge from the ImageNet dataset to other vision tasks motivates the use of pretrained deep Convolutional Neural Networks (CNN) to extract features for the targets image clustering tasks at hand. In the next chapter, we propose to study how to optimize knowledge transfer from ImageNet to a new target unsupervised image classification task. Unsupervised Robotic Sorting (URS) application, good image clustering algorithms are necessary. Therefore, this part of the manuscript focuses on image clustering. Recently, a common starting point for solving complex image clustering problems is to use generic features, extracted with deep Convolutional Neural Networks (CNN) pretrained on a large and versatile dataset (ImageNet). However, in most research, the CNN architecture for feature extraction is often arbitrarily chosen without justification. This chapter aims at providing insight on the use of pretrained deep CNN features for unsupervised problems. Different layers from various CNN architectures, combined with standard clustering algorithms, are evaluated on 8 datasets from 4 different subtasks of image clustering: Natural object recognition, Scene recognition, Face Recognition and Fine-grained recognition. Our key findings are that: 1. for all architectures and tasks, the last layer before softmax tends outperform earlier layers; 2. the choice of the CNN architecture can have a huge impact on the clustering results; 3. knowing which architecture to use is a difficult task.

  Figure 3.1: Definition of the image clustering problem. Examples of inputs and expected outputs on three natural images datasets.

Figure 3 . 2 :

 32 Figure 3.2: General form of the proposed Image Clustering pipeline.

Figure

  Layer-task interaction (mean and std across architectures, datasets and clustering algorithms).

Figure 3 . 3 :

 33 Figure 3.3: Influence of the layers on the clustering results (NMI).

Figure 3 . 4 :Figure 3 . 5 :Figure 3 . 6 :

 343536 Figure 3.4: Influence of the CNN architectures (L3) on the clustering results (NMI).

  Figure 4.1: Proposed multi-view generation + deep multi-view clustering (DMVC) approach to solve the Image Clustering problem.

4. 2

 2 From Image Clustering to Multi-View Clustering4.2.1 Related workEnsemble clustering (EC) consists in combining different clustering results in order to obtain a unified, final partition of the original data with improved clustering accuracy[START_REF] Vega-Pons | A survey of clustering ensemble algorithms[END_REF]). It is composed of two steps: generation, which deals with the creation of a set of partitions, and consensus, where all the partitions are integrated into a better set of clusters. In contrast to EC, Multi-View Clustering (MVC), is concerned with finding a unified partition from multi-view data[START_REF] Chao | A survey on multi-view clustering[END_REF]), which can be obtained by various sensors or represented with different descriptors. Recently, MVC has received a lot of attention: In ([Kumar

Figure 4

 4 Figure 4.3: 2d t-SNE visualization of features extracted by three pretrained CNNs for the UMist dataset. These features form different complementary views of the data.

  Figure 4.5, for all m ∈ {1, ..., 10} we report the mean and standard deviation across the C 10 m NMI scores. To generate Figure 4.5, for each dataset, we need to solve 10 m=1

Figure 4 . 5 :

 45 Figure 4.5: Evolution of the NMI score and total time (in sec) for different numbers of pretrained CNN feature extractors.

Figure 4 . 6 :

 46 Figure 4.6: Proposed deep multi-view clustering approach to solve multi-view clustering.

Figure 4 . 7 :

 47 Figure 4.7: Multi-view clustering results MIX 0.5 score values for the different MVC methods and datasets.

  Densenet169 features(b) Densenet169 + JULE (c) Concat (d) MVnet fix (e) MVnet

Figure 4

 4 Figure 4.9: 2d t-SNE visualization of the features extracted from the UMist dataset at different stages of the DMVC framework.

Figure 5 . 1 :

 51 Figure 5.1: Schematic representation of a real world classification system.

Figure 5

 5 Figure 5.2: Schematic representation of a decision making module of a real world Unsupervised Robotic Sorting application.

  Vision module description.

Figure 5 . 3 :

 53 Figure 5.3: Unsupervised Robotic Sorting with fixed camera poses.

  1 and sample images, illustrating the different objects, poses and BLC, can be seen on Figure 5.4a. The dataset, together with its description, can be downloaded at: https://github.com/ jorisguerin/toolClustering_dataset. For further evaluation of the robustness to lighting conditions, we also modify computationally the brightness of the pictures under conditions 2 only. This allows to isolate the influence of brightness in the clustering results. Example images with the applied brightness filters can be visualized on Figure 5.4b. Different poses, backgrounds and lighting conditions.(b) Artificial brightness modifications on BLC 2.

Figure 5 . 4 :

 54 Figure 5.4: Example images used to evaluate robustness to unmastered disturbances.

Figure 5 . 5 :

 55 Figure 5.5: Proposed MVEC approach to use multiple views of each object.

  Figure 6.1: Illustration of the Semantic View Selection Problem. The angular parameterization is defined in Section 6.3.

  2, we first propose a generic conceptual formulation of this problem, which we call the Semantic View Selection (SVS) problem. We then relax the problem by reducing it to the optimization of a clustering-based objective. To solve this problem, we introduce a new image dataset containing 144 objects, from 29 categories, under different poses and observed under various views. Both the data collection process and the dataset content are described in Section 6.3. A first approach using the clustering SVS problem formulation on the new dataset is then detailed in Section 6.4. It consists in training a multi-input deep convolutional neural network to map a top view and

Figure 6 . 2 :

 62 Figure 6.2: Example of a scene containing two objects under different viewpoints.

  .5) which is the average score under metric m of all possible clustering problems containing I. S a,m (v ζω (p c )) is high if the image generated with camera c in the pose p c , observing object ω in scene ζ ω , is good for clustering ω with algorithm a and low if not, where good means having a high score under m.

Figure 6 . 3 :

 63 Figure 6.3: Definition of the parameters used to sample camera poses (R, θ and ϕ).

Figure 6 . 4 :

 64 Figure 6.4: Best viewed in color. Subset of views for one instance of the sun glasses class in a particular scene. The two images with highest (resp. lowest) FM individual indexes are framed in green (resp. red). See Sections 6.4 and 6.5 for more details.

  To build the training set for the SVS problem, we start by generating a large set of N cp random clustering problems CP M C = {cp i , i ∈ N cp }. Then, for each (cp, I) pair, we define the cp is the number of elements in CP M C containing I. Likewise, for a given I, we can define a global semantic view score Ŝ(I) by replacing F M I I cp by F M I cp in (6.10).

  .5, where BN denotes a batch normalization layer, Drop(x) a dropout layer with x% drops and FC(k) a fully connected layer with k neurons. ReLu and Sigmoid are standard activation layers.

Figure 6 . 5 :

 65 Figure 6.5: Proposed SV-net architecture. Inputs are in blue and outputs in red. The Conv Block is the convolutional part of VGG19 (until "block4 pool" layer).

( a )

 a Figure 6.6: Examples of views predicted by SV-net.

Figure 7 . 1 :

 71 Figure 7.1: Block diagram to define a trajectory and summarize the notations.

Figure 7 . 2 :

 72 Figure 7.2: Different ways to compute the second order Taylor expansion of a Cartesian distance cost function from angular state and control vectors.

Figure 7 .

 7 3 represents the initial and final trajectories for the simulated positioning tasks. On this figure, the end-effector is represented by the little black connector at the extremity of the robots and the target by the center of the red floating sphere.The distance (cost function) is extracted directly from the software. In real physical applications, one can imagine it to be provided by any distance measurement sensor (e.g. laser tracker).

  Figure 7.3: Trajectory learned in simulation for positioning task for different industrial robots (KUKA LBR iiwa / ABB IRB 140 / UR 10 / F&P P-ARM, from left to right).

Figure 7 .

 7 Figure 7.3b represents final trajectories found for the positioning task. The algorithm stopped running when a precision of 0.1 mm was reached. As it can be seen, iLQG with no model of the robot and Cartesian cost function succeeds to reach its objective point with different robots. Indeed, they have different lengths, different numbers of links, but the algorithm can still converge towards a solution without being restricted on the cost function definition.

Figure 7 .

 7 Figure7.4 shows the influence of the number of samples on the rate of convergence on one specific robot (KUKA LBR iiwa). For a sample count above 40, the solution does not evolve anymore, which means that 40 samples is enough to characterize with high precision the local cost function.On the other hand, Figure7.4 also shows that if the number of samples gathered

Figure 7 . 4 :

 74 Figure 7.4: Distance to target point with respect to the number of iLQG passes. Different curves represent different number of samples generated for learning the cost and dynamics.

Figure 7 . 5 :

 75 Figure 7.5: Comparisons of two methods: Quadratic regression (plain blue lines) and modified state (dotted orange lines). Right-upper boxes on each figures are zoom on the final stages of iLQG.

Figure 7 . 6 :

 76 Figure 7.6: Trajectory learnt on V-REP software with a KUKA LBR iiwa.

Figure 7 . 7 :

 77 Figure 7.7: Influence of the initial covariance on the learning speed with other parameters set to v = 1, α = 10 -5 , ini = 1000

Figure 7 . 8 :

 78 Figure 7.8: Influence of other parameters on learning speed with σ ini = 1. When they do not vary, other parameters take the following values: v = 1, α = 10 -5 , ini = 1000

  (a) Initial configuration. (b) Final pose learned.

Figure 7 . 9 :

 79 Figure 7.9: Initial configuration and final pose learned for the Cartesian positioning task with the KUKA LBR iiwa.

  Figure 7.10: Learning curve for iLQG on the robot.

  Figure 7.11: Qualitative validation: Laser pointer target reaching task.

Figure 8 . 1 :

 81 Figure 8.1: Typical pipeline for stereo vision object localization.

Figure 8 . 2 :

 82 Figure 8.2: Representative subset of the images from the screw driver localization dataset.

  Figure 8.3: Computer vision pipeline using four images to generate an image containing only the studied objects.

  methodology is applied to build a dataset for screw driver localization. This dataset can then be used for training a regression model to learn 3D localization from pairs of images coming from our stereo system. Perspectives and future work regarding 3D stereo localization are discussed in Chapter 9. proposes a summary of the contributions developed throughout this manuscript and introduces various research perspectives, which arise naturally from the findings of this thesis.

Figure C. 4 :

 4 Figure C.4: NMI scores on face recognition datasets.

  Figure D.1: Made-up 2d example to illustrate the JULE methodology. N c stands for the number of clusters at a given stage of the JULE pipeline.

  Figure G.1: Deux exemples de problèmes de Tri Robotique Non-Supervisé (URS)

Figure G. 2 :

 2 Figure G.2: Robot utilisant notre algorithme des K-moyennes pondéré par rapport des écarts pour trier des objets à partir de caractéristiques de couleur et de forme.Une vidéo de démonstration peut être vue à l'url suivant : https://www.youtube.com/watch?v=korkcYs1EHM

Figure G. 3 :

 3 Figure G.3: Exemples de jeux de données pour différentes conditions de lumière.

Figure G. 4 :

 4 Figure G.4: Pourcentage d'expériences avec aucune classification erronée. L'expérience a été reproduite 98 fois sous différentes conditions de lumière et avec différents arrangements des blocs Lego. Le taux d'erreur présenté est moyenné sur ces 98 tentatives.

  Figure G.5: Comparaison de la stabilité des poids GR et CV. Étude de l'influence des exposants pour le partitionnement de briques Lego par K-means pondéré exponentiel.

G. 3 Figure G. 7 :

 37 Figure G.6: Définition du problème de partitionnement d'images. Exemples d'entrées et des sorties attendues associées pour trois jeux d'images naturelles.

Figure G. 8 :Figure G. 9 :

 89 Figure G.8: Influence de la couche d'extraction sur le partitionnement (NMI).

G. 4 . 2 Figure

 42 Figure G.12, est mise en oeuvre pour différents jeux de données. Pour chaque jeu de données, nous faisons varier le nombre d'architectures utilisées pour l'extraction et essayons toutes les permutations parmi 10 architectures étudiées. Les résultats de ces expériences sont présentés sur la Figure G.13, où il apparaît que pour tous les jeux

Figure

  Figure G.16: Visualisation t-SNE 2d des caractéristiques extraites du jeu de données UMist à différentes étapes de la structure DMVC.

Figure

  Figure G.20.

  Dans ce chapitre, une première mise en oeuvre d'URS est proposée, dans laquelle des images sont collectées avec une caméra placée verticalement au-dessus des différents objets à trier. Cette approche fonctionne plutôt bien pour le cas pratique industriel testé. Pour tester de manière plus complète la robustesse de cette approche aux changements d'arrière-plan et de luminosité, un jeu d'images représentant différents objets dans différentes conditions a été créé. Ces données sont difficiles pour l'IC.Les résultats obtenus sur ces données montrent que la robustesse de l'approche diminue quand la luminosité change trop et quand l'arrière-plan comporte des motifs. Des expériences multi-vues sont également menées avec différentes images de chaque objet et montrent que l'angle de vue sous lequel sont observés les objets influe sur la robustesse du système global. Ainsi, comme il semble impossible de définir une méthodologie robuste de prise de données par caméra fixe, nous proposons de développer un schéma de pose caméra adaptative et autonome dans le prochain cha-de vues sémantiquement riches Comme démontré dans le chapitre précédent, une mise en oeuvre robuste d'URS requiert non seulement un bon algorithme d'IC mais également une bonne stratégie d'acquisition d'images. Dans ce chapitre, afin de pouvoir choisir des images adaptées pour le partitionnement, une méthode de sélection de vues à fort contenu sémantique est proposée. La sélection de vues sémantique est une direction de recherche peu explorée jusqu'à maintenant, mais qui peut avoir de nombreux impacts positifs pour la robotique. L'utilisation de la mobilité d'un robot pour améliorer la compréhension des objets qu'il manipule semble une piste d'étude importante pour de nombreuses applications. Pour mieux comprendre le rôle de la prise de vue dans la classification d'objets, on peut observer la Figure G.22, où seule l'image du milieu permet au robot de comprendre que l'objet étudié est un peigne. Dans ce chapitre, nous proposons d'entraîner un réseau de convolution pour optimiser la pose d'un robot sériel, équipé d'une caméra en bout de bras, afin de maximiser le contenu sémantique de l'image obtenue et de comprendre la nature des objets observés. Les résultats expérimentaux obtenus démontrent que les poses retenues par le réseau surpassent les poses fixes pour la tâche d'URS.

( a )

 a Figure G.21: Illustration du problème de sélection de vue sémantique. La paramètrisation angulaire est définie plus tard.

( a )

 a Figure G.24: Exemples de vues prédites par SV-net.

Figure G. 26 :

 26 Figure G.26: Différentes options pour calculer le développement limité de second ordre de la fonction de coût cartésienne à partir de vecteurs d'état et d'un contrôle dans l'espace angulaire.

  Figure G.27: Trajectoires apprises en simulation pour une tâche de positionnement cartésien pour différents robots industriels (KUKA LBR iiwa / ABB IRB 140 / UR 10 / F&P P-ARM, de gauche à droite).

G. 8

 8 Figure G.30: Approche classique pour la localisation par vision stéréo.

Figure G. 31 :

 31 Figure G.31: Ensemble d'images représentatif du jeu de données de localisation de tournevis.

  K-means CV K-means CV p * K-means Figure 2.6: Percentage of experiments with zero misclassification. The experiment was run 98 times under different lighting conditions and with different layouts of the bricks. Error rates presented are averaged among these 98 runs.

	81%	100%	97%	100%	81%	100%
	8%				8%	17%	17%
	Original dataset	Slightly modified dataset
	K-means GR K-means GR p
	Considering the weighted K-means algorithms, CV K-means performs particularly
	well on the original dataset. However, after little modifications, it falls into very
	bad behavior. Such issue with CV K-means comes from the fact that coefficient of
	variation is not appropriate for interval scale data. In other words, CV K-means can
	succeed on the original dataset only because the bricks used do not present RGB

*

Table 2 . 1 :

 21 Statistics of the datasets used to validate GR K-means.

	Dataset	Iris	Wine
	Number of instances	150	178
	Number of attributes	4	13
	Number of classes	3	3
	Is linearly separable?	No	Yes
	Data type	Real Real and Integers
	Scale of measurement Ratio	Ratio

Table 2

 2

.2 summarizes clustering results for both datasets, using all previously described implementations of different algorithms. For each configuration, the algorithm was run 1000 times with different random initialization. The values reported in Table

2

.2 correspond to the average scores over the different runs.

Table 2

 2

		Iris	Wine
		PUR NMI PUR NMI
	K-means	0.83 0.65 0.97 0.88
	GR K-means 0.88 0.72 0.95 0.84
	GR 2 K-means 0.96 0.86 0.86 0.63
	CV K-means 0.96 0.85 0.93 0.79
	CV 2 K-means 0.96 0.87 0.87 0.67

.2: Results on other data sets. Accuracy and NMI scores averaged over 1000 runs of the algorithms from different centroid initializations. GR 2 and CV 2 denote the exponentiated versions of the algorithms (p = 2).

Table 3 . 1 :

 31 Statistics of the eight datasets used for the image clustering benchmark study.

	Dataset	Task	# images # classes Images size Balanced 1
	VOC2007 2 Natural object	2841	20	Variable	No
	COIL100 Natural object	7200	100	128 × 128	Yes
	Archi	Scene	4794	25	variable	No
	MIT	Scene	15620	67	variable	No
	Flowers	Fine-grained	400	17	variable	Yes
	Birds	Fine grained	2800	200	variable	No
	UMist	Face	564	20	220 × 220	Yes
	FEI	Face	6033	200	640 × 480	Yes
	1 A dataset is balanced if it contains a similar number of instances for each classes.
	2 We use a modified version of the VOC2007 test set. All the images presenting
	two or more labels have been removed in order to be able to evaluate clustering.

Table 3

 3

			.2: Names of the feature extraction layers studied.
			VGG16	VGG19	Inception	Xception	Resnet50
	L1	name block5 pool block5 pool shape 25,088 25,088	mixed7 221,952	add 12 102,400	activation 40 200,704
	L2	name shape	fc1 4,096	fc1 4,096	mixed10 block14 sepconv2 act activation 47 131,072 204,800 25,088
	L3	name shape	fc2 4,096	fc2 4,096	avg pool 2,048	avg pool 2,048	avg pool 2,048
	3.2.4 Clustering algorithms
	Over the last fifty years, many clustering algorithms have been developed. Different
	surveys propose different classifications of the clustering methods (([Xu and Wunsch,
	2005]), ([Berkhin, 2006])). However, a common bipartite classification of the different
	algorithms seem to emerge. The first group of algorithms are called partitioning
	methods. Data points are considered independently, as points in the feature space,
	and the clusters are created by separating the space into different areas. The other
	type of algorithms are called graph-based methods (or connectivity based methods)
	and consist in viewing the data as nodes on a graph, connected by a certain distance.

  table in Figure 4.3, InceptionResnet is performing way better than its two competitors with respect to all global metrics. However, looking at class 4, we can see that the two other networks present a significant improvement

		NMI	PUR	FM	FM C 4
	InceptionResnet 0.775 0.642 0.537 0.442
	VGG16	0.689 0.550 0.372 0.653
	Densenet121	0.684 0.553 0.384 0.700
	(a) InceptionResnet	(b) VGG16		(c) Densenet121

  only of ones, and W C k 1 ,C k 2 is the submatrix of W which points from samples in C k 1 to samples in C k 2 . This definition of cluster affinity also come from ([START_REF] Zhang | Graph degree linkage: Agglomerative clustering on a directed graph[END_REF]).

	Overview
	JULE is an iterative optimization process which leverages alternating optimization
	to obtain both good cluster assignments y[t f ] and a good new representation of the
	initial data Z θ[t f ] . Indeed, going from iteration t : (θ[t], y[t]) to iteration t + 1 consists
	in solving two subproblems:
	• Representation learning: The initial network ϕ θ[t] is trained on the dataset
	( X, y[t]) to generate a set of updated weights θ[t + 1].
	• Clusters merging: A new set of cluster assignments y[t + 1] are generated from
	similarities computed in Z θ[t+1]

  K-means. K-means is a simple clustering algorithm which performs best on representations presenting compact clusters, which are distant from each others. We choose Densenet169, which is the LNet, to represent the fixed CNN feature representation methods. Results are reported in Figure4.8 and show that for most datasets, better features are generated as we progress in the training of MVnet. For Birds, MIT and Archi, the results remain similar, which is likely to come from the fact that JULE does not perform well on these datasets.

		1
					VOC2007
					COIL100
	NMI scores	0.6 0.8			Archi MIT Flowers Birds UMist
					FEI
		LNet	LNet+JULE Concat	DMVC-fix	DMVC
	Figure 4.8: NMI scores for K-means applied to feature representations from
			different stages of the DMVC pipeline.
	4.4.4.2 Visualization
	In Figure 4.9, we also propose to visualize the evolution of the 2d t-SNE representation
	of features at different stages of the MVnet training for the UMist dataset. It also
	shows that this way of training MVnet produces representations that generate more
	compact clusters, which are distant from each others.

Table 5 . 1 :

 51 Statistics of the proposed tool clustering dataset (BLC stands for Background/Lighting Conditions).

	#Images Images size #Classes #Images per class #BLC
	560	640 x 480	7	12 to 24	5

  , the experiments are conducted on 10 clustering pipelines (5 CNN architectures and 2 clustering algorithms), and the results are used to choose the one presenting the strongest results. For a given BLC, a clustering problem is sampled by randomly taking one image (i.e. one position/orientation) for each object. The final experiments consist in sampling 1000 clustering problems for each of the BLC and cluster them with each of the pipelines. Results from these experiments are reported in both Tables 5.2 (physical BLC variation) and 5.3 (artificially modified brightness on BLC 2). For each BLC, the results reported are the means over the 1000 runs.

Table 5 .

 5 2: Clustering results for different CNN architectures and clustering algorithms on the tool clustering dataset. KMeans 0.84 0.83 0.87 0.86 0.82 0.82 0.66 0.66 0.80 0.80Table 5.3: Clustering results for different CNN architectures and clustering algorithms for different artificially modified lighting conditions on the BLC2 subset.

			BLC1	BLC2	BLC3	BLC4	BLC5
			NMI PUR NMI PUR NMI PUR NMI PUR NMI PUR
	Inception V3	Agg KMeans 0.80 0.79 0.79 0.78 0.76 0.76 0.63 0.64 0.75 0.73 0.82 0.81 0.82 0.81 0.80 0.80 0.65 0.65 0.79 0.76
	Resnet50	Agg KMeans 0.77 0.78 0.71 0.72 0.71 0.72 0.58 0.58 0.70 0.70 0.81 0.81 0.74 0.74 0.74 0.75 0.62 0.59 0.72 0.71
	VGG16	Agg KMeans 0.72 0.72 0.70 0.70 0.71 0.70 0.58 0.57 0.67 0.67 0.76 0.75 0.74 0.73 0.73 0.72 0.61 0.60 0.70 0.69
	VGG19	Agg KMeans 0.73 0.73 0.73 0.73 0.69 0.70 0.56 0.56 0.67 0.67 0.76 0.76 0.77 0.76 0.71 0.72 0.59 0.58 0.71 0.70
	Xception	Agg	0.86 0.85 0.90 0.90 0.84 0.85 0.69 0.69 0.83 0.81

Table 5 .

 5 4: Clustering results of MVEC for different BLC. For comparison, the corresponding single view results are reminded in parenthesis.

			NMI	Purity
		BLC1	0.95 (0.86) 0.96 (0.85)
		Very dark	0.91 (0.77) 0.93 (0.77)
		Dark	1.00 (0.88) 1.00 (0.89)
	BLC2	Normal	1.00 (0.90) 1.00 (0.90)
		Bright	0.96 (0.84) 0.96 (0.84)
		Very bright 0.84 (0.73) 0.86 (0.74)
		BLC3	0.95 (0.84) 0.96 (0.85)
		BLC4	0.84 (0.69) 0.82 (0.69)
		BLC5	0.95 (0.83) 0.96 (0.81)

  6.1.1 From URS to Semantic View SelectionA robust Unsupervised Robotic Sorting (URS) application requires both a good strategy for image acquisition and a good Image Clustering (IC) pipeline. In the previous chapter, a first straightforward pipeline for image acquisition is presented.It consists in gathering images of the different objects from fixed, top down camera poses. Although this approach, combined with a good IC pipeline, has demonstrated good results on a real-world industrial implementation, we have also shown that URS may benefit from an adaptive view selection scheme. In this chapter, we propose to study the problem of optimal view selection for clustering. This problem is embedded into the broader problem of optimal semantic view selection. As explained in Section 6.2, we believe that choosing views which are semantically meaningful and views which are good for clustering are two problems which are highly positively correlated.

In the meantime, viewing the problem as one of finding "semantic views" is more generic and has more potential direct and indirect applications. Moreover, formally defining the generic semantic view selection problem leaves the door open to different approaches for future research. Hence, this chapter deals with the problem of Semantically Meaningful View Selection (SVS) independently of any URS considera-

Table 6 . 1 :

 61 Statistics of the proposed multi-objects/multi-pose/multi-view image dataset.

	# Classes # Object/class (total ) # Poses/object (total ) # Images/pose (total )
	29	4-6 (144 )	3 (432 )	17-22 (9112 )

Table 6 .

 6 2: Semantic view scores validation. Comparison of clustering results among different view selectors on the training set. for each (c, m) pair, the best view selector is in bold.

			FM NMI PUR
		TOP	0.48 0.78 0.73
	XCE AGG	RAND 0.50 0.78 0.74 OPT glob 0.85 0.94 0.93
		OPT ind 0.87 0.95 0.94
		TOP	0.44 0.75 0.71
	XCE KM	RAND 0.46 0.76 0.72 OPT glob 0.81 0.93 0.91
		OPT ind 0.83 0.93 0.92
		TOP	0.39 0.72 0.67
	VGG AGG	RAND 0.38 0.72 0.66 OPT glob 0.49 0.78 0.73
		OPT ind 0.52 0.79 0.74

Table 6 .

 6 3: SV-net validation. Comparison of clustering results between different view selectors on the test set. for each (c, m) pair, the best view selector is in bold.

			FM NMI PUR
		TOP	0.44 0.51 0.70
	XCE AGG	RAND 0.48 0.56 0.74
		SV-net 0.55 0.63 0.78
		TOP	0.44 0.51 0.70
	XCE KM	RAND 0.48 0.55 0.73
		SV-net 0.55 0.62 0.78
		TOP	0.46 0.53 0.71
	VGG AGG	RAND 0.44 0.51 0.70
		SV-net 0.48 0.55 0.73

  t. δx t and δu t (respectively n and m entries) is obtained by solving a linear regression with input

	data in the form:
	[1, δx 1 , ..., δx n	, δx 2 1 , δx 1 δx 2 , ..., δx 2 n	, δu 1 , ..., δu m	, δu 2 1 , ..., δu 2 m	, δx 1 δu 1 , ..., δx n δu m
	n	n(n+1)	m	m(m+1)	n×m
		2		2

Table B . 1 :

 B1 Results on natural object recognition datasets.

				K-means K-means K-means K-means	L1 L2 L3 L1 L2 L3 L1 L2 L3 L1 L2 L3
	VOC2007 COIL100 Archi MIT Flowers Birds Umist FEI VGG16 Table B.2: Results on Scene recognition datasets. Table B.3: Results on fine-grained recognition datasets. Table B.4: Results on face recognition datasets. 0.516 0.578 0.673 VGG16 0.425 0.420 0.430 VGG16 0.588 0.657 0.688 VGG16 0.575 0.669 0.669 VGG16 0.525 0.608 0.651 VGG16 0.420 0.431 0.414 VGG16 0.616 0.661 0.644 VGG16 0.699 0.717 0.689 VGG16 0.895 0.943 0.945 VGG16 0.383 0.481 0.515 VGG16 0.375 0.529 0.557 VGG16 0.875 0.899 0.887 0.928 0.940 0.956 0.410 0.471 0.492 0.420 0.542 0.557 0.909 0.900 0.897 VGG16 VGG16 VGG16 VGG16	KM Agg KM Agg KM Agg KM Agg KM Agg KM Agg KM Agg KM Agg	nmi pur time nmi pur time nmi pur time nmi pur time nmi pur time nmi pur time nmi pur time nmi pur time nmi pur time nmi pur time nmi pur time nmi pur time nmi pur time nmi pur time nmi pur time nmi pur time VGG19 VGG16 L1 0.516 0.656 82.5 0.525 0.549 96.8 0.895 0.821 457 0.928 0.863 476 L2 0.578 0.557 18.3 0.608 0.615 16 0.943 0.895 72 0.940 0.875 71 L3 0.673 0.636 16.4 0.651 0.653 16.1 0.945 0.900 77 0.956 0.919 70 VGG16 L1 0.425 0.451 218 0.420 0.429 210 0.383 0.400 1,826 0.410 0.349 2,259 L2 0.420 0.398 35 0.431 0.413 31.2 0.481 0.357 285 0.471 0.360 328 L3 0.430 0.371 39 0.414 0.382 31.4 0.515 0.406 283 0.492 0.398 334 VGG16 L1 0.588 0.609 36.5 0.616 0.623 17.1 0.375 0.160 2,309 0.420 0.174 1,288 L2 0.657 0.652 5.5 0.661 0.685 2.5 0.529 0.301 399 0.542 0.325 190 L3 0.688 0.680 5.7 0.644 0.651 2.7 0.557 0.314 371 0.557 0.332 189 VGG16 L1 0.575 0.494 11.9 0.699 0.600 3.2 0.875 0.793 304 0.909 0.858 72 L2 0.669 0.543 1.7 0.717 0.560 0.5 0.899 0.834 52 0.900 0.835 11 L3 0.669 0.529 1.5 0.689 0.532 1.5 0.887 0.804 45 0.897 0.825 11 0.541 0.625 0.661 VGG19 0.408 0.415 0.416 VGG19 0.567 0.652 0.652 VGG19 0.623 0.719 0.700 VGG19 0.537 0.618 0.650 VGG19 0.431 0.426 0.398 VGG19 0.614 0.676 0.614 VGG19 0.717 0.766 0.740 VGG19 0.889 0.937 0.939 VGG19 0.388 0.484 0.510 VGG19 0.390 0.534 0.562 VGG19 0.888 0.905 0.908 0.922 0.949 0.948 0.411 0.480 0.491 0.428 0.544 0.557 0.923 0.918 0.915 VGG19 VGG19 VGG19 VGG19	VGG19 L2 0.625 0.593 17.6 0.618 0.642 16.1 0.937 0.881 67 0.949 0.892 71 VGG19 L2 0.415 0.392 37 0.426 0.400 31.4 0.484 0.380 288 0.480 0.420 331 VGG19 L2 0.652 0.699 5.5 0.676 0.717 2.5 0.534 0.303 377 0.544 0.315 189 VGG19 L2 0.719 0.597 1.5 0.766 0.671 0.4 0.905 0.837 47 0.918 0.862 15 Inception L3 0.661 0.628 18.5 0.650 0.631 16.8 0.939 0.883 62 0.948 0.894 70 Inception L1 0.216 0.364 825 0.298 0.382 856 0.799 0.642 4,340 0.846 0.719 4,455 L2 0.423 0.447 565 0.542 0.592 531 0.913 0.828 2,852 0.939 0.873 2,517 L3 0.408 0.359 34.1 0.398 0.362 31.2 0.510 0.389 289 0.491 0.400 331 Inception L1 0.184 0.207 1,747 0.178 0.250 1,909 0.209 0.492 170.4 0.378 0.291 36,824 L2 0.412 0.388 1,142 0.401 0.394 1,119 0.495 0.419 10,005 0.498 0.399 12,243 L3 0.652 0.696 5.3 0.646 0.688 2.5 0.562 0.331 383 0.557 0.333 187 Inception L1 0.241 0.367 364 0.329 0.385 155 0.143 0.144 145.7 0.306 0.078 17,966 L2 0.560 0.586 219 0.604 0.630 90.2 0.464 0.192 12,483 0.484 0.212 6,904 L3 0.700 0.577 1.5 0.740 0.635 0.4 0.908 0.848 45 0.915 0.857 13 Inception L1 0.588 0.499 104 0.641 0.530 28.1 0.881 0.788 3,366 0.920 0.868 663 L2 0.661 0.532 60.9 0.694 0.565 16.5 0.923 0.879 1,843 0.939 0.904 383 0.216 0.423 0.603 Agglomerative L1 0.541 0.605 83 0.537 0.533 98 0.889 0.791 440 0.922 0.845 476 L3 0.603 0.561 8.6 0.692 0.681 8.6 0.928 0.868 37 0.953 0.904 35 L1 0.416 0.447 219 0.431 0.449 211 0.388 0.389 1,749 0.411 0.339 2,235 L3 0.420 0.363 16.1 0.421 0.395 15.5 0.570 0.446 138 0.561 0.466 164 L1 0.567 0.710 35 0.614 0.660 17.1 0.390 0.170 2,369 0.428 0.182 1,282 L3 0.650 0.645 2.6 0.677 0.700 1.1 0.558 0.326 164 0.569 0.342 95 L1 0.623 0.490 12.7 0.717 0.624 3.1 0.888 0.821 311 0.923 0.878 93 L3 0.698 0.563 0.8 0.760 0.647 0.2 0.921 0.854 21 0.941 0.899 5.3 Inception 0.184 0.412 0.420 Agglomerative Inception 0.241 0.560 0.650 Agglomerative Inception 0.588 0.661 0.698 Agglomerative Inception 0.298 0.542 0.692 Inception 0.178 0.401 0.421 Inception 0.329 0.604 0.677 Inception 0.641 0.694 0.760 (a) VOC2007 Inception 0.799 0.913 0.928 K-means Agglomerative (a) Archi Inception 0.209 0.495 0.570 K-means Agglomerative (a) Flowers Inception 0.143 0.464 0.558 K-means Agglomerative (a) Umist Inception 0.881 0.923 0.921 K-means Agglomerative 0.846 0.939 0.953 0.378 0.498 0.561 0.306 0.464 0.569 0.920 0.939 0.941 Inception Inception Inception Inception (b) COIL100 (b) MIT (b) Birds (b) FEI	Xception L2 0.538 0.509 901 0.625 0.616 822 0.920 0.842 3,754 0.942 0.878 3,909 Xception L2 0.435 0.381 1,636 0.435 0.424 1,756 0.500 0.572 174.8 0.548 0.455 29,548 Xception L2 0.633 0.643 298 0.661 0.690 140 0.481 0.307 189.8 0.548 0.291 11,159 Xception L2 0.688 0.588 89.9 0.745 0.635 25.5 0.930 0.893 2,712 0.934 0.896 599 Xception L1 0.291 0.358 356 0.410 0.510 403 0.832 0.692 1,965 0.875 0.763 1,960 L3 0.687 0.630 8.7 0.719 0.709 8.1 0.938 0.881 32 0.955 0.911 35 L1 0.279 0.493 758 0.315 0.367 783 0.850 0.718 4,475 0.888 0.782 3,876 L1 0.165 0.246 595 0.174 0.243 859 0.430 0.390 7,307 0.457 0.378 9,177 L3 0.442 0.384 15.5 0.433 0.401 15.6 0.597 0.471 121 0.587 0.488 164 L1 0.248 0.282 1,827 0.270 0.285 1,674 0.193 0.598 165.7 0.393 0.321 23949 L1 0.425 0.501 139 0.495 0.599 69.7 0.341 0.106 8,919 0.366 0.122 5,194 L3 0.674 0.681 2.9 0.670 0.704 1.2 0.633 0.428 159 0.644 0.452 94 L1 0.543 0.611 298 0.589 0.626 137 0.146 0.413 202.3 0.331 0.102 10,994 L1 0.589 0.471 47.2 0.642 0.511 12.9 0.789 0.630 1,393 0.831 0.690 296 L3 0.683 0.593 0.8 0.731 0.607 0.2 0.913 0.852 20 0.928 0.884 5.2 L1 0.533 0.443 87.9 0.597 0.459 24.9 0.843 0.718 3,137 0.892 0.816 579 0.291 0.538 0.687 Xception 0.165 0.435 0.442 Xception 0.425 0.633 0.674 Xception 0.589 0.688 0.683 Xception 0.410 0.625 0.719 Xception 0.174 0.435 0.433 Xception 0.495 0.661 0.670 Xception 0.642 0.745 0.731 Xception 0.832 0.920 0.938 Xception 0.430 0.500 0.597 Xception 0.341 0.481 0.633 Xception 0.789 0.930 0.913 0.875 0.942 0.955 0.457 0.548 0.587 0.366 0.548 0.644 0.934 0.928 0.831 Xception Xception Xception Xception	Resnet50 L2 0.413 0.548 99 0.494 0.491 97 0.884 0.771 489 0.927 0.846 474 Resnet50 L2 0.377 0.368 238 0.389 0.402 210 0.367 0.365 1,684 0.395 0.315 2,252 Resnet50 L2 0.510 0.713 37.6 0.607 0.679 17.2 0.375 0.142 2,630 0.421 0.157 1,273 L2 0.555 0.463 11.3 0.623 0.510 3.1 0.903 0.840 324 0.931 0.897 72 Resnet50	L3 0.680 0.641 8.4 0.656 0.666 8 0.957 0.926 35 0.967 0.940 35 L3 0.455 0.410 17 0.447 0.414 22 0.539 0.426 133 0.529 0.443 164 L3 0.684 0.668 2.7 0.708 0.721 2.3 0.521 0.290 197 0.529 0.312 94 0.919 0.859 5.3 Resnet50 BoF 0.083 0.325 0.5 0.072 0.342 0.4 ------BoF 0.100 0.555 0.9 0.102 0.681 1.1 ------BoF 0.177 0.419 0.331 0.179 0.388 0.103 ---------0.279 0.413 0.680 Resnet50 0.248 0.377 0.455 Resnet50 0.543 0.510 0.684 Resnet50 0.533 0.555 0.658 25 ---Resnet50 0.315 0.494 0.656 Resnet50 0.270 0.389 0.447 Resnet50 0.589 0.607 0.708 Resnet50 0.597 0.623 0.717 L3 0.658 0.532 770 0.717 0.579 0.2 0.910 0.841 Resnet50 0.850 0.884 0.957 Resnet50 0.193 0.367 0.539 Resnet50 0.146 0.375 0.521 Resnet50 0.843 0.903 0.910 0.542 0.459 0.306 0.638 0.522 0.03 0.888 0.927 0.967 0.393 0.395 0.529 0.331 0.421 0.529 0.892 0.931 0.919 BoF Resnet50 Resnet50 Resnet50 Resnet50
	Figure C.1: NMI scores on natural object recognition datasets. Figure C.2: NMI scores on scene recognition datasets. Figure C.3: NMI scores on natural fine-grained datasets.

  Un robot industriel est un mécanisme motorisé complexe, conçu de manière à pouvoir être programmé pour différentes tâches. Cette flexibilité mécanique lui permet d'exécuter une large gamme de mouvements, ce qui a permis un essor important de la robotique dans les usines au cours des dernières décennies. Pendant de nombreuses années, la recherche autour de la programmation robotique s'est principalement concentrée sur l'étude de la cinématique et du contrôle de ces machines complexes. Cependant, cette vision du robot comme une simple machine de précision, dénuée de capacité de perception et d'adaptation, a limité leur utilisation à des environnements cloisonnés et à des tâches très répétitives, nécessitant de gros volumes de production pour être rentable.

	G.1 Introduction : vers des robots plus polyvalents
	G.1.1 L'apprentissage automatique au service de l'adaptabi-
	lité des robots
	Dans cette thèse, nous proposons de mener une réflexion autour de la généralisation
	des applications robotiques à des plages de fonctionnement plus larges. De part ses
	nombreux récents succès en terme de modélisation de systèmes complexes, l'appren-
	tissage automatique (ML) est un cadre intéressant pour mener cette étude. En effet,
	le ML permet de définir des tâches de manière plus systémique et a démontré des
	résultats moins dépendants d'une tâche donnée. Une introduction à l'apprentissage
	automatique est proposée en Annexe A. Le but de cette thèse est d'introduire de
	nouvelles méthodes de ML, permettant d'élargir la gamme de fonctionnement et le
	domaine d'applicabilité de diverses applications robotiques, et ainsi de mieux exploi-
	ter la richesse dans la conception de ces machines. Les améliorations introduites dans
	ce manuscrit sont présentées principalement dans le contexte du tri robotisé, introduit
	ci-après.

Ces dernières années ont vu émerger un nouveau contexte industriel, appelé industrie 4.0, qui propose une vision plus souple des unités de production, nécessitant des robots plus polyvalents, capables de collaborer avec d'autres machines ainsi que des humains. Avec l'apparition de ce nouveau besoin, les systèmes robotisés modernes ont été équipés de nombreux capteurs afin de leur donner des capacités de perception et ainsi leur permettre de résoudre une plus grande diversité de tâches.

  Dans cette thèse, nous nous intéressons principalement au module de prise de décision et proposons une approche plus générique que ce qui est fait classiquement. En définissant la tâche de tri de manière non-supervisée, l'application est alors programmée pour fonctionner avec tous types d'objets au lieu d'être spécialisée pour certaines catégories.

  problèmes de partitionnement. Cette approche est ensuite testée sur l'application robotique de tri de blocs Lego. L'étude est menée sur un cas pratique comportant trois catégories, présentées en Figure G.3. Cette expérience de tri est répétée 98 fois avec différentes conditions de lumière et différentes dispositions des objets. Les résultats obtenus sont comparés avec les méthodes de KM classique et de CVKM. Ils sont présentés en Figure G.4, où l'on peut également voir une étude de robustesse quand une des composantes de couleur se rapproche de zéro (histogramme de droite).

	81%	100%	97%	100%	81%	100%
	8%				8%	17%	17%
	Données d'origine	Données légèrement modifiées
	K-means GR K-means GR p
	chaque paire de points consécutifs pour chaque dimension. Le rapport de l'écart
	maximum sur l'écart moyen est ensuite calculé pour chaque dimension puis utilisé
	pour créer des poids qui sont réinjectés dans les données normalisées afin de trouver
	les dimensions les plus importantes. L'utilisation des écarts est motivée par l'intui-
	tion qu'un grand écart pour une dimension peut signifier que les valeurs viennent
	de deux distributions différentes et donc que les points appartiennent à deux classes

G.2.2 Approche proposée et résultats

L'algorithme GRKM proposé dans ce chapitre se base sur le calcul des écarts entre différentes. L'avantage d'utiliser des données relatives est qu'elles peuvent fonctionner pour différents types d'échelles. A travers de nombreux exemples simulés, qui sont présentés dans le corps du texte, la pertinence de cette approche est démontrée pour différents * K-means CV K-means CV p * K-means

  Les résultats complets obtenus sont présentés en annexe B. Ces résultats sont synthétisés sur les Figures G.8, G.10 et G.9, où les moyennes et écarts types pour Interaction couche/architecture (moyenne et écart type sur les différentes tâches et algorithmes). Interaction couche/tâche (moyenne et écart type sur les architectures, DS et algorithmes).

	différents couples de paramètres sont représentés.
	0.8
	0.6
	0.4
	VGG16	VGG19	Inception	Xception	Resnet50
	0.2 0.4 0.6 0.8 1	(a) Natural object	Scene L1 L2 L3 Fine-grained	Face
			(b)

  Le premier résultat intéressant de cette étude est que la dernière couche avant le softmax semble produire les caractéristiques les plus intéressantes pour le partitionnement. Les expériences ont également démontré que le choix de l'architecture d'extraction de caractéristiques est crucial pour le tri non-supervisé. Cependant, les résultats obtenus ne permettent pas de conclure quant au choix d'une telle architecture. Pour pallier à ce problème, une nouvelle méthode ensembliste est proposée dans le chapitre suivant. G.4 Améliorer le partitionnement d'images en utilisant plusieurs CNN pré-entraînés. 'utilisation de réseaux profonds pré-entraînés pour l'extraction de caractéristiques permet d'améliorer grandement les résultats d'IC. Cependant, les expériences menées dans le chapitre précédent ont démontré que le choix d'une bonne architecture pour un problème donné est particulièrement complexe. Une méthode pour s'affranchir de ce choix est introduite dans ce chapitre. En suivant l'intuition que différents réseaux préentraînés peuvent contenir des informations complémentaires, nous proposons d'utiliser plusieurs CNNs simultanément pour générer plusieurs représentations des données initiales. Cette approche revient à transformer le problème initial en un problème de partitionnement multi-vues (MVC). Le problème de MVC ainsi généré est résolu par la méthode de partitionnement profond JULE, que nous avons modifiée pour accepter des données multi-vues. Les résultats obtenus par la méthodologie proposée sont les meilleurs résultats connus dans la littérature pour plusieurs jeux de données. Cette approche présente également l'avantage de générer une représentation unifiée de basse dimension du jeu de données de départ.

	G.4.1 Introduction

L'utilisation d'un CNN pré-entraîné pour extraire des caractéristiques est maintenant devenue une pratique courante pour résoudre des problèmes d'IC. Cependant, le choix de l'architecture et de la couche d'extraction est souvent arbitraire. Dans ce chapitre, nous avons mené des expériences sur huit jeux de données classiques de vision provenant de quatre sous-tâches de classification. L

  Tableau G.2: Résultats de partitionnement pour différentes architectures de CNN et différents algorithmes de partitionnement sur l'ensemble d'images BLC2 avec différentes modifications artificielles de la luminosité. PUR NMI PUR NMI PUR NMI PUR NMI PUR Inception V3 Agg 0.74 0.72 0.81 0.79 0.82 0.81 0.80 0.80 0.71 0.70 KMeans 0.70 0.70 0.77 0.75 0.79 0.78 0.77 0.77 0.66 0.67 Resnet50 Agg 0.67 0.67 0.73 0.73 0.74 0.74 0.69 0.68 0.61 0.61 KMeans 0.65 0.66 0.70 0.71 0.71 0.72 0.66 0.66 0.58 0.59 VGG16 Agg 0.66 0.66 0.73 0.72 0.74 0.73 0.68 0.68 0.61 0.61 KMeans 0.62 0.63 0.69 0.69 0.70 0.70 0.65 0.66 0.57 0.58 VGG19 Agg 0.67 0.67 0.76 0.75 0.77 0.76 0.74 0.72 0.64 0.65 KMeans 0.64 0.65 0.73 0.73 0.73 0.73 0.71 0.70 0.59 0.62 Xception Agg 0.77 0.77 0.88 0.89 0.90 0.90 0.84 0.84 0.73 0.74 KMeans 0.74 0.74 0.85 0.86 0.87 0.86 0.82 0.82 0.70 0.71

	Very dark	Dark	Normal	Bright	Very bright
	NMI

  Table G.3, montrent que l'utilisation simultanée de différents points de vue des objets permet d'améliorer la robustesse aux différents éléments non maîtrisables. L'étude menée dans ce chapitre souligne donc l'importance d'étudier la méthodologie de collecte des images de 206 manière plus fine que l'approche initiale proposée. Tableau G.3: Résultats du partitionnement MVEC pour différentes BLC. Pour comparer, les résultats avec vue unique sont rappelés entre parenthèse.

			NMI	Purity
		BLC1	0.95 (0.86) 0.96 (0.85)
		Very dark	0.91 (0.77) 0.93 (0.77)
		Dark	1.00 (0.88) 1.00 (0.89)
	BLC2	Normal	1.00 (0.90) 1.00 (0.90)
		Bright	0.96 (0.84) 0.96 (0.84)
		Very bright 0.84 (0.73) 0.86 (0.74)
		BLC3	0.95 (0.84) 0.96 (0.85)
		BLC4	0.84 (0.69) 0.82 (0.69)
		BLC5	0.95 (0.83) 0.96 (0.81)

  Schéma bloc pour définir une trajectoire et résumer les notations. une approximation linéaire locale de la dynamique et obtenir de bons résultats d'optimisation de trajectoire. Cependant, pour ce qui est de la fonction de coût, la plupart des articles propose de la calculer à partir d'un modèle du robot ou encore d'ajouter des variables au vecteur d'état afin de pouvoir ensuite la calculer analytiquement. Dans ce chapitre, nous proposons d'étendre le processus d'exploration-régression à l'estimation de la fonction de coût. Cette méthode permet d'éviter de créer un estimateur quadratique calculé à partir d'une approximation linéaire. Les différentes méthodes possibles pour calculer le développement limité de deuxième ordre de la fonction de coût sont présentées sur la Figure G.26. Inclure la distance d t dans la représentation d'état. Apprendre l'approximation quadratique du coût.

	État initial x 0	État x t	Contrôleur Π t (x t )	Action u t	Dynamique F t (x t , u t )	État x t+1	État final x T
			Coût l t
	Figure G.25: x t , u t	Dynamique linéaire apprise	Modèle approximatif du robot	l t
			(a) Utiliser un modèle du robot.
					x t+1
	x t , u t	Dynamique linéaire
	d t		apprise	d t+1	Dérivation analytique	l t
		(b) x t , u t	Dynamique linéaire apprise	x t+1
				Fonction de coût quadratique apprise	l t
		(c)

https://artsetmetiers.fr/fr/techday-robotique

Remerciements

Ce projet a reçu des soutiens financiers du projet ColRobot (collaborative robotics for assembly and kitting in smart manufacturing, European Union's 2020 research and innovation program under grant agreement No.688807), de Fulbright Haut-de-France, ainsi que de la Fondation Arts et Métiers (convention No. 8130-01).

Chapter 7

Model independent trajectory optimization Abstract

The methods for object understanding proposed in the previous chapters are important skills to increase the operating range of many robotic applications. Similarly, this part of the thesis, presents contributions in two other tasks that can increase autonomy, flexibility and robustness of various industrial applications, including robotic sorting. Chapter 8 deals with 3D object localization and this chapter focuses on precise trajectory learning. Recent reinforcement learning methods have enabled to accomplish difficult high dimensional robotic tasks under unknown dynamics, using iterative Linear Quadratic Gaussian (iLQG) control theory. These algorithms are based on building a local time-varying linear model of the dynamics from data gathered through interaction with the environment. These techniques often require an accurate model of the manipulated system to converge, which can be impractical for tasks where such a model is unknown or changing frequently. This chapter proposes a model independent version of iLQG by regressing the quadratic cost function directly from the data. This way, any sensor information can be used to design the cost function, thus making the trajectory learning easier to define and reprogram.

The proposed approach is validated against another model independent method for a Cartesian positioning task, with various industrial robot models, in a simulated environment. Simulation is also leveraged to tune the hyperparameters of the method. These parameters are then transferred to a real industrial robot for both standard Cartesian positioning and a target reaching task with a laser pointer, for which a model cannot be computed in the general case.

Appendix A A beginner's definition of Machine Learning

Every morning, a fruit retailing company receives apples and oranges mixed in the same containers. Then, fruits go on a conveyor one by one and need to be placed in the crates corresponding to their type. For such task, we can think of different levels of automation. A first, very simple solution is to have someone stand by the end of the conveyor, who decides whether a fruit is an apple or an orange. Even though such method would work perfectly, it has the drawback of requiring a worker fully dedicated to this tedious task. To automate such procedure, another possibility is to automatically measure characteristics of the different fruits, such as color, size, etc. Such characteristics are called features and are then used to determine where the fruits should go. To do so, an engineer can analyze the features, plot them, compute statistics, etc. and come up with more or less complex rules to decide whether a fruit is an apple or an orange. Such automation can save valuable time and money for the company, but is limited to apples and oranges. If the company starts receiving melons, they would need to call the engineer again, so that he can come up with new rules that take melons into account. In this simple example, using Machine Learning (ML) can be thought of as automating the work of the engineer. By automatically analyzing some labelled examples, i.e. features measurements and corresponding categories, a machine learning algorithm can come up with general rules for classifying previously unseen fruits.

The famous example of apples and oranges classification is basic, but yet is a good illustration of why ML receives so much interest. It enables to define complex programs and models for specific cases, by generalizing from examples. It is easy, even for a non-specialist, to teach a well-designed ML algorithm to carry out new tasks.

Arthur Samuel [START_REF] Mccarthy | In memoriam: Arthur samuel: Pioneer in machine learning[END_REF]), one of the pioneers of ML who

F.1 Problem statement

This appendix presents a method to compute the Kullback-Leibler divergence (D KL ) between two Gaussian trajectory distributions, ρ and ρ. The initial states of these two trajectories are drawn from the same distribution

where N (µ x 0 , Σ x 0 ) is the normal distribution of mean vector µ x 0 and covariance matrix Σ x 0 . We also assume that these two trajectory distributions share the same dynamics

where x t and u t are respectively the state and action vectors at time step t and

In the context of learnt iLQG introduced in Chapter 7, this assumption is justified the small deviation rule for policy updates. Finally, both trajectory generate actions by following different time-varying linear-Gaussian controllers

F.5 Summary algorithm

To conclude this appendix, we propose the following pseudo-code for the computation of the KL divergence between Gaussian trajectories.

Algorithm 1: Pseudo-code for computation of KL divergence between two Gaussian trajectories.

Annexe G

Résumé étendu en français

Cette annexe propose une synthèse du mémoire de thèse rédigée en langue française.

Dans un soucis de concision, seule la méthodologie et les principaux résultats sont présentés. On notera que, par souci de cohérence du manuscrit, les acronymes en anglais sont utilisés dans ce résumé. Les traductions en français sont rappelées dans le glossaire, au début de la thèse.

Les travaux présentés ont été financés en partie par le projet européen ColRobot (robotique collaborative pour l'assemblage et la préparation de kits dans le contexte de

G.4.3 Méthode complète et résultats

G.8.3 Conclusions

Dans ce chapitre, nous avons proposé une méthodologie de construction de jeux de données supervisés pour la localisation stéréo. Un jeu de données a été créé pour des tournevis et mis à la disposition de la communauté. Ces jeux de données peuvent ensuite servir à entraîner des CNNs siamois à résoudre la tâche de localisation.

G.9 Conclusions générales

Les robots industriels modernes sont des machines qui peuvent être reprogrammées pour résoudre de nombreuses tâches sans toucher à leur architecture physique. Ces caractéristiques leur permettent de résoudre une grande diversité de tâches.

METHODES D'APPRENTISSAGE AUTOMATIQUE POUR DES APPLICATIONS ROBOTIQUES DANS UN CONTEXTE INDUSTRIEL :

ETUDE DE CAS DU TRI ROBOTISÉ RESUME: L'architecture multi-axes des robots industriels permet de les programmer pour effectuer des tâches diverses. Cependant, malgré qu'ils soient équipés de nombreux capteurs -ce qui devrait leur permettre de s'adapter à des changements d'environnement -l'utilisation de robots dans l'industrie se limite souvent à des tâches très répétables et ne nécessitant que peu d'adaptabilité. Dans un contexte industriel, la programmation de robots capables de s'adapter automatiquement à diverses applications, et étant robustes sous différentes conditions de fonctionnement est une source de progrès importante. Ainsi, dans cette thèse, plusieurs contributions en apprentissage automatique sont proposées dans le but de concevoir des robots intelligents, ayant une plus grande gamme de fonctionnements. Les méthodes présentées dans ce mémoire sont centrées autour du tri autonome d'objets mais peuvent servir à implémenter de nombreuses autres applications robotiques. Afin de concevoir des applications plus polyvalentes, des solutions aux problèmes de tri d'images non supervisé, de choix de vue optimal, d'apprentissage de trajectoires et de localisation stéréoscopique ont été développées. ABSTRACT: Thanks to their flexible mechanical design, modern industrial robots can be programmed for different tasks. However, despite the fact that they are highly instrumented -which should enable them to be responsive to their environment -the use of robots in industry is still often restricted to repeatable tasks with low level of adaptability. In an industrial context, it is essential to program robots that can autonomously adapt to different applications and are robust to changes in their working conditions. Hence, in this thesis, several machine learning contributions are presented, aiming at designing smarter robotic applications, with a broader operational range. The methods developed are centered on autonomous sorting, but may be useful to address problems in many other subfields of robotics. Throughout this thesis, new approaches are proposed to address image clustering, optimal view selection, trajectory learning and stereo localization, with the objective of designing more versatile robotic applications.

Keywords