Romain Brégier

Frédéric Devernay

Laetitia Leyrit

James L Crowley

Symmetry Aware Evaluation of 3D Object Detection and Pose Estimation in Scenes of Many Parts in Bulk

While 3D object detection and pose estimation has been studied for a long time, its evaluation is not yet completely satisfactory. Indeed, existing datasets typically consist in numerous acquisitions of only a few scenes because of the tediousness of pose annotation, and existing evaluation protocols cannot handle properly objects with symmetries. This work aims at addressing those two points. We first present automatic techniques to produce fully annotated RGBD data of many object instances in arbitrary poses, with which we produce a dataset of thousands of independent scenes of bulk parts composed of both real and synthetic images. We then propose a consistent evaluation methodology suitable for any rigid object, regardless of its symmetries. We illustrate it with two reference object detection and pose estimation methods on different objects, and show that incorporating symmetry considerations into pose estimation methods themselves can lead to significant performance gains. The proposed dataset is available at

Introduction

Detecting instances of 3D rigid objects and estimating their poses given visual data is of major interest for applications such as augmented reality, scene understanding and robotics, and has been an open field of research since the early days of computer vision. Nonetheless and despite important progresses in this field, existing 3D pose estimation methods still fundamentally cannot deal with any kind of rigid object, since they rely on the assumption that the pose of a rigid object corresponds to a single 6-degrees-offreedom rigid transformation [START_REF] Johnson | Using spin images for efficient object recognition in cluttered 3D scenes[END_REF][START_REF] Mian | Threedimensional model-based object recognition and segmentation in cluttered scenes[END_REF][START_REF] Drost | Model globally, match locally: Efficient and robust 3D object recognition[END_REF][START_REF] Hinterstoisser | Model based training, detection and pose estimation of texture-less 3D objects in heavily cluttered scenes[END_REF][START_REF] Tejani | Latent-Class hough forests for 3D object detection and pose estimation[END_REF][START_REF] Hodan | On evaluation of 6D object pose estimation[END_REF]. This hypothesis leads to ambiguities when dealing with objects showing some proper symmetries -i.e. invariances under some rigid transformations -notably for evaluation purposes, as the pose of such an object can be represented by multiple rigid transformations. Symmetries are actually common among manufactured objects, and while ad hoc solutions to this issue have been proposed for performance evaluation [START_REF] Hinterstoisser | Model based training, detection and pose estimation of texture-less 3D objects in heavily cluttered scenes[END_REF][START_REF] Hodan | On evaluation of 6D object pose estimation[END_REF], they consist in relaxing the pose estimation validation criteria and are therefore not suited for applications requiring precise positioning.

On an other level, a scenario of particular practical interest for object detection and pose estimation consists in the so-called bin-picking problem, where instances of a rigid object have to be detected and localized within a bin containing many instances in bulk. Despite the active research on 3D pose estimation, the current state of the art for bin-picking is relatively unknown. Secrecy is indeed the norm regarding the performances of industrial solutions, and available object recognition datasets are not representative of this scenario, as they typically consider only a limited number of object instances lying on a flat surface on a given face, because of the cost of pose annotation.

This article tries to address those issues. In section 3, we propose automatic techniques for generating fully annotated real range image and synthetic RGBD datasets of many instances of objects in arbitrary poses, with no redundancy between the acquisitions, contrary to existing approaches. We suggest in section 4 to take the object symmetries into account for both evaluating accuracy and improving performances of existing pose estimation methods. Section 5 is focused on the evaluation problem, for which we formalize performance metrics in order to better deal with scenes containing many object instances. These developments are supported by experiments on two well-established object detection and pose estimation methods, in section 6.

Related work on 3D object pose estimation evaluation

Several approaches have been proposed to evaluate 3D object detection and pose estimation for manipulation tasks based on robotic experiments [START_REF] Horn | Picking parts out of a bin[END_REF][START_REF] Liu | Fast object localization and pose estimation in heavy clutter for robotic bin picking[END_REF]9,[START_REF] Rodrigues | 6D pose estimation of textureless shiny objects using random ferns for bin-picking[END_REF], but pose accuracy is difficult to evaluate in such scenario due to the lack 1 Figure 1: Samples exhibiting the strong correlation of data within existing datasets. From left to right: datasets of Tejani et al. [START_REF] Tejani | Latent-Class hough forests for 3D object detection and pose estimation[END_REF], Hinterstoisser et al. [START_REF] Hinterstoisser | Model based training, detection and pose estimation of texture-less 3D objects in heavily cluttered scenes[END_REF]13], T-LESS [START_REF] Hodaň | T-LESS: An RGB-D dataset for 6D pose estimation of texture-less objects[END_REF] and Desk3D [START_REF] Bonde | Robust instance recognition in presence of occlusion and clutter[END_REF].

of ground truth, and its online nature makes reproducibility difficult. The use of datasets annotated with ground truth poses of every visible object instance instead enables a more quantitative and reproducible offline evaluation. Over the years, several publicly available datasets emerged and we summarize the characteristics of major ones in table 1. Other 3D object recognition datasets of interest obviously exist (see notably Firman [START_REF] Firman | RGBD datasets: Past, present and future[END_REF]) but we focus here on the case of rigid objects without intra-class variability. Those datasets (except for T-LESS [START_REF] Hodaň | T-LESS: An RGB-D dataset for 6D pose estimation of texture-less objects[END_REF]) consist in views of scenes of a few objects lying on a given face on a table, acquired from pan-tilt viewpoints with only limited roll along the sensor axis. As such, they provide useful material for working on tasks such as indoor scene understanding, but their limited variability of poses relative to the camera is not representative of the problem of localizing objects in arbitrary poses.

Moreover, and because manually annotating objects poses in each image is a tedious process, datasets of more than a few tens of images [START_REF] Hinterstoisser | Model based training, detection and pose estimation of texture-less 3D objects in heavily cluttered scenes[END_REF][START_REF] Tejani | Latent-Class hough forests for 3D object detection and pose estimation[END_REF][START_REF] Bonde | Robust instance recognition in presence of occlusion and clutter[END_REF][START_REF] Hodaň | T-LESS: An RGB-D dataset for 6D pose estimation of texture-less objects[END_REF][START_REF] Doumanoglou | Recovering 6d Object Pose and Predicting Next-Best-View in the Crowd[END_REF] rely on automatic annotation techniques. Objects instances are typically placed at given poses relatively to some fiducial markers, whose automatic detection in the scene enable to compute the poses of instances relatively to the camera. This approach enables to easily generate large annotated datasets, however consisting in numerous acquisitions of a few scenes from different viewpoints (see figure 1). This strong correlation between data samples makes those datasets less representative of the data distribution in a genuine application, and training or performance evaluation on such datasets is therefore likely to suffer from some overfitting effects.

Pose accuracy An essential part of the evaluation methodology consists in selecting a criterion to decide whether or not a pose hypothesis matches a ground truth pose. Such matching criterion is typically defined based on a measure of similarity between the pose hypothesis and the ground truth. Several similarity measures have been consid-ered for this task, such as the Intersection over Union ratio of 2D silhouettes, the translational and rotational errors [START_REF] Drost | Model globally, match locally: Efficient and robust 3D object recognition[END_REF], and the average displacement of vertices of a 3D model of the object [START_REF] Hinterstoisser | Model based training, detection and pose estimation of texture-less 3D objects in heavily cluttered scenes[END_REF]. The first criterion, while widely used for 2D object localization, is however not suited for 3D pose estimation, as multiple poses may have similar projected silhouette on the image plane. The latter two are not well defined for objects with symmetry properties, since there is no unique displacement between two poses of a symmetric object. The most widespread workaround to this issue consists in the use of an ad hoc dissimilarity measure [START_REF] Hinterstoisser | Model based training, detection and pose estimation of texture-less 3D objects in heavily cluttered scenes[END_REF], based on the distance between vertices of a 3D model M of the object at those given poses

avg x1∈M min x2∈M T 1 (x 1 ) -T 2 (x 2 ) , (1)
where T 1 , T 2 ∈ SE(3) are active rigid transforms representing the poses. However, such criterion remains problematic, as it cannot distinguish between poses of similar 3D shapes, such as the flipped poses of coffee cups depicted figure 3a. Hodan et al. [START_REF] Hodan | On evaluation of 6D object pose estimation[END_REF] recently suggested the use of an ambiguity-invariant pose error function, considering a pose hypothesis as valid if and only if it is plausible given the available data. While not denying the interest of dealing with ambiguities, especially in the context of active vision, we consider this approach problematic for evaluation, since numerous applications rely on a precise pose estimation and cannot be satisfied by plausible hypotheses. This is all the more true when dealing with highly occluded objects, for which the number of plausible hypotheses can be infinitely large.

Precision and recall Once able to classify pose results, one can define metrics to quantify the performances of a given algorithm. In the case of pose estimation of a single object instance per scene, performances are typically described by the recognition rate [START_REF] Johnson | Using spin images for efficient object recognition in cluttered 3D scenes[END_REF][START_REF] Drost | Model globally, match locally: Efficient and robust 3D object recognition[END_REF][START_REF] Hinterstoisser | Model based training, detection and pose estimation of texture-less 3D objects in heavily cluttered scenes[END_REF], that is the fraction of scenes for which the pose returned is the correct one on the whole dataset. This metric is however insufficient for scenes containing an unknown number of instances, as it provides no information on the false positive results produced. In such case, the performance is typically described in terms of precision and recall [START_REF] Tejani | Latent-Class hough forests for 3D object detection and pose estimation[END_REF], averaged over all the data samples. Such approach however focuses on the exhaustive retrieval of every instance in each scene, which might not be the actual objective of use cases dealing with scenes of many instances.

In this article, we propose a methodology to generate both real and synthetic data of many object instances in arbitrary poses, with no dependency between data samples, in order to overcome the limitations of existing datasets, and metrics adapted for performance evaluation in scenes of many object instances, with potential symmetries.

Generating annotated datasets of many instances

We propose in this section two methods to generate datasets of independent scenes of many object instances in arbitrary poses, automatically annotated with the poses of those objects. The first method is based on the annotation of real range data thanks to the use of tagged object instances. The second one relies on computer simulations. We consider in our experiments multiple instances of a single object because it is representative of the bin-picking problem, however multiple objects could be considered in a similar fashion.

Real data

We cover the surface of each instance of object with a set of unique fiducial markers [START_REF] Garrido-Jurado | Automatic generation and detection of highly reliable fiducial markers under occlusion[END_REF], densely enough such that at least one marker is always visible from any point of view. The pose of a marker relative to its corresponding object instance is assumed to be known, and in our experiments, we ensured the precise positioning of markers by carving their locations on the 3D model of the object, and by producing its instances by 3D printing. This shape modification is not required but is merely performed here for convenience, and precise positioning of markers could be achieved e.g. through the use of a specific assembly jig. Using a binocular stereoscopic system, we produce a range image for each scene through the use of a pseudo-random pattern projector and an off-the-shelf stereo matching algorithm. We also acquire intensity images from both camera with a diffuse lighting, depicted in figure 2. This second modality is exploited in order to automatically recover the ground truth poses.

Visible markers can indeed be quite reliably detected, identified and localized within the intensity images. Because each marker is uniquely assigned to a given instance of object, object detection is straightforward: a given instance is considered present in the scene if and only if at least one of its marker is detected. The potential occlusion of every marker of an instance is not a major issue for this approach, as it only occurs when the instance is nearly en- tirely occluded -provided that the object is densely enough covered by small enough markers -and pose annotation is of limited interest such case.

For each detected instance, marker detection in intensity image j ∈ {1, 2} provides us with the 2D corners coordinates p i,j ∈ R 2 , i ∈ M j of the detected markers associated with the instance. Because the pose of those markers relative to the object instance is known, each p i,j can be put in relation with its corresponding 3D point X i,j ∈ R 3 within the object frame. The pose [ R, t] of the considered instance -described by a 3 × 3 rotation matrix R and a 3D translation vector t -can therefore be estimated by solving the multiview perspective-n-point problem

[ R, t] = argmin R,t j∈{1,2} i∈Mj π j (RX i,j + t) -p i,j 2 ,
(2) where π j represents the projection from a 3D point in the reference coordinate system onto the image plane of camera j. The annotation procedure is therefore automatic, but we proceed in our experiments to a visual validation stage to ensure the quality of the ground truth. Validation of about 700 scenes required less than 40min for a single person, and 6% of images were discarded due to failures of the marker detector used, consisting either in false positives, false negatives, mislabeled markers or imprecision in the localization of markers' corners. This failure rate could probably be diminished significantly through more robust markers detection, but we considered it acceptable given that scene acquisition and validation of automatic annotations are quite fast to perform. Additional annotations such as the occlusion rate of each instance are produced by comparison with CGI-renderings of the instances at the annotated poses.

Synthetic data

While effective, this marker-based approach is intrusive, in that markers remain visible in intensity images, which limit the suitability of this approach for object recognition techniques based on such modality. Those issues can be avoided through the use of synthetic data, for which ideal ground truth can be produced. It is indeed possible to generate synthetic datasets reasonably representative of controlled environments such as an industrial setup for bin-picking, and for our experiments we produce synthetic scenes of bulk objects through the physical simulation of drop of instances into a bin, from which we synthesize topview RGBD images (see examples figure 4).

Sensor simulation Producing plausible synthetic RGB images is relatively simple thanks to existing ray-tracing renderers, but while range data simulators [18] or depth noise models (e.g. [START_REF] Buchholz | RANSAM for industrial Bin-Picking[END_REF]) have been proposed, they remain quite sensor-dependent and their tuning is not trivial. To overcome those issues, we render stereo image pairs of the synthetic scene lit by a virtual pseudo-random pattern projector using the Blender Cycles renderer [20], and perform on those images the same 3D stereo reconstruction as in our real experiments. This approach produces range images with a 3D reconstruction noise visually similar to the one observed in our real data, without the need for an explicit noise model, and despite a coarse modeling of the scene (we discuss this question further in section 6.2). The reconstruction technology we simulate here is similar to the one used in industrial cameras such as the Ensenso N10, and other triangulation-based sensors such as LASER scanners or Kinect v1 could be simulated in a similar fashion, given access to the processing algorithms used by these devices to transform raw input data into depth images.

Based on those approaches, we generated both real and synthetic datasets, considering objects with various symmetry properties, in scenes of various number of instances piled up in bulk (see fig. 4 and supplementary material). The clutter dataset is an exception depicting very cluttered scenes, to illustrate the suitability of our annotation proce-dure in such scenario.

Dealing with symmetric objects

Being able to quantify the accuracy of a pose estimate is a prerequisite for evaluation, but usual measures are not fit to deal with symmetric objects. In this section, we suggest the use of a distance suited to any bounded rigid object, and discuss how it can be used within 3D object detection and pose estimation techniques themselves, in order to increase performances.

Pose distance

Brégier et al. [START_REF] Brégier | Defining the pose of any 3D rigid object and an associated distance[END_REF] recently proposed a pose definition valid for any rigid object -including those with symmetries -as a distinguishable static state of the object. Within their framework, a pose can be identified to an equivalency class {T • G|G ∈ G} of SE(3), defined by an active rigid transformation T and up to any rigid transformation the object is invariant to (the proper symmetry group G ⊂ SE(3)).

Given a bounded rigid object, they propose a distance between two poses consisting in the length of the smallest displacement from one pose to an other, the length of a displacement being defined as the RMS displacement of surface points of the object

d(P 1 , P 2 ) min G1,G2∈G 1 S S T 2 • G 2 (x) -T 1 • G 1 (x) 2 ds, (3)
where S is the surface area of the object. This distance is physically meaningful, and accounts properly for object's symmetries contrary to the widespread measure (1) as illustrated figure 3. Moreover, it can be estimated efficiently in closed form, whereas the measure (1) requires to perform systematically a computation on every vertex of the model. To this aim, the authors propose a representation of a pose P as a finite set of points R(P) of at most 12 dimensions, depending on the proper symmetry class of the object. Those representations are listed in table 2 and in particular,

• A pose of an object without proper symmetries (such as the bunny covered with markers used in our experiments) is represented by a 12D point, consisting in the concatenation of the 3D position of its centroid t and of its rotation matrix R, anisotropically scaled by a matrix Λ to account for the object's geometry.

• A pose of an object with a finite non-trivial proper symmetry group is represented by several of those 12D points, two in the case of the brick object figure 4 which is invariant under the group of proper symmetries G = {I, R π z } consisting in the identity transformation and a rotation of 1/2 turn around a given axis. • Poses of revolution objects (such as pepper and gear figure 4) are represented by 6D vectors consisting in the concatenation of the 3D position of their centroids and the direction of their revolution axis, scaled to account for the object's geometry.

Using this representation, they show that the distance between two poses P 1 , P 2 can be evaluated as the Euclidean distance between any given point p 1 ∈ R(P 1 ) and the pointset R(P 2 ):

∀p 1 ∈ R(P 1 ), d(P 1 , P 2 ) = min p2∈R(P2) p 2 -p 1 . (4)
This "nearly Euclidean" structure enables to perform neighborhood queries such as nearest-neighbor or radius searches in an exact or approximate fashion using existing algorithms developed for Euclidean spaces (regular grids, kD-trees, etc.). It also enables to average poses of an object with potential symmetries efficiently. Thanks to those possibilities, high-level techniques can be developed in the space of poses, such as the Mean Shift algorithm, which finds local maxima of density distributions.

Given its advantages, we suggest to use this distance for quantifying the accuracy of a pose estimation. It expresses the RMS error in positioning of surface points of the object, and we define a criterion m(p, t) (d(p, t) < δ) to decide whether or not a pose hypothesis p matches a ground truth pose t based on a hard threshold δ, in length unit. The choice of such threshold is very application-dependent, but without further information we chose arbitrarily a value of 10% of the diameter of the smallest sphere enclosing the object and centered on the centroid of object's surface.

Considering symmetries for 3D pose estimation

Finding modes An important class of 3D detection and pose estimation methods are based on the aggregation of multiple votes for poses [START_REF] Drost | Model globally, match locally: Efficient and robust 3D object recognition[END_REF][START_REF] Kim | 3D object recognition in range images using visibility context[END_REF][START_REF] Rodrigues | 6D pose estimation of textureless shiny objects using random ferns for bin-picking[END_REF][START_REF] Tejani | Latent-Class hough forests for 3D object detection and pose estimation[END_REF]23,24] in order to generate pose hypotheses. They typically rely on density estimation, modes finding techniques such as Mean Shift, or clustering operations. Those operations depend on the choice of metric over the pose space, and can actually be performed efficiently considering the distance (3) (see [START_REF] Brégier | Defining the pose of any 3D rigid object and an associated distance[END_REF] for details). Contrary to a metric suited for SE(3), the suggested one enables to take object symmetries into account directly in the 3D pose estimation method, and to better exploit the aggregate of votes. We test this hypothesis in our experiments by adapting to this approach the method of Drost et al. [START_REF] Drost | Model globally, match locally: Efficient and robust 3D object recognition[END_REF], to which we will refer as PPF.

Filtering duplicates More generally, considering symmetries can improve the performance of any 3D pose estimation method thanks to early duplicate removal, as one can ignore pose hypotheses too similar to an already retained one, according to the distance (3). Removing duplicates is a simple way to increase the precision in object detection and pose estimation, and performing it at an early stage benefits to computation time, by limiting the number of pose hypotheses to refine and validate. Equivalently given a constant number of pose hypotheses retained, duplicates removal enables to consider a larger number of truly different pose hypotheses, which should benefit to the recall. We experiment this approach on two methods: PPF evoked previously; and the sliding window technique of Hinterstoisser et al. [START_REF] Hinterstoisser | Model based training, detection and pose estimation of texture-less 3D objects in heavily cluttered scenes[END_REF] (referred to as LINEMOD+).

Performance metrics

In scenes containing many parts, retrieving the pose of every single object instance is not always required. The pose of very occluded objects is indeed often ambiguous, and the retrieval of a limited number of instances is sufficient for many applications -e.g. in robotic manipulation. We propose here some adaptations of the usual performance metrics -precision and recall -to take those aspects into account. We discuss the case of a scene containing potentially multiple instances of a rigid object, but multiple scenes should obviously be considered for statistical robustness through the use of aggregated metrics, such as the mean precision and recall. The multi-object case can be handled similarly, as long as no specific relations between the objects are implied (e.g. no object categories). Let T be the set of ground truth poses of object instances within the scene, and P the set of result poses retrieved via an object detection and pose estimation method.

α z |α ∈ R} R δ x R α z δ ∈ {0, π} , α ∈ R SO(3) G ⊂ SO(3) Pose representatives R(P) (λ(Rez) , t ) ∈ R 6 (±λ(Rez) , t ) ⊂ R 6 t ∈ R 3 (vec(RGΛ) , t ) |G ∈ G ⊂ R 12
Assumptions: Object frame (O, ex, ey, ez) chosen such as O to be the center of mass of the object and the ez axis to be aligned with the symmetry axis for revolution objects. Notations: I is the identity rotation. R α

x , R α y , R α z represent rotations of angle α ∈ R around respectively ex, ey, ez axes.

Λ ( 1

S S xx ds) 1/2
, and λ λ 2 r + λ 2 z for revolution objects where Λ = diag(λr, λr, λz).

Instances of interest Only a subset T o ⊂ T of object instances present in the scene might be of interest to retrieve.

In our experiments, we choose T o {t ∈ T |o(t) < δ o } as the subset of instances t with an occlusion rate o(t) smaller than δ o = 50%. Given T o , we define the notions of true positives (T P ), false positives (F P ), and false negatives (F N )

T P = {(p, t) ∈ P ×T o |m(p, t) ∧ p = n P (t) ∧ t = n T (p)} F P = {p ∈ P |¬m(p, n T (p)) ∨ n P (n T (p)) = p} F N = {t ∈ T o |¬m(t, n P (t)) ∨ n T (n P (t)) = t} , (5)
where n S (q) argmin r∈S d(q, r) is the nearest pose within a set S from a pose q. Duplicates are considered as false positives with this definition, and a result corresponding to an instance whose retrieval is of no interest (within T \ T o ) is neither considered as a true nor a false positive. The complementary notions of precision and recall can then be derived as

Experiments

This section presents experiments performed on object detection and pose estimation using the PPF and LINEMOD+ methods, adapted to deal with symmetric objects (see section 4.2). Through these, we aim to illustrate our different evaluation proposals suited for scenes depicting multiple instances of any rigid object; and also to bring out the benefits of considering symmetries for object detection and pose estimation.

Protocol

Our experiments are based on the LINEMOD implementation of Stefan Holtzer available in PCL [START_REF] Rusu | 3D is here: Point cloud library (PCL)[END_REF], and on our own implementation of PPF. Prior to evaluation, we refine pose hypotheses thanks to a projective ICP algorithm -an additional step often performed to improve pose estimation accuracy [START_REF] Hinterstoisser | Model based training, detection and pose estimation of texture-less 3D objects in heavily cluttered scenes[END_REF]24,[START_REF] Kehl | Deep learning of local RGB-D patches for 3D object detection and 6D pose estimation[END_REF]. The evaluation therefore focuses on the ability to generate pose hypotheses within the convergence basin of poses of actual instances. We perform experiments on both our real and synthetic datasets (some of those generated using object models from T-LESS [START_REF] Hodaň | T-LESS: An RGB-D dataset for 6D pose estimation of texture-less objects[END_REF]), as well as the two bin-picking datasets from Doumanoglou et al. [START_REF] Doumanoglou | Recovering 6d Object Pose and Predicting Next-Best-View in the Crowd[END_REF]. Computation time is not evaluated here, as we used unoptimized code unrepresentative of the original methods.

Post-processing step The addition of a post-processing step (PP) for objects pose hypotheses has been shown to substantially improve performances [START_REF] Hinterstoisser | Model based training, detection and pose estimation of texture-less 3D objects in heavily cluttered scenes[END_REF]27]. We evaluate this effect by considering variants of PPF and LINEMOD+ consisting in keeping the 20 best hypotheses returned by the method, scoring them according to their consistency with the input data and filtering duplicates.

Considering symmetries As discussed in section 4.2, we also evaluate the impact of considering the proper symmetries of the object if any (suffixed by sym in table 3). We use the same set of templates for LINEMOD+ in both cases, so as not to bias the comparison.

Suitability of synthetic data for evaluation

The use of synthetic data raises the question of its suitability for evaluation purposes. To assess the usefulness of our depth sensor simulation procedure for this task, we generate a synthetic dataset of 308 images depicting virtual copies of real scenes of object instances lying on a flat background (markers flat), and we perform evaluations on both datasets. These virtual scenes are synthesized automatically based on the pose annotations of real data produced using markers, and a plane detection algorithm for the background. Parameters of the virtual cameras used for range data generation match those of real cameras, and figure 5 depicts an example of synthesized data.

Because we used simple and ideal material and lighting in our CGI renderings, the synthetic data produced is different from the real one, and of slightly better quality (96.6% of the pixels with depth information, against 96.2% for the real data). This quantitatively affects performances and every pose estimation method we tested showed slightly better performances on the synthetic dataset than on the real one (see markers flat results in table 3). Nonetheless, synthetic data remains plausible thanks to our depth sensor simulation procedure, and the performances of the evaluated methods compare to one another very similarly on both datasets, as illustrated figure 6. We therefore consider the synthetic data produced to be realistic enough for comparative evaluation purposes.

Effect of accounting for object symmetries

While precision-recall curves such as the ones figure 6 provide a finer-grained understanding of performances, we synthesize our results for the sake of readability in table 3 through different metrics. We consider the Average Precision (AP) [28], a usual metric consisting in the area under the precision-recall curve, given the goal of retrieval of instances less than 50% occluded. One might only be interested in the retrieval of a few object instances -e.g. for robotics manipulation -therefore we also present the Average Precision given at most n ∈ {1, 3} results returned (APn). The formalism enabling to compute those metrics is defined in section 5.

Results discussion

As expected, adding a basic postprocessing step (PP) to both PPF and LINEMOD+ significantly improves performances on every dataset (with an average AP improvement of respectively +59% and +60%).

Taking object symmetries into consideration also leads to great improvements for the PPF method, with or without postprocessing and regardless of the considered metrics (e.g. respectively +18% and +99% increases of AP3). The PPF method indeed relies on the aggregation of multiple weak pose hypotheses in order to generate stronger ones, and considering symmetries significantly helps in this aggregation, as discussed in section 4.2. LINEMOD+ also benefits from considering symmetries (+54% and +93% increases of AP respectively with and without post-processing). Since symmetry considerations are only used to filter out duplicates prior to pose refinement, we should not observe performance improvements when retrieving at most one pose hypothesis without postprocessing for this method. The small difference observed here (+1% increase of AP1) is merely an artifact of our implementation, which includes a basic filtering step of pose hypotheses outside the frustum of the camera after pose refinement. Accounting for symmetries however benefits in the other cases to both precision, by removing duplicates, and recall, as it allows to consider more truly different poses for a given number of pose hypotheses, a point we observe in the global performance improvements of LINEMOD+ PP (+54% AP and +11% AP3) which already includes a duplicates filtering step, even without symmetry considerations.

Conclusion

We focused in this article on the evaluation of 3D rigid object detection and pose estimation techniques, in scenes containing an arbitrary number of instances, in arbitrary poses. We proposed two methods to generate automatically annotated datasets of such scenes, and metrics suited for the performance evaluation of this generic scenario, even in the case of symmetric objects. We showed how those symmetry considerations could be adapted within existing pose estimation methods themselves, and our experimental results suggest that it leads to significant performance improvements.

Figure 2 :

 2 Figure 2: Automatic production of annotated range images of many object instances in bulk. Range images are produced by active stereo matching (left), while ground truth poses are annotated by detecting, in intensity images, markers placed on object instances (right).

  Hinterstoisser et al. [4].

Figure 3 :

 3 Figure 3: Dissimilarity (in % of the object's diameter D) between pose hypotheses (overlaid contours) and the corresponding ground truth poses. In this example from Doumanoglou's dataset[START_REF] Doumanoglou | Recovering 6d Object Pose and Predicting Next-Best-View in the Crowd[END_REF], each pose hypothesis would be considered as valid according to the widespread criterion of Hinterstoisser et al.[START_REF] Hinterstoisser | Model based training, detection and pose estimation of texture-less 3D objects in heavily cluttered scenes[END_REF] for symmetric objects (dissimilarity below 10% • D). The proposed distance on the other hand properly accounts for the object's symmetries, which enables to discriminate well true positives (green) and false positives (upside down hypotheses, in red).

6 )

 6 precision = |T P |/(|T P | + |F P |), recall = |T P |/(|F N | + |T P |). (Limited number of retrievals To properly handle the case where the number of results |P | is restricted to n ∈ N * , we propose to alter the definition of recall as follows recall ≤n results = |T P |/ min(n, |F N | + |T P |). (7)

Figure 4 :

 4 Figure 4: Examples of results from our experiments in object detection and pose estimation. Top: considered objects. Middle: RGB images, with silhouettes contours of the first 5 results returned by the PPF sym method with post-processing, classified as true positives (green) or false positives (red). Bottom: corresponding range data. * : the false positive brick highlighted in red is flipped upside down compared to the corresponding ground truth.

Figure 5 :

 5 Figure 5: Synthetic dataset depicting virtual copies of real scenes for comparison purposes (depth and intensity).

Figure 6 :

 6 Figure 6: Comparison of performances obtained with real and synthetic data. Precision-recall curves for the retrieval of less than 50% occluded instances.

Table 1 :

 1 Characteristics of typical rigid object detection and pose estimation datasets.We distinguish between localization and detection[START_REF] Hodan | On evaluation of 6D object pose estimation[END_REF], depending on whether the number of instances to retrieve is constant or not. A dataset is considered redundant if it contains numerous acquisitions of the same scene.

	Dataset	Modality	Problem class	Multiple instances	Multiple objects	Absence of data redundancy	Pose variability	Clutter
	Mian et al. [2]	Point cloud	Localization	no	yes	yes	limited	no
	Aldoma et al. [12]	Colored point cloud	Detection	no	yes	partial	limited	no
	Hinterstoisser et al. [4, 13]	RGBD	Localization	no	yes	no	limited	yes
	Desk3D [14]	Point cloud	Detection	no	yes	no	limited	yes
	Tejani et al. [5]	RGBD	Localization	yes	no	no	limited	yes
	Doumanoglou et al. [15]	RGBD	Detection	yes	yes	no	high	limited
	T-LESS [16]	RGBD	Detection	yes	yes	no	high	limited

Table 2 :

 2 Classification of every potential group of proper symmetry for a 3D bounded physical object, and expression of pose representatives enabling fast distance computations.

	Proper symmetry class	Revolution without rotoreflection invariance	Revolution with rotoreflection invariance	Spherical	Finite
	Proper symmetry group G	{R

Table 3 :

 3 Performances of two object detection and pose estimation methods, and their variants exploiting object symmetries (sym) if any, and post-processing of the best 20 pose hypotheses. Accounting for symmetries improves performances of both methods.

			Raw method	With post-processing of the best 20 pose hypotheses (PP)
		PPF	PPF sym	LINEMOD+ LINEMOD+ sym	PPF	PPF sym	LINEMOD+ LINEMOD+ sym
	Dataset	AP AP1 AP3 AP AP1 AP3 AP AP1 AP3 AP AP1 AP3	AP AP1 AP3 AP AP1 AP3 AP AP1 AP3 AP AP1 AP3
	markers bump .35 .66 .43 -	-	-
	Real data

.85 1.00 .96 -


Adrien Morel
email: adrien.morel@cea.fr

Adrien Badel

Pierre Gasnier

David Gibus

Gaël Pillonnet

Short Circuit Synchronized Electric Charge Extraction (SC-SECE): a tunable interface for wideband vibration energy harvesting

published or not. The documents may come   L'archive ouverte pluridisciplinaire

I. INTRODUCTION

In order to make small systems and sensors autonomous scavenging ambient energy has been widely investigated in the last two decades as an alternative to batteries. Piezoelectric energy harvesters (PEH) are of particular interest in closed confined environments, where there are few solar radiations and thermal gradients.

In order to maximize the energy harvested from piezoelectric harvesters, the electrical interface is a key point to consider. Several non-linear synchronous strategies, such as Synchronous Electric Charge Extraction (SECE) and Synchronized Switch Harvesting on Inductance (SSHI) have been introduced [START_REF] Lefeuvre | A comparison between several vibration-powered piezoelectric generators for standalone systems[END_REF], and have been implemented using discrete components [START_REF] Shi | An efficient self powered synchronous electric charge extraction interface circuit for piezoelectric energy harvesting systems[END_REF][START_REF] Wu | Piezoelectric vibration energy harvesting by optimized synchronous electric charge extraction[END_REF] or dedicated ASIC [START_REF] Quelen | A 30nA Quiescent 80nW to 14mW Power Range Shock-Optimized SECE-based Piezoelectric Harvesting Interface with 420% Harvested Energy Improvement[END_REF][START_REF] Hehn | A Fully Autonomous Integrated Interface Circuit for Piezoelectric Harvesters[END_REF]. Those strategies exhibit high performance for lowly coupled and/or highly damped piezoelectric harvesters. However, for highly coupled and/or lowly damped piezoelectric harvesters, these strategies may overdamp the mechanical resonator, leading to low performances. To face this challenge, researchers started to propose new tunable strategies inducing lower damping [START_REF] Morel | Regenerative synchronous electrical charge extraction for highly coupled piezoelectric generators[END_REF], and even tuning the PEH resonant frequency thanks to the important influences of the electrical interface on the mechanical resonator [START_REF] Morel | Short Circuit Synchronous Electric Charge Extraction(SC-SECE) Strategy for Wideband Vibration Energy Harvesting[END_REF].

In this paper, we introduce a strategy based on the SECE interface, which introduces a tunable short-circuit phase. As detailed extensively in this paper, this short-circuit allows to reduce the damping induced by the electrical interface. Furthermore, for high coupling harvesters, it allows to tune the PEH resonant frequency, leading to an enhanced harvesting bandwidth.

II. THEORETICAL MODELLING

A. Linear PEH modelling

A linear PEH under a periodic excitation can usually be modelled by a system of linear differential equations, given by [START_REF] Lefeuvre | A comparison between several vibration-powered piezoelectric generators for standalone systems[END_REF].

𝑀𝑥 + 𝐷𝑥 + 𝐾 )* 𝑥 + α𝑣 -= -F = -𝑀𝑦 𝑖 = α𝑥 -𝐶 -𝑣 - 𝑥 𝑡 = 𝑋 7 cos(θ) = 𝑋 7 cos (ω𝑡) (1)
where 𝑦, 𝐹 and 𝑥 stand for the ambient displacement, the force applied on the PEH, and the tip mass displacement, respectively. 𝑀, 𝐾 )* , 𝐷, 𝐶 -and 𝛼 stand for the equivalent mass of the PEH, its short-circuited stiffness, its mechanical damping, the capacitance of the piezoelectric material, and the piezoelectric force coefficient, respectively. Fig. 1 shows the electrical circuit modelling these equations.

B. Expression of the piezoelectric voltage

In order to find the harvested power expression, we have to solve (1) and find the mechanical displacement magnitude 𝑋 7 . As v C is a variable in (1), finding a linear expression of 𝑣 -would greatly simplify the calculations. The piezoelectric harvester is either working in open or short-circuit, as illustrated in Fig. 2. Thus, the piezoelectric voltage during a single vibration period is given by [START_REF] Shi | An efficient self powered synchronous electric charge extraction interface circuit for piezoelectric energy harvesting systems[END_REF].

v C θ = α C C 𝑥 θ dθ F G H IJGKL , ∀θ ∈ ]ϕ Q + Δϕ -π, ϕ Q ] 0, ∀θ ∈ ]ϕ Q , ϕ Q + Δϕ] α C C 𝑥 θ dθ F G H IJG , ∀θ ∈ ]ϕ Q + Δϕ, ϕ Q + π] 0, ∀θ ∈ ]ϕ Q + π, ϕ Q + Δϕ + π] (2)
ϕ Q ∈ [0, 𝜋] corresponds to the angular phase between the harvesting process and the precedent displacement extremum. Δϕ ∈ [0, 𝜋] stands for the angular time spent in the short-circuit phase. A system implementing this extraction strategy is depicted in Fig. 1, and an example of the voltage waveform is shown in Fig. 2. As expressed by (2), v C is not sinusoidal. Only the first harmonic of v C is considered in order to analytically solve (1) while simplifying the calculations. 2), we eventually find the expression of the Fourier series coefficients a X and b X associated with 𝑣 -:

a X = αX [ πC C [π -𝛥𝜙 + sin 2𝜙 ) + 2𝛥𝜙 2 + sin 2𝜙 ) 2 +2 cos 𝜙 ) + 𝛥𝜙 sin 𝜙 ) ] b X = - αX [ πC C cos 𝜙 a + cos 𝜙 a + ∆𝜙 c (3)
Hence, the first harmonic of v C , v C X , is expressed by [START_REF] Quelen | A 30nA Quiescent 80nW to 14mW Power Range Shock-Optimized SECE-based Piezoelectric Harvesting Interface with 420% Harvested Energy Improvement[END_REF].

v C X = x a X 𝑋 7 -j b X 𝑋 7 = x a X * -jb X * (4)
where 𝑥 is the mechanical displacement in the Fourier domain. a X * and b X * are the first Fourier coefficients a X and b X divided by the displacement amplitude, 𝑋 7 .

C. Expression of the harvested power

Due to the filtering effect of the resonator, we consider that the first voltage harmonic may impact the PEH dynamics. We can hence substitute the voltage expression (4) in (1) to find the displacement amplitude. Solving (1) in the Fourier domain, the mechanical displacement amplitude can be expressed by [START_REF] Hehn | A Fully Autonomous Integrated Interface Circuit for Piezoelectric Harvesters[END_REF].

𝑋 7 = 𝑀𝑦 𝐾 )* -𝑀ω c + αa X * c + ω𝐷 + αb X * c (5)
For each vibration's semi-period, the harvested energy is the one stored in 𝐶 -when θ = ϕ Q . Thus, from (1), we can derive the harvested power expression:

P hijk = ωα c 2π𝐶 - X [ c cos 𝜙 ) + 𝛥𝜙 + cos 𝜙 ) c (6)
This power P hijk , divided by the maximum harvestable power 𝑃 [im = 𝐷 KX 𝑀𝑦 c /8 [START_REF] Morel | Regenerative synchronous electrical charge extraction for highly coupled piezoelectric generators[END_REF] has been computed in Figure 3 with optimized parameters (𝜙 a , 𝛥𝜙) as a function of the normalized vibration frequency 𝛺 7 = 𝜔/𝜔 r with three normalized squared coupling coefficients 𝑘 7 c and a mechanical quality factor of 𝑄 7 =25. These normalized parameters have been extensively described in [START_REF] Morel | Regenerative synchronous electrical charge extraction for highly coupled piezoelectric generators[END_REF][START_REF] Morel | Short Circuit Synchronous Electric Charge Extraction(SC-SECE) Strategy for Wideband Vibration Energy Harvesting[END_REF]. We can observe that the bandwidth gain becomes more important as 𝑘 7 c is increased.

III. CONCLUSION

In this paper, we present the SC-SECE strategy, which is based on a combination of the SECE interface and a tunable short-circuit. Numerical results show that this strategy could be used to both enhance the harvested power and the harvesting bandwidth of highly coupled PEH Experiments confirming these theoretical predictions are currently being run, and will be presented during the conference.

Fig. 1 .Fig. 2 .

 12 Fig.1. System view of the proposed SC-SECE strategy[START_REF] Morel | Short Circuit Synchronous Electric Charge Extraction(SC-SECE) Strategy for Wideband Vibration Energy Harvesting[END_REF]

Fig. 3 .

 3 Fig. 3. Frequency responses of highly coupled piezoelectric harvesters with optimized resistive loads (dashed lines) and the proposed SC-SECE interface (straight lines)

/ " 0 = 0.29 / " 0 = 0.43 / " 0 = 0.86