Eder A Rodríguez Martínez

Guillaume Caron
email: guillaume.caron@u-picardie.fr

Claude P Égard
email: claude.pegard@u-picardie.fr

David Lara-Alabazares

David Lara
email: dlaraa@misantla.tecnm.mx

Photometric-Planner for Visual Path Following

Keywords: Vision-Based Robotic Navigation, Direct Visual Servoing, Aerial Robotics

Robotic navigation is the aspect of cognition related to robot robust mobility. It combines perception, some knowledge on the environment and a set of goal poses to reliably control the robot during a mission that involves displacement. Visionbased autonomous navigation is an instantiation of the latter discipline where visual perception is used to control the robot and to represent the environment. This paper presents a visionbased navigation system that uses its onboard camera to navigate and a visual path that represents the scene with a set of images. Being a memory-based system, the navigation is conceived as a concatenation of positioning tasks in the visual servoing scheme. The novelty on the proposed system relies on the generation of the images that compose the visual path. These images are rendered from a preobtained model of the scene. The experiments evaluate the performance of the system over three different scenes, contemplating indoor and outdoor environments, and using an Unmanned Aerial Vehicle as testbed.

I. INTRODUCTION

V ISION-based autonomous robot navigation is the process of making the robot being able to reach a final pose, which is out of sensing range from a start pose, using vision for perception. The robot, in order to navigate from the start to the final pose, requires a path, that is usually planned offline, online or both [START_REF] Yuncheng | A survey on vision-based UAV navigation[END_REF]. Once the planned path is known, most of the approaches estimate the robot pose, i.e. localization, while moving, using visual odometry [START_REF] Scaramuzza | Visual Odometry [Tutorial[END_REF] or Simultaneous Localization And Mapping (SLAM) [START_REF] Bresson | Simultaneous Localization and Mapping: A Survey of Current Trends in Autonomous Driving[END_REF]. Then, the estimated current pose is taken as input to control the robot in order to successively follow the intermediate poses that compose the path [START_REF] Sujit | Unmanned Aerial Vehicle Path Following: A Survey and Analysis of Algorithms for Fixed-Wing Unmanned Aerial Vehicles[END_REF]. The common scheme described above presents three distinct stages: path planning, visual perception and motion control.

The visual perception stage of the autonomous robotic navigation mostly deals with keypoints and geometric features detected and matched in images. Such geometric information can be related to the robot pose, considered in the planning as well as in the control stages. Nevertheless, robust extraction and matching of the features over a large environment is still a bottleneck for the visual navigation schemes [START_REF] Raj | Appearance-based indoor navigation by IBVS using mutual information[END_REF]. Therefore, an alternative to the geometric information is to consider the direct one. Indeed, a conventional camera captures light and not geometric features. However, due to the difference between the natures of photometry (visual acquisition) and geometry (3D space in which the robot moves), the direct information is less trivial to be considered for navigation. Recent progresses in direct photometric-based autonomous robot localization systems opened the way to reach similar practical performance than geometry-based approaches, and even better, since detection of geometric features and matching process are avoided [START_REF] Engel | Direct Sparse Odometry[END_REF]. Furthermore, dense methods are more precise for positioning tasks thanks to the visual feature redundancy [START_REF] Raj | Appearance-based indoor navigation by IBVS using mutual information[END_REF]. Although implicitly, the aforementioned system directly relates two common stages of vision-based navigation: image acquisition and pose estimation (robot attitude and position). Similarly, the system proposed in [START_REF] Caron | Photometric visual servoing for omnidirectional cameras[END_REF] directly relates the stages of visual perception with motion control. This system uses Photometric Visual Servoing (PVS) [START_REF] Collewet | Photometric visual servoing[END_REF], which considers the pixel intensities of the image as visual feature, to follow a visual path.

A. Problem Statement

On the one hand, the process of building a visual path generally starts with the acquisition of images from a first tele-operated trajectory [START_REF] Courbon | Vision-based navigation of unmanned aerial vehicles[END_REF]. This process requires the robot in situ and is time consuming. However, the visual path might also be planned from a preobtained dataset of images [START_REF] Remazeilles | Image-based robot navigation from an image memory[END_REF]. In the latter work, the visual path is represented by a weighted graph where each node is associated with an image. Then, the edges are weighted accordingly to the number of feature points matched between the images. Nevertheless, the dataset of images is required to be acquired in situ and by the same camera to ensure the point matching.

On the other hand, the scene might be completely represented by a model, e.g. a Computer-Aided Design model [START_REF] Gasteratos | PRONTO: a system for mobile robot navigation via CAD-model guidance[END_REF], obtained a priori. Nevertheless, some zones may lack of texture. Moreover, localization is required when navigating.

We propose to solve these problems by generating a visual path K from a preobtained model of the scene. First and foremost, the model is a rectangular mesh M, subdivided into two triangles (orange dotted lines in Fig. 1). The mesh M is textured by aligning its 4 vertices with those of an image of the scene, e.g. a satellite or high altitude aerial image. By using an Unmanned Aerial Vehicle (UAV) as testbed, the scene and the mesh M are considered flat in this paper. Secondly, the visual path K is composed by key images K ∈ K rendered from the mesh M. The generation of the mesh M and the rendering of each key image K are computed offline. Lastly, the autonomous navigation is defined as a concatenation of positioning tasks which goal is to sequentially servo each key image K; called visual path following.

The system proposed in this paper is presented next.

B. Proposed System

The main contribution in this paper is a vision-based navigation system composed by the Photometric-Planner and the Visual Path Follower.

The goal of the Photometric-Planner is to generate a navigable visual path K. Generally speaking, this process begins by rendering virtual images I V from the mesh M. Then a Gaussian filter is applied to these images and a Zero-mean Normalization transformation to their pixel intensities. Finally, some of these I V are selected; thus, they become the elements in the visual path K.

The Visual Path Follower is in charge of carrying out the autonomous navigation. Being a memory-based system, the UAV is controlled by comparing the onboard images I with those stored in the visual path K using a variation of PVS, called Enhanced PVS. Indeed, to servo a desired image I * using an UAV as testbed is a secondary contribution since this image is virtual (so, denoted as I * V ). The diagram in Fig. 1 illustrates the general workflow of the proposed system.

C. Paper Structure

The rest of the paper follows the next structure. Section II compares current works in the state-of-the-art of visionbased navigation with ours and Section III formally presents the novel Enhanced PVS and highlights two advantages over PVS. Then, Section IV details the process of the Photometric-Planner and the criterion used to select the key images K among other virtual images I V . Later, the Visual Path Follower is detailed in Section V as well as the criterion to switch from the current key image K to the next one in the visual path K. The evaluation of the complete system, implemented on the experimental setup, is presented in Section VI. The experimentation is composed by a group of positioning and navigation tasks performed over indoor and outdoor environments using an UAV as testbed. Finally, the paper concludes in Section VII by summarizing the main ideas, the interpretation of the results of the experiments and by offering new perspectives for future works.

II. RELATED WORKS

This Section reviews the works related to the one presented in this paper. The processes of visual perception and motion control used in [START_REF] Caron | Photometric visual servoing for omnidirectional cameras[END_REF] are similar of those used in the proposed system since a direct visual feature is considered for the pathplanning and navigation. In this paper we introduce a novel feature, called Gaussian Zero-mean Normalized (GZN) luminance Ǐ, which is a secondary contribution. GZN luminance Ǐ is a variation of the luminance feature I, being the stacking of every pixel intensities of an image. It is better adapted to compare virtual with onboard images I V and I respectively, and to compute the motion control of the UAV.

In order to accomplish the goal of the Photometric-Planner, the main difficulty is to select the key images K to build the visual path K. As stated in Section I-A, a first example to handle this problem is by maximizing common feature points between successive key frames of the visual path [START_REF] Remazeilles | Image-based robot navigation from an image memory[END_REF]. A second one is by means of the visual odometry and SLAMbased approaches, which minimize (relative) pose uncertainty. These approaches had been implemented for the case of ground robot navigation [START_REF] Otsu | Where to Look? Predictive Perception With Applications to Planetary Exploration[END_REF], [START_REF] Inoue | Active localization for planetary rovers[END_REF] or in the case of a flying robot [START_REF] Sadat | Feature-rich path planning for robust navigation of MAVs with Mono-SLAM[END_REF] along the followed path. The last two works are referred as Planning in Information Space, in the more general frame of active perception. In practice, this approach has the implicit behavior of maximizing the number of feature points acquired along the path, a critical constraint for the success of visual odometry and SLAM. However, the Photometric-Planner addresses the key image selection problem based on the cost C of the Enhanced PVS which directly relates pathplanning with motion control since the same visual feature is used.

The main difficulty of the Visual Path Follower is to compare the key images K with onboard images I using only photometry. This problem had been also solved in the literature [START_REF] Caron | Photometric visual servoing for omnidirectional cameras[END_REF] where a wheeled robot navigates by comparing the onboard images I from an omnidirectional camera with those stored in the visual path K. However, its visual path is obtained from a tele-operated navigation and not from a textured mesh M. Another similar approach is the active perception that extends the pose uncertainty minimization-based planning to a photometric criterion [START_REF] Costante | Exploiting Photometric Information for Planning Under Uncertainty[END_REF]. In that approach the robot pose uncertainty is computed as a function of the photometric information of the scene, called Perception-aware Path Planning. However, the system proposed in this paper does not require to build nor update a model while navigating, it rather represents the environment to navigate by a minimum number of key images K.

III. ENHANCED PHOTOMETRIC VISUAL SERVOING

The luminance feature I, used in PVS, is a dense feature more accurate than a sparse one since it stacks pixel intensities of the entire image I to perform a positioning task. However, it is not robust to light variation [START_REF] Raj | Appearance-based indoor navigation by IBVS using mutual information[END_REF]. Therefore, its implementation to vision-based navigation is restricted to indoor environments only.

Furthermore, the illumination in the textured mesh M and the scene are unlikely to be the same; thus, PVS is not suitable to compare virtual images I V with onboard images I by the Visual Path Follower. Therefore, a Zero-mean Normalization transformation is applied to the image I in order to make the system robust to global affine illumination changes. Additionally, a Gaussian filtering is also considered to prevent undesired behavior when applying the control input u to the UAV. The proposed system uses this slight variation of PVS, called Enhanced PVS.

A. Control Law

The Enhanced PVS considers the GZN luminance feature Ǐ which is obtained from the pixel intensities of the image I after a Gaussian filtering with the following kernel:

h(x, y) = 1 2πσ 2 e -x 2 +y 2 2σ 2 , ( 1
)
where σ is standard deviation, and a Zero-mean Normalization, becoming:

Ǐ(ξ) = ( Ǐ1• , Ǐ2• , . . . , ǏN• ) (2)
where Ǐi• ∈ R 1×M is the i-th line of the GZN image Ǐ, N is the width and M the height of Ǐ.

Then, the GZN photometric error is defined as:

ě(ξ) = Ǐ(ξ) -Ǐ(ξ * ), (3)
where Ǐ(ξ) and Ǐ(ξ * ) are obtained from the current and desired GZN images Ǐ and Ǐ * , respectively, at the current pose ξ = [X , Θ ] ∈ R 6 , where X = [X, Y, Z] ∈ R 3 is its location and Θ = [Θ, Φ, Ψ] ∈ R 3 its orientation, and desired pose ξ * . Analogously to PVS, the goal of Enhanced PVS is to minimize the value of the cost function:

C(ξ) = 1 2 ě(ξ) 2 , (4)
by non-linear optimization. The chosen control law that handles the motion of the robot is based on the Gauss-Newton method:

u = -λ L + Ǐ(ξ * ) ( Ǐ(ξ) -Ǐ(ξ * )), (5)
where

u = ξ = [v , ω ] is the control input composed by linear v = [v x , v y , v z ] and angular ω = [ω x , ω y , ω z ] ve- locities, L + Ǐ(ξ * )
is the Moore-Penrose pseudoinverse of the approximation of the interaction matrix LǏ (ξ * ) = -∇ Ǐ(ξ * ) L x related to Ǐ(ξ * ), LǏ (ξ * ) is of full rank, ∇ Ǐ(ξ * ) is the gradient of Ǐ(ξ * ) and:

L x = -1/Z 0 x/Z xy -(1 + x 2 ) y 0 -1/Z y/Z 1 + y 2 -xy -x , (6)
is the interaction matrix that relates the variations of the image point x = [x, y] to the velocity of the camera, as depicted in [START_REF] Chaumette | Visual servo control. Part I. Basic approaches[END_REF].

Although the Gauss-Newton optimization method is chosen to minimize the cost C, the Visual Path Follower is not restricted to any particular method.

B. Cost Function Analysis

An experimental study shows the main differences between PVS and Enhanced PVS. The study consists into computing the cost functions C(ξ) of PVS and Enhanced PVS by rendering virtual images I V (ξ) over the image plane around the desired pose ξ * . These images are generated by translating ξ along t X and t Y within the intervals of [-30mm, 30mm] in the X and Y axes with a sampling step of 0.5mm. Fig. 2 shows four desired virtual images I *

V used in the experimental study of the cost functions C(ξ). All of these images are rendered from the textured mesh M at the desired pose ξ * . Notice that the global illumination difference is drastically reduced after applying a Zero-mean Normalization transformation to the obscured I * V with respect to its corresponding non-obscured I * V .

Fig. 2: Four virtual images rendered from the textured mesh at the desired pose for the cost functions study.

Two considerations are made for the experimental study:

• All the feature vectors (I, I * , Ǐ and Ǐ * ) are generated from virtual images I V . The experimental study uses virtual images I V since the Photometric-Planner renders I V from the mesh M.

• The desired features (I * and Ǐ * ) are generated from obscured I V or ǏV . This consideration simulates the global illumination difference between the onboard image I and the key image K during the autonomous navigation. The cost functions C(ξ) of PVS, defined as the Sum of Squared Differences (SSD) of pixel intensities, and (4) (GZNSSD) appear in Fig. 3.

Notice that the value at the minimum of GZNSSD cost function C(ξ) reaches zero whereas the corresponding one of SSD does not. Moreover, the shape of the domain of convergence around the global minimum is smoother for GZNSSD which will prevent to compute drastic changes in the input velocity u. Such comparison justifies the choice of Enhanced PVS over PVS for the Photometric-Planner and Visual Path Follower, which are formally introduced in the next two Sections.

IV. PHOTOMETRIC-PLANNER

The goal of the Photometric-Planner is to generate a navigable visual path K that allows the Visual Path Follower to reach the final pose ξ f from the start pose ξ s . This path contains key images K = I V (ξ k ) ∈ K which are rendered from the textured mesh M, using any graphics engine, at key poses ξ k . Although several virtual images I V are rendered from the mesh M, just few of them are selected as key images K.

The Photometric-Planner begins by defining the start ξ s and final pose ξ f in the textured mesh M. For the sake of simplicity, ξ s and ξ f are considered to have the same orientation Θ. Then, the straight path from ξ f to ξ s is regularly sampled in 3D translation steps ∆X, into intermediate poses ξ i . Indeed, the same orientation Θ is preserved in ξ i since ∆X does not involve rotation. Later, ξ f is considered as the first intermediate pose ξ i and directly selected as the first key pose ξ k . Thus, its virtual image I V (ξ f ) = I V (ξ k ) is stored in the visual path K as the first key image K. Analogously, the virtual image rendered from the textured mesh M at the start pose I V (ξ s ) is considered as the last K in the visual path K. For the rest of the intermediate poses ξ i in the path from ξ f to ξ s , the selection process is carried out as follows. Let ξ i+1 = ξ i + ∆X be the next intermediate pose and ξ k the last key pose selected by the Photometric-Planner. Then, ξ i is considered as the next key pose ξ k+1 , if the cost increase ∆C i+1 falls below a threshold s :

∆C i+1 < s , (7)
where

∆C i+1 = C i+1 -C i , C i = 1 2 Ǐ(ξ i ) -Ǐ(ξ k ) 2 , C i+1 = 1 2 Ǐ(ξ i+1 ) -Ǐ(ξ k ) 2
and the GZN luminance features, Ǐ(ξ i ) and Ǐ(ξ i+1 ), are obtained from ǏV (ξ i ) and ǏV (ξ i+1 ), respectively (Section III). Generally speaking, such selection criterion does not depend on the image content but it is rather related to the control law convergence.

V. VISUAL PATH FOLLOWING

The process of visual path following (Section I-A) is carried out by the Visual Path Follower, which computes the control input u = ξ from:

u = -λ L + Ǐ(ξ k ) ( Ǐ(ξ) -Ǐ(ξ k )), (8)
where Ǐ(ξ k ) is obtained from the key image K = I V (ξ k ) in the visual path K by transforming I V into ǏV . When the current key image K is considered reached during the visual path following, the next key image is set as the current one. This process is called key image update and it occurs when the value of the cost C (4) satisfies:

Ǐ(ξ) -Ǐ(ξ k ) 2 < u , where u is a threshold. The complete process of visual path following is displayed in Fig. 4.

VI. EXPERIMENTATION

The performances of the Photometric-Planner and Visual Path Follower are evaluated by navigation tasks. Moreover, since comparing virtual images I V with onboard images I in a visual servoing scheme using an UAV is a novelty, the performance of the Enhanced PVS is evaluated by positioning tasks. Both types of tasks are carried out indoor and outdoor using the experimental setup which is composed by: an HP ZBook 15 computer as ground station, a Parrot Bebop 2 as testbed UAV, the Optitrack motion capture system (indoor) to track the pose ξ of the UAV, a printed tarpaulin (indoor), a Nikon camera to record external video and three textured meshes M. In all the indoor tasks, the same satellite image of 2952 × 3052 pixels is printed on the 4.8 m × 4.9 m tarpaulin and used to generate the textured mesh M. This image is provided by Google Earth [START_REF] Gorelick | Google Earth Engine: Planetary-scale geospatial analysis for everyone[END_REF] and displays a neighborhood of 1146 m × 1170 m Amiens, France.

The Bebop 2 is connected to the computer, that runs Ubuntu 16.04, through Wi-Fi using the bebop autonomy package 1 developed for the Robot Operating System [START_REF] Quigley | ROS: an open-source Robot Operating System[END_REF]. The Photometric-Planner runs offline on the computer using MATLAB and the corresponding textured mesh M for each navigation task. Indeed, the positioning tasks are an instantiation of the navigation tasks where the visual path K contains only one key image K. The size of all the images is 240×428 pixels. The Visual Path Follower runs in the computer and sends the control input u to the UAV. Generally speaking, the onboard images I are acquired by the camera of the Bebop 2 and sent to the ZBook that computes the control input u = ξ to be applied to the robot. However, the Bebop 2 does not obey velocities in the X and Y -axes (so, v x and v y ) but angular positions around the Y and X-axes (so, Φ and Θ). Fig. 5. displays the robot and world reference frames R F and W F, respectively. Therefore, a proportional controller is proposed to adapt the output of the control law [START_REF] Collewet | Photometric visual servoing[END_REF] to the input expected by the Bebop 2. Such controller was computed from a velocity characterization of second order and using the dynamic model of the quadrotor [START_REF] Luukkonen | Modelling and control of quadcopter[END_REF] that considers air resistance:

W v x (t) W v y (t) = g m c 0 0 0 c 1 tan( W Φ)(1 -e -t c 0 m ) tan( W Θ)(1 -e -t c 1 m ) , (9)
where W v x and W v y are the linear velocities in the XY plane, W Φ is the pitch and W Θ is the roll in the world reference frame W F, the acceleration caused by the gravity is g = 9.8 m•s -2 , the coefficients of the air in both axes are c 0 = c 1 = 0.35, t ∈ R is the time's variable, and the mass of the quadrotor m = 0.497 kg. Nevertheless, such controller is only valid at low speeds. A Proportional-Integral-Derivative controller [START_REF] Salih | Flight PID controller design for a UAV quadrotor[END_REF] would be preferred over a Proportional controller. However, it was not possible to design since the readings of the Inertial Measurement Unit (IMU) are retrieved at a lower rate (5 Hz) than the control input (30 Hz) at servoing mode. This issue is related with the update of the Robot Operating System topic that contains the velocity of the Bebop 2 rather than a poor performance of the IMU. This is the main reason to consider the Gaussian filtering in the visual feature.

Furthermore, an image latency, i.e. the time elapsed between an event in the real world and its corresponding change in 1 https://bebop-autonomy.readthedocs.io/ the image on the laptop side, was measured with a mean of 183.9 ms and a standard deviation of 11.26 ms. The latency was measured using a mbed 2 microcontroller that triggers a LED and starts a chronometer at the same time. Then, the chronometer stops when a dark pixel, that corresponds to the center of the LED, increments its intensity up to 200 (with a maximum level of 255). In order to mitigate the impact of the latency in the experiments, the robot switches from servoing mode to hovering mode, i.e. to apply a null control input u = 0, during 20 iterations at 30 Hz when a key image K is reached according to the key image update (Section V). Lastly, the tasks are considered failed when the value of the cost C is larger than a threshold f . The thresholds for the navigation tasks are u = 1.5 × 10 5 for the indoor experiment, u = 5.5×10 4 for the outdoor experiment and f = 2.75×10 5 for both. All the thresholds are experimentally determined.

The rest of the Section covers the positioning and navigation tasks.

A. Positioning Tasks

Three positioning tasks evaluate the performance of the Enhanced PVS (Section III). The goal of the robot is to servo a desired virtual image I * V , using (8) and real images I acquired with its onboard camera.

1) Indoor Nominal Positioning: This experiment consists in servoing a desired virtual image I * V (Fig. 6) without any external perturbation and at constant illumination. It is important to notice that the native hovering mode drifts and the robot eventually diverges since it is based on visual odometry, which is an open-loop system [START_REF] Zhang | Visual-lidar odometry and mapping: Lowdrift, robust, and fast[END_REF]. While applying the control law (8), the robot minimizes the cost C (4). Then, the experiment finished when the memory that stores the data of the experiment was filled up in about 5 minutes. The evolution of the cost C and control input u are displayed in Fig. 7.

2) Indoor Perturbed Positioning: This experiment is similar to the previous one except that a fan is actuated in oscillation mode 2 m behind the robot to cause external perturbation. If the native hovering mode is permanently active, the robot diverges in 90 s. However, the Enhanced PVS task lasted 163 s when a gust of wind pushed the robot strong enough to diverge. 3) Outdoor Positioning: This experiment took place on a crossroad (Fig. 8a) during the day but with presence of clouds. The desired virtual image I * V (Fig. 6) is also generated from the same textured mesh M as the previous two experiments. The robot stayed in place, at 10 m above the crossroad, without any important perturbation for the entire task with a duration of 240 s. Indeed, the robot hovered for the most part of the experiment except at the beginning, when servoing for the first time, and at t = 100 s, when a pedestrian caused an increase in the value of the cost C and triggered the servoing mode.

This group of experiments demonstrated that the servoing is performed at scale (concerning image variation and camera displacement) even if Z is not determined from this scheme. Therefore, perturbations that occur at lower altitude had a greater impact in these tasks since they triggered the servoing mode several times whereas perturbations in the image at higher altitude had less impact. The perturbation caused by the pedestrian showed that the Enhanced PVS is robust to partial occlusions. This experiment was conducted with no Global Navigation Satellite System information for guidance and navigation from ground station. Therefore, the quantitative evaluation of this experiment validates the interest of not relying only on Global Navigation Satellite System for robot positioning and navigation.

B. Navigation Tasks

The performance of the Photometric-Planner and Visual Path Follower is evaluated by two navigation tasks. Although any set of straight paths could be considered, e.g. an L-path, the motion capture system field of measure constrains the indoor experiment on a single straight path. A straight path is considered outdoor too for consistency. Therefore, to make the path following harder, these tasks are defined as a round trip on a straight line in order to validate the visual path K in forward and backward directions even if the latter is generated only in backward direction. A video3 is provided to present the navigation tasks. 1) Indoor Navigation: In this experiment (Fig. 9), the UAV is intended to travel a straight path of 55 cm. The Photometric-Planner generated a visual path K that contains 11 key images K (Fig. 10). Before diverging, the robot traveled approx. 2.7 m reaching 54 K: {K 0 , K 1 , . . . , K 10 }, {K 10 , K 9 , . . . , K 0 }, {K 0 , K 1 , . . . , K 10 }, {K 10 , K 9 , . . . , K 0 }, and {K 0 , K 1 , . . . , K 9 }. Fig. 11 shows the trajectory of the last round trip and Fig. 12 displays the cost C and the control input u = ξ evolution of this experiment. Fig. 13 displays an excerpt of the images of difference between the onboard image I and the key image K.

2) Outdoor Navigation: This experiment took place in the car park of the University of Picardie Jules Verne Campus, Amiens, France (Fig. 8b). The textured mesh M (Fig. 14) is obtained from a snapshot that the UAV took when flying at 30 m above the ground. However, the visual path K is generated so the robot navigates at a height of 13 m. The distance between the start and final poses ξ s and ξ f , respectively, is set to 8 m. The Photometric-Planner generated a visual path K composed by 5 key images K. Then, the robot successfully traveled the round trip, i.e. 16 m, using the visual path K but it diverged when it tried to restart from the first key image K. Fig. 16 shows an excerpt of the images related to this experiment and Fig. 15 displays the evolution of the cost C and the control input u = ξ from this task.

From this group of experiments, the proposed system completed the navigation tasks in round trip but they diverged when an important perturbation in the image or in the robot occurred.

VII. CONCLUSIONS

This paper presented a memory-based navigation system that generates its visual path from a textured mesh rather than from a previous taught path. Therefore, the visual pathplanning process is carried out ex situ and does not require the robot if the textured mesh was obtained a priori. Moreover, such mesh can be instantly generated from a snapshot model [START_REF] Salichs | Navigation of mobile robots: open questions[END_REF] as in the last experiment.

The Photometric-Planner generated navigable visual paths validated in both directions and the Visual Path Follower controlled an UAV through a total distance of 16 m outdoor using only its camera to navigate. In such experiment, navigation based on the Global Navigation Satellite System could not be possible since the signal of the latter was unavailable. Additionally, the Enhanced PVS proved to be able to compare virtual and onboard images while servoing a Fig. 16: Excerpt of some images from the outdoor navigation. Desired virtual images appear at the top row, onboard images appear at the central row and images of difference appear at the bottom row. commercial UAV. Nevertheless, the Photometric-Planner only validates the navigability of a visual path defined in straight line and does not explore other visual paths. Future works will consider a topological approach to generate paths not strictly straight. Additionally, they will consider another controller, e.g. Model Predictive Control [START_REF] Mercorelli | A model predictive control in robotino and its implementation using ros system[END_REF], or UAV so the control input be correctly applied and the Visual Path Follower be evaluated in more challenging situations. As a perspective for indoor navigation, the scene model might include obstacles mapped from a group of robots [START_REF] Sergiyenko | Data transferring model determination in robotic group[END_REF] prior to the process of the Photometric-Planner.

Fig. 1 :

 1 Fig. 1: Proposed vision-based navigation system. Once the start and final poses are defined over the mesh (top left), the Photometric-Planner generates the visual path (top right) to be used for the Visual Path Follower.

Fig. 3 :

 3 Fig. 3: Comparison of SSD (top) and GZNSSD (bottom) cost functions. Distances are shown in millimeters

Fig. 4 :

 4 Fig. 4: Visual Path Follower

Fig. 5 :

 5 Fig. 5: Reference frames of the robot and the world.

Fig. 6 :

 6 Fig. 6: Desired virtual images used for indoor nominal and perturbed positioning (left) and outdoor positioning (right).

Fig. 7 :

 7 Fig. 7: Cost and control input evolution from the indoor nominal positioning. Vertical lines indicate the switch from hovering mode to servoing mode.

  (a) Positioning. (b) Navigation.

Fig. 8 :

 8 Fig. 8: Outdoor experiments.

Fig. 9 :

 9 Fig. 9: Indoor navigation experiment. The arrows indicate the trajectory done by the robot at the beginning of the second round trip.

Fig. 10 :

 10 Fig. 10: Visual path of the indoor navigation experiment.

Fig. 11 :

 11 Fig. 11: Last round trip (cyan) of the indoor navigation. Green dots indicate the locations of the key poses over the straight path (pink) and red dots where the system failed. Distances appear in meters.

Fig. 12 :

 12 Fig. 12: Cost and control input evolution from the indoor navigation. Vertical lines indicate the key image update and red line when the system failed.

Fig. 13 :

 13 Fig. 13: Excerpt of four images of difference of the indoor navigation experiment.

Fig. 14 :

 14 Fig. 14: Textured mesh used for the outdoor navigation.

Fig. 15 :

 15 Fig. 15: Cost and control input evolution from the outdoor navigation. Vertical lines indicate the key image update.

Mbed website https://os.mbed.com/

http://mis.u-picardie.fr/ ˜g-caron/videos/2020_ IEEESensors.mp4

ACKNOWLEDGEMENTS

This research was partially supported by the National Council of Science and Technology (CONACYT) of Mexico.

This project was supported partially by the Mexican Council of Science and Technology, CONACYT-MEXICO.


Wajdi Ghezaiel
email: wajdi.ghezaiel@ensicaen.fr

Luc Brun
email: luc.brun@ensicaen.fr

Olivier L Ézoray
email: olivier.lezoray@unicaen.fr

Wavelet Scattering Transform and CNN for Closed Set Speaker Identification

Keywords: Speaker identification, short utterances, wavelet scattering transform, convolutional neural network, hybrid network

In real world applications, the performances of speaker identification systems degrade due to the reduction of both the amount and the quality of speech utterance. For that particular purpose, we propose a speaker identification system where short utterances with few training examples are used for person identification. Therefore, only a very small amount of data involving a sentence of 2-4 seconds is used. To achieve this, we propose a novel raw waveform end-to-end convolutional neural network (CNN) for text-independent speaker identification. We use wavelet scattering transform as a fixed initialization of the first layers of a CNN network, and learn the remaining layers in a supervised manner. The conducted experiments show that our hybrid architecture combining wavelet scattering transform and CNN can successfully perform efficient feature extraction for a speaker identification, even with a small number of short duration training samples.

I. INTRODUCTION

Smart home speakers have recently become very popular and allow the voice-based launching of applications. This can be problematic for security reasons, especially for applications that enable online purchase. In such cases, the speaker has to be identified to ensure a secure transaction. To enable this, the speaker has first to be enrolled. Such an enrollment is usually performed from few short sentences. In such a practical scenario there may be no constraints on such sentences. Moreover, these sentences are usually short and only few utterances are provided to the system. Different studies [START_REF] Kanagasundaram | i-vector based speaker recognition on short utterances[END_REF], [START_REF] Poddar | Speaker verification with short utterances: a review of challenges, trends and opportunities[END_REF] have shown that the use of short segments may induce a drastic drop of the performances of speaker identification systems. This drop in performance is mainly due to the low amount of information extracted for each speaker. Speaker identification with only few and short utterances is thus a challenging problem. Moreover, this work takes place within the framework of the HomeKeeper 1 project which aims at providing smart home speakers dedicated to regional needs This work was supported by BPI France, project HomeKeeper. 1 https://home-keeper.io/ (healthcare, local radios,. . . ). In order to avoid the need of an external identification server, the consortium of the project has decided to perform the identification step directly on the terminal. This means that each terminal has at its disposal only a closet set reduced database of short sentences corresponding typically to the local members of a family (less than 10 persons).

Traditionally, speaker identification systems are based on the extraction of features relying on speech production and perception such as Mel-Frequency Cepstral Coefficients (MFCCs). During the training phase, such features are extracted from a large amount of speakers. A Gaussian Mixture model (GMM) is then trained to build a Universal Background Model (UBM) [START_REF] Reynolds | Speaker Verification Using Adapted Gaussian Mixture Models[END_REF]. During the enrollment phase of a speaker, the mean of the GMM is adapted to fit the speaker's data. In GMM-UBM systems, the stacked mean vectors are directly used as the representation of the speaker. However, it has been shown [START_REF] Dehak | Frontend factor analysis for speaker verification[END_REF], [START_REF] Li | An improved i-vector extraction for speaker verification[END_REF] that it is beneficial to further process this vector by extracting intermediate vectors called i-vectors. During the identification phase, an i-vector is extracted from a given speech sample and is compared to the reference i-vector, either with a simple cosine distance or with more complex techniques such as Probabilistic Linear Discriminant Analysis (PLDA) [START_REF] Prince | Probabilistic linear discriminant analysis for inferences about identity[END_REF].

In recent years, deep learning has appeared in many pattern recognition fields. It has shown remarkable success in many fields such as image recognition [START_REF] Krizhevsky | Imagenet classification with deep convolutional neural networks[END_REF] and natural language processing [START_REF] Collobert | A unified architecture for natural language processing: Deep neural networks with multitask learning[END_REF]. In speaker identification, a similar trend has been observed. Deep Neural Networks (DNNs) have been used with the i-vector framework to compute Baum-Welch statistics [START_REF] Kenny | Deep neural networks for extracting baum-welch statistics for speaker recognition[END_REF], or for frame-level feature extraction [START_REF] Yaman | Bottleneck features for speaker recognition[END_REF]. DNNs have also been proposed for direct discriminative speaker classification, as witnessed by the recent literature on this topic [START_REF] Variani | Deep neural networks for small foot-print text-dependent speaker verification[END_REF], [START_REF] Snyder | X-vectors: Robust dnn embeddings for speaker recognition[END_REF]. Lately, there was an increasing number of studies using convolutional neural networks (CNNs) [START_REF] Lecun | Convolutional networks for images, speech, and time series[END_REF] in numerous speech tasks [START_REF] Abdel-Hamid | Convolutional neural networks for speech recognition[END_REF]. Some works have proposed to directly feed networks with spectrogram bins [START_REF] Zhang | Text-independent speaker verification based on triplet convolutional neural network embedding[END_REF] or even with raw waveforms [START_REF] Nagrani | Voxceleb:a large-scale speaker identification dataset[END_REF], [START_REF] Muckenhirn | To-wards directly modeling raw speech signal for speaker verification using CNNs[END_REF]. Among DNNs, CNNs have the most suitable architecture for processing raw speech samples, since weight sharing, local filtering, and pooling constitute precious tools to discover robust and invariant representations.

978-1-7281-9320-5/20/$31.00 ©2020 IEEE However, CNN networks require numerous labeled training examples along with considerable computational resources and time to achieve effective learning. In a setting where only few labeled data of short duration are available, the training becomes difficult and requires a lot of regularization.

Feature extraction and representation is a critical point of classification systems which, if correctly handled, can allow to reduce the size of labeled training datasets. The wavelet scattering transform, a rich representation, has enjoyed significant success in various audio [START_REF] Andén | Deep scattering spectrum[END_REF] and biomedical [START_REF] Chudáček | Scattering transform for intrapartum fetal heart rate variability fractal analysis: A case-control study[END_REF] signal classification tasks. Its structure is that of a convolutional neural network [START_REF] Andén | Deep scattering spectrum[END_REF], but with fixed filters. This last point is important when only few training samples are available. Specifically, this transformation alternates convolutions with wavelet filters and pointwise nonlinearities to ensure time-shift invariance and time-warping stability [START_REF] Mallat | Group invariant scattering[END_REF]. Scattering representations can be plugged into any classification or regression system, be it shallow or deep. The original experiments of Andén and Mallat [START_REF] Andén | Deep scattering spectrum[END_REF] on deep scattering spectrum relied on support vector machines (SVM) with linear or Gaussian kernels. For supervised large-vocabulary continuous speech recognition, replacing these locally linear classifiers by five layers of deep neural networks (DNN) or deep convolutional networks (ConvNets) only brought marginal improvements in accuracy [START_REF] Peddinti | Deep scattering spectrum with deep neural network[END_REF]. However, in the Zero Resource Speech Challenge [START_REF] Versteegh | Deep scattering spectrum with deep neural networks[END_REF], whose aim is to discover sub-word and word units from continuous speech in an unsupervised way, associating scattering representations with deep siamese network provided a substantial gain in the trade off between inter-class discriminability and inter-speaker robustness [START_REF] Zeghidour | A deep scattering spectrum-Deep Siamese network pipeline for unsupervised acoustic modeling[END_REF]. Wavelet scattering transform features demonstrated promising results on the dataset TIMIT for phonetic classification [START_REF] Andén | Deep scattering spectrum[END_REF] and recognition [START_REF] Peddinti | Deep scattering spectrum with deep neural network[END_REF]. In this paper, we explore the use of the Wavelet Scattering Transform (WST) for feature extraction along with convolutional neural network for closed set speaker identification. In this hybrid deep learning network, wavelet scattering coefficients generated in first few layers capture the dominant energy contained in the input data patterns.

The reminder of the paper is organized as follows. Section II discusses wavelet scattering transform. Section III describes the proposed hybrid architecture, which is a cascade of a wavelet scattering transform and a convolutional neural network. Section IV discusses the experimental setup and the corresponding results obtained by the proposed system as well as the ones provided by related systems.

II. WAVELET SCATTERING TRANSFORM

The wavelet scattering transform (WST), introduced in [START_REF] Andén | Deep scattering spectrum[END_REF], [START_REF] Mallat | Group invariant scattering[END_REF], is a deep representation, obtained by iterative application of the wavelet transform modulus. It has been defined so as to be invariant to translations of the input signal, and stable to small deformations. The authors of [START_REF] Andén | Deep scattering spectrum[END_REF], [START_REF] Mallat | Group invariant scattering[END_REF] have demonstrated how the WST transform can extract significant features at different scales. WST has been successfully applied to different classification tasks, textures [START_REF] Bruna | Invariant scattering convolution networks[END_REF], [START_REF] Bruna | Learning stable group invariant representations with convolutional networks[END_REF], small digits [START_REF] Bruna | Invariant scattering convolution networks[END_REF], sounds [START_REF] Andén | Deep scattering spectrum[END_REF] or complex image datasets with unsupervised representations [START_REF] Chudáček | Scattering transform for intrapartum fetal heart rate variability fractal analysis: A case-control study[END_REF]. Moreover, it has been proven [START_REF] Oyallon | Scaling the Scattering Transform: Deep Hybrid Networks[END_REF] that the WST coefficients are more informative than a Fourier transform when dealing with short variation signals or small deformation and rotation. WST consists in Convolution (Wavelet ψ(t))

x ψ(t) Let a wavelet ψ(t) be a band pass filter with a central frequency normalized to 1, and ψ λ (t) a wavelet filter bank, which is constructed by dilating the wavelet:

Nonlinearity (Modulus) |x ψ(t)| Averaging (low-pass filter) |x ψ(t)| φ(t)
ψ λ (t) = λψ(λt) (1)
where λ = 2 j Q , ∀j ∈ Z and Q is the number of wavelets per octave. The bandwidth of the wavelet ψ(t) is of the order 1 Q , and as a result, the filter bank is composed of band pass filters which are centered in the frequency domain in λ and have a frequency bandwidth λ Q . At the zero order, we have a single coefficient given by S 0 x(t) = x φ(t), where is the convolution operator. This coefficient is close to zero for speech signals. At the first order, we set Q 1 = 8 for speech signals, which defines wavelets having the same frequency resolution as mel-frequency filters. Approximate mel-frequency spectral coefficients are obtained by averaging the wavelet modulus coefficients with φ:

S 1 x(t, λ 1 ) = |x ψ λ1 | φ(t) (2)
The second order coefficients capture the high-frequency amplitude modulations occurring at each frequency band of the first layer and are obtained by:

S 2 x(t, λ 1 , λ 2 ) = ||x ψ λ1 | ψ λ2 | φ(t) (3)
The wavelets ψ λ2 have an octave resolution Q 2 which may be different from Q 1 . We set Q 2 = 1 for speech signals, to defines wavelets with more narrow time support, which are better adapted to characterize transients and attacks. We get a sparse representation which means concentrating the signal information over as few wavelet coefficients as possible. These coefficients are averaged by the low pass filter φ, which ensures local invariance to time-shifts, as with the first-order coefficients. Fig. 2 shows the hierarchy of wavelet scattering coefficients. This somewhat resembles to the structure of deep neural networks, although in the WST, each layer provides some output, while the only output of most of deep neural networks is provided by the last layer. This decomposition on first and second orders scattering coefficients, is applied to the time domain signal. Features of the second order are normalized by features of the first order, just to ensure that the higher order of scattering depends on the amplitude modulation component of the speech signal. The first and the second orders of the WST are concatenated to form a scattering feature vector for a given frame. The scattering features include log-mel features together with higher order features to preserve greater detail in the speech signal [START_REF] Andén | Deep scattering spectrum[END_REF]. This representation is invariant to time shifts and is stable to deformations. To ensure invariability to frequency translation on a logarithmic scale like translation of speaker formants, the logarithm is applied to each coefficients of the scattering feature vector. It is thus locally translation invariant in time and log frequency, and stable to time and frequency deformations.

III. HYBRID NETWORK ARCHITECTURE

The proposed hybrid network is composed of a WST for feature extraction and a convolution neural network for classification in the back end. We thus propose to initialize the first layer of our CNN with WST feature maps. This map of features shows n f feature vectors with n s coefficients of scattering. Each feature vector is defined as a log of normalized scattering feature vectors (section II) for each frame. It acts as a subsampled feature map for the first CNN layer. We use Gabor and Morlet wavelet to obtain the scattering features map. As stated in section II we set the quality factor to 8 filters per octave for the first level and set 1 filter per octave at the second level. This configuration was chosen to match the frequency resolution of Mel filters at the first level [START_REF] Andén | Deep scattering spectrum[END_REF]. The second order of the scattering transform recovers the lost information. Therefore, the representation of speech signals using the first and the second orders of the scattering transform extends the MFCC representation and doesn't loose information. These scattering coefficients are computed using 50% overlapping windows using a publicly available toolbox [START_REF] Andén | Deep scattering spectrum[END_REF]. The size of final scattering feature maps is n s × n f . The proposed CNN is composed with three convolution blocks and one fully connected layer. Each convolution block is formed by 1D convolution layer of length 3 and batch normalization. Each convolutional layer is followed by a maxpooling layer, with pooling size 1 × 2 and stride 1 × 2.

The network has 16, 32 and 64 filters, respectively. A fully connected layer with n sp hidden neurons, where n sp is the number of speakers to be identified, is connected to categorical softmax layer. We use rectified linear units as activation functions in all layers. This architecture takes raw speech with fixed length, to produce speaker embedding at framelevel. The proposed architecture is shown in Figure 3. Details such as number of filters and kernel sizes are summarized in Table I. The amount of parameters in this neural network is 18,1 millions.

IV. EXPERIMENTS

This section describes the experiments and the results obtained with our approach and related systems.

A. Dataset and experimental setting

Two datasets are used in the experiments, TIMIT [START_REF] Lamel | Speech Database Development: Design and Analysis of the Acoustic-Phonetic Corpus[END_REF] and LibriSpeech [START_REF] Panayotov | Librispeech: An ASR corpus based on public domain audio books[END_REF]. The TIMIT dataset contains studio quality recordings of 630 speakers (192 female, 438 male), sampled at 16 kHz, covering the eight major dialects of American English. Each speaker reads ten phonetically rich sentences. We consider only 462 speakers from TIMIT. We use only 8 sentences for each speaker, the "SX" (5 sentences) and the "SI" (3 sentences). The "SX" sentences are used to train the system, while the "SI" sentences are used to test. The average duration of "SX" sentences is about 4s and test sentences "SI" duration is about of 2-6 seconds. The LibriSpeech database consists in audio books read-out-loud by 2484 speakers, 1283 male and 1201 female volunteers who recorded their voices spontaneously. The speech signal is usually clean, but the recording device and channel conditions vary a lot between different utterances and speakers. 7 utterances have been randomly selected to exploit 12-15 seconds of training for each speaker, and 3 utterances lasting from 5 to 12 seconds as a fixed test set for evaluation. Experiments are only conducted on the short length conditions: our system uses raw waveforms of length of 2 and 4 seconds as input for training and testing phases. Speech utterances having initial duration upper to 4s are splitted on small speech chunk of 2s or 4s. Then utterances having duration lower than 2s or 4s are augmented with speech chunks of same speaker to reach desired duration. To validate the effectiveness of our model, we built 8 kHz and 16 kHz versions of our system. The Timit and Librispeech datasets are thus downsampled to 8 kHz. We did not apply any pre-processing to the raw waveforms, such as pre-emphasis, silence removal, detection and removal of unvoiced speech. Non speech intervals at the beginning and end of each sentence were conserved. Scattering transform was computed to the depth of 2. The first layer contained 8 Gabor wavelets per octave and the second had one Morlet wavelet per octave. The averaging window was set to 500ms of length. Later, coefficients are normalized and log-transformed. The number of the coefficients of a single frame for this setting is 77 and 356 for the first and the seconds orders of the scattering transform respectively. Stochastic gradient descent was used as an optimizer with a learning rate of 0.001 and 0.9 momentum. The network is trained with mini batches of 64 for 30 epochs. Our implementation is based on Scatnet [START_REF] Andén | Deep scattering spectrum[END_REF] and deep learning Matlab toolboxes.

B. Related systems

In this paper, we compare our system with SincNet [START_REF] Ravanelli | Speaker Recognition from raw waveform with SincNet[END_REF], CNN-Raw [START_REF] Muckenhirn | On Learning Vocal Tract System Related Speaker Discriminative Information from Raw Signal Using CNNs[END_REF] and deep system based on hand-crafted features for speaker identification. SincNet is a novel end-toend neural network architecture, that directly receives raw waveforms as inputs. The first 1D convolutional layer of SincNet is composed by Sinc functions. SincNet convolves the waveform with a set of parametric sinc functions that implement band-pass filters. The filters are initialized using the Mel-frequency filter bank and their low and high cutoff frequencies are adapted with standard back-propagation as any other layer. The first layer performs Sinc based convolutions, using 80 filters of length 251. The remaining two layers use 60 filters of length 5. Next, three fully-connected layers composed of 2048 neurons and normalized with batch normalization are applied. All hidden layers use leaky-ReLU non-linearity. Frame-level binary classification is performed by applying a softmax classifier and cross-entropy criteria [START_REF] Ravanelli | Speaker Recognition from raw waveform with SincNet[END_REF]. The number of parameters in Sincnet is about 26.5 millions.

In the CNN-Raw system, the raw waveform is fed directly to the first layer. This network sets on the same architecture as SincNet, but the sinc-based convolution layer is remplaced with standard convolution layer. Three convolution layers are used to perform feature mapping. Each convolution layer is composed of 80 filters followed by a max pooling. Next, three fully-connected layers composed of 2048 neurons and normalized with batch normalization are applied. All hidden layers use leaky-ReLU non-linearities. Frame-level binary classification is performed by applying a softmax classifier and cross-entropy criteria [START_REF] Muckenhirn | On Learning Vocal Tract System Related Speaker Discriminative Information from Raw Signal Using CNNs[END_REF]. The number of parameters in CNN-raw is about 27.6 millions.

The first row of Table II summarizes the number of learning parameters of all tested methods. We observe that the number of learning parameters required by our method is lower than the ones of SincNet and CNN-Raw by about 33%. The second raw of Table II shows the number of epoch required by each method on the Librispeech dataset. Our architecture requires only 30 epochs for training while both SincNet and CNN-Raw networks require 2900 epochs.

A comparison with popular hand-crafted features was also performed. To this end, we computed 39 MFCCs and 40 FBANKs. These features, were computed every 25 ms with 10 ms overlap. FBANK features were fed to the same CNN architecture used in our system (Section III), while a Multi-Layer Perceptron (MLP) was used for MFCCs.

C. Results

In order to evaluate our proposed speaker identification system we use the identification accuracy rate which is equal to the number of correct identification over the number of tests. Accuracy are only computed per frame for all tested methods. In Table III, we report the effect of sampling frequency on In order to compare our system with related ones, speech utterances of either 2s or 4s of duration are used for training and testing without any pre-processing for all systems. Results in Table IV, show that our hybrid network outperforms other systems on Librispeech dataset. For speech utterance with 4s duration, our system achieves a relative improvement of about 20% over CNN-raw and 17% over SincNet. Similarly, with speech utterance of 2s our system provides a relative improvement of about 20% over CNN-raw and 13% over SincNet. Also our system with WST outperforms classical hand-crafted features for both speech utterances of 4s and 2s duration. Results on the original SincNet and CNN-Raw systems [START_REF] Ravanelli | Speaker Recognition from raw waveform with SincNet[END_REF] were conducted on Librispeech with 12-15 seconds of training utterances for each speaker, and 3 utterances lasting from 5 to 16 seconds for evaluation. It is also important to note that our proposed method does not use any pre-processing such as voice or silence detection as this is performed in [START_REF] Ravanelli | Speaker Recognition from raw waveform with SincNet[END_REF]. Adding such pre-processings could possibly further enhance our results.

We report in Table V the effect of training utterances duration per speaker on performances. We split the training data to obtain a total duration of 8s or 12s per speaker. Full train duration is about 14s. In this experiment, the duration of training and testing utterances are set equal. Hence, a duration of 2s for testing and training with a total of 8s for training induces the use of 4 samples per speaker for training. As shown in Table V, varying the number of samples per speaker and thus the total duration for training induces a variation of only 2% of the accuracy. On the other hand, using 4s duration instead of 2s induces an increase of the accuracy of about 10%. Our system is thus able to construct discriminating speakers models with few number of training data but provides better results with test and train samples of at least 4s. V. CONCLUSION

In this paper, we proposed a speaker identification system that learns speaker discriminating information directly from short raw speech signals using wavelet scattering transform and CNNs. We have shown the effectiveness of this hybrid architecture for speaker identification under difficult conditions where we use both short utterances for testing and a low number of samples (of the same duration) for training. Experiments on Librispeech and Timit databases have shown that our hybrid method yields significant improvements over SincNet, CNN-Raw and classical feature combined with deep learning methods on the Librispeech database. The wavelet scattering transform has reduced instabilities on the CNN first layers. The proposed hybrid architecture has the ability to reduce the required depth and spatial dimension of the deep learning networks. In future works, we plan to extend this work in order to consider variable length speech utterance.

Fig. 1 .

 1 Fig. 1. Wavelet scattering transform processes, where x is the input data, ψ a wavelet function and φ an averaging low-pass filter.a cascade of wavelet transforms and modulus nonlinearities. To produce a wavelet scattering transform of an input signal x, three successive operations are required: convolution, nonlinearity, and averaging as described in Figure1. The WST coefficients are obtained with the averaging of wavelet modulus coefficients by a low-pass filter φ.

Fig. 2 .

 2 Fig. 2. Hierarchical representation of wavelet scattering coefficients at multiple layers [18].

Scat feature n s ×n f 1 × 3 Fig. 3 .

 133 Fig. 3. Proposed Hybrid Network.

TABLE I SCATTCNN

 I ARCHITECTURE. EACH ROW SPECIFIES THE # OF CONVOLUTIONAL FILTERS, THEIR SIZES, AND THE # FILTERS. THIS ARCHITECTURE HAS 18,1 MILLIONS PARAMETERS FOR SCATTERING FEATURE MAP OF SIZE 433 × 16 × 1.

	Layer name	ScatCNN	Output
	Input	inputlayer	ns × n f × 1
	Conv1 block	conv1D, 1 × 3, 16	ns × n f × 16
		bn
		relu
	Pooling	maxpool, 1 × 2, stride	ns × n f /2 × 16
		(1,2)
	Conv2 block	conv1D, 1 × 3, 32	ns × n f /2 × 32
		bn
		relu
	Pooling	maxpool, 1 × 2, stride	ns × n f /4 × 32
		(1,2)
	Conv3 block	conv1D, 1 × 3, 64	ns × n f /4 × 64
		bn
		relu
	Pooling	maxpool, 1 × 2, stride	ns × n f /8 × 64
		(1,2)
	Embedding	fc,nsp	nsp
	Loss	softmax	--

TABLE V IDENTIFICATION

 V ACCURACY RATE (%) OF THE PROPOSED SPEAKER IDENTIFICATION ON LIBRISPEECH 16KHZ DATASET TRAINED AND TESTED WITH UTTERANCES OF 4S AND 2S DURATIONS.

		Total train duration
	Test	8s	12s	full
	2s	77.16	78.03	79.86
	4s	86.27	87.63	88.04