Quentin Le Lidec
email: quentin.le-lidec@inria.fr

Fabian Schramm

Louis Montaut

Cordelia Schmid

Ivan Laptev

Justin Carpentier

Leveraging Randomized Smoothing for Optimal Control of Nonsmooth Dynamical Systems

Keywords: optimization, optimal control, reinforcement learning

Optimal control (OC) algorithms such as differential dynamic programming (DDP) take advantage of the derivatives of the dynamics to control physical systems efficiently. Yet, these algorithms are prone to failure when dealing with non-smooth dynamical systems. This can be attributed to factors such as the existence of discontinuities in the dynamics derivatives or the presence of non-informative gradients. On the contrary, reinforcement learning (RL) algorithms have shown better empirical results in scenarios exhibiting nonsmooth effects (contacts, frictions, etc.). Our approach leverages recent works on randomized smoothing (RS) to tackle non-smoothness issues commonly encountered in optimal control and provides key insights on the interplay between RL and OC through the prism of RS methods. This naturally leads us to introduce the randomized Differential Dynamic Programming (RDDP) algorithm accounting for deterministic but non-smooth dynamics in a very sample-efficient way. The experiments demonstrate that our method can solve classic robotic problems with dry friction and frictional contacts, where classical OC algorithms are likely to fail, and RL algorithms require, in practice, a prohibitive number of samples to find an optimal solution.

Introduction

Theories and applications of optimal control (OC) and reinforcement learning (RL) are all related to the problem of minimizing a cost (resp. maximizing a reward) while fulfilling the system dynamics and constraints over a given time duration. Nonetheless, the resulting algorithms to solve OC or RL problems are based on different approaches, leading to very different performances in practice. On the one hand, in their vast majority, RL algorithms only exploit samples, leading to zero-th order approaches. On the other hand, optimal control and trajectory optimization techniques such as the iterative Linear Quadratic Regulator (iLQR) [START_REF] Li | Iterative linear quadratic regulator design for nonlinear biological movement systems[END_REF] and Differential Dynamic Programming (DDP) [START_REF] Tassa | Synthesis and stabilization of complex behaviors through online trajectory optimization[END_REF] rely on first-order and second-order linearization of the dynamics. Exploiting this derivative information makes them much more sample-efficient than their zero-th order counterparts from the field of RL. However, when considering complex scenarios such as robots interacting with their environments, the system dynamics may depict some non-smooth physical phenomena (dry friction, contact constraints, etc.). These properties may induce non-informative or discontinuous gradients that make gradient-based strategies fail [START_REF] Werling | Fast and feature-complete differentiable physics engine for articulated rigid bodies with contact constraints[END_REF]. On the contrary, RL algorithms have proven to be able to get around these non-smoothness issues in such cases, leading to impressive results when considering contact interactions [START_REF] Hwangbo | Learning agile and dynamic motor skills for legged robots[END_REF]. By treating the dynamics as a black-box function, derivative-free algorithms such as standard RL methods circumvent the issues as mentioned above. They can transparently deal with arbitrarily complex and non-smooth dynamics. However, completely disregarding the specific structure of the dynamics comes at the cost of often requiring a large number of samples.

In a recent growing effort, differentiable physical simulators have emerged in the context of exploiting informative gradients for control [START_REF] De Avila Belbute-Peres | End-to-end differentiable physics for learning and control[END_REF] and estimation [START_REF] Le Lidec | Differentiable simulation for physical system identification[END_REF]. Gradients of rigid body dynamics were obtained by differentiating the classical rigid body algorithms [START_REF] Featherstone | Rigid body dynamics algorithms[END_REF][START_REF] Carpentier | Analytical derivatives of rigid body dynamics algorithms[END_REF]. Simulating and differentiating physics with frictional contacts is more challenging as it requires to also solve for contact forces [START_REF] De Avila Belbute-Peres | End-to-end differentiable physics for learning and control[END_REF][START_REF] Le Lidec | Differentiable simulation for physical system identification[END_REF] and was made possible by differentiable optimization techniques [START_REF] Amos | Optnet: Differentiable optimization as a layer in neural networks[END_REF][START_REF] Agrawal | Differentiable convex optimization layers[END_REF]. Yet, these dynamics derivatives may present some discontinuities or lack of regularity, drastically impacting gradient-based optimization techniques, especially in the context of classical control algorithms [START_REF] Werling | Fast and feature-complete differentiable physics engine for articulated rigid bodies with contact constraints[END_REF].

In this work, we propose to leverage randomized smoothing (RS) techniques [START_REF] Duchi | Randomized smoothing for stochastic optimization[END_REF][START_REF] Berthet | Learning with differentiable pertubed optimizers[END_REF] to cope with nonsmooth dynamical systems in optimal control problems. From a theoretical perspective, we notably demonstrate how RS methods applied to OC problems allow us to close the gap with the RL setting (Sec. 3) in general. A first connection of RL and OC for control tasks with infinite state and action spaces has been introduced by [START_REF] Bradtke | Reinforcement learning applied to linear quadratic regulation[END_REF][START_REF] Bradtke | Adaptive linear quadratic control using policy iteration[END_REF] for the linear quadratic regulator with a focus on convergence proofs. From a practical perspective, we propose to use RS techniques within the frame of Differential Dynamic Programming to deal with nonsmooth dynamics, which are hard to handle in the vanilla setting (Sec. 4). We also introduce an adaptive strategy to automatically reduce the smoothing noise inherent to RS techniques across the optimization procedure. We experimentally show the practical benefits of our approach on various robotic tasks with increasing complexity, ranging from inverted pendulum to locomotion (Sec. 5).

Similarly to our work, [START_REF] Suh | Bundled gradients through contact via randomized smoothing[END_REF] introduced the notion of randomized smoothing on the state and control spaces to get gradients from the dynamics through contacts. The same authors extended the approach to motion planning for complex manipulation tasks [START_REF] Pang | Global planning for contact-rich manipulation via local smoothing of quasi-dynamic contact models[END_REF] by exploring the influence of zero-th and first-order gradient estimators. We take a different point of view by establishing links between RL and OC through the lens of RS, which is a first step towards a stronger interplay between these two fields. This viewpoint implies a practical difference: we interpret RS as an exploration term and only smooth the control space, leading to a sampled space of reduced dimension. Our experiments demonstrate that this is sufficient to perform well on complex robotics tasks.

Background

Our work builds on optimal control, reinforcement learning, and randomized smoothing techniques applied to robotics, which are briefly introduced in this section.

Optimal control. We consider the OC problem of controlling a robot by minimizing a cost l while satisfying the system dynamics f :

min x,u T 0 l(τ, x(τ ), u(τ ))dτ (1a) s.t. ẋ(τ ) = f (x(τ ), u(τ )), (1b) x(0) = x0 , (1c)
where x(τ ) ∈ S and u(τ ) ∈ A are the state and the control action of the system at time τ respectively, x0 ∈ S is a given initial state and T is the time horizon. In this paper, we call smooth a dynamics whose corresponding function f is differentiable everywhere. While in robotics f is often considered smooth, contact interactions give rise to points where f is only sub-differentiable [START_REF] Brogliato | Nonsmooth mechanics[END_REF].

Two different approaches may be used to solve [START_REF] Li | Iterative linear quadratic regulator design for nonlinear biological movement systems[END_REF], namely direct and indirect approaches. Direct approaches translate the infinite dimensional problem (1) into a finite nonlinear programming problem (NLP) and exploit off-the-shelf constrained optimization solvers [START_REF] Diehl | Fast direct multiple shooting algorithms for optimal robot control[END_REF] for solving it. In particular, this leads to a discrete numerical problem of the form:

min x,u J(x,u) l N (x N ) + N -1 t=0 l t (x t , u t ) (2a) s.t. x t+1 = f (x t , u t ), ∀t ∈ [0, N -1] , (2b) x 0 = x0 , (2c)
where x = {x 0 , . . . , x N } and u = {u 0 , . . . , u N -1 }, with N being the number of time steps. Alternatively, indirect approaches first apply the optimality conditions of optimal control problems (e.g., Hamilton-Jacobi-Bellman or Pontryagin's maximum principles) and then discretize these conditions in order to solve the problems numerically. This notably offers the advantage of highlighting the underlying sparsity of constraints induced by times. In this line of work, iLQR [START_REF] Li | Iterative linear quadratic regulator design for nonlinear biological movement systems[END_REF] and DDP [START_REF] Mayne | A second-order gradient method for determining optimal trajectories of non-linear discrete-time systems[END_REF][START_REF] Tassa | Synthesis and stabilization of complex behaviors through online trajectory optimization[END_REF][START_REF] Mastalli | Crocoddyl: An efficient and versatile framework for multi-contact optimal control[END_REF] are the most well-known first and second-order algorithms, with linear or quadratic-type convergence rates respectively. Moreover, their recent adaptations can even handle trajectory constraints [START_REF] Howell | ALTRO: A fast solver for constrained trajectory optimization[END_REF][START_REF] Kazdadi | Equality constrained differential dynamic programming[END_REF][START_REF] Jallet | PROXDDP: Proximal Constrained Trajectory Optimization[END_REF] and implicit dynamics [START_REF] Jallet | Implicit Differential Dynamic Programming[END_REF]. More closely related to our work, sampled DDP [START_REF] Rajamäki | Sampled differential dynamic programming[END_REF] lies in between, as it uses stochastic estimates of the dynamics derivatives to deal with dynamics for which gradients are unavailable.

Reinforcement learning considers the dynamics as a black-box function and uses parameterized stochastic policies π θ to explore the action space better. This model-free approach can naturally handle complex or even unknown dynamics. During training, the cumulated sum of rewards R along the trajectories sampled from the distribution ρ θ induced by the policy π θ is maximized the along the trajectories sampled from the distribution ρ θ induced by the policy π θ is minimized (which is equivalent to minimize the average cost J = -R), leading to the following problem:

max θ E (x,u)∼ρ θ [R(x, u)] (3)
where trajectories are generated by the policy in the following way:

u t ∼ π θ (•|x t ), (4a) x t+1 = f (x t , u t ). ( 4b
)
Policy Gradient (PG) [START_REF] Williams | Simple statistical gradient-following algorithms for connectionist reinforcement learning[END_REF][START_REF] Sutton | Reinforcement learning: An introduction[END_REF] algorithms aim at maximizing (3) by using a zero-th order estimate of the gradient as a ascent direction:

∇ θ R P G = E (x,u)∼ρ θ [R(x, u)∇ θ log ρ θ (x, u)] (5)
The resulting randomness induces some variance in the gradient estimates, which slows down optimization and fosters exploratory behaviors, potentially leading to more global solutions. Finally, this makes RL an instance of random optimization, which will be discussed in more detail in Sec. 3.3. Random optimization techniques are among the earliest optimization schemes and were developed in order to tackle problems where the objective function is discontinuous or nonsmooth. In this situation, the gradients only provide limited information and are not suited for use in classic gradientbased optimization techniques [START_REF] Matyas | Random optimization[END_REF]. During a random search, one classical strategy consists in sampling a random direction at each step and then moving towards the corresponding directional derivative [START_REF] Polyak | Introduction to Optimization[END_REF], which is the basis of the Simultaneous Perturbation Stochastic Approximation algorithm [START_REF] Spall | Introduction to stochastic search and optimization: estimation, simulation, and control[END_REF]. Gradient Sampling (GS) is an alternative strategy that approximates the subgradient at non-differentiable points by taking the descent direction inside the convex hull of some gradients randomly sampled in a neighborhood [START_REF] Burke | A robust gradient sampling algorithm for nonsmooth, nonconvex optimization[END_REF].

Recent works [START_REF] Nesterov | Random gradient-free minimization of convex functions[END_REF] exploit the equivalence between random optimization techniques and perform stochastic gradient descent on a smoothed version of the original problem to get theoretical convergence bounds. In a parallel line of work, randomized smoothing was recently introduced in the machine learning community in order to be able to differentiate through Linear Programming (LP) argmin operators [START_REF] Abernethy | Perturbation techniques in online learning and optimization[END_REF][START_REF] Berthet | Learning with differentiable pertubed optimizers[END_REF]. It was also applied to make rendering [START_REF] Le Lidec | Differentiable Rendering with Perturbed Optimizers[END_REF], patch selection [START_REF] Cordonnier | Differentiable patch selection for image recognition[END_REF], clustering [START_REF] Stewart | Differentiable clustering with perturbed spanning forests[END_REF] and collision detection [START_REF] Montaut | Differentiable collision detection: a randomized smoothing approach[END_REF] operations differentiable. Unlike their classical counterparts, these perturbed optimizers are guaranteed to have non-null gradients everywhere, making it possible to use gradient-based optimization methods.

Concretely, let Z be a random variable whose probability distribution µ is a Gibbs measure. A function g can be approximated by convolving it with this probability distribution:

g (x) = E Z∼µ [g(x + Z)] (6)
which corresponds to the randomly smoothed counterpart of g and can be estimated with a Monte-Carlo estimator as follows:

g (x) ≈ 1 M M i=0 g(x + Z (i) ) (7)
where {Z (1) , . . . , Z (M ) } are i.i.d. samples and M is the number of samples. Using integration by part, we have the following expression of gradients:

∇ x g (x) = E Z∼µ -g(x + Z) ∇ log µ(Z) (8a) = E Z∼µ [∇g(x + Z)] , (8b)
where (8a) and (8b) corresponds respectively to the zero-th and first order expressions of ∇ x g . In practice, it is possible to approximate the smoothed gradient by the first order Monte-Carlo (MC) estimator (9b) or, because E Z∼µ ∇ log µ(Z) = 0, by the variance reduced zero-th order MC estimator (9a):

∇ x g (x) ≈ 1 M M i=1 g(x) -g(x + Z (i) ) ∇ log µ(Z (i) ) (9a) ≈ 1 M M i=1 ∇g(x + Z (i) ) (9b)
More intuitively, because of the local averaging effect of the convolution, g (x) is always smoother than g. Indeed, g is guaranteed to be differentiable [START_REF] Bertsekas | Stochastic optimization problems with nondifferentiable cost functionals[END_REF], uniformly close from g and its gradient to be Lipschitz-continuous [START_REF] Duchi | Randomized smoothing for stochastic optimization[END_REF][START_REF] Berthet | Learning with differentiable pertubed optimizers[END_REF]. Moreover, g -→0 ---→ g, so reducing the perturbation by decreasing leads to a reduced gap between g and the original function g but also results in a less smooth approximation. In terms of computational complexity, evaluating ∇g with M samples induces a complexity increased by a factor M . The computation being easily parallelizable, this, in fact, leads to a constant computational time without a critical impact on the memory footprint, as shown in the context of differentiable rendering in [START_REF] Le Lidec | Differentiable Rendering with Perturbed Optimizers[END_REF]. Adding stochasticity to gradients is also proven to help escape saddle points when optimizing nonconvex functions [START_REF] Ge | Escaping from saddle points-online stochastic gradient for tensor decomposition[END_REF], which constitutes a positive side effect of randomized smoothing. Finally, other previous works on the use of randomized smoothing demonstrate how it can improve convergence rates when optimizing nonsmooth functions [START_REF] Duchi | Randomized smoothing for stochastic optimization[END_REF] and lead to more robust solutions [START_REF] Cohen | Certified adversarial robustness via randomized smoothing[END_REF]. Non-smooth dynamics in robotics. Generating movements for locomotion and manipulation is considered a major task in robotics. Whether it is to move itself or an object, the robot has to interact with its environment by making and breaking contacts, which induce non-smooth dynamics [START_REF] Brogliato | Modeling, analysis and control of robot-object nonsmooth underactuated lagrangian systems: A tutorial overview and perspectives[END_REF]. It is common to address these issues by making the hypothesis of bilateral contacts either between feet and the ground for locomotion [START_REF] Farshidian | An efficient optimal planning and control framework for quadrupedal locomotion[END_REF] or between the fingers and the manipulated object [START_REF] Mason | Robot hands and the mechanics of manipulation[END_REF] as detailed in Appendix 6. This hypothesis limits the range of possible movements by enabling only conservative strategies and fixing the contact modes during predetermined phases. RL approaches go beyond these constraints by not making any assumption on the underlying physical model and, hence, are said to be model-free. To go towards more dynamical movements, model-based approaches should be able to handle the switches between various contact modes better [START_REF] Carpentier | Recent progress in legged robots locomotion control[END_REF][START_REF] Wensing | Optimization-based control for dynamic legged robots[END_REF]. Direct optimization of the sequence of contacts [START_REF] Posa | A direct method for trajectory optimization of rigid bodies through contact[END_REF][START_REF] Toussaint | Differentiable physics and stable modes for tool-use and manipulation planning[END_REF] avoids the tedious manual engineering by shifting the burden to mixed-integer solvers. Alternatively, the seminal work from [START_REF] Mordatch | Discovery of complex behaviors through contact-invariant optimization[END_REF] exploits an analytical smoothing of the dynamics to use classical smooth optimization techniques. This work and the concurrent [START_REF] Suh | Bundled gradients through contact via randomized smoothing[END_REF] propose to apply randomized smoothing on the dynamics before leveraging Trajectory Optimization algorithms.

x t + 1 f(0, ⋅ ) f ε (0, ⋅ ), (ε, M) = (1.0, 32) f ε (0, ⋅ ), (ε, M) = (4.0, 32) f ε (0, ⋅ ), (ε, M) = (4.0, 1024)

Bridging the gap between optimal control and reinforcement learning

In this section, we present the caveats of poorly informative gradients for classical control algorithms, which may, for instance, occur in the presence of nonsmooth dynamical systems. To overcome these limitations, we propose to exploit the randomized smoothing approach in the trajectory optimization paradigm. We also detail a connection between randomized smoothing and RL methods, thus explaining how they effectively solve problems involving poorly informative gradients.

Locally optimal solutions of optimal control problems

As discussed in Sec. 2, several approaches may be used for solving OC problems of the form [START_REF] Tassa | Synthesis and stabilization of complex behaviors through online trajectory optimization[END_REF]. In particular, one can substitute x 1 , . . . , x N thanks to the constraint on the dynamics [START_REF] Duchi | Randomized smoothing for stochastic optimization[END_REF] and express problem (2) only in terms of u 0 , . . . , u N -1 , leading to the following but equivalent unconstrained optimization problem: min

u J(x(u), u) (10)
where x(u) is recursively defined by:

x 0 = x0 and x t (u) = f (x t-1 (u), u t ), (11)
corresponding to an integration process (e.g., exploiting a dynamical simulator). Unrolling the successive integration steps facilitates the efficient differentiation of x(u). Solving the equivalent problem (10) can be achieved using a classical unconstrained optimization algorithm such as gradient descent, consisting of backpropagating through time [START_REF] Bengio | Learning long-term dependencies with gradient descent is difficult[END_REF]. In the case of a robotic system, the dynamic simulator involves making and breaking contacts and switching between sticking and sliding contacts, which induces several modes in the dynamics. More formally, the computation of f results from a Nonlinear Complementarity Problem (NCP), making it inherently non-smooth as detailed in the Appendix 6 and [START_REF] Acary | On solving contact problems with Coulomb friction: formulations and numerical comparisons[END_REF][START_REF] Lidec | Contact models in robotics: a comparative analysis[END_REF]. In addition, the cost function J, which often involves the robot kinematics via a penalty on the end-effector position as done in the experiments of Sec. 5, is non-convex. Thus, solving (2) with approaches described previously can lead to local solutions because of the inherent non-convexity and non-smoothness of the problem.

To illustrate this, one can think about the problem of lifting a cube [3] illustrated in Fig. 2. When u is initialized with null control, this results in ∇ u f = 0 because of the complementarity constraint arising from unilateral contacts (see IV. of [START_REF] Werling | Fast and feature-complete differentiable physics engine for articulated rigid bodies with contact constraints[END_REF]). By supposing that ∂J ∂u = 0 and applying the chain rule, we have that:

∇ u J(x(u), u) = ∂J ∂x ∂x ∂u + ∂J ∂u = ∂J ∂x ∂x ∂u . (12)
Moreover, since ∇ u f = 0, we have that starting from the first time step ∂x 1 ∂u = ∂f ∂u (x 0 , u 0 ) = 0 and with the time recursion, this leads to:

∂x t+1 ∂u = ∂f ∂u (x t , u t ) + ∂f ∂x (x t , u t ) ∂x t ∂u = 0, (13)
implying that ∂x ∂u = 0. Finally, because ∇ u J = 0, an algorithm exploiting only the local gradient information (e.g., gradient or Newton descent) will stop at this point, leaving the problem unsolved and blocked at a local maximum. The classical DDP algorithm would get stuck in a similar situation for the same reasons. It is also worth mentioning that even when equipped with proximal smoothing [START_REF] Parikh | Proximal algorithms[END_REF], such algorithms would still rely on local information and, thus, fail to overcome the issue of non-informative gradients.

Randomized smoothing of the system dynamics

The issue highlighted above is due to the inability of deterministic control algorithms to deal with non-smooth dynamics and their non-informative gradients, which often occur for physical systems involving contact or friction.

An intuitive way to circumvent this issue consists of introducing randomization in the optimization process in order to get a more exploratory behavior by collecting samples in a larger neighborhood around a given point when compared to classic methods such as the ones relying on local gradient information. This is precisely the motivation behind Randomized Smoothing and, to some extent, behind Reinforcement Learning, as discussed in Sec. 3.3. We adapt the formulation (2) by artificially smoothing the system dynamics using randomized smoothing, leading to the following smooth but approximated problem:

min x,u J(x, u) (14a) s.t. x t+1 = f (x t , u t ), ∀t ∈ [0, N -1] , (14b) x 0 = x0 . ( 14c
)
where

f (x, u) = E Z∼µ [f (x, u + Z)]
and µ is a noise distribution. It is worth mentioning that f corresponds to a randomized version of f , which is only perturbed with respect to the control input u. Perturbing the control but not the state ensures that only reachable states are explored. When → 0, problem [START_REF] Bradtke | Adaptive linear quadratic control using policy iteration[END_REF] converges to the original problem [START_REF] Tassa | Synthesis and stabilization of complex behaviors through online trajectory optimization[END_REF]. The proposed solution of using a smoothed approximation of the system dynamics is very generic. It can be easily instantiated in most of the existing trajectory optimization frameworks without major modifications. Yet, there is a priori no obvious choice of the sampling distribution µ, neither for the number of particles sufficient in the Monte-Carlo estimator of the system dynamics and gradient computations. Another difficulty lies in the proper scheduling of the noise intensity towards 0 in order to remove the effect of noise at convergence to recover the original problem.

In Sec. 4, we introduce an algorithmic variation of the so-called Differential Dynamic Programming algorithm, which relies on the smoothed dynamics f . In particular, we propose an automatic scheduling of the noise intensity and an auto-tuning strategy of the number of particles in the Monte-Carlo estimators.

Reinforcement learning through the prism of randomized smoothing

At this stage, one could wonder why RL demonstrated empirical success even in the case of the non-smooth dynamics evoked in 3.1. We provide the first possible explanation by drawing a parallel between the descent directions used in RL and the one from random optimization (8a) when applied to the OC problem [START_REF] Tassa | Synthesis and stabilization of complex behaviors through online trajectory optimization[END_REF].

Indeed, the ascent directions used in the classical RL algorithm REIN-FORCE with baseline (the same considerations remain valid for the closely related actor-critic algorithms) [START_REF] Greensmith | Variance reduction techniques for gradient estimates in reinforcement learning[END_REF][START_REF] Sutton | Reinforcement learning: An introduction[END_REF][START_REF] Mnih | Asynchronous methods for deep reinforcement learning[END_REF] can be written as:

∇ θ R P G = E (x,u)∼ρ θ R(x, u) -V (x 0 ) ∇ θ log ρ θ (x, u) , ( 15
)
where V is an estimate of the value function, which exactly corresponds to the variance-reduced version of the randomly smoothed approximation (9a). More intuitively, in order to deal with the non-smoothness of the dynamics or reward functions, Policy Gradient adds noise in the action space by sampling actions from a stochastic policy. Concretely, this causes ∇ θ R P G = 0 even in regions where ∇ u R = 0 and first or second order gradient methods fail, as discussed in Sec. 3.1. Thus, introducing some stochasticity allows RL to smooth the original problem and avoid the computation of gradients from the dynamics when they are unknown. Unfortunately, this generally comes at the cost of increased variance in the estimates of ∇R P G , which induces a slower convergence rate [START_REF] Nesterov | Random gradient-free minimization of convex functions[END_REF]. A similar study could be done with Evolution Strategies used in [START_REF] Salimans | Evolution strategies as a scalable alternative to reinforcement learning[END_REF][START_REF] Mania | Simple random search of static linear policies is competitive for reinforcement learning[END_REF], which prefer to generate trajectories with deterministic policies but parameterized by randomly sampled parameters.

Following this analysis, in a way similar to RL, we propose using randomized smoothing to compute informative gradients even when the standard ones are non-informative. The obtained gradients can then be exploited in the context of optimal control problems with higher-order optimization algorithms in order to get improved convergence rates, as shown in the context of the widely used Differential Dynamic Programming algorithm in the following section.

In earlier works, [START_REF] Bradtke | Reinforcement learning applied to linear quadratic regulation[END_REF][START_REF] Bradtke | Adaptive linear quadratic control using policy iteration[END_REF] derive a convergence proof for policy iteration methods in a deterministic, linear, and time-variant LQR setup for infinite state and action spaces. An algorithm was presented, that converges to the optimal policy and involves some additional exploration noise in the control due to a requirement in the underlying recursive least-square algorithm. However, the noise has a very different motivation from our usage of injected noise for randomized smoothing of non-smooth function surfaces, which is done by averaging over several noise samples.

Randomized Differential Dynamic Programming

This section introduces our randomized Differential Dynamic Programming algorithm. This novel formulation builds on the previous analysis to incorporate randomized smoothing in the optimal control paradigm in order to increase the exploration of classical DDP and efficiently solve problems involving non-smooth dynamical systems.

Dynamic programming with smoothed physics

The main idea behind randomized DDP consists in exploiting the formulation [START_REF] Bradtke | Adaptive linear quadratic control using policy iteration[END_REF] and replacing the original, possibly non-smooth, dynamics f by its randomly smoothed approximation f in the forward and backward passes of the vanilla DDP. Doing so allows to benefit from the efficiency of DDP even in situations where the original DDP algorithm will fail.

First, we introduce the cost-to-go function associated to our problem (14):

J t (x t , u t , . . . , u N -1 ) = l N (x N ) + N -1 j=t l j (x j , u j ) (16)
Algorithm 1: Randomized DDP algorithm Input: OC problem: J, f , initial trajectory: x, u, target noise and precision: * , α * , initial noise and precision: , α, adaptive scheme parameters: ρ, γ Output: Solution x, u of the OC problem (2)

1 repeat 2 repeat 3 k, K ← Backward Pass (19); 4 x, u ← Forward Pass (22); 5 until Q u Q -1 uu < α; 6 ← /ρ ; 7 α ← α/γ; 8 until α < α * and < * ;
and the value function which verifies the Bellman's equation:

V t (x t ) = min ut,...,u N -1 J t (x t , u t , . . . , u N -1 ) (17a) = min ut l t (x t , u t ) + V t+1 (f (x t , u t )) (17b)
with the terminal condition V N (x) = l N (x). Additionally, the Q-function is defined by:

Q t (x, u) = l t (x, u) + V t+1 (f (x, u)) (18)
As done in the classical Differential Dynamic Programming algorithm, we exploit the sparsity of constraints induced by time via Bellman's equation to solve the problem [START_REF] Bradtke | Adaptive linear quadratic control using policy iteration[END_REF]. To do so, local second-order approximations of the value and the Q functions are built by backpropagating the Bellman's equation [START_REF] Brogliato | Nonsmooth mechanics[END_REF] the backward pass equations:

Q xx = l xx + f x V xx f x + V x f xx (19a)
Q ux = l ux + f u V xx f x + V x f ux (19b)
Q uu = l uu + f u V xx f u + V x f uu (19c)
Q x = l x + V x f x (19d) Q u = l u + V x f u (19e) q = l + v , (19f)
where v = V t (x), and and the subscript on x and u are the usual notations for the partial derivative w.r.t the state and control variables, and the superscript V denotes the value function at the next time-step. Minimizing these local quadratic approximations of Q w.r.t u gives:

u = k + K(x -x), k = -Q -1 uu Q u and K = -Q -1 uu Q ux , (20)
and injecting [START_REF] Mastalli | Crocoddyl: An efficient and versatile framework for multi-contact optimal control[END_REF] in [START_REF] Brogliato | Nonsmooth mechanics[END_REF] gives rise to:

V xx = Q xx -K Q uu K (21a) V x = Q x -k Q uu K (21b) v = q - 1 2 k Q uu k. (21c)
Finally, u is updated with a line search during the forward computation:

u n t = u n-1 t + αk + K(x n t -x t ) (22a) x n t+1 = f (x n t , u n t ) (22b)
with the initial condition x n 0 = x0 and where the superscript n denotes the number of optimization iterations. During line search, the noise is fixed as described in [START_REF] Vaswani | Painless stochastic gradient: Interpolation, linesearch, and convergence rates[END_REF][START_REF] Nocedal | Numerical optimization[END_REF]. Repeating the forward and backward steps by taking the new trajectory x n as the new reference x allows to find the optimal control u under the form of a linear policy [START_REF] Mastalli | Crocoddyl: An efficient and versatile framework for multi-contact optimal control[END_REF] which is optimal around the trajectory.

As detailed previously in Sec. 2, smoothing the physics makes it possible to have non-null gradients ∇ u f thus inducing non-null Q u , as illustrated in Fig. 3. Alternatively, the stochasticity can also be interpreted as an exploration term that has proven to be crucial in the RL framework to escape regions where the local information from gradients does not provide exploitable insight on the problem being solved (see Sec. 3.3).

The main difference between our approach formulated at [START_REF] Bradtke | Adaptive linear quadratic control using policy iteration[END_REF] and "Policy Gradient"-type algorithms lies in the scope of the randomized smoothing. Indeed, RL smooths the whole problem while we "only" smooth the dynamics f : note that the expectation is on entire trajectories in (3) while it is at the time-step level in [START_REF] Bradtke | Adaptive linear quadratic control using policy iteration[END_REF] resulting in a reduced variance in the latter case [START_REF] Peters | Reinforcement learning of motor skills with policy gradients[END_REF]. Moreover, we find that preserving the original recursive structure of (2) is beneficial in the case of known dynamics f as it allows to benefit from the efficient dynamic programming backward passes [START_REF] Mayne | A second-order gradient method for determining optimal trajectories of non-linear discrete-time systems[END_REF] of DDP. While RL requires to also smooth the cost function to deal with sparse rewards, it is not necessary in our case as current trajectory optimization algorithms can handle hard constraints [START_REF] Kazdadi | Equality constrained differential dynamic programming[END_REF][START_REF] Jallet | Implicit Differential Dynamic Programming[END_REF][START_REF] Howell | ALTRO: A fast solver for constrained trajectory optimization[END_REF].

Adaptive smoothing

To enforce the convergence towards an optimal (local) solution, it remains crucial to reduce the noise injected via the randomized smoothing across the iterations. A first possible strategy [START_REF] Suh | Bundled gradients through contact via randomized smoothing[END_REF] consists in relying on Robbins-Monro rule [START_REF] Robbins | A Stochastic Approximation Method[END_REF] by decreasing the variance in a way such that k 2 k < ∞ to guarantee convergence towards a local minimum. In this work, we propose to decrease k in a way that adapts to the problem and avoids the smoothing being reduced too quickly, which would lead to performance similar to classical DDP, or too slowly, which would induce an unnecessarily large number Without randomized smoothing, the gradients are null and the system remains stuck in the downward position (blue).

of iterations. We adapt the smoothing by solving a cascade of randomly smoothed DDP problems. More concretely, ∇ u Q ∞ decreases towards 0 when converging towards an optimum. Whenever ∇ u Q ∞ is under a given precision threshold α we consider the smooth sub-problem solved and thus reduce the noise intensity and the precision threshold α, by a factor ρ and γ respectively, before solving the next sub-problem. Typically, ρ and γ are taken equal and set to the value 2. This adaptive scheme is summarized in Alg. 1 and is generic, so it could be transferred to any algorithm using randomized smoothing.

Experiments

In this section, we demonstrate the practical benefits of our randomized DDP algorithms on a set of different systems, ranging from simple toy examples to real robotic systems. We first examine the underlying mechanisms and pendulum and a cube and extend them afterward to a quadrotor and a quadrupedal robot. Our implementation is based on the open-source frameworks Crocoddyl [START_REF] Mastalli | Crocoddyl: An efficient and versatile framework for multi-contact optimal control[END_REF] for the DDP algorithm and on Pinocchio [START_REF] Carpentier | The Pinocchio C++ library -A fast and flexible implementation of rigid body dynamics algorithms and their analytical derivatives[END_REF] and [START_REF] Carpentier | Analytical derivatives of rigid body dynamics algorithms[END_REF][START_REF] Le Lidec | Differentiable simulation for physical system identification[END_REF] for the derivatives of the dynamics with and without contacts.

Avoiding local optima of smooth dynamics

We consider the task of raising a pendulum from the downward (θ = 0) to the upward vertical position (θ = π). The system's state is characterized by the angle θ, representing the deviation of the pendulum bar from the vertical axis, and by the angular velocity θ. The tip's position also called the endeffector, can be determined through a trigonometric relation and is denoted as p(θ), where p * is the desired position. We optimize the cost function

J(x, u) = w p p(θ N ) -p * 2 + N t=0 w u u t -u * 2 , (23)
over N = 400 time steps and each with a duration dt = 5 × 10 -3 s. The weights are defined as w p = 2, w u = 2 × 10 -5 and the system is described using the generalized coordinates x = (θ, θ) of position and velocity. As explained previously, due to the robot kinematics, the term of J involving the distance on the end-effector p(θ N ) -p * is non-convex leading to multiple potential local optima of the control problem. We run 24 iterations of the classical DDP and the RDDP algorithm optimizing for a trajectory with 400 time steps to swing the pendulum up. For the problem [START_REF] Jallet | PROXDDP: Proximal Constrained Trajectory Optimization[END_REF], the solution (u, x) = (0, 0) is a local extremum, and the classical DDP algorithm gets stuck at this point (Fig. 4). As discussed in Sec. 3.3, randomized smoothing using a smaller number of only M = 4 samples per control input to smooth and average the resulting state allows RDDP to avoid this local optimum (Fig. 3,4). Here, two distinct effects are at work: i) the Randomized Smoothing can smoothen out some local optima, and ii) the noise from the Monte-Carlo estimator helps to escape from unstable critical points as detailed in [START_REF] Ge | Escaping from saddle points-online stochastic gradient for tensor decomposition[END_REF]. In addition to converging towards better optima in the case of smooth dynamics, we primarily designed RDDP to provide a solution when it comes to the aforementioned issues from non-smooth dynamics (see Sec. 3.1 and Appendix 6), shown in the next section.

Controlling non-smooth dynamics

To illustrate the issues induced by the non-smoothness of unilateral contact and frictions, we consider the preliminary tasks of taking off or sliding a cube on a table (appearing in [START_REF] Werling | Fast and feature-complete differentiable physics engine for articulated rigid bodies with contact constraints[END_REF]). In these experiments, we use a cost function similar to the one of [START_REF] Jallet | PROXDDP: Proximal Constrained Trajectory Optimization[END_REF], optimizing the difference between a target configuration and the final configuration of the system at the end of the planned trajectory. As shown by Fig. 6 (left), our approach allows us to complete this task while we have shown in Sec. 3.1 that it is impossible when relying on classic gradient information. Similarly, it is possible to apply the RDDP algorithm to control more complex systems such as a double pendulum and a cartpole with dry friction on the joints (Fig. 5). The double pendulum and the cartpole are illustrative instances of underactuated systems whose dynamics are non-smooth due to friction. In both cases, classical DDP fails to optimize the control variable using the true dynamics. Once again, smoothing the dynamics with respect to the control u using M samples per time step allows to have non-null gradients almost everywhere and thus to apply classical DDP (Fig. 6, middle and right). Combining randomized smoothing with the DDP algorithm makes it possible to solve the problem of controlling a complex system involving contacts and friction very efficiently. Interestingly, the FDDP algorithm [START_REF] Mastalli | Crocoddyl: An efficient and versatile framework for multi-contact optimal control[END_REF], which allows for unfeasible trajectories, also failed to overcome the non-smoothness from dry frictions. We also studied the influence of the number of samples M used for the MC estimators (Fig. 5) and the adaptive scheme (Fig. 6) on the quality of the obtained solution. We noticed that good results could be obtained with a relatively small number of samples M for the first-order MC gradient estimator, meaning that the supplementary computational costs are limited when compared to classical DDP. On the contrary, using an adaptive scheme for decreasing the smoothing perturbation leads to a more precise solution by allowing the gradients of the trajectory optimization problems to converge to zero (Fig. 6). This represents an advantage of our method when compared to classical RL approaches, as the latter requires keeping a non-null noise to estimate gradients, preventing them from converging very precisely.

Controlling contact interactions in robotics

The following experiments explore the application of our method to control complex robotics systems interacting with their environments via fric- tional contacts.

Quadrotor. In our first experiment, we control the takeoff of a quadrotor. The drone starts in an initial state, landing on the ground, and the goal is to reach a target position one meter higher in the air. This case of a drone with four actuated thrusts represents a realistic and challenging robotics task as one could need to control its trajectory without having to plan for takeoff and landing phases, which would require breaking or making unilateral contacts.

Concretely, the cost of the control problem J is similar to (23) but with p now denoting the position of the base of the quadrotor with w p = 4 and w u = 10 -3 . As shown by Fig. 7, the classical DDP algorithm fails due to the issue of non-informative gradients whenever the drone rests on the ground. In contrast, our approach is quickly (less than 10 optimization steps) able to generate a trajectory that accurately reaches the target position.

Solo robot. Finally, we apply our algorithm to solve a task on a legged robotic system with frictional contacts. We use one leg of the Solo robot [START_REF] Grimminger | An open torque-controlled modular robot architecture for legged locomotion research[END_REF] similar to [START_REF] Viereck | Learning a structured neural network policy for a hopping task[END_REF], which results in a hopper with 3 degrees of freedom, namely the hip and knee joint, and a vertical translation of the base. Therefore, the state is fully described by the position q ∈ R 3 and velocity q ∈ R 3 . The system is underactuated as only the knee and the hip of the robot are actuated. The task is to make the system jump up from an initial stretched configuration (Fig. 8, Left) to a position 30cm above the ground along the vertical axis using a similar formulation for the control cost R as in [START_REF] Jallet | PROXDDP: Proximal Constrained Trajectory Optimization[END_REF]. Without randomized smoothing, the DDP algorithm has no gradient information in this configuration; the algorithm converges to a local optimum, and the system is not moving. On the contrary, using RDDP achieves solving the task by first bending the leg downward (Fig. 8, 3 rd image), to be then able to apply a control torque that leads to contact forces with the ground moving the leg up to the target position. It is worth noting that this movement was generated without pre-specifying contact phases, as the optimizer does the contact planning entirely. This represents an encouraging first step towards automatically generating locomotion movements via trajectory optimization.

Conclusion

Analyzing reinforcement learning via the theory of randomized smoothing allows us to understand how crucial the exploratory characteristic of these algorithms is to solve control problems with non-smooth dynamics. By transferring these ideas to the field of trajectory optimization, we have leveraged randomized smoothing in this paper to propose an approximate and smooth formulation of the original optimal control problem. Exploiting this new formulation the well-established DDP algorithm results in an approach that can cope with the presence of contact interactions and friction in a sample-efficient manner. We've also demonstrated the capacity of our method to properly cope with standard robotics systems (pendulum, cart pole, drone, Solo robot), including non-smooth dynamical effects (frictions, contacts) where classic optimization-based are likely to fail. In future work, we plan to investigate possible uses of the information contained in the variance of the several particles of the Monte-Carlo estimator. In particular, we believe this could help to build a more robust adaptive scheme by detecting when increasing the noise , the precision threshold α, or even the number of particles M is necessary. action of the environment on the system via the contact vector impulse λ ∈ R m and should be incorporated into the Lagrangian equations of motion (.1):

M v t+1 = M v f + J T λ (.5)
Unilateral contacts are usually used when a system is in contact with its environment and enforce the normal component of the separation vector, i.e. the signed distance function, to be non-negative: Φ(q) N ≥ 0 (.6)

where Φ(q) ∈ R 3×nc , n c is the number of contacts, and the subscripts N and T respectively account for the normal and tangential components. Similarly to (.4), this can be expressed as:

c N -c * N ≥ 0 ( .7)
where c = Jv t+1 ∈ R 3×nc is the velocity of contact points and c * N is set to model physical effects or improve the numerical accuracy of the solutions (commonly referred to as Baumgarte stabilization). Unilateral contacts constrain the possible efforts λ depending on the state of the system and the friction coefficient. In general, the contact forces λ can only be repulsive as they should not act in a glue-like fashion (the environment can only push and not pull on the feet of a legged robot) and, thus, are forced to be nonnegative. An impulse cannot occur when an object takes off, implying that the normal velocity and impulse cannot be non-null simultaneously.

In a frictionless situation, the tangential forces are null, which implies that λ T = 0. Combining these conditions, we obtain the so-called Signorini condition:

0 ≤ λ N ⊥ c N -c * N ≥ 0 ( .8)
However, such a condition does not define a mapping between λ N and c N , i.e., the contact forces are not a function of the penetration error. Indeed, its representation is an infinitely steep graph and this problem is referred to as a Linear Complementarity Problem (LCP). Therefore, the computation of f in [START_REF] Duchi | Randomized smoothing for stochastic optimization[END_REF] which corresponds to the computation of v t+1 in (.5) and q t+1 via integration is facing the issues of non-smoothness and non-convexity. If friction is included, additional conditions on λ T , i.e. lying in a friction cone and satisfying the maximum dissipation principle, lead to a reformulation of the problem as a nonlinear complementarity problem (NCP), see [START_REF] Lidec | Contact models in robotics: a comparative analysis[END_REF] for further details.

Figure 1 :

 1 Figure 1: Illustration of randomized smoothing effects on the front left leg of the Solo robot.

Figure 2 :

 2 Figure 2: Top: A slight vertical force cannot break the unilateral contact, leaving the cube on the floor and, thus, the state unchanged. Bottom: The non-smoothness of physics induces null gradients ∇ u J which results in the failure of classical optimization techniques (3.1).

Figure 3 :

 3 Figure 3: Left: The randomly-smoothed dynamics allow to get non-null gradients and escape the local maximum for the inverted pendulum. Right: Q 0 (x, •) value function of the pendulum with dry friction exhibits plateaus where Q u = 0 around u = 0 while randomly smoothed dynamics leads to Q u = 0.

= 1 ε init = 10 - 1 , M = 4 Figure 4 :

 110144 Figure 4: Randomized smoothing allows to escape from the local optima (x, u) = 0 for the inverted pendulum (see Fig 3) and the system reaches the upward position (orange).Without randomized smoothing, the gradients are null and the system remains stuck in the downward position (blue).

Figure 5 :

 5 Figure 5: Top: Application of RDDP to precisely control a cartpole movement requiring overcoming dry friction on the joints and moving the pole straight upward. Bottom:Using RDDP for the cartpole task, we compare the influence of the sample size on the solution. Only a few samples (here M=8) are necessary to get reasonable results. Increasing the number of samples in the first order Monte-Carlo estimator of the gradients leads to very marginal improvement.

= 1 εinit = 10 - 1 , M = 4 ε = 10 - 1 ,= 1 εinit = 10 - 1 , M = 4 ε = 10 - 1 ,= 1 εinit = 10 - 1 , M = 4 ε = 10 - 1 ,= 1 εinit = 10 - 1 , M = 4 ε = 10 - 1 ,= 1 εinit = 10 - 1 , M = 4 ε = 10 - 1 ,= 1 εinit = 10 - 1 , M = 4 ε

 110141011101410111014101110141011101410111014 = 10 -1 , M = 4

Figure 6 :

 6 Figure 6: This figure shows the effect of our adaptive noise scheduling (orange) RDDP in comparison against fixed noise (green) RDDP and classical DDP with no noise (blue) for three different tasks. Left: RDDP solves tasks requiring breaking contacts such as the cube-lifting from Fig.2Middle: Similarly, the approach solves classical control problems on underactuated systems such as an inverted double pendulum or, Right: a cartpole with dry frictions on the joints. While both RDDP variants solve the task contrary to classical DDP, observe the improvement on the convergence criteria Q u ∞ for the adaptive noise scheduling (orange).

  M = 1 ε init = 1. , M = 4

Figure 7 :

 7 Figure7: Comparison of RDDP (blue) against classical DDP (orange) to plan the take-off of a drone. Top: Sequence for take-off achieved by RDDP which allows to reach the goal position marked with a blue sphere on the pictures. Bottom: On the opposite, unilateral contacts induce null gradients which lead to the failure of classical DDP.

4 Figure 8 :

 48 Figure8: RDDP allows executing a single-legged jump (on a vertical plane) towards a target position marked with a blue sphere. Top: The initially bends downwards and then extends to perform the jumping motion. Bottom: The cost J and the norm Q u are constant when using classical DDP (orange) while RSDDP (blue) achieves to lower the cost. The vertical green lines denote updates in the noise level used in randomized smoothing.

  backward in time around a reference trajectory x, leading to

		+4e3					+4e3
	0 (x, u + Δu) Q	-0.0002 -0.0001 0.0000 -0.0003	True dynamics	0 (x, u + Δu) Q	0.000 -0.004 -0.002 -0.006	True dynamics
		-0.0004	Smooth dynamics	-0.008	Smooth dynamics
		-1.00 -0.75 -0.50 -0.25 0.00 Δu	0.25	0.50	0.75	1.00	-1.00 -0.75 -0.50 -0.25 0.00 Δu	0.25	0.50	0.75	1.00

Acknowledgements

We warmly thank Robin Strudel for the fruitful discussions and feedback on the paper. This work was supported in part by L'Agence d'Innovation Défense, the French government under the management of Agence Nationale de la Recherche through the projects INEXACT (ANR-22-CE33-0007-01) and NIMBLE (ANR-22-CE33-0008), as part of the "Investissements d'avenir" program, reference ANR-19-P3IA-0001 (PRAIRIE 3IA Institute), and by the European Union through the AGIMUS project (GA no.101070165) and the Louis Vuitton ENS Chair on Artificial Intelligence.

Appendix

This section presents the physical principles commonly used for rigid body simulation with point contact to show where non-smoothness and nonconvexity arise during the computation of f in [START_REF] Duchi | Randomized smoothing for stochastic optimization[END_REF]. In general, we describe the state of a system with its generalized coordinates q ∈ Q ∼ = R nq and denote by v ∈ T q Q = R nv the joint velocity. The principle of least action states that the path followed by a dynamical system should minimize the action functional which induces the Lagrangian equations of motion:

where M ∈ R nv×nv represents the joint space inertia matrix of the system, C(q, v)v accounts for the centrifugal and Coriolis effects, and g is the generalized gravity. In the following, we express the problem in terms of velocities rather than acceleration, thus discretizing (.1) to

where C and g are evaluated explicitly at (q t , v t ) We note v f the free velocity which is defined as the solution of (.2).

Constrained dynamics for robotic systems with kinematic loops or that establish contacts with the environment can define these constraints implicitly via Φ(q) = 0 (.

where Φ : R nq → R m is the implicit constraint function of dimension m depending on the nature of the constraint. (.3) is then derived w.r.t time to express the constraint as a constraint on joint velocities in the sequel:

where c = Jv t+1 is the constraint velocity, J = ∂Φ ∂q is the constraint Jacobian and c * is the reference velocity of the constraint which is set to either model physical effects or stabilize the numerics. Such a constraint is enforced by the


K Jezzine

Audrey Gardahaut

Nicolas Leymarie

Sylvain Chatillon

Evaluation of ray-based methods for the simulation of UT welds inspection

Keywords: Weld Inspection, Dynamic Ray Tracing, Anisotropy PACS: 81.70.Ex

est destinée au dépôt

INTRODUCTION

Ultrasonic techniques are used in various industrial sectors (nuclear, petrochemical, etc…) for the inspection of bimetallic or austenitic welds. The simulation of ultrasonic inspection of welds is a major challenge because of their anisotropic and heterogeneous nature. Because of the polycrystalline structure of the weld, the detection and characterization of defects may be complicated as the beam shows disturbances. Indeed, the effects of skewing of the beam can be observed on experimental structural echoes. Various models have been proposed in the literature to simulate the propagation of ultrasonic waves in welds. Alternatively to finite element method [1], semi-analytical propagation models based on ray theory have been implemented in the CIVA software [START_REF]CIVA software platform for simulating NDT techniques[END_REF][START_REF] Lhémery | [END_REF]. In this case, the weld is described as a set of several anisotropic homogeneous domains with a given crystallographic orientation in each domain. The rays propagate in straight line within those domains. At each interface, the reflection and refraction coefficients are calculated. If the domains have dimensions smaller than the wavelength, the results are valid provided there are small variations of impedance between two adjacent domains [4]. If this condition is not fulfilled, a continuously varying description of the grain orientation in the weld is to be preferred. In this case, dynamic ray tracing method (DRT) allows dealing with the propagation in anisotropic and inhomogeneous media [START_REF] Cerveny | Seismic Ray Theory[END_REF]. In this paper, we discuss the applicability of both ray based models using either a piecewise description of the weld or a smooth description.

CONTEXT: UT SIMULATION OF WELD INSPECTIONS

As far as simulation of UT weld inspection is concerned, two possible approaches may be investigated. Numerical methods such as finite elements (FE) or finite differences (FD) consist in giving locally an approximate solution of the equation of motion. If complex phenomena such as caustics or surface wave are implicitly taken into account in the simulation, computation time and memory requirements may be prohibitive, especially for 3D computations. On the other hand, ray-based approaches give a high frequency approximation of the exact solution of the wave equation at a moderate computation cost. One of the advantages of ray based methods is the possibility to interpret results in terms of ray trajectories, which can be very useful. A ray based approach is implemented in CIVA. The wave-field emitted by a transducer is approximated by using the Rayleigh surface integral. The surface of the transducer is discretized and each contribution is approached asymptotically using ray computation. The impulse response is subsequently obtained by summing the contributions from each elementary source.

The input data for any simulation code must feature the weld geometry, the elastic properties of the welding material, attenuation values and the crystallographic orientation at any position inside the weld. By applying image processing techniques on a macrograph of the weld, either a 'piecewise description' or a continuously varying description of the weld can be obtained [START_REF] Gardahaut | Modelling tools for ultrasonic inspection of bimetallic welds[END_REF]. In the case of a piecewise description of the weld, that is to say the weld is described as a set of homogeneous domains as on Fig 1 .a, a 'straight ray tracing' model has to be used. In the case of a smooth description, the weld is described with a continuously varying crystallographic orientation and a dynamic ray tracing model has to be used. In both cases, the propagation model consists in solving the basic system of equations of ray theory for an anisotropic medium. The eikonal equation relates the travel time T of the wave front associated to a ray to the local wave speed c(x).

    2 2 1 x c T   (1)
Solving this equation allows the ray paths and travel time of the wavefront associated to the ray to be computed.

A second equation, called the transport equation allows the ray amplitude to be computed. The transport equation expresses the conservation of energy inside a ray tube.

  0 ) ( ) ( 2 ) ( 2       x x x x T A T A (2)

RAY METHOD APPLIED ON PIECEWISE DESCRIPTIONS OF WELDS Straight Ray Tracing Model

For a piecewise description of the weld, the rays travel in straight lines inside each homogeneous domain. At each interface, a computation of the reflection and refraction coefficient is performed and the propagation within the next domains is pursued. The computation of the ray amplitude is performed through matrix products expressing the evolution of a paraxial ray describing a ray tube or pencil [START_REF] Gengembre | [END_REF]. The results are valid if the domains have dimensions larger than the wavelength and present small variations of impedance between two adjacent areas.

Results of this straight ray tracing model, already implemented in CIVA, has been compared to results obtained with ATHENA, an elastodynamic finite element code developed by Electricité de France (EDF) [START_REF] Chassignole | Influence of the Metallurgical Structure of Austenitic Stainless Steel Welds on the Ultrasonic Non Destructive Testing[END_REF]. A weld has been described as a set of homogeneous orthotropic domains. Each domain shows a given grain orientation represented by an arrow on Fig. 2. One can notice that the variations of impedance between neighboring domains are not important so that the validity domain of the straight ray tracing model is respected. A very good agreement can be observed on 2D computation results on Fig. 2. In particular, the beam deviation inside the weld is predicted by both CIVA and ATHENA codes.

Limits Of Validity

The ray theory being based on a high frequency approximation, its applicability is limited by several qualitative conditions [4]. One of them specifies that the characteristic lengths of the model should be larger than the wavelength. As far as the simulation of ultrasonic propagation in welds is concerned, this means that the presence of small domains with significant impedance contrast should be problematic.

To verify this assumption, a numerical experiment has been performed in which two fictitious weld descriptions are considered. The two descriptions show approximately the same impedance contrast between two neighboring transversely isotropic domains but the characteristic length of a domain is about three wavelengths for the first description (Fig. 3) whereas it is about a wavelength for the second description (Fig. 4). Thus, if the first description respects the ray theory validity condition, the second description is supposed to possibly give inaccurate results.

Simulation results obtained with the straight ray tracing model, corresponding to the response of a side drilled hole located inside the weld have been compared with finite element results. Finite element results are obtained with a hybrid code developed by CEA and EDF that we use as a validation tool. This code uses a finite element box surrounding the weld that takes as input data the incoming ultrasonic field on the box boundaries. This incoming wave field emitted by a transducer is computed with CIVA and coupled with a finite element computation carried out by ATHENA inside the box containing the weld [START_REF] Mahaut | [END_REF].

As the condition that the characteristic dimension of the domains should be larger than the wavelength is not fulfilled in the second description considered, we expect the first description to give more reliable results than the second one. This is confirmed on the Bscans corresponding to the responses of the side drilled hole positioned inside the weld (Fig. 3 and Fig. 4). For the first description, there is a very good agreement between results obtained with the hybrid finite element code and the straight ray tracing model. On the other hand, significant discrepancies between both models are observed for the second description. This confirms that one must pay attention to the smoothness of the description before performing a ray based simulation.

RAY METHOD APPLIED ON SMOOTH DESCRIPTIONS OF WELDS Dynamic Ray Tracing Model

In the case of a smooth cartography of crystallographic orientation (Fig. 1b), the solution of the eikonal equation ( 1) is not as simple as in the case of the piecewise weld descriptions previously mentioned. Indeed, the ray trajectories are not straight lines any more. A differential system can be derived from the eikonal equation [START_REF] Cerveny | Seismic Ray Theory[END_REF]: T is the travel time, cijkl are the elastic constants at the position xi,  the density, Si are the components of the slowness vector, Pi the components of the polarization vector and Ve stands for the energy velocity. These two equations are coupled. The first one expresses the variation of the current ray position relatively to the travel time. It explicitly depends on the slowness. The second equation expresses the variation of the current slowness on the travel time which also depends on the current position xi.

This system of ordinary differential equations can be solved for compressional or transverse waves to obtain the ray trajectories and slowness along the ray. Knowing the initial conditions, the system (3) can be solved by standard numerical techniques such as Euler's method or Runge-Kutta's method [START_REF] Journiac | Simulation of ultrasonic inspection of composite using bulk waves: application to curved components[END_REF].

To solve the system, we need to know the elastic constants values and the value of their derivatives at any position inside the weld. The latter may be obtained either numerically if the weld is described as a cartography of crystallographic orientations as featured on Fig 1 .b or analytically if the orientations are given by a law as proposed for example by Ogilvy [11]. The ray trajectory is computed iteratively as shown on Fig. 5. At each time step, position and slowness are updated. Subsequently, the computation of ray amplitude is performed as a first approximation by considering the medium to be locally homogeneous between each time-step. This approximation is valid only for slowly varying anisotropy and requires the use of a small time step when solving the differential system of equations. A more rigorous approach would consist in solving a differential system for paraxial quantities from the transport equation [START_REF] Cerveny | Seismic Ray Theory[END_REF]. This would allow the use of high order numerical techniques to solve equation ( 3) that would involve a larger timestep.

First results

To validate our approach, we use the simplified model proposed by Ogilvy that assumes an analytical law to describe the variation of the crystallographic orientation along the weld. The V-shaped austenitic region modeled as transversely isotropic medium is surrounded on either side by ferritic steel modeled as a isotropic medium. The grain orientation inside the weld is represented on Fig. 6.

(4)

T is proportional to the tangent of the grain axis at the sloping edge. and D are geometrical parameters shown on Fig. 6 and  is a parameter driving the variation of orientation in the x direction. The material is considered to be a transversely isotropic steel.

Our dynamic ray tracing model has been compared with results from Connolly's PhD thesis [START_REF] Connolly | Modelling of the propagation of ultrasound through austenitic steel welds[END_REF] taking , T=1, and D=2mm. Contrary to dynamic ray tracing model, Connolly's ray tracing model uses virtual boundary parallel to lines of constant crystallographic orientation. One can notice on Fig. 6 a very good agreement between both models for a ray trajectory crossing the weld. Finally, a comparison between the wave field obtained using the dynamic ray tracing model and finite element results is shown on Fig. 7 on the same weld description. For the finite elements results, as in the previous section, the incoming wave field emitted by a transducer is computed with CIVA and coupled with a finite element computation performed with ATHENA inside the box containing the weld. There is a good agreement between both models even if slight discrepancies have to be investigated.

CONCLUSION

Two ray based approaches have been assessed for the simulation of ultrasonic inspections of austenitic or bimetallic welds, depending either the weld is considered as a set of homogeneous domains of different crystallographic orientations or as an inhomogeneous medium with a continuously varying orientation. The first one involving a straight ray tracing model is presently implemented in the CIVA commercial package. Its validity domain has been underlined by comparison with a 2D Finite Element code. A special attention must be given to avoid the presence of small domains compared to the wavelength with significant impedance contrast between neighboring domains. Alternatively, the second approach based on dynamic ray tracing has been implemented in the case of a smooth weld description. In this case, ray trajectories are computed by solving a system of differential equations and the ray amplitudes are computed by considering the medium as locally homogeneous. Comparisons of 2D ray trajectories with literature and comparisons of wave field with finite elements have proven the relevance of the approach. Current work is in progress in order to connect the dynamic ray tracing model on smooth cartographies of crystallographic orientations obtained from image processing techniques applied on macrographs.

FIGURE 1 .

 1 FIGURE 1. Macrograph of a bimetallic weld. Thanks to image processing techniques, either a) a piecewise description of the crystallographic orientation or b) a smooth description can be obtained.

FIGURE 2 .

 2 FIGURE 2. .(a) Piecewise description of a weld as a set of homogeneous domains with different crystallographic orientations represented by arrows. (b) Maximum particle velocity field for L0 inspection at 2.25 MHz obtained with a straight ray tracing model (CIVA) and (c) finite element model (ATHENA).

FIGURE 3 .

 3 FIGURE 3. (a) Fictitious weld description respecting the ray theory validity conditions. The weld is described as a set of homogeneous domains with different crystallographic orientation represented by arrows (b) Simulated Bscan (L0 wave inspection, 2.25 MHz) obtained with hybrid code CIVA/ATHENA (c) Simulated Bscan obtained with a straight ray tracing model.

FIGURE 4 .

 4 FIGURE 4. (a) Fictitious weld description that does not respect the ray theory validity conditions. The weld is described as a set of homogeneous domains with different crystallographic orientation represented by arrows (b) Simulated Bscan (L0 wave inspection, 2.25 MHz) obtained with hybrid code CIVA/ATHENA (c) Simulated Bscan obtained with a straight ray tracing model.

FIGURE 5 .

 5 FIGURE 5. Iterative computation of a ray trajectory obtained by solving equation (3).

FIGURE 6 .

 6 FIGURE 6. (a) Typical weld description proposed by Ogilvy [11]. Crystallographic orientations inside the weld are determined by equation (4). (b) Comparison of present dynamic ray tracing model with Connolly's ray tracing model.

FIGURE 7 .

 7 FIGURE 7. Comparison of maximum particle velocity field (L60 inspection, 2.25 MHz) obtained with finite elements (hybrid code CIVA/ATHENA) and the dynamic ray tracing model.


domain: sdv.ib.ima math phys

text:
Caroline Vienne
email: caroline.vienne@cea.fr

Julie Escoda

Anthony Touron

Marius Costin

Assessing the influence of CT acquisition parameters on flaw detectability through simulation

Keywords: CT simulation, probability of detection, NDT C = 4.7 C

, which integrates in particular a radiographic and computed tomographic (RT/CT) module, allowing the modeling of a complete chain for Nondestructive Testing (NDT) inspection with X-ray CT. In this paper, we present the multitude of parameters and possible configurations available for parametric studies and apply this simulation framework for assessing the influence of such parameters on the detectability of a given flaw inserted in a 3D object. A 3D detectability criterion is proposed and will be integrated in the next versions of CIVA to establish probability of detection curves.

CIVA-CT module presentation

The X-ray simulation code models the physics of photon-matter interaction in the range of 1 keV to 100 MeV, through the three most important processes: photoelectric effect, scattering (coherent and incoherent) and pair production. A fast computation of the attenuation is performed with the Beer-Lambert law, to which a Monte Carlo computation is combined in order to estimate the scattering phenomena. Distinct images are computed and fused into a final one, which integrates the influence of the X-ray source (size, spectrum and filtering), the attenuation and scattering of the beam, the photonic noise and the detector response (modulation transfer function, noise model).

The object to inspect is defined as a simple parametric object or using a CAD model and flaws of different shapes can be included and positioned inside it. Figure 1 presents a 3D view of the simulated configuration with a classical circular path and a projection without and with noise.

Simulation study

As a first step towards Probability of Detection studies of Computed Tomography using CIVA, we present here a parametric variation of different acquisition parameters of the CT setup (scene geometry, trajectory, X-ray source, etc…) and their influence on the detectability of different flaws.

Use case and variation parameters

We consider here a classical CT configuration for the NDT inspection of a complex aluminium part coming from Additive Manufacturing. To try to answer to the common question on the detectability limit, we integrate in this part several flaws of spherical shape and increasing size. Variations are applied on the location of the flaw in the part as well as on the main acquisition parameters (X-ray tube voltage and filter, number of projections, orientation of the part with respect to the X-ray system …).

One of the new functionality of CIVA 2021 concerns the application of the reconstruction algorithm (FDK) directly following the generation of the simulation results. Therefore, when running batches of simulation, it is now possible to access the 3D image in addition to the set of radiographic images. As a further step, we aim at integrating a detectability criterion applied to such 3D image, similar to the 2D "Rose" criterion already implemented in CIVA for radiographic configurations [START_REF] Tisseur | A Modified Detectability Criterion for Conventional Radiography Simulation[END_REF].

Defining a detectability criterion

The objective of a detectability criterion is not to automatically detect a flaw in a reconstructed image but rather to quantify the detectability of an existing flaw in this image. We use the knowledge of the flaw we have modeled, such as its shape and location in the 3D volume, to build our new criterion. The proposed criterion involves the contrast value (mean grey value difference between the flaw and its neighborhood), the noise and a weighting factor linked to the size of the flaw in terms of pixels (see Figure 2).

Prelimiray results

Figure 3 shows one slice of the CT reconstruction result around the region of interest of one inserted flaw of 300 µm diameter for three configurations corresponding to different X-ray source parameters. The detectability criterion C computed on these images is displayed for each configuration and gives a first estimation of the best X-ray settings for the detection of this porosity.

Conclusion

We present here the potential of using X-ray simulation for conducting first assessment of the detectability of a specific flaw in a part, based on the chosen X-ray settings but also for helping in the optimization of the X-ray parameters. A detectability criterion is proposed and applied on simulated results obtained with CIVA.

Figure 1 :

 1 Figure 1: Overview of CIVA CT scene (left) and visualisation of the reconstruction result (right).

Figure 2 :

 2 Figure 2: Visualisation of the simulated flaw in the reconstructed image (a) and quantitative values computed from the 3D image to compute the detectability criterion (b).

Figure 3 :

 3 Figure 3: CT reconstruction results obtained with three different source tension / filter, respectively 100 kV without filter (a), 150 kV with 0.5 mm Cu (b) and 250 kV with 0.5 mm Ta


Gerardo E Granados
email: gerardo.granados@cea.fr

Giorgia Colombera #
email: giorgia.colombera@centralesupelec.frfilippo.gatti@centralesupelec.fr

Filippo Gatti #

Roberto Miorelli
email: roberto.miorelli@cea.fr

Didier Clouteau
email: didier.clouteau@centralesupelec.fr

Generatives models approach on TFM data based on variational inference for multi-fidelity data generation

Fast and reliable numerical models are commonly used in nondestructive testing and evaluation (NDT&E) domain in order to design inspection procedure, design probes, interpret acquisitions, etc. In ultrasound testing (UT) full matrix capture (FMC) acquisitions are often post-processed via the total focus method (TFM) imaging technique to facilitate detection, classification and characterization tasks based on the analysis of amplitude signal maps (i.e., mono-channel "images"). In this context, the use of simulated data for training machine learning (ML) algorithm to be applied on experimental data has shown to be a promising schema for NDT&E applications [2] in view of its possible application to help to decision and automatic diagnostic tools.

Despite the very good accuracy obtained by the most advanced numerical simulation tools, the impact of measure uncertainties and experimental noise remain the biggest hundreds to overcome in order to unleash the prediction power of the most advanced deep learning algorithms. That is, the use of realistic (i.e., as closer as possible to experimental measures) simulated data is of paramount importance to develop accurate suitable for this task since the data samples of a given experiment are virtually unlimited.

The presented approach aims at developing deep-learning meta-model to generate realistic NDT&E results. A conditional generative adversarial network (c-GAN) is trained on a TFM database, composed of experimental and simulated images respectively. A reduced-order latent manifold z is inferred via an auto-encoder architecture, trained in an adversarial way. An encoder F is trained to link the latent manifold space Z with the X space (e.g: TFM image data-set), as shown on equation [START_REF] Donahue | Large scale adversarial representation learning in Advances in Neural Information Processing Systems 33[END_REF]. G generates a (c, z g ) conditional sample x g , where c is a class (simulation or experimental TFM image), and z g contains all the information of x projected on Z . Additionally, the discriminator D x (x, x g ) is a neural network who learns the differences between real x and fake images x g . Likewise, Dz judges if z g = F(x) and z are equally distributed. The generator uses the scores form D x as a loss to learn to generate realistic images, while the encoder use D z to learn to generate z g from x. Additional consideration on the losses are done to ensure bijective F and G.

z g = F(x) X g = G(z, c) (1)
Where,

x ∈ X ; X = R D z ∈ Z ; Z = R d c ∈ C ; C = R C with D >> d and C = 2 z ∈ Z ; Z = R d with z ∼ N (I I I, 0 0 0)
The ML model learns the differences between simulation and experimental data from TFM images in a non-supervised frame. The objective is to generate hybrid data spanning different degrees of fidelity, up to the highest one corresponding to the experimental samples.

The regression on the reduced space conditionally generates samples with different degrees of fidelity. In our network, the hybrid generation starts with a simulated TFM input image as content to then add the realistic style, similarly to [1][3]. A consistent addition of noise and variations on the hybrid data is expected.

The possibility to create unlimited data is now extended to realistic hybrid data, which is indistinguishable from reality since it contains the style of the experimental data set. Furthermore, the inferred latent space contains the information of the parameters (labels) of the simulated data-set. This space is explored to condition the hybrid data-set generation when creating new instances of TFM realistic images labeled by the corresponding parameters.

The present work evaluates the translation from a numerical input into a higher fidelity output by transferring experimental style to simulated data. The hybrid and experimental data similarities are measured to validate the newly generated images. Future work aims to use the hybrid data to enhance the data generation to be used for training classification and regression models in view of decision support systems.


Gerardo E Granados
email: gerardo.granados@cea.fr

Giorgia Colombera

Filippo Gatti

Roberto Miorelli

Didier Clouteau

Sébastien Robert

Giorgia -Colombera

Filippo -Gatti

Sébastien -Robert

Roberto -Miorelli

-Clouteau

Didier

Generative model approach on TFM data based on variational inference for muti-fidelity data generation

Keywords:

Generative model applied to ultrasonic testing (UT) on TFM imaging Peculiarity, challenges, and issues of UT inspection problems:

• Access to reliable simulations (semi-analytical approaches finite elements models, etc.)  model driven ML approaches are feasible • Lack of public realistic datasets (security and secrecy constraint issues)

 data driven ML approaches are very often not feasible • Operational conditions may impact the measurements (from the ML perspective)

 non-negligible impact on the acquisitions Conditional U-Net as generator on TFM data (supervised) Encoder: FiLM [4] + ST [5] + Convolution Encoder (conditioned)

•
[4] E. Perez, F. Strub, H. De Vries, V. Dumoulin, and A. Courville ( 2022)

The new per-channel statistics modify the relative importance of features for the subsequent convolution operation.

32@64X64

32@64X64

[5] M. Jaderberg, K. Simonyan, A. Zisserman, and K. Kavukcuoglu (2015).

[4]

Localisation Net conditioning conditioning conditioning [5] Commissariat à l'énergie atomique et aux énergies alternatives gerardo.granados@cea.fr

12 Conditional U-Net as generator on TFM data (supervised) Encoder: Instance Normalisation for fast stylization [6] [6] D. Ulyanov, A. Vedaldi, and V. The neural network has a parametric input that modifies the input image in realtion to the input parameters.

Can we sample the parameter space to generate new images related to new labels? The architecture is not limited by the shape of the parameter vector.  Parametric metamodeling allows to generate new "experimental" instances, even without new simulation instances.  Regression on label space provides direct labelling for new data.

Conclusion Perspectives

 cU-net as realistic data generator for data-set creation provides the tool to have enogh data to train deeper models:

GAN as variational inference tool to link two fidelity data domains (e.g.: simulation and experience)  GAN approach is pertinent when couples on each fidelity domain are not informed.

G



  Inspection of parts during manufacturing process  Characterization of materials -Aeronautics and aerospace  Multi-layer structure  Composite materials NDT&E and SHM modelling -Data generation for diagnostic, life time prediction -Materials characterization (Advanced Manufacturing) -Performance demonstration -Probe optimization -Defect detection, classification and characterization Exploitation of fast and reliable models -Dimensioning and defect characterization (form/type/criticality) -Confidence intervals on estimation (e.g., uncertainties). -Assessment of the "well-posed" nature of the inverse problem -Rapid Inversion (Online diagnostic)

  Fig 1 -CIVA configuration for simulated welded structure NDT inspection.

  Fig 6 -Conditional CNN U-net architecture for realistic image generation.

Fig 7 -

 7 Fig 7 -Example for image generation vs ground truth. Comparison of maximum pixel value and (x,y) position.

Fig 8 -

 8 Fig 8 -Example for input image vs image generation.

  'énergie atomique et aux énergies alternatives gerardo.granados@cea.fr Commissariat à l'énergie atomique et aux énergies alternatives -www.cea.fr dimensional graph representation of the data then optimizes a low-dimensional graph to be as structurally similar as possible Conclusions and perspectives  Multitask neural network: 9 TFM reconstruction modes are learned. New modes requires DA or new training.  The model takes as HF domain the experience. Different degrees of data fidelity on HF side can be considerer (e.g: FEM).  A novel application of ST and FiLM modules is presented to learn parametric subrogated model on ML frame.  Relation between NDT image features and the simulation parameter are learned by the model.

Net as generator on TFM data Results on Realistic data-set generation Surrogate model Testing realistic data generation and TFM parametric modelling

		Decoder
				Surrogate model
	XS test +YS test	cU-NET	XE test	XS train +YS test	cU-NET	XE test
			Instance Normalisation
				Learned bias and scale
	❑ Generate experience images ❑ Need new simulations		❑ Metamodelling just changing parameters ❑ No new simulation needed

Lempitsky, "Instance Normalization: The Missing Ingredient for Fast Stylization," Jul. 2016, doi: 10.48550/arxiv.1607.08022.

Principal layers:  Convolutive Neural Network  Fast stylization through Instance Normalisation Task:  Learns the difference between classes (mse+ffl loss)  Take into account uncertainties from experience  Generate new experience data (new instances) in a supervised way Commissariat à l'énergie atomique et aux énergies alternatives gerardo.granados@cea.fr Commissariat à l'énergie atomique et aux énergies alternatives -www.cea.fr Conditional U-

Net as generator on TFM data (supervised) Results on Test: Realistic data-set generation

	Conditional U-
	Echo amplitude cU-Net as multifidelity generator	SIMULATION	Xs test +Ys test XS test +YS test EXPERIENCE	cU-NET cU-NET	Xe test XE test
				SIMULATION		EXPERIENCE
		Echo localisation
	Backwall angle
	Celerity T	Xs test +Ys test	cU-NET	Xe test

Test images + test param. Test mean Test images + test param. Test mean Test images + test param. Test mean Conditional U-cU-Net as multifidelity generator Sampling from simulation parameter space Examples of points is test set Examples of points is training set

Net as generator on TFM data (supervised) Results on Test: Realistic data-set generation

  Net surrogate model -regresion in parameter space(labels)

				New generated dataset
	XS test +YS test	cU-NET	XE test	SIMULATION	EXPERIENCE
	XS train +YS test	cU-NET	XE test
		New simulated samples

A point represents: simulated image + parametares vector cU-

Net as generator on TFM data (supervised) Results on Test: Surrogate model

	regression Conditional U-fixed Echo localisation	Backwall angle	XS train Celerity T XS train Echo amplitude

Examples of points to model

Examples of points is training set

Test images + test param. Train images + test param. Test images + test param. Train images + test param. Test images + test param. Train images + test param.

Conditional U-

Net as generator on TFM data (supervised) Generation error on reconstruction parameters regression

Commissariat à l'énergie atomique et aux énergies alternatives gerardo.granados@cea.fr 20 fixed Conditional U-

Net as generator on TFM data (supervised) Generation by reconstruction parameters regression Modelling point Examples of points is training set

	Backwall angle
	Celerity T

Commissariat à l'énergie atomique et aux énergies alternatives gerardo.granados@cea.fr 21 Test images + test param. Train images + test param. Test images + test param. Train images + test param. Test images + test param. Train images + test param.

de votre attention Contact mail: gerardo.granados@cea.fr Thanks you for you attention

	Loss	Metric	Metric
		Gerardo E. Granados Validation Training
	Loss	4h sur GPU NVIDIA Quadro RTX 6000 Batch 128 22 GB on GPU	Dn GAN architecture F	Dx
			dimanche, novembre 20, 2022

Commissariat à l'énergie atomique et aux énergies alternatives gerardo.granados@cea.fr Commissariat à l'énergie atomique et aux énergies alternatives -www.cea.fr Merci dimanche, novembre 20, 2022 Commissariat à l'énergie atomique et aux énergies alternatives gerardo.granados@cea.fr 26 Training: best model dimanche, novembre 20, 2022 Commissariat à l'énergie atomique et aux énergies alternatives gerardo.granados@cea.fr 27 Test images + test param. Train images + test param. Test images + test param. Train images + test param.

Metric

This project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 800945 -NUMERICS -H2020-MSCA-COFUND-2017 26th This project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 800945 -NUMERICS -H2020-MSCA-COFUND-2017 IUTAM DDMech 2022 -Paris