G Dif-Pradalier
email: guilhem.dif-pradalier@cea.fr

Ph Ghendrih

Y Sarazin

X Garbet

V Grandgirard

Y Munschy

R Varennes

L Vermare

Y Camenen

F Widmer

INTERPLAY BETWEEN CORE, EDGE AND SCRAPE-OFF LAYER IN TURBULENT MAGNETISED PLASMAS Lessons learnt from an information theoretic measure, the "Transfer Entropy" to assess causality in nonlinear plasma dynamics

The present manuscript is meant to provide a companion pedagogical approach to an information theoretic measure, the "Transfer Entropy" method, useful to assess causality in complex (time series) datasets. This manuscript describes useful basics for the forthcoming discussion on the causal chain of events that leads to the onset of a spontaneous and stable edge transport barrier in flux-driven gyrokinetics when core, edge and scrape-off layer regions self-consistently interplay.

INTRODUCTION

For over three decades, the observation of rapid core confinement improvement upon favourable modifications of edge operating conditions has been a nagging source of puzzlement for experimentalists investigating conditions for a lasting source of fusion energy in tokamaks. The transport properties of drift-wave turbulence and the interaction of the confined plasma with its material boundaries have long been recognised as essential to the resolution of this conundrum. Key aspects of the turbulent dynamics in the plasma edge are poorly quantified, owing to the disparity of temporal and spatial scales and the inadequacy of performing scale separations. A forthcoming paper [1] will be detailing three key results that arise when a material boundary is introduced in flux-driven gyrokinetics, effectively interconnecting core, edge and Scrape-Off Layer (SOL) regions. We show that (a) turbulence is not only locally driven by local gradients but nonlocally controlled by fluxes of turbulence activity, primarily though not exclusively borne at the edge. This `nonlocal' influence is mediated through vortex-flow localised interactions near the material boundaries and has two major consequences: (b) the nonlinear destabilisation of the linearly stable edge, providing a possible resolution for the so-called `shortfall' conundrum and (c) the spontaneous emergence of a stable and localised transport barrier at the closed/open field line transition, possible prelude to the formation of a pedestal. In the present manuscript, we focus on the latter point and discuss a general technique, the so-called ``Transfer Entropy" imported from information theory that provides a useful tool to assess the causal chain of events that leads to the onset of this stable edge transport barrier. This point is of course of basic interest: it allows to precisely test theoretical frameworks from the perspective of primitive equations and as such provides important guidelines for reduced modelling.

CHAIN OF CAUSALITY AND THE ``TRANSFER ENTROPY" (T.E.) METHOD

Unravelling causal chains of events in complex datasets is challenging yet especially rewarding when the dataset stems from either experimental measurements or has been computed from the primitive equations. Lowfrequency microturbulence in fusion plasmas is appropriately described by gyrokinetics. GYSELA [2] models ions and trapped electrons gyro-kinetically in the core and edge regions as well as the closed/open field line transition and the Scrape-Off Layer (SOL) through introduction of a simplified penalised limiter [3] mimicking the role of a heat and momentum sink. Core, edge and SOL are treated on an equal footing. As we find strong interplay between these distinct regions and the spontaneous development of a stable edge transport barrier, the flux of information across the edge is complex and typically not merely locally influenced. In other words, its evolution is not only subject to variations of the local free energy, ie. the mean local gradients. Investigating techniques that may allow to disentangle the causal chain of events is very desirable and would provide quantitative insight regarding the dominant feedback loops involved.

Causality detection in information theory is actively debated [4]. We will focus here on a simple nonlinear extension of the Granger causality, introduced by Schreiber [5] as the ``Transfer Entropy" (TE) method and investigated in magnetised plasmas by Van Milligen et al. [6] and Nicolau [7]. Let us state at this point that our goal here is not to provide a discussion on the concept of causality itself but rather put to use one of its possible manifestations -defining causality let alone inferring it is indeed still lively discussed [8]. The idea behind TE is simple yet powerful: let's consider a time series (xi) with 0  i  n, of realisations of observable X. If one can better predict its next realisation xn+1 using additional data from another time series (yj) with 0  j  n of observable Y, then we shall say that ``Y transfers information (i.e. causes) X", or more accurately as "Y forecasts X", which our definition of causality here. This idea is quantified measuring deviation of transition probabilities from independence, i.e. from a stationary Markov process. In its simplest expression, as used here, if processes X and Y are independent, then the following generalised Markov property holds for all 0  k  n: p(xn+1|xn-k,yn-k) = p(xn+1|xn-k). The standard notation for conditional probabilities is used here: p(a|b) is the probability of a knowing b. If now processes X and Y are not independent, the ratio of these two transition probabilities provides a measure of how much information they may share, i.e. how much knowing values in Y in addition to past values in X may help to better evaluate next-step xn+1. This idea leads to the following definition of the Transfer Entropy (TE) from process Y to process X:

where k is thus a time lag and represents the k-past of times series X and Y. The summation process is detailed below, in Eq.( 3). The TE can equivalently be recast as a conditional mutual information and represents the additional amount of information that must be added to adequately represent the studied process p(xn+1|xn-k,yn-k) with respect to its reference Markov process p(xn+1|xn-k). In the absence of information flow from Y to X, the logarithm vanishes as the state of Y has no influence on the transition probabilities of X -another way of saying that Eq.( 1) indeed quantifies a "transfer" of information. It also follows that TE is directional, i.e. TEYX  TEXY, which allows to infer causality between processes X and Y. It is also to be noted that TE is quite generic and displays interesting properties: it is independent of the relative magnitudes of signals X and Y; it may apply to either linear and nonlinear regimes; it is easy to evaluate (directly in real space rather than in Fourier space) and typically less demanding in terms of statistics than bispectral techniques. -k). In order to have sufficient statistics, the bin size "" is typically chosen as =2 or =3, depending on the available length of the time series (the longer the times series, the larger  can be). Eq.( 3) now clarifies the way to perform the summation in Eqs.( 1) and ( 2).

Here we have also introduced the additional exponent   1, which effectively represents a nonlinear threshold: low probabilities will be further reduced and higher ones amplified. In the following, we take =1. In a complex setting, information may flow both ways, from Y to X and inversely. It is thus especially useful to define the net transfer entropy X,Y(TE)[k] = TEYX[k]  TEXY[k], which provides the net flow of information between processes X and Y, at timelag k. This is the quantity that we now discuss in further detail.

MANUFACTURED SOLUTIONS AND PROOF OF CONCEPT

TE is found to be weakly dependent on a few choices that the user must perform: in the following, we discretise the pdf in Eq.( 2) using  d =2 d bins, with d the dimensionality of the pdf. The magnification exponent  is set to unity and X and Y are discretised at the same rate and chosen to enter the TE calculation with zero temporal mean. In order to provide a pedagogical introduction to the interpretation of transfer entropy graphs, let us detail the following manufactured solutions. In Fig. 1 we show the example for a random time series X. Y is constructed such that it exactly corresponds to X, with a forward in time delay of 9 units. As only past information may be shared between X and Y and as time lag k tests for similarities in shape of processes X and Y, TE(YX) should be vanishing for all time lags for a random time series and display a marked peak around time lag k=9 for the TE(XY). The net TE simply states that for all the 50 time lags considered here, the only meaningful flow of information can go from XY, with a time lag of 9. Similarly, in Fig. 2, focussing on the left panel, the blue peak at time=31 from process Y may transfer information to the two peaks of process X at times=40 and 140, the former with a time lag=9, the latter with a time lag=109. Inversely, considering the reverse TE(XY), the peak at time=40 may only transfer information towards peaks at time=69, 131, and 169 and not to the previous in time=31 peak. Another typical situation, common in nonlinear dynamics is illustrated in Fig. 3, simulating predator-prey behaviour from the Lotka-Volterra equations:

with parameters a = 1, b = 0.1, c = 0.075 and d = 1.5. The typical cyclic behaviour between predator and prey is readily seen through the Lissajoux display (bottom right panel) of process X (the prey) as a function of the predator Y. Where correlation analysis would merely pick up the periodicity of the predator and prey oscillations, this example illustrates the usefulness of the net Transfer Entropy X,Y(TE) (dashed green, lower left panel) as it provides guidance regarding the flow of information, as the nonlinear system evolves. Individual TE curves provide knowledge on typical waiting times for the system to reach a given state. For instance, TE(XY) [black, lower panel] tells us that starting from a large number of preys (X1), the system will need about k~6 time steps to reach a state with a large number of predators (Y1). Inversely, from TE(YX) starting from a large predator state (Y1), the system will require k~25 time lags to be able to sustain again a large number of preys (X3). Of course, in a complex system information keeps flowing between preys and predators during the whole oscillation period. Net TE X,Y(TE) captures inflexion points relevant for the dynamics. Thus, peak A corresponds to the situation where an increase in prey numbers or a large abundance of preys triggers a large predator growth with time lag~6. In stage B, predators in large numbers now eat more preys than their birthrate manages to compensate and decimates their numbers. During stage C, preys in fewer numbers trigger a decline in predator numbers with a time lag ~18. The low predator population in stage D now triggers with a time lag ~25 the increase in prey growth and the system loops again.

T.E. APPLIED TO THE FAR-EDGE REGION WHERE A TRANSPORT BARRIER DEVELOPS

We now systematically compute the TE algorithm to actual time series from large-scale GYSELA flux-driven computations with limiter boundary conditions in the last 5% inside the last closed flux surface (LCFS) where the spontaneous onset of a persistent transport barrier is observed. A vorticity equation [9] can be inferred from the primitive gyrokinetic equations including ExB drift and finite Larmor radius effects and can be recast at leading order as follows:

with r and  respectively the minor radius and poloidal angle and  denoting an average over toroidal angle . We have applied the TE algorithm to many of the possible permutations of quantities in Eq.( 6) and especially to the following set:

We focus here on the example where `X' is the vorticity and `Y' a "field advection" [first and fourth terms in the bracket of Eq.( 11)]. This latter term is somewhat uncommon; physically it corresponds to a poloidal advection by diamagnetic currents of poloidal anisotropies of the radial electric field Er. Each field is radially and poloidally averaged respectively over the last 5% of the plasma volume and /3 region symmetric about the limiter. The net transfer entropy X,Y(TE) between vorticity and poloidal tilt is displayed in Fig. 4. For two periodic fields, the investigated time lag should have an upper bound which, as can be intuitively understood from Figs. 2 and3 is the shortest of the two fields' periods. Here, none of X or Y are strictly periodic; the upper bound kmax for time lag k is chosen as the shortest "period" that is inferred from the dominant Fourier mode in each field. Upper bound kmax could also be inferred [6] As detailed above, net TE X,Y(TE) provides a quantitative measure of the direction and intensity of information flow. These facts are conveniently summarised diagrammatically (right panel of Fig. 4): (i) the direction of information flow (direction of the arrow) is given by X,Y(TE), either positive or negative whilst (ii) 𝐴 >0 [resp. 𝐴 <0 ] is a proxy for the amount of information transferred from YX [resp. XY] during one "period". In the example displayed in Fig. 4, 𝐴 >0 𝐴 <0 ⁄ ≈ 100 11 ⁄ ; the thickness of each arrow relative to the next in the diagrammatic representation is set to follow the same ratio so that respective thickness and direction of connecting arrows provide a visual representation of causality and amount of information actually transferred between considered fields. This analysis has been systematically performed, for all combination of terms in Eqs.( 6)-(10). Unexpectedly, the diamagnetic advection of vorticity [2 nd term in the bracket of Eq.( 6)] also appears as a key contributor to vorticity build-up in the early onset of the edge transport barrier. These facts certainty point towards an important role for diamagnetic currents, i.e. poloidal distribution of pressure fluctuations that stem from spontaneous symmetry breaking induced through plasma-boundary interplay. These results are discussed in a forthcoming paper.

CONCLUSIONS AND OUTLOOK

The present manuscript has focused on laying out useful basics to assess causality in time series analysis, through the use of the so-called ``Transfer Entropy" method, a nonlinear extension of the Granger causality. Practical use of this method, applied to the nonlinear onset of an edge transport barrier in advanced gyrokinetic computations of tokamak turbulence will be reported shortly elsewhere [1].

  Practically, TE is evaluated expressing the conditional probabilities as joint probabilities, eg. p(xn+1|xn-k, yn-k) = p(xn+1,xn-k,yn-k)/p(xn-k,yn-k) and computing the 4 multidimensional probability density functions (pdfs): as a function of time delay k and normalised such that 0  TE  1. The 4 pdfs in Eq.(2) result from a binning process of times series X and Y, such that Eq.(2) is estimated in practice as: where p 3d , p 2d xx, p 2d xy and p 1d are the discretised versions of respectively p(xn+1,xn-k,yn-k), p(xn+1,xn-k), p(xn-k,yn-k) and p(xn

Figure 1 .

 1 Figure 1. (Top left and right) Time traces of the two processes X and Y over which TE is computed. For visualisation purposes only, Y is offset by its maximum value. TE (YX) is represented (lower left) in purple and TE (XY) in black (lower right) as a function of time lag k. The net Transfer Entropy X,Y(TE) is displayed as well (dashed green).

Figure 2 .

 2 Figure 2. Time traces of processes X and Y and associated TE calculations; colour coding and legends are as in Fig.1.

Figure 3 .

 3 Figure 3. TE applied to the Lotka-Volterra equations; colour coding and legends are as in Fig.1.

  from a time-delayed self-mutual information; both methods render comparable estimations: kmax ~ 4000 Ωci, with Ωci denoting the ion cyclotron frequency. Let us also introduce 𝐴 >0 = ∫ max[0, ∆ 𝑋,𝑌 (𝑇𝐸)]𝑑𝑘 𝑘 𝑚𝑎𝑥 0 and 𝐴 <0 = ∫ min[0, ∆ 𝑋,𝑌 (𝑇𝐸)]𝑑𝑘 𝑘 𝑚𝑎𝑥 0 the areas of respectively positive (yellow area, Fig.4, centre) and negative net TE (purple area, same figure).

Figure 4 .

 4 Figure 4. Causality, as inferred from the application of the TE algorithm to the actual time series of vorticity and poloidal tilt (left panel), from flux-driven gyrokinetic computations near the LCFS, at inception of an edge transport barrier. The net TE is show in the central panel, the net flow of information reversing between positive and negative areas. The arrows on the right panel are a convenient way to synthetise this data, their direction indicating the flow of information and their width (resp. 100 vs 11) the amount of information actually transferred between both quantities.

ACKNOWLEDGEMENTS

The authors want to thank Ch. Passeron for continued help with the development of GYSELA and acknowledge stimulating discussions with P.H. Diamond and participants at the 2019 Festival de Théorie in Aix-en-Provence. The work has been carried out within the framework of the EUROfusion Consortium and was supported by the EUROfusion Theory and Advanced Simulation Coordination (E-TASC) initiative under the TSVV (Theory, Simulation, Verification and Validation) "L-H transition and pedestal physics" project (2019-2020). It has also received funding from the Euratom research and training program. The project has received funding from the European Union Horizon 2020 research and innovation program under grant agreement No 824158 (EoCoE-II). The views and opinions expressed herein do not necessarily reflect those of the European Commission. This work was also granted access to the HPC resources of TGCC and CINES made by GENCI, and to the EUROfusion High Performance Computer Marconi-Fusion. We acknowledge PRACE for awarding us access to Joliot-Curie at GENCI@CEA, France and MareNostrum at Barcelona Supercomputing Center (BSC), Spain.


Anatoliy Zavdoveev
email: avzavdoveev@gmail.com

Valeriy Poznyakov

Thierry Baudin
email: thierry.baudin@universite-paris-saclay.fr

Massimo Rogante

Hyoung Seop Kim
email: hskim@postech.ac.kr

Mark Heaton
email: mark.heaton@antsltd.com

Yuriy Demchenko

Victor Zhukov

Mykola Skoryk
email: mykolaskor@gmail.com

Effect of heat treatment on the mechanical properties and microstructure of HSLA steels processed by various technologies

Keywords: high-strength low-alloyed steels, heat treatment, mechanical properties, intensive grain growth, microstructure, electron backscatter diffraction, thermo-mechanical control process

ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

High-strength low alloyed (HSLA) steels are actively used in bridge construction, oil and gas offshore platforms, the supports of wind generators, ships, high-pressure vessels, as well as in the manufacture of railway transport [START_REF] Meester | The weldability of modern structural TMCP steels[END_REF][START_REF] Ufuah | Elevated Temperature Mechanical Properties of Butt-Welded Connections Made with High Strength Steel Grades S355 and S460M BT -Design, Fabrication and Economy of Metal Structures[END_REF][START_REF] Lee | Evaluation of high strength TMCP steel weld for use in cold regions[END_REF]. The important task of reducing metal consumption can be solved by increasing the strength of rolled steel. There are several ways to increase the strength of low carbon steels. The first is an increase in carbon concentration, which has limitations associated with decreased ductility and deterioration in weldability. The second way is additional doping with elements that offer solid solution hardening (e.g., Mn and Si). This enables an increase in the strength of metallic materials. However, to ensure ductility, the use of normalizing heat treatment is additionally required. An increase in the concentrations of Mn and Si, as in the first case, also leads to a deterioration in weldability [START_REF] Medina | Effects of v and Nb on static recrystallisation of austenite and precipitate size in microalloyed steels[END_REF]. Microalloying with carbide and nitride-forming elements (Nb, V, Ti) [START_REF] Fossaert | The effect of niobium on the hardenability of microalloyed austenite[END_REF][START_REF] Nazarov | Comparison of weldability of high-strength pipe steels microalloyed with niobium, niobium and vanadium[END_REF][START_REF] Reddy | Cold cracking studies on low alloy steel weldments: effect of filler metal composition[END_REF] is limited by the increased cost of rolled steels. The third strengthening method is to apply heat treatment after rolling (e.g., quenching in oil, high-temperature tempering, or normalization). This, though, significantly increases the cost of manufacturing rolled steel [START_REF] Ufuah | Elevated Temperature Mechanical Properties of Butt-Welded Connections Made with High Strength Steel Grades S355 and S460M BT -Design, Fabrication and Economy of Metal Structures[END_REF]. The fourth is by obtaining a fine-grained structure of rolled steels using the method of the thermo-mechanical control process (TMCP) [START_REF] Lee | Evaluation of high strength TMCP steel weld for use in cold regions[END_REF].

The most widespread technology used in manufacturing steel construction is welding [START_REF] Demarque | Evaluation of the effect of the thermal cycle on the characteristics of welded joints through the variation of the heat input of the austhenitic AISI 316L steels by the GMAW process[END_REF][START_REF] Lee | Correlation of microstructure and fracture properties in weld heat-affected zones of thermomechanically controlled processed steels[END_REF][START_REF] Natschläger | Thermomechanically Rolled Heavy Plates for PenstocksThermomechanisch gewalzte Grobbleche für Druckrohrleitungen[END_REF][START_REF] Brozda | The weldability of thermomechanically rolled S460ML steel[END_REF]. To create reasonable welded structures operating under dynamic and variable loading conditions, more than the chemical composition and structural state had to be considered. The behavior of the various steels under thermal treatment should be taken into account also. Namely, the thermal cycle of the welding makes the most vulnerable effect on the welded metal in the zone next to the fusion line. This area is called the heat-affected zone (HAZ) and is characterized by the quenching structure. On the other hand, in the process of manufacturing welded metal structures, heating can be used to temperatures in the range from 580 to 950 °C. High-temperature tempering (T = 580-620 °С) is used to relieve welding stresses. Heating to temperatures of 680-750 °C is carried out for thermal straightening of the weldments. In some cases, for example, during hot stamping, the welded plates are heated to a temperature in the range of 900-950 °C.

The mechanical properties can be lost due to the above-mentioned heat treatment of the steel (e.g., the welding thermal cycle or heat treatment associated with relatively slow heating/cooling rates). The literature states that the steel obtained using TMCP cannot be heated above 600 °C because the microstructure formed during the TMCP process after such heating cannot be restored [START_REF] Lee | Correlation of microstructure and fracture properties in weld heat-affected zones of thermomechanically controlled processed steels[END_REF][START_REF] Pashinska | Effect of rolling with shear technology on structure, properties and plasticity of low-carbon steel[END_REF][START_REF] Krasnowski | Effect of heat treatment of S420MC steel joints on their mechanical properties and fatigue strength[END_REF]. However, it was not possible to uncover a detailed justification with physical reasons for this. Gorka [START_REF] Górka | Weldability of Thermomechanically Treated Steels Having a High Yield Point[END_REF][START_REF] Mehditabar | Effect of Heat Treatment on the Properties of AlCu26Si8[END_REF] considered the thermal stability of high-strength steel (700 MPa) obtained using TMCP up to temperatures of 1300 °C. He proved that within a certain temperature range, there is a significant change in the structure and the hardening phase of the investigated steel. The question of the physical cause of these changes, however, remains unsolved.

The purpose of the present work is to study the effect of high-temperature heating on both the mechanical properties and microstructure of HSLA rolled sheet steels obtained using normalization and TMCP.

Experimental

Materials and preparation methods

The HSLA steels of grades S355J2 and S460M with similar chemical compositions for the main alloying elements, such as C, Mn, Si, P, and S, were chosen. It was manufactured according to the standard routine at the metallurgical plant: S355J2 is received in normalized condition and S460M after TMCP. The requirements for the chemical composition of these steels are given in Table 1. As noted earlier, the maximum value of alloying elements is specified in EN10025-4, and the minimum value is not regulated. This causes a significant difference in the chemical compositions of the same steel grades. In this regard, the actual chemical compositions of the investigated steels measured using an optical emission spectrometer are given in Table 1. For the tensile test of the steel, the cylindrical samples (three samples for each measurement) with a 6 mm diameter for their working parts were mechanically manufactured (type II following ISO 6892:1998). The tests were carried out at an ambient temperature. For the impact strength determination, the Charpy impact tests were carried out at +20 and -40 °C (according to the ASTM A370). The mechanical properties, namely the yield strength (YS), ultimate tensile strength (UTS), plasticity (δ, elongation for failure), and the impact strength (KCV) at -40 °C for the investigated steels in the initial state, are given in Table 2. Here, normalization is a condition when the steel plates are heated to 50 °C above their upper critical point (Ac3) and are kept at it for a short time before being cooled in air.

TMCP is a condition caused when steel plates are manufactured with deformation in the intercritical temperature zone (Ac1-Ac3) followed by accelerated cooling. Such technologies as the rolling of steel sheets provide an opportunity to reach a higher level of strength with a conserved CE value (Table .1). This is a critical factor because CE is strongly connected with the formation of the cold cracks during welding (i.e., with growing CE, there is an increasing probability of cold cracks). Thus, in addition to the chemical composition, it is also essential to indicate supply conditions. Depending on the method of producing the steel, the weldability and behavior during the subsequent heat treatment will differ [START_REF] Górka | Assessment of Steel Subjected to the Thermomechanical Control Process with Respect to Weldability[END_REF].

Heat treatment

As a result of the thermal cycle of welding the microstructure and properties of the initial metal are degraded. The most vulnerable point is the area of the coarse grain HAZ, whose size does not exceed 1.5-2 mm. Therefore, studies have been carried out on bulk samples to simulate the microstructure of this zone. For this purpose, a particular installation was used [START_REF] Zavdoveev | Weldability of S460M high strength low-alloyed steel[END_REF]. The samples were heated and cooled according to the thermal cycle of the welding, i.e., with the heating rate of 150 °C/s up to 1200 °C, and cooling in the temperature range of lower austenite stability (600-500 °С) at a rate of 25 °С/s.

For the thermal stability investigation of the selected HSLA steels, heat treatment was applied. The samples (initial state) were heated for 1-hour in a muffle furnace to 630, 750, and 950 °C before cooling in air. The simulated HAZ metal samples were heated to 950 °C, kept, and cooled in the same way as previous ones.

Structure characterization

The preparation of the metallographic specimens for microstructural studies was carried out according to standard methods using grinding papers of different roughness (P240, P400, P600, P1200, P2000). Final polishing was carried out on a diamond suspension with a polishing particle size of 1 μm. To reveal and identify the morphology of the microstructure of the specimen, etching with a 4% solution of HNO3 in alcohol was conducted. The exposure time of these specimens was 4 s, followed by washing and drying.

The average grain size was evaluated by applying the linear intercept method. The chemical composition of the structural components was measured using EDS (Oxford Instruments).

The crystallographic orientation of the crystallites was determined using the EBSD technique [START_REF] Zavdoveev | Preparation of metallographic specimens for electron backscatter diffraction[END_REF][START_REF] Vander | Metallographic Specimen Preparation for Electron Backscattered Diffraction Part I[END_REF]. For the formation of high-quality diffraction patterns, the surface relief should be minimal. Thus, the specimens for EBSD were prepared using standard procedures for metallographic investigation, except for the finishing stage. A silica-colloidal solution of 0.04 µm was applied for polishing. MD-Nap cloth was used, the rotation frequency was 200 rpm, the pressing force of thin sections was 27N, and the polishing time was 3-5

minutes. An electron-scanning microscope Tescan Mira 3 LMU equipped with a Nordlys detector and HKL Channel 5 software was used for EBSD analysis of microstructural changes. At least 1500 grains of the specimen were analyzed to obtain a representative sample. The procedure was to apply a magnification of (x600-700) over a scanning area of 150x100 µm 2 at a scanning step of 0.5 µm (at least five measuring points per grain), binning 4x4. The degree of indication was at least 80%.

The geometrically necessary dislocation (GND) density was estimated using the EBSD data. Since GNDs distort the crystal lattice, low angular misorientation changes can be measured and quantified with the help of the EBSD. For this purpose, local average misorientation (LAM) maps were constructed. These maps represent local orientation gradients. Thus, low LAM values indicate grains with relatively low GND density (depicted by blue) and high LAM values -increased GND density (represented by green and red) [START_REF] Karmakar | Effect of Starting Microstructure on the Grain Refinement in Cold-Rolled Low-Carbon Steel During Annealing at Two Different Heating Rates[END_REF][START_REF] Ma | Microstructure and Hardness Evolution during Deformation near Ae3 in a Cr-Mn-Ti Gear Steel[END_REF][START_REF] Beausir | Disclination densities from EBSD orientation mapping[END_REF]. In the samples processed by a thermal cycle of welding, the parent austenite grains were identified using the ARPGE program [START_REF] Cayron | ARPGE : a computer program to automatically reconstruct the parent grains from electron backscatter diffraction data[END_REF]. Based on orientational relations (Kurdjumov-Sachs relationship), this program allowed restoring the parent austenite from the bainite structure.

Results

Effect of heat treatment on the mechanical properties

The experimental results have shown that the strength and plastic characteristics (Figures 1 and2 Noticeable changes in the mechanical properties and impact strength of the S460M steel are observed only when heated to 750 °C. A yield strength decrease of 10% is observed, and the plasticity remains almost unchanged, while the impact strength rises

somewhat. An increase in the heating to 950 °C leads to a more substantial reduction in the yield strength of the S460M steel, by almost 25%. Its impact strength, at the same time,

decreases by 30% at the samples tested at 20 °C, and by 4-6 times at -40 °C (Figure 1).

Unlike the steel grade S460M, the steel grade S355J2 is more thermally stable.

Regardless of the heating temperature, it keeps the values of the mechanical properties (YS, UTS, δ) within the limits of 6% error, compared with the initial state (Figure 2). The impact strength, as in the case of the S460M steel sample, after heating to 750 °C, regardless of the test temperature, the impact strength slightly increased. With a further increase in heating temperature (up to 950°C), the impact strength values practically return to the initial state for the S355J2 steel. The mechanical properties of the simulated HAZ metal (rapid heating/cooling regime) are presented in Figure 3. It is clearly seen that such heat treatments lead to significant increase of the YS and UTS while impact strength drops drastically. Such behavior of the mechanical characteristics is widespread for the quenching processes.

Due to the fact that the most remarkable changes in the mechanical properties of S460M steel occurred after it was heated to 950 °C, this heat treatment temperature was used in further studies, which were carried out for the HAZ. It has been established that the strength characteristics of the HAZ metal drop after heating to 950 °C for both of the investigated steels, and that the plasticity increases significantly (Figure 3). The ultimate tensile strength of the HAZ metal of the S355J2 steel after exposure of the samples for 1 hour at 950 °C (Figure 3f) remains higher than in the initial state. It is important to note that the strength and plasticity characteristics of the HAZ metal for both of the investigated steels after exposure to 950 °C are similar. High-temperature heating (950 °C) of the HAZ metal leads to an increase in the impact strength of the S355J2 steel, and a slight decrease in S460M steel (Figure 3d,h).

Effect of heat treatment on the structure

The microstructure of the considered steels in the initial state is ferritic-pearlitic with average grain sizes of ~20 µm for S355J2 steel and 10 µm for S460М steel (see Figure 4).

In the case of steel production through the normalization procedure (S355J2 steel), the microstructure is characterized by a chaotic distribution of orientations. The steel produced through TMCP (S460M steel) has a significantly pronounced texture with a predominance of directions <101> (// normal direction to the rolling plane) as indicated by the EBSD orientation distribution maps and poles figures (Figure 5). For BCC lattice metals {101} crystallographic planes are most favorable for the dislocation slip. The pole figures (Figure

Discussion

The obtained results indicate that for the investigated steels in the initial state, the maximum changes in the mechanical properties and microstructure take place during hightemperature heating to 950 °C. In contrast, the HAZ metal is less susceptible to such influence.

To explain this effect, the factors affecting the mechanical properties of the steel were analyzed. One of the main mechanical properties is Yield strength (σу), which can be represented by a sum of terms [START_REF] Larzabal | Effect of Microstructure on Post-Rolling Induction Treatment in a Low C Ti-Mo Microalloyed Steel[END_REF][START_REF] Berdnikova | Structure and Properties of Laser-Welded Joints from High-Strength Steels[END_REF][START_REF] Mayo | On the characterization procedure to quantify the contribution of microstructure on mechanical properties in intercritically deformed low carbon HSLA steels[END_REF]:

σу = σ0 +σρ+σss+σppt+σgs, ( 1
)
where σ0 is the friction stress of the crystal lattice, σρ is the hardening stress due to dislocation interactions, σss is the solid solution hardening stress, σppt is the stress due to hardening by the precipitating particles, and σgs is the stress due to hardening by grains refinement, which increases the volume fraction of the grain boundaries. The Hall-Petch equation ( 2) is

σ gs = ∼ k y √d , ( 2
)
where ky is the strengthening coefficient = 0.74 for low-carbon ferritic steel [START_REF] Gol'dshtein | Special steels[END_REF] and d is the average grain size. These components describe the relationship between the yield strength and grain size, indicating that a significant contribution of the σgs term to the overall strengthening in low-carbon low-alloyed steels can reach 40%. The contributions of each of

the (σ0 = 2 • 𝐺𝐺 • 10 -4 ≤ 5-10%, σρ = 𝛼𝛼 • 𝑚𝑚 • 𝐺𝐺 • 𝑏𝑏 • 𝜌𝜌 1/2 ≤ 3-5%, σss = ∑ 𝐾𝐾 𝑖𝑖 𝐶𝐶 𝑖𝑖 𝑛𝑛 𝑖𝑖=1 ≤ 25%)
components to the total strengthening of the low-carbon low-alloyed steel do not exceed the value of σgs [START_REF] Gol'dshtein | Special steels[END_REF].

In the production of steel by applying the normalizing heat treatment, an important strengthening mechanism (further to a solid solution (σss)) is the presence of dispersed particles, such as carbides, nitrides, and intermetallic compounds (σppt). The steel is conditioned such that during the normalizing heat treatment, only slight grain growth occurs and strain-hardening is annihilated. Meanwhile, in the steels obtained by the TMCP method, strengthening due to the formation of a finely dispersed structure (σgs) and strain-hardening plays a key role.

With high-temperature heating, grain growth occurs that entails changes in the mechanical properties. The intensity of the grain growth process is determined by several factors, such as:

-heating temperature and holding time [START_REF] Shome | Effect of simulated thermal cycles on the microstructure of the heat-affected zone in HSLA-80 and HSLA-100 steel plates[END_REF];

-accumulated strain [START_REF] Han | Effect of strain and strain rate on the Bauschinger effect response of three different steels[END_REF];

-the presence of deformation texture (rolling) [START_REF] Rajan | Heat treatment: principles and techniques[END_REF];

-steel deoxidation [START_REF] Rajan | Heat treatment: principles and techniques[END_REF];

-carbonitrides [START_REF] Rajan | Heat treatment: principles and techniques[END_REF];

-dissolution/uncontrolled growth of precipitations (Zener pinning effect) [START_REF] Moon | Pinning efficiency of austenite grain boundary by a cubic shaped TiN particle in hot rolled HSLA steel[END_REF].

Thus, in further consideration, the primary attention is paid to the grain growth phenomenon during heating up to temperatures above Ac3.

Heating temperature and holding time

According to the phase diagram Fe-C, steels are characterized by the critical points Ac1 and Ac3. The transformation kinetics for any particular steels depends on the selected heating temperature and the range where this temperature will be chosen. The austenite grain growth is controlled by diffusion at the heat treatment temperature. The AC1and AC3 points are considered, to show the kinetic of the heat treatment process. For high temperature 950°C heat treatment, when the temperature gets AC1 the transition from ferritic-pearlitic into austenitic phase is started, and when the temperature gets AC3 fully austenitic structure is formed and only then austenitic grain start growing. But all is determined by the time of being above critical points AC1 and AC3. For the investigated steels, the critical points have the following different values (Figure 8): Ac1 = 714°C, Ac3 = 883°C for S460M; Ac1 = 750°C and Ac3 = 930°С for S355J2 (data obtained using Gleeble 3800 dilatometric unit [START_REF] Kostin | Structural Transformations of the Metal of Heat-Affected Zone of Welded Joints of High-Strength Armor Steels[END_REF]). Calculations of critical points (Eq. 3) using the Park equations [START_REF] Alibeyki | The effects of primary thermo-mechanical processing routes and intercritical annealing on the mechanical properties of st37 low carbon steel[END_REF][START_REF] Gorni | Steel forming and heat treating handbook[END_REF] have close (within a 3% error) values: Ac1 = 716°C, Ac3 = 878°С for S460M; Ac1 = 732°C, Ac3 = 911°С for S355J2.

Ас1(°С) = 739-22.8С-6.8Mn+18.2Si+11.7Cr-15Ni-6.4Mo-5V-28Cu

Ac3(°С) = 955-350C-25Mn+51Si+106Nb+100Ti+68Al-11Cr-33Ni-16Cu+67Mo (3)
An analysis of the equations shows that the differences in the critical points of these steels are due to differences in the micro-alloying elements. They differ in the content of such elements as V, S, Ti, and Nb. Unlike in S460M steel, the vanadium content is six times higher, while the sulfur is halved in S355J2 steel. Considering the heating temperatures used in post-weld treatments, it is possible to estimate the amount of overheating for the investigated steels (Figure 9). Thus, for S460M steel ∆TAc1 = 36°C and ∆TAc3 = 67°C, whereas for S355J2 steel ∆TAc1 = 0°C and ∆TAc3 = 20°C. This suggests that the transformation of austenite in S460M steel will proceed more intensively. The transition from α-Fe to γ-Fe (austenite) begins when the critical point Ac1 is reached and completely ends at Ac3. The completion of the phase transformation process is characterized by the formation of austenite and the disappearance of pearlite. This newly formed austenite, however, will be heterogeneous even in the volume of the selected grain. To obtain homogeneous austenite during heating, it is necessary not only to cross the Ac3 point but to overheat the steel above this point, or dwell at whatever the current temperature is. Such steps allow completing the diffusion processes inside the austenite grain. The rate of transformation largely depends on the initial structure of the steel, the dispersity of cementite, and its shape.

The smaller the cementite particles and the larger their total surface, the faster this transformation occurs. Consequently, the selected treatment temperatures for S460M steel will have a destructive effect on the structure and, therefore, on the mechanical properties.

Due to the slight overheating of S355J2 steel, this factor does not significantly affect strength. At these conditions, the grain size is increased, as shown above in 1.5 times for S355J2 steel (overheating is equal to 20 °C at Ac3 point), while for S460M steel (overheating is equal to 67 °C at Ac3 point) in 3.5 times. According to the Hall-Petch relation Eq. ( 2) the strength for the S460M steel is decreased by ∼ 125 MPa, which is in good agreement with the observed experimental data (Fig. 1a).

Fig. 9. Infographic of high-temperature heating of the investigated steels.

Accumulated strain

In addition to the heating temperature for treatment and the exposure time, the final grain size also depends on the initial metal structure after plastic deformation. After the various types of metal processing by plastic deformation (e.g., rolling, forging, and drawing [START_REF] Pashinskaya | Effect of drawing with shear on structure and properties of low-carbon steel[END_REF][START_REF] Pashinska | Formation of structure and properties of low-carbon steel under rolling with shear and cold drawing[END_REF]) in steel, the stability of austenite above the critical temperature changes. The dynamics of relaxation and recrystallization processes also varied. The metals subjected to plastic deformation are less thermally stable [START_REF] Song | Mechanical properties of an ultrafine grained C-Mn steel processed by warm deformation and annealing[END_REF] because they contain more defects in their crystal structure, and hence lower temperatures are necessary to activate their movement.

An increase in the accumulated strain leads to an increase in the dislocation density, mainly in the zones adjacent to the grain boundaries. In other words, the energy stored in grain boundaries, becomes the additional driving force for grain growth, i.e. the smaller grains the higher the driving force. At a critical accumulated strain, the grain grows very quickly due to the mechanism of grain growth. The dislocation structure of the grain boundaries, in this case, is such that, upon heating, several grains merge into one large grain. Dislocations of a different sign annihilate, and the boundary between the individual grains is cleared and gradually disappears. When the strain exceeds the critical level, the active grain growth mechanism is replaced by a primary recrystallization mechanism (occurrence of the new grains nuclei and their growth). Figure 10 presents the LAM maps for the investigated steels. From the above results, it is clearly seen that the areas with high dislocation density in S460M steel are noticeably larger than that in S355J2 steel. As above noted, this is one of the factors influencing the intensive grain growth in S460M steel during further hightemperature (950 °C) heating.

Steel deoxidizing

There are two types of steel microstructure: hereditarily fine-grained and coarsegrained. The first is characterized by a low tendency for grain growth, while the second has increased growth. The bigger grain of austenite, the bigger the ferrite grain after austenite decomposition [START_REF] Gol'dshtein | Special steels[END_REF]. Austenitic grains grow only when heated; hence, the maximum heating temperature of the steel and its hereditary origin determines the final grain size. More notably, even different melts of the same steel grade may have a different ability for grain growth. The steel deoxidation method matters here. Steel deoxidized only with ferromanganese or ferrosilicon is hereditarily coarse-grained, and steel additionally deoxidized with aluminum is fine-grained [START_REF] Gol'dshtein | Special steels[END_REF][START_REF] Rajan | Heat treatment: principles and techniques[END_REF]. The aluminum introduced into the liquid steel before crystallization forms the nitride compound AlN with the nitrogen dissolved in the liquid steel. This compound is dissolved in the liquid steel, and after its crystallization and subsequent cooling, it is released in the form of submicrocrystalline particles (so-called "nonmetallic dust"), which are located along the grain boundaries, preventing them from growing.

As shown in Figure 7, the average grain size of the HAZ metal of both steels after high-temperature heating is close enough, which is explained by the peculiarities of the microstructure of the HAZ metal before heating. First of all, the metal in the HAZ has no strain hardening as well as the texture of rolling. The HAZ of both steels has a mixture of bainite/ferrite structure, so to restore the grain size of the parent austenite, a reconstruction (a first approximation) was carried out using the ARPGE software (Figure 11). This reconstructed microstructure represents an estimation of parent austenite grains for the HAZ metal. The reconstructed grains of parent austenite are numbered and presented in different colors; black dots in the bulk of the grains and white regions correspond to unsolved points. It can be seen that parent austenitic grains for both the investigated steels have a close size distribution due to the influence of the welding thermal cycle (rapid heating, short exposure time, and rapid cooling). In this case, the exposure time of the metal above the phase recrystallization temperature (Ac3) is not enough for complete diffusion processes. Differences in the microstructure are significantly noticeable though when heating the steel in its initial state at high temperature. (Figure 6). Abnormal grain growth of 3.5 times compared with the initial state for S460M steel may indicate its hereditary coarse grain. It is also important to note that the aluminum content in it is almost half that of S355J2 steel.

Carbonitrides

The carbide-forming elements have a significant effect on the microstructure and mechanical characteristics of the steel, which is caused by the slowing down of diffusion processes. The mechanism of carbide action for II-group (interstitial phase, MeC) is that they are hardly soluble in austenite even at high temperatures, while carbides of the I -group (Me3C, Me23C6, Fe3C, Cr23C6) dissolve easily [START_REF] Gol'dshtein | Special steels[END_REF][START_REF] Adrian | Thermodynamic model for precipitation of carbonitrides in high strength low alloy steels containing up to three microalloying elements with or without additions of aluminium[END_REF]. The order of carbide dissolution is determined by their stability and the degree of transition into the solution, i.e., the carbide amounts. Non-carbide-forming elements (Si, Ni, Co) prevent the dissolution of carbides in austenite. Mn and Cr (the carbides of which dissolve in austenite at lower temperatures), when dissolved in austenite, will increase the solubility of the VC [START_REF] Gol'dshtein | Special steels[END_REF]. Titanium will not have such an effect since its carbide (in low-carbon steel C < 0.1 wt%) will transfer to austenite at a higher temperature. In the case of a treatment temperature of 950 °C, ∼0.1% V, ∼0.01%

Nb, ∼0.008% Ti dissolve in austenite [START_REF] Gol'dshtein | Special steels[END_REF] and the excessive content of these elements will form a hardening carbide phase which has a barrier effect on the migrating grain boundary.

The dissolution or coagulation of dispersed phases in austenite eliminates the barrier effect of the particles (Zener pinning effect). The maximum grain size of austenite d depends on the size of the particles and their quantity (Eq. Zener) [START_REF] Sun | Effect of heat treatments on microstructural evolution of additively manufactured and wrought 17-4PH stainless steel[END_REF]:

𝑑𝑑 = 4 3 • 𝑟𝑟 𝑓𝑓 , ( 4
)
where r is the radius of the particles, and f is their volumetric fraction. Assuming that the volume fraction of the hardening particles is 1%, then in the case of their size of 100 nm, d = 7 μm; and with an increase to 500 nm, d = 35 μm. Consequently, the larger the volume fraction of undissolved particles retained in the steel upon heating and the smaller their size, the more fine-grained austenite is obtained. The microstructures in Figure 12 illustrate the morphology of hardening particle distribution (marked with numbers 1-4) for the investigated steels after heating to 950 °C. It can be seen after heating to this temperature that S355J2 steel has more dispersed inclusions with a size of ~100 nm, while their number in S460M steel is less and can be found rather large. The results of the EDS analysis carried out at the marked points (Figure 12, Table 3) confirm that the selected particles in S355J2 steel are carbonitrides of Ti and Nb. In addition, the presence of AlN particles was also detected. This indicates that the highstrength state of S355J2 steel is achieved due to the formation of a hardening carbide phase, which inhibits grain growth at high heat treatment temperatures (T ≥ Ac3).

For the S460M steel, it was revealed that one inclusion is niobium and titanium carbonitride, while its dimensions (~500 nm) are significantly larger than those observed in the S355J2 steel. Considering that the volume fraction of hardening particles is small, it can be concluded that when heated above Ac3, they could go into solid solution, followed by segregation in the form of large carbides. This is an additional factor that induces intense grain growth in the TMCP (S460M) steel during heating. In such particular conditions, the pinning effect is suppressed that causes increasing the grain size of S460M steel up to 3.5 times (from 10 to 35 µm). Also, as the dimensions of the hardening precipitations increase, the coherence of the interfaces with the matrix is lost, leading to a drop in the mechanical characteristics [START_REF] Tao | Effects of cooling method after intercritical heat treatment on microstructural characteristics and mechanical properties of as-cast high-strength low-alloy steel[END_REF].

Conclusions

The effect of the high-temperature heat treatment on the mechanical properties and microstructure of the HSLA steels processed by various technologies have been investigated. It has been shown that at a temperature T ≤ 650 °C, associated with the treatment of relieving welding stresses, the mechanical properties of the investigated steels are stable within the error limits. At a temperature T ≤ 750° C (for example, enough for thermal shaping) in the steel, obtained by TMCP, the yield strength decreases from 15% to more than 25% at T ≤ 950 °C (for example, hot stamping). The mechanical properties of the normalized steel are stable at all studied temperatures. It has been established that the decrease in the mechanical properties of the TMCP steel is due to intensive grains growth at T > Ac3 conditioned by several factors, e.g., high dislocation density, rolling texture, dissolution, and uncontrolled growth of the hardening precipitation (carbonitrides). The stability of the properties for normalized steel at T > Ac3 is caused by the high stability of austenite due to the effect of preventing the intensive growth of the grains (Zener pinning).

The regularities of the changes in the mechanical properties of the HAZ metal (rapid heating/cooling) in both steels under heating to 950 °C are similar. It results from the similar sizes of the parent austenite grains in the HAZ metal. Such particular effects for the TMCP steel HAZ metal are due to the absence of factors such as deformation hardening and rolling texture. They are eliminated by heating by the thermal cycle of welding.

Thus, the factors associated with mechanical treatment procedure (deformation hardening, rolling texture) as well as microalloying (precipitation hardening) play key roles in structure stability under the following high-temperature heat treatment.

From the practical point of view, the HSLA steel obtained by TMCP can be used in the manufacturing of welded steel structures that do not require hot straightening and stamping operations. Heating is allowed not higher than Ac1 (714 °С) for welding stress relieving. The HSLA steel, obtained through normalizing, may be used in the manufacture of welded steel structures involving consequent treatment at temperatures of up to 950 °C.

Novelty:

• The decrease in the mechanical properties of TMCP steel at T > Ac3 is due to intensive grain growth

Fig. 1 .

 1 Fig. 1. Influence of heating on the mechanical properties of the as-rolled steel grade S460M: Yield stress, Ultimate tensile stress, Plasticity -a; Impact strength -b.

Fig. 2 .

 2 Fig. 2. Influence of the heating on the mechanical properties of the as-rolled steel grade S355J2: Yield stress, Ultimate tensile stress, Plasticity -a; Impact strength -b.

Fig. 3 .

 3 Fig. 3. The mechanical properties of the investigated steels after thermal treatment. Yield stress -a,e; Ultimate tensile stress -b,f; Plasticity -c,g; Impact strength -d,f. Steel grade S460M -a,b,c,d. Steel grade S355J2 -e,f,g,h. As-rolled (initial state) -1; HAZ -2; HAZ with heating to 950 °C -3.

4 . 5 . 6 . 7 .

 4567 5 c,d) for this structure confirm the presence of {110}<111> texture for steel grade S460M, and separate pole peaks for steel grade S355J2. The difference in grain size is also observed in S460M steel, whereas for S355J2 steel, the grain size distribution is more uniform. The SEM microstructures of the investigated steels in as-rolled condition (initial state). a,c -steel grade S460M; b,d -steel grade S355J2. The rolling direction is horizontal.The studied steels in the initial state heated to 950 °C exhibit completely different behavior. In S355J2 steel, there is a moderate grain growth from the initial state of 20 µm to 25-30 µm (1.5 times growth) and the dispersed distribution of pearlite. In comparison, S460M steel is characterized by significant grain growth (Figures 6a and 7a ) from 10 µm in the initial state to up to 35 µm after the heating up to 950 °C (i.e., 3.5 times) and a smaller fraction of pearlite component. Thus, during heating up to 950°C the transformation of the ferritic pearlitic into austenitic structure occurs with the following growth of the austenite grains size. As a sequence of further cooling, the austenitic structure is transformed back to the ferritic pearlitic and the size of last is determined by the grain boundaries of the previous austenite grains. The reasons for such grains growth evolution are considered below. The EBSD orientation maps (a, b) and pole figures (c,d) of the investigated steels in as-rolled condition (initial state). a,c -steel grade S460M; b,d -steel grade S355J2; IPF, crystallographic direction // ND (Normal direction) Microstructural analysis data has shown that the HAZ of the investigated steels has similar morphology and consists of a mixture of bainite and ferrite (Figure 6 c and d, Figure 7 c and d). The hardness values for such microstructures are 280±14 HV and 260±13 HV for S460M and S355J2 steels, respectively. The SEM microstructure of the as-rolled steels after high-temperature heating to 950 °C (a,b), HAZ metal (c,d), HAZ metal after high-temperature heating to 950 °C (e,f); (a,c,e) -steel grade S460M, (b,d,f) -steel grade S355J2 The EBSD orientation maps of the as-rolled steels after high-temperature heating to 950 °C (a,b), HAZ metal (c,d), HAZ metal after high-temperature heating to 950 °C (e,f); (a,c,e) -steel grade S460M, (b,d,f) -steel grade S355J2 The grain orientation analysis (Figure 7 c and d) shows no pronounced deformation texture for S460M steel. Under the heat treatment (950 °C), the simulated HAZ metal structure approaches a more equilibrium ferritic-pearlitic form. The average grain size for the investigated steel, in this case, is very close and equal to 25-30 µm (Figures 6 e,f and 7 e,f).

Fig. 8

 8 Fig.8 CCT diagrams for investigated steels. a -TMCP steel grade S460M, b -Normalized steel grade S355J2.

10 .

 10 The LAM maps for the as-rolled steels (initial state), constructed according to the EBSD data. a -steel grade S460M (TMCP); b -steel grade S355J2 (normalized);

11 .

 11 Parent austenite grains of the investigated steels HAZ metal, obtained by means of ARPGE software (The grains are described by different colors). a -steel grade S460M; b -steel grade S355J2

Fig. 12 .

 12 Fig. 12. The SEM microstructure (a and d) and selected areas EDS Spectra (b, c, e, and f) of the as-rolled steels after high-temperature heating to 950 °C. a, b, and c -steel grade S460M; d, e, and f -steel grade S355J2. EDS spectra numbers correspond to numbers on the SEM micrographs.

Table 1 .

 1 The chemical compositions (in wt%) and CE of the investigated HSLA steels.

	Grade	C	Si Mn	P	S	Nb	V	Ti	Cr	Ni	Mo Cu	Al	N	СE***
	S460M* 0.16 0.60 1.7 0.03 0.025 0.05 0.12 0.05 0.30 0.80 0.20 0.55 0.02 0.025 0.65
	S460M ** 0.15 0.23 1.30 0.017 0.013 0.050 0.01 0.012 0.09 0.019 -	-0.025 0.005 0.38
	S355J2* 0.16 0.55 1.7 0.035 0.03 0.06 0.12 0.06 0.35 0.55 0.13 0.60 0.015 0.025 0.64
	S355J2** 0.16 0.24 1.36 0.016 0.007 0.031 0.06 0.016 0.04 0.030 -	-0.040 0.007 0.4
	* -EN10025-4, ** -measured, *** carbon equivalent (CE)=%C+%Mn/6+(%Cr+%Mo+%V)/5+(%Cu+%Ni)/15

Table 2 .

 2 The mechanical properties of the investigated HSLA steels.

	Steel/ supply	YS, MPa UTS, MPa δ, % KCV -40, J/cm 2
	conditions
	S355J2/normalization*	370	530	32	200
	S460M/TMCP*	480	600	27	90
	* according to the certificate of quality for the steel

Table 3 .

 3 Chemical composition of the selected points for the investigated steels

	Spectrum	N	Al	Si	S	Ti	V	Mn	Fe	Nb	Total
				TMCP S460M steel
	Spectrum 1	10.84	0.13	0.26	1.36	30.09	-	2.40	44.29	10.63	100.00
	Spectrum 2	-	0.00	0.43	-	-	-	1.89	97.68	-	100.00
	Spectrum 3	-	0.00	0.34	-	-	-	2.60	97.06	-	100.00
	Spectrum 4	-	0.00	0.42	-	2.22	1.07	-	91.42	4.86	100.00
				Normalized S355J2 steel
	Spectrum 1	1.67	0.49	0.31	-	0.46	-	1.29 92.16	3.62	100.00
	Spectrum 2	2.72	2.14	0.30	-	0.52	-	1.72 92.21	0.39	100.00
	Spectrum 3	0.89	0.42	0.29	-	0.37	-	1.24 96.40	0.38	100.00
	Spectrum 4	-	0.00	0.33	-	-	-	2.08 97.59	-	100.00

•

  The high dislocation density, texture, dissolution of precipitates induces intensive grain growth under following high-temperature heating • The stability of the properties of normalized steel at T > Ac3 provided by high stability of the austenite • Zener pinning effect plays a key role in preventing the intensive growth of the grains at T > Ac3

Contributor Roles Taxonomy Anatoliy Zavdoveev, Thierry Baudin, Hyoung Seop Kim, Massimo Rogante and Mykola Skoryk Conceptualization, Methodology Anatoliy Zavdoveev, Hyoung Seop Kim, Valeriy Poznyakov Supervision Anatoliy Zavdoveev, Thierry Baudin, Yuriy Demchenko, Victor Zhukov and Mark Heaton

  Data curation, Writing-Original draft preparation. All authors discussed the results and commented on the manuscript.

Acknowledgments

The research work reported in this paper was funded by the National Academy of Sciences of Ukraine [grant number 0117U001666]. The authors are grateful to Dr. C. Cayron for kindly providing ARPGE program and Dr. B. Beausir for kindly providing ATEX software, Prof. D. Raabe, Dr. S.Mironov, Dr. A. Kulakowska for many fruitful discussions, Dr. S. Moskalyuk, Dr. A. Maksimenko, Dr. S. Jdanov, Dr. V.Kostin for help in experiments.

Data availability:

The raw/processed data required to reproduce these findings cannot be shared at this time due to technical or time limitations.


Manuel Diaz-Zapata

Özgür Erkent

Christian Laugier

Instance Segmentation with Unsupervised Adaptation to Different Domains for Autonomous Vehicles

Detection of the objects around a vehicle is important for a safe and successful navigation of an autonomous vehicle. Instance segmentation provides a fine and accurate classification of the objects such as cars, trucks, pedestrians, etc. In this study, we propose a fast and accurate approach which can detect and segment the object instances which can be adapted to new conditions without requiring the labels from the new condition. Furthermore, the performance of the instance segmentation does not degrade in detection of the objects in the original condition after it adapts to the new condition. To our knowledge, currently there are not other methods which perform unsupervised domain adaptation for the task of instance segmentation using non-synthetic datasets. We evaluate the adaptation capability of our method on two datasets. Firstly, we test its capacity of adapting to a new domain; secondly, we test its ability to adapt to new weather conditions. The results show that it can adapt to new conditions with an improved accuracy while preserving the accuracy of the original condition.

I. INTRODUCTION

An autonomous vehicle needs to detect the number and location of objects around it to perceive and comprehend its surroundings. Although several object detection algorithms propose to find the objects, this results in bounding boxes which is a coarse localization of the objects ( [START_REF] Ren | Faster r-cnn: Towards realtime object detection with region proposal networks[END_REF], [START_REF] Redmon | Yolov3: An incremental improvement[END_REF]). In this study, we deal with instance segmentation and its unsupervised adaptation to new domains in this study.

Instance segmentation can be defined as the detection of the pixels in an image belonging to an instance such as car, truck, pedestrian, rider, etc. It is considered to be harder than object detection or semantic segmentation since all the object instances must be found in an image and a bounding box is not sufficient [START_REF] De Brabandere | Semantic instance segmentation with a discriminative loss function[END_REF]. This is mainly due to the fact that in instance segmentation, you need to count the number of occurrences of an instance in an image, contrary to semantic segmentation where you don't discriminate between different instances of the same class, i.e. you cannot discriminate between two instances of a vehicle. Due to this difficulty, instance segmentation usually provides more detailed information with respect to semantic segmentation and object detection with the cost of an increased computation time. Since we are considering autonomous vehicles, we propose an approach which is fast and achieves a decent accuracy to be used on an autonomous vehicle.

This work was partially supported by EU project CPS4EU. This work was partially supported by Toyota Motor Europe. 1 Authors are with Univ. Grenoble Alpes, INRIA, Chroma Team, France INRIA, Rhône-Alpes, France. Correspondence: name.surname@inria.fr Fig. 1: A sample overview of the instance segmentation adaptation. The images are from the Cityscapes Dataset [START_REF] Sakaridis | Semantic Foggy Scene Understanding with Synthetic Data[END_REF] and its foggy variant [START_REF] Sakaridis | Semantic Foggy Scene Understanding with Synthetic Data[END_REF]. A model based on YOLOv3 is used [START_REF] Redmon | Yolov3: An incremental improvement[END_REF]. M γ is the model trained for the source domain (original condition), M γ is the adapted model based on only RGB images. x s : source domain, x t : target domain, ȳs : ground truth label that was used only for original model training, ŷs , ŷt : output of the M γ on source and target domain, y s , y t : output of M γ on source and target domain. The red circle shows the improvement (minivan detected after adaptation) in target domain instance segmentation.

An important problem with modern approaches for instance segmentation methods which depend on deep neural networks (DNNs) is their requirement for tremendous amount of data for training. Utilization of the same model in different conditions usually results in reduced performance. Although the performance can be increased by labeling the images from the new condition, this will increase the cost of training since labeling at pixel level is a high-cost task [START_REF] Cordts | The cityscapes dataset for semantic urban scene understanding[END_REF] which can take up to several hours for a single image.

To overcome this problem, we consider the unsupervised domain adaptation (UDA) which adapts the previously obtained DNN model into the new condition without requiring labels ( [START_REF] Long | Learning Transferable Features with Deep Adaptation Networks[END_REF], [START_REF] Erkent | Semantic segmentation with unsupervised domain adaptation under varying weather conditions for autonomous vehicles[END_REF]). We achieve this adaptation by making the features of both of the domains similar to each other in the initial layers of the network (see Fig. 1). In this context, source domain refers to the old condition where we train our model and target domain refers to the new condition without labels.

The contributions of this work can be listed as follows:

• A new instance segmentation method based on YOLOv3 [START_REF] Redmon | Yolov3: An incremental improvement[END_REF] which is fast and achieves a high accuracy • An unsupervised domain adaptation method for instance segmentation which does not require the labels neither from source nor target domain • The adapted instance segmentation model can still achieve a high accuracy in the source domain. We evaluate our approach on two datasets for instance segmentation and show that the proposed approach can achieve results similar or even better than supervised methods.

In Section. II, we briefly summarize the related literature; in Section. III, we explain the new instance segmentation method; in Section. IV, the instance segmentation domain adaptation method is explained and in Section. V, we provide the results of our evaluation on two datasets. Finally, we summarize the findings of our work.

II. RELATED WORK

First, we make a brief summary of instance segmentation methods and then we provide the previous studies on unsupervised domain adaptation in general with a specific focus on object detection and segmentation.

A. Instance segmentation

Currently, instance segmentation methods follow one of two variants: pixel-based methods and polygon-based methods. In this work, a pixel-based algorithm is proposed.

1) Pixel-based methods: Pixel-based methods gained a high popularity in recent years due to their success. Here, instances are described by masks that indicate which pixels belong to them. The creation of these masks is usually aided by region proposals from a parallel detector such as Faster R-CNN [START_REF] Ren | Faster r-cnn: Towards realtime object detection with region proposal networks[END_REF] in Mask R-CNN [START_REF] He | Mask r-cnn[END_REF] and a cascade variant of Mask R-CNN [START_REF] Chen | Hybrid task cascade for instance segmentation[END_REF], where an FCN [START_REF] Long | Fully convolutional networks for semantic segmentation[END_REF] is used to perform the foreground segmentation on the bounding box predictions in order to find which pixels belong to the object. In YOLACT [START_REF] Bolya | Yolact: Real-time instance segmentation[END_REF], instead of focusing its attention to specific parts of the image using the bounding boxes from a detector stage, prototypes for the entire image are found and linearly combined using a set of predicted coefficients.

Other instance segmentation methods such as AdaptIS [START_REF] Sofiiuk | Adaptis: Adaptive instance selection network[END_REF] and Deeperlab [START_REF] Yang | Deeperlab: Single-shot image parser[END_REF], follow a keypoint-based approach. In AdaptIS, the network adapts to an input point using adaptive instance normalization layers [START_REF] Huang | Arbitrary style transfer in real-time with adaptive instance normalization[END_REF], creating masks in an iterative process for each of the objects in the scene. Deeperlab models the relationships between predicted keypoints and the pixels of the instance using a keypoint heatmap and long, medium and short-range offset maps.

Methods like Recurrent Instance Segmentation [START_REF] Romera-Paredes | Recurrent instance segmentation[END_REF], performs sequential segmentation of the objects in an image and keeps track of which pixels have been assigned with a Convolutional LSTM module.

2) Polygon-based methods: Instead of predicting the pixels that belong to an instance, some methods predict instead the polygon that best fits the instance's boundary, which can help to reduce computation time. ESE-Seg [START_REF] Xu | Explicit shape encoding for realtime instance segmentation[END_REF] performs a Chebyshev polynomial fitting by adding a parallel decoder to a YOLOv3 [START_REF] Redmon | Yolov3: An incremental improvement[END_REF] network. In the case of the deep snake [START_REF] Peng | Deep snake for real-time instance segmentation[END_REF] approach, the network takes a contour as input and outputs vertex-wise offsets to deform the contour in order to match the boundary of the detected instance.

We implement a pixel-based instance segmentation method due to their high accuracy; however, since their speed is generally low for the autonomous vehicles, we propose a new approach which can adapt to new conditions.

B. Unsupervised Domain Adaptation

As aforementioned, deep learning methods require tremendous number of labeled images. However, it is not possible to label images for each new condition, such as a new city, a new vehicle or a new weather condition. Therefore, a number of studies have proposed to use unsupervised domain adaptation where the labeled images from the new domain are not available.

For example, a group of studies adapt to a new condition by using a small number of samples from the new domain and incrementally update the available network. However, this needs an incremental change in the conditions. For example Dai et al. [START_REF] Dai | Dark Model Adaptation: Semantic Image Segmentation from Daytime to Nighttime[END_REF] use different times of twilight images from lighter to darker hours of twilight for adaptation to darkness. Wulfmeier et al. [START_REF] Wulfmeier | Incremental Adversarial Domain Adaptation for Continually Changing Environments[END_REF] also use an incremental approach for adaptation to darkness. Since new domains do not always have the incremental changes, we don't follow such an approach.

Another possibility is to use the invariance of the sensors to the weather conditions such as the invariance property of laser range sensors in different weather conditions and dark. Kim et al. [START_REF] Kim | Season-Invariant Semantic Segmentation with A Deep Multimodal Network[END_REF] use Lidars in different weathers. Chen et al. [START_REF] Chen | Learning Semantic Segmentation from Synthetic Data: A Geometrically Guided Input-Output Adaptation Approach[END_REF] propose to use the depth information to adapt from synthetic domain to real-world domain. Erkent et al. [START_REF] Erkent | Semantic grid estimation with a hybrid bayesian and deep neural network approach[END_REF] also use the the semantic 2D maps to obtain an semantic grid of the scene. However, it is not always possible to find sensors which will be invariant to new weather conditions, therefore they need to be trained for each condition.

Another approach is to make the output of both domains similar to each other. For example, Vu et al. [START_REF] Vu | ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation[END_REF] use GAN training [START_REF] Goodfellow | Generative adversarial nets[END_REF] on the entropy of the output which is used to make these distributions similar to each other. However, if you measure the similarity on the output only, you need to convert these outputs into a high-dimensional space which will increase computation time.

Another line of work deals with the similarity of the distribution of the features since they already have a highdimensional data. Long et al. [START_REF] Long | Learning Transferable Features with Deep Adaptation Networks[END_REF] is one of the early works which consider to update the neural network to obtain similar features for domain adaptation. GANs are also used to make the features similar to each other which was initially proposed by Ganin et al. [START_REF] Ganin | Domain-adversarial training of neural networks[END_REF]. Due to its success, it has been implemented by several studies including [START_REF] Tsai | Learning to Adapt Structured Output Space for Semantic Segmentation[END_REF], [START_REF] Zhang | Fully Convolutional Adaptation Networks for Semantic Segmentation[END_REF], [START_REF] Sankaranarayanan | Learning from Synthetic Data: Addressing Domain Shift for Semantic Segmentation[END_REF] for semantic segmentation. Zhang et al. [START_REF] Zhang | Syntheticto-real domain adaptation for object instance segmentation[END_REF] use it for instance segmentation from synthetic to real domain . Adaptation approaches do not always use GANs, for example Peng et al. [START_REF] Peng | Domain Adaptation with One-step Transformation[END_REF] implement Wasserstein GANs (W-GANs) [START_REF] Arjovsky | Wasserstein gan[END_REF], Erkent et al. [START_REF] Erkent | Semantic segmentation with unsupervised domain adaptation under varying weather conditions for autonomous vehicles[END_REF] use W-GANs and MMDs for adaptation of semantic segmentation.

Other than output and feature similarity, some studies try to make the inputs similar to each other before they are provided to the classification network. Cycada [START_REF] Hoffman | CYCADA: Cycle-Consistent Adversarial Domain Adaptation[END_REF] trains in Synthetic data and then transforms the real images into a synthetic style by using GANs. [START_REF] Zhang | Curriculum Domain Adaptation for Semantic Segmentation of Urban Scenes[END_REF] and [START_REF] Li | Bidirectional Learning for Domain Adaptation of Semantic Segmentation[END_REF] also use a similar approach for semantically segmenting the scenes. It should be noted that this requires additional computation to pre-process the images.

Our approach, which adapts the instance segmentation network into new conditions is different from the aforementioned approaches in a few aspects. First, we propose a method which can preserve a similar accuracy for source domain while increasing the accuracy for the target domain for instance segmentation thanks to self-supervision loss. Secondly, we do not require the instance labels for neither source nor target domain for adaptation. In instance segmentation, converting the instance labels from one dataset to another can be a time-consuming task depending on the labeling conventions, or even impossible depending of the availability of the source domain labels.

III. INSTANCE SEGMENTATION NETWORK

We use a model based on YOLOv3 [START_REF] Redmon | Yolov3: An incremental improvement[END_REF] for instance segmentation. Here, a segmentation head is added to the Darknet-53 backbone in parallel to the YOLO detection head. This segmentation head uses the Pyramid Pooling Module (PPM) proposed in PSPNet [START_REF] Zhao | Pyramid Scene Parsing Network[END_REF] to produce enriched features.

First, the network takes three feature maps from different depths in the backbone to find multi-scale information. Upsampling is performed on the two feature maps with smaller spatial size, in order to match the shallowest feature map for concatenation. A 1 × 1 convolution is also applied to the shallowest map in order to increase its channel depth.

Once the feature maps are concatenated, average pooling is performed with bin sizes of [START_REF] Ren | Faster r-cnn: Towards realtime object detection with region proposal networks[END_REF][START_REF] Redmon | Yolov3: An incremental improvement[END_REF][START_REF] De Brabandere | Semantic instance segmentation with a discriminative loss function[END_REF][START_REF] Long | Learning Transferable Features with Deep Adaptation Networks[END_REF] together with a 1×1 convolution to reduce depth size by four, batch normalization and ReLU activation. The resulting four feature maps are appended to the original feature map, resulting in the set of features that will be used for instance segmentation. Two separate convolutions are applied to the resulting feature map, one to perform semantic segmentation and another one to decrease feature depth. Semantic segmentation is necessary to separate the foreground from the background for the instance segmentation.

With the semantic feature maps processed, the bounding boxes from YOLO are used to extract the detections to do foreground segmentation. First, non-maximum suppression is performed on the bounding boxes proposed by the detector head. Then, since YOLO's predictions are relative to the image size, the bounding boxes are scaled to the size of the feature map to crop the detections for instance segmentation. This processing of the features can be seen on Fig. 2.

Finally, foreground segmentation is performed in the same fashion as in [START_REF] He | Mask r-cnn[END_REF], where an FCN architecture is used to find the mask (Fig. 3). The cropped features are first bilinearly upsampled to have a spatial size of 20 × 20, then two 3 × 3 convolutions with stride of 1 are applied followed by a ReLU non-linearity. The resulting feature map is upsampled once more to a size of 40 × 40, followed by a 3 × 3 convolution and a 1 × 1 convolution, both with stride of 1, to find the instance mask.

IV. DOMAIN ADAPTATION FOR INSTANCE SEGMENTATION

We will make a brief reminder of the unsupervised domain adaptation (UDA) and explain how it is used for instance segmentation. First, we define a couple of terms. "Source domain": S = { x i s , ȳi s } ns i=1 , "target domain": X t = { x j t } nt j=1 , "source data" (no labels): X s = { x i s } ns i=1 , "target data": X t with n s and n t the number of samples for source and target domain respectively. A deep neural network is trained on "source domain" M γ to detect the instance of each pixel ŷi s . The aim of M γ is to make a prediction such that the cost

c s (M γ ) = Pr (x,ȳ)∼p [M γ (x) = ȳ] is minimized.
In UDA, since we don't have access to labels of the target domain, we try to approximate the distribution of the target domain to the source domain. For this, we investigate the probability distributions of the source and target domain Pr p and Pr q . The model M γ is updated such that the features of both domains become similar to each other while the c s is simultaneously minimized. An overview of the proposed adaptation method is given in Fig. 4. M γ is the initial model which is trained on source domain with its provided labels. This model can also be a pre-trained network. We separate this model into two parts: E θ and F α . γ, θ, γ, θ and α are the hyperparameters of the neural networks. M γ is replicated with its parameters as M γ . While the parameters of E θ and F α are fixed, the parameters of E θ are optimized so that the output of E θ for target domain data become similar in distribution to the source domain data. Therefore, the target cost optimization can be rewritten as c t (E θ (x t ) = f t ) = Pr (f ,y)∼q [F α (f ) = y] and hence the parameters to be optimized reduce from ||γ|| to ||θ||. This results in reduced training time since we need to optimize the parameters of a smaller network.

It should be noted that the capacity of E must be sufficient to produce similar features for both domains while providing features for a high quality classification. The optimization of θ can be given as: where θ ∈ Θ and f i t = E θ (x i t ) are target domain features. D is the network that gives the distribution between source and target domain features. λ > 0 is used as a regularization hyperparameter; l g ( fs

, E θ (x s )) = || fs -E θ (x s )|| 2 2 is the self- supervision loss.
We measure the distance D(Pr ( ft)∼p , Pr (E θ (xt))∼q ) between the source and target domain feature distributions by using the Wasserstein Distance.

W-GAN: m-th order Wasserstein distance is [START_REF] Arjovsky | Wasserstein gan[END_REF]:

W m (Pr ( f )∼p , Pr ((f ))∼q ) = inf Pr fs∼p,(ft)∼q E || fs -f t || m (2)
over all joint distributions. This optimal transport problem over joint probabilities has a dual formulation (Kantorovich-Rubinstein) [START_REF] Villani | Optimal transport: old and new[END_REF]. It can be rewritten for m = 1 as follows:

inf θ∈Θ inf φ∈Φ E D φ ( fs ) -E [D φ (E θ (x t ))]+λl g ( fs , E θ (x s )) (3)
where φ ∈ Φ represents the weights of the discriminator D. A solution to find the parameters of the discriminator has been proposed by Arjovsky et al. [START_REF] Arjovsky | Wasserstein gan[END_REF]. Furthermore, gradient penalty variant solves the problems related to convergence Algorithm 1: Self-Supervised Domain Adaptation

Require: Xs, Xt, E θ , E θ , D φ , λ > 0, n: Mini-batch size, ncrit, λp > 0:

Penalty gradient Initialize θ = θ Initialize the parameters φ of D φ while (θ) not converged do Sample { x i s } n i=1 ∈ Xs, { x i t } n i=1 ∈ Xt Compute f i s = E θ (x i s ), f i t = E θ (x i t ), f i s = E θ (x i s ); for k = 1, • • • , ncrit do Sample a random number ∼ U [0, 1]; f i = f i s + (1 -)f i t ; Update D φ by ascending 1 n n i=1 D( f i s ) -D(f i t ) -λp(||∇f D( f i )|| 2 -1) 2
Update E θ by descending

1 n n i=1 D(f i t ) + λlg(f i s , f i s )
Fig which we use in this study [START_REF] Gulrajani | Improved training of wasserstein gans[END_REF]. For a detailed understanding of the algorithm, please refer to Algorithm. 1. More details and variants of the algorithm can be found in [START_REF] Erkent | Semantic segmentation with unsupervised domain adaptation under varying weather conditions for autonomous vehicles[END_REF] with further experiments for the semantic segmentation problem.

V. EXPERIMENTS

We evaluate the adaptation capability of our instance segmentation method on two different domains: Semantic KITTI [START_REF] Alhaija | Augmented reality meets computer vision: Efficient data generation for urban driving scenes[END_REF] and Foggy Cityscapes Dataset (CS) [START_REF] Sakaridis | Semantic Foggy Scene Understanding with Synthetic Data[END_REF]. We use the weights of the YOLOv3 network [START_REF] Redmon | Yolov3: An incremental improvement[END_REF] which is trained on the COCO Dataset [START_REF] Lin | Microsoft coco: Common objects in context[END_REF]. The weights of the instance segmentation part are obtained by using the Cityscapes Dataset; however, since we freeze the YOLO object detection network, the object detection is not improved with this training and certain classes such as rider is not detected.

We use the source domain as Cityscapes in both of the datasets. For unsupervised domain adaptation the hyper parameters are selected to be n crit = 5, λ = 10, λ g = 20 and Adam optimizer with learning rate α = 1 × 10 -4 and β 1 = 0.7 and β 2 = 0.9. The time requirement of inference for images with size of 640x640 pixels is 66 ms while for 1024x1024 pixels, it is 165 ms. In the experiments, we use 1024x1024 pixel sized images. We use a shallow discriminator D as shown in Fig. 5.

For evaluation, we use AP and AP 50 % measures where AP is the average precision over the region level for each class. For AP, 10 different overlaps are used between [0.5, 0.95] in steps 0.05. Overlap is computed for each instance at region level. AP 50 % is also provided which gives the average precision with 50 % overlap.

Semantic KITTI [START_REF] Alhaija | Augmented reality meets computer vision: Efficient data generation for urban driving scenes[END_REF] is a variant of KITTI dataset where 200 images are selected and labeled in detail with same labels of Cityscapes. Although both datasets are obtained from the cameras of moving vehicles in traffic, the properties of the cameras and labeling strategies make these two datasets significantly different from each other and the datasets that are trained and achieve a high accuracy of instance segmentation accuracy cannot achieve the same high accuracy on KITTI instance segmentation as demonstrated on the benchmark1 . We use our network which is trained for instance segmentation on Cityscapes, and adapt it to this dataset by using 200 images of the KITTI dataset without using any of the labels. In the current benchmark, no unsupervised method has been reported until now; therefore, this is the first time UDA is performed on Semantic Instance Segmentation KITTI. The image sizes of Cityscapes and KITTI are different. We test the images with and without resizing the KITTI images to be in similar size of CS and find out that resizing improves the accuracy, therefore we use the resized images for the adaptation as shown in Table . I.

Results: As it can be seen, the average performance is highest when the UDA is applied on the method; however, this is not the case for all the classes. For example, the accuracy of the truck and bus detection reduces. This is probably due to the small number of occurrences of these classes. The trains are also recognized worse with respect to non-resized cases probably due to the visibility of the trams. There are only a few frames where these rare classes are visible. We don't report the rider class since we use the pretrained Coco dataset which has no rider class. In overall, the results are comparable with or better than some the state-of-art results in the KITTI benchmark although no supervision is used from the KITTI dataset. Furthermore, the accuracy of the source domain, CS dataset, drops from AP %50 = 25.7% to 24.8% which is less than the increase in KITTI.

The visualization of four samples are shown in Fig. 6. The ground truth is shown on the left column, the non-adapted resized samples in the middle and the adapted ones are on the right column. The important differences between them are shown with a red circle. On the top row, two vehicles are detected as single before adaptation. On the second row, a person is hallucinated before the adaptation. On the third column, a human is detected after adaptation correctly. In the last row, a vehicle which is not detected at a distance is detected after adaptation. It should be noted that although some of the results are corrected after adaptation, it is not sufficient to correct all. The accuracy of the initial model is important to achieve a better adaptation.

Foggy Cityscapes [START_REF] Sakaridis | Semantic Foggy Scene Understanding with Synthetic Data[END_REF], [START_REF] Cordts | The cityscapes dataset for semantic urban scene understanding[END_REF] is a collection of images in which the fog is overlaid to the original real images of the Cityscapes dataset. Therefore, it is a realistic synthetic dataset. The only additional noise factor is the fog, the camera parameters are same in both; therefore we don't resize the images. Three different variances of fog are provided in the CS Foggy dataset. Fog-02 refers to heavy fog while Fog-01 refers to lighter and Fog-005 refers to the lightest fog conditions. An example of three conditions is given in Fig. 7 with the original condition.

For source domain, we use 2975 training images of the CS, while for the target domain, 1500 validation images of foggy condition are used. We use a mixture of all foggy weather conditions since in real life, it is not possible to estimate the amount of fog easily. Again, no labeled images are used for adaptation training and the same parameters with KITTI UDA are used.

Results The results improve significantly for Fog-02 as it can be seen in Table . II. The accuracy of Fog-005 also improves slightly. However, the accuracy of Fog-01 does not improve. The slight or no change in the performance for Fog-01 and Fog-005 conditions can be attributed to the high similarity of source and target domains for instance segmentation. Furthermore, the performance on the original CS dataset does not degrade significantly (from 25.7 % to 24.6 %). To understand the effect of adaptation in detail, we further provide data for Fog-02 condition in Table . III. As it can be seen, nearly all classes improve in performance; only, bus class which has a few samples reduce slightly in performance. Although the detections of cars improve slightly, it is necessary to note that the car class has many samples; therefore, the increase in car class shows a significant increase in the overall dataset.

To visualize the effects of the adaptation, we show some of the results in Fig. 8. On the top row, the tram is recognized after adaptation, which is important in the scene. On the second row, a human is detected on a bike after detection. Since we don't have the rider class in training, this is not recognized as rider. On the third row, again the tram is recognized after adaptation. Finally, on the last row, a minivan is recognized after adaptation which is an important object in the scene.

VI. CONCLUSION

Our proposal depends on two main components: a fast instance segmentation network and an unsupervised domain adaptation approach. As it can be seen, a DNN, which uses Darknet as a backbone can be adapted to new conditions for instance segmentation. The performance increases significantly for the cases where the difference between the source and the target domain is significant such as thick fog condition, or a completely new dataset. For the KITTI dataset, our adaptation outperforms even some of the supervised methods in the KITTI benchmark. As previously mentioned, currently there are not other methods apart from this one which perform unsupervised domain adaptation between two real datasets for the task of instance segmentation. In the future work, we plan to train the backbone of our model on both datasets for new conditions to be able to compare its performance to supervised cases, propose an instance segmentation architecture for real-time inference and do domain adaptation for panoptic segmentation.

Fig. 2 :Fig. 3 :

 23 Fig. 2: Proposed instance segmentation network.

Fig. 4 :

 4 Fig. 4: Adaptation schema overview. Green dotted boxes are the simplified instance segmentation models. Dotted black lines use the back-propagation; while, the blue lines are the processes without back-propagation.

  ( fs)∼p , Pr (E θ (xt))∼q ) + λl g ( fs , E θ (x s ))[START_REF] Ren | Faster r-cnn: Towards realtime object detection with region proposal networks[END_REF]

. 5 :

 5 Fig. 5: Discriminator D. B1 and B2: 2D convolutional + instance normalization layers [40]. FC1 and FC2: fully connected layers. c 0 = 512, c 1 = c 2 = 64 and c 3 = 262144 are number of channels and FC2 outputs a scalar value.

Fig. 6 :

 6 Fig. 6: Samples from KITTI dataset. GT: Ground Truth, All instances are estimated after resize operation.

Fig. 7 :

 7 Fig. 7: Top-left: Normal condition, Top-right: Fog-02, Bottom-left: Fog-01 and Bottom-right: Fog-005.

Fig. 8 :

 8 Fig. 8: Samples from CS Fog-02 dataset. GT: Ground Truth. Red circle shows important differences.

TABLE I :

 I Results with CRF on the KITTI dataset with different architecture types.

		No Resize		Resized	Resized, Adapted
	(%)	AP	AP 50%	AP	AP 50%	AP	AP 50%
	person:	1.00	6.70	3.60	16.10	5.60	26.10
	car:	19.40	43.20	16.00	38.30	27.30	58.10
	truck:	3.00	4.70	12.80	25.60	8.30	14.40
	bus:	15.90	23.50	31.90	59.20	16.70	24.50
	train:	6.80	16.70	6.10	13.00	3.30	8.60
	motor:	0.00	0.00	4.40	18.60	28.00	60.00
	bicycle:	3.10	12.30	1.80	9.30	3.80	15.10
	average:	7.03	15.30	10.94	25.73	13.29	29.54

TABLE II :

 II Average Results for Three Fog Conditions

		Initial Model	Adapted Model
	(%)	AP	AP 50%	AP	AP 50%
	Original	10.9%	25.7%	9.7%	24.6%
	Fog-02	7.3%	16.2%	7.4%	17.8%
	Fog-01	9.1%	21.1%	8.6%	21.0%
	Fog-005	9.6%	22.5%	9.2%	23.2%

TABLE III :

 III Detailed Results for Fog-02

	Fog-02	Initial Model	Adapted Model
	(%)	AP	AP 50%	AP	AP 50%
	person:	2.5%	11.3%	3.0%	13.4%
	car:	12.8%	29.7%	13.2%	31.3%
	truck:	8.9%	16.0%	10.4%	18.2%
	bus:	21.7%	34.3%	19.6%	32.3%
	train:	1.3%	4.3%	0.9%	4.3%
	motorcycle:	2.3%	9.9%	3.1%	14.7%
	bicycle:	1.4%	8.1%	1.9%	10.2%
	average:	7.3%	16.2%	7.4%	17.8%

http://www.cvlibs.net/datasets/kitti/eval instance seg.php? bench-mark=instanceSeg2015

ACKNOWLEDGEMENT

Parts of the experiments presented in this paper were carried out using the Grid'5000 testbed, supported by a scientific interest group hosted by Inria and including CNRS, RENATER and several Universities as well as other organizations.