Hervé Nicolas

Lambert Romain

Nathalie Gey

Perroud Olivier

Slama Meriem

Poulat Ben Haj

Nicolas Hervé

Romain Lambert

Olivier Perroud

Meriem Ben

Haj Slama

Charlie Poulat

Static and impact toughness properties of LB-PBF-built and heat treated 33CrMoV12 steel

Keywords:

ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

Additive manufacturing (AM) processes refers to different techniques to build 3D parts layer by layer from a CAD model. These techniques allow to obtain complex geometrical parts that are impossible to manufacture with conventional methods like casting or forging. Among the AM processes, Laser Beam Powder Bed Fusion (LB-PBF) is a promising way to manufacture complex metallic parts. LB-PBF is a powder based AM technology using the power delivered by a laser to melt successive metallic powder layers [START_REF] Yap | Review of selective laser melting: Materials and applications[END_REF]. This process leads to near-net-shape objects that require no or few post-production steps. For example Chen et al. [START_REF] Chen | Microstructure and composition heterogeneity, tensile property and underlying thermal physical mechanism of selective laser melting tool steel parts[END_REF] managed to build high accuracy blades of 5CrNi4Mo. LB-PBF could also be considered to build mechanical parts with optimized geometries for lightening purposes and with the possibility to gain new functions. In the past decade, different studies showed that steel parts can be successfully processed by LB-PBF. This paper will focus on a low alloy and tool steel that can be selected for gear manufacturing. In the literature, different grades of steels were studied [START_REF] Bajaj | Steels in additive manufacturing: a review of their microstructure[END_REF], HY100 steel [START_REF] Dilip | Selective laser melting of HY100 steel : Process parameters microstructure and mechanical properties[END_REF], X40CrMoV5-1 H13 tool steel [START_REF] Krell | General investigations on processing tool steel X40CrMoV5-1 with selective[END_REF] [6] [START_REF] Mertens | Influence of powder bed preheating on microstructure and mechanical properties of H13 tool steel SLM parts[END_REF] or 4340 steel [START_REF] Jelis | Metallurgical and mechanical evaluation of 4340 steel produced by direct metal laser sintering[END_REF]. These grades undergo a complex transformation route including solidification in ferrite delta or austenite followed by further solid phase transformations to form tempered martensite or bainite.

The microstructures produced by LB-PBF process differ from those obtained by conventional methods and consequently the mechanical properties. First, the solidified microstructure often exhibits a cellular microstructure, reported by many studies, on different alloys [START_REF] Krell | General investigations on processing tool steel X40CrMoV5-1 with selective[END_REF] [START_REF] Holzweissig | Microstructural Characterization and Mechanical Performance of Hot Work Tool Steel Processed by Selective Laser Melting[END_REF] [10] [START_REF] Mostafa | Structure, Texture and Phases in 3D Printed IN718 Alloy Subjected to Homogenization and HIP Treatments[END_REF] [START_REF] Depinoy | Experimental determination of solute redistribution behavior during solidification of additively manufactured 316L[END_REF]. The authors suggest that this cellular microstructure (cells between 0.3µm and 1µm) results from segregations during very fast solidification. Saidi et al. [START_REF] Saeidi | Transformation of austenite to duplex austenite-ferrite assembly[END_REF] and Depinoy and al. [START_REF] Depinoy | Experimental determination of solute redistribution behavior during solidification of additively manufactured 316L[END_REF] [14] detected the segregation of chromium and molybdenum in austenite borders in 316L steel by mean of Energy Dispersive Spectroscopy. When an additional solid phase transformation takes place, for tool steels, a dual phased microstructure is observed, as for H13 tool steel (0.4% C, 5.1% Cr, 1.35% Mo and 1.1% V). Holzweissig and al. [START_REF] Holzweissig | Microstructural Characterization and Mechanical Performance of Hot Work Tool Steel Processed by Selective Laser Melting[END_REF], Krell and al. [START_REF] Krell | General investigations on processing tool steel X40CrMoV5-1 with selective[END_REF], Deirmina and al [START_REF] Deirmina | Heat treatment and properties of a hot work tool steel fabricated by additive manufacturing[END_REF] and Yuan and al [START_REF] Yuan | Characteristics of a modified H13 hotwork tool fabricated by means of laser beam powder bed fusion[END_REF] studied a modified H13 and reported a microstructure composed of martensite cells (diameter 0.5-2µm) surrounded by narrow borders of austenite (thickness 0.1-0.2µm). This microstructure significantly differs from that observed on conventional material. Krell and al. [START_REF] Krell | General investigations on processing tool steel X40CrMoV5-1 with selective[END_REF] estimated the segregation of alloying elements (C, Cr and Mo) by numerical simulation, using a Scheil-Gulliver model, and showed that it can prevent austenitic borders from transforming into martensite [START_REF] Krell | General investigations on processing tool steel X40CrMoV5-1 with selective[END_REF], an idea also suggested by Holzweissig and al. [START_REF] Holzweissig | Microstructural Characterization and Mechanical Performance of Hot Work Tool Steel Processed by Selective Laser Melting[END_REF], but no experimental evidence of carbon segregation was reported yet. Birnbaum and al. [START_REF] Birnbaum | Intrinsic strain ageing, Σ3 boundaries and origins of cellular microstructure in additively manufactured 316L[END_REF] proposed another mechanism: they suggested that the cellular microstructure doesn't emanate from segregation solidification in the liquid state but results from the segregation path in the solid state induced by the dislocation network formed during the fast cooling.

During the LB-PBF process, successive laser pathways induce complex heat flows in the material. On 316L, Shifeng and al [START_REF] Shifeng | Effect of molten pool boundaries on the mechanical properties of selective laser melting parts[END_REF] showed that the mechanical properties, especially toughness, depends on the type of molten pool boundaries. Krakhmalev and al. [START_REF] Krakhmalev | In situ heat treatment in selective laser melted martensitic AISI 420[END_REF] simulated the thermal cycling of different layers during the LB-PBF process and showed its influence on the microstructure. The previously built layers are tempered during further layer building. This in situ heat treatment directly impacts the microstructure and mechanical properties of LB-PBF parts, inducing a hardness heterogeneity between the first (softer) and the last (harder) layer. Martens and al. [START_REF] Mertens | Influence of powder bed preheating on microstructure and mechanical properties of H13 tool steel SLM parts[END_REF] and Holzweissig and al. [START_REF] Holzweissig | Microstructural Characterization and Mechanical Performance of Hot Work Tool Steel Processed by Selective Laser Melting[END_REF] observed this phenomenon on H13 steel, caused by tempering of the martensite in the lower layers. . Mertens and al. [START_REF] Mertens | Influence of powder bed preheating on microstructure and mechanical properties of H13 tool steel SLM parts[END_REF] also showed that a base plate temperature above Ms can modify the mechanical response of the H13 and attributed this phenomenon to bainite formation.

Hence, the solidification route and the inherited microstructure of LB-PBF build parts are complex and far from being completely understood, especially in tool steels with solid phase transformation upon cooling as well as evidences concerning the presence of carbon segregations.

H13 has been mechanically characterized in the As-built condition [START_REF] Lee | Microstructural effects on the tensile and fracture behavior of selective laser melted H13 tool steel under varying conditions[END_REF] Multiple teams [START_REF] Deirmina | Heat treatment and properties of a hot work tool steel fabricated by additive manufacturing[END_REF] [21] investigated the mechanical properties of H13 tool steel after direct tempering and quench and tempering, showing that tensile properties of additively built sample can reach the same level compared to conventional sample. Notably, Deirmina and al [START_REF] Deirmina | Heat treatment and properties of a hot work tool steel fabricated by additive manufacturing[END_REF] evidenced that As-built H13 material can achieve high fracture toughness despite a high hardness. On a modified H13 material, Yuan and al. [START_REF] Yuan | Characteristics of a modified H13 hotwork tool fabricated by means of laser beam powder bed fusion[END_REF] showed that As-built condition presents a lower impact toughness compared to quench and temper. Further characterizations are required, especially on impact and fracture toughness of additively built samples and the effect of heat treatment.

The present study is focused on the mechanical properties of LB-PBF-build 33CrMoV12 steel grade. This steel grade can be nitrided from 0,1 to 1mm [START_REF] Jegou | Gaseous nitriding behaviour of 33CrMoV12-9 steel: Evolution of the grain[END_REF] to be used for slightly or heavily loaded gears. Four different states (As-built, single tempered, double tempered and quenched-tempered) are analyzed and the resulting tensile and impact toughness properties, critical properties for an industrial application, are compared to conventional wrought material. A deep characterization of the inherited microstructures allows to discuss differences in mechanical properties, i.e. static and impact toughness, for several heat treatments. In addition, high resolution SIMS (Secondary Ion Mass Spectroscopy) experiments were performed on As-built specimen to identify and quantify carbon segregations. The result of this research offers new insight on microstructure formation and the relation between these microstructures and the mechanical properties on martensitic steels. The LB-PBF process and its ability to manufacture low alloy steel products.

Materials and Methods

Material

Gas-atomized 33CrMoV12 powder was supplied by Erasteel/Irun, Spain. Table 1

Selective Laser Melting

An LB-PBF machine M290 from EOS equipped with an Yb fiber laser (wavelength: 1064nm, focus diameter: 100µm, maximal power: 400W) was used. Samples were built vertically by applying a scanning strategy with a rotation of 67° between layers. An argon atmosphere was applied during the fabrication. Different parameters were tested in order to minimize the porosity density. The selected parameter set was a layer thickness of 40µm, an output power of 350W, a scan velocity of 867mm/s and a hatch-distance of 0.11mm. To avoid cracking during building, the baseplate was heated to 200°C. These parameters produced an energy of 92 J/mm² and resulted in a porosity density below 0,1%.

Heat Treatments

The As-built samples were further tempered to reproduce the final thermal history of conventional wrought 33CrMoV12, i.e. austenitizing at 930°C for 1h30, oil quenching and tempering at 615°C for 3h followed by air cooling.

Three types of heat treatments were tested: Single Tempering (ST), Double Tempering (DT) and Quenching-Tempering (QT). Single Tempering (ST) and double tempering (DT) were performed at 615°C for 3h followed by air cooling. Quenching-tempering (QT) consisted in an austenitization at 930°C for 1h30 followed by oil quenching then a tempering at 615°C for 3h followed by air cooling.

Microstructural characterization

Microstructural observations were performed on sample cross sections perpendicular to the direction of building polished down to 1µm and etched in 4% Nital. The SEM images were recorded using the secondary electron detector on the EVO25 and the SUPRA 40 from ZEISS.

The residual austenite was quantified by X-ray diffraction (XRD). The diffractometer (Siemens D5000) was equipped with a linear detector and the Cr-Kα radiation (kα= 2.29 A , 40 kV, 30 mA, step size= 0.04°, opening= 16°). To avoid texture effects, the samples were rotated 360° around the azimuthal angle and tilted up to 75°. The quantitative analysis was performed using the Rietveld method.

High-resolution SIMS (Secondary Ion Mass Spectrometry) imaging was used to study the carbon distribution in the As-built sample. The NanoSIMS instrument was operated with a cesium source to analyze carbon as negative ion. The focused primary beam (size of about 150 nm) was scanned over an area of (40 × 40) µm 2 down to (10 × 10) µm 2 . Analyzed areas of interest were systematically pre-sputtered to remove carbon contamination on the steel surface. Acquisition times were short (maximum 10 minutes) to limit the redeposition of carbon during the measurement. Further details on analytical methodology can be found in [START_REF] Valle | Study of the carbon distribution in multi-phase steels using nanoSIMS 50[END_REF]. With these analytical conditions, the detection limit of carbon can be as low as 60 ppm. To calibrate the carbon content of the measurements, the mean value of the cartography was set to 0.37%.

Mechanical characterization

Tensile tests, V-notch impact tests and hardness measurements were carried out on the As-built and heat treated samples.

Previous researches showed that vertical building is the most harmful regarding the mechanical properties of the samples [START_REF] Chen | Microstructure and composition heterogeneity, tensile property and underlying thermal physical mechanism of selective laser melting tool steel parts[END_REF] [START_REF] Dilip | Selective laser melting of HY100 steel : Process parameters microstructure and mechanical properties[END_REF]. Hence, in order to study the worst configuration, this building direction was used in the present work.

Tensile tests were performed at a constant strain rate of 0.05min -1 by using a ZWICK Z100 machine according to NF EN 2002-001/06. Cylindrical tensile samples with a gauge length of 38mm and a diameter of Ø=3.99mm were used.

V-notch impact tests were performed on a ZWICK RKP450 machine according to NF EN ISO 148-1/17 with a nominal energy of 300 J. For each heat treatment condition, three samples were tested.

Additional

Vickers micro hardness measurements were carried out using a 5100 series durometer from Buehler with a load of 1kg on cross sections perpendicular to the building direction. For each heat treatment condition, ten hardness values were measured and then averaged. XRD was performed on As-built, QT and ST specimens, plotted with the relative intensity I/Imax as a function of the diffraction angle (Fig. 2). The three samples show the presence of a bcc structure corresponding to ferrite, martensite or bainite. In the Asbuilt sample, austenite is also detected. A quantitative analysis indicates a volume fraction of austenite of 25%. As expected, no austenite is detected on QT sample, as well as on ST sample. Deirmina and al. [START_REF] Deirmina | Heat treatment and properties of a hot work tool steel fabricated by additive manufacturing[END_REF] also observed a decomposition of the austenite on H13 tool steel after tempering above 600°C. It should be noted that the half width of the bcc peak of As-built sample is broader compared to ST and QT sample, indicating the possible presence of high dislocation density [START_REF]Microstructural parameters from X-ray diffraction peak broadeningT. UNGAR[END_REF]. The cellular microstructure appears on this map but not as obviously as on SEM images, as the spot size is 150mm for a 200mm border width of the network. Blue cells with a low carbon concentration are surround by red borders up to seven times richer in carbon content. This level is very high compared to conventional 33CrMoV12 (375Hv) and even compared to H13 and X33CrMoNiW3-2 tool steel (530Hv), [START_REF] Yuan | Characteristics of a modified H13 hotwork tool fabricated by means of laser beam powder bed fusion[END_REF] which have far more carbide forming alloying elements. Hardness drops after ST and QT is still harder than conventional 33CrMoV12. The results of tensile tests are given in fig 2 . Properties of conventional material are extracted from [START_REF] Aubert&duval | GKH (W-YW) 33CrMoV12-9 High-performance martensitic steel for critical mechanical components[END_REF]. LB-PBF samples exhibits better tensile properties and are harder than conventional samples. Among these LB-PBF samples, As-built samples have the highest YS0.2 and UTS, as well as the highest hardness. A single tempering causes a slight hardness and YS0.2 drop while it increases the elongation (E%), as expected on a steel with no secondary hardening [START_REF] Grange | Hardness of Tempered Martensite in Carbon and Low Alloy Steels[END_REF]. . Applying a second tempering to ST samples causes a noticeable softening. ST samples have better tensile properties than DT or QT samples; these latter are both close to the conventional material. It should be noted that QT samples are harder than conventional 33CrMoV12 even if their heat treatments are identical.

Results

Microstructure

Mechanical properties of As-built, ST, DT and QT samples

For all tensile specimens, the fracture surfaces show both dimples and cleavage, indicating a mixed rupture mode. These fracture surfaces do not reveal any LB-PBF defects, such as gas porosity or lack of fusion.

A XRD analysis was carried on the As-built sample: the austenite content decreases in the vicinity of the fracture surface. As it can be seen on Fig 6, LB-PBF samples have lower impact toughness than conventional ones. However, LB-PBF QT samples almost reaches the impact toughness value of the conventional ones, despite its cellular microstructure. Tempering the samples after their fabrication causes an important drop in impact toughness. As-built samples show mostly ductile fracture surface, with clear shear lips (fig. 7a). Conversely, ST and DT samples show a brittle fracture with cleavage facets. Extremely low ductility is achieved as no shear lips are observed (Fig. 7c), even though the tensile elongation is higher than 14% (Fig. 5.). Even though presenting very similar tensile properties, the second tempering doesn't allow to restore the impact toughness to the level of conventional samples. As-built samples show a preferential failure orientation as observed on the fractured zone, with parallel strips of 100µm width (red double arrows on Fig. 7b). These strips may correspond to the laser pathways of the LB-PBF process, since the strip width (100µm) is close to the hatch spacing used (110µm). These strips are observed only on As-built samples.

Regarding mechanical properties, As-built samples have the best compromise between UTS and E%. Morever, their impact toughness, even if it is not as high as for conventionnal samples-is sufficient to ensure a ductile fracture during V-notch impact tests.

Discussion

Cellular microstructure

The quantitative XRD analysis indicates a volume fraction of austenite of 25%, in the same order of magnitude of the network proportion obtained by image analysis and consistent with the previous works on H13 tool steel by Deirmina and al [START_REF] Deirmina | Heat treatment and properties of a hot work tool steel fabricated by additive manufacturing[END_REF]and Krell and al. [START_REF] Krell | General investigations on processing tool steel X40CrMoV5-1 with selective[END_REF]. As-built samples of 33CrMoV12 contain a high amount of austenite (25%), located in the intercellular areas surrounding the martensitic cells. This result is unexpected in a 33CrMoV12 alloy that has an Ms temperature of 375°C. Conventional 33CrMoV12 is thus fully martensitic at room temperature. The cellular microstructure reported in the present study is similar to the LB-PBF-inherited microstructures observed on other materials: Al-Si alloys, CoCrMo alloys [START_REF] Prashanth | Formation of metastable cellular microstructures in selective laser[END_REF], austenitic stainless steels [START_REF] Prashanth | Formation of metastable cellular microstructures in selective laser[END_REF] [27] [START_REF] Cui | Cyclic response of additive manufactured 316L stainless steel: The role of cell structures[END_REF] and on martensitic stainless steels [START_REF] Krell | General investigations on processing tool steel X40CrMoV5-1 with selective[END_REF] [START_REF] Freeman | Influence of solidification cell structure on the martensitic transformation in additively manufactured steels[END_REF]. Its exact mechanism of formation remains to be understood, even though the role of segregation during solidification was reported [START_REF] Prashanth | Formation of metastable cellular microstructures in selective laser[END_REF]. For martensitic stainless steels, the austenite stabilizing effect of the small size of the cell was discussed by Freeman and al. [START_REF] Freeman | Influence of solidification cell structure on the martensitic transformation in additively manufactured steels[END_REF] and demonstrated as insufficient to explain the thermal stability of austenite even in liquid nitrogen Nano-SIMS experiment shows that the regions of high carbon content correspond mostly to the bright borders revealed in the etched microstructure (Fig 1 a and Fig. 3). Some borders have a carbon concentration as high as 1.2%. Using the Andrew's formula [30], such a carbon content leads to an Ms of -30°C. This negative Ms prevents the austenite borders from transformation into martensite during the building and the subsequent cooling down to room temperature. In some other austenite borders, carbon content is not high enough hence Ms remains above room temperature but below the baseplate temperature, the austenite borders are still stable during LB-PBF building. If the carbon content is low enough to keep Ms above the baseplate temperature, for borders with less than 0,65% of carbon, austenite can transform into martensitic during the building sequence.

Krell and al. [START_REF] Krell | General investigations on processing tool steel X40CrMoV5-1 with selective[END_REF] as well as Deirmina [START_REF] Deirmina | Heat treatment and properties of a hot work tool steel fabricated by additive manufacturing[END_REF] evidenced that LB-PBF H13 has a similar cell-like microstructure consisting of austenite and martensite. Based on a numerical simulation, Krell and al. [START_REF] Krell | General investigations on processing tool steel X40CrMoV5-1 with selective[END_REF] have suggested that carbon segregation occurs during the solidification. Their proposed solidification route is a germination on delta ferrite followed by an enrichment of the liquid phase that ultimately lead to the stabilization of austenite. Their numerical simulation shows that austenite borders are up to eight time richer in carbon than martensitic cells. The present work is in agreement with their results and bring experimental evidences of carbon segregation in austenitic borders have been found with nano-SIMS carbon repartition maps, with a maximal carbon content ratio between borders and cells of seven This demonstrates that the cooling rates reached on LB-PBF are high enough to freeze segregation of carbon on a nanometric scale and the reheating from the subsequent layers is not sufficient to decompose the austenite or homogenize the chemical composition. Recent studies on H13 steel showed that the cellular-like microstructure is not a specificity of additive manufacturing. Raw powder [START_REF] Olievera | Effect of microstructure and defect formation on the bending properties of additive manufactured H13tool steel[END_REF] also presents the cell like microstructure. Laser surface treatment [START_REF] Telasang | Structure-property correlation in laser surface treated AISI H13 tool steel for improved mechanical properties[END_REF]can also achieve cellular like structure. This supports the idea that the key factor for the cellular like structure is the melting of the material and a fast cooling rate to freeze the microstructure. This work didn't intend to settle the debate between the solidification segregation theory and the dislocation segregation theory [START_REF] Birnbaum | Intrinsic strain ageing, Σ3 boundaries and origins of cellular microstructure in additively manufactured 316L[END_REF], but the carbon levels observed by Nano-SIMS are consistent with Krell's simulations on solidification segregations.

Concerning the stability of the cellular structure after tempering, XRD on Fig. 2 shows that the austenite of 33CrMoV12 is decomposed after a single tempering. This results is in agreement with Deirmina and al. [START_REF] Deirmina | Heat treatment and properties of a hot work tool steel fabricated by additive manufacturing[END_REF] that evidenced the decomposition of the austenite border on H13 tool steel during tempering by using dilatometry. It should be noted that on a modified H13, X33CrMoNiW3-2 designed for improved elevated temperature resistance, Yuan and al. [START_REF] Yuan | Characteristics of a modified H13 hotwork tool fabricated by means of laser beam powder bed fusion[END_REF] observed a persistence of the austenite after a double direct tempering of 2h at 625°C. Hence, other substitutional elements can be involved in the stabilization on austenite depending on the chemical composition

Mechanical properties

As-built samples present the highest tensile properties (Fig. 5), significantly higher than those of the conventional 33CrMoV12. Their high strength can be attributed to the duplex microstructure of carbon enriched austenite and martensite cells that leads to a high density of phase interfaces, as well as a low equivalent tempering temperature during the reheating phases of the building sequence. XRD analysis was carried out on tensile tested sample and the austenite fraction near the fracture surface decreases, indicating a possible transformation into martensite during the tensile test, presumably by TRIP effect [START_REF] Zackay | The enhancement of ductility in high-strength steels[END_REF]. On H13, stable austenite was observed by Holzweissig and al. [START_REF] Holzweissig | Microstructural Characterization and Mechanical Performance of Hot Work Tool Steel Processed by Selective Laser Melting[END_REF] after deformation. As the H13 steel presents a higher content alpha forming elements such as vanadium, chromium and silicon, and a slightly higher content of carbon, a very strong gamma forming elements, further studies are required to understand why the 32CrMoV12-9 present this behavior. In addition, the martensite confined in an austenitic network may present a high dislocation density, as suggested by Figure 2, enhancing the yield stress. On 316L austenitic steel, without any phase transformation, high dislocation density are reported [START_REF] Dilip | Selective laser melting of HY100 steel : Process parameters microstructure and mechanical properties[END_REF] [12] [START_REF] Li | Fine-grain-embedded dislocationcell structures for high strength and ductility in additively manufactured steels[END_REF]. The combination of the duplex microstructure, the high dislocation density and TRIP effect can explain why the hardness of the 33CrMoV12 in the As-built condition is close to H13 or X33CrMoNiW3-2 [START_REF] Yuan | Characteristics of a modified H13 hotwork tool fabricated by means of laser beam powder bed fusion[END_REF] despite the difference of chemical composition. Moreover, As-built samples present an impact toughness of 40J, not as high as the conventional material but sufficient to ensure a ductile fracture as shown by Fig. 7. On H13 tool steel, Deirmina and al. [START_REF] Deirmina | Heat treatment and properties of a hot work tool steel fabricated by additive manufacturing[END_REF] also reported surprisingly high toughness of As-built samples. These good results can be attributed to both ductile austenite network and, presumably the composite effect of LB-PBF building. As observed on Fig. 7 b), the fracture surface is stripped with a characteristic distance consistent with hatch spacing, indicating preferential crack propagation. During the process, on each layer, differences in cooling rates and chemical compositions create a microstructure gradient which is repeated layer after layer. Further studies need to be carried out on As-built samples with different hatch spaces, still maintaining good density, to assess this effect. Still, As-built samples present an interesting compromise between strength and impact toughness and the thermal and mechanical stability of the austenite network will require further studies.

ST samples results are also worth mentionning because, even after tempering, their tensile properties remain higher than those of conventional 33CrMoV12 and close to the As-built samples. V-notch impact tests showed that ST samples have a low impact toughness (absorbed energy = 13J, Fig. 5). In these samples, XRD analysis does not detect any evidence of austenite. Hence tempering at 615°C allows the decomposition of austenite into carbides and fresh martensite and/or bainite, while the thermal affectation between the layers with the used set of parameters during building was insufficient to achieve this decomposition. The presence of brittle carbides in tempered samples can explain the low impact toughness and the lack of ductility While in the same time, fresh martensite in ST sample and the remnant cell structure can contribute to the increase of mechanical properties and hardness.

DT samples still present a low impact toughness and a significant drop on yield strength and ultimate tensile strength as reported on Fig. 3, showing that a second tempering cannot increase significantly the impact toughness of ST samples. These low results can thereby be mainly attributed to the carbides formed after the first tempering treatment, an increase of the ductility can be noted on Fig. .5, showing that fresh martensite or bainite is tempered during the second tempering treatment. This fresh martensite may also counterbalance the tempering of the martensite formed after the building, leading to the relatively small drop in tensile properties between As-built and ST samples. Dermina and al. [START_REF] Deirmina | Heat treatment and properties of a hot work tool steel fabricated by additive manufacturing[END_REF] reported high fracture toughness on DT H13 tool steel samples, however, coarse carbides can have a more detrimental effect on impact toughness compared to fracture toughness [START_REF] Im | Effects of carbide precipitation on the strength and Charpy impact properties of low carbon Mn-Ni-Mo bainitic steels[END_REF] [START_REF] Lee | Effect of carbide distribution on the fracture toughness in the transition temperature region of an SA 508 steel[END_REF]. On conventional material, impact toughness and fracture toughness are widely considered to be positively correlated, hence, an assessment of the fracture toughness of the 33CrMoV12 and a comparison with the results of Dermina and al. [START_REF] Deirmina | Heat treatment and properties of a hot work tool steel fabricated by additive manufacturing[END_REF] would be interesting.

QT samples have very close mechanical properties (tensile and impact toughness) to the conventional material. The slight gap can be attributed to a difference of grain size or chemical composition, the powder used for LB-PBF being slightly enriched in carbon as shown in Table 1 . The difference between QT samples and the other LB-PBF samples is the austenizing-quenching step. As the characteristic size of a cell is 700nm to 3µm, the carbon diffusivity is sufficient to ensure homogenization of the carbon micro segregations during austenitizing. Only primary martensite is formed during quenching and erases the microstructure inherited from LB-PBF. An adapted building strategy can achieve the same result during LB-PBF process by reheating the As-built layer with overlapping [START_REF] Chen | Microstructure and composition heterogeneity, tensile property and underlying thermal physical mechanism of selective laser melting tool steel parts[END_REF] and allow homogenization without further treatment. It should also be noted that if a quench and temper heat treatment is performed, the soaking time of the austenitizing must be adapted to the additively manufactured material as this step must ensure the carbon homogenization in addition to the dissolution of carbides and austenitizing compared to conventional low alloy treatment.

As impact toughness is restored after a conventional quench and temper heat treatment of the additively manufactured samples, the low impact toughness of ST and DT samples appears strictly linked to the microstructure and thermomechanical process route. These results discard chemical related rootcause such as difference in oxygen or nitrogen content. The rough intergranular carbides observed in Fig. 1 b) are detrimental to impact toughness properties compared to fine homogenous precipitation of conventional 33CrMoV12 steel.

Conclusion

In this study, the relation between microstructures and mechanical properties of LB-PBF-build 33CrMoV12 steel grade has been investigated by considering four material states: As-built, ST, DT and QT. Tensile and impact toughness properties as well as their microstructures have been characterized on these thermal states and compared to conventional wrought material. The main results are described below: 1) A network microstructure composed of austenite borders surrounding martensite cells is observed on As-built samples. Experimental evidences of nano-sized carbon segregations are found in the As-built microstructure using nano-SIMS. Nano-sized carbon enriched austenite borders, seven time richer than the cells, are stabilized at room temperature to a total volume fraction of 25%. These experimental evidences are consistent with the formation mechanism of austenite network proposed by Krell and al. [START_REF] Krell | General investigations on processing tool steel X40CrMoV5-1 with selective[END_REF] 2) As-built samples have the highest tensile properties and a good impact toughness. These properties are attributed to the specific microstructure produced by LB-PBF manufacturing and the occurrence of TRIP effect.

3) After single tempering, samples present a low impact toughness due to the precipitation of brittle phases resulting from the decomposition of the austenite. A second tempering does not fully restore impact toughness by tempering the brittle phase. This low impact toughness is thereby attributed to brittle coarse carbides precipitation and fresh martensite formation is evidenced by the increase of the ductility after a second tempering and the persistence of high tensile properties after the first tempering. 4) QT samples have mechanical properties very close to the conventional 33CrMoV12.

Compared to the different LB-PBF samples, they present the highest impact toughness but the lowest tensile properties. The austenizingquenching homogenizes carbon and erases the micro segregations inherited from LB-PBF process.

Finally, as the tensile properties are not degraded by LB-PBF process it can be promising to use this process on 33CrMoV12 steel grade in order to achieve parts with a complex geometry. Industrial parts require a compromise between impact toughness and tensile properties. For transmission parts, they must be compatible with a surface treatment like carburizing or nitriding. Samples that have the best compromise are As-built and QT samples. As built samples are better regarding tensile properties but have a lower impact toughness than QT samples. To complete this work, further studies are required to investigate the influence of the microstructure of As-built and QT samples on thermochemical treatment.

On large mechanical parts and with an appropriate composition and heat treatment, martensitic steels produced by additive manufacturing could challenge the staticimpact toughness tradeoff of baintic steels as the obtained properties are independent of the thickness of the parts, even if weldabilty of this kind of structures remains to be studied.

Fig 1 :

 1 Fig 1: SEM images of Nital etched As-built (a) and ST samples (b) (circle in red elongated cells and circle in blue equiaxed cells), Fig 1 shows the microstructure of the etched As-built and ST samples. For As-built samples (Fig 1.a), the etched surface is composed of bright borders surrounding dark cells. These cells and borders are typically respectively between 0.7µm and 3µm of diameter and 200nm of width. According to an image analysis using ImageJ, bright borders represents 26% of the surface. Depending on the polished plane, equiaxed cells (blue ellipse on Fig 1.a) or elongated cells (red ellipse on Fig 1.a) are observed, probably indicating a tubed shaped microstructure. In the elongated cells, a substructure similar to martensite or bainite can be observed on the etched surface. This microstructure is observed in the whole sample even in the last layer. After tensile tests on As-built specimens, this cellular microstructure remains present, even close to the surface fracture. However, ST samples do not exhibit the cellular microstructure anymore (Fig 1.b), but bright rough intergranular carbides are observed. These carbides can even achieve a continuous network.

Fig 2 :

 2 Fig 2: X-ray diffraction patterns for As-built sample (red), ST sample (grey) and QT sample (blue) Nano-SIMS was performed on the cross section of As-built samples to obtain the carbon distribution map shown in Fig 3. The color scale gives the carbon concentration in atomic percent.

Fig 3 :

 3 Fig 3 : Nano-SIMS carbon repartition map on the cross section of the As-built sample. The color scale gives the carbon concentration in percent.

Fig 4

 4 Fig 4 shows the mean hardness of As-built, ST, QT samples and conventional 33CrMoV12. As-built sample achieve the highest hardness reaching almost 500Hv. This level is very high compared to conventional 33CrMoV12 (375Hv) and even compared to H13 and X33CrMoNiW3-2 tool steel (530Hv),[START_REF] Yuan | Characteristics of a modified H13 hotwork tool fabricated by means of laser beam powder bed fusion[END_REF] which have far more carbide forming alloying elements. Hardness drops after ST and QT is still harder than conventional 33CrMoV12. The results of tensile tests are given in fig 2.Properties of conventional material are extracted from[START_REF] Aubert&duval | GKH (W-YW) 33CrMoV12-9 High-performance martensitic steel for critical mechanical components[END_REF].

Fig 4 :

 4 Fig 4: Micro-hardness (HV1) of As-built, single tempering (ST), quenching-tempering (QT) and conventional samples.

Fig 6 :

 6 Fig 6: Impact toughness of As-built, single tempering (ST), double tempering (DT), quenching-tempering (QT) and conventional samples at room temperature.Fig 6 shows the fracture energy of the different V-notch impact test samples. Fig 7 shows the fracture surface of As-built and ST V-notch impact test samples. Blue arrows indicate the position of the V notch.

Fig 7 :

 7 Fig 7: Fracture surface of As-built V-notch impact test samples (a) at low magnification and (b) at higher magnification, (c) fracture surface of ST V-notch impact test samples. Blue arrows indicate the position of the notch. Red circles indicate the fabrication direction. Red arrows on (b) highlight the presence of parallel strips of 100µm width.

Table 1 :

 1 Chemical composition of the 33CrMoV12 alloy

						summarized
	the chemical composition (in wt.%) of the atomized
	powder and standard AMS6481 33CrMoV12 alloy. Its
	Martensitic Start temperature (Ms) is 375°C [23]. The O
	content of the raw powder is 0.04%.
	Element	C	Cr	Mo	V	Mn	Si	Fe
	Wrought	0.29	2.80	0.70	0.15	0.40	0.10
	33CrMoV	-	-	-	-	-	-	Bal.
	[23]	0.36	3.30	1.20	0.35	0.70	0.40
	Powder	0.37	3.1	1.03 0.24 0.58 0.37 Bal.

Acknowledgement

This work was funded by Safran Transmission Systems

Research & Technology program. The authors gratefully acknowledge Volum-e company for the fabrication of LB-PBF samples. We also wish to thank Nathalie Valle of Materials Research and Technology (MRT) department of Luxembourg Institute of Science and Technology (LIST) who performed the Nano-SIMS characterization.

Références


Jing Xing

Contrastive Self-Supervised Learning on Crohn's Disease Detection

Keywords: Deep Learning, Self-Supervised Learning, Contrastive Learning, Crohn's Disease, Medical images classification

Crohn's disease is a type of inflammatory bowel illness that is typically identified via computer-aided diagnosis (CAD), which employs images from wireless capsule endoscopy (WCE). While deep learning has recently made significant advancements in Crohn's disease detection, its performance is still constrained by limited labeled data. We suggest using contrastive self-supervised learning methods to address these difficulties which was barely used in detection of Crohn's disease. Besides, we discovered that, unlike supervised learning, it is difficult to monitor contrastive self-supervised pretraining process in real time. So we propose a method for evaluating the model during contrastive pretraining (EDCP) based on the Euclidean distance of the sample representation, so that the model can be monitored during pretraining. Our comprehensive experiment results show that with contrastive self-supervised learning, better results in Crohn's disease detection can be obtained. EDCP has also been shown to reflect the model's training progress. Furthermore, we discovered some intriguing issues with using contrastive selfsupervised learning for small dataset tasks in our experiments that merit further investigation.

I. INTRODUCTION

Crohn's Disease is a common bowel disease [START_REF] Eliakim | The impact of panenteric capsule endoscopy on the management of crohn's disease[END_REF]. Wireless Capsule Endoscopy (WCE) is commonly used by doctors to aid in diagnosis. However, counting and identifying lesions of Crohn's disease in WCE videos is time-consuming. Many methods [START_REF] Soffer | Deep learning for wireless capsule endoscopy: a systematic review and meta-analysis[END_REF] have been developed to automatically detect intestinal abnormalities in WCE images, such as ulcer, bleeding, and erosion. These methods can be divided into two main categories. One is using Support Vector Machine(SVM) [START_REF] Hajo-Maghsoudi | Segmentation of crohn, lymphangiectasia, xanthoma, lymphoid hyperplasia and stenosis diseases in wce[END_REF]- [START_REF] Souaidi | Multi-scale analysis of ulcer disease detection from wce images[END_REF] with manually designed features. The other is based on deep learning [START_REF] Georgakopoulos | Weakly-supervised convolutional learning for detection of inflammatory gastrointestinal lesions[END_REF]- [START_REF] De Maissin | Multi-expert annotation of crohn's disease images of the small bowel for automatic detection using a convolutional recurrent attention neural network[END_REF]. In the early years, researchers primarily used SVM as classifiers based on manually designed features such as local binary pattern [START_REF] Yu | Ulcer detection in wireless capsule endoscopy images[END_REF], scale invariant feature transform [START_REF] Yu | Ulcer detection in wireless capsule endoscopy images[END_REF], texture feature [START_REF] Che | MM'12 : proceedings of the 20th ACM International Conference on Multimedia[END_REF], color feature [START_REF] Li | Computer-based detection of bleeding and ulcer in wireless capsule endoscopy images by chromaticity moments[END_REF], etc. However, deep learning has demonstrated tremendous power in a number of fields in recent years. The accuracy of Crohn's disease detection has increased significantly with deep neural networks to over 90% [START_REF] Soffer | Deep learning for wireless capsule endoscopy: a systematic review and meta-analysis[END_REF].

As shown in Fig. 1, the images for Crohn's disease detection are taken from WCE video, thus there are numerous unlabeled images. Nevertheless, because labeling WCE image is very time-consuming and requires professional doctors, labelled data is scarce. As a result, deep learning's effectiveness in the field of medical images, including Crohn's disease detection, is massively diminished. In this situation, self-supervised learning is a viable option. Given that it can train the network with unlabeled data to make up for the lack of enough labeled data. Self-supervised methods based on contrastive learning have recently been proven to be more effective in image classification tasks. The CrohnIPI [START_REF] Vallée | CrohnIPI: An endoscopic image database for the evaluation of automatic Crohn's disease lesions recognition algorithms[END_REF] dataset that we use is a professional public dataset that contains well-labeled data. To the best of our knowledge, existing research works use only self-supervised learning for Crohn's disease detection. Besides, most self-supervised studies use very large datasets as benchmark, such as ImageNet [START_REF] Deng | Imagenet: A large-scale hierarchical image database[END_REF]. However, WCE images are usually small and have few categories which is very different to ImageNet. Considering the gap between ImageNet and WCE images, we compare several state-of-the-art contrastive self-supervised learning methods' performance on WCE data to find out that whether self-supervised learning can improve the performance of Crohn's disease detection.

During our experiments, we discovered that self-supervised training, also referred pretraining, unlike supervised training, cannot use the validation set to observe the pretraining process. This also makes selecting model of pretraining and conducting early-stopping impossible. We can only use finetuning to verify the model's performance after pretraining, which significantly prolongs the model's tuning process and makes selecting the best model during the pretraining process for downstream tasks difficult. Therefore, we propose evaluation during contrastive pretraining (EDCP), a low-cost method for real-time monitoring of pretraining progress. Based on the Euclidean distance between sample representations, this method computes a value that reflects how well network is trained. We compare the EDCP to the results of few-epoch finetuning and discovered that the EDCP results are consistent with the few-epoch finetuning results, which demonstrats that EDCP can be used effectively to monitor contrastive selfsupervised learning pretraining.

978-1-6654-6819-0/22/ $31.00 ©2022 IEEE Fig. 1. Traditional supervised learning (top), our contrastive self-supervised learning method (bottom). Evaluation During Contrastive Pretraining(EDCP) is used to monitor pretraining(Bottom). The wireless capsule endoscopy (WCE) images are from CrohnIPI [START_REF] Vallée | CrohnIPI: An endoscopic image database for the evaluation of automatic Crohn's disease lesions recognition algorithms[END_REF]. Furthermore, we observed some contrastive learning techniques that are useful on large datasets but are ineffective or even harmful for Crohn's disease detection in experiments. A larger batch size, for example, will generally produce better results [START_REF] Chen | A simple framework for contrastive learning of visual representations[END_REF], but it has no effect on Crohn's disease detection, and even too many negative sample pairs will degrade the model's performance. These anomalous phenomena merit further investigation, as they may improve the model's performance and have some reference significance for similar classification tasks of small dataset.

The following are our primary contributions: (i)We are the first to apply self-supervised learning to the detection of Crohn's disease in images. (ii) We outperform traditional supervised learning in the detection of Crohn's disease. (iii) We propose using evaluation during contrastive pretraining (EDCP) to track the progress of contrastive self-supervised training.

II. PRELIMINARY AND RELATED WORK

Crohn's Disease Detection. Computer-aided diagnosis (CAD) has become an active search area in the past few years. WCE is crucial for the diagnosis of Crohn's disease because it allows medical professionals to see inside the patient's bowel in order to identify Crohn's disease as soon as possible. However, diagnosing with WCE images takes a long time for medical experts. Many methods for automatically detecting diseases in WCE based on deep learning have recently been proposed, and excellent results have been obtained [START_REF] Soffer | Deep learning for wireless capsule endoscopy: a systematic review and meta-analysis[END_REF]. Deep learning methods based on convolutional neural networks (CNN) have been proven to outperform prior machine learning methods which based on support vector machines (SVM) and manually designed features. However, there is still room for improvement due to the lack of labeled data. In fact, insufficient labeled data is a common issue in the medical imaging field. For example, Seguí et al [START_REF] Seguí | Generic feature learning for wireless capsule endoscopy analysis[END_REF] found that 10fold cross-validation increase in training data size improved the accuracy by 3% for motility movement classification in WCE. WCE images have the significant advantage of being easy to obtain a large number of unlabeled images since tens of thousands of frames are contained in each WCE video. To make use of these unlabeled data, we employ self-supervised learning.

Contrastive Self-Supervised Learning. Self-supervised learning (SSL) has recently achieved great success in a variety of fields. Liu et al. [START_REF] Liu | Self-supervised learning: Generative or contrastive[END_REF] summarize the mainstream self-supervised learning into three broad categories: generative, contrastive, and generative-contrastive. In generative tasks such as image colorization [START_REF] Zhang | Colorful image colorization[END_REF] and inpainting [START_REF] Pathak | Context encoders: Feature learning by inpainting[END_REF], generative and generative-contrastive learning are particularly effective. Contrastive learning, on the other hand, outperforms in classification tasks, particularly in computer versions (CV). Among contrastive learning methods, instance-instance contrast is proved perform better than context-instance contrast [START_REF] Liu | Self-supervised learning: Generative or contrastive[END_REF]. Many instance-instance [START_REF] Chen | A simple framework for contrastive learning of visual representations[END_REF], [START_REF] He | Momentum contrast for unsupervised visual representation learning[END_REF]- [START_REF] Caron | Unsupervised learning of visual features by contrasting cluster assignments[END_REF] have performed admirably on various CV benchmarks. The object of contrast in contrastive learning is commonly referred to as negative samples and positive samples. The different embeddings produced by transforming one image are referred to as positive samples from each other, whereas the embeddings produced by transforming different images are referred to as negative samples from each other. To obtain positive and negative samples, the contrastive learning network typically has two similar branches, and is therefore also known as siamese network [START_REF] Chen | Exploring simple siamese representation learning[END_REF]. The objective of contrastive learning is to maximize the similarity of negative pairs while minimize the similarity of positive pairs. InfoNCE [START_REF] Van Den Oord | Representation learning with contrastive predictive coding[END_REF] loss is commonly used to achieve this goal. The InfoNCE loss for sample q with distance measured by dot product is:

L q = -log exp (q • k + /τ ) K i=0 exp (q • k i /τ ) , ( 1
)
where τ is a temperature hyper-parameter. The sum is over one positive sample k + and K negative samples. Many studies [START_REF] Chen | A simple framework for contrastive learning of visual representations[END_REF], [START_REF] He | Momentum contrast for unsupervised visual representation learning[END_REF], [START_REF] Zbontar | Barlow twins: Self-supervised learning via redundancy reduction[END_REF] have shown that having a sufficient number of negative samples is critical for model performance and avoiding clloaps during pretraining.

State of the Art (SOTA) Methods. In the field of contrastive self-supervised learning, method iteration has been remarkably rapid in recent years. InvaSpread [START_REF] Ye | Unsupervised embedding learning via invariant and spreading instance feature[END_REF] presents an end-to-end method for contrastive learning. Following In-vaSpread, SimCLR made significant progress through refining the experiment and network structure including adding MLP head, more epochs of pretraining, much bigger batch size and more complex data augmentations. These manipulations are straightforward and effective, and they are followed by methods proposed later. Meanwhile, MoCo [START_REF] He | Momentum contrast for unsupervised visual representation learning[END_REF] introduced momentum encoder to reduce inconsistency caused by rapidly changing encoder, based on InstDisc [START_REF] Wu | Unsupervised feature learning via non-parametric instance discrimination[END_REF], which proposed memory bank that use a small amount of computational resources while producing a large number of negative samples. MoCo v2 later combined the useful SimCLR strategies on the basis of moco to achieve better results. Positive and negative pairs are sample-based in MoCo v2 and SimCLR, whereas in Barlow Twins [START_REF] Zbontar | Barlow twins: Self-supervised learning via redundancy reduction[END_REF], they are based on filters that generate embeddings. They believe that this will allow filters to learn more different features and reduce the redundancy of information. Barlow Twins does not require a large batch size or memory bank, but it does require more channels in the MLP layer in order to obtain enough negative samples. Most contrastive self-supervised learning methods proposed prior to BYOL [START_REF] Grill | Bootstrap your own latent-a new approach to self-supervised learning[END_REF] require negative samples to avoid trivial solutions. For example, if the network only needs to reduce the similarity of positive pairs, regardless of the input, the network can output the same feature vector. BYOL produces excellent results with only positive sample pairs and its performance is robust to small batch sizes. Soon after, SimSiam [START_REF] Chen | Exploring simple siamese representation learning[END_REF] proposed a simplified version of BYOL which remove momentum encoder and the predictor head while slightly reducing performance.

III. METHODS

A. Baseline

Our aim is to find out whether contrastive self-supervised learning can improve the performance of Crohn's disease detection. For comparison, the traditional supervised learning method is used. Following [START_REF] Vallée | CrohnIPI: An endoscopic image database for the evaluation of automatic Crohn's disease lesions recognition algorithms[END_REF], the network structure of baseline consists of an encoder and a fully connected layer. The encoder is ResNet [START_REF] He | Deep residual learning for image recognition[END_REF] without final fully connected layer.

B. Contrastive Self-Supervised Methods Comparison

SOTA methods mentioned in II can be divided into three categories according to the factors contrasted: positive and negative pairs based on instances(MoCo, MoCo v2, SimCLR, etc.), positive and negative pairs based on filters(Barlow Twins), and only positive pairs based on instances(BYOL, SimSiam). The best methods from each of these three categories for detecting Crohn's disease detection will be chosen.

We are primarily concerned with two aspects when comparing methods: performance and robustness to batch size. A general comparison of the performance of self-supervised learning is to compare the performance of linear evaluation on ImageNet which is shown in Table I. According to the Table I, it can be seen that MoCo v2, SimSiam, Barlow Twins, and BYOL perform well. Barlow Twins and BYOL stand out among them. When comparing the robustness of methods to batch size, the study [START_REF] Zbontar | Barlow twins: Self-supervised learning via redundancy reduction[END_REF] in Barlow Twins shows that the performance of Barlow Twins and BYOL are still good when the batch size decreases, while the performance of SimCLR drop a lot. Because the loss strategy in the BYOL and Barlow twins determines that their performance is less dependent on a large number of negative samples. MoCo v2's performance is also robust to batch size since its negative samples are primarily drawn from memory bank which is not influenced by batch size. As a result, three methods were chosen from the three categories based on their good performance and robustness of batch size: MoCo v2, BYOL, and Barlow Twins. Since the hyperparameters of each method are carefully tuned, we will try to keep the original hyperparameters of these methods during the experiment for fairness.

C. Framework

The three methods have similar structure which shown in the Fig. 2. The input image is augmented t and t ′ to produce two distinct views for two branches. The network's goal is that the representations generated by two branches for two views of the same image are similar, while the representations generated for different input images are dissimilar. The network, with the exception of the BYOL predictor, is made up of two symmetrical branches. Each branch is made up of an encoder f , which is a CNN followed by an projector network MLP. The encoder structure of the three networks is ResNet without final full-connected layer, but the projector structure is different. The projector MLP structures of MoCo v2, BYOL, and Barlow Twins are 512-128, 4096-256, and 8192-8192-8192, respectively. Following SimCLR, the projector MoCo v2 consists in a linear layer with output size 512 follwed by rectified linear units (ReLU) and a final linear layer with output dimension 256. The projector of BYOL consists in a linear layer with output size 4096 followed by batch normalization, ReLU, and a final linear layer with output dimension 256. The predictor of BYOL uses the same architecture as projector. The projector of Barlow Twins has three linear layers, each with 8192 output units. A batch normalization layer and ReLU are placed after the projector's first two layers. We can see that Barlow Twins require large output dimension to provide a large number of negative samples in order to perform well. Besides, despite the simplicity of MoCo v2's MLP structure, MoCo v2 requires memory banks to provide a large number of embeddings as negative samples.

The parameters of Barlow Twins are all updated by backpropagation. However, in the network structure of BYOL and MoCo v2, the stop gradient operation is performed at the end of one branch, and the parameter ξ of this branch is updated by exponential moving average (EMA) strategy which formulated by [START_REF] Soffer | Deep learning for wireless capsule endoscopy: a systematic review and meta-analysis[END_REF].Back-propagation updates the parameter θ of another branch.

ξ

← τ ξ + (1 -τ )θ (2)
where τ ⊂ [0, 1) is a coefficient. The embeddings used by the three methods to calculate the loss are different, as shown in the Fig. 2. MoCo v2 employs InfoNCE loss between projection z θ and embedding z m sampled from the memory bank. The loss in BYOL is calculated as the mean squared error between the l2normalized projection sg(z ′ ξ ) and the prediction q θ (z θ ). After normalize the embeddings along the batch dimension, Barlow Twins cleverly transposes one projection z θ and multiplies it with another projection z ′ ξ to produce a cross-correlation matrix with diagonal and off-diagonal elements representing the cosine similarity of positive and negative sample pairs, respectively. The identity matrix is used as the label, and the loss is calculated as the mean square error of the crosscorrelation matrix and the identity matrix in Barlow Twins.

D. Evaluation During Contrastive Petraining (EDCP)

The validation set is commonly used in supervised training to monitor training progress in order to avoid overfitting or insufficient training, and it can also be used to select models. However, because the evaluation of self-supervised learning is finetuning when pretraining is finished, evaluating the network during pretraining is difficult. Finetuning after each epoch during pretraining is a simple and straightforward method, but it adds a significant amount of time and computational resource consumption. Another straightforward approach is to monitor pretraining process directly through observing the loss, but these losses are not related to the classification task. First, this is an indirect method. There is inconsistency because the downstream task uses only the encoder rather than the entire pretraining network. Second, the loss may gradually decrease with training due to the strategy of some methods. During pretraining, for example, the momentum of BYOL will gradually increase, resulting in closer parameters of the two branches of the network, which brings the generated representation closer to each other and eventually leads to a smaller loss.

To address this problem, we design a method for real-time monitoring of model training progress that can be quickly computed and widely used in any contrastive learning pretraining process, called evaluation during contrastive pretraining (EDCP).

Based on the goal of contrastive learning, which is to decrease the positive sample pair's feature similarity close and increase the negative sample pair's feature similarity. Euclidean distance is used to measure similarity. We input a small amount of labeled data into the encoder to get the corresponding representations, and then calculate the Euclidean distance of the positive and negative sample pairs. Two subsets are sampled with no overlap, each subset has the same number of samples with different labels, and they are converted into representations set M, N by the encoder. Then subtract the distance of the negative representation pairs from the distance of the positive sample pair to get S EDCP . For example, for a representation m l from M with label l, find the closest representation m l ′ in M with different label l ′ and the closest representation n l , n ′ l from N . Then use (5) to calculate the L2 distance of m l and other representations to get d(m l , m ′ l ),d(m l , n l ) and d(m l , n ′ l ). Then subtract d(m l , n ′ l ) and d(m l , m ′ l ) by d(m l , n l ) to get the distance value for m l . Sum and all distance for each representation in M, N will get S EDCP . The calculation process is as follows:

S EDCP (M, N ) = 1 K [D(M, N ) + D(N, M )] (3)
D(M, N ) = i [min j d(m i,l , n j,l ) - 1 2 (min j d(m i,l , n j,l ′ ) + min j d(m i,l , m j,l ′ ))] (4) d(m, n) = ∥m -n∥ 2 (5)
Where m l,i is i-th sample of M with label l, n j,l is j-th sample of N with label l, K is the sum number of M, N . The smaller the value of S EDCP , the better the encoder is trained. So that monitoring process of pretraining can be achieved by observing the curve of S EDCP . And the encoder at the point where S EDCP reaches a minimum is likely to be the best pretrained encoder. Meanwhile, to assess if the trend of curve of S EDCP is correct, we conduct oneepoch-finetuning during pretraining. Because Crohn's disease detection is a relatively simple task that requires only a small number of epochs to complete finetuning, one-epochfinetuning can, to some extent, represent the final result of finetuning at a lower cost. Therefore, the ability of EDCP to reflect the encoder's pretraining trend can be determined by comparing the results of EDCP and one-epoch-finetuning.

IV. EXPERIMENT

To explore whether contrastive self-supervised learning can achieve better performance than traditional supervised learning on Crohn's disease detection, we first performed a baseline experiment to obtain results for comparison. Then, for each of the three contrastive self-supervised learning methods, perform pretraining and finetuning. The finetuning and baseline settings are the same. EDCP and one-epoch-finetuning are used during pretraining. All of our experiments were conducted on a 16GB Tesla T4.

A. Dataset.

For baseline and finetuning experiments, 3484 labeled images of CrohnIPI [START_REF] Vallée | CrohnIPI: An endoscopic image database for the evaluation of automatic Crohn's disease lesions recognition algorithms[END_REF] are used, including 1360 pathological images and 2124 non-pathological images. For pretraining, 35053 unlabeled images were obtained from two WCE videos, different from the labeled images. It is worth noting that the majority of the datasets used in the WCE image detection task research are private datasets or unspecified subsets of public databases. However, CrohnIPI is a publicly available and welllabeled dataset that can be widely used as a benchmark for Crohn's disease detection.

B. Baseline & Finetuning

Image Augmentations. The input image will be resized to 256x256 for training after a random horizontal flip, vertical flip, and rotation. During testing, the images are resized to 256x256.

Optimization. Following the settings in CrohnIPI [START_REF] Vallée | CrohnIPI: An endoscopic image database for the evaluation of automatic Crohn's disease lesions recognition algorithms[END_REF], ResNet is used as the encoder, followed by a fully connected layer as the classification head. For back-propagation, we use the cross-entropy loss. Adam optimizer is used to optimize the cross-entropy loss. For the entire network, we use a learning rate of 0.0003 and train for 300 epochs. We use a batch size of 16 and 5-fold cross-validation. For data splitting, there is 20% labeled data for testing and the rest for training and validation. The ratio of the images used for training and validation is 1:4. We use validation set to validate the model after each training epoch and select the model with the highest validation accuracy for test.

C. Pretraining

For fair comparison, we use the same encoder, number of epochs and batch size when pretraining the three contrastive learning methods, which are resnet18 (without the final classification layer), 300, and 96, respectively Image Augmentation. The data augmentation of pretraining comes after the augmentation in SimCLR [START_REF] Chen | A simple framework for contrastive learning of visual representations[END_REF]. First, a random patch of the image is chosen and resized to 256x256 with a random horizontal flip, then color distortion is applied with a random sequence of brightness, contrast, saturation, hue, and optional grayscale conversion adjustments. Gaussian blur and solarization are applied to the patch as a final step.

MoCo v2 on CrohnIPI. For optimization, MoCo v2 uses SGD as optimizer. The momentum of SGD is 0.9, and the weight decay of SGD is 0.0001. At 120 and 160 epochs, the learning rate was multiplied by 0.1. The learning rate is set to 0.03. A learning rate schedule based on cosine is used. The memory bank size is changed to 65472 instead of 65536 to accommodate the batch size of 96. Because the size of memory bank must be an integer multiple of the batch size BYOL on CrohnIPI. BYOL uses LARS Optimizer [START_REF] You | Scaling sgd batch size to 32k for imagenet training[END_REF] with a cosine decay learning rate schedule and 10 warmup epochs. The baseline learning rate was set to 0.2 and was linearly scaled [START_REF] Goyal | Accurate, large minibatch sgd: Training imagenet in 1 hour[END_REF] with batch size (LearningRate = 0.2 × BatchSize/256). BYOL uses a global weight decay parameter of 1.5 • 10 -6 while omitting the biases and batch normalization parameters from both LARS adaptation and weight decay. The target network's exponential moving average parameter τ starts at 0.996 and is raised to one Barlow Twins on CrohnIPI. Barlow Twins follow the optimization procedure outlined in BYOL [START_REF] Grill | Bootstrap your own latent-a new approach to self-supervised learning[END_REF] which use LARS optimizer for optimization. They utilize learning rates of 0.0048 for the biases and batch normalization parameters and 0.2 for the weights. They divide the learning rate by 256 and then multiply the result by the batch size. After a 10epoch warm-up period, they employ a cosine decay schedule to reduce the learning rate by a factor of 1000 [START_REF] Loshchilov | Sgdr: Stochastic gradient descent with warm restarts[END_REF]. The tradeoff parameter λ is set to 5 • 10 -3 . A weightdecay parameter of 1.5•10 6 is used. The biases and batch normalization parameters are excluded from LARS adaptation and weight decay.

D. Evaluation During Contrastive Pretraining

From the labeled dataset, we extracted two subsets. Each subset has 100 total images, of which 50 are pathological and 50 are non-pathological. We feed the image into the encoder every 20 epochs of pretraining to generate the corresponding representation. The S EDCP is calculated using [START_REF] Hajo-Maghsoudi | Segmentation of crohn, lymphangiectasia, xanthoma, lymphoid hyperplasia and stenosis diseases in wce[END_REF]. At the same time, we perform one-epoch-finetuning every 20 epochs and get value of accuracy. Theoretically, accuracy rises as S EDCP increases and vice versa. By comparing the trend of S EDCP and the trend of accuracy of one-epoch-finetuning, it is possible to demonstrate whether the EDCP method can be used to monitor the process of contrastive self-supervised learning pretraining.

V. EXPERIMENT RESULTS

A. Baseline

We have tried use Resnet18, Resnet34 and Resnet50 and find that the network with Resnet18 as encoder get best performance as shown in Table II. And the curve of validation and training accuracy shown in Fig. 3 demonstrates that 50 epochs are sufficient for the network to converge. Since detecting Crohn's disease is not a difficult task and the labeled dataset is small, the model can converge and achieve good accuracy without training for a long time.

B. Contrastive Self-Supervised Training

We utilize ResNet-34 as the encoder and pretrain MoCo v2. BYOL and Barlow Twins for 300 epochs with a batch size of 96. After pretraining, we finetuning with encoder initialized by pretrained parameters for 50 epochs. The results of finetuning are shown in Table III.

It can be seen from Table III that BYOL gets best performance in three self-supervised methods. It is intriguingly to find that MoCo v2 get relative lower performance. The reason may be due to the large number of positive samples considered as negative samples in binary classification tasks. This issue will be further investigated in VI-A.

According to studies on contrastive self-supervised learning, the batch size, encoder size, and number of pretraining epochs could all have a significant impact on performance. According to the result of baseline experiment, smaller encoder could have better performance. Therefore, based on BYOL, we compare the performance with different encoder, epochs and batch size. The results are shown in Table IV which also includes the baseline results for comparison.

From Table IV, about the size of encoder we can see that the performance is in line with baseline, that the smaller encoder performs better. Unlike in the contrastive self-supervised learning research on ImageNet, the larger encoder gets better performance does not work here. Furthermore, we discover that more epochs in BYOL pretraining can improve encoder training while larger batch size does not. When the ResNet-18 is pretrained after 600 epochs with 256 batch size, its performance outperforms the baseline which is inspiring. This result prove that contrastive self-supervised learning can improve the performance of Crohn's disease detection. However, there is still a lot of room for improvement. Since our primary goal is to investigate whether contrastive self-supervised learning is effective for Crohn's disease detection and which kind of contrastive method is more effective, we haven't done much tuning of the network structure and training parameters. We believe that many operations, such as training for longer epochs, using smaller encoders and simpler MLP layers, and using more specifically aimed data augmentation for WCE images could improve performance.

C. Evaluation During Contrastive Pretraining

When the encoder is better pretrained, the accuracy of one-epoch-finetuning should increase, while the representation IV. As a result, the curve demonstrates that s EDCP can effectively reflect the training degree during the pretraining process and can be used as an early-stopping and encoder selection tool in pretraining.

VI. DISSCUSSION

A. Noisy Negative Samples of MoCo v2

According to study in MoCo [START_REF] He | Momentum contrast for unsupervised visual representation learning[END_REF], increasing the number of features taken from the memory bank improves performance, similar to increasing the batch size in SimCLR. However, in our experiment, MoCo v2 does not perform well when there are a large number of negative sample pairs, according to the results shown in V-B. We believe that it is due to the noisy sample pairs, in which many positive samples are misidentified as negative samples. Because MoCo is an instance-instance contrastive self-supervised learning method. During pretraining, all other samples are negative samples for a sample. This strategy, however, is harmful for binary classification tasks. Because there will be a large number of samples that are considered negative during pretraining but positive during finetuning. This causes the encoder to undergo conflicting training and it becomes more severe as the number of negative samples increases when pretraining. There is no such issue in BYOL or Barlow Twins. Because BYOL does not use negative samples, and the negative samples used by the Barlow Twins are based on filters. The results of Table V show that reducing the number of negative samples improves the performance of MoCo v2 on Crohn's detection significantly. As a result, the sample-based contrastive self-supervised method may not be appropriate for binary classification or tasks on small dataset. Furthermore, the noise cancellation for samples merits further investigation.

B. Self-Supervised Learning on Small Dataset

Research on self-supervised methods is generally conducted on large benchmarks, such as ImageNet, because this provides a more objective evaluation of models and methods. Nevertheless, in the field of medical images, many datasets are small and the data diversity is limited due to the focus on specific disease or body part, such CrohnIPI. As a result, some methods and techniques that work on ImageNet may not work on small datasets. For instance, while a larger encoder and batch size are typically effective in improving performance on ImageNet, they are ineffective and even worsen performance in our experiments. Network and training strategy design for small datasets is lacking. These methods are critical in the practical application of deep learning methods, such as classification and segmentation of various medical image tasks. We hope that this study inspires more research into self-supervised learning on small datasets and we will also continue to investigate the application of self-supervision on small data sets.

VII. CONCLUSION

To alleviate the lack of sufficient labeled data for Crohn's disease detection, we investigated the use of contrastive selfsupervised learning methods combined with unlabeled data, which improved the accuracy to 94%. We also propose an evaluation during contrastive pretraining (EDCP) method for monitoring pretraining and model selection for self-supervised pretraining. Finally, we observed some intriguing issues with using contrastive self-supervised learning for small dataset tasks in our experiments that merit further investigation.

Fig. 2 .

 2 Fig. 2. The architecture of MoCo v2, BYOL and Barlow Twins. The black part represents the structure shared by all three methods, the red part represents MoCo v2, and the blue part represents BYOL. θ are the trained weights, ξ are exponential moving average of θ and sg denotes stop-gradient. At the end of training, everything except f θ is discarded, and y θ is used as the image representation. The embeddings used for calculating loss by three methods are shown in the lower left corner.

  training. BYOL specifically set k the current training step and K the maximum number of training steps to, τ ≜ 1 -(1 -τ base ) • (cos(πk/K) + 1)/2.

Fig. 3 .

 3 Fig. 3. Training, validation and test accuracy of baseline (with ResNet-18 encoder) to the number of training epochs in one of 5-fold cross-validation.

Fig. 4 .

 4 Fig. 4. The curve of one-epoch-finetuning accuracy and distance S of ECDP during pretraining. S is S EDCP in (3).

TABLE I TOP

 I -1 AND TOP-5 ACCURACIES (IN %) UNDER LINEAR EVALUATION ON IMAGENET. ALL MODELS USE RESNET-50 AS ENCODER. THE BATCH SIZE IS FOR PRETRAINING. THESE RESULTS ARE COME FROM CORRESPONDING PAPERS.

	Method	Top-1 Top-5 Batch Size
	Supervised	76.5	-	-
	MoCo	60.6	-	256
	SimCLR	69.3	89.0	4096
	MoCo v2	71.1	-	256
	SimSiam	71.3	-	256
	Barlow Twins 73.2	91.0	4096
	BYOL	74.3	91.6	4096

TABLE II TEST

 II ACCURACY OF BASELINE WITH DIFFERENT ENCODERS.

TABLE III FINETUNING

 III ACCURACY FOR MOCO V2 BYOL AND BARLOW TWINS. THE ENCODER IS RESNET35.

	Methods	Accuracy (%)
	MoCo v2	90.6
	BYOL	93.2
	Barlow Twins 91.5

TABLE IV OPTIMIZATION

 IV OF EPOCHS, BATCHS SIZE AND SIZE OF ENCODER OF PRETRAINING. WHERE ACCURACY EXCEEDS THE BASELINE, IT IS HIGHLIGHTED IN RED. BYOL IS USED AS PRETRAINING METHOD.

	encoder	number of epoch	batch size accuracy(%)
	Baseline ResNet-18 -	-	93.8
	Baseline ResNet-34 -	-	93.5
	ResNet-34	300	96	93.2
	ResNet-18	300	96	93.6
	ResNet-18	600	256	94.0
	ResNet-18	300	256	93.6