R T Cerbus

H Chraibi

M Tondusson

S Petit

D Soto

R Devillard

J P Delville

H Kellay

Experimental and numerical study of laser-induced secondary jetting

The generation of liquid jets and drops using tightly focused femtosecond laser pulses near a liquid-air interface is a convenient contactless solution for printing functional materials as well as bio-materials. Jets and drops emerge following the nucleation of a cavitation bubble in the liquid bulk by a laser-induced plasma. During the initial expansion of the bubble, a thin and fast jet is produced at the liquid surface. Moments later a second thick and slow jet emanates from the surface when the bubble has nearly deflated. Despite potential applications, little is known about the mechanism behind this complex phenomenology. Here experiments and simulations are used to investigate this two-jet process. Counter-intuitively, the second jet is not the result of bubble expansion, as with the first jet, but originates from the secondary flows induced by the bubble dynamics. Our study links the second jet properties to the control parameters of the problem and establishes a phase diagram for its emergence.

Introduction

Printing functional materials for organic electronics applications or biological inks in tissue engineering is an intricate challenge as the printing techniques must often be adapted to the material properties. These properties can be complex, with fluids ranging from Newtonian to viscoplastic and viscoelastic. Many printing techniques rely on physical contact between the ink and a blade, a nozzle, or a pattern [START_REF] Yan | A review of 3d printing technology for medical applications[END_REF][START_REF] Zhou | A review of 3d printing technologies for soft polymer materials[END_REF][START_REF] Glasser | Tuning the rheology of conducting polymer inks for various deposition processes[END_REF][START_REF] Deblais | Taming contact line instability for pattern formation[END_REF]. Contact between the fluid and the solid surfaces of the printing device can cause clogging or degradation of the material through substantial shear or extension. These modifications can be desirable for certain printing techniques. Shear thinning is useful for inkjet printing and the presence of a yield stress is useful for blade coating. Nevertheless, the high sensitivity of printing processes to both fluid and mechanical properties narrows the utility of each method. A robust and flexible technique which avoids such drawbacks is Laser Induced Forward Transfer (LIFT) [START_REF] Bohandy | Metal deposition from a supported metal film using an excimer laser[END_REF]. A submerged bubble is nucleated near a liquidgas interface by a laser-induced plasma, whose subsequent dynamics produces liquid jets accomplishing material transportation and deposition. As it is contactless and nozzle free, LIFT can be adapted to different materials from Newtonian or complex liquids to biological materials for printing, dispensing controlled amounts of fluid, or producing jets with very high velocities intended for hypodermic injections [START_REF] Desrus | Effective parameters for film-free femtosecond laser assisted bioprinting[END_REF][START_REF] Petit | Femtosecond versus picosecond laser pulses for film-free laser bioprinting[END_REF][START_REF] Brown | Impulsively actuated jets from thin liquid films for high-resolution printing applications[END_REF][START_REF] Yan | Effects of fluid properties and laser fluence on jet formation during laser direct writing of glycerol solution[END_REF][START_REF] Zhang | Time-resolved imaging study of jetting dynamics during laser printing of viscoelastic alginate solutions[END_REF]Jalaal et al. 2019b;[START_REF] Turkoz | Comparison of jets from newtonian and non-newtonian fluids induced by blister-actuated laser-induced forward transfer (BA-LIFT)[END_REF][START_REF] Turkoz | Impulsively induced jets from viscoelastic films for high-resolution printing[END_REF][START_REF] Mézel | Bioprinting by laser-induced forward transfer for tissue engineering applications: jet formation modeling[END_REF][START_REF] Ali | Controlling laser-induced jet formation for bioprinting mesenchymal stem cells with high viability and high resolution[END_REF][START_REF] Gruene | Dispensing pico to nanolitre of a natural hydrogel by laser-assisted bioprinting[END_REF][START_REF] Tagawa | Highly focused supersonic microjets[END_REF]. This technique is used in two configurations. The first configuration requires the presence of a sacrificial film, such as a metallic film, which is heated (and damaged) by the laser light to produce an expanding bubble and eventually liquid jets [START_REF] Brown | Impulsively actuated jets from thin liquid films for high-resolution printing applications[END_REF][START_REF] Yan | Effects of fluid properties and laser fluence on jet formation during laser direct writing of glycerol solution[END_REF][START_REF] Zhang | Time-resolved imaging study of jetting dynamics during laser printing of viscoelastic alginate solutions[END_REF][START_REF] Turkoz | Comparison of jets from newtonian and non-newtonian fluids induced by blister-actuated laser-induced forward transfer (BA-LIFT)[END_REF][START_REF] Turkoz | Impulsively induced jets from viscoelastic films for high-resolution printing[END_REF][START_REF] Ali | Controlling laser-induced jet formation for bioprinting mesenchymal stem cells with high viability and high resolution[END_REF][START_REF] Gruene | Dispensing pico to nanolitre of a natural hydrogel by laser-assisted bioprinting[END_REF]). In the second configuration, investigated here, the laser is focused directly in the bulk of a fluid layer just beneath the liquid-air interface [START_REF] Desrus | Effective parameters for film-free femtosecond laser assisted bioprinting[END_REF][START_REF] Petit | Femtosecond versus picosecond laser pulses for film-free laser bioprinting[END_REF]Patrascioiu et al. 2014b;Jalaal et al. 2019b). This configuration is attractive for applications including bio-printing as it minimizes the presence of impurities engendered by the eventual degradation of a sacrificial film.

An obstacle of both LIFT configurations is the complex phenomenology of the jet formation, with most reports showing that this occurs in two stages [START_REF] Petit | Femtosecond versus picosecond laser pulses for film-free laser bioprinting[END_REF]Patrascioiu et al. 2014b;Jalaal et al. 2019b). The first stage produces very thin (∼ 10 µm) and fast (∼ 10 m/s) jets. In the second stage, a relatively thick (∼ 100 µm) and slow (∼ 1 m/s) jet is produced. The first jet can be utilised for high resolution printing if the comparatively thicker second jet is avoided. However, the first jet is too thin to be suitable for printing functional materials such as suspensions of large particles or bio-materials such as cells, which the second jet is potentially capable of. If properly understood, this complexity can be exploited to provide a versatile printing method.

The fluid mechanics of the interaction between an oscillating bubble and a free surface have been studied for some time [START_REF] Lauterborn | Physics of bubble oscillations[END_REF][START_REF] Pearson | Bubble interactions near a free surface[END_REF], with military applications of underwater explosions providing an early motivation to understand this phenomenona [START_REF] Davies | The vertical motion of a spherical bubble and the pressure surrounding it[END_REF]. Observations and analysis of principally the first jet in laboratory-scale experiments have steadily provided new insights into the bubble dynamics and first jet formation, aided by improved technology and numerical simulations [START_REF] Chahine | Interaction Between an Oscillating Bubble and a Free Surface[END_REF][START_REF] Chahine | Oscillation and collapse of a cavitation bubble in the vicinity of a two-liquid interface[END_REF][START_REF] Blake | Cavitation bubbles near boundaries[END_REF][START_REF] Robinson | Interaction of cavitation bubbles with a free surface[END_REF]Patrascioiu et al. 2014b). There appears to be broad consensus that the first jet is engendered by the impulse of the initial expansion of the bubble after initiation (Patrascioiu et al. 2014a). In particular, there is good overall agreement between experiments and theoretical calculations which employ the Kelvin impulse [START_REF] Blake | Cavitation bubbles near boundaries[END_REF]. And more recent analysis using a boundary integral formulation can quantitatively reproduce the detailed features of the first jet and even its counter-jet [START_REF] Pearson | Bubble interactions near a free surface[END_REF]). The dynamics of and mechanism behind the second jet formation, on the other hand, is comparatively less studied and understood.

Although the second jet itself has not been as extensively studied, there are numerous studies that, while focusing on the first jet, also investigated the formation of a crown that can follow the emergence of the first jet [START_REF] Li | Bubble interactions and bursting behaviors near a free surface[END_REF][START_REF] Liu | Study on the interactions between two identical oscillation bubbles and a free surface in a tank[END_REF][START_REF] Koukouvinis | Simulation of bubble expansion and collapse in the vicinity of a free surface[END_REF][START_REF] Kiyama | Effects of a water hammer and cavitation on jet formation in a test tube[END_REF]. The formation of the crown and the formation of the second jet appear to be intricately linked. Although some have argued that compressibility effects such as large pressure waves are responsible for the crown [START_REF] Kiyama | Effects of a water hammer and cavitation on jet formation in a test tube[END_REF], explicitly incompressible simulations [START_REF] Liu | Study on the interactions between two identical oscillation bubbles and a free surface in a tank[END_REF]) that reproduce the crown demonstrate that compressibility is not essential to the phenomenon. Moreover the effects of the shock produced by the bubble nucleation last for only a few microseconds after bubble nucleation (Jalaal et al. 2019a), well before the formation of the second jet, and a recent parametric study of the crown formation concluded that the shock wave impinging on the surface is not responsible for the crown [START_REF] Saade | Crown formation from a cavitating bubble close to a free surface[END_REF].

Here, by carrying out a comprehensive experimental and numerical study, we clarify the mechanism behind the formation of the second jet and elucidate both its properties and the conditions for its emergence. We demonstrate that the mechanism behind the second jet formation is unique and unlike the mechanism behind the first jet formation.

In particular, the origin of the second jet is the secondary flow generated by the bubble dynamics. The paper is organized as follows. In §2, we provide information regarding the experimental setup and the numerical methods. In §3, we present our initial experimental and numerical results, showing the broad agreement between these two approaches. In §4 we elucidate the mechanism behind the formation of the second jet and use this to formulate a scaling argument to understand the dependence of the second jet velocity on parameters such as initial bubble depth and pressure. Finally, in §5 we summarize our findings.

Experimental and numerical methods

Our experimental setup consists of an amplified Titane Sapphire Femtosecond Laser emitting at 800 nm with a pulse duration of 65 fs, pulse energy of 3 mJ, and a repetition rate of 1kHz. A Pockels cell, synchronised with the laser, selects a smaller repetition rate going from a fraction of a Hz to several Hz. The laser beam, of typical radius 2mm, is then focused with a long working distance microscope objective (50x N.A. 0.5 (Olympus SLMPLN series)) in a volume of liquid (ultrapure water from a Millipore/Milli Q system) contained in a rectangular fused quartz cell (Hellma 101.150φs) with a square transparent base of 1 cm 2 . The pulse energy can be varied using a half-wave plate combined with a polarizing beamsplitter from 2.5 to 30 µJ. The objective was mounted on a stage for a precise positioning of the focal point with respect to the liquid-air interface. The Laser beam waist was estimated to be roughly 1 micron. The cell walls were rendered hydrophobic using OTS (octadecyltrisiloxane) in Toluene solution to minimize the effect of the meniscus on the visualization. A high speed camera (Phantom V 640) equipped with a magnifying lens (Navitar) was then used to visualize the dynamics of both bubble and jet formation. Typical frame rates were 110000 frames/s with an exposure time of 1 µs. The spatial resolution was 3.5 µm/pix. The acquisition of the images from the camera was triggered using a train of pulses delivered by an Agilent generator, itself synchronized to a recurring starting pulse powered by the Pockels cell driver.

For the numerical simulations, we use the open-source package OpenFOAM [START_REF] Jasak | Openfoam: A c++ library for complex physics simulations[END_REF]), a finite-volume solver for partial differential equations such as the Navier-Stokes equations. For the time integration we use an Euler scheme with the time step dynamically set to ensure a maximum Courant number less than 0.25. We use compressibleInterFOAM, a solver for two compressible, non-isothermal immiscible fluids, which uses a volume of fluid (VoF) method to solve for the volume fraction of the liquid phase γ with the transport equation [START_REF] Miller | A pressurebased, compressible, two-phase flow finite volume method for underwater explosions[END_REF][START_REF] Koch | Numerical modeling of laser generated cavitation bubbles with the finite volume and volume of fluid method, using openfoam[END_REF][START_REF] Zeng | Splitting and jetting of cavitation bubbles in thin gaps[END_REF])

∂γ ∂t + ∇ • (γu) + ∇ • (γ(1 -γ)U r = γ(1 -γ) 1 ρ g Dρ g Dp g - 1 ρ l Dρ l Dp l Dp Dt + γ∇ • u, (2.1)
where u is the velocity field, t is time, p is the pressure, ρ is the density, and the subscripts g and l represent the gas and liquid phases respectively. The final term on the left hand side is an artificial term that counteracts the numerical diffusion of the interface keeping it sharp [START_REF] Koch | Numerical modeling of laser generated cavitation bubbles with the finite volume and volume of fluid method, using openfoam[END_REF][START_REF] Rusche | Computational fluid dynamics of dispersed two-phase flows at high phase fractions[END_REF]. The compressibilities Dρ/Dp are determined by the equation of state for the gas and liquid phases, which for simplicity we choose to be the ideal gas and fluid laws, neglecting higher-order effects of temperature and pressure [START_REF] Miller | A pressurebased, compressible, two-phase flow finite volume method for underwater explosions[END_REF][START_REF] Koch | Numerical modeling of laser generated cavitation bubbles with the finite volume and volume of fluid method, using openfoam[END_REF]). Numerical simulations using compressibleInterFOAM have recently been performed to study cavitation and jetting in thin gaps [START_REF] Zeng | Splitting and jetting of cavitation bubbles in thin gaps[END_REF].

Heat and mass transfer are neglected, and the governing equations for the motion of the fluid are then the continuity equation

∂ρ ∂t + ∇ • (ρu) = 0, (2.2)
and the compressible Navier-Stokes equations

∂ρu ∂t + ∇ • (ρuu) = -∇p + ∇ • τ + F , (2.3)
where τ is the viscous stress tensor

τ = µ[∇u + ∇u T -2 3 (∇ • u)I],
with µ the dynamic viscosity and I the identity tensor. F is the force due to surface tension which is modelled using a continuous-surface-force method which preserves the sharpness of the interface in order to accurately estimate the curvature [START_REF] Koch | Numerical modeling of laser generated cavitation bubbles with the finite volume and volume of fluid method, using openfoam[END_REF][START_REF] Brackbill | A continuum method for modeling surface tension[END_REF].

Eqs. 2.1-2.3 are solved together in a rectilinear domain. The extent of the threedimensional (3D) simulation domain was 400 µm in both horizontal directions (x, y) and ranged from 800 µm to 1200 µm in the vertical direction (z), with the liquid-air interface (z = 0) fixed at 300 µm above the domain bottom. We used a uniform resolution of dx = 2 µm. The boundary conditions are zero-gradient at all domain boundaries. More information on the sensitivity to domain size and resolution can be found in App. A.

Simulating the exact experimental situation, including the laser-fluid interaction, is complex. To simplify our approach, we assume that this interaction simply nucleates a small bubble of initial high pressure P beneath the surface at z = -H (x = y = 0). We use a spherical bubble with initial radius R 0 = 10 µm, which remains roughly spherical during the initial dynamics. This is significantly different from experiments where the bubbles observed experimentally are elongated in the direction of the beam (the focal area is a vertically elongated ellipsoid due to the necessarily weak numerical aperture of the long working distance objective). As we show, despite these simplifications, our numerical simulations capture the essential features of the experimental observations.

Results

In experiments we vary both the focal point depth H below the interface as well as the pulse energy E. The typical phenomenology of the jet formation following bubble nucleation for a fixed E and H = 144 µm is shown in Fig. 1. After nucleation, the bubble rapidly expands, reaches a maximum size, then contracts and recoils away from the interface. Notwithstanding the presence of the interface and substantial asymmetry, the bubble dynamics is in fair agreement with the predicted behavior of the Rayleigh-Plesset equation [START_REF] Vogel | Energy balance of optical breakdown in water at nanosecond to femtosecond time scales[END_REF][START_REF] Lauterborn | Physics of bubble oscillations[END_REF]) (see App. B). The initial bubble expansion correlates with the formation of the first rapid jet. As this thin jet rises, the bubble deflates and recoils into the bulk fluid. Moments later (t ∼ 30 µs) a second bulge emerges from the interface and the slower, thicker second jet emerges as in previous observations (Patrascioiu et al. 2014b). During the emergence of this second jet the bubble has deflated completely as it recoils away from the interface. While the first jet disintegrates into a train of rapidly rising drops (of size ∼ 20 µm) the second jet becomes unstable at the tip and engenders drops of much larger size (D ∼ 100 µm).

The first jet, now well understood, is produced by the impulse of the initial bubble expansion. Previous work has suggested that the mechanism for the second jet is the same (Patrascioiu et al. 2014b), with the second jet corresponding to the second bubble expansion. To investigate this in more detail, we have carried out a numerical study which provides information not available in experiments and reveals the detailed velocity and pressure fields responsible for the second jet formation. As with experiments, we systematically vary H and P , which replaces E, to uncover their significance for the jet's behavior.

To estimate appropriate values of P to correspond with experiments is not a simple task as only a fraction of the pulse energy is transformed into bubble energy [START_REF] Vogel | Energy balance of optical breakdown in water at nanosecond to femtosecond time scales[END_REF]. Following [START_REF] Fabbro | Physical study of laser-produced plasma in confined geometry[END_REF] and Jalaal et al. (2019b), a characteristic pressure due to the laser pulse can be estimated using

P c = (α(I -I th )) 1/2 (ρc) 1/2 , (3.1)
where I is the laser intensity, I th is the plasma threshold intensity, ρ = 1000 kg/m 3 is the density of water, and c = 1500 m/s is the speed of sound in water, and α ( 0.01) is the small fraction of the laser energy that finally contributes to the bubble energy [START_REF] Vogel | Energy balance of optical breakdown in water at nanosecond to femtosecond time scales[END_REF]. With an estimated laser spot radius of ∼ 1 µm, an energy of E ∼ 1 µJ, and a plasma threshold of I th 6 × 10 16 W/m 2 (Linz et al. 2016), we estimate a characteristic pressure of P c ∼ 3 × 10 6 bars, which although large is of a similar order of magnitude to estimates in similar experiments [START_REF] Schaffer | Dynamics of femtosecond laser-induced breakdown in water from femtoseconds to microseconds[END_REF]. We confirm this estimate by comparing the oscillations of a bubble nucleated far from the interface with the predictions of the Rayleigh-Plesset equation (see App. B). This comparison suggests a similar order of magnitude for the pressure for an initial bubble radius R 0 ∼ 1 µm, a scale too small to resolve in our finite spatial resolution simulations. An important parameter, as we will see below, is the bubble energy ∼ P R 3 0 . Keeping this constant we thus increase the initial radius to R 0 = 10 µm and explore initial pressures 10 2 P 10 3 bars. Fig. 1 shows a sequence of images of the bubble and jetting dynamics from simulations that are remarkably similar to experiments. A first and second jet, droplet formation, and the bubble oscillations are all observed, as well as a counter-jet not easily observable in our experiments (Patrascioiu et al. 2014b).

We further compare the experimental and numerical results by studying the dynamics of jet formation for different values of H, E (for experiments), and P (for simulations). In this work we focus on H R 0 to avoid bubble bursting [START_REF] Sanjay | Bursting bubble in a viscoplastic medium[END_REF]. In Fig. 2 we plot the heights h of the first and second jets as a function of time, giving a quantitative picture of the phenomenology just described. At early times h corresponds to the jet tip, while at later times it corresponds to the furthest point of the released droplet, if any (see Fig. 1). For experiments, in Fig. 2a we systematically vary H for fixed E = 18.3 µJ, while in Fig. 2b we systematically vary E for relatively fixed H 144 µm. Within our precision the velocity of the first jet U 1 is hardly affected by varying H or E. The velocity U 2 of the second jet, on the other hand, is strongly affected by both H and E. The second jet first decelerates before reaching a steady velocity, U 2 , until it becomes unstable to a Rayleigh-Plateau-like instability engendering droplets with velocities close to that of the rising jet. The open circles in Figs. 2a,2b indicate the position of drop formation. For fixed E = 18.3 µJ (Fig. 2a) U 2 is a decreasing function of H until a threshold value of H = H c 300 µm, below which the jet retracts and no droplets are shed. Likewise for a fixed value of H 144 µm (Fig. 2b U 2 shows a systematic dependence on E with a threshold below which no jets can be produced [START_REF] Vogel | Energy balance of optical breakdown in water at nanosecond to femtosecond time scales[END_REF]. The diameter D of the The liquid (water) phase is dark and the gas (air) phase is light. Until time t 14 µs, when the bubble size reaches its first minimum, the two simulations (a) and (b) are the same with dynamics as in Fig. 1 for t 25µs. Then for (b), the bubble is artificially removed by setting the air phase below the interface to water and the simulation allowed to evolve. In this way the second expansion of the bubble is removed. A comparison of the second jet with (a) and without (b) the second expansion shows that the second jet is much the same in its absence.

shed droplets, however, depends only weakly on H and E (see SFig. 2). The sensitivity of U 2 to H and E suggests that it can be tuned to affect how droplets impact a surface for printing [START_REF] Zhang | Study of impingement types and printing quality during laser printing of viscoelastic alginate solutions[END_REF]. In Fig. 2c we show the jet time series for different values of H for fixed P = 400 bar, while in Fig. 2d we show the same for different values of P with fixed H = 40 µm. The time series from the simulations are both qualitatively and quantitatively similar to the experiments. The second jet emergence time (∼ 20 µs), the droplet detachment time (∼ 200 µs), and the first jet velocity (∼ 30 m/s), and second jet velocity (∼ 3 m/s) are all quantitatively similar for simulations and experiments. Likewise, the dependence of second jet velocity U 2 and drop diameter D on H and P mimic the experimental behavior with D being less sensitive than U 2 (see App. C). A notable difference between experiments and simulations is the significant dependence of U 1 on H and P in simulations (see Figs. 2c,2d), although our experiments may simply have insufficient temporal resolution to see a similar dependence.

Analysis and discussion

While there exists detailed analysis and theory for the initial jet formation due to the expanding bubble [START_REF] Blake | Cavitation bubbles near boundaries[END_REF][START_REF] Robinson | Interaction of cavitation bubbles with a free surface[END_REF]), there appears to be little understanding of the second, thicker jet. Previous work has argued that the mechanism for the second jet differs little from that of the first jet (Patrascioiu et al. 2014b): the secondary bubble expansion pushes the fluid away from the free surface into the air. However, in both experiments and simulations, the second jet emerges when the bubble is imploding and recoiling away from the interface, before it has begun its second expansion (as in Fig. 1 for t 25µs and Fig. 5a). If the second jet were the product of the second bubble expansion, then one would expect a delay even longer than the several µs between the first bubble expansion and first jet (see Fig. 1c) since the bubble is even further from the surface when the second jet emerges. The absence of any delay points to the failure of the argument that the second jet arises from the second expansion.

A qualitative test of the hypothesis of the importance of the secondary expansion for the second jet can be made by artificially removing the bubble when it reaches its minimum size in a simulation (see Fig. 3). When the bubble reaches its minimum size at t 14 µs, after its implosion, we set the phase in the submerged bubble to water, leaving the velocity, pressure and temperature fields as they are. In this way the second expansion is effectively, albeit artificially, removed. A comparison between a simulation with (Fig. 3a) and without (Fig. 3b) the second expansion shows only minute differences. The second jet still emerges and engenders a large droplet. If the second expansion is not responsible for the second jet, its source must be sought elsewhere.

Here the additional information provided by the simulations becomes essential. In Fig. 4 we show the velocity and pressure fields of a representative simulation where P = 200 bar and H = 40 µm. These fields are consistent with the explanation that the first jet is created by the initial bubble expansion pushing the fluid upward, the primary flow. The pressure gradient is directed radially away from the bubble, and the velocity is likewise directed radially outward. During the initial formation of the second jet, however, the velocity and pressure fields are significantly different. The pressure is much larger below the surface than at the surface, and the largest gradient occurs when the bubble is at its minimum size (t = 14 µs). The collapse of the bubble, which becomes toroidal, generates a rapidly retracting counter-jet and a substantial inflow into the region between the bubble and the surface (t = 13 µs). This creates the high-pressure region below the surface as well as a saddle flow (t = 14 µs). The lateral flow into the region between the bubble and the surface, which we term the secondary flow and which is established before the bubble expansion, appears to be the source of the second jet. Less than 1 µs later, the second jet has already emerged, well before the bubble re-expands. In Fig. 5a we show the time of the second jet emergence T jet vs. the time that the bubble collapses to its minimum size T min . The near perfect correlation between the two times for all pressures, energies, and depths reinforces our argument that the second jet is related to the bubble contraction, and the secondary flow it engenders, and not the bubble re-expansion. Identifying the secondary flow as the mechanism behind the second jet formation, as done qualitatively in [START_REF] Chen | Exploration of water jet generated by q-switched laser induced water breakdown with different depths beneath a flat free surface[END_REF], permits a simple analysis of the second jet velocity scaling.

Mass conservation determines a relationship between the secondary inflow velocity and the second jet velocity, U 2 . Although the degree of contribution to the secondary flow from the bubble collapse and counter-jet is unclear, we attribute the large downward flow in Fig. 4a (t 13 µs) to the counter-jet, which we assume has a velocity comparable to the first jet U 1 . By considering a cylindrical volume of liquid of diameter D c and length l c , we can write 2π

(D c /2)l c U ∼ π(D c /2) 2 U 1 , which yields U ∼ U 1 D c 4l c . (4.1)
We anticipate that D c corresponds to the maximum nucleated bubble size D max while the vertical length scale l c is as yet unknown. (Additional evidence for a single horizontal length scale proportional to D max is given in Fig. 6a, where we show that the droplet diameter D ∝ D max ). In Fig. 5b we plot both U 1 and U 2 versus H for simulations at P = 400 bar. Both velocities decrease approximately linearly with H and approach zero at H ∼ 90 µm, beyond which no jetting is observed. The ratio between U 2 and U 1 is thus The first column (t = 1 µs) shows the bubble several µs prior to the emergence of the first jet. The remaining columns are before (t = 13 µs), during (t = 14 µs), and after (t = 15 µs), the emergence of the second jet, which is accompanied by a substantial secondary flow. A substantial secondary flow accompanies the emergence of the second jet, with the flow drawn in by the bubble collapse and counter-jet feeding the second jet and creating the large pressure gradient that drives it. constant and given by D max /4l c (Eq. 4.1), which after measuring D max yields l c . In Fig. 6b we show that l c D max for several values of P . The bubble dimensions thus set the relation between U 2 and U 1 .

In Figs. 7a,7b we show that the linearity of U 2 with H up to a threshold value H c , as in Fig. 5b, is generic for both experiments with fixed E and for additional simulations with fixed P . Unsurprisingly, the critical depth H c increases while the slope ∂U/∂H decreases with an increase of E or P . The similarities between these graphs suggest that the experimental and numerical results can be brought into correspondence with a suitable normalization. Since we have argued that the second jet is associated with bubble collapse, we estimate a characteristic bubble collapse velocity as U σ = σ/ρD max , where σ is the surface tension between water and air, and normalize U 2 with U σ . To account for the dependence on a critical depth H c , we plot U 2 /U σ versus (H c -H)/D max , which is similar to but distinct from the standoff distance 2H/D max used in previous work Plot of first U1 and second U jet velocity vs. initial depth H from simulations for three different initial pressures P = 200, 300, and 400 bar. For all pressures, both U1 and U decrease nearly linearly to zero at a critical depth H * . The ratio of the slopes of these two lines is Dmax/4lc, as argued in the main text. Inset: Dmax/lc vs. P . The ratio is close to unity for all P , suggesting that there is only a single dominant length scale. [START_REF] Pearson | Bubble interactions near a free surface[END_REF]) by virtue of the inclusion of H c necessary for collapse. As Fig. 7c shows, the data collapse satisfactorily. Although the bubbles in the simulations are roughly spherical, the bubbles in experiments are strongly elliptical, so to account for this asymmetry we used the maximum bubble horizontal width D h max and vertical length D v max as the relevant length scales. In the simulations these two length scales are nearly the same. If different, as in experiments, we expect the vertical length scale D v max to affect the critical depth H c , while we expect the horizontal length D h max to control the bubble collapse speed. Thus D v max is used to normalize (H c -H) and D h max is used to normalize the velocity. With these choices the experiments and simulations collapse, but separately. To bring them into accord with each other, we introduce a parameter β 0.48 which is used only for the simulations to artificially modify the length scales D v max and D h max . Since the value of β is close to the ratio of the D h max /D v max for experiments (see App. B), this suggests that we have fully captured the behavior of both simulations and experiments with the normalization in Fig. 7c, excepting a dependence on the bubble asymmetry at its maximum size. We add further evidence to the correspondence between experiments and simulations with a combined phase diagram in Fig. 7d indicating the parameter ranges for jetting and no-jetting. For the simulations we systematically varied H and P to determine when jetting was observed, yielding a jetting to no-jetting boundary empirically found to be H/R 0 ∼ (P R 0 /σ) 1/2 . The scaling of the boundary can be understood by recasting the dependence as σH 2 c ∼ P R 3 0 , a relationship born out in the inset of Fig. 7d. This simply states that for jetting to occur, the bubble energy must overcome a surface energy [START_REF] Pearson | Bubble interactions near a free surface[END_REF][START_REF] Chen | Exploration of water jet generated by q-switched laser induced water breakdown with different depths beneath a flat free surface[END_REF]. We note that the length scale of the surface energy H c is itself simply related to the nucleated bubble dimensions at maximum expansion (see Fig. 10d). The surface in question is thus that which will protrude from the initially flat surface due to a bubble of size D max ∝ H c , with a curvature also set by these length scales. To include the experimental data in the phase diagram requires knowledge of P and R 0 , which we are not able to measure directly. Using a line calibrated to the simulation data in the inset of Fig. 7d, we determine the value of P R 3 0 corresponding to the value of σH 2 c , which we can determine experimentally. Using P estimated with Eq. 3.1, the value of R 0 turns out to be similar to the beam waist, ∼ 1 µm, as found in [START_REF] Schaffer | Dynamics of femtosecond laser-induced breakdown in water from femtoseconds to microseconds[END_REF]. The experimental data can then be brought into correspondence with the numerical data in a satisfactory manner as shown in Fig. 7d. Taken together, Figs. 7c and 7d provide a comprehensive understanding of the jet formation using LIFT as well as the correspondence between laser pulse energy and the engendered pressure giving rise to the cavitation bubble.

Conclusion

LIFT near a liquid-air interface results in a complex interaction between the dynamics of a cavitation bubble created by the laser-induced plasma and the shape of the interface. The fast expansion and subsequent implosion of this bubble near the interface generate an initial rapid jet as well as strong secondary flows which give rise to a second more prominent jet. The properties of this jet can be tuned by varying either the focus position of the laser beam with respect to the surface or the incident energy of the laser pulse. Our fully three-dimensional numerical simulations qualitatively and quantitatively reproduce the experimental observations. An important result of our combined numerical and experimental study is a phase diagram of nucleated bubble distance from the interface versus pressure which allows to delimit the jetting and no-jetting regimes. We also explicitly link properties of the jets such as their velocity to these two control parameters. Altogether, our study provides a more complete understanding of the secondary jetting phenomenon and thus how they might be controlled for use in different printing processes where the aim is to deposit materials of large dimensions such as cells or colloids. where P and R 0 are the initial pressure and radius of the bubble at rest, κ = 4/3 is the isentropic expansion factor of water, P ∞ = 1 bar is the pressure at infinity, µ = 8.9×10 -4 kg/m•s is the dynamic viscosity of water, and σ = 0.072 N/m is the surface tension between water and air. This equation is integrated forward in time using a fourth order Runge-Kutta method. Because the experimental bubble is axisymmetric, we plot a diameter that accommodates for the cigar shape of the bubble so that we can better compare with the Rayleigh-Plesset theory for symmetric spherical bubbles:

D bubble = (D v D 2 h ) 1/3
, where D v is the vertical length and D h is the shorter radial width. We found empirically that at their maximum values D h /D v 0.41, which is close to the value of β = 0.48 used in Fig. 7c. We estimated initial bubble diameters of R 0 1µm in §3 (see Fig. 7d). Assuming that this R 0 corresponds to the smaller horizontal width of the bubble, and using the same ratio of vertical to horizontal length scales found in experiments, the effective initial bubble size used in the numerical solution of the Rayleigh-Plesset equation is then R * 0 = (R 2 0 (D v R 0 /D h )) 1/3 1.3µm. We find that an estimate of P = 0.7 Mbar gives excellent agreement both in terms of spatial and temporal magnitude with the experimental observations, and is close to the estimated value of P ∼ 3 Mbar found using Eq. 3.1.

Figure 1 .

 1 Figure 1. (a)-(b) Sequence of side-view images illustrating the experimental bubble and jet dynamics, the nucleation depth H, the jet height h and the droplet diameter D. The laser pulse energy is E = 18.3 µJ and H = 144 µm. The first thin jet appears concomitantly with the bubble expansion while the second jet emerges ∼ 30 µs later when the bubble has deflated and recoiled away from the interface. (c)-(e) Numerical analog of the experimental images. Vertical slices (y -z plane) of the 3D numerical simulations of the bubble and jet dynamics. Here H = 80 µm and the initial bubble pressure is P = 600 bars. The second jet is beginning to emerge ∼ 30 µs after initiation. The simulations reproduce the main features of the experiments and show additional details such as the counter-jet (t = 17 µs) not easily observed in our experiments.

Figure 2 .

 2 Figure 2. (a) Experimental jet height h versus time t for E = 18.3 µJ and varying H. For large H no drop formation occurs at the tip of the second jet which simply retracts. (b) Experimental h versus time t for different energies E and H 120 µm. (c) h versus time from numerical simulations for different H and fixed initial pressure P = 400 bar. The first jet is indicated by open symbols. (d ) Same as (c) for different initial pressures P and fixed H = 40 µm. The open circles indicate the time at which drop detachment occurs. The data after this time indicate the position of the drop versus time. The drops travels at basically the same velocity as the jet in this regime of roughly constant velocity. The open squares indicate the retraction time.

Figure 3 .

 3 Figure 3. Vertical slices of the simulations at different times for P = 200 bars and H = 40 µm.The liquid (water) phase is dark and the gas (air) phase is light. Until time t 14 µs, when the bubble size reaches its first minimum, the two simulations (a) and (b) are the same with dynamics as in Fig.1for t 25µs. Then for (b), the bubble is artificially removed by setting the air phase below the interface to water and the simulation allowed to evolve. In this way the second expansion of the bubble is removed. A comparison of the second jet with (a) and without (b) the second expansion shows that the second jet is much the same in its absence.

Figure 4 .

 4 Figure 4. Vertical slices of the simulated flow fields at t = 1, 13, 14, and 15 µs for P = 200 bars and H = 40 µm, focusing on the region near the interface. (a) Plots of the liquid (dark) and air (light) phases with velocity vectors super-imposed. The velocity vector lengths at t = 1 µs correspond to the distance a fluid particle will travel in 1 µs, while the velocity vector lengths for the remaining times correspond to the distance a fluid particle will travel in 2 µs. (b) Color plots of the pressure field p. The phase boundaries are indicated by thick black lines. (c) Color plots of the horizontal velocity uy. (d ) Color plots of the vertical velocity uz.The first column (t = 1 µs) shows the bubble several µs prior to the emergence of the first jet. The remaining columns are before (t = 13 µs), during (t = 14 µs), and after (t = 15 µs), the emergence of the second jet, which is accompanied by a substantial secondary flow. A substantial secondary flow accompanies the emergence of the second jet, with the flow drawn in by the bubble collapse and counter-jet feeding the second jet and creating the large pressure gradient that drives it.

Figure 5 .

 5 Figure 5. (a) Plot of the second jet emergence time Tjet vs. the time when the submerged bubble reaches its minimum size Tmin. The uncertainties on the experimental and numerical results correspond to the time between recorded times, 9 µs and 1 µs respectively. (b) Plot of the first U1 and second U2 jet velocity vs. H for fixed P = 400 bar from simulations. Both vary approximately linearly near a threshold value of Hc 90 µm.

Figure 6 .

 6 Figure 6. (a) Plot of the droplet size D vs. horizontal maximum bubble diameter D h max from simulations. Combining the results from several depths indicates that D ∝ D h max . (b)Plot of first U1 and second U jet velocity vs. initial depth H from simulations for three different initial pressures P = 200, 300, and 400 bar. For all pressures, both U1 and U decrease nearly linearly to zero at a critical depth H * . The ratio of the slopes of these two lines is Dmax/4lc, as argued in the main text. Inset: Dmax/lc vs. P . The ratio is close to unity for all P , suggesting that there is only a single dominant length scale.

Figure 7 .

 7 Figure 7. The second jet velocity U2 versus initial depth H for (a) experiments with several fixed E and (b) simulations with several fixed P . All curves reveal the same linearity of U2 with H and a threshold Hc. (c) Second jet velocity U2 from experiments and simulations normalized by Uσ = (σ/ρβD h max ) 1/2 versus a normalized distance from the interface (Hc -H)/βD v max . Here β = 0.48 (close to the experimental ratio D h max /D v max 0.41) brings the experiments into accord with the simulations, including initially simulations where the initial bubble shape was elliptic (see App. D). (d ) Phase diagram for jetting using the two normalized control parameters H/R0 and P R0/σ. The interface between jetting and no-jetting scales as H/R0 ∝ (P R0/σ) 1/2 . Inset: Plot of σH 2 c versus P R 3 0 with the same symbols as the main figure. A linear calibration fit to the simulations determines P R 3 0 for experiments, with P from Eq. 3.1 then yielding R0.

Figure 8 .

 8 Figure 8. (a) Snapshots of simulations (vertical slices) at t = 25 µs after initiation for different domain widths. (b) Time series of first jet (open symbols) and second jet (closed symbols) positions for these same domain widths as well as two different resolutions dx. For domain sizes larger than 300 µm and resolutions smaller than dx = 4 µm, the results do not differ significantly.

Figure 10 .Figure 11 .

 1011 Figure 10. (a) Plot of second jet droplet diameter D vs. initial depth H for experiments at fixed energy E = 18.3µJ. (b) Plot of second jet droplet diameter D vs. energy E for fixed initial depth H = 125 µm. Unlike the second jet velocity, the droplet diameter D is relatively insensitive to variations in either H or E. (c) Plot of horizontal maximum bubble diameter D h max vs. initial pressure P for simulations at different depths and the numerical solution of the Rayleigh-Plesset equation. Although the results from simulations are below the prediction of the Rayleigh-Plesset equation, which unlike the simulations assumes the fluid surrounding the bubble extends to infinity in all directions, D h max ∝ P 0.38 for both. (d ) The empirically-determined critical depths Hc vs. empirically-determined vertical D v max and horizontal D h max maximum submerged bubble diameters. While both depend on E or P , Hc and Dmax are strongly correlated. While for simulations the D h max D v max , the asymmetry of the initial bubble in experiments results in D v max being nearly twice as large as D h max .

† Email address for correspondence: hamid.kellay@u-bordeaux.fr

Acknowledgements. We thank X. Aguerre, P. Lastanet, J. Flecher, D. Attoumani, for help with preliminary experiments and numerical simulations during their internships.

Funding. H.K. acknowledges support from Institut Universitaire de France, and R.T.C. gratefully acknowledges the support of a Marie Sk lodowska-Curie Action Individual Fellowship (MSCAIF).

Declaration of interest. The authors report no conflict of interest.

Appendix A. Simulation sensitivity to domain size and resolution

For the simulations we confirmed insensitivity to both lateral domain size and grid resolution by performing simulations at different lateral domain sizes of 300 µm, 400 µm, 600 µm and 800 µm, and at uniform resolutions of dx = 4 µm, dx = 2 µm, and dx = 1.33 µm. We found no significant change for a lateral domain size larger than 400 µm or a resolution smaller than dx = 2 µm. In Fig. 8a we show snapshots of simulations at t = 25 µs after initiation. As the jet time series in Fig. 8b confirms, lateral domains widths of 400 µm and a resolution of dx = 2 µm are sufficient to achieve insensitivity to these parameter choices. We thus adopted these values for the simulations.

Appendix B. Rayleigh-Plesset numerical solution

An additional estimate of the initial experimental bubble pressure is obtained by comparing the oscillations of the bubble diameter with the predictions of the Rayleigh-Plesset equation (see Fig. 9) [START_REF] Lauterborn | Physics of bubble oscillations[END_REF]. In this case the bubble was initiated far from the interface to avoid its influence on the dynamics. We solve the Rayleigh-Plesset equation for the radius R including the effects of viscosity and surface tension

Appendix C. Dependence of droplet and bubble diameter

Here we present the droplet and bubble diameter dependence on parameters such as the initial depth H, the initial energy E, or pressure P . The main significance of this presentation is that the droplet diameter D is only weakly dependent on these control parameters, and that the maximum bubble diameter D max is the dominant length scale. In Fig. 10a we show the dependence of the experimental droplet diameter D on the depth H, which follows a weak power law, D ∝ H 0.225 . In Fig. 10b we show the dependence of D on the initial energy E, which is also a weak power law, D ∝ E 0.321 . We are not aware of any theory that accounts for these power laws. The dependence of the maximum bubble diameter in simulations, D max on pressure P is shown in Fig. 10c. This also follows a weak power-law, D max ∝ P 0.383 that is consistent with prediction from the Rayleigh-Plesset equation (see App. B). Finally, Fig. 10d shows the dependence of the critical depth H c on the maximum bubble diameter D max for both simulations and experiments. The dependence is essentially linear, and is in keeping with our finding that D max is the dominant length scale and thus can be used to collapse the plots of velocity versus depth (see Fig. 7c). While in simulations the horizontal and vertical maximum bubble diameters Here the laser pulse was focused with a 50X magnification microscope objective. Dynamics of a bubble, the black region, nucleated at a large depth (H = ∞) far from any interface. The time between images is 9 µs. The bubble shape is axisymmetric but not spherical, a result of the non-spherical focused laser spot. (b) Plot of immersed bubble diameter D bubble vs. time t from experiments. The laser energy used to initiate this bubble is E = 18.3µJ. We also plot the numerical solution when R0 = 10µm (R * 0 13.4µm), which fits with the experiments when P = 800 bar. This result is consistent with P R 3 0 = const., as argued for Fig. 7d.

are essentially the same, they differ significantly in experiments. That the experiments and simulations should belong to different curves is not surprising, given the differences between them such as initial bubble size and asymmetry. The significance of Fig. 10d is that D max sets the critical depth H c , just as it also sets the droplet diameter (see Fig. 6a).

Appendix D. Dependence on initial bubble shape

In nearly all of the simulations presented the initial bubble was spherical with radius R 0 = 10 µm. In this appendix we present the results of simulations with the same initial volume, and thus the same initial energy P R 3 0 , but with an initially elliptical shape. The ratio of the vertical to horizontal length was 2. Keeping the initial bubble pressure fixed at P = 200 bar, we varied the position of the center of the bubble H. Fig. 11a reveals jet time series of the same character as Fig. 2c, with U 1 and U 2 steadily decreasing with increasing H as shown in Fig. 11b. Likewise the plot of U 1 and U 2 vs. H in Fig. 11b is similar to Figs. 5b,7b, with an approximately linear dependence of the asymptotic velocities on H near a critical H c , although the critical H c for the first and second jet is slightly different. The normalized velocities are included in Fig. 7c with β 0.48 as with the other simulations. The simulations collapse on themselves regardless of the initial bubble shape. Consistent with this, the maximum vertical bubble diameter D v max differs little from the horizontal diameter D h max , despite the different initial shape.


Romy Sauvayre
email: romy.sauvayre@uca.fr

What types of errors are hiding in Google Scholar data? A case study

Keywords: Reference accuracy, database reliability, false positive, academic publication, research evaluation, scientometrics, citation analysis

HAL is

Introduction

Google Scholar (GS) has become a major bibliographic and citation database. Soon after its creation in 2004, GS received major criticism (see Orduna-Malea et al. [START_REF] Orduna-Malea | Google Scholar as a source for scholarly evaluation: a bibliographic review of database errors[END_REF] review), but subsequently, further studies described it more positively [START_REF] Martín-Martín | Can we use Google Scholar to identify highly-cited documents?[END_REF][START_REF] Falagas | Comparison of PubMed, Scopus, Web of Science, and Google Scholar: strengths and weaknesses[END_REF]. Indeed, the literature acknowledges the free access offered by GS [START_REF] Falagas | Comparison of PubMed, Scopus, Web of Science, and Google Scholar: strengths and weaknesses[END_REF][START_REF] Haddaway | The Role of Google Scholar in Evidence Reviews and Its Applicability to Grey Literature Searching[END_REF][START_REF] Bornmann | Convergent validity of bibliometric Google Scholar data in the field of chemistry-Citation counts for papers that were accepted by Angewandte Chemie International Edition or rejected but published elsewhere, using Google Scholar[END_REF] and the quality of its coverage [START_REF] González-Teruel | Effectiveness of OpenAIRE, BASE, Recolecta, and Google Scholar at finding spanish articles in repositories[END_REF][START_REF] Orduna-Malea | Methods for estimating the size of Google Scholar[END_REF][START_REF] Mingers | Counting the citations: a comparison of Web of Science and Google Scholar in the field of business and management[END_REF][START_REF] Harzing | Google Scholar, Scopus and the Web of Science: a longitudinal and cross-disciplinary comparison[END_REF][START_REF] Moed | A new methodology for comparing Google Scholar and Scopus[END_REF][START_REF] Gehanno | Is the coverage of google scholar enough to be used alone for systematic reviews[END_REF][START_REF] Harzing | A preliminary test of Google Scholar as a source for citation data: a longitudinal study of Nobel prize winners[END_REF]. The coverage of GS is considered better than that of both Web of Science (WoS) [START_REF] Harzing | A preliminary test of Google Scholar as a source for citation data: a longitudinal study of Nobel prize winners[END_REF][START_REF] Mingers | The drivers of citations in management science journals[END_REF][START_REF] Franceschet | A comparison of bibliometric indicators for computer science scholars and journals on Web of Science and Google Scholar[END_REF][START_REF] Harzing | A longitudinal study of Google Scholar coverage between 2012 and 2013[END_REF] and Scopus [START_REF] Harzing | Google Scholar, Scopus and the Web of Science: a longitudinal and cross-disciplinary comparison[END_REF][START_REF] Moed | A new methodology for comparing Google Scholar and Scopus[END_REF], which are GS's fee-based competitors. This is particularly true regarding its coverage of social sciences and humanities research [START_REF] Moed | A new methodology for comparing Google Scholar and Scopus[END_REF][START_REF] Prins | Using Google Scholar in research evaluation of humanities and social science programs: a comparison with Web of Science data[END_REF][START_REF] Kousha | Assessing the citation impact of books: the role of Google Books, Google Scholar, and Scopus[END_REF], conference proceedings [START_REF] Moed | A new methodology for comparing Google Scholar and Scopus[END_REF][START_REF] Franceschet | A comparison of bibliometric indicators for computer science scholars and journals on Web of Science and Google Scholar[END_REF] and books [START_REF] Kousha | Assessing the citation impact of books: the role of Google Books, Google Scholar, and Scopus[END_REF]. The GS database has been substantially qualitatively [START_REF] Winter | The expansion of Google Scholar versus Web of Science: a longitudinal study[END_REF] and quantitatively [START_REF] Moed | A new methodology for comparing Google Scholar and Scopus[END_REF][START_REF] Wildgaard | A comparison of 17 author-level bibliometric indicators for researchers in Astronomy, Environmental Science, Philosophy and Public Health in Web of Science and Google Scholar[END_REF] improved in all scientific areas such that, according to De Winter et al. [START_REF] Winter | The expansion of Google Scholar versus Web of Science: a longitudinal study[END_REF], it could supplant WoS.

• development of appropriate methods to limit the negative effects of GS errors on the results produced.

The present case study aimed to examine the GS data extracted by PoP, provide a full count of the errors contained in these collected data, and present an epistemological reflection. By doing so, this study offers detailed categorizations of GS data that have not been provided by previous studies. The purpose is especially to address the following questions:

1) What types of GS errors could affect the data and results of researchers' studies? 2) What methodological problems may result from these errors? 3) How reliable can the citations of GS be without data cleaning?

Methods

Context

This GS study is part of broader research that aims to explore the diffusion process of a neuroimaging work that sought to alert the scientific community to the issue of false positives. Two references were examined: the first reference is a poster presented at the 15th Annual Meeting for the Organization for Human Brain Mapping (OHBM) [START_REF] Bennett | Neural correlates of interspecies perspective taking in the post-mortem Atlantic salmon: an argument for multiple comparisons correction[END_REF] and the second is an article published in the short-lived Journal of Serendipitous and Unexpected Results (JSUR) [START_REF] Bennett | Neural correlates of interspecies perspective taking in the post-mortem Atlantic salmon: an argument for proper multiple comparisons correction[END_REF]. The question was which researchers contributed to this diffusion or, in other words, who cited the OHBM poster or the JSUR article. The collection of those citation data then became necessary. Nevertheless, some full texts of the citing documents collected by GS did not cite either the OHBM poster or the JSUR article. From this, the present case study was born: the reliability of the GS data needed to be quantified to identify the limitations of the results produced with GS data before using these data in the diffusion study. This categorization of errors using these two references enables one to identify how GS works with literature not referenced by journal editors' websites. Google Scholar uses "automated software, known as 'parsers', to identify bibliographic data" [START_REF]Inclusion Guidelines for Webmasters[END_REF] of documents available on the Internet. Then, the parser software "typically" collects the same data from full documents without metadata as the two used in the present case study.

Data collection

To examine the reliability and accuracy of GS, the citations of both the OHBM poster and the JSUR article were analyzed. Note that GS was the only citation database available to collect the citation data for these two works because neither WoS nor Scopus have ever indexed them. PoP version 5 was used to extract references that cited the poster. According to Harzing [START_REF] Harzing | Accuracy: PoP vs GS for journals[END_REF], this software provides a perfect collection of GS data: "Publish or Perish is as accurate or as inaccurate as Google Scholar itself." In addition, PoP is a common tool in scientometric studies using GS data [START_REF] Franceschet | A comparison of bibliometric indicators for computer science scholars and journals on Web of Science and Google Scholar[END_REF][START_REF] De Groote | Coverage of Google Scholar, Scopus, and Web of Science: a case study of the h-index in nursing[END_REF][START_REF] Wildgaard | A comparison of 17 author-level bibliometric indicators for researchers in Astronomy, Environmental Science, Philosophy and Public Health in Web of Science and Google Scholar[END_REF][START_REF] Thelwall | ResearchGate versus Google Scholar: which finds more early citations?[END_REF].

The citation data were then collected from GS via PoP: the first author's name ("Bennett, Craig M") was entered without quotation marks, and the first part of the OHBM poster and JSUR article title ("Neural correlates of interspecies perspective taking in the post-mortem Atlantic Salmon") were entered with quotation marks into the "All of the words" software query box. As PoP's manual explains, the "All of the words" query "matches the search terms anywhere in the searched documents (author, title, source, abstract, references etc.)" [45], as GS does. Thus, this query was used to reproduce the same request with PoP and GS. This title is so specific that only two results appeared:

• 127 references cited the JSUR article [START_REF] Bennett | Neural correlates of interspecies perspective taking in the post-mortem Atlantic salmon: an argument for proper multiple comparisons correction[END_REF], and • 154 references cited the OHBM poster [START_REF] Bennett | Neural correlates of interspecies perspective taking in the post-mortem Atlantic salmon: an argument for multiple comparisons correction[END_REF]. In contrast, the reference that appeared in PoP and on GS was a paper supposedly published in a supplement of the famous NeuroImage journal and indexed by ScienceDirect. In reality, NeuroImage did not publish a journal article written by Bennett and colleagues in 2009 about the neuroimaging work. Bennett's NeuroImage paper does not exist. What this supplemental issue of NeuroImage does contain is the program of the OHBM conference. Therefore, when the citing documents cite the "NeuroImage ghost paper", they actually cite the OHBM poster. Note that the JSUR article title is almost identical to the OHBM poster title-only the term "proper" in the second part of the title differs. The advantage of this strong similarity is the ability to evaluate the capacity of GS to manage citations of similar references. A total of 281 references were extracted via PoP on October 6, 2017. Two CSV files were obtained (Multimedia Appendix 1)-one for each neuroimaging reference. In the present study, several columns that contained the following information were examined: authors, title of the citing document, publication year, publication or source, publisher, and web address of the citing document ("Article URL" as provided by GS). Each column was manually verified, and inaccuracies were counted and categorized in the following six steps:

1) The full text of accessible citing documents was downloaded and recorded;

2) The reference list of each citing document was consulted to verify and record the presence of the neuroimaging reference (OHBM poster, JSUR article, or both); 3) The document type was determined and recorded by reading it and searching for additional information on its source; 4) For each citing document, an accurate reference was elaborated for use as a standard and to determine whether GS data contain errors. An inductive and descriptive methodological approach was used to list and identify all the error types that occurred in the GS data. The reference accuracy literature served as a guide to avoid omitting the important errors in this field. A typology was elaborated and presented in the results section: 1) Data collection error (duplicates, reprints, translations, missing URLs, and inaccessible documents);

2) Academic publication collection error (retrieval of types of documents other than journal articles, books, book chapters and conference proceedings); 3) Citation error (false positive or citation counted by GS when the document does not cite the reference counted); 4) Author error (missing authors, added authors, missing part of the author's name, initials errors); 5) Title error (incorrect or incomplete title, spelling or typographical errors) ; 6) Publication year error (erroneous or missing date of publication); 7) Publication of source errors (journal name error identified in the "publication" column of GS); 8) Publisher error (book editor name error identified in the "publisher" column of GS). 5) The GS errors found in each extracted column were listed; and 6) The identified errors were aggregated by reference. This collection, verification, and aggregation process required approximately 170 hours of work.

Results

Number of errors

A total of 755 errors were detected in 281 references retrieved from GS, for an average of 2.7 errors (range: 0-7) (Multimedia Appendix 2). Furthermore, 279 of 281 references (99.3%) contained at least one error.

Typology of GS errors

After a manual examination of the references extracted from GS, eight types of errors were identified (Figure 2) and are thoroughly described above: 1) Data collection error; 2) Academic publication collection error; 3) Citation error (false positive); 4) Author error; 5) Title error; 6) Publication year error; 7) Publication error; 8) Publisher error.

Data collection errors

Data collection errors include duplicates, reprints, translations, missing URLs, and inaccessible documents (Multimedia Appendix 3). This type of error is identified in 33 of 281 references (11.7%), and among these errors, duplicates are detected in 16 of 281 references (5.7%). In addition, URL analysis indicates that none of GS data in any of the PoP extractions contain duplicate URLs. However, because 18 URLs were missing, a manual search for these references was then conducted to obtain and verify them. Among these missing-URL references, only 2 of the 18 citing documents were inaccessible, and 9 references were duplicates, translations, or reprints.

Academic publication collection error

Some scientometric studies have used document type as a variable. Consequently, some researchers have focused exclusively on journal articles [START_REF] Falagas | Comparison of PubMed, Scopus, Web of Science, and Google Scholar: strengths and weaknesses[END_REF][START_REF] González-Teruel | Effectiveness of OpenAIRE, BASE, Recolecta, and Google Scholar at finding spanish articles in repositories[END_REF][START_REF] De Groote | Coverage of Google Scholar, Scopus, and Web of Science: a case study of the h-index in nursing[END_REF][START_REF] Granter | Calciphylaxis and the Persistence of Medical Misinformation in the Era of Google[END_REF][START_REF] Cole | Scientific output and recognition: a study in the operation of the reward system in science[END_REF][START_REF] Price Dj De | Networks of scientific papers: the pattern of bibliographic references indicates the nature of the scientific research front[END_REF], whereas others have presented their collected citations per document type, including journal articles, books, book chapters and conference proceedings [START_REF] Martín-Martín | A two-sided academic landscape: snapshot of highly-cited documents in Google Scholar (1950-2013)[END_REF]. Furthermore, "grey literature" [START_REF] Haddaway | The Role of Google Scholar in Evidence Reviews and Its Applicability to Grey Literature Searching[END_REF], such as theses and research reports [START_REF] Winter | The expansion of Google Scholar versus Web of Science: a longitudinal study[END_REF], can also be included.

Considering the diversity of this research method, it will be interesting to further explore the document types that GS is likely to retrieve and count. GS describes itself as a database that "provides a simple way to broadly search for scholarly literature" [START_REF] About | Google Scholar[END_REF]. But what does "scholarly literature" mean for GS? The definition provided by GS-and used in the document inclusion processencompasses "journal papers, conference papers, technical reports, or their drafts, dissertations, pre-prints, post-prints, or abstracts" [START_REF]Inclusion Guidelines for Webmasters[END_REF]. On another webpage, GS mentions that users "can search across many disciplines and sources: articles, theses, books, abstracts and court opinions" [START_REF] About | Google Scholar[END_REF]. By contrast, GS excludes "news or magazine articles, book reviews, and editorials" [START_REF]Inclusion Guidelines for Webmasters[END_REF] because they are "not appropriate" [START_REF]Inclusion Guidelines for Webmasters[END_REF]. Nevertheless, there is no statement about the rejection of these undesirable documents from the GS index.

In the present study, the document type of each reference collected from GS was examined to determine whether the document in question were "academic publications". In this way, a document was considered an "academic publication" if and only if it was 1) an article that was published in a journal with an International Standard Serial Number (ISSN) or 2) a book, a book chapter or conference proceedings published with an International Standard Book Number (ISBN). All other document types (thesis, magazine, communication poster, bibliography, course, report, unpublished document)-so-called "nonacademic documents"-were classified as GS collection errors. Note that, according to this definition, a doctoral thesis is an academic work but not an academic publication. As Multimedia Appendix 5 shows, GS retrieved 203 of 281 (72.5%) academic publications, but included 77 nonacademic documents in the corpus. The error rate reached 27.5% according to the definition given in the literature, whereas the GS definition led to a lower error rate (6.8%). In addition, because GS data are asymmetrically distributed, a nonparametric (Mann-Whitney) test was conducted with SPSS 25 and revealed that the nonacademic documents tend to contain more errors than the academic publications (U=5117.0, P<.001) (Multimedia Appendix 4).

Citation error (false positive)

The reference list of each citing document was examined to determine which of the two references (OHBM poster or JSUR article) had been cited. A total of 271 full documents were available and were read. This assessment reveals that 81 of the documents (29.9%) do not cite the reference retrieved from GS. In other words, 29.9% of the citations counted by GS are false positives. In 8 of the 271 cases (3.0%), neither of the two references are found. In 12 of the 271 cases (4.4%), the JSUR article reference is found instead of the OHBM poster one, that is, in the extraction of citations attributed by GS to the OHBM poster. Conversely, in 61 of the 271 cases (22.5%), the OHBM poster reference is found instead of the JSUR article one. Additionally, these citation errors (false positives) affect eight times more OHBM poster references than JSUR article references (odds ratio = 7.77; 4.4 < CI < 13.71). Note that the OHBM poster reference is misreferenced in the citing documents more often than the JSUR article.

Author error

As Multimedia Appendix 6 shows, 53 of 273 references (19.4%) contain at least one author error. For example, initials are removed or added. Authors are missing in 41 of 273 references (15.0%). Surprisingly, they are replaced by a journal name or by the title of either their own book or their own book chapter. Finally, 104 authors are missing, while 20 authors are improperly added. In sum, 124 authors of 565 (22.0%) are inaccurate.

Title error

A thorough examination of the "title" column extracted from GS shows that 57 of 274 references (20.8%) contain at least one error (Multimedia Appendix 7). The incompleteness of the title is the most common error identified. As a result of this error, some incomplete titles are similar to other publication titles. Furthermore, several errors are more questionable, such as replacement of a book title with a chapter title from this aforementioned book or with the title of a different chapter from another book by an author who contributed a chapter to this book. Other questionable title errors are the assemblage of two-chapter parts published in the same book and the replacement of the publication title by its editor's name or the domain name of the website that hosts it. Surprisingly, irrelevant parts are added to the publication title, such as an ISBN number, the price of the book, the name of the book collection, and an excerpt from the front page of a thesis ("a dissertation submitted for the degree of Doctor of Philosophy"). Lastly, the reference titles also contain typographical or spelling errors.

Publication year error

Publication year errors are detected in 31 of 274 references (11.3%). In most cases, the years are missing-they are replaced by "zero" in 22 references. In other cases, the actual publication year of the JSUR article or the OHBM poster is increased by one year or decreased by 1, 3, 7, or 100 years (Multimedia Appendix 8).

Publication of source errors

The "publication" or "source" column retrieved from GS via PoP shows inconsistencies that depend on the document type of references (Multimedia Appendix 9). Indeed, it contains journal names, books, edited book titles, conference proceedings titles, magazine names, publisher names, domain names of websites that host the citing documents, irrelevant parts of references, and even an author's address. Furthermore, a large number of missing values (i.e., "not provided" in Multimedia Appendix 9) are found in these publication data, affecting one in three references (32.0%). These missing publication data are observed most often for theses (bachelor's, master's and doctoral) and book references.

In total, 133 of 280 references (47.5%) contain errors (Table 1). These errors are mostly identified in conference proceedings, edited books, and journal articles. In a The usable publication content for studies using academic publications is denoted by a "+". These errors were not easy to categorize because of the nonacademic documents. For instance, when the document type is a blog post or an unpublished draft, a journal name is not expected in the "publication" column and thus is counted as an inaccuracy. Nevertheless, this type of document had already been counted as a data collection error. Therefore, each document type was specifically analyzed to avoid falsely increasing the error count. However, the categorization was easier for other references, such as when the journal editor name was provided instead of the journal name. In addition, an examination of spelling and typographical errors, including capitalization errors, was conducted.

These source inconsistencies mainly occur in journal names as typographical errors, particularly capitalization errors (Multimedia Appendix 10). The second most frequent error is title and journal name incompleteness. Journal names are then heavily truncated, as shown in the following examples: "Journal of …" instead of "Journal of Advertising Research" and "Rev …" instead of "Revista de neurologia". The same type of inaccuracy was identified in the edited book titles: "… Routledge Handbook of …" instead of "The Routledge Handbook of Neuroethics" and "… Imaging of the …" instead of "Imaging of the Pelvis, Musculoskeletal System, and Special Applications to CAD". Furthermore, as several journal names begin with "Journal of" and several edited books begin with "Routledge Handbook of", the incompleteness of the GS data may cause difficulties.

Publisher error

The "publisher" column retrieved from GS provides a variety of content (Multimedia Appendix 11): editor name (including journal editor), journal name, domain name of the website that hosts the citing document (e.g., 42 of the domain names are "books.google.com"), digital library (i.e., JSTOR), and missing values. The "publisher" column contains the highest error rate found in the GS data, which is 244 of 281 references (86.8%) (Table 2). Indeed, the 248 inaccuracies detected in this column constitute a third (32.9%) of the total errors identified. Journal editors and domain names are frequently inaccurate. The utility of this publisher data is then limited to studies using academic publication data. Only the editor names of books, book chapters, and conference proceedings are usable, but they actually represent 35 of the 281 references (12.5%). Furthermore, an error rate of 37.1% is found in these usable data. For example, an editor's name is replaced by an irrelevant one: The Penguin Press by Australia Books; and Palgrave Macmillan by Springer.

Discussion

The aim of the present study was to examine the accuracy of citation data collected from GS via PoP and to provide a comprehensive description of the errors and miscounts identified. In fact, the extraction of raw data with inaccuracies from GS may generate incorrect results in several research areas, such as bibliometrics, scientometrics, and research evaluation. Despite the data cleaning performed by researchers (mainly duplicate removal), citation counts retrieved from GS are generally used without substantial caution. Furthermore, few comprehensive studies list the different types of GS errors, and no previous research seems to quantify the inherent problems of GS citations collected by PoP. The present study was therefore conducted to provide a meticulous analysis of GS data to anticipate the risk of errors that may affect data and the results of studies using them.

Ranking of GS errors

The GS errors were analyzed using 281 documents that cited a neuroimaging work performed to raise awareness of false positive results in the scientific community. The present study reveals an unprecedented error rate: 279 of 281 examined references (99.3%) contain at least one error. The academic publications are not free from errors: they account for 503 of the 755 detected errors (67.0%). However, the nonacademic documents tend to contain more errors than the academic publications (U=5117.0, P<.001).

The cumulative error rate detected in the present study-99.3% of references contain at least one error-differs from the median rate (14.6%) reported in the literature over the past 10 years. This difference may be explained by several aspects of previous research: 1) an automatic approach was generally used to clean the data, while a manual examination was conducted in the present study; 2) a varied but low number of variables were examined in these studies: a median of one type of error was examined in previous studies, while eight types of errors were examined in the present study; 3) the usual purpose of these studies was to compare the coverage of GS, WoS, and Scopus; thus, the researchers mainly verified duplicates in an aggregated corpus drawn from these three databases; and 4) these studies do not cumulate the number of errors identified per reference. These discrepancies make comparison difficult, but data provided by De Winter et al. [START_REF] Winter | The expansion of Google Scholar versus Web of Science: a longitudinal study[END_REF]-"Online Supplementary material 5 Excel File"-make it possible. Through these data, an error rate cumulated by reference was calculated to compare what is comparable. However, as these researchers used four error types, the comparison was performed for academic publication collection errors, author errors, title errors and duplicates. All other things being equal, the present study reports an error rate three times higher than that reported by De Winter et al. [START_REF] Winter | The expansion of Google Scholar versus Web of Science: a longitudinal study[END_REF] (64.8% and 20.5%, respectively). These findings suggest that citation counts and references extracted from GS are not fully reliable and may expose the researchers who use them to numerous errors. Note that the content of GS is the result of automatic indexing of websites by robots. The coverage depends on the indexed websites. Moreover, according to GS, "robots generally try to index every paper from every website they visit, including most major sources and also many lesser known ones" [START_REF]Google Scholar Search Tips[END_REF]. Thus, the reliability of GS is a type of "photography" of the reliability of authors and editors' websites. Since errors can happen, it is important to identify the possible impact of GS's lack of reliability with respect to research data.

The impact of GS errors in research data

What is the probable impact of GS errors in the citation analysis or research evaluation areas when citation counts and references are used without data cleaning? Sauvayre R., (in press) "What types of errors are hiding in Google Scholar data? A case study", Journal of Medical Internet Research.

13

Publisher error

The useful content that a researcher needs to find in the "publisher" column extracted from GS via PoP is the editor name for books, book chapters, and conference proceedings. However, this column mainly contains the domain name of the website hosting the citing document. Thus, only 7.8% of these collected data are free from errors and usable in an academic publication study. The "publication" column therefore requires meticulous examination before use. The first step is to determine the document type of each collected reference because GS still does not provide it.

Publication error

The PoP manual indicates that the "publication" column contains "journal name or similar"-"similar" is not explicitly defined-which is "not always available" and "sometimes wrong" [START_REF] Adams | Results list[END_REF]. However, the "publication" content is more disparate and incorrect than this. Indeed, it contains journal names, book titles, thesis titles, publisher names, and domain names, and one-third of this column is missing values. Only 21.4% of "publication" data are free from errors and usable in an academic publication study. In addition, GS errors can impact studies in the following ways. First, the GS error rate can negatively affect the evaluation of journal impact factors and the journal ranking. Second, missing values (32.4% of references) can alter relational database management [START_REF] Guagliardo | Correctness of SQL Queries on Databases with Nulls[END_REF]. Third, typographical errors (including capitalization errors) can lead to duplicates [START_REF] Haddaway | The Role of Google Scholar in Evidence Reviews and Its Applicability to Grey Literature Searching[END_REF]. Lastly, note that there is a risk in using this "publication" data because of the large number of errors detected in the journal names, edited book titles, and conference proceedings titles.

Citation error (false positive)

The GS citation count is distorted by documents that do not cite the reference retrieved. This point is often reported in the literature-for instance, as a "phantom citation" or "false citation" [START_REF] Franceschini | Empirical analysis and classification of database errors in Scopus and Web of Science[END_REF]-but the no reported error rate [START_REF] Winter | The expansion of Google Scholar versus Web of Science: a longitudinal study[END_REF][START_REF] Granter | Calciphylaxis and the Persistence of Medical Misinformation in the Era of Google[END_REF][START_REF] García-Pérez | Accuracy and completeness of publication and citation records in the Web of Science, PsycINFO, and Google Scholar: a case study for the computation of h indices in psychology[END_REF][START_REF] Baneyx | Publish or Perish" as citation metrics used to analyze scientific output in the humanities: international case studies in economics, geography, social sciences, philosophy, and history[END_REF][START_REF] Bar-Ilan | Citations to the "Introduction to informetrics" indexed by WOS, Scopus and Google Scholar[END_REF] is as high as the rate found in the present study (29.9%). This difference can be explained by the highly similar titles of the two references examined (OHBM poster and JSUR article). This finding also demonstrates the difficulties of addressing this type of similarity in GS data. Consequently, researchers may use data samples that contain false positive citations and then may obtain biased findings.

Academic publication collection error

GS failed to retrieve only academic publications. Indeed, 27.5% of the citing documents are nonacademic publications, including doctoral theses, magazine articles, preprints, reports, courses, bibliographies, and blog posts. This error rate confirms previous findings [START_REF] Granter | Calciphylaxis and the Persistence of Medical Misinformation in the Era of Google[END_REF]. However, if the GS definition of "scholarly literature" is applied, this error rate falls to 6.8%. This GS definition differs widely from the definition of "academic publication" used in the present study. Thus, GS seems to inaccurately report citation counts and references of academic publications, and consequently, it does not accurately reflect the dissemination of published work. Therefore, the results of many scientometric studies using GS data to examine the publication activity of scientists, particularly in research evaluation, may be questionable when these data are not verified and cleaned (document type and false positive). The citation counts and h-index scores calculated by GS are also questionable. This raises questions about the reliability of studies that compare the coverage of GS, WoS and Scopus and conclude that GS collects significantly more citations [START_REF] Harzing | A preliminary test of Google Scholar as a source for citation data: a longitudinal study of Nobel prize winners[END_REF][START_REF] Bergman | Finding citations to social work literature: the relative benefits of using Web of Science, Scopus, or Google Scholar[END_REF][START_REF] De Groote | Coverage of Google Scholar, Scopus, and Web of Science: a case study of the h-index in nursing[END_REF] than its competitors. Further research should explore the citation counts of these databases to determine how comparable they are.

Title error

The main issue with the titles retrieved from GS is incompleteness, which causes problems such as false positive matches. The similarity between the OHBM poster title and the JSUR article title demonstrates this GS difficulty. Other errorstypographical and spelling-cause problems in database management. More unwelcome is the missing title error: instead of the title, 6.2% of references contain, for example, editor names, domain names, or ISBN numbers. These missing title errors raise several problems: 1) references cannot be retrieved by a search by title, and 2) duplicates can be more frequent and more difficult to detect. duplicate, translation, and reprint references. Consequently, the duplicate search removes 7.8% of the references, whereas the irrelevant deletion of references without a URL address leads to the omission of 3.2% of the citing documents. Again, this may cause biased results.

Publication year error

Incorrect years have a lower frequency than missing years (3.3% and 8.0% of references, respectively). These missing values cause major problems in data collection. As GS limits the search results to the first 1000 citing references per query, certain researchers have collected data by publication year to obtain a larger corpus of GS citations [START_REF] Martín-Martín | A two-sided academic landscape: snapshot of highly-cited documents in Google Scholar (1950-2013)[END_REF] or to focus their analysis on a specific period of time [START_REF] Thelwall | ResearchGate versus Google Scholar: which finds more early citations?[END_REF]. Other researchers have removed references containing incorrect publication years [START_REF] Prins | Using Google Scholar in research evaluation of humanities and social science programs: a comparison with Web of Science data[END_REF]. Thus, these neglected references may lead to truncated data and biased results. Inherent to the failed indexation process of GS, this publication year error may cause sampling errors that affect the representativeness of findings.

Data verification versus biased results

The GS error rate seems to be negligible when types of errors are considered in isolation. These types of claims have been made about false positives [START_REF] Franceschini | Empirical analysis and classification of database errors in Scopus and Web of Science[END_REF], duplicates [START_REF] Moed | A new methodology for comparing Google Scholar and Scopus[END_REF], and incorrect publication years [START_REF] Martín-Martín | Can we use Google Scholar to identify highly-cited documents?[END_REF]. By contrast, with regard to GS errors, Harzing [55] argues fatalistically that "bibliometrics is an inexact science and that any data source has its own flaws". However, Hicks et al. [START_REF] Hicks | Bibliometrics: The Leiden Manifesto for research metrics[END_REF], in presenting the Leiden Manifesto, emphasize the importance of the quality of the data used in research evaluation. Conversely, when the GS error rates are observed as a whole, a worrying cumulative effect is revealed. Indeed, only 2 of 281 (0.71%) references collected from GS are free from errors. This raises a question about the reliability of the GS citation counts. In the present study, two neuroimaging works were cited 281 times according to GS. However, this citation count is incorrect. In fact, these works are cited 131 times in academic publications (i.e., excluding duplicates, reprints, translations, inaccessible documents, and false positives), which is 53.4% less than the GS claim. Thus, the full sample collected from GS (281 citations) can considerably differ from the proper sample (131 citations). There is thus a major risk of producing incorrect and biased results that do not accurately reflect the data examined. Consequently, meticulous verification and cleaning of GS data are essential before using them. Considering this, several precautions should be taken to improve the reliability of GS data:

1) Detect and remove duplicate, translation, and reprint references and subsequently merge their citation counts; 2) Consult the full-text documents of the full sample to remove false positive matches; and

3) Verify the document type of each reference to exclude nonacademic publications. Because results will be biased or wrong if these verification steps are not performed, is it possible to study a large-scale sample of GS citations (approximately several thousand)? It seems unlikely unless substantial resources are allocated for such verification. Indeed, Meho and Yang [START_REF] Meho | Impact of data sources on citation counts and rankings of LIS faculty: Web of Science versus scopus and google scholar[END_REF] were allowed 18 minutes per reference (3,000 hours of work for a 10,000 citation samples). In the present study, a work time of 32 minutes per reference was necessary to complete the verification (150 hours for 281 citations). What about automatization of the verification? Studies that cleaned large-scale data either in part or as a whole using an automatic cleaning process report a lower error rate and fewer error types than studies using a manual cleaning process. Therefore, it is reasonable to have doubts about the efficiency of this automatic cleaning. Finally, studying a small sample of GS data seems more adequate than studying a large sample in terms of obtaining reliable data and accurate findings. Nevertheless, there is a need to conduct further research to develop statistical tools for weighting the correlation calculation in a large-scale sample of GS data, which are widely used in database coverage studies. However, these tools may not correct the collection issue inherent to the GS database. Alternatively, according to the reference accuracy literature [START_REF] Harzing | Google Scholar, Scopus and the Web of Science: a longitudinal and cross-disciplinary comparison[END_REF][START_REF] Moed | A new methodology for comparing Google Scholar and Scopus[END_REF][START_REF] Harzing | A preliminary test of Google Scholar as a source for citation data: a longitudinal study of Nobel prize winners[END_REF][START_REF] Mingers | The drivers of citations in management science journals[END_REF][START_REF] Franceschet | A comparison of bibliometric indicators for computer science scholars and journals on Web of Science and Google Scholar[END_REF][START_REF] Harzing | A longitudinal study of Google Scholar coverage between 2012 and 2013[END_REF], databases such as WoS or Scopus can be used in place of GS because they are more reliable, though they have narrower coverage than GS. Indeed, WoS has an average error rate of 0.1% [START_REF] Haddaway | The Role of Google Scholar in Evidence Reviews and Its Applicability to Grey Literature Searching[END_REF][START_REF] Winter | The expansion of Google Scholar versus Web of Science: a longitudinal study[END_REF][START_REF] Kulkarni | Comparisons of Citations in Web of Science, Scopus, and Google Scholar for Articles Published in General Medical Journals[END_REF][START_REF] García-Pérez | Accuracy and completeness of publication and citation records in the Web of Science, PsycINFO, and Google Scholar: a case study for the computation of h indices in psychology[END_REF], and this rate is 1.0% for Scopus [START_REF] Moed | A new methodology for comparing Google Scholar and Scopus[END_REF][START_REF] Kulkarni | Comparisons of Citations in Web of Science, Scopus, and Google Scholar for Articles Published in General Medical Journals[END_REF]. However, since GS is a free database [START_REF] Basu | Global Distribution of Google Scholar Citations: A Size-independent Institution-based Analysis[END_REF], it may be the only possible way to conduct a study. However, knowing that all databases are likely to contain errors, verifying a sample of data is a useful precaution. To conclude, the categorization of the errors encountered in the data extracted from GS provides researchers with methodological and epistemological reflections so that they become aware, with precision, of the probable errors that they are likely to encounter and can consequently adjust their methodological choice. For example, the number of citations obtained by GS may not be completely accurate, or the names of the authors mentioned may not be completely correct. With a sample of several thousand references, these errors can have a noticeable impact on the results.

Conclusion

Almost all of the data retrieved from GS contain at least one error calling the reliability of GS data into question. Further, the reliability of studies using a large-scale sample without verification and data cleaning is also called into question. Moreover, studies using GS to evaluate research activity or compare the coverage of several databases (i.e., GS, WoS, Scopus) may be affected by substantial biases, including citation miscounts. However, researchers able to spend a considerable amount of time on the meticulous verification of their small samples can obtain various references for journal articles, books, edited book chapters, and conference proceedings from GS. This ability can be especially useful in bibliometric studies based on material published in research areas in which journal articles are less predominant than other publication types.

Limitations

Since the data used are limited and specific, the results obtained cannot be generalized. However, this case study provides a kind of "stress test" of GS to promote reflection on the limits of this free database.

Figure 1 .

 1 Figure 1. Pareto diagram: sum of errors detected (N = 755) as a function of document type. Light blue: academic publications; Gray: nonacademic documents. Dark blue: cumulative sum curve of errors detected.

Figure 2 .

 2 Figure 2. Typology of GS errors. Typology and proportion of errors identified (A) as a function of the number of valid references examined (271 < N < 281) and (B) as a function of the total number of errors detected (N = 755).

of Medical Internet Research. 10 addition

  Sauvayre R., (in press) "What types of errors are hiding in Google Scholar data? A case study", Journal , only half of the citations counted by GS are usable as academic publication material (Table1, "utility" column) because, for example, GS provides a domain name instead of the academic journal name. Among these usable data, 133 are inaccurate. Finally, in this corpus, only 60 of 280 references (21.4%) are proper usable data.

Table 1 .

 1 Accurate and inaccurate content identified in the "Publication" column retrieved from GS via PoP (n=280). a

	Type of errors, n (%)	Inaccurate publication	Accurate publication	Total references Utility
	Journal name	56 (51.9)	52 (48.1)	108 (100.0)	(+)
	Magazine name	1 (50.0)	1 (50.0)	2 (100.0)	(₋)
	Book title	13 (100.0)	0 (0.0)	13 (100.0)	(₋)
	Edited book title	21 (72.4)	8 (27.6)	29 (100.0)	(+)

Table 2 .

 2 Accurate and inaccurate content in the "Publisher" column retrieved from GS via PoP (N=281). The usable publication content for studies using academic publication data is denoted by a "+".

	Type of errors, n (%)	Inaccurate publication	Accurate publication	Total references	Utility
	Book & conference proceedings editor 13 (37.1)	22 (62.

Sauvayre R., (in press) "What types of errors are hiding in Google Scholar data? A case study", Journal of Medical Internet Research.

Author error

The citing documents examined are cowritten by 565 authors. Nevertheless, 124 authors (22.0%) are either incorrect or missing. These errors can cause problems in studies of the structure of scientific collaborative networks, which are commonly used graphs. Indeed, a fifth of the collaborative networks built may be incorrect and thus may generate imperfect relationships. First, the missing authors may truncate an important share of all the authors involved. Second, the irrelevant added authors may create a bias that a graph's algorithms can reinforce. Consequently, researchers may overestimate a relationship or ignore another determinant one.

Data collection error

Duplicates, translations, and reprints are frequent in GS data. As collected data can be biased by duplicates, their detection is the first step implemented in studies using GS data. The duplicate, translation, and reprint rates found in the present study are similar to those in previous findings [START_REF] Granter | Calciphylaxis and the Persistence of Medical Misinformation in the Era of Google[END_REF][START_REF] García-Pérez | Accuracy and completeness of publication and citation records in the Web of Science, PsycINFO, and Google Scholar: a case study for the computation of h indices in psychology[END_REF]. In addition, the URL address of citing documents is commonly used to detect duplicates and collect full text documents. Because a missing URL may cause difficulties, previous studies resolved this issue by automatically deleting a reference without a web address [START_REF] Prins | Using Google Scholar in research evaluation of humanities and social science programs: a comparison with Web of Science data[END_REF][START_REF] García-Pérez | Accuracy and completeness of publication and citation records in the Web of Science, PsycINFO, and Google Scholar: a case study for the computation of h indices in psychology[END_REF]. By contrast, in the present study, 6.4% of URL addresses are missing, but only 0.7% of them cannot be found with a manual search. Half of these found documents are usable, and half are

Conflicts of Interest

The author declares no competing interests.

Abbreviations

Multimedia Appendix 4

Results of the Mann-Whitney test on the type of reference (academic publication or nonacademic document) and the number of GS errors and data visualization with a box plot.

Multimedia Appendix 5

Document types of references collected from GS as a function of three categories (academic journal, nonacademic journal, and GS non-"scholarly literature").

Multimedia Appendix 6

Inaccurate content identified in the "Author" column retrieved from GS via PoP.

Multimedia Appendix 9

Content of the "Publication" column retrieved from GS via PoP as a function of reference document type.

Multimedia Appendix 10

Spelling and orthographical errors in the academic publications retrieved from GS via PoP: journal name, edited book title, and conference proceedings book title.

Multimedia Appendix 11

Content of the "Publisher" column retrieved from GS via PoP as a function of reference document type.