J J Van Der Werf
email: j.j.vanderwerf@utwente.nl

V Magar

J Malarkey
email: j.malarkey@bangor.ac.uk

K Guizien
email: guizien@obs-banyuls.fr

T O'donoghue
email: donoghue@eng.abdn.ac.uk

2DV MODELLING OF SEDIMENT TRANSPORT PROCESSES OVER FULL-SCALE RIPPLES IN REGULAR ASYMMETRIC OSCILLATORY FLOW

Keywords: Experimental data, mathematical models, oscillatory flow, sand ripples, sediment transport. 1

Wave-induced, steep vortex ripples are ubiquitous features in shallow coastal seas and it is therefore important to fully understand and model the sediment transport processes that occur over them. To this end, two two-dimensional vertical (2DV) models have been critically tested against detailed velocity and sediment concentration measurements above mobile ripples in regular asymmetric oscillatory flow. The two models are a k-ω turbulence-closure model and a discrete-vortex, particle-tracking (DVPT) model, while the data is obtained in the Aberdeen Oscillatory Flow Tunnel (AOFT). The models and the data demonstrate that the time-dependent velocity and suspended sediment

separation at the crest of the flow coming from the stoss-side of the ripple. The vortices are subsequently ejected into the flow at flow reversal and new vortices are generated on the opposite side of the ripple. This process is repeated each wave half cycle (see e.g. [START_REF] Earnshaw | Dynamics of ripple bed vortices[END_REF][START_REF] Marin | Eddy viscosity and Eulerian drift over rippled beds in waves[END_REF]. [START_REF] Fredsøe | Wave plus current over a ripple-covered bed[END_REF] demonstrated that when waves and currents are superimposed this pattern of behaviour persists almost unchanged because of the dominance of these periodic vortex structures near the ripples.

Associated with the generation and ejection of vortices, the suspended sediment above ripples has a well-organised temporal and spatial structure and coherent, sediment-rich, lee vortices represent the primary mechanism for entraining sediment from the bed into the water column (see e.g. [START_REF] Clubb | Experimental study of vortex ripples in full scale sinusoidal and asymmetric flows[END_REF][START_REF] Thorne | Measurements of near-bed intra-wave sand entrainment above vortex ripples[END_REF]. Above rippled beds, the phase of sediment pick-up from the bed during the wave cycle is thus linked to the phase of vortex shedding. This has potentially important consequences for the net sediment transport rate beneath asymmetrical waves, which can be 'offshore' in spite of the fact that 'onshore' near-bed velocities are stronger (see e.g [START_REF] Sato | Oscillatory boundary flow and sand movement over ripples[END_REF][START_REF] Clubb | Experimental study of vortex ripples in full scale sinusoidal and asymmetric flows[END_REF]Van der Werf et al., 2006).

Experimental results are important to understand the complex nature of ripple regime sediment transport. Numerous experimental studies on sediment transport processes over rippled beds in oscillatory flows have been carried out in the past, both in the field (e.g. [START_REF] Vincent | Field measurements of the suspended sand concentration profiles and fluxes and of the resuspension coefficient  0 over a rippled bed[END_REF][START_REF] Green | Suspended-sediment reference concentration under waves: field observations and critical analysis of two predictive models[END_REF][START_REF] Traykovski | Geometry, migration, and evolution of wave orbital ripples at LEO-15[END_REF][START_REF] Hanes | Wave-formed sand ripples at Duck, North Carolina[END_REF][START_REF] Doering | An investigation of the velocity field under regular and irregular waves over a sand beach[END_REF][START_REF] Williams | Field measurements of flow fields and sediment transport above mobile bed forms[END_REF][START_REF] Grasmeijer | Observed and predicted bed forms and their effect on suspended sand concentrations[END_REF] and in the laboratory (e.g. [START_REF] Bosman | Concentration measurements under oscillatory motion[END_REF][START_REF] Faraci | Geometry, migration and evolution of sea ripples[END_REF][START_REF] Thorne | Suspended sediment under waves measured in a large-scale flume facility[END_REF][START_REF] Marin | Eddy viscosity and Eulerian drift over rippled beds in waves[END_REF]. The advantages of laboratory experiments are that the conditions are controlled, a high measuring accuracy can be obtained and the total net sediment transport rate can be measured directly. However, due to limitations of the experimental facilities, many laboratory experiments were carried out under small-scale conditions, i.e. relatively short wave periods. One way to overcome this limitation is to perform experiments in a wave tunnel. This has the added advantage of isolating the processes from the effects of the wave's free surface, which aids in the measurement process (e.g. [START_REF] Sato | Oscillatory boundary flow and sand movement over ripples[END_REF][START_REF] Ribberink | Sediment transport in oscillatory boundary layers in case of rippled beds and sheet flow[END_REF][START_REF] Clubb | Experimental study of vortex ripples in full scale sinusoidal and asymmetric flows[END_REF][START_REF] Van Der Werf | Sand transport over rippled beds in oscillatory flow[END_REF].

Especially in the last decade many mathematical models have been developed to compute the sediment transport processes over ripples. Unsteady models, which are the most sophisticated, attempt to represent the detailed physical processes involved in sediment transport by waves and currents over rippled beds. They include one-dimensional models that resolve the vertical-and the time-dependent structure of the velocity and sediment concentration field (e.g. [START_REF] Davies | Modeling and measurement of sediment transport by waves in the vortex ripple regime[END_REF] for more practical applications, as well as two-and three-dimensional intra-wave models that represent the processes more completely (e.g. [START_REF] Longuet-Higgins | Oscillating flow over steep sand ripples[END_REF][START_REF] Hansen | Distribution of suspended sediment over wavegenerated ripples[END_REF][START_REF] Perrier | Numerical modelling of the flow and sand transport by waves and currents over a rippled bed[END_REF][START_REF] Watanabe | Three-dimensional boundary layer flow over ripples[END_REF][START_REF] Eidsvik | Large scale modelling of oscillatory flows over a rippled bottom[END_REF]. The net sediment transport can be obtained by integrating the cycle-and ripple-averaged sediment flux over the water depth. Intra-wave models can be divided into turbulenceclosure models, discrete-vortex models, Large Eddy Simulations (LES) models and Direct Numerical Simulations (DNS) models. DNS models have been used to represent the full three-dimensional nature of flows over rippled beds [START_REF] Scandura | Three-dimensional oscillatory flow over steep ripples[END_REF][START_REF] Blondeaux | Coherent structures in an oscillatory separated flow: numerical experiments[END_REF], but are currently limited to the laminar regime due to computational capacity restrictions. LES modelling is complex and still in development (Chang andScotti, 2003, 2006;[START_REF] Watanabe | Three-dimensional boundary layer flow over ripples[END_REF][START_REF] Tseng | Large-eddy simulation of turbulent wavy boundary flowillustration of vortex dynamics[END_REF]. These two modelling techniques are not included in the present study.

In this paper, we consider two two-dimensional vertical (2DV) sediment transport models: i) the k-ω model of [START_REF] Guizien | Equilibrium bottom roughness of ripples derived from a 2DHV ripple model[END_REF] and ii) the discrete-vortex, particle-tracking model developed by [START_REF] Malarkey | Discrete vortex modelling of oscillatory flows over ripples[END_REF] and [START_REF] Magar | Suspended sediment dynamics over rippled beds in oscillatory flows based on a 2DHV discrete-vortex/particle-tracking model[END_REF]. These models have not yet been validated against detailed measurements of sediment transport processes in oscillatory flows over full-scale, mobile ripples, mainly due to a lack of such measurements. Recently, a new dataset containing detailed oscillatory flow tunnel measurements of the transport processes over full-scale rippled beds for three different regular asymmetric oscillatory flows in a tunnel has been produced by [START_REF] Van Der Werf | Detailed measurements of velocities and suspended sand concentrations over full-scale ripples in regular oscillatory flow[END_REF]. In this paper we use the data from one of these experiments to test the two above-mentioned models. This is the first time that such a detailed intercomparison has been carried out.

This paper is organised as follows. Section 2 gives a short description of the two 2DV models. In Section 3 the experiment is presented. The ripple's periodicity and the sand's settling velocity are also discussed in relation to observations and modelling assumptions. In the next two sections the measured and predicted velocities, suspended sediment concentrations, suspended sediment fluxes and net transport rates are compared. Finally, Section 5 presents the conclusions.

2DV SEDIMENT TRANSPORT MODELS

The models represent the flow over one ripple cross-section assuming that the along-ripple variation can be neglected and the ripple remains fixed in space and time. They make use of a conformal mapping function to represent the ripple shape (see [START_REF] Malarkey | Discrete vortex modelling of oscillatory flows over ripples[END_REF]. The convention used in this paper is that the x-and y-axis represent the horizontal (cross-ripple) direction, and the vertical direction, respectively, with corresponding velocity components u and v, as shown schematically in Figure 1. All model quantities are periodic in the horizontal direction over a ripple wavelength λ. For simplicity, we chose the troughs to correspond to x = ±λ/2 (see Figure 1). Also, we assume that the presence of the sediment causes no stratification or turbulence damping effects. The models are described below.

k-ω model

The k-ω model of [START_REF] Guizien | Equilibrium bottom roughness of ripples derived from a 2DHV ripple model[END_REF] solves the conservation equations for the Reynolds-averaged velocity, u = (u,v), the turbulent kinetic energy, k, and the specific turbulent kinetic energy dissipation rate, ω, for the flow above ripples using the turbulence-closure proposed by [START_REF] Guizien | 1DV bottom boundary layer modeling under combined wave and current: turbulent separation and phase lag effects[END_REF]. By defining the velocity in terms of a stream function ψ: u = (∂ψ/∂y,-∂ψ/∂x) and the vorticity, Ω, as Ω = ∂u/∂y-∂v/∂x, the conservation equations can be rewritten as transport equations in Ω, k and ω. Besides, Ω and ψ can be related via Poisson's equation. This new system of equations together with one turbulence-closure equation defining the eddy viscosity or momentum diffusivity is solved numerically on an orthogonal, curvilinear grid based on the conformal mapping function with regular horizontal and exponential vertical spacing.

The boundary conditions for Ω and ψ are derived from those for the velocity: a no-slip condition on the ripple surface and the velocity being equal to the free-stream flow at the top of the model domain.

For k and ω the boundary conditions on the ripple surface are that k = 0 and that ω is specified in terms of the cross-ripple friction velocity and the Nikuradse roughness, based on law of the wall arguments [START_REF] Saffman | A model for inhomogeneous turbulent flow[END_REF]. At the top of the model domain, the vertical flux of k and ω are zero. Finally, as stated earlier, cyclic boundary conditions are applied to all quantities at the lateral boundaries of the computational domain.

With respect to the sediment dynamics, the transport equation for the concentration c is solved in an analogous way to the hydrodynamic equations, with the assumption that the sediment diffusivity is equal to the momentum diffusivity. At the bottom boundary, the concentration is taken as the largest of the reference concentration, c a, at two median grain diameters above the bed using the [START_REF] Engelund | A sediment transport model for straight alluvial channels[END_REF] formula and the concentration c b at the same height assuming zero flux perpendicular to the ripple surface. At the top of the model domain, a vertical zero-flux condition is applied. A more complete description of the k-ω model formulation can be found in [START_REF] Guizien | Equilibrium bottom roughness of ripples derived from a 2DHV ripple model[END_REF].

Discrete-vortex, particle-tracking (DVPT) model

The discrete-vortex, particle-tracking (DVPT) model of [START_REF] Malarkey | Discrete vortex modelling of oscillatory flows over ripples[END_REF] and [START_REF] Magar | Suspended sediment dynamics over rippled beds in oscillatory flows based on a 2DHV discrete-vortex/particle-tracking model[END_REF] solves the vorticity transport equation above the ripple directly by representing the vorticity field as a sum of a large number of point vortices. The point vortices are alternately advected by the surrounding flow and given a normally distributed random-walk jump of zero mean and fixed variance, scaled to mimic diffusion by the kinematic viscosity ν. The computations are carried out in a mapping plane with boundary conditions analogous to those used in the k-ω model, except that here the no-slip condition for the flow is achieved by introducing new vortices along the ripple surface at every time-step. The velocity and vorticity on the grid are calculated from the point vortices using the cloud-in-cell (CIC) method of [START_REF] Christiansen | Numerical simulation of hydrodynamics by the method of point vortices[END_REF]. The flow is phase-ensembled over 30 cycles and the resulting velocity and vorticity are comparable to their Reynolds-averaged counterparts from the kω model. A more complete description of the discrete vortex part of the model has been given by [START_REF] Malarkey | Discrete vortex modelling of oscillatory flows over ripples[END_REF].

The concentration transport equation is solved by an analogous method to the vorticity transport equation using a particle-tracking method (see [START_REF] Hansen | Distribution of suspended sediment over wavegenerated ripples[END_REF][START_REF] Perrier | Numerical modelling of the flow and sand transport by waves and currents over a rippled bed[END_REF] in a curvilinear coordinate system corresponding to the mapping plane. Sediment particles are continually released from the crest and advected by the phase-ensembled velocity, subject to a constant settling velocity and diffused via random-walk jumps with zero mean and fixed variance scaled to mimic the microscopic mixing coefficient ε. The size of ε was chosen as 400ν, as this value was found to produce a cycle-and ripple-averaged concentration profile that decayed exponentially with height, as expected from observations (e.g. [START_REF] Thorne | Measurements of near-bed intra-wave sand entrainment above vortex ripples[END_REF]. At its release, each 'notional' particle is assigned a mass based on an excess shear stress criterion [START_REF] Nielsen | Coastal bottom boundary layers and sand transport[END_REF]. The shear stress is calculated using Fredsøe´s (1984) boundary layer theory. At the ripple surface, particles having hit the bed through advection and settling are eliminated from the computations, while those having hit the bed through a random jump are reintroduced into the suspension layer. The concentration on the grid is determined by the CIC method and phase-ensembling is over 15 cycles. A more complete description of the particle-tracking part of the model can be found in [START_REF] Magar | Suspended sediment dynamics over rippled beds in oscillatory flows based on a 2DHV discrete-vortex/particle-tracking model[END_REF].

THE MR5B63 EXPERIMENT

Below follows a concise description of the Mr5b63 experiment. A more complete one can be found in [START_REF] Van Der Werf | Detailed measurements of velocities and suspended sand concentrations over full-scale ripples in regular oscillatory flow[END_REF].

Experimental facility

The Mr5b63 experiment was performed by [START_REF] Van Der Werf | Detailed measurements of velocities and suspended sand concentrations over full-scale ripples in regular oscillatory flow[END_REF] in the Aberdeen Oscillatory Flow Tunnel (AOFT). The AOFT is a large laboratory facility in which near-bed horizontal flows, equivalent in period and amplitude to the near-bed horizontal flows induced by full-scale waves, can be generated over sediment beds. The AOFT has an overall length of 16 m with a 10 m long, glasssided rectangular test section, 0.75 m high and 0.3 m wide, which has a closed top. The test section was filled with a 0.25 m thick sediment bed leaving 0.5 m for the flow.

Measurements

Flow velocities over the ripples were measured using a cross-correlation Particle Image Velocimetry (PIV) system based on a double-pulsed Nd-Yag laser light sheet and a 1k × 1k cross-correlation camera. The camera and laser are synchronised and the camera grabs a pair of images, with a 2 ms time separation, at a rate of approximately 13.2 Hz. For the present experiments, measurements were made using a camera viewing area of 400×400 mm. The image pairs were analysed using a 32×32 pixel interrogation area with 50% overlap. The spatial resolution of the resulting velocity field measurement is 6.4 mm horizontally and vertically and the velocity resolution is 20 mm/s. In order to minimise the effects of ripple migration and distortion, the PIV-measured velocities were phase- ensembled over 5 flow cycles. The limit that this comparatively small number of cycles places on the interpretation of the results is discussed later (Section 4.1).

The suspended sediment acted as the seeding agent, and therefore measured velocities are those of the suspended sediment rather than the water itself. The settling-velocity effect was removed from the PIV data by first forcing the velocity data to be periodic in the horizontal direction and then removing the ripple-averaged vertical velocity at each height (as this quantity must be zero at all heights to satisfy continuity). This procedure is explained in more detail in Appendix A. As seen from Figure A1 in the Appendix, the velocity satisfies better the condition of local tangency at the rippled bed with this correction than without it. The size of the ripple-averaged vertical velocity component that is removed will be returned to in the discussion of the choice of settling velocity. The method for calculating the vorticity from the PIV data is explained in Appendix B.

Ripple dimensions were measured using a Laser Displacement Sensor (LDS) mounted on a positioning carriage. The LDS made point measurements of bed elevation with a 0.05 mm resolution in the vertical direction. The laser displacement system was used to measure the equilibrium rippled bed. Six parallel profiles were measured, spaced at 40 mm intervals across the tunnel width, with heights measured every 5 mm along each profile.

An Acoustic Backscatter System (ABS) was deployed to measure time-dependent suspended sediment concentrations. The ABS collected backscatter profiles at 128 Hz. The ABS data were block averaged to give backscatter profiles measured at 8 Hz and these are subsequently converted to high-resolution concentration profiles. The averaging is required, because of the statistical nature of the backscattered signal. The system provides concentration profiles with a 0.005 m vertical spatial resolution.

Calibration was based on concentrations measured by an optical concentration meter (OPCON) and a transverse suction system (TSS). [START_REF] Van Der Werf | Detailed measurements of velocities and suspended sand concentrations over full-scale ripples in regular oscillatory flow[END_REF] showed that the ABS-and OPCONmeasured time-dependent concentration profiles were very similar, thus demonstrating the consistency of the concentration measurements.

10

The ABS measured continuously while six entire equilibrium ripples migrated underneath it. The bed level below the ABS was continuously monitored by a sonar ripple profiler. The ABS concentration measurements are accurate within approximately a factor of 2 due to uncertainties in the suspended sand grain size (Chris Vincent, personal communication) with an uncertainty in the vertical position of ±5 mm.

A good approximation of the concentration as a function of time can be found by phase-ensembling over a sufficiently large number of flow cycles. Since the ripples migrate, phase-ensembling over a number of flow cycles also implies bed-averaging. It is assumed that a spatial resolution of 5% of the ripple length (20 mm) is acceptable, which corresponds to phase-ensembling the ABS concentrations over 22 flow cycles. This approach is similar to that of [START_REF] Thorne | Measurements of near-bed intra-wave sand entrainment above vortex ripples[END_REF]. With this number of flow cycles the typical time-dependent concentration behaviour (number and timing of concentration peaks) is captured. As with the PIV data, the concentration is forced to be periodic over the ripple wavelength.

A Transverse Suction System (TSS) was used to measure the time-averaged suspended sediment concentration at 5 different elevations above the bed. The estimated total (random) error in the measured concentration is 5%, and the uncertainty in vertical position with respect to the ripple surface ±4 mm. This estimation does not include uncertainties associated with the ripple size and shape.

The net sediment transport rates were calculated using a mass conservation technique. Given the sediment masses in the traps at both ends of the test section and the volume change derived from the bed profiling system, the net sediment transport rates along the test section were calculated by integrating in from the left-hand and the right-hand boundaries. The calculated net transport rate in (or close to) the middle of the test section was taken as a representative value. Repeated measurements for the same condition show that the (random) error in the measured net transport rate is around 20%.

Experimental conditions

The sediment used for the Mr5b63 experiment was well-sorted with a median grain diameter D 50 = 0.44 mm and a grain size distribution as shown in Figure 2.

The free-stream velocity, u ∞ , (positive x corresponds to the 'onshore' direction) was based on a wavetunnel equivalent to near-bed flow beneath Stokes second-order waves, namely:

), γ σ ( 2 cos ) γ σ cos( 2 1      t U t U u (1)
where t is time,  = 2π/T is the angular frequency, T is the flow period, U 1 and U 2 are the first and second harmonic velocity amplitudes and  is the phase such that u  (0) = 0:

. 4

8 arccos γ 2 1 2 2 2 1            U U U U (2)
Here T = 5.0 s, U 1 = 0.54 m/s and U 2 = 0.09 m/s. This asymmetric oscillatory flow in the tunnel gave rise to a cycle-mean flow that was measured using the PIV system. The ripples for this experiment were highly two-dimensional with height η = 0.078 m and length λ = 0.41 m. The ripple size in the central 4 m of the test section (where the measurements were carried out) was very regular: the relative standard deviation of both the ripple height and length was 13%.

Model inputs

Both models were driven hydrodynamically by the free-stream velocity specified in Eqs. ( 1) and ( 2) and matched to the cycle-and ripple-averaged velocity as determined by the PIV measurements at a particular height in the same way as is done for 1DV modelling (e.g. [START_REF] Holmedal | Boundary layer flow and net sediment transport beneath asymmetric waves[END_REF].

The matching height above the ripple trough was taken to be 0.84λ since this represented a suitable height outside the wave boundary layer where the velocity shear was zero. The Nikuradse roughness on the ripple surface, k s , was given by the median grain size of the bed sediment D 50 , k s = 2.5D 50 . In this intercomparison the modellers were also provided with the experimental ripple profile and the settling velocity in suspension. In the interests of minimising the differences in inputs between models, both models used the same ripple profile and settling velocity in suspension. The method of calculating these two inputs are explained in the next two sections.

The ripple profile

Both models use the same conformal mapping function to define the orthogonal curvilinear grid. The ripple was forced to be periodic and then fitted to the mapping function (see [START_REF] Malarkey | Discrete vortex modelling of oscillatory flows over ripples[END_REF] using 16 coefficients. The ripple generated by the mapping function and that measured are depicted in Figure 3. It is clear that the fitted and the measured ripples are in excellent agreement and that forcing periodicity only results in a slight difference at the ripple troughs.

The settling velocity

For modelling purposes it is assumed that the grain size, D, is log-normally distributed, such that the cumulative frequency F(D) can be expressed as:

  . σ log 2 / log erf 1 5 . 0 ) ( g 50                   D D D F (3)
Eq. ( 3) was fitted to the lower end of the distribution using the experimental values of D 10 = 0.25 mm and D 50 = 0.44 mm to obtain the value of σ g (= 1.55). It can be seen in Figure 2, where the log-normal fit is compared to the measured distribution, that Eq. ( 3) fits the experimental cumulative frequency at the lower end of the grain size distribution reasonably well.

An estimation of the settling velocity, v s , can be obtained by calculating the maximum grain size that can get into suspension using [START_REF] Fredsøe | Mechanics of coastal sediment transport[END_REF] criterion. A k-ε model, see for example [START_REF] Malarkey | A simple model of unsteady sheet flow sediment transport[END_REF], with the Mr5b63 hydrodynamic conditions and a flat rough bed with k s = 13 2.5D 50 , produces a maximum skin friction u *m equal to 4.57 cm/s. Fredsøe and Deigaard's criterion states that the maximum settling velocity, v sc , of the sediment in suspension is 0.8u *m , which here leads to a maximum settling velocity of 3.65 cm/s. The maximum settling velocity corresponds to sediment grains with critical grain diameter D crit = 0.295 mm, as deduced from [START_REF] Hallermeier | Terminal settling velocity of commonly occuring sand grains[END_REF] formula. Now, from Eq. (3) it may be found that F(D crit ) = 0.183, which means that only 18% of the grain population can get into suspension. Since F(D 50,s ) = 0.5F(D crit ), this implies that D 50,s = 0.244 mm (D 50,s and D crit are shown in Figure 2). Thus, the (median) settling velocity of the sediment in suspension is v s = 2.97 cm/s. It should be pointed out that while the entire sediment population is mobile, only sizes finer than D crit = 0.295 mm are suspended and hence all suspended sediment quantities need to be multiplied by F(D crit ) = 0.183.

It is interesting to compare this settling velocity estimate with the ripple-averaged vertical velocities, <v>, which were removed from the PIV data. This comparison is shown in Figure 4. First of all it can be seen that <v> takes values that are consistent with a grain size dependent settling velocity: it is overwhelmingly negative and decreases in magnitude with height above the bed (representing the fining of sediment with height). At each height there is a lot of scatter in <v> but it appears to be quite consistent with v s , thereby confirming this choice of settling velocity in the model simulations.

INTERCOMPARISON OF THE HYDRODYNAMICS

Vorticity dynamics

Figure 5 shows the measured and predicted non-dimensional vorticity (λ/U 1 ) above the rippled bed at phase angles σt = 0° to 300° in steps of 60°. Positive, 'onshore' flow is directed to the right and σt = 0° corresponds to the off-onshore free-stream reversal. Each panel is one ripple wavelength long and one half ripple wavelength high. Figure 5 shows vorticity contours predicted by the two models, together with those from the PIV measurements.

At σt = 0° both models and data clearly show a vortex with negative (anti-clockwise) vorticity, being ejected into the water column as the flow begins to move from left to right. This vortex, labelled 'F o ', was generated in the previous offshore flow half cycle. The proximity of vortex F o to the bed, together with the no-slip condition on the bed, generates shear that results in a counter-rotating strip of vorticity, labelled 'C', beneath the vortex (see e.g. [START_REF] Malarkey | Discrete vortex modelling of oscillatory flows over ripples[END_REF]. This strip of vorticity is present at phases 0°, 120° and 300° for both the DVPT model and the data, but only present at phase 120° for the k-ω model. If we look at phase σt = 60°, the free-stream is moving onshore and accelerating. Vortex F o has now been advected onshore, and has reached the offshore side of the neighbouring onshore ripple and weakened in strength. By the time the phase reaches 120°, vortex F o appears to be in roughly the same position as at phase 0° (slightly to the left of the crest) but is now two ripple wavelengths away from its parent ripple. The elevated vortex labelled 'N o ', visible at phases of 0° and 60°in the DVPT model and 0° in the data, is actually the vortex created during the previous wave half cycle on the onshore side of the ripple.

The motion of the vortex N o during the first half cycle depicted in Figure 5 is in the onshore direction, just as with vortex F o , but by 120°the strength of N o is very small. Moreover, between σt = 0° and 120° a new vortex with positive vorticity, labelled 'N', has been forming on the onshore side of the ripple, starting with the contribution of the offshore counter-rotating strip of vorticity C at σt = 0°. As the phase advances, vortex N grows as it is in constant contact with the bed.

In general, over phases 0°, 60° and 120° both models predict the horizontal position of the vortices reasonably well. By the time the phase is 120° vortex F o in the data is just beginning to break up into more than one region of vorticity. The DVPT model predicts the vertical position, physical size and strength of the vortices better than the k-ω model. The underprediction by the k-ω model of the heights reached by the vortices particularly when they are growing is related to the underprediction by this model of the counter-rotating strip of vorticity, which is present in both the DVPT predictions and the experiments. As observed above, the k-ω model predicts this strip of vorticity only when the phase is 120°. This is possibly due to the boundary condition on the rippled bed: the k-ω model considers that the vorticity depends explicitly on the second derivative normal to the bed of the stream function and on the value of the Jacobian at the bed. The latter quantity is very sensitive to the steepness of the ripple. Therefore, given that in the experiment analysed here the ripple steepness is quite low, the k-ω model underpredicts the extent to which the vorticity separates. This is supported by the fact that in Magar et al. (2005), where an experiment with a steeper ripple was studied, the vorticity calculated by the k-ω model separated much more readily from the bed (see also [START_REF] Fredsøe | Wave plus current over a ripple-covered bed[END_REF].

Next, we compare the model predictions and experimental data for phase angles after on-offshore flow reversal, which occurs at σt = 162°. Thus, the first phase after flow reversal is σt = 180°, when the free-stream is moving offshore and the left side of the ripple is now the lee-side. Since the flow has reversed, vortex N has been ejected and is now being advected offshore. Given that at σt = 180° only about 18 o have elapsed since flow reversal, this vortex has not moved very far from its parent ripple, as can be clearly seen in Figure 5. However, there has been sufficient offshore flow for a new vortex F to start forming on the offshore side of the ripple. It is interesting to note that, in spite of the freestream velocity asymmetry, the horizontal positions of vortex F o between 0° and 120° and vortex N between 180° and 300° are the mirror images of one another. This is particularly true for the DVPT model, but less the case for the measurements and even lesser the case for the k-ω model. Thus in the DVPT model the slower offshore advection is more or less compensated for by considering phase instants which are 18 o ahead in phase, relative to flow reversal, than their onshore counterparts. The free-stream asymmetry, however, does result in a difference in strength between vortices generated in each of the two half flow cycles. It can be seen from both models and the data that the vorticity of vortex N is more diffuse than that of vortex F o . In the case of the data this results in the position of vortex N being more ill-defined than the vortex F o once ejected. When the phase is 360°, vortices N and F become N o and F o and the process repeats itself in the next flow cycle.

Finally, it should be noticed that the vorticity contours derived from the velocities measured experimentally appear quite noisy. This is probably related to the fact that the velocities were phaseensembled for only five cycles, which may not be enough to remove the turbulent component from the velocity field. This is also the reason for the ill-defined nature of the vortex positions, after they have been ejected. However, both the models and experiments reproduce qualitatively well the vortex dynamics above ripples that has been reported in previous studies, both for oscillatory flows (e.g. [START_REF] Malarkey | Discrete vortex modelling of oscillatory flows over ripples[END_REF] and for surface waves and currents (e.g. Fredsoe et al., 1999). The consistency and uncertainty of the vorticity is discussed further in Appendix B.

Cycle-averaged flow field and vorticity contours

The cycle-averaged flow fields and corresponding vorticity contours are shown in Figure 6. Cycleaveraging the time-varying flow and vorticity results in two circulation cells, one on either side of the ripple. These cells are mainly the result of the growing vortices, which are almost static when they are forming, and at their strongest just before ejection [START_REF] Malarkey | Discrete vortex modelling of oscillatory flows over ripples[END_REF]. It can be seen from the vorticity comparisons that the DVPT model predicts the position, strength and extent of the circulation cells reasonably well. The k-ω model, on the other hand, underpredicts quite substantially the height of the circulation cells and their extent, but predicts their strengths reasonably. These observations are consistent with those made for the time-varying vorticity comparisons.

Both models show an asymmetry in the circulation cells that is consistent with the data, as demonstrated by the relative position of the zero vorticity contour line. However, in the case of the DVPT model there is insufficient difference in the vertical position of the cells, so that the asymmetry appears weaker. In general however, the DVPT model produces comparable velocities to the data above the ripple but overpredicts the velocity close to it. The k-ω model, on the other hand, underpredicts the velocity higher above the ripple, but predicts to a better extent the velocity near the ripple. This is probably because only the k-ω model resolves the boundary layer properly, as it considers the grain roughness at the ripple surface (see [START_REF] Guizien | Equilibrium bottom roughness of ripples derived from a 2DHV ripple model[END_REF].

Cycle-and ripple-averaged horizontal velocities

Next, we consider the cycle-and ripple-averaged horizontal velocities above the ripple crest, shown in Figure 7. As explained in Section 3.4, both models were matched to the cycle-and ripple-averaged experimental velocity at a certain height (0.84λ above the ripple trough) and this can clearly be seen in Figure 7.

The DVPT model predicts the velocity profile reasonably well, including the correct position of the offshore and onshore maxima. However, the magnitude of the onshore maximum is slightly overpredicted and the magnitude of the offshore maximum is more substantially underpredicted, especially below the crest level. This latter underprediction probably relates to the insufficient difference in the vertical position of the circulation cells (discussed in Section 4.2). The k-ω model, on the other hand, predicts the velocity close to the ripple better than the DVPT model but fails to capture the onshore maximum because the circulation cells are too close to the bed.

INTERCOMPARISON OF THE SUSPENDED SEDIMENT DYNAMICS

Time-dependent suspended sediment concentrations

Figure 8 shows the measured and predicted suspended sediment concentration contours above the rippled bed at phase angles σt = 0° to 300° in steps of 60°. The concentration scale is logarithmic and the same in all the figures; concentrations are in [g/l]. As with Figure 5, each panel is one ripple wavelength long and one half ripple wavelength high. For clarity a minimum concentration of 0.01 g/l has been assumed for the models.

In the first instance, we compare the concentration contour plots from the DVPT model (left-hand panels in Figure 8) to the corresponding vorticity contour plots from that same model, as illustrated in Figure 5. The comparison of those plots indicates that the positions of the sediment clouds, at each phase, agree reasonably well with the positions of the vortices along and above the ripple surface, at that same phase. Also, there is good correlation between the regions of large vorticity magnitude and the regions of large sediment concentration.

Next, we compare the right-hand panels in Figure 8 showing the concentration contours predicted by the k-ω model with the k-ω model vorticity contours at the corresponding phases (shown in Figure 5).

Again, the sediment clouds seem to follow the vorticity contours quite closely, except at flow reversal.

For instance, when the phase is 180° we would have expected the sediment cloud seen at 120° to have moved close to the ripple crest or possibly further offshore, in response to the advection of the vortex ejected at around 162 o in the offshore direction. The same is true of the flow reversal at 0°, except here the sediment cloud appears to be ahead of the corresponding vortex. Similar behaviour is observed in the experiments at flow reversal. This happens because the suspended sediment is redistributed not only by advection but also by diffusion and settling. This may explain as well the differences between the concentration contours predicted by the twomodels. In fact, the DVPT model does not have any spatial variation in diffusion, nor is there any substantial source of sediment from the bed other than that released at the crest. This is probably why the concentration and vorticity mirror one another in the DVPT model results. The k-ω model, on the other hand, which does include entrainment of sediment along the entire ripple surface, may have too much variation in diffusion.

Finally, we investigate how each model compares with the experimental concentration plots, shown in the middle panels of Figure 8. The models are capable of reproducing the time-dependent concentration field reasonably well, both qualitatively and to some extent quantitatively. By analogy with the vorticity dynamics, a suspended sediment cloud develops on the onshore (right-hand) side of the ripple during the onshore flow half cycle. This suspended sediment cloud is ejected into the flow near the on-offshore flow reversal. During the offshore flow half cycle, a suspended sediment cloud is generated on the offshore side of the ripple, which is then ejected into the flow near the off-onshore flow reversal. The suspended sediment cloud ejected near the on-offshore flow reversal (σt = 162°) is larger and contains more sediment than the suspended sediment cloud ejected around the off-onshore flow reversal (σt = 0°). This is due to the flow asymmetry: the onshore velocities are stronger than the offshore velocities. This asymmetry seems to be well represented by the k-ω model, but tends to be underpredicted by the DVPT model. This is as expected since the diffusion in the DVPT model is not velocity dependent, unlike in the k-ω model, and the vorticity asymmetry is underpredicted.

Along the ripple surface the k-ω model is better able to represent the concentration than the DVPT model because of a better near-bed turbulence description. However, as we already observed in Figure 5, the vortices are not lifted high enough up into the water column in the k-ω model, and this is reflected in the suspended sediment dynamics (closest agreement between model and data occur at the same phases, 120 and 180º, for both the vorticity and concentration). The model produces sediment clouds that are very close to the bed at phase σt = 0°, whereas the measurements show considerable amounts of sediment in suspension (as a result of the clouds generated by vortex shedding) that have not settled to the bed completely. However at σt = 120°, and especially at σt = 180°, even though vorticity is underpredicted by the k-ω model, sediment diffusion appears to be sufficient to produce concentration contours that are comparable to the measured ones. This may suggest that diffusion in the k-ω model is possibly larger than it should be, as pointed out previously. However, this is not the case for the remaining phases depicted in Figure 5 (σt = 0, 60, 240 and 300°), when the sediment diffusion is not large enough to compensate for the underprediction of vorticity. It is difficult to test whether the amount of diffusion in the k-ω model is appropriate in this case for two reasons. Firstly, Reynolds stresses, on which diffusion is scaled, cannot be reliably obtained from the experimental data because of the limited number of flow cycles and the resolution of the velocity measurements.

Secondly, the vorticity is not being predicted correctly by the k-ω model: one would expect the distribution of Reynolds stresses to mirror the distribution of vorticity [START_REF] Sato | Oscillatory boundary flow and sand movement over ripples[END_REF][START_REF] Perrier | Numerical modelling of the flow and sand transport by waves and currents over a rippled bed[END_REF].

With the exception of the case when σt = 180° (discussed earlier), the DVPT model shows better agreement with the measurements than the k-ω model in terms of the positions and size of the suspension clouds (higher concentrations at higher elevations above the bed). However, it seems that the DVPT model does not include enough diffusion, especially higher above the rippled bed. As a result, the concentrations within the ejected vortices are overpredicted and the concentrations outside the ejected vortices are underpredicted.

Cycle-and ripple-averaged suspended sediment concentrations

Figure 9 shows the measured (by the ABS and TSS) and predicted vertical profile of the cycle-and ripple-averaged concentrations. The ABS-measured concentrations below the ripple crest level are noisy compared to those above because of the comparatively small number of non-zero measured concentration values contributing to the horizontal average. It can be seen that the ABS and TSS measurements are in good agreement with one another.

In general there is reasonable agreement between the measured and predicted concentration profiles.

The k-ω model prediction and the data show that the cycle-and ripple-averaged concentrations decay exponentially, as expected (see [START_REF] Nielsen | Coastal bottom boundary layers and sand transport[END_REF]. Close to the ripple crest (0.19 < y/λ < 0.3) the DVPT model slightly overpredicts the concentration (prediction is within a factor of 1.5) and the k-ω model underpredicts the concentration (prediction is within a factor of 3). Below the ripple crest (0 < y/λ < 0.19) the DVPT model predicts the concentration better than the k-ω model. This is partly because the DVPT model tends to produce the vortex positions and corresponding concentration contours that are closer to those observed as already discussed, and partly because the k-ω model produces concentrations that are very large right along the ripple surface. In the case of the DVPT model, the local over-and underpredicted concentrations described in Figure 8 for the DVPT model compensate for one another to produce a reasonable cycle-and ripple-averaged concentration. Above y/λ = 0.35 both models underpredict the concentration, but by y/λ = 0.43 the k-ω predicts the larger concentration. In this upper region the k-ω model produces the more reasonable concentration probably because of its better representation of background turbulence in the outer flow.

Net suspended sediment fluxes and transport rates

The time-varying velocity and ABS concentration measurements can be combined to produce timevarying suspended sediment fluxes. The net (cycle-averaged) suspended sediment flux has a waveand current-related component (defined by

      c u uc and    c u , respectively,
where the overbar denotes a cycle average and the angled brackets denote a ripple average, see also [START_REF] Van Der Werf | Detailed measurements of velocities and suspended sand concentrations over full-scale ripples in regular oscillatory flow[END_REF]. Figure 10 shows comparisons between the measured and predicted currentrelated, wave-related and net cycle-averaged, ripple-averaged suspended sediment fluxes, respectively. Again the measured fluxes below the ripple crest level are noisy compared to those above the crest because of the comparatively small number of non-zero measured flux values contributing to the horizontal average.

Both the DVPT and k-ω models accurately predict the current-related suspended sediment flux above the ripple crest level. Below this level the DVPT model underestimates and the k-ω model slightly overestimates the current-related flux. The current-related flux is in the offshore direction due to the relatively strong offshore mean flow in combination with the relatively high sediment concentrations above the onshore (right-hand side) ripple flank (see Figures 6 and8). Due to the phase lags between the near-bed peak flow velocities and suspended sediment concentrations related to the vortex ejection at times of flow reversal (see Figures 5 and8), the wave-related suspended sediment flux is offshore as well. This process is represented by both models. The k-ω model systematically underpredicts the wave-related flux above the ripple crest level and consequently the net cycle-averaged flux. The DVPT model does better, except close to the ripple's crest, where it predicts a large onshore waverelated flux peak. The latter is related to the fact that the DVPT model releases the majority of its sediment particles near the crest at a point where the velocity is already overpredicted. Thus, because the velocities are stronger and, consequently, the sediment particle 'masses' are larger for the onshore flow, the result is an onshore near-crest peak in the wave-related and net fluxes. This may indicate the need for a more diffuse injection point of particles near the crest, together with a better representation of the boundary layer.

If we vertically integrate these ripple-averaged suspended sediment fluxes from the ripple trough upwards, we get the corresponding suspended sediment transport rates (see Table 1). Again we distinguish between the wave-related, current-related and net cycle-averaged suspended transport rate.

The models predict an 'offshore' (negative) net suspended sediment transport, i.e. against the orbital flow asymmetry, as seen in the data. The k-ω model in particular predicts the value of the currentrelated suspended transport and the net suspended transport well. Both the wave-and current-related suspended transport rates are offshore and the current-related component dominates over the wave-related component. The k-ω model underpredicts the wave-related transport component, which is partly the result of an underprediction of the height to which the suspended sediment clouds are lifted (see Figures 5 and8). The values of the net suspended transport rate predicted by the DVPT model are lower than that of the data. This is caused by a lack of asymmetry in the velocity and suspended sediment concentration during onshore and offshore flow (see Figures 5 and8).

It is well known that the total transport rate in general and the bedload transport rate along the ripple surface in particular are responsible for ripple migration [START_REF] Traykovski | Geometry, migration, and evolution of wave orbital ripples at LEO-15[END_REF][START_REF] Hoekstra | Bedform migration and bedload transport on an intertidal shoal[END_REF]. However in this experiment, in the ripple-averaged sense, it is clearest to define the cycleaveraged total transport rate <q t > as simply:

,

r s t         q q q (4)
where <q s > is the net transport contribution from the cycle-and ripple-averaged suspended sediment, as defined and discussed above, and <q r > is the transport contribution from the migration of the ripple itself. During the experiment, the ripples migrated onshore with a fairly constant rate of 15 to 20 mm/min. If we assume that the ripple shape, as depicted in Figure 2, remains constant while it migrates, and that there is no transport below the ripple-trough level, <q r > can be estimated by: ,

η ) ε 1 ( p r r     f c q (5)
where c r is the ripple migration rate, f is the dimensionless shape factor (given by the area under the curve shown in Figure 3 multiplied by 1/(λη)) and ε p (≈ 0.4) is the bed porosity. Since f is 0.507 in this case, we find that <q r > = 6.91.0 mm 2 /s. The total net sediment transport, measured directly from mass conservation (see Section 3.2), is <q t > = -3.70.7 mm 2 /s, so the contribution of the cycle-and ripple-averaged suspended sediment may be deduced from Eq. ( 4) to be <q s > = -10.6±1.7 mm 2 /s. It can be seen that this value of <q s > agrees closely with that obtained previously from the integration of flux profiles (-11.9 mm 2 /s) considering the uncertainties in the measurements. For experiment Mr5b63, the total sediment transport is offshore as a result of the greater offshore suspended transport relative to the smaller onshore ripple-migration transport. Bearing in mind the grain size in suspension (Section 3.4.2), this implies that grains larger than 0.295 mm are being transported onshore while grains smaller than 0.295 mm are being transported offshore. While the suspended transport in this case has proved to be the most significant component, this discussion highlights the need for models to be able to predict migration rates as well.

CONCLUSIONS

We have tested two different 2DV sediment transport models against detailed velocity and sediment concentration measurements above full-scale mobile ripples in regular asymmetric oscillatory flow.

The models considered are the k-ω turbulence-closure model of [START_REF] Guizien | Equilibrium bottom roughness of ripples derived from a 2DHV ripple model[END_REF] and the discretevortex, particle-tracking (DVPT) model of [START_REF] Malarkey | Discrete vortex modelling of oscillatory flows over ripples[END_REF] and [START_REF] Magar | Suspended sediment dynamics over rippled beds in oscillatory flows based on a 2DHV discrete-vortex/particle-tracking model[END_REF]. This is the first time that such a detailed model intercomparison has been carried out.

A procedure to remove the settling velocity effect from the velocity data was successfully applied. The measured velocity field was first forced to be periodic in the horizontal direction and then the rippleaveraged vertical velocity at each height was removed from the data, because this quantity must be zero at all heights to satisfy continuity. This procedure was necessary since the measured velocities are those of the suspended sediment rather than the water itself. The resulting flow field satisfies better the condition of local tangency close to the rippled bed and the values of the removed ripple-averaged vertical velocities appear to agree reasonably well with the computed settling velocity of suspended sediment.

The models and the data demonstrate that the time-dependent velocity and suspended sediment concentration above rippled beds are dominated by the generation of lee-side vortices and their subsequent ejection at flow reversal. In general, the flow dynamics are reasonably well predicted by the models. The DVPT model predicts the horizontal and vertical positions and strengths of the vortices at all phases reasonably well, but tends to overpredict the velocity close to the ripple surface because it does not represent the turbulent boundary layer characteristics properly. The k-ω model, on the other hand, underpredicts the height reached by the vortices, and most of the vorticity tends to occur close to the ripple surface. However, this model is better able to predict the velocity close to the bed. The k-ω model's underprediction of the heights to which vorticity is lifted is thought to be related to the boundary condition on the vorticity at the ripple surface, which is sensitive to the ripple steepness. The quantitative agreement between the measured and predicted cycle-and ripple-averaged horizontal velocity is reasonable. Both models reproduce the measured offshore flow close to and below the ripple and the DVPT model reproduces the measured onshore flow higher up.

The DVPT model reproduces the horizontal and vertical extent of the growing and ejected sediment clouds reasonably well. However, it overpredicts the concentrations within the vortices and underpredicts the concentrations outside the vortices. This may be because sediment diffusion is not large enough in the DVPT model, and also because it is uniform across the field. The k-ω model underpredicts the height to which the suspended sediment clouds are lifted, which is the result of the vortices not being lifted high enough up into the water column. However, in the k-ω model the diffusion is possibly larger than it should be, but this is difficult to test. It is important to note here that the main shortcoming of the k-ω model, and of Reynolds-averaged Navier-Stokes models in general, is that it is difficult to obtain the right balance between vorticity and diffusion [START_REF] Chang | Entrainment and suspension of sediments into a turbulent flow over ripples[END_REF]. This depends on the turbulence-closure condition, and the difficulties in calibrating the parameters in it. The measured and predicted cycle-and ripple-averaged suspended sediment concentrations are in reasonable agreement and demonstrate a region of exponential decay. The concentrations are better predicted by the DVPT model in the vicinity of the growing and ejected vortices and by the k-ω model higher up in the water column and along the ripple surface.

The models are able to reproduce the observed 'offshore' cycle-and ripple-averaged current-and wave-related suspended sediment fluxes from the ripple troughs upwards. As a result, the predicted and measured net suspended sediment transport rates are also offshore and are in reasonable agreement with each other. The measured net suspended transport based on flux measurements was found to be consistent with the net suspended transport deduced from the difference between the measured total transport and the transport associated with the measured ripple migration. For the particular experiment considered in this paper, the net offshore suspended transport, which is the result of only a small fraction (18%) of the grain population of the bed, is greater than the net onshore transport due to ripple migration, which involves the entire grain population. However, other conditions of flow and sediment size may result in a net onshore rather than offshore transport, and numerical models, including the 2DV-type models considered here, need to account for ripple migration if they are to properly capture the overall transport process.

2. In anticipation of the vorticity calculation, the PIV velocity data was first interpolated onto a grid such that an integer number of grid cells corresponded to a ripple wavelength (Δx = 6.6129 mm as opposed to 6.4 mm such that λ = 62Δx but Δy was still 6.4 mm).

3. A ripple average of the vertical velocity was taken at each height and each time instant and then removed from the data (since the ripple-averaged vertical velocity must be zero in order to satisfy continuity).

The effect of these changes on the cycle-mean velocity vectors is shown in Figure A1 (the 'with correction' panel is the same velocity field as that depicted in Figure 6, except that here every point is shown). It can be seen that the procedure improves the velocity field estimates, especially near the ripple surface such that the vectors satisfy better the condition of local tangency close to the ripple. It can also be seen that the forcing of periodicity is useful because it helps to fill in the gaps in the data, using linear interpolation.

The two phases chosen (0 and 180º) are depicted in Figure A2 for phase-ensembling over 1, 2, 3, 4, and 5 wave cycles. Also depicted is the standard deviation of the individual realisations relative to the phase-ensemble.

In the 0º case, it is clear that the basic structure of the strongest vorticity in the field, around the newly ejected vortex F o , is preserved very well through successive ensembles. Also preserved through the phase-ensembling, though not as well, is the old onshore vortex, F o . However, in the regions of lower vorticity no discernable structure is preserved. From the standard deviation panel it can be seen that only the preserved features have a standard deviation that is smaller in magnitude than the feature itself. In the 180º case, the structure of the growing vortex, F, is again well preserved but the structure of the newly ejected vortex N is not preserved as well, as reflected by the larger and more scattered standard deviation. One initial feature that is lost to the phase-ensembling altogether is the negative vorticity that appears to the left of vortex N. The fact that this feature decreases in strength consistently and finally disappears suggests that it is probably spurious. It is clear that while phaseensembling over a larger number of cycles would probably better capture the vorticity structure, particularly in the 180º case, the most important features of the flow are preserved.

FIGURE CAPTIONS Figure 1

Definition sketch of the modelling domain for the 2DV models.

Figure 2

Grain size distribution for the Mr5b63 experiment (asterisks), showing the maximum critical grain diameter that can get into suspension (D crit ), the median grain diameter in suspension (D 50,s ) (open circles) and a log-normal fit to the lower part of the distribution (solid line) (see Section 3.4.2).

Figure 3

Comparison of experimental (dots) and fitted (solid line) ripple profile.

Figure 4

Comparison of the ripple-averaged vertical velocity from the PIV data and the settling velocity based on Fredsøe and Deigaard's (1992) approach.

Figure 5

Non-dimensional vorticity contours, λ/U 1 , (clockwise is positive, spacing = 2) predicted by the DVPT model (left panels), measured in the experiment (middle panels), and predicted by the k-ω model (right panels) for the phases shown (degrees).

N o and F o denote the on-and offshore generated vortices from the previous wave cycle, N and F denote the on-and offshore generated vortices from the present wave cycle and C denotes the counter rotating strip of vorticity. The arrows are proportional to the strength of the free-stream velocity.

Figure 6

Cycle-averaged velocity vectors and vorticity contours (non-dimensionalised by U 1 and U 1 /λ, respectively) for the data, the DVPT model, and the k-ω model (common velocity scale shown in DATA panel).

Figure 7

Cycle-and ripple-averaged horizontal velocities predicted by the DVPT (solid line) and k-ω (dashed line) models and measured in the experiment (closed circles).

Figure A1

The effect of forcing periodicity in the ripple wavelength and removing the horizontally-averaged vertical velocity on the cycle-mean velocity field (vertical lines correspond to where periodicity is forced and the common velocity scale is shown in the top panel).

Figure A2

Non-dimensional vorticity contours (Ωλ/U 1 ) at the labelled phases for phaseensembling over different numbers of wave cycles (number in brackets) and standard deviation of the five realisations. [START_REF] Van Der Werf | Detailed measurements of velocities and suspended sand concentrations over full-scale ripples in regular oscillatory flow[END_REF] because integration is performed down to the ripple surface rather than two grid cells above it.

Figure 8

 8 Figure 8Suspended sediment concentration contours [log 10 (c)] predicted by the DVPT model (left panels), measured in the experiment (middle panels), and predicted by the k-ω model (right panels) for the phases shown (degrees). The arrows are proportional to the strength of the free-stream velocity.

Figure 9

 9 Figure 9Cycle-and ripple-averaged suspended sediment concentrations predicted by the DVPT (solid line) and k-ω (dashed line) models, and measured in the experiment using ABS (open circles) and TSS (closed circles).

Figure 10

 10 Figure 10Current-related, wave-related and net cycle-averaged, ripple-averaged suspended sediment fluxes predicted by the DVPT model (solid lines) and the k-ω model (dashed lines) and measured in the experiment (open circles). The horizontal (solid) line at y/λ = 0.19 denotes the ripple crest level.

  Figure01

Table 1 :

 1 Measured and predicted current-related <q sc >, wave-related <q sw > and net cycle-averaged <q s > suspended transport rates in mm 2 /s.

		<q sc >	<q sw >	<q s >
	DVPT model	-1.6	-2.3	-3.9
	k-ω model	-6.6	-2.2	-8.8
	Data*	-7.3	-4.6	-11.9
	*These values differ from those quoted in

ACKNOWLEDGEMENTS

This work has been carried out within the SANDPIT project ('Sand Transport and Morphology of Offshore Sand Mining Pits') funded by the European Union (EVK3-2001-00056) and within the SANTOSS project ('Sand Transport in Oscillatory flows in the Sheet-flow regime') funded jointly by EPSRC in the UK (GR/T28089/01) and STW in the Netherlands (TCB.6586). Vanesa Magar was also partly funded by an EPSRC academic fellowship.

APPENDIX A: Modification of PIV data

This appendix explains how the PIV data was modified in order to correct for the settling velocity effect of the sediment, and then demonstrates the difference the modification makes.

The modifications made to the PIV data were as follows.

1. Periodicity was forced at the two ripple troughs:

APPENDIX B: Calculation of vorticity and its consistency

This appendix explains how the vorticity is calculated from the modified PIV data and then examines its consistency.

The vorticity,  , can be readily calculated from the corrected velocity field:

.

However, in order to produce the vorticity contours, it is necessary to use a running-mean or convolution process to reduce the noise in the signal and smooth Ω.

The consistency of the vorticity data is examined here by considering two phases in the wave cycle and showing the effect of increasing the number of cycles over which the data is phase-ensembled.


Alexi Turcotte
email: turcotte.al@northeastern.edu

Pierre Donat-Bouillud

Filip Křikava
email: filip.krikava@fit.cvut.cz

Jan Vitek
email: j.vitek@northeastern.edu

signatr: A Data-Driven Fuzzing Tool for R

Keywords: CCS Concepts:, Software and its engineering → Software notations and tools R, fuzzing, dynamic program analysis, dynamic programming languages

The fast-and-loose, permissive semantics of dynamic programming languages limit the power of static analyses. For that reason, soundness is often traded for precision through dynamic program analysis. Dynamic analysis is only as good as the available runnable code, and relying solely on test suites is fraught as they do not cover the full gamut of possible behaviors. Fuzzing is an approach for automatically exercising code, and could be used to obtain more runnable code. However, the shape of user-defined data in dynamic languages is difficult to intuit, limiting a fuzzer's reach.

We propose a feedback-driven blackbox fuzzing approach which draws inputs from a database of values recorded from existing code. We implement this approach in a tool called signatr for the R language. We present the insights of its design and implementation, and assess signatr's ability to uncover new behaviors by fuzzing 4,829 R functions from 100 R packages, revealing 1,195,184 new signatures.

Introduction

Dynamic analysis is often the only practical way to analyse code written in dynamic languages as the semantics of these languages severely limits static analyses. Dynamic analysis, however, requires both code to run and valid inputs for said code. To draw conclusions about a code base, one could run the existing, runnable code, e.g., tests, but such code paints an incomplete picture as it is challenging to fully cover the range of behaviors allowed by dynamic languages.

To increase coverage, one could make use of a fuzzer, a tool that exercises code by generating random inputs. Fuzz testing has seen widespread adoption, primarily to find bugs, performance pathologies, and security vulnerabilities. However, dynamically typed languages such as Python, JavaScript, and R pose unique challenges that revolve around the idea that dynamic code tends to hide runtime errors. For instance, accessing a non-existent field of an object in JavaScript yields the value undefined instead of crashing, and basic functions in R will readily coerce values whose types do not match. A tool that tries to run code automatically will thus have very little to go on vis-à-vis the correctness of the code being generated as there is no clear observable witness of an error. On top of this, the lack of static types leaves fuzzers with very little information about what values are expected to begin with. Finally, it is difficult to generate complex values automatically in dynamically typed languages; in a statically typed language like Java, the shapes of user-defined objects can be inferred from a static class definition. In contrast, there is no such guide in dynamic languages, limiting a fuzzer's ability to generate realistic inputs.

To get around this, we propose an approach to fuzzing that relies on an extensive database of observed values. We develop a tracer that collects information about function calls and values created during code execution, and store this information in a database with an expressive query API. Then, we leverage this database to generate new function calls using the recorded values. This approach is implemented in a tool called signatr for the R programming language.

To validate our tool, we revisit an application of dynamic analysis, namely trace typing [START_REF] Andreasen | Trace typing: An approach for evaluating retrofitted type systems[END_REF] where the goal is to guess the signatures of library functions by observing the values that they accept and return. We perfom that experiment on R libraries and use the contractr type inference tool [START_REF] Turcotte | Designing types for R, empirically[END_REF], wherein function types were inferred from recorded calls in R package test, example, and vignette code. Fuzzing 4,829 of those R functions with our tool generated 1.2M new unique signatures compared to the original study.

Background and Related Work

Related Work. One of the earliest fuzzers is Randoop, a feedback-driven random test generation tool for Java [START_REF] Pacheco | Randoop: feedback-directed random testing for Java[END_REF], where sequences of method calls are generated to test classes, and arguments are randomly generated for these calls. For primitive values, a random value is selected from a predefined, but user-extensible list; for reference types, a value is selected at random from those which have been seen, and if none are available then null is selected. While this technique is effective at generating tests involving non-trivial objects that are built up from a number of method calls, data science languages like R often require generating realistic data. QuickCheck is another interesting tool in this space [START_REF] Claessen | QuickCheck: A Lightweight Tool for Random Testing of Haskell Programs[END_REF]. While it can generate random inputs, the strategy for constructing inputs needs to be specified. The lack of type information in R limits the usefulness of such approaches. American Fuzzy Lop (AFL) is a state-of-the-art industrial fuzzer [START_REF] Afl | American fuzzy lop[END_REF]. AFL takes a program and one example file as input, calls the program with the input, and then uses a variety of heuristics to transform the input and fuzz the program. AFL aims to find defects, while our approach aims to find novel inputs that successfully execute.

The R Language. We present a fuzzing tool called signatr for R, a language which sports an unusual mix of language features making it a challenging target for tooling [START_REF] Morandat | Evaluating the Design of the R Language[END_REF]:

-The lack of a static type system, so there is There has been work on a static type system for R [START_REF] Turcotte | Designing types for R, empirically[END_REF]. A simple type annotation language has been evaluated on a corpus of 400 R packages. In Section 4, we will revisit that

Approach

There are two main phases to our approach: (1) recording R code as it runs (Fig. 1); this involves capturing function arguments and return values in our value database, and (2) fuzzing by drawing inputs to functions from the database (Fig. 2).

For the recording phase, runnable code is extracted from examples and tests in packages. This runnable code consists of a set of scripts that can be run independently. Next, the scripts are executed in parallel, and all arguments are recorded into separate instances of our database, called the sxpdb. Finally, all the database instances are merged into one, in the process duplicate values are discarded. GNU parallel is used for orchestration [START_REF] Tange | Gnu parallel-the command-line power tool[END_REF]. Every R package hosted on R's main package repository (called CRAN) is required to have runnable example scripts to show how the package should be used, so there is a lot of code available.

As for the fuzzing phase, a list of functions to fuzz as well as a sxpdb are taken as input. The functions are fuzzed in parallel. The output of the fuzzer is a list of successful function calls, where a successful call is defined as one that generated no warnings, errors, and did not cause R to crash. The fuzzer relies on hooks before and after function execution: the hook before invocation allows the fuzzer to process inputs, and the hook after allows errors to be signaled and handled and the return value to be processed. Concretely, the fuzzer runs the functions using an extended R virtual machine that supports attaching callbacks to various runtime events [START_REF] Goel | On the Design, Implementation, and Use of Laziness in R[END_REF]. Users can also plug a dynamic analysis directly into the pipeline.

Since R can crash (and does), to avoid losing results each instance of the fuzzer consists of two processes: a worker that fuzzes, and a supervisor that can spawn workers if needed. Furthermore, the fuzzer runs isolated in a container as calling arbitrary code could have dire consequences.

The tool itself consists of three standalone components: argtracer, the tracer responsible for running code and recording function invocation, sxpdb, the value database, and finally generatr, the fuzzer responsible for generating inputs. They are R packages written in a combination of R and C++.

Tracer

The tracer is built on top of the aforementioned extended virtual machine. The two runtime events we use are function exit, where all arguments are captured and stored, and a context jump, which is necessary to keep the call stack balanced as the interpreter uses long jumps for loop control flow, return statements, and error handling. (On a long jump, the exit hook is ignored, so we maintain our own version of the call stack to capture all function exits.) When the tracer sees a call, it only sees a pointer to a closure, and the function's name and its package can only be found by searching through the loaded namespaces and the symbols they contain. Thus, the tracer eagerly builds an index of package and function names when namespace functions are loaded.

Regarding performance, there is 1.2-7.8 (3.3 on average) slowdown when running with tracing enabled (based on the running / tracing 3,236 R scripts recording 8M unique values of 3GB size). This cost is mostly due to the value serialization. The tracer is written in 600 lines of C++ code.

Database of Values

The database is hand-written to leverage domain-knowledge of R values and optimize it for the queries supported by our API. It is implemented in 5K lines of C++ and 1.5K lines of R.

Storage. The database stores unique values. We use XXH-128 hashes for uniqueness in combination with a hash table based on RobinHood hashing 1 . While the hashing is fast, we need to lower R values into a binary format. R provides a binary serialization XDR, 2 but it is costly. We also strip the serialization of sources of non-determinism related to character encodings. Since many values are pushed to the database in an average recording session, we try to avoid serialization as much as possible. For this, we use the trace bit that is part of each value. 3 If not set, the value is fresh 1 cyan4973.github.io/xxHash, github.com/martinus/robin-hood-hashing 2 cran.r-project.org/doc/manuals/r-release/R-ints.html#Serialization-Formats 3 A bit in the C struct that represents a value, cran.r-project.org/doc/ manuals/r-release/R-ints.html#Rest-of-header. and we serialize it, compute its hash, store it. The trace bit is then set to avoid repeated serialization. In spite of R's copyon-write semantics, values can be modified in place before being shared, and these values may be serialized repeatedly to capture the updates performed by the program.

The database also stores metadata about values and their origin. We also keep a unique id for each sequence of arguments coming from the same call site. This allows us to replay the calls as they were observed.

The database maintains tables for the hashes, runtime metadata (e.g., how often a value was seen), static metadata, origins, call ids, and class names. Variable length data are stored in a combination of 2 tables, one giving an offset into the other table which holds the size of the value and the value itself. Origin strings and class names are interned, i.e., each unique string is stored separately, and referred to by pointer. Search indices are built using fast compressed bitsets [START_REF] Chambi | Better bitmap performance with roaring bitmaps[END_REF]. The database supports all values except external pointers (e.g. pointer to C allocated data). Environments and closures were not stored in the database during our experiments since they dramatically increased memory pressure.

Finally, opening the database in read mode only loads metadata. Retrieving values from disk is done on demand, making it possible to query larger-than-memory databases.

Queries. Values can be queried based on their typeoftype or class, on the presence of NAs, number of attributes, and dimensions. The database can be queried for a random value with the desired metadata, or can be queried by providing an existing value along with a list of search parameters to be relaxed.

Fuzzing

The value database is at the core of our fuzzing approach, which is similar in spirit to mutation-based fuzzing, where valid inputs are taken and mutated to try to exercise new functionality (rather than have inputs be randomly generated). Instead of mutating arguments to previous calls, new argument values are selected based on previous ones.

The fuzzer generates calls to a function and chooses arguments to these calls as depicted in Algorithm 1. In addition to the target function and database, the algorithm considers how many query parameters to relax on (numRelax), as well as all of the previously seen successful calls to the function (succs). For each parameter, the algorithm determines how to relax (this may change from one iteration to the next), finds all values that inhabited that parameter in successful calls, chooses one value, and queries the database for a value similar to it save for the relaxation. If no successful calls to the function have been observed, random values can be chosen.

The fuzzing approach itself is depicted in Algorithm 2. First, the collection of already known calls to the function is obtained from the database. The main idea of the approach is to start by selecting new arguments essentially at random by querying the database and relaxing on many parameters, and gradually reduce the number of parameters being relaxed as the fuzzer progresses. Concretely, the number of parameters being relaxed is reduced every tick, which is determined by dividing the total fuzzing budget by the number of parameters that can be relaxed (numRelaxParams). The function will be fuzzed for as long as the budget allows, and initially all database parameters will be relaxed. Arguments for a new call are generated through the approach depicted in Algorithm 1 (getArgs), the call is performed, and the results are saved in res. If there were no errors, warnings, or crashes, then the successful call is added to the list of successful calls succs and iteration continues until the budget is exhausted. ⊲ get all values that 𝑝 had in successful calls to 𝑓 return 𝑠𝑢𝑐𝑐𝑠 ⊲ the successful calls to 𝑓 20: end procedure

Algorithm 1 Selecting Arguments for A Call

Intended Use

The tracing discussed in Section 3.1 is costly in that running large swaths of R code takes time. Thankfully, this only needs to be done once to construct a database, which can then be reused by the fuzzer. For simplicity, our artifact4 contains instructions to download a 10GB sample database.

Assessment

We used signatr to stress-test one of the proposed type system for R [START_REF] Turcotte | Designing types for R, empirically[END_REF]. The type system was designed empirically, with the help of a dynamic analysis that inferred function signatures from the types of values observed while running extracted code from packages. In a nutshell, a type was inferred from each call to a function, and these types were unified into a single function type, and the more unique successful calls there are, the more precise the inferred signature-the types inferred from successful calls are referred to as call signatures. It would be therefore interesting to see how many additional successful calls can signatr generate?

We ran all of our experiments on two Ubuntu 18.04 servers, each with a 72 core Intel Xeon 6140 2.30GHz processor and 256GB of RAM.

Recording. First, we created a sxpdb for the fuzzer. For this we used the extracted runnable code from the same corpus as the original study which consists of 17,463 R scripts containing 389.7K lines of code (excluding comments and new lines). The database was generated in 16 hours and occupies 287.17 GB of disk space. It contains 39.4M unique values recorded from 20.5M calls to 38.1K functions in 652 packages. Figure 3 shows the distribution of main value types. The vast majority of values are vectors and matrices of real numbers which is unsurprising as R is mostly used for numerical computing. That said, many of them (47.2%) contain attributes which is what makes them interesting, as attributes add semantic meaning. The next big group are lists, which can be divided into two groups, data frames (two dimensional, column-major structure representing observations) and records. Note that 0.5% are logical vectors of varying length, not simply true or false.

Fuzzing. Armed with this database, signatr fuzzed 4,829 functions from 100 packages, a subset of the original corpus, in 19 hours. The fuzzing budget was set to 5,000, with 64 functions being fuzzed in parallel. In total, 24.1M calls were made, and out of that, 13.9% were successful, resulting in 3,351,753 traces of 2,315 functions coming from 98 packages. The vast majority of errors were exceptions, but in 211 cases, the R process crashed. While the aim of this work is not bug finding, we have investigated one such crash in stringi 5 , and found that it was caused by memory corruption by large input. The issue was reported, acknowledged and fixed. Results. The results of fuzzing is shown in Figure 4. From the 4,829 functions in our 100 packages corpus, the fuzzer managed to generate call signatures for 2,189 functions (45.3%). In comparison, tracing, i.e., recording calls by running the extracted code covers 3,135 functions (64.9%). While the fuzzer covered fewer functions, it generated over 1.2M new unique call signatures (on average 562.7 per function), i.e., 105.1 times more call signatures in comparison to tracing. Out of the 2,189 functions, 607 functions were covered only by fuzzing. Fuzzed signatures overlapped with traced ones in only 1,082 cases in 597 functions.

The reason that signatr failed to generate a single successful call for 2,640 functions is because they require a very specific shape of one or more of its arguments, or the arguments depended on one-another. As an example of the latter case, certain functions from the dplyr data manipulation package required a data frame alongside an unevaluated expression made up of column names from the data frame. Besides that, problematic functions include those that use non-standard scoping, manipulating arguments as unevaluated expressions and evaluating them in custom environments. In some cases the error message could have been used as a feedback to the fuzzer (and we plan to address it in the next iteration), but this is not easy to generalize.

Next, we looked at code coverage to see if the new call signatures translated to more code being exercised. Using the covr package 6 , we computed line coverage of R source code for 1,342 functions using the fuzzed calls, and separately using the traced calls. This is not all of the functions that signatr managed to fuzz, as running covr on some functions caused runtime errors, and also repeating certain calls failed (both fuzzed and traced). For 294 functions, signatr improved code coverage on average by 20.4% (as compared with coverage obtained by simply running package tests, examples, and vignettes). 1.2M new calls yielded only 20.4% more coverage-while new code paths were explored, the extensive polymorphism and use of coercion in R mean that most of these new calls did not exercise new R code. 6 The only tool for R code coverage, cf. https://covr.r-lib.org Summary. signatr uncovers many calls with new type signatures, significantly expanding on what tracing alone can discover, particularly for polymorphic functions. Together, existing and generated calls yields a wealth of interesting runnable code that will be essential for the further design of a possible type system for the R programming language.

Conclusions and Future Work

Fuzzing is a useful technique not only for finding bugs and security vulnerabilities, but also for getting insights about code. However, it is hindered by the permissive semantics of dynamic languages as well as the dynamic nature of how complex data is defined. In this work, we proposed a fuzzing approach that relies on a database of observed values to provide complex and realistic inputs for functions. We implement this approach in a tool called signatr for the R programming language. We show that signatr uncovers many new call signatures for R functions and it can be a useful tool in a data-driven language evolution toolbox.

There are various avenues to improve the tool itself. For instance, the fuzzing component of our approach could be enhanced to consider the types of default function parameters. Further, we could expand the database with knowledge of how values co-occur, which would add more dimensions to how values can be selected. While this approach was implemented in a tool for R, it is broadly applicable in all languages, and the only real language-specific aspect is the set of parameters in the database. For instance, one could implement a similar tool for object-oriented languages, where database metadata could include object field names (e.g., JavaScript).

A signatr demonstration

The following is a short demonstration of the basic signatr functionality, i.e., how to create the value database by running R code and then how to use it for fuzzing. The tool is packaged as an R library. It can be used both in a script or interactively from an R REPL.

We begin by starting R (concretely, R-dyntrace version 4.0.2 which has the hooks used by the argtracer) and loading signatr. In the following listings, the $ indicates shell prompt and > denotes the R REPL. To generate a database of values, we need some code to run. One way to get it is to extract it from an existing R package, for example stringr: > extract_package_code("stringr", output_dir = "demo") ... 7 examples/str_detect.Rd.R examples ... This will extract all the runnable snippets from the package documentation and tests into the given directory. For example:

$ cat demo/examples/str_detect.Rd.R ... fruit <-c("apple", "banana", "pear", "pinapple") str_detect(fruit, "a") str_detect(fruit, "^a") ... Next, we trace the file by executing it and recording all the calls using the trace_file function:

> trace_file("demo/examples/str_detect.Rd.R", db_path = "demo.sxpdb") status time db_size error 0 0.024 20 NA

The resulting database is stored as demo.sxpdb. In this example, after running the str_detect.Rd.R file, the database contains 20 unique values. This can be repeated for all the other files. To trace multiple files in parallel, one database is created per file, which are merged using the merge_dbs function once they are all available. This also allows us to run larger experiments on multiple machines.

Once the database is ready, we can start fuzzing. The fuzzer has a number of configuration points, but the easiest starting point is the quick_fuzz helper function: > R <-quick_fuzz("stringr", "str_detect", "demo.sxpdb", budget = 100, action = "infer") started a new runner:PROCESS 'R', running, pid 4157 fuzzing stringr:::str_detect [======] 100/100 (100%) 39s stopped runner:PROCESS 'R', running, pid 4157

The infer action will use the infer_call_signature function that infers types for each call argument and return value using the type annotation language of Turcotte et al. [START_REF] Turcotte | Designing types for R, empirically[END_REF] (with the contractr type inference tool). infer_call_signature returns a data frame with details for each call. Here, the data frame includes the inferred signature in the result column. One advantage of using R is that we can use R's many data analysis functions. For example, we can look at the resulting signatures: This shows that in three cases, the fuzzer managed to generate a call that was successful, and the signatures of those calls. If you are repeating these steps, it is possible that your results will be different since fuzzing is non-deterministic.

> count(R,

Figure 2 .

 2 Figure 2. Fuzzing pipeline.

1 :

 1 procedure getArgs(𝑓 , 𝑛𝑢𝑚𝑅𝑒𝑙𝑎𝑥, 𝑑𝑏, 𝑠𝑢𝑐𝑐𝑠)2: 𝑝𝑎𝑟𝑎𝑚𝑠 ← 𝑔𝑒𝑡𝑃𝑎𝑟𝑎𝑚𝑠 (𝑓 ) 3:for 𝑝 in 𝑝𝑎𝑟𝑎𝑚𝑠 do 4: ⊲ relax on 𝑛𝑢𝑚𝑅𝑒𝑙𝑎𝑥 params 5: 𝑟𝑒𝑙𝑎𝑥 ← 𝑝𝑖𝑐𝑘𝑆𝑜𝑚𝑒 (𝑟𝑒𝑙𝑎𝑥𝑃𝑎𝑟𝑎𝑚𝑠, 𝑛𝑢𝑚𝑅𝑒𝑙𝑎𝑥) 6:

⊲ 1 : 2 :

 12 sample a similar value from the database 11: 𝑎𝑟𝑔𝑠 [𝑝] ← 𝑠𝑎𝑚𝑝𝑙𝑒𝑆𝑖𝑚𝑖𝑙𝑎𝑟 (𝑣, 𝑑𝑏, 𝑟𝑒𝑙𝑎𝑥) procedure fuzzWithDB(𝑓 , 𝑑𝑏, 𝑏𝑢𝑑𝑔𝑒𝑡) 𝑠𝑢𝑐𝑐𝑠 ← 𝑔𝑒𝑡𝑆𝑢𝑐𝑐𝑒𝑠𝑠 𝑓 𝑢𝑙𝐶𝑎𝑙𝑙𝑠 (𝑓 , 𝑑𝑏) 𝑎𝑟𝑔𝑠 ← 𝑔𝑒𝑡𝐴𝑟𝑔𝑠 (𝑓 , 𝑟𝑒𝑙𝑎𝑥𝑇ℎ𝑖𝑠𝑇𝑖𝑚𝑒, 𝑑𝑏, 𝑠𝑢𝑐𝑐𝑠) 12: 𝑟𝑒𝑠 ← 𝑐𝑎𝑙𝑙 (𝑓 , 𝑎𝑟𝑔𝑠) 13: ⊲ add successful call to 𝑠𝑢𝑐𝑐𝑠 14:if no warnings, errors, crashes in 𝑟𝑒𝑠 then

Figure 3 .

 3 Figure 3. The sxpdb value type distribution

Figure 4 .

 4 Figure 4. Number of unique call signatures.

  .0.2 (2020-06-22) --"Taking Off Again" ... > library(signatr)

  The above listing shows two calls: a failed one (non-zero status) with an error message, and a successful one with an inferred signature. The args_idx column contains the indices in the sxpdb of the values; thus, the actual argument values can be obtained by looking up the args_idx in the sxpdb.

https://github.com/PRL-PRG/sle22-signatr-artifact

A string processing library, one of the most downloaded package in R.