Issam Raïs

Laurent Lefèvre

Anne-Cécile Orgerie

Anne Benoit

Exploiting the Table of Energy and Power Leverages

Large scale distributed systems and supercomputers consume huge amounts of energy. To address this issue, a large set of hardware and software capabilities and techniques (leverages) exist to modify power and energy consumption in large scale systems. Discovering, benchmarking and efficiently exploiting such leverages, remains a real challenge for most of the users. In this paper, we define leverages and the table of leverages, and we propose algorithms and predicates that ease the reading of the table of leverages and extract knowledge from it.

Introduction

Data centers worldwide consumed around 194 terawatt hours (TWh) of electricity in 2014, or about 1% of total demand [START_REF] Agency | Digitalization & Energy[END_REF]. This worrying consumption has direct financial and environmental consequences on data center managers, like Cloud providers and supercomputer operators. Several techniques have been developed in order to lower the electrical consumption of data centers. These techniques, that we call leverages, can improve the energy efficiency of data centers at different levels: hardware, middleware, and application. Hardware leverages include Dynamic Voltage and Frequency Scaling (DVFS) [START_REF] Suleiman | Dynamic voltage frequency scaling (DVFS) for microprocessors power and energy reduction[END_REF] and shutdown techniques [START_REF] Rais | Impact of Shutdown Techniques for Energy-Efficient Cloud Data Centers[END_REF]. At the middleware level, energy-efficient resource allocation policies for job managers are examples of leverages [START_REF] Georgiou | A scheduler-level incentive mechanism for energy efficiency in hpc[END_REF]. Finally, leverages at the application level include green programming [START_REF] Acar | Towards a Green and Sustainable Software[END_REF].

While many of these leverages have been independently studied in the literature, few works consider the utilization of several leverages at the same time, and no more than two leverages. Yet, the utilization of a given leverage can impact both the utilization and the efficiency of another leverage. The variety of leverages is added to the data center's complexity, in terms of size and hardware heterogeneity, and makes energy efficiency complex to reach for the users who have access to multiple leverages.

In this work, we aim at extending the current state of the art, which is studying the influence of one or two leverages at maximum at the same time, thus ignoring the impacts incurred by the utilization of more leverages. Thus, we proposed a generic definition, combination and knowledge extraction of multiple leverages in order to fully explore their combined impacts.

We propose a first approach toward a completely automated process to characterize the leverages available on a data center node. The key idea of our contribution consists in providing hints to users about the most suitable solution for their application from a defined score table with a value for each leverage combination and each studied metric. Through these tables could be derived knowledge about leverage combination and effects they incur on each other. From the definition of a table of leverages, a tool to help a user, a developer or an administrator to choose which leverage or leverage combination suits the best his objectives (here with a focus on energy or power metrics), the contribution of this paper consists in the algorithms proposed to extract knowledge about the interaction of leverages and their influence on a given metric.

The remaining of this paper is structured as follows. Section 2 formalizes the concept of leverages, and illustrates this formalism on the leverages under consideration in this paper. Section 3 defines and explains how to build the table of leverages. Section 4 presents the experimental setup and a first full example of table of leverages. Section 5 then shows how to exploit the raw data of the table of leverages and extract useful knowledge. Finally, Section 6 concludes this work and gives perspectives. 4

Leverage definition

In this section, we first propose a formalization of a leverage. Second, we apply this formalism to the leverages that we selected for this paper.

Definition 1 A leverage L is composed of S = {s 0 , s 1 , . . . , s n }, the set of available valid states of L, and s c , the current state of L.

Thus, an energy or power leverage is a leverage that has a high impact on the energy or power consumption of a device through its various states or through the modification of its current state. Switching from one state to another can have a cost in terms of time and energy. Yet, in the current work, we focus on studying the impacts of leverage combinations over a single intensive application phase [START_REF] Chetsa | A user friendly phase detection methodology for hpc systems' analysis[END_REF], and thus we do not study the switching costs between states.

In this paper, we consider multiple leverages available on current hardware, namely multi-thread, computation precision and vectorization. These leverages belong to different categories of leverages: application level with computation precision and vectorization techniques, and middleware level with multi-threading. These leverages are described hereafter.

Multi-thread leverage. The first studied leverage is a middleware-level leverage that permits the usage of multiple cores during computation. OpenMP [START_REF] Dagum | OpenMP: an industry standard API for shared-memory programming[END_REF], a well-known application programming interface abstraction for multi-threading, can be used to exploit this intra-node parallelism of multi-cores. It consists of a set of directives that modifies the behavior of the executed code, where a master thread forks a specific number of slave threads that run concurrently. This multi-thread leverage increases the CPU utilization of the node. Consequently, because of the non-power proportionality of current hardware architectures [START_REF] Rais | Impact of Shutdown Techniques for Energy-Efficient Cloud Data Centers[END_REF], this leverage can improve the energy efficiency of the node. In the rest of the paper, the multi-thread leverage is denoted by nbThreads with the set of states {1, . . . , n max }, where 1 means that one OpenMP thread is used, and n max corresponds to the maximum number of threads that could be launched simultaneously on the node. In this work, only the extreme states, 1 and n max , are explored.

Computation precision leverage. The second leverage belongs to the application level and exploits the various computation precision options available on actual hardware (i.e., int, float, double). Such a leverage alters the precision of the results computed by the application, but lower precision translates into shorter data representation and so, less computation and less energy consumption. At the application level, the user can specify a desired Quality-of-Service that can be expressed as accessible computation precision states.

This precision leverage is denoted by P recision, and the set of states is {int, float, double}, corresponding to the data format for the application. For each of these states, a different code version is provided.

Vectorization leverage. Finally, the last studied leverage concerns the application level. Current CPUs allow the usage of vectorization capabilities to exploit intra-core parallelism. On Intel architectures, it started with MMX instruction in Pentium P5 architectures in 1997 [START_REF] Peleg | MMX technology extension to the Intel architecture[END_REF]. It was then extended to SSE [START_REF] Gallas | Embedded Pentium (R) processor system design for Windows CE[END_REF]. SSE was then extended to SSE2, SSE3, SSSE3 and finally SSE4. AVX [START_REF] Lomont | Introduction to intel advanced vector extensions[END_REF] then introduces new instructions, followed by AVX2 and finally AVX512 available in XeonPhi architecture. In this paper, we focus on SSE3 and AVX2, which are representative of the SSE and AVX families. These instruction sets permit single instruction on multiple data (SIMD) at application level.

This vectorization leverage is denoted by V ectorization. The set of states is {none, SSE3, AVX2}, where none means that no vectorization is used. For each of these states, a different code version is provided using the specific intrinsics and adequate compilation flags for each version.

The proposed leverage formalism described above is used in the rest of the paper to easily describe the state of each considered leverage and the possible combinations of leverages. The three leverages studied here are chosen to be representative examples of available leverages on modern architectures and frequently used during HPC applications. The methodology proposed in this paper is designed to be applied to any number and any type of leverages.

The table of leverages

We describe the table of leverages, which relies on metrics and benchmarks to characterize the performance and energy impact of each leverage combination on a given node. For each metric and each benchmark, a score is attributed to a given leverage combination. The table is then used to extract knowledge about each leverage and evaluate impacts of leverage combinations in order to help the users to utilize their computing infrastructure in a more energy-efficient way.

Metrics. Leverages may influence the quality of service or performance of an application. For instance, shutdown techniques may induce latency in waking up the required nodes. Consequently, for these leverages, users need to determine their acceptable trade-off between energy-related metrics and performance metrics. The table of leverages relies on three different metrics that represent both energy and performance constraints. These metrics are measured for a given period of time corresponding to the time spent during benchmark execution.

The two first metrics are energy and power related metrics. To define them, we introduce the following notations: T = {t 0 , . . . , t N } is the set of time stamps of energy consumption measurements of a given run; t 0 and t N represent the starting and ending timestamps (with a distance of one second), respectively; p j , j ∈ [0, N ], represents the power consumption (in Watt), of the considered node for the timestamp t j . Metric 1: The average power consumption of an executable is denoted avrgW att, and it is defined as avrgW att = j∈[0,N ] p j /(N + 1). Metric 2: The energy consumption of an executable is denoted Joules. It represents the energy consumption of the complete node used between t 0 and t N . It is defined as Joules = j∈[0,N -1] (t j+1 -t j ) × p j . Metric 3: The last metric concerns the performance of the run, and is expressed as the execution time, denoted T ime. It includes whole execution time of an executable, including initialization.

Benchmarks. A benchmark corresponds to a self-contained application that is representative of typical applications or portions of applications. The benchmark is compiled before the run, and once launched, the metrics previously defined are collected during its execution.

Here, for the sake of clarity, we evaluate only one benchmark for a set of embedded leverages. We chose to focus on a well-known CPU intensive code: the line per line matrix multiplication (LpL MM) of dense random large squared matrices (8192 as dimension size). The same algorithm is implemented for the various leverage combinations. The considered leverages are multi-thread, computation precision and vectorization. For the last two leverages, a different state means a different version of code, here generated by hand using dedicated intrinsics and compilation flags (-O3 -msse3 -mavx2). We deactivated the auto vectorization of the compiler (-fno-tree-vectorize) to have a control over the chosen intrinsics and because auto generation of vectorizable code is not one of the focused leverage in this paper.

Formalization of the table of leverages.

Here, we describe how to compute the score associated to each metric for each leverage. Let X, Y, Z be the sets of available states of three leverages χ, ψ, ω (corresponding to S, the set of states for a given leverage L, from definition 1): X = {x 0 , . . . , x nx }, Y = {y 0 , . . . , y ny }, and Z = {z 0 , . . . , z nz }. Let g 1 , . . . , g m be the measured metric functions, as for instance avrgW att, Joules, and T ime. For all u (1 ≤ u ≤ m), g u (x i , y j , z k ) is the value of metric g u for the states x i , y j , z k for the leverages χ, ψ, ω.

In the table of leverages, each line corresponds to a combination of states for each leverage and the columns correspond to the measured metrics. We normalize each value on the minimum value for each metric. These normalized values constitute the scores indicated in the table of leverages. Let h 1 , . . . , h m be the normalized versions of g 1 , . . . , g m . So, we have, for 1

≤ u ≤ m, h u (x i , y j , z k ) = gu(xi,yj ,z k ) min x i
∈X,y j ∈Y,z k ∈Z gu(x i ,y j ,z k ) , with h u (x i , y j , z k ) being the value in the table of leverages in column of metric u and corresponding to the line for the states x i , y j , z k respectively for the leverages χ, ψ, ω.

For application-level leverages, here Precision and Vectorization, the chosen benchmarks correspond to a different combination of application leverage states. Leverage nbThreads changes its state through environment variable. When all states are covered, the table of leverages is complete for the considered benchmark. Reducing the creation time of such a table is not the focus of this paper.

Building and analyzing the table of leverages

In this section, we present the table of leverages built on a node from our experimental testbed, Grid'5000 [START_REF] Balouek | Adding virtualization capabilities to the Grid'5000 testbed[END_REF]. Grid'5000 deploys clusters linked with dedicated high performance networks in several cities in France. As our focus is on energy and performance related metrics, we used the Lyon site, where the energy consumption of every computing node is monitored through a dedicated wattmeter, exposing one power measurement per second with a 0.125 Watts accuracy. The Nova cluster from Lyon is used in the following. This cluster contains Dell Pow-erEdge R430 with 2 CPU E5-2620 v4 of 8 cores each, 32 GB of memory, 2 HDD disks of 300 GB each.

We applied our previous methodology for the three chosen leverages to the CPU intensive benchmark. This allows us to explore all possible states of chosen leverages, and thus to build a complete table of leverages. The table has the following format: the first three columns present the states of the nbT hreads, P recision, and V ectorization leverages respectively, while the last three columns show the normalized results of the three metrics avrgW att, Joules, and T ime, respectively, for every combination of leverage. As can be seen in Table 1 (first six columns), a line represents results of all gathered metrics for the execution of a representative load for a chosen combination of leverages. The results are normalized as explained before. The table of leverages gathers the knowledge of a Nova node, for a given workload done for multiple states of leverages combined.

Explanation of the table:

A lot of unexpected results, at first sight, are detected in Table of leverage 1, like the combination with int being better than float and double when 1 and none are the chosen state for the nbThread and Vectorization leverages, with this trend being reversed with nbThreads=32.

From the set of combination with 1 as the chosen state for leverage nbThreads, it is logic to see that int is quicker than float then double from a cache usage perspective. Indeed, more data can be brought into the cache to compute without the need to fetch new data compared to float or double representation that need more space for the same amount of elements. As for the SSE and AVX combinations, we have tremendous gain while using it compared to None, as it uses vectorial capabilities of the used core. Using a leverage usually comes with a cost. This statement is also true for the Vectorization leverage. An operation on vectors has costs, even if it is low. For instance, it is known that loading and saving vectors has a non null cost. With only one active thread, the current architecture, Broadwell here, allows turbo boost, a technology that permits to reach a much higher frequency that the available ones (here it can reach 3.0 GHz, when average frequency is 2.1 GHz). Also, when the OS detects too much load on a core, it context switches the running process and runs it on another core. Hence, the kernel saves the states (stack, registers) of the current process and loads it on another core, implying a storing and loading cost of the given process. This phenomenon can happened several times during a second. Thus, saving and charging states can create a lot of cache misses, which could be dramatical with usage of vectorization, where loading and saving vectors is not free.

As AVX has longer vectors, its operation costs on vectors can be longer than SSE. Thus, it starts to be beneficial only when comparing double combinations for such a Vectorization leverage.

When threads are up to 32, data is more likely to be shared between caches of various used cores. Without the previous struggles from caches for one core and because it is also well known that floating points operations(float and doubles here) are well optimized on current architectures and perform better than integers, {32, float, none} and {32, double, none} perform better than {32, int, none}. All threads are sharing data on separated cache, SSE and AVX outperforms the none configuration, with AVX always outperforming SSE for a fixed combination. Due to this data repartition between caches implied by the chosen configuration of the nbThreads leverage, there is enough computation to overcome costs of larger vector operations, here AVX for all combinations.

Note that the best combination for all metrics used here is always the {32, int, AVX2} combination. This result is the best combination to choose only if we have no constraints about leverage choices. It is expected to see variation, as leverages highly modulate the usage of nodes, either from intensity of usage for example of caches, core usage, availability of specific leverages (like seen with turbo boost with one thread). Results of metrics from combination of leverages is thus complicated to fully understand without a detailed knowledge of the architecture, the underlying used leverages and their influences on a given context. We propose predicates that helps a user underline such interesting points of interest from the table of leverages. For example, this table could help a user to choose a combination taking into account a fixed leverage state. Or to answer the following question: is there a leverage or a state of leverage that is always better for a given metric?

Exploiting the table of leverages

In this section, we describe the main contribution of this paper: a methodology to exploit the table of leverages and to extract useful knowledge, such as the influence and impact of one or multiple leverages on a given metric or set of metrics. We propose two focuses for extracting a score for each leverage. The first one corresponds to the actual table: it normalizes the results of a given metric for every explored configuration. The second one computes a ratio of contribution for each leverage in order to expose the most relevant leverage (the one with the largest contribution to the considered metric). We define four exploitation predicates that ease the analysis of the table, and answer questions. We illustrate these predicates and the answers of these questions on the selected table (Table 1). These questions target a single metric, h u .

Question 1: Is a selected combination of leverages states the best one for metric h u ? If a given combination is always the best, it means it should always be applied, if possible, if one wants to optimize h u . Consider a combination of states x a , y b , z c of leverages χ, ψ, ω for metric h u . We need to check whether for all i ∈ [0, . . . , n x ]\{a}, j ∈ [0, . . . , n y ]\{b}, and k ∈ [0, . . . , n z ]\{c}, we have h u (x a , y b , z c ) ≤ h u (x i , y j , z k ). On Nova nodes and for the three leverages (Table 1), the best combination for all three studied metrics is {32, int, AVX2}.

Question 2: When I fix a state, do I always improve metric h u ? Consider state x a of leverage χ. We want to check whether for all i ∈ [0, . . . , n x ]\{a}, for all l, j ∈ [0, . . . , n y ], and for all m, k ∈ [0, . . . , n z ], we have h u (x a , y l , z m ) ≤ h u (x i , y j , z k ). On the example of Table 1, for the Joules and T ime metric, only the n max (here, 32) state of nbThreads leverage answers this predicate, meaning that using this state will always be beneficial. No specific results can be obtained with this question for the avrgW att metric, meaning that no leverage state is always better for this metric when used.

Question 3: If some states are fixed for a subset of leverages, is a given state for the remaining leverages the best choice to optimize h u ? Consider that the state of leverages ψ, ω is fixed to y b , z c . We are asking whether state x a of leverage χ is the best choice for metric h u . Therefore, we need to check whether for all i ∈ [0, . . . , n x ]\{a}, we have h u (x a , y b , z c ) ≤ h u (x i , y b , z c ), which tells for instance that for the fixed combination {32, SSE3}, the best state for the Precision leverage is float, when considering the Joules or T ime metric (Table 1). Although, when focusing on avrgW att as the studied metric, for the {32, SSE3} fixed combination, the best state for the Precision metric is int.

If only state z c for leverage ω is fixed, and we consider states x a and y b of leverages χ and ψ respectively, we check whether for all i ∈ [0, . . . , n x ] and for all j ∈ [0, . . . , n y ], we have h u (x a , y b , z c ) ≤ h u (x i , y j , z c ). Concerning the Joules metric (Table 1) for the fixed state float of the Precision leverage, the best combination for the nbThreads and Vectorization leverages is {32, AVX2}. However, for the avrgW att metric, fixing again the state float of the Precision leverage, the best combination is now {32, SSE3}.

Applying this predicate allows us to extract some unexpected results. Concerning the Joules and T ime metrics, for the Precision and Vectorization leverages, no state emerges as the best one. In fact, it highly depends on the chosen state of other leverages. One could for instance expect int to always be the best state, but when comparing the {32, double, none} with {32, int, none}, we see that the double combination is more effective than the int combination. Similar conclusions can be drawn when the Vectorization leverage is used. AVX2 has larger vectors than SSE3, thus we would expect it to be always more efficient. However, when nbThreads state is equal to 1, {1, float, SSE3} is more effective than {1, float, AVX2}, leading to a different best choice when combined to the n max state (here, 32), where {32, float, AVX2} is more effective than {32, float, SSE3}. Note that this combination emerges as the best one when SSE3 is fixed.

Concerning the avrgW att metric, we also get unexpected knowledge. In opposition to the Joules and T ime metrics, no state emerges as the best one for none of the studied leverages. As AVX2 has larger vectors than SSE3, we would expect it to always stress more the CPU, thus always having higher values for this metric. It is the case with the {32, float} and {32, double} combinations. However, it is not observed with other combinations. When nbThreads=1, int is always the best choice to minimize this metric, whatever the chosen state for Precision and Vectorization leverages. Moreover, when Vectorization and nbThreads are set to any studied states, int is also always the best choice to minimize the avrgW att metric.

Question 4: Given a combination for all the leverages, how can we rank the states in terms of contribution for metric h u ? To answer this question, we consider a set of states x a , y b , z c of leverages χ, ψ, ω. Then, for each state w ∈ {x a , y b , z c }, we compute the contribution score mc(w) for this state on metric h u as follows. For state x a of leverage χ, mc(x a ) = hu(xa,y b ,zc) max i∈[0,...,nx ] hu(xi,y b ,zc) . We define similarly the contribution of states for the other leverages ψ and ω. Then, we rank the contribution scores mc(x a ), mc(y b ), mc(z c ) in ascending order to answer the question.

Table 1 (last three columns) presents the scoring related to the table of leverages. For the best combination {32, int, AVX2}, the ranking goes as follows for the Joules metric: "T,V,P" or "nbThreads, Vectorization, Precision", meaning that the chosen state for T here is the most contributing state in this combination, followed by the V, and then P states. Thus, for this combination, the precision leverage with the int position has the lowest contribution.

This ranking points out unexpected results for the Joules metric. We notice a switch between two positions of a given leverage for the fixed combination of other leverage states: {32, double}. In fact, when comparing the scoring of {32, double, SSE3} with {32, double, AVX2}, we get respectively "T,P,V" and "T,V,P". In the first case, double and SSE3 have the same worst possible score, 1.0, meaning that it is the worst state of this leverage for this combination. In the second case, AVX2 scores better than SSE3 and thus, it is above double. When nbThreads=1, we note that combinations including SSE3 and AVX2 states always have the Vectorization leverage state as the most contributing one, which leads to the conclusion that it is always better to use SSE3 and AVX2 states for the Vectorization leverage. For the {32, float, SSE3} combination, we get the scoring "T,P,V". float gets a better score and thus a better position than SSE3 because it is the best leverage state for the {32, SSE3} combination, leading to the conclusion that choosing float instead of other Precision leverage states contributes more than choosing SSE3 instead of other Vectorization leverage states for this combination. For the avrgW att metric, scoring underlines the fact that when choosing int as a state of Precision leverage, and for a fixed state of the Vectorization leverage, the sorting is always the same. In fact, {32, int, none}, {32, int, SSE3} and {32, int, AVX2} get the exact same sorting of contribution that {1, int, none}, {1, int, SSE3} and {1, int, AVX2}, respectively. Moreover, int is always the most contributing leverage state, which shows that int is always a good choice to improve this metric. This scoring also underlines the fact that in order to minimize the avrgW att metric, a user should better focus on P and V leverages, asT is never the most contributing one. This scoring highlights results that would have been difficult to notice just by looking at the table. It allows a user to quantify how much a leverage position used in a combination contributes to the overall performance for a given metric.

Conclusion

Energy efficiency is a growing concern. In the context of HPC and datacenters where the size of infrastructures grows drastically, energy consumption has to be taken into account as a high expense. There is a wide range of techniques, that we formally define as leverages, that permits to modulate the computing capabilities and/or the energy/power used by a device. We propose a generic solution to extract fine grain knowledge and hints from the table of leverages, thanks to the defined predicates. Our solution underlines new knowledge about leverages alone and about combinations of leverages. Thus, it allows us to extract influences of leverages on each other and understandable knowledge by the user.

Knowledge could be extracted from a table on CPU-intensive workload. For example, our solution underlines the fact that if Precision is set to the double state, it is always better to use it with AVX2 state for the Vectorization leverage to minimize the Joules metric. Also, for Vectorization fixed to the SSE3 state, our solution tells us that float is the best state to minimize the Joules metric. We also underline the fact that some unexpected behavior can be seen when combining leverages. For example, we underline the fact that changing float or int to double for Precision, and keeping the SSE3 state activated for Vectorization state, turns out to be counterproductive for the Joules metric.

The first short term future work is the parallelization of the creation of the table of leverages in order to improve the time needed to build it. Then, we plan to apply this methodology on other non CPU-intensive phases, such as IO, HDD, and RAM-intensive phases with appropriate leverages for every phase. Finally, a future working direction would be to extend this methodology to costly transition leverage states, as for instance shutdown policies. Also, we would like to investigate how to reduce the completion time for building such a table. In fact, the time to solution here could be greatly reduced, for example by predicting which run is not needed to know values of relevant metrics using learning or prediction techniques.

Table 1 :

 1 Normalized table of leverage states and ranked impact for line per line matrix multiplication (LpL MM) benchmark on a Nova node.

		Leverage states	Table of leverages	Ranked impact
	nbThreads (T) Prec. (P) Vector. (V) avrgWatt Joules Time avrgWatt Joules Time
	1	int	none	1.05	65.09 61.89	P,T,V P,T,V P,T,V
	1	int	SSE3	1.06	28.26 26.56	P,V,T V,P,T V,P,T
	1	int	AVX2	1.06	29.32 27.67	P,V,T V,P,T V,P,T
	1	float	none	1.05		69.67	P,V,T P,T,V P,T,V
	1	float	SSE3	1.06	33.8 31.89	V,P,T V,P,T V,P,T
	1	float	AVX2	1.05	36.8 34.89	P,V,T V,P,T V,P,T
	1	double none	1.06	81.59 76.89	P,T,V P,T,V P,T,V
	1	double SSE3	1.07	58.52 54.89	V,P,T V,P,T V,P,T
	1	double AVX2	1.06	57.72 54.22	P,V,T V,P,T V,P,T
	32	int	none	1.43	13.48 9.44	P,T,V T,P,V T,P,V
	32	int	SSE3	1.4	4.68 3.33	P,V,T T,V,P T,V,P
	32	int	AVX2	1.0	1.0	1.0	P,V,T T,V,P T,V,P
	32	float	none	1.45	7.4 5.11	P,T,V T,P,V T,P,V
	32	float	SSE3	1.41	3.76 2.67	V,P,T T,P,V T,P,V
	32	float	AVX2	1.56	3.11 2.0	P,V,T T,V,P T,V,P
	32	double none	1.53	8.34 5.44	P,T,V T,P,V T,P,V
	32	double SSE3	1.53	8.52 5.56	V,T,P T,P,V T,P,V
	32	double AVX2	1.54	7.0 4.56	P,T,V T,V,P T,V,P

This work is supported by the ELCI project, a French FSN project that associates academic and industrial partners to design and provide software environment for very high performance computing. Experiments were carried out using the Grid'5000 testbed, supported by a scientific interest group hosted by Inria and including CNRS, RENATER and several Universities (https://www.grid5000.fr).


André J Arruda
email: ajarruda@gmail.com

Elise Buisson

Peter Poschlod

Fernando A O Silveira

How have we studied seed rain in grasslands and what do we need to improve for better restoration?

Keywords: meadow, prairie, rangeland, seed limitation, seed trap, steppe Conceptual Implications

Seed rain, the number of seeds reaching an area, is a process that plays a key role in recruitment and regeneration in plant communities. A better understanding of seed rain dynamics is therefore a critical step for restoration practices. A wide variety of methods to study seed rain in grasslands are available, but there is little agreement to which is the most appropriate one. Here we: 1) assessed where, how and why research on seed rain has been carried out; 2) examined how methodological design and results have been reported and 3) provide guidelines for future research on seed rain in grasslands. We built a database of 185 papers from a systematic literature survey between 1980 and November 2016 and we found a remarkable unbalance of the numbers of studies between grassland types, which becomes even more dissimilar across global climatic ranges when the area covered by each grassland type is addressed. We also found a great disparity of methods and data being reported across studies. Despite recent progress in understanding seed rain dynamics, large knowledge gaps in important issues, such as the role of native dispersers, method efficiency and application of mechanistic models still persist. Finally, we propose guidelines for the implementation of minimum standardized methodology and data reporting, which will foster higher quality, transparency, reproducibility and value of seed rain studies and grassland restoration.

Introduction

Seed rain is the number of seeds reaching an area, and it usually is quantified and qualified by placing traps in the plant community to catch seeds that then are identified and counted (Baskin & Baskin 2014). Seed rain is a critical step in plant life cycle, as it represents a demographic bridge between the adult and seedling stage [START_REF] Harper | Population Biology of Plants[END_REF]. Seed rain reflects species dispersal potential, and therefore the persistence or potential for change of the standing vegetation [START_REF] Page | Effectiveness of three seed-trap designs[END_REF].

Additionally, it allows the arrival of seeds into suitable uncolonized microsites [START_REF] Baker | The evolution of weeds[END_REF], with major implications for biological invasions and restoration ecology by revealing relevant information about how target species can reach a restored site, and whether seed rain is effective and efficient to restore a given site [START_REF] Turnbull | Are plant populations seed-limited? a review of seed sowing experiments[END_REF].

Many plant communities are seed limited, meaning that microsites where seeds can arrive and germinate, remain vacant, which can be related to limited seed production and/or limited dispersal of available seeds among sites [START_REF] Clark | Stages and spatial scales of recruitment limitation in southern Appalachian forests[END_REF]. Hence, seed rain estimates can provide a hint of successional direction by predicting the probabilities of propagule arrival [START_REF] Facelli | The role of the permanent soil seed bank in early stages of a post-agricultural succession in the Inland Pampa, Argentina[END_REF].

Over the last decades, seed rain has been analyzed both i) indirectly, by studies of plant reproductive potential [START_REF] Boughton | Reproductive traits of Lachnanthes caroliniana (Lam.) Dandy related to patch formation following feral swine rooting disturbance[END_REF]) and seed bank dynamics [START_REF] Bertiller | Seed bank strategies in Patagonian semi-arid grasslands in relation to their management and conservation[END_REF], and ii) directly, by collecting either seeds visible on the ground, by observing the movements of granivore animals [START_REF] Izhaki | Seed shadows generated by frugivorous birds in an eastern Mediterranean scrub[END_REF] or from diaspore traps [START_REF] Kollmann | Notes on seed traps in terrestrial plant communities[END_REF]. The great diversity of methodologies reported for seed rain studies may have a close relation with the wide range of biological processes that may be involved, which makes the delimitations of this biological process often unclear (Fig. S1).

In essence, an ideal method for studying seed rain is the one that can assess what seed, how many seeds and when that seed arrives in the seed bank [START_REF] Schott | A seed trap for monitoring the seed rain in terrestrial communities[END_REF].

Despite this, few seed rain studies use standardized methods that would allow crossvegetation comparisons. As an example, numerous types of seed traps have been used in ecological studies with different characteristics, such as shape, size, height above soil surface and trap inclination, all of which can affect their effectiveness [START_REF] Bakker | Seed banks and seed dispersal: important topics in restoration ecology[END_REF][START_REF] Chabrerie | Comparison of three seed trap types in a chalk grassland: toward a standardised protocol[END_REF]. A wide variety of seed rain measuring methods is available in the literature. However, there is little agreement to which is the most appropriate method taking into account a wide range of possible variables. This is especially true for grassy biomes, for which knowledge on natural regeneration following disturbance lags behind that of forest ecosystems [START_REF] Meli | A global review of past land use, climate, and active vs. passive restoration effects on forest recovery[END_REF].

Grassy ecosystems cover around 52 million km², ca. 40% of the global land surface [START_REF] White | Pilot analysis of global ecosystems: grassland ecosystems[END_REF]Gibson 2009). Over the last decades huge areas of native, old-growth grasslands have been lost to agricultural expansion, desertification, mining, urbanization and other changes or have been degraded by changes in fire regimes, exotic species introduction, fertilization, drainage, liming, overgrazing, etc. [START_REF] White | Pilot analysis of global ecosystems: grassland ecosystems[END_REF][START_REF] Veldman | Where tree planting and forest expansion are bad for biodiversity and ecosystem services[END_REF]. In the face of increasing land-use pressures and increasing biodiversity loss in grasslands, biogeographical studies of spatial and temporal patterns of seed rain are essential to restoration and management practices [START_REF] Bakker | Seed banks and seed dispersal: important topics in restoration ecology[END_REF].

While seed rain patterns in grasslands are very difficult to assess and to predict, a better understanding about their influence on the resilience and succession in these environments is urgently needed, aiming to achieve biodiversity conservation goals, secure ecosystem services [START_REF] Parr | Tropical grassy biomes: misunderstood, neglected, and under threat[END_REF]) and provide basis for better restoration practices.

Given that seed rain plays a pivotal role in restoration ecology, we explore the interplay between seed rain research and restoration in grasslands 20 years after [START_REF] Bakker | Seed banks and seed dispersal: important topics in restoration ecology[END_REF] stressed the role of seed rain and seed banks in restoration ecology. Our goals were: 1) to assess where, how and why research on seed rain was carried out; 2) to examine methodological design and results reported; and 3) to provide guidelines for future research in seed rain, aiming to standardize and foster higher quality, transparency, reproducibility and value to seed rain studies in grasslands.

Material and Methods

Literature survey and grassland classification

We surveyed papers in the Web of Science which have been published from key words of papers: "seed rain" plus "grassy biome", "grassland", "meadow", "prairie", "rangeland", "savanna", or "steppe". We removed papers dealing only with seed production or seed bank, and included only studies that addressed the releasing process of diaspores from the parent plant and their movements until reaching the soil.

After removing redundant papers, our survey comprised 185 papers published between 1980 and 2016. From those, we removed the papers not related to grasslands (those addressing seed rain in forest ecosystems), and papers that did not study seed rain in detail, represented by studies that did not present any qualitative or quantitative data on seed rain over space or time. We were left with 98 papers that matched our criteria for this review.

For each study, we classified the grassland type according to [START_REF] Dixon | Distribution mapping of world grassland types[END_REF].

We considered seven grassland types: alpine grassland; boreal grassland; cool semidesert grassland; temperate grassland; Mediterranean grassland; tropical grassland and warm semi-desert grassland. Using ArcGIS software, we built a map plotting the geographic distribution of seed rain studies together with the global grassland distribution through the geographical coordinates available in [START_REF] Dixon | Distribution mapping of world grassland types[END_REF].

Relevance of seed rain studies to restoration ecology

We considered studies with direct relevance to restoration ecology, such as those that addressed any of the following processes: recolonization, succession and predictions after degradation, endangered species conservation, spread of non-native species and natural plant communities or species dynamics. We classified degradation types reported in each study as endogenous or exogenous disturbances (sensu McIntyre and Hobbs 1999). The endogenous category refers to disturbances to which ecosystems were repeatedly exposed through evolutionary time and which, with an appropriate regime, allow ecosystems to maintain biodiversity. On the other hand, the exogenous category refers to novel, mainly anthropogenic disturbances, which may be incompatible with the maintenance of grassland biodiversity. We classified the ecological restoration processes, whenever mentioned, as passive or active restoration.

Passive restoration stands for cases when only natural regeneration processes were evaluated, whereas active restoration refers to the implementation of restoration techniques, such as hay spreading or seed sowing [START_REF] Stuttgart | Creating woodland islets to reconcile ecological restoration, conservation, and agricultural land use[END_REF].

Methodological design

We classified the studies depending on the methodologies used for seed rain studies according to the nature and scope of the experimental design into two categories: i) direct measurements and ii) mathematical or mechanistic models and simulations. The direct measurements included the use of seed traps [START_REF] Bakker | Seed banks and seed dispersal: important topics in restoration ecology[END_REF] and focal or seed tracking observations (e.g. [START_REF] Ferguson | Influence of vegetation structure on spatial patterns of seed deposition by birds[END_REF][START_REF] Piazzon | Seed dispersal by lizards on a continental-shelf island: predicting interspecific variation in seed rain based on plant distribution and lizard movement patterns[END_REF]. Mathematical or mechanistic models and simulations used numerical methods for seed rain dynamics predictions and simulations (e.g. [START_REF] Doisy | Weed seed rain interception by grass cover depends on seed traits[END_REF].

In order to obtain additional information about the dispersal dynamics, we searched for studies that used methods to evaluate the effect of abiotic (such as water and wind) and biotic vectors of seed dispersal. Additionally, we recorded seed dispersal distance whenever provided in the original study.

Owing to the fact that seed trap characteristics may strongly influence results [START_REF] Bakker | Seed banks and seed dispersal: important topics in restoration ecology[END_REF][START_REF] Chabrerie | Comparison of three seed trap types in a chalk grassland: toward a standardised protocol[END_REF], we evaluated the following aspects for studies that employed seed traps as a method to estimate seed rain: trap type, total number of traps, trap efficiency in collecting seeds, trap protection against granivores, trap position on the ground, height categories for traps lodged above ground, trap position in relation to the main wind direction, and sticky traps slope angle. We classified the traps in the following categories: funnel trap, sticky trap, tray or gap trap and other traps.

Methodologies and data reporting

To evaluate standardization in methodologies and data reporting, we analyzed seed trap size and total surface measurements for the two most common trap types, funnels and sticky traps. Considering that traps with equivalent areas can have very different formats, which together with the total trap area, can affect the study outcome, we assessed how these measurements were reported in different ways, sorting them into three possible categories: i) trap area only; ii) format measures (width and length or diameter), iii) not informed. Secondly, we evaluated how many studies calculated the total area sampled by the traps. Thirdly, we evaluated how the data collected or estimated were reported. We created four categories of data reporting: seed density only, total number and density, total number only, not informed. Finally, we examined what seed traits [START_REF] Jiménez-Alfaro | Seed germination traits can contribute better to plant community ecology[END_REF] were reported in each study.

Results

We found an increase in the number of seed rain studies in grasslands over the two last decades, with the maximum number of studies/year of 10 in 2005. We found relative increases in the number of studies coinciding with the publication of review papers (Fig. S2).

We found seed rain studies in grasslands across all vegetated continents, in tropical (N=7), temperate (N=82) and boreal (N=3) ranges (Fig. 1). However, the geographic distribution of studies was strikingly uneven, with most studies concentrated in Europe (N=46) and in North America (N=21). More than half (57%) of the seed rain studies were conducted in temperate grasslands, while the rest was spread among warm semi-desert grasslands (14%), alpine grasslands (10%), Mediterranean grasslands (8%), tropical grasslands (6%), boreal grasslands (3%) and cool semi-desert grasslands (1%).

Only three studies were done in flooded grasslands, all belonging to the temperate grassland type. Eighty-four papers (85%) of the selected studies were directly or indirectly related to ecological restoration. Among the studies related to ecological restoration, 80 papers (95%) mentioned some type of disturbance, from which 73 (91%) reported exogenous disturbances, such as cultivation and mining and only seven (8%) of them addressed endogenous disturbance events. Regarding the latter, five studies addressed fire regimes, one mentioned drought regimes and fire and one simulated small soil disturbances naturally created by a native small mammal from a temperate grassland (Supplementary material I). Additionally, from the eighty-four studies applied to ecological restoration, 64% addressed passive restoration aspects only and 36% active restoration processes. From the 30 studies that addressed active restoration, only 11% tested seed sowing techniques.

From the ninety-eight selected studies, 97% used direct measurements and 3% mathematical or mechanistic models and simulations. Additionally, across all selected studies, 21 mentioned or surveyed biotic dispersal. Within these 21 studies, birds were the main dispersers studied (42% of the studies), followed by livestock (29%), such as sheep, cattle and donkeys, ants (29%), other mammals not including bats (14%), bats (10%) and lizards (5%). Only 9 studies mentioned the influence of abiotic factors, such as wind (8 studies) and water (1 study). From the eight studies that provide some information about the main wind direction, five only mentioned and three measured this variable. Additionally, we checked how many studies estimated seed dispersal distance and found that only 24 studies took seed dispersal distance into account, from which less than a quarter presented seed dispersal distance estimates or measurements.

Fifty-nine percent of seed rain studies were conducted at the community-level and 37% only at species-level. From the 37 studies that evaluated the seed rain at species-level, 51% included only one species. Additionally, from all studies, 20 mentioned or evaluated the impact of non-native species, of which 70% reported nonwoody species and 30% woody species.

Only 46 studies provided information on complementary physical and physiological seed traits, such as size, weight, dormancy and viability. From those, 54% provided information on a single seed trait and less than 5% on more than three traits.

Among the seed traits reported in these 46 studies, seed morphological adaptations for dispersal (other than size and weight) were the most common (39% of the studies), followed by seed size (33%), seed weight (30%), seed viability (24%), seed dormancy (15%), releasing height (11%), seed longevity (4%) and moisture content (2%).

Sampling length for the 79 studies that provided this kind of information ranged from less than 1 month up to 60 months. Additionally, 76 studies provided information on sampling frequency, which ranged from weekly to 24 months intervals. Regarding the methods for seed identification, 89 studies dealt with this aspect. From those, 50% used visual identification through reference collections, 33% did not mentioned any seed identification method and 15% used seedlings emerging from seeds to identify species.

From the 76 studies (77%) that used seed traps, 88% used only one type of seed traps, 8% used two types, 3% used three types and only one study used five types. The most common seed trap type was the funnel trap (34%), followed by the sticky trap (18%) and tray or gap trap (17%). Additionally, 18% of the studies used other types of traps. Surprisingly, we found information on the total number of traps only for 56 out of the 76 studies. Among these, the number of traps per study ranged from 14 to 576, with nearly half of the studies using less than 90 traps.

Among the 76 studies that used seed traps, 62 (82%) did not provide any information about the seed trap efficiency in collecting seeds. Only eight studies (10%) mentioned trap efficiency and only six studies (8%) tested seed traps for efficiency.

Only 18 studies (24%) provided information on trap protection against granivores.

Information on seed trap position related to the ground was found for 56 studies only.

Among these 56 studies, 52% placed traps aboveground, 50% at ground level and only 4% underground (some studies included more than one type). For aboveground traps, we found a minimum height value of 0.5 cm and maximum of 90 cm. We also found that 32% of aboveground traps were positioned at a height of 50 cm or more, followed by those at 15 to less than 30 cm (28%), at one to less than 5 cm (25%) and at less than 1 cm (21%).

Additionally, we analyzed separately some specific aspects of trap types. We evaluated the sloping angle for sticky traps in the 17 studies (22%) that used this kind of trap. From these, 76% did not report this variable. Among those which reported this information, the angle of 0° (parallel to soil surface) was the most common (3 studies), followed by the angles of 45° and 90°, each one present in one study.

We evaluated seed trap measurements for the most common traps, funnel and sticky traps. For the funnel and sticky traps, respectively 30% and 6% of the studies provided the seed trap surface area only. Subsequently, we evaluated trap area separately for each type of traps. For the funnel and sticky traps, most studies used particular trap area sizes. Additionally, only 14% of the studies calculated the total area covered by the funnel or sticky traps.

We also evaluated how the total number of seeds collected or estimated was reported for the 82 studies that reported these data. Half of the studies presented seed density only (50%), followed by total seed number and seed density (28%) and total seed number only (22%).

Discussion

Our data clearly shows a lack of standardization in approaches, methods and data being reported in seed rain studies across global grasslands. We found unbalanced distribution of seed rain studies among hemispheres, different grassland types and, more importantly, that even basic aspects of a study (experimental design and number of species/seeds sampled), were not properly reported in many cases. Seed rain studies have different goals, and one should expect a wide diversity of methods in the literature.

However, the lack of justification for the employment of different methods, and incomplete data reporting strongly hamper our ability to compare results among studies, hence preventing a better appreciation of the role played by seed rain in restoration ecology.

General and biogeographical information

The large proportion of seed rain studies related to ecological restoration underlines the great relevance of seed rain studies in restoration ecology [START_REF] Bakker | Seed banks and seed dispersal: important topics in restoration ecology[END_REF][START_REF] Freund | Restoration of a newly created inland dune complex as a model in practice: impact of substrate, minimized inoculation and grazing[END_REF]. The small number of studies evaluating the effect of endogenous disturbances points out that this issue still needs to be better explored: endogenous disturbances usually have to be re-established as a means of or after restoration. For example, fire-stimulated flowering [START_REF] Keeley | Role of fire in regeneration from seed[END_REF] and seed release [START_REF] Holmes | Protocols for restoration based on recruitment dynamics, community structure, and ecosystem function: perspectives from South African fynbos[END_REF]) may affect natural regeneration patterns. However, natural recovery of some ecosystems may be quite slow if important drivers of recovery, such as the availability of propagules or dispersers are limited [START_REF] Hubbell | Light gap disturbances, recruitment limitation, and tree diversity in a neotropical forest[END_REF][START_REF] Stradic | Restoration of Neotropical grasslands degraded by quarrying using hay transfer[END_REF]).

The geographical distribution of studies can substantially influence conclusions reached by ecologists, being therefore critical to know which biomes, regions, and landscapes remain understudied and undervalued [START_REF] Martin | Mapping where ecologists work: biases in the global distribution of terrestrial ecological observations[END_REF]. The remarkable unbalance between the numbers of studies worldwide indicates knowledge gaps, especially in old-growth tropical grasslands in the southern hemisphere [START_REF] Veldman | Where tree planting and forest expansion are bad for biodiversity and ecosystem services[END_REF]. Considering the total area per grassland type [START_REF] Dixon | Distribution mapping of world grassland types[END_REF], alpine grasslands showed the second lowest area among the formations, but had more seed rain studies than tropical grasslands, which have an approximately 27 times higher total area (Table 1).

High diversity of estimates of seed rain

The considerable dominance of direct measurements to evaluate seed rain patterns reflects the sampling effort carried out in field experiments with use of seed traps or focal observations over the last few decades. Despite the potential usefulness of mechanistic models in providing reliable estimates of plant dispersal distances, they are still little explored by ecologists [START_REF] Bullock | A synthesis of empirical plant dispersal kernels[END_REF]. The development of dispersal mechanistic models is crucial to improve our understanding about relevant topics on restoration ecology, such as population dynamics in fragmented landscapes [START_REF] Gilbert | Spreading speeds for stage structured plant populations in fragmented landscapes[END_REF]) and the arrival of non-native species [START_REF] Hastings | The spatial spread of invasions: new developments in theory and evidence[END_REF]. Meanwhile, there is still a long way to go from mechanistic models to processes applicable to seed dispersal, in order to reduce the effort required to measure dispersal directly in the field [START_REF] Bullock | Measuring plant dispersal: an introduction to field methods and experimental design[END_REF].

Animals play important roles in dispersing seeds of native and non-native species. Most studies surveyed explored the role of frugivorous passerines or livestock as dispersers of grassland species. However, we found a large knowledge gap on the ecological role played by other native animals, such as lizards [START_REF] Piazzon | Seed dispersal by lizards on a continental-shelf island: predicting interspecific variation in seed rain based on plant distribution and lizard movement patterns[END_REF], beetles or ants [START_REF] Nicolai | Harvester ants modify seed rain using nest vegetation and granivory[END_REF][START_REF] Lima | Interactions between ants and nonmyrmecochorous fruits in Miconia (Melastomataceae) in a neotropical savanna[END_REF]) and non-volant mammals [START_REF] Genrich | Duality of interaction outcomes in a plant-frugivore multilayer network[END_REF] in dispersing seeds of native grassland species. Additionally, future research should focus on the roles of seed dispersers in dispersing seeds of invasive species, which will provide relevant information for restoration actions and post-restoration management [START_REF] Buisson | Plant phenological research enhances ecological restoration[END_REF]. Spatial patterns of seed deposition and issues related to seed sourcing, seed quality, availability and dormancy-breaking are of great interest for evaluating the effectiveness of dispersal and relevant hurdles for plant community reassembly after degradation [START_REF] Dayrell | Overcoming challenges on using native seeds for restoration of megadiverse resource-poor environments: a reply to Madsen et al[END_REF]. However, assessing the contribution of local versus distant diaspore sources can be a great challenge [START_REF] Bullock | A synthesis of empirical plant dispersal kernels[END_REF], which may explain how estimating seed dispersal distance has been mostly overlooked by seed rain studies in grasslands.

Sampling length, sampling frequency and seed identification techniques can have a great influence on the results. Thus, whenever possible, studies should prioritize longer collection periods and short collection frequencies. Short-term studies may imply non-overlapping patterns of dispersal period of some species. Furthermore, results are likely to be affected by the frequency of checking, mainly when traps are insufficiently or not at all protected against seed predation [START_REF] Gorchov | The role of seed dispersal in the natural regeneration of rain forest after strip-cutting in the Peruvian Amazon[END_REF].

Sampling effort and lack of standardization in the use of seed traps

Measurements of seed rain in the field can be carried out using seed traps with different shapes, sizes, heights and inclinations [START_REF] Chabrerie | Comparison of three seed trap types in a chalk grassland: toward a standardised protocol[END_REF]. However, trap effectiveness at capturing seeds can vary not only among trap types and characteristics, but also over different vegetation or plant species and environmental conditions [START_REF] Kollmann | Notes on seed traps in terrestrial plant communities[END_REF]. Funnel traps have often been described in the literature as the best trapping method by catching the highest number of seeds [START_REF] Kollmann | Notes on seed traps in terrestrial plant communities[END_REF]. However, funnel trap efficiency may also retain diaspores belonging to seed banks through water run-off, and therefore ignoring secondary dispersal may result in overestimates of seed rain. Sticky traps, which are especially suitable for studying seed rain of anemochorous species, can be easily adapted to the local aerodynamic environment by the control of height, orientation and inclination [START_REF] Chabrerie | Comparison of three seed trap types in a chalk grassland: toward a standardised protocol[END_REF]. However, sticky traps can catch large amount of insects, drastically reducing their efficiency and making it difficult to identify the sampled seeds [START_REF] Poschlod | Vegetationsentwicklung in abgetorften Hochmooren des bayerischen Alpenvorlandes unter besonderer Berücksichtigung standortskundlicher und populationsbiologischer Faktoren[END_REF]). Alternatively, the use of soil gaps or pots with sterile soil have been often attributed as a more realistic method by the integration of seed arrival with other field natural conditions, such as secondary removal and predation [START_REF] Kollmann | Notes on seed traps in terrestrial plant communities[END_REF], despite the recommendation for the avoidance of these variables for other trap types [START_REF] Debussche | Bird-dispersed seed rain and seedling establishment in patchy Mediterranean vegetation[END_REF]. Considering that trapping success can be very variable not only between trap types, but across environments and plant communities, comparisons between studies should always be made with caution [START_REF] Kollmann | Notes on seed traps in terrestrial plant communities[END_REF].

Despite the fact that effects of seed trap area and height on sampling effectiveness have been debated since the seminal publications of [START_REF] Fischer | Investigations on population dynamics in the beginning of secondary succession[END_REF] and [START_REF] Jackel | Diaspore production and the influence of the size of diaspore traps on the quantitative result of seasonal diaspore rain in two calcareous grassland sites[END_REF], we still need to better explore how seed traits, such as size, weight and releasing height [START_REF] Fischer | Experimental studies on the dispersalof plants and animals on sheep in calcareous grasslands[END_REF][START_REF] Jackel | Diaspore production and the influence of the size of diaspore traps on the quantitative result of seasonal diaspore rain in two calcareous grassland sites[END_REF][START_REF] Tackenberg | Assessment of wind dispersal potential in plant species[END_REF]) can influence seed trap efficiency. Recommendations about the number of traps for seed rain studies have already been reported [START_REF] Fischer | Investigations on population dynamics in the beginning of secondary succession[END_REF][START_REF] Kollmann | Notes on seed traps in terrestrial plant communities[END_REF], but the number of traps needed greatly vary depending on the goals and background of the studies.

Seed trap effectiveness tests are fundamental in seed rain research and should be conducted prior to field experiments [START_REF] Debussche | Bird-dispersed seed rain and seedling establishment in patchy Mediterranean vegetation[END_REF]), but we found they are rarely carried out. Ignoring such issues may have serious consequences in the reliability of seed rain estimates. Standardization of sampling efforts [START_REF] Jackel | Diaspore production and the influence of the size of diaspore traps on the quantitative result of seasonal diaspore rain in two calcareous grassland sites[END_REF]) and in tests of trap efficiency in collecting, storing and protecting seeds against granivores will be fundamental for a better understanding of the operation of different seed traps along distinct plant communities and geoclimatic conditions.

Methodologies and data reporting

Basic information about methods and results must be thoroughly, clearly and transparently reported to enable comparisons and facilitate inclusion in meta-analyses [START_REF] Gerstner | Will your paper be used in a meta-analysis? Make the reach of your research broader and longer lasting[END_REF]. Seed trap format measures, area and the total number of seeds collected or estimated reported by the studies evaluated here are remarkably unstandardized, sometimes unclear or sometimes data were even not provided. We also found a great variety of sizes for both sticky and funnel traps, which were rarely accompanied by a clear methodological justification. Finally, only few studies provided information on the proportional seed trap area (seed trap area over total plot area).

Altogether, the lack of standardized methodologies in seed rain studies undermine the reliability of studies and prevent comparisons among sites, thereby preventing the evolution of the scientific knowledge on a key ecological process underpinning ecological restoration.

Implications and guidelines for future seed rain studies

When reporting data, researchers should have in mind that their data can be readily used by the scientific community to build-up knowledge from single studies [START_REF] Gerstner | Will your paper be used in a meta-analysis? Make the reach of your research broader and longer lasting[END_REF]. Therefore, it is imperative that not only experimental designs are clear and justified, but that data reporting adheres to minimum standards. Here, we propose guidelines for future research on seed rain studies in grasslands with the hope that standardized methodology, or at least better justified methods, will provide the scientific community with better conditions for the critical assessment of publications, fostering scientific knowledge.

First, we propose the standardization for the number of seeds collected over time and space, according to the minimum parameters: total number of seeds per species sampled at each sampling interval; total number of species sampled at each interval; total number of seeds/species per trap over the total sampling time and total number of seeds/m² per day. Second, given the disparity of methods and data reporting across studies that used seed traps, we propose that the following minimum standard is reported: trap format measurements (width and length or diameter); total trap area per unit (cm²); number of traps per plot; total area covered by traps per plot (cm²); and percentage (%) of plot area covered by the traps. We recognize that our modest proposal will not be enough to standardize research at global scale, but this is the first step towards common and shared vocabulary, methods and data reporting.

Prior to restoration, seed rain can be evaluated in order to assess the potential for regeneration and passive restoration. Restoration assessments should include seed rain data in post-restoration monitoring programs [START_REF] Jacquemyn | Management effects on the vegetation and soil seed bank of calcareous grasslands: An 11-year review[END_REF]) and increasing knowledge on pre-and post-dispersal seed predation [START_REF] Pardini | Effects of seed density and proximity to refuge habitat on seed predation rates for a rare and a common Lupinus species[END_REF]) are to play an increasing role in determining restoration success and should be integrated into seed rain studies whenever possible. Despite considerable progress in recent decades, we still face great challenges to improve our knowledge and increase the use of seed rain studies to better support grassland restoration (Table 2). More research on restoration of tropical and sub-tropical grasslands is required due to the fact that techniques used for temperate grassland restoration are not successful in restoring tropical ones [START_REF] Stradic | Restoration of Neotropical grasslands degraded by quarrying using hay transfer[END_REF]. Additionally, understanding to what extent anthropogenic modifications are relevant to seed limitation will be fundamental to predict the capacity of grasslands to respond to these changes.

Future successful restoration may also depend on seed addition, on which we still need to advance in the practical and theoretical framework [START_REF] Bakker | Contingency of grassland restoration on year, site, and competition from introduced grasses[END_REF]. We must also recognize the importance of considering several characteristics of seed rain, besides species composition and seed abundance. Spatial patterns of dispersal and seed traits, such as dormancy, longevity, releasing time and duration are fundamental for a better understanding on the formation of seed banks and the dynamics of community reassembly following disturbances [START_REF] Jiménez-Alfaro | Seed germination traits can contribute better to plant community ecology[END_REF].

Lastly, we need to critically stress the fact that cross-vegetation comparisons of seed rain data are prevented due to lack of standardized research protocols. We believe that the lack of standardization in methodology and data reporting creates a unique opportunity for the scientific community to put efforts on a global protocol of seed rain methods. An increased implementation of guidelines for methodology and data reporting will foster higher quality, transparency, reproducibility and value to seed rain studies in grasslands. Future collaborative, international efforts are paramount for a global assessment of the role played by seed rain in restoration ecology.

Table 1. Comparison between the number of seed rain studies with the total area for each grassland type according to Dixon et al. 2014 (=98 studies).

Grassland type

Total

Challenges Solutions

Seed rain spatial patterns,

Figure 1 :

 1 Figure 1: Geographic distribution of seed rain studies (black dots) in grasslands. Red

  heterogeneity and limitation Increase periods of observation and reduce sampling intervals Predict latitudinal and climate variations effects on seed rain Incorporate landscape metrics and estimate the relative importance of different dispersal modes Discuss how the number of traps should vary according to study goals Seed trap efficiency and lack of standardization in the use of seed traps Consider the main direction of seed import Consider vegetation growing height and dispersal syndromes Provide justification for the choice of seed traps (position on the ground, format, area, etc.) Test trap efficiency and protect seeds against granivory Critically evaluate and report negative aspects of each the role of overlooked vectors in the dispersal of non-woody species Hurdles for plant community reassembly after degradation Understand how issues related to seed sourcing, seed quality, availability and dormancy-breaking mechanisms Unstandardized or unclear methodology and lack of data reporting Methods and results must be thoroughly, clearly and transparently stated Standardization for the number of seeds collected over time and space Standardization for the trap format measurements and area within the sample area per plot

Table 2 .

 2 Future challenges and proposed solutions to improve our knowledge and increase the use of seed rain studies to support grassland restoration.

		area	N° of seed rain	Studies/100.000
		(Km²)	studies	km²
	Boreal Grassland	246.322	3	1.21
	Alpine Grassland	591.357	10	1.69
	Mediterranean	1.594.750	8	0.50
	Grassland
	Warm Semi-Desert	3.030.720	13	0.43
	Grassland
	Cool Semi-Desert	5.661.110	1	0.02
	Grassland
	Temperate Grassland	8.104.830	55	0.68
	Tropical Grassland	16.156.620	6	0.04

Acknowledgments

We thank RS Oliveira, QS Garcia, NPU Barbosa and LF Fuzessy for criticism during early stages of the manuscript. NPU Barbosa and RLC Dayrell also assisted with the construction of the map and figures. We thank the reviewers for their comments which significantly increased manuscript quality. AJA receives a PhD scholarship from CAPES and FAOS receives grants from CNPq and FAPEMIG.