François-Michel De Rainville

Michèle Sebag

Christian Gagné

Marc Schoenauer

Denis Laurendeau

Sustainable Cooperative Coevolution with a Multi-Armed Bandit

Keywords: Categories and Subject Descriptor I.2.8 [Artificial Intelligence]: Problem Solving, Control Methods, and Searchheuristic methods Cooperative Coevolution, Multi-Armed Bandit, Credit Assignment, Adaptivity Algorithm 1: Cooperative co-evolution

This paper proposes a self-adaptation mechanism to manage the resources allocated to the different species comprising a cooperative coevolutionary algorithm. The proposed approach relies on a dynamic extension to the well-known multi-armed bandit framework. At each iteration, the dynamic multi-armed bandit makes a decision on which species to evolve for a generation, using the history of progress made by the different species to guide the decisions. We show experimentally, on a benchmark and a real-world problem, that evolving the different populations at different paces allows not only to identify solutions more rapidly, but also improves the capacity of cooperative coevolution to solve more complex problems.

Introduction

Coevolutionary algorithms [START_REF] Axelrod | The evolution of strategies in the iterated prisoner's dilemma[END_REF][START_REF] Hillis | Co-evolving parasites improves simulated evolution as an optimization procedure[END_REF] are extending traditional evolutionary algorithms by making use of several subpopulations of individuals, often designated as species, and evaluating the fitness of the individuals jointly with individuals from other species. Cooperative coevolutionary algorithms [START_REF] Potter | Cooperative coevolution : An architecture for evolving co-adapted subcomponents[END_REF] exploit modularity to decompose a problem into multiple interacting subcomponents, in a divide-and-conquer fashion. It is a paradigm where subcomponents not only emerge naturally from the evolution, but also cooperate to solve much more complex problems than those that can be tackled using standard single-species genetic algorithms. Examples of successful applications of cooperative coevolution include ensemble pattern classification [START_REF] García-Pedrajas | Cooperative coevolution of artificial neural network ensembles for pattern classification[END_REF], multi-agent learning [START_REF] Panait | Cooperative multi-agent learning: The state of the art[END_REF], and sensor placement [START_REF] De Rainville | Co-adapting mobile sensor networks to maximize coverage in dynamic environments[END_REF], among others.

Fair competition between the subcomponent populations has been introduced in the original algorithms. Indeed, each population can run for an arbitrary number of generations before the other populations provide new cooperating individuals. Potter and De Jong [START_REF] Potter | Cooperative coevolution : An architecture for evolving co-adapted subcomponents[END_REF] present an asynchronous evolution of the populations, although only few have reported making their optimization in an asynchronous manner. However, for situations where the natural decomposition of the problem 1 yields subproblems of varying difficulty, it is natural to assign a different number of generations to each subpopulation. The issue is then to determine how the generations should be allocated to each species.

That issue can be formulated as a decision problem where we need to determine which population should be evolved at each time step, in order to maximize the global progress of the coevolutionary algorithm. The multi-armed bandit framework [START_REF] Gittins | Bandit processes and dynamic allocation indices[END_REF][START_REF] Lai | Asymptotically efficient adaptive allocation rules[END_REF], widely studied in the context of game theory, strives at proposing actions maximizing the expected reward while acquiring more knowledge on the decision process. In our context, it is thus a natural and promising avenue to follow for an automatic subpopulation selection mechanism. Therefore, we propose to apply a dynamic multi-armed bandit to allocate more computational time to the most promising species in order to sustain the coevolution. For this purpose, we also present an efficient strategy to evaluate the rewards, in order to balance the evolution between the species, as this is a non-stationary decision process.

The structure of the paper is as follows. First, in Sec. 2 and 3, we discuss the important concepts involved in cooperative coevolution and multi-armed bandits, respectively. Then, we present our framework for the selection of the most promising population in Sec. [START_REF] Costa | Adaptive operator selection with dynamic multi-armed bandits[END_REF]. Next, we present experiments with our framework in Sec. 5 on an artificial problem and a real-world scenario on sensor placement. Finally, we conclude the paper on the contributions of this article and possible future work in Sec. 6.

Cooperative Coevolution

Cooperative coevolution has been proposed by Potter and De Jong [START_REF] Potter | Cooperative coevolution : An architecture for evolving co-adapted subcomponents[END_REF] to exploit the modularity of a problem, similarly to the rules in Learning Classifier Systems and subroutines in Genetic Programming. In this paradigm, each subpopulation, the so-called species, represents a subcomponent of a solution, and a complete solution is obtained by combining a representative of each species. Each individual is assigned a fitness equal to the reward of the complete solution in which it participates. The number of species can change along the evolutionary process, new species being introduced when the evolution stagnates and species being removed when their best individual does not contribute enough to the complete solution. Potter and De Jong also propose to evolve the species asynchronously, which would allow to allocate more time to some species. To the authors knowledge, no work has been made to adjust the effort on each species to sustain the cooperative optimization. However, multiple avenues have been explored to improve the original algorithm, some are presented in this section.

A multiple interactions scheme for the cooperative coevolution has been studied by Wiegand [START_REF] Wiegand | An Analysis of Cooperative Coevolutionary Algorithms[END_REF] to increase the convergence of the algorithm, but it has been shown to generate subcomponents that are over-generalizing to fit to a multitude of collaborators. Furthermore, Bucci and Pollack [START_REF] Bucci | On identifying global optima in cooperative coevolution[END_REF] proposed a modification of the original cooperative coevolutionary algorithm by replacing the selection mechanism with a new one based on Pareto dominance. However, establishing the Pareto dominance among the individuals of each species requires a tremendous number of fitness evaluations when the population size or the number of populations increases. In fact, it requires O(nm k-1 ) evaluations, where n is the number of individuals in the evaluated species, m is the number of individuals in each population with whom each individual is evaluated (required for evaluating Pareto dominance), and k is the number of populations. Such solution is not always possible for real world problems and thus other avenues need to be explored.

1 initialize P ← {S i , i = 1, . . . , n} 2 choose R ← {select random(S i ), i = 1, . . . , n} 3 while ¬stop do 4 foreach species S i ∈ P do 5 S i ← apply variations(S i ) 6 evaluate(S i , R\r i ) 7 S i ← select(S i ) 8 R ← {select best(S i ), i = 1, . . . , n} 9 if improvement < T i then 10 remove species with contribution < T c 11 R ← R\R - 12 add a new species P ← P ∪ {S ′ } 13 R ← R ∪ {select random(S ′ )}
Panait et al. [START_REF] Panait | Biasing coevolution search for optimal multiagent behavior[END_REF] biased the coevolutionary process by augmenting the fitness of an individual with an estimate of its best possible reward if it was evaluated with its optimal collaborators. The problem is now how to find the optimal collaborators for an individual in a non trivial problem. The authors propose to pair the individuals with the most successful collaborators found yet; or with collaborators that have cooperated well with structurally similar individuals; among other ideas. To our knowledge, none of these options have been tested nor compared with their proposed method when the optimal collaborators are known, making this solution unapproachable for real world problems.

Wu and Banzhaf [START_REF] Wu | A hierarchical cooperative evolutionary algorithm[END_REF] introduced a hierarchical decomposition of the problem domain. Cooperation among the individuals of the population is rendered by "groups" that are evolved in the framework. Groups provide a collaborating mechanism by assembling different individuals together into a solution which can be evaluated. In their algorithm, individuals and groups have their own fitness, which is shared by all individuals in a sharing radius to preserve diversity in the populations. Their technique exploits the uniqueness of the contribution of each subcomponent in a group to avoid carrying "free riders" that would have the same effect as bloat in Genetic Programming. Such uniqueness measures might not always be available for real world problems.

Albeit the promising enhancements to the original algorithm, this work is based on the standard cooperative coevolution of Potter and De Jong [START_REF] Potter | Cooperative coevolution : An architecture for evolving co-adapted subcomponents[END_REF], for its simplicity, its general applicability to any problems, and its ease of implementation. The pseudo-code is presented in Algo. 1. First, a population P of n species S i is initialized randomly and one representative r i of each species is placed in the representative set R. Then, lines 4 to 7 show how each species is evolved independently of the others and how interaction through evaluations is achieved. More specifically, on line 6, the evaluation of the individuals of a species is made in collaboration with the representatives of the other species, thus excluding the representative of the current species from set R (R\r i ). The inner loop, beginning at line 4, is a classical genetic algorithm, which can be replaced by any other evolutionary algorithm. At each ecosystem generation, i.e. the outer loop beginning at line 3, the representatives are updated with the best individual of each species for the next generation. Then, the improvement of the evolution is verified at line 9. If the solution fitness, given by all representatives, has not improved over the last I generations by more than a given threshold T i , then the system is considered as stagnating. In this situation, unproductive species that contribute less than a threshold T c are removed from the population1 -their representatives (R -) are also removed from R -and one new species is added to the population. The evolution continues until the termination criterion is reached.

Multi-Armed Bandit

Multi-Armed Bandits (MAB), introduced by Robbins [START_REF] Robbins | Some aspects of the sequential design of experiments[END_REF], are used to model the explorationexploitation trade-off faced by an agent that takes actions in a given environment for which rewards are issued. In this setting, the agent has two opposite goals, which are: 1) to gain more knowledge about the environment and the rewards obtained (exploration), and 2) to exploit its current knowledge on the actions in order to maximize its immediate reward (exploitation). The standard MAB problem consists in a set of k arms, each arm representing a possible action. A fixed and unknown reward probability distribution r i ∈ [0, 1] is associated to each arm. At each time step of the problem, the agent selects an arm and receives a reward according to the corresponding reward probability.

Auer et al. [START_REF] Auer | Finite-time analysis of the multi-armed bandit problem[END_REF] proposed the Upper Confidence Bound (UCB) algorithm, which has been demonstrated to achieve asymptotically the optimal regret rate in the classical MAB case of independent and stationary reward probability distributions. This means that this algorithm strives at finding the best arm as fast as possible, while trying the other arms at an exponentially decreasing frequency. In an environment involving coevolutionary algorithms, none of the independent and stationary reward distribution assumptions are respected. Indeed, the quality of solutions in a species strongly depends on the solutions of the other species, which also makes the probability distributions non-stationary.

Da Costa et al. [START_REF] Costa | Adaptive operator selection with dynamic multi-armed bandits[END_REF] proposed a dynamic MAB combining the principles of the UCB and the Page-Hinkley statistics for restarting the method. This bandit was used to automatically select variation operators during the course of an evolutionary algorithm, with promising results. Fialho et al. [START_REF] Fialho | Toward comparison-based adaptive operator selection[END_REF] refined this dynamic MAB in order to improve the robustness and invariance to monotonic transformations. They added a rank-based credit assignment scheme based on the Area Under the Curve (AUC) measure, commonly used in machine learning.

Fialho et al. [START_REF] Fialho | Toward comparison-based adaptive operator selection[END_REF] AUC dynamic MAB has been used as a basic species selection mechanism for the current work. We slightly modified the credit assignment by not taking into account the ties in the ranked rewards, since our reward model is binary (ρ ∈ {0, 1}) instead of continuous (see Sec. 4). The order of appearance of the rewards is thus taken into consideration, as a more recent reward obtains a higher credit than an equal but older one. Algo. 2 presents the credit assignment scheme, with d ∈ [0, 1] being the decay factor proposed in [START_REF] Fialho | Toward comparison-based adaptive operator selection[END_REF] and w = [w 1 • • • w n ] contains the tuples rewards, index obtained by the arms in the last n iterations.

The selection mechanism of the MAB is the same as the one proposed in [START_REF] Fialho | Toward comparison-based adaptive operator selection[END_REF], it is presented in Algo. 3. First, the number of times each k arm is selected n

= [n 1 • • • n k ], the credit each arm obtained so far q = [q 1 • • • q k ]
, and the window of rewards w are all initialized. Then, we iterate over each selection step until the decision process ends. At first, if one or more arms have never been selected, we select randomly among these. Otherwise, we select an arm using the UCB formula, where C is a parameter to control exploration. Next, we apply the selected arm and Algorithm 2: Area under the curve credit assignment.

Input: arm index i, window of rewards w Output: credit q 1 sort lexicographically w according to rewards (best first), and ages in case of ties (newer first)

2 q ← y ← 0 3 forall the ranks r ∈ [1 • • • |w|] do 4 ρ, j ← w r 5 ρ ← d (r-1) (|w| -(r -1)) 6 if i = j then 7 y ← y + ρ 8 else 9 q ← q + yρ Algorithm 3: Multi-armed bandit selection. 1 n ← [0 • • • 0], q ← [0 • • • 0], w ← [] 2 while ¬stop do 3 if ∃n i ∈ n : n i = 0 then 4 arm ← select random({i|n i = 0}) 5 else 6 arm ← arg max i q i + C 2 log n j=1 n j n i 7
apply selected arm and retrieve reward

8 w ← [(reward , arm) w] 9 if |w| > W then 10 w ← [w i ∈ w, i = 0, . . . , W -1] 11 n ← [count(i, w), i = 1, . . . , n] 12 q ← [auc(i, w), i = 1, . . . , n]
retrieve the reward. A tuple made of the arm index and the reward is prepended to the window of rewards, from which the oldest tuple is removed if we have reached the maximum window size W . Finally, the number of times each arm is selected is updated according to the window and the credit is given by the AUC (Algo. 2) of each arm.

Selecting Species

The main idea at the source of the proposed algorithm is very simple. Instead of evolving every population at each generation, as presented in Sec. 2, we want to choose the most promising population to evolve with the dynamic MAB described in Sec. 3. This strategy has two major goals: it allows sustainable coevolution of the multiple subpopulations while balancing the global evolution resources allocated to each population, as they are not appearing in the evolution at the same time given the species addition and removal mechanisms used.

The evolution process starts similarly to the original algorithm. A number of species are initialized and one individual in each of them is selected as representative. The MAB is initialized with one arm associated with each species in the coevolution. Then, line 4 of Algo. 1 becomes lines 3 to 6 of Algo. 3, where the MAB selects the index i of the species to be evolved. The application of variation operators, evaluation, and selection of S i remain the same. Later, the best individual of S i replaces the previous representative of species i in R. The bandit is rewarded for the progress made by the representatives on the problem, and the rest of the iteration of Algo. 3 is run to complete the bandit update. The reward scheme will be detailed in the following Sec. 4.1. Finally, the cooperative process continues with the improvement verification of the standard algorithm (lines 9 to 13 of Algo. 1). The improvement length is set to the value in the original algorithm times the number of species, since we evolve only one species on each generation. The arms in the bandit are adjusted for each addition and removal of a species in the coevolution, as detailed later in Sec. 4.2.

Reward

Properly rewarding the selected arm is a crucial element that should be carefully designed to ensure success. The choice of the reward must encourage exploitation of the best arm while still allowing exploration of the other arms. The concept used in our case is borrowed from an assumption made in the original UCB, contrary to what is presented by Fialho et al. [START_REF] Fialho | Toward comparison-based adaptive operator selection[END_REF]. The reward given to the bandit is binary. An arm is assigned a reward of 1 if the fitness of the selected species representative is better than the one at the previous iteration, and 0 otherwise.

We argue that this reward method is the best that we can use in the context of population selection in cooperative coevolution compared to the direct and delta fitness assignment of Fialho et al. [START_REF] Fialho | Toward comparison-based adaptive operator selection[END_REF]. On the one hand, the direct fitness allocation suffers when the bandit is updated with a new fitness that caused a loss -if this fitness is still better than any other in the window that caused a gain, the arm will receive a higher credit for that loss than the other arm for its gain. This drawback has been addressed in the delta fitness assignment. On the other hand, this previous reward attribution is handicapped by the "log convergence" of optimization algorithms. The difference of fitness between consecutive iterations should decrease logarithmically with time. Sorting the delta fitness results in older data being privileged over new ones, which is counter intuitive as the current state of cooperation is probably closer to the next state than any older one. The Boolean reward allotment benefits from the properties of both assignments without their drawbacks, which explains its choice for the proposed method.

Adding and Removing Arms

As species are added and removed from the evolution, the number of arms must change. Instead of entirely restarting the bandit, we simply modify its internal state. When adding a species, we add entries in both the number of times each arm has been selected and the credit each arm received so far, that is n ←

[n 1 • • • n k 0] and q ← [q 1 • • • q k 0]
. This procedure forces the bandit to select the newly introduced species a couple of times, giving it a chance to find a suitable niche without being bumped out of the coevolution.

Removing an arm is conducted in a similar manner to the addition, with entries corresponding to the removed species in n and q being removed, leaving them with k -1 values each. For the window of reward w, we remove all entries in which the removed species (indexed l) is involved, w ← [∀(reward, arm) ∈ w|arm = l].

Experiments

The proposed algorithm (CCEA-MAB) is tested and compared against the original cooperative coevolution (CCEA) of Potter and De Jong [START_REF] Potter | Cooperative coevolution : An architecture for evolving co-adapted subcomponents[END_REF] on a string covering benchmark and a sensor placement problem.

String Covering

The binary string covering problem, introduced in [START_REF] Forrest | Using genetic algorithms to explore pattern recognition in the immune system[END_REF] and used by [START_REF] Potter | Cooperative coevolution : An architecture for evolving co-adapted subcomponents[END_REF][START_REF] Smith | Searching for diverse, cooperative populations with genetic algorithms[END_REF][START_REF] Wu | A hierarchical cooperative evolutionary algorithm[END_REF], benchmarks cooperative evolutionary algorithms. The problem consists in matching a number of strings in a test set T with strings in a match set M, where T is generally much larger than M. Thus, the strings in the match set must generalize the patterns in the target set to obtain an optimal cover. The match strength s(x, y) between two strings x and y is defined by the number of bits at the same position and of the same value that the two strings share. The strength of a match set S(M, T ) is the average maximum match strength its strings has against the target set,

S(M, T ) = 1 |T | t i ∈T max m j ∈M s(m j , t i ).
The cooperative algorithm evolves a match set that is evaluated against the fixed test set. A number of arrangements of the fixed and variable bits is predetermined, these arrangements are called schemata. The test set consists of an equal number of each schemata where the variable bits have been replaced by random bit values. In all experiments each set strings are 64 bits long.

The representation used for this problem is a 64 bits string. We use a two-point crossover and a flip bit mutation to generate the offspring, each variation operator being applied with a certain probability. The fitness of an individual is given by the match strength of the group it forms with the representatives of the other species. Tournament selection is used to select individuals in each species. The contribution of a species is computed by the number of times its representative has the best cover for a target string.

Four versions of the string covering problem are used to assess each property of the cooperative evolutionary algorithm of Potter and De Jong [START_REF] Potter | Cooperative coevolution : An architecture for evolving co-adapted subcomponents[END_REF]. While the two first tests gauge the ability of the algorithm to locate and cover multiple niches, and to evolve the appropriate level of generality, we are more interested in the capacity of adaptation to a changing number of species and the ability to evolve the right number of species. The parameters of the algorithms are shown in Tab. 1. The parameters concerning the evolutionary algorithms and the coevolutionary process are set similar to what is presented in Potter and De Jong [14, pp. 10 and 17] and those for the bandit are set accordingly to the conclusions made by Fialho [START_REF] Fialho | Toward comparison-based adaptive operator selection[END_REF].

Adaptation During the Evolution

The capability of adaptation of our version of the coevolutionary algorithm is crucial. As more time is allocated to more promising species, other species must still have enough time to adapt their representatives to the addition of new species. This test allows the ability of specialization of the species over time to be measured as more and more species are added to the evolution. For this experiment, the evolution is not in charge of adding or removing species, allowing to focus solely on the adaptation property of the algorithm. Starting with a single species, a new species is added at each 100 generations until there is the same number of species as schemata to be matched. At the end of the evolution all schemata should be covered. Three scenarios have been designed. Scenario 1 is similar to the one used by Potter and De Jong, where a 30 element target set is generated from the three following schemata, each containing 32 fixed bits and 32 variable bits.

1##1###1###11111##1##1111#1##1###1#1111##111111##1#11#1#11###### 1##1###1###11111##1##1000#0##0###0#0000##000000##0#00#0#00###### 0##0###0###00000##0##0000#0##0###0#0000##001111##1#11#1#11######
Scenario 2 is similar to the first one, with a 30 element target set. Five schemata are generated randomly to produce the target set, using the same noise pattern (variable bits) as in scenario 1, while the fixed bits correspond to those of a random integer uniformly chosen in [0, 2 32 -1]. Scenario 3 is composed of a 50 element target set generated from five schemata, with a varying number of variable bits. For each schema, we first draw uniformly from [START_REF] Smith | Searching for diverse, cooperative populations with genetic algorithms[END_REF]48] the number of variable bits (v) used. Then, the fixed bits are generated using the binary representation of a number taken uniformly in [0, 2 64-v -1]. Finally, the variable and fixed bits are shuffled together to form a 64-bit schema.

It should be stressed that for all scenarios, all runs of CCEA and CCEA-MAB are carried out in pairs, with the same target set used for each pair of runs. This allows a fair comparison between the methods.

On 100 runs for scenario 1, the original and the MAB-driven algorithms find the optimal string cover on all occasions. A typical run is shown in Fig. 1, where the match strength against each schemata is presented. Fig. 1(b) clearly illustrates the allocation to the promising species right after generation 100 and 200 where the slope of the number of bits matched is generally steeper for CCEA-MAB. We also notice that once a species has converged to a schemata, time is allocated to adjust the other species, if there can be a reward, as exposed by the difference between the number of times each species is evolved in CCEA-MAB. In the third part of Fig. 1(b), we observe that species 2, which has converged to schemata 3, is chosen by the bandit only 7 times over the 100 last generations while the two other species for which convergence is still possible are chosen 44 and 49 times. This process saves much effort since the progress made by evolving a species is not interrupted by evolving another less promising or already converged one.

For the second scenario with five schemata, CCEA finds the optimal cover in only 22% of the experiments, while CCEA-MAB succeeds 71% of the time. Fig. 2 exposes a typical run with this configuration. We see that the two algorithms have mostly the same behaviour for the first couple of phases, but when the fourth and fifth species are introduced, alternating between the species in a round-robin fashion does not leave enough generations to the important species to converge to a niche, as seen in Fig. 2(a). The greater number of generations offered to the promising populations by CCEA-MAB in the fourth and fifth phases greatly helps to find an optimal coverage (Fig. 2(b)).

The experiments with scenario 3 show a success rate of 83% for CCEA and 88% for CCEA-MAB. While this appears to be contradictory with the previous experiment, we find that the large amount of time allocated between the introduction of new species (100 generations) greatly helps CCEA while not affecting the MAB version much. In fact, just reducing the number of generations allocated per species introduction to 75 reduces the number of perfect coverages found by CCEA to 10% and those found by CCEA-MAB to 33%, conserving its superiority observed previously.

Finding the Appropriate Number of Species

In this problem, the coevolutionary algorithm aims not only at evolving the perfect cover for a given target set, but also strives at finding the optimal number of species for that problem. For this test, we expect the MAB version of the coevolutionary algorithm to perform much better, with less species removed of their inability to identify a suitable niche in the allocated time. This test is the closest one to a real world problem where we do not know how many species will be required, nor the time we should allocate after the addition of a species for the algorithm to stabilize.

For this experiment, the complete algorithms presented in Sec. 2 and 4 are used. The coevolution must concurrently determine the appropriate number of species, a niche for all those species, and the optimal coverage for the problem. Again starting with a single species, the cooperative coevolution mechanisms will be used to add and remove species. The evolution is run until the algorithm finds the perfect coverage or 500 generations have passed. We will use the same three scenarios as in the adaptation experiment. Fig. 3 shows the generation at which the perfect coverage was found for scenario 1. The average first hit generation is 111.9 for CCEA and 92.5 for CCEA-MAB. A Wilcoxon signed test reveals that the two algorithms are different with a significance level of 99.9%.

The first hit generation histogram for scenario 2 is shown in Fig. 4. The average convergence time for CCEA, excluding those exceeding the 500 generations allocated, is 315.0 generations, while it is 229.7 generations for CCEA-MAB. A Wilcoxon signed test indicates that the two algorithms are different at a significance level of 99.9%. We also notice that CCEA-MAB does find the optimal coverage on all of the 100 runs, while CCEA does not converge in more than 25% of the experiments. Fig. 5 presents the run with the median first hit time for each algorithm. The contribution of the different species is shown with the continuous lines and the collaboration fitness is presented with the dashed line. This clearly illustrates that CCEA consumes much more species than CCEA-MAB. In fact, it takes 12 species and approximately 350 generations for CCEA to have the right elements to entirely cover the target set, while only 6 species and less than 250 generations are required for CCEA-MAB. This clearly shows that allocating more steps to the promising species allows them to find suitable niches and support global improvements in the optimization, while the original algorithm would have removed those species at the outset.

Finally, the first hit generation histogram for scenario 3 is shown in Fig. 6. The average convergence time for CCEA and CCEA-MAB on this experiment, excluding those exceeding the 500 generations allocated, are respectively 313.8 and 245.6 generations. The runs that have not converged represent more than 50% of the trials for CCEA and only 11% of the runs for CCEA-MAB. Again, the Wilcoxon signed test indicates that the two algorithms are different at a significance level of 99.9% in this configuration. Fig. 7 exposes the same problem as in the previous scenario, showing the runs corresponding to the median first hit time of each algorithm. We see that in CCEA, an enormous amount of evaluations are wasted on species immediately removed after their introduction. However, these species should have helped the algorithm to increase its collaboration fitness. This problem is not present in CCEA-MAB as it allocated enough resources to the promising species.

Sensor Placement

The string covering problem exposed the properties of our new algorithm. This section will illustrate how this algorithm performs on a more complex case. As shown by De Rainville et al. [START_REF] De Rainville | Co-adapting mobile sensor networks to maximize coverage in dynamic environments[END_REF], cooperative coevolutionary algorithms apply well to the sensor placement optimization. We will study the effect of selecting the species to evolve on each generation on a similar scenario.

The sensor placement problem we face is to optimize the position of a group of sensors (i.e. directional cameras) so that an entire section of an environment is seen. The required number of sensors is unknown, thus the adaptation of the number of species in the evolution is essential. The representation used to tackle this problem is a real-valued vector representing the position and orientations of a sensor. Each species will evolve a single sensor and cooperation between species will produce multiple sensor solutions. We use a simulated binary crossover and a Gaussian mutation as variation operators. Selection between the individuals of a species is achieved by a tournament. The visibility of a sensor is computed by the integral of its resolution on the surface it senses. For space reasons, the details on computing a sensor visibility cannot be entirely exposed here.

The collaboration between species is evaluated by computing the complete visibility attained with all representatives. The fitness is the difference between the visibility required for the environment and the visibility given by the group of sensors. Fig. 8 shows the different visibility concepts implied in the sensor placement problem. In this figure, the environment desired view is determined by the resolution an omnidirectional camera would have of the environment if it was positioned on the dot. The visibility of a sensor is computed using the same process, a sensor placed at position "x" and oriented as shown by the arrow sees the thick dashed line. The fitness is the integral of the difference between the required visibility and the visibility of the sensor. Finally, a species contribution is computed as the difference of fitness the group of representatives would have with and without the individual from that species.

Experiments have been conducted for the environment of Fig. 8. The desired field of view is 360 • while the sensors field of view is 45 • , which results in a theoretical optimal solution with 8 sensors. We stopped the evolution for both techniques when a suitable coverage had been achieved, namely an error less than 10 4 . The average first hit generation and average resulting number of sensors, for 10 independent runs, are shown in Tab. 2. A Mann-Whitney-Wilcoxon test reveals that the two algorithms are different with a significance level of 99.9%. This clearly shows that the MAB maintained the focus of the optimization on the important sensors, and thus the algorithm converged more rapidly to a suitable solution. Fig. 9 shows the evolution of the collaboration fitness during the median run for both algorithms. As mentioned in the previous section, when many species are present, the MAB greatly increases the convergence of the algorithm because it allocates resources to the preferred population and offers some stability to the new species. Moreover, tuning the extinction threshold would force the algorithm to reach a better position for each sensor and not use the extra sensors, but it would also require more computational time to identify that solution. In addition, a fixed length real valued genetic algorithm has been run for several experiments with different number of sensors without success at producing any suitable solution even with carefully tuned parameters.

Conclusion

This paper presented an efficient way to automatically control the resources allocated to the cooperative populations in coevolution in order to sustain co-adaptation. Our technique exploits the complete potential of every species and learns when to select another species due to our multiarmed bandit. Our technique also alleviates the burden of precisely choosing the improvement length, as the allocation of the resources to the population is conducted automatically. Finally, we introduced a mechanism that responds to the difficulty of each problem by allocating the computing resources to the subpopulations which will sustain evolution. The string covering benchmark and the sensor placement problem clearly illustrate that our method helps the algorithm to converge more rapidly to a suitable solution in situations where the original algorithm struggles to keep up with the problem. This work can be extended in a number of ways. First, different multi-armed bandits exist and much work has been carried out to solve combinatorial problems with these learners. It would be interesting to incorporate the dynamic technique in these bandits and use them in our algorithm as they do not make the independence assumption between the populations. Second, preliminary results show that the population selection technique can be applied efficiently to other types of coevolution such as competitive coevolution. In this case, the bandit would balance the progress made by both competitors so that one does not entirely dominate the other. Third, the bandit technique would also be promising in multi-population algorithms to invest resources in the most promising areas of the search space.

Figure 1 :

 1 Figure 1: Maximum number of matched bits by a single match string on each schemata in scenario 1 with the number of times each species is evolved.

Figure 2 :

 2 Figure 2: Maximum number of matched bits by a single match string on each schemata in scenario 21 with the number of times each species is evolved.

Figure 3 :

 3 Figure 3: First hit generation histogram for scenario 1.

Figure 4 :

 4 Figure 4: First hit generation histogram for scenario 2.

Figure 5 :

 5 Figure 5: Collaboration (match set fitness) and contribution (fitness of each species independently) for scenario 2.

Figure 6 :

 6 Figure 6: First hit generation histogram for scenario 3. Algorithm First Hit Gen. # Sensors CCEA 429.4 8.8 CCEA-MAB 283.7 8.8

Figure 7 :

 7 Figure 7: Collaboration (match set fitness) and contribution (fitness of each species independently) for scenario 3.

Figure 8 :

 8 Figure 8: The environment used for the real world experiments. The desired view and the sensor view are shown respectively by the thick continuous and thick dashed lines.

Figure 9 :

 9 Figure 9: Fitness progress on the sensor problem.

Table 1 :

 1 Parameters used in the experiments.

	Parameter	CCEA CCEA-MAB
	Species Size	50	50
	Initial Number of Species	1	1
	Crossover Rate	0.6	0.6
	Mutation Rate	1.0	1.0
	Flip Bit Rate	1/64	1/64
	Tournament Size	3	3
	Window Size (W )	-	50
	Decay Factor (d)	-	1.0
	Exploration Factor (C)	-	1.0
	Improvement Length (I)	5	5
	Improvement Threshold (T i )	0.5	0.5
	Extinction Threshold (T c )	5.0	5.0

Table 2 :

 2 Results for the second environment.

Newer species are removed first and the contribution is re-calculated after each removal.

Acknowledgements

This work has been funded by the FRQNT (Québec) and NSERC (Canada). The authors wish to thank Annette Schwerdtfeger for proofreading the manuscript.


Elodie Reveillac
email: elodie.reveillac@gmail.com

Thomas Lacoue-Labarthe

François Oberhänsli

Jean- Louis Teyssie

Ross Jeffree

Jean-Pierre Gattuso

Sophie Martin

Ocean acidification reshapes the otolith-body allometry of growth in juvenile seabream

Keywords: climate change, ocean acidification, otolith calcification, somatic-otolith growth allometry, temperate coastal fish

The effects of elevated CO 2 partial pressure (pCO 2 ) on otolith calcification and on the coupling between the somatic and otolith growth were investigated in juvenile gilthead seabream Sparus aurata. Six-month old individuals were raised during seven weeks under four pCO 2 conditions set according to projected future ocean acidification scenarios. Body and otolith biometric parameters were measured throughout the experiment along with the otolith biomineralization monitored using a radiotracer technique based on 45 Ca incorporation. Seabream exhibited somatic growth resilience to all treatments. In contrast, increased growth rate and shape complexity of otoliths were observed with a pH T drop from 8.1 to 7.5. Hypercalcification was observed under lowered pH, with a rate of calcium incorporation increasing by up to 18% between pH T 8.1 and pH T 7.7. This work highlighted an uncoupling of otolith and body growth of juvenile seabream within 40 d at pH T 7.9 projected to be reached by the end of the century. As the otolith is an essential tool used in reconstructing fish life history, this work suggests that information resulting from otolith studies should be interpreted with caution with respect to the potential impacts that ocean acidification projected modifications could have on otolith biomineralization.

INTRODUCTION

On 9 th May 2013, the concentration of carbon dioxide (CO 2 ) in the atmosphere reached the symbolic threshold of 400 ppm in Mauna Loa, Hawaii (IPCC, 2013), a level never reached at this reference site. The increase of atmospheric CO 2 from a preindustrial value of 280 µatm is the result of fossil fuel combustion, cement production and land use change [START_REF] Ciais | Carbon and other biogeochemical cycles[END_REF].

IPCC projections suggest further increase in the coming decades with concentrations reaching 490 ppm in 2050 and more than 1370 ppm by 2100 (IPCC, 2013). The ocean is a major carbon sink, absorbing about 25% of anthropogenic CO 2 emissions thus limiting the greenhouse gas effects on climate (Le [START_REF] Quéré | The global carbon budget 1959-2011[END_REF]. The increase in pCO 2 in the ocean has already led to a pH decline of 0.1 unit since the industrial revolution and to major shifts in the ocean carbonate chemistry, i.e. increased concentrations of dissolved inorganic carbon and bicarbonate ions, decreased carbonate concentration, altogether leading to 'ocean acidification' [START_REF] Caldeira | Ocean model predictions of chemistry changes from carbon dioxide emissions to the atmosphere and ocean[END_REF]. The pH of the surface ocean is expected to decline by 0.06 to 0.32 units by the end of century [START_REF] Ciais | Carbon and other biogeochemical cycles[END_REF], resulting in an unprecedented change of seawater chemistry equilibrium since the last 800,000 years (Zeebe and Ridgwell, 2012).

For a decade, a growing body of experimental studies has examined the response of marine organisms to decreased pH across multiple taxa. Impacts on early life stages have been of particular concern in fish [START_REF] Baumann | Reduced early life growth and survival in a fish in direct response to increased carbon dioxide[END_REF] because it has been hypothesized that animal embryos and larvae may not be as resilient to physiological stress as juveniles and adults [START_REF] Pörtner | Ecosystem effects of ocean acidification in times of ocean warming: a physiologist's view[END_REF], and because recruitment cohorts lay the foundation for population success and connectivity [START_REF] Planes | Larval dispersal connects fish populations in a network of marine protected areas[END_REF]. Nevertheless juveniles of coastal fish may be more exposed to high CO 2 levels than earlier life stages and deserves particular attention. Indeed, unlike in the open ocean, seawater pCO 2 is known to vary considerably in coastal waters depending on landdriven eutrophication, which adds to the uptake of atmospheric CO 2 , locally amplifying ocean acidification [START_REF] Cai | Acidification of subsurface coastal waters enhanced by eutrophication[END_REF][START_REF] Guinotte | Ocean acidification and its potential effects on marine ecosystems[END_REF]. Juvenile fish settling in coastal areas during their high metabolic and fast growing phase could be severely challenged by hypercapnic conditions.

Based on the sparse current knowledge, adult physiological performance allows fish to cope with extracellular acidosis caused by increasing pCO 2 (e.g. Melzner et al., 2009b). But in early-life stages of multiple taxa including fish, increasing pCO 2 was shown to affect calcification of shells and skeletons due to a drop in the carbonate availability (e.g. [START_REF] Gattuso | CARBON AND CARBONATE METABOLISM IN COASTAL AQUATIC ECOSYSTEMS[END_REF][START_REF] Riebesell | Reduced calcification of marine plankton in response to increased atmospheric CO 2[END_REF]. Munday et al. (2011a) observed no effects on spiny damselfish otolith calcification at 850 atm, while Munday et al. (2011b) and [START_REF] Checkley | Elevated CO 2 enhances otolith growth in young fish[END_REF] highlighted an otolith hypercalcification in white sea bass larvae exposed at 993 and 2558 atm pCO 2 and in clownfish larvae at 1721 atm pCO 2 , respectively. In cases of calcification modulation, otolith morphology can be affected, which may have negative repercussions on the behaviour and acoustic function in fish, decreasing their survival probabilities (Bignami et al., 2013a;[START_REF] Popper | Why otoliths? Insights from inner ear physiology and fisheries biology[END_REF]. It has also been recently reported that otolith increment growth can be uncoupled from somatic growth in fish larva raised under high pCO 2 conditions, then having potential implications for the study of fish populations (Bignami et al. 2013b). Understanding the way ocean acidification can affect the process of otolith formation is decidedly important in many perspectives among which is seems crucial to delineate how the environment can affect its growth and influence its coupling with the structural skeleton growth. If otoliths indeed ensure the fish's biological hearing and balance functions, they are also an essential tool used in fisheries biology to reconstruct individual life history in terms of age and somatic growth relationship (age-length keys) and attended habitats [START_REF] Campana | Microstructure of fish otoliths[END_REF][START_REF] Campana | Otolith science entering the 21st century[END_REF].

This paper aims at evaluating the impacts of ocean acidification on the calcification rate of fish otoliths and on the understudied coupling between otolith and somatic growth in juveniles. These questions were investigated experimentally using a nuclear tracking approach, following the 45 Ca incorporation in otolith for 7 weeks of exposure to 4 realistic pCO 2 levels projected for the nearfuture. The gilthead seabream Sparus aurata was chosen due to its ecology and high economic value. Temperate and widely distributed over the North-Eastern Atlantic Ocean and the Mediterranean Sea, this coastal species is subjected to recreational and professional fishing and is increasingly aquacultured (FAO, 2014). In the Mediterranean Sea, it is the first marine fish species cultured with more than 140,000 tonnes produced in 2010 in 17 countries for a worth ca.

US$ 785 million (GFCM, 2013).

MATERIALS AND METHODS

Organisms, radiotracer and experimental procedures

Six-month old juveniles of seabream Sparus aurata have been purchased at the "Cannes Aquaculture" fish farm of Monaco and were placed for three weeks in an open-circuit 500 l tank in the IAEA-EL premises for acclimation.

Then, 200 fish juveniles of ca. 50 mm were randomly assigned in four 20 l circular tanks (one tank per treatment) filled with filtered (0.45 µm) and UV-sterilized Mediterranean seawater (38 p.s.u.) pumped from 30 m depth in the Bay of Monaco. In each experimental tank (closed system), seawater was constantly aerated. The light/dark cycle was 12h/12h. Fish were fed daily ad libitum with pellets. After feeding, not ingested food has been removed and 80% of the volume was renewed daily with sterilized and filtered seawater. Temperature was maintained at 21°C and controlled in each bath to within ± 0.5°C using temperature controllers connected to 300 W submersible heaters. Seawater pH (pH T on the total scale, [START_REF] Dickson | Guide to best practices for ocean CO 2 measurements[END_REF] was adjusted to the desired level from ambient pH T of 8.1 (corresponding to 475 µatm of pCO 2 ) to low pH T of 7.9, 7.7, and 7. 5 (700, 1200 and 2000 µatm, respectively), as derived from various models on trajectories of carbon emissions to the near-future (IPCC, 2013). The pH was controlled in each bottle to within ± 0.05 pH unit using a pH-stat system (IKS, Karlsbad). The experimental containers were continuously bubbled with CO 2 -free air and discrete amounts of pure CO 2 were added by the pH-stat system. pH and alkalinity were measured and set according to [START_REF] Martin | Early development and molecular plasticity in the Mediterranean sea urchin Paracentrotus lividus exposed to CO 2driven acidification[END_REF] using the R package seacarb [START_REF] Proye | Seacarb, an R package to calculate parameters of the seawater carbonate system. R Core Team[END_REF].

Seawater in each bottle was spiked with 45 Ca (10 kBq l -1 ). Radiotracers were purchased from Radioisotope Centre Polatum, Poland, 45 Ca [as 45 CaCl 2 ; T ½ = 163 d]. Stock solutions were prepared in H 2 O to obtain radioactivities that allowed the use of spikes of only a few microliters (typically 5-10 μl). The radiotracer spikes were renewed at each water change and were checked (i.e. counted in 1 ml of seawater) before and after each water renewal in order to maintain radiotracer concentrations.

Fish were maintained for 40 d at the four pH treatments and continuously exposed to dissolved 45 Ca in seawater. Five fishes were collected every 3 or 7 d in each tank, anesthetized with clove oil prior to decerebration and then measured in length and weight to the nearest 0.1 mm and 1 mg, respectively. Left and right otoliths (sagittae) were extracted and photographed using a Leica DFC420 camera mounted on a stereomicroscope (Leica LZ12). Otolith surface area (OSA, mm 2 ), maximum Feret diameter (OF, mm) and roundness (OR) were calculated using ImageJ software and weight measurement was made to the nearest 0.1 mg. 45 Ca content was then determined. Paired radiolabelled sagittal otoliths from the same individual were pooled and dissolved adding 300 μL of hydrochloric acid (HCl, 37%) at 80°C. After evaporation, the residues were dissolved in 1 mL of distilled water. Biological and seawater samples were counted after adding 10 mL of scintillation liquid, Ultima GoldTM XR (Perkin Elmer).

Emissions were measured with a liquid scintillation analyzer (Tri-Carb, Packarb 1600 TR or Perkin Elmer 2900 TR) calibrated with an appropriate standard for each counting that was used.

Counting times were adapted to obtain relative propagated errors less than 5% (from 10 min to 24 h). Corrections for the physical half-life time and background noise were done in order to determine the 45 Ca activities at the sampling time (Bq). The uptake of Ca in the otolith was then expressed as the amount of Ca incorporated (Q Ca , in μmol Ca g -1 of otolith dry wt) and following the equation [START_REF] Martin | Early development and molecular plasticity in the Mediterranean sea urchin Paracentrotus lividus exposed to CO 2driven acidification[END_REF]:

Q Ca = [(A cut / A sw ) x C sw ] x 10 3
where A cut is the total 45 Ca activity in each otolith (in Bq), A sw is the time-integrated activity (in Bq g -1 ) in seawater during the time of exposure and C sw is the total Ca concentration in Mediterranean seawater (0.0114 mmol g -1 ).

Data Analyses

Due to compromises with experiment cost, radioprotection rules and waste management possibilities with respect to the use of radioisotope 45 Ca, only one 20-l tank has been dedicated to each pH conditions. Therefore, fish sampled along the experiment course at regular sampling time (n = 5 (and 10) per sampling time (at 40 d) per condition) have been considered as pseudoreplicates [START_REF] Hurlbert | Pseudoreplication and the design of ecological field experiments[END_REF]. Accordingly, the effect of pH on measured physiological or morphological responses of fish has been tested through multiple linear regression considering the pH as a continuous covariate (Havenhand et al., 2011). Linear regressions were used to test the effects of pH (continuous covariate), on relationships between otolith biometrics (i.e. otolith surface area (OSA), weight (OW), maximum diameter (Feret's diameter, OF), roundness (OR)) and fish total length (FTL) or Time, including a FTL (or Time) x pH interactions. When FTL (or Time) x pH interaction was not significant, a simpler model without interaction (i.e. Time (or FTL) + pH) has been computed [START_REF] Crawley | Statistics: an introduction using R[END_REF].

To evaluate the effect of the pH treatment on the asymmetry of paired-otoliths (regarding leftright differences in OSA, OF and OR), log-transformed data were injected in a Brown-Forsythe robust Levene test of homogeneity of variances based on deviation from the median.

The amount of Ca incorporated in otoliths were standardized to the otolith weight and expressed according to the Time of experiment. Ca incorporation data (mmol g -1 otolith dry weight) have been fitted by multiple linear regression with Time x pH as continuous covariates.

Results are expressed as mean ± s.d. Significance was considered at p<0.005. All statistical treatments have been done using R (R Core Team, 2013).

RESULTS

Culture conditions

During the experiment, the fish biomass in each tank did not exceed 5 g.l -1 , i.e. far below the EU recommendations for fish welfare (CEE n°2092/91) which limit the biomass to 25 g.l -1 for fish aquaculture in marine environment. Along the 40 d of 45 Ca exposure, pH T was maintained at a mean (± s.d.) of 7.50 ± 0.06, 7.69 ± 0.03, 7.89 ± 0.05 and 8.04 ± 0.05 (Table 1), corresponding to pCO 2 of ca. 2000, 1200, 700, and 475 µatm, respectively. Mean temperature was 21.0°C ± 0.7.

Mean A T of renewed seawater was 2.595 ± 0.003 mmol.kg -1 . It changed by a maximum of 0.2 mmol kg -1 between two seawater renewals. C T increased from 2410 to 2690 µmol kg -1 whereas the CO 3 2-concentration decreased from 240 to 80 µmol kg -1 with decreasing pH. In all conditions, the saturation state with respect to aragonite was higher than 1.

Somatic versus otolith growth

Fish somatic growth in total length and weight was significant over the 40 days period of the experiment (p-value < 0.001, Table 2), slightly inflecting from day 30 (Figure 1, A,B). As a mean, fish grew by 0.36 mm.d -1 and 0.043 g.d -1 , a growth rate close to the 0.0534 g.d -1 value measured in juveniles seabream (1.24 ± 0.02 g) reared in closed system tank at a density of 1.5

g.l -1 [START_REF] Kalogeropoulos | Effects of dietary soybean and codliver oil levels on growth and body composition of gilthead bream (Sparus aurata)[END_REF]. Similarly [START_REF] Kim | Comparison of growth performance and whole-body amino acid composition in red seabream (Pagrus major) fed free or dipeptide form of phenylalanine[END_REF] reported a mean weight growth rate of 0.0364 g.d -1 for young juveniles of red seabream reared from 1.46 g for 42 days, arguing for adequate rearing conditions in our experiment allowing a normal growth and development of the juveniles (Figure 2). The pH treatments had no impact on those body growth parameters and intercepts did not differ between treatments meaning the initial group of juveniles was homogeneous (p-value > 0.05). In contrast, the allometric relationship between otolith surface area and fish total length (Figure 3.A), though still linear, was altered by the decreasing pH (pvalue < 0.001, Table 2) with a lower slope value observed at pH T = 8.1 compared to pH T = 7.9 and 7.7, and even more compared to pH T = 7.5 (Figure 3.A). The otolith's Feret diameter and fish total length relationship tended also to be modified in the two lowest pH conditions though not significantly (Figure 3.B, Table 2). The otolith roundness was not related to fish total length except at pH T 7.5 where the linear relationship was low but significant (Figure 3.C, p-value < 0.0046) indicating a faster evolution of the otolith toward a less round shape along with body growth at the lowest pH than at more elevated ones. Finally, the otolith weight was linearly correlated to the otolith surface area during all the experiment and whatever the pH treatment (Table 2), indicating that the density of the otolith was not affected by pCO 2 .

Asymmetry

Levels of both surface area and roundness asymmetry between left and right sagittal otoliths

Calcification

The calcification of otolith was followed through 45 Ca accumulation throughout the 40 days of experiment (Figure 5). The incorporation of Ca in otolith of juvenile seabreams followed a linear equation whatever the pH (Table 3), which is consistent with the linear growth of fish over the experiment. Based on the slope calculated for each pH, we could estimate that for the fish reared at pH T 8.1 rates of Ca incorporation were 13, 17 and 18% lower than rates performed at pH T 7.9, 7.7 and 7.5, respectively.

DISCUSSION

Assessing whether elevated pCO 2 and lowered pH impact marine fauna and flora and to what extent it can affect individual, population and ecosystem function is a major concern (Fabry et al., 2008;[START_REF] Hilmi | Towards improved socio-economic assessments of ocean acidification's impacts[END_REF]Melzner et al., 2009a). Recent studies have reported that hypercapnia can challenge fish physiology (e.g. Melzner et al., 2009b;[START_REF] Michaelidis | Effects of long-term acclimation to environmental hypercapnia on extracellular acid-base status and metabolic capacity in Mediterranean fish Sparus aurata[END_REF], affect the behaviour [START_REF] Simpson | Ocean acidification erodes crucial auditory behaviour in a marine fish[END_REF] and growth [START_REF] Baumann | Reduced early life growth and survival in a fish in direct response to increased carbon dioxide[END_REF] and impact calcified structures biomineralized from the early-life stages [START_REF] Checkley | Elevated CO 2 enhances otolith growth in young fish[END_REF][START_REF] Ries | Marine calcifiers exhibit mixed responses to CO 2 -induced ocean acidification[END_REF]). Our study shows that responses can be contrasted at the sub-individual integration level.

Effect of elevated pCO 2 on the coupling of fish somatic and otolith growth

Juvenile seabream Sparus aurata reared under 4 pCO 2 treatments during 40 d exhibited no variation in somatic growth. As documented, physiological performance of fish could be expected to allow them to cope with elevated pCO 2 (Melzner et al., 2009b) without significantly affecting the animal's growth [START_REF] Hurst | Resiliency of juvenile walleye pollock to projected levels of ocean acidification[END_REF][START_REF] Ishimatsu | Fishes in high-CO 2 , acidified oceans[END_REF]. [START_REF] Michaelidis | Effects of long-term acclimation to environmental hypercapnia on extracellular acid-base status and metabolic capacity in Mediterranean fish Sparus aurata[END_REF] demonstrated that long-term hypercapnia may cause a shift from aerobic to anaerobic metabolism in seabream Sparus aurata, but only at dramatically elevated pCO 2 (~5000 atm).

This suggests that this species is able to cope with realistic increasing pCO 2 , according to IPCC scenarios, without extra-energy demand and might thus maintain its juvenile somatic growth rate as observed in this study.

In contrast, a modulation of the otolith biomineralization was observed, with an increase of the calcification rate observed from 700 atm after 40 days of exposure. This means that juvenile seabream might be subjected to the hypercalcification of their otoliths by the end of the century with respect to IPCC projected CO 2 levels. Only Bignami et al. (2013b) reported such a shortterm effect of increased pCO 2 under the threshold of 800 atm, and this was at the larval stage of cobia Rachycentron canadum. Here, the higher growth rate of the otolith surface area under rising pCO 2 was directly linked with the increase of the net calcification rate calculated on the basis of 45 Ca incorporation. The linearity of calcium incorporation with time was not disrupted by pCO 2 treatments, but it did provide evidence of a critical difference in the mineralisation slopes. The hypermineralization of calcareous structures in organisms maintained in decreasing pH and in less favourable chemical condition for CaCO 3 precipitation (lower carbonate saturation state, Ω arg ) has been already observed in species of various taxonomic groups characterized by efficient capacities to maintain blood/hemolymph homeostasis, such as crustaceans [START_REF] Ries | Marine calcifiers exhibit mixed responses to CO 2 -induced ocean acidification[END_REF], cephalopods [START_REF] Dorey | Ocean acidification and temperature rise: effects on calcification during early development of the cuttlefish Sepia officinalis[END_REF][START_REF] Gutowska | Cuttlebone calcification increases during exposure to elevated seawater pCO 2 in the cephalopod Sepia officinalis[END_REF] and fishes (Bignami et al., 2013b;[START_REF] Checkley | Elevated CO 2 enhances otolith growth in young fish[END_REF]. Indeed, the active uptake of bicarbonate ions in internal medium to compensate blood acidosis is expected to contribute at increasing Ω arg in the endolymph fluid surrounding the aragonitic otolith [START_REF] Borelli | Daily variations of endolymph composition: relationship with the otolith calcification process in trout[END_REF]. The precipitation of HCO 3 -with Ca 2+ in endolymph leads in turn to active proton extrusion to avoid impeding the calcification process [START_REF] Allemand | Fish Otolith Calcification in Relation to Endolymph Chemistry[END_REF]. Nonetheless, the cellular mechanisms underlying the hypercalcification of otoliths in this context of ocean acidification remains poorly understood (Munday et al., 2011b) and require further insights.

The observed uncoupling of fish body and otolith growth may have implications in fisheries studies. Indeed, back-calculation of fish size from the otolith size is a common and quite highly effective tool used in fisheries biology [START_REF] Campana | Microstructure of fish otoliths[END_REF][START_REF] Mosegaard | Uncoupling of Somatic and Otolith Growth Rates in Arctic Char (Salvelinus alpinus) as an Effect of Differences in Temperature Response[END_REF][START_REF] Payan | Ionic composition of endolymph in Teleosts: origin and importance of endolymph alkalinity[END_REF]. It can be used to determine fish-prey size spectra from otoliths collected in the stomach contents of predators [START_REF] Blanco | Diet of the bottlenose dolphin (Tursiops truncatus) in the western Mediterranean Sea[END_REF][START_REF] Markaida | Food and feeding habits of jumbo squid Dosidicus gigas (Cephalopoda: Ommastrephidae) from the Gulf of California, Mexico[END_REF][START_REF] Ross | Use of fish-otolith-length regressions to infer size of double-crested cormorant prey fish from recovered otoliths in Lake Ontario[END_REF]. It is also commonly used to infer fish-size at age, based on otolith growthincrements reading [START_REF] Secor | Somatic growth effects on the otolith-fish size relationship in young pond-reared striped bass, Morone saxatilis[END_REF]). However to be reliable, body and otolith growth have to be uniquely and confidently linked throughout the developmental stage or time period of interest. A body of studies has shown how complex this relationship can be (reviewed in [START_REF] Wilson | The back-calculation of size and growth from otoliths: validation and comparison of models at an individual level[END_REF], being potentially impacted by environmental parameters fluctuations such as temperature [START_REF] Mosegaard | Uncoupling of Somatic and Otolith Growth Rates in Arctic Char (Salvelinus alpinus) as an Effect of Differences in Temperature Response[END_REF][START_REF] Takasuka | Growth effect on the otolith and somatic size relationship in Japanese anchovy and sardine larvae[END_REF]. To date, "growth-effect" and "ageeffect" were the two major explanations of the uncoupling between somatic and otolith growth [START_REF] Wilson | The back-calculation of size and growth from otoliths: validation and comparison of models at an individual level[END_REF]. This work presents the perspective that the distortion of the fish-otolith allometric relationship can also originate from the modulation of the otolith growth induced by variations of environmental parameters to which body growth is (more) resilient. Ocean acidification, as a compound of the global change, appears here to alter this relationship.

Effect of increased pCO 2 on otolith shape

Ocean acidification has also been reported to affect other otolith metrics that were monitored during this experiment. As otolith weight increased in the same way as the otolith surface area, no effect of pCO 2 increase up to 2000 atm was observed on seabream otolith density. These results are consistent with Bignami et al. (2013b) who did not observe any density modification of the otolith in cobia larvae until 800 atm. Nevertheless, they reported an increase of 6% density when they experimentally increased conditions to 2100 atm, suggesting an alteration of the otolith's mass-size relationship under extreme hypercapnia conditions. Seabream otolith shape seems to be also sensitive to pH conditions. Sagitta of juveniles reared at 475 atm where indeed significantly shapeless compared to those mineralized at 2000 atm. As no asymmetry between left and right otoliths was recorded, no sign of "anomalous" calcification appeared, being in agreement with [START_REF] Maneja | Effects of ocean acidification on the calcification of otoliths of larval Atlantic cod Gadus morhua[END_REF] who did not observe a deviation from symmetry in otoliths of Atlantic cod larvae raised at 1800 atm and even at 4200 atm. Then, the hereobserved lower otolith roundness with high pCO 2 seems to provide evidence that in response to hypercapnia, the otolith not only grows, but also develops more rapidly, acquiring at a higher rate its species-specific ornaments [START_REF] Campana | Stock Discrimination Using Otolith Shape Analysis[END_REF][START_REF] Popper | Why otoliths? Insights from inner ear physiology and fisheries biology[END_REF].

All these otolith descriptors are important in the otolithic apparatus functioning, as the main organ of the ear in the senses of balance, directed motion detection and hearing [START_REF] Lychakov | Otolith regularities[END_REF][START_REF] Popper | Why otoliths? Insights from inner ear physiology and fisheries biology[END_REF]. The otolith features indeed determine the compromise made between these three senses. In bottom-dwelling species such as Sparus aurata, otoliths tend to be bigger relative to the body size than in pelagic species [START_REF] Popper | Why otoliths? Insights from inner ear physiology and fisheries biology[END_REF], improving balance and hearing senses while impairing motion detection. Hearing in seabream has been shown to be a developed sense that is critical for the welfare of juveniles [START_REF] Filiciotto | Effect of acoustic environment on gilthead seabream (Sparus aurata): Sea and onshore aquaculture background noise[END_REF]. Thus, as reported in another Perciform and bottom-dwelling species (Bignami et al., 2013a), ocean acidification, in increasing the size of the otolith relative to the size of the body, may improve the hearing acuity of S. aurata. As highlighted by Bignami et al. (2013a) improving auditory capacity could also be advantageous or deleterious depending on the sound spectra newly accessible to the fish. As Filiciotto et al. ( 2013) reported, wild offshore sound at low pressure levels are more beneficial to S. aurata with individuals being less stressed and faster growing than individuals exposed to constant and high sound pressure levels in aquaculture tanks. In increasing hearing acuity, ocean acidification could thus induce a threshold shift of sound pressure tolerance [START_REF] Smith | Acoustical stress and hearing sensitivity in fishes: does the linear threshold shift hypothesis hold water?[END_REF] or displace the optimum window of frequency, making the baseline sound of the environment more disturbing and stressful [START_REF] Simpson | Ocean acidification erodes crucial auditory behaviour in a marine fish[END_REF][START_REF] Caiger | Chronic low-intensity noise exposure affects the hearing thresholds of juvenile snapper[END_REF][START_REF] Filiciotto | Effect of acoustic environment on gilthead seabream (Sparus aurata): Sea and onshore aquaculture background noise[END_REF]. As many other ecologically and economically important species fished in the wild and cultured in offshore farms, the consequences of ocean acidification on growth and survival performances related to balance and auditory capacities need to be considered and evaluated.

Potential repercussions of elevated pCO 2 on the otolith as a recorder of the seawater chemistry

Considerations have also to be raised in regard to the use by fisheries scientists of the chemical composition of the otolith as a biological tracking tool [START_REF] Campana | Otolith science entering the 21st century[END_REF][START_REF] Miller | Transition metal binding to cod otolith proteins[END_REF].

Here, calcium incorporation into the otolith was modulated by seawater pH. This questions the stability of the elemental:Ca ratio under environmental hypercapnia. During the biomineralization of the otolith, chemical elements such as metals and metalloids are supposed to substitute for calcium [START_REF] Campana | Chemistry and composition of fish otoliths: pathways, mechanisms and applications[END_REF] or, at least for some transition metals, complex with the organic matrix of the otolith via its constitutive metal-binding proteins [START_REF] Miller | Transition metal binding to cod otolith proteins[END_REF].

The changes of pH and seawater chemistry caused by increased CO 2 can modify the speciation of metals and therefore their bioavailability to organisms [START_REF] Millero | Effect of Ocean Acidification on the Speciation of Metals in Seawater[END_REF]. The physiological response of fish to hypercapnia might in turn stimulate processes to compensate for acidosis based on the key role of ion transporters such as the Na + /H + exchangers [START_REF] Hu | Development in a naturally acidified environment: Na + /H + -exchanger 3-based proton secretion leads to CO 2 tolerance in cephalopod embryos[END_REF] that are hypothesized to be a major accumulation pathway for some cationic elements [START_REF] Webb | Bioaccumulation and distribution of silver in four marine teleosts and two marine elasmobranchs: influence of exposure duration, concentration, and salinity[END_REF]. In this context, ocean acidification may interfere with trace element uptake and body concentrations [START_REF] Lacoue-Labarthe | Effects of increased pCO 2 and temperature on trace element (Ag, Cd and Zn) bioaccumulation in the eggs of the common cuttlefish, Sepia officinalis[END_REF] and therefore could affect microchemical signature recorded in fish otolith. Two previous studies observed that ocean acidification has no effect on alkaline metals Ba-, Mg-, Sr-Ca ratio in the otolith of larval and juvenile fish [START_REF] Hurst | Resiliency of juvenile walleye pollock to projected levels of ocean acidification[END_REF]Munday et al., 2011a). In contrast, in statoliths of squid larvae, the activity of the transition metal 65 Zn has been shown to increase with lowering pH [START_REF] Lacoue-Labarthe | Effects of ocean acidification on trace element accumulation in the early-life stages of squid Loligo vulgaris[END_REF]. Future research thus seems still needed to investigate the possibility that ocean acidification could impact the incorporation of other trace elements used to track movements of marine organisms [START_REF] Arkhipkin | Spatial and temporal variation in elemental signatures of statoliths from the Patagonian longfin squid (Loligo gahi)[END_REF][START_REF] Campana | Otolith elemental fingerprints as biological tracers of fish stocks[END_REF], depending on their chemical properties, molecular-binding affinities and incorporation pathway into the otolith.

Conclusions

In conclusion, this study demonstrates that, even under projected near-future pCO 2 levels, juvenile seabream exhibited an increase of their otolith calcification and development rates while their body growth rate was not affected. Highlighting an uncoupling of otolith and body growth rates which appeared within 40 d at a pCO 2 of 700 µatm projected to be reached by the end of the century, this study shows that juvenile seabream could be more resilient to the ongoing ocean acidification in terms of somatic growth than in terms of structural calcification. As the otolith is an essential tool used in reconstructing fish life history in terms of age, somatic growth and attended habitats, this work suggests that information resulting from otolith studies should be interpreted with caution with respect to the potential impacts that ocean acidification projected modifications could have on otolith biomineralization. individually in this study (all 4 pH conditions merged) and means (±sd for weight) reported by 577 [START_REF] Mihelakakis | Optimization of feeding rate for hatcheryproduced juvenile gilthead sea bream Sparus aurata[END_REF] and [START_REF] Kopecka-Pilarczyk | Biochemical response in gilthead seabream (Sparus aurata) to in vivo exposure to pyrene and fluorine[END_REF]. 578 5, 7.7, 7.9, and 8.1 (1 tank per treatment, see Table 2 for statistical significance). An otolith roundness value of 1 means a circular shape. For OR, only data recorded at pH 7.5 were significantly linearly distributed (full black line, p = 0.0046). -1 ± sd) in sagittal otolith of juvenile seabreams reared at 4 pH treatments 7.5, 7.7, 7.9, and 8.1 (1 tank per treatment) during 40 d. Offset in the x-axis was used to display the data per pH treatment. See Table 3 for data and statistical comparisons. Rates of Ca incorporation calculated on pseudoreplicates at pH T 8.1 are 18, 27 and 25% lower than rates at pH T 7.9, 7.7 and 7.5, respectively.

TABLES

(

  Figure 4) of juvenile seabream were not affected by pH treatments over the 40 days (Brown-Forsythe robust Levene Test based on deviation from the median; p > 0.05). Disequilibrium of the symmetry in the otolith surface area toward a greater OSA of the right otolith was observed during the first half of the experiment whatever the pH (Figure 4.A.), but disappeared in animals sampled during the last 10 days of experiment. No asymmetry of the otolith roundness was observed (Figure 4.B.).

FIGURES 570 571 Figure 1 .Figure 2 .

 57112 FIGURES 570

Figure 3 .

 3 Figure3. Sparus aurata. (A) Allometric relationship between the otolith surface area (OSA, mm 2 ) and the fish total length (FTL, mm), (B) Otolith Feret maximum diameter (OF) and (C) otolith roundness (OR) measured in juvenile seabreams raised during 40 d at four different pH T treatments 7.5, 7.7, 7.9, and 8.1 (1 tank per treatment, see Table2for statistical significance). An otolith roundness value of 1 means a circular shape. For OR, only data recorded at pH 7.5 were significantly linearly distributed (full black line, p = 0.0046).

Figure 4 .

 4 Figure 4. Sparus aurata. Asymmetry ([left -right]/mean left,right ) in (A) otolith surface area (OSA) and (B) otolith roundness (OR) between left and right sagitta of juveniles reared at 4 pH treatments 7.5, 7.7, 7.9, and 8.1 (1 tank per treatment) during 40 d. Dashed-line represents symmetry; offset in the x-axis was used to display the data per pH treatment.

Figure 5 .

 5 Figure 5. Sparus aurata. Incorporation of calcium (mmol.g otolith dry weight-1 ± sd) in sagittal otolith of juvenile seabreams reared at 4 pH treatments 7.5, 7.7, 7.9, and 8.1 (1 tank per treatment) during 40 d. Offset in the x-axis was used to display the data per pH treatment. See Table3for data and statistical comparisons. Rates of Ca incorporation calculated on pseudoreplicates at pH T 8.1 are 18, 27 and 25% lower than rates at pH T 7.9, 7.7 and 7.5, respectively.

Table 1 .

 1 Sparus aurata. Carbonate chemistry and pH (mean ± sd) in the four pH levels applied to reared juvenile of seabream for 40 d (1 tank per treatment). Partial pressure of CO 2 (pCO 2 ; atm),

	dissolved inorganic carbon (DIC; µmol.kg -1 ), CO 3 2-concentration (µmol.kg -1 ) and saturation state of seawater with
	respect to aragonite (Ω arg ) are calculated from pH T , temperature (21.0°C), salinity (38) and the total alkalinity
	varying with time between two water renewals according the following equation: TA (µmol. kg -1 ) = 2599 + 196 x
	Time (day). Data are means ± s.d. of measurements taken every 15 min from Day 0 to Day 40; N = 24,724.
	Treatment	pH T	pCO 2	DIC	CO 3 2-	Ω arg
	pH 8.1	8.04 ± 0.05	477 ± 78	2412 ± 341	238 ± 22	3.65 ± 0.34
	pH 7.9	7.89 ± 0.05	726 ± 101	2506 ± 247	178 ± 15	2.73 ± 0.23
	pH 7.7	7.69 ± 0.03	1198 ± 105	2603 ± 153	121 ± 8	1.85 ± 0.13
	pH 7.5	7.50 ± 0.06	1978 ± 202	2685 ± 244	80 ± 13	1.22 ± 0.19

Table 2 .

 2 Sparus aurata. Parameters of the multiple linear regressions between fish total length (FTL, mm) and Time (days), fish total weight (FTW, g) and Time (days) otolith surface area (OSA, mm 2 ) and FTL, otolith weight (OW) and OSA, otolith Feret maximum diameter (OF,

	mm) and FTL, and pH considered as a continuous co-variable, measured over the 40 days period
	of experiment in 4 tanks maintained at pH 7.5, 7.7, 7.9 and 8.1. Significance of p-values NS >
	0.05; * < 0.05; ** < 0.01; *** <0.001.
	Model	Parameter	Variable p-value	Model F-value	Model p-value
	FTL vs. Time	Time	< 0.001***	117.1	< 0.001***
		pH	0.654 NS
	FTW vs. Time	Time	< 0.001***	85.8	< 0.001***
		pH	0.372
	OSA vs. FTL	FTL	0.002**	246.9	< 0.001***
		pH	0.046*
		FTL x pH	0.021*
	OW vs. OSA	OSA	< 0.001***	737.3	< 0.001***
		pH	0.772 NS
	OF vs. FTL	FTL	0.002**	280.2	< 0.001***
		pH	0.636 NS
	Model degree of freedom: 3 and 186

Acknowledgments

We thank the staff of the hatchery "Cannes Aquaculture" in Monaco for providing seabream juveniles. The IAEA is grateful for the support provided to its Environment Laboratories by the Government of the Principality of Monaco. This work is a contribution to the "European Project on Ocean Acidification" (EPOCA), which received funding from the European Community (grant agreement 211384).