Alex Arenas

Juan Camacho

José A Cuesta

Rubén Requejo

The Joker effect: cooperation driven by destructive agents

Keywords: public goods, cooperation, destructive agents, cycles

Understanding the emergence of cooperation is a central issue in evolutionary game theory. The hardest setup for the attainment of cooperation in a population of individuals is the Public Goods game in which cooperative agents generate a common good at their own expenses, while defectors "free-ride" this good. Eventually this causes the exhaustion of the good, a situation which is bad for everybody. Previous results have shown that introducing reputation, allowing for volunteer participation, punishing defectors, rewarding cooperators or structuring agents, can enhance cooperation. Here we present a model which shows how the introduction of rare, malicious agents -that we term jokers-performing just destructive actions on the other agents induce bursts of cooperation. The appearance of jokers promotes a rock-paper-scissors dynamics, where jokers outbeat defectors and cooperators outperform jokers, which are subsequently invaded by defectors. Thus, paradoxically, the existence of destructive agents acting indiscriminately promotes cooperation.

Introduction

In the recent Hollywood movie The Dark Knight (2008) the comic character known as the Joker jeopardizes a whole society spreading chaos and destruction with no aim of benefit at it. The situation is so critical that even the mob is willing to cooperate with honest people to stop this nonsensical catastrophe. This fiction provides a visual metaphor of how an event like this can force exploiters of society to collaborate temporarily to fight the common enemy. Society is an emergent structure resulting from the cooperation among its members, and exploiters need society to survive, even if they do not contribute to it. Thus they are specially sensitive to the destruction of society precisely because, being selfish agents, society is their only source of survival. The appearance of the Joker provides a strong incentive for cooperation.

Beside situations like the one depicted by the Joker metaphor, the importance of the inclusion of malicious agents on the game is also illustrated in other scenarios. Here are a few examples. Temporary coalitions of rival parties are constantly formed whenever a common enemy arises, only to restore their old rivalry once this enemy has been wiped out. During the Second World War U.S.A. and U.S.S.R. were allied in fighting Hitler, but they got engaged in the Cold War for decades after the danger of Nazism had been ruled out. It is also well known that strong affective links between humans are created when they face a common difficult situation. Biology is another source of potential examples. For instance, it has been shown that the perception of an increase in the risk of predation can induce cooperative behavior in some bird species [START_REF] Krams | The increased risk of predation enhances cooperation[END_REF]. Indeed, prey species frequently form groups to increase the survival rate against predator attacks [START_REF] Hamilton | Geometry for the selfish herd[END_REF][START_REF] Krebs | An introduction to behavioural ecology[END_REF]. In some cases, this has been proven to happen even in the absence of kinship among its members, as in the collective defense of spiny lobsters [START_REF] Lavalli | Collective defense by spiny lobster (panulirus argus) against triggerfish (balistes capriscus): effects of number of attackers and defenders[END_REF].

The existence of these temporary coalitions for defense against a common danger in rational and irrational agents alike calls for an evolutionary explanation. In this article we propose a stylized evolutionary game [START_REF] Hofbauer | Evolutionary Games and Population Dynamics[END_REF] aimed at studying theoretically this enhancement of cooperation driven by the emergence of purely destructive agents. The game does not try to model any specific situation, but it proposes an abstract setting in which the role of the indiscriminate destructive action of these agents in enhancing cooperation is made clear. Our model is a modification of the standard Public Goods (PG) game [START_REF] Groves | Optimal allocation of public goods: A solution to the 'free rider' problem[END_REF], the n-players version of Prisoner's Dilemma and a paradigm of the risk of exploitation faced by cooperative behavior [START_REF] Hardin | The tragedy of the commons[END_REF]. It has been shown that several mechanisms involving reputation [START_REF] Milinski | Reputation helps solve the tragedy of the commons[END_REF], allowing for volunteer participation (Hauert et al., 2002a,b), punishing defectors [START_REF] Fehr | A theory of fairness, competition, and cooperation[END_REF][START_REF] Fehr | Cooperation and punishment in public goods experiments[END_REF], rewarding cooperators [START_REF] Sigmund | Reward and punishment[END_REF] or structuring agents [START_REF] Szabó | Phase transitions and volunteering in spatial public goods games[END_REF][START_REF] Wakano | Spatial dynamics of ecological public goods[END_REF][START_REF] Hauert | Ecological public goods games: cooperation and bifurcation[END_REF], can enhance cooperation. Here, we present a different mechanism for the enhancement of cooperation based on the existence of evil agents. The game involves n players who belong to one out of three different types: cooperators, who contribute to the public good at a cost for themselves; defectors, who free-ride the public good at no expense; and jokers, who do not participate in the public good -hence obtain no benefit whatsoever-and only inflict dam-age to the public good. Groups are formed randomly, and each player's strategy is established before the group is selected. Hence, players have no memory. Remarkably, the appearance of jokers promotes a rock-paper-scissors dynamics, where jokers outbeat defectors and cooperators outperform jokers, which are subsequently invaded by defectors. In contrast to previous models (Hauert et al., 2002a,b), the cycles induced by jokers are limit cycles, i.e. attractors of the dynamics, and exist in the presence of mutations; these properties make them robust evolutionary outcomes. Therefore, paradoxically, the existence of destructive agents acting indiscriminately promotes cooperation.

The paper is organized as follows. Section 2 exposes the model and shows the existence of cycles. Section 3 analyzes the dynamics for infinite populations, and section 4 compares the joker model with other RPS dynamics.

A Public Good game with jokers: existence of limit cycles

The PG game works as usual: every cooperator yields a benefit b = rc (r > 1) to be shared by cooperators and defectors alike, at a cost c for herself (this cost can be set to c = 1 without loss of generality: all other payoffs are given in units of c), and defectors produce no benefit at all but get their share of the public good. As for the new agents (jokers), every joker inflicts a damage -d < 0 to be shared equally by all non-jokers and gets no benefit. In a given game 0 ≤ m ≤ n denotes the number of cooperators, 0 ≤ j ≤ n the number of jokers, and nmj ≥ 0 the number of defectors; S = nj expresses the number of non-jokers. In this group, the payoff of a defector will be Π D (m, j) = (rmd j)/S , and that of a cooperator Π C = Π D -1. Then, in each group, defectors will always do better than cooperators. Jokers' payoff is always 0.

A usual requirement of PG games is that r < n. Without this requirement the solution in which all n players are defectors is no longer a Nash equilibrium -hence the dilemma goes away. As shown later, the evolutionary dynamics for infinite populations yields the same constraint, i.e., if r < n the dynamics asymptotically approaches the tragedy of the commons. However this is no longer true for finite populations, where the upper bound of r for which the tragedy of the commons takes place grows as M, the population size, decreases. In this case the tragedy of the commons arises whenever r < r max = n(M -1)/(Mn) (see Appendix A; notice in passing that for a population of M = n individuals, the evolutionary dynamics yields a tragedy of the commons for every r > 1).

An invasion analysis provides the clue as to why a rockpaper-scissors (RPS) cycle is to be expected when jokers intervene in the game. We shall assume that we have a population of M players of the same type and will consider putative mutations of one individual to any of the other two types. The mutation will thrive if the average payoff of the mutant after many interactions overcomes the average payoff of a non-mutant player. The result of this analysis (see Appendix A) is summarized in Fig. 1, which represents the three different patterns of invasion that can be observed within the region of interest 1 < r < r max , d > 0: III. Joker invasion: If r < 1+d/(M-1) jokers will invade any homogeneous population, so a homogeneous population of jokers is the only stable solution. Notice that this region disappears for large populations (M → ∞) because r > 1.

The RPS cycle C→D→J→C occurring in region I is the essence of the Joker effect.

Infinite populations

We can gain further insight into this effect by studying a replicator-mutator dynamics [START_REF] Smith | Evolution and the Theory of Games[END_REF]. We as-sume a very large population in which the three types are present at time t in fractions x (cooperators), y (defectors), and z = 1xz (jokers). Agents interact with the whole population by engaging in the above described game within groups of n randomly chosen individuals [START_REF] Hauert | Synergy and discounting of cooperation in social dilemmas[END_REF]. Average payoffs of a cooperator, a defector, and a joker are denoted P C (x, z), P D (x, z), and P J (x, z), respectively. Assuming individuals of a given type mutate to any other type at a rate μ 1, the replicator-mutator equations for this system will be

ẋ = x(P C -P) + μ(1 -3x), ẏ = y(P D -P) + μ(1 -3y), ż = z(P J -P) + μ(1 -3z), ( 1
)
where P = xP C + yP D + zP J is the mean payoff of the population at a given time. Explicit expressions for P C , P D , and P J can be obtained by averaging over all samples of groups of n players extracted from a population containing Mx cooperators, My defectors, and Mz jokers, in the limit of very large populations (M → ∞); the derivation can be found in Appendix B. Let us recall that the parameters of the game in the infinite population limit satisfy 1 < r < n and d > 0; the first condition enforces the public goods dilemma, and the second one implies that jokers beat defectors in the absence of cooperators, because defectors receive the damage inflicted by jokers thus obtaining a negative payoff.

The stability analysis of the dynamical system (1) recovers the picture displayed in Fig. 1 (taking M → ∞). When r < 1 + (n -1)d the system is in region II. The only stable equilibrium is a population of only jokers and any trajectory of (1) is asymptotically attracted to it. Thus, in this region the destructive power of jokers is high enough to wipe out the populations of both cooperators and defectors. But the most interesting situation takes place when

r > 1 + (n -1)d, (2)
i.e., in region I. In the absence of mutations the dynamical system (1) has three saddle points at the corners of the simplex as well as an unstable mixed equilibrium (see Appendix C). As a consequence, the attractor of the system is the heteroclinic orbit C → D → J → C. The period is infinite because the system delays more and more around the saddle points. When mutations occur the corners of the simplex are no longer equilibria, and one is left with the interior fixed point, which for small mutations is a repeller (see Appendix C). Since trajectories are confined within the closed region of the simplex, they are attracted to a stable limit cycle for any r > 1 (a direct consequence of the Poincaré-Bendixon theorem [START_REF] Simmons | Differential Equations: Theory, Technique and Practice[END_REF]), as shown in Fig. 2. The size of the cycle depends on the parameter values. It grows as d increases -i.e., when jokers play a more important role (Fig. 3)-and as the mutation rate decreases (Fig. 4). For both, large values of d [compatible with condition (2)] and very small mutations, the cycle closely follows the boundaries of the simplex (see Fig. 4a). By increasing the mutation rate (typically over 0.01), cycles disappear in a Hopf bifurcation yielding a stable mixed equilibrium (Figs. 4b-c).

Figure 2: The Joker effect in public goods games for large, well-mixed populations. The simplex describes the replicator-mutator dynamics, Eq. 1, for a population of cooperators, defectors and jokers with parameter values satisfying n > r > 1 + d(n -1), for which a rock-paper-scissor dynamics is expected (yellow region in Fig. 1). When mutation rates are small, the only equilibrium is a repeller (white dot), and trajectories end up in a stable limit cycle (black line). Thus the presence of jokers induces periodically a burst of cooperators.

Discussion and conclusions

This evolution has some resemblances with the effect of volunteering in a PG game (Hauert et al., 2002a,b), but the two games are fundamentally different. This can be told from the dynamic behavior of the system. In both cases, the existence of a third agent which does not participate in the game is the ultimate reason why cooperators periodically thrive through a Rock-Paper-Scissor dynamics. However, while the loners game leads to neutrally stable cycles around a center, trajectories in the Joker model are attracted by the heteroclinic cycle C-D-J-C. The difference is even more striking if mutations are included. Mutations replace the cycles in the loner model by a stable mixed equilibrium. In contrast, in the Joker model mutations substitute the heteroclinic orbit by a stable limit cycle, which undergoes a transition (Hopf bifurcation) to a stable mixed equilibrium above a threshold mutation rate.

These two scenarios can be understood from the analysis of general RPS games [START_REF] Hofbauer | Evolutionary Games and Population Dynamics[END_REF]. There are three situations: (a) orbits are attracted towards an asymptotically stable mixed equilibrium (the case of the loners game with mutations), (b) orbits cycle around a neutrally stable mixed equilibrium (the case of the loners game without mutations), and (c) orbits go away from an unstable mixed equilibrium and approach the heteroclinic orbit defined by the border of the simplex (the case of the Joker game without mutations). If mutations are added to the latter type of RPS games, limit cycles and a Hopf bifurcation upon increasing the mutation rate are also found [START_REF] Mobilia | Oscillatory dynamics in rock-paperscissors games with mutations[END_REF]. Limit cycles are robust to perturbations and have a well defined amplitude irrespective of the initial fractions of players (as long as it is not at the border of the simplex). Therefore, they are true attractors of the dynamics, and can thus be regarded as a robust evolutionary outcome, in contrast to neutrally stable cycles.

In contrast to loners, which do not participate in the game but receive a benefit outside of it, jokers do not receive any benefit at all and cause damage to players. Both loner and joker models coincide -in the absence of mutations-when the damage inflicted by jokers and the benefit obtained by loners are both zero. In this case both become simply non-participants in the game, and the only effect they produce is a reduction in the effective number of players in the game, which is not enough to induce an oscillatory dynamics (see Fig. 5). In other words, the appearance of the RPS cycle which periodically increases the population of cooperators in the presence of jokers can only happen, remarkably, provided d > 0, i.e., if jokers are truly destructive agents.

In this letter we have shed light on a still unexplored aspect of evolutionary game theory (the presence of a destructive strategy) in the prototypical PG game. We have shown, both theoretically and by numerical simulations, that the addition of purely destructive agents (jokers) to a standard PG game has, paradoxically, a positive effect on cooperation. Bursts of cooperators are induced through the appearance of a RPS cycle in which jokers beat defectors, who beat cooperators, who beat jokers in succession. The evolutionary dynamics provoked by the Joker, with periods of cooperation, defection and destruction of the PG, may help understand the appearance of cognitive abilities that allow individuals to foresee the destructive periods, promoting in advance the necessary cooperation to avoid them.

We have proven this "Joker effect" to occur both in finite and infinite populations, discarding the possibility of its being an artificial size-depending phenomenon. Further research is required to ascertain the scope of the constructive role of destruction in general settings. This provides a new framework for the evolution of cooperation that may find important implications in social, biological, economical, and even philosophical contexts, and that is worth exploring either with different variants of this game or with new, more specific games accounting for indiscriminate destruction.

Thus, when playing the game, the X player will always interact with n -1 Y players. Therefore P X = Π X (1X, (n -1)Y).

(A.1)

On the other hand, the n -1 opponents of a Y player can be of just two types: either all n -1 are Y players, or n -2 are Y players and one is the single X player. The latter situation occurs with probability (n -1)/(M -1). Therefore the average payoff of a Y player will be

P Y = Π Y (nY) M -n M -1 + Π Y (1X, (n -1)Y) n -1 M -1 . (A.2)
Next we derive the invasion conditions for homogeneous populations of three types of players. In this new scenario we must consider the six different situations arising form the pair interactions that can be formed:

(A) 1D + (M -1)C. P C = r -1 - r n n -1 M -1 , P D = r - r n . (A.
3)

The tragedy of the commons occurs when defectors overcome cooperators, i.e., P D > P C . This happens iff

r < n M -1 M -n . (A.4)
We will henceforth assume (A.4) to hold. This condition contains the dilemmatic region 1 < r < n of PG games. In the limit M → ∞, the inequality (A.4) reduces to r < n and both, the conditions for the dilemma and the tragedy of the commons coincide.

(B) 1C + (M -1)D.

P C = r n -1, P D = r n n -1 M -1 . (A.5)
Because of (A.4) P D > P C , so C never invades D.

(C) 1J + (M -1)C.

P C = r -1 - d M -1 , P J = 0. (A.6) Since P J > P C iff r < 1 + d M -1 , (A.7)
then J invades C iff (A.7) holds.

(D) 1C + (M -1)J.

P C = r -(n -1)d -1, P J = 0. (A.8) Since P C > P J iff r > 1 + (n -1)d, (A.9)
then C invades J iff (A.9) holds.

(E) 1D + (M -1)J.

P D = -(n -1)d, P J = 0. (A.10)
As long as d > 0 we will have P J > P D , then D never invades J.

(F) 1J + (M -1)D. .11) As long as d > 0 we will have P J > P D , then J always invades D.

P D = - d M -1 , P J = 0. ( A
Figure 1 illustrates the different regions of interest in this game. The most interesting one is that in which there is a rockpaper-scissor rotation between C, D, and J, which corresponds to

1 < r < n M -1 M -n , 0 < d < r -1 n -1 . (A.12)

Appendix B. Infinite populations: average payoffs

We evaluate here the average payoffs P X obtained by each strategy (i =C, D, J) in this game when the population is very large. These functions will determine the dynamics of the population through the replicator equation. As before, sample groups of n individuals playing the game are randomly formed, and it is assumed that each player is sampled a large number of times before payoffs are compared in order to update strategies. The payoff for a given strategy is therefore proportional to the average payoff that a player using this strategy obtains playing against the whole population. This average payoff will depend only on the player's strategy and the composition of the population, described by a fraction x of cooperators, z of jokers and y = 1xz of defectors. Notice that P J = 0 for any composition of the population, so only cooperators' and defectors' payoffs need to be calculated.

Appendix B.1. Defectors

The average payoff of a defector is

P D = rm -d j S , (B.1)
where the symbol • • • denotes an average over samples of n-1 opponents randomly selected from the population. The average m/S can be obtained as in (Hauert et al., 2002b), yielding Joining the two averages one gets the average payoff of a defector,

m S = x 1 -z 1 - 1 -z n n(1 -z) .
P D = r x 1 -z 1 - 1 -z n n(1 -z) -d 1 -z n 1 -z -1 , (B.2)
the first term arising from the exploitation of cooperators and the second one being the damage inflicted by jokers.

Figure 1 :

 1 Figure1: Dynamics of invasions in a Public Goods game with jokers. The axes represent the gain factor r of the Public Goods game (i.e., the payoff each cooperator yields to the public good) and the "damage" d > 0 that every joker inflicts on the public good. The tragedy of the commons occurs for 1 < r < r max = n(M -1)/(Mn) (see text), which includes the dilemmatic region 1 < r < n characteristic of PG games. Different colors are assigned to different invasion patterns: Light blue corresponds to a region where J invades both C and D (III); light green corresponds to a region where neither C nor J invades each other (there is bistability on the J-C line) but D invades C and is in turn invaded by J, so again everything ends up in J (II); finally, light yellow corresponds to a region where D invades C, J invades D, but C invades J back, thus generating a rock-paper-scissors cycle (I). The latter behavior is the essence of the Joker effect. The equations of the straight lines separating the three regions are (from top to bottom) r = 1 + d(n -1) and r = 1 + d/(M -1). Notice that this scheme is valid for arbitrary n > 1. Also, for fixed r, all three regions are crossed upon varying d, whereas vice versa is only true provided d < d 1 = M/(Mn). The Joker effect does not occur if d > d 1 . For large populations, M 1, the region for the rock-paper-scissors cycle simplifies to n > r > 1 + (n -1)d and d < 1.

  Figure2: The Joker effect in public goods games for large, well-mixed populations. The simplex describes the replicator-mutator dynamics, Eq. 1, for a population of cooperators, defectors and jokers with parameter values satisfying n > r > 1 + d(n -1), for which a rock-paper-scissor dynamics is expected (yellow region in Fig.1). When mutation rates are small, the only equilibrium is a repeller (white dot), and trajectories end up in a stable limit cycle (black line). Thus the presence of jokers induces periodically a burst of cooperators. Cooperators abound during short time spans, as shown by the small fraction of cooperators in the equilibrium point. Parameters: n = 5, r = 3, d = 0.4 and μ = 0.005. (Images generated using a modified version of the Dynamo Package[START_REF] Sandholm | Dynamo: Phase diagrams for evolutionary dynamics[END_REF]).

Figure 3 :

 3 Figure 3: Replicator-mutator dynamics as a function of the damage d inflicted by jokers. For a fixed mutation rate, the size of the cycles increases as the damage increases. Parameters: n = 5, r = 3 and μ = 0.001.

Figure 4 :

 4 Figure 4: Replicator-mutator dynamics as a function of the mutation rate μ. (a) For very small mutation rates cycles approach the boundary of the simplex. (b) As μ increases, the cycle amplitude decreases and, above a critical value (typically, μ c 0.01), cycles disappear in a Hopf bifurcation yielding a stable mixed equilibrium (c). Parameters: n = 5, r = 3 and d = 0.4.

Figure 5 :

 5 Figure 5: Replicator-mutator dynamics for d = 0. If jokers are just passive agents cooperators go extinct. (a) μ = 0. The system ends up in a point of the line DJ with a majority of defectors. (b) μ = 0.001. Mutation generates one single stable state made up mostly by defectors. Parameters: n = 5, r = 3 and d = 0.

  Since j = n -S , the second term in Eq. (B.1) can be written asn 1/Sz) S -1 z n-S 1 S ,the factor in front of 1/S in the summation being the probability of having S -1 non-jokers in a group of n -1 randomly chosen players. By using the identity a a

Acknowledgments

Financial support from Ministerio de Ciencia y Tecnología (Spain) under projects FIS2009-13730-C02-02 (A.A.), FIS2009-13370-C02-01 (J.C. and R.J.R.) and MOSAICO (J.A.C.); from the Director, Office of Science, Computational and Technology Research, U.S. Department of Energy under Contract No. DE-AC02-05CH11231 (A.A.); from the Barcelona Graduate School of Economics and of the Government of Catalonia (A.A.); from the Generalitat de Catalunya under project 2009SGR0838 (A.A.) 2009SGR0164 (J.C. and R.J.R.) and from Comunidad de Madrid under project MODELICO-CM (J.A.C.). R.J.R. acknowledges the financial support of the Universitat Autònoma de Barcelona (PIF grant) and the Spanish government (FPU grant).

Appendix A. Finite populations: invasion analysis

We shall consider the situation in which in a homogeneous population of M individuals with the same strategy Y, one of them mutates (changes) to a different type X. The new individual will invade provided its average payoff after many interactions, P X , is larger than the average payoff of a Y individual, i.e., P X > P Y . Average payoffs can be evaluated as follows. The population is made of one X player and M -1 Y players.

Appendix B.2. Cooperators

The difference P D -P C can be written as

because in a group of S -1 opponents switching from cooperation to defection yields a payoff increment of 1r/S : the defector's payoff gets reduced by r/S because there is one cooperator less in the group, but adds 1 to her payoff because she does not pay the cost of cooperating (Hauert et al., 2002b). The average in the r.h.s. of Eq. (B.3) just contains < 1/S >, thus yielding

Finally, from Eqs. (B.2) and (B.4) one gets

Appendix C. Infinite populations: proof of existence of limit cycles

To complete the proof that the system ends up in a limit cycle it remains to show that the interior equilibrium of Eqs (1) is a repeller, i.e., its two eigenvalues have positive real parts. The interior equilibrium and its stability can be evaluated in the limit of small mutation rates, the one we are interested in. In this case, one can neglect the dependence of μ in the position of the fixed point. We are thus faced with the solution of the dynamical system (1) without the mutation term. The calculation becomes simple for n = 2, and tractable for n > 3. The proofs are treated separately in the next subsections.

Appendix C.1. Interior fixed point for n = 2

The interior fixed point (x 0 , y 0 , z 0 ) satisfies P C = P D = 0. According to Eq. B.4, the first equality requires (1 + z 0 )r = 2, yielding

Since n = 2 > r > 1, one has 0 < z 0 < 1, as it should. The second equality, P D = 0, produces

Condition r > 1 + d from expression (2) guarantees that 0 < x 0 < 1 and 0 < y 0 = 1x 0z 0 < 1. In order to analyze the stability of this equilibrium, we consider frequencies x and z as the independent variables of the two-dimensional system. To prove that the equilibrium is a repeller it suffices to show that the trace and determinant of the Jacobian matrix at the fixed point are both positive. For n = 2, equations (1) become

2)

The Jacobian matrix in the interior equilibrium is

whose trace, T , and determinant, D, are

T is positive because n = 2 > r > 1. To prove that the determinant is positive, we should realize that the second bracket in its expression can be written as r(r -1)d(2r), which is larger than 2(r -1) 2 > 0 because r > 1 + d.

Appendix C.2. Interior fixed point for n > 3

We use the same procedure as in the previous case. The fraction of jokers z 0 of the interior equilibrium arises from P C = P D , namely Eq. (B.4). Once it is found, x 0 follows from P D = 0, c.f. Eq. (B.2).

Appendix C.2.1. Calculation of z 0

z 0 is obtained as the solution to

The latter equation has exactly one solution, namely the crossing of the polynomial in the l.h.s of Eq. (C.7) with the constant n/r > 1. Since r > 1, this occurs at 0 < z 0 < 1, consistent with the meaning of z 0 . There is no analytical solution to Eq. (C.6) for arbitrary n. There exists, however, a simple analytical solution in the limit of large n, which is indeed an excellent approximation for all n > 3. It can be obtained neglecting z n as compared to 1 in (C.6), which leads to

1, which holds, say, for r > 3. Notice that if r n the equilibrium approaches allJ, so that cycles get very close to this state in this limit.

Appendix C.2.2. Calculation of x 0

Let us impose P D = 0. Introducing (C.6) into (B.2) one finds

Conditions d > 0, n > r > 1, and r > 1 + (n -1)d yield 0 < x 0 < 1 and 0 < x 0 + z 0 < 1, so that the three fractions are smaller than 1. Substituting z 0 from expression (C.8) into (C.9) one finally obtains

.3. Stability of the interior equilibrium

We need to determine the Jacobian matrix for the equilibrium (x 0 , z 0 ) given by C.9 and C.8. The dynamical system (1) can be written as

The first equation is very cumbersome. Fortunately, as already explained, in the limit of large n and if r > 3 one can neglect terms of order z n and above. Using expressions (C.8) and (C.10), the Jacobian matrix J can be written as J = Yd(nr)/n, where

(Notice that the factor d(n-r)/n > 0.) As the diagonal elements of this matrix are positive, the trace is positive. Also Y zx < 0 and, as we show next, Y xz > 0, therefore the determinant turns out to be positive, and the interior equilibrium is a repeller. To see that Y xz > 0 we must show that the numerator is positive. This can be shown by writing it as nr(r -1) + d(rn)(r 2r + n) > (r -1) 2 (nr) 2 + nr n -1 > 0.

The first inequality follows from condition r -1 > (n -1)d.


J J Poos

Å Brännström

U Dieckmann

Harvest-induced maturation evolution under different life-history trade-offs and harvesting regimes

The potential of harvesting to induce adaptive changes in exploited populations is now increasingly recognized. While early studies predicted that elevated mortalities among larger individuals select for reduced maturation size, recent theoretical studies have shown conditions under which other, more complex evolutionary responses to size-selective mortality are expected. These new predictions are based on the assumption that, owing to the trade-off between growth and reproduction, earlier maturation implies reduced growth. Here we extend these findings by analyzing a model of a harvested size-structured population in continuous time, and by systematically exploring maturation evolution under all three traditionally acknowledged costs of early maturation: reduced fecundity, reduced growth, and/or increased natural mortality. We further extend this analysis to the two main types of harvest selectivity, with an individual's chance of getting harvested depending on its size and/or maturity stage. Surprisingly, we find that harvesting mature individuals not only favors late maturation when the costs of early maturation are low, but promotes early maturation when the costs of early maturation are high. To our knowledge, this study therefore is the first to show that harvesting mature individuals can induce early maturation.

Introduction

Concerns over evolutionary consequences of harvesting are mounting, as both theoretical and empirical studies have demonstrated the potential for rapid harvest-induced evolution (e.g., [START_REF] Law | Fishing, selection, and phenotypic evolution[END_REF][START_REF] Conover | Sustaining fisheries yields over evolutionary time scales[END_REF]Heino and Godø, 2002;[START_REF] Ashley | Evolutionarily enlightened management[END_REF][START_REF] Coltman | Undesirable evolutionary consequences of trophy hunting[END_REF][START_REF] Jørgensen | Managing evolving fish stocks[END_REF][START_REF] Dieckmann | The dawn of Darwinian fishery management[END_REF]. Life-history theory predicts that increased mortality devalues life-history processes occurring later in life, relative to those occurring earlier (e.g., [START_REF] Law | Optimal life histories under age-specific predation[END_REF][START_REF] Stearns | The Evolution of Life Histories[END_REF][START_REF] Ernande | Adaptive changes in harvested populations: plasticity and evolution of age and size at maturation[END_REF]. Therefore, systematic reductions in age and size at maturation observed for stocks exposed to heavy fishing (e.g., [START_REF] Jørgensen | Long-term changes in age at sexual maturity of Northeast Arctic cod (Gadus morhua L.)[END_REF]Rijnsdorp, 1993a;Trippel, 1995) are suggestive of fisheries-induced evolution (e.g., Heino et al., 2002;[START_REF] Grift | Fisheries-induced trends in reaction norms for maturation in North Sea plaice[END_REF][START_REF] Grift | Threedimensional maturation reaction norms for North Sea plaice[END_REF][START_REF] Barot | Long-term trend in the maturation reaction norm of two cod stocks[END_REF][START_REF] Barot | Maturation of Newfoundland American plaice (Hippoglossoides platessoides): long-term trends in maturation reaction norms despite low fishing mortality?[END_REF][START_REF] Olsen | Maturation trends indicative of rapid evolution preceded the collapse of northern cod[END_REF][START_REF] Olsen | Assessing changes in age and size at maturation in collapsing populations of Atlantic cod (Gadus morhua)[END_REF][START_REF] Baulier | Body condition and evolution of maturation of Atlantic cod in Newfoundland[END_REF][START_REF] Dieckmann | Probabilistic maturation reaction norms: their history, strengths, and limitations[END_REF][START_REF] Mollet | Fisheries-induced evolutionary changes in maturation reaction norms in North Sea sole Solea solea[END_REF][START_REF] Heino | Management of evolving fish stocks[END_REF]Dieckmann, 2008a, 2008b;[START_REF] Thériault | The impact of fishing-induced mortality on the evolution of alternative life-history tactics in brook charr[END_REF][START_REF] Arlinghaus | Quantifying selection differentials caused by recreational fishing: development of modeling framework and application to reproductive investment in pike (Esox lucius)[END_REF][START_REF] Okamoto | Mitigating fisheriesinduced evolution in lacustrine brook charr (Salvelinus fontinalis) in southern Quebec, Canada[END_REF]. These concerns add to those raised more broadly about the negative impact of fisheries on the ecosystems in which all fish stocks are embedded (e.g., [START_REF] Jennings | The effects of fishing on marine ecosystems[END_REF].

Concerns are exacerbated by the fact that effects of fisheries-induced evolution are building up cumulatively, and by model-based results indicating that such effects will often be difficult and slow to reverse (Dunlop et al., 2009a;[START_REF] Enberg | Implications of fisheries-induced evolution for stock rebuilding and recovery[END_REF].

Earlier theoretical studies using age-structured models have shown that increased harvesting pressures may cause evolutionary shifts toward earlier maturation [START_REF] Law | Evolution of yields from populations with age-specific cropping[END_REF][START_REF] Heino | Management of evolving fish stocks[END_REF][START_REF] Ernande | Adaptive changes in harvested populations: plasticity and evolution of age and size at maturation[END_REF]. The corresponding selection pressures are often enhanced when harvesting is positively size-selective, i.e., when larger fish are exploited more heavily than smaller fish. This picture has been complemented by recent studies demonstrating that in the presence of life-history trade-offs and/or predation, harvest-induced evolution may also result in delayed maturation [START_REF] Gårdmark | Life-history evolution in harvested populations: the role of natural predation[END_REF][START_REF] Gårdmark | Disparate maturation adaptations to size-dependent mortality[END_REF]. These newer predictions are based on specific assumptions about the trade-off between early maturation and growth, and therefore between early maturation and subsequent fecundity. Since there are several ways in which early maturation may cause fitness costs, and since the assumptions made about the resultant trade-offs are likely to impact the evolutionary predictions, further scrutiny of these dependences is warranted. As the major competing functions in an individual's allocation of available resources are reproduction, survival, and growth (e.g., [START_REF] Roff | An allocation model of growth and reproduction in fish[END_REF][START_REF] Charnov | Evolution of life history parameters in animals with indeterminate growth, particularly fish[END_REF][START_REF] Arendt | Adaptive intrinsic growth rates: an integration across taxa[END_REF][START_REF] Fonseca | Are fish early growth and condition patterns related to life-history strategies?[END_REF], it is natural to consider that early maturation may result in reduced fecundity, reduced growth, and/or increased natural mortality. In this study, we systematically explore the evolutionary impacts of fishing under all these scenarios. Also, earlier theoretical studies of fisheries-induced evolution have largely focused on size-selective fishing. This reflects the fact that many of the world's fisheries are harvested size-selectively, because of gear design, targeting incentives of fishers, or management regulations (e.g., [START_REF] Holden | The Common Fisheries Policy[END_REF][START_REF] Hart | Handbook of Fish Biology and Fisheries: Volume 2, Fisheries[END_REF][START_REF] Fromentin | Atlantic bluefin tuna: population dynamics, ecology, fisheries and management[END_REF]. Some fisheries, however, may also be selective with respect to the maturity stage of individuals, irrespective of their size. While such an exploitation pattern is less common, it readily occurs when fisheries selectively target a stock's spawning grounds, like in the cases of Northeast Arctic cod (e.g., [START_REF] Garrod | Population dynamics of the Arcto-Norwegian cod[END_REF][START_REF] Opdal | Fisheries change spawning ground distribution of northeast Arctic cod[END_REF] and Norwegian spring-spawning herring (e.g., [START_REF] Dragesund | Biology and population dynamics of the Norwegian spring spawning herring[END_REF][START_REF] Engelhard | Maturity changes in Norwegian spring-spawning herring Clupea harengus: compensatory or evolutionary responses[END_REF]. At first sight, such stage-specific selectivity may appear to be very similar to size-specific harvesting, as mature individuals tend to be large individuals and vice versa. It is therefore important to realize that the evolutionary implications of stage-specific exploitation may surprisingly differ from those of size-specific exploitation. In this study, we systematically explore the evolutionary impacts of fishing under both scenarios.

In the following sections, we investigate maturation evolution in a size-structured continuous-time model of a selectively harvested population. In particular, we extend previous analyses by systematically exploring the evolutionary implications of (1) different fitness costs of early maturation, which may imply reduced fecundity, reduced growth, and/or increased natural mortality; and of (2) different selectivities of the harvesting regime, which may be based on size or on maturity stage.

Model description

We consider a harvested population divided into three size classes, as illustrated in Figure 1. tilde always refer to the early-maturing life history. Table 1 provides an overview of all model variables and parameters.

Life-history processes

Life-history dynamics

The dynamics of individuals in the first size class are given by The dynamics of early-and late-maturing individuals in the second size class are given by, respectively,

2 1 2 2 3 3 3 3 1 1 1 1 2 2 2 3 3 3 3 2 3 3 3 3 3 3 dN f N f N f N m N r N dt . ( 1
2 1 1 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 dN r N m N h N r N dt J , ( 2a
) 2 1 1 2 2 2 2 2 2 (1 ) dN r N m N h N r N dt J . ( 2b
)
From left to right, the terms represent growth from the first size class, natural mortality, harvest mortality, and growth into the third size class.

Finally, the dynamics of early-and late-maturing individuals in the third size class are given by, respectively,

3 2 2 3 3 3 3 dN r N m N h N dt 3 N 3 r r 3 3 3 N h N 3 3 3 3 3 N 2 r N 2 r N r N r N 2 3 N m N 2 3 2 3 N m N N N , ( 3a
) 3 2 2 3 3 3 3 dN r N m N h N dt . ( 3b
)
From left to right, the terms represent growth from the second size class, natural mortality, and harvest mortality.

Life-history trade-offs

To explore maturation evolution in our model, we examine adaptations in the probability J that an individual matures early. Like [START_REF] Gårdmark | Life-history evolution in harvested populations: the role of natural predation[END_REF], we assume that earlier matura- ! m m ). Based on these trade-offs and the dynamics specified above, we now investigate the selection pressures harvesting imposes on the quantitative trait J .

Evolutionary consequences of size-specific harvesting

To determine the fitness differences between early-and late-maturing individuals, we consider their basic reproduction ratio,

0 2 2 3 3 R FT FT , (4)
where 2 F and 3 F denote, respectively, the average fecundity rate of an individual in the second and third size class, while 2 T and 3 T denote the average time individuals spend in these two size classes. Early maturation increases 2 F , but the trade-offs with growth, mortality, and/or fecundity will additionally affect 2 T , 3 T , and/or 3 F , making the overall effect on 0 R hard to intuit. Table 2 summarizes the qualitative impacts that early maturation has on 2 F , 3 F , 2 T , and 3 T under the three types of trade-offs. To proceed, a quantitative analysis is required.

Evolutionary invasion analysis

With

1 0 0 0 0 (1 ) R R R J J ,
2 2 2 2 2 3 2 3 3 3 2 2 2 1 § • ! ¨¸ © ¹ • • 2 1 § ! ¨2 3 § § 2 3 ¨2 3 ¨• 2 2 2 2 2 3 2 2 2 2 2 2 2 2 2 ¹ • 2 3 • • 2 3 • 2 3 • m h r f r f r f m h m h r . ( 7
)
This inequality allows us to conclude that harvesting the third size class always promotes early maturation. The consequences of harvesting the second size class at a stage-unspecific rate 2 2 2 h h are less obvious and are examined below for each of the three potential costs of early maturation. It should be noted, however, that too high harvest mortality rates may cause the population to go extinct. Hence, conclusions drawn from Ineq. (7) are meaningful only for parameter values at which the population persists. A comprehensive graphical representation of possible outcomes of harvesting the second size class, also accounting for the exploitation limits at which the population goes extinct, is given in Fig. 2.

Trade-off between early reproduction and fecundity rate

Assuming that the only cost of early maturation is reduced fecundity in the third size class,

3 3 3 3 f f , Ineq. (7) simplifies to 3 3 3 3 2 2 ) ( ~h m f f r f ! . ( 8
)
In this case, harvesting the second size class does not affect maturation evolution. This is because an increased mortality rate in the second size class equally reduces an individual's probability of reaching the third size class and its time spent in the second size class.

Trade-off between early reproduction and growth rate

When early maturation reduces the growth rate of mature individuals, 2 2 r r

2 2 r 2 r 2 , Ineq. (7) simpli- fies to ) )( ( ) )( ( ~2 2 2 3 3 2 2 2 2 3 2 r h m h m r r h m f f ! . ( 9
)
In contrast to the preceding case, the mortality rates in the second size class do affect this inequality, and may therefore cause maturation evolution. Harvesting individuals in the second size class causes the right-hand side of Ineq. ( 9) to increase, thus making early maturation less advantageous.

Trade-off between early maturation and mortality rate

When early maturation increases the natural mortality rate of mature individuals in the second

size class, 2 2 ! 2 2 ! m m , Ineq. (7) simplifies to ) )( ( ) ( ~2 2 2 3 3 2 2 2 3 2 r h m h m m m r f f ! . ( 10
)
Here, the right-hand side decreases if the harvesting mortality rate increases, so harvesting the second size class makes early maturation more advantageous.

Multiple trade-offs associated with early maturation

The

f with respect to 2 h , * 3 2 2 2 2 2 2 2 2 3 3 2 2 2 ( ) ( )( ) f r m m r r f h m h m h r w w 2 2 2 ) 2 2 2 2 2 2 2 2 2 2 2 2 . ( 11
)
We see that the sign of this derivative depends only on 2

2 2 2 m m r r 2 2 2 r 2 2 2 2 2 m r r 2 2 2
: harvesting the second size class makes early maturation more (less

) advantageous if 2 2 2 2 ( ) 0 ! 2 2 2 ( ) 0 2 2 2 ( m m r r , i.e., if 2 2 2 2 ( ) ! 2 2 2 2 ( ) 2 ! 2 ( ) m m
r r , i.e., if the increase in mortality rate caused by early maturation is larger (smaller) than the decrease in growth rate caused by early maturation.

Conclusion

We have analyzed the effects of size-specific harvesting on maturation evolution for three potential costs of early maturation: reduced fecundity, reduced growth, and/or increased natural mortality. Results are summarized in Table 3: harvesting the smallest size class has no effect on maturation evolution, harvesting the intermediate size class can favor either early maturation (when mortality costs exceed growth costs) or late maturation (when growth costs exceed mortality costs), while harvesting the largest size class always favors early maturation. As noted in Sec. 3.1, our analysis applies to the full parameter ranges in which the population remains extant. In Fig. 3, we demonstrate that each of the predicted outcomes does indeed occur.

Evolutionary consequences of stage-specific harvesting

Using an analysis similar to that in the previous section, we can explore the effects of stagespecific harvesting, to examine the evolutionary consequences we expect from harvesting either mature or immature individuals. This scenario applies when fisheries selectively target a stock's spawning grounds or nursery grounds.

Evolutionary invasion analysis

We first analyze the case of stage-specific harvesting in which only immature individuals are 7), we see that early maturation is advantageous if and only if

harvested: 2 3 0 h h 2 3 0 h h 2 3 2 3 h and 2 i m m h h . From Ineq. (
1 • © § ! 3 2 3 2 2 imm 2 2 2 3 2 1 ~f r f r r h m r m m f . ( 12
)
The evolutionary effects caused by the harvesting of immature individuals depend on whether the right-hand side of this inequality increases or decreases with respect to the harvest mortality rate imm h , given that the population persists. Since the right-hand side of Ineq. ( 12) is a decreasing function of imm h , harvesting immature individuals always promotes early maturation. This is true for any combination of costs of early maturation we may consider. The biological interpretation of this result derives from the fact that maturation in the second size class offers an effective refuge from a harvesting regime that exploits only immature individuals.

We now turn to stage-specific harvesting regimes that target only mature individuals: h h h . From Ineq. (7), we see that early maturation is then favored by se- lection if and only if

1 • © § ! 3 2 3 2 2 2 2 mat 2 3 2 1 ~f r f r r m r h m h m f mat . ( 13
)
As the evolutionary consequences of harvesting mature individuals are difficult to elucidate when multiple trade-offs are considered simultaneously, we explore these consequences for each of the three trade-offs in turn.

Trade-off between early reproduction and fecundity rate

When the sole cost of early maturation is a reduced fecundity rate in the third size class, the consequences of harvesting mature individuals are found by setting 2 2 r r 13) with respect to mat h . In Appendix B, we show that the sign of this derivative is positive for small positive values of 3 3 f f 3 f 3 , i.e., for small fecundity costs of early maturation. In that case, harvesting of mature individuals will make late maturation more advantageous. However, if the fecundity cost 3 3 f f 3 f 3 of early maturation is high, harvesting mature individuals may have the opposite effect, by making early maturation more advantageous.

Trade-off between early reproduction and growth rate

If early maturation instead induces slower growth from the second to the third size class, we find that the right-hand side of Ineq. ( 13) is an increasing function of mat h when

Understanding the evolutionary outcomes

The finding that harvesting mature individuals can induce not only late maturation (when costs of early maturation are low), but also early maturation (when costs of early maturation are high) is novel and surprising, but can be understood with the help of Fig. 1.

When costs of early maturation are negligible and harvesting is absent, early and late ma- Finally, when the trade-off between early maturation and mortality rate is sufficiently strong, early-maturing individuals experience a higher natural mortality in the second size class. In the absence of harvesting, this promotes late maturation. As the harvesting of mature individuals is intensified, however, this direct fitness cost of early maturation is increasingly overshadowed by fishing mortality. Consequently, when such harvesting is intensive enough, the natural evolutionary advantage of early maturation will dominate the outcome.

For all three life-history trade-offs, the high direct fitness costs of early maturation, which favor late maturation in the absence of harvesting, thus gradually vanish as mature individuals are harvested intensively, uncovering the natural evolutionary advantage early maturation confers under such conditions.

Conclusion

We have analyzed the effects of stage-specific harvesting on maturation evolution for three potential costs of early maturation: reduced fecundity, reduced growth, and/or increased natural mortality. Results are summarized in Table 3. Harvesting immature individuals always favors early maturation. Harvesting mature individuals favors late maturation when the costs of early maturation are low. When natural mortality in the largest size class exceeds that in the intermediate size class, this conclusion remains true even when the growth costs of early maturation are arbitrarily high. For all other cases, high costs of early maturation can reverse the evolutionary consequence of harvesting mature individuals, so that such harvesting then makes early maturation more advantageous. In Fig. 4, we demonstrate that each of the predicted outcomes does indeed occur.

Discussion

Here we have analyzed harvest-induced maturation evolution under different costs of early maturation and for harvest regimes with different selectivities. In particular, we have examined costs of early maturation that imply reduced fecundity, reduced growth, and/or increased natural mortality, while considering harvest regimes that are selective for either size or maturity stage. Our results provide a first systematic overview of how the evolutionary effects of harvesting vary with the trade-offs associated with early maturation and with the selectivities of harvesting regimes. The results of our investigation are summarized in Table 3, and paraphrased in Secs. 3.6 and 4.5.

Life-history traits are often coupled through trade-offs (e.g., [START_REF] Charnov | Evolution of life history parameters in animals with indeterminate growth, particularly fish[END_REF][START_REF] Stearns | The Evolution of Life Histories[END_REF]. In this study we have therefore considered trade-offs between size at maturation and three major life-history characteristics: fecundity, growth, and mortality. These tradeoffs can affect maturation evolution by altering the underlying selection pressures. The importance of considering multiple trade-offs when studying harvest-induced maturation evolution is demonstrated by our results: selection pressures on size at maturation caused by size-or stage-specific harvesting vary, often qualitatively, with the considered life-history trade-offs.

Early maturation becomes advantageous when the benefits of reproducing early exceed the total costs of early reproduction. We find that when the cost of early maturation reduces fecundity in the largest size class, the mortality rate in the intermediate size class has no bearing on whether early or late maturation is advantageous. At first sight, this finding contrasts with earlier results obtained by [START_REF] Gårdmark | Life-history evolution in harvested populations: the role of natural predation[END_REF]. However, this difference between the continuous-time size-structured model studied here and the discrete-time age-structured model studied by [START_REF] Gårdmark | Life-history evolution in harvested populations: the role of natural predation[END_REF] is unsurprising, since in our model a change in the mortality of the intermediate size class does not only affect the probability of reaching the largest size class, but also the time spent in the intermediate size class. This is not the case in the model by [START_REF] Gårdmark | Life-history evolution in harvested populations: the role of natural predation[END_REF], where the time spent in the intermediate age class is fixed.

Harvest regimes are often selective in the sense that the mortality caused by fishing may depend on size, age, and/or maturity stage (e.g., [START_REF] Ajiad | Sexual difference in maturation of Northeast Arctic cod[END_REF][START_REF] Law | Fishing, selection, and phenotypic evolution[END_REF]. While agespecific harvesting is rare, size-and stage-specific harvesting are common. The results of our study underscore that the latter harvest selectivities can qualitatively change the outcomes of maturation evolution: harvesting intermediately sized or mature individuals can favor either early or late maturation (depending on the costs of early maturation), whereas harvesting large or immature individuals always favors early maturation.

Previous studies have already shed some light on maturation evolution under harvest regimes that select for size or maturity stage. A study by [START_REF] Ernande | Adaptive changes in harvested populations: plasticity and evolution of age and size at maturation[END_REF] predicted that the harvesting of mature individuals induces maturation at an older age and larger size, whereas the harvesting of immature individuals induces maturation at a younger age and smaller size.

Their work generalized earlier findings by [START_REF] Law | Evolution of yields from populations with age-specific cropping[END_REF] and [START_REF] Heino | Management of evolving fish stocks[END_REF], which were obtained for directly evolving age or size at maturation, respectively, to ages and sizes at maturation that plastically varied with juvenile growth conditions. Our results differ from these findings by showing that harvesting of mature individuals may either increase or decrease the size at maturation that is favored by selection. Our results confirm findings by [START_REF] Gårdmark | Disparate maturation adaptations to size-dependent mortality[END_REF], who showed that evolutionary outcomes in size at maturation can both increase and decrease as a result of size-specific harvesting. Our results are novel by showing how the alternative fitness costs of early maturation determine the evolutionary consequences of harvesting.

For harvesting to decrease the size at maturation, the direct fitness cost of early maturation must be large. As an example, when this cost is expressed in terms of higher natural mortality, the harvesting of mature individuals may cause early maturation only if the additional mortality rate exceeds the natural mortality rate of the third size class. As another example, when the cost of early maturation is expressed in terms of lower growth, and natural mortality in the third size class is 10 percent lower than in the second, the harvesting of mature individuals may cause early maturation only if the early-maturing individuals grow 10 times slower than the late-maturing individuals. As a final example, when the cost of early maturation is expressed in terms of lower fecundity in the third size class, and the intrinsic growth and mortality rates are all equal, the fecundity must be reduced by half before harvesting mature individuals may cause early maturation.

With the number of empirical studies indicative of harvest-induced evolution in maturation schedules of commercially exploited marine fish stocks mounting rapidly (e.g., [START_REF] Grift | Fisheries-induced trends in reaction norms for maturation in North Sea plaice[END_REF][START_REF] Grift | Threedimensional maturation reaction norms for North Sea plaice[END_REF][START_REF] Olsen | Maturation trends indicative of rapid evolution preceded the collapse of northern cod[END_REF][START_REF] Olsen | Assessing changes in age and size at maturation in collapsing populations of Atlantic cod (Gadus morhua)[END_REF][START_REF] Barot | Maturation of Newfoundland American plaice (Hippoglossoides platessoides): long-term trends in maturation reaction norms despite low fishing mortality?[END_REF][START_REF] Baulier | Body condition and evolution of maturation of Atlantic cod in Newfoundland[END_REF][START_REF] Mollet | Fisheries-induced evolutionary changes in maturation reaction norms in North Sea sole Solea solea[END_REF][START_REF] Okamoto | Mitigating fisheriesinduced evolution in lacustrine brook charr (Salvelinus fontinalis) in southern Quebec, Canada[END_REF], it is becoming increasingly important to interpret observed trends in terms of sufficiently realistic eco-evolutionary models. Since, as a matter of principle, it will never be possible to prove fisheries-induced evolution as the unequivocal cause of maturation trends [START_REF] Dieckmann | Probabilistic maturation reaction norms: their history, strengths, and limitations[END_REF], models are key to assessing whether the observed trends comply with those predicted by life-history theory: the closer the match, the more an interpretation of observed trends in terms of fisheries-induced evolution is strengthened.

In response to this need, various eco-evolutionary modeling approaches have been devised

and analyzed. The pioneering work by [START_REF] Law | Evolution of yields from populations with age-specific cropping[END_REF] addressed harvest-induced maturation evolution in terms of simple age-structured population models (see also [START_REF] Rowell | The effects of fishing on the timing of maturity in North Sea cod (Gadus morhua L.)[END_REF]Rijnsdorp, 1993b;[START_REF] Heino | Management of evolving fish stocks[END_REF]. Since then, several studies have extended the modeling of harvest-induced evolution to, e.g., natural predation [START_REF] Gårdmark | Life-history evolution in harvested populations: the role of natural predation[END_REF], phenotypically plastic growth and maturation [START_REF] Ernande | Adaptive changes in harvested populations: plasticity and evolution of age and size at maturation[END_REF], the effects of marine reserves [START_REF] Baskett | Marine reserve design and the evolution of size at maturation in harvested fish[END_REF]Dunlop et al., 2009b;[START_REF] Miethe | Marine reserves and the evolutionary effects of fishing on size at maturation[END_REF], continuous size structure [START_REF] Gårdmark | Disparate maturation adaptations to size-dependent mortality[END_REF], density-dependent growth and genetic variation in multiple life-history traits (Dunlop et al., 2009a), density-dependent harvesting behavior [START_REF] Arlinghaus | Quantifying selection differentials caused by recreational fishing: development of modeling framework and application to reproductive investment in pike (Esox lucius)[END_REF], processes of stock recovery [START_REF] Enberg | Implications of fisheries-induced evolution for stock rebuilding and recovery[END_REF], and explicit resource dynamics [START_REF] Okamoto | Mitigating fisheriesinduced evolution in lacustrine brook charr (Salvelinus fontinalis) in southern Quebec, Canada[END_REF]. The present study adds to this portfolio of modeling approaches by elucidating the effects of life-history trade-offs on harvest-induced maturation evolution.

A strong feature of the analysis presented here is that all results have been derived analytically, rather than through numerical explorations. Naturally, this benefit comes at the cost of a relatively simple model structure, defined by three size classes. As a particularly promising direction for future research, it will therefore be desirable to generalize our analysis to populations with continuous size structure, and if possible, with phenotypically plastic growth and maturation. Until then, our analysis can best be applied to natural populations by recognizing that our model's intermediate size class is to be defined as spanning the range of sizes over which maturation can naturally occur.

Predictions of our study can readily be judged against empirical observations of maturation evolution. Here we mention just a few examples. Positively size-selective fishing is common among stocks that are harvested by trawl fisheries. These gears generally have catch selectivity curves that increase with fish length (e.g., [START_REF] Millar | Estimating the size-selectivity of fishing gear by conditioning on the total catch[END_REF]. Examples of such stocks include North Sea sole and North Sea plaice, where both mature and immature fish experience positively size-selective harvest mortality rates. Our results summarized in Table 3, for the "Harvesting of size class 3", are compatible with the observed trends towards maturation at smaller size (Rijnsdorp, 1993;[START_REF] Grift | Fisheries-induced trends in reaction norms for maturation in North Sea plaice[END_REF][START_REF] Grift | Threedimensional maturation reaction norms for North Sea plaice[END_REF][START_REF] Mollet | Fisheries-induced evolutionary changes in maturation reaction norms in North Sea sole Solea solea[END_REF] observed for these and many other stocks.

Stage-specific fishing is common among stocks that undergo spawning migrations. Examples of such stocks include Northeast Arctic cod and Norwegian spring-spawning herring, where a strong spatio-temporal segregation of mature and immature fish enables targeted fisheries. Our results summarized in Table 3, for the "Harvesting of mature individuals", are compatible with the maturation sizes observed for Northeast Arctic cod until about the middle of the 20 th century, recognizing that these sizes were unusually large compared with all other stocks of Atlantic cod, that costs of early maturation are thought to be relatively low in this stock, and that the historical fishing regime had predominantly targeted mature individuals [START_REF] Law | Evolution of yields from populations with age-specific cropping[END_REF][START_REF] Jørgensen | Long-term changes in age at sexual maturity of Northeast Arctic cod (Gadus morhua L.)[END_REF]. Furthermore, our results summarized in Table 3, for the "Harvesting of immature individuals", are compatible with the maturation sizes observed for Northeast Arctic cod since about the middle of the 20 th century, recognizing that harvest mortality rates of immature Northeast Arctic cod in the Barents Sea has increased by a factor of approximately 5-7 from 1920 to 1960. Finally, our results summarized in Table 3, for the "Harvesting of mature individuals", are compatible with the maturation sizes observed for Norwegian spring-spawning herring during the 20 th century. For this stock, a marked decrease in growth after maturation indicates that the trade-offs between early maturation and growth, and thus also between early maturation and future fecundity, are strong [START_REF] Engelhard | Age at maturation predicted from routine scale measurements in Norwegian spring-spawning herring Clupea harengus using discriminant and neural network analyses[END_REF]. In accordance with our results, the evolutionary changes in size at maturation caused by intense harvesting on the spawning grounds of Norwegian spring-spawning herring have been found to be very small [START_REF] Engelhard | Maturity changes in Norwegian spring-spawning herring Clupea harengus: compensatory or evolutionary responses[END_REF].

In conclusion, we have systematically explored how different aspects of individual lifehistory characteristics and harvesting regimes affect the relative evolutionary advantage of early maturation. To keep the analysis tractable, the ecological environment -consisting of prey, competitors, and predators -has not been included in our model, even though it may have confounding effects on maturation evolution (e.g., [START_REF] Gårdmark | Life-history evolution in harvested populations: the role of natural predation[END_REF]. Despite this simplification, we hope that the insights on the evolutionary consequence of harvesting under different life-history trade-offs obtained in this study make a helpful contribution to the rapidly growing body of knowledge on fisheries-induced adaptive changes required for rational fisheries management.

22

Appendix A: General invasion condition derived from basic reproduction ratio

An individual's basic reproduction ratio 0 R equals its total lifetime reproductive output. For our model, this can be expressed as 0 2 2 3 3 R FT FT , where 2 T and 3 T are the average times an individual will spend in the two larger size classes, and 2 F and 3 F are the average fecundity rates in these size classes.

Accounting for the fact that an individual may or may not mature early, we obtain 2 2 2 2 2

(1 )

T PD PD J J 2 (1 ) D (1 ) 2 D (1 D 2 (1 , (A.1)
where J is the individual's probability to mature early (i.e., in the second size class), 2 P is its probability of surviving from entering the first size class until entering the second size class, (1 )

P D f F P D P D J J J 2 D f 2 2 2 2 2 (1 ) P 2 D (1 ) J 2 D 2 (1 (1 2 2 2 2 2 . (A.2)
Analogously, the average time an individual spends in the third size class is (1 ) (1 ) and 2 N 2 N ), and large individuals (third size class, with densities 3 N and 3 N 3 N 3 ). Individuals can mature early (with probability J , bottom row) or late (with probability 1 J , top row). Quan- tities with a tilde refer to the early-maturing life history. The probability J of early maturation is evolving. The parameters 1 r , 2 r , and 2 2 r are growth rates; 2 2 f , 3 f , and 3 3 f are fe- cundity rates;

P D f P D f F P D P D J J J J 1 ) 1 1 3 P D f P D f 3 3 3 f (1 f 3 (1 (1 3 3 (1 P D (1 3 3 P D 3 3 (1 (1 3 3 3 3 3 . (A.4) For 0 R , we thus obtain 0 2 2 3 3 2 2 2 2 3 3 3 2 3 3 3 ( ) ( 1 ) R F T FT P D f P P D f P P D f J J 2 P P 2 2 ) (1 ) 2 2 2 2 2 2 2 2 ) (1 1 2 2 2 2 3 3 3 ) ( 3 3 3 3 3 3 3 3 ) ( ) ( 3
D f r D D f D D f r ! ( ( ) ( )) ( ) ( ) r f m r f m m r f h h m m r w w 3 2 2 3 2 2 ( ) (
1 1 / m is the carrying capacity of the first size class; 2 m , 2 2 m , and 3 m are natural mortality rates; and 2 h , 2 2 h , and 3 h are harvest mortality rates. (7) is not fulfilled and selection promotes late maturation (grey areas). As the harvesting rate is increased, there may be a switch in optimal maturation size (left panels). Alternatively, the switch in optimal maturation is predicted by the model at harvesting rates for which the population is no longer viable (central panels). Finally, model parameters can be chosen such that the switching point is never reached, independent of the harvesting rate (right panels). Parameter values are provided in Table C1 of Appendix C.

  Newborn individuals enter the first size class, grow into the second size class at rate 1 r , and from there grow into the third size class. With probability J , individuals mature when enter- ing the second size class (early maturation); otherwise, they mature when entering the third size class (late maturation). Late-maturing individuals grow into the third size class at rate 2 r , while early-maturing individuals grow into the third size class at rate 2 2 r . Late-maturing individuals produce offspring at rate 3 f , while early-maturing individuals produce offspring at rate 2 2 f in the second size class and at rate 3 3 f in the third. Late-maturing individuals expe- rience mortality rates 2 m and 3 m in the second and third size class, respectively, while earlymaturing individuals experience the corresponding mortality rates 2 2 m and 3 m . Natural mortality in the first size class is density-dependent and of logistic type, with carrying capacity 1 1 / m . Harvesting is assumed to be density-independent and to occur in the second and/or third size classes, with rates of harvest mortality denoted by 2 h (individuals in the second size class that are late-maturing and thus immature), 2 2 h (individuals in the second size class that are early-maturing and thus mature), and 3 h (individuals in the third size class, which are all mature). The densities of individuals are denoted by 1 N (individuals in the first size class, which are all immature), 2 N (individuals in the second size class that are late-maturing and thus immature), 2 2 N (individuals in the second size class that are early-maturing and thus mature), 3 N (late-maturing individuals in the third size class, which are all mature), and 3 3 N (early-maturing individuals in the third size class, which are all mature). Quantities with a

)

  The first two terms on the right-hand side represent recruitment from early-maturing individuals, while the third term represents recruitment from late-maturing individuals. The last two terms represent density-dependent mortality in the first size class and growth into the second size class.

  tion incurs life-history costs arising from energy-budget considerations. Limits in the energy individuals can allocate to different life-history functions naturally result in trade-offs between early maturation and reduced fecundity (

R

  are the basic reproduction ratios for early-and late-maturing individuals, respectively, we see that early maturation is favored by selection denotes the average time that a late-maturing individual entering size class 2,3 i will remain in that size class. Likewise, 2 2 D denotes the average time that an early-maturing 7 individual entering the second size class will remain in that size class. The difference between i D and i D i D on the one hand, and i T on the other, is that the former measure the average time an individual entering a size class will remain there, whereas the latter measure the average time a newborn individual will stay in size class i during the course of its life. Ineq. (5) can be interpreted as describing a trade-off between the fitness gains due to early maturation in the second size class and the fitness losses due to early maturation in the third size class that result from reduced fecundity rate, reduced growth rate, and/or increased natural mortality rate. The expected duration individuals spend in a size class is given by the inverse of their exit rate from that size class, Eqs. (6) into Ineq. (5) and reorganizing shows that early maturation is favored by selection if and only if

  analysis above shows how the evolutionary effects of harvesting the second size class vary with the considered trade-off. To understand what happens when the three trade-offs are considered simultaneously, we denote by * 2 f the right-hand side of Ineq. (7

  right-hand side of Ineq. (

  mature individuals makes late maturation more advantageous, independent of costs in terms of slower growth, if the natural mortality of late-maturing individuals is larger in the third size class than in the second. Otherwise, two disparate outcomes are possible. If the growth costs of early maturation are sufficiently small for Ineq. (14) to hold, i.e., if 2 2 r is not much smaller than 2 r , harvesting mature individuals makes late maturation more advantageous. For larger costs of early maturation, harvesting of mature individuals has the opposite effect, by making early maturation more advantageous.4.4 Trade-off between early maturation and mortality rateExamining Ineq. (13) shows that when early maturation increases the natural mortality rate in the second size class, harvesting of mature individuals makes late maturation more advantalow, but will have the opposite effect when this mortality cost exceeds the intrinsic mortality rate in the third size class.

  turing individuals have very similar life histories, which differ only in the additional offspring early-maturing individuals produce in the second size class. Evidently, early maturation hence is the better strategy, but its advantage may be arbitrarily small. When mature individuals are harvested, this advantage is altered, since early-maturing individuals experience harvest mortality in both the second and the third size class, whereas late-maturing individuals experience harvesting mortality only in the third size class. Provided that the advantage of early maturation is not too large in the absence of harvesting, the differential impact of harvesting mature individuals will thus tip the evolutionary balance in favor of late maturation. This tendency is already well known and understood.The reasons why harvesting mature individuals can induce early maturation when costs of early maturation are high are more subtle, and require appreciating two consequences of intensive harvesting: firstly, such harvesting makes it increasingly unlikely that early-maturing individuals reach the third size class, and secondly, it increasingly equalizes the expected durations early-maturing individuals spend in the second size class and late-maturing individuals spend in the third size class. Together, these effects imply a natural evolutionary advantage for early-maturing individuals under the intensive harvesting of mature individuals, since these individuals avoid the loss of time, and the resultant increase in mortality risk, that latematuring individuals experience in the second size class before they start to reproduce. Using these overarching observations, we now consider each life-history trade-off in turn.When the trade-off between early maturation and fecundity rate is sufficiently strong, the third size class effectively acts as a reproductive sink for early-maturing individuals, exerting a high direct fitness cost of early maturation. In the absence of harvesting, this promotes late maturation. This fitness cost, however, becomes less and less relevant as the intensive harvesting of mature individuals increasingly prevents early-maturing individuals from reaching the third size class. Because of the natural evolutionary advantage of early maturation described above, such harvesting just needs to become intensive enough to tip the evolutionary balance in favor of early maturation.When the trade-off between early maturation and growth rate is sufficiently strong, the growth of early-maturing individuals into the third size class is much impeded. If the natural mortality of late-maturing individuals is smaller in the third size class than in the second (Sect. 4.3), the expected duration that, in the absence of harvesting, late-maturing individuals spend in the third size class always exceeds that of early-maturing individuals in the second size class, which advantages late maturation. As the harvesting of mature individuals is increased, these durations become similar, so the aforementioned advantage of late maturation gradually disappears. At the same time, early-maturing individuals become unlikely to reach the third size class, whether they grow well or not, so the relative importance of the direct fitness cost of early maturation also gradually disappears. What remains is the natural evolutionary advantage of early maturation, which thus inevitably prevails once harvesting is intensive enough.

  while 2 D 2 D and 2 D are the average durations, respectively, that an early-and late-maturing individual entering the second size class will remain there. An individual's average fecundity rate 2 F in the second size class equals the average total number of offspring it has while staying in that size class, divided by the average time 2 T it spends in that size class,

  , are the probabilities with which an early-and late-maturing individual entering the second size class will survive until entering the third size class. The average fecundity in the third size class is then given by

  is positive, we can rewrite Ineq. (A.6early-maturing individual entering the second size class to survive until entering the third size class is 2 2 r D . Similarly, the probability 3 P 3 P 3 of a late-maturing individual entering the second size class to survive until entering the third size class is 2

  this derivative depends only on the last factor in the numerator, and therefore is positive i.e., harvesting mature individuals promotes late maturation when the fecundity costs of early maturation are low. Second, we instead assume that the only trade-off associated with early maturation is a higher natural mortality in the second size class. Denoting by * 2 f the right-hand side of Ineq. i.e., harvesting mature individuals promotes late maturation when the mortality costs of early maturation are low. Third, we instead assume that the only trade-off associated with early maturation is a lower growth rate from the second to the third size class. Denoting by * 2 f the right-hand side of Inin the numerator shows that this derivative is positive if and only is sufficiently close to 1, i.e., harvesting mature individuals promotes late maturation when the growth costs of early maturation are low. For 3 2 m m ! , i.e., when the natural mortality rate in the third size class exceeds that in the second size class, Ineq. (B.7) holds independent of these growth costs. In that case, * 2 f always is an increasing function of mat h , so that harvesting mature individuals always promotes late maturation.

Figures and captions Figure 1 .

 captions1 Figures and captions

Figure 2 .

 2 Figure 2. Numerical illustration of the possible evolutionary consequences of harvesting the second size class (increasing 2 h ). Harvesting the second size class can make maturation in either the second or the third size class advantageous. The population density 1 N of the first size class (thick gray line, right vertical axis) decreases with increased harvesting rate 2 h . If harvesting rates are too high for the population to persist, model predictions are not meaningful. The right-hand side of Ineq. (7) (thin black line, left vertical axis) can either increase with 2 h (top panels) or decrease with 2 h (lower panels), depending on model parameters. When the thin black line is below 2 f , marked here with a dashed horizontal line, Ineq. (7) is fulfilled and selection promotes early maturation. If instead the thin black line is above 2 f , Ineq.

Figure 3 .

 3 Figure 3. Numerical illustration showing that all predicted outcomes of size-selective harvesting (increasing 2 h or 3 h ) are possible. Figure elements are as in Fig. 2. For a trade-off between early maturation and fecundity rate, harvesting the second size class does not affect maturation in either the second or the third size class (top left panel). For a trade-off between early maturation and growth rate, harvesting the second size class promotes late maturation (top center panel). Finally, for a trade-off between early maturation and mortality rate, harvesting promotes early maturation (top right panel). Harvesting the third size class always promotes early maturation (bottom panels). Parameter values are provided in Table C1 of Appendix C.

Figure 4 .

 4 Figure 4. Numerical illustration showing that all analytically predicted outcomes of stageselective harvesting (increasing im m h or m at h ) are possible. Figure elements are as in Fig. 2. Harvesting immature individuals always promotes early maturation (top panels). For a tradeoff between early maturation and fecundity rate, harvesting mature individuals promotes late maturation when the reduction 3 3 f f in fecundity rate is small (middle left panel), and early

mmrr 2 FFTTPP

 2 Figure 2 Top left 0.8 0.31 0.40 1.20 1.50 1.30 0.54 2.1 1.80

Table 1 .

 1 Overview of model variables, parameters, and derived quantities.

2

Acknowledgments

JJP and UD are grateful for financial support by the Netherlands Organization for Scientific Research (NWO), in the form of a six-month postdoctoral scholarship that enabled JJP's work at IIASA. ÅB and UD acknowledges support by the European Marie Curie Research Training Network FishACE (Fisheries-induced Adaptive Changes in Exploited Stocks), funded through the European Community's Sixth Framework Programme (Contract MRTN-CT-2004-005578). UD also acknowledges support by European Science Foundation, the Austrian Science Fund, the Austrian Ministry for Science and Research, and the Vienna Science and Technology Fund, as well as by the European Community's Sixth Framework Programme, through the Specific Targeted Research Project FinE ("Fisheries-induced evolution").

Appendix B: Specific invasion conditions for harvesting mature individuals

We can determine whether the harvesting of mature individuals makes early maturation more The analyses presented in Secs. 3 and 4 apply to the full parameter ranges for which the population persists, as illustrated in Fig. 2. In Figs. 3 and4 we show that all predicted outcomes occur. To facilitate replication of these results, we provide in Table C1 the specific parameterizations that were used to produce each of the panels in Figs. 234.