Ingela Alger

Pau Piret Avila

Juan Bartroli

Péter Bayer

Konrad Dierks

Maria Kleshnina

Laurent Lehmann

Enrico Mattia Salonia

Esteban Muñoz

Jorge Peña

Jörgen Weibull

Evolutionarily stable preferences *

published or not. The documents may come

Introduction

What drives human behavior in their interactions with others? The premise in evolutionary game theory is that each individual is programmed to use a certain strategy. Since the typical life of a human being consists of a large number of different kinds of interactions, Nature should thus have equipped us with automatic play of a certain strategy tailored to each one of them, the ultimate driver of the strategies played in a population being natural selection [1]. Such a worldview is, however, at odds with the idea that we both understand the situations we find ourselves in and choose how to act. But if the latter is an accurate description of how strategies are selected, what then guides the strategy choice?

One theory comes from economics, where the overwhelmingly common premise is that each individual has preferences over the available strategies, which simply means that if presented with a pair of strategies, say A and B, (s)he can tell whether (s)he prefers A to B, (s)he prefers B to A, or is indifferent between the two strategies. In an interaction with others, the answer may depend on what strategies the others are expected to play. Rational behavior requires that a strategy that is preferred over the others be selected by the individual. A Nash equilibrium strategy profile is such that no interactant prefers to alter his/her strategy given the opponents' strategies. In this approach, the individual's preferences is the proximate driver of his/her behavior.

When combining these two strands of thought, the question that follows naturally is: if humans choose strategies in accordance with their preferences, which preferences should we expect evolutionary forces to favor, if any? The literature on preference evolution, initiated by Frank [2] and Güth and Yaari [3], provides some answers to this question. This article summarizes some of the key findings of this literature, found mostly in economics journals, and draws some parallels with related contributions by biologists, notably McNamara, Gasson, and Houston [START_REF] Mcnamara | Incorporating rules for responding into evolutionary games[END_REF], Taylor and Day [START_REF] Taylor | Stability in negotiation games and the emergence of cooperation[END_REF], Akc ¸ay et al. [START_REF] Akc ¸ay | A theory for the evolution of other-regard integrating proximate and ultimate perspectives[END_REF], and Hamilton [START_REF] Hamilton | The genetical evolution of social behaviour. I[END_REF][START_REF] Hamilton | The genetical evolution of social behaviour. II[END_REF].

Strategy evolution in biology 2.1 Framework and definition of ESS

Throughout I adopt the following assumptions, which are in line with the standard evolutionary game theory model [START_REF] Weibull | Evolutionary Game Theory[END_REF]. First, I follow John Maynard Smith by defining a " 'strategy' [as] a behavioral phenotype, i.e. it is a specification of what an individual will do in any situation in which it may find itself" ( [START_REF] Smith | Evolution and the Theory of Games[END_REF] p.10, see also the recent book by McNamara and Leimar [START_REF] Mcnamara | Game Theory in Biology: Concepts and Frontiers[END_REF]); this is also in line with standard vocabulary in non-cooperative game theory, see [START_REF] Fudenberg | Game Theory[END_REF]). Second, I examine a continuum population, in which individuals are randomly matched into pairs to interact, i.e., there is no partner choice. Third, I restrict attention to interactions in which both individuals have access to the same set of strategies, called X, and the material consequences of strategy choices are identical for all individuals in the population. Letting w(x, y) denote the (personal) fitness of an individual using strategy x when the other is using strategy y, I will refer to Γ = ⟨X, w⟩ as the fitness game. 1 The question at hand is whether some resident strategy x, present in a share 1 -ε of the population, would resist the invasion of some mutant strategy y, present in a small share ε > 0 of the population. An evolutionarily stable strategy is then formally defined as follows [START_REF] Weibull | Evolutionary Game Theory[END_REF]: Definition 1. Consider a population in which individuals are uniformly randomly matched into pairs to interact according to the fitness game Γ = ⟨X, w⟩. A strategy x ∈ X is evolutionarily stable (ES) against strategy y ∈ X, y ̸ = x, if there exists εy ∈ (0, 1) such that for all ε ∈ (0, εy ):

(1 -ε) • w (x, x) + ε • w (x, y) > (1 -ε) • w (y, x) + ε • w (y, y) . (1)
And x is an evolutionarily stable strategy (ESS) if it is evolutionarily stable against all y ∈ X, y ̸ = x.

The population being infinitely large and the interactants being matched in a uniformly random manner, any individual is matched with a resident (who plays x) with probability 1 -ε and with a mutant (who plays y) with probability ε. In (1) the left-hand side is thus the average fitness of individuals playing the resident strategy, while the right-hand side is the average fitness of individuals playing the mutant strategy, given the share ε of mutants in the population.

In words, then, an ESS is a strategy which, once it has become prevalent in a population, earns a higher average fitness than any rare mutant strategy.

Remark 1. The present model disregards a number of features that are present in many rich mathematical models in evolutionary biology, in which: the process by which individuals are matched to interact depends on the life-cycle and the population structure; there are individuals of different classes (e.g., men and women, old and young, etc.); there are explicit computations of the expected number of descendants of a single initial mutant; etc. For a recent such general model, see Lehmann and Rousset [START_REF] Lehmann | When do individuals maximize their inclusive fitness?[END_REF]. The model in Definition 1 is closely related to the special case of theirs, in which there is full dispersal, haploidy, and weak selection, the latter assumption ensuring that the probability of being matched with a mutant does not depend on the mutant strategy itself. In this case the invasion fitness (see their equation ( 2)) reduces to w(y, x)/w(x, x), and the condition for x to be uninvadable (see their equation (1)) is equivalent to w(y, x) ≤ w(x, x), a condition which below will be seen to be necessary for x to be ESS. 2This bare-bones model presents the advantage of making it easy to derive insights on preference evolution and understand the challenges that its analysis presents.

While much of the literature focuses on one-shot simultaneous-move games with a finite number of strategies, such as the Prisoner's dilemma or the Hawk-Dove game, the setting in fact encompasses a large number of other kinds interactions, for instance those with an infinite number of pure strategies, and/or where the interactants play in a sequential manner. Several such games are described in detail in Section 1 of the online appendix. 3 While several general results will be presented, I will often refer to the following two examples. (

) 2
The first term is the benefit and the second term the cost from contributing x. The parameter m > 0 measures the baseline marginal benefit from the contribution. The parameter k ∈ (-1, 1) measures the effect that the other individual's contribution y has on the marginal benefit from the contribution: if k ∈ (-1, 0), y reduces the marginal benefit by ky; if k ∈ (0, 1) it increases the marginal benefit by ky; finally, if k = 0 it has no effect on the marginal benefit.

Note that the parameter k, which determines how the other's contribution affects the individual's marginal benefit from contributing, is nothing but the cross-partial derivative of w:

∂ 2 w(x, y)/(∂x∂y) = k. The following definition proposes a general classification of fitness games depending on the sign of this cross-partial derivative. 4Definition 2. In a given fitness game Γ = ⟨X, w⟩, the strategies are:

1. strategic substitutes if ∂ 2 w(x, y)/(∂x∂y) < 0 for all (x, y) ∈ X 2 ;

2. strategic complements if ∂ 2 w(x, y)/(∂x∂y) > 0 for all (x, y) ∈ X 2 ;

3. strategically neutral if ∂ 2 w(x, y)/(∂x∂y) = 0 for all (x, y) ∈ X in which each individual has two actions-Stag (S) and Hare (H), and with payoffs as shown in Figure 1. This game captures interactions in which H ("going for the Hare" in the original story) gives a certain payoff of 1 but coordination by both players on S ("going for the Stag") leads to a higher payoff R > 1. A (mixed) strategy is the probability of playing S, with H being played with the complementary probability. Hence, the strategy set is X = [0, 1] and the fitness function is

w (x, y) = xyR + 1 -x. (3)
While the strategy labels stem from the original story of two hunters, this fitness game can be used to represent a host of interactions where coordination on one strategy enhances the fitness values of both individuals.

S H S R, R 0, 1 H 1, 0 1, 1

An "as if" interpretation of ESS

As a first step towards analysis of preference evolution, it is worth noting that a population in which an ESS is played can be viewed as being populated by individuals who seek to maximize own fitness.

To this end-and also to facilitate description of the analytical challenges that preference evolution sometimes entails-it proves useful to express the difference between the average fitness earned by residents and that earned by mutants as a function of the share of mutants ε, using what is called the score function [START_REF] Bomze | Game Theoretical Foundations of Evolutionary Stability[END_REF]:

S x,y (ε) = (1 -ε) • [w (x, x) -w (y, x)] + ε • [w (x, y) -w (y, y)]. (4)
This function being linear in ε, S x,y (0) ≥ 0 is a necessary condition for x to be ES against y, while S x,y (0) > 0 is a sufficient condition. 5 Moreover, if S x,y (0) = 0 then the slope of the score function must be strictly positive for x to be ES against y. This leads to the following result and also simple test for whether a strategy is evolutionarily stable:

Result 1.

1. If w(x, x) > w(y, x), then x is ES against y.

2. If w(x, x) = w(y, x), then x is ES against y only if w(x, y) > w(y, y).

3. If w(x, x) < w(y, x), then x is not ES against y.

As an illustration, in Example 1 the unique ESS is

x ESS = m/(2 -k).
The first-order condition for such a maximum is ∂w(y, x ESS ) ∂y

|y=x ESS = m + kx ESS -2y |y=x ESS = 0. (6)
This equation has as unique solution the ability to understand the situations they find themselves in, and (b) some preferences that guide their strategy choice in any given interactions? Which preferences should we then expect evolutionary forces to favor, if any?

x ESS = m/(2-k).
3 Preference evolution

In economics, preferences guide behavior

In their analyses of human behavior, economists typically rely on the premise that in any given situation each individual chooses the option that (s)he prefers among all the options that are feasible for him/her. Choosing an option other than a preferred one is deemed irrational. This simple idea is captured by positing that each individual is able to rank all the feasible options.

This ranking is then formalized as a preference ordering, which for any pair of feasible options

A and B tells whether the individual prefers A, B, or is indifferent between A and B. It turns out that under certain conditions, such a preference ordering can be fully described by a function that to each feasible option associates a real number: the number associated with option A is higher (resp. lower) than that associated with option B if and only if the individual prefers A over B (resp. B over A), and the same number to both options if the individual is indifferent between them (see, e.g., [START_REF] Mas-Colell | Microeconomic Theory[END_REF]). Such a function is called a utility function in economics. Here I will instead refer to it as a preference function. In any given situation, an individual is expected to choose that option that yields the highest possible value of the function, since this is the option (s)he prefers. Economists do not interpret this utility maximization literally: it is simply a mathematical tool that the researcher uses to describe behavior that amounts to choosing the preferred item from the feasible set.

In the context of a fitness game Γ = ⟨X, w⟩, an individual's feasible set is the set of strategies X. The individual with whom (s)he is matched to interact-the opponent-also chooses some strategy in the strategy set X. To capture the fact that an individual's ranking over own strategies may depend on what strategy the opponent is expected to use, a preference function is some function u : X 2 → R that to each pair of own and opponent's strategy associates a real number. If the individual strictly prefers some strategy profile, say (x, y), over another, say (x ′ , y ′ ), then u gives a strictly higher number to the former, while if the individual is indifferent, then u gives the same number to both strategy profiles. The question posed at the end of the previous section can now be formally stated as follows: given that personal fitness drives evolutionary success, should we expect evolution to favor the preference function which induces the individual to maximize personal fitness? Definition 3. In any given fitness game Γ = ⟨X, w⟩, a (personal) fitness-maximizer has a preference function that coincides with the personal fitness function, i.e., u(x, y) = w(x, y) for all strategies x, y ∈ X.

For any given strategy that a fitness-maximizer expects the opponent to choose, (s)he chooses a strategy that maximizes own fitness, since this is the strategy that (s)he is encoded to prefer.

Should we expect evolution to lead humans to be such fitness-maximizers? The literature has revealed that information plays a key role in this context.

Interactions under complete information

An example will serve as an introduction. Consider a population in which individuals are matched pairwise to interact according to a common-pool resource game, formalized as fitness game Γ 1 with parameter values m = 10 and k = -1/2 (see (2)). Having in mind situations such as fishing in a common lake, I will use "extraction effort" or simply "effort" to refer to a strategy. The negative value of k means that an individual's effort has a negative impact on the other's marginal return to effort, the idea being that this effort diminishes the size of the fish population available to the other.

Suppose first that all individuals in the population are fitness-maximizers, i.e., each individual has preferences u : X 2 → R defined in [START_REF] Hamilton | The genetical evolution of social behaviour. I[END_REF], namely u(x, y) = (m + ky) x -x 2 , over own and other's efforts, x and y. Suppose further that individuals who are matched to interact can observe each other's preference function, i.e., they interact under complete information ( [START_REF] Fudenberg | Game Theory[END_REF]).

In such an interaction, what pair of strategies will be played? While this question has no simple general answer (see, e.g., [START_REF] Aumann | Epistemic conditions for Nash equilibrium[END_REF]), it is common in economics to apply the Nash equilibrium concept.

Definition 4. In an interaction between two individuals with preferences u : X 2 → R and v : X 2 → R, respectively, a pair of strategies (x * , y * ) is a Nash equilibrium if neither individual would like to deviate from their strategy, given the other's strategy. Formally:

   x * ∈ arg max x∈X u (x, y * ) y * ∈ arg max y∈X v (y, x * ) . (8)
In the example, in an interaction between two fitness-maximizers, (x * , y * ) is a Nash equilibrium if and only if:

   x * ∈ arg max x∈X (10 -y * /2) x -x 2 y * ∈ arg max y∈X (10 -x * /2) y -y 2 . ( 9
)
Differentiability of the preference function together with an unbounded strategy set implies that (x * , y * ) must satisfy the following set of first-order conditions for x * and y * to be maxima (these conditions are also sufficient because of the strict concavity of the preference function):

   10 -y * /2 = 2x * 10 -x * /2 = 2y * . (10)
It follows that the unique Nash equilibrium strategy profile between two fitness-maximizers is (x * , y * ) = (4, 4) (note that this is also the ESS x ESS = m/(2 -k)). Suppose now that another preference function enters this population. For example suppose that some individuals have the preference function

v(x, y) = w(x, y) - 1 2 w(y, x). (11)
For any given strategy used by the individual, x, and any given strategy used by the opponent, y, in this function both the individual's own fitness, w(x, y), and the fitness of the opponent, w(y, x), appears. Since the former enters with a positive sign and the latter with a negative sign, this function means that the individual prefers strategy profiles (x, y) that give a higher fitness to itself and a lower fitness to the opponent. In economics such an individual is said to have spiteful preferences (see, e.g., [START_REF] Alger | A generalization of Hamilton's rule-love others how much?[END_REF]). In an interaction between one fitness-maximizer (with preference function u) and one spiteful individual (with preference function v) a pair of strategies (x, ŷ) is a Nash equilibrium if and only if:

   x ∈ arg max x∈X (10 -ŷ/2) x -x 2 ŷ ∈ arg max y∈X (10 -x/2) y -y 2 -1 2 [(10 -y/2) x -x2 ]. (12)
The following first-order conditions are necessary (and also sufficient because of the strict concavity of the preference functions):

   10 -ŷ/2 = 2x 10 -x/2 + x/4 = 2ŷ. (13)
It follows that there is a unique Nash equilibrium strategy profile (x, ŷ) = (120/31, 140/31) ≈ (3.87, 4.52). Note that ŷ > x * > x. By attaching a negative weight to the other's benefit, the spiteful individual extracts more from the common-pool resource than a fitness-maximizer; in turn, compared to when he interacts with another fitness-maximizer and makes extraction effort x * = 4, here the fitness-maximizer compensates for the higher extraction effort by his opponent by decreasing his extraction effort, to x ≈ 3.87, because the marginal benefit from extracting is reduced when the opponent is extracting more. Calling the preference function u the resident trait, and the spiteful preference function v the mutant trait, and letting ε denote the share of individuals with the mutant trait, it follows that the average fitness of the resident fitness-maximizers is

(1 -ε) • w (x * , x * ) + ε • w (x, ŷ) (14)
while the average fitness of the mutant spiteful individuals is

(1 -ε) • w (ŷ, x) + ε • w (x, x) , (15)
where = w(ŷ, x) > w (x * , x * ) = 16, it follows that for ε close enough to zero, the mutants obtain a strictly higher average fitness than the residents. Following the same logic as in standard evolutionary game theory, we conclude that a population of fitness-maximizers would not resist the invasion by mutants with the spiteful preference function v.

The conclusion that a population of fitness-maximizers would not resist the invasion by some other preference function, here reached in a simple example, has been shown by Heifetz, Shannon, and Spiegel [START_REF] Heifetz | What to maximize if you must[END_REF] to hold for any fitness game Γ = ⟨X, w⟩ such that w is a thrice differentiable function and X is an open subset of R (see also Ok and Vega-Redondo [START_REF] Ok | On the evolution of individualistic preferences: An incomplete information scenario[END_REF]).

They show this general result in a model which encompasses any preference function of the form

u (x, y) = w(x, y) + B(x, y, τ ), (16)
where τ ∈ E ⊆ R is the evolving trait and B is some thrice differentiable function (the fitnessmaximizer is the special case with B(x, y, τ ) = 0 for all (x, y) ∈ X 2 ).

This then leads to the question: which preference function, if any, is evolutionarily stable?

Definition 5. Consider a population in which individuals are uniformly matched into pairs to interact according to the fitness game Γ = ⟨X, w⟩ under complete information about each other's preferences. Let Θ denote the set of all possible preference functions u :

X 2 → R;
any u ∈ Θ is such that there exists a unique Nash equilibrium in each matched pair. Then, a preference function u ∈ Θ is evolutionarily stable under complete information (ESC) against preference function v ∈ Θ if there exists εv ∈ (0, 1) such that for all ε ∈ (0, εv ):

(1 -ε) • w (x * , x * ) + ε • w (x, ŷ) > (1 -ε) • w (ŷ, x) + ε • w (x, x) , (17)
where (x * , x * ) is the unique Nash equilibrium in an interaction between two residents, (x, ŷ) is the unique Nash equilibrium in an interaction between a resident and a mutant, and (x, x) is the unique Nash equilibrium in an interaction between two mutants.

The preference function u is an evolutionarily stable preference function under complete information (ESPFC) if it is evolutionarily stable against all preference functions v ∈ Θ,

v ̸ = u.
In words, an ESPFC is a preference function which, once it has become prevalent in a population, cannot be displaced by any other preference function, the criterion being fitness evaluated at Nash equilibrium. 6 It should be remarked that the definition can be generalized to encompass settings where there exist multiple Nash equilibria; however, most of the literature has restricted attention to settings with a unique Nash equilibrium (an exception is the model of Dekel, Ely, and Yilankaya [START_REF] Dekel | Evolution of preferences[END_REF], but their analysis is on the other hand restricted to fitness games with finite action sets).

In settings where there exists a unique Nash equilibrium in each matched pair, the score function is well defined:

S u,v (ε) = (1 -ε) • [w (x * , x * ) -w (ŷ, x)] + ε • [w (x, ŷ) -w (x, x)]. (18)
Since S u,v is linear in ε, the following result and simple test obtains:

Result 2. Let (x * , x * ) be the unique Nash equilibrium in an interaction between two residents, (x, ŷ) the unique Nash equilibrium in an interaction between a resident and a mutant, and (x, x)

the unique Nash equilibrium in an interaction between two mutants. Then:

1. If w(x * , x * ) > w(ŷ, x), then u is ESC against v. 2. If w(x * , x * ) = w(ŷ, x), then u is ESC against v only if w(x, ŷ) > w(x, x). 3. If w(x * , x * ) < w(ŷ, x), then u is not ESC against v.
A fundamental difference with strategy evolution is that the set of potential preference functions, Θ, is a priori undetermined. Hence, the researcher must make some assumption. Thus far most of the analyses of preference evolution under complete information have examined the parametric class of preferences originally proposed by Bester and Güth in their seminal paper [START_REF] Bester | Is altruism evolutionarily stable?[END_REF]. In a model with the fitness function given in (2), they examine preference functions of the form

u α (x, y) = w(x, y) + α • w(y, x), (19)
where α ∈ [0, 1] is the evolving trait. Bolle [START_REF] Bolle | Is altruism evolutionarily stable? and envy and malevolence? remarks on Bester and Güth[END_REF] and Possajennikov [START_REF] Possajennikov | On the evolutionary stability of altruistic and spiteful preferences[END_REF] generalized the original model by extending the range of possible values of α to R. Like in the example studied in detail above (which corresponded to the special case α = -1/2), a straightforward interpretation is that an individual with such a preference function attaches some weight, α, to the consequences of his/her strategy choice on the opponent. If α > 0, (s)he is willing to reduce own fitness in order to enhance that of the other, i.e., to act in a pro-social manner; economists refer to preferences with α > 0 as altruistic preferences [START_REF] Becker | A theory of social interactions[END_REF]. By contrast, an individual with α < 0 is willing to reduce own fitness in order to reduce that of the other, i.e., to act in an antisocial manner; economists refer to preferences with α < 0 as spiteful ones. Finally, fitnessmaximizing individuals correspond to the special case α = 0. Although this class of preferences thus encompasses altruism, self-interest, and spite, we will simply refer to α as the degree of altruism.

A key insight delivered by the analyses of fitness game Γ 1 by Bester and Güth [START_REF] Bester | Is altruism evolutionarily stable?[END_REF], Bolle [START_REF] Bolle | Is altruism evolutionarily stable? and envy and malevolence? remarks on Bester and Güth[END_REF], and Possajennikov [START_REF] Possajennikov | On the evolutionary stability of altruistic and spiteful preferences[END_REF], is that the evolutionary stability requires α to be of the same sign as k, the parameter that determines whether the strategies are strategic substitutes, strategic complements, or strategically neutral. In order to show how one identifies evolutionarily stable degrees of altruism in interactions under complete information, I turn to the model of Alger

and Weibull [START_REF] Alger | A generalization of Hamilton's rule-love others how much?[END_REF], which generalized the analysis of the same class of preference functions to any fitness game with a differentiable fitness function w such that there exists a unique and differentiable Nash equilibrium in any dyadic interaction; the analysis is further restricted to fitness functions w whereby an individual's strategy has some effect on the other's fitness, i.e., such that ∂w(x, y)/∂y ̸ = 0 for all (x, y) ∈ X 2 (the reason for this will be explained below).

Starting with the behavioral equilibrium in a dyad, let (x * (α, β), x * (β, α)) denote the Nash equilibrium strategy profile in a pair where one individual has degree of altruism α and the other has degree of altruism β. Recall that a Nash equilibrium strategy profile is such that neither individual would prefer to deviate to another strategy, given the other's strategy, i.e.:

   x * (α, β) ∈ arg max x∈X w(x, y) + α • w(y, x) x * (β, α) ∈ arg max y∈X w(y, x) + β • w(x, y). ( 20
)
Since w is differentiable, the pair of first-order conditions is:

   w 1 (x * (α, β), x * (β, α)) + α • w 2 (x * (β, α), x * (α, β)) = 0 w 1 (x * (β, α), x * (α, β)) + β • w 2 (x * (α, β), x * (β, α)) = 0, (21)
where the index 1 (resp. 2) indicates the partial derivative with respect to the first (resp. second) argument. Note that for altruism to have an effect on equilibrium strategies, w 2 (•) ̸ = 0 is necessary, and this explains why the analysis is restricted to such fitness functions, as announced above.

Turning now to the evolutionary stability analysis, let α denote the degree of altruism in the resident preference function and β that in the mutant preference function. Then, Result 2 implies that for the function with degree of altruism α to be an ESPFC, it is necessary that

w(x * (α, α), x * (α, α)) ≥ w(x * (β, α), x * (α, β)) ∀β ∈ R, (22)
or

α ∈ arg max β∈R w(x * (β, α), x * (α, β)). ( 23
)
Since w is differentiable, the necessary first-order condition is

[w 1 (x * (β, α), x * (α, β)) • x * 1 (β, α) + w 2 (x * (β, α), x * (α, β)) • x * 2 (α, β)] |β=α = 0, (24)
where again an index 1 (resp. 2) indicates the partial derivative with respect to the first (resp. second) argument. Recalling from ( 21) that

w 1 (x * (β, α), x * (α, β)) = -β•w 2 (x * (α, β), x * (β, α)),
equation ( 24) can be rewritten

[-β • w 2 (x * (α, β), x * (β, α)) • x * 1 (β, α) + w 2 (x * (β, α), x * (α, β)) • x * 2 (α, β)] |β=α = 0, (25)
which reduces to the following simple equation, which in this setting is necessary for the function with degree of altruism α to be an ESPFC [START_REF] Alger | A generalization of Hamilton's rule-love others how much?[END_REF]:

α • x * 1 (α, α) = x * 2 (α, α). (26)
In fitness game Γ 1 , simple calculations (i.e., solving [START_REF] Bester | Is altruism evolutionarily stable?[END_REF] for w defined in ( 2)) lead to

x * (α, β) = [2 + (1 + α)k]m 4 -(1 + α)(1 + β)k 2 . ( 27
)
In Section 2 of the online Appendix I reproduce the proof (found for different parameter values in [START_REF] Bester | Is altruism evolutionarily stable?[END_REF][START_REF] Bolle | Is altruism evolutionarily stable? and envy and malevolence? remarks on Bester and Güth[END_REF][START_REF] Possajennikov | On the evolutionary stability of altruistic and spiteful preferences[END_REF]) that the unique evolutionarily stable degree of altruism is

α * = k 2 -k (28)
for any k ∈ (-1, 0) ∪ (0, 1), implying that α * has the same sign as k.

Equation [START_REF] Alger | Evolution and Kantian morality[END_REF] shows that the observability of the opponent's preferences drives a wedge between fitness-maximizing preferences and evolutionarily stable preferences. Indeed, the righthand side represents the effect that an individual's preferences have on the opponent's equilibrium strategy, and fitness-maximizing preferences (α * = 0) are evolutionarily stable if and only if this effect is nil (x * 2 (0, 0) = 0). More generally, the characterization in ( 26) unveils a connection between the qualitative nature of the fitness function w and the sign of the evolutionarily stable value of α.

Result 3. [Alger and Weibull [START_REF] Alger | A generalization of Hamilton's rule-love others how much?[END_REF]] Let w be such that ∂w(x, y)/∂y ̸ = 0 for all (x, y) ∈ X 2 .

A preference function of the form [START_REF] Ok | On the evolution of individualistic preferences: An incomplete information scenario[END_REF] with α = α * is an ESPFC (within the set of all such functions) only if x * (α, alpha):

1. α * < 0 if the strategies are strategic substitutes; 2. α * > 0 if the strategies are strategic complements;

3. α * = 0 if the strategies are strategically neutral.

This result generates a clear prediction for the relationship between preferences on the one hand, and the specifics of the fitness function w on the other hand, where w presumably depends on the environment in which the population evolves. In environments with competition over resources, or where it pays off to free-ride on the other, evolutionary stability requires spiteful preferences. The reason is that spiteful preferences induce a commitment by the individual to engage in higher extraction effort (or to free-ride), and this in turn induces a lower extraction effort (or a higher contribution) by the other; hence they can invade a population consisting of fitness-maximizers or altruistic individuals, as in the introductory example. By contrast, in environments where teamwork is valuable, evolutionary stability requires altruistic preferences.

The reason is that altruistic preferences induce a commitment by the individual to engage in higher effort in the team, and this in turn induces a higher effort by the other; for a small enough degree of altruism this enables them to invade a population consisting of fitness-maximizers or spiteful individuals.

The prediction reported in Result 3 is testable if the researcher can measure whether individuals in the population at hand are willing to reduce own fitness in order to enhance that of the other (in which case α > 0), or rather to reduce it (in which case α < 0). If such direct measurement is impossible, the following comparison between the strategy that is employed by individuals in the population at hand and the ESS can be used as an indirect test:

Result 4. Suppose that the fitness game Γ = ⟨X, w⟩ is such that there is a unique ESS, denoted by x ESS , and that w(x, x) is increasing in x when evaluated around x = x ESS . Then in a population where a preference function of the form [START_REF] Ok | On the evolution of individualistic preferences: An incomplete information scenario[END_REF] with α = α * is an ESPFC (within the set of all such functions) and in which the (unique) Nash equilibrium strategy x * (α * , α * ) is employed:

1. x * (α * , α * ) < x ESS if the strategies are strategic substitutes; 2. x * (α * , α * ) > x ESS if the strategies are strategic complements; 3. x * (α * , α * ) = x ESS if the strategies are strategically neutral.
If the fitness game Γ = ⟨X, w⟩ is such that w(x, x) is decreasing in x when evaluated at x = x ESS , the reverse inequalities hold. Indeed, in fitness game Γ 1 , one obtains

x * (α * , α * ) = m 2 -(1 + α * )k . ( 29
)
Recalling that x ESS = m/(2 -k) from ( 6), one indeed obtains x * (α * , α * ) > x ESS if k ∈ (0, 1),

and x * (α * , α * ) < x ESS if k ∈ (-1, 0).

Related models in the biology literature

Following McNamara, Gasson, and Houston [START_REF] Mcnamara | Incorporating rules for responding into evolutionary games[END_REF], a series of contributions in biology have examined the evolutionary stability of negotiation rules. This literature takes interest in fitness games whereby individuals engage in a series of interaction rounds which eventually lead to a "negotiated outcome". Compared to the standard strategy evolution setting, where each individual is programmed to employ a certain strategy, here each individual is programmed with a response rule which specifies the strategy to play in response to the strategy used by the opponent in the previous round. This alternating process converges to a pair of strategies-the negotiated outcome-which the interactants then employ in the remaining rounds.

Like in the preference evolution literature, there is a priori no clear set of possible negotiation rules. McNamara, Gasson, and Houston [START_REF] Mcnamara | Incorporating rules for responding into evolutionary games[END_REF] and Taylor and Day [START_REF] Taylor | Stability in negotiation games and the emergence of cooperation[END_REF] posit the following rule for an individual playing x in response to the opponent's play of y in the previous round:

x = ρ -λ • y. (30)
The evolving trait is the vector (λ, ρ), which represents the slope and the intercept. A population consisting of individuals with the rule (λ, ρ) = (0, x ESS ) would play the ESS x ESS , and this rule is evolutionarily stable. However, there are also rules (λ, ρ) with λ ̸ = 0 that are evolutionarily stable [START_REF] Mcnamara | Incorporating rules for responding into evolutionary games[END_REF][START_REF] Taylor | Stability in negotiation games and the emergence of cooperation[END_REF]. Note that the non-degenerate slope λ ̸ = 0 implies that an individual's behavior is swayed by the opponent's behavior. The similarity with the non-nil effect of an individual's preferences on the opponent's behavior in the model on the evolution of altruistic preferences under complete information (i.e., x * 2 (•, •) ̸ = 0 in [START_REF] Alger | Evolution and Kantian morality[END_REF]) is thus clear. The following remark examines in greater detail the similarities and differences between the seminal models. Remark 2. An interesting parallel can be drawn between the response rule in [START_REF] Van Schaik | The Primate Origin of Human Behavior[END_REF] and the model analyzed by Bester and Güth [START_REF] Bester | Is altruism evolutionarily stable?[END_REF]. Recalling the fitness function that they posit (see (2)), an individual with altruistic preferences chooses a strategy x that maximizes the following expression, where y is the opponent's strategy:

(m -x + ky) x + α • (m -y + kx) y (31)
The necessary (and sufficient) first-order condition for this maximization is

m -2x + ky + α • ky = 0, (32)
or x = m 2 + k(1+α) 2 y, (33)
In other words, the best response of an individual with degree of altruism α to the opponent's strategy is equivalent to the response rule examined by McNamara, Gasson, and Houston [START_REF] Mcnamara | Incorporating rules for responding into evolutionary games[END_REF] and Taylor and Day [START_REF] Taylor | Stability in negotiation games and the emergence of cooperation[END_REF] (see [START_REF] Van Schaik | The Primate Origin of Human Behavior[END_REF]) for ρ = m 2 and λ = -k(1+α)

2

. Hence, the system of necessary conditions for a Nash equilibrium strategy profile in a dyad with degrees of altruism (α, α ′ ) at which Bester and Güth [START_REF] Bester | Is altruism evolutionarily stable?[END_REF] evaluate fitness,

   x * (α, α ′ ) = m 2 + k(1+α) 2 • x * (α ′ , α) x * (α ′ , α) = m 2 + k(1+α ′ ) 2 • x * (α, α ′ ) (34)
coincides with the system of equations that define the negotiated outcome in a dyad with response rules (ρ, λ), (ρ ′ , λ ′ ) at which McNamara, Gasson, and Houston [START_REF] Mcnamara | Incorporating rules for responding into evolutionary games[END_REF] and Taylor and Day [START_REF] Taylor | Stability in negotiation games and the emergence of cooperation[END_REF] evaluate fitness,

   x * ((ρ, λ), (ρ ′ , λ ′ )) = ρ -λ • x * ((ρ ′ , λ ′ ), (ρ, λ)) x * ((ρ ′ , λ ′ ), (ρ, λ)) = ρ ′ -λ ′ • x * ((ρ, λ), (ρ ′ , λ ′ )) (35)
if ρ = ρ ′ = m/2, λ = -k(1 + α)/2, and λ ′ = -k(1 + α ′ )/2.
This comparison highlights two differences between Bester and Güth [START_REF] Bester | Is altruism evolutionarily stable?[END_REF] on the one hand, and McNamara, Gasson, and Houston [START_REF] Mcnamara | Incorporating rules for responding into evolutionary games[END_REF] and Taylor and Day [START_REF] Taylor | Stability in negotiation games and the emergence of cooperation[END_REF] on the other hand. First, in the latter both the slope and the intercept of the response rule are evolving traits, while in the former only the slope evolves.

Second, they do not use the same fitness function.

This remark brings us to the contribution by Akc ¸ay et al. [START_REF] Akc ¸ay | A theory for the evolution of other-regard integrating proximate and ultimate perspectives[END_REF], which builds a nice bridge between the biology literature on the evolution of negotiation rules on the one hand, and the economics literature on preference evolution under complete information on the other hand. In a model with the fitness function

w(x, y) = y 1/2 -x 2 , (36)
they consider preference functions of the form

u β (x, y) = w(x, y) • [w(y, x)] β , (37)
and let β ≥ 0 be the evolving trait. They derive the best response of an individual with such preferences, they determine the conditions under which a negotiation phase would converge to Nash equilibrium in a complete information game between two individuals with such preferences, and they characterize the evolutionarily stable value of β. They further derive a result in a general model with generic but differentiable fitness and preference functions, such that in each dyad there exists a unique Nash equilibrium. This result can be described as follows:

Result 5. [Akc ¸ay et al. [START_REF] Akc ¸ay | A theory for the evolution of other-regard integrating proximate and ultimate perspectives[END_REF]] Suppose that the fitness game Γ = ⟨X, w⟩ is such that w(x, x) is increasing in x, and suppose that there is a unique ESS, denoted x ESS . Then in a population where a preference function of the form (37) with β = β * is an ESPFC and in which the (unique)

Nash equilibrium strategy x * (β * , β * ) is employed:

1. x * (β * , β * ) > x ESS if the strategies are strategic complements; 2. x * (β * , β * ) = x ESS if the strategies are strategically neutral.
The qualitative nature of this result is in line with that of a subset of the results found by Alger and Weibull [START_REF] Alger | A generalization of Hamilton's rule-love others how much?[END_REF] in the case of altruistic/spiteful preference functions (see Result 4 above), an observation which would be expected in light of the following remark.

Remark 3. It is well-known in economics that any preference ranking over items in an individual's choice set that can be described by some preference function u, can equally well be described by any positive monotone transformation of u. Taking the logarithm of the function posited by Akcay et al. [START_REF] Akc ¸ay | A theory for the evolution of other-regard integrating proximate and ultimate perspectives[END_REF] (see [START_REF] Miettinen | Revealed preferences in a sequential prisoners' dilemma: a horse-race between six utility functions[END_REF]), and defining, ũ (x, y) = ln w(x, y)

+ β • ln w(y, x), (38)
it is clear that this class of preference functions is qualitatively similar to the one adopted in the economics literature that built on Bester and Güth [START_REF] Bester | Is altruism evolutionarily stable?[END_REF] (see [START_REF] Ok | On the evolution of individualistic preferences: An incomplete information scenario[END_REF]).

It is still an open question whether the qualitative nature of Results 4 and 5 generalizes to other preference function classes. As mentioned earlier, it is a priori not clear which preference classes should be examined by modelers, a question that will be brought up again in the discussion section.

Interactions under incomplete information

In this subsection we will see that fitness-maximizers do prevail when interactions take place under incomplete information. By contrast to interactions that take place under complete information, in interactions where the individuals cannot observe each other's preference function their behavior cannot be swayed by the opponent's preference function. However, an individual may still adapt behavior to the distribution of preference functions present in the population. This is the assumption adopted in the analyses of preference evolution under incomplete information [START_REF] Ok | On the evolution of individualistic preferences: An incomplete information scenario[END_REF][START_REF] Dekel | Evolution of preferences[END_REF][START_REF] Alger | Homo moralis-preference evolution under incomplete information and assortative matching[END_REF]. This section summarizes results by closely following the modeling assumptions of Alger and Weibull [START_REF] Alger | Homo moralis-preference evolution under incomplete information and assortative matching[END_REF], for a reason that will become clear below. The set Θ of potential preferences is assumed to be less restrictive than above: it is the set of all continuous functions u : X 2 → R.

Let a population state s = (u, v, ε) be defined by the resident preference function u ∈ Θ, the mutant preference function v ∈ Θ, and the share ε of mutants. Under the same matching protocol as in the standard framework-i.e., that any individual faces a probability ε of being matched with a mutant-the criterion used in the literature is fitness evaluated at typehomogenous Bayesian Nash equilibrium strategy profiles (below these will simply be referred to as equilibrium strategy profiles, or equilibria), defined as follows.

Definition 6. In any state s = (u, v, ε) ∈ Θ 2 × (0, 1), a strategy pair (x * , y * ) ∈ X 2 is a type-homogenous Bayesian Nash Equilibrium (BNE) if

   x * ∈ arg max x∈X (1 -ε) • u (x, x * ) + ε • u (x, y * ) y * ∈ arg max y∈X (1 -ε) • v (y, x * ) + ε • v (y, y * ) . (39)
The first (resp. second) equation says that a resident (resp. a mutant) chooses a strategy that maximizes the expected value of the preference function u (resp. v), where the expectation is taken over the value that the preference function takes in a match with another resident (who plays x * ), which realizes with probability 1 -ε, and the value that it takes in a match with a mutant (who plays y * ), which realizes with probability ε. Type-homogeneity means that all individuals with the same preference function (or preference type) use the same strategy. For example, consider again fitness game Γ 1 , and assume that the resident preference function is of the form [START_REF] Ok | On the evolution of individualistic preferences: An incomplete information scenario[END_REF] with degree of altruism α while the mutant preference function is also of the form [START_REF] Ok | On the evolution of individualistic preferences: An incomplete information scenario[END_REF] but with degree of altruism β ̸ = α. It is straightforward to verify that there then exists a unique type-homogenous Bayesian Nash equilibrium strategy profile, which depends on the share of mutants ε as follows:

     x * (ε) = m[2+ε(α-β)k] 2[2-(1+α)k+ε(α-β)k] y * (ε) = m[2-(1-ε)(α-β)k] 2[2-(1+α)k+ε(α-β)k] (40)
Given some equilibrium strategy profile (x * (ε), y * (ε)) associated with population state s = (u, v, ε), define the equilibrium fitnesses of residents and mutants:

W u (x * (ε), y * (ε), ε) = (1 -ε) • w(x * (ε), x * (ε)) + ε • w(x * (ε), y * (ε)) (41)
W v (x * (ε), y * (ε), ε) = (1 -ε) • w(y * (ε), x * (ε)) + ε • w (y * (ε), y * (ε)) (42)
By contrast to the analyses of interactions under complete information, the typical approach for interactions under incomplete information consists in minimally constraining the set of possible preference functions, Θ. In particular it turns out that it is possible to derive general results even for settings in which there are states s = (u, v, ε) with multiple equilibria.

equilibrium profile (1/4, 1/4). 7 Suppose now that a share ε of the population strongly dislike killing stags, and has the mutant preference function of the form (44) with K = -4. Then a BNE satisfies:

   x * ∈ arg max x∈{0,1/4,1} (1 -ε) • (4xx * + 1 -x) + ε • (4xy * + 1 -x) y * ∈ arg max y∈{0,1/4,1} (1 -ε) • (-2yx * + 1 -y) + ε • (-2yy * + 1 -y). (45)
Clearly, for the mutants the pure strategy H is strictly dominant, meaning that they strictly prefer it to the other two strategies, independent of the opponent's strategy. Hence, in any BNE in this population, the mutants use strategy y * = 0. Hence, the residents choose some strategy such that

x * ∈ arg max x∈{0,1/4,1} (1 -ε) • (4xx * + 1 -x) + ε • (1 -x). (46)
This fixed-point problem admits the solution x * = 0 for any ε ∈ (0, 1) and the solution x * = 1

for any ε ∈ (0, 3/4], implying that (x * , y * ) = (1, 0) and (x * , y * ) = (0, 0) are both BNE for small values of ε. However, for any ε > 0, a resident is no longer indifferent between H and S when all the other residents play

x = 1/4, since (1 -ε)4/4 < 1. Hence (x * , y * ) = (1/4, 0)
is not a BNE, and there is thus a discontinuity at ε = 0: while residents may use the mixed strategy 1/4 when there are no mutants around, they no longer do so as soon as the mutant preference function at hand is present in the population.

The potential existence of multiple equilibria together with potential discontinuities in the set of BNE introduces a sharp contrast with the linearity in ε of the score functions under strategy evolution (4) and under preference evolution under complete information [START_REF] Heifetz | What to maximize if you must[END_REF], which implied that analysis of the score function at ε = 0 was sufficient to check evolutionary stability (recall Results 1 and 2). In spite of these challenges, there are conditions that render general analysis possible, even for settings with multiple equilibria. In particular, several authors have shown that the fitness-maximizing preference function is evolutionarily stable.

Prior to stating one such result, there is one additional issue that needs to be addressed, however. Recall that the goal here is to minimally constrain the set of possible preference functions, Θ. In particular, Θ contains several functions that give rise to the same strategy choices (recall Remark 3), and hence that in general there will be no preference function that is ESI against all the other preference functions (since residents must obtain a strictly higher average fitness than mutants for it to be evolutionarily stable, see Definition 7). The following definition of behavioral alikes addresses this issue.

Definition 8. Let X 0 be the set of type-homogenous Nash equilibria in a population consisting solely of fitness-maximizers. A preference function f is a behavioral alike to fitness-maximizers if there exists some x 0 ∈ X 0 and some z ∈ X such that z ∈ arg max x∈X w(x, x 0 ) and z ∈ arg max x∈X f (x, x 0 ).

In words (and somewhat loosely) a behavioral alike to fitness-maximizers is a preference type that would be willing to play a strategy z (perhaps different from x 0 ) that the fitnessmaximizer would also be willing to play, given that the opponent plays some x 0 ∈ X 0 . The following result identifies sufficient conditions for the fitness-maximizing preference function to be evolutionarily stable. This result is found in [START_REF] Alger | Evolution and Kantian morality[END_REF] (it is a slight variation of the result as stated in the earlier article [START_REF] Alger | Homo moralis-preference evolution under incomplete information and assortative matching[END_REF], the difference stemming from a slight difference in how behavioral alikes are defined; the core of the result is not affected, however).

Result 6. If the strategy set X is compact and convex, and all the preference functions in Θ as well as the fitness function w are continuous, then the fitness-maximizing preference function (see [START_REF] Hamilton | The genetical evolution of social behaviour. I[END_REF]) is ESI against any preference function that is not its behavioral alike.

The topological properties stated in the result ensure that the correspondence, which to each population state s = (u, v, ε) ∈ Θ 2 × (0, 1) associates the set of equilibrium strategy profiles, is upper-hemicontinuous in ε. Hence, even if the introduction of an infinitesimal share of mutants sways the equilibrium strategy of the residents away from the equilibrium strategy played in the absence of mutants, the "new" equilibrium strategy is arbitrarily close to some strategy that the fitness-maximizers could have played in the absence of mutants. Continuity of the fitness function then implies that any mutant which is not a behavioral alike to fitness-maximizers obtains a strictly lower equilibrium fitness than the fitness-maximizers.

Ok and Vega-Redondo [START_REF] Ok | On the evolution of individualistic preferences: An incomplete information scenario[END_REF] adopt similar topological properties, and they show that fitnessmaximizers are robust to the entry of non-fitness-maximizers even in finite but large enough populations. By contrast, in small populations the entry of mutants makes the resident fitnessmaximizers shift their strategy away from any strategy they would have played in the absence of mutants in many fitness games, and the result no longer holds (this is reminiscent of the fact that a strategy that is ES in infinite populations is not necessarily ES in finite populations [START_REF] Schaffer | Evolutionarily stable strategies for a finite population and a variable contest size[END_REF]).

Why should evolutionary biologists take interest in Result 6? After all, the prediction is not surprising: under incomplete information fitness-maximizers prevail in a panmictic setting. I'd argue that the result is valuable because it solves a dilemma inherent to fitness games with multiple evolutionarily stable strategies. Indeed, for such fitness games analysis under the assumption that selection operates at the level of strategies delivers no clear prediction. By contrast, analysis under the assumption that selection operates at the level of preferences predicts that one particular preference function stands out as being viable from an evolutionary perspective. 8 4 Strategy and preference evolution in the presence of relatedness All of the analyses summarized above were derived in the panmictic setting [START_REF] Smith | Evolution and the Theory of Games[END_REF], where the probability of being matched with a mutant is the same for residents and mutants, which implies that as the share of mutants tends to 0 the probability that a mutant is matched with another mutant tends to 0 as well. Relatedness means that a rare mutant is more likely to be matched with another mutant than a resident is to be matched with a mutant [START_REF] Wright | Coefficients of inbreeding and relationship[END_REF][START_REF] Hamilton | The genetical evolution of social behaviour. I[END_REF][START_REF] Hamilton | The genetical evolution of social behaviour. II[END_REF]. Relatedness arises in naturally structured populations [START_REF] Rousset | Genetic Structure and Selection in Subdivided Populations[END_REF] and is part of the environment of evolutionary adaptation of the human lineage [START_REF] Van Schaik | The Primate Origin of Human Behavior[END_REF]. It has been shown to depend on a number of factors, such as migration rates, and even the strategies present in the population (see, e.g., [START_REF] Lehmann | When do individuals maximize their inclusive fitness?[END_REF]). However, here I adopt the reduced-form approach originally proposed by Bergstrom [START_REF] Bergstrom | The algebra of assortative encounters and the evolution of cooperation[END_REF] for a model of strategy evolution (the term commonly used in the economics literature is assortativity of the matching process rather than relatedness).

Definition 9. For any given resident preference function u and mutant preference function v, 8 Note, however, that for such fitness games the set of Nash equilibria in a population consisting of fitnessmaximizers does not necessarily coincide with the set of evolutionarily stable strategies. For example, in the Stag-Hunt game with the (convex) set of strategies X = [0, 1] and parameter R = 3 in the fitness function (3), the set of symmetric Nash equilibrium strategies is {0, 1/3, 1}, while the set of evolutionarily stable strategies is {0, 1}. This can be understood by recalling Result 1: suppose there is strategy evolution, that the resident strategy is x = 1/3 and the mutant strategy is y = 0; then w(x, x) = w(y, x) = 1, but w(x, y) = 2/3 < 1 = w(y, y), and hence x is not ES against y. However, any ESS is a symmetric Nash equilibrium in a game between two fitness-maximizers (see [START_REF] Weibull | Evolutionary Game Theory[END_REF]). and share ε ∈ (0, 1) of mutants, let Pr [v|u, ε] denote the probability that a resident is matched with a mutant, and Pr [v|v, ε] the probability that a mutant is matched with another mutant.

Assuming that the conditional probability functions are continuous in ε, and that the probability that a mutant is matched with another mutant tends to some number r ∈ [0, 1] as the share of mutants tends to 0, i.e.,

lim ε→0 Pr [v|v, ε] = r, ( 47
)
the number r measures relatedness between interactants.

The analyses summarized above correspond to the special case Pr [v|u, ε] = Pr [v|v, ε] = ε for all (u, v, ε), and thus r = 0.

Prior to examining preference evolution, it is worth noting that the definition of an evolutionarily stable strategy is readily extended to encompass relatedness (an early analysis can be found in [START_REF] Grafen | The hawk-dove game played between relatives[END_REF]):

Definition 10. Consider a population in which individuals are randomly matched into pairs to interact according to the fitness game Γ = ⟨X, w⟩. A strategy x ∈ X is evolutionarily stable (ES) against strategy y ∈ X, y ̸ = x, if there exists εy ∈ (0, 1) such that for all ε ∈ (0, εy ):

Pr [x|x, ε] • w (x, x) + Pr [y|x, ε] • w (x, y) > Pr [x|y, ε] • w (y, x) + Pr [y|y, ε] • w (y, y) . ( 48
)
And x is an evolutionarily stable strategy (ESS) if it is evolutionarily stable against all y ∈ X,

y ̸ = x.
Hence, in a setting with relatedness r ∈ [0, 1], Result 1 generalizes to (see [START_REF] Alger | Homo moralis-preference evolution under incomplete information and assortative matching[END_REF]):

Result 7. 1. If w(x, x) > w(y, x) + r • [w(y, y) -w(y, x)], then x is ES against y. 2. If w(x, x) = w(y, x) + r • [w(y, y) -w(y, x)], then x is ES against y only if w(x, y) > w(y, y) + r • [w(y, y) -w(y, x)].
3. If w(x, x) < w(y, x) + r • [w(y, y) -w(y, x)], then x is not ES against y.

Remark 4. Recalling Remark 1 for the panmictic case, the model in Definition 10 can again be seen as the special case of the general model by Lehmann and Rousset [START_REF] Lehmann | When do individuals maximize their inclusive fitness?[END_REF], in which there is relatedness r, haploidy, and weak selection. In this case the invasion fitness (their equation ( 2)) reduces to [(1 -r)w(y, x) + rw(y, y)]/w(x, x), and the condition for x to be uninvadable (their equation ( 1)) is equivalent to (1 -r)w(y, x) + rw(y, y) ≤ w(x, x), a condition which is necessary for x to be ESS.

Turning now to a summary of the results for preference evolution, the distinction between complete and incomplete information is still called for.

Interactions under complete information

Starting with interactions under complete information and examining, as above, preference functions of the form ( 19) whereby an individual attaches some weight α R to the other's individual fitness, the definition of an evolutionarily stable preference function under complete information (see Definition 5) readily generalizes to encompass relatedness by replacing 1 -ε and ϵ by the appropriate conditional matching probabilities in [START_REF] Alger | A generalization of Hamilton's rule-love others how much?[END_REF], to obtain:

Pr [u|u, ε] • w (x * , x * ) + Pr [v|u, ε] • w (x, ŷ) > Pr [u|v, ε] • w (ŷ, x) + Pr [v|v, ε] • w (x, x) . ( 49
)
The score function in [START_REF] Heifetz | What to maximize if you must[END_REF] thus generalizes to:

S u,v (ε) = Pr [u|u, ε] • w (x * , x * ) + Pr [v|u, ε] • w (x, ŷ) -Pr [u|v, ε] • w (ŷ, x) -Pr [v|v, ε] • w (x, x) . ( 50
)
Recall that differentiability of this function facilitates analysis, since it is then sufficient to examine the value (and sometimes the derivative) of S u,v at ε = 0 to establish whether u is ESC against v. Such differentiability obtains if the conditional probability functions are differentiable. Positing such differentiability, Result 3 generalizes to:

Result 8. [Alger and Weibull [START_REF] Alger | A generalization of Hamilton's rule-love others how much?[END_REF]] In a population where the matching process entails relatedness r ∈ [0, 1], and the conditional probability functions are differentiable in ε, a preference function of the form [START_REF] Ok | On the evolution of individualistic preferences: An incomplete information scenario[END_REF] with α = α * is an ESPFC only if:

1. α * < r if the strategies are strategic substitutes;

2. α * > r if the strategies are strategic complements;

3. α * = r if the strategies are strategically neutral.

The complete information setting may be particularly well suited to represent interactions between relatives, especially in view of the fact that relatives often have the opportunity to observe each other's behaviors for many years. Given that the value of α determines how willing individuals are to act generously towards the other, the result suggests that evolution may have led to variation in the degree of intra-family generosity across different regions of the world. Indeed, in our evolutionary past the qualitative nature of the fitness game in any given region may have depended on the local ecological conditions. For example, in Arctic regions whale hunting was common, and such hunting arguably involves strategic complementarities. By contrast, in agricultural societies, food production would arguably have been a game in which production efforts were strategic substitutes. Furthermore, even for a given category of fitness game (i.e., whether strategies are strategic substitutes, complements, or neutral), the specifics of the fitness game is expected to matter. For example, in the production-and-sharing fitness game studied in [START_REF] Alger | Kinship, incentives, and evolution[END_REF], where the strategies are strategic substitutes, the evolutionarily stable degree of altruism is found to be lower in harsh than in generous environments, for a given relatedness r.

Interactions under incomplete information

Turning now to interactions under incomplete information, a straightforward generalization of the panmictic setting examined above is possible. Inserting the conditional probabilities into the system of best-response equations [START_REF] Andreoni | Impure altruism and donations to public goods: A theory of warm-glow giving[END_REF] in Definition 6 of a Bayesian Nash equilibrium,

   x * ∈ arg max x∈X Pr [u|u, ε] • u (x, x * ) + Pr [v|u, ε] • u (x, y * ) y * ∈ arg max y∈X Pr [u|v, ε] • v (y, x * ) + Pr [v|v, ε] • v (y, y * ) , (51)
and into the equilibrium fitnesses of residents and mutants (see [START_REF] Rabin | Incorporating fairness into game theory and economics[END_REF] and ( 42)),

W u (x * , y * , ε) = Pr [u|u, ε] • w(x * , x * ) + Pr [v|u, ε] • w(x * , y * ) (52)
W v (x * , y * , ε) = Pr [u|v, ε] • w(y * , x * ) + Pr [v|v, ε] • w(y * , y * ), (53)
the definition of an evolutionarily stable preference function under incomplete information applies as is (see Definition 7).

As was the case for preference evolution under incomplete information in the panmictic setting, the goal is to impose minimal restrictions on the set of potential preferences. The simple fitness-maximizing preference function (see [START_REF] Hamilton | The genetical evolution of social behaviour. I[END_REF]) is no longer evolutionarily stable, however.

Instead, the analyses in [START_REF] Alger | Homo moralis-preference evolution under incomplete information and assortative matching[END_REF][START_REF] Alger | Evolution and Kantian morality[END_REF] reveal that evolution favors Homo moralis preferences: Definition 11. An individual is a Homo moralis if his/her preference function is of the form

u κ (x, y) = (1 -κ) • w (x, y) + κ • w (x, x) , (54)
for some κ ∈ [0, 1], his/her degree of morality.

While it was the mathematical analysis that led to the "discovery" of this preference class, the name Homo moralis was inspired by the second term in [START_REF] Croson | Gender differences in preferences[END_REF], which can be interpreted as a concern for universalization, reminiscent of Kant's reasoning [START_REF] Kant | English: Groundwork of the Metaphysics of Morals[END_REF]: what would happen (to the individual's fitness) if the individual's strategy was universalized? The first term being the individual's fitness given own and opponent's actual strategies, the Homo moralis preference function can be thought of as representing a form of partial Kantian moral concern (see also [START_REF] Bergstrom | On the evolution of altruistic ethical rules for siblings[END_REF] for a similar "as if" interpretation, in a model with strategy evolution for interactions between siblings).

As before, it is necessary to precisely define behavioral alikes (recall Definition 8).

Definition 12. Let X r be the set of type-homogenous Nash equilibria in a population consisting solely of Homo moralis with degree of morality κ = r. A preference function f is a behavioral alike to such Homo moralis if there exists some x r ∈ X r and some z ∈ X such that z ∈

arg max x∈X (1 -r) • w(x, x r ) + r • w(x, x) and z ∈ arg max x∈X f (x, x r ).
In words, a behavioral alike to a Homo moralis with a degree of morality κ = r is a preference type that would be willing to play a strategy that such a Homo moralis would also be willing to play, given that the opponent uses some type-homogenous Bayesian Nash equilibrium strategy in a monomorphic population of such Homo moralis, x r ∈ X r . Then [START_REF] Alger | Evolution and Kantian morality[END_REF] show the following result (a slight variation can be found in [START_REF] Alger | Homo moralis-preference evolution under incomplete information and assortative matching[END_REF], where the main difference is a slight and unimportant difference in the definition of behavioral alikes, already referred to above).

Result 9. If the strategy set X is compact and convex, and all the preference functions in Θ as well as the fitness function w are continuous, then the Homo moralis preference function with degree of morality equal to the coefficient of relatedness, κ = r (see [START_REF] Mcnamara | Game Theory in Biology: Concepts and Frontiers[END_REF]) is ESI against any preference function that is not its behavioral alike.

A population of Homo moralis resists entry by mutants because their preferences make them select a strategy that pre-empts entry by mutants. To see this, note first that the average fitness of vanishingly rare mutants (see ( 53)), who play some strategy, say z, tends to the following value as ε tends to 0:9

(1 -r) • w(z, x r ) + r • w(z, z), (55)
where x r is some Nash equilibrium strategy in a monomorphic population consisting of Homo moralis:

x r ∈ arg max x∈X (1 -r) • w(x, x r ) + r • w(x, x). (56)
A mutant preference type that is not a behavioral alike to Homo moralis with degree of morality κ = r necessarily plays a strategy which does not belong to the set

arg max x∈X (1 -r) • w(x, x r )+r •w(x, x). Hence, (1-r)•w(z, x r )+r •w(z, z) < (1-r)•w(x r , x r )+r •w(x r , x r ) =
w(x r , x r ). This in turn implies that the average fitness of vanishingly rare mutants is strictly smaller than the average fitness of residents, which is arbitrarily close to w(x r , x r ). In sum, a population of Homo moralis resists entry by mutants because their preferences make them select a strategy which maximizes the average fitness of vanishingly rare mutants, given that the residents play this strategy.

Related models in the biology literature

Biologists will of course have recognized Hamilton's rule [START_REF] Hamilton | The genetical evolution of social behaviour. I[END_REF][START_REF] Hamilton | The genetical evolution of social behaviour. II[END_REF] in the results presented in this section (expressed at the relevant level of selection [START_REF] Okasha | Maynard Smith on the levels of selection question[END_REF], i.e., either at the level of strategies, or at the level of preferences). One may therefore wonder whether analysis of preference evolution brings fundamental new insights to the rich literature that followed in Hamilton's footsteps. I

argue that the answer is positive for two reasons.

First, economists propose a rigorous and general analysis of behavioral Nash equilibria that is absent from the biology literature. As shown above, this has made it possible to tackle new questions such as: "What can be said about evolutionarily stable preferences when there are multiple Nash equilibria?", and "What can be said about evolutionarily stable preferences when the set of potential preference functions is the set of all continuous functions?"

Second, and more speculatively, there may be an interesting parallel to be drawn between, on the one hand, the preference evolution literature, and on the other hand the literature that asks whether individuals who use an ESS can be viewed as if they are maximizing some goal function (recently reviewed by Lehmann and Rousset [START_REF] Lehmann | When do individuals maximize their inclusive fitness?[END_REF]; see also [START_REF] Bergstrom | On the evolution of altruistic ethical rules for siblings[END_REF] for an early such analysis in the economics literature). Specifically, the idea is that this parallel may perhaps shed additional light on the comparison of the gene-centered and the actor-centered view, discussed a length in [START_REF] Lehmann | When do individuals maximize their inclusive fitness?[END_REF] and in works surveyed therein.

The argument rests on a comparison between the altruistic preference function, defined in [START_REF] Ok | On the evolution of individualistic preferences: An incomplete information scenario[END_REF], and the Homo moralis preference function, defined in ( 54). An individual who has an altruistic preference function evaluates the consequences of his/her strategy on own individual fitness as well as on the interactant's individual fitness. This is reminiscent of the actor-centered view, which aggregates the effects of the individual on the others. An individual who has a Homo moralis preference function evaluates the consequences of his/her strategy on own individual fitness as well as on what his/her own individual fitness would be if, hypothetically, the interactant were to use the same strategy instead of the one (s)he is actually using. This is reminiscent of the gene-centered view, which aggregates the effects of others' behaviors on the individual's fitness.

The point is that the literature on preference evolution under incomplete information shows that Homo moralis preferences are more robust than altruistic preferences, in the following sense. Recall from Result 9 that Homo moralis preferences with degree of morality κ = r are evolutionarily stable. It turns out that there are fitness games for which the set of Nash equilibria in a monomorphic population of altruists with degree of altruism α = r coincides with the set of Nash equilibria in a monomorphic population of Homo moralis with degree of morality κ = r.

For such games neither preference function is evolutionarily stable against the other, and they are both evolutionarily stable against preference functions which are not behavioral alikes of Homo moralis. However, there are also fitness games in which the said set of Nash equilibria do not coincide. In such games, only Homo moralis preferences with degree of morality κ = r are evolutionarily stable, while altruistic preferences with degree of altruism α = r may fail to be evolutionarily stable. I will now show that the former situation arises in fitness game Γ 1 while the latter case arises in fitness game Γ 2 .

In fitness game Γ 1 , when interacting with an individual who uses strategy y a Homo moralis chooses a strategy that satisfies the necessary first-order condition for x to maximize u κ (x, y):

(1 -κ)(m + ky -2x) + κ(m + 2kx -2x) = 0. (57)
This implies that there is a unique any Nash equilibrium strategy in a monomorphic population consisting of Homo moralis with degree of morality κ = r, which is the unique solution to the equation

(1 -r)(m + kx -2x) + r(m + 2kx -2x) = 0. (58)
In this fitness game an individual with altruistic preferences chooses a strategy that satisfies the necessary first-order condition for x to maximize u α (x, y)

m + ky -2x + αky = 0, (59)
implying that the unique Nash equilibrium strategy in a monomorphic population consisting of such altruists with degree of altruism α = r solves

m + kx -2x + rkx = 0. (60)
Equations ( 58) and ( 60) both yield x = m/[2-(1+r)k]: the set of equilibria in both monomorphic populations being the same, both preference functions are evolutionarily stable against preference functions that are not behavioral alikes of Homo moralis with degree of morality κ = r.

Turning now to fitness game Γ 2 , a strategy x α is a symmetric Nash equilibrium in a monomorphic population consisting of altruists with degree of altruism α if and only if an individual would not like to deviate to any other strategy, given that the other plays x α :

(1 + α) (x α ) 2 R + 1 -x α ≥ (1 + α)xx α R + 1 -x + α(1 -x α ) ∀x ∈ [0, 1]. (61)
For any value of α ∈ [0, 1], there are three such equilibrium strategies: {0, 1/[(1 + α)R], 1}.

Turning now to Homo moralis preferences, a strategy x κ is a symmetric Nash equilibrium in a monomorphic population consisting of Homo moralis with degree of morality κ if and only if:

(x κ ) 2 R + 1 -x κ ≥ (1 -κ)(xx κ R + 1 -x) + κ (x 2 R + 1 -x) ∀x ∈ [0, 1]. (62)
Here the set of such equilibrium strategies is {0, 1} if and only if κ ∈ [0, 1/R], and {1} otherwise. In this fitness game, the Homo moralis preference function with κ = r is evolutionarily stable (as per Result 9). By contrast, the altruistic preference function with α = r is not. To see this, assume that this is the resident preference function and consider a mutant preference function that induces the mutants to choose y = 1. Then in one of the Bayesian Nash equilibria (see [START_REF] Sober | Unto Others: The Evolution and Psychology of Unselfish Behavior[END_REF]) residents play the mixed strategy

x = 1 -P r[v|u, ε](1 + α)R P r[u|u, ε](1 + α)R . (63)
Clearly, the mutants obtain a strictly higher average fitness than such resident altruists, when these use this mixed strategy. In this case, the altruistic preferences with α = r are thus not evolutionarily stable, while Homo moralis preferences are (as per Result 9).

Discussion

The first contributions to the literature on the evolution of preferences by natural selection extended the concept of evolutionary stability from the level of strategies [1] to the level of preferences guiding the choice of strategy, an approach that is sometimes referred to as indirect evolution [START_REF] Bester | Is altruism evolutionarily stable?[END_REF], since evolution then operates on strategies only indirectly, by "delegating" the strategy choice to the individual. Several novel insights were delivered by these contributions, as summarized above. This literature is arguably still in its infancy, and I here discuss some possible future paths.

To begin, some readers may wonder: is there really a deep difference between strategy evolution and preference evolution? After all, and as highlighted in this article, it is typically possible to reformulate preference evolution as evolution of response rules. I'd make the case that there is a fundamental difference, however. Strategies are mere descriptions of behavior.

Preferences are expressed within individuals as the result of some process which may involve reasoning, emotions, hormones, and/or other neurobiological mechanisms, and which may re-spond to the stimuli and information the individual receives. Hence, individuals are guided by their preferences even in completely novel situations, implying that their behavior may change when, for instance, a hitherto unknown preference function appears in the population. Furthermore, some preference classes present the advantage of lending themselves to psychological interpretation. For example, one possible interpretation of an individual with altruistic preferences of the form [START_REF] Ok | On the evolution of individualistic preferences: An incomplete information scenario[END_REF] with a positive degree of altruism α > 0 is that (s)he has emotions that are swayed by the fitness of the person with whom (s)he interacts: the better off is the opponent, the happier (s)he gets. By contrast, an individual Homo moralis preferences of the form [START_REF] Croson | Gender differences in preferences[END_REF] with a positive degree of morality κ > 0 would not react to information about the opponent's fitness: (s)he instead evaluates different courses of action by taking into account what own fitness would be if-hypothetically-the course of action was universalized to all the interactants.

Experimental research shows that these motivations can be distinguished empirically [START_REF] Miettinen | Revealed preferences in a sequential prisoners' dilemma: a horse-race between six utility functions[END_REF][START_REF] Van Leeuwen | Estimating social preferences and Kantian morality in strategic interactions[END_REF].

These observations further suggest three possible future research paths.

First, over the past few decades the behavioral economics literature has proposed and examined a wealth of preference classes to explain observed behaviors in social interactions: altruism [START_REF] Becker | A theory of social interactions[END_REF], warm glow [START_REF] Andreoni | Impure altruism and donations to public goods: A theory of warm-glow giving[END_REF], a preference for conformity [START_REF] Bernheim | A theory of conformity[END_REF], for reciprocity [START_REF] Rabin | Incorporating fairness into game theory and economics[END_REF][START_REF] Charness | Understanding social preferences with simple tests[END_REF][START_REF] Dufwenberg | A theory of sequential reciprocity[END_REF][START_REF] Falk | A theory of reciprocity[END_REF], inequity aversion [START_REF] Fehr | A theory of fairness, competition, and cooperation[END_REF][START_REF] Bolton | ERC: A theory of equity, reciprocity, and competition[END_REF], guilt aversion [START_REF] Charness | Promises and partnership[END_REF][START_REF] Battigalli | Guilt in games[END_REF], and image concerns [START_REF] Bénabou | Incentives and prosocial behavior[END_REF][START_REF] Ellingsen | Pride and prejudice: The human side of incentive theory[END_REF]. These preference classes were inspired mostly by research in psychology and sociology. Building the interdisciplinary bridge one step further by evaluating the evolutionary stability properties of these preferences classes would be interesting; note that this is related to the suggestion made by Sober and Wilson [START_REF] Sober | Unto Others: The Evolution and Psychology of Unselfish Behavior[END_REF] that evolutionary viability of psychological motives behind unselfish behaviors. It should be noted in this context that Homo moralis preferences [START_REF] Alger | Homo moralis-preference evolution under incomplete information and assortative matching[END_REF], studied in this article, are novel to behavioral economics: the theory of preference evolution may thus contribute to economics through the discovery of hitherto unstudied preference classes, and future analyses may make further similar discoveries.

Second, the theory of preference evolution may unveil ultimate drivers of the aforementioned preference classes (besides altruistic and Homo moralis preferences, already extensively studied). A question of particular interest is whether there may be stable polymorphismspopulations in which several preference classes co-exist-and if so, which factors are expected to affect the stable distribution of preferences. Such theories may help explain observed heterogeneity both within and between populations in survey and experimental data [START_REF] Falk | Global evidence on economic preferences[END_REF][START_REF] Nunnari | Meta-analysis of inequality aversion estimates[END_REF][START_REF] Croson | Gender differences in preferences[END_REF].

Third, researchers working with models of preference evolution must make assumptions on the set of potential preference functions. In reality, however, the set of potential preference functions available for a given organism may be determined by physiological constraints. An open question is thus whether findings on the neurobiology of our species would help reduce this set. Such an approach has already been used in the theoretical literature on the evolution of preference functions that govern choices in decision situations other than social interactions, see, e.g., [START_REF] Robson | The evolution of decision and experienced utilities[END_REF][START_REF] Robalino | The biological foundations of economic preferences[END_REF].

Readers may also ask: how realistic is the process by which individuals are matched together in preference evolution models that extend the standard evolutionary game theory model?

An important question is thus whether the results found under this assumption are robust to the extension to other matching processes. Two nascent paths can be mentioned in this context.

First, the model of preference under incomplete information found in [START_REF] Alger | Homo moralis-preference evolution under incomplete information and assortative matching[END_REF][START_REF] Alger | Evolution and Kantian morality[END_REF] has been incorporated by Alger, Weibull, and Lehmann [START_REF] Alger | Evolution of preferences in structured populations: genes, guns, and culture[END_REF] into a standard island model [START_REF] Wright | Evolution in mendelian populations[END_REF], in which the population is structured into groups between which there is limited migration. This approach allowed the researchers to distinguish between preference functions defined over fitness on the one hand and preference functions defined over material payoffs on the other hand. Arguably, the preference function defined over material payoffs that is found to be uninvadable in [START_REF] Alger | Evolution of preferences in structured populations: genes, guns, and culture[END_REF] is more relevant for social scientists who seek to estimate the preferences of individuals by way of observing their behavioral responses to trivial material payoff consequences, such as in the experimental economics literature [START_REF] Charness | Understanding social preferences with simple tests[END_REF][START_REF] Fisman | Individual preferences for giving[END_REF][START_REF] Blanco | A within-subject analysis of otherregarding preferences[END_REF][START_REF] Bruhin | The many faces of human sociality: Uncovering the distribution and stability of social preferences[END_REF][START_REF] Miettinen | Revealed preferences in a sequential prisoners' dilemma: a horse-race between six utility functions[END_REF]. This function combines material self-interest and a Kantian moral concern à la Homo moralis expressed at the material payoff level, and it also has a third component which can be interpreted as altruism/spite towards the opponent, again at the material payoff level. While this function thus differs from the Homo moralis function, the Homo moralis preference function is still uninvadable when defined over fitnesses rather than material payoffs, thus providing one first robustness test. It remains to be seen which preference functions-or distributions over preference functions-would resist the invasion of mutants in models with more sophisticated modeling of the migration decisions, such as in [START_REF] Mullon | Evolutionary stability of jointly evolving traits in subdivided populations[END_REF], for example.

Second, individuals are typically free to choose with whom they interact. Such active partner choice is known to matter for the evolution of cooperative strategies [START_REF] Mcnamara | The coevolution of choosiness and cooperation[END_REF]. How would it affect the evolution of preference functions? One possible formalization is provided by Hop-kins [START_REF] Hopkins | Competitive altruism, mentalizing and signalling[END_REF], in a model with altruistic preference functions where individuals differ in their ability to understand the mental processes of others.

Readers may further ask: if preferences emanate from mental and neurobiological processes, is it reasonable to assume that one can observe others' preferences? In the model proposed by Heller and Mohlin [START_REF] Heller | Coevolution of deception and preferences: Darwin and Nash meet Machiavelli[END_REF] this issue-reminiscent of the well-known "mimicry" issue in biology-is addressed by examining the co-evolution of preferences and the ability to deceive others about preferences and intentions. The extensive work on the commitment role that emotions may have played in our evolutionary past, and the concomitant ability to signal (e.g., through anger) and also detect such emotions (see, e.g., [START_REF] Tooby | The Evolutionary Psychology of the Emotions and their Relationship to Internal Regulatory Variables[END_REF], may perhaps also inspire formal work on emotions that can be incorporated into the theory of preference evolution.

The definition of an evolutionarily stable strategy [1] provided a key tool for theorists to model ultimate drivers of behavior in social interactions. Adding the idea that Nature delegates the strategy choice to the individuals by way of equipping them with preferences over strategies [2,3], arguably brings the theory closer to reality. Although the literature has already delivered many insights, most of the work on evolutionarily viable preferences undoubtedly still lays ahead of us. I hope that this article has underlined the fundamental role played by the bridges built between the models of biologists and economists, both in the past and in the future.

Online appendix to "Evolutionarily stable preferences" [2]); this is also in line with standard vocabulary in non-cooperative game theory, see [3]).

In one-shot simultaneous-move games-in which the individuals act simultaneously and only once-the set of strategies simply coincides with the set of actions available. In oneshot sequential-move games-in which the individuals are called to act sequentially and they can observe any previous action choices-a strategy is a plan that describes the action to be chosen depending on the order of play and previous action choices. In repeated games, the same simultaneous-move game is played several times, and a strategy is a plan that describes the action to be chosen depending on the action choices in the previous rounds.

To fix ideas, consider first a simultaneous-move one-shot Prisoners' dilemma (PD), in which there are two actions-Cooperate (C) and Defect (D), and with payoffs as shown in Figure 1.

In this interaction each individual will find itself in only one decision situation: a strategy can then be formalized as a probability of playing C, with D being played with the complementary * Toulouse School of Economics, CNRS, University of Toulouse Capitole, Toulouse, France, and Institute for Advanced Study in Toulouse. ingela.alger@tse-fr.eu 1 probability. In the simultaneous-move PD a strategy is thus a scalar in the interval [0, 1]. Consider now instead a Sequential prisoners' dilemma (SPD), played by two individuals, say i and j. Nature first draws the assignment of the individuals to the first-mover and secondmover roles, with equal probability for both assignments. The first-mover then chooses between the two actions C and D, following which the second-mover chooses between the two actions C and D. The game tree that represents this interaction is shown in Figure 2. Here an individual's strategy consists of specifying choices in the three situations it may find itself (i.e., at the decision nodes of the game tree, following standard vocabulary associated with sequential games, see [3]. Allowing again for randomization, a strategy is thus a three-dimensional vector in the simplex [0, 1] 3 . As shown in the game tree, I denote by x = (x 1 , x 2 , x 3 ) the strategy of i, where x 1 is the probability that i plays C as a first-mover, x 2 the probability that i plays C as a second-mover following play C by j, and x 3 the probability that i plays C as a second-mover following play D by j; likewise, y = (y 1 , y 2 , y 3 ) denotes the strategy used by individual j.

C D C R, R S, T D T, S P, P

Note that in both the PD and the SPD with role-randomization by Nature, the set of strategies is the same for both individuals: the interval [0, 1] in the PD and the simplex [0, 1] 3 in the SPD.

We will restrict attention to interactions sharing this feature, and call X the common strategy set. Letting w(x, y) denote the fitness of an individual using strategy x when the other is using strategy y, we will refer to Γ = ⟨X, w⟩ as the fitness game. 1for example:

• Simultaneous and one-shot games with a finite number (say, two) of pure strategies, like

Hawk-Dove and Coordination (see the payoff matrices in Figure 1). The strategy set is the set of mixed strategies, X = [0, 1].

• The sequential versions of the aforementioned games (following a similar structure as in the Sequential prisoners' dilemma described in detail above).

• Simultaneous and one-shot linear public goods games: X = [0, E] and w (x, y) = V (x + y) + E -x, for some endowment E > 0 and multiplication factor V ∈ (1/2, 1)

• Simultaneous and one-shot non-linear public goods games where strategies are strategic substitutes: X = R + and w (x, y) = (x + y) τ -x 2 , for some τ ∈ (0, 1) (strategies are strategic substitutes because ∂ 2 w(x, y)/(∂x∂y) < 0)

• Simultaneous and one-shot non-linear public goods games where strategies are strategic complements: X = R + and w (x, y) = (xy) µ -x 2 , for some µ ∈ (0, 1) (strategies are strategic complements because ∂ 2 w(x, y)/(∂x∂y) > 0)

• Simultaneous and one-shot common pool resource games: X = R + and w (x, y) = (a -x -y) x -cx, for some a > c ≥ 0.

• Helping games:

-Nature draws the initial wealth distribution: with probability 1/2, player 1's initial wealth is m H and 2's is m L ≤ m H , and with probability 1/2 the players' wealths are reversed the wealthier individual may transfer any amount of his/her wealth to the other w(x * (α, β), x * (β, α)) > w(x * (β, β), x * (β, β)).

3. If w(x * (α, α), x * (α, α)) < w(x * (β, α), x * (α, β), then u α is not ESC against u β .

To identify candidates for ESC values of α, I solve the equation (see equation [START_REF] Alger | Evolution and Kantian morality[END_REF] in the main text):

α • x * 1 (α, α) = x * 2 (α, α) (4)
for α. The partial derivative with respect to the mutant trait β of the mutant's equilibrium strategy,

x * 1 (β, α) = mk[4 -(1 + α)(1 + β)k 2 ] + m[2 + (1 + β)k](1 + α)k 2 [4 -(1 + α)(1 + β)k 2 ] 2 , (5)
simplifies to

x * 1 (β, α) = 2mk[2 + (1 + α)k] [4 -(1 + α)(1 + β)k 2 ] 2 . (6)
The partial derivative with respect to the mutant trait β of the resident's equilibrium strategy is

x * 2 (α, β) = mk 2 (1 + α)[2 + (1 + α)k] [4 -(1 + α)(1 + β)k 2 ] 2 . ( 7
)
Evaluating these partial derivatives for β = α gives:

x * 1 (α, α) = 2mk[2 + (1 + α)k] [4 -(1 + α) 2 k 2 ] 2 (8)
and

x * 2 (α, α) = mk 2 (1 + α)[2 + (1 + α)k] [4 -(1 + α) 2 k 2 ] 2 . ( 9
)
Inserting these expressions into equation (4) gives:

α • 2mk[2+(1+α)k] [4-(1+α) 2 k 2 ] 2 = mk 2 (1+α)[2+(1+α)k] [4-(1+α) 2 k 2 ] 2 , (10)
or

α • 2mk[2 + (1 + α)k] = mk 2 (1 + α)[2 + (1 + α)k]. (11)
For any k ∈ (-1, 0) ∪ (0, 1) this equation has two solutions, α = -2-k k and α = k 2-k , both of which are thus candidates for an evolutionarily stable α.

I apply the tests in Result 2 (bis) to the first candidate. First, note that for any β

w x * β, -2 -k k , x * -2 -k k , β = m 2 /4, (12)
which implies that

w x * -2 -k k , -2 -k k , x * -2 -k k , -2 -k k = w x * β, -2 -k k , x * β, -2 -k k . (13)
One must therefore check whether the strict inequality in the second part of the result is verified. A simple counter-example proves that this inequality is not satisfied for all mutant traits β ̸ = Next, I apply the tests in Result 2 (bis) to the second candidate, α = k 2-k . First, note that, upon simplification,

w x * k 2 -k , k 2 -k , x * k 2 -k , k 2 -k = m 2 (2 + k)(2 -k) 16(1 -k) (14)
and

w x * β, k 2 -k , x * k 2 -k , β = m 2 (2 -k)(2 + k)[4 -k 2 (1 + β) 2 ] 4[2(2 -k) -(1 + β)k 2 ] 2 . ( 15
)
Tedious calculations then lead to the following expression for w(x

* ( k 2-k , k 2-k ), x * ( k 2-k , k 2-k )) - w(x * (β, k 2-k ), x * ( k 2-k , β)): m 2 (2 + k)(2 -k)k 2 [(1 + β)k -2β] 2 16(1 -k)[2(2 -k) -(1 + β)k 2 ] 2 . (16)

Example 1 .

 1 [Fitness game Γ 1 ] This is a (simultaneous-move one-shot) public goods game with strategy set X = R + and fitness function w (x, y) = (m + ky) x -x 2 .

Figure 1 :

 1 Figure 1: The payoff matrix of the simultaneous-move Stag-Hunt game (R > 1).

Figure 1 :

 1 Figure 1: The payoff matrix of the simultaneous-move Prisoners dilemma (T > R > P > S).

Figure 2 :

 2 Figure 2: Meta-game protocol for the Sequential Prisoners' Dilemma (T > R > P > S)

- 2

 2 -k k . For example, if β = 0, then w(x * (β, β), x * (β, β)) = m 2 (2 -k) 2 while w(x * ( -2-k k , β), x * (β, -2-k k )) = 0, implying that the said condition fails to hold for β = 0. I conclude that α = -2-k k does not resist the invasion by the mutant trait β = 0.

  Since the second-order derivative with respect to y is strictly negative, it follows that w(x ESS , x ESS ) > w(y, x ESS ) for any y ̸ = x ESS .

Result 1 then implies that x ESS is ES against any y ∈ X, y ̸ = x ESS , and Definition 1 in turn implies that x ESS is an ESS. In Example 2, there are two ESS: play S with probability 1, and play H with probability 1.

Since x is ESS only if w(x, x) ≥ w(y, x) for all y ̸ = x, an individual in a population where the ESS x is played by everyone can be interpreted as if (s)he were choosing the strategy which maximizes his or her fitness, given that any individual (s)he interacts with uses strategy x. This observation brings us to the main question: what if, instead of equipping us with automatic play of a strategy tailored to each possible interaction, Nature has equipped humans with (a)

Personal fitness measures the individual's reproductive success, i.e., the number of offspring. Note that if the transmission is cultural, then these are not the biological but the cultural offspring

McNamara and Leimar [11] also view w(y, x) as a proxy of invasion fitness.

The framework even applies to both finitely and infinitely repeated games, in which a strategy specifies which action to undertake as a function of the history of play (see[START_REF] Fudenberg | Game Theory[END_REF]); due to the complex notation required to rigorously define repeated games and to space restrictions, however, I will not explicitly study these games here.

Note that fitness games for which ∂ 2 w(x, y)/(∂x∂y) does not have the same sign for all (x, y) ∈ X 2 fall in neither category.

Note that this is different from the selection gradient, which is the derivative with respect to the trait value; the slope of the score function instead indicates whether the difference between the average fitness earned by residents and that earned by mutants increases or decreases as the share of mutants ε increases. The selection gradient (for fitness game Γ 1 ) will be found in equation (6) below.

Researchers who have adopted this definition do not necessarily believe that individuals in real life do play some Nash equilibrium; however, it is a useful first approach, and the definition can be readily adapted to other equilibrium notions.

To see this, note that an individual is indeed indifferent between the two pure strategies S and H, given that the opponent chooses S with probability 1/4, since (R + K)/4 = 4/4 = 1. Such indifference is necessary and sufficient for the individual to be willing to randomize between H and S.

This observation follows from the upper hemi-continuity that is implied by the topological properties stated in the result (see the discussion following Result 6).

To simplify the exposition, I here refer to w(x, y) as the individual's fitness, although it should instead be thought of some proxy of invasion fitness, like in[2].

This expression is strictly positive for any k ∈ (-1, 0) ∪ (0, 1) and β ∈ (-1, 1), β ̸ = k 2-k , implying that the first condition of Result 2 (bis) is satisfied. Q.E.D.

I acknowledge funding from the European Research Council (ERC) under the European Union's Horizon 2020 research and innovation programme (grant agreement No 789111 -ERC EvolvingEconomics) and IAST funding from the French National Research Agency (ANR) under grant ANR-17-EURE-0010 (Investissements d'Avenir program).

Definition 7. [Alger and Weibull [START_REF] Alger | Homo moralis-preference evolution under incomplete information and assortative matching[END_REF]] A preference function u ∈ Θ is evolutionarily stable under incomplete information (ESI) against a function v ∈ Θ if there exists an ε > 0 such that W u (x * (ε), y * (ε), ε) > W v (x * (ε), y * (ε), ε) in all Nash equilibria (x * (ε), y * (ε)) in all states s = (u, v, ε) with ε ∈ (0, ε). A preference function u is an evolutionarily stable preference function under incomplete information (ESPFI) if it is ESI against all preference functions v ̸ = u in Θ.

To illustrate the analytical challenge that this setting presents, focus momentarily on a setting where in each state s = (u, v, ε) ∈ Θ 2 × (0, 1) there exists a unique equilibrium strategy profile. In such a setting the score function is:

Clearly, the score function is not necessarily linear in ε (not even in the simple fitness game Γ 1 , as can be readily inferred from the expressions in ( 40)). What's worse, without further assumptions the score function may even be discontinuous, since the equilibrium strategy profile may vary discontinuously with ε. To see this, recall the Stag-Hunt fitness game Γ 2 with fitness function w(x, y) = xyR + 1 -x (see (3)), and assume that the set of strategies is {0, 1/4, 1}, i.e., an individual can either choose the pure strategy H with certainty, the pure strategy S with certainty, or to use some randomization device that leads to S with probability 1/4 and to H with probability 3/4 (in the hunting story, perhaps the hunter has decided to go for the stag if and only if (s)he sees a snake on the path, an event which happens with probability 1/4).

Consider preference functions of the form

where K ∈ R can be interpreted, for example, as the joy (if K > 0) or the sadness (if K < 0) experienced when killing a stag. Suppose that R = 2 and that the resident preference function has K = 2. In a population consisting entirely of residents, there are three Nash equilibrium strategy profiles: the two pure strategy profiles (0, 0) and (1, 1), as well as the mixed-strategy let h : R + → R measure the material utility from net wealth m ∈ R + , where h ′ > 0 and h ′′ ≤ 0 letting x be player 1's transfer when rich and y 2's transfer when rich, with x, y ∈ X = 0, m H , then the (expected) material payoff is:

2 Calculating the evolutionarily stable value of α under complete information in fitness game Γ 1 A mutant's equilibrium strategy is

and a resident's equilibrium strategy is

(

Note that an individual's altruism has an effect on its equilibrium strategy only if k ̸ = 0. Hence, the subsequent analysis applies to k ∈ (-1, 0) ∪ (0, 1).

For any (α, β) ∈ (-1, 1), the equilibrium fitness is:

With the notation used here, Result 2 in the main text is: Result 2 (bis).

1. If w(x * (α, α), x * (α, α)) > w(x * (β, α), x * (α, β), then u α is ESC against u β .

2. If w(x * (α, α), x * (α, α)) = w(x * (β, α), x * (α, β), then u α is ESC against u β only if