Massimiliano Bonomi

Davide Branduardi

Giovanni Bussi

Carlo Camilloni

Davide Provasi

Paolo Raiteri

Davide Donadio

Fabrizio Marinelli

Fabio Pietrucci

Ricardo A Broglia

Michele Parrinello

PLUMED: a portable plugin for free-energy calculations with molecular dynamics

come

based methods [4], adaptive force bias [5] and metadynamics [START_REF] Laio | Proc. Natl. Acad. Sci. USA[END_REF]. At the same time, a multitude of MD codes have been developed through the years with focus on different fields of application. Some of these programs are more solid-state oriented, with a particular attention to the variety of implemented potentials (DL POLY [START_REF] Smith | [END_REF]). Others are more specialized in biomolecular systems, with specific potentials developed to that scope (CHARMM [8],GROMACS [9] and NAMD [10]), or implicit solvent capabilities (CHARMM, AMBER [11] and GROMOS [12,13]). Recently, a lot of effort has also been devoted to efficient parallelization, allowing a linear scaling reduction of computational time with the number of processors. In this respect, NAMD, DESMOND [START_REF] Bowers | scalable Algorithms for Molecular Dynamics Simulations[END_REF], GROMACS and pmemd in AMBER are currently among the best performing programs. This wealth of codes provides the user a wide range of capabilities, but only few of them offer interfaces for specific freeenergy methods, often with a limited set of collective variables. In particular, metadynamics has been implemented separately for some of these programs, and so far the only one freely distributed is GROMETA [START_REF] Camilloni | [END_REF] for GROMACS. This prompted us to develop a plugin compatible with many of the aforementioned codes so as to facilitate free-energy calculations with a unified input.

II. THEORETICAL BACKGROUND A. Free-energy methods

We consider a system made of N atoms characterized by microscopic coordinates r ∈ R 3N and a potential energy function U (r). We then introduce a set of d collective variables (CVs), s(r), where s ∈ R d . These variables are used as order parameters, i.e. to distinguish between macroscopically different configurations. The Helmholtz free energy as a function of these CVs is defined as:

F (s) = - 1 β ln dr e -βU(r) δ(s -s(r)) + C, (1)
where C is an immaterial constant. The free energy F (s) contains crucial informations about the thermodynamics of the system, and allows the calculation of the ensemble average of any observable that depends on the CVs s. Moreover, when the CVs are properly chosen, the free-energy profile can be used to model the out-of-equilibrium behavior of the system by means of a stochastic dynamics [16,[START_REF] Zwanzig | Nonequilibrium statistical mechanics[END_REF][START_REF] Yang | [END_REF]19].

The simplest way to obtain the Helmholtz free energy from an unbiased MD simulation is to evaluate the histogram of the visited configurations in the CVs space N (s), so that the estimated free energy F (s) reads:

F (s) = - 1 β ln N (s). (2)
However, in presence of rare events this method is completely impractical, since it would require an enormous computational time. Many methods have been proposed to tackle the rare-event problem and to calculate free-energy profiles. Some of them are aimed at enhancing the sampling of the canonical ensemble, so that the free energy is still estimated by Eq. ( 2). This class includes methods such as simulated tempering [20], parallel tempering [21,22], Hamiltonian replica exchange [23] and solute tempering [24]. Here we concentrate on a second class of methods, which are based on collecting configurations in a biased ensemble and require one to select the set of CVs prior to the simulation. The prototype of all these methods is umbrella sampling [1], where the simulation is performed with a fixed additional bias potential V (s), and the unbiased free energy is recovered as

F (s) = - 1 β ln N (s) -V (s). (3)
While Equation ( 3) is valid for any choice of the bias potential V (s), the efficiency of the sampling and convergence properties are strongly dependent on V (s). In particular, if an approximate free-energy estimate F ′ (s) is available before the simulation, the choice V (s) = -F ′ (s) would give an approximately flat histogram, thus helping in overcoming the free-energy barriers. However, it is rather difficult to have a reliable free-energy estimate before the simulation. Various improvements have been introduced so as to refine the bias potential on the fly (see, among others, Refs. [5,[START_REF] Laio | Proc. Natl. Acad. Sci. USA[END_REF]25,26]). We focus here on metadynamics, in its standard form [START_REF] Laio | Proc. Natl. Acad. Sci. USA[END_REF]27] or in the recently introduced well-tempered flavor [28]. In particular, we consider the direct version of metadynamics, where the bias is acting directly on the microscopic coordinates. For an excellent review of several variants of metadynamics see Ref. [29].

In standard metadynamics the bias potential is built during the simulation as a sum of Gaussian functions centered on the previously visited configurations in the CVs space. This manner of biasing the evolution by discouraging the visited configurations was first introduced in the taboo search [30] and, in the context of MD, by the local elevation method [31]. The approach is also closely related to the Wang and Landau algorithm [25], adaptive force bias [5] and self-healing umbrella sampling [26]. In metadynamics, the bias at time t is written as an integral on the past trajectory r(t):

V (s, t) = t 0 dt ′ ω exp - d i=1 (s i (r) -s i (r(t ′ )) 2 2σ 2 i . (4)
Here σ i is the Gaussian width corresponding to the i-th CV and represents the resolution for that CV, and ω is the rate at which the bias grows. As it was shown empirically [27] and analytically [32] for model Langevin dynamics, in the long run the bias will converge to the negative of the free energy and then oscillate around that profile. As a consequence, the final histogram will be approximately flat in the CVs space, allowing for an uniform exploration in spite of the free-energy barriers. We also observe that the bias performs a work on the system, which needs to be dissipated. Usually during a metadynamics simulation the thermostat keeps the system in thermal equilibrium, unless the growth rate of the bias is too large. Metadynamics has been historically used in two different and complementary manners. It has been used to escape free-energy minima (see e.g. Ref. [33]). i.e. to find a reasonable saddle point out of a local minimum. In this case, metadynamics should be stopped as soon as the system exits from the minimum and starts exploring a new region of space. In other applications, it has been used to exhaustively explore the CV space and reconstruct the free energy. In the examples presented in this paper, we focus more on this latter application. Its main advantage over umbrella-sampling technique is that it inherently explores the region of low free energy first. Here the simulation should be stopped when the motion of the CVs becomes diffusive in this region of interest, and the bias itself can be used as an estimate of the underlying free energy:

F (s, t) = -V (s, t) (5)
(note that if s does not include all relevant order parameters the bias may not converge in a reasonable simulation time). The free-energy estimate at time t, F (s, t), is indeed an unbiased estimator of the exact free energy F (s) [32]. However, F (s, t) fluctuates around F (s) with an amplitude which depends on both the diffusion coefficient in the CV space and on the metadynamics parameters ω and σ, and a more accurate calculation can be performed decreasing ω.

Clearly, a smaller ω means that more time is required to reconstruct the free-energy landscape, therefore a compromise needs to be found between speed and accuracy. One can also exploit the fact that F (s, t) is an unbiased estimate of F (s) at all times, and take the time average of all the profiles as done, for instance, in Ref. [34]. However, as the simulation continues, configurations of higher and higher free energy are explored and, in order to take the average it is necessary to force the system to remain inside the region by a suitable restraining potential. An alternative approach is the recently introduced well-tempered metadynamics [28]. Well-tempered metadynamics is a variant of the method that solves the problem of the fluctuations in a different way, and is more suitable for performing free-energy calculations in several dimensions since it allows avoiding the complication of restraining the dynamics inside a region. In the well-tempered algorithm, the rate at which the bias is grown is decreased during the simulation proportional to e -V (s,t)/∆T , where ∆T is a characteristic energy:

V (s, t) = t 0 dt ′ ωe -V (s(r(t ′ )),t ′ )/∆T exp - d i=1 (s i (r) -s i (r(t ′ )) 2 2σ 2 i . (6)
Over long time, it can be shown that the bias converges to a fraction of the exact free energy:

lim t→∞ V (s, t) = - ∆T T + ∆T F (s). (7)
Conversely, it is possible to estimate the free energy as:

F (s, t) = - ∆T + T ∆T V (s, t). (8)
This estimator does not suffer of the fluctuation problem of standard metadynamics. Moreover at variance with standard metadynamics, the exploration for large times will not be uniform in the CV space but instead it will satisfy the probability distribution:

P (s) ∝ e -F (s)/(T +∆T ) . (9)
Thus, the CVs will be sampled at a finite but arbitrarily high temperature T + ∆T . This is a rather important feature of well-tempered metadynamics, especially for d > 1, since it allows to focus the exploration on the low free-energy regions, such as the main minima and the saddle points. The explored free-energy range can be varied by tuning ∆T , and standard metadynamics is recovered for ∆T → ∞.

In practical implementations, the bias is updated with a finite temporal stride τ G , so that:

V (s, t) = t ′ <t t ′ =0,τG,2τG,... W e -V (s(r(t ′ )),t ′ )/∆T exp - d i=1 (s i (r) -s i (r(t ′ )) 2 2σ 2 i , (10)
where W = τ G ω is the height of a single Gaussian.

B. Metadynamics implementation

A metadynamics implementation should perform two basic tasks: (a) keep track of the visited configurations in the CV space or, equivalently, of the shape of the bias potential and (b) add the proper forces to the microscopic dynamics.

The first task is accomplished by maintaining a list of the Gaussians which have been added to the bias. This list is dynamic and grows during the simulation. The list is also stored in a file that can be used to restart a simulation, and to plot the bias with an external utility.

The second task requires evaluation of the bias forces, i.e. the derivatives of the bias potential with respect to the microscopic coordinates r. This derivative is calculated using the chain rule:

∂V (s, t) ∂r i = d j=1 ∂V (s, t) ∂s j ∂s j (r) ∂r i . (11)
The first part of the derivative is simply a sum of analytical derivatives of Gaussian functions. This sum gets more and more expensive as the simulation proceeds and the number of Gaussians grows [35]. Usually this is not a problem since, if we exclude the simplest test cases, this effort is incomparably smaller than that of evaluating the force-field. For specific needs, an implementation based on the storage of the bias potential on a grid could be faster. The second part of the derivative in Eq. ( 11) depends on the specific choice for the CVs. Thus, for each of the CVs that one wants to use, it is necessary to provide routines which, given the microscopic coordinates, return the value of the CV and its gradients. Writing and debugging these routines for a large number of CVs requires a noticeable effort. However, it is worthwhile pointing out that this effort is the main ingredient of many other free-energy methods, and thus our plugin has been adapted to perform other kinds of free-energy calculations, such as umbrella sampling [2,3] or Jarzynski equation [4] based methods, as it will be discussed in Section III.

C. Collective variables

The implementation of many different CVs is required to deal with the huge variety of problems of interest and to give a proper description of each. Here we describe all the possibilities present in the current package.

• Atom position. The absolute position of an atom or a group of atoms. This CV is implemented with several options that allow the user to restrict the bias to a given direction, e.g. z, or to bias the position of the particle as projected onto a selected segment or, in analogy with the path CV, to bias the atoms distance from a segment. This variable is not translationally invariant.

• Distance. The distance between two atoms or, more generally, the distance between the centers of mass of two groups of atoms identified as G 1 and G 2 :

s dist = i∈G1 m i r i i∈G1 m i -i∈G2 m i r i i∈G2 m i (12) = |r G1 -r G2 | , (13)
where m i and r i are the mass and the position of the ith atom respectively, and r G1 and r G2 are the centers of mass of the two groups.

• Angle. The angle defined by three atoms or, more generally, the angle defined by the centers of mass of three groups of atoms identified as G 1 , G 2 and G 3 :

r c = r G1 -r G2 (14)
r b = r G1 -r G3 (15)
r a = r G2 -r G3 (16)
s angle = cos -1 r 2 a + r 2 c -r 2 b 2|r a ||r c | . ( 17
)
• Torsion. The dihedral angle defined by four atoms or, more generally, the dihedral angle defined by the centers of mass of four groups of atoms identified as G 1 , G 2 , G 3 and G 4 :

r a = r G4 -r G3 (18)
r b = r G2 -r G3 (19)
r c = r G3 -r G2 (20)
r d = r G1 -r G2 (21)
s torsion = cos -1 (r a × r b ) • (r c × r d ) |r a × r b ||r c × r d | . (22)
• Minimum distance. The distance between the two closest atoms pertaining to two different groups G 1 and G 2 , approximately obtained with the following expression:

s mindist = b ln i∈G1 j∈G2 exp b |ri-rj | , ( 23
)
where b is a user-supplied smoothing parameter.

• Coordination. The coordination number of one atom, or more atoms, with respect to another atom or group of atoms (e.g. the coordination of an ion with respect to all the water molecules in the simulation box):

s coord = i∈G1 j∈G2 s ij , (24)
where

s ij =    1 if |r i -r j | < δ 1- " |r i -r j |-δ r 0 " n 1- " |r i -r j |-δ r 0 " m if |r i -r j | ≥ δ. (25)
The user-supplied parameters r 0 , δ, n and m allow a great flexibility to fine-tune the decay of the switching function, e.g. a more accurate counting of the atoms in the coordination shell. In general a good guess for these parameters can be achieved by looking at the pair distribution function between the first and the second group of atoms. A good starting point is to take δ as the position of the first peak in the pair distribution function, r 0 as the full width at half maximum of the peak and n and m to force s ij ≃ 0 at the first minimum of the pair distribution function. However, depending on the system properties, different choices may give better results.

• Hydrogen bonds. The number of intra-protein hydrogen bonds, defined as:

s hbonds = i∈GO j∈GH f (i, j) 1 - |ri-rj | r0 n 1 - |ri-rj | r0 m , (26)
where G O the group of oxygen atoms of the protein, G H the group of hydrogen atoms of the protein. Typically, r 0 = 2.5 Å, n = 6 and m = 10. The function f (i, j) selects a particular type of hydrogen bonds, depending on the user choice: all, α-helix pattern, β-strand pattern (even or odd).

• Interfacial water. This variable is intended to calculate the number of atoms of a certain group G 0 that are in contact with atoms of both groups G 1 and G 2 at the same time (e.g. the number of waters at the interface of two surfaces):

s waterbridge = i∈G0   j∈G1 1 - |ri-rj | r0 n 1 - |ri-rj | r0 m     k∈G2 1 -|ri-r k | r0 n 1 -|ri-r k | r0 m   . (27)
More precisely, the variable counts the number of atom triples (i ∈ G 0 , j ∈ G 1 , k ∈ G 2 ) with i in contact with both j and k (see also coordination parameters).

• Radius of gyration. The radius of gyration of a group G, defined as:

s gyration = i∈G m i |r i -r G | 2 i∈G m i . ( 28
)
Similarly, one can be interested in the trace of the inertia tensor, which can be shown to be equal to:

s Tr[I] = 2(s gyration ) 2 i∈G m i . (29)
• Dipole moment. The electric dipole of a group of atoms:

s dipole = | n i q i r i |, (30)
where q i is the charge of each atom i.

• Dihedral correlation. This variable measures the degree of similarity of a list of adjacent dihedral angles. It is defined by:

s dihecorr = n i=2 1 + cos 2 φ i -φ i-1 2 , ( 31
)
where φ i is the dihedral angle defined by four atoms. For proteins, the variable grows with the content of secondary structure.

• Alpha-beta similarity. This variable measures the similarity of dihedral angles with respect to reference values:

s α-β = 1 2 n i=1 1 + cos φ i -φ ref i , (32)
where reference dihedrals φ ref i are given as input. For proteins, this variable can be use to measure the amount of α or β secondary structure.

• Torsional rmsd. Root of mean square deviation of selected dihedral angles with respect to a reference configuration:

s torsrmsd = n i (θ i -θ ref i ) 2 n , ( 33
)
where n is the number of reference dihedrals.

• Path collective variables. Path collective variables are a general approach based on a previous (approximate) knowledge of the reaction path [36]. If one assumes that the transition from A to B can be described by a set of CVs S(r), which are in general non-linear vectorial functions of the microscopic variables r, then it is possible to define two associated variables. One aims at measuring the progress along a parametric path in CVs space S(l) composed of P frames:

s (r) = P l=1 l e -λ S(r)-S(l) 2 P l=1 e -λ S(r)-S(l) 2 , (34)
and the other measures the distance from the closest point along the path:

z (r) = - 1 λ ln P l=1 e -λ S(r)-S(l) 2 , (35)
where . . . is the metric that defines the distance between two configurations. With such definitions s (r) is a pure number that ranges from 1 to P and z (r) has the dimension of the chosen distance, squared. The parameter λ is in general chosen as 2.3/(∆d) 2 where ∆d is the average distance among adjacent frames.

The definition of the S(l) and of the measure . . . can be chosen by the user among the following:

-Root mean square displacement in Cartesian coordinates: The path S(l) may be defined as a series of configuration in Cartesian space. Each configuration is made of a subset of atoms of the system and the distance is calculated as root mean square displacement (RMSD) after optimal alignment [37].

-RMSD in distances: the path S(l) must be defined as a set of pair distance among atoms. The RMSD is calculated as a difference of distances:

S(r) -S(l)) = N dist i (d i (r) -d i (l)) 2 N dist , (36)
where d i is the distance between the atoms of the i-th couple and N dist is the total number of couples considered [38,39].

-Contact map distance: the path S(l) is defined by a set of contact maps [40], where each contact is defined as in Eq. ( 26). The distance is defined as:

S(r) -S(l) = Ncont i (C i (r) -C i (l)) 2 , (37)
where C i (r) is the i-th contact for the configuration r and N cont is the total number of contacts considered.

The CV z (r), when used with a single reference frame or contact map, is equivalent to the distance between a configuration and the reference structure, measured in the chosen metric (squared). Therefore, this CV should be used to reproduce the standard RMSD, distance RMSD or CMAP distance.

III. USAGE EXAMPLES

When a given program is instructed to use PLUMED (see the manual for specific implementations), a supplementary input file for the free-energy calculation must be provided. During the calculation the main output is a file containing a record of the values of CVs. This file is generally called COLVAR. It may contain also additional informations depending on the chosen free-energy method. In the following we illustrate the basic use of the different methods, available for all the MD codes specified in section VI, and of the algorithms currently implemented only in the GROMACS version. Many additional examples of CVs and sampling techniques, such as multiple walkers metadynamics [41], are contained in the test directory distributed with the code.

A. Metadynamics

A generic input for metadynamics appears as follows:

# switching on metadynamics and Gaussian parameters HILLS HEIGHT 0.1 W_STRIDE 100 # well-tempered metadynamics WELLTEMPERED SIMTEMP 300 BIASFACTOR 10 # instruction for CVs printout PRINT W_STRIDE 50 # a simple CV: the distance between atoms # or group of atoms (in this case between atom 13 and atoms group <g1>) DISTANCE LIST 13 <g1> SIGMA 0.35 g1-> 17 20 22 30 g1<-# wall potential UWALL CV 1 LIMIT 15.0 KAPPA 100.0 EXP 4.0 EPS 1.0 OFF 0.0 # end of the input ENDMETA Three kinds of keyword may exist: the directive (needed keyword to be placed in the first position along a line that specifies the intent of the following keywords), the parameter keyword (which specifies the attribute in the following field/s ) and flags which simply turn on or off a given option.

The HILLS is a directive and switches on metadynamics. The line containing this keyword also sets up the parameters for Gaussians deposition: HEIGHT (parameter keyword) followed by the Gaussian height in the energy unit of the program chosen and W_STRIDE (parameter keyword), which specifies the time between the deposition of two consecutive Gaussians (in number of timesteps). This input has the effect of producing an additional file, called HILLS, which has the following layout: The keyword STEER activates the steering on the collective variable specified by CV. The target value is determined by the parameter keyword TO, the velocity, in unit of CV/kilostep, by VEL and the spring constant by KAPPA. The functional form of the dragging potential is the same as the one of formula 39.

The directive ANGLE selects the type of CV, in this case an angle defined by three atoms or group of atoms. The printout on COLVAR file is analogous to umbrella sampling with the difference that in this method the restraint position changes during the run.

D. Replica-exchange metadynamics

When combined with GROMACS (both version 3.3 and 4), PLUMED can perform replica-exchange simulations coupled with metadynamics in two different ways: parallel tempering metadynamics (PTMetaD) [START_REF] Camilloni | [END_REF]43] and biasexchange metadynamics (BE-META) [44].

PTMetaD is particularly useful to increase the diffusion of the system in conformational space. It consists in defining several replicas of the system, controlled by the same CVs but coupled with thermal baths at different temperatures T i . As in standard parallel tempering [21,22], pairs of replicas can exchange at a given time t two conformations r i and r j . The probability of such an exchange is given by:

P i,j = min(1, exp[(β j -β i )(U (r j ) -U (r i )) + β i (V i (s(r i ), t) -V i (s(r j ), t)) + β j (V j (s(r j ), t) -V j (s(r i ), t))]), (40)
where β i = 1/K B T i is the inverse temperature, U (r) the internal potential, V i and V j the biasing potentials deposited by the two replicas. The effect of this algorithm is to sample the degrees of freedom perpendicular to the CVs more efficiently with respect to standard metadynamics. BE-META is designed to sample the system making use of a large number of CVs without the need of filling with Gaussians a high-dimensional space [44]. This is done employing several replicas of the system, controlled by a few different CVs for each replica. Usually one defines also a "neutral" replica, which evolves according to standard MD, i.e. without metadynamics. The temperature of the system is the same for all replicas.

The exchange probability for a pair of replicas i and j is:

P i,j = min(1, exp(β[V i (s(r i ), t) + V j (s(r j ), t) -V i (s(r j ), t) -V j (s(r i ), t)])). (41)
To run PTMetaD simulations, one has to follow the standard GROMACS procedure for parallel tempering (see GROMACS manual). A binary topology file must be prepared one for each replica, while only one PLUMED input file is required. This file looks as follows:

# switching on metadynamics and Gaussian parameters HILLS HEIGHT 0. The keyword PTMETAD switches on parallel tempering plus metadynamics. All replicas have the same CVs, in this case the radius of gyration defined by the group of atoms <CA>. The Gaussian height set by the keyword HEIGHT is automatically rescaled with temperature, following W i = W 0 Ti T0 , where i is the index of a replica and T i its temperature. The plugin will produce one COLVAR file and one HILLS file for each replica.

A similar procedure is used to run BE-META. A PLUMED input and a binary topology file must be provided, one for each replica. These files must end with the replica index (e.g., META_INP0, META_INP1, ...) and must contain all the CVs, in the same order, and the keyword BIASXMD. The first replica (META_INP0) must have the NOHILLS CV keyword for all the CVs; the other replicas must switch off the variables not used with a list of keywords NOHILLS CV. Also in this case, PLUMED will produce one COLVAR file and one HILLS file for each replica.

IV. OVERVIEW OF THE SOFTWARE STRUCTURE

PLUMED performs these basic functions:

• Initialization and parsing of the input file;

• Evaluation of the CVs value for a given microscopic configuration;

• Calculation of the forces coming from the Gaussians deposited along the CVs trajectory -in the case of metadynamics -or from a fixed/moving restraint acting on the CVs -in case of umbrella sampling/steered MD;

• Printout of CVs value on COLVAR file and, in the case of metadynamics, of the Gaussians deposited on HILLS file.

The initialization of the plugin is done by the routine init_metadyn contained in metadyn.c. This routine is called by the main MD code, which communicates to the plugin some critical information such as the number of atoms, masses and charges, length of the simulation or timestep. The parsing of the PLUMED input is performed by the routine read_restraint in read_restraint.c , which reads the file and, according to the CV chosen (let's say CVname), calls a specific parsing routine contained in a file called restraint_CVname.c. The second task is controlled mainly by the routine restraint in restraint.c, which receives by the MD code the atoms positions at every step of dynamics. This routine calls a specific function, contained in restraint_CVname.c, which calculates the CVs value and the derivatives with respect to the coordinates. The same routine, depending on the free-energy method chosen, calls a proper function to calculate the force acting on the atoms and controls the printout of CVs on the COLVAR file. The forces calculated by the plugin are communicated back to the main MD code and added to the internal forces before the following integration step. The interaction of PLUMED with the principal code is summarized schematically in Fig. 1.

MD code

V. DESCRIPTION OF THE INDIVIDUAL SOFTWARE COMPONENTS

The plugin package, distributed in a compressed tar archive, has the following directory structure:

• common_files. The directory containing all the basic routines that compose PLUMED.

• tests. A variety of examples of different CVs and free-energy methods provided with topology and input files for GROMACS, NAMD, AMBER (SANDER module) and DL POLY. These examples, combined with a script adapted from CP2K [45], work also as a regtest for the plugin.

• patches. A collection of patches to interface PLUMED with different codes (see section VI for more details).

• utilities. Two small utilities written in Fortran: sum_hills and driver. The former is a post-processing program which reads the HILLS file produced by the plugin in a metadynamics simulation and returns the free energy by summing the Gaussians that have been deposited. The latter is a tool that calculates the value of selected CVs along a MD trajectory. It requires a PDB file, a trajectory in DCD format and a file with the same syntax of the PLUMED input file.

• doc. A complete manual with detailed installation instructions for each code.

VI. INSTALLATION INSTRUCTIONS

The installation of PLUMED on every supported program is done through an automatic patch procedure specific to each code. This is done on the clean code and requires its recompilation. All the patching procedures are illustrated in detail in the manual. Currently supported codes are NAMD 2.6, GROMACS 3.3.3 and 4.0.4, DLPOLY 2.16 and 2.19, SANDER (AMBER 9 version).

VII. TEST RUNS DESCRIPTION

In the following we describe a few simple examples of the free-energy methods implemented in PLUMED applied to alanine dipeptide in vacuum at 300 K (see Fig. 2). The AMBER99SB force field [46] has been used throughout all the simulations. These test runs have been conducted with either NAMD or AMBER (SANDER module) code.

A. Metadynamics

Well-tempered metadynamics using the two dihedral angles Φ and Ψ as CVs (see Fig. 2) has been performed with the SANDER code included in AMBER 9. The bias factor chosen is 10, the initial Gaussian height 0.1 kcal/mol, the width 0.35 rad for both CVs and the deposition stride 1 ps. The total simulation time is 5 ns. The free energy (see Fig. 3) has been reconstructed from the Gaussians deposited during the run using sum_hills, the tool provided in the directory utilities.

B. Umbrella sampling

Two-dimensional umbrella sampling on the dihedral angles Φ and Ψ has been performed with NAMD code using 676 umbrella simulations of 10 ps each and a spring constant of 100 kcal mol -1 rad -2 . Umbrellas have been chosen in an adaptive way. The WHAM code by Alan Grossfield [47] has been used to produce the free-energy profile (see Fig. 4). One dimensional umbrella sampling on dihedral Ψ has been performed with NAMD code using 26 windows and running a simulation of 20 ps per umbrella (520 ps total). The spring constant used is 100 kcal mol -1 rad -2 . Thermodynamic integration has been completed dragging the dihedral Ψ from π to -π in 504 ps. The resulting free energies are shown in Fig. 5.

FIG. 5: Free-energy of alanine dipeptide as a function of the dihedral angle Ψ obtained from a one dimensional umbrella sampling calculation (full line) and from thermodynamic integration (dashed line).

VIII. AVAILABILITY

The plugin can be downloaded from http://merlino.mi.infn.it/plumed. Any questions regarding the installation and usage of PLUMED can be posted to the users mailing list at plumed-users@googlegroups.com.

IX. CONCLUSIONS AND OUTLOOK

In this paper we have presented PLUMED, a plugin aimed at performing the calculation of free energy landscapes using a number of state-of-the-art methods such as umbrella sampling, steered molecular dynamics and metadynamics. The unique feature of PLUMED is that it can be easily ported to four popular MD codes, namely AMBER, DL POLY, GROMACS and NAMD. In the next future, we plan to further expand this list. The possibility of using PLUMED with different host codes will allow people to choose the proper code on the basis of its capabilities (e.g., implicit solvent, parallelism, particular force fields), and also taking into account its performance relative to a specific application.

FIG. 1 :

 1 FIG.1: Schematic representation of the interaction of PLUMED with the main MD code.

FIG. 2 :

 2 FIG. 2: Ball and stick representation of alanine dipeptide (Ace-Ala-Nme) in vacuum. The dihedral angle Φ is defined by the set of atoms C-N-Cα-C while the angle Ψ by N-Cα-C-N.

FIG. 3 :

 3 FIG.3: Free-energy of the alanine dipeptide as a function of the two dihedral angles Φ and Ψ obtained with well-tempered metadynamics. Isoenergy lines are drawn every 1 kcal/mol.

FIG. 4 :

 4 FIG.4: Free-energy of the alanine dipeptide as a function of the two dihedral angles Φ and Ψ obtained with umbrella sampling. Isoenergy lines are drawn every 1 kcal/mol.

Acknowledgements

This work would not have been possible without the joint effort of many people in the course of the last seven years. Among these, we should like to thank (in alphabetical order): Alessandro Barducci, Anna Berteotti, Rosa Bulo, Matteo Ceccarelli, Michele Ceriotti, Paolo Elvati, Antonio Fortea-Rodriguez, Francesco Luigi Gervasio, Alessandro Laio, Matteo Masetti, Fawzi Mohamed, Ferenc Molnar, Gabriele Petraglio and Federica Trudu. Francesco Marini is kindly acknowledged for his technical support, Joost VandeVondele for permission to use his regtest script, Jim Pfaendtner for giving precious suggestions in writing the manuscript.

Alessandro Laio deserves a special acknowledgement for carefully reading the manuscript and giving a number of useful suggestions.

This is a record of the Gaussians put during the run: it displays the time step, the center of the Gaussian (one for each CV), the width (one for each CV) and the height. In case of well-tempered metadynamics the Gaussian height is rescaled using the bias factor printed in the last column in order to directly obtain the free energy (and not the bias), when summing all the Gaussians deposited during the run.

The WELLTEMPERED directive switches on well-tempered metadynamics. As explained in section II A, CVs are sampled at a fictitious higher temperature T + ∆T determined by the bias factor (T + ∆T )/T . The user must specify this bias factor using the keyword BIASFACTOR and the system temperature using SIMTEMP.

The PRINT directive allows one to monitor, during the simulation, the evolution of the CVs between the deposition of two Gaussians. The CVs values are printed on the COLVAR file with a frequency, expressed in timestep units, controlled by the parameter keyword W_STRIDE. The file produced looks as follows: where the first column is the time step, the next contains the CV value (one for each CV), the third is the bias potential and the last the potential due to a wall or a restraint. The DISTANCE directive selects the CV (in this case the distance between the center of mass of two groups of atoms). LIST (parameter keywords) specifies the two atoms or group of atoms whose distance is calculated. The atom indices range from 1 to N at in the order they appear in a reference structure produced by the program. In case of a group of atoms, the name of the group must be specified between brackets <>. The list of atoms belonging to the group can be placed anywhere in the input file. The parameter keyword SIGMA specifies the Gaussian width in CV units.

The UWALL directive switches on a wall potential on the collective variable CV. This potential starts acting on the system when the value of the CV is greater (or lower in the case of LWALL) then a certain limit (LIMIT) minus an offset (OFF). The functional form of this potential is the following:

where KAPPA is an energy constant in internal unit of the code, EPS a rescaling factor and EXP the exponent determining the power law. The multiple definition of CVs is allowed. The directive ENDMETA specifies the end of the input. All the following text will be discarded. The symbol # is a comment line which is ignored.

B. Umbrella Sampling

A general input for umbrella sampling calculation is the following: # a simple CV: a dihedral angle TORSION LIST 13 [START_REF] Camilloni | [END_REF] The directive UMBRELLA switches on umbrella sampling on the collective variable specified by the parameter keyword CV (in this case the first CV that appears in the input). The position s 0 of the umbrella restraint is determined by the keyword AT, and the spring constant -whose energy units depend on the MD code used -by the keyword KAPPA.

The functional form of the potential is the following:

The directive TORSION selects the type of CV, in this case a dihedral angle defined by four atoms or group of atoms.

The CVs value is printed on the COLVAR file with a stride fixed by the keyword W_STRIDE. In case of umbrella sampling, this file looks as follows:

0.000 -1.04742 0.000 0.225 RESTRAINT 1 -1.00000 20.000 -1.09302 0.000 0.865 RESTRAINT 1 -1.00000 40.000 -0.84990 0.000 2.253 RESTRAINT 1 -1.00000 60.000 -1.11383 0.000 1.296 RESTRAINT 1 -1.00000 80.000 -1.32902 0.000 10.825 RESTRAINT 1 -1.00000

This file contains, from left to right: the time step, the CV value (one for each CV), the potential coming from the Gaussians, the harmonic potential of umbrella sampling, the CV on which the restraint acts and the position of the restraint. The final calculation of the free energy as a function of this CV can be done using the weighted histogram analysis method, choosing one of the many possible implementations (see section VII B).

C. Thermodynamic integration and methods based on Jarzynski or Crooks relations

PLUMED can be used to drag a system to a target value in CV space using an harmonic potential moving at constant speed. If the process is reversible, i.e. for velocities tending to zero, the work done in the dragging corresponds to the free-energy difference between the initial and the final states. In case of finite velocity, it is still possible to obtain an estimate of the free energy from the work distribution using Jarzynski [4] or Crooks [42] relations.

A general input for a steered MD calculation is the following:

# a simple CV: a dihedral angle ANGLE LIST 13 15 17 # switching on steered MD


Stephen A Butterfill

Towards a Blueprint for a Social Animal

In this chapter, we attempt to answer the question, By what steps could members of a group capable of acting together with a purpose, coordinating flexibly, communicating cooperatively and deceiving competitors be constructed from creatures with minimal social skills and cognitive abilities? The method we use is creature construction: the idea is to adopt the perspective of a designer tasked with specifying a sequence of creatures, where each is independently viable and has the capacities of its predecessors together with some new capacity which enables it to overcome limits its predecessors faced. In creature construction, the aim is not to characterise actual species, nor to describe actual evolutionary or developmental processes. Instead the aims are to understand how various forms (or prototypes) of joint action are related to, and diverge from, each other; and to identify limits on what can be achieved with a given set of cognitive and social skills.

We start with Alphonso and his kin, whose social cognition is limited to tracking the goals of others' actions. We show that despite little cognitive sophistication, the salience and triangulation heuristics enables them to initiate simple joint actions requiring coordination. One group of their descendants, Beki and her kin, develop abilities to produce pointing gestures and object-directed vocalisations, that enable them to enlist others not yet as partners but as social tools, thus extending the range of situations in which they can rely on the salience and triangulation heuristics. Another group of Alphonso's descendants, Bemi's kin, learn the art of strategic deception, acquiring increasingly elaborate tactics for manipulating others' action possibilities. This advantages them in competition. Finally, the Kimi, who are mixed descendants of both the Beki and the Bemi, inherit the former's communicative abilities and the latter's abilities for tactical deception. Progressively integrating the two allows them to develop new capacities of selective deception.

We argue that although our creatures do not yet have all the cognitive capacities classical accounts imply are needed for joint action, they have proxies for some of these capacities. These proxies allow them to coordinate in a limited but useful range of ordinary circumstances. Further, relying on such proxies provide ways of avoiding both omni-doxasticity and omni-intentionality when acting together.

Introduction

Human social and societal life is built on thoughts, intentions, motivations and feelings that bind us and our actions together. One essential aspect of this is joint action. Despite its foundational role in all social and cultural life, joint action has only recently become a topic of inquiry in philosophy and the cognitive sciences (for example, see (Bratman 2014;[START_REF] Gilbert | Joint Commitment: How We Make the Social World[END_REF][START_REF] Tomasello | Why We Cooperate[END_REF][START_REF] Rakoczy | Development of Collective Intentionality[END_REF]. A number of classical accounts of joint action make substantive demands on cognitive abilities (as has been argued by [START_REF] Butterfill | Joint Action and Development[END_REF][START_REF] Pacherie | Framing Joint Action[END_REF]Pacherie 2013;[START_REF] Pacherie | From Mirror Neurons to Joint Actions[END_REF][START_REF] Tollefsen | Let's Pretend: Children and Joint Action[END_REF][START_REF] Obhi | Shared Representations: Sensorimotor Foundations of Social Life[END_REF]. By contrast, we pursue a minimalist agenda with the aim of isolating some undemanding forms of joint action (or prototypes for joint action) and their more modest cognitive requirements.

Our method is creature construction, which has a distinguished history in philosophy [START_REF] Bratman | Valuing and the Will[END_REF]Bratman 2014;[START_REF] Grice | Method in Philosophical Psychology (from the Banal to the Bizarre)[END_REF][START_REF] Strawson | Individuals[END_REF]). The idea is to adopt the perspective of a designer tasked with specifying a sequence of creatures, where each is independently viable and has the capacities of its predecessors together with some new capacity which enables it to overcome limits its predecessors faced. As Grice described it, "The method [...] is to construct (in imagination, of course) [...] a sequence of types of creature, to serve as [...] models for actual creatures. [...] The general idea is to develop sequentially the psychological theory for different brands [...] (which of course is unlikely ever to be more than partial)" (Grice 1974, 37). Whereas Grice's project was 'to compare what one thus generates with the psychological concepts we apply to suitably related actual creature', our aims here are more modest. We will not propose that any of our constructed creatures resembles actual individuals very closely. Our claim is merely that the creatures we construct exhibit possible cognitive profiles. These fictional creatures are, however, loosely inspired by discoveries about developmental, comparative and cognitive psychology, as our footnotes reveal.

We will use the method of creature construction to attempt to answer the question, By what steps could members of a group capable of acting together with a purpose, coordinating flexibly, communicating cooperatively and deceiving competitors be constructed from creatures with minimal social skills and cognitive abilities? The aims are to understand how various forms (or prototypes) of joint action are related to, and diverge from, each other; and to identify limits on what can be achieved with a given set of cognitive and social skills.

We take for granted in what follows that the creatures we are constructing have some ability to track cause and effect. This is, after all, a blueprint for a social animal.

Alphonso's Kin

Consider first Alphonso and his kin, whose social cognition is limited to tracking the goals of others' actions. This enables them to distinguish between, say, grasping and pushing actions. Their goal tracking is pure in the sense that it does not depend on representing intentions or other mental states (see [START_REF] Gergely | Taking the Intentional Stance at 12 Months of Age[END_REF][START_REF] Csibra | One-Year-Old Infants Use Teleological Representations of Actions Productively[END_REF] for one account of how this is possible). What joint actions are they capable of? Suppose Alphonso is walking in the mountains when he comes across one of his kin pushing against a small boulder that is blocking their paths. There is no way around the boulder: the only option is to move it. But it is manifestly too heavy for any individual to move. Alphonso joins in pushing the boulder. At first they are both pushing as hard as they can but it doesn't move much. Gradually they fall into a rhythm of pushing and releasing simultaneously, rocking it harder and harder until eventually it gives way altogether.

Moving the boulder counts as a joint action in at least a minimal sense. Minimally, a joint action is an event grounded1 by two or more agents' actions (compare [START_REF] Ludwig | Collective Intentional Behavior from the Standpoint of Semantics[END_REF]). This is a very broad notion of joint action. Whenever two or more agents' actions have a common effect and there is an event comprising the actions and their common effect, the actions will ground this event. Because Alphonso's and his kin's rocking actions have a common effect, namely moving the boulder, this is sufficient for their actions to ground a joint action.2 That Alphonso and his kin perform actions directed to a single goal is not entirely accidental because the environment has provided a single obstacle which they must overcome. But their both simply pushing is not enough: to move the boulder, coordinated action is required. How could their actions be coordinated? For Alphonso and his kin, coordination is nonintentional. In this case, the actions of Alphonso and his kin could be coordinated thanks to a combination of two things: the joint affordance a large, barely movable rock presents them;3 and entrainment, the process of synchronizing two or more rhythmic behaviours with respect to phase. 4Because their actions are coordinated in this way, there is a sense in which the joint action performed by Alphonso and his kin is purposive. Their actions are directed to moving the boulder. Importantly, because they are coordinated in such a way as to make the boulder's moving more likely, this is not just a matter of each agent's action being directed to moving the boulder. Rather their actions are collectively directed to this goal: it is thereby a joint goal in the following minimal sense.

A goal is an outcome towards which an action is directed (so not a mental state). And the actions which comprise a joint action can be collectively directed to an outcome in this sense: there is an outcome to which these actions are directed and this is not, or not only, a matter of each action being individually directed to that outcome. 5 A joint goal of a joint action is an outcome to which the actions comprising it are collectively directed. This sense of joint goal is broad enough to allow that the jointness may not be at the level of intentions or representations of the subject; it can instead be found at the level of the coordination mechanisms. So for Alphonso and his kin's actions to have a joint goal does not yet imply that they are aware of this, nor that the joint goal is an any sense a purpose they share in acting.

The boulder is a relatively simple case because the environment provides a single most salient goal to pursue. There is also just one most salient means to pursue this goal, and the demands on coordination are limited to the need for actions to be synchronized. But what happens when there is more than one goal to pursue, none more salient than the other? In such situations there will often be uncertainty about which goal any individual will pursue. On different trees there are two large bunches of jackfruits to harvest and efficient harvesting requires several people to participate, so that it would be futile for Alphonso and his kin to pursue different goals. How could Alphonso and his kin overcome such uncertainty, even without being aware that there is uncertainty about goals? If any of them makes a move to harvest jackfruits from one of the trees, she will be aware that some additional contribution to this action is needed (although she need not appreciate what the contribution is; compare [START_REF] Vesper | A Minimal Architecture for Joint Action[END_REF]). How is the additional contribution to be secured? Suppose Alphonso goes ahead unilaterally and attempts to start harvesting jackfruits as if on the assumption that the additional contribution will be forthcoming. 6 When he acts as if on this assumption, others can detect the goal of his actions. Providing they do so, his acting on this assumption makes the goal to which his actions are directed more salient or more attractive to others. This in turn makes it more likely that the assumption will turn out to be true, which can mean that acting on the assumption is reasonable.

Here, then, is a simple strategy that has the effect, not always but often enough, of enabling coordination when there is no single most salient goal to pursue (for example when several bunches of jackfruits could be harvested):

1. Pick a goal to pursue.

2. If it is not too costly to end up acting alone, go ahead and act as if on the assumption that any necessary additional contribution will be forthcoming.

3. If this turns out not to be the case, change your objectives.

A similar strategy is used by Alphonso and his kin when there are complementary roles and it is uncertain who is to do what. For example, to harvest the jackfruits, one agent needs to climb the tree and break the stems while the other should stand below and catch the falling bunches. Alphonso proceeds by selecting and performing a role, as if on the assumption that the other will perform the complementary role. Where this turns out not to be the case, he will eventually change role or objectives.

However, there are situations in which it is too dangerous or otherwise too costly to act as if on the assumption that others' contributions will be forthcoming. To illustrate, suppose Alphonso and his kin have dispersed in the woods to forage for mushrooms when a large pig comes by. This is an opportunity for them to hunt the pig, but their actions will need to be tightly synchronized as tackling the animal alone would be too dangerous. If only they had common knowledge, they would be able to safely rely on the assumption that each will join pursuit of the pig. But Alfonso and his kin do not attribute, and are not aware of, knowledge states. So they do not have common knowledge. To avoid both missing an opportunity and risking disaster, they need a proxy7 for common knowledge, one that requires no awareness of knowledge states.

One crude proxy for common knowledge involves stimuli such as loud noises or strong smells. Suppose the pig is making a loud and distinctive noise; or, more generally, that Alphonso and his kin are in a position to perform actions directed to goals which specify an object or event associated with a loud noise or strong smell. Suppose further that the noise or smell is salient enough that there could be little doubt that everyone nearby had picked it up, so that its existence would ensure common knowledge among people who were capable of having common knowledge.8 Where such salient stimuli identify the pig, the risk to Alphonso of relying on the assumption that his kin's goal will be the goal of hunting the pig is reduced. So Alphonso and his kin can use salience as a proxy for common knowledge. This is the salience heuristic: where they would not ordinarily rely on the assumption that additional necessary contributions will be forthcoming, Alphonso and his kin will rely on this assumption where an object or event associated with the target of potential actions is both sufficiently salient and sufficiently more salient than any other object or event.9

Relying exclusively on salience in this way would stringently limit the range of situations in which Alphonso and his kin can perform joint actions. There will be many cases in which an event is not very salient but others appear likely to be affected by it. For example, they may exhibit characteristic responses to it, such as a startled response or a certain twitching of a nose; or they may have a line of sight to it. 10 Suppose Alphonso, observing the pig, can observe that he and the pig are each linked to his kin, where this link is a matter of the thing (himself or the pig) causally influencing his kin, or else it is a matter of his kin having a line of sight to the thing (to himself or to the pig). This triangular situation ensures that if Alphonso acts, there is less of a risk that additional contributions will not be forthcoming. This is the triangulation heuristic. 11

Although neither is infallible, the salience and triangulation heuristics pick out situations in which, often enough, there could well be common knowledge among more sophisticated individuals. So Alphonso and his kin can use these heuristics as proxies for common knowledge. 12 These heuristics demonstrate that Alphonso and his kin can achieve a range of joint actions. And since the salience and triangulation heuristics work with larger groups as well as dyads, they can even achieve joint actions involving many individuals. But there are also limits on when they can act together, and on when they can avoid acting together.

To illustrate, consider two situations. First, Alphonso and one of his kin hear a pig approaching and hide in order to ambush it. But the pig they heard turns out to be not one but two pigs. This defeats the salience heuristic, as neither pig is sufficiently more salient than the other. And they have no other way to determine which pig to attack. Second, sometimes members of Alphonso's kin come across a resource, such as a berry patch, which they would ideally exploit alone. If others are alerted, the original discoverer will get little. They have no way to prevent more dominant individuals from pilfering berries. While Alphonso and his kin are unequipped to overcome either of these problems, his descendants are more fortunate.

Beki's kin

At this point in our construction, Alphonso's descendants divide into two groups, Beki's and Bemi's. Beki and her kin are frequently confronted with situations like the 'two pig' situation described above: situations in which there are multiple, equally salient possible goals achieving which would require joint action. In these situations, they desire to act, and are both excited by a potential target of action and frustrated by their inabilities to act. But Beki and her kin develop abilities to produce pointing gestures and objectdirected vocalizations. Initially, the gesture or vocalization is not a thoughtful attempt to communicate but merely an expression drawn out of them by a situation. Among the causes of their gestures and vocalizations is a combination of excitement at a potential target of action and frustration at not being able to act.

example, [START_REF] Kaminski | Goats' Behaviour in a Competitive Food Paradigm: Evidence for Perspective Taking?[END_REF][START_REF] Bräuer | Visual Perspective Taking in Dogs (Canis Familiaris) in the Presence of Barriers[END_REF][START_REF] Bugnyar | Ravens, Corvus Corax, Follow Gaze Direction of Humans Around Obstacles[END_REF][START_REF] Okamoto-Barth | Great Apes' Understanding of Other Individuals' Line of Sight[END_REF]).

11

The triangulation heuristic is to joint attention as the salience heuristic is to common knowledge (compare Rakoczy 2017).

12 Our approach is similar in spirit to that of [START_REF] Carpenter | Just How Joint Is Joint Action in Infancy?[END_REF] who discusses 'common knowledge, in the sense of what is known or has been experienced together' (p. 383). Note, however, that she is committed to characterising common knowledge rather than merely a proxy for it.

None of Beki's kin understand these gestures and vocalizations except perhaps as expressions of frustration. Despite this, on some occasions the gestures and vocalizations do function to draw others' attention to objects. They thereby have the effect, unintentionally, of extending the range of situations in which Beki and her kin can rely on the salience heuristic. For example, in the 'two pig' situation, Beki vocalizes at one of the pigs thereby making this pig more salient than the other to her kin. At this point they can rely on the salience heuristic (a proxy for common knowledge introduced above), and so capture the pig.

The gestures and vocalizations also extend the range of situations in which Beki and her kin can rely on the triangulation heuristic. To illustrate, consider the 'hidden pig' problem: there is a pig nearby which only Beki is linked to, and which is too dangerous to be tackled alone. Frustration at missing an opportunity to catch the pig triggers a vocalization directed to it. The vocalization draws Beki's kin's attention to the pig, thereby linking them to it as well. In this way, the conditions necessary for relying on the triangulation heuristic are met, and so Beki's and her kin capture the pig.

At this first stage, Beki's kin's communicative actions have limited effects because they are unintentional responses to exciting objects in frustrating situations (among other things). But over time they observe and become familiar with the causal effects of their gestures and vocalizations on their kin and on other animals around them. This allows them to produce gestures and vocalizations with the goal of bringing about their familiar effects. What was merely an expression of excitement and frustration (among other things) has become an instrumental action resembling a communicative act. This greatly expands the range of actions they can perform. Initially their goals were limited to acting on the physical environment. Now a new set of goals is open to them, namely influencing the behaviour of their kin. They can, for example, call to make another come towards them, or vocalize towards an object in order to direct another's actions to that object. This amounts to their coming to recognize each other as social tools [START_REF] Warneken | Collaborative Partner or Social Tool? New Evidence for Young Children's Understanding of Joint Intentions in Collaborative Activities[END_REF].

Over time, Beki's kin come to make and distinguish a range of gestures. Some of these center around danger. What was initially an involuntary vocal response to danger gradually becomes an action they produce with the goal of bringing about its familiar effects, so that it functions as a warning. 13 Merely involuntary vocalizing and gesturing enabled Beki's kin to solve the 'two pig' problem sporadically, when their actions happened to make the pig more salient or linked it to their kin. But having voluntary control over their gestures and vocalizations enables them to gesture and vocalize strategically, suppressing gestures and vocalizations when these could hinder success, and using them to enlist others as social tools when they promote success. This gives them a systematic way of solving the 'two pig' problem. And, more generally, it means that when uncertainty and the costs of acting 13 Vervet monkeys have a range of danger calls for eagles, for pythons and, for leopards. Infant vervet monkeys are not very discriminating: they will produce the alarm call whenever they see a big bird in the sky; only later do they become able to distinguish eagles from nonthreatening big birds [START_REF] Seyfarth | Vervet Monkey Alarm Calls: Semantic Communication in a Free-Ranging Primate[END_REF].

alone would otherwise prevent action, communicative actions can put them in a position to act.

Bemi's kin

Bemi's kin occupy a region in which resources are scarce. This confronts them with challenges quite different from those facing Beki's kin, who inhabit a region with abundant resources. Competition for food becomes intense, and those who come across ripe berries must either be able to defend their find or consume it before others notice. This is especially challenging for weaker individuals: to stray too far from the group is dangerous, but to forage too close to others means retaining little of the food found.

The weaker of Bemi's kin acquire abilities for tactical deception [START_REF] Byrne | Tactical Deception of Familiar Individuals in Baboons (Papio Ursinus)[END_REF]. When finding berries, the weaker individuals will occasionally refrain from exploiting the food source while others are around. Although such restraint will increase their chances of getting the berries for themselves, their actions need not be performed with any intention to gain an advantage. It may be, for example, that rising anxiety associated with anticipation of conflict over food immediately suppresses any inclination to feed. But at some point, the weaker among Bemi's kin learn that they can avoid having food stolen when foraging in proximity to others by delaying consumption of food for as long as possible while they are present. Now the acts (or omissions) of tactical deception are performed with an intention to avoid theft. But they are not yet performed with any insight into others' mental states.

Over time the value of freezing up or refraining from exploiting food is reduced as competitors come to associate these behaviours with the presence of food. This in turn leads to an escalation of tactical deception. The weaker among Bemi's kin begin to act as if the food was absent and walk on as others pass by. 14 And what at first was a tendency not to eat when others are around gradually becomes a tendency not to eat when others are linked to you or to the food. (As stipulated above, being linked is a matter either of having a line of sight to, or else of being causally influenced by, the food). They also become discriminating in when tactical deception is used, relying on it against stronger but not weaker competitors.

Eventually some of Bemi's kin realise that being linked to food is a precondition not only for stealing it but also for performing any action concerning it; and that what goes for food goes for any kind of object. So they appreciate that, for example, if someone is not 14 Compare De Waal (2016)'s description of what happened when experimenters hid grapefruits on the island where the colony of chimpanzees spend the day: 'After releasing the apes onto the island, a number of them passed over the site where we had hidden the fruits under the sand. Only a few small yellow patches were visible. Dandy, a young adult male, hardly slowed down when he ran over the place. Later in the afternoon, however, when all the apes were dozing off in the sun, he made a beeline for the spot. Without hesitation, he dug up the fruits and devoured them at his leisure, which he would never have been able to do had he stopped right when he saw them. He would have lost them to dominant group mates' (De Waal 2016 Chapter 2).

linked to a sleeping snake she will not be able to avoid it. In this situation, one seizes the other's head and forcibly links her to the snake.

Bemi's kin take an important further step when their experience of manipulating whether competitors and partners are linked to things gradually clues them into the realisation that you need not be linked to something right now in order to act on it; in many cases, it is sufficient to have been linked to it at some point in the recent past. This enhances their abilities to prevent more dominant individuals from stealing their berries. By concealing a cache of berries from a dominant competitor regardless of whether they are currently hungry enough to steal, members of Bemi's kin can avoid theft when the competitor later becomes hungry again.

The story of Bemi's kin is one of gradually elaborating tactics for manipulating others' action possibilities. Merely involuntary freezing in the presence of competitors enabled Bemi's kin to protect some of their food discoveries from theft. Involuntary freezing was sometimes ineffective and occasionally even led to missing opportunities to eat, but it did provide Bemi's kin with the experiences necessary to acquire more refined tactical deception. Learning that freezing is associated with having food stolen less often, they began intentionally to delay gathering or consuming food when others were currently linked to them. This further enriched their experiences of how others' actions are associated with facts about what they are, or have recently been, linked to, allowing them to manipulate those links. This in turn created new opportunities to learn about when others' actions succeed and when they fail.

The Kimi

Climate change forces Beki's and Bemi's groups to migrate, and they end up in close proximity. In their new environment, food comes mostly from large animals. Tight action coordination involving multiple complementary roles is therefore needed for catching prey. But food is also scarce enough that a find needs to be protected from pilfering by others, which will often require strategic deception.

Success in hunting large animals requires tight coordination among a fairly large number of individuals, a division of roles, and the ability to anticipate another's complementary action in order to coordinate your own with it. Through repeating successful behaviours, the joint actions of Beki's and Bemi's kin come to follow conventional patterns in the way they unfold and in who does what. These conventional patterns resemble action scripts and function as precursors of planning. Here Beki's kin have an advantage: as they identify patterns in their past successful behaviours, they can use their communicative abilities to assign roles. But in communicating, Beki's kin often alert Bemi's kin, and so end up losing much of the food.

For their part, Bemi's kin rely on the triangulation and salience heuristics to coordinate in capturing an animal, and so rarely succeed unless they are together when they encounter an animal. The infrequency of their successes means that their joint actions involve less conventional patterns than those of Beki's kin, which further widens the gap between their hunting and that of Beki's kin. But when Bemi's kin do capture an animal, their tactical deception means that they rarely suffer from pilfering.

Both Beki's and Bemi's kin are thus struggling to survive, although for different reasons.

As their societies disintegrate, members of the two groups occasionally reproduce and raise children who become skilled in both communication and tactical deception. These children belong to neither Beki's nor Bemi's group but are outcasts. As outcasts they are often thrust together, but without thereby forming a group in their own right. They are however sometimes forced to act together, as surviving alone is impossible. And in acting together they have an advantage over Beki's and Bemi's kin: they are simultaneously communicators, who enable coordination, and tactical deceivers, who can avoid pilfering. Repeated successes in acting together results in them forming stable groups. These are the Kimi.

Despite their advantages, the Kimi are initially vulnerable because their abilities to communicate are not fully integrated with their abilities for tactical deception. When a potential target appears, the Kimi will gesture to link other group members to the animal, so enabling cooperative action. In doing this they attract competitors, especially Bemi's kin, who, being much less successful at obtaining food themselves, rapidly learn to follow the Kimi and steal from them.

Eventually some of the Kimi realise that their gestures and vocalizations are drawing competitors to them at just the wrong moment. Just as their ancestors began to intentionally delay foraging, so they come to suppress gestures and vocalizations when competitors are around. Although this initially makes things better for them, Bemi's kin become so dependent on them that the new strategy only means the Kimi are rarely apart from their competitors. It is only when there is some pressing danger that Bemi's kin will not pursue the Kimi.

Through following the Kimi so closely, Bemi's kin gradually come to associate the Kimi danger calls with danger, running away whenever these calls are made. On detecting the association between danger calls and their competitors' flight, Kimi groups have the opportunity to put danger calls to a new use. Some now begin to use danger calls to scare Bemi's kin away. Initially these fake danger calls cause both Bemi's kin and the Kimi to respond as if there was danger. At this stage, they are only useful where a Kimi has found a food source and does not need to cooperate with her kin. But as more and more Kimi come to use or encounter fake danger calls, perhaps observing apparently anomalous combinations of danger calls followed by feeding behaviour, they gradually come to produce and respond to the danger calls in a more nuanced way. Meanwhile Bemi's kin, who are not so close, lack opportunities to observe the anomalous combinations and so fail to learn to differentiate genuine from tactically deceptive danger calls. As the new practice of tactical deception takes hold among the Kimi, they come to accompany danger calls with nonvocal communication about food, thereby ensuring cooperation from their group co-members nearby even while giving a danger call. The Kimi have now combined Bemi's tactical deception with Beki's communicative abilities. They can selectively deceive.

Conclusion

There in fact are no kin of Alphonso, Beki, Bemi or Kimi among us, we assume. But fiction is a philosophical tool that can help us to understand by what steps members of a group capable of acting together with a purpose, coordinating flexibly, communicating cooperatively and deceiving competitors could be constructed from creatures with minimal social skills and cognitive abilities. We started with Alphonso and his kin, whose social cognition is limited to tracking the goals of others' actions. Despite little cognitive sophistication, the salience and triangulation heuristics enables them to initiate simple joint actions requiring coordination such as gathering and hunting. But they are dependent on the environment to provide favourable circumstances for coordination, and vulnerable to pilfering by dominant individuals. One group of their descendants, Beki's kin, are equipped to coordinate in less serendipitous circumstances. Beki and her kin develop abilities to produce pointing gestures and object-directed vocalizations. They can suppress gestures and vocalizations when these could hinder success, and use them to enlist others not yet as partners but as social tools, thus extending the range of situations in which they can rely on the salience and triangulation heuristics. Meanwhile, another group of Alphonso's descendants, Bemi's kin, learn the art of strategic deception, acquiring increasingly elaborate tactics for manipulating others' action possibilities. This advantages them in competition. Finally, the Kimi, who are mixed descendants of both the Beki and the Bemi, inherit the former's communicative abilities and the latter's abilities for tactical deception. Progressively integrating the two allows them to develop a new capacity for selective deception.

Although the creatures we have been constructing do not yet have all the cognitive capacities classical accounts imply are needed for joint action, they have proxies for some of these capacities. Alphonso's kin already have proxies for common knowledge and joint attention. These proxies allow them to coordinate in a limited but useful range of ordinary circumstances. Whether or not the things they do together are strictly speaking cases of joint action, they will at least appear very much like joint actions.

Further, having these proxies continues to be useful for the descendants of our creatures.

It frees them from relying on explicit beliefs and sophisticated forms of reasoning in many ordinary situations. As Perry says, even though the descendants may have greater cognitive sophistication, their designers will want to avoid 'omnidoxasticity'. Instead, '[a] more efficient way for Mother Nature to proceed is to fit our psychology to the constant factors in our environment, and give us a capacity of belief for dealing with the rest' (Perry 1993, 202). Of course Perry's focus is an individual acting alone: for much of the time, at least, it would be unfortunate to have to rely on beliefs about gravitational forces in reaching for a glass of water, say. The proxies Alphonso, Beki, Bemi and Kimi rely on provide ways of avoiding both omnidoxasticity and omni-intentionality when acting together.

As our creatures become more complex, their social environments become more complex. There is more variability and less constancy, which makes it more likely that the limits of what can be done with the proxies will matter. Take the triangulation heuristic, for example. According to this heuristic, if you observe that another is linked to you and to the target of a potential joint action, where the joint action is mutually desirable, it is safe to perform the joint action on the assumption that the other will participate (see [sec:alphonso]). With their communicative capacities, Beki's kin are positioned to overcome many limits of this heuristic (as illustrated with the 'two pig' problem in [sec:beki]). However, their descendants, the Kimi, end up using their communicative capacities for deceptive ends. This may create a need to distinguish deceptive from sincere communication and to keep track of how frequently others deceive. Such a need might be met by the emergence of a crude theory of mind. 15 No less importantly, as our creatures and their descendants get better at navigating social complexity, they are increasingly likely to run up against another kind of limit. For example, action scripts and flexible role assignments allow them to exploit frequent and predictable events. But the better they get at coordinating around these events, the more need they may have to rapidly change their approach. When some of the Kimi chance on a better way of tracking their prey, there is no way for the group to exploit this fortuitous discovery. Introducing the better way of tracking prey cannot be done directly through conventions and action scripts, which can change only gradually. Instead this would require some form of planning ability. As these examples suggest, our creatures' proxies for common knowledge, joint attention and the rest may therefore be drivers of development in this sense: needs for greater cognitive sophistication arise from hitting their limits.

Events D1, ... Dn ground E just if: D1, ... Dn and E occur; D1, ... Dn are each part of E; and every event that is a part of E but does not overlap D1, ... Dn is caused by some or all of D1, ... Dn. (This is a generalisation of the notion specified by[START_REF] Pietroski | Actions, Adjuncts, and Agency[END_REF]

Joint action of this kind is found in nonhuman primates (e.g.[START_REF] Suchak | How Chimpanzees Cooperate in a Competitive World[END_REF][START_REF] Visco-Comandini | Do Non-Human Primates Cooperate? Evidences of Motor Coordination During a Joint Action Task in Macaque Monkeys[END_REF] and human infants (e.g. Rakoczy 2017).

3 A joint affordance is an affordance for the agents of a joint action collectively. That is, it is an affordance for these agents and this is not, or not only, a matter of its being an affordance for any of the individual agents. For evidence that joint affordances exist, see M. J.[START_REF] Richardson | Judging and Actualizing Intrapersonal and Interpersonal Affordances[END_REF][START_REF] Davis | Perceiving Affordances for Joint Actions[END_REF][START_REF] Doerrfeld | Expecting to Lift a Box Together Makes the Load Look Lighter[END_REF].

4 Entrainment is found in many species' behaviours (e.g.[START_REF] Backwell | Synchronized Courtship in Fiddler Crabs[END_REF]. In humans, entrainment can occur automatically-that is, independently of the subject's tasks and motivations (e.g.[START_REF] Varlet | Informational Constraints on Spontaneous Visuomotor Entrainment[END_REF]-and without awareness (e.g.[START_REF] Richardson | Effects of Visual and Verbal Interaction on Unintentional Interpersonal Coordination[END_REF]. For a review of emergent forms of coordination in joint action, see G.[START_REF] Knoblich | Psychological Research on Joint Action: Theory and Data[END_REF]

).5 Given how we have defined joint action (in terms of two or more agent's actions grounding a common effect), it follows that not all joint actions have joint goals. This is because for an outcome to be a joint goal, it is not sufficient that it be a common effect of two agent's actions. Rather, the actions must be collectively directed to this outcome in the above sense.

We write 'as if on the assumption' because Alphonso need not actually assume that the additional contribution will be forthcoming. Rather, his actions may be produced in such a way that they often rely for their success on others' contributions without Alphonso himself having any view on the matter. For comparison, an agent's actions can be produced in such a way that they rely on unsupported objects falling without the agent herself having any view on this. Note that this is possible even when there are significant exceptions (not all unsupported objects fall when underwater); comparePerry (1993, 202).

To say that A is a proxy for B is to say that, at least within a useful if limited range of circumstances, A can fulfil some of the functions of which B would fulfil were B present in those circumstances. Where there are two proxies for B, one may be a better proxy than the other insofar as it can serve more of the functions associated with B, or insofar as it can serve functions associated with B in a broader range of circumstances.

At least some nonhumans are sensitive to what others hear[START_REF] Santos | Rhesus Monkeys, Macaca Mulatta, Know What Others Can and Cannot Hear[END_REF], at least within limits(Bräuer, Call, and Tomasello 2008).

Note that it is salience itself, not a belief about, or representation of, salience, which drives the salience heuristic. This heuristic adapts Lewis' suggestion that common knowledge can be arrived at through public events((1969)). He suggests an event E is the basis for common knowledge that P if (i) E is public and (ii) E indicates that P. For instance, a pig's grunting is a public event and indicates the presence of a pig and is therefore a basis for common knowledge.

Line of sight calculations, in at least limited form (but perhaps along with more sophisticated forms of perspective taking), appear widespread in nonhumans. See, for


Elisabeth Pacherie

The topic of Talking to Our Selves is human agency and moral responsibility. John Doris argues that what the sciences of the mind tell us about what drives human conduct and how humans function as agents calls into question important strands of philosophical theorizing about human agency and moral responsibility. Doris does not advocate blanket skepticism about agency and moral responsibility. His view is not that empirical research shows that the practice of treating people as morally responsible agents is fundamentally misguided. Rather than on the practice itself, this research cast doubt on the main theoretical justifications philosophers have proposed in support of this practice.

The main target of Doris's criticism is reflectivism, a standard philosophical approach to agency that characterizes it in terms of deliberation informed by accurate selfawareness. The picture of human cognition and behavior emerging from decades of research in psychology suggests, however, that we lack accurate knowledge of what we do and why. We are thus faced with two options: either we retain the reflectivist conception of human agency and are forced to conclude that the exercise of human agency is the exception rather than the rule, or we jettison reflectivism and propose an alternative theory of human agency that provides more solid foundations for our practice of treating people as morally responsible agents. Doris goes for the latter option and develops a theory of human agency he characterizes as anti-reflectivist, valuational, collaborativist and pluralist.

The book is organized into two parts, each with four chapters. The first part offers a detailed argument that reflectivism, as a theory of human agency, lacks the resources needed to deflect the skeptical challenges it faces. The second part develops an alternative theory of human agency meant to answer these skeptical challenges.

I examine the two parts in turn, saying, in brief, what each of the chapters is about and then considering some objections one might have to Doris's arguments.

In chapter 1, Doris sets the stage for his enquiry into moral responsibility. Empirical investigations into the workings of the mind suggest that rather than as a unified system the mind should be seen as motley collection of at best only loosely integrated systems and subsystems. This empirical picture of psychological anarchy makes three vexing problems of self central to moral psychology -identity, continuity, and agency -appear even more intractable. If the disintegration of the mind brings with it a disintegration of the self, then skepticism about moral responsibility looms: if agents as unified loci of moral praise and blame vanish, moral responsibility vanishes with them. Doris proposes to resist this skeptical temptation but to do so while embracing "bare-knuckle naturalism". His brief is thus to offer an empirically adequate theory of human agency that answers the question: "under what circumstances, if any, are human beings able to function as morally responsible agents?" This is a tall order, but Doris gives himself a little breathing space by assuming that the question of moral responsibility is independent from the question of freedom: an account of moral responsibility need not require freedom from causal constraint.

Chapter 2 discusses reflectivist accounts of agency. The preoccupation with reflection is, Doris notes, a central feature of the Western philosophical tradition. The idea that reflection is what separates humans from non-human animals finds deep echoes in many areas of philosophy, ranging from epistemology, to political philosophy and moral psychology. Doris is first and foremost concerned with reflectivism about agency, at its most obvious in the Kantian tradition, but also common in other traditions. Approaches to moral responsibility generally assume that attribution of moral responsibility to someone for their behavior depends on whether this behavior is an exercise of agency. Hence, the tight connection between accounts of human agency and accounts of moral responsibility. It is also commonly held that a behavior is an exercise of agency when it is self-directed. The question then becomes, What does it take for behavior to be selfdirected? The reflectivist answer to this question can be schematically characterized , Doris proposes, as the doctrine that "the exercise of human agency consists in judgment and behavior ordered by self-conscious reflection about what to think and do" (p. 19), a doctrine whose corollary is that "the exercise of human agency requires accurate selfreflection" (p. 19).

pronouns in a text featuring first personal person pronouns are more likely to identify with "collectivist" values when asked afterwards to complete a values questionnaire then people who tasked with circling pronouns in a version of the text where the plural pronouns have been replaced by singular pronouns. Implicit and egotism biases are also well-documented. What these various effects have in common is that they are supposed to involve unconscious, effortless processing rather than conscious, effortful processing of the kind implicated in reflective deliberation. Together with this partition of human cognition into automatic and analytic processes comes the possibility of what Doris calls incongruence, where the outputs of automatic processes conflict with the output of simultaneous analytic processes or would conflict with the output of analytic processes were they engaged. This forms the basis for Doris's skeptical challenge. Instances where the causes of behavior would not be recognized by an agent as reasons for that behavior, were she aware of them at the time of performance, constitute defeaters. When defeaters obtain, agency doesn't. Unless defeaters can be ruled out, we are on the road to skepticism about agency. Doris considers two moves open to the reflectivists to try and deflect the skeptical challenge. The first move would involve challenging the force of the empirical evidence Doris musters. There are replication issues with a number of findings on priming or implicit effects. Scientific journals may be biased towards publishing studies showing the presence of surprising effects rather than studies showing their absence. The size of the surprising effects in question tends to be small and the effects themselves shortlived. Doris acknowledges all these issues, but takes it that the empirical evidence is so bountiful that even applying the most drastic criteria to screen experimental findings leaves us with effects that are together larger than reflectivist theories would wish. The second move open to reflectivism is what Doris calls the "triage" response, that allows that reflective-self direction may be relatively infrequent but insists that when the situation demands it, people reflect effectively. Doris' rejoinder is that situations where the stakes are high enough that people should reflect before acting are also typically emotionally charged and thus also situations in which unruly automatic processes may be at full play and taint reflection.

Chapter 4 considers the reply to the skeptical challenge that appeals to our experience of our agency as evidence for reflective agency. Doris uses the literature on confabulation to block this reply. He argues that two of the four main factors in clinical confabulation, deficient self-awareness and motivation, are also common, in milder form, in healthy subjects. We have little insight into the actual causes of our behaviors and we readily confabulate explanations for what we do. Furthermore, the explanations we come up with have a motivational basis, often reflecting a tendency towards self-enhancement. The conclusion then is that our experiences of agency are not the reliable guides to our actual agency they would need to be to support reflectivism.

Have the prospects of reflectivism been steamrolled into dust by Doris' skeptical argument and the mass of empirical evidence behind it? Reflectivists might want to welcome these empirical findings rather than looking at them with a jaundiced eye. If, like Socrates, they are much taken with the Delphic Maxim, "Know Thyself!", then, with him, they are probably well aware that self-knowledge does not come for free and that the first step on the path to self-knowledge is the acknowledgment of self-ignorance. The sciences of the mind tell us that this self-ignorance is deeper that previously thought and that unconscious influences on our behavior are multifarious. But a reflectivist might think that to be forewarned is to be forearmed. Doris notes that healthy people do not suffer the deficiencies in self-control characteristic of clinical confabulation. A reflectivist might argue that when so forewarned, we can exercise self-control to keep unwanted influences at bay when deliberating about what to do. As a real life example, some years ago my research institution decided that people sitting on its hiring committees would be systematically briefed on implicit gender biases in science. This policy was aimed at reducing the gender gap observed in certain disciplines and appears to have met with some success. Thus, reflectivists may take the empirical evidence mustered by Doris not as a cause for despair but as an opportunity to become better deliberators.

I now turn to Doris' positive account. Doris proposes a valuational approach to morally responsible agency, which locates the exercise of agency in the expression of an actor's value. His main purpose in the second part of the book is to show that this account of morally responsible agency is well equipped to meet the skeptical challenges that spell the doom of reflectivism.

In chapter 5, Doris rejects another hallmark of the Western philosophical tradition, individualism, defending instead collaborativism. Collaborativism maintains that optimal human reasoning is substantially social: people reason best when they reason in interaction with others rather than in isolation. Thus, substantial cognitive achievements, such as sciences and technology but also moral reasoning, are socially embedded. The first step in his positive argument is that collaborativism is not limited to reasoning but extends to agency. Sociality facilitates agency. For instance, dialogic interactions, such as talk therapies, can facilitate the expression of an agent's values.

The second step of his argument, developed in chapter 6, is that self-ignorance, together with sociality, can promote agency rather than hinder it. First, self-ignorance, when it takes the form of positive illusions of self can motivate us to act in ways that enhance agency. For instance, my believing that I am a good athlete when I am not may motivate me to train hard and indeed become a good athlete. Second, agency may be facilitated by self-ignorance and confabulation in tandem with collaboration. Absent accurate awareness of what makes us behave as we do, the exchange and negotiation of rationalizations can help us make sense of our behavior. Finally, these rationalizations provide the material for the self-narratives we construct in interaction with others to interpret our behavior and these self-narratives shape in turn our behavior. Because people are motivated to reduce cognitive dissonance between attitudes and behavior, self-narratives and behavior will tend to converge, thus insuring that behavior expresses the values endorsed by the self-narratives. This dialogic conception of agency forms the core of Doris's positive account.

Two chapters on responsibility and self conclude the book. Chapter 7 argues has the resources needed to address skeptical concerns about responsibility and can ground a normatively robust account of responsibility. While Doris recommends pluralism about agency and responsibility, and concedes that reflectivism way be warranted for some attributions of agency and responsibility he maintains that a dialogic conception of agency makes better sense of most attributions. Chapter 8 goes back to the problems of the self identified in chapter 1 and argues that the dialogic approach can be extended to continuity and identity.

Value and collaboration are the two pillars on which Doris builds his theory of agency. Let me briefly consider some concerns one might have regarding the solidity of these pillars. Doris extolls the virtues of collaboration, but collaboration may not be an unmixed blessing. While our scientific achievements may be taken as evidence that collaboration facilitates optimal reasoning, social psychology also has a rich store of counter-examples. The phenomenon known as groupthink, thought to have led to such fiascos as Pearl Harbor, the Bay of Pig Invasion or more recently the Invasion of Iraq, is a case in point. Rather than reducing biases, collaboration may exacerbate them when the desire for harmony and cohesion in the group leads group members to minimize conflict and suppress dissenting viewpoints. The success of science may be explained in part by the fact that in the scientific arena collaboration is peppered with a good dose of competition. Even if we agree with Doris that the exercise of agency is a substantially interpersonal phenomenon, it is unclear whether social interaction systematically facilitates optimal agency. Dialogic rationalizations of behavior may have more in common with the dynamics of groupthink than with the dynamics of science. Doris may think this is of little import. Even if these rationalizations are little more than social confabulations, once incorporated in our self-narratives, they start shaping our behavior and become expressions of our values. This takes me to the second pillar of Doris theory, values. Doris claims that behavior is an exercise of agency when it is self-directed rather than having external sources and that self-directed behavior is behavior expressive of the actor's values. However, one can regret that he doesn't spell out in more detail what values are, what makes behavior expressive of values and how we can tell whether behavior is indeed self-directed as opposed to externally directed. In particular, it isn't clear to me that his approach has the resources to identify instances of self-directed behavior with the confidence needed to rule out defeaters and dispel skeptical challenges. Since, on the approach, selfdirected agency is compatible with self-ignorance, the subjective reports and rationalizations of actors cannot be taken at face value and cannot form the basis for attributions of self-directed agency. Doris's answer to this epistemological problem is that we should consider extended behavioral processes rather that isolated behaviors are that attributions of agency are warranted when patterns of behavior and rationalizations emerge and are best explained a involving the expression of some value. But one may wonder whether this approach really gives us sufficient warrant for agency attributions. Suppose we could travel back in time and observe the life or ordinary Soviet citizens in the 1950's. We would certainly find strong trends in their behaviors and rationalizations, and, following Doris's method of attribution, would seem to be warranted in attributing them self-directed agency expressive of communist values. But is it really the case? Some Soviet citizens were no doubt bona fide communists; others might have been mere opportunists doing and saying what it took to become part of the soviet elite, and fear of the gulag was probably the main force driving the behavior of quite a few. Given that in all three cases the observable patterns of behavior would have been the same, we cannot with confidence attribute to them self-directed agency expressive of communist values, we cannot even attribute to them self-directive agency, whether expressive of communist values or not. Rather than being self-directed, their behavior could well have had its source in features of their oppressive social environment. When the soviet composer Dmitri Shostakovich was asked why he had such a passion for football, his answer was that the stadium was the only place where he could be himself and openly express his emotions. Of course some of us are lucky enough not to live under totalitarian regimes. Even in more open societies, however, social pressure to conform remains present even if in milder forms and creates smallerscale versions of the "Soviet attribution problem". It is not so clear that from temporally extended patterns of behavior we can infer with assurance whether behavior is selfdirected and if so what values it expresses.

Talking to Our Selves raises a series of fascinating challenges to the defenders of reflectivism and makes a strong case that philosophers should come to grips with the wealth of psychological findings relevant to their inquiries. The positive account Doris proposes shows that taking these findings seriously can inspire bold, new views of human agency and moral responsibility. This well argued, thought-provoking and, too rare a quality in philosophy writing, very entertaining book makes for a very stimulating and enjoyable read.

On a reflectivist account of agency as reflective self-directedness, agency can be imperiled either through failure of deliberation -the behavior is not the product of deliberation or is the product of invalid reasoning or through failure of self-awarenessthe agent is mistaken about his motives for acting. Doris puts forward three claims. First, empirical evidence shows that both types of failures are very common, thus throwing doubt on reflectivism interpreted as the view that reflective self-reflection is the most commonly observed form of agency. Second, we usually lack sufficient warrant for attributing the exercise of reflective agency, including on occasions of practical importance, thus throwing doubt on an interpretation of reflectivism as the view that reflective self-direction is practically dominant. Finally, he claims that it is possible to develop an account where exercising agency does not require reflection, thus creating trouble for reflectivism understood as the conceptual thesis that reflection is necessary for the exercise of agency. While the arguments for these three claims are left to later chapters, Doris immediately proceed to fill the vacuum created by his rejection of reflectivism about agency by outlining an alternative, valuational account of selfdirection, that locates the exercise of agency in the expression of an actor's values and where values are "associated with desires that exhibit some degree of strength, duration, ultimacy and non-fungibility, while playing a determinative-justificatory role in planning" (p. 28).Chapters 3 and 4 spell out the empirical and epistemological challenges to reflectivism. In chapter 3, Doris presents a wealth of empirical evidence identifying influences on behavior that are both unconscious and unexpected. For instance, if you have an honor box system for coffee at the office, placing an image of eyes near the box may help you avoid bankruptcy, as this makes people much more likely to pay their contribution (the Watching Eye Effect). Another example it the Pronoun Effect: people tasked with circling