I Zeiler

J P Caulkins
email: caulkins@cmu.edu

G Tragler
email: tragler@eos.tuwien.ac.at

Optimal Control of Interacting Systems with DNSS Property: The Case of Illicit Drug Use

Keywords: optimal control, indifference points, multiple equilibria, DNSS points, illicit

In this paper we generalize a one-dimensional optimal control problem with DNSS property to a two-dimensional optimal control problem. This is done by taking the direct product of the model with itself, i.e. we combine two similar system dynamics under a joint objective functional that is separable in both states and controls. This framework can be applied to the construction of various optimal control problems, such as optimal marketing of related products, optimal growth of separate but interacting economies, or optimal control of two related capital stocks.

We study such a system for a particular case drawn from the domain of drug control. The main result of this paper is that in this domain even a modest amount of interaction can sometimes make a very big difference. Hence, drawing conclusions by simplifying the real world into two independent, one-dimensional models may be problematic.

Methodologically the combination of two systems with DNSS property leads to a fascinating series of situations with multiple optimal steady states and associated threshold behavior. These instances reflect some important recent developments in optimal dynamic control theory.

Introduction

Optimal control models have played an important role in economic theory. Over a long period most of the models analyzed yielded a unique and stable equilibrium solution. In that situation the qualitative conclusions derived from such models are robust with respect to the initial conditions. [START_REF] Dechert | A complete characterization of optimal growth paths in an aggregated model with a non-concave production function[END_REF], [START_REF] Sethi | Nearest feasible paths in optimal control problems: Theory, examples, and counterexamples[END_REF][START_REF] Sethi | Optimal advertising policy with the contagion model[END_REF], and [START_REF] Skiba | Optimal growth with a convex-concave production function[END_REF] seem to be the first who presented examples of optimal control models with multiple locally optimal equilibria. Which equilibrium is optimal to converge to depends then on where one starts 1 ; hence the initial conditions play a crucial role. Naturally, such equilibria are separated by indifference or tipping points, which are more formally known as Skiba or DNSS points (henceforth, DNSS in honor of the four seminal authors). In a one-state system, DNSS points are initial conditions at which the decision maker is indifferent between moving "right" or "left" by applying a control such that the state variable approaches either a high-level or a low-level steady state.

A classic example is the Ramsey growth model with a convex-concave production function of [START_REF] Skiba | Optimal growth with a convex-concave production function[END_REF]. If a society starts with a relatively generous supply of capital, the welfare maximizing policy will involve further investment and growth toward a high-income equilibrium. In contrast, a society starting with a capital stock below the threshold level should remain in that low-capital, low-production, low-consumption domain because the long-run benefits of investment do not outweigh the short-run benefits of consumption.

Another well-known model with multiple steady states concerns marketing. [START_REF] Sethi | Optimal advertising policy with the contagion model[END_REF] analyzed a version of the contagion model for optimal advertising [START_REF] Gould | Diffusion processes and optimal advertising policy[END_REF]. He showed that when information spreads by word of mouth, rather than through a standard advertising medium, the optimal outcome may depend on the initial level of awareness about the firm.

1 An early analysis of multiple equilibria and history dependence is provided by [START_REF] Schelling | President's Comission on Law Enforcement and the Administration of Justice Task Force[END_REF][START_REF] Schelling | Hockey helmets, concealed weapons, and daylight saving: A study of binary choices with externalities[END_REF] who assumed that the probability that an agent engages in an illegal transaction increases with the total number of agents who also engage in the transaction. This results in three equilibrium points, a clean and a dirty society separated by an unstable interior tipping point.

Page 3 of 29 A c c e p t e d M a n u s c r i p t

For very low initial awareness, decaying awareness (forgetting) dominates the word of mouth transmission effects and the firm will become unknown and should go out of business. This is the low-level steady state. If initial awareness is high enough the firm should advertise at the maximum allowable level to attain a positive, high-level steady-state awareness fraction.

Besides applications in business and economics, tipping points are of great interest in economic-ecological models, e.g. [START_REF] Brock | Managing systems with nonconvex positive feedback[END_REF], [START_REF] Mäler | The economics of shallow lakes[END_REF][START_REF] Wagener | Skiba points and heteroclinic bifurcations, with applications to the shallow lake system[END_REF]. Other applications include fashion [START_REF] Caulkins | Explaining fashion cycles: Imitators chasing innovators in product space[END_REF], crime [START_REF] Feichtinger | Optimal dynamic law enforcement[END_REF], and migration [START_REF] Caulkins | Skiba thresholds in a model of controlled migration[END_REF].

Scholars very reasonably investigate one-dimensional systems first because they are more tractable and intuition developed with one-dimensional models often carries over to more complicated models. However, a model with a single state variable is almost always a substantial simplification that omits interaction with the outside world. To take the Ramsey growth model as an example, individual nations' economies are rarely isolated. Usually there is both trade and migration across international borders, so it seems sensible to ask how if at all the insights produced by the Ramsey model might be modified if there were two economies that interacted with each other in some fashion? Likewise, one might wish to generalize [START_REF] Sethi | Optimal advertising policy with the contagion model[END_REF] optimal advertising problem to two states, one for the firm and one for the firm's main product. Awareness of the product may then spill over to awareness of the firm and vice versa, and this spill over connects the two states by generating additional awareness in each state. Examples are a motor vehicle manufacturer and a specific car brand or a fashion company and its most known piece of clothing.

More generally one might ask what happens if one takes the "direct product" of two (or more) one-dimensional optimal control models that exhibit tipping points. Such a model, where one decision maker can control two related states, was first considered by [START_REF] Brock | The Generalized Maximum Principle[END_REF] but without any interaction between the states. So in this paper we link two otherwise independent systems, and ask what if any difference that linkage makes.

When the presence of an additional, parallel one-state system has little effect on the solution or policy prescriptions for the other one-state system, then we have some reassurance that not much was lost by simplifying the world down to a single one-state system and studying that system. On the other hand, if modest interaction with a second system greatly perturbs the solution to the original system, then the conclusions we thought we "knew"

A c c e p t e d M a n u s c r i p t based on the one-state model might need to be stated with an extra caveat or expressed in conditional terms, conditional on the absence of significant interactions with parallel systems.

What we do in this paper is in some sense a proof by example that introducing modest interaction to the direct product of two isolated one-state systems can make a substantial difference in the optimal solution. The particular case investigated is drawn from the domain of drug control and the main result here is that in this domain even a modest amount of interaction seems to sometimes make a very big difference. For example, one might expect some sort of monotonicity in the optimal outcome with growing interaction. However, we show that small perturbations of the linkage of the systems can lead to a total reversal of the optimal policy. This is all the more significant when one considers that not just the magnitude of the interaction was modest, but also that the nature of the interaction belongs to a specific class. In our context the interaction is drug users counted by one state variable moving over to the other state (e.g. switching drugs). In more general terms this means that the sum of the state equations only depends directly on the current states and not at all on the presence or the extent of interaction.

The paper is structured as follows. In Section 2 we present a general modeling framework for combining two independent systems under one objective functional and introduce the particular model of illicit drug use. The optimal control problem is solved by applying Pontryagin's Minimum Principle. Section 3 discusses the substantial changes that occur to the one-dimensional system when a parallel, interacting system is introduced. We do this by presenting three different numerical illustrations of the strong impact of interaction. The paper closes in Section 4 with a discussion of the results and conclusions for the optimal control of interacting, parallel systems.

The Model

General Modeling Framework

We discuss two parallel non-linear, autonomous, optimal control problems with infinite planning horizon min

u A (t) J A = ∞ 0 e -rt g A (x A (t), u A (t), ρ A )dt (1) Page 5 of 29 A c c e p t e d M a n u s c r i p t subject to ẋA (t) = f A (x A (t), u A (t), ρ A ),
and min

u B (t) J B = ∞ 0 e -rt g B (x B (t), u B (t), ρ B )dt (2) subject to ẋB (t) = f B (x B (t), u B (t), ρ B ),
where

x i (t) ∈ X i ⊂ R are the state variables, u i (t) ∈ U i ⊂ R n are the control or policy variables, ρ i ∈ Υ i ⊂ R q i are q i -dimensional parameters for i ∈ {A, B}.
For reasons of simplification we assume identical time discount rates, r A = r B = r. We call such systems "parallel" if they are related in content such that introducing interaction is sensible in the application.

The two-dimensional optimal control problem is built by summing up and weighting the cost functionals of each optimization problem; i.e. the two-dimensional model is generated by two jointly managed one-dimensional problems. The new objective functional is separable in the cost functionals, i.e. without further interaction an optimization of the joint system would lead to the same results as optimizing each individual system. This is altered when a functional is introduced that connects ẋA (t) and ẋB (t) in some form.

Letting h i (•), i ∈ {A, B}, denote the flow functionals and σ ∈ Υ I ⊂ R s an s-dimensional parameter that governs the extent of interaction between the systems, the resulting two-dimensional model reads as

min u A (t),u B (t) J = ∞ 0 e -rt (̟ A g A (x A (t), u A (t), ρ A ) + ̟ B g B (x B (t), u B (t), ρ B ))dt (3) subject to ẋA (t) = f A (x A (t), u A (t), ρ A ) + h A (x A (t), x B (t), ρ A , ρ B , σ), ẋB (t) = f B (x B (t), u B (t), ρ B ) + h B (x A (t), x B (t), ρ A , ρ B , σ), where (x A , x B ) ∈ X A × X B ⊂ R 2 denote the state variables, (u A , u B ) ∈ U A × U B ⊂ R n × R n the controls and (ρ A , ρ B ) ∈ Υ A × Υ B ⊂ R q A × R q B a
pair of parameters. The weights, ̟ A and ̟ B , reflect the decision maker's preferences.

In this paper we consider a special case for which h A (•) = -h B (•) so the interaction does not directly affect the sum of the states. The added flows A c c e p t e d M a n u s c r i p t are from one state to the other and vice versa, but not into or out of the system overall.

As stand-alone (non-interacting) systems, both models (1) and ( 2) are assumed to manifest two locally optimal steady states that are separated by a DNSS point. If x and x are two such optimal steady states, a DNSS point, x = x DN SS , is an initial state for which there are two distinct optimal controls, u 1 (t) and u 2 (t), and corresponding state trajectories, x 1 (t) and x 2 (t), with x 1 (0) = x 2 (0) = x DN SS , lim t→∞ x 1 (t) = x and lim t→∞ x 2 (t) = x, such that J(x 1 (t), u 1 (t)) = J(x 2 (t), u 2 (t)). Hence, when starting at a DNSS point a system may be driven to either steady state while incurring equal costs.

If the states in the two-dimensional problem (3) are decoupled, i.e. h i (•) = 0, the DNSS property of the one-dimensional case is naturally inherited to this case. In particular the two-dimensional DNSS thresholds create a simple cross (see [START_REF] Brock | The Generalized Maximum Principle[END_REF]. In the following we want to find out what happens to the DNSS thresholds when the connecting flows are nonzero and discuss this problem by the example of interacting drug epidemics.

The Drug Control Model

The particular example we use to construct a two-dimensional model of type (3) is drawn from the domain of drug use and control. Optimal control theory has been harnessed for about a decade to provide crucial insights concerning how drug control policy ought to adapt over the course of a drug epidemic. In particular, previous work (e.g. [START_REF] Tragler | Optimal dynamic allocation of treatment and enforcement in illicit drug control[END_REF] showed that the occurrence of DNSS thresholds is a common phenomenon that describes what is thought to hold true for drug epidemics. When control begins early, i.e. when the epidemic is still small, it is optimal to drive it down to a low level of use. It is common to refer to this as "eradication" even though the lowlevel equilibrium may not be at zero. If the size of the epidemic crosses the DNSS threshold then instead of eradication, an "accommodation" strategy is indicated that delays rather than controls the growth of the drug problem towards a high level of use.

The general idea that drug markets can be stable at either low-or highlevels of participation is borne out in data and observation. If the market is very small, for any given level of drug control (e.g. enforcement), the control effort per participant will be high and that potentially induces some to exit. Each exit shrinks the market size and further inflates the density of control per remaining participant. This can create a virtuous positive feedback loop in which that same fixed level of control can be enough to suppress the Page 7 of 29 A c c e p t e d M a n u s c r i p t market, and, in a subsequent maintenance phase, an even smaller level of control may be enough to keep an eliminated market from rebounding. So "eradication" refers to a stabilization of the epidemic at such a level, where the absence of scale economics makes it relatively easy to maintain the market at that size [START_REF] Kleiman | Enforcement swamping: A positive-feedback mechanism in rates of illicit activity[END_REF]. However, if the market is initially beyond some minimum threshold size, then the initial control effort per participant is low, which may give more people an incentive to enter. That lets the market expand to some high level equilibrium where an eradication strategy is not feasible or at least not optimal.

In the optimal control literature, so far only single populations of drug users have been considered. This is a simplification since most societies with a serious drug problem have substantial use of more than one substance, and/or have to deal with mobile drug users that may affect the course and costs of an epidemic in places other than their origin (see, e.g., [START_REF] Roxburgh | Changes in patterns of drug use among injecting drug users following changes in the availability of heroin in New South Wales, Australia[END_REF][START_REF] Rachlis | Migration and transmission of blood-borne infections among injection drug users: Understanding the epidemiologic bridge[END_REF]. So, besides discussing an important question on optimal control, we introduce here for the first time an optimal control model that considers two interacting populations of drug users.

The underlying one-state one-control model is based on Tragler et al's. ( 2001) work on the optimal dynamic allocation of treatment and enforcement. To better suit our needs we modified the model in two ways. First, we replaced Tragler et al.'s growth function (a power function) by the more familiar logistic function that models illicit drug use as diffusion process. This better describes the common "lifecycle" for a drug in a particular society, which is minimal levels of use for an extended period before some event stimulates market growth that then leads to convex growth in initiation and, eventually, stabilization at a higher endemic level of use [START_REF] Caulkins | Implications of inertia for assessing drug control policy[END_REF]. For more on the diffusion of drug use in a population, see also [START_REF] Ferrence | Diffusion theory and drug use[END_REF]. Second, we use a single control whose effect is removing users, referred to for convenience as "treatment". The price-raising control "law enforcement" is omitted for simplicity, which leads to a constant retail price. Constant prices are clearly a simplification, but it is common in drug policy analysis to assume a very high elasticity of supply and, hence, a price that varies little with prevalence, because there are few constrained factors of production. Bultmann et al. (2008a,b) discuss these modifications at length.

As in [START_REF] Tragler | Optimal dynamic allocation of treatment and enforcement in illicit drug control[END_REF] the objective is to minimize the discounted sum over time of the control costs, u(t), plus consumption, p -ω A(t), times the social costs per unit consumed, κ. The dynamics of A(t) is governed by three flows, a logistic inflow, an endogenous outflow, and a control driven The impact of control spending is as in [START_REF] Rydell | Enforcement or treatment: Modeling the relative efficacy of alternatives for controlling cocaine[END_REF] model of drug treatment. However, we prefer to think of it more generally as any control driven outflow from drug use. For example, deterring or incarcerating users is an alternative, non-treatment mechanism for removing users. Control has diminishing returns for the usual reasons. For example, for treatment there may be "cream skimming" in which those who are easiest to treat are treated first. Hence, the rate at which users exit due to control, β(A(t), u(t)), is concave and monotonically increasing in control spending u(t), i.e. assuming A > 0, then β ′ > 0 and β ′′ < 0. In particular, following [START_REF] Rydell | Enforcement or treatment: Modeling the relative efficacy of alternatives for controlling cocaine[END_REF] we have

β(A(t), u(t)) = c u(t) A(t) z
where c is the constant measuring the average efficiency of control and z the constant measuring the marginal efficiency of control with 0 < z < 1 (cf. [START_REF] Tragler | Optimal dynamic allocation of treatment and enforcement in illicit drug control[END_REF][START_REF] Behrens | Optimal control of drug epidemics: Prevent and treat -but not at the same time?[END_REF][START_REF] Behrens | Why present-oriented societies undergo cycles of drug epidemics[END_REF]. The endogenous outflow due to natural desistance occurs at a constant per capita rate. Hence, we investigate a special case of the optimal control problem (1) where Several forms of h(•) are possible. Here, the flows are chosen to be linear in the state variable and independent of the control and second state; thus,

g(A(t), u(t), ρ) = κp -ω A(t) + u(t), (4)
f (A(t), u(t), ρ) = kp -a A(t) (m -A(t)) -c u(t) A(t) z A(t) -µp b A(t). (5
h A (•) = -γ A A + γ B B and h B (•) = -h A (•)
, where γ A denotes the rate at which users leave population A by migrating into B, and γ B the rate at which users leave population B by migrating into A. At one moment in time an individual can only be part of one population, either A or B. So, "migration" can be understood as physical migration from e.g. one city to another or in a wider sense as a change of primary substance of abuse.

Furthermore we assume the cost functionals to be equally weighted in the joint objective functional, hence we have

̟ A = ̟ B = 1.
So when A(t) and B(t) denote the current numbers of users of a drug, the two-dimensional model of coupled parallel systems is given by min

u A (t)≥0, u B (t)≥0 J = ∞ 0 e -rt (κ A p -ω A A A(t) + κ B p -ω B B B(t) + u A (t) + u B (t)) dt, (6) subject to the system dynamics Ȧ(t) = k A p -a A A A(t)(m A -A(t)) -c A u A (t) A(t) z A A(t) (7) -µ A p b A A A(t) -γ A A(t) + γ B B(t), Ḃ(t) = k B p -a B B B(t)(m B -B(t)) -c B u B (t) B(t) z B B(t) (8) -µ B p b B B B(t) + γ A A(t) -γ B B(t).

Solution of the Optimal Control Problem

To solve the optimal control problem ( 6)-( 8) by applying Pontryagin's Minimum Principle (cf. [START_REF] Feichtinger | Optimale Kontrolle ökonomischer Prozesse -Anwendungen des Maximumsprinzips in den Wirtschaftswissenschaften[END_REF][START_REF] Grass | Optimal Control of Nonlinear Processes: With Applications in Drugs, Corruption and Terror[END_REF][START_REF] Leonard | Optimal Control Theory and Static Optimization in Economics[END_REF] we consider the current value Hamiltonian H = H(A, B, λ A , λ B , u A , u B ) defined by

H = λ 0 (g A (A, u A , ρ A ) + g B (B, u B , ρ B )) + λ A Ȧ + λ B Ḃ, ( 9
)
where λ A and λ B denote the costate variables in current value terms and λ 0 ≥ 0 is a nonnegative constant multiplier associated with the integrand of the objective function

g A (A, u A , ρ A ) + g B (B, u B , ρ B ) = κ A p -ω A A A + κ B p -ω B B B + u A + u B . (10)
A c c e p t e d M a n u s c r i p t

The time argument t is omitted for brevity. If we assumed that λ 0 = 0, then the Hamiltonian minimizing condition would yield at least one u i = ∞, i ∈ {A, B}, because H is monotonically decreasing in u i and there is no upper bound on the controls. That would cause the objective function ( 6) to be maximized which contradicts our purpose of minimizing the total societal costs. Hence, we can conclude that λ 0 = 0 and may set λ 0 = 1 without loss of generality.

There must exist two non-positive multipliers η A and η B for the nonnegativity constraints on the controls such that the partial derivatives of the Lagrangian function L = L(A, B, λ A , λ B , u A , u B , η A , η B ) with respect to the controls are equal to zero when evaluated along an optimal path. The Lagrangian function is defined by

L = H + η A u A + η B u B , (11)
and the complementary slackness conditions

η i ≤ 0 ∧ η i u i = 0, ∀ i ∈ {A, B} (12)
imply that as long as the optimal controls are in the interior of the feasible region, i.e. strictly positive, the Lagrangian function L reduces to the Hamiltonian H and the solutions of the constrained problem equals the unconstrained one. So, if ∀i ∈ {A, B} : u i > 0, the necessary2 optimality condition on the controls is to minimize the Hamiltonian subject to the constraints at any instant of time,

u ⋆ i = arg min u i H.
Using the convexity of the Hamiltonian with respect to the controls, which is proven by the positive definiteness of the Hessian matrix, and hold if states and costates approach a stable state. After substituting for the controls, the state and costate equations form the canonical system, and their two-dimensional stable manifolds satisfy the necessary optimality conditions provided by the Minimum Principle.

∂ 2 H ∂u i ∂u j = -1 A c A u A A z A -2 (z A -1)z A λ A 0 0 -1 B c B u B B z B -2 (z B -1)z B λ B , i, j ∈ {A, B}, we can derive u ⋆ A = A(c A z A λ A ) 1 1-z A , (13)
u ⋆ B = B(c B z B λ B ) 1 1-z B , (14)

Results

The goal of this section is to discuss how the optimal solution of decoupled parallel systems changes when interaction is introduced. Before discussing this interesting question we present the solution of two identically parameterized, jointly managed systems. This acts as base case for the analysis of coupled cases, i.e. it is an instance to which the later results can be compared. For the baseline parametrization we use the values of Bultmann et al. (2008b) (see Section 2.2) that roughly describe the current U.S. cocaine epidemic and yield in the one-state one-control model two optimal steady states, A and A, that are separated by a DNSS point, A DN SS . We then present three different illustrations for the strong impact of interaction. In every case the separate state's internal dynamics is similar, so these are modest departures from the noninteracting case. In particular, two illustrations involve identical systems; the third has asymmetry in that state B cannot be controlled and does not impose any costs directly (e.g. represents and adjoining region). For further asymmetric scenarios we refer to [START_REF] Zeiler | When two become one: Optimal control of interacting drug epidemics[END_REF].

We first show that in this model even very little interaction leads to profound structural changes in the optimal solutions. Second, if the system starts in a certain area of the state space, the optimal outcome reacts very sensitively to the degree of interaction. Third, when one system is uncontrollable, the outcome is determined, but interaction affects substantially the optimal paths that lead there.

A c c e p t e d M a n u s c r i p t

Optimal Control of Uncoupled, Jointly Managed Systems

Figure 1i shows a classic DNSS structure in a one-dimensional system. The candidates for optimality are the stable manifolds of the saddles A and A (the direction of the phase flows is indicated by arrows). Around the unstable, central focus-equilibrium the stable manifolds overlap. By calculating the costs of a solution starting on either of these manifolds the point of equity is found that indicates the DNSS point A DN SS (vertical line).

Figure 1ii shows the optimal solution of two parallel, decoupled systems that are jointly managed. Without interaction the optimal outcome depends on the initial value of each state only. If either of the states exceeds (falls below) A DN SS = B DN SS , that state converges to the corresponding high (low) steady-state value. Since each of the two states can independently approach either of two steady states, this results in four optimal steady states that are separated by straight DNSS lines intersecting at the point (A DN SS , A DN SS ).

If the system starts on the vertical DNSS line, for example at (A DN SS , B 0 ) with some B 0 > A DN SS , B converges independently of A to the high level of use, whereas for A there are two distinct optimal controls that drive this state to either A or A while incurring equal costs. The central DNSS point, (A DN SS , A DN SS ) is a trivial quadruple DNSS point, at which the decision maker has four different options to choose from.

Here and in the following the steady states are denoted by (L, L), (H, L), (L, H) and (H, H). L (low) and H (high) are descriptions of the location of the respective steady state coordinate and have to be seen relatively, not in absolute A or B values.

Interaction Leads to Global Changes in the Optimal Solutions

Here the two separate state dynamics are identical, but there is now a constant per capita flow at rate γ from each state to the other. In practice the flow rates out of the two states may be different, but we make them identical to show that the interesting results below do not depend on asymmetry in those flow rates. We analyze the changes of the optimal solutions on a global level by applying bifurcation theory.

To begin with we may anticipate some results for identically parameterized, connected systems from the uncoupled case. First, since the connecting flows annihilate each other when identical, the overall high and low level of use are not affected by growing interaction. Second, for all unique solutions that start on the 45 degree line, i.e. all solutions for (A 0 , B 0 ) with A 0 = B 0 = A DN SS the optimal solutions are invariant to changes of γ. Thus, Page 13 of 29 this central line represents the projection of the one-dimensional case into the two-dimensional system and is comprised of trajectories that appear in all later discussed cases.

Figure 2 shows phase portraits for four different levels of the interaction parameter γ in the AB-plane; (i) γ = 0.001, (ii) γ = 0.005, (iii) γ = 0.01, and (iv) γ = 1. (These should be construed as rates of flow per year, because the time scale underpinning the rest of the parametrization is that t = 1 is one year.)

In this series we can observe two structural changes. First, the number of steady states reduces from four to two between case (iii) and (iv). Second, at some point between (ii) and (iii) the asymmetric steady states (denoted by (L, H) and (H, L)) lose optimality. This causes the disappearance of the two corresponding DNSS thresholds that separated these steady states from (L, L) and (H, H). The first change is a local bifurcation, precisely a fold bifurcation, that does not directly affect the optimal vector field and is hence of minor interest.

However, the loss of optimality of the asymmetric steady states is a global phenomenon that indicates a qualitative change between the systems. At this point the entire structure of the optimal trajectories is changed. Keeping one state small while the other is large is (for some initial values) optimal when interaction is low, but such a discordant policy cannot be optimal when interaction is larger than a certain critical value. The exact bifurcation value represents an economic hairline case; its practical relevance is little. However, the magnitude of this value compared to all other parameters is of interest, because it shows how much interaction it takes to reduce policy prescription down to eradication versus accommodation of both epidemics. We use bifurcation theory to find this threshold.

Also, [START_REF] Wagener | Skiba points and heteroclinic bifurcations, with applications to the shallow lake system[END_REF] recognized in one-dimensional systems a connection between the existence of DNSS thresholds and bifurcation theory. He proved for one-dimensional optimal control problems that the region in the parameter space in which a DNSS threshold exists, is bounded by the occurrence of heteroclinic bifurcations. We show numerically that this result holds here as well.

A heteroclinic bifurcation is indicated by the existence of a heteroclinic connection between two equilibria. Such a connection is an orbit that lies in the stable manifold of one, and in the unstable manifold of the other equilibrium. That is, when the system starts on such a curve, it converges for t → ∞ to one steady state and for t → -∞ to the other.

With baseline parameterizations a connecting orbit occurs when γ is close to γ crit = 0.0051987. Figure 3 shows this connection in the state-costate space. When the system starts in a neighborhood of the asymmetric steady state it converges to (L, L), i.e. the stable manifold of (L, L) (that gives the optimal solutions) is in the unstable manifold of (L, H).

Figure 4 highlights how the DNSS threshold "disappears" at the bifurcation. It shows the Hamiltonian calculated for initial values along the intersection of the state space that is the direct connection between (L, L) and (L, H). The solid line gives the costs when the system starts at any given point on this intersection and converges to (L, L); the dashed line shows the costs for converging to (L, H). (L, L) is the optimal choice for all initial values with A < Â(L,H) since here H (L,L) < H (L,H) , but in Â(L,H) the Hamiltonians coincide, so this steady state is a weak DNSS point (cf. [START_REF] Grass | Optimal Control of Nonlinear Processes: With Applications in Drugs, Corruption and Terror[END_REF].

In this case of a weak DNSS point, the optimal solution is unique for each initial point, but depending on the initial state the long-run optimal solution may be different. In other words, weak DNSS points only exhibit the property of history dependence not that of multiplicity of the optimal solution. Here, when starting in (L, H), the system stays in (L, H), as (L, H) is a steady state that is not dominated by a different path (the values of the Hamiltonians are equal). So, the optimal solution is unique. However, small deviations of the initial values cause the system instead to tip to a different A c c e p t e d M a n u s c r i p t steady state, i.e., to converge to (L, L).

From the existence of the heteroclinic connection in the two-dimensional model we can draw implications both methodologically and substantively. Methodologically, our model illustrates that Wagener's general result on the existence of DNSS thresholds in one-dimensional models applies in this case to a two dimensional model.

Another interesting methodological finding is depicted in Figure 5, which shows the phase portrait for a value of γ that is only slightly larger (the difference is less than 10 -7 ) than its bifurcation value. The (L, H)-(L, L) DNSS curve does not just disappear but is replaced by an (L, L)-indifference curve that separates two different paths, e.g. pt 1 and pt 2 , that both approach (L, L) while incurring equal costs. Since very little is known on what exactly happens when a two-dimensional DNSS threshold disappears, this phenomenon raises questions, including whether DNSS curves are in general replaced by such indifference curves.

Substantively the important finding is that the bifurcation value γ crit = 0.0051987 is surprisingly low. When comparing it to the rate of natural quitting, µp b = 0.107692, γ crit is about one twentieth as large. So, even a very little migration is sufficient to achieve substantial changes in the optimal policy.

Changes of the Optimal Outcome are Non-Monotonic in the Strength of the Interaction

In the last section we saw that small levels of interaction between identical systems can lead to structural changes in the optimal solution. Here, we go to a more local level and present an example where the optimal policy for a certain initial value changes dramatically when the degree of interaction is varied. Furthermore, we discover a non-monotonicity of the optimal policy in the strength of interaction. One might expect that growing interaction favors either eradication or it favors accommodation, but we see that this is not the case.

We consider the initial value P = (10, 2) million, which corresponds to a well established population A that is close to its steady state value and a second population, B, that is on the verge of spreading out, perhaps because the substance has only recently become available to this population. The changes are illustrated with the cases depicted in Figure 2 where P is indicated by an "o"; the respective trajectories are shown in Figure 6. We can distinguish between three levels of interaction, γ < γ crit , γ > γ crit and γ >> γ crit .

γ < γ crit For small levels of interaction the system adheres to the basic structure of four optimal steady states and converges for P to (H, L) (Figures 2i and ii and path γ = 0.005 in Figure 6). The major population grows to its steady state level, while the smaller population is "eradicated". With growing interaction (e.g. from γ = 0.001 to γ = 0.005) the basins of attraction of the asymmetric steady states get smaller, because maintenance of the smaller epidemic at a low level gets harder with the stronger inflow from the larger epidemic.

γ > γ crit When interaction is just a little larger than the critical value derived in the last section the asymmetric steady states lose optimality, so we know that strategy is no longer optimal. P lies then in the basin of attraction of (L, L) and for both epidemics eradication is optimal

Page 21 of 29

A c c e p t e d M a n u s c r i p t

(Figure 2iii, γ = 0.01 in Figure 6). That seems to be counterintuitive because a larger interaction leads to a stronger bond between the epidemics and one could assume that the larger state drags the smaller to a high steady state.

A possible explanation is that without control, A and B would converge to the high level of use. Even in a situation where there are initially no users in B, A would cause a persistent drug problem B. Optimal control may prevent this. So even if drug A is past the (single) drug threshold where it is optimal to accommodate, the prevention of the emergence of a second drug problem can suddenly make it optimal to switch to an eradication strategy not only for drug B but also for drug A. This holds for a relatively moderate linkage of the systems.

γ >> γ crit With a further increase of interaction the situation changes again. Connecting flows of this scale (γ = 1 and larger) tie the systems so closely together that the optimal trajectories of A and B coincide after a short period of time. In the phase portrait (Figure 2iv) this manifests in the trajectories moving close to the 45 degree line.

The convexity of the DNSS curve that separates (L, L) from (H, H) lessens, thus the basin of attraction of (H, H) expands and the basin of (L, L) shrinks. As a consequence, the optimal policy for P changes yet again, this time from eradication to accommodation of both drugs. When the states are so strongly connected it is only the sum of the states that determines the optimal outcome. The DNSS curve is close to A + B = 2A DN SS , where A DN SS is the DNSS threshold of the corresponding one-dimensional system. In this case the two-dimensional system is merely the sum of the two one-dimensional systems and may be treated as a one-dimensional system.

This example shows that even small perturbations of the linkage of the systems can lead to a total reversal of the optimal policy. The non-monotonicity of the optimal policy with respect to the nature of the linkage of the systems underlines the importance of the potential impact of a second, parallel system. Since for certain initial values a small change or misunderstanding of the strength of interaction can change the optimal policy as profoundly as from eradication to accommodation, a very good understanding of that linkage is crucial.

A c c e p t e d M a n u s c r i p t One comment is in order. The central DNSS curve is convex in the states. That implies that for moderate levels of interaction the optimal long-run steady state depends on the relation and not only the sum of A(0) to B(0). The greater the differences are between A(0) and B(0), the more favorable is an eradication strategy. From an application point of view, this is a sensible result. Small epidemics are more easily controlled (cf. the marginal efficiency of control), so the decision maker can then concentrate on the larger epidemic, where a reduction of the number of users has (because of the nature of the interaction) a greater impact on the system.

In Two-Dimensional Systems Not All Thresholds are Tipping Points

So far we have discussed two identical systems that could be independently controlled. But what if a parallel system exists that is outside the influence of the decision maker? In the Ramsey growth model this would be a large economy in a neighboring country. For the control of fashion we could imagine a competing product that emerges. And in illicit drug use it could be drug users in an adjacent country that do not impose costs on the decision maker but interact by migration with the decision maker's drug problem.

Hence, in the following we discuss a case where parallel systems are symmetrically connected and only one of the systems, A, is controllable and costly to the decision maker, i.e. for the social costs parameter we have κ B = 0 and for the control, the efficiency of u B is c B = 0.

In such a situation two unconnected systems would yield three optimal steady states. The uncontrolled system converges for all initial values to a high steady state, whereas the controlled system converges to either of two locally optimal equilibria that are separated by a DNSS threshold. By introducing interaction with B > 0 a persistent flow of users from B to A exists that makes it difficult to preserve an overall low level of use. The level of interaction we have chosen is γ = 0.01. That is high enough that in Figure 2 the structure of four optimal steady states is not optimal anymore.

Figure 7 shows a phase portrait in the state space. There is only a single optimal steady state, where both epidemics are at a high level of use. Consequently there is no DNSS curve. However, the influence of the parallel system manifests in the occurrence of an "indifference" curve that separates two different optimal policies that lead to the same steady state. So, not only is the optimal outcome influenced but so is the way the system gets there. The same phenomenon occurred in the hairline case depicted in Figure 5, A c c e p t e d M a n u s c r i p t however the indifference curve here exists in a sufficiently large region of the parameter space to qualify as relevant for practical applications.

When the system starts at a point on the indifference curve, such as (A, B) = (2.1, 0.2), the decision maker can choose between optimal control strategies that result in either trajectory pt 1 or pt 2 . Either strategy incurs the same total costs. The number of costly users, A, is for the most part significantly lower on pt 1 than on pt 2 , but the control costs are higher which is reflected in the optimal controls along these paths, shown in Figure 7.

When choosing pt 1 , the decision maker applies u pt 1 A to a great extent, thus driving the number of users in A down at first. After passing this point, little control is applied and the epidemics grow to the steady state value. The sharp bend of u pt 1

A is due to the projection of the control into the A-u space. In the phase portrait the optimal path stays at a very low A-level for some time which cannot be shown in the projection. Along path pt 2 the epidemics head directly to the steady state, driven by control u pt 2 A , which stays at a moderate level throughout the course of the epidemic. The money saved by using a weaker control is offset by the larger number of users in the A state.

Hence, what we observe here is a point of indifference that does not separate the basins of attraction of different equilibria, but rather separates dramatically different strategies for getting to a single equilibrium.

This cannot happen in isolated one-dimensional systems, for which such indifference points can be excluded. The present example shows that in interacting systems answering "where should we go?" is not sufficient. One must also ask, "Even given where we should go, which among multiple optimal ways of getting there should we choose?". Hence, looking only at the isolated one-dimensional system may simplify to the point of distortion even the way the choices are framed. So the discussion has to become more specific and the "dimension" of the questions asked has to be adapted to fit the complexity of the system considered.

Discussion and Conclusions

In this paper we generalized a one-dimensional optimal control problem with a DNSS property to a two-dimensional optimal control problem by taking the "direct product" of the model with itself. The resulting problem describes a situation where two similar, parallel systems are jointly managed by a single decision maker. We use a particular example out of the domain of drug control to perform a case study on this class of models. The model is constructed in a way that it easily can be adapted to fit a whole range of relevant scenarios; for a deeper discussion see [START_REF] Zeiler | Optimal dynamic control with DNSS curves: Multiple equilibria in epidemic models of HIV/AIDS and illicit drug use[END_REF]. We start the analysis with two identically parameterized models and show by gradually introducing interaction that a little interaction can have a tremendous impact on the optimal solution. In fact, a minor change can actually reverse the policy prescription yielded from the analysis of an underlying one-dimensional system.

This finding is all the more significant when one considers that not just the magnitude of the interaction was modest, but also that the nature of the interaction belongs to a specific class, namely the migration from one system to the other with no net creation or destruction of the sum of the state variables in total caused (directly) by interaction of the two states. That is, the sum ẋ + ẏ depends only on (x, y) and not at all on the presence or extent of interaction.

Furthermore, we could show that some general invariance properties one might have expected to hold, do not in fact hold. For example, one might have expected that if in an already escalated situation a high level of drug use is optimal in the one-state model, that this carries over to the twodimensional coupled case. However, we show that this does not necessarily Page 25 of 29 A c c e p t e d M a n u s c r i p t hold true, because the optimal outcome depends non-monotonically on interaction. What is optimal for small interaction may be optimal for strong interaction, but not for an intermediate situation.

Another important finding in the drug control context is that the well established paradigm of "eradication" or "accommodation" has to be reconsidered. When two interacting drug markets are modeled, the optimal answer may be a mix of "eradication" and "accommodation" strategies, the optimality of which can crucially depend on the initial state of the system. So, in further analyses the whole system of drug use should be taken into account. So researchers could take this as inspiration for finding out more about the nature of linkages between substances of abuse.

The analysis performed here may be specific for drug control, however, our approach can be carried over to other optimal control problems that concern other domains. We presented here a proof by example that introducing modest interaction into the product of two isolated one-state systems can make a big difference. This suggests that a sensible line of further research would be to revisit various classic one-dimensional systems and pursue parallel analyses in those other domains as well.

A generalization of [START_REF] Sethi | Optimal advertising policy with the contagion model[END_REF] optimal advertising problem to two states for example, could reveal surprising results on advertising different, exchangeable products (e.g. two cars of the same automaker). The optimal advertising policy may be changed by the introduction of a new product. The question of whether to advertise or not may have to be extended by the second dimension to discordant strategies and the answer might depend crucially on the degree of interaction (i.e. the change of the product preferred by customers).

A scenario akin to the uncontrollable drug problem in an adjacent country (Section 3.4) can arise in a higher dimensional version of the Ramsey growth model. If there exists a second economy sufficiently large that it does not act strategically with respect to the first economy then this parallel, interacting system may have a large impact on the optimal strategy of the first country, even if the second economy does not play a direct role in the objective function. This setting could be studied in a game theoretic framework, as well. Here, the uncontrolled state could be controlled by another player who is influenced by the behavior of the focal player. DNSS thresholds in such a setting would enrich existing literature on this phenomenon (cf. [START_REF] Dechert | The stochastic lake game: A numerical solution[END_REF].

Further models for possible investigation include a higher dimensional

25

A c c e p t e d M a n u s c r i p t version of the shallow lake system investigated by e.g. [START_REF] Mäler | The economics of shallow lakes[END_REF], multi-dimensional models in the area of other economic problems, such as multiple capital stocks, and models of infectious disease dynamics.

Besides substantively interesting insights we found two methodological implications as well. First, the occurrence of the heteroclinic connection in our model that bounds the area in the parameter space where DNSS thresholds exist hints that Wagener's general result on the existence of DNSS thresholds in one-dimensional models might extend to higher dimensional models. This is relevant, because analytically results on DNSS thresholds are rare in the literature.

Second, by introducing some asymmetry to the model we could find an "indifference" curve of significant length. Although this phenomenon is not new in optimal control theory, it is rare and important enough to be highlighted here. In a one-dimensional model, tipping point behavior is restricted to DNSS points, starting at which there are two options, namely approaching a low-level steady state or a high-level steady state. In two-dimensional models, however, "low" or "high" are not the only choices, so the paths leading to an optimal outcome, can become a policy relevant question.

In conclusion, we set out to address the question "Can the presence of an additional, parallel one-state system substantially impact the policy prescriptions for the original one-state system?". We know now that the simple answer is "Yes, and it can do so in several ways."

Page 27 of 29

)

  Both the state and control variables must be nonnegative. The parameter set ρ at the base case values consists of κ 3.396819914 social costs per unit of consumption including a proportionality constant, p 0.12454 retail price per gram cocaine in 1000$, ω 0.5 absolute value of the price elasticity of demand, k 1.5813 • 10 -8 initiation proportionality constant, a 0.25 absolute value of the elasticity of participation, m 1.625 • 10 7 logistic initiation function constant, c 0.043229 control efficiency proportionality constant, z 0.5 control function exponent, µ 0.181282758 desistance proportionality constant, and b 0.25 elasticity of desistance. The time discount rate, r, is 0.04. Page 9 of 29 A c c e p t e d M a n u s c r i p t

  by setting H u i = 0. If any u i = 0, i ∈ {A, B}, (12) yields η i ≤ 0, where an explicit form for η i can be derived from L u i = 0.Furthermore, the costate equations are given by λA = rλ A -H A and λB = rλ B -H B .The limiting transversality conditions for the costates λ A and λ B are lim t→∞ e -rt λ A (t) = 0 and lim t→∞ e -rt λ B (t) = 0

Figure 1 :

 1 Figure 1: Phase portrait of the one-state one-control optimal control problem (i) and of two uncoupled systems (ii).

Figure 2 :

 2 Figure2: Phase portraits of identical epidemics for different levels of interaction, (i) γ = 0.001, (ii) γ = 0.005, (iii) γ = 0.01, and (iv) γ = 1. Bold lines are DNSS curves, thin lines are optimal trajectories for given initial values on the DNSS curves. Steady states are indicated by an "×"; "•" indicates a particular initial value, P . The direction of the phase flow is indicated by arrows.

Figure 3 :

 3 Figure 3: Heteroclinic connection of two equilibria at γ crit = 0.0051987.

Figure 4 :Figure 5 :

 45 Figure 4: Values of the Hamiltonian as a function of A. On the solid line the low level of use is approached; on the dashed line (L, H) is approached. The values of the Hamiltonian coincide in (L, H).

Figure 6 :

 6 Figure6: Optimal paths for different levels of interaction, all starting at a certain initial value, P .

Figure 7 :

 7 Figure 7: Similar drugs in different regions, κ B = 0, c B = 0 and γ = 0.01. The left figure shows a phase portrait with the "indifference" curve highlighted. The right figure shows the optimal controls along the paths starting on the indifference curve on (A, B) = (2.1, 0.2).

Sufficiency holds if the Hamiltonian is jointly convex in both the states and the controls.

Acknowledgement

This work was supported by the Austrian Science Fund (FWF) under contract No. P18527-G14, the Qatar National Research Fund, and by a Robert Wood Johnson Foundation Investigator Award in Health Policy Research. The views expressed imply no endorsement by the Robert Wood Johnson Foundation.

The authors gratefully acknowledge the helpful and inspiring comments of Gustav Feichtinger, Dieter Grass, Peter Kort, Alois Steindl, and of one anonymous reviewer.

A c c e p t e d M a n u s c r i p t


Loukas Balafoutas
email: loukas.balafoutas@uibk.ac.at.

Gary Charness

Ed Hopkins

Santiago Sanchez-Pages

Stephane Straub

Kohei Kawamura

Matthias Sutter

Keywords: psychological games, corruption, bureaucracy, guilt, third party. JEL codes: C73, D73 psychological games, corruption, bureaucracy, guilt, third party. JEL codes: C73, D73

Resear h highlights for: Pu li eliefs a d orruptio i a repeated psy hologi al ga e

 Public beliefs about corruption affect the behavior of a guilt averse bureaucrat.  A lobby pays a strategic bribe to the bureaucrat, and this bribe falls over time.  Corruption is more likely with a long time horizon and fast-moving beliefs.  Public beliefs about corruption are self-fulfilling.

A c c e p t e d M a n u s c r i p t

Introduction

The economic literature on corruption typically assumes that government officials rationally weigh their expected monetary costs and benefits from corruption, and accordingly decide whether to behave corruptly or not. Moreover, the principal-agent framework is typically used for a structural analysis of corruption (see, for example, Becker and Stigler, 1974; Rose-Ackerman, 1975, 1978; Klitgaard, 1988, 1991; Shleifer and Vishny, 1993; Acemoglu and Verdier, 2000). The resulting implications for policymakers are that corrupt behavior can be curbed by designing appropriate incentive schemes, which increase the costs of corruption for government officials (penalties, effective monitoring), reduce its benefits (limits to the discretionary power of officials), or increase the benefits of honest behavior (efficiency wages, outcome-contingent contracts). 1 In this paper we focus on a different factor that can affect corrupt behavior. We study the role of emotions for corruption in public administration and suggest that corruption may depend on the beliefs of the public in the context of a repeated psychological game.

Psychological games (see Geanakoplos et al., 1989; Battigalli and Dufwenberg, 2009) incorporate emotions in the motivation of players by including in their utility not only material payoffs, but also higher-order beliefs. The inclusion of beliefs in utility can lead to emotions such as guilt, embarrassment, disappointment, anger, surprise, or gratitude. In our model we consider guilt aversion as a driving factor for the behavior of bureaucrats, and in order to do this we extend their utility function beyond monetary costs and rewards and include in it the psychological cost of guilt that arises from corrupt behavior. This idea is already mentioned in Klitgaard (1991), who argues that raising the "moral costs" of corruption can be one of five broad sets of tools to control corruption. These "moral costs" of corruption refer mainly to organizational cultures and codes of ethics, something that is applicable to bureaucracies. 2 Guilt aversion is defined in Charness and Dufwenberg (2006) as the emotion that arises when a player "believes he hurts others relative to what they believe they will get".

Further theoretical models on guilt include Dufwenberg (2002), Battigalli and Dufwenberg (2007), while experimental evidence on the effect of guilt on decision-making is abundant (see, for instance, [START_REF] Dufwenberg | Measuring beliefs in an experimental lost wallet game[END_REF][START_REF] Guerra | Trust responsiveness and beliefs[END_REF][START_REF] Charness | Promises and partnership[END_REF][START_REF] Dufwenberg | The framing of games and the psychology of strategic choice[END_REF][START_REF] Bacharach | The self-fulfilling property of trust: An experimental study[END_REF]Battigalli and 1 On the question of which factors can help reduce the level of corruption, see also Mookherjee and Png (1995), Margit and Shi (1998), Waller et al. (2002). 2 See also van Winden (2007:57) for a discussion on the importance of incorporating emotions in the domain of public choice: "In my view, by now sufficient evidence exists to replace homo economicus by homo sapiens, a boundedly rational and emotional agent, in the private sector as well as the public sector."

A c c e p t e d M a n u s c r i p t 3 Dufwenberg, 2007; Reuben et al., 2009). Corazzini et al. (2007) offer experimental evidence in support of the hypothesis that politicians are motivated by guilt aversion from letting the electorate down. Although their setting is based on electoral competition, the results are important for us because they link the behavior of politicians to their beliefs about the expectations of the electorate, and therefore strengthen the case for a model that adds guilt aversion to the study of corruption in the domain of public administration.

Our stylized model of corruption in public administration shows that, when the bureaucrat is averse to letting the public down, corruption is less likely to emerge in equilibrium compared to a version of the game without psychological payoffs. The model predicts that -under certain assumptions-corruption can lead to more corruption, if the selffulfilling beliefs of the public enter a downward trajectory. Whether or not a society becomes entrapped in such a corruption spiral depends on a number of factors: corruption is more likely when the psychological component in the bureaucrat's utility is strong relative to the monetary incentives (i.e., when he is very guilt averse), when public beliefs are initially low and are updated fast, and when the expected horizon of the game is relatively long.

In broader terms, the paper adds to the literature that studies the role of beliefs and emotions in decision-making, by considering the impact of an inactive third party on a cooperation game played between two other players, under the assumption of guilt aversion.

The model takes a close look at the dynamic properties of beliefs and at how these affect the outcome of the game. Beliefs are initially constrained to be constant over time. We then relax this assumption and move to a version of the model in which players update their beliefs. In such a setting, players must take into account the effect of their actions on others' beliefs in the future; this introduces an additional element of strategic behaviour.

In another application of psychological game theory to the field of public administration and corruption, Huang and Wu (1994) look at a psychological game of trust, where a player's remorse from betraying another player's trust is proportional to their expectations. The central insight of Huang and Wu (1994) is that the expectation-dependent moral cost of betrayal leads to a multiplicity of equilibria, some of which feature (at least some degree of) honest behavior. The authors use corruption in bureaucratic organizations as their central example: bureaucrats may act less corruptly if they believe that the public has high expectations of them. Then, different equilibria with their corresponding sets of beliefs can be viewed as alternative social norms. Compared to Huang and Wu (1994), we endogenize the benefit from corruption by introducing a bribe-paying lobby. Moreover, as the bribe can vary over time, so can the moral cost of corruption. Another paper assuming that A c c e p t e d M a n u s c r i p t bureaucrats are averse to letting the public down is Varoufakis (2006), who studies the evolution of corruption in bureaucracies with the added twist of linking it to political participation.

A model of corruption in public administration

Players, structure and beliefs

Consider the following three-person game. Player A is a bureaucrat, player B is (some representative of) a business or lobby, and player C is the public. 3 The lobby can bribe the bureaucrat with the purpose of eliciting some favor from him (e.g., a procurement contract with favorable terms), which amounts to corruption. If the bribe is high enough, the lobby may be able to persuade the bureaucrat to act corruptly, which however generates a negative externality for the public. Consistent with the widely used definition of corruption as the "misuse of public office for private gain" (Rose-Ackerman, 1999), the bureaucrat can thus choose an action that hurts the public but ensures a higher payoff for him.

To formalize these ideas, let there be two distinct sets of monetary payoffs, called the high and the low set, Beliefs are defined as follows. Let p denote the probability that A will choose H, and let π denote C's expectation of p. Finally, q is A's expectation of C's expectation of p, in other words his second order belief. In brief, p=Pr(H), π=E C (p), q=E A (π). Player B's expectation of 3 I intentionally refer to a bureaucrat and not a politician. The latter is elected and therefore the public is far from being an inactive player. Bureaucrats, on the other hand, are appointed and their positions are often permanent (although it is true that their careers can depend on political affiliations). 4 Player A could in theory choose L even if the bribe is lower than (a H -a L ). This situation is not defined as corruption here, since it reduces A's payoff.

  L H ,   ,
A c c e p t e d M a n u s c r i p t 5 π equals that of player A, i.e., it equals q. 5 Unless otherwise stated, the term "beliefs" will refer throughout the paper to the bureaucrat's second order belief q.

The utility functions of players B and C comprise simply their monetary payoffs. As explained in the introduction, the central assumption of the model is that player A is guilt averse, i.e., he is averse to letting player C down. This is modeled as follows. If A chooses L, his utility includes the monetary payoff and the bribe (if any), but also a psychological cost from betraying C's expectations (see Figure 1). In other words, there is an endogenous benefit to corruption (equal to the bribe), and a dual cost to it. The bureaucrat may have to exert some effort in order to grant the favor or in order to preserve secrecy and avoid getting caught -this cost corresponds to the difference a H -a L . Alternatively, this difference can be thought of as the expected cost from being caught (probability of detection multiplied by the penalty). In addition to this direct cost, the bureaucrat suffers a moral cost of corruption. This is equal to γq, where the parameter γ measures the intensity of guilt aversion. The value of γ is common knowledge. 6 The model conditions guilt only on the beliefs of the public, and not on those of the lobby. Section 4 relaxes this assumption and looks at the implications of allowing the bureaucrat to feel guilt when he lets any of the two players down.

Figure 1: Player A's utility

Under our assumption of guilt aversion, the bureaucrat is not a purely selfish moneymaximizer and his motivation includes the fact that he feels he must live up to the expectations of the public. This sense of duty and responsibility is stronger, the higher the public's expectations. Consider a bureaucrat who acts corruptly. The term (-γq) in his utility function implies that this bureaucrat will feel less guilt in a society where corruption is widely regarded as very likely than in a society that holds high expectations of him. 5 The assumption that A and B have the same second order beliefs can be relaxed without leading to any qualitative changes in the results, but increasing the complexity of the analysis. 6 On this assumption, see the discussion in section 3.1.

The one-shot solution

Given the timing of events, backwards induction reveals that no bribe is paid in equilibrium, since player B correctly anticipates that A's dominant strategy is to choose H for every possible amount of the bribe. This is a standard commitment problem and corruption cannot be sustained in equilibrium. In this example, the psychological assumption regarding the bureaucrat's utility does not affect the outcome. Indeed, guilt aversion pushes him even stronger towards choosing H, since the choice of L generates an additional psychological cost.

However, beliefs can make a difference when the game is indefinitely (even if finitely) repeated.

The repeated game

Let the above game be played repeatedly and suppose that after every repetition the probability that the game will be played once more equals

) 1 , 0 (  r
, where r is constant across periods. In this dynamic context, players may be able to establish corruption by adopting some conditional collusive strategy. In particular, consider the possibility that B adopts the following grim trigger strategy (called S B ): "Pay a positive bribe x in the first period, and then only keep paying the same bribe provided A has colluded in the previous period; switch permanently to a zero bribe if in any period A does not collude". Turning to the bureaucrat, he may also adopt a trigger strategy of the following sort: "Collude as long as B has paid at least y, never collude if B pays less than y in any period".

Given these trigger strategies, an infinite number of equilibria are possible, some of which support corruption (those in which y x  ). The problem is that there is no natural way of selecting among these equilibria. 7 Although there are infinite intertemporal strategies that can be adopted by the players and supported in equilibrium, we shall confine ourselves in what follows to certain focal strategies. In particular, the analysis will focus here on the case where B plays S B and A plays the following simple strategy, called S A : "Collude as long as the payoff from corruption is at least equal to the payoff from no corruption". These strategies correspond to a situation where the lobby is able to extract the maximum benefit from corruption, while keeping the bureaucrat indifferent between corruption and no corruption. Moreover, when public beliefs are updated, these strategies have the interesting feature that they generate time-varying bribes, which are functions of those beliefs. In the appendix we discuss another extreme case, where strategies are such that the bureaucrat is 7 As Binmore and Shaked (2010:88) point out, "the case of multiple Nash equilibria is hard, because most game theorists regard the equilibrium selection problem as unsolved".

A c c e p t e d M a n u s c r i p t 7 able to extract the maximum possible bribe from the lobby. The choice of these two cases will allow us to pin down the respective corruption equilibria with extreme allocations: one where the lobby is able to capture the entire surplus from corruption, and one where the bureaucrat captures the entire surplus. For the sake of simplicity we will only study these two cases, although any intermediate sharing of the surplus is possible in some equilibrium of the repeated game.

Let q, the bureaucrat's second-order belief, be constant throughout the horizon of the game. We also require that q be correct in equilibrium, i.e., that the second-order belief correspond to reality. This assumption is necessary in order to use the concept of a psychological Nash equilibrium (henceforth PNE, see Geanakoplos et al., 1989) for the solution of the game. Both of these assumptions will be relaxed in the next section.

Player B plays S B and player A plays S A . As we have seen, this strategy amounts to A colluding provided that the bribe covers at least the difference between a H and a L , and it is the strategy that gives the lowest collusive payoff to A and the highest collusive payoff to B.

Given that B plays S B each period, and given that none of the parameters change over time, she will pay the same bribe k every period if there is corruption. Accordingly, A will either collude in every period or he will never collude. The associated respective total expected payoffs for A are:

) 1 ( ) ( r q k a L     , ) 1 /( r a k H  
. If A honours B's trust and colludes, then each period his utility equals the low monetary payoff a L plus the bribe minus the cost from guilt. If he chooses not to collude, then his utility is high in the first period, but in all subsequent periods he receives only a H . Comparing the two payoffs, it follows that player A will collude as long as:

k q a a r L H / ] ) [(    
.

(1)

This expression conditions the sustainability of corruption on the effect of guilt (γq), the monetary loss that player A suffers if he chooses L (a H -a L ), the expected duration of the interaction and the amount of the bribe. For A to collude, the bribe and the expected horizon of the game must be sufficiently high with respect to the difference in monetary payoffs and to the psychological cost of disappointing the public.

Lemma 1

Given S A and S B , if corruption is the equilibrium of the game B pays the following bribe k * every period:

r q a a k L H / ) ( *    

This follows from (1).

A c c e p t e d M a n u s c r i p t 8 A bribe of k * compensates A for the utility loss caused by the lower monetary payoff and the cost of guilt. A high continuation probability r reduces the amount of the bribe. This accounts for the fact that the bribe is paid in advance, so that in any given period T the benefit to A from corruption lies in the future while the cost is borne starting at T.

Proposition 1

Given that A and B follow strategies S A and S B respectively, the game has two possible equilibria:

(i) The corruption equilibrium, with Π=L in every period, p=q=0, and k * =(a H -a L )/r.

(ii) The no corruption equilibrium, with Π=H in every period, p=q=1, and k * =0.

If H L L H b b r a a     / ) (  , the equilibrium is necessarily (i) If H L L H b b r a a    / ) (
, the equilibrium is necessarily (ii)

Otherwise, both equilibria are possible.

Proof:

H L L H b b r a a     / ) (  means that, from (
1), B can always pay a bribe high enough (and still lower than b L -b H ) to sustain corruption, even if perceived expectations are at the highest possible level (q=1). Knowing this, the public is convinced that A and B will collude, q is actually zero and A confirms these expectations by choosing L. On the other hand, The above equilibria satisfy the definition of a PNE: players' strategies are optimal and beliefs correspond to actual play. There is a multiplicity (at least within certain parameter values), meaning that we could end up in any of the two equilibria. 8 In this repeated game, the psychological element in the bureaucrat's motivation makes corruption less likely to emerge in equilibrium. To see this, note that in the absence of guilt aversion (i.e., when γ=0) the necessary and sufficient condition for corruption is:

H L L H b b r a a    / ) ( means that, from (
H L L H b b r a a    / ) ( (2)
This is the solution to the standard, non-psychological version of the game. Comparing

(2) with the conditions given in Proposition 1 reveals that, for certain parameter values, corruption is always the outcome when γ=0 but not necessarily when γ>0.

8 Of course, as in any other infinitely repeated game of interest, many other equilibria exist if we consider strategies other than S A and S B , inclusive equilibria in mixed strategies (which necessarily involve less cooperation than the pure strategy equilibrium characterized in Proposition 1).

A c c e p t e d M a n u s c r i p t 9

Time-varying beliefs

General

So far it has been assumed that beliefs are constant and that they correspond to the equilibrium outcome, which is reached in the first period and does not change over the horizon of the game. These assumptions are now relaxed and beliefs are allowed to vary over time. Before we look at the specifics of the model, we begin with a remark regarding equilibrium. The model in this section will posit a specific behavioral assumption regarding the evolution of beliefs (in the form of an updating process), generating beliefs that are not necessarily correct in equilibrium. This means that the model no longer satisfies the assumptions required for the application of PNE or sequential equilibrium (SE) as solution concepts. 9 In light of this fact, we will consider a state in which players A and B are in equilibrium in their corruption game, but the inactive player C may hold out-of-equilibrium beliefs about its outcome.

One way to motivate the above ideas is to consider the possibility of asymmetric information in the model. It was assumed that all the parameters (payoffs and psychological factors) are common knowledge; this assumption appears unrealistic in many settings, particularly in the context of psychological games. 10 If player C knows all the parameters of the model as well as the strategies that A and B follow, beliefs will jump to a corner and we will end up with one of the equilibria described in Proposition 1. If, instead, information is incomplete, it seems appropriate to not constrain beliefs to be correct in equilibrium, but to allow them to take interior values. For this, it is sufficient to think that C may not know some of the parameters of the model (for instance the psychological parameter γ), or that he simply does not know the particular strategies S A and S B that the other players follow, which would mean that from C's point of view anything could be an equilibrium of the repeated game.

On a related issue, the analysis of the game hinges crucially on the assumption that B knows how guilt averse A is, and thus, she can pay the appropriate bribe. This is admittedly a strong assumption-even though it is common in the literature on psychological games. To motivate it, one can think that the lobby invests some resources in order to find out the extent to which a given bureaucrat is prone to corruption (e.g., based on his behavior in other 9 Both the SE (Battigalli and Dufwenberg, 2009) and PNE require that beliefs of any order be correct in equilibrium. See Battigali and Dufwenberg (2009: 16), for their consistency assumption as well as its analogy to PNE.

10 "Unless one models interaction within a family or amongst friends, it is probably not realistic to assume that players know each other's psychological propensities" (Battigalli and Dufwenberg 2009:27). Also, Rabin (1993) motivates the study of psychological games with incomplete information, where a player's knowledge of other players' motives may be limited.

A c c e p t e d M a n u s c r i p t 10 positions in the past). The public is not organized and able to undertake collective action in a similar way as the lobby, and therefore may not know the true value of γ.

The model with updated beliefs

Beliefs are initially at some exogenous level q 0 and they can change over time (q=q t ): the public updates its belief about A's choice each period taking into account A's choice in the preceding period, and players A and B update their perceived beliefs in exactly the same way.11 Before imposing a specific process, we begin with a general case where the only assumption is that beliefs fall every time player A chooses L, and that they rise every time he chooses H:

             0 , 1 , 1 1 t t t t t t t t q and L if q q q and H if q q
(3)

Following the discussion in section 3.1, we define an equilibrium of our corruption game with updated beliefs as a pair of strategies S A and S B of players A and B, and a belief q t of player C, such that: (i) in every period, S A and S B are best responses to each other, and (ii) q t is updated according to (3).

Proceeding as in section 2, we can determine the amount of the bribe that in every period makes player A indifferent between always colluding and never colluding (from that period onwards). This is given by: k * t = (a Ha L + γq t )/r (4) Proposition 2

Given (3), S A and S B , the unique optimal sequence of bribe payments made by player B in the corruption equilibrium is k * t as given by ( 4).

Proof: In any period T throughout the game, no bribe k T >k * T will be paid; since k * T is high enough to make player A prefer the collusive outcome subject to k * t being offered in all subsequent periods, a higher bribe unnecessarily reduces B's payoff. On the other hand, no k T <k * T is offered in period T either. Such a bribe is lower than what is needed to compensate player A, so the latter will not collude in period T with k T <k * T , unless he expects to be compensated for the loss in T in some future period(s). But such an expectation is not rational and any such commitment by player B would not be credible, since we have just seen that k t will never exceed k * t . Thus, k T must equal exactly k * T .

A c c e p t e d M a n u s c r i p t

Corollary 1

Given (3), S A and S B , the sequence of optimal bribe payments in the corruption equilibrium is strictly decreasing over time:

* 1 *   t t k k
This follows immediately from (3) and (4).

This corollary is saying that, in this particular setting, the bribe that the lobby must pay falls over time. Hence, corruption becomes cheaper for the lobby each period. The reason why this happens is that the bribe must compensate the bureaucrat for the cost of guilt, and this cost falls each period as expectations become lower. This result implies that, by lowering public expectations and eroding confidence in public administration over time, corruption paves the way for more corruption. 12 We will now examine the conditions under which the corruption equilibrium emerges.

In any period, paying bribe in exchange for the choice of L is profitable for B if k t <b L -b H , i.e., if her cost (bribe) is lower than her benefit. Moreover, from Corollary 1, if this action is profitable in the first period, it will be profitable in all subsequent periods. The intuition is that expectations fall over time and the required compensation diminishes as guilt becomes weaker. This leads to the fact that an equilibrium with corruption in every period is definitely profitable for player B if:

H L L H b b r q a a     / ) ( 0 
(5)

This condition ties the emergence of corruption to initial beliefs. But that is not the whole story. Condition (5) is sufficient for corruption to be profitable for B, but it is not necessary. It may be the case that B wants to sustain corruption even if it is unprofitable for her in the first period. The reason is that, as beliefs fall, k * t will also fall over time so that the total expected return from always paying the bribe may be positive. In other words, player B might make a monetary sacrifice today in the expectation that she will be compensated for it tomorrow, since the effect of this sacrifice will be to drive expectations down. In order to be able to say something about the outcome of the game in that case, we must assume a specific dynamic process for beliefs and assess the benefits and costs of setting k t =k * t as in (4) throughout the horizon of the game. Let beliefs be updated according to the following process: 12 Another paper in which corruption leads to more corruption is Tirole (1996). In a principal-agent model, Tirole (1996) shows how an economy can become "locked" in a high corruption steady state when the principal's source of information about the agent is the agent's group reputation.

A c c e p t e d M a n u s c r i p t

                0 , 1 ), 1 ( 1 1 t t t t t t t t t t q and L if q q q q and H if q q q   (6)
This process specifies that in any given period beliefs will move either upwards, if there was no corruption in the previous period, or downwards if there was corruption. 13 The

parameter ] 1 , 0 [  
measures the speed of beliefs adjustment. 14 If ρ =1, beliefs jump to a corner after the first period. In the opposite case where ρ=0, beliefs do not move over time and remain at their initial value of q 0 (as in the previous section). This process captures the main features of the adjustment of beliefs that we have in mind, although one can think of alternative specifications.

Solving the difference equations in (6) gives the following expressions for beliefs at any point in time as a function of initial beliefs:

            m equilibriu corruption the in q q m equilibriu corruption no the in q q t t t t t , ) 1 ( , ) 1 ( 1 ) 1 ( 0 0    (7)
From ( 4) and ( 7), we obtain the level of the bribe in any period t:

r q a a k t L H t / ] ) 1 ( [ 0 *       (8) Proposition 3
When beliefs follow the dynamic process given in ( 6), and given S A and S B , the game has two possible equilibria:

(i) If H L L H b b r r r q r a a        )] 1 ( 1 [ ) 1 ( ) ( 0   , ( 9
)
the outcome is corruption in every period with k t =k * t as given in ( 8). (ii) If ( 9) does not hold, corruption never occurs.

Proof: The corruption equilibrium emerges as long as the expected total value of the bribe payments throughout the game does not exceed the expected value of the total benefit of corruption to player B:

        0 0 * ) ( t t H L t t t r b b r k
, which translates to (9). In such a case, player B will always pay k * t , because

A c c e p t e d M a n u s c r i p t 13 to her this is the least costly way of sustaining corruption. On the contrary, if (9) does not hold, player B never pays a bribe and player A always chooses H.

Discussion: Factors that facilitate corruption

Starting from an initial level of beliefs and observing the incidence -or not-of corruption, the public updates its expectations every period. Proposition 3 predicts that, as long as the exogenous parameters do not change, corruption will occur either in all periods or in none. It therefore also predicts that trust in bureaucrats will keep moving, according to (6), either towards very high levels or towards complete disillusionment where everyone expects them to act corruptly. If that is the case, lobbies will eventually be able to capture bureaucrats with only small amounts of bribes. The intuition for this is that a bureaucrat who knows that the public considers him corrupt suffers a low moral cost when he actually behaves in a corrupt manner, so that he is eventually caught in a circle of self-fulfilling expectations.

Compared to the version of the game with constant beliefs, there is now no multiplicity of equilibria and the outcome is determined by (9). This condition reveals that corruption is more likely to emerge in equilibrium when: (i) the stakes of corruption (b L -b H ) are high and the cost to the bureaucrat (a H -a L ) is low, (ii) guilt aversion is relatively weak (low γ), (iii) initial beliefs q 0 are low, (iv) the expected horizon of the game is long: as r increases, the effect of guilt diminishes relative to the other factors that determine the outcome of the game, 15 making corruption less costly for the bureaucrat. In policy terms, this means that the frequent rotation of public servants can be an effective instrument against corruption, (v) Beliefs are updated fast (high ρ): if the public is quick to become disillusioned by a bureaucrat, then the latter is more likely to act corruptly in the first place. On the contrary, a bureaucrat is less likely to disappoint the public if beliefs are relatively resilient.

The intuition behind this result is that, if expectations decline fast following the incidence of corruption, then the psychological cost of guilt also diminishes fast and the bribe that is required to establish corruption falls. Note, also, that the model predicts that lobbies are likely to act strategically and offer very high bribes at initial stages of the corruption relationship, with the purpose of marring the image of public officials and making corruption easier and cheaper in the future. This strategy is more profitable, the higher is the speed of dynamic adjustment of beliefs.

The last two points (iv) and (v) highlight the fact that allowing beliefs to vary over time leads to a time-varying cost of guilt. If the public's expectations fall during the game,

15 This is because q t falls over time, whereas the other parameters remain constant.

A c c e p t e d M a n u s c r i p t 14 then so does the psychological cost to the bureaucrat of betraying these expectations, and vice versa. Summing up the main idea of the model, high perceived expectations make corruption harder to sustain because they imply a high guilt-related disutility for the bureaucrat; on the contrary, low expectations can actually lead to corruption. This highlights the self-fulfilling property of public beliefs in this game.

Guilt from letting both players down

A question that may arise concerning the specification of the game is the following: why does the bureaucrat feel guilt only when he lets the public down? In other words, why does he play a psychological game with one of the players and a standard (neoclassical) game with the other? To motivate this assumption, one may evoke social and psychological arguments. It is natural to assume that a bureaucrat has some sense of mission and responsibility towards the public that he is supposed to serve, but not towards a lobby. Nevertheless, this section modifies the model of section 3.2 by adding guilt from letting the lobby (player B) down. In particular, let us assume that, if a positive bribe is paid and player A selects H, he suffers guilt. In this context, guilt does not arise from the choice of H per se, but from the fact that A fails to respond to B's trust. 16 The only difference that this additional psychological assumption makes in the analysis of the game is that, if in period t player B pays a bribe and player A selects H, the latter's payoff becomes U H t =a H +k-ξμ t , where ξ measures the strength of guilt aversion and μ t is A's second-order belief, defined in the same way as q t : it is A's expectation of B's expectation of the probability that A will select L. Note however that, given S B , guilt can only occur once, in the first period when A does not collude. This is because in any subsequent period B will switch to a zero bribe strategy. Thus, the problem facing the bureaucrat is now not the same in every period. If he lets player B down he will have a lower utility in that particular period but will afterwards return to the high monetary payoff a H . For simplicity, let μ t =μ: since beliefs do not matter anymore after A selects H for the first time, there is no need for an updating rule similar to (3) or (6). Given S B and S A , the optimal sequence of bribes in the corruption equilibrium is now: k * t = [a Ha L + γq t -ξμ(1-r)]/r (10)

The difference between k * t in (4) and in (10) is the additional term -ξμ(1-r), which reduces the level of the bribe that is required to sustain corruption. Compared to the game of section 3.2, it is now relatively cheaper for the lobby to influence the bureaucrat, the more so the shorter is the horizon of the game. In fact, with the effect of guilt now going in two A c c e p t e d M a n u s c r i p t different directions, it is the relative disutility of letting the two players down that matters for the outcome of the game. The term (1-r) reflects the fact that guilt from letting B down can only occur once, so that only part of it enters A's utility seen from any point t in time. Then, given (6), the condition for corruption to be the equilibrium outcome becomes:

H L L H b b r r r r r q r a a          ) 1 ( ] ) 1 ( 1 [ ) 1 ( ) ( 0    (11)
In addition to the findings of section 3, we see here that corruption is more likely when player B's perceived beliefs (μ) are high and when A's guilt from disappointing B is strong (high ξ). The effect of the continuation probability r on the incidence of corruption is no longer clear-cut, because it works in two opposite directions through the second and third term on the left-hand side of (11).

Conclusion

The economic literature on corruption has largely neglected the role of emotions as a driving factor for the behavior of actors involved in corrupt activities. In this paper we consider how guilt aversion affects the behavior of a bureaucrat, and we do this by modelling corruption in public administration as the outcome of a game of collusion between a bureaucrat and a lobby, which generates an externality for the public. We have used a stylized model to show how guilt aversion can help mitigate the problem of corruption and how it affects the strategic behavior of lobbies and the likelihood that they are able to sustain corruption by bribing public officials. Some insights regarding the interaction between a bureaucrat, a lobby and the public are that declining public expectations allow the lobby to capture the bureaucrat with lower bribes; that this paves the way for more and more corruption as public beliefs enter a self-fulfilling downward trajectory; finally, that corruption is more likely when public beliefs are updated fast and when the interaction between the bureaucrat and the lobby has a long expected horizon.

The analysis of these issues has relied on specific assumptions regarding the psychological motivation of the bureaucrat, the updating of public beliefs and the common knowledge of certain parameters of the model, and it has concentrated on a focal subset of the strategies that players may adopt in the repeated game. In that sense, the paper leaves open many possibilities for future research, which could lead to a more general framework for the study of this topic.

  1), B cannot pay a bribe lower than b L -b H and still high enough to sustain corruption, even if q=0. Therefore, everyone expects the no-corruption equilibrium (p=q=1) and expectations are again confirmed.

The assumption that A and B have correct second order beliefs is not crucial. All that is required is that A and B have the same second order beliefs. See also footnote 5 on this.

The first part of the process that refers to the case where there is no corruption does not enter into any of the calculations and does not affect the results, because in the no corruption equilibrium the level of beliefs does not matter.

In fact, the speeds of upward and downward adjustment should not be constrained to be equal. This would account, for example, for the possibility that confidence might take a longer time to build than to destroy. Here we use only one parameter ρ for simplicity, since the upward path of beliefs is irrelevant to the outcome of the game.

If player A suffers guilt whenever he chooses H irrespective of whether a bribe has been paid, the results are almost identical to the ones that are obtained here.

Appendix: Considering a different set of strategies.

Consider once more the model of section 3.2. Player A may be able to maximize his payoff from corruption through the following trigger strategy, called S ' A : "Collude as long as the bribe is at least equal to y * =(b L -b H )". This more aggressive strategy maximizes player A's payoff, provided that player B follows the following simple strategy S' B , which is similar to S A : "Pay a bribe high enough to sustain corruption, as long as the payoff from corruption is at least equal to the payoff from no corruption".

The value of y * is such that player B is indifferent between corruption and no corruption. Given S ' A , S' B , and (3), the equilibrium of the game with updated beliefs is the following: B pays each period a bribe of k=y * and corruption is the equilibrium. B's total expected payoff from corruption is at its possible lowest level and equal to b H /(1-r), while A's

In this case we do not have a time-varying bribe as in the solution of section 3.2. The reason is that the bribe is at a maximum and does not fall over time given strategies S '

A and S '

B . The fall in public beliefs over time leads to an increasing payoff for the bureaucrat given that the bribe remains constant, while in section 3.2 it was the lobby that benefitted from the fall in beliefs.

If we consider a wider set of possible strategies by the two players, Corollary 1 on the decreasing bribe payments over time will hold as long as the moral cost of corruption for the bureaucrat, given by γq t , makes its way into the expression for y (the minimum bribe required by the bureaucrat according to his strategy). Of course, given any such set of strategies, corruption may or may not be the equilibrium outcome, depending on a number of factors captured by an inequality such as (9).