Éric Chien-Chung Huang

Colin De Verdière

Zhentao Li

Hang Zhou

Víctor Verdugo

Andrea Clementi

Tom Friedetzky

Reut Levi

Adrien Ash

Quay), Remy Tristan Sara

Till Thianhlanh

Keywords: stochastic processes, distributed computing, consensus, leader election, random walks, social networks Résumé processus stochastiques, processus distribués, consensus, élection de chef, marches aléatoires, réseaux sociaux stochastic processes, distributed computing, consensus, leader election, random walks, social networks

This thesis is devoted to the study of stochastic decentralized processes. Typical examples in the real world include the dynamics of weather and temperature, of traffic, the way we meet our friends, etc. We take the rich tool set from probability theory for the analysis of Markov Chains and employ it to study a wide range of such distributed processes: Forest Fire Model (social networks), Balls-into-Bins with Deleting Bins, and fundamental consensus dynamics and protocols such as the Voter Model, 2-Choices, and 3-Majority.

List of Figures

Introduction "So much of life, it seems to me, is determined by pure randomness."

-Sidney Poitier

This thesis is devoted to the study of stochastic distributed processes, i. e., processes evolving as a result of randomized decisions of interacting autonomous entities. Such processes surround us in many aspects of our daily lives and are imperative in the realm of distributed computing. Archetypal examples of such processes include the evolution of social networks, distribution of jobs in cloud computing, propagation of opinions and diseases, movement of atoms, evolution of stock markets, etc. 1

We focus on two groups of these processes: (i) dynamic processes, in which the entities arrive (and leave) over time, and (ii) consensus processes, in which the entities of a static network interact continuously with each other in order to reach consensus.

The dynamic processes we study in this thesis (Part I) are the Forest Fire Model, a model for the creation of social networks, and Balls-into-Bins with Deletions modeling the load distribution in systems. We are interested in the asymptotic behavior of these processes as the time goes to infinity.

The consensus processes (Part II) we study are: Voter, 2-Choices, 3-Majority, as well as faster, albeit slightly more sophisticated, processes.

At first glance, the wide-range of processes considered in this thesis appear to be unrelated; however, the common thread among them lies in our analysis. We relate each of these processes to (general) random walks and employ machinery from different areas of probability theory to analyze them.

1 Although not all of these processes are inherently stochastic-possibly even deterministic-accurate predictions by means of deterministic models are often infeasible. The sheer amount of data required alone renders this impractical, not to mention the complexity of the computations. Instead, treating theses processes as being random allows, in some cases, for good estimates such as predicting the number of times a die shows 6.

Part I: Dynamic Processes

The first process we study is the Forest Fire Process, a model for generating random graphs representing the evolution of social networks. The nodes in the generated graph represent the users and and edge represents friendship between the adjacent nodes.

Forest Fire Process. In the Forest Fire Process, a new user u arrives in every round, and connects to another user v chosen uniformly at random among those already present.

User u then becomes friends with a randomly chosen subset of v's friends. These friends then introduce u to some of their friends and so on. The process stops once there is no user left who is willing to introduce u to any of their friends. Once this happens, an edge is added between u and those nodes u was introduced to. This concludes the round and the next rounds begins with the arrival of a new node executing the same process. See Chapter 4 for the precise model and Figure 1.1 for an illustration.

Ten years ago, Leskovec et al. [START_REF] Leskovec | Graph evolution: Densification and shrinking diameters[END_REF] conjectured, based on their simulations, that the Forest Fire Process exhibits the small-world effect, i. e., the expected distance between any pair of users is constant. Under mild assumptions, we were able to prove this rigorously. Furthermore, we show under certain conditions-in which the presence of edges is more unlikely-that the expected distance is logarithmic in the number of rounds. eases [START_REF] Eubank | Modelling disease outbreaks in realistic urban social networks[END_REF], marketing [START_REF] Subramani | Knowledge-sharing and influence in online social networks via viral marketing[END_REF], romantic relations, and even political ideas. For example, in marketing it is wellknown that targeted advertisement is more effective than generic advertisement [START_REF] Johnson | Targeted advertising and advertising avoidance[END_REF].

Knowing a user's social network is therefore an important part of efficient marketing as suggested by the saying: "Show me your friends and I'll tell you who you are". Moreover, studying social networks can fundamentally change the way we think about social networks and society. A concrete example is the Milgram Experiment [START_REF] Travers | The small world problem[END_REF] in which participants are asked to send a letter to an unknown person by only forwarding it to people they know on a first-name basis. The recipients then forward the letters by following the same rule, and so on, until the letters reach their intended destination. The experiment had the surprising outcome that the average length of the chain, of the letters that arrived, was 6.2. This experiment branded the terms "small-world", and "six degrees of separation". Nevertheless, the experiment left the question open as to how social networks are structured; inviting mathematicians to design suitable models-such as the Forest Fire Process [START_REF] Leskovec | Graph evolution: Densification and shrinking diameters[END_REF]. The motivation behind studying the Forest Fire Process in particular are twofold. First, the Forest Fire Process models that we often meet our friends through mutual friends. Second, the Forest Fire Process encapsulates-as simulations suggest [START_REF] Leskovec | Graph evolution: Densification and shrinking diameters[END_REF]-three important properties observed in many social networks: (i) the small-world effect, (ii) the "densification" of edges, i. e., the number of edges in the network is super constant in the number of users, and (iii) a power-law distribution of the out-edges. See Chapter 4 for an overview of other social network models.

The key to the analysis is a potential function, which allows us to show that whenever the distance of a user in the network to the initial set of users is large, then the potential will decrease in expectation over the course of the arrival of the next two users. In particular, the potential can be modeled as a general random walk on the natural numbers with a drift towards zero (apart from finitely many states) and tail bounds on the absolute change per round. To bound the potential change, we couple the original process with a Galton-Watson Tree.

Balls-into-Bins with Deletions. The Balls-into-Bins with Deletions process models the load distribution in a system with n nodes (bins) representing n queues. Tasks (balls) arrive over time and move to bins according to different strategies. We consider two strategies:

Greedy[1] and Greedy [2]. The process works as follows. At each round a batch of 0 to n balls arrives: Each of n potential balls spawns w.p. λ < 1 and each of the spawned balls chooses (i) uniformly at random from n bins (Greedy[1]) or (ii) greedily from two bins sampled uniformly at random from n bins (Greedy[2]). At the end of each round, all non-empty bins delete one ball each. See Figure 1.2 for an illustration.

We give bounds on the load of the bins after an arbitrary number of steps (possibly super-exponential in the number of bins n) and show an exponential difference in the load of processes Greedy[1] and Greedy [2]. We show that the corresponding Markov Chains are positive recurrent and that there is an exponential difference in the load, similarly to the classical two-choices Balls-into-Bins process [START_REF] Azar | Balanced Allocations[END_REF].

Step t (beginning)

Step t (after assignment) In this example we have n = 5 and 4 balls arrive. Balls 1, 2, and 3 choose the same bin with a load of 2 and a bin with larger node and hence all move the same bin resulting in that bin having the highest load. Moreover, Ball 4 chooses two bins with equal load and chooses one of these uniformly at random. At the end of the round all non-empty bins delete one ball (marked gray).

The Balls-into-Bins with Deletions process can be used to model customers accessing a web-service are assigned to servers, collisions protocols used for contention resolution message routing, as well as (iii) real-life queues at airports, supermarkets, etc. [CMM+98, [START_REF] Eschenfeldt | Supermarket Queueing System in the Heavy Traffic Regime. Short Queue Dynamics[END_REF][START_REF] Mitzenmacher | The Power of Two Random Choices: A Survey of Techniques and Results[END_REF]. Further applications are hashing, shared memory emulations on distributed memory machines, load balancing with limited information, and low-congestion circuit routing [START_REF] Mitzenmacher | The Power of Two Random Choices: A Survey of Techniques and Results[END_REF]. Knowing the length of the queues, and therefore the time a user or task spends in the queue is helpful in the design of systems in which waiting times are a major concern.

The analysis follows-on a superficial level-the approach (which we also used in the Forest Fire Process) of reducing the underlying problem to a potential that performs a biased random walk with a drift towards zero (apart from finitely many states) and tail bounds on the absolute change per round. This time, however, it is not necessary to consider two consecutive time steps at once. On the other side, we are faced with a different challenge:

It seems hard to "condense" all relevant properties of the load distribution into one single potential. For that reason, our analysis builds on the careful analysis of the interplay of three different potentials. Each of these potentials characterizes features of the load distribution, by mapping the features of the load distribution to a natural number.

In the same spirit as in the Forest Fire Process, the changes in the potential can be treated as a general random walk with a drift. However, this time the potentials are in expectation no longer constant but a function of the time step t = ω(poly(n)). In order to get strong bounds at time t, we first use union bounds in an adaptive way to get rough bounds on the potentials at all time steps up to time step t. Using combinatorial arguments, we show that there must have been a state with a favorable load distribution at some round t-poly (n). From there on we can apply a more fine-grained potential analysis to characterize the load distribution at time t.

See Chapter 3 for further details about the high-level analysis and common aspects of both dynamic processes.

Part II: Consensus Processes

The second part of the thesis focuses on the study of consensus processes through the lens of Markov Chains. Consensus processes are processes in which each node of a graph starts with an opinion 2 and all nodes execute simple protocols with the goal of attaining consensus quickly, i. e., to agree on one opinion. We distinguish between consensus dynamics, which are very simple and require little memory and communication, and consensus protocols, which are faster in reaching consensus at the cost of being more complicated often requiring more memory and communication. See Chapter 7 for more details. It is well-known (e. g., [START_REF] Angluin | Local and Global Properties in Networks of Processors (Extended Abstract)[END_REF][START_REF] Lynch | A Hundred Impossibility Proofs for Distributed Computing[END_REF]) that reaching Voter is arguably the simplest randomized dynamic possible and 2-Choices and 3-Majority are almost equally simple. However, 2-Choices and 3-Majority are at the same time efficient self-stabilizing solutions for Byzantine agreement [START_REF] Pease | Reaching agreement in the presence of faults[END_REF][START_REF] Rabin | Randomized byzantine generals[END_REF]: achieving consensus in the presence of an adversary that can disrupt a bounded set of nodes each round [START_REF] Becchetti | Trevisan: Simple Dynamics for Plurality Consensus[END_REF][START_REF] Becchetti | Stabilizing Consensus with Many Opinions[END_REF][START_REF] Cooper | The Power of Two Choices in Distributed Voting[END_REF][START_REF] Elsässer | Efficient k-Party Voting with Two Choices[END_REF]. We generalize each of these protocols in different ways and settle the question of the fastest protocol among these three on the complete graph.

Applications of consensus dynamics and protocols are manifold: The nodes may represent machines in a network: Consider Bitcoin, where the need of reaching consensus in a distributed fashion arises frequently whenever two or more parties "mine" a new block (bundling transactions) simultaneously; otherwise a transaction could be accounted for multiple times. In fact, distributed consensus is one of the most fundamental problem in distributed computing with many applications [DGM+11, Pel02, PVV09, CIG+15, BMPS04].

Arguably the most prominent special case of consensus protocols is leader election, the 2 We assume that there is no ordering of the opinions.

heart of distributed computing. In leader election, all nodes start with distinct opinions and need to agree quickly on one opinion. Applications typically demand both very simple and space-efficient protocols.

Consensus processes can also be used to develop a better understanding of how opinions and believes spread in social networks, as well as for other application in social networks [START_REF] Mossel | Reaching Consensus on Social Networks[END_REF][START_REF] Mossel | Majority dynamics and aggregation of information in social networks[END_REF]. Such insights could be used to develop strategies such as determining how and where to distribute free samples of a new product in order to raise market shares.

Moreover, developing simple consensus dynamics helps to understand how communication in nature works, e. g., among ants and birds works, as well as many other aspects of nature [BDDS10, CER14, FPM+02, CDS+13, CC12]. In the following we give an overview of the results and techniques we show for each of the consensus processes.

The Voter model (see Chapter 8) works as follows. Initially, every node has a distinct opinion and in each synchronous round each node samples a neighbor uniformly at random and adopts its opinion. It is well-known that the consensus time follows the same distribution as the coalescence time, which is defined as follows. The coalescence time is expected time it takes for n independent random walks starting from different nodes to absorb one another. We thus study the more amenable consensus time and express it in terms of two fundamental quantities related to random walks: The mixing time and the meeting time.

As a side product, we obtain tail bounds on the meeting time of random walks prior to the meeting time. 3We then study 3-Majority (see Chapter 9), where every node samples three other nodes at random and changes its opinion to the majority among the samples, with ties broken arbitrarily in case all sampled colors are distinct. The analysis of 3-Majority rests on the shoulders of Voter: We show via Strassen's Theorem (Theorem 9.7) the existence of a coupling between the processes which allows to bound the progress of 3-Majority with the progress of Voter (Theorem 9.4). The latter allows, in the setting of many distinct opinions, a much better handle and notion of progress-by making use of the aforementioned "duality" with coalescing random walks. We also extend the well-known duality between the Voter and coalescing random walks to obtain bounds on the expected time required to reduce the number of opinions from n to k. This reduction to coalescing random walks (via theVoter) together with a potential approach allows us to derive the first unconditional bounds for the 3-Majority. Subsequently, we consider the 2-Choices protocol (see Chapter 10) in which a node only changes its opinion if both of its samples share the same opinion. We obtain the first results for the case of more than 2 different opinions). We complete the picture by showing that 2-Choices is slower than 3-Majority but at the same time gives better guarantees for plurality consensus, meaning that the initially most frequent option prevails.

In the wake of our study of simple consensus dynamics we move to consensus dynamics which achieve better guarantees on the plurality consensus at the price of being slightly more sophisticated. First, we harness the guarantees of 2-Choices to develop a considerably faster algorithm (Chapter 11) to reach plurality consensus on the complete graph. Finally, in Chapter 12 we appeal to load balancing to design protocols achieving plurality consensus on general graphs.

Conclusion

In essence, we relate a variety of seeming unrelated models to random walks and use powerful machinery developed in the past decades (such as Strassen's Theorem, Hajek's Theorem, Galton-Watson Trees, Póly urns, Doob-Martingales, etc.) to shed light on fundamental problems in distributed computing.

Introduction française «Une grande partie de la vie, me semble-t-il, est déterminée par le pur hasard» -Jean-Claude Pirotte

Cette thèse est consacrée à l'étude des processus stochastiques décentralisés qui sont omniprésents dans notre vie quotidienne. Ils décrivent par exemple le mouvement des atomes, les marchés boursiers, les personnes que nous rencontrons, les élections, etc. 4 , et nous étudions dans cette thèse une large gamme d'exemples modélisés par des processus stochastiques : les réseaux sociaux, les processus de balles dans les bacs et les dynamiques et protocoles fondamentaux de consensus.

La particularité du travail réalisé dans cette thèse est de ramener systématiquement l'étude de ces processus à l'étude de marches aléatoires sur des ensembles de basse dimension (par exemple les entiers naturels), à l'aide d'une large palette d'outils de probabilités.

Première partie : processus dynamiques en temps infini. La première partie de cette thèse est consacrée aux processus dynamiques en temps infini. Nous commencerons par nous pencher sur le processus dynamiques de nouveaux utilisateurs sur les réseaux sociaux.

En effet, l'étude des réseaux sociaux est primordiale dans la compréhension des sociétés dans lesquelles nous vivons -réelles et virtuelles. La communication et l'interaction avec autrui ont évolué : de visu (réel) vers les réseaux sociaux en ligne (virtuel). Les réseaux sociaux qui nous entourent vont bien au-delà des liens amicaux et incluent les liens commerciaux, les collaborations académiques, les relations amoureuses, voire l'échange d'idées politiques et la propagation des maladies. L'étude de ces réseaux sociaux est capitale dans divers domaines, y compris l'économie, les sciences sociales, le marketing, la propagation des maladies, la politique [Pik13, Jac+08, SR03, EGA+04, Jac05], etc. Afin de comprendre les réseaux sociaux il est nécessaire de trouver un modèle qui présente les mêmes caractéristiques que ces dernieres. Dans cet esprit, de nombreux modèles ont été proposés (voir Chapter 4 pour une vue d'ensemble). Un modèle bien connu est le Forest Fire Process introduit par [START_REF] Leskovec | Graph evolution: Densification and shrinking diameters[END_REF]. Le modèle Forest Fire est un modèle pour les réseaux sociaux dans lequel arrivent perpétuellement de nouveaux utilisateurs : à chaque instant, un utilisateur arrive et se connecte à un autre utilisateur choisi uniformément au hasard parmi ceux déjà présents. Le nouvel utilisateur exécute alors un processus récursif simple pour déterminer ses connexions, c'est-à-dire son voisinage (voir Chapter 4 pour le modèle précis et Figure 1.4 pour une illustration). En pratique, il est essentiel de pouvoir prédire la longueur des files d'attente et, par conséquent, le temps que l'utilisateur ou la tâche passe dans la file afin de concevoir efficacement les infrastructures.

Dans Chapter 5, nous montrons en quoi ce problème peut être modélisé par un problème des balles arrivant de façon aléatoire dans différents bacs. Nous étudions la version des boules dans les bacs qui fonctionne comme suit. À chaque étape du temps, un lot de 0 à n boules arrive : chaque n boule potentielle apparaît avec probabilité λ < 1 et chacune des boules apparues choisit (I) uniformément au hasard à partir de n bins (Greedy[1]) ou (ii) de manière glouton parmi deux bacs échantillonnés uniformément au hasard à partir de n bins (Greedy[2]). À la fin de chaque tour, chacun des bacs non vide supprime une boule (voir Figure 1.5 pour une illustration).

Step t (beginning)

Step t (after assignment) Nous attribuons des limites à la charge des bacs après un nombre arbitraire d'étapes (éventuellement super exponentielles dans le nombre de bacs n) et montrons une différence exponentielle dans la charge de processus Greedy[1] et Greedy [2]. Nous montrons que les chaînes de Markov correspondantes sont récurrentes positives et qu'il y a une différence exponentielle dans la charge, similaire à celle des processus classiques à deux choix de boules dans les bacs [START_REF] Azar | Balanced Allocations[END_REF]. Les applications sont Hashing, les émulations de mémoire partagée sur les machines à mémoire distribuée, l'équilibrage de charge avec des informations limitées et le routage des circuits à faible congestion [START_REF] Mitzenmacher | The Power of Two Random Choices: A Survey of Techniques and Results[END_REF].

Notre analyse s'appuie sur une analyse minutieuse de l'interaction entre différents potentiels que nous introduisons pour notre analyse. Chacun de ces potentiels représente les caractéristiques de la répartition de la charge, et l'évolution de ces potentiels s'apparente à une marche aléatoire générale avec une dérive. Afin d'obtenir des comportement asymptotiques, nous utilisons d'abord les limites de l'union d'une manière adaptative, puis à l'aide d'arguments combinatoires en d de l'approche générale (voir Forest Fire Process).

Deuxième partie : Dynamique de consensus et protocoles de consensus. La deuxième partie de la thèse est consacrée à l'étude de la dynamique et des protocoles de consensus à travers l'objectif des chaînes de Markov : les dynamiques de consensus sont des processus dans lesquels chaque noeud d'un graphe commence par une opinion5 et tous les noeuds exécutent des protocoles simples dans le but d'atteindre un consensus rapidement, c'est-à-dire de s'entendre sur un seul avis. Les noeuds peuvent être utilisés dans un réseau comme par exemple dans le cas de Bitcoin ou chaque fois que deux ou plusieurs parties « découvrent » un nouveau bloc (regroupement de transactions) simultanément. Dans ce cas, tous les joueurs doivent se mettre d'accord sur l'un de ces blocs de manière distribuée.

En fait, le consensus distribué est l'un des problèmes les plus fondamentaux en informatique distribuée avec de nombreuses applications dans l'informatique distribuée [DGM+11, Pel02, PVV09, CIG+15, BMPS04]. Le cas le plus important des protocoles de consensus est, sans doute, l'élection de chef, le coeur de l'informatique distribuée. Dans l'élection de chef, tous les noeuds commencent par des opinions distinctes et doivent s'entendre rapidement sur un seul avis. Toutes ces applications exigent généralement des protocoles très simples et économes en espace.

Les protocoles de consensus peuvent également être utilisés pour développer une meilleure compréhension de la façon dont les opinions et les croyances se diffusent dans les réseaux sociaux et autres applications des réseaux sociaux [START_REF] Mossel | Reaching Consensus on Social Networks[END_REF][START_REF] Mossel | Majority dynamics and aggregation of information in social networks[END_REF]. En outre, le développement d'une simple dynamique de consensus aide à comprendre comment la communication dans la nature, par exemple parmi les fourmis et les oiseaux, fonctionne ainsi que d'innombrables autres aspects de la nature, [BDDS10, CER14, FPM+02, CDS+13, CC12].

Il est bien connu (e. g., [START_REF] Angluin | Local and Global Properties in Networks of Processors (Extended Abstract)[END_REF][START_REF] Lynch | A Hundred Impossibility Proofs for Distributed Computing[END_REF]) que l'obtention d'un consensus de manière déterministe est impossible dans de nombreux contextes d'intérêts.

Une façon naturelle de défier les résultats impossibles est l'utilisation de protocoles randomisés -le noyau de cette thèse. La dynamique aléatoire prédominante dans cette zone est Voter, 2-Choices, et 3-Majority que nous présentons par la suite (voir Figure 1 pour atteindre un consensus de la pluralité sur le graph complet. Enfin, dans Chapter 12, nous faisons appel au domaine de la répartition de charge pour concevoir des protocoles obtenant un consensus de pluralité sur des graphes généraux.

Organization and Publications

In the first part of this thesis (Part I) we analyze two infinite dynamic processes: The Forest Fire Process (Chapter 4) and a Balls-into-Bins version (Chapter 5). See Chapter 3 for an introduction and overview of our results. The second half of the thesis (Part II) considers consensus dynamics and protocols and an introduction and overview can be found in Chapter 7. In this part we study Voter (Chapter 8), 2-Choices (Chapter 10), 3-Majority (Chapter 9), and other protocols (Chapter 11 and Chapter 12). Chapter 2 introduces the bulk of the notation used in this thesis and Appendice A provides the probabilistic preliminaries of this thesis.

Publications comprised in this thesis

• Chapter 4 considers Forest Fire Process and is based on V. Kanade

General Notation

We denote by N = {0, 1, . . . , }, R the set of all natural and real numbers, respectively.

Let [n] := {1, 2, . . . , n}. We write log for the logarithm with base 2 and ln for the natural logarithm. Let d ∈ N and x, y ∈ R d . We define

x 1 := i∈[d] x i and x 2 := i∈[d] x 2 i 1/2 .
Moreover, let x ↓ denote a permutation of x such that all components are sorted nonincreasingly. We write x y and say x majorizes y for

x 1 = y 1 if, for all l ∈ [d], we have i∈[l] x ↓ i ≥ i∈[l] y ↓ i .
In general, we will use bold-faced letters to denote vectors and capital letters to denote random variables and sets.

Graphs

Random Variables. For random variables X and Y we write

X ≤ st Y if X is stochas- tically dominated by Y , i. e., for all k ∈ R it holds P[ X ≥ k ] ≤ P[ Y ≥ k ].
We denote by F t the filtration (i. e.,, intuitively speaking, the history of all random decisions) up to time step t. Throughout this thesis, the expression w.h.p. (with high probability) means with probability at least 1 -n -Ω(1) and the expression w.c.p. (with constant probability) means with probability > 0.

Probability Distributions. We denote by Bernoulli(p) the Bernoulli distribution with success probability p, by Bin(n, p) the Binomial distribution, with n independent trials, each having success probability p, by Geom(p) the geometric distribution with success probability p, i. e., for X ∼ Geom(p) we have P[ X = i ] = (1 -p) i p for i = 0, 1, . . . ., by Poisson(λ) the Poisson distribution with mean λ, and by Uniform(b) the uniform distribution on the elements {0, 1, 2, . . . , b}. For a probability vector Θ ∈ [0, 1] d , we use Mult(m, Θ) to denote the multinomial distribution for m trials and d categories (the i-th category having probability Θ i ).

Markov Chains. Unless stated otherwise, all random walks are assumed to be discretetime (indexed by natural numbers) and lazy, i. e., if P denotes the n × n transition matrix of the random walk,

p u,u = 1 2 , p u,v = 1 2 deg(u)
for any edge (u, v) ∈ E and p u,v = 0 otherwise. We define p t u,v to be the probability that a random walk starting at u ∈ V is at node v ∈ V at time t ∈ N. Furthermore, let p t u,• be the probability distribution of the random walk after t time steps starting at u. By π we denote the stationary distribution, which satisfies,

for undirected graphs, π(u) = deg(u) 2m for all u ∈ V . Let d(t) := max u p t u,• -π TV and d(t) := max u,v p t u,• -p t v,
• TV , where • TV denotes the total variation distance. Following Aldous and Fill [AF02], we define the mixing time to be t mix (ε) := min{t ≥ 0 : d(t) ≤ ε} and for convenience we will write

t mix := t mix (1/e).
We define separation from stationarity at a given time step as follows: s(t) := min{ε :

p t u,v ≥ (1 -ε)π(v) for all u, v ∈ V }.
The definition ensures that s(•) is submultiplicative, so in particular, non-increasing [START_REF] Aldous | Reversible Markov Chains and Random Walks on Graphs[END_REF], and we can define the separation threshold time

t sep := min{t ≥ 0 : s(t) ≤ e -1 }
and, by [AF02, Lemma 4.11], t sep ≤ 4t mix . We write T hit (u, v) to denote the first time step t ≥ 0 at which a random walk starting at u hits v. In particular,

T hit (u, u) = 0. The hitting time t hit (u, v) = E[ T hit (u, v) ] of any pair of nodes u, v ∈ V is the expected time required for a random walk starting at u to hit v. Thus, t hit (u, v) is the expectation of T hit (u, v).
The hitting time of a graph

t hit := max u,v t hit (u, v)
is the maximum over all such pairs. For A ⊆ V , we use t hit (u, A), to denote the expected time required for a random walk starting to u to hit some node in the set A. Furthermore, we define t hit (π, u) :=

v∈V t hit (v, u) • π(v). Furthermore, we define t avg-hit := u,v∈V π(u) • π(v) • t hit (u, v).
For two random walks (X t ) t≥0 , (Y t ) t≥0 starting at u and v let

t meet (u, v) := min{t ≥ 0 : X t = Y t }
denote the expected meeting time, i. e., the first time step at which both walks are on the same node. We write t π meet to denote the expected meeting time of two random walks starting at two independent samples from the stationary distribution. Finally, let t meet := max u,v t meet (u, v) denoted the worst-case expected meeting time.

Consensus Processes. The processes are defined in Chapter 7 and we restrict ourselves to only introducing the important notation. In these processes we have n anonymous nodes connected by edges of a graph. Initially, each node supports one opinion from the set [k] := { 1, . . . , k }. We refer to these colors as C 1 , C 2 , . . . , C k . The system state after any round by an n-dimensional

integral vector c = (c i ) i∈[n] ∈ N n 0 with i∈[n] c i = n.
Here, the i-th component c i ∈ N 0 corresponds to the number of nodes supporting opinion i.

If k < n, then c i = 0 for all i ∈ { k + 1, k + 2, . . . , n }. Miscellaneous. A function f : R d → R is Schur-convex if x y ⇒ f (x) ≥ f (y).

Part I

Probabilistic Analysis of Distributed Dynamic Processes

Chapter 3

Contributions Dynamic Processes

The first part of the thesis concerns the analysis of infinite dynamic processes by means of carefully crafted potentials and the analysis of the potential through (general) random walks. We consider two processes: Forest Fire Process and Balls-into-Bins with Deletions.

Definition Forest Fire Process. The Forest Fire Process is a model for generating random graphs representing the evolution of social networks. The nodes in the generated graph represent the users and the each edges represents friendship between the adjacent nodes.

At every time step a user arrives and connects to another user chosen uniformly at random among those already present. The new user then executes a simple recursive process to determine their connections i. e., their neighborhood. Formally, the Forest Fire Process is defined iteratively, starting from a seed graph G 0 . Let G t-1 = (V t-1 , E t-1 ) denote the graph at the end of round t -1. In round t, a new node u t arrives, and chooses a node amb(u t ) ∈ V t-1 uniformly at random, where we call the node amb(u t ) the ambassador of the new node u t . After selecting the ambassador, we burn the ambassador, meaning we add the edge (u t , amb(u t )) to the graph. The graph generation process then continues as follows. First choose a random subset of the edges of G t-1 as active edges: every edge (u, v) of G t-1 is active independently with probability min{1, α deg + (u) }, where α is a parameter of the model and deg

+ (u) is u's out-degree.
Secondly, add an edge to all vertices of G t-1 , reachable from amb(u t ) by a path consisting of active edges. This construction of G t can be obtained by executing Algorithm 1 and Algorithm 2.

We show, under mild assumptions, that if the parameter α is a large enough constant, then this models exhibits indeed the small world effect, i. e., the expected distance between two users is constant. Conversely, if α is below some constant, then the expected distance is of order Ω(log n).

Algorithm 1: Forest Fire Process (G0) for t = 1, 2, . . . do upon arrival of node u t at time t: At each time step, we generate up to n balls, each with probability λ < 0. Each of the spawned balls (i) chooses the target bin uniformly at random (Greedy[1]) or (ii) chooses the target bin greedily among two bins chosen uniformly at random (Greedy[2]).

amb(u t ) ← a node chosen u.a.r. from V t-1 S ← Burn(G t-1 , amb(u t )) G t ← (V t-1 ∪ {u t }, E t-1 ∪ {(u t , w) : w ∈ S}) Algorithm 2: Burn(G = (V, E), v) // Outputs a subset of V reachable from v H ← ∅ for all (w, x) ∈ E do with probability min 1, α deg + G t (w) H ← H ∪ {(w, x)} return {x ∈ V :
To model the load of real system more realistically, we extend the model by adding the following ingredient. At the end of each round, all non-empty bins delete one ball each modeling a finished tasks. In the following we state an algorithm summarizing the above.

The algorithm is executed by each of the n generators. We show that both protocols are positive recurrent and that there is an exponential load difference between both protocols.

Analysis. At first glance, the Forest Fire Process and the Balls-into-Bins with Deletions process appear to be completely unrelated. The fabric connecting these processes is our analysis. We design custom-tailored potential functions for the Forest Fire Process, Stating the precise definitions of the potentials would overstrain this overview (and is therefore left to Chapter 4 and Chapter 5) but their intuition is simple. We show that, whenever any of our potentials surpasses a certain threshold, then it decreases in expectationregardless of the current state. Furthermore, the distribution of the potential change has an exponential tail-bound, i. e., the probability to increase the potential by k is 2 -Ω(k) . This allows us to model the potential via a general random walk on the real numbers. See It is worth mentioning that the potentials do not decrease (in expectation) in every case.

Greedy

In fact, when the potential is close to zero it increases in expectation.

Since both processes are infinite, it does happen-albeit very rarely-that the quantities of interest (e. g., the maximum load) attain a value which is a function of the time t. Our potential approach shows that this is a "rare" state and whenever the system is in such a state, it quickly recovers.

The potentials are designed in such a way that the following holds. Whenever the potentials are large at a given time step t, we simply assume that system is in the worstcase state and show that the potentials decrease in expectation. This technique proves to be very useful for the problems we study since both problems are of infinite nature.

The analysis of such potentials dates back to Hajek [START_REF] Hajek | Hitting-Time and Occupation-Time Bounds Implied by Drift Analysis with Applications[END_REF] (Theorem A.11) and less general versions have successfully applied to various areas and notably to evolutionary algorithms and drift theory [START_REF] Berenbrink | The Natural Work-Stealing Algorithm is Stable[END_REF][START_REF] Doerr | Adaptive Drift Analysis[END_REF][START_REF] Pemantle | Moment conditions for a sequence with negative drift to be uniformly bounded in Lr[END_REF].

In order to harness the aforementioned general approach for the analysis of both problems, a few problem specific enhancement are required: For the Forest Fire Process we were unable to find a potential that decrease in expectation in a single time step, regardless of the current state. Instead, we consider the potential drop over two consecutive time steps: The node arriving in the first time step t + 1 is likely to have a very "favorable" neighborhood, such that the node arriving at time step t+2 causes the potential to decrease. Our potential might be of general interest and refer the reader to Chapter 4 for an in-depth discussion.

As for the Balls-into-Bins process, we use three different potentials: Potential Φ(t) measuring the load difference to average load over time, potential Ψ(t) counting the total load in the system, and potential Γ(t) which interweaves both potentials and allows us use to prove positive recurrence. Because of the strong dependencies between the potential, we start by focusing solely on Φ(t) and we obtain bounds for arbitrary time steps (possibly superexponential in the number of bins). This allows to apply the general approach described above and gives a weak bound on the maximum load at arbitrary time steps.

To derive a stronger bound we combine the bounds on Φ with combinatorial arguments.

To do so use union bounds in an adaptive way to get rough bounds on the potentials at all time steps up to time step t. We then show that that there must have been a step t -poly(n) where Ψ was very small. Together with our rough bounds on Φ, we are able to establish strong bounds on the maximum load at time t. Finally, we are able to analyze the third potential Γ by means of the general approach: We reduce it to a general random walk with drift towards zero whenever it's large and apply Hajek's Theorem. Using Γ we show positive recurrence of the underlying Markov chain.

Potential of the approach. It seems that the infinite nature (of the processes we considered) renders many standard approaches futile: for example, the approach of relying solely on invariants which could fail over the course of time. On the other hand, the concepts developed for Markov chains such as positive recurrence, and drift theory capture the notion of "recovery", making them very useful in the study of infinite processes. Using these techniques reduces the analysis to finding suitable potentials encapsulating the key-properties of the underlying problem. We are optimistic that this general potential approach, as well as our ideas used to craft and analyze our potentials, carry over to other dynamic processes.

Chapter 4

Social Networks: The Forest Fire Model [START_REF] Kanade | Distance in the Forest Fire Model How far are you from Eve?[END_REF] Ten years ago, Leskovec, Kleinberg and Faloutsos introduced the Forest Fire model, a generative model to understand the dynamics of social networks over a long period [START_REF] Leskovec | Graph evolution: Densification and shrinking diameters[END_REF].

They examined real-world networks such as the ArXiv Citation Graph, the Patents Citation Graph, the Autonomous Systems Graph, Affiliation Graphs, the Email Network, the IMDB Actors-to-Movies Network, and a Product Recommendation Network. They observed that these social networks become denser over time. They also made the surprising observation that the effective diameter of the networks "shrinks" over time, instead of growing, as was previously thought. They suggested the Forest Fire model as an attempt to explain densification, shrinking diameter, and heavy-tailed distributions of vertex indegrees and outdegrees.

In this model, the evolution initially starts with a fixed seed graph. Time is discrete and at each time t a node u t arrives, picks a random node, w, in the current graph as its "ambassador" and connects to it. The ambassador is considered burned and all other nodes are considered unburnt. Node u t then generates two random numbers x and y and selects

x outgoing edges from w and y in-coming edges to w incident to nodes that have not yet been burned. If not enough outgoing or incoming edges are available, u t selects as many as it can. Let w 1 , w 2 , ..., w x+y denote the other endpoints of the edges selected. u t connects to w 1 , w 2 , ..., w x+y , marks them as burned, and then applies the previous step recursively to each w i . See

Formal Definition

Formally, the Forest Fire Process is defined iteratively, starting from a seed graph G 0 . Let

G t-1 = (V t-1 , E t-1
) denote the graph at the end of round t -1. In round t, a new node u t arrives, and chooses a node amb(u t ) ∈ V t-1 uniformly at random, where we call the node amb(u t ) the ambassador of the new node u t . After selecting the ambassador, we burn the ambassador, i. e., we add the edge (u t , amb(u t )) to the graph. This then propagates as follows.

First choose a random subset of the edges of G t-1 as active edges: every edge (u, v) of G t-1 is active independently with probability min{1, α deg + (u) }, where α is a parameter of the model. Second, burn all vertices of G t-1 , reachable from amb(u t ) by following directed active edges. Third, add an edge from u t to every burnt vertex. This construction of G t can be obtained by executing Algorithms 4 and 5. Although, it is more natural to view burning as a branching process in which we consider the burnt nodes "layer by layer", we describe the process as a percolation process 1 in order to avoid the need to define a specific order for the burning process: In a branching process, a node w could appear on several levels however we allow w to only be burnt once 2 and thus the order in which we burn nodes affects the random subtree of burnt nodes.

1 A percolation process is a process in which every edge of the graph is present independently with a fixed probability.

2 In Section 4.7 we discuss a model where we allow a node to burn several times for t = 1, 2, . . . do upon arrival of node u t at time t:

amb(u t ) ← a node chosen u.a.r. from V t-1 S ← Burn(G t-1 , amb(u t )) G t ← (V t-1 ∪ {u t }, E t-1 ∪ {(u t , w) : w ∈ S}) Algorithm 5: Burn(G = (V, E), v) // Outputs a subset of V reachable from v H ← ∅ for all (w, x) ∈ E do with probability min 1, α deg + G t (w)
H ← H ∪ {(w, x)} return {x ∈ V : there exists a directed path from v to x in H}

Results

We now state our two main results for the Forest Fire model. The parameters α and the input graph G 0 are fixed and we study the asymptotic properties of the graph G t . We have not optimised the constants in the theorem statements and expect them to be far from being tight. Theorem 4.1. Let α ≥ 100 and let G 0 be a directed cycle such that |G 0 | ≥ α 20 , the Forest Fire Process with parameters α and G 0 has the property of non-increasing distance to G 0 , i. e., for every t,

E[ dist Gt (u, G 0 ) ] = O(1),
where the expectation is over a node u, which is chosen uniformly at random in G t , and

dist(u, G 0 ) is the directed distance. 3
Remark 4.2. It is not critical that G 0 is a cycle. The main requirement is that conditioned on the Burn Process reaching G 0 , a large enough constant number of vertices of G 0 will be burnt. For example, G 0 being an expander, clique, or a strongly connected graph with large girth suffices. Simulations seem to indicate that G 0 being a single node also result in a similar behaviour. Theorem 4.3. Let α ≤ 1/(4e) and let G 0 be an arbitrary graph, the Forest Fire Process with parameters α and G 0 is such that

E[ dist Gt (u, G 0 ) ] = Ω(log t),
using the same notation as above.

Approach and Technical Contributions

The main idea is as follows. We first reduce the process to the line process in which the node at time t connects to the node which arrived at time t -1 (see Section 4.4). In this line process we define a potential φ(v t ) which measures, intuitively speaking, the "typical path length" of node v t arriving at time step t to the initial graph L 0 . We defer the formal definition to Section 4.5. We would like to argue that no matter what happens up to time t, dist(v t+1 , L 0 ) is less than dist(v t , L 0 ) in expectation whenever dist(v t , L 0 ) is large enough. This does not seem to be possible when using distance directly; we can construct graphs where this is not true. However, these graphs are unlikely to arise under the Line Fire Process. Analysing φ instead gets around this issue. In fact, assuming φ(v 2t ) > 2, we show that φ(v 2t+2 ) -φ(v 2t ) has negative expectation-irrespective of the history up to time 2t. The potential is designed such that v 2t+1 sets up a favourable situation such that v 2t+2 is able to decrease the potential w.r.t. to the value φ((v 2t ). The crucial part is that φ(v t ) is designed such that it dominates dist(v t+1 , L 0 ) and thus assuming we can bound E[ φ(v) ] = 0, we get, by triangle inequality, that for u, v chosen uniformly at random

E[ dist(u, v) ] ≤ E[ dist(u, L 0 ) ] + E[ dist(v, L 0 ) ] ≤ E[ φ(u) ] + E[ φ(v) ] = O(1).
Note that even though the edges added are directed, we treat the graph as undirected when we consider the distance of nodes.

Related work

There is a extensive variety of models for generating graphs of social networks, each reproducing a subset of properties observed in real-world social networks. The first major line of research considers static graphs, where the number of nodes does not change over the course of time: For example, in small-world like models, there is a fixed underlying graph which is augmented by additional links between the vertices. Kleinberg proposed a particular random augmentation of links on the grid and proved that this gives rise to a decentralised greedy algorithms to find short paths among nodes [START_REF] Kleinberg | Small-World Phenomena and the Dynamics of Information[END_REF]. In a more recent paper, Chaintreau et al. proposed a different model, in which similar results are achieved, where the grid is augmented with links generated by random walks on the grid with occasional resets [START_REF] Chaintreau | Networks Become Navigable as Nodes Move and Forget[END_REF].

Other static models focus mainly on reproducing both densification and small diameter simultaneously. One example is the model by Leskovec et al. which uses a matrix-operation, namely, the Kronecker product, to generate self-similar graphs recursively [START_REF] Leskovec | Realistic, Mathematically Tractable Graph Generation and Evolution, Using Kronecker Multiplication[END_REF]. They reproduce a vast number of properties including heavy tails for the in-and out-degree distribution and small diameter. However, the deterministic nature of this model produces unrealistic features. To remedy this drawback, they propose the Stochastic Kronecker Graph (SKG) model which has been very successful and is widely used in simulations. One disadvantage of SKG is that the adjustment of the parameters may have a huge influence on the properties of the resulting graphs. Recently, Seshadhri et al. [START_REF] Seshadhri | The Similarity Between Stochastic Kronecker and Chung-Lu Graph Models[END_REF] showed that in fact the SKG model bears resemblance to a variant of the Chung and Lu model [START_REF] Chung | The Average Distance in a Random Graph with Given Expected Degrees[END_REF] which generalises classical random graph models. Here for any given collection of n weights (w 1 , w 2 , . . . , w n ), the probability of an edge (i, j) is given by w i w j ( k w k ).

Additionally, Pinar et al. [START_REF] Pinar | The Block Two-level Erdos Renyi (BTER) Graph Model[END_REF] introduce the Block Two-Level Erdős Rény (BTER) model, and demonstrate that it captures observable properties of many real-world social networks. Given a degree sequence, the model works in two stages: In the first stage the nodes of roughly the same degree are grouped into clusters and the edges in each cluster are generated by ER (Erdős Rény) graphs for a given using another input parameter. Finally, the "excess" edges of the node i, i. e., the edges not yet used up by edges in the same cluster, are generated by randomly choosing two endpoints proportional to the excess edges of the nodes. Resulting self-loops and multi-edges are discarded.

The second major research line considers graph evolving over time where at each time step new vertices and edges are added to the evolving graph. Barabási et al. proposed the so called preferential attachment model in which new vertices attach preferentially to vertices with high degree, reproducing the power-law distribution over the in-degrees [START_REF] Barabási | Emergence of scaling in random networks[END_REF].

Building on preferential attachment, Cooper and Frieze propose a model in which exhibits a power-law of the degree as well as a shrinking diameter and densification; unfortunately, it involves many parameters [START_REF] Cooper | Frieze: A general model of web graphs[END_REF]. Roughly speaking, the graph at time G t is generated as follows. With some probability a new node is added with one or more edges to G t-1 and with the remaining probability an already existing vertex is selected and exta edges are added to it. Recently, Avin et al. extended the preferential attachment model to incorporate densification [START_REF] Avin | Core Size and Densification in Preferential Attachment Networks[END_REF]: Similarly as in [START_REF] Cooper | Frieze: A general model of web graphs[END_REF] either a new node arrives or new edges are added. In either case, the nodes are chosen according to preferential attachment. Krapivsky and Redner investigated the development of random networks as the attachment probability grows [START_REF] Krapivsky | Organization of growing random networks[END_REF].

The authors of [KKR+99, KRR+00] consider an edge copying evolution in which, on arrival, a new vertex picks an existing node and copies a subset of its neighbours. Another model is the Community Guided Attachment model, in which there is a hierarchical backbone structure that determines the linkage probabilities [START_REF] Leskovec | Graph evolution: Densification and shrinking diameters[END_REF]. Lattanzi and Sivakumar generate random graphs according to an underlying affiliation network: Each node picks a random subset of affiliations and in each affiliation the nodes are connected as a clique (additionally, there is a process of preferential attachment) [START_REF] Lattanzi | Affiliation networks[END_REF]. They show that this model exhibits shrinking diameter, densification, and a heavy-tailed degree distribution. Moreover, they connected the densification of the network to the non-linearity of the core. The recursive search model proposed by Vazquez is quite similar to the Forest-Fire model [START_REF] Vazquez | Disordered networks generated by recursive searches[END_REF]. In the recursive search model, vertices are added to the graph one by one; when a new vertex arrives it first connects to a random vertex and then recursively connects to a subset of its unvisited neighbours. The main difference is that in the Forest Fire model, the average number of neighbours visited out of the current node is constant, where as in the recursive search model this is a constant fraction. Thus, presence of high-degree nodes can make the two models quite different.

In the Random-Surfer Model (RSM), introduced by Blum et al. [START_REF] Blum | A Random-Surfer Web-Graph Model[END_REF], the nodes arrive one by one. Upon arrival, each node performs several random walks from random starting points and connects to the endpoints of the performed walks. Our Random Walk Process (RWP) share resemblance to the RSM. The main difference is that in the RWP a new node connects to all the visited nodes in the random walk (instead of just the endpoint). Chebolu and Melsted [START_REF] Chebolu | PageRank and the Random Surfer Model[END_REF]

Relating graph and line process

Line process

To prove the results for the Forest Fire, we study the related process which we call the Line Fire Process allowing us to reduce the graph process to a line process: When comparing the graph processes (Forest Fire Process), defined in Section 4.1, with the line process (Line Fire Process), the difference is that while in the graph process the first step is to select the ambassador at random; in the line process we skip this step, and force each new node to select the most recently added node as its ambassador, i. e., in the line process the first step is deterministic and follows the line structure.

We state two corresponding technical lemmas for the Line process; in the next subsection, we state coupling lemmas to relate the processes and prove the results of Sec-tion 4.1, using the corresponding related results on the line (whose proofs are deferred to later sections), together with the coupling.

Algorithm 6: Line Fire Process(L0) for t = 1, 2, . . . do upon arrival of node u t at time t:

amb(u t ) ← u t-1 (key difference) S ← Burn(L t-1 , amb(u t )) L t ← (V t-1 ∪ {u t }, E t-1 ∪ {(u t , w) : w ∈ S})
Next, we state the relevant lemmas for the Line Fire Process, that are used to prove the above theorems. The proofs of these lemmas are deferred to later sections.

Lemma 4.4. Let α ≥ 100 and let L 0 be a directed cycle, such that |L 0 | ≥ α 20 . Then, the Line Fire Process has the property that

∃c, ∃γ < 1 s.t. ∀t ∀j P[ dist Lt (v t , L 0 ) > j ] < cγ j .
To see why Theorem 4.1 follows from Lemma 4.4 observe that the subgraph of G t by the vertices on the path from v t to G 0 following edges to ambassadors has the same distribution as L τ for some τ -This concept is formalized in the remainder of Section 4.4, which the reader might wish to skip.

Lemma 4.5.

There exists an α * > 0 such that the following holds. Let α ≤ α * and let L 0 be an arbitrary graph. Then, the Line Fire Process with parameters α and L 0 is such that

E[ dist Lt (v t , L 0 ) ] = Ω(t).

The ambassador graph

Definition 4.6. The ambassador graph A t is the subgraph of G t = (V t , E t ), consisting of edges (u, amb(u)) induced by all nodes u ∈ G 0 . These edges are referred to as ambassador edges.

A t is a forest of directed trees, rooted at vertices of G 0 . First, we observe the following fact.

Fact 4.7. If (u, v) is an edge of G t \ G 0 , then there exists a path from u to v in A t .
To prove our lower bounds, we will use the following bound on the expected distance to the seed graph in the ambassador graph. The following lemma was originally proven in [Dev87, Theorem 10].

Lemma 4.8. Let u be a vertex in the ambassador graph A t chosen uniformly at random. Then

E[ dist At (u, G 0 ) ] = Θ(log t) .
Proof. Let v k denote the node which arrives at time k, where by convention the vertices of G 0 arrive at time 0. First we prove the upper bound

E[ dist At (u, G 0 ) ] ≤ 1 t 1≤k≤t E[ dist At (v k , G 0 ) ] . (4.1) Since dist At (v k , G 0 ) is at most k in the worst-case: E[ dist At (v k , G 0 ) ] ≤ 2 log 2 k + k • P[ dist At (v k , G 0 ) > 2 log k ] . (4.2) Recall that dist At (v k , G 0 ) is the length the path v k , amb(v k ), amb 2 (v k ), . . ., until we reach G 0 . Let X i denote the arrival time of amb i-1 (v k ) = v k-i+1 .
We have, by uniform choice of the ambassador of a node:

X 1 = k and thus E[ X i |X i-1 ] ≤ X i-1 /2, which implies E[ X i ] ≤ E[ X i-1 ]/2. We deduce that E[ X i ] ≤ k/2 i-1 for all i and E X 2 log 2 k+1 ≤ 1 k .
Moreover, we have Since those intervals are disjoint,

X dist A t (v k ,G 0 ) = 0. By Markov's inequality, P[ dist At (v k , G 0 ) > 2 log 2 k ] = P X 2 log 2 k+1 ≥ 1 ≤ E X 2 log 2 k+1 ≤ 1/k. ( 4
E[ dist At (v k , G 0 ) ] ≥ E   i≤log 2 k Y i   . (4.4)
To analyze Y i , let v denote the first vertex on p with arrival time in [0, 2 i ]. By uniform choice of the ambassador and monotonicity

P[ Y i = 1 ] = P v has arrival time > 2 i-1 = 2 i -2 i-1 2 i + |G 0 | ≥ 1 2(1 + |G 0 |) .
Thus,

E[ dist At (v k , G 0 ) ] ≥ E   i≤log 2 k Y i   ≥ log 2 k 2(|G 0 | + 1) . (4.5)

Coupling

The following lemma shows the relation between the Line Fire and Forest Fire Processes.

Definition 4.9. The level of a vertex u is its distance to G 0 in the ambassador graph, defined by: Proof. The ambassador graph is constructed independently of the Burn Process (Algorithm 5), so we can change the order in which the edges of G t are constructed, by generating the ambassador graph in a first phase, and then adding the other edges in a second phase.

(u) =    0 if u ∈ G 0 (amb(u)) +
In the first phase, a node only chooses a random ambassador and connects to it. In the second phase, every node invokes the Burn Process starting with the respective ambassador.

Consider the path in the ambassador graph, going from u t to G 0 , and label its vertices (u t , amb(u t ), amb 2 (u t ), . . . , amb (ut) (u t )) = (w (ut) , . . . , w 0 ), where amb k denotes k iterative applications of amb(•).

Thus w (ut) = u t , w i = amb(w i+1 ) for i < (u t ), and w 0 ∈ G 0 . We claim that the subgraph induced by G 0 ∪ {w 0 , . . . , w (ut) } in the Forest Fire Process, has exactly the same distribution as the graph L τ , produced by the Line Fire process with seed G 0 , for τ = (u t ).

To prove this, we couple the burning decisions of w i and v i . When i = 0, both graphs are G 0 . Assume by induction that the subgraphs induced by G 0 ∪ {w 0 , . . . , w i-1 }, in the Forest Fire Process and in the graph L i-1 in the Line Fire Process, are identically distributed, hence coupled. Then the Burn Process, starting at w i , can clearly also be coupled with the Burn Process of vertex v i , to give the desired result.

Corollary 4.11. For every b ≥ 0, we have

P[ dist Gt (u t , G 0 ) > b | (u t ) = τ ] = P[ dist Lτ (v τ , G 0 ) > b ].

Proofs of the

E[ dist Gt (u t , G 0 ) ] = t b=0 P[ dist Gt (u t , G 0 ) > b ] = t b=0 t τ =1 P[ dist Gt (u t , G 0 ) > b | (u t ) = τ ] • P[ (u t ) = τ ] = t b=0 t τ =1 P[ dist Lτ (v τ , G 0 ) > b ] • P[ (u t ) = τ ] = t τ =1 P[ (u t ) = τ ] t b=0 P[ dist Lτ (v τ , G 0 ) > b ]. (4.6)
From Lemma 4.4 for the Line Fire Process,

P[ dist Lτ (v τ , G 0 ) > b ] ≤ cγ b . Thus t b=0 P[ dist Lτ (v τ , G 0 ) > b ] ≤ t b=0 cγ b ≤ c 1 -γ = O(1).
Since t τ =1 P[ (u t ) = τ ] = 1, the result follows.

Proofs of Theorem 4.3. We follow the proof of Theorem 4.1 until (4.6) and get

E[ dist Gt (u t , G 0 ) ] = t τ =1 P[ (u t ) = τ ] t b=0 P[ dist Lτ (v τ , G 0 ) > b ] = t τ =1 P[ (u t ) = τ ]E[ dist Lτ (v τ , G 0 ) ].
Using Lemma 4.5, we get E[ dist Lτ (v τ , G 0 ) ] ≥ cτ for a suitable constant c. Thus,

E[ dist Gt (u t , G 0 ) ] ≥ t τ =1 P[ (u t ) = τ ]c • τ = c • E[ (u t ) ] = Ω(log t),
by Lemma 4.8. Hence, E[ dist Gt (u t , G 0 ) ] = Ω(log t) for every t ≥ t/2, hence we obtain the desired result.

Analysis of the Line Fire Process

Throughout Sections 4.5.1, 4.5.2, 4.5.3, and 4.5.4 we assume that α ≥ 100, L 0 is a directed cycle with |L 0 | ≥ α 20 . In order to prove Lemma 4.4, we define a function φ, such that for all t, dist Lt (v t ) ≤ φ(v t ), and which is more amenable to analysis. The function φ is defined as follows: Let δ := α 20 , then

φ(v) =                    0 if v ∈ L 0 max w∈N + (v) {φ(w)} + 1 if deg + (v) < δ max      φ(amb(v)) -2, max w∈N + (v) w =amb(v) φ(w) + 1     
otherwise.

We now give an intuitive description and we defer the reader to Section 4.5.1 and Section 4.5.5 for an in-depth discussion about the ingredients of φ.

High-Level Proof Overview

We first give some intuition about the definition of φ. We would like to argue that no matter what happens up to time t, dist(v t+1 , L 0 ) is less than dist(v t , L 0 ) in expectation whenever dist(v t , L 0 ) is large enough. This does not seem to be possible when using distance directly;

we can construct graphs where this is not true. However, these graphs are unlikely to arise under the Line Fire Process. Analysing φ instead gets around this issue. In fact, assuming φ(v 2t ) > 2, we show that φ(v 2t+2 ) -φ(v 2t ) has negative expectation -irrespective of the history up to time 2t. A low value φ(v t ) implies that not only is there one short path from v t to L 0 , but most paths from v t to L 0 are short. However, note that not all paths are short, in particular the path of ambassador edges v t , v t-1 , . . . , v 0 is of linear size.

Furthermore, while it is true for most nodes, it is not necessarily true that all nodes are well connected to the seed graph. Note that the definition of φ makes a special case for the ambassador when the degree is small. For an edge (v, u) if u = amb(v), φ(u) < φ(v). We will call edges (v, amb(u)) ambassador edges.

We start from an arbitrary history (and hence an arbitrary graph) at time 2t

v 2t v 2t-1 v 2t+1 φ(v 2t ) φ(v 2t ) -1 φ(v 2t ) -2 φ(v 2t ) -3 (a) v 2t v 2t-1 v 2t+1 φ(v 2t ) φ(v 2t ) -1 φ(v 2t ) -2 φ(v 2t ) -3 (b) v 2t v 2t-1 v 2t+1 φ(v 2t ) φ(v 2t ) -1 φ(v 2t ) -2 φ(v 2t ) -3 v 2t+2 (c)

Proof of Lemma 4.4

We now formalise the high-level ideas presented in the previous section. We begin by proving that φ indeed dominates the distance.

Fact 4.12. If v arrives at time t, then

dist Lt (v, L 0 ) ≤ φ(v).
Proof. The straightforward proof is by induction on t. For t = 0, v ∈ L 0 and then dist Lt (v, L 0 ) = 0 = φ(0), so the statement holds.

Assume the statement holds for all nodes in L t-1 . Note that in the graph L t all (directed) edges point to vertices that arrived earlier (i. e., for any edge (v t , v τ ), t > τ ). We get, by applying the induction hypothesis,

dist Lt (v, L 0 ) =      0 if v ∈ L 0 min w∈N + (v) {dist Lt (w, L 0 )} + 1 otherwise ind. ≤      0 if v ∈ L 0 min w∈N + (v) {φ(w)} + 1 otherwise ≤            0 if v ∈ L 0 φ(amb(v)) + 1 if deg + (v) = 1 min w∈N + (v)\{amb(v)} {φ(w)} + 1 otherwise ≤                    0 if v ∈ L 0 max w∈N + (v) {φ(w)} + 1 if deg + (v) < δ max      φ(amb(v)) -2, max w∈N + (v) w =amb(v) φ(w) + 1      otherwise. = φ(v).
The proof of Lemma 4.4 relies on Hajek's theorem (Theorem A.11), which we can be found in Section A.2.4.

Let (F t ) t≥0 denote the history of random choices up to time t for the Line Fire process.

We state the lemma that prove that (φ(v 2t+2 ) -φ(v 2t ) | F 2t ) satisfies the conditions of Hajek's theorem. The proofs of this appears in subsequent subsections.

Lemma 4.13 (Majorization and Negative bias). The following holds.

1. Let Z be the random variable taking values over all integers greater than or equal to 4, defined by: P[ Z = i ] = 3 -i/2 for i ≥ 5, and

P[ Z = 4 ] = 1 - 1 9( √ 3+1) . Then (|φ(v 2t+2 ) -φ(v 2t )| | F 2t ) ≤ st Z.
2. Let α and κ be large enough constants and assume |L 0 | ≥ δ ≥ α κ . There exists a constant ε 0 > 0 such that for every t we have,

E[ φ(v 2t+2 ) -φ(v 2t ) | F 2t , φ(v 2t ) > 2 ] ≤ -ε 0
We now prove Lemma 4.4.

Proof of Lemma 4.4. Let Y t = φ(v 2t ). For any constant λ < ln( √ 3)/2, we have,

E e λ Z ≤ e λ 4 • 1 + i≥3 e λ 2i • 3 -i/2 = O(1),
where the last equation follows from the geometric series. Thus E e λ Z is finite for the random variable Z defined in the statement of Lemma 4.13 and hence the sequence (Y t ) t≥0

with respect to the filtration (F 2t ) t≥0 satisfies the two conditions of Theorem A.11 by Lemma 4.13, hence, by using Fact 4.12, Lemma 4.4 follows.

Proof of Lemma 4.13

In our proofs, it is useful to rephrase the process Burn(G, α) defined in Algorithm 5 as a tree process, rather than a percolation process.We define BurnBFS(G, v) in Algorithm 7.

We assume that vertices have a natural order in the graph, for examples for graphs evolving in time, the vertices are ordered according to their time of arrival. Thus, when indexing a set we assume that the vertices are indexed in this order.

First, we note that if the burning decisions made for activation of edges (w, x) in Algorithm 7 are coupled with those made in Algorithm 5, the set of vertices returned by the two processes is exactly the same. Thus, this is indeed another view of the Burn Process.

The Burn Process BurnBFS produces a tree T with activated vertices ∪ j≥0 M j , and edges (w, x) for which the if condition in Algorithm 7 was satisfied (see Figure 4.4). (We remark M 0 := {v} for i = 1, 2, . . . do M i := ∅ for all w ∈ M i-1 do for all edges (w, x) do activate edge (w, x) with probability min{1, The figure depicts the percolation Burn Process (Algorithm 5) on the l.h.s. and the corresponding BFS burn process (BurnBFS(G, v) in Algorithm 7) on the r.h.s. Activated edges and burnt nodes are coloured red. In this example vt-5 (level M2) is burnt by two predecessors one on level M1 and one on level M2; due to the definition of the BFS burn process, it is placed on M2 (rather than M3).

α deg + G (w) } if (w, x) is activated and x ∈ j≤i M j then add x to M i set parent(x) := w return j≥0 M j v t v t-1 v t-2 v t-3 v t-4 v t-5 v t-1 v t-3 v t-4 v t-6 v t-5 M 3 M 0 M 1 M 2 v t-6
that if H v is the induced subgraph of H (defined in Algorithm 5) consisting of all nodes reachable from v, then T is simply the unique BFS tree of H v starting at v using the order on the vertices.)

In the present section, we will fix some graph G (say some L t produced by the Line Fire process), and look at calls made to BurnBFS with this graph as input. Thus, the only source of randomness is the activation decision of the edges.

In order to apply Hajek's Theorem to φ, the main idea of the following lemma is roughly the following: Given u, let T u denote the nodes burned by starting at u. Then, with probability at least 1 -Ω(3 -k ), we have that all nodes burnt in T u (and thus the new neighbourhood) have a φ-value of at most φ(u) + k which establishes a tail bound. Proposition 4.14 is proved by coupling the burning process with a Galton-Watson process (See Section A.5.1).

On the Galton-Watson tree, we can finally analyze a simpler function that majorizes φ. Proposition 4.14. Let u be a vertex burnt by BurnBFS and T u be the random subtree of T with root u. Then, for any k ≥ 0,

P[ for all w ∈ T u , φ(w) ≤ φ(u) + k | H ] ≥ 1 - 3 -k 12α 4 , where, if i is s. t. u ∈ M i in BurnBFS(G, v)
, then H denotes the history of all activation decisions made by BurnBFS until all nodes belonging to M i are added.

The proof can be found in Section 4.5.4. We continue by proving the following side lemma.

Lemma 4.15.

P deg + Lt (v t ) < δ | F t-1 ≤ 1 2α .
Proof. Observe that P Bin(d, min{1, α d }) = 0 ≤ e -α . Hence, for any vertex, with probability at least 1 -e -α , one or more of its outgoing edges are burned. Starting from amb(v t ) = v t-1 , we do the following.

w ← v t-1 repeat if at least one outgoing edge of w is burned then pick one such edges (w, x) w ← x until w has no outgoing edges that is burned All the vertices traversed by this process are neighbours of v t . Note that, by assumption, the only cycles are in L 0 and we assume, that L 0 is a cycle of length ≥ δ. Thus, since δ = α 20 and α large enough we get,

P deg + Lt (v t ) ≥ δ | F t-1 ≥ (1 -e -α ) δ-1 ≥ 1 -(δ -1)e -α ≥ 1 - 1 2α .
This completes the proof.

To prove the negative bias, we need to analyse the process over two consecutive steps.

We start from an arbitrary history F 2t . We first establish some properties that after one step hold with high probability (w.r. Proof. Let z = φ(v 2t ). Consider the process BurnBFS(G, v 2t ) executed to construct

N + (v 2t+1
). The number of activated edges

|M 1 | at v 2t is distributed |M 1 | ∼ Bin |N (v 2t )|, min 1, α |N (v 2t )| ≤ st Bin |N (v)|, α |N (v)|
and thus E[ |M 1 | ] ≤ α. By Chernoff bounds (Proposition A.3), with probability at least 1 -2 -6α we have |M 1 | ≤ 6α. Assume this holds.

We consider now the out-degree of v 2t . If it is less than δ, then, by definition of φ, all nodes of N + (v 2t ), and in particular all elements of M 1 , have φ-value at most z -1. If it is greater than or equal to δ, then by definition of φ all but one node (v 2t-1 ) of N + (v 2t ) have value less than or equal to z -1. In other words, in the former case all nodes have a small φ-value and in the latter case but one node have a low φ-value. We now consider the latter and argue that w.p. at least 1 -6α/δ the node v 2t-1 is not activated, i. e., v 2t-1 ∈ M 1 .

This follows trivially since we assumed that at most 6α neighbors were activated (chosen u.a.r.) and

|N + (v 2t )| ≥ δ.
Assuming this holds, all elements of M 1 have φ-value less than or equal to z -1. Then, Proposition 4.14 (with k = 0) applied to each sub-tree rooted at nodes of M 1 and taking a Union bound, shows that with probability at least 1 -6α/(12α 4 ), all other nodes visited by BurnBFS(G, v 2t ) have φ-value less than or equal to z -2. Assume this holds.

Moreover, by Lemma 4.15, with probability at least 1-1/(2α), we have deg

+ (v 2t+1 ) ≥ δ.

Assume this holds.

Assuming all those events hold, N + (v 2t+1 ) satisfies all the statements of the lemma.

The probability that one of the assumptions we made along the way fails to be realised is, by Union bound, at most

2 -6α + 6α δ + 6α 12α 4 + 1 2α ≤ 1 α .
We now prove the majorisation Lemma 4.13.

Proof of Lemma 4.13. We start by proving the first part of the statement.

Majorization. Recall that P[ Z = i ] = 3 -i/2 for i ≥ 5, and

P[ Z = 4 ] = 1 - 1 9( √ 3+1)
. Fix the history F 2t up to time 2t and let

∆ := φ(v 2t+2 ) -φ(v 2t ) = φ(v 2t+2 ) -φ(v 2t+1 ) + φ(v 2t+1 ) -φ(v 2t ) = (φ(v 2t+2 ) -φ(amb(v 2t+2 ))) + (φ(v 2t+1 ) -φ(amb(v 2t+1 ))).
If ∆ ≥ i ≥ 4, then at least one of the two expressions on the right hand side exceeds i/2 ≥ 2, so

P[ ∆ ≥ i | F 2t ] ≤ P[ φ(v 2t+1 ) -φ(amb(v 2t+1 )) ≥ i/2 | F 2t ] + P[ φ(v 2t+2 ) -φ(amb(v 2t+2 )) ≥ i/2 | F 2t+1 ].
Thus we need an upper bound on P[ φ(v) -φ(amb(v)) ≥ j | F ] for j ≥ 2, where F is the history right before the arrival of v.

Consider the process BurnBFS(G, amb(v)) with output T amb(v) executed to construct

N + (v)
. By definition of φ, the sub-tree T amb(v) needs to contain a node with φ-value at least φ(amb(v)) + j -1. We use Proposition 4.14 by setting u = amb(v), k = i/2 -1, and can therefore write, by Union bound,

P[ ∆ ≥ i ] ≤ 2 3 -( i/2 -1) 12α 4 = 3 -i/2 1 2α 4 . (4.7)
Thus for any i ≥ 5 and large enough constant α we have

P[ ∆ ≥ i ] ≤ 3 -i/2 1 2α 4 ≤ 3 -i/2 = P[ Z = i ] ≤ P[ Z ≥ i ]. (4.8)
Note that for i < 5 we have P[

Z ≥ i ] = 1 ≥ P[ ∆ ≥ i ]. Thus, we have ∆ ≤ st Z.
On the other hand, by definition of φ, φ(v 2t+2 ) -φ(v 2t ) ≥ -4, so -∆ ≤ st Z. Thus |∆| ≤ st Z, which yields the first part of Lemma 4.13. We now turn to the second statement.

Negative bias. Fix F 2t and consider the arrival of v 2t+1 . With probability at least 1 -1/α the situation described in Lemma 4.16 happens. Assume that to be the case, and consider the arrival of v 2t+2 . Consider the process BurnBFS(G, v 2t+1 ) executed to construct

N + (v 2t+2
φ(v 2t+2 ) ≤ φ(v 2t ) -1.
The probability that one of the assumptions we made along the way fails to be realised is

at most 1 α + 2 -6α + 6α(6α + 1) δ + 6α 12α 4 + 1 2α ≤ 2 α .
To recap, if we let ∆ = φ(v 2t+2 ) -φ(v 2t ), we have just proved that

P[ ∆ ≤ -1 ] ≥ 1 - 2 α .
To compute the expectation (implicitly conditioning on F 2t ), we now write

E[ ∆ ] ≤ k≥4 k • P[ ∆ = k ] + 3 • P[ 0 ≤ ∆ ≤ 3 ] -P[ ∆ ≤ -1 ] ≤ k≥4 P[ ∆ ≥ k ] + 3 • 2 α -1 - 2 α .
(4.9)

The first term on the right hand side can be bounded using (4.7):

k≥4 P[ ∆ ≥ k ] ≤ k≥4 3 -k/2 1 2α 4 ≤ 1 α 2 .
We finally obtain

E[ ∆ | F 2t ] ≤ -1 + 8 α + 1
α 2 < 0, hence the negative bias.

Proof of Proposition 4.14

Proof. The main idea of the proof is to couple the tree process defined by BurnBFS with a Galton-Watson Process. Let i be as in the statement of the lemma, and suppose that the sets M 0 , . . . , M i have already been fixed by the activation decisions in BurnBFS. We look at u ∈ M i , the designated vertex in the statement of the lemma.

Let w be some vertex in T , the tree generated by BurnBFS, and say w ∈ M k-1 . We are interested in understanding the random variable that is the number of children of w in T . Let M w k denote the set M k right after the activation decisions for edges of vertices in M k-1 that are before w in the ordering are completed. Let S = {x ∈ N + (w) | x ∈ ∪ j<k M j and x / ∈ M w k } be the random variable (depending on the choices made while determining M 0 , . . . , M k-1 , M w k \ {w}), that is the set of potential children of w. Let p = min{1, α deg + (w) }. Let B be obtained by adding each x ∈ S to B with probability p. Thus,

Z w := |B| ∼ Bin(|S|, p)
and B corresponds to the activated edges that lead to nodes not already in ∪ j<k M j ∪ M w k . We define R w to be a random variable

R w =    0 if deg + (w) < δ 1 if deg + (w) ≥ δ and amb(w) ∈ B.
We will call the edge (w, amb(w)) in T purple if R w = 1. We are interested in the random variables (Z w , R w ) (note that they are dependent on random choices made earlier in the process defined above; however, to minimise cumbersome notation we will not make this explicit).

We will now define a branching process that is completely independent of the Line Fire process. It is a Galton-Watson process (see Section A.5.1 for a definition), with some designated red edges. Let Z , R be random variables

Z ∼ 1 + eα + Poisson(eα)
that defines the offspring distribution of the Galton-Watson process. Furthermore,

R ∼ Bernoulli(α/δ),
is the indicator variable where R = 1 if and only if the edge between the node and its "first" child is marked red. We will show that this process stochastically dominates the branching process resulting from a call to Burn, in a particular technical sense. We have the following claim: Claim 4.17. Let w be some node in M k-1 , and let S, p, B be as defined above. Let (Z w , R w ) be the random variables defined above for the burn process. Let Z , R be as used to define the independent Galton-Watson process. Then, whenever α ≤ δ, there exists a coupling of the random variables such that Z w < Z and R w ≤ R . Subproof. We distinguish between three cases.

1. If deg + (w) < δ, then R w = 0 ≤ R . So we only need to define a coupling so that Z w < Z .

2. If deg + (w) ≥ δ, but amb(w) ∈ S (that is amb(w) is in some M j for j ≤ k already when the activation decisions for out-edges of w were made), then R w = 0 ≤ R , and again we just need to define a coupling so that Z w < Z . Let T denote the (possibly infinite) Galton-Watson tree with offspring distribution Z and some edges marked "red" as defined above. We define a coupling between the (random)

Finally

sub-tree T u generated by the Burn process (rooted at u ∈ M i ) and T inductively below:

This results in an injective map σ from V (T u ) to V (T ), where V (T ) denotes the vertices in tree T . Let ρ denote the root of T , then σ is defined as follows (through coupling and induction on dist Tu (u, w)). Note that u is the only vertex with dist Tu (u, u) = 0.

1. σ(u) = ρ 2. Suppose all w ∈ T u with dist Tu (u, w) ≤ ∆ are mapped under σ to some vertices in T .

We look at the time when activation decisions for some w such that dist Tu (u, w) = ∆ are made. For each such w, we apply the coupling defined in Claim 4.17. Let (Z w , R w ) be the corresponding random variables and let (Z , R ) be the independent instantiation of the random variables denoting the children of σ(w) in T . By the coupling, we have Z w < Z and R w ≤ R . If R w = 1, we set σ(amb(w)) to be the "red" (first) child of σ(w). The remaining Z w -1 children of w can be mapped to the subsequent Z w -1 children of σ(w), which is possible by the coupling. If R w = 0, all Z w children of w are mapped to the non-"red" children of σ(w), which again is possible since Z > Z w . This defines the map σ for all vertices w , such that dist Tu (u, w ) = ∆ + 1 and completes the inductive step.

We observe that the map σ satisfies the following properties by definition:

1. If (w, x) is an edge in T u , then (σ(w), σ(x)) is an edge in T , and furthermore the edge (σ(w), σ(x)) points away from the root.

2. If (w, x) is coloured purple in T u , then (σ(w), σ(x)) is coloured purple in T .

Finally, we define a function φ : V (T ) → N, on the nodes of the tree T as follows:

1. φ (ρ) = φ(u)

2. For w , let parent(w ) denote the parent of w in T . Then,

φ (w ) =    φ (parent(w )) + 2 if (parent(w ), w ) is red φ (parent(w )) -1 otherwise.
We check the following fact:

Claim 4.18. For every w ∈ V (T u ), φ(w) ≤ φ (σ(w))

Subproof. The proof is based on induction on dist Tu (u, w). Clearly, when dist Tu (u, w) = 0, it must be the case that w = u, and we have φ (σ(u)) = φ (ρ) = φ(u). Suppose, this holds for all w such that dist Tu (u, w) ≤ ∆. Consider an edge (w, x) in T u , such that dist Tu (u, x) = ∆ + 1. Then, we consider two cases:

• If (w, x) is coloured purple, x = amb(w). Also, in this case, the edge (σ(w), σ(x)) in T is also coloured purple. Hence by definition φ (σ(x)) = φ (σ(w)) + 2 ≥ φ(w) + 2.

On the other hand, since (w, x) is red, we have that deg + (w) ≥ δ and x = amb(w).

Thus, by definition of φ, a node cannot have potential difference of more than -2 w.r.t. its ambassador, i. e., φ(w) ≥ φ(x) -2. Hence, putting everything together, we have φ(x) ≤ φ(w) + 2 ≤ φ (σ(x)).

• On the other hand, if (w, x) is not red, we have that φ (σ(x)) = φ (σ(w))-1 ≥ φ(w)-1.

Also by definition of φ, we know that φ(w) ≥ φ(x) + 1 for all x ∈ N + (w) \ {amb(w)}.

Hence, putting everything together, we have φ(x) ≤ φ(w) -1 ≤ φ (σ(x)).

This yields the claim.

Using Claim 4.18 we have,

P[ ∃w ∈ T u s. t. φ(w) ≥ φ(u) + k | H ] ≤ P ∃w ∈ T s. t. φ (w) ≥ φ (ρ) + k . (4.10)
Thus, it only remains to analyze φ on T to bound the r.h.s. of (4.10) and thus yielding the proposition.

Claim 4.19. We have, for k ≥ 0

P ∃w ∈ T s. t. φ (w) ≥ φ (ρ) + k ≤ 3 -k 12α 4 ,
where ρ is the root.

Subproof. Let α = (1 + eα + eα);

for i ≥ 0, let b i = (k + 1)(6α ) i
and let N i be the random variable denoting the number of nodes of T at distance i from the root ρ. We have the following:

P ∃w ∈ T s. t. φ (w) ≥ φ (ρ) + k ≤ (4.11) ≤ P ∃w ∈ T s. t. φ (w) ≥ φ (ρ) + k i {N i < b i } • 1 + P i {N i ≥ b i }
We bound the two terms of (4.11) separately. To bound the first term, we use a Union bound:

P ∃w ∈ T s. t. φ (w) ≥ φ (ρ) + k i {N i < b i } ≤ (4.12) ≤ j≥1 b j max w P φ (w) ≥ φ (ρ) + k | dist(ρ, w) = j
We now bound for an arbitrary (directed) path P, with vertices v 0 = ρ, v 1 , . . . , v j = w in T , the probability that φ (w) ≥ φ (ρ) + k. Note that as we go down the tree T , the value of φ only decreases, except on red edges. Observe by definition of the tree, that the number of children of any node, distributed according to Z ∼ 1 + eα + Poisson(eα) is independent of whether or not the first node is coloured purple (since the random variable R and Z ≥ 1 are drawn independently). Therefore, by assuming that every edge along the path can potentially be red, we are only increasing the probability that for some node w, φ (w) ≥ φ (ρ) + k. Note that the probability that any edge out of a node is red, denoted by p r is α/δ (for the first child of a node, which always exists since the number of children is at least 1 + eα , the probability is p r , for the remaining it is 0). Let r denote the number of purple edges in the path v 0 = ρ, v 1 , . . . , v j = w, then the number of non-purple edges is j -r. Thus, by definition of φ , φ (w) = φ (ρ) + 2 • r -1 • (j -r) = φ (ρ) + 3r -j, and hence for φ (w) ≥ φ(ρ) + k to be true, it must be the case that r ≥ j+k 3 . Thus, we have

P At least j+k 3 edges in P are red ≤ j b= j+k 3 j b p b r (1 -p r ) (j-b) ≤ j b= j+k 3 ej b b p b r ≤ b≥ j+k 3 (3e • p r ) b ≤ (3e • p r ) j+k 3 • 1 1-3epr .
Substituting this bound in (4.12), we get using p r = α/δ = 1/α 19

P ∃w ∈ T s. t. φ (w) ≥ φ (ρ) + k i {N i < b i } ≤ ≤ j≥1 b j 1 1-3epr (3ep r ) 1/3 j+k ≤ (k + 1) 1 1 -3ep r (3ep r ) 1/3 k j≥1 6α (3ep r ) 1/3 j ≤ (k + 1) 1 1 -3ep r (3ep r ) 1/3 k • 6α (3ep r ) 1/3 1 -6α (3ep r ) 1/3 ≤ (k + 1)3 1 1 2 p 1/3 r k • 6α 3 • p 1/3 r 1 2 ≤ (k + 1)216α 1 α 6(k+1) ≤ 3 -k 24α 4 , (4.13)
whenever α ≥ 100. Now, we analyse the second term of (4.11). Let E i denote the event that N i ≥ b i . Thus, we are interested in bounding P[ i E i ]. Observe, that:

P i E i ≤ i≥1 P E i | E i-1 .
We observe that N i is a sum of N i-1 independent copies of Z ∼ 1 + eα + Poisson(eα).

Lemma A.10 proves that,

P E i | E i-1 ≤ 2 -6eab i-1
Thus, we have:

P i E i ≤ i≥1 2 -6eab i-1 ≤ 2 • 2 -6eab 0 = 2 • 2 -(k+1)(36eα) ≤ 3 -k 24α 4 . (4.14)
Substituting (4.13) and (4.14) in (4.11) concludes the proof:

P ∃w ∈ T s. t. φ (w) ≥ φ (ρ) + k ≤ 2 • 3 -k 24α 4 = 3 -k 12α 4 .
This completes the proof of Proposition 4.14.

The foundations of φ

In order to derive a constant expected distance, we require some notion of positive recurrence: Over the course of t time steps, subgraphs with a distance of ω(1) emerge with constant probability and we need to show that the graph "recovers quickly" from this. For this reason, we show that whenever φ is large at time t, it decreases in expectation regardless of the structure of the graph at time t. Such an analysis seems to break if one analyzes the distance directly: One can construct worst-case graphs, where the distance does not decrease in expectation in a constant number of steps. For example one can construct line-like graphs in which the distance is likely to increase considerably. The reason why we are able to show that φ decreases is that in such graphs the degree of the nodes is small and thus causing φ to be relative large (w.r.t. the distance) allowing the potential (in contrast to the distance) to decrease in expectation. On the other side, creating such a worst-case subgraph is extremely unlikely (otherwise φ would increase in expectation). These worst-case graphs are the intuition behind the second line of φ, namely

φ(v) = max w∈N + (v) {φ(w)} + 1 if deg + (v) < δ .
The third line of φ, namely

φ(v) = max      φ(amb(v)) -2, max w∈N + (v) w =amb(v) φ(w) + 1     
consists of two parts: The second part is the core of φ and measures essentially the longest path ignoring ambassador edges (note that the longest path from a node to G 0 is monotonically increasing due to to ambassador edges which is the reason why we disregard them).

The first part (of the third line) ensure that the potential cannot increase by too much if an ambassador edge of a node is activated; without the constraint, the potential increase caused by just activating one edge could be unbounded.

Lower bound -Proof of Lemma 4.5

We set α * = 1/(4e). We majorize the original process by a process P in which the following holds. First, every arriving node performs Poisson(eα) burns nodes upon arrival. As we show, this strictly majorizes the number of neighbours burns and hence increases a node's neighborhood and its distance to L 0 . Second, all the burns happen along the shortest path which, intuitively speaking, only decreases the distance to L 0 .

Let ∆ t = dist Lt (v t , L 0 ) -dist L t-1 (v t-1 , L 0 ). We have for process P that -∞ < ∆ t ≤ 1.
We show that it's very unlikely that the distance upon arrival of a new decreases, more precisely we show the tail bound k+1) by majorizing with a Galton Watson tree with offspring distribution Poisson(eα). From this we get,

P[ ∆ t = -k ] ≤ e -(
E[ ∆ t | F t-1 ] ≥ 1/2 which allows us to conlcude that E[ dist Lt (v t , L 0 ) ] = Ω(t).
Proof of Lemma 4.5. We set α * = 1/(4e). As mentioned earlier, the neighbours of a node v t in the Line Fire process can be represented by the vertices of a tree T rooted at v t-1 , in which every node v appears at most once. Furthermore, the number of edges percolated by

v is X v ∼ Bin |N (v)|, min 1, α |N (v)| = Bin |N (v)|, α |N (v)| for α < 1.
Since X v depends on the degree of node v we will make use of Lemma A.9 and majorize X v by the degreeindependent distribution Poisson(eα).

We define a process P in which, at the arrival of v t , node v t-1 percolates Y v t-1 ∼ Poisson(eα) outgoing edges uniformly at random. Moreover, whenever an edge (v 1 , v 2 ) is percolated, node v 2 percolates Y v 2 ∼ Poisson(eα) of its outgoing edges uniformly at random. Let T vt (T vt , respectively) be the resulting tree of percolated edges in the new process (original process, respectively). Node v t connects then to all nodes of T vt -If the same node v appears several times, then we only connect v t once to v. However, we allow v to be burnt several times: every time v is added, it chooses Poisson(eα) children u.a.r. and independent of former choices to percolate.

Since P r(X v ≥ k) ≤ P r(Y v ≥ k) for k ≥ 1 (by Lemma A.9), we can couple the trees T v
and T vt such that if v ∈ T , then v ∈ T vt . This implies that the neighborhood of v t in the original process is a subset of the neighborhood of v t in P . Hence, the distance of v t in the original process is at least the distance of v t in P .

Let ∆ t = dist Lt (v t , L 0 ) -dist L t-1 (v t-1 , L 0 ). We have -∞ < ∆ t ≤ 1. The distance of a node v to L 0 in L t equals
P[ ∆ t = -k|F t-1 ] ≤ P |T vt | ≥ k + 1 ≤ e -h(1)•(k+1) ≤ e -(k+1) ,
where h(1) ≥ 1 is the function defined in Proposition A.33. Hence,

E[ ∆ t |F t-1 ] ≥ 1 • P X v t-1 = 0 + k≥1 (-k) • P[ ∆ t = -k|F t-1 ] ≥ min d≥1 {(1 -α/d) d } + k≥1 -ke -(k+1) ≥ 1 -α -2/5 ≥ 1/2. Hence, E[ dist Lt (v t , L 0 )|F t-1 ] ≥ dist L t-1 (v t-1 , L 0 ) + 1/2. We have, E[ dist Lt (v t , L 0 ) ] = E[ E[ dist Lt (v t , L 0 )|F t-1 ] ] ≥ E dist L t-1 (v t-1 , L 0 ) + 1/2.
Hence, by repeating this iteratively, we get E[ dist Lt (v t , L 0 ) ] = Ω(t), which yields the claim.

Future Work and Conclusion

The Forest Fire model was proposed by Leskovec et al. to explain several properties of social networks, shrinking diameter being an important one, in addition to densification and power-law degree distributions [START_REF] Leskovec | Graph evolution: Densification and shrinking diameters[END_REF]. As the graphs generated are directed, we focused on distance to the seed graph, rather than diameter as the property of interest.

This work shows that in a restricted version of the Forest Fire model, we can prove that this distance remains bounded, even as the graph size increases, albeit with some conditions on the seed graph.

There are several natural open questions as to how to proceed from here. The obvious one is whether one can remove the conditions on the seed graph. Our simulation results seem to suggest that starting with a single node as a seed graph should also result in similar behaviour. The next is whether one can address densification. Without backward burning, it is clear that the out-degree of any vertex in G t can be at most logarithmic in t. This follows from the fact that the edges have to be on directed paths in the ambassador tree, which is of logarithmic depth. Thus, we cannot expect the average edge density to be more than logarithmic in the number of nodes. Also, for this reason the out-degrees cannot have a heavy tail. In simulations, the in-degrees did exhibit power-law behaviour.

Further models

As mentioned before, backward burning might prove interesting to study as it might result in a super polylogarithmic number of edges as observed in the simulations of [START_REF] Leskovec | Graph evolution: Densification and shrinking diameters[END_REF]. Our analysis however would break: Our potential φ was carefully designed to ensure that for every node at most one out of many outgoing (i. e., burnable) edges is "bad", i. e., leading to node with a high potential. This would no longer be true in the case of backward burning since it would be possible that many nodes with a high (possibly much higher) φ-value have an edge to a node v and thus, when v is "burnt", v might have many neighbours with high φ-values. Changing the potential function alone seems to be futile, as the backward edges of v t are only added after time step t and thus the potential of v t might change after time step t rendering our approach inapplicable.

On the other hand, there is a multitude of variations that our techniques allow to analyze.

For example the Random Surfer Model, in which at the arrival of a node a random walk is started at the ambassador, which then travers the out-edges of nodes stopping at every node with probability p. Even though this model could be analyzed with our techniques, the required bounds on p to have a constant diameter would not be sharp in contrast to [START_REF] Kanade | Distance in the Forest Fire Model How far are you from Eve?[END_REF] where we investigate tight bounds for this model. Another natural extension would be to allow "multi burning": in our model, a node is not allowed to "burn" several times, i. e., to introduce the arriving nodes several times to his own friends several times. Without this restriction the process becomes much easier to analyze: The random process behaves like a Galton-Watson process with a suitable mapping from the children in the resulting Galton-Watson tree to the current graph. If the Galton-Watson tree is infinite, then we must be in the cyclic seed graph and thus the distance to the root is 1. For a suitable constant α it is well-known that this is the case. A drawback of this is the resulting infinite edges to the seed-graph; A simple way around this would be to forbid multi-edges and to simply remove them when they occur (the analysis still works). In this model, it would even be possible the drop the assumption on the seed graph: Once a Galton-Watson tree exceeds a certain depth and by making use of the underlying mapping, we know that the new node must have reached the root-node.

Further models which easily fit in our framework are those where the probability of continuing decays as function of the depth as well as models where the out-edges chosen are always the edges which lead the"oldest" neighbour of a node. Finally, the following variation also falls into our framework: each outgoing edge is activated with constant probability, i. e., without normalizing by the out-degree of a node. This model behaves like the aforementioned Random Surfer model with additional edges. It is worth mentioning that it is unclear whether any of the above models exhibits the densification (of the edges) property.

Conclusion

To summarize, we showed that the Forest Fire model with forward burning exhibits a constant expected distance for large enough constant α and a logarithmic distance for small enough constant α. The machinery we developed and in particular the core ideas of the potential function allow the study of a wide-range of variants for models of social networks.

We were able to apply the high-level ideas of our approach to the at first glance unrelated appearing problem of balls-into-bins (see Chapter 5). We believe that our techniques find application in settings beyond the realm of social networks and balls-into-bins.

Chapter 5

Balls-into-Bins with Deleting Bins [BFK+16a]

One of the fundamental problems in distributed computing is the distribution of requests, tasks, or data items to a set of uniform servers. In order to simplify this process and to avoid a single point of failure, it is often advisable to use a simple, randomized strategy instead of a complex, centralized controller to allocate the requests to the servers. In the most naive strategy (Greedy[1]), each client sends its request to a server chosen uniformly at random.

A more elaborate scheme (Greedy[2]) chooses two servers, queries their current loads, and sends the request to the least loaded of them. Both approaches are typically modeled as balls-into-bins processes [Gon81, RS98, ABKU99, BCSV06, ACMR98, Ste96, BCN+15a],

where requests are represented as balls and servers as bins. While the latter approach leads to considerably better load distributions [START_REF] Azar | Balanced Allocations[END_REF][START_REF] Berenbrink | Balanced Allocations: The Heavily Loaded Case[END_REF], it loses some of its power in synchronous settings, where requests arrive in parallel and cannot take each other into account [START_REF] Adler | Parallel randomized load balancing[END_REF][START_REF] Stemann | Parallel Balanced Allocations[END_REF].

We propose and study a novel infinite batch-based balls-into-bins process to model the client-server scenario. In a round, each server (bin) consumes one of its current tasks (balls). Afterward, in expectation λn tasks arrive and are allocated using a given distribution scheme. The arrival rate λ is allowed to be a function of n (e.g., λ = 1 -1/ poly(n)).

Standard balls-into-bins results imply that, for high arrival rates, with high probability1 (w.h.p.) in each round there is a bin that receives Θ(log n/ log log n) balls. Most other infinite balls-into-bins-type processes limit the total number of concurrent balls in the system by n [ABKU99, BCN+15a] and show a fast recovery.

Results

Since we do not limit the number of balls, our process can, in principle, result in an arbitrary high system load. In particular, if starting in a high-load situation (e.g., exponentially many balls), we cannot recover in a polynomial number of steps. Instead, we regard the system load as a Markov chain and adapt the following notion of self-stabilization: The system is positive recurrent (expected return time to a typical low-load situation is finite), and taking a snapshot of the load situation at an arbitrary (even super-exponential large) time step yields (w.h.p.) a time-independent maximum load. Positive recurrence is a standard notion for stability and basically states that the system load is time-invariant.

For irreducible, aperiodic Markov chains it implies the existence of a unique stationary distribution (cf. Section 5.4). While this alone does not guarantee a good load in the stationary distribution, together with the snapshot property we can look at an arbitrary time window of polynomial size (even if it is exponentially far away from the start) and give strong load guarantees.

In particular, we give the following bounds on the load in addition to showing positive recurrence:

1-Choice Process:

The maximum load at an arbitrary time is (w.h.p.) bounded by

O 1 1-λ • log n 1-λ .
We also provide a lower bound which is asymptotically tight for λ ≤ 1 -1/ poly(n). While this implies that already the simple 1-Choice process is self-stabilizing, the load properties in a "typical" state are poor: even an arrival rate of only λ = 1 -1/n yields a superlinear maximum load.

2-Choice Process:

The maximum load at an arbitrary time is (w.h.p.) bounded by O log n 1-λ . This allows to maintain an exponentially better system load compared to the 1-Choice process; for any λ ≤ 1 -1/ poly(n) the maximum load remains logarithmic.

Note that the resulting processes can be seen as queuing processes.

Approach and Technical Contributions

For the analysis of Greedy[1] the main idea of the proof is to bound the maximum load for any bin i and to take union bound of all resources. The load of bin i decreases whenever it is large and, thus, performs a biased random walk towards a load of zero. However, when the load is zero, it increases in expectation, such that standard drift theorems cannot not be applied directly. Nevertheless, the increase of the load for any given state has an exponential tail, which allows us to apply Hajek's Theorem (Theorem A.11) to derive exponential tail bounds on the load of i at any (possibly super-exponential) number of time steps.

For the analysis of Greedy[2] we define three different potentials (see Section 5.6 for further details) which measure the total load in the system, the load difference as an exponential function (which was already sucuessfully used in [START_REF] Talwar | Balanced Allocations: A Simple Proof for the Heavily Loaded Case. English[END_REF]) as well as a weighted combination of the first two potentials. We're able to derive strong bounds on the load difference after an arbitrary number. In order to derive bounds on the maximum load of the system after an arbitrary number of steps we essentially use union bounds in an adaptive manner: While we cannot apply the same union bound over exponentially many time steps to bound to the load difference of the system at a given time step t, we apply union bounds at every τ < t which bound the load difference as a function of t -τ . This together with combinatorical properties of the potentials will allow us to derive strong bounds on the load of the system at an arbitrary point t in time. See Section 5.6.2 for further intuition.

Related Work

We will continue with an overview of related work. We start with classical results for sequential and finite balls-into-bins processes, go over to parallel settings, and give an overview of infinite and batch-based processes similar to ours. We also briefly mention some results from queuing theory (which is related but studies slightly different quality of service measures and system models).

Sequential Setting. There are many strong, well-known results for the classical, sequential balls-into-bins process. In the sequential setting, m balls are thrown one after another and allocated to n bins. For m = n, the maximum load of any bin is known to be (w.h.p.) In particular, note that the number of balls above the average grows with m for d = 1 but is independent of m for d ≥ 2. This fundamental difference is known as the power of two choices. A similar (if slightly weaker) result was shown by Talwar and Wieder [START_REF] Talwar | Balanced Allocations: A Simple Proof for the Heavily Loaded Case. English[END_REF] using a quite elegant proof technique (which we also employ and generalize for our analysis in Section 5.6). Czumaj and Stemann [START_REF] Czumaj | Randomized allocation processes[END_REF] study adaptive allocation processes where the number of a ball's choices depends on the load of queried bins. The authors subsequently analyze a scenario that allows reallocations.

Berenbrink et al. [BKSS13] adapt the threshold protocol from [ACMR98] (see below)

to a sequential setting and m ≥ n bins. Here, ball i randomly chooses bins until it sees a load smaller than 1 + i/n. While this is a relatively strong assumption on the balls, this protocol needs only O(m) choices in total (allocation time) and achieves an almost optimal maximum load of m/n + 1.

Parallel Setting. Several papers (e.g., [START_REF] Adler | Parallel randomized load balancing[END_REF][START_REF] Stemann | Parallel Balanced Allocations[END_REF]) investigated parallel settings of multiple-Choice games for the case m = n. Here, all m balls have to be allocated in parallel, but balls and bins might employ some (limited) communication. Adler et al. [START_REF] Adler | Parallel randomized load balancing[END_REF] consider a trade-off between the maximum load and the number of communication rounds r the balls need to decide for a target bin. Basically, bounds that are close to the classical (sequential) processes can only be achieved if r is close to the maximum load [START_REF] Adler | Parallel randomized load balancing[END_REF].

The authors also give a lower bound on the maximum load if r communication rounds are allowed, and Stemann [START_REF] Stemann | Parallel Balanced Allocations[END_REF] provides a matching upper bound via a collision-based protocol.

Infinite Processes. In infinite processes, the number of balls to be thrown is not fixed. Instead, in each of infinitely many rounds, balls are thrown or reallocated and bins (possibly) delete old balls. Azar et al. [START_REF] Azar | Balanced Allocations[END_REF] consider an infinite, sequential process starting with n balls arbitrarily assigned to n bins. In each round one random ball is reallocated using the d-Choice process. For any t > cn 2 log log n, the maximum load at time t is (w.h.p.) ln ln(n)/ ln d + O(1).

Adler et al.

[ABS98] consider a system where in each round m ≤ n/9 balls are allocated.

Bins have a FIFO queue, and each arriving ball is stored in the queue of two random bins.

After each round, every non-empty bin deletes its frontmost ball (which automatically removes its copy from the second random bin). It is shown that the expected waiting time is constant and the maximum waiting time is (w.h.p.) ln ln(n)/ ln d + O(1). The restriction m ≤ n/9 is the major drawback of this process. A further study of this process, based on differential methods and experiments, was conducted in [START_REF] Berenbrink | Infinite Parallel Job Allocation (Extended Abstract)[END_REF]. The balls' arrival times are binomially distributed with parameters n and λ = m/n. Their results indicate a stable behavior for λ ≤ 0.86. A similar model was considered by Mitzenmacher [START_REF] Mitzenmacher | The Power of Two Choices in Randomized Load Balancing[END_REF], who considers ball arrivals as a Poisson stream of rate λn for λ < 1. It is shown that the 2-Choice process reduces the waiting time exponentially compared to the 1-Choice process. Czumaj [START_REF] Czumaj | Recovery Time of Dynamic Allocation Processes[END_REF] presents a framework to study the recovery time of discrete-time dynamic allocation processes. In each round one of n balls is reallocated using the d-Choice process. Two models are considered: in the first, the ball to be reallocated is chosen by taking a ball from a random bin. In the second, the ball to be reallocated is chosen by selecting a random ball. From an arbitrary initial assignment, the system is shown to recover to the maximum load from [START_REF] Azar | Balanced Allocations[END_REF] within O n 2 ln n rounds in the former and O(n ln n) rounds in the latter case. Becchetti et al. [START_REF] Becchetti | Self-Stabilizing Repeated Balls-into-Bins[END_REF] consider a similar (but parallel) process. In each round one ball is chosen from every non-empty bin and reallocated to a randomly chosen bin (one Choice per ball). The authors show that (w.h.p.) starting from an arbitrary configuration, it takes O(n) rounds to reach a configuration with maximum load O(log n).

Moreover, if the process starts in a configuration with maximum load O(log n), then the maximum load stays in O(log n) for poly(n) rounds. An interesting connection to our work is that the analysis of [START_REF] Becchetti | Self-Stabilizing Repeated Balls-into-Bins[END_REF] is based on an auxiliary Tetris-process. This process can be seen a special version of our 1-Choice process and is defined as follows: starting from a state with at least n/4 empty bins, in each round every non-empty bin deletes one ball. Subsequently, exactly 3n/4 new balls are allocated to the bins (one choice per ball). Batch-Processes. Batch-based processes allocate m balls to n bins in batches of (usually) n balls each, where each batch is allocated in parallel. They lie between (pure) parallel and sequential processes. For m = τ • n, Stemann [START_REF] Stemann | Parallel Balanced Allocations[END_REF] investigates a scenario with n players each having m/n balls. To allocate a ball, every player independently chooses two bins and allocates copies of the ball to both of them. Every bin has two queues (one for first copies, one for second copies) and processes one ball from each queue per round. When a ball is processed, its copy is removed from the system and the player is allowed to initiate the allocation of the next ball. If τ = ln n, all balls are processed in O(ln n) rounds and the waiting time is (w.h.p.) O(ln ln n). Berenbrink et al. [START_REF] Berenbrink | Multiple-Choice Balanced Allocation in (Almost) Parallel. English[END_REF] study the d-Choice process in a scenario where m balls are allocated to n bins in batches of size n each. The authors show that the load of every bin is (w.h.p.) m/n ± O(log n). As noted in Lemma 5.9, our analysis can be used to derive the same result by easier means. Batch-processes have also been studied in the operations research community [Bai54, Dow55, BDJ98] though with more practical emphasis. Bailey [START_REF] Bailey | On Queueing Processes with Bulk Service[END_REF] and Downton [START_REF] Downton | Waiting Time in Bulk Service Queues[END_REF] study the process where users arrive and are processed once a sufficiently large number (batch) are present and Berg et al. [START_REF] Berg | Optimal Batch Provisioning to Customers Subject to a Delay-Limit[END_REF] studies a variant where manufactures deliver in batches and individually.

Queuing Processes. Batch arrival processes have also been considered in the context of queuing systems. A key motivation for such models stems from the asynchronous transfer mode (ATM) in telecommunication systems. Tasks arrive in batches, are stored in a FIFO queue and served by a fixed number of servers which remove the tasks from the queue and process them. Several papers [SZ92, Kam96, KCYK12, Alf03] consider scenarios where the number of arriving tasks is determined by a finite state Markov chain. Results study steady state properties of the system to determine properties of interest (e.g., waiting times or queue lengths). Sohraby and Zhang [START_REF] Sohraby | Spectral decomposition approach for transient analysis of multi-server discrete-time queues[END_REF] use spectral techniques to study a multi-server scenario with an infinite queue. Alfa [START_REF] Alfa | Algorithmic analysis of the BMAP/D/k system in discrete time[END_REF] considers a discrete-time process for n identical servers and tasks with constant service time s ≥ 1. To ensure a stable system, the arrival rate λ is assumed to be at most n/s and tasks are assigned cyclical, allowing to study an arbitrary server (instead of the complete system). Kamal [START_REF] Kamal | Efficient solution of multiple server queues with application to the modeling of ATM concentrators[END_REF] and Kim et al. [START_REF] Kim | A Complete and Simple Solution to a Discrete-Time Finite-Capacity BMAP/D/c Queue[END_REF] study a system with a finite capacity. The tasks which arrive when the buffer is full are lost. The authors study the steady state probability and give empirical results to show the decay of waiting times as n increases.

Step t (beginning)

Step t (after assignment) In this example we have n = 5 and 4 balls arrive. Balls 1, 2, and 3 choose the same bin with a load of 2 and a bin with larger node and hence all move the same bin resulting in that bin having the highest load. Moreover, Ball 4 chooses two bins with equal load and chooses one of these uniformly at random. At the end of the round all non-empty bins delete one ball (marked gray).

Model & Preliminaries

We model our load balancing problem as an infinite, parallel balls-into-bins process. Time is divided into discrete, synchronous rounds. There are n bins and n generators, and the initial system is assumed to be empty. At the start of each round, every non-empty bin deletes one ball. Afterward, every generator generates a ball with a probability of λ = λ(n) ∈ [0, 1] (the arrival rate). This generation scheme allows us to consider arrival rates that are arbitrarily close to one (like 1 -1/ poly(n)). Generated balls are distributed in the system using a distribution process. We analyze two specific distribution processes:

• The 1-Choice process Greedy[1] assigns every ball to a random bin.

• The 2-Choice process Greedy[2] assigns every ball to a least loaded among two randomly chosen bins.

See Figure 5.1 for an illustration. It is worth mentioning, that the maximum load in Greedy[2] does not need to be smaller than in Greedy[1] as the following (artificial) example shows. Consider two bins (n = 2) with different initial loads and λ = 1. In

Greedy[1] each bin receives n/2 ± c
√ n new balls for some constant c. On the other side, in Greedy[2] the bin with the smaller initial load receives 3n/4 ± c √ n new balls. However, as our results indicate, this effect becomes negligible when n grows.

Notation. The random variable X i (t) denotes the load (number of balls) of the i-th fullest bin at the end of round t. Thus, the load situation (configuration) after round t can be described by the load vector

X(t) = (X i (t)) i∈[n] ∈ N n . We define ∅(t) := 1 n n i=1 X i (t)
as the average load at the end of round t. The value ν(t) denotes the fraction of non-empty bins after round t and η(t) := 1 -ν(t) the fraction of empty bins after round t. It will be useful to define 1 i (t) := min 1, X i (t) and η i (t) := 1 i (t) -ν(t) (which equals η(t) if i is a non-empty bin and -ν(t) otherwise). For random variables X and Y we write

X ≤ st Y if X is stochastically dominated by Y . That is, if for all k we have P[ X ≥ k ] ≤ P[ Y ≥ k ].
Markov Chain Preliminaries. The random process (X(t)) t∈N has the Markov property, since X(t) depends only on X(t -1) and the random choices during round t. We refer to this Markov chain as X. Note that X is time-homogeneous (transition probabilities are time-independent), irreducible (every state is reachable from every other state 2 ), and aperiodic (path lengths have no period; in fact, our chain is lazy). Recall that such a Markov chain is positive recurrent (or ergodic) if the probability to return to the start state is 1 and the expected return time is finite. In particular, this implies the existence of a unique stationary distribution. Positive recurrence is a standard formalization of the intuitive concept of stability. See [START_REF] Levin | Markov Chains and Mixing Times[END_REF] for an excellent introduction into Markov chains and the involved terminology.

The 1-Choice Process

We present two main results for the 1-Choice process: Theorem 5.1 states the stability of the system under the 1-Choice process for an arbitrary λ, using the standard notion of positive recurrence as defined above. In particular, this implies the existence of a stationary distribution for the 1-Choice process. Theorem 5.2 strengthens this by giving a high probability bound on the maximum load for an arbitrary round t ∈ N. Together, both results imply that the 1-Choice process is self-stabilizing. That is, the system is positive recurrent and taking a snapshot of the load situation at an arbitrary time step yields (w.h.p.) a time-independent maximum load.

Theorem 5.1 (Stability). Let λ = λ(n) < 1. The Markov chain X of the 1-Choice process is positive recurrent.

Theorem 5.2 (Maximum Load). Let λ = λ(n) < 1. Fix an arbitrary round t of the 1-Choice process. The maximum load of all bins is (w.h.p.) bounded by O

1 1-λ • log n 1-λ .
Note that for high arrival rates of the form λ

(n) = 1 -ε(n), the bound given in The- orem 5.2 is inversely proportional to ε(n). For example, for ε(n) = 1/n the maximal load is O(n log n). Theorem 5.3
shows that this dependence is unavoidable: the bound given in Theorem 5.2 is tight for large values of λ.

Theorem 5.3. Let n be sufficiently large. Let λ = λ(n) ≥ 3/4 and consider step t := 9λ log(n)/(64(1 -λ) 2 ). With probability 1 -o(1) there is a bin i in step t with load Ω 1 1-λ • log n .
The proofs of these results can be found in the following subsections. We first prove a bound on the maximum load (Theorem 5.2). Afterward, we prove stability of the system (Theorem 5.1). Finally we prove the lower bound (Theorem 5.3). 2 The state space includes all vectors with non-increasing entries over N n .

Maximum Load -Proof of Theorem 5.2

The proof idea is to apply Hajek's Theorem (Theorem A.11) to derive exponential tail bounds on the load of i at any (possibly super-exponential) number of time steps.

Proof of Theorem 5.2. We prove Theorem 5.2 using a (slightly simplified) "drift theorem" from Hajek [START_REF] Hajek | Hitting-Time and Occupation-Time Bounds Implied by Drift Analysis with Applications[END_REF] (cf. Theorem A.11). As mentioned in Section 5.4, our process is a Markov chain, such that we need to condition only on the previous state (instead of the full filtration from Theorem A.11). Our goal is to bound the load of a fixed bin i at time t using Theorem A.11 and, subsequently, to use this with a union bound to bound the maximum load over all bins. To apply Theorem A.11, we have to prove that the maximum load difference of bin i between two rounds is exponentially bounded (Majorization) and that, given a high enough load, the system tends to lose load (Negative Bias). We start with the majorization. Recall that for random variables X and Y we write

X ≤ st Y if X is stochastically dominated by Y , i.e., for all k it holds P[ X ≥ k ] ≤ P[ Y ≥ k ]. The load difference |X i (t + 1) -X i (t)| is bounded by max(1, B i (t)) ≤ 1 + B i (t), where B i (t) is the number of tokens bin i receives during round t + 1. In particular, |X i (t + 1) -X i (t)| | X(t) ≤ st 1 + B i (t). Note that B i (t)
is binomially distributed with parameters n and λ/n since each of the potential n balls has probability λ to spawn and, given that it spawned, with probability 1/n it ends up in bin i. Using standard inequalities we bound

P[ B i (t) = k ] ≤ n k • λ n k ≤ e • n k k • 1 n k = e k k k (5.1)
and calculate

E e B i (t)+1 = e • n k=0 e k • e k k k = e • e 3 -1 k=0 e 2k k k + e • ∞ k=e 3 e 2k k k ≤ Θ(1) + ∞ k=1 e -k = Θ(1).
(5.2)

This shows that the Majorization condition from Theorem A.11 holds (with λ = 1 and

D = Θ(1)).
To see that the Negative Bias condition is also given, note that if bin i has non-zero load, it is guaranteed to delete one ball and receives in expectation n

• λ/n = λ balls. We get E[ X i (t + 1) -X i (t)|X i (t) > 0 ] ≤ λ -1 < 0, establishing the Negative Bias condition (with ε 0 = 1 -λ)
. Thus, we can apply Theorem A.11 with η := min(1, (1 -

λ)/(2D), 1/(2 -2λ)) = (1 -λ)/(2D) and get for b ≥ 1 P[ X i (t) ≥ b ] ≤ e -b•η + 2D η • (1 -λ) • e η•(-b) ≤ 2 • (2D) 2 (1 -λ) 2 • e (1-λ)•(-b) 2D ≤ (4D) 2 (1 -λ) 2 • e -b•(1-λ) (4D) 2 ≤ c (1 -λ) 2 • e -b•(1-λ) c , (5.3)
where c ≥ (4D) 2 denotes a suitable constant. Applying the Union bound to all n bins and

choosing b := c 1-λ •ln c•n h+1 (1-λ) 2 , where h > 2 is a constant, yields P max i∈[n] X i (t) ≥ b ≤ n -h . Since b = c 1 -λ • ln c • n h+1 (1 -λ) 2 ≤ c 2 • (h + 1) 1 -λ • ln n 1 -λ = O 1 1 -λ • ln n 1 -λ ,
(5.4)

we get the desired statement.

Stability -Proof of Theorem 5.1

In the following, we provide an auxiliary result that will prove useful for deriving the stability of the 1-Choice process.

Corollary 5.4. Let λ = λ(n) < 1. Fix an arbitrary round t of the 1-Choice process and a bin i. There is a constant c > 1 such that the expected load of bin i is bounded by

6c 1-λ • ln n 1-λ .
Proof. By Theorem 5.2, the maximum load of all bins is with high probability bounded by

c • 1 1-λ • log n 1-λ , for a sufficiently large constant c. Let γ := c 1 -λ • ln e • cn (1 -λ) 2 .
(5.5)

Partitioning time into windows of γ rounds and with (5.3), we calculate

E[ X i (t) ] = γ b=1 b • P[ X i (t) = b ] + ∞ k=1 (k+1)γ b=k•γ+1 b • P[ X i (t) = b ] ≤ γ + ∞ k=1 (k + 1)γ • P[ X i (t) > k • γ ] ≤ γ + ∞ k=1 (k + 1)γ • c (1 -λ) 2 • e -k•γ•(1-λ) c ≤ γ + ∞ k=1 (k + 1)γ • c (1 -λ) 2 • e -k • e -ln(cn/(1-λ) 2 ) ≤ γ + ∞ k=1 (k + 1)γ • e -k ≤ 3γ ≤ 6c 1 -λ • ln e • cn 1 -λ .
(5.6)

This finishes the proof.

Proof of Theorem 5.1. We prove Theorem 5.1 using a result from Fayolle et al. [START_REF] Fayolle | Topics in the Constructive Theory of Countable Markov Chains[END_REF] (cf. Theorem A.22). Note that X is a time-homogeneous irreducible Markov chain with a countable state space. In the following, let

∆ := 12e 2 • c 2 n 2 (1 -λ) 3 , (5.7)
where c is the constant from Corollary 5.4. For a configuration x, we define the auxiliary potential Ψ(x) := n i=1 x i as the total system load of configuration x. Consider the (finite) set C := { x | Ψ(x) ≤ n • ∆ } of all configurations with not too much load. To prove positive recurrence, it remains to show that item 1 (expected potential drop if not in a high-load configuration) and item 2 (finite potential) of Theorem A.22 hold. Let us start with item 1.

Fix a round t and let x = X(t) ∈ C. By definition of C, we have Ψ(x) > n • ∆. Hence, there is at least one bin i with load x i ≥ Ψ(x)/n > ∆. Thus, by definition of the process, during each of the next ∆ rounds bin i deletes exactly one ball. On the other hand, bin i

receives in expectation ∆ • λn • 1 n = λ∆ balls during the next ∆ rounds. We get E[ X i (t + ∆) -x i |X(t) = x ] = λ∆ -∆ = -(1 -λ) • ∆.
For any bin j = i, we assume pessimistically that no ball is deleted. Note that the expected load increase of each of these bins can be majorized by the load increase in an empty system running for ∆ rounds. Thus, we can use Corollary 5.4 to bound the expected load increase in each of these bins

by 6c 1-λ • ln 2•cn 1-λ ≤ 6e 2 •c 2 •n (1-λ) 2 = (1-λ)∆ 2n
, by definition of ∆. We get

E[ Ψ(X(t + ∆))|X(t) = x ] ≤ -(1 -λ) • ∆ + (n -1) • (1 -λ)∆ 2n ≤ - 1 -λ 2 • ∆.
(5.8)

This proves item 1 of Theorem A.22. For item 2, assume x = X(t) ∈ C. We bound the system load after ∆ rounds trivially by

E[ Ψ(X(t + ∆))|X(t) = x ] ≤ Ψ(x) + ∆ • n ≤ n • ∆ + ∆ • n < ∞, (5.9)
(note that the finiteness in Theorem A.22 is with respect to time, not n). This finishes the proof.

Lower Bound on Maximum Load -Proof of Theorem 5.3

In expectation, the load of any non-empty bin decreases. Thus, to derive a meaningful lower bound, we need to make use of the variance of the number of balls that are assigned to a bin over a period of suitable length. To do so, we make use of Theorem A.34 (due to Raab and Steger [START_REF] Raab | Balls into Bins" -A Simple and Tight Analysis[END_REF]; see appendix), which lower-bounds the maximum number of balls a bin receives when m balls are allocated into n bins.

Proof of Theorem 5.3. We assume that we start at an empty system and apply Theorem A.34 to m := λtn many balls. The theorem states that, due to the variance, one of the bins is likely to get more than c 1 λt + c 2 √ tλ log n many balls for suitable constants c 1 and c 2 . This allows us to show that the load of this bin is large, even if we assume, pessimistically, that it deletes a ball during each of the t time steps.

Let M (t ) be the number of balls allocated during the first t ∈ N steps, and let Y max (t )

be the maximum number of balls allocated to any bin. Set

t := 9λ log(n) 64(1 -λ) 2
(5.10) and let ε := (1 -λ)/λ. Since all balls are independent and

E[ M (t) ] = t • λn ≥ n log n (due to λ ≥ 3/4
), it follows by Chernoff's inequality that

P[ M (t) ≤ (1 -ε) • t • λn ] ≤ e -ε 2 E[ M (t) ]/2 ≤ 1 n 2 .
(5.11) By Theorem A.34 Cases 3 and 4 (depending on the size of 1 -λ) we get for α := 8/9 (w.h.p.)

Y max (t) ≥ ≥ (1 -ε) • t • λ + 2(1 -ε) • t • λ log n • min α, 1 - log log n 2α log n = (1 -ε) • t • λ + α 2(1 -ε) • t • λ log n.
(5.12)

Let X max (t) denote the load of the bin of maximum load. We derive,

X max (t) ≥ (1 -ε) • t • λ + (1 -ε) • 16 9 t • λ log n -t = (1 -ε) • t • λ + 1 -ε 4 • λ log n (1 -λ) -t = 1 -ε 4 • λ log n (1 -λ) -2(1 -λ)t = 1 -ε 4 • λ log n (1 -λ) - 9λ log(n) 32(1 -λ) =   1 -1-λ λ 4 - 9 32   • λ log n (1 -λ) = Ω λ log n 1 -λ , (5.13)
where the last inequality holds since λ ≥ 3/4.

The 2-Choice Process

We continue with the study of the 2-Choice process. Here, new balls are distributed according to Greedy[2] (cf. description in Section 5.4). Our main results are the following theorems, which are equivalents to the corresponding theorems for the 1-Choice process.

Theorem 5.5 (Stability).

Let λ = λ(n) ∈ [1/4, 1). The Markov chain X of the 2-Choice process is positive recurrent. Theorem 5.6 (Maximum Load). Let λ = λ(n) ∈ [1/4, 1)
. Fix an arbitrary round t of the 2-Choice process. The maximum load of all bins is (w.h.p.) bounded by O log n 1-λ .

Note that Theorem 5.6 implies a much better behaved system than we saw in Theorem 5.2 for the 1-Choice process. In particular, it allows for an exponentially higher arrival rate: for λ(n) = 1 -1/ poly(n) the 2-Choice process maintains a maximal load of O(log n).

In contrast, for the same arrival rate the 1-Choice process results in a system with maximal load Ω(poly(n)).

Our analysis of the 2-Choice process relies to a large part on a good bound on the smoothness (the maximum load difference between any two bins). This is stated in the following proposition. This result is of independent interest, showing that even if the arrival rate is λ(n) = 1-e -n , where we get a polynomial system load, the maximum load difference is still logarithmic. Analysis Overview. To prove these results, we combine three different potential functions: For a configuration x with average load ∅ and for a suitable constant α < 1 (to be fixed later), we define

Φ(x) := i∈[n] e α•(x i -∅) + i∈[n] e α•(∅-x i ) , Ψ(x) := i∈[n]
x i , and

Γ(x) := Φ(x) + n 1-λ • Ψ(x).
(5.14)

The potential Φ measures the smoothness (the maximum load difference to the average) of a configuration and is used to prove Proposition 5.7 (Section 5.6.1). The proof is based on the observation that whenever the load of a bin is far from the average load, it decreases in expectation. The potential Ψ measures the total load of a configuration and is used, in combination with our results on the smoothness, to prove Theorem 5.6 (Section 5.6.2). The potential Γ entangles the smoothness and total load, allowing us to prove Theorem 5.5 (Section 5.6.3). The proof is based on the fact that whenever Γ is large (i.e., the configuration is not smooth or it has a huge total load), it decreases in expectation.

Before we continue with our analysis, let us make a simple but useful observation concerning the smoothness: For any configuration x and value b ≥ 0, the inequality Φ(x) ≤ e α•b implies (by definition of Φ) max i |x i -∅| ≤ b. That is, the load difference of any bin to the average is at most b and, thus, the load difference between any two bins is at most 2b.

Observation 5.8. Consider a configuration x with average load ∅ and let b ≥ 0.

If Φ(x) ≤ e α•b , then |x i -∅| ≤ b for all i ∈ [n]. In particular, max i (x i ) -min i (x i ) ≤ 2b.

Bounding the Smoothness -Proof of Proposition 5.7

The goal of this section is to prove Proposition 5.7. The key ingredient for its proof is the following statement: There are values 0 < c < 1 and γ > 0 such that

E[ Φ(X(t + 1))|X(t) ] ≤ c • Φ(X(t)) + γ (5.15)
holds for all rounds t ≥ 0. Once (5.15) is proven, taking the expected value on both sides yields

E[ Φ(X(t + 1)) ] ≤ c • E[ Φ(X(t)) ] + γ. This recursion is solved by E[ Φ(X(t)) ] ≤ γ • (1 -c) -1 .
In the rest of this section, we prove that (5.15) holds for a constant c and γ = O(n), such that we immediately get the following bound on the expected smoothness (potential Φ) at an arbitrary time t:

Lemma 5.9. Let λ ∈ [1/4, 1]. Fix an arbitrary round t of the 2-Choice process. There is a constant ε > 0 such that E[ Φ(X(t)) ] ≤ n/ε.
In Lemma 5.9, we chose λ ∈ [1/4, 1] for convenience; the proof works with minor modifications for any λ = Θ(1) (i.e., for any constant λ, no matter whether λ < 1 or λ > 1). Also, our analysis easily adapts to the process without deletions by setting λ = 1 and η i (t) = 0.

This yields the same results as [START_REF] Berenbrink | Multiple-Choice Balanced Allocation in (Almost) Parallel. English[END_REF] using a simpler analysis.

Proposition 5.7 emerges by combining Observation 5.8, Lemma 5.9, and Markov's inequality:

P max i X i (t) -min i X i (t) ≥ 4 α • ln n ε ≤ P Φ(X(t)) ≥ n 2 ε 2 ≤ ε n .
It remains to prove (5.15). Our proof follows the lines of [PTW10, TW14]3 . We start by splitting the potential Φ(x) in two parts:

Φ(x) = Φ + (x) + Φ -(x), (5.16)
with the upper potential Φ + (x) := i e α•(x i -∅) and with the lower potential Φ -(x) := i e α•(∅-x i ) . For a fixed bin i, we use Φ i,+ (x) := e α•(x i -∅) and Φ i,-(x) := e α•(∅-x i ) to denote i's contribution to the upper and lower potential, respectively. When we consider the effect of a fixed round t + 1, we will sometimes omit the time parameter and use prime notation to denote the value of a parameter at the end of round t+1. For example, we write X i and X i for the load of bin i at the beginning and at the end of round t + 1, respectively. Two simple but useful identities regarding the potential drops ∆ i,+ (t + 1) := Φ i,+ (X(t + 1)) -Φ i,+ (X(t)) and ∆ i,-(t + 1) := Φ i,-(X(t + 1)) -Φ i,-(X(t)) due to a fixed bin i during round t + 1 are as follows:

Observation 5.10. Fix a bin i, let K denote the number of balls that are placed during round t + 1 and let k ≤ K be the number of these balls that fall into bin i. Then,

1. ∆ i,+ (t + 1) = Φ i,+ (X(t)) • e α•(k-η i (t)-K/n) -1 and 2. ∆ i,-(t + 1) = Φ i,-(X(t)) • e -α•(k-η i (t)-K/n) -1 .
Proof. Remember that 1 i is an indicator value which equals 1 if and only if the i-th bin is non-empty in configuration X. Bin i looses exactly 1 i balls and receives exactly k balls, such that X i -X i = -1 i + k. Similarly, we have ∅ -∅ = -ν + K/n for the change of the average load. With the identity η i = 1 i -ν (see Section 5.4), this yields

∆ i,+ (t + 1) = e α• X i -∅ -e α• X i -∅ = e α• X i -∅ • e α• -1 i +k+ν-K/n -1 = Φ i,+ • e α•(k-η i -K/n) -1 ,
(5.17)

proving the first statement. The second statement follows similarly.

Preliminaries to Bound the Potential Drop

We now derive the main technical lemma that states general bounds on the expected upper and lower potential change during one round. This will be used to derive different bounds on the potential change depending on the situation (Section 5.6.1). For this, let

p i := i n 2 -i-1 n 2 = 2i-1
n 2 (the probability that a ball thrown with Greedy[2] falls into the i-th fullest bin). We also define α := e α -1 and α := 1 -e -α .

(5.18)

Note that α ∈ (α, α + α 2 ) and α ∈ (α -α 2 , α) for α ∈ (0, 1.7). This follows from the

Taylor approximation e x ≤ 1 + x + x 2 , which holds for x ∈ (-∞, 1.7] (we will use this approximation several times in the analysis). Finally, let δi :

= λn • ( 1 /n • 1 --p i • α/α) and δi := λn • ( 1 /n • 1 + -p i • α/α), (5.19)
where 1 -:= 1 -α/n < 1 < 1 + := 1 + α/n. These δi and δi values can be thought of as upper/lower bounds on the expected difference in the number of balls that fall into bin i under the 1-Choice and 2-Choice process, respectively (note that 1 + , 1 -, α/α, and α/α are all close to 1).

Lemma 5.11. Consider a bin i after round t and a constant α ≤ 1.

1. For the expected change of i's upper potential during round t + 1 we have

E[ ∆ i,+ (t + 1)|X(t) ] Φ i,+ (X(t)) ≤ -α • η i + δi + α 2 • η i + δi 2 .
(5.20)

2. For the expected change of i's lower potential during round t + 1 we have

E[ ∆ i,-(t + 1)|X(t) ] Φ i,-(X(t)) ≤ α • η i + δi + α 2 • η i + δi 2 .
(5.21)

Proof. For the first statement, we use Observation 5.10 to calculate

E[ ∆ i,+ (t)|X ]/Φ i,+ = = n K=0 K k=0 n K K k (p i λ) k • (1 -p i )λ K-k • (1 -λ) n-K • e α•(k-η i -K/n) -1 = n K=0 n K (1 -λ) n-K λ K K k=0 K k • p k i • (1 -p i ) K-k • e α•(k-η i -K/n) -1 = n K=0 n K (1 -λ) n-K λ K • e -α(η i +K/n) K k=0 K k (e α • p i ) k (1 -p i ) K-k -1 = n K=0 n K (1 -λ) n-K λ K • e -α(η i +K/n) • (1 + α • p i ) K -1 ,
where we first apply the law of total expectation together with Observation 5.10 and, afterward, twice the binomial theorem. Continuing the calculation using the aforementioned

Taylor approximation e x ≤ 1+x+x 2 (which holds for any x ∈ (-∞, 1.7]), and the definition

of δi yields = e -αη i • 1 -λ + λe -α/n • (1 + α • p i ) n -1 ≤ e -αη i • 1 -λ(1 -e -α/n ) + λ • α • p i n -1 ≤ e -αη i • 1 - λ • α n • (1 -α/n) + λ • α • p i n -1 ≤ e -αη i • 1 - α n • δi n -1 ≤ e -α•(η i + δi) -1.
Now, the claim follows by another application of the Taylor approximation. The second statement follows similarly.

Before we apply Lemma 5.11 to derive different bounds on the potential drop for various situations, we provide three auxiliary claims:

Claim 5.12. Consider a bin i and the values δi and δi as defined before Lemma 5.11. If

α ≤ ln(10/9), then max(| δi |, | δi |) ≤ 5λ/4. Proof. Remember that δi = λn • ( 1 /n • 1 --p i • α/α) and δi = λn • ( 1 /n • 1 + -p i • α/α),
where

1 -= 1 -α/n < 1 < 1 + α/n = 1 + (see proof of Lemma 5.11). Note that if α ≤ ln(10/9),
we have 1 + < 5/4 and 1 -> 8/9. Since the p i are non-decreasing in i, it is sufficient to consider the extreme cases i = 1 and i = n.

The claims hold trivially for i = 1, since p 1 = 1/n 2 and both

| 1 /n • 1 --p i • α/α| ≤ 1/n and | 1 /n • 1 + -p i • α/α| ≤ 1 + /n.
For the other extreme, i = n, we have p n ≤ 2/n. From this and the definition of α = e α -1, we get

| δi | ≤ 5 4 λ, since 2 n • α α -1 n • 1 -≤ 2 n 10/9-1 ln(10/9) -1 n • 1 -< 5 4n . Similarly, | δi | ≤ 5 4 λ follows together with 2 n α α -1 n • 1 + < 1 n (which holds for any α > 0).
Claim 5.13. There is a constant ε > 0 such that

1. i≤ 3 4 n p i • Φ i,+ ≤ (1 -2ε) • Φ + n and 2. i∈[n] p i • Φ i,-≥ (1 + 2ε) • Φ --i≤ n 4 Φ i,- n .
Proof. For part1, note that the Φ i,+ are non-increasing in i, that they sum up to Φ + , and that the p i are non-decreasing in i. Thus, the left hand side of the claim's first statement is maximized if Φ i,+ = 4Φ + 3n for all i. Now note that there is a constant ε such that4 i>3n/4 p i ≥ 1 4 + ε. We get i≤3n/4 p i ≤ 3 4 -ε. With this, the result follows by

i≤ 3 4 n p i • Φ i,+ ≤ 3 4 -ε 4Φ + 3n = 1 - 4ε 3n • Φ + ≤ (1 -2ε) • Φ + n . (5.22)
Part 2 follows similarly.

Claim 5.14. Consider a round t and a constant α ≥ 0. Then:

1. i∈[n] αη i (αη i -1) • Φ i,+ (X(t)) ≤ α 2 ην • min n, Φ + (X(t)) and 2. i∈[n] αη i (αη i + 1) • Φ i,-(X(t)) ≤ α 2 ην • Φ -(X(t)).
Proof. For the first statement, we calculate

i∈[n] αη i (αη i -1) • Φ i,+ (X(t)) = i≤νn αη i (αη i -1) • Φ i,+ (X(t)) + i>νn αη i (αη i -1) • Φ i,+ (X(t)) = αη(αη -1) • i≤νn Φ i,+ (X(t)) + αν(1 + αν) • i>νn Φ i,+ (X(t)) ≤ αη(αη -1) • ν • Φ + (X(t)) + αν(1 + αν) • η • min n, Φ + (X(t)) ≤ α 2 ην • min n, Φ + (X(t)) ,
(5.23)

where the first inequality uses that Φ i,+ (X(t)) is non-increasing in i and that Φ i,+ (X(t)) ≤ 1 for all i > νn. The claim's second statement follows by a similar calculation, using that Φ i,-(X(t)) is non-decreasing in i (note that we cannot apply the same trick as above to get

min n, Φ -(X(t)) instead of Φ -(X(t))).

Bounding the Potential Drop in Different Situations

With these tools in place, we can derive the bounds on the potential drop in different situations. We start with a relative bound on the upper potential change ∆ + (t + 1) :=

E[ ∆ R (t + 1)|X(t) ] < 2αλ • Φ R (X(t)).
(5.24)

Proof. We prove the statement for R = +. The case R = -follows similarly. Using Lemma 5.11 and summing up over all i ∈ [n] we get

E[ ∆ + (t + 1)|X ] ≤ i∈[n] -α • (η i + δi ) + α 2 • (η i + δi ) 2 • Φ i,+ = i∈[n] η i α(η i α -1) + α 2 • (2η i δi + δ2 i ) -α • δi • Φ i,+ ≤ i∈[n] η i α(η i α -1) + 5α 2 λ + 5 4 αλ • Φ i,+ .
Here, the last inequality uses λ ≤ 1 and | δi | ≤ 5 4 λ (Claim 5.12). We now apply Claim 5.14, νη ≤ 1/4 ≤ λ, and α < 1/8 to get

E[ ∆ + (t)|X ] ≤ α 2 λ + 5α 2 λ + 5 4 αλ • Φ + < 2αλ • Φ + , (5.25)
the desired statement.

The next two lemmas derive bounds that are used to bound the upper/lower potential change in reasonably balanced configurations.

Lemma 5.16. Consider a round t and the constants ε (from Claim 5.13) and α ≤ min(ln(10/9), ε/4). Let λ ∈ [1/4, 1] and assume X 3 4 n (t) ≤ ∅(t). For the expected upper potential drop during round t + 1 we have

E[ ∆ + (t + 1)|X(t) ] ≤ -εαλ • Φ + (X(t)) + 2αλn.
(5.26)

Proof.
To calculate the expected upper potential change, we use Lemma 5.11 and sum up over all i ∈ [n] (using similar inequalities as in the proof of Lemma 5.15 and the definition of δi ):

E[ ∆ + (t + 1)|X ] ≤ 6α 2 λ • Φ + - i∈[n] α • δi • Φ i,+ = 6α 2 λ -αλ • 1 -• Φ + + αλn i∈[n] p i • Φ i,+ .
(5.27)

We now use that Φ i,+ = e α•(X i -∅) ≤ 1 for all i > 3 4 n (by our assumption on X 3 4 n ). This yields

E[ ∆ + (t + 1)|X ] ≤ 6α 2 λ -αλ • 1 -• Φ + + αλn i≤ 3 4 n p i • Φ i,+ + 2αλn.
(5.28)

Finally, we apply Claim 5.13 and the definition of 1 -and α to get

E[ ∆ + (t + 1)|X ] ≤ 6α 2 λ -αλ • 1 -+ (1 -2ε) • αλ • Φ + + 2αλn ≤ 4α 2 λ -2ε • αλ • Φ + + 2αλn.
(5.29)

Using α ≤ ε/4 yields the desired result.

Lemma 5.17. Consider a round t and the constants ε (from Claim 5.13) and α ≤ min(ln(10/9), ε/8). Let λ ∈ [1/4, 1] and assume X n 4 (t) ≥ ∅(t). For the expected lower potential drop during round t we have

E[ ∆ -(t + 1)|X(t) ] ≤ -εαλ • Φ -(X(t)) + αλn 2 .
(5.30)

Proof.

To calculate the expected lower potential change, we use Lemma 5.11 and sum up over all i ∈ [n] (as in the proof of Lemma 5.16):

E[ ∆ -(t + 1)|X ] ≤ 6α 2 λ • Φ -+ i∈[n] α • δi • Φ i,- = 6α 2 λ + αλ • 1 + • Φ --αλn i∈[n] p i • Φ i,-.
(5.31)

We now use that Φ i,-= e α•(∅-X i ) ≤ 1 for all i ≤ n 4 (by our assumption on X n 4 ) and apply Claim 5.13 to get

E[ ∆ -(t)|X ] ≤ 6α 2 λ + αλ • 1 + • Φ --(1 + 2ε) • αλn • Φ --n 4 n = 6α 2 λ + αλ • 1 + -(1 + 2ε) • αλ • Φ -+ (1 + 2ε) • αλn 4 ≤ 8α 2 λ -2ε • αλ • Φ -+ αλn 2 ,
(5.32)

where the last inequality used the definitions of 1 + , α, as well as α > α -α 2 . Using α ≤ ε/8 yields the desired result.

The following two lemmas bound the potential drop in configurations with many balls far below the average to the right and with many balls far above the average to the left.

Lemma 5.18. Consider a round t and constants α ≤ 1/46 (< ln(10/9)) and ε ≤ 1/3.

Let λ ∈ [1/4, 1] and assume X 3 4 n (t) ≥ ∅(t) and E[ ∆ + (t + 1)|X(t) ] ≥ -εαλ 4 • Φ + (X(t)). Then, Φ + (X(t)) ≤ ε 4 • Φ -(X(t)) or Φ(X(t)) = ε -8 • O(n). Proof. Let L := i∈[n] max(X i -∅, 0) = i∈[n] max(∅ -X i , 0
) be the "excess load" above and below the average. First note that the assumption

X 3 4 n ≥ ∅ implies Φ -≥ n 4
• exp( αL n/4 ) (using Jensen's inequality). On the other hand, we can use the assumption E[ ∆ + (t + 1)|X ] ≥ -εαλ 4 • Φ + to show an upper bound on Φ + . To this end, we use Lemma 5.11 and sum up over all i ∈ [n] (as in the proof of Lemma 5.16):

E[ ∆ + (t + 1)|X ] ≤ 6α 2 λ • Φ + - i∈[n] α • δi • Φ i,+ = 6α 2 λ • Φ + - i≤ n 3 α • δi • Φ i,+ - i> n 3 α • δi • Φ i,+ .
(5.33) For i ≤ n/3 we have p i = 2i-1 n 2 ≤ 2 3n and, using the definition of 1 -and α, δi = λn

• 1 /n • 1 --p i • α/α ≥ (1 -5α)λ/3
. Setting Φ ≤n/3,+ := i≤n/3 Φ i,+ and Φ >n/3,+ := i>n/3 Φ i,+ , together with Claim 5.12 this yields

E[ ∆ + (t + 1)|X ] ≤ ≤ 6α 2 λ • Φ + - α(1 -5α)λ 3 • Φ ≤n/3,+ + 5 4 αλ • Φ >n/3,+ = 6α 2 λ - α(1 -5α)λ 3 • Φ + + 5 4 αλ + α(1 -5α)λ 3 • Φ >n/3,+ ≤ - εαλ 2 • Φ + + 2αλ • Φ >n/3,+ ,
(5.34)

where the last inequality uses α ≤ 1/46 ≤ 1 23 -3 46 ε. With this, the assumption

E[ ∆ + (t + 1)|X ] ≥ -εαλ 4 • Φ + implies Φ + ≤ 8 ε • Φ >n/3,+ ≤ 8 ε • 2n 3 e αL n/3 = 16n 3ε e 3αL n
(the last inequality uses that none of the 2n/3 remaining bins can have a load higher than L/(n/3)).

To finish the proof, assume Φ + > ε 4 • Φ -(otherwise the lemma holds). Combining this with the upper bound on Φ + and with the lower bound on Φ -, we get 16n 3ε e

3αL n ≥ Φ + > ε 4 • Φ -≥ εn 16 • e 4αL n .
(5.35)

Thus, the excess load can be bounded by

L < n α • ln 256 3ε 2 . Now, the lemma's statement follows from Φ = Φ + + Φ -< 5 ε • Φ + ≤ 80n 3ε 2 e 3αL n = ε -8 • O(n).
Lemma 5.19. Consider a round t and constants α ≤ 1/32 (< ln(10/9)) and ε ≤ 1.

Let λ ∈ [1/4, 1] and assume X n 4 (t) ≤ ∅(t) and E[ ∆ -(t + 1)|X(t) ] ≥ -εαλ 4 • Φ -(X(t)). Then, Φ -(X(t)) ≤ ε 4 • Φ + (X(t)) or Φ(X(t)) = ε -8 • O(n). Proof. Let L := i∈[n] max(X i -∅, 0) = i∈[n] max(∅ -X i , 0
) be the "excess load" above and below the average. First note that the assumption X n 4 ≤ ∅ implies Φ + ≥ n 4 •e αL n/4 (using Jensen's inequality). On the other hand, we can use the assumption E[ ∆ -(t + 1)|X ] ≥ -εαλ 4 • Φ -to show an upper bound on Φ -. To this end, we use Lemma 5.11 and sum up over all i ∈ [n] (as in the proof of Lemma 5.17):

E[ ∆ -(t + 1)|X ] ≤ 6α 2 λ • Φ -+ i∈[n] α • δi • Φ i,- = 6α 2 λ • Φ -+ i≤ 2n 3 α • δi • Φ i,-+ i> 2n 3 α • δi • Φ i,-.
(5.36)

For i ≥ 2n/3 we have

p i = 2i-1 n 2 ≥ 4 3n -1 n 2 . Using this with p i ≤ p n ≤ 2/n and α ≥ α -α 2 , we can bound δi = λn • 1 /n • 1 + -p i • α/α ≤ λ • (-1 /3 + 1+α n ) + 2αλ ≤ -λ/6 + 2αλ
. Setting Φ ≤2n/3,-:= i≤2n/3 Φ i,-and Φ >2n/3,-:= i>2n/3 Φ i,-, together with Claim 5.12 74 this yields

E[ ∆ -(t + 1)|X ] ≤ ≤ 6α 2 λ • Φ -+ 5 4 αλ • Φ ≤2n/3,-- αλ 6 • Φ >2n/3,-+ 2α 2 λ • Φ >2n/3,- ≤ 8α 2 λ -αλ/6 • Φ -+ 5 4 αλ + αλ/6 • Φ ≤2n/3,- ≤ - εαλ 2 • Φ -+ 2αλ • Φ ≤2n/3,-,
(5.37)

where the last inequality uses α ≤ 1/32 ≤ 1 16 -1 48 ε. With this, the assumption

E[ ∆ -(t + 1)|X ] ≥ -εαλ 4 • Φ -implies that Φ -≤ 8 ε • Φ ≤2n/3,-≤ 8 ε • 2n 3 e αL n/3 = 16n 3ε e 3αL n
(the last inequality uses that none of the 2n/3 remaining bins can have a load higher than L/(n/3)). To finish the proof, assume Φ -> ε 4 •Φ + (otherwise the lemma holds). Combining this with the upper bound on Φ -and with the lower bound on Φ + , we get

16n 3ε e 3αL n ≥ Φ -> ε 4 • Φ + ≥ εn 16 • e 4αL n .
(5.38)

Thus, the excess load can be bounded by

L < n α • ln 256 3ε 2 . Now, the lemma's statement follows from Φ = Φ + + Φ -< 5 ε • Φ -≤ 80n 3ε 2 e 3αL n = ε -8 • O(n).

Proving (5.15)

With the lemmas from Section 5.6.1, we are finally ready to prove (5.15). More exactly, we argue that for the constant ε from Claim 5.13 and α ≤ min(1/32, ε/8), for any λ ∈ [1/4, 1]

we have

E[ Φ(X(t + 1))|X(t) ] ≤ 1 - εαλ 4 • Φ(X(t)) + ε -8 • O(n).
(5.39)

This follows via a case analysis analogously to [START_REF] Talwar | Balanced Allocations: A Simple Proof for the Heavily Loaded Case. English[END_REF]:

Case 1: x n 4 ≥ ∅ and x 3n 4 ≤ ∅
The bound follows from Lemma 5.16 and Lemma 5.17.

Case 2:

x n 4 ≥ x 3n 4 > ∅ For E[ ∆ + (t + 1)|X(t) ] ≤ -εαλ 4 • Φ + the results follows from Lemma 5.17. Otherwise, E[ ∆ + (t + 1)|X(t) ] > -εαλ
4 • Φ + and Lemma 5.18 yields two subcases:

Case 2.1: Φ + (X(t)) ≤ ε 4 • Φ -(X(t))
Using Lemma 5.15 and Lemma 5.17 we obtain

E[ ∆(t + 1)|X(t) ] ≤ ≤ 2αλ • Φ + (X(t)) -εαλ • Φ -(X(t)) + αλn 2 ≤ - εαλ 2 • Φ -(X(t)) + αλn 2 ≤ - εαλ 4 • Φ(X(t)) + ε -8 • O(n).
(5.40)

Case 2.2: Φ(X(t)) = ε -8 • O(n) Using Lemma 5.15 we get E[ ∆(t + 1)|X(t) ] ≤ 2αλε -8 • O(n).
Our choice of α (< 1/8), λ (< 1), and ε ( 1) yields 2αλ ≤ (1 -εαλ/4). Using the case assumption, we compute

E[ ∆(t + 1)|X(t) ] ≤ 2αλε -8 • O(n) ≤ 1 - εαλ 4 • ε -8 • O(n) ≤ - εαλ 4 • Φ(X(t)) + ε -8 • O(n).
(5.41)

Case 3: x 3n 4 ≤ x n 4 ≤ ∅ Similar to the previous case, for E[ ∆ -(t + 1)|X(t) ] ≤ -εαn 4 • Φ -the result follows from Lemma 5.16. For E[ ∆ -(t + 1)|X(t) ] ≥ -εαn 4
• Φ -, Lemma 5.19 yields two subcases that are proven analogously to Cases 2.1 and 2.2 (using Lemma 5.16 instead of Lemma 5.17).

Thus, all cases lead to (5.39).

Maximum Load -Proof of Theorem 5.6

The goal of this section is to prove Theorem 5.6. Recall the definitions of Φ(x) and Ψ(x) from (5.14). For any fixed round t, we will prove that (w.h.p.) Ψ(X(t)) = O(n • ln n) and that the average load is ∅ = O(ln n). Using Union bounds and Proposition 5.7, we see that (w.h.p.) the maximum load at the end of round t is bounded by

∅ + O(ln n) = O(ln n).
It remains to prove a high probability bound on Ψ(X(t)) for arbitrary t. To get an intuition for our analysis, consider the toy case t = poly(n) and assume that exactly λ•n ≤ n balls are thrown each round. Here, we can combine Observation 5.8 and Lemma 5.9 to bound (w.h.p.) the load difference between any pair of bins and for all t < t by O(ln n) (via a union bound over poly(n) rounds). Given this bound on load difference, we can use the following combinatorial observation (formally stated in Lemma 5.20). If the load distance to the average is bounded by some b ≥ 0, the bound on the number of balls Ψ ≤ 2b • n is invariant under the 2-Choice process, since under our assumptions all bins are non-empty time t bound on load difference (Φ(t)) min load min i {X i (t)} t T

Figure 5.2: To bound the system load at time t, consider the minimum load and our bound on the load difference over time. We consider the last time T when there was an empty bin. The system load can only increase if there is an empty bin, and this increase is bounded by our bound on the load difference. Using that the system load decreases linearly in time while every increase is bounded by our logarithmic bound on the load difference, we find a small interval [t , t] containing T . Due to the monotonic of our bound on Ψ, this will allow us to derive strong bounds on Ψ(t) and on the maximum load.

and thus at least as many balls are deleted as spawn. In particular, we get for b

= O(ln n) that Ψ(X(t)) ≤ 2b • n = O(n • ln n), as required.
The case t = ω(poly(n)) is considerably more involved. In particular, the fact that the number of balls in the system is only guaranteed to decrease when the total load is high and the load distance to the average is low makes it challenging to design a suitable potential function that drops fast enough when it is high. Thus, we deviate from this standard technique and elaborate on the idea of the toy case: Instead of bounding (w.h.p.) the load difference between any pair of bins by O(ln n) for all t < t (which is not possible for t poly(n)), we prove (w.h.p.) an adaptive bound of O(ln(t -t ) • f (λ)) for all t < t, where f is a suitable function (Lemma 5.21 and Lemma 5.22). Then we consider the last round T < t with an empty bin. Observation 5.8 yields a bound of Ψ(X(T )

) = 2 • O(ln(t -T ) • f (λ)) • n
on the total load at time T . Using the same combinatorial observation as in the toy case, we get that (w.h.p.) Ψ(X(t) Proof. We distinguish two cases: if there is no empty bin, then all n bins delete one ball.

) ≤ Ψ(X(T )) = 2 • O(ln(t -T ) • f (λ)) • n.
Since the maximum number of new balls is n, the number of balls cannot increase. That is, we have Ψ(x ) ≤ Ψ(x) ≤ 2b • n. Now consider the case that there is at least one empty bin. Let η ∈ (0, 1] denote the fraction of empty bins (i.e., there are exactly η • n > 0 empty bins). Since the minimal load is zero, Observation 5.8 implies max i x i ≤ 2b. Thus, the total number of balls in configuration x is at most (1 -η)n • 2b. Exactly (1 -η)n balls are deleted (one from each non-empty bin) and at most n new balls enter the system. We get

Ψ(x ) ≤ (1 -η)n • 2b -(1 -η)n + n = (1 -η)n • (2b -1) + n ≤ 2b • n.
The next lemma bounds the probability of two events: First, it bounds Φ over an arbitrary time interval [0, t) using a union bound over all past rounds t < t. Note that t can be arbitrary large. Thus, in order to get a high probability bound, we must make the bound adaptive and allow for larger errors the further back in time we go. Second, the lemma shows that (w.h.p.) not too many balls are created.

Lemma 5.21.

Let λ ∈ [1/4, 1). Fix a round t. For i ∈ N with t -i • 8 log n 1-λ ≥ 0 define I i := [t -i • 8 ln n 1-λ , t]. Let Y i be the number of balls which spawn in I i . 1. Define the (good) smooth event S t := t <t { Φ(X(t )) ≤ |t -t | 2 • n 2 }. Then, P[ S t ] = 1 -O n -1 .

Define the (good) bounded balls event B

t := i { Y i ≤ 1+λ 2 • |I i | • n }. Then, P[ B t ] = 1 -O n -1 .
Proof. Consider an arbitrary time t < t. By Lemma 5.9 we have E[ Φ(t ) ] ≤ n/ε. Using Markov's inequality, this implies

P Φ(t ) ≥ |t -t | 2 • n 2 ≤ 1/(ε • |t -t | 2 • n).
(5.42)

Using the union bound over all t < t we calculate

P St ≤ t <t P Φ(t ) ≥ |t -t | 2 • n 2 ≤ 1 εn • t <t 1 |t -t | 2 ≤ π 2 6ε • n = O n -1 ,
where the last inequality uses the solution to the Basel problem. This proves the first statement.

For the second statement, let Z i := |I i |•n-Y i be the number of balls that did not spawn

during I i . Note that Z i is a sum of |I i | • n independent indicator variables with E[ Z i ] = (1 -λ) • |I i | • n = 8i • ln n. Chernoff yields P[ Z i ≤ (1 -λ) • |I i | • n/2 ] ≤ e -8i•ln n/8 = n -i .
The desired statement follows from applying the identity Lemma 5.22. Fix a round t and assume that both S t and B t hold. Then,

Z i = |I i | • n -Y i
Ψ(X(t)) ≤ 9n α • ln n 1 -λ .
(5.43)

Proof. Let T < t be the last time when there was an empty bin and set ∆ := t -T . Note that T is well defined, as we have X i (0) = 0 for all i ∈ [n]. Since S t holds, we have

Φ(X(T )) ≤ ∆ 2 • n 2 = exp ln(∆ 2 • n 2 ) .
(5.44)

By definition of T , we have min i X i (T ) = 0. Together with Observation 5.8 we get

max i X i (T ) ≤ 2 ln ∆ 2 • n 2 /α.
(5.45)

Summing up over all bins (and pulling out the square), this implies that Ψ(X(T ))

≤ 4n • ln ∆ • n /α. Applying Lemma 5.20 yields Ψ(X(T + 1)) ≤ 4n • ln ∆ • n /α. (5.46)
By the definition of T , is must be the case that there is no empty bin in X(t ) for all t ∈ { T + 1, T + 2, . . . , t -1 }. Thus, during each of these rounds exactly n balls are deleted. To bound the number of deleted balls, let i be maximal with I i ⊆ [T, t] (as defined in Lemma 5.21). Recall that

I i = [t -i • 8 ln n 1-λ , t]
. Since B t holds and using the maximality of i, the number of balls Y that spawn during [T, t] is bounded by

(1 + λ)|I i | • n/2 + 8 ln n 1 -λ • n ≤ (1 + λ)∆ • n/2 + 8 ln n 1 -λ • n. (5.47) We calculate Ψ(X(t)) ≤ Ψ(X(T + 1)) -∆ • n + Y ≤ 4n α ln(∆ • n) - 1 -λ 2 ∆ • n + 8 ln n 1 -λ • n = 1 -λ 2 • n • 8 α(1 -λ) • ln(∆ • n) -∆ + 16 ln n (1 -λ) 2 ≤ 1 -λ 2 • ∆ • n • 24 α(1 -λ) 2 • ln(∆ • n) ∆ -1 .
(5.48)

With f = f (λ) := 24/ α(1 -λ) 2 the last factor becomes f • ln(∆ • n)/∆ -1. It is negative if and only if ∆ > f • ln(∆ • n). This inequality holds for any ∆ > -f • W -1 (-1 f •n )
, where W -1 denotes the lower branch of the Lambert W function5 . This implies that ∆ ≤ -f •

W -1 (-1 f •n ), since otherwise we would have Ψ(X(t)) < 0, which is clearly a contradiction. Using the Taylor approximation W -1 (x) = ln(-x) -ln ln(-1/x) -o(1) as x → -0, we get ∆ ≤ -f • W -1 - 1 f • n ≤ f • ln(f • n) + f • ln ln(f • n) + f ≤ 2f • ln(f • n).
(5.49)

Finally, we use this bound on ∆ to get

Ψ(X(t)) ≤ Ψ(X(T + 1)) ≤ 4n α • ln(∆ • n) ≤ 4n α • ln 2f n • ln(f n) ≤ 4n α • ln 48n α(1 -λ) 2 • ln 24n α(1 -λ) 2 ≤ 9n α • ln n 1 -λ .
(5.50) Now, by combining Lemma 5.22 with the fact that the events S t and B t hold with high probability (Lemma 5.21), we immediately get that (w.h.p.) Ψ(X(t)) = O(n • ln n).

As described at the beginning of this section, combining this with Proposition 5.7 proves Theorem 5.6.

Stability -Proof of Theorem 5.5

This section proves Theorem 5.5. In order to do so, we consider the potential Γ (defined in (5.14)) and show that, for a sufficiently high value of, this potential decreases (Lemma 5.23).6

To show this drop, we argue along the following lines. For the potential to be large and since the potential is the sum of two potentials Φ and Ψ, one of must have size at least Γ(x)/2.

If Φ(x) is large, then we can even assume a worst-case increase of Ψ and invoke (5.39) to show that Φ drops considerably resulting in an overall potential drop of Γ. Similarly, if Ψ(x) ≥ Γ(x)/2, then, due to the careful construction of Γ, we can show that all bins are non-empty, and the overall potential decreases in expectation. This overall potential decrease of Γ allows to apply Theorem A.22 yielding stability.

Lemma 5.23

(Negative Bias Γ). Let λ ∈ [1/4, 1). If Γ(X(t)) ≥ 2 n 4 (1-λ) 2 λ , then E[ Γ(X(t + 1)) -Γ(X(t))|X(t) ] ≤ -1.
(5.51)

Proof. Assume X(t) = x is fixed. By definition of Γ(•), we have Φ(x) ≥ Γ(x)/2 or Ψ(x) ≥ Γ(x)/2. We now show that in both cases

E[ Γ(X(t + 1)) -Γ(x)|X(t) = x ] ≤ -1.
(5.52)

1. If Φ(x) ≥ Γ(x)/2, then we have, by (5.39), a potential drop of

E[ Φ(X(t + 1)) -Φ(x)|X(t) = x ] ≤ -(εαλ/4) • Φ(x) + n log n ≤ -(εαλ/8) • Γ(x) + n log n.
(5.53)

Note that, by definition of Ψ, Ψ(X(t + 1)) -Ψ(x) ≤ n. Together with Γ(x) ≥

8(n log n+n 2 /(1-λ)+1) eαλ , E[ Γ(X(t + 1)) -Γ(x)|X(t) = x ] ≤ - εαλ 8 Γ(x) + n log n + (n/(1 -λ)) • n ≤ -1.
(5.54) 2. Otherwise, i.e., if Φ(x) < Γ(x)/2, we have that (i) the load difference is, by Observation 5.8, bounded by 2 ln(Γ(x)/2)/α, and

(ii) Ψ(x) ≥ Γ(x)/2 must hold. This implies that ∅ ≥ 1 n Γ(x)/2 n 1-λ = (1-λ)•Γ(x) 2n 2 .
From (i) and (ii) we have that the minimum load is at least

(1-λ)•Γ(x) 2n 2
ln(Γ(x)/2)/α. From Lemma 5.24 and Γ(x) ≥ 2 n 4

(1-λ) 2 λ , it follows that every bin has load at least load 1. Thus each bin will delete one ball and the number of balls arriving is λn in expectation. Hence,

E[ Ψ(X(t + 1)) -Ψ(x)|X(t) = x ] = - n 1 -λ (1 -λ)n.
(5.55)

Now,

E[ Γ(X(t + 1)) -Γ(x)|X(t) = x ] = E[ Φ(X(t + 1)) -Φ(x)|X(t) = x ] - n 1 -λ (1 -λ)n ≤ n log n - n 1 -λ (1 -λ)n ≤ -1.
(5.56)

Thus, E[ Γ(X(t + 1)) -Γ(x)|X(t) = x ] ≤ -1, which yields the claim.

We now proceed with a technical result.

Lemma 5.24. For all

x ≥ 2 n 4 (1-λ) 2 λ it holds that (1-λ)•x 2n 2 -2 ln(x/2)/α ≥ 1. Proof. Define f (x) = (1-λ)•x 2n 2 -2 ln(x/2)/α. We have f 2 n 4 (1-λ) 2 λ ≥ n 2 (1-λ)λ -2 α ln n 4
(1-λ) 2 λ ≥ 1, where the last inequality holds for large enough of n since α is a constant. Moreover, for all x ≥ 2 n 4

(1-λ) 2 λ we have f (x) = 1-λ n 2 -2 αx ≥ 0. Thus, the claim follows.

We are ready to prove Theorem 5.5.

Proof of Theorem 5.5. The proof proceeds by applying Theorem A.22. We now define the parameters of Theorem A.22. Let ζ(t) = X(t) and hence Ω is the state space of X. First we observe that Ω is countable since there are a constant number of bins (n is consider a constant in this matter) each having a load which is a natural number. We define φ(X(t))

to be Γ(X(t)). We define

C = { x | Γ(x) ≤ 2 n 4
(1-λ) 2 λ }. Define β(x) = 1 and η = 1. We now show that the preconditions (a) and (b) of Theorem A.22 are fulfilled.

• Let x ∈ C. By definition of C and φ(X(t)), and from Lemma 5.23 we have

E[ φ(X(t + 1)) -φ(x)|X(t) = x ] ≤ E[ Γ(X(t + 1)) -Γ(x)|X(t) = x ] ≤ -1.
(5.57)

• Let x ∈ C. Recall that Γ(X(t)) = Φ(X(t)) + Ψ(X(t)). By Lemma 5.19 and the fact that the number of balls arriving in one round is bounded by n, we derive,

E[ φ(X(t + 1))|X(t) = x ] = = E[ Φ(X(t + 1))|X(t) = x ] + E[ Ψ(X(t + 1))|X(t) = x ] ≤ 1 - εαλ 4 2 n 4 (1-λ) 2 λ + n 1 -λ n < ∞.
(5.58)

The claim follows by applying Theorem A.22 with (5.57) and (5.58).

Chapter 6

Future Work -Dynamic Processes

Apart from the variants of the Forest Fire Process discussed in Section 4.7 we suspect that there are many social networks and other dynamic processes that can be studied by using the potential approach of Chapter 3 and similar ideas and methods as we used for the study of Forest Fire Process and Balls-into-Bins with Deletions.

It would be interesting to develop some further notion of what it means that a process is self-stabilizing in the sense that whenever the process is in a "bad" state (e. g., the distance is super-constant), then it quickly recovers. This notion of self-stabilization is different than the related the notion of positive recurrence of the underlying Markov chain we use in Chapter 5 since we would like the notion to include the distance in the Forest Fire Process-however, the underlying state space is growing since the graph is growing and the underlying Markov chain is not positive recurrent. Nevertheless, as our analysis of the Forest Fire Process shows, whenever the potential maximizing the distance is large, then it decreases in expectation. It would be interesting to identify necessary properties of such self-stabilizing processes-what's the common thread about self-stabilizing processes?

Another interesting research direction consists of deriving lower bounds complementing Theorem A.11 and to understand how tight the results of Theorem A.11 are-can the first condition (the bound on all moments), be relaxed?

One of the the technically most challenging task consists of deriving tools to deal with potentials that change in more convoluted ways: We studied the Forest Fire Process for the case without backward burning since with backward burning the φ of the node arrive at time t changes by nodes arriving after time t. We are not aware of any tools that would permit the analysis of such a potential and we believe that such tools would allow us to study much more involved processes-rendering the potential analysis of dynamic processes even more powerful.

Chapter 7

Contributions Consensus Processes

Since reaching consensus is impossible for deterministic distributed algorithm in many settings of interest (e. g., [START_REF] Angluin | Local and Global Properties in Networks of Processors (Extended Abstract)[END_REF][START_REF] Lynch | A Hundred Impossibility Proofs for Distributed Computing[END_REF]), we focus on randomized consensus processes executing simple algorithms. The system consists of n anonymous nodes connected by edges of a graph. Initially, each node supports one opinion from the set [k] := { 1, . . . , k }. We refer to these opinions as colors. We describe the system state after any round by an n-

dimensional integral vector c = (c i ) i∈[n] ∈ N n 0 with i∈[n] c i = n.
Here, the i-th component c i ∈ N 0 corresponds to the number of nodes supporting opinion i. If k < n, then c i = 0 for all i ∈ { k + 1, k + 2, . . . , n }. A consensus process is specified by an update rule that is executed by each node. The goal is to reach a state in which all nodes support the same color; the special case where nodes start with pairwise distinct colors is leader election, an important primitive in distributed computing. The quantity of interest is the consensus time which is the expected time of a consensus process to reach consensus. In some cases we are also interested in plurality consensus in which the goal is that all nodes agree on the initially most dominant color.

We assume a severely restricted and simple communication mechanism known as Uniform Pull [DGH+87, KSSV00, KDG03]. Here, in each discrete round, nodes independently pull information from some (typically constant) number of randomly sampled nodes. We are interested in protocols with low memory and low message size.

We distinguish between Consensus Dynamics and Consensus Protocols: Consensus Dynamics are a subset of Consensus Protocols in which the message size and memory is bounded by O(log k)-the memory is enough to share a constant number of opinions but no more. Typically, consensus dynamics are simple and memoryless. Consensus protocols on the other side can be more complex: they are trading off additional memory and larger message sizes against faster consensus or better guarantees on the plurality consensus (i. e., they require a smaller initial bias to ensure that plurality color wins).

Consensus Dynamics

We start by giving an overview over the most studied protocols in this class. The Voter process (also known as Polling) uses the most naïve update rule: In every round, with probability 1/2, each node samples one neighbor independently and uniformly at random, and it adopts that node's color. Otherwise, the node keeps its current color. We will also consider the non-lazy version of the model, where every node samples one neighbor in every round.

Two further natural and prominent consensus processes are the 2-Choices and the 3-Majority processes. Their corresponding update rules, executed synchronously by every node, are as follows:

• 2-Choices: Sample two neighboring nodes independently and uniformly at random.

If the samples have the same color, adopt it. Otherwise, ignore them and keep the current color.

• 3-Majority: Sample three neighboring nodes independently and uniformly at random. If a color is supported by at least two samples, adopt it. Otherwise, adopt the color of one of them at random1 . See Figure 7.1 for an illustration. In the following we give an overview of our results in these models.

Overview of Results

We start by considering the Voter model on general undirected graphs.

Results

Voter model. The Voter is the dual2 of the so-called coalescing random walk model, which is a fundamental stochastic process on connected and undirected graphs: At the beginning of the process there is one particle on each node in the graph. At discrete time steps, every particle performs independently one step of a random walk.3 Whenever two or more particles arrive at the same node at the same time step, they merge into a single particle and continue as a single random walk. We are interested in the (expected) first time step when only one particle remains, to which we refer to as coalescence time.

The aforementioned duality (see [START_REF] Hassin | Distributed Probabilistic Polling and Applications to Proportionate Agreement[END_REF][START_REF] Aldous | Reversible Markov Chains and Random Walks on Graphs[END_REF] and Proposition 9.11 (more rigorously))

states that the Voter process viewed backwards is exactly the same as the coalescence process starting with a random walk on every node; thus, the coalescence time t coal and consensus time t V have the same distribution. In other words, a bound on the coalescence time yields a bound on consensus time and vice versa.

When starting with two particles, the coalescence time is referred to as the meeting time: we denote by t meet the worst-case expected meeting time over all pairs of starting nodes and let t coal denote the expected coalescence time starting from one particle on every node. Surprisingly, little is known about the relationship between t meet and t coal .

Question 1: What are the bounds for the expected consensus time t coal in terms of the meeting time t meet ? Question 2: For which graphs do we have t coal = Θ(t meet )?

The first question can partially be answered by

t coal ∈ [t meet , O(t coal log n)]. (7.1)
The lower bound follows my means of a coupling argument stating that it takes for all n walks at least as long to meet as for 2 random walks. The upper bound follows by dividing time into periods of length 2t meet : Say we are left with k random walks, then any pair of random walks meets after 2t meet time steps w.p. at least 1/2, by Markov inequality.

Repeating this argument shows that after O(log n) periods only one walk prevails (see Proposition 8.6).

In Chapter 8, we generalize (7.1) and give at the same time sufficient conditions in answer to the second question. More precisely, we relate t coal to the ratio of the mixing time t mix and t meet : Whenever t meet is marginally larger than t mix (a factor of log 2 n), then t coal = Θ(t meet ). This result as well as (7.1) are special cases of our more general bound covering the entire spectrum t meet /t mix ∈ [1, log 2 n]. We show that (Theorem 8.1), using

the duality t V = t coal t V = t coal = O t meet 1 + t mix t meet • log n .
We complement this by giving a matching lower bound (Theorem 8.2).

While the Voter model is very natural and useful in many settings-due to its simplicity, the Voter model has two downsides: 1) the consensus time on the clique is fairly large Θ(n)

and 2) the probability for an opinion to win is proportional to the sum of the degrees of nodes of that color ( [START_REF] Hassin | Distributed Probabilistic Polling and Applications to Proportionate Agreement[END_REF]) and thus even with a large bias the second most dominant color can easily win, which is in certain settings intolerable. To circumvent both issues two other simple dynamics were proposed: 3-Majority and 2-Choices.

Results 3-Majority. All known results for 3-Majority (and 2-Choices) apply only if the number of opinions is small thus not solving the leader election problem. Most results require-in addition to a limited number of opinions-a sufficient bias between the largest and second largest color. The reason for the lack of more general results is the following. While it it easy to apply first moment analysis, it seems challenging to analyze the true behavior of the process, due to the considerable variance. Intuitively, in the difficult regime where the number of opinions exceeds n 2/3 , the process behaves similarly to the Voter in the sense that majorities shift easily, rendering many standard approaches futile. We embrace this resemblance and show a stochastic majorization between the processes (Proposition 9.10) which allows us to analyze the challenging regime by reducing to the easy-to-analyze Voter model. By doing so we obtain the first unconditional results on the clique: We show (Theorem 9.8) that the expected consensus time is bounded by

t 3M = O n 3/4 log 7/8 n .
This stochastic majorization is a consequence of our general result (Theorem 9.4) whose essence is the following. We define a potential φ measuring the progress towards consensus, and a family of memoryless processes (AC-processes defined in Definition 9.2) comprising 3-Majority and the non-lazy voter model. For any two process P, P of this family we show that P reaches consensus faster than P , i. e., T P ≤ st T P if the following holds.

t 2C = O(n/c 1 • log n)
rounds even in presence of an adversary which can change up to c 1 (c 1 -c 2 )/n nodes per round, where c i is the size of the i'th largest opinion. Furthermore, we prove a series of lower bounds which show that leader election takes Ω(n/ log n) rounds (Theorem 10.5), and that plurality consensus is likely to fail if the initial bias is smaller (w.r.t. the bias of our upper bound) by a factor of √ log n (Theorem 10.6).

At first glance 2-Choices and 3-Majority appear to very similar: Under the first moment method both processes behave identically meaning that when started from the same configuration the expected sized of all colors after one round are identical. Nevertheless, we show stark differences between processes resulting in an interesting trade-off: 2-Choices trades a (considerably) worst-case slower consensus time for better guarantees on the plurality color. These strong plurality guarantees prove to be very useful in the design of consensus protocols as we show by using 2-Choices as the core of our fast protocol RapidPlurali-tyConsensus.

Consensus Protocols

The goal of our consensus protocols is not only to reach consensus more efficiently, but also to relax the synchronicity requirement, i. e.,, the requirement that all nodes perform synchronous rounds. Instead, we would like our algorithm to work even in settings such the asynchronous model, sequential model.

Overview of Results

We consider three protocols RapidPluralityConsensus as well as two protocols inspired by load balancing: Balance and Shuffle.

Results

RapidPluralityConsensus.

The key to our fast plurality protocol is the combination of the 2-Choices process with an information dissemination process. We divide time into phases: A 2-Choices sub-phase and a dissemination sub-phase. The first sub-phase is just one round of 2-Choices and every node that changed its color propagates the adopted color in the second sub-phase via a process which is essentially pull rumor spreading with multiple rumors (representing the adopted colors).

While this process is not too difficult to analyze in the synchronous model it becomes very challenging in the asynchronous (continuous) setting or sequential settings in which the nodes are selected u.a.r. to perform a tick (see Chapter 11). The reason for the arising difficulties is that the aforementioned protocol relies heavily on simultaneously execution of the phases which is no longer the case in the asynchronous realm.

Nevertheless, we are able to adapt the aforementioned protocol to obtain (Theorem 11.1) a plurality consensus time5 of

t async = O(log n).
Incidentally, this is the best possible consensus time since some nodes will not have even ticked once after Ω(log n) and have therefore not even queried a single. To make our algorithm work, we develop the following weaker notion of synchronicity. At any time we only require a (1 -o(1)) fraction of the nodes to be almost synchronous. This relaxes full synchronicity in three ways: First, nodes are only "almost synchronous", meaning that for any two nodes their clocks (adjusted over time) may differ by up to ∆ = Θ(log n/ log log n).

Secondly, we allow o(n) nodes to be poorly synchronized. Finally, we require this to hold only with high probability.

The above notion does not require the nodes to synchronize actively per se, since their number of ticks is to some extent concentrated even without active synchronization. However, it turns out that without synchronizing perpetually, the number of poorly synchronized nodes in each phase will become larger than the initial bias (we are interested in) towards the plurality opinion c 1 -c 2 and could therefore influence the consensus significantly. We thus synchronize actively (see Algorithm 10) nodes at the end of each phase to decrease the fraction of poorly synchronized nodes such that their number is in o(c 1 -c 2 ), resulting in a negligible influence of those nodes.

Results Consensus via Load Balancing.

To the best of our knowledge, no effort has been made to ensure plurality consensus in arbitrary undirected graphs. We develop two protocols Balance and Shuffle which are heavily inspired by distributed load balancing.

In the spirit of generalization, our protocols work in a multitude of different environments:

Asynchronous, sequential, synchronous, and random matchings. The beauty of our protocols is that they're transparent to the underlying communication environment and use the results from load balancing as a black box.

Protocol Balance (Algorithm 13) uses load balancing in the most natural way possible:

Each node creates a polynomial number of balls of its own color and then balances the balls of all colors separately for

t Balance = O log n 1 -λ 2
rounds, where λ 2 is the second largest eigenvalue. Afterwards, each node simply chooses the color of which it has the most number of balls. This will with high probability be the plurality opinion (Theorem 12.9). While this is very efficient in the settings with few colors (small k), the required memory per node is Θ(k log n). Reducing this memory is the goal of our algorithm Shuffle (Algorithm 12), which essentially performs "blind" load-balancing as opposed to color separated load balancing as done by Balance: In the "diffusion setting"

where all nodes can communicate to all of the neighbors in every round the algorithm essential just chooses half of its tokens and randomly sends them in equal shares to each of the neighbors. Our results (Theorem 12.9) show a trade-off between memory usage and consensus time of Shuffle. The challenge in analyzing Shuffle is the dependencies between the tokens: The number of tokens per node is invariant and thus if we reveal the positions of some tokens, we affect the probability distribution of the remaining tokens.

Fortunately, it turns out that the the dependencies work in the right direction: the tokens are negatively associated (Lemma 12.6) allowing us to apply concentration inequalities.

Related Work

Voter Model

4 n + π -2 2 ) • (1 -λ 2 ) -1 )
, where λ 2 is the second largest eigenvalue of the transition matrix of the random walk and π is the stationary distribution. Berenbrink et al. [START_REF] Berenbrink | Bounds on the Voter Model in Dynamic Networks[END_REF] show that and to the best of our knowledge, there is no general way in which results in continuous time can be transferred to discrete time or vice versa, even when the random walks in discretetime are lazy. In the continuous time setting, Cox [START_REF] Cox | Coalescing Random Walks and Voter Model Consensus Times on the Torus in Z d[END_REF] show that the coalescence time is bounded by Θ(t hit ) for tori. Oliveira [START_REF] Oliveira | On the coalescence time of reversible random walks[END_REF] showed that the coalescence time is O(t hit ) in general. In a different work, Oliveira [START_REF] Oliveira | Mean field conditions for coalescing random walks[END_REF] proved so-called mean field conditions, which are sufficient conditions for the coalescing process on a graph to behave similarly to that on a complete graph. His main result in [START_REF] Oliveira | Mean field conditions for coalescing random walks[END_REF], Theorem 1.2, implies that

t coal = O(m/(d min • Φ)),
t coal = O(t meet ) whenever t mix • π max = O(1/ log 4 n).

2-Choices

The expected convergence time of the Voter process is at least Ω(n) on many graphs, such as regular expanders and complete graphs. Taking into account that solutions to many other fundamental problems in distributed computing, such as information dissemination [START_REF] Karp | Randomized rumor spreading[END_REF] or aggregate computation [START_REF] Kempe | Gossip-Based Computation of Aggregate Information[END_REF], are known to run much more efficiently, Cooper et al. noted that there is room for improvement. To address this issue, Cooper et al. [START_REF] Cooper | The Power of Two Choices in Distributed Voting[END_REF] introduced the 2-Choices voting process. In this modified process, one is given a graph G = (V, E) where each node has one of two possible opinions. The process runs in discrete rounds during which, unlike in the Voter process, every node is allowed to contact two neighbors chosen uniformly at random. As mentioned earlier, if both neighbors have the same opinion, then this opinion is adopted, otherwise the calling vertex retains its current opinion in this round. , the results from [START_REF] Cooper | The Power of Two Choices in Distributed Voting[END_REF] have been extended to general expander graphs, cutting out the restrictions on the node degrees but nevertheless proving that the convergence time remains in O(log n). Recently, the authors of [START_REF] Cooper | Fast plurality consensus in regular expanders[END_REF] showed the following bound on the consensus time in regular expanders. If the initial bias between the largest and second-largest opinion is at least c 1 -c 2 ≥ Cn max{ log n/c 1 , λ 2 }, where λ 2 is the second largest eigenvalue and C is a suitable constant, then the largest opinion wins in O((n log n)/c 1 ) steps, with high probability. Very recently Ghaffari and Lengler [START_REF] Ghaffari | Tight Analysis for the 3-Majority Consensus Dynamics[END_REF] show for 2-Choices that for k = O( √ n log n) the consensus time is O(k log n) which is known to be tight. The authors suggest that their results extend to 3-Majority.

3-Majority

All theoretical results for 3-Majority consider the complete graph. The authors of [START_REF] Becchetti | Trevisan: Simple Dynamics for Plurality Consensus[END_REF] assume that the bias is Ω min { √ 2k, (n/ log n) 1/6 }• √ n log n . Under this assumption, they prove that consensus is reached with high probability in O min { k, (n/ log n) 1/3 } • log n rounds, and that this is tight if k ≤ (n/ log n) 1/4 . The only result without bias [START_REF] Becchetti | Stabilizing Consensus with Many Opinions[END_REF] restricts the number of initial colors to k = o n 1/3 . Under this assumption, they prove that 3-Majority reaches consensus with high probability in

O (k 2 (log n) 1/2 + k log n) • (k + log n)
rounds. Their analysis considers phases of length O k 2 log n and shows that, at the end of each phase, one of the initial colors disappears with high probability. Note that this approach is so far the only one not assuming any bias cannot yield sublinear bounds with respect to k.

Further Consensus Dynamics

A related consensus process is 2-Median [START_REF] Doerr | Stabilizing consensus with the power of two choices[END_REF]. Here, every node updates its color (a numerical value) to the median of its own value and two randomly sampled nodes. Without assuming any initial bias, the authors show that this process reaches consensus with high probability in O(log k • log log n + log n) rounds. This is seemingly stronger than the bounds achieved for 3-Majority and 2-Choices without bias. However, it comes at the price of a complete order on the colors (our processes require colors only to be testable for identity).

Consensus Protocols

There is a diverse body of literature that analyzes consensus problems under various models and assumptions. Results differ in the considered network topology (e.g., arbitrary or complete), the restrictions on model parameters (e.g., the number of opinions or the initial bias7 ), the time model (synchronous or asynchronous), or the required knowledge (e.g., n, maximal degree, or spanning tree). To capture this diverse spectrum, we classify8 results into population protocols and sensor networks. We will not discuss work whose focus is too far away from this paper's, e.g., consensus on some arbitrary opinion, leader election, robustness concerns, or Byzantine models.

Population Protocols. The first area of work we consider comes from population protocols. Population protocols model interactions between large populations of very simple entities (like molecules). Entities are modeled as finite state machines with a small state space and communicate asynchronously. In each step, an edge is chosen uniformly at ran- arbitrary arbitrary

T • t mix T • log n/(1 -λ 2 ) for (d-regular graph) sync & async see Theo- rem 12.1 Bala.
arbitrary arbitrary

τ log n/(1 -λ 2 ) for (d-regular graph) sync & async k • log n [KLS08] arbitrary arbitrary D + F 2 n 2 1 • log(k) broadcast - [MNRS14] 2 arbitrary n 5 async 1 [DV12] 2 arbitrary log n/δ(G, n 1 /n) async 1 [CER+15] expander 2 vol(1) -vol(2) ≥ 4λ 2 2 |E| log n sync 1 [CER14] random d-reg 2 1/d + d/n log n sync 1 [BCN+14a] ≤ n min k, 3 n log n • log n n min k, 3 n log n • log n sync log(k) [BCN+15b] O( 3 n log n ) ε • n 2 /n md(c) • log n sync log(k) [EFK+16] O(n ) log n/n k + log n sync log(k) [BFGK16] o( n log n ) log n/n log n • log log n sync log(k) [AAG17] 2 arbitrary O(log 2 (n)) async s = O(log n) states [AGV15] 2 arbitrary log 2 (n) sα + log 2 (n) async s = O(n) states [AAG17] 2 arbitrary O(log 2 n) async O(log n) states [GS17] 2 arbitrary O(log n) async O(log (2) n) states [AAE08] 2 log n/ √ n log n async 1
Shuffle assumes rough bounds on t mix and n. Bounds on α can reduce the space requirements of our protocols.

[KLS08] requires a spanning tree and a common set of quasi-random hash functions. In contrast to these results, our protocols consider the case of arbitrary number k ≥ 2 of opinions. Also, with the notable exception of [START_REF] Mertzios | Determining Majority in Networks with Local Interactions and Very Small Local Memory[END_REF], the above results are restricted to the complete graph. These restrictions are not surprising, given that these protocols operate on a very constrained state space. Moreover, [START_REF] Gasieniec | Fast Space Optimal Leader Election in Population Protocols[END_REF] provided a protocol for leader election which uses O(log log n) states and reaches consensus in O(log 2 n) rounds. Very recently [START_REF] Alistarh | Space-Optimal Majority in Population Protocols[END_REF] showed that for a protocol converging to the majority opinion in O(n c ), c ≤ 1 time steps regardless of the initial configuration, Ω(log n) states are necessary. Moreover, they give a protocol requiring O log 2 (n) time steps to converge and O(log n) states.

Sensor Networks. Another line of work has a background in sensor networks. Quantized interval consensus draws its motivation from signal processing. Initially, nodes measure quantized values (signals) and then communicate through a network to agree on the quantized values that enclose the average. This can be used to solve majority consensus (k = 2).

The communication model is typically sequential.

Bénézit et al. [BTV09]

propose a protocol that is equivalent to the 4-state population protocol of [START_REF] Mertzios | Determining Majority in Networks with Local Interactions and Very Small Local Memory[END_REF] and prove that with probability 1 it converges in finite time, but without bounds on that convergence time.

A more recent result by Draief and Vojnovic [START_REF] Draief | Convergence Speed of Binary Interval Consensus[END_REF] shows that this protocol (and thus [START_REF] Mertzios | Determining Majority in Networks with Local Interactions and Very Small Local Memory[END_REF]) needs O log n δ(Q S ,α) steps in expectation. Here, δ(Q S , α) depends on the bias α and on the spectrum of a set of matrices Q S related to the underlying graph. The authors give concrete bounds for several specific graphs (e.g., in the complete graph the consensus time is of order11 O(log n/α)). The only related result for k > 2 we are aware of is [START_REF] Bénézit | The Distributed Multiple Voting Problem[END_REF] which again proves only convergence in finite time.

Another consensus variant is mode computation. For example, Kuhn et al. [START_REF] Kuhn | Distributed computation of the mode[END_REF] consider a graph of diameter D where each node has one or several of k distinct elements.

The authors use a protocol based on a complex hashing scheme to compute the mode (the

most frequent element) w.h.p. in time O D + F 2 /n 2 1 • log k . Here, F 2 = i n 2
i is the second frequency moment and n i the frequency of the i-th most common element. F 2 /n 2 1 ∈ [1, k] can be seen as an alternative bias measure. Nodes communicate via synchronous broadcasts and need a precomputed spanning tree and hash functions. [START_REF] Kuhn | Distributed computation of the mode[END_REF] can also be used for aggregate computation as done by Kempe et al. [START_REF] Kempe | Gossip-Based Computation of Aggregate Information[END_REF] (where the authors provide an elegant protocol to compute sums or averages in complete graphs).

Overall, [START_REF] Draief | Convergence Speed of Binary Interval Consensus[END_REF] and [START_REF] Kuhn | Distributed computation of the mode[END_REF] are probably the most closely related to our work since they consider arbitrary graphs. However, we our work consider more general communication models, including dynamic graphs. Similar to [START_REF] Draief | Convergence Speed of Binary Interval Consensus[END_REF], our results for k = 2 rely on spectral properties of the underlying graph (and are asymptotically the same for their concrete Chapter 8

Voter Model [KMS16]

In the voter model, each node of a graph has an opinion, and in every round each node chooses independently a random neighbor and adopts its opinion. We are interested in the consensus time, which is the first point in time where all nodes have the same opinion. In order to derive strong bound on the voter model we will study the dual of this problem which is called Coalescing random walks. Coalescing random walks is a fundamental stochastic process on connected and undirected graphs. The process begins with particles on some subset of the nodes in the graph. At discrete time steps, every particle performs one step of a random walk.1 Whenever two or more particles arrive at the same node at the same time step, they merge into a single particle and continue as a single random walk. The coalescence time is defined as the first time step when only one particle remains. The coalescence time depends on the number and starting positions of the particles.

When starting with two particles, the coalescence time is referred to as the meeting time.

Let t meet denote worst-case expected meeting time over all pairs of starting nodes and let t coal denote the expected coalescence time starting from one particle on every node. It is clear that t meet ≤ t coal ; as for an upper bound, it can be shown that

t coal = O(t meet log n),
where n is the number of nodes in the graph. The main idea used to obtain the bound is that the number of surviving random walks halves roughly every t meet steps. A proof of the result appears implicitly in the work of Hassin and Peleg [START_REF] Hassin | Distributed Probabilistic Polling and Applications to Proportionate Agreement[END_REF].

Results

In this work, we provide several results relating the coalescence too two fundamental quantities related to random walks: the mixing time and the meeting time. In particular, our focus is on understanding for which graphs the coalescence time is the same as meeting time, as we know that t coal is always in the rather narrow interval of [Ω(t meet ), O(t meet • log n)]. As a consequence of our results, we derive new bounds on the coalescence times for several graph families of interest. Formal definitions of all quantities used below appear in Chapter 2 and Section 8.3.1.

Our first main result relates t coal to t meet and t mix . As already mentioned in the introduction, the crude bound t coal = O(t meet log n) is well-known. However, this bound is not in general tight, as demonstrated by our result below.

Theorem 8.1. For any graph G, we have

t coal = O t meet 1 + t mix t meet • log n Consequently, when t meet ≥ t mix log 2 n, t coal = O(t meet ).
The proof of Theorem 8.1 appears in Section 8.3. One interesting aspect about this bound is that it can be used to establish t coal = Θ(t meet ) even without having to know the quantities t meet or t mix . This flexibility turns out to particularly useful when dealing with random graph models for "real world" networks. From this we immediately derive the coalescence time for Erdős Rény graphs, random regular graphs, hypercubes, tours for any d > 2 as well as for many "real word" networks: Common features of real world graph models are (i) a power-law degree distribution with exponent β ∈ (2, 3) and (ii) high expansion, i. e., 1 -λ 2 is not too large, and hence

t mix = O(log n).
Notice that (i) β ∈ (2, 3) implies that w.h.p. we have ∆ = O(n 1-), and hence π2 2 ≤ max u∈V π(u) ≤ n -, for ε > 0. It is easy to see that t coal ≥ t meet = Ω( π -2 2 ) which implies for the above defined networks that t coal = Ω(n ). This implies that for a large range of parameters we have that t coal = Θ(t meet ).

Our next main result shows that the bound in Theorem 8.1 is tight up to a constant factor, which we establish by constructing an explicit family of graphs. Interestingly, for this family of almost-regular graphs we also have t hit t meet , thus showing that t hit may be a rather loose upper bound for t coal in some cases. 2 Theorem 8.2. For any sequence (α n ) n≥0 , α n ∈ [1, log 2 n] there exists a family of almostregular graphs (G n ), with G n having Θ(n) nodes and satisfying tmeet t mix = Θ(α n ) such that

t coal = Ω t meet • 1 + t mix t meet • log n .
The above two results show that that t meet /t mix should be Ω(log 2 n) to guarantee that

t coal = O(t meet ).

Approach and Technical Contributions

When dealing with processes involving concurrent random walks, a significant challenge is often to understand the behavior of "short" random walks. This challenge appears in several settings, e. g., in the context of cover time of multiple random walks [START_REF] Alon | Many Random Walks Are Faster Than One[END_REF][START_REF] Efremenko | How Well Do Random Walks Parallelize?[END_REF], where In our setting, we also face these generic problems and devise different methods to get a handle on the meeting time distribution before its expectation.

Bounds on t coal in terms of t mix and t meet

The key ingredient in the proof of Theorem 8.1, where we express t coal as a trade-off between t meet and t mix is the following: a tight bound on the probability p that two random walks meet before time steps, for in the range [t mix , t meet ]. Arguing about meeting probabilities of walks that are much shorter than t meet allows us to understand the rate at which the number of alive, random walks is decreasing.

Optimistically, one may hope that starting with k random walks, as there are k 2 possible meeting events, roughly k 2 • p meetings may have occurred after time steps. However, the non-independence of these events turns out to be a serious issue and we require a significantly more sophisticated approach to account for the dependencies. We divide the k random walks into disjoint groups G 1 and G 2 (with |G 1 | usually being much smaller than |G 2 |) and walks of G 1 can't be eliminated. The domination of the real process by the grouprestricted one is established by introducing a formal concept called immortal process at the beginning of Section 8.3.2. In this stochastic process, we can expose the random walks of G 1 first and consider meetings with random walks in G 2 (for an illustration, see Figure 8.2 on page 103). Conditioning on a specific exposed walk in G 1 , the events of the different walks in G 2 meeting this exposed walk are indeed independent. In fact, we will also use the symmetric case where the roles of G 1 and G 2 are switched. Thus, the problem then reduces to calculating the likelihood of a random walk in G 2 having a 'good trajectory', i. e., one which many random walks in G 1 would meet with large enough probability. Surprisingly, it suffices to divide trajectories into only two categories (Lemma 8.5).

Although, one may expect that a more fine-grained classification of trajectories would result in better bounds, it turns out not to be the case. In fact, the bound that we derive on the coalescing time in Theorem 8.1 is tight, and this is precisely due to the tightness of Lemma 8.5. The tightness is established by the following construction (cf. Figure 8.1).

z G 2 z 1 G 1 1 z 2 G 2 1 z 3 G 3 1 z κ G κ 1 Figure 8.1: The graph described in Section 8.3.5 with t coal = Ω(tmeet + √ α • log n • tmix).
The graph is designed such that the vast majority of meetings (between any two random walks) occur in a relatively small part of the graph (G 2 in Figure 8.1). On average, it takes a considerable number of time steps before random walks actually get to this part of the graph.

What this implies is that for relatively short trajectories (of length significantly smaller than t meet ), it is quite likely that other random walks will not meet them (cf. Lemma 8.5). There is a bit of a dichotomy here, once a walk reaches G 2 it is likely that many random walks will meet it; however, a random walk not reaching G 2 is unlikely to be met by any other random walk.

Equipped with Theorem 8.1, we can bound t coal = Θ(t meet ) for all graphs satisfying t meet /t mix ≥ log 2 n. Therefore, the problem of bounding t coal reduces to bounding t meet . For some of the other results including Theorem 8.2, we will need a more fine-grained approach to derive lower (or upper bounds) on the probability that two walks meet during a certain number of steps, which may or may not be smaller than the mixing time or meeting time. The starting point is the following simple observation. If we have two random walks (X t ) t≥0 and (Y t ) t≥0 , and count the number of collisions Z := τ -1 t=0 1 Xt=Yt before time step τ ,then

P[ Z ≥ 1 ] = E[ Z ] E[ Z | Z ≥ 1 ] . (8.1)
If we further assume that both walks start from the stationary distribution, then we have

P[ Z ≥ 1 ] = τ • π 2 2 E[ Z | Z ≥ 1 ]
.

We should mention that this generic approach is of course not new, an analogous variant of counting visits to a vertex has been used by Cooper and Frieze in several works (e. g., [START_REF] Cooper | The Cover Time of Random Regular Graphs[END_REF])

to derive very accurate bounds on the hitting (and cover time) on various classes of random graphs, or in Barlow et al. [START_REF] Barlow | Collisions of Random Walks[END_REF] to bound the collisions of random walks on infinite graphs. However, using (8.1), we are able to obtain several improvements to existing bounds on the meeting time, and as a consequence for coalescing time. We believe that our work further highlights the power of this identity.

The crux of (8.1) is that in order to lower (or upper) bound the probability that the two walks meet, we need to derive a corresponding bound on E[ Z | Z ≥ 1 ], i. e., the number of collisions conditioning on the occurrence of at least one collision.

Our results employ various tools to get a handle on this quantity, but here we mention one that is relatively simple:

E[ Z | Z ≥ 1 ] ≤ max u∈V τ -1 t=0 v∈V p t u,v 2 . (8.2)
The inner summand v∈V (p t u,v ) 2 is the probability that two walks starting from the same vertex u will meet after further t steps. Thus, summing over t and conditioning on the first meeting happening (i. e., the condition Z ≥ 1) at some vertex u before time step τ yields the bound in (8.2). Despite the seemingly crude nature of this bound, it can be used to derive new results for t hit , t meet and t coal that significantly improve over the state-of-the-art for regular graphs.

Bounding the coalescence time

In this section we prove Theorem 8.1, one of our main results. We refer the reader to Section 8.2 for a high-level description of the proof ideas.

The coalescence process

We define the coalescence process as a stochastic process as follows: Let S 0 ⊆ V be the set of nodes for which there is initially one random walk on it, and for all v ∈ S t let

Y v (t) =    u ∈ N (v) w.p. 1 2|N (v)| v w.p. 1 2
The set of active nodes in step t+1 is given by means with probability > 0. We use log n for the natural logarithm. Appendice A contains some known results about Markov Chains that we frequently use in our proofs.

S t+1 = {Y v (t) | v ∈ S t }

A more amenable process

In order to prove our first main result, it is helpful to consider a more general stochastic process, P imm , called the immortal process, involving multiple independent random walks.

In the immortal process, whenever several random walks arrive at the same node at the same time a subset of them (rather than just one) may survive, while the remaining are merged with one of the surviving walks. To identify the random walks, we assume that each walk has a natural number (in N) as an identifier. In order to define this process formally, we introduce some additional notation and definitions; then we state and prove some auxiliary lemmas. A related concept was introduced in [Oli12, Section 3.4] under the name of "allowed killings". As mentioned before, we assume that every random walk r has a unique identifier id(r) ∈ N. We divide the ids into two groups G 1 , the group of immortal walks and G 2 the group of the remaining (mortal) walks. Whenever two or more walks collide at a node, then all walks with ids in G 1 survive, while all walks with ids in G 2 are killed (merged with some walk with id in G 1 ). Furthermore, if all walks have ids in G 2 , i. e., there are no walks with id in G 1 , then the walk with the minimum id among these walks survives. The ids along with the assignment of ids to groups determine which of the random walks that arrive at a given node at the same time survive.

G 1 = {1, 2} G 2 = {3, 4, 5} t = 0
Formally, let P imm denote the following process:

1. At time 0, S 0 = {(u r , id(r))}, where u r is the starting node of random walk r and id(r) is its identifier.

2. At time t, several random walks may arrive at the same node. The process P imm allows some subset of them to survive, while the rest 'coalesce' with one of the surviving walks. Formally, S t+1 is defined using S t as follows. Define the (random) next-step position of the random walk with id i ∈ N which is on node v ∈ V to be

Y v,i (t) :=    u where u ∈ N (v) w.p. 1 2|N (v)| v w.p. 1 2 , Let R v (t) := {(Y v,i (t), i) | (v, i) ∈ S t }, v ∈ V
= {(v, i) | ∃u ∈ V, (v, i) ∈ R u (t)}
be the random walks that have arrived at node v at time step t+1, just before merging happens. Then, merging happens w.r.t. the ids as follows:

(a) If there exists i ∈ G 1 such that (v, i) ∈ Rv (t) (at least one walk with id in G 1 arrives at v), then

S v (t + 1) := {(v, j) | (v, j) ∈ Rv (t), j ∈ G 1 } (b) If there is no i ∈ G 1 , such that (v, i) ∈ Rv (t)
and Rv (t) = ∅ (no walk with id in G 1 arrives at v, but at least one walk arrives at v), then

S v (t + 1) := {(v, j)}, where j = min{i | (v, i) ∈ Rv (t)}.
(c) Otherwise, S v (t + 1) := ∅, i. e., no walk arrived at v.

Finally, let

S t+1 := v∈V S v (t + 1).
104

We now relate this more general process, P imm , to the coalescing process defined in Section 8.3.1. Let P be regarded as a special instance of P imm with G 1 = {1}. In process P , only one of several walks arriving at the same node survives and by convention the one having the smallest id is chosen. Let (S t ) ∞ t=0 denote the stochastic process P . If we define St := {v | (v, i) ∈ S t }, then ( St ) ∞ t=0 is a coalescence process as defined in Section 8.3.1. Moreover, P represented by (S t ) ∞ t=0 is the coalescence process which additionally keeps track of the ids. Throughout this chapter, we assume that every random walk of S 0 is on a distinct node.

In the following we show that the time it takes to reduce to k random walks in the original process P is majorized by the time it takes in P imm to reduce to k random walks.

While this might be intuitive, one needs to be very careful about the dependencies between the meetings of different random walks: For instance a random walk which is immortal in P imm might eliminate many other random walks whereas the corresponding coupled random walk in P might be eliminated early and therefore cannot eliminate said random walks.

Proposition 8.3. Consider the following two processes:

1. Process P is the standard process of coalescing random walks, viewed as a special case of P imm with G 1 = {1} as described above.

2. Process P imm is the process defined above using groups G 1 and G 2 , where 1 ∈ G 1 . Let T k , T k imm be the stopping times given by the condition that fewer than k random walks remain for the two processes respectively. Assume both processes start with the same initial configuration, i. e., the vertices occupied by walks in both processes are identical and there is only one walk per vertex in either process. Then, there exists a coupling such that

T k ≤ T k imm .
Proof. We will give a coupling between the moves of walks in P imm and P int , a new process that is essentially intermediate between P and P imm ; furthermore, we will show that the original process P is essentially a restricted view of the process P int . The process P int will label the walks dead, alive, and phantom. We emphasize that a phantom walk is not considered alive. Note that the processes P and P imm can be viewed as processes which assign labels to each random walk of the type alive and dead.

Let S Q t denote the set of tuples of alive walks in process Q ∈ {P, P int , P imm } at time t.

Let SQ t = {v | (v, i) ∈ S Q t }
for Q ∈ {P, P int , P imm } be the set of nodes which are occupied by at least one alive walk (there might be several in P imm at t ≥ 1). In order to prove the proposition, we show that there exists a coupling, such that for any t ∈ N

SP

T k = min{t ≥ 0 : | SP t | ≤ k} ≤ min{t ≥ 0 : | SP imm t | ≤ k} = T k imm .
We now define P int . As mentioned above, the walks in P int will be given three kinds of labels alive, dead, or phantom; the dead walks do not continue ahead in time; alive and phantom walks do.

Formally, P int using the groups G 1 and G 2 is defined as follows. We say that walk r is Note, that walks of G 1 are either alive or phantom and walks of G 2 are either alive or dead. Also, note that in the process P int , there is at most one alive walk at any given node.

of type G i , if id(r) ∈ G i for i ∈ {1,
Throughout the proof we regard the processes in two stages: First, each random walk selects a destination (possibly the same node it was on) and moves there. In the second phase the walks are merged according to the process. See Figure 8.3 for an illustration. We prove (8.4) by induction on t starting from the same initial configuration: if v ∈ SP t , then v ∈ SP int t . Consider the inductive step from t to t + 1 and assume that the claim holds at the end of round t (after merging happened). For the (unique) random walk at v ∈ SP t under process P , we couple its transition to node Y v (t + 1) (where we possibly have Y v (t + 1) = v) with the corresponding alive walk of SP int t (there might be several walks of P int , however only one is alive and we couple with this alive walk). Let S be the set of nodes to which a random walk in P moved, i. e., S = {Y v (t + 1) : v ∈ SP t }. Observe, that before the merging takes place in round t + 1 (but moves have been made), there is, by induction hypothesis and the coupling, at least one alive walk of P int on each node of S. Furthermore, the definition of P int ensures that whenever an alive random walk moves to a node, then after merging takes place, at least 4 one alive walk remains. Thus, our coupling

ensures that if v ∈ SP t+1 , then v ∈ SP int t+1 .
In words, if one looks at the subsets where there is an alive walk of P int , this is essentially the standard coalescence process. This finishes the proof of (8.4) and we turn to proving (8.5).

When starting from the same initial configuration, we will provide a coupling that satisfies the following invariants.

1. There is a bijective map from the alive and phantom walks of P int to the alive walks of P imm , such that the following holds. All walks of P int of type G i are mapped to walks of P imm of type G i , for i ∈ {1, 2}.

4 By definition, there is actually exactly one alive walk. 107 2. Whenever a walk of type G 2 is labeled dead in P imm , then it is also labeled dead in P int and vice versa.

At the beginning there are no dead or phantom walks in P int , there are no dead walks in P imm , all walks are alive and as the starting positions in P imm and P int are the same, an arbitrary bijective mapping may be chosen, so long as it respects node positions and walk types.

Assume the invariant holds at time t. We take one random walk step for each alive or phantom random walk in P int . These are coupled with the corresponding walks in P imm , under the chosen map. Walks that are already dead are neither simulated in P int nor in P imm . Hence, we can ensure the bijection between the walks of G 1 in both processes holds at time t + 1.

We now prove the second invariant. Note that whenever a walk r of type G 2 in P imm (P int ) is labeled dead, this implies there must have been another walk r on the same node at the same time. Since there is a bijective map, r must be on the same node in P int (P imm ). We have that either r is of type G 1 or r is of type G 2 and that id(r ) < id(r). In either case, r is also killed (labeled dead) in P int (P imm ). Hence, we can ensure the bijection between the walks of G 2 in both processes holds at time t + 1. Thus, the invariant holds at time t + 1.

By induction, and since the alive walks of P int are a subset of the alive walks of P imm the invariant holds throughout the process and yielding (8.5). This finishes the proof.

Meeting Time Distribution Prior to t meet

Let (X t ) t≥0 and (Y t ) t≥0 be independent random walks starting at arbitrary positions. For τ a multiple of t mix , the following lemma gives a lower bound on the probability of intersection of the two random walks in τ steps. Lemma 8.4. Let (X t ) t≥0 and (Y t ) t≥0 be two independent random walks starting at arbitrary positions. Let intersect(X t , Y t , τ ) be the event that there exists 0 ≤ s ≤ τ , such that X s = Y s .

Then

P[ intersect(X t , Y t , 5t mix ) ] ≥ 1 32α
, where α = t meet /t mix . Furthermore, for any 1 ≤ b ≤ e-1 e • α, there exists a constant c > 0, such that

P[ intersect(X t , Y t , c b t mix ) ] ≥ b α ,
Proof. First, let ( Xt ) t≥0 and ( Ỹt ) t≥0 be two random walks that start from two independent samples drawn from the stationary distribution and are run for := 2 α t mix steps. Notice that ≥ 2t meet , and hence, by Markov's inequality,

P intersect( Xt , Ỹt , ) ≥ 1 2 . ( 8
P[ intersect(X t , Y t , 5t mix ) ] ≥ (1 -e -1 ) 2 • P intersect( Xt , Ỹt , t mix ) ≥ (1 -e -1 ) 2 • 1 4 α .
Observing that for any α ≥ 1, the RHS above expression is greater than 1/(32α) completes the proof of the first part. For the second part, we consider k blocks of length 5t mix . Due to independence of different blocks, the probability of that the two walks meet in at least one of the k blocks is at least 1 -(1 -1 32α ) k . We set k := * 32b/(1 -e -1 ), x := 1/(32α). We distinguish between two cases.

Case k • x < 1: We use the fact that (1 -x) k ≤ e -xk ≤ 1 -(1 -e -1 )xk for 0 ≤ x < 1, k ≥ 0 and xk ≤ 1. We derive that the probability of intersecting after k blocks is at least 1

-(1 -1 32α ) k ≥ (1 -e -1 )k/(32α) = b/α. Case k • x ≥ 1: We have 1 -(1 -1 32α ) k ≥ 1 -(1 -1 32α ) 32α ≥ 1 -1/e ≥ b/α.
In both cases the second part follows.

At the heart of the proof of Theorem 8.1 lies the following lemma that analyses the marginal distribution of the meeting time distribution. That is, we only expose the first random walk (X t ) τ t=0 , and look at how this affects the probability of meeting. In essence, we show that at least one of the two "orthogonal" cases hold. In Case 1 (corresponding to set C 1 ), there is at least a modest probability that after exposing (X t ), (Y t ) will intersect with significant probability. Otherwise, in Case 2 (corresponding to set C 2 ), there is a significant probability that after exposing (X t ), (Y t ) will intersect with at least a modest probability. Lemma 8.5. Fix τ ∈ N and a graph G. Let (X t ) τ t=0 and (Y t ) τ t=0 be independent random walks, where the starting nodes X 0 and Y 0 are drawn independently from the stationary distribution π (w.r.t. to G), and the walks are run for τ steps. Let p = P[ intersect(X t , Y t , τ ) ] and let T τ denote the set all possible trajectories of a walk of length τ in G (including possible self-loops). We define the following two categories C 1 and

C 2 with C 1 ⊆ C 2 C 1 := {(z 0 , . . . , z τ ) ∈ T τ : P[ ∃0 ≤ s ≤ τ, Y s = z s ] ≥ √ p} C 2 := {(z 0 , . . . , z τ ) ∈ T τ : P[ ∃0 ≤ s ≤ τ, Y s = z s ] ≥ p/3}. Then, P[ (X t ) τ t=0 ∈ C 1 ] ≥ p 3 or P[ (X t ) τ t=0 ∈ C 2 ] ≥ √ p 3 .
While the actual lower bounds on the probabilities appear rather crude, it turns out that the "significant probability" √ p/3 is best possible, as we demonstrate in our lower bound construction later. Remarkably, the fact that the "modest probability" is only p/3 and much smaller than √ p/3 does not affect the tightness of our bound, since in Claim 8.7, we can make up for this gap in both cases through a simple amplification argument over the unexposed random walks.

Proof. Let us suppose that P[ (X t ) τ t=0 ∈ C 1 ] < p 3 . We show that this implies P[

(X t ) τ t=0 ∈ C 2 ] ≥ √ p 3 . Assume for the sake of contradiction P[ (X t ) τ t=0 ∈ C 2 ] < √ p
3 . We have

p = P[ intersect(X t , Y t , τ ) ] ≤ P[ (X t ) τ t=0 ∈ C 1 ] • 1 + P[ (X t ) τ t=0 ∈ (C 2 \ C 1 ) ] • √ p + P[ (X t ) τ t=0 ∈ C 2 ] • p 3 < p/3 + √ p/3 • √ p + p/3 ≤ p,
a contradiction. This completes the proof.

It is well-known that starting with k random walks, the coalescence time is bounded by O(t meet log k), this can be deduced from the proof presented in [START_REF] Hassin | Distributed Probabilistic Polling and Applications to Proportionate Agreement[END_REF]. For the sake of completeness, we give a self-contained proof5 .

Proposition 8.6. We have t coal (S 0 ) = O(t meet log |S 0 |).

Proof. Let P be the coalescing process (with ids) defined in Section 8.3.2. Recall that We divide time into blocks of length τ := c e-1 e t meet + 4t mix , where c is the constant of Lemma 8.4, i. e., P intersect(X t , Y t , c e-1 e t meet ) ≥ e-1 e . We are primarily interested in what happens at the end of the blocks, i. e., at time steps t 0 , t 0 + τ, t 0 + 2τ, . . .. For simplicity, we will start counting time from 0 at the beginning of each block. Let (X t ) t≥0 be the random walk with id 1. After 4t mix steps, we can couple the state of the random walk (X t ) t≥4t mix with a node drawn from π with probability at least (1 -e -1 ), since 4t mix ≥ t sep (see Lemma A.27). Further, note that conditioned on this coupling, the statement of Lemma 8.5 implies that (X t ) t≥4t mix ∈ C 2 w.p. at least p/3, where we used C 2 ⊆ C 1 , and where p := P intersect( Xt , Ỹt , c • e-1 e • t meet ) ≥ e-1 e for X0 , Ỹ0 ∼ π. We condition on the successful coupling of X 4t mix with a node drawn from π and that (X t ) t≥4t mix ∈ C 2 , which happens with probability at least (1-e -1 )p/3 = (e-1) 2 3e 2 (called event E). Finally, consider any random walk (Y t ) t≥0 b with id other than 1. Again with probability at least 1-e -1 we can couple Y 4t mix with a node drawn from π and conditioned on successful coupling, (Y t ) t≥4t mix meets (X t ) t≥4t mix between time steps [4t mix , τ ] with probability at least p/3, by definition of C 2 . Thus, conditioned on event E, each walk of G 2 vanishes w.p.

G 1 = {1}. Let S t
(1 -e -1 )p/3 = (e-1) 2 3e 2 and thus the expected fraction of walks killed in the τ time steps is at least (e-1) 2 3e . Let Z = |S t 0 + •τ | denote the number of random walks alive at the beginning of block .

E Z | F t 0 +( -1)•τ ≤ Z -1 -(Z -1 -1) • (e -1) 4 9e 4 ≤ Z -1 - Z -1 100 .
The above holds as long as Z -1 ≥ 100. We can therefore apply Lemma A.14 with parameters g = 99/100 • S 0 and β = 99/100 to obtain that E[ T ] = O(τ ) = O(t meet ), which completes the proof.

Upper Bound -Proof of Theorem 8.1

We commence by considering the process P imm defined in Section 8.3.2. This allows us to establish Claim 8.7 providing us with the following trade-off. For a given period τ of length at least t mix we obtain a bound on the required number of periods to reduce the number of random walks by an arbitrary factor. The proof relies heavily on Lemma 8.5 which divides the walks of G 1 into two groups allowing us to expose the walks of G 1 first and then to calculate the probability of the walks of G 2 to intersect with them. In fact, we will also use the symmetric case where the roles of G 1 and G 2 are switched. We define

= {(v i , i) | 1 ≤ i ≤ k }. • Let G 1 = {1, . . . ,
IDs(S t ) := {id(r) | (u r , id(r)) ∈ S t }, t ∈ N.
The following lemma gives the expected time it takes to reduce the number of random walks in G 2 from k -k to some arbitrary integer g ≥ k: given a period of length τ and integer g, assuming that k = |G 1 | is large enough, we derive a bound on the number of periods of length τ until the walks in G 2 are reduced to g. The required size of k is a function of the probability for two random walks drawn from π intersecting after τ time steps.

Claim 8.7. Let τ ∈ N, let (X t ) τ t=0 and (Y t ) τ t=0 be independent random walks run for τ steps, with X 0 and Y 0 drawn independently from π. Let p τ ≤ P[ intersect(X t , Y t , τ ) ] be a lower bound on the probability of the intersection of the two walks during the τ steps. Consider an instantiation of P imm (S 0 , k). Suppose that k ≥

3 (1-e -1 )•pτ . For some 1 ≤ g ≤ |S 0 | -k, define the stopping condition T g = min{t ≥ 0 | | IDs(S t ) ∩ G 2 | ≤ g}. Then the expected stopping time satisfies E[ T g ] = O (4t mix + τ ) • 1 p τ • (log |G 2 | -log g) .
We first describe the high-level proof idea, before delving into the formal proof. We divide time into blocks of size 4t mix + τ . For any random walk (Z t ) 4t mix +τ t=0 we can couple its position after 4t mix ≥ t sep w.c.p. with a node drawn from π. Thus, conditioning on the success of this coupling we have, by Lemma 8.5,

P (Z t ) 4t mix +τ t=4t mix ∈ C 1 ≥ pτ 3 or P (Z t ) 4t mix +τ t=4t mix ∈ C 2 ≥ √ pτ
3 . In the former case we have that w.c.p. there is at least one random walk r in G 1 which is, due to independence of the walks, in class C 1 . The hypothetical extension of the trajectory of any random walk in r ∈ G 2 intersects with r w.p. c √ p τ /3, where the constant arises due to the fact that we also need to couple the state of r at time 4t mix to a node drawn according to π. (We need to consider the hypothetical extension because the walk r may get eliminated sooner-this only helps us.) Thus, r gets eliminated w.p. at least c √ p τ for a suitable constant c.

In the latter case we have that w.p. at least c √ p τ /3 a random walks of G 2 is in class C 2 . Every random walk in that class intersects w.c.p. with at least one of the walks of G 1 . Thus, in both cases, we have that in each block a random walk of G 2 is eliminated w.p.

a least c √ p τ for some constant c. Thus, the number of random walks in G 2 decrease in expectation by a factor of c √ p τ .

Proof. We will consider the process in blocks each consisting of 4t mix +τ time steps. For convenience in the proof, we'll restart counting time steps from 0 at the beginning of each block;

we keep track of the total number of time steps by counting the number of blocks. Let C 1 and C 2 be as defined in Lemma 8.5. Then we perform a case analysis by considering the two possible outcomes described in Lemma 8.5 separately. We define

Z j = | IDs(S j•(4t mix +τ ) ) ∩ G 2 |,
i. e., the number of walks remaining in G 2 after j blocks of time have passed. For any j ≥ 1, we will show that there exists a constant c > 0 such that,

E[ Z j | F j-1 ] ≤ Z j-1 • (1 -c √ p τ ).
By using Lemma A.14, we get

E[ T g ] = O (4t mix + τ ) • 1 √ pτ • (log |G 2 | -log g) (
the factor (4t mix + τ ) appears as the size of the block). Recall that F j is the filtration up to end of the jth block. In the remainder we show that we have indeed

E[ Z j | F j-1 ] ≤ Z j-1 • 1 -c √ p τ .
Case 1. P[ (X t ) τ t=0 ∈ C 1 ] ≥ pτ 3 : Consider any random walk r in G 1 at the beginning of a block. Using Lemma A.27, after 4t mix steps we can couple the state of the random walk with a node drawn from π with probability at least (1 -e -1 ). Furthermore, conditioned on this coupling, the portion of the random walk between time steps 4t mix and 4t mix + τ of the walk is in class C 1 with probability at least pτ 3 . Since k ≥ 3 pτ •(1-e -1 ) , w.p. c 1 > 0, in any block, there exists a walk in G 1 that has the portion between time steps 4t mix and 4t mix + τ in C 1 .

Fix a block and condition on the event that there is a walk in G 1 , denoted by r 1 , whose portion between time steps 4t mix and 4t mix + τ is in C 1 . Consider any walk in G 2 , denoted by r 2 , at the beginning of the block. We want to argue that this walk r 2 has a reasonable probability of intersecting some walk in G 1 in this block of time steps. First, consider (the possibly hypothetical continuation of r 2 ) walk r 2 for the entire length of the block. The reason for this is that if r 2 and some walk from G 1 are at the same node at the same time sometime in the block, r 2 will be eliminated in the process P imm (S 0 , k); however, we can consider its hypothetical extension to the entire length of the block. Using Lemma A.27 the state of the walk r 2 at time step 4t mix can be coupled with a node drawn from π with probability at least c 2 := 1 -e -1 . Then conditioned on successful coupling, the probability that r 2 and r 1 collide during time steps 4t mix and 4t mix + τ is at least √ p τ (by definition of C 1 in Lemma 8.5). Thus, the probability that r 2 hits at least one walk in G 1 is at least

c 1 • c 2 • √ p τ .
Note that it is also possible for r 2 to be eliminated by another walk from G 2 .

In any case, we have that r 2 is eliminated w.p. at least c √ p τ and we get

E[ Z j | F j-1 ] ≤ Z j-1 • (1 -c 1 • c 2 √ p τ ). Case 2. P[ (X t ) τ t=0 ∈ C 2 ] ≥ √ pτ
3 : Consider a walk in G 2 , denoted by r 2 , at the beginning of a block; as in the previous case, we will consider a possibly hypothetical continuation r 2 of r 2 . Using Lemma A.27 we can couple the state of r 2 at time step 4t mix with a node drawn from π with probability at least 1 -e -1 . Furthermore, conditioned on the successful coupling, with probability at least √ pτ 3 the trajectory of r 2 between the time steps 4t mix to 4t mix + τ is in C 2 . Thus, with probability at least p := (1 -e -1 ) √ pτ 3 , r 2 has a trajectory between time steps 4t mix and 4t mix + τ that lies in C 2 . Now consider any random walk r 1 ∈ G 1 at the beginning of the block. Again, using Lemma A.27 with probability at least 1 -e -1 , we can couple the state of the random walk at time 4t mix with a node drawn from π. Conditioned on this between time steps 4t mix to 4t mix + τ , this random walk hits any trajectory whose portion between time steps 4t mix to 4t mix + τ lies in C 2 with probability at least p τ /3 (by definition of C 2 in Lemma 8.5). Since k = |G 1 | ≥ 3 (1-e -1 )•pτ , with at least constant probability c 1 > 0 there is some walk in G 1 that intersects any fixed trajectory whose portion between time steps 4t mix to 4t mix + τ lies in C 2 . Since the random walks in G 1 are independent, by the definition of the immortal process, we have that any walk in G 2 is eliminated by the end of the block with probability at least c 1 • p = c √ p τ for some constant c > 0. Similarly as before, it is possible that r 2 is eliminated by at least one of the walks of G 2 , which only increases the probability for r 2 of being eliminated. We get

E[ Z j | F j-1 ] ≤ Z j-1 • (1 -c √ p τ ).
In the following we bound the time T required to reduce to 2 α random walks. The claim follows by applying Claim 8.7 to derive a bound on T imm for processP imm , and using the majorization of T by T imm (Proposition 8.3).

Corollary 8.8. Consider the coalescence process starting with set S 0 and let α = t meet /t mix .

Let

T 1 = min{t ≥ 0 | |S t | ≤ 2 α }. Then E[ T 1 ] = O(t mix • √ α • log |S 0 |).
Proof. We consider the process P (defined in Section 8.3.2), which is identical to the coalescence process, but in addition also keeps track of ids of random walks and that al-lows only the walk with the smallest id to survive. We assume that the ids are from the set {1, 2, . . . , |S 0 |}. Let S 0 = {(v 1 , 1), . . . , (v |S 0 | , |S 0 |)} and S0 = {i : (v, i) ∈ S 0 }. We consider the process P imm (S 0 , k) and k = α . Let T * 1 be the stopping time defined by | IDs( St ) ∩ G 2 | ≤ α for the process P imm (S 0 , k). By definition of P imm and Proposition 8.3, it follows that T imm stochastically dominates T . Thus, it suffices to bound E[ T imm ]. W.l.o.g. we assume that α ≥ 6 e-1 e , otherwise the claim follows directly from Proposition 8.6. We apply Lemma 8.4 with b = 6 and derive that for some suitable constant c,

p = P[ intersect(X t≥0 , Y t≥0 , 6ct mix ) ] ≥ 6 α , Thus, we have 3 (1 -e -1 ) • p ≤ 3 1 2 • p ≤ α ≤ k
Applying Claim 8.7 with g = α, τ = 6ct mix (where c is a constant as given by Lemma 8.4), p τ = 6/α, and observing that k ≥

3

(1-e -1 )•pτ , we get the required result.

In the following we bound the time T required to reduce from 2 α random walks to a single random walk. The proof uses the same ideas as before (Corollary 8.8) however, this time we consider several phases and in each we reduce the number of random walks by a constant factor. The expected time per phase is geometrically increasing as the number of walks decreases and the overall time is essentially dominated by the time for a constant number of random walks to meet, which is O(t meet ).

Lemma 8.9. Consider the coalescence process starting with set S 0 , satisfying

|S 0 | ≤ 4α log α, where α = t meet /t mix . Let T 2 := min{t ≥ 0 | |S t | ≤ 1}. Then E[ T 2 ] = O(t meet ).
Proof. We will consider the coalescence process in phases. Let be the largest integer such that |S 0 | ≥ 4 3 . For j ≥ 1, the jþ phase ends when

|S t | < 4 3 -j+1
. The (j + 1)þ phase begins as soon as the jþ phase ends. Note that it may be the case that some phases are empty. Let T 2 (j) denote the time for phase j to last. We will only consider phases up to which -j + 1 ≥ 32.

Now we focus on a particular phase j. Let t j be the time when the jþ phase begins and let S t j denote the corresponding set at that time. Thus, we have 4 3

-j+1 ≤ |S t j | < 4 3 -j+2 (8.8)
We consider the process P imm defined in Section 8.3.4 as follows. Define n j = |S t j |. Fix a phase j and define S 0 = {(v 1 , 1), . . . , (v n j , n j )} and S 0 = {v 1 , . . . , v n j }. Then, consider again the set of occupied vertices (ignoring the labels) St j +t = {v | ∃i ∈ N, (v, i) ∈ S t } with t ∈ N. Thus, phase j ends when

|S t | = | St j +t | < 4 3 -j+1
. Let

k j := |S 0 | 2
be the size of G 1 and consider the process P imm (S 0 , k j ) as defined in Section 8.3.4. Let

g j := |S 0 | -k j 3
and

T * 2 (j) := min{t | | IDs(S t ) ∩ G 2 | ≤ g j }.
We note that as long as -j + 1 ≥ 32, g j ≥ 1 and at time T * 2 (j),

|S t | ≤ g j + k j ≤ |S 0 | -k j 3 + k j = |S 0 | 3 + 2k j 3 ≤ |S 0 | 3 + |S 0 | 3 + 2 3 < 3 4 • |S 0 |.

By

τ j := cb j t mix ,
for independent random walks (X t )

τ j t=0 , (Y t ) τ j t=0 , P[ intersect(X t , Y t , τ j ) ] ≥ p j ,
where

p j := 32 log(4/3)( -j + 1)(3/4) -j+1 .
We seek to apply Claim 8.7 to bound E[ T * 2 (j) ]. We first verify that the conditions of Claim 8.7 are fulfilled. In particular, we verify that k j ≥ 8 p j ; to see this consider the following:

8 p j = 8 32 log(4/3)( -j + 1) (4/3) -j+1 ≤ 1 4 • 4 3 -j+1 ≤ 1 2 • |S 0 | ≤ k j ,
where we used (8.8) and |S 0 | = |S t j | in the second-last inequality. Thus we can apply Claim 8.7 and derive

E[ T * 2 (j) ] ≤ (τ j + 4t mix ) • 1 √ p j • log | IDs(S 0 ) ∩ G 2 | -log g j
and we continue by dissecting that bound. Since b j ≥ 1, there exists a suitably large constant c 1 , so that τ j + 4t mix ≤ c 1 b j t mix . Furthermore,

b j √ p j = 32α log(4/3)( -j + 1)(3/4) -j+1
32 log(4/3)( -j + 1)(3/4) -j+1

= O α -j + 1 • 3 4 ( -j+1)/2 . Observe that, by definition, | IDs(S 0 ) ∩ G 2 |/g j ≤ 3, hence log | IDs(S 0 ) ∩ G 2 | -log g j ≤ log(3).
Putting everything together, we get that there is a constant c 2 such that,

E[ T * 2 (j) ] ≤ c 2 • t mix • α • -j + 1 3 4 ( -j+1)/2
(8.9)

Note that since we stop when -j + 1 < 32, there are at most -30 phases considered.

Let T be the random variable denoting the time step when the last phase ends; at this point

|S T | = O(1). Therefore, using Proposition 8.6, E T 2 -T | T = O(t meet ). But, clearly T is stochastically dominated by -30 j=0 T * 2 (j)
. Thus, we have

E[ T 2 ] = E T + E E T 2 -T | T ≤ c 2 • t mix • α -30 j=0 -j + 1 3 4 ( -j+1)/2 + c 3 t meet (8.10) ≤ c 2 • t mix • α + c 3 t meet = O(t meet ) (8.11)
Above, in (8.10) we used (8.9) and the fact that E T 2 -T | T ≤ c 3 t meet for some constant c 3 > 0 and in step (8.11), we used the fact that ∞ j=32 jc j < 1 for c ≤ 3/4.

Thus, the first phase (Corollary 8.8) and the second phase (Lemma 8.9) take together

O( √ α • log n • t mix + t meet )
time steps, which yields Theorem 8.1.

Lower Bound -Proof of Theorem 8.2

In this section we give a construction of a graph family in order to establish lower bounds on t coal (G) in terms of t meet (G) and t mix (G) demonstrating that Theorem 8.1 is asymptotically tight. Additionally, our construction generalizes a claim of Aldous and Fill [AF02, Chapter 14]: They mention that it is possible construct regular graphs that mimic the n-star in the sense that the t meet = o(t avg-hit ), without giving further details of the construction. Our construction shows that even the coalescence time can be significantly smaller than the

z G 2 z 1 G 1 1 z 2 G 2 1 z 3 G 3 1 z κ G κ 1 Figure 8
.4: The graph described in Section 8.3.5 with

t coal = Ω(tmeet + √ α • log n • tmix).
average hitting time for almost-regular graphs. For our family of almost-regular graphs, there is a polynomial gap between t meet and t avg-hit . More importantly, we show that these almost-regular graphs have a gap of t mix /t meet •log n between coalescing and meeting time.

This shows that the bound in Theorem 8.1 is best possible, even if we constrain G to be almost-regular. We refer the reader to Section 8.2 for a high-level description of the proof ideas.

More precisely, in the proof of Theorem 8.2 we shall give an explicit construction of a graph family G = G n with t coal = Ω( √ α n • log n • t mix ), where α n = t meet /t mix . For the remainder of this section, we will drop the dependence on n and will simply use G instead of G n and α instead α n .

The construction of G (see Figure 8.4 for an illustration) is based on two building blocks,

G 1 and G 2 . First, let G 1 = (V 1 , E 1 ) be a clique of size √ n. Let G 2 = (V 2 , E 2 ) be a √ n- regular bipartite Ramanujan Graph on n/ √ α nodes [MSS15], where α = max{α, 2 20 • C 2 }, where C > 1 is the universal constant of Corollary A.30. The graph G is made of one copy of G 2 , κ = √ n copies of G 1 (denoted by G 1 1 , G 2 1 , . . . , G κ 1 )
, and a node z, which has an edge to n/α distinct nodes of G 2 and to each of the designated nodes

z i ∈ V i 1 in G i 1 for i ∈ [1, κ].
It is not difficult to see that this graph is almost-regular, i. e., maximum and minimum degree differ by at most a constant.

In Lemma 8.13, Lemma 8.14, Lemma 8.15 and Lemma 8.16 respectively we show that

t mix = Θ(n), t meet = Θ(α n), t coal = Ω( √ α • n log n),
and t avg-hit = Ω(n 3/2 ). We start with the following auxiliary lemma which shows that the walk restricted to V 2 behaves similarly to the walk restricted to V 2 ∪ { z}, meaning that the walks have very similar t-step probabilities.

Lemma 8.10. Let P denote the transition matrix of the random walk on G, Q the transition matrix of the random walk on G 2 and Q be the transition matrix of the random walk on the subgraph of G induced by V 2 ∪ { z}. Let S * = {u ∈ V 2 ∩ N ( z)}. Then the following statements hold:

(i) For any u, v ∈ V 2 we have p t u,• -q t u,• TV ≤ t-1 i=1 p i u,S * /(2 √ n) ≤ t/(2 √ n). (ii) For any u, v ∈ V 2 we have q t u,• -q t u,• TV ≤ t-1 i=1 p i u,S * /(2 √ n) ≤ t/(2 √ n).
(iii) For any u, v ∈ V 2 we have that after

t = t mix (G 2 ) time steps p t u,• -p t v,• TV ≤ o(1)+2/e.
Proof. Let (X t ) t≥0 be the Markov chain with transition matrix P and let (Y t ) t≥0 be the Markov chain with transition matrix Q. We will inductively couple these two random walks starting from X 0 = Y 0 = u. Given that we coupled both chains up to time t -1, we can couple (X t , Y t ) such that X t = Y t with an error probability

P[ X t = Y t | X t-1 = Y t-1 ] = P[ X t = Y t | X t-1 = Y t-1 , X t-1 ∈ S * ] • P[ X t-1 ∈ S * ] + P[ X t = Y t | X t-1 = Y t-1 , X t-1 ∈ V 2 \ S * ] • P[ X t-1 ∈ V 2 \ S * ] ≤ p t-1 u,S * /(2 √ n) + 0.
We have, by [LPW06, Proposition 4.7],

p t u,• -p t v,• TV = inf{P[ X = Y ] | (X, Y ) is a coupling of p t u,• and p t v,• }.
Hence, by a union bound over t steps,

p t u,• -p t v,• TV = inf{P[ X = Y ] | (X, Y ) is a coupling of p t u,• and p t v,• } ≤ P[ X t = Y t ] ≤ t-1 i=1 p i u,S * /(2 √ n) ≤ t 2 √ n .
To prove the second part we redefine (X t ) t≥0 to be the Markov chain with transition matrix Q and the proof is identical. We proceed with the last part. For u, v ∈ V 2 we have that after t = t mix (G 2 ) time steps, by the triangle inequality and using that t mix (G 2 ) = O(1), by Proposition A.29, we get

p t u,• -π G 2 (•) TV ≤ p t u,• -q t u,• TV + q t u,• -π G 2 (•) TV ≤ t mix (G 2 ) 2 √ n + q t u,• -π G 2 (•) TV ≤ o(1) + q t u,• -π G 2 (•) TV ≤ o(1) + 1/e,
where the last inequality follows form the definition of mixing time. Again, by the triangle inequality,

p t u,• -p t v,• TV ≤ o(1) + 2/e.
Based on Lemma 8.10, we can now bound the hitting time to reach z, which will later be used to establish the bounds on the mixing and meeting time of the whole graph G. But first, we prove that the mixing time of the graph G induced by V 2 ∪ { z} is constant and that after mixing on G, the random walk has a probability of Ω(1/n) to hit z in a constant number of time steps.

Lemma 8.11. The following three statements hold.

(i) Let G be the induced graph by the vertices

V 2 ∪ { z}. Then t mix ( G) = O(1). (ii) Let u ∈ V \ { z}. Then there exists a constant c ≥ 1 such that P[ T hit (u, z) ≥ n/c ] ≥ 1/2. (iii) Let u ∈ V \ { z}. Then t hit (u, z) = O(n).
Proof. We prove the statements one by one. (8.12)

For any such D ∈ D, we have, by definition of the total variation distance, Exercise 4.1] we have the following identity for d Q (t). Let D * be the set of all distributions over V (G 2 ), then

q t u∼D ,• -π G (•) TV = 0 + 1 2 v∈V 2 q t u∼D ,v -π G (v) + 1 2 q t u∼D , z -π G ( z) . For u ∈ V 2 observe that π G (u) ∈ [π G 2 (u)(1 -ζ), π G 2 (u)(1 + ζ)] for some ζ = o(1). By [LPW06,
d Q (t) = max D∈D * q t u∼D,• -π G 2 (•) TV ≥ max D ∈D q t u∼D ,• -π G 2 (•) TV .
Thus, for δ v := | q t u,v -q t u,v |, we get by using triangle inequality,

1 2 v∈V 2 q t u∼D ,v -π G (v) ≤ 1 2 v∈V 2 q t u∼D ,v -π G 2 (v) + 1 2 v∈V 2 |π G 2 (v) -π G (v)| ≤ 1 2 v∈V 2 q t u∼D ,v -π G 2 (v) + 1 2 v∈V 2 π G 2 (v)ζ ≤ 1 2 v∈V 2 q t u∼D ,v -π G 2 (v) + 1 2 v∈V 2 |δ v | + 1 2 v∈V 2 π G 2 (v)|ζ| ≤ d Q (t) + 1/32 + ζ 2 , ≤ d Q (t) + 1/32 + 1/32, (8.13)
where the second-last inequality is due to Lemma 8.10.(ii),

1 2 v∈V |δ v | ≤ t/(2 √ n) ≤ 1 32
. By definition of the t mix (G 2 ) and by sub-multiplicativity we have

d Q (t) ≤ d Q (2t mix (G 2 )) ≤ 1/e 2 .
The above equation (8.13) only consider the variation distance w.r.t. V 2 . For z we

have 1 2 | q t u∼D , z -π G ( z)| ≤ (2t mix (G 2 ) + 7)/ √ n ≤ 1/32.
Putting everything together we get we get Consider the random walk starting at z and let (X 0 , X 1 , . . . ) denote its trajectory.

q t u∼D ,• -π G (•) TV = 1 2 v∈V 2 q t u∼D ,v -π G (v) + 1 2 q t u∼D , z -π G ( z) ≤ d Q (t
Observe that at time 7 we have

q 7 z, z ≤ 1 2 7 + i≤7 v∈N ( z) q i-1 z,v • 1 2( √ n + 1) ≤ 1 2 7 + 7 2 √ n ≤ 1/32.
The set of distribution for the position of the random walk at time 7 conditioning on X 7 = z gives the same distribution D as defined in (8.12). Let D z ∈ D be distribution of the random at time 7 starting at z. Hence, by(8.14), we get

q 2t mix (G 2 )+7 z,• -π G (•) TV ≤ q z,V ( G)\{ z} • q 2t mix (G 2 ) u∼D z ,• -π G (•) TV + q z, z • 1 (8.16) ≤ 1 • (1/e 2 + 3/32) + 1/32 ≤ 1/e.
(8.17)

Thus, for t = 2t mix (G 2 ) + 7 we have q t z,• -π G (•) TV ≤ 1/e. Together with (8.14), we conclude that for all u ∈ V , q t u,• -π G (•) TV ≤ 1/e and by definition of t mix and we get t mix ( G) ≤ 2t mix + 7 = O(1).

(ii) To prove P[ T hit (u, z) ≥ n/c ] ≥ 1/2 for u ∈ V 2 we show that the random walk restricted to G does not hit z after n/c 1 steps w.c.p. for some large enough constant c 1 . By the Union bound, for some large constants c 1 , c 2 that

P T G hit (u, z) ≤ n/c 1 = P T G hit (u, z) ≤ n/c 1 ≤ n/c 1 t=1 q t u, z ≤ c 2 log n t=1 1/ √ n + n/c 1 t=c 2 log n q t u, z ≤ o(1) + n/c 1 • (π G ( z) + 1/n 2 ) ≤ 1/2,
where we used

q t u, z ≤ π G ( z) + π G ( z) π G (u) λ 2 ( G) t (Proposition A.26).
We proceed by bounding that

P[ T hit (u, z) ≥ n/c 1 ] ≥ 1/2 for u ∈ V 1 .
Consider first a random walk ( Xt ) t≥0 restricted to G 1 1 = G 1 that starts at vertex z 1 and let P denote the transition matrix. Furthermore, in order to couple the random walk Xt restricted to G 1 with a random walk in G, we will consider the random variable

Z := t G 1 sep t=0 1 Xt=z1 . Since G 1 is a clique, t G 1 sep = O(1), and pt z 1 ,z 1 ≤ 1 √ n + λ 2 (G 1 )
t by Proposition A.26, where λ 2 (G 1 ) is some constant bounded away form 1. Therefore,

E Z = n/c 1 t=0 pt z 1 ,z 1 ≤ 2 √ n/c 1 . Let γ := 4 • E Z .
Then, by Markov's inequality

P Z ≥ γ ≤ 1/4.
Consider now the straightforward coupling between a random walk (X t ) t≥1 in G that starts at vertex z 1 and the random walk ( Xt ) t≥1 restricted to G i 1 that starts at the same vertex. Whenever the random walk Xt is at a vertex different from z 1 , then the random walk X t makes the same transition. If the random walk Xt is at vertex z 1 , then there is a coupling so that the random walk X t makes the same transition as Xt with probability 2 √ n-1 2 √ n . Conditional on the event Z ≤ γ occurring, the random walk Xt follows the random walk X t up until step n/c 1 with probability at least

p := 2 √ n -1 2 √ n γ ≥ 3/4,
since the random walk Xt has at most γ visits to z 1 . Therefore, by the Union bound,

P T G hit (u, z) ≥ n/c 1 ≥ P ∪ n/c 1 t=0 X t = Xt ≥ 1 -P Z ≥ γ -(1 -p) ≥ 1/2
and the proof is complete.

(iii) We proceed by showing

t hit (u, z) = O(n) for u ∈ V 2 .
Let Q be the transition matrix of the random walk restricted to G 2 . Let u ∈ V 2 and S * = N ( z) be the neighbors of z in G 2 . For every v ∈ S * we have

π G 2 (v) = √ n+1 n √ α √ n+ √ n √ α ≥ √ α
1.2n . Hence, after t = t sep (G 2 ) we have that

q t u,S * := v∈S * q t u,v ≥ v∈S * π G 2 (v)(1 -e -1 ) ≥ √ n √ α • √ α 1.2n (1 -e -1 ) = 1 -e -1 1.2 √ n .
By Lemma 8.10, we have for any

u ∈ V 2 that p t u,• -q t u,• TV ≤ t sep (G 2 )/(2 √ n). To bound T G hit (u, z)
we show that after t sep + 1 = O(1) steps the random walk hits z w.p. Ω(1/n).

We distinguish between two cases. (a) For all i ≤ t we have p t u,S * ≤ 1/t sep (G 2 ). Thus, by Lemma 8.10.(i)

p t u,S * = v∈S * p t u,v ≥ q t u,S * -p t u,• -q t u,• TV ≥ 1 -e -1 1.2 √ n - t-1 i=1 p i u,S * /(2 √ n) ≥ 1 -e -1 1.2 √ n - t sep (G 2 ) t sep (G 2 )2 √ n = Ω(1/ √ n).
Hence, the random walk hits z after t sep (G 2 )+1 w.p. at least p t u,S * •min v∈S * {p v, z } = Ω(1/n).

(b) Otherwise there exists a t * such that p t * u,S * > 1/t sep (G 2 ). Thus the random walk hits z after t sep (G 2 ) + 1 w.p. at least

p t * u,S * • min v∈S * {p v, z } = Ω(1/n).
Thus after O(1) steps the random walk hits z w.p. Ω(1/n).

We now show a similar statement if u ∈ V 1 . Let (X t ) t≥0 be a random walk on G starting on u. Observe that X t (the walk on G) hits z with probability

p 1 u,z 1 • p 1 z 1 , z = Ω(1/n) in 2 time steps. Hence, for any u ∈ V we P[ T hit (u, z) = O(1) ] = Ω(1/n).
Thus, repeating this iteratively and using independence yields

t hit (u, z) = O(n) for u ∈ V .
To establish a bound on the mixing time of G, we will make use of a following result of Peres and Sousi [PS15] (Theorem A.31 in Appendice A) to relate the mixing time of a graph to the hitting time of large sets. Peres and Sousi [START_REF] Peres | Mixing Times are Hitting Times of Large Sets[END_REF] show the following. For any β < 1/2, let t H (β) = max u,A:π(A)≥β t hit (u, A). Then there exist positive constants c β and 123

c β such that c β • t H (β) ≤ t mix (1/4) ≤ c β • t H (β).
In the following we show for any β close enough to 1/2, that any A ⊆ V satisfying π(A) ≥ β must include at least a constant fraction of nodes from a constant fraction of copies of G 1 . Claim 8.12. Let β = 1/2 -10 -3 . For any

A ⊆ V with π(A) ≥ β, define H(A) = {i | |G i 1 ∩ A| ≥ |V 1 |/(2e)}. Then, |H(A)| ≥ κ/(2e).
Proof. This follows from a simple pigeon-hole argument: Suppose |H(A)| < κ/(2e) was true. Then,

π(A) ≤ |H(A)| • π(V 1 ) + (κ -|H(A)|) • π(V 1 ) 2e + π(z i ) + π(V 2 ) + π( z) < κ 2e • π(V 1 ) + κ • π(V 1 ) 2e + π(z i ) + 1/20 < β ≤ π(A),
which is a contradiction and hence choice of A must fulfill |H(A)| ≥ κ/(2e).

We are now ready to determine the mixing time of G. The lower bound is a simple application of Cheeger's inequality, while the upper bound combines the previous lemmas with Theorem A.31.

Lemma 8.13. Let G be the graph described at the beginning of Section 8.3.5. We have

t mix (G) = Θ(n).
Proof. First we show

t mix = Ω(n). The conductance of G = (V, E) is defined by Φ(G) = min U ⊆V, 0<vol(U )≤vol(V )/2 |E(U,V \U )| vol(U ) . In particular, for U = V 1 we get that Φ(G) ≤ 4 n .

Hence, by

Cheeger's inequality and

1 1-λ 2 (G) -1 • log( e 2 ) ≤ t mix (1/e) (see, e. g., [LPW06, Chapter 12]), n 4 ≤ 1 Φ(G) ≤ 2 1 -λ 2 (G) = 2 1 -λ 2 (G) -2 + 2 ≤ 2t mix log e 2 + 2.
Rearranging the terms yields t mix = Ω(n).

We proceed with the upper bound on the mixing time. Let β = 1/2 -10 -3 and let A ⊆ V be an arbitrary set satisfying π(A) ≥ β. First, we apply Claim 8.12 to conclude that

|H(A)| ≥ κ/(2e). This immediately implies that with

Z := {z i : i ∈ H(A)}, |Z| ≥ κ/(2e).
The remainder of the proof is divided into the following three parts:

(i) Starting from any vertex u ∈ V , with probability at least 1/2, the random walk hits z * after 2 max u∈V t hit (u, z) = O(n) steps.

(ii) With constant probability p 1 > 0, the random walk moves from z * to a vertex in Z.

(iii) With constant probability p 2 > 0 a random walk starting from a vertex in Z will hit A after one step.

It is clear that combining these three results shows that with constant probability 1 2 p 1 p 2 > 0, a random walk starting from an arbitrary vertex u ∈ V hits a vertex in A after O(n)+1+1 time steps. Iterating this and using independence shows that t hit (u, A) = O(n), and hence by Theorem A.31, t mix = O(n) as needed.

Part (i).

Consider max u∈V t hit (u, z). For u ∈ V , Lemma 8.11.(iii) implies t hit (u, z) = O(n).

Part (ii).

If the random walk is on z * , then since deg(z * ) = κ + n/α , |Z| ≥ κ/(2e), it follows that the random walk hits a vertex in Z after one step with constant probability

p 1 := |Z| 2(κ+ √ n/α ) > 0.
Part (iii). Finally, for any z ∈ Z we have that

p 2 = p z,A = |V 1 |/(2e) 2 √ n
> 0 and the proof is complete.

In the following we establish the bound on the meeting time. As it turns out, any meeting is very likely to happen on V 2 and it takes about Θ(α n) time steps until both walks reach V 2 simultaneously. The lower bound then follows from our common analysis method (8.1). The upper bound combines the mixing time bound of O(n) (Lemma 8.13), and that once a random walk reaches a copy of G 1 , it says there for Θ(n) steps with constant probability Lemma 8.11.(ii). Lemma 8.14. Let G be the graph described at the beginning of Section 8.3.5. We have

t meet (G) = Θ(α n).
Proof. We start by proving t meet = Ω(α n): Consider two non-interactingDEGreplace these by something, random walks with starting positions drawn from the stationary distribution π. Let = c α n, for some small enough constant c > 0. Let Z 1 be the number of collisions of the two random walks on the nodes in

V 1 1 ∪ V 2 1 ∪ • • • ∪ V κ 1 .
Let Z 2 be the number of collisions of the two random walks on the nodes in V 2 . Let Z * be the number of collisions of the two random walks on the node z.

Let Z be the number of collisions of the two walks during the first time steps, i. e.,

Z = Z 1 + Z 2 + Z * .
Using the Union bound we derive

P[ Z ≥ 1 ] ≤ P[ Z 1 ≥ 1 ] + P[ Z 2 ≥ 1 ] + P[ Z * ≥ 1 ] ≤ E[ Z 1 ] E[ Z 1 | Z 1 ≥ 1 ] + E[ Z 2 ] E[ Z 2 | Z 2 ≥ 1 ] + E[ Z * ] E[ Z * | Z * ≥ 1 ] . (8.18)
We have

E[ Z 1 ] ≤ n 2 n 2 , E[ Z 2 ] ≤ n √ α 2 n 2
, and

E[ Z * ] ≤ 2 n 2 , since max u π(u) ≤ 2/n.
p t u,v = 1 implying that v∈V 1 ( p t u,v ) 2 ≥ v∈V 1 1 |V 1 | 2 = 1/|V 1 |. Hence, we get E[ Z 1 | Z 1 ≥ 1 ] ≥ E[ Z 1 | Z 1 ≥ 1, E 1 ]•P[ E 1 ] ≥ 1 2 min u∈V 1 n/c-1 t=0 v∈V 1 ( p t u,v ) 2 ≥ 1 4 n/c-1 t=0 1/|V 1 | = √ n 4c
.

Using an exactly analogous analysis for Z 2 we can upper bound E[ Z 2 | Z 2 ≥ 1 ] as follows:

E[ Z 2 | Z 2 ≥ 1 ] ≥ E[ Z 2 | Z 2 ≥ 1, E 2 ]•P[ E 2 ] ≥ 1 4 min u∈V 2 n/c-1 t=0 v∈V 2 ( p t u,v ) 2 ≥ 1 4 n/c-1 t=0 1/|V 2 | = √ α 4c ,
where E 2 is the event that for u ∈ V 2 we have T hit (u, z) ≥ n/c for some large enough constant c. Plugging everything into (8.18) and using = c α n yields

P[ Z ≥ 1 ] ≤ E[ Z 1 ] E[ Z 1 | Z 1 ≥ 1 ] + E[ Z 2 ] E[ Z 2 | Z 2 ≥ 1 ] + E[ Z * ] E[ Z * | Z * ≥ 1 ] ≤ n 2 n 2 √ n 4c + n √ α 2 n 2 √ α 4c + 2 n 2 1 ≤ o(1) + 16c • c + o(1) ≤ 1/2,
for any constant c ∈ (0, 1 33c ]. This finishes the proof of t meet = Ω(α n). In the remainder we prove t meet = O(α n). Consider two independent walks (X t ) t≥0 and (Y t ) t≥0 on G, both starting from arbitrary nodes. Note t sep = t sep (G) ≤ 4t mix = O(n) by Lemma 8.13, and

p 0 := P X tsep ∈ V 2 ∩ Y tsep ∈ V 2 ≥   u∈V 2 (1 -e)π(u)   2 = Ω 1/ √ α 2 = Ω 1/α .
We assume in the following that

X tsep ∈ V 2 ∩ Y tsep ∈ V 2 . We have t mix (G 2 ) = O(
E Z ≤ t mix (G 2 ) + tsep+n/c-1 t=tsep+t mix (G 2 )+1 |N ( z)|(π G 2 (z) + d P (t)) ≤ t mix (G 2 ) + |N ( z)|(n/c) + O(1) ≤ (1 + 1/e) √
∈ N ( z),
then there is a coupling so that the random walk X t makes the same transition as Xt with probability 2 √ n 2 √ n+2 . Conditional on the event { Z ≤ γ} occurring, the random walk Xt follows the random walk X t up until step n/c with probability at least

p 1 := 2 √ n 2 √ n + 2 γ = 1 - 1 √ n + 1 γ ≥ 3
p 2 := P Xt+tsep(G 2 ) = Ỹt+tsep(G 2 ) | F t ≥ (1 -1/e) 2 π G 2 2 2 ≥ √ α 8n .
Recall that α ≥ 2 20 t sep (G 2 ) 2 by definition. Therefore, the probability that Xt and Ỹt do not meet in the time-interval [t sep (G 1 ), t sep (G 1 ) + n/c -1] is at most

p 3 := (1 -p 2 ) n/(tsep(G 2 )c) ≤ (1 -p 2 ) 2 10 n/( √ α c) ≤ 1/4.
Therefore, by the Union bound,

P ∪ tsep(G 1 )+n/c-1 t=0 X t = Y t ≥ p 0 • 1 -P Z ≥ γ -2 • (1 -p 1 ) -p 3 = Ω(α ).
Repeating this O(1/p 3 ) times and using the independence yields that the expected meeting time is O((t sep (G 1 ) + n/c -1)/p 3 ) = O(α n) and the proof is complete.

Finally, we analyze the coalescing time of G. The proof idea is to consider 5 √ n random walks starting from π and show that meetings only occur on V 2 and that at least one random walk requires Ω( √ α • n log n) time steps to reach V 2 .

Lemma 8.15. Let G be the graph described at the beginning of Section 8.3.5. We have

t coal (G) = Ω( √ α • n log n).
Proof. Let ε = 1/5. We show that even the coalescing time of n ε random walks requires Ω( √ α • n log n) time steps w.c.p.. Let R be a collection of n ε independent, i. e., noninteracting, random walks with starting positions drawn from the stationary distribution π. We define the following three bad events: (i) Let E 1 be the event that any of the n ε random walks meet on a node

V \ V 2 in √ α • n log 2 n steps.
(ii) Let E 2 be the event that fewer than n ε /4 random walks start on copies of G 1 , i. e., on nodes V \ (V 2 ∪ z).

(iii) Let E 3 be the event that all random walks starting from a copy of G 1 require fewer

than c • √ α • n log n time steps for leaving V \ (V 2 ∪ z * )
for some constant c > 0 to be determined later.

In the following we show that P[ E 1 ] = o(1), P[ E 2 ] = o(1), and P E 3 | E 2 < 1/e, which implies, by union bound,

P E 1 ∩ E 2 ∩ E 3 ≥ P E 1 -(1 -P E 2 ∩ E 3 ) ≥ 1 -o(1) -1 -(1 -o(1)) • 1 - 1 e ≥ 1 - 1 2e .
Conditioning on E 1 ∩ E 2 ∩ E 3 , none of the independent random walks meet on any node V \ V 2 and hence they are indistinguishable from coalescing random walks until they reach V 2 . Therefore, it is necessary for all random walks to reach G 2 in order to coalesce. Hence, we conclude that t coal (G) = Ω( √ α • n log n) yielding the lemma.

(i) We now prove P[ E 1 ] = o(1). Consider any pair of the random walks R. Since both random walks start from the stationary distribution, the probability for them to meet on a node on z in a fixed step t ≥ 0 is at most O(1/n 2 ).

Hence, by the Union bound over n 2 pairs of random walks and √ α •n log 2 n ≤ n log 3 n steps, the probability of any two random walks meeting on z is at most 1), since = 1 5 . Furthermore, the probability that no two walks start on the same copy of G 1 is at most p 1) by the Union bound. Moreover, using a Chernoff bound together with Lemma 8.11.(ii), it follows that a random walks visits the vertex z * at most 10 log 3 n times during n log 3 n steps with probability at least 1 -n -2 . By the Union bound over all random walks, it follows that w.p. at least 1 -n -1 , each random walk visits at most 10 log 3 n different copies of G 1 , and by construction of G each such copy is chosen uniformly and independently at random among G 1 1 , G 2 1 , . . . , G κ 1 . Therefore, the probability that there exists a copy of G 1 which is visited by at least two random walks in n log 3 n steps is at most

p 1 := n 2 • n log 3 n • O(1/n 2 ) = o(
2 := n ε • n ε √ n = o(
p 3 := n -1 + n ε (10 log 3 n + 1) • n ε (10 log 3 n + 1) √ n = o(1). (8.19)
Putting everything together, using union bound, yields

P[ E 1 ] ≤ p 1 + p 2 + p 3 = o(1).
(ii) We now prove P[ E 2 ] = o(1). The probability p for each random walk to start on a

node of V \ (V 2 ∪ z) is π(V \ (V 2 ∪ z)) ≥ 1/2.
For each of the random walks with label 1 ≤ i ≤ n ε we define the indicator variable X i to be one, if that random walk

starts on V \ (V 2 ∪ z). Let X = n ε i=1 X i . We have E[ X ] = n ε • E[ X i ] ≥ n ε /2.
Since the starting positions of the n ε random walks are drawn independently, by a Chernoff bound

P[ E 2 ] = P X ≤ 1 4 n ε ≤ P[ X ≤ E[ X ]/2 ] ≤ e -n ε /16 = o(1).
(iii) We now prove P E 3 | E 2 < 1/4. From Lemma 8.11.(ii) we get that w.p. at least 1/2 a random walk starting at any node u ∈ V 1 does not leave G 1 , i. e., does not reach z * , after c 1 n time steps for some constant c 1 > 0. It is easy to see that the number of visits to z required before the random walk hits G 2 instead of returning to G 1 is w.c.p. at least √ α /2; this is because the fraction of edges from z to G 2 is n/α /( n/α + √ n). Using a Chernoff bound, we conclude that any random walk starting at G 1 doesn't hit G 2 during the first T = c 1 • √ α n/2 time steps with constant probability p > 0. Thus the probability that a random walk does not reach G 2 after λ • T time steps is at least p λ , for any integer λ ≥ 1. Setting λ = • log(1/p) • log(n/4), the probability that all of the at least 1 4 n ε random walks starting from

G 1 reach G 2 within λ • T = Ω( √ α • n log n) steps is P E 3 | E 2 ≤ (1 -p λ ) 1 4 n ε ≤ 1/e,
completing the proof.

The following lemma establishes a bound on the average hitting time.

Lemma 8.16. Let G be the graph described at the beginning of Section 8.3.5. We have

t avg-hit = Ω(n 3/2 )
Proof. Consider a random walk that starts from an arbitrary vertex u ∈ V . By Lemma 8.11.(ii), every time a vertex z i is visited, with probability at least c > 0 it takes Ω(n) time steps to visit another vertex z j , j = i. Using a Chernoff bound, it follows that with probability larger than 1/2 it takes at least Ω(n 3/2 ) time steps to visit at least half of the nodes in {z 1 , z 2 , . . . , z κ }. By symmetry, it follows that for every vertex in a copy of G 1 there are Ω(n) vertices to which the hitting time is Ω(n 3/2 ). Thus, by symmetry,

t avg-hit = u,v∈V π(u) • π(v) • t hit (u, v) = Ω(n 2 1 n 2 n 3/2 ) = Ω(n 3/2 ).
Chapter 9

3-Majority [BCE+17]

In the 3-Majority, each node of a graph has an opinion, and in every round each node chooses independently three random neighbours and adopts the opinion of the majority where ties are broken arbitrarily. We are interested in the consensus time, which is the first point in time where all nodes have the same opinion. The system consists of n anonymous nodes connected by a complete graph. Initially, each node supports one opinion from the set [k] := { 1, . . . , k }. We refer to these opinions as colors. The system state is modeled as a configuration vector c, whose i-th component c i denotes the number (support) of nodes with color i.

A consensus process is specified by an update rule that is executed by each node. The so-called Voter process (also known as Polling), uses the most naïve update rule: In every round, each node samples one neighbor independently and uniformly at random and adopts that node's color. Two further natural and prominent consensus processes are the 2-Choices and the 3-Majority process. Their corresponding update rules, as executed synchronously by each node, are as follows:

• 2-Choices: Sample two nodes independently and uniformly at random. If the samples have the same color, adopt it. Otherwise, ignore them and keep your current color.

• 3-Majority: Sample three nodes independently and uniformly at random. If a color is supported by at least two samples, adopt it. Otherwise, adopt the color of one of them at random 1 .

One reason for the interest in these processes is that they represent simple and efficient self- This highlights the fact that 3-Majority is a combination of 2-Choices and Voter: Each node u performs the update rule of 2-Choices. If the sampled colors do not match, instead of keeping its color, u executes the update rule of Voter. Interestingly enough, both 3-Majority and 2-Choices behave identical in expectation2 . In comparison to Voter, both 2-Choices and 3-Majority exhibit a drift: they favor colors with a large support, for which it is more likely that the first two samples match. In particular, if there is a certain initial bias3 towards one color, Voter still needs linear time (in n) to reach consensus, while both 2-Choices and 3-Majority can exploit the bias to achieve sublinear time. On the other hand, it is unknown how 2-Choices and 3-Majority behave when they start from configurations having a large number of colors and no (or small) bias since for neither of the models reasonable bounds are unknown in the general setting with up to n colors.

Results

In this chapter, we give the first unconditional sublinear bound on any of these processesan open issue from, e.g., [START_REF] Becchetti | Stabilizing Consensus with Many Opinions[END_REF]).

The following theorem states slightly simplified version of our upper bound (see Theorem 9.8.

Theorem 9.1 (Simplified). Starting from an arbitrary configuration, 3-Majority reaches consensus with high probability in O n 3/4 log 7/8 n rounds.

The proof is more based on a combination of various techniques and results from different contexts. This approach not only results in a concise proof of the upper bound, but yields some additional, interesting results along the way. We give a brief overview of our approach in the next paragraph.

Should the bound of Ghaffari and Lengler [START_REF] Ghaffari | Tight Analysis for the 3-Majority Consensus Dynamics[END_REF] carry over from 2-Choices to 3-Majority, then together with our results (see Theorem 9.8 and Section 9.4), we would get that the expected consensus time is O(k log n) for any k.

Approach and Technical Contributions

To derive our upper bound on the time to consensus required by 3-Majority, we split the analysis in two phases: (a) the time needed to go from n to ≈ n 1/4 colors and (b) the time needed to go from ≈ n 1/4 to one color. The runtime of the second phase follows by a simple application of [START_REF] Becchetti | Stabilizing Consensus with Many Opinions[END_REF] and is Õ n 3/4 . Bounding the runtime of the first phase is more challenging: we cannot rely on the drift from a bias or similar effects, and it is not clear how to perform a direct analysis in this setting (3-Majority is geared towards biased configurations). To overcome this issue, we resort to a coupling between Voter and 3-Majority. Since the construction of such a coupling seems elusive, we use some machinery from majorization theory [START_REF] Marshall | Inequalities: Theory of Majorization and Its Applications[END_REF] to merely prove the existence of the coupling (see next paragraph). As a consequence of (the existence of) this coupling, we get that the time needed by 3-Majority to reduce the number of colors to a fixed value is stochastically dominated by the time Voter needs for this (Proposition 9.10). This, finally, allows us to upper bound the time needed by 3-Majority 4 to go from ≈ n to ≈ n 1/4 colors by the time Voter needs for this (which, in turn, we bound in Lemma 9.12 by Õ(n/k)).

The technically most interesting part of our analysis is the proof of the stochastic dominance between 3-Majority and Voter. It works for a wide class of processes (including Voter and 3-Majority), which we call anonymous consensus (AC-) processes (see Definition 9.2).

These are defined by an update rule that causes each node to adopt any color i with the same probability α i that depends only on the current frequency of colors.

In the following, we provide a natural way to compare two processes. First, we define a way to compare two configurations c and c . We use vector majorization for this purpose: c majorizes c (c c ) if the total support of the j largest colors in c is not smaller than that in c for all j ∈ [k]. In particular, note that a configuration where all nodes have the same color majorizes any other configuration. Intuitively, this can be thought of as a potential φ with φ(c) ≥ φ(c ) if and only if c c . Let us write P (c) for the (random) configuration obtained by performing one step of a process P on configuration c. Consider two processes P, P and two configurations c, c with c c . We say P dominates P if, for all j ∈ [k], the following holds:

For every pair of color distributions c, c such that φ(c ) ≤ φ(c) after one-step P remains closer towards consensus than

P , i. e., E[ φ(P (c )) ] ≤ E[ φ(P (c)) ].
Note that this definition is not restricted to AC-processes.

Our main technical result (Theorem 9.4) proves that, for two AC-processes, P dominating P implies that the time needed by P to reduce the number of colors to a fixed value stochastically dominates the time P needs for this. Note that while this statement might sound obvious, it is not true in general (if one of the processes is not an AC-process):

2-Choices dominates Voter, but it is much slower in reducing the number of colors when there are many colors.

Consensus Model & Technical Framework

This section introduces our technical framework using concepts from majorization theory, which is used in Section 9.4 to derive the sublinear upper bound on 3-Majority. We provide a few definitions and state the main result of this section (Theorem 9.4).

Comparing Anonymous Consensus Processes

We first define a class of processes defined by update rules that depend only on the current configuration. The update rule states that each nodes adopts a color i with the same probability α i (c), where c ∈ C is the current configuration. In particular, node IDs (including the sampling node's ID) do not influence the outcome. In this sense, such update rules are anonymous.

Definition 9.2 (Anonymous Consensus Processes). Given a distributed system of n nodes, an anonymous consensus process P α is characterized by a process function α :

C → [0, 1] n with i∈[n] α i (c) = 1 for all c ∈ C. When in configuration c ∈ C, each node independently adopts opinion i ∈ [k]
with probability α i (c). We use the shorthand AC-processes to refer to this class.

Given an AC-process P α and a fixed initial configuration, let5 P α (t) denote the configuration of P α at time t. By Definition 9.2, P α (t) t≥0 is a Markov chain, since P α (t) depends only on P α (t -1). Another immediate consequence of Definition 9.2 is that P α (t) conditional on P α (t -1) = c is distributed according to Mult n, α(c) ; in other words, the 1-step distribution of an AC-process is a multinomial distribution. Two important examples of AC-processes include Voter and 3-Majority:

• In the Voter process P α (V) , each node samples one node (according to the pull mechanism) and (always) adopts that node's opinion. Thus

α (V) i (c) = c i n . (9.1)
• In the 3-Majority process P α (3M ) , each node samples independently and uniformly at random three nodes. If a color is supported by at least two of the samples, adopt it. Otherwise, adopt a random one of the sampled colors. Simple calculations (see [START_REF] Becchetti | Trevisan: Simple Dynamics for Plurality Consensus[END_REF]) show

α (3M ) i (c) = c i n • 1 + c i n - c n 2 2 . (9.2)
For any protocol P starting with configuration c ∈ C let T κ P (c) denote the first time step where the number of remaining colors reduces to κ where κ ∈ N. The next definition introduces dominance between protocol. Intuitively, a protocol P dominates another protocol P if their expected behavior preserves majorization. Definition 9.3 (Protocol Dominance). Consider two (not necessarily AC-) processes P, P . We say P dominates P if for all c, c ∈ C with c c we have that E[ P (c) ] E P (c) holds.

Note that, in the case of AC-protocols, Definition 9.3 can be stated as follows: P α dominates P α if and only if c c ⇒ α(c) α(c) for all c, c ∈ C with c c. With this, the main result of our framework can be stated as follows.

Theorem 9.4. Consider two AC-Processes P and P where P dominates P . Assume P and P are started from the same configuration c ∈ C. Then, for any κ ∈ N, the time needed by P to reduce the number of remaining colors to κ dominates the time P needs for this, i.e.,

T κ P (c) ≥ st T κ P (c).
One should note that the statement of Theorem 9.4 is not true in general (i.e., for non-AC-processes). In particular, 2-Choices dominates Voter, but our upper bound on Voter (Lemma 9.12) and our lower bound on 2-Choices (Theorem 10.5 in Chapter 10) contradict the statement of Theorem 9.4: The 2-Choices process takes Ω(n/ log n) time steps to reduce to k = n/ log n opinions whereas Voter reduces to k w.h.p. in only

O(n/k log n) = O(log 2 (n)) time steps.

Coupling two AC-Processes -Proof of Theorem 9.4

In order to prove Theorem 9.4, we formulate a strong 1-step coupling property for ACprocesses: Since the configuration space C is a finite subset of R n , it is closed and so is { (x, y) | x y }.

We now apply Theorem 9.7 (Strassen's Theorem) to get that there exists a coupling between X and Y such that6 X Y. This finishes the proof.

Note that Theorem 9.4 is an immediate consequence of Lemma 9.5: Since P dominates P (which is, for AC-processes, equivalent to α(c) α(c) for all c, c with c c) we can apply Lemma 9.5 iteratively to get Theorem 9.4. The fine-grained comparison enabled by Lemma 9.5 is based on three observations:

1. The (pre-) order " " on the set of configurations naturally measures the closeness to consensus. Indeed, a configuration with only one remaining color is maximal with respect to " ". Similarly, the n-color configuration is minimal.

2. We can define a vector variant "≤ st " of stochastic domination (see Definition 9.6)

such that Θ 1 Θ 2 ⇒ Mult m, Θ 1 ≤ st Mult m, Θ 2 ([MOA11, Proposition 11.E.11]
or Proposition A.20).

Consider two configurations

c, c ∈ C with α(c) α(c). Since P α (c) ∼ Mult(n, α(c))
and P α ∼ Mult(n, α(c)), the previous observations imply that one step of P α on c is stochastically "better" than one step of P α on c. Our goal is to apply Lemma 9.5 iteratively to get Theorem 9.4. For this, we prove a coupling showing majorization between the resulting configurations. We achieve this via a variant of Strassen's Theorem (see Theorem 9.7 below), which translates stochastic domination among random vectors to the existence of such a coupling.

We now give a definition of stochastic majorization that is compatible with the preorder " " on the configuration space C (cf. [MOA11, Chapter 11]).

Definition 9.6 (Stochastic Majorization). For two random vectors X and Y in R d , we write X ≤ st Y and say that Y stochastically majorizes

X if E[ f (X) ] ≤ E[ f (Y) ] for all
Schur-convex functions f on R d such that the expectations are defined.

We proceed by stating the aforementioned variant (Theorem 9.7) of Strassen's Theorem (Theorem A.21).

Theorem 9.7 (Strassen's Theorem (variant)). Consider a closed subset A ⊆ R n such that the set { (x, y) | x y } is closed. For two random vectors X and Y over A, the following conditions are equivalent:

(i) (Stochastic Majorization) X ≤ st Y and
(ii) (Coupling) there is a coupling between X and Y such that P[ X Y ] = 1.

Proof. Consider the cone

C := { f : A → R | f is Schur-convex }
of real-valued Schur-convex functions on A. This cone implies a preorder "≤ C " on A by the definition x ≤ C y :⇔ f (x) ≤ f (y) for all f ∈ C. One can show that this preorder is the vector majorization " " (cf. [MOA11, Example 14.E.5])7 . Now, "≤ C " being equal to " " has two implications:

(a) The stochastic majorization "≤ st C " implied by the preorder "≤ C " is the stochastic majorization "≤ st " from Definition 9.6 (cf.

Upper Bound for 3-Majority -Proof of Theorem 9.8

In this section, we provide a sublinear upper bound on the time needed by 3-Majority to reach consensus with high probability. This is one of our main results and is formulated in the following theorem.

Theorem 9.8. Starting from any configuration c ∈ C, 3-Majority reaches consensus w.h.p. in O n 3/4 log 7/8 n rounds.

The analysis is split into two phases, each consisting of O n 3/4 log 7/8 n rounds.

Phase 1: From up to n to n 1/4 log 1/8 colors. This is the crucial part of the analysis.

Instead of analyzing 3-Majority directly, we use our machinery from Section 9.3.1 to show that 3-Majority is not slower than Voter (Proposition 9.10). Then, we prove that Voter reaches O n 1/4 colors in O n 3/4 log 7/8 n rounds (Lemma 9.12).

Proof. By Theorem 9.4, all we have to prove is c c ⇒ α (3M ) c α (V) (c) (see Section 9.3.1).

To this end, consider two configurations c, c ∈ C with c c. Let p := α (3M ) c and p := α (V) (c). We have to show p p. Since these are probability vectors, we have p 1 = 1 = p 1 . It remains to consider the partial sums for k ∈ [n]. For this, let x := c/n and x := c/n. Remember that

p i = x 2 i + (1 -x 2 2 ) • x i ((9.
2)) and pi = xi ((9.1)). In the following, we assume (w.l.o.g.) p = p ↓ and p = p↓ (this implies x = x ↓ and x = x↓ ). We compute

k i=1 p i - k i=1 pi = k i=1 x 2 i + k i=1 x i -x 2 2 k i=1 x i - k i=1 xi ≥ k i=1 x 2 i -x 2 2 k i=1 x i . (9.3)
We have to show that this last expression is non-negative, which is equivalent to

x 2 2 ≤ k i=1 x 2 i / k i=1 x i . (9.4)
This holds trivially for k = n (where we have equality). Thus, it is sufficient to show that

( k i=1 x 2 i )/( k i=1 x i ) is non-increasing in k. That is, for any k ∈ [n -1] we seek to show the inequality k+1 i=1 x 2 i k+1 i=1 x i = k i=1 x 2 i + x 2 k+1 k i=1 x i + x k+1 ≤ k i=1 x 2 i k i=1 x i . (9.5)
This inequality is of the form A+x B+x ≤ A B , where A, B, x > 0. Rearranging shows that this is equivalent to x ≤ A/B. Thus, (9.5) holds if and only if

x k+1 ≤ ( k i=1 x 2 i )/( k i=1 x i ). This last inequality holds via x k+1 • k i=1 x i = k i=1 x i • x k+1 ≤ k i=1 x i • x i = k i=1 x 2
i , where we used x = x ↓ . This finishes the proof.

Analysis of Phase 1: A Bound for Voter

We analyze the time the Voter process takes to reduce the number of remaining colors from n to k. One should note that [BGKM16] studies a similar process. However, their analysis relies critically on the fact that their process is lazy (i.e., nodes do not sample another node with probability 1/2), while our proof does not require any laziness.

We make use of the well-known duality (via time reversal) between the Voter process and coalescing random walks. In the coalescing random walks process there are initially n independent random walks, one placed at each of the n nodes. While performing synchronous steps, whenever two or more random walks meet, they coalesce into a single random walk.

Let T k

C denote the number of steps it takes to reduce the number of random walks from n to k in the coalescing random walks process (the coalescence time). Similarly, let T k V denote the number of rounds it takes Voter to reduce the number of remaining colors from n to k. Running the coalescence process from right to left (an edge from u to v means that the token on u -if any-moves to v) yields that after T = 4 rounds the number of random walks reduces to k = 2. Using the same random choices (black arrows) for the voter process and running the process from left to right (an edge from u to v means that u pulls v's opinion) we derive that the number of opinions after T = 4 rounds is also 2. This is no coincidence as we show in Proposition 9.11.

The following lemma uses the high-level idea of the proof presented in [AF02, Chapter 14] which only considers the case k = 1. For the purposes of our proof we would only require a coupling with T k V ≤ T k C , but for the sake of completeness we show the stronger claim

T k V = T k C .
Proposition 9.11. For any graph G = (V, E), there exists a coupling such that

T k C = T k V .
Proof. For t ∈ N and for u ∈ V define the random variables Y t (u) with Y t (u) ∼ uniform(N (u)), where uniform(•) denotes the uniform distribution and N (u) denotes the neighborhood of u. Hence, Y t (u) = v means that u pulls information from node v in step t.

In the Coalescence process, the random variable Y t (u) ∈ N (u), t ∈ [0, T k C ) captures the transition performed by the random walk which is at u at time t (if any). In other words, these random variables define the arrows in Figure 9.1. For the voter process Y t (u) = v means that in step t node u adopts the opinion of v.

Let X(u) = (X 0 (u) = u, X 1 (u), . . . , X T k C (u)
) be the trajectory of the random walk starting at u. We can thus express

X t (u) =    u if t = 0 Y t-1 (X t-1 (u)) otherwise.
(9.6) Thus, this trajectory X(u) and the random variable T k C are completely determined by the random variables

Y = {Y t (u) : t ∈ N, u ∈ V }. Let V T k C
be the Voter process whose starting time t = 0 equals the time T k C of the coalescence process (see also Figure 9

.1). Let O * T k C -t (u) be the opinion of u at time t of V T k C . For every node u ∈ V and t ∈ [0, T k C ] we can thus express O * T k C -t (u) =      u if t = 0 O * T k C -(t -1) (Y T k C -t (u)) otherwise.
(9.7)

Note that (9.7) constructs a coupling between the Voter process and the coalescence process through the common usage of the random variables Y in (9.6) and (9.7). In particular, by unrolling (9.6) and (9.7) we get

X T k C (u) = Y T k C -1 (Y T k C -2 (. . . (Y 0 (X 0 (u))) . . . )) (a) = Y T k C -1 (Y T k C -2 (. . . (Y 0 (u)) . . . )) O * 0 (u) = O * T k C (Y T k C -1 (Y T k C -2 (. . . (Y 0 (u)) . . . ))) (b) = Y T k C -1 (Y T k C -2 (. . . (Y 0 (u) . . . )),
where and (a) we used that X 0 (u) = u and in (b) we used that O *

T k C (v) = v for all v.
The above equations imply

X T k C (u) = O * 0 (u). (9.8)
Let Z t = {X t (u) : u ∈ V } denote the positions of the remaining walks in the coalescence process at time t. Observe that

|Z 0 | = n, |Z T k C | ≤ k, by definition of T k C .
We have, by (9.8), that

Z T k C = {X T k C (u) : u ∈ V } = {O * 0 (u) : u ∈ V } =: O * 0 .
(9.9) From (9.9) we infer

|O * 0 | = |Z T k C | ≤ k, which implies that T k V ≤ T k C .
In the reminder we generalize the previous coupling to show that

T k V = T k C .
In particular, we consider the Voter process for all starting position τ < T k C (all nodes have different colors at round t) and show that the resulting number of opinions is strictly more than k.

Let V τ be the Voter process that starts at time τ ∈ [0, T k C ), and let O τ T k C -t (u) be the opinion of u at time t of V τ . For every node u ∈ V and t ∈ [0, τ ] we have

O τ τ -t (u) =    u if t = τ O τ τ -(t -1) (Y τ -t (u)) otherwise.
(9.10) Similarly as before, by unrolling (9.6) and (9.10) we get

X τ (u) = Y τ -1 (Y τ -2 (. . . (Y 0 (X 0 (u))) . . . )) (a) = Y τ -1 (Y τ -2 (. . . (Y 0 (u)) . . . )) O τ 0 (u) = O τ τ (Y τ -1 (Y τ -2 (. . . (Y 0 (u)) . . . ))) (b) = Y τ -1 (Y τ -2 (. . . (Y 0 (u) . . . )),
where and (a) we used that X 0 (u) = u and in (b) we used that

O τ τ (v) = v for all v. By defining O 0 T k C -t = {O 0 T k C -t (u) : u ∈ V }, from the above equations we get that X τ (u) = O τ 0 (u).
Hence, Given the above duality, we are ready to prove the lower bound on Voter. Lemma 9.12. Consider an arbitrary initial configuration c ∈ C. Voter reaches a configuration c having at most k remaining colors w.h.p. in O n k log n rounds, i.e.,

Z τ = {X τ (u) : u ∈ V } = {O τ 0 (u) : u ∈ V } =: O τ 0 . ( 9
P T k V = O n k log n ≥ 1 -1/n.
Proof. We prove the lemma using the well-known duality (via time reversal) between the Voter process and coalescing random walks.

It is well-known (e.g., [START_REF] Aldous | Reversible Markov Chains and Random Walks on Graphs[END_REF]), that T 1 V = T 1 C . This statement generalizes for all k ∈ [n] (see Proposition 9.11 for a proof) to

T k V = T k C . (9.12)
Thanks to the previous identity, we can prove the lemma's statement by proving that w.h.p.

T k C = O n k log n . To this end, we show that E T k C = O(n/k).
In order to get the claimed bound in concentration, we can apply the following standard argument. Consider the process as a sequence of phases, each one of length 2E T k C . We say that a phase is successful when the number of remaining random walks drops below n/k. Thanks to our bound in expectation above and the Markov inequality, we easily get that every phase has probability Ω(1) to be successful. So, with high probability, there will be at least one success within the first O(log n) phases.

Let X t denote the number of coalescing random walks at time t. We have

X 0 = n and T k C = min { t ≥ 0 | X t ≤ k }.
We seek to apply drift theory (Theorem A.13 to derive a bound on E T k C . Next, we compute an upper bound on E

[ X t+1 -X t | X t = x ]. Let us begin assuming that k is any constant. It holds in general that E[ X t+1 -X t | X t ≥ 2 ] ≤ -1/n,
since in expectation two random walks collide w.p. 1/n in a given time step. Hence we can directly apply8 Theorem A.13 with parameters h(x) = 1/n to reduce from k random walks to 1, yielding the bound

E T k C = O(n/k) = O(n),
where in the latter equality we used that k is constant.

We now consider the case where k is larger than a big constant, say k > 100. Assume that in every time step the random walks move in two phases. Let W 1 denote an arbitrary set of X t /2 random walks and let W 2 denote the remaining ones. We first look at how the random walks in W 1 coalesce, then we consider the movement of the remaining walks W 2 . Let E be the event that the walks in W 1 move onto more than X t /4 distinct nodes. This would imply that each walk in W 2 coalesces with one in W 1 with probability at least

X t /4 /n. We thus have E[ X t+1 | X t = x, E ] ≤ x -x/2 • x/4 n ≤ x - x 2 10n .
Moreover, conditioning on E implies that there were at least X t /2 -X t /4 collisions during the first phase. Thus,

E X t+1 | X t = x, E ≤ x -( x/2 -x/4 ) ≤ x - x 2 10n .
Hence, by law of total expectation,

E[ X t+1 | X t = x ] = E[ X t+1 | X t = x, E ]P[ E ] + E X t+1 | X t = x, E P E ≤ x - x 2 10n .
In order to apply Theorem A.13, we define the random variables (Y t ) t≥0 as follows

Y t =    X t if X t > k, 0 otherwise. Let T * = {t ≥ 0 | Y t = 0}. Since by construction we have Y t = X t for t < T k C and Y T k C = 0 otherwise, it follows that T k C = T * . (9.13)
Therefore,

E[ Y t+1 | Y t = y, Y t > k ] ≤ y - y 2 10n .
We can thus apply Theorem A.13 for the random variables (Y t ) t≥0 with x min = k, x max = n, and h(x) = x 2 10n , obtaining

E[ T * ] ≤ k k 2 /(10n) + n k 1 h(u) ≤ 10n k + 10n - 1 n -- 1 k ≤ 20 n k . (9.14)
Finally, from (9.12), (9.13) and (9.14) we get

E T k V = E T k C = E[ T * ] ≤ 20 n k , ( 9.15)
concluding the proof.

Limitations of 1-Step Coupling

In this section we show that there are configurations c c such that α (hM ) (c) α (h+1M ) (c).

This means that, Lemma 9.5 is not strong enough to derive Conjecture 13.1. Consider the configurations x := (1/2, 1/6, 1/6, 1/6) (1/2, 1/2, 0, 0) =: x (for simplicity, we use the fraction vectors x = c/n). For symmetry reasons, we immediately get that α (h+1M ) (c) =

(1/2, 1/2, 0, 0) = c. However, even for h = 3, for the second configuration we get that the expected fraction of the nodes which adopt the first opinion after one step is

1 • 3 0 • 1 2 3 + 1 • 3 1 • 1 2 2 • 3 6 + 1 3 • 3 2 • 1 2 • 3 6 • 2 6 = 7 12 . (9.16)
The three terms of the sum on the left hand side correspond to the cases and probabilities for which the first color is adopted:

• all samples choose color 1 (probability to win is 1, number of cases 3 0 ),

• two samples choose color 1 (probability to win is 1, number of cases 3 1 ), or

• 1 sample chooses color 1 and the other samples choose different colors (probability to win is 1/3, number of cases 3 2 ).

Thus, for n large enough, with high probability the configuration resulting from (h + 1)-Majority will not majorize the one resulting from h-M ajority.

Chapter 10

2-Choices [EFK+16, BCE+17]

The 2-Choices process works as follows. Each node of a graph has an opinion, and in every round each node chooses independently two random neighbours and adopts their opinion if they coincide; otherwise the node keeps its own opinion. We are interested in the plurality consensus time, which is the first point in time where all nodes have the opinion of the initially most dominant opinion. In this chapter we put emphasize on the difference to

Stability

In our analysis, we will show that the 2-Choices process can tolerate the presence of an adversary which is allowed to arbitrarily change the opinion of up to F = c 1 (c 1 -c 2 )/(8n) arbitrarily selected nodes after every round. We will show that under these assumptions our 2-Choices process still guarantees that with high probability a vast majority of nodes accept the plurality opinion, that is, the initially dominant opinion. Observe that, similarly, all our theorems also hold if the adversary is allowed to change opinions at the beginning of a round. We use a definition similar to the definition by Becchetti et al. [START_REF] Becchetti | Stabilizing Consensus with Many Opinions[END_REF], which in turn has its roots in [START_REF] Angluin | A simple population protocol for fast robust approximate majority[END_REF][START_REF] Angluin | Stabilizing consensus in mobile networks[END_REF].

Definition 10.1. A stabilizing near-plurality protocol ensures the following properties:

1. Almost agreement. Starting from any initial configuration, in a finite number of rounds, the system must reach a regime of configurations where all but a negligible bad subset of nodes of size at most O(n ε ) for some constant ε < 1 support the same opinion.

2. Almost validity. Given a large enough initial bias, the system is required to converge to the plurality opinion A, with high probability, where all but a negligible bad set of nodes have opinion A.

3. Non-termination. In dynamic distributed systems, nodes represent simple and anonymous computing units which are not necessarily able to detect any global property.

Stability.

The convergence to such a weaker form of agreement is only guaranteed to hold with high probability.

Results

The 2-Choices protocol seems to be very efficient if the number of colors is two [START_REF] Cooper | The Power of Two Choices in Distributed Voting[END_REF].

The following result can be seen as an extension of Cooper et al. [START_REF] Cooper | The Power of Two Choices in Distributed Voting[END_REF] on the complete graph when initially the number of opinions is larger than two. That is, we assume that every node of the complete graph G = (V, E) has one of k possible opinions at the beginning,

where k = O(n ) for some small positive constant . Then, the following theorem holds.

Theorem 10.2. Consider the synchronous model on the complete graph with n nodes. Let k = O(n ε ) be the number of opinions for some small constant ε > 0. Assume the initial bias is at least c 1 -c 2 ≥ z • √ n log n for some constant z. Then 2-Choices plurality consensus process converges with high probability to the initially most dominant color within

t 2C = O(n/c 1 • log n)
rounds. Moreover, the process fulfills the stabilizing near-plurality conditions in presence of any F = c 1 (c 1 -c 2 )/(8n)-dynamic adversary.

The difficulty in the analysis lies in the possibly diminishingly small initial mass of A in comparison to the mass of all other colors. Interestingly, the required initial gap does not depend on the number of opinions present. Moreover, we also show that if c 1 -c 2 = O( √ n), then B wins with constant probability. Slightly later, Cooper et al. [START_REF] Cooper | Fast plurality consensus in regular expanders[END_REF] proved the same run time in a much more general form by considering the class of regular expander graphs, albeit assuming a slightly more restrictive initial bias.

Finally, in Theorem 10.5, we show that the it takes Ω(n/ log n) rounds to reach consensus if all nodes start with distinct colors, i. e., in the leader election setting. This shows that 3-Majority is polynomial faster (Chapter 9). On the other side our upper bound on 2-Choices shows that it beats 3-Majority in terms of the required initial bias. We defer the reader to Section 10.5 for an in-depth discussion on the compression between both dynamics.

Approach and Technical Contributions

This chapter rests on the shoulders of careful applications of Chernoff bounds. The main idea of the proof of the upper bound is to show that, by carefully applying Chernoff bounds, the initial bias of color 1 to all other colors increases sufficiently fast. Intuitively, the difficulty lies in the sheer number of initial opinions we allow. In contrast to what is permitted in most previous work, their total mass may significantly exceed the initial mass of c 1 . We denote the number of nodes changing their opinion from color i (C i ) to color j (C j ) by ∆ ij . As mentioned before, we use c 1 , c 2 , . . . , c k to denote the support of the colors {1, 2, . . . , k} at the beginning of a round. We will use c 1 , . . . , c k to denote the number of nodes of corresponding colors after the switching has been performed before the adversary changes F arbitrary nodes. Note that, c 1 , . . . , c k are not necessary monotonically decreasing (as opposed to c 1 , c 2 , . . . , c k ).

Whenever we fix a configuration, we assume, w.l.o.g. that colors are ordered in descend-

ing order such that c 1 ≥ c 2 ≥ • • • ≥ c k .
Observe that in the complete graph the number ∆ ij of nodes switching from color 1

to color 2 has a binomial distribution with parameters ∆ ij ∼ Bin(c i , c 2 j /n 2 ). Clearly, the expectation and variance of ∆ ij are

E[ ∆ ij ] = c i • c 2 j n 2
As for the lower bound in Theorem 10.5 showing that t 2C = Ω(n/ log n), we refrain from applying Chernoff bounds in every step the result would be too weak. Instead we consider larger periods together with stochastic domination to derive stronger bounds.

Plurality Consensus with Two Choices

In Section 10.3.1 we show the upper bound Theorem 10.2 on the 2-Choices process. We show that if the initial bias is Ω(n log n), then the initially most dominant color wins with high probability in O(k • log n) rounds.

In Section 10.4.2 we show two lower bounds: We show that if the initial bias is of order O( √ n), then with constant probability a color different than color 1 will win (Theorem 10.6). Furthermore, we show that there are configurations from which we require Ω(k + log n) rounds until any opinion wins (Theorem 10.7).

Upper bound -Proof of Theorem 10.2

The algorithm discussed in this section is formally defined in Algorithm 8. The interesting regime is when the largest color c 1 has non yet reached an absolute majority. Otherwise, that is if c 1 ≥ (1/2 + ε 1 )n for some constant ε 1 > 0, the process converges within O(log n) steps with high probability. This follows from [START_REF] Cooper | The Power of Two Choices in Distributed Voting[END_REF] since in the case of c 1 ≥ (1/2 + ε 1 )n the size of the largest color is stochastically dominated by the size of the largest color when all other colors are merged into one single color.

For the sake of readability we assume in the following that a ≤ n/2. Furthermore, observe that c 1 > n/k, since color 1 is the largest of k color classes. We introduce the following notation.

Let S ⊆ C be a set of colors. We will use the random variable ∆ iS to denote the sum of all flows from color i to any color in S and ∆ Si to denote the sum of all flows from any color in S to i. We have in expectation

E[ ∆ Si ] = j∈S c j • c 2 i n 2 and E[ ∆ iS ] = j∈S c i • c 2 j n 2 .
For color i define i be the set of all other colors, in symbols i = {j : j ∈ C with j = i}.

We observe that after one round the new number of nodes supporting color i is a random variable

c i = c i + j =i ∆ ji - j =i ∆ ij = c i + ∆ ii -∆ ii .
Since all nodes perform their choices independently, the first sum ∆ C i i has a binomial distribution with parameters ∆

C i i ∼ B(n -c i , c 2 i /n 2 ).
Furthermore, every node of color i changes its color away from C i to any other opinion with probability j =i c 2 j /n 2 . Moreover, ∆ ii also has a binomial distribution with parameters ∆ ii ∼ Bin(c i , j =i c 2 j /n 2 ). In expectation

E c i = c i + (n -c i )c 2 i n 2 - c i n 2 j =i c 2 j . (10.1)
Note that these expected values are monotone w.r.t. the current size.

Observation 10.3. Let r and s be two colors.

It holds that if c r ≤ c s then E[ c r ] ≤ E[ c s ].
Proof. By (10.1),

E c i = c i + c 2 i n - c i n 2 C j c 2 j = c i   1 + c i n - C j c 2 j n 2   .
Thus,

E c r = c r   1 + c r n - j c 2 j n 2   (cr≤cs) ≤ c s   1 + c s n - j c 2 j n 2   = E c s .
We are ready to prove the following lemma.

Lemma 10.4. Assume that c 1 -c 2 > z • √ n log n. There exists a constant z such that with high probability

c 1 -c 2 > (c 1 -c 2 )(1 + c 1 /4n)
In the following proof we utilize certain methods which have also been used in [START_REF] Cooper | The Power of Two Choices in Distributed Voting[END_REF] for the two-opinion plurality consensus process with two choices in more general graphs.

Proof. First we observe that

i c 2 i = c 2 1 + i =1 c 2 i ≤ c 2 1 + i =1 c i • c 2 = c 2 1 + (n -c 1 ) • c 2 ≤ c 2 1 + n • c 2 (10.2)
We derive,

E c 1 -c 2 = c 1 + E ∆ 11 -E ∆ 11 -c 2 -E ∆ 22 + E ∆ 22 = a + (n -c 1 ) • c 2 1 n 2 - c 1 n 2 i =1 c 2 i -c 2 -(n -c 2 ) • c 2 2 n 2 + c 2 n 2 i =2 c 2 i = c 1 -c 2 + 1 n 2   c 2 1 n -c 3 1 -c 2 2 n + c 3 2 -c 1 i =1 c 2 i + c 2 i =2 c 2 i   = c 1 -c 2 + 1 n 2   n c 2 1 -c 2 2 -c 1   c 2 1 + i =1 c 2 i   + c 2   c 2 2 + i =2 c 2 i     = c 1 -c 2 + 1 n c 2 1 -c 2 2 - 1 n 2 c 1 i c 2 i -c 2 i c 2 i = c 1 -c 2 + (c 1 -c 2 )(c 1 + c 2 ) n - 1 n 2 i c 2 i (c 1 -c 2 ) = (c 1 -c 2 ) • 1 + (c 1 + c 2 ) n - 1 n 2 i c 2 i . (10.2) ≥ (c 1 -c 2 ) 1 + (c 1 + c 2 ) n - c 2 1 + n • c 2 n 2 ≥ (c 1 -c 2 ) 1 + c 1 n • 1 - c 1 n ≥ (c 1 -c 2 ) 1 + c 1 2n ,
where the last inequality uses c 1 ≤ n/2. Before we apply Chernoff bounds to c 1 -c 2 we introduce the following notation.

E ∆ 11 = (n -c 1 ) c 2 1 n 2 , E ∆ 11 = c 1 n 2 i =1 c 2 i , E ∆ 22 = (n -c 2 ) c 2 2 n 2 , E ∆ 22 = c 2 n 2 i =2 c 2 i .
Furthermore, let δ 11 , δ 11 , δ 22 , δ 22 be defined as

δ 11 = 2 √ n log n c 1 , δ 11 = 2n √ log n c 1 i =1 c 2 i , δ 22 = 2 √ n log n c 2 , δ 22 = 2n √ log n c 2 i =2 c 2 i
Since c 1 ≤ n/2 we know for the second largest color (color 2) that c 2 ≥ n/2k. Together with c 1 ≥ n/k ≥ n 1-ε we get 0 < δ < 1 and δ 2 xy • ∆ xy = Ω(log n) for (δ xy , ∆ xy ) defined above. We now apply Chernoff bounds to c 1 -c 2 and obtain with high probability

c 1 -c 2 ≥ (c 1 -c 2 ) • 1 + c 1 2n -σ
where the "Chernoff deviation" σ is bounded as follows.

σ := δ 11 • E ∆ 11 + δ 11 • E ∆ 11 + δ 22 • E ∆ 22 + δ 22 • E ∆ 22 = 2 √ n log n n 2   c 1 n -c 2 1 + c 1 n i =1 c 2 i + c 2 n -c 2 2 + c 2 n i =2 c 2 i   ≤ 2 √ n log n n 2   n i c 2 i ( √ c 1 + √ c 2 ) + c 1 n + c 2 n   ≤ 2 √ n log n n 2 (2c 1 n + c 1 n + c 2 n) ≤ 8a √ n log n n ,
where we used that i c 2 i ≤ i c 1 • c i ≤ c 1 n. From the conditions in the statement of the lemma we know that (c 1 -c 2 ) ≥ z • √ n log n for some constant z. If we assume that z is large enough, e.g., z ≥ 32, then we get with high probability

c 1 -c 2 ≥ (c 1 -c 2 ) • 1 + c 1 4n .
While Lemma 10.4 shows that in the absence of an adversary the difference between the most dominant color and the second most dominant color increases in every round with high probability. It is easy to see that the same holds for any third color, i. e., the increase in the distance between the most dominant color and any other given color is lower bounded by Lemma 10.4. By taking Union bound over all other colors we derive that the gap increases every round. To obtain a strong upper bound on the runtime, we will analyze how in the following (Theorem 10.2). the increase

Proof. Assume c 1 -c 2 ≥ z • √
n log n for a sufficiently large constant z. From Lemma 10.4 we know that c 1 -c 2 ≥ (c 1 -c 2 ) • (1 + c 1 /4n) with high probability. By using a standard coupling, we get that c 1 -

c j ≥ c 1 -c 2 ≥ (c 1 -c 2 ) • (1 + c 1 /4n
). Note that it may very well happen, especially if all colors have the same size except for color 1, that another color j "overtakes" color 2. However, with high probability

c 1 -max j≥2 c j ≥ (c 1 -c 2 ) • (1 + c 1 /4n).
We now take care of the adversary who may change up to F arbitrary nodes. Let c i denote the support of color i the next round after the adversary influenced nodes. Clearly, we have

|c i -c i | ≤ F for all i ∈ C. We conclude that We have c 1 -c j ≥ c 1 -c j -2F ≥ (c 1 -c 2 ) • (1 + c 1 /4n -2F/c 1 -c 2 ) ≥ (c 1 -c 2 ) • (1 + c 1 /8n), since F = a(c 1 -c 2 )/8n.
Taking the union bound over all colors, we conclude that the distance between the most dominant color and every other color grows in every round by a factor of at least (1 + c 1 /4n) with high probability. Therefore, after τ = 4n/c 1 rounds, the relative distance between color 1 and any other color doubles with high probability. Hence, the required time for color 1 to reach a size of at least (1/2 + ε 1 ) • n for a constant ε 1 > 0 is bounded by O(n/c 1 • log n). This bias is large enough that we assume in the following that all nodes which are not of color 1 are of the same color, say 2.

In absence of an adversary, we can see that after additional O(log n) rounds every node has the same color 1, with high probability; see [START_REF] Cooper | The Power of Two Choices in Distributed Voting[END_REF]. In each individual round, the growth described in Lemma 10.4 takes place with high probability. A union bound over all O(n/c 1 • log n) rounds yields that the protocol indeed converges to color 1 within O(n/c 1 • log n) rounds with high probability. The same analysis of [START_REF] Cooper | The Power of Two Choices in Distributed Voting[END_REF] can be used even in the presence of an adversary. However, in this case we can only reach almost validity according to Definition 10.1, since the adversary is allowed to change F = o(n) nodes per round.

Finally, we argue that the 2-Choices process trivially fulfills the property almost agreement according to Definition 10.1. Starting from an arbitrary initial distribution of colors, there is in every round a positive (albeit super-exponentially small in n) probability that all nodes adopt the same color.

Lower Bound for 2-Choices

This section give lower bounds for 2-Choices. We first show that the consensus time can be almost linear with high probability starting from n colors (Section 10.4.1). We then complement on upper bounds of Section 10.3: Even if the initial bias is of order √ n (as opposed to √ n log n), then the second largest color might win with constant probability. Additionally, we give almost matching lower bounds on the consensus time in the setting where the initial bias is of order √ n log n.

Worst-Case Lower bound -Theorem 10.5

It turns out that, when started from an almost balanced configuration, the consensus time is dictated by the time it takes for one of the colors to gain a support of Ω(log n). To prove this result, we prove a slightly stronger statement, that captures the slow initial part of the process when started from configurations with a maximal load of .

Theorem 10.5. Let γ be a sufficiently large constant. Consider the 2-Choices process starting from any initial configuration c ∈ C. Let := max i c i (0) be the support of the largest color. Then, for := max{2 , γ log n}, it holds with high probability that no color has a support larger than for n/(γ ) rounds. In symbols,

P max i c i (t) > for some t < n γ ≤ 1 n . (10.3)
In particular, starting from the n-color configuration, it holds with high probability that no color has a support larger than γ log n for n γ 2 log n rounds.

Proof.

Let T i = min{t ≥ 0 | c i (t) > }.
For any fixed opinion i ≤ k we show that j is a Bernoulli random variable with P[ X i = 1 ] = p and, by a standard coupling, it is 1 whenever node j sees two times color i at round t (note that the latter event happens with probability at most p for any t < T i ). By definition, if t < T i it holds c i (t) ≤ , which implies that the probability that any node in the original process gets opinion i is at most p. Thus, we can couple 2-Choices and P for t ≤ T i so that c i (t) ≤ P (t). This implies that

P[ T i < n/(γ ) ] ≤ 1/n 2 ,
T := min{t ≥ 0 | P (t) ≥ } T i .
(10.4)

In the remainder we show that P[ T < n/(γ ) ] < 1/n 2 . For any round t + 1, we define ∆ t+1 := P (t + 1) -P (t) = i≤n X i . Observe that ∆ t+1 ∼ Bin(n, p). Let t 0 = n/(γ ). In the following we bound

B := P (t 0 ) -P (0) = t 0 i=1 ∆ i .
Observe that B ∼ Bin(t 0 • n, p) and thus E[ B ] = t 0 • n • p. Using Chernoff bounds, e.g., [START_REF] Mitzenmacher | Probability and Computing: Randomized Algorithms and Probabilistic Analysis[END_REF]Theorem 4.4] we derive for any γ ≥ 18

P P (t 0 ) ≥ = P B ≥ -≤ P B ≥ max 2E[ B ], γ 2 log n = P B ≥ E[ B ] • max 2, 1 + γ 2 log n E[ B ] ≤ exp - γ 2 log n 3 ≤ 1/n 3 , ( 10.5)
where we used that

max 2E[ B ], γ 2 log n = max 2t 0 • n • p, γ 2 log n ≤ max ( ) 2 γ , γ 2 log n ≤ max 2 , γ 2 log n ≤ 2 = -.
Putting everything together yields

P T < n/(γ ) = P[ T < t 0 ] (10.6) (a) ≤ nP[ T i < t 0 ] (b) ≤ nP T < t 0 (c) ≤ nP P (t 0 ) ≥ (d)
≤ n -2 , (10.7)

where in (a) we used union bound over all colors, in (b) we used (10.4), in (c) we used that "T < t 0 " =⇒ "P (t 0 ) ≥ " and in (d) we used (10.5). This completes the proof.

10.4.2 Complementing our Upper bounds Theorem 10.6 and Theorem 10.7

In this section we give lower bounds complementing our positive results: In the previous section, we proved that the process converges to color 1 with high probability if the initial imbalance c 1 -c 2 is not too small. Precisely, Theorem 10.2 states that if c 1 -c 2 ≥ z • √ n log n for some constant z, color 1 wins with high probability. Conversely, in the following section we examine a lower bound on the initial bias. We will show, as stated in Theorem 10.6, that for an initial bias c 1 -c 2 ≤ z • √ n for some constant z we have a constant probability that color 2 "overtakes" color 1 in the first round, that is, P[ c 1 < c 2 ] = Ω(1) implying that color 2 wins w.c.p.. Our lower bound is based on the approximation of the binomial distribution with the normal distribution, which allows us to obtain a lower bound. In order to so, we apply the DeMoivre-Laplace limit theorem (Theorem A.6)

We now prove Theorem 10.6 which states that there exists an initial color assignment for which c 1 = c 2 + z • √ n but color 2 wins with constant probability even in absence of an adversary.

Theorem 10.6 (Lower Bound on the Initial Bias). For any k ≤ √ n and constant z there exists an initial assignment of colors to nodes for which

c 1 = c 2 + z • √ n but P[ c 1 < c 2 ] = Ω(1)
) = n + z • √ n , n -z • √ n , 1, . . . , 1 .
Clearly, C j c j = n. In the following we will omit the floor and ceiling functions for the sake of readability reasons. First, we start by giving an upper bound on the number of nodes which change their color away from color 2. Now recall that ∆ 22 follows a binomial distribution ∆ 22 ∼ Bin(b, Cj =B c 2 j /n 2 ) with expected value

E ∆ 22 = c 2 • c 2 1 + k -2 n 2 = n -z • √ n • (n + z • √ n) 2 + k -2 n 2 ≤ (n + z • √ n) 3 + k -2 n 2 ≤ n 8 + 4z √ n .
Applying Chernoff bounds to ∆ 22 gives us

P ∆ 22 ≥ 1 + 3/E ∆ 22 E ∆ 22 ≤ 1/e . (10.8)
That is, with constant probability at least 1 -1/e we have

∆ 22 ≤ 1 + 3/E ∆ 22 E ∆ 22 ≤ n 8 + 4z √ n + 3E ∆ 22 ≤ n 8 + (4z + 1) √ n .
Secondly, we give the following lower bound on the number of nodes which change their color from color 1 to color 2 and ∆ 12 ∼ Bin a, b 2 /n 2 with expected value

E[ ∆ 12 ] = n + z • √ n • (n -z • √ n) 2 n 2 ≥ (n -z • √ n) 3 n 2 ≥ (n/2 -(z + 1/2) √ n) 3 n 2 ≥ n 8 -4z √ n and variance Var[ ∆ 12 ] = E[ ∆ 12 ] • 1 - (n -z • √ n) 2 n 2 ≥ n 9 • 1 2 = n 18 .
We now apply Theorem A.6 to ∆ 12 . Let x = √ 18 2 (18z + 4). We derive

P ∆ 12 ≥ E[ ∆ 12 ] + x • Var[ ∆ 12 ] = 1 √ 2π • x exp -x 2 /2 ± o(1) = Ω(1) .
That is, we have with constant probability

∆ 12 ≥ E[ ∆ 12 ] + x • Var[ ∆ 12 ] ≥ n 8 -4z √ n + x • n 18 .
(10.9)

Finally, assume that in the worst-case every node of colors 3, . . . , k changes to color 1 but not a single node changes away from color 1 to these colors 3, . . . , k. Observe that ∆ 22 is an upper bound on ∆ 21 . Therefore,

c 1 -c 2 ≤ (c 1 + k -2 + ∆ 21 -∆ 21 ) -c 2 + ∆ 12 -∆ 22 ≤ c 1 -c 2 + k -2 + 2∆ 22 -2∆ 12 ≤ 2z • √ n + k -2 + 2∆ 22 -2∆ 12 ≤ (2z + 1) • √ n + 2∆ 22 -2∆ 12 .
We plug in (10.8) and (10.9) to bound the random variables ∆ 12 and ∆ 22 and obtain with constant probability

c 1 -c 2 ≤ (2z + 1) • √ n + 2 n 8 + (4z + 1) √ n -2 n 8 -4z √ n + x • n 18 = (2z + 1 + 8z + 2 + 8z -2x/ √ 18) • √ n = (18z + 3 -2x/ √ 18) • √ n which gives us c 1 -c 2 < 0 for x = √ 18
2 (18z + 4). Therefore, we have P[ c 1 < c 2 ] = Ω(1) and thus we conclude that color 2 wins with constant probability.

Theorem 10.7 (Lower Bound on the Run Time). Assume the initial bias is exactly z √ n log n for some constant z. The number of rounds required for the plurality consensus process defined in Algorithm 8 to converge is at least Ω(n/c 1 + log n) with constant probability, even in absence of an adversary.

Proof. Let c 1 (t) denote the size of color 1 in round t and of initial size c 1 (0) = n/k + z • √ n log n. Furthermore, assume that k ≥ 3 • z. We show by induction on the rounds that

c 1 (t) ≤ c 1 (0) • (1 + 3 • c 1 (0)/n) t for 1 ≤ t ≤ n/(10 • c 1 (0)) with probability 1 -t/n. First we note that c 1 (t) ≤ c 1 (0) • 1 + 3 • c 1 (0) n t ≤ c 1 (0) • 1 + 3 • c 1 (0) n n/(10•c 1 (0)) ≤ c 1 (0) • exp(1/2) ≤ 2 • c 1 (0) (10.10)
We now prove the induction claim. The base case holds trivially. Consider step t + 1.

By induction hypothesis we have with probability at least 1 -t/n that c 1 (t) ≤ c 1 (0) •

(1 + 3 • c 1 (0)/n) t . Note that we have with high probability

c 1 (t + 1) ≤ c 1 (t) + ∆ 11 ≤ c 1 (t) +   1 + √ 3 log n E ∆ 11   • E ∆ 11 ,
where the latter inequality follows by Chernoff bounds. Using (10.10) and c 1 (t) ≤ c 1 (0), we derive

c 1 (t + 1) ≤ c 1 (t) + 1 + √ 3 log n c 1 (t) 2 /(2 • n) c 1 (t) 2 n ≤ c 1 (t) + 1 + √ 3 log n c 1 (0) 2 /(2 • n) c 1 (t) 2 n ≤ c 1 (t) + 3 2 • c 1 (t) 2 n = c 1 (t) • 1 + 3 2 • c 1 (t) n ≤ c 1 (t) • 1 + 3 • c 1 (0) n .
From the induction hypothesis we therefore obtain

c 1 (t + 1) ≤ c 1 (0) • 1 + 3 • c 1 (0) n t • 1 + 3 • c 1 (0) n = c 1 (0) • 1 + 3 • c 1 (0) n t+1 .
Using a union bound to account for all errors, we derive that with probability at least 1 -(t + 1)/n we have c 1 (t + 1) ≤ c 1 (0) • (1 + 3 • c 1 (0)/n) t+1 , which completes the proof of the induction and proves the lower bound of Ω(n/a).

In the remainder we establish the bound Ω(log n). Assume only two colors 1 and color 2 where color 1 initial size c 1 (0) = n/2 + √ n log n. We show by induction on the rounds that c 1 (t) ≤ c 1 (0) + 6 t √ n log n for 1 ≤ t ≤ log n/20 with probability 1 -2t/n. First we note that c 1 (t) ≥ c 1 (0) and

c 1 (t) ≤ c 1 (0) + 6 t √ n log n ≤ n/2 + n 5/6 < n .
We now prove the induction claim. The base case holds trivially. Consider step t + 1.

By induction hypothesis we have with probability at least 1 -2t/n that c 1 (t) ≤ c 1 (0) + 6 t √ n log n. We have, using c 1 = c 1 (t) and σ = 6 t √ n log n,

n 2 • E ∆ 11 -∆ 11 = (n -1)c 2 1 -c 1 • (n -c 1 ) 2 = (n -c 1 )c 1 (2c 1 -n) ≤ n/2 • c 1 • 2σ = n • σ(n + σ) = n 2 • σ + n • σ 2 .
Similar to before, we obtain by Chernoff bounds that with high probability

c 1 (t + 1) -c 1 (t) = ∆ 11 -∆ 11 ≤   1 + √ 3 log n E ∆ 11   E ∆ 11 -   1 - √ 3 log n E ∆ 11   E ∆ 11 ≤ E ∆ 11 -∆ 11 + 2 3 log n • E ∆ 11 ≤ σ + σ 2 /n + 2 3 log n • √ n ≤ 3σ .
From the induction hypothesis we therefore obtain

c 1 (t + 1) ≤ c 1 (0) + 6 t √ n log n + 3σ ≤ c 1 (0) + 6 t+1 √ n log n ,
which completes the induction and yields the lower bound of Ω(log n).

Comparison with the 3-Majority Process

In this section we elaborate on the difference between the 2-Choices process and the 3-Majority rule [START_REF] Becchetti | Trevisan: Simple Dynamics for Plurality Consensus[END_REF], where in the latter each node pulls the opinion of three random neighbors and adopts the majority opinion among those three, breaking ties uniformly at random. As mentioned before, the 3-Majority process of [START_REF] Becchetti | Trevisan: Simple Dynamics for Plurality Consensus[END_REF] uses O(log k) memory bits and the authors prove a tight run time of Θ(k • log n) for this protocol, given a sufficiently high bias c 1 -c 2 . Moreover, they show that if the bias is only of order √ kn, then with constant probability the difference c 1 -c 2 decreases. This is fundamentally different in the 2-Choices process, which requires only a bias of O(

√ n log n).
here is the 2-Choices process where in each round every node samples two nodes chosen uniformly at random, with replacement. If the chosen nodes' colors coincide, then the node adopts this color. We denote this process as the plurality consensus process with two choices.

Our first two results will be shown w.r.t. this synchronous model.

Parallel Asynchronous Model

In the asynchronous model we consider, every node v is equipped with a random clock which ticks according to a Poisson distribution with parameter λ = 1. Whenever a node ticks, it may sample nodes chosen uniformly at random and update its opinion based on the sampled values.

As discussed in [START_REF] Boyd | Randomized gossip algorithms[END_REF][START_REF] Mosk-Aoyama | Fast Distributed Algorithms for Computing Seperable Functions[END_REF], this model is equivalent to the setting, in which a global clock ticks according to a Poisson process of rate n. If T i denotes the i'th tick of this clock, then the time differences T i -T i-1 are identically distributed independent random variables which have exponential distribution with parameter n. At each tick of this global clock, a node is chosen uniformly at random, and the global tick corresponds to the local tick at this node. As shown before, there is a strong correspondence between the number of ticks and the absolute time, meaning that with probability 1 -e -Ω(n) the k'th tick of the global clock occurs at some time Θ(k/n) for any k = Ω(n).

Sequential Asynchronous Model

While the parallel model described above represents real-world processes for which event

Results

Our main contribution is an efficient algorithm for plurality consensus in the asynchronous setting. As discussed below in more detail, a straight-forward observation is that in the sequential asynchronous model many nodes may remain unselected for up to O(log n) time, which implies that no algorithm can converge in o(log n) time. Thus, our aim is to construct a protocol that solves plurality consensus in O(log n) time. We show that if the difference between the numbers of the largest two opinions is at least Ω(c 2 ), where c 2 is the size of the second largest opinion, and k = n O(1/ log log n) , then our algorithm solves plurality consensus and achieves the best possible run time of O(log n), provided a node is allowed to communicate with at most constantly many other nodes in a step.

The key to the rapidity of our protocol is that we pair a phase in which all nodes execute the 2-Choices process with a phase in which successful opinions are propagated quickly-much like in broadcasting. For this to work it is crucial to separate the two phases.

While this is trivial in the synchronous setting, it is impossible in the asynchronous setting.

The number of activations of different nodes can easily differ by Θ(log n), rendering any attempt of full synchronization futile if one aims for a run time of O(log n). Thus, we restrict ourselves to the concept of weak synchronicity as follows. At any time we only require that a (1 -o(1))-fraction of nodes are almost synchronous. To cope with the influence of the remaining nodes, we rely on a toolkit of gadgets, which we believe are interesting in their own right. Our result is formally stated in the following theorem. The result of Theorem 11.1 can be extended to the following asynchronous model that takes into account latency and response times when a node is contacted. In this extended model, every node ticks according to a Poisson process with parameter λ = 1, and the response time of a node has exponential distribution with some constant parameter. Furthermore, we believe that the result of the theorem, and the methods used in its proof, can be extended beyond the Poisson clock process-see Section 11.11 for a discussion.

Approach and Technical Contributions

In this section, we introduce our asynchronous consensus algorithm and give the intuition behind the proof of Theorem 11.1. To simplify the presentation we start by discussing our synchronous consensus protocol. A detailed description and its proofs can be found in [START_REF] Elsässer | Efficient k-Party Voting with Two Choices[END_REF]. Afterwards, we introduce a framework allowing us to adapt the algorithm to the asynchronous setting. In order to achieve a poly-logarithmic run time we do the following. We combine the guarantee of the 2-Choices process to reach plurality consensus with the speed of broadcasting. More specifically, the protocol consists of Θ(log(n/c 1 ) + log log n) phases which in turn consist of two sub-phases: (i) one round of the 2-Choices process and (ii) several rounds of the so-called Bit-Propagation sub-phase in which each node that changed its opinion during the preceding two-choice step broadcasts its new opinion.

An Ideal World: Synchronous Consensus

More precisely, we equip each node with an additional bit of memory which is set to True if and only if it changed its opinion in the 2-Choices sub-phase. In the Bit-Propagation subphase, each node u samples nodes randomly until a node v with its bit set to True is found.

Then u adopts v's opinion and sets its own bit to True, which means that subsequently any node sampling u will set their bit as well.

The first sub-phase ensures that after the 2-Choices round the number of nodes holding opinion C 1 and having their bit set to True is concentrated around c 2 1 /n. After the Bit-Propagation sub-phase all nodes will have their bit set, and the distribution and the size of C j 's support is concentrated around c 2 j /x(1), where x(1) is the total number of bits set after the 2-Choices sub-phase. This is enough to show that after O(log(n/c 1 )) rounds the distance between C 1 and any opinion

C j = C 1 increases quadratically, that is, c 1 /c j ≥ (1 -o(1))c 2 1 /c 2 j .
Due to the quadratic growth in the distance between C 1 and every other opinion, the number of required phases is only of order Θ(log(n/a) + log log n). We assume that every node is aware of (upper bounds on) n and k, allowing us to use these values within the algorithm, and in particular to run it in multiple phases of length Θ(log k + log log n) each.

Towards an Asynchronous Algorithm. We now introduce our asynchronous protocol to solve plurality consensus. In the sequential asynchronous model we assume that a sequence of discrete time steps is given, where at each time step one node is chosen uniformly at random to perform its tick.

C 1 lgorithmtwo-choices(G = (V, E), color : V → C) for round t = 1 to |C| • log |V | do C 1 tNodev let u 1 , u 2 ∈ N (v) uniformly at random; if color(u 1 ) = color(u 2 ) then color(v) ← color(u 1 );
The key to the speed of the synchronous algorithm is the combination of the two-choice process with an information dissemination process. However, this interweaving of these processes requires that the nodes execute the sub-phases simultaneously. While this is trivially the case in the synchronous setting, it is extremely unlikely in the asynchronous setting, since the numbers of ticks of different nodes may differ by up to O(log n). Therefore, any attempt to reach full synchronization is futile if one aims for a run time of O(log n).

do-nothing-blocks are used, in combination with the following result on synchronicity, to ensure that a large fraction of nodes executes critical instructions at almost the same time.

That is, for a large fraction of nodes we will show that these nodes execute instructions as if they were bulk synchronized, which they clearly are not.

The first phase is the 2-Choices sub-phase, which consists of two instructions, the 2-Choices step and the commit step. In the 2-Choices step, every node samples two neighbors uniformly at random. If and only if these neighbors' colors coincide, the node sets an intermediate color to the neighbors' colors. In the commit step, nodes change their color if they have their intermediate color set and then set their bit accordingly. The second phase is the Bit-Propagation sub-phase, which closely resembles the synchronous counter part.

Finally, in the third phase, all nodes execute the so-called Sync Gadget. In this gadget, nodes adjust their working time in order to synchronize. Our perpetual synchronization mechanism is described after the following definitions.

For the analysis of the asynchronous algorithm we will use the following notation and definitions.

Definitions. Let κ and denote sufficiently large positive constants. We refer to a series of n consecutive time steps as a period, and we combine T = κ • log n/ log log n periods to a phase. The first part of the asynchronous protocol consists of • log log n phases. Intuitively, a period is the number of time steps during which each node ticks in expectation once. We define a reference point τ to be a time step which marks the end of a period τ . In particular, at reference point τ there have been τ •n time steps, and each node has ticked in expectation τ times. At the beginning of the algorithm, both, the real time and the working time are initialized to 0. Since at each time step one node is chosen to tick independently and uniformly at random, T v (τ ) has a binomial distribution T v (τ ) ∼ Bin(τ • n, 1/n) with expected value E[ T v (τ ) ] = τ . It will prove convenient to regard a reference point as the one instruction in the algorithm which would be executed in the corresponding period if every node ticked exactly once in every period.

Weak Perpetual Synchronization. In the asynchronous algorithm, when a node is selected to tick, all operations are performed based on the node's current working time. In contrast, the real time of a node is used to always the total number of ticks performed so far by this node. In the Sync Gadget, the working time T v of a node v, denoted as workingtime in Algorithm 10, is adjusted as follows.

The Sync Gadget consists of a sampling sub-phase [τ m1 , τ m2 ] and a jump step τ jump . The sampling sub-phase of the Sync Gadget consists of log 3 log n ticks. During these ticks, every node samples a neighbor uniformly at random and collects the real time T u of the sampled neighbor u. Additionally, the node increments all real times sampled so far by 1 until the jump step is executed. At the jump step, the node sets its working time to the median of the samples.

During the entire phase, according to Algorithm 10, there are multiple blocks of instructions where nodes literally do nothing. These blocks are used, in combination with the following result on synchronicity, to ensure that a large fraction of nodes executes critical instructions at almost the same time. That is, for a large fraction of nodes we will show that these nodes execute instructions as if they were bulk synchronized, which they clearly are not.

The Key Lemmas

The use of the Sync Gadget and the following definition of ∆-closeness allow us to show Proposition 11.3 which forms the basis for our adaption of the synchronous protocol to the asynchronous setting.

|S| ≥ n • (1 -exp(-8 log n/ log log n)).
The proof idea is as follows. We first observe that roughly n• 1 -explog n/ log 2 log n nodes are (∆/16)-close throughout the execution of the algorithm. As argued before, the resulting number of poorly synchronized nodes is too large and could tip the balance.

Furthermore, we show, by careful induction, that thanks to the perpetual synchronization in each phase, a large fraction f = (1 -exp(-9 log n/ log log n)) of the nodes which were (∆/2)-close throughout the first i phases, will remain (∆/2)-close in phase i + 1: (i) a fraction f of these nodes will tick equally often in each interval in this phase, up to an error of ∆/16, and (ii) among these nodes again a fraction f will adapt their working time by selecting the median of a sample of nodes. That median will be (∆/16)-close. Accounting for numerous other sources of error we obtain overall (∆/2)-closeness for a large fraction of nodes.

Equipped with Proposition 11.3 we analyze the 2-Choices and Bit-Propagation subphases. Instead of describing the distribution of colors after every 2-Choices and Bit-Propagation sub-phase, we restrict ourselves to the distribution of colors among the wellsynchronized nodes in S. In fact, throughout the analysis, we assume for all other nodes in (V \ S) the worst-case. However, based on the Sync Gadget and Proposition 11.3, their number is small enough such to prevent them from tipping the balance.

Our next key-lemma is Proposition 11.4 which establishes that the number of nodes which pick up a bit for color C j is with high probability concentrated around the expectation.

Analogously to the synchronous case, we consider in the following definitions and propositions an arbitrary but fixed phase of Algorithm 10. Let ĉj (τ ) denote the number of nodes belonging to S having color C j at reference point τ , that is, at time step τ • n. Let furthermore x j (τ ) denote the set of nodes belonging to S having color C j and having their bit set at reference point τ and let finally x(τ ) = j x j (τ ).

Proposition 11.4. Assume |S| ≥ n • (1 -exp(-8 log n/ log log n)). Let C j be an arbitrary but fixed color. With high probability, the number of nodes in S having a bit set for color C j after the 2-Choices sub-phase at reference point τ bp1 is bounded as follows.

x 1 (τ bp1 ) ≥ ĉj (τ 0 ) 2 n (1 -o(1)) and x i (τ bp1 ) ≤ ĉj (τ 0 ) 2 n (1 + o(1)) + O n 1-14/log log n .
Building on the concentration of bits given by Proposition 11.4 at τ bp1 , the following proposition bounds the number of nodes of each color after the Bit-Propagation sub-phase at τ bp2 . As before, we only characterize those nodes which are part of S.

Proposition 11.5. Assume |S| ≥ n • (1 -exp(-8 log n/ log log n)). Let C j be an arbitrary but fixed color. With high probability, the number of nodes in S of color C j after the Bit-Propagation sub-phase is bounded as follows.

ĉ1 (τ bp2 ) ≥ ĉ1 (τ 0 ) 2 x(τ bp1 ) • (1 -o(1)) and ĉj (τ bp2 ) ≤ ĉj (τ 0 ) 2 x(τ bp1 ) • (1 + o(1)) + O n 1-4/log log n .
In the proof we analyze the Bit-Propagation by the means of the Pólya urn process.

In particular, we show that the fraction of nodes supporting each color C j remains concentrated throughout the Bit-Propagation sub-phase. The proofs can be found in Section 11.5, Section 11.6, and Section 11.7, respectively.

Concentration of the Clocks: Proof of Proposition 11.3

In the following we show that throughout the entire process there do not exist nodes which perform more than O(log n) ticks, with high probability.

Observation 11.6. For any reference point τ we have that the working time of any node is bounded by the minimum and maximum real times, that is, for all u ∈ V and τ ∈ N we have

T v (τ ) ∈ min u∈V T u (τ ), max u∈V T u (τ ) . ( 11
T ≤ 3/2 • κ • • n log n . (11.2)
Furthermore, we have with high probability that

max v∈V {T v (T)} < 2 • κ • • log n and max v∈V T v (T) < 2 • κ • • log n . (11.3)
Proof Sketch. The proof idea is the following. (11.1) follows from the fact that at every tick the working time and the real time are simultaneously increased by one, and whenever the working time is set to the median of the sampled real times, which are also incremented upon each tick, the property also holds. For the proof of (11.2) and (11.3), observe that according to Algorithm 10 a node completes the execution of the algorithm when T v reaches κ • • log n. The proof of (11.2) and (11.3) follows, for κ • large enough, from an application of Chernoff bounds to T v (T) and union bound over all nodes, where we use (11.1) to show the second part of (11.3).

We proceed to show that most nodes are almost synchronous at carefully chosen reference points. Intuitively, a huge fraction of nodes has a number of ticks that is concentrated around the expected value and therefore most nodes will execute instructions which are close together. We formalize this concept in the following lemma which is based on Definition 11.2.

The lemma establishes in its first part that n • 1 -exp -Θ log n/ log 2 log n nodes will be (∆/6)-close w.r.t. the real time over the course of the algorithm.

In the second statement we consider shorter intervals of the length of a phase and claim that a much larger number of nodes, to be specific, n

|Y | ≥ |Y | • (1 -exp(-10 log n/ log log n)) -Õ( √ n) .
Proof. Let E v (τ ) be the event that a node v is (∆/16)-close to τ , that is,

E v (τ ) = τ -∆/16 ≤ T v (τ ) ≤ τ + ∆/16 .
We apply Chernoff bounds to T v (t) and obtain

P[ E v (τ ) ] ≥ 1 -exp -Ω log n log 2 log n , ( 11.4)
Let in the following Y v (τ ) be an indicator random variable for a node v and a reference point τ defined as

Y v (τ ) =    1, if E v (τ ) , 0, otherwise.
Summing up over all nodes gives us Y

(τ ) = v∈V Y v (τ ). By linearity of expectation, we have E[ Y (τ ) ] ≥ n • 1 -exp -Θ log n/(log 2 log n)
. Note that the random variables T v (τ ), and therefore also the random variables Y v (τ ), are not independent. We thus consider the process of uncovering Y v (τ ) one node after the other in order to obtain the Doob martingale of Y (τ ) as follows. We define the sequence Z j (τ ) as

Z j (τ ) = E[ Y (τ )|T j (τ ), . . . , T 1 (τ ) ] with Z 0 (τ ) = E[ Y (τ ) ]. We have E[ Z j (τ )|T j-1 (τ ), . . . , T 1 (τ ) ] = E[ E[ Y (τ )|T j (τ ), . . . , T 1 (τ ) ]|T j-1 (τ ), . . . , T 1 (τ ) ]
which, applying the tower property, gives us that

E[ Z j (τ )|T j-1 (τ ), . . . , T 1 (τ ) ] = E[ Y (τ )|T j-1 (τ ), . . . , T 1 (τ ) ] = Z j-1 (τ ) . Therefore Z j (τ ) is indeed the Doob martingale of Y (τ ).
According to Observation 11.6 each node ticks at most 2c•log n times, that is, |T j+1 (τ )-

T j (τ )| ≤ 2c • log n.
This holds with high probability in the original process P and with probability 1 in the coupled process P . Since at most 2c • log n of the random variables Y j+1 (τ ), . . . , Y n (τ ) differ, we have

|Z j+1 (τ ) -Z j (τ )| = E[ Y n (τ ) + • • • + Y 1 (τ )|T j+1 (τ ), . . . , T 1 (τ ) ] -E[ Y n (τ ) + • • • + Y 1 (τ )|T j (τ ), . . . , T 1 (τ ) ] ≤ 2c • log n .
Let G be the set of these bad nodes. Let Z denote the number of samples drawn which are bad. Thus, by Theorem A.4 with parameters α = 1/2 and p = L/n, we derive

P Z ≥ αc log 2 log n/2 ≤ (2p) 1/2 (2(1 -p)) 1/2 c log 2 log n ≤ 2 c log 2 log n • (p 1/2 (1 -p)) c log 2 log n ≤ √ n • (L/n) c log 2 log n/2 = √ n • n -c•c log 2 log n 2 log 2 log n ≤ 1/n 2 ,
for large enough c . In the following, we show by induction that with high probability Let J s be the set of nodes which are selected τ r -τ l ± ∆/16 times to tick in every interval [τ l , τ r ]. Since there are at most T 2 such intervals, we get by (11.5) that with high probability

|J s | ≥ n 1 -T 2 • s • exp(-9 log n/ log log n) .
|J s | ≥ |J s | 1 -T 2 • exp(-9 log n/ log log n) .
Let v be an arbitrary but fixed node. Let ϑ v be the exact time step at which v jumps and observe that ϑ v is a random variable. Let furthermore τ v denote the first reference point after time step ϑ v , that is,

τ v = ϑ v /n .
T v (τ ) -T v (ϑ v /n) = T v (τ ) -T v (ϑ v /n) .
(11.7)

We now show that every node v ∈ J s jumps exactly once. Recall that τ jump is the instruction at which every node executes the jump step. That is, if any nodes has a working time of s • T + τ jump , then that node jumps We claim that every node v ∈ J s must have jumped prior to (s + 1) • T , that is, we have τ v ≤ (s + 1) • T . To see this, assume that v didn't jump. By (11.7),

T v ((s + 1) • T ) = T v ((s + 1) • T ) -T v (s • T ) + T v (s • T ) ≥ (s + 1) • T -s • T -∆/16 + T v (s • T ) ≥ (s + 1) • T -s • T -∆/16 + s • T -5∆/16 > (s + 1) • T -∆/2 ≥ s • T + τ jump ,
where the first inequality follows from the definition of J s and the second inequality follows from the induction hypothesis. The the above inequality implies that v must have executed the jump instruction and thus must have jumped.

Symmetrically, we claim that every node v ∈ J s will jump at most once per phase with high probability. It suffices to show that no node of J s jumps before reference point τ := τ m2 + ∆/2, since, informally speaking, at reference point τ all nodes of J s will have a real time exceeding τ m2 (similarly as before, this can be shown using the definition of J s and the induction hypothesis). Thus, by Lemma 11.8 and the due to the immense size of J s , node v will set its working time to the median of sampled real times which will be larger than τ m2 . Node v will not execute the jump instruction again in this phase. To show this claim we need to show that T v (τ ) < s • T + τ jump , which is true since (11.7),

T v (τ ) = T v (τ ) -T v (s • T ) + T v (s • T ) ≤ τ m2 + ∆/2 + ∆/16 + T v (s • T ) ≤ τ m2 + ∆/2 + ∆/16 + s • T -5∆/16 ≤ (s + 1) • T -∆/2 = s • T + τ jump ,
where the first inequality follows from the definition of J s and the second inequality follows from the induction hypothesis. Thus, v jumped at most once. We therefore conclude that every node v ∈ J s jumps exactly once.

We will now argue the following. For every v ∈ J s chooses with high probability

|T v (ϑ v /n) -ϑ v /n| ≤ 2∆/16 + 1 . (11.8)
To see this, first observe that, by Lemma 11.8, the median taken from log 3 log n samples of the real time is (∆/6)-close. Second, we need to account for the fact that median is not taken directly, but rather over time. If all samples were taken directly before jumping, then the median would indeed be (∆/6)-close. However, since v ∈ J s , it holds that the value of any sample is (∆/6)-close w.r.t. the value it would have if it were sampled directly before v jumps. Accounting for all errors, using triangle inequality and that τ u = ϑ v /n , (11.8) follows.

We proceed by showing that after v ∈ J s jumps its working-time well-concentrated, that is,

|T v (τ ) -τ | ≤ 5∆/16 , ( 11.9)
for any reference point τ in [τ v , (s + 1) • T ]. We have

T v (τ ) (11.7) = T v (ϑ v /n) + T v (τ ) -T v (ϑ v /n) (11.8) ≤ ϑ v /n + 2∆/16 + 1 + T v (τ ) -T v (ϑ v /n) (11.6) ≤ ϑ v /n + 2∆/16 + 1 + T v (τ ) -T v (τ v ) + ∆/16 def. J s ≤ ϑ v /n + 2∆/16 + 1 + ((τ -τ v ) + ∆/16) + ∆/16 def. τv ≤ τ v + 1 + 2∆/16 + 1 + ((τ -τ v ) + ∆/16) + ∆/16
≤ τ + 5∆/16, Symmetrically, we have

T v (τ ) (11.7) = T v (ϑ v /n) + T v (τ ) -T v (ϑ v /n) (11.8) ≥ ϑ v /n -2∆/16 -1 + T v (τ ) -T v (ϑ v /n) (11.6) ≥ ϑ v /n -2∆/16 -1 + T v (τ ) -T v (τ v ) -∆/16 def. J s ≥ ϑ v /n -2∆/16 -1 + ((τ -τ v ) + ∆/16) -∆/16 def. τv ≥ τ v -1 -2∆/16 -1 + ((τ -τ v ) + ∆/16) -∆/16 ≥ τ -5∆/16 ,
This shows (11.9). Define J s+1 = J s . This shows that v ∈ J s+1 is (5∆/16)-close at (s + 1) • T . Furthermore, at reference point s • T , v was, by induction hypothesis, (5∆/16)close and, since J s+1 = J s , at every reference point τ before u jumped we can derive

|T v (τ ) -τ | ≤ 5∆/16 + ∆/16 ≤ ∆/2. Furthermore, (11.9) implies that v was also (∆/2)close after jumping and thus v was ∆/2 at each reference point in [s • T, (s + 1) • T ].

We now show that |J s+1 | is large enough. Using the induction hypothesis, we have

|J s+1 | = |J s | ≥ |J s | 1 -T 2 • exp(-9 log n/ log log n) ≥ n 1 -sT 2 • exp(-9 log n/ log log n) 1 -T 2 • exp(-9 log n/ log log n) ≥ n 1 -(s + 1)T 2 • exp(-9 log n/ log log n) .
This finishes the induction step. Finally, observe that for any s = O(log log n) we have definition the working time of all nodes of S is larger than τ 0 and smaller than τ set . Let u be a node of S. Then, u samples at two nodes (that is, when its working time is τ tc ), then its probability of sampling two nodes of color C j with probability at least (ĉ j (τ 0 )/n) 2 and at most ((ĉ j (τ 0 ) + E)/n) 2 .

n • 1 -s • T 2 • exp(-9 log n/ log log n) ≥ n(1 -exp(-8 log n/ log log n)) .
By Chernoff bounds,

x 1 (τ bp1 ) ≥ |S| • (ĉ j (τ 0 )/n) 2 - √ n log n ≥ ĉj (τ 0 ) 2 n (1 -o(1)),
where we used the fact that all nodes of S must have executed the instruction at τ set at reference point τ bp1 .

We now distinguish between two cases. If ĉj (τ 0 ) ≤ n 1-7/log log n we have, ĉj (τ 0 ) + E = O n 1-7/log log n . Thus, by Chernoff bounds, with high probability

x j (τ bp1 ) ≤ n • ((ĉ j (τ 0 ) + E)/n) 2 + √ n log n = O n 1-14/log log n .
Otherwise, ĉj (τ bp1 ) > n 1-7/log log n and we have ĉj (τ 0 ) + E = ĉj (τ 0 )(1 + o(1)). Thus, by

Chernoff bounds, we obtain with high probability that

x j (τ bp1 ) ≤ n • ((ĉ j (τ 0 ) + E)/n) 2 = ĉj (τ 0 ) 2 /n • (1 + o(1)).
This finishes the proof. Proof. We split the proof into three parts, in each of which we will rely on the fact that at each reference point the nodes of S are (∆/2)-close. We argue that with high probability

Analysis of the

(i) x(τ 2 ) ≥ n/2, (ii) x(τ 4 ) ≥ |S| • 1 -n -2/ log log n , and (iii) x(τ bp2 ) = |S|.

Part (i).

To show the first part, we first consider a sequence of ∆ periods from τ 1 to τ 2 . Recall that each period consists of n consecutive time steps. We will show by induction

over i ∈ [τ 1 , τ 2 ) that x(i) ≥ min n 2 , n 2k • 1 + 1 5 i .
Let i be an arbitrary but fixed period in [τ 1 , τ 2 ) and assume that x(i -1) < n/2. For a node v ∈ A(i) ∩ H(i) in period i, we define X v to be the indicator random variable for the event that v sets the bit. Note that all X v are independent and P

[ X v = 1 ] ≥ x(i -1)/n. Let X = X i . By Chernoff bounds, X ≥ |A(i) ∩ H(i)| • x(i -1)/n • (1 -o(1)) ≥
x(i -1)/5 with high probability. We therefore get that with high probability

x(i) ≥ x(i -1) + X ≥ x(i -1) 1 + 1 5 IH ≥ n 2k • 1 + 1 5 i ,
which completes the induction. We now obtain, using τ

2 -τ 1 ≥ 4 log k, that x(τ 2 ) ≥ n 2k 1 + 1 5 τ 2 -τ 1 ≥ n 2k • k = n/2 .
This completes the proof of Part (i).

Part (ii). Let H(τ 2 ) ⊆ S be the set of nodes in S which do not have a bit set at reference point τ 2 . We consider an arbitrary but fixed node v ∈ H(τ 2 ) at reference point τ 4 . Since v is in S and thus (∆/2)-close at both, τ 2 and τ 4 , we observe that it ticked at least τ 4 -τ 2 -2 • ∆/2 = ∆/2 times between time steps τ 2 • n and τ 4 • n corresponding to these reference points. The probability that the node v never sampled a node with the bit set is thus at most 2 -∆/2 . Hence, by using independence and Chernoff bounds, the number of nodes remaining in H(τ 4 ) is, for ∆ large enough, at most |S| • n -2/ log log n with high probability.

Part (iii).

As before, let H(τ 4 ) ⊆ S be the set of nodes in S which do not have a bit set at reference point τ 4 . We again consider an arbitrary but fixed node v ∈ H(τ 4 ).

Since v is in S and thus (∆/2)-close at both, τ 4 and τ bp2 , we observe that it performed at least τ 5 -τ 4 = ∆/2 Bit-Propagation ticks. The probability that v samples in one of these ticks a node in S without the bit set or that v samples a node not in S is at most n -2/ log log n + n -8/ log log n ≤ n -1/ log log n . Therefore, the probability that this node never obtains the bit is at most n -1/ log log n ∆/2 ≤ n -ω(1) . From union bound we derive that all nodes in S therefore have the bit set at reference point τ bp2 .

In the following we analyze the individual colors during the Bit-Propagation sub-phase.

Our main observation is that the Bit-Propagation process can be modeled by so-called Pólya urns [START_REF] Johnson | Urn Models and Their Application: An Approach to Modern Discrete Probability Theory[END_REF]. In this model, we are given an urn containing marbles of two colors, black and white. In every step, one marble is drawn uniformly at random from the urn. Its color is observed, the marble is returned to the urn and one more marble of the same color is added.

For any color, the ratio of marbles with that given color over the total number of marbles is a martingale. We will use this urn process to model the Bit-Propagation sub-phase, which then can be analyzed by means of martingale techniques. Formally, the Pólya urn process is defined as follows.

Definition 11.10 (Pólya Urn Process). Let Pólya(α 1 , α 2 ) with α 1 , α 2 ∈ Z + 0 be the following urn process. At the beginning there are α 1 black marbles and α 2 white marbles in the urn. The process runs in multiple steps where α 1 (i) and α 2 (i) denote the number of black and white marbles in the urn, respectively, for every time step i. In every time step i, a black marble is added with probability α 1 (i)/(α 1 (i) + α 2 (i)), and with remaining probability α 2 (i)/(α 1 (i) + α 2 (i)) a white marble is added.

We now use this urn model to show our main result for the Bit-Propagation sub-phase, Proposition 11.5. We start by performing a worst-case analysis for color 1 in order to give a lower bound on the number of nodes of color 1 after the Bit-Propagation sub-phase.

Similarly, we will upper bound any large color C j . Then we will show that after each phase the gap between color 1 and C j grows quadratically. We will use bounds resulting from Proposition 11.4 for the numbers of nodes with bits and their color distribution among S.

For the worst-case analysis, we will assume that any node which is not in S has color C j and its bit set. We now give the formal proof.

Proof of Proposition 11.5. We consider an arbitrary but fixed Bit-Propagation sub-phase which we model by Pólya(α 1 , α 2 ) as follows. Initially, we place for each node in S of color 1 which has its bit set at reference point τ bp1 a black marble in the urn, that is, α 1 = x 1 (τ bp1 ).

Additionally, we add for each node in S which has its bit set for any color C j = C 1 a white marble in the urn. Finally, in order to perform a worst-case analysis, we add a white marble for any node which is not in S, that is, we add an additional number of |V \S| white marbles.

We therefore have α 1 + α 2 = x(τ bp1 ) + |V \ S|. We now consider only those time steps of the Bit-Propagation sub-phase, where a node in S without bit samples another node with bit.

We couple these very steps with the Pólya urn process, where we assume that a marble is added based on the adopted color in the Bit-Propagation process, that is, if a node newly adopts a bit for color 1, we add a black marble, and if otherwise a node adopts a bit for color C j = C 1 , we add a white marble. For the worst-case analysis we assume in the Bit-Propagation process that all nodes in V \ S have a bit set for a color C j = C 1 throughout the entire process. This corresponds to the additional |V \ S| white marbles initially added to the urn.

As before, we will use the notation that x(τ ) denotes the number of nodes in S which have a bit set at reference point τ and x j (τ ) denotes the number of nodes in S of color C j which have a bit set at reference point τ . Let M be a lower bound on x(τ bp1 ), the number of bits set at the beginning of the Bit-Propagation sub-phase, and recall that according to the proof of Proposition 11.4 we have with high probability M ≥ n/(2k) .

(11.10)

We now consider the Pólya urn process. Let F (i) be the fraction of black marbles in step i of the Pólya urn process. As mentioned before, this fraction of black marbles in the Pólya urn process is a martingale. Observe furthermore that |F (i) -F (i -1)| ≤ 1/M throughout the entire urn process. Let I be the last step of the Pólya urn process and observe that I ≤ n. Applying Azuma's inequality to F (i) for any i ≤ I gives us

P[ |F (i) -F (1)| ≥ δ ] ≤ 2 • exp - δ 2 2 • i j=1 1/M 2 ≤ 2 • exp - δ 2 • M 2 2 • i .
We set δ = 4 • k • log n/n and obtain using (11.10)

P |F (i) -F (1)| ≥ 4 • k • log n/n ≤ 2 • exp - 2 • k 2 • M 2 • log n n • i ≤ 2 • exp(-2 • log n) , (11.11)
where we used that x(τ bp1 ) ≥ n/(2k) with high probability.

From the calculation above we see that with high probability the fraction of black marbles in the urn remains concentrated around the initial value. To derive a lower bound on the absolute number of black marbles at the end of the process we first bound F (1). By Proposition 11.3, we have |V \ S| ≤ n 1-8/ log log n and thus

F (1) ≥ x 1 (τ bp1 ) x(τ bp1 ) + |V \ S| ≥ x 1 (τ bp1 ) x(τ bp1 ) + n 1-8/ log log n = x 1 (τ bp1 ) x(τ bp1 )
• (1 -o(1)) (11.12) Using (11.11), we get for the end of the Bit-Propagation sub-phase that at reference point τ bp2 with high probability

F (I) ≥ F (1) -4 • k • log n/n = x 1 (τ bp1 ) x(τ bp1 ) • (1 -o(1)) -4 • n 1/ log log n log n/n = x 1 (τ bp1 ) x(τ bp1 ) • (1 -o(1)) ,
where we used that x 1 (τ bp1 ) ≥ n/(2k 2 ) ≥ n 1-3/ log log n with high probability and x(τ bp) ≤ n.

Hence,

x 1 (τ bp2 ) ≥ x(τ bp2 ) x 1 (τ bp1 ) x(τ bp1 ) • (1 -o(1)) (11.13)
It remains to establish an upper bound on x j (τ bp2 ) for every other large color C j = C 1 . We will use a symmetric argument. Let C j = A be an arbitrary but fixed color and let F (i) be the fraction of black marbles in another Pólya urn process which we use to bound the size of color C j . As before, we use the black marbles to represent C j , the color under investigation, and the white marbles to represent all other colors C i = C j . For the worst-case analysis, we again assume that all nodes of V \ S have their bit set for color C j . We apply a similar computation as before and observe, now for color C j , that

F (1) ≤ x j (τ bp1 ) + |V \ S| x(τ bp1 ) + |V \ S| ≤ x j (τ bp1 ) + |V \ S| x(τ bp1 ) ≤ x j (τ bp1 ) + n 1-8/ log log n x(τ bp1 ) ≤ x j (τ bp1 ) x(τ bp1 ) + n 1-8/ log log n n 1-3/ log log n ≤ x j (τ bp1 ) x(τ bp1 ) + n -5/ log log n .
Again using (11.11), we get with high probability

F (I) ≤ F (1) + 4 • k • log n/n = x 1 (τ bp1 ) x(τ bp1 ) + n -5/ log log n + n -1/3 ≤ x 1 (τ bp1 )
x(τ bp1 ) + 2n -5/ log log n .

Thus, using that x(τ bp2 )/x(τ bp2 ) ≤ 2k with high probability we get

x j (τ bp2 ) ≤ x(τ bp2 ) • x j (τ bp1 ) x(τ bp1 ) + 2n -1/ log log n • 2n -5/ log log n = x(τ bp2 ) • x j (τ bp1 ) x(τ bp1 ) + 4n -5/ log log n .
Furthermore, from the calculation above and (11.13) we obtain for all C j that with high probability

x j (τ bp2 ) = x j (τ bp1 ) • x(τ bp2 ) x(τ bp1 ) • (1 ± o(1)) + O n -5/log log n .
By Proposition 11.4, we have that with high probability

x j (τ bp1 ) = ĉj (τ set ) 2 n (1 ± o(1)) + O n 1-5/log log n .
Moreover, by Lemma 11.9 and Definition 11.2, we have

x(τ bp2 ) ∈ [n • (1 -o(1)), n] .
Putting everything together, we derive that with high probability We define the following reference points for Part 2.

x j (τ bp2 ) = ĉj (τ 0 ) 2 x(τ bp1 ) (1 ± o(1)) + O n 1-4/log log n .
τ end0 = 3/2 • τ end0 τ end1 = 2 • τ end0 τ end2 = 3 • τ end0 τ end3 = 4 • τ end0 τ end4 = 5 • τ end0
Observe that according to the definition of Part 2 given in Algorithm 11 we only consider the working time (and not the real time . We conclude that there are at least n/20 nodes that have already passed τ end2 and changed their color away from C 1 . However, by assumption of the lemma, these nodes have not yet passed τ end4 .

These nodes can thus switch to C 1 if they are selected to tick and choose two nodes of color 1.

We define the random variable X t as 1 when a node of color C j = C 1 is selected to tick and changes its color to C 1 and as -1 if a node of color C 1 is selected to tick and changes its color to any other color C j = C 1 . If neither of these cases apply, we define X t to be zero. Observe, that the probability for X t to be negative is maximized when b t = n -a t . Therefore, we have

X t =           
1 with probability at least 1/20 Proof. By Proposition 11.5 we have

x j (τ bp2 ) = ĉj (τ 0 ) 2 x(τ bp1 ) • (1 ± o(1)) + O n 1-4/log log n .
Observe that due to the definition of x j and S, we have x j (τ t ) = x j (τ bp2 ). Furthermore, note that ĉ1 (τ 0 ) ≥ n/k ≥ n 1-1/ log log n and hence

ĉ1 (τ 0 ) 2 x(τ bp1 ) ≥ n 1-2/ log log n = ω n 1-4/log log n
Let a := ĉ1 (τ 0 + T ) the nodes of color 1 belonging to S at the the beginning of the next round. Define b analogously for color B. We consider the ratio between and show a quadratic growth w.r.t. ĉ1 (τ 0 ) 2 /ĉ 2 (τ 0 + T ) 2 . We derive

a b ≥ ĉ1 (τ 0 ) 2 x(τ bp1 ) • (1 -o(1)) ĉ2 (τ 0 ) 2 x(τtc) • 1 + o(1)) + O n 1-4/log log n ≥ ĉ1 (τ 0 ) 2 ĉ2 (τ 0 ) 2 • (1 -o(1)).
Hence, for sufficiently large constant , we have after

Increasing the Number of Opinions

In our proofs we considered the setting k ≤ exp(log n/ log log n).

However, it is possible to allow for any k = O(n ε ) (we still require that a ≥ (1 + ε)b).

This requires the algorithm to have a bound on k so that the length of block ∆ is adapted to ∆ = Θ(log k + log n/ log log n). This is sufficient to get an equivalent notion of weak synchronicity. Due to the quadratic doubling, the algorithm requires O(log log n) phases.

The length of the second part of the algorithm remains untouched resulting in a run time of O(log k • log log n + log n).

Conclusions and Further Work

We introduced an algorithm to solve the plurality consensus in the asynchronous setting.

Our algorithm achieves the best the possible asymptotic run time in the setting where the number of opinions k is bounded by exp(log n/ log log n).

It remains an open question whether there exists an algorithm with the same run time allowing for k = O(n ε ) opinions;

we note that even in the synchronous setting this questions is open.

We believe that the concept of weak synchronicity (including the Sync Gadget and the tactical waiting) as well as our analysis techniques may well prove to be of independent interest.

We showed our main result assuming independent Poisson clocks with parameter 1.

However, our techniques should carry over to a much more general setting where the nodes' clocks follow distributions satisfying the following properties: in Θ(log n/ log log n) periods n -n/e O(log n/ log log n) nodes tick Θ(log n/ log log n) times with high probability; a message is spread to n -n/e O(log n/ log log n) nodes within Θ(log n/ log log n) periods; at each (global) time t at least n -n/e Ω(log n/ log 2 log n) nodes ticked t ± O(log n/ log log n) many times; and the two-choice protocol converges within O(log n) steps if the majority opinion is supported by 19/20 of the nodes. We believe we may also be able to relax the assumption that the nodes' clocks tick independently.

Moreover, we assumed that once a node contacts another node, it receives that node's response without any delay. This assumption, however, might be unrealistic in real networks (or other models of asynchronicity). We may address this issue by extending our model to allow for response delays following some exponential distribution with constant parameter (which need not be 1, but must be independent of n).

Finally, we feel that the ideas presented here may be applicable to the adaptation of synchronous protocols to asynchronous settings for a much wider class of problems, perhaps even eventually leading to a generic framework.

Chapter 12

Consensus via Load Balancing [BFK+16b]

In this chapter we consider plurality consensus on arbitrary connected and undirected graphs and a wide range of communication modes: From anything between simple sequential communication with a single neighbor (often used in biological settings as a simple variant of asynchronous communication [START_REF] Aspnes | An Introduction to Population Protocols[END_REF]) to fully parallel communication where all nodes communicate with all their neighbors simultaneously (e.g. broadcasting models in distributed computing). This diversity turns out to be a major obstacle for algorithm design, since protocols (and their analysis) to a large degree depend upon the employed communication mechanism.

We present two simple protocols for the plurality consensus problem called Shuffle and Balance. Both protocols work in a very general discrete-time communication model.

The communication partners are determined by a (possibly randomized) sequence (M t ) t≥0

of communication matrices, where we assume1 N to be some suitably large polynomial in n. That is, nodes u and v can communicate in round t if and only if M t [u, v] = 1. In that case, we call the edge { u, v } active (see [START_REF] Avin | How to Explore a Fast-Changing World (Cover Time of a Simple Random Walk on Evolving Graphs)[END_REF][START_REF] Sauerwald | Tight Bounds for Randomized Load Balancing on Arbitrary Network Topologies[END_REF] for related graph models). Our results allow for a wide class of communication patterns (which can even vary over time)

as long as the communication matrices have certain "mixing" properties (cf. Section 12.3).

In fact, load balancing is the source of inspiration for our protocols. Initially, each node creates a suitably chosen number of tokens labeled with its own opinion. Our Balance protocol then performs discrete load balancing on these tokens, allowing each node to get an estimate on the total number of tokens for each opinion. The Shuffle protocol keeps the number of tokens on every node fixed, but shuffles tokens between communication partners.

By keeping track of how many tokens of their own opinion (label) were exchanged in total, nodes gain an estimate on the total (global) number of such tokens. Together with a simple broadcast routine, all nodes can determine the plurality opinion.

The running time of our protocols is the smallest time t where all nodes have stabilized on the plurality opinion. That is, all nodes have determined the plurality opinion and will not change. This time depends on the network G, the communication pattern (M t ) t≥0 , and the initial bias towards the plurality opinion (cf. Section 12.3). For both protocols we show a strong correlation between their running time, the mixing time of certain random walks, both of which are used in the analysis of recent load balancing results [START_REF] Sauerwald | Tight Bounds for Randomized Load Balancing on Arbitrary Network Topologies[END_REF].

To • T in the diffusion model (all nodes communicate with all their neighbors at once).

To the best of our knowledge, these match the best known bounds in the corresponding models. For an arbitrary bias (in particular, arbitrarily small bias), the protocols differ in their time and space requirements. More details of our results can be found in Section 12.1.

Results

We introduce two protocols for plurality consensus, called Shuffle and Balance. Both solve plurality consensus under a diverse set of (randomized or adversarial) communication patterns in arbitrary graphs for any positive bias. We continue with a detailed description of our results.

Shuffle.

Our main result is the Shuffle protocol. In the first time step each node generates γ tokens labeled with its initial opinion. During round t, any pair of nodes connected by an active edge (as specified by the communication pattern (M t ) t≤N ) exchanges tokens. We show that Shuffle solves plurality consensus and allows for a trade-off between running time and memory.

More exactly, let the number of tokens be γ = O log n/(α2 • T ) , where T is a parameter to control the trade-off between memory and running time Balance. The previous protocol, Shuffle, allows for a nice trade-off between running time and memory. If the number of opinions is relatively small, our much simpler Balance protocol gives better results.

In Balance, each node u maintains a k-dimensional load vector. Where j denotes u's initial opinion, the j-th dimension of this load vector is initialized with γ ∈ N (a sufficiently large value) and any other dimension is initialized with zero. In each time step, all nodes perform a simple, discrete load balancing on each dimension of these load vectors. Our results imply, for example, that plurality consensus on expanders in the sequential model is achieved in only O(n • log n) time steps with O(k) memory bits per node (assuming a constant initial bias).

Balance can be thought of as a (slightly simplified) version of [START_REF] Alistarh | Fast and Exact Majority in Population Protocols[END_REF] or [KDG03] that generalizes naturally to k ≥ 2 and arbitrary (even dynamic) graphs. In the setting of [START_REF] Alistarh | Fast and Exact Majority in Population Protocols[END_REF] (but as opposed to [START_REF] Alistarh | Fast and Exact Majority in Population Protocols[END_REF]

Approach and Technical Contributions

While our protocol Shuffle is relatively simple, the analysis is quite involved. The idea is to observe that after t mix time steps, each single token is on any node with (roughly) the same probability; the difficulty is that token movements are not independent. The main ingredients for our analysis are Lemma 12.5 and Lemma 12.6, which generalize a result by Sauerwald and Sun [START_REF] Sauerwald | Tight Bounds for Randomized Load Balancing on Arbitrary Network Topologies[END_REF] (we believe that this generalization is interesting in its own right).

These lemmas show that the joint distribution of token locations is negatively correlated, allowing us to derive a suitable Chernoff bound. Once this is proven, nodes can "count" tokens every t mix time steps, building up over time an estimate of the total number of tokens Generalized Mixing Time Consider a fixed sequence (M t ) t≥1 of communication matrices.

In the following we generalize the standard definition for the mixing time to our general communication patterns. Let d(t) = max x,t 1 x • t 1 +t t =t 1 P t -π TV and d(t) = max x,y,t 1 x • t 1 +t t =t 1 P t -y • t 1 +t t =t 1 P t TV , where • TV denotes the total variation distance. We define the mixing time to be t mix (ε) = min{t ≥ 0 : d(t) ≤ ε}. For convenience we adapt the definition to only include t 1 ∈ poly(n); otherwise, the mixing time might be arbitrary large for many communication patterns such as the sequential model.

The mixing time can be seen as the worst-case time required by a random walk to get "close" to the uniform distribution. If the parameter ε is not explicitly stated, we consider t mix := t mix (n -5 ).

Protocol Shuffle -Theorem 12.1

Our main result is the following theorem, stating the correctness as well as the time and space-efficiency of Shuffle. The protocol is described in Section 12.4.1, followed by its analysis in Section 12.4.2. some notation for the shuffle part of our protocol at time t. To ease the discussion, we consider u as a neighbor of itself and speak of deg u (t) + 1 neighbors. For i ∈ [deg u (t) + 1],

let N t (u, i) ∈ V denote the i-th neighbor of u (in an arbitrary order). Fix a node u and let u's tokens be numbered from 1 to γ. Our assumption on γ allows us to partition the tokens into deg u (t) + 1 disjoint subsets (slots) S i ⊆ [γ] of size P t [u, v] • γ each, where v = N t (u, i).

Let π t,u : [γ] → [γ] be a random permutation of u's tokens at time t. All tokens j of node u at time t with π t,u (j) ∈ S i are sent to u's i-th neighbor. To ease notation, we drop the time index t and write π u instead of π t,u (and, similarly for deg u and N (u, i)).

A configuration c describes the location of all γn tokens at a given point in time. For a token j ∈ [γn] we use u j ∈ V to denote its location in configuration c (which will always be clear from the context). For each such token j we define a random variable

X j ∈ [deg u j +1]
with X j = i if and only if π u j (j) ∈ S i . In other words, X j indicates to which of u j 's neighbors token j is sent. Our key technical lemma (Lemma 12.5) establishes the negative regression condition for these X = (X j ) j∈[γn] variables.

Formally, negative regression is defined as follows:

Definition 12.3 (Neg. Regression [DR98, Def. 21]). For n ∈ N, a vector (X 1 , X 2 , . . . , X n ) of random variables is said to satisfy the negative regression condition (NRC) if

E[ f (X l , l ∈ L )|X r = x r , r ∈ R ]
is non-increasing in each x r for any disjoint L , R ⊆ [n] and for any non-decreasing function f .

The intuition is as follows. Consider two disjoint subsets R and L of [γn] and let X R , X L be the corresponding sets of random variables. The higher the values of the random variables X R we condition on, the (monotoinically) smaller the expectation of the random variables X L .

Lemma 12.4 ([DR98, Lemma 26]). Let n ∈ N and assume (X 1 , X 2 , . . . , X n ) satisfy the negative regression condition. Consider an arbitrary index set I ⊆ [n] as well as any family of non-decreasing functions f i (i ∈ I). Then, we have

E i∈I f i (X i ) ≤ i∈I E[ f i (X i ) ].
(12.2) Lemma 12.5 (NRC). Fix a configuration c and consider the random variables (X j ) j∈ [γn] . Then (X j ) j∈[γn] satisfies the negative regression condition.

Proof. Recall that u j is the location of token j in configuration c and that X j ∈ [deg

u j +1]
indicates the neighbor of u that token j is sent in the next step. We show for any u ∈ V that (X j ) j : u j =u satisfies the NRC. The lemma's statement follows since the π u are chosen independently: if two independent vectors (X j ) and (Y j ) satisfy the NRC, then so do both together. That is, we need to show

E[ f (X l , l ∈ L )|X r = x r , r ∈ R ] ≤ E[ f (X l , l ∈ L )|X r = xr , r ∈ R ], (12.3)
where x r = xr holds for all r ∈ R \ { r } and x r > xr for a fixed index r ∈ R.

We prove Inequality (12.3) via a coupling of the processes on the left-hand side (LHS process) and right-hand side (RHS process) of that inequality. Since x r = xr , these processes involve two slightly different probability spaces Ω and Ω, respectively. To couple these, we employ a common uniform random variable U i ∈ [0, 1). By partitioning [0, 1) into d + 1 suitable slots for each process (corresponding to the slots S i mentioned above), we can use the outcome of U i to set the X j in both Ω and Ω. We first explain how to handle the case x r -xr = 1. The case x r -xr > 1 follows from this by a simple reordering argument. So assume x r -xr = 1. We reveal the yet unset random variables X j (i.e., j ∈ R) one by one in order of increasing indices. To ease the description assume (w.l.o.g.) that the tokens from R are numbered from 1 to |R|. When we reveal the j-th variable (which indicates the new location of the j-th token), note that the probability p j,i that token j is assigned to N (u, i) (the i'th neighbour of u) depends solely on the number of previous tokens j < j that were assigned to N (u, i). Thus, we can denote by p j,i : N → [0, 1] a function mapping

x ∈ N to the probability that j is assigned to N (u, i) conditioned on the event that exactly

x previous tokens were assigned to N (u, i). We observe that p j,i is non-increasing. For a vector x ∈ N d+1 , we define a threshold function T j,i : N d+1 → [0, 1] by T j,i (x) := i ≤i p j,i (x i )

for each i ∈ [d + 1]. To define our coupling, let β j,i := |{ j < j | X j = i }| denote the number of already revealed variables with value i in the LHS process and define, similarly, βj,i := |{ j < j | Xj = i }| for the RHS process. We use β j , βj ∈ N d+1 to denote the corresponding vectors. Now, to assign token j we consider a uniform random variable U j ∈ [0, 1) and assign j in both processes using customized partitions of the unit interval. To this end, let T j,i := T j,i (β j ) LHS T j,1 T j,2 T j,3 T j,4 The first figure depicts the situation after the random decision of the j-th token is made and the second figure depicts the coupling for the subsequence token j + 1. In this example, there are d + 1 = 4 different slots for the LHS and RHS process, x r = 3 and xr = 2. On the left, the uniform random variable Uj falls into slot [Tj,1, Tj,2) for the LHS process (causing j to be sent to node N (u, 2)) and into slot [ Tj,2, Tj,3) for the RHS process (causing j to be sent to node N (u, 3)).

RHS

and Tj,i := T j,i ( βj ) for each i ∈ [d + 1]. We assign X j in the LHS and RHS process as follows:

• LHS Process: X j = x j = i if and only if U j ∈ [T j,i-1 , T j,i ),

• RHS Process: X j = xj = i if and only if U j ∈ [ Tj,i-1 , Tj,i ).

See Figure 12.1 for an illustration. Our construction guarantees that the coupling is valid (i.e., considered in isolation, both the LHS and RHS process behave correctly).

At the beginning of this coupling, only the variables X r corresponding to tokens r ∈ R are set, and these differ in the LHS and RHS process only for the index r ∈ R, for which we have X r = x r (LHS) and X r = xr = x r -1 (RHS). For the first revealed token j = |R| + 1, this implies β j,x r = βj,x r + 1, β j,x r -1 = βj,x r -1 -1, and β j,i = βj,i for all i ∈ { x r, x r -1 }.

By the definitions of the slots for both processes, we get T j,i = Tj,i for all i = x r -1 and T j,x r -1 > Tj,x r -1 (cf. Figure 12.1). Thus, the LHS and RHS process behave differently if and only if U i ∈ [ Tj,x r -1 , T j,x r -1 ). If this happens, we get x j < xj (i.e., token j is assigned to a smaller neighbor in the LHS process). This implies β j+1 = βj+1 and both processes behave identical from now on. Otherwise, if U i ∈ [ Tj,x r -1 , T j,x r -1 ), we have βj+1 -β j+1 = βjβ j and we can repeat the above argument. Thus, after all X j are revealed, there is at most one j ∈ L for which x j = xj , and for this we have x j < xj . Since f is non-decreasing, this guarantees Inequality (12.3). To handle the case x r -xr > 1, note that we can reorder the intervals [T j,i-1 , T j,i ) used for the assignment of the variables such that the corresponding slots for x r and xr are neighboring. Formally, this merely changes in which order we consider the neighbors in the definition of the functions T j,i . With this change, the same arguments as above apply.

Before proving the majorization of tokens with random walks (Lemma 12.6) we require further notation. Let S denote our random Shuffle process, and W the random walk process in which each of the γn tokens performs an independent random walk according to the sequence of random walk matrices (P t ) t∈N (i.e., a token on u uses P t (u, •) for the configuration c. Then the following identities hold: = P E t |c(t ) = c .

P E t
Using the law of total probability, we conclude P[ E t +1 ] ≤ P[ E t ], as required.

In the remainder we prove (12.6) and (12.7). Remember the definitions from Lemma 12.6 and its proof. We use the shorthand deg u j = deg u j (t + 1). Remember that each X j indicates to which of the deg u j +1 neighbors of u j (where u j is considered a neighbor of itself) a token j moves during time step t + 1. Thus, given the configuration c(t ) = c immediately before time step t + 1, there is a bijection between any possible configuration c(t + 1) and outcomes of the random variable vector X = (X j ) j∈ [γn] . Let c x denote the configuration corresponding to a concrete outcome X = x ∈ [deg u j +1] γn . Thus, we where (a) follows from law of total probability, (b) follows by using the bijection between c(t + 1) and X (if c(t ) is given) and that the process S W (t + 1) consists of independent random walks if c(t + 1) is fixed, (c) we use the definition of the auxiliary functions h j (i), which equal the probability that a random walk starting at time t +1 from u j 's i-th neighbor reaches a node from D.

For the claim's second statement, we do a similar calculation for the process S W (t ). By definition, this process consists already from time t onward of a collection of independent random walks. For a fixed configuration cc = cc(t ) let X denote the vector obtained after each token performed a step of an independent random walk. Thus, following the same arguments as before we obtain, The difference to before is that the Xj are independent and thus we obtain,

P E t |c(t ) = c = E   j∈B h j ( Xj )|c(t ) = c   = j∈B E h j ( Xj )|c(t ) = c = j∈B E h j (X j )|c(t ) = c ,
where the last equality stems from the fact that if consider only one token, then its distribution is the same in both processes.

Separating the Plurality via Chernoff

We rely on a bound of Theorem A.5 by [START_REF] Azar | Balanced Allocations[END_REF] which allows us to majorize a sequence of "weakly dependent" variables by a sequence of binomially distributed variables.

We are finally able to prove the following Chernoff-like bound.

Lemma 12.7 (Token Concentration). Consider any subset B of tokens, a node u ∈ V , and an integer T . Let X := 1≤t≤T j∈B X j,t , where X j,t is 1 if token j is on node u at time and, thus, there exists a path from u to v (with respect to the communication matrices).

If there is such a path for every node v, the counter of u was also propagated to that v and we have τ ≤ t mix . Consequently, at time t all nodes have the correct majority opinion.

This implies the desired time bound. For the memory requirements, note that each node u stores γ tokens with a label from the set [k] (γ • O(log k) bits), three opinions (its own, its plurality guess, and the dominating opinion; O(log k) bits), the two counters c u and e u and the time step counter. The memory to store the counter c u and e u is O(γT ). Finally, the time step counter is bounded by O(log(T • t mix )) bits. This yields the claimed space bound.

12.5 Protocol Balance -Theorem 12.9

Protocol Description. The idea of our Balance protocol is quite simple: Every node u stores a k-dimensional vector t (u) with k integer entries, one for each opinion. Balance performs an entry-wise load balancing on t (u) according to the communication pattern M = (M t ) t≥0 and the corresponding transition matrices P t (cf. Section 12.3). Once the load is properly balanced, the nodes look at their largest entry and assume that this is the plurality opinion (stored in the variable plu u ).

In order to ensure a low memory footprint, we must not send fractional loads over active edges. To this end, we use a rounding scheme from [BCF+15, SS12], which works as follows:

Consider a dimension i ∈ [k] and let i,t (u) ∈ N denote the current (integral) load at u in dimension i, then u sends i,t (u) • P t [u, v] tokens to all neighbors v with M t [u, v] = 1.

This results in at most deg u (t) remaining excess tokens ( i,t (u) minus the total number of tokens sent out). These are then randomly distributed (without replacement), where neighbor v receives a token with probability P t [u, v]. In the following we call the resulting balancing algorithm the Vertex-Based Balancer algorithm.

The formal description of protocol Balance is given in Listing 13.

Algorithm 13: Protocol Balance as executed by node u at time t. At time zero, each node initializes ou,0(u) := γ and j,0(u) := 0 for all j = ou. . ≤ e -eαn • (e 2 αn) 6eαb i-1 (6eαb i-1 ) 6eαb i-1 ≤ 1 • (e 2 αb i-1 ) 6eαb i-1 (6eαb i-1 ) 6eαb i-1 ≤ 2 -6eαb i-1 . Proof. The statement of the theorem provided in [START_REF] Hajek | Hitting-Time and Occupation-Time Bounds Implied by Drift Analysis with Applications[END_REF] requires besides (i) and (ii) to choose constants η, and ρ such that 0 < ρ ≤ λ , η < ε 0 /c and ρ = 1 -ε 0 • η + cη 2 where

Algorithm

A.2.4 Hajek's Theorem

c = E e λ Z -(1 + λ E[ Z ]) λ 2 = ∞ k=2 λ k-2 k! E Z k .
With these requirements it then holds that for all b and t -b) . (A.4)

P[ Y (t) ≥ b|F(0) ] ≤ ρ t e η(Y (0)-b) + 1 -ρ t 1 -ρ • D • e η(a
In the following we bound (A.4) by setting η = min { λ , ε 0 • λ 2 /(2D), 1/(2ε 0 ) }. The following upper and lower bound on ρ follow.

• ρ = 1 -ε 0 • η + cη 2 ≤ 1 -ε 0 • η + ε 0 • η • c • λ 2 /(2D) ≤ 1 -ε 0 • η + ε 0 • η/2 = 1 -ε 0 • η/2,
where we used c ≤ D/λ 2 .

• ρ = 1 -ε 0 • η + cη 2 ≥ 1 -ε 0 /(2ε 0 ) ≥ 0.

We derive, from (A.4) using that for any t ≥ 0 we have 0 ≤ ρ t < 1

A.2.6 Doob-Martingale

We thus consider the process of uncovering Y v (τ ) one node after the other in order to obtain the Doob martingale of Y (τ ) as follows. We define the sequence Z j (τ ) as Therefore Z j (τ ) is indeed the Doob martingale of Y (τ ).

A.2.7 Azuma-Hoeffding Inequality

The following definitions and propositions were introduced in [DP09].

Definition A.15 (Martingale).

A martingale is a sequence of random variables X 0 , X 1 , . . . such that for all i ≥ 1 E[ X i | X 0 , X 1 , . . . , X i-1 ] = X i-1 .

A.3.3 Positive Recurrence

A Markov chain X is time-homogeneous (transition probabilities are time-independent), irreducible (every state is reachable from every other state1 ), and aperiodic (path lengths have no period). Recall that such a Markov chain is positive recurrent (or ergodic) if the probability to return to the start state is 1 and the expected return time is finite. In particular, this implies the existence of a unique stationary distribution. Positive recurrence is a standard formalization of the intuitive concept of stability. See [START_REF] Levin | Markov Chains and Mixing Times[END_REF] for an excellent introduction into Markov chains and the involved terminology.

Theorem A.22 (Fayolle et

• Z 0 = s ∈ [0, b] • P[ Z t = i | Z t-1 = i -1 ] = p for i ∈ [1, b -1], t ≥ 1 • P[ Z t = i | Z t-1 = i + 1 ] = 1 -p for i ∈ [1, b -1], t ≥ 1 • P[ Z t = i | Z t-1 = i ] = 1 for i ∈ {0,

A.4.1 Spectral Gap

We will frequently use the following basic fact about lazy random walks, which in fact also holds for arbitrary reversible Markov chains: where u k is the corresponding eigenvector to λ k . Since all eigenvalues are non-negative, we conclude from (A.7) that p t x,x is non-increasing in t as needed. Since G is bipartite and regular, it is not difficult to verify that λ n = -1 and u n (x) = 1/n if x ∈ V 1 and u n (x) = -1/n if x ∈ V 2 is the corresponding eigenvector. Hence, p t

Proposition
x,y -

2 n • 1 + (-1) t-1 ≤ π(y) • n-1 k=2 u k (x) • u k (y) • λ t k ≤ 2 n • max 2≤k≤n-1 λ t k • 1 n • n-1 k=2 |u k (x) • u k (y)| ≤ 2 n • max 2≤k≤n-1 λ t k • n-1 k=2 u k (x) 2 • n-1 k=2 u k (y) 2
at least 1-1/e, the distribution of Y s is equal to that of a stationary distribution. Hence with probability (1 -1/e) 2 , X s and Y s are drawn independently from the stationary distribution.

In this case, it follows by Markov's inequality that the two random walks meet before step s + 2t π meet with probability at least 1/2. Overall, we have shown that with probability at least (1 -1/e) 2 • 1/2, a meeting occurs in a single epoch. Since this lower bound holds for every epoch, independent of the outcomes in previous epochs, the upper bound on the expected time t meet follows. The upper bound on t coal in terms of t π coal is shown in exactly the same way.

A.4.3 Ramanujan graphs

The following is a simple corollary from a recent work by Marcus et al. [START_REF] Marcus | Interlacing Families IV: Bipartite Ramanujan Graphs of All Sizes[END_REF] on the existence of Ramanujan graphs. Furthermore note that q t x,y ≥ 2/n due to Proposition A.26.(iii) for even (or odd) t depending on whether x and y are in the same partitions. 20n , where we note that such a t exists due to max{λ 2 ( Q), |λ n-1 ( Q)|} = O(1/ √ d). We choose s to be the smallest odd integer being greater than 20t. To translate from the non-lazy random walk Q to a lazy-random walk P , let Z denote the number of non-loops performed by a lazy random walk of length s. Since, the probability for a self-loop is 1/2 and the number of self-loops is binomially distributed, we have

P[ Z ≥ t ] ≥ 19/20.
By symmetry and the fact that s is odd, P[ Z is even ] = 1 2 . Hence, by the Union bound,

P[ Z is even | Z ≥ t ] ≥ P[ Z is even ∩ Z ≥ t ] ≥ P[ Z is even ] -P[ Z < t ] ≥ 9 20 ,

Figure 1 .

 1 Figure 1.1 A social network generated by the Forest Fire Process. . . . . . . .

  Figure 1.1 A social network generated by the Forest Fire Process. . . . . . . .

Figure 1. 2

 2 Figure 1.2 Typical round of Greedy[2]. . . . . . . . . . . . . . . . . . . . . . .

Figure 1. 3

 3 Figure 1.3 Illustration of the central consensus dynamics. . . . . . . . . . . . .

Figure 1

 1 Figure 1.4 A réseau social généré par le forest fire processus. . . . . . . . . . .

Figure 1. 5

 5 Figure 1.5 Un tour typique de Greedy[2]. . . . . . . . . . . . . . . . . . . . .

Figure 1. 6

 6 Figure 1.6 Illustration des dynamiques aléatoires. . . . . . . . . . . . . . . . .

Figure 3. 1

 1 Figure 3.1 The general potential approach. . . . . . . . . . . . . . . . . . . . .

Figure 4

 4 Figure 4.1 A social network generated by the Forest Fire Process. . . . . . . .

Figure 4. 2

 2 Figure 4.2The change of the potential φ in the Forest Fire Process. . . . . . .

Figure 4

 4 Figure 4.3 A different take on the potential change φ in the Forest Fire Process model. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Figure 4. 4

 4 Figure 4.4 The percolation Burn Process (Algorithm 5). . . . . . . . . . . . . .

Figure 5. 1

 1 Figure 5.1 Typical round of Greedy[2]. . . . . . . . . . . . . . . . . . . . . . .

Figure 5. 2

 2 Figure 5.2Combinatorial approach to bounding the maximum load. . . . . . .

Figure 7. 1

 1 Figure 7.1 Illustration of the central consensus dynamics. . . . . . . . . . . . .

Figure 8. 1

 1 Figure 8.1 The graph described in Section 8.3.5 with t coal = Ω(t meet + √ α•log n• t mix ). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Figure 8.2 Illustration of the process P imm . . . . . . . . . . . . . . . . . . . . .

Figure 8. 3

 3 Figure 8.3 An illustration of couplings between the processes. . . . . . . . . .

Figure 8. 4

 4 Figure 8.4 The graph described in Section 8.3.5 with t coal = Ω(t meet + √ α•log n• t mix ). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Figure 9. 1

 1 Figure 9.1 Reducing from consensus to coalescing. . . . . . . . . . . . . . . . .

Figure 11. 1

 1 Figure 11.1 Graphical representation of one phase of Algorithm 10. Each phase consists of T = 10 • ∆ ticks. . . . . . . . . . . . . . . . . . . . . . .

Figure 12. 1

 1 Figure 12.1 The coupling of tokens. . . . . . . . . . . . . . . . . . . . . . . . . .

Figure 12

 12 Figure 12.2 High-level idea of Lemma 12.8. . . . . . . . . . . . . . . . . . . . .

Figure 1 . 1 :

 11 Figure 1.1: A social network generated by the Forest Fire Process. The node sizes and the colors are a function of the degrees and the node labels correspond to the arrival times.

ball 1 ball 2 ball 3 ball 4 Figure 1 . 2 :

 412 Figure 1.2: The figure depicts a typical round of Greedy[2].In this example we have n = 5 and 4 balls arrive. Balls 1, 2, and 3 choose the same bin with a load of 2 and a bin with larger node and hence all move the same bin resulting in that bin having the highest load. Moreover, Ball 4 chooses two bins with equal load and chooses one of these uniformly at random. At the end of the round all non-empty bins delete one ball (marked gray).

Figure 1 . 3 :

 13 Figure 1.3: The figure depicts the three models with two examples each.

  consensus deterministically is impossible in many settings of interest. A natural way of defying the impossibility results is by using randomized protocols -the nucleus of this thesis. The prevailing randomized dynamics in this area are Voter, 2-Choices, and 3-Majority, which we define below (see also Figure 1.3 for an illustration).

Figure 1

 1 Figure 1.4: A réseau social généré par le forest fire processus.

ball 1 ball 2 ball 3 ballFigure 1 . 5 :

 15 Figure 1.5: Un tour typique de Greedy[2].

  .6 pour une illustration). Voter est sans doute la dynamique aléatoire la plus simple possible et 2-Choices et 3-Majority sont tout aussi simples et sont en même temps des solutions auto-stabilisantes efficaces pour l'accord byzantin [PSL80, Rab83] : parvenir à un consensus en présence d'un adversaire qui peut perturber un ensemble borné de noeuds à chaque tour [BCN+14b, BCN+16, CER14, EFK+16].

Figure 1 . 6 :

 16 Figure 1.6: Illustration des dynamiques aléatoires.

.

  Graphs G = (V, E) considered in this thesis may be directed or undirected; typically we assume |V | = n and |E| = m, though if there is scope for confusion we use |V | or |E| explicitly. For undirected graphs, for a node v ∈ V we denote by N (v) its neighbourhood, i. e., N (v) := {w | {v, w} ∈ E}, and its degree by deg(v) := |N (v)|. In the case of directed graphs, we denote by N + (v) := {w | (v, w) ∈ E} its out-neighbourhood and by deg + (v) := |N + (v)| its out-degree. Similarly, N -(v) := {u | (u, v) ∈ E} denotes its inneighbourhood and deg -(v) := |N -(v)| its in-degree. Furthermore, d avg := v∈V deg(v)/n denotes the average degree. We use d max and d min to refer the maximum and minimum degree. Whenever there is scope for confusion, we use the notations deg G (u), N G (v), d avg (G), etc. to emphasize that the terms are with respect to graph G.

  there exists a directed path from v to x in H} Definition Balls-into-Bins with Deletions. The Balls-into-Bins with Deletions process (see Chapter 5) is a fundamental process modeling, among other things, the load distribution in distributed systems. The system consists of n bins and balls which arrive over time according to two strategies: Greedy[1] and Greedy[2]. The process works as follows.

  [1], and Greedy[2].

  A representation of the potential as a random walk: The arrow depict the transition probabilities of the walk for current node. ∆φ prob. (b) The density function of the potential change.

Figure 3 . 1 :

 31 Figure 3.1: The potential has a negative drift and the potential change exhibits a tail-bound.

Figure 3 .

 3 Figure 3.1 for an illustration.

  Figure 4.1 for an illustration. Leskovec et al. observed through simulation, that the Forest Fire Model appears to have the shrinking diameter property, but leave open the question of providing a rigorous proof: "Rigorous analysis of the Forest Fire model appears to be quite difficult. However in simulations we find that [...] we can produce graphs that [...] have diameter that decrease."

Figure 4

 4 Figure 4.1: A social network generated by the Forest Fire Process. The node sizes and the colors are a function of the degrees and the node labels correspond to the arrival times.

  proved that the RSM and the PageRank-based selection model, proposed by Pandurangan et al. [PRU06], are equivalent and also proved that the expected in-degree of vertices follows a powerlaw distribution. More recently, Mehrabian and Wormald [MW14] proved logarithmic upper bounds for the diameter in the RSM and the PageRank-based selection model as well as a logarithmic lower bound for a special case where the generated graph is a tree. The only rigorous work thus far on the Forest Fire model is by Mehrabian [MW14] who provide a logarithmic upper bound to the diameter of the Forest Fire model as well as for other well known models, e.g., the copying model and the PageRank-based selection model.

  .3) Combining (4.1), (4.2), and (4.3), we obtain E[ dist At (u, G 0 ) ] = O(log t), as desired. We now prove the lower bound on the expectation. Again, let v k denote the node which arrives at time k and recall that the vertices of G 0 arrive at time 0. Consider the path p from v k to G 0 in A k and observe that arrival times are decreasing along p. For i ≤ log 2 k let Y i denote the indicator variable of the event that some vertex of p has an arrival time in (2 i-1 , 2 i ].

Figure 4 .

 4 Figure 4.3 offers a slightly different point of view: Here we illustrate the essence of φ by focusing on the Line Fire process as opposed to focusing on the branching nature of the process.

Algorithm 7 :

 7 BurnBFS(G, v)

Figure

  Figure 4.4:The figure depicts the percolation Burn Process (Algorithm 5) on the l.h.s. and the corresponding BFS burn process (BurnBFS(G, v) in Algorithm 7) on the r.h.s. Activated edges and burnt nodes are coloured red. In this example vt-5 (level M2) is burnt by two predecessors one on level M1 and one on level M2; due to the definition of the BFS burn process, it is placed on M2 (rather than M3).

  the number of nodes on the shortest path plus one. Hence, we obtain a crude bound on P[ ∆ t = -k ] by bounding P |T vt | ≥ k + 1 . As we will argue in the following, P[ |T | ≥ k + 1 ] has an exponential tail distribution. Observe, that T vt is GW-tree with offspring distribution Poisson(eα). We have, by Proposition A.33, using that eα ≤ eα * = 1/4,

( 1 +

 1 o(1)) • ln(n)/ ln ln n for the 1-Choice process [Gon81, RS98] and ln ln(n)/ ln d + Θ(1) for the d-Choice process with d ≥ 2 [ABKU99]. If m ≥ n • ln n, the maximum load increases to m/n + Θ m • ln(n)/n [RS98] and m/n + ln ln(n)/ ln d + Θ(1) [BCSV06], respectively.

ball 1 ball 2 ball 3 ballFigure 5 . 1 :

 51 Figure 5.1: The figure depicts a typical round of Greedy[2].In this example we have n = 5 and 4 balls arrive. Balls 1, 2, and 3 choose the same bin with a load of 2 and a bin with larger node and hence all move the same bin resulting in that bin having the highest load. Moreover, Ball 4 chooses two bins with equal load and chooses one of these uniformly at random. At the end of the round all non-empty bins delete one ball (marked gray).

Proposition 5. 7

 7 (Smoothness). Let λ = λ(n) ∈ [1/4, 1]. Fix an arbitrary round t of the 2-Choice process. The load difference of all bins is (w.h.p.) bounded by O(ln n).

  Using Ψ(t) ≤ Ψ(T ) (due to the definition of T ) together with the smoothness bounds of Lemma 5.21 yields the claim. See Figure 5.2 for an illustration. Lemma 5.20. Let b ≥ 0 and consider a configuration x with Ψ(x) ≤ 2b•n and Φ(x) ≤ e α•b . Let x denote the configuration after one step of the 2-Choice process. Then, Ψ(x ) ≤ 2b • n.

  and taking the union bound.

Figure 7 . 1 :

 71 Figure 7.1: The figure depicts the three models with two examples each.

  For every pair of color distributions c, c such that φ(c ) ≤ φ(c) after one-step P remains closer towards consensus than P , i. e., E[ φ(P (c )) ] ≤ E[ φ(P (c)) ]. These requirements are satisfied by Voter and 3-Majority (Proposition 9.10) establishing the stochastic majorization between the two processes. Therefore, we use the following two-state approach: We can use our reduction from 3-Majority to Voter in order to reduce the number of opinion until we are in a regime where the known results apply. From there one we simply use the known results to reduce to one opinion.Results 2-Choices. For the 2-Choices dynamics we generalize to the setting of k > 2 colors and highlight the advantage of 2-Choices (in comparison to 3-Majority) when it comes to plurality consensus (converging towards the initially most dominant color) in the self-stabilizing setting, i. e., in presence of adversaries. We show for the complete graph (Theorem 10.2) starting with k = O(n ε ) opinions for some small constant ε > 0 and for a sufficient bias (between the most and second most dominant color) of Ω( √ n log n) that 2-Choices converges with high probability 4 to the initially most dominant color within

  where m is the number of edges, d min is the minimum degree and Φ is the conductance. Their result improves on that of Cooper et al. for certain graph classes, e. g., cycles. Their bounds hold in dynamic graphs where edges change, in a restricted way, over time. Despite the recent progress due to Cooper et al. [CEOR13] as well as Berenbrink et al. [BGKM16], for many fundamental graphs such as the hypercube and the (d-dimensional) torus, the coalescing time in the discrete setting remains unsettled. The coalescing random walk process has also been studied in continuous time; in this case, particles jump to a random neighboring node when activated according to a Poisson clock with mean 1. As Cooper and Rivera [CR16] recently pointed out "It is however, not clear whether the continuous-time results apply to the discrete-time setting."

  They show that in random d-regular graphs, with high probability all nodes agree after O(log n) steps on the initially most frequent opinion, provided that c 1 -c 2 = K • (n 1/d + d/n) for K large enough, where c 1 and c 2 denote the support of the initially most frequent and second-most frequent colors. For an arbitrary d-regular graph G, they need c 1 -c 2 = K • λ 2 • n, where λ 2 is the second largest eigenvalue. In the more recent work by Cooper et al. [CER+15]

Moreover, 2 -

 2 Median is not self-stabilizing for Byzantine agreement (unlike 3-Majority and 2-Choices [BCN+16, EFK+16]): it cannot guarantee validity 6 [BCN+16]. Another consensus process is the UndecidedDynamics. Here, each node randomly samples one neighbor and, if the sample has a different color, adopts a special "undecided" color. In subsequent rounds, it tries to find a new (real) color by sampling one random neighbor. The most recent results [BCN+15b] show that, for a large enough bias, consensus is reached with high probability in at most O(k log n) rounds. Slightly more involved variants yield improved bounds of O(log k • log n) [BFGK16, GP16, EFK+16]. However, observe that for k = n all nodes become undecided with constant probability instead of agreeing on a color. Another natural variant is five-sample voting in d-regular graphs with d ≥ 5, where in each round at least five distinct neighbors are consulted. Abdullah and Draief showed for the case k = 2 an O(log d log d n) bound [AD15], which is tight for a wider class of voting protocols. A more general analysis of multi-sample voting has been conducted by Cruise and Ganesh [CG14] on the complete graph.

  dom and only the two connected nodes communicate. We refer to this communication model as the sequential model. See [AR07, AAER07] for detailed introductions. Angluin et al. [AAE08] propose a 3-state population protocol for majority voting (i.e., k = 2) on the clique. If the initial bias α is ω(log n/ √ n), their protocol agrees (w.h.p.) 9 on the majority opinion in O(n • log n) steps. Mertzios et al. [MNRS14] suggest a 4-state protocol for exact majority voting, which always returns the majority opinion (independent of α) in time O n 6 in arbitrary graphs and in time O n 2 • log(n)/α in the clique. This is optimal in that no population protocol for exact majority can have fewer than four states [MNRS14]. Alistarh et al. [AGV15] gives a protocol for k = 2 in the clique that allows for a speed-memory trade-off. It solves exact majority and has expected parallel running time 10

  Efremenko and Reingold [ER09, Section 6] highlight the difficulty in analyzing the hitting time distribution before its expectation. In the context of concentration inequalities for Markov chains, Lezaud [Lez89, p. 863] points out the requirement to spend at least mixing time steps before taking any samples. Related to that, in property testing, dealing with graphs that are far from expanders has been mentioned as one of the major challenges to test the expansion of the graph by Czumaj and Sohler[START_REF] Czumaj | Testing Expansion in Bounded-Degree Graphs[END_REF].

Figure 8 . 2 :

 82 Figure 8.2: Illustration of the process Pimm.

Figure 8 . 3 :

 83 Figure 8.3: An illustration of couplings between the processes. The squares depict the random walks.Walks of G1 are colored black and gray (phantom) and the nodes of G2 are white. The blue arrows denote the moving decisions. Observe that in Pint a phantom becomes alive (and a walk of G2 is labeled dead).

  Proposition 8.3, T * 2 (j) stochastically dominates T 2 (j) and hence it suffices to bound E[ T * 2 (j) ]. In order to bound E[ T * 2 (j) ], we define b j := 32α log(4/3)( -j + 1)(3/4) -j+1 . Since we only consider phases with j respecting -j + 1 ≥ 32 we have b j ≤ b -31 ≤ ((e -1)/e)α. Furthermore, we have b j ≥ b 0 ≥ 4α log α(3/4) ≥ 1, where the last inequality follows from (4/3) ≤ |S 0 | ≤ 4α log α, which in turn follows from definition of and the assumed bound on |S 0 |. Applying Lemma 8.4 with this value of b j , we get that for

  (i) Let Q be the transition matrix of a random walk restricted to G 2 . Let d Q (t) be the total variation distance w.r.t. the transition matrix Q. Further, let Q be the transition matrix of a random walk restricted to G. Recall that t mix (G 2 ) = O(1), by Proposition A.29. Fix an arbitrary t ∈ [2t mix (G 2 ), 2t mix (G 2 ) + 7]. In the following we show q t u,•π G (•) TV ≤ 1/e. We first consider any start vertex u ∈ V 2 \ { z} and afterwards the vertex u = z. Let D be the set of distributions over V ( G) = V 2 ∪ { z} assigning no probability mass to z, i. e., D = {D : for u ∼ D we have P[ u = z ] = 0}.

  stabilizing solutions for Byzantine agreement[START_REF] Pease | Reaching agreement in the presence of faults[END_REF][START_REF] Rabin | Randomized byzantine generals[END_REF]: achieving consensus in the presence of an adversary that can disrupt a bounded set of nodes each round[START_REF] Becchetti | Trevisan: Simple Dynamics for Plurality Consensus[END_REF][START_REF] Becchetti | Stabilizing Consensus with Many Opinions[END_REF][START_REF] Cooper | The Power of Two Choices in Distributed Voting[END_REF][START_REF] Elsässer | Efficient k-Party Voting with Two Choices[END_REF]. Further interest stems from the fact that they capture aspects 1 Equivalently, the node may adopt the color of a fixed sample (the first, or second, or third). of how agreement is reached in social networks and biological systems [BDDS10, CER14, FPM+02]. At first glance, the above processes look quite different. But a slight reformulation of 3-Majority's update rule reveals an intriguing connection: • 3-Majority (alt.): Sample two nodes independently and uniformly at random. If the samples have the same color, adopt it. Otherwise, sample a new neighbor and adopt its color.

Lemma 9. 5

 5 (1-Step Coupling). Let P α and P α be two AC-processes. Consider any two configurations c, c ∈ C with α(c) α(c). Let c and c be the configurations of P α and P α after one round, respectively. Then, there exists a coupling such that c c . Proof. Consider the processes P α and P α with the configurations c and c from the theorem's statement. Let Y = c and X = c denote the configurations resulting after one round of P α on c and P α on c, respectively. Let Θ := α(c) and Θ := α(c). As observed earlier in Section 9.3.1, we have Y ∼ Mult(n, Θ 1 ) and X ∼ Mult(n, Θ 2 ). By the theorem's assumption, we have Θ Θ. Since, by Proposition A.20, the function Θ → E f Mult(n, Θ) is Schurconvex for any Schur-convex function f (•) for which the expectation exists, we get X ≤ st Y.

  [MOA11, Definition 17.B.1]). (b) Since a cone C is complete if it is maximal with respect to functions preserving the preorder "≤ C " (cf. [MOA11, Definition 14.E.2]), C is complete (Schur-convex functions are by definition the set of all functions preserving the majorization preorder). From (a) we get that Condition (i) is actually Condition (i) of Theorem A.21. The same holds for Condition (ii). From (b) we get that C = C * = C + (cf. [MOA11, Proposition 17.B.3]), such that Conditions (i) and (ii) are equivalent by Theorem A.21. This finishes the proof. With this, Lemma 9.5 follows by a straightforward combination of the aforementioned machinery.

  Figure 9.1: Running the coalescence process from right to left (an edge from u to v means that the token on u -if any-moves to v) yields that after T = 4 rounds the number of random walks reduces to k = 2. Using the same random choices (black arrows) for the voter process and running the process from left to right (an edge from u to v means that u pulls v's opinion) we derive that the number of opinions after T = 4 rounds is also 2. This is no coincidence as we show in Proposition 9.11.

  .11) Since for all τ < T k C we have |Z τ | > k, from (9.11) it follows that |O τ 0 | = |Z τ | > k which yields the claim.

3-

  Majority and we establish a lower bound on 2-Choices showing a polynomial difference in the consensus time of both protocols. Another focus of this chapter is Byzantine agreement, where the goal is to achieve consensus in spite of an adversary who is allowed to change the opinion of a bounded number of nodes per round [PSL80, Rab83, BCN+14b, BCN+16, CER14, EFK+16].

Algorithm 8 :

 8 Distributed Voting Protocol with Two Choices Algorithm 2-Choices (G = (V, E), color : V → C) for round t = 1 to |C| • log |V | do at each node v do in parallel let u 1 , u 2 ∈ N (v) uniformly at random; if color(u 1 ) = color(u 2 ) then color(v) ← color(u 1 );

.

  even in absence of an adversary. Proof. Let z = z /2 and n = n-k+2 2 Assume that we have the following initial color distribution among the nodes. (c 1 , c 2 , c 3 , . . . , c k

  frequencies are commonly modeled by Poisson clocks, we give in the following a more theoretical model. Instead of considering the asynchronous parallel process in continuous time, we rather analyze the process in the sequential model. In this sequential model, we assume that a discrete time is given by the sequence of ticks, and at any of the discrete time steps, a node is selected uniformly at random from the set of all nodes to perform its task. According to the global clock model described above, and using the fact that for any k = Ω(n) the k'th tick of the global clock occurs at some time Θ(k/n), with probability 1 -e -Ω(n) , we can relate the number of ticks in the sequential model to the continuous time in the asynchronous model. Since k ticks in the global clock model occur within time Θ(k/n) for any k = Ω(n) with probability 1 -e -Ω(n) , the number of ticks divided by the number of nodes in the sequential model and the time in the global clock model are asymptotically the same with probability 1 -e -Ω(n) .

Theorem 11. 1 .

 1 Consider the asynchronous model. Let G = K n be the complete graph with n nodes. Let k = O(exp(log n/ log log n)) be the number of opinions. Let ε bias > 0 be a constant. Assume c 1 ≥ (1 + ε bias )•c i for all i ≥ 2, then the asynchronous plurality consensus process defined in Section 11.2 on G converges within time Θ(log n) to the majority opinion C 1 , with high probability.

  A simple algorithm for reaching consensus is 2-Choices. As we show in detail in the full version, a bias of c 1 -c 2 = O √ n log n between the two largest colors suffices to guarantee that the largest color 1 wins after O(n/c 1 • log n)rounds with high probability.

Algorithm 9 :

 9 Distributed Voting Protocol with Two Choices

. 1 )

 1 Let T denote the total number of time steps until all nodes have completed the execution of Part 1 of the asynchronous protocol defined in Algorithm 10 w.r.t. their working time. With high probability, we have

  Proof of Proposition 11.3. For every phase s = O(log log n), let J s be the set of nodes which are 1. (5∆/16)-close w.r.t. the working time at any reference point τ = s • T and 2. (∆/2)-close w.r.t. the working time at any reference point in [(s -1) • T, s • T ].

For s = 0

 0 this holds trivially since |J 0 | = n. Suppose the claims holds for phase s and consider phase s+1. We seek to show that the claim holds in the interval [s•T, (s+1)•T ]. Let τ l , τ r with τ l < τ r be an arbitrary pair of reference points with τ l ≥ s•T and τ r ≤ (s+1)•T . Let furthermore J ⊂ J s denote the set of nodes which are selected to tick τ r -τ l ± ∆/16 times in any interval [τ l , τ r ]. By Part 2 of Lemma 11.7, we have |J | ≥ |J s |(1 -exp(-9 log n/ log log n)) .(11.5)

  We now focus on the analysis of the Bit-Propagation sub-phase. Similar to the analysis of the synchronous case, we first analyze the number of bits which are set during the Bit-Propagation sub-phase without taking their color into consideration. The following lemma is based on the observation that the Bit-Propagation can be modeled by a simple asynchronous randomized-gossip-based information dissemination process.Lemma 11.9. Consider an arbitrary but fixed phase and let x(τ ) be the number of nodes in S which have a bit set at reference point τ in that phase. Assume that |S| ≥ n •(1 -exp(-8 log n/ log log n)) and that x(τ bp1 ) ≥ n/(2k). Then we have x(τ bp2 ) = |S| with high probability.

Algorithm 11 :Figure 3 :

 113 Figure 3: graphical representation of the asynchronous protocol, showing Part 1 (Algorithm 10) and Part 2 (Algorithm 11)

  give some more concrete examples of our results, let T := O(log n/(1 -λ 2 )), where 1 -λ 2 is the spectral gap of G. If the bias is sufficiently high, then both our protocols Shuffle and Balance determine the plurality opinion in time • n • T in the sequential model (only one pair of nodes communicates per time step); • d • T in the balancing circuit model (communication partners are chosen according to d (deterministic) perfect matchings in a round-robin fashion); and

  for arbitrary k), it achieves plurality consensus with probability 1 -o(1) in parallel time O(log n) and uses O(k • log(1/α)) = O(k • log n) bits per node (Corollary 12.10), an improvement by a log(n) factor.

Theorem 12. 1 .

 1 Let α = n 1 -n 2 n ∈ [ 1 /n,1] denote the initial bias. Consider a fixed communication pattern (M t ) t≥1 and let T ∈ N. Protocol Shuffle ensures that all nodes know the plurality opinion after O T • t mix (n -5 ) rounds 6 (w.h.p.) and requires 12• log(n) α 2 •T + 2∆ + 4 • log(k) + 4 log 12•log(n) α 2+ log(T • t mix ) memory bits per node.The parameter T in the statement serves as a lever to trade running time for memory.Since t mix (n -5 ) depends on the graph and communication pattern, Theorem 12.1 might look a bit unwieldy. The following corollary gives a few concrete examples for common communication patterns on general graphs.Corollary 12.2. Let G be an arbitrary d-regular graph. Shuffle ensures that all nodes agree on the plurality opinion (w.h.p.) using 12• log(n) α 2 •T + 2∆ + 4 • log(k) + 4 log 12•log(n) α 2 + log(T • t mix ) bits of memory in time • O T • log(n) 1-λ 2 in the diffusion model, • O T d•p min • log(n) 1-λ 2 in the random matching model, • O T • d • log(n)1-λ 2 in the balancing circuit model, and• O T • n • log(n)1-λ 2 in the sequential model.

Fix a node u and

  disjoint subsets L , R ⊆ { j ∈ [γn] : u j = u } of tokens on u. Define d := deg u and let f : [d + 1] |L | → R be an arbitrary non-decreasing function. We have to show that E[ f (X l , l ∈ L )|X r = x r , r ∈ R ] is non-increasing in each x r (cf. Definition 12.3).

Figure 12 . 1 :

 121 Figure12.1: The figure illustrates the coupling between the two process for the token j and the token j +1. The first figure depicts the situation after the random decision of the j-th token is made and the second figure depicts the coupling for the subsequence token j + 1. In this example, there are d + 1 = 4 different slots for the LHS and RHS process, x r = 3 and xr = 2. On the left, the uniform random variable Uj falls into slot [Tj,1, Tj,2) for the LHS process (causing j to be sent to node N (u, 2)) and into slot [ Tj,2, Tj,3) for the RHS process (causing j to be sent to node N (u, 3)).

P

  +1 |c(t ) = c = E   j∈B h j (X j )|c(t ) = c E t |c(t ) = c = j∈B E h j (X j )|c(t ) = c .(12.7)We defer the proof the end of the lemma. Using this assumption, we computeP E t +1 |c(t ) = c (12.6) = E   j∈B h j (X j )|c(t ) = c Lemma 12.4 = j∈B E h j (X j )|c(t ) = c(12.7)

  have P[ c(t + 1) = c x |c(t ) = c ] = P[ X = x|c(t ) = c ], and conditioning on c(t + 1) is equivalent to conditioning on X and c(t ). For the claim's first statement, we calculateP E t +1 |c(t ) = c = (a) = cx P E t +1 |c(t + 1) = c x • P c(t + 1) = c x |c(t ) = c ∈ D|X = x, c(t ) = c • P X = x|c(t ) = c (c) = cx j∈B h j (x j ) • P X = x|c(t ) = c = x j∈B h j (x j ) • P X = x|c(t ) = c = E   j∈B h j (X j )|c(t ) = c   ,

P

  E t |c(t ) = c = = c x P E t |c(t + 1) = c x • P c(t + 1) = c x|c(t ) = c = c x j∈B P w S W (t ) j (t) ∈ D| X = x, c(t ) = c • P X = x|c(t ) = c = c x j∈B h j (x j ) • P X = x|c(t ) = c = x j∈B h j (x j ) • P X = x|c(t ) = c

For (a)

  we show that for any given node the number of tokens received is not much larger than its exception. See Figure12.2 for an illustration. Showing (b) is slightly more involved. Due to the negative association, we refrain from showing directly that the number of tokens received by a node u of opinion o v = 1 is lower bounded by its expectation minus some deviation. Instead, we show (b) by arguing that out of the total number of tokens considered which do not end up on node v with o v = 1 is concentrated. This gives a lower bound on the number of tokens which did land on v.

  plu u := i with i,t (u) ≥ j,t (u) ∀1 ≤ i, j ≤ k ; // plurality guess definition b i and by applying the Poisson tail bound given in[START_REF] Mitzenmacher | Probability and Computing: Randomized Algorithms and Probabilistic Analysis[END_REF],P[ N i ≥ b i |n < b i-1 ] ≤ P[ Poisson(eαn) + n(1 + eα ) ≥ b i |n < b i-1 ] ≤ P Poisson(eαn) ≥ (6α )b i-1 -b i-1 (1 + eα )|n < b i-1 = P[ Poisson(eαn) ≥ 6eαb i-1 |n < b i-1 ]

Theorem A. 11 (

 11 Hajek's Theorem -Simplified version of[START_REF] Hajek | Hitting-Time and Occupation-Time Bounds Implied by Drift Analysis with Applications[END_REF] Theorem 2.3]). Let (Y (t)) t≥0 be a sequence of random variables on a probability space (Ω, F, P ) with respect to the filtration (F(t)) t≥0 . Assume the following two conditions hold:(i) (Majorization) There exists a random variable Z and a constant λ > 0, such that E e λ Z ≤ D for some finite D, and (|Y (t + 1) -Y (t)| F(t)) ≤ st Z for all t ≥ 0; and (ii) (Negative Bias) There exist a, ε 0 > 0, such for all t we haveE[ Y (t + 1) -Y (t)|F(t), Y (t) > a ] ≤ -ε 0 . Let η = min{λ , ε 0 • λ 2 /(2D), 1/(2ε 0 )}.Then, for all b and t we have P[ Y (t) ≥ b|F(0) ] ≤ e η(Y (0)-b) + 2D ε 0 • η • e η(a-b) .

P

  [ τ (g) ≥ i ] ≤ log β (g/(2X 0 )) + ∞ λ=1 log β (g/(2X 0 )) • P τ (g) > λ • log β (g/(2X 0 ) ≤ 2 • log β (g/(2X 0 )) .

  Z j (τ ) = E[ Y (τ )|T j (τ ), . . . , T 1 (τ ) ] with Z 0 (τ ) = E[ Y (τ ) ]. We have E[ Z j (τ )|T j-1 (τ ), . . . , T 1 (τ ) ] = E[ E[ Y (τ )|T j (τ ), . . . , T 1 (τ ) ]|T j-1 (τ ), . . . , T 1 (τ ) ]which, applying the tower property, gives us thatE[ Z j (τ )|T j-1 (τ ), . . . , T 1 (τ ) ] = E[ Y (τ )|T j-1 (τ ), . . . , T 1 (τ ) ] = Z j-1 (τ ).

2 i=0 1

 21 b}, t ≥ 1 Let T = min{t ≥ 0 | Z t ∈ {0, b}}. Then P[ Z T = b ] = Consider two random walks (X t ) t≥0 and (Y t ) t≥0 starting on nodes drawn from the stationary distribution. Fix an arbitrary t ∈ N. Define the collisioncounting random variables Z 1= t/Xt=Yt , Z 2 = t i= t/2 +1 1 Xt=Yt , and Z = Z 1 + Z 2 . Then P[ Z 1 ≥ 1 | Z ≥ 1 ] ≥ 1 2 . Proof. Since both nodes start from the stationary distribution, P[ Z 1 ≥ 1 ] ≥ P[ Z 2 ≥ 1 ]. By the Union bound, P[ Z ≥ 1 ] ≤ P[ Z 1 ≥ 1 ] + P[ Z 2 ≥ 1 ] ≤ 2 • P[ Z 1 ≥ 1 ]. By law of total probability, P[ Z 1 ] = P[ Z 1 ≥ 1 | Z ≥ 1 ] • P[ Z ≥ 1 ].Putting everything together yieldsP[ Z 1 ≥ 1 | Z ≥ 1 ] ≥ 1 2 .

  Proposition A.29 (cf.[START_REF] Marcus | Interlacing Families IV: Bipartite Ramanujan Graphs of All Sizes[END_REF]). For any integer d ≥ 3, there are d-regular graphs H = (V, E) with t mix = O(log n/ log(d)).Proof. Marcus et al.[START_REF] Marcus | Interlacing Families IV: Bipartite Ramanujan Graphs of All Sizes[END_REF] show that the existence of a d-regular bipartite Ramanujan graphH such that max{λ 2 ( Q), |λ n-1 ( Q)|} = O(1/ √ d), where Q = 1 d A is the transition matrix of a non-lazy random walk where A is the adjacency matrix. By the second statement of Proposition A.26, for any pair of states x, y in the same partition qtx,y ≤2 n • 1 + (-1) t-1 + 2(max{λ 2 , |λ n-1 |}) t .Similarly, x and y are in opposite partitions, qt x,y ≤ 2 n • 1 + (-1) t + 2(max{λ 2 , |λ n-1 |}) t .

Fix

  t = O(log n/ log d) such that 2 max{λ 2 ( Q), |λ n-1 ( Q)|} t ≤ 1

  Chapter 11 considers our asynchronous protocol and is based on the preceding publi-

	cation
	• Chapter 12 considers our consensus protocol inspired by load balancing and is based on org/10.4230/LIPIcs.MFCS.2016.55
	P. Berenbrink, T. Friedetzky, P. Kling, F. Mallmann-Trenn, and C. Wastell: Plural-• P. Berenbrink, B. Krayenhoff, and F. Mallmann-Trenn: Estimating the number of
	ity Consensus in Arbitrary Graphs: Lessons Learned from Load Balancing. In 24th connected components in sublinear time. In Inf. Process. Lett. volume 114 (11), 2014,
	Annual European Symposium on Algorithms ESA'16. Volume 57. Schloss Dagstuhl-pages 639-642. url: http://dx.adoi.org/10.1016/j.ipl.2014.05.008
	Leibniz-Zentrum fuer Informatik, 2016, pages 10:1-10:18. url: http : / / drops . • P. Kling, A. Cord-Landwehr, and F. Mallmann-Trenn: Slow Down and Sleep for dagstuhl.de/opus/volltexte/2016/6361 Profit in Online Deadline Scheduling. In Design and Analysis of Algorithms -First
	Publications of the authors not covered in the thesis Mediterranean Conference on Algorithms, MedAlg'12, 2012, pages 234-247. url:
	http://dx.adoi.org/10.1007/978-3-642-34862-4_17
	DC, USA, July 25-27, 2017, 2017, pages 335-344. url: http://doi.acm.org/10.
	1145/3087801.3087817
	• Chapter 10 considers the 2-Choices and is based on
	R. Elsässer, T. Friedetzky, D. Kaaser, F. Mallmann-Trenn, and H. Trinker: Efficient

, R. Levi, Z. Lotker, F. Mallmann-Trenn, and C. Mathieu: Distance in the Forest Fire Model How far are you from Eve? In Proceedings of the Twenty-Seventh Annual ACM-SIAM Symposium on Discrete Algorithms, SODA'16, 2016, pages 1602-1620. url: http://dx.adoi.org/10.1137/1.9781611974331.ch109 • Chapter 5 considers the Balls-into-bins process and is based on P. Berenbrink, T. Friedetzky, P. Kling, F. Mallmann-Trenn, L. Nagel, and C. Wastell: Self-stabilizing Balls & Bins in Batches: The Power of Leaky Bins. In Proceedings of the 2016 ACM Symposium on Principles of Distributed Computing. PODC'16, 2016, pages 83-92. url: http://adoi.acm.org/10.1145/2933057.2933092 • Chapter 8 considers the Voter and is based on V. Kanade, F. Mallmann-Trenn, and T. Sauerwald: On coalescence time in graphs-When is coalescing as fast as meeting? In CoRR, volume abs/1611.02460, 2016. url: http://arxiv.org/abs/1611.02460 • Chapter 9 considers the 3-Majority and is based on P. Berenbrink, A. E. F. Clementi, R. Elsässer, P. Kling, F. Mallmann-Trenn, and E. Natale: Ignore or Comply?: On Breaking Symmetry in Consensus. In Proceedings of the ACM Symposium on Principles of Distributed Computing, PODC'17, Washington, k-Party Voting with Two Choices. In CoRR, volume abs/1602.04667, 2016. url: http://arxiv.org/abs/1602.04667 and the preceding publication

• • V. Cohen-Addad, V. Kanade, and F. Mallmann-Trenn: Hierarchical Clustering Beyond the Worst-Case. In, 2017. To appear in NIPS 2017. url: http://arxiv.org/ abs/1702.03959 • P. Berenbrink, R. Klasing, A. Kosowski, F. Mallmann-Trenn, and P. Uznanski: Improved Analysis of Deterministic Load-Balancing Schemes. In Proceedings of the 2015 ACM Symposium on Principles of Distributed Computing, PODC'15, 2015, pages 301-310. url: http://adoi.acm.org/10.1145/2767386.2767413 • P. Berenbrink, G. Giakkoupis, A. Kermarrec, and F. Mallmann-Trenn: Bounds on the Voter Model in Dynamic Networks. In 43rd International Colloquium on Automata, Languages, and Programming, ICALP'16, 2016, pages 146:1-146:15. url: http : //dx.adoi.org/10.4230/LIPIcs.ICALP.2016.146 • V. Cohen-Addad, V. Kanade, F. Mallmann-Trenn, and C. Mathieu: Hierarchical Clustering: Objective Functions and Algorithms. In CoRR, volume abs/1704.02147, 2017. url: http://arxiv.org/abs/1704.02147 • P. Berenbrink, T. Friedetzky, F. Mallmann-Trenn, S. Meshkinfamfard, and C. Wastell: Threshold Load Balancing with Weighted Tasks. In 2015 IEEE International Parallel and Distributed Processing Symposium, IPDPS'15, 2015, pages 550-558. url: http: //dx.adoi.org/10.1109/IPDPS.2015.73 • P. Berenbrink, F. Ergün, F. Mallmann-Trenn, and E. S. Azer: Palindrome Recognition In The Streaming Model. In 31st International Symposium on Theoretical Aspects of Computer Science STACS. Volume 25, 2014, pages 149-161. url: http://drops. dagstuhl.de/opus/volltexte/2014/4454 • V. Kanade, F. Mallmann-Trenn, and V. Verdugo: How large is your graph? In CoRR, volume abs/1702.03959, 2017. To appear in DISC'17. url: http://arxiv.org/abs/ 1702.03959 • D. Kaaser, F. Mallmann-Trenn, and E. Natale: On the Voting Time of the Deterministic Majority Process. In 41st International Symposium on Mathematical Foundations of Computer Science, MFCS'16, 2016, pages 55:1-55:15. url: http://dx.adoi. • F. Mallmann-Trenn, C. Mathieu, and V. Verdugo: Skyline Computation with Noisy Comparisons. Unpublished manuscript Chapter 2

  Move to bin with smallest load among sample 1 , Sample 2 , . . . , Sample d

Algorithm 3: Greedy[d] for arrival rate λ, d ∈ [1, 2] Spawn a ball w.p. λ < 0. if a spawned is spaaned then for all choice i ∈ [d] do sample i = Uniform(bin 1 , bin 2 , . . . , bin n )

  1 otherwise. Let t ≥ τ ≥ 1. Consider the Forest Fire Process with seed graph G 0 , conditioned on (u t ) = τ . Then, the subgraph of G t , consisting of G 0 and of all vertices on the path from u t to G 0 in A t , and of all edges out of those vertices; has the same distribution as the graph L τ , with seed graph L 0 = G 0 . In particular dist Gt (u t , G 0 ) in the Forest Fire Process conditioned on (u t ) = τ has the same distribution as dist Lτ (v τ , G 0 ) in the Line

	Lemma 4.10. Fire Process.

Graph results from the Line results -Proof of Theo- rem 4.1 and Theorem 4.3

	Before proving Lemma 4.4, let us see how it implies Theorem 4.1.
	Proof of Theorem 4.1. Let c be a large enough constant. Since dist(u t , G 0 ), once defined
	at time t, never changes, it suffices to show that E[ dist Gt (u t , G 0 ) ] = O(1). Thus, by law of
	total probability,

  t. α).

	Lemma 4.16. Fix F 2t and consider the arrival of v 2t+1 . With probability at least 1 -1/α
	the following holds: node v 2t+1 has outdegree at least δ, and among the nodes of N + (v 2t+1 )
	only v 2t has value φ(v 2t ), at most 6α nodes have value φ(v 2t ) -1, and all other nodes have
	value ≤ φ(v 2t ) -2.

  , consider the case deg + (w) ≥ δ and amb(w) ∈ S. Note that Z w = R w +

	Zw , where R w ∼ Bernoulli(p) and Zw ∼ Bin(|S| -1, p) (when deg + (w) ≥ δ, p =
	min{1, α/ deg + (w)} = α/ deg + (w), as long as δ ≥ α). Note that (Z w , R w ) have the
	exact same joint distribution as defined above, since effectively we are making the
	choice of whether or not amb(w) should be included in B independently of the other
	elements. Since α/ deg + (w) ≤ α/δ, it is clear that we can couple R w and R so that
	R w ≤ R . Thus, again it remains only to show a coupling such that Z w < Z .
	For all α ≥ 1, it follows that Bin(n, p) is stochastically dominated by Poisson(eα) when-
	ever p ≤ α/n and n ≥ eα (see e. g., [KM10]). When, n < eα, clearly Bin(n, p) is stochasti-
	cally dominated by 1 + eα . The additional 1 in the definition of Z takes care of the strict
	inequality made in the claim. This completes the proof.

  Due to the duality, one easily obtains the bound O(t meet log n) on the coalescence time.This bound appears implicitly in the work of Hassin and Peleg[START_REF] Hassin | Distributed Probabilistic Polling and Applications to Proportionate Agreement[END_REF]. In recent work, Cooper et al.[START_REF] Cooper | Coalescing Random Walks and Voting on Connected Graphs[END_REF] provide results that are better than O(t meet log n) for several in-

teresting graph classes, notably expanders and power-law graphs. They show that t coal = O((log

Table 7 . 1 :

 71 Summary of plurality consensus results.

	Arbitrary Graph of Opinions Number	Required Bias α O-notation	Time O-notation	Model	Space O-notation
	Shuf.

  + log n • log s and (w.h.p.) O log 2 n s•α + log 2 n . Here, s is the number of states and must be in the range s = O(n) and s = Ω(log n • log log n).

	O log n s•α
	Time in the async model
	use parallel time. All results, except for [DV12], hold w.p. 1 -o(1). [AGV15] also gives an expected time of
	o(log(n)/(sα) + log(n) • log(s)).

  t is the filtration up to time t. Finally, we define the time of coalescence as t coal (S 0 ) = min{t ≥ 0 | |S t | = 1}. Let t coal (S 0 ) be defined as expected consensus time when initially only the nodes of S 0 are occupied by random walks; we use the notation t coal when S 0 = V . Throughout this paper, the expression w.h.p. (with high probability) means with probability at least 1 -n -Ω(1) and the expression w.c.p. (with constant probability)

. The process satisfies the Markov property, i. e.,

P[ S t+1 | F t ] = P[ S t+1 | S t ],

(8.3)

where F

  2}. Whenever at least one walk arrives 3 on a node, then the following happens.

1. At least one of the walks is of type G 1 (a) At least one walk of type G 1 is alive i. the walk of G 1 with the smallest id is labeled as alive (even if it was labeled phantom before)

ii. all other walks of type G 1 (if there are any) are labeled as phantom iii. alive walks of type G 2 are labeled dead (if present). (b) All walks of type G 1 are phantom walks i. There is no walk of type G 2 A. No label is changed ii. There is at least one walk of type G 2 A. the walk of type G 1 with the smallest id is labeled as alive B. all other walks of type G 1 (if there are any) are labeled as phantom C. alive walks of type G 2 are labeled dead. 2. All walks are of type G 2 (a) the walk of G 2 with the smallest id is labeled as alive (b) all other walks are labeled as dead.

  .6) Furthermore, if we divide the interval [1, ] into 2 α consecutive sections of length t mix each, the probability for a collision in each of these section is identical and therefore the Consider now two independent random walks (X t ) t≥0 and (Y t ) t≥0 starting at arbitrary positions. By applying Lemma A.27 to both walks, with probability at least (1 -e -1 ) 2 both X 4t mix and Y 4t mix are drawn independently from the stationary distribution since 4t mix ≥ t sep . Therefore,

	union bound implies
	P intersect( Xt , Ỹt , ) ≤ 2 α • P intersect( Xt , Ỹt , t mix ) ,	(8.7)
	and hence combining equation (8.6) and (8.7) yields
	P intersect( Xt , Ỹt , t mix ) ≥	1 4 α	.

  be set of coalescing random walks at an arbitrary time step t. In the following we show the slightly stronger claim that the expected time to reduce the number of random walks by a constant factor is O(t meet ). Formally, we fix an arbitrary time step t 0 . With T := min{t ≥ t 0 : |S t | ≤ 99/100 • |S t 0 |, |S t 0 | ≥ 100} denoting the first time step the number of coalescing random walks reduces by a factor of 99/100, we will prove that E[ T ] = O(t meet ). Iterating the argument O(log |S 0 |) times implies that the expected time it takes to reduce to 100 random walks is O(t meet log |S 0 |). Note that the expected time to reduce from 100 random walks to 1 is bounded by O(t meet ). Hence, the claim t coal (S 0 ) = O(t meet log |S 0 |) follows.It remains to show that the expected number of time steps it takes to reduce the number of random walks by a factor of 99/100 is indeed O(t meet ).

  meet /t mix (Corollary 8.8). From there on we employ Claim 8.7 to reduce the number of walks to 1 in O(t meet ) time steps. Melding both phases together yields the bound of Theorem 8.1.We now define a process P imm (S 0 , k) with k < |S 0 |, which is a parameterized version of the process P imm defined in Section 8.3.2:

These probabilities are derived from the time-probability trade-off presented in Lemma 8.4. We then use Claim 8.7 to derive a bound on the number of time steps it takes to reduce the number of walks to 2α , where α = t • Let |S 0 | = k ; there are k random walks with ids 1, . . . , k and starting nodes v 1 , . . . , v k . Thus, S 0

  k} and G 2 = {k + 1, . . . , k }. Recall that, by definition of P imm , we have that if some random walks with ids in G 1 and some with ids in G 2 are present on the same node at the same time, only the ones with ids in G 1 survive. If all the random walks have ids in only in G 1 , then all of them survive. If all random walks have ids only in G 2 , then only the one with the smallest id survives.

  Conditioning on Z 1 ≥ 1 and since both random walks start from the stationary distribution, we have, by Observation A.25, that the first meeting happens in the first /2 time steps w.p. at least 1/2.Consider E[ Z 1 | Z 1 ≥ 1 ].Suppose the meeting occurred at node u ∈ V 1 . Let E 1 be the event that for u ∈ V 1 we have T hit (u, z) ≥ n/c for both walks, where c > 0 is a large enough constant. By Lemma 8.11.(ii), we have that P[ E 1 ] ≥ (1/2) 2 = 1/4 due to independence of the walks. For any t < n/c let p t u,• be the distribution of the random walk on G 1 starting on u after t time steps under the conditioning E 1 . Observe that v∈V 1

  1), byProposition A.29. Consider a random walk ( Xt ) t≥tsep restricted to G 2 that starts at vertex X tsep ∈ V 2 and let P denote the transition matrix. Furthermore, in order to couple the random walk Xt restricted to G 2 with a random walk in G, we will consider the random

	variable	tsep+n/c-1
	Z :=
		t=tsep	z∈N ( z)

1 Xt=z , 126 for c = 32. Thus, for any z ∈ N ( z),

  Consider now the straightforward coupling between a random walk (X t ) t≥tsep in G that starts at vertex Xtsep ∈ V 2 and the random walk ( Xt ) t≥tsep restricted to G 2 that starts at the same vertex. Whenever the random walk Xt is at a vertex in V 2 \ {N ( z)}, then the random walk X t makes the same transition. If the random walk Xt is at vertex z

			n/c.
	Let γ := 8(1 + 1/e)	√	n/c. Then, by Markov's inequality
			P Z ≥ γ ≤ 1/8.

  so that, by a union bound over all opinions and using that T = min{T i | i ≤ k}, we obtain P[ T < n/(γ ) ] ≤ 1/n. Intuitively, we would like to show that, conditioning on c i ≤ , the expected number of nodes joining opinion i is dominated by a binomial distribution with parameters n and p = ( /n) 2 . The main obstacle to this is that naïvely applying Chernoff bounds for every time step yields a weak bound, since with constant probability at each round at least one color increases its support by a constant number of nodes. Instead, we consider a new process P in which the number P (t) of nodes supporting color i at time t majorizes c i (t) as long as P (t) ≤ ; we will then show that, after a certain time w.h.p. P (t) is still smaller than implying that P indeed majorizes the original process. Using the fact that in P we can simply apply Chernoff bounds over several rounds, we can finally get c i ≤ P (t) ≤ w.h.p..

	Formally, process P is defined as follows. P (0) := and P (t + 1) = P (t) + j≤n X j , (t)
	where X	(t)

  • Let T v (t) denote the random variable for the real time, the number of ticks of node v after the first t • n time steps. That is, T v (t) denotes the number of times v was scheduled during the first t • n ticks.• Let T v (t) denote the random variable for the working time, the current instruction counter of node v after the first t • n time steps. Note that T v (t) can differ from T v (t) since the working time is adjusted with the goal of synchronization in Algorithm 10.

  Definition 11.2. We say a node is ∆-close to a reference point τ w.r.t. the real time T v or the working time T v , if |T v (τ ) -τ | ≤ ∆ or |T v (τ ) -τ | ≤ ∆, respectively. If we say a node is ∆-close without specifying a reference point, we mean that it is ∆-close to the expected number of ticks.

	Proposition 11.3. Let S be set of synchronized nodes that are (∆/2)-close w.r.t. the
	working time throughout the entire process. With high probability,

  Consider the number of times v is selected to tick in the interval of time steps [ϑ Let τ be any reference point in [τ v , (s + 1) • T ]. Since the working time increases afterwards whenever v is selected to tick, we have

v , τ v • n]. By a standard balls-into-bins argument [RS98],

we can argue that with high probability

|T (τ v ) -T (ϑ v /n)| ≤ ∆/16 .

(11.6)

11.6 Analysis of the 2-Choices sub-phase: Proof of Proposi- tion 11.4

  Proof of Proposition 11.4. Recall that S is the set of nodes v that are (∆/2)-close w.r.t.

T (v) throughout the entire process. By Proposition 11.3, |S| ≥ n -E, with E ≤ n • exp(-8 log n/ log log n)) = n 1-8/log log n . When a node of S samples two nodes, then by

  ). As Observation 11.6 Part 1 suggests, the working times of the nodes are sandwiched by the real time of the nodes and thus if we bound the real times of nodes, we get bounds on the working times as well. Therefore, we conclude that all nodes have completed Part 1 before any node starts the two choices process of Part 2 at reference point τ end1 . More precisely, all nodes are with high probability in [τ end0 , τ end1 ] before the first node passes τ end1 .Since the real times are sandwiched, we get from Chernoff bounds that when the first node reaches τ end2 , all nodes are with high probability in [τ end1 , τ end2 ] w.r.t. the real time.We assume that nodes which are in [τ end0 , τ end4 ] respond, when queried, with the color they last set, possibly in Part 1 of the algorithm.The remainder of this section is structured as follows. In Lemma 11.11 we give a lower bound on the size of C 1 throughout the execution of Algorithm 9. This lower bound on C 1 allows us to show that the number of nodes having any other color C j = C 1 decreases quickly in expectation. This expected drop lets us apply a standard drift theorem, Theorem A.12, to obtain a bound on the required time until C 1 prevails and all other colors vanish. Finally, this will allow us to show that with high probability all nodes have set their color to C 1 by the end of the the asynchronous algorithm at τ end4 .For the next two lemmas, we will use the following notation. Consider an arbitrary but fixed time step t. Let a t and b t be the number of nodes of color 1 and B at time step t, Assume that all nodes have a working time in [τ end0 , τ end4 ] during the time steps in[n • τ end0 , n • τ end3 ].Assume furthermore that at time step t = n • τ end0 we have a t ≥ 19n/20. Then for any later tick t in [n • τ end0 , n • τ end4 ] we have a t ≥ 4n/5, with high probability. Proof. To show the claim, we split Part 2 of the asynchronous algorithm into phases of n/100 consecutive time steps each. Based on these phases, we show the claim by an induction over every phase i ∈ [100 • τ end0 , 100 • τ end4 ]. By induction, we will show that we have with high probability at time step t Furthermore, by assumption of the lemma we have a t ≥ 19n/20 at time step t = n • τ end0

	respectively.
	Lemma 11.11.

From Observation 11.6 we obtain that all nodes have finished Part 1 at time step T after at most T ≤ 3/2 • κ • • log n = τ end0 ticks w.r.t. the working time. Furthermore, also due to Observation 11.6, we have that no node has yet reached τ end1 w.r.t. the working time at

time step T. i = i • 100 • n a t i ≥ 17n/20 -i • √ n • log n .

Let now i be an arbitrary but fixed phase. We distinguish two cases. Case 1: a t i ≥ 18n/20. In this case the induction step holds trivially, since in the worstcase a t i+1 ≥ a t i -(t i+1 -t i ) = 18n/20 -n/100 > 17n/20. Case 2: a t i ≤ 18n/20. Observe that we have, by induction hypothesis, that for every t ∈ [t i , t i+1 ] that a t ≥ 17n/20 -i • √ n • log n -n/100 ≥ 16.5n/20.

  Consider the asynchronous model. Let G = K n be the complete graph with n nodes. Let k = O(exp(log n/ log log n)) be the number of opinions. Let ε bias > 0 be a constant. Assume c 1 ≥ (1 + ε bias )•c i for all i ≥ 2, then the asynchronous plurality consensus process defined in Section 11.2 on G converges within time Θ(log n) to the majority opinion C 1 , with high probability.

	Theorem 11.1.
		• (16.5n/20) 2 /n 2 = 272.25/20 3
	-1 with probability at most 19/20 • (3.5n/20) 2 /n 2 = 232.75/20 3
	0	otherwise.

  end3 . Clearly, no node can change to any other color afterwards and, by Chernoff bounds, after additional Θ(log n) periods all nodes will have completed the execution of Algorithm 11. Thus the total run time is Θ(log n).

	• log log n phases
	ĉ1 ≥ 19n/20 .	(11.14)
	As mentioned before (see Observation 11.6), using Chernoff bounds, we can show that
	with high probability:
	1. All nodes have a working time in [τ end0 , τ end1 ) at reference point τ end0 . This implies
	that no node starts with two choices phase before all nodes finished Part 1 (Algo-
	rithm 10).
	2. All nodes have a working time in [τ end0 , τ end4 ] during the reference points in
	[τ end0 , τ end3 ]. This together with above statement and (11.14) are the assumptions of
	Lemma 11.11.
	3. All nodes have a working time in [τ end1 , τ end4 ] during the reference points in
	[τ end2 , τ end3 ]. This is the assumption required by Lemma 11.12.

Thus, by Lemma 11.11 and Lemma 11.12, with high probability all nodes agree on C 1 at τ

  2 . Moreover, let t mix be such that any time interval [t, t + t mix ] is ε-smoothing 3 (cf. Section 12.3). Given knowledge of the maximum number degree d max and the mixing time t mix of the underlying communication pattern 4 , Shuffle lets all nodes agree on the plurality opinion in O(T • t mix ) rounds (w.h.p.), using O log n/(α 2 T ) • log k + log(T • t mix ) memory bits per node. This implies, for example, that plurality consensus on expanders in the sequential model is achieved in O(T • n log n) time steps and with O(log n • log k/T + log(T n)) memory bits (assuming a constant initial bias). For arbitrary graphs, arbitrary bias, and many natural communication patterns (e.g., communicating with all neighbors in every round or communicating via random matchings), the time for plurality consensus is closely related to the spectral gap of the underlying communication network (cf. Corollary 12.2).

  t • t mix . With µ := (1/n + 1/n 5 ) • |B| • T , we have P[ X ≥ (1 + δ) • µ ] ≤ e δ 2 µ/3 .Proof. Let v j,t denote the location of token j at time (t -1) • t mix . For all t ∈ {1, . . . , T } and ∈ N define the random indicator variable Y j,t to be 1 if and only if the random walk starting at v j,t is at node u after t mix time steps. By Lemma 12.6 we have for each B ⊆ B and t ∈ {1, . . . , T } that Let us define p := 1/n + 1/n 5 . By the definition of t mix , we have for all j ∈ B and t ≤ T that P Y j,t = 1|Y 1,1 , Y 2,1 , . . . , Y |B|,1 , Y 1,2 , . . . , Y j-1,t ≤ p.

				
	P		X j,t = 1	 ≤	P[ Y j,t = 1 ].	(12.8)
			i∈B			j∈B
	Hence for all t ≤ T and ∈ N we have P	j∈B X j,t ≥		≤ P	j∈B Y j,t ≥	and
									
	P[ X ≥ ] = P		X j,t ≥	 ≤ P			Y j,t ≥	 .	(12.9)
			1≤t≤T j∈B				1≤t≤T j∈B
					e δ (1 + δ) 1+δ	µ	≤ e δ 2 µ/3 ,	(12.11)
	which yields the desired statement.

(12.10) Combining our observations with Theorem A.5 (see above), we get P[ X ≥ ] ≤ Bin(T • |B|, p). Recall that µ = T • |B| • p. Thus, by applying standard Chernoff bounds we get

P[ X ≥ (1 + δ)µ ] ≤

Together, these lemmas generalize a result given in

[START_REF] Sauerwald | Tight Bounds for Randomized Load Balancing on Arbitrary Network Topologies[END_REF]

to a setting with considerably more dependencies. Equipped with this Chernoff bound, we prove concentration of the counter values. Lemma 12.8 (Counter Separation). Let c ≥ 12. For every time t ≥ c • T • t mix there exist values > ⊥ such that 1. For all nodes w with o w ≥ 2 we have (w.h.p.) c w ≤ ⊥ . 2. For all nodes v with o v = 1 we have (w.h.p.) c v ≥ .

  The Doob sequence of a function defines a martingale, i. e.,E[ Y i | X i-1 ] = Y i-1 , i ∈ {0, 1, . . . , n}.Proposition A.18 (Azuma-Hoeffding Inequality). Let Y 0 , Y 1 , . . . be a martingale with respect to the sequence X 0 , X 1 , . . .. Suppose also that fora i ≤ Y i -Y i-1 ≤ b i for i ≥ 1. Then, P[ |Y n -Y 0 | + t ] ≤ exp -2t 2 i∈[n] (b i -a i ) 2 .

A.16. The Doob sequence of a function f w.r.t. a sequence of random variables X 1 , . . . , X n is defined by

Y i := E[ f |X i ], i ∈ {0, 1, . . . , n}.

Proposition A.17.

  al. [FMM95, Theorem 2.2.4]). A time-homogeneous irreducible aperiodic Markov chain ζ with a countable state space Ω is positive recurrent if and only if there exists a positive function φ(x), x ∈ Ω, a number η > 0, a positive integer-valued function β(x), x ∈ Ω, and a finite set C ⊆ Ω such that the following inequalities hold:1. E[φ(ζ(t + β(x))) -φ(x)|ζ(t) = x] ≤ -ηβ(x), x ∈ C

2. E[φ(ζ(t + β(x)))|ζ(t) = x] < ∞, x ∈ C

A.4 Markov Chains: Random Walks

Proposition A.23

[START_REF] Levin | Markov chains and mixing times[END_REF]

). Consider two irreducible Markov chains (X t ) t≥0 , (Y t ) t≥0 with transition matrix P , X 0 = x, and

Y 0 = y. Let {(X t , Y t )} be a coupling satisfying that if X s = Y s , then X t = Y t for t ≥ s. Let T = min{t ≥ 0 | X t = Y t }. Then p t (x, •)p t (y, •) TV ≤ P[ T > t ].

Proposition A.24. Biased Random Walk [Fel68, Chapter XIV.2]] Let p ∈ (0, 1/2) and b, s ∈ N. Consider a discrete time Markov chain (Z t ) t≥0 with state space Ω = [0, b] where

  A.26 (cf.[START_REF] Levin | Markov chains and mixing times[END_REF] Chapter 12]). Let P be the transition matrix of a reversible Markov chain with state space Ω. Then the following statements hold: (i) If P is irreducible, then for any two states x, y ∈ Ω, where λ := max{λ 2 , |λ n |} and λ 1 ≥ λ 2 ≥ • • • ≥ λ n are the n real eigenvalues of the matrix P .(ii) If the Markov chain is a non-lazy random walk on a bipartite regular graph with two partitions V 1 and V 2 , then for any pair of states x, y in the same partition

	p t x,y ≤ π(y) +	π(y) π(x)	• λ t ,
	p t x,y ≤ • 1 + (-1) p t 2 n x,y ≤ 2 n • 1 + (-1)

t-1 + 2(max{λ 2 , |λ n-1 |}) t . Similarly, if x and y are in opposite partitions, t + 2(max{λ 2 , |λ n-1 |}) t . (iii) If the Markov chain is lazy, then for any state x ∈ Ω, p t x,x is non-increasing in t. In particular, p t x,x ≥ π(u). Proof. The first statement can be found in [LPW06, Equation 12.11]. For the second statement, recall the spectral representation [LPW06, Lemma 12.2 (iii)] p t x,y = π(y) + π(y) • n k=2 u k (x) • u k (y) • λ t k , (A.7)

The meeting time of two nodes is the expected time for random walks starting from these two nodes to meet and the meeting time of a graph is the maximum over all pairs of nodes of the graph.

Bien que tous ces processus ne soient pas intrinsèquement stochastiques, les méthodes de prédiction basées sur des modèles déterministes nécessitent bien souvent des calculs infaisables en pratique car trop complexes. Une façon de remédier à cette complexité est les traiter de façon stochastique. Des prédictions précises par des modèles déterministes sont souvent infaisables face à la complexité des calculs. Le traitement de ces processus comme inhérents aléatoires permet de bonnes estimations, comme par example la prédiction du nombre de fois où un dé montre 6.

Nous supposons qu'il n'y a pas de ordre des opinions.

Le temps de réunion de deux noeuds est le temps prévue pour les marches aléatoires commençant sur les noeuds respectifs et le de temps réunion d'un graphe est le maximum sur toutes les paires de noeuds du graphe.

Note that dist(vt, G0), once defined at time t, never changes

An event E occurs with high probability (w.h.p.) if P[ E ] = 1 -n -Ω(1) .

Talwar and Wieder [TW14] use the same potential function to analyze variants of the sequential d-Choice process without deletions. Our analysis turns out a bit more involved, since we have to consider deletions and argue over whole batches (of random size) instead of single balls.

This is easily verified by hand. Alternatively, [TW14, Appendix A] gives i≥3n/4 pi ≥ 1 4 + ε and the statement follows by noting that p 3n/4 = o(1).

Note that -1 f •n ≥ -1/e, so that W-1(-1 f •n ) is well defined.

It might look tempting to use Γ together with Hajek's theorem to bound the maximum system load. However, this would require (exponentially) sharper bounds on Φ. Furthermore, it might be tempting to use the stability of Greedy[1] to prove stability of Greedy[2], however, as discussed earlier, it is not clear to achieve this, as it seems challenging to couple or majorize the processes.

Equivalently, the node may adopt the color of a fixed sample (the first, or second, or third).

The meaning of "dual" in this field differs from usage in other fields. As we elaborate later, the duality states that either of the processes can be viewed as the other the exception that time runs "backwards". This is made rigorously in Proposition 9.11.

Throughout this thesis, we use random walk and particle interchangeably, assuming that every random walk has an identifier.

Throughout this thesis, the expression with high probability means a probability of at least 1 -n -Ω(1) .

To allow for an easier comparison with the synchronous model, we will normalize the run time of all sequential algorithms and continuous processes throughout this thesis by dividing their run time by n[START_REF] Alistarh | Fast and Exact Majority in Population Protocols[END_REF].

Byzantine agreement requires that the system does not converge to a color that was initially not supported by at least one non-corrupted node.

The bias is α := (n1 -n2)/n, n1 and n2 being the support of the most and second most common opinions.

This classification is neither unique nor injective but merely an attempt to make the overview more accessible.

We say an event happens with high probability (w.h.p.) if its probability is at least 1 -1/n c for c ∈ N.

The number of steps divided by n. A typical measure for population protocols, based on the intuition that each node communicates roughly once in n steps.

We state their bound in terms of our α = (n1 -n2)/n; their definition of α differs slightly.

Throughout this thesis, we use random walk and particle interchangeably, assuming that every random walk has an identifier.

Note that the star also exhibits t hit tmeet. However, the star is not almost-regular.

Throughout, by arrive we take into account that walks may arrive at a node from the same node through laziness.

One might be tempted to pair random walks in groups of two and run them for 2tmeet time steps so that, by Markov inequality, they meet with probability at least 1/2. Repeating this iteratively would yield the claim. To formalize such an argument one would need to disallow coalescence between different pairs of random walk which differs from the stochastic process we reduce to in Section 8.3.2.

Simple calculations[START_REF] Becchetti | Trevisan: Simple Dynamics for Plurality Consensus[END_REF][START_REF] Elsässer | Efficient k-Party Voting with Two Choices[END_REF] show that, for both processes, if xi is the current fraction of nodes with color i then the expected fraction of nodes with color i after one round isx 2 i + (1 -x 2 j ) • xi.

 3 The bias is the difference between the number of nodes supporting the most and second most common color.

Note that for a large number of colors, a node executing 3-Majority behaves with high probability like a node performing Voter. Thus, it is relatively tight to bound 3-Majority by Voter in this parameter regime.

Recall that, with a slight abuse of notation we also write P (c) for the (random) configuration obtained by performing one step of a process P on configuration c.

Observe that Strassen's Theorem gives us that P[ X Y ] = 1. That is, X Y holds almost surely. However, since C is finite, this actually means that c = X Y = c holds (surely).

Alternatively, one checks this manually: The direction x y ⇒ x ≤C y is trivial by the definition of Schur-convexity. For x ≤C y ⇒ x y consider the n + 1 Schur-convex functions z → j∈[i] z ↓ for i ∈ [n] and z → -z 1 .

Technically, one would have to define a new random variable which is 0 whenever the number of random walks reduces to 1. We illustrate this technicality shortly, for case k > 100 below.

For simplicity and without loss of generality; our protocols run in polynomial time in all considered models.

The protocol works for any integral choice of γ (this fixes the trade-off parameter T ).

Intuitively, this means that the communication pattern has good load balancing properties during any time window of length tmix. This coincides with the worst-case mixing time of a lazy random walk on active edges.

For static graphs, dmax is the maximal degree which can be easily computed in a distributed way, see for example[START_REF] Boyd | Randomized gossip algorithms[END_REF]. For tmix, good bounds are known for many static graphs [AF02, Chapter

5].

Note that there are several simple, distributed protocols to obtain such matchings[START_REF] Ghosh | Dynamic Load Balancing by Random Matchings[END_REF][START_REF] Boyd | Randomized gossip algorithms[END_REF].

This state is then maintained for a poly(n) many rounds. For an exponential number of time steps, one would require larger counters to guarantee correctness.

Shuffle needs not to know α, it works for any choice of γ; such a choice merely fixes the trade-off parameter T .

The state space includes all vectors with non-increasing entries over N n .

Acknowledgments

First and foremost, I'd like to thank my advisors Petra Berenbrink and Claire Mathieu for their indispensable support and advice extending far beyond research. I thank Colin Cooper and Pierre Fraigniaud for having accepted to review this manuscript and I'm grateful to Bartek Błaszczyszyn, Andrei Bulatov, and Philippe Jacquet for being part of the jury. Special thanks go to Vincent Cohen-Addad and Varun Kanade, who I consider to be my academic big brothers and valuable friends. I also thank Robert Elsässer for his support and all the intuition he shared about randomized processes. Thanks go to Ralf Klasing and Adrian Kosowski for inviting me, and to George Giakkoupis and Anne-Marie Kermarrec for the research visit me during my Masters. I thank Thomas Sauerwald for inviting me to Cambridge and for sharing so much of his knowledge about random walks.

Part II

Probabilistic Analysis of Consensus Dynamics and Protocols

examples). However, our bounds are related to well-studied load balancing bounds and mixing times of random walks (which we believe are easier to get a handle on than their δ(Q S , α)).

Further Consensus Protocols.

The authors of [START_REF] Feinerman | Breathe before speaking: efficient information dissemination despite noisy, limited and anonymous communication[END_REF][START_REF] Fraigniaud | Noisy Rumor Spreading and Plurality Consensus[END_REF] give efficient protocols for plurality consensus in the setting where transmission are subject the noise.

Phase 2: From up to n 1/4 log 1/8 n to 1 color (consensus). Once we reached a configuration with n 1/4 log 1/8 n colors, we can apply [BCN+16, Theorem 3.1] (see Theorem 9.9), a previous analysis of 3-Majority. It works only for initial configurations with at most k ≤ n 1/3-colors ( > 0 arbitrarily small). In that case, [BCN+16, Theorem 3.1] yields a runtime of O (k 2 log 1/2 n + k log n) • (k + log n) . Since the first phase leaves us with O n 1/4 colors, this immediately implies that the second phase takes O n 3/4 log 7/8 n rounds.

This section proceeds by proving the runtime of Phase 1 in two steps: dominating the runtime of 3-Majority by that of Voter (Section 9.4.1) and proving the corresponding runtime for Voter (Section 9.4.2). In the end, we can combine these results together with [BCN+16, Theorem 3.1] to prove Theorem 9.8.

Proof of Theorem 9.8. Consider any initial configuration c ∈ C. By applying Lemma 9.12 for k = n 1/4 , we get that Voter reduces the number of remaining colors w.h.p. from initially at most n to n 1/4 in O n 3/4 log 7/8 n rounds. By Proposition 9.10, the time it takes 3-Majority to reach some fixed number of remaining colors is dominated by the time it takes

Voter to reach the same number of remaining colors. In particular, we get that 3-Majority also reduces the number of remaining colors w.h.p. to n 1/4 in O n 3/4 log 7/8 n rounds. That is, the first phase takes O n 3/4 log 7/8 n rounds.

For the second phase, we apply [BCN+16, Theorem 3.1] (see Theorem 9.9) for k = n 1/4 = o n 1/3 . This immediately yields that the second phase takes O n 3/4 log 7/8 n rounds, finishing the proof.

Theorem 9.9 ([BCN+16, Theorem 3.1]). Let ε > 0 be an arbitrarily small constant.

Starting from any initial configuration with k ≤ n 1/3-ε colors, 3-Majority reaches consensus w.h.p. in

Analysis of Phase 1: 3-Majority vs. Voter

We prove the following proposition.

Proposition 9.10. Consider the clique. We have tht Voter (V) and 3-Majority (3M ) started from the same initial configuration c ∈ C. There is a coupling such that after any round, the number of remaining colors in Voter is not smaller than those in 3-Majority. In particular, the time Voter needs to reach consensus stochastically dominates the time needed by 3-Majority to reach consensus, i.e.,

The reasons are twofold. First, the variance of the 3-Majority process can be orders of magnitude larger and second, the expected increase in the difference between the largest and second largest color in the 3-Majority process is only of order of the variance. As for the variance, consider an initial setting where all colors are of sublinear size and colors 1 and 2 are larger than all other colors, such that c 1 = o(n) and c 2 = c 1 -c n log n > c j and

for all 3 ≤ j ≤ k with k = n ε for constants ε and c. Observe that the expected numbers of color switches differ significantly. In the 2-Choices process it is very unlikely for a node to pick the same color twice and the probability of switching is o(1). In contrast to this, the probability of switching in the 3-Majority process is 1 -o(1).

More illustratively, consider the number of switches to color 2. By Lemma 2.1 of [START_REF] Becchetti | Trevisan: Simple Dynamics for Plurality Consensus[END_REF], the probability that a node switches to color 2 in the 3-Majority process is p ∈ [c 2 /n, 2c 2 /n] and the variance becomes n • p • (1 -p) ≥ c 2 /2. However, in the 2-Choices process, the probability of switching to color 2 is q = c 2 2 /n 2 and the variance is thus at most n • q • (1 -q) ≤ n • q = c 2 2 /n, which is considerably smaller than c 2 /2. This high variance paired with the small expected increase in the difference between color 1 and color 2 easily becomes fatal. Again, by Lemma 2.1 of [START_REF] Becchetti | Trevisan: Simple Dynamics for Plurality Consensus[END_REF], one can verify that

2 )/n. Now we have P[ c 1 ≤ E[ c 1 ] ] = Ω(1) and, using the large variance, we obtain from Theorem A.6 that

Thus the distance between color 1 and 2 decreases with constant probability, that is,

In comparison to this, we have seen in Section 10.3 that in the given setting the distance between color 1 and color 2 in the 2-Choices process increases with high probability.

Chapter 11

Rapid Asynchronous Consensus [EFK+16]

We consider the following plurality consensus process on the complete graph of size n. We consider the asynchronous setting (cf. e.g. [START_REF] Mosk-Aoyama | Fast Distributed Algorithms for Computing Seperable Functions[END_REF]). In the asynchronous model, we assume that each node has a random clock which ticks according to a Poisson distribution, once per unit of time in expectation [START_REF] Mosk-Aoyama | Fast Distributed Algorithms for Computing Seperable Functions[END_REF] (note that this model is equivalent to the continuous-time model of [START_REF] Aldous | Reversible Markov Chains and Random Walks on Graphs[END_REF], page 26 (ii), according to which random walks move along the vertices of a graph). Again, upon activation a node updates its opinion according to a sample of its neighborhood.

Regardless of the underlying model of synchronicity, if eventually all nodes agree on one opinion, we say this opinion wins, and the process converges. Typically, one would demand from such a voting procedure to run accurately, that is, the opinion with the largest initial support should win with decent probability (1 -o(1)), and to be efficient, that is, the voting process should converge within as few communication steps as possible.

Our goal is to design a simple algorithm which reaches quickly plurality consensus.

Model

In the following section, we will introduce formally the model which we consider in the remainder of this chapter. We give a formal definition of the consensus process in the synchronous and the asynchronous model followed by an overview of our results in Section 10.1.

We consider the following plurality consensus process on the complete graph of size n.

Synchronous Model

In the synchronous model we assume that the protocol operates in discrete rounds. In each round, the nodes may simultaneously sample other nodes uniformly at random and then simultaneously change their opinion as a function of the observed samples. An example To overcome this restriction, we adopt the following weaker notion of synchronicity. At any time we only require a (1 -o(1)) fraction of the nodes to be almost synchronous. This relaxes full synchronicity in three ways: First, nodes are only almost synchronous, meaning that for any two nodes their working times may differ by up to ∆ = Θ(log n/ log log n).

Secondly, we allow o(n) nodes to be poorly synchronized. Finally, we require this to hold only with high probability.

The above notion does not require the nodes to synchronize actively per se, since their number of ticks is to some extent concentrated even without active synchronization. However, it turns out that without synchronizing perpetually, the number of poorly synchronized nodes in each phase will become larger than the initial bias towards the plurality opinion c 1 -c 2 and could therefore influence the consensus significantly. We thus actively synchronize nodes at the end of each phase to decrease the fraction of poorly synchronized nodes such that their number is in o(c 1 -c 2 ), resulting in a negligible influence of those nodes.

Once several technical challenges are resolved, the resulting weak synchronicity allows us to reuse the high-level structure of the synchronous algorithm(OneExtraBit). As in the synchronous case, the asynchronous protocol consists of one 2-Choices sub-phase and one Bit-Propagation sub-phase, the latter of which propagates the choices of the 2-Choices phase to all nodes in the network. In addition to these sub-phases we have a third sub-phase in which we synchronize nodes.

After executing the first two sub-phases, the relative difference between C 1 and any opinion C j = A increases quadratically and thus we only require O(log log n) such phases.

Each of the sub-phases has a length of O(log n/ log log n), amounting to a total run-time of O(log n). While superficially the asynchronous version looks very similar to the synchronous protocol (OneExtraBit), the analysis differs greatly from the synchronous case, in both approach and technical execution.

The Asynchronous Protocol

Our asynchronous protocol consists of two parts, Part 1 defined in Algorithm 10 later in this section and Part 2 defined in Algorithm 11 in Section 11.8. In these formal definitions, we specify the operations that each node performs when selected to tick. The goal of the first part is to increase the number of nodes of color 1 to at least c 1 ≥ (1 -ε Part1 )•n for some small constant ε Part1 . Once the execution of the first part has finished, the nodes execute a simple 2-Choices algorithm in an asynchronous manner. We will show that after the second part, C 1 wins with high probability. Our main contribution is the analysis of the first part.

For the sake of completeness, we formally analyzethe second part in Section 11.8.

In contrast to the formal definitions, it is more convenient and instructive to represent the algorithm executed by each node in a graphical way. This graphical representation for a single phase of the first part is shown in Figure 11.1. In this graphical representation, the instructions are drawn on a line from left to right, starting with the first instruction at the left endpoint.

As in the synchronous case, the asynchronous algorithm operates in multiple phases.

Each of these phases is split into three sub-phases. Each sub-phase consists of multiple blocks of length ∆ each. During these sub-phases, according to Algorithm 10, there are multiple blocks of instructions where nodes for a long time literally do nothing. These Applying the Azuma-Hoeffding bound to Y (τ ) = v∈V Y v (τ ) gives us

Observe that c 3 • n • log 3 n ≤ n • exp -Θ log n/ log 2 log n . We finally conclude that, with high probability, at least n • 1 -exp -Θ log n/ log 2 log n nodes are synchronous up to a deviation of at most ∆ = Θ(log n/ log log n) ticks from the expected number of ticks at the given reference point τ .

We now turn to the second part of the statement. Recall that ∆ = c ∆ log n/ log log n and c ∆ is a large enough constant. Observe that, by definition of our algorithm, T = 10∆.

The proof of the second part follows in a similar way as before. We define an analogous event E v (τ 1 ) for node v to hold, then the number of ticks it receives t/n ± ∆/16 out of t ticks. We have

Observe that this is bound is much stronger than (11.4). Similarly, as before,

In the following we show that the median taken will be concentrated around the expected real time.

Lemma 11.8. The median real-time of a uniform sample of Ω(log 2 log n) nodes is (∆/16)close with high probability at any reference point τ ≤ κ • • log n.

Proof. In this proof we assume for simplicity that the c log 2 log n sampled nodes are taken in one single step. First, we show that the median of the sampled times is close to the average of all (real) times, with high probability. The median real-time of the sample is no (∆/16)-close if at least half of the sample contained nodes which were not (∆/16)-close. By Lemma 11.7, we know that for some constant c > 0 there are with high probability at most

nodes u which are not (∆/16)-close w.r.t. T u during any point of the execution of the algorithm.

We now define Y t as Y t = k≤t X k and show that Y t is a sub-martingale.

which yields that the induction steps hold with high probability. This completes the proof.

In the following (Lemma 11.12) we make use of multiplicative drift theorem (Theorem A.12 in Appendice A) which will allow us to derive a bound on the number of required periods until all nodes agree on one opinion.

Lemma 11.12. Assume that all nodes have a working time in

Furthermore assume that a t ≥ 4n/5 for any time step

We have

Let δ = 12/(25n) and define Φ(x t ) = b t . Note that Φ(x max ) ≤ n and at any time step t

Let T be the first point in time where all nodes agree on color 1, that is, T = min{t ≥ 0 : Φ(x t ) = 0}. We derive from Theorem A.12 with parameters δ and k = 5 log

, where we used the Taylor series approximation for log(1 -δ). Since τ end3 -τ end2 ≥ 20/δ ln n, the claim follows.

Putting Everything Together: Proof of Theorem 11.1

We use Proposition 11.5 (which builds on Proposition 11.4) and Lemma 11.12 to show Theorem 11.1, which is restated as follows.

labeled with their own opinion. By broadcasting these estimates, all nodes determine the plurality opinion.

On the other hand protocol Balance analysis' is straightforward: Even if the bias of the plurality color just one (the second largest color has support which is smaller by one), by construction, there will be ≥ γ = poly(n) many more tokens of the plurality token.

The analysis of [START_REF] Sauerwald | Tight Bounds for Randomized Load Balancing on Arbitrary Network Topologies[END_REF] then shows that sufficiently many time steps, the load discrepancy between any pair of nodes is so small that these additional γ tokens ensure that each node has strictly more tokens of the plurality color than of any other color.

Communication Model and Notation

We consider an undirected graph G = (V, E) of n ∈ N nodes and let 1 -λ 2 denote the eigenvalue (or spectral) gap of G. Each node u is assigned an opinion o u ∈ { 1, 2, . . . , k }.

For i ∈ { 1, 2, . . . , k }, we use n i ∈ N to denote the number of nodes which have initially opinion i. Without loss of generality (w.l.o.g), we assume

is the opinion that is initially supported by the largest subset of nodes. We also say that 1 is the plurality opinion. The value

denotes the initial bias towards the plurality opinion. In the plurality consensus problem, the goal is to design simple, distributed protocols that let all nodes agree on the plurality opinion. Time is measured in discrete rounds, such that the (randomized) running time of our protocols is the number of rounds it takes until all nodes are aware of the plurality opinion. Further to the running time we also consider the total number of memory bits per node that are required by our protocols. All our statements and proofs assume n to be sufficiently large.

Communication Model. In any given round, two nodes u and v can communicate if and only if the edge between u and v is active. We use M t to denote the symmetric

u] = 0 otherwise. We assume (w.l.o.g) M t [u, u] = 1 (allowing nodes to "communicate" with themselves). Typically, the sequence M = (M t ) t∈N of communication matrices (the communication pattern) is either randomized or adversarial, and our statements merely require that M satisfies certain smoothing properties (see below). For the ease of presentation, we restrict ourselves to polynomial number of time steps and consider only communication patterns M = (M t ) t≥0 where N = N (n) is an arbitrarily large polynomial. Let us briefly mention some natural and common communication models covered by such patterns:

• Diffusion Model: In every round t, all edges of the graph are activated.

• Random matching model: In every round t, the active edges are given by a random matching. We require that random matchings from different rounds are mu-tually independent 5 . Results for the random matching model dependent on p min :=

• Balancing Circuit Model: There are d perfect matchings M 0 , M 1 , . . . , M d-1 given.

They are used in a round-robin fashion, such that for t ≥ d we have M t = M t mod d .

• Sequential Model: In each round t an edge { u, v } ∈ E is activated uniformly random.

Notation. We use x to denote the -norm of vector x, where the ∞-norm is the vector's maximum absolute entry. In general, bold font indicates vectors and matrices, and x(i) refers to the i-th component of x. The discrepancy of x is defined as disc(x) := max i x(i)-min i x(i). For i ∈ N, we define [i] := { 1, 2, . . . , i } as the set of the first i integers.

We use log x to denote the binary logarithm of x ∈ R >0 . We write a | b if a divides b. For any node u ∈ V , we use deg u to denote u's degree in G and deg u (t

its active degree at time t (i.e., its degree when restricted to active edges). Similarly, N (u) and N t (u) refer to u's (active) neighborhood respectively. Moreover, d max := max t,u deg u (t) is the maximum active degree of any node. We assume knowledge of d max . On static graphs it can be computed efficiently in a distributed manner [START_REF] Boyd | Randomized gossip algorithms[END_REF] and it is given by many dynamic graph models (e.g., 1 for the sequential model, d for balancing circuits). Let c(t) be the configuration (state) of the all nodes at time t. We say an event happens with high probability (w.h.p.) if its probability is at least 1 -1/n c for c ∈ N. Random Walks. The running time of our protocols is closely related to the running time ("smoothing time") of diffusion load balancing algorithms, which in turn is a function of the mixing time of a random walk on G (see also [START_REF] Avin | How to Explore a Fast-Changing World (Cover Time of a Simple Random Walk on Evolving Graphs)[END_REF][START_REF] Sauerwald | Tight Bounds for Randomized Load Balancing on Arbitrary Network Topologies[END_REF]). More exactly, we consider a random walk on G that is restricted to the active edges in each time step. As indicated in Section 12.1, this random walk should converge towards the uniform distribution over the nodes of G. This leads to the following definition of the random walk's transition matrices P t based on the communication matrices M t :

Obviously, P t is doubly stochastic for all t ∈ N. Moreover, note that the random walk is trivial in any matching-based model, while we get P t [u, v] = 1 2d for every edge { u, v } ∈ E in the diffusion model on a d-regular graph. We are now ready to define the required mixing property.

Protocol Description

We continue to explain the Shuffle protocol given in Listing 12. Our protocol consists of three parts that are executed in each time step: the shuffle part, the broadcast part, and the update part.

Every node u is initialized with γ ∈ N tokens labeled with u's opinion o u . Our protocol sends 2 d max tokens chosen uniformly at random (without replacement) over each edge

Here, γ ≥ 2 d max is a parameter depending on T and α to be fixed during the analysis 7 .

Shuffle maintains the invariant that, at any time, all nodes have exactly γ tokens.

In addition to storing the tokens, each node maintains a set of auxiliary variables. The variable c u is increased during the update part of the protocol and counts tokens labeled o u . The variable pair (dom u , e u ) is a temporary guess of the plurality opinion and its frequency. During the broadcast part of the protocol, nodes broadcast these pairs, replacing their own pair whenever they observe a pair with higher frequency. Finally, the variable plu u represents the opinion currently believed to be the plurality opinion. The shuffle and broadcast parts of the protocol are executed in each time step, while the update part is executed only every t mix (n -5 ) time steps

Waiting t mix (n -5 ) time steps for each update gives the broadcast enough time to inform all nodes and ensures that the tokens of each opinion are well distributed. The latter implies that, if we consider a node u with opinion o u = i at time T • t mix (n -5 ), the value c u is a good estimate of T • γn i /n (which is maximized for the plurality opinion). When we reset the broadcast (Line 11), the subsequent t mix (n -5 ) broadcast steps ensure that all nodes get to know the pair (o u , c u ) for which c u is maximal. Thus, if we can ensure that c u is a good enough approximation of T • γn i /n, all nodes get to know the plurality.

Algorithm 12: Protocol Shuffle as executed by node u at time t. At time zero, each node u creates γ tokens labeled ou and sets cu := 0 and (domu, eu) := (ou, cu).

Algorithm Shuffle

// shuffle sub-phase

Analysis of Shuffle

Fix a communication pattern (M t ) t≥0 and an arbitrary parameter T ∈ N. Remember that t mix := t mix (n -5 ) ensure that a random walk starting at any node and for any time step t and run for t mix (n -5 ) time steps will be n -5 -close to the stationary distribution. We set the number of tokens stored in each node to γ := c • log n α 2 T , where c is a suitable constant. The analysis of Shuffle is largely based on Lemma 12.8, which states that, after O T • t mix (n -5 ) time steps, the counter values c u can be used to reliably separate the plurality opinion from any other opinion. The main technical difficulty is the dependency between the tokens' movements, rendering standard Chernoff-bounds inapplicable. Instead, we show that certain random variables satisfy the negative regression condition (Lemma 12.5), which allows us to majorize the token distribution by a random walk (Lemma 12.6) and to derive the Chernoff type bound in Lemma 12.7. This Chernoff type bound can be used to show that all counter values are concentrated which is the main pillar of the proof of Theorem 12.1.

Majorizing Shuffle by Random Walks

While our Shuffle protocol assumes that 2 d max divides γ, here we assume the slightly weaker requirement that P t [u, v] • γ ∈ N for any u, v ∈ V and t ∈ N. Let us first introduce transition probabilities). We use w P j (t) to denote the position of token j after t steps of a process P. We assume (w.l.o.g.) w S j (0) = w W j (0) for all j. While there are strong correlations between the tokens' movements in S (e.g., not all tokens can move to the same neighbor), Lemma 12.6 shows that these correlations are negative.

Lemma 12.6 (Majorizing RWs). Consider a time t ≥ 0, a token j, and node v. Let B ⊆ [γn] and D ⊆ V be arbitrary subsets of tokens and nodes, respectively. The following holds:

Proof. The first statement follows immediately from the definition of our process. For the second statement, note that the equality on the right-hand side holds trivially, since the tokens perform independent random walks in W . for all t ∈ { 0, 1, . . . , t -1 }. Combining these inequalities yields the desired result.

Fix an arbitrary t ∈ { 0, 1, . . . , t -1 } and note that S W (t ) and S W (t + 1) behave identical up to and including step t . Hence, we can fix an arbitrary configuration (i.e., the location of each token) c(t ) = c immediately before time step t +1. Remember that u j ∈ V denotes the location of j in configuration c. The auxiliary functions h j : [deg

describe the probability that a random walk starting at time t + 1 from u j 's i-th neighbor ends up in a node from D. Formally,

We can assume (w.l.o.g.) that all h j are non-decreasing (by reordering the neighborhood of u j ). Now, by Lemma 12.5 the variables (X j ) j∈B satisfy the negative regression condition.

Thus, we can apply Lemma 12.4 (a well-known characterization of negative regression) to the functions h j . Using another simple auxiliary result ((12.6) and (12.7)) we can relate the (conditioned) probabilities of the events E t and E t +1 to the expectations over the different h j (X j ). We assume the following. Fix a time t ∈ { 0, 1, . . . , t -1 } and consider an arbitrary Proof. Let v and w be two nodes with o v = 1 and o w ≥ 2. We define for all i ∈ {2, . . . , k}

For i ∈ {2, . . . , k} define

We set ⊥ := ⊥ (2). We first show that

, where c γ is a suitable constant.

The figure depicts the random variables in the proof of Lemma 12.8 for the case of two nodes where node 1 supports opinion 1 and node 2 supports opinion 2. The total expected number of tokens counter by a node supporting the most prominent color 1 (2, respectively) is approximatively µ (µ2, respectively) up to an second order error accounting for the difference between the distribution of a random walk after tmix time steps and the stationary distribution. We will show that w.h.p. the number of tokens counted will be in [µ -λ , ∞] and in [0, µ2 + λ(2)] for node 2. These intervals are separated by the carefully chosen quantities ⊥ (2) and where ⊥ (2) < . This establishes the separation of the counters.

We have

where the last inequality holds for c γ ≥ 5. Now, let all γn tokens be labeled from 1 to γn.

We proceed by showing the lemma's statements:

• For the first statement, consider a node w with o w ≥ 2 and set

where the last line follows by Lemma 12.7 applied to c w = c•T j=1 i∈B X i,j•t mix and setting B to the set of all tokens with label o w . Hence, the claim follows for c large enough after taking the union bound over all n -n 1 ≤ n nodes w with o w ≥ 2.

• For the lemma's second statement, consider a node v with o v = 1 and set

Define the random indicator variable Y i,t to be 1 if and only if token i is on node v at time t and if i'

, where B is the set of all tokens with opinion different than 1. Note that

where the first inequality follows by Lemma 12.7 applied to Y . Hence, the claim follows for c large enough after taking the union bound over all n 1 ≤ n nodes v with o u ≥ 2.

We now give the proof of our main theorem.

Theorem 12.1. Fix an arbitrary time t ∈ [c • T • t mix , N ] with t mix dividing t, where c is the constant from the statement of Lemma 12.8. From Lemma 12.8 we have that (w.h.p.) the node u with the highest counter c u has o u = 1 (ties are broken arbitrarily). In the following we condition on o u = 1. We claim that at time t = t + t mix all nodes v ∈ V have plu v = 1. This is because the counters during the "broadcast part" (Lines 3 to 7)

propagate the highest counter received after time t. The time τ until all nodes v ∈ V have plu v = 1 is bounded by the t mix (n -5 ) by definition: In order for [t, t ] to be 1/n 5 -smoothing, the random walk starting at u at time t is with probability at least 1/n -1/n 5 on node v

Analysis of Balance.

Consider initial load vectors 0 with 0 ∞ ≤ n 5 . Let τ := τ (g, M ) be the first time step when Vertex-Based Balancer under the (fixed) communication pattern M = (M t ) t≥0 is able to balance any such vector 0 up to a g-discrepancy.

With this, we show:

Theorem 12.9. Let α = n 1 -n 2 n ∈ [ 1 /n, 1] denote the initial bias. Consider a fixed communication pattern M = (M t ) t≥0 and let γ ∈ [3 • g α , n 5 ] be an arbitrary integer. Protocol Balance ensures that all nodes know the plurality opinion after τ (g, M ) rounds and requires k • log(γ) memory bits per node.

The definition of τ (g, M ) implies 1,t (u) ≥ ¯ 1 -g and i,t (u) ≤ ¯ i + g for all nodes u and i ≥ 2. Consequently, we get

(12.13) Thus, every node u has the correct plurality guess at time t.

The memory usage of Balance depends on the number of opinions (k) and on the number of tokens generated on every node (γ). The algorithm is very efficient for small values of k but it becomes rather impractical if k is large. Note that if one chooses γ sufficiently large, it is easy to adjust the algorithm such that every node knows the frequency of all opinions in the network. The next corollary gives a few concrete examples for common communication patterns on general graphs.

Corollary 12.10. Let G be an arbitrary d-regular graph. Balance ensures that all nodes agree on the plurality opinion with probability 1 -e -(log(n)) c for some constant c Chapter 13

Future Work -Distributed Consensus Processes

The results for consensus dynamics in Part II can be generalized in many ways-the most obvious being to general graphs. Another interesting open problem would be to analyze 3-Majority and faster protocols in presence of an adversary that is allowed to change the opinion of say √ n nodes per round-starting from all configurations. In the following we elaborate on these research directions.

Fault Tolerance. As mentioned in Chapter 9, previous studies [BCN+14b, BCN+16, CER14, EFK+16] show that 2-Choices and 3-Majority are consensus protocols that can tolerate dynamic, worst-case adversarial faults. More specifically, the protocols work even in the presence of an adversary that can, in every round, corrupt the state of a bounded set of nodes. The goal in this setting is to achieve a stable regime in which "almost-all" nodes support the same valid color (i.e. a color initially supported by at least one noncorrupted node). The size of the corrupted set is one of the studied quality parameters and depends on the number k of colors and/or on the bias in the starting configuration. For instance, in [START_REF] Becchetti | Stabilizing Consensus with Many Opinions[END_REF] it is proven that, for k = o n 1/3 , 3-Majority tolerates a corrupted

A natural important open issue is to investigate whether our framework for AC-processes can be used to make statements about fault-tolerance properties in this (or in similar) adversarial models. We moderately lean toward thinking that our analysis is sufficiently general and "robust" to be suitably adapted in order to cope with this adversarial scenario over a wider range of k and bias w.r.t. the relative previous analyses. Finally, we ask whether there exists any poly-log consensus time protocols agreeing on the plurality color in presence of an adversary? Towards a Hierarchy. Consider the process functions of the general h-Majority process for arbitrary h ∈ N. Intuitively, h-Majority should be (stochastically) slower than (h + 1)-Majority. We strongly believe this result holds. However, naïvely applying our machinery to prove this does not work and needs to be amended. Our conjecture that such a "hierarchy" for h-M ajority for different h ∈ N holds is backed by the proof of Proposition 9.10 (which shows this for h ∈ { 1, 2, 3 }, since the Voter process is actually equivalent to 1-Majority and 2-Majority).

Conjecture 13.1. For h ∈ N, we can couple h-Majority and (h + 1)-Majority such that the latter never has more remaining colors than the former. In particular, (h + 1)-Majority is stochastically faster than h-Majority.

However, as we have shown in Section 9.5 via a counterexample, it turns out that Lemma 9.5 is not strong enough to derive Conjecture 13.1. In fact, our failed attempts in adapting our approach may suggest that similar counterexamples exist for any majorization attempt that uses a total order on vectors.

General Graphs. To obtain result for 3-Majority on general graphs, it seems natural to try generalizing the coupling of Proposition 9.10 to general graphs. However, there are graphs for which this is not true: versions of the stochastic block model seem to require a super-exponential number of rounds whereas Voter requires at most O(n 3 ) rounds [START_REF] Kanade | On coalescence time in graphs-When is coalescing as fast as meeting?[END_REF].

Nevertheless, 3-Majority seems to perform at least as good as Voter on graphs like the cycle, the grid and the star. It is natural to ask whether there are any dynamics reaching consensus faster than Voter, 3-Majority, and 2-Choices on any graph.

The Right Model. The most flagrant and yet most difficult and long-term based challenge is to find the right model. We are far from a general understanding of the domain partially due to the vast choice of parameters: Synchronous vs asynchronous time steps, knowledge of the graph size n, execution starts for all nodes at the same time, presence of adversaries, trade-off consensus guarantees and space etc. Finding the correct model and assumptions could benefit greatly from industrial applications as well as applications in computational biology modelling the behaviour and communication of species. Alternatively, it would be very interesting to obtain a general understand of how results in one setting relate to another-is it possible to translate results for synchronous dynamics to asynchronous dynamics and vice versa?

Appendix A Probabilistic Preliminaries

220

A.1 Elementary Bounds

Proposition A.1 (Markov's Inequality). Let X be a random variable that assumes only nonnegative values. Then, for all a > 0

Proposition A.2 (Union Bound). For any countable set of events E 1 , E 2 , . . . , E n we have

A.2 Concentration Inequalities and Drift Analysis

A.2.1 Binomial Distribution

Proposition A.3 (Chernoff bound [MU05, Theorem 4.4 and 4.5]). Let X = i X i be the sum of 0/1 independent random variables. Then, 1. for any δ > 0,

.

for any

Here, Bin(n, p) denotes the binomial distribution with parameters n and p.

We adapt Theorem 2 and equation (6.7) from [START_REF] Feller | An Introduction to Probability Theory and Its Applications[END_REF] as follows.

Theorem A.6 (DeMoivre-Laplace limit theorem [START_REF] Feller | An Introduction to Probability Theory and Its Applications[END_REF]). Let X be a random variable with binomial distribution X ∼ Bin(N, p). It holds for any

A.2.2 Negative Regression

Negative regression is defined as follows.

Definition A.7 (Neg. Regression [DR98, Def. 21]). A vector (X 1 , X 2 , . . . , X n ) of random variables is said -to satisfy the negative regression condition if

is non-increasing in each x r for any disjoint L , R ⊆ [n] and for any non-decreasing function f .

Proposition A.8 ([DR98, Lemma 26]). Let (X 1 , X 2 , . . . , X n ) satisfy the negative regression condition and consider an arbitrary index set I ⊆ [n] as well as any family of non-decreasing functions f i (i ∈ { I }). Then, we have

A.2.3 Poisson Distribution

Lemma A.9. Let X n ∼ Bin(n, α/n) for all n. Let Y ∼ P oi(eα). Let α ≤ 1/e. For all n and k ≥ 1 we have P r(X n ≥ k) ≤ P r(Y ≥ k).

Proof. Fix an arbitrary n and k ≥ 1. P r(

Proof. Due to the independence of the random variables, we have N i-1 • Poisson(eα) = Poisson(eαN i-1 ). In the following we fix N i-1 = n for n < b i-1 . We derive, by using the

This yields the claim.

A.2.5 Drift Theorem

The following is a version of the multiplicative drift theorem which we will use in Lemma 11.12 to derive a bound on the number of required periods until all nodes agree on one opinion.

Theorem A.12 ([LS16, Theorem 5]). Let (X t ) t∈N 0 be a Markov chain with state space S ⊆ {0} ∪ [1, ∞) and with X 0 = n. Let T be the random variable that denotes the earliest point in time t ≥ 0 such that X t = 0. Assume that there is δ > 0 such that for all

Then

Theorem A.13 (Variable Drift Theorem [LW14, Corollary 1.(i)]). Let (X t ) t≥0 , be a stochastic process over some state space S ⊆ {0} ∪ [x min , x max ], where x min ≥ 0. Let h : [x min , x max ] → R + be a differentiable function. Then the following statements hold for the first hitting time T := min{t

dy.

Lemma A.14. Let (X t ) t≥0 be a stochastic process satisfying

Proof. By the iterative law of expectation, we have

Furthermore, by Markov's inequality, for any λ ≥ 1

A.3 Markov Chains

A.3.1 Couplings

The following lemma will be helpful to define a coupling between distributions that are close to the stationary distribution and the exact stationary distribution. (A very similar lemma has been derived in [ES11, Lemma 2.8])

Lemma A.19. Let ε ∈ (0, 1] be an arbitrary value. Let Z 1 and Z 2 be two probability distributions over {1, . . . , n} so that P[

Then, there is a coupling ( Z1 , Z2 ) of (Z 1 , Z 2 ) and an event E with P[ E ] ≥ so that

Proof. Let U ∈ [0, 1] be a uniform random variable. We next define our coupling ( Z1 , Z2 ) of Z 1 and Z 2 that will depend on the outcome of U . First, if U ∈ [0, ε), then we set

For the case where U ∈ (ε, 1), it is clear that the definition of U can be extended in a way so that Z1 has the same distribution as Z 1 , and Z2 has the same distribution as Z 2 . Furthermore, notice that if U ∈ [0, ε) happens, then Z1 has the same distribution as Z 2 , and Z1 = Z2 . Observing that P[ U ∈ [0, ε) ] = ε completes the proof.

A.3.2 Strassen's Theorem

Proposition A.20 ([MOA11, Proposition 11.E.11], [START_REF] Rinott | An Inequality for Multivariate Normal Probabilities with Application to a Design Problem[END_REF]). For N ∈ N and a probability vector Θ ∈ [0, 1] l , consider a random vector X having the multinomial distribution Mult(N, Θ). Let φ : As in [LPW06, Proof of Theorem 12.3], using the orthonormality of the eigenvectors, we have

and the second statement follows if u and v are in the same partition. The case where u and v are in different partitions follows analogously.

For the third statement, first note that by [LPW06, Exercise 12.3], all eigenvalues of the transition matrix M are non-negative. Since all eigenvalues are non-negative, we conclude from (A.7) that p t x,x is non-increasing in t as needed. Due to this, we get that p t x,x ≥ π(u). This can be verified by means of a simple proof by contradiction. The existence of a t with p t

x,x < π(u) and the fact that p t x,x is non-increasing in t implies that the expected number of visits to u during a sufficiently long period of length τ starting from the stationary distribution would be strictly less than τ • π(u). A contradiction.

A.4.2 Separation Time (t sep )

The following lemma is an immediate consequence of Lemma A.19. Lemma A.27. Consider a random walk (X t ) t≥0 , starting from an arbitrary but fixed vertex x 0 . Then with probability at least 1 -1/e, we can couple X 4t mix with the stationary distribution.

Proof. Consider the random walk (X t ) t≥0 after step s := t sep ≤ 4t mix . By definition of t sep , p s u,v ≥ (1 -1/e)π(v). Applying Lemma A.19, where Z 1 is the distribution given by p t u,v and Z 2 is the stationary distribution shows that with probability at least 1 -1/e, X s has the same distribution as π. If this is the case, then the same holds for X 4t mix as well.

Lemma A.28. For any graph G,

and similarly, t coal ≤ 4 • (4t mix + 2t π coal ).

Proof. We begin by proving the lower bound on t meet . First, consider two independent random walks (X t ) t≥0 and (Y t ) t≥0 that are run for t = e • t meet time steps. Then, we have

where the first inequality is due to the coupling method [LPW06, Theorem 5.3] and the second inequality follows by Markov's inequality. The above inequality implies t mix ≤ e • t meet . Furthermore, t π meet ≤ t meet holds by definition, and the lower bound follows. For the upper bound, we divide the two random walks into consecutive epochs of length := 4t mix + 2t π meet . For the statement it suffices to prove that in each such epoch, regardless of the start vertices of the two random walks, a meeting occurs with probability at least (1 -1/e) 2 • 1/2.

Consider the first random walk (X t ) t≥0 starting from an arbitrary vertex after s := 4t mix steps. By Lemma A.27, we obtain that with probability at least 1 -1/e, the distribution of X s is equal to that of a stationary random walk. Similarly, we obtain that with probability and similarly,

A.4.4 Mixing time bounds via Hitting time

The following result of Peres and Sousi is useful to establish a bound on the mixing time.

Theorem A.31 ([PS15]

). For any β < 1/2, let t hit (β) = max u,A:π(A)≥β t hit (u, A). Then there exist positive constants c β and c β such that

A.5 Random Processes

A.5.1 Galton-Watson Trees

The analysis of the Forest Fire Process uses reductions to Galton-Watson branching processes.

Definition A.32 (Galton-Watson Tree). A Galton-Watson process is a stochastic process

{X n } which evolves according to the recurrence formula X 0 = 1 and X n+1 = Xn j=1 ξ

(n) j , where {ξ (n) j : n, j ∈ N} is a set of i.i.d. natural number-valued random variables.

The interpretation is as follows: the process builds a random tree. X n can be thought of as the number of descendants of the root in the n th generation, and ξ (n) j can be thought of as the number of children (in generation n+1) of the j th of these (n th generation) descendants. The recurrence relation states that the number of descendants in the (n + 1) st generation is the sum, over all n th generation descendants, of the number of children of that descendant. For more information, see [START_REF] Lyons | Probability on Trees and Networks[END_REF].

A.5.3 Póly-Urn

The Póly urns [START_REF] Johnson | Urn Models and Their Application: An Approach to Modern Discrete Probability Theory[END_REF]. In this model, we are given an urn containing marbles of two colors, black and white. In every step, one marble is drawn uniformly at random from the urn. Its color is observed, the marble is returned to the urn and one more marble of the same color is added. For any color, the ratio of marbles with that given color over the total number of marbles is a martingale. Formally, the Póly urn process is defined as follows.

Definition A.35 (Pólya Urn Process). Let Pólya(α 1 , α 2 ) with α 1 , α 2 ∈ Z + 0 be the following urn process. At the beginning there are α 1 black marbles and α 2 white marbles in the urn. The process runs in multiple steps where α 1 (i) and α 2 (i) denote the number of black and white marbles in the urn, respectively, for every time step i. In every time step i, a black marble is added with probability α 1 (i)/(α 1 (i) + α 2 (i)), and with remaining probability α 2 (i)/(α 1 (i) + α 2 (i)) a white marble is added. Proposition A.36. Consider a Pólya urn starting with a fraction X 1 > 0 of white balls. Then, the sequence X 1 , X 2 , . . . forms a martingale.

The statement and the proposition are folklore.

Proof. Let i denote the number of balls at time step i. Let E i be the event that the ball drawn at step i is white. We have Typical examples in the real world include the dynamics of weather and temperature, of traffic, the way we meet our friends, etc. We take the rich tool set from probability theory for the analysis of Markov Chains and employ it to study a wide range of such distributed processes: Forest Fire Model (social networks), Balls-into-Bins with Deleting Bins, and fundamental consensus dynamics and protocols such as the Voter Model, 2-Choices, and 3-Majority.

Résumé