Keywords: parallel programs, concurrent data structures, synchronization, performance, granularity control, algorithms Resumé programmes parallèles, structures de données simultanées, synchronisation, performance, contrôle de la granularité, algorithmes

To use the computational power of modern computing machines, we have to deal with concurrent programs. Writing efficient concurrent programs is notoriously difficult, primarily due to the need of harnessing synchronization costs. In this thesis, we focus on synchronization costs in parallel programs and concurrent data structures.

First, we present a novel granularity control technique for parallel programs designed for the dynamic multithreading environment. Then in the context of concurrent data structures, we consider the notion of concurrency-optimality and propose the first implementation of a concurrency-optimal binary search tree that, intuitively, accepts a concurrent schedule if and only if the schedule is correct. Also, we propose parallel combining, a technique that enables efficient implementations of concurrent data structures from their parallel batched counterparts. We validate the proposed techniques via experimental evaluations showing superior or comparable performance with respect to state-of-the-art algorithms.

From a more formal perspective, we consider the phenomenon of helping in concurrent data structures. Intuitively, helping is observed when the order of some operation in a linearization is fixed by a step of another process. We show that no wait-free linearizable implementation of stack using read, write, compare&swap and fetch&add primitives can be help-free, correcting a mistake in an earlier proof by Censor-Hillel et al. Finally, we propose a simple way to analytically predict the throughput of data structures based on coarse-grained locking.

Introduction

In 1965, Intel co-founder Gordon E. Moore made a prediction [START_REF]Cramming more components onto integrated circuits[END_REF], widely known as Moore's law, that the number of transistors per integrated circuit would increase exponentially leading to an exponential increase of a chip performance. However, around 2004, such increase in performance became economically inefficient due to hardware limitations [START_REF] Sutter | The free lunch is over: A fundamental turn toward concurrency in software[END_REF]. To resolve this, hardware manufacturers had to find another way to improve the performance. They decided to combine multiple computational units (cores) on the same processor, named multicore processor. Further, we avoid the term core and use the term process instead: a core is a physical object while a process is an abstraction that represents a sequential code running on a core.

Concurrent Programs

The invention of multicore processors, known as the "multicore revolution" [START_REF] Charles E Leiserson | How to survive the multicore software revolution (or at least survive the hype)[END_REF], changed the whole paradigm of computing. To benefit from the new design of processors, sequential algorithms had to be replaced by concurrent ones.

In this thesis, we consider two types of concurrent programs: parallel programs and concurrent data structures. The distinction between these types of programs is based on the relation between inputs and outputs.

Similarly to sequential programs, a parallel program, given an input, should produce an output that satisfies a specification. A specification maps each possible input to a set of allowed outputs. The main advantage of parallel programs in comparison to sequential programs is that they can use multiple processes to improve performance (Figure 1.1).

Consider a sorting algorithm as an example. Its specification is: given an input array a the algorithm should output an array of elements of a in the ascending order. The sequential mergesort algorithm sorts an array of size m in O(m log m) time, while its parallel version [START_REF] Cole | Parallel merge sort[END_REF] only needs O( m n • log m) time where n is the number of processes. In concurrent data structures, e.g., concurrent stacks and concurrent binary search trees, inputs and outputs are distributed across the concurrent processes (Figure 1.2). The input of a process consists of operations called on the data structure by that process. The process performs each operation and appends its result to the output. Of course, these results should satisfy some correctness property. Typically, the correctness properties of concurrent data structures are split into two categories: safety or liveness.

A safety property asserts that "nothing bad ever happens". The most reknown safety property is linearizability [START_REF] Maurice | Linearizability: A correctness condition for concurrent objects[END_REF] -the invocations and responses of high-level operations observed in the execution should constitute a correct sequential history.

A liveness property asserts that "something good eventually happens". The most common liveness properties are:

• wait-freedom -every process eventually completes its operation;

• lock-freedom -at least one process eventually completes its operation;

• starvation-freedom -every process eventually completes its operation if each process takes an infinite number of steps;

• deadlock-freedom -at least one process eventually completes its operation if each process takes an infinite number of steps. Note that starvation-freedom and deadlock-freedom allow the implementation to use locks, while wait-freedom and lock-freedom do not [START_REF] Herlihy | On the nature of progress[END_REF].

Synchronization

To ensure correctness of parallel programs and concurrent data structures, we need synchronization, i.e., coordination between the processes. Informally, synchronization is used to handle conflicts on the shared data, e.g., resolving data races, and on shared resources, e.g., memory allocation and deallocation.

Let us have a look at how synchronization is instantiated in parallel programs. Parallel programs are typically written for two environments: for static multithreading and for dynamic multithreading [START_REF] Thomas H Cormen | Introduction to algorithms[END_REF].

In static multithreading, each process is provided with its own program. These programs are written as a composition of supersteps. During a superstep, a process performs independent local computations. After the superstep is completed, the processes synchronize to accumulate the results of their independent computations.

In dynamic multithreading, the program is written using fork-join mechanism (or a similar one, for example, # pragma omp parallel in OpenMP [START_REF]Openmp application interface[END_REF]). Fork-join takes two functions, named branches, as arguments and executes them in parallel until their completion. One fork-join call incurs the synchronization overhead at least on [START_REF] Frigo | The implementation of the cilk-5 multithreaded language[END_REF]:

1. Allocating and deallocating a thread: the shared memory should be able to process concurrent requests.

2. Scheduling of a thread. The scheduler should accept concurrent requests to execute threads and should control migration of threads. The migration happens when a thread changes the process-owner: this typically involves significant overhead spent on transfering the necessary data.

3. Joining of the two threads. A process that completes a thread decrements a concurrent counter of the corresponding fork-join call. If the counter is zero the process proceeds with the code after the fork-join, otherwise, it flags the scheduler that it is idle.

An important property of most parallel programs (written either for static or dynamic multithreading) is that they are designed in a way that there are no data races except for the races described above. In contrast, data races arising in concurrent data structures, typically, are way more diverse because each process has its own input and invocations of operations on different processes are not synchronized. As a consequence, synchronization is individually crafted from scratch for each concurrent data structure. For example, lock-based data structures get rid of data races using locks that enable an exclusive access to data. In wait-free and lock-free data structures, data races are allowed and they are resolved using different techniques, for example, helping where one process may perform some work on behalf of other processes. This complexity of synchronization makes concurrent data structures more challenging to design than parallel programs.

Overview of the Results

Synchronization, when it is not implemented properly, can induce an overhead overwhelming the benefits of parallelism. Ideally, the programmer should use just the right amount of synchronization to ensure correctness. In this thesis, we discuss how to improve performance in different settings by reducing the synchronization overhead.

At first, we provide a new practical way to solve the granularity control problem [START_REF]Intel. Intel threading building blocks[END_REF]. Suppose that we are provided with a parallel program written for dynamic multithreading using the fork-join mechanism. Informally, the granularity control problem consists in deciding, for each fork-join in a given program, whether to call it or to execute its branches sequentially in order to achieve the best performance. On one hand, if too many calls are executed, the total synchronization overhead (spent on allocation, scheduling, and joining of threads) can take a considerable fraction of the total execution time. On the other hand, if too many calls are executed sequentially, we lose potential benefits of multiprocessor machine.

Then we describe two techniques to build concurrent data structures with potentially low synchronization overhead:

• concurrency-optimality, a technique that constructs a concurrent data structure from a sequential implementation with the theoretically "optimal" synchronization;

• parallel combining, a technique that uses explicit synchronization to gather all concurrent accesses to a concurrent data structure and then calls a parallel algorithm that requires a small amount of synchronization, thus, reducing the total synchronization overhead.

Further, we consider synchronization in wait-free and lock-free data structures that typically appears in a form of helping. We identify a mistake in an earlier proof by Censor-Hillel et al. [START_REF] Censor-Hillel | Help![END_REF] of the fact that stack does not have a wait-free help-free implementation and, then, we reprove that fact.

Finally, we describe a simple method that theoretically measures the throughput of simple coarse-grained lock-based concurrent data structures exporting one method: a critical section of size C guarded by CLH lock [START_REF] Craig | Building fifo and priorityqueuing spin locks from atomic swap[END_REF] followed by a parallel section of size P .

Below, we describe the results in more details.

Granularity Control Problem

Over past decades many programming languages and systems for parallel computing have been developed, such as Cilk, OpenMP, Intel Threading Building Blocks, etc., to provide dynamic multithreading [START_REF] Thomas H Cormen | Introduction to algorithms[END_REF] (sometimes called implicit parallelism). Although these systems raise the level of abstraction and hide synchronization details from the user, the programmer still has to perform extensive optimization in order to improve performance. One such optimization is granularity control, which requires the programmer to determine when and how parallel tasks should be sequentialized.

We present an automatic algorithm to control granularity using an oracle: the oracle predicts the execution time of a parallel code and helps the algorithm decide whether the code should be sequentialized or not. We prove that this algorithm has the desired theoretical properties, i.e., the overhead on synchronization is small compared to the total execution time. Finally, we implement it in C++ as an extention to Cilk and evaluate its performance. The results show that our technique can eliminate hand tuning in many cases and closely match the performance of hand-tuned code.

To get an intuition behind granularity control, suppose that a dynamic multithreading is implemented via the fork-join primitive [START_REF] Thomas H Cormen | Introduction to algorithms[END_REF]. This primitive takes two functions, called program branches, as arguments and executes them in parallel until their completion. Despite its simplicity, fork-join is powerful enough to implement other common parallel expressions, including parallel loops and nested parallel computations [START_REF] Michael D Mccool | Structured parallel programming: patterns for efficient computation[END_REF].

One important aspect of dynamic multithreading is that it is possible to express the efficiency of a parallel program using so-called Work-Span Framework: work is the time spent to execute the program using an idealized machine with one process and span is the time spent to execute a program using an idealized machine with an infinite supply of processes.

Widely accepted as complexity metrics for parallel programs, work and span do not account for the real overhead hidden in a fork-join call -they consider this call to cost one unit of work (one instruction). However, in practice, this cost is non-negligible -one fork-join call induces at least an allocation of a new thread, its scheduling and, finally, the joining. Frequent calls of this primitive can take a considerable fraction of the total execution time. Thus, for each fork-join call, we have to decide whether to call it or to execute an alternative sequential algorithm, e.g., execute its branches sequentially. The resulting granularity control problem is about the synchronization overhead: we want to call fork-joins rarely enough so that the time spent on the overhead of these calls is a small fraction of the total execution time, and, on the other hand, we want to call them often enough to make processes busy.

The most popular solution to this problem is to cover each fork-join call with a special condition-guard [START_REF]Intel. Intel threading building blocks[END_REF]: if the work in both branches exceeds some grain-size constant, i.e., the approximation of the execution time is much bigger than the overhead in fork-join, then we are allowed to call it; otherwise, we execute an alternative sequential algorithm (e.g., execute branches sequentially).

However, such an approach is quite inconvenient. First, a program can have several fork-join calls that have distinct condition-guards with different grain-size constants. To improve performance, we have to tune all these constants. This tuning is tedious, especially when the constants cannot be tuned separately. For example, two constants of different condition-guards that correspond to two nested fork-join calls depend on each other. There is a further performance issue: everytime we want to check whether the chosen constants provide good performance the program has to be re-executed. Second, the tuned constants are non-portable, i.e., the constants tuned for one machine can provide bad performance on other machine.

In order to resolve these two issues, difficulty of tuning and non-portability, we present an automatic granularity control technique in a form of a sp-guard (a substitution of a condition-guard).

At first, the user should provide the asymptotic cost function of the code covered by each sp-guard. Notice that, as these functions depend only on the algorithm, they are portable.

Second, the user should provide a machine dependent variable κ. It represents the smallest amount of work that is big enough in comparison to an overhead in a fork-join call.

Our automatic granularity control algorithm works with each sp-guard g separately. At first, it determines the biggest value N g of the cost function for which the sequential execution time of the covered fork-join does not exceed κ. Then, when the fork-join is about to be called, the cost is compared with N g . If N g is smaller than the cost, then the execution time exceeds κ and we can call the fork-join, otherwise, an alternative sequential For our granularity control technique, we provide a theoretical analysis in the Work-Span Framework [START_REF] Jájá | An introduction to parallel algorithms[END_REF] together with a rigorous experimental analysis on a wide range of parallel programs taken from the PBBS suite [START_REF] Shun | Brief announcement: the problem based benchmark suite[END_REF] on three machines with different architectures. Experiments show that the performance of programs written with our automatic granularity control is comparable or even better (up to 32% faster and no more than 6% slower) than the performance of hand tuned code.

A Concurrency-Optimal Binary Search Tree

Implementing a concurrent data structure typically begins from a sequential implementation. When used as is in a concurrent environment, a sequential data structure may expose incorrect behaviour: lost updates, incorrect responses, etc. One way to ensure correctness is to protect portions of the sequential code operating on the shared data with synchronization primitives. By that, certain schedules, i.e., interleavings of the steps of the sequential implementation, are rejected, and the more synchronization we use the more schedules we reject.

Using this notion of schedules, we can compare the "concurrency" of algorithms: linearizable implementation A is "more concurrent" than linearizable implementation B if the set of schedules accepted by A is a strict superset of the set of schedules accepted by B. Thus, a linearizable implementation is concurrency-optimal [START_REF] Gramoli | In the search for optimal concurrency[END_REF] if it accepts all linearizable schedules. We expect this implementation to perform well since it uses exactly as little synchronization as necessary to ensure the linearizability of the data structure. Note that the notion of concurrency-optimality is theoretical and, thus, we expect concurrencyoptimal implementations to perform well on any machine.

As an example, consider the sequential partially-external binary search tree and its following schedule (Figure 1.3):

1. initially the tree consists of one node with value 1; 2. π 1 = insert(2) finds that 2 should be in the right child of the node with 1; 3. π 2 = insert(3) finds that 3 should be in the right child of the node with 1; 4. π 1 and π 2 create new nodes; 5. π 1 links its node as the right child of the node with 1; 6. π 2 overwrites the right child of the node with 1. This schedule is non-linearizable: operation insert(2) is lost and, thus, it should be rejected by any concurrency-optimal implementation.

We design a concurrency-optimal binary search tree. We implement it and compare against the state-of-the-art implementations [START_REF] Nathan G Bronson | A practical concurrent binary search tree[END_REF][START_REF] Crain | A contention-friendly binary search tree[END_REF][START_REF] Drachsler | Practical concurrent binary search trees via logical ordering[END_REF][START_REF] Ellen | Nonblocking binary search trees[END_REF]. To check the portability of concurrency-optimal technique we perform experiments on two machines with different architectures. The evaluation shows that our implementation is one of the best and, thus, the concurrency-optimal approach can be an adequate design principle for building efficient portable data structures.

Parallel Combining

We discussed two portable methods that try to provide optimal synchronization: an automatic granularity control for parallel programs and the concurrency-optimal design principle for concurrent data structure. The first method is automatic: it uses the fact that the synchronization in parallel programs appears in the special form and, by that, it almost does not require to change the original program. By contrast, concurrency-optimality requires the programmer to design the data structure from scratch, which might be nontrivial. It would be interesting to check whether it is possible to automatically build efficient concurrent data structures from parallel programs.

Efficient "concurrency-friendly" data structures (e.g., sets based on linked lists [START_REF] Harris | A pragmatic implementation of non-blocking linked-lists[END_REF][START_REF] Heller | A lazy concurrent list-based set algorithm[END_REF] or binary search trees [START_REF] Crain | A contention-friendly binary search tree[END_REF][START_REF] Drachsler | Practical concurrent binary search trees via logical ordering[END_REF]) are, typically, not subject to frequent data races and are designed using hand-crafted fine-grained locking.. In contrast, "concurrency-averse" data structures (e.g., stacks and queues) are subject to frequent sequential bottlenecks and solutions based on combining (e.g., [START_REF] Hendler | Flat combining and the synchronization-parallelism tradeoff[END_REF]), where requests are synchronized into batches and each batch is applied sequentially, perform surprisingly well compared to fine-grained ones [START_REF] Hendler | Flat combining and the synchronization-parallelism tradeoff[END_REF]. A general data structure typically combines features of "concurrency-friendliness" and "concurrency-averseness", and an immediate question is how to implement it in the most efficient way.

We propose parallel combining, a technique that can be used to build a concurrent data structure from a parallel batched counterpart [START_REF] Agrawal | Provably good scheduling for parallel programs that use data structures through implicit batching[END_REF]. A parallel batched data structure is implemented as a parallel program (apply function) that applies a set of data-structure operations to the underlying sequential data structure. As a correctness criteria, typically, after the execution of apply function we expect that the results of operations and the state of the underlying sequential data structure satisfy some sequential application of the operations. Parallel batched data structures appeared a long time ago, and, perhaps, the oldest one is 2-3 search tree by Paul et al. [START_REF] Paul | Parallel dictionaries on 2-3 trees[END_REF]. As other examples, we can consider parallel batched priority queues (e.g., [START_REF] Sanders | Randomized priority queues for fast parallel access[END_REF]) and parallel batched graphs (e.g., [START_REF] Umut A Acar | Brief announcement: Parallel dynamic tree contraction via self-adjusting computation[END_REF]).

In parallel combining, processes share a set of active requests using any combining algorithm [START_REF] Drachsler-Cohen | Lcd: Local combining on demand[END_REF]. One of the active processes becomes a combiner and forms a batch from the requests in set. Then, under the coordination of the combiner, owners of the collected requests, called clients, execute apply method on the parallel batched data structure.

This technique becomes handy when the overhead on combining algorithm is compensated by the advantages of using the parallel batched data structure. We show that combining and parallel batching pay off for data structures that offer some degree of parallelism, but still have sequential bottlenecks.

We discuss three applications of parallel combining. First, we design concurrent implementations optimized for read-dominated workloads given sequential data structures. Intuitively, updates are performed sequentially by the combiner and read-only operations are performed by the clients in parallel. In our performance analysis we considered a sequential dynamic graph data structure D [START_REF] Holm | Poly-logarithmic deterministic fully-dynamic algorithms for connectivity, minimum spanning tree, 2-edge, and biconnectivity[END_REF], that allows adding and removing edges (updates), and checking a connectivity between two vertices (read-only). We implement the resulting concurrent dynamic graph and compare it against three implementations: the first uses the global lock to access D; the second uses the read-write lock, i.e., a connectivity query takes the read lock and other queries take the write lock; and, finally, the last one uses flat combining [START_REF] Hendler | Flat combining and the synchronization-parallelism tradeoff[END_REF] to access D. The experimental analysis shows that our implementation has throughput up to six times higher on the loads consisting mostly of connectivity queries that can be parallelized.

Second, we apply parallel combining to the parallel batched binary search tree [START_REF] Guy E Blelloch | Just join for parallel ordered sets[END_REF]. The resulting concurrent algorithm performs somewhat worse than the state-of-the-art implementations [START_REF] Nathan G Bronson | A practical concurrent binary search tree[END_REF][START_REF] Brown | A general technique for non-blocking trees[END_REF][START_REF] Crain | A contention-friendly binary search tree[END_REF][START_REF] Drachsler | Practical concurrent binary search trees via logical ordering[END_REF][START_REF] Ellen | Nonblocking binary search trees[END_REF][START_REF] Shane | A non-blocking internal binary search tree[END_REF][START_REF] Natarajan | Fast concurrent lock-free binary search trees[END_REF][START_REF] Natarajan | Concurrent wait-free red black trees[END_REF], which is expected: a binary search tree is a concurrency-friendly data structure. But from the theoretical point of view, our concurrent tree provides a guarantee (inherited from the parallel batched algorithm) that it is always strictly balanced compared to the state-of-the-art concurrent implementations with relaxed balancing schemes [START_REF] Nathan G Bronson | A practical concurrent binary search tree[END_REF][START_REF] Brown | A general technique for non-blocking trees[END_REF][START_REF] Crain | A contention-friendly binary search tree[END_REF][START_REF] Drachsler | Practical concurrent binary search trees via logical ordering[END_REF].

Finally, we apply parallel combining to priority queue. As a side contribution, we propose a novel parallel batched priority queue. We compare the resulting concurrent algorithm with the state-of-the-art algorithms [START_REF] Herlihy | The art of multiprocessor programming[END_REF][START_REF] Lindén | A skiplist-based concurrent priority queue with minimal memory contention[END_REF][START_REF] Shavit | Skiplist-based concurrent priority queues[END_REF] and show that our implementation is among the best ones.

To summarize, our performance analysis shows that parallel combining can be used to construct efficient concurrent data structures.

Helping as a Synchronization Technique

Up to this point we only considered synchronization costs in lock-based (deadlock-free or starvation-free) concurrent data structures. In such data structures, data races are resolved by protecting a code operating on the shared data with the mutual exclusion mechanisms, i.e., locks.

Wait-freedom or lock-freedom guarantees cannot be achieved using locks and should use other types of synchronization. To ensure these progress guarantees when one process blocks indefinitely, it might be necessary for the remaining processes to complete the operation of the blocked process. This behaviour is a special type of synchronization known as "helping".

Censor-Hillel et al. [START_REF] Censor-Hillel | Help![END_REF] proposed a natural formalization of helping, based on the notion of linearization: a process p helps an operation of a process q in a given execution if a step of p determines that an operation of q linearizes before some other operation by a different process in any possible extention. It was claimed that in a system provided with read, write, compare&swap and fetch&add shared memory primitives helping is required for any wait-free linearizable implementation of an exact order data type. Informally, a sequential data type is exact order if for some operation sequence any change in the relative order of two operations affects the result of at least one other operation. As examples of exact order data types, Censor-Hillel et al. gave queue and stack.

However, we observe that the stack data type is not exact order. As we show, in any sequential execution on stack, we can reorder any two operations in such a way that no other operation will see the difference. Hence, the original proof for exact order types by Censor-Hillel et al. does not apply to stack.

In this thesis, we present a direct proof that stack does not have a wait-free helpfree implementation. Our proof is similar to the original one: we choose any help-free implementation and build an execution history such that one process makes an infinite number of steps, but never completes, proving that the implementation is not wait-free.

Performance Prediction of Coarse-Grained Programs

Reasoning about concurrency-optimality provides an analytical way to compare synchronization overheads of implementations using sets of schedules. However, this metric might be not always useful capturing synchronization costs. Implementations are, in this sense, incomparable when their sets of schedules are not related by containment, though their actual performance may differ a lot. Furthermore, the scope of implementations that enable the schedule-based metric is limited, e.g., lock-free algorithms are hard to model this way.

The common way to compare two implementations is to compare their throughput, i.e., the number of operations per unit of time. In the last technical chapter of this thesis, we discuss an analytical way to evaluate throughput of an implementation. Typically, throughput is measured through experiments, and soundness of the results is subject to experimental setting, the workload, the machine, etc. It is therefore interesting whether we can quantify throughput theoretically.

We describe a simple model that can be used to predict throughput of coarse-grained lock-based algorithm. We show that our model works well for the simple data structures that export one method: perform a critical section of size C guarded by CLH lock [START_REF] Craig | Building fifo and priorityqueuing spin locks from atomic swap[END_REF] (acquire the lock, perform work of size C and release the lock) followed by a parallel section of size P (simply, work of size P ).

Publications

The results presented in this thesis appeared originally in the following papers.

[1] Umut A Acar, Vitaly Aksenov, Arthur Charguéraud, and Mike Rainey. Performance challenges in modular parallel programs. In Proceedings of the twenty third ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming (PPoPP), pages 381-382. ACM, 2018 [START_REF] Umut | Algorithm Design: Theory and Practice[END_REF] Vitaly Aksenov, Vincent Gramoli, Petr Kuznetsov, Anna Malova, and Srivatsan Ravi. A concurrency-optimal binary search tree. In European Conference on Parallel Processing (Euro-Par), pages 580-593. Springer, 2017 [START_REF] Umut A Acar | Scheduling parallel programs by work stealing with private deques[END_REF] Vitaly Aksenov, Petr Kuznetsov, and Anatoly Shalyto. Parallel combining: Benefits of explicit synchronization. arXiv preprint arXiv:1710.07588, 2018 [START_REF] Umut A Acar | A work-efficient algorithm for parallel unordered depth-first search[END_REF] Vitaly Aksenov, Petr Kuznetsov, and Anatoly Shalyto. On helping and stacks. In Proceedings of NETYS 2018 [START_REF] Umut A Acar | Oracle-guided scheduling for controlling granularity in implicitly parallel languages[END_REF] Vitaly Aksenov, Dan Alistarh, and Petr Kuznetsov. Brief-announcement: Performance prediction of coarse-grained programs. In Proceedings of the thirty seventh annual ACM Symposium on Principles of distributed computing (PODC), pages 411-413, 2018

In parallel with this doctoral work, the author was also involved in the following paper:

[1] Umut A Acar, Vitaly Aksenov, and Sam Westrick. Brief-announsement: Parallel dynamic tree contraction via self-adjusting computation. In Proceedings of the twentyninth ACM Symposium on Parallelism in Algorithms and Architectures (SPAA), pages 275-277. ACM, 2017

Roadmap

In Chapter 2, (1) we present different models of computation (Section 2.1); (2) we describe schedulers and complexity models for parallel programs written for dynamic multithreading (Section 2.2); (3) we provide definitions concerning concurrent data structures: correctness and progress guarantees, complexity models and data types (Section 2.3).

In Chapter 3, we describe state-of-the-art implementations of binary search tree, skiplist and priority queue, used for experimental analyses in Chapters 5 and 6.

Chapter 4 presents our automatic granularity control algorithm with the theoretical bounds on the execution time of parallel programs using it and with the rigorous performance analysis.

Chapter 5 presents the concurrency-optimal binary search tree. Chapter 6 presents the parallel combining technique together with several applications. Chapter 7 presents a proof of the fact that stack does not have a wait-free help-free implementation in systems with reads, writes, compare&swap and fetch&add primitives, correcting a mistake in an earlier proof by Censor-Hillel et al.

Chapter 8 presents a simple method that predicts the throughput of simple coarsegrained lock-based data structures.

We conclude in Chapter 9 with a discussion of open questions and future work.

Background: Models and Definitions

In this chapter we present models, definitions and notations that will be used throughout the thesis. Individual chapters will be provided with additional definitions and notations specific to the chapter.

Parallel Models

In each considered parallel model there are n processes p 1 , . . . , p n that communicate via reads, writes and, possibly, other primitives (defined explicitly) on shared atomic registers. Each process is provided with its own program.

Parallel Random Access Machine

Parallel Random Access Machine (PRAM) model was proposed by Fortune and Wyllie [START_REF] Fortune | Parallelism in random access machines[END_REF] as a simple extension of the Random Access Machine model used in the design and analysis of sequential algorithms. A global clock synchronizes the operations of processes: one operation (or skip) of each process is executed during one tick of the clock. The memory model of PRAM is the strongest known consistency model [START_REF] Sarita | Shared memory consistency models: A tutorial[END_REF]: a write during clock tick t becomes globally visible to all processes in the beginning of clock tick t + 1. There are several variations of PRAM model based on how the concurrent writes and reads are handled. The exclusive read exclusive write (EREW) PRAM does not allow any simultaneous access to a single memory location. The concurrent read exclusive write (CREW) PRAM allows for concurrent reads but not concurrent writes. The concurrent read concurrent write (CRCW) PRAM allows for both concurrent reads and writes. The CRCW models can be distinguished further: common allows concurrent writes only when all processors are attempting to write the same value; arbitrary allows an arbitrary write to succeed; and priority allows the process with the smallest index to succeed.

There are further variations of PRAM model. For example, QRQW PRAM [START_REF] Phillip B Gibbons | The queue-read queuewrite pram model: Accounting for contention in parallel algorithms[END_REF] in which reads and writes are added into the queue and are served one per tick; and scan PRAM [START_REF] Guy | Scans as primitive parallel operations[END_REF], a version of EREW PRAM in which scan (prefix sum) operation takes one tick of the global clock.

PRAM model is unique in that it supports deterministic parallel computation, and it can be regarded as one of the most programmer-friendly models available. Numerous algorithms have been developed for PRAM model, for example, see the book by JáJá [START_REF] Jájá | An introduction to parallel algorithms[END_REF]. Unfortunately, this model is not useful in practice since it induces high overhead on the synchronization of processes during each tick.

Bulk Synchronous Parallelism

The bulk-synchronous shared-memory parallel (BSP) model, proposed by Valiant [START_REF] Leslie | A bridging model for parallel computation[END_REF], is an extension of PRAM. The computation is split into supersteps. Each superstep consists of a computation phase, during which the processes peform local computations, and a global interprocesor comunication phase, during which processes can share a data. After each superstep, the processes are synchronized using a barrier.

Asynchronous Shared Memory

An asynchronous shared memory model resembles the PRAM model, but the processes run asynchronously and all potentially conflicting accesses to the shared memory must be resolved by the programmer. If we extend this model with a special instruction -the synchronization of the subset of processes, it becomes Asynchronous PRAM [START_REF] Phillip | A more practical pram model[END_REF].

In this thesis, we use the Asynchronous PRAM model. Typically, in addition to standard primitives on shared atomic registers (reads and writes), we provide this model with more powerful ones such as compare&swap and fetch&add, etc.

Useful Primitives

The compare&swap primitive takes a target location, an expected value and a new value. The value stored in the location is compared to the expected value. If they are equal, then the value in the location is replaced with the new value and true is returned (we say that the operation is successfull). Otherwise, the operation fails (i.e., the operation is failed) and returns false.

The fetch&add primitive takes a target location and an integer value. The primitive augments the value in the location by the provided value and returns the original value.

Expressions of Parallel Programs

We consider two ways of expressing the parallel programs: for static multithreading or for dynamic multithreading.

Static Multithreading

For static multithreading the user writes a program for each process separately [START_REF] Thomas H Cormen | Introduction to algorithms[END_REF]. Typically, it is assumed that the number of processes executing the program is kept constant during the execution (note, that this number has to be less than n, the total number of processes).

Dynamic Multithreading

For dynamic multithreading, or nested fork-join parallelism, the programs are written using two constructions: fork that specifies procedures that can be called in parallel, and join that specifies a synchronization point among procedures [START_REF] Thomas H Cormen | Introduction to algorithms[END_REF]. Fork and join constructs can be nested, making this type of programs useful for divide-and-conquer algorithms.

An execution of programs written for dynamic multithreading can be modeled as a directed acyclic graph (dag) that unfolds dynamically. In this execution dag, each node represents a unit-time sequential subcomputation and each edge represents control-flow dependencies between nodes. A node that corresponds to a "fork" has two or more outgoing edges, and a node corresponding to a "join" has two or more incoming edges.

To execute a program some processes are chosen to work under guidance of a scheduler [START_REF] Umut | Algorithm Design: Theory and Practice[END_REF]. The scheduler is responsible for choosing which nodes to execute on each process during each timestep. It chooses for execution only ready nodes: unexecuted nodes whose predecessors have all been executed.

Schedulers

There exist two types of schedulers for dynamic multithreading: offline and online. Online scheduler works with a dynamically unfolding dag, while offline scheduler works with a pre-provided full computation dag. Typically, online schedulers are used in practice, since the execution dag is not known a priori and offline scheduler cannot be used. However, offline schedulers are easier to implement and analyse.

The commonly used offline schedulers [START_REF] Umut | Algorithm Design: Theory and Practice[END_REF] are greedy schedulers. A scheduler is greedy if it never leaves a process idle unless there are no ready nodes.

Online schedulers obey the following scheme. The algorithm maintains a pool of work, consisting of ready nodes. Execution starts with the root node in the pool. It ends when the pool becomes empty. During the execution, a process withdraws a node from the pool, executes it and puts the ready successor nodes into the pool.

One of the most popular online scheduler is the work-stealing scheduler [START_REF] Robert | Scheduling multithreaded computations by work stealing[END_REF]. Each process maintains a deque, doubly ended queue, of nodes. Each process tries to work on its local deque as much as possible: a process pops the node from the bottom of its deque and executes it. When a node is executed, the process puts all ready successor nodes to the bottom of the deque. If a process finds its deque empty, it becomes a thief. A thief picks a victim process at random and attempts to steal a node from the top of the victim's deque. If the victim's deque is not empty, the steal attempt succeeds, otherwise, it fails.

Implementations

The support of nested parallelism dates back at least to Dijkstra's parbegin-parend constuct. Many parallel languages support nested parallelism includling NESL [START_REF] Guy | Nesl: A nested data-parallel language[END_REF], Cilk [START_REF] Frigo | The implementation of the cilk-5 multithreaded language[END_REF], the Java fork-join framework [START_REF]Java Fork-Join[END_REF], OpenMP [START_REF]Openmp application interface[END_REF], X10 [START_REF] Charles | X10: an objectoriented approach to non-uniform cluster computing[END_REF], Habanero [START_REF] Budimlić | The design and implementation of the habanerojava parallel programming language[END_REF], Intel Threading Building Blocks [START_REF]Intel. Intel threading building blocks[END_REF], and Task Parallel Library [112].

Algorithmic Complexity Model

Parallel algorithms are generally analyzed using Work-Span Framework [START_REF] Guy | Programming parallel algorithms[END_REF]. Work W is the total number of nodes in the computation dag and span S is the length of the longest path in the dag.

There exist several theorems that bound the execution time of a parallel program with work W and span S with different schedulers.

The first one is the renown Brent's theorem [START_REF] Richard | The parallel evaluation of general arithmetic expressions[END_REF].

Theorem 2.2.1 (Brent's theorem). With any greedy scheduler the execution time of program on n processes does not exceed W n + S.

The bound on the execution time with the work-stealing scheduler was provided by Aurora et al. [START_REF] Nimar S Arora | Thread scheduling for multiprogrammed multiprocessors[END_REF].

Theorem 2.2.2. With the work-stealing scheduler the expected execution time of program on n processes is O(

W n + S).
Moreover, for any > 0, with probability at least 1 -, the execution time on n processes is O( W n + S + log 1 ).

Work and Span of PRAM Algorithm

In this thesis, we will talk about work and span of PRAM algorithms even though this notation is only applicable to programs written for dynamic multhreading. Here, we explain how to convert PRAM algorithm to its counterpart written to dynamic multhreding. Instead of presenting the precise transformation algorithm, we explain how the computation dag should look like for each input (this should be enough to talk about work and span). At first, we execute the PRAM algorithm on that input and suppose that it takes T ticks. Our dag will have T layers of at most n nodes, the i-th layer contains nodes that correspond to the operations performed by processes during the i-th tick of the PRAM algorithm, and the nodes in the i-th level depend on the nodes in i -1-th level. Now, we can state that the work and span of the PRAM algorithm is work and span of that dag.

However, typically, the span of the PRAM algorithm, re-written for dynamic mutithreading, is log n times bigger. This is due to the fact that our dag uses a complex dependency between layers, while in practice we only can have nodes with a constant number of predecessors and successors. This replacement increases the span of the algorithm by log n, while the work increases by no more than a constant.

An Example

To give an intuition behind the expression of parallel programs for dynamic multithreading and the use of the Work-Span Framework, consider an example: exclusive scan operation. Given an array a of s integers exclusive scan operation returns an array of length s such that its i-th value is the sum of the first i -1 values of a. Below, we provide a standard algorithm of exclusive scan written for dynamic multithreading.

In the pseudocode, instead of two constructions fork and join, we use one primitive fork2join that takes two functions, named branches, as arguments and executes them in parallel until their completion.

1 phase1(a, tmp, l, r): We briefly explain how the algorithm works. During the execution, we implicitly build a segment tree with height O(log s), i.e., a binary tree whose root corresponds to segment [0, s), a node corresponding to a segment [l, r) is either a leaf, if l equals to r -1, or a parent of two nodes corresponding to segments [l, l+r 2 ) and [ l+r 2 , r). The algorithm consists of two phases. During the first phase, we calculate an array tmp of length s: suppose that m = l+r 2 is a split point of a segment [l, r) in the segment tree, then tmp[m] equals to

2 if l = r -1: 3 return a[l]
a[l] + . . . + a[m -1] (note that each i ∈ [1, n) is a split point of exactly one segment).
During the second phase, we calculate the results array ans: ans[m] is the sum of the values on the segments to the left of m.

The total work performed by the algorithm is O(s): we visit each node in the segment tree O(1) times and the number of nodes is O(s). The span of the algorithm is O(log s) as the height of the segment tree. Thus, the expected execution time of the algorithm on n processes with work-stealing scheduler by Theorem 2.2.2 is O( s n + log s).

Data Structures

A data type is a tuple (Φ, Γ, Q, q 0 , θ), where Φ is a set of operations, Γ is a set of responses, Q is a set of states, q 0 is an initial state and θ ⊆ Q × Φ × Q × Γ is a transition function, that determines, for each state and each operation, the set of possible resulting states and produced responses.

A sequential implementation (or sequential data structure) corresponding to a given data type specifies, for each operation, a sequential read-write algorithm, so that the specification of the data type is respected in every sequential execution.

A batched version [START_REF] Neiger | Set-linearizability[END_REF] of a data type τ = (Φ, Γ, Q, q 0 , θ) is specified by the transition

function β ⊆ θ ∪ +∞ i=2
Q×Φ i ×Q×Γ i that determines, for each state and a batch of requests, the set of possible resulting states and produced responses for each request in the batch.

In the simplest batched version of the data type the resulting state and the responses to the operations from the batch correspond to the sequential application of the operations in some order.

A batched implementation (or batched data structure) corresponding to a given data type τ specifies an algorithm of apply operation that applies a batch of operations, so that the specification of the batched version of τ is respected.

When apply operation is implemented as a parallel program, we call the batched implementation parallel.

A concurrent implementation (or concurrent data structure) corresponding to a given data type assigns for each process and each operation type a deterministic state machine that specifies the sequence of steps (primitives on the shared atomic registers) that the process needs to perform to complete the operation.

Corectness, Progress Guarantees and Complexity Model of Concurrent Data Structures

Correctness

Here, we describe common corectness criteria of concurrent implementations in the asynchronous shared memory model. A low-level history (or an execution) is a finite or infinite sequence of primitive steps: invocations and responses of high-level operations, invocations and responses of primitives on the shared registers (reads, writes, etc.). We assume that executions are well-formed: no process invokes a new primitive, or high-level operation before the previous primitive, or a high-level operation, respectively, returns or takes steps outside its operation's interval.

An operation is complete in execution α if the invocation event is followed by a matching response; otherwise, it is incomplete.

A high-level history (or simply a history) of execution α on high-level object O is the subsequence of α consisting of all invocations and responses of operations on O.

Here we define three correctness criteria that appear in the thesis: quiescent consistency [START_REF] Aspnes | Counting networks[END_REF], linearizability [START_REF] Maurice | Linearizability: A correctness condition for concurrent objects[END_REF] and set-linearizability [START_REF] Neiger | Set-linearizability[END_REF].

Definition 2.3.1 (Quiescent consistency). A period of quiescence in a history h is a period between two consecutive events such that all invoked operations are complete.

A history h defines a partial order on operations: op 1 preceeds op 2 (op 1 ≺ h op 2 ) if they are separated by a period of quiescence. A quiescence given a history h is a sequential high-level history of operations Q such that:

1. Q consists of all the operations completed in h and, possibly, some operations that have started but not yet completed in h;

the operations have the same input and the same output as the corresponding operations in h;

for every two operations op

1 ≺ h op 2 if op 2 is included in Q, then op 1 preceeds op 2 in Q (op 1 ≺ Q op 2 ).
A concurrent implementation of a data type is quiescently consistent if, for each of its histories, there exists a quiescence.

Definition 2.3.2 (Linearizability).

A history h defines a partial order on operations: op 1 preceeds op 2 (op 1 ≺ h op 2 ) if op 1 is completed before op 2 begins. A linearization of a history h is a sequential high-level history of operations L such that:

1. L contains all the operations completed in h and, possibly, some operations that have started but not yet completed in h;

the operations have the same input and the same output as the corresponding operations in h;

for every two operations op

1 ≺ h op 2 if op 2 is included in L, then op 1 preceeds op 2 in L (op 1 ≺ L op 2 ).
A concurrent implementation of a data type is linearizable if, for each of its histories, there exists a linearization.

Definition 2.3.3 (Set-linearizability). A set-linearization of a history h is a sequence S of batches of operations with set-linearization points such that:

1. a union of batches in S contains all the operations completed in h and, possibly, some operations that have started but not yet completed in h;

for any batch b from S, its set-linearization point lies between invocation and response of all operations of b in h;

3. the operations have the same input and the same output in h as if they are applied sequentially in batches of S.

A concurrent implementation of a batched data type is set-linearizable if, for each of its histories, there exists a set-linearization.

In an infinite execution a process is faulty if it stops taking steps before completing its operation. A process which is not faulty is called correct.

Progress Guarantees

A method implementation is non-blocking if a failure or a suspension of any other process does not prevent other threads from making progress during the execution of this method. A method that is not non-blocking is called blocking.

Non-blocking.

A method implementation is wait-free if it guarantees that every call of this method eventually completes. A method implementation is bounded wait-free if every call of this method completes in a bounded number of steps. We say that a data structure is wait-free if its methods are wait-free.

A method implementation is lock-free if it guarantees that some call of this method eventually completes. We say that a data structure is lock-free if its methods are lockfree. Wait-freedom is stricter than lock-freedom: a wait-free method is lock-free, but not vice versa.

We say that a method call is executed in isolation for a duration if no other threads take steps during that time. A method implementation is obstruction-free if it guarantees that every call completes if the corresponding thread executes in isolation for long enough.

Blocking. Blocking implementations can provide two progress guarantees: starvationfreedom and deadlock-freedom.

A method is starvation-free if it guarantees that every call of this method eventually completes if each process takes an infinite number of steps. We say that a data structure is starvation-free if its methods are starvation-free.

A method is deadlock-free if it guarantees that an infinite number of calls of this method completes if each process takes an infinite number of steps We say that a data structure is deadlock-free if its methods are deadlock-free. Note that starvation-freedom implies deadlock-freedom.

Complexity for Concurrent Data Structure

Two architecture paradigms are considered in the literature that allow shared variables to be locally accessed: the cache-coherent (CC) and the distributed shared memory (DSM) machines [START_REF] James H Anderson | Shared-memory mutual exclusion: major research trends since 1986[END_REF].

In a CC machine, each process has a private cache, and some hardware protocol is used to enforce cache consistency (i.e., to ensure that all copies of the same variable in different local caches are consistent). A shared variable becomes locally accessible by migrating to a local cache line. Memory references to the variable stored on the cache line in a process's cache are called local and are much faster than ones that has to load a cache line (i.e., cache misses), called remote memory references (RMRs).

In a DSM machine, each processor has its own memory module that can be accessed without accessing the global interconnection network. On such a machine, a shared variable can be made localy accessible by storing it in a local memory module. An access to a memory location in a process's own memory module is local and a reference to another process's memory module is an RMR.

Complexity of concurrent data structures' operations, typically, cannot be expressed in steps: the standard spin-wait loop (the loop that spins on a local boolean variable) can take infinite number of them. Instead, complexity is measured in RMRs and, for example, the described spin-wait loop takes 1 RMRs. However, the algorithm can have different complexities on CC and DSM machines. For example, on DSM machine, the spin-wait loop on a non-local variable can take more than 1 RMRs.

Data Types Considered in This Thesis

Stack

Stack is a data type that maintains a multiset of values and supports two operations:

• push(v) -inserts value v into the set;

• pop() -extracts the most recently added element that was not yet extracted, or returns ⊥ if the set is empty.

Queue

Queue is a data type that maintains a multiset of values and supports two operations:

• enqueue(v) -inserts value v into the set;

• dequeue() -extracts the earliest added element that was not yet extracted, or returns ⊥ if the set is empty.

Set

Set is a data type that maintains a set of elements and supports three operations:

• insert(v) -inserts value v into the set, returns true, if and only if v is absent in the set;

• delete(v) -deletes value v from the set, returns false, if and only if v is present in the set;

• contains(v) -returns whether v is in the set or not.

Sets are generally implemented in two ways: using hash tables or some sort of balanced (tree-like) data structures, e.g., binary, B-, (a, b)-trees and skip-lists. In the thesis we are interested in binary search trees and skip-lists implementations. Such implementations support operations in logarithmic time. We discuss their state-of-the-art implementation in Section 3.1 and 3.2.

Priority Queue

Priority queue is a data type that maintains an ordered multiset and supports two operations:

• insert(v) -inserts value v into the set;

• extractMin() -extracts the smallest value from the set, or returns ⊥, if the set is empty.

We discuss its state-of-the-art implementations in Section 3.3.

Dynamic graph

Dynamic graph is a data type that maintains a graph on N vertices and supports three operations:

• insert(u, v) -adds an edge between vertices u and v;

• delete(u, v) -removes an edge between vertices u and v;

• isConnected(u, v) -returns whether u and v are connected.

In the thesis, we use only one implementation of dynamic graph presented by Thorup et al. [START_REF] Holm | Poly-logarithmic deterministic fully-dynamic algorithms for connectivity, minimum spanning tree, 2-edge, and biconnectivity[END_REF]: on a graph with N vertices insertion and deletion take O(log 2 N ) amortized time, and connectivity query takes O(log N ) time.

Dynamic forest

Dynamic forest is a special case of dynamic graph: the sequence of operations can be appled only if after each operation a graph is a forest, i.e., after each operation there exists no more than one simple path between any pair of vertices.

Overview of Data Structure Implementations

In this chapter, we overview the state-of-the-art sequential, parallel batched and concurrent implementations of set and priority queue data types. Most of the implementations described here are used to compare against our novel data structures, either empirically or theoretically. However, if you are not interested in the overview, you can safely skip this chapter.

We are primarily interested in set implementations based on binary search tree and priority queue implementations. Since the majority of priority queue implementations are based on skip-list, we also discuss set implementations based on skip-list.

Binary Search Trees

Binary search tree (BST) is a rooted binary tree, whose nodes each store a value (and, possibly, other information) and have two subtrees, commonly denoted left and right (the roots of these subtrees are called left and right children, respectively, if subtree is not empty). The tree additionally satisfies the order property: the value in each node is strictly greater than values in its left subtree and strictly smaller than values in the right subtree. The node is named a leaf, if it does not have any child. Otherwise, the node is called inner.

Three types of binary search trees are distinguished in literature: internal, external and partially-external. In an internal tree, the set of an internal tree consists of the values in all nodes. In an external tree, the set consists of the values in the leaves; and each inner node has two children and is called routing. In a partially-external tree, each node can be either routing or data; each routing node has exactly two children; and the set consists of the values in data nodes.

To provide logarithmic bounds for the operations, a binary search tree is augmented with a balancing scheme. In a few words, balancing scheme is an algorithm that explains how to restructure the binary tree after an update, so that a special condition, called an invariant, is satisfied. Some of the schemes are:

• AVL scheme [START_REF] Adelson | An algorithm for the organization of information[END_REF]. Invariant: for any node the heights of left and right subtrees differ by at most one.

• Red-Black scheme [START_REF] Leo | A dichromatic framework for balanced trees[END_REF]. Each node has a color: red or black. Invariant consists of two parts: 1) no red node has a red child; and 2) the number of black nodes on every path from the root down to any leaf is the same.

• Weight-balanced scheme or BB[α] [START_REF] Nievergelt | Binary search trees of bounded balance[END_REF]. Let w(T ) be the number of nodes in tree T and let T l be the left subtree of T . Invariant: for any node with subtree T , α ≤ w(T l ) w(T ) ≤ 1 -α.

• Treap scheme [START_REF] Seidel | Randomized search trees[END_REF]. Each node has a uniformly random priority. Invariant: the priority of each node is not less than the priorities of its children.

One of the most important features of the presented trees is that they allow an operation join in logarithmic time. Join unites two trees, such that the maximal key of the first is less than the minimal key of the second.

Parallel Batched Implementations

The first parallel batched binary search tree was presented by Blelloch and Reid-Miller [START_REF] Guy | Fast set operations using treaps[END_REF]: they used treap balancing scheme. The algorithm is implemented by a recursive function apply(T, A) that returns a tree after applying operations A, sorted by the arguments, to tree T . apply(T, A) splits the tree T by the argument of the middle operation A[m ← l+r 2 ] to T l and T r , calls recursively in parallel T l ← apply(T l , A[1, . . . , m -1]) and T r ← apply(T r , A[m + 1, . . . , |A|]), and, finally, returns the join of T l , argument of A[m] and T r , if A[m] is Insert, and return the join of T l and T r , if A[m] is Delete.

It was shown that given the size of the tree k and the size of the batch m the algorithm is work-efficient, i.e., the work is O(m • log( k m + 1)), and is highly parallel, i.e., the span is O(log 2 k). We called this algorithm work-efficient, because in order to apply a batch of size m to a tree of size k any sequential algorithm should make at least log k+m m = θ(m • log( k m + 1)) comparisons. The second paper by Blelloch et al. [START_REF] Guy E Blelloch | Just join for parallel ordered sets[END_REF] is a generalization of the previous one. They took the same algorithm and applied it to four different implementations of binary search tree (one for each balancing scheme described earlier). For all four algorithms the work is

O(m • log( k m + 1)) and the span is O(log k • log m),
where k is the size of the tree and m is the size of the batch.

Concurrent Implementations

Lock-Based Partially-External BST by Bronson et al.

We start with the first practical balanced concurrent BST by Bronson et al. [START_REF] Nathan G Bronson | A practical concurrent binary search tree[END_REF]. The tree is a partially-external BST and it uses relaxed AVL balancing scheme [START_REF] Bougé | Heightrelaxed avl rebalancing: A unified, fine-grained approach to concurrent dictionaries[END_REF]. Any operation starts with a traversal that is implemented as a recursive function: it goes optimistically down the tree and the recursion saves for us a path of nodes that we traversed from the root. If someone has updated the links on the nodes in such a way that we cannot continue the traversal correctly, e.g., the current node is removed or there was a "bad" rotation, we have to backtrack (return from recursive calls) and find the lowest position on the traversed path from the root, saved by the recursion, from which we can continue the traversal down. After the traversal, an Insert or Delete operation properly updates the node and goes up the tree to fix every node on the path to the root using locks: remove routing nodes with one child and rebalance if necessary. Rebalance is performed using relaxed AVL scheme [START_REF] Bougé | Heightrelaxed avl rebalancing: A unified, fine-grained approach to concurrent dictionaries[END_REF]: we optimistically calculate the difference in heights (possibly, stale) between the children; if the absolute difference is bigger than one, we lock necessary nodes (the node, the parent and the proper child) and perform the proper rotation; and, finally, we update the height of the node from the heights of its children. It is argued that when there is no pending operation, the tree is a strict AVL tree.

Non-Blocking External BST by Ellen et al.

The first practical lock-free BST was proposed by Ellen et al. [START_REF] Ellen | Nonblocking binary search trees[END_REF]. The tree is implemented as an external BST. The blocking version of external BST is quite straightforward and Barnes's technique [START_REF] Barnes | A method for implementing lock-free shared-data structures[END_REF] is used to make the algorithm lock-free. Each node of a tree is expanded with a pointer info to Info record. Each operation starts with a simple traversal. When an operation reaches the effective node node the operation indicates that it wants to change node: it creates Info record and sets the pointer info to this record. Info record contains enough information to help complete the original operation so that the node can be unflagged. Typically, helping often leads to poor performance because several processes try to perform the same piece of work. In this implementation a process p helps another process's operation only if this operation prevents p's progress. Because of this policy traversals do not help at all. Wait-Free Red-black Tree by Natarajan et al. Natarajan et al. [121] presented a wait-free red-black tree. As a basis they took an external red-black tree with top-down operations by Tarjan [START_REF] Robert E Tarjan | Efficient top-down updating of red-black trees[END_REF]. To translate a sequential algorithm into wait-free concurrent one they used Tsay and Li's framework [START_REF] Tsay | Lock-free concurrent tree structures for multiprocessor systems[END_REF]. In this framework each operation holds a window (subtree) that moves down the tree. If the window of an operation intersects with some lower window of a different operation, the process tries to move the lower window first. To move a window a process copies the subtree from the window, updates the copy, links the copy instead of the original subtree and, finally, moves a window lower. The straightforward application of Tsay and Li's framework is very expensive, because of that some optimizations are proposed which we do not discuss here.

Non-Blocking Internal BST by Howley et al.

Howley et al. [START_REF] Shane | A non-blocking internal binary search tree[END_REF] presented an internal lock-free binary BST. They used the similar technique of Info records as in BST by Ellen et al. [START_REF] Ellen | Nonblocking binary search trees[END_REF]. Nevertheless, this BST is more complicated to implement: the traversal should be accurately taken care of since the desired value can be moved up the tree; furthermore, a delete operation now has to search for the replacement value. Traversals and insertions in Howley and Jones's algorithm are generally faster than in Ellen et al.'s algorithm: the traversed path in an external tree terminates at a leaf node while in an internal tree it may terminate at an internal node. However, delete operations in an internal tree are generally slower than those in an external tree.

Lock-Based Partially-External BST with Helper Thread by Crain et al.

Crain et al. [START_REF] Crain | A contention-friendly binary search tree[END_REF] presented a contention-friendly lock-based partially-external BST with relaxed AVL balancing scheme. Given that the traversal is the longest part of the operation concurrent BST should avoid unnecessary verifications and backtracking which happen to be an issue of the previous balanced BST by Bronson et al. Crain et al. simplified an implementation: the operations are decoupled from structural adaptations, e.g., physical removal and rebalancing. These adaptations are performed in a separate daemon thread that continually recursively restructures the tree. To get rid of verifications during the traversals the daemon thread performs restructures as follows: 1) to remove a routing node it takes locks on the node and its parent, reroutes pointers to the children to point onto the parent and marks the node as deleted; 2) to rebalance a node it takes necessary locks (on the node, its parent and the proper child), replaces a current node with a new node and marks the current node as deleted. By that all operations on a data structure become short and lightweight.

Lock-Based Internal BST with Logical Ordering by Drachsler et al.

Drachsler et al. [START_REF] Drachsler | Practical concurrent binary search trees via logical ordering[END_REF] proposed a practical lock-based internal concurrent BST with relaxed AVL balancing scheme. Their main goal was also to reduce overheads during the traversals. For that they stitch the nodes of BST in sorted order with a linked list. Now, to find a node with the given value in BST an operation starts a wait-free traversal, stops at some node, and then moves to the left or to the right of this node in the linked list. Note that the overhead of any modification increases since the algorithm in addition to updating links of the tree updates links in the linked list.

Lock-Free External BST with Low Overhead by Natarajan et Mittal

Natarajan and Mittal [START_REF] Natarajan | Fast concurrent lock-free binary search trees[END_REF] proposed an external lock-free BST as an improvement of BST by Ellen et al. We describe three key ideas of this improvement. At first, the algorithm operates on edge-level (marks edges) whereas Ellen et al.'s algorithm operates at node-level (marks nodes). This leads to the smaller contention window: any two operations that can be executed concurrently in Ellen et al.'s algorithm can also be executed concurrently in Natarajan et al.'s algorithm, but not vice versa. For example, the algorithm by Natarajan et al. allows setting two children of a node independently. The second improvement is that Info records are not used, instead only edges are being marked, and the help is performed only for delete operations (insertions do not need help). Finally, they optimized the operations: the number of allocated objects and the number of executed atomic instructions are less than in lock-free algorithms by Ellen et al. and Howley et al.

Lock-Free External Balanced BST by Brown et al.

Brown et al. [START_REF] Brown | A general technique for non-blocking trees[END_REF] presented a lock-free balanced BST based on an external chromatic BST [START_REF] Nurmi | Chromatic binary search trees[END_REF], which is a relaxed version of the red-black tree. To implement BST they introduced generalized versions of LL and SC primitives: LLX and SCX [START_REF] Brown | Pragmatic primitives for non-blocking data structures[END_REF] which operate on Records consisting of a fixed number of mutable and immutable fields. LLX(r) attempts to take a snapshot of the mutable fields of a Record r. SCX(V , R, f ld, new) takes as arguments a sequence V of Records, a subsequence R of V , a pointer f ld to a mutable field of one Record in V , and a new value new for that field. SCX tries to atomically store the value new in the field that f ld points to and to finalize each Record in R, i.e., make all fields in these Records to be immutable. A traversal in this algorithm is performed in a standard wait-free manner by following links from the root. Insertions and deletions use LLX and SCX (on two and three Records, correspondingly) to update the nodes, then they start a cleanup phase: continually rebalance the first unbalanced node on the path from the root to the position of the value. Brown et al. proved that the height of the tree at any time is O(c + log k), where k is the size of the tree and c is the number of updates in progress.

Skip-Lists

A skip-list data structure was proposed in 1990 by Pugh [START_REF] Pugh | Skip lists: a probabilistic alternative to balanced trees[END_REF]. It maintains a collection of sorted linked lists, which mimics, in a subtle way, a balanced search tree.

Each list has a level, ranging from 0 to a maximum (constant MAX_LEVEL). Each value has a corresponding node that is linked into a subset of lists. The bottom-level list contains all nodes, and each higher-level list is a sublist of the lower-level lists. The higher-level lists are shortcuts into lower-level lists.

The structure itself is randomized: the node is created with a random top level (or simply level), and belongs to all lists up to that level. Top levels are chosen in such a way that the expected number of nodes decreases exponentially with the increase of level. For example, if the probability of the node to appear at level i is 1 2 i , then, roughly speaking, each link at level i skips 2 i nodes in the bottom-level list.

To simplify implementations of skip-lists we prepopulate them with head (value -∞) and tail (value +∞) sentinel nodes that appear in all the lists. Each node has an array next of references: one reference for each list the node belongs to.

In the algorithms we use the following additional notations: in a sorted linked list, the node with the biggest value smaller than value v is called the predecessor of v, while the node with the smallest value bigger than value v is called the successor of v. Also, we sometimes call the top level of a node simply as the level of a node.

The sequential algorithm works as follows. Given value v each operation starts with the traversal. The traversal starts from the top-level at sentinel node head and goes down the levels one after the other until it reaches the predecessor of v at the bottom level: at each level i, the algorithm takes the last visited node and follows next pointers until it reaches the predecessor of v; we store the found predecessor in preds [i]

Concurrent Implementations

Lock-Based Skip-List by Pugh

The first concurrent lock-based skip-list was presented by Pugh [START_REF] Pugh | Concurrent maintenance of skip lists[END_REF]. At first, he described a lock-based linked list. To find a position to insert or delete, an operation with argument v does a lock-free lookup for the node pred of v by following next pointers. Then the operation takes a lock on the next pointer of pred, and using hand-over-hand locking follows next pointers to adjust the predecessor pred of v. Insert operation creates a new node and links it after pred in the list. Delete operation takes an additional lock on the next pointer of the node node with value v, sets the next pointer of node to pred and unlinks node.

The skip-list algorithm works similarly to the described linked list. An operation with an argument v traverses the skip-list down to the bottom level using the sequential algorithm and stores the predecessors preds at each level. Insert creates a new node with level and iterates through all the lists from level 0 to level : at each level i, it finds the real predecessor of v using the estimation preds[i] (as in lock-based linked list algorithm) and links the new node. Delete finds a node node with value v and iterates through all the lists from the level of the node downto level 0: at each level i, it finds the real predecessor using the estimation preds[i] (as in lock-based linked list algorithm), sets the next pointer node[i].next to the predecessor and unlinks node.

Lock-Free Skip-List by Sundell and Tsigas

Sundell and Tsigas [START_REF] Sundell | Scalable and lock-free concurrent dictionaries[END_REF] presented the first lock-free skip-list. They extended nodes with:

(1) a reference prev that points to some predecessor (not necessarily, exact) at some level if the node is under deletion; and (2) an integer validLevel that indicates the current level up to which the node is linked. Also, for each node, each pointer in next array is now extended with a boolean mark that indicates whether the node is under deletion.

The algorithm uses auxiliary functions scan and helpDelete. Given a node node, a level and a value v, scan follows next pointers from node on that level until it finds the predecessor of v, and at the same time it helps to delete nodes with the marked next pointer at that level using helpDelete. The helping function helpDelete, given a node with value v and a level , deletes the node and returns the current predecessor of v at level : mark all next pointers from top-level up to level ; then find a current predecessor of v: traverse either from the node at prev pointer or from the head, if prev has a small level; and, finally, unlink a node at level .

Each operation with an argument v starts with the traversal: start at the top level, then, at each level, the traversal calls scan function given the predecessor from the previous (upper) level. An insertion then creates and links a new node: at each level from bottom to top, find a current predecessor of v using scan on the estimation from the traversal and link the new node. A deletion sets prev pointer to the estimated predecessor of v at level node.level/2 from the traversal, marks the next pointers from top to bottom and, finally, unlinks a node: at each level from top to bottom, find a current predecessor of v using scan given the estimation from the traversal and unlink the node. Note that the traversal call in contains can skip the help to delete marked nodes.

Lock-Free Skip-List by Fomitchev and Ruppert

Fomitchev and Ruppert [START_REF] Fomitchev | Lock-free linked lists and skip lists[END_REF] designed a lock-free skip-list similar to the lock-based skip-lit by Pugh. At first, they present a lock-free linked list with the bounds on the running time of operations. They augment nodes with a backlink pointer and augment each next pointer with two bits: flag and mark. Mark bit is used to register the deletion of the node, while flag bit is a warning that a deletion of the next node is in progress. Flag bit is introduced only to ensure the bounds on the running time.

Each operation starts with the traversal that finds a predecessor pred of an argument v: start from the head, follow next pointers and help to delete marked nodes on its way. Insert then creates a new node and tries to link it. If the attempt to link is unsuccessful then: (1) if the next pointer of pred is flagged, then the operation help the concurrent delete operation; then, (2) while the next pointer of pred is marked the operation follows the backlink pointers; finally, (3) the operation adjusts pred. Delete takes node node with value v and attempts to perform four steps: 1) flag prev.next; 2) set the backlink of node to pred; 3) mark node.next; 4) unlink node and unmark node.next. If the attempt is unsuccessful the operation helps the concurrent delete operation, adjusts prev and retries.

This lock-free linked list is a basic building block of the proposed lock-free skip-list. This skip-list has a non-standard representation of a node: a node is represented with a linked list (a tower) from the top-level subnode (tower_root) to the bottom-level subnode. Each subnode stores a successor (next pointer) in the level-list, a subnode below and a link to tower_root. Additionally, tower_root is extended with the value field, other subnodes do not contain values. Each operation starts with the traversal: the standard traversal algorithm that helps to delete subnodes from top to bottom of a node which tower_root marked for deletion. Insert links a new tower from bottom to top: at each level i, it inserts a subnode in the linked list trying with preds[i] as the predecessor. Delete unlinks a tower from top to bottom: at each level i, it deletes a subnode from the linked list trying with preds[i] as the predecessor.

Lock-Based Lazy Skip-List by Herlihy et al.

Herlihy et al. [START_REF] Herlihy | A simple optimistic skiplist algorithm[END_REF] proposed a simple lock-based "lazy" skip-list. Each node is extended with a mark that indicates whether the node is under deletion. Each operation with an argument v starts with the standard traversal that finds the predecessors preds and the successors succs of v.

Insert then tries to lock the predecessors at each level from bottom to top. When all the locks are taken successfully, the algorithm checks that: (1) the predecessors are not marked; and (2) for each level i, preds [i].next still equals to succs [i]. If the conditions are satisfied, the insertion creates a new node and links it from the bottom level; otherwise, it restarts from the beginning.

Deletion locks the node node with value v and marks it. Then it locks the predecessors at each level from bottom to top and checks: (1) the predecessors are not marked; and (2) for each level i, prev [level].next still equals to node. If the conditions are satisfied, the deletion unlinks a node from top to bottom; otherwise, the operation restarts.

Lock-Free Skip-List by Herlihy et at.

The last skip-list implementation described here is a simple lock-free skip-list by Herlihy et al [START_REF] Herlihy | A lock-free concurrent skiplist with wait-free search[END_REF]. Next pointers are augmented with a mark that indicates whether the node is under deletion. Each modification operation with an argument v starts with the traversal: the standard traversal algorithm that at the same time unlinks all the traversed nodes with a marked next pointer. Insert creates a new node and links it from bottom to top, but if at some level the real predecessor and successor of v are not the same as found by the previous traversal -the traversal is repeated. Delete goes from top to bottom and only marks the next pointers of a node with value v obtained from the traversal. If it marked the next pointer at the bottom level it calls the traversal to physically delete the node. Contains function uses the standard traversal that does not modify anything.

Priority Queues

Sequential Binary Heap

We start with the simplest sequential priority queue and its simple modification. This implementation (or its modification) is a basis of various priority queue implementations, either parallel batched or concurrent, including a novel parallel batched implementation presented in Section 6.4.

A binary heap of size m is represented by a complete binary tree with nodes indexed from 1 to m. Each node v has at most two children: 2v and 2v + 1 (to exist, 2v and 2v + 1 should be less than or equal to m). For each node, the heap property should be satisfied: the value stored in the node is less than or equal to the values stored in its children.

The heap is represented with size m and an array a where a[v] is the value at node v. At first, we describe the classical binary heap algorithm [START_REF] Thomas H Cormen | Introduction to algorithms[END_REF]: An important modification of that algorithm was presented by Gonnet and Munro [START_REF] Gaston | Heaps on heaps[END_REF]. extractMin() remains the same while insert(x) works from top to bottom. insert(x) initializes a variable val with x, increments m and traverses the path from the root to a new node m. For each node v on the path, if val < a [v], then the two values are swapped. Upon reaching node m the operation sets a[m] to val.

• extractMin()

Parallel Batched Implementations Parallel Batched Heap By Pinotti and Pucci for CREW PRAM

The first parallel batched priority queue was presented by Pinotti and Pucci [START_REF] Pinotti | Parallel priority queues[END_REF]. The algorithm works in CREW PRAM with n processes. The priority queue is represented by a complete binary heap and each node contains a sorted array of n values. The arrays in nodes satisfy extended heap property: the minimal value in a node is bigger or equal than the maximal value in the parent. This data structure supports an insertion of n values and an extraction of n minimums.

To insert n values a new node is created, the provided n values are preliminarily sorted using Cole's algorithm [START_REF] Cole | Parallel merge sort[END_REF] in O(log n) time and merged with all values in the ancestors of the new node using Kruskal's merging algorithm [START_REF] Kruskal | Searching, merging, and sorting in parallel computation[END_REF] in O(log m + log log n) time, where m is the number of nodes. Then the sorted array is distributed between the nodes on the path from the new node to the root.

To extract n values, at first, an array in the root is saved as a response and is replaced with an array from the last node. Then we construct the minimal path µ: suppose we are now at node v, we add the child c with the smallest maximal value to µ and continue with c. For each node v in µ, we merge an array in v with an array in its sibling, split this array into halves, give the smallest half to v and the other half to the sibling in O(log n) total time. Then we merge arrays of all the nodes v on a path µ and distribute the resulting array to the nodes at µ.

Summarizing, insertions and extractions in batches of size n take O(log m + log log n) time, and O(n • (log m + log log n)) work where m is the number of nodes.

Parallel Batched Heap by Deo and Prasad for EREW PRAM

Next we discuss a parallel batched priority queue by Deo and Prasad [START_REF] Deo | Parallel heap: An optimal parallel priority queue[END_REF]. The algorithm is described for EREW PRAM with n processes. As in the previous algorithm the priority queue is represented by a complete binary tree where each node contains n sorted values and satisfies extended heap property.

An insertion of c ≤ n keys works similarly to the sequential algorithm by Gonnet and Munro. At first, the operation sorts arguments using Cole's algorithm and stores them in a new array x. The operation starts at the root and traverses towards the last leaf: at each node the array at the node is merged with array x; then, first n values are stored in the node, while x is populated with the values left.

An extraction of c ≤ n minimums retrieves the first c values from the root as the answer, then replaces them with c values from the last leaf. Then the operation starts a traversal from the root towards some leaf. Suppose the traversal is currently at node curr and let child be the child with the biggest maximal value. The array at curr is merged with arrays in children; the smallest n values are stored in curr, the next n values are stored in child; and the traversal continues with the other child.

Thus, a batch of size c ≤ n can be applied in O(log m • log n) time and O(n • log m • log n) work where m is the number of nodes. In the same paper, they discuss how the pipelining approach can reduce time to O(log m). Unfortunately, this idea can be applied only if there is a continuous flow of batches to apply.

Parallel Batched Priority Queue by Sanders et al. for Asynchronous PRAM

We finish the survey of parallel batched priority queue with the algorithm by Sanders [START_REF] Sanders | Randomized priority queues for fast parallel access[END_REF]. The algorithm is designed for asynchronous shared memory model with n processes. Each process has its own queue which is split into two parts: a sorted array and a heap.

An insertion of c ≤ n values randomly chooses c out of n local queues and inserts each value into the array of the chosen queue. Once in log n insertions to a local queue the values from the array are flushed into the heap. By that the size of the array in any local queue does not exceed log n.

The algorithm to apply a batch of c ≤ n extractMin operations is more involved. At first, each process operates on its own local queue: move the smallest values one-by-one from the heap to the array until all arrays of local queues together contain c smallest values (they can also contain other values). Then the smallest c values from the union of arrays are found using the algorithm similar to quicksort and, finally, these values are returned.

An application of a batch of c ≤ n insert operations takes O(c • log m) work and O(log m) span, while an application of a batch of c ≤ n extractMin operations takes O(n • log m) work and O(log m • log n) span, where m is the size of the priority queue.

Concurrent Implementations

Lock-Based Heap by Hunt et al.

We start with the coarse-grained heap-based priority queue by Hunt et al. [START_REF] Hunt | An efficient algorithm for concurrent priority queue heaps[END_REF]. They used an algorithm similar to the sequential binary heap that we described earlier. At first, an operation takes a global lock, changes the number of nodes m, calculates m as a bit-reverse of m (the least significant becomes the most significant and etc.), locks node m and, if the operation is extractMin, also locks root, and, finally, releases the global lock.

Insert writes a value into a[ m] and sifts up from m. To guarantee correctness the operation uses hand-over-hand locking: takes the lock on the parent, then, possibly, swaps values, releases the lock on the node and continues with the parent.

ExtractMin swaps a[1] with a[ m], unlocks node m and sifts down. To guarantee correctness the operation uses hand-over-hand locking: takes a lock on both children, then, possibly, swaps values with one of them, releases the lock on the node and continues with the proper child.

Priority Queue from Skip-Lists

Typically, all concurrent skip-lists algorithms can be transformed into concurrent priority queue implementations in a manner firstly proposed by Lotan and Shavit [START_REF] Shavit | Skiplist-based concurrent priority queues[END_REF]. They took the concurrent skip-list by Pugh [START_REF] Pugh | Concurrent maintenance of skip lists[END_REF] and augmented it with a findMin function (a subfunction of extractMin) that simply goes through the bottom level searching for a non-marked node. When such node is found the operation performs its deletion.

It is argued that such an implementation is not linearizable, but quiescently consistent: suppose that the skip-list contains only one value 2 and the first process performing findMin is currently at that node; then the second process inserts 1, inserts 3, and marks 2 for deletion; the first process wakes up and finds 3 which never was a minimum. To ensure the linearizability they added the timestamp for each operation and findMin function only searches for the nodes that were inserted before the extractMin operation starts.

Next, Sundell and Tsigas proposed a lock-free priority queue [START_REF] Sundell | Fast and lock-free concurrent priority queues for multi-thread systems[END_REF] which is based on their lock-free skip-list [START_REF] Sundell | Scalable and lock-free concurrent dictionaries[END_REF]. Finally, the lazy lock-based skip-list by Herlihy [START_REF] Herlihy | A simple optimistic skiplist algorithm[END_REF] and the lock-free skip-list by Herlihy [START_REF] Herlihy | A lock-free concurrent skiplist with wait-free search[END_REF] were transformed into priority queues in [START_REF] Herlihy | The art of multiprocessor programming[END_REF]. We name them Lazy and SkipQueue, correspondingly.

Lock-Free Priority Queue by Johnson and Linden

The concurrent priority queues obtained from concurrent skip-lists in a manner described above are subject to severe contention: several processes can try to mark the same node at the same time, where the losers proceed together trying to mark the next node and so on; plus the nodes to be physically removed are likely to be neighbours and, consequently, are likely to share predecessors, thus, leading to contention.

Johnson and Linden addressed the issue with contention during physical removals and presented a lock-free priority queue [START_REF] Lindén | A skiplist-based concurrent priority queue with minimal memory contention[END_REF]. A key idea is: instead of physically removing each logically deleted node separately, we remove them in batches, i.e., several at a time. As a basis for priority queue they take a skip-list similar to the lock-free skip-list by Herlihy [START_REF] Herlihy | A lock-free concurrent skiplist with wait-free search[END_REF]. As usual the nodes can be logically and physically deleted. The major distinction from the previous skip-list algorithms, is that the logically-deletion flag is stored not in the next pointer of the node, but in the next pointer of its predecessor. By that the algorithm can guarantee that logically deleted nodes represent a prefix of the skip-list.

Given a value v an insertion starts with a traversal that finds, at each level i, the first not logically deleted node succs[i] with the value bigger than v and its predecessor preds[i]. Since logically deleted nodes always form the prefix, this traversal does not differ much from the standard one. Then the insertion creates a new node and inserts it starting from the bottom: at each level i, inserts between preds[i] and succs[i] using compare&swap, if unsuccessful then restarts the traversal to get new predecessors.

An extraction of minimum iterates through the lower level in order to mark the first not logically deleted node. When found, the node is marked and the number of traversed nodes is compared to the threshold. If it exceeds some predefined threshold, the next pointers from the head node are changed in order to unlink the prefix of logically deleted nodes.

Lock-Based Priority Queue with Elimination and Combining by Calciu et al.

The last data structure still suffers from the contention during the marking of the nodes. Calciu et al. [START_REF] Calciu | The adaptive priority queue with elimination and combining[END_REF] tried to resolve this issue by splitting the queue into two parts: sequential and concurrent skip-lists. The first skip-list contains values smaller than a threshold and is accessed through a request buffer. The requests from the buffer are performed sequentially by a dedicated server thread. The second skip-list stores all other values and is instantiated as some concurrent skip-list.

Insert decides in which skip-list to insert: if a value is bigger than the threshold it is inserted into the concurrent skip-list, otherwise, the operation lefts a request in the buffer to the sequential skip-list. ExtractMin simply publishes a request in the buffer to the sequential skip-list. After an operation is performed the threshold can be adjusted in order to improve performance: the sequential skip-list should be not very small and also not very big.

Lock-Free Cache-Friendly Priority Queue by Braginsky et al. Braginsky et al. [36] choose some lock-free skip-list implementation and make a node contain not one value but an array of values with the size of one cache line. The skip-list is logically split into two parts: the first node, which array is sorted, and the others.

Insert a node curr in the skip-list that should contain the argument and tries to insert it. There are two cases: curr is the first node or not. In the first case, the operation adds its request into a special buffer associated with the first node. Then the operation freezes curr, creates a new node with the sorted values from curr and the buffer, and tries to replace curr with a new node in the skip-list. In the second case Insert appends the value to the array in curr using one fetch&add instruction. In both cases, if the array becomes full the operation freezes node curr to split it into two. The freeze procedure is implemented in such a manner that other operations working on curr help to unfreeze it.

ExtractMin tries to take the first element from the first node using one fetch&add instruction. If the first node becomes empty, the operation freezes the first node and the second node in order to sort values in the second node and move them to the first node.

Lock-Based and Lock-Free Priority Queues by Liu and Spear

We finish the overview of concurrent priority queues with Mounds by Liu and Spear [START_REF] Liu | Mounds: Array-based concurrent priority queues[END_REF]. This priority queue is represented by a binary tree with a sorted list of values in each node and extended heap property: the smallest value at a node is less than or equal to the smallest values in the children (note that this extended property differs from the one appeared before).

Insert(v) starts with several attempts to find a node with the first value bigger than v: take a random leaf and find the highest node on the path to root that has the first value bigger than v using binary search. If such node is not found, the binary tree is extended with a new level of nodes and the operation restarts. The operation simply finishes by inserting v at the beginning of the list of the found node.

ExtractMin operation removes the first value from the list in the root and marks the root as dirty. All dirty nodes have to be moundified, or in other words they have to be sift down that compares the first values of the lists.

Two implementations of this data structure were described: lock-based and lock-free. Lock-based implementation simply uses hand-over-hand locking as the priority queue by Hunt et al. In lock-free implementation operations help each other to moundify nodes they are working on.

Automatic Oracle-Guided Granularity Control

Introduction

At the very beginning of the era of multicore processors parallel programs were written for static multithreading: each process is provided with its own program. These programs are written as a composition of supersteps. During a superstep, each process performs independent local computations. After each superstep, the processes synchronizae to accumulate the results of their independent computations. When the number of processes exceeds the number of cores, the programmer has to think about low-level execution details, such as scheduling the processes onto cores and controlling the overhead spent on the synchronization between processes. The intricacy of these low-level details has motivated interest in dynamic multithreading (also known as implicit parallelism). Dynamic multithreading seeks to make parallel programming easier by delegating the tedious details, such as tasks scheduling, to compiler or run-time system. Much work has been done in that area resulting in various implementations: OpenMP [START_REF]Openmp application interface[END_REF], Cilk [START_REF] Frigo | The implementation of the cilk-5 multithreaded language[END_REF], Fork/Join Java [START_REF]Java Fork-Join[END_REF], Habanero Java [START_REF] Budimlić | The design and implementation of the habanerojava parallel programming language[END_REF], X10 [START_REF] Charles | X10: an objectoriented approach to non-uniform cluster computing[END_REF], Intel TBB [START_REF]Intel. Intel threading building blocks[END_REF], NESL [START_REF] Guy | Nesl: A nested data-parallel language[END_REF], TPL [112], parallel ML [START_REF] Jagannathan | The design rationale for multi-mlton[END_REF] and parallel Haskell [START_REF] Manuel Mt Chakravarty | Data parallel haskell: a status report[END_REF].

Most of these implementations provide the opportunity for parallelism via lightweight language constructs. In this work, we will use Cilk Plus [START_REF]Intel. Intel cilk plus manual[END_REF] runtime system that provides just two keywords: spawn and sync. Spawn indicates a computation that can be executed in parallel and sync indicates a computation that must be synchronized with. These keywords are enough to express common parallel constructs such as nested parallelism and parallel loop [START_REF] Michael D Mccool | Structured parallel programming: patterns for efficient computation[END_REF].

As a classic application of implicit parallelism, consider a map function that takes an array a and a function f , and returns an array b with b i = f (a i ). Using templates (a powerful feature of C++) our map implementation (Figure 4.1) enables different forms of mapping. For example, a can be an array of integers and f can be an increment. Alternatively, a can be an array of vectors from R n and f can be a "dot product" with vector specified beforehand. As can be seen, templates simplify a life of the programmer: it is unnecessary to write the separate code for each type of a and f . However, as discussed further this unification introduces a new obstacle on the path to good performance.

Even though implicit parallelism allows writing high-level code, it is non-trivial to make the code perform well. During the execution of a parallel program the run-time system induces overheads related to creation and scheduling of tasks. These overheads can be large enough to wipe out benefits of parallelism. For example, the simple map example above could run as much as 10-100x slower than an optimized implementation (Section 4.2).

There exists two complementary types of approaches to reduce the overheads of parallelism. An approach of the first type (e.g., [START_REF] Kumar | Work-stealing without the baggage[END_REF]) optimizes the run-time system itself, for example, by reducing overheads spent per task creation, and does not require any changes to the code. An approach of the second type (e.g., [START_REF]Intel. Intel threading building blocks[END_REF]) controls granularity. Granularity control requires the programmer to tune the code in order to ensure that only big enough computations are parallelised and for small and only small computations an alternative sequential algorithm is executed. By that, the total overhead spent by run-time system takes a small fraction of the total execution time.

As an example of granularity control, imagine our map function that takes an integer array as a and an increment function as f . To perform this operation efficiently, the pro- grammer should group operations on elements into blocks that are executed sequentially and that are large enough to amortize the cost of parallelism. Figure 4.2 shows such a tuned implementation of map, where GRAIN_SIZE determines the block size. The only thing left unspecified is how to choose the setting for grain-size constant GRAIN_SIZE. While it may appear simple, this process turns out to be a challenge, because the optimal setting for GRAIN_SIZE depends on the the architecture as well as the specific inputs [START_REF] Bergstrom | Lazy tree splitting[END_REF][START_REF] Sk Debray | A methodology for granularity-based control of parallelism in logic programs[END_REF][START_REF] Feeley | A message passing implementation of lazy task creation[END_REF][START_REF] Hiraishi | Backtrackingbased load balancing[END_REF][START_REF] Sanchez | Flexible architectural support for fine-grain scheduling[END_REF][START_REF] Tzannes | Lazy binary-splitting: a run-time adaptive work-stealing scheduler[END_REF]. The programmer has to perform the tuning process, i.e., repeatedly execute the program with different inputs so that the right setting can be found [START_REF]Intel. Intel threading building blocks[END_REF][START_REF] Tzannes | Lazy binary-splitting: a run-time adaptive work-stealing scheduler[END_REF].

[i] ← f(a[i]) return mid ← (l + r) / 2 spawn [&] { map(f, a, b, l, mid) } [&] { map(f, a, b, mid, r) } sync return
In simple parallel programs, such as map discussed above, grain-size constants are independent from each other and can be tuned separately. However, when programs become more complicated, for example, by using the nested parallelism: the grain-size constants that correspond to recursive calls become dependent. The dependency between the constants greatly complicates the tuning process.

Furthermore, for generic code, such as map, it can be impossible to perform such tuning, because grain-size constants depend on the template parameters as well as the actual arguments.

These issues emerge an interest in an automatic algorithms that help the programmer to control granularity [START_REF] Sk Debray | A methodology for granularity-based control of parallelism in logic programs[END_REF][START_REF] Lorenz Huelsbergen | Using the run-time sizes of data structures to guide parallel-thread creation[END_REF][START_REF] Joseph | Low-cost process creation and dynamic partitioning in qlisp[END_REF][START_REF] Joseph S Weening | Parallel execution of lisp programs[END_REF]. However, all these approaches fall short to deliver strong theoretical and practical efficiency guarantees. A more recent paper on automatic oracle-guided algorithm [START_REF] Umut A Acar | Oracle-guided scheduling for controlling granularity in implicitly parallel languages[END_REF] show that strong bounds can be achieved if we are provided with an oracle that can predict the execution time precisely. The authors present an implementation of such an oracle, but this implementation cannot be used in programs with nested parallelism. This limitation excludes nearly all interesting parallel programs expressible in high-level parallel programs.

In this chapter, we present provably and practically efficient automatic granularity control technique for the class of nested parallel programs. To use our technique the programmer is asked to provide an "asymtotic" cost function of each piece of parallel code. Our novel online algorithm works as follows: (1) uses these cost functions to predict the expected execution time of that code on one process; (2) uses this prediction to decide whether to execute the parallel or sequential version of the code.

We prove that our algorithm has the desired theoretical properties, i.e., the overhead on synchronization is small in comparison to the total execution time, for a broad class of parallel programs. We implement the algorithm as a C++ library that extends Cilk Plus [START_REF]Intel. Intel cilk plus manual[END_REF] and compare it against the hand-tuned code from the PBBS suite [START_REF] Shun | Brief announcement: the problem based benchmark suite[END_REF]. As the results show, our automatic approach to granularity control can eliminate the need for hand-tuning in many cases.

To the best of our knowledge, this is the first result that provides the provable and practically efficient automatic granularity control.

Roadmap

In Section 4.2, we present several challenges of the granularity problem and present the high-level idea behind our solution. In Section 4.3, we present our online algorithm for automatic granularity control. This algorithm is fully online and does not require additional tuning or specific compiler support. Then we present end-to-end tight bounds on the execution time of a parallel algorithm that uses our algorithm. In Section 4.4, we provide the proofs of the bounds. In Section 4.5, we present the results of the evaluation on a broad collection of hand-tuned benchmarks. In Section 4.6, we discuss the related work. We conclude in Section 4.7.

Overview

We present an overview of the challenges of the granularity problem and our proposed solution.

Example Program

Consider a simple, data-mining problem: given an array of elements of type T, find the number of elements that satisfy a given predicate p. Using C++ templates, we specify such a generic function as follows:

template<T, P> match(T* lo, T* hi, P p): int ...

Because match is generic, we can set T to be char, and define the predicate to be a function (a C++ lambda function [START_REF] Stroustrup | The c++ programming language[END_REF]Sec.11.4]) that tests equality of the character to '#' as follows.

p = [&] (char* c) { return *c = '#' }
Similarly, we can perform matches over arrays whose elements are 1024-character strings, by setting T to be char [1024]. For example, we may count the strings whose hash code matches a particular value, say 2017, by instantiating the predicate as follows.

p = [&] (T* x) { return hash(x, x + sizeof(T)) = 2017 }
A classic way to implement a parallel match is to divide the input array into two halves, recursively call on each half, and compute the sum of the number of occurrences from both halves. Figure 4.3 shows the code for such an implementation, where match takes as arguments the input array, specified by a reference a and length l, and a predicate function, p. To control granularity, we stop the recursion when the input contains fewer than grain elements and switch to a fast sequential algorithm, match_seq. When the input array is large, it is divided in half and solved recursively in parallel using fork2join.

Choice of the Grain Size

To ensure good performance, the programmer must choose the setting for grain; but what should this setting be? The challenge is that there is no a priori suitable setting for grain, because the suitability depends on the particular hardware and software environment and, in fact, on the inputs to the function match. Applying the established practice of manual granularity control leads to poor results even on the same machine and with the same software environment. Table 4.1 illustrates the issue. It shows the 40-core run times for different types T and different grain settings. (The experiment is run on an Intel machine described in Section 4.5.) The input size (total number of elements) is chosen to ensure a sequential execution time of a few seconds. When T is the type char, we use character equality as predicate. When T is an array of characters, we compare the hash values, for a standard polynomial hashing function. For each setting of T, we consider various values of grain, including the "recommended" grain value determined by following the process described in Intel's TBB manual [START_REF]Intel. Intel threading building blocks[END_REF]: start by setting the grain to the value 10, 000 and halve it until the 1-processor run-time stops decreasing by more than 10%. Such tuning maximizes the exposed parallelism by considering the smallest grain value for which the overheads are not prohibitive.

At first, observe that the TBB-recommended value of grain changes for different settings of T. For char it is 5000; for char [START_REF] Fatourou | A highly-efficient wait-free universal construction[END_REF] it is 10; for char[2048] it is 1. Secondly, observe that a grain optimal in one setting may induce a very significant slowdown in a different setting. For example, when T is char, setting the grain to 1 instead of 5000 results in a 100-fold slowdown. Thus, we conclude that, there is no one value of grain that works well for all instances of match.

One might attempt to select the grain size based on the arguments provided to the match function. For example, the grain could be set to C/sizeof(T), for some constant C, to ensure use of a smaller grain size when processing bigger elements. This approach helps in some cases, but it does not solve the problem in general: note that in match, the grain depends not only on the type T, but also on the predicate passed as the second argument. If T is set to char [START_REF] Fatourou | A highly-efficient wait-free universal construction[END_REF] and the predicate is instantiated as a standard polynomial hash function, the optimal grain size is 10; however, providing a different, more computationally expensive predicate function, for example, the number of different substrings, causes the optimal grain size to be 1. Selecting the right grain for different predicates would require the ability to predict the execution time of a function, an intractable problem.

To control granularity in cases of nested-parallel programs, the programmer will likely have to specialize the code for each predicate and apply granularity control to each such specialization, thus losing the key benefits of generic functions. The problems illustrated by the simple example above are neither carefully chosen ones nor isolated cases. They are common; more realistic benchmarks exhibit even more complex behavior. In fact, as discussed in more detail in Section 4.5, in the state of the art PBBS benchmarking [START_REF] Shun | Brief announcement: the problem based benchmark suite[END_REF], nearly every benchmark relies on carefully written, custom granularity control techniques.

Our Approach

Our goal is to delegate the task of granularity control to a smart library implementation. To this end, we ask the programmer to provide for each parallel function a series-parallel guard, by using the keyword spguard. A spguard consists of: a parallel body, which is a lambda function that performs a programmer-specified parallel computation; a sequential body, which is a lambda function that performs a purely sequential computation equivalent to the parallel body, i.e. performing the same side-effects and delivering the same result; and, a cost function, which gives an abstract measure, as a positive number, of the work (run-time cost) that would be performed by the sequential body.

At a high level, a spguard exploits the result of the cost function to determine whether the computation involved is small enough to be executed sequentially, i.e., without attempting to spawn any subcomputation. If so, the spguard executes the sequential body.

Otherwise, it executes the parallel body, which would typically spawn smaller subcomputations, each of them being similarly guarded by a spguard.

The cost function may be any programmer-specified piece of code that, given the context, computes a value in proportion to the one-processor execution time of the sequential body. Typically, the cost function depends on the arguments provided to the current function call. A good choice for the cost function is the average asymptotic complexity of the sequential body, e.g., n lg n, or n, or √ n, where n denotes the size of the input. The programmer need not worry about constant factors because spguards are able to infer them on-line, with sufficient accuracy.

In a real implementation, the sequential body can be left implicit in many cases, because it can be inferred automatically. For example, the sequential body for a parallel-for loop can be obtained by replacing the parallel-for primitive with a sequential for. Likewise, in many instances, the complexity function is linear, allowing us to set it to the default when not specified. In our library and experiments, we use this approach to reduce dramatically the annotations needed.

Figure 4.4 shows the code for our example match function using spguard. Compared with the original code from Figure 4.3, the only difference is the code being structured as a spguard with three arguments: cost function, parallel body, and sequential body. There, the cost function simply returns the input size, written n, because the sequential body (match_seq) uses a linear-time, sequential matching algorithm.

As we show in this chapter, once a parallel algorithm is modified with the insertion of spguards like in Figure 4.4, the information provided by the cost function suffices for our run-time system to control granularity effectively for all settings of the parameters. As shown in Table 4.1, our oracle-guided version matches the performance achieved by the grain settings recommended by the TBB method, but without any of the manual tuning effort and code modifications.

Algorithmic Granularity Control

Our algorithm aims at sequentializing computations that involve no more than a small amount of work. To quantify this amount, let us introduce the parallelism unit, written κ, to denote the smallest amount of work (as units of time) that would be profitable to parallelize on the host architecture. The value of κ should be just large enough to amortize the cost of creating and managing a single parallel task. On modern computers, this cost can be from hundreds to thousands of cycles. Practical values for κ therefore range between 25 and 500 microseconds.

To see the basic idea behind our algorithm, consider the following simple example, involving a single spguard. Suppose that we have a parallel function f(x) which, given an argument x, performs some computation in divide-and-conquer fashion, by recursively calling itself in parallel, as shown below. Assume the body of this function to be controlled by a spguard, with c(x) denoting the cost function, and g(x) denoting the sequential body, that is, a purely sequential function that computes the same result as f(x).

f(x): spguard( [&] { c(x) }, // cost function [&] { // parallel body if x = 1: ... else: (x1, x2) ← divide(x) r1 ← null r2 ← null spawn [&] { r1 ← f(x1) } [&] { r2 ← f(x2) } sync conquer(r1, r2) }, [&] { g(x) } // sequential body )
Intuitively, we aim at enforcing the following policy: if the result of f(x) can be obtained by evaluating the sequential body g(x) in time less than κ, then g(x) should be used. Under a small number of assumptions detailed further in this chapter, this policy leads to provably efficient granularity control.

Making Predictions

One central question is how to predict whether a call to g(x) would take less than κ units of time. Assume, to begin with, a favorable environment where (1) the hardware is predictable in the sense that the execution time is in proportion to the number of instructions, and (2) the cost function c(x) gives an estimate of the asymptotic number of instructions involved in the execution of g(x). For a given input x, let N denote the value of c(x), and let T denote the execution time of g(x). By definition of "asymptotic", there exists a constant C such that: T ≈ C • N . Our algorithm aims at computing C by sampling executions, and then it exploits this constant to predict whether particular calls to g(•) take less than κ units of time.

More precisely, for an input x_i being evaluated sequentially, that is, through a call to g(x_i), we may measure the execution time of g(x_i), written T i , and we may compute the value of c(x_i), written N i . From a collection of samples of the form (T i , N i ), we may evaluate the value of the constant C by computing the ratios T i /N i . In reality, the actual value of the constant can vary dramatically depending on the size of the computation, in particular due to cache effects -we will later return to that point. There is a much bigger catch to be addressed first.

In order to decide which computations are safe to execute using the sequential body g(•), our algorithm needs to first know the constant C. Indeed, without a sufficiently accurate estimate of the constant, the algorithm might end up invoking g(•) on a large input, thereby potentially destroying all available parallelism. Yet, at the same time, in order to estimate the constant C, the algorithm needs to measure the execution time of invocations of g(•). Thus, determining the value of C and executing the algorithm g(•) are interdependent. Resolving this critical circular dependency is a key technical challenge.

At a high level, our algorithm progressively sequentializes larger and larger computations. It begins by sequentializing only the base case, and ultimately converges to computations of duration κ. Each time that we sequentialize a computation by calling g(•) instead of f(•), we obtain a new time measure. This measure may be subsequently used to predict that another, slightly larger input may also be processed sequentially. We are careful to increase the input size progressively, in order to always remain on the safe side, making sure that our algorithm never executes sequentially a computation significantly longer than κ.

As our algorithm increases the cost (as measured by the cost function) of sequentialized computations each time at most by a multiplicative factor, called α, it converges after just a logarithmic number of steps. The growth rate, α controls how fast sequentialized computations are allowed to grow. Any α > 1 could be used; values between 1.2 and 3 work well in practice.

Dealing with Nested Parallelism

When dealing with a single spguard, the process described above generally suffices to infer the constant associated with that spguard. However, the process falls short for programs involving nested parallelism, e.g., nested loops or mutually-recursive functions. To see why, consider a function h(•) that consists of a spguard whose parallel body performs some local processing then spawns a number of calls to completely independent functions. Because h(•) is not a recursive function with a base case, the algorithm described so far is unable does not have a chance to follow the convergence process; the spguard of h(•) would have no information whatsoever about its constant, and it would always invoke the parallel body, failing to control granularity.

To address this issue and support the general case of nested parallelism, we introduce an additional mechanism. When executing the parallel body of a spguard, our algorithm computes the sum of the durations of all the pieces of sequential computation involved in the execution of that parallel body. This value gives an upper bound on the time that the sequential body would have taken to execute. This upper bound enables deriving an over-approximation of the constant. Our algorithm uses this mechanism to make safe sequentialization decisions, i.e., to sequentialize computations that certainly require less than κ time. By measuring the duration of such sequential runs, our algorithm is then able to refine its estimate of the constant. It may subsequently sequentialize computations of size closer to κ.

Overall, our algorithm still progressively sequentializes larger and larger subcomputations, only it is able to do so by traversing distinct spguards with different constant factors.

Dealing with Real Hardware

Our discussion so far assumes that execution times may be predicted by the relationship T ≈ C • N . But in reality, this assumption is not the case. The reason is that the ratios T /N may significantly depend on the input size. For example, we observed in several programs that processing an input that does not fit into the L3 cache may take up to 10 times longer to execute than a just slightly smaller input that fits in the cache. In fact, even two calls to the same function on the same input may have measured time several folds apart, for example, if the first call needs to load the data into the cache but not the second one.

We design our algorithm to be robust in the face of large variations of the execution times typical of modern hardware. In addition to validating empirically that our algorithm behaves well on hundreds of runs of our full benchmark suite, we formally prove its robustness property. To that end, we consider a relatively realistic model that takes into account the variability of execution times typical of current hardware. More precisely, we develop our theory with respect to an abstract notion of "work", which we connect both to runtime measures and to results of costs functions, as described next.

At first, we assume that runtime measures may vary by no more than a multiplicative factor E from the work, in either direction. Second, we require that, for the program and the growth rate α considered, there exists a value β such that, for any cost function involved in the program, the following property holds: if the cost function applied to an input J returns a value no more than α bigger than for input I, then the work associated with input J is at most β times bigger than the work for input I. Intuitively, this property ensures that the work increases with the cost, but not exceeding a maximal rate of increase. In practice, for α ≤ 2, we typically observe β ≤ 10. The value 10 typically corresponds to the maximal slowdown occurring when the input data exceeds the size of the cache. Note that the algorithm does not require knowledge of β; it is only involved in the analysis.

Analysis Summary

Our algorithm relies on several important assumptions on programs. These assumptions, whose formal statements may be found in Analysis Section 4.4.1, are matched by a large class of realistic parallel algorithms. At first, our algorithm assumes that, for each spguard, the sequential body evaluates no slower than the corresponding parallel body on a single process. (The sequential body might always be obtained by omitting spawns in the parallel body.) But at the same time, the sequential body should not run arbitrarily faster than the parallel body executed with all its inner spguards being sequentialized. This assumption is required to know that it is safe to exploit the execution time of the parallel body to over-approximate that of the sequential body.

Second, our algorithm assumes that spguards are called regularly enough through the call tree. Without this assumption, the program could have a spguard called on an input that takes much longer than κ to process, with the immediately nested spguard called on an input that takes much less than κ, leaving no opportunity for our algorithm to sequentialize computations of duration close to κ.

Last, our algorithm assumes some balance between the branches of a fork-join. Without this assumption, the program could be a right-leaning tree, with all left branches containing tiny computations. Such an ill-balanced program is a poorly-designed parallel program that would be slow in any case.

As we prove through a careful analysis detailed in Section 4.4, under these assumptions, our algorithm is efficient. Our bound generalizes Brent's Theorem 2.2.1 (T P ≤ W P + S), by taking into account the overheads of thread creation (cost of spawning and managing threads) and granularity control (including the cost of evaluating cost functions, which are assumed to be executed in constant time). A simplified statement of the bound, using big-O notations, appears next. Theorem 4.3.1. Under the above assumptions, with parallelism unit κ, the runtime on P processes using any greedy scheduler of a program involving work w and span s is bounded by:

T P ≤ 1 + O(1) κ • w P + O(κ) • s + O log 2 κ .
The most important term in this bound is the first term: the overheads impact the work term w by only a small factor O(1)/κ, which can be reduced by controlling κ. The second term states that doing so increases the span by a small factor O(κ), and that the granularity control comes at a small logarithmic overhead O(log 2 κ), which is due to our granularity control algorithm. We note that while our theorem is stated for simplicity in terms of greedy schedulers, it could easily be adapted to other schedulers, e.g., to work-stealing schedulers using Theorem 2.2.2.

High-Level Pseudo-Code for the Estimator and Spguard

In what follows, we present pseudo-code for the core of our algorithm. For each syntactic instance of a spguard, we require a unique estimator data structure to store the information required to estimate the corresponding constant factor. (In the case of templated code, we instrument the template system to ensure that each template instantiation allocates one independent estimator data structure.) The estimator data structure appears on the left of Figure 4.5. It maintains only two variables: the variable C, a running estimate of the constant, and Nmax, which tracks the maximum (abstract) work sampled thus far.

The function report provides the estimator with samples. It takes as argument T, the execution time and N, the abstract work. If T is less than the parallelism threshold κ and N is greater than Nmax, then C is atomically updated as T/N and Nmax is updated to N. The function report protects against data races by using an atomic block. Our library implements this atomic block using a single compare-and-swap (CAS) operation, by packing the variables C (represented in single-precision) and Nmax (represented on 32 bits) on a single machine word. In the case of a race, compare-and-swap would fail and our code would try again until it succeeds, or Nmax becomes greater than N.

The function is_small, takes as argument a cost N and returns a boolean indicating whether the time to execute this much work is predicted to be less that α • κ. (Allowing sequentialization of computations of size slightly larger than κ is needed to obtain proper estimates near κ.) If N is smaller than Nmax, then the function returns true. Intuitively, this computation is smaller than a previously-sampled computation that took less than κ time. If N exceeds Nmax, but no more than by a factor α, the function extrapolates using its most current information: if the product C • N does not exceeds ακ, then it returns true.

The functions fork2join and spguard appear on the right of Figure 4.5. The fork2join function executes its two branches in parallel and resumes after completion of both branches. The function spguard takes an estimator, a parallel body, a sequential body and a cost function that return the abstract work of the sequential body. It begins by computing the abstract cost N for the sequential body and consults the estimator. If the work is predicted to be small, it runs the sequential body, else the parallel body. It relies on a function called measured_run to measure the sum of the duration of the pieces of sequential code executed. This function is not entirely trivial to implement, but can be implemented efficiently with little code as detailed further.

Implementing Time Measurements

Above, for simplicity, we have abstracted an important detail: measuring the work of a sequential or a parallel function. Now, we present a refinement of the algorithm that performs such measurements. The basic strategy is to keep several pieces of information as processor local state to accumulate relevant timer information and combine them carefully to compute the total work of a function. Figure 4.6 shows the full specification, which we describe below.

Time Measures

We assume a function called now() that returns the current time. Importantly, we never need to assume a global clock synchronized between the various processors, since all our time measurements are always local to a processor. We nevertheless need to assume that the clocks on the various processors deliver homogeneous results, that is, that the clocks tick (roughly) at the same pace. The precision of the clock should be sufficient to measure time interval one or two order of magnitude smaller than κ, that is, roughly in the thousands of cycles. In practice, we rely on hardware cycle counters, which are both very precise and very cheap to query.

Sequential Work Time

We define the sequential work time of a computation to be the sum of the durations of all the sequential subcomputations performed over the scope of that computation.

Invariants

Our algorithm maintains some information as processor-local state. More precisely, each processor manipulates two variables, called total and timer. These two variables are used for evaluating the sequential work time associated with the innermost call to the measured_run function, according to the following two invariant. First, the timer variable stores either the time of beginning of the innermost measured_run call, or a point in time posterior to it. Second, the total variable stores the sequential work time that was performed between the timestamp associated with the beginning of the innermost measured_run call and the timestamp stored in the variable timer.

The auxiliary function total_now returns the sequential work time since the beginning of the innermost call to measured_run. It is implemented by computing the sum of the contents of the total variable and the width of the time interval between the timer and the current time.

Remark: when outside of the scope of any measured_run call, the two processor local variables keep track of the sequential work time since the beginning of the program. Our code never exploits such values.

Transitions

When entering the scope of a new call to measured_run (Line 7), the variable total is set to zero, and the variable timer is set to the current time. When exiting the scope of a measured_run call (Line 11), the auxiliary function total_now is used to compute the total sequential work time over this call.

When entering the scope of a new call to measured_run, it is essential to save the relevant information associated with the current (immediately outer) call, otherwise this valuable information would get lost. The variable t_before serves this purpose, by saving the sequential work time performed so far in the current call, just before entering the scope of the new call (Lines 15 and 33). When subsequently leaving the scope of this new call, we restore the invariant by setting total to t_before plus the time spent during the new call (Lines 21 and 36), and by setting timer to the current time (Lines 22 and 37).

In case the innermost measured_run call executes a fork-join (Line 14), the value of the local variable t_before is captured by the join continuation. In technical terms, the value of t_before is part of the call frame associated with the join thread that executes after the sync. This join thread could be executed on a different processor than the one that initiated the spawn, but such possibility does not harm the correctness of our algorithm. Regardless of potential thread migration, we correctly set the timer and the total immediately after the sync (Lines 21 and 22). More precisely, the processor that executes the join continuation sets its total variable to be the sum of the sequential work time performed before the spawn, plus that performed in each of the two branches (which may execute on different processors), and it resets its timer variable to the current time.

In the simple case of a spguard executing its sequential body (Lines 28-31), our algorithm does not bother calling measured_run. Instead, it directly measures the time before and after the sequential body, and then compute the difference between the two values. This simpler scheme applies because the sequential body involves no spguard nor any fork-join call.

Programming Interface

We highlight two interesting features of our C++ implementation. At first, our implementation supports spguard without an explicitly-provided sequential body. In this case, the parallel body is executed, with all spawns processed sequentially. Second, our implementation supports defining higher-level abstractions on top of fork2join, such as parallel-for, map, reduce, map_reduce, scan, filter, etc. For these abstractions, the programmer may either indicate a custom cost function or simply rely on the default one, which assumes a constant-time processing for each item or iteration. This default mechanism saves a large number of cost functions.

For example, we implemented a more concise version of the match function in Figure 4.4 using map_reduce, as follows.

Analysis

Definitions and Assumptions

Work and Span

To take into account the actual overheads of parallelism that granularity control aims to amortize, our analysis accounts for the cost of parallel task creation, written by τ , as well as the cost of evaluating cost functions, written by φ. Although these costs may vary in practice, we assume τ and φ to be upper bounds for them. In particular, we assume cost functions to evaluate in constant time, and we exploit the fact that our algorithm makes predictions and handles time measurements in constant time for each spguard call. For the latter, we make the simplifying assumption that the resolution of data races during reports in an estimator incurs no more than a fixed cost. (In practice, CAS-conflicts are quite rare in our experiments; a more refined cost model taking contention into account would be required to account for the cost of CAS conflicts.)

Our analysis establishes bounds on the work and on the span, including the overheads of parallelism, of the execution of a program under the guidance of our automatic granularity control algorithm. We name these entities total work and total span. Our bounds on the total work and span are expressed with respect to the work and span of the erasure version of that program, in which all spguards are replaced with their parallel bodies. The erasure of a program can be viewed as a program that is not granularity controlled at all, but instead exposes all available parallelism.

We define the work and span of the erasure program, written w and s respectively, as the work and the span of its erasure, where work and span are defined in the standard manner [START_REF] Umut | Algorithm Design: Theory and Practice[END_REF]. In short, the work of two expressions composed sequentially is the sum of the work of the two expressions; the work of two expressions composed in parallel is the sum of the work of the two plus one. The span of two expressions composed sequentially is the sum of the spans of the two; the span of two expressions composed in parallel is the maximum of the spans of the two plus one. Note that to avoid ambiguity we call w and s the raw work and the raw span, respectively.

We define the total work, written W, and the total span, written S, to measure the actual work and span of an execution (rather than a program), accounting also for the cost of parallelism and granularity control. More specifically, total work and total span include the overheads: each fork2join is assumed to incur an extra cost τ (covering in particular the cost of spawning threads and dealing with their scheduling), and each estimator operation (including the evaluation of the cost function, the call to is_small and to report, and the time measurements) is assumed to incur an extra cost φ. When computing total work and span, we compute work/span of a spguard based on the branch that it took: if the sequential branch is taken, then the total work/span of the spguard is the total work/span of the sequential branch plus φ. Otherwise, if the parallel branch is taken, then the total work/span of the spguard is the total work/span of the parallel branch plus "τ + φ".

Finally, for the purpose of stating the assumptions of our theorems, we define the sequential work, written W s (t, I), as the work of a piece of code t executed on input I when this term executes fully sequentially, that is, where all spguards are forced to execute the sequential body. For such a sequential execution, no parallelism is created and no cost function is evaluated, thus, the sequential work is equal both to the work and to the total work. S on input I, its measured time, written M (S, I), satisfies:

M (S, I) W s (S, I) ∈ [ 1 E , E].

Well-Defined Spguards

We say that a spguard is well-defined if the relative sequential work cost of the sequential body and the parallel body of each spguard can be lower and upper bounded by a constant, i.e., are asymptotically the same. Specifically, consider a spguard g, with a cost function F , sequential body S, and parallel body B, executed on some input I. Let W s (S, I) denote the sequential work of the sequential body of this call. Let W s (B, I) denote the sequential work of parallel body of this call, that is, the sequential execution time of the parallel body when all spguards choose their sequential body. The spguard is well-defined if there exists a constant D such that:

1 ≤ W s (B, I) W s (S, I) ≤ D.
For the analysis, we consider programs where each spguard is well-defined with respect to the same constant D.

The lower bounds assert that the sequential body induces no more work than the parallel body. This is justified because the parallel body can always serve as a sequential body by replacing fork2join calls with function calls. Note that without this assumption, the sequentialization of subcomputations can increase the total work making it impossible to control granularity.

The upper bound asserts that the sequential body cannot be arbitrarily faster than executing the parallel body sequentially. This requires the parallel body to use spguards to fall back to a sequential body without doing too much work. This is not difficult to achieve for many nested-parallel algorithms, because they present an opportunity to use a sequential body at each nest level, which usually corresponds to a recursive call. Without this assumption, our algorithm is not guaranteed to converge to the desired constant factor due to the gap between the parallel and sequential bodies.

Accurate Cost Functions

We make several accuracy assumptions on cost functions. At first, as mentioned previously, we assume that cost functions are evaluated in constant time, and that this amount of time does not exceed φ.

Second, we assume that the work increases with the cost. Formally, we assume that, for any spguard g and associated cost function F , and for any pair of input I and J such that F (I) ≤ F (J), we have: W s (g, I) ≤ W s (g, J).

Finally, we assume that the work increases with the cost no faster than at some maximal rate (recall Section 4.3). Formally, we require that, for the program and the value considered for the parameter α, there exists a value β such that, for any guard g and associated cost function F involved in the program, and for any pair of inputs I and J such that

F (I) ≤ α • F (J), we have: W s (g, I) ≤ β • W s (g, J).

Syntactic Forms of Spguards

For the analysis, we assume that we are given a program written by using the two parallelism primitives that we offer: fork2join and spguards (described in Section 4.3). To facilitate and simplify the analysis, we make two syntactic assumptions about programs. These assumptions are not necessary for the algorithm or our implementation but are used purely to facilitate analysis.

First, we assume that the parallel body of a spguard consists of fork-join call surrounded by two pieces of sequential code, one to split the input and one to merge the output. In C++ syntax:

spguard(F , [&]{ S p ;fork2join(L,R);S m }, S).
This assumption causes no loss of generality because more complex expressions can be guarded by sequencing and nesting spguards.

Second, we treat the body of spguards as functions that operate on some input. More precisely, an execution of the guard above requires some input I and proceeds by first executing F (I) to compute the cost, and then running with I either the sequential body S(I) or the parallel body based on the outcome of the estimator as described in Section 4.3. The parallel body is of the form:

S p (I); fork2join(L(I L ),R(I R )); S m (I m )
where I L and I R are the inputs to branches and I m is the input to S m . The inputs I L and I R are obtained after processing of the input I by S p , and I m is an output produced by branches L and R.

Regularity of Forks

As explained in Section 4.3, to allow efficient granularity control, we need to rule out ill-balanced programs. We assume the existence of a constant γ (with γ ≥ 1), called the regularity factor, satisfying the following requirements.

• When considering a fork-join, there must be at least some balance between the left and the right branch. Formally, for any execution on input I of a spguard with sequential body S and branches L and R, we assume:

W s (L, I) W s (S, I) and W s (R, I) W s (S, I) ∈ 1 γ , 1 - 1 γ .
Without this assumption, the program can be a right-leaning tree, with all left branches containing only a tiny subcomputation; in such a program the overheads of forks cannot be amortized.

• spguards must be called sufficiently frequently in the call tree. Formally, for any call to a spguard that has sequential body S and executes on input I, if the immediate outer spguard call has a sequential body S and executes on input I , we assume:

W s (S, I) W s (S , I ) ≥ 1 γ .
Without this assumption, the program can have a spguard called on an input that takes much longer than κ to execute, with the immediate inner spguard called on an input that takes much less than κ, leaving no opportunity to sequentialize a subcomputation that takes approximately time κ to execute.

• For an outermost call to a spguard, we need to assume that it involves a nontrivial amount of work. Formally, if an outermost call to a spguard with sequential body S executes on input I, we assume W s (S, I) ≥ κ. Without this assumption, the program can consist of a sequential loop that iterates calls to a spguard on tiny input; and, again, the overheads would not be amortized. (Note that, technically, the requirement W s (S, I) ≥ κ γDE would suffice.)

Results Overview

To establish a bound on the parallel execution time of our programs, we first bound the total span and total work. We then derive a bound on the parallel execution time under a greedy scheduler. For the bounds, we assume programs that are 1) γ-regular, 2) where spguards are well-defined, and 3) where all cost functions are accurate. We express our bounds with respect to work and span, which do not account for the overheads as well as the parameters of the analysis that account for various overheads and factors. The span may grow by a multiplicative factor, but no more. As our final bound will show, for a program with sufficient parallelism (i.e., with w s P ), this increase in the span has no visible impact on the parallel run time.

W ≤ 1 + τ + 2φ κ • w + P F GH • (τ + 2γφ).
The first component of the left hand side asserts that the work grows by no more than a multiplicative factor 1 + , where may be tamed to a couple of percents by the choice of a sufficiently large κ. The second component asserts that the total overheads associated with the sequentialization of tiny subcomputations during the convergence phase of the estimators are bounded by some constant cost, proportional to the number of estimators and to the product

P • F • H, which is P • O(log 2 κ).
To bound the parallel run time, we exploit Brent's Theorem 2.2.1, which asserts that, for any greedy scheduler, T p ≤ W P + S, where P stands for the number of processes. To simplify the final statement, we make some over-approximation, and we also exploit several inequalities that are always satisfied in practice. We assume κ ≥ τ and κ ≥ 1 + φ, since in practice the user always sets κ max(τ, φ) to ensure small overheads.

Theorem 4.4.3 (Bound on the parallel run time). Let P denote the number of processes and G denote the number of spguards.

T P ≤ 1 + γED • (τ + 2φ) κ • w P + (Eβ + 1) • κ • s + O G • log 2 κ • (τ + 2γφ) .
Theorem 4.3.1 (Simplified bound on the parallel run time). For fixed hardware and any program, all parameters of the analysis except for κ (the unit of parallelism) can be replaced with constants, leaving us with the following bound:

T P ≤ 1 + O(1) κ w P + O(κ) • s + O log 2 κ .
The most important term in this bound is the first term that says that the overhead of various practical factors impact the work term w by only a small factor O(1)/κ, which can be reduced by controlling κ. The second term states that doing so increases the span by a small factor O(κ) and that all of this comes at a small logarithmic overhead, which is due to our granularity control algorithm O(log 2 κ).

Our theorem establishes that a non-granularity controlled nested parallel program with work w and span s can be guaranteed to be executed fast on a real machine using our granularity control algorithm. Our experiments (Section 4.5) show that the analysis appears to be valid in practice.

Additional Definitions

Syntax of Programs

The BNF grammar for programs with spguards is thus as follows. For brevity, we focus on the language constructs that matter to the analysis: sequences, conditionals, and spguards combined with fork-join.

• B ::= a boolean variable • S ::= a purely sequential piece of code, without forks nor guards

• t ::= S | (t; t) | if B then t else t | spguard(S, p, S)
• p ::= S; fork2join(t, t); S

Detailed Definitions of Work and Span

The table below shows the definition of raw work and span, which correspond to the standard definitions used in parallel program analysis, as well as the definition of total work and span, which include the overheads of thread creation, written τ , and the overheads associated with the estimator, written φ. Note that in the definitions below we are ultimately referring to the work of a sequential piece of code, written W (S), which we assume to be defined as the number of instructions involved in the execution of S, or as the number of cycles involved if the considered execution model assigns a cost to each instruction. Definition 4.4.1 (Work and span, raw and total, for each construct).

W (S) W (S) W (S) (t1; t2) W (t1) + W (t2) S(t1) + S(t2) W(t1) + W(t2) S(t1) + S(t2) if B then t1 else t2 when B is true 1 + W (t1) 1 + S(t1) 1 + W(t1) 1 + S(t1) if B then t1 else t2 when B is false 1 + W (t2) 1 + S(t2) 1 + W(t2) 1 + S(t2)
spguard(F , (Sp; fork2join(L, R); Sm), S) when parallel body is chosen

W (Sp) + W (L) + W (R)+ +1 + W (Sm) S(Sp) + max(S(L), S(R))+ +1 + S(Sm) W(Sp) + W(L) + W(R)+ +φ + τ + W(Sm) S(Sp) + max(S(L), S(R))+ +φ + τ + S(Sm) spguard(F , (Sp; fork2join(L, R); Sm), S)
when sequential body is chosen

W (S) W (S) W (S) + φ W (S) + φ
Due to the fact that spguard may dynamically select between the execution of the sequential or the parallel branch, it does not make sense to speak about the work and span of a program. We may only speak about the work and span of a particular execution of the program, that is, of the work and span of an execution trace describing which spguards have been sequentialized and which have not.

Definition 4.4.2 (Execution trace). A particular execution of a source term t on an input

I corresponds to a trace, written X, that describes, for each evaluation of a spguard during the program execution, whether the sequential body or the parallel body is selected by the spguard. Definition 4.4.3 (Work and span, raw and total, of an execution trace). For an execution of a source term t on an input I producing a trace X, we let:

• W (t, I, X) denote the raw work,

• S(t, I, X) denote the raw span,

• W(t, I, X) denote the total work,

• S(t, I, X) denote the total span.

The definitions are obtained by applying the appropriate rules from the previous table.

When the arguments t, I and X are obvious from the context, we write simply W , S, W, and S. Definition 4.4.4 (Sequential work). For a term t and an input I, we define the sequential work, written W s (t, I), as the raw work W (t, I, X), where X is the trace that systematically selects the sequential bodies. Definition 4.4.5 (Work and span). For a term t and an input O, we define:

• the work w(t, I) as the raw work W (t, I, X) where X is the trace that systematically selects parallel bodies.

• the span s(t, I) as the raw span S(t, I, X) where X is the trace that systematically selects parallel bodies.

For a purely sequential subcomputation, the work is equal to the raw work. (It also matches the span since there is no parallelism involved.) We use this fact implicitly in several places through the proofs.

Time Measurements

In addition to the definition already given for the measurement of a sequential execution time, written M (t, I), we need for the analysis to quantify the total sequential work time involved in a parallel execution, written M (t, I, X). (Recall Appendix 4.3.6.) Definition 4.4.6 (Measured time). We let:

• M (t, I) denote the measured time of the sequential execution of the term t on input I.

• M (t, I, X) denote the sum of the measured time of all the pieces of sequential subcomputations involved in the evaluation of the term t on input I according to trace X. The measure thus ignores the overheads of fork-join operations and the overheads associated with our runtime decisions. Note that this is exactly the time we measure in our algorithm. • parallel non-small call (when the spguard executes its parallel body on a non-small call).

Classification of Spguard

• parallel small call (when the spguard executes its parallel body on a small call).

• covered sequential call (when the spguard executes its sequential body and is dominated by a parallel small call; note that a covered sequentiall call is a small call).

• non-covered sequential call (when the spguard executes its sequential body but is not directly dominated by a parallel small call; in this case, it can be dominated by a parallel non-small call, or not dominated at all).

Definition 4.4.11 (Critical parallel calls). A small parallel call to a spguard is said to be a critical if all the calls that it dominates are sequential small calls.

This classification simplifies the presentation of the proof. Any non-small call amortizes the overheads in a standard manner, i.e., provides multiplicative factor to the work and span. So, our major task is to bound the overhead spend during small calls. Our bound is obtained in several steps.

1. We bound the number of critical parallel calls (that is, a parallel small call that dominates only sequential small calls) in Lemma 4.4.7. To that end, we exploit the fact that, after each critical parallel call, the algorithm performs a report that increases the value of Nmax by at least some constant factor.

2. We show that the number of parallel small calls is bounded in terms of the number of critical parallel calls in Lemma 4.4.9. To that end, we observe that each parallel small call features at least one nested critical parallel call, and that, reciprocally, each critical parallel call is nested in at most a logarithmic number of small parallel calls.

3. We bound the number of covered sequential calls in terms of the number of parallel small calls in Lemma 4.4.10. We do so by arguing that each parallel small call can have at most a logarithmic number of nested sequential calls.

4. We independently show that the non-covered sequential calls correspond to non-small calls, so their overheads are properly amortized in Lemma 4.4.11.

All together, we derive the bound on the overheads associated with all small calls in Lemma 4.4.13. In the definitions of the auxiliary constants F and H, we assume that the number of cycles involved in the sequential execution of a spguard always exceeds the number returned by the cost function. The bounds can be easily adapted by adding a constant factors to F and H if this was not the case.

Parameters Involved in the

To bound the total work W, our proof first bounds the total work excluding overheads involved in parallel small calls and covered sequential calls, which we call W , then bounds the excluded overheads, that is, the value of W -W . Definition 4.4.16 (Total work excluding the overheads involved in parallel small calls). We let W denote the subset of the total work W obtained by excluding the overheads (τ and φ) involved in small parallel calls, in the sense that when reaching such a call, we only count the raw work involved in this call, regardless of the decisions involved in the subcomputations. Proof. Immediate from the definitions. Lemma 4.4.2 (Sequential work associated with a spguard). In the particular case where the term t corresponds to some spguard g, the sequential work is that of its sequential body S, which does not contain any spguard. Thus, we have:

Basic Auxiliary Lemmas

W s (g, I) = W s (S, I) = W (S, I, X) = W(S, I, X) -φ
for any trace X.

Lemma 4.4.3 (Relationship between work and measured time of a computation).

For any term t executed on input I according to trace X, we have:

M (t, I, X) W (t, I, X) ∈ [ 1 E , E].
Proof. Recall our hypothesis: M (S,I) W (S,I) ∈ [ 1 E , E] for all sequential computation S. First, we prove that M (t, I, X) ≤ E • W (t, I, X) by induction on the execution tree.

• Case t = S. By assumption on the variability of time measurements, we have:

M (t, I, X) = M (S, I, X) = M (S, I) ≤ E • W s (S, I) = E • W s (t, I). Using the fact W s (S, I) = W (S, I, X) from Lemma 4.4.2 we get M (t, I, X) ≤ E • W (t, I, X).
• Case t = (t 1 , t 2 ). We have:

M (t, I, X) = M (t 1 , I, X)+M (t 2 , I, X) ≤ E•W (t 1 , I, X)+ E • W (t 2 , I, X) = E • W (t, I, X).
• Case t = if B then T 1 else T 2 . Similar to the previous case, after performing the case analysis.

• Case t = spguard(F , (S p ; fork2join(L, R); S m ), S).

-First case: spguard chooses the parallel body. Then,

M (t, I, X) = M (S p , I, X)+ M (L, I, X)+M (R, I, X)+1+M (S m , I, X) ≤ E •W (S p , I, X)+E •W (L, I, X)+ E • W (R, I, X) + 1 + E • W (S m , I, X) ≤ E • (W (S p , I, X) + W (L, I, X) + W (R, I, X) + 1 + W (S m , I, X)) = E • W (t, I, X).
-Second case: spguard chooses the sequential body. Similar to the case t = S.

The second inequality W (t, I, X) ≤ E • M (t, I, X) again can be proved by induction on the execution tree. The proof is identical to the proof of the inequality above with the only difference: M and W should be swapped.

We assumed, for each spguard, that the execution of the sequential body is faster than the sequential execution of the parallel body, thus, when the trace chooses the parallel body the execution should be slower than the corresponding sequential execution. Lemma 4.4.4. Consider a term t with well-defined spguards. For any execution t on input I with trace X, we have:

W s (t, I) ≤ W (t, I, X).
Proof. We prove this by induction on the execution tree.

• Case t = S. We have: W s (S, I) = W (S, I, X).

• Case t = (t 1 , t 2 ). We have:

W s (t, I) = W s (t 1 , I) + W s (t 2 , I) ≤ W (t 1 , I, X) + W (t 2 , I, X) = W (t, I, X).
• Case t = if B then t 1 else t 2 . Similar to the previous case, after performing case analysis.

• Case t = spguard(F , (S p ; fork2join(L, R); S m , S). We distinguish two cases:

-First case: spguard chooses the parallel body. Then, using the property of welldefined spguards, W s (t,

I) ≤ W s (S p , I) + 1 + W s (L, I) + W s (R, I) + W s (S m , I) ≤ W (S p , I, X) + 1 + W (L, I, X) + W (R, I, X) + W (S m , I, X) = W (t, I, X).
-Second case: spguard chooses the sequential body. Then, W s (t,

I) = W s (S, I) = W (S, I, X) = W (t, I, X).
Because work considers full parallelization, it never selects the sequential bodies of spguards, which are assumed to execute faster than parallel bodies on one process. Thus, the work always exceeds the raw work. Lemma 4.4.5. Consider a term t with well-defined spguards. For any execution of t on input I with trace X, we have: w(t, I) ≥ W (t, I, X).

Proof. We prove this by induction on the execution tree.

• Case t = S. We have: w(S, I) = W (S, I, X).

• Case t = (t 1 , t 2 ). We have:

w(t, I) = w(t 1 , I)+w(t 2 , I) ≥ W (t 1 , I, X)+W (t 2 , I, X) = W (t, I, X) • Case t = if B then t 1 else t 2 .
Similar to the previous case, after performing the case analysis.

• Case t = spguard(F , (S p ; fork2join(L, R); S m ), S). We distinguish two cases:

-First case: spguard chooses the parallel body. Then, w(t, I) = w(S p , I)

+ 1 + w(L, I) + w(R, I) + w(S m , I) ≥ W (S p , I, X) + 1 + W (L, I, X) + W (R, I, X) + W (S m , I, X) = W (t, I, X).
-Second case: spguard chooses the sequential body. Since N is non-negative, the condition may only evaluate to true if Nmax is non-zero, indicating that at least one previous report has been stored for this spguard. Let J be the input on which this previous report has been obtained, in other words Nmax = F (J), let X denote the trace of spguards choices, and let M (g, J, X) denote the measured time at this previous report. Since the report was stored, according to the implementation of function report, we know that M (g, J, X) ≤ κ and that C = M (g,J,X) F (J) . By Lemma 4.4.2 on sequential work we know that W s (S, J) = W s (g, J). Lemma 4.4.4 provides us with W s (g, J) ≤ W (g, J, X). And Lemma 4.4.3 on measured time in the execution on input J gives W (g, I, X) ≤ E • M (g, J, X). Thus, the raw work on input J can be bounded as follows:

W s (g, I) ≤ E • β • κ. Proof. At first, observe that W s (g, I) = W s (S,
W s (S, J) = W s (g, J) ≤ W (g, J, X) ≤ E • M (g, J, X).
In what follows, we establish the inequality W s (S, I) ≤ β • W s (S, J). By exploiting that W s (S, J) ≤ E • M (g, J, X) and M (g, J, X) ≤ κ, this inequality allows us to conclude as follows:

W s (g, I) = W s (S, I) ≤ β • W s (S, J) ≤ β • E • M (g, J, X) ≤ E • β • κ.
The desired inequality is deduced from the fact that the boolean condition evaluates to true. Indeed, we know that

N ≤ Nmax is true, or that N ≤ α • Nmax and N • C ≤ α • κ are true.
• In the first case, the condition reformulates to F (I) ≤ F (J). By the property of well-defined spguards, we deduce W s (S, I) ≤ W s (S, J). By exploiting β > 1, we conclude W s (S, I) ≤ β • W s (S, J), as desired.

• In the second case, the condition reformulates to:

F (I) ≤ α • F (J) and F (I) • M F (J) ≤ α • κ.
By the property of well-defined spguards, exploiting the first inequality, we deduce: W s (S, I) ≤ β • W s (S, J), the desired inequality. Theorem 4.4.1 (Bound on the total span). For any execution of a program with welldefined spguards, we have:

S ≤ (1 + φ + max(τ, Eβκ)) • s
Proof. Let ρ be a shorthand for 1 + φ + max(τ, Eβκ). We establish the inequality S ≤ ρ • s by induction on the execution tree.

• Case t = S. The program is sequential, so S = s ≤ ρ • s.

• Case t = (t 1 ; t 2 ). We have:

S = S(t 1 ) + S(t 2 ) ≤ ρ • s(t 1 ) + ρ • s(t 2 ) = ρ • s.
• Case t = if B then t 1 else t 2 . Similar to the previous case, after considering the two cases.

• Case t = spguard(F , (S p ; fork2join(L, R); S m ), S). We have two cases:

-First case: the spguard chooses the parallel body. Then, S = S(S p )+max(S(L), S(R)

)+ φ + τ + S(S m ) ≤ ρ • s(S p ) + max(ρ • s(L), ρ • s(R)) + ρ + ρ • s(S m ) = ρ • (s(S m ) + max(s(L), s(R)) + s(S p ) + 1) = ρ • s.
-Second case: the spguard chooses the sequential body. In this case by Lemma 4.4.6 we know that the sequential work of this spguard call is bounded: W s (g, I) ≤ Eβκ. Thus, using the fact that the span is nonnegative (s ≥ 1), we have:

S = W s (S, I) + φ ≤ Eβκ + φ ≤ ρ ≤ ρ • s.
Lemma 4.4.7 (Bound on the number of critical calls). A given estimator involves no more than P • F critical calls.

Proof. Let us consider the critical call on input I with measured time M by process p.

At first, we prove that M does not exceed κ. From Lemma 4.4.3, we have M W ≤ E. Because the critical call is small, we know that W s (S) ≤ κ DE and that all directly dominated calls are sequentialized, thus

W = W s (S p ) + W s (L) + W s (R) + 1 + W s (S m ). By the property of well-defined spguards W s (S p ) + W s (L) + W s (R) + 1 + W s (S m ) ≤ D • W s (S).
Combining all these facts, we get

M ≤ E•W = E•(W s (S p )+W s (L)+W s (R)+1+W s (S m )) ≤ E • D • W s (S) ≤ E • D • κ DE = κ.
Second, after the report Nmax is at least F (I): either another process updated Nmax to be not less than F (I), or this report successfully updates Nmax and sets it to F (I). This report can be successful since the reported time M does not exceed κ.

Next, we show that the cost of the next critical call by process p increases at least by a factor α. Suppose that the input of the next critical call is J. Our goal is to show F (J) > α • F (I). For the spguard to choose parallel body on J, the boolean condition in is_small

function (F (J) ≤ F (Y ), or ((F (J) ≤ α • F (Y )) and (F (J) • T /F (Y ) ≤ α • κ),
where Y and T are the latest reported input and the measured time, respectively), needs to evaluate to false. Note that F (I) ≤ F (Y ), since after the last report by process p the value of Nmax was at least F (I), and T ≤ κ, since the measure for Y was reported. Thus, at least one of the two following conditions needs to be satisfied:

F (J) > α • F (Y ), or F (J) > F (Y ) and F (J) > ακ•F (Y )
T . The first case directly gives us the desired inequality, since F (Y ) ≥ F (I).

For the second case, exploiting T ≤ κ, we have:

F (J) > α•κ•F (Y ) T ≥ α•κ•F (I) κ = α • F (I). In summary, F (J) > α • F (I).
Since critical calls are small calls, their work cannot exceeed κ DE . By the assumption that the number of cycles exceeds the cost, the value of the cost function cannot exceed κ DE , with κ expressed in cycles. The cost associated with the first critical report is at least 1 unit, and since the value increases by a factor α at least between every two consecutive critical call by the same process, the number of such critical calls cannot exceed P • (1 + log α κ DE ). We denoted 1 + log α κ DE as F . Lemma 4.4.8 (Bound on the number of nested small calls). Small spguard calls may be nested at no more than on depth H. Proof. Consider a set of nested small calls. The outermost call is small, so it involves work at most κ D•E . As we argue next, the ratio between the work of a call directly nested into another one is at least γ γ-1 . Combining the two, we can deduce that the number of nested small calls is bounded by log γ/(γ-1)

κ D•E .
To bound the ratio between two directly nested calls, we proceed as follows. Consider a spguard with sequential body S on input I, directly dominated by a call to a spguard with sequential body S on input I . Assume, without loss of generality, that the inner spguard call occurs in the left branch, call it L, of the outer spguard call. From the γ-regularity assumption, we know that Ws(L ,I ) Ws(S ,I ) ≤ 1 -1 γ . Furthermore, since S executes as a subcomputation of the sequential execution of L , we have: W s (S, I) ≤ W s (L , I ). Combining the two inequalities gives: W s (S, I) ≤ (1 -1 γ ) • W s (S , I ), which can be reformulated as

Ws(S ,I )

Ws(S,I) ≥ γ γ-1 , meaning that the ratio between the two nested calls is at least γ γ-1 .

Lemma 4.4.9 (Bound on the number of parallel small calls). A given spguard involves no more than P • F • H parallel small calls.

Proof. First, we observe that each parallel small call must dominate at least one critical call. Indeed, when following the computation tree, there must be a moment at which we reach a spguard such that all dominated spguards choose sequential bodies, or such that there are no dominated spguards in the body. Thus, we may bound the number of parallel small calls by multiplying the number of critical calls with the number of parallel small calls that dominate it (including the critical call, which is itself a parallel small call). Of course, we may be counting a same parallel call several times, but this over-approximation is good enough to achieve our bound. More precisely, we multiply the bound from Lemma 4.4.8 with the bound from Lemma 4.4.7.

Lemma 4.4.10 (Bound on the number of covered sequential calls). A given spguard involves no more than

P • F • H • (2γ -2) covered sequential calls.
Proof. A covered sequential call is dominated by a parallel small call. The claimed bound follows from the bound of Lemma 4.4.9 and the fact that, for each parallel small call, we can have at most 2γ -2 directly dominated sequential small calls. We next prove this last claim.

Consider a parallel small call on input I to a spguard with sequential body S and branches L and R . Consider a directly dominated call on input I c to a spguard with sequential body S c . By γ-regularity, we have:

W s (S c , I c ) ≥ 1 γ • W s (S , I ).
From the structure of the program, we know:

W s (L ) + W s (R ) ≥ c∈C W s (S c , I c )
, where C is the set of all directly dominated sequential small calls. Also, by the first property of γregular programs:

W s (L ), W s (R ) ≤ (1 -1 γ ) • W s (S , I ), giving us additional inequality W s (L )+W s (R ) ≤ (2-2 γ )•W s (S , I )
. By combining the facts, we get:

|C|• 1 γ •W s (S , I ) ≤ c∈C W s (S c , I c ) ≤ W s (L ) + W s (R ) ≤ (2 -2 γ ) • W s (S , I
). Thus, we deduce that the number of directly dominated sequential small calls |C| does not exceed 2γ -2.

Lemma 4.4.11 (Work involved in a non-covered sequential call). If a call to a spguard g on input I is a non-covered sequential call, then it involves at least some substantial amout of work, in the sense that:

W s (g, I) ≥ κ γDE .

Proof. A call can be a non-covered sequential call for one of two reasons.

• First case: the call is directly dominated by another spguard call. This spguard call is necessarily parallel, and by assumption it is a non-small call (otherwise, the inner call would be covered). This non-small parallel call occurs on some spguard g with sequential body S executed on input I . This call is not small, meaning that W s (S , I ) > κ DE holds. Besides, by the definition of γ-regularity, we have:

Ws(S,I)
Ws(S ,I ) ≥ 1 γ . Combining the two inequalities gives: W s (S, I) > κ γDE . • Second case: the call is not dominated by any other spguard call. Then, by the last assumption from the definition of γ-regularity, we have W s (S) ≥ κ γDE .

Lemma 4.4.12 (Bound on the work excluding overheads during small calls).

W ≤ 1 + DEγ • (τ + 2φ) κ • w Proof. Let us introduce the shorthand κ = κ γDE . The bound is equivalent to W ≤ 1 + 1 κ τ + 2 κ φ • w. Let B(w) = w + (w-κ ) + κ τ + (2w-κ ) + κ φ
, where x + is defined as x if x is non-negative, and 0, otherwise.

We establish the slightly tighter inequality W ≤ B(w), by induction on the execution tree.

• Case t = S. The program is sequential, so W = W = w ≤ B(w).

• Case t = (t 1 ; t 2 ). W = W (t 1 ) + W (t 2 ) ≤ B(w(t 1 )) + B(w(t 2 )) ≤ B(w 1 + w 2 ). The last inequality holds, because B(x) + B(y) ≤ B(x + y) due to the fact (x -k) + + (y -k) + ≤ (x + y -k) + .

• Case t = if B then t 1 else t 2 . Considering two different cases, similar to the previous case.

• Case t = spguard(F , (S p ; fork2join(L, R); S m ), S). Let I be the input and X be the trace for this call. Thereafter, we write W s (S) as short for W s (S, I) ad w as short for w(t, I).

By Lemma 4.4.4, we know that: W s (t, I) ≤ W (t, I, X). By Lemma 4.4.5, we have W (t, I, X) ≤ w. Combining the two gives W s (t) ≤ w.

We then distinguish four cases:

-First case: the call is the parallel non-small call. By definition of small, this means: W s (S) > κ DE . Let us focus first on the left branch. By the definition of γ-regularity, we know Ws(L) Ws(S) ≥ 1 γ . Besides, for the same reason as we have W s (t) ≤ w, we have W s (L) ≤ w(L). Combining these results, we get:

w(L) ≥ 1 γ • W s (S) > κ γDE = κ .
By symmetry, we have w(R) > κ . Also, we know that B(x) + B(y) ≤ B(x + y). Putting everything together gives:

W = W(S p ) + W (L) + W (R) + τ + φ + W(S m ) ≤ B(w(S p )) + B(w(L)) + B(w(R)) + τ + φ + B(w(S m )) = B(w(S p )) + (w(L) + w(R)) + (w(L)-κ )+(w(R)-κ )+κ κ τ + (2w(L)-κ )+(2w(R)-κ )+κ κ φ + B(w(S m )) ≤ B(w(S p )) + B(w(L)+w(R))+B(w(S m )) ≤ B(w(S p )+w(L)+w(R)+w(S m )) = B(w -1) ≤ B(w).
-Second case: the call is the parallel small call. By definition of W , we do not count the overheads within the scope of this call and only count the raw work, thus W ≤ W (t, I, X). Recall that W (t, I, X) ≤ w. Besides, by definition of B, we have: w ≤ B(w). Combining these results gives: W ≤ B(w).

-Third case: the call is the covered sequential call. In this case, the call is dominated by a parallel small call. Thus, such calls are never reached by our proof by induction, because to reach them one would necessarily first go through the case that treats parallel small calls, case which does not exploit an induction hypothesis. (Recall that the definition of W excludes all the overheads involved throughout the execution of a parallel small call.)

-Fourth case: the call is the non-covered sequential call. In this case, the work equals to the sequential work, w = W s (S), and W does not exclude the overheads, so W = W s (S)+φ. By Lemma 4.4.11, we have: W s (S) ≥ κ γDE . In other words, W s (S) ≥ κ . This inequality may be reformulated as: 2Ws(S)-κ κ ≥ 1. Recall that we have W s (S) ≤ w, since here W s (t) = W s (S). Combining all these results yields:

W = W s (S) + φ ≤ W s (S) + 2W s (S) -κ κ φ ≤ B(W s (S)) ≤ B(w).
Lemma 4.4.13 (Bound on the overheads associated with small calls).

W -W ≤ P F GH • (τ + (2γ -1) • φ)
Proof. This bound on the overheads associated with small calls is obtained as the sum of the overheads associated with covered sequential calls and the overheads associated with parallel small calls. For the former, each covered sequential call induces an overhead of φ, and there are at most P F H • (2γ -2) of them per spguard, by Lemma 4.4.10. For the latter, each parallel small call induces an overhead of τ + φ, and there are at most P F H of them per spguard, by Lemma 4.4.9. Multiplying by G, the number of spguards, and factorizing the sum leads to the aforementioned bound.

Theorem 4.4.2 (Bound on the total work using our algorithm).

W ≤ 1 + DEγ • (τ + 2φ) κ • w + P F GH • (τ + 2γ • φ)
Proof. Obtained by summing the bound on W obtained from Lemma 4.4.12 and the bound on W -W obtained from Lemma 4.4.13 with the fact that 2γ -1 < 2γ. We have:

T P ≤ 1 + γED • (τ + 2φ) κ • w P + (1 + φ + max(τ, Eβκ)) • s + F GH • (τ + 2γφ).
Proof. Combining Theorem 4.4.1 about the total span and Theorem 4.4.2 about the total work together with Brent's Theorem 2.2.1, we get our main theorem.

Theorem 4.4.3 (Bound on the parallel run time). Under two assumptions κ ≥ τ and κ ≥ 1 + φ, the bound on the parallel running time from the previous theorem can be slightly simplified:

T P ≤ 1 + γED • (τ + 2φ) κ • w P + (Eβ + 1) • κ • s + O G • log 2 κ • (τ + 2γφ) .
Proof. The proof is straightforward. At first, since κ ≥ τ , κ ≥ 1 + φ and E, β ≥ 1 we obtain:

(1 + φ + max(τ, Eβκ)) ≤ (1 + Eβ) • κ. Secondly, F = 1 + log α κ DE and H = log γ/(γ-1) κ DE , thus F • H = O(log 2 κ).
Theorem 4.3.1 (Simplified bound on the parallel run time). For fixed hardware and any program, all parameters of the analysis except for κ which represents the unit of parallelism can be replaced with constants, leaving us with the following bound:

T P ≤ 1 + O(1) κ w P + O(κ) • s + O log 2 κ .
Proof. Immediate from the previous theorem.

Experimental Evaluation

For our experiments we chose 11 benchmarks from the PBBS benchmark suite [START_REF] Shun | Brief announcement: the problem based benchmark suite[END_REF], which covers diverse algorithms and is designed to compare different parallel programming methodologies in terms of performance. Here we list the benchmarks: blockradix-sort stably sorts fixed-length unsigned integer keys with the ability to carry along fixed-length auxiliary data; comparison-sort sorts a sequence of any type given a comparison function; suffix-array sorts all suffixes of a given string; convex-hull build a convex hull over a set of points in 2D; nearest-neighours given a set of points in 2D or 3D finds k nearest neighbours for each point; ray-cast given a set of triangles inside a 3D bounding box and a set of rays calculates for each ray the first triangle it intersects; delaunay given a set of points in 2D calculates the Delaunay triangulation for them; mis given an undirected graph calculates a maximal independent set, i.e., a set of vertices, such any pair of points are not adjacent and no other vertex can join this set; mst given a weighted undirected graph calculates a minimum spanning tree (or forest), i.e., a subgraph without cycles, in which any pair of vertices is connected, and which has the minimum possible total edge weight; spanning given an undirected graph calculates any spanning tree (or forest); BFS given an unweighted undirected graph and a source calculates the shortest path from the source to all vertices. PBBS includes only four more benchmarks: delaunay-refine given a Delaunay triangulation in 2D calculates new Dealunay triangulation with all original points plus points such that no triangle has an angle smaller than a threshold θ; n-body given n points in 3D calculates the gravitational force vector on each point due to all other points; removeduplicates removes duplicates from a sequence of any type; maximal-matching given an undirected graph calculates a maximal matching, i.e., the set edges such that any pair of edges does not share a vertex and that no other edge can join this set. We do not consider them in our evaluation analysis because: delaunay-refine and n-body have critical code regions that do not readily admit a cost function; remove-duplicates and maximal-matching exhibit a very high variability in the running time of their critical loops, due to heavy use of atomic operations that trigger massive bus contention. We leave it to future work to investigate how to generalize our approach to support these algorithms.

The code provided for these benchmarks in the PBBS suite is highly-optimized. It relies on a number of sophisticated techniques to control granularity, with careful engineering to select the technique and hand-tune threshold settings. Two of the benchmarks we consider, namely ray-cast and delaunay, are among the largest and most complex codes in PBBS. The BFS and nearest-neighbour benchmarks are also notable because these benchmarks were identified by the PBBS authors as problem cases for granularity control.

For our programs, we ported the original code by modifying only what was needed to set up our automatic granularity. Overall, we frequently relied on default cost functions provided by our library, and only provided 24 explicit cost functions. Changes are summarized as follows.

• Several divide-and-conquer algorithms involved a manually-fixed grain to control granularity (like in Figure 4.3). We replaced them with a spguard (like in Figure 4.4), thereby eliminating the hardware-dependent magic numbers.

• A number of loops were parallelized by splitting in fixed size blocks of 2048 items each -a pattern widely used throughout the PBBS sequence library. We replaced all of them with our automatically-controlled parallel-for loop.

• Other loops exploited Cilk parallel for-loops, which essentially split the loop range in 8P blocks, where P is the number of cores. We also replaced all of them with our automatically-controlled loops.

• A number of inner loops were forced to be sequential, even though they could have been parallel. As we learned from private communication with the authors, the purpose was to tame the overheads. We restored parallelism, using our automaticallycontrolled loops.

• A number of loops were forced to always make one spawn per iteration, using a Cilk for-loop with grain size 1. Doing so is needed in situations were the loop body itself may need to generate parallel spawns, because nontrivial grain size might dramatically reduce parallelism. Again, we replaced all such loops with our automaticallycontrolled loops, which properly support nested parallelism.

In summary, from a programmer's perspective, automatic granularity control enables replacing careful selection of techniques with a single, uniform technique for controlling granularity. Furthermore, our automatic approach does not require labor-intensive tuning of grain sizes, and it automatically adapts to the hardware -performance is portable. These significant benefits are not completely free. Indeed, our algorithm needs to infer granularity thresholds online, through a convergence phase. Nevertheless, as established by our analysis (and, as we confirm through our experimental results), the cost of the convergence phase generally accounts for at most a few percent of the parallel run time.

In addition to its engineering benefits, automatic granularity control may also deliver better results. Depending on the situations, the elimination of Cilk for-loops may either increase the number of spawns, possibly increasing utilization without noticeably increasing the overheads; or it may decrease the number of spawns, possibly decreasing overheads without noticeably decreasing parallelism. Likewise, the parallelization of inner sequential loops may increase the parallelism available without incurring significant overheads. Thus, we may expect to see, on a number of benchmarks, significant performance improvement.

Experimental Setup

Our primary test harness is an Intel machine with 40 cores. We compiled the code using GCC (version 6.3) using the extensions for Cilk Plus (options -O2 -march=native -fcilkplus). Our 40-core machine has four 10-core Intel E7-4870 chips, at 2.4GHz, with 32Kb of L1 and 256Kb L2 cache per core, 30Mb of L3 cache per chip, and 32GB RAM, and runs Ubuntu Linux kernel v3.13.0-66-generic. For this machine, we used κ = 25 microseconds and α = 1.5. For each data point, we report the average running time over 30 runs. The variation in the running times is negligible: overall, we observed only a few cases where the standard deviation is 5%, but it was usually below 3%.

The benchmarks in the PBBS suite were originally implemented in C++, using Cilk extensions to realize parallelism. As baselines we use the author's original code for each benchmark. Note that that code was tuned by the authors offline, on a collection of inputs, using a test machine similar to ours, and using GCC compiler like we did.

Input Data Description

For input data, we used mostly the same data sets used in the original PBBS study [START_REF] Shun | Brief announcement: the problem based benchmark suite[END_REF], but in a number of cases using newly acquired data.

For radix-sort, we used a variety of inputs of 10 8 items. The random input consists of 32-bit integers ∈ [0, 2 31 ) drawn from the uniform distribution and exponential from the exponential distribution. The input random kvp 256 consists of integer pairs (k, v) such that k ∈ [0, 2 31 ) and v ∈ [0, 256), and random kvp 10 8 with v ∈ [0, 10 8 ).

For comparison-sort, we used a variety of inputs of 10 8 items. The random input consists of 64-bit floats ∈ [0, 1) from a uniform distribution, and exponential from an exponential distribution. The almost sorted input consists of a sorted sequence 64-bit floats ∈ [0, 10 8 ) that is updated with 10 4 random swaps. The trigrams input consists of strings generated using trigram distribution.

For suffix-array, we used one synthetic and three non-synthetic inputs. The trigrams input consists of a string of length 10 8 generated using trigram distribution. The dna input consists of a DNA sequence and has about 32 million charachters. The text input consists of about 105 million characters drawn from Project Gutenberg. The wiki input consists of 100 million characters taken from wikipedia's xml source files.

For convex-hull, we used a variety of inputs of 10 8 2D points. The in circle input consists of points inside the unit circle centered at the origin, on circle consists of points on the unit circle centered at the origin, and kuzmin consists of points from Kuzmin's distribution [START_REF] Guy E Blelloch | Design and implementation of a practical parallel delaunay algorithm[END_REF].

For nearest neighbors, we used a variety of inputs of 10 8 2D and 3D points. The inputs in square and on square consist of 2D points in and on, respectively, the unit square centered at the origin. The input kuzmin consists of 2D points drawn from the Kuzmin distribution. The input plummer consists of 3D points drawn from the Plummer distribution [START_REF] Crozier | On the problem of distribution in globular star clusters[END_REF]. The inputs in sphere and on sphere consist of 3D points in and on, respectively, the unit sphere centered at the origin.

For ray-cast, we used a variety of synthetic and non-synthetic inputs. The input in cube consists of 10 6 triangles with vertices drawn from the unit cube centered at the origin, and on sphere consists of 10 6 triangles with vertices drawn from the unit circle centered at the origin. The input happy consists of happy Buddha mesh from the Stanford 3D Scanning Repository, and it consists of 1087716 triangles. The input xyz-rgb-manusript comes from the same repository and consists of 4305818 triangles. The input turbine comes from a different repository and consists of 1765388 triangles.1 For each of the mesh with n triangles, n rays were generated: the start of each ray is randomly drawn from the lowest side of the bounding box of the mesh and the end of each ray is randomly drawn from the upper side of the bounding box of the mesh.

For delaunay, we used two inputs consisting of 10 7 2D points. The input in square consists of points in the unit square, and kuzmin consists of points drawn from the Kuzmin distribution.

For mis and spanning, we used three input graphs. The graph cube-grid (33076161 vertices and 99228483 edges) is represented as follows: the vertices are the unit cubes of the cube with side 321 and an edge exist between two vertices if the cubes share the side. The graph rMat24 (rMat27 ) is a synthetic graph on 16777216 vertices and 119845397 (118768790) generated with power-law distribution degrees [START_REF] Chakrabarti | R-mat: A recursive model for graph mining[END_REF] with settings a = 0. For mst, we used the same three graphs as above. A weight of each edge is generated uniformly at random from [0, 2 31 ).

For bfs, we used 13 different graphs that is a representative subset of the graphs used in a different performance study in [START_REF] Umut A Acar | A work-efficient algorithm for parallel unordered depth-first search[END_REF]. This subset includes small-world graphs, such as the networks of livejournal and twitter, and high-diameter graphs, such as a map of Europe and cube-grid. We already described three graphs cube-grid, rMat24 and rMat27 above. The graphs livejournal and twitter describe social networks [1,[START_REF] Kwak | What is twitter, a social network or a news media[END_REF]: the first has 4 847 571 vertices and 68 993 773 edges, and the second has 41 652 231 vertices and 1 468 365 182 edges. The graph wikipedia (as of 6 February 2007) is taken from University of Florida sparse-matrix collection [START_REF] Timothy | The university of florida sparse matrix collection[END_REF]: 3 566 907 vertices and 45 030 389 edges. The graph europe is chosen from DIMACS challenge problems [124]: 50 912 018 vertices and 108 109 320 edges. The remaining six graphs are synthetic. The directed graph square-grid (49 999 041 vertices and 99 998 082 edges) is represented as follows: the vertices are the cells of the square with side 7071 and a cell has at most two out-edges, to the right neighbour cell and to the up neighbour cell. The directed graph par-chains-100 (50 000 002 vertices and 50 000 100 edges) is 100 directed chains with 500 000 vertices each that start in one source vertex and finish in one sink vertex. The directed graph trunk-first (10 000 001 vertices and 10 000 000 edges) is a directed chain of length 5 000 000 each vertex of which (except for the last one) has additional out-edge to a vertex with out-degree zero. The directed graph phases-10-d-2 (33 333 331 vertices and 93 333 306 edges) has a source vertex and 10 layers each with 3 333 333 vertices, a source has out-edges to all vertices of the first layer, exactly one vertex of each layer has out-edges to all the vertices of the next layer, and any other vertex has out-edges to 2 random vertices of the next layer. The directed graph phases-50-d-5 (40 000 001 vertices and 196 800 000 edges) has a source vertex and 50 layers each with 800 000 vertices, a source has out-edges to all vertices of the first layer, exactly one vertex of each layer has out-edges to all the vertices of the next layer, and any other vertex has out-edges to 5 random vertices of the next layer. The directed graph trees-524k (99 712 001 vertices and 99 712 000 edges) is a directed chain of length 381 each vertex of which (except for the last one) has 524 287 additional out-edges to vertices with out-degree zero. The last three graphs are cube-grid, rMat24 and rMat27 described above.

Main PBBS Results

The performance results appear in Table 4.2. The first column indicates the execution time of the baseline, while the second one indicates the relative performance of our version featuring automatic granularity control. For example, -10% indicates that our code is 10% faster. Overall, our oracle-guided algorithm performs effectively the same and, in six cases, between 23% and 36% better than the unaltered authors' code.

Interestingly, we learned from private communication that the PBBS authors were aware of granularity problems affecting nearest-neighbors and BFS, but not for other codes, such as blockradix-sort, sample-sort, and convex-hull. Even carefully hand-tuned codes can contain inefficiencies related to granularity control that are not necessarily obvious and can easily escape detection.

Parallel BFS

In addition to the other PBBS programs, we considered the non-deterministic nested BFS benchmark, named ndBFS in the PBBS paper. We chose ndBFS because it is the fastest among the alternative BFS implementations in the PBBS suite and, to the best of our knowledge, the fastest publicly available Cilk implementation of BFS.

There are two versions of BFS: the flat and the nested version (ndBFS). In the flat version, the traversal over the neighbors of each vertex is performed sequentially. In the nested version, it is performed using a parallel loop, allowing to process in parallel the out-edges of vertices with high out-degree.

PBBS currently uses the flat version. Via personal communication, we learned that the authors sequentialized the inner loop because (1) the graphs they considered did not feature such high-degree vertices, and (2) they observed that performance improved for some of their test graphs when this loop was serialized. Columns 2 and 4 from Figure 4 parallel cilk_for loop over the edges. The figures confirm that overheads of parallelization using cilk_for are significant, sometimes above 50%.

In contrast, our algorithm, which supports nested parallelism, delivers significantly better results for nested BFS, as reported in the 5th column from Figure 4.7 ("nested/ours"). The last column from this figure shows our main result: it compares the performance of the original authors' flat BFS versus our nested BFS with automatic granularity control. Our version either performs about as well or up to 86% faster.

Portability Study

We run the same set of experiments on two additional machines. As baselines we use the PBBS author's code, unaltered.

Portability: 48-core AMD machine This machine has four 12-core AMD Opteron 6172 processors running at 2.1GHz. Each core has 64KB of L1 instruction and data cache and a 512KB L2 cache. Each processor has two 6MB L3 cache that is shared with the four cores of the processor. The system has 128Gb of RAM and runs Ubuntu Linux 10.04.2 LTS. We tuned κ and α to be 30 microseconds and 1.1, respectively. Results are shown in Table 4.10 and Figure 4.8.

Portability: 72-core Intel Xeon machine

This machine features four Intel Xeon E7-8867 chips, each with 18 cores. Each core has a 32Kb L1 cache and a 256Kb L2 cache, and there is one 45Mb shared cache per chip. The system has 1Tb of RAM and runs is running Ubuntu v16.04 LTS (with Linux kernel version 4.10.0-27). We tuned κ and α to be 40 microseconds and 1.2, respectively. Results are shown in Table 4.11 and Figure 4.9.

Although the architecture of this machine is similar to our primary 40-core Intel machine, this 72-core machine is also useful to test the scalability of our granularity control approach.

One outlier value it the +14% for the BFS algorithm on the cube-grid graph. This slowdown is due to the particular graph structure and its particular size: the BFS traversal of this graph exhibits a critically-low amount of parallelism, explaining why the slowdown did not show up on the other two test machines. For this input graph, the cilk_for loop manages to exploit slightly more parallelism within each round of a succession of several BFS frontiers. It does so by spawning 8P parallel tasks, each of size less than

Summary

These results show that, as expected, individual benchmarks could perform slightly differently on different machines, but also that the overall trends remain the same. Therefore, we conclude that, as suggested by our theoretical results, our technique appears to be portable across different machines.

Related Work

Controlling the overheads of parallelism has been an important open problem since the early 1980's, when it was first recognized that practical overheads can overwhelm the benefits of parallelism [START_REF] Robert | Implementation of multilisp: Lisp on a multiprocessor[END_REF]. Since then, researchers have explored two separate approaches that reduce overheads by reducing the number of created threads. Note that these approaches are complementary to approaches that reduce overheads by reducing per-task costs [START_REF] Umut A Acar | Scheduling parallel programs by work stealing with private deques[END_REF][START_REF] Frigo | The implementation of the cilk-5 multithreaded language[END_REF][START_REF] Hiraishi | Backtrackingbased load balancing[END_REF][START_REF] Kumar | Work-stealing without the baggage[END_REF][START_REF] Sanchez | Flexible architectural support for fine-grain scheduling[END_REF][START_REF] Tardieu | A work-stealing scheduler for x10's task parallelism with suspension[END_REF].

Granularity Control

The granularity control is a technique that reduces the number of threads by selectively switching from a parallel code to its, possibly faster, sequential alternative. Perhaps the oldest granularity control technique is to use condition-guards for each task creation: "cutoff" conditions that switch from parallel to a sequential mode of execution. Researchers have addressed the limitations of such manual granularity control by using various forms of automation. Weening et al. [START_REF] Joseph | Low-cost process creation and dynamic partitioning in qlisp[END_REF][START_REF] Joseph S Weening | Parallel execution of lisp programs[END_REF] proposed two methods based on height and depth cutoffs. The "height cutoff" approach approximates the height of the current subcomputation in the execution tree and then compares it against the single properly chosen cutoff constant. The two drawbacks are: (1) sometimes the height of the subcomputation cannot be deduced;

(2) the increase of the problem size results in the increase of the number of created threads and, thus, the cutoff constant should be adjusted. The "depth cutoff" approach compares the depth, i.e., the distance from the root to a current subcomputation in a computation tree, with a single properly chosen cutoff constant. Its disadvantages are: (1) the depth cannot provide an approximation of the size of the task and on skewed computation trees this approach can create very small tasks; (2) when the code has several functions that depend on each other, it becomes non-trivial to find the optimal cutoff constant. Suppose that a function f has a sole argument -a list l. The algorithm by Huelsbergen et al. [START_REF] Lorenz Huelsbergen | Using the run-time sizes of data structures to guide parallel-thread creation[END_REF] compares the length of l with a cutoff constant. This cutoff constant C is deduced statically by abstractly executing f on lists of different sizes: C is the smallest value such that the execution time of f on the list of length C exceeds the overhead threshold T . Note that this approach works only for functions which execution time depends on the length of exactly one argument.

Lopez et al. [START_REF] Sk Debray | A methodology for granularity-based control of parallelism in logic programs[END_REF] proposed an approach similar to ours, but in the context of logic programming: the programmer is required to provide cost functions and, then, the approach decides to execute in parallel or sequentially by comparing costs with cutoff constants. However, these cutoff constants are precomputed statically, while our algorithm does not require preliminary tuning.

The algorithm by Duran et al. [START_REF] Duran | An adaptive cut-off for task parallelism[END_REF] uses an average execution time of subcomputations on each depth to make predictions: if the average time for the current depth exceeds some cutoff constant then the code is executed in parallel, otherwise, sequentially. To find the average time for each depth, the algorithm measures the sequential execution time of the subcomputations (as the sum of the execution times of sequential pieces) and, then, reports it. If the number of reports for the current depth is big enough, e.g., 100, then the average time is computed as an average time of reports. Despite the fact that this approach is fully automatic, the average time is not a proper approximation of the work. For example, if we received reports from 100 nodes on depth d with very little work then any node on depth d with a lot of work will be executed sequentially leading to the loss of parallelism.

Another approach to control granularity is proposed by Thoman et al. [START_REF] Thoman | Adaptive granularity control in task parallel programs using multiversioning[END_REF] Suppose a process withdraws a huge task (for example, the program is irregular, and this is the biggest task created) to execute and sees that all processes are busy. The process decides to sequentialize that task and, thus, the parallelism is lost.

The latest technique was proposed by Iwasaki et al. [START_REF] Iwasaki | A static cut-off for task parallel programs[END_REF]. For each function it uses a static analysis to find a condition on arguments in which the height of the subcomputation is within a specifically chosen constant height H. Thus, when a function is executed the condition is checked: if it is satisfied, the function is executed sequentially, otherwise, it is executed in parallel. Further, the technique uses a sophisticated compiler support to provide static optimizations, such as code-bloar-free inlining (inlining of the immediate function calls) and loopification (transforming tasks into vectorizable loops). The major drawback of this approach is that it uses height cutoffs: as shown by Weening [START_REF] Joseph S Weening | Parallel execution of lisp programs[END_REF] the height is not a good approximation of the sequential execution time. In addition, in complex programs the static analysis cannot deduce the conditions and the approach has to decide on runtime information such as depth and the number of ready tasks: as shown in prior works none of these properties can reasonably approximate the sequential execution time.

Lazy Task Creation

In contrast to granularity control, lazy task creation (or, sometimes, lazy scheduling) focuses on reducing the number of tasks by observing the load in the system and creating tasks only when necessary [START_REF] Bergstrom | Lazy tree splitting[END_REF][START_REF] Feeley | A message passing implementation of lazy task creation[END_REF][START_REF] Mohr | Lazy task creation: A technique for increasing the granularity of parallel programs[END_REF][START_REF] Tzannes | Lazy binary-splitting: a run-time adaptive work-stealing scheduler[END_REF]. Although, lazy task creation can reduce overheads of parallelism, it has some important limitations. First, there is no way to account for the fact that an alternative sequential algorithm is more efficient than the original parallel one executed on one process. So, it is reasonably common, that an algorithm under lazy task creation works 3-10 times slower than the same algorithm under manual granularity control.

Another issue with lazy task creation is that it requires regular checks whether some process made a work request. Unfortunately, the simplest way to implement it is to use a compiler support that injects polling checks [START_REF] Feeley | Polling efficiently on stock hardware[END_REF].

However, we think that granularity control, such as our algorithm, and lazy task creation are complementary. For example, when it is impossible to provide a cost function lazy task creation can improve performance, and when it is possible our granularity control can improve performance by switching to a sequential alternative.

Conclusion

The problem of managing parallelism-related overheads effectively is an important problem facing parallel programming. The current state of the art in granularity control places the burden of the tuning on the programmer. The tuning process is labor intensive and results in highly engineered, possibly performance-brittle code. In this paper, we show that it is possible to control granularity in a more principled fashion. The key to our approach is an online algorithm for efficiently and accurately estimating the work of parallel computations. We show that our algorithm can be integrated into a state-of-the-art, implicitly parallel language and can deliver executables that compete with and sometimes even beat expertly hand-tuned programs.

A Concurrency-Optimal Binary Search Tree

Introduction

To meet modern computational demands and to overcome the fundamental limitations of computing hardware, the traditional single-CPU architecture is being replaced by a concurrent system based on multi-cores or even many-cores. Therefore, at least until the next technological revolution, the only way to respond to the growing computing demand is to invest in smarter concurrent algorithms. Synchronization, one of the principal challenges in concurrent programming, consists in arbitrating concurrent accesses to shared data structures: lists, hash tables, trees, etc. Intuitively, an efficient data structure must be highly concurrent: it should allow multiple processes to "make progress" on it in parallel. Indeed, every new implementation of a concurrent data structure is usually claimed to enable such parallelism. But what does "making progress" means precisely?

Optimal Concurrency

If we zoom in the code of an operation on a typical concurrent data structure, we can distinguish data accesses, i.e., reads and updates to the data structure itself, performed as though the operation works on the data in the absence of concurrency. To ensure that concurrent operations do not violate correctness of the implemented high-level data type (e.g., linearizability [START_REF] Herlihy | Wait-free synchronization[END_REF] of the implemented set abstraction), data accesses are "protected" with synchronization primitives, e.g., acquisitions and releases of locks or atomic read-modify-write instructions like compare&swap. Intuitively, a process makes progress by performing "sequential" data accesses to the shared data, e.g., traversing the data structure and modifying its content. In contrast, synchronization tasks, though necessary for correctness, do not contribute to the progress of an operation.

Hence, "making progress in parallel" can be seen as allowing concurrent execution of pieces of locally sequential fragments of code. The more synchronization we use to protect "critical" pieces of the sequential code, the less schedules, i.e., interleavings of data accesses, we accept. Intuitively, we would like to use exactly as little synchronization as sufficient for ensuring linearizability of the high-level implemented abstraction. This expectation brings up the notion of a concurrency-optimal implementation [START_REF] Gramoli | In the search for optimal concurrency[END_REF] that only rejects a schedule if it does violate linearizability.

To be able to reason about the "amount of concurrency" exhibited by implementations employing different synchronization techniques, we consider the recently introduced notion of "local serializability" (on top of linearizability) and the metric of the "amount of concurrency" defined via sets of accepted (locally serializable) schedules [START_REF] Gramoli | In the search for optimal concurrency[END_REF]. Local serializability, intuitively, requires the sequence of sequential steps locally observed by every given process to be consistent with some execution of the sequential algorithm. Note that these sequential executions can be different for different processes, i.e., the execution may not be serializable [START_REF] Papadimitriou | The serializability of concurrent database updates[END_REF]. Combined with the standard correctness criterion of linearizability [START_REF] Attiya | Distributed computing: fundamentals, simulations, and advanced topics[END_REF][START_REF] Maurice | Linearizability: A correctness condition for concurrent objects[END_REF]), local serializability implies our basic correctness criterion called LS-linearizability. The concurrency properties of LS-linearizable data structures can be compared on the same level: implementation A is "more concurrent" than implementation B if the set of schedules accepted by A is a strict superset of the set of schedules accepted by B. Thus, a concurrency-optimal implementation accepts all correct (LS-linearizable) schedules.

A Concurrency-Optimal Binary Search Tree

It is interesting to consider binary search trees (BSTs) from the optimal concurrency perspective, as they are believed, as a representative of search data structures [START_REF] Vinay | Safe locking policies for dynamic databases[END_REF], to be "concurrency-friendly" [START_REF] Sutter | Choose concurrency-friendly data structures[END_REF]: updates concerning different keys are likely to operate on disjoint sets of tree nodes (in contrast with, e.g., operations on queues or stacks).

We present a novel LS-linearizable concurrent BST-based set implementation. We prove that the implementation is concurrency-optimal with respect to a standard partiallyexternal sequential tree [START_REF] Gramoli | In the search for optimal concurrency[END_REF]. The proposed implementation employs the optimistic "lazy" locking approach [START_REF] Heller | A lazy concurrent list-based set algorithm[END_REF] that distinguishes logical and physical deletion of a node and makes sure that read-only operations are wait-free [START_REF] Herlihy | Wait-free synchronization[END_REF], i.e., cannot be delayed by concurrent processes.

The algorithm also offers a few algorithmic novelties. Unlike most implementations of concurrent trees, the algorithm uses multiple locks per node: one lock for the state of the node, and one lock for each of its descendants. To ensure that only conflicting operations can delay each other, we use conditional read-write locks, where the lock can be acquired only under certain condition. Intuitively, only changes in the relevant part of the tree structure may prevent a thread from acquiring the lock. The fine-grained conditional read-write locking of nodes and edges allows us to ensure that an implementation rejects a schedule only if it violates linearizability.

Concurrency-Optimality and Performance

Of course, optimal concurrency does not necessarily imply performance nor maximum progress (à la wait-freedom [START_REF] Herlihy | On the nature of progress[END_REF]). An extreme example is the transactional memory (TM) data structure. TMs typically require restrictions of serializability as a correctness criterion. And it is known that rejecting a schedule that is rejected only if it is not serializable (the property known as permissiveness) requires very heavy local computations [START_REF] Guerraoui | Permissiveness in transactional memories[END_REF][START_REF] Kuznetsov | On the cost of concurrency in transactional memory[END_REF]. But the intuition is that looking for concurrency-optimal search data structures like trees pays off. In this chapter, we show that optimal concurrency can pay off: we demonstrate empirically that the Java implementation of our concurrency-optimal BST outperforms state-of-the-art BST implementations [START_REF] Nathan G Bronson | A practical concurrent binary search tree[END_REF][START_REF] Crain | A contention-friendly binary search tree[END_REF][START_REF] Drachsler | Practical concurrent binary search trees via logical ordering[END_REF][START_REF] Ellen | Nonblocking binary search trees[END_REF] on most workloads. Apart from the obvious benefit of producing a highly efficient BST, this work suggests that optimizing the set of accepted schedules of the sequential code can be an adequate design principle for building efficient concurrent data structures.

Roadmap

The rest of the paper is organized as follows. In Section 5.2, we describe the details of our concurrency-optimal BST implementation together with a novel conditional readwrite lock abstraction. In Section 5.3, we formalize the notion of concurrency-optimality and sketch the corresponding proof. In Section 5.4, we give a proof of correctness of our concurrent algorithm and, in Section 5.5, we show its concurrency-optimality. In Section 5.6, we provides details of our experimental methodology and extensive evaluation of our Java implementation. In Section 5.7, we presents concluding remarks.

Binary Search Tree Implementation

This section consists of two parts. At first, we describe in details the sequential implementation of the set based on partially-external binary search tree which we briefly mentioned in Section 3.1. Then, we build the concurrent implementation on top of the sequential one by adding synchronization separately for each field of a node. Our implementation takes only the locks that are neccessary to perform correct modifications of the tree structure. Moreover, if the field is not going to be modified, the algorithm takes the read lock instead of the write lock. The last is done only to improve the performance of the algorithm, i.e., allow more schedules of the concurrent algorithm, while, as we shall see, this does not affect the optimality.

Sequential Implementation

As for a sequential implementation we chose the well-known partially-external binary search tree. The partially-external tree supports two types of nodes: routing and data. The set is represented by the values stored by the data nodes. To bound the number of routing vertices by the number of data nodes the tree should satisfy the following condition: all routing nodes should have exactly two children.

The pseudocode of the sequential implementation is presented in Figure 5.1. Here, we give a brief description. The traversal function takes a value v and traverses down the tree from the root following the corresponding links as long as the current node is not null and its value is not v. This function returns the last three visited nodes. The contains function takes a value v and returns whether the last visited node is null or not.

The insert function takes a value v and uses the traversal function to find the place to insert the value. If the last node returned by the traversal is not null, the algorithm checks whether the node is data or routing: in the former case it is impossible to insert; in the latter case, the algorithm simply changes the state of the node from routing to data. If the last node is null, then the algorithm assumes that value v is not in the set and inserts a new node with value v as the child of the latest non-null node visited by the traversal.

The delete function takes a value v and uses the traversal function to find the node with value v to delete. If the last node returned by the traversal is null or its state is routing, the algorithm assumes that value v is not in the set and finishes. Otherwise, there are three cases depending on the number of children that the found node has: (i) if the node has two children, then the algorithm changes its state from data to routing; (ii) if the node has one child, then the algorithm unlinks the node; (iii) finally, if the node is a leaf then the algorithm unlinks the node, and if the parent is a routing node then it also unlinks the parent.

Concurrent Implementation

As the basis of our concurrent implementation we took the idea of optimistic algorithms, where the algorithm reads all necessary variables without synchronization and right before the modification, the algorithm takes all the locks and checks the consistency of all the information it read. As we show in the next section, the obtained concurrent partiallyexternal BST appears to be concurrency-optimal. The algorithm is presented at Figure 5.4 while the necessary read-write lock and node classes are presented at Figures 5.2 and 5.3, respectively. Now, we discuss the details.

Deleted Mark

As usual in concurrent algorithms with wait-free traversals, the deletion of any node happens in two stages. At first, the delete operation logically removes a node from the tree by setting the boolean flag deleted (Figure 5

Traversal function

The traversal function is slightly different from the sequential implementation, but we omit it from the pseudocode. It traverses as the sequential algorithm, but if the target node is "under-deletion", i.e., its deleted mark is set, the traversal is restarted.

Locks

In the beginning of the section we noted that we have locks separately for each field of a node: llock for the reference to the left child, rlock for the reference to the right child and slock for the state (Figure 5.3 Lines 8-10). Also, as mentioned, the algorithm uses read-write locks, implemented using one integer variable lock (Figure 5.2). The least significant bit of lock indicates whether the write lock is taken or not, the remaining bits represent the number of readers that have taken the lock. In other words, lock is zero if the lock is not taken, lock is one if the write lock is taken, otherwise, lock divided by two represents the number of times the read lock is taken. The locking and unlocking are done using the atomic compare&swap primitive. Throughout the algorithm the locks are acquired using helper functions that are provided by the Node class (Figure 5.3). These functions take a lock and verify several conditions: they check whether the future modification is possible. If the conditions are not satisfied, the guarded modification will corrupt the data structure (in the next subsection we explain what we mean by corruption). In our concurrent algorithm when the helping function returns false (Figure 5.4 Lines 15, 20, 24, 42, 47, 48, 51, 71, 76, 81, 90-92 and 98-100), all locks taken up to this moment (including this one) by the high-level "parent" function call, i.e., contains, insert and delete, are released and that "parent" call is restarted.

These are the helper functions:

• tryReadLockState() (Figure 5.3 Lines 36-40) takes a read lock on slock and checks that the node is not marked as deleted;

• try(Read|Write)LockState(exp) (Figure 5.3 Lines 42-46 and Lines 48-52) takes a (read|write) lock on slock, checks that the state of the node is equal to exp and the node is not marked as deleted;

• tryLock(Left|Right)Ref(exp) (Figure 5.3 Lines 12-16) takes a write lock on (llock|rlock), checks that the reference to the (left|right) child is equal to exp (to verify that the child did not change) and the node is not marked as deleted;

• tryLock(Left|Right)Val(exp) (Figure 5. to exp (to verify that the value in the child did not change) and the node is not marked as deleted.

We use the notation of bar as a shorthand notation for avoiding name duplication; such notation should be read as either choosing the first option or the second option. Momentarily we make two remarks. At first, in the algorithm we typically use tryLockEdge(Ref|Val)(node) (Figure 5.3 Lines 18-22 and Lines 30-34) instead of tryLock(Left|Right)Edge(Ref|Val)(exp). Such a substitution, given non-null node, decides whether the node is the left child or the right child of the current node and calls the corresponding function providing node or node.val. Secondly, to improve the performance we implement these functions using double-verification: check the conditions, only if they are satisfied take a lock and then check the conditions again.

Concurrency-Optimality and Correctness. Overview

In this section, we discuss correctness and concurrency-optimality [START_REF] Gramoli | In the search for optimal concurrency[END_REF] of our implementation (the proofs of these properties are provided in Sections 5.4 and 5.5, respectively). Intuitively, a concurrency-optimal implementation employs as much synchronization as necessary for ensuring correctness of the implemented high-level abstraction -in our case, the linearizable set object [START_REF] Herlihy | Wait-free synchronization[END_REF].

Recall our sequential BST implementation and imagine that we run it in a concurrent environment. We refer to an execution of this concurrent algorithm as a schedule. A schedule, thus, consists of reads, writes, node creation events, and invocation and responses of high-level operations. Note that each operation performs at most one write to which we can simply refer as a write of the operation.

Notice that in every such schedule, any operation locally witnesses a consistent tree state, since it cannot distinguish the execution from a sequential one. It is easy to see that the local views across operations may not be mutually consistent, and this simplistic con- Release all locks current algorithm is not linearizable. For example, two insert operations that concurrently traverse the tree may update the same node so that one of the operations "overwrites" the other (so called the "lost update" problem). Obviously, such a schedule is "incorrect" since it is not linearizable.

To reach the concurrency-optimality we want the concurrent algorithm to avoid such "incorrect" schedules, while accepting all others. More precisely, a schedule σ is accepted by an algorithm if it has an execution in which the sequence of high-level invocations and responses, reads, writes, and node creation events (modulo the restarted fragments) is σ [START_REF] Gramoli | In the search for optimal concurrency[END_REF]. Now, we formalize what does it mean for the schedule to be "incorrect" by introducing the opposite notion of observable correctness.

Definition 5.3.1. A schedule is observably correct if each of its prefixes σ satisfies the following conditions:

• subsequence of high-level invocations and responses of operations that performed their write in σ has a linearization with respect to the set type;

• a set of nodes reachable from the root after performing σ is a BST B: (i) they form a tree rooted at node root; (ii) this tree satisfies the order property: for each node with value v all the values in the left subtree are less than v and all the values in the right subtree are bigger than v; (iii) each routing node in this tree has two children.

• BST after performing σ does not contain a node x such that there exist σ and σ , such that σ is a prefix of σ , σ is a prefix of σ (σ ≺ σ ≺ σ), x is in the BST after σ , and x is not in the BST after σ .

In other words, a schedule is observably correct if it is linearizable, during its execution the tree is always a partially-external BST and, finally, the unlinked node is never linked back.

Now we can define the concurrency-optimality property.

Definition 5.3.2. A concurrent implementation is concurrency-optimal if it accepts all observably correct schedules.

At first, in Section 5.4, we prove the correctness of our algorithm.

Theorem 5.3.1 (Correctness). The algorithm is correct:

• The schedule corresponding to any execution of our BST implementation is observably correct.

• The algorithm is deadlock-free.

Further, in Section 5.5 we prove that, in a strict sense, our algorithm accepts all correct schedules.

Theorem 5.3.2 (Optimality). Our BST implementation is concurrency-optimal.

The intuition behind the proof of Theorem 5.3.2 is the following. We show that for each observably correct schedule there exists a matching execution of our implementation. Therefore, only schedules not observably correct can be rejected by our algorithm. The construction of an execution that matches an observably correct schedule is possible, in particular, due to the fact that every critical section in our algorithm contains exactly one event of the schedule. Thus, the only reason to reject a schedule is that some condition on a critical section does not hold and, as a result, the operation must be restarted. By accounting for all the conditions under which an operation restarts, we show that the restart may only happen if the schedule violates observable correctness. ; rejected by all the popular BSTs [START_REF] Nathan G Bronson | A practical concurrent binary search tree[END_REF][START_REF] Crain | A contention-friendly binary search tree[END_REF][START_REF] Drachsler | Practical concurrent binary search trees via logical ordering[END_REF][START_REF] Ellen | Nonblocking binary search trees[END_REF][START_REF] Natarajan | Fast concurrent lock-free binary search trees[END_REF], it is accepted by a concurrency-optimal BST

Suboptimality of Related BST Algorithms

To understand the hardness of building linearizable concurrency-optimal BSTs, we present a schedule that is rejected by current state-of-the-art BST algorithms against which we evaluate the performance of our algorithm. This schedule is shown in the Figure 5.5. There is one operation p = delete(3) performed on a tree shown in part a). It traverses to leaf v with value 3. Then, some concurrent operation delete(3) unlinks node v (part b)). Later, another concurrent operation inserts a new leaf with value 3 (part c)). Operation p wakes up and locks a link since the value 3 is the same (part d)). Finally, p unlinks the node with value 3 (part e)). Note that this is a correct schedule since both delete operations can be successful; however, all the BSTs we are aware of reject this schedule or similar ones [START_REF] Nathan G Bronson | A practical concurrent binary search tree[END_REF][START_REF] Crain | A contention-friendly binary search tree[END_REF][START_REF] Drachsler | Practical concurrent binary search trees via logical ordering[END_REF][START_REF] Ellen | Nonblocking binary search trees[END_REF][START_REF] Natarajan | Fast concurrent lock-free binary search trees[END_REF]. By contrast, there is an execution of our concurrency-optimal BST that accepts this schedule.

Proof of Correctness

In general, the correctness of the parallel algorithm is carried by the proofs of linearizability and deadlock-freedom. Here, we have additional constraints on the possible executions of our algorithm: they have to carry the observably correct schedules (Definition 5.3.1). The theorem about the correctness of the algorithm can be stated as follows.

Theorem 5.

The algorithm is correct if:

• the schedule corresponding to any execution of the algorithm is observably correct.

• the algorithm is deadlock-free.

We split the proof of the first statement into two parts: the structural properties, i.e., the tree is a BST and an unlinked node cannot be linked back and the linearizability. In total our proof consists of three parts.

Structural Correctness

At first, we prove that our search tree satisfies the structural properties at any point in time, i.e., the second and the third properties of observably correctness.

Theorem 5.4.1. The following properties are satisfied at any point in time during the execution:

• The order property of BST is preserved.

• Every routing node has two children.

• Any non-physically deleted node is reachable from the root.

• Any physically deleted node is non-reachable from the root.

Proof. The first two properties are non-trivial by themselves, but we refer to papers [START_REF] Nathan G Bronson | A practical concurrent binary search tree[END_REF] and [START_REF] Crain | A contention-friendly binary search tree[END_REF] that use the similar partially-external algorithm.

The last two properties follow straightforwardly from the fact that during physical deletion the algorithm takes locks.

Linearizability

We prove for each execution that it is linearizable.

High-Level Histories and Linearizability

Suppose we are given a history H. To prove the linearizability of H we perform three steps (see Definition 2.3.2). At first, we complete H obtaining the completion H: choose a subset of incomplete operations to complete and discard all other incomplete. Then, we provide linearization points to each remaining operation giving us a total order on these operations. And, finally, we prove that the resulting order makes sense: the responses of operations remain the same in H and in the linearization (order) S.

Completions

We obtain a completion H of history H as follows. The invocation of an incomplete contains operation is discarded. An incomplete π = insert operation that has not performed a write at Lines 16,22 (26) of Figure 5.4 are discarded; otherwise, π is completed with the response true. An incomplete π = delete operation that has not performed a write at Lines 45, 60 [START_REF] Fatourou | A highly-efficient wait-free universal construction[END_REF], 79 [START_REF] Harris | A pragmatic implementation of non-blocking linked-lists[END_REF], 96 (104) of Figure 5.4 is discarded; otherwise, π is completed with the response true.

Note that the described completions correspond to the completions in which the completed operations made a write of the sequential algorithm.

Linearization Points

We obtain a sequential high-level history S equivalent to H by associating a linearization point l π with each operation π. In some cases our choice of the linearization point depends on the interval between the invocation and the response of the execution of some operation π, later referred to as the interval of π. For example, the linearization point of π in should lie in the interval of π.

Below we specify the linearization point of the operation π depending on its type.

Insert.

For π = insert(v) that returns true, we have two cases:

1. A routing node with value v was found in the tree. Then l π is associated with the write in Line 16 of Figure 5.4.

2.

A node with value v was not found in the tree. Then l π is associated with the writes in Lines 22 or 26 of Figure 5.4, depending on whether the newly inserted node is left or right child.

For π = insert(v) that returns false, we have three cases:

1. If there exists a successful insert(v) whose linearization point lies in the interval of π, then we take the first such π = insert(v) and linearize right after l π .

2. If there exists a successful delete(v) whose linearization point lies in the interval of π, then we take the first such π = delete(v) and linearize right before l π .

3. Otherwise, l π is the call point of π.

Delete.

For π = delete(v) that returns true we have four cases, depending on the number of children of the node with value v, i.e., the node curr:

1. curr has two children. Then l π is associated with the write in Line 45 of Figure 5.4.

2. curr has one child. Then l π is associated somewhere between the writes in Line 59 [START_REF] Fajstrup | Directed algebraic topology and concurrency[END_REF] and in Line 60 [START_REF] Fatourou | A highly-efficient wait-free universal construction[END_REF] of Figure 5.4, depending on whether curr is left or right child. The exact position is calculated as what comes last: Line 59 [START_REF] Fajstrup | Directed algebraic topology and concurrency[END_REF] or the last invocation of unsuccessful insert(v) or contains(v) that reads the node curr.

3. curr is a leaf with a data parent. Then l π is associated somewhere between the writes in Line 78 [START_REF] Robert | Implementation of multilisp: Lisp on a multiprocessor[END_REF] and in Line 79 (84) of Figure 5.4, depending on whether curr is left or right child. The exact position is calculated as what comes last: Line 78 [START_REF] Robert | Implementation of multilisp: Lisp on a multiprocessor[END_REF] or the last invocation of unsuccessful insert(v) or contains(v) that reads the node curr.

4. curr is a leaf with a routing parent. Then l π is associated between the writes in Line 95 [START_REF] Jagannathan | The design rationale for multi-mlton[END_REF] and in Line 96 (104) of Figure 5.4, depending on whether prev is left or right child. The exact position is calculated as what comes last: Line 95 [START_REF] Jagannathan | The design rationale for multi-mlton[END_REF] or the last invocation of unsuccessful insert(v) or contains(v) that reads the node curr.

For every π = delete(v) that returns false, we have three cases:

1. If there exists a successful delete(v) whose linearization point lies in the interval of π, then we take the first such π = delete(v) and linearize right after l π .

2. If there exists a successful insert(v) whose linearization point lies in the interval of π, then we take the first such π = insert(v) and linearize right before l π .

3. Otherwise, l π is the invocation point of π.

Contains.

For π = contains(v) that returns true, we have three cases:

1. If there exists a successful insert(v) whose linearization point lies in the interval of π, then we take the first such π = insert(v) and linearize right after l π .

2. If there exists a successful delete(v) whose linearization point lies in the interval of π, then we take the first such π = delete(v) and linearize right before l π .

3. Otherwise, l π is the invocation point of π.

For π = contains(v) that returns false, we have three cases:

1. If there exists successful delete(v) whose linearization point lies in the interval of π, then we take the first such π = delete(v) and linearize right after l π .

2. If there exists successful insert(v) which linearization point lies in the interval of π, then we take the first such π = insert(v) and linearize right before l π .

3. Otherwise, l π is the invocation point of π.

To confirm our choice of linearization points, we need an auxiliary lemma. Proof. Consider a list A(u) of ancestors of node u: root = w 1 , . . . , w n-1 , w n = u (starting from the root) in BST at the moment of the invocation of π.

Let us prove that at any point in time the child of w i in the direction of value v is w j for some j > i. The only way for w i to change the proper child is to perform a physical deletion on this child. Consider the physical deletions of w i in their order in execution. In a base case, when no deletions happened, our invariant is satisfied. Suppose that we performed first d deletions and now we consider the deletion of w j . Let w i be an ancestor of w j and w k be a child of w j in proper direction. After relinking w k becomes a child of w i in proper direction, so the invariant is satisfied for w i because i ≤ j ≤ k, while the children of other vertices remain unchanged.

Summing up, π starts at root, i.e., w 1 , and traverses only the vertices from A(u) in strictly increasing order. Thus, π eventually reaches u and returns it. Theorem 5.4.2 (Linearizability). The algorithm is linearizable with respect to set type.

Proof. We split our proof into three parts. First, we prove the linearizability of the subhistory with only successful insert and delete operations because other operations do not affect the structure of the tree. Then, we prove the linearizability of the subhistory with only update operations, i.e., successful and unsuccessful insert(v) and delete(v). And finally, we present the proof for the history with all types of operations.

Successful Update Operations.

Let Sk succ be the prefix of S consisting of the first k complete successful operations insert(v) or delete(v) with respect to their linearization points. We prove by induction on k that the sequence Sk succ is consistent with respect to set type. The base case k = 0, i.e., there are no complete operations, is trivial. The transition from k to k + 1. Suppose that Sk succ is consistent with set type. Let π with argument v ∈ Z and its response r π be the last operation in Sk+1 succ . We want to prove that Sk+1 succ is consistent with π. For that, we check all possible types of π.

1. π = insert(v) returns true.
By induction, it is enough to prove that there is no preceding operation with an argument v or the last preceding operation with an argument v in Sk+1 succ is delete(v). Suppose the contrary: let the last preceding operation with an argument v be π = insert(v). We need to consider two cases of insertion: whether π finds the node with value v in the tree or not.

In the first case, π finds a node u with value v. π should have inserted or modified u. Otherwise, the BST at l π would contain two vertices with value v and this fact violates the structural correctness. If π has inserted u, then π has no choice but only to read the state of u as data, which is impossible because π is successful. If π has changed the state of u to data, then π has to read the state of u as data, because the linearization points of π and π are guarded by the lock on state. This contradicts the fact that π is successful.

In the second case, π does not find a node with value v. We know that π and π are both successful. Suppose for a moment that π wants to insert v as a child of node p, while π inserts v in some other place. Then the tree at l π has two vertices with value v, violating the structural correctness. This means, that π and π both want to insert v as a child of node p. Because l π precedes l π and these linearization points are guarded by the lock on the corresponding link of p, π takes a lock first, modifies the link to a child of p and by that forces π to restart. During the second traversal, π finds newly inserted node with value v by Lemma 5.4.1 and becomes unsuccessful. This contradicts the fact that π is successful.

2. π = delete(v) returns true.

By induction it is enough to prove that the preceding operation with an argument v in Sk+1

succ is insert(v). Suppose the opposite: let the last preceding operation with v be delete(v) or there is no preceding operation with an argument v.

At first, consider the simplest case: there is no such operation. This means that π could not find a node with value v. Because if it finds a node then there exists another operation that should have inserted this node and consequently its linearization point would have been earlier. In this case, π cannot successfully delete, which contradicts the result of π.

The only remaining possibility is that the previous successful operation is π = delete(v). Because π is successful, it finds a non-deleted node u with value v. π should have found the same node u by Lemma 5.4.1, otherwise, the BST right before l π would contain two vertices with value v, violating the structural correctness. So, both π and π take locks on the state of u to perform an operation. Because l π precedes l π , π has taken the lock earlier and set the state of u to routing or marks u as deleted. When π obtains the lock, it could not read state as data and, as a result, cannot delete the node. This contradicts the fact that π is successful.

Update Operations.

Let Sk m be the prefix of S consisting of the first k complete operations insert(v) or delete(v) with respect to their linearization points. We prove by induction on k that the sequence Sk m is consistent with respect to set type. We already proved that successful operations are consistent, then we should prove that the linearization points of unsuccessful operations are consistent too.

The base case k = 0, i.e., there are no complete operations, is trivial. The transition from k to k + 1. Suppose that Sk m is consistent with set type. Let π with argument v ∈ Z and response r π be the last operation in Sk+1 m . We want to prove that Sk+1 m is consistent with π. For that, we check all the possible types of π. If k + 1-th operation is successful then it is consistent with the previous operations, because it is consistent with successful operations while unsuccessful operations do not change the structure of the tree.

If k + 1-th operation is unsuccessful, we have two cases.

1. π = insert(v) returns false. When we set the linearization point of π relying on a successful operation in the interval of π, the linearization point is correct: if we linearize right after successful insert(v) then π correctly returns false; if we linearize right before successful π = delete(v) then by the proof of linearizability for successful operations there exists successful insert(v) preceding π , thus, π correctly returns false.

It remains to consider the case when no successful update operation was linearized in the interval of π. By induction, it is enough to prove that the last preceding successful operation with v in Sk+1 m is insert(v). Suppose the opposite: let the last preceding successful operation with an argument v be delete(v) or there is no preceding operation with an argument v. If there is no such operation then π could not find a node with value v, because, otherwise, another operation should have inserted the node and its linearization point would have come earlier. Thus, π can successfully insert a new node with value v, which contradicts the fact that π is unsuccessful.

The only remaining possibility is that the last preceding successful operation is π = delete(v). We know that l π does not lie inside the interval of π, since, otherwise, we linearized π with respect to π . Thus, π has to find either the routing node with value v or do not find such a node at all, since π has already successfully unlinked it. (This happened because we set linearization points of successful delete operations quite tricky) In both cases, insert operation could be performed successfully. This contradicts the fact that π is unsuccessful.

2. π = delete(v) returns false.

When we set the linearization point of π relying on the successful operation in the interval of π, the linearization point is correct: if we linearize right after successful delete(v) then π correctly returns false; if we linearize right before successful π = insert(v) then by the proof of linearizability for successful operations there exists successful delete(v) preceding π or there are no successful operation with an argument v in Sk+1 m before π , thus, π correctly returns false.

It remains to consider the case when no successful operation was linearized in the interval of π. By induction, it is enough to prove that there is no preceding successful operation with v or the last preceding successful operation with v in Sk+1 m is delete(v). Again, suppose the opposite: let the previous successful operation with v be π = insert(v). By Lemma 5.4.1, π finds the data node u with value v and π can successfully remove it because no other operation with argument v has a linearization point during the execution of π. This contradicts the fact that π is unsuccessful.

All Operations.

Finally, we prove the correctness of the linearization points of all operations.

Let Sk be the prefix of S consisting of the first k complete operations ordered by their linearization points. We prove by induction on k that the sequence Sk is consistent with respect to the set type. We already proved that update operations are consistent, then we should prove that the linearization points of contains operations are consistent too.

The base case k = 0, i.e., there are no complete operations, is trivial. The transition from k to k + 1. Suppose that Sk is consistent with the set type. Let π with argument v ∈ Z be the last operation in Sk+1 . We want to prove that Sk+1 is consistent for the operation π. For that, we check all the possible types of π.

If k + 1-th operation is insert(v) and delete(v) then it is consistent with the previous insert(v) and delete(v) operations while contains(v) operations do not change the structure of the tree.

If the operation is π = contains(v), we have two cases:

1. π returns true.

When we set the linearization point of π relying on a successful update operation in the interval of π, then the linearization point is correct: if we linearize right after successful insert(v), then π correctly returns true; if we linearize right before successful π = delete(v), then, by the proof of the linearizability on successful operations, there exists successful insert(v) preceding π , thus π correctly returns true.

We are left with the case when no successful operation has its linearization point in the interval of π. By induction, it is enough to prove that the last preceding successful operation with v in Sk+1 is insert(v). Suppose the opposite: the last preceding successful operation with an argument v is delete(v) or there is no preceding successful operation with v. If there is no successful operation then π could not find a node with value v, otherwise, some operation has inserted a node before and its linearization point would have come earlier. This contradicts the fact that π is successful.

It remains to check if there exists a preceding π = delete(v) operation. Since l π does not lie inside the interval of π then π has to find either the routing node with value v or do not find such node, since π has unlinked it. This contradicts the fact that π returns true.

π returns false.

When we set the linearization point of π relying on a successful update operation in the interval of π, then the linearization point is correct: if we linearize right after successful delete(v), then π correctly returns false; if we linearize right before successful π = insert(v) then, by the proof of linearizability on successful operations either there exists a preceding π successful delete(v) or there exists no operation with an argument v in S before π . Thus π correctly returns false.

We are left with the case when no successful operation has its linearization point in the interval of π. By induction, it is enough to prove that there is no preceding successful operation with an argument v or the last preceding successful operation with an argument v in Sk+1 is delete(v). Again, suppose the opposite: the last preceding successful operation with an argument v is π = insert(v).

By Lemma 5.4.1 π finds the data node u with value v. This contradicts the fact that π returns false and π should return false.

Deadlock-Freedom

Theorem 5.4.3 (Deadlock-freedom). The algorithm is deadlock-free: assuming the infinite execution and that no thread fails in the middle of its update operation, the infinite number of operations is completed.

Proof. contains does not take locks at all, so, it is deadlock-free. We note that during insert and delete operations the locks are taken in a top-down manner: when the first lock is taken, the other locks are acquired only after the "parent" lock is taken: the "parent" lock for the state lock is the lock on the edge from parent; and the "parent" lock for the edge lock is the lock on the state in the source node. Also, let a lock-level of an operation be the number of "ancestors" towards the root of the first lock that the operation attempts to acquire.

In such a manner, when it is not possible for an operation to take a lock then there is another operation with a higher lock-level that acquired that lock. Then, if the second operation cannot proceed there exists the third blocking operation with lock-level even bigger, and so on. Since the height of the tree is finite, there exist the lowest operation that blocks everybody: this operation can proceed. This lowest operation either successfully completes or the conditions are not satisfied and the operation is restarted. However, in the last case, the operation, that forces the restart, made progress.

Proof of Concurrency-Optimality

Theorem 5.3.2 (Optimality). Our binary search tree implementation is concurrencyoptimal with respect to the sequential algorithm provided in Figure 5.1.

Proof. Consider all executions of our algorithm in which all critical sections are executed sequentially, i.e., the primitives of a critical section are executed sequentially. Since all critical sections in our algorithm contain only one operation from the sequential algorithm, the implementation accepts all the schedules in which the operation is not restarted by failing some condition in the critical sections.

Suppose that we are given a schedule σ. We show that each condition that forces the restart is crucial, i.e., if the operation ignores it, then σ is not observably correct.

By the first property of observable correctness (Definition ??) when we talk about the linearization of the schedule we talk about the linearization of operations that made the write.

To simplify the proof by exhaustion we consider three common situations (later referred to as Cases 1, 2 or 3) that appear under consideration, and show that in each of these cases the schedule σ is not observably correct.

Let two functions I(T, v) and D(T, v) be the number of insert and delete operations with argument v that made the single write of the sequential algorithm in the prefix of the schedule σ of length T , i.e., T primitives, later referred as σ(T ). 1. The modification, i.e., a corresponding write, in the critical section of operation π = insert(v) (the case of delete(v) is considered similarly) does not change the set of values represented by our tree: (1) for each node reachable from the root the fields (left and right link, and state) do not change; or (2) for each node reachable from the root (except for one) the fields do not change, and for the vertex left one routing child can be unlinked. Note that we do not discuss the deleted field, since the sequential algorithm does not know about its existence and, thus, modifications of this field do not belong to schedules.

Let this modification be the T -th primitive of the current schedule σ. Consider two prefixes of this schedule: σ(T -1) and σ(T ). There are two cases:

• If the value v is present in the set after σ(T -1), then

I(T -1, v) = D(T -1, v)+1, since σ(T -1) is linearizable.
As the T -th primitive, π made the write and, since π is insert, I(T, v) = D(T, v) + 2. Any completion of σ(T ) (considering only operations that made a write) is non-linearizable, because the number of successful insert operations cannot exceed the number of successful delete operations by more than one. This means that σ(T ) is non-linearizable, and, thus, σ is not observably correct.

• If the value v is not present in the set after σ(T -1), then

I(T -1, v) = D(T -1, v), since σ(T -1) is linearizable.
As the T -th primitive, operation π made a write and, since π is insert, I(T, v) = D(T, v) + 1,. Also, the node with value v is unreachable from the root, since the set did not change. Let us consider any linearizable completion of σ(T ). By completing operations, the set cannot change, since all writes by the operations were already performed, and, thus, v does not belong to the set. However, the number of successful insertion exceeds the number of successful deletions, and, thus, σ(T ) cannot be linearizable. This means that any completion of σ(T ) is non-linearizable, thus, σ(T ) is non-linearizable and, consequently, σ is not observably correct.

2. After the modification in the critical section of operation π with argument v a whole subtree of node u with a value different from v becomes unreachable from the root. Let this modification be the T -th primitive of the current schedule σ. Because of the structure of the partially-external tree, subtree of node u should contain at least one data vertex with value x not equal to v: either u is a data leaf, or the subtree contains at least two data nodes. Since x was reachable before the modification and σ(T -1) is linearizable, we assume

I(T -1, x) = D(T -1, x) + 1.
The number of successful update operations with argument x does not change after the modification, so I(T, x) = D(T, x) + 1. But the value x is not reachable from the root after σ(T ), meaning that any completion of (operations that made a write in) σ(T ) cannot be linearized. Thus, σ(T ) is non-linearizable and, consequently, σ is not observably correct.

3. After the modification in the critical section of operation π the node u with deleted mark becomes reachable from the root. Let this write be the T -th primitive of the current schedule σ. Let the write that was done in the same critical section as the deleted mark of u was set be the T -th primitive of σ. It could be seen that u is reachable from the root after σ( T -1) and after σ(T ), but u is not reachable from the root after σ( T ). Thus, σ does not satisfy the third requirement for observably correct schedules, meaning that σ is not observably correct. Now, we want to prove that all conditions that precede each modification operation are necessary and their omission leads to not observably correct schedule. The proof is done by induction on the position of the modification operation in the execution. The base case, when there are no modification operations done, is trivial. Suppose that we showed the correctness of the statement for the first i -1 modifications and want to prove it for the i-th. Let this modification be the T -th primitive of the schedule σ. We ignore each condition that precedes the modification one by one in some order and show that their omission makes σ not observably correct.

• Operation π = insert(v) restarts in Line 15 of Figure 5.4. As we explicitely did not write in the pseudocode for simplicity we want to remind that an update operation restarts whenether some condition checked while the locks are held is not satisfied. This means, that at least one of the following condition holds:

curr is not routing (Line 15). Then the guarded operation does not change the set of values and we can apply Case 1.

deleted mark of curr is set (later, we simply say curr is deleted) (Line 15). then curr is already unlinked, so the modification in Line 16 does not change the set of values and we can apply Case 1.

• Operation π = insert(v) restarts in Lines 20, 21 (24, 25) of Figure 5.4. This means, that at least one of the following conditions holds:

prev is deleted (Line 20 [START_REF] Barnes | A method for implementing lock-free shared-data structures[END_REF]). Then prev is already unlinked and is not reachable from the root. This means, that the modification in Line 22 [START_REF] Guy | Scans as primitive parallel operations[END_REF] links the new node to already unlinked node prev, not changing the set of values, and we can apply Case 1.

-The corresponding child of prev is not null (Line 21 [START_REF] Bergstrom | Lazy tree splitting[END_REF]). Then the write in Line 22 [START_REF] Guy | Scans as primitive parallel operations[END_REF] unlinks a whole subtree of the current child and we can apply Case 2.

• Operation π = delete(v) restarts in Lines 42 and 44 of Figure 5.4. This means that either curr is not a data node, curr is deleted or curr does not have two children.

-If curr is not a data node or it is deleted (Line 42), then the write at Line 45 does not change the set of values and we can apply Case 1.

-If curr does not have two children (Line 44), then after the write in Line 45 the tree has the routing node curr with less than two children. Thus, after σ the tree does not satisfy the second requirement for observably correct schedules.

• Operation π = delete(v) restarts in Lines 47-51 of Figure 5.4. This means that at least one of the following conditions holds:

prev is deleted (Line 47). Then the write at Line 60 [START_REF] Fatourou | A highly-efficient wait-free universal construction[END_REF] does not change the set of values and we can apply. For later cases, we already assume, that prev is not deleted.

-There is no link from curr to child (Line 51). The link can cease to exist only of one of the nodes are deleted. * child is deleted. Then after the write at Line 60 (64) the deleted node child becomes reachable from the root, since prev is not deleted, and we can apply Case 3. From hereon, we assume that child is not deleted. * curr is deleted. We know that prev and child are not deleted, thus prev already has child as its child. By that, the write at Line 60 [START_REF] Fatourou | A highly-efficient wait-free universal construction[END_REF] does not change the set of values and we can apply Case 1. From hereon, we assume that curr is not deleted.

-There is no link from prev to curr (Line 47). This can happen only if prev or curr is deleted. But we already considered these cases.

curr is not a data node (Line 48). Then the write in Line 60 [START_REF] Fatourou | A highly-efficient wait-free universal construction[END_REF] does not change the set of values and we can apply Case 1.

curr does not have exactly one child (Line 50). Since none of curr and child are deleted, the link from curr to child exists in the tree. Thus, the only possible way to violate is that curr has two children. Finally, the write in Line 60 [START_REF] Fatourou | A highly-efficient wait-free universal construction[END_REF] unlinks a whole subtree of the other child of curr and we can apply Case 2.

• Operation π = delete(v) restarts in Lines 66-73 and 76 (81) of Figure 5.4. This means that at least one of the following conditions holds:

prev is deleted (Line 76 [START_REF] Leo | A dichromatic framework for balanced trees[END_REF]). Then the write in Line 79 [START_REF] Harris | A pragmatic implementation of non-blocking linked-lists[END_REF] does not change the set of values and we can apply Case 1. From hereon, we assume that prev is not deleted.

prev is not a data node (Line 76 [START_REF] Leo | A dichromatic framework for balanced trees[END_REF]). Then after the write in Line 79 [START_REF] Harris | A pragmatic implementation of non-blocking linked-lists[END_REF] the tree contains a routing node with less than two children. Thus, σ(T ) does not satisfy the second requirement for observably correct schedules.

-The child c of prev in the direction of v is null (Line 66). Then the write in Line 79 [START_REF] Harris | A pragmatic implementation of non-blocking linked-lists[END_REF] does not change the set of values and we can apply Case 1.

-The child c of prev in the direction of v has a key different from v (Line 66).

(Note that c cannot be deleted since the link from prev to c is locked and, thus, in the tree now.) The write in Line 79 [START_REF] Harris | A pragmatic implementation of non-blocking linked-lists[END_REF] unlinks a whole subtree of c and we can apply Case 2.

-The child c of prev in the direction of v is not a leaf (Line 73). Then the write in Line 79 [START_REF] Harris | A pragmatic implementation of non-blocking linked-lists[END_REF] removes whole subtree of c with at least one data node with the value different from v and we can apply Case 2. In last case we assume that c is a leaf.

-The child c of prev in the direction of v is a routing node (Line 71). Then before the write in Line 79 [START_REF] Harris | A pragmatic implementation of non-blocking linked-lists[END_REF] the tree contained a routing leaf c. Thus, σ(T -1) does not satisfy the second requirement for observably correct schedule.

• Operation π = delete(v) restarts in Lines 66-73 and 90-92 (98-100) of Figure 5.4. This means that at least one of the following conditions holds:

gprev is deleted (Line 90 [START_REF] Lorenz Huelsbergen | Using the run-time sizes of data structures to guide parallel-thread creation[END_REF]). Then the write in Line 96 [START_REF] Jájá | An introduction to parallel algorithms[END_REF] does not change the set of values and we can apply Case 1 σ. From hereon, we assume that gprev is not deleted.

child is deleted, then the write in Line 96 [START_REF] Jájá | An introduction to parallel algorithms[END_REF] links the deleted node back to the tree and we can apply Case 3. Later, we assume that child is not deleted.

prev is not a child of gprev. (Line 90 (98)) Since gprev is not deleted, this case can happen only if prev is physically deleted. From this follows that the current child of gprev in a direction of v is child. Thus, the write in Line 96 [START_REF] Jájá | An introduction to parallel algorithms[END_REF] does not change the set of values and by Case 1 σ is not observably correct. Later, we assume that prev is not deleted.

prev is a data node (Line 91 [START_REF] Hunt | An efficient algorithm for concurrent priority queue heaps[END_REF]). Then the write in Line 96 [START_REF] Jájá | An introduction to parallel algorithms[END_REF] unlinks prev from the tree and we can use the same reasoning as in Case 2.

child is not a current child of prev (Line 92 [START_REF]Intel. Intel threading building blocks[END_REF]). However, we already showed in the previous cases that prev and child are not deleted.

-The last four cases are identical to the last four cases for delete(v) that restarts in Lines 66-73 and 91 [START_REF] Hunt | An efficient algorithm for concurrent priority queue heaps[END_REF].

We showed that the restart of operation in the execution happens only if the the corresponding sequential schedule is not observably correct. Thus, our algorithm is indeed concurrency-optimal. due to the fact that with bigger size the cleanup-thread in CF-tree implementation spends more time to clean the tree out of logically deleted vertices, thus, the traversals has more chances to pass over deleted vertices, leading to longer traversals. By this fact and the trend shown, we could assume that CO-tree outperforms CF-tree on bigger sizes. On the other hand, BCCO-tree was much worse on 2 15 and became similar to CO-tree on 2 21 . This happened because the races for the locks become more unlikely. This helped much to BCCO-tree, because it uses high-grained locking. On bigger sizes we expect that our implementation will continue to perform similarly to BCCO-tree, because the difference in our and BCCO-tree implementations is only in grabbing locks. By that, we can state that our algorithm works well not depending on the size. Finally, in most workloads our algorithm seems to perform better than other trees.

Conclusion Measuring Concurrency

Measuring concurrency via comparing a concurrent data structure to its sequential counterpart was originally proposed [START_REF] Gramoli | From sequential to concurrent: correctness and relative efficiency (brief announcement)[END_REF]. The metric was later applied to construct a concurrencyoptimal linked list [START_REF] Gramoli | Brief announcement: a concurrency-optimal list-based set[END_REF], and to compare synchronization techniques used for concurrent search data structures, organizing nodes in a directed acyclic graph [START_REF] Gramoli | In the search for optimal concurrency[END_REF]. Although lots of efforts have been devoted to improve the performance of BSTs as under growing concurrency, to our knowledge, the existence of a concurrency-optimal BST has not been earlier addressed.

Search for Concurrency-Optimal Data Structures

Concurrent BSTs have been studied extensively in literature; yet by choosing to focus on minimizing the amount of synchronization, we identified an extremely high-performing concurrent BST implementation. We proved our implementation to be formally correct and established the concurrency-optimality of our algorithm. Apart from the intellectual merit of understanding what it means for an implementation to be highly concurrent, our findings suggest a relation between concurrency-optimality and efficiency. We hope this work will inspire the design of other concurrency-optimal data structures that currently lack efficient implementations.

6 Parallel Combining: Benefits of Explicit Synchronization

Introduction

Efficient concurrent data structures have to balance parallelism and synchronization. Parallelism implies performance, while synchronization maintains consistency. Efficient "concurrency-friendly" data structures (e.g., sets based on linked lists [START_REF] Harris | A pragmatic implementation of non-blocking linked-lists[END_REF][START_REF] Heller | A lazy concurrent list-based set algorithm[END_REF] or binary search trees [START_REF] Crain | A contention-friendly binary search tree[END_REF][START_REF] Drachsler | Practical concurrent binary search trees via logical ordering[END_REF]) are conventionally designed using hand-crafted fine-grained locking. In contrast, "concurrency-averse" data structures (e.g., staks and queues) are subject to frequent sequential bottlenecks and solutions based on combining (e.g., [START_REF] Hendler | Flat combining and the synchronization-parallelism tradeoff[END_REF]), where all requests are synchronized and applied sequentially, perform surprisingly well compared to fine-grained ones [START_REF] Hendler | Flat combining and the synchronization-parallelism tradeoff[END_REF]. Typically, a general data structure combines features of "concurrency-friendliness" and "concurrency-averseness", and an immediate question is how to implement it in the most efficient way.

In this paper, we suggest a methodology of building a concurrent data structure from its parallel batched counterpart [START_REF] Agrawal | Provably good scheduling for parallel programs that use data structures through implicit batching[END_REF]. A parallel batched data structure applies a batch (set) of operations using parallelism. The algorithm distributes the work between the processes assuming the synchronized application of operations that reduces the number of possible interleavings. As a result, designing parallel batched algorithms is much easier than their concurrent counterparts.

In the technique proposed here, we explicitly synchronize concurrent operations, assemble them into batches, and apply these batches on an emulated parallel batched data structure. Processes share a set of active requests using any combining algorithm [START_REF] Drachsler-Cohen | Lcd: Local combining on demand[END_REF]: one of the active processes becomes a combiner and forms a batch from the requests in the set. Under the coordination of the combiner, the owners of the collected requests, called clients, apply the requests in the batch to the parallel batched data structure.

This technique becomes handy when the overhead on explicit synchronization of processes is compensated by the advantages of involving clients into the combining procedure using the parallel batched data structure. In the extreme case of concurrency-averse data structures, such as queues and stacks, having control over the batch can be used to eliminate certain requests and bypass sequential bottlenecks, even though there are no benefits of parallelizing the execution.1 But as we show in the paper, combining and parallel batching pay off for data structures that offer some degree of parallelism, such as dynamic graphs and priority queues.

We discuss three applications of parallel combining and we experimentally validate the benefits for two of them. First, we design concurrent implementations optimized for readdominated workloads given a sequential data structure. Intuitively, updates are performed sequentially and read-only operations are performed by the clients in parallel under the coordination of the combiner. In our performance analysis, we considered a dynamic graph data structure [START_REF] Holm | Poly-logarithmic deterministic fully-dynamic algorithms for connectivity, minimum spanning tree, 2-edge, and biconnectivity[END_REF] that can be accessed for adding and removing edges (updates), as well as for checking connectivity between pairs of vertices (read-only). Second, we apply parallel combining to the parallel batched binary search tree algorithm by Blelloch et al. [START_REF] Guy E Blelloch | Just join for parallel ordered sets[END_REF] in Section 3.1. Our concurrent data structure performs worse than the state-of-the-art implementations, since the operations on them are not likely to contend. Despite this, our implementation provides theoretical bounds on the height of the tree and the running time of operations in RMRs. Finally, we apply parallel combining to priority queue that is subject to sequential bottlenecks for minimal-element extractions, while most insertions can be applied concurrently. As a side contribution, we propose a novel parallel batched priority queue, as no existing batched priority queue we are aware of can be efficiently used in our context. Our perfomance analysis shows that implementations based on parallel combining may outperform state-of-the-art algorithms.

Roadmap

The rest of the chapter is organized as follows. In Section 6.2, we outline parallel combining and provide several applications. In Section 6.3, we present the parallel batched binary search tree by Blelloch et al. [START_REF] Guy E Blelloch | Just join for parallel ordered sets[END_REF] in a form convenient for parallel combining. In Section 6.4, we present a novel parallel batched priority queue in a form convenient for parallel combining. In Section 6.5, we report the results of experiments. In Section 6.6, we overview the related work. We conclude in Section 6.7.

Parallel Combining and Applications

In this section, we describe the parallel combining technique in a parameterized form: the parameters are specified depending on the application.

Combining Data Structure

Our technique relies on a combining data structure C (e.g., [START_REF] Hendler | Flat combining and the synchronization-parallelism tradeoff[END_REF]) that maintains a set of requests to data structure and determines which process is a combiner. If the set of requests is not empty then exactly one process should be a combiner.

Elements stored in C are of Request type consisting of the following fields: 1) the method to be called and its input; 2) the response field; 3) the status of the request with a value from an application-specific STATUS_SET; 4) application-specific auxiliary fields. In our applications STATUS_SET contains at least INITIAL and FINISHED values: INITIAL is set during the initialization and FINISHED means that the request is served.

C supports three operations: 1) addRequest(r : Request) inserts request r into the set and returns whether this process becomes a combiner or a client; 2) getRequests() returns a non-empty set of requests; and 3) release() is issued by the combiner to make C find another process to be a combiner.

In the rest of the paper we assume any black-box implementation of C [START_REF] Drachsler-Cohen | Lcd: Local combining on demand[END_REF][START_REF] Fatourou | Revisiting the combining synchronization technique[END_REF][START_REF] Hendler | Flat combining and the synchronization-parallelism tradeoff[END_REF][START_REF] Oyama | Executing parallel programs with synchronization bottlenecks efficiently[END_REF].

Specifying Parameters

To perform an operation, a process executes the following steps (Figure 6.1): 1) it inserts the request into C using addRequest(•) and checks whether it becomes a combiner; 2) if it is the combiner, it collects requests from C using getRequests(), then it executes algorithm COMBINER_CODE, and, finally, calls release() to enable another active process to become a combiner; 3) if the process is a client, it waits until the status of the request is not INITIAL and, then, executes algorithm CLIENT_CODE.

To use our technique, one should therefore specify COMBINER_CODE, CLIENT_CODE, and appropriately modify Request type and STATUS_SET.

Note that sequential combining [START_REF] Drachsler-Cohen | Lcd: Local combining on demand[END_REF][START_REF] Fatourou | Revisiting the combining synchronization technique[END_REF][START_REF] Hendler | Flat combining and the synchronization-parallelism tradeoff[END_REF][START_REF] Oyama | Executing parallel programs with synchronization bottlenecks efficiently[END_REF] is a special case of parallel combining: we simply need to use the proper algorithm behind black box C. Now we discuss two interesting applications of parallel combining.

Read-optimized Concurrent Data Structures

Parallel combining can boost a sequential data structures under read-dominated workloads, i.e., when most operations do not modify the data structure. Such operations are called Suppose that we are given a sequential data structure D that suports update and readonly operations. Now we explain how to set parameters of parallel combining for this application. At first, STATUS_SET consists of three elements INITIAL, STARTED and FINISHED. Request type does not have auxiliary fields.

In COMBINER_CODE (Figure 6.2 Line 1), the combiner iterates through the set of collected requests A: if a request contains an update operation then the combiner executes it and sets its status to FINISHED; otherwise, the combiner adds the request to set R. Then the combiner sets the status of requests in R to STARTED. After that the combiner checks whether its own request is read-only. If so, it executes the method and sets the status of its request to FINISHED. Finally, the combiner waits until the status of the requests in R become FINISHED.

In CLIENT_CODE (Figure 6.2 Line 21), the client checks whether its method is read-only. If so, the client executes the method and sets the status of the request to FINISHED. Theorem 6.2.1. Algorithm in Figure 6.2 produces a linearizable concurrent data structure from a sequential one.

Proof. Any execution can be split into combining phases (Figure 6.2 Lines 2-19) which do not intersect. We can group the operations into batches by the combining phase in which they are applied.

Each update operation is linearized at the point when the combiner applies this operation. Note that this is a correct linearization since all operations that are linearized before are already applied: the operations from preceding combining phases were applied during the preceding phases, while the operations from the current combining phase are applied sequentially by the combiner.

Each read-only operation is linearized at the point when the combiner sets the status of the corresponding request to STARTED. By the algorithm, a read-only operation observes all update operations that are applied before and during the current combining phase. Thus, the chosen linearization is correct.

To evaluate the approach we apply it to the sequential dynamic graph data structure by Holm et al. [START_REF] Holm | Poly-logarithmic deterministic fully-dynamic algorithms for connectivity, minimum spanning tree, 2-edge, and biconnectivity[END_REF] (Section 6.5.1).

Parallel Batched Algorithms

We discuss below how to build a concurrent implementation given a parallel batched one in one of two forms: for static or dynamic multithreading.

Suppose that we are given a parallel batched implementation for dynamic multithreading. One can turn it into a concurrent one using parallel combining with the work-stealing scheduler. We enrich STATUS_SET with STARTED. In COMBINER_CODE the combiner collects the requests and sets their status to STARTED. Then the combiner creates a working deque, puts there a new node of computational DAG with apply function and starts the workstealing routine on processes-clients. Finally, the combiner waits for the clients to become FINISHED. In CLIENT_CODE the client creates a working deque and starts the work-stealing routine.

In the static multithreading case, each process is provided with a distinct version of apply function. Again, we enrich STATUS_SET with STARTED. In COMBINER_CODE the combiner collects the requests, sets their status to STARTED, performs the code of apply and waits for the clients to become FINISHED. In CLIENT_CODE the client waits until its request has STARTED status, performs the code of apply and sets the status of its request to FINISHED.

We apply this technique to the binary search tree and to the priority queue. For the binary search tree we present the parallel batched algorithm by Blelloch et al. [START_REF] Guy E Blelloch | Just join for parallel ordered sets[END_REF] in a form of COMBINER_CODE and CLIENT_CODE. For the priority queue we introduce a novel parallel batched implementation in a form of COMBINER_CODE and CLIENT_CODE. We had to design a new algorithm since no known implementation [START_REF] Gerth | A parallel priority queue with constant time operations[END_REF][START_REF] Deo | Parallel heap: An optimal parallel priority queue[END_REF][START_REF] Pinotti | Parallel priority queues[END_REF][START_REF] Sanders | Randomized priority queues for fast parallel access[END_REF] can be efficiently used in our context: their complexity inherently depends on the total number of processes in the system, regardless of the actual batch size.

Binary Search Tree with Parallel Combining

In this section, we explain how to implement a parallel batched binary search tree algorithm by Blelloch et al. [START_REF] Guy E Blelloch | Just join for parallel ordered sets[END_REF] in the form of COMBINER_CODE and CLIENT_CODE for the parallel combining framework described in the previous section.

Initially, we choose a sequential balanced binary search tree implementation that supports four operations with logarithmic complexity: 1. T ← join(T L , k, T R ) -takes two trees T L and T R , and a value k, and returns a new tree for which the in-order values are a concatenation of the in-order values of T L , k, and the in-order of T R .

2. T ← join2(T L , T R ) -takes two trees T L and T R , and returns a new tree for which the in-order values are a concatenation of the in-order values of T L and T R .

3. (T L , b, T R ) ← split(T , k) -takes a tree T and a value k, and returns two trees T L and T R , and the boolean b, where T L contains all the values of T that are less than k, T R contains all the values of T that are greater than k, and b represents whether k is in T .

4. b ← contains(T , k) -takes a tree T and a value k, and returns if k is in T .

As shown in the paper by Blelloch et al. [START_REF] Guy E Blelloch | Just join for parallel ordered sets[END_REF] at least four state-of-the-art sequential implementations satisfy the above requirements: AVL [START_REF] Adelson | An algorithm for the organization of information[END_REF], Red-Black [START_REF] Leo | A dichromatic framework for balanced trees[END_REF], Weight-balanced [START_REF] Nievergelt | Binary search trees of bounded balance[END_REF] and Treap [START_REF] Seidel | Randomized search trees[END_REF]. In our parallel batched implementation any sequential tree is represented by an object of Tree class. This class together with the functions described above appears in Figure 6.3 Lines 1-25.

We briefly describe the parallel batched algorithm by Blelloch et al. [START_REF] Guy E Blelloch | Just join for parallel ordered sets[END_REF]. It is implemented as one recursive function apply that takes an interval of requests S[l..r] and a sequential binary search tree T , and applies the requests to the tree. apply(S[l..r], T ) works in three stages: (1) splits T by the argument of the middle request S[mid], where mid = l+r 2 , into left T L and right T R trees; (2) calls fork-join with two branches: apply(S[l..mid -1], T L ) that returns tree T L and apply(S[mid + 1..r], T R ) that returns tree T R ; and, finally, (3) joins two trees T L and T R , possibly, with the argument of S[mid] if it is insert. In order to apply all requests we should call root ← apply(S[1..|S|], root), where root stores the tree.

This algorithm works correctly: if an Insert request has argument v then the resulting tree contains v since we join trees with v; if a Delete request has argument v then the resulting tree does not contain v since we do not join trees with v; and the resulting tree is perfectly balanced. The algorithm takes O(c • log( m c + 1)) work and O(log c • log m) span to apply a batch of size c to a tree of size m. Now we describe the high-level idea behind the transformation from the described parallel batched data structure into the version written for parallel combining. The main difference between the algorithms is that we wake up clients instead of calling fork-join. Suppose that we have requests S .|S|] and tree root. We present the code of the algorithm. The code for necessary classes is presented in Figure 6.3, COMBINER_CODE is presented in Figure 6.4 and CLIENT_CODE is presented in Figure 6.5.

STATUS_SET consists of five elements INITIAL, CONTAINS, CONTAINS_FINISHED, UPDATE and FINISHED.

A Request object (Figure 6.3 Lines 30-39) consists of: a method method to be called (Insert, Delete or Contains) and its input argument v; a result res field; a status field; a tree tree to which the client-owner c of this request should apply requests; a lef tR request of the client that is awaken by c on the left part of requests; a rightR request of the client that is awaken by c on the right part of requests.

The purpose of the last three fields will be clear from the code.

Contains Phase

Combiner: preparation (Figure 6.4 Lines 11-19). First, the combiner withdraws requests A from the combining data structure C (Line 11) and sorts them by their arguments (Line 13). Then it copies all update operations from A into set U and sets the status of all requests in A to CONTAINS (Lines 16-19). Clients (Figure 6.5 Lines 2-6). Each client finds whether its argument appears in the tree (Line 2) and stores the result in res field of its request. Then if the request is contains (Line 3), the client sets the status of its request to FINISHED (Line 4) and ends

1 class Tree: the execution. Otherwise, the request is an update and the client sets the status of its request to CONTAINS_FINISHED (Line 6) in order to signal the combiner that it finished the contains phase. The combiner runs the same code as clients (Figure 6.4 Lines 21-26). Then it considers the contains phase finished when all requests in A changes their state from CONTAINS either to FINISHED (contains requests) or to CONTAINS_FINISHED (update requests) (Lines 28-30).

Update Phase

Combiner: preparation (Figure 6.4 Lines 32-68). As the first step (Lines 32-64), the combiner computes the results of each request and chooses at most one request for each value of the argument to apply. The combiner iterates through batches of update requests in U grouped by argument (Lines 38-51), i.e., U[l..r -1]. For each batch the combiner reads the current state of the argument (Line 36): whether it is present in the tree. Then it logically applies requests with this argument one by one and calculates their responses. Finally, the combiner chooses at most one request that should be applied to the tree and adds it to set S (Lines 53-62): it is the last successful request, i.e., returns true. All requests that are not in S can be released and their status are set to FINISHED. Now the combiner prepares the requests in S to be applied to the binary search tree. As discussed earlier in our algorithm client c = C[ l+r 2 ] that applies S[l..r] wakes up two clients L on the left half and R on the right half of requests: in Line 66, the combiner prefills the fields leftR and rightR of the request of c with the requests of L and R, respectively. Finally, the combiner passes the tree argument to the first working client C[ 1+|S| 2 ], i.e., the client that owns the middle request, by writing the tree into tree field of the request (Line 67) and, then, wakes up that client by setting its status to UPDATE (Line 68). Clients (Figure 6.5 Lines 8-12). Each client waits until its status changes from CONTAINS_FINISHED (Lines 8-9). If the new status is FINISHED then the client finishes its execution, otherwise, To prove that this linearization is correct it is enough to show for each different argument that the operations with that argument are linearizable. Consider operations that have the same argument v. The results of these operations definitely satisfy the sequential execution since we apply them logically in the sorted order knowing whether v is in the tree prior to this combining phase. (Figure 6.4 Lines 32-64) The only thing left to show is that the resulting tree after the i-th combining phase contains v if the last successful operation is an insert, and does not have, otherwise. This directly follows from the algorithm since we apply to the tree only the last successful operation: if it is insert then the value is inserted into the tree (Figure 6.5 Line 42), otherwise, the value is not inserted (Figure 6.5 Line 44). Finally, since we join and split trees using the sequential algorithm -the resulting tree is perfectly balanced. The combiner waits for the change of the status of the clients (O(c) RMRs, Line 28-30). Then it sets the responses of the clients and choose the requests to apply (O(c) RMRs, Lines 32-64). After that, the combiner fills the dependencies between requests (O(c) RMRs, Line 66) and wakes up the first working client (O(1) RMRs, Lines 67-68).

The clients and the combiner participate in Update phase. The client waits for its status to change (1 RMR, Lines 8-9). Then it splits the tree (O(log m) primitive steps and RMRs, Line 20), wakes up two clients (O(1) RMRs, Lines 22-27), waits until they finish and change their status (O(1) RMRs, Lines 29-39) and, finally, joins the resulting trees, possibly, with the argument of its request (O(log m) primitive steps and RMRs, Lines [START_REF] Brown | A general technique for non-blocking trees[END_REF][START_REF] Budimlić | The design and implementation of the habanerojava parallel programming language[END_REF][START_REF] Calciu | The adaptive priority queue with elimination and combining[END_REF][START_REF] Censor-Hillel | Help![END_REF][START_REF] Chakrabarti | R-mat: A recursive model for graph mining[END_REF][START_REF] Manuel Mt Chakravarty | Data parallel haskell: a status report[END_REF].

The combiner waits the first working client to finish (O(1) RMRs, Line 77-79) and ends the execution.

Summing up, the combiner performs O(c + log m) RMRs and each client performs O(log m) RMRs, giving us in total O(c • log m) RMRs. Remark 6.3.1. The above bounds also hold in DSM model for the version of the described algorithm. For that we have to simply make spin-loops to loop on the local variables of process. Luckily for us, in our algorithm the purpose of each spin-loop is to wake up some process and at each point when we set the variable we know (or can deduce by a simple modification of the algorithm) which process is going to wake up. Thus, it is enough for each spin-loop to create a separate variable in the memory of the target process. Such transformation (in reality, it is slightly more technical than described above) of our algorithm provides an algorithm with the same bounds on RMRs but in DSM model.

Priority Queue with Parallel Combining

Before we introduce our novel parallel batched priority queue algorithm we have to mention that all previous parallel batched algorithms (see Section 3.3) do not satisfy us because 1 class InsertSet: they are tailored to a fixed number of processes p. Such a restriction leads to two major issues. Each of the known algorithm has a constraint on the size of batch: the algorithm by Pinotti and Pucci [START_REF] Pinotti | Parallel priority queues[END_REF] allows only batches with size p, the algorithm by Deo and Prasad [START_REF] Deo | Parallel heap: An optimal parallel priority queue[END_REF] allows only batches with size not exceeding p, and, finally, the algorithm by Sanders [START_REF] Sanders | Randomized priority queues for fast parallel access[END_REF] allows a batch to have no more than p extractMin operations. Thus it is not possible to introduce new processes to the system and use them efficiently, when the data structure is already constructed, and, consequently, p is chosen to be n, the total number of processes in the system. Secondly, we expect a parallel batched implementation to be work-efficient: it should be able to apply a batch of size c to a priority queue of size m in a time not worse than O(c log m). Unfortunately, the known algorithms are not work-efficient when the size of the batch is always less than p (which is the common case in our environment, since p is chosen to be n): the algorithm by Pinotti and Pucci [START_REF] Pinotti | Parallel priority queues[END_REF] induces O(p • (log m + log log p)) work, the algorithm by Deo and Prasad [START_REF] Deo | Parallel heap: An optimal parallel priority queue[END_REF]

Combiner and Client. Classes

Now, we describe our novel parallel batched priority queue in the form of COMBINER_CODE and CLIENT_CODE that fits the parallel combining framework described in Section 6.2. It is based on the sequential binary heap by Gonnet and Munro [START_REF] Gaston | Heaps on heaps[END_REF] described in Section 3.3. The code of necessary classes is presented in Figure 6.6, COMBINER_CODE is presented in Figure 6.7, and CLIENT_CODE is presented in Figure 6.8.

We introduce a sequential object InsertSet (Figure 6 Note that by the algorithm S.B contains only values that were stored in the nodes above node v, thus, any value in S.B cannot be bigger than a [v].val and after appending a [v].val S.B remains sorted. Then the client calculates the number inL of target nodes in the subtree of the left child of v and the number inR of target nodes in the subtree of the right child of v (Lines 63-66, to calculate these numbers in constant time we use fields l and r of the request). If inL = 0, then all the values in S should be inserted into the subtree of the right child of v, and c proceeds with the right child 2v + 1 (Lines 69-70). If inR = 0, then, symmetrically, c proceeds with the left child 2v (Lines 73-74). Otherwise, if inL = 0 and inR = 0, v is a split node and, thus, there is a client that waits at the right child 2v + 1. Hence, c splits S to (X, Y) ← S.split(inL) (Line 77): the values in X should be inserted into the subtree of node 2v and the values in Y should be inserted into the subtree of node 2v + 1. Then c sets a[2v + 1].split to Y , sets S to X and proceeds to node 2v (Lines 78-80). When c reaches a leaf v it sets the value a [v].val to the only value in S (Lines 82-85) and sets the status of the request req to FINISHED (Line 7).

If the request of the combiner is an incompleted Insert, it runs the code above as a client (Figure 6.7 Lines 94-95). The combiner considers the Insert phase completed when all requests in I have status FINISHED (Lines 97-99).

Analysis

Now we provide theorems on correctness and time bounds. Theorem 6.4.1. Our concurrent priority queue implementation is linearizable.

Proof. The execution can be split into combining phases (Figure 6.7 Lines 1-99) which do not intersect. We group the operations into batches corresponding to the combining phase in which they are applied.

Consider the i-th combining phase. We linearize all the operations from the i-th phase right after the end of the corresponding getRequests() (Line 1) in the following order: at first, we linearize ExtractMin operations in the increasing order of their responses, then, we linearize Insert operations in any order.

To see that this linearization is correct it is enough to prove that the combiner and the clients apply the batch correctly. Lemma 6.4.1. Suppose that the batch of the i-th combining phase contains a ExtractMin operations and b Insert operations with arguments x 1 , . . . , x b . Let V be the set of values stored in the priority queue before the i-th phase. The combiner and the clients apply this batch correctly:

• The minimal a values y 1 , . . . , y a in V are returned to ExtractMin operations.

• After an execution the set of values stored in the queue is equal to V ∪ {x 1 , . . . , x b } \ {y 1 , . . . , y a }. and the values are stored in nodes with identifiers 1, . . . , |V | -b + a.

• After an execution the heap property is satisfied for each node.

Proof. The first statement is correct, because the combiner chooses the smallest a elements from the priority queue and sets them as the results of ExtractMin requests (Lines 18-33).

The second statement about the set of values straightforwardly follows from the algorithm. During ExtractMin phase the combiner finds a smallest elements, replaces them with x 1 , . . The third statement is slightly tricky. At first, the combiner finds a smallest elements that should be removed and replaces them with x 1 , . . . , x min(a,b) and with values from the last a -min(a, b) nodes of the heap. Suppose that these a smallest elements were at nodes v 1 , . . . , v a , sorted by their depth (the length of the shortest path from the root) in nonincreasing order. These nodes form a connected subtree where v a is the root of the heap. Suppose that they do not form a connected tree or v a is not the root of the heap. Then there exists a node v i which parent p is not v j for any j. This means that the values in nodes v 1 , . . . , v a are not the smallest a values: by the heap property a value at p is smaller than the value at v i . Now a processes perform sift down from the nodes v 1 , . . . , v a . We show that when a node v is unlocked, i.e., its locked field is set to false, the value at v is the smallest value in the subtree of v. This statement is enough to show that the heap property holds for all nodes after ExtractMin phase, because at that point all nodes are unlocked.

Consider an execution of sift down. We prove the statement by induction on the number of unlock operations. Base. No unlock happened and the statement is satisfied for all unlocked nodes, i.e., all the nodes except for v 1 , . . . , v a . Transition. Let us look right before the k-th unlock: the unlock of a node v. The left child l of v should be unlocked and, thus, l contains a value that is the smallest in its subtree. The same statement holds for the right child r of v. v chooses the smallest value between the value at v and the values at l and r. This value is the smallest in the subtree of v. Thus, the statement holds for v when unlocked.

After that, the algorithm applies the incompleted b -min(a, b) Insert operations. We name the nodes with at least one target node in the subtree as modified. Modified nodes are the only nodes whose value can be changed and, also, each modified node is visited by exactly one client. To prove that after the execution the heap property for each modified node holds: we show by induction on the depth of a modified node that if a node v is visited by a client with InsertSet S then: (1) S.A is sorted; (2) S.B is sorted and contains only values that were stored in ancestors of v after ExtractMin phase; and (3) v contains the smallest value in its subtree when the client finishes with it. Base. In the root S.A is sorted, S.B is empty and the new value in the root is either the first value in S.A or the current value in the root, thus, it is the smallest value in the heap. To get the improved bound we carefully calculate the total number of RMRs spent on the splits of InsertSets in Line 77. This number equals to the number of values that are moved to newly created sets during the splits. For simplicity we suppose that inserted values are bigger than all the values in the priority queue and, thus, each InsertSet contains only the newly inserted values. This assumption does not affect the bound. Consider now the inserted value v. Suppose that v was moved k times and at the i-th time it was moved during the split of InsertSet with size s i . Because v is moved during split only to the set with the smaller size: Remark 6.4.1. The above bounds also hold in DSM model for the version of the described algorithm. For that we have to simply make spin-loops to loop on the local variable of processes. In our algorithm the purpose of each spin-loops is to wake up some process. At most places in our algorithm when we set the variable on which we spin we know (or can deduce by a simple modification of the algorithm) which process is going to wake up. For each spin-loop it is enough to create a separate variable in the memory of the target process.

s 1 ≥ 2 • s 2 ≥ . . . ≥ 2 k-1 • s k . k is less than log c, because s 1 ≤ c,
The only two non-trivial spin-loops are in CLIENT_CODE (Lines [START_REF] Agrawal | Provably good scheduling for parallel programs that use data structures through implicit batching[END_REF][START_REF] Aksenov | On helping and stacks[END_REF][START_REF] Aksenov | A concurrency-optimal binary search tree[END_REF][START_REF] Aksenov | Brief-announcement: Performance prediction of coarse-grained programs[END_REF][START_REF] Aksenov | Parallel combining: Benefits of explicit synchronization[END_REF] where we do not know a process that is going to wake up. To elliviate this issue we expand each Node object with the pointer to process proc. When the process wants to sift-down, first, it registers itself in a [v] Note that the total number of notify local variables that is needed by each process is logarithmic from the size of the queue.

The described transformation (in reality, it is slightly more technical than described above) of our algorithm provides an algorithm with the same bounds on RMRs but in DSM model.

Experiments

We evaluate Java implementations of our data structures on a 4-processor AMD Opteron 6378 2.4 GHz server with 16 threads per processor (yielding 64 threads in total), 512 Gb of RAM, running Ubuntu 14.04.5 with Java 1.8.0_111-b14 and HotSpot JVM 25.111-b14.

Concurrent Dynamic Graph

To illustrate how parallel combining can be used to construct read-optimized concurrent data structures, we took the sequential dynamic graph implementation by Holm et al. [START_REF] Holm | Poly-logarithmic deterministic fully-dynamic algorithms for connectivity, minimum spanning tree, 2-edge, and biconnectivity[END_REF]. This data structure supports two update methods: an insertion of an edge and a deletion of an edge; and one read-only method: a connectivity query that tests whether two vertices are connected.

We compare our implementation based on parallel combining (PC) against three others: (1) Lock, based on ReentrantLock from java.util.concurrent; (2) RW Lock, based on Reen-trantReadWriteLock from java.util.concurrent; and (3) FC, based on flat combining [START_REF] Hendler | Flat combining and the synchronization-parallelism tradeoff[END_REF]. The code is available at https://github.com/Aksenov239/concurrent-graph.

We consider workloads parametrized with: 1) the fraction x of connectivity queries (50%, 80% or 100%, as we consider read-dominated workloads); 2) the set of edges E: edges of a single random tree, or edges of ten random trees; 3) the number of processes P (from 1 to 64). We prepopulate the graph on 10 5 vertices with edges from E: we insert each edge with probability 1 2 . Then we start P processes. Each process repeatedly performs operations: 1) with probability x, it calls a connectivity query on two vertices chosen uniformly at random; 2) with probability 1 -x 2 , it inserts an edge chosen uniformly at random from E; 3) with probability 1-x 2 , it deletes an edge chosen uniformly at random from E.

We denote the workloads with E as a single tree as Tree workloads, and other workloads as Trees workloads. Tree workloads are interesting because they show the degenerate case: the dynamic graph behaves as a dynamic tree. In this case, about 50% of update operations successfully change the spanning tree, while other update operations only check the existence of the edge and do not modify the graph. Trees workloads are interesting because a reasonably small number (approximately, 5-10%) of update operations modify the set of all edges and the underlying complex data structure that maintains a spanning forest (giving in total the squared logarithmic complexity), while other update operations only can modify the set of edges and cannot modify the underlying complex data structure (giving in total the logarithmic complexity).

For each setting and each algorithm, we run the corresponding workload for 10 seconds to warmup HotSpot JVM and then we run the workload five more times for 10 seconds. The average throughput of the last five runs is reported in Figure 6. [START_REF] Sarita | Shared memory consistency models: A tutorial[END_REF].

From the plots we can infer two general observations: PC exhibits the highest thoughput over all considered implementations and it is the only one whose throughput scales up with the number of the processes. On the 100% workload we expect the throughput curve to be almost linear since all operations are read-only and can run in parallel. The plots almost confirm our expectation: the curve of the throughput is a linear function with coefficient 1 2 (instead of the ideal coefficient 1). We note that this is almost the best we can achieve: a combiner typically collects operations of only approximately half the number of working processes. In addition, the induced overhead is still perceptible, since each connectivity query works in just logarithmic time. With the decrease of the fraction of read-only operations we expect that the throughput curve becomes flatter, as plots for the 50% and 80% workloads confirm.

It is also interesting to point out several features of other implementations. At first, FC implementation works slightly worse than Lock and RW Lock. This might be explained as follows. Lock implementations (ReentrantLock and ReentrantReadWriteLock) behind Lock and RWLock implementations are based on CLH Lock [START_REF] Craig | Building fifo and priorityqueuing spin locks from atomic swap[END_REF] organized as a queue: every competing process is appended to the queue and then waits until the previous one releases the lock. Operations on the dynamic graph take significant amount of time, so under high load when the process finishes its operation it appends itself to the queue in the lock without any contention. Indeed, all other processes are likely to be in the queue and, thus, no process can contend. By that the operations by processes are serialized with almost no overhead. In contrast, the combining procedure in FC introduces non-negligible Second, it is interesting to observe that, against the intuition, RWLock is not so superior with respect to Lock on read-only workloads. As can be seen, when there are update operations in the workload RWLock works even worse than Lock. We relate this to the fact that the overhead hidden inside ReentrantReadWriteLock spent on manipulation with read and write requests is bigger than the overhead spent by ReentrantLock. With the increase of the percentage of read-only operations the difference between Lock and RWLock diminishes and RWLock becomes dominant since read operations become more likely to be applied concurrently (for example, on 50% it is normal to have an execution without any parallelization: read operation, write operation, read operation, and so on). However, on 100% one could expect that RWLock should exhibit ideal throughput. Unfortunately, in this case, under the hood ReentrantReadWriteLock uses compare&swap on the shared variable that represents the number of current read operations. Read-only operations take enough time but not enough to amortize the considerable traffic introduced by concurrent compare&swaps. Thus, the plot for RWLock is almost flat, getting even slightly worse with the increase of the number of processes, and we blame the traffic for this.

Binary Search Tree

We implement our algorithm using AVL binary search tree as the sequential implementation, since it seems to be the fastest. Then we compare it with the state-of-the-art concurrent binary search trees with relaxed AVL-balancing scheme [START_REF] Nathan G Bronson | A practical concurrent binary search tree[END_REF][START_REF] Crain | A contention-friendly binary search tree[END_REF][START_REF] Drachsler | Practical concurrent binary search trees via logical ordering[END_REF].

We consider typical workloads parametrized by: 1) the fraction x of update queries (0%, 20%, 100%), 2) the key range R ([0, 2 • 10 6 ] or [0, 2 • 10 7 ]), and 3) the number of processes P (from 1 to 64). We prepopulate the tree with values from the key range: we add a value with probability 1 2 . Then we start P processes. Each process repeatedly performs operations: 1) with probability 1 -x, it calls a contains query given a value chosen uniformly at random from the range; 2) with probability x 2 , it inserts a value We do not show here the plots, but we give the main conclusion: our implementation works worse than others by up to 7 times. This is expected, since when the number of concurrent operations on binary search tree is moderately small in comparison to the size of the tree, the operations are performed almost without data races. Suppose that there are no data races at all, then we can implement a parallel batched algorithm that calls operations from a batch concurrently giving O(log m) span, where m is the size of the tree. At the same time, the parallel batched algorithm by Blelloch et al. [START_REF] Guy E Blelloch | Just join for parallel ordered sets[END_REF] has O(log c • log m) span, where c is the size of the batch and m is the size of the queue. Thus, we can expect the parallel batched algorithm to work log c ≈ 5 times slower even not considering the explicit synchronization.

To summarize, the binary search tree is an example of a data structure that does not benefit from parallel combining on small batches (proportional to the number of processes): the synchronization used by the concurrent algorithm is smaller than the synchronization used by the parallel batched algorithm.

Priority Queue

We run our algorithm (PC) against six state-of-the-art concurrent priority queues: (1) the lock-free skip-list by Linden and Johnson (Linden SL [START_REF] Lindén | A skiplist-based concurrent priority queue with minimal memory contention[END_REF]), (2) the lazy lock-based skiplist (Lazy SL [START_REF] Herlihy | The art of multiprocessor programming[END_REF]), (3) the non-linearizable lock-free skip-list by Herlihy and Shavit (SkipQueue [START_REF] Herlihy | The art of multiprocessor programming[END_REF]) as an adaptation of Lotan and Shavit's algorithm [START_REF] Shavit | Skiplist-based concurrent priority queues[END_REF], (4) the lockfree skip-list from Java library (JavaLib), ( 5) the binary heap with flat combining (FC Binary [START_REF] Hendler | Flat combining and the synchronization-parallelism tradeoff[END_REF]), and (6) the pairing heap with flat combining (FC Pairing [START_REF] Hendler | Flat combining and the synchronization-parallelism tradeoff[END_REF]). 2 The code is available at https://github.com/Aksenov239/FC-heap.

We consider workloads parametrized by: 1) the initial size S (8 • 10 5 or 8 • 10 6 ); and 2) the number P of working processes (from 1 to 64). We prepopulate the queue with S random integers chosen uniformly from the range [0, 2 31 -1]. Then we start P processes, and each process repeatedly performs operations: with equal probability it either inserts a random value taken uniformly from [0, 2 31 -1] or extracts the minimum value.

For each setting and each algorithm, we run the corresponding workload for 10 seconds to warmup HotSpot JVM and then we run the workload five more times for 10 seconds. The average throughput of the last five runs is reported in Figure 6.11.

On a small number of processes (< 15), PC performs worse than other algorithms.

With respect to Linden SL, Lazy SL, SkipQueue and JavaLib this can be explained by two different issues:

• Synchronization incurred by PC is not compensated by the work done;

• Typically, a combiner collects operations of only approximately half the processes, thus, we "overserialize", i.e., only n 2 operations can be performed in parallel. In contrast, on small number of processes, the other four algorithms can perform operations almost with no contention. With respect to algorithms based on flat combining, FC Binary and FC Pairing, our algorithm is simply slower on one process than the simplest sequential binary and pairing heap algorithms.

With the increase of the number of processes the synchronization overhead significantly increases for all algorithms (in addition to the fact that FC Binary and FC Pairing cannot scale). As a result, starting from 15 processes, PC outperforms all algorithms except for Linden SL. Linden SL relaxes the contention during ExtractMin operations, and it helps to keep the throughput approximately constant. At approximately 40 processes the benefits of the parallel batched algorithm in PC starts prevailing the costs of explicit synchronization, and our algorithms overtakes Linden SL.

It is interesting to note that FC Binary performs very well when the number of processes is small: the overhead on the synchronization is very small, the processes are from the same core and the simplest binary heap performs operations very fast.

Related Work

To the best of our knowledge, the first attempt to combine concurrent operations was introduced by Yew et al. [START_REF] Nian-Feng | Distributing hot-spot addressing in large-scale multiprocessors[END_REF]. They introduced a combining tree: processes start at the leaves and traverse upwards to gain exclusive access by reaching the root. If, during the traversal, two processes access the same tree node, one of them adopts the operations of another and continues the traversal, while the other stops its traversal and waits until its operations are completed. Several improvements of this technique have been discussed, such as adaptive combining tree [START_REF] Shavit | Combining funnels: a dynamic approach to software combining[END_REF], barrier implementations [START_REF] Gupta | A scalable implementation of barrier synchronization using an adaptive combining tree[END_REF][START_REF] Scott | Algorithms for scalable synchronization on shared-memory multiprocessors[END_REF] and counting networks [START_REF] Shavit | Diffracting trees[END_REF].

A different approach was proposed by Oyama et al. [START_REF] Oyama | Executing parallel programs with synchronization bottlenecks efficiently[END_REF]. Here the data structure is protected by a lock. A thread with a new operation to be performed adds it to a list of submitted requests and then tries to acquire the lock. The process that acquires the lock performs the pending requests on behalf of other processes from the list in LIFO order, and later removes them from the list. Its main drawback is that all processes have to perform CAS on the head of the list. The flat combining technique presented by Hendler et al. [START_REF] Hendler | Flat combining and the synchronization-parallelism tradeoff[END_REF] addresses this issue by replacing the list of requests with a publication list which maintains a distinct publication record per participating process. A process puts its new operation in its publication record, and the publication record is only maintained in the list if the process is sufficiently active. This way the process generally does not need to perform CAS on the head of the list. Variations of flat combining were later proposed for various contexts [START_REF] Drachsler-Cohen | Lcd: Local combining on demand[END_REF][START_REF] Fatourou | A highly-efficient wait-free universal construction[END_REF][START_REF] Fatourou | Revisiting the combining synchronization technique[END_REF][START_REF] Holt | Flat combining synchronized global data structures[END_REF].

Hierarchical combining [START_REF] Hendler | Scalable flat-combining based synchronous queues[END_REF] is the first attempt to improve the performance of combining using the computation power of clients. The list of requests is split into blocks, and each of these blocks has its own combiner. The combiners push the combined requests from the block into the second layer implemented as the standard flat combining with one combiner. This approach, however, may be sub-optimal as it does not have all clients participating. Moreover, this approach works only for specific data structures, such as stacks or unfair synchronous queues, where operations could be combined without accessing the data structure.

Agrawal et al. [START_REF] Agrawal | Provably good scheduling for parallel programs that use data structures through implicit batching[END_REF] decide to use a parallel batched data structure instead of a concurrent one in a different context. They provide provable bounds on the running time of a dynamic multithreaded parallel program using P processes and a specified scheduler. The proposed scheduler extends the work-stealing scheduler by maintaining separate batch work-stealing deques that are accessed whenever processes have operations to be performed on the abstract data type. A process with a task to be performed on the data structure stores it in a request array and tries to acquire a global lock. If succeeded, the process puts the task to perform the batch update in its batch deque. Then all the processes with requests in the request array run the work-stealing routine on the batch deques until there are no tasks left. The idea of [START_REF] Agrawal | Provably good scheduling for parallel programs that use data structures through implicit batching[END_REF] is similar to ours. However, our goals are different: we aim at improving the performance of a concurrent data structure while their goal was to establish bounds on the running time of a parallel program for dynamic multithreading. Implementing a concurrent data structure from its parallel batched counterpart for dynamic multithreading is only one of the applications of our parallel combining, as shown in Section 6.2.2.

Conclusion

In this chapter, we showed that parallel combining can provide performance gains and some theoretical guarantees. We can come up with another important advantage of our technique: it can enable the first ever concurrent data structures of some data types, that are efficient for requests of any type, either read-only or update. As a good example we can provide a dynamic tree -a data type that supports the same operations as the dynamic graph, but with the condition that the graph is always a forest. It does not have concurrent implementation but has a parallel batched one [START_REF] Umut A Acar | Brief announcement: Parallel dynamic tree contraction via self-adjusting computation[END_REF].

As shown in Section 6.5, our concurrent priority queue performs well compared to state-of-the-art algorithms. We assume that one of the reasons is that the underlying parallel batched implementation is designed for static multithreading and, thus, it has little synchronization overhead. This might not be the case for implementations based on dynamic multithreading, where the overhead induced by the scheduler can be much higher. We intend to check this in the forthcoming work.

On Helping and Stacks

Introduction

In a wait-free data structure, every process is guaranteed to make progress in its own speed, regardless of the behavior of other processes [START_REF] Herlihy | Wait-free synchronization[END_REF]. It has been observed, however, that achieving wait-freedom typically involves some helping mechanism (e.g., [START_REF] Fatourou | A highly-efficient wait-free universal construction[END_REF][START_REF] Feldman | A wait-free multi-word compare-and-swap operation[END_REF][START_REF] Peng | Fa-stack: A fast array-based stack with wait-free progress guarantee[END_REF][START_REF] Timnat | Wait-free linked-lists[END_REF]). Informally, helping means that a process may perform additional work on behalf of other processes. But how to define formally the phenomenon of helping? There are three attempts done up to date: the linearization-based helping by Censor-Hillerl et al. [START_REF] Censor-Hillel | Help![END_REF] and two, valency-based and universal, by Attiya et al. [START_REF] Attiya | Nontrivial and universal helping for wait-free queues and stacks[END_REF].

This chapter is devoted to the linearization-based helping, but at first we explain what makes us take helping into the consideration. Initially, we started with a question that was open for many years: does queue belong to Common2?

To explain the meaning of the question we provide several definitions. In the consensus problem, each process proposes a value and is required to decide on a value such that the following properties are satisfied in every execution:

• Termination. Every correct process decides.

• Agreement. No two processes decide on different values.

• Validity. The decided value was proposed by some process.

The consensus number of a primitive on a shared register [START_REF] Herlihy | Wait-free synchronization[END_REF] is the maximum number n such that it is possible to solve consensus on n processes in a wait-free manner from reads, writes and this primitive. For example, test&set, the primitive that retrieves the boolean value from the memory cell and sets the value in the cell to true, has consensus number 2. Finally, Common2 is the family of data types that have wait-free implementions from primitives with consensus number 2 for any number of processes.

It was proven by Afek et al. [START_REF] Afek | Common2 extended to stacks and unbounded concurrency[END_REF] that stack belongs to Common2: they provided a waitfree algorithm implemented with reads, writes and test&set primitives. For queue, known Common2 implementations work only in special cases: each process either only enqueues or dequeues, and the number of processes that dequeue [START_REF] Li | Non-blocking implementations of queues in asynchronous distributed shared-memory systems[END_REF] or the number of processes that enqueue [START_REF] Eisenstat | Two-enqueuer queue in common2[END_REF] should not exceed two.

But how the question about Common2 queue implementation is related to helping? Attiya et al. [START_REF] Attiya | Nontrivial and universal helping for wait-free queues and stacks[END_REF] searched for the wait-free implementations of stacks and queues that do not have the non-trivial valency-based helping and are implemented only using reads, writes and test&set primities, i.e., a relaxed version of Common2. They found that such an implementation exists for stack, but does not exist for queue. This can give us an intuition that queue does not belong to Common2.

The valency-based helping has the disadvantage that it cannot capture the helping between the operations that return trivial values: for example, in the case of stack, two pushes do "help" each other. Because of that we switched our attention to the linearization-based helping by Censor-Hillel et al. [START_REF] Censor-Hillel | Help![END_REF]. They proposed a natural formalization based on the notion of linearization: a process p helps an operation of a process q in a given execution if a step of p determines that an operation of q takes effect, or linearizes, before some other operation in any possible extension. At first glance, there is no connection between the question about Common2 queue and this definition: they claimed that stack and queue are exact order types and, then, they proved that any exact order type does not have a wait-free help-free implementation using reads, writes and compare&swap primitives. Informally, a sequential data type is exact order if for some sequence of operations every change in the relative order of two operations affects the result of some other operations. However, we found that that the stack data type is not exact order. Hence, the proof of help-free impossibility for exact order types given in [START_REF] Censor-Hillel | Help![END_REF] does not apply to stack, and there can exist a distinction between stack and queue! Unfortunately, we reassert the result by Censor-Hillel et al. via a direct proof that stack does not have a wait-free and help-free implementation using reads, writes, compare&swap and fetch&add primitives. At first, we show the result for implementations using read, write and compare&swap operations in systems with at least three processes, and, then, extend the proof to those additionally using fetch&add in systems with at least four proccesses. The structure of these two proofs resembles the structure of the proofs from the paper by Censor-Hillel et al. [START_REF] Censor-Hillel | Help![END_REF], but the underlying reasoning is novel. Unlike their approach our proofs argue about the order of operations given their responses only after we empty the data structure. As a result, certain steps of the proof become more technically involved.

Roadmap

The chapter is organized as follows. In Section 7.2, we present a computational model and necessary definitions. In Section 7.3, we recall the definition of helping and highlight the mistake in [START_REF] Censor-Hillel | Help![END_REF]. In Section 7.4, we give our direct proof. In Section 7.5, we explain how to build a help-free wait-free implementation of any data type given move&increment primitive. In Section 7.6, we discuss the related work. And, finally, we conclude in Section 7.7.

Model and Definitions

We briefly remind the model and definitions. All missed definitions can be found in Chapter 2.

We consider a system of n processes p 1 , . . . , p n communicating via invocations of primitives on a shared memory. During this chapter we assume that primitives are read, write, compare&swap, fetch&add and move&increment. The definition of the last primitive appear in Section 7.5.

In this chapter, we discuss stack implementations. To simplify the reasoning all considered implementations are deterministic. Nevertheless, the provided proofs can be easily extended to randomized implementations. For the rest of the section we fix some implementation of stack.

A program of a process specifies a sequence of operations calls on an object. The program may include local computations and can choose which operation to execute depending on the results of the previous operations.

A (low-level) history is a finite or infinite sequence of primitive steps. Note that in this chapter when we mention history we mean that it is low-level (if not specified otherwise explicitly). Each step is coupled with a specific operation that is being executed by the process performing this step. The first step of an operation always comes with the input parameters of the operation, and the last step of an operation is associated with the return of the operation. Given two histories h 1 and h 2 we denote by h 1 • h 2 the concatenation of h 1 and h 2 .

A schedule is a finite or infinite sequence of processes' identifiers. Given a schedule, an implementation and programs provided to the processes, one can unambiguously determine the corresponding history. And vice versa, given a history one can always build a schedule by substituting the steps of history to the process that performed it. Assuming a fixed program for each process (these programs will be clear from the context), and a history h, we denote by h • p i the history derived from scheduling process p i to take the next step (if any) following its program immediately after h.

The set of histories H induced by an implementation consists of all possible histories induced by all possible processes' programs with all possible schedules. An implementation of a data type is linearizable if each history from the set of histories has a linearization (Definition 2.3.2). A linearization function defined over a set of linearizable histories H maps every history in H to its linearization. Note that a linearizable implementations may have multiple linearization functions defined on the set of its histories.

Helping and Exact Order Types

In this section, we recall the definitions of helping and exact order type in [START_REF] Censor-Hillel | Help![END_REF], and show that stack is not exact order. An implementation is help-free, if there exists a linearization function f such that the set of histories of this implementation with f is help-free.

Following the formalism of [START_REF] Censor-Hillel | Help![END_REF], if S is a sequence of operations, we denote by S(n) the first n operations in S, and by S n the n-th operation of S. We denote by (S + op?) the set of sequences that contains S and all sequences that are similar to S, except that a single operation op is inserted somewhere between (or before, or after) the operations of S.

Definition 7.3.3 (Exact Order Types

). An exact order type is a data type for which there exists an operation op, an infinite sequence of operations W , and a (finite or infinite) sequence of operations R, such that for every integer n ≥ 0 there exists an integer m ≥ 1, such that for any sequence A from W (n + 1) • (R(m) + op?) and any sequence B from W (n) • op • (R(m) + W n+1 ?) at least one operation in R(m) has different responses in A and B, where • is a concatenation of sequences.

It is shown in [START_REF] Censor-Hillel | Help![END_REF] that the implementations of exact order types require helping if they use only read, write, and compare&swap primitives. The paper also sketches the proof of a more general result for implementations that, additionally, use fetch&add. Further, it is claimed in [START_REF] Censor-Hillel | Help![END_REF] that stack and queue are exact order types. Indeed, at first glance, if you swap two subsequent operations, further operations have to acknowledge this difference. However, the definition of an exact order type is slightly more complicated, as it allows not only to swap operations but also move them. This relaxation does not affect queue, but, unfortunately, it affects stack. Theorem 7.3.1. Stack is not an exact order type.

Proof. We prove that for any fixed op, W , R and n there does not exist m that satisfies Definition 7.3.3. Note that the claim is stronger than what is needed to prove the theorem: it would be sufficient to prove that for all op, W and R, the condition does not hold for some n. In a sense, this suggests that stack is far from being exact order. Suppose, by contradiction, that there exists m that satisfies Definition 7.3.3 for fixed op, W , R and n. There are four cases for op and W n+1 : pop-pop, push-pop, pop-push or push-push. For each of these cases, we find two sequences from W (n + 1) • (R(m) + op?) and W (n) • op • (R(m) + W n+1 ?) for which all operations in R(m) return the same responses. Since in h , only p 2 takes steps starting from h, op 1 must be decided before op in hotherwise, h has a prefix h such that op 1 is not decided before op in h and op 1 is decided before op in h • p 2 -a contradiction with the assumption that Q is help-free. Now we build an infinite history h in which p 1 executes infinitely many failed com-pare&swap steps, yet it never completes its operation. We assume that p 1 , p 2 and p 3 are assigned the following programs: p 1 tries to perform op 1 = push(1); p 2 applies an infinite sequence of operations push(2), push(3), push(4), . . .; and p 3 is about to perform an infinite sequence of pop() operations.

1 h ← 2 op 1 ← push(1) 3 id 2 ←
The algorithm for constructing this "contradiction" history is given in Figure 7.1. Initially, p 1 invokes op 1 = push(1) and, concurrently, p 2 invokes op 2 = push(2). Then we interleave steps of p 1 and p 2 until a critical history h is located: op 1 is decided before op 2 in h • p 1 and op 2 is decided before op 1 in h • p 2 . We let p 2 and p 1 take the next step and, then, run op 2 after h • p 2 • p 1 until it completes. We will show that op 1 cannot complete and that we can reiterate the construction by allowing p 2 to invoke concurrent operations push(3), push(4), etc. In the resulting infinite history, p 1 takes infinitely many steps without completing op 1 .

To ensure that at each iteration op 1 is not completed, we show that, at the start of each iteration of the outer loop (Line 6), the constructed history satisfies the following two invariants:

• op 1 is not decided before op 2 or before any operation of p 3 ;

• the operations of p 2 prior to op 2 are decided before op 1 .

At the first iteration, the invariants trivially hold, since neither op 1 nor op 2 is started. Observation 7.4.2. The order between op 1 and op 2 cannot be decided during (and right after) the inner loop (Lines 6-13).

Lemma 7.4.3. During (and right after) the execution of the inner loop (Lines 6-13) op 1 and op 2 cannot be decided before any operation of p 3 .

Proof. Suppose that during an execution of the inner loop op 1 or op 2 is decided before some operation of p 3 .

Before entering the inner loop, neither op 1 nor op 2 is decided before any operation of p 3 : op 1 is not decided because of the first invariant that holds at the beginning of this iteration, while op 2 is not started (Observation 7.4.1 (2)). Thus, at least one step is performed by p 1 or p 2 during the execution of the inner loop.

Let us execute the inner loop until the first point in time when op 1 or op 2 is decided before an operation of p 3 . Let this history be h. Note, that because Q is help-free only one of op 1 and op 2 is decided before an operation of p 3 in h. Suppose, that op 1 is decided before some op 3 of p 3 , while op 2 is not decided before any operation of p 3 . (The case when op 2 is decided before some op 3 is symmetric)

Now p 3 runs pop operations until it completes operation op 3 and then, further, until the first pop operation returns ⊥, i.e., the stack becomes empty. Let the resulting extension of h be h .

Recall that op 2 is not decided before any operation of p 3 in h and, since Q is help-free and only p 3 takes steps after h, op 2 cannot be decided before any operation of p 3 in h . Hence, none of the completed operations of p 3 can return id 2 , the argument of op 2 , due to the fact that all push operations have different arguments. Since the operations of p 3 empty the stack and they cannot pop id 2 , op 2 has to linearize after them, making op 3 be decided before op 2 in h . By Transitivity Lemma 7.4.1, op 1 is decided before op 2 in h . Finally, since Q is help-free and only p 3 takes steps after h op 1 has to be decided before op 2 in h, contradicting Observation 7.4.2. Lemma 7.4.4. op 1 and op 2 cannot be completed after the inner loop (Lines 6-13).

Proof. Suppose the contrary. By Observation 7.4.1 (1), op 1 has to be decided before all operations of p 3 , contradicting Lemma 7.4.3. Lemma 7.4.5. The execution of the inner loop (Lines 6-13) is finite.

Proof. Suppose that the execution is infinite. By Lemma 7.4.4, neither of op 1 and op 2 is completed in h. Thus, in our infinite execution either op 1 or op 2 takes infinite number of steps, contradicting wait-freedom of Q. Lemma 7.4.6. Just before Line 14 the following holds:

1. The next primitive step by p 1 and p 2 is to the same memory location.

2. The next primitive step by p 1 and p 2 is a compare&swap. This way we build a history in which p 1 takes infinitely many steps, but op 1 is never completed. This contradicts the assumption that Q is wait-free. Theorem 7.4.1. In a system with at least three processes and primitives read, write and compare&swap there does not exist a wait-free and help-free stack implementation.

The expected value of the compare&swap steps of p

Adding Fetch&Add

Now suppose that the implementation is allowed to additionally use fetch&add primitives. We prove that there is no wait-free and help-free stack implementation in a system with at least four processes.

Again, by contradiction, suppose that such an implementation Q exists. We build an infinite history h in which either p 1 or p 2 executes infinitely many failed compare&swap steps, yet it never completes its operation, contradicting wait-freedom. In h, processes p 1 , p 2 , p 3 and p 4 follow the following programs: for 1 ≤ i ≤ 2, p i tries to perform op i = push(i); p 3 applies an infinite sequence of operations push(3), push(4), push(5), . . .; and p 4 is about to perform an infinite sequence of pop() operations. The algorithm for constructing this "contradiction" history is given in Figure 7.2.

Similar to the proof of Theorem 7.4.1, we show that the following three invariants hold at the beginning of each iteration of the outer loop (Line 6):

• the order between any two operations among op 1 , op 2 and op 3 is not decided;

• op 1 and op 2 are not decided before any operation of p 4 ;

• all the operations of p 3 prior to op 3 are decided before op 1 and op 2 .

At the beginning of the first iteration, the invariants hold trivially, since none of op i is started.

Observation 7.4.5. The order between op i and op j for 1 ≤ i = j ≤ 3 cannot be decided during (and right after) the inner loop (Lines 7 -14).

Suppose that the next primitive step of some p i is a write. We take any other process p j and build two histories:

h = h • p j • p i • complete op i and h = h • p i • complete op i .
As in the proof of Lemma 7.4.6, p i performs two pop() operations (op i and op i ) and p j completes its operation after h : by Lemmas 7.4.1 and 7.4.12, op i and op i have to return the argument of op i and the argument of op j , respectively. The local states of p i and the shared memory states after h and h are identical, thus, op i and op i after h should also return the arguments of op i and op j . Hence, op j has to be decided before op i in h = h • perform op i • perform op i . By Lemma 7.4.2, op j is decided before any operation of p 4 in h. And, finally, since Q is help-free and p j does not take steps in h after h, op j has to be decided before any operation of p 4 in h, contradicting Lemma 7.4.9.

A similar argument applies to the case when the next primitive step of some p i is fetch&add with argument zero, or compare&swap which expected value differs from the value in the designated location or the new value is equal to the expected. We take any other process p j (1 ≤ j ≤ 3) and build two histories h = h • p i • p j • complete p j and h = h • p j • complete p j . The proof for the previous case applies except that now the roles of p i and p j are swapped. Proof. Suppose that p 1 and p 2 have fetch&add as their next primitive step. Consider two histories h = h • p 1 • p 2 and h = h • p 2 • p 1 . From Lemma 7.4.12 op 1 is decided before op 2 in h , thus, by Lemma 7.4.2, op 1 is decided before the first operation op 4 of p 4 . After h p 4 performs k pop operations until one of them returns ⊥, i.e., the stack is empty. One pop has to return 1, because if we now complete op 1 it has to be linearized before op 4 . The same with h : p 4 performs k pops until one of them returns ⊥, and one of these pop's return 2. Since the local states of p 4 and the shared memory states after h and h are the same: two pop operations pop 1 () and pop 2 () of k (= k ) operations of p 4 after h and h return 1 and 2. Now we show that op 1 and op 2 are decided before op 3 in h . The same can be shown for h . Consider a history h: h continued with k pop operations by p 4 . By Lemma 7.4.12, op 1 is decided before op 3 in h . From Lemma 7.4.9 and two facts that Q is help-free and op 3 does not make any steps after h in h, it follows that op 3 cannot be decided before any operation of p 4 in h and, consequently, the operations of p 4 cannot pop an argument of op 3 . Since k pops of op 4 empty the stack, op 3 has to linearize after them, making operation pop 2 () to be decided before op 3 . Since pop 2 () returns 2, it has to be decided after op 2 . By Transitivity Lemma 7.4.1, op 2 is decided before op 3 in h. Q is help-free and only p 4 takes steps after h , thus, op 2 is decided before op 3 in h . Now consider two histories h • complete op 3 and h • complete op 3 . In both of these histories, op 1 and op 2 are decided before op 3 . After the first history let p 4 perform three pop operations and p 1 and p 2 complete push(1) and push(2): the three pops return id 3 , 2 and 1, respectively. Analogously, after the second history three pop return id 3 , 1 and 2. This is impossible, since the local states of p 4 and the memory states after these two histories are identical. Proof. We prove the claim for op 1 , the case of op 2 is similar.

If p 2 took a step at Line 18, then, by Lemma 7.4.9 and the fact that the steps by p 2 or p 3 cannot fix the order between op 1 and any operation of p 4 due to help-freedom, op 1 is not decided before any operation of p 4 .

If p 1 took a step at Line 18, then, by Lemma 7.4.9 and the fact that the steps by p 3 cannot fix the order between op 1 and any operation of p 4 due to help-freedom, the only step that could fix the order is a step by p 1 at Line 18, i.e., a failed compare&swap. Suppose that op 1 is decided before some op 4 of p 4 after Line 18. We consider two histories h = h • p 3 • p 1 and h = h • p 3 . Let p 4 run solo after h until it completes op 4 , and then further until some pop returns ⊥, i.e., the stack becomes empty. Since op 1 is decided before op 4 , some completed operation op 4 of p 4 has to return 1: if we now complete op 1 it has to be linearized before op 4 . Now, let p 4 to run the same number of pop operations after h . Since the local states of p 4 and the shared memory states after h and h are identical, op 4 returns 1. Thus, op 1 is decided before op 4 in h . As Q is help-free and p 1 does not take steps after h in h , op 1 has to be decided before op 4 in h, contradicting Lemma 7.4.9.

Lemma 7.4.16. At the end of the outer loop (Line 21), the order between any two operations among op 1 , op 2 and the next op 3 = push(id 3 + 1) is not yet decided.

Proof. The operation op 3 is not yet started, thus, it cannot be decided before op i , i = 1, 2, by Observation 7.4.1 [START_REF] Umut | Algorithm Design: Theory and Practice[END_REF]. Suppose that op i , i = 1, 2, is decided before op j , then by Lemma 7.4.2 op i has to be decided before all operations of p 4 , contradicting Lemma 7.4.15.

We started with three invariants that hold before any iteration of the loop. By Observation 7.4.7 and Lemmas 7.4.15 and 7.4.16, the invariants hold after the iteration, and at least one of p 1 and p 2 made at least one primitive step.

This way we build a history in which one of op 1 and op 2 never completes its operation, even though it takes infinitely many steps. This contradicts the assumption that Q is wait-free. Theorem 7.4.2. In a system with at least four processes and primitives read, write, compare&swap and fetch&add, there does not exist a wait-free and help-free stack implementation.

Universal Construction with Move&Increment

At first, we define atomic move&increment primitive. It takes two pointers to integers to_move and to_increment. The operation copies a value from to_increment to to_move and increments the value in to_increment.

This primitive is obviously more powerful than fetch&add, but it seems to be less complex than atomic double compare&swap: a primitive that takes a pair of arbitrary memory locations, a pair of expected values and a pair of new values, and sets new values in the provided memory locations only if the current values in the memory locations are equal to the provided expected values. Theorem 7.5.1. In a system with n processes and primitives read, write and move&increment any data type has a help-free wait-free implementation provided a sequential implementation.

Proof. As for the proof we simply provide such an implementation in Figure 7.3.

In the algorithm, we use an infinite array of operations operations, an identifier array id_by_p of size n and an operation array op_by_p of size n, where n is the number of processes. The algorithm builds a total order of operations in operations array. Process pid tries to apply an operation op by calling apply(pid, op). At first, the process registers his operation in op_by_p[id] array (Line 7). Second, it gets an identifier id_by_p of its operation in the sequence of all operations, i.e., the array operations (Line 8). Then the process helps to copy all the operations into the sequence operations that should come prior to op together with op. For that it iterates through all processes and for each process j (p j ) it reads the current identifier (Line 11) and operation (Line 12). Then it verifies whether the operation by p j was not yet applied (Lines 14 and 15): the operation is op or should come prior to op and the corresponding slot in operations is still empty. Finally, it applies the all operations in the order of array operations to new sequential object.

Each c = apply(pid, op) call has a unique identifier id(c) calculated in Line 7. To prove the correctness of the algorithm it is enough to show that: 1) op from a call c = apply(pid, op) can only appear at operations[id(c)], and 2) before Line 18 of call c all cells of operations up to id(c) are non-null.

We prove this by the identifier of the call k.

Base k = 0. No calls are performed and everything is correct. Consider a call c = apply(pid, op) with identifier k. We know that for all calls prior to k the two statements are correct.

Instead of proving the two statements we prove for each p j separately: all operations of p j from the calls with identifiers up to k are copied to proper positions in operations array before Line 18 of call c.

For p j , p pid reads a current identifier op_id_j in Line 11 and a current operation op_j in Line 12. Suppose that op_j was written to op_by_p[j] during a call apply(j, op') with identifier uid op and op_id_j was written to id_by_p[j] during a call apply(j, op") with identifier uid id .

We note that uid id is always less than or equal to uid op , since we read identifier and operation in the opposite order we write them.

Let lid be the identifier of the last call by p j with the identifier less than or equal to k. Note that uid id and uid op cannot be less than lid. This is due to the fact that uid op was already set to lid before move&increment primitive by pid during c which happened prior to the read of op_id_j.

There are two cases. If uid op is bigger than lid then the call with identifier lid is already completed when op_j is read. This means by induction that all the operations by p j prior to k are already copied into their proper positions.

Otherwise, uid op is equal to lid. Thus, since uid id ≤ uid op , we get that uid id = uid op = lid. Thus, the write at Line 16 by p pid can copy operation only into proper position and it is always copied if the proper position is empty. Note that this is the only situation when the operation is copied into operations array and it is copied only into the proper position.

Since the operations by p j from calls before uid id are successfully copied by induction, and the operation from the last call by p j with identifier less than ort equal to k is copied, after Line 16 all operations by p j with identifier less than or equal to k are copied.

Note that the presented implementation is obviously wait-free. Also, it is help-free since we can linearize any operation in Line 8. Thus, the operation is linearized by its operation and does not require the linearization-based helping.

Related Work

Helping is often observed in wait-free (e.g., [START_REF] Fatourou | A highly-efficient wait-free universal construction[END_REF][START_REF] Feldman | A wait-free multi-word compare-and-swap operation[END_REF][START_REF] Peng | Fa-stack: A fast array-based stack with wait-free progress guarantee[END_REF][START_REF] Timnat | Wait-free linked-lists[END_REF]) and lock-free implementations (e.g., [START_REF] Arbel | Reuse, don't recycle: Transforming lockfree algorithms that throw away descriptors[END_REF][START_REF] Shane | A non-blocking internal binary search tree[END_REF][START_REF] Maged | High performance dynamic lock-free hash tables and list-based sets[END_REF][START_REF] Natarajan | Fast concurrent lock-free binary search trees[END_REF]): operations of a slow or crashed process may be finished by other processes. Typically, to benefit from helping, an operation should register a descriptor (either in a dedicated "announce" array or attached in the data items) that can be used by concurrent processes to help completing it.

We are aware of three alternative definitions of helping: (1) linearization-based by Censor-Hillel et al. [START_REF] Censor-Hillel | Help![END_REF] considered here, (2) valency-based by Attiya et al. [START_REF] Attiya | Nontrivial and universal helping for wait-free queues and stacks[END_REF] and (3) universal by Attiya et al. [START_REF] Attiya | Nontrivial and universal helping for wait-free queues and stacks[END_REF].

Valency-based helping [START_REF] Attiya | Nontrivial and universal helping for wait-free queues and stacks[END_REF] captures helping through the values returned by the operations, which makes it quite restrictive. In particular, for stack, the definition cannot capture helping relation between two push operations. They distinguish trivial and nontrivial helping: for non-trivial helping, the operation that is being helped should return a data-structure-specific non-trivial (e.g., non-empty for stacks and queues) value. It is shown in [START_REF] Attiya | Nontrivial and universal helping for wait-free queues and stacks[END_REF] that any wait-free implementation of queue has non-trivial helping, while there exists a wait-free implementation of stack without non-trivial helping. This is an interesting result, given notorious attempts of showing that queue is in Common2 [START_REF] Afek | A completeness theorem for a class of synchronization objects[END_REF], i.e., that they can be implemented using reads, writes and primitives with consensus number 2, while stack has been shown to be in Common2 [START_REF] Afek | Common2 extended to stacks and unbounded concurrency[END_REF].

Attiya et al. [START_REF] Attiya | Nontrivial and universal helping for wait-free queues and stacks[END_REF] also introduced a very strong notion of helping -universal helpingwhich essentially boils down to requiring that every invoked operation eventually takes effect. This property is typically satisfied in universal constructions parameterized with object types. But most algorithms that involve helping in a more conventional (weaker) sense do not meet it, which makes the use of universal helping very limited.

Linearization-based helping [START_REF] Censor-Hillel | Help![END_REF] considered here is based on the order between two operations in a possible linearization. Compared to valency-based definitions, this notion of helping operates on the linearization order and, thus, can be applied to all operations, not only to those that return (non-trivial) values. By relating "helping" to fixing positions in the linearization, this definiton appears to be more intuitive: one process helps another make a "progress", i.e., linearize earlier. Censor-Hillel et al. [START_REF] Censor-Hillel | Help![END_REF] also introduced two classes of data types: exact order types (queue as an example) and global view types (snapshot and counter as examples). They showed that no wait-free implementation of data types from these two classes can be help-free. By assuming stack to be exact order, they deduced that this kind of helping is required for wait-free stack implementations. In this chapter, we showed that stack is in fact not an exact order type, and give a direct proof of their claim.

Conclusion

In this chapter, we gave a direct proof that any wait-free implementation of stack in a system with read, write, comare&swap and fetch&add primitives is subject to linearizationbased helping. This corrects a mistake in the indirect proof via exact order types in [START_REF] Censor-Hillel | Help![END_REF].

Let us come back to the original intuition of helping as a process performing work on behalf of other processes. One may say that linearization-based helping introduced by Censor-Hillel et al. and used here does not adequately capture this intuition. For example, by examining the wait-free stack implementation by Afek et al. [START_REF] Afek | Common2 extended to stacks and unbounded concurrency[END_REF], we find out that none of the processes explicitly performs work for the others: to perform pop() a process goes down the stack from the current top until it reaches some value or the bottom of the stack; while to perform push(x) a process simply increments the top of the stack and deposits x there. But we just showed that any wait-free stack implementation has linearization-based helping, and indeed this algorithm has it. So, we might think that valency-based helping is superior to linearization-based one, since the algorithm by Afek et al. does not have non-trivial valency-based helping. Nevertheless, the aforementioned algorithm has trivial valency-based helping, and, thus, the (quite unnatural) distinction between trivial and non-trivial helping seems to be chosen specifically to allow the algorithm by Afek et al. to be help-free.

A very interesting challenge is therefore to find a definition of linearization-based helping that would naturally reflect help-freedom of the algorithm by Afek et al., while queue does not have a wait-free and help-free implementation.

Performance Prediction for

Coarse-Grained Programs

Introduction

A standard design pattern found in many concurrent data structures, such as hash tables or ordered containers, is an alternation of parallelizable sections that incur no data conflicts and critical sections that must run sequentially and are protected with locks. A lock can be viewed as a queue that arbitrates the order in which the critical sections are executed, and a natural question is whether we can use theoretical analysis to predict the resulting throughput. As a preliminary evidence to the affirmative, we describe a simple model that can be used to predict the throughput of coarse-grained lock-based algorithms. We show that our model works well for CLH lock, and we expect it to work for other popular lock designs such as TTAS, MCS, etc.

Roadmap

In Section 8.2, we describe the problem in more details. In Section 8.3, we list assumptions on the abstract machine on which data structures are executed. In Section 8.4, we obtain a formula for the throughput. In Section 8.5, we verify our theoretical result through experimental analysis. We conclude in Section 8.6.

Abstract Coarse-Grained Synchronization

Conventionally, the performance of a concurrent data structure is evaluated via experiments, and it is notoriously difficult to account for all significant experimental parameters so that the outcomes are meaningful. Our motivation here is to complement experimental evaluation with an analytical model that can be used to predict the performance rather than measure it. As a first step towards this goal, we attempt to predict the throughput of a class of algorithms that use coarse-grained synchronization. Consider a concurrent system with N processes that obey the following simple uniform scheduler: at every time step, each process performs a step of computation. This scheduler, resembling the well-known PRAM model [START_REF] Jájá | An introduction to parallel algorithms[END_REF], appears to be a reasonable approximation of a real-life concurrent system. Suppose that the processes share a data structure exporting a single operation(). If the operation induces a work of size P and incurs no synchronization, the resulting throughput is N • α/P operations in a unit of time: each process performs α/P operations in a unit of time, where α indicates the amount of work that is performed by one process in a unit of time. One way to evaluate the constant α experimentally is to count the total number F of operations, each of work P , completed by N processes in time T . Then we get α = F/N P . The longer is T , the more accurate is the estimation of α. Now suppose that, additionally, the operation performed by each process contains a critical section of size C. In the operation, described in Figure 8.1, every process takes a global lock, performs the critical section of size C, releases the lock and, finally, performs the parallel section of size P .

Here, as a unit of work, we take the number of CPU cycles spent during one iteration of the loop in Lines 3-4 or 6-7. The iteration consists of a nop instruction, an increment One cache miss can happen at the first iteration of the loop if the read of locked returns true. The last process that grabbed the lock already invalidated this cache line in Line 15 during its penultimate operation. MESI reloads the cache line and changes its state from I (or none if it was not loaded previously) to S.

The other cache miss happens in every execution when the operation reads next.locked and gets false. In this case, the cache line was invalidated in Line 13 during the last operation of the last process that grabbed the lock. MESI reloads the cache line and changes its state from I (or none) to S.

Each of the described cache misses induces the work of size R I . Thus, the work induced in Line 10 is of size R I (if only the second miss happens) or 2 • R I (if both misses happen).

3. In Lines 11-12, the critical section with work of size C is performed.

4. In Line 13, my_node.locked is set to false. There are two cases: if my_node.locked

is not yet loaded by any other process in Line 10 then the state remains M; otherwise, MESI changes the state from S to M and sends a signal to invalidate this cache line.

In both cases, the induced work is of size W .

5. In Line 14, the operation performs an assignment on local variables, without contributing to the total work.

6. In Line 15, my_node.locked is set to true. From the end of the while loop at Line 10 the corresponding cache line is in state S. MESI changes the state to M and sends a signal to invalidate this cache line inducing work of size W .

7. In Lines 16-17, the parallel work of size P is performed.

Evaluating Throughput

To evaluate the throughput of the resulting program under the uniform scheduler, take a closer look on how N processes continuously perform the operation from Figure 8.2. Process 1 executes: its first swap (taking at most X units); the critical section (blue, Lines 10-13): acknowledges the ownership of the lock by reading false in Line 10 (takes R I units), performs the work of size C and releases the lock in Line 13 (takes W units); the parallel section (red, Lines 15-17 and 9): sets my_node.locked to true (takes W ), performs the work of size P , performs a non-contended swap (takes W ) and, possibly, reads true in Line 10 (takes R I ). (Here, the swap operation performed after the very first completed critical section is counted in the parallel work, as it is executed in the absence of contention.) Every other process i operates in the same way: it swaps as early as possible (taking at most X), waits until process i -1 releases the lock, and then performs its critical (blue) and parallel (red) sections. Depending on the parameters N , C, P , W , and R I , two types of executions are possible.

In Case 1 (Figure 8.3a), at the moment when process 1 finishes its parallel section, process N already finished its critical section, i.e., P + 2 • W > (N -1) • (C + R I + W ). Therefore, in the steady case, at every moment of time, each process does not wait and executes either the parallel or critical section, and the read in Line 10 cannot return true because the lock is already released. Thus, the throughput, measured as the number of operations completed in a unit of time, equals to N • α (P +2•W )+(C+R I +W ) . In Case 2 (Figure 8.3b), before proceeding to the next operation, process 1 has to wait until process N completes its critical section from the previous round of operations; process 2 waits for process 1, process 3 waits for process 2, etc. Thus, there is always some process in the critical section, giving the throughput of α C+R I +W . Therefore, given the number of processes N , the sizes C and P of critical and parallel sections, the throughput can be calculated as follows:

   α C+R I +W if P + 2 • W ≤ (N -1) • (C + R I + W ) α•N (P +2•W )+(C+R I +W )) otherwise

Experiments

For our measurements, we used a server with four 10-core Intel Xeon E7-4870 chips of 2.4 GHz (yielding 40 hardware processes in total), running Ubuntu Linux kernel v3.13.0-66generic. We compiled the code with MinGW GCC 5.2.0 (with -O0 flag to avoid compiler optimizations, such as function inlining, that can screw up our benchmarking environment). The code is available at https://github.com/Aksenov239/complexity-lock-with-libslock.

We considered the following experimental settings: the number of processes is 39; the size of the critical section C ∈ {100, 500, 5000}; and the multiplier x ∈ [1, 150] (we choose

Conclusion and Future Work

In this thesis, we study how to balance synchronization and parallelism in a range of concurrent programs. We presented an automatic granularity control for parallel programs and two novel ways (concurrency-optimality and parallel combining) to design concurrent data structures with potentially low synchronization overhead. Then we discussed synchronization in a form of helping in wait-free and lock-free data structures. Finally, we proposed a way to predict the throughput of simple coarse-grained concurrent data structure.

Each of these contributions provides us with open questions for future research. In this chapter, we briefly recall these questions.

Automatic Granularity Control

In the current proposal, the programmer has to provide an asymptotic cost function for each sp-guard separately. These functions can be non-trivial and the programmer might not be able to provide them. However, he often can provide the sizes of the data on which the code is going to work. An interesting question is whether we can design a similar granularity control technique that uses only the information about sizes?

The intuition tells that we can find the appropriate cost functions using some machine learning algorithm and, after that, we can use our original algorithm. At first, new algorithm produces data points for each sp-guard by measuring the execution time: these points map a size of the data to the execution time. When the set of sample points for the sp-guard becomes representative enough, the algorithm fits the cost function using some machine learning algorithm, e.g., symbolic regression by genertic programming [START_REF] John R Koza | Genetic programming as a means for programming computers by natural selection[END_REF].

Another open question is what to do when complexity functions are not helpful, e.g., the execution time of the code with the same cost function fluctuates a lot. In this case, our predictions can be useless and the performance can degrade.

As an example, consider the simplest algorithm that compares two strings, symbol by symbol. For this algorithm, we expect the cost function to depend on the lengths of the strings, but not on the data itself (otherwise, the cost function is not lightweight). Thus, for the following two inputs the cost is the same while the execution time differs much:

(1) if one string consists of n symbols 'a' and the other consists of n symbols 'b' then the execution time is constant; (2) if both strings consist of n symbols 'a' then the execution time depends on n.

As a second example, we consider a maximal matching algorithm from the PBBS suite [START_REF] Shun | Brief announcement: the problem based benchmark suite[END_REF]: given a graph it finds a set of edges such that no pair of edges share a vertex and any other edge shares a vertex with some edge from the set. The algorithm uses the compare&swap primitive [START_REF] Guy E Blelloch | Internally deterministic parallel algorithms can be fast[END_REF]. Because the pattern of the compare&swap invocations is not fixed, the contention during the execution of the same code and with same complexity can differ much, which results in the fluctuation of the execution time. We did not present the results of the execution of this algorithm in Chapter 4 since our approach did not perform well on some inputs. Sometimes our version worked up to 30% worse than the PBBS one. Is it possible to improve our granularity control algorithm in a way that it could handle such behaviour?

Concurrency-Optimal Binary Search Tree

In Chapter 5, we provided a concurrency-optimal implementation of a binary search tree. This is the second known concurrency-optimal implementation after linked-list based set [START_REF] Gramoli | Brief announcement: a concurrency-optimal list-based set[END_REF]. Do there exist other data structures that can benefit from concurrency-optimality? We expect that skip-list should be a good candidate because it can be seen as a composition of linked lists.

Right now the concurrency-optimality proposes to measure "concurrency" of algorithms in sets of schedules. The immediate question is whether it is possible to express this metric quantitatively, so that it would be easier to compare "concurrency" of implementations? For example, as a new integer metric we can propose the number of dihomotopy classes of executions [START_REF] Fajstrup | Directed algebraic topology and concurrency[END_REF]. Intuitively, two executions are equivalent (or dihomotopic) if one can be obtained from the other by permuting independent instructions. Is it the reasonable metric?

Finally, we can imagine a design principle similar to concurrency-optimality which intends to maximize the set of exported executions, i.e., interleavings of steps of a concurrent algorithm, instead of schedules. Below we explain this idea on the example.

Consider two concurrent implementations of partially-external binary search trees. In the first one, there is one lock per node and an insertion of a new node is performed as follows: take a lock on the parent and set the proper child to the newly created node. In the second one, there is a read-write lock per node and a lock per each link to the children, and an insertion of a new node is performed as follows: take a read lock on the parent, take a lock on a link to the proper child and set a proper child to the newly created node.

It is obvious that the second implementation is more fine-grained and it allows more executions than the first one: two new nodes can be added as a left and a right child almost concurrently. This separation of one lock per node into two locks on the links was proven to provide better performance in practice [START_REF] Natarajan | Fast concurrent lock-free binary search trees[END_REF].

Can we provide a formal guide for this approach: specify the ways to increase the number of executions and specify the granularity, i.e., the number of executions, after which the performance starts to degrade?

Parallel Combining

In Chapter 6 we provide three application of the parallel combining technique. However, for two of them, binary search trees and priority queues, there already existed efficient fine-grained concurrent implementations. An immediate question is: which new concurrent data structures can we create from their parallel batched counterparts? For sure, we can build a concurrent dynamic tree since its parallel batched version is known [START_REF] Umut A Acar | Brief announcement: Parallel dynamic tree contraction via self-adjusting computation[END_REF]: it exports the same operations as a dynamic graph but must remain a tree (forest) at any moment of time. Also, we expect that it should be possible to design a parallel batched dynamic graph from a parallel batched dynamic tree and, consequently, it should be possible to design a concurrent dynamic graph that is more efficient than its coarse-grained implementations.

Right now, to use parallel combining, we have to provide a code of a parallel batched data structure in a form of COMBINER_CODE and CLIENT_CODE. It is less convenient than using the original code, typically written for dynamic multithreading, without changes. It is known that, the algorithms for dynamic multithreading are typically executed using some scheduler, for example, work-stealing [START_REF] Michael D Mccool | Structured parallel programming: patterns for efficient computation[END_REF]. Thus, the follow-up question is: can we implement efficient scheduler that uses only the processes from a specified subset, i.e., clients? This question is not entirely trivial, since up-to-date schedulers use all processes, and not the specific subset.

In Chapter 6 we consider only the linearizability correctness criterion. We present a new weaker notion of correctness: for each operation op there exist a linearizable set of operations S that contains: (1) op, (2) all operations finished before the invocation of op, and, (3) possibly, some other operations.

As an example, we build a simple concurrent implementation of a persistent binary search tree that satisfies our criterion while the linearized implementation is unknown and we expect it to be non-trivial. A persistent data structure [START_REF] James R Driscoll | Making data structures persistent[END_REF] is a data structure that creates a new version on an update and provides an access to all the previous versions.

We get our concurrent persistent binary search tree using parallel combining. Suppose that before the current combining phase the tree is T . During the combining phase the algorithm works in two phases: (1) we apply each operation to T separately in parallel using standard sequential persistent algorithm, i.e., as a resulting version an operation gets a tree T with this operation applied; [START_REF] Umut | Algorithm Design: Theory and Practice[END_REF] we apply all operations together to T using the parallel batched persistent algorithm similar to the algorithm by Blelloch et al. [START_REF] Guy E Blelloch | Just join for parallel ordered sets[END_REF]. This implementation satisfies our criterion: for each operation op, the set S contains op and all operations that were applied in preceding combining phases -the version of the tree returned by op is a tree that satisfies some sequential application of operations from S.

It would be interesting to investigate this correctness criterion further and find other data structures that can benefit from it, i.e., the implementation with new criteria is more efficient than the linearizable implementation.

On Helping and Stacks

In Chapter 7, we only considered helping in wait-free implementations. Thus, an immediate question is whether it is possible to consider helping for lock-free algorithms. Can we identify data types that do not have a lock-free help-free implementation?

As shown in this thesis, linearization-based helping does not distinguish stack and queue: none of these data structures has a wait-free help-free implementation. But examining the wait-free stack by Afek et al. [START_REF] Afek | Common2 extended to stacks and unbounded concurrency[END_REF], we find out that none of the processes explicitly performs work for the others: to perform pop() a process goes down the stack from the current top until it reaches some value or the bottom of the stack; while to perform push(x) a process simply increments the top of the stack and deposits x there. Thus, this algorithm does not have helping in the simplest intuitive form. However, there is no paradox, since linearization-based helping is still present in this algorithm.

Therefore, we might think that valency-based helping [START_REF] Attiya | Nontrivial and universal helping for wait-free queues and stacks[END_REF] is more suitable. The algorithm by Afek et al. does not have non-trivial (valency-based) helping. Nevertheless, valency-based helping has two issues: (1) the aforementioned algorithm by Afek et al. does have trivial helping, i.e., one pop can make another pop to return ⊥, and the quite unnatural distinction between trivial and non-trivial helping seems to be chosen specifically to allow this algorithm to be help-free; (2) the definition does not capture helping in operations that do not return non-trivial responses, for example, no operation helps push operation on stack by default.

Thus, it is appealing to find a variant of linearization-based helping that would capture help-freedom of the algorithm by Afek et al.

Performance Prediction of Coarse-Grained Programs

To perform our simplistic theoretical analysis in Chapter 8 we posed several constraints and it would interesting to get rid of them.

At first, can we replace CLH Lock by other popular lock designs such as TTAS, MCS, etc.?

Second, we assumed that the architecture has symmetric cache and read and write costs are independent on the status of a cache line. Can we introduce more details and make the architecture more realistic? Finally, all considered programs have only one lock, one critical and one parallel sections of fixed size. It would be interesting to investigate throughput of more complex programs, for example, programs with several locks or with the parallel section of variable size.

  Figure 1.1: Execution of a parallel program

Figure 1 . 2 :

 12 Figure 1.2: Execution of a concurrent data structure

6 Figure 1 . 3 :

 613 Figure 1.3: A non-linearizable schedule

10 left

 10 ← phase1(a, tmp, l, m) , ans, m, r, sum + tmp[m])

template

  <F, T, S> map(F f, T* a, S* b, int n): map(f, a, b, 0, n) return template <F, T, S> map(F f, T* a, S* b, int l, int r): if r -l = 1: b[l] ← f(a[l]) return mid ← (l + r) / 2 spawn [&] { map(f, a, b, l, mid) } [&] { map(f, a, b, mid, r) } sync return

Figure 4 .

 4 Figure 4.1: Parallel map, naive implementation.

Figure 4 . 2 :

 42 Figure 4.2: Parallel map, coarsened implementation.

1 21 res2

 21 int grain = ... // determined by tuning 2 template <T, P> 3 match(T* lo, int n, P p): ← match(mid, sr, p)

Figure 4 . 3 :

 43 Figure 4.3: Find matches with manual granularity control.

1 2 template 4 res ← 0 5 spguard 21 res2 22 } 23 ) 24 res ← res1 + res2 25 }Figure 4 . 4 :

 245212223242544 Figure 4.4: Find matches with automatic granularity control.

12 if T ≤ κ and N > Nmax: 13 C 19 return (N ≤ Nmax) or 20 ( 37 returnFigure 4 . 5 :

 121319203745 Figure 4.5: Pseudocode for the core algorithm.

1 core_local time total 2 core_local time timer 3 4 18 spawn 26 Par_bodyFigure 4 . 6 :

 3182646 Figure 4.6: Implementation of time measurements.

1 5 return x + y 6 }, 7 [ 9 } 10 )

 567910 template <T, P> 2 match(T* lo, int l, P p): int 3 return map_reduce(lo, lo + l, 0, 4 [&] (int x, int y) { // associative combining operator &] (T* i) { // leaf-level operation 8 return p(*i)

Theorem 4 . 4 . 1 (

 441 Bound on the total span). S ≤ (1 + φ + max(τ, Eβκ)) • s.

Theorem 4 . 4 . 2 (

 442 Bound on the total work). Let κ = κ DEγ , and F = 1 + log α κ DE , and H = log γ/(γ-1) κ DE , and P denote the number of processes, and G denote the number of spguards occuring in the code. Then, we have

Lemma 4 . 4 . 1 .

 441 If term S does not contain any spguard, then w(S, I) = W s (S, I).

  I), where S denotes the sequential body of the spguard. Let N denote the value of F (I). According to the implementation of function is_small, a spguard selects the sequential body according to the boolean condition: (N ≤ Nmax) or ((N ≤ α • Nmax) and (N • C ≤ α • κ)), where Nmax and C are the values stored in the estimator for this spguard.

Theorem 4 . 4 . 4 (

 444 Bound on the running time of γ-regular program). Consider the γregular program with well-defined spguards. Let T P be the running time of the program on a machine with P processes and a greedy scheduler. Let w and s be the work and span of the program, correspondingly. G, F and H are as defined in4.4.13, 4.4.14 and 4.4.15.

  5, b = c = 0.1 and d = 0.3 (a = 0.57, b = c = 0.19, d = 0.05).

Figure 4 . 7 :

 47 Figure 4.7: Parallel BFS experiment on 40 cores (30 runs).

Figure 4 . 9 :

 49 Figure 4.9: Results from the BFS experiment, executed on the 75-core Intel machine, averaged over 30 runs.

Figure 5 . 3 :

 53 Figure 5.3: Node clasess

Figure 5 . 5 :

 55 Figure 5.5: Scenario depicting an execution of two concurrent delete(3) operations, followed by a successful insert(3); rejected by all the popular BSTs [39, 52, 57, 62, 120], it is accepted by a concurrency-optimal BST

Lemma 5 . 4 . 1 .

 541 Consider the call π = traverse(v). If BST at the moment of the invocation of π contains the node u with value v and there is no linearization point of successful delete(v) operation in the interval of π, then π returns u.

Figure 5 . 6 :

 56 Figure 5.6: Performance evaluation of concurrent BSTs

5 if 13 if 22 ifFigure 6 . 2 :

 5132262 Figure 6.1: Parallel combining: pseudocode

2 ] 2 ]

 22 [1..|S|] and corresponding clients-owners C[1..|S|]. When the algorithm by Blelloch et al. calls a branch of fork-join with apply(S[l..r], T ), our algorithm wakes up client C[ l+r 2 ] to apply the same requests. Thus, typically, an application of S[l..r] to tree T works in four stages: (1) client C[mid], where mid = l+r 2 , splits T by the argument of S[mid] into T L and T R ; (2) C[mid] wakes up client C[ l+(mid-1) to apply S[l..mid -1] to T L and client C[ (mid+1)+r to apply S[mid + 1..r] to T R ; (3) C[mid] waits until two awoken clients finish their work and return new trees; and, finally, (4) C[mid] joins two new trees, possibly, with the argument of S[mid] if it is insert. To start the algorithm we wake up client S[ 1+|S| 2 ] with S[1.

2 V v 3 left 4 right 5 6 // some balancing information 7 8

 23467 TreeTree // joins two trees and value v 9 join(Tree l, V v, Tree r): Tree 10 ...

Figure 6 . 3 :

 63 Figure 6.3: Binary Search Tree. Classes

29 if 39 r

 2939 ← contains(root, req.v) res, r) ← split(req.tree, res.v) 21 22 if req.leftR = null: 23 req.leftR.tree ← l 24 req.leftR.status ← UPDATE 25 if req.rightR = null: 26 req.rightR.tree ← r 27 req.rightR.status ← UPDATE 28 req.leftR = null: ← req.rightR.tree 40

Theorem 6 . 3 . 2 .

 632 Suppose that the combiner collects c requests using getRequests(). To apply the collected requests to the tree, the combiner spends O(c + log m) RMRs in CC model, each client spends O(log m) RMRs in CC model and, in total, the algorithm spendsO(c • log m) RMRs in CC model.Proof. Suppose that the batch consists of only update operations. This assumption slightly simplify the presentation, but it does not affect the bounds.The combiner gathers requests (O(c) RMRs, Line 11) and sorts them (O(c • log c) primitive steps but O(c) RMRs, Line 13). Then the combiner sets the status of requests to CONTAINS (O(c) RMRs, Line[START_REF] Aksenov | A concurrency-optimal binary search tree[END_REF][START_REF] Aksenov | Brief-announcement: Performance prediction of coarse-grained programs[END_REF][START_REF] Aksenov | Parallel combining: Benefits of explicit synchronization[END_REF][START_REF] James H Anderson | Shared-memory mutual exclusion: major research trends since 1986[END_REF][START_REF] Arbel | Reuse, don't recycle: Transforming lockfree algorithms that throw away descriptors[END_REF].The clients and the combiner participate in Contains phase. At first, each client waits for its status to change from INITIAL (1 RMR). Then the client calls contains function (O(log m) primitive steps and RMRs, Line 2). It finishes contains phase with changing its status to CONTAINS_FINISHED (O(1) RMRs, Lines 3-6).

2 List 6 L 7 X 34 // specifies the segment of leaves 35 /Figure 6 . 6 :

 267343566 Figure 6.6: Parallel Priority Queue. Classes

  induces O(p • log m • log p) work and the algorithm by Sanders [137] induces O(p • log m) work. We respond to these issues with a new parallel batched algorithm that applies a batch of size c to a queue of size m in O(c • (log c + log m)) RMRs in CC or DSM models. From this result we can get the theoretical bounds in the Work-Span framework: O(c•(log m+log c)) work and O(c•log c+log m) span. By that, our algorithm can use up to c ≈ log m processes efficiently.

Figure 6 . 9 :

 69 Figure 6.9: Split and target nodes Since we have |I| target nodes, there are exactly |I| -1 split nodes u 1 , . . . , u |I|-1 : u i is the lowest common ancestor of nodes m + i and m + i + 1. They can be found in O(|I|+log m) primitive steps (Lines 55-72): starting with node m+i go up the heap until a node becomes a left child of some node pr; this pr is u i . We omit the discussion about the fields l and r of I[i]: they represent the smallest and the largest leaf identifiers in the subtree of u i at the lowest level, and they are used to calculate the number of leaves at the lowest level that are newly inserted, i.e., m+1, . . . , m+|I|, in constant time. The combiner sets I[1].start to the root (the node with identifier 1), (Line 55) and, for each i ∈ [2, |I|], it sets I[i].start to the right child of u i-1 (node 2 • u i-1 + 1) (Line 70). Then the combiner creates an InsertSet object X, sorts the arguments of the requests in I, puts them to X.A and sets a[1].split to X (Lines 81-89). Finally, it sets the status fields of all requests in I to SIFT (Lines 91-92). Clients: Insert phase (Lines 41-85). Consider a client c with an incompleted request req. It waits while a[req.start].split is null (Lines 42-43). Now c is going to insert values from InsertSet a[req.start].split to the subtree of req.start. Let S be a local InsertSet variable initialized with a[req.start].split. For each node v on the path, c inserts values from S into the subtree of v. c calculates the minimum value x in S (Lines 51-53): the first element of S.A or the first element of S.B. If a[v].val is bigger than x, then the client removes x from S, appends a[v].val to the end of S.B and sets a[v].val to x (Lines 54-60).Note that by the algorithm S.B contains only values that were stored in the nodes above node v, thus, any value in S.B cannot be bigger than a[v].val and after appending a[v].val S.B remains sorted. Then the client calculates the number inL of target nodes in the subtree of the left child of v and the number inR of target nodes in the subtree of the right child of v (Lines 63-66, to calculate these numbers in constant time we use fields l and r of the request). If inL = 0, then all the values in S should be inserted into the subtree of the right child of v, and c proceeds with the right child 2v + 1 (Lines 69-70). If inR = 0, then, symmetrically, c proceeds with the left child 2v (Lines 73-74). Otherwise, if inL = 0 and inR = 0, v is a split node and, thus, there is a client that waits at the right child 2v + 1. Hence, c splits S to (X, Y) ← S.split(inL) (Line 77): the values in X should be inserted into the subtree of node 2v and the values in Y should be inserted into the subtree of node 2v + 1. Then c sets a[2v + 1].split to Y , sets S to X and proceeds to node 2v (Lines 78-80). When c reaches a leaf v it sets the value a[v].val to the only value in S (Lines 82-85) and sets the status of the request req to FINISHED (Line 7).If the request of the combiner is an incompleted Insert, it runs the code above as a client (Figure6.7 Lines 94-95). The combiner considers the Insert phase completed when all requests in I have status FINISHED (Lines 97-99).

  . , x min(a,b) and with values from the last a -min(a, b) nodes of the heap: the set of values in the priority queue becomes V ∪ {x 1 , . . . , x min(a,b) } \ {y 1 , . . . , y b } and the values are stored in nodes 1, . . . , |V | -a + min(a, b). Then, the sift down is initiated, but it does not change the set of values and it does not touch nodes other than 1, . . . , |V |-a+min(a, b). During Insert phase the values x min(a,b)+1 , . . . , x b are inserted and new nodes which are used in Insert phase are |V | -a + min(a, b) + 1, . . . , |V | -a + b. Thus, the final set of values is V ∪ {x 1 , . . . , x b } \ {y 1 , . . . , y a } and the values are stored in nodes 1, . . . , |V | -a + b.

Theorem 6 . 4 . 2 .

 642 Suppose that the combiner collects c requests using getRequests().Then the combiner and the clients apply these requests to a priority queue of size m using O(c + log m) RMRs in CC model each and O(c • (log c + log m)) RMRs in CC model in total.Proof. Suppose that the batch consists of a ExtractMin operations and b Insert operations.The combiner splits requests into two sets E and I (O(c) RMRs, Lines 9-16). Then it finds a nodes with the smallest values (O(a log a) primitive steps, but O(a) RMRs, Lines 18-28) using Dijkstra-like algorithm. After that, the combiner sets up ExtractMin requests, sets their status to SIFT and pairs some Insert requests with ExtractMin requests (O(a) RMRs,.The clients participate in ExtractMin phase. At first, each client waits for its status to change (1 RMR). Then the client performs at most log m iterations of the loop (Line 12): waits on the locked fields of the children (O(1) RMRs, Lines 13-17); reads the values in the children (O(1) RMRs, Line 20); compares these values with the value at the node, possibly, swap the values, lock the proper child and unlock the node (O(1) RMRs, Lines 22-29). When the client stops it changes the status (1 RMR, Line 7). The combiner waits for the change of the status of the clients (O(a) RMRs, Lines 50-52). Summing up, in ExtractMin phase each client performs O(log m) RMRs and the combiner performs O(a + log m) RMRs, giving O(c + c • log m) RMRs in total. The combiner throws away completed Insert requests (O(b) primitive steps and 0 RMRs, Line 53). Then it finds the split nodes (O(log m+b) primitive steps, but 0 RMRs, Lines 55-72). After that the combiner sorts arguments of remaining Insert requests, sets their status to SIFT and sets up the initial InsertSet (O(b • log b) primitive steps, but O(b) RMRs, Line 81 and Line 92). The clients participate in Insert phase. At first, a client t waits while the corresponding InsertSet is null (1 RMR, Lines 41-43). Suppose that it reads the InsertSet S and starts the traversal down. The client performs at most log m iterations of the loop (Line 50): choose the smallest value (O(1) RMRs, Lines 51-60), find whether to split InsertSet (O(1) RMRs, Lines 62-74), split InsertSet (calculated below, Line 77) and passe one InsertSet to another client (O(1) RMRs, Lines 78-80). Now let us calculate the number of RMRs spent in Line 77. Suppose that there are k iterations of the loop and the size of S at iteration i is s i . At the i-th iteration split works in O(min(s i+1 , s i -s i+1 )) = O(s i -s i+1 ) primitive steps and RMRs. Summing up through all iterations we get O(s 1 ) = O(b) RMRs spent by t in Line 77. Finally, t sets the value in the leaf (O(1) RMRs, Lines 82-85) and changes the status (O(1) RMRs, Line 7). The combiner waits for the change of the status of the clients (O(b) RMRs, Lines 97-99). Summing up, in Insert phase the clients and the combiner perform O(b + log m) RMRs each. Consequently, the straightforward bound on the total number of RMRs is O(c 2 + c • log m) RMRs.

  and, thus, v was moved no more than log c times. This means, that in total during the splits of InsertSets no more than c • log c values are moved to new sets, giving O(c • log c) RMRs during the splits. This gives us a total bound of O(c • (log c + log m)) RMRs during Insert phase. To summarize, the combiner and the clients perform O(c + log m) RMRs each and O(c • (log c + log m)) RMRs in total.

Figure 6 . 10 :

 610 Figure 6.10: Dynamic graph implementations

Figure 6 . 11 :

 611 Figure 6.11: Priority Queue implementations

20 h ← h • p 3 21 id 3 ←Figure 7 . 2 :

 2021372 Figure 7.2: Constructing the history for the proof of Theorem 7.4.2

Lemma 7 . 4 . 14 .

 7414 At most one out of p 1 and p 2 can have fetch&add as their next primitive step.

1 5 6

 5 operations[1...] 2 op_by_p[1..n] 3 id_by_p[1..n] 4 id ← 1 apply(pid, op): // process pid applies operation op 7 op_by_p[pid] ← op 8 move&increment(&id_by_p[pid], &id)

13 14 if 18 o

 131418 op_id_j ≤ id_by_p[pid] and 15 operations[op_id_j] = null: 16 operations[op_id_j] ← op_j 17 ← new Object() 19 for j in 1..id_by_p[pid]: 20 o.apply(operations[j]) 21 22 return o.apply(op)

Figure 7 . 3 :

 73 Figure 7.3: The universal help-free wait-free construction using move&increment

  Case 1: W + P + W ≥ (N -1) • (RI + C + W ). Each process enters the critical section without waiting in the queue. Case 2: W + P + W ≤ (N -1) • (RI + C + W ). Each process waits in the queue before entering the critical section.

Figure 8 . 3 :

 83 Figure 8.3: Examples of executions of the coarse-grained algorithm from Figure 8.2. Blue intervals depict critical sections and red intervals depict parallel sections.

  and the successor of v in succs[i]. Then insert, if a node with value v does not exist (i.e., value in preds[0].next[0] is not v), creates a new node node (with random top level as described above) and links it from bottom up to its level: preds[i].next[i] is set to the new node and node.next[i] is set to succs[i]. And delete, if a node node with value v exists (i.e., value in preds[0].next[0] is v), unlinks node at each level up to the level of node: preds[i].next[i] is set to succs[i].next[i].

Table 4

 4

	Type T	Size grain	Time Comment
	char	800M 1	1.963 100x slower
		10	0.330 17x slower
		5000 (TBB-rec.) 0.020 optimal
		auto (ours)	0.020 optimal
	char[64]	200M 1	0.129 78% slower
		10 (TBB-rec.)	0.077 6% slower
		5000	0.072 optimal
		auto (ours)	0.073 optimal
	char[2048] 0.4M 1 (TBB-rec.)	0.049 optimal
		10	0.050 optimal
		5000	0.057 16% slower
		auto (ours)	0.050 optimal
	char[131072] 0.01M 1 (TBB-rec.)	0.075 optimal
		10	0.075 optimal
		5000	1.419 19x slower
		auto (ours)	0.075 optimal

.1: Running times on 40 cores for inputs of various sizes, for manually-fixed grain sizes, including the one obtained following Intel's TBB manual, and for our algorithm.

Statement of the Bounds Definition 4.4.12

  (Bound on the number of processes). Let P denote the number of processes involved in the evaluation.

Definition 4.4.13 (Bound on the number of spguards). Let G denote the total number of different spguards occuring in the source code of the program. Definition 4.4.14 (Auxiliary parameter F ). We define F = 1 + log α κ DE , where κ is expressed in number of machine cycles. Definition 4.4.15 (Auxiliary parameter H). We let H = log γ/(γ-1) κ DE , where κ is expressed in number of machine cycles.

  Thus, the raw work corresponds to the sequential work, i.e. W (t, I, X) = W s (t, I). Besides, by definition of work, we know that w(t, I) = W (t, I, X ), where X is the trace that systematically selects parallel bodies. By Lemma 4.4.4, w(t, I) = W (t, I, X ) ≥ W

s (t, I) = W (t, I, X)

4.4.5 Proofs

Lemma 4.4.6. If a spguard g executes its sequential body, then its sequential work is bounded as follows:

Table 4 .

 4 .7 give the execution time for the flat PBBS BFS, and for its nested counterpart, using a

		PBBS (s) Ours
	blockradix-sort
	random	0.20	-7.4%
	exponential	0.19	-8.4%
	random kvp 256	0.49	-23.9%
	random kvp 10 8	0.49	-27.7%
	comparison-sort
	random	1.13	-36.4%
	exponential	0.82	-31.3%
	almost sorted	0.63	-18.8%
	suffix-array
	trigrams	3.58	-6.3%
	dna	1.29	-6.7%
	text	4.11	-7.4%
	wiki	3.66	-5.3%
	convex-hull
	in circle	0.61	+5.8%
	kuzmin	0.41	-6.9%
	on circle	8.26	-32.4%
	nearest-neighbours
	in square	5.75	-2.2%
	kuzmin	22.00	-2.5%
	in cube	7.90	-6.5%
	on sphere	14.60	-31.2%
	plummer	23.54	-2.5%
	ray-cast
	in cube	7.90	-1.9%
	on sphere	0.87	-0.2%
	happy	0.50	-1.9%
	xyz-rgb manuscript	9.46	+0.3%
	turbine	4.10	-2.1%
	delaunay
	in square	3.39	-4.1%
	kuzmin	3.99	-4.4%
	mis
	cube-grid	0.12	+1.2%
	rMat24	0.07	+2.7%
	rMat27	0.06	+2.7%
	mst
	cube-grid	2.28	-9.9%
	rMat24	2.21	-13.3%
	rMat27	1.89	-16.3%
	spanning
	cube-grid	0.62	-5.8%
	rMat24	0.44	-0.6%
	rMat27	0.33	-5.0%

2: Results from PBBS benchmarks, executed on 40 cores, averaged over 30 runs. Figures show that automatic granularity control can be achieved at the expense of small overheads, and that in fact it often decreases the parallel runtime compared with carefully hand-tuned code.

  . It radically differs from the previous techniques. At first, it compiles different versions of each function: the version p 1 where all immediate calls in p are inlined; the version p 2 where all immediate calls in p 1 are inlined; etc. These versions are used to avoid overheads related to function calls. Second, the algorithm maintains two variables: a task demand (keeps track of other worker's unfulfilled attempts to steal tasks from the worker) and a queue length (how many tasks the worker currently has). Using a task demand and a queue length the algorithm decides which version of a function to execute: s, p, p 1 , p 2 , etc. Unfortunately, this approach can oversequentialize.

		PBBS (s) Ours PBBS (s) Ours
	blockradix-sort blockradix-sort
	random random	0.33 0.12	-2.3% -10.0%
	exponential exponential	0.32 0.11	-2.5% -7.8%
	random kvp 256 random kvp 256	0.58 0.24	+1.1% -20.6%
	random kvp 10 8 random kvp 10 8	0.58 0.24	-0.0% -20.6%
	comparison-sort comparison-sort
	random random	1.24 0.50	-31.2% -34.9%
	exponential exponential	0.96 0.38	-25.5% -29.6%
	almost sorted almost sorted	0.82 0.26	-19.9% -15.3%
	suffix-array suffix-array
	trigrams trigrams	7.07 2.17	+0.8% +0.4%
	dna dna	4.01 1.19	+2.0% -3.2%
	text text	9.21 2.80	+0.9% -1.1%
	wiki wiki	7.60 2.34	+2.2% +0.7%
	convex-hull convex-hull
	in circle in circle	1.04 0.39	+5.6% -4.2%
	kuzmin kuzmin	0.73 0.25	+2.1% -11.9%
	on circle on circle	10.53 3.86	-18.1% -30.5%
	nearest-neighbours nearest-neighbours
	in square in square	8.65 2.69	+1.6% -1.1%
	kuzmin kuzmin	32.57 12.54	+1.3% +1.9%
	in cube in cube	10.75 3.42	+0.3% -2.4%
	on sphere on sphere	33.38 14.40	+0.0% +5.0%
	plummer plummer	36.51 16.39	+0.3% -0.3%
	ray-cast ray-cast
	in cube in cube	12.94 3.01	-3.5% -4.4%
	on sphere on sphere	2.09 0.77	+1.8% +4.3%
	happy happy	1.75 0.69	+4.8% +5.6%
	xyz-rgb manuscript xyz-rgb manuscript	15.78 5.81	-0.4% +2.8%
	turbine turbine	9.80 2.40	-0.2% +2.6%
	delaunay delaunay
	in square in square	7.33 2.43	-1.0% -4.3%
	kuzmin kuzmin	8.73 2.89	-4.5% -8.9%
	mis mis
	cube-grid cube-grid	0.25 0.09	-1.4% -5.7%
	rMat24 rMat24	0.17 0.06	+0.4% +3.0%
	rMat27 rMat27	0.14 0.05	-4.5% +1.8%
	mst mst
	cube-grid cube-grid	3.94 1.34	-9.6% -13.0%
	rMat24 rMat24	4.09 1.24	-9.0% -8.4%
	rMat27 rMat27	3.29 1.12	-6.5% -11.6%
	spanning spanning
	cube-grid cube-grid	1.09 0.37	+0.0% -8.3%
	rMat24 rMat24	0.81 0.27	+0.1% -3.8%
	rMat27 rMat27	0.58 0.21	-3.0% -2.3%
	Figure 4.10: Results from PBBS benchmarks, executed on the 48-core AMD machine,
	averaged over 30 runs.

Figure 4.11: Results from PBBS benchmarks, executed on the 72-core Intel machine, averaged over 30 runs. the parallel version p; the sequentialized version s;

  .4 Lines 59, 63, 78, 83, 94-95 and 102-103). Secondly, the delete operation updates the links to physically remove the node (Figure 5.4 Lines 60, 64, 79, 84, 96 and 104).

	1 class RWLock:	15	unlock():
	2	int lock ← 0		16	if lock = 1:
	3			17	compare&swap(lock, 1, 0)
	4	writeLock():		18	else:
	5	while true:		19	while true:
	6	if compare&swap(lock, 0, 1):	20	now ← lock
	7	break		21	if compare&swap(lock, now, now -2):
	8			22	break
	9	readLock():
	10	while true:
	11	now ← lock
	12	Shared variables if now % 2 = 0 and	34	prev.left ← newNode
	13	node is a record with fields: compare&swap(lock, now, now + 2):	35	else:
	14	val, its value break		36	prev.right ← newNode
		left, its pointer to the left child
		right, its pointer to the right child state ∈ {DATA, ROUTING}, its state Figure 5.2: Read-write lock implementation 37 delete(v): bool 38 gprev, prev, curr ← traversal(v)
				39	if curr = null or curr.state = DATA
		Initially the tree contains one node root, root.val ← +∞	40	return false	// v is not in the set
		root.state ← DATA
	1 traversal(v): Node, Node, Node
	2	// wait-free traversal
	3	gprev ← null
	4	prev ← null
	5	curr ← root	// start from root
	24	else:	// prev is a place to insert
	25	Lines 32-36
	26	return true
		Update existing node
	28	if curr.state = DATA:
	29	return false	// v is already in the set
	30	curr.state ← DATA
		Insert new node
	32	newNode.val ← v // allocate a new node
	33	if v < prev.val:

6 while curr = null: 7 if curr.val = v: 8 break 9 else: 10 gprev ← prev 11 prev ← curr 12 if curr.val < v: 13 curr ← curr.left 14 else: 15 curr ← curr.right 16 return gprev, prev, curr 17 contains(v): bool // wait-free contains 18 gprev, prev, curr ← traversal(v) 19 return curr = null and curr.state = DATA 20 insert(v): bool 21 gprev, prev, curr ← traversal(v) 22 if curr = null: // curr has value v 23 Lines 28-30

  Transition from depth k to depth k + 1. Consider a modified node v at depth k + 1 and its parent p. Suppose that p was visited by a client with InsertSet S p . By induction, S p .A is sorted and S p .B is sorted and contains only the values that were in ancestors. Then the client chooses the smallest value in p: either a[p], the first value of S p .A or the first value of S p .B. Note that after any of these three cases S p .A and S p .B are sorted and S p .B contains only values from ancestors and node p: • a[p] is the smallest, then S p .A and S p .B are not modified; • we poll the first element of S p .A or S p .B; S p .A and S p .B are still sorted; then we append a[p] to S p .B, and a[p] has to be the biggest element in S p .B, since S p .B contains only the values from ancestors. Then the client splits S p and some client, possibly, another one, works on v with IntegerSet S. Since, S is a subset of S p then S.A is sorted and S.B is sorted and contains only the values from ancestors (ancestors of p and, possibly, p). Finally, the client chooses the smallest value to appear in the subtree: the first value of S.A, the first value of S.B and a[v].

  .proc and, then, checks a[2v].locked and a[2v + 1].locked. If some of them are true then it spins on specifically created local variables: on notif y 2v if a[2v].locked is true, and on notif y 2v+1 if a[2v + 1].locked is true. Then, the algorithm standardly performs swapping routine. At the end, it unlocks the node, i.e., sets a[v].locked to false, then, reads a process a[v/2].proc and notifies it by setting its corresponding variable notif y v .

  Definition 7.3.1 (Decided before). For a history h in a set of histories H, a linearization function f over H, and two operations op 1 and op 2 , we say that op 1 is decided before op 2 in h with respect to f and H, if there exists no extension s ∈ H of h such that op 2 ≺ f (s) op 1 . (Helping). A set of histories H with a linearization function f over H is help-free if for every h ∈ H, every two operations op 1 , op 2 , and a single computation step γ such that h • γ ∈ H it holds that if op 1 is decided before op 2 in h • γ and op 1 is not decided before op 2 in h then γ is a step in the execution of op 1 .

	Definition 7.3.2

  Proof. Suppose that op 1 is not decided before op 3 in h then there exists a extension s of h for which op 3 ≺ f (s) op 1 . Since op 2 is linearized in f (s) and op 1 is decided before op 2 then op 1 ≺ f (s) op 2 . Together, op 3 ≺ f (s) op 1 ≺ f (s) op 2 contradicting our assumption that op 2 is decided before op 3 in h.

		2
	4 while true:	// outer loop
	5	op 2 ← push(id 2 )
	6	while true:	// inner loop
	12	continue
	13	break
	14	h ← h • p 2
	15	h ← h • p 1
	16	while op 2 is not completed:
	17	h ← h • p 2
	18	id 2 ← id 2 + 1
		Figure 7.1: Constructing the history for the proof of Theorem 7.4.1

7

if op 1 is not decided before op 2 in h • p 1 :

8 h ← h • p 1 9 continue 10 if op 2 is not decided before op 1 in h • p 2 : 11 h ← h • p 2 Lemma 7.4.2.

For any linearization function f and finite history h, if an operation op 1 of a process p 1 is decided before an operation op 2 of a process p 2 , then op 1 must be decided before any operation op that has not started in h.

Proof. Consider h , the extension of h, in which p 2 runs solo until op 2 completes. Such an extension exists, as our considered implementation Q is wait-free. By Observation 7.4.1 (1), op 2 is decided before op in h , and, consequently, by Transitivity Lemma 7.4.1, op 1 is decided before op in h .

  1 and p 2 is the value that appears in the designated address.

		while true:	// inner loop
	8	moved ← False
	9	for i in 1..3:
	11	h ← h • p i
	12	moved ← True
	13	if not moved:
	14	break
	15
	16	h ← h • p 3
	17	// let p k be the process whose next primitive is compare&swap
	18	h ← h • p k
	19	while op 3 is not completed:

10

if op i is not decided before any op j in h • p i :

Observation 7.4.6.

  From the previous lemma we know that the next primitive step of at least one process p 1 or p 2 is compare&swap. Let it be process p k . By algorithm, p 3 takes a step at Line 16 changing the memory location either by fetch&add or by a successful compare&swap, thus, the next step of p k at Line 18 should be a failed compare&swap. Immediately after Line 16, op 3 is decided before op 1 and op 2 . Immediately after Line 18, op 1 and op 2 are not decided before any operation of p 4 .

	Observation 7.4.7. Lemma 7.4.15.

  Node next ← swap(&head, my_node) // W or X The coarse-grained operation with inlined lock and unlock functions 2. In Line 10, the algorithm loops on a field next.locked. During this loop one or two cache misses happens.

	2	bool locked
	3
	4 global Node head ← new Node() // global
	5 Node my_node	// per process
	6	my_node.locked ← true
	7
	8 operation():
	10	while (next.locked) {}	// R I or 2 • R I
	11	for i in 1..C:	// C
	12	nop
	13	my_node.locked ← false	// W
	14	my_node ← next
	15	my_node.locked ← true	// W
	16	for i in 1..P:	// P
	17	nop
		Figure 8.2:

1 class Node: 9

Large Geometric Models Archive at Georgia Institute of Technology http://www.cc.gatech.edu/ projects/large_models/blade.html

Note that sequential combining[START_REF] Drachsler-Cohen | Lcd: Local combining on demand[END_REF][START_REF] Fatourou | Revisiting the combining synchronization technique[END_REF][START_REF] Hendler | Flat combining and the synchronization-parallelism tradeoff[END_REF][START_REF] Oyama | Executing parallel programs with synchronization bottlenecks efficiently[END_REF], typically used in the concurrency-averse case, is a degenerate case of our parallel combining in which the combiner applies the batch sequentially.

We are aware of the cache-friendly priority queue by Braginsky et al.[START_REF] Braginsky | Cbpq: High performance lock-free priority queue[END_REF], but we do not have its Java implementation.

The new values of the compare&swap steps of p 1 and p 2 are different from the expected value.

Accuracy of Time Measurements

We assume that time measured for a sequential execution may diverge from the sequential work by up to some multiplicative factor E, in either direction. Formally, we assume the existence of constant E such that, for any sequential execution of a purely sequential term

 [START_REF] Gramoli | More than you ever wanted to know about synchronization: synchrobench, measuring the impact of the synchronization on concurrent algorithms[END_REF]. In order to make the comparison equitable, we remove rotation routines from the CF-, LO-and BCCO-trees implementations. We are aware of efficient lock-free tree by Natarajan and Mittal ([120]), but unfortunately we were unable to find it written on Java.

Experimental Methodology

For our experiments, we use the environment provided by the synchrobench library. To compare the performance we considered the following parameters:

• Workloads. Each workload distribution is characterized by the percent x% of update operations. This means that the tree will be requested to make 100 -x% of contains calls, x/2% of insert calls and x/2% of delete calls. We considered three different workload distributions: 0%, 20% and 100%.

• Tree size. On the workloads described above, the tree size depends on the size of the key space (the size is approximately half of the range). We consider three different key ranges [0, R), where R is 2 15 , 2 19 or 2 21 .

• Degree of contention. This depends on the number of cores in a machine. We take enough points to reason about the behaviour of curves.

We prepopulate the tree with, approximately, R/2 values: we take each value from the range with probability 1 2 . Then we start P processes. Each process repeatedly performs operations: (1) with probability 1 -x, it searches for a random value taken uniformly from range; (2) with probability x 2 , it inserts a random value taken uniformly from range; (3) with probability x 2 , it deletes a random value taken uniformly from range. Under such workloads, the size of the tree is always approximately half the range, R/2, and, thus, only half of update operations succeed.

We chose the settings such that we had two extremes and one middle point. For workload, we chose 20% of attempted updates as a middle point.

Results

To get meaningful results we average through up to 25 runs. Each run is carried out for 10 seconds with a warmup of 5 seconds. Figure 5.6a (and resp. 5.6b) contains the results of executions on Intel (and resp. AMD) machine. It can be seen that with the increase of the size the performance of our algorithm becomes better relatively to CF-tree. This is At first, the client splits the provided tree in tree field by the argument v into two trees l and r. Then the client writes the tree l to tree field of leftR request (Line 23) and wakes up the owner of leftR by setting its status to UPDATE (Line 24). It does the same with rightR request (Lines 25-27). The client waits until the owner of leftR request finishes its work (Lines 30-32) , i.e., the status changes from CONTAINS_FINISHED, and stores the resulting tree in l (Line 33). It does the same with rightR request (Lines 35-39). Then the client joins trees l and r with the argument of its request if the request is insert (Line 42) and without, otherwise (Line 44). Finally, it sets the status of its request to FINISHED and ends the execution.

The combiner runs the same code as clients (Lines 70-75). Then it waits until the first working client finishes, i.e., its status changes from CONTAINS_FINISHED (Lines 77-78) and, finally, updates the tree, i.e., root (Line 79).

Analysis

We provide theorems on correctness and time bounds. Theorem 6.3.1. Our concurrent binary search tree implementation is linearizable.

Proof. The execution can be split into combining phases (Figure 6.4) which do not intersect. We group the operations into the batches corresponding to the combining phase in which they are applied.

Consider the i-th combining phase. We linearize all the operations from the i-th phase right after the end of the corresponding getRequests() (Figure 6.4 Line 11) in order: at first, contains operations and, then, update operations ordered by sorting in Figure 6.4 Line 13. The heap is defined by its size m and an array a of Node objects. Node object (Figure 6.6 Lines 21-24) has three fields: value val, boolean locked and InsertSet split.

STATUS_SET consists of three items: INITIAL, SIFT and FINISHED.

A Request object (Figure 6.6 Lines 26-36) consists of: a method method to be called and its input argument v; a result res field; a status field; a node identifier start.

ExtractMin Phase

Combiner: ExtractMin preparation (Figure 6.7 Lines 1-52). First, the combiner withdraws requests A from the combining data structure C (Line 1). If the size of A is larger than m, the combiner serves the requests sequentially (Lines 3-7). Intuitively, in this case, there is no way to parallelize the execution. For example, if A consists of only Insert requests and if there are more Insert requests than the number of nodes in the corresponding binary tree, we cannot insert them in parallel.

In the following, we assume that the size of the queue is at least the size of A. The combiner splits A into sets E and I (Lines Finally, the combiner sets the status of all requests in E to SIFT (Lines 44-45). Clients: ExtractMin phase (Figure 6.8 Lines 11-29). Briefly, the clients sift down the values in nodes v 1 , . . . , v |E| in parallel using hand-over-hand locking: the locked field of a node is set whenever there is a sift down operation working on that node.

A client c waits until the status of its request becomes SIFT. c starts sifting down from req.start. Suppose that c is currently at node v. c waits until the locked fields of the children become false (Lines 13-17). If a [v].val, the value of v, is less than the values in its children, then sift down is finished (Lines 23-24): c unsets a [v].locked and sets the status of its request to FINISHED. Otherwise, let w be the child with the smallest value.

Then c swaps a[v].val and a[w].val, sets a[w].locked, unsets a[v].locked and continues with node w (Lines 26-29).

If the request of the combiner is ExtractMin, it also runs the code above as a client (Figure 6.7 Lines 47-48). The combiner considers the ExtractMin phase completed when all requests in E have status FINISHED (Lines 50-52).

Insert Phase

Combiner: Insert preparation (Figure 6.7 Lines 53-99). For Insert requests, the combiner removes all completed requests from I (Line 53). Nodes m + 1, . . . , m + |I| have to be leaves, because we assume that the size of I is at most the size of the queue. We call these leaves target nodes. The combiner then finds all split nodes: nodes for which the subtrees of both children contain at least one target node. (See Figure 6.9 for an example of how target and split nodes can be defined.)

• op = pop, W n+1 = pop. Then, as two sequences we can choose W (n + 1)

and

In both of them the operations in R return the same responses, since W n+1 • op and op • W n+1 perform two pop operations.

• op = push(a), W n+1 = pop. For the first sequence we take A = W (n+1) • op • R(m). Now, we choose the second sequence B from W (n) • op • (R(m)+W n+1 ?). Let W n+1 pop in A the x-th element from the bottom of the stack. We extend W (n) • op in B with operations from R(m) until some operation op tries to pop the x-th element from the bottom. Note that all operations R(m) up to op (not including op ) return the same results in A and B. If such op does not exist then we are done. Otherwise, we insert W n+1 right before op , i.e., pop this element. Subsequent operations in R(m) are not affected, i.e., results of operations in R(m) are the same in A and B.

• op = pop, W n+1 = push(b). This case is symmetric to the previous one.

• op = push(a), W n+1 = push(b). For the first sequence, we take The contradiction implies that stack is not an exact order type.

Wait-Free Stack Cannot Be Help-Free

In this section, we prove that there does not exist a help-free wait-free implementation of stack in a system with reads, writes, and compare&swaps. We then extend the proof to the case when a system has one more primitive fetch&add.

Help-Free Stacks Using Reads, Writes and Compare&Swap

Suppose that there exists such a help-free stack implementation Q that uses read, write, and compare&swap primitives. All further lemmas and theorems assume this implementation. We establish a contradiction by presenting a history h in which some operation takes infinitely many steps without completing. All further lemmas and theorems assume the We start with two observations that immediately follow from the definition of linearizability.

Observation 7.4.1. In any history h:

1. Once an operation is completed it must be decided before all operations that have not yet started;

2. If an operation is not started it cannot be decided before any operation of a different process.

Lemma 7.4.1 (Transitivity). For any linearization function f and finite history h, if an operation op 2 is completed in h, an operation op 1 is decided before op 2 in h and op 2 is decided before an operation op 3 in h then op 1 is decided before op 3 in h.

Proof. Suppose that the next primitive steps by p 1 and p 2 are to different locations. Consider two histories:

Let us look at the first two pop() operations by p 3 . Executed after h they have to return id 2 then 1, since op 1 is decided before op 2 in h and both of them are completed. While executed after h they have to return 1 then id 2 . But the local states of p 3 and shared memory states after h and h are identical and, thus, two pops of p 3 must return the same values -a contradiction. The same argument will apply when both steps by p 1 and p 2 are reads. Suppose that the next operation of p 1 is a write. (The case when the next operation of p 2 is write is symmetric) Consider two histories:

Let the process p 1 perform two pop() operations (op 1 and op 1 ) and p 2 complete its operation after h : op 1 and op 1 have to return 1 and id 2 , correspondingly, since op 1 and op 2 are completed and op 2 is decided before op 1 in h . Again, since the local states of p 1 and the shared memory states after h and h are identical, op 1 and op 1 performed by p 1 after h must return 1 and id 2 . Hence, op 2 has to be decided before op 1 in h = h • perform op 1 • perform op 1 and, by Lemma 7.4.2, op 2 has to be decided before any operation of p 3 in h. Since only p 1 performs steps after h in h and Q is help-free, op 2 has to be decided before any operation of p 3 at h, contradicting Lemma 7.4.3. Thus, both primitives have to be compare&swap.

By the same argument both compare&swap steps by p 1 and p 2 have the expected value that is equal to the current value in the designated memory location, and the new value is different from the expected. If it does not hold, either the local states of p 1 and the shared memory states after h • p 1 and h • p 2 • p 1 are identical or the local state of p 2 and the shared memory states after h • p 2 and h • p 1 • p 2 are identical. Proof. By Lemma 7.4.3, the order between op 1 and any operation of p 3 is not decided before Line 14. Since Q is help-free, the steps by p 2 cannot fix the order between op 1 and any operation of p 3 . Thus, the only step that can fix the order of op 1 and some operation of p 3 is a step by p 1 at Line 15, i.e., a failed compare&swap.

Suppose that op 1 is decided before some operation op 3 of p 3 after Line 15. Let h be the history right before Line 14. Consider two histories h = h • p 2 • p 1 and h = h • p 2 . Let p 3 solo run pop operations after h until it completes operation op 3 and then, further, until pop operation returns ⊥, i.e., the stack is empty. Since op 1 is decided before op 3 , some completed operation op 3 of p 3 has to return 1: if we now complete op 1 it should be linearized before op 3 . Now let p 3 to perform after h the same number of operations as it did after h . Since the local states of p 3 and the shared memory states after h and h are identical (p 1 makes the failed compare&swap), op 3 after h has to return 1 as after h . Thus, op 1 is decided before op 3 in h . Since Q is help-free and p 1 does not take steps after h in h , op 1 has to be decided before op 3 before Line 14, contradicting Lemma 7.4.3. Lemma 7.4.8. At the end of the outer loop (Line 18) the order between op 1 and next op 2 = push(id 2 + 1) is not yet decided.

Proof. The operation op 2 is not started, thus, it cannot be decided before op 1 by Observation 7.4.1 [START_REF] Umut | Algorithm Design: Theory and Practice[END_REF].

Suppose that op 1 is decided before op 2 . By Lemma 7.4.2 op 1 has to be decided before all operations of p 3 , contradicting Lemma 7.4.7.

Proof. From the first invariant, op i cannot be decided before op j prior to the inner loop (Lines 7-14). Since Q is help-free, during the inner loop op i can become decided before op j only after a step by p i which is impossible due to the check in Line 10. Lemma 7.4.9. During (and right after) an execution of the inner loop (Lines 7-14) op 1 , op 2 and op 3 cannot be decided before any operation of p 4 .

Proof. Suppose that during an execution of the inner loop op 1 , op 2 or op 3 is decided before some operation of p 4 .

At the beginning of the loop, none of op 1 , op 2 and op 3 is decided before any operation of p 4 : op 1 and op 2 are not decided because of the second invariant, while op 3 is not yet started. Suppose that during the execution of the inner loop some op i becomes decided before some operations of p 4 .

Let us look at the execution and find the first point in time when some op k of p k is decided before some operation op 4 of p 4 . Using the same argument as in the proof of Lemma 7.4.3, we can show that op k has to be decided before any other op j contradicting Observation 7.4.5: we let p 4 run until the operation op 4 is completed and, further, while stack is not empty; op 4 becomes decided before op j ; by Transitivity Lemma 7.4.1, op k is decided before op j .

The proofs of the following two lemmas are identical to those of Lemmas 7.4.4 and 7.4.5.

Lemma 7.4.10. For each i, 1 ≤ i ≤ 3, op i cannot be completed after the inner loop (Lines 7-14). Lemma 7.4.11. The execution of the inner loop (Lines 7-14) is finite. Lemma 7.4.12. For all i, j,

Proof. Consider an operation of process i. At the end of the inner loop op i should be decided before some op k in h • p i , otherwise, p i can make at least one more step during the inner loop. Thus, by Lemma 7.4.2 op i should be decided before op 4 , the first operation of p 4 . Let p 4 run pop operations after h • p i until one of them returns ⊥, i.e., the stack is empty. Let this history be h . By Lemma 7.4.9, op j is not decided before any operation of p 4 in h. Since Q is help-free and only p i and p 4 takes steps in h after h, op j cannot be decided before any operation of p 4 in h , and, consequently, operations of p 4 cannot pop an argument of op j . Since the operations of p 4 empty the stack, op j must be linearized after them. Thus, op 4 is decided before op j in h . By Transitivity Lemma 7.4.1, op i is decided before op j in h . Finally, since Q is help-free and only p 4 takes steps in h after h • p i , op i is decided before op j in h • p i . Lemma 7.4.13. Immediately before Line 16 the following holds:

1. The next primitive step by p i for 1 ≤ i ≤ 3 is to the same memory location.

2. The next primitive step by p i for 1 ≤ i ≤ 3 is fetch&add with a non-zero argument or compare&swap for which the expected value is the value that appears in the designated location and the new value is different from the expected one.

Proof. Suppose that for some pair p i and p j the next steps are to different memory locations. We consider two histories h

Model Assumptions

Below we list basic assumptions on the abstract machine used for our analytical throughput prediction.

First, we assume that coherence of caches is maintained by a variant of MESI protocol [START_REF] Mark | A low-overhead coherence solution for multiprocessors with private cache memories[END_REF]. Each cache line can be in one of four states: Modified (M), Exclusive (E), Shared (S) and Invalid (I). MESI regulates transitions between states of a cache line and responses depending on the request (read or write) to the cache line by a process or on the request to the memory bus. The important transitions for us are: (1) upon reading, the state of the cache line does not change if it was not I, otherwise, the state becomes S, and, if the state was I, then a read request is sent to the bus; (2) upon writing, the state of the cache line becomes M, and, if the state was S or I, an invalidation request is sent to the bus.

We assume that the caches are symmetric: for each MESI state st, there exist two constants R st and W st such that any read from any cache line with status st takes R st work and any write to a cache line with status st takes W st work. Tudor et al. [START_REF] David | Everything you always wanted to know about synchronization but were afraid to ask[END_REF] showed that for an Intel Xeon machine (similar to the one we use in our experimental validation below), given the relative location of a cache line with respect to the process (whether they are located on the same socket or not), the following hypotheses hold:

(1) writes induce the same work, regardless of the state of the cache line; (2) swaps, not concurrent with other swaps, induce the same work as writes. Therefore, we assume that (1) W = W M = W E = W S = W I and (2) any contention-free swap induces a work of size W .

CLH Lock

Multiple lock implementations have been previously proposed, from simple spinlocks and TTAS to more advanced MCS [START_REF] Scott | Algorithms for scalable synchronization on shared-memory multiprocessors[END_REF] and CLH [START_REF] Craig | Building fifo and priorityqueuing spin locks from atomic swap[END_REF]. For our analysis, we choose CLH, as the simplest lock among those considered to be efficient. In Figure 8.2, we inline lock and unlock calls to CLH lock in our abstract coarse-grained operation.

Cost of an Operation

Let us zoom into what happens during the execution of the operation.

Note that at the beginning of an operation (unless it is the very first invocation), my_node.locked is loaded into the cache and the corresponding cache line is in state M, because of the set in Line 15 during the previous operation by the same process.

1. The operation starts with swap (Line 9) that induces a work of size W , if not concurrent with other swaps, and a work of size at most X, otherwise. In Figure 8.4, we show our experimental results (blue curves) compared with our theoretical prediction (red curves). The two curves match very closely, except for the case of small C and P where our predicted throughput underestimates the real one. We relate this to the fact that we oversimplified the abstract machine: any write induces the work of constant size W , regardless of the relative location of the cache line with respect to the process. For small C and P , two processes from the same socket are more likely to take the lock one after the other and, thus, on average, a write might induce less work than W , and, consequently, the throughput can be higher than predicted.

Conclusion

In this short chapter, we showed that a simple theoretical analysis may quite accurately predict the throughput of data structures implemented using coarse-grained synchronization. For the moment, our analysis is restricted to algorithms using CLH-based locking in systems obeying the uniform scheduler. As a future work, we intend to extend the analysis to more realistic algorithm designs, lock implementations and architectures.