PROGRAMMATION BSP IMPÉRATIVE

1.2

Programmation BSP impérative

Différents types d'opérations BSP

.1 résume et compare les bibliothèques considérées pour la programmation bsp.

Choix pour BSP-Why

Les bibliothèques bsp présentent un vaste ensemble de primitives, et il est nécessaire de faire un choix pour les routines qui seront autorisées dans notre langage bsp. L'un des objectifs à court terme est de pouvoir prouver des programmes mpi bien structurés, pour cette raison il est nécessaire d'inclure les opérations collectives et la synchronisation sur un sous-groupe. Au contraire, les opérations hauteperformance présentes dans la bibliothèque PUB rompent le déterminisme du modèle bsp, à la base de notre approche, et ne seront donc pas autorisées.

Erreurs fréquentes dans les programmes BSP

Dans l'optique de vérifier des programmes bsp, il est naturel de s'intéresser aux nouveaux types d'erreurs (bugs) introduites dans les programmes bsp, en plus des erreurs classiques de la programmation séquentielle.

• Les deadlocks (interblocages) sont presque entièrement éliminés par le modèle bsp. Il reste cependant une situation où un deadlock peut se produire. C'est le cas lorsqu'au moins un processeur termine son exécution, alors que d'autres processeurs attendaient une synchronisation.

• Communications en dehors des bornes et non-déterministes. Si un processeur essaie d'écrire sur un buffer d'un autre processeur, au dela de sa taille, une erreur risque de se produire. D'autre part, si deux (ou plusieurs) processeurs tentent d'écrire sur la même variable du même processeur lors d'une même synchronisation, il est a priori impossible de prédire le résultat.

L'outil bsp-why devra permettre de répondre à ces deux types d'erreurs. Nous verrons dans le chapitre 3 comment cela est fait.

L'outil BSP-Why

Le langage BSP-Why-ML

Comme expliqué en introduction, l'idée de notre travail est de transformer un programme bsp en un programme séquentiel qui simule l'exécution du programme parallèle, puis d'utiliser les outils existants pour la vérification de programmes séquentiels. Nous avons choisi d'utiliser l'outil why pour prouver le programme séquentiel, qui sera donc écrit dans le langage why-ML.

Nous travaillerons sur des programmes parallèles écrits dans le langage bsp-why-ML. Nous avons défini ce langage comme une extension de why-ML, en ajoutant les opérations nécessaires pour le parallélisme.

Ainsi, les instructions pour la synchronisation et les communications ont été ajoutées. Le langage logique des assertions est également complété, afin de permettre d'exprimer des propriétés sur les valeurs d'une variable sur les différents processeurs.

Transformation des programmes BSP-Why-ML

Le programme parallèle bsp-why-ML est transformé par notre outil en un programme séquentiel why-ML. Cette transformation s'effectue en plusieurs phases, présentées dans la figure 1.2.

• Tout d'abord, nous effectuons une détection des blocs de code séquentiels du programme. Il s'agit des codes dans lequel le parallèlisme n'intervient pas (il n'y a donc pas de synchronisation).

Cette étape de la transformation a pour résultat un arbre, dont les feuilles sont des blocs de code séquentiels, et dont les noeuds correspondent à la structure du programme parallèle.

• L'étape suivante est alors la transformation de cet arbre, depuis un code bsp-why-ML vers un code why-ML. Cette transformation mets en place la structure du programme séquentiel qui sera généré.

1.6. CONCLUSION

Conclusion 1.6.1 Résumé des contributions

Dans la chapitre 3 de la thèse, nous avons réalisé une étude comparatives des différentes librairies pour la programmation bsp, afin de déterminer quelles fonctionnalités sont essentielles à étudier. Dans le chapitre 4, nous avons présenté bsp-why, notre outil de preuve de programmes parallèles. Nous avons présenté le langage bsp-why-ML, une extension du langage existant why-ML, et donné une transformation du programme parallèle vers le programme séquentiel afin de pouvoir le vérifier avec why. Dans le chapitre 5, nous avons donné des applications de notre outil bsp-why. Nous avons prouvé plusieurs algorithmes bsp classiques, ainsi qu'un exemple plus complexe de génération de l'espace d'états pour le model checking. Enfin, dans le chapitre 6, nous avons présenté les élements nécessaires pour une plus grande confiance en l'outil bsp-why. Nous avons défini des sémantiques formelles pour le langage, et prouvé les propriétés naturelles de celui-ci. Enfin, nous avons donné une preuve de la transformation du programme parallèle vers le programme séquentiel.

Perspectives

Nous avons l'intention de poursuivre le travail de cette thèse dans différentes directions. De même que le langage intermédiaire why peut-être utilisé pour prouver des programmes écrits en C ou Java, il serait souhaitable de pouvoir vérifier directement des programmes écrits en C-BSP. Un autre objectif est la vérification d'algorithmes mpi, lorsqu'il sont écrits dans un style bsp, c'est à dire avec des opérations collectives. Cela pourra être réalisé en utilisation la sous-synchronisation. Une autre piste de travail est la poursuite de la vérification d'algorithmes de model-checking, pour des algorithmes plus complexes et plus proches de ceux utilisés en pratique (avec par exemple la logique LTL, CTL*, etc.)

Introduction

Contexte

De nos jours, l'informatique des programmes séquentiels laisse de plus en plus sa place au parallèlisme, qu'il s'agisse d'un simple téléphone (les smartphones modernes ont bien souvent quatre coeurs), d'un ordinateur de bureau ou d'un super-calculateur.

Cependant, si la programmation séquentielle a évolué pour donner lieux à des méthodes de programmation robustes et de haut niveau, ce n'est pas encore le cas pour la programmation parallèle, qui utilise bien souvent des langages de bas niveau, avec des coûts de débogage importants. De plus, un échecs lors d'un calcul parallèle a généralement un impact plus important que pour un programme séquentiel, par exemple s'il faut ré-exécuter un calcul sur un super-calculateur.

Pour ces raisons, il semble essentiel de pouvoir vérifier formellement les programmes parallèles. Pour les programmes séquentiels, les méthodes existent et sont bien rodées, mais plusieurs difficultés compliquent la tache dans le cas du parallèlisme. Les programmes ont une complexité accrue. Comme le code s'exécute simultanément sur plusieurs processeurs, il y a un entrelacement des instructions, et plusieurs scénarios d'exécution sont possibles, d'où un non-déterminisme du résultat. Il est plus facile pour le programmeur de faire des erreurs lors de la conception du programme, et il est également plus difficile de tester le programme fini dans tous les cas possibles.

Une idée naturelle est de ré-utiliser les méthodes utilisées pour la vérification de programmes séquentiels, et de les adapter aux programmes parallèles.

Preuve vérifiée par ordinateur de programmes

De nombreuses méthodes existent pour vérifier par ordinateur des programmes séquentiels.

• Les méthodes de test et de model-checking reposent sur une exploration des cas d'exécution possibles, dans le but de découvrir des bugs.

• L'analyse statique, l'interprétation abstraite, et l'exécution symbolique reposent sur une étude du code source du programme, pour inférer des propriétés vérifiées par le code. Contrairement au point précédent, les programmes ne sont ici pas réellement exécutés.

• L'utilisation des assistants de preuve, et la méthode B, permettent de générer des programmes qui seront corrects par construction.

Enfin, la logique de Hoare et la vérification déductive reposent sur des sémantiques axiomatiques permettant de vérifier formellement que des propriétés logiques seront toujours vraies à un point donné du programme. Formellement, on se base sur des triplés de la forme {P } e {Q}, où P et Q sont des prédicats logiques, et e un code à exécuter. Ce triplet a pour signification que dès lors que la prédicat P est vérifié, si l'ont exécute le programme e, alors le prédicat Q sera toujours vérifié après l'exécution.

Les sémantiques axiomatiques donnent des règles inductives permettant de construire des triplets corrects pour un programme complet. Plus adapté pour une preuve automatique, le wp-calculus (pour weakest precondition calculus) permet de trouver algorithmiquement le "meilleur" prédicat P , pour un programme e et un prédicat Q, afin que le triplet {P } e {Q} soit correct.

Parallèlisme

Le développement de méthodes formelles pour la preuve de programmes parallèle est rendue plus complexe par la diversité des types de parallélisme. Il est par exemple possible de classifier les ordinateurs parallèles par le modèle de Flynn, suivant que tous les composants de la machine parallèle exécutent le même code ou 1 non, et suivant qu'ils partagent les mêmes données ou non (ce qui donne les acronymes bien connus MIMD pour Multiple Instructions, Multiple Data, etc.). Dans cette thèse, on s'intéresse au modèle SPMD (Simgle Program, Multiple Data), ce qui correspond au calcul distribué, plus adapté à des quantités importantes de processeurs. La variété des architectures parallèles se traduit aussi par une variété des styles de programmation parallèle. On a ainsi l'opposition entre les programmes dans le style "mémoire partagée", et les programmes dans le style "passage de messages". Une librairie très utilisée pour ce dernier cas est mpi (Message Passing Interface).

Cependant, les super-calculateurs évoluent de plus en plus vers des architectures "hybrides", par exemple avec un cluster de multi-coeurs.

Le modèle bsp (Bulk Synchronous Parallelism est un modèle "pont" entre l'exécution abstraite et les machines parallèles concrètes. Il a l'avantage de permettre de modéliser un grand nombre d'architectures parallèles, que ce soit en mémoire partagée, en calcul distribué, ou les architectures hybrides.

Vérification de programmes parallèles (a) Méthodes existantes

Il existe des travaux pour la vérification de programmes parallèles dans le standard mpi. En particulier, il est possible de vérifier par model-checking qu'un programme mpi ne rencontrera pas de deadlock. Cependant, il n'est possible de faire cette vérification que pour un nombre fixé de processeurs. Il n'est pas non plus possible de vérifier la correction complète du programme.

(b) Notre approche

Dans cette thèse, nous avons choisi de vérifier des programmes dans le modèle bsp, afin d'avoir une approche la plus générale possible. Pour éviter les limitations des travaux cités ci-dessus, nous avons adopté la vérification déductive, qui permet a priori de prouver n'importe quelle propriété d'un programme, et ce quel que soit le nombre de processeurs sur lequel il sera exécuté.

Afin de profiter des méthodes et outils déja existant dans le monde de la programmation séquentielle, nous avons choisi de vérifier un programme bsp en le simulant par un programme séquentiel. Cela est rendu possible par le déterminisme du modèle bsp.

Nous nous sommes ainsi basé sur l'outil why, qui est un VCG (Verification Condition Generator), prenant en entrée un programme (séquentiel) annoté dans le langage spécifique de l'outil why-ML, et générant à l'aide d'un wp-calculus des obligations de preuve. Ces obligations de preuve sont des lemmes logiques qui doivent être démontrés afin de garantir la correction du programme annoté. why présente l'avantage de permettre l'interface avec un grand nombre de prouveurs, automatiques et interactifs, pour décharger ces obligations de preuve. Il existe également des outils permettant de prouver des programmes Java et C à l'aide de why. Ces avantages en font le candidat idéal comme outil sur lequel baser notre travail.

Plan

Le chapitre 2 de cette thèse présente un panorama des outils et langages disponibles pour la programmation bsp. Ceci est essentiel pour trouver le cadre commun que l'on utilisera dans notre outil de vérification de programmes bsp.

Le chapitre 3 présente l'outil bsp-why, qui est notre extension de l'outil why pour prouver des programmes parallèles dans le modèle bsp.

Dans le chapitre 4, nous donnons quelques exemples d'application de bsp-why, sur des algorithmes élémentaires, et un cas d'exécution plus complexe pour l'exploration d'espace d'états (la base du modelchecking).

Enfin, le chapitre 5 présente la preuve que notre approche est correcte. Nous définissons formellement les sémantiques pour notre langage, mathématiquement et dans l'assistant de preuve coq. Nous donnons ensuite la preuve que la transformation de bsp-why vers why est correcte par rapport à ces sémantiques. * : avec l'utilisation d'une autre librairie ** : optimisations dans certaines implémentations *** : serait facile à ajouter, mais n'est pas actuellement implémenté + : fonctionne en données partagées A l'issue de cette transformation, on obtient un programme why-ML, qu'il est ensuite possible de vérifier avec l'outil why, comme n'importe quel autre programme séquentiel.

Étude de cas

Nous avons appliqué bsp-why à la vérification de plusieurs algorithmes.

Algorithmes BSP basiques

• Notre premier exemple est le calcul des préfixes parallèles, qui est une opération très courante dans le domaine du calcul scientifique, permettant de résoudre des problèmes tel que le problème des N corps. Nous donnons deux variantes de cet algorithme, tout d'abord un code naïf, en une seule super-étape, puis une variante logarithmique, plus optimisée.

• Nous prouvons ensuite un algorithme de tri parallèle, le PSRS (Parallel Sorting by Regular Sampling). Cet algorithme est légerement plus complexe, et nécessite quatre super-étapes.

Après transformation par notre outil bsp-why, nous utilisons why pour vérifier le programme séquentiel généré. Nous présentons comme résultat le nombre d'obligations de preuves générées, et de celles déchargées par les prouveurs automatiques, dans le tableau suivant.

Programme

Correction/AP Sûreté/AP Préfixes (direct) [START_REF] Bonorden | The Paderborn University bsp (pub) library[END_REF] On peut noter que le premier exemple est vérifié entièrement automatiquement, mais ce n'est pas le cas des deux autres exemples.

Construction parallèle d'espaces d'états

Nous présentons un exemple plus complexe, celui de la construction de l'espace d'états pour le Model Checking.

• Nous avons commencé par définir l'algorithme séquentiel le plus basique de construction de l'espace d'états, dans le langage de why. Cet algorithme est facilement vérifié automatiquement par les prouveurs.

• Nous avons ensuite vérifié un algorithme parallèle qui repose sur une fonction de hashage pour associer un processeur à chaque état. L'idée est qu'un processeur calculera uniquement les successeurs des états qui lui appartiennent.

Cet algorithme est entièrement vérifié automatiquement, c'est à dire que toutes les obligations de preuve sont déchargées par les prouveurs automatiques.

Sémantiques formelles

L'outil bsp-why permet de vérifier des programmes bsp écrits dans le langage bsp-why-ML. Cependant, la confiance en cette vérification serait limitée sans étude formelle de la transformation effectuée par bsp-why. Dans ce chapitre, nous allons définir les outils nécessaires pour cette étude, et conclure par la preuve de la correction de la méthode.

Sémantiques à grand-pas, et à petit-pas

Nous avons tout d'abord défini différentes sémantiques pour le langage bsp-why-ML.

• Une sémantique à grands pas, la plus naturelle pour étudier un programme. Elle nous permet de définir le cadre formelle de l'exécution d'un programme bsp-why-ML.

• Une sémantique à petits pas, qui permet de modéliser plus finement l'exécution du programme parallèle. Cette sémantique sera utile pour les preuves de transformation de programmes.

• Des sémantiques co-inductives, à la fois en grands pas et petits pas, ont également été définies pour permettre l'étude des programmes qui tournent "à l'infini".

• Enfin, pour chacunes de ces sémantiques, nous avons aussi défini les modifications qui sont nécessaires pour traiter de la sous-synchronisation.

Sémantiques en Coq

Les sémantiques ont toutes été définies dans l'assistant de preuve Coq, ce qui nous donne une plus grande confiance dans les résultats prouvés. La définition des sémantiques suit le cadre habituel, avec des définitions sous forme de types inductifs. L'environnement d'exécution, plus complexe que pour un langage séquentiel, est défini sous la forme d'un Record contenant les informations sur tous les processeurs.

Nous avons ensuite pu énoncer et prouver dans Coq les différentes propriétés attendues sur nos sémantiques. Cela inclut les propriétés de déterminisme, de confluence, ainsi que l'équivalence entre les différentes sémantiques. Toutes ces propriétés sont donc prouvées formellement, et vérifiées par l'ordinateur.

Preuve de la transformation

Enfin, nous avons utilisé les sémantiques formelles pour donner la preuve de la correction de la transformation d'un programme bsp-why-ML vers un programme bsp-why. C'est en effet sur cette transformation que repose l'ensemble de la vérification des programmes bsp à l'aide de notre outil.

La preuve mathématique est effectuée à partir des sémantiques formelles définies précédement. Nous n'avons pas eu le temps au cours de cette thèse de réaliser la preuve entièrement dans l'assistant de preuve coq, cependant nous donnons des éléments de la preuve en Coq, afin de montrer que cette adaptation est possible.

The main topics of this thesis are formal proofs, using methods inspired by Hoare logic, and the mechanical proof checking of parallel programs.

In this introduction, we provide a general background on program verification, and different forms of parallelism. We will then introduce two technologies that form the basis of our work: Bulk Synchronous Parallelism (bsp) and the why tool for deductive verification of programs.

2.1

Context of the Work: Generalities and Background

Why Verify Parallel Programs?

Currently, computing is evolving from sequential programming towards the parallelisation of calculations.

New smartphones now include quad-core processors. Personal computers use increasing number of cores. Software now uses gpus to perform massively parallel computations. On another scale, supercomputers reach new records of performance, allowing massively parallel programs to be executed in various domains, such as meteorology, scientific calculations, etc. However, the common practices of sequential programming (code reuse and structuring) that evolved from Djisktra's "Goto statement considered harmful" paper [START_REF] Dijkstra | Letters to the Editor: Goto Statement Considered Harmful[END_REF] are not the norm for parallel code. Most parallel programming tasks are still performed using low level tools and languages, leading to poor quality code and high debugging and upkeep costs [START_REF] Gorlatch | Send-receive Considered Harmful: Myths and Realities of Message Passing[END_REF].

Because parallel code is the norm in many areas (an attempt to list all of them would certainly fail1 [START_REF] Hendrickson | Computational Science: Emerging Opportunities and Challenges[END_REF]), formal verification [START_REF] Hoare | The Verified Software Initiative: A Manifesto[END_REF][START_REF] Jones | Verified Software: A Grand Challenge[END_REF] of parallel programs is necessary. Indeed formal verification seems essential when considering the growing number of parallel architectures (gpus, multi-cores, etc. [START_REF] Balaji | mpi on Millions of Cores[END_REF]), the complexity of parallel architectures and the cost of conducting large-scale simulations (the losses due to faulty programs, unreliable results, unexpected crashing simulations, etc.). This is especially true when parallel programs are executed on architectures which are expensive and consume many resources.

Checking programs for correctness has always been a major issue. In critical domains, such as in aeronautics, medicine, or for military applications, a bug can be disastrous, causing the loss of billions of dollars, or even human lives 2 . Early programmers 3 were fully aware that ensuring the correctness of programs was an unavoidable issue. Given the strong heterogeneity of massively parallel architectures and their complexity, a frontal attack of the problem is a daunting task which is unlikely to materialise. Therefore, it seems more appropriate to find errors before the said programs are executed. This is called "a priori verification".

Why is Verification of Parallel Programs Hard?

First we consider the reasons why many researchers think that verification of parallel programs is hard:

• Added complexity; there is no magic method for designing a task decomposition and data distribution; if the wrong decomposition or distribution is chosen, poor performance results (due to too much communication between machines) and unexpected results can appear; for example, think of distributed graph algorithms [START_REF] Chan | Implementing and Testing CGM Graph Algorithms on PC Clusters and Shared Memory Machines[END_REF] that mainly make the hypothesis of a sorted distribution of edges. Compared to the verification of sequential programs, there are three main inherent difficulties in verifying parallel programs:

1. There is not just one flow of execution but many (as there are multiple units of computation);

2. The interleaving of instructions can introduce data-races 4 and unexpected results by the presence of non-determinism. A choice should be made between forbidding these cases (by raising a warning or an exception) or computing all possible traces of executions and results. [START_REF] Alt | Algorithm Design and Performance Prediction in a java-Based Grid System with Skeletons[END_REF]. In distributed computations the use of a network for reading/writing values in different memories and primitives of communications are introduced. This makes the reading and the semantics of programs harder: synchronous/buffered sending can generate a deadlock 5 and an unexpected result.

Tools for the verification of parallel codes exist but are not much used, mainly because they are usually inferior to their sequential counterparts. Most researchers on parallel computing are not accustomed to formal methods [START_REF] Gopalakrishnan | Formal Analysis of mpi-based Parallel Programs: Present and Future[END_REF].

• Parallel programming is error-prone even for "Gurus" of parallel computing, some subtle errors can be introduced and only appear after a large number of program executions; for example, a lack of verification of the call of mpi's collective operators mixed with asynchronous sending and multi-threading can crash the whole machine quickly [START_REF] Gorlatch | Send-receive Considered Harmful: Myths and Realities of Message Passing[END_REF]. Parallel libraries are generally well documented but even experienced programmers can misunderstand these apis, especially when they are informally described.

• Too few powerful abstractions are available in the languages used for parallel programming [START_REF] Lee | The Problem with Threads[END_REF]. Parallel programming languages are often very low level [START_REF] Gorlatch | Send-receive Considered Harmful: Myths and Realities of Message Passing[END_REF], as each communication or synchronisation operation has to be managed in detail by the programmer. Verification becomes much easier when the programmer can rely on powerful libraries to encapsulate complex behaviours -these generate less possible cases/traces of execution to prove. Work on algorithmic skeletons and high-level patterns is established but there is little knowledge of innovative parallel programming languages and tools for the verification of codes. Furthermore, it is difficult to know when you can compose parallel codes and if the results are as intended. For example, how to call a mpi code within an openmp [[ 1 ]] one?

• Parallel code testing is hard. Many libraries (e.g. mpi, openmp, etc.) allow you to test your parallel programs on your personal computer by emulating parallelism using OS processes; but not all interleavings can be tested in this way and some combinations that generate a deadlock may be missed; only formal verification gives this assurance. Testing is a first step and without test try, it can be hard to know "where to go" in a proof of correctness.

We conclude that High Performance Computing (hpc) programming is hard because every detail of parallelism is left to the programmer. Large fractions of code are technical and program complexity becomes excessive, resulting in a truly incessant debugging and proof. Because hpc is widely used in research in science and engineering (these themes are present in hpc conferences), for the long term viability of this area it is absolutely essential that we have verification tools to help application developers gain confidence in their software.

In the last decades, methods have been developed to study the correctness of sequential programs. For the verification of parallel programs it should be possible to avoid starting again. A natural question immediately follows: how easy is it to adapt the methods developed to ensure the safety of sequential programs for a parallel environment? In the following, we will summarize some of the methods that are frequently used to ensure that programs are error-free, focusing on machine-checked approaches. Then we will describe more precisely what we mean by parallel computing and which model we will use. Finally pre-existing tools for verification of parallel codes (algorithms and programs) will be presented.

Machine-checked Proof of Programs

Developing tools and methods to find logical bugs or to help programmers have "from the start" (or at a given time of the development) correct programs is an old but still pertinent area of research. For sequential (imperative, functional, object or real-time) algorithms and programs, many methods (and their associated tools) exist: programming through theorem proving, b method, model-checking, test, deductive verification, abstract interpretation, algebraic rules, etc.

A common and well accepted approach is conducting machine-checked proofs. These are formal derivations in some given formal system, assuming that the underlying logic is coherent. Humans, even great mathematicians, are fallible. The use of a formal system for doing proofs on a machine forbids false proofs. For example, special cases that seem a priori trivially true (or very similar to other cases and therefore unattractive) are often forgotten about but later revealed to be false. The use of a formal system gives thus greater confidence in the results. Parallel programming tends to be much more complex than sequential computing. This means that rigorous program development techniques are all the more necessary.

Common Methods for Having Machine-checked Correct Programs

There are different methods and tools for proving the correctness of programs and systems or for generating machine-checked programs. Without being exhaustive, we present here some of the best known methods.

(a) Testing, Debugging and Model-checking

This first class of methods we consider is generally not used for proving the correctness of programs but rather for finding bugs. Software testing [START_REF] Kaner | Testing Computer Software[END_REF] involves running a program for different scenarios (inputs of the program) and comparing the traces of execution against a formal specification of expected results. Scenarios can be automatically extracted by a testing tool from the program (for example, when there is a division to test the program against a division by zero) or given by hand by the programmer. Some languages such as Python have a specific syntax for describing the expected results of a function (which is also used to document the programs). Testing can not prove the correctness of a given program but it is very useful for finding most of the small careless mistakes of programmers.

Verification through model checking [START_REF] Clarke | Model Checking[END_REF] consists in: (1) defining a formal model of the system to be analysed; [START_REF] Aldinucci | MUSKEL: An Expandable Skeleton Environment[END_REF] expressing expected properties formally (generally in a temporal logic); (3) using automated tools to check whether the model fulfils the properties. This is done by calculating the state-space of the model representing all the different configurations of the execution of the program. The state-space construction problem is that of computing the explicit representation of a given model from the implicit one. In most cases, this space is constructed by exploring all the states reachable through a successor function from an initial one. Building a state-space is the first step in verification by model-checking of logical properties. Temporal logics are mainly used to compare all traces of execution against a given formal specification. Systems that are to be analysed are mostly concurrent communicated "agents", i.e. small programs. Model-checking can only find unexpected results for a given finite number of agents but not for any number of agents. However, model-checkers use different techniques to reduce the state-space (or the time to compute it) and to check the logical formula. That allows the checking of more agents or more complicated systems (since less memory and time are needed in the verification).

(b) Static Analysis, Abstract Interpretation and Symbolic Execution

This class of automatic methods is based on the analysis of a (full) source code, sometimes on the executable code (after compilation) and therefore performed without truly executing the code. These methods do not prove the "full" correctness of programs but rather prove that some properties hold during the entire execution of the programs. Those properties can be safe execution, absence of deadlock or (integer/buffer/...) overflow, termination of the computation, security issues, etc.

Static program analysers are mainly based on some kind of type systems. If the source code is valid for the type system then some simple properties hold during the execution of the program. Type systems give some (polymorphic) types to elementary objects and sub-expressions of the code. Then, some specific rules are used to compose these types. ml, haskell, java, ada (and many others) are common examples of typed languages. There are other kinds of specific analysers which extend the traditional type systems of programming languages.

Abstract interpretation is a generalisation of type systems, which uses Galois connections and monotonic functions over ordered sets [START_REF] Cousot | Abstract Interpretation: a Unified Lattice Model for Static Analysis of Programs by Construction or Approximation of Fixpoints[END_REF]. It can be viewed as a partial execution of a computer program which gains information about its semantics by running an approximation of the execution. Approximation is used to allow for vague answers to questions; this simplifies problems making them amenable to automatic solutions. One crucial requirement is to add sufficient vagueness so as to make problems manageable while still retaining enough precision for answering the important questions (such as "will the program crash?"). Abstract interpretation thus works on abstract domains of the data of the analysed program. When one chooses an abstract domain, one typically has to strike a balance between keeping fine-grained relationships, and high computational costs. Different kind of abstractions exist such as convex polyhedra, congruence relations, etc.

Symbolic execution is a special case of abstract interpretation and works by tracking symbolic rather than actual values of the code and is used to reason about all the inputs that take the same path through a program. Mainly, true values are substituted by symbolic variables and a constraint solver is used to test assertions in the program or to test if a given property holds during a symbolic execution. For branching constructs, all feasible paths run symbolically. But the number of feasible paths in a program grows exponentially with an increase in program size. Thus the method is inappropriate for large programs.

(c) Programming Through Theorem Proving and B Method

Machine-checked proofs (such as mechanized semantics of programming languages) are realised using theorem provers. A theorem is given to the system together with some tactics for solving the theorem using the logic rules of the system. Some theorem provers are fully automatic while others have special tactics to check automatically some simple goals. Several theorem provers (such as coq) are based on the Curry-Howard isomorphism which claims that "a proof is a program, the formula it proves is a type for the program". In this way, proving a property is as giving a (functional) program that computes the property. For example, the property "for any list of objects, there exist a list with the same objects but arranged in an ordered fashion" corresponds to giving a function for sorting lists. Programming through theorem proving [START_REF] Bertot | Interactive Theorem Proving and Program Development[END_REF] is thus proving that the formal specification (a property) is valid and then extracting the program from the proof. The resulting program would be automatically correct (assuming a correct extraction) since it is taken from the mechanized proof.

In contrast, top-down design of a program starts from an abstract formal specification. The specification is then refined into a more concrete form that incorporates design decisions reflecting a commitment to algorithmic and data representation. An executable code can be then extracted. The b method yields such a refinement technique of abstract machines. Correctness of abstract machines, mainly with logical invariants of the computations, are performed using a kind of Hoare logic (described below).

Hoare Logic and Deductive Verification of Programs

Hoare triples. In [START_REF] Hoare | An Axiomatic Basis for Computer Programming[END_REF], Hoare introduces the use of triples to describe the execution of a piece of code. A Hoare triple is written in the form {P } e {Q}, where P and Q are predicates, and e is a program expression (a command). P is called the precondition, and Q the postcondition. The meaning of a triple in Hoare logic is that when executed from an environment that satisfies the predicate P , the program e will result in an environment satisfying the predicate Q.

In [START_REF] Hoare | An Axiomatic Basis for Computer Programming[END_REF], Hoare gives a set of rules for reasoning about the instructions of a minimalist programming language. This allows valid Hoare triples to be built for a whole program incrementally. The rules are given in figure 2. 1. A few points need to be underlined:

• In the assignment rule, P [E/x] denotes the expression P where all the instances of x have been replaced with E. It is important to understand that this rule is only valid when the language does not contain aliases to the same value. valid according to the Hoare logic, but would be invalid in a program where x and y designate the same value. This limitation also holds for methods that are based on the Hoare logic, such as the why verification condition generator. We will discuss more about such consequences when we introduce why.

• The while rule introduces the notion of a loop invariant, P , that must remain true at the start and end of each iteration of the loop. We can state that P is true at the end of the loop, assuming that it was true initially. Hence the given Hoare triple for while.

• The last rule is used to strengthen a precondition, or weaken a postcondition.

Total correctness. The rules given by Hoare in [START_REF] Hoare | An Axiomatic Basis for Computer Programming[END_REF] have a limitation, discussed by the author in the original article. Even if Hoare triples are proved correct, there is still no guarantee that the program terminates. It is possible that a while statement will loop indefinitely. For this reason, a Hoare triple {P } e {Q} should be read as: provided that e successfully terminates, the result of its execution is as described by Q. Classical Hoare logic only proves partial correctness. However, it is possible to enrich the declaration of the loop rule, so that it includes the definition of a variant, a quantity that is decreasing according to a well-founded order. The new rule for a loop statement is written as follows:

wf (<) {P ∧ B ∧ v = v 0 } e {P ∧ v < v 0 } {P } while B do e done {P ∧ (¬B)}
Assuming that the relation < is well-founded, this means that in every iteration of the loop the quantity v, called the variant, decreases. A finite chain is formed, ensuring the termination of the loop.

Weakest Precondition. Hoare logic allows us to write triples that do not carry meaningful information. For instance, the triple {f alse} e {P } is always true, but very rarely relevant. Dijkstra's weakest precondition (wp) calculus [START_REF] Dijkstra | Guarded Commands, Nondeterminacy and Formal Derivation of Programs[END_REF] can be understood to give the most general precondition, for a program and its postcondition. As such, {wp(e, P )} e {P } is always true, and for all Q such as {Q} e {P } holds, we have Q → wp(e, P ). Most modern verification condition generators (vcg) are based on the Weakest Precondition calculus.

One of the main advantages of the weakest precondition calculus, compared to the basic Hoare logic, is that it is more easily adapted to the generation of mechanical proofs of programs. The wp-calculus can be computed by the formula given in figure 2. [START_REF] Aldinucci | MUSKEL: An Expandable Skeleton Environment[END_REF]. However, in Hoare logic, to prove a simple triple such as {P } e 1 ; e 2 {Q}, one has to devise a middle predicate that will allow the use of the induction rule. This can usually be done by a human; for a mechanical proof, the wp-calculus is more appropriate.

Parallel Computing

Different Forms of Parallelism (a) Flynn's Classification

A parallel computer or multi-processor system is a computer utilizing more than one processor (or unit of computation). It is common to classify parallel computers by distinguishing them by how processors access the system's main memory. Memory access heavily influences the usage and programming of a system. Flynn defines a classification of computer architectures, based upon the number of concurrent instructions (or controls) and data streams available in the architecture [START_REF] Duncan | A Survey of Parallel Computer Architectures[END_REF][START_REF] Flynn | Some Computer Organizations and their Effectiveness[END_REF]. Two major classes of distributed memory computers can be distinguished: distributed memory and shared memory systems.

Flynn's classification is as follow:

Single Instruction Multiple Instructions Single data SISD MISD Multiple data SIMD MIMD where:

• SISD is "Single Instruction, Single Data stream" that is a sequential machine;

• SIMD is "Single Instruction, Multiple Data streams" that is mostly array processors and gpu;

• MISD is "Multiple Instruction, Single Data stream" that is pipeline of data (pipe skeleton);

• MIMD is "Multiple Instruction, Multiple Data streams" that is clusters of cpus.

Distributed memory

No Remote Memory Access (norma) computers do not have any special hardware to support access to another node's local memory directly. The nodes are only connected through a computer network. Processors obtain data from remote memory by exchanging messages over this network between processes on the requesting and the supplying node. Computers in this class are sometimes also called Network Of Workstations (now). In case of shared memory systems, Remote Memory Access (rma) computers allow access to remote memory via specialized operations implemented by hardware. However the hardware does not provide a global address space.

The major advantage of distributed memory systems is their ability to scale to a very large number of nodes. In contrast, a shared memory architecture provides (in hardware) a global address space, i.e. all memory locations can be accessed via load and store operations. Such a system is much easier to program. Shared memory systems can only be scaled to moderate numbers of processors. We concentrate on the mind streams model, and especially on the so-called Single Program Multiple Data (spmd) model, which is widely used for programming parallel computers. In the spmd model, the same program runs on each processor but it computes on different parts of the data (distributed over the processors).

There are two main programming models: message passing and shared memory. These offer different features for implementing applications parallelized by domain decomposition. Shared memory allows multiple processes to read and write data from the same location. In the message passing model each process can send messages to the other processes.

(b) Shared Memory Model

In the shared memory model, a program starts as a single process (known as a master thread) which executes on a single processor. The programmer designates parallel regions in the program. When the master thread reaches a parallel region, a fork operation is executed. This creates a team of threads, which execute the parallel region on multiple processors. At the end of the parallel region, a join operation terminates the fork, leaving only the master thread to continue on a single processor. Now, we give some well known examples of libraries for the shared memory programming model.

openmp [START_REF] Basumallik | Programming Distributed Memory Sytems Using OpenMP[END_REF][START_REF] Chapman | Using OpenMP: Portable Shared Memory Parallel Programming[END_REF] is a directive-based programming interface for the shared memory programming model. It consists of a set of directives and runtime routines for Fortran, and a corresponding set of pragmas for c and c++ (1998). Directives are special comments that are interpreted by the compiler. Directives have the advantage that the code is still sequential and can be executed on sequential machines (by ignoring the directives/pragmas). There is no need to maintain separate sequential and parallel versions.

Intel Threading Building Blocks (Intel tbb [[ 2 ]] ) library [START_REF] Kukanov | The Foundations for Scalable Multi-core Software in Intel Threading Building Blocks[END_REF] which is a library in c++ that supports scalable parallel programming. The evaluation is specifically for pipeline applications which are implemented using the filter and pipeline class provided by the library. Various features of the library which help during pipeline application development are evaluated. Different applications are developed using the library and are evaluated in terms of their usability and expressibility [START_REF] Joseph | Evaluating Threading Building Blocks Pipelines[END_REF].

Recently Graphic Processing Units (gpu) have been used in hpc, due to their tremendous computing power, favorable cost effectiveness, and energy efficiency. The Compute Unified Device Architecture (cuda) [[ 3 ]] [START_REF] Nickolls | Scalable Parallel Programming with cuda[END_REF] has enabled graphics processors to be explicitly programmed as general-purpose sharedmemory multi-core processors with a high level of parallelism. In recent years, graphics processing units (gpus) have advanced from being specialized fixed-functions to being programmable and parallel computing devices. With the introduction of cuda, gpus are no longer exclusively programmed using graphics apis. In cuda, a gpu can be exposed to the programmer as a set of general-purpose sharedmemory simd multi-core processors. The number of threads that can be executed in parallel on such devices is currently in the order of hundreds and is expected to multiply soon. Many applications that are not yet able to achieve satisfactory performance on cpus can benefit from the massive parallelism provided by such devices.

Some take the view that models (generally concurrent languages) based on shared memory are easier to program because they provide an abstraction for a single, shared address space. While shared memory reduces the need for placement, it created a need to control simultaneous access to the same location. This requires either careful crafting of programs, or expensive lock management. Implementing sharedmemory abstractions requires a large fraction of a computer's resources to be devoted to communication and the maintenance of coherence. Worse, the technology required to provide the abstraction is not at all a commodity nature, and hence even more expensive [START_REF] Lee | The Problem with Threads[END_REF].

(c) Distributed Memory Model and the Message Passing Interface (MPI)

The message passing model is based on a set of processes with private data structures. Processes communicate by exchanging messages with special send and receive operations. It is widely used for programming distributed memory machines but it can be also used on shared memory computers.

The most popular message passing technology is the Message Passing Interface (mpi) [[ 4 ]] [START_REF] Snir | mpi the Complete Reference[END_REF], a message passing library for c and fortran. mpi is an industry standard and is implemented on a wide range of parallel computers. Details of the underlying network protocols and infrastructure are hidden from the programmer. This helps achieve portability while enabling programmers to focus on writing parallel code rather than networking code.

It includes routines for point-to-point communication, collective communication, one-sided communication, parallel io, and dynamic task creation. Later in this thesis more details about the sub-part of mpi (that interests us) will be given.

(d) Hybrid Architectures

In addition to shared-memory and distributed models, modern parallel architectures now provide hybrid models: we have now clusters of clusters of • • • of multi-processors of multi-cores with gpus. These architectures need a new way of programming.

Clusters of Multi-processors/cores.

Clusters have become the de-facto standard in parallel processing due to their high performance to price ratio. Symmetric MultiProcessing (smp) clusters are also gaining popularity, mainly under the assumption of fast interconnection networks and memory buses. smp clusters can be thought of as an hierarchical two-level parallel architecture, since they combine features of shared and distributed memory machines. As a consequence, there is interest in hybrid parallel programming models, e.g. models that perform communication both through message passing and memory access.

Intuitively, a paradigm that uses memory access for intra-node communication and message passing for internode communication seems to exploit better the characteristics of smp clusters [START_REF] Drosinos | Performance Comparison of Pure mpi vs Hybrid mpi-openmp Parallelization Models on smp Clusters[END_REF]. Hybrid models have already been applied to scientific applications [START_REF] Henty | Performance of Hybrid Message-Passing and Shared-Memory Parallelism for Discrete Element Modeling[END_REF], including probabilistic model-checking [START_REF] Hamidouche | Three High Performance Architectures in the Parallel APMC Boat[END_REF].

Skeleton Paradigm. Many parallel algorithms can be characterised and classified by their adherence to a small number of generic patterns of computation -farm, pipe, map, reduce, etc. Skeletal programming proposes that such patterns be abstracted and provided as a programmer's toolkit with specifications which transcend architectural variations but implementations which recognise them to enhance performance [START_REF] Cole | Bringing Skeletons out of the Closet: A Pragmatic Manifesto for Skeletal Parallel Programming[END_REF][START_REF] Falcou | Parallel Programming with Skeletons[END_REF][START_REF] González-Vélez | A Survey of Algorithmic Skeleton Frameworks: High-level Structured Parallel Programming Enablers[END_REF][START_REF] Gorlatch | Parallel Skeletons[END_REF]. The core principle of skeletal programming is conceptually straightforward. Its simplicity is its strength. There are two kinds of skeletons [START_REF] Kuchen | The Integration of Task and Data Parallel Skeletons[END_REF]: 1. Flow skeletons; These manipulate a "flow" of data by applying functions; All data flows (asynchronously) from machines to machines until a global condition occurs; For example, the ocamlp3l

[67] [[ 5 ]
] skeletons language (p3l [START_REF] Bacci | P3L: a Structured High-level Parallel Language, and its Structure Support[END_REF]'s set of skeletons for ocaml);

2. Data-parallel skeletons; They mainly manipulate distributed data-structures such as arrays, lists, trees, etc. by applying functions to the data.

Skeletons are patterns of parallel computations [START_REF] González-Vélez | A Survey of Algorithmic Skeleton Frameworks: High-level Structured Parallel Programming Enablers[END_REF][START_REF]Patterns and Skeletons for Parallel and Distributed Computing[END_REF]. They can be seen as high-order functions that provide parallelism. They fall into the category of functional extensions [START_REF] Kuchen | Features from Functional Programming for a C++ Skeleton Library[END_REF] -following their semantics [1]. Most skeleton libraries extend a language (mostly java, haskell, ml, c/c ++) to provide high level primitives.

Currently, the best known library is Google's mapreduce [START_REF] Dean | MapReduce: a Flexible Data Processing Tool[END_REF]. It is a framework to process embarrassingly parallel problems across huge datasets -originally for the page-ranking algorithm. Different implementations for java or c exist. But only two skeletons are provided which limits expressiveness.

Many authors have provided skeleton libraries for different language. For example, [START_REF] Falcou | QUAFF : Efficient C++ Design for Parallel Skeletons[END_REF] provides a set of flow skeletons for c ++. Templates are used to provide an efficient compilation of programs: for each program, a graph of communicating processes is generated which is then transformed into a classical mpi program. Templates have also been used in [START_REF] Hammond | Automatic Skeletons in Template Haskell[END_REF]. For java, many libraries exist such as [START_REF] Alt | Algorithm Design and Performance Prediction in a java-Based Grid System with Skeletons[END_REF][START_REF] Leyton | Skandium: Multi-core Programming with Algorithmic Skeletons[END_REF] and [START_REF] Aldinucci | MUSKEL: An Expandable Skeleton Environment[END_REF]. The latter has been extended for multi-core architectures [START_REF] Ciechanowicz | Enhancing Muesli's Data Parallel Skeletons for Multi-core Computer Architectures[END_REF]. A study of how to type java's skeletons is given in [START_REF] Caromel | Type Safe Algorithmic Skeletons[END_REF]. The authors note that some libraries of data-flow skeletons use a unique generic type for data (even if it is an integer or a string), which can cause a clash of the jvm. They explain how to avoid this problem, using a simple type system. There is also the work of [START_REF] Cole | Bringing Skeletons out of the Closet: A Pragmatic Manifesto for Skeletal Parallel Programming[END_REF] which describes how to add skeletons in mpi (the eskel library), as well as giving some experiments. It also gives convincing and pragmatic arguments to mix message passing libraries and skeleton programming. Some benchmarks of an ocaml implementation of data-flow skeletons for a numerical problem are described in [START_REF] Clément | Domain Decomposition and Skeleton Programming with OCamlP3l[END_REF].

Skeletons programs can be optimised to run on hybrid architectures (even if currently, this has not been really done).

The Bulk-Synchronous Parallelism (a) The Model

The Bulk-Synchronous Parallel (bsp) model is a bridging model [START_REF] Maggs | Models of Parallel Computation: A Survey and Synthesis[END_REF] between abstract execution and concrete parallel systems and was introduce by Valiant in [START_REF] Valiant | A Bridging Model for Parallel Computation[END_REF] and developed by McColl et al. [START_REF] Skillicorn | Questions and Answers about bsp[END_REF]. Its initial goal is to have a portable and scalable performance prediction for parallel programs. Without dealing with low-level details of parallel architectures, the programmer can focus on algorithm designcomplexity, correctness, etc. An introduction to its "philosophy" can be found in [START_REF] Skillicorn | Questions and Answers about bsp[END_REF] and a complete book of bsp numerical algorithms is [START_REF] Bisseling | Parallel Scientific Computation. A structured approach using bsp and mpi[END_REF]. A recent presentation on bsp can be found in [START_REF] Kielmann | Bandwidth-Latency Models (bsp, LogP)[END_REF].

A bsp computer has three components: (1) a homogeneous set of uniform processor-memory pairs; (2) a communication network allowing inter processor delivery of messages; (3) a global synchronisation unit which executes collective requests for a synchronisation barrier.

A wide range of actual architectures can be seen as bsp computers. For example shared memory machines could be used so that each processor only accesses a sub-part of the shared memory (which is then "private") and communications could be performed using a dedicated part of the shared memory. Moreover the synchronisation unit is very rarely a hardware entity but rather a software component [START_REF] Hill | Practical Barrier Synchronisation[END_REF]. Supercomputers, clusters of pc s [START_REF] Bisseling | Parallel Scientific Computation. A structured approach using bsp and mpi[END_REF], multi-cores [START_REF] Ghuloum | Ct: A Flexible Parallel Programming Model for Tera-scale Architectures[END_REF][START_REF] Yzelman | An Object-oriented Bulk Synchronous Parallel Library for Multicore Programming[END_REF] and gpus [START_REF] Hou | BSGP: Bulk-Synchronous gpu Programming[END_REF], etc. can be thus considered as bsp computers. There are different libraries and languages for bsp programming. We will describe them in the next chapter.

A bsp program is executed as a sequence of supersteps, each one divided into (at most) three successive disjoint phases (see left): (1) each processor only uses its local data to perform sequential computations and to request data transfers to/from other nodes; (2) the network delivers the requested data; (3) a global (collective) synchronisation barrier occurs, making the transferred data available for the next super-step.

The performance of the bsp machine is characterised by 4 parameters: (1) the local processing speed r; [START_REF] Aldinucci | MUSKEL: An Expandable Skeleton Environment[END_REF] the number of processor p; (3) the time L required for a barrier; (4) and the time g for collectively delivering a 1-relation, a communication phase where every processor receives/sends at most one word. The network can deliver an h-relation (every processor receives/sends at most h words) in time g×h. To accurately estimate the execution time of a bsp program these 4 parameters could be benchmarked [START_REF] Bisseling | Parallel Scientific Computation. A structured approach using bsp and mpi[END_REF]. The execution time (cost) of a super-step s is the sum of the maximal of the local processing, the data delivery and the global synchronisation times. It is expressed by the following formula:

Cost(s) = max 0≤i<p w s i + max 0≤i<p h s i × g + L
where w s i = local processing time on processor i during superstep s and h s i is the maximal number of words transmitted or received by processor i during superstep s. The total cost (execution time) of a bsp program is the sum of its super-steps's costs.

(b) Advantages and Disadvantages

It is stated in [START_REF] Dehne | Special Issue on Coarse-grained Parallel Algorithms[END_REF] that "A comparison of the proceedings of the eminent conference in the field, the acm Symposium on Parallel Algorithms and Architectures between the late eighties and the time from the mid-nineties to today reveals a startling change in research focus. Today, the majority of research in parallel algorithms is within the coarse-grained, bsp style, domain".

This structured model of parallelism enforces a strict separation of communication and computation: during a super-step, no communication between the processors is allowed; only at a synchronisation barrier can information be exchanged 6 . This execution policy has two main advantages. First, it removes non-determinism and guarantees the absence of deadlocks. This is the most visible aspect of a parallel model that shifts the responsibility for timing and synchronisation issues from the applications to the communications library 7 . Second, it allows for an accurate model of performance prediction based on the throughput and latency of the interconnection network, and on the speed of processors. This performance prediction model can even be used at runtime to dynamically make decisions, for instance choose whether to communicate in order to re-balance data, or to continue an unbalanced computation.

However, on many basic distributed architectures, barriers are often expensive when the number of processors dramatically increasese.g. more than 10 000. But proprietary architectures and future shared memory architecture developments (such as multi-cores and gpus) may make barriers much faster. Barriers have a number of advantages: it is harder to introduce livelock, since barriers do not create circular data dependencies. Barriers also permit novel forms of fault tolerance [START_REF] Skillicorn | Questions and Answers about bsp[END_REF].

The bsp model considers communication actions en masse. This is less flexible than asynchronous messages, but easier to debug, since there are many simultaneous communication actions in a parallel program, and their interactions are typically complex. Bulk sending offers better performance since, from an implementation point of view, grouping communication together in a separate program phase permits a global optimisation of the data exchange by the communications library. Moreover it is easy to measure during the execution of a bsp program, the time spent to communicate and to synchronise by just comparing the time before and after the primitive of synchronisation. This is mainly use to compare different algorithms.

Since bsp programs are portable and have a cost estimate which may model power consumption, they have the potential to be implemented in the cloud [START_REF] Armbrust | Above the Clouds: A Berkeley View of Cloud Computing[END_REF]: we can imagine a scheduler server that distributes a bsp program depending on its bsp cost so as to optimise power consumption and network traffic.

As with other low/high level design decisions, the applications programmer gains simplicity but gives up some flexibility and performance. In fact, the performance issue is not as simple as it seems: while a skilled programmer can in principle always produce more efficient code with a low-level tool (be it message passing or assembly language), it is not at all clear that a program, produced in a finite amount of time, can actually realise that theoretical advantage, especially when the program is to be used on a wide range of machines [START_REF] Gorlatch | Send-receive Considered Harmful: Myths and Realities of Message Passing[END_REF][START_REF] Lee | The Problem with Threads[END_REF].

One last advantage of bsp is that it greatly facilitates debugging. The computations going on during a super-step are completely independent and can thus be debugged independently. This facility will be used here to formally prove the correctness of our algorithms. This simplicity (for programming, debugging and proof) combined with its efficiency makes it a good framework for teaching.

The runtime system of bsp knows precisely which computations are independent. In a asynchronous message-passing system as mpi, the independent sections tend to be smaller, and identifying them is much harder. But, using bsp, programmers should be aware that some parallel patterns are not really bsp friendly, e.g. in pipeline and master/slave paradigm (also known as farm of processes). It is still possible to have reasonably efficient bsp programs for these schemes as we will see later. Some parallel computations and optimisations are unsuited for bsp [START_REF] Eisenbiegler | BSP, LogP, and oblivious programs[END_REF]. This is the disadvantage of all restricted models of computations.

Note that there are other (bridging) models of parallel computations. The best known is "logp" [START_REF] Bilardi | bsp versus LogP[END_REF][START_REF] Culler | LogP: Towards a Realistic Model of Parallel Ccomputation[END_REF] but there are also d-bsp, e-bsp, clumps, qsm, etc. A comprehensive list can be found in [START_REF] Cappello | An Algorithmic Model for Heterogeneous Hyper-clusters: Rationale and Experience[END_REF]. Except for logp, which has also been used to optimise some programs [START_REF] Chui | The LogP and MLogP Models for Parallel Image Processing with Multi-core Microprocessor[END_REF][START_REF] Martinez | Accurate Analytical Performance Model of Communications in mpi applications[END_REF] and mpi's collective operations implementations [START_REF] Pješivac-Grbović | Performance Analysis of mpi Collective Operations[END_REF], only bsp is widely used. The bsp model has also been used with success in a wide variety of problems such scientific computing [START_REF] Bisseling | Parallel Scientific Computation. A structured approach using bsp and mpi[END_REF][START_REF] Bisseling | Scientific Computing on Bulk Synchronous Parallel Architectures[END_REF][START_REF] Deo | Coarse-Grained Parallelization of Distance-Bound Smoothing for the Molecular Conformation Problem[END_REF][START_REF] Gerbessiotis | Topics in Parallel and Distributed Computation[END_REF][START_REF] Gerbessiotis | Direct Bulk-Synchronous Parallel Algorithms[END_REF][START_REF] Horvitz | Designing a bsp version of ScaLAPACK[END_REF][START_REF] Tiskin | The Design and Analysis of Bulk-Synchronous Parallel Algorithms[END_REF], parallel data-structure [START_REF] Gerbessiotis | Parallel priority queue and list contraction: The bsp approach[END_REF][START_REF] Guérin-Lassous | Portable List Ranking: an Experimental Study[END_REF], genetic algorithms [START_REF] Braud | A Parallel Genetic Algorithm based on the bsp Model[END_REF] and genetic programming [START_REF] Dracopoulos | Speeding up Genetic Programming: A Parallel bsp Implementation[END_REF], neural network [START_REF] Rogers | Using the bsp Cost Model to Optimise Parallel Neural Network Training[END_REF], parallel data-bases [START_REF] Bamha | Pipelining a Skew-Insensitive Parallel Join Algorithm[END_REF][START_REF] Bamha | Frequency-adaptive Join for Shared Nothing Machines[END_REF][START_REF] Bamha | An Efficient Equi-semi-join Algorithm for Distributed Architectures[END_REF], constraints solver [START_REF] Granvilliers | A System for the High-level Parallelization and Cooperation of Constraint Solvers[END_REF], graphs [START_REF] Chan | Implementing and Testing CGM Graph Algorithms on PC Clusters and Shared Memory Machines[END_REF][START_REF] Ferreira | Parallel Computation on Interval Graphs: Algorithms and Experiments[END_REF][START_REF] Lassous | Algorithmes Paralleles de Traitement de Graphes: une Approche Basee sur l'Analyse Experimentale[END_REF][START_REF] Tiskin | The Design and Analysis of Bulk-Synchronous Parallel Algorithms[END_REF], geometry [START_REF] Dehne | Scalable Parallel Computational Geometry for Coarse Grained Multicomputers[END_REF], string search [START_REF] Deo | Coarse-Grained Parallelization of Distance-Bound Smoothing for the Molecular Conformation Problem[END_REF][START_REF] Ferragina | String Search in Coarse-Grained Parallel Computers[END_REF][START_REF] Krusche | New Algorithms for Efficient Parallel String Comparison[END_REF], implementation of tree skeletons [START_REF] Matsuzaki | Efficient Implementation of Tree Accumulations on Distributed-Memroy Parallel Computers[END_REF], search engine (queries to textual databases) [START_REF] Costa | A Parallel Search Engine with bsp[END_REF], 3-sat solver [START_REF] Bonorden | Versatility of Bulk-Synchronous Parallel Computing: From the Heterogeneous Cluster to the System on Chip[END_REF], algebra [START_REF] Bonorden | Factoring a Binary Polynomial of Degree over one Million[END_REF][START_REF] Salem | Factorisation Algorithms for Univariate and Bivariate Polynomials over Finite Fields[END_REF][START_REF] Salem | Parallel Methods for Absolute Irreducibility Testing[END_REF], discrete event simulation [START_REF] Calinescu | Bulk Synchronous Parallel Algorithms for Optimistic Discrete Event Simulation[END_REF], bio-computing [START_REF] Huang | High-performance Parallel Bio-computing[END_REF][START_REF] Krusche | New Algorithms for Efficient Parallel String Comparison[END_REF], scheduling threads [START_REF] Da Rosa Righi | Observing the Impact of Multiple Metrics and Runtime Adaptations on bsp Process Rescheduling[END_REF], multiagent services [START_REF] Cheng | Hierarchical Resource Load Balancing Based on Multi-Agent in Service bsp Model[END_REF], image processing [START_REF] Da Rosa Righi | MigBSP: A Novel Migration Model for Bulk-Synchronous Parallel Processes Rescheduling[END_REF][START_REF] Hou | BSGP: Bulk-Synchronous gpu Programming[END_REF], etc. bsp was adopted because it represents a common model for writing successful parallel programs that exhibit phase-based computational behaviour [START_REF] Hendrickson | Computational Science: Emerging Opportunities and Challenges[END_REF].

2.4

Verification of Parallel Programs

Generalities

The correctness of parallel programs is of paramount importance, especially considering the growing number of parallel architecture (gpus, multi-cores, etc.) and the cost of conducting large-scale simulations -the losses due to faulty programs, unreliable results or crashing simulations is enormous. Also, avoiding deadlocks is insufficient to ensure that the programs will not crash. We need to check buffer and integer overflows (safety) and liveness. For critical systems and libraries, one needs to ensure that results are as intended. Formal verification tools that display parallel concepts are useful for program understanding and debugging. With multi-cores, gpus and peta-scale revolutions looming, such tools are long overdue.

Given the strong heterogeneity of these massively parallel architectures and their complexity, a frontal attack of the problem of verification of parallel programs is a daunting task that is unlikely to materialize. An approach would be to consider well-defined subsets that include interesting structural properties [START_REF] Cappello | On Communication Determinism in Parallel hpc Applications[END_REF]. In fact, many programs are not as unstructured as they appear: it is the skeletons and bsp main idea.

Advantages of Bridging Models for Correctness of Parallel Programs

One inherent difficulty in the development of scientific software is the reconciliation of the requirements that code be both correct and efficient. Today, the number of parallel computation models and languages may exceed the number of different architectures available. Most are inadequate because they make it hard to achieve portability, correctness and performance. Such systems are prone to deadlocks. Furthermore, the performance of such programs is typically difficult to predict because of the interaction of large number of individual data transfers. Placing too much emphasis on correctness may result in an abstract, but inefficient, programming model. Alternatively, striving for optimal efficiency can run the risk of comprising software correctness and can result in the employment of architecture-specific programming models. A common (but still insufficient) solution is the use of a bridging model.

In computer science, a bridging model is an abstract model of a computer which provides a conceptual bridge between the physical implementation of the machine and the abstraction available to a programmer of that machine; in other words, it is intended to provide a common level of understanding between hardware and software engineers. A bridging model provides software developers with an attractive escape route from the world of architecture-dependent parallel software. A successful bridging model is one which can be efficiently implemented in reality and efficiently targeted by programmers; in particular, it should be possible for a compiler to produce good code from a typical high-level language. The term was introduced in Leslie Valiant's 1990 paper "A Bridging Model for Parallel Computation" [START_REF] Valiant | A Bridging Model for Parallel Computation[END_REF], which argued that the strength of the von Neumann model was largely responsible for the success of computing. The paper goes on to develop the bsp model as an analogous model for parallel computing.

There exist other tentatives of bridging models such as Logp [START_REF] Bilardi | bsp versus LogP[END_REF][START_REF] Culler | LogP: Towards a Realistic Model of Parallel Ccomputation[END_REF] (and many variants such as cgm [START_REF] Chan | Implementing and Testing CGM Graph Algorithms on PC Clusters and Shared Memory Machines[END_REF], loggp, logpc [START_REF] Frank | LoPC: Modeling Contention in Parallel Algorithms[END_REF]), clumps, e-bsp, d-bsp, sqm, etc. None are as effective as bsp. For hybrid and hierarchical architectures, new bridging models such as the Hierarchical Hyper Clusters of Heterogeneous Processors (hihcohp) model [START_REF] Cappello | An Algorithmic Model for Heterogeneous Hyper-clusters: Rationale and Experience[END_REF] or the Multi-bsp model [START_REF] Valiant | A Bridging Model for Parallel Computation[END_REF] have been designed. But these models are too complex in comparison with bsp: compare algorithms is thus much harder (even if in a low-level parallel model, some trick optimisations are easier to find) and especially, for our purpose, proof of correctness would be harder because parallel routines could be much harder (more side-effects, unspecified behaviour etc.) to formalize.

The only sensible way to evaluate an architecture-independent model of parallel computation such as bsp is to consider it in terms of all of its properties, that is (a) its usefulness as a basis for the design and analysis of algorithms; (b) its applicability across the whole range of general-purpose architectures and its ability to provide efficient, scalable performance on them; (c) its support for the design of fully-portable programs; and (d) software engineering tools such as those for correctness or debug can be easily adapt to programs of this bridging model.

Take for example, a proof of correctness of a gpu-like program. Although interesting in itself it cannot be used directly for clusters of pcs. A bridging model has the advantage that if a program is correct, then this is the case for "all" physical architectures. Note that it is also the case for portable libraries such as mpi but algorithm design would be clearly architecture independent, which will be not the case using a bridging model. Moreover, it is known and accepted that correctness of programs is more costly in terms of work than just programming and designing algorithms. Hence the choice in this thesis of the bsp bridging model to provide both portability for proofs of correctness and a model for algorithmic design and efficient programs.

Our Approach: Deductive Verification of BSP Programs (a) The Traditional Solutions for the MPI Standard

Some works on the formal verification of programs based on standard mpi exist [START_REF] Gopalakrishnan | Formal Analysis of mpi-based Parallel Programs: Present and Future[END_REF]. We can find dynamic testing and runtime verification [START_REF] Gopalakrishnan | Runtime verification methods for mpi[END_REF][START_REF] Luecke | mpi-CHECK: a tool for checking Fortran 90 mpi programs[END_REF][START_REF] Vetter | Dynamic Software Testing of mpi Applications with Umpire[END_REF] 8 or debugging solutions [START_REF] Desouza | Automated, Scalable Debugging of mpi programs with Intel R Message Checker[END_REF]. The traditional solution for checking mpi codes is model-checking [START_REF] Grudenic | Modeling and Verification of mpi Based Distributed Software[END_REF][START_REF] Ohly | Automated mpi Correctness Checking: What if There Were a Magic Option? In High-Performance Clustered Computing[END_REF] which works most of time only for some specific properties [START_REF] Siegel | Verification of Halting Properties for mpi Programs Using Nonblocking Operations[END_REF].

Model-checkers of mpi programs [START_REF] Siegel | Verifying Parallel Programs with mpi-Spin[END_REF][START_REF] Vo | Formal Verification of Practical mpi Programs[END_REF] work on the c/mpi source code by using an abstraction of the mpi calls (the schemes of communications). An engineer, by push-button, can mainly verify that the program does not contain any possible deadlocks. A mechanised semantics of c/mpi primitives has been done in [START_REF] Li | Formal Specification of the mpi-2.0 Standard in TLA+[END_REF] (using tla + [[ 6 ]], the Temporal Logic of Actions) which can be used for the model-checking of mpi programs -and dynamically testing logical assertions. Model-checking of c/mpi programs has also been done in [START_REF] Krammer | MARMOT: An mpi Analysis and Checking Tool[END_REF]

[[ 7 ]] [250] [[ 8 ]] [276, 283] [[ 9 ]] [[ 10 ]].
The drawback of these methods is that checking is limited to a predefined number of processors: it is impossible to verify that the program is deadlock free for any number of processors (a scaling problem). Increasing the number of processes increases the verification time (at most exponentially) even if partial order reductions are used to reduce the state space. Most of these works are aimed at standard concurrency properties, rather than the functional correctness of the computations carried out by an mpi program. Those tools do no check/prove statically the correctness of the results -only assertion violations. They focus on important properties as deadlocks, resource leaks or data-races. This greatly increases the confidence that we have in the programs but it is not enough if we want to guarantee correctness.

Proving a program for any number of processors brings an additional difficulty: many properties are proved by induction (over p the number of processors). This kind of proof is problematic for automatic provers. We have sought another solution for verifying bsp programs and for the "bsp like" like subpart of mpi which is collective routines. Our proposed solution is deductive verification of bsp programs.

(b) Proposed Solution: Deductive Verification of BSP Programs

Avoiding deadlocks (or data-races) is not sufficient to ensure that programs will not crash. We need to check buffer and integer overflows or liveness. And one can also want a better trust in the code: are results as intended? It is recognised that "correctness" is not only the safe execution of a program but also a formal characterisation of the intended results and formal properties that hold during the execution. Using a Verification Condition Generator (vcg) tool is the proposed solution. A vcg takes an annotated program as input and produces verification conditions (proof obligations) as output to ensure correctness of the properties given in the annotations. An advantage of this approach is that manual proof of properties using proof assistants can be mixed with automatised checks of simple properties using automatic decision procedures.

As stated by J.-C. Filliâtre: Deductive program verification is the art of turning the correctness of a program into a mathematical statement and then proving it.

Deductive verification emphasises the use of a logic, to specify and to prove. This discipline is interested in the verification of small and challenging programs rather than huge systems. Deductive verification can be used to verify algorithms as well as programs, if the tool has been designed to verify codes that are executable -in our current work we only provide a tool for algorithm verification. The main concept (known nowadays as Hoare triple [START_REF] Hoare | An Axiomatic Basis for Computer Programming[END_REF]) binds together a precondition P , a program statement s, and a postcondition Q. This triple is usually noted {P }s{Q}. The basic method is the insertion of logical annotations in the programs (called annotated programs) e.g. an invariant of a loop 9 .

From this annotated program (or algorithm), a vcg would produces verification conditions for provers -proof assistants or automatic provers. Dijkstra's weakest preconditions calculus (wp calculus) is used.

The key advantages of using a vcg are:

• It allows the verification of simple properties (such as "no overflow") of a program without formally proving its entire correctness;

• Some tools are able to automatically insert annotations for some simple properties10 ;

• Using automatic provers enables the quick detection of simple errors;

• The manual proof of properties (using proof assistants) can be mixed with automatised checks of simple properties using automatic decision procedures

We provide a tool for the verification of properties of a special class of parallel programs by providing annotations and generation of proof obligations using a vcg. We choose bsp programs for three main reasons: (1) mpi programs with collective operators can be seen as bsp programs; (2) it is intrinsically a deterministic model; (3) as in coq proofs of bsml programs [START_REF] Bousdira | A verified library of algorithmic skeletons on evenly distributed arrays[END_REF][START_REF] Gava | Formal Proofs of Functional bsp Programs[END_REF][START_REF] Gava | Une Bibliothèque Certifiée de Programmes Fonctionnels bsp[END_REF][START_REF] Gesbert | Systematic Development of Correct Bulk Synchronous Parallel Programs[END_REF][START_REF] Tesson | A Verified Bulk Synchronous Parallel ml Heat Diffusion Simulation[END_REF], the structured nature of bsp programs (sequences of super-steps with clear separation between communications and computations) allows bsp programs to be executed in a sequential manner. This latter property will be used by our tool to generate sequential programs from bsp ones and by using a traditional vcg as a back-end to generate goals -logical conditions. Also, the structured nature of the bsp programs allows programs to be decomposed into sequences of blocks of code, each block mainly corresponding to a super-step.

Writing a proof assistant or a vcg is a huge task which should be left to the experts. The main idea of our work is to simulate bsp parallelism by transforming the parallel code into a sequential form. Therefore, my goal is to use a "well defined" verification tool for sequential programs as a back-end for our own verification tool of parallel programs. Furthermore, implementing a vcg for a realistic programming language needs a lot of work: too many constructs require specific treatment. Reducing the vcg to a core language is a good approach. Another advantage of generating a sequential program with assertions is that we would be able to use any kind of dedicated tools for code analysis that work on annotated programs; thus avoiding the need to recreate these tools for the bsp model.

(c) The VCG Why and its programming language

We choose the vcg why [START_REF] Bobot | Why3: Shepherd Your Herd of Provers[END_REF][START_REF] Filliâtre | Verification of Non-Functional Programs using Interpretations in Type Theory[END_REF][START_REF] Filliâtre | Verifying Two Lines of C with Why3: an Exercise in Program Verification[END_REF][START_REF] Filliâtre | The why/Krakatoa/Caduceus Platform for Deductive Program Verification[END_REF] [ [ 11 ]]. First, it takes as input a small language (why-ml, close to ml) avoiding the need to handle all the constructs of a real language 11 . Instead, realistic programming languages can be compiled into the why input language: why currently interprets c programs, java and ada programs with the help of companion tools [START_REF] Filliâtre | Multi-Prover Verification of C Programs[END_REF][START_REF] Filliâtre | The why/Krakatoa/Caduceus Platform for Deductive Program Verification[END_REF][START_REF] Guitton | why Hi-Lite ada?[END_REF] -Krakatoa for java and hit-lit for ada. Second, why is currently interfaced with the main proof assistants (coq

[[ 12 ]], pvs [[ 13 ]], isabelle [[ 14 ]], hol [[ 15 ]]) and automatic decision procedures (simplify [[ 16 ]], alt-ergo [[ 17 ]], z3 [[ 18 ]], cvc3 [[ 19 ]], yices [[ 20 ]], vampire [[ 21 ]]
) as back-ends for proofs obligations. These provers can be used for the proof obligations obtained from the parallel programs.

One might ask, why not work directly on parallel c programs then? After all, the main bsp library are written for the c language, and the programs that an end-user will want to prove are likely to be written in c, too. There are two aspects to answering this question. First, from a practical point of view, it is much easier to transform a parallel program in a syntax close to why-ml to a sequential why-ml program. The transformation (simulation) itself becomes almost a direct rewriting in many parts, and there is the added advantage that we were able to re-use the why parser, slightly modified to handle the parallel specificities, for our tool (bsp-why). The other reason is that, much as why-ml is intended to be an intermediate language, bsp-why-ml is ultimately aiming at being an intermediate in a larger chain. The framework frama-c [[ 22 ]] allows the transformation of c programs into why-ml programs, with the use of the jessie plug-in. Other languages allow a transformation in why too, such as java with the use of krakatoa. bsp-why could be used in the same way, as an intermediate between true parallel programs in c or other programming languages, and the why platform.

Informally, the input syntax of the vcg why is an intermediate and specific alias-free ml language dedicated to program verification. As a programming language, it is a ml language which (1) has limited side effects (only mutable variables that cannot be aliased), [START_REF] Aldinucci | MUSKEL: An Expandable Skeleton Environment[END_REF] provides no built-in data type, (3) proposes basic control statements (assignment, if, while, exceptions [START_REF] Filliâtre | The why/Krakatoa/Caduceus Platform for Deductive Program Verification[END_REF]) (4) uses program labels to refer to the values of variables at specific program points. The full syntax and semantics of why can be found in [START_REF] Filliâtre | Verification of Non-Functional Programs using Interpretations in Type Theory[END_REF].

A why program is a set of functions, annotated with pre-and post-conditions. Those are written in a general purpose specification language (polymorphic multi-sorted first-order logic). This logic can be used to introduce abstract data types, by declaring new sorts, function symbols, predicates and axioms. The verification condition generation is based on a Weakest Precondition calculus completed by a functional interpretation of the imperative features [START_REF] Filliâtre | Verification of Non-Functional Programs using Interpretations in Type Theory[END_REF], incorporating exceptional post-conditions and computation of effects over mutable variables. The why language also provides the possibility of defining axioms, pure logical assertions and parameters: primitives that have type definitions (with possibles side-effects and logical assertions) but no implementation.

In the case of alias-free programs, simpler proof obligations is the main goal of why: for parallel codes, our own tool can generate obligations that are "hard" to read; the simplest the obligations are for the sequential parts of a program, the less complex they are for the parallel parts.

In Fig 2 .3, we give a simple but representative example of why-ml code -from the why distribution. It allows us to give an informal presentation of why. The example is traditional binary search for a value in a sorted array -searching for v in array t. First, we include the package of arrays. The type 'a array is a built-in shortcut for 'a farray ref, where 'a farray is the following abstract type for purely applicative arrays. In fact, only the statement ← (":=") and parameters can modify in place data. For example in "array.why", we find:

( * t[e]
is syntactic sugar for (array_get t e) * ) parameter array_get: a:'a array → i:int → {0≤ i<array_length(a)} 'a reads a {result=access(a,i)} ( * t [e1] ←e2 is syntactic sugar for (array_set t e1 e2) * ) parameter array_set : a:'a array → i:int → v:'a → {0≤ i<array_length(a)} unit writes a {a=update(a@,i,v)} that is accesses to an element in an array at index i is only possible if i is in the bound of the array. A parameter gives a computation and can have thus side-effects (writes in an argument or raise an exception). A predicate (a syntactic sugar) or a logic give only logical properties and appear in logical annotations. A logic can also appear in the computations.

Then we have the value to search for and a function to compute the middle of a part of an array (with an axiom about this computation). Moreover, we give a predicate which gives the meaning of the presence of v in a part of the array. Finally, we take the array for the search and auxiliary variables. The search has the pre-condition that the array is sorted. The loop has an invariant which says that v is potentially present within the two index limits l (left) and u (up). Note the use of an assert to force another property (for the rest of the obligations). At the end, we have the final post-condition which says that at index p of the array, we have found the search value; otherwise the index is 0 and v is not present in the array (in this example only, the values in the arrays are indexed from 1).

Outline

In Chapter 3, we present different libraries for imperative bsp programming (mainly for c and java): the primitives and their informal semantics as well as their advantages and disadvantages. That allows us to show their differences and what they have in common, to extract the essential for our tool of deductive verification. Finally we present some choices for our core language, called bsp-why-ml, which tries to generalise the various imperative bsp libraries.

In Chapter 4, we show how bsp-why-ml can be used to specify a parallel program. We will then explain how our bsp-why tool works, that is how the transformation of a bsp-why-ml program into a sequential one is achieved. We also show how our model can be extended to prove the correction of programs with subgroup synchronisation.

In Chapter 5, we give several applications of bsp-why. We prove several classic parallel algorithms, such as prefix computation and parallel sorting. Finally, we give a mechanised proof of a more complex example, the parallel generation of the state-space in model-checking.

In Chapter 6, we will show the correctness of our approach. This is done by defining formally, mathematically and then mechanically in the coq proof assistant the formal operational semantics of bspwhy-ml. We then give the proof that the transformation from bsp-why to why is correct with respect to the given semantics. Since the design of the bsp model of execution there are different libraries and languages for bsp programming. Those libraries mainly exist as extensions of common sequential programming languages (c/c++, ocaml, python, java, etc.) and some were designed for specific architectures. In this chapter, we study those extensions and compare them. That will be used later to show what we can process using our bsp-why-ml language (presented in the next chapter) and how it differs from the previous libraries.

Different Kinds of BSP Operations

Generalities

Historically, the bsp library was the bsplib [START_REF] Hill | bsplib: The bsp Programming Library[END_REF] ] by adding subset synchronisations, high-performance operations and migration of threads (only using tcp/ip). The pub library has also been adapted to the java language and grid computing in [START_REF] Bonorden | A Web Computing Environment for Parallel Algorithms in java[END_REF]. In [START_REF] Diamos | Ocelot: A Dynamic Optimization Framework for Bulk-Synchronous Applications in Heterogeneous Systems[END_REF], the authors have extended the bsplib to execute programs in heterogeneous systems.

For the java language, different libraries exist. To our knowledge, the first one is [START_REF] Gu | JBSP: A bsp programming library in java[END_REF]. There is also bsponmulticore [START_REF] Yzelman | An Object-oriented Bulk Synchronous Parallel Library for Multicore Programming[END_REF] (which also works for c [START_REF] Yzelman | MulticoreBSP for C: A High-performance Library for Shared-memory Parallel Programming[END_REF]) [[ 27 multi-core technologies. There is also the JMigBSP library [START_REF] Graebin | jMigBSP: Object Migration and Asynchronous One-Sided Communication for bsp Applications[END_REF] which also provides code migrationusing the facilities of java to serialise objects. More recently, this library, with scheduling and migration on grid environments of bsp threads, has been re-implemented in [START_REF] Da Rosa Righi | MigBSP: A Novel Migration Model for Bulk-Synchronous Parallel Processes Rescheduling[END_REF] -the scheduling is implicit but the migration can be explicit. But the most known library is hama [245] [[ 28 ]]. It allows bsp's communications and the implementation is provided by a "mapreduce like" framework of the foundation apache. A comparison with the Google's mapreduce language [START_REF] Dean | MapReduce: a Flexible Data Processing Tool[END_REF] is described in [START_REF] Pace | bsp vs MapReduce[END_REF]. The author notes that some graph algorithms could not be efficiently implemented using mapreduce. Notice that another Google's language call pregel [START_REF] Malewicz | Pregel: A System for Large-scale Graph Processing[END_REF] [[ 29 ]] (without any public implementation) is also used by this company to perform bsp computations on graphs. We can also highlight the work of neststep [START_REF] Keßler | NestStep: Nested Parallelism and Virtual Shared Memory for the bsp Model[END_REF] [ [ 30 ]] which is c/java library for bsp computing, which authorizes nested computations in case of a cluster of multi-cores but without any safety.

For the gpu architectures, a first bsp library was provided in [START_REF] Hou | BSGP: Bulk-Synchronous gpu Programming[END_REF] [[ 31 ]]. It is mainly the primitives to remote access memory of the bsplib/pub. The bsml primitives (bsp for ml) [START_REF] Gesbert | Bulk Synchronous Parallel ml with Exceptions[END_REF] were adapted for c++ in bsp++ [START_REF] Hamidouche | A Framework for an Automatic Hybrid mpi +openmp Code Generation[END_REF] [ [ 32 ]]: this library provides nested computation in the case of a cluster of multi-cores (mpi +open-mp). The bsml language also inspired bsp python [START_REF] Hinsen | Using bsp and python to Simplify Parallel Programming[END_REF] [ [ 33 ]]. bsml also inspired bsphaskell [START_REF] Miller | bsp in a Lazy Functional Context[END_REF]. The library [START_REF] Ghuloum | Ct: A Flexible Parallel Programming Model for Tera-scale Architectures[END_REF] provides collective operators in a bsp fashion for multi-core architectures.

mpi programs that only use collective operations are close to bsp ones [START_REF] Bisseling | Parallel Scientific Computation. A structured approach using bsp and mpi[END_REF]. [START_REF] Cappello | On Communication Determinism in Parallel hpc Applications[END_REF] argues that most mpi programs can be thought as a sequence of collective patterns: most asynchronous communications are used to simulate a collective pattern that is not present in mpi. [START_REF] Martino | Parallel Program Analysis and Restructuring by Detection of Point-to-point Interaction Patterns and their Transformation into Collective Communication Constructs[END_REF] proposes an automatic tool (not yet fully implemented) to transform asynchronous communications into some calls of collective operations.

Description of these Routines (a) Sending Messages and One-sided Communication

Bulk Sending (BSMP). The first, simple and most frequent way for bsp programming is bulk sending. It is mainly a routine of the form "send(v,i)" to send a value to remote processor i. This differs from mpi send/received because the sending is non-blocking and the value is only available for processor i at the next super-step. It is the underlying system that decides when to truly send the value on the networkor a copy of the value in case of a shared memory architecture. The reception is performed using a routine of the form v=received(i,n) to read the nth value received from processor i during the past super-step. However some libraries do not give any order for the reception of messages but allow to "tag" messages to distinct them. The bsp libraries also give access to the number of received messages and/or get the tag of a message.

bsmp is more convenient then drma for computations where the volumes of data being communicated in super-steps are irregular and data dependent, and where the computation to be performed in each super-step depends on the quantity and form of sent data received at the start of this super-step.

Remote Data and Communication. Another way of communication is through drma (which stands for Direct Remote Memory Access): after every processor has registered a variable for direct access, all processors can read or write the value on other processors. This is usually called one-sided communication, because only one process needs to issue a send or receive call to achieve the communication. This scheme can simplify programming in cases where the memory locations that must be updated or interrogated are known on only one side of a communicating pair of processes. drma allows processes to specify shared memories and distant read/write in these memories.

The "put(x,v,i)" routine allow a processor to write the value v in the memory (variable v) of another processor i. It thus assumes that the source processor knows the memory location on the destination processor where the data must be put. The source processor is the initiator of the action, whereas the destination processor is passive. Thus, we assume implicitly that each processor allows all others to put data into its memory. To enable a processor to write into a remote variable, there must be a way to link the local name to the correct remote address. Linking is done by the registration primitive "push reg(x)". A variable is deregistered by a call to "pop reg(x)".

Note that for some case, using a "put" can be simpler than using a matching "send"/"receive" pair, as it is done in mpi-like algorithms: the program text of such an algorithm must contain additional ifstatements to distinguish between sends and receives; careful checking is needed to make sure that pairs match in all possible executions of the program and even if every send has a matching receive, this does not guarantee correct communication as intended by the designer of the algorithm and a deadlock can occur. Such a problem cannot happen when using synchronous puts.

Sometimes, it may be necessary to let the destination processor initiate the communication. This may happen in irregular computations, where the destination processor knows that it needs data, but the source processor is unaware of this need. In that case, the destination processor must fetch the data from the source processor. This is done by a statement of the form "get". Note that bsp put, get and sends operations do not block progress within their super-step: after a put or get is initiated, the program proceeds immediately. Most bsp libraries do not guarantee to exploit potential overlap of communication and computations. Instead, delaying all communication gives more scope for optimization, since this allows the system to combine different messages from the same source to the same destination and to reorder the messages with the aim of balancing the communication traffic.

(b) High-performance Routines and Oblivious Synchronisation

In some bsp libraries, message passing and drma routines have their high-performance counterpart. Those routines are less safe but can be more efficient. They are unbuffered and nonblocking that is when a processor executes hp-send(v,i) to send a value v to processor i, then it is unspecified when the value v will be sent. In this way, if the program changes the value of v, it is unknown which value will be received by processor i. In case of a send(v,i), it is a copy of v that is sent and thus the program can then modify v for its own purpose without modifying the meaning of the sending. Note that unlike some "mpi send routines", all "bsp's send routines" are nonblocking in the sense that the sender does not wait until the receiver truly receives the value; the good reception is guaranteed by the bsp barrier, i.e. the end of the super-step. Note also that some "mpi send routines" are also asynchronous as well as reception: the famous isend and irecv where mainly a "wait" for a request is performed to know when the message is truly received. It is thus possible to program bsp algorithms using mpi but it is more difficult and error prone since one has to manage all the waiting of data.

The oblivious synchronization is the high-performance pending of the traditional bsp synchronisation. It should be used if the programmer knows the number of messages each processor will receive in a superstep. Thus, in the oblivious synchronization each processor waits until "nmsgs" are received. This type of synchronization is cheap and much faster than the other one because no additional communication is needed to determine that processors can move on to the next super-step.

To our knowledge, The pub library has been the first to propose this kind of synchronisation.

(c) Collective Operations

A collective operation is a routine in which data are simultaneously sent to or received from many nodes.

Common examples of collective operations (those of the standard mpi) are:

• Broadcast, where the same data is sent to all nodes;

• Gather, where data is collected from all nodes;

• Scatter, where a set of data is broken up into pieces, and each piece is sent to a different node;

• Allgather, where every node send a data to other nodes;

• Alltoall, where every node send a different value (even empty) to each node;

• Global reduces, where an associative operation is performed over each data of each node. The advantage of this kind of operations is that their implementation can be optimised for some specific architectures since they provide a clear semantics. They also simplify the reading and understanding of the programs. Collective operations of bsp libraries generally work on the whole machine and are synchronous. mpi ones work on a communicator that is a subgroup of processors that have been registered to be within the communicator -using a specific routine. Another collective operation is the sendrcv in which two processors send (synchronously) each other a value.

Collective operations can be used to simulated the traditional bsp message passing and drma routines. This is the case of the bsponmpi library and bsml where all the routines accumulate values in some proper structures and finally, to finish the super-step, an mpi's alltoall is performed.

(d) Subgroup Synchronisation

The bsp model is based on a global synchronisation. However, in some cases, a parallel algorithm may include problems that can be solved using only a subset of processors. Some libraries extend the basic bsp model, and allow the definition of subgroups, which are pairwise disjoint subsets of the set of processors. It is then possible to write a part of the parallel program with the subgroup acting as an independent bsp computer. A call to the bsp_sync function will then synchronise over the subgroup, instead of the whole parallel computer. We then speak of partitioned synchronisation.

In the left, we show an example of execution of subgroup synchronisation. In this example, the overall group of processors S is split into two subgroups (S1 and S2) which run independent bsp computations. Finally the two subgroups are merged and the whole machine continue its work doing global barriers.

The idea of subgroup synchronisation is to allow the synchronisation to work on a subset of processors. This means that the communication procedures need to be able to tell in which group they are working. This is especially important for the synchronisation procedure. An additional argument is thus added to all the parallel procedures, representing a group of processors linked together. It is called a communicator. The mpi standard also allows to create sub-communicators: in this way, collective operations are performed only on a subset of the processors, those which participate to the communicator.

(e) Thread Migration

Some bsp libraries (mainly for java) offer the possibility to have more threads of computations than true nodes/processors available in the machine. They provide object rescheduling by using two different solutions: (1) migration directives on the application code directly or (2) through automatic load balancing at middleware level. One can speak of migratable virtual processors. It was designed to work on grid/cloud computing environments since thread migration and rescheduling allow a thread remapping in response to application and infrastructure behaviour. This technique is thus useful to migrate entities for executing faster on lightly-loaded resources and/or approximating those ones that communicate frequently. The execution time of such a parallel program can be significantly improved because it is possible to migrate its processes at run-time to other hosts with currently offer more available computation power.

As for garbage collectors of high-level languages, the second solution provides both transparent and effortless mechanism of migration in the user's point of view. The first solution allows the user to find an appropriate rescheduling but with the drawback or missing a good mapping or rescheduling too often, which can induce too much communication and thus bad performances: the explicit rescheduling requires a developer with expertise in load balancing algorithms; in this way, the programmer may be required to collect data about processors' capacity and to load for decision making on process relocation manually.

(f) Hybrid and Hierarchical Architectures

Coupling smp clusters (clusters of multi-cores) combine the packaging efficiencies of shared-memory multiprocessors with the scaling advantages of distributed-memory architectures. The result is a computer architecture that can scale more cost-effectively in size. Unfortunately, these systems come at the price of a more complex programming environment to deal with the two different modes of parallel execution. These architectures are called hybrid due to the these two different modes and are also called hierarchical since the computational units are mainly organised as a tree. While tools exist for shared-memory systems and for distributed-memory systems, solving problems on parallel computers with smp nodes is not as simple as combining two tools.

But bsp is a flat parallel model and the view of a parallel machine as a set of communicating sequential machines remains true but is more than incomplete. This is why some libraries such as NestStep or pub allow nested or subgroup synchronisation. Moreover, there are bsp libraries for different kinds of architectures such as multi-cores or gpus. For example, bsp++ allows nested bsp computations and uses mpi over the network of the cluster and openmp to share memory multi-cores; it uses cuda for gpus. We will discuss this issue in the last chapter because currently, our work is based on a rather flat bsp model even if we can manage subgroup synchronisation.

3.2

Some Details About Some BSP Libraries

The Message Passing Interface (MPI)

The Message Passing Interface (mpi) [START_REF] Snir | mpi the Complete Reference[END_REF] is a standard api for communication in distributed-memory parallel applications. There are numerous implementations of this api, both commercial and open source (e.g. open-mpi, mpich, etc.). mpi's goals are high performance, scalability, and portability. mpi is de facto the standard for high-performance computing today. And mpi is the main library used for implementing other parallel languages such as skeletons ones, bsp, etc. The mpi standard defines the informal semantics of routines useful to a wide range of users writing portable distributed programs in fortran and c. However, there are many stubs codes for languages such as java, ocaml, etc. mpi contains a huge number of routines, thus it is not possible to present all of them. We thus only present the routines that are close to bsp such as collective operations and operations for the communicators manipulation 1 . We also present only mpi-2 routines as, mpi-3 was released in September 2012 and is not yet fully implemented by any mpi library. Fig 3.3 gives the c types of these routines. The first are for initialize (or terminate) the mpi computation and know the number of processes or their "id" for a group of processor. There is an initial communicator MPI_COMM_WORLD which contains all the processes participating to the computation.

A collective communication is defined as communication that involves a group of processes, i.e. a communicator. All processes in the group identified by the communicator must call the collective routine. We do not give explanation of all the arguments but resume the most important ones for our purpose. In many cases, collective communication can occur "in place" for communicators, with the output buffer being identical to the input buffer. This is specified by providing a special argument value, MPI_IN_PLACE, instead of the send buffer or the receive buffer argument, depending on the performed operation. The collective operations are for broadcasting, scattering, gathering and all-to-all exchanges. There is mainly the buffer to send (resp. to receive the data) with the number of sending data and their types -MPI_INT, MPI_DOUBLE, etc. to obtain portable programs. As many c routines, they return an integer that indicate if the operation has been well realized. The int MPI_Alltoallw routines is the most general form of all-to-all. For example, by making all processes have sendcounts[i] = 0, this achieves an MPI_scatterw. This is a powerful routine but hard to use. There is also a routine a bit different: sendrecv. This routine involves only two processors which exchange data in a synchronous way as a barrier. In fact, this is a sub-group of only two processors. Also, reducing operations involve an associative operator (MPI_op) on the data -MPI_Datatype. They enable to apply these operator on all the data in the distributed arrays.

Collective operations are not really bsp communications since they are not synchronous. For example, using a broadcast, if the emitter changes the buffer to send after the call of the routine, it is unspecified which value will be received by other processors. A call to a MPI_barrier() is needed to forbid this case. In practise, most mpi libraries implement collective operations in a synchronous way but this is not standard.

There are many routines for managing communicators -we count 35 in mpi-2. However, the main routine is MPI_comm_split which creates a new communicator by partitioning the group into disjoint subgroups using a set of colors one for each value of color. The colors are define in the api. Each subgroup contains all processes of the same color. Within each subgroup, the processes are ranked in the order defined by the value of the argument key, with ties broken according to their rank in the old group. This is a collective call, but each process is permitted to provide different values for color and key. This is an extremely powerful mechanism for dividing a single communicating group of processes into k subgroups and many routines, such as MPI_comm_create, are equivalent to a specific call to MPI_comm_split. Unlike the pub (see thereafter) function, this mpi routine requires communications between the processors of the group. In fact, exchanging the colors can be seen as a call to the MPI_Allgather collective operation. Communicators in mpi are thus both more flexible and complex. Unlike in pub, there is almost no restriction to the way groups are formed. Processes can be in more than one group, they have an unique rank within each group. A traditional example of use of such communicators would be in linear algebra, where it is often useful to split processors by rows and by columns.

mpi also provides routines for asynchronous drma accesses, the windows. A programmer can create or free a window. It is a synchronous operation that every processors perform. After, the processors can write or read on remote processors. There is also a kind of barrier, the fence, for processors that have been registered to the window. And using the Win_complete, we can force a processor for waiting that all remote readings and writings have been performed on a window after a Win_start.

Furthermore, mpi obviously provides asynchronous send/received of data. These routines are outside the scope of this thesis.

BSPLIB Like Libraries for C

The bsplib [START_REF] Hill | bsplib: The bsp Programming Library[END_REF] is a c-library of communication routines to support the development of parallel algorithms based on the bsp model. There is also the pub [START_REF] Bonorden | The Paderborn University bsp (pub) library[END_REF] which provides additional features such as oblivious synchronization (where each processor waits until n messages are received), subgroup synchronisation (where only a part of the processors synchronise) and thread migration -not described in this document because too complex and too architecture dependant. Both libraries offer functions for both message passing (bsmp) and remote memory access (drma). Both are implemented using tcp/ip or mpi. Some collective communication operations like broadcast are also provided in the pub, which can easily be simulated by bsmp operations. Fig 3.4 gives the routines of the standard bsplib. Fig 3.6 gives the routines of pub. Because they are the most used libraries for bsp programming, we now details these routines and their differences. Note that bsponmpi is only a new implementation of the mpi implementation of bsplib.

(a) The Routines of the BSPLIB [START_REF] Hill | bsplib: The bsp Programming Library[END_REF] Tool Routine. As in standard mpi, we first need to initialise our parallel computation, using the function bsp_init, and then we can have different bsp computations, each beginning with bsp_begin and terminating with bsp_end. Now, within a bsp computation, we can query some information about the machine: bsp_nprocs returns the number of processors p and bsp_pid returns the processor "id" which is in the range 0, . . . , p -1.

Synchronisation.

According to the bsp model, all messages are received during the synchronisation barrier and cannot be read before. The barrier is done using bsp_sync which blocks the node until all other nodes have called bsp_sync and all messages sent to it in the current super-step have been received.

Message Passing.

In bsmp, a non-blocking send operation is provided that delivers (a packet of) messages to a system buffer associated with the destination process. The message is guaranteed to be in the destination buffer at the beginning of the subsequent super-step, and can be accessed by the destination process only during that super-step. If the message is not accessed during that super-step it is removed from the buffer. Using the bsplib, the destination buffer of a processor may therefore be viewed as a queue, where the incoming messages are enqueued in arbitrary order.

Sending a packet (in a buffering mode) is done using bsp_send and is based on the idea of two-part messages, a fixed-length part carrying tagging information that will help the receiver to interpret the message, and a variable-length part containing the main data payload. The length of the tag is required to be fixed during any particular super-step, but can vary between super-steps. Choosing a tag is done using bsp_set_tagsize where the user specifies the size of the fixed-length portion of every message in the subsequent super-steps. Allowing the user to set the tag size enables the use of tags that are appropriate for the communication requirements of each super-step. This should be particularly useful in the development of subroutines either in user programs or in libraries. The procedure must be called collectively by all processes. A change in tag size takes effect in the following super-step; it then becomes valid.

The programmer can know the number of received messages as well as the total size of received data (in bytes) using the routine bsp_qsize. This routine work on the queue of received messages. To receive a message, the user should use the procedures bsp_get_tag and bsp_move. The operation bsp_get_tag returns the tag of the first message in the queue and the size of the payload (status is -1 if the queue is empty). The operation bsp_move copies the payload of the first message in the system queue, i.e. the buffer call payload, and removes that message from the queue. Then, the system will advance to the next message.

DRMA Routines. Registering a variable or deleting it from global access is done using: void bsp_push_reg (ident,size) and bsp_popregister(ident). Due to the spmd structure of bsp programs, if p instances share the same name, they will not, in general, have the same physical address. To allow bsp programs to execute correctly, the bsplib provides a mechanism for relating these various addresses by creating associations called registrations. A registration is created when each process calls void bsp_push_reg and, respectively, provides the address and the extent of a local area of memory: registration takes effect at the next barrier synchronisation and newer registrations replace older ones. This scheme does not impose a strict nesting of push-pop pairs. For example: On processor 0:

On processor 1:

void x[5]
,y [START_REF] Armand | Extending coq with Imperative Features and Its Application to SAT Verification[END_REF]; bsp_push_reg(x,5) bsp_push_reg(y,5) void x [START_REF] Armand | Extending coq with Imperative Features and Its Application to SAT Verification[END_REF],y [START_REF] Armand | Extending coq with Imperative Features and Its Application to SAT Verification[END_REF]; bsp_push_reg(y,5) bsp_push_reg(x,5) The variable x on processor 0 would be associated with y on processor id 1. Note that this example is clearly not a good way of programming in bsp. In the same manner, a registration association is destroyed when each process calls bsp_popregister and provides the address of its local area participating in that registration. A run-time error will be raised if these addresses (i.e. one address per process) do All get and put operations are executed during the synchronisation step and all get are served before a put overwrites a value. For example:

SOME DETAILS ABOUT SOME BSP LIBRARIES

int array [START_REF] Ferreira | Parallel Computation on Interval Graphs: Algorithms and Experiments[END_REF], privateArray [START_REF] Ferreira | Parallel Computation on Interval Graphs: Algorithms and Experiments[END_REF]; src = 0; dest = 0; offset = 0; bsp_push_reg (&bsp, array, sizeof(array)); bsp_get (&bsp, src, array, offset, privateArray, sizeof(array)); bsp_put (&bsp, dest, &privateArray [1], array, bsp_pid(bsp) * sizeof(int), sizeof(int)); bsp_sync (&bsp); bsp_pop_reg (&bsp, array);

In this example every processor gets a copy of array on processor 0 in his own privateArray (bsp_get).

After that each processor writes privateArray [1] into array[pid] on processor 0. You could also exchange the put and the get and you would get the same result since "gets" are served before "puts".

(b) The Routines of the PUB [37]

The pub's routines are close to the ones of the bsplib. The true differences are the features for subgroup synchronisation and collective operations -not described here because trivially simulated by bsmp routines. During initialisation, the added parameter can be BSPLIB_STDPARAMS or a pointer to a t_bsplib_params that is initialized with bsplib_params_init. Is is as the MPI_COMM_WORLD of mpi. Now, let us describe the differences. DRMA Routines. The drma routines are the same except that they work for a subgroup. Note that in the pub library, if different variables have to be registered/unregistered, all processors must call the functions in the same order. The previous solution is simpler, thus we have currently considered only it.

But at synchronisation, it is easy to compare the local lists of registered variables to test if variables are correctly registered or not.

Message Passing. Sending a single message can be done using bsp_send or bsp_sendmsg once created.

The arguments are obvious. After calling one of these routines the buffer (or the message) may be overwritten or freed. Which sent operation should be used in which situation? If there is not much communication in your program or super-step then the routine bsp_send is preferred. If there is a lot of data in not continuous memory for the same destination processor then it is better to use the routine bsp_sendmsg. In the next super-step, each processor can access the received messages of type t_bspmsg.

This can be done using bsp_findmsg where proc_id is the "id" of the source-node and index is the index of the message. To access to the message, we need bspmsg_data which returns a pointer to the sending block of data and bspmsg_size which returns its size. Also bsp_nmsgs() returns the number of messages and buffers received in the last super-step. Note that the messages of the last super-step are available until the next synchronisation call. At this point the memory used for these messages will be deallocated.

As for drma operations, these routines work in the scope of a subgroup.

Subgroup Primitives. The function involved in the process are bsp_dup, bsp_partition and bsp_done. The bsp_dup function creates a subgroup that is a duplicate of the current bsp computer. This is useful to organize algorithms in a compositional manner, since a code working on the duplicate will not be affected by previous messages that were already in the queue, waiting to be sent. The synchronisation on the duplicate subgroup will only complete the communications that were requested with that subgroup. It is thus possible to define a function for a sub-algorithm, starting with a call to bsp_dup so that it can be called from anywhere in a parallel program, without interfering. The bsp_partition function is the way to create proper subgroups. The subgroups are described as a partition of the current bsp computer in contiguous subsets. The starting indexes of these subsets are given in argument in the partition integer array. To be a correct partition, the array has to be sorted. Finally, the bsp_done function indicates that we have finished the work with the subgroup. In the pub library, it is impossible to synchronize or create other subgroups from the parent object until the current subgroup is released with the routine bsp_done. It is however possible to create different new subgroups of the current subgroup. The organisation of subgroups is similar to a stack, with only the lowest sub-group being allowed to create new subgroups. Here is an example of a program, in the pub library, with subgroup synchronisation: Communicators are thus present both in mpi and in the pub library as group which are called bsp objects. Although they play a similar role, there are notable differences between both. The pub library allows to partition the set of processors into pairwise disjoint subsets. Each subset acts like an independent bsp computer. Subgroups are created by a call to the bsp_partition function. A call to the function creates a new t_bsp object, that can be used in any bsp call. It is also still possible to use the parent bsp object. A call to bsp_partition does not require communications between the processors, since each processor has to explicitly give the whole partition table. However, there are limitations. It is not possible to create other subgroups of the parent bsp object, until the current subgroup sub has been disposed of using the bsp_done function. The subgroups work like a stack: it is only possible to create subgroups of the lowest subgroup. Another limitation comes from the way the subgroups are defined. The partition has to be made of subgroups of consecutive processors. It is not possible to combine the processors 1 and 3 in a group, and 2 and 4 in another one, for a parallel machine of four computers.

t_bsp subbsp; int part[2]; part[0] = 2; part[1] =

BSP Programming over Multi-cores and GPUs (a) The Ct Library [130]

The Intel ct library is now part of an Intel framework for parallel computing called tbb [START_REF] Kukanov | The Foundations for Scalable Multi-core Software in Intel Threading Building Blocks[END_REF]. tbb contains parallel libraries and compilers, tools for debugging, etc. ct is a deterministic nested dataparallel c ++ library intended to leverage the best features of gpus while fully exploiting multi-cores flexibility. The main goal for Intel is power efficiency because "power consumption is the ultimate limiter to improving computational performance in silicon technology" [START_REF] Ghuloum | Ct: A Flexible Parallel Programming Model for Tera-scale Architectures[END_REF].

ct introduces a new (template-style) polymorphic type, called a TVEC. TVECs are write-once vectors that reside in a vector space segregated from native c ++ types, e.g. TVEC<F64> stands for a vector of doubles. For manipulating these vectors, there are three ct operator categories:

1. Element-wise operators that support simple unary, binary and n-ary operators, such as addition, multiplication, etc. For example, TVEC<F64> product = multiply(nonzeros,expv) or as operator overloading TVEC<F64> product = nonzeros * expv;

2. Collective communication operations, such as reduction, prefix-sum or combining-send. For example, TVEC<F64> innerproduct = addReduce(product,RowIdx);

3. Permutation operations which allow both structured and unstructured reordering and replication of data. For example, TVEC<F64> expv = distribute(v,ColP).

ct operators are logically free of side effects, from the programmer's perspective. That way, each ct operator logically returns a new TVEC. ct's support for nested vectors is a generalization that allows a greater degree of flexibility. Vectors may be flat or regular multi-dimensional vectors. They also may be nested vectors of varying length, which allows for very expressive coding of irregular algorithms. Fig 3.7 gives an overview of the ct primitives. All the primitives are synchronous, even for collective communications. Thus, ct allows some kind of nested bsp computations. ct is implemented using owner Intel low-level libraries.

(b) BSPonMultiCore for C [287]

This library is a full implementation of the routines of bsplib but optimised for multi-cores architectures.

The bsplib has a posix PThreading implementation, thus it works well for multi-core architectures. But there were no optimisations using shared caches and bus interfaces (from cores to the main memory) or current multi-core architectures. However, in the bsplib, these optimisations has been done for other (older) specific architectures such as cray or sgi origin massive computers. (c) BSPGP: BSP programming over GPU Architectures [START_REF] Hou | BSGP: Bulk-Synchronous gpu Programming[END_REF] bsgp is a bsp programming language for gpu. bsgp extends the c language with few primitives for spawning threads and performing barriers. The statements between two barriers are automatically deduced as a super-step and translated into a gpu kernel by the bsgp compiler. In a bsgp program, data dependencies are defined implicitly because local variables are visible and shared across super-steps.

As for gpu, the main goal of bsgp is stream processing and especially stream of pixels for image analysis. However The bsgp programming model does not directly match the gpu's stream processing architecture, and the bsgp compiler converts bsgp programs into efficient stream programs: (1) the compiler automatically adds context saving code making the barriers conform to the semantics and (2) due to the presence of shared variables across the super-steps (data dependencies between super-steps), the compiler uses a graph optimization to automatically allocate temporary streams to save local variable values and thus, for having a efficient codes, it minimizes the total number of temporary streams. The bsgp compiler generates c+cuda codes. A current limitation is the inability to handle flow control across barriers such as while or if statements.

bsgp allows to: (1) thread creation, destruction (with operations fork and kill) and load balancing/reassignment of thread across the kernel units (the small units of computations) of a gpu; (2) remote variable access intrinsic for efficient communications between threads and (3) collective primitive operations (reduce, scan and sort). Fig 3.8 resumes the statements and primitives of bsgp. Here some details:

• A spawn statement executes a block of gpu code using the total number of threads as a parameter.

• barrier(RANK REASSIGNED) allows rank reassigning barriers. Since physical thread ranks cannot be changed within a kernel, a logical rank reassignment is performed by shuffling stored thread contexts at a barrier.

• In stream processing, some operations such as resource allocation and detailed kernel launch configuration are only available to the cpu (control processor); To address this issue, bsgp allow the control processor code to be inserted into bsgp source code as a require statement block. At run time, code inserted this way is executed before the containing super-step is launched.

• In case of independent super-steps, bsgp provides the par construct to let the programmer control the bundling behaviour by restructuring the code. The par construct specifies that all statements in the block are independent of each other; During compilation, the bsgp compiler aligns barriers in the statements and bundles the corresponding super-steps together. It is close to the superposition of bsml

• Create/destroy threads can improve load balancing as the application data is amplified/reduced. It is the main goal of fork and kill primitives.

• Using Thread.split, the rank is reassigned such that a thread with a false side has a smaller rank than a thread with a true side. Relative rank order is preserved among threads of the same side;

• In the same manner, thread.sort allows to reassigned thread on kernels depending on a specific order given by the array key

BSP Programming in Java

There are some libraries for bsp programming using java. The most known and promising is hama whereas the older, to our knowledge is jbsp. There are also libraries for multi-core architectures (bsponmulticore) and for peer-to-peer/grid environments (pub-web and JMigBSP).

(a) Hama and JBSP JBSP [START_REF] Gu | JBSP: A bsp programming library in java[END_REF]. jbsp is mainly a bsplib for java. It provides programmers with both explicit message passing and remote memory access routines. Due to the use of the java's object serialization, the author notes an overhead in communication compared to bsplib. The implementation relies to rmi and tcp/ip socket routines of the standard library of java. A class that must perform bsp computations only need to extend jbspTask and to overload the method void run(). Fig 3 .9 resumes the routines. The library provides drma routines which are in fact bsmp routines but with a dedicated tag. We thus do not present them. bsmp is performed by sending a tagged object which can be serialized. Reception is performed by removing objects from the queue of received messages. As in the bsplib, there is no order for messages in the queue. Thus we can only have the tag of the message at the top of the queue. ] is an open-source framework for distributed computing which is mainly a clone of the Google's MapReduce framework. hama is thus a framework for cloud computing well adapted to search-engines and graph manipulations for data-intensive scientific applications such as machine learning, information retrieval, bioinformatics, and social network analysis. The hama's run-time works using a master/peer paradigm where the master performs the barrier by sending messages to slaves and managing the input/ouput stream of data.

The way to create your own bsp computation is to create a class that extends the BSP class: BSPPeer <K1, V1, K2, V2, M extends Writable> peer) throws ... where K1 (resp. V1) represent an input stream of Then, in the main class, we need to initialise the distributed computation as an Hadoop program and which machine will be the master. Optionally, there are also setup() and cleanup() which will be called at the beginning of your computation, respectively at the end of the computation. For this, we just have to override these methods. Fig. 3.10 gives the main methods of the BSP class without the possible thrown exceptions. They are like the bsmp ones of the bsplib. There is also methods to write/read data in the streams.

As one of the goals of hama is the handling of large graphs, it also offers a special api for manipulating graphs à la pregel. We will present pregel thereafter.

(b) BSPonMultiCore [START_REF] Yzelman | An Object-oriented Bulk Synchronous Parallel Library for Multicore Programming[END_REF] bsponmulticore is a java library for bsp computing over multi-core architectures. It has been first developed for testing if bsp is applicable for shared-memory multi-core systems and can attain proper speedups. bsponmulticore communication library is thus an object-oriented adaptation of bsplib, written in java, and targeting only shared-memory systems. But stills a spmd library. The implementation relies on the capacity of multi-threading of java.

A generic bsp program is defined to be a class, BSP_PROGRAM having at least the functions defined in Fig 3.11. Any specific bsp algorithm is extended from this superclass, and must implement the two following virtual methods: (1) main_part, only executed by a single process to prepare data and can only call bsp_begin(n) for starting the parallel computation with n threads; [START_REF] Aldinucci | MUSKEL: An Expandable Skeleton Environment[END_REF] parallel_part which defines the code run in a spmd fashion; once this method is reached, all bsp functions can be called, with the exception of bsp_begin.

bsponmulticore allows sending or sharing an object by the use of an abstract class call BSP_COMM. Any sending or sharing objects will have to extend this class. Since an object has no meaning if not connected to a parallel program, the constructor must take a BSP_PROGRAM as parameter, thus linking the object and the parallel program it is used in. The methods are standard bsplib primitives with the use of a queue of received messages at the beginning of each super-step. Note the methods bsp_unregister frees up all memory it uses at all threads; this should never be called inside a super-step where the object is still used. As in the bsplib, there is also asynchronous (but unsafe) high-performance versions of the methods for benefit of the shared-memory of multi-cores architectures. However, the uses must use this routines carefully.

Each thread can also define a shared array. Such an array could be defined using a BSP_REGISTER<T> and standard java array constructs. bsponmulticore provides some subclasses of BSP_COMM dedicated for manipulating arrays of data. The drma methods have been modified to copy a number of length elements of the array in a single communication request (which is thus closer to the bsplib routines). The getData method, giving the programmer access to the underlying array. The main implementation of BSP_COMM_ARR is BSP_ARRAY<T> which makes available an array with elements of type T at each process. The type T must still support cloning. For efficiency (especially for numerical applications), when T is an int or a double primitive type, bsponmulticore has defined the specialised BSP_INT_ARRAY and BSP_DOUBLE_ARRAY classes.

(c) PUB-web [36]

pub-web (PUBWCL) [[ 35 ]] is a bsp java library and run-time environment that allows to execute tightly coupled, bsp algorithms on PCs distributed over the internet whose owners are willing to donate their unused computation power. pub-web is realized as a peer-to-peer system and features migration and restoration of bsp processes executed on it. With the pub-web client, any user can donate its unused computation power and also run its own parallel programs. As the unused computation power of the participating pcs in the web is unpredictable, pub-web uses a dynamic strategy for load balancing the computations across the pcs. bsp-web implementation uses the java-go rmi library for communicated objects and processes in a secure fashion. java-go extends java with these features: (1) migrations are performed using the keyword go and (2) all methods, inside which a migration may take place, have to be declared migratory.

Using pub-web, in order to write a bsp and migratable program, the interface BSPMigratableProgram has to be implemented which means that the main method of the main class of the program has this signature: migratory void bspMain(BSPMigratable bspLib, String[] args) throws AbortedException, NotifyGone. Fig 3 .12 gives the bsp routines. They are traditional bsmp send/received routines. In migratable programs, there is also a method available which may be called to mark additional points inside long super-steps where a migration is safe, i.e. no open files etc..

One single process receiving little computation power can slow down the execution of the whole bsp program due to the barrier synchronization. Migrating these "slow" bsp processes can therefore significantly improve the execution time of the bsp program. In [START_REF] Bonorden | A Web Computing Environment for Parallel Algorithms in java[END_REF] the authors present and benchmark different heuristics for load-balancing/migrate processes.

(d) JMigBSP [75, 139]

jMigBSP is also a java programming library à la bsplib but which offers object (threads) rescheduling. It is implemented using the ProActive framework [[ 36 ]]. JMigBSP was designed to work on grid computing environments in two different ways: (1) by using migration directives on the application code directly and (2) through automatic load balancing at middleware level. A distributed object must extend the jMigBSP class. In this way, the user can override the migration for its own purpose. The initial one allows migrating the caller object to a remote host. JMigBSP proposes automatic load balancing of objects at each barrier with a potential of migration function. For this, the user must override the bsp_sync with this code: The number of messages in the queue

Message getMessage(int index)

Get the nth received message

Message[] getAllMessages()

Getting all received messages

Message findMessage(int src, int index)

Get the nth message sended by a process

Message[] findAllMessages(int src)

Getting all received messages sended by a process Serializable Message.getContent()

Getting

Other Libraries (a) NestStep [175]

NestStep is a set of extensions to existing imperative programming languages like java or c for programming bsp algorithms. It adds language constructs and run-time support for the explicit control of parallel program execution and the sharing of program objects: by default, shared scalar variables and objects are replicated across the processors; within a super-step only the local copies of shared variables are modified and the changes are committed to all remote copies at the end of the super-step. Also, NestStep introduces static and dynamic nesting of super-steps.

NestStep mainly provides the step{...} statement to denotes a super-step that is executed by entire groups of processors. Thus, a step statement always expects all processors of the current group to arrive at this program point -which is in the charge of the programmer. It implies a barrier at the beginning and the end of statement for performing communications (the combine phase) even if the implementation tries to avoid duplicate barriers. A step statement with parameters deactivates and splits the current group into disjoint subgroups that execute the step body independently of each other -in a nested manner. The parent group is reactivated and resumes when all subgroups have finished execution of the step body. For example, step<k, #>=1>{...} splits the current group into k subgroups where at least there one processor in each subgroup. For each group the run-time system holds on each processor its own class, group object. In particular, it contains the group size, the group "id", and the processor's rank within the group.

By default, objects (or basic variables) are private. Thus, they exist on every processor, executing their declarations. Sharing is explicitly specified by a type qualifier sh type x at declaration or for objects allocation with one of the following parameters:

• sh<0> declares a variable x of arbitrary type where the copy of the group leader (the processor with rank 0) is broadcast at the combine phase. All other local copies are ignored even if they have been written to.

• sh<?> denotes that an arbitrary updated copy is chosen and broadcast. If only one processor updates the variable in a step, this is deterministic.

• sh<=>, the programmer asserts that he always assigned the same value to x on all processors of the declaring group; thus, combining is not necessary for x.

There is also a declaration for distributed array (block or cyclic distribution) in order to save space and to exploit locality where each access to a non-local element is automatically resolved by a blocking point-topoint communication with its owner, in order to guarantee a sequential consistency. The implementation uses tcp/ip or mpi. (2) the property is ensured that for any bsml program, the sequential simulation (a top-level) gives the same results than the truly parallel run. bsml programs can mostly be read as ocaml ones, in particular, the execution order should not seem unexpected to a programmer used to ocaml, even though the program is parallel. That allows the parallelisation to be done incrementally from a sequential program. A parallel vector has type 'a par and embeds p values of any type 'a at each of the p different processors in the parallel machine.

The nesting of parallel vectors is not allowed and a type system prevents this forbidden use of vectors.

We use the following notation to describe a parallel vector: x 0 , x 1 , . . . , x p-1 where p is the (constant) number of processors throughout the execution of the program. It can be accessed in bsml using the integer constant bsp_p -the other bsp parameters are also accessible as float values through constants bsp_g, bsp_l and bsp_r. We distinguish a parallel vector from an usual array of size p because the different values, that will be called local, are blind from each other; it is only possible to access the local value x i in two cases: locally, on processor i (by the use of a specific syntax), or after some communications. In this way, the vector x 0 , x 1 , . . . , x p-1 holds the value x i at processor i. Since a bsml program deals with a whole parallel machine and individual processors at the same time, a distinction between the levels of execution that take place will be needed:

• Replicated execution is the default. Code that does not involve bsml's parallel vectors is run by the parallel machine as it would be by a single processor. Replicated code is executed at the same time by every processor, and leads to the same result everywhere.

• Local execution is what happens inside parallel vectors, on each of their components: The processor uses its local data to do computation that may be different from the other's.

• Global execution concerns the set of all processors together, but as a whole and not as a single processor. Typical example is the use of communication primitives.

The distinction between local and replicated is strict. Hence, the replicated code can not depend on local information, and remains replicated. We say that we lose replicated consistency if an expression outside of a parallel vector holds different values at different processors. Fig. 3.14 subsumes the use of these primitives. Informally, they works as follows.

Let ≪ x ≫ be the vector holding x everywhere -on each processor. The ≪ ≫ indicates that we enter a local section and pass to the local level. Replicated information is available inside the vector. To access to local information, we add the syntax x to open the vector x and get the local value it contains, which can obviously be used only within local sections. The local pid can be accessed with pid. Thus ≪ pid ≫ will contain i on the processor i. The proj primitive is the only way to extract a local value from a vector. Given a parallel vector, it returns a function such that applied to the pid of a processor, it returns the value of the vector at this processor. proj performs communications to make local results available globally within the returned function. Hence it establishes a meeting point for all processors and, in bsp terms, ends the current super-step. The put primitive allows any local value to be transferred to any other processor. As such, it is more flexible than proj. It is as well synchronous, and ends the current super-step. The parameter of put is a vector that, at each processor, holds a function of type (int → 'a) returning the data to be sent to processor j when applied to j. The result of put is another vector of functions: At a processor j the function, when applied to i, yields the value received from processor i by processor j. The last primitive allows the evaluation of two bsml expressions E 1 and E 2 as "super-threads". From the programmer's point of view, the semantics of the super is the same as pairing i.e., building the pair (E • par<T> encapsulates the concept of parallel vector. This class can be built from a large selection of c++ constructions ranging from c-style array, c++ standard container, function or lambdafunction. For example, bsp::par< vector <double > > v creates a parallel vector of arrays of double.

In this way, each processor contains an array of double. Local access to a parallel vector data is done through the traditional c++ dereferencing operators.

• pid_ is a global parallel vector that evaluates to the "pid" of current processors.

• sync performs an explicit synchronization and ends the current super-step.

• proj returns a function object that maps the identifier of a processor to the contents of the parallel vector held by this processor and ends the current super-step.

• put allows the local values to be transferred to any other processor and ends the current super-step.

The return of put is a parallel vector of function object of type T(int) that returns the data received from processor i when applied to i.

A special feature of bsp++ is the ability to nested parallel vector for hybrid architectures: the possibility to call a BSP_SECTION within a BSP_SECTION. The implementation thus uses mpi, Open-MP or cuda depending of the architecture.

The Python Version: BSP-Python [START_REF] Hinsen | Using bsp and python to Simplify Parallel Programming[END_REF]. bsp-python is an adaptation of the bsml primitives for python. The implementation relies of mpi or of bsplib. Most scientists did not consider python's programs sufficiently fast for number crunching. However, what make this language interesting in such applications was the idea of multiple language coding: the usually small parts of the code in which most of the cpu time is spent are written in a compiled language, usually fortran, c, or c++, whereas the bulk of the code can be written in a high-level language. There are several ways to create global object (parallel vectors), corresponding to their typical uses:

• ParConstant(v); a global object represents the constant v that is available on all processors.

• ParData(lambda pid, nprocs: v); defines the local representation as a function of the processor number and the total number of processors to v.

• ParSequence(v); distributes its argument v (which must be a python sequence) over the processors as evenly as possible.

bsp-python communication operations are defined as methods on global objects. An immediate consequence is that no communication is possible within local functions or methods of local classes. bsppython propose a set of communication patterns implemented as methods in all of the global data classes. For example, we have:

• put(pid list) which sends the local value to all processors in pid list (a global object whose local value is a list of processor identifiers) and returns a global data object whose local value is a list of all the values received from other processors, in unspecified order;

• fullExchange() which sends the local value of each processor to all other processors and returns a global data object whose local value is a list of all the received values, in unspecified order;

• accumulate(operator, zero) which performs an accumulation with operator over the local values of all processors using zero as initial value and the result is a global data object whose local value on each processor is the reduction of the values from all processors with lower or equal number.

In the communication operations described until now, it is always the local value of the global data type that is sent, either to one or to several receiving processors. In some situations, it is necessary to send different values to different processors. In principle this can be achieved by a series of put operations, but a special communication operation is both more efficient and allows more readable code. For this purpose, bsp-python provides a specialized global data type called ParMessages. Its local values are lists (or sets) of data-processor identifier pairs. The method exchange() sends each data item to the corresponding processor and returns another ParMessages object storing the received data items stored in a list in an unspecified order, thus each processor can easily iterate this.

(c) BSP Large Data/Graph Manipulation: BSP over the Cloud

With the new opportunities brought by cloud computing, large-scale applications that were restricted before to large research centers can now be executed with modest investments on infrastructure and maintenance. The typical cloud applications generates and process huge quantities of data. One of the main challenges is now how to deal with problems on how to store, manipulate and analyse this huge amount of data. If the application presents a good data parallelism, MapReduce framework achieves very good performance. If not, some recent research have proposed more sophisticated ways to achieve better performances on cloud computing platforms mainly using the bsp model [START_REF] Goldman | The suitability of bsp/CGM model for Clouds[END_REF][START_REF] Liu | BSPCloud: A Programming Model for Cloud Computing[END_REF][START_REF] Redekopp | Optimizations and Analysis of bsp Graph Processing Models on Public Clouds[END_REF]. We here shortly present them.

Google's Pregel [START_REF] Malewicz | Pregel: A System for Large-scale Graph Processing[END_REF]. pregel is a powerful c++ dedicated parallel language to express graph algorithms and to run them efficiently on several clusters. The approach centers to the computations on the vertices of the graph. Graph algorithms are expressed by the use of specific bsp primitives. The implementation relies of low level Google's libraries with checkpoint at each barrier enable fault tolerant.

Each vertex of the graph has a unique "id", an associated value and a list of output weighted edges. The computation is organized using a master/slave architecture and the input data is arbitrarily partitioned and stored on a distributed storage system. The pregel system maintains the vertices and edges stored on the node that will perform the computation but the work is migrated to where the data are stored. On each super-step, each worker node invoke a procedure Compute() of each active vertex that is under its control. This procedure is responsible for the execution of the business rule of the algorithm and is allowed, among other actions, to invoke other methods, compute new values for the vertex, add or remove vertices and edges, and send messages to other vertices. These messages are exchanged directly among the vertices, even if the vertices are being executed on different machines of the platform. The messages are sent asynchronously in order to allow the overlapping of computation and communication, but are delivered to the destination vertex only on the beginning of the next superstep. If a vertex declares that all its processing was done, it sends a message informing all the other nodes and deactivates itself. The master node stops the execution of the application after receiving this message from all the participants.

An important feature required by graph algorithms is the ability of change the topology of the graph. For example, a clustering algorithm might replace each cluster with a single vertex, and a minimum spanning tree algorithm might keep only the tree edges. In order to avoid conflicting changes on the topology (for instance, multiple vertices could issue a command to create a vertex V with different initial values), Pregel uses two mechanisms to achieve determinism: (1) a partial ordering: removals are performed first, with edge removal before vertex removal and additions follow removals, with vertex addition before edge addition; (2) a handlers for user defined conflict-resolution policies. Any change on the topology is only performed on the next super-step, before the invocation of the Compute procedure. pregel is considered as the main reference of graph applications on the cloud. And it is use by Google for their own applications. Due its proprietary nature, it is not possible having the code source nor the full api. Fig. 3.15 gives the available methods of the class Vertex. For example, a Pregel implementation of the traditional Google's page-ranking is: The example works as follow. The graph is initialized so that in superstep 0, the value of each vertex is 1/NumVertices(). In each of the first super-steps, each vertex sends along each outgoing edge its tentative page-rank divided by the number of outgoing edges. Each vertex sums up the values arriving on messages into sum and sets its own tentative page-rank. Note that, a truly page-rank algorithm would run until a convergence was achieve. hama has also a dedicated api for manipulating graph. It is mainly an implementation of some typical algorithms and parallel iterators over the distributed vertices or edges. The papers [START_REF] Goldman | The suitability of bsp/CGM model for Clouds[END_REF][START_REF] Liu | BSPCloud: A Programming Model for Cloud Computing[END_REF][START_REF] Redekopp | Optimizations and Analysis of bsp Graph Processing Models on Public Clouds[END_REF] give many other graph libraries but we have currently not yet had time to study them. And most of them are still in their beta version.

Which Routines for BSP-Why

After having seen all kinds of routines in the whole spectrum of libraries, we need to choose the most relevant ones to be treated in our tool bsp-why. One of the short term objectives of this research is to be able to prove with bsp-why: (1) mpi programs that are well structured and (2) most of the bsp programs from the above libraries. For (1), this can be done by working with mpi programs that only use collective operations. These ensure that the processors will synchronize to exchange their data, in a way similar to bsp. However, collective operations in mpi and some bsp libraries (e.g. the pub) come with the notion of groups and communicators. It is thus necessary to extend our model to be able to work with programs that synchronise over a restricted group of processors. In bsp-why, we chose to take into account the approaches of both mpi and the pub library to subgroup synchronisation.

Both bsplib and pub have high performance versions of their primitives which are using unbuffered communications (the buffer is sent, not a copy) in a asynchronous fashion (at an unspecified time before the synchronisation) which means that if the buffer is modified, it is unspecified what will be sent. Those primitives are not studied here because they are (1) too complex; (2) and with unspecified interaction with traditional bsp primitives; (3) and too non-deterministic. For communicating routines, we have chosen:

• bsmp primitives à la pub for just a historical reason (formal study of the semantics in [START_REF] Gava | Formal Semantics of a Subset of the Paderborn's bsplib[END_REF][START_REF] Gava | Two Formal Semantics of a Subset of the Paderborn University bsplib[END_REF]).

bsmp primitives using a queue for reading the received messages (in bsplib and java libraries) can trivially be added.

• drma primitives à la bsplib. The main reason is that the global registration seems cleaner than the constraint to have an order using the push primitives. Note that this method could be simulated by checking if the registration has the expected data.

• User defined collective operations. The user can define its own collective operations and thus its own schemes of communication. For example, in Chapter 5, we will use a kind of all-to-all collective operation.

bsml like and NestStep primitives have not been really taken into account. However, using the analysis (the block decomposition) of bsp-why, we will meet again the parallel vectors of bsml and the "steps" of NestStep because those structures are mainly used to distinct what are local computations (only for processors) and what is for the whole parallel machine. Also, bsp-why currently does not provide primitives for thread migration nor oblivious synchronisation nor nested computation (hybrid architectures). It is for future work.

Common Errors Introduced in BSP Programming

Proving the correct execution of parallel programs is actually significantly more difficult than proving that of sequential programs, because of several reasons. Let us talk a bit about the main problems that arise from the parallelisation of the computations. We thus try to enumerate the possible bugs that can be introduced by a faulty use of parallel/distributed routines. We then show how this possible errors can occur in bsp like computations.

Common Errors and Bugs (a) Deadlocks

The possibility of a deadlock is probably the most well-known problem when dealing with the parallelism.

A deadlock is a situation in which two or more processes are each waiting for the other to terminate, hence remaining in a blocked state. Let us give a basic mpi example of a deadlock situation. We consider a parallel machine with two processors, with the following parallel communication calls:

Process 0 Process 1 v'=Recv(1) v'=Recv(0) Send(1,v)
Send(0,v) where Send(i,v) is for sending a value v to processor i and Recv(j) is for receiving a value from processor j. In this example, we assume that the Recv and Send procedures are blocking, meaning they await a result before the remaining of the program is executed. The consequence is that when executing this code, the processor 0 will be waiting for data from the processor 1, while the processor 1 will be waiting for data from the processor 0. The Send instruction can never be executed, and the two processes will wait in the deadlock state forever. A correct program could be for instance:

Process 0 Process 1 v'=Recv(1) Send(0,v) Send(1,v) v'=Recv(0)
Here the Send and Recv operation are matched, so no deadlock occurs. Various models have been proposed to reduce the risk of deadlocks. In particular, using bsp or mpi collective operators or skeletons or high-level patterns, significantly decreases the risk. For instance, our example could be rewritten using the mpi sendrecv routine, which combines the send and receive operations.

Process 0 Process 1 v'=Sendrecv(1,v) v'=Sendrecv(0,v)

(b) Interleaving of Computations (Data Race Condition)

When executing a sequential program, one can immediately see when reading the code the order in which the instructions will be executed. However, when analysing a parallel program where processors share data, one must take into account that some processors might progress faster than others. Let us give this simple example:

Process 0 Process 1 x=0 x=0 x=x+1 x=x+1 y=hpget(x,1) y=hpget(x,0)
print(y) print(y) where hpget(x,i) returns (read) "immediately" the value of x at processor ihpget is the highperformance drma communication routine explained below. In this example, every processor executes the same sequence of instructions. This is consistent with the spmd form of parallelism, which will be the focus of our study. We can imagine that the expected behaviour of this program was to display the results of the computation of both processors. Indeed, if the processors execute always the same instruction simultaneously, they would both store in x the result of the computation, then get the result from the other processor, and then display both y. However, usually when looking at parallel programs it is impossible to ensure that all processors execute the same statement at the same time. In this example, if for instance the processor 0 finishes the computations faster than the processor 1, he might execute the next instruction, y=hpget(x,1), while the second processor is still computing, and his x is still 0. The output on the processor 0 would then be 0, and 1 for the other.

In this example the problem was fairly obvious. However, such a bug can be more difficult to spot. The behaviour of such programs is often highly non-deterministic, which makes the bug even harder to anticipate and correct. Note that this problem does not occur using pure bsp routines since all communications are performed during the super-step's phase of communication. Furthermore, this kind of bugs is generally the subject of concurrent analysis but can also appear in mpi programs with the use of asynchronous "window"drma routines, also call "one-sided communication".

These Errors in BSP Programs (a) Deadlocks

Even if the bsp model is deadlock and data-race free in its principle and simplifies the writing of parallel codes, many errors, even deadlocks, can appear in bsp programs (or at least in mpi programs written in bsp style) in addition of the classical sequential programming errors (buffer and integer overflows, non terminating loops, etc.). Take for example the following c codes using bsp or mpi: here, a deadlock can occur and the parallel machine would crash on some architectures. The same problem arises when: (1) one processor performs an oblivious synchronisation while other processors perform a global barrier; or (2) when a processor performs an oblivious synchronisation, waiting for instance for n messages and only n-1 messages are received. Communication can also generate errors. In addition to sending a value to an "id>nprocs", the reading of a bad number of values can induce bugs. For instance: the last processor would write over the limits of x of processor 0 and a segfault will occur.

( *
Subgroup synchronisation can also be a source of potential bugs. For example, using the pub, the creation of subgroups can be nested, but they must be created and destroyed in a lifo order. Otherwise a deadlock can occur or we can have unspecified (non portable) and non-deterministic results. Moreover, for both bsp and mpi, the subgroups must be disjoint when subgroups are created. Using mpi, a deadlock can occur when a processor is within two groups and each one performs a barrier. For instance: in this example, any processor that has an id (of MPI_COMM_WORLD) multiple of 2 and 3 (e.g. [START_REF] Armando | ltl Model Checking for Security Protocols[END_REF]) is within the two subgroups (subcommunicators) comm1 and comm2. In this way, some processors would perform the barrier of communicator comm1, others that of communicator comm2. Those are within the two subcommunicators would generate a deadlock. Note that this problem can not happen using pub because there is a nested decomposition and at each level, the routine bsp_partition tests if there is truly a creation of disjoint subgroups.

Our last example is not really an error since it does not crash the machine but it gives non-deterministic results and thus can disturb the meaning of a program. What happens when there are two distant writings (using the put primitive) of two different processors over the same area of memory? For instance: Two solutions are possible. First, forbid this case by adding logical conditions for distant writings. Second, suppose an order of writing for the processors. We have currently chosen the second solution since we suppose a deterministic semantics of bsp programs. However, changing to the first solution would be trivial.

Many other errors can be cited: forgetting to register a variable, bad size/number of messages, forgetting a barrier and all errors with pointers of the messages that one can imagine. Proving that programs do not have these incorrect uses of the routines would increase confidence in the codes. This will be even better if you also formally prove the behaviour of your bsp programs -at least, the more important parts of your code.

Related Works

There are really many parallel languages or parallel extensions of a sequential language (functional, iterative, object-oriented, etc.). It would be too long to list all of them. We chose to point out those that were the most important to our mind. Notice that, except in [START_REF] Loidl | Comparing Parallel Functional Languages: Programming and Performance[END_REF], there is a lack of comparisons between parallel languages. But it is hard to compare them since many parameters have to be taken into account: efficiency, scalability, expressiveness, etc. ) is an array programming language. The programmer gives instructions (with specific primitives) to manipulate the arrays (e.g. to copy a value in the cells above) and the compiler automatically distributes the computations and the data. It is close to sac but it is not a data-parallel language. This kind of operations can also be found in co-array fortran

Parallel Libraries and Imperative Programming

[[ 47 ]].
cilk [START_REF] Blumofe | Cilk: An Efficient Multithreaded Runtime System[END_REF] is an extension of c where the programmer can spawn a procedure like another thread of execution. Communications are performed using shared variables and synchronisations. occam-pi [[ 48 ]] was designed following the Communicating Sequential Processes (csp) process algebra and the π-calculus. The business code can be either haskell or c code. They appear in π-terms using a specific construction.

charm++ [172] [[ 49 ]
] is an extension of c ++ based on a migratable-objects programming model. The programmer decomposes the program into a large number of "chare", which can migrate and interact with each other via asynchronous method invocations. The system maintains a "work-pool" consisting of seeds for new chares, and messages for existing chares.

All these languages/libraries have generally no semantics or are too asynchronous for the study of this thesis. However, they are still used by programmers. Simulated this kind of work is for future work.

Other Parallel Paradigms (a) Parallel Functional Languages

Two nice introductions (with many references) to parallel functional programming can be found in [START_REF] Hammond | Parallel Functional Programming: An Introduction[END_REF] and [START_REF] Hammond | Research Directions in Parallel Functional Programming[END_REF]. They have been used as a basis for the following classification -with some updates.

The authors explain three reasons to use functional languages in the parallel programming. First, they ease the partition of a parallel program (task decomposition). Second, most of them are deadlock free: the value is independent of the evaluation order that is chosen and thus any program that delivers a value when run sequentially will deliver the same value when run in parallel. Third, they have a straightforward semantics which is great for debugging. Testing and debugging can be done on a sequential machine: functional programs have the same semantic value when evaluated in parallel as when evaluated sequentially.

The main drawbacks are: (1) their efficiency; and (2) the lack of libraries and tools; and (3) perhaps too hard to use for many programmers. It is still an old debate and it is not the subject of this thesis. bsp-why-ml is mostly an imperative language but there is some functional features.

(b) Algorithmic Skeletons and their Implementation

As described previously, skeletons are patterns of parallel computations. They can be seen as high-order functions that provide parallelism. They fall into the category of functional extensions following their semantics [1]. Most skeleton libraries extend a language (mostly java, haskell, ml, c/c++) to provide those high level primitives. Currently, the most known library is the Google's mapreduce [START_REF] Dean | MapReduce: a Flexible Data Processing Tool[END_REF]. It is a framework to process embarrassingly parallel problems across huge data sets -originally for the pageranking algorithm. Different implementations for java or c exist. But only two skeletons are provided 3. [START_REF] Appel | Separation Logic for Small-Step Cminor[END_REF]. RELATED WORKS which limit expressiveness. [START_REF] Falcou | QUAFF : Efficient C++ Design for Parallel Skeletons[END_REF] provides a set of flow skeletons for c++. Templates are used to provide an efficient compilation of the programs: for each program, a graph of communicating processes is generated and is then transformed into a classical mpi program.

For java, many libraries exist such as the ones of [START_REF] Alt | Algorithm Design and Performance Prediction in a java-Based Grid System with Skeletons[END_REF][START_REF] Leyton | Skandium: Multi-core Programming with Algorithmic Skeletons[END_REF] and [START_REF] Aldinucci | MUSKEL: An Expandable Skeleton Environment[END_REF]. The latter has been extended for multi-core architectures [START_REF] Ciechanowicz | Enhancing Muesli's Data Parallel Skeletons for Multi-core Computer Architectures[END_REF]. A study of how to type java's skeletons has been done in [START_REF] Caromel | Type Safe Algorithmic Skeletons[END_REF]. The authors note that some libraries of data-flow skeletons use a unique generic type for the data (even if it is an integer or a string), which can cause a clash of the jvm. They explain how to avoid this problem, using a simple type system.

[64] describes how to add skeletons in mpi (the eskel library), as well as some experiments. It also gives convincing and pragmatic arguments to mix message passing libraries and skeleton programming. We think that using ocaml in parallel programming (hpc applications) is not a wrong choice, since the generated code is often very competitive with the c counterparts. Some benchmarks of an ocaml implementation of data-flow skeletons for a numerical problem are described in [START_REF] Clément | Domain Decomposition and Skeleton Programming with OCamlP3l[END_REF]. But, the implementation currently sucks to tcp/ip. The first study of how to integrate flow skeletons and data-parallel ones was in [START_REF] Kuchen | The Integration of Task and Data Parallel Skeletons[END_REF]. Implementing skeletons using a bsp library was first done in [START_REF] Zavanella | Skeletons, bsp and Performance Portability[END_REF] and an application of the bsp cost prediction was also done in [START_REF] Hayashi | Automated bsp Cost Analysis of a Parallel Maximum Segment Sum Program Derivation[END_REF]. It seems possible to prove a bsp implementation of skeletons using bsp-why. It is for future work.

(c) Data-parallel Languages

To our knowledge, the first important data-parallel functional language was nesl [29] [ [ 50 ]]. This language allows to create specific arrays and nested computations within these arrays. The abstract machine (or the compiler) is responsible for the distribution of the data and computations over the available processors. Two extensions of nesl for ml programming are nepal [[ 51 ]] and manticore [[ 52 ]] [START_REF] Fluet | Implicitly-threaded Parallelism in Manticore[END_REF]; the latter is clearly a mix between nesl and concurrent-ml [START_REF] Reppy | Concurrent Programming in ML[END_REF] [[ 53 ]]concurrent-ml is a language of the creation of asynchronous threads and send/received messages in ml.

We can also cite the following data-parallel languages. First, sac

(Single Assignment c) [141] [[ 54 ]
] is a lazy functional language (with a syntax close to c) for array processing. Some higher-order operations on multi-dimensional arrays are provided and the compiler is responsible for generating an efficient parallel code. An extension of the famous lazy functional language haskell is data-parallel haskell [START_REF] Chakravarty | Data Parallel Haskell: A Status Report[END_REF]

[[ 55 ]].
It allows to create data arrays that are distributed across the processors. And some specific operations permit to manipulate them. The main drawback of these languages is that cost analysis (for comparing algorithms) is hard to do since the system is responsible for the data distribution.

bsp-why is also a kind of data-parallel language. But we are currently not taken into account all their features. It is for future work. In this chapter, we will present how our bsp-why tool works. In the first section, we will show the syntax of the language and of a core-calculus, the informal semantics and simple examples which will be used along this chapter. We will explain how the tool allows to ensure the safety and correctness of parallel (bsp) programs, mainly by transforming parallel programs (bsp-why-ML ones) into equivalent sequential ones (why-ML ones) that can be checked by the why tool.

Next, in the second section, we will explain the inner working of the bsp-why tool, and detail the transformation that is made to create a sequential why-ML program from the bsp-why-ML input. We thus give formally this transformation for a core-calculus.

Finally, we explain how to deal with subgroup synchronisation without much change.

The BSP-Why-ML Intermediate Language

As explain previously, we used a "sequentialisation" of the bsp-why-ml programs for their verification, that is we transform bsp-why-ml programs into why-ml ones and thus use the why1 tools to generate adequate conditions for the correctness of the programs. The main idea is to extract the biggest blocks of code without synchronization (that is purely sequential) and then each block is transformed into a for loop: for each processor, we execute the block of code. Fig. 4.1 illustrates this idea.

The main idea of our approach is thus to simulate the execution of a bsp program on a parallel machine by a sequential execution which will simulate the entire parallel machine. This way we are able to use the why tools. But in doing so, we need to simulate the memory (environment) of all the computers in a single computer. We also need to simulate the functioning of the communication operations.

The result is that each program written in bsp-why-ml, and then sequentialized into a why-ml program share the same structures: they use the same kind of environments to keep track of the parallel operations, the same data types (p-arrays, lists of messages, etc.), the same primitives to manipulate these environments. It is thus convenient to regroup all of these declarations in a separate file.

In the same way that why uses prelude files to define basic operations common to all why-ml programs, we use a bspwhyprelude.mlw file, which contains the common data types, the basic operations on these data types, the axiomatisation of the bsp operations, and of the memory model used.

We thus first present the syntax of a core-calculus bsp-why-ml that will be used to define the formal "sequentialisation". We also describe how bsp-why-ml extends why-ml. We then describe the memory model used for the simulation, that is how we deal with p different values for each variable in a bspwhy-ml program -how we represent them in a single memory.

Syntax of BSP-Why

The core idea of our approach is to generate a sequential code in the why-ml language, so that we can re-use its powerful back-end. It is thus mandatory that we generate why-ml code. However, it would be in theory possible to have a totally different language as our parallel language.

Our set of communication primitives follows an idealised version of the standard BSPlib. We thus have drma primitives (bsp_push et bsp_pop, bsp_put and bsp_get), as well as the message passing style primitives (bsp_send and bsp_findmsg).

The so-called High-performance primitives of the BSPLib/PUB library or thread migration, however, are outside the scope of our approach. They introduce nondeterminism, and can break the bsp model if not used carefully. In addition, it is not needed to have them in our basic language; in [START_REF] Fortin | From bsp Routines to High-performance Ones: Formal Verification of a Transformation Case[END_REF], we showed how it is possible to first program using only the standard primitives, and then use a certified optimisation procedure to automatically use the high-performance routines when it is possible in a safe way -this work has currently not been updated for the full bsp-why-ml language.

The syntax of bsp-why-ml language is thus the one of why-ml with an additional syntax for parallel instructions, see Fig. [START_REF] Appel | Separation Logic for Small-Step Cminor[END_REF].2 for the core-calculus. A program P is a list of declarations. A declaration d is either a program expression introduced with let or a declaration introduced with parameter, or an exception declaration. The full language also contains definitions: logic terms, axioms, parameters, etc.

(a) Programs Expressions

Program expressions mostly follows ml's ones. Fig. 4.3 gives two simple examples that will be used through this chapter to illustrate the manipulation and transformations made by our bsp-why tool. Remark that envCsendIs(j,bsp_pid + 1,y,j,x) is here a syntactic sugar (to simplify the reading of the example) to say that ∀j y ≤ j < pid + 1 the j-th element of the environment (a list) of sending messages envC is the value of x. envC contains (at least) the necessary values for the prefix computation when j = p. Fig. 4.4 gives an example of a more complete bsp-why-ml program without logical annotations (left) and with the full program expression (right). This example will be explained in details in Section 5. 1.1. Programs contain pure terms (t e ) made of constants (integers, booleans, void, etc.), variables, dereferences (written !x), application and application of function symbols from the logic to pure terms. A

Pure terms:

te special constant nprocs (equal to p) and a special variable bsp pid (with range 0, . . . , p -1) were added to why expressions. In pure terms (terms without possible side effects), we also have introduced the two special function symbols bsp nmsg(t) and bsp findmsg t 1 t 2 : the former corresponds to the number of messages received from a processor id t (C function bsp_nmsgs(t)) and the latter to get the t 2 -th message from processor t 1 (C function bsp_findmsg(t1,t2)).

::= c | x | !x | φ(te, .
ref e introduces a new reference initialized with e. loop e {invariant p variant t l } is an infinite loop of body e, invariant p and which termination is ensured by the variant t l . The raise construct is annotated with a type τ since there is no polymorphism.

In the core-calculus, the five parallel operations are: (1) bsp push x, registers a variable x for global access; (2) bsp pop x, delete x from global access; (3) bsp put e x y, distant writing of x to y of processor e; (4) bsp get e x y, distant reading from x to y; [START_REF] Armand | Extending coq with Imperative Features and Its Application to SAT Verification[END_REF] bsp send e 1 e 2 , sending value of e 1 to processor e 2 . In order to simplify the presentation of bsp-why, parallel operations of the core-calculus (notably drma primitives) take simple variables as argument, instead of buffers. In practice, bsp-why does manipulate buffers, and adds proof obligations to avoid buffer overflows.

There are two ways to insert proof obligations in programs: assert {p}; e places an assertion p to be checked right before e and e {q} places a post-condition q to be checked right after e.

(b) Logical Annotations and Types

One can remark that bsp-why-ml or why-ml programs are ML programs with logical annotations inside the brackets. As in why, programs can take parameters that correspond to external values. The prefix "global" corresponds to a value that would be available on each processor -and possibly different. The why language is typed using a simple monomorphic type system for program expressions and traditional polymorphic ML type system for pure expressions -logics and terms. This type system is also completed with effects: each expression is given a type together with the sets of possibly accessed and possibly modified variables and the set of possibly raised exceptions. bsp-why adds also a special effect to global parameters called "sync" which corresponds to the fact that the code behind the parameter performs at least a global synchronisation -this is obviously the case of the bsp sync. That allows the user to define its own patterns of communications -as MPI collective operators or else. Note that a parameter with a "sync" effect suppose that all modifications of variables have been done and that the exception is only raised after the synchronisation.

In the core-calculus, annotations are written using a simple minimal first-order logic. A logic term t l can be a constant c, a variable x, the contents of a reference x (written !x) or the application of a function symbol φ. Note that φ is a function symbol belonging to the logic which is thus not defined in the program. The construct old(t l ) denotes the value of term t l in the precondition state and the From this simple minimal first-order logic, we add the construct t<i> which denotes the value of a term t at processor id i, and <x> denotes the p-array x (a value on each processor) by opposition to the notation x which means the value of x on the current processor. We also add f parray to logical term which is the abstract type for purely applicative arrays of size p (with some obvious axioms) and list. They are used for the description of p-values, one value per process and for the environment of communications. This will be explained later.

We assume the existence of a set of pure types (β) in the logical world, containing at least unit, bool, int and messages value. As in [START_REF] Filliâtre | Verification of Non-Functional Programs using Interpretations in Type Theory[END_REF], predicates necessarily include conjunction, implication and universal quantification. An atomic predicate is the application of a predicate symbol A and is not interpreted.

As in [START_REF] Filliâtre | Verification of Non-Functional Programs using Interpretations in Type Theory[END_REF], a value of type τ is either an immutable variable of a pure type (β), a reference containing a value of a pure type (β ref) or a function of type (x : τ ) → {p} β ǫ {q} mapping the formal parameter x to the specification of its body, that is a precondition p, the type τ for the returned value, an effect ǫ and a post-condition q. An effect is made of three lists of variables: the references possibly accessed (reads), the references possibly modified (writes) and the exceptions possibly raised (raises). And the the effect sync is used to define functions that perform synchronisations. A post-condition q is made of several parts: one for the normal termination and one for each possibly raised exception (E stands for an exception name).

For synchronous routines (a global parameters), it can be useful, in order to prove the correctness of a program, to give a logic assertion just before the routine instruction: the routines holds a pre-condition. This assertion should describe the computations done during the previous superstep and define how are the environments of communications of the processes at the end of the previous super-step. In the transformation to the why code, this assertion is used in the invariant of the loop executing sequentially the code of each processor. If the assertion is void, it will not be possible to prove more than the safety of execution of the program, i.e. the fact that the program will terminate without failing by an array overflow, an illegal message read, etc. If the logic assertion has not been added, the bsp-why tool would search it automatically. This will be explain latter when speaking of the generation of invariants of "for loops". We also explain in Section 4.3 how to deal with subgroup synchronisation.

(c) Syntactic Sugars

As the core-calculus used in this thesis (for the formal definition of the transformation) is missing commonly used statements, we can define some syntactic sugar. We consider the following classical syntax sugar:

e 1 ;e 2 ≡ let _ = e 1 in e 2 raise E ≡ raise (E void):unit and the traditional while is also a syntactic sugar for a combination of an infinite loop with the use of an exception Exit to exit the loop. That is, while e 1 {invariant p variant t} do e 2 done is interpreted as: We can note that the core calculus (and also the why tool) does not contain any assignment (x := e). This is due to the fact that is can be simulated by the following parameter with effect: In the same manner, the bsp sync operation, barrier of synchronisation, can be simulated by: global parameter bsp_sync: unit → {} unit writes ... sync {Do_the_Com} that is a specific parameter that holds a sync effect and where "Do the Com" is for the modification of all the environments of communication. As explained above, that allows users to define their own patterns of communications. Such a parameter must have a write effect on every modified communication environment.

We also write τ as a syntactic sugar for the function specification {p} β ǫ {q} when no precondition and no post-condition (both being true) and no effect (ǫ is made of three empty lists) are defined. Note that functions can be partially applied.

(d) Informal Semantics of the Primitives

Even if the formal semantics (defined in Chapter 6) contains many rules and many environments (due to the parallel routines), there is no surprise and it has to be read naturally. bsp programs are spmd ones so an expression e is started p times. We model this as a p-vector of e with its environments. A final configuration is a value on all processors.

Basically, a primitive adds the corresponding message to the environment. Values to be sent and distant reading/writing are stored in environment of communications as simple lists of messages. There are thus six additional components in the environment, one per primitive that needs communications. Each of them is a special variable for the assertions. For drma primitives, there is also the registration T which is described later (push and pop need communications for keeping the registration of each processor coherent). As with the BSPlib, drma variables are also registered using a registration mechanism that is each processor contains a registration T which is a list of registered variables: the first one in the list of a processor i corresponds to the first one of the processor j.

To avoid confusion between a new reference and those that have been registered before, one could not declare a reference that has been created before. This is not a problem since why always forbids this case to only have alias-free programs.

The Parallel Memory Model in Why (a) Returned Values

With bsp-why, we transform a parallel bsp program into a sequential code simulating its execution. At any given point in the execution of the program, a parallel program is likely to return p different values, on the p processors. For this reason, we chose to make the sequential expressions in the translated program return p-arrays of values.

For instance, let us say that we have a parallel function f defined in our parallel program, returning an integer value, that is guaranted to always be the processor id:

let f () = ... ; !x {return = bsp_pid}
Then the translated program will also be a function f , but instead of returning an integer value, it will return a p-array of integers:

let f () = ... ; !x {∀ proc_i, isproc(proc_i) → return[proc_i] = proc_i}
where isproc is a predicate to know if proc_i is a valid processor number, that is 0 ≤proc_i< p. This transformation of the logical assertion into a forall statement will be explained in Section 4.2.5.

(b) Data Types

Several data types are used in the transformation, and are defined in the prelude file. The fparray type, a functional array of length p, is used every time we need to have data on each processor. The parray type corresponds to the mutable array, which is a reference to the fparray.

Lists are used in several ways for the communication environments, and are defined in this file too. Various other data types are defined, such as the value data type used to transmit any kind of data, and the rvaluet type used to represent the values received with send messages.

(c) Communication Environments

As the semantics (defined in Chapter 6) suggest, three separates message queues, send_queue (C send ), put_queue (C put ), and get_queue (C get ), are used to store the communication requests before synchronisation. Each queue is defined as a list, with the usual constructors nil and cons. Similar queues are used for the push (C push ) and pop (C pop ) mechanisms.

To be as close as possible to the semantics, the communication procedures send, put, get, and likewise the synchronisation sync are defined as parameters. As such, we only give the type of the procedure, and an axiomatization given by the post-condition, not the effective sequential code used: an actual sequential code would make more proofs, the additional verification conditions for this extra code.

In the EnvR section of the file, we describe R which contains messages sent during the previous superstep. Since it is possible to send different types of value with the communication instructions, a generic type value is used, and one function of serialisation and one of deserialisation are needed for each data type used in the program. One axiom for each data type ensures that the composition of deserialisation and serialisation is the identity.

As an example, we show the different parts of the prelude file used to model the behaviour of bsmp communications. First, we define the type used to store the messages waiting to be sent, using the usual list definition (nil and cons): type send_queue logic add_send : value , send_queue → send_queue logic nil_send : send_queue

The logical function add_send (⊕ in the semantics) will be used to effectively add a send message in a send_queue. Each processor has p send_queue, containing the messages to be sent to the p processors of the parallel machine. Next, we define some useful operations on these lists, using an abstract logic definition, and an axiomatisation for each logic function. We give for example the axiomatisation of the nsend function, used to determine the number of messages waiting (Size(s.R, to) in the semantics): logic nsend : send_queue → int logic in_send_n : send_queue,int,value → prop axiom nsend_nil : nsend(lfnil_send) = 0 axiom nsend_cons : forall q:send_queue. forall n:int. forall v:value. nsend(q) = n → nsend(lfadd_send(v,q)) = n+1 axiom nsend_cons2 : forall q:send_queue. forall v:value. nsend(lfadd_send(v,q)) = nsend(q) + 1 The in_send_n function is used to test the fact that a message is in the list. Lastly, we can define the variable used for the global environment. For each processor, we have a parray of send_queues, hence the final type, and the method bsp_send defined in the semantics. isproc is a useful predicate defined earlier in the prelude file, stating that an index is a valid processor id (i.e. 0 ≤ i < p):

let x = ref 0 in let y =
parameter envCsend : send_queue fparray fparray ref parameter bsp_send: dest0:int → v:value → { isproc(proc_i) } unit reads proc_i writes envCsend {envCsend=pupdate(envCsend@, proc_i, pupdate(paccess(envCsend@,proc_i),dest0, lfadd_send(v,(paccess(paccess(envCsend@,proc_i),dest0)))))}

We now define the environment used to store the values received during the previous synchronisation: The logic function rvalue_get allows to retrieve the n-nth message sent by a processor src. envR, as previously, is defined as a parray. The bsp_findmsg is the corresponding parameter, and it can be used in the bsp-why programs.

(d) Global synchronisation: Example of a "sync" Effect

The only remaining part of the bsmp process is the synchronisation function, which is defined, as in the semantics, by the use of a Comm predicate. We give here the part of the predicate concerning the bsmp communications:

predicate comm_send(envCsend:send_queue fparray fparray,envCsend':send_queue fparray fparray,envR':rvaluet fparray) = (forall i,j: int. isproc(i) → isproc(j) → (paccess(paccess(envCsend',i),j) = lfnil_send)) and (forall i: int. isproc(i) → (forall j:int. forall n:int. forall v:value.

(rvalue_get(paccess(envR',i),j,n)=v) ↔ (in_send_n(paccess(paccess(envCsend,j),i),n,v))))

predicate comm(envCsend:send_queue fparray fparray,envCsend':send_queue fparray fparray,envR':rvaluet fparray, ...) = comm_send(envCsend,envCsend',envR') and ... The comm_send predicate is specific to the send messages, and is called from the comm predicate. Lastly, the bsp_sync parameter ensures that the comm predicate is true.

(e) Dealing with DRMA Primitives

The most complex part of the prelude file is the definition of the drma mechanism. A typical drma communication is something such as bsp_put i x y. The meaning of this function call is that at the next synchronisation, the current processor will write the content of its variable x (taken at the function call) in the variable y of the processor i. The queued message should thus contain:

• An integer value, representing the target processor identifier;

• A value (of any type), representing the data that will be sent;

• A reference to a variable, representing the place where the value should be written. However, why does not allow the use of pointers, for the reasons explained in the introduction section on the Hoare logic and wp-calculus. It is thus impossible to simply put a reference to the target variable in a message queue. To solve this difficulty, we use a global two-dimensional array, named global, to store all the variables that need drma access. A special type is used to describe such variables, and for each variable x with drma in the program, a logical value x of type variable is added in the generated why file. This way, global[x][i] contains the value of variable x, on processor i.

To be in accordance with most bsplib-like libraries with drma routines, we define a registration T . The push instruction can associate different variables on different processors. This is modelled using an additional array, which stores the association of the variables on different processors. For instance, if even processors push the variable x while odd processors push the variable y, with p = 6, the next sync operation will add a line [x, y, x, y, x, y] in the association table. The index used in the global array is the variable on the first processor. For example, for the following code: We also recall that in the pub library, the order of push and pop should be the same on every processor, allowing to not use an additional synchronisation to check which block of memory (a variable) of a processor is linked to another block of another processor. It is easy (but currently not performed due to a lack of time) to deal with this memory model using the global array: at each synchronisation, we just have to check that all variables have been registered in the same order, that is appear in the same order in the lists of the global array.

4.2

Transformation of BSP-Why-ML Programs: the Inner Working

Generalities

Now that we have the necessary structures to simulate the environments and communication functions of the parallel machine (axiomatisation of the bsp routines), we can define the actual transformation of a bsp-why-ml program into a why-ml one, that will simulate its parallel execution. This transformation is composed of several steps, which we summarized in the scheme of Fig. [START_REF] Appel | Separation Logic for Small-Step Cminor[END_REF]. [START_REF] Armando | ltl Model Checking for Security Protocols[END_REF]. We also give the formal transformation for the core-calculus.

Note that the bsp-why tool was written in the ocaml language. Since we chose to keep the syntax of the language very close to why, we were able to re-use the open-source code of why for a large part of the program. It is the case in particular for the parsing and printing part of the program. The actual transformation however had to be written entirely from scratch.

Let us now detail these different steps of the transformation. A part of this transformation for the core-calculus will be machine-checked in Chapter 6.

(a) Notations

During the different steps of the transformation, we encounter different kinds of trees representing our expressions, which are naturally very close since they represent the same program, but subtly different from each other. In our implementation of bsp-why we used prefixes on each constructor to be able to differentiate the different types. For instance, a simple if e1 then e2 else e3 might be successively represented with the following constructors:

• bspIfThenElse(e1,e2,e3) for the if-then-else as a bsp-why statement • blockIfThenElse(e1,e2,e3) in the block tree • wblockIfThenElse(e1,e2,e3) in the block tree in its sequential (why) form • whyIfThenElse(e1,e2,e3), finally, in the resulting why program This was necessary in our ocaml program. However, in the mathematical notation of the transformation, we will keep the notations light and use the same keywords at the different stages of the transformation. Thus, we could write

[[if e1 then e2 else e3]] = if [[e1]] then [[e2]] else [[e3]
], even if this means going from the parallel block tree to the sequential block tree. We will detail the transformation of the program expression and logic assertions in the following sections.

Identification of Sequential Blocks

The first step of the transformation is a decomposition of the program into blocks of sequential instructions, as Fig. [START_REF] Appel | Separation Logic for Small-Step Cminor[END_REF].7 shows. The aim is to be able to simulate the execution of a sequential block consecutively for each processor executing it, in a sequential way, instead of the normal parallel execution on each processor at the "same time". In order to obtain the best efficiency, we are trying to isolate the largest blocks of code that are purely sequential.

The result of this phase of the transformation is what we will call from now on a block tree, meaning an abstract syntax tree, but where the leaves of the tree are blocks of sequential code, instead of basic expressions. The block tree is constructed as the abstract syntax tree (AST), with the addition of a basic constructor for a block of non synchronising code. For this, we first must tag which parts of the code are purely sequential and which are not, that is tagging the parallel parts of the code. Then we can extract the blocks.

(a) Tagging of the Parallel Parts of the Code

In order to do the block decomposition correctly, we need to be able to tell if an instruction has a parallel effect or not. Two instructions potentially influence the parallelism of the program:

1. A bsp-why parameter defined with the synchronize effect (such as the the bsp sync instruction);

A function call, if the function body is determined to have a parallel code.

The first step of our transformation is to tag each node of the abstract syntax tree, with a boolean that says whether the subtree includes parallel code, or not. One might note that the following instructions will not be tagged as parallel:

• A function call, if the function body is entirely sequential;

• A call to a bsp procedure, such as bsp_send, bsp_push, etc. the reason is that the parallel communication associated with such functions are only in effect done during the next synchronisation.

The tagging algorithm is a recurive pattern matching of the bsp-why abstract syntax tree. It operates with side effects (instead of returning a new tree, we modify a mutable field in the current one), and also returns a boolean indicating whether the expression is parallel or not. An example of a generic case would be the if-then-else instruction: tagging the if e1 then e2 else e3 node of the tree is done by recursively tagging the three subtrees e1, e2 and e3; then, if at least one of the subexpressions was found to be parallel, we set the parallel tag of our node to true; else, we set it to f alse. For example, a part of the code of this tagging function is: 4.8 gives the formal definition of this tagging for the core-calculus. It is easy to read inductive rules where max(t 1 , t 2 ) is true if either t 1 or t 2 are true, f alse otherwise (a boolean "or"). An expression tagged true is able to perform a synchronisation and thus could not appear into a sequential block.

e 1 :
Example: Tagging of Simple Expressions. We are going to illustrate the different steps of the transformation on the sample code given as example in Fig. [START_REF] Appel | Separation Logic for Small-Step Cminor[END_REF]. [START_REF] Alt | Algorithm Design and Performance Prediction in a java-Based Grid System with Skeletons[END_REF]. Since the transformation works on the abstract syntax trees, we start by giving the tree corresponding to the SimpleCode1 and SimpleCode2 expressions. This is done in Fig. [START_REF] Appel | Separation Logic for Small-Step Cminor[END_REF]. [START_REF] Armbrust | Above the Clouds: A Berkeley View of Cloud Computing[END_REF].

We can now apply the tagging algorithm on the trees. The result is shown in Fig. [START_REF] Appel | Separation Logic for Small-Step Cminor[END_REF]. [START_REF] Bacci | P3L: a Structured High-level Parallel Language, and its Structure Support[END_REF]. Here we can make a few remarks on the examples:

(b) Function of the Block Decomposition

Once we have tagged the syntax tree as described above, the transformation to the block tree is straightforward. It is simply a recursive algorithm on the syntax tree, with two cases at each node:

1. If the node is tagged as having no parallel effect, we simply return a newly constructed Block, with the sequential code being the subexpression at the node;

2. If the node is tagged as parallel, we return the corresponding block tree constructor, with his children obtained by recursively calling our algorithm on the children of the current node.

For example, a part of the code of this decomposition is: .12 gives the formal definition of this block decomposition for the core-calculus. The first rules says that if the expression e is tagged as f alse then it can be a full sequential block which contain e. Otherwise, we build a tree like the expression.

let

(c) Maximising Sequential Blocks

It is interesting to note that for a program written mainly as a sequence of instructions, such as A;B;C;D;E, the order in which the sequences are parsed modifies the block decomposition. Let us take a look at our example SimpleCode1, assuming that the sequences are read in a different way. The corresponding tree is represented in Fig. [START_REF] Appel | Separation Logic for Small-Step Cminor[END_REF]. [START_REF] Bamha | Pipelining a Skew-Insensitive Parallel Join Algorithm[END_REF]. It appears immediately that one more node of the tree is considered to be a parallel code. Furthermore, there are now 3 isolated sequential sub-trees, which means we will get 3 sequential blocks in our transformation. This does not change the result of the execution in any way, so it could be ignored from a theoretical point of view. However, having 3 blocks instead of 2 also means that we will have 3 for loops in the resulting code instead of 2, so less readability, and more proof obligations. For this reason, we decided to maximise the blocks in sequences of instruction. This is done by making a simple observation. There are two elementary patterns in which the block decomposition is clearly not optimal, corresponding to the codes A;(B;C) where C has synchronisations, and (A;B);C where A has synchronisations. We can thus re-organise the code when we meet such a pattern, as described on

< e, f alse >

= Block(e) < e1; e2, true > = < e1 > ; < e2 > < let x = e1 in e2, true > = let x = < e1 > in < e2 > < let x = ref e1 in e2, true > = let x = ref < e1 > in < e2 > < x := e, true > = x := < e > < if e1 then e2 else e3, true > = if < e1 > then < e2 > else < e3 > < while e1 do e2 done, true > = while < e1 > do < e2 > done < raise (E

(d) Limitation

This strict block decomposition does not accept all valid bsp programs. In practice and by reading many bsp algorithms (those cited in Section 2. [START_REF] Alt | Algorithm Design and Performance Prediction in a java-Based Grid System with Skeletons[END_REF].2), we only find this problem in reduction-like (logarithmic) loops where the code can clearly be re-factored. This does not seem too restrictive.

Block Tree Transformation (a) Tree Transformation

After having regrouped the sequential parts of the program into blocks, the rest of the tree is just the structure of the parallel mechanisms, and can not be altered. Thus, the transformation on the block tree is made with a traversal of the tree where we apply recursively the transformation. We write e ′ = [[e]] where e ′ is the result of the tree transformation.

We take a bsp-why block tree, and return a why block tree, which is structuraly identical, but with different constructors. For example, a part of the code of this tree transformation is: However, several cases need more attention. First, the base case in this recursive algorithm is the transformation of a sequential block. It is the next step of the global transformation, and will be detailed in Section 4. 2.4. Then, the if and while statements needs an additional treatment: when transforming a if or while structure at the block tree level, there is a risk that a bsp_sync instruction might be executed on a processor and not on the other. We generate an assertion to forbid this case, ensuring that the condition associated with the instruction will always be true on every processor at the same time.

For instance, if the source code is:

while ( (pow_int 2 !i) < bsp_nprocs ) do (...) bsp_sync void; (...) i:=!i + 1 done then the assertion generated would be: assert {forall proc_i, isproc(proc_i) → (((pow_int(2,paccess(i,proc_i))) < bsp_nprocs) ↔ (forall proc_j, isproc(proc_i) → (pow_int(2,paccess(i,proc_j))) < bsp_nprocs) ) };

If the condition is true for a processor (proc_i), then it must be true for any other processor. The valid Parameter. In practice, instead of explicitly generating the assertion, we indirectly enforce it by a call to a why parameter, called valid. This parameter is called on the result of the boolean expression. For instance, the transformation of the if e1 then e2 else e3 expression will be a why expression under the form:

if valid([[e1]]) then [[e2]] else [[e3]].
The valid parameter is then defined in the file bspwhyprelude.mlw. The why-ML code of its definition is as follows:

parameter valid : a:'a fparray ref →

{ ∀i:int. ∀j:int. isproc(i) → isproc(j) → paccess(a,i) = paccess(a,j)} 'a reads a {∀ i:int. isproc(i) → result = paccess(a,i)}

It takes a p-array of values in argument.

A proof obligation will then be generated by why to ensure that the precondition is met. This translates into a proof obligation equivalent to the one described in the assert above. Fig. [START_REF] Appel | Separation Logic for Small-Step Cminor[END_REF]. [START_REF] Bamha | An Efficient Equi-semi-join Algorithm for Distributed Architectures[END_REF] gives the formal definition of this tree transformation for the core-calculus and Fig. 4.16 gives the results of the examples of Fig. [START_REF] Appel | Separation Logic for Small-Step Cminor[END_REF]. [START_REF] Alt | Algorithm Design and Performance Prediction in a java-Based Grid System with Skeletons[END_REF]. The first rule says that sequential block of code is transformed into a "for-loop" as described in Fig 4 .1 that is we execute the code p times one per processor. Other rules are just inductive cases except for the while and if statements: we need to introduce the valid parameter to test if all processors go well and together inside the same sequential block.

Translation of Sequential Blocks (a) Local Block Transformation

The last step of the transformation is the transformation of a block of sequential code. The idea of bspwhy is that for a block of instructions that would be run in parallel on each processor simultaneously, we simulate its parallel execution by the use of a "for loop" that will run the code sequentially, with one execution for each processor.

The transformation is thus the following: [[Block(e)]] = f orp([[e]] i ). The notation f orp(e) is a shortened notation for the "for loop". When making explicit the loop, one thing is immediately visible. it is necessary to give an invariant to the while loop, if we hope to prove anything about the program. Thankfully, the invariant can in general be inferred automatically which will be explained later. The loop consists of the independent execution of the sequential code e, simulated for the processors 0 to p. This means that one iteration of the loop will have executed the code e for one processor. Hence, if we know the post condition post that we would like to ensure after the block e, the invariant at the i-th iteration should include: ∀j:int. 0≤j<i → post[j].

The meaning is that for all the processors j < i, the code e has already been executed, so the post condition post at the processor j holds. However, because parallel variables are simulated with the use of arrays in the why sequential program, there needs to be another part in the invariant, ensuring that at the ith iteration of the loop, we have not modified the array for the processors i to p -1 yet. In other words, for each variable v modified in the loop, we need to have: ∀j:int. i≤j< p v[j]=v[j]@loopstart that is the value of v for the processor j must still be the same as the value at the beginning of the loop, denoted by the label @loopstart.

Variables Accessed Within the Loop. Managing the variables that are accessed within the loop raises a few difficulties. The first difficulty comes from the fact that a variable x can be translated in three different ways depending on its use:

1. If the variable is declared locally, and is only used within the sequential block, it is simply translated in a similar variable x;

If the variable is used outside of the block, it can have different values depending on the processor.

If it is not used with a push instruction, it can simply be translated by an array of variables of the same type;

3. If the variable is used with a push instruction, it is more difficult to use directly an array, because it is not possible in why to transfer pointers to a variable, which would be necessary during the communications; in that case, we chose to use a bigger array, containing all the variables used in drma accesses; that way, we can transfer in the communications the index of the variable in the array, rather than the variable itself.

In our translation, we chose to solve both difficulties by adding an intermediate step inside the loop. At each iteration, before and after the translated code, we add lines of code to store temporarily the variables in their "natural" name. There are several advantages with this method. First, the translation itself is made easier. A statement with x in bsp-why can directly be translated into a statement with x too, in most cases. Perhaps more importantly, the translated code is easier to understand. If the source bsp-why program was talking about x, the resulting why program will be talking about x. It might not be so big an issue, since the programmer is not expected to read the why program anyway. But it also translates directly in the proof obligations generated by why. If an obligation is not discharged automatically by the provers, it is necessary that the user be able to look at it, understand what part of the code it comes from, and make adjustments to the program assertions if it is the correct action. This is much easier if the proof obligation includes references to x, and not parray_get x i, when the original program defined a variable x.

Also when translating the logic expressions, it is necessary to translate the variable in the same way as previously. When it is necessary to refer to the variable x as an array <x>, or to the variable on a different processor than the current one, x<i> is transformed in the access to the i-th component of x.

(c) Translation of the Local Code

Finally, we have to give the translation of a single block, denoted by [[e]] i , to the code that can be executed within the "for" loop. The transformation will have the index i of the processor as a parameter, named proc_i. The transformation of control instructions is straightforward, in the same way as previously, by walking the tree recursively. Fig. [START_REF] Appel | Separation Logic for Small-Step Cminor[END_REF].17 gives the formal definition of this block transformation for the core-calculus where the parallel instructions (put, send, etc.) are not directly translated in an equivalent sequential code but are replaced by calls to the parameters axiomatised in the prelude file.

(d) Finding the Invariant of the "for loop"

When writing our implementation of bsp-why, we encountered an unplanned difficulty: generating the invariant of the "for loop". We will discuss how we dealt with them, and a possible evolution of the bsp-why tool. It is relatively easy to find all the variables that are modified by the block, and generate the second part of the invariant for these variables. Guessing an appropriate post-condition for the sequential code is however a bit trickier. In bsp-why, we try to guess the most general post-condition by calling the why program on the block of code, and inferring the post-condition. In case the programmer wishes to utilise a different invariant, he can do so by giving explicitly a post-condition to the block of sequential code.

The Problem. In our planning of the bsp-why tool, we did not anticipate the difficulty of generating invariants for the loop executing a block of code on every processor successively. At first glance, it appears to be a trivial problem. The "forp loops" are simple in their concept, they execute the same independent code on the processors of our parallel machine, from the processor 0 to the processor p -1.

Since the processors execute independently from each other, the invariant form is really trivial: it just states that the code has been executed, for the processors 0 to i, where i is the current processor.

However, the difficulty comes from the logic expression representing the execution of the local block of code. With our approach, it is not easy to obtain it automatically. bsp-why is designed as a tool that transforms a program from one parallel language to a sequential program, and then trusting why to certify the correct execution of the sequential program. It is not designed to infer a correct postcondition from a block of code.

Past Solution.

In our first version of bsp-why, we partly solved the problem by asking the programmer to give, for every block of local code, a postcondition describing the result of the execution of the block of code. In a program such as the prefix calculus presented before, this is not too much of a difficulty: there are only two blocks of code, one leading to the synchronisation, and one leading to the end of the function. A function always has a postcondition, and it did not seem unreasonable to ask the programmer to give an assertion describing the environment state just before a synchronisation.

Using these postconditions, we were able to generate the correct invariants for the forp loops. However, this approach raises some major issues.

First of all, even with such a simple example, the programmer has to provide an assertion in the middle of the program. It might make sense because of the synchronisation that immediately follow, so the programmer should have a clear vision of what messages are about to be sent, and be able to describe it in a logic formula. For instance, the direct prefix example introduced before is in a way an "ideal" situation. The program is composed of only two blocks of code, both pretty apparent to the programmer, one before the synchronisation, and one after. However, there are many algorithms that would translate in bsp-why programs in a much less intuitive manner and that detracts from the usefulness of the why tool, where the idea is for a programmer to give a specification of the program, i.e. a precondition and postcondition for the function, and to obtain automatically the proof of all necessary proof obligations, as given by the wp-calculus. Having to give intermediate assertions at different points of the program was actually the main problem of the basic Hoare logic, and this is why the wp-calculus is used instead. For this reason, asking for the postcondition after every sequential block of code seems like a step in the wrong direction. Let us take a simple example:

let j = ref 0 in while(!j<nprocs) do
{ invariant 0<=j<=nprocs variant nprocsj } bsp_send(!j,0); bsp_sync(); j:= !j+1 done This is a basic example, sending a message (containing 0) to every processor successively in a while loop, synchronising after each message. It might not be obvious for someone not familiar with the bsp-why internal functioning, but if we apply the basic transformation as defined in the previous sections, such a program would result in four distinct blocks of sequential code. This means that with the basic solution of asking the programmer a postcondition for each block, he would have to provide four logic assertions in such a code. It would clearly be totally unreasonable.

Proposed Solution. bsp-why does contain some optimisations to help improve the readability of the transformed code. In particular, the first block of sequential code of this example is simply the constant 0 given when initialising the variable j. For such a code, bsp-why will not go through the full transformation and execution of a "forp loop", since it is obviously not needed. So in this example, the programmer would not actually need to give the full four postconditions. However, these optimisations would only reduce the scale of the problem, without solving it entirely. It would still be very inconvenient to have to give the logic assertions.

For this reason, the current implementation of bsp-why tries to guess the correct postcondition, by running the why tool on a specially crafted file, and inferring the postcondition from the proof obligations generated by why. This is not the cleanest but it was the only convenient way that we found to generate this invariant without changing totally the design choices of bsp-why.

It is important to understand that this issue is not really relevant to the basic logic of our transformation. The method of transforming a parallel program into a sequential program, and then applying a tool to prove sequential programs, is proved to be correct in our work. However, the limitations of the specific tool that we chose to use, why2, raise this implementation issue that complicated our work. If we used a sequential proving tool able to guess the invariants of some loops, we might not have encountered this issue at all. In addition, bsp-why still allows the user to specify a postcondition on a block of code for the case where it would not be inferred automatically correctly. So even if it is not perfect, the user will still be able to prove his parallel program.

Another important point is that even though this invariant generation is not formally proved in any way, this does not detract from the insurance that if why accepts the sequential program as being correct, then the bsp-why source program is correct. The invariant is used as a necessary step to be able to prove some of the generated proof obligation, but a wrong invariant would never allow to fully prove an incorrect program. It is also interesting to note that the use of why3 might allow us to solve this particular issue in a much more satisfying way. why2 was presented as an opaque tool, transforming a program into proof obligations that were sent to a set of provers. why3 is much more flexible, with the ability to use an api to manipulate why logic terms.

Translation of Logic Assertions

In the previous sections, we described how we generate a sequential program from the parallel program. The new program will have the same result as the parallel code, which means that if we prove that the sequential code is correct, then the parallel code is correct too. However, in order to be able to prove the correctness of the sequential code, we have to translate all of the logic annotations in your bsp-why program.

There are two parts in this translation: (1) generated annotations from those that concerning only the local memory of one processor; [START_REF] Aldinucci | MUSKEL: An Expandable Skeleton Environment[END_REF] annotations that can describing the whole parallel state of execution.

(a) Translation of Local Assertions

The syntax of logic assertions was described in Fig. [START_REF] Appel | Separation Logic for Small-Step Cminor[END_REF]

.2. We write [[t]

] i the translation of the logic term t, applied on the processor i. Similarly, we write [[p]] i the translation of the logic predicate p, applied on the processor i. The latter depends on the former, because logic predicates are constructed from logic terms. We give in Fig. [START_REF] Appel | Separation Logic for Small-Step Cminor[END_REF].19 the rules of the transformation, on both logic terms and predicates where:

• Because of the underlying memory model, variables x and dereferencing !x are both translated into a p-array access x[i] (Section 4. 1.2 described in more details the choices behind the memory model).

• In the case of a logic function call, the transformation is simply applied recursively to the arguments.

• For references with old(t) and at(t, l), the translation will be the same reference, but on the translated term.

• Finally, we have the new notations introduced by the bsp-why language. x<j> corresponds to the variable x on the processor j, so it is translated as a p-array access, but on the j-th component:

x[j]. <x> corresponds to all the values of x, seen as a p-array; because of the memory model, that is exactly x.

• The translation of logic formula is a simple recursive call to the components of the formula, translated logic terms in a function call as stated previously.

(b) Translation of Global Assertions

There are two ways to give a global assertion in bsp-why.

1. First, the x notation allows to give a general statement about the x variables of all processes, seen as an array. For instance, one could have the statement sorted(<x>) to say that the values found on the different processors are increasing along with the pid.

Another kind of global annotations is when a local annotation is implicitly seen as global.

For instance, if the postcondition of a parallel function f is of the form x=pid, what is meant is that on every processor, x will be equal to the processor identifier.

We could have had a separate translation for the two kinds of assertions. However, we found it easier to always translate in the same way. The translation of a logic predicate, in a parallel code, is defined as follows: • A parallel function f has for postcondition the predicate x=5. It is a local predicate, seen as global by stating that it is true on every processor. The translation is: ∀proc_i:int, isproc(proc_i) → parray_get(x,proc_i)=5; This is exactly what we intended when we wrote the postcondition, so the translation works well.

[[p]] ≡ ∀i, 0 ≤ i < p → [[p]] i that
• A sequential function g has for postcondition x=5. Since the function is not tagged as parallel, its postcondition will be translated only with the local translation. The result is thus x=5, once again as intended.

Dealing with Exceptions

Problematic. In the previous sections, we have ignored a problem that comes from having exception handling in the language. To better understand the issue, let us consider this example that includes a try statement:

try ( (if pid=0 then raise E else void); bsp_sync() ) with E → ...

If we naively follow the transformation as described earlier, the first step would result in the block tree show in Fig. [START_REF] Appel | Separation Logic for Small-Step Cminor[END_REF]. [START_REF] Basumallik | Programming Distributed Memory Sytems Using OpenMP[END_REF]. The if statement would be tagged as not being parallel, so it would constitute a block. The main step (the transformation from the bsp-why block tree to the why block tree) is mainly concerned with the safety of if and while nodes of the block tree. Since there are none here, it would not change things. The problem comes to light with the final step, the transformation of the sequential When running this code, the exception E will be raised in the first iteration of the "for loop", and will actually stop the loop immediately. It means that the code of the other processors was not executed at all. This means the transformation is not valid for such a program without modification. Another point of note is that the parallel program itself is incorrect in this example, since it would result in the processor 0 alone not doing the synchronisation, and a deadlock would ensue.

Solution.

To fix both of theses issues, we modified the tagging of parallel code in the first step of the transformation, so that it recognizes that a raise instruction can have an effect on the parallelism of the program. However, tagging all of the raise instructions in this way would be too restrictive, since the raise can be in a purely local computation that does not need to be guarded like a parallel one -it is mainly the case for the while statement which is a syntactic sugar to an infinite loop adjunct to a raise of an exception when the loop is over.

A better solution is tagging a raise instruction if and only if the matching try subtree contains some other parallel code. It is the case in the example above, so the raise would be tagged as well. This is accomplished by modifying the tag_parallel algorithm, so that it takes an additional parameter, a list of exceptions that are to be tagged. Then, the program processes in two steps when it encounters a try statement. In a first step, the whole subtree is tagged without adding the exception in the list. If the result is a not-parallel subtree, it is enough. However, if the try subtree is tagged as parallel, then the matching raise statements will need to be considered as parallel too. This is done by tagging a second time the subtree, but this time with the exception added in the parameter. For example, a part of the code of this tagging function is: Applying this algorithm to our example gives the block tree of Fig. [START_REF] Appel | Separation Logic for Small-Step Cminor[END_REF]. [START_REF] Bertot | Interactive Theorem Proving and Program Development[END_REF]. For this example, the algorithm works as follow. First, the raise is recognized as having a parallel effect, so it will not be put inside a "for loop". Second, as a result, the if statement becomes a node of the block tree, and will therefore be guarded with a proof obligation that the condition holds true for all the processors at the same time. In this example, it is not provable, so the deadlock is correctly detected.

Dealing with Subgroup Synchronisation

We have presented so far the transformation of programs in the strict bsp model, as defined in the bsplib standard. However, some extensions of this model are of a particular interest to us. In this section, we will show how to extend our transformation to include the possibility of synchronising over a subgroup of processors -mainly the possibility offered by the pub library and MPI as introduced in Section 3.1.2. Because this functionality can be seen as outside the strict bsp model, we have treated it later in our work and we give it a dedicated section. In this section we will thus discuss the various changes that were necessary in our transformation to be able to deal with subgroup synchronisation.

Subgroup Definition (a) Extending the Operators of BSP-Why-ML

bsp-why is based on the bsp model. As thus, it seemed necessary to implement communicators in the bsp-pub manner. However, bsp-why also aims as being able to model MPI programs that use only collective operations. For this reason, it is important to be able to simulate the use of communicators as it is done in MPI. We thus decided to provide the two ways of creating communicators in bsp-why:

1. The bsp-why parameter bsp_partition returns a communicator, without synchronisation, in the same way as it is done in the PUB library (this was detailed in Section 3.2), and with the same restrictions in the partition created.

2. The parameter mpi_createcomm, on the other hand, creates a communicator without any restriction on the processors in the groups.

In bsp-why, both parameters are without a synchronisation effect. However, it is possible to define a synchronising parameter that would simulate the MPI_Comm_split function using mpi_createcomm.

The bsp-why prelude file defines the datatypes used for the subgroup synchronisation. First, a subgroup of processors is defined as an array of booleans of size p (bool fparray). This is easily interpreted: each processor of the bsp machine can either be part of the subgroup, or not. It is not possible, however, to simply assimilate a communicator with a subgroup. Several distinct communicators can match the same subset of processors. For this reason, communicators are stored in a list associating a communicator identifier and the corresponding subgroup of processors, with the bsp_partition and mpi_createcomm parameters returning such an identifier.

bsp-why does not as of now offer specific tools to write logic assertions about subgroups. A possible extension would be to provide, in addition to the existing x<i> and <x> notations, the possibility of referring to a variable on a specific processor inside a subgroup, or as the array of the values taken inside a subgroup. In the abstract example of Fig. [START_REF] Appel | Separation Logic for Small-Step Cminor[END_REF].23, a set of processors S is computed using "eval" to know which processors will effectively run the same branch of the if. Then, for each sequential block, we do not run the code for every processor, instead restricting to the processors in S, in this example the processors that have effectively their "id" lower than 2. Now the synchronisation is performed as above but we also check if S and the communicator sub match to forbid a deadlock: one processor does not synchronise with other processors of the same group. This match is described below. Finally, for processors that are not in S we execute the else branch that is, for this example, for processors that have not their "id" less to 2. In fact, the bsp_pid(comm) now depends of the communicator and has thus not the same value every time. In this way, a "true processor" can change of "id" depending on the communicator.

(b) Main Idea of the Transformation

(c) New Pre-conditions to Primitives that Synchronise and Limitation

A New Pre-condition. It would be senseless to keep guarding the conditional statements as before, since it would only allow the synchronisation of all the processors. We already saw that in the bsp-why program, an additional argument is given to the synchronise primitive to tell which subgroup has to synchronise: the communicator. We thus need to verify in the execution of our why translated program that all the processors in the communicator are synchronising properly.

To do this, we now dynamically maintain a variable S during the execution of the translated program, that contains the set of the processors that are running the same branch of the code. To avoid deadlocks, for each bsp_sync(comm,S), we check that all the processors of a subgroup will synchronise at the same time: assert((∀i:int ∈S, comm[i] ⊆ S) and (∀i:int ∈S, ∀j:int ∈comm

[i], comm[i] = comm[j])).
That is bsp_sync (or every parameter with a synchronous effect) is now defined with this precondition. It ensures that it is called on a coherent set of processors at any time. For every processor in the set S, which is the set of the processors that will execute the call to bsp_sync, the subgroup that includes the processor is included in S. The subgroup of a processor i of S is denoted here by comm[i], since it is an information contained in the communicator argument. The second part of the assertion states that if one processor synchronises over a communicator, then all the other processors of the communicator synchronises on it too. In the actual implementation, the pre-condition is harder to read, since why lacks the clarity of mathematical formula:

(forall i:int. isproc(i) → paccess(S,i) = true → incl(paccess(comm,i),S)) and (forall i:int. isproc(i) → paccess(S,i) = true → forall j:int. isproc(j) → in_comm(j,paccess(comm,i)) → paccess(comm,i) = paccess(comm,j))

One might wonder why in the precondition of the sync statement, we write comm[i] ⊆ S, instead of for instance comm[i]=S. Let us consider again the simple example of Section 3.2. This is a program written in the pub library that we would like to model with bsp-why. To understand better the second part of the invariant, we can take a simple example. Let us consider 3 processors synchronising over the subgroups (1, 2), (2, 3) and (1, 3). Clearly, they would verify the first part of the assertion and still deadlock. It is thus mandatory to check that all the processors within a given subgroup are synchronising over this specific subgroup.

t_bsp subbsp; int part [START_REF] Aldinucci | MUSKEL: An Expandable Skeleton Environment[END_REF]; part[0] = 2; part [1] = bsp_nprocs(bsp); bsp_partition (bsp, &subbsp, 2, part); ... bsp_sync(&subbsp); . .

. bsp_done (&subbsp);

The same program, in bsp-why, would look like this: In this example, we do not have if statements to separate the behaviours of the disjoints subgroups. The sync operation is simultaneously executed by processors belonging to different groups. The important property is that inside every subgroups, all the processors do synchronise at the same time. It is true in this example, because the &subbsp are a partition of the bigger group.

In the precondition of the synchronisation, the variable S contains the set of the processors that execute the instruction. This variable is constructed dynamically by the why program, through additional instructions added during the transformation from bsp-why to why. We will present these modifications in the transformation when dealing with the subsynchronisation in the next section.

Limitation. So if one processor in a subgroup calls the synchronisation on that subgroup, every processors in the subgroup are executing it too. The restriction imposed by bsp-why is a bit more restrictive than what could be done in, for instance, the PUB library. But if a program can be executed normally in the bsp model, it is generally easy to transform it in an accepted bsp-why program. For example: let S = {0,1} in if pid=0 then computation 1 ; bsp_sync(S); computation 2 ; else if pid=1 then computation 3 ; bsp_sync(S); computation 4 ;

This program is correct in the bsp logic with subgroup synchronisation: the two processors of the subgroub S synchronise together, there is no deadlock. However, it is not a valid bsp-why program. The reason is that bsp-why sees that a sync operation is requested inside the if. To ensure that there can not be a deadlock, bsp-why asks that all the processors that enter the same branch of the if synchronise together. In the first branch, the processor 0 executes the sync, so all the processors that are with 0 in S must enter the same branch of the if conditional instruction. It is not true here (the processor 1 does not enter the same branch), so the pre-condition of the synchronisation will not be provable. In such cases, as previously, it is easy to rewrite the program, by factorising the synchronisation: One can easily see that the result of the execution will be the same as for the previous program. However, this program is accepted by bsp-why. The first if instruction has a synchronisation instruction inside a branch, so bsp-why will check at that synchronisation that all the processors of a group enters the branch before synchronising, which is trivially true. On the opposite, the inside if instruction does not lead to a synchronisation, so it is simply handled inside a sequential block as seen previously, without additional requirements. The slight loss of efficiency due to the new if statement does not matter since the goal here is only to formally prove the algorithm, not to get the fastest program.

let S = {0,

Transformation of Programs with the Subgroup Synchronisation

In the previous section, we presented parallel programs in the most usual bsp model, where all the processors of the parallel machine always synchronise together. The feature of subgroup synchronisation needs an extra treatment of the "sequentialisation" that is of the previous transformations.

Trivially, the first step of the transformation, the "decomposition into blocks", remains almost unchanged: tagging of the parallel parts of the code stay identical and subgroup do not change this fact. The major changes come with the next steps of the transformation, that is the second step ("tree transformation") and the third one ("local block transformation").

First, the pre-condition of primitives of synchronisation need to be treated depending of the subgroup: not all the processors synchronise. Second, the "for loop" would only deal with a subgroup of the processors and we must keep the fact that other processors have no work or a different one depending of the program. As we say above, the main idea is to maintain the subgroups and to generate an appropriate code depending of these subgroups and to check where the flow of instructions depends on these subgroups: where to branch of the if statement diverge depending of the subgroups.

(a) A Tree Transformation that Deals with Subgroups

New rules. The main difference in this transformation, compared to the previously defined transformation without subgroup synchronisation, is that the transformation [[t]], applied to the block tree, is now refined in a more precise transformation, [[t]] S , which means that the block tree t will be executed on the processors of the set S. This is useful to handle the if statement, as we have seen in the example, and the while statement is similar.

We also define a parameter that will be used to find what processors have to execute a given code in the case of a if statement. evalCond c S returns the subset of the processors of S for which the condition c holds. Note that the rules for the other structures are exactly the same as without sub-synchronisation with the exception that they now carry the S parameter. We now give the new rules of the transformation using these definitions. [

[if c 1 then c 2 else c 3 ]] S =⇒ let S 1 = evalCond c 1 S in [[c 2 ]] S1 ; [[c 3 ]] S\S1
Handling the Exceptions and the While Statement. As before, special care must be taken when considering the exception handling. Let us look again at the same example as in Section 4.2, the block decomposition of a simple try structure, with a if statement inside (Fig. 4.24).

The problem becomes apparent with the transformation of the if statement. With our new transformation, the generated code is supposed to be the succession of 3 steps:

1. we evaluate the condition (pid = 0), and the subset S of processors for which it is true ([0]); 2. we execute the then part of the code for all processors in S; 3. we execute the else part of the code for all processors not in S.

However, in this case, the step 2 will actually raise an exception, that is only caught outside of the scope of the if statement. Thus the whole step 3 would be ignored, which is clearly not what we want.

There are several approaches to solve this problem. We chose to implement one that is not the most general, without having an overly restrictive impact on the scope of accepted programs. If we think back about the issue when we did not have sub-synchronisation, the problem was solved by the valid parameter on the if condition, ensuring that all the processors would raise the exception at the same time. We extend this idea by now imposing a pre-condition to the raise statement itself: it must be executed by all the processors that initiated the matching try statement. Thus, the transformation rules for the try and raise statements are now as follows:

[[try c 1 with E x → c 2 end]] l S =⇒ try [[c 1 ]] (E,S)::l S with E x → [[c 2 ]] l S end [[raise (E c)]] (E,S)::l) S ′ =⇒ assert {S = S ′ }; raise (E [[c]] (E,S)::l) S ′ )
The transformation takes one more argument, a list of the exceptions that are being caught, and the subgroup of processors that executed the try. With this information, it is easy to add an proof obligation for the raise statement, to ensure that all the processors in the subgroup do raise the exception.

The principal user of exceptions is of course the while loop, and it is directly affected by this choice. In practice, the restriction is that for a while loop in the parallel structure of the program (while loops inside purely sequential calculations are still free), all the processors that execute the loop must exit it at the same time. This is true of many parallel programs with subgroup synchronisation, but not all of them.

It would be possible to give a more general transformation that accounts for any kind of exception handling imaginable. This would significantly complicate the transformation, however, and would be of little interest since the programs we study almost never use exceptions in this way. In practice, only the while limitation really could be felt. In the actual transformation, bsp-why does not use the loop expansion of the while statement, preferring to translate it directly in its while form to generate a more readable program in output. Because of this, it is possible to give two variants of the while transformation; one following the choice that we made for the exception handling, and one variant more complex, but allowing some processors to leave the loop before others.

[[while c 1 do c 2 {invariant i variant v}]] S =⇒ while valid([[c 1 ]] S , S) do [[c 2 ]] S {invariant [[i]] variant [[v]]} [[while c 1 do c 2 {invariant i variant v}]] S =⇒ S ′ := S ; while S ′ := evalCond c 1 S ′ do [[c 2 ]] S ′ {invariant [[i]] variant [[v]]}
The first option closely follows the previous transformation for the while loop: the valid parameter ensures that the condition remains true on every processor that executes the loop. The only difference is that now, the loop can be executed within a subgroup instead of all the processors. The invariant and variant are thus obtained in the same way as before.

In the second option, we enrich the loop by allowing processors to exit it while it progresses. Because of this, we need to update, at each iteration, the set of the processors that are currently executing the loop. This is done by introducing a variable S ′ , which is updated from the computation of the condition on each processor. While the invariant is still very much the same, the variant generation is a bit more tricky, since there is no way of knowing beforehand which processor will stay the longest in the loop. Instead, we need to provide some measure obtained from the variants on every processors, for instance their sum.

Because of the added complexity in the second option, both in code and with the proof obligations, we chose to provide the first option by default, with the possibility to request the generation of the more complex loop when necessary.

(b) A new Local Block Transformation that Deals with Subgroups

The idea of the transformation of a local block is similar to the one without subgroup synchronisation. However, instead of executing the block for all the processors successively, we only execute it for the processors that are running that part of the code. This is exactly what is denoted by the variable S. For this, we introduce a shortened notation: for i in S do c. The "for loop" means that we execute sequentially the instructions c for all the processors in S. This generates a code of the following form:

let i = ref 0 in while !i<nprocs do { invariant inv variant nprocs -i } if proc∈ i S then c; i← !i+1; done
where the procIn functions just tells if i is in the set S -the treatment of the invariant inv is described above. With this notation, our transformation is now the following:

[[Block(e)]] S = for i in S do [[e]] i,S

(c) A new Generation of Invariants

The determination of the invariant of "for loop" can be made very similarly to what we did before. The difference is that for the processors that are not in S, we always need the second form of invariant to hold. That is: ∀j:int. 0≤ j<i and j ∈S → post[j]. And, as above, we also need to express that the computation has not yet been processed for some processors. This is done using the two following invariants 1. ∀j:int. i≤ j<nprocs → v[j]=v[j]@loopstart, the computation has not been done for these processors; 2. ∀j:int. 0≤ j<i and ¬(j ∈S) → v[j]=v[j]@loopstart, processors not in S will never do the computation;

(d) Transformation of Local Code

When transforming local code from bsp-why to why, there are almost no modification compared to the standard bsp model. The main point here is that for synchronising parameters, we add the S argument containing the set of the processors that run the synchronisation. This is necessary to ensure the proper synchronisation. As explained before, the synchronisation parameters are defined with two arguments, the array of the communicators and the set of the processors that execute the synchronisation. Both arguments are then used to define the correct precondition.

Similarly, an argument is added for all of the function calls, when the function contains parallel code. This is needed to transmit the information of S inside the functions, which could be needed for a synchronisation.

4.4

Related Work

Other Verification Condition Generators

We can cite who [START_REF] Kanig | Who: A Verifier for Effectful Higher-order Programs[END_REF] [ [ 56 ]] which is a derivation of why for ml programming with polymorphic functions, and in the same spirit the pangolin system [START_REF] Régis-Gianas | A Hoare Logic for Call-by-Value Functional Programs[END_REF]. Both could be interesting for proofs of ml extensions of bsp [START_REF] Gava | Formal Proofs of Functional bsp Programs[END_REF]. But these tools are not yet stable enough to be used as basis for a parallel extension.

Close to why, we can cite the boogie programming language (with the spec # annotation language) [START_REF] Barnett | Specification and Verification: The Spec# Experience[END_REF] [ [ 57 ]]. The ynot System [START_REF] Nanevski | Ynot: Reasoning with the Awkward Squad[END_REF] is an extension to the coq proof assistant, able to reason about imperative higher-order programs, including functions with side effects as arguments, modular reasoning, while being able to reason about aliasing situations thanks to the separation logic.

Concurrent (Shared Memory) Programs

To our knowledge, the first work on deductive verification of concurrent programs is the famous Owicki-Gries proof system [START_REF] Owicki | Verifying Properties of Parallel Programs: An Axiomatic Approach[END_REF]. It is an extension of Hoare logic to parallel programs with shared-variable concurrency. It provides a methodology for breaking down correctness proofs into simpler pieces. First, the sequential components of the program are annotated with suitable assertions. Then, the proof reduces to showing that the annotation of each component is correct, and that each assertion of an annotation is invariant under the execution of the actions of the other components -the so-called interferencefreedom of proof outlines. The main drawback of the Owicki-Gries method is that it is not compositional: to perform the interference-freedom tests for some component once requires information about the implementation of all other components. Another drawback is the massive use of ghost codes (auxiliary variables) needed for proving parallel programs using the Owicki-Gries method. For example, the number of interference-freedom tests is polynomial to the number of sequential components. Note that a formal analysis of what is provable without these ghost codes has been done in [START_REF] Malkis | On the Strength of Owicki-Gries for Resources[END_REF].

In order to remedy to the above problems, different solutions have been proposed. In [START_REF] Nieto | Verification of Parallel Programs with the Owicki-Gries and Rely-Guarantee Methods in Isabelle/HOL[END_REF][START_REF] Nipkow | Owicki/Gries in Isabelle/HOL[END_REF], the authors proposed a compositional extension of the Owicki-Gries's rules, a wp-calculus, each implemented in the theorem prover isabelle. The authors also applied their method for verifying a concurrent garbage collector [START_REF] Nieto | Verifying Single and Multi-mutator Garbage Collectors with Owicki-Gries in Isabelle/HOL[END_REF]. Three advantages of the method are (1) the use of a theorem prover ensures a great confidence in the implementation; (2) the method allows verification of open systems, i.e. systems which interaction with the environment can be specified without knowing the precise implementation of the environment; this makes the method suitable for top-down design; (3) parametrised concurrent programs (programs with an unknown and unbound number of threads) can be directly verified in the system; this is achieved by modelling the parallel constructor such that its argument is a list of component programs; then, the length of the list can be fixed or left as a parameter. The main drawback (except that it only works for shared-memory programs) is still the number of generated conditions, that rely on "relyguarantee" which seems to be not provable by automatic provers: they require extensive human guidance.

Notice also the work of [START_REF] Gerth | A Proof System for Concurrent ada Programs[END_REF] where annotations are used to prove safety of concurrent ada programs, that is, they are deadlock free and without failure. Now, there are also some studies of proof obligations for concurrent programs; for example [START_REF] O'hearn | Resources, Concurrency and Local Reasoning[END_REF] presented a Concurrent Separation Logic as an extension of Separation Logic for reasoning about shared-memory concurrent programs with Dijkstra semaphores. [START_REF] Hobor | Oracle Semantics for Concurrent Separation Logic[END_REF] presents an operational semantics for Concurrent c minor which preserves as much sequentially as possible (coarse-grained spirit), by talking about permissions within a single thread instead of concurrency. This semantics is based on ideas from Concurrent Separation Logic: the resource invariant of each lock is an explicit part of the operational model. This model is well suited for correctness proofs done interactively in a proof assistant, or safety proofs done automatically by a shape analysis such as [START_REF] Gotsman | Thread-modular Shape Analysis[END_REF]. However, currently no tools for generating proof obligations are provided and it is not clear how hard the obligations would be. A first work in this direction has been done in [START_REF] Bobot | Separation Predicates: a Taste of Separation Logic in First-Order Logic[END_REF].

[41] presents a type system that prevents data races and deadlocks (as in [START_REF] Hobor | Oracle Semantics for Concurrent Separation Logic[END_REF]) by enforcing that locks are acquired in a given locking order, and this order can be changed dynamically. This system supports thread-local objects and coarse-grained locking of shared objects. The type system allows concurrent reading only for immutable objects.

In the same way, [START_REF] Leino | A Basis for Verifying Multi-threaded Programs[END_REF] presents a sound and modular verification methodology (implemented for an experimental language with some not-trivial examples) that can handle advanced concurrency patterns in multi-threaded, object-based programs. It prescribes the generation of verification conditions in first-order logic (well-suited for solvers). The language supports concepts such as multi-object monitor invariants, thread-local and shared objects, thread pre-and post-conditions, and deadlock prevention with a dynamically changeable locking order. [START_REF] Jacobs | A Statically Verifiable Programming Model for Concurrent Objectoriented Programs[END_REF] extends the boogie framework for concurrent programs with a kind of locking strategy. In the same way, a vcg for concurrent c programs has been designed in [START_REF] Cohen | VCC: A Practical System for Verifying Concurrent C[END_REF] [ [ 58 ]]. This vcg has been used for the development of the Microsoft Hyper-V hypervisor -see Chapter 7 for a discussion on hypervisors. The hypervisor is highly optimised for multi-core hardware and thus contains a number of custom concurrency control mechanisms and algorithms, mostly using fine-grained concurrency control. The model, with uniform treatment of objects and threads, is very similar to the one employed in Concurrent spec # and chalice [START_REF] Leino | Verifying Concurrent Programs with Chalice[END_REF]. There is also the verifast tool [START_REF] Jacobs | VeriFast: A Powerful, Sound, Predictable, Fast Verifier for C and java[END_REF][START_REF] Philippaerts | Software Verification with VeriFast: Industrial Case Studies[END_REF] for deductive verification of c and java programs using separation logic [[ 59 ]]. Other vcg for separation logic are describe in [START_REF] Philippaerts | Software Verification with VeriFast: Industrial Case Studies[END_REF]. All these tools are well defined for share-memory concurrency but not for distributed memory and hpc computing.

It is not clear if locking strategies are very suitable for high-performance applications [START_REF] Lee | The Problem with Threads[END_REF]. bsp is by nature a coarse-grained and deadlock-free model which is used for high-performance problems and now in multi-core/gpu applications. Even if proof of concurrent programs is clearly useful (servers, etc.), parallel programming is not concurrent programming. High-performance programs are much simpler [START_REF] Lee | The Problem with Threads[END_REF] (many time more coarse-grained) and bsp programs are even simpler. They can clearly be simulated by share-memory fork/lock concurrent programs by explicitly separating the local memories and allowing communications with copies of the data [START_REF] Tiskin | The Design and Analysis of Bulk-Synchronous Parallel Algorithms[END_REF]. Global synchronisation would be implemented using a sufficient number of locks. But, that would not use the structural nature of the bsp programs and the understanding of the program to simplify the obligations.

Distributed and MPI Programs

A first work on distributed programs was done in [START_REF] Lamport | The Hoare Logic of CSP, and All That[END_REF]. It has served as a basis for the tla + logic. But no work or a tool for vcg calculus was proposed.

In [START_REF] Daum | Reasoning on Data-Parallel Programs in Isabelle/Hol[END_REF], the author proposes a data-parallel (simd, Single Instruction Multi Data) extension of c and its embedding into a verification environment based on isabelle. The data-parallel operations are mostly binary like operations on arrays. Another work on data-parallel programs is the one of [START_REF] Bougé | Formal Validation of Data-Parallel Programs: a Two-Component Assertional Proof System for a Simple Language[END_REF]. The authors have designed a set of Hoare's rules and a wp-calculus for proving data-parallel programs. Exchange of data is done using shared arrays. The two drawbacks are (1) there is a lack of mechanised proofs; (2) the programs need a lot of annotations (not only loop invariants) which can repel a user; (3) the method does not hold if a program modifies the variables in the assertions (a data-race). For the latter, auxiliary variables can be used but make the proofs slightly more complex.

mpi is the most used library for high-performance computing. It is therefore natural to study safety issues related to mpi programs. But this is very challenging due to the number of routines (more than one hundred), the concurrent nature of these primitives, the lack of formal specifications -even if works such as [START_REF] Li | Formal Specification of the mpi-2.0 Standard in TLA+[END_REF][START_REF] Vakkalanka | Reduced Execution Semantics of mpi: From Theory to Practice[END_REF] exist, some cases are not taken into account because of the lack of specification and too much dependence on the architecture. This enormous number of routines in the api makes difficult to trust any formal specification of a complete mpie.g., a non-trivial case could have been forgotten?

They are many works and tools dedicated to mpi. Surveys could be found in [START_REF] Desouza | Automated, Scalable Debugging of mpi programs with Intel R Message Checker[END_REF][START_REF] Gopalakrishnan | Formal Analysis of mpi-based Parallel Programs: Present and Future[END_REF][START_REF] Krammer | MARMOT: An mpi Analysis and Checking Tool[END_REF][START_REF] Luecke | mpi-CHECK: a tool for checking Fortran 90 mpi programs[END_REF][START_REF] Sharma | A Survey of mpi Related Debuggers and Tools[END_REF][START_REF] Vetter | Dynamic Software Testing of mpi Applications with Umpire[END_REF]] and in hpcbugdatabase [[ 60 ]]. These tools help to find some classical errors, but not all of them. For example, in [START_REF] Traeff | Verifying Collective mpi Calls[END_REF], the authors propose to check dynamically if the parameters of collective operators are consistente.g. if the size of data to send is non-negative. Note that this kind of tools works well for many situations in development phases, but is not sufficient.

[283] presents a tool that directly model-check the mpi source code, executing its interleaving with the help of a verification scheduler (producing error traces of deadlocks and assertion violations). It allows an engineer to check its mpi code against deadlocks on a p-processors machine before running it on this machine. A large number of mpi routines (including drma ones [START_REF] Pervez | Formal Verification of Programs that use mpi One-sided Communication[END_REF]) are considered. This technique was also used to find irrelevant barriers [START_REF] Sharma | A Formal Approach to Detect Functionally Irrelevant Barriers in mpi Programs[END_REF]. The main idea (which is also the one of [START_REF] Siegel | Verifying Parallel Programs with mpi-Spin[END_REF][START_REF] Siegel | Combining Symbolic Execution with Model Checking to Verify Parallel Numerical Programs[END_REF]) is to analyse the code by abstract interpretation and to produce an output for model-checkers by eliminating all parts of the code which are not in the communication schemes. First implementations used the model-checker spin, but now specific and more efficient checkers are considered. The main advantage of these methods is to be "push-button". The two main drawbacks are (1) they only consider deadlocks and assertion violations (it is still an important problem for mpi programs) (2) programs are model-checked for a predefined number of processors which is less than 64 in [START_REF] Vo | Formal Verification of Practical mpi Programs[END_REF]. That is clearly not a scalable approach. The bsp spirit is to write programs for any number of processors, so proofs of bsp programs should do the same thing. A last problem with model checking parallel programs is state explosion, i.e. the fact that the number of states of a program typically can grow dramatically with the number of processes.

As said above, these methods generally cannot scale beyond a relatively small number of processes, but defects, which usually appear only in large configurations, can often be detected in much smaller configurations using symbolic execution. This original solution is the tass [START_REF] Siegel | TASS: The Toolkit for Accurate Scientific Software[END_REF] [ [ 61 ]] and fevs tools [254] [[ 62 ]]. tass uses symbolic execution and explicit state enumeration techniques to verify that safety properties of mpi programs hold for all possible executions within user-specified bounds. tass has its own internal support for simplifying symbolic expressions, placing them into a canonical form, and dispatching some of the proof obligations; for those it cannot dispatch itself, it uses cvc3. tass can also use comparative symbolic execution (in the spirit of fevs) to verify that two programs are functionally equivalent (i.e. "input-output equivalent"). The basic idea is to construct a model in which the two programs run sequentially, one after the other, and at the end the outputs are compared. The state space of this model is then exhaustively explored. This technique is particularly useful in computational science, to compare a complex parallel version of a program (the "implementation") to a simple, trusted sequential version (the "specification"). Techniques to verify program assertions using symbolic execution exhibit a significant limitation: they typically require to impose (small) bounds be imposed on the number of loop iterations. The use of loop invariants allows to overcome this limitation. In [START_REF] Siegel | Loop Invariant Symbolic Execution for Parallel Programs[END_REF], the author proposes a solution using a new symbolic execution technique (with special "collective loop invariants") that he uses to verify assertions in mpi programs with unbounded loops. Programs are then checked (a model-checking technique using POR reductions) for unknown sizes of data -but still for a fixed number of processor only. But the author note that discovering these collective loop invariants is currently to the charge of the programmer (as in our work) and these invariants are limited; for example, there is no way to express that the number of messages is invariant. The method still has the advantage of greatly reducing the number of necessary loop invariants.

The approaches of symbolic verification as well as vcg tools, suffer to the main limitation: as it now stands, models of the programs must be built by hand. This requires significant effort and a degree of skill from the user. The ideal situation would be to have tools that automatically extract the models from source code, at least for specific domains [START_REF] Furia | Inferring Loop Invariants Using Postconditions[END_REF][START_REF] Siegel | Combining Symbolic Execution with Model Checking to Verify Parallel Numerical Programs[END_REF].

Currently, we are either not aware of verification condition generators tools for mpi programs. We think that performing a sequential simulation (as done by our bsp-why tool) of any kind of mpi programs is not reasonable. Continuations would be necessary to simulate the interleaving of processes: that would generate unintelligible assertions. But collective mpi routines, which simplifies the life of parallel programming [START_REF] Squyres | Zen and the Art of mpi Collectives[END_REF], can be seen as bsp programs, and certainly many mpi programs could be transformed into bsp ones. Automatically translating this class of programs is a possible way to analyse mpi programs [START_REF] Martino | Parallel Program Analysis and Restructuring by Detection of Point-to-point Interaction Patterns and their Transformation into Collective Communication Constructs[END_REF]. We leave the aim of substantiating this claim for future work.

The only exception is the work of [START_REF] Villard | Heaps and Hops[END_REF][START_REF] Villard | Proving Copyless Message Passing[END_REF] with their prototype tool call heap-hop [[ 63 ]]. heap-hop is a vcg for programs with synchronous and copyless sending. A concurrent separation logic is used for the assertions (pre-and post-conditions and loop invariants). As an example, the authors propose to verify a load balancing algorithm for binary trees for two processes. This tool is thus able to take into account some mpi programs. But it is still limited to a fixed number of processese.g. two for the load balancing algorithm of a consumer/producer algorithm. Using bsp-why, the user can prove its programs for any number of processors.

Proof of BSP Programs

Different approaches for proofs of bsp programs have been studied. In [START_REF] Gava | Formal Proofs of Functional bsp Programs[END_REF][START_REF] Gesbert | Systematic Development of Correct Bulk Synchronous Parallel Programs[END_REF][START_REF] Tesson | A Verified Bulk Synchronous Parallel ml Heat Diffusion Simulation[END_REF], functional bsp programs have been proved correct using Coq. The bsp primitives of extension of ML call BSML were axiomatised in Coq. In this way, one can use Coq for programming bsp algorithms and BSML programs can be extracted from Coq in the form of OCaml code. In [START_REF] Gava | Formal Semantics of a Subset of the Paderborn's bsplib[END_REF], we presented the correctness of a classical numerical computation (the N-body problem) using a mechanised operational semantics. But, by using semantics inside coq, proofs of correctness were too hard.

The derivation of imperative bsp programs using the Hoare's axiom semantics [START_REF] Chen | Logic of Global Synchrony[END_REF] followed by the generation of correct c code [START_REF] Zhou | Generating C Code from LOGS Specifications[END_REF] also exists. The two main drawbacks of this approach is a lack of an implementation of a dedicated tool for the logical derivation, which implies a lack of safety; users make hand proofs which are not machine checked; moreover, it is impossible to verified users existing codes.

The derivation of imperative bsp programs using the Hoare's axiom semantics has also been studied in [START_REF] Chen | Top-Down Design of Bulk-Synchronous Parallel Programs[END_REF][START_REF] Jifeng | Algebraic Laws for bsp Programming[END_REF][START_REF] Stewart | Axiomatic Frameworks for Developing BSP-Style Programs[END_REF][START_REF] Stewart | Barrier Synchronisation: Axiomatisation and Relaxation[END_REF]. More recently, these works were extended for subgroup synchronisation in [START_REF] Stewart | A Programming Model for bsp with Partitioned Synchronisation[END_REF]. All of these approaches lack of mechanised proofs. Moreover, they are close to refinement "à la b" since they give logical rules (close to Hoare logic's axioms and inference rules) for derived algorithms from specifications -a wp calculus is also given in [START_REF] Stewart | A Programming Model for bsp with Partitioned Synchronisation[END_REF]. On the contrary, using deductive verification, we begin with a program and by adding logical assertions, we prove the correctness of the said program.

In this chapter, we first give some simple examples of use of bsp-why and mainly show how many proof obligations are discharged by automatic provers. Then, we focus on using bsp-why, for the verification of state-space construction algorithms which is the basis of model-checking. We mainly study three algorithms (one sequential, one distributed and another one dedicated to the state-space of security protocols) as a first step towards mechanically-assisted deductive verification of model-checkers. Because of a lack of time, we are currently not able to provide an example that uses the subgroup syncronisation capabilities of bsp-why.

Simple BSP Algorithms

All codes which are not given in the thesis are available as examples at the bsp-why web page 1 . In this section, we first present two examples: parallel prefix calculations (Section 5. 1.1), and a parallel sorting algorithm (Section 5.1.2). We then give the results obtained with automatic provers in Section 5.1.3.

Parallel Prefix Reductions (a) Direct Method

Our first example is a simple one-step parallel prefix reduction, that is obtaining the ⊕ k i=0 v i on each processor k, with processor i initially holding v i (this is the classical MPI_SCAN) for an operation ⊕.

Here, we used integers and addition for ⊕ but a polymorphic program can be considered. Using bsmp routines, we can give the bsp-why code of Fig. 5.1 (left).

The program starts with a distributed parameter x (with the notation parameterg), which contains the initial values, with one value on each processor. The prefixes are computed by the program in the z parameter. We use the user-defined logic term sigma prefix(X,n 1 ,n 2 ) to describe the partial sums, that is Σ n2 i=n1 X[i]. The programs is mainly composed of two while loops. In the first loop, each processor sends its value in a message to each processor with a greater pid than itself. The instruction bsp_sync

Mechanical Proof

In this table, we can show how many verification conditions are generated for the above examples. We also show this number when no assertions are given for the correctness of the programs (it is just to have safe execution of the programs without buffer overflow or out-of-bound read of messages etc.). We also show (noted AP) the number of obligations that are automatically discharged by automatic procedures. We used the following automatic provers: alt-ergo (0.9), simplify (1. 5.4) For a simple example such as the direct prefix, all the proof obligations are automatically discharged (proved) by automatic provers. For more complex examples, a few proof obligations are not automatically discharged yet. But safety (no deadlock, no buffer overflow, no out-of-the-bound sending messages etc.) is automatically ensured for all examples (except the log prefix) which is an interesting first result.

Not having all the properties given automatically is sad since the generated proof obligations are generally hard to read for why and even more for bsp-why: this is due to the use of loops over p for local computations. That also generated harder proof obligations for the provers. Furthermore, automatic provers are also a work in progress. For example, logarithm's (and power-of-two) properties are not currently well interpreted by any automatic prover and thus they fail to prove check bounds accesses in a logarithmic loop.

The key to evaluating the promise of a translation-based technique is in studying the effort needed to prove the generated proof obligations. Currently, many of them are automatically proved and it is thus an encouraging result regarding that it also happens for sequential computations and that our work is to our knowledge the first of its kind. Since these examples are not too difficult, we also believe that by giving more axioms (e.g. for log, sqrt, sort, etc. which are currently given to the minimum in the why library) all the proof obligations can be automatically proved. Furthermore, the complete proof of the proof obligations generated for these examples is still a work in progress.

Parallel State-space Construction

As any software, model-checkers are subject to bugs. They can thus report false negatives or validate a model that they should not. Different methods, such as theorem provers or Proof-Carrying Code [START_REF] Necula | Proof-Carrying Code[END_REF] (pcc for short) have been used to gain more confidence in the results of model-checkers. In this section, we focus on using the verification condition generator why to study two generic2 algorithms (one sequential and one distributed) of state-space construction and one distributed and dedicated to the state-space of security protocols. This work is a first step towards mechanically-assisted deductive verification of model-checkers.

Motivations and Background

Model-checkers (mcs for short) are often used to verify safety-critical systems. The correctness of their answers is thus vital: many mcs produce the answer "yes" or generate a counterexample computation (if a property of the model fails), which forces, in the two cases, to assume that the algorithm and its implementation are both correct. But mcs, like any software are subject to bugs and there exist surprisingly few attempts to prove them correct. Three main reasons can explain this fact [START_REF] Namjoshi | Certifying Model Checkers[END_REF]: (1) mcs involve complicated logics, algorithms and sophisticated state reduction techniques; (2) because efficiency is essential, mcs are often highly optimised, which implies that they may not be designed to be proved correct; (3) mcs are often updated. But there is a more and more pressing need from the industrial community, as well as from national authorities, to get not just a boolean answer, but also a formal proof -which could be checked by an established tool such as the theorem prover Coq. This is required in Common Criteria certification of computer products at the highest assurance level eal 7http://www.commoncriteriaportal.org/. And hand proofs are not sufficient for eal 7, mechanical proofs are needed. The author of [START_REF] Shankar | Trust and Automation in Verification Tools[END_REF] summarizes the problem: Quis custodiet ipsos custodes ? (Who will watch the watchmen? that is, who will verify the verifier?). We want to be able to trust the results of model-checkers with a high degree of confidence.

(a) Different Solutions for Verifying Model-checkers

For verifying model-checkers, different solutions have been proposed. The first one is to prove mcs inside theorem provers and use the extraction facilities to get pure functional machine-checked programs such as in the works of [START_REF] Sprenger | A Verified Model Checker for the Modal µ-calculus in coq[END_REF] and [START_REF] Esparza | A Fully Verified Executable ltl Model Checker[END_REF]. The second and more common approach, in the spirit of pcc, is to generate a "certificate" during the execution of the mc that can be checked later or on-the-fly by a dedicated tool or a theorem prover. This is the so-called "certifying model-checking" [START_REF] Namjoshi | Certifying Model Checkers[END_REF]. In this way, users can reexecute the certificate/trace and have some safety guarantees because even if the mc is buggy, its results can be checked by a trustworthy tool.

But, any explicit mc may enumerate a very large state-space (the famous state-space explosion problem), and mimicking this enumeration with proof rules inside any theorem prover (or with pccs) would be foolish, even if specific techniques and optimisations of the abstract machine of theorem provers [START_REF] Armand | Extending coq with Imperative Features and Its Application to SAT Verification[END_REF] are used. Note that this problem does not arise when finding a refutation of the logical formula (the trace is generally short) but when the answer is "yes" since the entire explicit state-space (or at least a symbolic representation) needs to verify the checked properties. In this way, certificate generation could also hamstring both the functionality and the efficiency of the automation that can be built from theorem provers (functional programs can be too memory consuming) and pcc tools (too big certificates) [START_REF] Shankar | Trust and Automation in Verification Tools[END_REF]. Because model-checking can cause memory crashing on single or multiple processor systems, it has led to consider exploiting the larger memory space available in distributed systems [START_REF] Garavel | Parallel State Space Construction for Model-Checking[END_REF], which also gives the opportunity to reduce the overall execution time. Paralleling the state-space construction on several machines is thus done in order to benefit from each machine's complete storage and computing resources. Only efficient, imperative and distributed programs can override the state-space explosion problem.

And mcs, especially distributed ones [START_REF] Garavel | Parallel State Space Construction for Model-Checking[END_REF], like any complex software are subject to bugs. And generating distributed certificates to be later machine-checked using a theorem prover is thus currently not reasonable since provers are critical softwares that can not be altered without much attention. For this purpose, we proposed to prove the correctness of distributed mcs themselves and not their results as "certifying mc" [START_REF] Namjoshi | Certifying Model Checkers[END_REF] generally does.

Another solution, proposed in [START_REF] Sun | Model Checking a Model Checker: A Code Contract Combined Approach[END_REF] for a mc call pat, is to use coding assumptions directly in the source code. They indeed use Spec# and a check of the object invariants (the contracts) is generated. Nevertheless, they cannot completely verify the correctness of pat and they thus focus on some safety properties (as no overflows, no deadlocks) of the underlying data structures of pat (which can run on a multi-core architecture) and check if some options may conflict with each other.

(b) The Proposed Solution

Our contribution follows the approach of [START_REF] Sun | Model Checking a Model Checker: A Code Contract Combined Approach[END_REF] but by using why and by extending the verification to the correctness of the final result: has the full state-space been well computed without adding unknown states?

Since the why-ml language is not immediately executable but a higher-level algorithmic language, we only focus on algorithms. We can thus focus on which formal properties need to be preserved and not be obstructed by problems specific to a particular programming language. Even if most of the bugs in mcs will not be due to wrong algorithms but rather due to subtle errors in the implementation of some complex data structures and bad interactions between these structures and compression aspects, we must first check the algorithms to get an idea of the amount of work necessary to verify a true model-checker.

Our goal is then a mechanically-assisted proof that these annotated algorithms terminate and indeed compute the expected finite state-space. This is an interesting first step before verifying mcs themselves: it allows to test if this approach is doable or not. This is also challenging due to the nature of modelchecking (critical system) and to the algorithmic complexity. The main contribution of this section is to demonstrate the ability of a vcg such as why to tackle the wide range of verification issues involved in the proof of correctness of imperative codes of mcs.

(c) Application to Security Protocols

Security protocols are small and standard components of systems that communicate over untrusted networks. Their relatively small size, combined with their critical role, makes them a suitable target for formal analysis [START_REF] Comon-Lundh | How to Prove Security of Communication Protocols? A Discussion on the Soundness of Formal Models w.r.t. Computational Ones[END_REF]. But verifying secure protocols is a challenging problem. In spite of their apparent simplicity, they are notoriously error-prone. Attacks exploit weaknesses in the protocol that are due to the complex and unexpected interleaving of different protocol sessions generated by an intruder (malicious) which resides in the network. The intruder is assumed to have complete network control and to be powerful enough to perform potentially dangerous actions such as intercepting messages flowing over the network, or replacing them by new ones using the knowledge he has previously gained [START_REF] Dolev | On the Security of Public Key Protocols[END_REF]. Unfortunately, the question of whether a protocol achieves its security requirements or not is, in the general case, undecidable [START_REF] Durgin | Undecidability of Bounded Security Protocols[END_REF] or NP-complet in case of bounded number of agents [START_REF] Rusinowith | Protocol Insecurity with Finite Number of Sessions is NP-complete[END_REF].

Model-checking (mc) is common solution to find flaws [START_REF] Armando | ltl Model Checking for Security Protocols[END_REF]. By focusing on verification of a bounded number of sessions, mc of a protocol can be done by simply enumerating and exploring all traces of the execution of the protocol and looking for a violation of some of the requirements. Verification through model-checking thus consists in defining a formal model of the system to be analysed and then using automated tools to check whether the expected properties are met or not on the state-space of the model that is having compute all the different configurations of the execution of the agents evolving in the protocol. Ideally, we would also like to have a proof of the protocol's correctness or of the attack found: generate a "certificate" [START_REF] Namjoshi | Certifying Model Checkers[END_REF] that can be checked later by a trusted theorem prover (e.g. coq).

But the greatest problem with explicit model-checking in general (and for security protocols in particular) is the so-called state explosion: the fact that the number of states typically grows dramatically with the number of agents. This is especially true when complex data-structures are used in the model such as the knowledge of an intruder in a security protocol. The same problem applies for the generation of large discrete state spaces of some non traditional protocols [START_REF] Sanjabi | Modelling, Verification, and Formal Analysis of Security Properties in a P2P System[END_REF] (especially when complex data-structures are used by the agents such as lists of trusted servers etc.)

As said above, we will use deductive verification of state-space algorithms. For security protocols, we will verify a specialised distributed algorithm for security protocols designed in [START_REF] Gava | A bsp Algorithm for the State Space Construction of Security Protocols[END_REF].

(d) Definition of Security Protocols

Security protocols specify an exchange of cryptographic messages between principals, i.e. the agents (e.g., users or servers) participating in the protocol. Messages are sent over open networks, such as the Internet, that are not secured. As a consequence, protocols should be designed to work fine even if messages may be eavesdropped or tampered with by an intrudere.g., a dishonest or careless agent. Each protocol is aimed to provide security guarantees, such as authentication or secrecy of some pieces of information.

We assume the use of keys sufficiently long of the best-known cryptographic algorithms to prevent a brute force attack in a reasonable time. This is the well-known perfect cryptography assumption. The idea is that an encrypted message can be decrypted only by using the appropriate decryption key, i.e., it is possible to retrieve M from the message encrypted {M } K only by using K -1 as decryption key and it is hopeless to compute K -1 from K.

The abilities of the attackers and relationship between participants and attackers together constitute a threat model and the almost exclusively used threat model is the one proposed by Dolev and Yao [START_REF] Dolev | On the Security of Public Key Protocols[END_REF]. The Dolev/Yao threat model is a worst-case model in the sense that the network, over which the participants communicate (agents perform "ping-pong" data exchanges), is thought as being totally controlled by an omnipotent intruder. In this thesis, we thus consider a Dolev/Yao attacker that resides on the network. An execution of such a model is thus a series of message exchanges as follows. (1) An agent sends a message on the network. [START_REF] Aldinucci | MUSKEL: An Expandable Skeleton Environment[END_REF] This message is captured by the attacker that tries to learn from it by recursively decomposing the message or decrypting it when the key to do so is known. Then, the attacker forges all possible messages from newly as well as previously learnt information (i.e., attacker's knowledge). Finally, these messages (including the original one) are made available on the network. (3) The agents waiting for a message reception accept some of the messages forged by the attacker, according to the protocol rules.

Definitions and Verification of a Sequential State-space Algorithm

We now describe how we model the state-space, and present the verification of a well-known algorithm. The annotated source codes are available at http://lacl.fr/gava/cert-mc.tar.gz.

(a) Definition of the Finite State-space

Let us recall that the finite state-space construction problem is computing the explicit graph representation (also known as Kripke structure) of a given model from the implicit one. This graph is constructed by exploring all the states reachable through a successor function succ (which returns a set of states) from an initial state s 0 . Generally, during this operation, all the explored states must be kept in memory in order to avoid multiple explorations of a same state.

In this thesis, all algorithms only compute the state-space, noted StSpace. This is done without loss of generality and it is a trivial extension to compute the full Kripke structure -usually preferred for checking temporal logic formulas. To represent StSpace in the logic of why, we used the following axiom contain_state_space (for consistency, it has been proved in coq using an inductive definition of the state-space, also available in the source code): i.e. defines which sets can contain the state-space. Now ss is the state-space (ss=StSpace) if and only if, the two following properties holds: (A) ss ⊆ StSpace and (B) StSpace ⊆ ss; that is equality of sets using extensionality. Note that using this first-order definition makes the automatic (mainly SMT) solvers prove more proof obligations than using an inductive definition for the state-space.

(b) Sequential Algorithms for State-space Construction

Fig. 5.4 gives the common sequential (random walk) algorithm in why-ml using an appropriate syntax for set operations. In left, without adding proof annotations for a best reading. All computations in this algorithm are set operations where a set called known contains all the states that have been processed and would finally contain StSpace. It involves a set of states todo that is used to hold all the states whose successors have not been constructed yet; each state s from todo is processed in turn (lines 4-5) and added to known (line 6) while its successors are added to todo unless they are known already -line 7. Note this algorithm can be made strictly depth-first by choosing the most-recently discovered state (i.e. todo as a stack), and breadth-first by choosing the least-recently discovered state. This has not been studied here.

For correctness, all our algorithms need three properties: (1) they do not fail (no rule of reduction) ≡ safety; (2) they indeed compute the state-space; (3) and they terminate. The first property is immediate for the sequential algorithm since the only operation that could fail is pick (where the precondition is "not take any element from an empty set") and this is assured by the guard of the while loop. The four invariants are: (1) known and todo are subsets of StSpace; at the end, (3) and ( 4) known is a subset of StSpace and has the "same" inductive property; and when todo will be empty, then known contains StSpace -property (B).

The termination is ensured by the following variant: |StSpace \ known| since, at each iteration of the loop, the algorithm only adds a new state s because (known ∩ todo)=∅.

All the obligations produced by the vcg of why are automatically discharged by a combination of automatic provers. For each prover, we give a timeout of 10 seconds -otherwise some obligations are not proved. In the following table, we give the number of generated obligations (column Total) and then how many are discharged by the provers: One could notice that the smt solvers simplify and z3 give the best results. In practice, we mostly used them. simplify is the faster and z3 sometime verified some obligations that had not be discharged by simplify. We have worked with the provers as black-boxes, without detailed knowledge of their inner working, and we have thus no explanation for this fact. Proof obligations are as expected when working with a vcg such as why.

let naive_state_space () = let total = ref 1 in let known = ref ∅ in let todo = ref ∅ in let pastsend = ref ∅ in if cpu(s0) =

Verification of a Generic Distributed State-space Algorithm

Parallelize the construction of the state-space on several machines is a standard method [START_REF] Garavel | Parallel State Space Construction for Model-Checking[END_REF]. In this section, we give an example of how to verify a generic (in the sense of independent of the function of successor succ) distributed (bsp) algorithm and show that it is more challenging but feasible.

(a) A Generic BSP Algorithm for State-space Construction

The above algorithm can be parallelised using a partition function cpu (a hashing) that returns for each state a processor id, i.e., the processor numbered cpu(s) is the owner of s: logic cpu: state → int and axiom cpu_range: ∀s:state. 0≤ cpu(s)<nprocs

The idea is that each process computes the successors for only the states it owns. This is rendered as the bsp algorithm of Fig. 5. [START_REF] Armand | Extending coq with Imperative Features and Its Application to SAT Verification[END_REF]. This is thus a spmd algorithm so that each processor executes it. Initially, only state s 0 is known and only its owner puts it in its todo set. This is performed in lines 6-7, where bsp_pid evaluates locally to each processor to its own identifier.

Sets known and todo are still used but become local to each processor and thus provide only a partial view on the ongoing computation.

Function local_successors computes the successors of the states in todo where each computed state that is not owned by the local processor is recorded in the array (of size nprocs) of sets of states tosend depending of its owner number. The set pastsend contains all the states that have been sent during the past super-steps -the past exchanges. This prevents returning a state already sent by the processor: this feature is not necessary for correctness and consumes more memory but it is generally more efficient mostly when the state-space contains many cycles.

Then, function exchange is responsible for performing the actual communications between processors. It assigns to todo the set of received states that are not yet known locally together with the new value of total. The primitive BSP_EXCHANGE performs a global (collective) synchronisation barrier which makes data available for the next super-step so that all the processors are now synchronised. The synchronous routine BSP_EXCHANGE sends each state s from the set tosend[i] to the processor i and returns the set of states received from the other processors, together with the total number of exchanged states -it is mainly the mpi's alltoall primitive. Notice that, by postponing communication, this function allows buffered sending and forbids sending several times the same state. More formally, at processor mypid:

BSP_EXCHANGE(tosend) = total = nprocs-1 k=0 nprocs-1 i=0 |tosend[[k]][i]| rcv = nprocs-1 i=0 tosend[[i]][mypid]
where tosend [[i]] represents the array tosend at processor i.

To ensure termination of the algorithm, we use the additional variable total in which we count the total number of sent states. We have thus not used any complicated methods as the ones presented in [START_REF] Barnat | Distributed Memory ltl Model Checking[END_REF]. It can be noted that the value of total may be greater than the intended count of states in todo sets. Indeed, it may happen that two processors compute a same state owned by a third processor, in which case two states are exchanged but then only one is kept upon reception. In the worst case, the termination requires one more super-step during which all the processors will process an empty todo, resulting in an empty exchange and thus total=0 on every processor, yielding the termination.

(b) Verification of this BSP Algorithm

For care of brevity, we do not present exchange which is technical and without really interesting properties and still available in the source code: the exchange procedure is only a permutation of the states that is, from a global point of view, only states in arrays have moved and there is no loss of states and a state has not magically appeared during the communications. Fig. 5.6 gives the annotated main parallel loop of the algorithm. We also use the following predicates:

• isproc(i) is defined what is a valid processor id that is 0≤ i<nprocs;

• (<p_set>) is the union of the sets of the p-value p_set that is p i=0 p_set<i>;

• GoodPart(<p_set>) is used to indicate that each processor only contains the states it owns that is ∀i:int. isproc(i) → ∀s:state. s ∈p_set<i> → cpu(s)=i;

As before, we need to prove that (1) the code does not fail; (2) indeed computes the entire state-space and (3) terminates. The first property follows immediately since only the routine pick is used as before; and to also prove that the code is deadlock free (the loop contains exchange which implies a global synchronisation of all the processors), we can maintain that total (which gives the condition for termination) has the same value on all the processors during the entire execution of the algorithm. Let us now focus on the two other properties.

Correctness of the Main Parallel Loop (Fig. 5.6). The invariants (lines 9 -18) of the main parallel loop work as follows: (1) as in sequential algorithm, we need to maintain that known (even distributed) is a subset of StSpace which finally ensures (A) when todo is empty; (2) as usual, the states to be treated are not already known; (3) our sets are well distributed (there is no duplicate state that is, each state is only kept in a unique processor); (4) total is a global variable, we thus ensure that it has the same value on each processor; [START_REF] Armand | Extending coq with Imperative Features and Its Application to SAT Verification[END_REF] ensures that no state remains in todo (to be processed) when leaving the loop since total is at least as big as the cardinality of todo, total is an over-approximation of the number of sent states; [START_REF] Armando | ltl Model Checking for Security Protocols[END_REF][START_REF] Armando | SAT-based Model-checking for Security Protocols Analysis[END_REF][START_REF] Armando | The AVISPA Tool for the Automated Validation of Internet Security Protocols and Applications[END_REF], as usual, ensure property (B); (9) past sending states are in the state-space; (10) pastsend only contains states that are not owned by the processor and (11) all these states, that were sent, are finally received and stored by a processor.

In the post-condition (line 26), we can also ensures that the result is well distributed: the state-space is complete and each processor only contains the states it owns according to the partition function cpu.

Termination of the Main Parallel Loop (Fig. 5.6). The main loop is more subtle: total is an overapproximation and thus could be greater to 0 whereas todo is empty. This happens when all the received states are already in known. The termination has thus two cases: (a) in general the set known globally (that is, from a global point of view, of all processors) grows and we have thus the cardinality of StSpace minus known which is strictly decreasing; (b) if there is no state in any todo of a processor (case of the last super-step), no new states would be computed and thus total would be equal to 0 in the last stage of the main loop.

We thus used a lexicographic order (this is well-founded ensuring termination) on the two values: sum of known across all processors; and total (which is the same on all processors) when no new states are computed and thus when no state would be sent during the next super-step. At least, one processor cannot received any state during a super-step. We thus need an invariant in the local_successors for maintaining the fact that the set known potentially grows with at least the states of todo. We also maintain that if todo is empty then no state would be sent (in local_successors) and received, making total equal to 0in exchange. This is due to the fact that eventually, in the last super-step, no new state are computed or added to known by any processor.

Local Computations. The termination is ensured as in the sequential algorithm since known can only grow when entering the loop. Note that the inner loop needs to define a ghost3 variable ghost_diff which represents the set of states of the initial new_states that have been processed since new_state decreases and we lack this information for proving that all states have been really well processed.

The annotations work as follow. First, in the pre-condition to local_successors, we have that all manipulated sets of states are subset of the StSpace (property to prove (A)); known and todo are disjoint sets; all states are owned by the right processor and finally that any state in known has its successor in known or have been previously sent in a past super-step.

Second, the nine invariants of the outer loop are the following: (1-3) as in the pre-condition; (4) any new state to be send has not been sent in a past super-step; (5) the sets are in tosend (to be send) as cpu needs it; (6) the processor does not send states to itself; [START_REF] Armando | SAT-based Model-checking for Security Protocols Analysis[END_REF] any state in known has its successors to be processed, to be send or still known (this is for property (B) since todo would be finally empty as well sets of states to be send; known has the "same" inductive property than StSpace); (8) known grows with at least the states from todo; (9) if there is no state to process then there will be no state to send nor to process (it is useful for the termination property of the main parallel loop).

Third, the inner loop maintains the same kind of invariants but with also the set new_states, the states that have been computed from one state of todo. We have also the fact (invariants (e) and (f)) that all states of new_states are processed in the loop (we do not miss a state).

Fourth, the post-conditions are a mix of the pre-conditions and of the above invariants. Mainly, we have that all manipulated sets of states are subset of the StSpace; states still been well partitioned; if there is no state to process then there will be no state to send.

Mechanical Proof.

With some obvious axioms on the above predicates (such as <∅,...,∅>=∅) so that solvers can handle the predicates, all the produced obligations are automatically discharged by a combination of the solvers. In the following table, for each part of the parallel algorithm, we give the number of obligations and how many are discharged by the provers (some proof obligations require long timeouts e.g. Now the combination of all provers is needed since none of them is able to prove all the obligations. This is certainly due to their different heuristics. We also note that simplify and z3 remain the most efficient. Some obligations are hard to follow due to the parallel computations. But reading them carefully, we can find the good annotations.

Dedicated Algorithms for Security protocols (a) Specific Properties of Security Protocols [120]

As said before, we model security protocols as an explicit set of states (or Kripke structure) such that any state can be represented by a function from a set of locations to an arbitrary data domain. For instance, locations may correspond to local variables of agents, buffers, etc. Our approach is largely independent of the chosen formalism and it is enough to assume that the four following properties hold:

(P1) The function successor succ can be partitioned into two successor succ R and succ L (R and L form a partition of the locations) that correspond respectively to execute the transitions upon which an agent (except the intruder) receives information (and stores it), and to execute all the other transitions;

(P2) There is an initial state s 0 and there exists a function slice from states to natural numbers (a measure) such that if s ′ ∈ succ R (s) then there is no path from s ′ to any state s ′′ such that slice(s) ≤ slice(s ′′ ) and slice(s ′ ) = slice(s) + 1 (it is often call a sweep-line progression); (P3) There exists also a function cpu R from states to natural numbers such that for all state s if s ′ ∈ succ L (s) then cpu R (s) = cpu R (s ′ ); mainly, the knowledge of the intruder is not taken into account to compute the hash of a state;

(P4) If s 1 , s 2 ∈ succ R (s) and cpu R (s 1 ) = cpu R (s 2 ) then there is no possible path from s 1 to s 2 and vice versa.

More precisely: for all state s and all

s ′ ∈ succ(s), if s ′ | R = s| R then s ′ ∈ succ L (s), else s ′ ∈ succ R (s)
; where s| R denotes the state s whose domain is restricted to the locations in R. Intuitively, succ R corresponds to transitions upon which an agent receives information and stores it, and R are the locations where the agents (except the attacker) store the information they receive.

On concrete models, it is generally easy to distinguish syntactically the transitions that correspond to a message reception in the protocol with information storage. Thus, is it easy to partition succ as above and, for most protocol models, it is also easy to check that the above properties are satisfied. Note also that our approach is independent of using partial order reductions as in [START_REF] Fokkink | Partial Order Reduction for Branching Security Protocols[END_REF] where the main idea is that the knowledge of the intruder always grows and thus it is safe to prioritise the sending transitions with respect to receptions and local computations of agents. A simple modification of the successors functions is sufficient.

It also to notice that some security properties such as secret (confidentiality), authentification, integrity, anonymity can be expressed only using a state-space computation since these properties force to a reachability analysis, i.e., finding a state that breaks one on the above properties. However, more complex property usually need to resort to temporal logics.

We can give (part) of these properties in the why's logic using the following:

logic slice: state → int ( *

(b) Algorithms for State-space Construction of Security Protocols [120]

Based on the above properties, we have designed, in an incremental manner, bsp algorithms for efficiently computing the state-space of security protocols. Successive improvements of the generic bsp algorithm will result in a parallel algorithm that remains quite simple in its expression (and efficient [START_REF] Gava | Performance Evaluations of a bsp Algorithm for State-space Construction of Security Protocols[END_REF]) but that actually relies on a precise use of a consistent set of observations and algorithmic modifications. Only the functions local_successors and exchange have been really modified in the distributed algorithms.

Increasing Local Computation Time. Using the naive parallel algorithm, function cpu distributes evenly the states over the processors. However, each super-step is likely to compute few states because only too few computed successors are locally owned. This results in a bad balance of the time spent in computation with respect to the time spent in communication. If more states can be computed locally, this balance improves but also the total communication time decreases because more states are computed during each call to local_successors. To achieve this result, we consider a peculiarity of the models we are analysing (see the scheme on the left). The learning phase (2) of the attacker is computationally expensive, in particular when a message can be actually decomposed, which leads to recompose a lot of new messages. Among the many forged messages, only a (usually) small proportion are accepted for reception by agents. Each such reception gives rise to a new state. This whole process can be kept local to the processor and so without cross-transitions. To do so, we need to design our partition function cpu R such that, for all states s 1 and s 2 , if

s 1 | R = s 2 | R then cpu R (s 1 ) = cpu R (s 2 ).
For instance, this can be obtained by computing a hash (modulo the number of processors) using only the locations from R, i.e., the locations where the honest agents store received information.

In this first improvement, when local_successors is called, then all new states from succ L are added in todo (states to be proceeded) and states from succ R are sent to be treated at the next super-step, enforcing an order of exploration of the state-space that matches the progression of the protocol in slices. Another difference is that no state could be sent twice due to this order. Fig. 5.8 gives the new function local_successors. The main loop of the parallel algorithm still unchanged. We call this new algorithm "Incr". Another difference is the forgotten variable "pastsend" since no state could be sent twice due to this order. With respect to the "Naive" algorithm, this one splits the local computations, avoiding calls to cpu R when they are not required. This may yield a performance improvement, both because cpu R is likely to be faster than cpu and because we only call it when necessary. But the main benefits in the use of cpu R instead of cpu is to generate less cross transitions since less states are need to be sent. Finally, notice that, on some states, cpu R may return the number of the local processor, in which case the computation of the successors for such states will occur in the next super-step. We now show how this can be exploited.

Decreasing Local Storage.

One can observe that the structure of the computations now matching the structure of the protocol execution: each super-step computes the executions of the protocol until a message is received. As a consequence, from the states exchanged at the end of a super-step, it is not possible to reach states computed in any previous super-step.

This kind of progression in a model execution is the basis of the sweep-line method [START_REF] Christensen | A Sweep-Line Method for State-space Exploration[END_REF] that aims at reducing the memory footstep of a state-space computation by exploring states in an order compatible with progression. It thus becomes possible to regularly dump from the main memory all the states that cannot be reached anymore -a disk-based backup can also be done if necessary for restoring the trace of a forbidden computation. Thus, statement dump(known) resets known to an empty set, possibly saving its content to disk if this is desirable. The rest of function exchange is simplified accordingly.

Enforcing such an exploration order is usually made by defining on states a measure of progression. In our case, such a measure is not needed because of the match between the protocol progression and the super-steps succession. So we can apply the sweep-line method by making a simple modification of the above algorithm: we empty known at the beginning of each super-step and \ !known is now not needed after the successor function succ R because all these states need to be send for the next super-step/slice. Fig. 5.8 gives the new function exchange. We call this new algorithm "Sweep".

Balancing the Computations. During our benchmark, we have found that using cpu R can introduce a bad balance of the computations due to a lack of information when hashing only on R. Thus, the final optimisation step aims at balancing the workload. To do so, we exploit the following observation: for all the protocols we have studied so far, the number of computed states during a super-step is usually closely related to the number of states received at the beginning of the super-step. So, before to exchange the states themselves, we can first exchange information about how many states each processor has to send and how they will be spread onto the other processors. Using this information, we can anticipate and compensate balancing problems. To compute the balancing information, we use a new partition function cpu B that is equivalent to cpu R without modulo. This function defines classes of states for which cpu B returns the same value. Those classes are like a "bag-of-tasks" [START_REF] Iosup | The Performance of Bags-of-tasks in Large-scale Distributed Systems[END_REF] that can be distributed over the processors independently (see scheme on the left). To do so, we compute a histogram of these classes on each processor, which summarises how cpu R would dispatch the states. This information is then globally exchanged, yielding a global histogram that is exploited to compute on each processor a better dispatching of the states it has to send. This is made by placing the classes according to a simple heuristic for the bin packing problem: the largest class is placed onto the less charged processor, which is repeated until all the classes have been placed.

It is worth noting that this placement is computed with respect to the global histogram, but then, each processor dispatches only the states it actually holds, using this global placement. Moreover, if several processors compute a same state, these identical states will be in the same class and so every processor that holds such states will send them to the same target. So there is no possibility of duplicated computation because of dynamic states remapping. We call this algorithm "Balance". Classes of states (consistent with partition function cpu B ) are grouped on processors so there is no possibility of duplicated computation. This "Balance" algorithm gives better performances than a naive distributed one for security protocols [START_REF] Gava | Performance Evaluations of a bsp Algorithm for State-space Construction of Security Protocols[END_REF].

(c) Verification of these Dedicated Parallel Algorithms

For all these algorithms, termination and safety are proved as the naive one.

Algorithm "Incr". The invariants are the same of Fig. 5.6 but with these changes. First, we need to forget all the behaviour about pastsend in the invariants that is invariants (9 -11) since we no longer use this variable -due to the slice progression. Second, we introduce these two new invariants: [START_REF] Balaji | mpi on Millions of Cores[END_REF] and (∀ e:state. e ∈ (<known>) → slice(e)<ghost_slice) [START_REF] Bamha | Pipelining a Skew-Insensitive Parallel Join Algorithm[END_REF] and (∀ e:state. e ∈ (<todo>) → slice(e)=ghost_slice)

It is easier for the proofs of correctness to introduce the ghost variable ghost_slice which is incremented at each super-step and thus corresponds to the measure of progression of the protocol. Invariant ( 12) is needed to prove that the set known contains only states of the past slices and invariant [START_REF] Bamha | Pipelining a Skew-Insensitive Parallel Join Algorithm[END_REF] proves that todo contains only states of the current slice. The assertions of the local computations need also to be changed. Most of them are as for the "Naive" algorithm. They are available in Fig 5. [START_REF] Armbrust | Above the Clouds: A Berkeley View of Cloud Computing[END_REF].

We only detail the true changes. First, In the pre-conditions and in all the annotations, we now used cpu R ; Second, any state in known has it slice measure less than the current slice (super-step) since it is a state that has been previously computed; but any state in todo has its slice equal to the current slice. We maintain this fact in the invariants [START_REF] Bacci | P3L: a Structured High-level Parallel Language, and its Structure Support[END_REF][START_REF] Bal | Orca: A Language For Parallel Programming of Distributed Systems[END_REF][START_REF] Balaji | mpi on Millions of Cores[END_REF] where the slice measure of states to be send are equal to the current slice plus one (for the next slice/super-step). Finally, the post-conditions are as above and with the properties of the slice measure.

Algorithm "Sweep". When dumping known at each begin of super-step/slice, we can thus no longer use known as the variable which contains the full state-space. We thus introduce another ghost variable called ghost_known which will grow at each super-step by recovering all the states of known. In this way and for having the correctness of this algorithm, in all the previous invariants, ghost_known replaces known. The rest is unchanged. Finally, it is thus an easy modification (for the algorithm and its correctness) but memory efficient. Algorithm "Balance". Using another partition function that is cpu B which partitioned (at each slice) the state-space into independent classes of states, we cannot remains of the cpu R partition. In this way, tosend is not a fix (size nprocs) array of sets of states, but rather a mapping from the hashing representing the classes (integer compute using cpu B ) to sets of states (the states of the classes). For the correctness of the algorithm, we thus need a new predicate class(e,e') that logically define that two states belong to the same class. We also need to redefine the predicate GoodPart(<p_set>) as follow:

∀i,j:int. isproc(i) → isproc(j) → i = j → ∀s,s':state. s ∈p_set<i> → s' ∈p_set<j> → ¬class(e,e')"

which denotes that two states that belong to two different processors are not in the same class. We also need to assert that after the computation of the balance (currently axiomatised since it is a heuristic of a np-problem), sent states respect the predicate GoodPart. We also need the following invariant: which denotes that known states respect the fact that all states in a class belong to the same slice at the same processor. The local_successor function should verify this fact in its invariants which is rather very simple to do. The complete annotated codes are also available in the source of this paper.

Proof Obligations. In the following As above, only the combination of all provers is able to prove all the obligations. And few of them need that provers run minutes. simplify and z3 still remain the most efficient. An interesting point is that this work has been partially done by a master student: he was able to perform the job (annotate these parallel algorithms) in three months. Based on this fact, it seems conceivable that a more seasoned team in formal methods can tackle more substantial algorithms (of model-checking) in a real programming language.

Related Work

Other Methods for Proving the Correctness of Model-checkers

Fig. [START_REF] Armand | Extending coq with Imperative Features and Its Application to SAT Verification[END_REF].10 summarises different methods that have been used for verifying mcs where each arrow corresponds to a proof of correctness (using a theorem prover or a pcc approach) and the papers related to the work. The state-space explosion can be a problem for mcs extracted from theorem provers. They are pure functional programs such as the ones of [START_REF] Esparza | A Fully Verified Executable ltl Model Checker[END_REF][START_REF] Sprenger | A Verified Model Checker for the Modal µ-calculus in coq[END_REF]. They certainly would be too slow for big models even if there work on obtaining imperative programs from extracted (pure) functional programs.

A sequential state-space algorithm (with a partial order reduction) has been checked in b in [START_REF] Turner | A Refinement-based Correctness Proof of Symmetry Reduced Model-checking[END_REF].

The "certifying model-checking" is an established research field [START_REF] Peled | From Falsification to Verification[END_REF][START_REF] Tan | Evidence-Based Model Checking[END_REF]. But, the performance issue of pcc is discussed in [START_REF] Verma | Reflecting BDDs in coq[END_REF] and [START_REF] Rival | Experiments with Finite Tree Automata in coq[END_REF] where the authors present developments (and model-checking benchmarks) of bdds and tree automata using theorem provers: bdds are common data-structures used by mcs and tree automata is an approach for having a formal successor function. pcc only focuses on the generation of independently-checkable evidences as the compiled code satisfies a simple behavioural specification such as memory safety; the evidence can then be checked efficiently. Using pcc for state-space is the same as computing it a "second time". In fact, the drawback of proof certificates is that verification tools have to be instrumented to generate them, and the size of fully expanded proofs may be too large. Authors of [START_REF] Rival | Experiments with Finite Tree Automata in coq[END_REF][START_REF] Verma | Reflecting BDDs in coq[END_REF] conclude that pccs are here inadequate and we can conclude that mcs themselves need to be proved. It is also the conclusion of [START_REF] Ford | Formal Verification of a Combination Decision Procedure[END_REF] where the authors note that "to avoid the inefficiency of fully expansive proof generations, a number of researchers have advocated the verification of decision procedures".

In [START_REF] Ford | Formal Verification of a Combination Decision Procedure[END_REF][START_REF] Shankar | The Mechanical Verification of a DPLL-Based Satisfiability Solver[END_REF], the authors have done a mechanical verification (using the theorem prover pvs) of automatic solvers methods: they note that the inefficiency of fully expansive proof generation can be avoided through verifying the decision procedures. One of the authors argues that trust need not be achieved at the expense of automation, and outlines a lightweight approach where the results of untrusted verifiers are checked by a trusted offline checker. The trusted checker is a verified reference kernel that contains a satisfiability solver to support the robust and efficient checking of untrusted tools. He summarizes the problem: Quis custodiet ipsos custodes ? (Who will watch the watchmen? that is, who will verify the verifier?). We want to be able to trust the results of provers/model-checkers with a high degree of confidence. But currently, only an approach using functional programs is presented with the same main drawback as above: less efficiency.

In our work, we also only use automatic solvers for proving the generated goals of the vcg why and thus we do not use any "elaborate" theorem prover such as coq. The correctness of our results depends on the correctness of (1) the why tool (correct generation of goals) and ( 2) the results of the solvers. Relying on modules like smt solvers has the advantage that these tools would certainly be verified in a close future. The work of [START_REF] Herms | Certification of a Chain for Deductive Program Verification[END_REF] is a first approach for (1) and the work of [START_REF] Böhme | Fast LCF-Style Proof Reconstruction for Z3[END_REF] is a pcc approach for (2). Moreover, a smt solver has been proved using a theorem prover [START_REF] Stump | SMT Proof Checking using a Logical Framework[END_REF]. In a close future, we can hope to achieve the same confidence in our codes as the mcs extracted from [START_REF] Esparza | A Fully Verified Executable ltl Model Checker[END_REF][START_REF] Sprenger | A Verified Model Checker for the Modal µ-calculus in coq[END_REF], as well as better performances since our codes are realistic imperative codes -and not functional ones from theorem provers. Finally, we think that using annotations (and a vcg tool) has the advantage of being "easy". And we can prove the correctness of programs or limit the work to some safety properties if the full correctness is too difficult to obtain. And it extends to parallel programs which is not easy using pccs or theorem provers.

A mechanically assisted proof using Isabelle of how ltl formulae can be transformed into Büchi automata is presented in [START_REF] Schimpf | Construction of Büchi Automata for ltl Model Checking Verified in Isabelle/HOL[END_REF]. ctl* temporal logic is also available in coq [START_REF] Tsai | Formalization of CTL* in the Calculus of Inductive Constructions[END_REF] (and ltl in [START_REF] Coupet-Grimal | An Axiomatization of Linear Temporal Logic in the Calculus of Inductive Constructions[END_REF]). All these works are interesting since logical theories may be axiomatised in why.

Model compilation is one of the numerous techniques to speedup model-checking: it relies on generating source code (then compiled into machine code) to produce a high-performance implementation of the state-space exploration primitives, mainly the successor function. In [START_REF] Fronc | Towards a Certified Petri Net Model-Checker[END_REF], authors propose a way to prove the correctness of such an approach. More precisely, they focus on generated Low-Level Virtual Machine (llvm) code from high-level Petri nets and manually prove that the object computed by its execution is a representation of the compiled model. If such a work can be redone using a theorem prover, we will have a machine-checked successor function which is currently axiomatised in why.

Verification of Security Protocols

Gavin Lowe has discovered the now well-known attack on the Needham-Schroeder public-key protocol using the model-checker fdr [START_REF] Lowe | Breaking and Fixing the Needham-Schroeder Public-Key Protocol Using FDR[END_REF]. In spite of this, over the last two decades, a wide variety of security protocol analysis tools have been developed that are able to detect attacks on protocols or, in some cases, establish their correctness. We distinguish three classes: tools that attempt verification (proving a protocol correct), those that attempt falsification (finding flaws, i.e., counterexamples), and hybrids that attempt to provide both proofs and counterexamples. In the first category, we find the use of theorem provers [START_REF] Paulson | The Inductive Approach to Verifying Cryptographic Protocols[END_REF] and dedicated tools such as proverif [START_REF] Blanchet | An Efficient Cryptographic Protocol Verifier Based on Prolog Rules[END_REF] [[ 64 ]] or scyther [START_REF] Cremers | Scyther -Semantics and Verification of Security Protocols[END_REF] [[ 65 ]], athena [START_REF] Song | Athena: A Novel Approach to Efficient Automatic Security Protocol Analysis[END_REF], etc., falsification is the domain of model-checking [START_REF] Armando | ltl Model Checking for Security Protocols[END_REF][START_REF] Armando | SAT-based Model-checking for Security Protocols Analysis[END_REF], and the latter of model-checker with lazy intruders such as avispa [START_REF] Armando | The AVISPA Tool for the Automated Validation of Internet Security Protocols and Applications[END_REF]

[[ 66 ]].
Papers [START_REF] Cortier | A Survey of Symbolic Methods in Computational Analysis of Cryptographic Systems[END_REF][START_REF] Cremers | Comparing State-spaces in Automatic Security Protocol Analysis[END_REF][START_REF] Dalal | A Comparative Analysis of Tools for Verification of Security Protocols[END_REF] present different cases study of verifying security protocols with various standard tools. To summarise, there is currently no tool that provides all the expected requirements. Most of them limit possible kinds of attacks or limit in their model language how addresses of agents can be manipulated in ad-hoc protocols (using arithmetic operations). Paper [START_REF] Dalal | A Comparative Analysis of Tools for Verification of Security Protocols[END_REF] presents different cases study of verifying security protocols with various standard tools. To summarise, there is currently no tool that provides all the expected requirements. Model-checking is the one we choose in this work.

Model-checking security protocols is not new [START_REF] Armando | ltl Model Checking for Security Protocols[END_REF][START_REF] Basin | An On-the-Fly Model-Checker for Security Protocol Analysis[END_REF][START_REF] Li | On-the-fly Model Checking of Security Protocols[END_REF][START_REF] Lowe | Casper: A Compiler for the Analysis of Security Protocols[END_REF]. In the work of [START_REF] Marrero | Brutus: a Model-checker for Security Protocols[END_REF], a por reduction of security protocols is used and in the work of [START_REF] Armando | ltl Model Checking for Security Protocols[END_REF], a lazy intruder was used -both are sequential algorithms. Our work has the advantage to take into account distributed architectures. In [START_REF] Mitchell | Automated analysis of cryptographic protocols using Murphi[END_REF], the authors have used the murphi modelling language and different parallel programs for murphi now exist. But the algorithm [START_REF] Stern | Parallelizing the µrj Verifier[END_REF] uses a naive random hash function. Notice also the work [START_REF] Blom | Analysing the BKE-security Protocol with µCRL[END_REF] for the µcrl checker. For finite checking scenarios (and enumerate state-space construction), the most well known tool is certainly avispa [START_REF] Armando | The AVISPA Tool for the Automated Validation of Internet Security Protocols and Applications[END_REF]. We believe that our observations and the subsequent optimisations are general enough to be adapted to the tools dedicated to protocol verification: we worked in a general setting of Kripke structure, defined by an initial state and a successor function. Our only requirements are the four simple conditions (P1 to P4) that can be easily fulfilled within most concrete modelling formalisms. [START_REF] Gao | Analysis of Security Protocols by Annotations[END_REF] allows to verify some properties about some classes of protocols for an infinite number of sessions and with some possibility of replay using a process algebra. Nevertheless,each time a new property is needed, a new theorem has to be proved. That could lead to a complex maintenance of the method. Furthermore, the method cannot be applied to some protocols, e.g., the Yahalom one. On the contrary, our approach is based on explicit state-space construction, that is not tied to any particular application domain. This is a well-desired feature for checking p2p security protocols as in [START_REF] Chaou | Evaluating a Peer-to-peer Storage System in Presence of Malicious Peers[END_REF].

To our knowledge, there are three existing approaches to automatically generate machine-checked protocol security proofs. The first approach is in [START_REF] Goubault-Larrecq | Finite Models for Formal Security Proofs[END_REF] where a protocol and its properties are modelled as a set of Horn-clauses and where the certificate is machine-checked in coq. The second [START_REF] Brucker | Integrating Automated and Interactive Protocol Verification[END_REF] uses the theorem prover isabelle and computes a fixpoint of an abstraction of the transition relations of the protocol of interest -this fixpoint over-approximates the set of reachable states of the protocol. The latter [START_REF] Barthe | Computer-Aided Cryptographic Proofs[END_REF][START_REF] Meier | Strong Invariants for the Efficient Construction of Machine-Checked Protocol Security Proofs[END_REF] also uses coq or isabelle and invariants are derived from an operational semantics of the protocols. We see three main drawbacks to these approaches. First, they limit (reasonably) protocols and properties that can be checked. Second, each time, the proof of the tested property of the protocol needs to be machine-checked; in our approach, the results of the mc are correct by construction. Third, there is currently no possibility of distributed computations for larger protocols -hours of computations are needed for some protocols. But they have one evident advantage: only the final result has to be correct (intermediate computations do not need to be verified) and it saves time for correct design (machinechecked) of algorithms. All these methods are also based on perfect cryptography. The author of [START_REF] Hartog | Towards Mechanized Correctness Proofs for Cryptographic Algorithms: Axiomatization of a Probabilistic Hoare Style Logic[END_REF] annotated cryptographic algorithms to mechanically prove their correctness.

Distributed State-space Construction

The main idea of most known approaches to the distributed memory state space generation is similar to the naive algorithm [START_REF] Garavel | Parallel State Space Construction for Model-Checking[END_REF].

Close to our hashing technique, [START_REF] Petcu | Parallel Explicit State Reachability Analysis and State-space Construction[END_REF] presents a hashing function that ensures that most of the successors are local: the partition function is computed by a round-robin on the successor states. This improves the locality of the computations but can duplicate states. Moreover, it works well only when network communication is substantially slower than computation, which is not the case on modern architectures for explicit model-checking. For load balancing, [START_REF] Lluch-Lafuente | Simplified Distributed Model-checking by Localizing Cycles[END_REF] presents a new dynamic partition function scheme that builds a dynamic remapping, based on the fact that the state space is partitioned into more pieces than the number of involved machines. When load on a machine is too high, the machine releases one of the partitions it is assigned and if it is the last machine owning the partition it sends the partition to a less loaded machine.

In [START_REF] Lerda | Distributed-Memory Model Checking with SPIN[END_REF], a state-space exploration algorithm derived from the spin model-checker is implemented using a master/slave model of computation. Several spin-specific partition functions are experimented, the most advantageous one being a function that takes into account only a fraction of the state vector, similarly to our function cpu R . The algorithm performs well on homogeneous networks of machines, but it does not outperform the standard implementation except for problems that do not fit into the main memory of a single machine. Moreover, no clue is provided about how to correctly choose the fraction of states to consider for hashing, while we have relied on reception locations from L -R.

In [START_REF] Pajault | Model Checking Parallèle et Réparti de Réseaux de Petri Colorés de Haut-niveau[END_REF] various technics from the literature are extended in order to avoid sending a state away from the current processor if its 2nd-generation successors are local. This is complemented with a mechanism that prevents re-sending already sent states. The idea is to compute the missing states when they become necessary for model-checking, which can be faster than sending it. That clearly improves communications but our method achieves similar goals, in a much simpler way, without ignoring any state.

There also exist approaches, such as [START_REF] Knottenbelt | Probability, Parallelism and the State Space Exploration Problem[END_REF], in which parallelization is applied to "partial verification", i.e. state enumeration in which some states can be omitted with a low probability. In our project, we only address exact, exhaustive verification issues. For the partition function, different technics have been used. In [START_REF] Garavel | Parallel State Space Construction for Model-Checking[END_REF] authors used of a prime number of virtual processors and map them to real processor. This improves load balancing but has no real impact on cross transitions. In [START_REF] Orzan | A State Space Distributed Policy Based on Abstract Interpretation[END_REF], a user defined abstract interpretation is used to reduce the size of the state space and then it allows to distribute the abstract graph; the concrete graphs is then computed in parallel for each part of the distributed abstract graph. In contrast, our distribution method is fully automated and does not require input from the user.

To our knowledge, the first bsp algorithm for state-space construction is the one of [START_REF] Martin | Parallel Algorithms for Deadlock and Livelock Analysis of Concurrent Systems[END_REF]. The authors limit their study to dead/livelock detection of csp expressions. Termination is ensuring by an additional global "reduction operation" to discover the total number of pending states which remain in the system. They also used fixed size buckets and processors computed until their own buckets are fulls which induce a greater number of super-steps but a better balancing.

Formal Semantics

This chapter subsumes the work of [START_REF] Gava | Formal Semantics of a Subset of the Paderborn's bsplib[END_REF][START_REF] Gava | Two Formal Semantics of a Subset of the Paderborn University bsplib[END_REF].

. . 124

In chapter 4, we presented a transformation from a bsp-why-ml program to a why-ml program, and explained how it was thus possible to use the why tools to prove the correctness of the original program. However, one can only trust the proof obtained in this way as much as the bsp-why tool is trusted. In this chapter, we present the tools needed to acquire this trust.

First, we define formal semantics of the bsp-why-ml language, which is necessary to be able to reason formally about the programs and the transformation. We start by giving a mathematical definition of the semantics in Section 6. 1. A natural big-steps operational semantics is presented, close to the intuition of the language. We also give co-inductive infinite semantics, allowing us to study programs that are running indefinitely. Those big-steps semantics can be seen as a formal specification. In the Section 6.2, we give a more refined small-step semantics, using continuations. This semantics, although less intuitive, is better suited for complex proofs on bsp-why-ml programs. To keep a high level of trust, the semantics is proved equivalent to the big-step semantics, the specification. The challenges added if we allow the subgroup synchronisation, and the corresponding semantics, are also studied along these sections.

In Section 6.3, we show more in details how those operational semantics are formalised (machine-checked) in coq using a set of (co-)inductive definitions. In this chapter, all the proofs that have been machine-checked in coq, are concluded with a symbol of a rooster. Finally, in Section 6.4, we will present the proof of the transformation done by bsp-why. The structure of the proof was also checked by the machine in coq, though not all the steps are currently proved in coq because of a lack of time. 99

6.1

Big-steps Semantics

Semantics Rules (a) Problematic

Values to be sent and distant reading/writing are stored in environments of communications as simple list of messages. There are thus six additional components in the environment (R, C send , C put , C get , C pop , C push ), one per primitive that needs communications. Each is a simple list of messages. For drma primitives, there is also the registration T as described in Section. [START_REF] Appel | Separation Logic for Small-Step Cminor[END_REF].2 (push and pop need communications to keep the registration of every processor coherent).

In [?], J.-C. Filliâtre gives the formal definitions and rules for a big-step semantics of the why-ml language. The notions of values v and states for bsp-why-ml are similar to the why-ml semantics with the additional possible value SYNC(e), which describes the awaiting of a synchronisation with e as program to be executed after the global synchronisation: v ::= c | (Ec) | SYNC(e). A value v can be a constant (integer, boolean, etc.), an exception E carrying a constant c, or the synchronisation state.

We note s for the environment of a processor. It is a 8-tuple E, T , R, C send , C put , C get , C pop , C push . We note s.X to access to the component X of the environment s, ⊕ the update of a component of an environment without modifying other components and ∈ to test the presence of an item in the component.

As the bsplib, drma variables are registered using a registration mechanism that is each processor contains a registration T which is a list of registered variables: the first one in the list of a processor i corresponds to first one of the processor j.

(b) Local Rules

We first have semantics rules for the local execution of a program, on a processor i. We note s, e ⇓ i s ′ , v for these local reductions rules (e.g. one at each processor i): e is the program to be executed, v is the value after execution, s is the environment before the execution, s ′ the environment after the execution.

Rules for the local control flows are fully defined in Fig. 6. [START_REF] Aldinucci | MUSKEL: An Expandable Skeleton Environment[END_REF]. For each control instruction, it is necessary to give several rules, depending on the result of the execution of the different sub-instructions: one when an execution leads to a synchronisation (when processors finish a super-step), and one if it returns directly a value. We have thus to memorise as a value the next instructions of each processor. These intermediate local configurations are noted SYNC(e). To avoid confusion between a new reference and those that have been registered before, one can not declare a reference that has been created before. This is not a problem since why already forbids this case. Rules of the bsp operations are given in Fig. [START_REF] Armando | ltl Model Checking for Security Protocols[END_REF].1 (executed on a single processor i). Basically, a primitives adds the corresponding message to the environment.

(c) Parallel Rules and Results

bsp programs are spmd ones so an expression e is started p times. We model this as a p-vector of e with its environments. A final configuration is a value on all processors. We note ⇓ for this evaluation and the two needed rules are given in Fig. 6. [START_REF] Alt | Algorithm Design and Performance Prediction in a java-Based Grid System with Skeletons[END_REF].

First rule gives the base case, when each processor i executes a local (sequential) evaluation ⇓ i to a final value. The second rule describes the synchronisation process when all processors execute to a SYNC(c) state: the communication are effectively done during the synchronisation phase and the current super-step is finished. The AllComm function models the exchanges of messages and thus specifies the order of the messages. It modifies the environment of each processor i. For the sake of brevity, we do not present this function which is a little painful to read and is just a reordering of the p environments.

Note that if at least one processor finishes his execution while others are waiting for a synchronisation, a deadlock will occur. Finally we have the following results. Proof. Trivially proved by induction on the evaluation.

Co-inductive Semantics

In addition to the standard big-step operational semantics, it is often useful to define co-inductive (or infinite) semantics. They allow to characterize the behaviour of a program that runs indefinitely. Defining divergence (infinite evaluations) is also done using inference rules but interpreted coinductively. More precisely, the relation is the greatest fixpoint of the rules, or, equivalently, the conclusions of infinite derivation trees built from these rules [START_REF] Leroy | Coinductive Big-step Operational Semantics[END_REF]. Throughout this thesis, double horizontal lines in inference rules denote inference rules that are to be interpreted coinductively; single horizontal lines denote the inductive interpretation. s, bsp sync ⇓ i s, SYNC(void) The co-inductive rules for the local control flow are as expected. They can easily be inferred from the regular big-step semantics. We give as an example one of the co-inductive rules for the if instruction:

s ′ = s.C push ⊕ x s, bsp push x ⇓ i s ′ , void s ′ = s.C pop ⊕ x s, bsp pop x ⇓ i s ′ ,
s, e 1 ⇓ i s 1 , true s 1 , e 2 ⇓ i ∞ s, if e 1 then e 2 else e 3 ⇓ i ∞
The rule can be read as follow: "If e 1 evaluates to true, and if e 2 runs infinitely, then the program if e 1 then e 2 else e 3 will run infinitely". Local bsp operation (push, send, etc.) always terminates, so the co-inductive are as expected. On the other hand, and more interestingly, we define the co-inductive rules for the parallel operations: The first rule states that if one of the processors runs infinitely, then the parallel program will run infinitely. In the second rule, it is said that if all the processors reach a synchronisation barrier, and if the parallel program runs infinitely starting from the resulting state, then the parallel program runs infinitely. Some results linking the two semantics are easily proved: Co-inductive semantics is also deterministic in the sense that the constructed infinite tree will always be the same. But as we do not currently give the execution traces, this property is not interesting and we felt non-provable in coq.

∃i s i , e i ⇓ i

Adding the Subgroup Synchronisation

In Section 4.3, we introduced a variation of bsp-why, that is not strictly dealing with the pure bsp model, instead allowing processors to synchronise within a subgroup. This gives us the possibility to work on parallel programs written for the bsp pub library, which allows for some limited subgroup synchronisation, but also on some families of mpi programs.

However, the subgroup synchronisation also introduces a slightly more complex language and parallel model, thus it is necessary to give the corresponding semantics. Let us describe the modifications that are needed.

(a) Local Operations

The semantics of local control flow operations of bsp-why-ml is not modified when introducing the subgroup synchronisation. There are some modifications in the way conditional instructions and loops are handled by bsp-why, but the language still follows the same semantics. Thus the rules defined in Fig. [START_REF] Armando | ltl Model Checking for Security Protocols[END_REF].2 are still valid.

With the subgroup synchronisation model, bsp operations are done in the scope of a communicator. Every bsp call thus takes an additional argument, this communicator, which describes the subset of processors in which the communications are done. However, apart from this addition, everything else remains the same. For example, the inductive rule for C send is now:

s, e ⇓ i s ′ , to cmt ∈ s ′ 0 ≤ to < p cmt {x → v} ∈ s ′ .E s ′′ = s ′ .C send cmt ⊕ {to, v} s, bsp send x e cmt ⇓ i s ′′ , void
where "cmt" is a valid communicator in the environment. The message to processor "to" of the communicator "cmt" is added to the queue of message.

(b) Global Reduction Rules

The major changes in the semantics are located within the parallel rules. Instead of having all the p processors synchronise together, and communicate together, it is now possible for a subgroup to synchronize together and make the needed communications. Several subgroups can also work independently from each other, and synchronise at the same time. There are two major options for the semantics in this situation:

1. All processors execute their code locally, until they reach a synchronisation state (or they terminate). We execute all of the possible subgroup synchronisations, and then start again the local computations. We call this options AllSub.

2.

A subgroup of processors execute their code locally, until they reach their synchronisation. We execute the synchronisation and the associated communications, then start again. We call this options Diamond.

With the subgroup synchronisation, the bsp notion of superstep is less clearly defined, and the two options could be seen as two different definitions of a superstep in this model.

There are pros and cons to both formulations. In the AllSub option, the rule is complex to write, even more so in the coq proof assistant. However, it is perhaps the rule matching most closely the execution of a parallel program where all processors compute in parallel. The Diamond definition is much easier to write and understand, but artificially gives priority to one subgroup over another one. This ordering of the subgroups execution leads to another issue: with this definition, the semantics loses its determinism, since when several subgroup are synchronising, it is possible to choose any subgroup to execute first.

We first give the formulation of Allsub:

{0, . . . , p -1} = N ⊕ C 1 ⊕ • • • ⊕ C k ∀i ∈ C j s i , e i ⇓ i s ′ i , SYNC(C j , e ′ i ) ∀i ∈ N s i , e i ⇓ i s ′ i , v i AllCommSimul{C 1 . . . C k , v, (s ′ 0 , e ′ 0 ), . . . (s ′ p-1 , e ′ p-1 )} ⇓ All (s ′′ 0 , v 0 ), . . . (s ′′ p-1 , v p-1 ) (s 0 , e 0 ), . . . (s p-1 , e p-1 ) ⇓ All (s ′′ 0 , v 0 ), . . . (s ′′ p-1 , v p-1 )
In this rule, we first partition the set of processors in k subsets that will synchronise, plus a subset N of processors that do not synchronise. AllCommSimul is then similar to the AllComm function defined before. However, because there can be several subgroups synchronizing, its exact definition is more complicated.

• It accepts as arguments the set of the communicators used in the synchronisation.

• In addition, it accepts as argument the array of the final values v i already reached in the superstep. For i ∈ N , the i-th processor terminates without synchronisation with the value v i , so the i-th component of the result of AllCommSimul will be the couple (s ′ i , v i ).

• For every set of processors matching a communicator C j , all the communications corresponding to the communicator are done.

• Among the messages of these processors, it only considers the ones that were sent within the matching communicator.

We now give the formulation of Diamond:

∀i ∈ C s i , e i ⇓ i s ′ i , SYNC(C, e ′ i ) AllCommC{C, (s ′ 0 , e ′ 0 ), . . . (s ′ p-1 , e ′ p-1 )} ⇓ Diam (s ′′ 0 , v 0 ), . . . (s ′′ p-1 , v p-1 ) (s 0 , e 0 ), . . . (s p-1 , e p-1 ) ⇓ Diam (s ′′ 0 , v 0 ), . . . (s ′′ p-1 , v p-1 )
Here, AllCommC is similar to the AllComm function defined before, with the following differences:

• It accepts a second argument (a communicator).

• It only modifies the environments of the processors in the range of the communicator.

• Among the messages of these processors, it only considers the ones that were sent within the matching communicator.

It is easy to see that even though this semantics leads to non-determinism, it is still confluent. The reason is that the only source of non-determinism is the communication rules, for which any matching communicator can be chosen. However, at any given point, the communications between two communicators are independent, since each processor leads to a synchronisation in one communicator only. Thus, the diamond property holds. We justify our use of the second, shortest rule with the following lemma:

Property 1

The Diamond semantics is confluent.

Proof. It is a direct consequence of the diamond property.

Lemma 4

Both semantics are equivalent.

Proof. By induction on the number k of communicators used in a AllSub super-step, we can show that there is a Diamond derivation, formed by k synchronizations, that simulates the global AllSub synchronization rule.

Since the Diamond semantics is confluent, any derivation will lead to the same result. We thus have the following: A configuration is completed with a control stack κ. The final configuration is (s, void . ǫ), an empty control stack. The control stack represents what has not been executed. There are sequential control operators to handle local control flow. This is close to an abstract machine. In Fig. 6.4 we give the local rules of control flow and in Fig. 6.5 those for the local bsp operations. Note that currently, no rule is needed for the bsp sync instruction. In the control stack we find expressions with holes. Each hole represents the sub-expression that is currently evaluated.

s, e ⇓ Diam s ′ , v ∧ s, e ⇓ All s ′′ , v ′ ⇒ s ′ = s ′′ ∧ v = v ′
Most instructions are dealt with by several rules. Generally, the first rule simply puts the instruction in the continuation stack, and sets the first basic element of the instruction to compute as the main program. Then, one or more rules will match the possible results of this execution, and perform the necessary operations for the control instruction. For instance, with the first rule of the "if" statement, the "if" continuation is put in the control stack and, depending on the result of e 1 , the control stack gives the evaluation of e 2 or e 3 with the rest of the stack. A communication primitive consists in simply adding a new value in the environment and in practice all rules of bsp primitives are merged into a generic one.

The raise and try rules behave a bit differently, and deserve an explanation. The first raise rule will compute the argument as usual, but the second rule will simply gives an "error state", with the given error type and the computed value. The error states then go through the control stack, ignoring everything but the try continuations. The try rules try to match such an error state with the exception being caught. If it is another exception, the try will simply be removed from the stack; however, if it is the matching exception, the error treatment will be started, with the variable taking the value given in the error state.

(c) Global Rules

As in the big-step semantics, global rules are mainly used to call local ones and p configurations have to be reduced. Figure 6.6 gives those rules.

First, the global reduction calls a local one. This represents a reduction by a single processor, which thus introduces an interleaving of computations. Communications and bsp synchronisation are done

Co-inductive Semantics

Co-inductive semantics are much easier to define with the small-step semantics. A program runs indefinitely if it has an infinite sequence of small-step reductions. Thus, the definition of ⇀ ∞ is as follow:

s, e ⇀ s ′ , e ′ s ′ , e ′ ⇀ ∞ s, e ⇀ ∞

Equivalence Between the Semantics

The small-step semantics have less rules than the big-step ones. But finding them is much harder and it is thus less a formal specification of the language. We give now some results of equivalence.

Lemma 5

⇀ is confluent.

Proof. The small-step semantics verifies the diamond property: if from one state two steps are possible, they can only be local executions on two different processors. Since the executions are independent within a super-step, we can reach a common state by executing the other computation. The confluence follows as a direct consequence.

Lemma 6

⇀ * and ⇓ (resp. ⇀ ∞ and ⇓ ∞ ) are equivalent.

Proof. The proof has some common ground with classic big-step to small-step equivalences, with two main difficulties.

• The small-step semantics allows operations to execute on the different processors in any order, while the big-step semantics fixes the order.

• The continuations, coupled with the synchronisation, introduce the need for a notion of equivalence between a program and a continuation.

The first implication (big-step to small-step) is done by induction. However, an induction directly on the stated theorem would not be enough, and we need to generalize the result by defining a notion of equivalence between pairs (program, continuation). This is because after a synchronisation, in the bigstep semantics we still have a program to execute, while in the small-step semantics it is a continuation. It is then a straightforward induction on the derivation of the big-step execution.

For the second implication, we rely of the previous lemma. Since the small-step semantics is confluent, we can order the local executions in any order, in particular the order chosen with the big-step semantics (execution on the first processor first until the synchronisation barrier, then the second, etc.) The proof is then done by induction on the derivation.

Lemma 7

⇀ and ⇀ ∞ are mutually exclusive.

Proof.

By induction and co-induction.

Adding the Subgroup Synchronisation

Small-step semantics with subgroup synchronisation follow the same ideas seen with the big-step semantics.

(a) Local Operations

As for the big-step semantics, the bsp operations take an additional argument, the communicator, and the rules are modified accordingly.

For instance, the rules for the send operation will be as follow: In the second rule, O i is used to represent an outcome of a local execution, and C a communicator. The rule, similar to the big-step Diamond rule, states that if all the processor inside a communicator reach a synchronisation state on this communicator, then the communications are done along the AllCommC function. This function is defined in almost the same way as for the big-step semantics:

• It accepts a second argument (a communicator).

• It only modifies the environments of the processors in the range of the communicator.

• Among the messages of these processors, it only considers the ones that were sent within the matching communicator.

With this definition of the small-step semantics, we can state the same lemma of equivalence with the big-step semantics.

Lemma 8

⇀ * and ⇓ are equivalent.

Proof. The proof follows the same structure as the proof without sub-synchronisation.

Semantics in Coq

All of the semantics presented in this chapter were formally defined in the coq proof assistant. The whole coq development is available at http://www.lacl.fr/fortin/dev-coq.tar.gz. In this section, rather than giving the full code (which is several thousands of lines long), we will focus on key points of the development, explaining the main choices we took in the course of the modeling. We first present the semantics in coq without the subgroup synchronisation and then show how to add this feature.

Mechanized Semantics in Coq

The coq system (see [START_REF] Bertot | Interactive Theorem Proving and Program Development[END_REF] for a nice introduction to this theorem-proving system) is a proof assistant based on a logic which is a variant of type theory, following the "propositions-as-types, proofs-as-terms" paradigm, enriched with built-in support for inductive and co-inductive definitions of predicates and data types. From a user's perspective, coq offers a rich specification language to define problems and state theorems about them. This language includes (1) constructive logic with all the usual connectives and quantifiers; (2) inductive definitions via inference rules and axioms; (3) a pure functional programming language with structural recursion. Proofs are developed interactively using tactics that build incrementally the proof term behind the scene. These tactics range from the trivial (intro, which adds an abstraction to the proof term) to rather complex decision procedures (omega for Presburger arithmetic).

For example, let us define in coq a toy language of numerical expressions e := n | e 1 +e 2 | e1 e2 | inf where "inf" is an infinite computation and e1 e2 is possible only if e 2 = 0 otherwise it is an infinite computation. More examples of semantics on simple languages (λ-calculus, small imperative language, abstract machine, etc.) can be found in [START_REF] Leroy | Coinductive Big-step Operational Semantics[END_REF]. Now let us define our expressions using an inductive: (integers are noted Z). Now, its natural (big-step) semantics i.e. abstract finite evaluation to integers would be defined by the following inductive:

Inductive eval: expr → Z → Prop := eval_num: ∀n:Z, eval (num n) n | eval_plus: ∀e1 e2 n1 n2, eval e1 n1 → eval e2 n2 → eval (plus e1 e2) (n1+n2) | eval_div: ∀e1 e2 n1 n2, eval e1 n1 → eval e2 n2 → n2 = 0 → eval (div e1 e2) (n1/n2) and we noted (eval_expr e n) the evaluation of expression e to integer n. Now the determinism of the semantics is expressed with the following lemma and its proof: A parameter does not strictly have a code in why (or bsp-why). Thus, it is impossible to give a semantics execution of it with only the parameter specification. For this reason, we write the semantics assuming that we have a predicate, eval_param, that tells us how the parameters are evaluated. We can then say that a program is correct if its execution is correct for any such eval_param matching the parameters specification. Lastly, the rule for the synchronisation is as follow:

| eval_Esync : ∀e, eval_expr e (Esync) e (Outsync sync Evoid)

that is it does not modify the environment, simply returning a request for synchronisation.

Parallel Rules. We had two rules in the mathematical definition of the semantics, so we have two matching rules in our coq development.

( * Parallel evaluation of expressions. * ) Inductive eval_par: par_env → par_expr → par_env → par_outcome → Prop := | evalpar_nosync : ∀pe pa pe' po, final_outcome po → ∀i:pid, eval_expr i (pe i) (pa i) (pe' i) (po i) → eval_par pe pa pe' po | evalpar_sync : ∀pe pa pe' po po' pec pe'', ∀i:pid, eval_expr i (pe i) (pa i) (pe' i) (po i) → sync_outcome po → pec = comm(pe) → eval_par pec pa pe'' po' → eval_par pe pa pe'' po'

The final_outcome and sync_outcome are predicates describing outcomes where all processors reach a value, or a synchronisation request, respectively.

(b) Small-Step Semantics

The small-step semantics use the same environments and memory model as the big-step semantics.

Continuations. However, the semantics are significantly different. First of all, since we chose to use continuation semantics, we need to define the continuations. For each statement in the language, zero, one, or several continuations will be defined, depending on the number of computations that are done sequentially in the statement. For instance, labels and asserts will not have any continuation, but a "if" statement has one. The continuation stack is defined inductively, in a similar fashion as a list:

Inductive cont : Type := | Kempty : cont | Kif : expr → expr → cont → cont | Klet : ident → expr → cont → cont | Kassign : ident → cont → cont | Kraise : exn → cont → cont | Ktry : exn → ident → expr → cont → cont
Kempty is the empty continuation, and every other continuation is linked to a previous continuation.

Local States.

A major difference with the big-step semantics is the notion of program execution "state".

For the natural semantics, a state was simply the association of a program to execute and an environment.

In the small-step semantics however, we define four kinds of states.

1.

A "normal" state is similar to the notion of state in the natural semantics.

2. Result states are the values returned when a computation is finished.

3. An Error state is characterized by an error type, and a parameter value. [START_REF] Appel | Separation Logic for Small-Step Cminor[END_REF]. Finally, the synchronisation state is the result of a call to a synchronising parameter.

States are thus naturally defined as an inductive type, with four constructors. There is always a continuation in a state, but it can be the empty continuation.

Proof of the Translation

Our bsp-why tool transforms a parallel program, written in bsp-why, into a sequential why program.

We explained that the why program simulates the execution of the parallel program, thus allowing to reason on the sequential program, with the existing tools and provers, instead of the parallel program. However, doubts could be raised on the validity of the bsp-why transformation. In this section, we alleviate those doubts by proving formally that bsp-why does, indeed, properly simulates the parallel program. Our main result will be as follow: For a given parallel program P, if the transformation of P by bsp-why is correct, then P itself is correct. The proof of this theorem is detailed in this section.

Program Correctness

First, we need to formally define the notion of program correctness.

(a) An Unsatisfactory Solution: the Hoare Correctness

A why (or bsp-why) function (a program) is defined with a pre-condition and post-condition. As such, it can be seen as a Hoare triple, as defined in Section 2. [START_REF] Aldinucci | MUSKEL: An Expandable Skeleton Environment[END_REF]. With the standard Hoare logic, the meaning of a triple {p} e {q} is: if e is executed in a state satisfying its precondition p, then if it terminates, the resulting state satisfies its postcondition q. In [109], we gave a mathematical proof of the correctness of bsp-why, based on this definition. However, there are several limitations with this approach:

• The first drawback is that with the standard Hoare logic, only partial correctness is proved.

Termination must be ensured separately. It is however possible to extend the Hoare logic so that it includes the termination, by adding a variant in the while rule.

• Such a definition of the correctness only ensures the validity of the global Hoare triple defined by a function. If there are internal assertions in the definition of a bsp-why program, their correctness is not guaranteed by such a proof.

• Perhaps most importantly, in our proof based on the Hoare correctness, we had to assume that the bsp-why program was structurally sound, in regard to the synchronisations, for the transformation to be valid. The reason is linked to the previous point. In the bsp-why transformation, in order to ensure that there will be no deadlocks, an internal assertion is added whenever an instruction might be problematic. The assertion then generates a proof obligation, which is ensured to be correct by running the why tool on the sequential program. However, if we only assume that the sequential program is correct with the standard definition of a Hoare triple, then nothing guarantees that the assertion will be true.

The heart of the problem here is that the standard Hoare correctness does not give us all the information that we get with a why proof of the sequential program. why ensures both the total correctness, and the fact that every internal assertion will hold true. A definition of the correctness that includes all the information is needed to get a satisfactory proof of the bsp-why transformation.

(b) Correctness with Blocking Semantics

We achieve a more satisfactory definition of the correctness with the use of refined semantics for the bsp-why and why languages. Starting from the operational semantics that we defined in Section 6.1, we modify the rules so that in addition of describing the execution of a program, the semantics checks for the validity of the logic assertions that are present inside the program. In Fig. [START_REF] Armando | ltl Model Checking for Security Protocols[END_REF].7, we give the big-step blocking semantics for why. Only three rules are changed compared to the basic big-step semantics. They are the rules with a logical assertion: • For the assert rule, a premise is added to the inference rule, stating that the assertion is valid in the environment before the execution.

s, c ⇓ s, c s, e 1 ⇓ s ′ , v s ′ [x ← v], e 2 ⇓ s ′′ , o s, let x = e 1 in e 2 ⇓ s ′′ , o s, e 1 ⇓ s ′ , v s ′ [x ← v],
V(s, v) = v V(s, x) = s.E(x) V(s, !x) = s.E(x) V(s, f (t)) = [[f ]](V(s, t)) . . .
• For the post-condition rule, a premise is added to the inference rule, stating that the assertion is valid in the environment after the execution.

• For the loop rule, a premise is added to the inference rule, stating that the invariant is valid in the environment before the execution of an iteration.

The semantics does not check if the variant decreases. Instead, the termination will be as before an immediate consequence of any derivation s, e ⇓ s ′ , r. The rules for the bsp-why semantics are modified in a similar fashion. Only the local control flow rules are modified; the local bsp operations and global rules do not include logical assertions. Of course, we also need to define formally the s |= p relation. The bsp-why logic language was defined in Section 4.1, as an extension of the why logic language. Logic formulas are defined from predicates and logic terms. The rules for the predicates are given in Fig. 6.8. Logic terms are evaluated according the the environment, as described in Fig. 6.9, where the evaluation of the term t in the state s is written V(s, t). We remind that the notation s.E(x) is used to denote the value of x in the state s.

Lemma 10

Let a be a bsp-why expression, and a ′ = [[a]] its transformation into why.

s 1 ∼ s ′ 1 ⇒ s ′ 1 , [[a]] ⇓ s ′ 2 , o ′ ⇒ ∃s 2 , o such as (s1, a ⇓ s 2 , o) ∧ (s 2 ∼ s ′ 2 ) ∧ (o ∼ o ′ )
Proof. Detailed in the next pages.

(b) Equivalence Between Semantics Rules

As we described in Section 4.1, the bsp-why transformation is a complex process, with several distinct stages. Attempting to prove the whole transformation in one big induction would most likely be unsuccessful. A better approach consists in splitting the transformation, and proving that we maintain an equivalence for the different stages. In the course of the bsp-why transformation, the program is part of several data-types:

• At the beginning, we have a parallel program in the bsp-why language.

• It is then transformed by the block decomposition, where the biggest sequential blocks of codes are isolated. The result of this step is a block tree, which corresponds to a parallel program control flow tree.

• The leaves of the block tree are blocks, a purely sequential code, without synchronisation.

• Finally, a sequential why program is generated.

In order to show that the equivalence is maintained during the transformation, we need to give a semantics for all of these data-types. We already defined the blocking semantics for bsp-why parallel programs, as well as for why sequential code. The semantics of a block of sequential code can be seen as a subset of the ⇓ i semantics: apart from the fact that there are no synchronisation (and thus the result is either a value, or an exception), it is identical. The only thing missing is thus to give a semantics for the execution of a block tree. We will start by defining such a semantics. Block Semantics. The Block Semantics (⇓ b ) (not to be confused with blocking semantics) describes a theoretical execution of a parallel program, when it is in the block tree form. The block tree function is to extract the essential parallel structure of a program, while putting aside (in the so-called blocks) the sequential computations. As we explained in Section 4.1, we require that all processors always follow the same control flow inside the block tree. If a processor enters a branch of a if statement that affects the parallelism (for instance because there is a synchronisation inside), then all the other processors must do the same. Because of this, the block semantics does not allow processors to advance independently in the execution of the program. Instead, it requires a simultaneous progression in the control flow for all processors. The full rules for the block semantics are given in Fig. 6. [START_REF] Bacci | P3L: a Structured High-level Parallel Language, and its Structure Support[END_REF].

Let us explain more in detail the key rules:

• The first rule applies to the leaves of the tree, the blocks. In this case, the execution consists simply in executing the block of code on all processors simultaneously. Their execution is independant, and can not lead to a synchronisation.

• The bsp sync rule is significantly different from what we have seen before. Here, it simply has to do all the communications to end the super-step, then there is nothing more to do.

• For a generic rule, such as the instruction let x = t 1 in t 2 (where t 1 and t 2 are both block trees), t 1 is first executed, returning a parallel outcome. If this outcome is made of p values (the vector v, then we continue with the execution of t 2 , after updating the environment s so that for any processor i, x is now the i-th component of the parallel outcome returned by t 1 .

• In the semantics, E( v) is used to denote a parallel outcome where all the individual outcomes are the exception E, with a value v i .

• In the if rule, we enforce that all processors must always choose the same branch of a conditional instruction on the parallel level. For this reason, the parallel outcome must be either the vector with all true, or all false.

• Finally, the semantics is also blocking in the loop, assert and post rules. The assertions must be true on every processor for the rule to be applied.

s 1 , B ⇓ i s ′ 1 , r 1 . . . s p , B ⇓ i s ′ p , r p s, B ⇓ b s ′ , r s, bsp sync ⇓ b AllComm(s), void s, t 1 ⇓ b s ′ , v s ′ [x ← v], t 2 ⇓ b s ′′ , o s, let x = t 1 in t 2 ⇓ b s ′′ , o s, t 1 ⇓ b s ′ , E( v) s, let x = t 1 in t 2 ⇓ b s ′ , E( v) s, t ⇓ b s ′ , v s, x := t ⇓ b s ′ [x ← v], void s, t ⇓ b s ′ , E( v) s, x := t ⇓ b s ′ , E( v) s, t 1 ⇓ b s ′ , true s ′ , t 2 ⇓ b s ′′ , o s, if t 1 then t 2 else t 3 ⇓ b s ′′ , o s, t 1 ⇓ b s ′ , f alse s ′ , t 3 ⇓ b s ′′ , o s, if t 1 then t 2 else t 3 ⇓ b s ′′ , o s, t 1 ⇓ b s ′ , E( v) s, if t 1 then t 2 else t 3 ⇓ b s ′ , E( v) ∀k, s k |= i s, t; loop t {invariant i variant v} ⇓ b s ′ , o s, loop t {invariant i variant v} ⇓ b s ′ , o s, t ⇓ b s ′ , v s, raise (E t) ⇓ b s ′ , E( v) s, t ⇓ b s ′ , E ′ ( v) s, raise (E t) ⇓ b s ′ , E ′ ( v) s, t 1 ⇓ b s ′ , v s, try t 1 with E x → t 2 end ⇓ b s ′ , v s, t 1 ⇓ b s ′ , E( v) s ′ [x ← v], t 2 ⇓ b s ′′ , o s, try t 1 with E x → t 2 end ⇓ b s ′′ , o ∀i, s i |= p s, t ⇓ b s ′ , o s, assert {p}; t ⇓ b s ′ , o s, t ⇓ b s ′ , o ∀i, s ′ i |= p s, t {p} ⇓ b s ′ , o s, t ⇓ b s ′ , o s, l:t ⇓ b s ′ , o
s 1 ∼ s ′ 1 ⇒ s ′ 1 , [[b]] i ⇓ s ′ 2 , o ′ ⇒ ∃s 2 o such as (s1, b ⇓ i s 2 , o) ∧ (s 2 ∼ s ′ 2 ) ∧ (o ∼ o ′ )
Proof. By induction of the sequential code b. It does not contain parallel code, so synchronisation and communications are not a concern here. We will not detail all the induction cases here, but give the relevant ones.

• If b is a term, Lemma 9 ensures that the outcome is the same for both derivations.

s 1 ∼ s ′ 1 ⇒ s ′ 1 , [[b]] ⇓ s ′ 2 , o ′ ⇒ ∃s 2 , o such as (s1, b ⇓ b s 2 , o) ∧ (s 2 ∼ s ′ 2 ) ∧ (o ∼ o ′ ) Proof. Here, [[b]] is a for loop, executing [[b]] i for i = 0 to p -1.
s 1 ∼ s ′ 1 ⇒ s ′ 1 , [[t]] ⇓ s ′ 2 , o ′ ⇒ ∃s 2 , o such as (s1, t ⇓ b s 2 , o) ∧ (s 2 ∼ s ′ 2 ) ∧ (o ∼ o ′ )
Proof. By induction on t.

• If t is a block, Lemma 13 gives the result directly. Theorem correct_transformation : ∀prog, wcorrect (bspwhy prog) → correct prog.

Related Work

To our knowledge, the first work on a formal operational semantic of bsp is [START_REF] Lecomber | A Semantics for Parallel Programming with bsp[END_REF]: the author gives a small-step semantic using its own primitives of its own core language. Neither mechanised work nor applications have been done. The interests and examples of the use of mechanised semantics for certified program verifiers are given in [START_REF] Chlipala | Modular Development of Certified Program Verifiers with a Proof Assistant[END_REF]. In [START_REF] Herms | Certification of a Chain for Deductive Program Verification[END_REF], the author gives a mechanised proof of the results of the weakest preconditions calculus used in why. A mechanised big-step semantic of why were given. The author used massively dependent types whereas we choose a simple model of the language in the spirit of [START_REF] Leroy | Formal Verification of a Realistic Compiler[END_REF]. But our work on the proof of the transformation (of bsp-why codes to why ones) and the results of [START_REF] Herms | Certification of a Chain for Deductive Program Verification[END_REF] can clearly be associated to gain more confidence in the outputs of bsp-why.

A work on proving determinism, using assertions in the code, of multi-threaded java programs with barriers can be found in [START_REF] Burnim | Asserting and Checking Determinism for Multithreaded Programs[END_REF]. The authors note that there seemed to be no obvious simpler, traditional assertions that would aid in catching non-deterministic parallelism. In our case of bsp programs, this work is simple -but still limited to bsp programs.

Another work on concurrent threading with barriers is [START_REF] Hobor | Barriers in Concurrent Separation Logic: Now With Tool Support! Logical Methods in[END_REF]. The authors have developed and proved sound a concurrent separation logic for barriers of threads. An interesting point is that the proofs are machine-checked in coq. The authors also showcase a program verification toolset that automatically applies the logic rules (Hoare logic) and discharges the associated proof obligations. It is thus a work for derivation of formal specification into correct parallel programs. The drawback (as in [START_REF] Villard | Proving Copyless Message Passing[END_REF] and partly in [START_REF] Nieto | Verification of Parallel Programs with the Owicki-Gries and Rely-Guarantee Methods in Isabelle/HOL[END_REF]) is that only programs with a predefined constant number of threads (e.g. two for a producerconsumer problem) can be considered. For hpc, we prefer to have correct programs for an unknown number of processors in a data-parallel fashion. Another interesting work is [START_REF] Malkis | Verification of Software Barriers[END_REF] in which, using the theorem prover coq, the authors give a mechanised deductive verification of shared-memory concurrent algorithms for software barriers (of synchronisation) of multi-threaded programs. But assertions are hard to understand and especially the number of threads is still bounded.

It is time to conclude and sketch possible improvements and future work. We first briefly summarize our contributions and then we discuss how close we have come to the initial goal of the thesis, namely to be able to prove parallel (bsp) programs. Finally, we discuss remaining challenges and future work.

Summary of the Contributions

Parallel/distributed programs have applications in many different areas because they offer the possibility of computing data much faster than sequential programs. However, even for small parallel programs, error is more often the rule, not the exception. That seems to create a need for verification of parallel programs. Verification is the rigorous demonstration of the correctness of a program with respect to a specification of its intended behaviour. And formal methods are increasingly being applied for this goal which also create a strong need for on-machine (mechanised) formalisation and verification of the used tools -programming language semantics, compilers, static/dynamic analysis, etc.

Since the seminal paper "Goto Statement Considered Harmful", structured sequential programming is the norm. And there are now many relevant methods and tools for the verification of sequential programs. But it is not the case for parallel programming. Beside a true zoo of parallel paradigms (models and languages), the main reason is that programmers have kept the habit to use low-level parallel routines (such as asynchronous and unbuffered send/receive of e.g. mpi) [START_REF] Gorlatch | Send-receive Considered Harmful: Myths and Realities of Message Passing[END_REF] or concurrent languages [START_REF] Lee | The Problem with Threads[END_REF]. They more or less manage the interleaving of the instructions adjuncting with the communications with the usual problems of date race, deadlocks and non-determinism. That makes the verification of parallel programs harder. By consequence, programmers cannot focus on the correctness of their algorithms in addition to the detection of typical bugse.g. such as buffer overflow. This makes the design of robust tools for the verification of parallel programs an important area of research.

In this thesis, we have presented an intermediate language aimed at the verification of bsp programs. It is an extension of the language of the verification tool why which is designed for sequential programs. We summarize the contributions done throughout this thesis thereafter and in Fig. 7.2.

Chapter 3: A comparison of different BSP libraries. In this chapter, we have tried to present the common routines for imperative bsp programming as well as more exotic bsp and dedicated languages. We mainly found buffered sending, drma primitives and collective operations. We also present common extensions (such as subgroup synchronisation, high-performance send/drma routines and thread migration) of the bsp model with their advantages and inconveniences. We have finished this chapter with a presentation of the common bugs, like deadlocks, that can be found in distributed/parallel computations and mainly in bsp programs.

Chapter 4: A tool for deductive verification of BSP programs. In this chapter, we have presented a spmd bsp extension of the why-ml language. Then, we gave a block transformation of the bsp-why-ml language into the why-ml one, featuring obligation generation for bsp programs -for the correctness. It is the main work of the tool bsp-why. The output programs rely on a generic library of logical axioms and definitions. The transformation uses a particularity of the bsp model: since bsp programs are decomposed in super-steps (local computations separated with global barrier of synchronisation), parallelism can be removed by replacing a portion of code between barriers with a loop to repeat that portion for every process. The bsp-why transformation is formally given for a core-calculus, mainly using inductive rules. We also gave some simple examples of the transformation to illustrate its working.

The bsp-why transformation processes as follow. First, decomposing the program into its blocks of local computations; then adding some piece of logical information for maintaining that every processor would execute the block properly; finally transforming each block by detecting which variable is 125

transformed into an array of p-values (where p is the number of processors) which also arise for logical assertions. why processed by applying a wp-calculus (for generating proof obligations) whereas bsp-why "sequentialise" (simulate) the parallelism by directly transforming the parallel program (with its logical annotations) into a sequential one. And then a wp-calculus can be applied. We think that building upon an existing tool for program verification (and not do this work from scratch) is quite appealing since generated proof obligations need a lot of work -in theory and in practice.

Finally, we have also presented how this transformation can be adapted to an extension of the bsp model, the subgroup synchronisation. We have shown that some modifications are needed as well as additional new logical assertions in the generated sequential why programs in order to achieve safety of execution and absence of deadlock. We believe that the restriction of the block decomposition still is an efficient heuristic for proving bsp algorithms.

Chapter 5: Applications of the BSP-Why tool. The bsp-why has first been applied to some classical bsp algorithm examples, namely reductions and sorting. We showed the results of our tool on these examples by counting how many generated proof obligations were automatically discharged. As might be expected, not all the goals are discharged and some still remain to be proved, using coq for example. Nevertheless, those for safety are discharged which is an encouraging first step -we have the proof for any number of processors. Thus, in view of this ratio "number to prove/proved", we believe our tool is far from perfect (notably for correctness) but nevertheless, it can rapidly increase the confidence to be placed in the code since at least the safety properties are massively proved automatically.

We have then used bsp-why for the formal analysis of model-checking (restricted to state-space generation) algorithms -with one dedicated to security protocols because model checkers are specialised software, using sophisticated algorithms, whose correctness is vital. For example, verifying security protocols is complex and often error prone: various attacks are reported in the literature to protocols thought to be "correct" for many years. There are now many tools that check the security of cryptographic protocols and model-checking is one of the solutions [START_REF] Comon-Lundh | How to Prove Security of Communication Protocols? A Discussion on the Soundness of Formal Models w.r.t. Computational Ones[END_REF]. But model checkers could miss a state which can be an unknown attack of the security protocol.

In this work, we focus on correctness of a well-known sequential algorithm for finite state-space construction (which is the basis for explicit model-checking) and distributed ones. We annotated the algorithms for finite set operations (available in coq) to obtain goals that were entirely checked by automatic solvers. These goals ensure the termination of the algorithms as well as their correctness for any successor function -assumed correct and generating a finite state-space. We thus gained more confidence in the code. We also hope to have convinced the reader that this approach is humanly feasible and applicable to real (parallel or sequential) model-checking algorithms.

Chapter 6: Mechanized semantics of a parallel core calculus. In this chapter, we have presented mechanized operational semantics for bsp programs using coq for a core-calculus of our bsp-why-ml language. Fig 7 .2 outlines the scheme of the results. We have given a big-step semantics as well as a small-step one and co-inductive rules for both semantics in order to reason on infinite computations. We have prove that they are both deterministic and equivalent. We have then extended this work for subgroup synchronisation -but the complete mechanised proof of equivalence of the semantics must still be done. Using the semantics, we also have sketched the proof of correctness of the bsp-why transformation ("sequentialisation"): transform (under some conditions) a bsp code with annotations into an equivalent sequential one.

The big-steps semantics uses two kinds of rules: one for local (sequential) computation and one for the whole parallel machine. There are also different kinds of returning values depending of the context: returning a truly value, an exception or a call to synchronisation. The last one is used in the parallel rules to finish the super-step and perform the communications. For this, the small-steps semantics has been written in a continuation style: the continuation keeps all the computations that still remain to perform. And there is no local rule to reduce a call to synchronisation: only a global rule, with all the continuations, can terminate the bsp super-step. Adding subgroup synchronisation does not really change the rules, except that only a group of processors synchronises which induces a "diamond" property. The proofs of confluence are fairly standard inductions. The proof of equivalence is also standard but required more efforts due to the nature of the semantics: continuations and a distributed memory model. Our mechanised semantics do not use at all dependent types (in contrary of [START_REF] Herms | Certification of a Chain for Deductive Program Verification[END_REF]) for the simple reason that we do not manage them in our proofs -even for p-values where p is a fixed integer and thus we could have p-vectors. Our semantics are thus close (in their design) to the ones of [START_REF] Leroy | Formal Verification of a Realistic Compiler[END_REF]: we want to extend this work for bsp programs so it seems natural to consider the same kind of semantics. Up to now, most (all?) of the tools for verification of parallel programs have never been formally proved to be sound. The verification tools needs thus to be trusted, even if it can become quite complex, in order to gain confidence in their results -the correctness of the verified programs. One can say that verification tools do not (always) respect the "de Bruijn criterion" that is having the correctness of the system depending of a trusted small kernel1 -such as coq. Given the difficulties and the amount of time needed to perform the soundness proof of the sequentialisation, one may wonder whether it would have been easier to avoid this semantics preservation proof altogether by proving the soundness of a wp-calculus directly on bsp-why-ml without going through an intermediate language as why-ml. At least, this would incite focusing on the real issues of a wp-calculus [START_REF] Herms | Certification of a Chain for Deductive Program Verification[END_REF][START_REF] Herms | Certification of a Tool Chain for Deductive Program Verification[END_REF][START_REF] Vogels | A Machine-checked Soundness Proof for an Efficient Verification Condition Generator[END_REF] and implicit block decomposition (to avoid deadlock) would have remained as hard. An "originality" of this chapter is that the semantics of the language as well as the transformation have been written in the specification language of coq. A part of the proof of observational semantic equivalence between the source and the generated code has been partially machine-checked using coq which ensures a better trust in the results. It also used an intermediate semantics. An executable compiler can be obtained by automatic extraction of executable ocaml code from coq -as in [START_REF] Herms | Certification of a Chain for Deductive Program Verification[END_REF][START_REF] Herms | Certification of a Tool Chain for Deductive Program Verification[END_REF][START_REF] Vogels | A Machine-checked Soundness Proof for an Efficient Verification Condition Generator[END_REF]. This work is our first experiment to create a certified software for deductive verification. The main goal of this work is an environment where programmers could prove correctness of their bsp programs in a trusted way.

To finish, Drawbacks and Lessons Learnt from this Work. Our bsp-why tool extends why for bsp algorithms and has been partially mechanically proved. We are not aware of such an environment (with such a confidence in the results) for parallel programs and for an unbound number of processors. It in part meets the request of a proof tool presented in the introduction. We can enumerate the four distinct lacks:

1. The user still needs to annotate himself the programs. This can be tedious and needs a great degree of skill of the user. The ideal situation would be to have tools that automatically extract some properties (the full specification is not decidable), and indeed a great deal of research on this subject has been carried out, at least for some specific domains such as numerical computations [START_REF] Furia | Inferring Loop Invariants Using Postconditions[END_REF][START_REF] Siegel | Combining Symbolic Execution with Model Checking to Verify Parallel Numerical Programs[END_REF].

2. Currently, bsp-why does not work for c or java programs;

3. There are no "error messages" available (except syntax error) which does not make it a really useful and diffusable tool;

4. The sequentialisation induces too many goals and those that are not automatically proved are too hard to read; thus, bsp-why does not truly scale for big programs.

bsp-why is our first experiment about the design of a tool for deductive verification of parallel programs. It was rewarding and showed us unexpected difficulties. First, the invariants for the "for loop" were hard to find -a technical difficulty which did not imply a fundamental change of the tool. Second, testing the implementation were hard due to the big number of generated goals and the fact that automatic solvers could not prove anything even for trivial cases. Third, the small-step semantics with continuations is not a thing easily handled in coq and the sketch of the semantics preservation proof were hard to find because of a vicious circle in the proof: to prove the additional logical assertions due to the block decomposition, we need the fact that the sequential program is correct, but to prove it we need to prove that the block decomposition had done its job. Nevertheless, we have learnt many things. First, creating your own tool for deductive verification needs a great carefulness and having machinechecked proofs is not only a way to do well in the academic: it is during the design of the mechanised semantics that we highlighted some issues with the translation. Thus, by formalising our semantics and a part of the methods of bsp-why in coq we have gained a full understanding of the difficulty involved in designing correct proof methods for the verification of parallel programs. The level of detail required for such a formal study naturally leads to approaching each step of the formalisation with a critical eye, considering first to investigate alternatives that could simplify the formalisation. However, to understand the theorem proving techniques, which are involved in many formalisations, requires a great deal of time, effort and mistakes.

Second, an interesting part of a verification process is to understand the program and to find an intuitive proof of the correctness. However, the projection of this intuition into assertions implies, in general, changing and tuning the assertions too many times and a great deal of effort is expended to get the details right. For example, the generated conditions of bsp-why are really unreadable and only automatic provers could manage them. But automatic provers regularly give strange results: sometimes they found the result and sometimes not without any comprehensible reason; sometimes they failed due to the adding of a new axiom that has nothing to do with the current job. This is tiring when experiments are performed. bsp-why does not directly help in finding the right annotations but at least automatizes the iterative process of changing assertions and checking the proof again. From our experience, we do not recommend trusting paper and pencil proofs of correctness of parallel algorithms and programs. Especially if the invariants of the loops are implicitly given -which is traditionally done in the field of parallel model-checking.

Finally, we hope that the modest contributions presented in this thesis will convince the reader that the bsp-why tool will be viable for the verification of parallel programs for a large number of areas. And we will now present some remaining challenges, how to improve bsp-why and expected future work.

7.2

Future Work and Perspectives

Close Future Work (a) For the BSP-Why Tool

We need to test our method on realistic bsp computations even if the results of our examples are encouraging. Programs of [START_REF] Bisseling | Parallel Scientific Computation. A structured approach using bsp and mpi[END_REF][START_REF] Dehne | Special Issue on Coarse-grained Parallel Algorithms[END_REF] (LU decomposition, fast Fourier transform, etc.) will be used. The current prototype implementation of bsp-why is still limited. We plan to extend it in several ways. First, we intend to add a companion tool for c programs as in [START_REF] Filliâtre | The why/Krakatoa/Caduceus Platform for Deductive Program Verification[END_REF] and for java programs. The tool for c programming is a plug-in called jessie for the frama-c framework -note that another plug-in call wp is also available but it directly generates conditions without using why. jessie generates why codes from c ones. For java programs, the krakatoa tool will be considered -it also generates why codes. java programs which use hama [START_REF] Seo | hama: An Efficient Matrix Computation with the MapReduce Framework[END_REF]) could be considered as hama's communication primitives are close to those of bsp-why.

Second, conditions generated by why from bsp-why programs are not friendly, even for theorem provers (especially for them?), and a lot of them are not proved automatically. This is mainly due to the massive use of p-loops generated to simulate the p-asynchronous computations. Special tactics are needed to simplify the analysis. Also, many parts of the programs work in the same manner and only differ by the manipulated data. For example, variables such as counters for loops, local sorting of data. Finding these parts to factorize them and generate more friendly conditions would simplify the work of provers. In the same manner, syntactic sugar to manipulate the environment of communications (list of messages) is needed to facilitate the writing of logical assertions.

Third, they are still bugs in the current implementation. We need to correct them. And we need to deal with error messages to facilitate the use of the tool. Fourth, we currently work on why-2 whereas why-3 now exists -close to why-2 except for the logical parts and some new features such as patternmatching of union types. We need to adapt our work to the new syntax which should mainly be only a technical work. Fifth, we are currently not taking into account oblivious synchronisation (obl_sync(n)) but it seems feasible quickly by counting the number of received message after the communication and adding this new assertion: n strictly equal to this counting and every processor calls this routines. This will induce a safe use of oblivious synchronisation.

(b) Mechanised Semantics

First, we need to finish the proofs of soundness of the sequentialisation, of the equivalence of semantics for subgroups synchronisation and of the soundness of the sequentialisation with subgroups. It will be again mostly a technical work but needs to be done for a gain of confidence in our methodology.

We also plan to extend our semantics studies to the semantics of the aforementioned languages as in [START_REF] Blazy | Mechanized Semantics for the Clight Subset of the C Language[END_REF][START_REF] Herms | Certification of a Tool Chain for Deductive Program Verification[END_REF] for the CompCert (c compiler certified in coq) [START_REF] Leroy | Formal Verification of a Realistic Compiler[END_REF]. In [START_REF] Herms | Certification of a Tool Chain for Deductive Program Verification[END_REF], the author gives (and proves using coq) an equivalence of the big-step semantics of Clight with the semantics of a transformation of Clight into "why". The semantics were close to ours, and we believe that we could achieve this goal for bsp-why.

(c) Machine-checked Model-checking

We are currently proving state-space algorithms and not the effective code. Regarding the code structure, this is not really an issue and translating the resulting proof into a verification tool for true programs should be straightforward, if high level data-structures are used. Also, machine-checked model-checkers would certainly be less efficient than traditional ones. But they could be used in addition when it seems necessary to give greater confidence in the results. We also believe that another interesting application of a verified tool (such as we are envisaging it) would be to serve as a reference implementation that is used to compare the results of an efficient implementation over a set of benchmark problems.

Long Term Perspectives (a) For the BSP-Why Tool

bsp is an interesting model because it features a realistic cost model for an estimation of the execution time of its programs. Formally giving these costs by extended pre-, post-condition and invariants is an interesting challenge: one could speak of a cost certification. In fact, many scientific algorithms (numeric computations such as matrix ones) do not have too complicated complexities: it is often a polynomial number of super-steps. In the case of cloud-computing [START_REF] Armbrust | Above the Clouds: A Berkeley View of Cloud Computing[END_REF], we can imagine a scheduler server that distributes the parallel programs depending on the cost certificates to optimise power consumption. Our current case studies prove that bsp-why is an interesting possibility for this topics. For example, we have currently (not presented here) proven the bsp cost of the prefix computation by traditionally adding ghost variables for counting: (1) the operations; (2) the loop iterations; (3) and the number of super-steps. For communication we have tested: (1) manipulating a matrix which represents the counting of communication (a transposition allows to count the bsp communication) but automatic solvers were not able to prove anything; (2) adding a "heuristic" where during the "for loop" a processor increases the number of received messages of the other processors; automatic provers would be more efficient in this case but we are not yet convinced that we can deal enough algorithms in this way. Worst-case, average case, amortised worst-case, complexity, etc. give different meanings and thus would need different studies.

Furthermore, in a previous work [START_REF] Fortin | From bsp Routines to High-performance Ones: Formal Verification of a Transformation Case[END_REF], we have mechanically proven a simple optimization of the source code that transforms traditional bsp routines to their high-performance versions. These routines are proposed to programmers by most bsp libraries for improved speedup of their programs even if they are unsafe: they are unbuffered and do not really follow the safe bsp model of execution. Replacing bsp routines by their high-performance pendants remains to the responsibility of the programmer or of a non-formally verified compiler analyser such as in [START_REF] Danalis | Automatic mpi Application Transformation with ASPhALT[END_REF]. By using the logical assertions (and those of the block decomposition), we believe that we will have more information to optimise the programs. Using our semantics, this new function of optimisation could (and should) be done using coq as in [START_REF] Fortin | From bsp Routines to High-performance Ones: Formal Verification of a Transformation Case[END_REF].

Last, there are many more mpi programs than bsp ones. Our tool is not intended to manage all mpi programs. It cannot be used to model asynchronous send/receive ones -with possible deadlocks depending on the mpi scheduler [START_REF] Siegel | Verifying Parallel Programs with mpi-Spin[END_REF][START_REF] Vo | Formal Verification of Practical mpi Programs[END_REF]. Only programs which are bsp-like (e.g. mpi's collective primitives) [START_REF] Cappello | On Communication Determinism in Parallel hpc Applications[END_REF] will be considered. Analysing mpi programs to find out which ones are bsp-like and to interpret them in bsp-why is a great challenge, as in [START_REF] Martino | Parallel Program Analysis and Restructuring by Detection of Point-to-point Interaction Patterns and their Transformation into Collective Communication Constructs[END_REF].

(b) Machine-checked Model Checking

Future goals are clear. First, we would like to adapt our work for true model-checking algorithms -as those for ltl/ctl* [START_REF] Bhat | Efficient On-the-Fly Model Checking for CTL*[END_REF]. Model-checking algorithms are mainly Tarjan like algorithms or ndfs (Nested Depth-First Searches) for considered Strongly Connected Components (scc) in graphs. This is challenging in general but using an appropriate tool, we believe that a team could "quickly" do it. Second, the successor function (computation of the transitions of the state-space) is currently an abstract function. We think about proving the work of [START_REF] Fronc | Towards a Certified Petri Net Model-Checker[END_REF] in a mechanically-assisted way in order to compensate this deficiency. Third, compression aspects like symmetry, partial order, etc. must be studied since they can generate wrong algorithms. The work of [START_REF] Turner | A Refinement-based Correctness Proof of Symmetry Reduced Model-checking[END_REF] which use the b method could be a good basis.

And to finish, all these approaches require mathematical foundations (ctl* as in [START_REF] Tsai | Formalization of CTL* in the Calculus of Inductive Constructions[END_REF], scc of proofgraphs for [START_REF] Bhat | Efficient On-the-Fly Model Checking for CTL*[END_REF], etc.) which need to be machine-checked. The effort for such a project and thus for verifying the whole stack of Fig. 5.10 is not at all within the reach of a single team. But our guess is that each of these single steps is largely feasible. Also, as for the problematic of compilers [START_REF] Leroy | Formal Verification of a Realistic Compiler[END_REF], machinechecked model-checkers would certainly be less efficient than traditional ones. But they could be used in addition when it is desired to give greater confidence in the results.

(c) Hybrid and Hierarchical Computing

The flat view of a parallel machine as a set of communicating sequential machines (as bsp reminds us) remains useful, but it is nowadays incomplete [START_REF] Balaji | mpi on Millions of Cores[END_REF]. For example, gpu processors have a master-worker architecture and clusters of multi-cores make the core share a few network cards, which implies a bottling and a congestion of the communications. Moreover, we can observe that heterogeneous chip multiprocessors (as cell and gpu's feature) present unique opportunities for improving system throughput and reducing processor power: the trend towards green-computing puts even more pressure on the optimal use of architectures that are not only highly scalable but hierarchical and non-homogeneous. We do not know how to design algorithms for nested level systems with the original flat bsp model. The hierarchical architectures share communication resources inside every level but not between different levels. The flat bsp cost model is not thus suitable for this kind of architectures which are the future of parallel and high-performance computing.

With these issues in mind, the authors of [START_REF] Li | SGL: Towards a Bridging Model for Heterogeneous Hierarchical Platforms[END_REF] proposed a language called sgl (for scatter/gather language) which is a master-worker language based on the multi-bsp model [START_REF] Valiant | A bridging model for multi-core computing[END_REF]: a multi-level variant of the bsp model -to our knowledge, the first multi-bsp model was proposed in [START_REF] Williams | The Heterogeneous Bulk Synchronous Parallel Model[END_REF]. The one of [START_REF] Valiant | A bridging model for multi-core computing[END_REF] is more realistic for "cluster of clusters of multi-cores with gpu processors". This recursive model abstracts parallel architectures as trees of processes (each branch is a master and a child, except for leaves) and thus needs recursive algorithms. But sgl does not seem appropriate for more complicated algorithms such as the ones dedicated to model-checking. By allowing only scatter and gather operations, it is not possible to have point-to-point communications. In [START_REF] Hamidouche | A Framework for an Automatic Hybrid mpi +openmp Code Generation[END_REF], the authors propose an explicit call to nested parallel computations: the nested bsp computation has just a pre-function to scatter the data and a post-function to gather them. We should continue such approaches.

Currently, bsp-why can only deal with the flat parallelism. But, as in [START_REF] Stewart | A Programming Model for bsp with Partitioned Synchronisation[END_REF], bsp-why can manage subgroup synchronisation. Subgroups could be the cores of the processors; another group could be the whole machine. In this way, we can deal with hybrid/hierarchical programs. But that is too static and explicit nested parallel computations should be considered in order to take into account this future architectures or programs. Finally, one could wish to mix bsp computations and less structured models. Future reflections will deal with how to mix deductive verification of different models for the same program.

Résumé.

Cette thèse s'inscrit dans le domaine de la vérification formelle de programmes parallèles. L'enjeu de la vérification formelle est de s'assurer qu'un programme va bien fonctionner comme il le devrait, sans commettre d'erreur, se bloquer, ou se terminer anormalement. Cela est d'autant plus important dans le domaine du calcul parallèle, où le coût des calculs est parfois très élevé. Le modèle bsp (Bulk Synchronous Parallelism) est un modèle de parallélisme bien adapté à l'utilisation des méthodes formelles. Il garantit une forme de structure dans le programme parallèle, en l'organisant en super-étapes où chacune d'entre elle est composée d'une phase de calculs, puis d'une phase de communications entre les unités de calculs. Dans cette thèse, nous avons choisi d'étendre un outil actuel pour l'adapter à la preuve de programmes bsp. Nous nous sommes basés sur why, un vcg (générateur de condition de vérification) qui a l'avantage de pouvoir s'interfacer avec plusieurs prouveurs automatiques et assistants de preuve pour décharger les obligations de preuves. Les contributions de cette thèse sont multiples. Dans un premier temps, nous présentons une comparaison des différentes librairies bsp disponibles, afin de mettre en évidence les primitives de programmation bsp les plus utilisées, donc les plus intéressantes à formaliser. Nous présentons ensuite bsp-why, notre outil de preuve des programmes bsp. Cet outil repose sur la génération d'un programme séquentiel qui simule le programme parallèle, permettant ainsi d'utiliser why et les nombreux prouveurs automatiques associés pour prouver les obligations de preuves. Nous montrons ensuite comment bsp-why peut-être utilisé pour prouver la correction de quelques algorithmes bsp simples, mais aussi pour un exemple plus complexe qu'est la construction distribuée de l'espace d'états (model-checking) de systèmes et plus particulièrement dans les protocoles de sécurité. Enfin, pour garantir la plus grande confiance possible dans l'outil bsp-why, nous formalisons les sémantiques du langage dans l'assistant de preuve coq. Nous démontrons également la correction de la transformation utilisée pour passer d'un programme parallèle à un programme séquentiel.

Mots clefs.

Parallélisme, bsp, Preuves Déductives, why, vcg, State-space, Model-checking, Sémantiques formelles, coq Abstract. This thesis falls within the formal verification of parallel programs. The aim of formal verification is to ensure that a program will run as it should, without making mistakes, blocking, or terminating abnormally. This is even more important in the parallel computation field, where the cost of calculations can be very high. The bsp model (Bulk Synchronous Parallelism) is a model of parallelism well suited for the use of formal methods. It guarantees a structure in the parallel program, by organising it into super-steps, each of them consisting in a phase of computations, followed by communications between the processes. In this thesis, we chose to extend an existing tool to adapt it for the proof of bsp programs. We based our work on why, a vcg (verification condition generator) that has the advantage of being able to interface with several automatic provers and proof assistants to discharge the proof obligations. There are multiple contributions in this thesis. In a first part, we present a comparison of the existing bsp libraries, in order to show the most used bsp primitives, which are the most interesting to formalise. We then present bsp-why, our tool for the proof of bsp programs. This tool generates a sequential program to simulate the parallel program in input, thus allowing the use of why and the numerous associated provers to solve the proof obligations. We then show how bsp-why can be used to prove the correctness of some basic bsp algorithms. We also present a more complex example, the generation of the state-space (model-checking) of systems, especially for security protocols. Finally, in order to ensure the greatest confidence in the bsp-why tool, we give a formalisation of the language semantics, in the coq proof assistant. We also prove the correctness of the transformation used to go from a parallel program to a sequential program.

Keywords.

Parallelism, bsp, Deductive Verification, why, vcg, State-space, Model-checking, Formal Semantics, coq

Figure 1 . 1 .Figure 1 . 2 .

 1112 Figure 1.1. Une comparaison des bibliothèques bsp

2 let 3 while todo = ∅ do 4 let s = pick todo in 5 known←!known ⊕ s; 6 let 7 while new_states = ∅ do 8 let new_s = pick new_states in 9 let tgt=cpu(new_s) in 10 if tgt=bsp_pid 11 then todo←!todo ⊕ new_s 12 elseFigure 1 . 3 .

 23456789111213 Figure 1.3. Algorithm parallèle (distribué) pour la construction de l'espace d'états en bsp-why-ml

Figure 2 . 1 .

 21 Figure 2.1. Inference rules for the Hoare logic

Figure 2 . 2 .

 22 Figure 2.2. Typical rules of the weakest precondition calculus.

Figure 2 . 3 .

 23 Figure 2.3. A simple example of why-ml codes (from the why distribution): the binary search.

Fig. 3 .

 3 Fig. 3.1 summarises and compares the considered libraries. We give below more details about those features. Like many other communications libraries, bsp libraries adopt a Single Program Multiple Data (spmd) programming model. The task of writing an spmd program will typically involve mapping a problem that manipulates a data structure of size N into p instances of a program that each manipulate an N p sized block of the original domain.

Figure 3 . 1 .

 31 Figure 3.1. A comparison of bsp libraries.

Figure 3 . 2 .

 32 Figure 3.2. mpi collective operations.

Fig 3 . 2

 32 Fig3.2 resumes the below operations. The advantage of this kind of operations is that their implementation can be optimised for some specific architectures since they provide a clear semantics. They also simplify the reading and understanding of the programs. Collective operations of bsp libraries generally work on the whole machine and are synchronous. mpi ones work on a communicator that is a subgroup of processors that have been registered to be within the communicator -using a specific routine. Another collective operation is the sendrcv in which two processors send (synchronously) each other a value.Collective operations can be used to simulated the traditional bsp message passing and drma routines. This is the case of the bsponmpi library and bsml where all the routines accumulate values in some proper structures and finally, to finish the super-step, an mpi's alltoall is performed.

Figure 3 . 3 .

 33 Figure 3.3. The mpi primitives.

Figure 3 . 4 .

 34 Figure 3.4. The bsplib primitives.

Figure 3 . 5 .

 35 Figure 3.5. drma bsp operations: "put" and "get".

Figure 3 . 6 .

 36 Figure 3.6. The pub primitives.

Figure 3 . 7 .

 37 Figure 3.7. Some ct primitives.

Figure 3 . 8 .

 38 Figure 3.8. The bsgp statements and primitives.

Figure 3 . 9 .

 39 Figure 3.9. The main methods of jbsp.

Hama [ 245 ]

 245 . hama is an Apache project for a pure java bsp computing framework on top of hdfs -Hadoop Distributed File System. Hadoop [[ 34 ]

  Configuration getConfiguration() Get the Hadoop configuration void sync() Barrier int getPeerIndex() Return the index of this peer String[] getAllPeerNames() The names of all the peers bsmp primitives: M getCurrentMessage() A received message in the queue int getNumCurrentMessages() The number of messages in the queue void send(String peerName, M msg) Send a data to another peer Stream I/O: void write(K2 key, V2 value) Writes a key/value pair to the output collector boolean readNext(K1 key, V1 value) Test if there are no records to read anymore KeyValuePair<K1, V1> readNext() Reads the next key value pair

Figure 3 . 10 .

 310 Figure 3.10. The main methods of hama.

Figure 3 . 11 .

 311 Figure 3.11. The main methods of bsponmulticore-java.

  public void bsp_sync(){ double[] vec_steps = new double[MAX_SUPERSTEPS]; computeSuperstepTime(vec_steps); if (isSuperstepRescheduling()) { next_call = computeBalance(vec_steps); exchageDataAmongSetManagers(computePM()); machine = willMigrate(); if (machine != null) bsp_migrate(machine); }But he can also choose its own heuristics for migration. Fig3.13 shows the primitives.

Figure 3 . 12 .

 312 Figure 3.12. The main routines of pub-web.

  Barrier void run(int p)Launch the bsp computation with p objects int bsp_nprocs() The number of objects in bsp_pid()The "id" of the object bsmp primitives: void bsp_put(Object o, int dest) Sending a value Object bsp_get(int id)Reading a sending value from object "id" Migration: void bsp_migrate(int dest) Migrate the caller object to another processor

Figure 3 . 13 .

 313 Figure 3.13. The main methods of jMigBSP.

(

  b) BSML, BSP++ and BSP-Python BSP Programming in ML[START_REF] Gesbert | Bulk Synchronous Parallel ml with Exceptions[END_REF]. bsml is an extension of ml to code bsp algorithms. bsml uses a small set of primitives (over a parallel data structure, called parallel vector) which are currently implemented as a parallel library [[ 37 ]] for the ml programming language objective caml, i.e. ocaml [[ 38 ]]. All communications in bsml are collective and deadlocks are avoided by a strict distinction between local and global computation. Two features of bsml are (1) its deterministic semantics; and

  Facilities: const string& vertex_id() const The "id" of a vertex int64 superstep() const The total number of threads const VertexValue& GetValue() Get the value of a vertex const int NumVertices() Get the number of vertices const int NumEdges() Get the number of edges VertexValue * MutableValue()

Other frameworks .

 . Several open source initiatives of pregel exist such as Giraph [[ 39 ]] and GoldenOrb [[ 40 ]]. Their implementations relies on Hadoop and their apis are for java.

  ( * * * * * * * * * * * * BSP * * * * * * * * * * * * ) if (bsp_pid()==0) bsp_sync(); else asynchronous_computation(); ( * * * * * * * * * * * * MPI * * * * * * * * * * * * ) comm=MPI_COMM_WORLD; MPI_Comm_rank(comm,&me); if (me==0) MPI_Barrier(comm); else asynchronous_computation();

  (b) Out-of-bound and Non-deterministic Communication Communication can be done outside bounds of buffers and thus bug. For instance: ( * * * * * * * * * * * * BSP * * * * * * * * * * * * ) int x[bsp_nprocs()]; bsp_push_reg((void * )x,bsp_nprocs() * sizeof(int)); bsp_sync(); ( * All processors except 0 write to the x of processor 0 * ) if (bsp_pid()!=0) bsp_put(0,(void * )x,(void * )x,bsp_pid()+1,1 * sizeof(int)); bsp_sync(); ( * * * * * * * * * * * * MPI * * * * * * * * * * * * )

  int x[bsp_nprocs()]; bsp_push_reg((void * )x,bsp_nprocs() * sizeof(int)); bsp_sync(); bsp_put(0,(void * )x,(void * )x,0,1 * sizeof(int)); bsp_sync();

Contents 4 . 1

 41 The BSP-Why-ML Intermediate Language

Figure 4 . 1 .

 41 Figure 4.1. Main idea of the transformation.

Figure 4 . 2 .

 42 Figure 4.2. Syntax of bsp-why: expressions (left), assertions, types and programs (right).

  block of local computations and sending values * ) while(!i<nprocs) do { invariant ... variant nprocs-i} bsp_send i m; i:= !i+1 done; ( * end of the first super-step * ) bsp_sync(); ( * another block of local computations by reading a sended value * ) y:=bsp_findmsg 0; x:=!y+1 let SimpleCode2 = ( * doing p super-steps * ) while !j<=nprocs do { invariant ... variant nprocs-j} bsp_sync(); j:=!j+1 done

Figure 4 . 3 .

 43 Figure 4.3. Simple examples of bsp-why codes.

try loop if e 1

 1 then e 2 else raise Exit {invariant p variant t} done with Exit → void end

Figure 4 . 4 .

 44 Figure 4.4. bsp-why code of the direct prefix computation (left) and with its logical annotations (right).

  parameter ref_set : x:'a ref → v:'a → {} unit writes x { x = v }

Figure 4 . 5 .

 45 Figure 4.5. Use of a global array drma communication.

  type rvaluet logic rvalue_get : rvaluet, int, int → value parameter envR : rvaluet fparray ref parameter bsp_findmsg: src:int → n:int → {} value reads proc_i,envR {result=rvalue_get(paccess(envR,proc_i),src,n)}

  parameter bsp_sync : unit → {} unit writes envCsend, envR, ... {comm(envCsend@, envCsend, envR, ...) }

Figure 4 . 6 .

 46 Figure 4.6. Scheme of transformation of bsp-why programs into why ones.

if(bsp_pid % 2 = 1 )

 21 then bsp_push(x) else bsp_push(y) we will have the following global array: P1 P2 P3 P4 P5 P6 . . . . . . . . . . . . . . . .

Figure 4 . 7 .

 47 Figure 4.7. Main idea of the sequential block decomposition.

Figure 4 . 8 .

 48 Figure 4.8. Tagging of the parallel parts of the code of the core-calculus.

Figure 4 . 9 .

 49 Figure 4.9. Abstract syntax trees of SimpleCode1 (left) and SimpleCode2 (right).

Figure 4 . 10 .

 410 Figure 4.10. Application of the tagging algorithm for SimpleCode1 (left) and SimpleCode2 (right).

Figure 4 . 11 .

 411 Figure 4.11. Application of the block decomposition for SimpleCode1 (left) and SimpleCode2 (right).

  rec blocktree(tree) = match tree with | ... → ... | bspIfThenElse(e1,e2,e3),tag → if(!tag) then Block(tree) else blockIfThenElse(blocktree(e1),blocktree(e2),blocktree(e3)) | ... → ...

Fig. 4 .

 4 Fig. 4.11 gives the results of the simple examples of Fig.[START_REF] Appel | Separation Logic for Small-Step Cminor[END_REF].[START_REF] Alt | Algorithm Design and Performance Prediction in a java-Based Grid System with Skeletons[END_REF]. Fig.[START_REF] Appel | Separation Logic for Small-Step Cminor[END_REF].12 gives the formal definition of this block decomposition for the core-calculus. The first rules says that if the expression e is tagged as f alse then it can be a full sequential block which contain e. Otherwise, we build a tree like the expression.

  e), true > = raise (E < e >) < try e1 with E e2 → e3 end, true > = try < e1 > with E < e2 >→< e3 > end < e1 e2, true > = < e1 > < e2 > < l:e, true > = l: < e > < assert {p}; e, true > = assert {p}; < e > < e {p}, true > = < e > {p}

Figure 4 . 12 .

 412 Figure 4.12. Formal definition of the block decomposition.

Figure 4 . 13 .

 413 Figure 4.13. Example SimpleCode1, with a different parsing of the sequences.

Figure 4 . 14 .

 414 Figure 4.14. Re-organising sequences.

Fig. 4 .

 4 14. This is done by a simple recursive transformation on the tree: let rec maxseq_tree(tree) = match tree with | ... → ... | bspSeq(e1,e2),t → begin match maxseq_tree(e1), maxseq_tree(e2) with | (a,false) , (bsqSeq((b,false),(c,true)),_) → bspSeq((bsqSeq((a,false),(b,false)),false),(c,true)),true | (bsqSeq((a,true),(b,false)),_) , (c,false) → bspSeq((a,true), (bsqSeq((b,false),(c,false)),false)), true | e'1 , e'2 → bspSeq(e'1,e'2),t end | ... → ...

  let rec whyblocktree(btree) = match btree with | ... → ... | blockLetin(v,e1,e2) → wblockLetin(v,whyblocktree(e1),whyblocktree(e2)) | ... → ... | blockBlock(b1) → wblockBlock(block_transform(b1))

Figure 4 . 15 .

 415 Figure 4.15. Transformation of the block tree in the core-calculus.

( b )Figure 4 . 16 .

 b416 Figure 4.16. Application of the tree transformation for SimpleCode1 (up) and SimpleCode2 (down).

Figure 4 . 17 .

 417 Figure 4.17. Formal rules for the transformation of sequential blocks.

  proc_i := 0 ; loop0: while (!proc_i < bsp_nprocs) do { invariant (proc_i>=0) and ... variant bsp_nprocs -proc_i } let i = ref (parray_get i_par !proc_i) in let m = ref (parray_get m_par !proc_i) in localloop0: while(!i<bsp_nprocs) do { invariant ... variant bsp_nprocs-i} bsp_send !i !m; i:= !i+1 done; parray_set i_par !proc_i !i; parray_set m_par !proc_i !m; proc_i:=!proc_i + 1 done; bsp_sync (void) ; proc_i := 0 ; loop1: while (!proc_i < bsp_nprocs) do { invariant (proc_i>=0) and ... variant bsp_nprocs -proc_i } let x = ref (parray_get x_par !proc_i) in let y = ref (parray_get y_par !proc_i) in y:=bsp_findmsg 0; x:=!y+1; parray_set x_par !proc_i !x; parray_set y_par !proc_i !y; proc_i:=!proc_i + 1 done

Figure 4 . 18 .

 418 Figure 4.18. Full generated why program of SimpleCode1.

  Fig. 4.18 gives the final results of this transformation for the simple examples of Fig.4.3.

Figure 4 . 19 .

 419 Figure 4.19. Rules of transformation of logic terms, and logic predicates.

Fig 4 .

 4 Fig4.20 gives the full why code from the direct prefix example of Fig.4.4.

Figure 4 . 22 .

 422 Figure 4.22. Solution to the block decomposition with exceptions.

  let rec tag_parallel(tree,exc_l) = match tree with | ... → ... | bspTryWith(e1,exc,e2),tag → if(tag_parallel(e1,l) || tag_parallel(e2,l)) then (tag←true; tag_parallel(e1,exc::l)) else tag←false; !tag | ... → ... | bspRaise(exc),tag → tag←in(exc,exc_l); !tag

Figure 4 . 23 .

 423 Figure 4.23. Main idea of the transformation with subgroup synchronisation.

Fig. 4 .

 4 Fig. 4.23 illustrates the main idea of the simulation of a bsp program with subgroup synchronisation by a sequential program. The block decomposition remains the same as before. However, if statements in the parallel structure need another treatment because they allow the program to branch over different paths of execution depending on the subgroups.In the abstract example of Fig.[START_REF] Appel | Separation Logic for Small-Step Cminor[END_REF].23, a set of processors S is computed using "eval" to know which processors will effectively run the same branch of the if. Then, for each sequential block, we do not run the code for every processor, instead restricting to the processors in S, in this example the processors that have effectively their "id" lower than 2. Now the synchronisation is performed as above but we also check if S and the communicator sub match to forbid a deadlock: one processor does not synchronise

Figure 4 . 24 .

 424 Figure 4.24. Block Decomposition with Exceptions.

2 let known = ref ∅ in 3 let todo = ref {s0} in 4 while todo = ∅ do 5 { 6 invariant ( 1 ) 7 and ( 2 ) 8 and ( 3 ) 9 and ( 4 ) 10 variant |StSpace \ known| 11 } 12 let s = pick todo in 13 known←!known ⊕ s; 14 todo← 17 {result=StSpace}Figure 5 . 4 .

 456172839410111213141754 Figure 5.4. Sequential why-ml algorithm.

  logic s0: state logic succ: state → state set logic StSpace: state set axiom contain_state_space: ∀ss:state set. StSpace ⊆ ss ↔ (s0 ∈ ss and (∀ s:state. s ∈ ss → s ∈ StSpace → succ(s) ⊆ ss))

2 let 3 while todo = ∅ do 4 let s = pick todo in 5 known←!known ⊕ s; 6 let 7 while new_states = ∅ do 8 let new_s = pick new_states in 9 let tgt=cpu(new_s) in 10 if tgt=bsp_pid 11 then todo←!todo ⊕ new_s 12 elseFigure 5 . 5 .

 23456789111255 Figure 5.5. Parallel (distributed) bsp-why-ml algorithm for state-space construction.

  Fig 5.7 gives the annotated sequential part of the generic bsp algorithm.

and ( 3 )Figure 5 . 6 .

 356 Figure 5.6. Annotated bsp generic algorithm for state-space construction.

Figure 5 . 7 .

 57 Figure 5.7. Local computations of the naive parallel algorithm.

(*Figure 5 . 8 .

 58 Figure 5.8. Dedicated bsp algorithms for state-space construction of security protocols.

Figure 5 . 9 .

 59 Figure 5.9. Annotated computations of the "Incr" algorithm.

( 14 )

 14 and (∀ i:int. isproc(i) → ∀e,e':state. e ∈ghost_known<i> → class(e,e') → e' ∈ghost_known<i>)

Figure 5 . 10 .

 510 Figure 5.10. Different ways for proving model-checking algorithms.

Lemma 1 ∀i

 1 ⇓ i is deterministic.

Proof . 2 ⇓

 .2 Trivially proved by induction on the evaluation.Lemma is deterministic.

Figure 6 . 1 .

 61 Figure 6.1. Big-step semantics: local sequential operations.

∞

  (s 0 , e 0 ), .. . , (sp-1 , e p-1 ) ⇓ ∞ ∀i s i , e i ⇓ i s ′ i , SYNC(e ′ i )AllComm{ (s ′ 0 , e ′ 0 ), . . . , (s ′ p-1 , e ′ p-1 ) } ⇓ ∞ (s 0 , e 0 ), . . . , (s p-1 , e p-1 ) ⇓ ∞

Lemma 3 ⇓

 3 and ⇓ ∞ are mutually exclusive.

Proof .

 . Trivially proved by induction and co-induction on the evaluations.

Figure 6 . 4 .

 64 Figure 6.4. Small-step semantics: local sequential operations.

  Inductive expr:Set := num:Z→ expr | plus:expr→ expr→ expr | div: expr→ expr→ expr | inf: expr

Figure 6 . 8 .

 68 Figure 6.8. Semantics of the why Logic Language (Predicates).

Figure 6 . 9 .

 69 Figure 6.9. Semantics of the why Logic Language (Terms).

Figure 6 . 10 .

 610 Figure 6.10. Big-step block semantics.

7. 1 .

 1 SUMMARY OF THE CONTRIBUTIONSPrograms in bsp-why-ml

Figure 7 . 1 .

 71 Figure 7.1. Scheme of the main results of the Chapters 4, 5 and 6.

128 CHAPTERFigure 7 . 2 .

 12872 Figure 7.2. Scheme of the contributions of the Chapter 6 -mechanized semantics investigation.

4

  CHAPTER 1. RÉSUMÉ Bibliothèque Langage bsmp drma Op. coll. Op. H.-P. Oblivious sync Sous-groupes Migr. threads Archi. hybride

	mpi	c/fortran oui	oui	oui	oui	oui	oui	non standard oui**
	bsplib	c	oui	oui	oui	oui	non	non	non standard non
	pub	c	oui	oui	oui	oui	oui	oui	non portable	non
	bsgp	c	non	oui	oui	non	non	non	oui	oui*
	ct	c++	non	oui+	oui	non	non	oui	non	oui*
	bsponmulticore c	oui	oui	non	oui	non	non	non	oui*
	bsponmulticore java	oui	oui	non	oui	non	non	pas encore	oui*
	NestStep	c	non	oui+	non	non	non	oui	oui	oui
	NestStep	java	non	oui+	non	non	non	oui	oui	oui
	pub-Web	java	oui	non	oui	non	oui	non	oui	non
	hama	java	oui	non	non	non	non	non	oui	oui**
	jbsp	java	oui	non	non	non	non	non	non	non
	jMigBSP	java	oui	non	non	non	non	non	oui	non
	bsml	ocaml	oui	non	non	non	non	non	non	non
	bsp++	c++	oui	non	non	non	non	non	non	oui
	bsp-python	python	oui	non	non	non	non	non	non	non
	bsp-why	WhyML	oui	oui	oui	non	pas encore***	oui	non	non
	avec les abbréviations suivantes :

• bsmp ⇒ Message Passing (send/receive) • drma ⇒ Direct Remote Memory Access • Op. coll. ⇒ Opérations/routines collectives • Op. H.-P. ⇒ Opérations Hautes-Performances (envois asynchrones/non bufferisés) • Oblivious sync ⇒ Oblivious synchronisation • Sous-groupes ⇒ Synchronisation sur des sous-groupes • Migr. threads ⇒ Migration de threads et équilibrage automatique des calculs répartis sur les processeurs • Archi. hybride ⇒ Architectures hybrides et hierarchiques (cluster de multi-coeurs) avec les précisions suivantes :

3.2 Some Details About Some BSP Libraries . . . . . . . . . . . . . . . . . . . . 3

  . . 3.1.2 Description of these Routines .. . . . . . . . . . . . . . . . . . . . . . . . . . ..2.1 The Message Passing Interface (MPI) . . . . . . . . . . . . . . . . . . . . . . .3.2.2 BSPLIB Like Librariesfor C . . . . . . . . . . . . . . . . . . . . . . . . . . . .3.2.3 BSP Programming over Multi-cores andGPUs . . . . . . . . . . . . . . . . . .3.2.4 BSP Programming inJava . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.5 Other Libraries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . 3.2.6 Which Routines for BSP-Why. . . . . . . . . . . . . . . . . . . . . . . . . . .

3.3 Common Errors Introduced in BSP Programming . . . . . . . . . . . . . . 3

  .[START_REF] Alt | Algorithm Design and Performance Prediction in a java-Based Grid System with Skeletons[END_REF].1 Common Errors andBugs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3.3.2 These Errors in BSP Programs

  . . 3.4.1 Parallel Libraries and Imperative Programming . . . . . . . . . . . . . . . . . . 3.4.2 Other Parallel Paradigms. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Direct Remote Memory Access

	CHAPTER 3. IMPERATIVE BSP PROGRAMMING	Library mpi bsplib pub bsgp c • bsmp ⇒ Message Passing (send/receive) Language bsmp drma Coll ops H.-P. ops Oblivious sync Subgroups sync Thread mig c/fortran yes yes yes yes yes yes not standard yes** Hybrid archi c yes yes yes yes no no not standard no c yes yes yes yes yes yes not portable no
	24
		]] which aims to provide a bsp library for

• drma ⇒ • Coll ops ⇒ Collective operations/routines • H.-P.

ops ⇒ High-Performances operations

	(asynchronous/unbuffered sends)
	• Oblivious sync ⇒ Oblivious synchronisation
	• Subgroups sync ⇒ Subgroups/Subsets synchronisation
	• Thread mig ⇒ Thread migration and automatic system balance of the computations over the processors
	• Hybrid archi ⇒ Hybrid and hierarchical architecture (cluster of multi-cores)
	and where:
	* : with the used of another library
	** : optimisations in some implementations
	*** : easy to add but currently not yet implemented
	+ : Work with shared data

  MPI_Datatype sendtype, void * recvbuf, int recvcount, MPI_Datatype recvtype, int root, MPI_Comm comm) int MPI_Allgather(void * sendbuf, int sendcount, MPI_Datatype sendtype, void * recvbuf, int recvcount, MPI_Datatype recvtype, MPI_Comm comm) int MPI_Allgatherv(void * sendbuf, int sendcount, MPI_Datatype sendtype, void * recvbuf, int * recvcounts, int * displs, MPI_Datatype recvtype, MPI_Comm comm) int MPI_Alltoall(void * sendbuf, int sendcount, MPI_Datatype sendtype, void * recvbuf, int recvcount, MPI_Datatype recvtype, MPI_Comm comm) int MPI_Alltoallv(void * sendbuf, int * sendcounts, int * sdispls, MPI_Datatype sendtype, void * recvbuf, int * recvcounts, int * rdispls, MPI_Datatype recvtype, MPI_Comm comm) int MPI_Alltoallw(void * sendbuf, int sendcounts[], int sdispls[], MPI_Datatype sendtypes[], void * recvbuf, int recvcounts[], int rdispls[], MPI_Datatype recvtypes[], MPI_Comm comm)

	Tools:
	int MPI_Init(int * argc,char * * * argv)	Initialize the mpi execution environment
	int MPI_Finalize(void)	Terminates mpi execution environment
	int MPI_Comm_size(MPI_Comm comm,int * size)	Determines the size of the group associated with a communicator
	int MPI_Comm_rank(MPI_Comm comm,int * rank)	Determines the rank of the calling process in the communicator
	Collective communicating operations:
	int MPI_Barrier(MPI_Comm comm)
	int MPI_Sendrecv(void * sendbuf, int sendcount, MPI_Datatype sendtype, int dest, int sendtag, void * recvbuf, int recvcount, MPI_Datatype recvtype,
	int source, int recvtag, MPI_Comm comm, MPI_Status * status)
	int MPI_Bcast(void * buffer, int count, MPI_Datatype datatype, int root, MPI_Comm comm)
	int MPI_Gather(void * sendbuf, int sendcount, MPI_Datatype sendtype, void * recvbuf, int recvcount, MPI_Datatype recvtype, int root, MPI_Comm comm)
	int MPI_Gatherv(void * sendbuf, int sendcount, MPI_Datatype sendtype, void * recvbuf, int * recvcounts, int * displs, MPI_Datatype recvtype, int root, MPI_Comm comm)
	int MPI_Scatter(void * sendbuf, int sendcount, MPI_Datatype sendtype, void * recvbuf, int recvcount, MPI_Datatype recvtype, int root, MPI_Comm comm)
	int MPI_Scatterv(void * sendbuf, int * sendcounts, int * displs, Collective reducing operations:
	int MPI_Reduce(void * sendbuf, void * recvbuf, int count, MPI_Datatype datatype, MPI_Op op, int root, MPI_Comm comm)
	int MPI_Allreduce(void * sendbuf, void * recvbuf, int count, MPI_Datatype datatype, MPI_Op op, MPI_Comm comm)
	int MPI_Reduce_scatter(void * sendbuf, void * recvbuf, int * recvcounts, MPI_Datatype datatype, MPI_Op op, MPI_Comm comm)
	int MPI_Scan(void * sendbuf, void * recvbuf, int count, MPI_Datatype datatype, MPI_Op op, MPI_Comm comm)
	int MPI_Exscan(void * sendbuf, void * recvbuf, int count, MPI_Datatype datatype, MPI_Op op, MPI_Comm comm)
	int MPI_Reduce_scatter_block(void * sendbuf, void * recvbuf, int recvcount, MPI_Datatype datatype, MPI_Op op, MPI_Comm comm)
	Communicator manipulation:
	int MPI_Comm_dup(MPI_Comm comm,MPI_Comm * newcomm)	Duplicates an existing communicator
	int MPI_Comm_free(MPI_Comm * comm)		Deallocation of the communicator
	int MPI_Comm_split(MPI_Comm comm,int color,int key,MPI_Comm * newcomm)	Creates new communicators based on colors and keys
	int MPI_Intercomm_create(MPI_Comm local_comm,int local_leader,
	MPI_Comm peer_comm,int remote_leader,int tag,MPI_Comm * newintercomm)	Creates a communicator from two others
	int MPI_Intercomm_merge(MPI_Comm intercomm,int high,MPI_Comm * newintracomm)	Creates a communicator from a communicator
	Windows for drma:
	int MPI_Win_create(void * base,MPI_Aint size,int disp_unit,MPI_Info info,MPI_Comm comm,MPI_Win * win)	Creation of a window
	int MPI_Win_free(MPI_Win * win)		Free a mpi window
	int MPI_Win_fence(int assert,MPI_Win win)		Fence synchronisation
	int MPI_Win_complete(MPI_Win win)		Complete a drma operations begun after a MPI_Win_start
	int MPI_Win_start(MPI_Group group,int assert,MPI_Win win)	Start drma accesses for mpi
	int MPI_Put(void * origin_addr,int origin_count,MPI_Datatype origin_datatype,int target_rank,
	MPI_Aint target_disp,int target_count,MPI_Datatype target_datatype,MPI_Win win)	Put data into a memory window on a remote process
	int MPI_Get(void * origin_addr,int origin_count,MPI_Datatype origin_datatype,int target_rank,
	MPI_Aint target_disp,int target_count,MPI_Datatype target_datatype,MPI_Win win)	Get data from a memory window on a remote process

  initialises the arguments in some architectures void bsplib_params_init(t_bsplib_params * parameter) initialises the bsp cost parameters void bsplib_done() exits and frees resources int bsp_nprocs(t_bsp * bsp) returns the number of processors in the group int bsp_pid(t_bsp * bsp) returns own processor-id in the group void bsp_sync(t_bsp * bsp) bsp synchronization of the group bsmp primitives: void bsp_send(t_bsp * bsp, int dest, void * buffer, int size) Bulk sending of a buffer void bsp_sendmsg(t_bsp * bsp, int dest, t_bspmsg * msg, int size) Bulk sending of a message t_bspmsg * bsp_createmsg(t_bsp * bsp, int size) create a message int bsp_nmsgs(t_bsp * bsp) the number of received messages or buffers t_bspmsg * bsp_findmsg(t_bsp * bsp, int proc_id, int index) find a message in the queue t_bspmsg * bsp_getmsg (t_bsp * bsp, int index) get a message in the queue void * bspmsg_data(t_bspmsg * msg) returns a pointer to the data of a message. int bspmsg_size(t_bspmsg * msg) returns the size of the message contain int bspmsg_src(t_bspmsg * msg) returns the node-id of the sender drma primitives: void bsp_push_reg(t_bsp * bsp, void * ident, int size) register a variable for remote access void bsp_pop_reg(t_bsp * bsp, void * ident) delete the registration of a variable

Tools: void bsplib_init(t_bsplib_params * parameter, t_bsp * bsp) initialises the bsp computation void bsplib_saveargs(int * argc, char * * * argv) void bsp_put(t_bsp * bsp, int destPID, void * src, void * dest, int offset, int nbytes) remote writing to another processor void bsp_get(t_bsp * bsp, int srcPID, void * src, int offset, void * dest, int nbytes) remote reading from another processor Subgroup routines: void bsp_dup(t_bsp * bsp, t_bsp * dup) a new group as a copy of the group void bsp_partition(t_bsp * bsp, t_bsp * sub, int nr, int * partition) creates a new subgroup void bsp_done(t_bsp * bsp) destroy a subgroup

  Managed Vector/Native Space Copying cat, repeat, replicate, replace, index, copy, newVector Vector Generators extract, copy, length Vector Utilities newNestedVector, applyNesting, copyNesting, getNesting, etc. Nested Vectors Element-wise: add, mul, div, equal, min, mod, greater, and, select, map, etc. Vector-vector addVectorScalar, equalVectorScalar, andVectorScalar, etc.

	Facilities:
	copyIn, copyin2D, copyin3D, copyout
		Vector-Scalar
	abs, not, log, sin, cos, etc.	Unary
	Collective Communication:
	addReduce, mulReduce, andReduce, reduce, etc. Reduction
	addScan, mulScan, andScan, scan, etc.	Scan/Prefix-Sum
	Permutation:
	pack, unpack		Pack/Unpack
	scatter, gather		Scatter/Gather
	leftShiftPermute, leftRotatePermute, shiftDefaultPermute, rotateDefaultPermute Shift/Rotate
	partition, unpartition		Partition
	defaultPermute, omegaPermute, butterflyPermute, distribute	Miscellaneous

  Thread manipulation: spawn(int n) {...} Creates n threads on the gpu to execute the enclosed statements thread.rank The rank (id) of a thread thread.size The total number of threads barrier bsp barrier barrier(RANK_REASSIGNED) Barrier with load-balancing of alive threads require {...} memory of the gpu controller par {...} Rank reassignment sorting. drma operations: thread.get as in bsplib (without the need of a push registering) thread.put as in bsplib

			Reducing barriers
	Collective data parallel primitives:
	reduce(op, x)		Reduction of x using associative operator op
	scan(op, x)		Forward exclusive scan of x using associative; overwrites x
	compact(list, src, keep, flag)	Stream compaction
	split(list, src, side, flag)	Stream splitting
	sort_idx(data)		Sorting
	Supported rank adjusting primitives:
	thread.kill(flag)	Kill the calling thread if flag is true
	thread.fork(n)	Fork n child threads
	thread.split(side)	Split threads
	thread.sortby(key)

  1 , E 2 ) but the evaluation of super E 1 E 2 is less costly because it merges Summary of the bsml primitives.the communication and the synchronization phases of E 1 and E 2 : using the superposition, the phases of asynchronous computation of E 1 and E 2 are run; then the communication phase of E 1 is merged with that of E 2 ; the messages are obtained by simple concatenation of the messages of each super-thread and only one barrier occurs; if the evaluation of E 1 needs more super-steps than that of E 2 then the evaluation of E 1 continues -and vice versa. bsml is implemented using mpi or tcp/ip.

	primitive	type	informal semantics
	≪ e ≫	t par (if e : t)	e, . . . , e
	pid (within a vector) int	i on processor i
	v (within a vector)

t (if v : t par) vi on processor i (if v = v0, . . . , vp-1 ) proj 'apar→ (int→ 'a) x0, . . . , xp-1 → (fun i → xi) put (int→ 'a)par→ (int→ 'a)par f0, . . . , fp-1 → (fun i → fi 0), . . . , (fun i → fi (p -1)) super (unit→ 'a)→ (unit→ 'a)→ 'a * 'b fa → f b → (fa (), f b ()) Figure 3.14. The C++ Version: BSP++ [147]. The bsp++ interface is a template/object-oriented implementation of bsml for c++. As bsml, it uses parallel vectors which are now represented as a c++ class. Functional programming is done using templates. Any bsp computation/section must be started with the use of the BSP_SECTION(){...} macro where ... is a code where is is possible to define parallel vectors and using the following routines, close to the ones of bsml:

  Get a pointer to a vertex enable changing the value bsp computations:virtual void Compute(MessageIterator * msgs);

		Local computations
	OutEdgeIterator GetOutEdgeIterator()	Get the output edges of a vertex
	void SendMessageTo(const string& dest_vertex,const MessageValue& message) void VoteToHalt()	Send a message (vertex to vertex)
	class PageRankVertex: public Vertex<double, void, double> {
	public:
	virtual void Compute(MessageIterator * msgs) {
	if (superstep() >= 1) {
	double sum = 0;
	for (; !msgs->Done(); msgs->Next())
	sum += msgs->Value();
	* MutableValue() =0.15 / NumVertices() + 0.85 * sum;
	}
	if (superstep() < 30) {
	const int64 n = GetOutEdgeIterator().size();
	SendMessageToAllNeighbors(GetValue() / n);
	}
	else {VoteToHalt();}
	}
	}

Barrier

Figure 3.15. The pregel's vertex methods.

  read a message from itself too, which does not exist. A segfault will occur for both the bsp program and a deadlock can occur for the mpi one.

	if (me==0)
	for(int i=0;i<=mpi_comm_size()-1;i++)
	mpi_ircv(i,0,z,comm);
	y+=z
	processor 0 will
	int me=bsp_pid();
	( * All processors except 0 send a message to 0 * )
	if (me!=0) bsp_send(0,(void * )me,sizeof(int));
	bsp_sync();
	( * processor 0 reads these messages * )
	if (me==0)
	for(int i=0;i<=bsp_nprocs()-1;i++)
	y+=(int)bspmsg_data(bsp_findmsg(i,0));
	( * * * * * * * * * * * * MPI * * * * * * * * * * * * )

* * * * * * * * * * * BSP * * * * * * * * * * * * ) comm=MPI_COMM_WORLD; MPI_Comm_rank(comm,&me); ( * All processors except 0 send a message to 0 * ) if (me!=0) mpi_isend(0,(MPI_INT)me,MPI_INT,comm); mpi_barrier(comm); ( * processor 0 reads these messages * )

  There are many parallel libraries but the two most frequently used are certainly mpi and open-mp. Both are working under c/fortran -even if we can find bindings for java, ocaml and other languages. mpi contains many operations such as asynchronous sending values, collective operations, parallel i/o accesses, etc. for distributed architectures. open-mp has been designed for working on shared-memory architectures and allows the creation/synchronisation of threads. It mainly works using annotations in the code, e.g. indicates which loop can be achieved in parallel.The "parallel loop paradigm" (where the compiler automatically distributes the computations) has given rise to many extensions of c and java. We can cite split-c[START_REF] Culler | Parallel Programming in Split-C[END_REF] [[ 41 ]], titanium and unified parallel c [95] [[ 42 ]], cilk [[ 43 ]], fortress [[44 ]], etc. Mainly, the management of the processes and the implementation are the only differences for these languages.

zpl [50] [[ 45 ]] (and its ancestor orca [11] [[ 46 ]]

  . . 49 4.1.1 Syntax of BSP-Why. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . 50 4.1.2 The Parallel Memory Model in Why . . . . . . . . . . . . . . . . . . . . . . . . 54

4.2 Transformation of BSP-Why-ML Programs: the Inner

Working . . . . . . 56 4.2.1 Generalities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . 56 4.2.2 Identification of Sequential Blocks . . . . . . . . . . . . . . . . . . . . . . . . . 57 4.2.3 Block Tree Transformation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 4.2.4 Translation of Sequential Blocks . . . . . . . . . . . . . . . . . . . . . . . . . . 62 4.2.5 Translation of Logic Assertions . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 4.2.6 Dealing with Exceptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 4.3 Dealing with Subgroup Synchronisation . . . . . . . . . . . . . . . . . . . . . 69 4.3.1 Subgroup Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 4.3.2 Transformation of Programs with the Subgroup Synchronisation . . . . . . . . 72 4.4 Related Work . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. 75 4.4.1 Other Verification Condition Generators . . . . . . . . . . . . . . . . . . . . . . 75 4.4.2 Concurrent (Shared Memory) Programs . . . . . . . . . . . . . . . . . . . . . . 75 4.4.3 Distributed and MPI Programs . . . . . . . . . . . . . . . . . . . . . . . . . . . 76 4.4.4 Proof of BSP Programs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77

  ::= c | x | !x | φ(t l , . . . , t l ) | old(t l ) | at(t l , L) | t l <i> | <x> | bsp pid | nprocs Predicates: p ::= A(t l , . . . , t l ) | ∀x : β.p | p ⇒ p | p ∧ p | . . .

						Assertions:
	Expressions: e	::=	| | | | | | | | | | | |	. . , te) let x = ref e in e te let x = e in e if e then e else e loop e {invariant p variant t l } L:e raise (E e) : τ try e with E x → e end assert {p}; e e {q} fun (x : τ ) → {p} e rec x (x : τ ) . . . (x : τ ) : β e e {variant t l } = {p} e	term declaration variable conditional infinite loop label exception catch it assertion test condition test pure function recursive one application	Logic terms: t Types: τ ::= β | β ref | (x : τ ) → κ κ ::= {p} τ ǫ {q} q ::= p; E ⇒ p; . . . ; E ⇒ p ǫ ::= reads x, . . . , x writes x, . . . , x raises E, . . . , E sync
			| | | | |	bsp push x bsp pop x bsp put e x y bsp get e x y bsp send e e	registering deregistering drma writing drma reading bsmp sending	Programs: P ::= d ::= | |	∅ | d P let x = e val x : τ exception E of β

l

  t 1 e 2 : t 2 e 1 ; e 2 : max(t 1 , t 2 ) e 1 : t 1 e 2 : t 2 let x = e 1 in e 2 : max(t 1 , t 2 ) e 1 : t 1 e 2 : t 2 let x = ref e 1 in e 2 : max(t 1 , t 2 ) if e 1 then e 2 else e 3 : max(t 1 , t 2 , t 3 ) e 1 : t 1 e 2 : t 2 while e 1 do e 2 done : max(t 1 , t 2 )

	e : t	e 1 : t 1	e 2 : t 2	e 3 : t 3
	x := e : t

bsp sync : true e : t raise (E e) : t e 1 : t 1 e 2 : t 2 e3 : t 3 try e 1 with E e 2 → e 3 end : max(t 1 , t 2 , t 3 ) e 1 : t 1 e 2 : t 2 e 1 e 2 : max(t 1 , t 2 ) e : t l:e : t e : t assert {p}; e : t e : t e {p} : t

  Take for example the following code: which can be used for instance for a logarithmic computation (see Section 5.1). This is a case where our block decomposition fails: not all the processors run the same bsp sync and our tool will generate unprovable assertions. But the program can be rewritten by factoring the two bsp sync 2 :

	if (bsp_pid >= !i) then
	begin
	computation 1 ;
	bsp_sync();
	computation 2 ;
	end
	else
	computation 3 ;
	bsp_sync;
	computation 4 ;

if (bsp_pid >= !i) then computation 1 ; else computation 3 ; bsp_sync(); if (bsp_pid >= !i) then computation 2 ; else computation 4 ;

  , z3 (2.6), yices (1.0.27), gappa (0.13.0) and cvc3 (2.2).

	Prog	Correctness/AP Safety/AP
	Direct Prefix 37/37	19/19
	Log Prefix	41/37	21/19
	psrs	51/45	27/27

  10 mins):

	part/Solvers	Total	alt-ergo	simplify	z3	cvc3	yices	vampire
	main	106	74	98	101	0	54	78
	successor	46	16	42	41	24	14	32
	exchange	24	20	22	23	0	16	15

  table, for each part of each parallel algorithm, we give the number of obligations and how many are discharged by the provers:

	Algorithm	Part	Total	alt-ergo	simplify	z3	cvc3	yices	vampire
	Incr
		main	109	50	93	85	0	0	85
		successor	105	55	102	101	77	0	73
		exchange	32	15	28	22	19	0	27
	Sweep
		main	129	62	114	109	0	0	92
		successor	107	58	103	102	81	0	78
		exchange	31	14	29	23	21	0	28
	Balance
		main	135	71	123	119	0	0	102
		successor	113	62	111	108	87	0	81
		exchange	38	16	31	29	22	0	29

Contents 6.1 Big-steps Semantics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 6

  .1.1 Semantics Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 6.1.2 Co-inductive Semantics . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . 101 6.1.3 Adding the Subgroup Synchronisation . . . . . . . . . . . . . . . . . . . . . . . 103

6.2 Small-steps Semantics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 6

  .2.1 Semantics Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 6.2.2 Co-inductive Semantics . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . 107 6.2.3 Equivalence Between the Semantics . . . . . . . . . . . . . . . . . . . . . . . . 107 6.2.4 Adding the Subgroup Synchronisation . . . . . . . . . . . . . . . . . . . . . . . 108 Mechanized Semantics in Coq. . . . . . . . . . . . . . . . . . . . . . . . . . . . 109 6.3.2 Memory Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 6.3.3 Environment of Execution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 6.3.4 Semantics Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . 112 6.3.5 Adding the Subgroup Synchronisation . . . . . . . . . . . . . . . . . . . . . . . 114

6.3 Semantics in

Coq . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109 6.3.1

6.4 Proof of the Translation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116 6

  .4.1 Program Correctness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . 116 6.4.2 Equivalence Between Elements of the Semantics . . . . . . . . . . . . . . . . . 118 6.4.3 Elements of Proof in Coq. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

122 6.5 Related Work

  s, c ⇓ i s, c s, e 1 ⇓ i s ′ , v s ′ [x ← v], e 2 ⇓ i s ′′ , o s, let x = e 1 in e 2 ⇓ i s ′′ , o s, e 1 ⇓ i s ′ , E(v) s, let x = e 1 in e 2 ⇓ i s ′ , E(v) s, e 1 ⇓ i s ′ , SYNC(e ′ ) s, let x = e 1 in e 2 ⇓ i s ′ , SYNC(let x = e ′ in e 2 ) s, e 1 ⇓ i s ′ , v s ′ [x ← v], e 2 ⇓ i s ′′ , o s, let x = ref e 1 in e 2 ⇓ i s ′′ , o s, e 1 ⇓ i s ′ , E(v) s, let x = ref e 1 in e 2 ⇓ i s ′ , E(v) s, e 1 ⇓ i s ′ , SYNC(e ′ ) s, let x = ref e 1 in e 2 ⇓ i s ′ , SYNC(let x = ref e ′ in e 2 ) ⇓ i s ′ , SYNC(e ′ ) s, x := e ⇓ i s ′ , SYNC(x := e ′ ) s, e 1 ⇓ i s ′ , true s ′ ,e 2 ⇓ i s ′′ , o s, if e 1 then e 2 else e 3 ⇓ i s ′′ , o s, e 1 ⇓ i s ′ , false s ′ , e 3 ⇓ i s ′′ , o s, if e 1 then e 2 else e 3 ⇓ i s ′′ , o s, e 1 ⇓ i s ′ , E(v) s, if e 1 then e 2 else e 3 ⇓ i s ′ , E(v) s, e 1 ⇓ i s ′ , SYNC(e ′ ) s, if e 1 then e 2 else e 3 ⇓ i s ′ , SYNC(if e ′ then e 2 else e 3 ) s, e; loop e {invariant i variant v} ⇓ i s ′ , o s, loop e {invariant i variant v} ⇓ i s ′ , o ⇓ i s ′ , SYNC(e ′ ) s, raise (E e) ⇓ i s ′ , SYNC(raise (E e ′ )) s, e 1 ⇓ i s ′ , v s, try e 1 with E x → e 2 end ⇓ i s ′ , v s, e 1 ⇓ i s ′ , E(v) s ′ [x ← v], e 2 ⇓ i s ′′ , o s, try e 1 with E x → e 2 end ⇓ i s ′′ , o s, e 1 ⇓ i s ′ , SYNC(e ′ )

	s, e ⇓ i s ′ , v s, x := e ⇓ i s ′ [x ← v], void s, e s, e ⇓ i s ′ , v s, e ⇓ i s ′ , E(v) s, x := e ⇓ i s ′ , E(v) s, e ⇓ i s ′ , E ′ (v) s, e
	s, raise (E e) ⇓ i s ′ , E(v)	s, raise (E e) ⇓ i s ′ , E ′ (v)

s, try e 1 with E x → e 2 end ⇓ i s ′ , SYNC(try e ′ with E x → e 2 end) s, e ⇓ i s ′ , o s, assert {p}; e ⇓ i s ′ , o s, e ⇓ i s ′ , o s, e {p} ⇓ i s ′ , o s, e ⇓ i s ′ , o s, e:l ⇓ i s ′ , o

  ′′ = s ′ .C put ⊕ {to, c, n} s, bsp put e x y ⇓ i s ′′ , voids, e ⇓ i s ′ , SYNC(e ′ ) s, bsp put e x y ⇓ i s ′ , SYNC(bsp put e ′ x y) s, e ⇓ i s ′ , SYNC(e ′ ) s, bsp get e x y ⇓ i s ′ , SYNC(bsp get e ′ x y) s, e ⇓ i s ′ , SYNC(e ′ ) s, bsp send x e ⇓ i s ′ , SYNC(bsp send x e ′ ) s, e ⇓ i s ′ , to 0 ≤ to < p {x → c} ∈ s ′ .E n th (s ′ .T , y) = n s ′′ = s ′ .C get ⊕ {to, n, x} s, bsp get e x y ⇓ i s ′′ , void s, e ⇓ i s ′ , to 0 ≤ to < p {x → c} ∈ s ′ .E s ′′ = s.C send ⊕ {to, c} s, bsp send x e ⇓ i s ′′ , void s, t 1 ⇓ i s ′ , to 0 ≤ to < p s ′ , t 2 ⇓ i s ′′ , n {to, n, c} ∈ s ′′ .R s, bsp findmsg t 1 t 2 ⇓ i s ′′ , c

			void
			s, nprocs ⇓ i s, p	s, pid ⇓ i s, i
	s, e ⇓ i s ′ , to	0 ≤ to < p s s, t ⇓ i s, to {x → c} ∈ s ′ .E n th (s ′ .T , y) = n 0 ≤ to < p n = Size(s.R, to)
			s, bsp nmsg(t) ⇓ i s, n
		Figure 6.2. Big-step semantics: local bsp operations.
			∀i s i , e i ⇓ i s ′ i , v i
		(s 0 , c 0 ), . . . (s p-1 , c p-1 ) ⇓ (s ′ 0 , v 0 ), . . . (s ′ p-1 , v p-1 )
	∀i s i , e i ⇓ i s ′ i , SYNC(e ′ i )	AllComm{(s ′ 0 , e ′ 0 ), . . . (s ′ p-1 , e ′ p-1 )} ⇓ (s ′′ 0 , v 0 ), . . . (s ′′ p-1 , v p-1 )
		(s 0 , e 0 ), . . . (s p-1 , e p-1 ) ⇓ (s ′′ 0 , v 0 ), . . . (s ′′ p-1 , v p-1 )
		Figure 6.3. Big-step semantics: global reductions.

  if e 1 then e 2 else e 3 . κ i ⇀ s, e 1 . (if then e 2 else e 3 ) . κ s, true . (if then e 2 else e 3 ) . κ i ⇀ s, e 2 . κ s, false . (if then e 2 else e 3 ) . κ i ⇀ s, e 3 . κ s, let x = e 1 in e 2 . κ i ⇀ s, e 1 . (let x = in e 2 ) . κ s, v . (let x = in e 2 ) . κ ← v], e 2 . κ s, let x = ref e 1 in e 2 . κ i ⇀ s, e 1 . (let x = ref in e 2 ) . κ s, v . (let x = ref in e 2 ) . κ try e 1 with E x → e 2 end . κ (try with E x → e 2 end) . κ s, v . (try with E x → e 2 end) . κ ′ v . (try with E x → e 2 end) . κ ′ v . κ s, E v . (try with E x → e 2 end) . κ

	s, nprocs . κ	i ⇀ s, p . κ
	s, pid . κ	i ⇀ s, i . κ
	s, i ⇀ s[x ← v], e 2 . κ
	s, x := e . κ	i ⇀ s, e . (x := ) . κ
	s, v . (x := ) . κ	i ⇀ s[x ← v], void . κ
	s, loop e {invariant i variant v} . κ	i ⇀ s, e; loop e {invariant i variant v}. κ
	s, raise (E e) . κ	i ⇀ s, e . (raise (E )) . κ
	s, v . (raise (E )) . κ	i ⇀ s, E v . κ
	s, i ⇀ s, v . κ
	s, E i ⇀ s[x ← v], e 2 . κ
	s, E v . k . κ

i ⇀ s[x i ⇀ s, e 1 . i ⇀ s, E i ⇀ s, E v . κ, k =

try with → end s, assert {p}; e . κ i ⇀ s, e. κ s, e {p} . κ i ⇀ s, e. κ s, e:l . κ i ⇀ s, e. κ

  ′ , κ if 0 ≤ to < p and s ′ = s.C send ⊕ {to, x} s, bsp put e x y . κ ′ , κ if 0 ≤ to < p and s ′ = s.C put ⊕ {to, x, y} s, bsp get e x y . κ ′ , κ if 0 ≤ to < p and s ′ = s.C get ⊕ {to, x, y} s, bsp push x . κ ′ , κ if s ′ = s.C push ⊕ {x} s, bsp pop x . κ ′ , κ if s ′ = s.C pop ⊕ {x} Figure 6.5. Small-step semantics: local bsp operations. , e 0 . κ 0 ), . . . , (s i , e i . κ i ), . . . , (s p-1 , e p-1 . .κ p-1 ) ⇀ (s 0 , e 0 . κ 0 ), . . . , (s ′ i , e ′ i . κ ′

			6.2. SMALL-STEPS SEMANTICS
	s, bsp send e x . κ	i ⇀ s, e . bsp send x . κ
	s, to . bsp send e x . κ
		i ⇀ s, e . bsp put x y . κ
	s, to . bsp put x y . κ
		i ⇀ s, e . bsp get x y . κ
	s, to . bsp get x y . κ
	s i , e i . κ i	i ⇀ s ′ i , e ′ i . κ ′ i
	(s 0

i ⇀ s i ⇀ s i ⇀ s i ⇀ s i ⇀ s i ), . . . , (s p-1 , e p-1 ) (s 0 , bsp sync . κ 0 ), . . . , (s p-1 , bsp sync . κ p-1 ) ⇀ AllComm{ (s 0 , κ 0 ), . . . , (s p-1 , κ p-1 ) } Figure 6.6. Small-step semantics: global reductions.

with the second rule: each processor is in the case of a bsp sync with its control stack. The AllComm function computes the communications, and returns the new environments. Then what remains to be executed is only the control stacks since the synchronisation has been performed -with communications. Finally, the whole evaluation ⇀ * is inductively define as follow:

s 1 ,

e 1 ⇀ s 2 , e 2 s 2 , e 2 ⇀ * s 3 , e 3 s 1 , e 1 ⇀ * s 3 , e 3 s, e ⇀ * s, e where (s, e) represents globally the p local configurations.

b) Global Reduction Rules Unlike

  in the big-step semantics, defining the global reduction rules in the small-step semantics is relatively straightforward. The small-step semantics aims at providing elementary reductions, so there is no reason to provide a rule matching the big-step AllComm. Instead, the Diamond rule can be directly translated to the small-step language.The global rules are now as follows: , e 0 . κ 0 ), . . . , (s i , e i . κ i ), . . . , (s p-1 , e p-1 . .κ p-1 ) ⇀ (s 0 , e 0 . κ 0 ), . . . , (s ′

	s i , e i . κ i	i ⇀ s ′ i , e ′ i . κ ′ i
	(s 0 i , e ′ i . κ ′ i ), . . . , (s p-1 , e p-1 )
	∀i ∈ C, O

s, bsp send cmt e x . κ i ⇀ s, e . bsp send cmt x . κ s, to . bsp send cmt e x . κ i ⇀ s ′ , κ if 0 ≤ to < p cmt and s ′ = s.C send cmt ⊕ {to, x} (i = bsp sync C . κ i (s 0 , O 0 ), . . . , (s p-1 , O p-1 ) ⇀ AllCommC{C, (s 0 , O 0 ), . . . , (s p-1 , O p-1 ) }

  Inductive pstep : pstate → pstate → Prop := | pstep_lstep : ∀i s s' ge, step i (s i) (s' i) → ∀j, j = i → (s j) = (s' j) → pstep (PState s ge) (PState s' ge) | pstep_sync : ∀cm s ge ge' k e e' sp, all_sync_cm cm s sp e k → comm2_cm cm sp e ge e' ge' → pstep (PState s ge) (mkpstate_cm cm e e' s k) ge').

  e 2 ⇓ s ′′ , o s, let x = ref e 1 in e 2 ⇓ s ′′ , o s, e ⇓ s ′ , v s, x := e ⇓ s ′ [x ← v], void s, e 1 ⇓ s ′ , true s ′ , e 2 ⇓ s ′′ , o s, if e 1 then e 2 else e 3 ⇓ s ′′ , o s, e 1 ⇓ s ′ , f alse s ′ , e 3 ⇓ s ′′ , o s, if e 1 then e 2 else e 3 ⇓ s ′′ , o

	s |= i	s, e; loop e {invariant i variant v} ⇓ s ′ , o	s, e ⇓ s ′ , v
		s, loop e {invariant i variant v} ⇓ s ′ , o	s, raise (E e) ⇓ s ′ , E(v)
		s, e 1 ⇓ s ′ , v		s, e 1 ⇓ s ′ , E(v)	s ′ [x ← v], e 2 ⇓ s ′′ , o
	s, try e 1 with E x → e 2 end ⇓ s ′ , v	s, try e 1 with E x → e 2 end ⇓ s ′′ , o
		s |= p	s, e ⇓ s ′ , o	s, e ⇓ s ′ , o	s ′ |= p	s, e ⇓ s ′ , o
		s, assert {p}; e ⇓ s ′ , o		s, e {p} ⇓ s ′ , o	s, l:e ⇓ s ′ , o
		Figure 6.7. Big-step Blocking Semantics for why.

s |= ∀x : β, p iif ∀v : β, s ⊕ {x = v} |= p s |= p ∧ p ′ iif s |= p and s |= p ′ s |= p ∨ p ′ iif s |= p or s |= p ′ . . .

  If from a state s ′ 1 , a ′ executes to a state s ′ 2 with an outcome o ′ , and if s ′ 1 is equivalent to a parallel state s 1 , then a will execute to a state s 2 and an outcome o, with s 2 equivalent to s 1 and o equivalent to o ′ :

  • A generic induction case would be the if-then-else statement. Let us assume b = if e 1 then e 2 else e 3 .The translation is [[b]]i = if [[e 1 ]] i then [[e 2 ]] i else [[e 3 ]] i .Several semantics rules might have been applied in the why derivation, depending on the result of the execution of e 1 . For s′ 1 , [[e 1 ]] i ⇓ s ′ 0 , o ′ 0 ,we have by induction s 0 and o 0 such as s 0 ∼ s ′ 0 and o 0 ∼ o ′ 0 . In the hypothesis that o ′ 0 = true, the result of the execution of [[b]] i is obtained by executing [[e 2 ]] i . By induction, we then have a result equivalent for the execution of e 2 , and thus b has a result equivalent to [[b]] i . The same holds true when o ′ 0 = f alse or if an exception is raised. • Other cases are similar. Let b be a bsp-why sequential block, and b ′ = [[b]] its transformation into why.If from a state s ′ 1 , b ′ executes to a state s ′ 2 with an outcome o ′ , and if s ′ 1 is equivalent to a parallel state s 1 , then b will execute to a state s 2 and an outcome o, with s 2 equivalent to s 1 and o equivalent to o ′ .

	Lemma 13

  The proof is thus an induction on p, by applying repeatedly the Lemma 12.

Block Tree Translation. Lemma 14 Let t be a bsp-why block tree, and a = [[t]] its transformation into why. If from a state s ′ 1 , a executes to a state s ′ 2 with an outcome o ′ , and if s ′ 1 is equivalent to a parallel state s 1 , then t will execute to a state s 2 and an outcome o, with s 2 equivalent to s ′ 2 and o equivalent to o ′ :

  The use of the valid parameter is the key point that allows the transformation to work. Since a has a derivation, the only rule that can apply is a rule in which valid([[t 1 ]]) has a derivation. Because we are using a blocking semantics, this means that the pre-condition of valid holds, hence either [[e 1 ]] returns a p-array with all true, or all false. By the induction hypothesis, this means we have a derivation s1, t 1 ⇓ b s 2 , o with o the parallel outcome with all values true. We can thus apply the block semantics rule for the if instruction. By applying the induction hypothesis once more with either t 2 or t 3 , we get the result. The other induction cases are similar to the if , without the difficulty introduced by the valid.Let a be a bsp-why expression, and t =< a > the block tree result of the block decomposition. If from a state s 1 , t executes to a state s 2 with an outcome o, then a will execute to the same state s 2 and the same outcome o:s 1 , < a >⇓ b s 2 , o ⇒ s1, a ⇓ s 2 , oProof. By induction on the block tree. The general induction case is straightforward. For t = if t 1 then t 2 else t 3 , inverting the block tree decomposition gives us that a = if a 1 then a 2 else a 3 , with t i =< a i >. Inverting the derivation of b gives us three possible rules with the if statement. Let us consider the example Finally, we can state the Theorem 1:

	Block Decomposition.
	Lemma 15

• For t = if t 1 then t 2 else t 3 . The translation is a = if valid([[t 1 ]]) then [[t 2 ]] else [[t 3 ]]. • •

These include numerical computations, analysis of texts in biology, social sciences and humanities, symbolic calculations such as model-checking, analysis of large graphs representing, for instance, social networks.

Two famous and dramatic examples are the first Ariane-V rocket, which was destructed following a software bug and the code controlling the Therac-25 radiation therapy machine, which was directly responsible for some patient deaths.

In 1949 already, Turing writes an article in which he formally proves an algorithm.

When at least two computation units access together to a common resource.

When at least two computation units are each waiting for the other to finish.

For performances issue, a bsp library can send messages during the computation phase of a super-step, but it is hidden to programmers.

bsp libraries are generally implemented using mpi[START_REF] Snir | mpi the Complete Reference[END_REF] or low level routines of the given specifics architectures.

The verification is performed during the execution of the program: this is not a a priori verification.

A property that is logically valid before the loop and during any step of the loop.

Many engineers do not want to write anything else than the programs and thus want push-button tools (such as modelchecking) to provide at least safety and liveness, there are some tools[START_REF] Kovács | Finding Loop Invariants for Programs over Arrays Using a Theorem Prover[END_REF] that automatically provide annotations for some properties. But defining the exact meaning of a computation is clearly a human activity. We have not yet studied this problem and we add assertions manually.

This also happens when modeling mpi[START_REF] Li | Formal Specification of the mpi-2.0 Standard in TLA+[END_REF]; it forces to use the smallest possible number of routines. But for mpi, this subset still largely bigger than bsp, a hundred of routines compare to approximatively ten of bsp.

We certainly missed some, considering the amount of existing ones, but we present those that are the most used.

( * * * * * * * * * * * * MPI * * * * * * * * * * * * ) MPI_Comm comm1,comm2; MPI_Comm_rank(MPI_COMM_WORLD, &me); color1 = me % 2; color2 = me % 3; MPI_Comm_split(MPI_COMM_WORLD, color1, me, &comm1); MPI_Comm_split(MPI_COMM_WORLD, color2, me, &comm2); if (me % 2==0) MPI_Barrier(comm1) else MPI_Barrier(comm2);

Note that when we started writing our implementation of bsp-why, the only version of why was why2. Due to a lack of time, we did not adapt our work to why3. However, the modifications introduced by why3 are mostly syntactic or in the logic language, and it should not be too difficult to adapt our work in the future.

• The parallel exchanges come from the bsp_sync() statement. It is important to understand that the bsp_send, etc. instructions are only requests for communication. As such, they modify the local environment, but they do not lead to immediate parallel code. The communications will be done during the synchronisation. For this reason, the whole while loop in SimpleCode1 is tagged as being not parallel.• In SimpleCode1, the only parallel nodes are sequence nodes. However, it if fully possible to have all of the control flow instructions tagged as parallel. If a bsp_sync appears inside a while loop, the loop will be tagged as parallel, etc. This is illustrated in SimpleCode2.

Note that doing this transformation automatically is perhaps possible in some specific cases but this is not the subject of this doctoral thesis.

http://www.lacl.fr/fortin/BSP-Why/

In the sense of independent of the considered model.

Additional codes not participating in the computation but accessing the program data and allowing the verification of the original code.

Using this semantics we also get for free the evaluation of control structures in c (e.g. break and continue in loops) if we want to move to a realistic programming language such as c.

That is, we derive for each possible constructor of an evaluation, all the necessary conditions that should hold for the evaluation to be proved by the constructor.

It is generally call "Trusted Code Base" (TCB) the part of the tool on which the soundness depends on. For example, coq have a very small TCB.

and (forall proc_j:int. 0<=proc_j<proc_i → (forall j:int. (proc_j + 1<=j<nprocs) → in_send_n(paccess(envCsend,proc_j),j-(proc_j+1),lcast_int (paccess(x,proc_j)))) and nsend(paccess(envCsend,proc_j))=nprocs-(proc_j+1)) and (forall proc_j:int. proc_i<=proc_j<nprocs → paccess(envCsend,proc_j) = paccess(envCsend@loop0,proc_j)) variant nprocs -proc_i } let y = ref (bsp_pid (void))+1 in while (!y < nprocs) do { invariant (forall j:int. (proc_i + 1<=j<y) → in_send_n(paccess(envCsend,proc_i),j-(proc_i+1),lcast_int (paccess(x,proc_i)))) and nsend(paccess(envCsend,proc_i))=y-(proc_i+1) variant nprocsy } bsp_send !y (cast_int (parray_get x !proc_i)) ; y:=!y+1 done; proc_i:=!proc_i + 1 done; bsp_sync (void); proc_i:=0; loop1: while (!proc_i<nprocs) do { invariant (proc_i>=0) and (forall proc_j:int. 0<=proc_j<proc_i → paccess(z,proc_j)=sigma_prefix(x, proc_j)) variant nprocs -proc_i } parray_set z !proc_i (parray_get x !proc_i); let y = ref 0 in while (!y < bsp_pid void) do { invariant paccess(z,proc_i)=paccess(x,proc_i)+sigma_prefix(x, y) variant proc_iy } parray_set z (!proc_i) ((parray_get z !proc_i) + (uncast_int (bsp_findmsg !y 0))); y:=!y+1 done; proc_i:=!proc_i + 1 done {forall proc_i:int. isproc(proc_i) → paccess(z,proc_i)=sigma_prefix(x, proc_i)} -→ then executes the synchronisation barrier. In the second loop, each processor computes the sum of all the received values. Note the use of our notations in the program: x designs the value on the current processor, <x> refers to the global array and x <i> refers to the value of x at processor i. envCsendIs is a syntactic sugar to describe the communication environment, without having to use the internal list description and its associated functions. The bsp_send and bsp_findmsg functions can be used to transfer any type of data. For this reason, we use the cast_int and uncast_int functions, that encapsulates the data in a generic value data type.

The generated why code is in Fig. 5.1 (right). The bsp-why engine has, as expected, separated the program into two sequential blocks, linked by the synchronisation operation. Around those two blocks, a while loop has been constructed, so that the code is executed sequentially for each processor proc_i. As described in Section 4. 2.4, the invariant generated for the while loops are quite complex, and for a better readability we omitted large parts of them. We can note that the distributed variables, such as x and z, are translated into arrays of size p, using the type parray. Reading or writing such a variable is done with the parray_get and parray_set functions, or in the logic world their counterparts paccess and pupdate.

Local variables, with a lifespan within a sequential block do not need to be translated into an array. For instance, an access to y will remain the same. Note that the why source code generated by bsp-why is actually not supposed to be manipulated by the end-user, and is in general significantly less readable by a human. It is now possible to use the generated code, and feed it to the why program, in order to generate the proof obligations for any supported back-end. The results will be presented in Section 5. 1.3.

(b) Logarithmic Reduction

The above reduction does not make use of parallelism and we may prefer to reduce in a multi-step manner, the classical logarithmic way, doing the combinations locally. In our second example, the algorithm combines the values of processors i and i + 2 n at processor i + 2 n for every step n from 0 to ⌈log 2 p⌉.

Parallel Sorting Algorithm

Parallel Sorting by Regular Sampling Algorithm. Our last example is the Parallel Sorting by Regular Sampling algorithm (psrs) of Schaeffer in its bsp version [START_REF] Tiskin | The Design and Analysis of Bulk-Synchronous Parallel Algorithms[END_REF]. The goal is to have data locally sorted and that processor i have smaller elements than those of processor i + 1. Data also need to be well balanced. We assume n elements to sort where p 3 ≤ n and elements are well distributed over the processorseach processor have n p elements. The psrs algorithm proceeds as follows. First, the local lists of the processors are sorted independently with a sequential sort algorithm. The problem now consists of merging the p sorted lists. Each process selects from its list p + 1 elements for the primary sample and there is a total exchange of these values. In the second super-step, each process reads the p × (p + 1) primary samples, sorts them and selects p secondary (main) samples. Note that the main sample is thus the same on each processor. That allows a global choice of how to remap the data. In the third super-step, each processor picks a secondary block and gathers elements that do belong to the assigned secondary block. In order to do this, each processor i sends to processor j all its elements that may intersect with the assigned secondary blocks of processor j. To simplify we suppose elements of the "same size". where the time of merging elements (in a sorting way) is of order of n/p. The code is given in Fig 5 .3. We currently make the hypothesis that arrays merged in this way give the expected result. A proof of this fact needs to be done in coq in the future.

The precondition (line 3) ensures that each processor has enough data. The bsp-why program is then made of four super-steps, since in addition to the three super-steps of the algorithm, a first super-step is necessary to register the variables for drma access. The code and its logical assertions are as follow:

• The first super-step of the algorithm comprises the lines 2 to 17. On each processor, the local data is first sorted (line 7), then a sampling is created. The super-step ends with the registration of a variable for the drma accesses.

• In the second super-step (lines 19 to 31), the samplings are exchanged between the processors, using the put routine. The invariant of the loop is similar to the one in the first loop of our prefix example: the loop is simply adding one message in the put message queue. The operation done during the loop is actually a total exchange of the sampling: each processor puts it on every other processor. As such, we could have instead used a global parameter with a synchronising effect.

• The third super-step of the program (lines 33 to 56) starts by a local sort on all the received sample values. A new sample is then extracted from the sample values. Because the sample array is the same on every processor and the sampling function is deterministic, the new sample will also be the same on every processor; this is stated in the assertion line 39. In the following while loop, a processor will send data to every other processor, according to the partition described by the second sampling. The selection of the values of a according to the sampling is a purely sequential process, and is thus abstracted by a parameter called selection. The communication of these data is done with a send operation, line 45. Once again, the invariant of the loop is straightforward, since each iteration only adds one send message in the communication environment.

• Finally, in the last super-step (lines 58 to 71), we start by resetting the array a. We then insert all the data received, starting from the processor 0. This is done by a call to the sequential parameter merge_arrays that merges two sorted arrays. The invariant for this loop is that after i iterations, a contains the merged data from the received messages from processor 0 to the processor i.

The post-condition of the function (lines 76 to 79) states that the array a is locally sorted, with the values on a processor i lower than the values on the next processor, and that it is globally the same array (same values) as the original array.

Small-steps Semantics

Big-step semantics offers rules that are both easily readable and intuitive. They are thus most adapted for a specification of the bsp-why language. For basic properties (such as the termination of a given program, etc.), these semantics allow for a natural proof. However, in order to formally prove more complex results, it is often necessary to reason precisely on the parallel execution of a program. Having a semantics that describes in the closest possible way the execution, including for instance the interleaving of the computations on the different processors, is needed. Proving the correctness of the bsp-why tool, i.e. that the generated sequential program will be correct if and only if the parallel program is correct, is such a result that benefits from additional control in the semantics. For this reason, we provide a small-step operational semantics for the bsp-why language. In this section, we first define the small-step semantics mathematically, then explain its definition in coq. We then prove the equivalence between the small-step semantics, and the big-step semantics that was given as the definition of the language. This proof is entirely checked by the coq proof assistant.

Semantics Rules

Small-step semantics specify the execution of a program, one step at a time For diverging programs, we note the whole (co-inductive) reduction ⇒ ∞ . All of our semantics are thus a set of "rewriting" rules. As we will see, the small-step semantic is harder to define than the big-step one.

(a) Problematic

As for the big-step semantic, most of rules are commons ones. Synchronisation is the only problem. A naive solution is the following global rule: that is all processors are waiting for a global synchronisation and then each processor executes what remains to be done. The problem with this rule is that it cannot evaluate a synchronisation inside a control instruction e.g. if e 1 then bsp sync else e 3 . Different solutions exist:

• Adding specific global rules for the synchronisation inside each control instruction; the drawback is that this implies too much rules;

• Using a global rule with "contexts" (a context is an expression with a hole): the bsp sync instruction replaces the hole within a context on each processor; the drawback is that the use of contexts is not friendly when using a theorem prover such as coq;

• In [START_REF] Tesson | Formal Semantics for the drma Programming Style Subset of the bsplib Library[END_REF] the authors propose the following rule: s, bsp sync i ⇀ s, W ait(skip) in adjunction with rules to propagate this waiting (as the ones of the big-step semantic) and the following rule

but two subtleties persist: (1) the rules add a skip instruction that complicates the proofs; (2) in their work, (e 1 ; bsp sync); e 2 cannot be evaluated, only e 1 ; (bsp sync; e 2 ) can.

To remedy to the latter problem, in [START_REF] Gava | Two Formal Semantics of a Subset of the Paderborn University bsplib[END_REF], we choose to add the congruence (equivalence) (e 1 ; bsp sync); e 2 ≡ e 1 ; (bsp sync; e 2 ) but that also complicates the proofs. Another solution would consist in having only lists of instructions (and not e 1 ; e 2 ) but that complicates the proofs too.

(b) Local Rules

The solution we propose is the use of a "continuation semantics", in the spirit of the semantics described in [START_REF] Appel | Separation Logic for Small-Step Cminor[END_REF] 1 . This semantics mainly allows a uniform representation of configurations that facilitates the design of lemmas. where we have performed a proof by induction on the evaluation of "e" and for each case, a proof by inversion 2 on the second evaluation.

The co-inductive for infinite computations would be defined as follows:

CoInductive eval_inf: expr → Prop := evalinf_inf: eval_inf inf | evalinf_plus1: ∀e1 e2 n1, eval e1 n1 → evalinf e2 → eval_inf (plus e1 e2) | evalinf_plus2: ∀e1 e2 n2, evalinf e1 → eval e2 n2 → eval_inf (plus e1 e2) | evalinf_div1: ∀e1 e2 n1, eval e1 n1 → evalinf e2 → eval_inf (div e1 e2) | evalinf_div2: ∀e1 e2 n2, evalinf e1 → eval e2 n2 → n2 = 0 → eval_inf (div e1 e2). | evalinf_div3: ∀e1 e2 n1 n2, eval e1 n1 → eval e2 n2 → n2=0 → eval_inf (div e1 e2) that is "inf" is an infinite computation and if one sub-expression of e 1 + e 2 or e1 e2 is "inf" then it is an infinite computation. Now the fact that the semantics are mutually exclusive is expressed and proved as follows:

Lemma eval_eval_inf_exclusive: ∀a v, eval a v → eval_inf a → False. that is by induction on the evaluation of "a" and proving by case that it is not possible to have the same co-inductive evaluation. Note the use of the deterministic result of "eval" to prove that it is impossible to have both n 2 = 0 and n 2 = 0 to split an infinite evaluation to a finite one.

For a small-step semantics, we will first define a one-step reduction with a left-to-right order for subexpressions: The small-step semantics i.e. abstract evaluation of expression e to integer n is noted (step_star e (num n)). Now we can have the following results. First the equivalence between small-step semantics and big-step ones:

( * finite evaluations * ) Lemma step_star_to_eval: ∀a v, step_star a (num v) ↔ eval a v.

( * infinite evaluations * ) Lemma step_int_to_eval_int: ∀a, step_inf a ↔ eval_inf a.

Now that the small-step semantics is deterministic (very easy to prove using above lemmas):

Lemma step_star_determinist: ∀e n1, step_star e (num n1) → ∀n2, step_star e (num n2) → n1=n2.

And to finish, that also for small-step semantics, finite and infinite reductions are mutually exclusive:

Definition notred (a: expr) : Prop := ∀b, ~(one_step a b). Lemma step_star_or_step_inf: ∀a, (exists b, step_star a b ∧ notred b) ∨ step_inf a

Memory Model

In bsp-why, all variables exchanged contain data of the generic type value, which can represent any elementary type. We define a corresponding type in coq:

Parameter value : Type.

A few special values are also defined, such as null, void, true, false, etc. The memory is then defined as a function from memory blocks to values. The blocks are numbered according to numbers: Thus the initial empty memory is the function that always returns the null value:

It is then easy to define a memory modification:

We then need to define variables. For this, we represente identifiers as positive numbers:

The link between a variable and its memory block is then stored as a part of the execution environment, which we will detail more in the next section:

Definition envE_t := ident → mblock.

Environment of Execution

We first define the environment for a single processor. The environment is composed of 8 parts, which are regrouped into a record type. where:

• The first component, envE, was described in the previous section. It is the part of the environment that describes the variables in the program, and give their associated memory block.

• The last component, mem, is the memory itself, which was also described in the previous section.

• The remaining components of the record all deal with the parallel communication model of bspwhy. envR is used to store the values that were received during the previous super-step, and that can be accessed trough the bsp_findmsg function. It is defined as a function, which takes in argument the pid and the number of the message, and returns its associated value.

Definition envR_t := pid → positive → value.

• Finally, envCs, envCp, envCg, envCpu and envCpo all share the same role and structure. They are lists keeping track of the parallel messages that will be sent during the next synchronisation. One list is used per type of message: send, put, get, push, and pop:

Definition envCs_t := list msg_send and envCp_t := list msg_put and envCg_t := list msg_get and envCpu_t := list msg_push and envCpo_t := list msg_pop

The types for representing the different kind of messages are themselves record type. Here is for instance the definition of the send messages:

Record msg_send : Type := mkmsg_send { ms_dest : pid; ms_value : value } Of course, bsp programs run on parallel machines, so we have to define the parallel environment. As usual, we do this by the use of a function from the pid to the local environments.

Definition par_env := pid → env.

Semantics Rules

It is now possible to define our semantics in coq, according to the rules given in previous sections. For the big-step operational semantics, there are two parts in the semantics. First, we will define the local reduction rules, which represent the evaluation on a single processor, and then we will give the parallel reduction rules.

(a) Big-step Semantics

Local Reduction Rules. As is usual in coq, the semantics rules are given as an inductive predicate. eval_expr i e a e' o defines the evaluation of the expression a in the environment e on the processor i.

Variable i: pid.

The definition of an outcome directly follows the definition given in Section 6.1: there are three possible outcomes, either the computation returns a value, raises an exception, or requests a synchronisation, with another expression remaining to be executed. The definition of the eval_expr predicate closely matches the mathematical definition of the semantics given in Section 6.1. We thus have several rules for each language instruction, depending on the kind of outcome obtained during the evaluation of the sub-expressions. There is typically one rule for a value outcome, one rule for an exception outcome, one rule for a synchronisation outcome, etc.. The complete rules are naturally available in the coq development. Let us focus of a few representative rules in the reduction.

The evaluation of a term is the most basic rule. It never returns an exception or a synchronisation request, and does not change the environment, thus it is the simplest rule:

| eval_Eterm : ∀(e:env) (t:term), eval_expr e (Eterm t) e (Outval (evalterm i t e))

We can see that the rule depends on a simple call to the evaluation of the term with the function evalterm. This matches the mathematical rule given before.

For most language instructions, we have at least three rules, as explained before. In the coq development, we distinguish the rules for when an exception occurs with a _e suffix in its name, and _s when a synchronisation happens. For instance, here are the three rules for the let ... in structure: Once again, these rules closely match the mathematical rules given before. The rule for the parameter application is a special case. Local Steps. A step in the semantics is defined as an inductive from states to states. The definition closely matches the rules that were given in Fig. 6. [START_REF] Appel | Separation Logic for Small-Step Cminor[END_REF]. As an illustration, here are the two rules for the "let" construction: From there, the transitive closure pstar is defined in a standard manner.

Adding the Subgroup Synchronisation

In order to allow the subgroup synchronisation, it is necessary to make some changes to the definitions of the parallel model. First, a subgroup is defined as a function from the processor identifiers to the booleans.

Definition subgroup := pid → bool.

As we explained in in Section 4.3, a communicator can not be simply seen as a subgroup. Instead, we define a communicator as a unique positive number, as we did for other identifiers. The next change concerns the communication environments. We saw that a message is now sent within the context of a communicator, and it needs to be reflected in the definition of the environment. This is simply done by modifying the envC definition: it is now a function from the communicators to the previous envC definition.

Record env : Type := mkenv { envE : envE_t; envR : envR_t; envC : comm → envC_t; mem : mem_t }.

Finally, we modify the definition of a bsp-why expression, to

Inductive expr : Type :

The only difference here is that every bsp primitive takes an additional argument, the communicator.

(a) Big-step Semantics with Subgroup Synchronisation

With these modifications, we can now change the semantics rules. The send rule changes as follows:

Inductive eval_expr: env → expr → env → outcome → Prop := ...

| eval_Esend : ∀e e' e'' a a' v v' cm, eval_expr e a e' (Outval v) → val_is_pid v → eval_expr e' a' e'' (Outval v') → eval_expr e (Esend cm a a') (add_envCs_cm e'' cm (mkmsg_send (val_to_pid v) v')) o_void ... Instead of adding the message directly in a global envC, it is added to the specific envC associated with the communicator cm, by the add_envCs_cm function.

We saw that apart from the bsp operations, the other local rules do not change significantly. However, we need to give definitions for the two candidate global rules, AllSub and Diamond.

First, we define the Diamond rule, that will be our final choice for the semantics.

Inductive eval_par1: genv → par_expr → genv → par_outcome → Prop := | evalpar1_nosync : ∀pe pa pe' po, final_outcome po → (∀ i:pid, eval_expr i (pe.(ge_penv) i) (pa i) (pe'.(ge_penv) i) (po i)) → eval_par1 pe pa pe' po | evalpar1_sync : ∀pe pa pe' pa' po po' pec pe'' id cm, sync_outcome_cm pe id cm pa' po → ∀i:pid, (pe.(ge_comms) cm) i = true → eval_expr i (pe.(ge_penv) i) (pa i) (pe'.(ge_penv) i) (po i) → ∀i:pid, (pe.(ge_comms) cm) i = false → ((pe.(ge_penv) i) = (pe'.(ge_penv) i)) ∧ (pa i = pa' i) → pec = comm_r id cm pe' → eval_par1 pec pa' pe'' po' → eval_par1 pe pa pe'' po'

The rule closely follows the mathematical definition that we gave before. The comm_r function corresponds to the mathematical AllCommC.

For the AllSub, it is much more complicated to exactly translate the mathematical definition. Instead, we give the following complex coq definition:

| evalpar2_sync : ∀pe pa pe' pa' po po' pec pe'', ( * When we execute all programs locally * ) ∀i:pid, eval_expr i (pe.(ge_penv) i) (pa i) (pe'.(ge_penv) i) (po i) → ( * If we have at least one sync request * ) sync_outcome po → ( * And if, for any processor in a communicator, a request means that * all * the processors in the communicator request the synchronisation, for the same kind of synchronization * ) (∀ cm:comm, ∀i:pid, (pe.(ge_comms) cm) i = true → ∀id e, sync_outcome_cm_i id cm e (po i) → sync_outcome_cm pe id cm pa' po) → ( * And if for other processors, pa and pe remain unchanged * ) (∀ i:pid, final_outcome_i (po i) → (pa i = pa' i) ∧ ((pe.(ge_penv) i) = (pe'.(ge_penv) i))) → ( * then we execute all the synchronizations * ) pec = comm_r_all po pe' → eval_par2 pec pa' pe'' po' → eval_par2 pe pa pe'' po'

It is clear from this definition that the Diamond semantics is preferable.

(b) Small-step Semantics with Subgroup Synchronisation

The subgroup synchronisation changes the small-step semantics in a very similar way that it changed the big-step semantics.

The first modification concerns the local states in the execution. It is necessary to keep memorised the communicator when a synchronisation is requested. Then the put rules are modified in a similar manner: And finally, there is only the need for one parallel rule change:

Definition 1 (A correct function).

A function f (for why or bsp-why), defined syntactically by a pre-condition p, a body b and a postcondition q, is said to be correct if and only if for any state s in which p holds, then there exists a derivation according to the blocking semantics, towards a state s ′ , in which q holds, and with a result r: Such a definition allows us to make links between the state of a parallel environment, and a sequential one. In particular, we can formulate a lemma concerning the evaluation of terms (which only depends on the environment):

Lemma 9

If s and s ′ are equivalent, then the result of the evaluation of a bsp-why term t in s on the processor i is the same as the result of the evaluation of the why term

Proof. By induction on the term t.

• If t is a constant, its translation is the same constant.

• If t is a variable x, its translation is the access to the p-array variable x in why. By definition of the equivalence s ∼ s ′ , we have the equality.

• If t is a call to the bsp_pid function, its translation is simply the variable i, which gives the same value in both cases.

• The other cases do not raise more difficulty.

In addition to the equivalence between environments, we also define the (much simpler) equivalence between outcomes. Also, with this definition of equivalence, our main lemma can be written as follows:

of the true rule. t 1 evaluate to the vector of true values, and t 2 evaluates to o. By induction, a 1 evaluates to the same true vector, and a 2 evaluates to o. Thus a evaluates to o.

• The interesting case here is when t is a block (t = Block(b)). By the block tree decomposition rules, we know that a = b is an expression, tagged as having no parallel effects. The block semantics rules gives us that for every processor i, b is executed locally to the outcome o i . Thus the global rule of the semantics can be used, and we have s1, a ⇓ s 2 , o.

(c) Putting Pieces Back Together

We have now all the tools needed to prove our main lemma.

Lemma 16

Let a be a bsp-why expression, and a ′ = [[a]] its transformation into why. We can finally prove the theorem that we announced in the beginning of the section.

Theorem 1

For a given parallel program P, if the transformation of P by bsp-why is correct, then P itself is correct.

Proof. Let (p, b, q) be the pre-condition, body and post-condition of P. Let s 1 be a parallel environment that satisfies the pre-condition of P (s 1 |= p). We want to prove that P is correct, i.

(d) Correctness of Infinite Programs

In this thesis, we have only proved the correctness of the transformation for terminating programs. However, it is also possible to define the correctness of an infinite program, by using the co-inductive semantics seen in Section. [START_REF] Armando | ltl Model Checking for Security Protocols[END_REF].1. It is easy to modify these semantics to include blocking rules when dealing with logic assertions. A correct infinite program would then be a program that accepts an infinite derivation, according to the semantics. For instance, one could check the correctness of a server program made of an infinite loop, with an assertion in the loop body ensuring the desired property. The proof of the bsp-why transformation in this case would follow a similar reasoning, but has not been done because of a lack of time.

Elements of Proof in Coq

We did not have enough time to provide a full correctness proof in coq in the scope of this thesis. However, we give an outline of the proof, with the necessary structures implemented in coq.

(a) Semantics Definitions

We start by defining the evaluation of a logical term, and the satisfaction of a logical formula. The evaluation of a term is a straightforward recursive function: The semantics are then modified to include the blocking rules. For instance, the why big-step semantics now has the following three rules with logic assertions:

(c) Proof Skeleton

The four lemmas used to separate the proof (Lemma 12, Lemma 13, 14 and 15) are given next: The main Lemma 16 is easily proved with the last two lemmas.