Parmi Eux

Pierre Villalon

François-Xavier Pasquier

Thierry Porcher

Janice Onanian Mcmahon

Onil Nazra

Per- Sada Goubier

Yannick Langlois

Pierre Jouvelot

Karim Barkati

Antoniu Pop

Amira Mensi

Rachid Habel

Claude Tadonki

Georges-André Silber

Claire Medrala

Laurent Daverio

Jacqueline Altimira

J'ai apprécié d'y rencontrer et d'échanger sur la compilation avec

i t p a s é t é p o s s i b l e s a n s l e s o u t i e n indéfectible de ma tendre épouse, ces trois dernières années n'ont pas été les plus reposantes et j'admire sa patience et sa tolérance vis à vis de son mari travaillant à 500km de notre domicile et s'absentant régulièrement pour toutes sortes de conférences ou formations.

Mes pensées vont évidement à mon fils Tim, mon moteur principal aujourd'hui, mais aussi celui qui m'épuise à longueur de journées (du moins le peu de celles que j'ai passé avec lui à la maison ces dernières années). A croire qu'il puise son énergie qui semble infinie dans la notre. J'ai la chance d'avoir des parents formidables à qui j'ai posé beaucoup de difficultés qu'ils ont su surmonter pour me pousser à faire des études. Peut-être que Tim qui me le rend bien me fait mesurer l'ampleur de la tâche. Je les en remercie milles fois.

Une aventure commence souvent avec une rencontre, et pour satisfaire ma nature nostalgique, je vais la raconter. C'était le 4 juin 2009, lorsque mon étoile m'a conduit à assister à la Journée jeunes chercheurs sur les Multiprocesseurs et Multicoeurs àP a r i s . Quelle chance d'être encore là à la fin de la journée lorsqu'un curieux personnage (non, je ne me risquerai pas à une description àl aZola) s'est levé pour annoncer qu'il cherchait des candidats passionnés à un projet un peu fou (ou des candidats un peu fous pour un projet passionnant, je ne sais plus très bien...). Il n'en fallait pas plus pour piquer ma curiosité et, après une description plus détaillée, j'étais fasciné par la folie apparente du projet et j'oubliais être le fou potentiel. Je repartais vers mon TGV, une carte de visite dans la poche. Peu après minuit le soir même, sitôt rentré chez moi, j'envoyais mon CV à Ronan Keryel l.L ed o i g td a n sl ' e n g r e n a g e . . . Trois jours plus tard, j'étais invité par François Irigoin àl u ir e n d r ev i s i t eà Fontainebleau. C'est finalement le 1er juillet que je me suis rendu dans les locaux du Centre de Recherche en Informatique (CRI) pour y rencontrer celui qui allait me proposer de diriger cette thèse. Neuf jours plus tard, Ronan me proposait de réaliser ces travaux dans une petite entreprise. Était-ce le goût du risque ? Ou peut-être le nom tellement cool de HPC Project ?L ec h a l l e n g ed el at h è s en ' é t a i tp a sa s s e zd i ffi c i l eàr e l e v e re nl u im ê m e qu'il fallait y associer une entreprise ? A moins que Ronan n'ait simplement su employer les arguments qui font mouche pour un esprit pur: Nous sommes prêts à te prendre en thèse CIFRE entre HPC Project et CRI, histoire de combiner le fun de la thèse avec une rémunération raisonnable et de voir la vie à 2 endroits différents.

Le 6 décembre j'étais encore ingénieur à l'Université de Strasbourg, le 7 décembre 2009 l'aventure commençait vraiment. Le jour même je m'envolais pour une école thématique sur le calcul hautes performances sur accélérateurs matériels et je rencontrais deux personnes dont j'ignorais l'importance de nos futurs interactions: Béatrice Creusillet et Stéphanie Even.J en ep o u v a i sr ê v e rm e i l l e u rd é p a r t .

Ac e sr e n c o n t r e ss ' e ns o n ts u c c é d é e sd en o m b r e u s e sa u t r e s ,e ti lm ' e s ti m p o s s i b l ed e toutes les mentionner ici. Certaines ont été plus marquantes, ou ont eu plus d'impact dans le cadre de mes travaux, que ce soit vis-à-vis de l'orientation de mes recherches ou simplement de mes conditions de "travail".

Bien sûr Ronan et François, pour m'avoir supporté, ont la plus grande part de mérites, ils ont été en quelque sorte le Yin et le Yang, le Chaud et le Froid, l'Alpha et l'Omega, enfin bref les éléments essentiels au maintient d'un équilibre pendant ces trois années. J'ai le privilège de continuer à travailler avec Ronan aujourd'hui. Fabien Coelho,q u iar e j o i n t l'aventure en cours de route (et il n'avait pas l'excuse de l'ignorance, il avait déjà eu à me supporter sur pipsdev@), a su apporter une couleur différente et compléter efficacement la direction de mes travaux. J'ai beaucoup progressé à leur contact, techniquement et humainement. Leurs relectures attentives et leur nombreux commentaires ont contribué significativement à l'amélioration de ce manuscrit, et j'atteste que tout ce qui peut empêcher ce dernier de répondre à l'exigence du lecteur reste de mon seul fait, que ce soit de ma paresse ou de mon incompétence (je me plais à croire au premier et à douter du second).

Au rang de mes collègues, j'ai eu la chance de côtoyer des gens très brillant. Il est très facile d'être modeste quand on évolue dans un pareil environnement.

Je reste admiratif devant la qualité du travail de Béatrice qui continue à m'impressionner chaque semaine par son efficacité. Je déconseille la lecture de son manuscrit de thèse à tout doctorant: la qualité (et la taille) de l'ouvrage pose une base à décourager de commencer sa propre rédaction. Si elle n'était pas si gentille, dévouée, et tellement agréable, je pourrais peut-être songer à lui trouver un défaut. Au même titre que Ronan, je suis chanceux de pouvoir continuer à la compter parmi mes collègues pour la suite de nos travaux sur Par4All.

Je suis heureux d'avoir pu travailler avec Serge papil lon Guelton pour la teneur nos discussions techniques, pour sa fraîcheur,e tp o u rs o ne n t h o u s i a s m ec o m m u n i c a t i f . N o s différences ont été sources d'enrichissement, que j'espère mutuel.

Le choix de m'engager avec Silkan (HPC-Project à cette époque) comportait des risques, mais ce fut payant. En dehors de l'expérience intéressante qui a consister à trouver l'équilibre entre la recherche et les problématiques terre-à-terre d'une jeune entreprise, j'ai côtoyé (et continue à travailler avec) des gens brillants.

all solution has emerged.

This thesis proposes a compiler-based solution to partially answer the three "P "p r o perties: Performance, Portability, and Programmability. The goal is to transform automatically a sequential program into an equivalent program accelerated with a GPU. A prototype, Par4All, is implemented and validated with numerous experiences. The programmability and portability are enforced by definition, and the performance may not be as good as what can be obtained by an expert programmer, but still has been measured excellent for a wide range of kernels and applications.

A survey of the GPU architectures and the trends in the languages and framework design is presented. The data movement between the host and the accelerator is managed without involving the developer. An algorithm is proposed to optimize the communication by sending data to the GPU as early as possible and keeping them on the GPU as long as they are not required by the host. Loop transformations techniques for kernel code generation are involved, and even well-known ones have to be adapted to match specific GPU constraints. They are combined in a coherent and flexible way and dynamically scheduled within the compilation process of an interprocedural compiler. Some preliminary work is presented about the extension of the approach toward multiple GPUs.

Résumé

Depuis le début des années 2000, la performance brute des coeurs des processeurs a cessé son augmentation exponentielle. Les circuits graphiques (GPUs) modernes ont été conçus comme des circuits composés d'une véritable grille de plusieurs centaines voir milliers d'unités de calcul. Leur capacité de calcul les a amenés à être rapidement détournés de leur fonction première d'affichage pour être exploités comme accélérateurs de calculs généralistes. Toutefois programmer un GPU efficacement en dehors du rendu de scènes 3D reste un défi.

La jungle qui règne dans l'écosystème du matériel se reflète dans le monde du logiciel, avec de plus en plus de modèles de programmation, langages, ou API, sans laisser émerger de solution universelle.

Cette thèse propose une solution de compilation pour répondre partiellement aux trois "P " propriétés : Performance, Portabilité, et Programmabilité. Le but est de transformer automatiquement un programme séquentiel en un programme équivalent accéléré à l'aide d'un GPU. Un prototype, Par4All, est implémenté et validé par de nombreuses expériences. La programmabilité et la portabilité sont assurées par définition, et si la performance n'est pas toujours au niveau de ce qu'obtiendrait un développeur expert, elle reste excellente sur une large gamme de noyaux et d'applications.

Une étude des architectures des GPUs et les tendances dans la conception des langages et cadres de programmation est présentée. Le placement des données entre l'hôte et l'accélérateur est réalisé sans impliquer le développeur. Un algorithme d'optimisation des communications est proposé pour envoyer les données sur le GPU dès que possible et les y conserver aussi longtemps qu'elle ne sont pas requises sur l'hôte. Des techniques de transformations de boucles pour la génération de code noyau sont utilisées, et même certaines connues et éprouvées doivent être adaptées aux contraintes posées par les GPUs. Elles sont assemblées de manière cohérente, et ordonnancées dans le flot d'un compilateur interprocédural. Des travaux préliminaires sont présentés au sujet de l'extension de l'approche pour cibler de multiples GPUs. 3 Example of a trivial pass-through GLSL geometry shader, which emits a vertex directly for each input vertex (source wikipedia [Wikipedia 2012b]). 17

List of Figures

2.4 Example of a simple saxpy using BrookGPU (taken from [Buck et al. 2004]

2.23

Influence of runtime parameters on performance for different kernels and different architectures, Nvidia Fermi and AMD Evergreen. On the left the launch configuration for different kernels shows that there is no universal work-group size. On the right a matrix multiply kernel without local data store optimization is used with one to four elements processed in each thread. The upper part shows the impact on performance for both architectures while the lower part shows the occupancy of the AMD GPU and the VLIW packing ratio. Taken from [Zhang et al. 2011b](c 2011 IEEE). .......

2.24

Performance of two different versions of matrix multiply kernel, a horizontal scheme and a vertical scheme, without local memory usage for a given architecture (Fermi), depending on the input size and the activation or not of the L1 cache. Taken from [Zhang et al. 2011b](c 2011 IEEE). .....

2.25

Google trends for the word GPU during last decade. .............

2.26

Source-to-source compilation scheme for GPU (source [Guelton 2011a]). .. 12 Example of a loop fusion scheme to extend the iteration set of a loop nest.

4.13 Example of manual kernel fusion using Thrust library and a SAXPY example. The first version is expressed using native Thrust operators and requires temporary arrays, the second version fuses the three steps in one user-defined kernel (source [START_REF] Hoberock | Jared Hoberock and Nathan Bell. Thrust Quick Start Guide[END_REF]). ............

4.14

On the left, a sequence of statements, in the middle the associated Dependence Graph (DG), and on the right the corresponding Reduced Dependence Graph (RDG) obtained after clustering the vertices that belong to the same loop. In solid red the flow dependences, in dashed blue the anti-dependence, and in dotted green the special dependences that model the declaration. The DG view showed here is simplified for the sake of clarity, for instance output dependences and the loop carried dependences are omitted. ........

4.15

The algorithm begins with a pruning phase. For each direct edge between two vertices it ensures that there is no other path between them. ..... The tasks have been declared with attributes in order to drive StarPU, specifying the suitable target platform for each implementation. Task parameters are declared with a const qualifier when used as read-only, and with an __attribute__((output)) when used as write-only. .........

6.

3 Execution timeline when using parallel tasks on one GPU (upper part) and on two GPUs (lower part) for the 3mm code presented in Figure 6.2. .... The example is the matmul kernel shown in Figure 4.22,p a g e129, with

n i = n j = n k =2048.
The execution times presented here are kernel execution times. The GTX 8800 results are given aside because they are one order of magnitude slower. This architecture does not perform double precision computations: doubles are rounded to float before being processed. Hence, no OpenCL results are available in double precision for this GPU. ..... 192 7.12 Execution times in µs(bestovertwentyruns)andspeedupsforscalarization using CUDA and OpenCL for different AMD and Nvidia GPUs. The code is the Scilab script after conversion to C and loop fusion shown in

The Prophecy

Once upon a time, software programmers were merrily writing their code with the simple von Neumann architecture in mind (see in Figure 1.1). Performance was important of course, but they were also protected by a godsend that let them launch a project requiring computing power that was not yet available. Indeed, the time-to-market period was for sure the scene of huge improvement in hardware performance. The prophecy that every programmer was relying on is known as Moore's law. It is commonly quoted as (see [Srinivasan 2012, Manegold 2002[START_REF] Yang | AP a r a l l e lL o o pS e l f -Scheduling on Extremely Heterogeneous PC Clusters[END_REF]) the CPU clock speed will double every eighteen months. This short and simple sentence has been immersed in the mind of generations of programmers for decades. Everything was going along fine until a bird of ill omen came and stated it cannot continue forever. The nature of exponentials is that you push them out and eventually disaster happens.

He was not the first one to challenge the prophecy, but this time it was Gordon Moore himself [Dubash 2005], the author of the prophecy. It was terrible for the programmers, and most of them locked themselves into denial. Little by little, the idea that the clock speed does not continue to grow as before made its way. As matter of fact, the original prophecy could probably ranked close second on the Top 10 list for misquoted statements, right behind "Luke, I am your father." Actually Moore originally stated [Moore 1965]that the complexity for minimum component costs has increased at a rate of roughly a factor of two per year. . . . Certainly over the short term this rate can be expected to continue, if not to increase. Over the longer term, the rate of increase is a bit more uncertain, although there is no reason to believe it will not remain nearly constant for at least ten years. That means by 1975, the number of components per integrated circuit for minimum cost will be 65,000. Ibe l i e v et h a ts u c hal a r g ec i r c u i tc a nbeb u i l to nas i n g l ew a f e r .

The forty years of easy life ended, as shown in Figure 1.2,a n dp r o g r a m m e r sw e r ea bo u tt o face a new challenge. In fact, hardware designers, facing the frequency wall, jumped right into the parallel world. 1 The processor frequency was limited and they bypassed this issue Programmers discovered a new universe: the execution times of their programs were no longer reduced when a new processor was released. In this new world they had to rethink their algorithms to make use of multiple processors. As if it were not complicated enough, some hardware designers, who probably embraced the dark side of the force, started to introduce some more exotic pieces of hardware. These hardware platforms were highly parallel but very tricky to target. The white knight programmer taking up the challenge had not only to rethink algorithms, but also to manage some complex memory hierarchies for which hardware designers left the management on behalf of the programmer.

Welcome to the heterogeneous computing universe!

Motivation

"Your free lunch will soon be over." Herb Sutter started his 2005 article [START_REF] Sutter | The Free Lunch Is Over: A Fundamental Turn Toward Concurrency in Software[END_REF]] with this declaration to software developers. The limits of frequency scaling now forbid automatic performance increase for sequential programs. The future is heterogeneous, from the embedded world of smartphones to the largest supercomputers. Sutter wrote a sequel to this article [START_REF] Sutter | Welcome to the Jungle[END_REF]] in which he states quite accurately: "Now welcome to the hardware jungle." Figure 1.3 illustrates this evolution. In the embedded system world, current high-end smartphones are based on a multicore processor, and they include vector processing units and also a Graphics Processing Unit (GPU). For instance the A5 processor, used in the Apple iPhone 4S, is a dualcore ARM Cortex-A9 MPCore Central Processing Unit (CPU) together with a dual-core GPU [AnandTech 2011]. The same processor is used in the Apple iPad 2. The latest Tegra 3 processor from Nvidia is a quad-core ARM Cortex-A9 MPCore and a twelvecore GPU [Nvidia 2012b]. In both case, each core includes a 128-bit wide NEON vector unit [Wikipedia 2012a]. The next Tegra generation will support General-Purpose Processing on Graphics Processing Units (GPGPU) computing using Open Computing Language (OpenCL).

In the supercomputing world, parallelism has been present for decades now. Vector ma- chines have been replaced by clusters of multicore multiprocessor systems in the Top500 list [TOP500 Supercomputing Sites 2012]. The new trend is now adding hardware accelerators to these systems, mostly using GPUs, adding a new layer of complexity. The June 2011 Top500 list includes three GPU-based systems in the top five, but there are also five GPUbased systems in the Green500 list [START_REF] Feng | The Green500 List: Encouraging Sustainable Supercomputing[END_REF]]amongthetenfirstentries [The Green500 2011]. The Nvidia Tesla K20 based Titan supercomputer trusts currently the last November 2012 list [TOP500 Supercomputing Sites 2012], and it is interesting to note that Intel with its Xeon Phi coprocessor enters at the 7th rank.

One cannot find a single-core Personal Computer (PC) nowadays. Dual-core is the standard at the entry level, quad-core in the mid-end, and it currently goes up to six-core in the high end. Required by the gaming industry, GPUs shipped with PCs are more and more powerful and are used in a growing set of applications beyond their primary usage: 3D rendering and graphic display.

The concern that arises now, as these heterogeneous platforms are widely available, can be summarized as the three P properties [Adve 1993, Benkner et al. 2011, Adve 2011] shown in Figure 1.4:

• Performance: the program makes use of the peak capability of the hardware.

• Portability: the code written by the programmer runs on a large range of platforms.

• Programmability: the programmer write his algorithms quickly.

Af o u r t hP can now be added: Power. Not only because our cell phones have small batteries, but also because in 2007 each of the ten biggest supercomputers consumed as much energy as a city of forty thousand people [START_REF] Feng | The Green500 List: Encouraging Sustainable Supercomputing[END_REF]. People are looking for software that is power aware [START_REF] Hsu | [END_REF], using trade-offs between performance Solutions to address these properties have been sought for a long time, since clusters of computers entered the market. The programming complexity is increased when very specialized hardware accelerators are added in these machines. Many types of resources must be connected, and it becomes too much of a burden for the developer.

Performance has improved with compilers, allowing new languages to be competitive with the king C language, which is still the reference when close control of the hardware is necessary.

On the portability side, it is hardly possible to maintain a huge C code making use of a wide range of heterogeneous devices. A usual practice is then to restrict to a common subset of hardware features, limiting the practical performance one can expect with respect to the theoretical peak performance depending on the application.

Finally, the programmability has been largely addressed by Application Programming Interface (API) providers and language designers. For instance, UPC [UPC Consortium 2005], Co-Array Fortran [ISO 2010], or Titanium [START_REF] Yelick | [END_REF]]e x p l o i tt h e Partitioned Global Address Space (PGAS) model. The global memory address space is logically partitioned and physically distributed among processors [Coarfa et al. 2005]. The locality of references is then exploited by the runtime system with strategies like the owner 1.3. Outline computes rule. The purpose of these languages is to let the programmer ignore the remote memory accesses, which leads to simpler code. This simple flat model has then evolved to Asynchronous Partitioned Global Address Space (APGAS) with the X10 [START_REF] Ebcioğlu | [END_REF] or the Chapel [START_REF] Chamberlain | [END_REF] languages. Concurrency has been made explicit and the programmers express asynchronous constructions on multiple levels. While the programmers have to change their usual approach to express their algorithms, these languages provide high-level abstractions of architecture in a layered manner. However, these languages are new and not widely adopted by developers. Criticisms about performance has been expressed: the code has to be optimized only with a good knowledge of the target architecture [Zhang et al. 2011a].

The recent OpenCL standard [Khronos OpenCL Working Group 2008, Khronos OpenCL Working Group 2011]hasbeendev elopedtoprogramaccelerators. Itpro videsan abstraction of the hardware, based on an API to manage the device, and a language derived from a subset of C to write kernels, i.e., functions to be executed on an accelerator. This standard provides some portability across vendors and programmability at the C level. However, performance portability is difficult to achieve [START_REF] Komatsu | [END_REF]. Another approach is directive-based languages, following the well-known OpenMP standard [OpenMP Architecture Review Board 1997, OpenMP Architecture Review Board 2011]f o rs h a r e d memory systems. For example, some sets of directives like Hybrid Multicore Parallel Programming (HMPP) [Wikipedia 2012c], PGI Accelerator [Wolfe 2010], or more recently OpenACC [OpenACC Consortium 2012]p r o v i d ea ne a s i e rw a yt op r o g r a ma c c e l e r a t o r s , while preserving code portability.

Outline

The goal of this dissertation is to explore the potential of compilers to provide a solution to the three P s: Performance, Portability, and Programmability. The solution considered is the automatic code transformation of plain C or Fortran sequential code to acceleratorenabled equivalent code. The main target machines are accelerators like GPUs:m a s s i v e l y parallel, with embedded memories in the GB range. A source-to-source approach takes advantage of the Compute Unified Device Architecture (CUDA) and the standard OpenCL APIs. Programmability and portability are enforced by the fully automatic approach. Numerous measurements are provided to show that performance is not sacrificed.

The approach is pragmatic and the ideas and schemes presented are implemented in a 8C h a p t e r 1 . I n t r o d u c t i o n new automatic source-to-source compiler, Par4All [SILKAN 2010 (perso)], and validated using benchmarks. The main goal is to provide a full end-to-end compilation chain, from the sequential code to the GPU-enabled binary, good enough as a prototype for an industrial solution. Therefore, instead of being deeply focused on a limited part of the problem, this work contributes to different aspects of the problem and attempts to explore and solve all the issues raised when building such a full compilation chain. This compiler approach is useful for legacy applications and new developments as well. A compiler lowers the entry cost but also the exit cost when a new platform has to be targeted. Debugging and maintenance are easier since the code is written with a sequential semantics that is suitable for most programmers. When the compiled code is not executing fast enough, some specific costly parts of the code, the hot spots,canbemanuallyoptimized for a particular architecture: a source-to-source compiler makes manual optimizations possible on the code after processing by the heterogeneous compiler.

The choice of the C and Fortran languages is driven by their broad use in the highperformance community. C is also a common choice for other tools that generate code from a high-level representation or a scripting language. In order to illustrate the interest of this approach, examples of Scilab [Scilab Consortium 2003] code are included. They are automatically converted to sequential C with a Scilab compiler, and then transformed to exploit accelerators using the different methods presented in this dissertation.

Ip r e s e n tt h eh i s t o r yo fGPUs and the emergence of GPGPU in Chapter 2. The hardware evolution is mirrored by the associated programming languages that all failed to match the three P sc r i t e r i a . Ii n t r o d u c eh ea r c h i t e c t u r e so fGPUs and their evolutions to show the constraints that should be met by a compiler to achieve performance: distributed memory, memory access patterns on GPUs,fi n eg r a i n e dp a r a l l e l i s m ,a n ds u p po r t for atomic operations.

In Chapter 3,Ie x p l o r es o l u t i o n st ot h ea u t o m a t i cd i s t r i b u t i o no ft h ed a t ao n t ot h e CPU and accelerator memories. The convex array region abstract representation is first presented. A simple process to generate communications based on array regions is then explained. I propose a new interprocedural optimizing scheme, and I validate it using experiments. The algorithm relies on a new static analysis, Kernel Data Mapping,a n d minimizes the communications by preserving data on the GPU memory and avoiding redundant communications. IidentifyacollectionofprogramandlooptransformationstoisolateandoptimizeGPU codes in Chapter 4. I propose a flexible mapping of parallel loop nests on the different layers 1.3. Outline of GPUs.Id e s i g n e da n di m p l e m e n t e dat r a n s f o r m a t i o nt os u b s t i t u t ei n d u c t i o nv a r i a b l e s and enable further parallelization. I present two different loop parallelization algorithms and the consequences on code generation and performance. I modified them to handle reduction schemes and introduced Coarse Grained with Reductions (CGR). I designed and implemented a new transformation to benefit from hardware atomic operations when parallelizing loop nests with reductions. I designed and implemented a new loop fusion scheme, and I proposed heuristics to drive loop fusion to fit the GPUs'c o n s t r a i n t s . I present Three different scalarization schemes. I modified the existing transformation to provide better performance on GPUs.I a l s o p r e s e n t t h e i m p a c t o f l o o p u n r o l l i n g a n d array linearization. I validated all these transformations with measurements.

I present the whole compilation process in Chapter 5,fromthesequentialsourcecodeto the final binary and the runtime associated at execution. The flexibility of a programmable pass manager is used to produce the compilation chain. Interprocedural analyses are used, and they require processing the source code of all functions in the call graph. It is an issue for external libraries. I defined a dynamic solution to feed the compiler on demand during the process.

Ie x p l o r ep e r s p e c t i v e sa b o u te x t e n s i o n sf o rm u l t i p l eGPUs in Chapter 6.Is t u d yt w o different schemes to extract parallelism. I implemented a simple task parallelism extraction, and modified the existing symbolic tiling transformation. The StarPU runtime library is used to exploit task parallelism and schedule tasks on multiple GPUs.

I present all experimental results in Chapter 7 to validate the solutions defined in the previous chapters. I extracted twenty test cases from Polybench and Rodinia test suites. Ia l s ou s e dar e a ln u m e r i c a ln-body simulation to show that speedups can be obtained automatically on application larger than the kernel benchmarks. Several target GPU boards from Nvidia and Advanced Micro Devices (AMD) are used to show how the impact of program transformations on performance depends on architectures.

Due to the variety of subjects tackled in this work, the presentation of the related works is included in each chapter.

Finally, to pay a tribute to the environment in which this work takes place, a summary in French is provided for each chapter at the end of the thesis.

The reign of the classical Central Processing Unit (CPU) is no longer hegemonic and the computing world is now heterogeneous. The Graphics Processing Units (GPUs) have been candidate as CPUs co-processors for more than a decade now. Other architectures were also developed like the Intel Larabee [START_REF] Seiler | [END_REF], which never really reached the market as GPU and was released recently as a co-processor under the name Xeon Phi 1 by the end of 2012, and the IBM and Sony Cell [Hofstee 2005], which was used in the Sony PlayStation 3. However, although many researchers have tried to map efficient algorithms on its complex architecture, it was discontinued. This failure resulted from its difficult programming and memory models, especially facing the emergence of alternatives in the industry: the GPU manufacturers entered the general computing market. Dedicated graphic hardware units offer, generally via their drivers, access to a standard Application Programming Interface (API) such as OpenGL [Khronos OpenGL Working Group 1994, Khronos OpenGL Working Group 2012]a n dD i r e c t X [ Microsoft 1995, Microsoft 2012]. These APIs are specific to graphic processing, the main application domain for this kind of hardware. Graphic processing makes use of many vector operations, and GPUs can multiply a vector by a scalar in one operation. This capability has been hijacked from graphic processing toward general-purpose computations.

This chapter first presents in Section 2.1 the history of the general-purpose computing using GPUs,t h e nS e c t i o n2.2 gives insights about the evolution of the programming model and the different initiatives taken to pave the way to General-Purpose Processing on Graphics Processing Units (GPGPU). The OpenCL standard is introduced with more details in Section 2.3. The contemporary GPU architectures are presented in Section 2.4. Finally I list the many programming challenges these architectures offer to programmers and compiler designers.

History

The use of graphic hardware for general-purpose computing has been a research domain for more than twenty years. Harris et al. proposed [START_REF] Harris | [END_REF]] a history starting with am a c h i n el i k et h eI k o n a s [START_REF] England | Asystemforinter activemo delingofphysic alcurve dsurfac e objects[END_REF]], the Pixel Machine [START_REF] Potmesil | [END_REF], and Pixel-Planes 5 [START_REF] Rhoades | [END_REF]. [START_REF] Trendall | [END_REF][START_REF] Trendall | [END_REF][START_REF] Trendall | [END_REF] gave an overview of the past experiments with graphics hardware. Lengyel et al. [START_REF] Lengyel | [END_REF]]p e r f o r m e dr e a l -t i m er o b o tm o t i o np l a n n i n gu s i n gr a s t e r i z i n g capabilities of graphics hardware. Bohn [Bohn 1998]i n t e r p r e t sar e c t a n g l eo fp i x e l sa sa four-dimensional vector function, to do computation on a Kohonen feature map. [START_REF] Hoff | [END_REF]]d e s c r i beh o wt oc o m p u t eV o r o n o id i a g r a m su s i n gz-buffers. Kedem et al. [Kedem & Ishihara 1999] use the PixelFlow SIMD graphics computer [START_REF] Eyles | [END_REF] to decrypt Unix passwords. Finally some raytracing was performed on GPU in [START_REF] Carr | [END_REF]a n d [ Purcell et al. 2002]. A survey of GPGPU computation can be found in [START_REF] Owens | [END_REF].

Until 2007, the GPUs exposed a graphic pipeline through the OpenGL API.A l lt h e élégance of this research rested in the mapping of general mathematical computations on this pipeline [START_REF] Trendall | [END_REF]. A key limitation was that, at that time, GPU hardware offered only single-precision floating point units, although double precision floating point is often required for engineering and most scientific simulations.

GPUs have spread during the last decades, with an excellent cost/performance ratio that led to a trend in experimental research to use these specialized pieces of hardware. This trend was mirrored first with the evolution of the programming interface. Both OpenGL and DirectX introduced shaders (see Section 2.2.2)i n2 0 0 1 ,a n dt h u sa d d e dp r o g r a m m ability and flexibility to the graphic pipeline. However, using one of the graphic APIs was still mandatory and therefore General-Purpose Processing on Graphics Processing Units (GPGPU)w a se v e nm o r ec h a l l e n g i n gt h a ni ti sc u r r e n t l y .

In 2003 Buck et al. [Buck et al. 2004]i m p l e m e n t e das u b s e to ft h eB r o o ks t r e a m i n g language to program GPUs. This new language, called BrookGPU, does not expose at all the graphic pipeline. The code is compiled toward DirectX and OpenGL. BrookGPU is used for instance in the Folding@home project [Pande lab Stanford University 2012]. More insight about Brook and BrookGPU is given in Section 2.2.3.

Ian Buck, who designed Brook and BrookGPU, has joined Nvidia to design the Compute Unified Device Architecture (CUDA) language, which shares similarities with BrookGPU. However, while BrookGPU is generic, CUDA API is specific to Nvidia and its then new scalar GPU architecture introduced with CUDA is presented in Section 2.4.5. CUDA is an API and a language to program GPUs more easily. The graphic pipeline does not exist anymore as such and the architecture is unified and exposed as multi-Single Instruction stream, Multiple Data streams (SIMD)-like processors. CUDA is introduced with more details in Section 2.2.4.

From 2004 to 2012, the evolution of GPUs'fl o a t i n gpo i n tpe r f o r m a n c ei n c r e a s e dm u c h The increased interest in GPGPU attracted more attention and led to the standardization of a dedicated API and language to program accelerators: the Open Computing Language known as OpenCL (see Section 2.3).

Others programming models are emerging, such as directive-based languages. These let the programmers write portable, maintainable, and hopefully efficient code. Pragma-like directives are added to a sequential code to tell the compiler which pieces of code should be executed on accelerator. This method is less intrusive but may provide limited performance currently. Several sets of directives are presented in Section 2.2.10.

Languages, Frameworks, and Programming Models

The programming language history includes many languages, frameworks, and programming models that have been designed to program accelerators. Some were designed for the initial purpose of the accelerator, i.e., graphic computing, and were later diverted to-ward general-purpose computation. Others were designed entirely from scratch to address GPGPU needs. This section surveys the major contributions, approaches, and paradigms involved during the last decade to program hardware accelerators in general-purpose computations.

Open Graphics Library (OpenGL)

Open Graphics Library (OpenGL) is a specification for a multiplatform API that was developed in 1992 by Silicon Graphics Inc. It is used to program software that make use of 3D or 2D graphic processing and provides an abstraction of the different graphic units, hiding the complexities of interfacing with different 3D accelerators. OpenGL manipulates objects such as points, lines and polygons, and converts them into pixels via a graphics pipeline, parametrized with the OpenGL state machine.

OpenGL is a procedural API containing low-level primitives that must be used by the programmer to render a scene. OpenGL was designed upon a state machine that mimics the graphic hardwares available at that time. The programmer must have a good knowledge of the graphics pipeline.

OpenGL commands mostly issue objects (points, lines and polygons) to the graphics pipeline, or configure the pipeline stages that process these objects. Basically, each stage of the pipeline performs a fixed function and is configurable only within tight limits. But since OpenGL 2.0 [Khronos OpenGL Working Group 2004]a n dt h ei n t r o d u c t i o no f shaders and the OpenGL Shading Language (GLSL) language, several stages are now fully programmable.

In august 2012, the version 4.3 is announced with a new feature: the possibility of executing compute shaders such as the saxpy example shown in Figure 2.2 without using the full OpenGL state machine. The shader program is executed by every single threads in parallel. Then conducting the same operation over a vector, which usually exhibits a loop, involves here an implicit iteration space. Figure 2.2 illustrates this execution model with one thread per iteration. An classic CPU version of saxpy is shown in Figure 2.4a.

Shaders

Shaders are small programs used in graphics processing to operate at a specific stage of the pipeline. They are used to describe light absorption and diffusion, the textures to apply, reflections and refractions, shadowing, moving primitives, or some other post-processing effects. The rendering process makes the shaders perfect candidates for parallel execution on vector graphic processors, relieving the CPU and producing the result faster. Three types of shaders exist:

• Vertex shaders are executed on each vertex given to the GPU. The purpose is to transform each 3D position in the virtual space into the 2D coordinates on the target display, and a depth value for the Z-buffer. The vertex shaders can manipulate properties like position, color, and texture coordinates, but cannot spawn new vertices. The vertex shader output is transferred to the next graphic pipeline stage, a geometry shader if any, or directly to the rasterizer.

• Geometry shaders are able to add or remove vertices of a lattice and their output is sent to the rasterizer for the rendering of the final graphic picture.

• Pixel shaders, also known as fragment shaders, compute the color of each pixel individually. The input comes from the rasterizer, which fills the polygons sent in the pipeline. Pixel shaders are typically used for lighting and associated effects like bump mapping and color tone adjustment. Pixel shaders are often called many times per pixel on the display, one for each object, even if it is hidden. The Z-buffer is later used to sort objects and display only visible parts. Shaders are flexible and efficient. Complicated surfaces can be rendered from a simple geometry. For instance a shader can be used to generate a tiled floor from a plane description.

# version 120 # extension

Initially languages close to assembly, shaders became more popular in 2001 with the definition of higher level languages and their adoption as extensions in OpenGL and Di-rectX. Shaders made it easier to use GPUs for a wider kind of algorithms. They are close to C and implicitly run in a parallel way on the GPU,b u ti ft h e ya d dfl e x i b i l i t ya n dp r ogrammability to the graphic pipeline for general-purpose computation, they do not provide the programmer with a way to abstract the graphic APIs.F i g u r e2.3 contains an example of a simple pass-through GLSL geometry shader.

Brook and BrookGPU

Brook is a direct successor of the Stanford Merrimac project [START_REF] Dally | [END_REF]. The goal of this project was to take advantage of a new compute model called streaming. This model offers two main advantages over classical languages:

• Data parallelism: Brook lets the programmer specify how to apply the same operation to different pieces of array elements.

• Arithmetic intensity: the programmer is encouraged to execute operations on data that minimize communications and maximize local computation.

The Merrimac project aimed at offering better performance than distributed memory [Project 2003], but using the same technology. A language is designed to take parallel processing concepts into a familiar and efficient language, using the streaming model. [Buck et al. 2004]).

Brook is designed as a simple extension of ANSI C.

Until 2003, the only way to benefit from graphics hardware resources was the general APIs OpenGL and DirectX, and the shader programming. BrookGPU [Buck et al. 2004] implements a subset from the Brook specification [Buck 2003]t ot a r g e tGPUs. It allows compiling the same code in different target languages, OpenGL and DirectX of course, but also Nvidia Cg shaders and later the generalist Advanced Micro Devices (AMD) Close To Metal (CTM) API. BrookGPU was used for instance in the Folding@home project. Figure 2.4 illustrates a simple SAXPY operation using BrookGPU.

Nvidia Compute Unified Device Architecture (CUDA)

Nvidia hired Ian Buck, the main author of Brook and BrookGPU, to design CUDA. Thus there are similarities between CUDA and BrookGPU. However, BrookGPU is generic and has different back ends while CUDA exhibits features specific to Nvidia GPUs. CUDA offers features and low-level tuning unavailable in a portable and generic language such as BrookGPU. CUDA removes also many limitations found in Brook, such as the memory model, which is quite rigid in Brook. Indeed it requires the programmers to map their algorithm around a fairly limited memory access pattern [START_REF] Buck | [END_REF]].

CUDA technology was published by Nvidia in February 2007. It is a set of components shipped by Nvidia to program their GPUs: a driver, a runtime, libraries (BLAS, FFT, . . . ), a language based on a extension to a C++ subset, and an API that exhibits an abstraction model for the architecture.

The code that runs on the GPU is written in a C-like form and allows direct random accesses to the GPU memory. The CUDA API is high level and abstracts the hardware. However, to obtain a good percentage of the peak performance, the code must be tuned with a good knowledge of the underlying architecture. CUDA allows the programmers to bypass the compiler and to write directly code in Parallel Thread eXecution (PTX),a pseudo-assembly SIMD language that exhibits an infinite number of registers. The PTX is Just In Time (JIT) compiled by the CUDA driver for a given GPU using its own Instruction Set Architecture (ISA). This allows Nvidia to evolve their architecture while being backward compatible, thanks to the JIT compilation capability of the driver.

CUDA has many advantages over classic GPGPU schemes using the OpenGL API for instance:

• Use of the C language (with extensions) instead of the classical graphic API:akernel is close to a function call.

• Possibility for sparse memory writes: the code can access a single address in memory.

• Threads can share up to 48 kB of local memory, that is nearly as fast as registers.

• Memory transfers between host and GPU are faster using page-locked memory.

• The instruction set is more extensive, for instance integer and bitwise operations and double precision computation are supported.

However, CUDA exhibits also some limits when compared to classic CPU programming:

• Texture rendering is supp orted in a limited way.

• Only the most recent architectures support function calls.

• The IEEE 754 floating point standard is not fully implemented.

• Threads execute by groups of thirty-two in a SIMD fashion, such a group is denoted warp by Nvidia. Branches do not impact performance significantly as long as all thirty-two threads in a group take the same path.

• GPUs compatible with CUDA are exclusively produced by Nvidia.

Nvidia has shipped dedicated boards for GPGPU: the Tesla series. These GPUs boards do not always have any display port and therefore can be used only for intensive compute processing. Usually Tesla boards provide dedicated features such as Error-correcting code (ECC) memory, larger memory sizes, and higher double precision peak performances.

AMD Accelerated Parallel Processing, FireStream

FireStream is the AMD GPGPU solution. The name refers to both the hardware and the software shipped by AMD. The hardware was released in 2006 under the name AMD Stream Processor. AMD claims that it was the industry's first commercially available hardware stream processing solution [Advanced Micro Devices 2006]. AMD introduced at the same time their own GPGPU API: Close To Metal (CTM). This API is very close to the hardware as it gives developers direct access to the native instruction set and memory, but the trade-off that arises when choosing a very low level API and language is the usual one: it raises the effort required from the programmer. AMD soon after proposed a new solution called Stream Computing Software Development Kit (SDK).ItisacompleteSDK and a compiler for Brook+, a high-level language based on Brook (see Section 2.2.3). At the same time they renamed CTM as Compute Abstraction Layer (CAL) 2 , which is the target API for Brook+. CAL provides the API to control the device (open, close, managing context, transfer data from or to the device, . . .). It comes with the language CAL Intermediate Language (IL),anintermediateassembly-likelanguageforAMD GPUs. IL is then compiled for the target ISA using the CAL API.

The latest version of AMD's technology is now called Accelerated Parallel Processing (APP)andisbaseduponOpen Computing Language (OpenCL). The support for Brook+ and CTM has been discontinued, and CAL API is now deprecated in favor of OpenCL. The IL language is still the target language for the OpenCL compiler.

The FireStream GPU series, just as the Nvidia Tesla series, does not always provide any graphic output, and is intended to be a pure GPGPU solution.

Open Computing Language (OpenCL)

OpenCL is a software stack designed to write programs portable over a wide range of platforms like CPUs, GPUs, Field Programmable Gate Array (FPGA) or other em-2. AMD CAL is unrelated to the eponymous language from Berkeley bedded hardwares. It includes a language, based on the C99 standard, to write code for heterogeneous hardwares. It defines an API to manage the dedicated hardware from the host. OpenCL was proposed by Apple to the Khronos Group in 2008 to unify the various frameworks in one standard, which was defined later in the same year [Khronos OpenCL Working Group 2008]. I study OpenCL in detail in Section 2.3

Microsoft DirectCompute

Microsoft proposes its own dedicated GPGPU solution with DirectCompute [Microsoft 2010]. It was released in fall 2009 as part of DirectX 11. The DirectCompute API leverage the High Level Shader Language (HLSL) (same as Nvidia Cg) and provides asolutionthatb ypassestheclassicalgraphicpipelineinfa v orofadirectaccesslik eCUDA or OpenCL. Programmers familiar with HLSL/Cg are then able to transfer buffers directly to or from the GPU, and set shader-like kernels for processing these buffers. Figure 2.5 shows an example of such a shader. The input matrices d_A and d_B are multiplied into d_C, using a straightforward block matrix multiplication algorithm. The three matrices are size ⇤ size. The mm function is executed by size ⇤ size number of threads. The scheduler is instructed to group the threads by workgroups of 16 ⇤ 16 number of threads. This virtual organization is mapped on the hardware by ensuring that all threads in a virtual workgroup share some resources, at least till the point where they can be synchronized. The groupshared declaration of local_a and local_b is linked to this thread organization, these arrays are shared by all the threads in a virtual workgroup. The local_a and local_b array holds the current block of the input matrices during the computation. They are loaded by the threads among a group, and a synchronization enforce that they are fully loaded before each thread perform the multiplication on the blocks using these shared arrays. The shared arrays can be seen as a cache memory that is explicitly managed by the programmer. Example of a Cg/HLSL shader for DirectCompute (source Microsoft [START_REF] Deitz | [END_REF].

C++ Accelerated Massive Parallelism (AMP)

the CPU to the GPU and back.

Unlike Direct Compute presented in Section 2.2.7,t h e r ei sn os e p a r a t i o nb e t w e e nt h e code running on the accelerator and the host code. Offloading a computation involves writing a kernel using a lambda function and a dedicated construction to express the iteration set like parallel_for_each.F i g u r e2.6 contains an example of C++ code before and after its conversion to C++ AMP. This example is a simple sum of two arrays. The concurrency namespace allows the use of AMP specific constructions and functions, such as array_view for example. The code exhibits a call to discard_data() on the array_view object sum. This call is intended to hint the runtime so that an initial copy to the accelerator memory is avoided since sum does not contain any data.

C++ AMP does not seem to provide a new paradigm, but leverages C++ power and flexibility to provide a more relaxed programming model than Direct Compute or OpenCL. It seems to compete more against a directive-based language such as OpenACC,presen ted in Section 2.2.10.

ΣC and the MPPA Accelerator

While far from being a new paradigm, process network language may benefit from more consideration in the future. For instance Kalray leverages the ΣCl a n g u a g e [ Goubier et al. 2011]f o ri t s( y e tu n r e l e a s e d )Multi-Purpose Processor Array (MPPA) accelerator [START_REF] Kalray | Kalray. MPPA : Multi-Purpose Processor Array[END_REF]]. It integrates a network of 256 Very Long Instruction Word (VLIW) processors, organized in sixteen clusters of sixteen processors, interconnected using a highbandwidth network-on-chip, but embeds only a few tens of MB of memory. This accelerator leverages low consumption (estimated at around 5 W) when compared to power-hungry GPUs. For example, the Nvidia Tesla C2070 eats up to 238 W.

ΣC is based on the Kahn process network theory [START_REF] Kahn | The Semantics of Simple Language for Parallel Programming[END_REF]]. It has been designed to enforce properties like being deadlock-free and provides memory-bounded execution. Formal analysis is leveraged to achieve this goal. The ΣCprogrammingmodelin v olv esagen ts as the most basic units. An agent is a stateless independent thread with its own memory space. Agents communicate via First In, First Out (FIFO) queues. Then an application is designed by a set of communicating agents forming a graph. In a ΣCa p p l i c a t i o n ,t h e graph is static during all the life of the application, no agent creation or destruction can occur neither any change to the graph topology.

Directive-Based Language and Frameworks

Addressing the programmers' difficulties to write efficient, portable, and maintainable code, as well as the ability to convert progressively existing sequential version toward GPGPU,s e v e r a li n i t i a t i v e sw e r el a u n c h e d ,b a s e do nd i r e c t i v e si n s e r t e di nCo rF o r t r a n sequential code.

On the basis of the popular Open Multi Processing (OpenMP) [Microsoft Corporation 2012b]).

• OpenMP is efficient at expressing loop-level parallelism in applications, which is an ideal target for utilizing the highly parallel GPU computing units to accelerate data parallel computations.

• The concept of a master thread and a pool of worker threads in OpenMP's fork-join model represents well the relationship between the master thread running in a host CPU and a pool of threads in a GPU device.

• Incremental parallelization of applications, which is one of OpenMP's features, can add the same benefit to GPGPU programming.

Following the same idea, the OMPCUDA project [Ohshima et al. 2010]e x t e n d e dt h e OMNI OpenMP Compiler to target CUDA.

As OpenMP is designed for shared memory systems, it can be difficult to convert automatically an OpenMP code optimized for CPU into a heterogeneous architecture. Thus other projects bypassed this issue and introduced new directives. Han and Abdelrahman propose with hiCUDA [Han & Abdelrahman 2009]as e to fd i r e c t i v e st om a n a g ed a t a allocation and transfers, and kernel mapping on GPU. The main drawback is that even if it is simpler to write, hiCUDA still requires the programmer to have good knowledge of the target architecture and the way the algorithm maps onto the GPU.I ti su n c l e a r how the code written this way is portable across architectures. Figure 2.7 shows a sample matrix multiplication using hiCUDA. The directives are tied to a particular architecture: the workgroup size is statically defined, so is the strip-mining width.

Bodin and Bihan propose Hybrid Multicore Parallel Programming (HMPP) [START_REF] Bodin | son code source pour une architecture ou un modèle d'exécution[END_REF], another set of directives to perform heterogeneous computing. HMPP was then promoted as a standard, Open Hybrid Multicore Parallel Programming (OpenHMPP), in a consortium joining CAPS Entreprise and PathScale. HMPP requires that the code follows some restrictions. The code to be run on an accelerator must be wrapped in a separate function called a codelet.H e r ea r et h ec o d e l e tp r o p e r t i e s[ Consortium 2011]:

• It is a pure function.

• It does not contain static or volatile variable declarations or refer to any global variables unless these have been declared by a HMPP directive "resident."

• It does not contain any function calls with an invisible body (that cannot be inlined). This includes the use of libraries and system functions such as malloc, printf. . . . • Every function call must refer to a static pure function (no function pointers).

• It does not return any value (void function in C or a subroutine in Fortran).

• The number of arguments should be set (i.e., it can not be a variadic function as in stdarg.h in C).

• It is not recursive.

• Its parameters are assumed to be non-aliased. Simple example for HMPP directive-based code writing (source wikipedia [Wikipedia 2012c]).

• It does not contain call site directives (i.e., RPC to another codelet) or other HMPP directives.

HMPP requires less effort from the programmer, and the HMPP compiler can manage automatically to map a given codelet on the GPU,asw ellashandlingthedatamo v emen t. The compiler can automatically detect the parallelism in a loop nest and take any decision involved in the process of generating the accelerator code. However HMPP offers advanced directive that allows the programmer to tune the compilation process to get better performance. But with the same drawbacks as in hiCUDA: the code is then likely to come tied to a specific target. Figure 2.8 contains a sample code written using HMPP without any specific directive.

PGI introduced the PGI Accelerator [Wolfe 2010], which uses the same idea as HMPP. The proposed directives are written àl aOpenMP. The code is not outlined in a codelet by the programmer.

The initial PGI Accelerator provided a limited set of directives. supposed to automatically do the conversion work. It was later updated with more possibilities available to the programmer to help the compiler to manage the data movements. HMPP includes also similar directives. Figure 2.9 shows a simple code written using these directives.

In November 2011 at the SuperComputing Conference, Nvidia, Cray, PGI, and CAPS announced that they agreed on a standard for directives: OpenACC. The OpenMP Architecture Review Board CEO Michael Wong declared at this occasion that he looked forward to work within the OpenMP organization to merge OpenACC with other ideas to create ac o m m o ns p e c i fi c a t i o nt h a te x t e n d sOpenMP to support accelerators. The OpenACC standard [START_REF] Nvidia | NVIDIA, Cray, PGI, CAPS. The OpenACC Specification, version 1.0[END_REF] seems to be based upon the PGI Accelerator solution: the directives show close similarities.

JCUDA [START_REF][END_REF]] is a programming interface for Java that allows invoking CUDA kernels. JCUDA defines an extension of Java that needs to be preprocessed to

double[][] I_a= new double[NUM1][NUM2]; double[][][] I_aout = new double[NUM1][NUM2][NUM3]; double[][] I_aex= new double[NUM1][NUM2];
initArray(I_A); initArray(I_aex); // i n i t i a l i z e value in array [START_REF][END_REF]). generate the pure Java code and the Java Native Interface (JNI) glue to link against CUDA kernels. The programmers make use of annotation (IN, OUT, INOUT) in front of kernel arguments and the data transfers are managed automatically by JCUDA based only on the annotation, it implies that a mistake from the programmer in an annotation leads to a wrong code. A simple example of Java code invoking a kernel with JCUDA is showed in Figure 2.10.H o w e v e r , u s e l e s s t r a n s f e r s c a n n o t b e a v o i d e d i n t h i s m o d e l : the programmer has no control to preserve data on the accelerator between two kernel calls, while the directive approach offer the possibility to the programmer to manage data movement across the whole program.

int[] ThreadsPerBlock = {16, 16, 1}; int[] BlocksPerGrid = new int[3]; BlocksPerGrid[3] = 1; BlocksPerGrid[0] = (NUM1+ThreadsPerBlock[0]-1)/ThreadsPerBlock[0]; BlocksPerGrid[1] = (NUM2+ThreadsPerBlock[1]-

Automatic Parallelization for GPGPU

Not much work has been done about the automatic parallelization of a sequential program toward GPUs.L e u n ge ta l . [ Leung et al. 2009]p r o p o s ea ne x t e n s i o nt oaJ a v a JIT compiler that executes a parallel loop nest on the GPU. The major part of their contributions seems to be the handling of Java exception semantics and Java aliasing at runtime. Nugteren et al. [Nugteren et al. 2011]p r e s e n tat e c h n i q u et oa u t o m a t i c a l l ym a pc o d e on a GPU based on skeletonization. This technique is based on a predefined set of skeletons for image processing algorithms. A Skeletonization step recognizes the algorithm's functionalities in the sequential code using techniques like pattern matching, and replaces them with another implementations for the GPU selected from the available predefined implementations.

Reservoir Labs claims that its R-Stream parallelizing C compiler offers automatic parallelization from C code to CUDA since 2010 [Reservoir Labs 2012]. However, R-Stream is proprietary software not freely available and without academic or evaluation licensing, the few academic publications about this work are vague and there is no way to reproduce their claims and results. CUDA-Chill [START_REF][END_REF]p r o v i d e sa u t o m a t i cp r o g r a mt r a n s f o r m a t i o nf o rGPU using the Chill framework for composing high-level loop transformations. However, the recipes have to be adapted to each input program, limiting the applicability and portability obtained. [START_REF] Baskaran | [END_REF]]introduceapolyhedralapproachtotheautomatic parallelization, using Pluto [Bondhugula et al. 2008c], of affine loop nest from C to CUDA. More recently, the on-going PPCG [Verdo olaege et al. 2013] project follows the same path and produces optimized kernels for GPU using the polyhedral model.

Focus on OpenCL

Open Computing Language (OpenCL)i sa no p e nr o y a l t y -f r e es t a n d a r d for general-purpose parallel programming across CPUs, GPUs and other processors, giving software developers a portable and efficient access to the power of these heterogeneous processing platforms [Khronos OpenCL Working Group 2011].

Introduction

Promoted first by Apple in early 2008, OpenCL was quickly supported by many other vendors such as IBM, Nvidia, AMD,a n dI n t e l . I tp r o v i d e sas o f t w a r es t a c kt h a ta d d r e s s e s the challenges of programming heterogeneous parallel processing platforms. The first revision of the standard exhibits a logical model close to the Nvidia CUDA programming model. OpenCL does not limit itself to the dual CPU vs GPU issue, but also takes into account mobile devices up to high-performance computers, as well as desktop computer systems. It can target different kind of accelerators, like multicore CPUs and GPUs,b u t also more specific devices like Digital Signal Processing (DSP) processors and the Cell processor.

OpenCL API abstracts the hardware at a rather low level. The purpose is to provide high performance by being close-to-metal, and keeping it simple enough for compilers so that the implementation can be easy for a wider range of vendors. OpenCL targets expert programmers who want to write portable and efficient code. Thus it can be seen as the lower level upon which portable libraries, middleware, or software can be built. It also represents a first choice as a backend target for code-generating tools from higher level languages or constructions.

The OpenCL model is split between a host and computing devices in a master-andslaves fashion. The host manages the devices and acts as a choreographer driving the process using the OpenCL API.O nt h ed e v i c es i d e ,t h ec o d et h a ti st ob ee x e c u t e di s contained in kernels. These kernels are written in a language that is based on a subset of ISO C99 with extensions for parallelism. The OpenCL API lets the host indifferently schedule data parallel kernels or task-based kernels or a combination of both.

OpenCL Architecture

The OpenCL standard is organized into four parts: the platform model (see Section 2.3.2.1), the memory model (see Section 2.3.2.3), the execution model (see Section 2.3.2.2), and the programming model (see Section 2.3.2.4).

The whole OpenCL abstract model is shown in Figures 2.11 typically correspond to a GPU, a multi-core CPU, and other processors such as DSPs and the Cell/B.E. processor.T om a n a g et h ed e v i c e s ,t h eh o s tp r o g r a mh a st oc r e a t eo n eo r more contexts. A context is defined as the environment within which the kernels execute and the domain in which synchronization and memory management is defined. The context includes a set of devices, the memory accessible to those devices, the corresponding memory properties and one or more command-queues used to schedule execution of a kernel(s) or operations on memory objects.

Platform Model

OpenCL is strongly based on the concept of one host directly connected to a set of dedicated computing devices. This is the platform in OpenCL terminology. The host plays the role of an orchestrator and manages the devices. These can include many compute units, each made up of many processing elements.

For instance, current OpenCL implementations map a multicore CPU as a single device with as many compute units as the number of cores. The number of processing elements per compute units (per core) depends on the vectorizing capabilities of the OpenCL runtime. The Intel OpenCL runtime for instance, reports sixteen processing elements so that the code can self-align on multiples of sixteen and allows faster loads in vector registers. A GPU is shown as a single device, with the number of compute units corresponding to the available hardware.

Execution Model

The user program drives the host part of the OpenCL model. It acts as an orchestrator for the kernel part of the program. The host part is responsible for managing contexts of execution for the kernels, initializing the devices, controlling the data movements, and scheduling the execution of the kernels on the devices. To achieve this, it creates at least one context.

Contexts are created and managed using an API defined by the standard. A device can be associated with many contexts, and a single context can manage multiple devices. For a given context, each device has its own command queue. A command queue is the only way for the host to request any data transfer by device, or to launch a kernel.

On the kernel side, the execution model is very close to the CUDA programming model: a huge number of virtual threads are mapped onto real hardware threads using what Nvidia calls in CUDA the Single Instruction stream, Multiple Thread streams (SIMT) paradigm. Work-items are group ed in work-groups. Work-items inside a work-group execute on the same compute unit, using multiple processing elements to achieve parallelism. Synchronization can be performed in a work-group but not across different work-groups. A work-group shares also a dedicated memory space (see Section 2.3.2.3). Work-groups are assigned a unique id in the global NDRange the same way as work-items do.

Memory Model

OpenCL exhibits a hierarchical memory model with four distinct spaces:

• The Global Memory is local to a given device, but shared across all work-items for the device. It is accessible for reading or/and writing, depending on how it is allocated. It can be cached or not depending on the underlying hardware.

• The Constant Memory is a part of global memory accessible read-only from the kernel. It has to be initialized from the host. It is usually advised to make use of the constant memory for performance reasons. On some devices it is more likely to be cached and optimized for read access.

• The Local Memory is a small dedicated memory for sharing temporary data between work-items in a work-group, to avoid redundant accesses to the global memory. Depending on the underlying hardware, it can be implemented with a very fast on-chip memory or emulated with a portion of the global memory. For instance on Nvidia Fermi architecture, it is nearly as fast as register accesses.

• This hierarchy is represented in Figure 2.14. There is no guarantee that all these memory areas are really separated on the hardware. The right part of the figure illustrates this situation. For example, classical multicore CPUs do not exhibit any separated memory space or software managed cache, embedded into each core or not. Then a kernel optimized for the more complex memory architecture on the left may lead to spurious costly memory duplication when using local or private memory on the simpler architecture.

At a more global level, the OpenCL runtime manipulates buffers, i.e., linear areas of memory that the host registers with the runtime before any use as a kernel argument. The host can then write to or read from these memory areas using the OpenCL API,o r even directly map the memory area into the host memory space. The physical location of the buffer is undefined by the standard and is implementation specific. From OpenCL version 1.2 on, the programmer can explicitly request a buffer to be moved to a particular device. In any case, before a kernel is launched on a particular device, the OpenCL runtime ensures that the buffers used by the kernel are physically allocated and copied to the device. Therefore it has to keep track of the locations of the buffers and invalidate other copies when a buffer is written by a kernel. The programmer can optimize this management by giving hints at buffer creation times using flags like read-only or write-only. However, these are holding for the whole lifetime of the buffer and thus are not helpful when a buffer is read or written only by some kernels. The const qualifier in the kernel declaration arguments can be used as a hint to the runtime to avoid invalidating other copies of a buffer after a kernel execution.

Programming Model

The OpenCL programming model is a mix of the data parallel and task parallel paradigms. The data parallel one is the preferred way to program OpenCL devices like GPUs.

As explained in Section 2.3.2.2,t h ed a t ap a r a l l e lm o d e li n v o l v e san u m b e ro fw o r kitems that spread over an index set, computing different data in a SIMD/SIMT fashion. The model is relaxed and does not require that each work-item produces one element, and therefore a single work-item can produce as much output as required, or on the other hand only some work-items can produce output. This latter situation occurs when work-items in a work-group work together to produce a single reduced result. Then only one work-item in the work-group is in charge of recording it in the global memory. OpenCL provides full flexibility on this aspect.

The task parallel model is exposed by considering each kernel execution as a task. The parallelism between tasks can then be exploited in two different ways. First, the programmers can issue different tasks to different command queues and thus rely on the Chapter 2. General-Purpose Processing on GPU : History and Context OpenCL runtime to schedule them in parallel. But command queues can also be defined as out-of-order, and then again the OpenCL runtime is free to schedule at the same time as many tasks as submitted to such a queue.

The programmer can issue barriers in the queue to ensure synchronization points, but he can also make use of OpenCL events to enforce dependencies between tasks in a common context, either kernel launches or memory transfers. When a task is submitted to a command queue, a handler on this request is recorded as an OpenCL event. A collection of events can then be used when a new task is submitted in the same context, possibly in a different queue. All events in this collection have to complete before the new task starts.

OpenCL Language

The OpenCL language is a subset of the International Organization for Standardization (ISO) C99 standard. It is used only to create kernels in the OpenCL model.

When compared to plain C, the main differences are the following:

• vector types are natively present, for sizes 2, 3, 4, 8, and 16, and for the native types char, uchar, short, ushort, int, uint, long, ulong, float,a n ddouble;

• the alignment in memory is always guaranteed to be a multiple of the type size. For instance an int16 vector is aligned to a 16*sizeof(int) boundary;

• shuffle can be written directly in a flexible way, for instance a double4 a can be initialized from double4 b and double4 c: a=( b . w ,c . z y x ) ,e q u i v a l e n tt ot h es e q u e n c e :

a.x=b.w; a.y=c.z; a.z=c.y; a.w=c.x;

• keywords are defined for the different memory spaces: __global, __local, __constant and __private.A n yp o i n t e rm u s tm a k eu s eo fo n eo ft h e ms ot h a tt h em e m o r ys p a c e to dereference is always known by the compiler;

• as p e c i a li m a g eo b j e c tc a nb ed e c l a r e da s__read_only or __write_only in kernel argument lists:

__kernel void foo (__read_only image2d_t imageA , __write_only image2d_t imageB);

• the qualifier __kernel is used in front of a kernel declaration. Such a function always returns void. It identifies functions that can be used in an NDRange object issued in a command queue;

• variable length arrays and structures with flexible (or unsized) arrays are not supported;

• variadic macros and functions are not supported;

• The library functions defined in the C99 standard headers assert.h, ctype.h, complex.h, errno.h, fenv.h, float.h, inttypes.h, limits.h, locale.h, setjmp.h, signal.h, stdarg.h, stdio.h, stdlib.h, string.h, tgmath.h, time.h, wchar.h and wctype.h are not available and cannot be included;

• recursion is not supported;

• built-in functions are provided to manage work-items, perform asynchronous or atomic memory operations.

Conclusion

OpenCL is a standard, which by itself is already a good thing for programmers concerned with portability. However, there are some caveats with OpenCL. The performance portability is not enforced and programmers have to write kernels for a given target. Another issue is programmability: OpenCL API is verbose and is rather designed as a target for libraries, frameworks, or code generators. In this case, OpenCL provides all the control that can be wished. Therefore it is suitable as a target for a source-to-source compiler such as the one proposed in this work.

Target Architectures

This thesis focuses on hardware accelerators like GPUs. The common characteristics of such accelerators are as follows:

• large embedded memory: over 1 GB;

• high level of parallelism: from a few tens of processing elements, to many thousands, possibly highly threaded;

• compliance with the OpenCL programming model introduced in Section 2.3.

The most widespread matching hardware platforms are manufactured by AMD and Nvidia, and are indeed ubiquitous in modern desktops. This section introduces some GPU architectures starting from a high-level view to a deeper comparison between the two current leading architectures. It also explains how two kinds of parallelism are exploited: Instruction Level Parallelism (ILP)a n dThread Level Parallelism (TLP).

From Specialized Hardware to a Massively Parallel Device

Dedicated graphic circuits were introduced in the 1980s to offload 2D primitives processing from the main CPU.A tt h a tt i m et h ep u r p o s ew a st od r a ws i m p l eo b j e c t sl i k ea line, a rectangle, or to write some text in the video memory (framebuffer) displayed on the screen.

GPUs then evolved in the 1990s with the introduction of more 3D graphic processing. Starting with the OpenGL API (see Section 2.2.1) and later with Direct3D, a common set of features began to be used by game developers, leading to more and more vendors implementing these features in hardware in the mid-1990s. At that time, GPUs were not programmable at all and provided hardware for a limited set of operations, but there was already some parallel processing involved under the hood. However, it is only during the 2000s that GPUs became programmable, with the introduction of shaders (see Section 2.2.2). GPU designers then continued to fuse pipeline stages into unified programmable units emulating the plain old OpenGL graphic pipeline. This architecture survey starts with the AMD architecture. Then the Nvidia G80 that came along with CUDA,a n dt h ee v o l u t i o no ft h ea r c h i t e c t u r et ot h ec u r r e n tg e n e r a t i o n , are introduced. Finally the impact of architectural choices on high-level code writing is presented. This section focuses exclusively on main breakthroughs that are relevant for GPGPU. Thus it simply ignores changes that introduce only improvements very specific to graphic workloads.

Building a GPU

A GPU is a huge and complicated piece of hardware. It was traditionally built upon units very specialized for fixed graphic processing functions. With the introduction of shaders, it became more and more programmable. As GPGPU are the main focus of this work, only the computation power of shader parts and the memory hierarchy capabilities and specificities are surveyed.

At the lowest level we find the Processing Element (PE),c a p a b l eo fb a s i co p e r a t i o n s like addition or multiplication, or at best a Fused Multiply-Add (FMA). Usually they are limited to single-precision floating point and integer operations. There can also be onsteroid PEs able to compute transcendental functions such as trigonometric, exponential, or square roots. Such a unit is called a Special Function Unit (SFU).

Multiple PEs are then grouped together in a Compute Unit (CU).ACU includes all A GPU chip can then be built by assembling many CUs with an interconnecting network, adding a global hardware scheduler to distribute the work among the CUs and some memory controllers. Sometimes CUs are grouped before being added to the network, and this group shares some resources like cache, on-ship memory network interconnect or also usually graphic centric units. Figure 2.15 illustrates this view of a GPU architecture. Such a view is not so far from what can be seen in a multicore CPU,buttheDevilisin the details. And the choices that are made at each level on the number of elements and the way they are grouped together have a significant impact on the resulting programmability. In general, unlike a CPU,m o s to ft h ed i es p a c ei naGPU is used for computing logics. This is why it has a lot of PEs with complex grouping, little to no cache, an important memory bandwidth, but also a long latency.

Most of the time, designers keep CUs as simple as possible and do not include any out-of-order execution capabilities, thus the main source of parallelism is Thread Level Parallelism (TLP).H o w e v e r ,Instruction Level Parallelism (ILP) can be exploited by the compiler using a VLIW instruction set, or by the hardware scheduler to keep the pipeline full and to mask memory latency if there is not enough TLP.F i g u r e2.16 illustrates the difference between ILP and TLP.

Hardware Atomic Operations

Hardware atomic operations are important for many parallel algorithms and widen the set of applications benefiting from a hardware accelerator. For instance, atomic operations on the GPU have been used to implement barrier synchronizations within a kernel [Xiao &c h u nF e n g2 0 1 0 ], to build high-level programming frameworks such as MapReduce [START_REF] Ji | Using Shared Memory to Accelerate MapReduce on Graphics Processing Units[END_REF]], a memory allocator for MapReduce [START_REF] Hong | [END_REF]], an histogram [Aubert et al. 2009 (perso)].

Nvidia does not disclose any detail about the hardware implementation of atomic operations. It is only known that these units are located in each of the memory controllers [Collange 2010b] on GT200 and directly in the L2 cache since Fermi [START_REF] Halfhill | [END_REF], Patterson 2009, Collange 2010a].

AMD hardware implementation is slower, so much so that some proposals using software emulation were presented as faster [START_REF] Elteir | [END_REF].

OpenCL supports as of version 1.2 the following integer atomic operations in 32-bit mode:

• add:a d d sa ni n t e g e rt oav a l u ea tam e m o r yl o c a t i o n ;

• sub:s u b t r a c t sa ni n t e g e rt oav a l u ea tam e m o r yl o c a t i o n ;

• xchg: swaps an integer with the value at a memory location;

• inc:i n c r e m e n t sav a l u ea tam e m o r yl o c a t i o n ;

• dec:d e c r e m e n t sav a l u ea tam e m o r yl o c a t i o n ;

• cmpxchg:c o m p a r e sa ni n t e g e rt ot h ev a l u ea tam e m o r yl o c a t i o na n dxchg if they are equal;

Figure 2.17: AMD R600 Compute Unit (CU) is built on top of 5-way VLIW instructions set. Four Processing Elements (PE) and a Special Function Unit (SFU) are grouped together in a Processing Unit (PU) to process instructions. These PUs are organized in a 16-wide SIMD array.

• min:c o m p a r e sa ni n t e g e rt oav a l u ea tam e m o r yl o c a t i o na n ds t o r e st h es m a l l e s t value;

• max:c o m p a r e sa ni n t e g e rt oav a l u ea tam e m o r yl o c a t i o na n ds t o r e st h el a r g e s t value;

• and:c o m p a r e sa ni n t e g e rt oav a l u ea tam e m o r yl o c a t i o na n ds t o r e st h er e s u l to fa bitwise and operation;

• or:c o m p a r e sa ni n t e g e rt oav a l u ea tam e m o r yl o c a t i o na n ds t o r e st h er e s u l to fa bitwise or operation;

• xor,c o m p a r e sa ni n t e g e rt oav a l u ea tam e m o r yl o c a t i o na n ds t o r e st h er e s u l to fa bitwise xor operation.

All these functions operate either in global or local memory, and return the old value. The standard specifies 64-bit versions for all these operations, but the implementation is optional and programmers have to check the availability using OpenCL extensions.B o t h 32-bit and 64-bit versions are supported by Nvidia GPUs since Fermi.

AMD, from R300 to Graphics Core Next

Historically, AMD has used a vector instruction set, and then in 2002 introduced a 2-way VLIW ISA at the beginning of computing shaders with the R300. This architecture was proven to be efficient for handling graphics workload until DirectX 10 and its novelties in shading were introduced.

At that time, shading was quite new and pixel and vertex shaders were separate entities. Vertex shader designers decided that VLIW was the ideal architecture for a vertex shader. It allows processing at the same time one SIMD operation on a four-component vector (e.g., w, x, y, z)a n do n eo t h e ro pe r a t i o no nas e p a r a t es c a l a rc o m po n e n t( e . g . ,l i g h t i n g ) . This organization relies on the compiler to pack the VLIW bundles from the Instruction Level Parallelism (ILP)thatcanbefoundinashaderprogram. Bycon trast,Thread Level Parallelism (TLP) is handled by replicating these processing units. The static scheduling done by the compiler simplifies the hardware and allows using more of the die space for compute units instead of a complex hardware scheduler.

DirectX 10 introduces the new geometry shaders (see Section 2.2.2)a n du n i fi e st h e programming language for vertex and pixel shaders. These changes pushed GPU designers to unify the architecture. The same units are in charge of all kind of shaders. For AMD GPUs, this change happened with the R600 chip. To achieve such a change, the hardware had to evolve and include more control logic to schedule the different threads that compete for the computing resources. The introduction of hardware schedulers is an important point for GPGPU.I th a sb e e nc r i t i c a lt oe n a b l ef u r t h e rh a r d w a r ee v o l u t i o n so nl a t e r architectures.

The novelties introduced by the new DirectX 10 version of the HLSL language drove the designers at AMD to choose a more flexible architecture. While previously based on a 2-way vector/scalar VLIW,theR600introduceda5-waypurescalarVLIW instruction set. This way, as before, five individual elements can be processed in each cycle. But the vector has been split. So instead of the same operation on four packed elements, it is possible now to execute five different operations.

ILP is still managed by the compiler that has to pack VLIW bundles. It is even more critical now that five different operations can be packed together. AMD introduced another SIMD level that is exploited implicitly by TLP. The new VLIW units are grouped in a SIMD array of sixteen units. The SIMD benefits only from TLP.A teac hcycle,oneshader 5-way VLIW instruction is scheduled for sixteen different threads. From a graphic workload point of view, it means that a SIMD processing unit handles pixels or vertices by blocks of sixteen, as shown in Figure 2.17.

To increase the computing power of the architecture without increasing the complexity, the control units are limited as much as possible in favor of processing units. A common technique is to use a logical SIMD width wider than the hardware. AMD chose to rely on a virtual sixty-four wide SIMD so that if each cycle a block of sixteen threads is processed, the instruction scheduler can feed the processing units with a VLIW instruction every four cycles on average. This allows the scheduler to run at a lower frequency than the compute units.

SIMD in such GPUs is managed differently than CPU extensions like Streaming SIMD Extension (SSE) or Advanced Vector eXtensions (AVX). GPU registers are not vectors but dynamically reconfigurable arrays of scalar values. The SIMD execution is implicit and managed by hardware. Another effect is that the mapping from the registers to the lightweight threads that run on the GPU is trivially reconfigurable, offering flexibility on the resource sharing.

While no divergence3 occurs between the sixty-four threads, all units execute the instruction. If a branch occurs, then threads diverge and PEs are predicated4 .S i n c et h e r ei s only one program counter for a SIMD unit, the different branches are executed sequentially. This behavior offers flexibility to the programmer, who is able to code in a scalar fashion even if he has to keep in mind the architecture characteristics to avoid divergence as much as possible to maximize performance.

The two next generations R700 and Evergreen (R800) did not introduce major new breakthroughs. R700 scales up the R600: it increases frequency, supports Graphic Double Data Rate (GDDR) Dynamic Random Access Memory (DRAM) in version five, and improves the internal bus. Evergreen again extends R700 with more PEs and CUs. Fused Multiply-Add (FMA) and new DirectX 11 instructions are supported, and also improves PEs precision to be IEEE 754-2008 compliant.

The Radeon HD 6900 series, codename Cayman (R900), was released in 2010. This new generation switched to a narrower 4-way VLIW. This reduces the complexity of the units and it is more efficient on the average according to AMD internal tests. Indeed the VLIW average occupation was established to be 3.4 on common workloads. While shaders that were able to fill the VLIW with four-scalar operation and a transcendental operation at the same time suffer from a performance drop, these are not so common. All other shaders benefit from the increased number of SIMD units and the higher TLP.

The main limitation of VLIW comes from the inherent ILP that the compiler is statically able to find in the source program. Moreover, memory accesses are distinct instructions and have to be separated from Arithmetic and Logical Unit (ALU) instructions in what AMD calls a clause.I fs i m p l eg r a p h i cw o r k l o a d sa r ew e l ls u i t e dt ot h i sc o n s t r a i n t , it can be a difficult limitation for complex shaders and more specially for GPGPU.F i gure 2.18 shows some statistics about the ALU occupation and the VLIW packing ratio 5for some computing kernels.

The last generation, codename Graphics Core Next (GCN)(R1000),introducesamajor breakthrough. Driven by the success of GPGPU, AMD chose to quit their VLIW ISA in favor of a scalar architecture. Basically they have split their 4-way VLIW into four separate SIMD pipelines. It means that ILP will no longer be exhibited by the compiler and that these units exploits TLP instead. As long as the workload exhibits enough threads, it is easier to reach the architectural peak performance.

Another novelty from Graphics Core Next (GCN) is that that these four SIMDs are packed along with an ALU scalar unit. Among other uses, this unit can perform pure scalar computation and avoid wasting resources underusing a SIMD for branch or mask prediction computations, a function call, or a jump. An overview of a GCN's compute unit is given in Figure 2.19.

These changes put more pressure on the hardware scheduler. Early announcements about GCN mention that it is able to schedule ten groups of sixty-four threads per SIMD, that is 2560 threads per compute unit. Such a high number of threads helps to hide memory latency. The presence of four different pipelines to feed increases also the requirements on the scheduler. While previously one VLIW instruction was processed in four cycles by the SIMD pipeline, the scheduler has now to feed a separate instruction every cycle. Indeed it considers each cycle thread for one of the four SIMD and issues up to five instructions among these: one for the vector unit, one for the scalar ALU,o n ef o rav e c t o rm e m o r y access, one for the branching unit, the local data store, for the global data share, or an internal one. 6 GCN also includes for the first a time a fully hierarchical hardware-managed cache, while the previous architecture only had an L2 cache and a software-managed Local Data Store (LDS) located within each CU.

As of early 2012, GCN is not released and we have thus no way to experiment with this new architecture. It is the result of a four-year effort, starting nearly from scratch with a full redesign. While in previous Nvidia architecture the compute units were specialized, the G80 is a complete unification of many stages. As for the AMD R600, one of the most visible and effective change from a GPGPU viewpoint is that the compute units are now indifferently able to process any kind of shader (see Section 2.2.2).

Another important novelty is that the compute units offer a scalar interface, similar to the one AMD announced with AMD GCN,b u ts i xy e a r se a r l i e r .

The G80 has many groups (typically sixteen) of eight scalar processors. Each group is considered by Nvidia as a multiprocessor. There is one shared instruction issue unit for a group, responsible of feeding the eight processors. This feeding unit runs at half the frequency of the scalar processors and needs two cycles to feed an instruction. The GPU efficiency is maximized when thirty-two threads execute the same instruction in a SIMD fashion (see Figure 2.21). Again TLP is exploited and ILP is not directly exhibited by the architecture, but can be exploited to hide memory latency as shown by Volkov [Volkov 2010]. The scheduler can benefit from independent instructions to issue In 2010, Nvidia released a major revision of the GT200: Fermi. It comes with a large number of improvements in a GPGPU perspective:

• Indirect control flow is now supported and opens the gate to C++ and virtual functions.

• Fine grained exception handling has been added to support C++ try-and-catch clause.

• Unified address space allows a simpler memory model where the hardware automatically resolves the location of an address (thread private, shared, global, system).

• Hardware-managed hierarchical caches are introduced for the first time. While the previous generation had read-only caches for texture, Fermi comes with a L1 cache located in each CUs,a n dag l o b a lL 2c a c h e . • The PCI Express (PCIe) interface is now able to execute Direct Memory Access (DMA) in both direction at the same time.

• The global scheduler is now able to process multiple kernels at the same time. A CU still only has in-flight threads from one kernel at a time.

• FMA is supported in the PEs,a n dt h eIEEE-754 rounding modes are all supported.

• Atomic operations execute directly in the L2 cache without having to write back the result in global memory.

• ECC is supported in the global memory, the L1 and L2 caches, and the register file.

Figure 2.22 illustrates the evolution of CU between GT200 and Fermi side by side. The number of PEs is increased from eight to thirty-two, split in two separate pipelines and two schedulers. An instruction is processed each cycle for sixteen threads. Two instructions can be issued every two cycles, ending up with a logical thirty-two wide SIMD view as in G80 and GT200.

The Special Function Units (SFUs) have their own pipeline shared between the two schedulers. Since there are only four SFUs,t h et h r o u g h p u ti sf o u rt i m e sl o n g e rt h a n for ALU operations. ILP allows a scheduler to feed the ALU pipeline while some SFU computations are running. Therefore there can be forty-eight threads processed per CU at the same time.

March 2012 has seen the release of a new major architecture by Nvidia, codename Kepler. The most important point that Nvidia emphasizes with Kepler is the performanceper-watt ratio, that they achieve mostly by reducing the frequency of the PEs by a half to be the same as the instruction scheduling unit. The four big new architectural improvements are the following:

• Dynamic Parallelism adds the capability for the GPU to generate new work for itself.

It make available the CUDA host API directly in the device code. A kernel can then initiate memory transfers, or launch other kernels. This provides a disruptive change in the CUDA programming model as known for years.

• Hyper-Q is the mechanism that allows up to thirty-two host threads to initiate command to the GPU in parallel, allowing more concurrent kernel parallelism to be exploited.

• Grid Management Unit is the basis piece of hardware that enables Dynamic Parallelism. It replaces the previous scheduler providing flexibility in the dispatch, queuing, and dependency of up to 2000 different kernel launches waiting for execution.

• GPU Direct allows transfer of data between different GPUs or between a GPU and any other PCIe piece of hardware directly over the PCIe bus without involving the host.

Kepler is currently available only for gaming and graphical usage with the GTX 680. It does not currently include all the novelty of the architecture that will be available with the Tesla K20 by the end of 2012 along with CUDA 5. Other than these four key features, the most visible change at that time is the organization of the PEs in the CUs.W h i l et h e y were previously grouped by eight on the G80 and thirty-two or forty-eight on Fermi, Kepler is shipped with 192 PEs per CU while keeping the classical logical 32-wide SIMD view. The number of schedulers is doubled to four, but operating now at the same frequency as the PE,i tp r o v i d e st h es a m er a t i oa st h ef o r t y -e i g h tPE Fermi CU.Al s ot h er a t i obe t w e e n double precision and single precision goes down to one third while it was one half on Fermi.

The Gefore GTX 680 is currently shipped with eight CUs,butNvidiaannouncedfifteen CUs,i . e . ,2 8 8 0PE in the Tesla K20, resulting in over one TFlop of double precision throughput and over four TFlops using single precision.

On the memory side, the GDDR5h a sb e e ni m p r o v e da n ds h o u l dp r o v i d ep e r f o r m a n c e closer to the theoretical peak. The announced bandwidth for the Tesla K20 is raised to 320 GB/s, which is nearly twice Fermi's capability. The L2 cache is also doubled both in bandwidth and size, as the memory bandwidth.

It is interesting to note that the balance of resources per CU when compared to Fermi shows that the capacity of each CU scheduling has been doubled in term of number of workgroups but multiplied by only 1.3 in terms of number of threads 7 .I ts e e m st ob ei n favor of smaller workgroups when compared to Fermi. The L1 cache keeps the same size while the number of PE increases, leading to more potential concurrency and contention.

Other less important improvements reside in the more efficient atomic operations, the ECC overhead reduced by 66% on average, the GPU Boost technology that increases or decrease the frequency dynamically to keep the power consumption in a given limit, and a new shuffle instruction to exchange data between threads of a same warp.

Impact on Code Generation

In this dissertation, no particular architecture is targeted and we want to be able to generate code that runs efficiently on all the architectures introduced previously. The main question is this: to what extent is performance portable from one architecture to another? Since scalar and VLIW targets are exposed, it is difficult to expect a unique universal solution. Extracting more ILP may require exposing less TLP and thus might lead to starving on a scalar architecture.

Chapter 7 presents various experiments, and the comparison of the performance obtained on different architectures after various transformations confirms that improving the performance for a given architecture reduces it on another architecture.

Another concern is about predicting statically that one version of a kernel will run faster than another. Even given a particular architecture it is a complex issue. For instance, for av e r ys i m p l ek e r n e l ,t h en u m b e ro fw o r k -i t e m st h a tw ea l l o c a t ei naw o r k -g r o u ph a sa n important impact on the resulting performance. Figure 2.23 shows the influence of runtime parameters on performance of Nvidia Fermi and AMD Evergreen. The left graphic shows different launch configurations for a set of kernels. While Evergreen is not very sensitive to it, Fermi shows up to a speedup of two by adjusting the workgroup size. The comparison of BinomialOption and Matmul_no_smem indicates that there is no universal work-group size. Zhang et al. [Zhang et al. 2011b] demonstrate that the loss in performance when increasing the work-group size from 128 to 256 for BinomialOption is correlated to a larger number of global memory accesses. In this case improving parallelism degrades the overall performance. On the right a matrix multiplication kernel, without any local 7. The maximum number of resident workgroups per multiprocessor is eight on Fermi and sixteen on Kepler, the maximum number of resident threads per multiprocessor is 1536 on Fermi and 2048 on Kepler. data store optimization, is tested with one to four elements produced in each thread. The upper part shows the impact on performance for both architectures while the lower part shows the occupancy 8 of the GPU and, for AMD,t h eVLIW packing ratio. The Evergreen VLIW architecture proves to be very sensitive as more elements to process means more opportunities for the compiler to extract ILP. The performance (upper graphic) and the packing ratio (lower graphic) are correlated and confirm this analysis. Fermi and its scalar architecture are less impacted by this change and exhibit nearly constant performance. But the lower graphic shows that the occupancy drops significantly, leading to fewer opportunities for TLP,a n dt h u spo t e n t i a l l yf e w e ro p po r t u n i t i e st om a s km e m o r y latency with computations in some kernels.

The impact of the launch configuration is explored with more detail in Section 5.8.

8.

Occupancy is ratio of the number of eligible threads over the maximum number of resident threads. Finally, the input size, which generally defines the maximum number of work-items that are exploited, is known only at runtime, which limits the possibilities of one universal version for a particular kernel. Figure 2.24 shows the performance evolution depending on matrix size for two different versions of matrix multiplication on Fermi with and without L1 cache enabled. The local memory is not used in any of these two versions. The fact that L1 can be activated or not on a per kernel basis is another parameter that might influence the choice of a particular kernel version to get the best performance. Section 5.8 covers in detail the implication of the launch configuration over performance, and in Section 7.4 provides experimental results.

Summary

GPUs exhibit massively parallel architecture. They rely mostly on Thread Level Parallelism (TLP)t oe x po s ep a r a l l e l i s ma st h o u s a n d so ft h r e a d s ,b u t ,d e pe n d i n go nt h eu n d e rlying architecture, Instruction Level Parallelism (ILP)m a ya l s ob eam u s tt og e td e c e n t performance.

For a deep er insight in the architectural mysteries, such as latency for each op eration, deep understanding of caches and so on, the reader is referred to the work of Taylor and Li on benchmarking the AMD architecture [START_REF] Taylor | [END_REF], the work of Wong et al. [Wong et al. 2010] and Collange [Collange 2010b] for the GT200, and Lindholm et al. [Lindholm

Conclusion

Adecadeago,theGeneral-Purpose Processing on Graphics Processing Units (GPGPU) computing was in its early days. Since then, it has been an intense field of research and it still very active as shown in Figure 2.25. As shown in this chapter, many languages, frameworks, and other solutions have been introduced to help programmers write programs that exploit accelerators. All these approaches provide varied trade-offs of the three P s: Performance, Portability, Programmability. The performance portability is a challenge in the context of GPGPU. The programmability has been addressed by several specialized programming languages.

The landscape in 2012 is very different from what it looked like more than ten years ago, when researchers were trying to leverage the pure graphic-oriented OpenGL pipeline to abstract mathematical operators [START_REF] Trendall | [END_REF]]o rt ou s ei ta sat a r g e tf o r compiling streaming language.

The programmability was very challenging. However, when a program was successfully mapped to the OpenGL API, performance and portability were obtained. The introduction of shaders in 2002 brought flexibility and exposed more features, resulting in improved programmability.

More recent approaches tried dedicated languages such as Brooks to trade performance for programmability. The underlying streaming programming model is a convenient interface for programmers, but is not flexible enough to be mainstream.

The evolution of DirectX drove GPU manufacturers toward more programmability. But the tipping point is the introduction of CUDA,t h e nf o l l o w e db ya l t e r n a t i v ep r o g r a m m i n g languages and frameworks.

The CUDA and OpenCL programming models leverage the experience with shaders to provide an equivalent level of programmability but without all the rigid mechanisms implied by the graphic API.H o w e v e r ,p r o g r a m m e r sh a v et ok n o ww e l lt h ea r c h i t e c t u r et o write efficient kernels: the portability is traded for performance.

Directive-based languages such as hiCUDA, JCUDA, HMPP, PGI Accelerator, or Ope-nACC (see Section 2.2.10)a r el e s si n v a s i v ea n dp r o v i d eg oodpo r t a b i l i t ya tt h ee x pe n s eo f performance. The directives can be specialized for a given target to increase the performance, but at the price of portability.

My PhD work started just after the first release of OpenCL 1.0 for GPU by Nvidia in spring 2009 [START_REF] Ramey | Languages, APIs and Development Tools for GPU Computing[END_REF]]. The goal of my work was to provide an end-to-end solution that relieves programmers of adapting their codes to hardware accelerators.

The programmability is as good as possible, since programmers write their codes using standard sequential programming languages, ignoring their heterogeneous targets. The compiler extracts the parallelism and the code to be executed on the accelerator. The performance may not match what an expert would get with effort. However, the trade-off on performance is acceptable if it is limited, such as for example ten, twenty, or thirty percent depending on the application domain.

Very few p eople tried to provide full automatic parallelization and transformation (see Section 2.2.11 page 30)f r o ms e q u e n t i a lc o d et oGPU. Most are limited in applicability or focus only on part of the problem. My work tries to process a whole application, generate kernels, optimize them all, and generate the required communication, without any user input.

Guelton proposes in his PhD thesis [Guelton 2011a]ag e n e r a lh i g h -l e v e ls c h e m ef o ra n heterogeneous compiler targeting GPUs. The compiler transforms the code, separating the host code and the kernel code, with the required glue. Each part is then compiled by dedicated binary compilers for the target. This is shown in Figure 2.26.

My work instantiates this high-level compilation scheme. An overview of my compiler structure and organization is presented in Figure 2.27.I ta d d r e s s e sa l lc o m p i l a t i o ni s s u e s raised by heterogeneous computing with CPUs and GPUs.W h i l en o te x p l o r i n gd e e p l y each concern, this dissertation provides solutions to many issues related to automatic parallelization for GPUs,r a n g i n gf r o mp a r a l l e l i s md e t e c t i o nt oc o d eg e n e r a t i o n ,p a s s i n g through loop nests optimizations and management of data mapping and consistency. GPU-like accelerators process data located in their own memory. Indeed, an accelerator board embeds a few gigabytes of memory with high bandwidth to feed their many CUs as discussed in Section 2.4. The difficulty is that this embedded memory is not visible from the host CPU and reciprocally host memory is not visible from the GPU. 1 The programmers then have to explicitly transfer input data from the host memory to the accelerator's before launching a kernel and then execute some opposite transfers from the accelerator memory to the host's after kernel execution for the data produced by the kernel.

These explicit communications use slow I/O buses. For example, PCIe 2.0 bus offers a peak 8 GB/s, to be compared with a few hundreds of GB/s available using the on-board GDDR memory. This is generally assumed to be the most important bottleneck for hybrid systems [START_REF][END_REF]].

Work has b een done to address this issue either using simplified input from programmers [START_REF][END_REF], CAPS Entreprise 2010[START_REF] Wolfe | [END_REF][START_REF] Nvidia | NVIDIA, Cray, PGI, CAPS. The OpenACC Specification, version 1.0[END_REF], or automatically [Amini et al. 2011c (perso), [START_REF] Ventroux | [END_REF], Guelton 2011a, Alias et al. 2011, Wolfe 2010]u s i n gc o m p i l e r s . Al a z ys c h e m eh a sa l s ob e e n proposed by [START_REF] Enmyren | SkePU: a multibackend skeleton programming library for multi-GPU systems[END_REF] in the SkePU C++ template library, a skeleton programming framework for multicore CPUs and multi-GPU systems.

This chapter studies the issues associated with the generation of communication in the context of automatically or semi-automatically offloading work to an accelerator and presents several contributions to address this issue: array regions are exploited to optimize the amount of data to transfer per kernel and a static interprocedural communication optimization scheme is designed and implemented in Paralléliseur Interprocédural de Programmes Scientifiques (PIPS).

PIPS is a twenty-year-old compiler framework [Irigoin et al. 1991, Amini et al. 2011a (perso)] that offers semantic analysis and transformation passes. Initially targeting Fortan 77 as an input, it has been then extended to handle C code. It aims at exploring different program optimizations using interprocedural analyses. Unlike heroes from other projects that target binary level parallelization [START_REF] Pradelle | [END_REF], Kotha et al. 2010], PIPS operates at source level trying to regenerate a code as close as possible to the input.

First, the targeted program class is introduced with a case study: a cosmological simulation. Then the convex array region abstraction, which is the basis of most of the transformations this work relies on, is introduced in Section 3.2. The most basic mapping The parallel promotion presented in Section 3.7 may help as a complement to loop fusion to reduce synchronization and sometimes memory transfers.

Finally, the related work about optimizing the communication for GPGPU is presented in Section 3.8.

Case Study

Small benchmarks like those used in the Polybench suite [START_REF] Pouchet | The Polyhedral Benchmark suite 2.0[END_REF]]forexample,are limited to a few kernels in sequence, sometimes surrounded by a time step loop. Therefore, if they are well suited for studying the pure performance of the GPUs,t h e yc a n n o tb e considered representative of a whole application when it comes to evaluating a global problem like mapping of data between the host and the accelerator. // Read i n i t i a l conditions from a f i l e init_data(argv [START_REF]O n l i n e ; a c c e s s e d 24-February-2012[END_REF]); // Time loop for(t = 0; t < T; t += DT) { // Do computations for each i t e r a t i o n } // Output r e s u l t s to a f i l e write_data(argv [2]); } Ca tObservatoire Astronomique de Strasbourg and was later rewritten and optimized by hand using CUDA to target GPUs [Aubert et al. 2009 (perso)].

This simulation models the gravitational interactions between particles in space. It discretizes the three-dimensional space with a discrete grid on which particles are mapped. Initial conditions are read from a file. A sequential loop iterates over successive time steps, where the core of the simulation is computed. Results are finally extracted from the final grid state and stored in an output file. This general organization is shown in the simplified code in Figure 3.2. It is a common scheme in numerical simulations, while the core of each iteration can vary widely from one domain to the other. The sub-steps performed for processing a single time step in Stars-PM are illustrated Figure 3.3.

Array Region Analysis

Several transformations in the compilation flow used to target hardware accelerators are based on array regions. This section provides a basic introduction to this representation. Three examples are used throughout this section to illustrate this approach: the code in Figure 3.4 requires interprocedural array accesses analysis, the code in Figure 3.5 contains a while loop, for which the memory access pattern requires an approximated analysis, and the code in Figure 3.6 features a nontrivial switch-case.

Convex array regions were first introduced by Triolet [START_REF] Triolet | Contribution à la Parallélisation Automatique de Programmes Fortran Comportant des Appels de Procédures[END_REF][START_REF] Triolet | Rémi Triolet, François Irigoin and Paul Feautrier. Direct parallelization of call statements[END_REF] with the initial purpose of summarizing the memory accesses performed on array element sets by function calls. The concept was later generalized and formally defined for any program statements by Creusillet [Creusillet & Irigoin 1996b, Creusillet 1996]a n di m p l emented in the PIPS compiler framework.

void iteration(coord pos[NP][NP][NP], coord vel[NP][NP][NP], float dens[NP][NP][NP], int data[NP][NP][NP], int histo[NP][NP][NP]) { //
Informally, the read (resp. write)r e g i o n sf o ras t a t e m e n ts are the set of all scalar variables and array elements that are read (resp. written) during the execution of s. This set generally depends on the values of some program variables at the entry point of statement s:t h eread regions are said to be a function of the memory state σ preceding the statement execution, and they are collectively denoted R(s, σ) (resp. W(s, σ) for the write regions).

For instance the read regions associated to the for statement in function kernel in Figure 3.4 are these: where φ x is used to describe the constraints on the xth dimension of an array, and where σ(i) denotes the value of the program variable i in the memory state σ.F r o mt h i sp o i n t , i is used instead of σ(i) when there is no ambiguity.

R(s, σ)={{v}, {i}, {j}, {src(φ 1 ) | φ 1 = σ(i)+σ(j)}, {m(φ 1 ) | φ 1 = σ(j)}} Chapter 3. Data Mapping, Communications and Consistency // R(src)={src[φ 1 ] | i  φ 1  i + k -1} // W(dst)={dst[φ 1 ] | φ 1 = i} // R(m)={m[φ 1 ] | 0  φ 1  k -1} int kernel(int i,int n,int k,int src[n],int dst[n-k],int m[k]) { int v=0; for( int j=0 ;j<k ;+ + j) v+ =s r c [i+j]*m [j] ; dst[i]=v; } void fir( int n, int k, int src[n], int dst[n-k], int m[k]) { for( int i=0 ;i<n-k +1 ;+ + i) // R(src)={src[φ 1 ] | i  φ 1  i + k -1, 0  i  n -k} // R(m)={m[φ 1 ] | 0  φ 1  k -1} // W(dst)={dst[φ 1 ] | φ 1 = i} kernel(i, n, k, src, dst, m); }
// R(randv)={randv[φ 1 ] | N -3  4 ⇥ φ 1 ;3⇥ φ 1  N } // W(a)={a[φ 1 ] | N -3  4 ⇥ φ 1 ; 12 ⇥ φ 1  5 ⇥ N +9} void foo(int N, int a[N], int randv[N]) { int x=N/
The regions given above correspond to a very simple statement; however, they can be computed for every level of compound statements. For instance, the read regions of the for loop on line 6 in the code in Figure 3.4 are these:

R(s, σ)={{v}, {i}, {src(φ 1 ) | i  φ 1  i + k -1}, {m(φ 1 ) | 0  φ 1  k -1}}
However, computing exact sets is not always possible, either because the compiler lacks information about the values of variables or the program control flow, or because the regions

// R(in)={in[φ 1 ] | i  φ 1  i +2} // W(out)={out[φ 1 ] | φ 1 = i} void foo(int n, int i, char c, int out[n], int in[n]) { switch(c){ case 'a': case 'e': out[i]=in[i]; break; default: out[i]=in[3*(i/3)+2]; } } Figure 3
.6: Array regions on a code with a switch case. cannot be exactly represented by a convex polyhedron. In these cases, over-approximated convex sets (denoted R and W) are computed. In the following example, the approximation is due to the fact that the exact set contains holes, and cannot be represented by a convex polyhedron:

W(Jfor(int i=0; i<n; i++) if (i != 3) a[i]=0;K,σ)={{n} , {a[φ 0 ] | 0  φ 0 <n}}
whereas in the next example, the approximation is due to the fact that the condition and its negation are nonlinear expressions that cannot be represented exactly in PIPS framework:

R(Jif(a[i]>3) b[i]=1; else c[i]=1K,σ)= {{i} , {a[φ 0 ] | φ 0 = i} , {b[φ 0 ] | φ 0 = i} , {c[φ 0 ] | φ 0 = i}}
Under-approximations (denoted R and W) are required when computing region differences (see [Creusillet & Irigoin 1996a] for more details on approximations when using the convex polyhedron lattice).

read and write regions summarize the effects of statements and functions upon array elements, but they do not take into account the flow of array element values. For that purpose, in and out regions have been introduced in [Creusillet & Irigoin 1996b]t ot a k e array kills into account, that is to say, redefinitions of individual array elements:

• in regions contain the array elements whose values are imported by the considered statement, which means the elements that are read before being possibly redefined by another instruction of the statement.

• out regions contain the array elements defined by the considered statement, which are used afterwards in the program continuation. They are the live or exported array elements.

As for read and write regions, in and out regions may be over-or under-approximated.

There is a strong analogy between the array regions of a statement and the memory used in this statement, at least from an external point of view, which means excluding its privately declared variables. Intuitively, the memory footprint of a statement can be obtained by counting the points in its associated array regions. In the same way, the read (or in)a n dwrite (or out)r e g i o n sc a nb eu s e dt oc o m p u t et h em e m o r yt r a n s f e r s required to execute this statement in a new memory space built from the original space. This analogy is analyzed and leveraged in the following sections.

Basic Transformation Process

The most basic process for mapping data to the accelerator consists in sending to the accelerator all arrays that are referenced in a kernel prior executing it. The same set of arrays has to be transferred back from the accelerator memory at the end of kernel execution. This basic process is the most basic that can be used by automatic tools. It is represented in Figure 3.7.

The main issue arises when it is needed to count the number of array elements to transfer. Depending on the target language or framework, the information can be hard to figure out. [START_REF] Leung | [END_REF] and JCUDA [START_REF][END_REF]]t a r g e tJ a v a and benefit from runtime information about array sizes. Others such as Verdoolaege and Grosser [Verdo olaege et al. 2013] handle C code but are limited to arrays with size known at compile time. The algorithms used by proprietary software like R-Stream, HMPP,o r PGI Accelerator are unknown, but they are most likely based on the same kind of scheme.

The proposed tool that comes along with this thesis, Par4All [SILKAN 2010 (perso), Amini et al. 2012b (perso)]( s e ed e t a i l e dp r e s e n t a t i o ni nS e c t i o n5.1), relies on the same scheme in its most basic version, relaxing this constraint by handling C99 Variable Length Array (VLA). The effective size is then known only at runtime but the information is available symbolically at compile time. Some polyhedral automatic tools do not consider this problem at all. While converting and automatically optimizing loop nests written in C code into CUDA or OpenCL kernels, they rely on the programmer to generate the host code. This is the case at least for This basic process is illustrated below using as an example the first step of Stars-PM main iteration, the function discretization().F i g u r e3.8 shows the sequential code of this function in its initial version.

The loop nest is detected as parallel and selected to be promoted as a kernel. The mapping on the accelerator is performed according to the technique presented forward in Section 4.2. The loop body is outlined to a new function that will be executed by the accelerator, and the loop nest is replaced by a call to a kernel launch function. Memory transfers are generated according to the basic technique introduced in this section. The resulting code is shown in Figure 3.9.

Looking at the original loop nest, it is clear that the pos array is used in the kernel, whereas the data array is written. Therefore two transfers have to be generated as can be seen in Figure 3.9. The first one ensures that pos are moved to the accelerator before kernel execution while the second one gets the data back into the host memory after the kernel execution.

void discretization(coord pos[NP][NP][NP], int data[NP][NP][NP]

){ int i, j, k; float x, y, z; for (i = 0; i < NP; i++) for (j = 0; j < NP; j++)

for (k = 0; k < NP; k++) { x=p o s [ i ] [ j ] [ k ] . x ; y=p o s [ i ] [ j ] [ k ] . y ; z=p o s [ i ] [ j ] [ k ] . z ; data[i][j][k] = (int)(x/DX)*NP*NP +( int)(y/DX)*NP +( int)(z/DX); } }

Region Refinement Scheme

This section introduces refinement of the basic scheme based on array declarations from the previous section using the convex array regions in Section 3.2.I ta l s oi l l u s t r a t e s informally the process of statement isolation described formally in Guelton's PhD thesis [Guelton 2011b]. It turns a statement s into a new statement Isol(s) that shares no memory area with the remainder of the code, and is surrounded by the required memory transfers between the two memory spaces. In other words, if s is evaluated in a memory state function, σ, Isol(s) does not reference any element of Domain(σ). The generated memory transfers to and from the new memory space ensure the consistency and validity of the values used in the extended memory space during the execution of Isol(s) and once again, back to the original execution path.

To illustrate how the convex array regions are leveraged, the while loop in Figure 3.5 is used as an example. The exact and over-approximated array regions for this statement are as follows: // (2) memcpy(&X, &x, sizeof(x)); memcpy(&Y, &y, sizeof(y)); // ( 3)

R = {{x} , {y}} R(randv)={randv[φ 1 ] | N -3  4 ⇥ φ 1 ;3⇥ φ 1  N } W = {{x} , {y}} W(a)={a[φ 1 ] | N -3  4 ⇥ φ 1 ;12⇥ φ 1  5 ⇥ N +9} void discretization(coord pos[NP][NP][NP],int data[NP][NP][NP]) { //
while(X<=N/3) { A[X+Y-(N-3)/4] = X+Y; if (RANDV[X-Y-(N-3)/4]) X = X+2; else X++,Y++; } memcpy(a+(N-3)/4, A, N/6*sizeof( int));
// ( 4) memcpy(&x, &X, sizeof(x)); memcpy(&y, &Y, sizeof(y)); // (5) } The basic idea is to turn each region into a newly allocated variable, large enough to hold the region, then to generate data transfers from the original variables to the new ones, and finally to perform the required copy from the new variables to the original ones. This results in the code shown in Figure 3.10, where isolated variables have been put in uppercase. Statements (3) and ( 5) correspond to the exact regions on scalar variables. Statements (2) and (4) correspond to the over-approximated regions on array variables. Statement (1) is used to ensure data consistency, as explained later. Notice how memcpy system calls are used here to simulate data transfers, and, in particular, how the sizes of the transfers are constrained with respect to the array regions.

The benefits of using new variables to simulate the extended memory space and of relying on a regular function to simulate the DMA are twofold:

1. The generated code can be executed on a general-purpose processor. It makes it possible to verify and validate the result without the need of an accelerator or a simulator.

2. The generated code is independent of the hardware target: specializing its implementation for a given accelerator requires only a specific implementation of the memory transfer instructions (here memcpy).

Converting Convex Array Regions into Data Transfers

From this p oint on, the availability of data transfer op erators that can transfer rectangular subparts of n-dimensional arrays to or from the accelerator is assumed. For instance, size_t memcpy2d(void*d e s t ,void*s r c , size_t dim1 , size_t offset1 , size_t count1 , size_t dim2 , size_t offset2 , size_t count2); copies from src to dest the rectangular zone between (offset1, offset2) and (offset1 + count1, offset2+count2). dim1 and dim2 are the sizes of the memory areas pointed to by src and dest on the host memory, and are used to compute the addresses of the memory elements to transfer.

We show how convex array regions are used to generate calls to these op erators. Let src be a n-dimensional variable, and

{src[φ 1 ] ...[φ n ] | ψ(φ 1 ,...,φ n )} be a convex region of this variable.
As native DMA instructions are very seldom capable of transferring anything other than a rectangular memory area, the rectangular hull, denoted d•e,i sfi r s tc o m p u t e ds o that the region is expressed in the form

{src[φ 1 ] ...[φ n ] | α 1  φ 1 <β 1 ,...,α n  φ n <β n }
This transformation can lead to a loss of accuracy and the region approximation can thus shift from exact to may. This shift is performed when the original region is not equal to its rectangular envelope.

The call to the transfer function can then be generated with offsetk = α k and

countk = β n -α k for each k in [1 ...n].
For a statement s,t h em e m o r yt r a n s f e r sf r o mσ are generated using its read regions (R(s, σ)): any array element read by s must have an up-to-date value in the extended memory space with respect to σ.S y m m e t r i c a l l y ,t h em e m o r yt r a n s f e r sb a c kt oσ must include all updated values, represented by the written regions (W(s, σ 0 )), where σ 0 is the memory state once s is executed from σ. 2However, if the written region is over-approximated, part of the values it contains may not have been updated by the execution of Isol(s). Therefore, to guarantee the consistency of the values transferred back to σ,t h e ym u s tfi r s tb ec o r r e c t l yi n i t i a l i z e dd u r i n gt h e transfer from σ. These observations lead to the following equations for the convex array Chapter 3. Data Mapping, Communications and Consistency regions transferred from and to σ,r e s pe c t i v e l yd e n o t e dLoad(s, σ) and Store(s, σ):

Store(s, σ)=dW(s, σ)e Load(s, σ)=dR(s, σ) [ (Store(s, σ) -W(s, σ))e
Load(s, σ) and Store(s, σ) are rectangular regions by definition and can be converted into memory transfers, as detailed previously. The new variables with ad-hoc dimensions are declared and a substitution taking into account the shifts is performed on s to generate Isol(s).

Managing Variable Substitutions

For each variable v to be transferred according to Load(s, σ),an e wv a r i a b l eV is declared, which must contain enough space to hold the loaded region. For instance if v holds short integers and

Load(s, σ)={v[φ 1 ][φ 2 ] | α 1  φ 1 <β 1 ,α 2  φ 2 <β 2 } then V will be declared as short int V[β 1 -α 1 ][β 2 -α 2 ]. The translation of an intraproce- dural reference to v into a reference to V is straightforward as 8i, j, V[i][j]=v[i+α 1 ][j +α 2 ].
The combination of this variable substitution with convex array regions is what makes the isolate statement a powerful tool: all the complexity is hidden by the region abstraction. For instance, once the regions of the switch case in Figure 3.6 are computed as

R(c)={c}R (i)={i} W(out)={out[φ 1 ] | φ 1 = i} R(in)={in[φ 1 ] | i  φ 1  i +2}
the data transfer generation and variable substitutions lead to the isolated code given in Figure 3.11. The complexity of the isolated statement does not matter as long as it has been modeled by the convex array region analysis.

For interpro cedural translation, a new version of the called function is created using the following scheme: for each transferred variable passed as an actual parameter, and for each of its dimensions, an extra parameter is added to the call and to the new function, holding the value of the corresponding offset. These extra parameters are then used to The output of the whole process applied to the outermost loop of the Finite Impulse Response (FIR) is illustrated in Figure 3.12, where a new KERNEL function with two extra parameters is now called instead of the original kernel function. These parameters hold the offsets between the original array variables src and m and the isolated ones SRC and M.

void fir( int n, int k, int src[n], int dst[n-k], int m[k]) { int N=n -k+ 1; for( int i=0 ;i<N ;+ + i){ int DST[1],SRC[k],M[k]; memcpy(SRC,
The body of the new KERNEL function is given in Figure 3.13. The extra offset parameters are used to perform the translation on the array parameters. The same scheme applies for multidimensional arrays, with one offset per dimension.

void KERNEL(int i, int n, int k, int SRC[k], int DST[1], int M[k], int SRC_offset , int DST_offset , int M_offset) { int v=0; for( int j=0 ;j<k ;+ + j) v+ =S R C [ i + j -S R C _ o f f s e t ] * M [ j -M _ o f f s e t ] ; DST[i-SRC_offset]=v; }

Limits

Data exchanges between host and accelerator are performed as DMA transfers between Random Access Memory (RAM) memories across the PCI Express bus, which currently offers a theoretical bandwidth of 8 GB/s. This is really small compared to the GPU inner memory bandwidth, which exceeds often 150 GB/s. This low bandwidth can annihilate all gains obtained when offloading computations in kernels, unless they are really computeintensive.

With the available hardware (see Section 7.1), up to 5.6 GB/s was measured from the host to the GPU,a n d6.2 GB/s back. This throughput is obtained for blocks of a few tens of MB, but decreases dramatically for smaller blocks. Moreover, this bandwidth is reduced by more than half when the transferred memory areas are not pinned ;i . e .s u bj e c t to paging by the virtual memory manager of the operating system. Figure 3.14 illustrates this behavior.

Using as reference a cube with 128 cells per edge and as many particles as cells, for af u n c t i o nl i k ediscretization,o n ec o p yt ot h eGPU for particle positions is a block of 25 MB. After execution, one copy back from the GPU for the particle-to-cell association is an 8 MB block.

The communication time for these two copies is about 5 ms. Recent GPUs offer ECC hardware memory error checking that more than doubles time needed for the same copies to 12 ms. Moreover, each buffer allocation and deallocation require 10 ms. In comparison, kernel execution for discretization and this problem size requires only 0.37 ms on the GPU,b u t3 7m so nt h eCPU.

Note that memory transfers and buffer allocations represent the largest part of the total execution time for the discretization step, and therefore the highest potential for obtaining accelerations. This is why the next section exposes a static interprocedural optimization to map data transfers more efficiently.

Communication Optimization Algorithm

Much work has been done regarding communication optimization for distributed computers. Examples include message fusion in the context of Single Program Distributed Data (SPDD) [START_REF] Gerndt | Optimizing communication in SUPERB[END_REF], data flow analysis based on array regions to eliminate redundant communications and to overlap the remaining communications with computations operations [START_REF] Gong | [END_REF], and distribution in the context of High Performance Fortran (HPF) compilation [Coelho 1996[START_REF] Coelho | Optimal Compilation of HPF Remappings[END_REF].

Similar methods are applied in this section to offload computation in the context of a host-accelerator relationship and to integrate in a parallelizing compiler a transformation that limit the amount of CPU-GPU communications at compile time.

This section introduces a new data flow analysis designed to drive the static generation of memory transfers between host and accelerator. The main scheme is first presented and the intraprocedural algorithm is detailed. Then the interprocedural extension of the algo-Chapter 3. Data Mapping, Communications and Consistency rithm is presented. The metric used to evaluate the scheme is introduced and experiments are performed using a 12-core Xeon multiprocessor machine with a Nvidia Tesla GPU C2050. The proposed solution is evaluated on well-known benchmarks [START_REF] Pouchet | The Polyhedral Benchmark suite 2.0[END_REF], Che et al. 2009] before showing how it scales with the real numerical cosmological simulation Stars-PM.

It is assumed for this section that the memory of the GPU is large enough to handle the arrays to process. While this assumption can represent an unacceptable constraint for some workloads, like those encountered when dealing with out-of-core computing, the many gigabytes of memory embedded in modern GPUs are large enough for a wide range of simulations.

A New Analysis: Kernel Data Mapping

At each time step, the function iteration (see in Figure 3.3)u s e sd a t ad e fi n e db y the previous step. The parallelized code performs many transfers to the GPU followed immediately by the opposite transfer.

Our simulation (see in Figure 3.2)p r e s e n t st h ec o m m o np a t t e r no fd a t ad e p e n d e n c i e s between loop iterations, where the current iteration uses data defined during previous ones. Such data should remain on the GPU, with copies back to the host only as needed for checkpoints and final results.

An e wa n a l y s i si si n t r o d u c e df o rt h ec o m p i l e rm i d d l e -e n dt og e n e r a t ee ffi c i e n th o s t -GPU data copying. The host and the accelerator have separated memory spaces; my analysis annotates internally the source program with information about the locations of up-to-date copies: in host memory and/or GPU memory. This allows a further additional transformation to statically determine interesting places to insert asynchronous transfers with a simple strategy: Launch transfers from host to GPU as early as possible and launch those from GPU back to host as late as possible, while still guaranteeing data integrity.

Additionally, launching transfers inside loops is avoided whenever possible. A heuristic is used to place transfers as high as possible in the call graph and in the Abstract Syntax Tree (AST). PIPS uses a hierarchical control flow graph [Irigoin et al. 1991, Amini et al. 2011a (perso)]t op r e s e r v ea sm u c ha sp o s s i b l eo ft h eAST.H o w e v e r ,t os i m p l i f y the presentation of the analyses, equations are written in a classical way assuming that a traditional Control Flow Graph (CFG) is available.

The sets used in the data flow analysis are first introduced. Then the equations used for intraprocedural construction are presented, before extending them to interprocedural construction. Finally the generation of transfers and the lightweight runtime involved to support the copy process are illustrated.

Definitions

The analysis computes the following sets for each statement:

• U is the set of arrays known to be used next (>)b yt h ea c c e l e r a t o r ;

• D is the set of arrays known to be last (<) d efined by the accelerator, and not used on the host in the meantime;

• T H!A is the set of arrays to transfer to the accelerator memory space immediately after the statement;

• T A!H is the set of arrays to transfer from the accelerator to the host immediately before the statement.

These sets are initially empty for every statement. Note that even if array regions are used in the following equations to compute these sets, the granularity is the array. Dataflow equations presented in the next sections describe the computation of these sets on the control-flow graph of the global program. Let S denotes a statement of the program. It can be complex but in order to simplify in the following it is considered that statements are assignments or function calls. A call to a kernel on the GPU is handled through different equations. Such a statement is denoted S k . The control-flow graph is represented with pred(S) for the set of statements that can precede immediately S at execution. Symmetrically, succ(S) stands for the set of statements that can be executed immediately after

S.

As explained in Section 3.2, PIPS computes array regions. These analyses produce fine grained resources; these local fine grained pieces of information are used to build a coarse grained analysis in which arrays are represented atomically. Therefore the equations presented in the following do not require a deep understanding of array regions. The interested reader is referred to Béatrice Creusillet's PhD thesis [Creusillet & Irigoin 1996b] for more information.

In the equations below, the arrays totally or partially written by a statement S are denoted W(S).S i m i l a r l y ,t h ea r r a y sr e a db yS are denoted R(S).

When S is a function call, the set represents the summary of the function, i.e., the set of effects that can be seen on function parameters and on global variables. Moreover, OUT (S k ) represents the set of arrays modified by a kernel for which PIPS established that they are alive, i.e., their value is potentially used by a later statement in the program. By contrast, IN(S k ) stands for the set of arrays consumed by the kernel, i.e., those for which a value is read without being previously produced in the kernel.

Intraprocedural Phase

The analysis begins with the set D in a forward pass through the control flow graph. An array is defined on the GPU for a statement S iff it is also the case for all its immediate predecessors in the control flow graph and if the array is not used or defined by the host, i.e., is not in the set R(S) or W(S) computed by PIPS:

D(S)= 0 @ \ S 0 2pred(S) D(S 0 ) 1 A -R(S) -W(S) (3.1)
The initialization is performed by the first kernel call S k with the arrays defined by the kernel k and used later, OUT (S k ). The following equation is involved at each kernel call site:

D(S k )=OUT (S k ) [ 0 @ \ S 0 2pred(S k ) D(S 0 ) 1 A (3.2)
Abac kw ardpassisthenperformedinordertocomputeU .F o ras t a t e m e n tS,anarra y has its next use on the accelerator iff it is also the case for all statements immediately following in the control flow graph, and if it is not defined by S.

U (S)= 0 @ [ S 0 2succ(S) U (S 0 ) 1 A -W(S) (3.3)
As above with D, U is initially empty and is first initialized at kernel call sites with the arrays necessary to run the kernel, IN(S k ),a n dt h ea r r a y sd e fi n e db yt h ek e r n e l ,W(S k ). These defined arrays have to be transferred to the GPU if it cannot be proved that they are written entirely by the kernel. Otherwise, if not all the values have been updated in the GPU memory, the transfer may overwrite still-valid data on the CPU when copying back the array from the GPU after kernel execution:

U (S k )=IN(S k ) [ W(S k ) [ 0 @ [ S 0 2succ(S k ) U (S 0 ) 1 A (3.4)
An array must be transferred from the accelerator to the host after a statement S iff its last definition is in a kernel and if it is not the case for at least one of the immediately following statements:

T A!H (S)=D(S) - \ S 0 2succ(S) D(S 0 ) (3.5)
This set is used to generate a copy operation at the latest possible location.

An array must be transferred from the host to the accelerator if it has a next use on the accelerator. In order to perform the communication at the earliest, its launch is placed immediately after the statement that defines it, i.e., the statement whose W(S) set contains it. The following equation applies for any S which is not a kernel call.

T H!A (S)=W(S) \ 0 @ [ S 0 2succ(S) U (S 0 ) 1 A (3.6)

Interprocedural Extension

Kernel calls are potentially localized deep in the call graph. Consequently, a reuse between kernels requires interprocedural analysis. The function iteration (see in Figure 3.3) illustrates this situation: each step corresponds to one or more kernel executions.

My approach is to perform a backward analysis on the call graph. For each function f , summary sets D(f ) and U (f ) are computed. They summarize information about the formal parameters of the function and the global variables. These sets can be viewed as contracts. They specify a data mapping that the call site must conform to. All arrays present in U (f ) must be transferred to the GPU before the call, and all arrays defined in D(f ) must be transferred back from the GPU before any use on the host.

Chapter 3. Data Mapping, Communications and Consistency

These sets are required in the computation of D and U when a call site is encountered. Indeed, at a call site c for a function f ,e a c ha r g u m e n to ft h ec a l lt h a tc o r r e s p o n d st oa formal parameter present in U must be transferred to the GPU before the call, because we know that the first use in the called function occurs in a kernel. Similarly, an argument that is present in D has been defined in a kernel during the call and not already transferred back when the call ends. This transfer can be scheduled later, but before any use on the host.

Equations 3.1 and 3.3 are modified for a call site by adding a translation operator, trans f!c ,be t w e e na r g u m e n t sa n df o r m a lp a r a m e t e r s :

D(c)= " trans f!c (D(f )) [ \ S 0 2pred(c) D(S 0 ) !# -R(c) -W(c) (3.7) U (c)= " trans f!c (U (f )) [ [ S 0 2succ(c) U (S 0 ) !# -W(c) (3.8)
The whole process implied by these equations is shown in Figure 3.15.

In the code in Figure 3.16,c o m p a r i n gt h er e s u l to ft h ei n t e r p r oc e d u r a lo p t i m i z e dc od e with the very local approach of Figure 3.9 shows that all communications and memory management instructions (allocation/deallocation) have been eliminated from the main loop.

Runtime Library

Our compiler Par4All includes a lightweight runtime library that lets the generated code be independent from the target (currently OpenCL and CUDA). Par4All also supports common functions such as memory allocation at kernel call sites and memory transfer sites. The runtime relies on a hash table that maps host addresses to GPU addresses. This hash table allows flexibility in the handling of the memory allocations. Using it, the user call sites and function signatures can be preserved, avoiding more advanced and heavy transformations, i.e., duplicating the function arguments for the arrays in the whole call graph and at all call sites to carry the CPU and GPU addresses.

The memory management in the runtime does not free the GPU buffers immediately after they have been used, but preserves them as long as there is enough memory on the GPU.W h e nak e r n e le x e c u t i o nr e q u i r e sm o r em e m o r yt h a ni sa v a i l a b l e ,t h er u n t i m ef r e e s some buffers. The policy used for selecting a buffer to free can be the same as for cache and virtual memory management, for instance Least Recently Used (LRU) or Least Frequently Used (LFU). This behavior requires updating hardware caches in Symmetric MultiProcessing (SMP) with protocols such as MESI. The scheme involved keeps a copy of the data up to date in the CPU and the accelerator memory at the same time. When the host or the accelerator writes data, the copy in the other one is invalidated and a transfer may be scheduled if necessary.

The calls to the runtime that retrieves addresses in the accelerator memory space for arrays pos and data can be noticed in Figure 3 in the accelerator, a lazy allocation is done the first time. The code is lighter than the previous version shown in Figure 3.9,a n de a s i e rt og e n e r a t ef r o mt h ec o m p i l e rp o i n to f view.

Sequential Promotion

Two parallel lo op nests can b e separated by some sequential co de. When this sequential code uses or produces the data involved in the parallel computations, transfers may occur between the host memory and the accelerator memory.

As o l u t i o nt ot h i si s s u ei st op r o m o t et h es e q u e n t i a lc o d ea sparal lel, with only one thread that executes it. Using one thread on the GPU is totally inefficient. However, the slowdown can be dramatically lower than the cost of communication if the code is small enough. This issue is similar to the decision about the profitability whether or not to offload a kernel to the accelerator that is discussed more generally in Section 5.6 page 156.

The gramschmidt example mentioned in the previous section is shown in Figure 3.17.

for (k = 0; k < n; k++) { // The f o l l o w i n g i s s e q u e n t i a l nrm = 0; for

(i = 0; i < m; i++) nrm += A[i][k] * A[i][k]; R[k][k] = sqrt(nrm); // The f o l l o w i n g i s p a r a l l e l for (i = 0; i < m; i++) Q[i][k] = A[i][k] / R[k][k]; for (j = k + 1; j < n; j++) { R[k][j] = 0; for (i = 0; i < m; i++) R[k][j] += Q[i][k] * A[i][j]; for (i = 0; i < m; i++) A[i][j] = A[i][j] -Q[i][k] * R[k][j]; } } kernel_0(R, n); copy_from_accel(R); for(k = 0; k <= n-1; k += 1) { // Sequential nrm = 0; for(i = 0; i <= m-1; i += 1) nrm += A[i][k]*A[i][k]; R[k][k] = sqrt(nrm); // P a r a l l e l region copy_to_accel(R); kernel_1(A, Q, R, k, m); kernel_2(A, Q, R, k, m, n); kernel_3(A, Q, R, k, m, n); copy_from_accel(A); copy_from_accel(R); } (a) Usual Host code. copy_to_accel(A); kernel_0(R, n); for(k = 0; k <= n-1; k += 1) { // Sequential code promoted // on the GPU sequential_kernel(A,R,m,k); // No more // t r a n s f e r s // here kernel_1(A, Q, R, k, m); kernel_2(A, Q, R, k, m, n); kernel_3(A, Q, R, k, m, n); } // t

r a n s f e r s i s outside of the loop copy_from_accel(A);

Chapter 3. Data Mapping, Communications and Consistency

The codes generated with and without sequential promotion illustrate how this trade-off can reduce the communication. The difficulty is to evaluate the trade-off. This depends on both the GPU PEs's pe e da n dt h ePCIe bandwidth. Section 7.8.4,p a g e201, contains measurements showing up to eight times speedup over the optimized scheme for the gramschmidt example, but also up to thirty-seven times speedup for the durbin example from the Polybench suite.

In case of inaccurate evaluation, the performance can be dramatically degraded. This transformation requires a careful evaluation of the execution time of both versions. One possibility to overcome this issue is to perform an off-line profiling with just one iteration of the sequential loop on the GPU and then decide at runtime if the overhead is worth the transfers that must be performed. Such an approach is explored in Section 5.7,p a g e158, however I did not study how it can be mixed with the communication optimization scheme introduced in this chapter.

Experimental Results

Section 7.8,p a g e197,p r e s e n t sd e t a i l e de x pe r i m e n t a lr e s u l t sf o rt h eo p t i m i z i n gs c h e m e introduced in this chapter.

The first question is: what should we measure? While speedup are a very effective metric commercially speaking, in the context of this optimization it is biased because it is largely impacted by input parameters (see Section 7.8.1,p a g e197). The very same benchmark exhibits speedups ranging from 1.4 to fourteen just by changing the input parameters.

Am o r eo bj e c t i v em e a s u r e m e n tf o re v a l u a t i n gt h ep r o po s e da p p r o a c hi st h en u m be ro f communications removed and the comparison with a scheme written by an expert programmer. Focusing on the speedup would also emphasize the parallelizer capabilities.

Using benchmarks from Polybench 2.0 suite and Rodinia, along with the Stars-PM numerical simulation introduced in Section 3.1,S e c t i o n7.8.2,p a g e199 illustrates the performance of the optimizing scheme using this metric, and shows that the optimized code performs almost as well as a hand-written code.

One noticeable exception is gramschmidt. Communications cannot be moved out of any loop due to data dependencies introduced by some sequential code. The parallel promotion scheme shown in Section 3.7 helps by accepting a more slowly generated code and allowing data to stay on the accelerator. This is still valuable while the slowdown is significantly smaller than the communication overhead. The difficulty for the compiler is to evaluate the slowdown and to attempt parallelization only if optimized communications lead to a net performance increase. The result of this scheme, shown in Section 7.8.4,page201,exhibits promising results with a speedup of up to thirty-seven, depending on the test case. Finally Section 7.8.3,p a g e199,e x p l o r e st h epe r f o r m a n c ei m p a c to fd e f e r r i n gt h ed e c ision at runtime using the StarPU library; speedup of up to five times is obtained with the proposed static approach. Although StarPU is a library that has capabilities ranging far beyond the issue of optimizing communications, my static scheme is relevant.

Related Work

Among the compilers that I evaluated, none implement such an automatic static interprocedural optimization. While Lee et al. address this issue [Lee et al. 2009, §4 . 2 . 3 ] , their work is limited to liveness of data and thus quite similar to the unoptimized scheme proposed in Section 3.3.L e u n ga d d r e s s e st h ec a s eo fas e q u e n t i a ll o o ps u r r o u n d i n gak e r n e l launch and moves the communications out of the loop [START_REF] Leung | [END_REF]].

The optimizing scheme proposed in this chapter is independent of the parallelizing scheme involved, and is applicable to systems that transform OpenMP in CUDA or OpenCL like OMPCUDA [Ohshima et al. 2010] My approach can be compared to the algorithm proposed by Alias et al. [START_REF] Alias | [END_REF], Alias et al. 2012b, Alias et al. 2012a]. This work studies, at a very fine grained level, the loading and unloading of data from memory for a tiled code running on a FPGA. My scheme optimizes at a coarse grained level and keeps the data on the accelerator as late as possible.

In a recent paper [START_REF] Jablin | [END_REF]], Jablin et al. introduce CGCM, a system targeting exactly the same issue. CGCM, just like my scheme, is focused on transferring full allocation units. While my granularity is the array, CGCM is coarser and considers a structure of arrays as a single allocation unit. While my decision process is fully static, CGCM makes decisions dynamically. It relies on a complete runtime to handle general pointers to the middle of any heap-allocated structure, which we do not support at this time.

Io b t a i ns i m i l a ro v e r a l ls p e e d u pr e s u l t s ,a n dIu s e dt h es a m ei n p u ts i z e s . H o w e v e r , CGCM is not publicly available and the author does not provide us with a version. Therefore it has not been possible to reproduce their results and compare my solution in the same experimental conditions. Jablin et al. measured a less-than-eight geometric mean speedup vs. mines of more than fourteen. However, a direct comparison of my measurement is hazardous. I used GNU C Compiler (GCC) while Jablin et al. used Clang, which produces a sequential reference code slower than GCC. I measured a slowdown of up to 20% on this benchmark set. Moreover, I made my measurements on a Xeon Westmere while they use an older Core2Quad Kentsfield. They generate their GPU version using a PTX generator for Low Level Virtual Machine (LLVM) while I used NVCC, the Nvidia compiler toolchain.

Finally, a key point is the scope on which the optimization is applied. Jablin et al. perform the optimization across the whole program and measured wall clock time, while I exclude the initialization functions from the scope of my compiler and exclude them from my measurements. Indeed, if I do not do so, the initializations of small benchmarks like the one in the Polybench suite would be parallelized and offloaded on the GPU,t h e nn o copy to the GPU would be required. Therefore I limit myself from optimization possibilities because I consider that this way is closer to what can be seen in real-world programs where initializations cannot usually be parallelized.

The overhead introduced by the runtime system in CGCM is thus impossible to evaluate by a direct comparison of the speedups obtained by my implementation.

Redundant Load-Store Elimination

Note that PIPS also includes another approach to the communication optimization issue that has been described formally in Guelton's thesis [Guelton 2011b]. This section informally describes how this approach uses step-by-step propagation of the memory transfers across the CFG of the host program. PIPS represents the program using a Hierarchical Control Flow Graph (HCFG):f o re x a m p l et h es t a t e m e n t st h a ta r ep a r to fal o o pb o d y are stored at lower level that the loop header. The representation is close to an AST. The main idea is the same as the one expressed earlier in Section 3.6,i . e . ,t om o v eload operations upward in the HCFG so that they are executed as soon as possible, while store operations are symmetrically moved so that they are executed as late as possible. Redundant load-store elimination is performed in the meantime. For instance, load sa n dstores inside a loop may be moved outwards, which is similar to invariant code motion. But this propagation is also performed interprocedurally, as data transfers are also moved outward function boundaries whenever possible.

Interprocedural Propagation

When a load is performed at the entry point of a function, it may be interesting to move it at the call sites. However, this is valid only if the memory state before the call site is the same as the memory state at the function entry point, that is, if there is no write effect during the effective parameter evaluation. In that case, the load statement can be moved before the call sites, after backward translation from formal parameters to effective parameters.

Similarly, if the same store statement is found at each exit point of a function, it may be possible to move it past its call sites. Validity criteria include that the store statement depends only on formal parameters and that these parameters are not written by the function. If this the case, the store statement can be removed from the function call and added after each call site after backward translation of the formal parameters.

Combining Load and Store Elimination

In the meanwhile, the intraprocedural and interprocedural propagation of DMA may trigger other optimization opportunities. Load sandstoresmayforinstanceinteractacross loop iterations, when the loop body is surrounded by a load and a store; or when a kernel is called in a function to produce data immediately consumed by a kernel hosted in another function, and the DMA have been moved in the calling function.

The optimization then consists in removing load and store operations when they are in direct sequence.

This relies on the following property: considering that the statement denoted by "memcpy (a,b,10*sizeof(in))"i saDMA and its reciprocal is denoted by "memcpy(b,a,10*sizeof(in))", then in the sequence memcpy(a,b,10*sizeof(in));memcpy(b,a,10*sizeof(in)),t h es e c o n dc a l lc a nb er e m o v e d since it would not change the values already stored in a. [2], int k [2]) { memload(k, j, sizeof( int)*2); // load moved before c a l l bar(0, j, k); memstore(j, k, sizeof( int)*2);// redundant load eliminated bar(1, j, k); memstore(j, k, sizeof( int)*2);// store moved a f t e r c a l l } Figure 3.18: Illustration of the redundant load-store elimination algorithm. for( int i=0 ;i<N ; (DDR) in the context of a tiled loop nest, where the tiling is done such that tiles execute sequentially on the accelerator while the computation inside each tile can be parallelized.

bar(int i, int j[2], int k[2]) { while (i-->=0) { memload(k, j, sizeof( int)*2); k[0]++; memstore(j, k, sizeof( int)*2); } } void foo(int j[2], int k[2]) { bar(0, j, k); bar(1, j, k); } + void bar(int i, int j[2], int k[2]) { memload(k, j, sizeof( int)*2); // moved outside of the loop while (i-->=0) k[0]++; memstore(j, k, sizeof( int)); } + void bar(int i, int j[2], int k[2]) { while (i-->=0) k[0]++; } void foo(int j

Optimizing a Tiled Loop Nest

+ + i){ if(i==0) { memcpy(SRC,src,k*sizeof( int)); memcpy(M,m,k*sizeof( int)); } else { memcpy(&SRC[i+k-1],&src[i+k-1],1*sizeof( int)); } kernel(i, n, k, SRC, DST, m); if(i<N-1) { memcpy(&dst[i],&DST[i],1*sizeof( int)); } else { memcpy(&dst[i],&DST[i],1*sizeof( int)); } } (b)
While their work is based on the Quasi-Affine Selection Tree (QUAST) abstraction, this section recalls how their algorithm can be used with the less expensive convex array region abstraction.

The classical scheme proposed to isolate kernels would exhibit full communications as shown in Figure 3.19a. An inter-iteration analysis allows avoiding redundant communications and produces the code shown in Figure 3.19b. The inter-iteration analysis is performed on a do loop, but with the array regions. The code part to isolate is not bound by static control constraints.

The theorem proposed for exact sets in [START_REF] Alias | [END_REF], Alias et al. 2012b, Alias et al. 2012a] is the following: 3 3. Regions are supposed exact here; the equation can be adapted to under-and over-approximations.

Load(T )=R(T ) -R(t<T) [ W(t<T) (3.9) Store(T )=W(T ) -W(t>T) (3.10)
where T represents a tile, t<T represents the tiles scheduled for execution before the tile T ,a n dt>T represents the tiles scheduled for execution after T . The denotation W(t>T) corresponds to S t>T W(t). In Theorem 3.1, a difference exists for each loop between the first iteration, the last one, and the rest of the iteration set. Indeed, the first iteration cannot benefit from reuse from previously transferred data and has to transfer all needed data. In other words, R(t<T) and W(t<T ) are empty for the first iteration while W(t>T ) is empty for the last iteration.

For instance, in the co de presented in Figure 3.19a,t h r e ec a s e sa r ec o n s i d e r e d : i =0 ,

0 <i<N-1 and i = N -1.
Using the array region abstraction available in PIPS, a refinement with respect to the naive case can be carried out to compute each case, starting with the full region, adding the necessary constraints and performing a difference.

For example, the region computed by PIPS to represent the set of elements read for array src,i s ,f o re a c ht i l e( h e r ec o r r e s po n d i n gt oas i n g l ei t e r a t i o ni)

R(i)={src[φ 1 ] | i  φ 1  i + k -1, 0  i<N}
For each iteration i of the loop except the first one (here i>0), the region of src that is read minus the elements read in all previous iterations i 0 <ihas to be processed; that is,

S i 0 R(i 0 <i). R(i 0 <i) is built from R(i)
by renaming i as i 0 and adding the constraint 0  i 0 <ito the polyhedron:

R(i 0 <i)={src[φ 1 ] | i 0  φ 1  i 0 + k -1, 0  i 0 < i, 1  i<N} i 0 is then eliminated to obtain S i 0 R(i 0 <i): [ i 0 R(i 0 <i)={src[φ 1 ] | 0  φ 1  i + k -2, 1  i<N}
The result of the subtraction R(i>0) -S i 0 R(i 0 <i) is then the following region:4

Load(i>0) = {src[φ 1 ] | φ 1 = i + k -1, 1  i<N}
This region is then exploited for generating the load sf o ra l li t e r a t i o n sb u tt h efi r s to n e . The resulting code after optimization is presented in Figure 3.19b.W h i l et h en a i v ev e r s i o n loads i ⇥ k ⇥ 2 elements, the optimized version exhibits loads only for i +2⇥ k elements.

Conclusion

With the increasing use of hardware accelerators, automatic or semi-automatic transformations assisted by directives take on an ever-greater importance.

The communication impact is critical when targeting hardware accelerators for massively parallel code like numerical simulations. Optimizing data movements is thus a key to high performance.

An optimizing scheme that addresses this issue has been designed and implemented in PIPS and Par4All.

The proposed approach has been validated against twenty benchmarks of the Polybench 2.0 suite, three from Rodinia, and on one real numerical simulation code. They are presented in Sections 3.1 and 7. As for future work, the cache management in the runtime can be improved further than ac l a s s i cc a c h em a n a g e m e n ta l g o r i t h mbe c a u s e ,u n l i k eah a r d w a r ec a c h e ,t h er u n t i m et h a t comes along the proposed optimizing scheme is software managed and can be dynamically controlled by the compiler inserting hints in the code. Indeed data flow analyses provide knowledge on the potential future course of execution of the program. This can be used in metrics to choose the next buffer to free from the cache. Buffers unlikely to be used again should be discarded first, while those that are certain to be used again should be freed last.

The execution times measured with multicore processors show that attention should be paid to work sharing between hosts and accelerators rather than keeping the host idle during the completion of a kernel. Multicore and multi-GPU configurations are another path to explore, with new requirements to determine accurate array region based transfers and computation localization.

The contributions of this chapter leverage some of the previously existing transformations in PIPS,e x t e n d i n gs o m eo ft h e mt oh a n d l eCc o d e ,i m p r o v i n go t h e r sf o rs p e c i fi c requirements of GPU code generation, and finally introducing new ones.

Introduction

The path leading from a sequential code to efficient parallel code for GPU includes many analyses and transformations. Moreover, some specificities of the input programs have to be taken into account. For instance, hand-written programs do not exhibit the same patterns as automatically generated code from high-level tools or languages. The code in Figure 4.1 shows how a three-line-long Scilab script ends up with temporary arrays and five loop nests.

The whole compilation scheme involved going from the sequential code down to the GPU code is presented in Figure 2.27 page 58 and identifies the contributions presented in this chapter. Section 4.2 explains my scheme to map a parallel loop nest to a GPU,a n dh o wi t leverages an implicit blocking and loop interchange at runtime for the NDRange and workgroup decomposition (see Section 2.3.2.2,p a g e32).

I studied the combination of two different parallelizing algorithms, with analysis of the impact on code generation of both of them in Section 4.3.

Ii m p r o v e dt h ee x i s t i n gr e d u c t i o nd e t e c t i o na n a l y s i st oh a n d l eCc od em o r ea c c u r a t e l y , and leveraged this analysis to enable parallelization of loops with reduction by improving the existing parallelization algorithms. I implemented a mapping scheme for some loops with reductions onto the GPU using atomic operations supported by OpenCL and CUDA (see Section 4.4,p a g e105). Actually, I proposed a new generic scheme for parallelizing loops with reduction, and implemented it in PIPS.I tp r o v i d e si m p r o v e m e n t sf o ro t h e r targets like multicore using OpenMP code generation.

Idesignedandimplementedanewinductionv ariablesubstitutiontransformationbased on linear precondition analysis (see Section 4.5). This transformation can enable the parallelization of loops that contain induction variables.

Ii m p l e m e n t e dt w ol o o pf u s i o nt r a n s f o r m a t i o np h a s e s : o n eb a s e do nt h ed e p e n d e n c e graph and the other on array regions (see in Section 4.6). I designed heuristics to drive the fusion in order to target GPUs. This is particularly critical when processing code generated from high-level tools and languages, such as Scilab, which include many loops.

a=o n e s ( 1 0 0 0 , 1 0 0 0 ) ; b=a + a + a c=b * 2 . + 3 . for(i=0; i<1000; i++) for(j=0; j<1000; j++)

t0[i][j] = a[i][j]+a[i][j]; double b[1000][1000]; for(i=0; i<1000; i++) for(j=0; j<1000; j++) b[i][j] = t0[i][j]+a[i][j]; disp_s0d2("b" ,

Loop Nest Mapping on GPU

Scheduling a parallel loop nest on a GPU using CUDA or OpenCL requires an elaborate mapping from the iteration set of the loop nest to the abstraction of the threaded GPU execution exhibited by NDRange (see Section 2.3.2.2 page 32).

Previous works [START_REF] Baghdadi | [END_REF], Lee et al. 2009, Baskaran et al. 2010]m a d e the compiler aware of the whole execution model hierarchy and tried to express it using nested loops. The transformations performed are principally multilevel tilings with some restructuring including loop interchanges or index set splittings. The approach implemented in our Par4All compiler [Amini et al. 2012b (perso)]isquite different and does not expose any explicit multilevel tiling operation. Instead the source code generated by PIPS keeps a sequential semantics and is specialized at post-processing. Let us assume that loops are first normalized, i.e., that they start at zero and have an increment of one. This is to express the iteration set using the OpenCL concept of NDRange introduced in Section 2.3.2.2.F i g u r e4.3 gives the four steps included in this transformation. First, the body of the initial parallel loop nest in Figure 4.3 is outlined to a new function, the kernel executed by each thread on the GPU. The loop indices are rebuilt in the kernel using two macros P4A_vp_x for each virtual processor dimension. The sequential execution is performed with an expansion of #define P4A_vp_1 ti and #define P4A_vp_0 tj. The parallel loop nest is then annotated with the iteration set as shown in Figure 4.3c.I n fact, the rectangular hull of the iteration set is represented, as it is not possible to be more precise using either OpenCL or CUDA. 1Finally, a post-processing phase matches the annotation and contracts the loop nest to ap s e u d o -c a l lt oaCall_kernel_xd() macro with the NDRange dimension x ranging from one to three. The result of the contraction is shown in Figure 4.3d. This macro abstracts the parallel execution of the kernel on an l ⇥ m ⇥ n grid of threads. The work-group size is not expressed in this abstraction and can then be chosen at runtime according to different parameters.

The proposed abstraction is used to expand the macro at compile time according to the effective target. CUDA, OpenCL, and OpenMP back ends have been implemented. The latter is particularly useful for debugging purposes, since the parallel execution of the kernel is emulated using CPU threads with dedicated buffers in the host memory to simulate the separate memory spaces.

As the iteration set is approximated by a rectangular hull, there may be more threads than necessary to execute the kernel. This situation can occurs in two cases, (1) because of the over-approximation of some triangular iteration sets for example, and ( 2) because the CUDA API requires the iteration set as a multiple of the work-group size. While the former can be detected at compile time, the latter is known only at runtime when the work-group size can be known. The iteration set is then systematically clamped using a guard, as shown in Figure 4.4.

A key point when targeting GPU is memory coalescing. To benefit from the memory bandwidth without suffering from the latency, consecutive threads in a block should access a contiguous memory area. This constraint is naturally respected when writing a code for the CPU. Programmers are taught to write loop nests in such a way that two consecutive iterations access contiguous array elements to exploit spatial locality in the caches. CUDA and OpenCL schedule consecutive threads along the first dimension of the work-group, then along the second dimension, and finally along the last one. Therefore the loop nest must be mapped with the innermost loop along the first work-group dimension. In the proposed representation, the mapping of threads to the work-group dimension is performed in the kernel with the index recovery shown in Figure 4.3b. The macros P4A_vp_x mask the dimension of the work-group along which the index implicitly iterates.

The tiling is implicit since each loop iteration set is potentially split according to the work-group size chosen. Again, the macros P4A_vp_x are involved to perform implicitly this transformation. [Boulet et al. 1998], nonexhaustive).

// p a
The expressiveness of these macros is limited, but still allows application of transformations that would otherwise require some work in the compiler. For instance, loop interchange, strip mining, or tiling can be achieve by using different macros. This representation allows postponing some decisions about the transformations, therefore providing a code that is more target independent while simplifying the compiler internal representation.

Parallelism Detection

Parallelism detection is the foundation of our process. It consists in proving that a loop can be scheduled for a parallel execution. Such techniques are well known in the compiler community, at least since the hyperplane method by Lamport in 1974 [Lamport 1974]. Among all parallelizing algorithms, the most famous is certainly the one from Allen and Kennedy [START_REF] Allen | [END_REF], Allen & Kennedy 1987]. Darte et Vivien, and Boulet et al. survey existing parallelization algorithms [START_REF] Darte | Parallelizing Nested Loops with Approximations of Distance Vectors: A Survey[END_REF], Boulet et al. 1998] and classify them according to the underlying dependence abstraction involved.

[i][j]=i+j; b[i][j]=b[i][j-1]+a[i][j]*c[i-1][j]; c[i][j]=2*b[i][j]+a[i][j]; } } (a) Input code.
# pragma omp parallel for for(i=0; i<N; i++) { #pragma omp parallel for for(j=0 the other one is based on Creusillet's array region analysis [Creusillet & Irigoin 1996b]I detail further now.

; j<N; j++) { a[i][j]=i+j; } } for(i=0; i<N; i++) { for(j=0; j<N; j++) { b[i][j]=b[i][j-1]+a[i][j]*c[i-1][j]; } #pragma omp parallel for for(j=0; j<N; j++) { c[i][j]=2*b[i][j]+a[i][j]; } } (b) After parallelization.

Allen and Kennedy

The Allen and Kennedy algorithm is based on the dependence graph with levels. This algorithm has been proven optimal by Darte and Vivien [Darte & Vivien 1996b]f o rs u c h dependence abstraction. This algorithm was designed for vector machines, and thus in its basic version distributes the loops as much as possible and maximizes parallelism.

The implementation in PIPS uses a dependence graph built using a dependence test [Irigoin et al. 1991] based on a variation of the Fourier-Motzkin pairwise elimina-tion technique [START_REF] Triolet | Rémi Triolet, François Irigoin and Paul Feautrier. Direct parallelization of call statements[END_REF]]. Allen and Kennedy's algorithm is implemented by structuring the dependence graph into strongly connected components, each of which is recursively analyzed with an incremented dependence level.

An example illustrating the result of the processing of the PIPS implementation of the Allen and Kennedy algorithm is presented in Figure 4.5. The loop distribution involved exhibits the maximum parallelism but adds implicit synchronization barriers. Moreover, it can break cache temporal reuse and prevent array scalarization. In the code in Figure 4.5a the same reference to array a appears in the three statements, thus the corresponding array element may stay in the cache. Moreover, if a is not used later in the computation, the reference can be scalarized. The resulting code after transformation (shown in Figure 4.5b) shows that a cannot any longer be scalarized since it is referenced in more than one loop now. Moreover, it is less likely to remain in the caches and the ratio of the number of arithmetic operations to the number of memory accesses decreases. The drawback of loop distribution can be circumvented using a loop fusion algorithm presented in Section 4.6.

Another issue is that this algorithm in this basic form (the one implemented in PIPS) has restrictions on the control flow; for instance, no test is allowed in the loop body. The algorithm introduced in the next section addressed these issues by providing a coarse grained parallelization algorithm based on convex summarized array regions [Creusillet & Irigoin 1996b].

Coarse Grained Parallelization

The second parallelization algorithm is a coarse grained scheme that relies on array region analyses [Creusillet & Irigoin 1996b]. No specific loop transformation is involved. The details about this parallelization method have been published in [Irigoin et al. 2011 (perso)]. The process is summarized below.

Berstein's conditions [START_REF] Bernstein | Analysis of programs for parallel processing[END_REF]]a r eu s e dbe t w e e nt w oi t e r a t i o n sa n de x t e n d st h e original definition to array regions. A loop is scheduled as parallel if no iteration reads or write an array element written by another iteration. It is expressed for any variable v:

{φ |9σ, σ 0 2 P B φ 2 (R B,v (σ) _ W B,v (σ)) ^φ 2 W B,v (σ 0 ) ^(σ(i) 6 = σ 0 (i))} = ∅
with σ the store, P B the preconditions over the loop body, R B,v (σ) the read array region for the whole loop body for variable v,a n dfi n a l l yW B,v (σ 0 ) the written array region for the whole loop body for variable v.

It can be rewritten more precisely:

{φ |9σ, σ 0 2 P B φ 2 (R B,v (σ) _ W B,v (σ)) ^φ 2 W B,v (σ 0 ) ^TB,B (σ, σ 0 )} = ∅
where T B,B stands for the transformer expressing the transition of at least one iteration on the store, i.e., the transitive closure for one iteration T B,B = T + B considering that B includes going to the next iteration.

This algorithm is used extensively in PIPS because it is complementary with Allen and Kennedy. When parallelization is sought instead of vectorization, the Allen and Kennedy distribution adds undesired implicit synchronization barriers. Moreover, no dependence graph is involved, computation of which can be costly. The array regions can be costly as well, but while the dependence graph complexity depends on the number of statements involved, the complexity depends on the size of the linear algebra system resulting from the array accesses. There are no restrictions on the loop body such as on the control flow or function calls as introduced in Section 3.2 page 64;i ta v o i d sl o o pd i s t r i b u t i o na n d thus improves the locality and size of the loops. The main limitation is that the current implementation does not integrate an array privatization phase [Creusillet 1996]a n da reduction detection. This latter point is addressed separately in PIPS as presented in the following section.

Impact on Code Generation

As shown above, there are two existing different parallelization algorithms implemented in PIPS.F i g u r e4.6 illustrates the impact of using one algorithm over the other. While Allen and Kennedy distribute the original loop nest in three different perfectly nested loop nests expressing two-dimensional parallelism, the coarse grained parallelization keeps the original nesting and detects one dimension as parallel. Moreover, the parallel dimension is inside a sequential one, which means that it leads to m kernel launches. Section 7.3 provides experiments about the parallelizing algorithms and shows that overall the Allen and Kennedy scheme leads to a more efficient code on all tested architectures with respect to coarse grained parallelization. While the acceleration is very limited on old architectures such as the G80, dramatic improvement is observable on more recent GPUs with an execution time up to eight times faster on Fermi and four times on Kepler using the example figure 4

[i][j] += data[k][i] * data[k][j]; symmat[j][i] = symmat[i][j]; } } (a) Input code.
for(i=1;i<m;i++)// P a r a l l e l for(j=i;j<=m;j++)// P a r a l l e l symmat[i][j] = 0.0; for(i=1;i<m;i++)// P a r a l l e l for(j=i;j<=m;j++)// P a r a l l e l for(k=1;k<=n;k++)

symmat[i][j] += data[k][i]*data[k][j]
; for(i=1;i<m;i++)// P a r a l l e l for(j=i;j<=m;j++)// P a r a l l e l symmat

[j][i] = symmat[i][j];
(b) After parallelization using Allen and Kennedy.

for(i=1;i<m;i++) { for(j=i;j<=m;j++) {// P a r a l l e l symmat[i][j] = 0.0; for(k=1;k<=n;k++)

symmat[i][j] += data[k][i]*data[k][j]; symmat[j][i] = symmat[i][j]; } }
(c) After parallelization using Coarse Grained method. Figure 4.6: The impact of the two parallelization schemes on a example of code performing a correlation. Allen and Kennedy algorithm results to three different parallel loop nests expressing the maximum parallelism, while the coarse grained algorithm detects only one parallel loop leading to less synchronization but also less exposed parallelism.

Reduction Parallelization

PIPS provides an algorithm for reduction detection based on the unified framework introduced by Jouvelot and Dehbonei [START_REF] Jouvelot | Pierre Jouvelot and Babak Dehbonei. Au n i fi e ds e m a n t i ca pproach for the vectorization and parallelization of generalized reductions[END_REF]]. The implementation is rather straightforward yet powerful. Once detected, the reductions can be parallelized depending on the target capabilities.

Detection

The algorithm is interprocedural and requires that a summary is produced for all the callees in a function. This implies that the algorithm has to be applied first on the leaf of the call graph before handling callers. Intraprocedurally, the following algorithm detects reductions in statements like // c a l l sum [ s [ a ] ] , sum [ b ] ,

s [a] = s [a]+b++;

where you can see the comment added by PIPS that indicates that two reductions have been detected, one on s [a] and the other on b.S t a t e m e n t sa r efi r s tp r o c e s s e di n d i v i d u a l l y and reductions are extracted according to these properties:

1. If the statement is a call, then get the translated interprocedural summary for this call site.

2. The statement is not a call, then perform a recursion on the inner expression to find an operation that would correspond to either an assignment, an update, or an unary operator. The recognized operators are the following: +=, -=, *=, /=, |=, &=, ^=, ++ (pre and post), and --(pre and post).

3. For other than unary op erators, assert that the left-hand side is a reference, either a scalar or an array.

4. Both the left-hand side and the right-hand side expressions (if any) must be side effect free. i.e., if any call occurs it has to be a call to a pure function.

5. In the case of an assignment, the right-hand side has to use a compatible operator, i.e., one of the following: +, -, *, /, min, max, &&, ||, &, |,a n d^.

6. In the case of an assignment, the right-hand side also has to include a reference to the same reference as the one on the left-hand side. Perform this search with a recursion through the right-hand side expression accepting only compatible operators.

7. Assert that there is no effect that may conflict with the reduced variable other than the ones touching reference in the left-hand side of the considered reduction and the reference found in the right-hand side. This prevents the wrong catching of the following two examples The first function call shows that the reduction is duplicated for S and P,b e c a u s et h e y are present for both SUMPROD and FSUMPROD.S i n c et h e ya r ec o m p a t i b l e( o n l ys u mo ro n l y product on a given array) they can be merged and kept, this is not the case for the second call where the two reductions are mutually exclusive (sum and product) and are eliminated of the detected reductions by the algorithm.

The reduction information is summarized at each level of the PIPS' HCFG.F orinstance, it is summarized at loop level so that this information can be used for parallelization. The summarization ensures that there is no other write of the reduced reference that would be incompatible with the reduction.

Reduction Parallelization for GPU

The parallelization of loops with reductions can be handled in different ways. PIPS used to provide only a simple parallelization for OpenMP. This implementation was a simple transformation that was adding an OpenMP pragma with a reduce clause to loops whose statements were all scalar-only reductions.

I have designed and implemented a new method, called Coarse Grained Reductions (CGR) that fits within the existing parallelization algorithms. The implementation is made in the coarse grained parallelization algorithm presented in Section 4.3.2.

The coarse grained parallelization uses array region analysis to find conflicts between two iterations of a loop. Such conflicts prevent parallel scheduling of the loop. The algorithm has been adapted to handle reductions by ignoring conflicts related to references involved in the reduction. If ignoring a conflict eliminates all cycles, the loop is marked as potential ly parallel. Potentially because another transformation replacing the reduction with a parallel compliant equivalent operation is necessary to execute the loop in parallel.

The fact that the parallelization phase does not directly modify the schedule but provides only the information that a potential parallelization may occur provides a decoupling of the reduction detection and the transformation. While this indicates the maximum potential parallelism that may be found in a code, not all reductions can be parallelized depending on the capabilities of the target. The OpenMP output benefits directly from this approach. It is generated by a new transformation that parallelizes a wider range of loops, since it is no longer limited to loops with a body that contains only reduction statements as was the legacy transformation available in PIPS.

Targeting GPU, one way of parallelizing loops with reductions is to make use of hardware atomic operations introduced in Section 2.4.3,P a g e42. Since different GPUs do not share the same capabilities, and since CUDA and OpenCL do not exhibit the exact same set of functions, a rough description of the target capabilities is provided to the compiler. A new implemented transformation exploits this description to select compatible previously detected reductions. If the target supports the corresponding atomic operation, then the substitution is made and the loop is declared as parallel in order to be transformed as a kernel in a further phase.

Parallel Prefix Operations on GPUs

To parallelize some reductions, parallel prefix op erations can b e used. In 1980, Ladner and Fischer introduced parallel prefix reductions [Ladner & Fischer 1980]: this has been a widely studied field since then. In 2004, Buck and Purcell [Buck & Purcell 2004] explained how map, reduce, scan, and sort can be implemented on a GPU using graphic primitives. Sengupta et al. presented later the implementation of parallel prefix scan using CUDA [Sengupta et al. 2007].

In 2010, Ravi et al. introduced a runtime system and framework to generate code from ah i g h -l e v e ld e s c r i p t i o no ft h er e d u c t i o n s [ Ravi et al. 2010]. The runtime scheduling is flexible enough to share the workload between GPUs and multicore CPUs. This system can be seen as a potential back end for an automatic parallelizer like ours.

The recognition and the parallelization of reductions that do not match the classic patterns like the ones detected in PIPS have been widely studied, and is still an active field [START_REF] Redon | Detection of Recurrences in Sequential Programs with Loops[END_REF], Fisher & Ghuloum 1994, Matsuzaki et al. 2006[START_REF] Zou | jopadhye. Scan detection and parallelization in "inherently sequential" nested loop programs[END_REF].

There has been work to provide efficient implementation for parallel prefix operations on GPUs [Sengupta et al. 2007, Harris et al. 2007, Capannini 2011]. An automatic scheme that detects such operations could be associated with a code generator that targets such libraries.

Induction Variable Substitution

Induction variable substitution is a classical transformation to enable parallelization. It is the opposite of strength reduction. Induction variables are usually detected in loops using pattern matching on the initialization and on the updates in the loop body. This section shows how the PIPS precondition analysis [START_REF] Irigoin | Polyèdres et Compilation.I nR e ncontres Francophones du Parallélisme (RenPar'20)[END_REF]i su s e dt od e fi n ea new algorithm to detect and replace induction variables. Given a loop L,t h ea l g o r i t h m processes every statement S in its body, and performs the following steps:

1. Fetch the precondition P associated to S. 4.9b), thanks to the comma operator that evaluates its first operand and discards the result, and then evaluates the second operand and returns this value. The transformed source code is as close as possible to the initial code and the number of statements is left unchanged.

{ sum = B[j-k][k] + A[k]; A[k++] = sum; } } k=S I Z E ; for(i=0;i<SIZE;i++) { if(k--) A[k] += B[j-k][k]; if(--k) A[k] += B[j-k][k]; } (a) Original code. k=-1 ; for(i=0;i<SIZE;i++) { k=i ; for(j=0;j<SIZE;j++) { sum = B[j-(i+j)][i+j]+A[i+j]; A[k = i+j+1, k-1] = sum; } } k=S I Z E ; for(i=0;i<SIZE;i++) { if(k = -2*i+SIZE-1, k-1) A[-2*i+SIZE-1] += B[j-((-2)*i+SIZE -1)][-2*i+SIZE -1]; if(k = -2*i+SIZE -2) A[-2*i+SIZE-2] += B[j-((-2)*i+SIZE -2)][-2*i+SIZE -2]; } (b) After induction substitution.

Loop Fusion

Loop fusion is a transformation that consists in collapsing two or more loops together into one loop. It has been widely studied for a long time [START_REF] Allen | A catalogue of optimizing transformations[END_REF], Burstall &D a r l i n g t o n1 9 7 7 , [START_REF] Kuck | [END_REF][START_REF][END_REF], Goldberg & Paige 1984[START_REF] Wolfe | [END_REF], Bondhugula et al. 2010]. Finding an optimal solution to the global fusion problem is all but trivial [Darte 2000] and there are many ways to address the problem, as well as different for(i=0; i<n; i++) for(j=0; j<m; j++) A[i][j] = (double)1 . 0 ; for(i=0; i<n; i++) for(j=0; j<m; j++) B[i][j] = (double)1 . 0 ; for(i=0; i<n; i++) for(j=0; j<m; j++) a) The code before fusion.

C[i][j] = A[i][j]+B[i][j]; (
for(i=0; i<n; i++) for(j=0; j<m; j++) { A This transformation helps to reduce the overhead of the branching and incrementation by eliminating loop headers and increase the size of the body. It exhibits several benefits such as

[i][j] = (double)1 . 0 ; B[i][j] = (double)1 . 0 ; C[i][j] = A[i][j]+B[i][j]; } (b) After fusion.
• more opportunities for data reuse, mostly temporal locality,

• more instructions can be scheduled, better pipeline usage or ILP,

• further array contraction [START_REF] Gao | [END_REF][START_REF] Sarkar | [END_REF]( s e ea l s oF i g u r e4.21 and Section 4.7).

However, loop fusion has some disadvantages. For instance, the pressure on the instruction cache and on the registers usage within the loop increases.

Legality

Loop fusion is not always legal as it may modify the semantics of the program. An invalid loop fusion can lead to a reverse order of dependent computations. Data dependence analysis is used to determine when the fusion is legal or not.

The validity of loop fusion has been widely studied [START_REF] Allen | A catalogue of optimizing transformations[END_REF], Warren 1984, Aho et al. 1986], but can be expressed in different ways. Allen and Cocke propose simple conditions for the validity of loop fusion in [START_REF] Allen | A catalogue of optimizing transformations[END_REF]]:

1. the control conditions are unique among the loops, 2. the loop headers control the same number of iterations, 3. the loops are not linked by a data dependence.

However, it restricts the number of loops that can fused and prevents any array contraction since no data dependence is allowed. Warren [Warren 1984]p r o p o s e sas l i g h t l y relaxed but still simple set of conditions:

1. the candidate loop nests are consecutive in the source code, 2. induction variables of both loops iterate toward the same upper bound after loop normalization,

3. the fused bodies preserve all the dependences from the first loop to the second loop.

To summarize, the first condition in b oth prop osals aims at avoiding control dep endences, i.e., it ensures that both loops always share the same execution path. The second condition intends to enforce that loops are compatible without any sophisticated transformations such as loop shifting or index set splitting for example, i.e., they have the same number of iterations. Finally, the last conditions guarantee the semantic equivalence of the two fused loops with the initial code. The first proposal is more restrictive because it prevents any data dependence between the two loops, while the second proposal is more general.

The last condition is key in establishing the validity of a loop fusion. It has been shown in [Warren 1984] that dependence with a distance vector allows stating whether the fusion is possible or not. If the distance is positive or null then the fusion is valid. Another definition without distance was given in [START_REF] Kennedy | [END_REF]]. The fusion is valid if no dependence arc from the first loop body to the second is inverted after fusion. Such dependence arcs are called fusion-preventing in the next section. [Irigoin et al. 2011 (perso)]b a s e do na r r a y region analysis [Creusillet & Irigoin 1996b]. The proposal allows identifying these dependences without any dependence graph (see Section 4.6.5).

Irigoin et al. conjectured another solution

When fusing parallel loops, a legal fusion may end up with a sequential loop. This happens when the dependence distance is positive, or, with the alternative definition, when the dependence after fusion becomes carried by the loop. Such dependences might also be considered as fusion-preventing depending on the goals of the algorithm, as shown in

Different Goals

While the earlier algorithms intended to maximize the number of fusions or minimize the total number of loops [START_REF] Allen | A catalogue of optimizing transformations[END_REF], Kuck et al. 1981, Warren 1984]t or e d u c e the control overhead, later contributions extended the goals of loop fusion.

When any bad memory access pattern resulted in swapping a memory page, Kuck et al. studied the applicability of loop fusion for improving performance in environment with virtual memory management [START_REF] Kuck | [END_REF]. It was also used to maximize the usage of vector registers [START_REF] Kuck | [END_REF][START_REF][END_REF] or to enable more effective scalar optimizations such as common subexpression elimination [START_REF] Wolfe | [END_REF]]. Later, fusion was used to increase locality [START_REF] Manjikian | Fusion of Loops for Parallelism and Locality[END_REF], Bondhugula et al. 2010], to generate better access patterns for hardware prefetchers [Bondhugula et al. 2010], or even to reduce power consumption [Zhu et al. 2004, Wang et al. 2010].

Other algorithms using loop fusion were designed to maximize task parallelism with minimum barrier synchronization [START_REF] Allen | Automatic decomposition of scientific programs for parallel execution[END_REF], Callahan 1987].

Kennedy and McKinley introduced an algorithm that focuses on maximizing the parallelism, the ordered typed fusion [START_REF] Kennedy | [END_REF], by avoiding fusing sequential and parallel loops. The type carries the information about the schedule of the loop: parallel or sequential. This solution is minimal in term of number of parallel loops.

They extended this algorithm to handle an arbitrary number of types [START_REF] Kennedy | [END_REF]] in order to handle noncompatible loop headers. They obtained a solution they claim to be minimal in the number of parallel loops and the total number of loops.

They then introduced the general weighted fusion algorithms [Kennedy & Mckin-ley 1993] to maximize reuse. The weight represents any metric that would indicate that it is preferable to fuse a couple of loops instead of another one.

Gao et al. also proposed a weighted loop fusion to maximize array contraction [START_REF] Gao | [END_REF]] based on the maximum-flow/minimum-cut algorithm [START_REF] Dantzig | GD a n t z i g ,RF u l k e r s o na n dSJ o h n s o n . Solution of a large-scale traveling-salesman problem[END_REF]] but they did not address loops with different headers. Their algorithm relies on a Loop Dependence Graph (LDG) (see Section 4.6.4) where edges can be of three types: nonfusible, fusible and contractable, and fusible but noncontractable.

The weighted fusion problem was shown in 1994 to be NP-hard first in [START_REF] Kennedy | [END_REF], then Darte proved it for a broader class of unweighted fusion [Darte 2000], including the typed fusion for two types or more. Loop fusion can also be used to extend the iteration set of a loop, using some index set splitting as shown in Figure 4.12. This technique is explored by [START_REF] Wang | [END_REF].

The loop-fusion transformation has been widely studied in different contexts. The next section present a GPGPU perspective and the associated specific constraint. Items one and two are common benefits when dealing with loop fusion. Since recent GPU architectures include multilevel hardware caches (see Section 2.4), they may benefit from such reuse. However, caches are small when compared to the number of threads, and are intended mostly to provide spatial locality. We expect loop fusion to allow to keep data in registers, avoiding external memory accesses. Finally a fused-kernel exhibits more opportunities for data reuse in local memory, as shown in Section. 2.3.

Loop Fusion for GPGPU

The third benefit exposes potentially more ILP to the hardware scheduler and the compiler. This is helpful for small kernels, but also increases the register pressure and the code size dramatically for larger kernels. Section 2.4.2 demonstrates how ILP is exploited in modern GPUs.

Finally, the last benefit is directly linked to the reduction of the number of kernels, and thus the number of kernel calls. Launching a kernel requires the driver to send the binary code for the kernel, the parameters, and the launch configuration (number of threads, workgroup size) to the GPU over the PCIe bus. Then the hardware scheduler begins scheduling and filling the multiprocessors with many thousands of threads (see Section 2.4). Finally, at the end of the computation, the scheduler has to wait for all the threads to finish, leaving potentially some multiprocessors stalled. These operations are not negligible for small kernels. Stock et al. measured the overhead of starting a kernel of the GPU as 20 µs for an Nvidia GTS 8800 512 and as 40 µs for an Nvidia GTX 280 [START_REF] Stock | Af a s tG P Ui m p l e m e n t a t i o nf o r solving sparse ill-posed linear equation systems[END_REF]. They manually performed loop fusions to reduce the number of kernel launches, and thus the overhead.

Overall, these fusions can improve significantly the performance. Membarth et al. obtained a 2.3 speedup by manually applying loop fusion on a multiresolution filtering application [START_REF] Membarth | [END_REF]]. Wang et al. published measurements with a speedup of five after loop fusion [START_REF] Wang | [END_REF]. Fousek et al. evaluated the fusion of CUDA kernels in the context of predefined parallel map operations. The algorithm they proposed performs an oriented search over the set of valid fusions. They predict the resulting execution times, based on off-line benchmarking of predefined functions. On a simple example that chains six operations (matrix-matrix multiply, matrix-vector multiply, vector normalization, matrixmatrix multiply, matrix-matrix add, matrix-scalar multiply) they obtained a 2.49 speedup. Wang et al. [START_REF] Wang | [END_REF]] study three different types of loop fusion, extending the iteration set to concatenate the two original loop iteration set (see Figure 4.12). Their goal is reduction of power consumption and they do not improve performance with respect to the classical loop fusion scheme implemented in PIPS. Modern GPUs are able to schedule more than one kernel at a time on different multiprocessors. Therefore the performance improvement of this approach, even on small iteration set, is rather theoretical and only the launch overhead of the fused kernel may be avoided.

Loop fusion is also used in the GPU implementation of MapReduce [START_REF] Catanzaro | [END_REF]. Map kernels are fused to reduce synchronizations, communications, and enabling data exchange in on-chip memories.

Finally the Thrust library manual [Bell & Hoberock 2011]r e c o m m e n d sp r o g r a m m e r s to fuse explicitly several computation functions into a single kernel, as shown Figure 4.13. This is presented as a good practice and is a key point in order to get good performance.

Loop Fusion in PIPS

In 2010, PIPS did not include any algorithm for loop fusion. I implemented a heuristicbased algorithm that performs unweighted typed loop fusion. It can take into account two types: parallel and sequential loops.

Most algorithms from the previous sections are based on the Loop Dependence Graph (LDG) [ Gao et al. 1993[START_REF] Megiddo | Optimal weighted loop fusion for parallel programs[END_REF]]. It represents a sequence of loop nests and can be seen as a specialization of the Program Dependence Graph (PDG) [Ferrante et al. 1987]. Each loop nest is represented as a node of the LDG. Edges correspond to dependences between statements that belong to the two loop nest bodies. An edge represents a dependence and the information whether the corresponding dependence prevents The first version is expressed using native Thrust operators and requires temporary arrays, the second version fuses the three steps in one user-defined kernel (source [START_REF] Hoberock | Jared Hoberock and Nathan Bell. Thrust Quick Start Guide[END_REF]).

for(i=0; i<N; i++){//S1 a[i]=b[i]; //S2 a[i]+=2*c[i]; //S3 } for(i=1; i<M; i++){//S4 e[i]=c[i]; //S5 e[i]+=2*b[i]; //S6 } int k;
//S7 k=d [START_REF]O n l i n e ; a c c e s s e d 24-February-2012[END_REF] ;

//S8 int l;

//S9 l=e [START_REF]O n l i n e ; a c c e s s e d 24-February-2012[END_REF] ;

//S10 for(i=1; i<M; i++){//S11 d[i]=2*e[i]; //S12 d[i]+=b[i]; //S13 } for(i=0; i<N; i++){//S14 c[i]+=a[i]; //S15 c[i]+=k+l;
//S16 } In solid red the flow dependences, in dashed blue the anti-dependence, and in dotted green the special dependences that model the declaration. The DG view showed here is simplified for the sake of clarity, for instance output dependences and the loop carried dependences are omitted.

the fusion of its two vertices. This information is a key point for many algorithms from the previous section.

The direction or distance vector dependence graphs, introduced in Section 4.6.1,a r e used to build the LDG with the fusion-compliant status attached to all the edges. But since the PIPS dependence graph exhibits an abstraction based on the dependence level, it does not provide the distance vector in the graph. It would require re-implementing the PIPS dependence test [Irigoin et al. 1991]andthede pe nde nc egraphre pre s e n tationto provide this information.

Without this information, the alternative method for identifying the fusion-preventing nature of an edge (see Section 4.6.1) requires first effectively fusing the loop nests and recomputing a dependence graph on the new body to establish if a dependence is fusionpreventing or not by a comparison with the original graph. However, it would require a lot of spurious computations to do that preventively for all possible combinations of two loop nests.

Instead I designed a heuristic that provides features similar to the typed fusion introduced by [START_REF] Kennedy | [END_REF], but operating on an LDG that does not include any information on the fusion-preventing nature of the edges. The vertices are selected by the algorithm and the fusion is tried. It is only at that time that the validity is checked. The algorithm has to proceed using a trial-and-error strategy.

This algorithm traverses the PIPS Hierarchical Control Flow Graph (HCFG)a n dc o nsiders statements that share the same control in the same way as [Ferrante et al. 1987]. In PIPS terminology these statements are in the same sequence of statements, such as ac o m p o u n db l o c k{. . .}in the C language, with no goto from or to the statements directly in the sequence.

In a sequence, we have to take into account not only loops but also any kind of constructs such as tests, assignments, function calls, etc. The nodes of our graph are not necessarily loop nests. Therefore I use the generic term Reduced Dependence Graph (RDG) instead of LDG, with an example shown on Figure 4.14.

PIPS HCFG represents all these constructs as statements, and they are stored in a linked list to represent the sequence. The RDG is then built by 1. creating a vertex for each statement in the sequence, 2. mapping all the inner statements to the vertex statement, and 3. adding an edge for each edge in the dependence graph to the RDG using the previously computed mappings.

It considers only the dependence arcs related to the statements within the sequence, and then the dependence graph is acyclic, so is the RDG obtained. The algorithm begins with a pruning phase. For each direct edge between two vertices it ensures that there is no other path between them.

The originality of the algorithm is to prune the RDG so that any direct arc between two statements guarantees that no other statement needs to be scheduled between the two statements. Therefore this pruning allows traversing the graph in any topological order and trying to fuse two adjacent statements without having first to check if a fusion would introduce a cycle.

Since PIPS is a source-to-source compiler, the readability of the transformed source is an important point. Each transformation must be designed to keep the code structure as close as possible to the original input code. For example, declarations are represented in the Internal Representation (IR) like any other statements. The DG includes edges from these declarations to every use of the variable. Therefore a transformation algorithm relying on the DG naturally keeps the declarations consistent with the uses of variables. My fusion algorithm allows declarations everywhere in the code without preventing fusion.

My algorithm traverses the graph in three independent ways. Each traversal has a dedicated goal:

1. The first traversal favors array contraction, and thus follows the edges in the RDG that correspond to flow dependences. These represent the opportunities to fuse to get reduced liveness, from the definition to the use of a value.

2. The second traversal improves temporal locality. It fuses vertices of the RDG that are linked by edge corresponding to a read-read dependence. In the context of GPGPU, the purposes are multiple. First, accesses to the same memory location are likely to 4.6. Loop Fusion 123 stay in a register. Then the accesses among a block of threads are more likely to benefit from the cache levels. Finally at a coarser grain level, some transfers between the CPU and the GPU may be avoided.

3. The last traversal is a greedy one that intends to minimize the total number of loops and therefore fuse all vertices in the graph that are not connected. The motivation is to minimize the number of kernel launches and to exhibit more instructions in kernels, leading to more ILP. The drawback is a potentially increased register pressure, and therefore spilling may occur in large kernels.

Note that the second traversal relies on edges that do not exist in the classic DG in PIPS. Moreover, it is not desirable to use them the same way as the other edges since they do not carry the same constraints: they do not imply any order between the connected vertices. One may think about these edges as being undirected arcs. However, since the RDG is a Directed Acyclic Graph (DAG),i tc a n n o ti n c l u d ea n yu n d i r e c t e de d g e . A separated undirected graph structure over the RDG vertices is computed to keep track of data reuse.

The first traversal starts with each vertex that does not have any predecessor in the graph. It then tries to fuse the vertex with each of its successors, and recurses on each, following edges in the RDG as paths. The algorithm presented in Figure 4.16 illustrates this process.

The second traversal is based on edges representing data reuse. The algorithm is similar to the one used for the first traversal and illustrated in Figure 4.16. Edges are pruned like arcs. The pruning involves the information about the arcs. In fact, an edge between two vertices is kept only if there are no paths between the two vertices in the directed graph. This guarantees that fusing two vertices linked by an undirected edge is always valid from the graph point of view, i.e., it does not create any cycle in the RDG.

Finally the last traversal is a greedy one; every possible pair of vertices that are not linked by any path in the RDG are tried for fusion. Figure 4.17 presents the algorithm involved in pruning the graph when merging two vertices. The resulting process of this algorithm on the code shown in Figure 4.14 is presented in Figure 4.18.N o t eh o wt h ed e c l a r a t i o na n di n i t i a l i z a t i o no fk are moved to ensure consistency.

The order these three traversals are performed matters. Since for example in a sequence of three loops, it can arise that the first one can be fused with the second or the third but not both. Therefore the heuristic to chose which fusion to perform instead of the other has // v is modified as side-effect function fuse_rdg(V, E) to consider the criteria that is likely to provide the best performance.

for v 2 V do if v

Loop Fusion Using Array Regions

Section 4.6.1 presents the classic way of determining the legality of a fusion based on the dependence graph. Irigoin conjectured another solution [Irigoin et al. 2011 (perso)]exploiting the array region analysis [Creusillet & Irigoin 1996b]. I designed and implemented in

function fuse_vertices(v, v 0 ) // Fuse v 0 into v pred(v) = pred(v) S pred(v 0 ) \{v} succ(v)=succ(v)
S succ(v 0 ) for s 2 succ(v 0 ) do // For each successor of v 0 ,r e p l a c ev 0 with v as a predecessor pred(s)= pred(s) \{v 0 } S {v} end for for p 2 pred(v 0 ) do if p 6 = v then // For each predecessor of v 0 ,r e p l a c ev 0 with v as a successor succ(p)= succ(p) \{v 0 } S {v} end if end for // Prune the graph, traversing all paths in both direction from v prune(v) end function Figure 4.17: Merging two vertices in the graph while enforcing pruning as introduced in Figure 4.15.

int k;

//S7 k=d [START_REF]O n l i n e ; a c c e s s e d 24-February-2012[END_REF] ; PIPS the corresponding algorithm, FusionWithRegions,t h a ti sb a s e do nas u m m a r i z a t i o n mechanism using array regions (see Section 3.2).

//S8 for(i=1; i<M; i++){//S4 e[i] = c[i]; //S5 e[i] += 2*b[i]; //S6 d[i] = 2*e[i]; //S12 d[i] += b[i]; //S13 } int l; //S9 l=e [ 0 ] ; //S10 for(i=0; i<N; i++){//S1 a[i] = b[i]; //S2 a[i] += 2*c[i]; //S3 c[i] += a[i]; //S15 c[i] += k+l; //S16 }
PIPS computes array region accesses for each structure of its HCFG,i n c l u d i n gl o o p s and their body. This information, summarized at body-level, enables establishing if the loops can be fused with an extended dependence test [START_REF] Triolet | Rémi Triolet, François Irigoin and Paul Feautrier. Direct parallelization of call statements[END_REF]]. Array regions are convex polyhedra. The linear system is used directly in the PIPS dependence test [Irigoin et al. 1991]t oe s t a b l i s ht h ed e p e n d e n c eb e t w e e nt h ea r r a yr e g i o n sa s s o c i a t e dt ot h el o o p bodies.

In the case of conflict, the dependence test states whether the dependence is loop-carried or not, and whether it is a backward or a forward dependence. A forward loop-independent dependence is totally harmless and therefore can be safely ignored for the fusion.

Ab a c k w a r dl o o p -c a r r i e dd e p e n d e n c eb r e a k st h es e m a n t i c so ft h ep r o g r a ma n da l w a y s has to be considered as fusion-preventing [Warren 1984[START_REF] Kennedy | [END_REF].

Finally a forward loop-carried dependence does not break the semantics of the program but serializes the execution of the loop. If the loop-fusion algorithm has to maximize the parallelism, then such a dependence has to be considered as fusion-preventing, if at least one of the original loops is parallel.

The main interest of the FusionWithRegions algorithm is the simplicity of its implementation in PIPS.I tr e l i e so naw e l l -t r i e dp o l y h e d r a lf r a m e w o r ku s e df o ra r r a yr e g i o n s . This solution allows avoiding recomputing a full dependence graph each time a fusion is attempted by the algorithm FUSE_RDG.

Further Special Considerations

As mentioned in Section 4.2,themappingofaloopnestonGPU involves only perfectly nested parallel loops. The fusion algorithm can be parametrized to enforce this property. When a fusion succeeds, if the original loops both contained a single parallel loop as body then a fusion is tried on these inner loops. In case of failure, the fusion of the outer loops is reverted.

The algorithm presented at the previous section has to be applied in sequences. The order sequences are picked for processing during HCFG traversal matters. on the GPUs,p a r a l l e ll oo p st h a ta r el oc a t e di nt h ebod yo fa no u t e rp a r a l l e ll oo pm u s tbe declared as sequential. If they are not, the fusion of the inner parallel loop with another inner sequential loop is prevented. This situation is illustrated in Figure 4.20. The second inner loop should not be declared parallel, so that it can be fused with the previous loop.

Scalarization

Scalarization is a transformation that replaces constant array references to arrays with scalars. This transformation can occur in the usual backend compiler, when it comes to keeping in a register a value fetched from memory as long as possible. Intuitively it means that performing this transformation at source level might increase the pressure on registers and lead to spilling. This transformation can also eliminate temporary arrays, most of the time after loop fusion and especially in the context of automatically generated code from high-level languages and tools. The generated C code from a three-line Scilab program contains three temporary arrays that can be totally replaced with scalars after loop fusion (see in Figure 4.21).

In the context of targeting accelerators like GPUs,t h i st r a n s f o r m a t i o ni se v e nm o r e critical than on a shared memory system. Indeed, the generated kernel will be faster by performing fewer memory accesses, but it is probably from the reduced memory transfers over the PCIe bus that most of the gains are to be expected.

Array scalarization has been widely studied in the past [START_REF] Gao | [END_REF][START_REF] Sarkar | [END_REF][START_REF] Darte | New Results on Array Contraction[END_REF][START_REF] Carribault | Applications of storage mapping optimization to register promotion[END_REF]. This section explores different schemes to apply this transformation in the context of offloading kernels to the GPU. The performance impact is evaluated for different GPU architectures.

Scalarization inside Kernel

As i m p l em a t r i xm u l t i p l i c a t i o nn a i v e l ym a p pe do n t ot h eGPU is shown in Figure 4.22. This kernel includes a sequential loop with a constant array reference. This reference can be kept in a scalar variable during the whole loop. These transformations could be done by the target backend compiler. However, the measurement presented in Figure 7.11, Page 192,indicatesthatperformingitatsourcelev elisv aluableonallarc hitecturestested, with speedup up to 2.39.

Scalarization after Loop Fusion

Loop fusion generates code where definitions and uses of temporary arrays are in the same loop body. The arrays can be totally removed, saving both memory bandwidth and memory footprint. In the context of automatically generated code from high-level languages and tools, this situation is a common pattern. To eliminate a temp orary array, its elements must not b e used later in the program execution. This is checked in PIPS with OUT regions (see Section 3.2,P a g e64).

Perfect Nesting of Loops

In some cases, scalarization can break the perfect nesting of loops. Figure 4.23 illustrates such a situation. Constant references in the innermost loop are assigned to scalars and thus the two loops are no longer perfectly nested. Since only perfectly nested loops are mapped, then here only one of the two loops can be executed in parallel after transformation. The useful parallelism is reduced: fewer threads can execute on the GPU.

The kernel generated without scalarization is shown in Figure 4.23c, while the kernel generated for the outer loop is shown in Figure 4.23e:i tc o n t a i n sas e q u e n t i a ll o o pi n the kernel. Finally, Figure 4.23d illustrates the mapping of the inner loop resulting in a sequential loop on the host. These three versions differ on the host side; as shown on the host code in Figure 4.23d,n oDMA operation is required for u1 and u2. The drawback is that, while the other two versions include only one kernel launch, this one requires as many launches as iterations of the outer loop.

Evaluating which of these three versions leads to the best performance is highly dependent on data size. Here the two loops have different iteration numbers, N and M.F i r s to fa l l , the transfer times of arrays u1 and u2 for the versions in Figure 4.23e and in Figure 4.23c increase with N. The number of threads mapped on the GPU scales with N and M for version in Figure 4.23c, with N for version in Figure 4.23e,a n dM for version 4.23d.B yi n c r e a s i n g the number of threads, more computation has to be done on the accelerator but also more parallelism is exposed and thus it is potentially more likely to keep the GPU busy. The time for computing one kernel with one thread is constant for the versions in Figures 4.23c and 4.23d but scales with M in version in Figure 4.23e.F i n a l l y ,v e r s i o ni nF i g u r e4.23d may suffer from a high number of kernel launches when N grows. This issue is multidimensional, and does not even take into account the difference in performance from one version to the other linked to architectural details like the memory accesses patterns or the potential ILP exposed.

Unless N is large and M is small, version 4.23e suffers from less parallelism exposed with respect to version 4.23c. When we compare this latter with version 4.23d,as m a l lN and a high M may provide a slight advantage to version 4.23d because no data transfer is performed. Although the same amount of data has to be transferred, it will be performed using the arguments of the kernel call instead of a separate DMA.Sincekernelsareexecuted asynchronously, while the first kernel executes the argument for the second kernel are transfered, providing a kind of overlapping of transfers and computation. However, the overhead of N kernel launches can be a lot higher than a single DMA.

preserve the perfect nesting of loops. I modified the existing implementation to enforce this property. Detailed experiments are presented in Section 7.5,P a g e181,a n ds p e e d u p s ranging from 1.12 to 2.39 are obtained.

Loop Unrolling

Loop unrolling (or unwinding) is a well-known loop transformation [START_REF] Aho | [END_REF] to improve the performance of loop execution time. The basic principle is to replicate the loop body many times to perform many iterations. The trip count is then reduced. Figure 4.24 shows an example of such a transformation. The original loop contains only one statement. After unrolling by a factor of four, the new loop body corresponds to four iterations of the original loop. A second loop is added to compute the remaining iterations. Indeed, the unrolled loop can compute only multiples of four iterations and thus, depending on the total number of iterations, the remainder must be processed separately. This transformation is used to reduce the execution time. The original loop in Figure 4.24 contains just a few computations per iteration, thus the overhead of the loop header and the hazard associated to the branching may be significant. Moreover, the Instruction Level Parallelism available to the hardware scheduler is poor. The unrolled loop addresses these issues and exhibits also a larger potential for further optimization. This is obtained by means of increased register pressure [START_REF] Bachir | Post-pass periodic register al location to minimise loop unrol ling degree[END_REF]]a n dal a r g e rc o d e that might break the instruction cache. These shortcomings can annihilate any of the aforementioned benefits.

In spite of its drawbacks, unrolling is a common optimization technique implemented in all mainstream compilers. In the context of GPU programming, this transformation is interesting for two reasons. The first arises when sequential loops are encountered inside kernels, while the second consists in unrolling parallel loops that are mapped on threads as shown in Section 4.2 page 98.I nt h i sl a t t e rc a s e ,i ti sat r a d e -o ffs i n c ei tr e d u c e st h e TLP exposed in favor of potentially more ILP. Section 2.4 presents the GPU architectures, and more especially how ILP can be exploited by current GPU architectures. For instance, some AMD architectures are based on a VLIW instruction set. Sohi & Vajapeyam show that loop unrolling is a must to get speedup for VLIW architectures [Sohi & Vajapeyam 1989]. Lee et al. show interesting speedups obtained with loop unrolling for superscalar architectures [START_REF] Lee | [END_REF]]. Nvidia architectures and the latest AMD one include a hardware scheduler that can ben- Section 7.6 presents the performance gains that I obtained by simply unrolling the inner loop in the kernel from the code in Figure 4.22.A na c c e l e r a t i o no fu pt o1 . 4c a nb e observed depending on the architecture. The register pressure impact is also studied, and it is shown that loop unrolling impacts the register consumption in kernels.

Array Linearization

Fortran and C programs make use of multidimensional arrays. However, this is an issue when using OpenCL:thestandarddoesnotallo wtheuseofm ultidimensionalarra ys. These have to be converted to pointers or 1D arrays and the accesses have to be linearized. This transformation is also mandatory when using CUDA and C99 VLA arrays that are not supported.

The result of this transformation is illustrated in Figure 4.25.

The impact on performance for CUDA code is checked in Section 7.7. This transformation can lead to a slowdown up to twenty percent, but can also, in one configuration, leads to a small speedup of about two percent. There is no reason in my opinion why a standard like OpenCL forbids the use of multidimensional arrays in the formal parameters of the functions, and, considering the performance impact, we hope that a future release will remove this constraint.

void k(int ni ,int nj ,int nk , double A[ni][nk], double B[nk][nj], double C[ni][nj], int i, int j) { int k; C[i][j] = 0; for(k=0; k<nk; ++k) C[i][j]+=A[i][k]* B[k][j]; } (a) Original kernel.

Toward a Compilation Scheme

This chapter presents many individual transformations that are applicable at different times in the whole process. The chaining of all these transformations can be tough and different choices in the process will lead to different performance results.

Iproposedandimplemen tedasimplebutflexiblemappingofparallelloopnestsonthe GPU. This mapping allows keeping the internal representation unaware of the full hierarchy implied by the OpenCL NDRange. I designed and implemented an induction variable substitution to enable parallelization of some loops, using an original scheme based on linear preconditions. I improved the parallelism detection in PIPS,e s pe c i a l l yt h ec o u p l i n g with the reduction detection, and studied the impact of the two different algorithms on the code generation and on the performance on GPUs.Id e s i g n e da n di m p l e m e n t e dd e d i c a t e d parallelization of reductions using the atomic operations available on GPUs.Id e s i g n e d and implemented a loop fusion phase in PIPS, including several different heuristics to favor ap e r f o r m i n gm a p p i n go n t oGPUs.Ii m p r o v e dt h ee x i s t i n gs c a l a r i z a t i o nt r a n s f o r m a t i o n in PIPS to keep the perfect nesting of loops. I identified three different schemes and analyzed the impact of scalarization in these cases. I conducted experiments to validate the individual impact of each of the transformations presented in this chapter. While most concepts are well known, it is shown that for many transformations, the state-of-the-art scheme has to be adapted to the specific requirements of the massively parallel pieces of hardware that are GPUs.

PIPS offers a flexible framework, but the compilation flow among these individual transformations has to be driven to provide an automatic end-to-end solution, as shown in Figure 2.27 on page 58. The next chapter motivates and introduces the concepts of the programmable pass manager implemented in PIPS, which Par4All leverages to provide automated process driving all the transformation steps. While previous chapters are focused on individual transformations implemented in the PIPS framework, this chapter addresses the issue of automating the whole compilation process, from the original source code to the final binary. To this end, we introduce Par4All [SILKAN 2010 (perso), Amini et al. 2012b (perso)]i nS e c t i o n5.1.P a r 4 A l li sa n Open Source initiative that we propose to incubate efforts made around compilers to allow automatic parallelization of applications to hybrid architectures.

As hardware platforms grow in complexity, compiler infrastructures need more flexibility: due to the heterogeneity of these platforms, compiler phases must be combined in unusual and dynamic ways, and several tools may have to be combined to handle specific parts of the compilation process efficiently. The need for flexibility also appears in iterative compilation, when different phases orderings are explored.

In this context, we need to assemble pieces of software like compiler phases without having to dive into the tool internals. The entity in charge of this phase management in as t a n d a r dm o n o l i t h i cc o m p i l e ri sc a l l e dapass manager.W h i l ep a s sm a n a g e r su s u a l l y rely on a statically defined schedule, the introduction of plug-ins in GCC and the current trends in compiler design showcased by LLVM pave the way for dynamic pass schedulers. Moreover, the heterogeneity of targets requires the combination of different tools in the compilation chain. In this context, automating the collaboration of such different tools requires defining a higher level meta pass manager.

The source-to-source aspect is key in this process, as explained in Section 5.2.A programmable pass manager is then introduced in Section 5.3.

Numerical simulations often make use of external libraries such as Basic Linear Algebra Subprograms (BLAS) or Fast Fourier transform (FFT) for example. Section 5.4 presents the handling of such specific libraries for mapping these computations on a GPU. Section 5.5 gives insights on how different tools can collaborate. The profitability decision of offload computation is studied in Section 5.6.S o l u t i o n sf o rs e l e c t i n ga m o n g different versions of a kernel at runtime are presented Section 5.7.F i n a l l y ,S e c t i o n 5.8 explores the impact of launch configuration on kernel performance.

Par4All Project

Recent compilers propose an incremental way for converting software toward accelerators. For instance, the PGI Accelerator [Wolfe 2010]o rHMPP [START_REF] Bodin | son code source pour une architecture ou un modèle d'exécution[END_REF] require the use of directives. The programmer must select the pieces of source that are to be executed on the accelerator. He provides optional directives that act as hints for data allocations and transfers. The compiler then automatically generates a transformed code that targets a specific platform. JCUDA [START_REF][END_REF]] offers a simpler interface to target CUDA from Java. There have been several attempts to automate transformations for OpenMP annotated source code to CUDA [Lee et al. 2009, Ohshima et al. 2010]. The GPU programming model and the host accelerator paradigm greatly restrict the potential of this approach, since OpenMP is designed for a shared memory computer. Recent work [Han & Abdelrahman 2009, Lee & Eigenmann 2010]a d d se x t e n s i o n st oOpenMP to account for CUDA specificity. These make programs easier to write, but the developer is still responsible for designing and writing communications code, and usually the programmer has to specialize his source code for a particular architecture. These previous works are presented with more detail in Section 2.2.

Unlike these approaches, Par4All [SILKAN 2010 (perso), Amini et al. 2012b (perso)] is an automatic parallelizing and optimizing compiler for C and Fortran sequential programs funded by the SILKAN company. The purpose of this source-to-source compiler is to integrate several compilation tools into an easy-to-use yet powerful compiler that automatically transforms existing programs to target various hardware platforms. Heterogeneity is everywhere nowadays, from the supercomputers to the mobile world, and the future seems to be more and more heterogeneous. Thus automatically adapting programs to targets such as multicore systems, embedded systems, high-performance computers and GPUs is ac r i t i c a lc h a l l e n g e .

Par4All is currently mainly based on the PIPS [Irigoin et al. 1991, Amini et al. 2011a (perso)]source-to-sourcecompilerinfrastructureandbenefitsfromitsinterproceduralcapabilities like memory effects, reduction detection, parallelism detection, but also polyhedralbased analyses such as convex array regions [Creusillet & Irigoin 1996b]andpreconditions.

The source-to-source nature of Par4All makes it easy to integrate third-party tools into the compilation flow. For instance, PIPS is used to identify parts that are of interest in a whole program, and then Par4All relies on the PoCC [Pouchet et al. 2010a] or PPCG [Verdoolaege et al. 2013]polyhedralloopoptimizerstoperformmemoryaccessesoptimizations on these parts, in order to benefit from local memory for instance, as shown in Section 5.5.

The combination of PIPS'a n a l y s e st o g e t h e ra n dt h ei n s e r t i o no fo t h e ro p t i m i z e r si n the middle of the compilation flow is automated by Par4All using a programmable pass manager (see Section 5.3) to perform whole-program analysis, spot parallel loops and generate mostly OpenMP, CUDA or OpenCL code.

To that end, we mainly face two challenges: parallelism detection and data transfer generation. The OpenMP directive generation relies on coarse grained parallelization and semantic-based reduction detection, as presented in Section 4.4. The CUDA and OpenCL targets add the difficulty of data transfer management. PIPS helps tackling this using convex array regions that are translated into optimized, interprocedural data transfers between host and accelerator as described in Chapter 3.

Source-to-Source Transformation System

Many previous successful compilers are source-to-source compilers [Bozkus et al. 1994, Frigo et al. 1998, Ayguadé et al. 1999[START_REF] Munk | Acotes project: Advanced compiler technologies for embedded streaming[END_REF]]o rb a s e do ns o u r c e -t o -s o u r c e compiler infrastructures [Irigoin et al. 1991, Wilson et al. 1994, Quinlan 2000, ik Lee et al. 2003, Derrien et al. 2012]. They provide interesting transformations in the context of heterogeneous computing, such as parallelism detection algorithms (see Section 4.3), variable privatization, and many others including those presented in Chapter 4.

In the heterogeneous world, it is common to rely on specific hardware compilers to generate binary code for the part of the application intended to be run on a particular hardware. Such compilers usually take a C dialect as input language to generate assembly code. Thus, it is mandatory for the whole toolbox to be able to generate C code as the result of its processing.

In addition to the intuitive collaboration with hardware compilers, source-to-source compilers can also collaborate with each other to achieve their goal, using source files as a common medium, at the expense of extra conversions between the Textual Representation (TR) and the IR.F i g u r e5.1 illustrates this generic behavior and, in Section 5.5,t h eu s e of external polyhedral tools for some loop nest optimizations is presented. Moreover, two source-to-source compilers written in the same infrastructure can be combined in that way. For instance, an SIMD instruction generator has been used for both the generation of SSE instructions on Intel processors and enhancing the code generated by the CUDA/OpenCL generator presented in Chapter 4.

More traditional advantages of source-to-source compilers include their ease of debugging: the IR can be dumped as a TR at anytime and executed. For the same reason, they are very pedagogical tools and make it easier to illustrate the behavior of a transformation.

Programmable Pass Managers

The continuous search for performance leads to numerous different hardware architectures, as showcased by current trends in heterogeneous computing. To use these architectures efficiently, new languages and paradigms are often introduced, but they typically target only specific architectures. For instance AV X intrinsics target vector registers of recent x86 processors, OpenMP directives target multicores, and most noticeably CUDA targets Nvidia's GPU. It is difficult to master all these language-hardware bindings without losing control of the original code. Thus compilers play a key role for "filling the gap" between generic sequential languages and specific parallel languages [START_REF] Asanović | [END_REF]]. Because of the diversity of targeted architecture, flexibility and retargetability are critical properties for compiler frameworks that must keep up with the ongoing work of hardware designers and founders. Also, applications targeting heterogeneous architectures, e.g. GPGPU with an x86 host, raise new problems such as the generation of different codes in different assembly languages, remote memory management, data transfer generations, etc., thus requiring new functionalities that are not available in current mainline compilers.

A recurrent point when compiling for heterogeneous platforms is the need to dynamically create new functions that will be mapped onto specific pieces of hardware, using a transformation called outlining. This transformation dynamically creates new functions and new compilation units depending on the processed code, so it does not fit well into static pass managers.

Additionally, iterative compilation [Goldberg 1989, Kulkarni et al. 2003]-the process of iteratively transforming, compiling and evaluating a program to maximize a fitness function-is more and more considered as an alternative to standard program optimizations to solve complex problems, but it requires a dynamic reconfiguration of the compilation process. In a compiler infrastructure, the latter is managed by the pass manager. Because of the much more complicated compilation schemes, this pass manager must be flexible and provide ways of overtaking the traditional hard-coded pass sequence to al-low compiler developers to manipulate the interactions between passes and the compiled program dynamically. This section is organized as follows: Section 5.3.1 overviews Pythonic PIPS (PyPS),the pass manager implemented in the PIPS source-to-source compiler framework. It involves an API with a high-level abstraction of compilation entities such as analyses, passes, functions, loops, etc. Building upon the fact that an API is relevant only when used extensively, some cases of use are mentioned in Section 5.3.1.5, with the summarized compilation scheme of distinct compiler prototypes using this model. Finally, related works is presented in Section 5.3.2.

PyPS

Af o r m a lm o d e ld e s c r i p t i o ni sa v a i l a b l ei n [ Guelton et al. 2011a (perso), Guelton et al. 2011b (perso), Guelton 2011a]. Multiple operators are proposed to describe transformations, error handling, and different pass-combination schemes. Guelton also improves it and provides a significantly extended version in his PhD thesis [Guelton 2011a]. The assumption made in PyPS is that the resources required to execute a given pass are transparently provided by an underlying component. In PIPS, the consistency manager PIPS-Make is present. It takes care of executing the analysis required by a compiling pass and keeps track of any change that invalidates the results of an analysis.

Instead of introducing yet another new domain-specific language to express these operators, we benefit from existing tools and languages, taking advantage of the similarity with existing control flow operators. Indeed the transformation composition is similar to af u n c t i o nd e fi n i t i o n ;t h ef a i l s a f eo p e r a t o rc a nb ei m p l e m e n t e du s i n ge x c e p t i o nh a n d l i n g and the conditional composition performs a branching operation. This leads to the idea of using a general-purpose language coupled with an existing compiler infrastructure, while clearly separating the concerns.

Benefiting from Python: on the shoulders of giants

Using a programming language to manage pass interactions offers all the flexibility needed to drive complex compilation processes, without the need of much insight on the actual IR. Conceptually, a scripting language is not required. However, it speeds up the development process without being a burden in terms of performance as all the time is spent in the transformations themselves. Some approaches introduced dedicated language [Yi 2011]forthepassmanagement,but we rather follow the well-known Bernard de Chartres' motto: "on the shoulders of giants" and thus use Python as our base language. This choice proved to be better than expected by not only providing high-level constructions in the language but also by opening access to a rich ecosystem that widens the set of possibilities, at the expense of a dependency on the Python interpreter.

Program Abstractions

In the model presented in [Guelton et al. 2011b (perso), Guelton et al. 2011a (perso)], transformations process the program as a whole. However, transformations can proceed at lower granularity: compilation unit level,1 function level2 or loop level :

• at compilation unit level, decisions based upon the target can be made following the rule of thumb "one compilation unit per target.' This helps drive the compilation process by applying different transformations to different compilation units;

• most functions that consider stack-allocated variables work at the function level:

common subexpression elimination, forward substitution or partial evaluation are good examples;

• a lot of optimizations are dedicated to loop nests, without taking care of the surrounding statements. This is the case for polyhedral transformations.

Interprocedural transformations, like building the cal lgraph, require knowledge of the whole program, or can be improved by such knowledge (e.g. constant propagation ), thus the program granularity is still relevant.

The class diagram in Figure 5.2 shows the relations between all these abstractions. These-and only these-are exposed to the pass manager. The builder,i nc h a r g eo ft h e compilation process, is introduced in Section 5.3.1.4.

Control Structures

The main control structures involved are introduced here. The complete formal description of the operators is found in [Guelton et al. 2011a (perso)

#f i r s t a r g u m e n t i s t h e a r g u m e n tn a m e , s e c o n d i s t h e d e f a u l t v a l u e if conditions.get('if_conversion',False):

module.if_conversion() Conditionals Conditionals are used when transformation scheduling depends on user input or on the current compilation state. Figure 5.3,extractedfromtheSIMD Architecture Compiler (SAC) compiler [Guelton 2011a], illustrates the use of conditionals to implement the -fno-pass-name/-fpass-name switch as in GCC.

For Lo ops For lo ops are used to p erform rep etitive tasks (see in Figure 5.4):

1. applying a transformation to each function or loop of a set;

2. applying a transformation iteratively with varying parameters.

Figure 5.4a illustrates a basic iteration over selected modules in the compiler for Terapix [Guelton 2011a]. Figure 5.4b from the SAC compiler [Guelton 2011a] shows how to unroll all inner loops by a factor of four. Finally, in the SAC compiler, early pattern recognition is also performed. Figure 5.4c demonstrates the use of loops to apply this pass for try: while True: module.redundant_load_store_elimination() module.dead_code_elimination() except: Exceptions A PyPS exception can be raised in three situations:

#T r y m o s t e f f i c i e n t p a r a l l e l i z a t i o n p a s s f i r s t:

#a n e x c e p t i o n i s r a i s e d b y t h e p r e v i o u s #p a s s e s w h e n n o t h i n g i s c l e a n e d u p pass

• au s e ri n p u ti si n v a l i d ;

• ap a s sf a i l s ;

• an internal error happens.

Each of them has a different meaning and can be caught or propagated depending on the pass manager developer goals. A sample usage is given in Figure 5.5.F i g u r e5.6 illustrates the error propagation operator [Guelton et al. 2011a (perso), Guelton et al. 2011b (perso)]o nas e q u e n c eo ft w op a s s e si nal o o p : dead_code_elimination is applied if redundant_load_store_elimination succeeds.

While Loops While loops are useful to achieve a goal within an unknown number of iterations. In the SAC compiler, while loops are combined with exception handling to reach a fix point. Figure 5.6 shows this pattern on a data transfer optimizer.

Chapter 5. Heterogeneous Compiler Design and Automation

Builder

As code for heterogeneous targets is generated in the form of several source code files, afinalcompilationstepisneededtogetthefinalbinary(orbinaries). Stricto sensu,thisis not the role of the pass manager. However, as it is in charge of the source generation, it has information about the mapping of the different files onto the targets and hence about which specific compilers to use. Moreover, complex transformations for hybrid or distributed architectures might require the addition of runtime calls. They are frequently shipped as shared library and thus the final link step is dependent on the transformations performed. For instance, if some FFT operations using FFTW3 are automatically transformed into the equivalent GPU operations with CuFFT, and the link process has to be adjusted. The pass manager can keep track of this. As a consequence, it is useful to provide primitives to drive the final compilation process.

The point here is to associate an automatically generated Makefile to the generated sources. This makefile is a template that holds generic rules to compile CUDA/OpenCL/AV X /OpenMP code, and it is filled with the proper dependencies to run the compilation with appropriate compiler/linker flags. This approach offers the advantage of relieving the user by delegating most of the difficulties to third-party tools, in as o u r c e -t o -s o u r c em a n n e r .

Heterogeneous Compiler Developements

Several real-world architectures have benefited from the flexibility of the proposed pass manager:

Terapix The Terapix architecture [START_REF] Bonnot | [END_REF]i saFPGA-based accelerator for image processing developed by Thales. It achieves high throughput and low energy consumption thanks to a highly specialized architecture and a limited ISA.I np a r t i c u l a ri t uses VLIW instructions written in a dedicated assembly. The compilation scheme summarized in Figure 5.7 shows how the name of generated functions takes part to the process. An external VLIW code compactor is used to compact the generated code, enforcing tool reuse. The generated makefile calls a C compiler for the host code and a specific assembler for the accelerator. This was implemented by Guelton in 2011 and described in detail in his PhD thesis [Guelton 2011a].

The whole compilation scheme is written with 411 Python Source Lines Of Code (SLOC). SAC SAC is an automatic generator of multimedia instructions that can output either AV X , SSE or NEON intrinsics. It is presented in [Guelton 2011a]. It combines polyhedral transformations at the loop level and pattern matching at the function level to generate vector instructions. Figure 5.8 shows an extract of its compilation scheme in charge of tiling as many loops as possible.

The whole compilation scheme is 220 Python SLOC.

An Iterative Compiler PyPS makes it easier to build an iterative compiler. Cloning the workspace (saving the internal state of the compiler), iterating over a parameter domain for transformations such as loop unrolling, is straightforward. A genetic algorithm was implemented to take advantage of PyPS abstraction to test different transformation schedules, benchmark the resulting applications and so forth. Additionally, each iteration is embedded in a remote process, àl aErlang, which gives a free performance boost on distributed multicore machines. The whole compiler is 600 Python SLOC. #directive generation module.generate_omp_pragma() Figure 5.9: OpenMP compilation scheme.

Par4All Par4All (presented in Section 5. 1) is an open-source initiative to incubate efforts made around compilers to allow automatic use of hybrid architectures. It relies heavily on PyPS capabilities. The compilation process is driven by user switches that select different back ends:

• the OpenMP back end uses a classic compilation scheme summarized in Figure 5.9, but still uses the basic pass manager. Two parallelization algorithms are tried successively regardless of the method applied, directives are generated.

• CUDA and OpenCL back ends rely on the parallelization algorithms used for OpenMP and add complex handling in the pass management process to address cases like C99 idioms unsupported by the Nvidia compiler or Fortran to CUDA interface. The subtle differences between the two back ends are mostly handled at runtime. Some other few differences are addressed in the process of pass selection using branches wherever required.

• SIMD targets SSE, AV X , NEON and is eligible as a stand-alone target or as a postoptimization step for OpenMP and OpenCL back ends. A high-level entry point is presented and hooked in the compilation process flow depending on command line options.

• the SCMP back end implemented by Creusillet [Creusillet 2011]generatestask-based applications for an heterogeneous MultiProcessor System-on-Chip (MP-SoC) archi-tecture designed by CEA for the execution of dynamic or streaming applications [Ventroux & David 2010]. To generate intertask communications, the compiler relies on a PIPS phase also used by the CUDA back end. The resulting code is then postprocessed by a phase directly coded in Python for further adaptation to a runtime specific to the SCMP architecture.

The Par4All project was started in 2009, with a first release in July 2010. It is already a large project. It ensures the portability of the original source code to several targets with different options. It is a perfect case study for validating the concepts that were proposed within PyPS,a n di t si m p l e m e n t a t i o ni sa c t u a l l ym a d ee a s i e rb yt h er e u s ea n dt h ee a s y combination of the different constructs available in PyPS.

Par4All represents around ten thousand lines of Python source code, including many other features than the pass management. For the pass management part, the core system takes only seventy-one SLOC,theOpenMP part sixteen, the SCMP part thirty-eight, and the CUDA/OpenCL part 117. This one is larger than expected because the transformation of Fortran or C99 to CUDA requires much special handling.

Related Work

In traditional compilers such as GCC,ac o m p i l a t i o ns c h e m ei ss i m p l yas e q u e n c eo f transformations chained one after the other and applied iteratively to each function of the original source code [Wikibooks 2009]. This rigid behavior led to the development of ap l u g -i nm e c h a n i s mm o t i v a t e db yt h ed i ffi c u l t yo fp r o v i d i n ga d d i t i o n a lf e a t u r e st ot h e existing infrastructure. Samples of successful plug-ins include "dragonegg" and "graphite." GCC's shortcoming led to the development of LLVM [Lattner & Adve 2004], which addresses the lack of flexibility by providing an advanced pass manager that can change the transformation chaining at runtime.

The source-to-source ROSE Compiler [Quinlan 2000, Schordan & Quinlan 2003]d o e s not include any pass manager but provides full access to all analyses, passes and IR through a clean C++ API. A Haskell wrapper provides scripting facilities. Although this approach offers good flexibility (any kind of pass composition is possible), it offers neither clean concept separation nor any abstraction. This leads to complicated interactions with the compiler. The addition of the scripting wrapper does not solve the problem because it is a raw one-to-one binding of the full API. The Poet [START_REF] Yi | Automated Programmable Control and Parameterization of Compiler Optimizations[END_REF]]l a n g u a g e ,d e s i g n e dt o build compilers, suffers from the same drawbacks, as there is no clean separation between compiler internals and pass management: everything is bundled all together.

Compiler directives offer a non-intrusive way to drive the compilation process, much as a pass manager does. They are extensively used in several projects [Kusano & Sato 1999, Donadio et al. 2006[START_REF] Nvidia | NVIDIA, Cray, PGI, CAPS. The OpenACC Specification, version 1.0[END_REF]t og e n e r a t ee ffi c i e n tc od et a r g e t i n g multicores, GPGPU or multimedia instruction set. They can specify a sequential ordering of passes, but they do not provide extra control flow, and may have complex composition semantics.

Transformation recip es, i.e., sp ecialized pass manager implementations, are presented in [START_REF] Hall | [END_REF]. This section emphasizes the need for a common API to manipulate compiler passes. It leads to a Lua-based 3 approach proposed in [START_REF][END_REF]t o optimize the generation of CUDA codes. It provides bindings in this scripting language for a limited number of parametric polyhedral loop transformations. The addition of scripting facilities makes it possible to benefit from complex control flow, but the interface is limited to polyhedral transformations. The approach is procedural and the API allows access to analysis results such as control flow graphs, dependence graphs, etc.

The need for iterative compilation led to the development of an extension of the GCC pass manager [START_REF] Fursin | [END_REF]. It provides a GCC plug-in that can monitor or replace the GCC pass manager. It is used to apply passes to specific functions but the ordering is still sequential. The underlying middleware, ICI, is also used in [START_REF] Fursin & Cohen ; | Building a practical iterative interactive compiler[END_REF] where an interactive compiler interface is presented. The authors enforce the need for clear encapsulation and concept separation, and propose an Extensible Markup Language (XML) based interface to manage the transformation sequencing.

This approach basically turns a compiler into a program transformation system, which is an active research area. FermaT [Ward 1999]f o c u s e so np r o g r a mr e fi n e m e n ta n dc o mposition, but is limited to transformation pipelining. Stratego [Olmos et al. 2005]soft w are transformation framework does not allow much more than basic chaining of transformations using pipelines. CIL [Necula et al. 2002]p r o v i d e so n l yafl a g -b a s e dc o m p o s i t i o n system that is activation or deactivation of passes in a predefined sequence without taking into account any feedback from the processing.

Conclusion

This section introduced PyPS,apassmanagerAPI for the PIPS source-to-source compilation infrastructure. This compilation framework is reusable, flexible and maintainable, which are the three properties required to develop new compilers at low cost while meet-3. Lua is a scripting language.

ing time-to-market constraints. This is due to a clear separation of concepts between the internal representation, the passes, the consistency manager and the pass manager. This clear separation is not implemented in GCC and is not fully exploited either in research compilers, although pass management is becoming a research topic. Five specific compilers and other auxiliary tools are implemented using this API: OpenMP, SSE, SCMP, and CUDA/OpenCL generators, and an optimizing iterative compiler. We show how it is possible to address retargetability by combining different transformation tools in an endto-end parallelizing compiler for heterogeneous architectures. Moreover, it is done in an elegant and efficient way relying on the clear separation of concepts that we identified and on facilities offered by Python ecosystem.

This work was previously published in [Guelton et al. 2011a(perso), Guelton et al. 2011b (perso)].

Library Handling

Developers often make use of external libraries, highly optimized for dedicated computing such as linear algebra, image processing, or signal processing. The issue, in the context of automatic parallelization of such programs, is that most of the time the source code of the library is not available either because it is a proprietary piece of code or simply because there is only the binary present on the computer. Even if the code were available, it may represent a huge amount of code to process. Moreover, it would be probably so tied to a given CPU architecture that any transformation for the GPU is far beyond what may be achieved by an automatic parallelizer.

PIPS is an interprocedural compiler. It analyses the whole call tree and propagates results of analyses to call sites. A current limitation of the implementation is that the whole call tree is required, i.e., the source code of all leave functions must be available for processing. As a major consequence, PIPS cannot process a source code that makes use of external libraries. The source is not always available or optimized for the CPU and too complicated to be properly analyzed and automatically transformed to a GPU version.

On the other hand, many libraries are available for GPUs and provide similar if not equal functionalities. This section covers how the handling of libraries is performed in the compilation chain. First, an automatic way of providing stubs on demand to the compiler is presented in Section 5.4.1. Then different techniques are used to replace the CPU library code with a GPU equivalent implementation in the backend (see Section 5.4.2).

Stubs Broker

As an interprocedural compiler, PIPS stops as soon as it has to analyze a function whose source code is not available. To avoid the problem, the usual process requires providing stubs to the compiler. Stubs are fake functions, with the exact same signature, that mimic the effects of a library call on the memory, since PIPS uses this information for parallelization purpose, for example. Memory effects can be of read or written types, and are handling part or the whole of the call arguments, but they can also involve manipulating library-internal variables generating side effects.

In the context of the Par4All tool, the goal is to provide a seamless experience to the end user. The user writes his own code and provides it to the compiler, with a flag that indicates which library is in use. Par4All includes a generic mechanism to handle libraries and to provide retargetability. We have historically called this mechanism the stubs broker. This piece of code is invoked by many steps of the compilation scheme. Firstly, when PIPS requires a source code for a function, the stubs broker is in charge of retrieving the required code and feeding PIPS with it. Then, when it is time to call the backend compiler, it has the responsibility to retrieve the actual implementation by providing either a link flag to a binary library or a source code to compile.

The process is the following:

1. PIPS needs to analyze a function with a call site to a missing function.

2. An exception is raised with the name of the missing function.

3. The exception is passed to the stubs broker handler.

4. The stubs broker handler goes through the list of available brokers and requires a stub for the missing function.

5. When a broker has a stub for this function, the broker handler registers the stub and the broker that has given it. There are also some meta-data associated with the stub, such as the different implementations available (for instance CUDA or OpenCL).

6. The code for the stub is given to PIPS, and the analyses continue. The stub has been flagged in order not to be modified in any way by PIPS,s i n c ei ti si n t e n d e d to be replaced by another implementation in the backend compiler. For example, inlining of this stub is not desirable. The meta-data are also used to indicate that communications have to be generated in case a given stub corresponds to a GPU implementation.

7. At the end of the process, the resulting code is passed to the Par4All back end. At that time the back end is fed with the parameters given by the broker handler for the retrieved stubs. A new source code can be added, or some compilation flag to link against some libraries could be added.

This flow is represented in Figure 5.10. This scheme has been validated with the implementation of a parallelizing Scilab-to-C compiler (the Wild Cruncher from SILKAN). A specific compiler is in charge of producing C code from Scilab scripts. The resulting code makes use of a large number of runtime functions to interact with Scilab internal (e.g. for the display) or to perform some specific optimized computations (e.g. particular numeric solvers).

In this context, feeding PIPS with thousands of unused functions was not an option as the processing would take hours! Moreover, the source code is not available; the runtime is distributed as a binary shared library.

Instead, stubs are provided; most of them are generated automatically while some are hand-written. The stubs broker interface is simple enough to ease the implementation of any new custom broker. Moreover, Par4All exposes entry points to add dynamically a new stubs broker in addition to the default ones. For the Scilab runtime, a third-party broker has been written to handle the huge number of stubs included. Less than fifty lines of Python were sufficient to provide a complete stubs broker.

Handling Multiple Implementations of an API: Dealing with External Libraries

Ab r o k e ri sn o ti nc h a r g eo fag i v e nl i b r a r yb u tr a t h e ro fafeature.F o ri n s t a n c e ,a broker can be in charge of the FFT, whatever libraries are included. The reason is that while a client code may use one particular CPU FFT library, we ought to replace it with another library on the GPU. This organization is emphasized in Figure 5.10;t h eb r o k e r handler is in charge of orchestrating the whole process. Let us use an example based on the FFTWl i b r a r y [START_REF] Frigo | [END_REF] with the cosmological simulation named Stars-PM [Aubert et al. 2009 (perso)]p r e s e n t e di nd e t a i l in Section 3.1. The FFT is used during the third step exposed in Figure 3.3,p a g e65. The involved calls to FFTWa r ee x h i b i t e di nF i g u r e5.11. These two calls include only one parameter: the plan.I ti sa no p a q u eFFTW structure. This structure has to be initialized with all the information required to perform the operation. It includes the pointers to the 2) An exception is raised for a missing function.

3) A stub for the missing function is requested from the handler.

4) The request is forwarded to all available brokers.

5)

If the function is matched, the broker returns the stub and associated metadatas.

6)

The handler bookkeeps the metadata and gives the stub and the metadatas back to Par4All manager. 7) PIPS is fed with the stub.

8)

The handler is asked for extra source les and parameters for the backend compilation 9) Backend compilation source and destination arrays, the size of the arrays, the direction for the FFT,a n do t h e r parameters. The initialization of the two involved plans is shown in Figure 5.12.

Par4All is launched with a flag -fftw that indicates the use of the FFTWl i b r a r y , but it could also be automatically detected. The FFT broker then automatically feeds PIPS when it complains about the FFTWm i s s i n gf u n c t i o n s ,a se x p l a i n e di nt h ep r e v i o u s section. At the end of the source-to-source processing, the Par4All stubs-broker manager adds sources and/or parameters to the backend process. For instance, it may add some C files that provide a wrapping feature around FFTWt or e t a r g e tt ot h eCUDA equivalent, i.e., CuFFT.I tm a ya l s oa d dafl a gf o rt h el i n k e r . F o rFFTWi tw o u l dbe-lcufft.

Tool Combinations

As heterogeneity grows, it is less likely to have one compiler framework that fulfills all needs. Some frameworks can have an internal representation that better fits some specific optimizations but not all. For example, PIPS benefits from interprocedural analyses, but inherently lacks capabilities that others have, such as PoCC [Pouchet et al. 2010a], Pluto [Bondhugula et al. 2008b], or PPCG [Verdo olaege et al. 2013]. These are some optimizers dedicated to the polyhedral model, but restricted to a particular class of programs.

At some point. depending on the input and the target, it is interesting to combine the right set of tools to perform the best chain of analyses and transformations. Some experiments were conducted in Par4All to make use of PoCC [Pouchet et al. 2010a]a ts o m ep o i n to ft h ep r o c e s s . PIPS takes care of the first phase, in which a detection of static control parts is performed. They are the code sections in which all loops are DO loops whose bounds are functions of parameters, numerical constants and enclosing loops iterators as defined in [START_REF] Feautrier | Dataflow analysis of array and scalar references[END_REF]]. PIPS benefits from interprocedural capabilities that allow it to assert some properties on function calls, like detecting that the called function is pure. Indeed, PoCC assumes that any function call in a code portion flagged as static control is pure.

The code sections detected by PIPS as static control code can be processed by PoCC, applying various optimizations. Currently no optimizations in PoCC are dedicated to the GPU. The extension of Pluto, the polyhedral engine in PoCC, to optimize a loop nest for the GPGPU computing is left as future work.

Another polyhedral optimizer that includes transformations that target GPUs is PPCG [Verdo olaege et al. 2013]. This is a very recent project but it seems very promising and complementary to an interprocedural compiler like PIPS.Othercompilerstak eCUDA kernels as an input like Yang et al. [START_REF][END_REF].

Af u t u r ep i e c eo fw o r kc u r r e n t l yi d e n t i fi e di st op a s sk e r n e l st oak e r n e lo p t i m i z e rt o optimize them for different architectures, and get back different versions of a kernel to be incorporated in the final binary with some mechanisms to select the right one at runtime, using one of the methods presented in Section 5.7.

To achieve all these combinations of different tools, the source level, eventually augmented with semantic information by pragmas, is perfectly suitable as explained in Section 5.2,p a g e140. The C language is a widely used language and it has been extended many times using proprietary extensions, e.g. CUDA,o ru s i n gp r a g m a st oi n c l u d em e t ainformations in the code, e.g. OpenMP [OpenMP Architecture Review Board 2011].

Profitability Criteria

Deciding whether to offload a computation on the GPU or not is not a trivial problem. Two approaches can be compared. The first one makes a decision based on static information. The second postpones the decision until runtime, benefiting from more information, but at the expense of cost and complexity at runtime.

Static Approach

The static approach requires an estimation of the execution time on both the GPU and the CPU. Moreover, the time to transfer the data has to be estimated too. This transfer can be accounted for both computations; it depends on the current data location. If the data are located in the GPU memory, even if the computation is a little bit faster on the CPU,thetimerequiredtotransferthedatama ybeinfa v orofoffloadingthecomputation to the GPU. This situation is mentioned in Section 3.7,p a g e84.Ap o r t i o no fs e q u e n t i a l code is offloaded to the GPU using only one thread to avoid a memory transfer.

Assuming the data location could be known at compile time, the decision still requires evaluating the time required to process the kernel both on CPU and GPU.T ot h i se n d , there is no single solution that would perfectly address this issue; the execution time estimation is a problem.

Even if the loop bounds are known, there is no single formula out there that would fit all the different architectures. Leung et al. proposed a parametrized formula [START_REF] Leung | [END_REF]]t o match a class of GPUs. The model is quite simple and uses off-line a set of "representative" micro-benchmarks to compute the instruction throughput of both the CPU and the GPU. This throughput is then multiplied by the number of instructions in the kernel and the number of threads. The GPU-estimated time also takes into account a constant startup time (a few microseconds) and the memory transfer time. Another assumption is that branches are taken half of the time. They show that this model is accurate enough for simple matrix multiplication but it is doubtful that it can predict as accurately a wide range of kernels. More precise models have been introduced [START_REF] Kothapalli | Ap e r f o r m a n c ep r e d i ction model for the CUDA GPGPU platform[END_REF], Liu et al. 2007[START_REF] Hong | An analytical model for a GPU architecture with memory-level and thread-level parallelism awareness[END_REF], Baghsorkhi et al. 2010], but they are limited in the level of detail taken into account like diverging control flow, memory bank conflicts, and SIMD pipeline delay. Moreover, they are quickly outdated by the evolution of architectures, for instance the introduction of L1/L2 caches in Fermi (see in Section 2.4.5).

Par4All does not include any performance model specific for GPU architectures. It can use a generic complexity analysis [Zhou 1992]f r o mPIPS that can be used to avoid parallelizing loops with small amount of computation.

Runtime Approach

Decisions taken based on runtime information are more accurate. For instance the StarPU runtime system includes many scheduler algorithms, some able to track past executions of a kernel and extrapolate the performance based on a nonlinear regression a ⇥ size b + c. The scheduler can then take decisions about whether to schedule a kernel on an accelerator or not, based on a performance model, the actual locality of the data, but also on the current availability of the resources. Section 6.2 gives more information about this scheme.

Conclusion

The profitability criteria is still an unsolved issue. Several approaches may be used. The next section presents a solution that involves runtime selection between multiple versions, based on off-line profiling.

Version Selection at Runtime

Another approach was explored by Dollinger and Loechner [START_REF] Dollinger | [END_REF]. At compile time they execute multiple versions of a kernel using a large set of possible iteration sets, and record the results to postpone until runtime the scheduling decision based on the actual size of the loop bound. They show that one kernel version can be more adapted for small dataset while another version provides better performance for a large dataset. The drawback is that the target accelerator must be available at compile time for execution.

This approach can be related to and shares the same basic idea as the ATLAS library [Whaley & Dongarra 1998]t h a ta u t o -t u n e sf o rt h et a r g e tp l a t f o r mb yr u n n i n gm u ltiple versions once at installation. Another path is to simulate the execution of a kernel to predict the performance on various architectures. Recently, various simulators for Nvidia G80 [Bakhoda et al. 2009a, Collange et al. 2010], for AMD Evergreen [START_REF] Ubal | Multi2Sim: A Simulation Framework to Evaluate Multicore-Multithreaded Processors[END_REF], or multiple architectures [Bakhoda et al. 2009b, Diamos et al. 2010]ha v ebeenin troduced. Ho w ev er,thesesim ulatorscantak e av e r yl o n gt i m e .

The main limitation of this approach is probably that some kernels need data initialized in a particular way, for instance because of data-dependent branches; then this method requires a representative set of data available at compile time, which is not always possible, or not in all possible sizes. This method is complementary with StarPU middleware [Augonnet et al. 2010b]. The latter accepts multiple kernel versions, with a custom callback function, to decide which one should be executed considering the actual arguments and the available architecture.

Launch Configuration Heuristic

CUDA and OpenCL require expressing the iteration set in the form of a threedimensional grid with two levels of refinement: the main grid is itself split into threedimensional blocks of threads. This is the NDRange in OpenCL terminology introduced in Section 2.3.2.2,p a g e32.

This grid has to be configured manually by the programmers. To get the best performance, they usually have to use a trial-and-error approach. Moreover, finding the best configuration is architecture dependent and data size dependent, and thus no generic formula is available.

Tuning the Work-Group Size

The block (or work-group) size has a potentially large impact on performance. An obvious practice is to have a block size multiple of the SIMD width; for instance it is always thirty-two on currently released Nvidia architectures. The CUDA best practice guide [Nvidia 2012a]m e n t i o n st h a tt h en u m b e ro ft h r e a d ss h o u l db eam u l t i p l eo fs i x t yfour in order for the compiler and the hardware thread scheduler to schedule instructions as optimally as possible to avoid register memory bank conflicts. It also advises defining a configuration that allows using several smaller thread blocks rather than one large thread block per multiprocessor. This is particularly beneficial to kernels that often call __syncthreads().F i n a l l y i t s t a t e s t h a t a g o o d c h o i c e i s u s u a l l y b e t w e e n 1 2 8 a n d 2 5 6 threads per block in general.

Ak e yp o i n ti nt h eCUDA programming guide is presented as maximizing occupancy. Even if Volkov showed that it is not always the metric to maximize to get the best performance [Volkov 2010], it is at least a good starting point for the kernels that can be generated automatically when using Par4All. Moreover, this metric has the merit to be easily maximized, adjusting the launch configuration for a given kernel and a given architecture. Since all these parameters are known at runtime, the launch configuration can be dynamically adjusted. The occupancy is the ratio of the number of threads ready to execute considering the resource used by the kernel and the block size to the maximum number of threads the hardware scheduler can handle. It is intended to ensure that the hardware scheduler has enough threads available to hide the memory latency with other thread operations.

The occupancy is computed with the following parameters: • The resource usage of the kernel: number of registers per thread KernelRegistersPerThread and requested shared memory size per block KernelSharedPerBlock. These are given by the OpenCL and CUDA APIs.

For a candidate blo ck size BlockSize, the occupancy can be computed in the following way. First, considering the shared memory usage, the limit in term of number of blocks per CU is SharedMemBound = SharedSize KernelSharedPerBlock Then considering separately the register usage per thread, the limit of number of blocks per CU is RegistersBound = NRegisterPerCU KernelRegistersPerThread ⇥ BlockSize For the candidate blo ck size, the maximum numb er of blo cks p er CU is

BlockThreadsBound = MaxThreadsPerCU BlockSize
The effective number of blocks schedulable is then the lowest of the previous computed values:

SchedulableBlocks =min(SharedMemBound, RegistersBound, BlockThreadsBound, MaxBlocksPerCU)
Finally, the occupancy is then the expected ratio of the number of threads schedulable per CU over the capacity of the scheduler:

Occupancy = SchedulableBlocks ⇥ BlockSize MaxThreadsPerCU
To ensure that the resource limits are not overwhelmed, both OpenCL and CUDA API provide the maximum block size possible for a given kernel and a given device. The algorithm to find the block size that would maximize the occupancy, implemented in the Par4All runtime, is shown in Figure 5.13. However, maximizing the occupancy is nonsense if the iteration set is too small and leads to too few blocks globally. For instance, using a Tesla C2050 shipped with fourteen CU,a tl e a s tt w e n t y -e i g h ta n dp r e f e r a b l yf o r t y -t w ob l o c k sh a v et ob el a u n c h e dt oe x p e c t good usage of the GPU.

M axSize =min

✓

BlockSizeForOccupancy,

⇠

GlobalWorkSize/NCU WarpSize

⇡◆

This formula also ensures that the block size is rounded to the next multiple of the warp size. While the algorithm BlockSizeForOccupancy presented in Figure 5.13 requires being processed only the first time a kernel is executed on a given device, this last formula depends on the GlobalWorkSize (i.e., the iteration set), and thus has to be checked each time a kernel is launched. This is particularly visible when a triangular loop nest has an external sequential loop sequential. Then for each iteration of the outer loop, a kernel that maps the inner loop is launched. The original code could look like this: for(i=0;i<N-1;i++) // Sequential for(j=i;j<N;j++) // P a r a l l e l // . . .

The kernel corresponding to the inner loop has a constant block size that maximizes the occupancy for a given architecture. However, as i grows, the global work size for this kernel decreases until not enough blocks to fill the GPU are available. Then the limitation for the block size in the runtime decreases the occupancy of each individual CU to favor a better occupancy at the scale of the whole GPU.

Figure 7.2 shows over a range of benchmarks and on various architectures how a carefully chosen block size leads to an acceleration up to 2.4 when compared to a naive constant allocation.

Tuning the Block Dimensions

The CUDA best practice guide [Nvidia 2012a]s t a t e st h a tt h em u l t i d i m e n s i o n a la s pe c t of the block size allows easier mapping of multidimensional problems to CUDA but does not impact performance. However, while it does not impact the hardware scheduler, it changes the memory access pattern and may lead to a very different performance.

The performance obtained for a large range of possible configurations for multiple block sizes is shown in Figures 7. 3, 7.4, 7.5,a n d7.6 page 183 for the code illustrated in Figure 5.14,a n di nF i g u r e s7.7, 7.8, 7.9,a n d7.10 page 187 for the matmul example presented in The measures conducted in Section 7.4,p a g e180,l e a dt oa n o t h e rr u n t i m eh e u r i s t i c . For a given block size, the shape of the block is as much as possible a square, but with an u m b e ro ft h r e a d sa sc l o s ea sp o s s i b l et oam u l t i p l eo ft h i r t y -t w o( o rs i x t e e no no l d e r GPUs)t h r e a d so nt h efi r s td i m e n s i o n ,i . e . ,aw a r ps i z e( o rh a l fw a r ps i z eo no l d e rGPUs). This heuristic is implemented in the Par4All runtime.

for(i=0;i<n;i++) // P a r a l l e l for(j=0;j<n;j++) // P a r a l l e l for(k=0;k<m;k++) // Sequential

C[i][j]+=alpha* A[i][k]*A[j][k]; int i, j, k; i=P 4 A _ v p _ 1 ; j=P 4 A _ v p _ 0 ; if (i<N && j<N) { double C_0 = 0; for(k=0;k<m;k++) { C_0+=alpha* A[i][k]*A[j][k]; } C[i][j] = C_0; }

Conclusion

This chapter addresses the issue of automating the whole compilation process, from the original source code to the final binary, but also the runtime.

Due to the heterogeneity of the current platforms, compiler phases need to be combined in unusual and dynamic ways. Moreover, several tools may need to be combined to handle specific parts of the compilation process.

The analyses and transformations presented Chapters 3 and 4 are chained to form a coherent automatic process. I implemented a fully automated solution, within the Par4All project [Amini et al. 2012b (perso)], that exploits this process and includes a dedicated runtime for CUDA and OpenCL.

This chapter provides several concepts that I tested and validated within Par4All: the advantage of the source-to-source aspect, the role of a programmable pass manager, handling libraries, making different tools to collaborate, profitability-based scheduling, and choosing a launch configuration at runtime.

Chapter 6

Management of Multi-GPUs Most of the work presented in this dissertation deals with a host and only one accelerator. This chapter presents some preliminary experiments carried out with multiple accelerators attached to a unique host.

Introduction

While the CPU manufacturers have faced the frequency wall for a few years now, a lot of research has been conducted on the benefit of adding an accelerator to a classical processor (see Chapter 2). The next step is seen naturally as adding more accelerators per host. While it is cheap from a hardware point of view, it represents a tough challenge for software programmers who are still having trouble programming one accelerator. This is even more challenging when the ambition is to provide an automatic solution, as in this dissertation. Some related work is described in Section 6.4.

Two major approaches are considered in this chapter. The first one relies on task parallelism where each task is a kernel that can be executed either on an accelerator, or on a CPU,o ro nb o t h . I ti sp r e s e n t e di nS e c t i o n6.2. The second approach presented in Section 6.3 consists in splitting the iteration set and the data among different accelerators. Some experiments are presented in Section 7.10,ev enthoughthisc hapterpresen tsonly some early work on this subject, promising more developments in the next decade.

Task Parallelism

Improving the state-of-the-art automatic task extraction is beyond the scope of this dissertation. Assuming that a satisfying scheme in the compiler can extract task parallelism automatically or that the programmer expresses it, how can it be exploited to target multiple accelerators?

This section studies how a runtime system like StarPU [Augonnet et al. 2011]c a nb e suitable as a target in the back end of a compiler targeting multiple accelerators.

The StarPU Runtime

StarPU [START_REF] Augonnet | [END_REF], Augonnet et al. 2010b] is a runtime system that offers task abstraction to programmers. The computation performed by a program has to be split in separated functions, the codelets, with some restrictions. For instance, no global or static variables can be used in task functions. The tasks take as parameters scalars or buffers. These latter have to be registered with StarPU first, and once registered cannot be used outside of a task without acquiring the buffer from StarPU. This model is very similar to the OpenCL runtime specification (see Section 2.3,p a g e30).

For a task to run on several architectures, the programmer has to give one implementation per target architecture, or even more than one. A custom helper function can select the best implementation using runtime information provided by StarPU, such as the exact GPU model selected for execution.

StarPU then takes care of the scheduling and executes those codelets as efficiently as possible over all the resources available at runtime. Memory coherency is enforced over registered buffers. Moreover, the scheduler can decide where to run a computation based on the actual location of the input data.

StarPU provides interesting performance when distributing work among multiple GPUs and CPUs concurrently [Augonnet et al. 2010a].

Task Extraction in PIPS

Ataskextractionphasew asaddedinPIPS to support the experimentations conducted in this chapter. This implementation was naive at that time, far from the interprocedural and hierarchical schemes mentioned in Section 6.4. Moreover, it is based on a simple heuristic based on the presence of loops instead of a connection with the complexity analysis [Zhou 1992]p r o v i d e db yPIPS to make profitability-based decisions. However, in the context of this chapter, it is sufficient for simple benchmarks like the Polybench or on some Scilab script automatically compiled to C.

The principle is to group all computations in identified tasks, i.e., functions that will be called by the StarPU runtime system. This process extensively uses the outlining phase (illustrated in Figure 4.3b page 100), which extracts a sequence of statements from an existing function and replaces them with a call to a new function built with that sequence.

The heuristic detects loop nests in a sequence and checks that they cannot be grouped in a common task. Thus any optimization that would be impossible by this separation in functions like the loop fusion transformation (see Section 4.6 page 112)i se x p e c t e dt ob e applied first. This transformation also ignores loop nests that have already been converted into kernel calls and that can be considered as tasks. The tasks have been declared with attributes in order to drive StarPU, specifying the suitable target platform for each implementation. Task parameters are declared with a const qualifier when used as read-only, and with an __attribute__((output)) when used as write-only. Hints can be given to StarPU about buffers. A buffer used only for reading by a task can be flagged as constant and StarPU will not generate a copy-out for it or invalidate other copies. Moreover, another task using the same buffer can be scheduled in parallel. In the same way, buffers can be flagged as write-only so that no copy-in is generated. The array region analysis introduced in Section 3.2 page 64 provides this information at compile time. Figure 6.2 illustrates the result of this transformation on the 3mm example. Tasks have been generated for each matrix multiplication, pragmas have been generated so that StarPU can track the buffers, and task parameters have been declared with a const qualifier or an __attribute__((output)).

Data Parallelism Using Loop Nest Tiling

Section 3.9 page 90 explored the minimization of communications when mapping a sequential tiled loop nest on an accelerator. This section introduces the mapping of a loop nest composed of fully parallel tiles onto multiple GPUs.

Loop tiling is a well known transformation [Wolfe 1989[START_REF] Irigoin | François Irigoin and Rémi Triolet. Supernode partitioning[END_REF], Dongarra &S c h r e i b e r1 9 9 0 , Wolf & Lam 1991b, Xue 2000[START_REF] Andonov | [END_REF]], but it is still an active field [START_REF] Hartono | [END_REF][START_REF] Kim | i n g so ft h e2 2 n di n t e r n a t i o n a lc o n f e r e n c eo nL a n g u a g e s and Compilers for Parallel Computing[END_REF][START_REF]Automatic creation of tile size selection models[END_REF][START_REF] Bao | Defensive loop tiling for multi-core processor[END_REF], Zhou et al. 2012, Lin et al. 2011, Renganarayanan et al. 2012]. Guelton introduced in PIPS a Symbolic Tiling scheme to adjust the memory footprint [Guelton 2011a] of kernels. The tile size can be instantiated at runtime according to the accelerator used. His goal was to generate code for an embedded accelerator with only a few kilobytes of memory. I have implemented another slightly different symbolic tiling scheme in PIPS in order to instantiate at runtime the number of tiles instead of the tile size. To feed multiple GPUs,t h el oo pn e s ti ss p l i ti n t ot i l e st h a ta r ed i s t r i b u t e do v e rt h ea c c e l e r a t o r s . Figure 6.4b illustrates this transformation, and shows the array regions computed by PIPS on this example. The final code with the DMA instructions is shown in Figure 6.4c. The calls to functions prefixed with P4A_ are asynchronous calls.

At execution time, a number of OpenCL queues are created and associated with some devices and the loop nest is dynamically split into as many tiles as queues, thanks to the symbolic tiling scheme. A dedicated runtime system was developed to support this scheme. It registers the data to transfer for each tile, and asynchronously schedules the communications and the kernel launches. It automatically provides the correct OpenCL events to express the dependences between commands in the different queues.

As a side effect, and because the OpenCL-based runtime is able to associate multiple command queues with each device, this scheme can be used to overlap communication and computation on one device. When the computation and communication times are perfectly balanced, this could lead to a speedup up of two. However, in practice this is seldom the case. Moreover, the bandwidth between the GPU memory and the PEs suffers from contention due to the concurrent transfers between the host and the GPU.

Performance

Section 7.10,p a g e205,p r o v i d e se x p e r i m e n t a lr e s u l t sf o rt h et w os c h e m e sp r e s e n t e di n the previous sections. The task parallelism scheme shows that it can be relevant for some kinds of workloads. For instance the 3mm example from Polybench shown in Figure 6.1a is sped up by thirty percent when using two Nvidia C1060 instead of one. However, this approach is limited by the number of tasks (or kernels) that can be run in parallel. The 3mm example has only two kernels that can be run in parallel, followed by a synchronization point before the last kernel. Adding more than two GPUs is useless in this case.

Another limitation is the StarPU runtime scheduling cost. The default greedy scheduler provides a rather low overhead, However, since it does not take data locality into account, it ends up with a mapping resulting in too many data transfers to expect any improvements. The problem can be observed when the 3mm example is executed with three GPUs. The last kernel is then mapped on the third GPU resulting in two additive data transfers, one from each of the other two GPUs.I ti se v e nc l e a r e ro nt h ebicg Polybench test case. A large number of dependent kernels are scheduled on different GPUs resulting in a flip-flop situation with respect to the memory location of the data. The performance impact on this last example is dramatic with a slowdown of 8000 when compared to the same code executed with only one GPU worker. Fortunately, StarPU provides other schedulers including some that take into account the task affinity and manage to provide better mapping. The measurement Section 7.10 page 205 confirms that using a data-aware scheduler avoids this issue.

The results show that this approach may provide good performance, even if the Nvidia OpenCL implementation limits currently to 1.63 the maximum speedup I managed to obtain, either using two GPUs or overlapping communications and computations on one GPU. Some future work would replace our custom runtime by a more flexible library like StarPU, with a more dynamic adaptation to the workload, and capabilities to distribute tasks on both GPUs and multicore CPUs.

Related Work

Al o to fp r e v i o u sw o r kh a sb e e nd o n et oe x t r a c tt a s k sf r o ms e q u e n t i a lc o d e ,s t a r ting with the work of Sarkar and Henessy [Sarkar & Hennessy 1986]a b o u tc o m p i l e -t i m e partitioning and scheduling techniques and Cytron et al. [START_REF] Cytron | [END_REF]]o nt h ep a rallelism extraction from a DAG. From a sequential program with the data and control dependences, their implementation in the PTRAN system [START_REF][END_REF]]p r o d u c e sa n equivalent fork-join parallel program. A hierarchical task graph is used to extract coarse grained task level parallelism by Girkar et al. [START_REF] Girkar | [END_REF], Girkar & Polychronopoulos 1995]. Hall et al. use interprocedural analyses in the SUIF compiler to extract coarse grained parallelism [Hall et al. 1995]. Sarkar [Sarkar 1991 [START_REF] Ceng | [END_REF]. They derive a weighted control data flow graph from the sequential code and iterate, using a heuristic and user inputs, to cluster and generate tasks. Cordes et al. [Cordes et al. 2010] makes use of Integer Linear Programming to steer the task granularity for a given hardware platform, whose characteristics have to be known at compile time. Stuart et al. [Stuart et al. 2010]d e v e l o p e dal i b r a r yt ot a r g e tm u l t i p l eGPUs using the MapReduce programming model [START_REF] Dean | [END_REF]. Other work has been conducted for ordering tasks at runtime [Augonnet et al. 2010a] or with preemption and priority handling for critical application [START_REF] Membarth | [END_REF].

When sharing work between CPUs and GPUs,H e r m a n n [START_REF] Hermann | Multi-GPU and multi-CPU parallelization for interactive physics simulations[END_REF]f o u n d that placing small tasks on the CPUs makes mapping GPUs more efficient and leads to a speedup greater than the sum of the speedups separately obtained on the GPUs and the CPUs. Following this idea, complexity analysis like the one existing in PIPS [Zhou 1992] might be used to generate hints about the relative size of the different tasks and to lead to better decision by the runtime system. Chen et al. [START_REF][END_REF]]m a n u a l l ye x p l o r eb o t ht h et a s ka n dd a t ap a r a l l e l i s m for unstructured grid applications, using multiple GPUs and overlapping computation and communication.

Enmyren et al. [START_REF] Enmyren | SkePU: a multibackend skeleton programming library for multi-GPU systems[END_REF], Dastgeer et al. 2011] propose SkePU, a C++ template library that provides a unified interface for specifying data parallel computations with the help of skeletons. They make use of lazy memory copying to avoid useless transfer between the host and the accelerator.

The automatic extraction of data parallelism for distributed memory systems has been studied too. For instance, Ancourt et al. proposed a linear algebra framework for static HPF code distribution [START_REF] Ancourt | [END_REF], Ancourt et al. 1997]. STEP [START_REF] Millot | [END_REF]] is a framework to convert OpenMP code to MPI using PIPS.S u c hw o r k ss h a r ec o m m o n problems with the multiple-GPU target, like the distributed memory space and data coherency issues.

Not much work has been done extracting data parallelism automatically for multiple GPUs. Many experiments were carried out manually by scientists in different fields (see for example [Aubert 2010]). [START_REF] Leung | [END_REF]]g a v es o m ei d e a sa b o u tu s i n g the polyhedral model to map a loop nest on a GPU,b u td e s p i t et h ea t t r a c t i v et i t l et h e specific issues of multi-GPUs are not addressed, e.g. the memory transfers between the host and the accelerators.

Conclusion

The free lunch is over :p a r a l l e lp r o g r a m m i n gi sn o wm a n d a t o r y .H e t e r o g e n e i t yi sn o t the future but the present. The last years have seen a lot of research about software development for accelerators. The compilers and tools field has been mostly limited to one GPU and one host, but the next step seems clearly to be inching toward more heterogeneity using multiple accelerators.

This chapter presents two different ways of extracting parallelism in order to map kernels onto multiple accelerators. The first one relies on task parallelism where each task is a kernel that can execute on an accelerator, on a CPU,oronboth. Iimplemen tedav ery simple automatic task parallelism extraction in PIPS,e v e ni ft h ea p p r o a c hi si n d e pe n d e n t of the extraction. I also implemented a code generation phase for the StarPU runtime system, and validated the full chain with experiences. The second approach consists in splitting the iteration set and the data among different accelerators. I modified the existing parallel symbolic tiling transformation in PIPS to provide the expected resulting code. I implemented and validated with experiences a dedicated runtime to map the tiles' execution over multiple GPUs.

Some limitations in the OpenCL implementation provided by Nvidia still limit the performance that can be obtained, but future releases and the forthcoming Kepler-based Tesla with improved GPU-GPU communications should improve the situation.

The experimental results obtained with these two basic schemes are encouraging, and many more deeper researches are expected in this challenging field for the next decades.

The next chapter presents the experiments conducted to validate all the proposal developed in the previous chapters. This chapter presents the experiments that validate the different program transformations introduced in the previous chapters. Various benchmarks (see Section 7.2)andseveral hardware platforms (see Section 7.1)h a v ebe e nu s e d .

Hardware Platforms Used

To explore the validity of transformations or other schemes among different architecture generations, many hardware platforms have been used involved in my tests, using Nvidia and AMD GPUs.

The different platforms used are listed in Table 7.1.

• The 8800GTX is the first CUDA-enabled GPU.I tw a si n t r o d u c e di n2 0 0 7 . I tp r ovides low performance when compared to more recent hardware, However, its architecture with very naive memory controllers exhibits interesting patterns. The board is plugged into a host with a 2-core Xeon 5130 running at 2 GHz.

• The C1060 is the first Tesla, i.e., the first GPU from Nvidia to be especially floatingpoint computation oriented. Two Intel X5570 CPUs (4-core each) are running the host at 2.93 GHz.

• The C2070 can have ECC enabled or not, impacting the performance and the total available memory. It is still today the most high-end compute-oriented solution from Nvidia while awaiting the Kepler-based Tesla at the end of 2012. The board is plugged into a host with two six-core Intel Xeon X5670 processors running at 2.93 GHz.

• The AMD 6970 is now outdated by the more recent 7970 based on the GCN architecture (see Section 2.4.4,p a g e43). The host runs a 6-core AMD FX-6100 at 3.3 GHz. It is the same generation as the Fermi-based Nvidia boards.

• Finally, the GTX 680 is based on the new Nvidia Kepler architecture, but is not compute oriented and lacks many of the Kepler novelties, like the Dynamic Parallelism feature. Also, it does not include the new GDDR5m e m o r yi n t e r f a c ea n dp r o v i d e s a theoretical bandwidth lower than the 320 GB/s announced for the yet unreleased Kepler-based Tesla. This chip should provide 2888 cores with 1/3 float to double precision ratio, while the GTX 680 that we use ships only 1536 cores and only 1/24 float to double precision ratio. The CPU is an 4-core Intel Core i7 3820 running at 3.6 GHz.

GPU

Experiments

Benchmarks involved in the experiments conducted in this chapter are taken from the Polybench 2.0 suite and from the Rodinia suite. Some kernels were extracted to exhibit interesting behaviors and validate some transformations. Am o r er e p r e s e n t a t i v ereal-world example introduced in Section 3.1 page 63 is also used to evaluate the whole tool chain with optimized data mapping (see Chapter 3). This numerical simulation, called Stars-PM, is a particle mesh cosmological N -body code.

Some Scilab examples, first compiled into an intermediate C version, are also used.

All these codes were used as is, except those from Rodinia. Those were rewritten partially to conform to some coding rules [Amini et al. 2012c (perso)]. The Rodina rules specify mostly that array accesses should not be linearized and C99 should be used as much as possible to type precisely the arrays. Therefore it is only a syntactic rewrite: C99 arrays are used and references adjusted without algorithmic modifications. This rewriting is due to the parallelization schemes implemented in PIPS and not to my communication optimization scheme (see Section 3.6,p a g e77).

For this reason, and b ecause some tests are intrinsically not adapted to GPU computing, only the suitable part of the benchmark suites are presented here. Some Polybench test cases use default sizes that are too small to amortize even the initial data transfer to the accelerator. Following Jablin et al. [START_REF] Jablin | [END_REF], the input sizes were adapted to match the capabilities of the GPUs.

Parallelization Algorithm

As explained in Section 4.3.3 page 104,t h ec h o i c eo ft h ep a r a l l e l i z a t i o na l g o r i t h mh a s an impact on the parallelism expressed and on the resulting number of kernels. Because the Allen and Kennedy algorithm introduced in Section 4.3.1 distributes the loops and leads to more kernels, it expresses finer parallelism than the coarse grained algorithm presented in Section 4.3.2 page 103.

Figure 7.1 shows how the Allen and Kennedy scheme leads to a more efficient code on all architectures. While the difference is very limited on old architectures such as the G80, it is dramatic on recent GPUs, which benefit from the larger amount of parallelism exposed. The execution time is up to eight times faster on Fermi and four times on Kepler in favor of Allen and Kennedy's parallelization.

Launch Configuration

OpenCL and CUDA split the iteration set into a grid of work-items (or threads) and group them into work-groups (or blocks), as explained in Section 2.3.2.2 page 32.I ti su p to the programmer to chose the size and the shape of the work-groups.

Figure 7.2 shows the impact of the work-group size on a large range of benchmarks. All work-items are allocated to the first dimension of the work-group. The performance impact of the work-group size is important, with an acceleration factor up to 2.4 for a given test case. More interestingly there is no general optimal size; that depends on the kernel and also on the architecture. The evolution of the performance as a function of the work-group size varies across tests, but also across different architectures for the very same test. This means that the block sizes have to be adjusted at runtime, kernel per kernel, depending on the global work-grid dimensions, and on the GPU architecture.

For a given work-group size, another parameter is the balance b etween the two dimension, which we call dimensionality. The impact of the different dimensionalities for various work-group sizes is illustrated for several architectures in Figures 7. 3, 7.4, 7.5,a n d7.6 for the syrk example Figure 5.14 page 163,andinFigures7.7, 7.8, 7.9,and7.10 for the matmul example of Figure 4.22 page 129.

The balance of work-items between the first two dimensions in a work-group largely impacts the performance. A common pattern is visible, with better performance when exploiting the two dimensions as much as possible. While the improvement over mapping all work-items on the first dimension is quite low on matmul,o nsyrk it can lead to a speedup of more than fifteen for some work-group sizes. Choosing the best work-group size on the latest GTX 680 Nvidia GPU achieves a four times speedup by adjusting the dimensionality.

Scalarization

The transformation presented in Section 4.7,p a g e127,e x h i b i t st h r e ei n t e r e s t i n gp a tterns that are measured here:

• scalarization of array accesses in a sequential loop inside the kernel (see Section 4.7.1, page 128);

• full array contraction, which eliminates temporary arrays, usually after loop fusion (see Section 4.7.2,p a g e128);

• scalarization can break the perfect nesting of loops (see Section 4.7.3 page 130).

Scalarization inside Kernel

The first transformation to measure is presented in Figure 4.22 page 129 with a simple matrix multiplication example. The interest is to keep in a register the reference to the array and therefore reduce memory accesses. This transformation could be done in the target backend compiler (nvcc for Nvidia): it is exposed at source level here. The speedup is between 1.12 and 1.59 in double precision floating point, but more impressive in single precision with a speedup ranging from 1.37 up to 2.39, and no slowdown is observed (see Figure 7.11).

Full Array Contraction

The second transformation allows a total array contraction, usually after loop fusion as shown with the Scilab script of Section 4.7 (see Figure 4.21,p a g e129). The speedups obtained vary from 1.96 up to 5.75 (see Figure 7.12). Moreover, three arrays no longer need to be allocated on the accelerator, reducing the kernel memory footprint, and the compiler does not generate any transfers, which would each take as long as the kernel execution time. Unlike the previous pattern, this one cannot be performed by the target binary compiler, and therefore it is necessary to detect this opportunity and to perform the transformation at source level.

Perfect Nesting of Loops

My last transformation explores the three possibilities exposed by the code in Figure 4.23,p a g e131,a n dd i s c u s s e di nS e c t i o n4.7.3,p a g e130. The scalarization breaks the perfect nesting of the loops, and thus there is only one iteration space mapped onto the GPU. Instead of one kernel launch, there are now multiple launches with fewer threads each. Figure 7.13 shows the performance comparison of the original version and the version after scalarization with the inner loop mapping. The value for M has been fixed at 8192 and the values for N follow the x axis. The third version is not considered because it is by far slower than the others. As expected, the execution time scales nearly linearly and quickly with N for the scalarized version with inner mapping, while the time for the non-scalarized version remains low. Small jumps are observed on the latter because of the SIMD width of thirty-two on this architecture, and because of the various overheads not overlapped with computation because only N threads are available (see Sections 4.7.3). The inner mapping scheme is interesting only for very small values of N. An off-line profiling of the two versions would provide the test to select the best versions according the M and N values at runtime (see Section 5.7,p a g e158).

7.6. Loop Unrolling 195

Loop Unrolling

Some kernels include a sequential loop inside the kernel and an unrolling loop at source level as shown in Section 4.8, which can have some impact on the performance. Figure 7.14 explores the performance impact of the unrolling of the inner loop in the matrix multiplication kernel presented in Figure 4.22 page 129. Many unroll factors are tested, showing different results depending on the target architecture.

It is possible to get better performance after unrolling on all architectures. However, the factors differ and so do the speedups. While the Fermi architecture benefits from a 1.35 speedup using an unroll factor of eight, the Kepler architecture exhibits only a 1.025 acceleration using an unroll factor of two. The G80 does not benefit from loop unrolling as much as the last architectures, with a 1.04 speedup using an unroll factor of thirty-two.

Finally, the impact of unrolling on the register pressure is shown Figure 7.15 for Nvidia GPUs. The code generated for Fermi shows that the register count is increased by a factor up to ten and then stays relatively stable. On the other hand, older architectures have a generated code using a number of registers that scales linearly with the unroll factor. The Nvidia compiler targets a pseudo-assembly language: PTX. The PTX is using an arbitrary number of registers, and the output of the compiler is Simple Static Assignment (SSA) form. The Nvidia binary compiler, ptxas, reports the final number of register when compiling the PTX to ISA. The ISA is not released by Nvidia, and therefore the origin of register usage is hard to track in the PTX.

Excessive register pressure implies more spilling. Since the number of registers is shared among threads in flight on a CU,increasingtherequiremen tmak estheoccupancydramatically decrease; i.e., the hardware scheduler does not have enough thread in flight to hide memory latency. This issue is also explored in Section 5.8.1,p a g e159, with the selection of the launch configuration.

Array Linearization

While the array linearization presented in Section 4.9 is a transformation mandatory to produce a valid OpenCL kernel, the performance impact is not known. Since CUDA makes it possible to apply it or not, it is possible to generate and compare both versions.

The benchmark used the simple matrix multiplication example presented in Figure 4.22. The result presented in Figure 7.16 shows that the impact varies with the target architecture. It can lead to a slowdown of up to twenty percent, but also in one configuration-the GTX 680 with single precision-can lead to a small speedup of about two percent.

Communication Optimization

This section details the experiments carried out to validate the communication optimizing scheme presented in Chapter 3.

Metric

The first question is: what should we measure? While speedups in terms of CPU and wall clock time are most important to users, if not to administrators, many parameters impact the results obtained. Let us consider for example the popular hotspot benchmark [START_REF] Huang | [END_REF]. Its execution time depends on two parameters: the matrix size and the number of time steps. In the general context of GPU,t h em a t r i xs i z es h o u l db e large enough to fully load the GPU.F orm ycomm unicationoptimizationsc heme,thetime step parameter is at least as important since data transfers are hoisted out of the time step loop. Figure 7.17 shows how hotspot is affected by the number of time step iterations and approaches an asymptote; acceleration ranges from 1.4 to 14. The single speedup metric is not adapted to properly evaluate my scheme.

Amoreo bj e c tiv eme as ure me n tf ore v aluatingm yapproac hist hen um be rofc omm unications removed and the comparison with a scheme written by an expert programmer. Not only being inappropriate, focusing on the speedup would also emphasize the parallelizer capabilities.

The number of memory transfers generated for each version of the code is counted. When the communication occurs in a loop, this metric is parametrized by the surrounding loop iteration space. For instance, many benchmarks are parametrized with a number of time steps t,t h u si ft h r e em e m o r yt r a n s f e r sa r ep r e s e n ti nt h et i m es t e pl oo p ,a n dt w oa r e outside of the loop, the number of communications will be expressed as (3 ⇥ t)+2.

Sometimes a loop that iterates over a matrix dimension cannot be parallelized, either intrinsically or because of the limited capability of the compiler. Memory transfers in such loop have a huge performance impact. In this case n is used to emphasize the difference with the time step iteration set.

Results

This optimization is evaluated using the metric introduced in the previous section. It is more relevant to evaluate my optimization with the number of communications removed. The result in Table 7.2 shows that, when counting the number of memory transfers, the optimized code performed is very close to a hand-written mapping. One noticeable exception is gramschmidt.A se x p l a i n e di nS e c t i o n3.7.1,p a g e86,c o m m u n i c a t i o n sc a n n o tb em o v e d out of any loop due to data dependencies introduced by some sequential code. The parallel promotion scheme shown in Section 3.7,p a g e84,a n de v a l u a t e di nS e c t i o n7.8.4 helps by accepting a more slowly generated code and allowing data to stay on the accelerator.

Comparison with Respect to a Fully Dynamic Approach

This section compares the static scheme to a runtime-based system like the StarPU [Augonnet et al. 2011] Table 7.2: Numb er of memory transfers after parallelization using Par4All naive allo cation, using my automatic optimizing scheme, and as a developer would have put it.

A test case included with the Par4All distribution is used and rewritten using StarPU in order to evaluate the overhead of the dynamic management of communication compared to my static scheme. This example performs 400 iterations of a simple Jacobi scheme on a 512 ⇥ 512 pixel picture loaded from a file and stores the result in another file. Figure 7.18 presents the core computation of this program and the processing using Par4All, while Figure 7.19 shows the encapsulation of the previously shown kernel in a StarPU task, and the use of StarPU to manage the task.

StarPU's traces show that all spurious communications are removed, just as my static scheme does. The manual rewrite using StarPU and a GPU with CUDA offers a 162 ms execution time for the StarPU version while the statically optimized scheme performed in 32 ms. The speedup is about five.

Although StarPU is a library that has capabilities ranging far beyond the issue of optimizing communications, the measured overhead confirmed that my static approach can be relevant.

; if (i<SIZE -2&&j<SIZE -2) dest[i+1][j+1] = 0.25*(src[i+1-1][j+1]+src[i+1+1][j+1]+ src[i+1][j+1-1]+src[i+1][j+1+1]); } (a)

Sequential Promotion

The transformation introduced in Section 3.7 consists of mapping some intrinsically sequential code onto one GPU thread. This transformation is performed to avoid some costly transfer over the PCIe bus. Two examples from the Polybench suite provide interesting results when using this scheme, gramschmidt shown in Figure 3.17 page 85 and durbin shown in Figure 7.20. The communication count goes from 2n +1 down to 3 for gramschmidt and from 3n +4 down to 2 for durbin.

In terms of execution times, Figure 7.21 presents the times measured on two architec- gettimeofday(&time_begin , NULL); for(t = 0; t < 400; t++) { starpu_task_submit(task); } starpu_task_wait_for_all(); // Require a copy-out from the runtime starpu_data_acquire(space_handle , STARPU_R); gettimeofday(&time_end , NULL); tures, a Tesla C1060 and a Fermi C2070. The speedups are very high for the durbin example, respectively 37.2 and 35.6, and less impressive but still very interesting for gramschmidt with respectively 8.3 and 6.5. Kernels are exactly the same for the two automatically generated CUDA versions using Par4All, the naive and the one with communication optimization. The Nvidia CUDA SDK 4.0, and GCC 4.4.5 with -O3 were used.

Overall Results

for (k = 1; k < n; k++) { beta[k] = beta[k-1] - alpha[k-1]*alpha[k-1]*beta[k-1]; sum[0][k] = r[k]; for (i = 0; i <= k-1; i++) sum[i + 1][k] = sum[i][k] + r[k-i-1] * y[i][k-1]; alpha[k] = -sum[k][k] * beta[k]; for (i = 0; i <= k-1; i++) y[i][k] = y[i][k-1] + alpha[k] * y[k-i-1][k-1]; y[k][k] = alpha[k]; } for (i = 0; i < n; i++) out[i] = y[i][N-1]; copy_to_accel(r); kernel_0(r, sum, n); copy_from_accel(sum); for(k = 1; k <= n-1; k += 1) { beta[k] = beta[k-1]-alpha[k-1]* alpha[k-1]*beta[k-1]; for(i = 0; i <= k-1; i += 1) sum[i+1][k] = sum[i][k]+ r[k-i-1]*y[i][k-1]; alpha[k] = -sum[k][k]*beta[k]; copy_to_accel(alpha); kernel_1(alpha , y, k); copy_from_accel(y); y[k][k] = alpha[k]; copy_to_accel(y); } kernel_2(out, y, n); copy_from_accel(out);
For Polyb ench, Par4All was forced to ignore array initializations b ecause they are b oth easily parallelized and occur before any of the timed algorithms. The normal optimizer configuration would thus have "optimized away" the initial data transfer to the GPU within the timed code. It seems more relevant to prevent this optimization even though it is not advantaging Par4All results.

The measurements in Figure 7.22 include all communications.

For the cosmological simulation (see Section 3.1), the communication optimization speeds up execution by a factor of 14 compared to the version without my optimization, and 52 compared to the sequential code.

Results are provided for HMPP and PGI Accelerator. Only basic directives were added by hand. We did not use more advanced options of the directives, thus the compiler does not have any hints on how to optimize communications.

The geometric mean over all test cases shows that this optimization improves by a factor of 4 to 5 over Par4All, PGI Accelerator and HMPP naive versions.

One noticeable exception is gramschmidt. Communications cannot be moved out of any loop due to data dependencies introduced by some sequential code. The results for the parallel promotion scheme (see Section 7.8.4)s h o wt h a ta c c e p t i n gas l o w e rg e n e r a t e d code but allowing data to stay on the accelerator leads to performance improvement on such code. My measure provides a 6.5 times speedup over the optimized scheme on this architecture, which means a speedup of 17.5 over the sequential code, closer to HMPP results.

The geometric mean of the speedup obtained with the communication optimization scheme is over 14, while a naive parallelization using CUDA achieves a speedup of 2.22, and on the CPU only the automatic OpenMP loop parallelization one of 3.9.

Multiple GPUs

Task Parallelism

This section measures the benefit obtained by attaching multiple GPUs to one host, using current technologies and a runtime like StarPU introduced in Section 6.2.1.T w o different examples were selected from the Polybench benchmark suite. The first one is 3mm (see Figure 6.1 page 168)a n dt h es e c o n do n ei sbicg (see Figure 7.23).

The 3mm example uses three kernels. The first two can run independently while the third one consumes the results of the two previous ones. The expected execution timeline is shown in Figure 6.3,p a g e170. The bicg example does not exhibit any task parallelism but tests the scheduling capacity and overhead by launching more than 8000 dependent kernels. A greedy scheduler that allocates a new task to a new GPU for each launch would result in a very inefficient mapping. Indeed a large number of memory transfers would occur while ensuring synchronization since every kernel is dependent on the result of the previous one.

The 3mm example contains large kernels whose execution times dominate the communication times. On the contrary, the bicg example contains a large number of very small kernels whose execution times are lower than the communication times individually, but a good mapping avoids most of the communications. Much can be observed from these measurements. Firstly, using one GPU,t h eg r e e d y scheduler provides better results than the static scheduling. The static scheduling is synchronous, while StarPU uses asynchronous scheduling and thus some communication and computation overlapping occurs. Also, the time to take scheduling decisions on the host is hidden by the computation time on the GPU.

Using two GPUs,t h e3mm example shows a speedup up to 1.47, which is consistent with the expected speedup from the timeline in Figure 6.3. The bicg example with two GPUs shows the impact of the scheduler selection. A data-aware scheduler, while not perfect, ends up about only two times slower than the same code using one GPU.O nt h e other hand, the default greedy scheduler ends up with totally inefficient mapping and an execution time up to 8127 slower than the time measured using only one GPU.

Finally, the three GPU configuration confirms the slowdown exhibited by the default StarPU scheduler on the bicg example with two GPUs. The 3mm example, thanks to its favorable task graph, does not benefit from the scheduler, but at least it confirms that the overhead due to the more complex scheduler is hidden by the computations that run asynchronously.

Data Parallelism

Section 6.3 illustrates how parallel tiles in a loop nest can be generated with symbolic tiling, delaying the choice of the tile size and the mapping on the hardware at runtime. This section evaluates this scheme using a variant of the vector scaling example shown in Figure 7.25. The kernel is modified to include a loop that artificially increases the computation time. This is necessary in order to experiment with the current mapping of Nvidia OpenCL runtime on high-end hardware.

Figure 7.26 shows the mapping at runtime using one GPU and from one to ten queues associated with it. Figure 7.27 involves two GPUs and shows the mapping for one to five queues associated with each of them. The main observation is that the Nvidia OpenCL for(i=0;i<ny;i++) s[i] = 0; for(i=0;i<nx;i++){ q[i] = 0; for(j=0;j<ny;j++){ s

[j] = s[j] + r[i] * A[i][j]; q[i] = q[i] + A[i][j] * p[j]; } } (a) Input code.
for(i=0;i<ny;i++)// P a r a l l e l s[i] = 0; for(i=0;i<nx;i++){// P a r a l l e l q[i] = 0; for(j=0;j<ny;j++)// Sequential q runtime implementation is far from being perfect, especially when overloaded with many parallel queues.

[i] = q[i] + A[i][j] *
However, using only one GPU,t h eo v e r l a p p i n go fc o m p u t a t i o n sa n dc o m m u n i c a t i o n s shows interesting speedups of up to 1.63. Using two GPUs does not provide any further improvements. We suppose that the main limitation is in the OpenCL runtime implementation. It serializes of the transfers to and from the two GPUs. The timelines presented in Figures 7.26 and 7.27 are promising, as the kernels are mapped on the different GPUs available and the communications are overlapped. One can expect future implementations of the runtime and future pieces of hardware to be able to provide higher speedups for this mapping scheme.

Conclusions

Many experiments are presented in this chapter to validate the transformations and the different schemes presented in Chapters 3,4,5,a n d6 in this dissertation. Multiple architectures are tested to ensure that the approach is portable enough to provide a good retargetability.

The results show that the approach is rather robust. Measures on a n-body numerical simulation code show speedups of twelve compared to a naive parallelization and eight compared to the automatically generated OpenMP version on two six-core processors.

Some early work about multi-GPUs mapping is presented, and the measurements show that while the Nvidia OpenCL implementation is not perfect yet, opportunities for speedup already exist. Future work in this area is promising. The mapping is done using two C2070s. The copy-out do not overlap properly with copy-in, which is unexpected and limits the acceleration that can be achieved.

Chapter 8

Conclusion

Welcome to the hardware jungle. Herb Sutter's statement [START_REF] Sutter | Welcome to the Jungle[END_REF]]i sm o r et h a ne v e rab u r n i n gi s s u e . F r o mt h ec e l l phones in every pocket to the supercomputers, the hardware is more and more heterogeneous and challenging to program.

No satisfying solution has been proposed for the three P properties: Performance, Portability, and Programmability. At least no solution is close to having unanimous support from programmers.

The jungle is now also visible in the software world, with more and more programming models, new languages, different APIs,etc. Althoughnoone-fits-all solution has emerged, Iha v edesignedanddev elopedacompiler-basedautomaticsolutiontopartiallyansw erthe problem. The programmability and portability are enforced by definition, the performance may not be as good as what can be obtained by an expert programmer, but is still excellent for a wide range of kernels and applications. By using a GPU instead of the CPU,Iobtained aspeedincreasebyafactoroffifty-twoforann-body numerical simulation when compared to a naive sequential code compiled with GCC.

This work explores the many issues associated with a fully automatic solution, addressing the three P issues by changing neither the programming model nor the language, but by implementing compiler transformations and runtime support backed by semantics analyses in a compiler. The portability and programmability are obviously matched; the programmer does not have to know well the underlying platform.

The trade on performance is limited, as shown by many experiments presented in Chapter 7. They also show that portability of performance can be achieved over multiple architectures.

Contributions

Id e s i g n e da n di m p l e m e n t e ds e v e r a ln e wt r a n s f o r m a t i o n sa n da n a l y s e si nP a r 4 A l la n d in the PIPS source-to-source framework compiler. I also modified state-of-the-art trans-formations to obtained a GPU-specific scheme, as shown in Figure 8.1.

If o l l o wt h ec h r o n o l o g i c a lp r oc e s so ft h efi g u r er a t h e rt h a nt hec h a p t e ro r d e rt op r e s e n t my contributions.

Idesignedandimplemen tednewinductionv ariablesubstitutiontransformationsbased on linear precondition analysis (see Section 4.5,p a g e111). This transformation enables the parallelization of loops in spite of induction variables.

I studied the combinations of two different parallelizing algorithms, with an analysis of the impact on code generation of both of them in Section 4.3,p a g e101.

Ii m p r o v e dt h ee x i s t i n gr e d u c t i o nd e t e c t i o na n a l y s i st oh a n d l eCc od em o r ea c c u r a t e l y , and leveraged this analysis to enable parallelization of loops with reduction by improving the existing parallelization algorithms. I implemented a mapping scheme for some loops with reductions onto the GPU using atomic operations supported by OpenCL and CUDA (see Section 4.4 Loop unrolling was presented in Section 4.8,p a g e132, and its impact was shown and analyzed with several experiments. Ar u n t i m es y s t e mt om a pl o o p so nt h eGPU in a flexible manner, including transformations like tiling or loop interchange, is proposed in Section 4.2,p a g e98.

The issue of generating code for mapping data communications between the CPU and the GPU is studied in Chapter 3. Convex array regions are used to generate accurate communication. I designed and implemented an analysis, Kernel Data Mapping,at r a n sformation, and a runtime system to perform interprocedural optimization to preserve the data on the GPU as long as possible to avoid redundant communications.

Code generation for multiple GPUs is addressed in Chapter 6. Two different sources of parallelism are presented and compared. I implemented a simple task extraction trans-formation, and a code generator targeting the StarPU runtime system. I also modified the existing symbolic tiling transformation to distribute loop nests on multiple GPUs.I implemented a dedicated runtime system to perform this distribution using OpenCL.

The whole process is automated thanks to a programmable pass manager (see Chapter 5). I implemented a generic stubs broker in charge of handling external library used in the processed code. It integrates the compilation chain and provides information to the final binary linking process.

Ialsoimplemen tedarun timesystemtomanagek ernellaunc hes;especiallyIdev eloped ah e u r i s t i ct oc h o o s ea tr u n t i m eal a u n c hc o n fi g u r a t i o nb a s e do nt h ed a t as i z ea n do nt h e target architecture.

Finally, I conducted a large range of experiments and analyses in Chapter 7 to validate all proposed transformations, schemes, and the implemented runtime systems. Twenty benchmarks of the Polybench suite, three from Rodinia, and the Stars-PM n-body application were used. The geometric mean speedup over all test cases that I obtain is over fourteen when compared to the sequential input code compiler with GCC. Stars-PM is accelerated by a factor fifty-two. Using multiple GPUs provides a speedup up to 1.63 when compared to the automatic compilation using only one GPU. Following Arch Robison's statement [START_REF] Robison | [END_REF] Compile-time program optimizations are similar to poetry: more are written than are actually published in commercial compilers.

The major contribution of this work, beyond the concepts, is included in an industrial integrated prototype released as open-source: the Par4All compiler. Based on the PIPS framework, it provides developers an automatic solution to achieve retargetability. The interface is minimal and Par4All can be invoked as simply as: p4a --opencl my_code.c -o my_binary.

Ia l s oi m p l e m e n t e daF o r t r a n9 5f r o n t e n di nPIPS.I ti sp r e s e n ti nt h ef o r mo fab r i d g e that converts the abstract syntax tree from the Fortran 95 parser in gfortran to the PIPS internal representation.

Moreover, the SILKAN company has applied the techniques presented in this dissertation in a user-friendly environment and delivered to Scilab programmers a one-click access to the power of hardware accelerators like GPUs.

Future Work

Chapter 6 contains some preliminary work about multiple GPUs, with encouraging experimental results. More work has to be done on kernel-specific optimization for multiple GPUs.F o ri n s t a n c e ,l o o pn e s t sh a v et ob er e s t r u c t u r e dt oe n s u r eb e t t e rl o c a l i t ya n dt o optimize the communications when distributing the data onto multiple GPUs.

In general, the trade-offs between the static decisions made by the compiler off-line and those made by the runtime systems have to be explored in more detail. The compiler could generate hints for the runtime system, such as estimation of the execution time of a kernel, or the usage of memory buffers as shown with StarPU in Chapter 6.

The data layout transformations are not studied in this dissertation. Some polyhedral tools may be well suited to perform this kind of transformation. Coupling such tools within Par4All should be explored further.

The use of OpenCL local memory is not addressed. It would be interesting to implement a scheme based on array regions and to compare it with polyhedral based solution.

The three P s Performance, Portability, and Programmability are nowadays completed with a fourth P :P o w e r . S o m el i b r a r i e so rDomain Specific Language (DSL) are designed to handle power-aware execution. Trading the existing three P sinapo w er-a w arecompiler offers room for further research.

Other paradigms emerge, for instance the ΣC dataflow language that comes along with the MPPA accelerator, both introduced in Section 2.2.9.W h i l et h ea u t h o r so fΣCe x p e c t that it can be targeted using tools like Par4All [Goubier et al. 2011], more research is needed to achieve that goal.

Finally, the hardware improves year after year. This work started before Nvidia Fermi was available and the next generation Kepler is now about to be released with new capabilities. The programming model of the GPUs changes. The dynamic parallelism feature introduced by Nvidia paves the way toward more and more flexibility exposed by the hardware through APIs like CUDA. Automatically mapping programs without hints is more difficult in this new context.

The collaboration between code generators from high-level tools or programming languages and a retargetable compiler like Par4All is a crucial way to explore. The high-level semantic informations are known by the code generator, and be capable to propagate them in the optimizing compiler seems to be the path to follow.

The GPU computing era and more generally the heterogeneous computing era is still at its beginning and many interesting challenges will be thrown to compiler developers.

Cette section contient pour chaque chapitre une traduction de son introduction et un résumé de chacune de ses sections. Le lecteur intéressé est invité à se reporter au chapitre correspondant pour obtenir des précisions sur un sujet particulier.

1I n t r o d u c t i o n 1.1 La Prophétie

Il était une fois, un monde dans lequel les développeurs écrivaient joyeusement leur code avec en tête la simple architecture de von Neumann (voir Figure 1.1). La performance était un facteur important bien sûr, mais ils étaient alors protégés par une bénédiction qui leur permettait de démarrer un projet nécessitant une puissance de calcul alors indisponible. En effet, le temps que le projet aboutisse était l'occasion d'évolutions significatives des performances des processeurs. La prophétie sur laquelle comptait chaque développeur était connue sous le nom de Loi de Moore. Elle est régulièrement citée sous la forme suivante (see [Srinivasan 2012, Manegold 2002[START_REF] Yang | AP a r a l l e lL o o pS e l f -Scheduling on Extremely Heterogeneous PC Clusters[END_REF]) : la fréquence des CPUs doublera tous les dix-huit mois. 1 Cette courte et simple affirmation a été insérée dans l'esprit de générations de développeurs pendant des décennies. Tout allait pour le mieux jusqu'à ce qu'un oiseau de mauvais augure annonça : ça ne peut pas continuer éternellement. La nature des exponentielles est que vous les poussez trop loin et finalement une catastrophe se produit. 2Il n'était pas le premier à remettre en cause la Prophétie, mais cette fois ce n'était autre que Gordon Moore lui-même [Dubash 2005], l'auteur de la Prophétie. Ce fût terrible pour les développeurs, et la plupart d'entre-eux s'enfermèrent dans le déni. Petit à petit, l'idée que la fréquence des processeurs n'allait pas continuer à augmenter comme auparavant a fait son chemin. En réalité, la Prophétie originelle peut figurer dans le top 10 des affirmations déformées, juste après "Luke, je suis ton père." En fait Moore avait affirmé à l'origine que : La complexité du coût minimum des composants a augmenté environ d'un facteur deux chaque année . . . . Certainement qu'à court terme on peut s'at-tendre à ce que ce taux continue, s'il n'augmente pas. Sur le long terme, le taux d'augmentation est plus incertain, bien qu'il n'y ait pas de raison de croire qu'il ne va pas rester environ constant pour au moins dix années. 3 Les quarante années de vie facile étaient terminées, comme l'illustre la Figure 1.2,e tl e s développeurs étaient sur le point de faire face à un nouveau défi. En effet les concepteurs de circuits, face à l'impossibilité d'augmenter la fréquence, ont sauté à pieds joints dans un monde parallèle 4 .I l so n ta l o r sc o n t o u r n él al i m i t ed ef r é q u e n c ed e sp r o c e s s e u r se n agrégeant plusieurs processeurs par circuit, multipliant la performance brute maximale théorique atteignable par un circuit. L'ère du multicoeurs avait débuté.

Les développeurs découvraient un nouvel univers : le temps d'exécution de leurs programmes n'était plus réduit significativement quand un nouveau processeur apparaissait sur le marché. Dans ce nouvel univers ils devaient repenser leurs algorithmes pour exploiter de multiples processeurs. Comme si ce n'était pas assez compliqué, certains concepteurs de circuits, qui avaient probablement rejoint le côté obscur de la force, commençaient à introduire des composants exotiques. Ces plate-formes matérielles étaient hautement parallèles, mais également très difficiles à programmer. Ainsi le chevalier blanc des développeurs qui souhaitait relever le défi devait non seulement repenser ses algorithmes mais également gérer manuellement des hiérarchies mémoires complexes.

Bienvenue dans le monde du calcul hétérogène !

Motivation

"Votre déjeuner gratuit est bientôt terminé" 5 Herb Sutter commença son article en 2005 [START_REF] Sutter | The Free Lunch Is Over: A Fundamental Turn Toward Concurrency in Software[END_REF]]aveccettedéclaration.Lalimitedel'augmentationdefréquencein terdit maintenant d'espérer une augmentation significative de la performance des programmes séquentiels. Le futur est hétérogène, du monde de l'embarqué des téléphones intelligents 6 et des tablettes jusqu'aux plus grands super-calculateurs. Sutter a écrit une suite à son 3. "the complexity for minimum component costs has increased at a rate of roughly a factor of two per year. . . . Certainly over the short term this rate can be expected to continue, if not to increase. Over the longer term, the rate of increase is a bit more uncertain, although there is no reason to believe it will not remain nearly constant for at least ten years."

4. Le parallélisme a été présent dans les processeurs à coeur unique depuis 1989 avec le processeur Very Long Instruction Word (VLIW) i860, et ensuite avec le jeux d'instruction Matrix Math eXtension (MMX) dans le Pentium. Depuis les développeurs avaient la possibilité d'exprimer du parallélisme fin dans le jeu d'instruction.

5. "Your free lunch will soon be over." 6. smartphones article [START_REF] Sutter | Welcome to the Jungle[END_REF]] dans laquelle il déclare : "Maintenant bienvenue dans la jungle des circuits matériels." 7 .L aF i g u r e1.3 illustre cette évolution.

Dans le monde des systèmes embarqués, les téléphones intelligents sont basés sur des processeurs multicoeurs et incluent des unités de calcul vectorielles ainsi qu'un processeur graphique 8 . Par exemple le processeur A5, utilisé dans l'iPhone 4S, comporte deux coeurs généralistes couplés à deux coeurs spécialisés pour les calculs graphiques [Anand-Tech 2011]. Le même processeur est également utilisé dans la tablette Apple iPad 2. Le dernier processeur Tegra 3 de Nvidia comporte quatre coeurs généralistes et douze coeurs graphiques [Nvidia 2012b]. Dans les deux cas, chaque coeur inclut une unité vectorielle NEON d'une largeur de 128 bits [Wikipedia 2012a]. La prochaine génération de Tegra offrira l'accès aux coeurs graphiques pour du calcul généraliste 9 àl ' a i d ed el ' Application Programming Interface (API) standard Open Computing Language (OpenCL).

Dans le monde des supercalculateurs, le parallélisme est présent depuis des décennies maintenant. Les machines vectorielles ont été remplacées par des grappes de systèmes multiprocesseurs multicoeurs dans la liste du Top500 [TOP500 Supercomputing Sites 2012]. La nouvelle tendance est maintenant d'ajouter des accélérateurs matériels dans ces systèmes, pour la plupart des processeurs graphiques (GPUs), ajoutant un nouveau niveau de complexité. La liste Top500 de juin 2011 comportait trois systèmes comportant des GPUs parmi les cinq premières places. Il y avait aussi cinq systèmes basés sur des GPUs dans la liste Green500 [START_REF] Feng | The Green500 List: Encouraging Sustainable Supercomputing[END_REF]]parmilesdixpremièresplaces [The Green500 2011]. Le supercalculateur Titan, à base de carte graphique Nvidia Tesla K20, décroche la première place du dernier classement du Top500 de novembre 2012 [TOP500 Supercomputing Sites 2012], et il est aussi intéressant de noter l'entrée à la septième place d'un système qui inclut des coprocesseurs Intel Xeon Phi.

Il est difficile voir impossible de trouver un ordinateur personnel ne possédant qu'un seul coeur de nos jours. Le double-coeurs est le standard en entrée de gamme, le quadcoeurs occupe les gammes intermédiaires, et le haut de gamme offre six coeurs et plus. Rendu nécessaire par l'industrie du jeu vidéo, les cartes graphiques livrées avec les ordinateurs personnels sont de plus en plus puissantes et sont utilisées dans un nombre croissant d'applications en dehors de leur utilisation première : le rendu et l'affichage de scènes 3D.

Le problème qui se pose maintenant, alors que les plate-formes hétérogènes sont largement disponibles, peut être résumé sous la forme des propriétés triple-P [Adve 1993, Benk-7. "Now welcome to the hardware jungle." 8. Graphics Processing Unit (GPU) 9. General-Purpose Processing on Graphics Processing Units (GPGPU) ner et al. 2011, Adve 2011]i l l u s t r é e sp a rl aF i g u r e1.4 :

• Performance : le programme exploite les capacités maximales du matériel.

• Portabilité : le code écrit par le développeur s'exécute sur un grand nombre de plateformes.

• Programmabilité : le développeur implémente ses algorithmes rapidement.

Une quatrième propriété peut être ajoutée : l'économie d'énergie 10 .N o ns e u l e m e n t parce que la batterie de nos téléphones intelligents est limitée, mais également car en 2007 chacun des dix plus grands supercalculateurs consommait autant d'énergie qu'une ville de quarante mille habitants [START_REF] Feng | The Green500 List: Encouraging Sustainable Supercomputing[END_REF]. Le concept de programmes qui adaptent leur exécution pour économiser de l'énergie est étudié [START_REF] Hsu | [END_REF], en faisant appel àu nc o m p r o m i se n t r epe r f o r m a n c ee tc o n s o m m a t i o nd ' é n e r g i e .

Des solutions qui tentent de répondre à toutes ces propriétés ont été explorées depuis des décennies. La complexité de programmation augmente avec la spécialisation des circuits tels que les accélérateurs matériels, au point de rendre la programmation impraticable pour la majorité des développeurs.

La performance s'est améliorée avec les compilateurs, permettant à de nouveaux langages de devenir concurrentiels face au langage C, la référence. Ce dernier reste incontesté quand il est nécessaire d'avoir un contrôle précis de l'exécution d'un programme pour du matériel donné.

Du point de vue de la portabilité, il est difficile de maintenir un programme de grande taille écrit en C qui exploite plusieurs plate-formes hétérogènes. Une pratique courante est alors de se restreindre à un ensemble de fonctionnalités communes à ces plate-formes, limitant alors les performances atteignables en pratique par rapport à la capacité maximale du matériel.

Finalement, la programmabilité a été traitée sous la forme de bibliothèques et de nouveaux langages. Par exemple, les langages UPC [UPC Consortium 2005], Co-Array Fortran [ISO 2010], ou Titanium [START_REF] Yelick | [END_REF]]u t i l i s el em od è l ed ' u ne s p a c ed ' a d r e s s a g e global partitionné (Partitioned Global Address Space (PGAS)). Le partitionnement est effectué de telle sorte que la mémoire est répartie entre les différents processeurs [Coarfa et al. 2005]. La localité des références mémoires est exploitée lors de l'exécution à l'aide de stratégies telles que celui qui possède la donnée effectue le calcul.L'objectifdeceslangages est de soulager le développeur de l'application du poids de la gestion des accès mémoires distants. Ce modèle mémoire plat a ensuite évolué avec l'ajout de l'asynchronisme dans 10. "Power"e na n g l a i s les languages dits "Asynchronous Partitioned Global Address Space (APGAS)"t e l sq u e X10 [START_REF] Ebcioğlu | [END_REF] ou Chapel [START_REF] Chamberlain | [END_REF]]. L'exécution concurrente est explicite et les développeurs expriment des constructions asynchrones à plusieurs niveaux. Alors que ces langages exigent des développeurs qu'ils modifient leur conception des algorithmes utilisés, des interfaces de haut-niveau sont fournies pour abstraire l'architecture matérielle sous forme de couches. Toutefois ces langages sont récents et pas encore très répandus. Des critiques ont été exprimées au sujet de la performance atteignable : dans la pratique le code semble devoir être optimisé pour une plate-forme cible pour obtenir de bonne performance [Zhang et al. 2011a].

Le standard OpenCL [Khronos OpenCL Working Group 2008, Khronos OpenCL Working Group 2011]aé t éd é v e l o p p ép o u rc i b l e rl e sa c c é l é r a t e u r sm a t é r i e l s .I lf o u r n i tu n e abstraction du matériel, basée sur une interface pour gérer des accélérateurs attachés à un hôte, et un langage dérivé d'un sous-ensemble de C pour écrire des noyaux,c ' e s tàd i r e des fonctions destinées à s'exécuter sur un accélérateur. Ce standard fournit une portabilité entre différents fournisseurs de matériel et place la programmabilité au niveau du langage C. Toutefois, la portabilité de la performance est difficile voir impossible à atteindre [START_REF] Komatsu | [END_REF]. Une autre approche est basée sur des langages de directives, dans la lignée du célèbre standard OpenMP pour les systèmes à mémoire partagée. Par exemple, des ensembles de directives telles que proposées par Hybrid Multicore Parallel Programming (HMPP) [Wikipedia 2012c], PGI Accelerator [Wolfe 2010], ou plus récemment OpenACC [OpenACC Consortium 2012]pe r m e t t e n ta u xd é v e l o p pe u r sd ep r o g r a m m e rf acilement des accélérateurs, tout en préservant la portabilité du code.

Plan

Le but de cette thèse est d'explorer le potentiel des compilateurs pour fournir une solution aux trois emphP : Performance, Portabilité, et Programmabilité. La solution considérée est la transformation automatique de code C ou Fortran vers un code équivalent exploitant un accélérateur matériel. La cible principale est le cas des accélérateurs de type GPUs : massivement parallèles et embarquant des giga-octets de mémoire. L'approche source-àsource bénéficie des interfaces de programmation Compute Unified Device Architecture (CUDA) et OpenCL.L ap r o g r a m m a b i l i t ée tl ap o r t a b i l i t és o n ta s s u r é e sp a rl ' a p p r o c h e entièrement automatique. Les résultats de nombreuses mesures sont fournies pour montrer que la performance n'est pas sacrifiée. L'approche retenue est pragmatique et les idées et stratégies présentées sont implémentées dans un nouveau compilateur source-à-source, Par4All [SILKAN 2010 (perso)], et validée avec des jeux d'essai. Le but principal est de fournir un prototype industriel d'une chaîne de compilation complète de bout en bout : du code séquentiel jusqu'au binaire prêt às ' e x é c u t e re nt i r a n tp a r t i ed ' u na c c é l é r a t e u rm a t é r i e l .P a rc o n s é q u e n t ,p l u t ô tq u ed es e concentrer sur une partie très limitée du problème, cette thèse contribue à différent aspects de la problématique et tente d'explorer et de résoudre les problèmes qui se posent quand on construit une telle chaîne complète de compilation.

Cette approche d'un compilateur est utile pour les applications existantes comme pour les nouveaux développements. Un compilateur efficace diminue le coût d'entrée mais également le coût de sortie quand une nouvelle plate-forme doit être ciblée. La maintenance est facilitée dans la mesure où le code est exprimé avec une sémantique séquentielle cohérence avec les habitudes des développeurs. Quand le code compilé n'est pas assez rapide, des parties spécifiques et particulièrement coûteuses peuvent être optimisées manuellement pour une architecture donnée : l'approche source-à-source rend possible des optimisations sur le code obtenu après traitement par le compilateur hétérogène. Le Chapitre 2 présente l'histoire des GPUs et l'émergence de l'utilisation des coeurs graphiques pour du calcul généraliste (GPGPU). L'évolution des circuits matériels est suivie par celle des langages de programmation qui échouent à répondre aux trois P s àl af o i s .L e sa r c h i t e c t u r ed e sGPUs et leur évolution sont introduites pour illustrer les contraintes à respecter par un compilateur pour exploiter la capacité maximale des circuits : mémoires distribuées, motifs d'accès la mémoire sur GPU,p a r a l l é l i s m ed eg r a i nfi n ,e t support d'opérations atomiques.

Le Chapitre 3 explore des solutions à la distribution automatique des données sur le Central Processing Unit (CPU) et les mémoires de l'accélérateur. La représentation abstraite sous forme régions de tableaux convexes est d'abord présentée. Une génération des communications hôte-accélérateur basée sur les régions de tableau est ensuite exposée.

Une nouvelle méthode interprocédurale d'optimisation est ensuite proposée et validée à l'aide d'expériences. L'algorithme se base sur une analyse statique et minimise les communications en préservant les données dans la mémoire de l'accélérateur pour éviter les communications redondantes.

Un ensemble de transformations de programmes et de boucles pour isoler et optimiser le code destiné à être exécuté par l'accélérateur sont développés dans le Chapitre 4. Une méthode souple est proposée pour faire correspondre l'espace d'itération d'un nid de boucles parallèles aux différentes couches d'exécution des GPUs.J ' a ic o n ç ue ti m p l é m e n t é une transformation pour substituer les variables d'induction dans des nids de boucles et permettre leur parallélisation. Deux différents algorithmes de parallélisation de nids de boucles sont présentés. Leur impact respectif, ainsi que l'impact de leur combinaison est étudié. J'ai proposé et implémenté une modification de ces algorithmes pour paralléliser en présence de réductions. J'ai conçu et implémenté une nouvelle transformation pour exploiter les capacités des circuits matériel à exécuter des opérations atomiques pour paralléliser les nids de boucles comportant des réductions. J'ai conçu et implémenté une méthode de fusion de boucles, et j'ai proposé des heuristiques pour conduire le choix des fusions à effectuer pour répondre aux contraintes particulières posées par les GPUs. Trois différents motifs de remplacement de tableaux par des scalaires sont présentés. J'ai modifié l'implémentation existante de cette transformation pour améliorer les performances dans le cas particulier des GPUs.L'impactdudéroulagedebouclesetdelalinéarisationdesaccèsaux tableaux est également présenté. Toutes ces transformations sont validées par des mesures de performance.

La chaîne complète de compilation est présentée dans le Chapitre 5,d uc o d es o u r c e séquentiel jusqu'au binaire final ainsi que le support exécutif proposé. Un gestionnaire de passes de compilation programmable est présenté, et la flexibilité qu'il offre est exploitée pour produire la chaîne de compilation. Des analyses interprocédurales sont mises en oeuvre et requièrent le code source de toutes les fonctions présentent dans le graphe d'appel. Cette contrainte peut poser un problème, particulièrement pour les librairies externes. J'ai proposé et implémenté une solution souple qui répond dynamiquement aux besoins du compilateur lors des différentes phases du processus de compilation.

Les perspectives d'extensions pour le cas de multiples accélérateurs sont explorées dans le Chapitre 6.D e u xa p p r oc h e spo u re x t r a i r el ep a r a l l é l i s m es o n té t u d i é e s .J ' a ii m p l é m e n t é une extraction simple du parallélisme de tâche, et modifié l'implémentation existante du tuilage symbolique pour l'adapter à cette problématique. Le cadre exécutif StarPU est utilisé pour exploiter le parallélisme de tâches et gérer l'ordonnancement sur plusieurs GPUs.

Finalement, les résultats expérimentaux sont présentés dans le Chapitre 7 pour valider les solutions définies dans les chapitres précédents. Vingt jeux d'essai sont extraits des suites Polybench et Rodinia. Une simulation numérique réelle de type n-corps est utilisée pour montrer qu'une accélération peut être obtenue automatiquement sur une application qui dépasse le cadre du simple jeu d'essai. Plusieurs cibles matérielles de Nvidia et d'Advanced Micro Devices (AMD) sont utilisées pour montrer de quelle manière les transformations de programmes mises en oeuvre impactent les performances de différentes manières en fonction des architectures.

En raison de la variété des sujets adressés dans cette thèse, la présentation des travaux liés est répartie dans les différents chapitres.

2C a l c u l g é n é r a l i s t e s u r p r o c e s s e u r s g r a p h i q u e s : h i stoire et contexte Le règne du processeur généraliste sous sa forme classique n'est plus hégémonique et le monde des processeurs est maintenant hétérogène. Les GPUs sont candidats au rôle de coprocesseurs depuis plus d'une décennie maintenant. D'autres architectures ont aussi été développées, tel que le Larabee d'Intel [START_REF] Seiler | [END_REF]. Ce dernier n'a jamais vraiment été mis sur le marché en tant que processeur graphique et a récemment trouvé une autre voie sous la forme de coprocesseur pour calculs intensifs en adoptant le nom commercial de Xeon Phi à la fin de l'année 2012. Citons également le processeur Cell [Hofstee 2005] développé par IBM et popularisé par son utilisation dans la console de jeu PlayStation 3 de Sony. Toutefois, bien que plusieurs travaux de recherches ont visé à adapter des algorithmes pour tirer parti de son architecture complexe, le Cell a été abandonné. Cet échec est lié às o nm o d è l em é m o i r ee ts o nm o d è l ed ep r o g r a m m a t i o nt r o pc o m p l e x e ,p a r t i c u l i è r e m e n t face aux alternatives industrielles : les fabricants de circuits graphiques sont entrés dans le marché du calcul généraliste.

Les unités dédiées au calcul graphique offrent, en général via leurs pilotes, un accès àu n ei n t e r f a c ed ep r o g r a m m a t i o ns t a n d a r dt e lq u ' O p e n G L [ Khronos OpenGL Working Group 1994, Khronos OpenGL Working Group 2012]e tD i r e c t X [ Microsoft 1995, Microsoft 2012]. Ces interfaces sont spécifiques au traitement graphique, l'application principale de ce type de circuits. Le traitement graphique fait appel à de nombreuses opérations naturellement vectorielles, les GPUs ont donc été dotés de capacités visant à multiplier un vecteur par un scalaire en une opération unique. Ces capacités vectorielles ont été détournées du traitement graphique vers des calculs généralistes.

Ce chapitre présente d'abord dans la Section 2.1 l'histoire du calcul généraliste basé sur circuits graphiques (GPGPU), puis la Section 2.2 relève les points clés de l'évolution des modèles de programmation, ainsi que les différentes initiatives qui ont été prises pour placer au premier plan la pratique du GPGPU. Le standard OpenCL est introduit en détail Section 2.3.L e sa r c h i t e c t u r e sGPU contemporaines sont présentées Section 2.4.F i n a l ement je liste les différents défis que ces architectures présentent aux développeurs et aux concepteurs de compilateurs.

Historique

L'utilisation de matériel graphique pour des calculs généralistes a été un sujet de recherche pendant plus de vingt ans. Harris et al. propose [Harris et al. 2002]u nh i s t o r i q u e qui commence avec la machine Ikonas [START_REF] England | Asystemforinter activemo delingofphysic alcurve dsurfac e objects[END_REF]], la Pixel Machine [START_REF] Potmesil | [END_REF], et Pixel-Planes 5 [START_REF] Rhoades | [END_REF] D'autres modèles de programmation émergent, tels que les langages à base de directives. Ces langages laissent les développeurs écrire du code portable, maintenable, et, avec un peu de chance, efficace. Les directives sous forme de pragmass o n taj o u t é e sa uc o d es é q u e n t i e l pour indiquer au compilateur quelle partie du code doit être exécutée sur l'accélérateur. Cette méthode est moins intrusive mais aboutit actuellement à des performances plus limitées. Plusieurs propositions de directives sont présentées à la Section 2.2.10.

Langages, cadres d'applications, et modèles de programmation

L'historique des langages de programmation comporte de nombreux langages, cadres d'exécution, et modèles de programmation conçus pour programmer les accélérateurs. Certains ont été conçus dans le but spécifique de la cible d'origine des GPUs,c ' e s tàd i r el e calcul graphique, et ont été détournés plus tard pour la pratique du GPGPU.D'autreson t été conçus dès la base pour répondre au besoin de la pratique du GPGPU.

Cette section passe en revue les contributions, approches, et paradigmes majeurs impliqués au cours de la dernière décennie dans la programmation des accélérateurs matériels.

Open Graphics Library (OpenGL)

En 1992, Silicon Graphics Inc. introduit la spécification d'OpenGL. Cette interface est utilisée pour programmer les applications qui font appel à du calcul graphique 2D et 3D et fournit une abstraction du flux de traitement graphique. Les objets manipulés sont des points, lignes et polygones. Ils sont convertis en pixels via le flux d'exécution graphique, paramétré par la machine à état d'OpenGL. A la base, chaque étage du flux de traitement n'était capable que d'effectuer des opérations prédéfinies et n'était configurable que dans des limites restreintes. Cependant depuis l'introduction des shaders dans Open Graphics Library (OpenGL) 2.0 [Khronos OpenGL Working Group 2004], le langage OpenGL Shading Language (GLSL) rend plusieurs étages complètement programmables. La Figure 2.2 montre l'exemple d'un shader de calcul généraliste.

Shaders

Les shaders sont de petits programmes utilisés dans le cadre du traitement graphique à différents étages du flux de traitement. Ils sont utilisés pour décrire l'absorption et la diffusion de la lumiére ainsi que les textures à appliquer, les réflexions et réfractions, l'ombrage, et le déplacement de primitives graphiques. Le processus de rendu place les shaders comme de parfaits candidats pour une exécution parallèle sur des processeurs graphiques vectoriels, libérant le CPU de cette tâche coûteuse et produisant des résultats plus rapidement.

Le niveau de programmation est actuellement proche du langage C et le parallélisme est implicite. Cependant s'ils ajoutent de la flexibilité et de la programmabilité au flux d'exécution graphique pour les calculs généralistes, ils ne permettent pas aux développeurs de s'abstraire de l'interface graphique et des primitives associées. La Figure 2.3 illustre un exemple de shader géométrique utilisant le langage GLSL.

Brook and BrookGPU

Jusqu'à 2003, le seul moyen de bénéficier des ressources du matériel graphique était d'utiliser les shaders proposés par les interfaces graphiques OpenGL et DirectX. BrookGPU [Buck et al. 2004]i m p l é m e n t eu ns o u s -e n s e m b l ed el as pé c i fi c a t i o nd ul a n g a g e Brook [Buck 2003]p o u rc i b l e rl e sGPUs. Il offre la possibilité de compiler le même code pour différentes cibles, OpenGL and DirectX bien sûr, mais aussi les shaders Nvidia Cg et plus tard l'interface généraliste d'AMD : Close To Metal (CTM).L aF i g u r e2.4 reproduit définit une interface de programmation pour gérer les accélérateurs depuis l'hôte. OpenCL a été proposé par Apple au groupe Khronos en 2008 pour unifier les différents environnements de programmation dans un standard, qui a été finalement adopté plus tard la même année [Khronos OpenCL Working Group 2008]. La Section 2.3 fournit plus de détails sur OpenCL.

Microsoft DirectCompute

Microsoft propose sa propre solution pour la pratique du GPGPU avec DirectCompute [Microsoft 2010]l i v r ée nt a n tq u ec o m p o s a n td eD i r e c t X1 1fi n2 0 0 9 .L ' i n t e r f a c ed e DirectCompute se base sur le langage de shader High Level Shader Language (HLSL) (identique à Nvidia Cg) pour fournir une solution qui s'affranchie du flux d'exécution graphique en faveur d'un accès direct tel que CUDA ou OpenCL.L e sd é v e l o p p e u r sf a m i l i e r s avec HLSL/Cg peuvent alors transférer des zones mémoires directement vers ou depuis la mémoire embarquée de l'accelérateur, puis exécuter des shaders (ou noyaux de calcul) pour traiter ces zones mémoires. La Figure 2.5 montre un exemple d'un tel noyau de calculs. Contrairement à Direct Compute présenté Section 2,i ln ' yap a sd es é p a r a t i o ne n t r el e code qui tourne sur l'accélérateur et le code hôte. Les calculs à exécuter sur un accélérateur sont isolés sous forme de lambda fonctions encapsulées dans une des constructions introduite par la spécification pour exprimer l'espace d'itération, telle que parallel_for_each par exemple. La Figure 2.6 montre un exemple de code C++ avant et après conversion vers C++ AMP.

Langage à base de directives et cadre de programmation

Suivant l'exemple du populaire standard Open Multi Processing (OpenMP) pour les machines à mémoire partagée, plusieurs propositions ont été faites pour l'ajout de directives, principalement aux langages C et Fortran. L'objectif est de répondre aux difficultés rencontrées par les développeurs pour écrire du code efficace, portable, et maintenable, tout en convertissant progressivement des applications séquentielles existantes vers l'utilisation d'accélérateurs.

Le principal inconvénient, est que même si les directives rendent le code plus simple àé c r i r e ,l ed é v e l o p p e u rd o i tc e p e n d a n tp o s s é d e ru n eb o n n ec o n n a i s s a n c ed el ac i b l ee td e la manière dont l'algorithme peut être appliqué sur un accélérateur. Le code résultant de l'ajout des directives est finalement optimisé de manière spécifique à une architecture.

Les Figures 2.7, 2.8, 2.9,et2.10 illustrent des exemples de traitements avec les langages de directives hiCUDA, HMPP, PGI Accelerator, et JCUDA.

Parallélisation automatique pour la pratique du GPGPU

Les travaux de parallélisation automatique de code séquentiel pour cibler des GPUs sont récents. [START_REF] Leung | [END_REF]]p r o po s e n tu n ee x t e n s i o nàu nc o m p i l a t e u rJust In Time (JIT) Java qui exécute des boucles parallèles sur un GPU.Nugterenetal. [Nugteren et al. 2011]p r é s e n t e n tu n et e c h n i q u ep o u ra u t o m a t i q u e m e n tf a i r ec o r r e s p o n d r ed uc o d e séquentiel à une implémentation sur un accélérateur en utilisant une approche de skeletonization. Cette technique est basée sur un jeu de squelettes prédéfinis pour le traitement d'images. L'étape de skeletonization reconnaît les fonctionnalités de l'algorithme à l'aide de reconnaissance de motifs, puis les remplace avec une autre implémentation prédéfinie pour l'accélérateur cible.

CUDA-Chill [START_REF][END_REF]]fournitunetransformationautomatiquedeprogrammes pour GPU utilisant le cadre de compilation Chill pour assembler des transformations de boucles. Toutefois les recettes Chill doivent être adaptées à chaque programme à traiter, limitant la portabilité de l'approche.

Baskaran et al. [START_REF] Baskaran | [END_REF]in troduisen tuneapproc hepoly édriqueàlaparallélisation automatique vers CUDA de nids de boucles affines exprimées en C dans l'outil Pluto [Bondhugula et al. 2008c]. Plus récemment, le projet PPCG [Verdo olaege et al. 2013] suit le même chemin et produit des noyaux optimisés pour GPU en utilisant le modèle polyédrique.

Architectures cibles

Cette thèse se concentre sur les accélérateurs matériels tels que les GPUs.L e sc a r a c t éristiques communes de tels accélérateurs sont les suivantes :

• une large mémoire embarquée : de l'ordre de quelques giga-octets ;

• un parallélisme massif : de quelques dizaines d'unités de calcul, à plusieurs milliers ;

• compatibilité avec le modèle de programmation défini par le standard OpenCL introduit dans la section 2.

Conception d'un circuit graphique (GPU)

Un GPU est un circuit énorme et complexe, traditionnellement construit sur des unités de calcul très spécialisées pour traiter efficacement des opérateurs graphiques prédéfinis. Ave c l ' i nt r o d u c t i o n d e s shaders,c e sc i r c u i t ss o n td e v e n u sd ep l u se np l u sfl e x i b l e se tp r ogrammables. Puisque que la pratique du GPGPU est l'objectif principal de cette thèse, seule la puissance de calcul dédiée aux shader et les hiérarchies mémoires sont importantes. Les circuits dédiés au traitement graphique sont volontairement omis dans cette section.

Au niveau le plus fin, nous trouvons les unités de calcul (Processing Element (PE)), capables d'opérations basiques telles que des additions, des multiplications, ou au mieux des opérations combinées (Fused Multiply-Add (FMA)). Certains PEs sous stéroïdes sont capables de traiter des fonctions transcendantes telles que des opérations trigonométriques, des exponentielles, ou des racines carrées. De telles unités sont nommées Special Function Unit (SFU).

Plusieurs PEs sont alors groupés ensemble dans un multiprocesseur (Compute Unit (CU)). Une CU inclue les circuits de logiques partagés par les PEs,t e l sq u el ed é c o d a g e des instructions, les registres, le cache, un ordonnanceur, etc.

Un circuit graphique est ensuite conçu par assemblage de plusieurs CUs àl ' a i d ed ' u n réseau d'interconnexion. Un ordonnanceur matériel global est ensuite ajouté pour distribué le travail aux différentesCUs,e te n fi nd e sc o n t r ô l e u r sm é m o i r ei n c l u a n tu né v e n t u e lc a c h e global offrent un accès aux giga-octets de DDR embarqués. Les CUs peuvent également être groupées avant d'être reliées au réseau d'interconnexion et ce groupe partage certaines ressources tels que le cache ou des unités de traitement graphique. La Figure 2.15 illustre cette vue conceptuelle de l'architecture d'un GPU.

Une telle vue n'est pas si éloignée de ce qu'on retrouve dans un processeur multicoeurs, mais le diable est dans les détails. Les choix qui sont réalisés à chaque niveau sur le nombre d'éléments et la manière dont ils sont groupés et partagent des ressources a un impact important sur la programmabilité. En général, par rapport à un CPU,l ' e s s e n t i e l de la surface du circuit est utilisée pour de la logique de calcul. C'est pourquoi il y a beaucoup de PEs avec un groupement complexe partageant les ressources qui n'effectuent pas de calculs, peu de mémoire cache, et une bande passante mémoire importante, mais également une latence élevée.

En général, les concepteurs essayent d'obtenir des CUs aussi simple que possible et n'incluent pas de capacité de réordonnancement du flux d'instruction 12 ,e td o n cl as o u r c e principale de parallélisme est le parallélisme de fils d'exécution (Thread Level Parallelism (TLP)). Toutefois, le parallélisme dans le flux d'instruction (Instruction Level Parallelism (ILP))p e u tê t r ee x p l o i t ép a rl ec o m p i l a t e u ro ul ed é v e l o p p e u re nu t i l i s a n tu nj e ud ' i n struction spécifique (Very Long Instruction Word (VLIW)), ou par l'ordonnanceur matériel en gérant les dépendances entre instruction pour conserver un fort taux de remplissage du pipeline.L aF i g u r e2.16 illustre les différences entre l'ILP et le TLP. Jusqu'à ce moment, les shaders étaient encore nouveaux et séparés dans des circuits dédiés pour les shaders de géométrie et les shaders de sommets. Les concepteurs de circuits graphiques avaient conclu qu'un jeu d'instruction VLIW était l'architecture idéale pour les shaders de sommets. Ce jeu d'instruction permet le traitement simultané d'une opération Single Instruction stream, Multiple Data streams (SIMD) sur un vecteur à quatre composants (w, x, y, z)e tu n ea u t r eo p é r a t i o ns u ru nc o m p o s a n ts c a l a i r es é p a r é ,p a re x e m p l el a luminosité.

Le compilateur joue alors un rôle crucial pour assembler les opérations dans une instruction VLIW en exploitant l'ILP qui peut être trouvée dans un shaders.P a ro p po s i t i o n , le TLP est géré en répliquant les unités de calcul et en traitant plus de données simultanément. L'ordonnancement statique effecuté par le compilateur simplifie le matériel et permet de dédier une plus grande proportion des transistors au profit des unités de calcul plutôt que pour complexifier l'ordonnanceur matériel.

DirectX 10 unifie la programmation des différents type de shaders. Ces changements ont 12. "out-of-order execution" poussé les concepteurs de circuits graphiques à refléter ces changements dans l'architecture. Les mêmes unités sont alors en charge de l'exécution de tous les types de shaders.P o u r AMD ce changement est introduit dans le circuit R600. Des nouveaux circuits de logique sont introduits pour prendre en charge l'ordonnancement des différents fils d'exécution en provenance des différents types de shaders qui entrent en concurrence pour les ressources. L'introduction d'un ordonnanceur matériel est un point important pour l'évolution de la pratique du GPGPU comme cela s'est confirmé par la suite des évolutions matérielles qui ont proposé des fonctionnalités construites au dessus de cet ordonnanceur. La dernière révision majeure en date de l'architecture Nvidia a été livrée en mars 2012 et porte le nom de Kepler. Nvidia met l'accent sur le ratio de la capacité de calcul par watt, amélioré en réduisant la fréquence des PEs de moitié de sorte à être identique aux unités d'ordonnancement. Les quatre nouveautés architecturales principales sont les suivantes :

• Le parallélisme dynamique offre la capacité de générer de nouveaux fils d'exécution sur le GPU depuis l'exécution d'un noyaux. Un noyaux peut initier des transferts mémoires entre l'hôte et l'accélérateur, ou lancer l'exécution de nouveaux noyaux. C'est un changement important dans le modèle de programmation existant.

• Le mécanisme de files multiples Hyper-Q permet jusqu'à trente-deux fils d'exécution de l'hôte de soumettre des commandes à un accélérateur dans des files indépendantes, permettant un partage plus efficace d'un accélérateur.

• L'unité de gestion de grille est le circuit matériel de base pour activer le parallélisme dynamique. Il remplace l'ordonnanceur précédent en offrant plus de flexibilité dans la répartition, la mise en file, et les dépendances pour jusqu'à 2000 différents contextes d'exécution simultanés.

• GPU Direct est la technologie qui autorise à transférer des données directement entre deux accélérateurs ou entre la mémoire d'un GPU et un autre périphérique connecté au bus PCIe,t e lq u ' u na d a p t a t e u rr é s e a uI n fi n i b a n dp a re x e m p l e .

Impact sur la génération de code

Dans cette thèse, je ne cible pas une architecture en particulier et je souhaite offrir la possibilité de générer du code qui s'exécutera efficacement sur l'ensemble des architectures introduites précédemment. La question principale est la suivante : à quel point la performance obtenue en transformant un code est portable d'une architecture à une autre ? les travaux de Wong et al. [Wong et al. 2010] et Collange [Collange 2010b] pour le GT200, et [START_REF] Lindholm | NVIDIA Tesla: A Unified Graphics and Computing Architecture[END_REF]] et la thèse de Collange [Collange 2010a]p o u r Fermi.

Conclusion

Il y a une décennie, la pratique du GPGPU n'en était qu'à ses débuts. Depuis elle a été l'objet d'intenses recherches et est toujours un domaine très actif, comme illustré sur la Figure 2.25. Ce chapitre a montré que plusieurs langages, cadres de programmation, et autres solutions ont été proposés et expérimentés pour aider les développeurs à écrire des programmes qui exploitent les accélérateurs. Toutes ces approches se basent sur différents compromis des propriétés trois P s : Performance, Portabilité, Programmabilité. La performance portabilité est un challenge dans le contexte de la pratique du GPGPU.L a programmabilité a été traitée par divers langages spécialisés.

Le paysage en 2012 est très différent de ce qu'il était il y a dix ans quand les chercheurs contournaient le flux d'exécution purement graphique d'OpenGL pour traiter des opérations mathématiques abstraites [START_REF] Trendall | [END_REF].

La programmabilité était un défi. Toutefois, quand un programme était converti vers l'interface OpenGL, les performances et la portabilité étaient garantis. L'introduction des shaders en 2002 a apporté de la flexibilité et offert plus de fonctionnalités et amélioré la programmabilité.

Des approches plus récentes tel que le langage Brooks ont tenté de faire un compromis sur les performances au profit de la programmabilité. L'interface de programmation par flux répond à certains besoins des développeurs, mais n'est pas suffisamment flexible pour se généraliser. L'évolution de DirectX a conduit les fabricants de GPUs vers plus de programmabilité. Cependant le point clé dans la démocratisation de la pratique du GPGPU aétél'in troduction de CUDA,s u i v ie n s u i t ep a rd e sl a n g a g e sa l t e r n a t i f se td e sc a d r e sd ep r o g r a m m a t i o n .

Les modèles de programmation CUDA et OpenCL bénéficient de l'expérience des shaders pour fournir un niveau équivalent de programmabilité mais en s'affranchissant des mécanismes rigides et inadaptés imposés par l'utilisation des flux de traitement graphiques d'OpenGL et Direct X. Toutefois les développeurs doivent connaître l'architecture pour écrire des noyaux de calcul efficaces : la performance est favorisée au détriment de la portabilité .

Les langages à base de directives tels que hiCUDA, JCUDA, HMPP, PGI Accelerator, ou OpenACC (voir Section 2.2.10) sont moins intrusifs et offrent une bonne portabilité au prix de la performance. Les directives peuvent être spécialisées pour une cible donnée pour favoriser la performance, mais en sacrifiant la portabilité.

Mon travail de doctorat a démarré juste après l'annonce de la première version du standard OpenCL au printemps 2009 [START_REF] Ramey | Languages, APIs and Development Tools for GPU Computing[END_REF]]. Le but de ce travail était de fournir une solution de bout en bout qui évite aux développeurs l'adaptation manuelle de leurs codes pour les accélérateurs graphiques.

La programmabilité est aussi bonne que possible, puisque les développeurs écrivent leurs codes en utilisant des langages de programmation séquentiels et en ignorant la présence de l'accélérateur. Le compilateur extrait le parallélisme et le code à exécuter sur l'accélérateur. La performance peut ne pas atteindre ce qu'un expert pourrait obtenir avec des efforts. Toutefois le compromis sur la performance est acceptable si la différence avec le code produit par un expert pour une architecture donnée est limitée, par exemple dix, vingt, ou trente pour cent selon les domaines d'application.

Très p eu de travaux ont p orté sur des chaînes entièrement automatiques de parallélisation automatique et de transformation (voir Section 2.2.11 page 30)d ' u nc o d es é q u e n t i e l vers du code GPU.Laplupartdespropositionsson td'applicationslimitéesouseconcentre seulement sur une partie du problème. Mon travail tente de traiter une application complète, de générer des noyaux, de les optimiser, et de générer les communications requises, sans impliquer l'utilisateur dans le processus de décision. Il a été montré à la Section 2.4.6 de quelle manière l'évolution des architectures des GPUs impacte la génération de code.

Guelton propose dans sa thèse [Guelton 2011a]unevuedehaut-niveaud'uncompilateur hétérogène pour cibler des accélérateurs. Le compilateur transforme le code, sépare le code hôte et le code des noyaux, avec la col le nécessaire pour assembler les morceaux. Chaque partie est ensuite compilée par le compilateur binaire de la cible. La Figure 2.26 illustre cette vue de haut-niveau.

Mon travail propose d'instancier cette vue de haut-niveau dans un compilateur pour GPUss. Un aperçu de la structure du compilateur proposé est présenté dans la Figure 2.27, qui distingue mes contributions. Sans explorer tous les aspects de chaque élément de la chaîne, cette thèse fournit des solutions à plusieurs problématiques liées à la parallélisation automatique pour GPUs,allan tdeladétectionduparallélismeàlagénérationdecode,en passant par les optimisations de nids de boucles et la gestion du placement des données.

3P l a c e m e n t d e s d o n n é e s , c o m m u n i c a t i o n s , e t c o h érence

Les accélérateurs tels que les GPUs traitent des données placées dans leur mémoire embarquée. En effet un circuit graphique est accompagné d'une mémoire dédiée à hautdébit de plusieurs giga-octets, comme exposé à la Section 2.4.L ad i ffi c u l t ée s tq u ec e t t e mémoire embarquée n'est pas directement accessible depuis l'hôte et réciproquement la mémoire de l'hôte n'est pas directement accessible depuis l'accélérateur. Les développeurs doivent explicitement transférer les données à traiter dans la mémoire de l'accélérateur avant d'exécuter un noyau, et transférer dans le sens opposé les données résultant du traitement sur l'accélérateur.

Ces communications explicites utilisent des bus à la bande passante limitée. Par exemple, le bus PCIe 2.0 offre une bande passante théorique maximale de 8 GB/s, ce qui reste peu comparé à la bande passante de la mémoire embarquée des GPUs de l'ordre de centaines de GB/s. Cette limitation est régulièrement considérée comme le goulet d'étranglement pour la programmation hybride [START_REF][END_REF].

Des travaux ont été menés pour répondre à cette problématique, soit en utilisant des informations fournies par le développeur [START_REF][END_REF], CAPS Entreprise 2010[START_REF] Wolfe | [END_REF][START_REF] Nvidia | NVIDIA, Cray, PGI, CAPS. The OpenACC Specification, version 1.0[END_REF], soit automatiquement [Amini et al. 2011c(perso), Ventroux et al. 2012, Guelton 2011a, Alias et al. 2011, Wolfe 2010]pardesanalysesdecompila-tion. Une méthode paresseuse a également été proposée par Enmyren et Kessler [START_REF] Enmyren | SkePU: a multibackend skeleton programming library for multi-GPU systems[END_REF] dans la bibliothèque de squelette C++ SkePU.

Ce chapitre étudie la problématique de la génération des communications dans le contexte de la parallélisation automatique ou semi-automatique et présente plusieurs contributions pour résoudre ce problème : les régions convexes de tableau sont exploitées pour optimiser la quantité de données à transférer par noyau, ainsi qu'une méthode de placement statique basée sur une analyse interprocédurale.

Paralléliseur Interprocédural de Programmes Scientifiques (PIPS) est un cadre de compilation développé depuis plus de vingt ans [Irigoin et al. 1991, Amini et al. 2011a (perso)] qui comporte un ensemble de phases d'analyse sémantique et de phases de transformation de code. Initialement traitant le langage Fortan 77, il a été étendu pour gérer le langage C. PIPS génère un code source modifié, aussi proche que possible du code d'entrée. D'abord, le type de programmes ciblés est présenté avec une étude de cas : une simulation cosmologique. Ensuite l'abstraction des régions de tableaux convexes, qui est à la base de la plupart des transformations, est présentée à la Section 3.2.L ep l a c e m e n tl e plus simple est ensuite décrit à la Section 3.3 pour introduire les mécanismes mis en jeu. Les régions de tableaux convexes sont ensuite utilisées à la Section 3.4 pour améliorer le résultat de la section précédente. Les limites de l'approche sont exposées à la Section 3.5. Une optimisation interprocédurale est proposée Section 3.6 pour placer efficacement les données sur un accélérateur et limiter le nombre de transfers.

La promotion parallèle de code séquentiel présentée à la Section 3.7 peut aider en complément de la fusion de boucles pour réduire les synchronisations entre deux noyaux mais également en éliminant certains transferts mémoires.

Finalement, les travaux liés à l'optimisation de communication pour la pratique du GPGPU sont présentés Section 3.8.

Étude de cas

Les jeux d'essai tels que ceux utilisés dans la suite Polybench [START_REF] Pouchet | The Polyhedral Benchmark suite 2.0[END_REF]]p a r exemple, sont limités à quelques noyaux en séquence, de temps en temps encapsulés dans une boucle de temps. Par conséquent, s'ils sont adaptés à l'étude la performance brute de l'exécution d'un noyau sur GPU,ilsnepeuv en têtreconsidéréscommereprésen tatifsd'une application complète lorsqu'il s'agit d'évaluer la problématique du placement des données entre hôte et accélérateur.

L'application présentée dans cette section est plus représentative des simulations numé-σ(i) dénote la valeur de la variable i dans l'état mémoire σ.

Processus basique de transformation

Le processus le plus basique pour placer les données sur l'accélérateur consiste à envoyer vers l'accélérateur tous les tableaux référencés dans un noyau avant de l'exécuter. Le même ensemble de tableaux doit être transféré depuis l'accélérateur à la fin de l'exécution du noyau. Ce processus basique, le plus simple qui puisse être mis en oeuvre par des outils automatiques, est représenté Figure 3.7.

La problématique principale survient quand il est nécessaire de compter le nombre d'éléments de tableau à transférer. Selon le langage, l'information peut être délicate à trouver. [START_REF] Leung | [END_REF] et JCUDA [START_REF][END_REF] L'outil proposé avec cette thèse, Par4All [SILKAN 2010 (perso), Amini et al. 2012b (perso)]( p r é s e n t ée nd é t a i lS e c t i o n5.1), se base sur le même principe dans sa version la plus basique, en étant toutefois un peu plus souple en acceptant les tableaux C99 "Variable Length Array (VLA)". La taille concrète ne sera connue qu'à l'exécution mais l'information est utilisée de manière symbolique lors de la compilation.

Raffinement à l'aide de régions convexes

Cette section présente un raffinement du modèle basique basé sur les déclarations des tableaux introduit dans la section précédente. Les régions convexes de tableaux introduites àl as e c t i o nS e c t i o n3.2 sont mises en oeuvre. Le principe sous-jacent, l'isolation d'instructions, a été décrit de manière formelle dans la thèse de Guelton's [Guelton 2011b].

Pour illustrer comment les régions de tableau sont exploitées, l'exemple de la boucle while sur la Figure 3.5 illustre le résultat du calcul des régions sur-approximées : Il est intéressant de noter comment les appels systèmes memcpy sont utilisés ici pour simuler les transferts de données dans un nouvel espace mémoire, et en particulier comment la taille des transferts est contrainte d'après les régions de tableau.

R = {{x} , {y}} R(randv)={randv[φ 1 ] | N -3  4 ⇥ φ 1 ;3⇥ φ 1  N } W = {{x} , {y}} W(a)={a[φ 1 ] | N -3  4 ⇥ φ 1 ;12⇥ φ 1  5 ⇥ N +9} L'

Limites

L'échange des données entre l'hôte et l'accélérateur sont effectués sous forme de transferts Direct Memory Access (DMA) entre mémoires à travers du bus PCI Express,q u i offre actuellement une bande passante théorique de 8 GB/s. C'est très lent face à la bande passante de la mémoire embarquée du GPU, qui dépasse souvent 150 GB/s. Cette bande passante faible peut annihiler tous les gains obtenus lors du déport de noyaux.

Le matériel que nous avons utilisé (voir Section 7.1)n o u sp e r m e td em e s u r e rj u s q u ' à 5.6 GB/s lors des transferts de l'hôte vers le GPU,e t6.2 GB/s dans l'autre sens. Cette bande passante est obtenue pour des morceaux de plusieurs MB, mais diminue pour des morceaux plus petits. De plus cette bande passante maximale est réduite de moitié environ quand les zones mémoires sur l'hôte ne sont pas allouées de manière épinglée ;c ' e s tà dire sujet à la gestion de la mémoire virtuelle paginée par le système d'exploitation, la Figure 3.14 illustre l'évolution de la bande passante en fonction de la taille de la zone mémoire à transférer et de l'allocation des données.

Algorithme d'optimisation des communications

Beaucoup de travaux ont été fait au sujet des optimisations de communications pour les ordinateurs distribués, tels que la fusion de message dans le contexte des programmes unique à mémoires distribuées (Single Program Distributed Data (SPDD)) [START_REF] Gerndt | Optimizing communication in SUPERB[END_REF]], des analyses de flots de données basées sur des régions de tableau pour éliminer les communications redondantes et masquer les communications par des calculs [START_REF] Gong | [END_REF], et la distribution dans le contexte de la compilation High Performance Fortran (HPF) [Coelho 1996[START_REF] Coelho | Optimal Compilation of HPF Remappings[END_REF].

Des méthodes similaires sont appliquées dans cette section au déport des calculs dans le contexte d'une relation hôte-accélérateur et pour intégrer dans un compilateur parallélisant une transformation qui limite la quantité de communication CPU-GPU àl ac o m p i l a t i o n .

Cette section introduit une analyse de flot de données conçue pour améliorer la génération statique des transferts mémoires entre l'hôte et l'accélérateur. Une hypothèse de base dans cette section est que la mémoire du GPU est suffisamment grande pour contenir l'ensemble des tableaux à traiter. Alors que cette supposition peut sembler une contrainte inacceptable pour certaines applications, les mémoires des GPUs modernes de plusieurs giga-octets sont suffisamment importantes pour une large gamme de simulations.

Promotion de code séquentiel

Des nids de boucles parallèles peuvent être séparés par du code séquentiel. Quand ce code séquentiel utilise ou produit des données utilisées ou produites par les noyaux avant et après, des transferts entre la mémoire de l'hôte et celle de l'accélérateur sont nécessaires.

Une solution à ce problème est de promouvoir ce code séquentiel sur un unique fil d'exécution sur l'accélérateur. Alors que l'exécution d'un unique fil d'exécution sur le GPU est inefficace, la perte peut être bien moindre que le coût de la communication. La problématique de la profitabilité de cette transformation rejoint celle de la profitabilité de l'exécution d'un noyau sur un accélérateur, discutée plus spécifiquement à la Section 5.6 page 156. L'exemple gramschmidt mentionné dans la section précédente est illustré par la Figure 3.17.L e sc o d e sg é n é r é sa v e ce ts a n sp r o m o t i o ns é q u e n t i e l l ei l l u s t r e n tc o m m e n tl e compromis réalisé sur la performance du code séquentiel peut réduire les communications. La Section 7.8.4,p a g e201,m o n t r en o sm e s u r e sq u ii n d i q u e n tu n ea c c é l é r a t i o nj u s q u ' à huit fois pour l'exemple gramschmidt,m a i sa u s s ij u s q u ' àt r e n t e -s e p tf o i sp o u rl ' e x e m p l e durbin de la suite Polybench.

En revanche, en cas de mauvais évaluation de la profitabilité, les performances peuvent chuter drastiquement. Cette transformation requiert une évaluation précise de l'exécution de chacune des versions. Une manière de résoudre ce problème est d'effectuer une mesure àf r o i davec un échantillon d'une unique itération de la boucle séquentielle sur le GPU et de retarder la décision au moment de l'exécution. Cette approche est explorée à la Section 5.7,p a g e158,s e r a i tt o u t e f o i sd i ffi c i l eàc o m b i n e ra v e cl am é t h o d ed ' o p t i m i s a t i o n de communication introduite dans ce chapitre.

Résultats expérimentaux

La Section 7.8,p a g e197,p r é s e n t ed e sr é s u l t a t se x p é r i m e n t a u xp o u rl am é t h o d ed e limitation des communications introduite dans ce chapitre.

JCUDA et hiCUDA [Han & Abdelrahman 2009]. Elle pourrait aussi compléter les travaux effectués sur OpenMPC [START_REF] Lee | OpenMPC: Extended OpenMP Programming and Tuning for GPUs[END_REF]e ndé plaç an tle sc omm unic ationsdansle graphe d'appels.

Cette approche peut-être comparée à l'algorithme proposé par Alias et al. [START_REF] Alias | [END_REF], Alias et al. 2012b, Alias et al. 2012a], toutefois à un niveau de granularité différent.

Dans un article récent [START_REF] Jablin | [END_REF] Alors que leur travaux sont basés sur l'abstraction sous forme de Quasi-Affine Selection Tree (QUAST),c e t t es e c t i o nm o n t r ec o m m e n tl e u ra l g o r i t h m ep e u tê t r eu t i l i s éa v e cl e s régions de tableaux convexes.

Conclusion

Ave c l ' a u g m e nt a t i o n d e l ' u t i l i s a t i o n d e s a c c é l é r a t e u r s m a t é r i e l s , l a p a r a l l é l i s a t i o n a u t omatique ou semi-automatique assistée par des directives prend une importance croissante. L'impact des communications est critique quand on cible des accélérateurs pour des programmes massivement parallèles telles que les simulations numériques. L'optimisation du placement des données est donc clé pour l'obtention de performance.

Un schéma d'optimisation qui traite cette problématique a été conçu et implémenté dans PIPS et dans Par4All.

L'approche proposée a été validée à l'aide de vingt jeux d'essai de la suite Polybench, trois de la suite Rodinia, et une simulation numérique réelle. La méthode proposée donne des résultats proche d'un placement manuel en terme de nombre de communications effectuées.

Le prochain Chapitre présente les différentes transformations effectuées sur le code séquentiel pour obtenir un code GPU.

4T r a n s f o r m a t i o n s p o u r G P G P U Les contributions de ce chapitre se basent sur les analyses et transformations existantes dans l'environnement de compilation PIPS, étendant certaines pour gérer le langage C, améliorant d'autres pour les besoins spécifiques de la génération de code pour GPU,e t finalement en introduisant de nouvelles transformations quand PIPS ne proposait pas de solution adaptée.

Introduction

Le chemin qui mène du code séquentiel vers du code parallèle efficace pour GPU est pavé de nombreuses analyses et transformations. De plus, certaines spécificités des programmes d'entrée doivent être prises en compte. Par exemple, des programmes écrits par un développeur ne présentent pas les mêmes motifs inclus dans des programmes automatiquement générés depuis des outils ou langages de conception de haut-niveau. Le code de la Le flot de compilation complet pour transformer un code séquentiel en code GPU est présenté à la Figure 2.27 page 58 et identifie les contributions présentées dans ce chapitre. La Section 4.2 explique la manière dont un nid de boucles est placé sur GPU,ainsique la manière dont le tuilage implicite et l'échange de boucles sont mis en oeuvre à l'exécution (voir la Section 2.3.2.2,p a g e32).

J'ai conçu et implémenté une nouvelle transformation de substitution de variables d'induction basée sur l'analyse de préconditions linéaires, présentée à la Section 4.5. Cette transformation peut permettre la parallélisation des boucles qui contiennent des variables d'induction. J'ai étudié la combinaison de deux algorithmes de parallélisation, en analysant l'impact sur la génération de code pour chacun d'entre eux à la Section 4. Finalement, une étape d'après-traitement remplace l'annotation et contracte le nid de boucles dans un pseudo-appel à une macro Call_kernel_xd() avec la dimension x allant de un à trois, pour respecter les contraintes du NDRange OpenCL.L er é s u l t a td el ac o n t r a ction est montré sur la Figure 4.3d. Cette macro masque l'exécution parallèle du noyau sur une grille de l ⇥ m ⇥ n fils d'exécution. La taille de groupement des fils d'exécution sur la grille n'est pas exprimée explicitement est n'est jamais manipulée dans la représentation interne du compilateur, mais construite dynamique à l'exécution selon différents paramètres.

Cette représentation permet de retarder certaines décisions sur des transformations, et donc conserve un code plus indépendant de la cible tout en simplifiant la représentation interne du compilateur.

Détection du parallélisme

La détection du parallélisme est à la base du processus. Cela consiste à prouver qu'une boucle peut être ordonnancé de manière parallèle. De telles techniques sont bien connues dans la communauté de compilation, au moins depuis la méthode des hyperplans proposée par Lamport en 1974[Lamport 1974]. Une autre limitation de cet algorithme réside dans les limitations imposées sur le flot de contrôle du code à traiter, par exemple le corps de boucles ne peut contenir de branchement. L'algorithme introduit à la prochaine section fournit une solution à ce problème en fournissant une parallélisation à gros grain basé sur un résumé de régions de tableaux [Creusillet &I r i g o i n1 9 9 6 b ].

Allen et Kennedy

Parallélisation à gros grain

Le deuxième algorithme de parallélisation est une méthode à gros grain qui repose sur les analyses de régions de tableaux [Creusillet & Irigoin 1996b]. Aucune transformation de boucles n'est mise en oeuvre. Les détails au sujet de cette méthode de parallélisation ont été publiés dans [Irigoin et al. 2011 (perso)]. Le processus est résumé dans cette section.

Cet algorithme est utilisé de manière extensive dans PIPS en raison de sa complémentarité avec l'algorithme d'Allen et Kennedy. En effet les boucles ne sont pas distribuées, et cet algorithme ne comporte pas de restrictions sur le flot de contrôle.

Impact sur la génération de code

Comme montré précédemment, il y a deux algorithmes de parallélisation implémentés dans PIPS.L aF i g u r e4.6 illustre l'impact de l'utilisation d'un algorithme par rapport à l'autre. Alors que Allen et Kennedy distribuent le nid de boucles d'origine dans trois nids de boucles différentes parfaitement imbriquées exprimant un parallélisme à deux dimensions. L'algorithme de parallélisation à gros grain conserve l'imbrication d'origine et détecte la dimension externe comme parallèle. De plus, la dimension parallèle est à l'intérieur d'une boucle séquentielle, ce qui implique m lancements de noyaux. La Section 7.3 présente des expériences sur les algorithmes de parallélisation et montre que la méthode d'Allen et Kennedy aboutit à un code plus efficace sur toutes les architectures testées par rapport à la parallélisation à gros grain. Alors que l'accélération est limitée sur les architectures anciennes tel que le G80, elle est notable sur les GPUs récents et atteint jusqu'à huit sur Fermi et quatre sur Kepler.

Parallélisation des réductions

PIPS inclut un algorithme pour la détection des réductions basé sur un cadre introduit par Jouvelot et Dehbonei [START_REF] Jouvelot | Pierre Jouvelot and Babak Dehbonei. Au n i fi e ds e m a n t i ca pproach for the vectorization and parallelization of generalized reductions[END_REF]. Après détection les réductions peuvent être parallélisées selon les capacités de l'architecture cible. s [a] = s [a]+b++;

dans laquelle on observe le commentaire ajouté par PIPS qui indique que deux réductions ont été détectées.

Parallélisation des réductions pour GPU

La parallélisation des boucles avec réductions peut être être réalisées de différentes façons. J'ai conçu et implémenté une nouvelle méthode qui s'insère dans les algorithmes de parallélisation. L'implémentation dans l'algorithme de parallélisation à gros grain présenté àl aS e c t i o n4.3.2.

La parallélisation à gros grain utilise l'analyse de région de tableaux pour trouver les conflits entre deux itérations d'une boucle. De tels conflits empêchent la parallélisation de la boucle. L'algorithme a été adapté pour gérer les réductions en ignorant les conflits liés aux références impliquées dans la réduction. Si ignorer un conflit élimine tous les cycles, la boucle est alors marquée comme potentiel lement parallèle. Une autre transformation peut potentiellement remplacer la réduction avec une opération équivalente qui permettra l'exécution parallèle de la boucle.

Substitution des variables d'induction

La substitution des variables d'induction est une transformation classique qui permet la parallélisation de certaines boucles. Les variables d'induction sont en général détectées par des boucles qui utilisent de la reconnaissance de motifs basée sur l'initialisation et les mises à jour d'une variable dans le corps de boucles. Cette section indique comment les analyses de préconditions linéaires dans PIPS [START_REF] Irigoin | Polyèdres et Compilation.I nR e ncontres Francophones du Parallélisme (RenPar'20)[END_REF]

Fusion de boucles

La fusion de boucles est une transformation qui consiste à fusionner deux boucles ou plus en une seule. Cette transformation a été très étudiée pendant des décennies [START_REF] Allen | A catalogue of optimizing transformations[END_REF][START_REF] Burstall | [END_REF], Kuck et al. 1981[START_REF][END_REF], Goldberg & Paige 1984[START_REF] Wolfe | [END_REF], Bondhugula et al. 2010]. Trouver une solution optimale à la problématique globale de la fusion est tout sauf trivial [Darte 2000]etilyaplusieursmanières de traiter ce problème, tout comme il y a différentes définitions du problème lui-même.

Cette transformation peut réduire la surcharge induite par le branchement et l'incrémentation en éliminant des en-têtes de boucles et en augmentant la taille du corps de boucle. Cependant la pression sur le cache d'instruction et sur l'utilisation des registres dans le corps de la boucle est plus forte.

Légalité

La fusion de boucles n'est pas toujours valide et peut modifier la sémantique du programme. Une fusion invalide peut aboutir à l'ordre inverse des calculs dépendants. L'analyse de dépendance de données est utilisée pour déterminer quand la fusion est valide ou pas.

La validité de la fusion de boucles a été largement étudiée [START_REF] Allen | A catalogue of optimizing transformations[END_REF], Warren 1984, Aho et al. 1986], mais peut être exprimée de différentes manières. [START_REF] Kennedy | [END_REF] Cette section présente les différents objectifs de la fusion de boucles dans les travaux précédents, de la réduction de consommation d'énergie à une meilleure utilisation de registres vectoriels, puis présente l'intérêt de cette transformation dans le contexte de la pratique du GPGPU. C'est à dire la réduction du nombre de noyaux exécutés et l'augmentation de leur taille, la réutilisation de données, et la contraction des tableaux. L'algorithme proposé et implémenté dans PIPS est ensuite détaillé, puis une alternative basée sur les régions de tableaux est introduite. Enfin des considérations particulières liées à la génération de code pour accélérateurs sont abordées.

Remplacement des tableaux par des scalaires

Le remplacement de tableaux par des scalaires remplace les références constantes à des tableaux à l'aide de scalaires. Cette transformation peut être rapprochée de qui est réalisé dans la phase d'allocation des registres du compilateur chargé de générer le binaire final, quand il s'agit de conserver dans un registre une valeur mémoire le plus longtemps possible. Intuitivement, cela signifie qu'effectuer cette transformation au niveau du code source peut augmenter la pression sur les registres et aboutir à la sauvegarde de registres en mémoire.

Cette transformation peut aussi éliminer les tableaux temporaires, la plupart du temps après la fusion de boucles et particulièrement dans le contexte de codes générés automatiquement à partir de représentations de plus haut-niveaux. La Figure 4.21 indique comment trois lignes d'un script Scilab aboutissent à un trois tableaux temporaires qui peuvent être totalement remplacés par des scalaires après fusion de boucles.

Dans le contexte des accélérateurs, cette transformation est encore plus critique que sur un système à mémoires partagées. En effet les noyaux générés seront plus rapides en effectuant moins d'accès mémoires, mais la source principale de gain sera probablement l'élimination de transferts sur le bus PCIe.

Le remplacement de tableaux par des scalaires a été largement étudié [START_REF] Gao | [END_REF][START_REF] Sarkar | [END_REF][START_REF] Darte | New Results on Array Contraction[END_REF][START_REF] Carribault | Applications of storage mapping optimization to register promotion[END_REF]. Cette section explore les différentes méthodes pour appliquer cette transformation dans le contexte du déport de noyaux vers un GPU. L'impact sur la performance est évalué pour différentes architectures de GPU.

A l'intérieur du noyau

Une simple multiplication de matrices naïvement exécutée sur GPU est montrée à la Figure 4.22. Ce noyau inclut une boucle séquentiel avec une référence constante à un tableau. Cette référence peut être conservée dans une variable scalaire pendant l'exécution complète de la boucle. Cette transformation peut être réalisée par le compilateur qui génère le binaire. Les mesures présentées à la Figure 7.11,p a g e192, indiquent que l'effectuer au niveau du source est intéressant sur toutes les architectures testées, avec une accélération jusqu'à 2,39.

Après la fusion de boucles

La fusion de boucles génère un code dans lequel les instructions qui produisent et utilisent un tableau temporaire sont autant que possible dans le même corps de boucles. Certain de ces tableaux peuvent alors être complètement supprimés, économisant la bande passante et l'empreinte mémoire. Dans le contexte de code automatiquement généré depuis des représentations de haut-niveau, cette situation est courante. La Figure 4.21 montre un exemple d'un code généré depuis un script de trois lignes de Scilab. Après la fusion de boucles, le code C généré contient trois tableaux temporaires qui peuvent être remplacés par des scalaire comme illustré sur la Figure 4.21b.

Pour éliminer un tableau temporaire, ses éléments ne doivent pas être utilisés plus tard dans l'exécution du programme. PIPS fournit l'information de régions vivantes (OUT ,voir Section 3.2,p a g e64).

La Section 7.5.2,p a g e191,m o n t r ec o m m e n tc es i m p l ee x e m p l ea bo u t i tau n ea c c é l é r ation de 1,96 à 5,75 selon l'architecture. Contrairement à l'exemple précédent, dans ce cas le compilateur qui génère le binaire du noyau est démuni et il est donc critique d'effectuer cette transformation au niveau source.

Perfect Nesting of Loops

Dans certains cas, cette transformation peut casser l'imbrication parfaite des boucles. La Figure 4.23 illustre une telle situation. Les références constantes dans la boucle la plus interne sont affectées à des scalaires entre les deux boucles qui ne sont donc plus parfaitement imbriquées. Puisque seules les boucles parfaitement imbriquées sont exécutées en parallèle, ici seule l'une des deux boucles pourra être exécutée en parallèle après transformation. Le parallélisme est réduit : peu de fils d'exécution peuvent s'exécuter sur le GPU.

L'impact sur la performance pour du code CUDA est évalué à la Section 7.7. Cette transformation peut aboutir à un ralentissement jusqu'à vingt pour cent mais également, dans une seule configuration, aboutir à une petite accélération de deux pour cent.

Il n'y a pas de raison selon mon opinion qu'un standard tel que OpenCL interdise l'utilisation de tableaux multi-dimensionnels dans les paramètres formels des noyaux, et, considérant l'impact sur les performances, nous espérons qu'une version future supprimera cette contrainte.

Vers une chaîne de compilation

Ce chapitre présente plusieurs transformations individuelles qui sont applicables à différent moment du processus complet. L'enchaînement de toutes ces transformations peut être délicat et différent choix dans le processus aboutiront à différents résultats en terme de performance. J'ai proposé et implémenté une méthode de mise en correspondance simple mais flexible des nids de boucles sur le GPU. Cette méthode permet de conserver une représentation interne simple et ignore la hiérarchie complète induite par l'utilisation de NDRange OpenCL. J'ai conçu et implémenté une phase de substitution de variable d'induction basée sur une méthode originale utilisant des préconditions linéaires. J'ai amélioré la détection du parallélisme dans PIPS,p a r t i c u l i è r e m e n tl ec o u p l a g ea v e cl ad é t e c t i o nd e sr é d u c t i o n s ,e tj ' a i étudié l'impact de la combinaison de deux algorithmes différents sur la génération du code et la performance obtenue lors de l'exécution sur GPU.J'aiconçuetimplémentéunephase de parallélisation dédiée aux réductions en utilisant les opérations disponibles sur GPU. J'ai conçu et implémenté une phase de fusion de boucles dans PIPS,incluantplusieursheuristiques qui favorisent la mise en oeuvre sur GPU.J'aiaméliorélaphasedecon tractionde tableaux existante dans PIPS pour garder l'imbrication parfaite de boucles. J'ai identifié trois motifs différents et analysé l'impact de la scalarisation sur ces motifs. J'ai conduit des expériences pour valider l'impact individuel de chaque transformation présentée dans ce chapitre. Alors que la plupart des concepts sont bien connus, il est montré que pour plusieurs transformations, la méthode de l'état de l'art doit être adaptée pour répondre parfaitement aux contraintes des circuits massivement parallèles des GPUs.

PIPS offre un cadre de compilation flexible, mais le flot de compilation doit être piloté, comme indiqué sur la Figure 2.27 de la page 58.Leprochainchapitremotiveetintroduitles concepts d'un gestionnaire de passes programmable, implémenté dans PIPS,e ts u rl e q u e l Par4All construit le processus qui automatise les étapes de transformation. Alors que les plate-formes matérielles grandissent en complexité, les infrastructures de compilation ont besoin de plus de flexibilité : à cause de l'hétérogénéité des plate-formes, les phases de compilation doivent être combinées de manière inhabituelle et dynamique, et plusieurs outils doivent être combinés pour gérer efficacement des parties spécifiques du processus de compilation. Le besoin de flexibilité apparaît également lors de la mise en oeuvre de processus itératif de compilation, quand différentes combinaisons de phases de transformation sont explorées.

Dans ce contexte, il est nécessaire d'assembler des composants logiciels telles que les phases de compilation, sans avoir à plonger dans les arcanes des différents outils. L'entité en charge de la combinaison des phases de compilation est le gestionnaire de phases dans un compilateur monolithique classique. Alors que les gestionnaires de phases se basent habituellement sur un ordonnancement statique, l'introduction de plug-ins dans GCC et les tendances actuelles dans la conception des compilateurs ouvrent la voie des gestionnaires de phases dynamiques. De plus, la combinaison de différents outils dans la chaîne de compilation est rendue obligatoire par l'hétérogénéité des cibles. Dans ce contexte, l'automatisation de la collaboration d'outils si différents implique de définir un méta-gestionnaire de phases de haut-niveau.

L'aspect source-à-source est clé dans ce processus, comme expliqué dans la Section 5.2. Un gestionnaire de phases programmable est alors introduit à la Section 5.3.

Les simulations numériques font souvent appel à des bibliothèques de fonctions externes telles que Basic Linear Algebra Subprograms (BLAS) ou Fast Fourier transform (FFT) par exemple. La Section 5.4 présente comment l'utilisation de telles bibliothèques est exploitée pour déporter les calculs sur un GPU. Finalement, la Section 5.5 propose un aperçu sur la manière dont différents outils Alors que le code qui exploite des cibles matérielles hétérogènes est généré depuis plusieurs sources, une étape finale de compilation est requise pour obtenir le ou les binaires finaux. Ce n'est pas à proprement parler le rôle du gestionnaire de phases, toutefois comme pièce centrale il possède l'information précise requise pour piloter ce processus et les différents outils de construction du binaire à mettre en oeuvre.

Travaux liés

Dans les compilateurs traditionnels tel que GNU C Compiler (GCC),u np r o c e s s u sd e compilation est simplement une séquence de transformations chaînées les unes après les autres et appliquées de manière itérative sur chaque fonction du code source d'origine [Wikibooks 2009]. Ce comportement rigide a mené au développement d'un mécanisme de motivé par la difficulté à fournir des fonctionnalités additionnelles à l'infrastructure existante. Des exemples de plug-ins à succès incluent "dragonegg" et "graphite." Certaines limitations de GCC ont conduit au développement de Low Level Virtual Machine (LLVM) [Lattner &A d v e2 0 0 4 ], qui traite le manque de flexibilité en fournissant un gestionnaire de phases plus élaboré qui peut modifier le chaînage des phases à l'exécution.

Le compilateur source-à-source ROSE [Quinlan 2000, Schordan & Quinlan 2003]n ' i nclut pas de gestionnaire de phases particulier, mais fournit une interface C++ et Haskell. Le langage Poet [START_REF] Yi | Automated Programmable Control and Parameterization of Compiler Optimizations[END_REF]] est conçu pour construire des compilateurs, mais souffre de certaines limitations, comme il n'y a pas de séparation claire des principes entre les interfaces internes du compilateur le gestionnaire de phases.

Les directives de compilation offrent un moyen non-intrusif d'intervenir sur le processus de compilation. Elles sont intensivement utilisées dans plusieurs projets [Kusano & Sato 1999, Donadio et al. 2006[START_REF] Nvidia | NVIDIA, Cray, PGI, CAPS. The OpenACC Specification, version 1.0[END_REF]p o u rg é n é r e rd uc o d e efficace qui cible des multicoeurs, des GPUs,oudesjeuxd'instructionvectorielsIlspeuvent spécifier un ordre séquentiel des phases, mais ils ne fournissent pas de contrôle supplémentaire, et peuvent avoir une sémantique de composition complexe.

Conclusion

Cette section introduit PyPS,uneinterfacedeprogrammationdegestionnairedephases pour l'infrastructure de compilation source-à-source PIPS. Cette infrastructure de compilation est réutilisable, flexible, et maintenable. Ce sont les trois propriétés requises dans le développement de nouveaux compilateurs à bas coût tout en respectant des contraintes de délai de mise sur le marché. Ceci est possible grâce à une séparation claire des concepts entre les interfaces internes, les phases de compilation, le gestionnaire de cohérence, et le gestionnaire de phases. Cette séparation claire n'est pas présente dans GCC et n'est pas complètement mis en oeuvre non plus dans les compilateurs de recherche, bien que la problématique de la gestion des phases devient un sujet de recherche actif. Cinq compilateurs spécifiques et d'autres outils annexes sont implémentés avec cette interface de programmation : de générateurs OpenMP, SSE, SCMP, et CUDA/OpenCL,a i n s iq u ' u nc o m p i l a t e u r itératif optimisant.

Les travaux présentés dans cette section ont été publiés précédemment dans [Guelton et al. 2011a (perso), Guelton et al. 2011b (perso)].

Traitement des bibliothèques externes

Les développeurs font souvent usage de bibliothèques externes, hautement optimisées pour des traitements particuliers, comme de l'algèbre linéaire, du traitement d'image, du traitement du signal, etc. Le problème, dans le contexte de la parallélisation automatique de tels programmes, est qu'en général le code source de la bibliothèque n'est pas disponible soit parce que le code est propriétaire, ou simplement car le processus de compilation normal ne requiert que le binaire installé sur la machine. Même si le code était disponible, ça pourrait représenter une énorme quantité de code à traiter. De plus le code serait probablement tellement optimisé pour une architecture de processeur généraliste qu'une transformation automatique pour GPU ou autre accélérateur semble utopique.

PIPS est un compilateur interprocédural, il analyse le graphe d'appel complet et propage les résultats des analyses aux sites d'appel. Une limitation actuelle de l'implémentation est que les fonctions de tout le graphe d'appel doivent être disponibles pour le traitement. La conséquence principal est qu'un code qui contient des appels à des bibliothèques ne peut être traité par PIPS. D'un autre côté, plusieurs bibliothèques sont disponibles pour les GPUs et fournissent des fonctionnalités similaires, voir équivalentes. Cette section couvre la manière dont la gestion des bibliothèques est réalisée dans la chaîne de compilation. D'abord, une méthode automatique pour fournir des fonctions mimes (stubs)àlademandeaucompilateurestprésentée à la Section 5.4.1. Ces différentes techniques sont utilisées pour remplacer les appels au code de bibliothèque CPU par l'équivalent GPU,c o m m ei n t r od u i tàl aS e c t i o n5.4.2.

Fournisseur de fonctions mimes

En tant que compilateur purement interprocédural, PIPS arrête son traitement dès qu'il a besoin du résultat d'analyse d'une fonction pour laquelle le code source n'est pas disponible. Des fonctions mimes sont utilisées pour éviter le problème. Ce sont des fonctions qui possèdent la même signature et miment les effets de l'appel à la bibliothèque sur la mémoire, puisque PIPS utilise cette information, à fin de parallélisation par exemple.

Dans le contexte de l'outil Par4All, le but est de fournir une expérience décousue à l'utilisateur. Le développeur écrit son propre code et le fournit au compilateur, avec un paramètre qui indique la bibliothèque utilisée. Par4All inclut un mécanisme générique pour gérer les bibliothèques, le fournisseur de fonctions mimes, qui est invoqué par plusieurs étapes du processus de compilation.

Le flot qui met en oeuvre le fournisseur de fonctions mimes est présenté sur la Figure 5.10. Ce processus a été validée avec l'implémentation d'un compilation Scilab-to-C (Wild Cruncher de SILKAN). Les fonctions mimes pour le support exécutif fourni par ce compilateur sont produites automatiquement à partir d'une description de plus haut niveau. Le code résultant fait appel à des bibliothèques internes de Scilab, par exemple 306 Résumé en français pour l'affichage, ou pour des calculs optimisés.

Gérer plusieurs implémentations d'une API : gestion des bibliothèques externes

Le fournisseur de fonctions mimes n'est en fait pas en charge d'une bibliothèque donnée mais d'une fonctionnalité donnée. Par exemple, un fournisseur peut être en charge de la FFT. Cette organisation est présentée par la Figure 5.10. L'exemple Stars-PM [Aubert et al. 2009 (perso)]p r é s e n t ée nd é t a i làl aS e c t i o n3.1 utilise la bibliothèque FFTW [START_REF] Frigo | [END_REF]. La FFT est effectué lors de la troisième étape montrée sur la Figure 3.3,p a g e65.L e sa p p e l sàl ab i b l i o t h è q u eFFTW correspondants sont montrés sur la Figure 5.11.

Le fournisseur de fonctions mimes dédié à la FFT fournit alors automatiquement à PIPS quand il se plaint des fonctions manquantes de la bibiolthèque FFT,commeexpliquédans la section précédente. A la fin du processus de compilation source-à-source, Par4All ajoute les sources et/ou les paramètres indiquées par le fournisseur à l'étape finale de génération du code. Par exemple un paramètre peut être ajouté à l'édition des liens, comme -lcufft pour le remplacement la bibliothèque FFTWsurGPU,etdessourcespeuv en têtreajoutés pour faire l'interface.

Combinaisons d'outils

Ave c l ' h é t é r o g é n é i t é c r o i s s a nt e , i l e s t m o i n s f r é q u e nt d ' avo i r u n e i n f r a s t r u c t u r e d e c o mpilation qui répond à tous les besoins. Certaines infrastructures peuvent avoir une représentation interne qui intègre mieux certaines optimisations particulières, mais pas toutes. Par exemple, PIPS bénéficie d'analyses interprocédurales, mais n'a pas certaines capacités que d'autres outils tels que PoCC [Pouchet et al. 2010a], Pluto [Bondhugula et al. 2008b], ou PPCG [Verdo olaege et al. 2013] ont. Cet optimiseurs sont dédiés au modèle polyédrique, et donc restreints à une catégorie de programme en particulier.

Des expériences ont été menées dans Par4All pour exploiter PoCC [Pouchet et al. 2010a] àcertainniv eauduprocessus.PIPS détecte les parties à contrôle statique [START_REF] Feautrier | Dataflow analysis of array and scalar references[END_REF]], et les extrait pour être traitée par un outil polyédrique.

Pour effectuer de telles combinaisons d'outils, le source semble un niveau intéressant de représentation intermédiaire. Et le langage C, avec des directives, a fait ses preuves comme présenté à la Section 5.2,p a g e140.

Critères de profitabilité

Décider de l'intérêt de déporter un calcul sur le GPU n'est pas un problème simple.Deux sur deux processeurs à six coeurs.

Des travaux préliminaires au sujet d'une exécution distribuée sur plusieurs GPUs sont présentés, et les mesures montrent que des opportunités d'accélération existent déjà. Les travaux futurs dans ce domaine sont prometteurs.

8C o n c l u s i o n

Bienvenue dans la jungle matérielle 13 . Dix ans après l'écriture de cette affirmation par Herb Sutter [START_REF] Sutter | Welcome to the Jungle[END_REF]], elle touche plus que jamais une problématique brûlante. Des téléphones mobiles dans chaque poche jusqu'aux super-calculateurs, l'exploitation du matériel représente un défi croissant pour les développeurs.

Il n'y a pas de solution satisfaisante qui a été proposée pour les trois "P "p r o p r i é t é s: Performance, Portabilité, et Programmabilité.

La jungle actuelle dans l'écosystème des circuits matériels est maintenant reflétée dans le monde du logiciel, avec de plus en plus de modèles de programmation, de nouveaux langages, différentes interfaces de programmation, etc. Cependant aucune solution universelle n'a encore émergée. J'ai conçu et implémenté une solution automatique basée sur un compilateur pour répondre en partie au problème. La programmabilité et la portabilité sont assurées par définition, et si la performance n'est pas toujours au niveau de ce qu'obtiendrait un développeur expert, elle reste excellente sur une large gamme de noyaux et d'applications. J'ai obtenu une accélération de cinquante-deux pour une simulation numérique n-corps par rapport au code séquentiel compilé avec GCC.

Ce travail explore les problèmes associés avec une solution entièrement automatique, traitant la problématique des trois P se nn ec h a n g e a n tn il em od è l ed ep r o g r a m m a t i o n ,n i le langage, mais en implémentant des transformations avancées et un support exécutif. La portabilité et la programmabilité sont évidement atteintes : le développeur n'a pas besoin d'avoir une bonne connaissance de la plate-forme sous-jacente.

Le compromis sur les performances est limité, comme illustré par les expériences du Chapitre 7. Elles montrent également que la portabilité des performances peut être atteinte sur de multiples architectures. Abstract: Since the beginning of the 2000s, the raw performance of processors stopped its exponential increase. The modern graphic processing units (GPUs) have been designed as array of hundreds or thousands of compute units. The GPUs' compute capacity quickly leads them to be diverted from their original target to be used as accelerators for general purpose computation. However programming a GPU efficiently to perform other computations than 3D rendering remains challenging. The current jungle in the hardware ecosystem is mirrored by the software world, with more and more programming models, new languages, different APIs, etc. But no one-fits-all solution has emerged.

This thesis proposes a compiler-based solution to partially answer the three "P" properties: Performance, Portability, and Programmability. The goal is to transform automatically a sequential program into an equivalent program accelerated with a GPU. A prototype, Par4All, is implemented and validated with numerous experiences. The programmability and portability are enforced by definition, and the performance may not be as good as what can be obtained by an expert programmer, but still has been measured excellent for a wide range of kernels and applications.

A survey of the GPU architectures and the trends in the languages and framework design is presented. The data movement between the host and the accelerator is managed without involving the developer. An algorithm is proposed to optimize the communication by sending data to the GPU as early as possible and keeping them on the GPU as long as they are not required by the host. Loop transformations techniques for kernel code generation are involved, and even well-known ones have to be adapted to match specific GPU constraints. They are combined in a coherent and flexible way and dynamically scheduled within the compilation process of an interprocedural compiler. Some preliminary work is presented about the extension of the approach toward multiple GPUs.

Keywords: GPU, CUDA, OpenCL, Automatic Parallelization, Compilation

1. 1 von 3 1. 2 4 1. 3 5 1. 4

 1324354 Neumann architecture ........................... More than three decades of prosperity, the misquoted Moore's law (source [Sutter 2005], updated 2009, reprinted here with the kind permission of the author). ................................. The free lunch is over. Now welcome to the hardware jungle (source [Sutter 2011], reprinted here with the kind permission of the author). ..... The three P properties. ............................. 6 2.1 Performance evolution for single-precision floating point computation, for both Nvidia GPUs and Intel CPUs between 2003 and 2012, computed from vendors' datasheets. ............................... 14 2.2 Example of a saxpy OpenGL 4.4 compute shader (adapted from [Kilgard 2012]). ................................... 16 2.

2. 27 6

 276 Overview of the global compilation scheme. .................. 3.1 Stars-PM is a N -body cosmological simulation. Here a satellite triggers a bar and spiral arms in a galactic disc. ..................... 3.2 Simplified global scheme commonly used in numerical simulations. .... 3.3 Outline of one time step in the Stars-PM cosmological simulation code. ..3.4 Array regions on a code with a function call. ................. 3.5 Array regions on a code with a while loop. .................. 3.6 Array regions on a code with a switch case. ................. 3.7 Basic process for mapping data to the accelerator (source [Yan et al. 2009], c 2011 Springer-Verlag)g. ........................... 3.8 Sequential source code for function discretization,t h efi r s ts t e po fe a c h Stars-PM simulation main iteration. ...................... xx List of Figures 3.9 Code for function discretization after automatic GPU code generation. . 3.10 Isolation of the irregular while loop from Figure 3.5 using array region analysis. ....................................... 3.11 Code with a switch case from Figure 3.6 after isolation. ........... 3.12 Interprocedural isolation of the outermost loop of a Finite Impulse Response. 3.13 Isolated version of the KERNEL function of the Finite Impulse Response (see Figure 3.4). ................................... 3.14 Bandwidth for memory transfers over the PCI-Express 2.0 bus as a function of block size. Results are shown for transfers from the host to the GPU (H-TO-D) and in the opposite direction (D-TO-H), each for pinned or standard allocated memory. ................................ 3.15 Illustration of set construction using the intraprocedural analysis on the function iteration. The different calls to step functions use and produce data on the GPU via kernel calls. Sometimes in the main loop, array a is read to display or to checkpoint. The interprocedural translation exploits at call site the summary computed on function iteration.Afi xp o i n ti s sought on the loop. ............................... 3.16 Simplified code for functions discretization and main after interprocedural communication optimization. .......................... 3.17 gramschmidt example taken from Polybench suite. The first part of the loop body is sequential while the following are parallel loop nests. The sequential promotion on the GPU avoids costly memory transfers. ........... 3.18 Illustration of the redundant load-store elimination algorithm. ....... 3.19 Code with communication for FIR function presented in Figure 3.4. .... 4.1 Example of a short Scilab program with the generated C file. ........ 4.2 Example from Baghdadi et al. [Baghdadi et al. 2010]thatillustratesho wto tile a loop nest to map the GPU execution. .................. 4.3 Illustration of the successive steps performed to map a loop nest on the GPU.100 4.4 The iteration set is over-approximated with a rectangular hull; a guard is added to clamp it. ............................... 4.5 Example of Allen and Kennedy algorithm as implemented in PIPS: loops are distributed and parallelism is expressed using OpenMP pragmas. ... The impact of the two parallelization schemes on a example of code performing a correlation. Allen and Kennedy algorithm results to three different parallel loop nests expressing the maximum parallelism, while the coarse grained algorithm detects only one parallel loop leading to less synchronization but also less exposed parallelism. ..................... 4.7 Example of reduction detection and interprocedural propagation in PIPS. . 4.8 An example of reduction parallelization of an histogram using hardware atomic operations. ............................... 4.9 Example of induction variable substitution to enable loop nest parallelization.112 4.10 Example of loop fusion. ............................. 4.11 Example of two parallel loops that can be legally fused, but the resulting loop nest would be sequential. ......................... 4.

4. 16 2

 162 Heuristic algorithm FUSE_RDG to traverse the RDG and apply fusion. The graph is modified as side effect. ........................ 4.17 Merging two vertices in the graph while enforcing pruning as introduced in Figure 4.15. ................................... 4.18 The resulting code after applying the loop-fusion algorithm on the code presented in Figure 4.14a. ........................... 4.19 Sample code showing that inner loops have to be fused first in order to be able to fuse the outer loops without breaking the perfect nesting. ..... Code generated automatically with pragmas to drive the StarPU runtime.

6. 4

 4 Tiling transformation process illustrated on a simple vector scaling multiplication example. ................................ 7.1 Kernel execution times in ms (best over twenty runs) and speedups between the Coarse Grained and the Allen and Kennedy parallelization algorithms using CUDA for different Nvidia GPUs. The example used here is the correlation loop nest shown in Figure 4.6 with m =3000. ......... 7.2 Impact of work-group size on GT200 and Fermi architectures, speedup normalized with respect to a size of 512 work-items per work-group. The CUDA API is used in this experiment. ........................ 7.3 Impact of block dimensionality on performance for different block sizess expressed in µs for G80 architecture. The reference kernel here is the main loop nest from syrk (Polybench suite) shown in Figure 5.14 page 163. .. 7.4 Impact of block dimensionality on performance for different block sizes expressed in µs for GT200 architecture. The reference kernel here is the main loop nest from syrk (Polybench suite) shown in Figure 5.14 page 163. .. 7.5 Impact of block dimensionality on performance for different block sizes expressed in µs for Fermi architecture. The reference kernel here is the main loop nest from syrk (Polybench suite) shown in Figure 5.14 page 163. .. 7.6 Impact of block dimensionality on performance for different block sizes expressed in µs for Kepler architecture. The reference kernel here is the main loop nest from syrk (Polybench suite) shown in Figure 5.14 page 163. .. 7.7 Impact of block dimensionality on performance for different block sizes expressed in µs for G80 architecture. The reference kernel is the matmul example shown in Figure 4.22 page 129. .................... 7.8 Impact of block dimensionality on performance for different block sizes expressed in µs for GT200 architecture. The reference kernel is the matmul example shown in Figure 4.22 page 129. ................... xxiv List of Figures 7.9 Impact of block dimensionality on performance for different block sizes expressed in µs for Fermi architecture. The reference kernel is the matmul example shown in Figure 4.22 page 129. ................... 189 7.10 Impact of block dimensionality on performance for different block sizes expressed in µs for Kepler architecture. The reference kernel is the matmul example shown in Figure 4.22 page 129. ................... 190 7.11 Execution times in ms (best over twenty runs) and speedups for scalarization using CUDA and OpenCL for different AMD and Nvidia GPUs.

  Figure 4.21b, page 129. The execution times shown are the kernel execution times. The GTX 8800 results are given aside because they are one order of magnitude slower. This architecture does not perform double precision computations: doubles are rounded to float before being processed. Hence, no OpenCL results are available in double precision for this GPU. ........... 193 7.13 Execution times in µs (best over twenty runs) for the code shown in Figure 4.23c and Figure 4.23d. The value of M is 8192 and the values of N are on the x axis. The Nvidia board shows clearly a shift on the red curve for N =32corresponding to the the warp size. ................. 194 7.14 Execution time in µs (best over twenty runs) and speedup for different unrolling factor using CUDA and OpenCL for different AMD and Nvidia GPUs. The execution times are the kernel execution times. Single precision floating point is used. The example is the code presented in Figure 4.22, page 129,usedintheprevioussection. Loopunrollingisappliedafterscalarization to obtain the code shown in Figure 4.24,p a g e133. The GTX 8800 results are given aside because they are one order of magnitude slower. .. 196 List of Figures xxv 7.15 Register counts for different Nvidia architectures and different unrolling factors. ....................................... 7.16 Kernel execution times in ms (best over twenty runs) and speedups for the array linearization transformation and different Nvidia GPUs, and with and without the scalarization illustrated in Figure 7.11. The example is the matmul kernel shown in Figure 4.22,p a g e129, with n i = n j = n k =2048. . 7.17 Execution times and speedups for versions of hotspot on GPU, with different iteration counts. ................................. 7.18 Illustration for the code used to measure performance of the static approach on a Jacobi 2D scheme. ............................. 7.19 Illustration for the code used to measure performance for the StarPU version of the Jacobi 2D scheme. ............................ 7.20 durbin example from Polybench that shows the interest of sequential promotion. ..................................... 7.21 Execution times in ms and speedups for CUDA execution with communication optimization, using the classic scheme and the sequential promotion. The result are based on the average over five runs for durbin and gramschmidt examples (see Figures 7.20 and 3.17). ..................... 7.22 Speedup relative to naive sequential version for an OpenMP version on the CPU, a version with basic PGI Accelerator and HMPP directives, a naive CUDA version, and an optimized CUDA version, all automatically generated from the naive sequential code. ........................ 7.23 bicg example from Polybench that shows the impact of the different StarPU schedulers on a sequential example. There is a direct dependence between each of the kernels. Here nx = ny =8 0 0 0 ,t h u st h ekernel3() is executed 8000 times. ................................... 7.24 Average execution time in ms over ten runs for 3mm and bicg examples. Note the impact of different StarPU schedulers, the default greedy one and the data-aware, dmda. ................................ 7.25 The vector scaling example presented in Figure 6.4,modifiedtoincreasethe computation time. ............................... Prophecy ............................... 1.2 Motivation ................................ 1.3 Outline ................................... 2C h a p t e r 1 . I n t r o d u c t i o n

Figure 1 . 1 :

 11 Figure 1.1: von Neumann architecture

Figure 1 . 2 :

 12 Figure 1.2: More than three decades of prosperity, the misquoted Moore's law (source [Sutter 2005], updated 2009, reprinted here with the kind permission of the author).

Figure 1 . 3 :

 13 Figure 1.3: The free lunch is over. Now welcome to the hardware jungle (source [Sutter 2011], reprinted here with the kind permission of the author).

Figure 1 . 4 :

 14 Figure 1.4: The three P properties.

Figure 2 . 1 :

 21 Figure 2.1: Performance evolution for single-precision floating point computation, for both Nvidia GPUs and Intel CPUs between 2003 and 2012, computed from vendors' datasheets.

Figure 2 . 2 :

 22 Figure 2.2: Example of a saxpy OpenGL 4.4 compute shader (adapted from [Kilgard 2012]).

Figure 2 . 3 :

 23 Figure 2.3: Example of a trivial pass-through GLSL geometry shader, which emits a vertex directly for each input vertex (source wikipedia [Wikipedia 2012b]).

Figure 2 . 4 :

 24 Figure 2.4: Example of a simple saxpy using BrookGPU (taken from[Buck et al. 2004]).

Figure 2 . 6 :

 26 Figure 2.6: Rewriting a C++ computation using C++ AMP. The example shows the use of a lambda function and a parallel_for_each construct to express the parallelism (source Microsoft [Microsoft Corporation 2012b]).

Figure 2 .

 2 Figure 2.7: A sample matrix multiplication code with hiCUDA directives (source [Han & Abdelrahman 2009]).

Figure 2 . 9 :

 29 Figure 2.9: Example of a PGI Accelerator code using data movement optimization (source PGI Insider [Wolfe 2011]).

  and 2.12. The host entry point is the platform. It represents the vendor implementation. The host program sees as many platforms as there are vendor runtimes in the machine. After selecting one or several platforms, the host program can query a list of devices available for this platform. A device is defined in the OpenCL standard as ac o l l e c t i o no fc o m p u t eu n i t s . [ . . . ] O p e n C Ld e v i c e s Chapter 2. General-Purpose Processing on GPU : History and Context

Figure 2 . 11 :

 211 Figure 2.11: Simplified view of the OpenCL abstraction model. A host is connected to multiple devices (GPUs, FPGAs, DPSs, . . .). OpenCL platforms are vendors' implementations that target some types of devices. A context is created for a given platform and a set of devices. Memory objects and events are created context-wise. Devices are then controlled in a given context using command queues. There can be multiple command queues per device, and a device can be associated with queues from multiple contexts and platforms.

Figure 2 .

 2 Figure 2.12: UML representation of the OpenCL abstraction model (see Figure 2.11)taken from the Standard [Khronos OpenCL Working Group 2011].

Figure 2 .

 2 Figure 2.13 shows how a simple two-dimensional parallel loop nest can be mapped onto an OpenCL index set.

Figure 2 .

 2 Figure 2.13: A mapping example of a two-dimensional loop nest iteration set into an OpenCL index range. The mapping is the simplest possible; one work-item executes one iteration of the original loop nest. The work-group size used as an illustration on figure c is a two-dimensional square with an edge of five. Values for get_global_id() and get_local_id() OpenCL primitives are exhibited for a particular work-group.

Figure 2 .

 2 Figure 2.14: Visual example of the OpenCL memory model. Two possible mappings are illustrated: data caches are optional, and private, local, and constant memories are not necessarily dedicated. On the right the simplest mapping, for instance a CPU, merges all memory spaces onto the same piece of hardware.

Figure 2 .

 2 Figure 2.15: High-level simplified GPGPU-oriented view of generic GPU architecture.

Figure 2 .

 2 Figure 2.16: Instruction Level Parallelism (ILP) versus Thread Level Parallelism (TPL), two different ways of extracting parallelism in GPUs.

Figure 2 .

 2 Figure 2.18: Table summarizing the ALU occupation and the VLIW packing ratio for some computing kernels, taken from [Zhang et al. 2011b](c 2011 IEEE).

Figure 2 .

 2 Figure 2.19: The 2012 AMD architecture Graphics Core Next. No longer VLIW, the four separate SIMD pipelines are independent. A new integer scalar unit is introduced. The scheduler feeds each SIMD every four cycles (one per cycle) with a 64-wide virtual SIMD instruction.

Figure 2 .

 2 Figure 2.20 summarizes the evolution of PE grouping across AMD architectures. AMD has later released this information about GCN in a white paper [AMD 2012].

Figure 2 .

 2 Figure 2.20: Evolution of Processing Element (PE) grouping across AMD architectures.

2. 4 . 5

 45 Nvidia Computing Unified Device Architecture, from G80 to Kepler In the fall of 2006, Nvidia released the G80. It was the first DirectX 10 compliant GPU.

Figure 2 .

 2 Figure2.21: The GeForce 8800 architecture (G80) introduced unified shaders where shader programmable processors can be used to replace multiple stages of the classic graphic pipeline. There are still specialized units for some graphics operations. (Source:Nvidia)

Figure 2 .

 2 Figure 2.22: GT200 compute unit (CU) on the left, Fermi's on the right. Processing Elements (PE) upgrade from eight to sixteen per pipeline, but the logical SIMD width is unchanged, threads are scheduled by groups of thirty-two (source Nvidia).

Figure 2 .

 2 Figure 2.23: Influence of runtime parameters on performance for different kernels and different architectures, Nvidia Fermi and AMD Evergreen. On the left the launch configuration for different kernels shows that there is no universal work-group size. On the right a matrix multiply kernel without local data store optimization is used with one to four elements processed in each thread. The upper part shows the impact on performance for both architectures while the lower part shows the occupancy of the AMD GPU and the VLIW packing ratio. Taken from [Zhang et al. 2011b](c 2011 IEEE).

Figure 2 .

 2 Figure 2.24: Performance of two different versions of matrix multiply kernel, a horizontal scheme and a vertical scheme, without local memory usage for a given architecture (Fermi), depending on the input size and the activation or not of the L1 cache. Taken from [Zhang et al. 2011b](c 2011 IEEE).

Figure 2 .

 2 Figure 2.25: Google trends for the word GPU during last decade.

Figure 2 .

 2 Figure 2.26: Source-to-source compilation scheme for GPU (source [Guelton 2011a]).

Figure 2 .

 2 Figure 2.27 illustrates the compilation chain. The source code is first analyzed to find parallelism, and transformed before extracting the code to be executed on the GPU in new functions: the kernels.S o m eo p t i m i z a t i o np h a s e sc a nb ea p p l i e ds u c ha sl o o pf u s i o n ,a r r a y scalarization, array linearization, symbolic tiling, or unrolling. This part of the process is presented in Chapter 4.A f t e rk e r n e lg e n e r a t i o n ,a n a l y s e sa n dt r a n s f o r m a t i o n st og e n e r a t e communications are required. Array regions are used in Chapter 3 to achieve accurate communication generation. An interprocedural static analysis is proposed to optimize the communication by leaving data as much as possible on the GPU.A n o t h e rp a t hi st h e generation of tasks that are scheduled at runtime on multiple GPUs using StarPU. The task extraction and code generation for StarPU are presented in Chapter 6, along with another mapping on multiple GPUs based on symbolic tiling. The whole process is driven by the new Par4All driver, from the input source code to the final binary. It is based on a flexible pass manager. The challenge of automating the whole process is presented in Chapter 5. The experimental results are presented and discussed in Chapter 7.

Figure 3 . 1 :

 31 Figure 3.1: Stars-PM is a N -body cosmological simulation. Here a satellite triggers a bar and spiral arms in a galactic disc.

  To address this issue, my study is based on a program more representative of numerical simulations. It is a real numerical simulation called Stars-PM, a particle mesh cosmological N -body code whose output is shown in Figure3.1. The sequential version was written in Chapter 3. Data Mapping, Communications and Consistency int main(int argc , char *argv[]) {

Figure 3 . 2 :

 32 Figure 3.2: Simplified global scheme commonly used in numerical simulations.

Figure 3 . 3 :

 33 Figure 3.3: Outline of one time step in the Stars-PM cosmological simulation code.

Figure 3 . 4 :

 34 Figure 3.4: Array regions on a code with a function call.

Figure 3 . 5 :

 35 Figure 3.5: Array regions on a code with a while loop.

Figure 3 .

 3 Figure 3.7: Basic process for mapping data to the accelerator (source [Yan et al. 2009], c 2011 Springer-Verlag)g.

Figure 3 . 8 :

 38 Figure 3.8: Sequential source code for function discretization,t hefirststepofeac hStars-PM simulation main iteration.

Figure 3 . 9 :

 39 Figure 3.9: Code for function discretization after automatic GPU code generation.

Figure 3 .

 3 Figure 3.10: Isolation of the irregular while loop from Figure 3.5 using array region analysis.

Figure 3 . 11 :

 311 Figure 3.11: Code with a switch case from Figure 3.6 after isolation.

Figure 3 . 12 :

 312 Figure 3.12: Interprocedural isolation of the outermost loop of a Finite Impulse Response.

Figure 3 .

 3 Figure 3.13: Isolated version of the KERNEL function of the Finite Impulse Response (see Figure 3.4).

Figure 3 .

 3 Figure 3.14: Bandwidth for memory transfers over the PCI-Express 2.0 bus as a function of block size. Results are shown for transfers from the host to the GPU (H-TO-D) and in the opposite direction (D-TO-H), each for pinned or standard allocated memory.

Figure 3 .

 3 Figure 3.15: Illustration of set construction using the intraprocedural analysis on the function iteration. The different calls to step functions use and produce data on the GPU via kernel calls. Sometimes in the main loop, array a is read to display or to checkpoint. The interprocedural translation exploits at call site the summary computed on function iteration.Afi xp o i n ti ss o u g h to nt h el o o p .

  Figure 3.16: Simplified code for functions discretization and main after interprocedural communication optimization.

  (b) Host code after sequential promotion.

Figure 3 .

 3 Figure 3.17: gramschmidt example taken from Polybench suite. The first part of the loop body is sequential while the following are parallel loop nests. The sequential promotion on the GPU avoids costly memory transfers.

  o rOpenMP to GPU[Lee et al. 2009]. It is also relevant for a directive-based compiler, such as JCUDA and hiCUDA[Han & Abdelrahman 2009]. It would also complete the work done on OpenMPC[START_REF] Lee | OpenMPC: Extended OpenMP Programming and Tuning for GPUs[END_REF]b yn o to n l yr e m o v i n gu s e l e s sc o m m u n i c a t i o n sb u tm o v i n gt h e mu pi nt h ec a l l graph. Finally it would free the programmer of the task of adding directives to manage data movements in HMPP[START_REF] Bodin | son code source pour une architecture ou un modèle d'exécution[END_REF] and PGI Accelerator[Wolfe 2010].

Figure 3 .

 3 Figure 3.18,p a g e90,i l l u s t r a t e st h er e s u l to ft h ea l g o r i t h mo na ne x a m p l et a k e nf r o m the PIPS validation suite. It demonstrates the interprocedural elimination of data communications represented by the memload and memstore functions. These function calls are first moved outside of the loop, then outside of the bar function; finally, redundant load s are eliminated.

  void

  After the inter-iterations redundant elimination.

Figure 3 .

 3 Figure 3.19: Code with communication for FIR function presented in Figure 3.4.

Chapter 3 .

 3 Data Mapping, Communications and Consistency Theorem 3.1

  2.I tw a sf o u n dt h a tt h ep r o p o s e ds c h e m ep e r f o r m sv e r y close to a hand-written mapping in terms of number of communications.

( a )

 a Scilab source.

  double a[1000][1000]; for(i=0; i<1000; i++) for(j=0; j<1000; j++) a[i][j] = (double)( 1 . 0 ) ; double t0[1000][1000];

  1000,1000,b); double t1[1000][1000]; for(i=0; i<1000; i++) for(j=0; j<1000; j++) t1[i][j] = b[i][j]*2.; double c[1000][1000]; for(i=0; i<1000; i++) for(j=0; j<1000; j++) c[i][j] = t1[i][j]+3.; disp_s0d2("c" ,1000,1000,c); (b) Generated C code.

Figure 4 . 1 :

 41 Figure 4.1: Example of a short Scilab program with the generated C file.

Finally, Section 4 .

 4 10 summarizes the contributions of this chapter and how they are connected together in the next chapter to form a complete compilation chain. for (i=1; i<=n; i++) for (j=1; j<=n; j++) computation statements (a) Input code. for (T=...) // Sequential , on the CPU parfor (P=pl(T) to pu(T)) // NDRange decomposition for (t=...) // Sequential , on the GPU parfor (p=...) // Thread p a r a l l e l i s m inside work-group computation statements (b) After loop transformation for GPU mapping.

Figure 4 . 2 :

 42 Figure 4.2: Example from Baghdadi et al. [Baghdadi et al. 2010]t h a ti l l u s t r a t e sh o wt o tile a loop nest to map the GPU execution.

  Figure 4.2 illustrates an example of how a loop nest is tiled to map the two-level GPU execution model.

Figure 4 . 5 :

 45 Figure 4.5: Example of Allen and Kennedy algorithm as implemented in PIPS: loops are distributed and parallelism is expressed using OpenMP pragmas.

  , conduct a sanity pass to avoid the declaration of two reductions on the same variable. If compatible, they are merged. The last step prevents situations such as the two following function calls with side effects: Cs u m m a r y s u m [ X ] , p r o d [ m a r y s u m [ X ] , p r o d [ Y ] , SUBROUTINE SUMPROD(X, Y, Z) Cc a l l s u m [ X ] , l p r o d [ P ] , s u m [ S ] , p r o d [ P ] , s u m [ S ] , CALL SUMPROD(S, P, FSUMPROD(S, P, 3.)) Cc a l l p r o d [ S ] , s u m [ P ] , p r o d [ P ] , s u m [ S ] ,CALL SUMPROD(P, S, FSUMPROD(S, P, 3.))

Figure 4 .Figure 4 . 7 :

 447 Figure 4.7: Example of reduction detection and interprocedural propagation in PIPS.

Figure 4 .

 4 8 contains an example of a sequential histogram code and the resulting code after reduction detection and replacement with an atomic operation. static void _histogram(int data[NP][NP][NP], int histo[NP][NP][NP]) { int i,j,k; for (i = 0; i < NP; i++) { for (j = 0; j < NP; j++) { for (k = 0; k < NP; k++) { int x=f l o o r ( ( ( float)data[i][j][k]) / (float)(NP * NP)); int y=f l o o r ( ( ( float)(data[i][j][k] -x * NP * NP)) /( float)(NP)); int z=d a t a [ i ] [ j ] [ k ]-x*N P*N P-y*N P ; ++histo[x][y][z]; } } } } (a) Input code. static void _histogram(int data[NP][NP][NP], int histo[NP][NP][NP]) { int i,j,k; for (i = 0; i < NP; i++) { // Scheduled as p a r a l l e l for (j = 0; j < NP; j++) { // Scheduled as p a r a l l e l for (k = 0; k < NP; k++) { // Scheduled as p a r a l l e l int x=f l o o r ( ( ( float)data[i][j][k]) / (float)(NP * NP)); int y=f l o o r ( ( ( float)(data[i][j][k] -x * NP * NP)) /( float)(NP)); int z=d a t a [ i ] [ j ] [ k ]-x*N P*N P-y*N P ; atomicAddInt(&histo[x][y][z],1After replacement with atomic operation.

Figure 4 . 8 :

 48 Figure 4.8: An example of reduction parallelization of an histogram using hardware atomic operations.

2 .

 2 Evaluate individually each linear relation in P equations and inequations: (a) find in the relation a variable k modified in the loop body, (b) verify that all other variables are either loop indices or loop invariant, (c) construct the linear expression to replace k. k=-1 ; for(i=0;i<SIZE;i++) { k=i ; for(j=0;j<SIZE;j++)

Figure 4 . 9 :

 49 Figure 4.9: Example of induction variable substitution to enable loop nest parallelization.

Figure 4 .

 4 9a gives an example of such challenge. The C language allows side effects in references, for instance A[k++] = .... The solution that I designed and implemented handles these references with respect to the C standard. For instance A[k++] = ... is replaced by A[k = i+j+1, k-1] = ... (see in Figure

Figure 4 .

 4 Figure 4.10: Example of loop fusion.

Figure 4 . 11 .

 411 Some advanced transformations can remove fusion-preventing dependences. For example, Xue et al. eliminate anti-dependences using array copying[Xue 2005]. Shifting and peeling 2 techniques described in[START_REF] Manjikian | Fusion of Loops for Parallelism and Locality[END_REF]a l l o wt h ef u s i o na n d 2. These transformations enable loop fusion for loops with different iteration set. for(i=1;i<N;i++) // P a r a l l e l a[i]=0; for(i=1;i<N;i++) // P a r a l l e l b[i]=a[i-1]; (a) Original code: two parallel fusable loops. for(i=1;i<N;i++) { // Sequential a[i]=0; b[i]=a[i-1]; } (b) After fusion, the loop is sequential.

Figure 4 . 11 :

 411 Figure 4.11: Example of two parallel loops that can be legally fused, but the resulting loop nest would be sequential.

Figure 4 .

 4 Figure 4.10 illustrates a simple loop fusion.

  Kennedy and McKinley proposed two polynomial-time heuristics[START_REF] Kennedy | [END_REF]]a sas o l u t i o nf o rt h ew e i g h t e dl o o pf u s i o n[ Gao et al. 1993]. Finally Kennedy proposed a fast greedy weighted loop fusion heuristic[Kennedy 2001].Megiddo et al. present a linear-sized integer programming formulation for weighted loop fusion[START_REF] Megiddo | Optimal weighted loop fusion for parallel programs[END_REF]]. They claim that despite the NP-hardness of the problem, an optimal solution can be found within time constraints corresponding to a product-quality optimizing compiler.Bondhugula et al. used the polyhedral model to provide first maximal fusion [Bondhugula et al. 2008c, Bondhugula et al. 2008a], and later a metric-based [Bondhugula et al. 2010]a l g o r i t h mt h a to p t i m i z e sa tt h es a m et i m ef o rh a r d w a r ep r e f e t c h ,l o c a l i t y , and parallelism. Pouchet et al. [Pouchet et al. 2010b]b u i l dac o n v e xs e tt h a tm od e l st h es e to fa l ll e g a l possibilities on which an iterative empirical search is performed.

  In the context of GPGPU,l o o pf u s i o ni sd i r e c t l yr e l a t e dt ot h en u m b e ro fk e r n e l s obtained and their size, as presented in Section 4.2.F o u rm a j o rb e n e fi t sa r ee x p e c t e df r o m loop fusion:1. data reuse, 2. array contraction (see Section 4.7), 3. increasing the size of the kernel, 4. reducing the overhead associated to the kernel launch.

  Loop fusion to schedule more threads on the GPU.

Figure 4 . 12 :

 412 Figure 4.12: Example of a loop fusion scheme to extend the iteration set of a loop nest.

  void saxpy_slow(float A, thrust::device_vector <float >& X, thrust::device_vector <float >& Y) { thrust::device_vector <float >t e m p ( X . s i z e ( ) ) ; // temp <-A thrust::fill(temp.begin(), temp.end(), A); // temp <-A ⇤ X thrust::transform(X.begin(), X.end(), temp.begin(), temp.begin(), thrust::multiplies <float >()); // Y <-A ⇤ X+Y thrust::transform(temp.begin(), temp.end(), Y.begin(), Y.begin(), thrust::plus <float >()); } (a) Using native Thrust operator, performing 4N reads and 3N writes.

  <-A ⇤ X+Y void saxpy_fast(float A, thrust::device_vector <float >& X, thrust::device_vector <float >& Y) { thrust::transform(X.begin(), X.end(), Y.begin(), Y.begin(), saxpy_functor(A)); } (b) Using a user-defined kernel, performing 2N reads and N writes.

Figure 4 .

 4 Figure 4.13: Example of manual kernel fusion using Thrust library and a SAXPY example.The first version is expressed using native Thrust operators and requires temporary arrays, the second version fuses the three steps in one user-defined kernel (source[START_REF] Hoberock | Jared Hoberock and Nathan Bell. Thrust Quick Start Guide[END_REF]).

Figure 4 .

 4 Figure 4.14: On the left, a sequence of statements, in the middle the associated Dependence Graph (DG), and on the right the corresponding Reduced Dependence Graph (RDG) obtained after clustering the vertices that belong to the same loop.In solid red the flow dependences, in dashed blue the anti-dependence, and in dotted green the special dependences that model the declaration. The DG view showed here is simplified for the sake of clarity, for instance output dependences and the loop carried dependences are omitted.

Figure 4 .

 4 14a,P a g e113, contains a sequence of statements including some loops. The resulting (simplified) Dependence Graph (DG) and the RDG computed by PIPS are presented inFigures 4.14b

  and 4.14c.A ta l lt i m e s ,t h e r ei sao n e -t o -o n em a p p i n gf r o mt h e vertices in the RDG and the statements in the sequence.

( a )

 a The original RDG.(b) The RDG after pruning.

Figure 4 .

 4 Figure 4.15: The algorithm begins with a pruning phase. For each direct edge between two vertices it ensures that there is no other path between them.

Figure 4 .

 4 Figure 4.16: Heuristic algorithm FUSE_RDG to traverse the RDG and apply fusion. The graph is modified as side effect.

Figure 4 .

 4 Figure 4.18: The resulting code after applying the loop-fusion algorithm on the code presented in Figure 4.14a.

Figure 4 .Figure 4 .

 44 Figure 4.19: Sample code showing that inner loops have to be fused first in order to be able to fuse the outer loops without breaking the perfect nesting.

Figure 4 .

 4 21 shows an example of such generated code from a three-line Scilab program. After loop fusion, the generated C code contains three temporary arrays that can be replaced by scalars as shown in Figure 4.21b.

  k < nk; k += 1) { D_scalar += alpha*A[i][k]*B[k][j]; } (a) Original code. for(k = 0; k < 4*((nk)/4); k += 4) { D_scalar += alpha*A[i][k]*B[k][j]; D_scalar += alpha*A[i][k+1]*B[k+1][j]; D_scalar += alpha*A[i][k+2]*B[k+2][j]; D_scalar += alpha*A[i][k+3]*B[k+3][j]; } // Epilogue for(; k < nk; k += 1) { D_scalar += alpha*A[i][k]*B[k][j]; } (b) After loop unrolling.

Figure 4 .

 4 Figure 4.24: Example of loop unrolling with a factor four.

Figure 4 .

 4 Figure 4.25: Simple matrix multiplication example to illustrate array linearization interest.

Figure 5 . 1 :

 51 Figure 5.1: Source-to-source cooperation with external tools.

  , Guelton et al. 2011b (perso), Guelton 2011a].

Figure 5 . 2 :

 52 Figure 5.2: PyPS class hierarchy (source [Guelton et al. 2011a (perso), Guelton et al. 2011b (perso)]).

Figure 5 . 3 :

 53 Figure 5.3: Conditionals in PyPS.

Figure 5 . 4 :

 54 Figure 5.4: For loop is a control structure commonly involved in PyPS

Figure 5 . 5 :

 55 Figure 5.5: Using exceptions to adapt the compilation process.

Figure 5 . 6 :

 56 Figure 5.6: Searching fix point.

Figure 5 . 8 :

 58 Figure 5.7: Terapix compilation scheme.

Figure 5 .

 5 Figure 5.10: The brokers infrastructure and compilation flow in Par4All.

Figure 5 . 11 :Figure 5 . 12 :

 511512 Figure 5.11: The original source code for the potential step that involves two calls to FFTW library.

•

  The current GPU architecture capabilities: shared memory size SharedSize,n u mber of registers per CU NRegisterPerCU,m a x i m u mn u m b e ro ft h r e a d sp e rCU MaxThreadsPerCU,m a x i m u mn u m b e ro ft h r e a d sp e rb l o c kMaxThreadsPerBlock, maximum number of block per CU MaxBlocksPerCU.

Figure 5 .

 5 Figure 5.13: Computing the block size that maximizes the occupancy for a given kernel and a given GPU.

Figure 4 .

 4 22.F o rag i v e nb l o c ks i z e ,c h o o s i n gr e c t a n g u l a rm a p p i n gf o rt h et h r e a d sl e a d s to a speedup of two to fifteen and more. These two examples show a loop nest with two-dimensional array accesses and an inner sequential loop in the kernel.

Figure 5 .

 5 Figure 5.14: The main loop nest in syrk benchmark from the Polybench suite and the resulting kernel.

Figure 6 .

 6 Figure 6.1 illustrates this process on the simple 3mm example from the Polybench suite.

Figure 6 . 2 :

 62 Figure 6.2: Code generated automatically with pragmas to drive the StarPU runtime.The tasks have been declared with attributes in order to drive StarPU, specifying the suitable target platform for each implementation. Task parameters are declared with a const qualifier when used as read-only, and with an __attribute__((output)) when used as write-only.

Figure 6 . 3 :

 63 Figure 6.3: Execution timeline when using parallel tasks on one GPU (upper part) and on two GPUs (lower part) for the 3mm code presented in Figure6.2.

  ]a n dO t t o n i[ Ottoni et al. 2005]u s es o m ed e r i v e dp r o g r a md e p e n d e n c eg r a p h st op a r a l l e l i z ea p p l i c a t i o n . Verdo olaege et al. [Verdo olaege et al. 2007]u s ep r o c e s sn e t w o r k st oe x t r a c tp a r a l l e l i s m from a sequential application, however, limited to affine loops. Ceng et al. proposed a semi-automatic scheme

  Huynh et al. introduced a framework [Huynh et al. 2012]a sab a c ke n do ft h e StreamIt [Gordon et al. 2002]c o m p i l e r . S t r e a m I ti sal a n g u a g eu s i n gt h es t r e a m i n gp r ogramming model. The technique employed to identify suitable partitions relies on a modified well-known k -way graph partitioning algorithm [Karypis & Kumar 1998]. It computes load-balanced partitions with minimum communications.

Figure 7 . 1 :

 71 Figure 7.1: Kernel execution times in ms (best over twenty runs) and speedups between the Coarse Grained and the Allen and Kennedy parallelization algorithms using CUDA for different Nvidia GPUs. The example used here is the correlation loop nest shown in Figure 4.6 with m =3000.

Figure 7 . 2 :

 72 Figure 7.2: Impact of work-group size on GT200 and Fermi architectures, speedup normalized with respect to a size of 512 work-items per work-group. The CUDA API is used in this experiment.

Figure 7 . 3 :

 73 Figure 7.3: Impact of block dimensionality on performance for different block sizess expressed in µs for G80 architecture. The reference kernel here is the main loop nest from syrk (Polybench suite) shown in Figure 5.14 page 163.

Figure 7 . 4 :

 74 Figure 7.4: Impact of block dimensionality on performance for different block sizes expressed in µs for GT200 architecture. The reference kernel here is the main loop nest from syrk (Polybench suite) shown in Figure 5.14 page 163.

Figure 7 . 5 :

 75 Figure 7.5: Impact of block dimensionality on performance for different block sizes expressed in µs for Fermi architecture. The reference kernel here is the main loop nest from syrk (Polybench suite) shown in Figure 5.14 page 163.

Figure 7 . 6 :

 76 Figure 7.6: Impact of block dimensionality on performance for different block sizes expressed in µs for Kepler architecture. The reference kernel here is the main loop nest from syrk (Polybench suite) shown in Figure 5.14 page 163.

Figure 7 . 7 :

 77 Figure 7.7: Impact of block dimensionality on performance for different block sizes expressed in µs for G80 architecture. The reference kernel is the matmul example shown in Figure 4.22 page 129.

Figure 7 . 8 :

 78 Figure 7.8: Impact of block dimensionality on performance for different block sizes expressed in µs for GT200 architecture. The reference kernel is the matmul example shown in Figure 4.22 page 129.

Figure 7 . 9 :

 79 Figure 7.9: Impact of block dimensionality on performance for different block sizes expressed in µs for Fermi architecture. The reference kernel is the matmul example shown in Figure 4.22 page 129.

Figure 7 .

 7 Figure 7.10: Impact of block dimensionality on performance for different block sizes expressed in µs for Kepler architecture. The reference kernel is the matmul example shown in Figure 4.22 page 129.

Figure 7 . 11 :

 711 Figure 7.11: Execution times in ms (best over twenty runs) and speedups for scalarization using CUDA and OpenCL for different AMD and Nvidia GPUs. The example is the matmul kernel shown in Figure 4.22,p a g e129, with n i = n j = n k =2 0 4 8 . The execution times presented here are kernel execution times. The GTX 8800 results are given aside because they are one order of magnitude slower. This architecture does not perform double precision computations: doubles are rounded to float before being processed. Hence, no OpenCL results are available in double precision for this GPU.

Figure 7 . 12 :

 712 Figure 7.12: Execution times in µs( be s to v e rt w e n t yr u n s )a n ds pe e d u p sf o rs c a l a r i z a t i o n using CUDA and OpenCL for different AMD and Nvidia GPUs. The code is the Scilab script after conversion to C and loop fusion shown in Figure 4.21b,page129. The execution times shown are the kernel execution times. The GTX 8800 results are given aside because they are one order of magnitude slower. This architecture does not perform double precision computations: doubles are rounded to float before being processed. Hence, no OpenCL results are available in double precision for this GPU.

Figure 7

 7 Figure 7.13: Execution times in µs (best over twenty runs) for the code shown in Figure 4.23c and Figure 4.23d. The value of M is 8192 and the values of N are on the x axis. The Nvidia board shows clearly a shift on the red curve for N =3 2corresponding to the the warp size.

Figure 7 .

 7 Figure 7.14: Execution time in µs (best over twenty runs) and speedup for different unrolling factor using CUDA and OpenCL for different AMD and Nvidia GPUs. The execution times are the kernel execution times. Single precision floating point is used. The example is the code presented in Figure 4.22,p a g e129,u s e di nt h ep r e v i o u ss e c t i o n . L oo p unrolling is applied after scalarization to obtain the code shown in Figure 4.24,p a g e133. The GTX 8800 results are given aside because they are one order of magnitude slower.

Figure 7 .

 7 Figure 7.15: Register counts for different Nvidia architectures and different unrolling factors.

( a )

 a single precision, no scalarization. (b) single precision, with scalarization. (c) Double precision, no scalarization. (d) Double precision, with scalarization.

Figure 7 .

 7 Figure 7.16: Kernel execution times in ms (best over twenty runs) and speedups for the array linearization transformation and different Nvidia GPUs, and with and without the scalarization illustrated in Figure 7.11. The example is the matmul kernel shown in Figure 4.22,p a g e129, with n i = n j = n k =2048.

Figure 7

 7 Figure7.17: Execution times and speedups for versions of hotspot on GPU, with different iteration counts.

  static __global__ void flipflop(real_t src[SIZE][SIZE], real_t dest[SIZE][SIZE]) { int i=b l o c k I d x . y*b l o c k D i m . y+t h r e a d I d x . y ; int j=b l o c k I d x . x*b l o c k D i m . x+t h r e a d I d x . x

  (b) Host code using Par4All runtime.

Figure 7 .

 7 Figure7.18: Illustration for the code used to measure performance of the static approach on a Jacobi 2D scheme.

( a )

 a StarPU task to encapsulate the kernel launch.

  (b) Host code using the StarPU runtime.

Figure 7 .

 7 Figure 7.19: Illustration for the code used to measure performance for the StarPU version of the Jacobi 2D scheme.

Figure 7 .

 7 Figure 7.22 shows results for twenty benchmarks of the Polybench suite, three from Rodinia, and the Stars-PM application (see Section 3.1). Measurements were performed on a machine with two Xeon Westmere X5670 (twelves cores at 2.93 GHz) and a Nvidia GPU Tesla C2050. The OpenMP versions used for the experiments are generated automatically by Par4All and are not manually optimized or modified. The OpenMP version runs on the twelves X5760 cores only and does not use the GPU.

  code after sequential promotion.

Figure

  Figure 7.20: durbin example from Polybench that shows the interest of sequential promotion.

Figure 7 .

 7 Figure 7.21: Execution times in ms and speedups for CUDA execution with communication optimization, using the classic scheme and the sequential promotion. The result are based on the average over five runs for durbin and gramschmidt examples (see Figures 7.20 and 3.17).

Figure 7 . 22 :

 722 Figure 7.22: Speedup relative to naive sequential version for an OpenMP version on the CPU, a version with basic PGI Accelerator and HMPP directives, a naive CUDA version, and an optimized CUDA version, all automatically generated from the naive sequential code.

Figure 7 .

 7 Figure 7.23 shows the average execution times over ten runs for both examples on two different multi-GPU configurations, one with two C2070 and the other with three C1060.

Figure 7 .

 7 Figure 7.24: Average execution time in ms over ten runs for 3mm and bicg examples. Note the impact of different StarPU schedulers, the default greedy one and the data-aware, dmda.

Figure 7 .

 7 Figure7.26: Output from Nvidia Visual Profiler showing the communications in brown and the kernel executions in blue. The mapping is done using one C2070 only. The copy-out do not overlap properly with copy-in, which is unexpected and limits the acceleration that can be achieved.

Figure 7 .

 7 Figure7.27: Output from Nvidia Visual Profiler showing the communications in brown and the kernel executions in blue. The mapping is done using two C2070s. The copy-out do not overlap properly with copy-in, which is unexpected and limits the acceleration that can be achieved.

  ,p a g e105). Actually, I proposed a new generic scheme for parallelizing loops with reduction, and implemented it in PIPS.I tp r o v i d e si m p r o v e m e n t sf o ro t h e r targets like multicore using OpenMP code generation. Ii m p l e m e n t e dt w ol o o pf u s i o nt r a n s f o r m a t i o np h a s e s : o n eb a s e do nt h ed e p e n d e n c e graph and the other on array regions. I designed heuristics to drive the fusion in order to target GPUs. This is particularly critical when processing code generated from high-level tools and languages, such as Scilab, which include many loops. This transformation allows al a t e rs c a l a r i z a t i o np h a s et or e m o v em a n yt e m po r a r ya r r a y sg e n e r a t e db ys u c ht oo l s . I studied different array scalarization schemes in the context of GPGPU in Section 4.7, page 127,a n dIm o d i fi e dt h ePIPS implementation to match requirements for GPU code generation, especially to enforce the perfect nesting of loops.

  Le choix des langages C et Fortran est motivé par leur utilisation courante dans le domaine des simulations numériques et du calcul à haute performance. Le langage C est également un choix courant pour les outils générant du code depuis une représentation de plus haut niveau ou un langage de script. Pour illustrer l'intérêt de ce choix, des exemples de code Scilab[Scilab Consortium 2003]son tinclus.Ilsson tautomatiquemen tcon v ertisen code C séquentiel à l'aide d'un compilateur Scilab, et ensuite transformés pour exploiter les accélérateurs matériels en utilisant les différentes techniques présentées dans ce manuscrit.

  ].En 2000, Trendall et Stewart[Trendall &S t e w a r t2 0 0 0 ]é t a b l i s s a i e n tu na p e r ç ud e se x p é r i e n c e sp a s s é e se x p l o i t a n td um a t é r i e l graphique. Jusqu'en 2007, les GPUs n'exposaient qu'un flux d'exécution propre au traitement graphique au travers d'interface de programmation tel qu'OpenGL. Toute l'élégance la recherche d'alors résidait dans la correspondance établie entre des opérations mathématiques généralistes et les opérations graphiques possibles dans ce flux d'exécution[Trendall &S t e w a r t2 0 0 0 ]. Une limitation clé était qu'à cette époque les circuits graphiques n'offraien t que des unités de calculs flottants en simple précision, alors que les calculs en double précision sont souvent requis pour la plupart des simulations numériques.Les GPUs se sont popularisés pendant les dernières décennies, avec un excellent ratio coût/performance. Une conséquence a été la naissance d'une forte mode dans la recherche expérimentale pour exploiter ces circuits spécialisés. Cette tendance s'est reflétée d'abord dans l'évolution des interfaces de programmation. A la fois OpenGL et DirectX ont introduit les shaders en 2001, et par là même ajouter programmabilité et flexibilité au flux d'exécution graphique. Toutefois, utiliser une de ces interfaces de programmation graphique était toujours un point de passage obligatoire. Par conséquent la pratique du GPGPU était un défi encore plus important qu'il ne l'est aujourd'hui. En 2003 Buck et al. [Buck et al. 2004]o n ti m p l é m e n t éu ns o u s -e n s e m b l ed ul a n g a g e de streaming Brook pour cibler les GPUs. Ce nouveau langage, nommé BrookGPU, ne présente pas au développeur le flux d'exécution graphique. Le code est compilé vers les interfaces DirectX ou OpenGL. BrookGPU a été utilisé par exemple dans le cadre du projet Folding@home [Pande lab Stanford University 2012]. Plus d'informations au sujet de Brook et BrookGPU sont données à la Section 2.2.3. Ian Buck, qui a conçu Brook et BrookGPU, a rejoint Nvidia pour concevoir l'environnement CUDA.L el a n g a g ed é fi n ip a rCUDA partage des similitudes avec BrookGPU. Toutefois, alors que BrookGPU propose une abstraction générique, l'interface CUDA est spécifique à Nvidia et à la nouvelle architecture scalaire introduite simultanément. La Section 2.4.5 présente cette architecture de manière plus détaillée. CUDA est à la fois une interface de programmation et un langage pour programmer les GPUs plus facilement. Le flux d'exécution graphique n'existe plus en tant que tel et l'architecture matérielle est unifiée et exposée comme un ensemble de processeurs vectoriels. CUDA est introduit avec plus de détail à la Section 2.2.4. De 2004 à 2012, l'évolution de la performance en calcul flottant des GPUs aa u g m e n t é bien plus rapidement que celle des CPUs,c o m m ei l l u s t r ép a rl aF i g u r e2.1.L ap r o g r a mmabilité offerte par CUDA,c o m b i n é ea v e cl ' a v a n t a g ed ep e r f o r m a n c ed e sGPUs,ar e n d u la pratique du GPGPU de plus en plus populaire pour les calculs scientifiques au cours des cinq dernières années. L'intérêt croissant dans la pratique du GPGPU ac o n c e n t r éb e a u c o u pd ' a t t e n t i o n s , au point d'aboutir à la standardisation d'une interface de programmation dédiée et d'un langage pour programmer les accélérateurs : le Open Computing Language connu sous le sigle OpenCL (voir Section 2.3 pour plus de détail).

2. 2 . 8

 28 C++ Accelerated Massive Parallelism (AMP) Microsoft C++ Accelerated Massive Parallelism (AMP) est une spécification ouverte [Microsoft Corporation 2012a]p o u rp e r m e t t r ed up a r a l l é l i s m ed ed o n n é e se nt a n t qu'extension C++. La suite AMP de Microsoft a été livrée pour la première fois en janvier 2012. Elle est composée d'une extension au langage C++, d'un compilateur, d'un support exécutif.

3 .

 3 Les plateformes matérielles les plus répandues qui correspondent à ces critères sont fabriquées par AMD et Nvidia, et sont présentes dans les ordinateurs grand public. Cette section introduit et compare les architectures actuelles des GPUs.Lesdeuxt ypesdeparallélisme exploité sont présentés : le parallélisme au niveau du flux d'instruction (Instruction Level Parallelism (ILP)) et le parallélisme entre fils d'exécution (Thread Level Parallelism (TLP)).

  2.3.2 AMD, de l'architecture R300 à Graphics Core Next Historiquement, AMD au t i l i s éu nj e ud ' i n s t r u c t i o nv e c t o r i e l ,e tai n t r o d u i te n2 0 0 2 avec l'architecture R300 un jeu d'instruction VLIW àd e u xv o i e sp o u ra c c o m p a g n e rl e s débuts des calculs dans les shaders Cette architecture s'est révélée efficace pour traiter les problèmes posés par les applications graphiques, jusqu'à l'arrivée de DirectX 10 et des nouvelles fonctionnalités introduites dans les shaders.

  Les nouveautés introduites dans le langage HLSL par DirectX 10 a conduit les concepteurs des puces AMD àc h o i s i ru n ea r c h i t e c t u r ep l u sfl e x i b l e .L eR 6 0 0r e m p l a c el ' a r c h i t e cture VLIW mixte vecteur/scalaire par un jeu d'instructions VLIW àc i n qv o i e sp u r e m e n t scalaires. De cette manière, comme précédemment, cinq éléments individuels peuvent être traités à chaque cycle. De cette manière, comme précédemment, cinq éléments peuvent être traités à chaque cycle. Cependant le vecteur a été séparé et au lieu de devoir effectuer la même opération sur quatre éléments il est alors possible maintenant d'exécuter cinq opérations différentes.L'exécution SIMD dans de tels GPUs est gérée différemment que les extensions CPU telles que Streaming SIMD Extension (SSE) ou Advanced Vector eXtensions (AVX).L e s registres des GPUs ne sont pas vectoriels mais de larges zones reconfigurables dynamiquement à souhait. L'exécution SIMD est implicite et gérée par l'ordonnanceur matériel. Les PEs partagent une unité de décodage d'instruction, et tant que les différents fils d'exécution ne divergent pas et continuent d'exécuter la même instruction, tous les PEs sont exploités. En cas de branchement dans le flot de contrôle qui ne serait pas suivi par tous les fils d'exécution, les branches sont exécutées en séquence, et des PEs sont désactivés, réduisant le taux utilisation du GPU.La série Radeon HD 6900 series, de nom de code Cayman (R900), est livrée en 2010. Cette nouvelle génération évolue vers un jeu d'instruction VLIW de quatre voies, sensé être plus efficace d'après les tests effectués par AMD sur le taux de remplissage VLIW moyen obtenu en pratique par le compilateur. Cette simplification permet d'augmenter le nombre d'unité SIMD qui profite au TLP.La limitation principale du jeu d'instruction VLIW réside dans la pression mise sur le compilateur chargé d'extraire l'ILP statiquement dans le programme source. Cette limitation est acceptable pour des domaines restreints, tels que les applications graphiques, mais peut handicaper les performances atteignables par la pratique du GPGPU.L aF i -gure 2.18 montre des statistiques sur le taux d'utilisation des unités de calcul et du taux de remplissage des instructions VLIW. La dernière génération de circuit AMD,n o md ec o d eGraphics Core Next (GCN) (R1000), introduit une évolution majeure. L'architecture VLIW est abandonnée pour une architecture complètement scalaire. Les instructions VLIW àq u a t r ev o i e ss o n ts é p a r é e s dans quatre flux d'exécution SIMD. Cela signifie que l'ILP n'est plus exploitée par le compilateur et que les unités de calcul bénéficient implicitement du TLP.L e sa p p l i c a t i o n s doivent alors fournir un nombre suffisant de fils d'exécution pour utiliser toutes les unités de calcul. Le prix à payer est la présence d'un ordonnanceur matériel plus complexe.La Figure2.20 résume les évolutions du groupement des unités de calcul au fur et à mesure de l'évolution des architectures AMD.2.3.3 Nvidia Computing Unified Device Architecture, from G80 to KeplerA la fin de l'année 2006, Nvidia livrait le circuit G80. C'était le premier circuit compatible avec DirectX 10. Alors que les architectures précédentes proposées par Nvidia comportaient pour l'essentiel des unités de calculs spécialisées pour un nombre fixe de traitements graphiques, le G80 unifie les étages du flux d'exécution graphique en proposant des unités de calcul flexibles. Comme pour le R600 d'AMD,lechangementmajeurpourlapratiquedu GPGPU est que les unités de calcul peuvent traiter indifféremment tous types de shaders.Une autre nouveauté est que les unités de calcul exploitent un jeu d'instruction scalaire, similaire à la solution annoncée par AMD six ans plus tard avec Graphics Core Next (GCN).Le G80 possède plusieurs CUs composées de huit PEs.U n ei n s t r u c t i o ne s td é c o d é e par CU tous les deux cycles. Les PEs fonctionnent à une fréquence double du décodage d'instruction et chaque instruction est donc exécutée trente-deux fois (sur des données différentes). L'utilisation du GPU est maximisée quand trente-deux fils d'exécution traitent la même instruction simultanément, le SIMD est implicite. A nouveau le TLP est exploité et l'ILP n'est pas directement exposée par l'architecture mais peut être bénéfique pour masquer la latence mémoire, comme démontré par Volkov[Volkov 2010]. L'ordonnanceur bénéficie d'instructions indépendantes en séquence pour les émettre sans délai dans le flux d'exécution.Le GT200 succède au G80 en 2008. L'évolution principale du point de vue de la pratique du GPGPU est l'introduction d'unités de calcul flottant à double précision. Une autre nouveauté importante est la présence d'opération atomique en mémoire globale.En 2010, Nvidia livre une révision majeure de l'architecture avec Fermi. De nombreuses améliorations sont présentes du point de vue de la pratique du GPGPU :l es u p p o r td e s contrôles de flot indirect ouvrant la voie au C++ et aux fonctions virtuelles, une hiérarchie de caches gérés par le matériel sont introduits, les transferts de données sur le bus PCI Express (PCIe) s'effectuent dans les deux sens simultanément, plusieurs noyaux peuvent être ordonnancer simultanément se partageant les ressources matérielles, et l'introduction de la mémoire à correction d'erreur Error-correcting code (ECC).La Figure 2.22 illustre l'évolution des CUs introduite avec Fermi. Le nombre de PEs est augmenté de huit à trente-deux, et ceux-ci sont séparés en deux flux d'exécution qui possèdent chacun leur ordonnanceur. Une instruction est traitée à chaque cycle pour seize fils d'exécution simultanément, soit la même vue d'une largeur logique SIMD de trente-deux que dans les architectures précédentes.

  c i b l e n tJ a v ae tb é n é fi c i e n t d'informations à l'exécution sur la taille des tableaux. D'autres comme Verdoolaege and Grosser [Verdo olaege et al. 2013]traiten tlelangageCmaisson tlimitésauxtableauxdon t la taille est connue à la compilation. Les algorithmes utilisés par les outils propriétaires tels que R-Stream, HMPP,o uP G IA c c e l e r a t o rs o n ti n c o n n u s ,m a i si l ss e m b l e n ts u i v r el e même principe.

  3.10 Optimisation d'un nid de boucles tuiléesAlias et al. ont publié des travaux intéressants sur l'optimisation fine des communications dans le contexte des FPGAs[START_REF] Alias | [END_REF], Alias et al. 2012b, Alias et al. 2012a]. Le fait qu'ils ciblent des FPGAs change des considérations sur la taille de la mémoire : les FPGAs embarquent en général très peu de mémoire en comparaison des GPUs.L a proposition d'Alias et al. se focalise sur l'optimisation des chargements depuis la mémoire Double Data Rate (DDR) dans le cas d'un nid de boucles tuilées, pour lequel le tuilage est effectué de sorte que les tuiles s'exécutent en séquence sur l'accélérateur alors que les itérations à l'intérieur des tuiles peuvent être parallélisées.

Figure 4 .

 4 1 illustre de quelle manière un script de trois lignes de Scilab aboutit à un code Ca v e cd e st a b l e a u xt e m po r a i r e se tc i n qn i d sd ebo u c l e s .

  3. J'ai amélioré la phase de détection de réductions existante dans PIPS pour traiter le langage C plus précisément, et j'ai exploité cette analyse pour permettre la parallélisation en utilisant le concept de NDRange d'OpenCL introduit à la Section 2.3.2.2.LaFigure4.3 indique les quatre étapes impliquées dans cette transformation. D'abord, le corps du nid de boucles initial dans la Figure 4.3 est détouré dans une nouvelle fonction, le noyau, qui sera exécuté dans chaque fil d'exécution sur le GPU.L e si n d i c e sd eb o u c l e ss o n tr e c o n s t r u i t s dans le noyaux à l'aide de deux macros P4A_vp_x pour chaque dimension. L'exécution séquentielle est effectuée avec une simple expansion de ces macros #define P4A_vp_1 ti et #define P4A_vp_0 tj.Leniddebouclesparallèlesestalorsannotéa vecl'espaced'itération, comme illustré par la Figure 4.3c.

L

  'algorithme d'Allen et Kennedy est basé sur un graphe de dépendance à niveaux. Cet algorithme a été prouvé optimal par Darte et Vivien [Darte & Vivien 1996b]pourunetelle abstraction de dépendances. Cet algorithme a été conçu pour les machines vectorielles, et donc dans sa version basique distribue les boucles autant que possible et maximise le parallélisme. Un exemple qui illustre le traitement de l'implémentation dans PIPS de l'algorithme d'Allen et Kennedy est présenté à la Figure 4.5.L ad i s t r i b u t i o nd eb o u c l e se x p o s el e maximum de parallélisme mais augmente le nombre de barrières implicites. De plus les op-portunités de réutilisation temporelle du cache et de contraction de tableaux sont moindres, comme illustré sur les Figures 4.5a

  et 4.5b. Ces inconvénients de la distribution de boucles peuvent être limités par une transformation de fusion de boucles, telle que présentée à la Section 4.6.

  est interprocédural et requiert un résumé pour les fonctions appelées pour traiter une fonction. De manière intraprocédurale, l'algorithme détecte les réductions dans les suites d'instructions telle que : // c a l l sum [ s [ a ] ] , sum [ b ] ,

  Pour les GPUs,u nm o y e nd ep a r a l l é l i s e rl e sb o u c l e sq u ic o m p o r t e n tl e sr é d u c t i o n se s t de faire usage des opérations atomiques présentées à la Section 2.4.3,p a g e42.S il ac i b l e est compatible avec les opérations atomiques nécessaire, alors la substitution est faite et la boucle est déclarée parallèle pour une future extraction en noyau. La Figure 4.8 contient un exemple de code séquentiel d'histogramme, ainsi que le code résultant après détection des réductions et remplacement par des opérations atomiques.

  sontutiliséespourdéfinir un nouvel algorithme pour détecter et remplacer les variables d'induction. Considérant une boucle L,l ' a l g o r i t h m et r a i t et o u t e sl e si n s t r u c t i o n sd a n sl ec o r p sd el abo u c l ee tu t i l i s el e s préconditions linéaires associées pour remplacer les variables d'induction. Cette transformation est un défi dans le cadre du source-à-source et du langage C. La Figure 4.9a donne un exemple de code C avec des effets de bord dans les références, par exemple A[k++] = ....L as o l u t i o nq u ej ' a ic o n ç u ee ti m p l é m e n t é et r a i t ec e sr é f é r e n c e s selon le standard C. Par exemple A[k++] = ... est remplacé par A[k = i+j+1, k-1] = ... comme montré sur la Figure 4.9b.L ec od es o u r c et r a n s f o r m éc o n s e r v ea u t a n tq u epo s s i b l e la structure du code initial.

  l ad é fi n i tc o m m es ia u c u na r cd ed é p e n d a n c ed uc o r p sd el a première boucle vers le corps de la seconde est inversé après fusion. Irigoin et al. proposent une autre solution [Irigoin et al. 2011 (perso)]b a s é es u rl e s régions de tableaux [Creusillet & Irigoin 1996b]. Cette proposition, qui permet d'identifier ces dépendances sans aucun graphe de dépendances, est présentée à la Section 4.6.5. La fusion de boucles parallèles peut aboutir à une boucle séquentielle. Cette situation, illustrée par la Figure 4.11,p e u tê t r ec o n s i d é r é ec o m m eb l o q u a n t ep o u rl af u s i o nd a n su n objectif de maximisation du parallélisme. La Figure 4.10 illustre une simple fusion de boucles.

  5C o n c e p t i o n d e c o m p i l a t e u r s h é t é r o g è n e s e t a u t o m a t isation Alors que les chapitres précédents sont focalisés sur des transformations individuelles implémentées principalement dans PIPS,c ec h a p i t r et r a i t el ep r o c e s s u sd ec o m p i l a t i o n complet, du code source original jusqu'au binaire final. A cette fin, nous introduisons Par4All [SILKAN 2010 (perso), Amini et al. 2012b (perso)]àl aS e c t i o n5.1.P a r 4 A l le s t une initiative Open Source visant à fédérer les efforts réalisés sur les compilateurs pour permettre la parallélisation automatique des applications pour architectures hybrides.

  De plus, des transformations complexes pour architectures hybrides ou distribuées requièrent l'ajout d'appel à un support exécutif, souvent livré sous forme de bibliothèque partagée, et donc l'étape d'édition des liens est dépendantes des transformations effectuées. Par exemple si dans le flot de compilation, une opération de FFT avec la bibliothèque FFTW3 est transformée dans son équivalent GPU avec CuFFT,l eg e s t i o n n a i r ed ep a s s e s peut propager cette information pour l'étape d'édition des liens. Plusieurs architectures existantes et passées ont bénéficié de la flexibilité du gestionnaire de phases programmable implémenté dans PIPS : -Terapix est un accélérateur à base de FPGA pour le traitement d'image proposé par Thales qui met en oeuvre un jeu d'instruction VLIW,pourlequelleflotdecompilation est résumé à la Figure 5.7. Ce flot de compilation a été implémenté par Guelton et décrit en détail dans sa thèse [Guelton 2011a]. -SAC est un générateur d'instructions vectorielles introduit dans [Guelton 2011a]. La Figure 5.8 montre un extrait de son processus de compilation en charge du tuilage. -Un compilateur itératif qui requiert le clonage de l'état interne du compilateur et itère sur un domaine paramétrique de transformations en utilisant un algorithme génétique a été implémenté aisément à l'aide des abstractions fournies par PyPS. -Par4All, présenté à la Section 5.1,e s tu nc o m p i l a t e u rm u l t i -c i b l e s( OpenMP,CUDA, OpenCL, SCMP, etc.) qui traite du Fortran et du C. Il repose fortement sur les capacités de PyPS et bénéficie de l'environnement Python.

  13. Welcome to the hardware jungle INSTITUT DES SCIENCES ET TECHNOLOGIES Transformations de programme automatiques et source-à-source pour accélérateurs matériels de type GPU Résumé : Depuis le début des années 2000, la performance brute des coeurs des processeurs a cessé son augmentation exponentielle. Les circuits graphiques (GPUs) modernes ont été conçus comme des circuits composés d'une véritable grille de plusieurs centaines voir milliers d'unités de calcul. Leur capacité de calcul les a amenés à être rapidement détournés de leur fonction première d'affichage pour être exploités comme accélérateurs de calculs généralistes. Toutefois programmer un GPU efficacement en dehors du rendu de scènes 3D reste un défi. La jungle qui règne dans l'écosystème du matériel se reflète dans le monde du logiciel, avec de plus en plus de modèles de programmation, langages, ou API, sans laisser émerger de solution universelle. Cette thèse propose une solution de compilation pour répondre partiellement aux trois "P" propriétés : Performance, Portabilité, et Programmabilité. Le but est de transformer automatiquement un programme séquentiel en un programme équivalent accéléré à l'aide d'un GPU. Un prototype, Par4All, est implémenté et validé par de nombreuses expériences. La programmabilité et la portabilité sont assurées par définition, et si la performance n'est pas toujours au niveau de ce qu'obtiendrait un développeur expert, elle reste excellente sur une large gamme de noyaux et d'applications. Une étude des architectures des GPUs et les tendances dans la conception des langages et cadres de programmation est présentée. Le placement des données entre l'hôte et l'accélérateur est réalisé sans impliquer le développeur. Un algorithme d'optimisation des communications est proposé pour envoyer les données sur le GPU dès que possible et les y conserver aussi longtemps qu'elle ne sont pas requises sur l'hôte. Des techniques de transformations de boucles pour la génération de code noyau sont utilisées, et même certaines connues et éprouvées doivent être adaptées aux contraintes posées par les GPUs. Elles sont assemblées de manière cohérente, et ordonnancées dans le flot d'un compilateur interprocédural. Des travaux préliminaires sont présentés au sujet de l'extension de l'approche pour cibler de multiples GPUs. Mots clés : GPU, CUDA, OpenCL, Parallélisation automatisée, Compilation Source-to-Source Automatic Program Transformations for GPU-like Hardware Accelerators

  Simple example for HMPP directive-based code writing (source wikipedia [Wikipedia 2012c]). ......................... 27 2.9 Example of a PGI Accelerator code using data movement optimization (source PGI Insider [Wolfe 2011]). ....................... 28 2.10 A simple JCUDA example. Note the IN, OUT,a n dINOUT attributes in the kernel declaration that drive automatic memory transfers (source [Yan Simplified view of the OpenCL abstraction model. A host is connected to multiple devices (GPUs, FPGAs, DPSs, . . .). OpenCL platforms are vendors' implementations that target some types of devices. A context is created for a given platform and a set of devices. Memory objects and events are created context-wise. Devices are then controlled in a given context using command queues. There can be multiple command queues per device, and a device can be associated with queues from multiple contexts and platforms. 33 2.12 UML representation of the OpenCL abstraction model (see Figure 2.11) taken from the Standard [Khronos OpenCL Working Group 2011]. .... 34 2.13 A mapping example of a two-dimensional loop nest iteration set into an OpenCL index range. The mapping is the simplest possible; one work-item executes one iteration of the original loop nest. The work-group size used as an illustration on figure c is a two-dimensional square with an edge of These PUs are organized in a 16-wide SIMD array. ................. 43 2.18 Table summarizing the ALU occupation and the VLIW packing ratio for some computing kernels, taken from [Zhang et al. 2011b](c 2011 IEEE). . 46 2.19 The 2012 AMD architecture Graphics Core Next. No longer VLIW, the four separate SIMD pipelines are independent. A new integer scalar unit is introduced. The scheduler feeds each SIMD every four cycles (one per cycle) with a 64-wide virtual SIMD instruction. ................... 46 2.20 Evolution of Processing Element (PE) grouping across AMD architectures. The GeForce 8800 architecture (G80) introduced unified shaders where shader programmable processors can be used to replace multiple stages of the classic graphic pipeline. There are still specialized units for some graphics operations. (Source: Nvidia) ........................ 2.22 GT200 compute unit (CU) on the left, Fermi's on the right. Processing Elements (PE) upgrade from eight to sixteen per pipeline, but the logical SIMD width is unchanged, threads are scheduled by groups of thirty-two (source Nvidia). ................................

	List of Figures	xix
	2.21

). 18 2.5 Example of a Cg/HLSL shader for DirectCompute (source Microsoft [Deitz 2012]). .............................. 22 2.6 Rewriting a C++ computation using C++ AMP. The example shows the use of a lambda function and a parallel_for_each construct to express the parallelism (source Microsoft [Microsoft Corporation 2012b]). ........ 24 2.7 A sample matrix multiplication code with hiCUDA directives (source [Han &A bd e l r a h m a n2 0 0 9 ]). ............................. 26 2.8 et al. 2009]). .................................. 29 xviii List of Figures 2.11 five. Values for get_global_id() and get_local_id() OpenCL primitives are exhibited for a particular work-group. .................. 35 2.14 Visual example of the OpenCL memory model. Two possible mappings are illustrated: data caches are optional, and private, local, and constant memories are not necessarily dedicated. On the right the simplest mapping, for instance a CPU, merges all memory spaces onto the same piece of hardware. 37 2.15 High-level simplified GPGPU-oriented view of generic GPU architecture. . 41 2.16 Instruction Level Parallelism (ILP) versus Thread Level Parallelism (TPL), two different ways of extracting parallelism in GPUs. ............ 42 2.17 AMD R600 Compute Unit (CU) is built on top of 5-way VLIW instructions set. Four Processing Elements (PE) and a Special Function Unit (SFU) are grouped together in a Processing Unit (PU) to process instructions.

  Microsoft C++ Accelerated Massive Parallelism (AMP) is an open specification [Microsoft Corporation 2012a] for enabling data parallelism directly in C++. It was first released in January 2012. It is composed of a C++ language extension, a compiler, a runtime, and a programming model.The C++ AMP programming model supports multidimensional arrays, indexing, memory transfer, and tiling. Some language extensions control the ways data are moved from

	cbuffer CB : register(b0)
	{
	int size;
	};
	StructuredBuffer <float >d _ A:register(t0);
	StructuredBuffer <float >d _ B:register(t1);
	RWStructuredBuffer <float >d _ C:register(u0);
	groupshared float local_a[16][16];
	groupshared float local_b[16][16];
	[numthreads(16, 16, 1)]
	void mm(uint3 DTid : SV_DispatchThreadID , uint3 GTid : SV_GroupThreadID)
	{
	int row = GTid.y;
	int col = GTid.x;
	float sum = 0.0f;
	AllMemoryBarrierWithGroupSync();
	for ( int k=0 ;k<1 6 ;k + + ){
	sum += local_a[row][k] * local_b[k][col];
	}
	AllMemoryBarrierWithGroupSync();
	}
	d_C[DTid.y * size + DTid.x] = sum;
	}
	Figure 2.5:

for ( int i=0 ;i<s i z e ;i+ =1 6 ){ local_a[row][col] = d_A[DTid.y * size + i + col]; local_b[row][col] = d_B[(i + row) * size + DTid.x];

  standard,Lee et al. propose OpenMP for GPGPU[Lee et al. 2009]. They justify the advantages of OpenMP as a programming paradigm for GPGPU as follows:

	Chapter 2. General-Purpose Processing on GPU : History and Context
		# include <amp.h>
	# include <iostream >	# include <iostream >
		using namespace concurrency;
	const int size = 5;	const int size = 5;
	void StandardMethod() {	void CppAmpMethod() {
	int aCPP[]={1,2,3,4,5};	int aCPP[]={1, 2, 3, 4, 5};
	int bCPP[]={6,7,8,9,10};	int bCPP[]={6, 7, 8, 9, 10};
	int sumCPP[size];	int sumCPP[size];
		// Create C++ AMP o b j e c t s .
		array_view <const int ,1> a(size ,aCPP);
		array_view <const int ,1> b(size ,bCPP);
		array_view <int ,1 >s u m ( s i z e ,s u m C P P ) ;
		sum.discard_data();
		parallel_for_each(
		// Define the compute domain , which
		// i s the set of threads that are
		// created .
		sum.extent ,
		// Define the code to run on each
		// thread on the accelerator .
	for( int idx=0;idx <5;idx++)	[=](index <1> idx) restrict(amp)
	{	{
	sumCPP[idx]=	sum[idx] = a[idx] + b[idx];
	aCPP[idx]+bCPP[idx];	}
	}	);
	for( int idx=0;idx <5;idx++)	// Print the r e s u l t s . The expected
	{	// output i s "7 , 9 , 11 , 13 , 15".
	std::cout <<sumCPP[idx]	for ( int i=0 ;i<s i z e ;i + + ){
	<<"\n";	std::cout << sum[i] << "\n";
	}	}
	}	}
	(a) Pure C++.	(b) Using C++ AMP.

  Figure 2.10: A simple JCUDA example. Note the IN, OUT,a n dINOUT attributes in the kernel declaration that drive automatic memory transfers (source

	1)/ThreadsPerBlock[1];
	/⇤ invoke device on this block/thread grid ⇤/
	cudafoo.foo1 <<<<BlocksPerGrid , ThreadsPerBlock >>>>(I_a,
	I_aout ,
	I_aex);
	printArray(I_a); printArray(I_aout); printArray(I_aex);
	......
	static lib cudafoo("cfoo","/opt/cudafoo/lib") {
	acc void foo1(IN double[][] a,
	OUT int[][] aout ,
	INOUT float[][] aex);
	acc void foo2(IN short[][] a,
	INOUT double[][][] aex,
	IN int total);
	}

  Case Study ................................

	Chapter 3
	Data Mapping, Communications and
	Consistency
	Contents
	3.1

3.2 Array Region Analysis ......................... 3.3 Basic Transformation Process ..................... 3.4 Region Refinement Scheme ...................... 3.4.1 Converting Convex Array Regions into Data Transfers ........ 3.4.2 Managing Variable Substitutions .................... 3.5 Limits ................................... 3.6 Communication Optimization Algorithm .............. 3.6.1 A New Analysis: Kernel Data Mapping ................ 3.6.2 Definitions ................................ 3.6.3 Intraprocedural Phase .......................... 3.6.4 Interprocedural Extension ........................ 3.6.5 Runtime Library ............................. 3.7 Sequential Promotion .......................... 3.7.1 Experimental Results .......................... 3.8 Related Work ............................... 3.8.1 Redundant Load-Store Elimination ................... 3.9 Optimizing a Tiled Loop Nest ..................... 3.10 Conclusion .................................

  Alias et al. have published an interesting study about fine grained optimization of communications in the context of FPGA[START_REF] Alias | [END_REF], Alias et al. 2012b, Alias et al. 2012a]. The fact that they target FPGAs changes some considerations on the memory size: FPGAs usually embed a very small memory compared to the many gigabytes available in a GPU board. The proposal from Alias et al. focuses on optimizing loads from Double Data Rate

	for( int i=0 ;i<N ;+ + i){
	memcpy(M,m,k*sizeof( int));
	memcpy(&SRC[i],&src[i],k*sizeof( int));
	kernel(i, n, k, SRC, DST, M);
	memcpy(&dst[i],&DST[i],1*sizeof( int));
	}
	(a) With naive communication scheme.

Table 4 .

 4 Figure 4.3: Illustration of the successive steps performed to map a loop nest on the GPU. 1: A comparison of some lo op parallel algorithms (from data published in

		void kernel(int ti ,int tj ,...) {
		int i=P 4 A _ v p _ 1 ;
		int j=P 4 A _ v p _ 0 ;
		// computation statements
		...
	r a l l e l	}
	for (i=0; i<=n; i++)
	// p a r a l l e l	// p a r a l l e l
	for (j=0; j<=m; j++) {	for (ti=0; ti<=n; ti++)
	// computation statements	// p a r a l l e l
	...	for (tj=0; tj<=m; tj++)
	}	kernel(ti,tj ,...);
	(a) Input code.
		Call_kernel_2d(n, m, kernel ,...);
	(c) Annotated loop nest iteration set.	(d) The loop nest replaced by an abstract macro call.
	void kernel(int ti ,int tj ,...) {
	int i=P 4 A _ v p _ 1 ;
	int j=P 4 A _ v p _ 0 ;
	if(i<n&&j<m) { // Guard
	// computation statements
	...
	}
	}
	Figure 4.4: The iteration set is over-approximated with a rectangular hull; a guard is added
	to clamp it.

(b) Body outlined in a new function. // Loop nest P4A begin ,2D(n , n) // p a r a l l e l for (ti=0; ti<=n; ti++) // p a r a l l e l for (tj=0; tj<=m; tj++) { // Loop nest P4A end kernel(ti,tj ,...); }

Table 4 .

 4 1 reproduces the summary that they established. PIPS relies on two parallelization algorithms, the first one is Allen and Kennedy's and

	for(i=0; i<N; i++) {
	for(j=0; j<N; j++) {
	a

  .6.

	/⇤ Calculate the m ⇤ mc o r r e l a t i o n m a t r i x . ⇤/
	for(i=1;i<m;i++) {
	for(j=i;j<=m;j++) {
	symmat[i][j] = 0.0;
	for(k=1;k<=n;k++)
	symmat

  Par4All Project .............................

	Chapter 5
	Heterogeneous Compiler Design and
	Automation
	Contents
	5.1

5.2 Source-to-Source Transformation System .............. 5.3 Programmable Pass Managers ..................... 5.3.1 PyPS ................................... 5.3.2 Related Work ............................... 5.3.3 Conclusion ................................ 5.4 Library Handling ............................. 5.4.1 Stubs Broker ............................... 5.4.2 Handling Multiple Implementations of an API: Dealing with External Libraries .................................. 5.5 Tool Combinations ............................ 5.6 Profitability Criteria ........................... 5.6.1 Static Approach ............................. 5.6.2 Runtime Approach ............................ 5.6.3 Conclusion ................................ 5.7 Version Selection at Runtime ..................... 5.8 Launch Configuration Heuristic .................... 5.8.1 Tuning the Work-Group Size ...................... 5.8.2 Tuning the Block Dimensions ...................... 5.9 Conclusion .................................

  Contents 6.1 Introduction ............................... 6.2 Task Parallelism ............................. 6.2.1 The StarPU Runtime ..........................

6.2.2 Task Extraction in PIPS ......................... 6.2.3 Code Generation for StarPU ...................... 6.3 Data Parallelism Using Loop Nest Tiling .............. 6.3.1 Performance ................................ 6.4 Related Work ............................... 6.5 Conclusion .................................

Table 7 .

 7 1: The different hardware platforms available for the experiments.

		8800GTX 3xC1060	2xC2070	6970	GTX680
	Chipset	G80	GT200	Fermi	Cayman	Kepler
	GPU Memory (GB)	0.768	3x42 x6	22
	GPU Cores	128	3x240	2x448	1536	1536
	GPU Freq (MHz)	1350	1300	1150	880	1006-1058
	GPU Theoretical Bw (GB/s)	86.4	102	144	176	192.2
	Host	X5130	2 ⇥ X5570 2 ⇥ E5620 FX-6100	i7-3820
	CPU Physical Cores	2	2 ⇥ 42 ⇥ 4	64
	Host Memory (GB)	4	24	24	8	8

7.2 Benchmarks, Kernels, and Applications Used for

  library. StarPU is used here in its 0.4 version, configured and compiled with its default options.

	200						Chapter 7. Experiments
		2mm 3mm adi atax bicg correlation covariance doitgen fdtd-2d
	Naive	8	12	15t	6	6	18	15	5	13t
				+9nt	+4n	+4n
	Opt.	5	6	4	3	5	2	2	3	6
	Hand	4	5	4	3	5	2	2	3	6
		gauss-filter gemm gemver gesummv gramschmidt jacobi-1d jacobi-2d
	Naive	7		4	17	7	9n		6t	6t
	Opt.	4		4	10	3	2n+1		3	3
	Hand	4		4	10	3	3		3	3
		lu mvt symm-exp syrk syr2k hotspot99 lud99 srad99 Stars-PM
	Naive 4n	8	6n	5	6	5t	4n	24t	25t
	Opt.	2	7	4	3	4	3	2	t+1	3
	Hand	2	7	4	3	4	3	2	t+1	3

  CUDA kernel for one Jacobi 2D iteration.

	gettimeofday(&p4a_time_begin , NULL);
	// Send data
	P4A_runtime_copy_to_accel(space , sizeof(real_t)*SIZE*SIZE);
	P4A_runtime_copy_to_accel(save, sizeof(real_t)*SIZE*SIZE);
	for(t = 0; t < 400; t++) {
	dim3 dimGrid(SIZE/32+SIZE%32,SIZE/8+SIZE%8,1);
	dim3 dimBlock(32,8,1);
	real_t (*d_space)[SIZE] = P4A_get_accel_ptr(space);
	real_t (*d_save)[SIZE] = P4A_get_accel_ptr(save);
	flipflop <<<dimGrid ,dimBlock >>>(d_space , d_save);
	flipflop <<<dimGrid ,dimBlock >>>(d_save , d_space);
	cudaThreadSynchronize();
	}
	// Get back data from the GPU
	P4A_runtime_copy_from_accel(space, sizeof(real_t)*SIZE*SIZE);
	gettimeofday(&p4a_time_end , NULL);

  Figure 7.23: bicg example from Polybench that shows the impact of the different StarPU schedulers on a sequential example. There is a direct dependence between each of the kernels. Here nx = ny =8000,t h u st h ekernel3() is executed 8000 times.

			p[j];
	}
	for(i=0;i<nx;i++)// Sequential
	for(j=0;j<ny;j++)// P a r a l l e l
	s[j] = s[j] + r[i] * A[i][j];
	(b) After parallelization, ready to be transformed
	into kernels.
	kernel1(s, ny);
	kernel2(A, p, q, nx, ny);
	for(i=0;i<nx;i++)
	kernel3(A, s, i, ny, r_0);
	(c) After task transformation.
		3mm	bicg
	One C2070 static synchronous scheduling	962.3	175.3
	One C2070 StarPU default scheduler	943.8	163.9
	One C2070 StarPU data-aware scheduler	942.6	208.1
	Two C2070 StarPU default schedulers	809.5 859517.2
	Two C2070 StarPU data-aware schedulers	724.8	404.7
	One C1060 static synchronous scheduling	3231.8	301.2
	One C1060 StarPU default scheduler	3167.6	280.2
	One C1060 StarPU data-aware scheduler	3177.0	279.8
	Two C1060 StarPU default schedulers	2151.5 2446426.6
	Two C1060 StarPU data-aware schedulers	2151.1	629.6
	Three C1060 StarPU default schedulers	2162.5 2199509.7
	Three C1060 StarPU data-aware schedulers 2155.8	599.7

  La Figure2.27 illustre cette chaîne de compilation. Le code source est d'abord analysé pour trouver le parallélisme, puis transformé, avant d'extraire le code à exécuter sur le GPU dans de nouvelles fonctions : les noyaux. Certaines phases d'optimisation peuvent être appliquées, telle que la fusion de boucles, le remplacement de tableaux par des scalaires, la linéarisation des accès au tableaux, le tuilage symbolique, ou encore le déroulage de boucles. Cette partie du processus est présentée au Chapitre 4.A p r è sl ag é n é r a t i o n des noyaux, des analyses et des transformations pour générer les communications sont nécessaires. Les régions convexes de tableaux sont utilisées dans le Chapitre 3 pour générer des communications efficaces. Une analyse interprocédurale statique est proposée pour optimiser les communications en laissant les données autant que possible sur le GPU.U n e autre voie est la génération de tâches qui sont ordonnancées à l'exécution sur plusieurs accélérateurs à l'aide du support exécutif StarPU.L ' e x t r a c t i o nd et â c h e se tl ag é n é r a t i o n de code pour StarPU sont présentées dans le Chapitre 6,t o u tc o m m eu n ea u t r em é t h o d e basée sur du tuilage symbolique. Le processus complet est piloté par Par4All, depuis le code source d'entrée jusqu'au binaire final. Par4All est basé sur un gestionnaire de phases de compilation flexibles. Le défi d'automatiser un processus complet est présenté dans le Chapitre 5. Les résultats expérimentaux sont présentés et discutés dans le Chapitre 7.

  idée de base est de se servir des régions pour créer une nouvelle allocation suffisamment grande, puis de générer les transferts mémoires depuis la zone mémoire d'origine vers cette nouvelle allocation, et enfin d'effectuer la copie de cette allocation vers la zone d'origine. La Figure3.10 illustre le résultat de ce traitement, dans lequel les variables isolées sont en majuscules. Les instructions (3) et (5) correspondent à la région exacte sur les variables scalaires. Les instructions (2) et (4) correspondent aux régions sur-approximées sur les tableaux. L'instruction (1) est utilisée pour s'assurer de la cohérence des données, comme expliqué dans la suite.

  ], Jablin et al. introduisent CGCM, un système qui vise exactement la même problématique. CGCM, tout comme la méthode proposée ici, est focalisée sur le transfert d'allocation mémoire entière. Alors que le processus ici est entièrement statique, CGCM prend des décisions à l'exécution. Le support exécutif gère les pointeurs en général, y compris ceux qui renvoient sur le milieu d'une structure allouée sur le tas, ce qui n'est pas géré par la méthode que je propose.

There is divergence when the code includes conditional branching and not all threads take the same execution path.

When a branch occurs and threads diverge, both paths are executed sequentially and the PEs corresponding to the threads that took the other path are disabled ; they are predicated.

The packing ratio indicates on the average how many instructions are packed in the VLIW by the compiler with respect to the VLIW width.

Some recent solutions like Nvidia Zero-Copy allow mapping directly the host memory in the GPU virtual space and thus avoid the explicit copy. However, they do not provide good performance in the general case.

Most of the time, variables used in the region description are not modified by the isolated statement and we can safely use W(s, σ). Otherwise, e.g. a[i++]=1,m e t h od sd e t a i l e di n[ Creusillet & Irigoin 1996b] must be applied to express the region in the right memory state.

As the write regions are empty for src,t h i sc o r r e s p o n d st ot h el o a d s .

CUDA 5 and Kepler GPU, which should both be released by the end of 2012, bring a potential solution introducing what Nvidia calls Dynamic Parallelism.

A source file in C.

Also referred as "module level."

(a) single precision. (b) Double precision.

"it cannot continue forever. The nature of exponentials is that you push them out and eventually disaster happens."

Remerciements

Acronyms PTX Parallel Thread eXecution. 19, 88, 195 PyPS Pythonic PIPS. 142, 145,[147][148][149][150]300 QUAST Quasi-Affine Selection Tree. 91, 283 RAM Random Access Memory. 76 RDG Reduced Dependence Graph. 121-123 SAC SIMD Architecture Compiler. 144, 145 SDK Software Development Kit. 20, 202 SFU Special Function Unit. 40, 50, 268 SIMD Single Instruction stream, Multiple Data streams. 13, 19, 37, 44, 45, 47, 50, 51, 140, 157, 159, 191, 269-272, 297, 303, 305 SIMT Single Instruction stream, Multiple Thread streams. 32, 37 SLOC Source Lines Of Code. 146, 147, 149 SMP Symmetric MultiProcessing. 83 SPDD Single Program Distributed Data. 77, 280 SSA Simple Static Assignment. 195 SSE Streaming SIMD Extension. 3, 45, 140, 147, 148, 151, 270, 297, 300 TLP Thread Level Parallelism. 39, 41, 44, 45, 47, 48,[52][53][54] 132,[268][269][270][271] 273, 293 TR Textual Representation. 140, 141, 297 VLA Variable Length Array. 68, 133, 279, 293 VLIW Very Long Instruction Word. 2, 23, 41,[43][44][45][46][47] 52, 53, 132, 146, 255,[269][270][271] 299 XML Extensible Markup Language. 150

Résumé en français

Most of the work described in this chapter was published in [Amini et al. 2011b (perso), Amini et al. 2011c (perso), Guelton et al. 2012 (perso), Amini et al. 2012a (perso)].

The next chapter presents the different steps performed on the sequential input code to achieve parallelization and GPU code generation. for (i = 0; i < ni; i++) { for (j = 0; j < nj; j++) { C Section 7.5.2,P age191, shows how this simple case exhibits speedup ranging from 1.96 up to 5.75. In this case the backend compiler cannot do anything and thus it is critical to apply this transformation at source level.

for(i=0; i<N; i++) { // u1 [ i ] and u2 [ i ] are constant // references in the inner loop for(j=0; j<M; j++

+u1_s+u2_s; } } copy_to_accel(u1); copy_to_accel(u2); k_out(A,u1,u2);

(e) Kernel when mapping the outer loop.

Conclusion

This section surveys a well-known transformation, scalarization, and shows how the implementation in PIPS,l e v e r a g i n ga r r a yr e g i o na n a l y s i s ,c a nh e l pr e d u c i n gt h em e m o r y footprint and improving overall performance when targeting GPU.

The GPGPU puts some unusual constraints in such state-of-the-art transformation:

Chapter 6. Management of Multi-GPUs /⇤ Initialize array. ⇤/ init_array();

/⇤ E: =A ⇤B ⇤/ for(i=0;i<=ni-1;i++) for(j=0;j<=nj-1;j++) { E

Code Generation for StarPU

After the task generation has been performed, the code has to be decorated with the pragmas that will trigger the StarPU runtime to catch calls to task functions.

All arrays in the code are registered with StarPU as buffers. StarPU then takes care of the dependences between tasks, based on the arguments, at runtime. It also automatically transfers the data if the task is expected to be run on a GPU. The host code calls StarPU tasks asynchronously. Some global synchronizations can be inserted, or alternatively synchronizations based on the availability of a buffer, i.e., waiting for a buffer to be filled.

for(i = 0; i <= N-1; i += 1)

A