Arlie Russell Hochschild

Le

Version précédent la mise en forme de la revue

Initialement publié aux Presses de l'université de Californie -Berkeley en 1983, l'ouvrage d'Arlie R. Hochschild The Managed Heart est un classique de la sociologie du travail et de l'approche sociologique des émotions. Sa traduction française vient combler un manque certain pour le lectorat francophone qui ne disposait jusque-là que de quelques articles de son auteure (en particulier « Travail émotionnel, règles de sentiments et structure sociale », dans Travailler, n°9, 2003), pour discuter et mobiliser son concept phare -le travail émotionnel.

L'objet du livre est d'interroger les mécanismes et les coûts subjectifs du décalage entre les sentiments ressentis et les émotions exprimées. Contre un psychologisme rapide qui voudrait que l'émotion soit le produit direct d'un stimulus, Hochschild soutient que s'intercalent souvent, entre le stimulus et la réponse, des actes d'évaluation, de catégorisation et de gestion de l'émotion. Ce travail sur les émotions, plus ou moins conscient, permet de masquer une émotion véritablement éprouvée, ou de transformer un sentiment pour le rendre plus acceptable. S'il s'agit bien là d'un travail, c'est qu'il puise dans notre énergie et qu'il a pour finalité de modifier le cours des actions sociales. Maintenir le climat serein d'un repas de famille qui s'envenime, être aimable avec une clientèle agaçante, ou trouver l'assurance d'exiger le respect d'une règle quand on a peur des autres, suppose un travail émotionnel.

Pour investiguer cette dimension du comportement social -et montrer les influences sociales qui l'oriente -, l'auteure construit et mobilise trois ensembles de données : un questionnaire administré auprès de 261 étudiants portant sur des récits d'expérience émotionnelle, une enquête qualitative dans le monde des hôtesses de l'air (basée sur une cinquantaine d'entretiens avec les différents types de personnels d'une compagnie aérienne, l'observation des séances de formation des hôtesses et stewards, l'analyse des publicités de la compagnie), et une enquête plus modeste auprès des agents de recouvrement de la même compagnie.

Dans une première partie (intitulée « La vie privée »), l'auteure s'appuie notamment sur les récits d'émotions des étudiants pour construire son cadre analytique. S'appuyant sur Sigmund Freud, Hochschild rappelle que l'émotion a une « fonction signal » : elle nous indique la manière dont nous entrons en relation avec la situation, elle nous en apprend sur nous mêmes et sur nos tendances à l'action. Lorsque l'émotion peut s'exprimer sans risque de sanction sociale, il n'y a pas de travail émotionnel. Cependant, il apparaît souvent des situations problématiques dans lesquelles savoir se comporter s'apparente à une performance d'acteur. Le « jeu » avec les émotions peut être « de surface », lorsque, « comme pour les individus observés par Erving Goffman, l'action se situe dans le langage du corps » (p.56). Mais il peut aussi se fabriquer « en profondeur », lorsque le sentiment a été « induit par l'individu luimême » (p.57). Les étudiants témoignent d'hésitations et de réflexions sur ce qu'ils auraient dû, pu, ou voulu ressentir. Des glissements émotionnels apparaissent lorsque la situation est envisagée sous d'autres aspects. Ce faisant, ils s'efforcent de ressentir l'émotion qui leur semble appropriée.

Conformément à la tradition interactionniste que l'auteure revendique, le travail sur les émotions s'organise en fonction des rôles que nous nous attribuons et des attentes à notre égard que se forgent les personnes impliquées dans la situation. Les caractéristiques socialesle statut, le pouvoir, l'âge, le genre, etc. -sont déterminantes. Mais le travail émotionnel est aussi guidé par des « règles de sentiments », qui oeuvrent à « l'élaboration du sens du droit et du devoir qui gouverne les échanges émotionnels » (p.77). Potentiellement imposées par des cadrages cérémoniels (la joie de mise à un mariage), ces règles de sentiment varient historiquement et culturellement. La gestion individuelle des émotions n'est donc pas laissée à la discrétion de chacun.

La notion de travail émotionnel posée, l'auteure s'emploie à montrer les différences existant entre les interactions de la sphère privée et les interactions en contexte professionnel. Dans les relations amoureuses, amicales, ou familiales, les règles de sentiment seraient, selon Hochschild, plus intériorisées, plus inconscientes, et vécues comme plus naturelles que dans les autres relations sociales. Le travail émotionnel serait une contribution au lien social, un don à la collectivité. Mais ce don n'est pas gratuit. Il appelle au contraire à une réciprocité. L'émotion travaillée et partagée fonctionne comme une monnaie d'échange immatérielle. L'échange s'appuierait inconsciemment sur « un registre comptable à deux colonnes -'dû' et 'reçu' -pour la reconnaissance, l'amour, la colère, la culpabilité » (p.98), dans un contexte où « nous sommes libres de remettre en cause le taux de change usuel et d'en négocier un nouveau » (p.105). Dans la vie professionnelle, au contraire, les règles de sentiments seraient imposées et feraient l'objet d'un contrôle.

Dans la seconde partie (intitulée « la vie publique »), l'auteure s'intéresse au problème de la « transmutation » du système émotionnel privé dans la vie professionnelle. La sélection des hôtesses de l'air, leur formation, leur activité concrète et le contrôle exercé par leur hiérarchie sont mis au coeur de l'analyse. Reprenant l'idée du « marché de la personnalité » diagnostiqué par Charles W. Mills dès les années 1950 (dans Les cols blancs, Maspero, 1951), Hochschild montre que les hôtesses sont sélectionnées en fonction de leur capacité à faire montre d'une personnalité non agressive, chaleureuse, sincère, « naturelle », à la fois calme et capable d'enthousiasme (p.116). Il s'agit bien ici seulement de paraître, car l'objet des formations sera précisément d'apprendre aux hôtesses à ne pas montrer leurs véritables sentiments.

Pour la compagnie, la satisfaction de la clientèle nécessite une qualité de service qui passe par une formation et un contrôle de l'état d'esprit et du comportement des hôtesses. L'idée d'un transfert vers le monde du travail des compétences émotionnelles féminines traditionnellement développées dans la sphère domestique est à peine voilée. Les hôtesses sont invitées à construire des représentations spécifiques : la compagnie est une « grande famille » ; l'avion est leur salon, et les passagers leurs invités ; lorsqu'un passager est outrancier, elles devraient imaginer qu'il s'agit d'un jeune enfant agité. En toute circonstance, le sourire est de rigueur et l'agacement est proscrit. L'activité de travail est décrite par les hôtesses comme une succession d'efforts réalisés pour se conformer aux règles de sentiments imposées par la compagnie. Faisant l'objet d'une codification et d'un énoncé public, les règles de sentiment au travail réduisent les marges de manoeuvre des hôtesses. Elles doivent accepter davantage d'échanges inégaux que dans la sphère privée. Intégrée à la relation de subordination salariale, la gestion des émotions acquiert une dimension marchande. Mobilisant le vocabulaire marxiste, Hochschild considère que le travail émotionnel n'a plus seulement une valeur d'usage, il devient une valeur d'échange. Cependant, les coûts psychiques de l'instrumentalisation commerciale des sentiments et de l'implication subjective (le « prix des sentiments ») seraient sous-considérés. Les hôtesses ressentent en effet une dépossession de leur identité, une « confusion identitaire » (p. 151). En gouvernant les émotions, se crée un écart entre le rôle et le soi, entre l'illusion et l'authenticité, entre les identités professionnelle et personnelle. Selon l'intensité du trouble, apparaissent gêne, cynisme, aliénation ou retrait émotionnel.

Centré sur le travail émotionnel « interne » de ces professionnelles, l'ouvrage ne néglige pas les facteurs sociaux qui le déterminent : l'augmentation de la taille des avions et de la longueur des trajets lorsque les effectifs sont constants, ou l'augmentation de la concurrence dans le secteur aérien américain dans les années 1980, notamment, ont augmenté le niveau de stress des hôtesses de l'air. Plus largement, Hochschild intègre la description fine de l'activité émotionnelle aux structures socio-économiques : l'augmentation du taux d'emploi féminin dans la seconde moitié du 20 e siècle, la surreprésentation des femmes dans les emplois de service et la surreprésentation du travail émotionnel dans les classes moyennes. L'intérêt de la perspective du genre s'observe dans le chapitre consacré aux agents de recouvrement de la compagnie aérienne étudiée : chargés de réclamer les impayés, ces agents, majoritairement masculins, doivent s'interdire d'avoir de l'empathie, considérer les débiteurs comme des voleurs potentiels, agresser verbalement le débiteur et savoir s'énerver « comme un homme » (p.165). Enfin, les influences sociales sur le comportement émotionnel sont interrogées sur le terrain de l'éducation. En corrélation avec le milieu social, certains styles éducatifs propres aux classes moyennes et supérieures favoriseraient le contrôle de et par les sentiments. En lien avec le genre, l'éducation expliquerait en partie pourquoi, dans le questionnaire adressé aux étudiants, « les femmes plus que les hommes (33% contre 18%) utilisaient spontanément le langage du travail émotionnel pour décrire leurs sentiments » (p.187).Certaines analyses semblent désormais datées : celles sur le genre ou les types d'éducation se comprennent dans le contexte des années 1970. Mais la thèse centrale de l'ouvrage demeure d'une grande actualité : il nous est dit qu'on ne peut étudier le travail sans s'intéresser à l'activité ellemême et à ce qu'elle implique quant à la maîtrise de soi et les interactions avec autrui, en lien avec le fonctionnement de l'émotion. On sait combien les problèmes de la souffrance au travail et des « risques psychosociaux » sont cruciaux pour la santé publique et le développement économique : ils ne peuvent se comprendre sans analyser la manière dont les sentiments sont pris comme objet de gestion, par les organisations comme par les travailleurs eux-mêmes.

4

La traduction française de cette oeuvre pionnière de la sociologie des émotions pourra donc permettre aux études déjà nombreuses dans la sociologie francophone sur les émotions au travail d'affiner la compréhension des mécanismes du travail émotionnel et de l'interpénétration des domaines privés et publics. Le travail d'Hochschild offre en outre un regard proprement sociologique sur la souffrance au travail : il vient utilement compléter la perspective de la psychodynamique du travail, théoriquement fondée sur la psychanalyse, qui, par d'autres voies -les dénis du risque émotionnel comme stratégies de défense -aboutit au même constat d'une aliénation affective des travailleurs.

Au-delà de la sociologie du travail, l'ouvrage ouvre également à une interrogation sur l'individualisme et le culte de l'authenticité dans la société contemporaine. « Plus les émotions sont contrôlées, plus nous attachons d'importance à celles qui ne le sont pas » (p.213). Dans un dialogue avec la philosophie, Hochschild suggère que la condition affective contemporaine serait marquée par un néoromantisme, où, paradoxalement, il faudrait travailler à retrouver des sentiments spontanés (une problématique travaillée notamment par Eva Illouz dans Les sentiments du capitalisme, Seuil, 2006). L'ouvrage comporte enfin une importante annexe sur la théorie sociologique de l'émotion, dans laquelle Hochschild -qui puise ses inspirations chez les classiques que sont Marx, Durkheim, Mills et Goffmandiscute de sa conception de l'émotion et des apports et limites de Darwin, Freud, Dewey, ou Collins.

Julien Bernard Université Paris Nanterre


Écologie des affects

par Julien Bernard D'où viennent nos émotions ? Sont-elles propres à notre sensibilité ou sont-elles produites par notre environnement ? Et comment deviennent-elles collectives ? Questions majeures, que L. Quéré entend reprendre à nouveaux frais.

Dé-subjectiver (et désintellectualiser) l'émotion

L'orientation proposée suppose de se départir de la tendance à définir l'émotion par l'expérience du « Je » et par les sensations qui l'accompagnent. La subjectivation produite par le primat accordé au « vécu » dans le sens commun risquerait en effet de réduire les émotions à des états ou à des processus corporels et mentaux. Or, en s'appuyant sur Wittgenstein, l'auteur estime que ce cadrage réifie les émotions et sousestime l'importance de l'action en cours, de l'environnement, et du caractère expressif des émotions. Il conduirait en outre à postuler l'antécédence du psychisme sur le comportement et donc à faire des émotions un révélateur rapide de l'identité et des valeurs individuelles. Au contraire, penser l'émotion dans l'action permettrait de voir l'émotion davantage comme un mode de comportement en transaction avec l'environnement que comme un « état ».

  Loin d'établir un dualisme individu / environnement, l'émotion manifesterait en effet l'interdépendance entre « l'intérieur » et « l'extérieur » des individus. Dans une approche écologique, elle serait une partie d'un processus plus global d'engagement, de rupture et d'ajustement avec l'environnement qui « se développe dans une structure d'intrigue » (p. 404). Avec Merleau-Ponty, l'auteur estime qu'elle serait « une variation de nos rapports avec autrui et avec le monde lisible dans notre attitude corporelle » (p. 49). Plus précisément, l'émotion serait insérée dans un cours d'action, duquel elle émerge, et qui serait ensuite modifié par les nouveaux comportements engendrés par l'émotion. En fonction de ses orientations motrices et mentales, l'individu projette en effet sur la situation un sens et une idée de son déroulement. L'émotion surgirait lors des ruptures de coordination avec le monde, lesquelles opéreraient un « désajustement des fonctions organiques » (p. 84). Suivant la théorie des émotions de Dewey, l'auteur prend ses distances tant avec le schéma béhavioriste « stimulus-réponse » qu'avec les conceptions qui donnent le primat à l'appréciation cognitive de l'acteur (appraisal). Pour Louis Quéré, il ne saurait exister de perception a priori neutre suivie d'une évaluation de la situation donnant lieu à une émotion. Selon lui, « le stimulus est toujours précédé d'une coordination sensori-motrice, dont il émerge ». Le jugement de l'objet en cause est aussi modulé par l'émotion elle-même (« l'émotion configure ce qui la provoque »). L'auteur cherche ainsi à cerner la composante cognitive de l'émotion « sans surintellectualiser l'appraisal. » La description du processus émotionnel proposée relie perception, évaluation, émotion et reconfiguration de l'action. La rupture de coordination engagerait un premier mouvement, dit de « valuations immédiates » des impressions. Celles-ci seraient ressenties de manière positive ou négative, dans le continuum entre attraction et répulsion ; elles manifesteraient le trouble affectif et orienteraient la saisie de l'événement. Dans un deuxième temps s'enclencherait une triple dynamique : pratique (tendance à l'action) ; cognitive (configuration de l'objet-stimulus qui est « discriminé, sélectionné et identifié » (p. 96)) ; et affective (attribution de valeur à la situation et à son émotion). L'utilité de l'émotion serait d'être un signifiant pour autrui (p. 85), mais aussi pour soi. En sélectionnant des données et en les ré-agençant, elle activerait « l'organisation de l'expérience », ce que l'auteur nomme « travail de l'émotion ». Dans ce schéma, la dimension cognitive n'est pas absente, mais seconde. Les véritables jugements ne se produiraient en effet qu'ensuite, dans un troisième temps, lorsque, après « enquête », l'objet de l'émotion, et l'émotion exprimée ellemême, font l'objet d'une analyse morale (p. 191). Les émotions permettraient alors de sonder nos valeurs rétrospectivement, par analyse réflexive. Une théorie de « l'émotion distribuée » et des « habitudes » Loin d'être centrée sur l'individu, la théorie proposée considère comme essentielles les caractéristiques et les variations du cadre de la situation. L'émotion serait le produit de la rencontre de deux dynamiques : celle, subjective, de l'individu, et celle, « objective » (p. 211), des variations de l'environnement. Les « choses » de l'environnement -les propriétés du cadre, le comportement des autres individus…auraient selon l'auteur des « qualités affectives immédiates, qui sont senties » (p. 105). Les qualités sensorielles éprouvées par l'organisme comporteraient per se une qualité affective, la perception étant soumise à « l'action des choses sur les organes corporels qui la subissent et y répondent » (p. 120). Le point de vue selon lequel « les choses présentent d'emblée des qualités affectives/émotionnelles entremêlées à leurs qualités sensorielles » (p. 184) conduirait à considérer que l'émotion n'est pas foncièrement dans les individus : elle est « distribuée » sur les différents acteurs et paramètres du cadre. Parallèlement, une autre modalité de distribution des émotions se trouverait dans leur institutionnalisation (p.103). La mise en forme de l'émotion s'appuierait sur l'« arrière-plan de la connaissance commune » (Garfinkel) ou l'« attitude naturelle » (Schütz) ayant cours dans le milieu social. Autrement dit, les façons ordinaires de se comporter et les « valuations » produites dans le milieu agiraient comme des institutions au sens durkheimien, i.e. des manières de faire déjà-là, dépersonnalisées, servant de guides pour produire l'émotion appropriée. Les émotions s'apparenteraient ainsi à des « énergies organiques canalisées » donnant lieu à des habitudes émotionnelles, par incorporation. Celle-ci permettrait de comprendre comment les émotions, faits organiques, peuvent être « socialement construites ». Des émotions « moulées dans une forme collective » Quid, à partir de cette théorie, des émotions collectives ? Dans la seconde partie du livre, Louis Quéré entreprend un examen critique des différentes théories de l'émotion collective avec pour souci de la comprendre sans perdre de vue le caractère corporel et dynamique des émotions individuelles, mais sans pour autant la considérer comme simple agrégation d'émotions individuelles. Ainsi le paradigme théorique de la foule, basée sur l'idée de « contagion » des émotions, négligerait-il l'importance du partage d'idéaux. La théorie durkheimienne, à l'inverse, pêcherait par un poids excessif donné aux représentations par rapport au corps. L'approche intersubjectiviste phénoménologique, qui conceptualise la synchronisation des perceptions, sous-estimerait le poids de l'organisation du groupe et de ses « médiations institutionnelles » (p. 263). L'approche « collectivistenormativiste », qui suppose la conformation des individus aux émotions de leurs groupes, comporterait le problème d'attribuer aux normes un pouvoir causal. Enfin, les approches de psychologie sociale pointant l'importance de l'identification à un groupe buteraient sur la difficulté à préciser la notion d'affiliation, très variable, étant donné que « les collectifs émotionnels (…) [peuvent être] des collectifs occasionnels et éphémères, aux contours instables » (p. 324). Seule l'anthropologie du rituel inspirée de Mauss permettrait de comprendre la dynamique et le « moulage dans une forme collective » des émotions individuelles. Ce qui se met en commun dans ce cadre théorique serait davantage des comportements que des ressentis. Le collectif serait alors produit par des « accomplissements concertés médiatisés par des usages, des objets, des dispositifs » (p. 328). Dans ces séquences d'action, le rythme et la motricité favoriseraient l'émergence d'émotion collective. Par suite, « la symbolisation de celle-ci dans un espace public, à l'aide de médiations matérielles et symboliques » (p. 352) donnerait à l'émotion collective un caractère public. Celui-ci s'accompagnerait d'une prétention de l'émotion à s'imposer comme légitime, ce qui lui donnerait une importance politique. Cette transition amène l'auteur à un dernier chapitre très différent dans la forme des précédents, dans la mesure où il aborde une émotion (ou un sentiment) en particulier : le ressentiment. Dans un contexte social actuel qui serait marqué par une prépondérance des « passions tristes », le ressentiment est analysé au regard des faits sociaux que constituent le développement des fake news sur les réseaux sociaux, les tendances à ne pas séparer faits et opinions, celle de s'entourer de gens qui pensent comme soi, ou la méfiance grandissante à l'égard des autorités culturelles… L'auteur s'interroge sur la potentialité du ressentiment à produire des mouvements sociaux, tels que, par exemple, celui des Gilets jaunes, mais aussi sur celle de l'éducation à constituer un remède contre les « passions tristes ». Une théorie à l'épreuve de la variété des expériences L'ouvrage comporte à la fois une analyse critique assez complète des théories sociologiques et philosophiques sur les émotions et une puissante théorie des émotions basée sur une synthèse des apports de Dewey et Mauss. Il comporte cependant un certain nombre de propositions ou de parti-pris -assumés -qui ne manquent pas de poser question -ce qui n'est pas la moindre qualité de l'ouvrage. La focalisation sur l'action semble ainsi exclure du champ de l'analyse les émotions se produisant sans que le cours d'action n'ait réellement d'orientation préalable, celles qui émergent par exemple dans la solitude de ses souvenirs, ou certains états affectifs qui n'entraînent pas de tendance à l'action (ennui, nostalgie…) La théorie proposée ne rend guère compte, par ailleurs, des différences entre, d'une part, les émotions plutôt liées aux réactions instinctives qui sont parfois activées avant même l'identification du stimulus, et, d'autre part, les émotions plus cognitives, liées à une identification et à une évaluation progressives qui donnent lieu à un éveil (arousal) ou une « montée » de l'émotion plus longue et plus lente : elle se situe entre les deux. La théorie de « l'émotion distribuée » sur les variations de l'environnement, lesquelles comporteraient des « qualités affectives immédiates », pourra, elle aussi, surprendre. En voulant donner toute sa part aux influences extérieures sur la corporéité, l'auteur ne considère pas -ou pas clairement -que, toutes objectives qu'elles soient, ces « choses » sont subjectivement perçues, investies de sens, de qualité. On pourrait donc objecter que la « qualité affective des choses » n'existe pas hors de leur saisie par les individus et que les impressions sensorielles ne produisent pas mécaniquement des affects ad hoc. Enfin, si le travail de l'émotion (ce que nous fait l'émotion) est mis à l'honneur, le travail sur l'émotion (ce que nous en faisons) pourra sembler vite évacué. L'émotion semble en effet apparaître immédiatement comme un comportement expressif. Or la

perception des normes sociales peut entrer en ligne de compte dans la mise en forme de l'émotion. En fonction des normes, nous pouvons non seulement camoufler ou amplifier l'expression de nos émotions, mais aussi oeuvrer à les transformer si nous considérons qu'elles ne seront pas comprises ou admises par les autres. La perception de la norme peut ainsi être considérée comme une composante intrinsèque de la genèse émotionnelle, et non seulement, comme le suggère l'auteur, comme un facteur externe agissant après coup, lorsque nous faisons face aux contraintes de la situation.Publié dans laviedesidees.fr, le 24 février 2022


Abderaouf Nassim Amalou
email: abderaouf.amalou@irisa.fr

Elisa Fromont
email: elisa.fromont@irisa.fr

Isabelle Puaut
email: isabelle.puaut@irisa.fr

Fast and Accurate Context-Aware Basic Block Timing Prediction using Transformers

Keywords: Execution time estimation, Machine learning, long-short term memory, transformers model

This paper introduces ORXESTRA, a context-aware execution time prediction model based on Transformers XL, specifically designed to accurately estimate performance in embedded system applications. Unlike traditional machine learning models that often overlook contextual information, resulting in biased predictions for individual isolated basic blocks, ORXESTRA overcomes this limitation by incorporating execution context awareness. By doing so, ORXESTRA effectively accounts for the processor micro-architecture without explicitly modeling micro-architectural elements such as caches, pipelines, and branch predictors. Our evaluations demonstrate ORXESTRA's ability to provide precise timing estimations for different ARM targets (Cortex M4, M7, A53, and A72), surpassing existing machine learningbased approaches in both prediction accuracy and prediction speed.

Introduction

Over the past few years, numerous studies have explored the use of machine learning (ML) techniques for predicting performance metrics such as execution time, energy consumption, and resource utilization [START_REF] Abderaouf | CATREEN: Context-Aware Code Timing Estimation with Stacked Recurrent Networks[END_REF][START_REF] Maas | A taxonomy of ML for systems problems[END_REF][START_REF] Mendis | Ithemal: Accurate, portable and fast basic block throughput estimation using deep neural networks[END_REF][START_REF] Jun S Shim | DeepPM: transformer-based power and performance prediction for energy-aware software[END_REF][START_REF] Wang | Machine learning in compiler optimization[END_REF]. These techniques have become increasingly popular for complex processors, for which developing a precise analytical model of the micro-architecture is challenging, error-prone, or sometimes impossible to perform, due to a lack of detailed documentation and human expertise to design the model.

The basic block 1 (BB) granularity is frequently used to profile program performance (execution time or energy consumption). However, existing techniques for BB timing estimation, for example, tools [START_REF] Mendis | Ithemal: Accurate, portable and fast basic block throughput estimation using deep neural networks[END_REF][START_REF] Sỳkora | GRANITE: A Graph Neural Network Model for Basic Block Throughput Estimation[END_REF], consider each BB in isolation, which results in biased solutions that do not account for the impact of pipelines, branch predictors, and cache memory, thereby lacking consideration of execution context in BB timing estimation. Some studies have attempted to address this limitation by incorporating context awareness into BB execution time estimation using stacked Long-short Term Memory (LSTM) networks [START_REF] Abderaouf | CATREEN: Context-Aware Code Timing Estimation with Stacked Recurrent Networks[END_REF]. Nevertheless, LSTMs struggle with long sequences (exceeding 200 tokens, as empirically shown in [START_REF] Khandelwal | Sharp nearby, fuzzy far away: How neural language models use context[END_REF]). More recently, Transformers architectures have shown promise in overcoming the limitations of LSTMs, leading researchers to use them in binary analysis [START_REF] Li | Palmtree: Learning an assembly language model for instruction embedding[END_REF].

This paper presents ORXESTRA 2 , for cOntext-awaRe eX-Ecution Time eStimation using TRAnsformers. ORXESTRA accurately predicts the execution time of BBs 3 within compiled binaries. Inspired by deep learning techniques commonly used in natural language processing, ORXESTRA uses a type of ML technique named Transformers [START_REF] Vaswani | Attention is all you need[END_REF], a particular attention-based architecture, to deliver fast and precise predictions. More specifically, our approach predicts execution time at the BB granularity using a Transformers XL [START_REF] Dai | Transformer-xl: Attentive language models beyond a fixed-length context[END_REF], a recurrent variant of Transformers. The timing prediction of a BB uses the execution context of the BB (i.e., instructions executed before the BB under study). Furthermore, unlike LSTM-based models or traditional Transformers, ORXESTRA 1 A Basic Block (BB) is a straight-line sequence of instructions with no branches in, except to the entry, and no branches out except at the end. 2 The code and dataset for training ORXESTRA will be made available upon acceptance of the paper. 3 Our approach is portable for higher granularity like code segments, the choice of BB was made for the sole purpose of comparison with existing work.

provides accurate predictions for long basic blocks, typically present in real-world applications (e.g., see the study conducted in [START_REF] Jun S Shim | DeepPM: transformer-based power and performance prediction for energy-aware software[END_REF]) and long basic block sequences. ORXESTRA further includes automatic recognition of the structure of machine code (list of instructions, operands, addressing modes), avoiding the cumbersome task of manually expressing this information from hardware documentation.

ORXESTRA has been tested on a range of Arm embedded processors, each with its own level of complexity. Although it can be used for x86 architecture, we focused on testing our solution across architectures of varying complexity, and the ARM architecture was a good fit for this purpose.

The tested processors include the Cortex-M4, a basic pipelineonly processor [START_REF]ARM Cortex-M4 Processor[END_REF], the Cortex-M7 [START_REF]ARM Cortex-M7 Processor[END_REF] which features a data cache, an instruction cache, and a branch predictor, the super-scalar Cortex-A53 [START_REF]ARM Cortex-A53 Processor[END_REF], and the out-of-order Cortex-A72 [START_REF]ARM Cortex-A72 Processor[END_REF]. Our experimental results demonstrate that ORXES-TRA surpasses LSTM-based models [START_REF] Abderaouf | CATREEN: Context-Aware Code Timing Estimation with Stacked Recurrent Networks[END_REF][START_REF] Mendis | Ithemal: Accurate, portable and fast basic block throughput estimation using deep neural networks[END_REF] and traditional Transformers-based solution [START_REF] Li | Palmtree: Learning an assembly language model for instruction embedding[END_REF], showing a 28% improvement in estimation accuracy over the best competitor CATREEN (on average across all targets), while also being 98% faster in prediction timing than CATREEN [START_REF] Abderaouf | CATREEN: Context-Aware Code Timing Estimation with Stacked Recurrent Networks[END_REF].

ORXESTRA comes as a standalone tool that can be used directly by the application developer to profile programs' performance. In this case, its prediction efficiency allows ORXES-TRA to replace costly performance measurement campaigns, directly on the target processor or on a processor simulator. ORXESTRA can also be used as a companion tool for the compiler, allow the compiler to quickly estimate the quality of the compilation optimizations adopted, for example when using iterative compilation [START_REF] Chen | Deconstructing Iterative Optimization[END_REF][START_REF] Peter | Iterative Compilation[END_REF].

The remainder of this paper is organized as follows. Section 2 presents an overview of related works that have utilized Transformers or LSTMs to learn the representations of basic blocks (BBs) or their performance. Section 3 contains a comprehensive presentation of ORXESTRA, both regarding training and actual predictions. The dataset, competitors, processor targets and the experimental setup are described in Section 4. The performance of ORXESTRA during pretraining and fine-tuning is compared to against state-of-the-art techniques in Section 5. Section 6 discusses the limitations of ORXESTRA and outlines potential future research directions. Lastly, Section 7 concludes the paper.

Related works

Performance estimation techniques using heuristics have been developed over the years to guide code optimization. Such techniques range from simple methods, such as counting the number of instructions, to complex machine learning (ML) techniques that predict the performance of code snippets, using, for example multilayer perceptrons, recurrent neural networks such as Long Short-Term Memory (LSTM), and, more recently, Transformers. This section provides an overview of works using ML-based techniques to estimate execution time for complex micro-architectures, as well as related research for automatic learning of the representation of machine code.

Execution time estimation ML-based techniques

ITHEMAL [START_REF] Mendis | Ithemal: Accurate, portable and fast basic block throughput estimation using deep neural networks[END_REF] is the first model utilizing a hierarchical multi-scale RNN, specifically LSTM layers, to predict basic block performance, focusing on best-case execution time. It effectively captures interactions between instructions within the same basic block by sequentially processing each element (operations and operands). Alongside, the authors introduced BHive [START_REF] Chen | BHive: A benchmark suite and measurement framework for validating x86-64 basic block performance models[END_REF], a dataset of basic blocks for X86. However, BHive's data collection methodology, including isolating basic blocks, ensuring first-level cache memory accesses, and omitting branch instructions, does not truly represent processor execution. Our aim, in contrast to BHive, is to realize execution time representation authentic to the processor, enabling precise differentiation at the cycle level.

Granit [START_REF] Sỳkora | GRANITE: A Graph Neural Network Model for Basic Block Throughput Estimation[END_REF] employs a Graph Neural Network (GNN) approach to model the dependencies between instructions within a basic block. This method relies on the user's expertise to define dependencies, which are then enforced by the GNN structure. In contrast, ORXESTRA adopts a matrix representation for depicting these dependencies. Specifically, it utilizes Transformers-based mechanisms, where dependencies are learned through matrix attention techniques. ORXESTRA's approach is more autonomous, enabling it to uncover microarchitectural nuances without the need for predefined dependency structures by the user. This fundamental difference highlights ORXESTRA's ability to adaptively learn and discover intricate relationships within instruction sets, potentially offering a more efficient and user-independent pathway for modeling instruction dependencies.

DeepPM [START_REF] Jun S Shim | DeepPM: transformer-based power and performance prediction for energy-aware software[END_REF] in the same fashion as ITHEMAL [START_REF] Mendis | Ithemal: Accurate, portable and fast basic block throughput estimation using deep neural networks[END_REF] predicts the execution time of a basic block in isolation using a simplified Transformers architecture, the lack of details on the paper and the unavailability of the code, made the reproduction of this solution impossible.

In contrast to these three previous works, CATREEN [START_REF] Abderaouf | CATREEN: Context-Aware Code Timing Estimation with Stacked Recurrent Networks[END_REF] relaxes the assumption that a basic block is executed in isolation and predicts the execution time of a basic block based on its actual context of execution, using three LSTM layers. However, ORXESTRA advances beyond this by employing Transformers XL, offering superior management of execution contexts, including support for longer sequences and more efficient handling of historical information. The differences between CATREEN and ORXESTRA, however, extend beyond just the choice of model. In the experiments Section 5, and for a fair comparison, we align CATREEN to the same configuration as ORXESTRA, still the two approach differs in several key aspects:

• Input language: CATREEN uses a predefined input language where all memory addresses are a constant, while ORXESTRA learns its input language, treating each address as a separate token for better cache effect learning. • Deployment: ORXESTRA is more practical to use than CATREEN as it automates the calculation of all execution contexts leading to a basic block, which is not the case in CATREEN. • Dataset and targets: CATREEN has only been trained using a dataset formed of synthetic programs and on a single target (Cortex M7). In contrast, ORXESTRA uses a dataset formed of real programs and is trained on several different targets of varying complexity.

2.2 Learning code representation for performance estimation Word2Vec [START_REF] Mikolov | Distributed representations of words and phrases and their compositionality[END_REF] has been widely utilized in instruction representation learning, including performance models like ITHE-MAL and CATREEN. However, there are notable limitations associated with Word2Vec: the absence of embeddings for out-of-vocabulary words poses a challenge when working with specialized languages, such as assembly code, where the vocabulary may contain domain-specific or rare words. In addition, Word2Vec treats each word as a single entity, disregarding its multiple meanings. This can lead to ambiguities in word representations, which becomes problematic when dealing with data dependencies in pipelines, vital for performance analysis. While Word2Vec has its merits and offers computational efficiency, Transformers have surpassed it in various aspects by employing more advanced techniques and models. Transformers provide enhanced contextualization, improved handling of long-range dependencies, and achieve state-of-the-art performance. Palmtree [START_REF] Li | Palmtree: Learning an assembly language model for instruction embedding[END_REF] is a BERT-based [START_REF] Devlin | BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding[END_REF] assembly encoder specifically designed to capture the characteristics of machine instruction sequences by generating general-purpose instruction embeddings through pre-training in assembly language. However, our work differs from Palmtree in two significant ways. Firstly, we employ a Transformers XL architecture to overcome limitations present in Transformers like BERT when dealing with sequence size. Secondly, we concentrate on ARM assembly code instead of the Intel assembly code. Nevertheless, both Palmtree and ORXESTRA can be adapted to function with either Instruction Set Architecture (ISA). Other research efforts focusing on code representations, such Graphcodebert [START_REF] Daya Guo | GraphCode{BERT}: Pre-training Code Representations with Data Flow[END_REF] and CodeBert [START_REF] Feng | CodeBERT: A Pre-Trained Model for Programming and Natural Languages[END_REF], are primarily interested in high-level code for tasks like code debugging, commenting, or code generation. In contrast, our research focuses on low-level code (which makes it a different language to understand so they are not usable for our case) for performance estimation, aiming to improve performance estimation models for embedded systems.

3 Overview of ORXESTRA ORXESTRA uses Transformer-XL for performance prediction. Introduced in [START_REF] Vaswani | Attention is all you need[END_REF], Transformers are neural architectures initially designed for Natural Language Processing (NLP) tasks, like language translation and text summarization. They incorporate self-attention mechanisms, which enable the model to give appropriate importance to different parts of a sequence in the input data. This capability allows them to effectively capture the relationships between elements in a sequence and consider the overall context. However, the original Transformer architecture [START_REF] Vaswani | Attention is all you need[END_REF] has a fixed context window length, which limits its effectiveness for sequential data with long-term dependencies. This issue leads to 'context fragmentation, ' where longer inputs are broken into smaller, independently processed fragments. Dai et al. [START_REF] Dai | Transformer-xl: Attentive language models beyond a fixed-length context[END_REF] addressed this limitation by developing Transformer-XL (TXL), designed to more efficiently handle longer sequences.

Transformer XL

Transformers XL, as illustrated in Figure 1, marks a significant advancement in Transformers architectures by addressing the challenge of context fragmentation. One of its standout features is its capacity to "recall" or remember previously treated fragments of data. Instead of processing each fragment in isolation, Transformers XL integrates information from previous fragments, using this accumulated knowledge as a foundation when interpreting new data. This continuity is facilitated by a technique known as the "recurrence mechanism", which is somewhat akin to the workings of RNNs or LSTMs. In Figure 1, the "Old Memory" represents retained information, while the "New Memory" captures the latest processed data. As data flows through Transformers XL, these two components interact, ensuring a holistic understanding of sequences and interconnecting even distant pieces of information.

Architecture of ORXESTRA

ORXESTRA's architecture, as depicted in Figure 2, consists of several components. At its core, it utilizes a tokenizer called Sentencepiece [START_REF] Kudo | Sentencepiece: A simple and language independent subword tokenizer and detokenizer for neural text processing[END_REF]. A tokenizer is a standard NLP function that converts elements of a sequence into codes (often integers) that are suitable to be processed by a neural network.

The entire system uses two TXL, each serving a specific purpose. The first Transformers, displayed on the right side of the figure, focuses on creating an embedding representation of the execution context. The second Transformers, depicted on the left side of the figure, takes care of processing the basic block under analysis (BBUA), as it holds the main information to predict the execution time.

When these two Transformers operate, they produce an attention matrix (i.e. a weigh matrix -attention matrix-of size #𝑡𝑜𝑘𝑒𝑛 × #𝑡𝑜𝑘𝑒𝑛) with a fixed size of #𝑡𝑜𝑘𝑒𝑛 = 512. Essentially, when a context or a BBUA is presented as input, it is divided into smaller segments of 512 tokens. Thanks to the TXL memory mechanism, the system retains information from the previously processed 512 tokens while continuing to process longer sequences. Padding is applied at the beginning of each token sequence in order to obtain a sequence with a size that is a multiple of 512. ORXESTRA generates an attention matrix when processing the context (called here the global attention matrix), and a local attention matrix for the BBUA. These two matrices are concatenated and provided as input to a subsequent feed-forward neural network (bottom of the figure). This network estimates the average execution time (AET) associated with the BBUA.

Training of ORXESTRA

Training ORXESTRA consists in two main stages: pre-training each TXL (in practice, the same pre-trained model is used twice) and fine-tuning the entire architecture.

Pre-training. During the pre-training phase, the TXL is first trained using a self-supervised learning approach. Selfsupervised pre-training is a technique often used in machine learning to leverage unlabeled data in order to "initialize" the weights of an (often large) neural network and help it to learn better data representation to solve the final task. Training for this final task often involves a much smaller set of labeled data than the number of data used during the pre-training phase. A pretext supervised learning task is designed from the unlabeled data. For example, in NLP, a standard pretext task consists in predicting a (randomly chosen) "masked" word in a sequence. This learning task is thus supervised since the masked word is known from the model when parsing the sentence but the label is generated automatically (randomly) without any human intervention (thus, it is called "self-supervised" learning). In our working context, our goal for this pre-training phase is to enable the model to understand the structure of assembly instructions presented in a textual format. This is also achieved by masking random operations or operands within the instruction sequence and training the model to predict them as output in a self-supervised training scheme. By doing so, we can leverage a very large (unlabeled) dataset consisting of thousands of disassembled binary programs to learn a trustworthy representation of assembly code (i.e. to have a relevant initialization of the weights of our TXL) that can be used for the final training phase. This final training phase consists in "fine-tuning" the pre-trained models.

Fine-Tuning. In the fine-tuning stage, ORXESTRA is trained to predict the execution time of individual BB in context. In this phase, a specific set of programs, a target processor, and a measurement tool are employed. The execution times of basic blocks are measured (to obtain the training labels), and the corresponding instruction sequences are tokenized using the Sentencepiece technique presented before. To construct the training dataset, each BB's median execution time is considered, along with the tokenized BB itself and its associated context. The context size, which represents the number of basic blocks, serves as a hyperparameter for the TXL architecture.

3.4 Timing predictions using ORXESTRA Figure 3 shows an overview of the deployment of ORXESTRA (e.g. its inputs and outputs). Once trained, using ORXESTRA requires performing three main steps:

Dataset, Competitors, Targets and Setup

We detail the data used to train (for both phases) and test ORXESTRA in Section 4.1. Section 4.2 describes the baseline methods used to assess the effectiveness of ORXESTRA. The hardware and software setups are described in Section 4.3. Finally, Section 4.4 provides detailed information about the learning setup of ORXESTRA, including all relevant hyperparameters. 4 Because parametric loops handling is a complex problem itself.

Datasets and benchmarks

As stated before, training ORXESTRA involves two primary steps: pre-training and fine-tuning. In the pre-training phase, which learns the structure of machine code, a large dataset of programs from CodeNet [START_REF] Puri | CodeNet: A large-scale AI for code dataset for learning a diversity of coding tasks[END_REF] is used. This dataset contains about 900,000 C programs obtained from public submissions on competitive programming websites. These programs undergo cross-compilation to the target architecture using O3 optimization option and are disassembled using the GNU binary tool objdump. From each assembly program, we extract relevant information such as instruction addresses and identification of BBs limits (the start of a basic block and the end of it). It is important to note that the programs themselves are not executed during this process; they are used as a form of natural language data. The pre-training dataset is further used by Sentencepiece which performs its own learning phase and produces a model to tokenize our data. The Sentencepiece procedure progressively merges characters and character sequences, generating a vocabulary of smaller subword units based on statistical patterns. These subword units are then used by the model to tokenize the input text, facilitating language modeling and addressing the challenge of out-ofvocabulary [START_REF] Luong | Better Word Representations with Recursive Neural Networks for Morphology[END_REF] 5 words. Once trained, Sentencepiece is able to tokenize binary programs written in the target instruction set, enabling efficient processing by the Transformers.

To fine-tune ORXESTRA, a varied set of publicly available programs is employed 6 , namely The Algorithms7 , MiBench [START_REF] Matthew R Guthaus | MiBench: A free, commercially representative embedded benchmark suite[END_REF], and Polybench [START_REF] Tomofumi | Understanding polybench/c 3.2 kernels[END_REF]. Basic blocks and their respective contexts are extracted from these programs with a few modifications to obtain relevant data. For example, all instances of printf and system calls, which introduce redundant BBs and can potentially lead to overfitting, are eliminated from these programs. In Table 1, a summary of each benchmark suite is provided, including the number of programs in each dataset and the total count of basic blocks encountered during the execution of each program.

Baselines

ORXESTRA is compared to three context-agnostic timing predictors and one context-aware timing predictor. The first context-agnostic competitor is a Multi-Layer Perceptron (MLP) regressor, loosely referred to as a neural network (NN) [START_REF] Abderaouf | WE-HML: hybrid WCET estimation using machine learning for architectures with caches[END_REF]. Although not a naive approach, the neural network follows a feed-forward architecture that does not incorporate context information, and further requires a fixed-size input. For our NN implementation, we input 233 static features from the basic blocks, mainly consisting of the proportions of various machine instruction types (e.g., MOV, ADD, LDR) with the associated access type (direct, indirect, immediate). We use a greedy search algorithm to determine the optimal hyperparameters for the NN, including its number of hidden layers, the optimizer, the learning rate, and the loss function. Based on the validation dataset, the best parameters are: hidden layer sizes set to 512, 256, 128; learning rate set to 'adaptive' with an initialization at 0.001, and use of 'adam' solver. These hyper-parameters are coherent with what is used in [START_REF] Abderaouf | WE-HML: hybrid WCET estimation using machine learning for architectures with caches[END_REF].

Our second context-agnostic baseline is ITHEMAL [START_REF] Mendis | Ithemal: Accurate, portable and fast basic block throughput estimation using deep neural networks[END_REF], which uses LSTMs for execution time prediction. We reimplemented ITHEMAL from the original paper, porting the tokenization and embedding step of ITHEMAL to the ARM instruction set. Additionally, we tuned the model's hyperparameters to better fit the new data.

The third context-agnostic baseline is a re-implementation of BERT [START_REF] Devlin | BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding[END_REF] 8 here called "PalmTree ARM" [START_REF] Li | Palmtree: Learning an assembly language model for instruction embedding[END_REF]. This approach as Palmtree, involves pre-training BERT, and similarly to ORXESTRA, using the masked language modeling pretext task specifically designed for ARM assembly code. Palmtree ARM takes a single basic block as input and predicts its timing hence without considering the execution context information. Our objective with this competitor is to compare ORXESTRA with a Transformers model that performs similarly to ITHEMAL to assess the influence of the context awareness on the same type of neural architecture.

Finally, ORXESTRA is compared with its closest competitor, CATREEN, a context-aware execution time predictor introduced in [START_REF] Abderaouf | CATREEN: Context-Aware Code Timing Estimation with Stacked Recurrent Networks[END_REF]. We re-implemented CATREEN using the same hyperparameters as in the original study. Unlike ORXESTRA, which relies on Transformers XL, CATREEN uses LSTMs which are recurrent neural architectures. Transformers XL excel at capturing dependencies using self-attention mechanisms, while LSTMs struggle due to the vanishing gradient problem when capturing long-range dependencies. Additionally, Transformers architectures are easier to design and train compared to LSTMs, even with pre-training. 8 BERT (Bidirectional Encoder Representations from Transformers) is a type of language model based on the Transformers architecture.

In Table 2, a summary of the hyperparameters employed by all competitors is presented. To ensure a fair comparison, the context size parameter for both ORXESTRA and CATREEN was selected to be the same.

Hardware and software setups

To obtain the timing values (labels) necessary to train ORXES-TRA in the fine-tuning phase, we employ either a hardwarebased or a software-based approach, depending on the availability of each solution. The hardware solution is always preferred when available for its negligible interference with execution (probe effect).

The hardware-based timing instrumentation leverages the Joint Test Action Group (JTAG) interface for the hardware solution and utilizes the J-Trace Pro trace solution from Segger [START_REF] Segger | J-Trace PRO -The Leading Trace Solution[END_REF]. This allows us to connect to the JTAG interface of the target processors, specifically the Cortex-M4 and Cortex-M7 in our case. To generate execution traces, we use Ozone [START_REF]Ozone User Guide & Reference Manual[END_REF], a cross-platform debugger and performance analyzer, in conjunction with J-Trace Pro. These traces provide valuable information such as the cycle counter value, instruction address, opcode, operands, and corresponding assembly code for each instruction.

The software-based solution involves adding instrumentation code to measure the execution time of individual BBs in a program. We retrieve the execution trace using GDB (the GNU Debugger) to obtain context and assembly code for the timed BB.

Our experiments encompass a variety of Arm processors, summarized in Table 3. The Cortex-M4 processor features a simple in-order pipeline with three stages and no cache. This processor enables us to validate our method on a deterministic processor with precise timing measurements obtained through the JTAG interface. The more advanced Cortex-M7 processor possesses a 6-stage in-order pipeline, data and instruction caches, and a branch predictor. The Cortex-A53 processor, hosted in a Raspberry Pi 3 features an 8-stage inorder pipeline, two levels of data and instruction caches, and a branch predictor. The Cortex-A72 processor, hosted in a Raspberry Pi 4 differs from the A53 through its out-of-order pipeline. Since the Cortex-A53 and Cortex-A72 lack a JTAG interface, we rely on reading the cycle counter register for timing measurements.

Setup for the learning phase

PyTorch was used to implement our model and the baseline ones. ORXESTRA was trained on a Tesla V100. Each setting (processor) required two days for ORXESTRA training: 1,5 days for pre-training and 0,5 days to fine-tune the model. Perplexity [START_REF] Stanley | An empirical study of smoothing techniques for language modeling[END_REF] 9 score was chosen as the value to optimize during pre-training (see Equation 2) and Mean absolute percentage errors were used as loss during fine-tuning (see Equation 1). All the datasets (even in the pre-training phase) were split into training (70%), validation (10%), and test (the rest) sets containing different BBs. The MAPE (Mean Absolute Percentage Error) is also used to assess the performance of each model. It evaluates how far (as a percentage) the prediction is from the true timing. It is defined as:

𝑀𝐴𝑃𝐸 𝑙𝑜𝑠𝑠 = 1 𝑛 * 𝑛 ∑ 𝑖=0 |𝑝𝑟𝑒𝑑𝑖𝑐𝑡 𝑖 -𝑎𝑐𝑡𝑢𝑎𝑙 𝑖 | 𝑎𝑐𝑡𝑢𝑎𝑙 𝑖 (1)
Perplexity(𝐷) = 𝑁 𝑁 𝑖=1 1 𝑃 (𝑤 𝑖 ) (2)
Where:

• 𝑁 is the total number of events or words in the dataset.

• 𝑤 𝑖 represents the 𝑖th event or word in the dataset.

• 𝑃 (𝑤 𝑖 ) is the probability assigned to event 𝑤 𝑖 by the probability model. It is the estimated likelihood of observing event 𝑤 𝑖 based on the model.

Experimental results

Following the typical evaluation methodology of language models, ORXESTRA is evaluated in two ways: through intrinsic evaluation and extrinsic evaluation. Intrinsic evaluation involves assessing the model obtained at the end of the pretraining phase using specific unsupervised learning metrics (e.g. the perplexity score). It is provided in Section 5.1. On the other hand, extrinsic evaluation involves employing the models in actual tasks, such as the execution time estimation in our case, and evaluating their performance based on metrics such as the mean absolute percentage error (MAPE). This is provided in Section 5.2. The context-awareness feature of ORXESTRA is evaluated in Section 5.3, in which the impact of the context size on the estimations is investigated. Section 5.4 demonstrates that ORXESTRA exhibits superior scalability compared to other models when estimating the timing of large BBs. Section 5.5 study the robustness of the models to the unused optimization options. Lastly, Section 5.6, provides insights into the processing speed of each model by offering estimations of the number of instructions that can be processed per second.

Evaluation of pre-training

In the intrinsic evaluation experiment, both ORXESTRA and the PalmTree ARM10 were evaluated using the same dataset. This evaluation assesses their performance in a masked language modeling task, specifically in recovering the masked operation/operand. The perplexity values for each model are provided in Table 4. The best results in the table are highlighted in bold. A lower perplexity score indicates that the language model is better at predicting masked words. The results clearly indicate that ORXESTRA outperforms the PalmTree ARM across all targets. This observation is not surprising, as Transformers XL, unlike PalmTree ARM, exhibit superior memory capabilities for handling long sequences. In contrast, PalmTree ARM are limited by the restricted number of tokens that can be processed, which restricts their ability to capture dependencies beyond their specified context length. This hard sequence segmentation leads to context fragmentation, inefficient optimization, and ultimately a decline in performance.

Prediction results on the test dataset

We generated two distinct test sets for extrinsic evaluation purposes. The first test set consists of 500 BBs with fewer than 50 instructions and 500 BBs with more than 50 instructions. The second test set is created specifically for all the BBs whose timing can be successfully predicted by the PalmTree ARM. To accommodate the limitations of PalmTree ARM, which cannot handle BBs with token sequence sizes exceeding their capacity. This adjustment was essential to ensure a fair comparison between the models.

In Table 5, we use the first test dataset, and thus we do not provide the results for the PalmTree ARM (which could only perform predictions on the second test dataset and not all BB from the first set). The results include the Mean Absolute Percentage Error (MAPE), where lower percentages indicate better model performance. The best results in the table are highlighted in bold. Additionally, the Pearson correlation score is utilized as another evaluation metric to estimate how correlated the predictions are with the ground truth. In this case, higher scores indicate better model performance.

Table 5. Test results of Neural Networks (NN), ITHEMAL [START_REF] Mendis | Ithemal: Accurate, portable and fast basic block throughput estimation using deep neural networks[END_REF], CATREEN [START_REF] Abderaouf | CATREEN: Context-Aware Code Timing Estimation with Stacked Recurrent Networks[END_REF], and ORXESTRA on various ARM Cortex targets: M4, M7, A53, and A72. The results are based on the first test dataset, which includes a balance between the number of small and large-sized BBs. Evaluation metrics: mean absolute percentage error (MAPE) and Pearson correlation (Corr.). Table 5 shows that ORXESTRA obtains better MAPE performance than all other techniques for all the target architectures. The second best-performing model is CATREEN, which, like ORXESTRA, considers the execution context of BBs. The worst-performing model is the Neural Network this shows the importance of accounting for the sequential information. The context-agnostic techniques, ITHEMAL and, as shown in Table 6 for the second test set, the PalmTree ARM are positioned after the context-aware techniques. The correlation is high for all models and better for the model designed to process sequential data.

The complexity of the target architecture plays a role in the final results, although its influence varies depending on the measurement method employed. When measuring timings on Cortex M4 and M7 using JTAG, we observe that errors on M7 are higher than for Cortex M4. This discrepancy is due to the deterministic nature of the Cortex M4 architecture, while M7 incorporates a cache with a random replacement policy, which introduces timing variability. However, this observation does not hold for more sophisticated architectures such as Cortex A53 and A72. For these processors, measurement methods involving software instrumentation were necessary, which introduced additional cycles into the measurements. The insertion of measurement instruments disrupts the execution, particularly affecting memory plans and cache behavior. As a result, the data (and in particular, the timing labels) obtained for these processors are slightly less accurate compared to processors with a JTAG interface. Consequently, making a direct comparison between these architectures is challenging.

Table 6 reports the results obtained on the second dataset. We can notice that the trends observed in the previous table 5 remain consistent.

Impact of the context size

In the previous Section, we saw the importance of context awareness to make accurate timing predictions (ORXESTRA and CATREEN were the most accurate models thanks to this). However, an important question arises: how much context is necessary? To explore this, we conducted investigations on CATREEN and ORXESTRA for the different target architectures. Experimental results are reported in Table 7 and Table 8 respectively. The best results in the table are highlighted in bold. The results highlight the importance of context in timing estimates. Both CATREEN and ORXESTRA, when deprived of context, produce estimates akin to standard neural networks. As the context size grows, the prediction errors diminish. Yet, there's a limit to this improvement. For ORXESTRA,

Impact of the basic block size

Figure 4 shows the mean absolute error 11 for six sets of 500 basic blocks (BBs) in the test dataset, categorized by instruction count: under 10, 10-20, 20-30, 30-40, 40-50, 50-100, and over 100 instructions. ORXESTRA generally shows lower errors across BBs of varying sizes and architectures. CATREEN and ITHEMAL's error increases significantly for BBs over 100 instructions. This confirms that LSTM-based models scale poorly, even in a task like predicting the execution time of basic blocks, probably due to the vanishing gradient problem. We also observe that the ITHEMAL errors are higher than CATREEN's error, which is surely due to the context management, which will be more influential for larger basic blocks that have a higher probability of making more memory accesses and being more dependent on the context than small-sized basic blocks. As deducted in Section 5.2, ORXESTRA has lower average errors for complex architectures like Cortex A53 and A72, but higher errors for in-order pipeline architectures. Analysis of execution time standard deviations reveals lower variability in more advanced architectures (M4: 252 cycles, M7: 113 cycles, A53: 83 cycles, A72: 64 cycles), which likely contributes to reduced prediction errors.

Optimization effect on prediction

In this experiment, we want to use different GCC optimization levels (O0, O1, O2, O3) to compile and generate the test dataset. This allows for an investigation into how sensitive the models are to changes in optimization levels. The goal 11 Mean Absolute Error (MAE) is calculated as

1 𝑛 * 𝑖=𝑛 1 |𝑝𝑟𝑒𝑑𝑖𝑐𝑡𝑖𝑜𝑛 𝑖 -𝑡𝑟𝑢𝑡ℎ 𝑖 |.
is to answer the question: are the models robust enough to maintain performance across different optimization levels? Table 9 gives the results of this experimentation, and we observe that ORXESTRA consistently outperforms other models across all targets and optimization levels, demonstrating its robustness. In contrast, Neural Networks exhibit the highest error in results, and the highest in term of variability making them the most sensitive to compiler optimization. ITHEMAL consistently outperforms Neural Networks and occasionally surpasses CATREEN.

Inference throughput

Limits and future work

While ORXESTRA has demonstrated superior performance in predicting average execution time compared to existing models, it is important to consider potential limitations that could affect the validity of the study. This section discusses these threats and highlights areas for further improvement.

• Context size: Performance decline with increasing basic blocks is concerning. Investigating if this relates to our Transformer XL network's size is crucial. If accuracy benefits from larger networks, GPU-based parallel training might be needed, and balancing network size with efficient inference time during deployment should be also considered. • Toward explainability: Neural networks' "black-box" nature makes understanding their decision-making process challenging, particularly in precise timing tasks. "Explainability" [START_REF] Zini | On the explainability of natural language processing deep models[END_REF] becomes critical here, providing insight into the model's decisions, building trust, and allowing identification of any model biases or errors. This may allow us to understand the difference in performance from processor architecture to another. • Quicker inference for compiler integration: As noted in Subsection 5.6, neural networks are timeefficient for estimation. Thus, knowledge distillation could be key for integrating ORXESTRA into compilers, with ORXESTRA training a smaller, efficient 'student model' (multilayer perceptron) for practical use while maintaining at best ORXESTRA's performance.

Conclusion

ORXESTRA is a tool that estimates the timing of basic blocks within a program. It takes into account the execution context formed by previously executed basic blocks. Experimental results have shown that its timing predictions are 28% better than those of state-of-the-art context-aware LSTM based CATREEN, while being 98% faster than the latter. Moving forward, we plan to develop a larger model that can incorporate all execution contexts, to capture memory accesses in the control flow graph and represent them as an embedding.

Introduce explainability for a more accurate results analysis and create faster model using transfer learning.

Figure 1 .Figure 2 .

 12 Figure1. Transformers XL architecture[START_REF] Dai | Transformer-xl: Attentive language models beyond a fixed-length context[END_REF]

Figure 4 .

 4 Figure 4. Mean absolute average cycle error boxplot comparison of ITHEMAL (blue), CATREEN (orange), PalmTree ARM (green), and ORXESTRA (red) for different processors (M4, M7, A53, A72) and six Categories of basic blocks. The most left category represents basic blocks with a size of 10 or less instructions (-<=10), while the most right category includes basic blocks with a number of instructions exceeding 100 instructions (100<). Each subfigure represents a processor.

) CFG extraction Binary program CFG (2) BB and context extraction (3) ORXESTRA Context size

  Context extraction: to extract the sequence of BB forming the context, we currently ask the user to annotate each loop with its maximum number of iterations4 and use this information to unroll the loops. The degree to which we unroll these loops depends on the desired context size. As for function calls, they are virtually inlined during context extraction (Reccursivity is not handled in our case). Using the resulted CFG, we unroll loops, and eliminate backedges resulting a creation of an acyclic graph. The edges in the graph are then reversed. Next, utilizing a depth-limited Breadth-first search (BFS) algorithm, we locate paths from each Basic Block to its predecessors. The BFS algorithm takes the context size as input to limit the search. The resulting extracted context BB sequences are provided as inputs to ORXESTRA. 3. Inference: In the final step, ORXESTRA loads the model based on the selected target CPU and proceeds to analyze all the given sequence of BB as inputs. It then predicts the execution time for the last BB of each sequence. If a BB appears at the end of multiple sequences, the BB is assigned the average execution time from all of these sequences. The final prediction gives the execution time among predictions for each BB.

	<output_file>
	...
	<BB4> (mean_AET=5)
	<t0BB4> (AET=4) </t0BB4>
	<t1BB4> (AET=6) </t1BB4>
	<t2BB4> (AET=5) </t2BB4>
	</BB4>
	...
	</output_file>
	Target
	CPU
	1 2 context 1 2 context 3 5 BB average ET ? cycles 3 5 BB average ET ? cycles 1 2 3 5 context BB average ET ? cycles 1 2 3 5 context BB average ET ? cycles 1 2 3 5 context BB average ET ? cycles 1 2 3 5 context BB average ET ? cycles 1 2 3 5 context BB average ET context BB average ET context BB average ET ? cycles 1 2 3 5 ? cycles 1 2 3 5 ? cycles
	Figure 3. Timing predictions using ORXESTRA
	tools, such as Angr [28], can be used for this CFG
	generation.
	2.

1. CFG generation: the process of generating a Control

Flow Graph involves creating a CFG from the binary or assembly code of programs. Various binary analysis (1

Table 1 .

 1 Composition of the dataset for the finetuning phase, showing benchmarks, the number of programs, and the total count of basic blocks retrieved per program. This dataset serves to the training and testing of all competitors also

	Dataset name	Nb. of programs Nb. of BB
	The Algorithms	200	12123
	PolyBench	30	11224
	MiBench	14	8324
	Total	244	31671

Table 2 .

 2 hyper-parameters used for ITHEMAL CATREEN, the PalmTree ARM, and ORXESTRA. (NA: Non Applicable).

	Hyperparameter	ITHEMAL CATREEN PalmTree ARM ORXESTRA
	Loss function	MAPE	sMAPE	MAPE	MAPE
	Optimizer	'SGD'	'Adam'	'Adam'	'Adam'
	Learning rate	0.001	0.0001	0.0001	0.0001
	Embedding size	512	512	512	512
	Feed forward structure and size	128	256, 256	512, 256, 128	512, 256, 128
	Number of layers	2 LSTMs	3 LSTMs	6	4
	Number of attention head	NA	NA	8	4
	Memory length	NA	NA	NA	1024

Table 3 .

 3 Summary of the processors used and their microarchitectural features; I: Inordre processor, S: Superscalar, O: Out of Ordre processor.

	Target	M4	M7	A53	A72
	Measurement tool JTAG	JTAG	Software Software
	OS?	Baremetal Baremetal Linux	Linux
	Pipeline/#stages	I 3	I 6	I-S 8	O 15
	Branch predictor	No	Yes	Yes	Yes
	Cache memory	No	L1	L2	L2

Table 4 .

 4 Perplexity scores obtained by ORXESTRA and the PalmTree ARM in the pre-training phase.

	Target	M4	M7	A53 A72
	PalmTree ARM 24.1 26.4 25.3 25.7
	ORXESTRA	19.2 23.1 22.2 21.8

Table 6 .

 6 Test Results: Mean Absolute Percentage Error (MAPE) on Different Targets (M4, M7, A53, and A72) using the second test set. The Test Set is specifically chosen to be within the prediction capabilities of PalmTree ARM, ensuring a fairer comparison among models.

	Target	M4	M7	A53	A72
	Scores	MAPE MAPE MAPE MAPE
	Neural Networks 27.3% 24.5% 27.8% 25.9%
	ITHEMAL	10.0% 14.4% 12.1% 13.0%
	CATREEN	9.6%	14.5% 10.3% 11.8%
	PalmTree ARM	9.1%	13.8% 13.5% 13.3%
	ORXESTRA	8.7%	6.8%	6.1%	7.5%

Table 7 .

 7 Impact of the context size (number of BB considered as context) on the Mean Absolute Percentage Error of CATREEN.

	Target	M4	M7	A53	A72
	None 13.0% 26.1% 36.3% 18.2%
	1	12.5% 15.2% 21.0% 18.4%
	3	8.8% 15.5% 8.5% 10.4%
	6	9.3% 13.3% 12.5% 15.5%
	20	10.2% 14.2% 9.5% 11.4%

Table 8 .

 8 Impact of the context size (number of BB considered as context) on the Mean Absolute Percentage Error of ORXESTRA.

	Target	M4	M7	A53	A72
	None 12.5% 24.5% 34.6% 13.3%
	1	11.9% 14.6% 20.5% 13.1%
	3	7.8% 14.5% 22.9% 14.5%
	6	8.8%	9.6% 5.2% 6.9%
	20	9.2% 13.7% 8.3%	8.8%
	errors stabilize after including 3 BBs for the M4 architecture
	and 6 BBs for other processors. For CATREEN, most proces-
	sors stabilize at 3 BBs, but M7 needs 6 BBs. This plateau in
	error reduction can be attributed to the inherent constraints
	of LSTM architectures. Overly long contexts can overload
	the context vector, making it less effective within the set
	hyperparameters of both models. In future research, we in-
	tend to delve deeper into this phenomenon, necessitating
	significant computational resources to gain further insights
	and confirm our intuition.

  Table 10 displays the instruction rate per second achieved by each machine learning model. Interestingly, this time is independent of the complexity of the target processor architecture, so we report the average over all processors. The throughput calculation is based on the 1000 basic blocks utilized in the previous experiments (the first test set). To ensure a fair comparison, we present the results in the first column with a batch size of 1, followed by the results with a batch size of 32 in the second column for all techniques. Notably, neural networks demonstrate the highest execution speed, despite their lower accuracy. PalmTree ARM, which does not consider the execution context, follows closely. ORXES-TRA, which processes this context. Both Transformers based solutions provides a better execution speed compared to LSTM-based networks (ITHEMAL and CATREEN), which require sequential processing of each instruction. Consequently, CATREEN is the slowest among them due to the additional context processing involved.

Table 9 .

 9 MAPE performance of ORXESTRA, CATREEN, ITHEMAL and Neural Networks across various GCC optimization levels (O0, O1, O2 and O3) and architectural targets 1% 23.2% 21.9% 26.4% 21.1% 19.9% 28.3% 22.7% 25.2% 40.4% 37.1% 38.4% 18.2% 17.1% 16.4% 16.7% ITHEMAL 14.1% 14.5% 14.5% 14.4% 18.2% 18.2% 17.7% 17.6% 9.1% 11.0% 10.6% 10.1% 12.2% 12.3% 11.5% 11.4% CATREEN 8.9% 8.2% 8.9% 8.8% 13.4% 13.1% 12.8% 13.3% 9.7% 9.7% 9.2% 8.5% 11.1% 11.4% 10.8% 10.4% ORXESTRA 7.6% 7.7% 7.7% 7.8% 8.9% 8.2% 9.9% 9.6% 6.2% 6.1% 6.3% 5.2% 7.9% 7.2% 7.4% 6.9%

	Target			M4				M7				A53				A72
	Optimization	O0	O1	O2	O3	O0	O1	O2	O3	O0	O1	O2	O3	O0	O1	O2	O3
	Neural Networks 28.

Table 10 .

 10 The mean throughput over all processors, when treating 1000 BB for each technique (with a batch size of 1 and batch size of 32).

		Throughput Instruction/second
	Batch size	1	32
	Neural Networks 5131	162140
	ITHEMAL	1627	45379
	CATREEN	1356	32644
	PalmTree ARM 3809	112468
	ORXESTRA	2691	74172

OOV in machine learning can have detrimental effects as it reduces generalization, results in information loss, triggers cascading errors, and presents difficulties in domain adaptation

[START_REF] Abderaouf | WE-HML: hybrid WCET estimation using machine learning for architectures with caches[END_REF] We also cross-compile this program using O3

option.[START_REF] Stanley | An empirical study of smoothing techniques for language modeling[END_REF] Available here: https://github.com/TheAlgorithms/C

Perplexity is a measure of how well a probability model predicts a sample or a sequence of events. A lower perplexity indicates a better model fit to the data.

A Palmtree[START_REF] Li | Palmtree: Learning an assembly language model for instruction embedding[END_REF] model trained for ARM ISA. We only compare to Palmtree as it already showed it superiority to other embedding models like word2vec[START_REF] Mikolov | Distributed representations of words and phrases and their compositionality[END_REF].