Anonymisation is a technique of data processing that can be applied to personal data to achieve irreversible deidentification. One of the advantages of anonymisation is that it allows for research that would otherwise not be possible due to privacy concerns. This means removing direct identifiers such as names, birth dates and addresses, although this alone may not be sufficient to avoid the identification of individuals. Therefore, each case must be considered in its entirety to ensure that no individuals are identifiable.

Effective anonymisation prevents singling out an individual in a dataset by linking several records within the same dataset (or between several separate datasets) or inferring relevant information from such dataset(s). The aim of anonymisation is to ensure that the data no longer relates to identifiable persons, which means it is no longer considered "personal data". Therefore, once a dataset is properly anonymised and individuals are no longer identifiable, the data falls outside the scope of data protection rules under the GDPR. In 2018, the Court of Justice of the European Union introduced measures it was adopting to comply with the GDPR in the publication of its decisions. 2 National legislation regarding privacy and data management is also relevant in this regard.

Pseudonymisation consists of replacing one attribute with another. It is defined in the GDPR as "the processing of personal data in such a manner that the personal data can no longer be attributed to a specific data subject without the use of additional information, provided that such additional information is kept separately and is subject to technical and organisational measures to ensure that the personal data are not attributed to an identified or identifiable natural person." 3 Most EU legal systems use a two-step technique to achieve the most effective outcome: First, the names of the parties to the litigation are changed to their initials. Then, personal identifiers such as ID numbers or addresses are replaced with "x" characters. Pseudonymisation, however, may require further redaction to comply with the GDPR. Even when the name of a party is completely replaced with initials, if the decision describes a situation that is unique or that is very well-known, the decision may still contain personal data. Similarly, if, by using techniques of data mining, users could find cross-references leading to specific identifiers of persons involved, the data may still be within the scope of the GDPR and national legislation. It is important to note that member states have implemented, interpreted, and applied the GDPR somewhat differently across the EU. Hence, an ad hoc assessment of the balance between the risk of disclosing personal data and a legitimate purpose of personal data processing may be required in particular cases.

Generalisation is another anonymisation technique. This approach consists of generalising or diluting the attributes of data subjects by modifying the relevant scale or order of magnitude (e.g. using a region rather than a city, a month rather than a week, etc.). Whilst generalisation can be effective to prevent the singling out of individuals, and may be useful as a supplemental tool, it does not allow for effective anonymisation in all cases.

Processing personal data for research purposes occupies a privileged position within the GDPR. 4 CULTEXP (re)publishes caselaw according to a legitimate interest of facilitating access to justice on topics related to cultural expertise. 5 In the phase of data-treatment, the EURO-EXPERT team removed all personal data from their data sets, unless a clearly specified exception applies. EURO-EXPERT has adopted the following combination of anonymisation and pseudonymisation, which is explained in further detail below:

1) The names of the parties are reduced to initials or have been redacted or removed. 2) All personal identifiers such as ID numbers, addresses, social security numbers and so on are removed or redacted.

What to anonymise?

EURO-EXPERT requires anonymisation of:

1. Names of all living individuals unless otherwise specified: Defendants, plaintiffs, claimants, applicants, appellants, respondents, victims, witnesses, other parties, lawyers, experts and consultants.

2. Names of places: Addresses (home address and other); specific locations (villages, towns, and cities, depending on the risk that individuals might be identifiable with this information -consider generalisation); specific places (hotels, shopping centres, shops, farms, churches, accommodation, hospitals, buildings or monuments, boats, etc).

3. Dates: Birth dates (consider generalisation: a year instead of a date); other dates or years which could make a person identifiable (dates suggesting specific and recognisable events in the life of subject, such as the date of marriage, birth dates of children); dates of other decisions, orders and notifications (asylum application, decision by an administrative authority, etc).

a. In some cases, redacting only part of the date may be sufficient if the timeline is relevant for understanding the case. Consider whether the month/year may be retained, for instance, in cases where the age of the individual is relevant. For example: 4. Numbers: National identification numbers, passport numbers, fiscal codes, social security numbers, bank accounts, vehicle registration numbers, identification numbers (e.g. personal identification under the asylum procedure), police report numbers (codes of recordings, codes for intercepts), telephone numbers, postal codes, etc, if they provide information that could lead to the identification of individuals.

5. Signatures: In some jurisdictions case files include signatures. Note also signatures written in the margins of the documents (e.g. Italy). Signatures of judges do not need to be redacted if the names of judges are unredacted. Many expert reports include signatures, which should be removed or redacted.

6. Brands, labels, and corporate logos (including those of legal database services) if they provide information that could lead to the identification of individuals. a. In a case of terrorism, the brand of the shirt of the accused was redacted because the eyewitness identified the accused by the branding of their clothes.

b. In the UK cases, many decisions were downloaded from Westlaw, and their logo has been redacted.

7. In some cases, names of private associations, institutions, and commercial enterprises are also redacted if those can be linked to and used to identify a natural person. This is addressed in detail in Annex 1 on legal persons and anonymisation. See also Annex 2 on processing decisions related to legal persons. Consider the context, scope and level of detail: the CEO of a named small local company is identifiable, whereas a salesperson working for Carrefour or Tesco in a large city may not be.

a. If names of companies are redacted, "nesting" corporate entities may be necessary to retain readability of the case. For example:

"Marvel Studios is a subsidiary of Disney Corp, which purchased it from Marvel Entertainment Group."

[Corp 1.1] is a subsidiary of [Corp 1], which purchased it from [Corp 2]
8. In some cases, the case numbers have been redacted. If there is no imminent risk of or access to sensitive information via the case number, this does not have to be redacted. A balancing assessment should be made between the legitimate purpose of the database and the indirect risk of disclosure of personal data; note that the case number might be disclosed for the purpose of legal citations. In some jurisdictions, anyone can order the case from a court with a case number and could access the personal data that way.

In the Italian context, the case numbers of decisions of the Commissione Territoriale reveal the social security number of the applicant, which should be removed or redacted.

a. In other Italian judgments, some cases are identified as follows: "Cass. Sez. 6^, 20 ottobre 1999, Bajrami" (or with the name of judge).

"Cass. Sez. 6^, 20 ottobre 1999, B[removed]"

What not to anonymise?

Do not remove or redact the following, unless the data can lead to the identification of individuals:

1. Names of judges and court officials, unless country legislation or practice dictates otherwise, or where revealing the name of the judge may put them at a risk of reprisal. For further discussion, see the next section on special categories.

2. Names of deceased persons, historical figures, famous persons, and public figures. However, if a public figure is a party to a case, they may have a right to privacy; consider also issues such as libel suits. a. In some cases, well-known public figures may have been indirectly involved in circumstances surrounding the case. In such instances, consider if it may be useful to not only redact their name but also their role and title. For example:

"Emmanuel Macron was president of France when he signed the contract with John Doe."

[EM] was [Redacted] when he signed the contract with [JD]

3. Names of criminal organizations (Cosa Nostra, Islamic State, Black AXE, etc).

4. Names of provinces or states, unless such information could lead to the identification of the person(s)see for example:

"asylum seeker from Hashela district of Keren in Eritrea living in the small town of Grândola in Portugal"

"asylum seeker from Hashela district of Keren in Eritrea living in the small town of Grândola in Portugal" 5. URLs.

Bibliographical references (authors, titles).

7. Public and international associations or organisations (United Nations, Red Cross, World Health Organization, etc).

8. Names and other data related to legal persons (companies, associations, etc.) when those cannot be linked to an individual and do not lead to the identification of a natural person. Note that, under the GDPR, only natural persons have personal data. This is addressed in detail in Annex 1. 9. Legal precedents that contain the names of the parties (unless these lead to identification of the individuals involved in the case that you are anonymising).

a. Consider redacting names when the court refers to names of individuals in discussing the facts of the precedent. For example:

"In this case of Appellant we rely on the Parker case. In that case Mr Parker was driving down the street when he got in an altercation with the driver of another vehicle."

In this case of Appellant we rely on the Parker case. In that case [Redacted] was driving down the street when he got in an altercation with the driver of another vehicle.

Special categories

As a general principle, the GDPR applies to all personal data. However, national legislation and practices on the implementation of the GDPR vary significantly. At EURO-EXPERT, we have debated on whether to anonymise the names of the judges. To the best of our knowledge, no binding interpretation on the application of GDPR to legal documents has been issued to date and available guidelines do not provide instructions on this specific matter. Article 6 (1) (f) GDPR entails an analysis of proportionality: regarding the publication of the names of court staff, and especially judges, the legitimate interest of the publication prevails over the risk posed to their privacy. The publication of the names of judges contributes to democratic debate and may be protected by article 85 of the GDPR. The principle of transparency in the judicial process and the principle of open justice as enshrined in article 47 of the EU Charter of Fundamental Rights support the publication of the names of judges to foster the effective independence and impartiality of the courts. It has been noted that one of the main reasons for publishing court decisions at the national level is the public scrutiny of the judge. [START_REF] Bobek | Data protection, anonymity and courts[END_REF] Further, it has been observed that "in most countries the names of professionals involved [in a court case] are not rendered anonymous […]". [START_REF] Marc Van Opijnen | Court Decisions on the Internet: Development of a Legal Framework in Europe[END_REF] According to the expert opinion received by Prof. Jougleux (European University Cyprus), there is no explicit reference in the EU legislation to the names of judges and lawyers in court decisions; furthermore, the mention of the names of the judges and the lawyers does not generally present privacy risks and should therefore not be included in the anonymisation mechanism. Hence, as a general rule, the names of people involved in a court case in their professional capacity are not anonymised in CULTEXP; for example judges, clerks, lawyers, bailiffs, court experts, interpreters and custodians. [START_REF] Van Opijnen | On-line Publication of Court Decisions in the EU: Report of the Policy Group of the Project 'Building on the European Case Law Identifier[END_REF] Exceptions to this may include specific country practices -the respective Data Protection Authorities of different EURO-EXPERT countries have varying opinions and practices on what information should be included in the public court files -and in situations presenting a high risk of reprisals (e.g. terrorism and organised crimes cases). 9

As country practices vary and it is expected that more detailed guidelines will be developed, EURO-EXPERT endorses a cautious approach overall. EURO-EXPERT abides by the national legislations and regional practices that require the redaction of the judges and court staff's personal data, constantly monitors the development of the implementation of the GDPR and will adapt to future changes.

At this stage, EURO-EXPERT anonymises or pseudonymises the personal data on lawyers and experts involved in a court case. As the guidelines for the processing of personal data in the publication of judgements in legal databases evolve, EURO-EXPERT may review and relax this precautionary approach. The EURO-EXPERT team is advised to also use their expert knowledge of the context and national legislation and practices in combination with GDPR regulations and always to err on the side of caution if in doubt.

How to anonymise?

1. Keep two folders on the Shared Drive: one for the original documents (for reference and if there is a need to check details, but this folder is to be deleted at the end) and one for the anonymised documents. Avoid keeping any original documents in locations (physical or digital) which could be accessed by others or could pose any risk of a data breach.

2. When saving the anonymised document, make sure the file name does not contain any names or other personal data. The same applies to its 'document properties' or metadata.

3. Depending on your dataset, you may consider different options for anonymising or pseudonymising the names of individuals: a. Use initials representing the first name and surname. b. If initials might reveal the identity of the person, use letters or numbers but try to be consistent throughout the case, as long as the letter or number will not reveal the identity: i. A1, A2 etc for claimants/plaintiffs and B1, B2 etc for respondents/defendants) ii.

For experts, use E1, E2, etc. iii.

For witnesses, use W1, W2, etc. iv.

For judges, use J1, J2, etc. v.

For lawyers, counsel, attorneys and other legal representatives of parties, use L1, L2, etc. b. If there is still a risk of identifying individuals, use --or xx for names of all or some of the individuals involved in the case.

4. If the document makes a passing reference to any person, you can redact, or use "-" or "xx". These can be used when the role of that individual is irrelevant to the narrative as a whole.

5. When going through the case, you may find it useful to keep a pen and paper at hand to write down your "code", e.g. "Dr Carol Smith = E1". Make sure to properly destroy these notes afterwards.

6. In MS Word, under "Edit", you will find "Find" under which there is the function "Replace". In some versions of MS Word, you will find this function under "Home" > "Editing". Use this with caution. See below for details.

7. If you work with PDF documents, using Adobe may be the only secure option. Remember to sanitise the document at the end! 8. Do not use any online PDF-Word converters for documents containing personal data or sensitive information. Adobe Pro includes a PDF-Word converter. If you have any problems, contact the EURO-EXPERT team.

Challenges and tips

Removing direct identifiers is not enough to ensure that identification of the data subject is no longer possible; pseudonymised data is not the same as anonymised data! Simply altering the main identifier (usually the name) does not prevent someone from identifying a data subject if quasi-identifiers (such as places, dates, age and so on) remain in the dataset. Extra steps should be taken, such as removing and generalising attributes or deleting the original data, or at least bringing them to a highly aggregated level.

• Example: if a person is described as 'a man', the anonymity set size is three and a half billion, but if he is described as 'a middle-aged Dutchman with a beard' the anonymity set size is reduced to, say, half a million. If he is described as 'a middle-aged Dutchman with a beard who lives near Cambridge' the anonymity set size is only three or four and consequently allows for a high probability of identification.

Make sure you:

• Read and understand the case and the roles of persons involved.

• Clarify whether the data will be anonymised (the link to the data subject will be destroyed) or pseudonymised (the data could be reversible).

• Do not rely on the "find-replace" function: o Documents may be partly corrupt and not all references are identifiable by the search function.

o Documents may contain inaccuracies, misprints or mistakes. o The same name, place, or other identifier may appear with several spellings in the same document (e.g. Mohammed, Mohamed, Muhammad, Muhammed etc).

o In certain languages (e.g. Finnish) suffixes may alter the core of the word, rendering it undiscoverable by the search function (e.g. Mäki -Mäen).

o The find-replace function can be a useful supplemental tool in the final check of the document to see if any identifiers remain.

• Remember to check the case name/citation for any identifiers!

• Remember to check also footnotes for any personal data, if applicable.

• If some data will not be anonymised, explain why you cannot anonymise the data.

• If the data will be coded, describe the coding system, and who will have access to it; and confirm that it cannot be traced back to individuals unless essential for the study.

• Assess whether the remaining data combined with additional information might permit the identification of an individual.

• Prepare a short statement disclosing your anonymisation technique or the mix of techniques that you have used, and highlight any caveats, challenges, or problems. Doing so will help to show that your work is conscientious and reflective and will serve to maintain the quality and legal compliance of the outcomes of the project.

Annex 1: Identifying legal persons for anonymisation

Introduction

There are several reasons for anonymising or redacting the name of a legal person, such as a corporation. The two most common reasons are (1) to maintain readability, without having to sacrifice the strict standards of GDPR compliance in EURO-EXPERT, and (2) to remove any data which could reveal information about unrelated parties.

The need to anonymise or redact the name of a legal person arises most commonly in proceedings where the actions of natural persons are key to litigation between legal persons, for example, when the CEO of a company is accused of defrauding another company's corporate officers. In these situations, there are two options: (1) to anonymise or redact the natural person's name and relationship with the legal person, or (2) to anonymise or redact the natural person's name and the legal person's name.

Maintaining comprehension and readability

In Excerpt 2.1, it is clear that the natural person is the founder of a corporate entity. The two options are shown below to fully anonymise the paragraph: Excerpt 2.1: In excerpt 2.1(a) the relationship that VAK has with the company is lost, although we know the name of the company. However, with Excerpt 2.1(b) the pseudonymised name of the legal person is "Corp1" (Corporate 1), so we know that VAK is the founder of "Corp1" and more information is preserved to improve readability. Furthermore (in Excerpt 2.1(b)), leaving "the founder" unredacted does not risk disclosure of personal data but maintains the importance of this person's position in the company as a measure of their decision-making power, or to contextualise the later evidence discussed in the case. For example, with excerpt 2.2: Excerpt 2.2:

In Excerpt 2.
In Excerpt 2.2(a) below, the testimony of the witness would be useless unless one is allowed to know that YZ, who was CEO at the time, convinced the founder of "Corp1" that the witness should become GD of Corp1's subsidiary "Corp1.1" as below would maintain:

Excerpt 2.2(a):

Another example (Excerpt 2.3, below) of this arises in relation to real property (buildings, land, apartments, etc.). In the example below, there is a question of whether Loc3 (Location 3) is owned by Corp10.1 or VA. By redacting the precise location and the company name, this context can be maintained so that the reader understands the dispute between Corp10's subsidiary (Corp10.1) and VA. Excerpt 2.3:

Protecting the identity of third parties uninvolved in litigation

In Excerpt 3.1, disclosing the legal entity's name would reveal the involvement of a natural person who is entirely uninvolved in the case: Excerpt 3.1

In Excerpt 3.1, redacting the names of two companies is necessary to protect the identity of CS, but it also conceals further information about individual DJT, who is not a party or witness in the case. By redacting the names of the companies, we do not have to redact "had developed a prior working relationship with Corp5 over a number of years".

Annex 2: Practical steps for redacting information related to legal persons

When anonymising and assigning pseudonyms to cases involving legal persons (corporations), it may become necessary to redact some data of the legal persons to protect the identity of natural persons (individuals). This is especially important when looking at cases examining the action and behaviour of named corporate officers (CEOs, CFOs, CTOs, etc.) or named corporate employees. Unfortunately, these types of cases, especially involving accusations of "white collar crimes" (such as embezzlement, tax evasion, and fraud) often include complex and opaque webs of corporations, shell companies, offshore accounts, and natural persons. As demonstrated below, in order to maintain readability of the document, this complex web should be understood by the redactor before they begin. This can be achieved by: (1) mapping out the corporate structures, (2) identifying outside organisations, (3) identifying properties owned by companies and (4) identifying extraneous information about the company which may not relate to the case at hand but can lead to the identification of the company (and by extension to the involved natural persons).

Mapping out the corporate structure

To maintain readability, it is important to make sure the pseudonyms reflect the relationships between parties according to the corporate structure, and to have a clear plan on how the information about these relationships can be incorporated into the pseudonymisation and to ensure not to double number them. The easiest and most intuitive way to do this, without using too much space, is to label the highest-level company with a whole number (Corp1, Corp2, Corp3, etc.), and as you move down within a single corporate structure add decimals to reflect where the company in question lies on the corporate map (Corp1.1 being the first company encountered which is a subsidiary of Corp1). Using the fictional example below, this can look as follows:

Example 1.1:

The After mapping out the corporate structure, it is possible to replace the names (and the natural persons involved) with the assigned pseudonyms. Be sure to leave names of countries, when possible, as this helps the reader to follow judicial logic in some situations, especially when dealing with offshore shell companies as will be seen in a later example. This should look as follows: Example 1.1(a)

Unlike in Example 1.1 and 1.1(a) often the relationship between companies is not apparent until later on in the decision, as demonstrated in Example 1.2 below: Example 1.2 Again, when redacting the above example, if one had not made it to paragraph 102 yet, it is easy to confuse Acme Holding as Corp1, and find out later that Acme Inc. is actually the parent company. This would require one to go back and correct the mistake. To avoid this, check the entire document first to see if the relationship is described. Thus, when done correctly it should look like Example 1.2(a) below.

Example 1.2(a)

When a company's relationship to other companies is in question, or the source of litigation, it can make deciding where to place them in the corporate map difficult. Such an example can be seen below in Example 1.3:

Example 1.3
In Example 1.3, the relationship between Roi Mug SARL and Glassware Holding Ltd. is unclear, and may be subject to judicial review to determine whether it is actually related to Glassware holding Ltd. In this case, refer to Roi Mug with a letter, rather than a number. In Example 1.3(a) I referred to Roi Mug SARL as Corp X, but if more than one corporation is involved, consider pseudonymising in alphabetical order (CorpA, CorpB, CorpC). Furthermore, if more than 26 companies (because there are 26 letters in the English alphabet) are involved, double letters may be necessary (CorpAA, CorpAB, CorpAC). Example 1.3(a)

Outside Corporations and Groups

When a company is not related to another company, simply assign it the next number (for example If you have already assigned Corp1, it would be Corp2, then Corp3). In more complex cases, you can end up easily with over 50 different and unrelated companies. It is important, however, to distinguish between banks (Use Bank1, Bank2, etc.) and companies -also keeping in mind the necessary corporate maps which may be needed. Good News Ltd. (Corp2) In Example 2.1, Good News is not affiliated with the Delivery Company family of companies, nor is its relation in question, it is only potentially entering into an agreement with Delivery Company's subsidiary, thus it would be Corp2. Secondly, the First Bank of the United States of Earth (Bank2) has a subsidiary too, USE NNY Bank and so this should be given the pseudonym Bank2.1. When we apply these redactions and their associated pseudonyms, the information of who is a Bank, who is a company, and which companies are related is maintained. As we see in Example 2.1(a) this would look like this: Example 2.1(a)

This distinction between banks and companies is useful to the reader, even though a bank is technically a corporation. If this distinction is not made, it could confuse the reader into thinking that a company gave money to another company, as opposed to the very different legal implications of a loan from a bank.

Identifying properties owned by a company

When a company owns a property, especially real property, it could lead to identification of the company -especially when the property is well known. As redactors, we should not assume what properties are well known to the local population or even internationally and redact all properties which are associated with the company.

Example 3.1

There must be a distinction here between the three types of locations used: (1) the cities, places and countries where events and properties are located such as Amsterdam, Den Haag, the Netherlands and Voorburg Station; (2) exact addresses such as Zonweg 21; and (3) buildings such as Den Haag Municipal Library or the warehouse. Only the latter two need to be redacted, as the location of meetings and countries in which the property is located can have important legal repercussions, and generally do not risk disclosure of personal data. This is also the case in Examples 1. In Example 3.1(a), notice that the entirety of "the warehouse at Zonweg 21" in paragraph 41, and similarly in paragraph 100 "the library" and "the warehouse" are redacted. This is because, as a redactor who is unfamiliar with the local geography, it would be too time-consuming to look up the details and the identifiability of the property, thus it is safer to redact. Furthermore, as with "the warehouse" in paragraph 100, for consistency and clarity one should continue to label "the warehouse" as Location1 and "the library" as Location2.

Also in Example 3.1(a), although one could redact "near Voorburg train station", the previously redacted information is sufficient to protect the identity of Jane Doe (the person whose identity we are trying to protect). The goal is to reach the maximum protection of personal data with the minimum number of redactions.

Another example of a name of a property needing to be redacted is when naming a chattel (a tangible thing) can lead to the identification of natural persons. Chattel can be (non-exhaustively) ships, train lines, airplanes or products. This is demonstrated below in Example 3.2: Example 3.2 One must use their judgement to determine if the information can lead to the identification of a legal or natural person. In the above example, as ships are given unique names which make them easy to identify, USS Enterprise should be redacted. Additionally, "Spice Line" is easily identifiable as a specific train line. Finally, a "Tribble", a fictional product in the above example, should be redacted as it may clearly connect the tribbles to Planet Express Inc.

Example 3.2(a)

In Example 3.2(a), each form of property was generalised. For example: Tribbles are products, the USS Enterprise is a ship, the Spice Line is a train line and the Daedalus is an airplane. Generalisation makes it easiest for the reader to follow the decision, especially when the products may only be mentioned hundreds of pages apart. Although named, it may not be necessary to redact Airbus; There are hundreds of companies which own Airbus planes, and thus it is not necessary to pseudonymise it into "Plane2".

Identifying extraneous information about the company

Sometimes, there may be information about a company which does not fit into the above categories, but which can nonetheless lead to the identification of the company, and by extension the people affiliated with the company. For example, in Example 4.1 below, the company Houseplant Inc. was a principal cause of the economic crash in 2030.

  1(a), the natural person's data redacted: Excerpt 2.1(a): In Excerpt 2.1(b), the legal person's data redacted: Excerpt 2.1(b):

  map for Example 1.1 would look as follows: Acme Inc. (Corp1) Acme Holding Ltd. (Corp1.1) Acme Financial Services A/S (Corp1.2) Fresh Foods SARL (Corp1.1.1) New Wave Financial Ltd. (Corp1.2.1) New Cars ZAO (Corp1.1.2) Heavy Anvils GmbH (Corp1.1.3) Wet Paint BV (Corp1.1.4)

  For example, in Example 2.1 below: Example 2.1 In Example 2.1, the following maps would be created: Delivery Company Ltd. (Corp1) Bank of New York (Bank1) First Bank of the US of Earth (Bank2) Planet Express Ltd. (Corp1.1) USE NNY Bank (Bank2.1)

  1 and 1.1(a), which name the country where each company is located and in Examples 2.1 and 2.1(a) where New York is named. When redacted, Example 3.1 should look like this: Example 3.1(a)

Complete Guide to GDPR Compliance: https://gdpr.eu/.

Example 4.1

In the above (obviously fictional) example, there are two "well-known" events, which a reader may recognise and connect with the company. The first is events which triggered the Third Great Recession of 2030 (in a way many people today would know Lehman Brothers triggered the Great Recession in 2008). The second is the public apology, which would be recognisable in a way that the BP Oil Spill apology by BP's CEO would be recognisable to many readers. To approach this, we would redact as below in Example 4.1(a):

In this Example 4.1(a), both instances of "the Third Great Recession of 2030" were redacted, although their nationalisation was not. This is because the fact that the company was nationalised may be important to the decision. The fact that Houseplant Inc. is responsible for the Third Great Recession of 2030 would likely be easily traced back to Mr. John Doe. Therefore, the scientific value of leaving the phrase "the Third Great Recession" in paragraph 99 does not outweigh the risk of identifying a natural person -it should be redacted. Additionally, "made its infamous televised address to the public" was redacted as, if it is infamous, it would be recognisable to the reader, and should again be redacted to protect the identity of John Doe.


Benoît Viollet

Louise Lantier

Jocelyne Devin-Leclerc

Sophie Hébrard

Chloé Amouyal

Rémi Mounier

Marc Foretz

Fabrizio Andreelli

à la diffusion de documents scientifiques de niveau

recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Targeting the AMPK pathway for the treatment of Type 2 diabetes.

Figure 3 .

 3 Figure 3. AMPK and the regulation of skeletal muscle metabolism. Proposed model for the role of AMPK in the regulation of lipid and glucose metabolism in skeletal muscle. AMPK activity may be increased by an altered energy nucleotide or by hormonal action. This activation of AMPK may result in an increase in glucose transport as well as an increase in fatty acid oxidation. ACC, acetyl-CoA carboxylase; AMPK, AMP-activated protein kinase; AS160, Akt substrate of 160kDa; CPT1-α, carnitine palmitoyl transferase-1; Glut4, glucose tranporter 4; MCD, malonyl-CoA decarboxylase; PGC1α, PPARγ co-activator 1α; LCACoA, Long Chain acyl CoAs.

Figure 4 .

 4 Figure 4. AMPK and the regulation of hepatic metabolism. Activation of AMPK leads to the inhibition of cholesterol synthesis by the phopshorylation of HMG-CoA reductase. By inhibiting ACC and activating MCD, AMPK increases fatty acid oxidation via the regulation of malonyl CoA levels, which is both a critical precursor for biosynthesis of fatty acids and a potent inhibitor of CPT-1, the shuttle that controls the transfer of LCACoA into the mitochondria. AMPK inhibits hepatic glucose production via the phosphorylation of TORC2 and inhibition gene expression for key gluconeogenic enzymes, G6Pase and PEPCK, and for the transcriptional coactivator PGC-1α. ACC, acetyl-CoA carboxylase; AMPK, AMP-activated protein kinase; CPT1-α, carnitine palmitoyl transferase-1; G6Pase, glucose-6-phosphatase; LCACoA, Long Chain acyl CoAs; MCD, malonyl-CoA decarboxylase; PEPCK, phosphoenolpyruvate carboxykinase; PGC1α, PPARγ co-activator 1α; TORC2, transducer of regulated CREB activity 2.


Mireille Fares

Catherine Pelachaud

Nicolas Obin

Transformer Network for Semantically-Aware and Speech-Driven Upper-Face Generation

Keywords: Semantically and Speech Driven Gestures, Transformers, Visual Prosody, Embodied Conversational Agents

We propose a semantically-aware speech driven model to generate expressive and natural upper-facial and head motion for Embodied Conversational Agents (ECA). In this work, we aim to produce natural and continuous head motion and upper-facial gestures synchronized with speech. We propose a model that generates these gestures based on multimodal input features: the first modality is text, and the second one is speech prosody. Our model makes use of Transformers and Convolutions to map the multimodal features that correspond to an utterance to continuous eyebrows and head gestures. We conduct subjective and objective evaluations to validate our approach and compare it with state of the art.

I. INTRODUCTION

A. Context

The human face is a key channel of communication in human-human interaction. During speech, humans spontaneously and continuously display various facial and head gestures that convey a large panel of information to the interlocutors. These gestures are known as "visual prosody" [START_REF] Graf | Visual prosody: facial movements accompanying speech[END_REF]: facial or head movement are produced in conjunction with verbal communication. During speech, the Fundamental Frequency (F0) variations and upper-facial movements are highly correlated [START_REF] Yehia | Linking facial animation, head motion and speech acoustics[END_REF]; they are the results of linguistic and conversational choices. Eyebrow motion can also occur during pauses. Another kinematic-acoustic relation that happens during production of speech is between F0 and head motion [START_REF] Yehia | Linking facial animation, head motion and speech acoustics[END_REF]. Natural and rhythmic head movements are one of the key factors in producing natural animations [START_REF] Ding | Modeling multimodal behaviors from speech prosody[END_REF], [START_REF] Chen | Talking-head generation with rhythmic head motion[END_REF].

B. Related works and our contributions

A large number of head motion generation systems has been proposed in previous works [START_REF] Hofer | Automatic head motion prediction from speech data[END_REF]- [START_REF] Wang | Audio2head: Audio-driven one-shot talking-head generation with natural head motion[END_REF]. A variety of generative statistical models aimed to predict the multimodal behavior of a virtual agent. Hidden Markov Models (HMM) [START_REF] Hofer | Automatic head motion prediction from speech data[END_REF], Recurrent Neural Networks (RNN) [START_REF] Haag | Bidirectional LSTM networks employing stacked bottleneck features for expressive speech-driven head motion synthesis[END_REF], [START_REF] Wang | Audio2head: Audio-driven one-shot talking-head generation with natural head motion[END_REF], and Dynamic Bayesian Networks (DBN) [START_REF] Mariooryad | Generating human-like behaviors using joint, speech-driven models for conversational agents[END_REF], [START_REF] Sadoughi | Speech-driven animation with meaningful behaviors[END_REF] have been used to generate head motion from speech; Generative Adversarial Networks (GAN) have been proposed to produce facial gestures from speech [START_REF] Vougioukas | Realistic speech-driven facial animation with gans[END_REF], [START_REF] Karras | Audio-driven facial animation by joint end-to-end learning of pose and emotion[END_REF]. [START_REF] Kucherenko | Gesticulator: A framework for semanticallyaware speech-driven gesture generation[END_REF] generates continuous 3D hand gestures based on acoustics and semantics. However, most of the aforementioned approaches exploit as input one modality only, namely speech, This work was performed within the Labex SMART (ANR-11-LABX-65) supported by French state funds managed by the ANR within the Investissements d'Avenir programme under reference ANR-11-IDEX-0004-02.

and neglect to render their approach using semantic information. Also, most of them focus only on facial expressions while the correlation between facial expressions and head movements are crucial to produce a natural behavior. For instance, [START_REF] Hofer | Automatic head motion prediction from speech data[END_REF]- [START_REF] Lu | Prediction of head motion from speech waveforms with a canonical-correlation-constrained autoencoder[END_REF], [START_REF] Sadoughi | Speech-driven animation with meaningful behaviors[END_REF] do not generate eyebrow motion along with head motion, which are both correlated to F0 [START_REF] Yehia | Linking facial animation, head motion and speech acoustics[END_REF], and therefore are correlated to each other. On the other hand, transformer networks and attention mechanisms have been recently proved to be very efficient for sequence-to-sequence modelling, with particular advances for modelling multimodal processes. For instance, Transformers were previously used for translating speech to text (ASR) [START_REF] Hrinchuk | Correction of automatic speech recognition with transformer sequence-to-sequence model[END_REF], [START_REF] Mohamed | Transformers with convolutional context for asr[END_REF], and multimodal learning of images based on text [START_REF] Yao | Multimodal transformer for multimodal machine translation[END_REF]. To overcome those limitations, we propose a novel approach for upper-facial and head gestures generation based on a multimodal transformer network. Our contributions can be listed as follows: 1) a transformer network operating on multi-modal input text and speech information in order to generate upper-facial and head movements, and 2) a cross-attention module that can efficiently exploit semantic and speech information. 1 The paper is organised as follows. The next section describes the proposed architecture of our model and its multimodal features. Then our objective and subjective evaluation experiments are presented including the LSTM-based Baseline Model. We finally discuss our results.

II. PROPOSED ARCHITECTURE

A. Multimodal Input/Output Features

The upper-facial movements and head rotations are represented by mean of Action Units (AUs) as defined in the Facial Action Coding Systems (FACS) [START_REF] Ekman | What the face reveals: Basic and applied studies of spontaneous expression using the Facial Action Coding System (FACS)[END_REF] and 3D head angles. The work presented in this paper only considers the AUs that represent eyebrows movements which are: inner raise eyebrow AU1, outer raise eyebrow AU2, frown AU4, upper lid raiser AU5, cheek raiser AU6, and lid tightener AU7. Head rotations have three degrees of freedom, represented by the Euler angles roll, pitch and yaw. They are represented by RX, RY and RZ which are the rotation of the head with respect to the X, Y and Z axes. In this paper, F0 values were extracted from the speech signal at a 5ms audio frame rate, linearly interpolated between unvoiced segments, and clipped to the range of 50 to 550Hz which represents the F0 range of human speech. Since F0, AU intensities (AU) and head rotations (R) are continuous, they were quantized to produce a finite set of discrete values to reduce the model size and energy consumption [START_REF] Guo | A survey on methods and theories of quantized neural networks[END_REF]. Finally, BERT word embeddings were extracted from the text transcription. The proposed architecture aims at mapping multimodal speech and text feature sequence into continuous facial and head gestures. This problem is treated as a multimodal sequence-to-sequence problem, for which a transformer network operating at the word level is presented as illustrated in Fig. 1 (a). The inputs and outputs of the transformer network consist of one feature vector for each word W of the input text sequence. The input text corresponds to an Inter-Pausal Unit (IPU) which is a sequence of words separated by silent pauses longer than 0.2 seconds. F0 input sequence as well as AU intensities AU and head rotations R output sequences have a variable length. We set the maximum F0 input sequence length to 100, and the maximum AU/R output length to 124. Shorter sequences were padded to the maximum length, and longer ones were truncated. In order to handle continuous flow of input and output information with different timing, the transformer is wrapped with a F0 encoder module at its input and to a AU/R decoder at its output. The objective of the F0 encoder is to encode the continuous F0-values at the word level and the objective of the AU/R decoder is to reconstruct the continuous values for AU and R from a word-level encoding. Each spoken word W is represented by a word-level F0 embedding vector X

(1) W which at its turn corresponds to the sequence of F0values X (1) = (F 0 1 , . . . , F 0 N W ), where N W is the number of F0-values corresponding to the spoken word W; and X

(2) W is the BERT word embedding vector corresponding to the spoken word W -including silences. Silences less than 0.2 secs may belong to IPUs. They do not have a contextual BERT embedding. Hence, we replaced them by a comma -",". Each spoken word W is also represented by a word-level AU vector Y

(j) W,AU (resp. R vector Y (j) W,R
) which in turn corresponds to the sequence of values dY

(j) AU = (AU (j) 1 , . . . , AU (j) N ′ W ) (resp. Y (j) R = (R (j) 1 , . . . , R (j) N ′ W
)) where j denotes the j th AU/R and N ′ W the number AU/R-values corresponding to the word W. F0 Encoder: As depicted in Fig. 1 (a), for each W, three one-dimensional convolutional layers are applied to project the input F0 sequence X W is then fed as an input to the Transformer Encoder.

Transformer Encoder: The transformer encoder architecture is depicted in Fig. 1 (b); it is similar to the one proposed in [START_REF] Vaswani | Attention is all you need[END_REF]. In our work, it is composed of a stack of N = 4 identical layers. Each layer has two sub-layers: the first one is a multihead self attention mechanism with 4 attention heads, and the second one is a position-wise fully connected feed-forward network. As the original transformer encoder, we employ a residual connection around each of the 2 sub-layers, followed by layer normalization.

Cross-Modality Attention Module (CMAM): The output of the transformer encoder Z, as well as X

(2)

W are fed as inputs to the Cross-Modality Attention Module (see Fig. 1 (a)). This Module has the same structure as the Transformer Decoder in [START_REF] Vaswani | Attention is all you need[END_REF]. It generates a representation that can take into account both modalities, text and speech. The representation learning is done in a master/slave manner, where one modality -the master -is used to highlight the extracted features in the other modality -the slave. This module takes X

(2) W -Text Modality -as master, and Z -Speech Modality -as slave. Thus, it performs cross-attention such that the attention mask is derived from text modality, and is harnessed to leverage the latent features from the speech modality.

Transformer Decoder: The decoder is composed of N = 4 identical decoding layers, with 4 attention heads. Similar to the one proposed in [START_REF] Vaswani | Attention is all you need[END_REF], it is composed of residual connections applied around each of the sub-layers, followed by layer normalization. As depicted in Fig. 1 (c), the self-attention sublayer in the decoder stack is modified to prevent positions from attending to subsequent positions. The output predictions which are offset by one position, and this masking ensure that the predictions for position index j depend only on the known outputs at positions less than j. We use 6 Transformer Decoders, one for each AU/R. For simplicity, Fig. 1 (a) only illustrates one decoder. AU/R decoder: As depicted in Fig. 1 (a), the Transformer Decoder outputs are concatenated together, then fed to 3 onedimensional convolutional layers that include 64 filters, with a kernel size equal to 3, to learn the correlation between the 6 output features, and therefore the correlation between facial and head movements. Finally, a Dense layer with a Softmax activation function is applied on each of the outputs, to convert the outputs to predicted next-token probabilities. The final output sequences are Y AU and Y R .

Transformer Sub-Layers and Hyperparameters: the transformer encoder and decoders have attention sub-layers, and contain fully connected feed-forward networks which are applied to each position separately and identically. Similarly to other sequence to sequence models, we use learned embeddings to convert the input tokens and output tokens to vectors of dimension d model = 64. All sub-layers and embedding layers therefore use this dimension. The inner feed-forward layers are of dimension d f f = 400. Positional encodings are applied to the inputs of the transformer encoder and decoders. They have the same dimension as the embeddings, so that they can be added together. We use sine and cosine functions, similar to [START_REF] Vaswani | Attention is all you need[END_REF].

III. EXPERIMENTS A. Material and Experimental Setups

We trained it on a subset of the TED dataset collected in [START_REF] Fares | Towards multimodal human-like characteristics and expressive visual prosody in virtual agents[END_REF], containing preprocessed AUs/R, F0s, and BERT embeddings of filtered shots where speakers' face and head are visible and close to the camera. Our subset consists of the features of 200 videos. Videos vary between 2 and 25 minutes, with a frame rate of 24 FPS, the total numbers of IPUs is 919, and of words is 62307. We shuffled all the IPUs, then split them into: training set (80%), validation set (10%) and test set (10%). There are two test conditions: SD (Speaker Dependent) and SI (Speaker Independent). The SD condition aims to assess to what extent the model can generalize on new sentences pronounced by a speaker seen during training -training set included multiple speakers. The SI condition aims to assess the extent to which gestures predictions can be extrapolated to unseen speakers. Each training batch contained 128 pairs of word embeddings, F0 sequences, and their corresponding AUs, RX, RY and RZ. The loss function used is the categorical cross-entropy between predicted and actual values. We used Adam optimizer with β 1 = 0.9, β 2 = 0.98 and ϵ = 10 9 . We used a learning rate scheduler as in [START_REF] Vaswani | Attention is all you need[END_REF], with warmup steps = 4000. We applied a dropout = 0.1 to the output of each sub-layer of the transformer, and to the sums of the positional encodings in the Transformer encoder and decoder stacks. All features values were normalized between 0 and 1. The total number of the model's parameters is 2051133.

B. Objective Evaluation

To assess the quality of the generated gestures, we used the following measures: Root Mean Squared Error (RMSE), Pearson Correlation Coefficient (PCC), Activity Hit Ratio (AHR) and Non-Activity Hit Ratio (NAHR). AHR and NAHR were proposed by Freeman et al. [START_REF] Ong | Real-time robust voice activity detection using the upper envelope weighted entropy measure and the dual-rate adaptive nonlinear filter[END_REF], to evaluate the performance of Voice Activity Detector (VAD) systems. We also considered them since evaluating AU activity looks similar to VAD evaluation. We considered an AU as "Activated" when its value is greater than 0.5, otherwise it is "Not-Activated". AHR is the percentage of predicted AU activation with respect to ground truth. If it is greater than 100%, it means that the model is predicting more activation than the amount of activation that is in the ground truth. NAHR is the same but for non-activity. We assessed the full model using the SD test set. We additionally evaluated its capacity to generalize on the SI set. To evaluate the different parts of our architecture, we conducted an ablation study as follows:

(1) speech ablation, (2) text ablation, (3) CMAM ablation, and (4) AU/R decoder ablation. The ablation study was evaluated using RMSE metric.

C. Comparing to LSTM-based Baseline Model

We compared our approach to a sequence to sequence LSTM-based model [START_REF] Fares | Towards multimodal human-like characteristics and expressive visual prosody in virtual agents[END_REF] to predict upper-face movements based on speech and text. This LSTM model [START_REF] Fares | Towards multimodal human-like characteristics and expressive visual prosody in virtual agents[END_REF] was developed only for predicting eyebrows movements (AUs). We extended it to include head movements (R). This extended model consists of mapping sequences of F0 and the Bert embedding that correspond to a Word (W), to the sequences of (AUs/R) of that corresponding W. It employs 2 layers of Bidirectional LSTMs to encode the concatenation of wordlevel F0 contours and Bert embeddings. The hidden internal states are then transmitted to 6 decoders to produce the corresponding AUs and R for a given input. The decoders are followed by Dense Layers with Softmax Activation. The hyperparameters of the extended LSTM model are as follows: in both encoder and decoders, the first layer of bidirectional LSTM has 200 units, and the second one has 100 units. The activation function used in these layers is LeakyReLU with alpha = 0.01. We trained and tested this model using the same training, validation and test sets described in Section III -A. We trained it on 300 epochs, using a batch size = 128. We used Root Mean Squared Propagation (RMSProp) optimizer, and Categorical Cross Entropy Loss. We used the same metrics described in III.B to compare this SOTA Baseline model to our Transformer-based model.

D. Subjective Evaluation

To investigate human perception of the facial gestures produced by our model, we conducted two different experimental studies using the virtual agent Greta [START_REF] Pelachaud | Greta: a conversing socio-emotional agent[END_REF]. We followed the recommendations proposed in [START_REF] Wolfert | A review of evaluation practices of gesture generation in embodied conversational agents[END_REF], by adapting them to facial gesture generation and assessed the naturalness, coherence, and human-likeness of the virtual agent's gestures. Since we are not evaluating deictic and iconic gestures, we did not use the metrics appropriateness, and intelligibility as proposed in [START_REF] Wolfert | A review of evaluation practices of gesture generation in embodied conversational agents[END_REF]. We added the metrics synchronization, and alignment to evaluate the gestures' temporal property with speech.

Participants in both studies were fluent in English, with a University degree, and recruited on Prolific, a crowd sourcing website. We added attention checks at the beginning of our perceptual evaluations, to filter out inattentive participants.

The first study was done by 35 participants, and consisted of presenting 16 videos: each video showed the virtual agent saying a sequence of words that corresponds to a sequence of IPUs. We considered 4 conditions: 4 videos (condition M) used our full model of SD gestures predictions; 4 videos (condition GT) were simulated using the gestures extracted from TED videos, which serve as ground truth; 4 videos of the virtual agent were simulated using the LSTM-based Baseline model of SD predictions (condition SOTA). The remaining 4 videos were produced using predicted gesture animation of IPUs with the sound of other IPUs (condition E). The second study was conducted by 55 participants. The goal of the second study was to evaluate our model when simulated with SI data, and therefore its capability to generalise to new speakers. It included 8 videos: 4 were simulated with our model's SI predictions, and 4 using SI gestures extracted from SI set (described in III.A), which serve as ground truth. For each video in both studies, participants were asked to rate the 5 factors, namely naturalness, coherence, human-likeness, synchronization, and alignment of the virtual agent's gestures on a 1 to 7 likert scale [START_REF] Wolfert | A review of evaluation practices of gesture generation in embodied conversational agents[END_REF]. The questions were listed in a random order. The agent's mouth movements were blurred to prevent participants from getting distracted by these gestures which were not inferred by our model, and therefore focus on the model's generated gestures.

IV. RESULTS AND DISCUSSION Table I reports the model's as well as the SOTA's objective evaluation results using the same SD test set. Results reveal that RMSE errors are much smaller for M (0.0229 ≤error≤ 0.3) than SOTA (0.2046 ≤error≤0.9786). On the other hand, PCC coefficients show that M's predictions (0.94≤PCC≤1.0) are more correlated than SOTA's predictions (-0.0525≤PCC≤0.0179) to GT. AHR and NAHR were calculated to measure the activation of AUs only, since they are not applicable for R. Results show that M predicts better the activation rate AHR (AHR≥98.0) than SOTA (20.211≤AHR≤60.314). The non-activation rate is higher for SOTA (107.084≤NAHR≤135.95) than for M(100≤NAHR≤130). This constitutes objective validation that M gives better results than SOTA in terms of error, correlation, and AU's activation rate. Ablation studies conducted on our model resulted in higher RMSE errors when performing Speech and Text Ablation for some AUs/R. For instance, AU01 RMSE error (0.0819 for full model) increased to 0.0836 with text ablation and to 0.09 with speech ablation. On the contrary, RZ RMSE score (0.3036 for full model) increased to 0.3089 with text ablation, and 0.3111 with speech ablation. AU/R Decoder ablation resulted in even higher RMSE errors especially for head rotations. AUs and R RMSE scores increased after CMAM ablation (i.e. AU01 RMSE increased to 0.0901). This constitutes objective validation that the use of multi-modal inputs (speech and text modalities) in M improves predictions. Thus we can also conclude that CMAM module is an efficient and a key component of our model, as it improves the generation accuracy of face gestures and head rotations. As mentioned previously, we also tested our model on the SI set. RMSE errors are between 0.301 and 0.89 for AUs, and between 0.25 and 0.93 for R. As we could expect, we got higher errors than the errors we had for the SD condition (Table I) since the speakers in SI set were not seen by our model during the training phase.

For our first perceptive study conducted on SD data, Fig. 2 (a) shows the mean scores obtained on the 5 factors for the 4 conditions: our model M, the baseline SOTA, the ground truth GT, and the error E. For the 5 factors, M is perceived as much closer to GT than SOTA and E, especially in terms of Alignment and Synchronization between speech and gestures. The mean difference between M and GT is 0.72 for Alignment and 0.74 for Synchronization (Fig. 2 (a)). SOTA is perceived as the closest to E especially in terms of Coherence, and Alignment: the mean difference is 0.4 and 0.44 respectively (Fig. 2 (a)). We also conducted ANOVA Between-Subjects to test whether there are significant differences between the 4 conditions along all the 5 factors. Results showed that for all factors, tests were significant (p<0.001). We additionally performed a post-hoc Fisher's LSD Test to do pair-wise comparisons of the means between the factors of all conditions. Significant results (p<0.001) were found for all the factors when comparing the condition GT with the 3 other conditions and when comparing our model M with SOTA and E. In particular, there were significant differences between M and SOTA in terms of the 5 factors (p<0.007). This constitutes experimental validation that when used with SD data, condition M is perceived significantly better than SOTA and E for all the factors. The difference for the 5 factors between M and GT is not significant. For our second perceptive study conducted on SI data, Fig. 2 (b) shows the means scores obtained on all factors for conditions M and GT: for the 5 factors M is perceived as close to GT. As our data were not normally distributed (Shapiro test's p<0.5), we conducted a post-hoc unpaired Wilcoxon test on each factor for the two conditions. For all tests, we could not find significant differences (p>0.05). Thus our model when used with SI data and GT received similar values for the 5 factors.

V. CONCLUSION

We have presented a new approach for modelling upper facial and head gestures using Transformer model. We conducted objective and subjective evaluations that showed that our model produces animations that are close to the ground truth in term of expressivity while ensuring that speech and computed gestures are aligned and synchronized. In a next future, we plan to expand our model to learn different speaker styles for gesture generation.

Fig. 1 :

 1 Fig. 1: Face Gesture Generation Model Architecture B. Face Gesture Generation Model Architecture

( 1 )

 1 W into a word-level representation of F0 contours covering local context of F0 variations. These convolutional layers include 64 filters, with a kernel size equal to 3. The generated output vector of the latter layers X[START_REF] Graf | Visual prosody: facial movements accompanying speech[END_REF]

Fig. 2 :

 2 Fig. 2: Subjective Evaluation Results

TABLE I :

 I Objective Evaluation: comparison of proposed transformer model vs. SOTA baseline model


Sandra Krim
email: sandra.krim@univ-paris1.fr

CONCEPTUALIZING THE MEDIA OF ARTIFICATION: A CONTRIBUTION TO THE THEORY OF ARTIFICATION OF LUXURY BRANDS

Keywords:

Luxury has long thrived on a storytelling where craftmanship, rarity, exclusivity and uniqueness were central. Both the industrialization and the financialization of luxury industry have led to a risk of losing its rarity and exclusivity values associated to the idea of craftmanship. Leveraging on the structural proximity that exists between art and luxury, many luxury brands have been developing connections to art to capitalize on the benefits that could arise in terms of positioning and legitimacy. This conceptual paper studies and extensively investigates the different kinds of existing links between art and luxury brands as well as their related constructs in existing literature, which appear to be marginal and lacking consensus. However, this study identifies a conceptualization, which encapsulates all types of linkage between luxury brands and art that aim at enhancing the perception of luxury brands and luxury goods by consumers. It further proposes a definition of this conceptualization, which will benefit both academics and practitioners.

CONCEPTUALIZING THE MEDIA OF ARTIFICATION: A CONTRIBUTION TO THE THEORY OF ARTIFICATION OF LUXURY BRANDS

Managerial context of the research

Luxury is a highly-performing sector: sales of personal luxury goods amounted 281 billions Euros in 2019 and should recover this level by 2022 (Bain & Co., 2021). Luxury groups perform so well that in 2021 LVMH became the first European market capitalization.

Luxury is also a very financialized sector, with a high concentration of key players (Deloitte Power of Luxury Goods, 2020): In 2019, the ten biggest luxury companies accounted for more than 50% of sales while multi-sectorial luxury groups represented 30% of global sales.

Theoretical context of the research

Over the last decades, the growth of the luxury sector and its financialization have undergone a major shift in the legitimization of the price and symbolic power of luxury. No longer able to thrive on rarity, craftmanship and uniqueness, many luxury brands turned to art to reinforce their moral, aesthetic and symbolic value, while conferring themselves non-commercial connotations and legitimizing their high prices [START_REF] Kapferer | The artification of luxury: From artisans to artists[END_REF][START_REF] Chailan | Art as a means to recreate luxury brands' rarity and value[END_REF].

Due to their structural proximity [START_REF] Kapferer | The luxury strategy: Break the rules of marketing to build luxury brands[END_REF], art and luxury have been related throughout history. However, their relationships are no longer sporadic or fortuitous (or very rarely) but scheduled and strategic [START_REF] Riot | Murakami on the bag: Louis Vuitton's decommoditization strategy[END_REF][START_REF] Kapferer | The artification of luxury: From artisans to artists[END_REF]Passebois-Ducros & Silhouette-Dercourt, 2015) Despite the growing marketing literature on the relationships between art and luxury, the conceptualization of these links between art and luxury brands or goods is only marginally addressed by scholars [START_REF] Chailan | Art as a means to recreate luxury brands' rarity and value[END_REF]. Moreover, academic publications employ various terms to describe such relationships, reflecting a "lack of consensus in conceptualizing the phenomenon" (Koronaki et al., 2017).

Research objective

Ø Provide a global definition of the various strategies used by luxury brands to enhance their perception through links with art.

Approach

This paper is a conceptual exploration. It aims at proposing a global construct that may encompass all forms of linkage between art and luxury brands which lead to the artification of those brands.

Litterature review 1. Contextualization of the artification of luxury

1.1.Phenomenology of the artification of luxury brands

The artification theory developed in luxury brand management by [START_REF] Kapferer | The artification of luxury: From artisans to artists[END_REF] is originally a cross-disciplinary concept. Grounded on a philosophical basis [START_REF] Böhme | Critique of aesthetic capitalism[END_REF] and the development of the artistic capitalism theory [START_REF] Lipovetsky | L'esthétisation du monde. Vivre à l'âge du capitalisme artiste[END_REF], the artification neologism was first introduced by two French sociologists [START_REF] Shapiro | Qu'est-ce que l'artification?[END_REF]Heinich & Shapiro, 2012). According to Heinich and Shapiro (2012), "the artification can be defined as the process of transforming non-art into art, the result of complex work that is an outcome of a change in the definition and status of people, objects and activities". Applied to marketing, the process of artification consists in a positioning strategy with a purpose of universality [START_REF] Kapferer | The artification of luxury: From artisans to artists[END_REF], aiming at providing luxury brand with enhanced legitimacy, reputation and differentiation (Lipovetsky & Roux, 2013).

Despite the fact that the initiators of the artification concept appreciated the luxury industry as a privileged domain of artification (Heinich & Shapiro, 2012), certain necessary conditions must be met so that a specific relationship between art and a luxury brand could be viewed as an artification. Among them, recognition and legitimization by the artistic community and institutions is a mandatory condition (Heinich & Shapiro, 2012;[START_REF] Kapferer | The artification of luxury: From artisans to artists[END_REF].

1.2.Effects of the artification of luxury brands

Art has connotations of excellence, luxury and exclusivity (Hoffman, 2002) and is a strategic opportunity for luxury brands that can enhance both their own symbolic value [START_REF] Kapferer | The artification of luxury: From artisans to artists[END_REF] and the perception of luxury goods by consumers [START_REF] Valette-Florence | Le Luxe ou la quête du Saint Graal![END_REF].

Regarding luxury goods, artification allows the restoration of their exclusivity and rarity [START_REF] Chailan | Art as a means to recreate luxury brands' rarity and value[END_REF], at least what Kapferer considers as abundant rarity (2012). Legitimacy of luxury products is no longer grounded in their objective scarcity but in their status as works of art [START_REF] Passebois-Ducros | Stratégies d'artification dans le domaine du luxe Le cas des vins de prestige[END_REF], which further justify their high price [START_REF] Kapferer | The artification of luxury: From artisans to artists[END_REF]. In this way, artification may be considered as a de-commoditization strategy of luxury products in order to protect and enhance their aura as well as to maintain the value perceived by the consumer [START_REF] Riot | Murakami on the bag: Louis Vuitton's decommoditization strategy[END_REF].

Notwithstanding, artification is more than a mere de-commoditization strategy [START_REF] Jelinek | Art as strategic branding tool for luxury fashion brands[END_REF] as it reinforces luxury brands' singularity and symbolic value [START_REF] Kapferer | The artification of luxury: From artisans to artists[END_REF][START_REF] Chailan | Art as a means to recreate luxury brands' rarity and value[END_REF] through the cultural elitism of those brands [START_REF] Hagtvedt | Art infusion: The influence of visual art on the perception and evaluation of consumer products[END_REF]. In doing so, the artification of luxury brands transcend their status and allows them to be viewed as cultural players [START_REF] Kapferer | The artification of luxury: From artisans to artists[END_REF][START_REF] Joy | M (Art) worlds: consumer perceptions of how luxury brand stores become art institutions[END_REF] as well as to benefit from an "irrefutable aura" (Masè, 2020).

Artification legitimizes what is coined abundant rarity, "where feelings of privilege are attached to the brand itself, seducing through its experiential facets, pricing, prestige and the world it symbolizes" [START_REF] Kapferer | Beyond rarity: the paths of luxury desire. How luxury brands grow yet remain desirable[END_REF]

Overview of the various types of linkage between art and luxury brands and their related constructs through a systematic literature review

The terms used to describe the relationships between luxury brand and arts in existing academic literature are manifold, from the concept of artification initially theorized in marketing by [START_REF] Kapferer | The artification of luxury: From artisans to artists[END_REF], to typologies like the "Luxury Brand-Art Collaborations" (LBCAs) developed by [START_REF] Baumgarth | Arts meet luxury brands. The Management of Luxury[END_REF], or even narrower concepts dedicated to a specific kind of relationship between art and luxury brands like "M(Art)Worlds" at the store level [START_REF] Joy | M (Art) worlds: consumer perceptions of how luxury brand stores become art institutions[END_REF], "art-infused products" [START_REF] Kim | In the eye of the beholder: Cross-pollination between art-infused products and retail spaces[END_REF] or "Luxury Items with Visual Artistic Elements" (LIVAE) [START_REF] Ochkovskaya | Collaboration with art in the luxury industry as a marketing tool for value creation[END_REF] at the product level.

However, three major trends arise from the systematic literature review to define the types of links that bind luxury brands to art (see Table I below).

First, it has been extensively acknowledged that the various links between luxury brands and art may be seen as collaborations [START_REF] Hagtvedt | Art infusion: The influence of visual art on the perception and evaluation of consumer products[END_REF][START_REF] Dion | Retail luxury strategy: assembling charisma through art and magic[END_REF][START_REF] Kastner | When luxury meets art: Forms of collaboration between luxury brands and the arts[END_REF][START_REF] Baumgarth | Arts meet luxury brands. The Management of Luxury[END_REF][START_REF] Jelinek | Art as strategic branding tool for luxury fashion brands[END_REF]. This trend, that was the first one to emerge, is also the most widely spread among marketing scholars [START_REF] Lee | The role of visual art in enhancing perceived prestige of luxury brands[END_REF][START_REF] Codignola | The blending of luxury fashion brands and contemporary art: A global strategy for value creation[END_REF]Baumgarth & Kastner, 2017), even though its use is tending to decrease over the years. This disaffection is due to the weaknesses of the collaboration construct in embracing certain kinds of linkage between art and luxury brands, especially regarding cases where luxury brands are inspired by or refer to art, without actual collaboration with an artist (Baumghart, 2018;[START_REF] Ochkovskaya | Collaboration with art in the luxury industry as a marketing tool for value creation[END_REF]Batat, 2019).

This terminology has been challenged by a term with a wider spectrum, that originates in the management and organization literature. Initially used to describe art-related activities targeted at internal publics within organizations, the Arts-based initiatives concept has been theorized by [START_REF] Schiuma | Mapping arts-based initiatives[END_REF]. Building on a broader definition of arts-based initiative (including "the exploitation of the arts to create intangible value to be incorporated into products") later developed by [START_REF] Schiuma | The value of arts for business[END_REF], another host of scholars elaborated on this specific construct to define the strategic interlinking of luxury brands and art [START_REF] Estes | The value of art in marketing: An emotionbased model of how artworks in ads improve product evaluations[END_REF][START_REF] Koronaki | The emotional value of artsbased initiatives: Strengthening the luxury brand-consumer relationship[END_REF]Batat, 2019;[START_REF] Yang | Creative Art-based Initiatives Enabling Value Co-creation in the Luxury Fashion Industry[END_REF].

More recently, Chailan identified four types of relationships with the arts adopted by luxury brands (namely Business Collaboration, Patronage, Foundations and Artistic Mentoring) that he globally defined as "art to luxury brand" binding modes (2018). This new marketing definition has subsequently been taken up by other scholars (Masè, 2020;[START_REF] Massi | When Fashion Meets Art: The Artification of Luxury Fashion Brands[END_REF]. Based on the fact that the Business Collaboration form fails to capture the bridging of art and luxury in cases where the brand is inspired by or refer to art, it has been suggested to add collaboration with art heritage to Chailan's classification [START_REF] Ochkovskaya | Collaboration with art in the luxury industry as a marketing tool for value creation[END_REF].

Other references

Besides the three main terms used in academic literature to define and conceptualize the different types of links between art and luxury brands (namely collaborations, arts-based initiatives and "art to luxury brand" binding modes), those links may also be defined around two marketing concepts in existing literature: Some authors refer to the strategic dimension of such links by using strategies [START_REF] Joy | M (Art) worlds: consumer perceptions of how luxury brand stores become art institutions[END_REF], art-based strategies [START_REF] Masè | Louis Vuitton's art-based strategy to communicate exclusivity and prestige[END_REF] or even art as strategic branding tool [START_REF] Jelinek | Art as strategic branding tool for luxury fashion brands[END_REF], while other scholars focus on the artification concept (Kapferer, 2015;[START_REF] Vukadin | Opportunities and risks of combining shopping experience and artistic elements in the same store: A contribution to the magical functions of the point of sale[END_REF]. According to some academics, the concept of artification strategies can merge both the aforementioned constructs [START_REF] Passebois-Ducros | Stratégies d'artification dans le domaine du luxe Le cas des vins de prestige[END_REF][START_REF] Massi | When Fashion Meets Art: The Artification of Luxury Fashion Brands[END_REF].

However, the concept of strategy cover neither genuine artistic collaborations under the impulse of the artistic director of the brand nor artistic references made by luxury brands, despite the fact that such types of links may enhance the perception of luxury brands [START_REF] Peluso | Luxury advertising and recognizable artworks: New insights on the "art infusion" effect[END_REF][START_REF] Ochkovskaya | Collaboration with art in the luxury industry as a marketing tool for value creation[END_REF]Batat, 2019).

Despite the merit of the above theoretical contributions in shedding light into the types of links connecting art and luxury brands, it should be acknowledged that none of the proposed terminologies allows to encapsulate all types of linkage between arts and luxury brands while taking into account their processual dimension as well as their brand positioning effects.

Proposed conceptualization

The medium of artification as a much-needed marketing concept

Among the three most commonly used terms that arise from existing literature to conceptualize the different types of relationships between art and luxury brands -i.e. collaborations, artsbased initiatives and "art to luxury brand" binding modes -the broader concept that may embrace the many forms of artification is the one of arts-based initiatives.

However, its wide spectrum may be seen as its own weakness for marketing purposes. The artsbased initiatives concept, which is mostly used in management and organizational studies, covers such a large range of art-related activities that it may be seen as not precise enough to address specific marketing strategies. Its 2,2 millions occurrences on Google Scholar tend to confirm its lack of accuracy to conceptualize a specific marketing strategy like artification.

Therefore, building on Kapferer' seminal article on artification of luxury ( 2014), this study suggest to use the term of medium of artification, that is not defined but mentioned by Kapferer on the side of his proposed artification scale (which itself does not refer to any medium and is more focused on collaborative forms of artification). Such concept is less blurry than the one of arts-based initiatives and more appropriate for marketing studies.

The medium of artification as a global concept embracing all types of linkage between art and luxury brands

The careful analysis of a large body of literature allows to identify three main constructs aiming at encompassing the different types of links connecting art to luxury brands. However, to the exception of arts-based initiatives (which is not suitable for the abovementioned reasons), none of the collaborations or the "art to luxury brand" binding modes -as defined by [START_REF] Chailan | Art as a means to recreate luxury brands' rarity and value[END_REF] -would permit to these concepts to embrace all types of links connecting art to luxury and leading to an enhanced perception of the luxury brand by consumers. Chailan' suggested binding modes, which he defines as "the type and nature of the relationship between an artist or an artistic event and a luxury brand", cover four main types of relationship, namely business collaboration, patronage, foundations and artistic mentoring. References or tributes to art and artists are then outside of the scope of this construct.

The solution suggested by [START_REF] Ochkovskaya | Collaboration with art in the luxury industry as a marketing tool for value creation[END_REF] to remedy this issue is to add collaboration with art heritage to Chailan's classification. This suggestion raises two difficulties. First, the concept of collaboration does not reflect the inspiration process (Thrash & Eliot, 2003) which is at stake when it comes to references and tributes to art or artists by luxury brands, as there is no actual collaborator on the artistic side, which is required by the very definition of a collaboration (Cambridge Dictionary). A fifth type of relationship should then be added.

Second, the definition of so-called binding modes given by Chailan should also be broadened as it only covers "the relationship between an artist or artistic event and a luxury brand".

Therefore, and despite the potential of this terminology, "art to luxury brand" binding modes is too narrow to encompass all kinds of strategic interlinking of art and luxury brands.

Conversely, referring to media of artification allows to embrace all types of linkage between art and luxury brands. It should be further noted that this construct can cover both existing kind of links and future kind of links that could arise (from digitalization, for example).

The medium of artification construct built on Kapferer' seminal article seems the most relevant concept and ought to be defined accordingly.

Proposed definition of the media of artification

According to [START_REF] Kapferer | The artification of luxury: From artisans to artists[END_REF], and despite the fact that he did not give any definition of the concept, there are "multiple media of artification" that are supposed to capture all kinds of strategic interlinking between art and luxury brands aiming at the artification of the latter.

Moreover, by using the artification term, this terminology integrates both the idea of process and the brand positioning effect of the theory when applied to brand management.

Consequently, this study suggest the following definition of media of artification : the media of artification should be referred to as art-related supports or events which aim at enhancing the perception of luxury brands and their products by consumers through products, communication, retail or e-tailing.

Application of the media of artification's concept to the existing forms of linkage between luxury brands and art

By referring to art-related supports or events which aim at enhancing the perception of luxury brands and their products by consumers through products, communication, retail or e-tailing, the media of artification construct can be applied to all known forms of links between art and luxury brands that purport to the artification of such brands.

Possible forms of artification might exist at the product level, the communication level and the store level.

Artification at the product level

Luxury products may be media of artification, either through inspiration by an artist or an art piece [START_REF] Ochkovskaya | Collaboration with art in the luxury industry as a marketing tool for value creation[END_REF]Batat, 2019) or through an actual collaboration with an artist [START_REF] Michel | The Art of Successful Brand Collaborations: Partnerships with Artists, Designers, Museums, Territories[END_REF] or his/her estate.

Product collaborations between luxury brands and artists do mainly occur for luxury fashion and leather good products [START_REF] Massi | When Fashion Meets Art: The Artification of Luxury Fashion Brands[END_REF], as well as wines and spirits [START_REF] Passebois-Ducros | Stratégies d'artification dans le domaine du luxe Le cas des vins de prestige[END_REF]. Louis Vuitton was the pioneer brand to collaborate with artists to offer cobranded products [START_REF] Riot | Murakami on the bag: Louis Vuitton's decommoditization strategy[END_REF][START_REF] Masè | Louis Vuitton's art-based strategy to communicate exclusivity and prestige[END_REF]. It is more scarce as far as high jewelry or fine watchmaking products are concerned.

Collaborations with artist may also be an opportunity for luxury brands to extend their offer, like Givenchy which offered their first digital artwork in the form of NFT (Non-Fungible Tokens) in collaboration with artist Chito in November 2021.

Artification at the communication level

Brand communication and corporate communication, in many of their forms, may be qualified as media of artification, either through inspiration by an artist or an art piece, through a reference or tribute to an artist, an art piece, a type of art or art in general, or through an actual collaboration with an artist or his/her estate or foundation.

Regarding brand communication, advertising pioneered as a medium of artification based on the collaboration of brands with artists [START_REF] Michel | The Art of Successful Brand Collaborations: Partnerships with Artists, Designers, Museums, Territories[END_REF] as well as by incorporating artistic references without any collaboration [START_REF] Lehu | Ceci n'est pas une marque[END_REF][START_REF] Peluso | Luxury advertising and recognizable artworks: New insights on the "art infusion" effect[END_REF].

The whole range of brand management communication tools may be supports or events that proceed to the artification of the brand through collaborations with artists. This includes, among others, fashion shows (Pinchera & Rinallo, 2020), store windows (Wiedmann et al., 2013), popup stores [START_REF] Michel | The Art of Successful Brand Collaborations: Partnerships with Artists, Designers, Museums, Territories[END_REF] as well as digital brand content [START_REF] Kastner | When luxury meets art: Forms of collaboration between luxury brands and the arts[END_REF]. Temporary museum exhibitions of luxury brands proceed to artification as well (De Lassus & Silhouette-Dercourt, 2015).

With respect to corporate communication, it is the first media of artification to have been used by luxury brand through corporate art collections (Kapferer, 2015), artistic mentoring [START_REF] Chailan | Preserving luxury exclusivity through arts. The Management of Luxury[END_REF], patronage of artists, artworks or events [START_REF] Koronaki | The emotional value of artsbased initiatives: Strengthening the luxury brand-consumer relationship[END_REF][START_REF] Massi | When Fashion Meets Art: The Artification of Luxury Fashion Brands[END_REF], corporate art museums (Caru et al., 2017) and foundations [START_REF] Riot | The Louis Vuitton Foundation: The Gift of a CSR Leader and Art Lover to the World[END_REF]Batat, 2019).

Artification at the store level

Retail is viewed as a growing medium of artification, as store artification through the display of artworks and the store atmospherics [START_REF] Dion | Retail luxury strategy: assembling charisma through art and magic[END_REF] has proved to foster customer experience (Venkatesh & Meamber, 2008) as well as to enhance store and product image [START_REF] Vukadin | Store artification and retail performance[END_REF][START_REF] Logkizidou | Why museological merchandise displays enhance luxury product evaluations: An extended art infusion effect[END_REF]. In certain cases, artified stores may be seen by customers as hybrid "art institutions" that enhance brand image [START_REF] Joy | M (Art) worlds: consumer perceptions of how luxury brand stores become art institutions[END_REF].

Theoretical implications

This study aims at bringing a conceptual contribution in the field of luxury brand management through artification strategy, by proposing a global concept referring to all existing and forthcoming types of linkage between art and luxury brand which, to the best of my knowledge, does not exist to date.

Managerial implications

In practice, this study allows luxury professionals to be aware of the whole range of existing or potential media of artification that are or would be available to proceed to an artification strategy for a luxury brand.

Limitations and directions for future research

Beyond the need for a global marketing concept encapsulating all types of linkage between art and luxury brands, it would have been interesting to further investigate those types of links and provide a comprehensive typology of the different media of artification that may be used by luxury brands.

To date, literature is fragmented on this topic as well. We have seen that some authors have categorized different forms of collaborations, whether such categories are divided into arty limited editions, philanthropic art and luxury brand collaboration, experimental art and luxury brand collaborations [START_REF] Baumgarth | Arts meet luxury brands. The Management of Luxury[END_REF] or artistic collaborations, patronage, foundations and artistic mentoring [START_REF] Chailan | Art as a means to recreate luxury brands' rarity and value[END_REF]. There has been other attempts to categorize the different forms of collaborations between luxury brands and art through a marketing-mix approach (Batat, 2019), while a few authors favor artification scales (Kapferer, 2015;[START_REF] Massi | When Fashion Meets Art: The Artification of Luxury Fashion Brands[END_REF]. Moreover, existing categorizations hardly encapsulate all types of linkage between luxury brands and art that may lead to artification.

I therefore propose to follow up this conceptual exploration with later empirical research aiming at defining a model conceptualizing all potential media of artification that may be used by luxury brands. In . Springer Gabler, Wiesbaden.

Bibliography

  Batat, W. (2019). Transforming luxury brand experiences through artification: A marketing and consumer research perspective. Marche et organisations, (2), 135-151. Baumgarth, C. (2014). Kunst und Marke (Teil 2) arts and brands (part 1) Kollaborationen zwischen zwei Welten. transfer-Werbeforschung & Praxis, Vol.60 No. 2 pp. 32-41. Baumgarth, C. (2018). Brand management and the world of the arts: Collaboration, cooperation, co-creation, and inspiration. Journal of Product & Brand Management. Baumgarth, C., & Kastner, O. L. (2017). Luxusmarken durch Kunst stärken-Konzept, Typen und Erfolgsfaktoren von Kunst-Luxusmarken-Kollaborationen.

Table I -Taxonomy of the main constructs used in literature to describe the relationships between luxury brands and art

 I

	Construct Collaborations	Arts-based initiatives	Binding modes
	Authors	Baumgarth et al. (2014) Schiuma (2011)	Chailan (2018)
	Definition None.	"An ABI can be	"By binding
			interpreted as any	modes, we refer to
			management action using	the type and nature
			one or more art forms to	of the relationship
			enable people to undergo	between an artist or
			an aesthetic experience	an artistic event
			within an organisation or	and a luxury
			at the intersection between	brand."
			the organisation and its
			external environment, as
			well as to embed the arts
			as a business asset."
	Typologies Arty limited editions,	Training, coaching,	Artistic
		philanthropic art and	residency activity, team-	collaborations,
		luxury brand	building, creative	patronage,
		collaboration,	investigation, event, art	foundations and
		experimental art and	collection, sponsorship,	artistic mentoring.
		luxury brand	arts and architecture, art
		collaborations.	and design, corporate
			social responsibility,
			embedding the arts into
			organisational life.

[START_REF] Hagtvedt | Art infusion: The influence of visual art on the perception and evaluation of consumer products[END_REF][START_REF] Dion | Retail luxury strategy: assembling charisma through art and magic[END_REF][START_REF] Baumgarth | Arts meet luxury brands. The Management of Luxury[END_REF][START_REF] Lee | The role of visual art in enhancing perceived prestige of luxury brands[END_REF][START_REF] Codignola | The blending of luxury fashion brands and contemporary art: A global strategy for value creation[END_REF] Baumgarth & Kastner, 2017 ; Jelinek,2018, ... Estes et al., 2017;[START_REF] Koronaki | The emotional value of artsbased initiatives: Strengthening the luxury brand-consumer relationship[END_REF][START_REF] Yang | Creative Art-based Initiatives Enabling Value Co-creation in the Luxury Fashion Industry[END_REF][START_REF] Chailan | Preserving luxury exclusivity through arts. The Management of Luxury[END_REF] Masè, 2020(a);Masè , 2020(b), …