Louise-Laure Mariani

Mohamed Doulazmi

Véronique Chaigneau

Christine Brefel-Courbon

Nicolas Carrière

Teodor Danaila

Luc Defebvre

Gilles Defer

Estelle Dellapina

Anne Doé De Maindreville

Christian Geny

David Maltête

Wassilios G Meissner

Olivier Rascol

Stéphane Thobois

Frederic Torny

Christine Tranchant

Marie Vidailhet

Jean-Christophe Corvol

Bertrand Degos
email: bertrand.degos@aphp.fr

Descriptive analysis of the French NS-Park registry: Towards a nation-wide Parkinson's disease cohort?

Keywords:

de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

As expected, ICDs were associated to dopaminergic agonist use but other associations, such as ICDs and sleep disturbances for instance, or anxiety and depression, were found.

Conclusions:

Our results report one of the biggest PD registries ever reported and demonstrate the feasibility of implementing a nation-wide registry of PD patients in France, a potent tool for future longitudinal studies and clinical trials' population selection, and for pharmacoepidemiology and cost-effectiveness studies.

INTRODUCTION:

Parkinson's disease (PD) is the second most common neurodegenerative disorder after The objective of NS-Park is to promote clinical research in PD and movement disorders in order to (i) understand underlying disease mechanisms, and (ii) develop innovative therapeutic strategies. To achieve these scientific objectives, one of the initiative planned by NS-Park was to create a patient registry to i) report the PECs' medical activity to the Ministry of Health, ii) facilitate clinical trials pre-screening of PD patients, and iii) provide a source of "real-life" clinical data for epidemiology and pharmaco-epidemiology studies.

Here, we describe the main clinical characteristics of the population included into the registry during the implementation phase. In addition to classic descriptive statistics, we investigate potential relationships between variables using MCA, an additional descriptive clustering analysis tool. We demonstrate the feasibility of implementing a nation-wide PD registry, globally representative of the PD population reported in clinical studies. Based on these results, NS-Park is now moving towards a longitudinal cohort supported by a web-based solution.

PATIENTS AND METHODS

Study participants

This was an observational cross-sectional study performed between January 2012 and January 2016 in 11 French PECs (Bordeaux, Caen, Lille, Limoges, Lyon, Montpellier, Paris, Reims, Rouen, Strasbourg and Toulouse). Subjects included into the database were patients with a parkinsonian syndrome referred to the Movement Disorders clinics of the aforementioned PECs.

Eligibility criteria were all consecutive patients diagnosed with any cause of Parkinsonism by one of the Movement Disorder specialist of the PECs. Movement disorders specialists were then asked to characterize the patients according to international criteria for diagnosis of PD, multiple system atrophy (MSA), supranuclear palsy (PSP), Lewy body dementia, corticobasal syndrome (CBS), or as "other". Patients were assessed at the time of their first visit since the PEC creation. Diagnosis of PD, MSA, PSP, CBS or LBD was made according to UK Parkinson's Disease Society Brain Bank Criteria for PD [START_REF] Hughes | Accuracy of clinical diagnosis of idiopathic Parkinson's disease: a clinico-pathological study of 100 cases[END_REF], and usual criteria [START_REF] Armstrong | Criteria for the diagnosis of corticobasal degeneration[END_REF][START_REF] Gilman | Second consensus statement on the diagnosis of multiple system atrophy[END_REF][START_REF] Litvan | Clinical research criteria for the diagnosis of progressive supranuclear palsy (Steele-Richardson-Olszewski syndrome): report of the NINDS-SPSP international workshop[END_REF][START_REF] Litvan | Accuracy of clinical criteria for the diagnosis of progressive supranuclear palsy (Steele-Richardson-Olszewski syndrome)[END_REF]. Patients diagnosed with PD were then further assessed.

NS-Park data collection

Data were prospectively collected using a structured questionnaire including demographic data and a physical examination form. Data were obtained during a regular outpatient visit at the PEC. Data were collected by a Movement Disorders specialist interviewing patients and/or their caregivers. Patient characteristics were captured on a standardized paper form which included: age, diagnosis, disease duration, sex, geographical origin, presence or absence of motor and non-motor symptoms, and current PD treatments. Patients were also asked if they would agree or not to be contacted for future clinical trials or studies. Motor and non-motor symptoms were systematically assessed as present or absent as per the opinion of the Movement Disorders specialist, and classified as follows: (i) motor complications (motor fluctuations and dyskinesia);

Standard Protocol Approvals, Registrations, and Patient Consents

According to French regulations at the time this study was conducted, the database was registered at the French National Commission for Data Protection and Liberties (CNIL) and the proposed analysis of the database received approval from the CNIL after a positive advisory opinion from the CCTIRS (the French Advisory Board regarding data processing in Health Research), the medical and scientific section of the CNIL, on the 17 th of November 2016. Oral consent was obtained for all patients by the movement disorder specialist entering the data in the database.

Statistical analysis

Extraction of individual data was performed locally at each center, and then centralized for a pooled data monitoring and analysis. Centralized analyses were done using anonymized data to protect privacy. Individual anonymized data were made available for age at examination and disease duration from which age at onset was calculated. For categorical variables, centers were asked to provide summary statistics with the total number of patients with a positive (Yes), negative (No), or missing data (MD) for each symptom. Data management on missing data and outliers was centrally performed on the pooled dataset, and queries were sent to each center for corrections. For the MCA performed only on data from the Paris center, individual data were made available for all variables. We performed a descriptive analysis for each variable.

Categorical variables are expressed as the ratio of the number of patients presenting the symptom to the total number of patients assessed and as percentages [n/N (%)] so missing data were reported for each item; and continuous variables as mean ± SD.

To further explore and describe relationships between variables, individual data were extracted from one center (Paris center), and a more detailed analysis was performed. The relationship between categorical variables was summarized by cross-tabulations and analyzed by multiple correspondence analyses (MCA) which studies cross-frequency tables (contingency tables). MCA explores the simultaneous relationships between variables in an n-dimensional space, and then uses the distance between the variables in each dimension to establish the similarity degree of variables. The position of the category-points in MCA maps is the basis for revealing the relationship among the investigated variables. MCA was chosen because as compared to other methods, it allows a description of the structure of the data without assuming their underlying distribution. With most techniques for cross-classification methods [START_REF] Fienberg | The Analysis of Cross-Classified Categorical Data[END_REF] categorical data analysis relies on the use of models. In log-linear analysis (one of the most widely used methods), for example, a distribution is assumed under which the data are collected, then a model for the data is hypothesized and estimations are made under the assumption that this model is true. In MCA, it is claimed that no underlying distribution has to be assumed and no model has to be hypothesized, but a decomposition of the data is obtained to study the 'structure' in the data [START_REF] Roux | Geometric Data Analysis: From Correspondence Analysis to Stuctured Data Analysis[END_REF]. Numerical variables such as "Disease duration" were transformed into categorical values and used as a supplementary quantitative descriptive variable not included in the MCA calculations. For the MCA analysis, the 14 main variables were chosen using the heads of each clinical complications category from the standardized form, and the treatments. The 14

RESULTS

Patients' inclusion and case demographics

Between 2012 and 2016, 11,157 patients with a parkinsonian syndrome were included into the database, among which 9,454 (85%) had PD (Figure 1). The rate of recruitment of PD patients was of 197 new patients per month. The distribution of PD patients from the 11 French PECs was 690 from Bordeaux, 334 from Caen, 1,347 from Lille, 350 from Limoges, 1,532 from Lyon, 432 from Montpellier, 2,553 from Paris, 188 from Reims, 529 from Rouen, 413 from Strasbourg and 1086 from Toulouse. Demographic data of PD patients are shown on Table 1a. Mean age at examination was 67.8 ± 10.5 years and mean disease duration at examination was 9.3 ± 7.2 years, with a sex ratio of 1.4:1 (Table 1a). There was no more than 10% of missing data for all parameters but two (RBD and periodic movements), in which data were particularly missing (Table 1a).

Motor and non-motor symptoms

Table 1a describes the motor and non-motor symptoms observed in the PD population. The most frequent complications reported were Levodopa-induced motor complications in 4,620 (51%) patients, followed by sleep disturbances in 4,307 (48%) and axial signs in 3,690 (42%).

Neurovegetative symptoms and psychiatric disorders were reported in a third of the patients (Table 1a), cognitive disorders in 1,584 (18%) and hallucinations in 1045 (12%) patients. ICDs were present in 8% of the patients.

Regarding Levodopa-induced motor complications, fluctuations were more frequent than dyskinesia.

Treatments

Table 1a shows the various treatments under which patients were at their first visit since the PEC creation. Half of the patients received L-DOPA associated with dopaminergic agonists. L-DOPA monotherapy was given to 2,686 (28%) patients. Only 7% of the patients received dopaminergic agonists as monotherapy. MAO and COMT inhibitors were respectively taken in 26% and 15% of patients. Anticholinergic drugs were used in 2% of the patients. Antidepressants (15%) and anxiolytics (10%) were the most frequently used psychiatric medications. 850 (9%) of the patients underwent deep brain stimulation. The main target was the STN and more rarely the VIM nucleus of the thalamus or the GPi.

Detailed analysis of Paris Center data

To describe in more detail how the clinical data may capture the relationship between variables, we further analyzed individual data from the Paris center. Characteristics, complications and treatments of the 2,553 PD patients from Paris center were similar to the general population of the global dataset (see Table 1b and Supplementary Table 1).

Among the 2,553 patients from this center, 536 had disease durations below one year (21%) and 1,248 below five years (49%) (Table 1b). Overall, 309 patients were drug naive "de novo" patients. Ages at examination were between 25 and 94 years and their distribution is shown in Supplementary Figure 1, with a mean age of 66.2 years. Early onset PD was observed in 165 (6.5%) patients with an age at diagnosis below 40 years (Table 1b). Age at diagnosis was below 50 years in 592 (23%) of the patients (Table 1b). One hundred and ninety (7%) patients had lateonset PD with an age at diagnosis above 75 years (Table 1b).

A large majority of patients (71%) was not opposed to being contacted to participate into a forthcoming clinical study. Further detailed characteristics of this NS-Park population fitting the profile as a potential clinical study population are summarized in Table 1b.

The clustering approach of the 14 main variables identified various association of variables depending on disease duration as shown in Figure 2 and Supplementary Figures 234. The contribution of each individual to each parameter is available to the readers upon request.

The respective positions of the variables on the map or on the dendrogram, close to one variable or the other, reflects if there is a stronger association with these variables than the others. Short disease duration of less than 1 year, was more closely correlated to the absence of treatment with L-dopa and neither motor complications nor axial signs as the closest variables were the absence of treatment by L-dopa (L-DA_N) and the absence of motor complications or axial signs (motor complications_N, axial signs_N) (Figure 2, Supplementary Figure 2). Patients with disease duration from 1 to 5 years were likely treated by MAO-B inhibitor and showed strong correlations to having none of the complications and no other treatments (amantadine, dopaminergic agonists, antipsychotic or antidepressants). Disease duration between 5 and 10 years were associated with L-dopa treatment. Disease duration over 10 years showed closer correlations to motor and non-motor complications, deep brain stimulation and treatments for complications (antidepressants, antipsychotics, antidyskinetic drugs as amantadine and against fluctuations as COMT Inhibitors) (Figure 3, Supplementary Table 2). Presence of ICDs was closely related to dopaminergic agonist use (Supplementary Figure 2). An analysis of all 35 variables showed very similar characteristics (Supplementary Figure 3; Supplementary Table 3) and other associations between subcategories of symptoms such as ICDs and sleep disturbances for instance, or anxiety and depression. As an example, the association of anxiety and depression can be seen on the MCA map, as both Anxiety-Yes and Depression-Yes are close to each other, and in the same way, Anxiety-No and Depression-No are close to each other (Supplementary Figure 3).

Based on the results of MCA, we applied hierarchical clustering on variables combined with a heat map representation showing the profile of the PD population during disease progression (Figure 3). The frequency of motor complications, falls, cognitive disorders and L-DA use increased over time. Neurovegetative symptoms, psychiatric disorders, sleep disturbances and ICDs seem to be stable over time. Treatments use and their distribution over disease course reflect the frequency of occurrence of the symptoms they are used to treat (Figure 3).

DISCUSSION

This feasibility study including over 11,000 patients of the NS-Park database is one of the largest datasets of PD patients ever reported. The number of PD patients in France is estimated at 150,000 [START_REF] Elbaz | Epidemiology of Parkinson's disease[END_REF], thus this initial report of the NS-Park registry represents approximately 10% of the French PD population. A recent review reported the results of 44 studies with a total of 14,666 participants (cohorts' median: 138; range: 23-3.090) [START_REF] Heinzel | Global, Yet Incomplete Overview of Cohort Studies in Parkinson's disease[END_REF], hence placing the number of patients in the NS-Park database above any of the cohorts previously reported.

Compared to the literature, our results provide a congruent picture as compared to the general parkinsonian population. The relative frequencies of the various parkinsonian syndromes reported here are in accordance to previous reports [START_REF] Hughes | Accuracy of clinical diagnosis of idiopathic Parkinson's disease: a clinico-pathological study of 100 cases[END_REF][START_REF] Schrag | Prevalence of progressive supranuclear palsy and multiple system atrophy: a cross-sectional study[END_REF][START_REF] Golbe | Prevalence and natural history of progressive supranuclear palsy[END_REF][START_REF] Bower | Incidence of progressive supranuclear palsy and multiple system atrophy in Olmsted County, Minnesota, 1976 to 1990[END_REF][START_REF] Erkkinen | Clinical Neurology and Epidemiology of the Major Neurodegenerative Diseases[END_REF][START_REF] Chrysostome | Epidemiology of Parkinsonian syndromes[END_REF][START_REF] Marsden | Parkinson's disease[END_REF][START_REF] Schrag | Cross sectional prevalence survey of Parkinson's disease and Parkinsonism in London[END_REF]. PD is by far the most frequent parkinsonian syndrome with a male-to-female sex ratio and age as previously reported [START_REF] Hughes | Accuracy of clinical diagnosis of idiopathic Parkinson's disease: a clinico-pathological study of 100 cases[END_REF][START_REF] Schrag | Rate of clinical progression in Parkinson's disease. A prospective study[END_REF]. Rates of complications reported here are also in accordance to previous reports [START_REF] Schrag | Rate of clinical progression in Parkinson's disease. A prospective study[END_REF][START_REF] Schrag | How common are complications of Parkinson's disease?[END_REF][START_REF] Ahlskog | Frequency of levodopa-related dyskinesias and motor fluctuations as estimated from the cumulative literature[END_REF][START_REF] Reijnders | A systematic review of prevalence studies of depression in Parkinson's disease[END_REF][START_REF] Katunina | The Epidemiology of Nonmotor Symptoms in Parkinson's Disease (Cohort and Other Studies)[END_REF]. Additionally, depression and anxiety also seem to be closely correlated on the MCA [START_REF] Rutten | The bidirectional longitudinal relationship between insomnia, depression and anxiety in patients with early-stage, medication-naïve Parkinson's disease[END_REF][START_REF] Mele | Detecting anxiety in individuals with Parkinson disease: A systematic review[END_REF][START_REF] Goodarzi | Guidelines for dementia or Parkinson's disease with depression or anxiety: a systematic review[END_REF].

The clustering with MCA shows relevant relationships between disease duration, motor and non-motor symptoms, and treatments. It shows the NS-Park/F-CRIN database can easily provide extended correlation information on a large-scale population as individual level data is made available to the researchers of the NS-Park network in the new computerized version of the NS-Park/F-CRIN database. It is a powerful tool to screen for patients with specific clinical characteristics. The global pooled analysis and the MCA analysis from one center both confirmed that the NS-Park PD population reported here seems representative and congruent with the literature [START_REF] Hughes | Accuracy of clinical diagnosis of idiopathic Parkinson's disease: a clinico-pathological study of 100 cases[END_REF][START_REF] Schrag | Prevalence of progressive supranuclear palsy and multiple system atrophy: a cross-sectional study[END_REF][START_REF] Golbe | Prevalence and natural history of progressive supranuclear palsy[END_REF][START_REF] Bower | Incidence of progressive supranuclear palsy and multiple system atrophy in Olmsted County, Minnesota, 1976 to 1990[END_REF][START_REF] Erkkinen | Clinical Neurology and Epidemiology of the Major Neurodegenerative Diseases[END_REF][START_REF] Chrysostome | Epidemiology of Parkinsonian syndromes[END_REF][START_REF] Marsden | Parkinson's disease[END_REF][START_REF] Schrag | Cross sectional prevalence survey of Parkinson's disease and Parkinsonism in London[END_REF][START_REF] Rodriguez-Sabate | The Multiple Correspondence Analysis Method and Brain Functional Connectivity: Its Application to the Study of the Nonlinear Relationships of Motor Cortex and Basal Ganglia[END_REF]. Almost all parameters are very similar between the Paris Center and the global dataset from all centers. The percentage of patients with short disease durations is slightly higher at the Paris Center. This may be due to the specific early state outpatient clinics, the "De Novo" consultation, which was set up in this center in order to offer an expert opinion to patients referred by physicians early during disease course [START_REF] Ruggeri | De-novo" consultation: Evaluation of an outpatient's clinic dedicated to early diagnosis of parkinsonian syndromes[END_REF]. The NS-Park/F-CRIN database may also help identifying or validating novel clinical correlation findings.

For instance, we showed an association between ICDs and sleep disturbances, an observation previously suggested by a smaller sample study reporting an association between RBD and pathological gambling [START_REF] Fantini | Increased risk of impulse control symptoms in Parkinson's disease with REM sleep behaviour disorder[END_REF]. In the same way, we showed an association between OSAS, depression and anxiety, when a previous report failed to show any correlation, probably because of small sample size [START_REF] Mery | Reduced cognitive function in patients with Parkinson disease and obstructive sleep apnea[END_REF].

MCA was chosen because it allows analyzing all outcomes simultaneously, characterizing the patients' profiles and considering outcomes together with the other exploratory variables. MCA is a useful tool for investigating the interrelationships among categorical variables. The MCA has several features and some advantages that distinguish it from other techniques of data analysis [START_REF] Rodriguez-Sabate | The Multiple Correspondence Analysis Method and Brain Functional Connectivity: Its Application to the Study of the Nonlinear Relationships of Motor Cortex and Basal Ganglia[END_REF]. This multivariate nature allows revealing relationships, which could occur during a series of pair wise comparisons of variables. As opposed to log-linear analysis, no underlying distribution has to be assumed and no model has to be hypothesized, but a decomposition of the data is obtained to study the 'structure' in the data [START_REF] Mery | Reduced cognitive function in patients with Parkinson disease and obstructive sleep apnea[END_REF]. Another advantage is related to joint graphical displays. This graphical display produces two dual displays whose row (patients) and column (variables) geometries have similar interpretations. This facilitates the analysis to detect different relationships. In other multivariate approaches to graphical data representation, this duality is not present. MCA helps to describe patterns of relationships distinctively using geometrical methods by locating each variable of analysis as a point in a lowdimensional space. MCA is useful to map both variables and individuals, allowing the construction of complex visual maps, structuring of which can be interpreted. The MCA works effectively for the large data matrix structure either unknown or poorly understood.

A limitation of the results presented here is that they are cross-sectional and not longitudinal yet. Results correlating disease duration with symptoms and treatment has thus to be taken with caution. However, patients being evaluated at the first visit were of various profiles: de novo, short or long disease duration. Hence, a partial picture of the PD population with different disease duration and treatments was already available. In the future version of the NS-Park database, it is planned to prospectively assess patients at each visit at the center, i.e. usually once to twice a year. The population referred to PECs, tertiary university-based centers with special expertise in movement disorders, may differ from the general PD population, for instance regarding the age at onset, the percentage of De novo patients with no treatment or the percentage of patients undergoing DBS treatment. Our results here however show congruency with the characteristics of the PD population usually reported in the literature, probably because such results are often reported by similar tertiary centers. Another limitation is that motor and non-motor symptom assessments were not quantitative, and they were based on movement disorder specialists' opinion, not standardized definitions. To prevent both of these limitations, the future NS-Park/F-CRIN cohort will allow further evaluation of each parameter by using the international validated scoring of the MDS-UPDRS scale [START_REF] Goetz | Movement Disorder Society-sponsored revision of the Unified Parkinson's Disease Rating Scale (MDS-UPDRS): scale presentation and clinimetric testing results[END_REF]. A webbased interface has been implemented for easier collection by the Movement Disorder specialist to limit missing data.

We demonstrate here the feasibility to implement, at a national level, a unique homogenous multicenter database, a potent tool for future longitudinal studies and clinical trials' population selection. Since this feasibility study reporting the feasibility of the recruitment of a large registry, NS-Park/F-CRIN has moved towards a more structured cohort of PD patients involving all PECs in France, using semi-quantitative items for motor and non-motor symptoms, a web-based interface for collecting data, and a quality management plan. The perspective of this new longitudinal cohort will have to be reported in a future study to confirm the feasibility of the prospective evaluation. This initiative is in agreement with precompetitive data sharing to address unmet needs in PD encouraged by others [START_REF] Stephenson | Precompetitive Data Sharing as a Catalyst to Address Unmet Needs in Parkinson's Disease[END_REF][START_REF] Ohmann | Sharing and reuse of individual participant data from clinical trials: principles and recommendations[END_REF]. Pairing with national Health insurance databases will soon be available and will allow for pharmaco-epidemiology and costeffectiveness studies in the French population as it has also been recently reported in the Netherlands [START_REF] Ypinga | Effectiveness and costs of specialised physiotherapy given via ParkinsonNet: a retrospective analysis of medical claims data[END_REF]. The future NS-Park/F-CRIN cohort should be a tremendous and compelling opportunity to acquire nation-wide information in terms of epidemiological, pharmacological and clinical data in PD. The perspective of such clinical cohorts in the long term will be to aggregate biological and brain imaging information to clinical data in order to become suitable platforms for precision clinical trials and personalized medicine.

TABLE/FIGURE LEGENDS:

4 -

 4 esupp-"Online-only text": listing of the coinvestigators members of the NS-Park/F-CRIN Network study group Corresponding author: Bertrand Degos Department of Neurology AP-HP, Hôpital Avicenne, Hôpitaux Universitaires de Paris -Seine Saint Denis 125 Rue de Stalingrad 93022, Bobigny, France Phone: +33 148955400; Fax: +33 148955409 email address: bertrand.degos@aphp.fr Declaration of Interest: Financial Disclosure/Conflict of Interest: Declarations of interest relevant to the study: none. Full Financial Disclosures unrelated to the current research: L-L.M. has received research support grants from INSERM, JNLF, The L'Oreal Foundation; speech honoraria from Lundbeck, Sanofi-Genzyme and Teva; and received travel funding from the Movement Disorders Society, ANAINF, Merz, Teva, Sanofi-Genzyme, AbbVie and Medtronic, outside the submitted work. M. D. declares no conflict of interest. V.C. declares no conflict of interest. C. B-C. has received research support grants from Fondation de France PHRC National, France Parkinson Association; speech honoraria from UCB, Teva, ABBVIE, Aguettant, Orkyn, Zambon and received travel funding ABBVIE, Aguettant, Orkyn, Zambon N.C. has received speech honoraria from Abbvie, Orkyn T.D. declares no conflict of interest. L.D. has received speech honoraria from Abbvie, Orkyn, Zambon, UCB G.D. declares no conflict of interest. E.D. declares no conflict of interest. A.D.d.M. declares no conflict of interest. C.G. received travel funding from Home Air. D.M. declares no conflict of interest. W.M. has received fees for editorial activities with Springer, for consultancy activities from Sanofi, Lundbeck and Affiris, teaching honoraria from UCB and MDS, as well as research support from the Michael J Fox Foundation, the University Hospital Bordeaux, the French Health Ministry, the European Community, ANR, ARAMISE, PSP-France, MSA Coalition, ARAMISE, LABEX Excellence Initiative. O.R. has received scientific grants from: Agence Nationale de la Recherche (ANR), CHU(FP7, H2020) de Toulouse, France-Parkinson, INSERM-DHOS Recherche Clinique Translationnelle, MJFox Foundation, Programme Hospitalier de Recherche Clinique, European Commission (FP7 & H2020) and has acted as a scientific advisor for: AbbVie, Adamas, Acorda, Addex, AlzProtect, Apopharma, Astra Zeneca, Bial, Biogen, Britannia, Clevexel, Cynapsus, INC Reasearch, Lundbeck, Merck, MundiPharma, Neuroderm, Novartis, Osmotica, Oxford Biomedica, Parexel, Pfizer, Prexton Therapeutics, Quintiles, Sanofi, Servier, Teva, UCB, XenoPort, Zambon. S.T. reports grants from Fondation pour la Recherche Médicale, France Parkinson, Neurodis and personal fees from UCB, Medtronic, Teva, St Jude, Novartis, Aguettant, Zambon, Abbvie, outside the submitted work. F.T. declares no conflict of interest. C.T. acted as a scientific advisor for Abbvie and Zambon. M.V. declares no conflict of interest. J.C.C. declares no conflict of interest. B.D. has received research support grants from Fondation de France, INSERM (COSSEC), ANR; speech honoraria from Ipsen, MERZ Pharma, Orkyn; and received travel funding from MERZ Pharma, Elivie. ABSTRACT: Introduction: Parkinson's disease (PD) is the second most common neurodegenerative disorder after Alzheimer's. The French clinical research network for PD (NS-Park) has created a national patient registry to i)report medical activity of Parkinson Expert Centers (PECs) to the Ministry of Health, ii)facilitate PD patients pre-screening for clinical trials, iii) provide a source for pharmaco-epidemiology studies. Objective: Assess the French Parkinsonian population at a nation-wide level and discover new clinical characteristics. Methods: In this feasibility study, PECs prospectively collected clinical data in a standardized manner. The population main clinical characteristics are described, focusing on motor and nonmotor symptoms and treatments, assessing its representativeness. By using an unbiased clustering with multiple correspondence analysis (MCA), we also investigate potential relationships between multiple variables like symptoms and treatments, as clues for future studies. Results: Between 2012 and 2016, among 11,157 included parkinsonian syndromes, 9,454 (85%) had PD. MCA identified various profiles depending on disease duration. Occurrences of motor complications, axial signs, cognitive disorders and Levodopa use increase over time. Neurovegetative symptoms, psychiatric disorders, sleep disturbances and impulse control disorders (ICDs) seem stable over time.

  Alzheimer's disease. Symptomatic dopaminergic replacement therapy does not prevent debilitating complications occurring during disease course. Development of new symptomatic or disease modifying drugs remains a major challenge in PD. The French Clinical Research Infrastructure network (F-CRIN; www.F-CRIN.org/en) is the French counterpart of ECRIN, the pan-European support infrastructure for multinational clinical trials (www.ecrin.org). The missions of F-CRIN are to: (i) promote French clinical research abroad, (ii) support large academic multicenter clinical trials under French coordination, (iii) encourage early and innovative studies such as proof of concept studies in partnership with industries. In 2013, F-CRIN supported national clinical research networks focusing on specific diseases. Meanwhile, the French Government labeled 24 Parkinson expert centers (PECs) through the 2012-2016 plan for PD[1]. The pre-existing network of 16 clinical investigation centers involved in PD research was extended to the 24 PECs to create the French clinical research network for PD and movement disorders (NS-Park) which was retained for funding by F-CRIN.

(

  ii) axial signs (falls or postural instability, freezing of gait, camptocormia, dysarthria, swallowing difficulties); (iii) neurovegetative symptoms (gastro-intestinal, sphincter or erectile dysfunction, postural hypotension); (iv) sleep disturbances (daytime sleepiness, insomnia, REM-sleep behavioral disorder (RBD), restless leg syndrome (RLS), periodic movements, obstructive sleep apnea syndrome (OSAS)); (v) cognitive disorders (apathy, and dementia); (vi) psychiatric disorders (depression, anxiety, psychosis); (vii) hallucinations; (viii) impulsive-compulsive disorders (ICDs) (pathological gambling, pathological shopping, hypersexuality, binge eating, addiction to L-Dopa, punding). Treatments were assessed as antiparkinsonian medication subtypes (L-dopa, dopamine agonists, monoamine oxidase (MAO) or cathechol-O-methyltransferase (COMT) inhibitors, anticholinergic drugs), psychiatric medication subtypes (antidepressants, antipsychotics, anxiolytics). Presence/absence of deep-brain stimulation (DBS) and its target (Subthalamic nucleus (STN), Ventral intermediate medial nucleus (VIM) of the thalamus or the internal Globus pallidus (GPi)) were also assessed.

  variables were motor complications, axial signs, neurovegetative symptoms, sleep disturbances, cognitive disorders, psychiatric disorders, ICDs, L-DA, dopaminergic agonists, MAO-B-I, COMT-I, DBS, antidepressants and antipsychotics. All 35 variables were including all subcategories of symptoms from the standardized form, for a more detailed description : motor fluctuations, dyskinesia, falls or postural instability, freezing of gait, camptocormia, dysarthria, swallowing difficulties, gastro-intestinal dysfunction, sphincter or erectile dysfunction, postural hypotension, daytime sleepiness, insomnia, RBD, RLS, periodic movements, OSAS, apathy, dementia, depression, anxiety, psychosis, hallucinations, pathological gambling, pathological shopping, hypersexuality, binge eating, addiction to L-DA, punding, L-DA, dopaminergic agonists, MAO-B-I, COMT-I, DBS, antidepressants, antipsychotics. Missing values were taken into account as "missing data" in the MCA analysis. Based on the results of MCA, we applied hierarchical clustering on variables combined with a heat map representation to identify different profiles. Clustering techniques allow classifying patients or variables within homogeneous subsets (clusters) through the definition of a distance between individuals on the basis of their characteristics. The score (coordinates) obtained on the factorial axes identified through the MCA was used to calculate distances. The classic descriptive analysis was performed using Microsoft Excel and Graphpad Prism softwares. For MCA, statistical analysis were implemented with R (3.3.0) with FactoMineR and ClustOfVar packages.

  Abbreviations: COMT: Cathechol-O-methyl-transferase; DBS: Deep brain stimulation; MAO: Monoamine oxidase; REM-sleep: Rapid Eye Movement sleep; SD: Standard Deviation; Vim: Ventral intermediate medial nucleus Table 1b: Parkinson's disease patients from the Parkinson Expert Center in which Multiple Correspondence Analyses was performed Categorical variables are expressed as the number of patients and as percentages [n (%)] and continuous variables as mean ± SD [range] (number of patients assessed).

Figure 1 :

 1 Figure 1: Flow chart of the patients with a parkinsonian syndrome included in the NS-Park/F-CRIN database. Abbreviations: PSP: Progressive Supranuclear Palsy, MSA: Multiple System Atrophy, CBS: Cortico-Basal Syndrome, LBD: Lewy Body Dementia.

Figure 2 :

 2 Figure 2: Multiple correspondence analysis map showing the positions and the association between 14 variables Multiple correspondence analysis map showing the positions and the association between 14 variables for patients with Parkinson's disease in the Expert Center where MCA was performed (N=2553). The horizontal and vertical axes represent the first and second principal dimensions, respectively. The first two axes accounted for 20.4% and 12.1% of inertia. Contribution from the third dimension was only 7.2 %. Motor complications, axial signs, neurovegetative symptoms and sleep disturbances are the most important variables contributing to the variance accounted for by axis 1. Axial signs, motor complications, cognitive disorder and disease duration are the most important variables contributing to the variance accounted for by axis 2 (See Supplementary Table 2). The modalities of categorical variables are specified as a suffix for Yes "_Y" colored in blue and No "_N" colored in black. Abbreviations: COMT-I: Cathechol-O-methyl-transferase inhibitor; DBS: Deep Brain Stimulation; L-DA: L-DOPA; MAO-I: Monoamine oxidase inhibitor.

Figure 3 :

 3 Figure 3: Heatmap of the similarity matrix of the Parkinson's disease "patients' stratifications" Heatmap of the similarity matrix of the "patients' stratifications" of patients with Parkinson's disease in the Expert Center in which MCA was performed. Each row and column means a patient and categorical variable respectively. In the heat map, the blue, white and red colors reflect the modalities of categorical variables (Yes, No and Missing Data respectively). Abbreviations: COMT-I: Cathechol-O-methyl-transferase inhibitor; DBS: Deep Brain Stimulation; L-DA: L-DOPA; MAO-I: Monoamine oxidase inhibitor, MD: Missing Data, N: No, Y: Yes.

Table 1 : Description of patients with Parkinson's disease Table 1a: Parkinson's disease patients' demographics, complications and medications

 1

ACKNOWLEDGMENTS

Funding sources for study:

The research leading to these results has received funding from the program "Investissements d'avenir" ANR-10-IAIHU-06, Institut national de la santé et de la recherche médicale (INSERM), Ministère des Solidarités et de la Santé -DGOS, Paris, France Agence nationale de la recherche, PIA 1 "INBS", F-CRIN, Paris, France. We are very grateful to the patients and family members who participated in the study. The authors would like to thank the following contributors: Carole Dongmo-Kenfack (CRA) and Minh-ha Morvan (CRA), CIC Neurosciences, Parkinson Expert Center, Pitié-Salpêtrière Hospital, Paris, France ; Rachida Bari (CRA), Department of Neurology, Caen University-Hospital, Normandie University, Caen, France; Nadia Barun (CRA) and Fanny Huselstein (CRA), Department of Neurology, Hopitaux Universitaires, Strasbourg, France Margaux Bonnaire (CRA), Department of Neurology, Hôpital Maison Blanche, Reims, France Morgane Gaboreau (CRA), Service de Neurologie, Hôpital Pellegrin, CHU de Bordeaux, 33000 Bordeaux, Univ. de Bordeaux, Institut des Maladies Neurodégénératives, CNRS UMR 5293, 33000 Bordeaux, France ; Claudia Verna (CRA); Department of Neurology, CHRU Montpellier, Montpellier, France ; Elise Metereau (CRA); Parkinson Expert Center, Hôpital Neurologique Pierre Wertheimer, Hospices Civils de Lyon, Lyon, France ; Estelle Harroch (CRA), Parkinson Expert Center, CHU Toulouse ; Toulouse; Olivier Villeneuve (CRA) ; Service de Neurologie, Hôpital Dupuytren, CHU de Limoges, 87042 Limoges Cedex, France for their participation in the acquisition of data. The authors would also like to thank the coinvestigators of the NS-Park/F-CRIN network study

group listed in the "online-only text". The horizontal and vertical axes represent the first and second principal dimensions, respectively. The first two axes accounted for 24.3% and 6.5% of inertia. Contribution from the third dimension was 4.4%. Motor fluctuations, dyskinesia, falls or postural instability, freezing of gait, camptocormia, dysarthria and swallowing difficulties are the most important variables contributing to the variance accounted for by axis 1. Falls or postural instability, dysarthria, freezing of gait, hallucinations, dementia, motor fluctuations and disease duration are the most important variables contributing to the variance accounted for by axis 2 (See Supplementary Table 3). The modalities of categorical variables are specified as a suffix for Yes "_Y" colored in blue and No "_N" colored in black.

Online-only tables

Supplementary Table 1, Supplementary


Mohamed Ben Ammar

Maud Nouri-Neuville

François H Cornelis
email: francois.cornelis@aphp.fr

Ben Ammar

Percutaneous Image-guided therapies of primary liver tumors: techniques and outcomes

Keywords: Barcelona Clinic Liver Classification CT, computed tomography DEB, drug eluting beads HCC, hepatocelular carcinoma MRI, magnetic resonance imaging MWA, microwave ablation PET/CT, positron emission tomography -computed tomography radiofrequency ablation TACE, transarterial hepatic chemoembolization ultrasound Y90, Yttrium-90

Percutaneous Image-guided therapies of primary liver tumors: techniques and outcomes

Introduction

Treatments of primary and secondary liver tumors have progressively gain acceptance worldwide. Image-guided procedures such as embolization and percutaneous ablations became well established therapeutic options. They allow to treat liver tumors under minimally invasive conditions [START_REF] Covey | Liver-Directed Therapy for Hepatocellular Carcinoma: An Overview of Techniques, Outcomes, and Posttreatment Imaging Findings[END_REF][START_REF] Cornelis | Treatment of Primary Liver Tumors and Liver Metastases, Part 2: Non-Nuclear Medicine Techniques[END_REF]. It may result in more rapid recovery, fewer complications, shorter hospital stays and reduced medical costs [START_REF] Solomon | Interventional Molecular Imaging[END_REF][START_REF] Cornelis | The interventional oncologist: The fourth musketeer of cancer care[END_REF]. Selection of a technique over another follows guidelines, such as the Barcelona Clinic Liver Classification (BCLC) for hepatocelular carcinoma (HCC) [START_REF] Vogel | Hepatocellular carcinoma: ESMO Clinical Practice Guidelines for diagnosis, treatment and follow-up[END_REF] but also patient's status and local availability of the techniques. The aim of this review is to present the techniques performed routinely in Interventional Radiology Departments.

Transarterial chemoembolization

The rationale for transarterial hepatic chemoembolization (TACE) is the preferential blood supply of HCC derived from the hepatic artery [START_REF] Idée | Use of Lipiodol as a drug-delivery system for transcatheter arterial chemoembolization of hepatocellular carcinoma: A review[END_REF]. TACE is recognized as the primary treatment for BCLC stage B, asymptomatic, non-invasive and multinodular HCC and as an option to treat cholangiocarcinoma and metastases of colorectal cancer, neuroendocrine tumors, breast cancer or melanoma [START_REF] Covey | Liver-Directed Therapy for Hepatocellular Carcinoma: An Overview of Techniques, Outcomes, and Posttreatment Imaging Findings[END_REF]. In general, patients with unresectable disease involving less than 50% of the liver with well-compensated cirrhosis (Child-Pugh score A or B) may be candidates [START_REF] Idée | Use of Lipiodol as a drug-delivery system for transcatheter arterial chemoembolization of hepatocellular carcinoma: A review[END_REF]. The schedule of the procedures is adaptated according to the individual response to treatment.

The aim of this technique is to concomitantly perform both local embolization and chemotherapy. However, it requires that afferent arteries must be adequately selected to avoid damaging the surrounding healthy parenchyma (Figure 1). A precise planning using cone beam computed tomography and navigation softwares improves the outcomes [START_REF] Cornelis | Hepatic Arterial Embolization Using Cone Beam CT with Tumor Feeding Vessel Detection Software: Impact on Hepatocellular Carcinoma Response[END_REF]. The selective catheterization of the hepatic artery is performed mainly via the femoral artery followed by selective embolization of the hepatic artery branches feeding the tumor [START_REF] Pesapane | New concepts in embolotherapy of HCC[END_REF]. A solution, emulsion or suspension of chemotherapy is injected intra-arterially. It is mixed with a water-soluble contrast medium and an embolic agent in order to monitore the injection, to retain the drug in the tumor and to exert embolic effects. The chemotherapy used in this setting is doxorubicin mainly, cisplatin, epirubicin, mitoxantrone, or mitomycin C. Embolic agents can be spherical or non-spherical, calibrated or non-calibrated, resorbable or nonresorbable. All of them induce or enhance tumour ischaemic necrosis [START_REF] Brown | Randomized Trial of Hepatic Artery Embolization for Hepatocellular Carcinoma Using Doxorubicin-Eluting Microspheres Compared With Embolization With Microspheres Alone[END_REF]. Conventional TACE use a solution for lipophilic drugs or an emulsion of the drug and ethiodized oil (Lipiodol, Guerbet, France) [START_REF] Idée | Use of Lipiodol as a drug-delivery system for transcatheter arterial chemoembolization of hepatocellular carcinoma: A review[END_REF]. The resulting emulsification procedure is operator-dependent.

Depending on the tumour size, vascularity, and fluoroscopic findings, a total volume of about 10-15 mL of Lipiodol is injected mixed with a similar volume of drug. Alternatively, drug eluting beads (DEB)-TACE is performed by injecting through the catether spherical embolic beads (in the 40-900 micrometers range) loaded with the chemotherapeutic agent [START_REF] Pesapane | New concepts in embolotherapy of HCC[END_REF]. These beads were developed to overcome the difficulty in obtaining reproducible emulsions with Lipiodol and to standardize the size of occlusion marterial. As now the size of these beads may be less than 100 micrometers, the embolic effects may also be increased requiring supraselective embolization of the afferent vessels to the tumor. The bond between the beads and chemo allows slow and sustained release of the drug. Less systemic chemotherapy exposure than in conventional TACE has been reported. However, none of the agents used thus far has been demonstrated to be clinically superior to others. No randomized clinical trial have been specifically conducted to date.

The largest retrospective study of embolization included 322 HCC patients and showed an overall survival of 21 months [START_REF] Maluccio | Comparison of survival rates after bland arterial embolization and ablation versus surgical resection for treating solitary hepatocellular carcinoma up to 7 cm[END_REF]. In a study involving 112 patients [START_REF] Llovet | Arterial embolisation or chemoembolisation versus symptomatic treatment in patients with unresectable hepatocellular carcinoma: a randomised controlled trial[END_REF], 1-and 2year survival probabilities were 75% and 50% for with embolization using gelatin sponge particles, a resorbable embolic agent, and 82% and 63% for conventional TACE (using doxorubicin as cytotoxic agent) and 63% and 27% for control (symptomatic treatment), respectively (p = 0.009 for TACE vs. control). A meta-analysis assessing overall survival for primary treatment of HCC (503 patients) (13) showed a significant 2-year survival benefit associated with TACE compared with conservative management (odds ratio 0.53 [0.32-0.89], p = 0.017). Several studies compared conventional TACE with DEB-TACE [START_REF] Covey | Liver-Directed Therapy for Hepatocellular Carcinoma: An Overview of Techniques, Outcomes, and Posttreatment Imaging Findings[END_REF]. None of those found difference in complete response, overall response, or disease control rates between study arms but reported fewer side-effects in patients treated with DEB-TACE.

Similarly, in a prospective randomized trial including 101 patients with inoperable HCC [START_REF] Brown | Randomized Trial of Hepatic Artery Embolization for Hepatocellular Carcinoma Using Doxorubicin-Eluting Microspheres Compared With Embolization With Microspheres Alone[END_REF], the authors compared DEB-TACE and TAE with unloaded beads and found no difference in time to progression, progression-free survival, or overall survival between groups. The median survival time was 20.8 months in the DEB-TACE arm and 19.6 months in the TAE arm, which was not signifiacant.

These treatments are generally considered to be well tolerated. Major complications occur in only 4-7% of procedures and the 30-day mortality rate is of approximately 1% [START_REF] Monier | Liver and biliary damages following transarterial chemoembolization of hepatocellular carcinoma: comparison between drug-eluting beads and lipiodol emulsion[END_REF].

The most serious complications of TACE are liver abscess or liver infarction and cholecystitis (about 2% of patients). However, a postembolization syndrome occurs in up to 90% of patients and consists of a combination of fever, nausea, vomiting and abdominal pain. It may last up to three days and justify to carefully follow the patients after TACE. Other serious complications may occur: nontarget embolization (i.e. inadvertent reflux of cytotoxic drugs and/or embolic material into unintended territories); liver failure; tumor rupture (<1% of cases) or biloma; and vessel injury. TACE is infrequently associated with bone marrow suppression and alopecia. With varying frequency, each intraarterial therapy is associated with arterial sclerosis, dissection and occlusion, which can render new procedure more challenging.

Transarterial radioembolization

Yttrium-90 (Y90) radioembolization is an emerging technique aiming to enrich the therapeutic arsenal for the treatment of liver tumors [START_REF] Pesapane | New concepts in embolotherapy of HCC[END_REF][START_REF] Venkatanarasimha | Radioembolisation of hepatocellular carcinoma: a primer[END_REF][START_REF] Study | EASL Clinical Practice Guidelines: Management of hepatocellular carcinoma[END_REF]. The indications include patients presenting with BCLC stage B or C with portal vein tumoral thrombosis HCC or progression after TACE. Patients must have ECOG status 0-2, and Child Pugh score A-B. In some studies, it has been also investigated for bridging patients to transplantation. Y90 is a pure β-emitter which is incorporated into glass or resin microspheres to be injected intraarterially [START_REF] Pesapane | New concepts in embolotherapy of HCC[END_REF]. Compared to TACE, radioembolization requires a meticulous planning before performing the treatment [START_REF] Venkatanarasimha | Radioembolisation of hepatocellular carcinoma: a primer[END_REF]. Therefore, a hepatic angiography is combined with technetium-99m labelles macro-aggregrated albumin scintigraphy 1-2 weeks prior to radioembolization to study the vascular supply of the tumor. The purpose is to quantify liver-to-lung shunt and detect shunts to the gastrointestinal tract which are contraindication of radioembolization. If present, these shunts must be occluded.

However, recent studies have not already demonstrated any benefits of radioembolization on overall survival compare to TACE, while it provides longer time to progression and better safety profile [START_REF] Pesapane | New concepts in embolotherapy of HCC[END_REF][START_REF] Venkatanarasimha | Radioembolisation of hepatocellular carcinoma: a primer[END_REF][START_REF] Study | EASL Clinical Practice Guidelines: Management of hepatocellular carcinoma[END_REF].

Percutaneous image-guided ablations

Percutaneous ablation of liver tumors can be achieved under imaging guidance (US, CT-scan, MRI or PET/CT) using different ablative techniques [START_REF] Solomon | Interventional Molecular Imaging[END_REF]. Chemical injection of alcool; heating by radiofrequency ablation (RFA) or microwave ablation (MWA); cooling using cryoablation; and disrupting cell membranes by irreversible electroporation or electrochemotherapy are all effective tumoricid procedures (17-20) (Figure 2). The resulting volume of ablation depends of the technology, needle shape and the amount of energy locally delivered. Other factors may affect strongly the volume and/or the effectiveness of the ablation, such as the properties of the local tumor environment (water content, cirrhotic versus non-cirrhotic parenchyma, all influencing the thermal conductivity), or the proximity to large vessels resulting in heat sink effect.

RFA is recognized as one of the primary treatment of BCLC stage A (single tumor any size or up to 3 nodules < 3cm with preserved liver function), and as an option to treat cholangiocarcinoma and metastases of colorectal cancer, neuroendocrine tumours, breast cancer or melanoma [START_REF] Covey | Liver-Directed Therapy for Hepatocellular Carcinoma: An Overview of Techniques, Outcomes, and Posttreatment Imaging Findings[END_REF]. In general, patients with unresectable disease involving less than 50% of the liver with well-compensated cirrhosis (Child-Pugh score A or B) may be candidates [START_REF] Idée | Use of Lipiodol as a drug-delivery system for transcatheter arterial chemoembolization of hepatocellular carcinoma: A review[END_REF]. Therefore, a technique has to be select to adequately treat the tumor targeted while avoiding complications based on the patients' and tumors' characteristics.

Currently, RFA, MWA and in a lesser extent cryoablation are the most frequently used techniques to treat liver tumor percutaneously in this setting. All these techniques present specific advantages and limits [START_REF] Vroomen | Irreversible electroporation and thermal ablation of tumors in the liver, lung, kidney and bone: What are the differences?[END_REF]. The resulting ablation zone differs substantially among the techniques in particular in the margins of ablation [START_REF] Cornelis | A Comparative Study of Ablation Boundary Sharpness After Percutaneous Radiofrequency, Cryo-, Microwave, and Irreversible Electroporation Ablation in Normal Swine Liver and Kidneys[END_REF]. During RFA or MWA, temperature rises up to 100 0 C, causing cell death by protein denaturation and cell membrane dysfunction. Cryoablation use the Joule-Thompson effect to decrease temperature to less than -40°C and cell death occurs mainly by crystal formation and osmotic shock. With all these techniques, an ablative margin of at least 5 mm is recommended, as it has been associated with the best local tumor control [START_REF] Kaye | Volumetric 3D assessment of ablation zones after thermal ablation of colorectal liver metastases to improve prediction of local tumor progression[END_REF].

While a single applicator may be used to achieve this objective, well-positioned overlapping ablations or multiple electrodes or probes activated simultaneously are used to increase the ablation volume and ensure the efficiency of ablation. Larger tumors (>3cm) and those with complex geometry or close contact with surrounding organs or sensitive tissue are more difficult to treat and justify the development of the combined strategies discussed later [START_REF] Higgins | Combining locoregional therapies in the treatment of hepatocellular carcinoma[END_REF].

The mortality rate is low up to 0.3% [START_REF] Livraghi | Complications of microwave ablation for liver tumors: results of a multicenter study[END_REF] whereas the rate of major complications is 2.2%. These complications justify further improving image-guidance or using protective techniques such as hydro or carbodissection [START_REF] Solomon | Interventional Molecular Imaging[END_REF]. During the last decade, three prospective randomized trials compared RFA and resection of solitary HCC. Chen et al. [START_REF] Chen | A prospective randomized trial comparing percutaneous local ablative therapy and partial hepatectomy for small hepatocellular carcinoma[END_REF] reported on 180 patients with solitary <5-cm HCC overall survival rates at 1, 2, 3, and 4 years of 96%, 82%, 71%, and 68% for ablation compared with 93%, 82%, 73%, and 64% for resection. In another randomized trial on 230 patients (26) reported overall 1-, 2-, 3-, 4-, and 5-year survival rates for the ablation group of 87%, 76%, 70%, 66%, and 55%, and 98%, 96%, 92%, 83%, and 76% for the surgical group. A meta-analysis showed no difference between resection and percutaneous ablation with respect to overall survival up to 4 years for HCC up to 3 cm [START_REF] Chen | Radiofrequency ablation versus surgical resection for intrahepatic hepatocellular carcinoma recurrence: a meta-analysis[END_REF]. Although RFA was firstly introduced 20 years ago and appeared effective in long term evaluations [START_REF] Li | Radiofrequency ablation combined with percutaneous ethanol injection for hepatocellular carcinoma: a systematic review and meta-analysis[END_REF], the advantage of MWA over RFA is to provide higher energy within the tumor and be less sensitive to heat sink effect [START_REF] Meloni | Microwave ablation in primary and secondary liver tumours: technical and clinical approaches[END_REF]. A recent single-center study of 75 patients treated with MWA showed primary technique effectiveness of 93.7% in tumors 4 cm or smaller, and 75% in tumors greater than 4 cm (30).

Combined therapies

Recent studies have demonstrated that patients with large, unresectable HCC experienced significantly longer survival with combination therapy using both ablation and embolization when compared with patients who had undergone ablation alone [START_REF] Meloni | Microwave ablation in primary and secondary liver tumours: technical and clinical approaches[END_REF].

Performing ablation with embolization optimizes outcomes by combining two different tumoricid method ensuring adequate tumor destruction. Arterial embolization and subsequent tumor constrast uptake may increase the efficiency of ablation by reducing heat sink but also by increasing the sensitivity of tumor cells to elevated temperatures, enhancing potentially the efficacy of the ablation [START_REF] Higgins | Combining locoregional therapies in the treatment of hepatocellular carcinoma[END_REF]. It also may help to improve tumor visualization by the retention of embolic material. Improved overall survival was observed in patients presenting with 3-to 5-cm tumors. In a study including 139 patients with recurrent HCC after resection [START_REF] Peng | Recurrent hepatocellular carcinoma treated with sequential transcatheter arterial chemoembolization and RF ablation versus RF ablation alone: a prospective randomized trial[END_REF], the 1-, 3-, and 5-year overall survival rates were 94%, 69%, and 46% for the combination therapy group (RFA +TACE) and 82%, 47%, and 36% for the RFA alone group. In another study on 189 HCCs [START_REF] Peng | Radiofrequency Ablation With or Without Transcatheter Arterial Chemoembolization in the Treatment of Hepatocellular Carcinoma: A Prospective Randomized Trial[END_REF], overall survival and progression free survival rates to be signicantly better in the embolization-ablation group than in the group who underwent RFA alone.

Interventional radiologic procedures may be also used in combination with surgery to enlarge the curative options [START_REF] Pesapane | New concepts in embolotherapy of HCC[END_REF]. To be considered as a suitable surgical candidate, a patient presenting with HCC must have enough functional liver remaining after the operation. Portal vein embolization is a procedure that induces regrowth on one side of the liver before a planned hepatic resection on the other side (Figure 1). After selective cathererism of the portal vein, embolization is performed using various embolic agents such as glue, microspheres, coils or plugs. This blockade of the portal blood supply induces the other side of the liver to regrow and after several weeks, the non-embolized side has often grown enough so that surgery is a viable option. In case of bilateral tumors, the future remnant liver may also be preconditioned by resecting or ablating the tumors on this side.

Conclusion

Interventional Oncologic procedures performed in a daily practice such as embolization and percutaneous ablations allow to treat efficiently primary liver tumors under minimally invasive conditions. It may result in more rapid recovery, fewer complications, shorter hospital stays and reduced medical costs. Selecting the appropriate therapeutic option for the individual patient is mandatory justifying further clinical evaluations.

Figures

Figure 1 :

 1 Figure 1: Transarterial chemoembolization (TACE) of a 80mm BCLC stage B hepatocellular

Figure 2 :

 2 Figure 2: Ultrasound guided liver microwave ablation (MWA) of a 20mm BCLC stage A

Conflict of interests: None

F-No enhancement was observed after contrast injection on the axial contrast enhanced T1weighted MR image.


Quentin Ferro
email: quentin.ferro@lip6.fr

Stef Graillat
email: stef.graillat@lip6.fr

Thibault Hilaire
email: thibault.hilaire@lip6.fr

Fabienne Jézéquel
email: fabienne.jezequel@lip6.fr

Basile Lewandowski

Neural Network Precision Tuning Using Stochastic Arithmetic

Keywords: Precision, Neural Networks, Auto-Tuning, Floating-Point, Stochastic Arithmetic

Neural networks can be costly in terms of memory and execution time. Reducing their cost has become an objective, especially when integrated in an embedded system with limited resources. A possible solution consists in reducing the precision of their neurons parameters. In this article, we present how to use auto-tuning on neural networks to lower their precision while keeping an accurate output. To do so, we use a floating-point auto-tuning tool on different kinds of neural networks. We show that, to some extent, we can lower the precision of several neural network parameters without compromising the accuracy requirement.

Introduction

Neural networks are nowadays massively used and are becoming larger and larger. They often need a lot of resources, which can be a problem, especially when used in a critical embedded system with limited computing power and memory. Therefore it can be very beneficial to optimise the numerical formats used in a neural network. This article describes how to perform precision autotuning of neural networks. From a neural network application and an accuracy requirement on its results, it is shown how to obtain a mixed precision version using the PROMISE tool [START_REF] Graillat | Auto-Tuning for Floating-Point Precision with Discrete Stochastic Arithmetic[END_REF]. A particularity of PROMISE is the fact that it uses stochastic arithmetic [START_REF]Discrete Stochastic Arithmetic for Validating Results of Numerical Software[END_REF] to control rounding errors in the programs it provides.

Minimizing the format of variables in a numerical simulation can offer advantages in terms of execution time, volume of data exchanged and energy consumption. During the past years several algorithms and tools have been proposed for precision auto-tuning. On the one hand, tools such as FPTuner [START_REF] Chiang | Rigorous Floating-Point Mixed-Precision Tuning[END_REF], Salsa [START_REF] Damouche | Mixed Precision Tuning with Salsa[END_REF], Rosa/Daisy [START_REF] Darulova | Towards a Compiler for Reals[END_REF][START_REF] Darulova | Daisy -Framework for Analysis and Optimization of Numerical Programs[END_REF], TAFFO [START_REF] Cherubin | TAFFO: Tuning Assistant for Floating to Fixed Point Optimization[END_REF], POP [START_REF] Adjé | Fast and Efficient Bit-Level Precision Tuning[END_REF] rely on a static approach and are not intended to be used on very large code. On the other hand, dynamic tools such as CRAFT HPC [24], Precimonious [START_REF] Rubio-González | Precimonious: Tuning Assistant for Floating-Point Precision[END_REF], HiFPTuner [START_REF] Guo | Exploiting Community Structure for Floating-Point Precision Tuning[END_REF], ADAPT [START_REF] Menon | ADAPT: Algorithmic Differentiation Applied to Floating-Point Precision Tuning[END_REF],

FloatSmith [START_REF] Lam | Tool Integration for Source-Level Mixed Precision[END_REF], PROMISE [START_REF] Graillat | Auto-Tuning for Floating-Point Precision with Discrete Stochastic Arithmetic[END_REF], have been proposed for precision auto-tuning in large HPC code. Moreover, tools have been recently developed for precision auto-tuning on GPUs: AMPT-GA [START_REF] Kotipalli | AMPT-GA: Automatic Mixed Precision Floating Point Tuning for GPU Applications[END_REF], GPUMixer [START_REF] Laguna | GPUMixer: Performance-Driven Floating-Point Tuning for GPU Scientific Applications[END_REF], GRAM [START_REF] Ho | GRAM: A Framework for Dynamically Mixing Precisions in GPU Applications[END_REF]. A specificity of PROMISE lies in the fact that it provides mixed precision programs validated owing to stochastic arithmetic [START_REF]Discrete Stochastic Arithmetic for Validating Results of Numerical Software[END_REF], whereas other dynamic tools rely on a reference result possibly affected by rounding errors. PROMISE has been used in various applications based on linear algebra kernels, but not yet for precision auto-tuning in deep neural networks.

While much effort has been devoted to the safety and robustness of deep learning code (see for instance [START_REF] Dutta | Output Range Analysis for Deep Feedforward Neural Networks[END_REF][START_REF] Singh | Fast and Effective Robustness Certification[END_REF][START_REF] Madry | Towards Deep Learning Models Resistant to Adversarial Attacks, 6th International Conference on Learning Representations[END_REF][START_REF] Lin | Robustness Verification of Classification Deep Neural Networks via Linear Programming[END_REF][START_REF] Rakin | RA-BNN: Constructing Robust & Accurate Binary Neural Network to Simultaneously Defend Adversarial Bit-Flip Attack and Improve Accuracy[END_REF]) a few studies have been carried out on the effects of rounding error propagation on neural networks. Verifiers such as MIPVerify [START_REF] Tjeng | Evaluating Robustness of Neural Networks with Mixed Integer Programming[END_REF] are designed to check properties of neural networks and measure their robustness. However, the impact of floating-point arithmetic both on neural networks and on verifiers is pointed out in [START_REF]Fooling a Complete Neural Network Verifier[END_REF]. Because of rounding errors, the actual robustness and the robustness provided by a verifier may radically differ. In [START_REF] Csendes | Adversarial Example Free Zones for Specific Inputs and Neural Networks[END_REF][START_REF] Zombori | Verification of Artificial Neural Networks via MIPVerify and SCIP[END_REF] it is shown how to control the robustness of different neural networks with greater efficiency using interval arithmetic based algorithms. In [START_REF] Lauter | A Framework for Semi-Automatic Precision and Accuracy Analysis for Fast and Rigorous Deep Learning[END_REF] a software framework is presented for semi-automatic floating-point error analysis in the inference phase of deep neural networks. This analysis provides absolute and relative error bounds owing to interval and affine arithmetics.

Precision tuning of neural networks using fixed-point arithmetic has been studied in [START_REF] Benmaghnia | Fixed-Point Code Synthesis For Neural Networks[END_REF]. Owing to the solution of a system of linear contraints, the fixedpoint precision of each neuron is determined, taking into account a certain error threshold.

In this article, we consider floating-point precision tuning, which is also studied in [START_REF] Ioualalen | Neural Network Precision Tuning[END_REF]. Focusing on interpolator networks, i.e. networks computing mathematical functions, the authors propose an algorithm that takes into account a given tolerance δ on the relative error between the assumed correctly computed function and the function computed by the network. The main difference with the present article is the auto-tuning algorithm: in [START_REF] Ioualalen | Neural Network Precision Tuning[END_REF] the precision is optimized by solving a linear programming problem, while PROMISE uses a hypothesistrial-result approach through the Delta-Debug algorithm [START_REF] Zeller | Simplifying and Isolating Failure-Inducing Input[END_REF]. Furthermore, the algorithm in [START_REF] Ioualalen | Neural Network Precision Tuning[END_REF] relies on a reference result that may be altered by rounding errors, while PROMISE uses stochastic arithmetic for the numerical validation of its results.

Stochastic arithmetic uses for rounding error estimation a random rounding mode: the result of each arithmetic operation is rounded up or down with the same probability. As a remark, another stochastic rounding often used in neural network training and inference uses a probability that depends on the position of the exact result with respect to the rounded ones (see for instance [START_REF] Gupta | Deep Learning with Limited Numerical Precision[END_REF][START_REF] Na | On-Chip Training of Recurrent Neural Networks with Limited Numerical Precision[END_REF][START_REF] Wang | Training Deep Neural Networks with 8-bit Floating Point Numbers[END_REF][START_REF] Essam | Dynamic Point Stochastic Rounding Algorithm for Limited Precision Arithmetic in Deep Belief Network Training[END_REF][START_REF] Mellempudi | Mixed Precision Training with 8-bit Floating Point[END_REF][START_REF] Su | Towards High Performance Low Bitwidth Training for Deep Neural Networks[END_REF]). This stochastic rounding does not aim at estimating rounding errors, it enables the update of small parameters and avoids stagnations that may be observed with round to nearest.

In this work, we consider tuning the precision of an already trained neural network. One of our contributions is a methodology for tuning the precision of a neural network using PROMISE in order to obtain the lowest precision for each of its parameters, while keeping a certain accuracy on its results. We present and compare the results obtained for different neural networks: an approximation of the sine function, an image classifier processing the MNIST dataset (2D pictures of handwritten digits), another image classifier using this time convolutional layers and processing the CIFAR10 dataset (3D images of different classes), and the last one introduced in [START_REF] Chang | Neural Lyapunov Control, 33rd Conference on Neural Information Processing Systems (NeurIPS 2019)[END_REF] and used in [START_REF] Lauter | A Framework for Semi-Automatic Precision and Accuracy Analysis for Fast and Rigorous Deep Learning[END_REF] that aims at approximating a Lyapunov function of a nonlinear controller for an inverted pendulum.

After a preliminary reminder on deep neural networks and stochastic arithmetic in Section 2, Section 3 describes our methodology and Section 4 presents our results considering the different neural networks previously mentioned.

Preliminary

Neural Networks

An artificial neural network is a computing system defined by several neurons distributed on different layers. Generally, we consider dense layers that take as an input a vector and in which the main computation is a matrix-vector product. In this case, from one layer to another, a vector of neurons x (k) ∈ R n k with k ∈ N is transformed into a vector x (k+1) ∈ R n k+1 by the following equation

x (k+1) = g (k) (W (k) x (k) + b (k) ) (1)
where W (k) ∈ R n k+1 ×n k is a weight matrix, b (k) ∈ R n k+1 a bias vector and g (k) an activation function. The activation function is a non-linear and often monotonous function. Most common activation functions are described below.

-Sigmoid: computes σ(x) = 1/(1 + e -x ) ∀x ∈ R -Hyperbolic Tangent: applies tanh(x) ∀x ∈ R -Rectified Linear Unit: ReLU(x) = max(x, 0) ∀x ∈ R -Softmax: normalizes an input vector x into a probability distribution over the output classes. For each element x i in x, softmax(x i ) = e xi / e xj Figure 1 shows a graphical representation of a neural network with two layers. Dense layers are sometimes generalized to multidimensional arrays and involve tensor products. Other layers exist such as convolution layers often used on multi-dimensional arrays to extract features out of the data. To do so, a convolution kernel is applied to the input data to produce the output. Different kinds of layers, for instance pooling layers and flatten layers, do not require weight nor bias. Pooling layers reduce the size of the input data by summarizing it by zones given a function such as the maximum or the average. Flatten layers are intended to change the shape of data, from a 3D tensor to a vector for example and can be used to pass from a convolutional layer to a dense layer.

Floating-Point Arithmetic

A floating-point number x in base β is defined by :

x = (-1) s × m × β e (2)
with s its sign being either 1 or 0, m its significand being an integer and e its exponent being also an integer. In this paper, we consider binary floating-point numbers, i.e. numbers in base β=2 that adhere to the IEEE 754 Standard [START_REF]IEEE Standard for Floating-Point Arithmetic, IEEE Std[END_REF].

The IEEE 754 Standard defines different formats with a fixed number of bits for the significand and the exponent. The number of bits for the significand is the precision p, hence the significand can take values ranging from 0 to β p -1. The exponent e ranges from e min to e max with e min = 1-e max and e max = 2 len(e)-1 -1, with len(e) the exponent length in bits. The sizes of the three different formats used in this paper, commonly named half, single (or float), and double, are sum up in Table 1. As a remark, another 16-bit format called bfloat16 exists, for example on ARM NEON CPUs. Owing to their 8-bit-large exponent, bfloat16 numbers benefit from a wider range, but have a lower significand precision (8 bits instead of 11). Discrete Stochastic Arithmetic (DSA) is a method for rounding error analysis based on the CESTAC method [START_REF] Vignes | A Stochastic Arithmetic for Reliable Scientific Computation[END_REF][START_REF] Chesneaux | Numerical Validation and Assessment of Numerical Accuracy[END_REF]. The CESTAC method allows the estima-tion of round-off error propagation that occurs when computing with floatingpoint numbers. Based on a probabilistic approach, it uses a random rounding mode: at each operation, the result is either rounded up or down with the same probability. Using this rounding mode, the same program is run N times giving us N samples R 1 , . . . , R N of the computed result R. The accuracy of the computed result (i.e. its number of exact significant digits) can be estimated using Student's law with the confidence level 95%. In practice the sample size is N = 3. Indeed, it has been shown that N = 3 is in some reasonable sense the optimal value. The estimation with N = 3 is more reliable than with N = 2 and increasing the sample size does not improve the quality of the estimation. Theoretical elements can be found in [START_REF] Chesneaux | L'arithmétique stochastique et le logiciel CADNA[END_REF][START_REF] Vignes | A Stochastic Arithmetic for Reliable Scientific Computation[END_REF].

The CADNA [START_REF] Jézéquel | Numerical Validation of Half Precision Simulations[END_REF][START_REF] Vignes | A Stochastic Arithmetic for Reliable Scientific Computation[END_REF][START_REF] Chesneaux | L'arithmétique stochastique et le logiciel CADNA[END_REF] (Control of Accuracy and Debugging for Numerical Applications) software 4 implements DSA in code written in C, C++ or Fortran. It introduces new variable types, the stochastic types. Each stochastic variable contains three floating-point values and one integer being the exact number of correct digits. CADNA can print each computed value with only its exact significant digits. In practice, owing to operator overloading, the use of CADNA only requires to change declaration of variables and input/output statements.

The PROMISE Software

The PROMISE software5 aims at reducing the precision of the variables in a given program. From an initial code and a required accuracy, it returns a mixed precision code, lowering the precision of the different variables while keeping a result that satisfies the accuracy constraint. To do so, some variables are declared as custom typed variables that PROMISE recognizes. PROMISE will consider tweaking their precisions. Different variables can be forced to have the same precision by giving them the same custom type. It may be useful to avoid compilation errors or casts of variables.

PROMISE computes a reference result using CADNA and relies on the Delta-Debug algorithm [START_REF] Zeller | Simplifying and Isolating Failure-Inducing Input[END_REF] to test different type configurations, until a suitable one lowering the precision while satisfying the accuracy requirement is found. PROMISE provides a transformed program that can mix half, single and double precision. Half precision can be either native on CPUs that support it or emulated using a library developed by C. Rau6 . PROMISE dataflow is presented in Figure 2. After computing the reference result, PROMISE tries to lower the precision of the variables from double to single, then from single to half, using twice the Delta-Debug algorithm. The accuracy requirement may concern one or several variables (e.g. in an array). PROMISE checks that the number of common digits between the computed result(s) and the reference result(s) is at least the required accuracy. In the case of several variables, the requirement has to be fulfilled by all of them. Both of them also allow us to save our model in HDF5 (Hierarchical Data Format) 9 , a file format designed to store and organize large data. HDF5 uses only two types of objects: datasets that are multidimensional arrays of homogeneous type, and groups, which contain datasets or other groups. HDF5 files can be read by Python programs using the h5py package. The associated data can be manipulated with Pandas10 , a Python library that proposes data structures and operations to manage large amount of data. Keras is used to develop, train and save our neural network models, except in the case of the inverted pendulum which uses PyTorch. As already mentioned in the introduction, precision tuning is performed on trained models, hence in the inference stage. The process path is summarized in Figure 3. For each neural network, we first convert the HDF file to CSV files using a Python script. The script loads the HDF file, stores the parameters in Pandas DataFrames, and then saves the parameters in CSV files using the Pandas DataFrame to csv function. For each layer that needs it, we create a CSV file with the weights of the layer and a CSV file with the bias of the layer. Indeed, some layers do not need weights nor bias, for example flattening layers that only change the data shape (from 2 dimensions to 1 dimension for example). Secondly, we use the data in the CSV files to create a C++ program, once again using a Python script that reads the CSV files and creates the necessary variables and computation. The translation scripts are based on the work done in the keras2c11 library. Once the C++ version created, we apply PROMISE on it.

Experimental Results

Results obtained for four different neural networks are presented in this section. For the neural networks using a database, test data[i] refers to (i+1) th test input provided by the database. PROMISE is applied to each neural network considering one type by neuron (half, single or double), then one type per layer, i.e. all the parameters of a layer have the same precision. In our analysis, the difference between the two approaches lies in the number of different type declarations in the code. However, it must be pointed out that, in dense layers, having one type per neuron implies independent dot products, whereas having one type per layer would enable one to compute matrix-vector products that could perform better.

In our experiments, the input is in double precision. In accordance with Figure 2, for any neural network, the reference value is the value computed at the very first step of PROMISE. All the results presented in this section have been obtained on a 2.80 GHz Intel Core i5-8400 CPU having 6 cores with 16GB RAM except indicated otherwise.

Sine Neural Network

To approximate the sine function, we use a classical densely-connected neural network with 3 layers. It is a toy problem, since using a neural network to compute sine is not necessary. However, this simple example validates our approach. The tanh activation function is used in the 3 different dense layers. The layers have respectively 20, 6 and 1 neuron(s) and the input is a scalar value x. Figure 4 presents the computation carried out by the neural network, considering one type per neuron. Colored variables are PROMISE variables, the precision of which can be tweaked. Variables with the same color have the same precision. The parameters of a neuron (weight(s) and bias) have the same color, hence the same type. The output type of each layer is also tuned. In this example, we assign types to 27 neurons and 3 outputs.

... w

(2) 6,20

x

...

x (2) 20 b (2) 1 b (2) 2
...

b

(2) 6

x

...

x (3) 6 tanh × + = Input L2 Weights Bias Output L2 w (3) 1,1
... w

... Figure 5 displays the distribution of the different types with input value 0.5 considering one type per neuron. The x-axis presents the required accuracy on the results, i.e. the number of significant digts in common with the reference result computed using CADNA. We can notice the evolution of the distribution depending on the number of exact significant digits required on the result. As expected, first we only have half precision variables, then some of them start to be in single, then in double precision, until eventually all of them are in double precision. Therefore, requiring the highest accuracy is not compatible with lowering the precision in this neural network. But still, a good compromise can be found, since we only have single and half precision variables for a required accuracy up to 7 digits, and still have 1/3 of single precision variables for a required accuracy up to 9 digits.

Figure 5 also presents the computation time of PROMISE in seconds for each required accuracy. It consists of the time to compute the reference result, and the time to apply the Delta-Debug algorithm twice (from double to single precision then from single to half precision), compiling and executing the tested distribution each time. It can be noticed that the computation time (less than 2 minutes) remains reasonable given the 3 30 possibilities. Figure 6 shows the type distribution considering one precision per layer. This approach per layer enables one to decrease the execution time of PROMISE, but it does not really help lowering the precision of the network parameters. Indeed, each time a parameter in a layer requires a higher precision, all the parameters of the same layer pass in higher precision. But still, it can be noticed that the first layer (that represents 2/3 of the neurons) stays in half precision for a required accuracy up to 4 digits. show that the input value can have a slight impact on the type distribution. We compare the results for two input values randomly chosen: 0.5 and 2.37. From Figure 5, with input value 0.5, PROMISE provides a type distribution with 8 half precision neurons and 22 single precision neurons for a required accuracy of 5 digits. From Appendix A.1, with input value 2.37, only 4 half precision neurons are obtained for a required accuracy of 5 digits. Actually, if 3, 4, or 5 digits are required, less neurons are in half precision than with input 0.5. But if 8 digits are required, one neuron remains in half precision with input 2.37, while no neuron is in half precision with input 0.5. Nevertheless, the type distribution with respect to the required accuracy remains globally the same, parameters all starting as half precision variables, then passing to higher precision.

MNIST Neural Network

Experiments have been carried out with an image classification neural network processing MNIST data of handwritten digits 12 . This neural network also uses classical dense layers. The main difference in this case is that the entry is a vector of size 784 (flatten image) and the output is a vector of size 10. This neural network consists of two layers: the first one with 64 neurons and the activation function ReLU, and the second one with 10 neurons and the activation function softmax which provides the probability distribution for the 10 different classes. Considering as previously one type per neuron, plus one type for the output of each layer, 76 different types have to be set either to half, single, or double precision.

Figure 7 shows the type distribution considering one type per neuron with the input image test data[61]: the 62 nd test data out of the 10,000 provided by MNIST. The x-axis represents the required accuracy on the output consisting in a vector of size 10. The maximum accuracy on the output is 13 digits, higher expectation could not be matched. Such high accuracy is nonetheless not realistic because not necessary for a classifier. However, exhaustive tests have been performed: all possible accuracies in double precision have been successively required.

The main difference with the sine neural network lies in the fact that a significant number of neurons stay in half precision no matters the required accuracy. Depending on the input, around 50% of neurons can stay in half precision and sometimes nearly 60% as shown in Appendix B.2. Thus, applying PROMISE to this neural network, even when requiring the highest accuracy, can help lowering the precision of its parameters. Neurons that keep the lowest precision are not the same depending on the input, but they always belong to the first layer. Hence, the first layer seems to have less impact on the output accuracy than the second one.

The computation times also reported in Figure 7 are much higher than for the sine approximation network. For the majority of the accuracy requirements, more than 15 minutes are necessary to obtain a mixed precision version of the neural network. The MNIST neural network has one layer less than the sine approximation network, but more neurons. Since we consider one precision per neuron, the number of possible type configurations (3 76 ) is much higher, hence the computation time difference. However, the execution time of PROMISE remains reasonable and performing such a tuning by hand would have been much more time consuming.

With both the sine neural network and MNIST neural network, PROMISE execution time tends to increase with the accuracy requirement. This can be explained by the Delta-Debug algorithm in PROMISE. As previously described, PROMISE firstly checks whether the accuracy requirement can be satisfied with double precision. Then, owing to the Delta-Debug algorithm, PROMISE tries to lower the precision of most variables from double to single, and this can be very fast if single precision is enough to match the required accuracy. Finally, PROMISE tries to transform the single precision declarations into half precision ones, and this can be fast if half precision is suitable for all these declarations. The number of programs compiled and executed by PROMISE tends to increase with the required accuracy on the results. For instance, in the case of MNIST neural network, if 1 or 2 digits are required, 18 type configurations are tested by PROMISE, whereas if 7 digits are required, 260 configurations are tested.

Results obtained considering one precision per layer are presented in Appendix B.1. The analysis is similar to the one previously given for the sine neural network. With the approach per layer, the execution time of PROMISE is lower than with the approach per neuron. But this approach forces some variables to be declared in higher precision. It can be noticed that with the approach per layer, both layers have the same precision. All the network parameters share the

CIFAR Neural Network

The neural network considered here is also an image classifier, but this time processing the CIFAR10 dataset. CIFAR 13 is a dataset having 100 classes of colored images and the CIFAR10 dataset is reduced to 10 classes. Because images are of size 32 × 32 × 3 (32 pixels width, 32 pixels height, 3 color channels), the network input is a 3D tensor of shape [START_REF] Rakin | RA-BNN: Constructing Robust & Accurate Binary Neural Network to Simultaneously Defend Adversarial Bit-Flip Attack and Improve Accuracy[END_REF][START_REF] Rakin | RA-BNN: Constructing Robust & Accurate Binary Neural Network to Simultaneously Defend Adversarial Bit-Flip Attack and Improve Accuracy[END_REF][START_REF] Benmaghnia | Fixed-Point Code Synthesis For Neural Networks[END_REF]. The neural network consists of 5 layers: a convolutional layer having 32 neurons with activation function ReLU, followed by a max pooling of size 2x2, a convolutional layer having 64 neurons with activation function ReLU, a flatten layer, and finally a dense layer of 10 neurons with activation function softmax. Taking into account one type per neuron and one type for each layer output, 111 types can be set.

Results presented here have been obtained on a 2.80 GHz Intel Core i9-10900 CPU having 20 cores and 64GB RAM. The maximum possible accuracy on the results is 13 digits. Although such a high accuracy is not necessary in a classifier network, exhaustive tests have been performed, like for the MNIST neural network. Results reported here refer to two input images out of the 10,000 provided by CIFAR10. Figure 8 As previously observed, considering one type per layer results in lower PROMISE computation times, but often in uniform precision programs. Again, the input image slightly impacts the type configurations provided by PROMISE and the same trend can be observed.

Experiments have been carried out with neural networks also processing CI-FAR10, but with more layers (up to 8 layers). Again, PROMISE could provide suitable type configurations taking into account accuracy requirements. However, PROMISE execution (that includes the compilation and execution of various programs) makes exhaustive tests more difficult with such neural networks. Possible PROMISE improvements described in Section 5 would enable precision tuning in larger neural networks that are themselves time consuming.

Inverted Pendulum

We present here results obtained with a neural network introduced in [START_REF] Chang | Neural Lyapunov Control, 33rd Conference on Neural Information Processing Systems (NeurIPS 2019)[END_REF] in the context of reinforcement learning for autonomous control. In [START_REF] Chang | Neural Lyapunov Control, 33rd Conference on Neural Information Processing Systems (NeurIPS 2019)[END_REF] methods are proposed for certified approximation of neural controllers and this neural network, related to an inverted pendulum, is used in a program that provides an approximation of a Lyapunov function. This neural network consists of 2 dense layers and uses the tanh activation function. The input is a state vector x ∈ R 2 and the output, a scalar value in R is an approximated value of the Lyapunov function for the state vector input. The first layer has 6 neurons and the second layer only one. Given the proximity between the two neural network models, results are expected to be close to the ones obtained for the sine approximation.

Figure 9 presents both the distribution of the different precisions and the execution time of PROMISE with respect to the accuracy requirement for input (0.5, 0.5). We consider here one type per neuron. As expected, the trend observed for the type configurations is the same as with the sine approximation. As the required accuracy increases, the precision of the network parameters also increases. If one digit is required, all the parameters can be declared in half precision, and if at least 11 digits are required all the parameters must be in double precision. The computation time remains reasonable whatever the required accuracy. As previously observed, the computation time tends to increase with the required accuracy because of the number of type configurations tested by PROMISE.

  x

2 Fig. 1 :

 21 Fig. 1: Neural network with two layers

Fig. 2 :

 2 Fig. 2: PROMISE dataflow

Fig. 3 :

 3 Fig. 3: Flowchart of the translation from a Python neural network to a C++ program with PROMISE variables

Fig. 4 :

 4 Fig. 4: Computation carried out in the sine neural network

Fig. 5 :

 5 Fig. 5: Number of variables of each type and computation time for the sine neural network with input value 0.5

Fig. 6 :

 6 Fig. 6: Precision of each layer for the sine neural network with input value 0.5

Fig. 7 :

 7 Fig. 7: Number of variables of each type and computation time for MNIST neural network with test data[61] input

  and Appendix C.2 present the type configurations given by PROMISE with respectively test data[386] and test data[731], considering one type per neuron. Appendix C.1 and Appendix C.3 show the results obtained with the same input images considering one type per layer.

Fig. 8 :

 8 Fig. 8: Number of variables of each type and computation time for CIFAR neural network with test data[386] input

Fig. 9 :Fig. 13 :

 913 Fig. 9: Number of variables of each type and computation time for the pendulum neural network with input (0.5,0.5)

Table 1 :

 1 Basic binary IEEE 754 formats 2.3 Discrete Stochastic Arithmetic (DSA)

	Name	Format	Length	Sign	Significand length 3	Exponent length
	Half	binary16	16 bits	1 bit	11 bits	5 bits
	Single	binary32	32 bits	1 bit	24 bits	8 bits
	Double	binary64	64 bits	1 bit	53 bits	11 bits

Including the implicit bit (which always equals 1 for normal numbers, and 0 for subnormal numbers). The implicit bit is not stored in memory.

http://cadna.lip6.fr

http://promise.lip6.fr

http://half.sourceforge.net

https://keras.io

https://pytorch.org

https://www.hdfgroup.org

https://pandas.pydata.org

https://f0uriest.github.io/keras2c/

http://yann.lecun.com/exdb/mnist

Acknowledgements

This work was supported by the InterFLOP (ANR-20-CE46-0009) project of the French National Agency for Research (ANR).

Appendix D.1 presents the results obtained considering one type per layer. Again, with this approach, the execution time of PROMISE is lower, but most configurations are actually in uniform precision. Results in Appendix D.2 and Appendix D.3 refer to an input consisting of two negative values (-3,-6). As previously observed, changing the input induces no significant difference.

Conclusion and Perspectives

We have shown with different kinds of neural networks having different types of layers how to lower the precision of their parameters while still satisfying a desired accuracy. Considering one type per neuron, mixed precision programs can be provided by PROMISE. Considering one type per layer enables one to reduce PROMISE execution time, however this approach often leads to uniform precision programs. It has been observed that with both approaches input values have actually a low impact on the type configurations obtained.

We plan to analyse the execution time of the mixed precision programs obtained with PROMISE on a processor with native half precision. Other perspectives consist in improving PROMISE. Another accuracy test more adapted to image classification networks could be proposed. We could improve the Delta-Debug algorithm used in PROMISE. Optimizations of the Delta-Debug algorithm are described in [START_REF] Hodován | Practical Improvements to the Minimizing Delta Debugging Algorithm[END_REF], including the parallelization potential. We could also consider the parallelization of PROMISE itself, i.e. applying PROMISE to different parts of a code in parallel. The extension of PROMISE and CADNA to other floating-point formats such as bfloat16 is another perspective. Taking benefit from the GPU version of CADNA, PROMISE could also be extended to GPUs. Floating-point auto-tuning in arbitrary precision is also a possible perspective that would enable the automatic generation of programs with a suitable type configuration for architectures such as FPGAs.