Flavien Lucas
email: flavien.lucas@univ-ubs@imt-atlantique.fr

Romain Billot
email: romain.billot@imt-atlantique.fr

Marc Sevaux
email: marc.sevaux@univ-ubs.fr

A comment on "what makes a VRP solution good? The generation of problem-specific knowledge for heuristics"

Keywords: Machine Learning, Optimization, Vehicle routing, Data mining, Principal Component Analysis, Random Forest

We propose a comment about the article "What makes a VRP solution good? The generation of problem-specific knowledge for heuristics" [1] by Florian Arnold and Kenneth Sörensen. In the original contribution, the authors implemented several Machine Learning (ML) algorithms in order to predict good vs. not good solutions. Then, some outcomes of the algorithms were discussed in terms of the predictive power of the solutions features. The purpose was then to use the extracted knowledge to improve existing heuristics. The first contribution of our comment is to validate and complement some of the conclusions of the authors. Then, we argue than most of the extracted knowledge can be retrieved by classical data reduction methods such as Principal Component Analysis (PCA). Hence, instead of ML-based predictions, a factorial analysis provides a powerful and synthetic view of the variables inter-dependencies in the light of solution quality. Thanks to the datasets provided by the authors in the original article, new experimental results are conducted. Finally, the integration of these results into future "boosted" heuristics is discussed.

Introduction

With the recent advances in machine learning, the number of articles using data mining for operations research has increased a lot over the last five years. However, combining machine learning and operations research is not new. It has been a goal since the populationbased algorithms (genetic algorithm [START_REF] Davis | Handbook of genetic algorithms[END_REF], ant colony [START_REF] Dorigo | Ant Colony Optimization and Swarm Intelligence: 6th International Conference, ANTS 2008[END_REF], etc) keeping the best characteristics of a set of solutions. More recently, some specific methods have proposed a two-step methodology: instances are first analyzed and then solved in a second step [START_REF] Barbalho | A hybrid data mining GRASP with path-relinking[END_REF][START_REF] Martins | Data mining in stochastic local search[END_REF]. Moreover, some other attempts use machine learning to find good parameters for metaheuristics algorithms [START_REF] Birattari | The problem of tuning metaheuristics as seen from a machine learning perspective[END_REF]. Another way to proceed is to conduct statistical studies on the effectiveness of certain heuristics and metaheuristics. These results are used to select which heuristic to execute, depending on the instance to solve [START_REF] Solano-Charris | Local search based metaheuristics for the robust vehicle routing problem with discrete scenarios[END_REF], or the solution to improve [START_REF] Veerapen | Pareto autonomous local search[END_REF]. The shared target of this wide range of methods is to directly improve the quality of existing methods. A survey of methods mixing machine learning and combinatorial optimization was recently published [START_REF] Bengio | Machine learning for combinatorial optimization: a methodological tour d'horizon[END_REF]. In the sequel, we will see how the paper of Arnold and Sörensen had opened a new perspective using the explanatory power of these machine learning methods, while a classical factorial approach (PCA) is another way to tackle similarly this challenge.

Summary of the original paper

K. Sörensen and F. Arnold studied some machine learning methods in order to improve the resolution of a combinatorial optimization problem. In this section we will summarize the problem addressed, the initial goal of their paper and the methodology.

Brief problem description

The Vehicle Routing Problem consists in delivering customers' demand with a fleet of vehicles. All routes start and end at the depot. All vehicles are identical and limited by their capacity. Each customer must be visited and served at once. This problem is N P-hard, explaining why only small instances (less than 200 customers) can be solved in reasonable time. The most studied algorithms are now approximation methods, such as Multiple Neighborhood Search (MNS) [START_REF] Soto | Multiple neighborhood search, tabu search and ejection chains for the multi-depot open vehicle routing problem[END_REF], Iterated Local Search (ILS) [START_REF] Subramanian | A hybrid algorithm for a class of vehicle routing problems[END_REF] and Unified Hybrid Genetic Search (UHGS) [START_REF] Vidal | A unified solution framework for multi-attribute vehicle routing problems[END_REF].

Goal and methodology of the article

In their paper, K. Sörensen and F. Arnold tried to discover the main features characterizing near-optimal and non-optimal solutions. This knowledge is helpful for guiding heuristics and thus improving the problem resolution. Table 1 shows the list of solutions features used. However, two solutions with same features values but from two different instances may potentially have different qualities. In order to verify this hypothesis, the authors have tested 8 additional features, depending on the instances. These features are listed in Table 2. More details are available in the original paper [START_REF] Arnold | What makes a VRP solution good? The generation of problem-specific knowledge for heuristics[END_REF]. S1 Average number of intersections per customer S2 Longest distance between two connected customers, per route S3 Average distance between depot to directly-connected customers S4 Average distance between routes (their centers of gravity) S5 Average width per route S6 Average span per route S7 Average compactness per route, by width S8 Average compactness per route, by radian S9 Average depth per route S10 Standard deviation of the number of customers per route

On the explanatory power of machine learning approaches

In [START_REF] Arnold | What makes a VRP solution good? The generation of problem-specific knowledge for heuristics[END_REF], F. Arnold and K. Sörensen exploit the 18 features previously described to predict the quality (near-optimal or non-optimal) of a solution. We have executed some experiments to check their results and develop further the relationship between features and quality of solutions. All figures have been generated from the same set of instances and solutions (large number of customers, high variance, depot at the center, and objective values between near-optimal and non-optimal solutions near 2%). It correspond to the file "gap2 large center highVariance.csv". Nevertheless, variations between sets of instances will also be discussed throughout this comment.

How to distinguish useful versus useless features?

The decision tree (DT) is a powerful method for understanding the contribution of each feature to the outcome of a targeted variable. Each leaf represents a fraction of the features space and contains a proportion of near-optimal and non-optimal solutions. Thus, the DT shows the areas where a majority of near-optimal, or non-optimal solutions are placed. Each leaf contains three types of information, the majority category (0 = majority of non-optimal solutions, 1 = majority of near-optimal solutions), the proportion of non-optimal solutions and the proportion of near-optimal solutions. For instance, the first leaf on Figure 1 contains a majority of non optimal solutions. More precisely, 80% of solutions contained in this leaf are non-optimal solutions and 20% are near optimal.

Decision trees allow us to understand the links between the features and the quality of the solutions. For example, on Figure 1, the three leaves on the right, with a small value of S8 contain more near-optimal solutions than leaves with a high value of S8. This feature corresponds to the routes compactness and needs to be as low as possible. Same conclusion can be drawn for the feature S3, the average distance between depot to directly-connected customers and feature S5, the average width per route. The DT made by F. Arnold and K Sörensen splits features S8 near 0.25 and S3 on 167, 199, 222, 240, 294. Our tree splits feature S8 on 0.25 and S3 on 229 and 248. The features chosen and their pivot value are very close, validating their results.

However, one aspect omitted by the authors is the extension of such interpretation to ensemble methods such as random forests, that combine hundreds of DT in order to reach better predictive performances than a single DT. Figure 2 the variables importance plot. In Figure 2, the MeanDecreaseAccuracy graph indicates the mean loss of accuracy by permuting features, one by one. The MeanDecreaseGini graph presents the mean variation of the Gini coefficient [START_REF] Breiman | Classification and regression trees[END_REF] when splitting a leaf. This coefficient represents how heterogeneous the leaves are. The higher the Gini coefficient, the more heterogeneous the leaves. Oppositely, a value of 0 means the complete homogeneity of the leaves. This coefficient must therefore be minimized.

In the original paper [START_REF] Arnold | What makes a VRP solution good? The generation of problem-specific knowledge for heuristics[END_REF], the authors distinguish two types of features : instances vs. solutions features. To identify the type of each feature, the instances features are printed in red and the solutions features are printed in blue. It seems that features I1 to I8 have much less significance than features S1 to S10. This idea has been confirmed by all our new experiments. As for the importance plot, the mean decrease accuracy and Gini values are respectively lower than 20 and 30. Oppositely, features S3, S5, S6, S8 and S10 have a MeanDecreaseAccuracy and a MeanDecreaseGini values larger than 60. Considering all the other instances provided by the authors, a meta-analysis has been run. A feature is considered as relevant if the MeanDecreaseAccuracy and MeanDecreaseGini associated is greater than 60. Features S1, S3, S5, S6, S7 and S8 are considered as significant for at least 6 sets of instances. The other features are not considered as significant for at least 14 sets of instances. In order to confirm that characteristics I1 . . . I8 are unnecessary, experimentations have been carried out. For this purpose, we will observe the efficiency of a random forest and a SVM with all the characteristics I1 . . . I8, then without these characteristics. Each pair Algorithm/Features is executed 50 times, with different training and test sets. Table 3 details the average results. We can observe a very low evolution of the performances with and without knowledge of I1. . . I8. These features do not affect the quality of the estimations. As a conclusion, features I1. . . I8 are useless for the set of instances "gap2 large center highVariance.csv".

Features

Random Forest Support-Vector Machine I1. . . I8, S1. . . S10 75.16% 77.28% S1. . . S10 76.10% 77.16% As seen in section 3, there are many methods of machine learning giving some keys to understand each feature significance. Nevertheless, this section will show that a classical factorial analysis, namely Principal Component Analysis (PCA) is able to address most of the issues tackled by the authors.

Preliminary step: correlation analysis

Figure 3 shows the Pearson correlation coefficient between all pairs of features. A value near 1 is equivalent to a strong correlation. A value near -1 is equivalent to a strong anticorrelation. Finally, a value near 0 means the features have no correlation. As notified on the right side of the legend, the colour of each circle indicates if the correlation is positive or negative and its degree by the intensity of the colour. The size of the circles grows with the absolute value of the correlation. With a correlation value greater than 0.7, features S5, S6, S7 and S8 are peer-to-peer strongly correlated. The same applies to S4 and S9. Oppositely, with a correlation value lower than -0.7, features S3 and S10 are anti-correlated. This very simple descriptive statistical analysis points out initial redundancies between the chosen variables. This trend has been checked for the other sets of instances. Correlations between features S5, S6, S7 and S8 are observed for each set of instances. Features S4 and S9 are strongly correlated in 9 sets of instances and non-correlated in the other sets of instances. Features S3 and S10 are always more or less anti-correlated, with correlation always less than -0.4. Finally, features S3 and S6 are strongly anti-correlated in at least 11 sets of instances. Once again, it is possible to reach global conclusions for all sets of instances.

PCA: principle

The Principal Component Analysis (PCA) projects the initial variables onto a factorial space, i.e. the principal components, which are linear combinations of the S1 to S10 variables. Figures 4 and5 are a part of those projections onto two of the most significant dimensions (a dimension is significant if it explains a large part of variability). On those projections, the longest an arrow, the best their corresponding features are represented on the factorial space. The angle between 2 arrows indicates the correlation between 2 associated features. The blue ellipse (E1) contains 95% of the near-optimum solutions, the red ellipse (E0) contains 95% of the non-optimal solutions.

PCA results

Figure 4 represents the correlation circle over the two most significant dimensions which explain all together 59% of the data variance (respectively 39% and 20% of the variability is by Dim1 and Dim2). With respect to the length of their arrows, features S6, S5, and S7 are well represented, oppositely to S2, very badly represented. The angle between features S6 and S7 is extremely low, indicating a strong correlation between them. On the opposite there is no correlation between features S1 and S8, with an angle close to π 2 . Finally, the angle between features S3 and S10 is close to π and shows anti-correlation between them. Regarding the solution quality, the blue and red ellipses, containing respectively 95% of the near-optimal and non-optimal solutions, are mostly intersected.

By observing which features are most described by each dimension, we can deduce the meaning of each one. We keep only features with correlation larger than 0.7 in absolute value with the different dimensions. Thus, the first dimension is mainly represented by features S6, S5, S7 and S8. All these features aggregated in dim 1 are different ways to represent the compactness of routes. Consequently, dim1 represents the compactness of each route. Equivalently, dim2 is represented by features S9 and S4. Thus, this dimension represents how deep and well-balanced are the routes.

Considering a third dimension helps to better explain differences between near-optimal and non-optimal solutions. The third dimension represents mostly the distance between the first/last customer and the depot. Figure 5 adds more precision about the features and the differences between near-optimal and non-optimal solutions. Blue and red ellipses are less intersected on the projection with respect to Dim3. While Figure 4 shows a bad representation of feature S2, this trend is confirmed by Figure 5, proving that looking at the longest distance between two connected customers is not as efficient as it seemed. Regardless which dimensions are used, the angles between arrows associated to features S5, S6, S7, S8 are always small, showing their correlations. It seems logical as they correspond to four different representations of routes compactness. Theses features are mainly associated with non-optimal solutions: keeping compact routes is not a priority and we have to minimize theses features as much as possible.

Toward a refined variable selection process for machine learning

The factorial analysis helps us to improve the variables selection process. It also opens a new area for improving machine learning predictions. First, in order to validate our hypothesis about the uselessness of features I1..I8, several experiments have been carried out. For each test, 50 random forests are created, containing exactly 2000 trees. Initially, the random forest methods success to classifies 75.16% of the tested instances, on average, with all I1 to I8 and S1 to S10. If we classify only with features S1 to S10, we obtain a better average success of 76.1%. Moreover, removing feature S2 allows to reach an accuracy score of 76.32%. Consequently, removing a priory useless features increases slightly this accuracy. Hence, we have proven uselessness of features I1 . . . I8 and S2 for predicting the efficiency of a solution. Finally, adding useless features seems to reduce the impact of significant features, reducing the accuracy of the final prediction.

Conclusion

With this comment, we firstly would like to underline the pioneer work of F. Arnold and K. Sörensen which is an important attempt to bridge the gap between machine learning and optimization. Then, we argue that a simple data mining approach such as PCA can do the job: to address the initial question "what makes a VRP solution good? ", a descriptive method with a high explanatory power is more useful than predictive black-box algorithms. With more interpretation of modern data mining or machine learning techniques, we can change our way to solve vehicle routing problems. For example, in their article, F. Arnold and K. Sörensen are using some penalties to guide a metaheuristic to find better solutions. Using the significance of each feature and their correlation could be used to improve the guidance of any optimization method. By the significance of each feature and their correlation, we are able to specialize which feature usage and the value of their penalties. As a perspective, an ongoing work consists in implementing the conclusions about solutions features into an adapted neighborhood selection local search method.

Figure 2 :

 2 Figure 2: Importance plot of predictive variables.

Figure 3 :

 3 Figure 3: Correlation plot of the solution metrics

Figure 4 :Figure 5 :

 45 Figure 4: PCA with the first two dimensions

Table 1 :

 1 Solution Features

	I1 Number of customers
	I2 Number of routes
	I3 Degree of capacity utilisation
	I4 Average distance between each pair of customers
	I5 Standard deviation of the pairwise distance between customers
	I6 Average distance from customers to the depot
	I7 Standard deviation of the distance from customers to the depot
	I8 Standard deviation of the radians of customers towards the depot

Table 2 :

 2 Instance Features

  fills this gap by representing

					1 1
				yes yes	Is S8 >= 0.25? no Is S8 >= 0.25? no
		2 2						3 3
		Is S3 >= 229? Is S3 >= 229?						Is S3 >= 248? Is S3 >= 248?
			5 5				6 6
		Is S8 >= 0.3? Is S8 >= 0.3?				Is S5 >= 255? Is S5 >= 255?
					11 11
				Is S10 < 3? Is S10 < 3?
			22 22
			Is S5 >= 274? Is S5 >= 274?
	4 4	10 10	44 44	45 45		23 23	12 12	13 13	7 7
	0	0	0	1		1	0	1	1
	.80 .20	.70 .30	.79 .21	.33 .67	.32 .68	.65 .35	.27 .73	.20 .80

Figure 1: Best decision tree for class 6 instances (2% gap).

Table 3 :

 3 Experiments on the utility of I1. . .I8

4.

No need for Machine Learning ? How PCA enables a synthetic view of the solutions characteristics

Acknowledgement

The authors would like to thank F. Arnold and K. Sörensen for providing their data, available on http://antor.uantwerpen.be/problem-knowledge/. Our data and the R script used to create these plots are also available on demand.


J Delanne

S Nambot

A Chassagne

A Pelissier

C Peyron

E Gautier

J Thevenon

E Cretin

A L Bruel

V Goussot

F Ghiringhelli

R Boidot

F Tran- Mau-Them

C Philippe

A Vitobello

L Demougeot

C Vernin

A S Lapointe

M Bardou

M Luu

C Binquet

C Lejeune

L Joly

C Juif

A Baurand

C Sawka

G Bertolone

Y Duffourd

D Sanlaville

P Pujol

D Geneviève

F Houdayer

C Thauvin-Robinet

MD-PhD Laurence Faivre
email: laurence.faivre@chu-dijon.fr

MD-PhD Christel Thauvin
email: christel.thauvin@chu-dijon.fr

Secondary findings from whole-exome/genome sequencing evaluating stakeholder perspectives. A review of the literature

With the development of next generation sequencing, beyond identifying the cause of manifestations that justified prescription of the test, other information with potential interest for patients and their families, defined as secondary findings (SF), can be provided once patients have given informed consent, in particular when therapeutic and preventive options are available. The disclosure of such findings has caused much debate. The aim of this work was to summarize all opinion-based studies focusing on SF, so as to shed light on the concerns that this question generate.

A review of the literature was performed, focusing on all PubMed articles reporting qualitative, quantitative or mixed studies that interviewed healthcare providers, participants, or society regarding this subject. The methodology was carefully analysed, in particular whether or not studies made the distinction between actionable and non-actionable SF, in a clinical or research context.

From 2010 to 2016, 39 articles were compiled. A total of 14,868 people were interviewed (1,259 participants, 6,104 healthcare providers, 7,505 representatives of society). When actionable and non-actionable SF were distinguished (20 articles), 92% of respondents were keen to have results regarding actionable SF (participants: 88%, healthcare providers: 86%, society: 97%), against 70% (participants: 83%, healthcare providers: 62%, society: 73%) for non-actionable SF. These percentages were slightly lower in the specific situation of children probands.

For respondents, the notion of the «patient's choice» is crucial. For healthcare providers, the importance of defining policies for SF among diagnostic lab, learning societies and/or countries is outlined, in particular regarding the content and extension of the list of actionable genes to propose, the modalities of information, and the access to information about adultonset diseases in minors.

However, the existing literature should be taken with caution, since most articles lack a clear definition of SF and actionability, and referred to hypothetical scenarios with limited information to respondents. Studies conducted by multidisciplinary teams involving patients with access to results are sadly lacking, in particular in the medium term after the results have been given. Such studies would feed the debate and make it possible to measure the impact of such findings and their benefit-risk ratio.

INTRODUCTION

Recent technical progress led to development of genomic medicine, which now makes it possible to sequence the genome of individuals at a reasonable cost and in a time compatible with care. This development was associated with an increase in knowledge concerning the medical significance of the variants identified. Since 2009, next generation sequencing (NGS) has become an innovative technology to analyse the DNA of an individual's genome (or its coding regions, the exome, which corresponds to 1.5% of the entire genome). NGS, and in particular, sequencing of the whole exome (WES), is now widely used in research and has led to significant advances in our understanding of the molecular bases of Mendelian diseases, which could not be determined using classical approaches. Thus, genomic medicine is based on the analysis and interpretation of genomic information in a medical context, but major obstacles need to be overcome before its full implementation.

The very recent implementation of WES for rare developmental diseases, which are particularly heterogeneous, has led to a diagnostic yield greater than 25%, making it by far the most efficient individual diagnostic test for such conditions, in particular in the absence of a precise clinical diagnosis [START_REF] Ligt | Diagnostic Exome Sequencing in Persons with Severe Intellectual Disability[END_REF][START_REF] Retterer | Clinical Application of Whole-Exome Sequencing across Clinical Indications[END_REF][START_REF] Yang | Clinical Whole-Exome Sequencing for the Diagnosis of Mendelian Disorders[END_REF][START_REF] Yang | Molecular Findings among Patients Referred for Clinical Whole-Exome Sequencing[END_REF]. Proof-of-concept studies suggest that whole genome sequencing (WGS) could increase the diagnostic yield to 60% for rare developmental diseases [START_REF] Gilissen | Genome Sequencing Identifies Major Causes of Severe Intellectual Disability[END_REF].

Despite its considerable interest in care, the implementation of WES and WGS varies widely from country to country.

Beyond the identification of pathogenic variants for the diagnosis of rare Mendelian diseases, the sequencing of an individual's genome may lead to the detection of numerous variants unrelated to the primary diagnosis for which NGS was prescribed but may be pertinent in terms of prevention for the patients and/or their families. Indeed, these variants may predispose the individual to diseases with a later onset that could be accessible to prevention or treatment, or they may provide information on the risk of transmitting a serious disease, and could thus be useful for family genetic counselling, or finally, these variants may predispose an individual to toxicity or the absence of efficacy of a given drug, and such knowledge may make it possible to adapt a treatment if necessary. These data can be particularly useful for highly medicated patients with rare diseases, whose families are deeply affected by the disease.

These variants, which correspond to findings not related to the disease for which the test was initially prescribed, were until very recently called incidental findings in the North-American literature [START_REF] Green | ACMG Recommendations for Reporting of Incidental Findings in Clinical Exome and Genome Sequencing[END_REF]. This term is no longer appropriate in cases of active screening for variants in certain genes. In the latest recommendations of the American College of Medical Genetics and Genomics (ACMG) and of the European Society of Human Genetics (ESHG), the term was progressively replaced by 'secondary findings' (SF) in an effort to standardise [START_REF] Kalia | Recommendations for Reporting of Secondary Findings in Clinical Exome and Genome Sequencing, 2016 Update (ACMG SF v2.0): A Policy Statement of the American College of Medical Genetics and Genomics[END_REF][START_REF] Matthijs | Guidelines for Diagnostic next-Generation Sequencing[END_REF].

The ACMG was the first organisation to define and propose an initial minimal list of 56 genes that were considered medically «actionable». Among these, pathogenic variants or potentially pathogenic variants must be reported by all clinical laboratories that propose WES and WGS [START_REF] Green | ACMG Recommendations for Reporting of Incidental Findings in Clinical Exome and Genome Sequencing[END_REF], unless the patient has previously refused to have access to this information in the informed consent process (American College of Medical Genetics and Genomics 2014). In 2016, this list, which is intended to be regularly revised, was extended to 59 genes [START_REF] Kalia | Recommendations for Reporting of Secondary Findings in Clinical Exome and Genome Sequencing, 2016 Update (ACMG SF v2.0): A Policy Statement of the American College of Medical Genetics and Genomics[END_REF]. An "actionable" gene is defined as a gene which, in the case of a pathogenic or potentially pathogenic mutation, will increase the risk of developing a genetic disease accessible to prevention or treatment, especially if the treatment is more effective if given early. Such interventions would thus improve the prognosis in terms of mortality or morbidity [START_REF] Green | ACMG Recommendations for Reporting of Incidental Findings in Clinical Exome and Genome Sequencing[END_REF]. These are essentially genes with pathogenic or probably pathogenic variants that predispose individuals to hereditary heart diseases or cancers. Certain laboratories propose to screen for a more complete list of genes with actionable variants [START_REF] Amendola | Actionable Exomic Incidental Findings in 6503 Participants: Challenges of Variant Classification[END_REF][START_REF] Dorschner | Actionable, Pathogenic Incidental Findings in 1,000 Participants' Exomes[END_REF], including genes in which pathogenic or probably pathogenic mutations have been identified in roughly 2% of patients. A few publications have proposed methods to evaluate the actionable nature of a gene, so as to complete the list of genes to target in the future depending on progress in our knowledge [START_REF] Berg | A Semiquantitative Metric for Evaluating Clinical Actionability of Incidental or Secondary Findings from Genome-Scale Sequencing[END_REF][START_REF] Hunter | A Standardized, Evidence-Based Protocol to Assess Clinical Actionability of Genetic Disorders Associated with Genomic Variation[END_REF]. The implications in terms of healthcare organisation have been little discussed [START_REF] Bennette | The cost-effectiveness of returning incidental findings from next-generation genomic sequencing[END_REF], but it is clear that interpreting such findings will increase the workload of healthcare professionals as much for the clinicians as for the biologists. In term of patient management, we should highlight the risk of utilizing unnecessary health care that may not be needed, because of reduced penetrance, and that could be more harmful than beneficial.

The analysis of these variants, if requested by the patient or the patient's guardian, could become the first step in genomic medicine [START_REF] Bowdin | Recommendations for the Integration of Genomics into Clinical Practice[END_REF]Manolio et al., 2015].

However, the interpretation of such variants, which are not related to the primary cause of the disease, will require additional investment from healthcare professionals, notably in terms of time. Although benefits can be derived in terms of prevention, it is a major issue to take into account the point of view of patients and more widely that of society at large. This article thus investigates the point of view of patients, of healthcare professionals and of society via a review of the literature on SF. We have synthesized the principal studies focusing on the return of SF, so as to shed light on the concerns that this question generates.

METHODS

The aim of this study was to synthesize all relevant studies focusing on the return of SF published in referenced scientific journals, so as to determine the principal expectations and concerns of patients, research participants, healthcare professionals and the public with regard to this issue. In this study, we generally refer to "participants" instead of "patients" because respondents in a number of these studies were not patients but research participants. To this end, in October 2016, we used the PubMed search tool with the following key words: "Incidental findings", "Secondary findings", "Ancillary findings", "Unsolicited findings", "Opportunistic findings", or "Accessory findings". These could be accompanied by accessory key words, such as: «Genetics»; «Clinical actionability»; «Whole exome sequencing»; «Healthcare provider»; «Geneticist»; «Professional»; «Society»; «Patient». The selection was limited to articles available in English or in French, to articles interviewing more than five people, and to articles that focused on the subject of secondary findings.

We chose to separate the articles into three principal categories depending on the population involved: i) participants, ii) healthcare professionals, iii) society. It is commonly admitted that point of views differs between patients/participants and healthcare professionals. We also decided to separate participant which could be directly concerned by WES/WGS and society (representing people with no direct concern in WES/WGS for health purposes) as their emotional involvement, their knowledge and personal expectations may alter their vision of WES/WGS. We then analysed each category with regard to the methodology: quantitative, qualitative, or mixed.

With the aim to make comparisons, we distinguished between actionable and non-actionable SF. For some articles, ACMG definitions was explicitly used, others simply used examples this highlight this difference, as stated in table 2. Qualitative and quantitative data were analysed separately (Table 2). Only quantitative data from quantitative or mixed article were used to calculate percentages. Qualitative studies were used as references to emphasize or counterbalance our results. In order to facilitate the reading and understanding, we decided to add to each reference in the results section their corresponding methodology: "Ql" for qualitative, "Qt" quantitative and "M" for mixed studies. As some articles focuses on different populations (participants/healthcare providers/society), they can use different methodology on each population. Therefore, the same reference can be classified in a different section of table 2 depending on the population of interest. Furthermore, some of the studies involved opinions regarding the return of results of SF in a research context, whereas others focused on the return of results in a diagnostic context (Table 1 and2).

RESULTS

General data (Figure 1 and 2)

Thirty-nine articles were selected according to the criteria mentioned above (Table 1

a, b, c).

The oldest article was published in 2010 [Meacham et al., 2010 (Ql)] (Figure 1a). Thirtythree articles were form North America, three were from Europe, and three were from Australia (Figure 1b).

Initially, we focused on the interpretation of the results (Figure 2), as it is now clear that it is essential to distinguish between opinions on the actionable or non-actionable nature of the SF.

Nonetheless, as half of the articles made no difference between actionable and non-actionable SF, the results below must be interpreted with caution. Among the 39 articles, 20 mentioned this difference, and half of them were referring to a research context whereas the other half were referring to a diagnosis context. Eleven of these referred to the ACMG list for actionability, and the other nine made use of precise examples. In these 20 articles, 92% of the persons interviewed were in favour of conveying the results for actionable SF, and 70% were in favour for non-actionable SF (Table 2). This percentage was slightly lower when respondents were questioned about the specific case of children. In this case, 84% of persons interviewed were in favour of disclosing actionable SF to children or to their legal guardians, and 68% for non-actionable SF. The percentages were lower in a diagnostic context than in a research context (Table 2).

Studies concerning patients/participants (Table 1a)

Thirteen articles were identified, including five quantitative studies, four qualitative studies and four mixed studies (Table 1a). Altogether, there were 1259 participants in these studies (1010 in the quantitative studies, 85 in the qualitative studies, and the remaining 164 in the mixed studies) (Figure 2); 1097 index cases were adults and 162 were children. Five of the 13 studies focused on participants with cancer, for a total of 438 participants.

Concerning the quantitative [START_REF] Fernandez | Attitudes of Parents toward the Return of Targeted and Incidental Genomic Research Findings in Children[END_REF][START_REF] Fernandez | Stability of Attitudes to the Ethical Issues Raised by the Return of Incidental Genomic Research Findings in Children: A Follow-Up Study[END_REF][START_REF] Gray | Oncologists' and Cancer Patients' Views on Whole-Exome Sequencing and Incidental Findings: Results from The CanSeq Study[END_REF][START_REF] Jelsig | Research participants in NGS studies want to know about incidental findings[END_REF][START_REF] Shahmirzadi | Patient Decisions for Disclosure of Secondary Findings among the First 200 Individuals Undergoing Clinical Diagnostic Exome Sequencing[END_REF] and mixed studies [START_REF] Appelbaum | Informed Consent for Return of Incidental Findings in Genomic Research[END_REF][START_REF] Gray | Attitudes of Patients with Cancer about Personalized Medicine and Somatic Genetic Testing[END_REF][START_REF] Kaphingst | Preferences for return of incidental findings from genome sequencing among women diagnosed with breast cancer at a young age[END_REF][START_REF] Oberg | Overcoming Challenges to Meaningful Informed Consent for Whole Genome Sequencing in Pediatric Cancer Research[END_REF], the majority of participants wished to know the SF found in the genetic investigations. When the notion of actionability was mentioned, this percentage was 88% if the SF were actionable and 83% if they were not actionable. In a diagnosis context, only one article [Shahmirzadi et al., 2013 (Qt)] focuses on the opinion of participant with 94% of them in favour of the disclosure of actionable secondary findings. In a research context, 87% of the participants were in favour of the disclosure of actionable secondary findings and 83% for non-actionable ones.

Following two studies in oncology conducted by Gray [Gray et al., 2012 (M); Gray et al., 2016 (Qt)], the strong desire of participants to learn about any type of SF showed the importance of providing high-quality information to patients to help them choose the type of secondary findings they wished to know about. A study conducted by Shahmirzadi and coll. in 2013 [Shahmirzadi et al., 2013 (Qt)] evaluated the choice of having access to SF in the first 200 patients referred to Ambry Genetics for WES following a consultation with a genetic counsellor during which information was given (162 parents of affected children and 38 adult patients). The persons whose choice were retrospectively analysed, could choose to receive the results for pathogenic or probably pathogenic variants in four categories of SF: healthy carrier of a recessive disease; predisposition to actionable later-onset diseases; actionable predisposition to cancers; predisposition to actionable childhood-onset diseases. The parents of affected children were more likely to choose to know the SF for their children (96%) than were adult patients for SF for themselves (84%). In 2015, Fernandez and coll. [Fernadez et al., 2015 (Qt)] re-surveyed 154 parents of children with rare diseases after a first survey 18 months earlier [Fernadez et al., 2014 (Qt)], and showed strong stability in their responses.

Altogether, 85% of the parents believed that they should be given the choice to have access to SF whether actionable or not. Only 74% believed that these results should be shared with atrisk relatives. The qualitative data (Table 1a) showed that the participants wanted to be involved in the choice of variants to be disclosed. They were afraid, however, that these results would cause anxiety [START_REF] Oberg | Overcoming Challenges to Meaningful Informed Consent for Whole Genome Sequencing in Pediatric Cancer Research[END_REF]]. In a study in 37 persons who underwent WES for a rare disease or cancer, Clift and coll. [Clift et al., 2015 (Ql)] showed that patients had great expectations with regard to these SF. The patients were interviewed on four occasions: before the first consultation with a genetic counsellor, after the first consultation with a genetic counsellor, while they were waiting for the results, and after the results had been given. One person justified the choice of knowing the results, including for non-actionable SF, by saying that he wanted to be sure that he wouldn't be a burden for his loved ones. Others said that it would let them contribute to research to help find a preventive or curative treatment for these diseases. The study of [START_REF] Sapp | Parental Attitudes, Values, and Beliefs toward the Return of Results from Exome Sequencing in Children[END_REF]coll. in 2014 [Sapp et al., 2014 (Ql)] in 25 parents of 13 children with various genetic diseases showed that 60% of parents did not want to receive the results for non-actionable SF. They said they needed more time to think about it and were concerned about the uncertainty and the worry caused by these disclosures. The question of research on the carrier status in recessive diseases was also raised, and 56% of parents preferred to leave the choice of knowing these results to their children once they became adults. In 2016, Kaphingst and coll [Kaphingst et al., 2016 (M)] conducted 60 semi-structured interviewed with young women diagnosed with breast cancer.

All women were interested in receiving actionable secondary findings, also comprising pharmacogenetic variants.

Studies concerning healthcare professionals (Table 1b)

Twenty-two articles focused on the point of view of healthcare professionals (Table 1b), eight quantitative, eight qualitative and six mixed studies. When the number was given, a total of 6104 medical geneticists, oncologists, paediatricians, genetic counsellors, genetic researchers, reviewers and other specialists were interviewed. Of these, 3031 took part in a quantitative study, 230 in a qualitative study and 2843 in a mixed study (Figure 2). When asked about their opinion on the disclosure of actionable secondary findings, healthcare providers where in favor of their return 86% of the time. Percentages were comparable in a diagnosis context (86%) and in a research context (91%). In the case of non-actionable secondary findings, 62%

were in favour of their disclosure in a diagnosis context, but they were slightly less favourable for their disclosure in a research context (55%).

The article by Gliwa and coll. [Gliwa et al., 2016 (Qt)] focused on non-geneticist reviewers, and 74% of respondents said they had had some experience with SF in genetics, but only 47% felt ready to disclose such findings to patients. Gray and coll. [Gray et al., 2016 (M)] focused on the point of view of oncologists, among whom 67% were not in favour of restricting the results given to patients to actionable genes only; 52% believed that patients should be given all of the results they wanted, even raw sequencing data. In this respect, Hitch and coll. [Hitch et al., 2014 (Ql)] reported that «compared with the cancer, negative information was surmountable». The study of Strong and coll. [Strong et al., 2014 (M)] included 258 paediatricians. Among these, 91% believed that actionable SF for adult-onset diseases should be given to families if they so wished and 96% believed so for actionable SF for childhood-onset diseases, compared with 47% for non-actionable SF. Concerning medical geneticists, overall, 95% were prepared to disclose medically actionable SF [Lohn et al., 2013 (Qt)]. In the article of Yu and coll. [Yu et al., 2014 (Qt)] in which 760 American geneticists were surveyed, 85% believed that SF in ACMG genes should be given to adult patients and 74% were of this opinion for parents of affected children. Sixty-two per cent believed that SF should be given to the parents of affected children with adult-onset diseases or carrier status (62%), and 81% believed that the preferences of patients should guide the practitioner in the choice of variants to disclose. In 2016, Middleton and coll. [Middleton et al., 2016 (M)] surveyed 533 medical geneticists, 843 non-genetic healthcare professionals and 607 genomic researchers. Ninety-two per cent of geneticists and 94% of non-geneticist healthcare professionals and genomic researchers believed that pertinent SF from sequencing should be returned to patients if they so wished. In this article, the authors distinguished between a prescription for NGS for diagnostic purposes and for research. Only 34% maintained their position if the search for these SF could compromise the ability of researchers to analyse the primary data, in particular because of the additional workload due to the analysis of secondary findings.

Several articles converged regarding the difficulties related to the absence of national guidelines, which should remain flexible enough to be adapted to individual situations, to the evolution of knowledge, and to the choice of patients [Downing et al., 2013 (Ql); Keogh et al., 2014 (Ql); Simon et al., 2011 (Ql); Yu et al., 2014 (Qt)]. Scheuner and coll [Scheuner et al., 2014 (Qt)] identified four factors that explained the variance of opinion between healthcare professionals in their decision to disclose or not SF: the clinical and laboratory best practices, patients' preferences, the guidance for practice and the informed consent. This raises the issue of dynamic consent [Keogh et al., 2014 (Ql)]. Grove and coll [Grove et al., 2014 (Ql)] spoke of it as an iterative and flexible process with unlocking results to "take into account not only the changing interpretations of the genome, but also the changing values and contexts of patients over time".

Only three articles dealt specifically with the question of pharmacogenetics [Gray et al., 2016 (M); Grove et al., 2014 (Ql); Klitzman et al., 2013 (M)] (Table 1b).

Studies concerning society at large (Table 1c)

Nine articles were identified [Christenhusz et al., 2015 (Ql); Daack-Hirsch et al., 2013 (Ql);Flemming et al., 2015 (Qt); Haga et al., 2012 (Ql); Hufnagel et al., 2016 (Qt); Middleton et al., 2016 (Qt); Regier et al., 2015 (Qt); Strong et al., 2014 (M); Townsend et al., 2012 (Ql)], in which 7505 people were recruited. The vast majority were recruited in four quantitative studies (7243 respondents), 165 took part in four qualitative studies and 97 in a study employing a mixed methodology. Four articles made the distinction between actionable secondary findings and non-actionable ones. Half of them referred to a diagnosis context [Hufnagel et al., 2016 (Qt); Strong et al., 2014 (M)] and the other half to a research context [Fleming et al., 2015 (Qt); Middleton et al., 2016 (Qt)] (Figure 1c).

In their study, Middleton and coll. [Middleton et al., 2016 (Qt)] surveyed 4961 persons representing the public, of whom 83% had never had any contact with genetics (79% lived in Ireland or the United Kingdom). In this study, surveyed participants were asked to put themselves in an hypothetical scenario of a patient who has been given a prescription for WES. It showed that 98% of participants wished to know about actionable SF and 73% also wanted the return of non-actionable SF. In the study by Fleming and coll. [Flemming et al., 2015 (Qt)] involving 800 Australian adults, 94% wanted the return of information defined as important for their health or for the treatment of their hypothetical disease; 84% wanted information on their risk of being a healthy carrier of a hereditary genetic disease; 70% wanted to receive «every type of SF (including non-actionable SF) ». Hufnagel and coll [Hufnagel et al., 2016 (Qt)] surveyed 282 students with scientific training aged for the most part between 12 and 18 years old. Of these, 83% wanted access to adult-onset SF that were not actionable during childhood; 72% wanted to be involved in the decision-making process; 73% thought that parents of a child less than 12 years old should have access to this information at the same time as their child. Different articles converged with regard to leaving the choice to the patient [Daack-Hirsch et al., 2013 (Ql); Strong et al., 2014 (M); Townsend et al., 2012 (Ql)].

DISCUSSION

This review of the literature with the main opinion-based articles on SF showed that the vast majority of participants and society at large were in favour of returning actionable SF, and that the majority of doctors were prepared to screen for and disclose these actionable SF to their patients. However, the results of the studies must be interpreted with caution, since most articles referred to hypothetical scenarios. Hypothetical situations are a notoriously flawed way to predict ultimate behaviour. It is worth noting that principally because of the anxiogenic nature of these results, a small percentage of those interviewed did not wish to know about these SF [Oberg et al., 2015, Table 2]. In this context, it is necessary to explore the preferences of the patients and to obtain their consent. To this end, the search for SF must be explained clearly to patients to ensure that they have understood what SF are and what the implications and repercussions of finding them could be [START_REF] Hehir-Kwa | Towards a European Consensus for Reporting Incidental Findings during Clinical NGS Testing[END_REF]. Depending on the analysis circuit, it could take a long time to obtain the results for WES and WGS, which gives time to patients to change their mind. The solution proposed would be to develop computerised «dynamic consent» [START_REF] Kaye | Dynamic Consent: A Patient Interface for Twenty-First Century Research Networks[END_REF], which can allow consent preferences to 'travel' with the patient. This, however, would be difficult to implement and could exclude persons in precarious situations. An alternative would be deferred consent for SF [START_REF] Oberg | Overcoming Challenges to Meaningful Informed Consent for Whole Genome Sequencing in Pediatric Cancer Research[END_REF]. When participants were interviewed about the content of this consent, they principally expected detailed consent forms indicating the possibilities of possible prevention for certain variants, the fact that there may be errors in the interpretation of certain variants, the possibility that certain results could have a negative psychological impact, the possibility that results could evolve with advances in scientific knowledge, and finally the possibility of limiting the adverse effects of certain drugs if pharmacogenetic variants are searched [START_REF] Appelbaum | Informed Consent for Return of Incidental Findings in Genomic Research[END_REF].

It is important to note that managing these SF is time-consuming, in terms of both clinical time, as it takes additional time to explain SF during consultations with patients, and time for interpreting the results. Participants would like consultations to last as long as necessary, up to 2h, while geneticists consider that no more than 30 min should be spent on this pre-test phase (77%) [START_REF] Appelbaum | Informed Consent for Return of Incidental Findings in Genomic Research[END_REF].

In parallel of our initiative, another group reviewed the literature relating on SF but focused more on qualitative studies [START_REF] Mackley | Stakeholder views on secondary findings in wholegenome and whole-exome sequencing: a systematic review of quantitative and qualitative studies[END_REF]. Our methodology also focused on quantitative studies that permitted to collect high numbers of opinions, in parallel of qualitative studies. The limits of this review of the literature are principally related to the heterogeneity of the articles studied. Indeed, they were conducted on different populations, in different countries with different legislation on patients' rights and different cultures, and proposed different methodologies that cannot be combined. While the total numbers of people concerned with these articles appear impressive (around 15,000), it turns out that only slightly more than 1200 people were offered and chose to learn such findings, while the others were confronted to hypothetical situations. Some of the studies collected opinions regarding SF in a research context, whereas others focused on SF in a diagnostic context, with different obligations for researchers and clinicians. When analysing in subgroups, the number of articles/respondants diminished. Methodological rigour varied and this aspect was difficult to take into account in compiling and comparing results. The quality and the way information was delivered also differed as it was given by medical geneticists, specialist doctors or genetic counsellors or in some cases by an information sheet or even a brief explanation by telephone.

One of the major concerns among scholars on this issue is whether patient/participant/public responses to the question of the return of secondary findings were well-informed, i.e., if respondents are taking into account the uncertain risk of developing the disease and limited available actions related to many of the findings that they say they want. The quote from an article by Cliff and coll. [Cliff et al., 2015], that "one person justified the choice of knowing the results, including for non-actionable SF, by saying that he wanted to be sure that he wouldn't be a burden for his loved ones" is indicative of the inflated perception of clinical utility that people assume secondary findings possess, probably leading to their broad acceptance. Results may also differ depending on the study populations, in patients with rare diseases or cancer, interviewing adult patients, children or parents of children, certain articles making no distinction between the three. Patients with cancer are expected to have different opinions about the value of additional information as compared to people with long standing chronic diseases or parents of children with rare diseases. The information provided to society at large is usually more succinct than that given to participants, who are often better informed about genetic aspects because of their disease. It seems also essential to distinguish between guidelines for tests in children and those in adults, even though the differences between the studies were not significant. The only study involving surveys with teenagers showed that 83% of those surveyed wanted to receive non-actionable SF, and 72% believed that the decision on whether to receive SF should be shared between parents and children [START_REF] Hufnagel | Adolescents' Preferences Regarding Disclosure of Incidental Findings in Genomic Sequencing That Are Not Medically Actionable in Childhood[END_REF]. In another study, [START_REF] Sapp | Parental Attitudes, Values, and Beliefs toward the Return of Results from Exome Sequencing in Children[END_REF]coll. 2014 [Sapp et al., 2014] showed that 56% of parents of children with various genetic diseases would prefer to leave the choice to their child once he/she had reached adulthood, in order to preserve their independence and freedom. The question of access to results for late-onset diseases in children is not resolved, and contrasts with the absence of any possibility to obtain a pre-symptomatic diagnosis in children if there is no direct benefit in childhood. Certain teams proposed getting in touch with the children again when they became adults, but the risk of losing contact in the intervening period is huge. When parents also undergo sequencing as part of the diagnostic investigations, the question of informing only the transmitting parent appears to be a good alternative to avoid lost opportunities for relatives.

In general, a clear definition of SF and the concept of actionability was lacking. This review of the literature showed that it is essential to distinguish between actionable and non-actionable genes when discussing SF. The notion of actionability, however, is sometimes vague as it can sometimes refer to actionability for patients themselves, and in other circumstances, the concept is widened to genetic counselling for the family. Unfortunately, half of the articles made no difference between actionable and non-actionable secondary findings. All articles written prior to 2013 could not make use of the ACMG definition of actionability [START_REF] Green | ACMG Recommendations for Reporting of Incidental Findings in Clinical Exome and Genome Sequencing[END_REF], and thus most of the time were based on examples. The percentage of healthcare professionals prepared to return non-actionable SF needs to be interpreted with care, as, in general, the positive responses only reflect the fact that they found the minimal list of actionable genes too restrictive. In 2013, Dorschner and coll. [START_REF] Dorschner | Actionable, Pathogenic Incidental Findings in 1,000 Participants' Exomes[END_REF] increased the list of ACMG genes to return by 62 additional actionable genes. Other lists have also been proposed in the literature. These lists are reported as non-exhaustive, containing a minimal number of genes that can be revised by laboratories. Certain authors proposed to screen for pharmacogenetic variants that may be relevant in chronic diseases subject to more medical prescriptions; or to search for heterozygosity in the most frequent rare recessive diseases, which may be relevant in terms of information for parenthood projects, in families already highly affected by the disease and the resulting handicap. It is worth noting that one study investigated the perception of pharmacogenetic data. It also appeared that doctors were not particularly inclined to use these data: 52% of the doctors surveyed did not think they would use pharmacogenetic data in current practice if such data were available [START_REF] Sauver | Integrating Pharmacogenomics into Clinical Practice: Promise vs Reality[END_REF].

Globally, based to the literature, a consensus came to light regarding the return of SF when patients requested them in teams where clinicians and biologists have validated a procedure to allow such disclosures. However, the existing literature on SF should be taken with caution, since most articles lack a clear definition of SF and the concept of actionability, focused on different populations of respondents, propose different methodologies that cannot be combined, and above all, referred to hypothetical scenarios. Studies conducted by multidisciplinary teams involving patients with access to results are sadly lacking in particular in the medium term after the results have been given. Such studies would feed the debate and make it possible to measure the impact of such findings and their benefit-risk ratio.

ACKNOWLEDGEMENTS: The authors thank the Burgundy Regional Council through the plan d'actions régional pour l'innovation (PARI 2016) and the European Union through the PO FEDER-FSE Bourgogne 2014/2020 programs for their financial support for this project. All types of genetic diseases 94% of patients chose to receive information on actionable secondary findings proposed (92% for carrier status in recessive diseases, 89% for the actionable genes with a predisposition for cancer, 92% for genes with an actionable predisposition for adult-onset diseases, 95% for genes with an actionable predisposition to early-onset diseases).

LEGENDS

Parents of affected children were more likely to want to receive secondary findings for their child (96%, 155/162) than were affected adults for themselves (32/38, 84%) and those with rare diseases, and 92% wanted the results to include predispositions to child-onset diseases (97% if there was possible treatment or prevention, 83% otherwise). 83% wanted to be informed about secondary findings even if they concerned untreatable and fatal diseases; 70% wanted to be informed for variants of unknown significance. 93% of those interviewed believed that their at-risk relatives should be informed about their risk in cases of a serious treatable disease, and 88% in cases of an untreatable disease.

74% believed that the results returned after the death of a patient should be disclosed. The majority of parents preferred to receive all types of result. They all wanted to receive the actionable secondary findings to guide the medical management of their child, but also for the benefits that they themselves could take advantage of. Concerning non-actionable data, 15 parents did not to receive these findings or were reluctant. They expressed a need to think more about this option. The results concerning recessive variants were more mixed; 14 parents preferred to leave the choice to their children when they reached adulthood.

Kleiderman et al., 2014

Returning

work groups of 3 persons each 9 interviews by telephone

The parents appeared enthusiastic about the idea of receiving secondary findings concerning their children. they underlined, however, 4 important points for them:

The right to information: meaning their right as parents to receive secondary findings concerning their children

The benefit-risk ratio of knowing these secondary findings: all of the participants thought that knowing these secondary findings would enable them to better plan the care for their child.

The way the results should be given:

Between the expert and the patient, they asked for true, clear and concise information from a doctor with expertise in genetics.

Between them and their family: they agreed that it was up to them to explain the results to their child and the majority thought it was their responsibility to tell their families about these results

The provision of « support » to accompany patients during the return of secondary findings: this could be done by genetic counsellors, psychologists, support groups. They recognised that the Internet had provided a lot of information but the majority of parents preferred to get information during consultations with a clinician. The interviews were conducted on 4 different occasions: before the before the first consultation with a genetic counsellor, after the first consultation with a genetic counsellor, while waiting for the results, after their return.

Clift et al., 2015

Diversity of opinions on the types of results to return, but consensus on the importance for the patients of taking part in their choice. The majority thought that knowing the results for the secondary findings would improve their quality of life and allow them to prevent or at least prepare for future diseases. The aim of this study was to report the point of view of patients on the consent process and their preferences with regard to the procedure.

Mixed methodology (quantitative + qualitative)

This study identified the principal benefits related to the return of secondary findings. For patients:

-95% (19/20) thought that a predisposition to a treatable disease could be identified -95% (19/20) thought that preventive measures could be implemented -85% (17/20) thought that genetic counselling could be implemented for the descendants -90% (18/20) thought that the pharmacogenetic variants could make it possible to adapt treatments for patients

The majority of patients thought that the consent for HTS should contain information on secondary findings: the possible prevention for certain variants, the possibility to limit the adverse effects of certain drugs for pharmacogenetic variants, the fact that there may have been errors in the interpretation of certain variants, the possibility that certain results could have a negative psychological effect, the possibility of evolution of results with advances in scientific knowledge. The desire to receive secondary findings was high in all groups (80%), but certain persons did not want these results because they feared it would cause anxiety. All preferred consent in 2 steps, and giving consent for secondary findings after a delay of two months.

Kaphingst et al., 2016

Preferences for Return of

Incidental Findings from

Genome Sequencing Among Women Diagnosed with Breast Cancer at a Young Age 60 semi-structured interviewed of women diagnosed with breast cancer at age 40 or younger.

They evaluate patient interest toward receiving 6 categories of incidental findings:

related to risk of a preventable or treatable disease (100% were interested) related to a risk of an unpreventable or untreatable disease (35% were interested) affecting treatment response (100% were interested))

uncertain or unknown significance (VUS) (65% were interested) carrier status (87% were interested)

no health meaning (77% were interested)

When ask to classify in those categories the one that they would choose first, it was "related to risk of a preventable or treatable disease" (83%). If the HTS concerned an adult patient (Ad) or a child patient (Ch): The decision to return the data or not was based on 3 concepts: the quality of the information (corresponding to the scientific validity of the variants identified), the respect of rules and the welfare of the patient. They agreed on the need to provide clear, personalised information to patients.

- 94% (
The researchers implemented various actions to prepare for the discovery of secondary findings. These included: adding specific wording to the informed consent document to clearly indicate how researchers will convey these data;

exploring how the participants should be prepared to make decisions, the way in which they themselves would like to receive these secondary findings;

developing procedures for the disclosure and then individualised follow-up. Aim: examine the perspectives of geneticists with regard to the disclosure of secondary findings.

The problems encountered concerned the information given to patients and the return of secondary findings. Participant underlined their concern about causing psychological harm to patients. There was no consensus about whether a secondary finding had to be clinically significant and/or actionable to be returned to patients. They were in favour of creating guidelines, but only if they were flexible enough to be adapted to individual situations.

Keogh et al., 2014

How do researchers manage genetic results in practice?

The This article showed the considerable problems of diversity, the challenges and the costs encountered in practice, for the return of secondary findings whose utility has been established. The difficulties the most often mentioned were:

the absence of guidelines, the problems in terms of cost and logistics due to the signature of a second consent form, the limited implication of genetic counsellors in certain centres. Aim: provide suggestions to investigators to shape the consent process for the return of secondary findings in research

The majority of researchers and participants raised several problems related to the return of secondary findings: the risks, the benefits, the impact on family members, data security and the return of results in cases of death or loss of contact. All expressed concern that patients would be overwhelmed by this information. The majority of researchers did not want to spend more than 30min returning these results.

This study identified the principal benefits of returning secondary findings.

For researchers:

- The majority wanted to receive information on secondary findings if they were taking part in clinical studies.

94% wanted to receive «information that was important for their health or on the treatment of their disease », 84% « on their risk of being a carrier of an inherited genetic disease » 70% « wanted to receive information not directly concerning their disease ». The participants wanted to be informed about secondary findings in clinical as well as research contexts. They wanted to be given the choice of being informed or not. The potential of personally benefiting from the findings was the principal reason. As well as the medical and scientific aspects, the participants underlined the interest of taking the parent-child relationship and the context into account. They did not want the results to be restricted to variants concerning potentially deadly diseases. Several times, the members of a group used the same arguments to justify opposite opinions.

Mixed methodology (quantitative + qualitative)

Figure 1 :

 1 Figure 1: Data describing the demographic and geographical characteristics of the articles a -Number of articles per year of publication and by type of study. b -Distribution of articles by continent.

Figure 2 :

 2 Figure 2: Number of subjects according to the type of study and to the methodology

  Figure 1a

  were more than 50 years old Survey then 19-item questionnaire The responses were stable over time, 85% declared that they had the right to receive secondary findings discovered through research even for predispositions for incurable diseases. 74% believed that the results must be shared with at-risk members of the family.

  children with cancer split into 4 groups and semi-structured interviews in a control group of 10 patients without cancer to establish recommendations for consent for genetic investigations in children with cancer.

  Ad) and 96% (Ch) were in favour of returning of variants with a predisposition to serious diseases and accessible to treatment -75% (Ad) and 70% (Ch) were in favour of returning of pharmacogenetic variants -57% (Ad) and 40% (Ch) were in favour of returning variants with a predisposition to untreatable serious diseases -73% (Ad) and 40% (Ch) were in favour of returning the researchers feel a strong responsibility to look for meaningful incidental results. But if an incidental result was noted, 78% think they had an obligation to transmit this result. Medical geneticists (57%) were significantly more likely than genomics researchers (18%) to report a feeling of responsibility to examine the data set for incidental clinically relevant findings. Association of Researcher Characteristics with Views on Return of Incidental Findings from Genomic Research Exome sequencing: research 241 included Questionnaire with fixed-response questions and opportunities to enter free-text comments. Aim: see how the characteristics and the experience of researchers were associated with their opinions concerning secondary findings. Researchers with no clinical training, who did not provide care to patients, and who were not involved in returning results were more likely to return secondary findings to patients. Among the researchers who wished to return secondary findings themselves, 48% of those with and 58% of those without clinical training thought that it would be beneficial to return pharmacogenetic data to patients; 51% of those with clinical training wanted to return highpenetrance variants with no possible intervention, versus 65% for those without clinical training; 98% of those with clinical training wanted to return actionable high-penetrance variants, versus 94% for those without clinical training. Gliwa et al., 2016 IRB (institutional review boards) perspectives on obligations to disclose genetic incidental findings to research participants Exome sequencing: research 796 reviewers belonging to the group "Public Responsibility in Medicine and Research" Survey Aim: study the opinions of research ethics committees concerning the return of results and the interpretation of secondary findings 74% of respondents said they had had experience with secondary findings in genetics, but only 47% felt ready to return secondary findings (actionable or not). 96% recognised the right not to be informed. 7% thought that the time and effort devoted to the analysis of these secondary findings was an obstacle to the obligation to return SF to patients. the responses of researchers to a hypothetical discovery of data of unknown significance following HTS for research purposes.

  experience of the multinational Colon Cancer Family Registry Exome sequencing: research 6 researchers representing 6 centres of the « Colon Cancer Family Registry ». Questionnaires: fixed-choice and open questions. Aim: describe the protocols used to return clinically pertinent results, report the use of these results, examine the challenges raised by the return of these results and propose recommendations for future practice.

  The participants expressed interest in receiving findings for severe or very severe diseases with a lifetime risk ≥80%, for which there is an effective treatment, and for heterozygous variants associated with recessive hereditary diseases. However, they expressed no interest in variants exposing them to diseases with no available treatments or lifestyle interventions. Patients' preferences tended towards the possibility of having the choice of what investigations to have public, 98% (4199/4304) wanted to receive actionable secondary findings and 73% (3146/4304) wanted to receive non-actionable secondary findings. to secondary findings that were non-actionable during childhood. 72% wanted to take part in decision-making. 73% thought that the parents of children of less than 12 years old must have access to this information. the opinions of the public on carrying out hypothetical pharmacogenetic tests and on the discovery of secondary findings during these tests.The inhabitants were rather interested in the contribution of pharmacogenetics. 3 of the 4 groups recognised the benefit of knowing the risk of adverse effects of treatments. They thought that the clinician should return these results but they were apprehensive about receiving results concerning predispositions to late-

Table 1a : Studies of opinions centred on patients and classified according to their methodology: quantitative, qualitative or mixed (quantitative + qualitative).

 1a

	Author / Year	Title			Number of subjects studied/Method	Main result
	of publication
					Quantitative methodology
	Shahmirzadi et	Patient decisions for disclosure	Exome sequencing: diagnosis
	al., 2013	of secondary findings among the	200 retrospective analyses of the choices of patients
		first 200 individuals undergoing	for the return of secondary findings classified in 4
		clinical	diagnostic	exome	categories, 162 parents of affected children (mean
		sequencing			age 5 years), 38 affected adults (mean age 38 years).

  wanted access to actionable SF, only 62% when the benefit was not clearly established.

	Gray et al., 2012 81% would have Appelbaum et Attitudes of Patients With Cancer About Personalized Medicine and Somatic Genetic Testing Hypothetical situation 69 patients with colorectal, lung or breast cancer. Median age 59 years (32-86 years) Semi-structured interviews and binary-choice questions Informed Consent for Return of Exome sequencing: research
	al., 2014	Incidental Findings in Genomic	20 patients (mean age 44 years) who had
		Research	undergone WES participated in the study (semi-
			structured interviews).
			254 researchers in genetics, 28 genomic researchers
			(semi-structured interviews)

Table 1b : Studies of opinions centred on healthcare professionals and classified according to the methodology used: quantitative, qualitative or mixed (quantitative + qualitative).

 1b

	Author	/	Title	Number of subjects studied/Method	Main result
	Year	of
	publication
				Quantitative methodology
	Lohn et al.,	Genetics Professionals' Perspectives on Reporting	114 geneticists and 96 genetic counsellors
	2012		Incidental Findings From Clinical Genome-Wide	Survey sent by email.
			Sequencing

  General consensus on the importance of taking the choice of the patient into account but also the evolution of such choices over time.

	Simon et al.,	Informed Consent and Genomic Incidental Findings: IRB	34 Chairs of institutional review boards
	2011		Chair Perspectives	Semi-structured interviews
				Aim: study the opinion of IRB members on the
				return of secondary findings in a context of
				informed consent processes
	Dressler	et	IRB perspectives on the return of individual results from	Exome sequencing: research	Secondary findings in a research context must be returned when they are
	al., 2012		genomic research	31 reviewers	medically actionable, but only if the patient wants to know the result. The
				In-depth interviews	majority of participants were aware of the need to develop guidelines, but did
				Aim: analyse the perspectives and the approaches	not wish this to be done in isolation.
				of IRB with regard to the return of secondary
				findings in a research context.
	Townsend et	''I Want to Know What's in Pandora's Box'': Comparing	Genome sequencing: diagnosis	See Table 1a
	al., 2012		Stakeholder Perspectives on Incidental Findings in Clinical	10 Doctors
			Whole Genomic Sequencing	8 Patients, 10 persons representing society at large.
				Debates following a guide, led by a moderator.
	Downing et	Genetics specialists' perspectives on disclosure of genomic	50 geneticists
	al., 2013		incidental findings in the clinical setting	Structured telephone interviews.

Table 1c : Studies of opinions centred on society at large and classified according to their methodology: quantitative, qualitative or mixed (quantitative + qualitative).

 1c

	Author / Year	Title	Number of subjects	Main result
	of publication		studied/Method
			Quantitative methodology
	Fleming et al.,	Attitudes of the general public towards the	Exome sequencing: research
	2015	disclosure of individual research results and	800 Australians aged 18 years and
		incidental findings from biobank genomic	older, representative of the
		research in Australia	Australian population.
			Mean age 58.2 years
			Telephone survey.

KEYWORDS

Secondary findings, Actionability, Opinion based studies, Literature Review. Prospective study with surveys

The vast majority of patients (≥95%) wanted to know actionable secondary findings whether or not they were related to the disease. 85% of patients also wanted to be informed about predispositions to nonactionable diseases. 99% of the respondents were interested to find out about pharmacogenetic data relative to their cancer and 96% wished to know about pharmacogenetic data not related to the cancer. The authors brought to light 5 themes relative to the return of secondary findings:

Qualitative methodology

-Pre-test information: all of the groups agreed that this step was crucial to give patients an informed choice -The patient's choice: even if the healthcare professionals agreed with the principle of patient's choice, they insisted on the fact that it was necessary to adapt what the patient wanted to know to what the patient was able to understand. For patients, their choice should take precedence of that of the healthcare professional.

-Responsibility: patients and professionals agreed that it was up to the patient to contact the clinician in cases of technological progress.

-The return of secondary findings: the practitioners recognised that the return of these secondary findings took time, and required empathy and expertise in genetics, making it difficult for a non-specialist doctor to return the results. For patients and society at large, the most important was neither experience nor expertise of the doctor, but his/her pedagogy in providing explanations.

-

The impact and implications of secondary findings: the biggest problem for professionals was the anxiety that disclosure of these findings could cause in patients. For the patients and society at large, this anxiety should not be an obstacle to the return of secondary findings; it was up to the patient alone to choose the data he/she wanted to receive and to judge the impact that they would have on him/her. Surveys sent by email and post.

Aim: examine the opinion of doctors with regard to the recommendations of the ACMG >80% thought that the children and their parents had the right to choose to be informed or not.

35% of members of the AAP and of the SOB supported the recommendations of the ACMG compared with 71% for the SGBD. This difference can be explained by the fact that the SOB is essentially against giving access to the results for late-onset diseases affecting children.

30% of each group thought that the parents should not have access to data on adult-onset diseases.

No consensus for reporting mutations if management would only benefit the parents: 34.5% were in favour in the group AAP-SOB vs. 80% of members of the SGBD.