M Courtaudiere
email: melanie.courtaudiere@edu.univ-fcomte.fr

N Andreff

S Koch
email: skoch@chu-besancon.fr

Preliminary Design of Bilio-digestive Anastomosis by Magnetic Compression

Keywords: Magnetic compression, bile duct, duodenum, ferrofluid, obstruction, jaundice

Some experiments have been realized to determine the characteristics of magnets allowing for bilio-digestive anastomosis by magnetic compression using a ferromagnetic gel and a magnet. The attraction distance between the ferromagnetic gel and the magnet was determined experimentally, under several configurations: magnet shape, strength, number and arrangements. Two magnets with a strength of 14 000 Gauss and a diameter of 12,7mm were necessary to attract 0,353mL of ferrofluid contained in a plastic sheath placed at 1cm of the magnet. So the magnet is able to attract the ferrofluid. However, this procedure needs a large powerful magnet which is not compatible for an immediate use in the human body. Nevertheless, this preliminary result opens promising investigation paths.

INTRODUCTION

Currently two treatments are available to treat a biliary obstruction. The first one is a non-invasive treatment: the ERCP (endoscopic retrograde cholangio pancreatography) using a duodenoscope and a guide wire through the obstruction allows to insert a biliary stent and thus to treat the obstruction However if the guide wire can't be inserted through the tumor, the biliary stent cannot be placed and a cholangitis occurs and kills the patient. The second method is the invasive surgery to skirt the obstruction. For instance, duct-to-duct anastomosis by invasive surgery are performed. As a promising alternative to surgery, anastomosis by magnetic compression is currently practiced for intestinal anastomosis [START_REF] Uygun | Magnetic compression gastrostomy in the rat in PEDIATR SURG INT[END_REF][2][3][4] [START_REF] Itoi | Magnetic Compression Duct-to-duct Anastomosis for Biliary Obstruction in a Patient with Living Donor Liver Transplantation in GUT AND LIVER[END_REF] . However, the reported magnetic duct-to-duct compressions seem to require surgical insertion of the magnet placed upstream the obstruction (and may be also a surgical removal) [START_REF] Fan | Sutureless Intestinal Anastomosis with a Novel Device of Magnetic Compression Anastomosis in CHIN MED SCI[END_REF] . To avoid invasive surgery, the creation of an anastomosis by magnetic compression between the blocked bile duct and the duodenum could be the solution. However, the question of inserting a magnet in the bile duct upstream the obstruction without surgery is left unanswered yet.

Our proposal is to investigate the injection of a biocompatible ferromagnetic gel in the biliary duct and then study its attraction by the magnet placed in the duodenum.

METHOD A novel clinical procedure

More precisely, the novel clinical protocol we foresee consists of the injection under EUS (endosonography) of a ferromagnetic gel by a FN with a diameter of 0.9mm (19G) above the obstruction of the bile duct. Then a magnet would be placed in the duodenum with a video endoscope. Removal of the magnet and gel would occur by simple natural elimination in the feces.

Anatomical constraints

Before this protocol can be tested, the choice of materials must be tested in accordance to the anatomical constraints. Ideally, the magnet would be placed through the operator channel, thus limiting the diameter of the magnet to some 4.2 mm. Nevertheless, using a magnet with a somewhat larger diameter could be enabled by holding over the scope with a nylon wire [8] . The common bile duct has an internal diameter included between 3mm and 8 mm in normal state (>10mm after cholecystectomy or in case of obstruction). The volume of ferromagnetic gel that needs to be considered must cope with the bile duct lumen. The bile duct measures about 10cm (hepatic duct and common duct) and the anastomosis can be located only in front of the first duodenum. So the distance between the gel and the duodenum magnet will be less than 1cm. Therefore, we need to determine the attraction strength of the magnets which are necessary to the attraction of a ferromagnetic gel localized at 1cm. This is coherent with all the published studies, where the magnets were placed very close from one to another, with a maximal attraction distance between the magnets lower than 20mm. Moreover, the diameter of the different magnets varied between 2mm and 5mm. However, this does not tell much on how this would extend to bilio-digestive anastomosis. Indeed, replacing one magnet by ferromagnetic gel in order to avoid a surgical insertion in the bile duct completely changes the magnetic characteristics of the set-up. The use of two kinds of magnets is reported:

magnets in Samarium-Cobalt magnets in Neodymium-Iron-Bore Both kinds are rare-earth magnets, identical in term of magnetic flow density. Each of them is appropriated for the anastomosis by magnetic compression but the obstructive strength of the magnets in Samarium-Cobalt is higher and so better. The magnets used in previous studies had a strength included between 3000 Gauss (~0.3 Tesla) and 3700 Gauss (~0.37 Tesla). Furthermore, to ensure their biocompatibility, many covering materials can be used, such as chromium, titanium nitride or polycarbonate. Different magnets shapes were used: spherical, cylindrical and ring-shaped. The spherical magnets show a smooth and round surface, thus they can move and roll up.

Experimental set-up and method

The experiments that were set-up consists of the use of off-the-shelf ferrofluid and magnets in various shapes, strengths and configurations. A ferrofluid is a colloidal solution of ferromagnetic nanoparticles, having a size of 10nm, put in solution in a solvent or in water. The liquid becomes magnetic when an external magnetic field is applied however it keeps its stability colloidal. When the magnetic field is adequate, the ferrofluid is disheveled of points few rigid whose the topology varies in function of the parameters of the field. They are, most frequently, composed of nanoparticles of magnetite (Fe3O4) or of magnesite (γ-Fe2O3), which are both iron oxide. The ferrofluids have an application in many fields especially in medicine and biology (trying of biological groups, contrast substance, cancerology).

The magnets used were made in neodymium iron bore and had the following characteristics, determined thanks to the state of art: The magnets were placed in 3 different configurations: horizontal, vertical and « loop » when it was possible. They were illustrated thereafter: The magnets were used alone, by two or by three. Their simultaneous utilization was always made with magnets which had the same characteristics (there was no mix between the different types of magnets).

To measure the necessary distance for the attraction between the ferrofluid and the magnet, the following assembly was realized thanks to a gauge (Fig. 7). The gauge has a magnetic fixation system so the magnet(s) were simply fixed by magnetization. To represent the bile duct of the patient it was necessary to use a material which had the same characteristics as the human body. A plastic sheath possessed the matching properties. The ferromagnetic liquid was inserted with a syringe B-D Plastipak (capacity of 1mL) and a needle with a diameter of 0,45mm. The plastic sheath had a diameter of 3mm and a length of 5cm. The quantity of ferrofluid inserted was:

The needle was made of stainless steel which was not magnetic so there was not interaction with the ferrofluid. The experimental protocol was to place the sheath on the gauge base with the magnets far enough from it. Then, the magnets were slowly descended until the sheath was lifted up in the air by the magnetic attraction. The acquisition of the movement was made by a camera Mikrotron EoSens 4 CXP with a 25mm Computar objective. When the acquisition was finished, the attraction distance could be calculated. It corresponded at the distance between the base and the end of the plastic sheath, once it had been attracted by the magnet. The distance was directly legible on the screen thanks to the graduation of the gauge (Fig. 9). To increase the reliability of the measures, each were realized 5 times, then the mean of the results was calculated.

RESULTS

The results are assembled in the following table. For a single spherical magnet, the horizontal and the vertical measure with one magnet is the same, so it appears only once in the table. The measures with the cylindrical magnets have been realized on the "short side". The difference of attraction between the horizontal and the vertical position for the same number of magnets was very low.

With the « loop » configuration the attraction length decreased of 50%. This configuration was not efficient, so it will be important to avoid this positioning for the magnets when they will be inserted in the human body.

For the same type of magnet, the attraction distance fluctuated in function of the number of magnet, sometimes the attraction distance is higher with 2 magnets than with 3. This fluctuation was caused by the positioning of the magnets one over the other.

Indeed the magnetization between them could be in different directions, thus it influenced the total properties of the magnet. These results showed the variability of the attraction in function of the positioning of the magnets. When they will be inserted in the body it will be necessary to care of their orientation. The utilization of several magnets increased the attraction distance. This increasing stays low. Moreover for the patient, the utilization of many magnets is not recommended for safety reasons (e.g. undesired magnetic compression in the small bowel), unless a specific device allows to keep them together.

DISCUSSION

The bile duct and the duodenum are separated by a distance of 1cm, so an attraction distance higher or equal is necessary. However, the weight of the sheath was measured to 1g, which means that the attraction force is only 10mN at 1cm distance. This might not be enough for starting the compression. Moreover, only the utilization of many spherical magnets with a diameter of 12.7mm and an afterglow of 14 000 Gauss (1.4 Tesla) allows to achieve this value. The diameter of such a magnet is not perfectly adapted for being used in the body because it is both heavy and yet still small enough to pass into the small bowel. The migration of this magnet should be avoided.

The alternative we will study now it the use of a set of weaker magnets (Fig. 11), attached together onto a muco-adhesive patch, in order to increase both the biocompatibility and efficiency of the device. The patch would be inserted in the duodenum thanks to the operator channel of the endoscope, then it would be placed on the duodenum wall thanks to a self-expandable fully covered metallic stent.

To avoid the attraction of the magnets when the patch is in the operator canal and to allow its extension, the magnets should be oriented in the same direction of polarity (all the faces are on the same side). To permit the adhesion of the patch on the duodenum wall, the patch could be covered with a muco-adhesive film. Finally, the ferrofluid could be replaced by a magneto-rheologic fluid. Such fluids are made of lager ferromagnetic particles and thus have better magnetic properties. They are considered to have the drawback of sedimenting, contrary to ferrofluids where nanoparticles stay apart. Nevertheless, this might not necessarily be a concern in our case, if properly handled, because it would improve the removal of all magnetic material from the body at the end of the treatment.

Figure 1 :

 1 Figure 1: Anatomic diagram of the gallbladder and the duodenum.

Figure 2 :

 2 Figure 2: Anatomic diagram of the biliary duct and the duodenum + obstruction and considered treatment.

Figure 3 :

 3 Figure 3: Magnets used in the experiments.

Figure 4 :

 4 Figure 4: The 3 magnets in position horizontal.

Figure 5 :

 5 Figure 5: The 3 magnets in position vertical.

Figure 6 :

 6 Figure 6: The 3 magnets in position "loop".

Figure 7 :

 7 Figure 7: Experimental set-up.

Figure 8 :

 8 Figure 8: Plastic sheath of 3mm filled with ferrofluid.

Figure 9 :

 9 Figure 9: Result of an acquisition.

Figure 10 :

 10 Figure 10: The two possible positions for the cylindrical magnets.

Figure 11 :

 11 Figure 11: The medical device developed.

Table 1 :

 1 The different magnets used in the experiments

	Magnet	Diameter (mm)	Afterglow (Gauss)	Magnetization
	Sphere 1	12,7	14 000	N42
	Sphere 2	8	12 400	N38
	Cylinder 1	5	13 500	N45
	Sphere1		Sphere 2	Cylinder 1

Table 2 :

 2 The results of the experiments with the 3 magnets.

			Attractio
	Configurations	Number of magnets	n distance
			(cm)
		1 (diameter 12,7mm)
	Horizontal	1 magnet	0,98
		2 magnets	0,98
		3 magnets	1,08
	Vertical	2 magnets	1,04
		3 magnets	1
	« Loop »	3 magnets	0,56
		2 (diameter 8 mm)
	Horizontal	1 magnet	0,64
		2 magnets	0,44

ACKNOWLEDGEMENTS

This work partly supported by the French ANR Labex ACTION (contract "ANR-11-LABX-01-01").


Ivana Roche
email: ivana.roche@inist.fr

Dominique Besagni
email: dominique.besagni@inist.fr

Claire François
email: claire.francois@inist.fr

Marianne Hörlesberger
email: marianne.hoerlesberger@ait.ac.at

Edgar Schiebel
email: edgar.schiebel@ait.ac.at

Dirk Holste
email: dirk.holste@ait.ac.at

Measuring interdisciplinarity of research grant applications. An indicator developed to model this selection criterion in the ERC's peer-review process

Introduction

Scientific experts having to evaluate the interest of research projects submitted for grant to the European Research Council (ERC) must lean on the four "frontier research" criteria among which: "… it pursues questions irrespective of established disciplinary boundaries, involves multi-, inter-or trans-disciplinary research that brings together researchers from different disciplinary backgrounds, with different theoretical and conceptual approaches, techniques, methodologies and instrumentation, perhaps even different goals and motivations", according to the EC's High Level Expert Group report (2005) assertions and that we defined in our DBF project (Hörlesbergeret al., 2013) as interdisciplinarity.

Background

Interdisciplinarity is used as a proxy to infer selfconsistently the presence and proportions of characteristic terminology associated with individual ERC panels, thereby revealing the intra or inter-panel character of a proposal. It was built upon a previously successfully tested approach that the frequency of occurrence and distribution of research field specific keywords can classify and characterize research fields. While the core of the approach has been retained, the computation has been adopted and fine-tuned to the grant scheme under study. The underlying basic hypothesis is that the larger the proportion of inter-panel keywords, the more interdisciplinary is the proposal. To this end, each keyword is labeled according to its statistical frequency across all panels, filters are applied to distinguish relevant from irrelevant keywords, and the tallying of keywords with their assigned panels is assessed to classify each proposal with respect to its share of inter-panel keywords.

Data and Methodology

Primary data are directly obtained from the documents supplied by the researcher. Each project proposal is indexed by keywords (KW) extracted from its textual information. After validation, we apply the diffusion model approach. First, the socalled Home Panel (H-P) terms are defined. We assumed KW which are specific for a panel occurred with a higher probability in that panel rather than in others. This probability is defined by the frequency of one KW in a panel divided by the number of proposals in this panel, namely the relative term frequency (rtf KW/panel ). Then, for a KW, we calculate its rtf KW/panel in each panel and the panel with the highest probability is declared to be its H-P. So, we obtain for each H-P the list of its H-P terms. Therefore the complete terminology associated to a panel consists of the union of its H-P term list and the set of terms imported from the other panels and we refer to as "abroad terms". In order to make the lists of H-P terms more consistent, we use the Gini index, a measure of the inequality of a distribution, varying from 0 to 1, a value of 0 expressing total equality and a value of 1 maximal inequality. It is commonly used in economics as a measure of inequality of the income or wealth. It is, in this study, employed as a measure of the dispersion of a KW. A Gini index of 1 tells us that the KW is very specifically limited to only one panel. Conversely, a Gini index equal to 0 means a completely uniform distribution and indicates that the term occurs in all the considered panels. We define a cut-off threshold to discard KW which dispersion will be considered excessive. Finally, the tallying of KW with their newly assigned H-P is assessed to calculate an index for the share of abroad terms in the list of KW representing the content of each project proposal. The underlying assumption is that the larger the proportion of abroad terms, the more interdisciplinary a project proposal is.

Figure 1.Methodological schema of the evaluation process of the interdisciplinarity of research grant applications

The Case Study We applied our methodology to a case study coming from project proposals submitted to the ERC 2009 Starting Grant Call.

Table 1. Distribution of successful and nonsuccessful proposals submitted to the ERC 2009

Starting Grant Call

Among the 19 ERC panels representing Life Sciences (LS) and Mathematics, Physics, Chemistry, Engineering & Earth Sciences (PE), we chose 6 panels with a balance between LS and PE as well as between basic and applied fields.

The analysis shows the results for ERC panel PE1 (Mathematics and Mathematical foundations) which received 39 submissions, 11 of them successful. The value of Interdisciplinarity in this example is the share of abroad terms calculated with a Gini index cut-off threshold of 0.1.

In the DBF project, we introduced a statistical discrete choice model (DCM) to estimate the decision probability for a proposal to be accepted on the basis of measured attributes of "frontier research" (Scherngellet al., 2013).

We studied in a proof-of-concept approach the influence of those attributes on the success probability and conducted an initial analysis of the ex-post comparison between the indicator-based scientometric evaluation and the empirical peerreview process.Interdisciplinarity is one of the five indicators we developed in the context of this project and it is an element that proves to influence significantly the selection probability of the projects.

The results obtained with Interdisciplinarity are encouraging and we are experimenting with it on a different dataset from e-Corda. We also have to introduce another dimension to that indicator: the diversity of the sources (H-P) of abroad terms, on top of just counting them.

Acknowledgments

Work accomplished in the context of the DBF (Development and Verification of a Bibliometric Model for the Identification of Frontier Research) project within the European Research Council's CSA of the EU's 7 th Framework Programme.


Jean-Charles Lamirel

Pascal Cuxac

Kafil Hajlaoui

Aneesh Sreevallabh Chivukula

A new feature selection and feature contrasting approach based on quality metric: application to efficient classification of complex textual data

Feature maximization is a cluster quality metric which favors clusters with maximum feature representation as regard to their associated data. This metric has already been successfully exploited, altogether, for defining unbiased clustering quality indexes, for efficient cluster labeling, as well as for substituting to distance in the clustering process, like in the IGNGF incremental clustering method. In this paper we go one step further showing that a straightforward adaptation of such metric can provide a highly efficient feature selection and feature contrasting model in the context of supervised classification. We more especially show that this technique can enhance the performance of classification methods whilst very significantly outperforming (+80%) the state-of-the art variable selection techniques in the case of the classification of unbalanced, highly multidimensional and noisy textual data, with a high degree of similarity between the classes. Our experimental dataset is a reference dataset of 7000 publications related to patents classes issued from a reference classification in the domain of pharmacology.

Introduction

Since the 1990s, advances in computing and storage capacity allow the manipulation of very large data. Whether in bio-informatics or in text mining, it is not uncommon to have description space of several thousand or even tens of thousands of variables. One might think that classification algorithms are more efficient if there are a large number of variables. However, the situation is not as simple as this. The first problem that arises is the increase in computation time. Moreover, the fact that a significant number of variables are redundant or irrelevant to the task of classification significantly perturbs the operation of the classifiers. In addition, as soon as most learning algorithms exploit probabilities, probability distributions can be difficult to estimate in the case of the presence of a very high number of variables. The integration of a variable selection process in the framework of the classification of high dimensional data becomes thus a central challenge.

In the literature, three types of approaches for variable selection are mainly proposed: the integrated (embedded) approaches, the "wrapper" methods and the filter approaches. An exhaustive overview of the state-of-the-art techniques in this domain has been achieved by many authors, like Ladha The integrated (embedded) approaches incorporate the selection of the variables in the learning process [BRE 84]. The most popular methods of this category are the SVM based methods and the neural based methods. SVM-EFR (Recursive Feature Elimination for Support Vector Machines) [ GUY 02] is an integrated process that performs the selection of variables an iterative basis using a SVM classifier. The process starts with the complete feature set and remove the variables given as the least important by the SVM. The original version uses a linear kernel. However, some extension using non-linear kernels have been proposed to consider potential nonlinear dependencies between variables. In an alternative way, the basic idea of the approaches of the FS-P (Feature Selection-Perceptron) family is to perform a supervised learning based on a perceptron neural model and to exploit the resulting interconnection weights between neurons as indicators of the feature that may be relevant and provide a ranking [MEJ 06].

On their own side, "wrapper" methods explicitly use a performance criterion for searching a subset of relevant predictors. More often it's error rate (but this can be a prediction cost or the area under the ROC curve). As an example, the WrapperSubsetEval method evaluates the attribute sets using a learning approach. Cross-Validation is used to estimate the accuracy of the learning for a given set of attributes. The algorithm starts with the empty set of attributes and continues until adding attributes does not improve performance [WIT 05].

Forman presents a remarkable work of methods comparison in [FOR 03]. As other similar works, this comparison clearly highlights that, disregarding of their efficiency, one of the main drawbacks of embedded and of the wrapper methods is that they are very computationally intensive. This prohibits their use in the case of highly multidimensional data description space. A potential alternative is thus to exploit filter-based methods in such context.

Filter approaches are selection procedures that are used prior and independently to the learning algorithm. They are based on statistical tests. They are thus lighter in terms of computation time than the other approaches and the obtained features can generally be ranked regarding to the tests' results.

The Chi-square method exploits a usual statistical test that measures the discrepancy to an expected distribution assuming that a variable is independent of a class label. Like any statistical test, he is known to have erratic behavior for very low expected frequencies (which is common case in text classification) [LAD 11].

The information gain is also one of the most common methods of evaluation of the attributes. This univariate filter provides an ordered classification of all variables. Based on to this approach, selected variables are those who obtain a positive value of information gain [HAL 99b].

In the MIFS (Mutual Information Feature Selection) method, a variable X is added to the subset M (of cardinality m) of already selected variables if it maximizes the quantity:

, | = , - * , ∈
Thus, a variable is considered to be interesting if its link with the target Y surpasses his average connection with already selected predictors. The method takes into account both the relevance and redundancy. The search stops when the best variable is X such I Y, X In this paper, we show that, despite of their diversity, all the existing filter-based approaches fail to successfully solve the variable selection task in the case they are faced with highly unbalanced, highly multidimensional and noisy textual data, with a high degree of similarity between the classes. We thus propose a new filter-based variable selection approach which relies on the exploitation of a class quality measure based on a specific feature maximization metric. Such metric already demonstrated high potential in the framework of unsupervised learning.

The paper is structured as follows. The first section presents the feature maximization principle along with the new proposed technique. The second section describes our dataset and our experiment which is performed experimental is a reference dataset of 7000 publications related to patents classes issued from a reference classification in the domain of pharmacology. The last section draws our conclusion and our perspectives.

Feature maximization for variable selection

Feature maximization metric principles in unsupervised learning

Feature maximization is an unbiased cluster quality metrics that exploits the properties of the data associated to each cluster without prior consideration of clusters profiles. This metrics has been initially proposed in [LAM 04]. Its main advantage is to be independent altogether of the clustering methods and of their operating mode. Whenever it is used during the clustering process, it can substitute to distance during that process [LAM 11b]. In a complementary way, whenever it is used after learning, it can be exploited to set up overall clustering quality indexes [LAM 10][GHR 10] or for cluster labeling [LAM 08].

Let us consider a set of clusters C resulting from a clustering method applied on a set of data D represented with a set of descriptive features F, feature maximization is a metric which favors clusters with maximum Feature F-measure. The Feature F-measure of a feature f associated to a cluster c is defined as the harmonic mean of Feature Recall and Feature Precision indexes which in turn are defined as:

= ∑ ! ∈" ∑ ∑ ! ∈" # " # ∈$ , = ∑ ! ∈" ∑ ! # ! # ∈% " , ∈" (1) = 2 ' () " * * (+ " * () " * ,(+ " * - (2)
where . / * represents the weight of the feature f for data d and F c represent the set of features occurring in the data associated to the cluster c.

An important application of the feature maximization metric is related to the estimation of the overall clustering quality. For that purpose, averaged Macro-Recall (MR) and Macro-Precision (MP) indexes can be directly derived from the former indexes.

They are expressed as:

= 0 |1| ∑ 0 |( " | ∑ *2( " 21 , = 0 |1| ∑ 0 |( " | ∑ *2( " 21 3
Macro-Recall and Macro-Precision indexes have opposite behaviors according to the number of clusters. Thus, these indexes permit to estimate in a global way an optimal number of clusters for a given method and a given dataset. The best data partition, or clustering result, is in this case the one which minimizes the difference between their values [LAM 04]. Conversely to classical distance-based indexes, they are independent of the clustering process. Moreover, it has been demonstrated in [LAM 11] that straightforward adaptations of these indexes permits to detect degenerated clustering results, whenever those jointly include a small number of heterogeneous or "garbage" clusters with large size and a big number of "chunk" clusters with very small size.

Another As regards to this approach, a feature is then said to be maximal or prevalent for a given cluster iff its Feature F-measure is higher for that cluster than for any other cluster. Thus the set L c of prevalent features of a cluster c can be defined as:

4 = 5 ∈ 1 | = 67 # ∈1 # 8 4
Whenever it has been exploited in combination with hypertree representation, this technique has highlighted promising results, as compared to the state-of-the-art labeling techniques, like Chi-square labeling, for synthetizing complex clustering output issued from the management of highly multidimensional data [LAM 08]. Additionally, the combination of this technique with unsupervised Bayesian reasoning resulted in the proposal of the first parameter-free fully unsupervised approach for analyzing the textual information evolving over time [LAM 10b].

Exhaustive experiments on large reference datasets of bibliographic records have shown that the approach is reliable and likely to produce accurate and meaningful results for diachronic scientometrics studies [LAM 12].

Last but not least, a central application of feature maximization metric is related to incremental clustering. The IGNGF (Incremental Neural Gas with Feature Maximization) clustering method is a neural-based parameter-free incremental clustering algorithm that substitutes feature maximization to usual distance in the clustering process. Thanks to this approach, the IGNGF clustering process is roughly the following. During learning, an incoming data point d is temporary added to every existing cluster, its feature profile is updated (i.e. each cluster is associated with its maximal features) and its average Feature F-measure is computed. Then the winning cluster is the cluster which maximizes the Kappa criteria given by:

: ; = ∆ * | ∩ / | - >? @ABC D,/ EFAGHC (5)
where ∆ represents the gain in Feature F-measure for the new cluster and ∩ / are the features shared by cluster c and the data point d. This way, those clusters are preferred which share more features with the new data point and clusters which don't have any common feature with the data point are ignored. The gain in Feature F-measure multiplied by the number of shared features can be optionally adjusted by the Euclidean distance of the new data point d to the cluster centroid vector ; D. Clusters with negative Kappa score are ignored. The data point is then added to the cluster c with maximal Kappa and Hebbian connections between winner and its neighbors are updated. If not such cluster is found, a new cluster is created.

The IGNGF method was shown to outperform other usual neural and non neural methods for clustering tasks on relatively clean data, and especially if said data are sparse and/or highly multidimensional [Lam 11]. The first applications of the IGNF method for clustering of textual data revealed very promising results. Especially, this method was exploited for the automatic classification of the French verbs using syntactic and semantic features issued from several reference lexicons. The method showed significantly better performance (+20%) than the best state-of-the-art methods of the field, including the reference methods based on spectral clustering [FAL 12]. In the context of the websites' classification, it has been also shown that the IGNGF method allowed, in an unattended way, to obtain better results (in terms of sensibility and purity) than those provided by the supervised methods this by automatically isolating latent, not originally labeled, classes [LAM 12b].

Adaptation of feature maximization metric for feature selection in supervised learning

Taking into consideration the basic definition of feature maximization metric presented in the former section, its exploitation for the task of feature selection in the context of supervised learning becomes a straightforward process, as soon as this generic metric can apply on data associated to a class as well as to those associated to a cluster. The feature maximization-based selection process can thus be defined as a class-based process in which a class feature is characterized using both its capacity to discriminate a given class from the others ( index and its capacity to accurately represent the class data (FR P f index . Finally, the set of all the selected features S C is the subset of F defined as:

R 1 = ⋃ R ∈1 (7)
Features that are judged relevant for a given class are the features whose representation is altogether better than their average representation in all the classes including those features and better than the average representation of all the features, as regard to the feature F-measure metric.

In the specific framework of the feature maximization process, a contrast enhancement step can be exploited complementary to the former feature selection step. The role of this step is to fit the description of each data to the specific characteristic of its associated class which have been formerly highlighted by the feature selection step. In the case of our metric, it consists in modifying the weighting scheme of the data specifically to each class by taking into consideration the information gain provided by the Feature F-measures of the features, locally to that class. This step more precisely operates as described in Algorithm 1.

Thanks to the former strategy, the information gain provided by a feature in a given class is proportional to the ratio between the value of the Feature F-measure of this feature in the class and the average value of the Feature F-measure of the said feature on all the partition. For a given data and a given feature describing this data, the resulting gain acts a contrast weight factorizing with any existing feature weight that can be issued from data preprocessing. Moreover, each data description can be optionally reduced to the features which are characteristic of its associated class. If it is present, normalization of the data description is discarded by those operations. Optional renormalization can also be performed in the curse of the algorithm.

Algorithm

Experimental data and results

Data extraction and preprocessing

The data is a collection of patent documents related to pharmacology domain. The bibliographic citations in the patents are extracted from the Medline database1 . The source data contains 6387 patents in XML format, grouped into 15 subclasses of the A61K class (medical preparation). 25887 citations have been extracted from 6387 patents [HAL 12]. Then the Medline database is queried with extracted citations for related scientific articles. The querying gives 7501 articles with 90% recall. Each article is then labeled by the class code of the citing patent. The set of labeled articles represents the final document set on which the training is performed. The final document set is unbalanced, with smallest class containing 22 articles (A61K41 class) and largest class containing 2500 articles (A61K31 class). Inter-class similarity computed using cosine correlation indicates that more than 70% of classes' couples have a similarity between 0.5 and 0.9. Thus the ability of any classification model to precisely detect the right class is curtailed. A common solution to deal with unbalance in dataset is undersampling majority classes and oversampling minority classes. However sampling that introduces redundancy in dataset does not improve the performance in this dataset, as it has been shown in [HAL 12]. So that bootstrapping of train and test data may solve problems of classification sensibility, stability, scalability and dimensionality but does not improve accuracy computation over the sampled correlations. Conversely, pruning irrelevant features and contrasting the relevant ones has we propose hereafter seems thus to be a good alternative.

The document set is converted to a bag of words model [SAL 71] using the TreeTagger tool [SCH 94] developed by the Institute for Computational Linguistics of the University of Stuttgart. This tool is both a lemmatizer and a tagger. A lemmatizer associates a lemma, or a syntactic root, to each word in the text and a tagger automatically annotates text with morpho-syntactic information. In our case, the documents are firstly lemmatized and the tagging process is performed on lemmatized items (in the case when a word is unknown to the lemmatizer, its original form is conserved). The punctuation signs and the numbers identified by the tagger are deleted. The feature selection according to grammatical categories allows identifying salient features for the documents classification according to document types or opinions.

Every document is represented as a term vector filled with keyword frequencies. The description space generated by the tagger has dimensionality 31214. To reduce noise generated by the TreeTager tool, a frequency threshold of 45 (i.e. an average threshold of 3/class) is applied on the extracted descriptors. It resulted in a thresholded description space of dimensionality 1804. The whole text collection is then represented as a (N+1) x J matrix where J is number of articles in the collection in a N-dimensional space. Each line j of this matrix is an N-dimensional bag of words vector for article j, plus its class label. The Term Frequency-Inverse Document Frequency (TF-IDF) weighting scheme [SAL 88] gives a sparse matrix representation of the text collection.

Testing process

To perform our experiments we firstly take into consideration different classification algorithms which are implemented in the Weka toolkit: Most of these algorithms are general purpose classification algorithms, except from DMNBtext which is a Discriminative Multinomial Naïve Bayes classifier especially developed for text classification. As compared to classical Multinomial Naïve Bayes classifier this algorithm cumulate the computational efficiency of Naïve Bayes approaches and the accuracy of Discriminating approaches by taking into account both the likelihood and the classification objectives during the frequency counting. Other general purpose algorithms whose accuracy has especially been reported for text classification are SMO and KNN [ZHA 02]. Default parameters are used when executing these algorithms, except for KNN for which the number of neighbors is optimized.

To more especially focus on the efficiency testing of the variable (i.e. feature) selection approaches including our new proposal. We include in our test a panel of filter-based approaches which are computationally tractable with high dimensional data, making again use of their Weka toolkit implementation. The panel of tested methods includes: -Weka's Chi-square method: weka.attributeSelection.ChiSquaredAttributeEval [LAD 11] ; -Weka's Information gain method: weka.attributeSelection.InfoGainAttributeEval [HAL 99b] ; -Weka's CBF method: weka.attributeSelection.ConsistencySubsetEval [DAS 03] ; -Weka's SU method: weka.attributeSelection.SymmetricalUncertaintyAttributeEval [YUL 03] ; -Weka's ReliefF algorithm: weka.attributeSelection.ReliefFAttributeEval [KIR 92] ; -Weka's Principal Component Analysis: weka.attributeSelection.PrincipalComponents [PER 01] ; -Feature maximization based method including contrasting (our current proposal).

Defaults parameters are also used for most this methods, except for PCA for which the percentage of explained variance is tuned for optimization.

We first experiment the methods separately. In a second phase we combine the feature selection provided by the method with the feature contrasting technique we have proposed. 10-fold cross validation is used on all our experiments.

Results

The different results are reported in tables 1 to 5 and in figure 1. Tables present standard performance measures (True Positive, False Positive, Precision, Recall, F-measure and ROC) weighted and averaged over all classes. For each table, and each combination of selection and classification methods, a performance increase indicator is computed using the DMNBtext True Positive results on the original data as the reference. Finally, as soon as the results are identical for Chi-square, Information Gain and Symmetrical Uncertainty, they are thus reported only once in the tables as Chi-square results.

Table 1 highlights that performance of all classification methods are low on the considered dataset if no feature selection process is performed. They also confirm the superiority of the DMNBtext, SMO and KNN methods on the two other tree-based methods in that context. Additionally, DMNBtext provides the best overall performance in terms of discrimination as it is illustrated by its highest ROC value.

Whenever a usual feature selection process is performed in combination with the best method, that is DMNBtext method, the exploitation of the usual feature selection strategies slightly alters the quality of the results, instead of bringing up an added value, as it is shown in table 2. Alternatively, same table highlights that even if the feature reduction effect is less with the F-max selection method, its combination with F-max data description contrasting boosts the performance of the method (+81%), leading to excellent classification results (Accuracy of 0.96) in a very complex classification context.

Even if the benefit of the former use of F-max selection and contrasting approach is very high with the DMNBtext method, table 3 shows that the added value provided by this preprocessing approach also concerns, to a lesser extent, all the other classifiers, leading to an average increase of their performance of 45% as compared to the reference result. Another interesting phenomenon that can be observed is that, with such help, tree-based classification methods significantly, and unusually, outperform the KNN method on text.

The results presented in table 4 more specifically illustrates the efficiency of the F-max contrasting procedure that acts on the data descriptions. In the experiments related to that table, F-max contrasting is performed individually on the features extracted by each selection method and, in a second step, DMNBtext classifier is applied on the resulting contrasted data (see algo 1). The results show that, whatever is the kind of feature selection technique that is used, resulting classification performance is enhanced whenever is a former step of F-max data description contrasting is performed. The average performance increase is 44%. Table 5 and figure 1 illustrate the capabilities of the F-max approach to efficiently cope with the class imbalance problem. Hence, examination of the confusion matrices of figure 1 shows that the data attraction effect of the majority class that occurs at a high level in the case of the exploitation of the original data (figure 1(a)) is quite completely overcome whenever the F-max approach is exploited (figure 1(b)). The capability of the approach to correct class imbalance is also clearly highlighted by the homogeneous distribution of the selected variables in the classes it provides, despite of their very different sizes (table 5).

Conclusion

Feature maximization is a cluster quality metric which favors clusters with maximum feature representation as regard to their associated data. In this paper, we have proposed a straightforward adaptation of such metric, which has already demonstrated several generic advantages in the framework of unsupervised learning, to the context of supervised classification. Our main goal was to build up an efficient feature selection and feature contrasting model that could overcome the usual problems arising in the supervised classification of large volume of data, and more especially in that of large full text data. These problems relate to classes' imbalance, high dimensionality, noise, and high degree of similarity between classes. Through our experiments on a large dataset constituted of bibliographical records extracted from a patents' classification, we more especially showed that our approach can naturally cope with the said handicaps. Hence, in such context, whereas the state-of-the-art variable selection techniques remain inoperative, feature maximization-based variable selection and contrasting can very significantly enhance the performance of classification methods (+80%). Another important advantage of this technique is that it is a parameter-free approach and it can thus be used in a larger scope, like in the one of semi-supervised learning.

  et al. [LAD 11], Bolón-Canedo et al. [BOL 12] Guyon et al. [GUY 03] or Daviet [DAV 09]. We thus only provide hereafter a rapid overview of existing approaches and related methods.

  The set S c of features that are characteristic of a given class c belonging to an overall class set C results in: and C /f represent the restriction of the set C to the classes in which the feature f is represented.

-

  Weka's Decision Tree algorithm: weka.classifiers.trees.J48 [QUI 93] ; -Weka's Random Forest algorithm: weka.classifiers.trees.RandomForest [BRE 01] ; -Weka's KNN algorithm: weka.classifiers.lazy.IBk [AHA 91] ; -Weka's Bayesian Network algorithm: weka.classifiers.bayes.DMNBtext [SU 08] ; -Weka's SVM algorithm: weka.classifiers.functions.SMO [SCH 98], [KEE 01].

Figure 1 :

 1 Figure 1: confusion matrices of the optimal results -before (1) and after (2) feature selection (Classification: DMNBtext -Feature selection: F-max + Contrast).

  important application of feature maximization metric is related to clusters' labeling whose role is to highlight the prevalent features of the clusters associated to a clustering model at a given time. Labeling can thus be used altogether for visualizing or synthesizing clustering results and for optimizing the learning process of a clustering method [ATT 06]. It can rely on endogenous data properties or on exogenous ones. Endogenous data properties represent the ones being used during the clustering process. Exogenous data properties represent either complementary properties or specific validation properties. Exploiting feature maximization metric for cluster labeling results in a parameter-free labeling technique [LAM 08].

Table 1 :

 1 classification results on initial data.

		TP	FP	Precision Recall	F-	ROC	TP
		Rate	Rate			measure		Incr. /Ref
	J48	0.42	0.16	0.40	0.42	0.40	0.63	-23%
	Random Forest	0.45	0.23	0.46	0.45	0.38	0.72	-17%
	SMO	0.54	0.14	0.53	0.54	0.52	0.80	0%
	DMNBtext	0.54	0.15	0.53	0.54	0.50	0.82	0% (Ref)
	KNN (k=3)	0.53	0.16	0.53	0.53	0.51	0.77	-2%

Table 2 :

 2 classification results after feature selection (DMNBtext classification).

		TP Rate	FP Rate	Precision Recall	F-measure	ROC	Nbr. of select. features	TP Incr./Ref
	ChiSquare (+..)	0.52	0.17	0.51	0.52	0.47	0.80	282	-4%
	CBF	0.47	0.21	0.44	0.47	0.41	0.75	37	-13%
	PCA (50% vr.)	0.47	0.18	0.47	0.47	0.44	0.77	483	-13%
	Relief	0.52	0.16	0.53	0.52	0.48	0.81	937	-4%
	F-max sel. + contrast	0.96	0.01	0.96	0.96	0.96	0.999	1419	+81%
		TP Rate	FP Rate	Precision Recall	F-measure	ROC	TP Incr./Ref
	J48	0.80	0.05	0.79	0.80	0.79	0.92	+48%
	Random Forest	0.76	0.09	0.79	0.76	0.73	0.96	+40%
	SMO	0.92	0.03	0.92	0.92	0.91	0.98	+70%
	DMNBtext	0.96	0.01	0.96	0.96	0.96	0.999	+81%
	KNN (k=3)	0.66	0.14	0.71	0.66	0.63	0.85	+22%

Table 3 :

 3 classification results after F-max + contrast feature selection (all classification methods).

		TP Rate	FP Rate	Precision Recall	F-measure	ROC	Nbr. select. features	TP Incr./Ref
	ChiSquare (+..)	0.79	0.08	0.82	0.79	0.78	0.98	282	+46%
	CBF	0.63	0.15	0.69	0.63	0.59	0.90	37	+16%
	PCA (50% vr.)	0.71	0.11	0.73	0.71	0.67	0.53	483	+31%
	Relief	0.79	0.08	0.81	0.79	0.78	0.98	937	+46%
	F-max sel. + contrast	0.96	0.01	0.96	0.96	0.96	0.999	1419	+81%

Table 3 :

 3 classification results after feature selection by all methods and F-max contrasting (DMNBtext classification).

	Class label	Class size	Selected features	TP Rate
	a61k31	2533	223	0.999
	a61k33	60	276	0.77
	a61k35	459	262	0.97
	a61k36	212	278	0.89
	a61k38	1110	237	0.99
	a61k39	1141	240	0.99
	a61k41	22	225	0.14
	a61k45	304	275	0.83
	a61k47	304	278	0.91
	a61k48	140	265	0.76
	a61k49	90	302	0.76
	a61k51	78	251	0.90
	a61k6	47	270	0.55
	a61k8	87	292	0.74
	a61k9	759	250	0.97
	Distinct features		1419

Table 5 :

 5 class data and F-max selected features/class.

http://www.ncbi.nlm.nih.gov/pubmed/