Kevin Jaouen

Renaud Cornut

Dominique Ausserré

Stéphane Campidelli

Vincent Derycke

Ideal optical contrast for 2D materials observation using bi-layer antireflection absorbing substrates

teaching and research institutions in France or abroad, or from public or private research centers.

Ideal optical contrast for 2D materials observation using bi-layer antireflection absorbing substrates

Introduction

In nanoscience, the capability of observing extremely thin films and materials with a high contrast by optical microscopy is crucial. This is very well illustrated by the role played by optical microscopy from the very early stage of the study of graphene 1 and 2D materials. [START_REF] Novoselov | Proc. Natl. Acad. Sci[END_REF] Indeed, the use of a transparent antireflection layer of appropriate thickness allowed observing graphene, [START_REF] Abergel | [END_REF][4][5][6][7][8][9][10] graphene oxide 11,10 and other 2D materials 12-18,9 despite their atomic-scale thickness. This was notably of great value for the identification of the number of layers or the fabrication of van der Waals heterostructures. It is however very important to note that in all these experiments, antireflection coatings were used and optimized in conditions where light travels from the low to the high refractive index of the observation medium (mostly air) and the microscope sample support (mostly silicon or glass) respectively. The benefits of antireflection coatings in optics have been studied since the end of the nineteenth century and such coatings are now widely used notably on camera and telescope lenses, eyeglasses, solar cells, optical fibers, and on some optical microscopy substrates. [19][20][21] The most widespread antireflection coatings use transparent layers which cancel the reflected light thanks to destructive interferences. When a high transmittance is not needed in combination with the low reflectance, light-absorbing materials can also be used (alone or in combination with transparent ones). [19][20][21][22][23][24] Stacks of multiple transparent and absorbing materials allow adjusting precisely the optical properties of surfaces in a given range of wavelength and incident angles by selecting the appropriate thickness, refractive index and extinction coefficient of each layer following the classical laws of optics.

Backside Absorbing Layer Microscopy is a recently developed optical microscopy technique which draw its strength from the used of light-absorbing substrates specifically in a reverse microscope configuration, i.e. with light propagating from the high to the low refractive index medium (typically from glass to air or solvent). [25][26][27] The layers used are Anti-Reflecting layers made of Absorbing materials and are thus called ARA layers.

For instance, with a single metallic layer deposited on top of a glass substrate used in the backside configuration, there is a preferential metal thickness that allows approaching the ideal antireflection conditions. This thickness depends on the nature of the top medium (air, water, other solvent) and is typically close to 7 nm of gold for the glass/gold/air system and of 2-3 nm of gold for the glass/gold/water one. This configuration allows many types of observations with exceptional contrast 25,26,28,29 in particular for 2D materials. 26 Most importantly, the reverse geometry provides a full half space available for different techniques so that BALM can be easily coupled with other techniques such as electrochemistry. 25,26,28,29 However, when implemented in this way (i.e. using a single few-nm thick metallic layer as ARA material), BALM can be further improved. Indeed, even at the most appropriate metal thickness, antireflection conditions can only be approached (for a complete treatment see Ref.

26

). The equation n metal2k metal2 = n 0 .n 3 (with n 0 and n 3 the refractive indexes of the bottom (glass) and top media (air, water, solvent)) cannot be exactly satisfied with metals. Therefore, when the metal thickness (e metal ) verifies the equation e metal ≈ (λ/4π)(n 0n 3 )/(n metal .k metal ), reflection is reduced to a very low value but is not cancelled. These layers, that we call near-ARA layers, leave thus room for improvement. In addition, producing stable, homogeneous, ultra-flat and stable metal films on glass in the 2-3 nm thickness range is challenging. 30,31 This range is the one adapted for observation in liquids and experiments in water or other solvents are very important for a wide range of studies related to chemical reactivity, bio-sensing, etc. Most of these studies could in principle be further improved if one could design substrates with ARA properties closer to ideal and a thickness higher than 2 nm.

In the following, we show how double antireflection coatings can positively impact the BALM technique and establish the required conditions in different configurations of interest for a broad range of studies, in particular for the observation of 2D materials with enhanced contrast. To do so, we first use the following experiment: the real-time observation and monitoring of the precisely controlled deposition of a thin (0-30 nm) organic layer on a metallic antireflection substrate. Precisely, we follow in-situ and at video rate the light-induced deposition of molecules on gold from a solution of 4nitrobenzenediazonium tetrafluoroborate (NBDT). Indeed, this compound is soluble in water but light is known to induce the coupling of this molecule forming a water-insoluble species that deposits on the surface. 32 Using such photochemical reaction, we benefit from some of the main characteristics of the BALM technique: (i) The organic film is deposited only in the observed area (the size of which can be selected using a diaphragm); (ii) The deposition speed is adjusted by the optical power; (iii) We can record movies of the deposition process allowing very precise extraction of the thickness/contrast relation (in combination with a single AFM calibration step of the final thickness).

Results and discussion

Fig. 1 presents the principle of such experiment and a significant example. A glass slide covered with 3 nm of gold is placed on a reverse microscope equipped with a x63 immersion objective (Fig. 1a) and at t = 0, a solution of NBDT (10 -3 mol.L -1 ) in water is added on top. The octagonal diaphragm is set at an intermediate size (~50 µm) and the reflected light image is recorded (3 Red/Green/Blue channels with 0-255 gray-scale levels on each). For the sake of clarity and to facilitate thickness measurements at a later step, the position of the substrate is moved laterally by ~20 µm (without stopping data acquisition) after some time (9 minutes in Fig. 1b). After 20 minutes, the solution is replaced with water, the diaphragm is fully open and the global picture reported in Fig. It is striking that depending on the thickness, the organic material appears either darker or brighter than the gold substrate. The red curve in Fig. 1c reports the raw camera signal as a function of time in the central area (gray-scale of the green channel only, averaged over 100x100 pixels). This red curve brings several key pieces of information: starting at t = 0 from a BALM configuration already selected for its low reflectivity, the signal first decreases by as much as a factor of 3 after ~7 min. It thus provides the first proof that a metal/organic double layer can improve very significantly the antireflection conditions in such BALM configuration. This is important since minimized reflectivity is the key feature allowing very high contrast. Reflectivity in the centre of the image reaches a minimum and then increases again. It yet remains below the initial value (i.e. it constitutes a more efficient antireflection coating than gold alone) for the first 12 min of the experiment before antireflection efficiency is progressively lost. The same experiment is then reproduced at a different location of the same substrate using in addition an optical filter that reduces the light intensity to 40 % of its initial value (once again the sample was moved laterally during the experiment after ca. 14 min in this case) and the green curve in Fig. 1c is obtained. After rinsing the sample in water, we measured the different height profiles by AFM (see Fig. S1 in the Electronic Supplementary Information). This allows converting the time scale of Fig. 1c into a height scale in Fig. 1d. As expected, both curves overlap since the deposition rate is simply proportional to the incoming light intensity. But more importantly, this conversion permits to extract precisely the thickness that minimizes light reflection (24 ± 1 nm). At this point, it is interesting to mention the remarkable potential of the BALM technique as a tool to follow in real time the deposition of thin organic layers on surfaces. At a typical recording rate of 25 frames/sec, the intensity evolution can be very precisely analysed. Fig. 2a reports a zoom on the red curve of Fig. 1d (zoom width 3 nm, zoom position 6.5 nm, 100x100 pixels area corresponding to ~7x7 µm 2

). The lower panel represents the distribution of the data around the mean slope obtained by a linear fit and shows a standard deviation as low as 10 pm. More importantly, when the gray-scale value of a single pixel (corresponding to a ~70x70 nm 2 area) is plotted (Fig. 2b), the linear decrease of the reflected signal is still very well resolved and the standard deviation of the dispersion is as low as 0.147 nm. This is smaller than the size of the deposited molecules (mass spectrometry data reported in Fig. S2 shows that the film is fully composed of biphenyl molecules resulting from the dimerization of the initial diazonium derivative) and comparable to the surface roughness. It thus proves that molecular-scale changes can be monitored in real time with exceptional vertical resolution at each individual pixel of images and at video rate. This is notably of particular relevance for the use of BALM as an ultrasensitive chemical-or bio-sensing technique. The deposition of an organic thin film from a NBDT solution was selected here as a model system for which the lightinduced mechanism 32 allows a continuous evaluation of the optical characteristics of the metal/organic double layer. However, the results are generic and absolutely not limited to either this particular compound or this particular deposition method. By adjusting the six parameters of the two layers (thickness, real and imaginary part of the complex refractive index of the metal and organic layers) one can indeed select the optical properties of the substrate that are best adapted for a given microscopy experiments. To illustrate this versatility, Fig. 3 reports numerical estimations obtained by computing the reflectivity of the glass/gold/layer2/water stack based on Fresnel equations. It shows the evolution of the lightreflection for 3 nm of gold (with n gold and k gold taken from

33

) as a function of the thickness of a second layer with refractive index (n layer2 , k layer2 ). The calculations are performed at a single wavelength (λ = 550 nm) and at normal incidence. Despite these approximations, they qualitatively capture the main features of Fig. 1d. They notably show (Fig. 3a) how the refractive index n layer2 impacts the ideal layer thickness for minimal reflection. For n layer2 ~ 1.55 and k layer2 = 0, the minimum of reflectivity is obtained for a film thickness of 25-30 nm, in the same range as the experimental value observed in Fig. 1. In Fig. 3b, the thickness that minimizes the reflectivity is plotted as a function of n layer2 for different values of k layer2 (see Fig. S3 for details on the impact of k layer2 ). Fig. 3 and S3 can be used as an indicative guide to select a second layer coating on gold.

The particular examples presented here are relevant for large classes of experiments since: (i) gold layers with a thickness of 3 nm are stable and conductive (in contrast with sub-2 nm gold layers), (ii) k layer2 in the 0-0.3 range includes a huge numbers of conventional coatings (polymers, oxides...), (iii) k layer2 = 0 yields a monotonous evolution of the ideal thickness as a function of n layer2 (Fig. 3b) which, most importantly, is in a range of thicknesses ideally suited from a practical point of view (i.e. in Note, that only the red curve in Fig. 3b is compatible with the data in Fig. 1, which was expected for a fully transparent organic layer (k layer2 = 0). This illustrates how BALM could in principle be used to provide a quick estimate of (n, k) values for arbitrary coating layers on gold. A quantitative evaluation of the accuracy of such strategy requires the precise knowledge of additional experimental parameters (notably the λ-dependence of the n and k values of ultrathin gold films) and is beyond the scope of this article.

Having established how double-layer metal/organic substrates improve antireflection performances, we now show how this can be used to obtain contrast enhancement for the observation of materials at the nanoscale. For such illustration, we selected graphene oxide (GO) as a model system because it is an important material in nanoscience and in particular in the field of energy conversion and storage. 34,35 In addition, contrarily to graphene, reduced-graphene-oxide (r-GO) and most 2D materials (such as MoS 2 ), GO is transparent. It is thus a challenging material for optical microscopy making contrast enhancement particularly important (note that as stated in the introduction, the use of transparent antireflection coatings in the conventional observation direction and in air-only was already reported for GO 11,14

). We showed in Ref. 26 how GO behaves on a metallic ARA single layer in BALM experiments, which helps highlighting the advantages of double-layers. In particular, it was shown that GO flakes usually appear with a negative contrast (darker that the substrate), which was indicating room for improvement towards better ARA layers. As a last advantage, GO monolayers of very large lateral size and very limited number of folds and wrinkles can be deposited on any substrate, using for example the method reported in Ref. 36 and used here. S4 for details). Strikingly, this is a factor 3.6 higher than in the initial gold-only conditions, which represents a considerable improvement. This evolution of the contrast of GO on gold at the early stage of the deposition of an organic layer can be very well explained using the simple model. Fig. 5a displays the calculated contrast between bare gold (3 nm) and a single layer of graphene oxide (at λ = 550 nm with n GO = 2, k GO = 0.3) on top of gold, as a function of the thickness of a transparent material deposited above the GO. The n layer2 = 1.55 and k layer2 = 0 values selected for this organic layer are those that best represent the deposition from NBDT in Fig. 3 and the (n GO , k GO ) values are extracted from Ref. 11 . For a very thin organic film (0-13 nm) the contrast is first enhanced (GO becomes progressively darker) in agreement with Fig. 4. For intermediate thicknesses, the contrast becomes lower (it cancels around 25 nm) and then finally reverses. For ~35 nm, GO should appear much brighter than the gold surface. This calculated behaviour qualitatively matches what is observed experimentally in Fig. 5b, which presents an experiment comparable to the one of Fig. 4 but conducted over a longer time period so as to observe the contrast inversion. The red curve represents the reflected signal on gold and the blue curve the (GO vs. gold) contrast during the deposition of the organic material, extracted from every frame of the movie. The selected images in Fig. 5c highlight the key features: initial contrast enhancement, intermediate point of minimal GO visibility, and contrast inversion. Another important information that can be extracted from Fig. 5b is that: it is not at the minimum of the gold reflectivity (ca. 20 nm in Fig. 5b upper x-axis, dotted red arrow) that occurs the best contrast for the observation of GO monolayers (ca. 8 nm in Fig. 5b upper x-axis, dotted blue arrow). It is of great relevance when one wants to select a coating ideally adapted to a specific study.

The methodology used up to here may look specific to the deposition method employed. However, it is important to note that the observed effects are very generic and can be extended in several ways so as to tackle other types of studies in very different contexts. Numerical estimations in Fig. 3 already showed that it is not limited to the organic material used in this study but that minimal reflection conditions exist for most n layer2 values in a large and relevant range (for the impact of k layer2 see also Fig S3). We now exemplify the fact that contrast enhancement: (1) can be obtained with more conventional 2 nd layer materials, ( 2) is not limited to data acquired in water, and most importantly, ( 3) is also achieved when the material under study (GO in our case) is on top of the second layer coating and not between gold and this second layer. This configuration is of course the one used in conventional microscopy experiments. 6a reports numerical results. In water, with this particular set of parameters, the contrast can only degrade (and get reversed at higher PMMA thicknesses). In air however, there is a very significant contrast enhancement effect in the 20-30 nm range of PMMA. Note that this particular calculation was performed with 4 nm of gold and that the calculated contrast is always higher than the measured one. The difference come from the calculation approximations (single wavelength, normal incidence only vs. high numerical aperture in the experiment) and from the presence of 0.1 nm of chromium as adhesion layer. From Fig. 4, 5 and 6, it is clear that strong contrast enhancement can be achieved with BALM using double-layer antireflection coating in very different and generic configurations, irrespective of the second layer material (organic or not) and deposition method, stacking order or observation medium (air, water, solvent), one can immediately foresee the benefits of using double inorganic near-ARA layers (typically composed of oxide layers on metal) in place of organic coatings. This opens the way to robust, stable, thickness controlled BALM substrates with improved performances.

Conclusions

In conclusion, we first demonstrated that BALM microscopy can be used to monitor in real time the deposition of molecular films with a precision in the vertical direction of the order of 0.15 nm at the single pixel level and at video rate. Such precision is simultaneously obtained on every pixel of each image of the movie so that textured surfaces and surfaces with pre-deposited materials (typically 2D materials) can be readily studied. In addition, if single-pixel lateral resolution is not required, vertical precision as good as 10 pm is achieved at the 100x100 pixel level (corresponding to a ~7x7 µm 2 area). This molecular-scale sensitivity and video-rate operation confirm the high potential of the technique for chemical-and bio-sensing applications.

We then proposed qualitative guidelines for the selection of near-to-ideal double-layer antireflection coatings. By selecting the thickness, refractive index and extinction coefficient of both layers, observation supports optimized for the BALM geometry (light traveling from glass to air or solvent) and adapted to the system under study (observation of transparent or absorbing materials, observation in air or in a solvent, conducting or insulating supports, etc.) can be produced. Beside the enhanced sensitivity, this new study brings an additional asset in terms of surface chemistry: in sensing experiments the material on which sensing probes (typically bio-probes) would be attached can be selected in an extended range of materials to match different functionalization methods. Finally, double ARA layers lead to very significant contrast improvement for 2D materials. This should facilitate the study of such materials and the fabrication of their heterostructures in an increasing number of laboratories. Moreover, the fact that BALM substrates can be insulating makes possible the GO transfer. The metal coated glass substrates were cleaned again by exposition to UV-ozone for 30 min and finally washed with isopropanol, dried in a N 2 flux and heated at 70°C for 10 min. After the drying process, the slides were placed in a chamber for GO deposition using the so-called bubble deposition technique.

  1b is acquired. One can clearly observe in Fig. 1b the light reflected on the bare gold surface and the light reflected on three areas covered with an organic film with different thicknesses (corresponding to light exposure during 9 min on the right, 20 min in the centre and 11 min on the left part of the pattern).

Figure 1 .

 1 Figure 1. Light-induced controlled deposition of an organic layer on gold. (a) BALM configuration of the experiment. (b) Final BALM image showing the organic layer deposition at maximum light intensity. The substrate was moved by ~20 µm at mid-experiment along the arrow direction. (c) Reflected light as a function of time for two light intensities (I0 and 0.4*I0). The plots display the gray-scale data from the green channel of the camera averaged on a 100x100 pixels area (see text for the impact of this area size), (d) Same as in c after conversion of the time scale into a height scale using AFM measurements.

Figure 2 .

 2 Figure 2. BALM vertical sensitivity for real-time monitoring the deposition of organic ultrathin films. (a) Detail of the 5-8 nm part of the red curve of Fig. 1d corresponding to a 100x100 pixel area. (b) Same data extracted on a single pixel of the movie (red) and linear fit (black line), (c,d) Distribution of the data around the linear fit. The gray value is converted into nm using the slope of the linear fit. The standard deviation is 10 pm and 147 pm in c and d respectively.

Figure 3 .

 3 Figure 3. Calculated estimation of the reflected light intensity (at λ = 550 nm and normal incidence) on a 3 nm-gold/second-layer stack. (a) For klayer2 = 0 and as a function of the second layer thickness and refractive index nlayer2. (b) Thickness of the second layer giving minimal reflection for different values of nlayer2 and klayer2.

  nm thickness range adapted to spin-coating, evaporation and other deposition techniques, including grafted chemical or biological layers for surface functionalization).

Figure 4 .

 4 Figure 4. BALM images of GO flakes on gold extracted from a movie (green channel data from movie_1 in ESI) recorded during the controlled deposition of an organic layer. The only correction to the raw data is as follows: the 0-255 gray scaled range was restricted for clarity to 0-128, 0-96, 0-64 for the 3 images respectively. This simply compensates for the progressive darkening of the images as the reflection decreases, but does not impact contrast values. (a) At t = 0 s, C = -0.046, (b) at t = 50 s, C = -0.119, (c) at t = 156 s, C = -0.167. Raw data and details on contrast extraction are presented in Fig. S4.

Fig. 4

 4 Fig. 4 displays three images (green channel) extracted from a full movie (See video_1 in ESI) that records the evolution of the GO visibility on gold during the light-induced deposition of the organic NBDT-based layer. At t = 0, the contrast defined as C =

Figure 5 .

 5 Figure 5. Contrast of monolayer graphene oxide on gold upon molecule deposition. (a) Contrast calculation for nlayer2 = 1.55 and klayer2 = 0. (b) Light reflected on gold (red curve) and experimental contrast between gold and GO (blue curve), (c) BALM images at t = 0, t = 3 min 6 sec, t = 5 min 12 sec and t = 6 min 43 sec. GO goes from darker than gold at t = 0 to brighter than gold. The images are extracted from the green channel of a movie. The only correction to the raw data is as follows: the 0-255 gray scaled range was restricted for clarity to 0-196, 0-100, 0-75 and 0-75 for the 4 images respectively. This simply compensates for the progressive darkening of the images as the overall reflection decreases.

Figure 6 .

 6 Figure 6. Contrast of GO monolayers above a PMMA/gold antireflection double-layer. (a) Calculated contrast for a GO monolayer deposited above a PMMA film on 4 nm of gold either in air (open squares) or in water (filled squares) as a function of the PMMA thickness, (b) Experimental contrast of GO monolayers on 3 nm of gold in air and water as a function of PMMA thickness. (c) Principle of the experiment. (d) Example of BALM image (green channel) of GO monolayers directly on gold (left part) and on 13 nm of PMMA on gold (right part). For clarity, the 0-255 gray scale range was restricted to 0-150.

  direct coupling of high-contrast observation with electrical measurements. Altogether, this study opens new and exciting possibilities for the use of BALM microscopy as a versatile and powerful tool for nanoscience.ExperimentalBALM. The BALM instrument was setup with Watch Live S.A.S23, Lyon, France. Microscope cover slides from VWR were used as glass support for single-layer (gold) and bilayers (NBDT/Gold, PMMA/Gold) antireflection absorbing layers. The near-ARA layers are composed of ~0.1 nm of chromium and 3 nm of gold.

36

  Organic layer deposition. The BALM slides were placed in a home-made cell standing on top of a reverse microscope equipped with a Zeiss oil immersion objective, a plan Apo x 63 with numerical aperture NA = 1.40, a Canon EOS 6D camera and an halogen lamp (64655 HLX) filtered with A Thorlabs FESH0700 filter. For the light intensity dependence experiment, a reflective ND filter 0.4 OD from Edmund Optics was added. Water was added on the support to setup the focus, light intensity, spot size and the camera settings. Then water was removed and a movie was started. Then 4-Nitrobenzenediazonium tetrafluoroborate from Sigma-Aldrich (M = 236.92 g.mol -1 ) at 10 -3 mol.L -1 in water was added for 20 min. Movies were recorded with an exposure time of 25 ms and an ISO sensitivity of 800, 2000 and 12800 respectively for experiments at I 0 (maximum light intensity), at 0.4*I 0 and at minimum light intensity. At the end of the experiments the solution was removed and the surface was washed twice with DI water.

Acknowledgements

This work is supported by a public grant overseen by the French National Research Agency (ANR) as part of the "Investissements d'Avenir" program (Labex NanoSaclay, reference: ANR-10-LABX-0035). We thank T. Bernard and P. Surugue for technical support.

Conflicts of interest

There are no conflicts to declare.


Tamara Álvarez-López

Milagros Fernández-Gavilanes

Enrique Costa-Montenegro

Patrice Bellot
email: patrice.bellot@univ-amu.fr

A Proposal for Book Oriented Aspect Based Sentiment Analysis: Comparison Over Domains

Keywords: Aspect-based sentiment analysis, book reviews, datasets, annotation, evaluation

Aspect-based sentiment analysis (ABSA) deals with extracting opinions at a fine-grained level from texts, providing a very useful information for companies which want to know what people think about them or their products. Most of the systems developed in this field are based on supervised machine learning techniques and need a high amount of annotated data, nevertheless not many resources can be found due to their high cost of preparation. In this paper we present an analysis of a recently published dataset, covering different subtasks, which are aspect extraction, category detection, and sentiment analysis. It contains book reviews published in Amazon, which is a new domain of application in ABSA literature. The annotation process and its characteristics are described, as well as a comparison with other datasets. This paper focuses on this comparison, addressing the different subtasks and analyzing their performance and properties.

Introduction

People have access to a huge amount of information available online about a wide diversity of subjects and users can express their opinions about products or services by means of social networks or blogs. Analyzing the user-generated content on the Web became very interesting both for consumers and companies, helping them to better understand what people think about them and their products. For this reason, many studies arose in the fields of Opinion Mining and Sentiment Analysis [START_REF] Liu | Sentiment Analysis and Opinion Mining[END_REF], trying to extract the most relevant information in an automatic way. Most of these works aim at extracting the global sentiment from the whole text, by means of different approaches, which can be supervised [START_REF] Pak | Twitter as a Corpus for Sentiment Analysis and Opinion Mining[END_REF], normally based on classifiers, or unsupervised, such as [START_REF] Fernández-Gavilanes | Unsupervised Method for Sentiment Analysis in Online Texts[END_REF], where the authors aim at capturing and modeling linguistic knowledge by using rule-based techniques. In the last years many works can also be found applying deep learning techniques [START_REF] Santos | Deep Convolutional Neural Networks for Sentiment Analysis of Short Texts[END_REF].

However, extracting the specific entities or aspects to which a sentiment is expressed provides a more specific insight of what people think about a particular product, and this is exactly the aim of the so-called ABSA. Several studies emerged in the ABSA field [START_REF] Schouten | Survey on Aspect-Level Sentiment Analysis[END_REF][START_REF] Akhtar | Feature Selection and Ensemble Construction: A Two-step Method for Aspect Based Sentiment Analysis[END_REF], as well as competitions like SemEval [START_REF] Pontiki | SemEval-2016 task 5: Aspect based sentiment analysis[END_REF], where researchers are encouraged to submit and evaluate their systems over common datasets. Again we can find supervised approaches based on classifiers and conditional random fields (CRFs) [START_REF] Jakob | Extracting Opinion Targets in a Single and Cross Domain Setting With Conditional Random Fields[END_REF] and unsupervised ones based on frequency studies or syntax dependencies [START_REF] Poria | A Rule-Based Approach to Aspect Extraction from Product Reviews[END_REF].

For developing ABSA systems, annotated datasets are essential for training and testing. However, manually annotating the user-generated reviews is a tough task, time and resource consuming, and not many datasets are available. We can find the following ones. The first one and widely used in the literature is on electronic products [START_REF] Hu | Mining and Summarizing Customer Reviews[END_REF], tagged with aspects and the sentiment associated. Then, [START_REF] Ganu | Beyond the Stars: Improving Rating Predictions using Review Text Content[END_REF] works with restaurant reviews, annotated with categories from a predefined list and their sentiments. Similarly annotated with predefined categories, we find [START_REF] Thet | Aspect-Based Sentiment Analysis of Movie Reviews on Discussion Boards[END_REF], which is about movie reviews. Finally we have to mention the datasets created for SemEval [START_REF] Pontiki | SemEval-2016 task 5: Aspect based sentiment analysis[END_REF], providing datasets for 8 languages and 7 domains, including restaurants, laptops, mobile phones, digital cameras, hotels and museums. Particularly for the book domain, only two works can be found on annotated corpora, one for Arabic language [START_REF] Al-Smadi | Human Annotated Arabic Dataset of Book Reviews for Aspect Based Sentiment Analysis[END_REF] and the other one for Portuguese [START_REF] Freitas | Sparkling Vampire... lol! Annotating Opinions in a Book Review Corpus. New Language Technologies and Linguistic Research: A Two-Way Road[END_REF].

Motivated by the few amount of resources found, we developed a new dataset for English language in the book domain, which was firstly presented in [START_REF] Álvarez-López | A Book Reviews Dataset for Aspect Based Sentiment Analysis[END_REF]. The aim of this paper is to provide a new glance of this dataset, its structure and annotation process, as well as to provide an evaluation benchmark for different subtasks in ABSA and to compare its performance to other datasets available, analyzing the particularities of each one.

The remaining of this article is structured as follows. In Section 2 the datasets under analysis are described. Section 3 presents the new dataset we created, including inter-annotator agreement levels and different statistics. In Section 4 the evaluation and comparison of all the datasets are shown. Finally, Section 5 provides some conclusions and future work.

ABSA Datasets

In this section we present the different datasets considered, all of them manually annotated for ABSA. Not much work on annotated datasets for this task can be found, so we use those which are publicly available, in electronics, restaurant and laptop domains, and were widely used in the literature.

Electronic Product Reviews

The Customer Reviews Dataset [START_REF] Hu | Mining and Summarizing Customer Reviews[END_REF] contains customer reviews of five electronic products, bringing a total number of 314 reviews, collected from Amazon.com and C|net.com and manually annotated. The features for which an opinion is expressed are tagged, along with the sentiment, represented by a numerical value from +3 (most positive) to -3 (most negative). In this dataset3 both explicit and implicit features (not explicitly appearing in the text) were annotated, however only the explicit ones were taken into account for this work. In Table 1 detailed information about this dataset is shown. For the next experiments, we will test each dataset separately, using the other four as training.

SemEval Datasets

In the following, the two datasets provided by SemEval workshop [START_REF] Pontiki | SemEval-2016 task 5: Aspect based sentiment analysis[END_REF] for English language are examined, in the domains of restaurants and laptops. They both have a similar structure, containing several reviews annotated at sentence level.

In every sentence the specific aspects, called Opinion Target Expressions (OTE), are identified; the aspect category to which the OTE belongs, chosen from a predefined list; and the polarity (positive, negative or neutral ). The list of categories designed for restaurants is formed by a combination of entity#attribute and it is composed of 12 different ones, such as restaurant#prices, food#quality or ambience#general. However, for the laptop dataset the categories are much more specific, combining 22 different entities with 14 attributes, obtaining a great number of possibilities. For the aim of this paper we shorten this list by regrouping the entities and attributes, so we can obtain a similar number of categories for all the datasets under evaluation. We only keep the entities laptop, software, support and company, while the entities hardware, os, warranty and shipping are removed due to their low frequency of appearance. Finally, the rest of the entities are grouped in components, as they all refer to different components of a laptop. About the attributes, we keep all of them, but only associated to the laptop entity. For the rest of entities, the attribute is always general. Like that we obtain a list of 13 categories.

Both datasets are divided into training and test and more detailed information is displayed in Table 2. The information shown about the laptop dataset belongs to the new annotations we created by summarizing the categories. 3 New Book Reviews Dataset

Performing ABSA on book reviews is very useful for different kind of users, including professional as well as non-expert readers, helping them when searching for a book which fits in certain requirements. It can be later applied to recommendation in digital libraries or electronic book shops.

In the following subsections we present the new dataset, the annotation process, its structure and some statistics. It is publicly available online 4 and it was previously introduced in [START_REF] Álvarez-López | A Book Reviews Dataset for Aspect Based Sentiment Analysis[END_REF], where some information about the annotation process and the dataset properties can be found. Throughout this work, we focus in providing additional information, as well as a baseline for its evaluation, comparing it to other datasets available for the same task.

Data Collection

For the construction of this dataset, 40 book records were selected randomly from the Amazon/LibraryThing corpus in English language provided by the Social Book Search Lab [START_REF] Koolen | Overview of the CLEF 2016 Social Book Search Lab[END_REF], a track which is part of the CLEF (Conference and Labs of the Evaluation Forum). Its main goal is to develop techniques to support readers in complex book search tasks, providing a dataset for the book recommendation task, consisting of 2.8 million book descriptions from Amazon, where each description contains metadata about the booktitle, author or isbn, as well as user generated content, like user ratings and reviews. However, it does not contain any annotations about the aspects or sentiments from the reviews.

Our dataset is composed by the textual content of the reviews associated to each of the 40 books selected, obtaining a total number of 2977 sentences from 300 reviews, which were annotated with aspects, categories and sentiment information.

Task Description

This new corpus is intended to cover different subtasks in the ABSA field:

1. Aspect Extraction. The aim is to extract the specific features mentioned in the text, which are related to the authors, characters, writing quality, etc. In our dataset we differentiate between explicit and implicit aspects. Implicit aspects are those which are not explicitly written in the analyzed sentence, but can be figured out by the context or the previous sentences: e.g.: Sent 1 → When Arthur is suffering from asthma,... (target = Arthur) Sent 2 → Then, he starts seeing a large house. (implicitTarget = Arthur) 2. Category Detection. Each target detected in a sentence is classified at a more coarse-grained level, assigning a category from a predefined list. The categories defined for the book domain try to cover most of the features that readers mention in their reviews and are divided into two groups, the ones related to the book itself and those related to its content. In the first group: general, author, title, audience (type of readers which the book was written for), quality (about the writing style), structure (related to the chapters, index, etc.), period (when the book was written or published), length and price. Then the categories included in the second group are: characters, plot, genre (related to the literary genre) and period (when the plot passes). 3. Sentiment Polarity. This last task consists of assigning a polarity to every aspect detected from three possible ones: positive, negative and neutral.

Annotation Process

In order to construct the annotated dataset and support all the tasks previously defined, different tags are attached at sentence level: Out of Scope, Target, Occurrence, Implicit Target, Category and Polarity. More information about them can be found in [START_REF] Álvarez-López | A Book Reviews Dataset for Aspect Based Sentiment Analysis[END_REF] and an example of an annotated sentence is shown in Figure 1. The reviews selected for the dataset were annotated by 3 different annotators, researchers in the field of NLP and, in particular, in Sentiment Analysis from the University of Vigo. In order to ensure the consistency of the final corpus, some guidelines were given to the annotators, as well as training sessions to solve the doubts arisen. Finally, only those annotations in which at least two of the three annotators agreed were taken into account.

When integrating the results from the three annotators, one of the main difficulties found was to determine the boundaries of a particular target. For example, in the sentence "I would recommend ages 8 and over" one annotator can tag ages 8 and over as an aspect, whilst other can annotate just ages 8. For this situation we consider that both annotators agreed and the longest one is taken into account, as so they do the authors in [START_REF] Freitas | Sparkling Vampire... lol! Annotating Opinions in a Book Review Corpus. New Language Technologies and Linguistic Research: A Two-Way Road[END_REF] .

In order to calculate the inter-annotator agreement, the Dice coefficient (D i ) is used, instead of Cohen's Kappa coefficient, as the annotators may tag any word sequence from the sentences, which leads to a very large set of possible classes:

D i = 2 • |A i ∩ B i | |A i | + |B i | (1)
where A i , B i are the aspects tagged by annotators A and B, respectively. In Table 3 the agreement between annotators A and B (A|B), A and C (A|C) and B and C (B|C) are shown, as well as the average of the three. The inter-annotator agreement study is also applied to the implicit aspects detected, the category identification, the polarity assigned to an aspect when there was already an agreement in annotating the aspect concerned and the pairs aspect-category, which means that both annotators extracted the same aspect and also assigned the same category to it. As we can see in Table 3, for the task of identifying the specific aspects, the 75% of agreement is reached, which is a good result due to the difficulty of the task. Then similarly, the annotators tag the same categories for a particular sentence with an agreement of 72%, regardless of the aspects extracted. When taking into account the tuple <aspect, category>, the Dice coefficient decreases to 65%, which is normal as in this case they have to agree simultaeously on two parameters. Once the annotators agree on a particular aspect, they also coincide in the polarity annotation with an agreement of 75%. Finally, the Dice measure obtained for the implicit aspect extraction task is 37%, which means a very poor agreement. This can be explained due to the complexity of detecting implicit aspects, as they do not appear written in the sentence, so it makes more difficult the annotation task. There is still much room for improvement in this last task and for the aim of this paper we will not take them into account for the experiments.

Dataset Characteristics

The complete dataset is composed of 300 reviews, belonging to 40 different books, with a total of 2977 sentences. For the following experiments the dataset was divided into training and test, selecting around the 25% of the reviews for testing and the rest for training, making sure that the reviews included in each one belong to different books, in order to avoid biased results. In Table 4 some additional information is shown. In Table 4 we can also see the polarity distribution (positive (P), negative (N) and neutral (NEU)) of explicit aspects, which is similar for training and test. Moreover, it can also be observed that the number of neutral aspects is quite higher that the rest of the polarities. This is due to the annotation schema, as in this dataset not only the opinionated aspects are annotated, but also those with no opinion associated (neutral ). We found very common to mention characteristics of the book related to the plot or the characters in an informative way and annotating also this kind of aspects will be useful for future tasks, such as recommendation or summarization.

Finally, in Figure 2 the distribution of the category annotations is shown for training and test, being similar across both of them. We find that the most common categories are characters and plot. However, categories like the price or the length of the book are not so usual in this kind of reviews.

Performance Evaluation: Comparative

In this section we present the evaluation results for the datasets previously described according to the three different tasks. A baseline system was designed for each task and applied to every dataset, analyzing and comparing the results.

For the evaluation of aspect term extraction and category detection we use precision, recall and F-measure, whilst for sentiment analysis we use the accuracy, following the same evaluation procedure as in the SemEval workshop [START_REF] Pontiki | SemEval-2016 task 5: Aspect based sentiment analysis[END_REF].

Aspect Extraction Task

For this task the baseline system consists of CRFs, using the CRF++ tool [START_REF] Kudo | CRF++: Yet another CRF toolkit[END_REF]. We extract for each single word the following features: words, lemmas, bigrams, POS tags and entity recognition, obtained by means of Stanford CoreNLP tool [START_REF] Manning | The Stanford CoreNLP Natural Language Processing Toolkit[END_REF]. We also extract the value of each two successive features in the the range -2,2 (the previous and subsequent two words of actual word) for the each feature. Moreover when the system identifies two or more consecutive words as aspects, we consider them as a single multiword aspect.

In Table 5 we can see the results obtained for the three datasets (the laptop dataset is not annotated with aspects, therefore it cannot be evaluated for this task), in terms of precision, recall and F-measure. It can be observed that applying the same baseline for all the datasets, the restaurant dataset obtains the highest F-measure. However, for electronics, and especially for books, we find quite low recall, what means that it is harder to extract all the aspects annotated, maybe due to the kind of texts which can be more complex in these two domains, in terms of the diversity of vocabulary.

Analyzing some properties of the datasets, we can see in Figure 3 the relation between the number of sentences, the vocabulary size and the target extraction performance. The vocabulary size of a dataset is determined by the number of unique word unigrams that it contains. As we can see in Figure 3, increasing the number of sentences does not always imply an increase in the vocabulary size. However, when the vocabulary size increases, the performance of the aspect extraction task becomes lower. With a higher amount of different words, there should also be more different aspects to detect.

If we inspect the list of aspects annotated in the test dataset for each domain, we find that for restaurants there are 312 different aspects, 278 different ones for electronics and 417 for books. Moreover, for the book dataset we find more terms which are considered as aspects just once in the whole dataset. For the electronics domain, even if there are less different aspects, the biggest difficulty is to differentiate when the same term should be considered as aspect or not. We find that the terms which are most frequently correctly detected (true positive) are usually the same as those which are most frequently not detected (false This situation arises a very interesting challenge for aspect extraction, as it is not easy to decide when a particular word should be considered as aspect or not, according to the specific context or sentence.

Category Detection Task

The baseline developed here is based on an ensamble of binary SVM classifiers with a linear kernel, one for each category. The library libsvm5 was used, with the following binary features at sentence level: words, lemmas, POS tags and bigrams. Then, for each sentence, one, several or no categories can be found.

The datasets evaluated for this task are restaurants, laptops and books, as for electronics there are no category annotations, and the performance results, in terms of precision, recall and F-measure, can be seen in Table 5.

We can see again that the restaurant dataset obtains the highest results, while the worst performance is obtained for the book dataset. When the evaluation is performed for each category separately, some differences can be found for the three domains. For the restaurant dataset the F-measure results obtained for the different categories are more similar to each other than those obtained for the categories in the book domain. For laptop dataset, and especially for book dataset, we can find some categories with really low performance. While the worst performance in the restaurant domain is 23% for the restaurant#miscellaneous category, in laptops we find four different categories whose F-measure is lower than 25%, rising to eight categories in the book domain. However, we also have to highlight the low representation of certain categories in the book dataset, making it more difficult for the classifier to learn when to annotate them for a particular sentence.

In addition to this, we find the categorization in general more difficult for the book domain. In this dataset the category which achieves the highest F-measure is characters, as there are many sentences both in training and test annotated for it. However, the F-measure is still 50%, while in the restaurant domain the categories which are more accurately detected obtain results around 80%.

The categories which obtain the best performance are restaurant#general, food#quality and ambience for restaurants; laptop#general, laptop#price and laptop#quality for laptop dataset; and finally for the book domain they are general, characters and author. These categories are also the most common in both training and test sets for each domain and the classifiers tend to work better with bigger amount of data.

Aspect-Based Sentiment Analysis Task

For extracting the sentiment associated to each aspect, we consider a window of five words before and after the specific aspect. The window size was determined by performing several experiments varying this parameter, so the highest results were obtained when it was equal to five. Then, we add the polarities of all the words included, considered as a numerical value and extracted from a sentiment lexicon, SOCAL [START_REF] Taboada | Lexicon-based methods for sentiment analysis[END_REF]. This dictionary is composed of a list of words with a polarity associated, expressed on a scale between -5 (most negative) and +5 (most positive). Then, if the addition of the polarities divided by the number of words with sentiment associated is higher than 1, the target is considered positive; if it is lower than -1 it is tagged as negative; and neutral otherwise. This task is applied to electronics, restaurant and book datasets at aspect level. For laptop dataset, as there are no annotated aspects, we extract the sentiment of the whole sentence and assign it to the annotated categories. In Table 6 we can see the results obtained. It can be observed that the weighted average accuracy is quite similar for all the datasets. The highest results are obtained for positive aspects. One of the reasons is the high amount of positive aspects in relation to negative or neutral in most of the datasets. In the restaurant domain, the 71% of the aspects are positive, whilst only 5% of them are tagged as neutral. Similar percentages can be obtained from the laptop domain. In electronics, 65% of the aspects are positive and the other 35% are negative. Finally, for the book dataset it can be seen that the accuracy is similar for the three classes. In this case the 61% of the aspects are neutral, 28% are positive and only 10% are negative. However, if we take into account the F-measure instead of accuracy, we observe that the highest results are obtained for neutral aspects (70%), followed by positive (58%) and negative (24%).

Conclusions

The aim of this paper was to provide a common evaluation benchmark for the ABSA task and its different subtasks: aspect extraction, category detection and sentiment analysis at the aspect level. Different datasets available in the literature were studied and compared, belonging to electronics, restaurant and laptop domains. Moreover, different characteristics and the annotation process were described for the new dataset in the domain of books, not yet explored in the state of the art. Then, different baselines were proposed for the evaluation and comparison. The aim here was not to provide a new approach for ABSA, but to analyze the distinctive features of reviews from different domains and how they affect to the ABSA performance, as well as to perform the evaluation of the new dataset developed.

As future work we plan to continue the research in the book domain and develop new baselines which fit better for this kind of reviews, which seem to present bigger challenges for ABSA. Moreover, we would like to work in the integration of aspect extraction from book reviews for improving book recommendation systems, introducing them as new inputs, so that the system could apply a reranking of the recommendation list according to this new information.

Fig. 1 .

 1 Fig. 1. Example of an annotated sentence from the corpus

Fig. 2 .

 2 Fig. 2. Aspect category distribution in training and test sets for books

Fig. 3 .

 3 Fig. 3. Number of sentences, vocabulary size, and F-measure for the aspect extraction task for the different datasets

Table 1 .

 1 Number of: reviews, sentences, aspects extracted and aspects tagged as positive or negative, for each product in the electronics dataset

	Data	Product	#Revs. #Sent. #Aspects #Pos.Asp. #Neg.Asp.
	D1 Digital camera	45	597	256	205	51
	D2 Digital camera	34	346	185	155	30
	D3 Cell phone	41	546	309	230	79
	D4 MP3 player	95	1716	734	441	293
	D5 DVD player	99	740	349	158	191
		Total	314	3945	1833	1189	644

Table 2 .

 2 Number of: reviews, sentences and categories, distinguishing between positive, negative and neutral ones, for restaurants and laptops Domain #Revs. #Sent. #Cat. #Pos.Cat. #Neg.Cat. #Neu.Cat.

	Restaurants	Train 350 Test 90	2000 676	2506 859	1657 611	751 204	98 44
	Laptops	Train 450 Test 80	2500 808	2730 749	1547 454	1002 254	181 41

Table 3 .

 3 Detailed Dice coefficients for aspect, category, aspect+category, polarity and implicit aspect annotations

	Annotators Aspect Cat. Asp.+Cat Polarity Implicit
	A|B	0.76 0.72	0.65	0.77	0.36
	A|C	0.76 0.74	0.68	0.77	0.37
	B|C	0.74 0.70	0.63	0.71	0.37
	Avg.	0.75 0.72	0.65	0.75	0.37

Table 4 .

 4 Number of: sentences, annotated aspects (explicit and implicit), and sentences tagged as out of scope for the book dataset

	Dataset #Sent.	#Explicit Asp.	#Implicit #OutOfScope
			#P #N #NEU Total
	Train	2219 726 296 1663 2685	265	469
	Test	758 230 88	501	819	64	182

Table 5 .

 5 Precision, recall and F-measure for aspect and category detection tasks

	Dataset	Aspect Precision Recall F-measure Precision Recall F-measure Category
	Restaurants 0.69	0.64	0.66	0.72	0.64	0.68
	Electronics	0.64	0.31	0.42	-	-	-
	Laptops	-	-	-	0.66	0.40	0.50
	Books	0.59	0.14	0.22	0.53	0.30	0.38

Table 6 .

 6 Accuracy for aspect polarity extraction task Dataset Positive Negative Neutral Weight.Avg.

	Restaurants 0.72	0.52	0.32	0.66
	Electronics	0.68	0.51	-	0.62
	Laptops	0.6	0.5	0.35	0.55
	Books	0.57	0.56	0.57	0.57

Available online at https://www.cs.uic.edu/ liub/FBS/sentiment-analysis.html

http://www.gti.uvigo.es/index.php/en/book-reviews-annotated-dataset-for-aspectbased-sentiment-analysis

https://www.csie.ntu.edu.tw/ cjlin/libsvm/

Acknowledgments This work is supported by Mineco grant TEC2016-C2-2-R, Xunta de Galicia grant GRC and the French ANR program "Investissements d'Avenir" EquipEx DILOH (ANR-11-EQPX-0013).


Messaoud Chaa
email: mchaa@cerist.dz

Omar Nouali
email: onouali@cerist.dz

Patrice Bellot
email: patrice.bellot@lsis.org

Combining Tags and Reviews to Improve Social Book Search Performance

Keywords: Social Information Retrieval, Social Book Search, Document Length Normalization, User-Generated Content, Tag Based Model, Review Based Model

The emergence of Web 2.0 and social networks have provided important amounts of information that led researchers from different fields to exploit it. Social information retrieval is one of the areas that aim to use this social information to improve the information retrieval performance. This information can be textual, like tags or reviews, or non textual like ratings, number of likes, number of shares, etc. In this paper, we focus on the integration of social textual information in the research model. As it seems logical that integrating tags in the retrieval model should not be in the same way taken to integrate reviews, we will analyze the different influences of using tags and reviews on both the settings of retrieval parameters and the retrieval effectiveness. After several experiments, on the CLEF social book search collection, we concluded that combining the results obtained from two separate indexes and two models with specific parameters for tags and reviews gives good results compared to when using a single index and a single model.

Introduction

The rise of social media has changed the role of users in the Web. From simple users who just consume information to users that can produce it. Many kinds of social information are generated like tags, reviews, rating, users' relation or users' interests. This social information has been exploited in several application domains such as marketing [START_REF] Kim | Do social media marketing activities enhance customer equity? an empirical study of luxury fashion brand[END_REF], commerce [START_REF] Swamynathan | Do social networks improve e-commerce?: a study on social marketplaces[END_REF], etc. The field of information retrieval is no exception to this. A few years ago, a new area of research namely social information retrieval (SIR) emerged and gained popularity. The objective of SIR systems is to identify and integrate that social information (tags, reviews, ratings...) in a search process in order to improve the IR performance [START_REF] Bouadjenek | Social networks and information retrieval, how are they converging? a survey, a taxonomy and an analysis of social information retrieval approaches and platforms[END_REF][START_REF] Park | Leveraging user reviews to improve accuracy for mobile app retrieval[END_REF].

On the other hand, social digital libraries such as LibraryThing 4 and Goodreads 5 are social cataloging websites for books that enable users to store and share book catalogues. Users can rate, review, tag, and discuss their books. They can also create groups with other members, create discussions, and ask for book recommendations. While suggestions and recommendations can be sent to users in response to their request. This generates a large volume of data for books and represent a great challenge for information retrieval tasks in general and book search in particular. In order to exploit this wealth of information, the CLEF track social book search (SBS) has come to use the social information to search and develop techniques to support users in book search tasks.

Social Book Search

Social book search (SBS) 6 was a CLEF lab (2011-2016) that particularly investigated book search in social media. It was interested in the use of user-generated content from social media to support users of LibraryThing forum in finding documents(books) 7 that interest them and that are relevant to their request [START_REF] Koolen | Overview of the clef 2016 social book search lab[END_REF][START_REF] Koolen | Social book search: comparing topical relevance judgements and book suggestions for evaluation[END_REF]. The organizers of INEX SBS have used Amazon 8 and Librarything(LT) to provide a document collection which consists of 2.8 million books. This collection contains both textual and non-textual social information about books. Textual social information consists of users' reviews and tags that are respectively extracted from Amazon and LT. As to non-textual information like rating, number of time a book is catalogued, etc, they are extracted from LT whereas ratings, number of reviews, of total votes and helpful votes are extracted from Amazon. In order to evaluate systems in SBS, a set of topics with relevance assessments are provided. These topics are based on discussion threads from LT forums, where users express their needs and ask suggestions and recommendations about books to other forum users. These topics contain many fields : group, title, narrative and examples (see Fig. 1). The group field designates the discussion group in which a user posts their thread, the title is the short representation of the topic which often contains a brief summary about the user's need. Narrative is a long representation of the topic in which the user utilizes natural language to explain their needs in details. As to the field examples, it consists of some similar books add by some LT users in order to indicate the kind of books they want.

Related Works

Since its first edition in 2011, participants of SBS have proposed approaches and submitted their results (runs) to get them evaluated. The majority of these approaches use textual information (tags and reviews) to estimate the initial score of books then non-textual information like (rating, popularity, number of tags, number of reviews, profile of user, etc) is used to re-rank the initial ranking and improve the initial text-based search results. The authors in [START_REF] Bogers | Rslis at inex 2012: Social book search track[END_REF] used the language modeling with Jelinek-Mercer (JM) smoothing to build the initial content based results. They also experimented different re-ranking approaches using different information sources, such as user ratings, tags, authorship, and Amazon's similar products. The results show that the re-ranking approaches are often successful.

In [START_REF] Feng | Ustb at social book search 2016 suggestion task: Active books set and re-ranking[END_REF][START_REF] Zhang | Ustb at inex2014: Social book search track[END_REF], the probability of the query content produced by the language model is used to rank the documents based on textual information. Ratings, number of reviews, popularity and high frequent books are then used to re-rank the initial ranking. Finally, random forest was used to learn the different combinations of scores and the results are better than the initial ranking.

In [START_REF] Hafsi | Lahc at inex 2014: Social book search track[END_REF] the BM25F model was used to optimize the weight of the four book fields (title, summary, tags, reviews). Popularity, reputation, ratings and similarity between users were used to improve the results however, their integration did not give any improvement. [START_REF] Imhof | Multimodal social book search[END_REF] employ the textual model BM25 and enhance it by using social signals such as rating. Finally, they applied a random forest learning to improve the results by including non-textual modalities like price and number of pages according to the user's preferences. This approach improves the results and shows good performances.

The authors in [START_REF] Benkoussas | Collaborative filtering for book recommandation[END_REF] used two textual retrieval model Divergence from randomness model(DFR) and Sequential Dependence Model(SDM) before combining the results with the ratings of books. The best results were obtained when using DFR model with textual information only. In [START_REF] Chaa | Verbose query reduction by learning to rank for social book search track[END_REF][START_REF] Chaa | New technique to deal with verbose queries in social book search[END_REF], the authors use only tags as textual information about books. They investigate the representation of the query by transforming the long verbose queries to a reduced queries before applying the BM15 retrieval model. The result of this approach shows also good performance despite not using Amazon reviews at all.

To sum up, the majority of these works used textual information as a baseline before using non-textual information for reranking. As for indexing, they used users' tags and reviews together to represent books. However, some of them [START_REF] Bogers | Rslis at inex 2012: Social book search track[END_REF][START_REF] Hafsi | Lahc at inex 2014: Social book search track[END_REF][START_REF] Koolen | user reviews in the search index? that'll never work![END_REF][START_REF] Koolen | Social book search: comparing topical relevance judgements and book suggestions for evaluation[END_REF] studied the influence of using tags only, reviews only or tags with reviews as books representation on the retrieval performance. To our knowledge, there is no study that has adapted a specific model for each document representation. Hence, when combining tags and reviews, the authors in [START_REF] Bogers | Rslis at inex 2012: Social book search track[END_REF][START_REF] Hafsi | Lahc at inex 2014: Social book search track[END_REF][START_REF] Koolen | user reviews in the search index? that'll never work![END_REF][START_REF] Koolen | Social book search: comparing topical relevance judgements and book suggestions for evaluation[END_REF] used the same retrieval model to compute scores and rank documents for each query.

In this paper, we use both textual and non textual information of books as used in the studies mentioned above. However, instead of using one index for all textual fields and one function to compute scores of documents, we build a separate index for each field. We build two models, Tag Based Model (TBM) and Review Based Model (RBM) which respectively use the index of tags and the index of reviews. Then, we analyse the different influences of using tags and reviews on the settings of retrieval parameters as well as on the retrieval effectiveness. Then, we combine the results of the two models to form the textual score. Finally, we combine textual and non textual score to form the final result.

Experimental Setup

In our experiments we have used the collection provided by the organizers of SBS. For documents, we have considered two kinds of representations : LT users' tags and Amazon users' reviews. Table 1 summarizes the statistics of the collection. The numbers of tokens and single terms are calculated after stopword removal and Porter stemming. Table 1 shows that the number of books that have been tagged is greater than the number of books that have been reviewed however, the number of tokens (all occurrences ) in reviews are greater than those of tags. This because, for reviews the users use natural language to give their opinions and speak freely about books. However for tags the users assign a few keywords or terms to describe books. As to queries, we have used all the six-year topics (1646 topics) provided by SBS from 2011 to 2016. Because each query is composed of title and narrative, we have considered two types of queries: short and long queries. The short query is constructed from the title of the topic while the long query is the narrative and title+narrative. The prime target of our study is to investigate the impact of using the different query representations as well as the two representations of documents on the retrieval performance. To achieve this, we built three indexes, the first contains tags assigned by users to books in LT, the second contains reviews extracted from Amazon while the last one merges tags and reviews in the same index. The Terrier IR platform [START_REF] Ounis | Terrier: A high performance and scalable information retrieval platform[END_REF] was used to index the collection by applying basic stopword filtering and Porter stemming algorithm. The BM25 model [START_REF] Robertson | Experimentation as a way of life: Okapi at trec[END_REF] was used for querying. Using the BM25 model, the relevance score of a book d for query Q is given by:

S(d, Q) = t∈Q (k 1 + 1)tf td tf td + k 1 (1 -b + b. |D| avgdl )
.idf (t).

(k 3 + 1)tf tq k 3 + tf tq (1)
Where tf td and tf tq are respectively the frequency of term t in document d and in query Q. the three free parameters of the function are : k 1 and k 3 that respectively controls term frequency scaling of the document and the query, the parameter b controls the document length normalization. idf (t) is the inverse document frequency of term t, given as follow:

idf (t) = log |D| -df (t) + 0.5 df (t) + 0.5 (2)
Where df(t) is the number of documents where the term t appears, and |D| is the number of documents in the collection.

Length Normalization vs Document and Query Representation

Document length normalization is a technique that attempts to adjust the term frequency or the relevance score in order to normalize the effect of document length on the document ranking. Several works [START_REF] Cummins | The effect of query length on normalisation in information retrieval[END_REF][START_REF] He | Term frequency normalisation tuning for bm25 and dfr models[END_REF][START_REF] Lv | Lower-bounding term frequency normalization[END_REF] show that this technique has an important impact on the performance of the model. In order to determine the sensitivity of the model performance as to the length normalization, using the different cases of document representations as well as query representations, we have set the BM25 standard parameters for k1 and k3 (k1 = 2, k3 = 1000) and varied the length normalization parameter b from 0 to 0.75 (in steps of 0.05) then we evaluated the results in terms of NDCG@10 on both indexes (tags and reviews). The results obtained for all queries together are shown in Figure 2. From Figure 2, we can see that in TBM, the performance of the model is very sensitive to the normalization of the length of the document. NDCG@10 drops from 0.1415 (b=0) to 0.0375 (b=0.75) in the case of title (short queries). It increases from 0.0368 (b=0) to 0.0961 (b=0.05) then drops to 0.0321 (b=0.75) when using narrative (long queries).The same sensitivity of the model was shown When combining title and narrative to represent the query, NDCG@10 increases from 0.0695 (b=0) to 0.1429 (b=0.1) and then drops to 0.0490 (b=0.75).

In the case of RBM, the model performance is not very sensitive to the normalization of the document length compared to TBM. The values of NDCG@10 are very close so there is no big difference between them especially when varying b from 0 to 0.35. NDCG@10 is 0.1096 (b=0) and 0.1003 (b=0.35) in the case of short queries. The same measure increases from 0.0731 (b=0) to 0.1042 (b=0.25) and then decreases to 0.0803 (b=0.75) in the case of long queries. In the same way, the NDCG@10 increases from 0.0957 (b=0) to 0.1370 (b=0.25) and then decreases to 0.1062 (b=0.75) when combining title and narrative.

The same figure clearly shows that in short queries the best performance is obtained when b is very small, b=0 for the tags (no length normalization is required) and b=0.1 for reviews. However, when using long queries we find that the best performance is obtained when b=0.1 for tags and b=0.25 for reviews (length normalization is required). We can conclude that the setting parameters of the model are not the same in the TBM model or in the RBM model. Each of them has its specific parameters.

Because these evaluations are obtained for all queries together and to make our results more meaningful, we have decided to learn the parameter b by selecting the set of topics of each year as the testing set and the remaining sets of topics (of other years) as the training set. The evaluation in term of NDCG@10 of the results, for Single index(b=0.3), TBM model (b=0.05) and RBM Model(b=0.25), obtained after training and testing is shown in Table 2.

As shown in Table 2, TBM gave the best results, compared to RBM and Single index, except for 2011 when RBM gave the best results. By the way, we were surprised that, in most cases, using tags only (TBM) or reviews only (RBM) to index documents gave better results compared to when using tags and reviews together. The only exceptional case was in 2011 where single index (NDCG=0.2810) was better than TBM (NDCG=0.2459).

Query Expansion Using Example Books

In addition to title and narrative, the topics of SBS contain an example field which includes a list of books that the user has mentioned in their topic to show the kind of books he/she requests. In our approach, we consider this list of books as relevant books; therefore we use the query expansion technique to extract the highly relevant terms of these and expand the original query. The Rocchio function [START_REF] Rocchio | Relevance feedback in information retrieval[END_REF] used to expand the query is as follows:

- → Q new = - → Q T +N + β |EXP | d∈EXP - → d ( 3
)
Where -→ Q new is the expansion query, -→ Q T +N is the original query represented by Title+Narrative.

-→ d denotes the weighted term vector of the example book d using the default term weighting model Bo1(Bose-Einstein 1). EXP is the set of example books and |EXP| is the number of example books mentioned in the topic. The function was used with their default parameter settings β =0.4, and the number of terms selected from each example book was set to 10.

Table 3 shows the results after the query expansion for the Single index, TBM model as well as for the RBM model. The same table shows that the query expansion technique has improved the results for the three indexes. We also notice that TBM gave the best results for all cases except for 2011 when RBM gave the best NDCG@10 (0.3418) and for 2013 when Single index has an NDCG@10=0.1496.

Combining the Scores of the Two Models

Once the parameters of TBM and RBM are optimized, a combination of the two scores obtained is necessary to obtain the final textual score of each book with respect to the query. The linear combination function is as follows:

S(d, Q) = α.S T BM (d, Q) + (1 -α).S RBM (d, Q) (4)
Where S(d, Q) is the final score of document d with respect to query Q. S T BM (d, Q) and S RBM (d, Q) are the scores of document d with respect to query Q respectively obtained from TBM and RBM models. The query Q is represented by Title+Narrative and expanded using similar books as explained in the previous section. α [0 1] is a free parameter that controls the weight of the two models.This parameter was tuned using six-fold cross-validation in the same way it was performed to tuned the parameter b as indicated in section 4.1. Thus, a single set of topics of one year is used for testing the model and the remaining five-year topics are used as a training set. After the process is repeated six times (once for each year), the results have been summed up in Table 4. This table shows the results in term of NDCG@10 obtained from the combination and compares them to the results obtained from the previous experiments. The best results of the combination are obtained when α ∈ is set to 0.4. From the results, we note that for all years the combination of the two scores gave good results, compared to the results of each index. For all queries, the NDCG@10 increased from 0.1619 (single model), 0.1686 (TBM model) and 0.1639 (RBM model) to 0.2091 when combining the two scores so there is an improvement of 29.15%, 24.02% and 27.54% compared to the three models. This results shows that the technique of using two separate indexes and combining the results is an effective technique to get best results.

Non textual Information to Re-rank documents

The non textual information of documents in social media like number of likes, number of rating, number of times the document was catalogued or rated represents an important information and can be used to re-rank the documents to improve the results. Several Re-ranking approaches were proposed by [START_REF] Zhang | Ustb at inex2014: Social book search track[END_REF] at INEX2014 and [START_REF] Bogers | Rslis at inex 2012: Social book search track[END_REF] in 2012 , which proved to be effective. In this paper we combine the textual score obtained above with the the number of times the book was rated to re-rank the documents. The combination of scores is calculated by the flowing function:

S(d, Q) = λ.S T extual (d, Q) + (1 -λ).S N on-textual (d) (5)
Where S(d, Q) is the final score of document d with respect to query Q. S T extual (d, Q) is the textual score of document d obtained by combining the scores of TBM and RBM as explained in the previous section. λ [0 1] is a free parameter that controls the weight of the two scores. S N on-textual (d) is the normalized non-textual score of document d calculated as follow:

S N on-textual = N b_rated -M in_nb_rated M ax_nb_rated -M in_nb_rated ( 6
)
Where nb_rated is the number of times the document is rated, Min_nb_rated and Max_nb_rated is the minimum and the maximum of the number of times that all books of the collection have. Table 5 shows the results obtained when using the non-textual information for re-ranking. The best results obtained when λ ∈ was set to 0.9. From this Table we show that when using the Non-Textual information to re-rank documents gave good results of all years of topics, except in 2011 when the NDCG@10(0.3595) obtained by using the textual information only is better than the NDCG@10(0.3551) after re-ranking the documents.

Finally, we compare the performance of our best results, obtained by our approach, to the best official and non official runs. The official runs are those that have been submitted by participants to SBS during the six last years. The non official runs are the results obtained recently in the works of [START_REF] Yin | Isart: A generic framework for searching books with social information[END_REF][START_REF] Zhang | Building your own reading list anytime via embedding relevance, quality, timeliness and diversity[END_REF]. Table 6 represents the comparative results. The results show that the NDCG@10 value of our approach is better than the best official runs of the four years (2011, 2012, 2013 and 2014) but lower than the best runs of the two years (2015 and 2016). This table shows also that our approach gave good results in the three first years compared to the non official runs.

Analysis

From the results, we have noticed that TBM model requires a smaller value of b for optimal performance whether for short or long queries compared to RBM model. This led us to ask the following question :

Why document length normalization is required when using reviews as a document representation and not required when using tags as document representation?

As an answer to the question, this may be due to the fact that the users' reviews are a natural language text in which users can repeat freely the same term many times in the same review. Hence, the frequency value of any given term present in the query will be increased thereby increasing the relevance scores of long documents that contain long reviews. That is why document length normalization is required to penalize very long documents. Contrary to the reviews, the users' tags are keywords that users assign them to documents and the same user can not assign the same term many times to the same document. Hence, for long documents when the frequency value of any query term is great this means that this document was tagged by several users using the same term then it is relevant to the topic and it is unreasonable to penalize them. Therefore, the document length normalization is not required for tag representation.

In this paper, we have studied the exploitation of user tags and reviews in social book search. Three indexes have been created, the first for tags, the second for reviews and the third that merges the two. After several experimentations, we concluded that the Tag Based Model does not require a document length normalization especially for short queries. The best results of this model were obtained when b has very low values, This may be due to the fact that the number of tags assigned to books by users cannot be regarded as a length of text documents. However, the Review Based Model requires the document length normalization, this may be because the user reviews are long and be seen as a classical textual document. We have noticed that using two indexes for tags and reviews separately and combining the results of the models gives good and better results compared to when using a single index and a single model. We have also demonstrated that the proposed combination has given satisfactory results, especially when using non-textual information to re-rank documents, and outperforms the best runs submitted to SBS in the four first years.

Fig. 2 .

 2 Fig. 2. Sensitivity of ndcg@10 for length normalization for Tag based model and Review based model.

  <topic id="41306"> <request>Spanish Civil War : For Whom The Bell Tolls :: French Revolution: ?? I don't think there's a right answer, I'm just looking for suggestions for a good fictional book set during the French Revolution. (My interest was piqued after reading the Jim Shepard short story "Sans Farine" in The Best American Short Stories

	2007.) <request>
	<group>Book talk</group>
	<title>Fill in this historical fiction analogy</title>
	<examples>
	<example>
	<booktitle>For Whom the Bell Tolls</booktitle>
	<author>Ernest Hemingway</author>
	<workid>10084</workid>
	</example>
	<example>
	<booktitle>The Best American Short Stories 2007</booktitle>
	<author>Stephen King</author>
	<workid>3539369</workid>
	</example>
	</examples>
	Fig. 1. XML file representing an example topic in Social Book Search.

Table 1 .

 1 Statistics on tags and reviews of SBS collection

	Number of books	2,781,400
	Number of books that have been reviewed at least once	1,915,336
	Number of books that have been tagged at least once	2,306,368
	Number of tokens in reviews	1,161,240,462
	Number of single terms in reviews	1,135,910
	Number of tokens in tags	246,552,598
	Number of single terms in tags	194,487

Table 2 .

 2 Results in term of NDCG@10 of the three index , using Title+Narrative as a representation of the topic; best results are shown in bold.

	Year	Single Index	TBM	RBM
	2011	0.2810	0.2459	0.3007
	2012	0.1387	0.2012	0.1479
	2013	0.1143	0.1193	0.1191
	2014	0.1169	0.1291	0.1175
	2015	0.0682	0.1222	0.0794
	2016	0.0803	0.0951	0.0906
	All	0.1307	0.1429	0.1370

Table 3 .

 3 NDCG@10 results obtained after applying a query expansion technique.

		Single Index	TBM	RBM
	Year	Before expansion	After expansion	Before expansion	After expansion	Before expansion	After expansion
	2011 0.281	0.3310	0.2459	0.2725	0.3007 0.3418
	2012 0.1387	0.1667	0.2012 0.2341	0.1479	0.1754
	2013 0.1143	0.1496	0.1193	0.1452	0.1191	0.1478
	2014 0.1169	0.1436	0.1291	0.1525	0.1175	0.1423
	2015 0.0682	0.0887	0.1222	0.1409	0.0794	0.0907
	2016 0.0803	0.1088	0.0951	0.1367	0.0906	0.1205
	All	0.1307	0.1619	0.1429	0.1686	0.1370	0.1639

Table 4 .

 4 NDCG@10 obtained by the combination after tuning the parameter α using six-fold cross-validation compared to a single index, RBM model and TBM Model.

	Asterisks indicate statistically significant differences compared with single index (Stu-
	dent's t-test, P < 0.05)
	Year Single Index TBM	RBM Combination
	2011	0.3310	0.2725 0.3418	0.3595
	2012	0.1667	0.2341 0.1754	0.2425*
	2013	0.1496	0.1452 0.1478	0.1888*
	2014	0.1436	0.1525 0.1423	0.1886*
	2015	0.0887	0.1409 0.0907	0.1526*
	2016	0.1088	0.1367 0.1205	0.1793*
	All	0.1619	0.1686 0.1639	0.2091*

Table 5 .

 5 NDCG@10 obtained by the combination of Textual and Non-textual scores

		Textual score only	Textual and Non-Textual scores
	2011	0.3595	0.3551
	2012	0.2425	0.2469
	2013	0.1888	0.1907
	2014	0.1886	0.1943
	2015	0.1526	0.1585
	2016	0.1793	0.1935
	All	0.2091	0.2133

Table 6 .

 6 Comparison between the results of our approach and the best runs submitted to the different years of SBS; best results are shown in bold.

		Our approach Best non official runs Best official runs
	2011	0.3551	0.3423	0.3101
	2012	0.2469	0.2325	0.1456
	2013	0.1907	0.1856	0.1361
	2014	0.1943	0.1960	0.1420
	2015	0.1585	0.2040	0.1870
	2016	0.1935	0.2157	0.2157

www.librarything.com

www.goodreads.com

www.social-book-search.humanities.uva.nl

We use the terms "document" and "book" interchangeably in this paper.

www.amazon.com