Gilles Hubert
email: gilles.hubert@irit.fr

Jose G Moreno
email: jose.moreno@irit.fr

Karen Pinel-Sauvagnat
email: karen.sauvagnat@irit.fr

Yoann Pitarch
email: yoann.pitarch@irit.fr

Some thoughts from IRIT about the scenario A of the TREC RTS 2016 and 2017 tracks ⋆

The TREC Real-Time Summarization (RTS) track provides a framework for evaluating systems monitoring the Twitter stream and pushing tweets to users according to given profiles. It includes metrics, files, settings and hypothesis provided by the organizers. In this work, we perform a thorough analysis of each component of the framework used for batch evaluation of scenario A in 2016 and 2017. We found some weaknesses of the metrics and took advantage of these limitations to submit our official run in the 2017 edition. The good evaluation results validate our findings. This paper also gives clear recommendations to fairly reuse the collection.

Introduction

A common usage of Twitter is to watch other users' tweets and never post anything. This usage scenario considers Twitter as a real time information source by scanning incessantly the tweet stream. The users adopting this usage aim to catch new (information they did not hear about before), fresh (information that appeared very recently) and precise (information that concerns them) information. There is growing interest in systems that could address these issues by providing information that satisfy this type of users with respect to their information needs.

The TREC campaign took an interest in the evaluation of such systems through various tracks and notably the ongoing Real-Time Summarization (RTS) track. As usual in information retrieval and evaluation campaigns, the researchers who tackle the issues on which focuses a track test their approaches using the framework provided for the track during the campaign period. Many researchers also test their solutions using the framework after the campaign period.

Our participations to these successive tracks have motivated a thorough analysis of the provided evaluation frameworks. This paper presents our main findings about the batch evaluation of the scenario A of the 2016 and 2017 benchmarks. On the one hand, it highlights some limitations of the provided evaluation framework with respect to the organizers' settings. On the other hand, it identifies precautions to take when reusing the evaluation framework after the campaign period. Leaving aside these precautions would lead to erroneous evaluation results and invalidate conclusions on system performance.

The remainder of this paper is organized as follows. Section 2 presents an overview of the TREC RTS track while Section 3 describes the metrics defined for the evaluations of systems corresponding to the track scenario A. The limitations highlighted on the evaluation framework are discussed in Sections 5 and 6. Section 8 introduces the precautions to take to obtain valid results when reusing the evaluation framework after the TREC campaign. Finally, Section 9 concludes the paper.

Introduced in 2016 and continued in 2017, the RTS track merges some previous TREC tracks: the Microblog (MB) track ran from 2010 to 2015 and the Temporal Summarization (TS) track run from 2013 to 2015. It intends to promote the development of systems that automatically monitor a document stream to keep the user up-to-date on topics of interest, by proposing a framework to evaluate such systems. The track considers two scenarios: scenario A -Push notifications -and scenario B -Email digest. The scenario A corresponds to the systems intended to send immediately the posts identified as relevant. The scenario B corresponds to systems intended to send once a day a summarization of the relevant posts of the day. This paper sheds some light on the evaluation framework defined for the scenario A. Each participant to the task must process a publicly accessible sample provided by Twitter which corresponds to the 1 % of the total available tweets. The evaluation period is partitioned in days, making 8 or 10 days long the evaluation window. To identify relevant tweets, a set of profiles is provided. Each profile (called topic in the TREC jargon) is composed of a title, a description and a narrative of the interest profiles. Table 1 provides some statistics about the 2016 and 2017 tasks. Each system must push at most 10 tweets per profile per day to a central system called the broker. Note that silence of a system is a desired effect when there are no relevant tweets during a day.

Two ways of evaluation were performed: online judgments and batch judgments. The earlier was performed during the evaluation period and the latter was performed once the challenge was over. Some works studied these two ways of evaluation and showed they are correlated [START_REF] Roegiest | Online in-situ interleaved evaluation of real-time push notification systems[END_REF][START_REF] Tan | An exploration of evaluation metrics for mobile push notifications[END_REF]. This work is interested only in the latter due to the reusability problems already identified in the earlier one [START_REF] Tan | On the reusability of "living labs" test collections: A case study of real-time summarization[END_REF]. In order to perform the batch judgments, a pool of tweets was built using all the pushed tweets in both scenarios A and B. The combined set of tweets was annotated following a two-step methodology. Given a tweet, assessors first assigned a relevance score. In all editions of this task, three levels of relevance (not relevant, relevant and very relevant) were considered. However without loss of generality, we consider only two levels of relevance to simplify our study, i.e., relevant tweets are considered as very relevant. Then, a unique cluster1 was assigned to each relevant tweet. A tweet is considered relevant if its content is related to one profile. The clusters were found following the Tweet Timeline Generation (TTG) approach [START_REF] Wang | Assessor differences and user preferences in tweet timeline generation[END_REF] which takes into account the creation timestamp to sort relevant tweets. Tweets are examined one per one traveling from past to future. A new cluster is created if the current tweet content is substantially dissimilar to all the previous tweets seen. All clusters are then considered equally important in the evaluation metrics.

Metrics

The RTS track in its guidelines asks for effectiveness (tweet quality) and efficiency (no latency). As participant systems might favour effectiveness or efficiency depending on their approaches, the organizers decided from 2016 to compute metrics for quality and latency separately [START_REF] Lin | Overview of the trec 2016 real-time summarization track[END_REF].

Notations and Preliminary Definition

The notations used in this paper are summarized in Table 2. The creation date of the tweet t Πi(t)

The date at which the tweet t has been pushed to the user by the system Si W = {w1, . . . , wT } The set of temporal windows, i.e., the set of days considered during the evaluation campaign N

The maximum number of tweets to push per window Ti(wj)

The list of tweets published during the window wj and pushed by the system Si ordered by their Θ(t) Ri

The set of relevant tweets pushed by the system Si

A key concept for all the metrics is how relevance is defined. A tweet is considered as relevant for a system S i if it satisfies two criteria: it is contained in a relevant cluster and it is the first tweet returned by S i for this cluster. Once a tweet from a cluster has been retrieved, all the other tweets from the same cluster are redundant and automatically become not relevant [START_REF] Lin | Overview of the trec-2015 microblog track[END_REF]. This implies that the relevance of a tweet is system-dependent.

Gain-Oriented Metrics

Gain. The three metrics proposed to evaluate quality are based on the concept of gain, i.e., the usefulness of a tweet in the list of the tweets pushed by the system. The way the gain is evaluated is thus decisive. Given a time window w j and T i (w j ), i.e., the tweets returned by the system S i published during w j , the gain G(w j , S i ) is evaluated as follows:

G(S i , w j ) = t∈Ti(wj ) g(t) (1)
where g(t) is the gain of the tweet t: g(t) = 1 if the tweet is relevant, g(t) = 0 otherwise, i.e., t is non relevant or redundant. It should be noted that this definition has been clarified from [START_REF]Trec 2017 evaluation guidelines[END_REF][START_REF] Lin | Overview of the trec 2016 real-time summarization track[END_REF] by specifying that the tweets considered during w j are picked using Θ(t) rather than Π i (t). We now detail the official metrics that rely on the gain.

Expected gain. The expected gain metric, denoted by EG, is adapted from [START_REF] Aslam | Trec 2015 temporal summarization track overview[END_REF]. Given a time window w j , it is evaluated as:

EG(w j , S i ) = 1 |T i (w j )| • G(S i , w j ) (2)
where |T i (w j )| is the number of tweets returned by S i and published during w j .

An important question about this metric is how to score systems during the so-called silent days, i.e., the days where no relevant tweets are published. Some variants of the EG metric have been introduced differing on how the silent days are considered:

-EG-0 in which systems receive a gain of 0 during the silent days no matter the tweets they returned. -EG-1 in which systems receive a gain of 1 during the silent days when they do not return any tweet published during the day, 0 otherwise. It should be noted that this definition has been slightly extended from [START_REF] Lin | Overview of the trec 2016 real-time summarization track[END_REF] to perfectly fit with the evaluation tool. This will be further discussed in Section 5. -EG-p in which the proportion of tweets returned during a silent day is considered: a system receives a score of N -|t| N , where |t| is the number of non-relevant tweets published during the day and returned by the system. For instance, if a system pushes one tweet published during the day but not relevant (instead of 0), it gets a score of 0.9; two nonrelevant tweets imply a score of 0.8, etc. Similarly to EG-1, this definition has been slightly extended.

The way the silent days are considered is crucial, since a huge impact of silent vs. eventful days is observed in the evaluation [START_REF] Tan | An exploration of evaluation metrics for mobile push notifications[END_REF].

Normalized Cumulative Gain. Given a time window w j , the nCG metric is evaluated as follows:

nCG(w j , S i ) = 1 Z • G(S i , w j ) (3)
Z is the maximum possible gain (given the N tweets per day limit). As for EG, three variants are considered regarding how the silent days are taken into account: nCG-1, nCG-0, and nCG-p.

Gain Minus Pain. The GMP metric evaluates the utility of the run:

GMP (S i , w j ) = α G(S i , w j ) -(1 -α) • P (S i , w j ) (4)
The gain Gain G(S i , w j ) is computed in the same manner as above, the pain P (S i , w j ) is the number of non-relevant tweets published during w j and returned by the system S i , and α controls the balance between the two. Three α settings were considered: 0.33, 0.50, and 0.66.

Latency-Oriented Metric

The latency metric is defined as:

Latency(S k ) = t (•) i ∈R k Π k (t (•) i ) -Θ(t 1 i ) ( 5
)
where t

(•)

i is the oldest tweet pushed by the system S k for the cluster C i . In other terms, latency is evaluated only for tweets contributing to the gain as the difference between the time a tweet was pushed and the first tweet in the semantic cluster that the tweet belongs to.

!" #" $" !%" #!" #&" !" θ(') %( #!" ##" " # #" !# %" !)" #*" " $ !( !!( !$" !+" *" % $ % # % & % ' % ( ' ! ! Π ! " (') Π ! # (') ' # ! ' * ! ' ! !
Fig. 1: Examples of runs retrieved by S1 and S2 as well as the associated ground truth (GS). In this example, a time window wi lasts 50 seconds.

Table 3: Behaviours of the studied metrics with respect to the metrics.

Metrics Systems

S 1 S 2
EG-0 ( 1 3 * 1 + 0 + 0 + 1 3 * 1 + 1 1 * 1)/5 = 0.33 (0 + 1 1 * 1 + 0 + 1 1 * 1 + 1 1 * 1)/5 = 0.6 EG-1 ( 1 3 * 1 + 0 + 0 + 1 3 * 1 + 1 1 * 1)/5 = 0.33 (0 + 1 1 * 1 + 1 + 1 1 * 1 + 1 1 * 1)/5 = 0.8 EG-p ( 1 3 * 1 + 9 10 + 9 10 + 1 3 * 1 + 1 1 * 1)/5 = 0.69 (0 + 1 1 * 1 + 1 + 1 1 * 1 + 1 1 * 1)/5 = 0.8 nCG-0 ( 1 1 * 1 + 0 + 0 + 1 1 * 1 + 1 2 * 1)/5 = 0.5 (0 + 1 1 * 1 + 0 + 1 1 * 1 + 1 2 * 1)/5 = 0.5 nCG-1 ( 1 1 * 1 + 0 + 0 + 1 1 * 1 + 1 2 * 1)/5 = 0.5 (0 + 1 1 * 1 + 1 + 1 1 * 1 + 1 2 * 1)/5 = 0.7 nCG-p (( 1 1 * 1 + 9 10 + 9 10 + 1 1 * 1 + 1 2 * 1)/5 = 0.86 (0 + 1 1 * 1 + 1 + 1 1 * 1 + 1 2 *

Metrics Exemplifications

Fig. 1 and Table 3 run through some examples of systems and the way the official metrics are calculated. The results presented in Table 3 are decomposed and were checked using the 2016 and 2017 official evaluation tools of the track2 .

In the EG family of metrics, the gain in a time window is divided by the number of tweets returned by the system and published during the time window. For instance and considering the system S 1 , the gain in the time window w 1 is divided by 3 while it is divided by 1 in w 2 . In the nCG family of metrics, the gain in a time window is divided by the optimal gain. For instance, in the time window w 5 the optimal gain is 2 (2 new clusters C 3 and C 4 ), but neither S 1 nor S 2 reach this optimal gain.

If we now consider the silent days, EG-1 and nCG-1 reward the systems for returning no tweets, and strongly penalize them otherwise. For instance, S 1 breaks the silence during w 3 and thus obtains a score of 0 for this window. The silent days can be different from one system to another: S 1 breaks the silence during w 2 since t 3 1 is not relevant in this case (the C 1 cluster has already been retrieved), whereas this is not the case for S 2 for which C 1 was not retrieved at this time. Conversely, S 1 and S 2 receive a score of 0 for w 3 considering the EG-0 and nCG-0 metrics, whereas S 2 has a perfect behavior during this window. Whatever the systems return, the silent days are associated with a score of 0, and it never hurts to push tweets. The "silent days effect" is lowered for the evaluation of EG-p et nCG-p: S 1 receives a score of 9/10 on w 2 and w 3 (whereas it receives a score of 0 for EG-0, nCG-0, EG-1, and nCG-1 metrics).

For all the gain-oriented metrics, a tweet participates to the gain of the time window on which it was published (and not on which it was pushed by the systems). As a consequence, if we consider the system S 2 and the window w 5 , the t 1 2 tweet participates to the gain of w 4 (which is the window in which it was published). S 2 has thus a non-zero score for w 4 whereas it did not return any tweet. Another point to discuss relates to redundant tweets. As expected, t 2 1 returned by S 1 during w 1 is considered as not relevant, only t 1 1 participates to the gain.

At last, the Latency metric is evaluated independently of the time windows as the difference between the first tweet found in the cluster by the system and the publication date of the first tweet in the cluster in the gold standard. A side effect of this metric is that a perfect latency can be obtained without returning any relevant tweets. Table 4 provides additional information about the metrics. The EG metrics attempt to capture precision while the nCG ones are recall-oriented. The GMP metrics aim to fill the gap between these two contradictory objectives and thus represent a trade-off between precision and recall. As stated in [START_REF] Roegiest | Online in-situ interleaved evaluation of real-time push notification systems[END_REF], EG-0 and nCG-0 metrics are poorly formulated metrics and were thus abandoned in 2017. The gain-oriented metrics are computed for each interest profile and each window w j . The score for a competitor is the mean of the scores for each day over all the profiles. Since each profile contains the same number of days, there is no distinction between micro-vs. macro-averages. The EG-1 and EG-p metrics were respectively considered as the official metrics in 2016 and 2017.

Metric Integration in the Evaluation Framework

Hypotheses and Settings of the Evaluation Framework

The metric evaluation is based on two hypotheses assumed by the organizers.

H1 -Redundant information is non relevant. As mentioned in [START_REF] Lin | Overview of the trec 2016 real-time summarization track[END_REF]:

Once a tweet from a cluster is retrieved, all other tweets from the same cluster automatically become not relevant. This penalizes systems for returning redundant information.

H2 -A perfect daily score is obtained when silence is respected. As mentioned in [START_REF] Lin | Overview of the trec 2016 real-time summarization track[END_REF]:

In the EG-1 and nCG-1 variants of the metrics, on a "silent day", the system receives a score of one (i.e., perfect score) if it does not push any tweets, or zero otherwise.

Note that since relevance is system-dependent, it implies that the silent days are systemdependent as well. Moreover, we would like to shed some light on two settings of the framework.

S1 -N = 10. This consists in forcing the systems to push a maximum of only 10 tweets per day and per profile. There is a twofold explanation for the value chosen for this parameter: first, to impose to the systems a realistic limit to the number of daily tweets that could be desired by a user and second, to impose a reasonable limit for the annotation phase.

S2 -Evaluation window. For the gain-oriented metrics, whatever the Π i (t) value for a tweet, only Θ(t) is considered for the evaluation of G(•, •) (see equation 1). In other terms, each returned tweet is sent back to its emission window. This can affect the systems that use buffering-based strategies as suggested by the guidelines. This setting is implicitly defined by the organizers since the latency metric is calculated separately from the main metrics.

Metrics Adequacy under RTS Hypotheses

We now refute the aforementioned hypotheses through 2 counterexamples. H1. Considering the example of Fig. 2 and Table 5, the system S 2 has higher scores than S 1 on the EG metrics, whereas both return results supposed as equivalent (the first tweet of the cluster C 1 during w 1 and respectively a redundant and non relevant tweet during w 2 ). S 1 is more penalized for returning a redundant tweet than a non relevant one. This thus violates H1.

!"

#"

!"

θ($) " # %" !& %" " $ % $ % # &" !& Π !" ($) Π !# ($) $ ! & $ ! ! $ ! ! $ ! ! $ & ! $ & ! Fig. 2:
Examples of runs retrieved by S1 and S2 as well as the associated ground truth (GS) with respect to H1. In this example, a time window wi lasts 50 seconds.

Metrics Systems Table 5: Behaviours of the studied metrics with respect to H1.

S 1 S 2 EG-0 ( 1 2 * 1 + 0)/2 = 0.25 ( 1 1 * 1 + 0)/2 = 0.5 EG-1 ( 1 2 * 1 + 0)/2 = 0.25 ( 1 1 * 1 + 0)/2 = 0.5 EG-p ( 1 2 * 1 + 0)/2 = 0.25 ( 1 1 * 1 + 0)/2 = 0.5 nCG-0 ( 1 1 * 1 + 1 1 * 0)/2 = 0.5 ( 1 1 * 1 + 0)/2 = 0.5 nCG-1 ( 1 1 * 1 + 0)/2 = 0.5 ( 1 1 * 1 + 0)/2 = 0.5 nCG-p ( 1 1 * 1 + 0/2 = 0.5 ( 1 1 * 1 + 0)/2 =

H2.

Considering the example of Fig. 3 and Table 6, w 2 is a silent day for both systems. S 2 breaks the silence with t 2 1 and however obtains a perfect score on this day, as S 1 which did not push any tweet. This thus violates H2.

!" !" θ(#) # $ # % $" # ! ! # $ ! " $ !$ Π !" (#) # ! ! !$ %" " % Π !# (#) # ! ! # $ !
Fig. 3: Examples of runs retrieved by S1 and S2 as well as the associated ground truth (GS) with respect to H2. In this example, a time window wi lasts 50 seconds.

Metrics Systems

S 1 S 2 EG-0 ( 1 1 * 1 + 0)/2 = 0.5 ( 1 2 * 1 + 0)/2 = 0.25 EG-1 ( 1 1 * 1 + 1)/2 = 1 ( 1 2 * 1 + 1)/2 = 0.75 EG-p ( 1 1 * 1 + 1)/2 = 1 ( 1 2 * 1 + 1)/2 = 0.75 nCG-0 ( 1 1 * 1 + 0)/2 = 0.5 ( 1 1 * 1 + 0)/2 = 0.5 nCG-1 ( 1 1 * 1 + 1)/2 = 1 ( 1 1 * 1 + 1)/2 = 1 nCG-p ( 1 1 * 1 + 1)/2 = 1 ( 1 1 * 1 + 1)/2 = 1
GMP.50 ((0.5 * 1) + (0))/2 = 0.25 ((0.5 * 1 -0.5 * 1) + (0))/2 = 0 Latency 2 2

Table 6: Behaviours of the studied metrics with respect to H2.

These two counterexamples are a side effect of S2.

6 Discussion of the RTS Settings S1. Allowing up to 10 tweets to be pushed per profile per day is an arbitrary limit of the task. In this section, we wonder how much the 2016 official metric would have been impacted by a modification of this value. With this aim in mind, we adopt the following methodology.

Given N ∈ {1..10}, we apply three distinct strategies to restrict the 2016 official runs to push only N tweets per profile per day and then calculate the average value of the EG-1 metric3 . The three strategies are as follows:

-In the First strategy, the first N tweets according to their pushing date are considered. This strategy intuitively simulates a change in the setting but no self-adaptation of the systems to this tighter constraint. -In the Gold strategy, N tweets are chosen to maximize the number of clusters and thus the official metric. Given a window w j , a profile p, and a system S i , if N is greater than the number of clusters retrieved by S i during w j for p, non-relevant tweets, i.e., either redundant or irrelevant tweets, are pushed to fulfill our requirement. Contrary to the First strategy, this strategy simulates a self-adaptation of the systems under this tighter constraint. -In the Random strategy, N tweets are randomly chosen. To overcome any bias in the sampling, 100 random draws were performed and the EG-1 metric values for all these 100 runs were then averaged. This strategy represents a fair compromise between the naive First and the optimal Gold strategies.

It should be noted that if less than N tweets have been pushed by a system S i for a profile p during a window w j , the set returned by any strategy is the same as the original set of pushed tweets. Finally, to fairly evaluate the impact of varying N , we compare the results obtained by these strategies to the average value of EG-1 in the official runs. The impact of the window size on the aforementioned strategies is shown in Fig. 4. Several strong conclusions can be drawn from these results. First, whatever the strategy, it is always beneficial to return very few tweets reinforcing the idea that EG-1 is essentially a precision-oriented metric. This conclusion is obviously even more true for the Gold strategy. Second, the performances of the First and Random strategies are very close suggesting that relevant tweets retrieved by the systems are uniformly distributed over the time window. Last but not least, pushing only one tweet adopting a very basic strategy, i.e., either the Random or the First one, without any guarantees that this tweet is relevant, provides similar or better results than pushing 3 or more tweets using a sophisticated strategy such as the Gold one, i.e., in which the number of retrieved clusters is maximized. This very interesting result reinforces our claim about the regrettable non-consideration of the coverage in the official metric. This point will be further discussed in Section 9.

!"#$! !"#$% !"&!! !"&!% !"&#! !"&#% !"&&! !"&&% !"&'! # & ' ( % ) * + $ #!
!"#$ % ,-./ 01234 567/-8 Fig. 4: Impact of S1 on the EG-1 metric S2. While the side effect of S2 have been assessed in Section 3, we now focus on its practical consequences during the TREC RTS 2016 track. For this purpose, we simulated an alternative evaluation framework in which the tweets are not sent back to their publication window. We observe from these statistics that:

very few tweets have been pushed in another window than their creation window. Specifically, this concerns only 0.12 % (53/45751) of the pushed tweets all the systems taken together. -these 53 tweets have been pushed by 8 different systems over the 41 runs. Notably, one of these systems have pushed 42 tweets among the 53 tweets while the other 7 systems have pushed only 1 or 2 tweets outside their creation window. -due to the rarity of these push window gaps, there is no differences in the rankings whatever the date taken into account for the evaluation.

Note that S2 could theoretically impact the performances of the systems, but this situation is not observed in the 2016 runs. Moreover, S2 may send back tweets to a window without any restrictions and making it greater than N , calling into question S1.

Experimental validation of the metrics limitations

We submitted to the 2017 official track one baseline run which exploited the above-discussed conclusions about the evaluation metrics. Its principle is as follows: after a standard preprocessing step, i.e., stopwords removal and stemming, the first tweet of the day containing all the query terms is returned. We thus return at most one tweet per day and per profile.

The results of our run for mobile and batch evaluation are respectively presented in Tables 7 and8. Our run, although being very simple, can be considered as a very strong baseline. We are ranked 2nd and 4th for the mobile and batch evaluation, while maintaining latency as low as possible.

These a posteriori results clearly confirm the need for a deeper reflection and discussion about the official batch metrics and their possible biases. Since mobile evaluation also seems to be concerned by the bias, we advocate for its inclusion in future discussions.

Reusability of the Test Collection

In order to enable comparison of new solutions against the TREC RTS 2016 and 2017 results, the organizers publicly provide an evaluation script as well as 3 ground truth files: (i) the qrels file that contains the relevance level of each tweet from the pool, (ii) the cluster file that gives cluster for each profile and (iii) the epoch file that contains the publication date of tweets from the pool. We conducted a standard "leaved-one out" analysis to evaluate the reusability of the 2016 collection. To do so, we simulated a rerun setup for all the 41 runs submitted during 2016 and evaluated them using the official metric, EG-1. The ground truth files, i.e., cluster, qrels, and epoch files, were created for each run i as if it has not taken part in the track by removing its unique tweets. For each of these new 41 evaluation files, an alternative ranking i was obtained using the EG-1 metric. The official ranking of each run i was then compared to this new ranking i in order to determine how effective would have been this run i in a rerun setup. The position of each run i in the ranking i showed either improvement or no variability with respect to its position in the original ranking, resulting in an average gain of 2.1 positions. This very surprising result has motivated a deep analysis of the evaluation tool. We observed a very odd behavior on how the unassessed tweets, i.e., the tweets that are not referenced in the ground truth files, are considered. Indeed, such tweets are simply ignored instead of being considered as irrelevant as traditionally done in classical evaluation setups. This point is even more problematic since the way the runs deal with the silent days is crucial for the calculation of gain-oriented metrics. By decreasing the number of tweets per profile/day and increasing chances to respect the silent days, the performances of new runs in the rerun setup are artificially increased. This situation is only attenuated, but still not solved, thanks to the introduction of the EG-p and nCG-p metrics in 2017. However, ignoring the tweets that must be considered as not relevant will still increase the score obtained by those metrics during the silent days. This bias in the (re)evaluation can be solved by including all the tweets of the Twitter stream during the evaluation period (11.5 M tweets) in the epoch file. In this case, our results showed a different ranking behaviour. None of the runs improved its position in its respective ranking, dealing with an average lost of 0.6 positions when compared to their original position.

Regarding the settings, we would like to draw attention to the fact that S1 is under the responsibility of each user of this collection. This setting was automatically handled by the organizers through the broker during the task. Not respecting this limit during the rerunning leads to underestimated performance since the gain is calculated only over the first 10 tweets but it is normalized by the total number of tweets, which could be greater than 10. Contrary to S1, S2 is always applied without user intervention.

Finally, users of the collection must consider analysis and remarks presented in Sections 5 and 6 because they are also valid under the rerun setup. We confirm the reusability of the 2016 and 2017 collections only under the aforementioned conditions, in particular, use of a complete epoch file and strict application of S1.

Recommendation and Conclusion

To conclude, we would like to summarize our main findings in this paper:

we clarified some definitions and assumptions of the track guidelines. We highlight here two of them, which are not clearly stated in the guidelines and overviews of the track although crucial for a good understanding of the evaluation framework. Only a deep analysis of the evaluation tool lead us to these conclusions, causing us to believe that some participants may not be conscious of these findings:

• the evaluation window used in EG and nCG metrics is not the window corresponding to the tweet push-timestamp. Each returned tweet is sent back to its emission window, which significantly impacts the way metrics are evaluated. • silent days are system-dependent. This is thus non-sense to elaborate approaches that try to detect silent days independently of already returned tweets. -we shed the light on the fact that coverage is not really evaluated by the official metrics.

The systems would better return few tweets that are very likely relevant to optimize the metrics. Trying to maximize the coverage and thus returning many tweets will probably lead to a result degradation. As a consequence, when developing a system for the track, all the improvements against the metrics should be compared to a very simple run returning at most one tweet per time window. This behavior of the results has already been noticed by the track organizers [START_REF] Qian | Interleaved evaluation for retrospective summarization and prospective notification on document streams[END_REF], but this was credited to misconfigurations of the systems that returned very few tweets. On the contrary, we do think that, given the metrics and the way the silent days are considered, systems should return few tweets to be top-ranked. This unusual behavior of the metrics is not observed on the other traditionally-used precisionoriented metrics such as P@K and MAP. Our official results on the 2017 track confirm these findings. We submitted a baseline run returning the first tweet of the day containing all the query terms (i.e., at most one tweet per profile and per day was returned). This very simple baseline allowed us to be ranked 2 nd on the mobile evaluation and 4 th (out of 41 participants) on the batch one of Scenario A [START_REF] Lin | Overview of the trec 2017 real-time summarization track[END_REF]. -Concerning the reusability of the collection, we found a problem on the epoch file used in evaluation. In case of rerun, researchers should add all their tweets to the official epoch file, which is not mentioned in the evaluation tool documentation. Otherwise, the results are largely over-evaluated since the evaluation does not consider the non-relevant tweets that are absent from the epoch file. As this problem has never been mentioned before by track organizers or participants, it is very likely that some already-published research papers using the TREC RTS collection as evaluation framework report over-evaluated results.

In future and concerning the metrics, since the track will be pursued in 2018, we suggest to focus on the relative importance of clusters. For instance, let us consider the 2017 profile RTS60 entitled "Beyonce's babies". The very famous photo posted on Instagram in which Beyonce officially announced the names of the twins with their first image is a crucial information for this profile. Other information such as the name of the nurse is also relevant but less crucial.

With equal numbers of retrieved clusters, the systems that find the cluster about this first announcement should thus be more rewarded than the systems that do not find it.

Separating latency and effectiveness should also be (re)considered. In 2015 the TREC microblog track included a very first version of the task (named Scenario A -Push notification) where a latency penalty was applied to the EG metric [START_REF] Lin | Overview of the trec-2015 microblog track[END_REF]. The metric has been given up since 2016 to understand the potential tradeoffs between quality and latency. However, we think that separating latency and gain metrics may lead to some side effects that could be avoided with a single-point metric.

  1)/5 = 0.7 GMP.50 ((0.5 * 1 -0.5 * 2) + (0) + (-0.5 * 1) + (0.5 * 1 -0.5 * 2) + (0.5 * 1))/5 = -0.1 ((-0.5 * 1) + (0.5 * 1) + (0) + (0.5 * 1) + (0.5 * 1))/5 = 0.2 Latency 2 + 10 + 20 = 32 65 + 40 + 10 = 115

0. 5 GMP 5 * 1 )

 551 .50 ((0.5 * 1 -0.5 * 1) + (0))/2 = 0 ((0.5 * 1) + (-0.

Table 1 :

 1 Statistics of scenario A in 2016 and 2017. Times are provided in UTC in order to have fixed time intervals regardless the participant location.

	Year Evaluation period	# judged topics # competitors
	2016 From 02/08/2016 00:00:00 to 11/08/2016 23:59:59	56	41
	2017 From 29/07/2017 00:00:00 to 05/08/2017 23:59:59	97	41

Table 2 :

 2 Notations used throughout the paper

	Notation	Definition
	C = {C1, . . . , C k } The set of clusters
	t j i	The j th tweet belonging to the cluster Ci
	t	A non relevant tweet
	Si	A system
	Θ(t)

Table 4 :

 4 Official metrics for the 2016 and 2017 tracks. The primary metric for each year is denoted with *.

	Metrics	Variants Years	Recall Precision Utility Latency Averaged over
		EG-1	2016*, 2017	✔	profiles
	EG	EG-0	2016	✔	and
		EG-p	2017*	✔	days
	nCG	nCG-1 nCG-0	2016, 2017 2016	✔ ✔	profiles and
		nCG-p	2017	✔	days
		GMP.33 2016, 2017	✔	profiles
	GMP	GMP.50 2016, 2017	✔	and
		GMP.66 2016, 2017	✔	days
	Latency		2016, 2017	✔	profiles

Table 7 :

 7 Official results of our system (run IRIT-Run1-14) and median for Scenario A (mobile evaluation). The mean (τ ) and median ( τ ) latency of submitted tweets in seconds. Strict (Ps) and lenient (P l ) precision. Average metrics were calculated using the 41 participant scores.

	Run	Online utility (strict) Online utility (lenient) τ	τ	Ps	P l
	1st participant run	-93	-25	1	1 0.4337 0.4822
	IRIT-Run1-14	-198	-46	1	1 0.4200 0.4814
	3rd participant run	-262	-66	296 31 0.4140 0.4783
	Average	-1507	-1071	6742 5580 0.3043 0.3699
	Median	-805	-456	102 35 0.3403 0.4174
	Run	EG-p EG-1 nCG-p nCG-1 GMP.33 GMP.50 GMP.66	τ	τ
	1st participant run 0.3630 0.2088 0.2808 0.1266 -0.2720 -0.1566 -0.0479 119374 56744
	2nd participant run 0.3318 0.1811 0.2610 0.1102 -0.3118 -0.1936 -0.0824 116649 49154
	3rd participant run 0.3226 0.2622 0.2489 0.1886 -0.1952 -0.1105 -0.0308 118653 55781
	IRIT-Run1-14	0.2918 0.2571 0.2321 0.1974 -0.1195 -0.0615 -0.0070 67555	1
	5th participant run 0.2907 0.2571 0.2285 0.1949 -0.1190 -0.0622 -0.0087 126484 60685
	Average	0.2273 0.18740 0.2187 0.1787 -0.5106 -0.3501 -0.1990 77911 13175
	Median	0.2194 0.1951 0.2095 0.1826 -0.2630 -0.1707 -0.0839 71463	80

Table 8 :

 8 Official results of our system (run IRIT-Run1-14) and median for Scenario A (batch evaluation). The mean (τ ) and median ( τ ) latency of submitted tweets in seconds. Average metrics were calculated using the 41 participant scores.

A cluster can be considered as a group of tweets sharing the same semantic information.

Official evaluation tools are available at http://trec.nist.gov/data/rts2016.html (2016) and http://trec.nist.gov/act_part/act_part.html (2017), last checked: October 6, 2017.

At the time of the paper submission, the 2017 official runs are not available.


Hamza Athmani

Christian Kijatkin

Rim Benali-Cherif

Sébastien Pillet

Dominik Schaniel

Mirco Imlau

Nourredine Benali-Cherif

& El-Eulmi Bendeif
email: el-eulmi.bendeif@univ-lorraine.fr

Nonlinear optical organic-inorganic crystals: Synthesis, structural analysis, and verification of harmonic generation in tri-(o-Chloroanilinium Nitrate)

Structural and non-linear optical (NLO) properties of a new anilinium hybrid crystal of chemical formula [(C6H7NCl + , NO3 -)3] have been investigated. The crystal structure was determined from single crystal X-ray diffraction measurements performed at a temperature of 100K and shows that the compound crystalizes in a noncentrosymmetric space group (Pna2 1 ). The structural analysis was coupled to the Hirshfeld surfaces analysis to evaluate the contribution of the different intermolecular interactions to the formation of supramolecular assemblies in the solid state that obey nonlinear optical features. This analysis reveals that the studied compound is characterized by a three-dimensional network of hydrogen bonds and the main contributions are provided by the O…H, C…H, H... H and Cl...H interactions, which alone represent ~ 85% of the total contributions to the Hirshfeld surfaces. It is noteworthy that the Halogen…H contributions are quite comparable to those of H…H contacts. The nonlinear optical properties were investigated by nonlinear diffuse femtosecond-pulse reflectometry and the obtained results were compared to those of the reference material LiNbO3. The hybrid crystals exhibit notable second (SHG) and third (THG) harmonic generation that confirms its polarity generated by the different intermolecular interactions. These measurements also highlight that the THG signal of the new anilinium compound normalized to its SHG counterpart is more pronounced than for LiNbO3.

Introduction

Organic-inorganic hybrid materials have been extensively investigated due to their remarkable versatile properties. Among others, anilinium derivatives have received considerable attention in recent years, due to their interesting applications as starting materials in multiple industrial and pharmaceutical processes ( Below et al., 2004;[START_REF] Yunnikova | [END_REF]. The anilinium halides are used as precursors in the synthesis of azo dyes, pigments, pesticides, insecticides, antioxidants and cosmetic products (Cao et al., 2010;Wang et al., 2010). The m-chloroaniline is for example used as a precursor for the production of the widely used antimicrobial and bactericide Chlorohexidine (Bayar et al., 2017). These compounds are also employed in the elaboration of optoelectronic devices in particular with a focus to non-linear optical (NLO) features. A pronounced NLO efficiency in these materials is correlated to the high polarizability of the π-conjugated molecules of the organic part and to the extended network of the intermolecular interactions between the organic cations and the inorganic anions, which promotes the charge transfer within the materials. On the other hand, these compounds are well investigated because of the ease of synthetic approaches, for their chemical and mechanical stability and their ability to crystallize in non-centrosymmetric structures appropriate for even order NLO applications. Although the development of anilinium derivatives systems continues to grow and being the main research topic of several research groups, a search in the Cambridge Structural Database CSD (ConQuest Version 1.22, 2018) (Allen, 2002) for crystal structures containing chloroanilinium molecules results in 37 hits with only seven compounds crystallizing in a non-centrosymmetric space groups. Based on our experience in the synthesis of anilinium derivatives hybrid compounds (Bendeif et al., 2005;Boutobba et al., 2010;Dadda et al., 2012;Takouachet et al., 2016) we prepared a new Chloroanilinium nitrate compound: tri(o-Chloroanilinium Nitrate): [(C6H7NCl + , NO3 -)3] named hereafter o-ClAN. The combination of the organic anilinium with the inorganic planar NO3 -anion is expected to yield enhanced NLO properties with respect to non-planar anions (Bouchouit et al., 2010) if crystallization in a non-centrosymmetric space group is achieved.

Experimental 2.1. Synthesis and crystallization

Crystals of tri(o-chloroanilinium nitrate): [(C6H7NCl + , NO3 -)3] (Fig. 1) were obtained by adding nitric acid to an aqueous solution containing o-chloroaniline in a 1:1 stoichiometric ratio. After three weeks of slow evaporation at room temperature, small brown single crystals appeared. All chemicals (reagents and solvents) used for the synthesis were purchased from Sigma Aldrich, and used without further purification.

Single crystal X-ray measurements

The single crystal X-ray diffraction experiments were performed at 100 K on a SuperNova Dual Wavelength Microfocus diffractometer equipped with a 135 mm Atlas CCD detector, using Mo K  radiation (= 0.71073 Å). To ensure high redundancies, the intensity data were accurately collected using  oscillation scans of 1° per frame repeated at different  positions. The radiation exposure time was 30 s per frame. The CrysAlis program suite (Rigaku Oxford Diffraction, 2017) was used for the unit cell determination, data reduction and analytical absorption corrections. The diffraction frames have been indexed and integrated yielding 117559 reflections up to a maximum resolution of sinmax/=0.864 Å -1 and merged to 13028 unique reflections (Rint = 0.072) with a coverage of reciprocal space of more than 99.6%. The corresponding structure was solved in the space group P-1 by direct methods and successive Fourier difference syntheses and refined against F 2 by weighted full-matrix least-squares methods using the SHELXL97 program (Sheldrick, 2008). All non-H atoms were refined anisotropically. All calculations were carried out using the WinGX software package (Farrugia, 1999).The crystallographic data, measurements and refinement details are summarized in Table 1.

Hirshfeld surfaces calculations

The Hirshfeld surface (HS) is a useful concept for describing the influence of molecular interactions by partitioning the total electron density in the crystal into molecular fragments where the electron distribution of a sum of spherical atoms for the molecule dominates the corresponding sum over the crystal (Hirshfeld, 1977;Spackman & Jayatilaka., 2009). Thus, the HS can be considered as the boundaries delimiting the space occupied by the molecular electron density from those of its neighboring molecules in a crystal. The size and shape of the HS also allow qualitative and quantitative investigations as well as the visualization of the different intermolecular interactions. The HS presented in this work were calculated with the program Crystal Explorer [START_REF] Wolff | CrystalExplorer, Version 3.1[END_REF] and are mapped using the normalized contact distance (dnorm), which is calculated using the following formula:

d norm = 𝑑 𝑖 -𝑟 𝑖 𝑣𝑑𝑊 𝑟 𝑖 𝑣𝑑𝑊 + 𝑑 𝑒 -𝑟 𝑒 𝑣𝑑𝑊 𝑟 𝑒 𝑣𝑑𝑊
where dnorm represent the sum of the distances of any surface point to the nearest external (de) and internal (di) atoms, normalized by the Van der Waals radii of the atoms . It is also worth noting that HS maps are characterized by a red-blue-white color scheme: red regions represent closer contacts and negative dnorm value; blue regions represent longer contacts and positive dnorm value; and white regions correspond to interactions with bond lengths exactly equivalent to the Vander Waals separation and with a dnorm value of zero.

From the 3-D dnorm surface, we can obtain 2-D fingerprint plots, allowing easy attribution and quantitative analysis of the contribution of all intermolecular contacts at the same time. Thus, these 2D plots summarize the nature and type as well as the contributions of all intermolecular interactions.

UV-Visible spectra

The UV-Vis spectra of o-Chloroaniline and o-Chloroanilinium Nitrate (o-ClAN) samples are shown in Figure 2. The o-Chloroaniline compound has been measured for comparison. The measurements were performed in 10 mm quartz cell using a solution of low concentration (c =10 -5 mol/l) by dissolving the compounds in ethanol. The measurements were recorded at room temperature in the 250-900 nm range using a Cary 4000 Varian UV-Visible spectrophotometer.

The spectra display mainly two absorption maxima at 293 nm and 350 nm for o-Chloroaniline and 269 nm and 450 nm for the hybrid compound (o-ClAN). The high energy absorption bands are assigned to the π→π* transition and are attributed to the aromatic entities, while the lower energy absorption bands are assigned to the n→π* transition. We observe thus a clear red-shift of the low-energy band when forming the o-ClAN salt. .

Nonlinear optical measurement

Further structural information regarding the polarity of the sample were acquired through nonlinear diffuse reflectometry at room temperature. For more details, see reference (Kijatkin et al., 2017). For characterization, as-synthesized dry o-ClAN salt was compressed into a pellet for later irradiation. Using regeneratively amplified and spectrally tunable femtosecond pulses from a Ti:Sapphire laser system with a successive OPA (Astrella HE+, Coherent Inc. and TOPAS Prime, Light Conversion), light was slightly focused onto the sample surface. Incident peak intensities on the order of 10 14 W/m 2 with a time-averaged power of 59 mW at  = 1400 nm were applied for the generation of efficient second (SHG) and third harmonic emission (THG). Diffusely emitted light was collected at a small angle with a glass fiber and coupled into a spectrograph (IsoPlane SCT320 and PIXIS:2KB/BUV, Princeton Instruments). In order to compare absolute harmonic emission intensities, a second pellet was prepared from LiNbO3:Mg nanopowder (henceforth abbreviated to LNO, particle diameter approx. 70 nm) whose nonlinear optical properties have been well-investigated and applied (Staedler et al., 2012., Joulaud et al., 2013;Dutto et al., 2011;[START_REF] Rogov | [END_REF]Bonacina, 2013).

3-Results and discussion

Structure and crystal packing

The asymmetric unit of o-ClAN consists of three structurally independent ochloroanilinium cations (ClA-1: C1 to C6, ClA-2: C7 to C12 and ClA-3: C13 to C18) and three nitrate anions (N4-O1 to O3, N5-O4 to O6 and N6-O7 to O9) (Fig. 1). The organic cations and the nitrate anions are interconnected through hydrogen bonds and halogen-oxygen interactions (Fig. 3). This type of interactions has been observed in only one structure containing the ochloroanilinium cation (Balamurugan et al., 2010) among the five structures deposited in the Cambridge Structural Database CSD (ConQuest Version 1.22, 2018) (Allen, 2002) . The crystal packing of o-ClAN is built of anionic zig-zag chains formed by the nitrate anions alternating with the o-chloroanilinium cations stacked layers parallel to the crystallographic c-axis (Fig. 4). The three o-chloroanilinium cations are similar and show typical geometrical features to those observed in similar compounds (Bendeif et al., 2005;Boutobba et al., 2010;Dadda et al., 2012;Takouachet et al., 2016). The average C-N bond length for the o-chloroanilinium cations (1.456(2) Å) (Table 2) is longer compared to that observed for the o-chloroaniline compound (1.375(2) Å) (Nayak et al., 2009), indicating that the proton is transferred from the nitric acid to the amino group of the o-chloroaniline moieties. Compared to the o-chloroaniline structure (Nayak et al., 2009) 3). Such intermolecular interactions produce ring motifs 𝑅 3 5 (16) between the organic cations and the nitrate anions (Fig. 4). The geometrical features of the three nitrate entities show a distorted sp2 configuration. The O-N-O angles are relatively similar ranging from 117.41( 16)° to 122.24(16)°. Each NO3 -anion is characterized by one shorter N-O bond length and two longer ones (see Table 2 for more details). These significant differences in N-O distances [ranging from 1.227(2) Å to 1.278( 2) Å] can be explained by the fact that the longest N-O bond lengths (Table 2) correspond to oxygen atoms involved in hydrogen bonds, however the shortest ones (N4-O3, N5-O5 and N6) correspond to those not implicated in any intermolecular interactions with organic cations. This behavior was also reported in a 4-nitroanilinium nitrate structure (Perpétuo et al., 2004).

Molecular Hirshfeld surfaces

The Hirshfeld surfaces of compound o-ClAN were mapped over dnorm (0.5 to 1.5 Å) and are illustrated in Figure 5. The red circular depressions visible in the front and back surface views indicate hydrogen bonding contacts (Fig. 5). The strongest and shortest interactions between N-H (N1-H1A, N2-H2A, N3-H3A and N3-H3C) and O (O2, O6 and O8) atoms manifest in the Hirshfeld surfaces as the eight ( 08 The 2D fingerprint plots are used for identification and separation of intermolecular interactions and relative contribution of these interactions can be obtained from the area of the surfaces. The 2D fingerprint plots of the studied compound are shown in Fig. 6.

D-H•••A D-H (Å) H•••A (Å) D•••A (Å) D-H•••A (°) N1-H1C•••O4 0.82(3) 1.98(3) 2.800(2) 173(3) N1-H1A•••O6 i 0.85(3) 1.99(3) 2.825(2) 169(3) N1-H1B•••O1 0.89(3) 1.99(3) 2.858(2) 166(3) N1-H1B•••O2 0.89(3) 2.43(3) 3.095(2) 132(2) N2-H2B•••O1 0.87(3) 1.93(3) 2.799(2) 174(3) N2-H2A•••O2 iv 0.88(3) 1.96(3) 2.821(2) 166(3) N2-H2C•••O7 0.94(3) 1.90(3) 2.823(2) 166(2) N2-H2C•••O8 0.94(3) 2.53(3) 3.185(2) 127(2) N3-H3B•••O7 0.84(3) 1.95(3) 2.793(2) 175(2) N3-H3A•••O8 i 0.87(2) 1.96(3) 2.819(2) 171(3) N3-H3C•••O6 ii 0.87(4) 2.08(3) 2.904(2) 159(2) N3-H3C•••O4 ii 0.87(4) 2.34(3) 3.038(2) 138(2) Symmetry code(s): (i) x, y-1, z; (ii) -x+1/2, y-1/2, z+1/2; (iii) x, y+1, z; (iv) x-1/2, -y+3/2, z.
The O…H/H…O contacts, which are attributed to N-H…O hydrogen-bonding interactions, appear as two sharp symmetric spikes in the two dimensional fingerprint maps (Fig. 6b). They have the most significant contribution to the total Hirshfeld surfaces, accounting for more than 40.9%. The presence of these long spikes is characteristic of strong and short hydrogen bonds associated with the cation-anion interactions (Table 3). Moreover, significant contributions to the surfaces are made by the H…C/C…H contacts that appear as symmetrical wings (Fig. 6c) in the fingerprint plots and have a contribution of 19% of the total HS. The H…H contacts appear in the middle of the scattered points in the two-dimensional fingerprint maps (Fig. 6d).

They were found to be the third highest contributors towards the Hirshfeld surface, after the H…O/O…H and H…C/C…H contacts, covering 13.9%. It is important to note that this analysis reveals a significant contribution of H...Halogen interactions (10.9%) comparable to that of H…H contacts. The H…Halogen interactions appear as wings on the top left (H…Cl) and bottom right (Cl...H) of the 2D fingerprint plots (Fig. 6e). Apart from the above interactions, the other interactions (Cl…C/C…Cl, O…O, Cl…O/O…Cl, N…H/H…N and C…C) are also observed and are highlighted in Fig. 6f-j. Their contributions are less than 5% of the total area of the HS.

From this analysis, we observe that the major contributions come from the four molecular interactions mentioned above, namely: the O…H/H…O, H…C/C…H, H...H and Cl... H/H…Cl. These interactions represent ~ 85% of the total contributions to the Hirshfeld surfaces and show the diversity of the intermolecular interactions leading to unusual nonlinear optical properties of this molecular compound. It has been also shown that the efficiency of the hydrogen-bond interactions play a key role in the significant enhancement of NLO properties compared to similar compounds ((Bouchouit et al., 2010;Sajana et al., 2010;Tanak et al., 2014).

NLO properties

Upon irradiation with femtosecond pulses (Incident peak intensities on the order of 1014 W/m 2 with a time-averaged power of 59 mW at  = 1400 nm ), o-ClAN hybrid crystal exhibit notable second and third harmonic generation at wavelengths of 700 nm and 467 nm, respectively. This is underlined by the measured harmonic ratio fR (see reference (Kijatkin et al., 2017) for the definition of f R ) ranging up to 5.6×10 7 which approaches that of LNO being 1.0×10 9 . This finding has two consequences: 1) because fR is well above unity, this measurement confirms the polarity of the o-ClAN compound (Kijatkin et al., 2017), generated by the different intermolecular interactions discussed above and 2) the THG signal of o-ClAN normalized to its SHG counterpart is more pronounced than for LNO. Visual inspection confirms this finding: whereas LNO irradiated with a wavelength  = 1400 nm appears in a red-magenta hue, o-ClAN incorporates relatively larger cyan components at 467 nm, thereby appearing in more blue hue. However, the nonlinear optical signal of the o-ClAN compound is highly time-dependent and deteriorates significantly upon prolonged exposure (Fig. 7). Additional measurements at different positions on the sample revealed that the o-ClAN compound can occasionally challenge harmonic emission of polar materials for short timescales. In terms of the emitted SHG-signal at an incident wavelength  = 1400 nm, o-ClAN typically ranges between 1/10 and 1/300 of the conversion efficiency of the LNOnanocrystalline powder sample.

Conclusion

In summary, we have reported in this work the synthesis, structural characterization and nonlinear optical (NLO) properties of anilinium organic-inorganic crystal of formula [(C6H7NCl + , NO3)3] (o-ClAN). The elaboration of this material was driven by the association of two molecular entities characterized by an electronic configuration favorable to charge transfer allowing therefore high polarizability expected to yield enhanced NLO properties.

The accurate structural analysis performed at low temperature (100K) shows that the crystal structure can be described in the pyroelectric crystalline symmetry class. The Hirshfeld surfaces analysis coupled to the structural investigation reveal that crystal packing is characterized by a three-dimensional network of hydrogen bonds and the main contributions are provided by the O…H, C…H, H... H and Cl...H interactions, which alone represent ~ 85% of the total contributions to the Hirshfeld surfaces. The different intermolecular interactions acting in accordance with the significant and directional H…H and H…Halogen contacts are responsible for steering molecules in the non-centrosymmetric space group leading to interesting NLO properties. The NLO measurements shows that the o-ClAN exhibits remarkable second (SHG) and third (THG) harmonic generation that confirms its polarity driven by the different intermolecular interactions. The obtained results were also compared to those of the reference material LiNbO3 and show that THG to SHG ratio of the studied compound is more pronounced than for LiNbO3.

Figure 1 .

 1 Figure 1. A view of the asymmetric unit o-ClAN with atom numbering scheme. Dashed lines indicate hydrogen bonds and halogen-oxygen interactions.

Figure 2 .

 2 Figure 2. The absorption spectra of o-chloroaniline (o-ClA) and o-ClAN compounds.

  , the internal C-C-C angles at the halogen atoms decrease significantly [120.23(2)° vs 123.3(5)°] and the internal C-C-C angles at ammonium groups increase [120.39(13)° vs 115.6(5)°]. These angular changes are the consequences of the protonation of the amino groups (N1, N2 and N3) and of the implication of chlorine atoms in relatively short Cl•••O halogen-oxygen interactions (Cl1•••O7 = 3.087(2) Å and Cl2•••O6 = 3.084(2) Å and Cl3…O1 = 3.288(2) Å). It is noteworthy that the three cations (ClA-1, ClA-2 and ClA3) form through their ammonium groups (N1, N2 and N3) four N-H•••O hydrogen bonds with three different NO3 -anionic neighbours (Table

Figure 3 .

 3 Figure 3. Projection of the o-ClAN structure along: (a) the crystallographic a-axis and (b) the b direction showing a periodic arrangement of anionic zig-zag chains formed by the nitrate anions and ochloroanilinium cationic layers. Dashed lines indicate intermolecular interactions.

Figure 4 .

 4 Figure 4. 3D view of the crystal packing of o-ClAN, illustrating the formation of ring motifs 𝑅 3 5 (16) between organic cations and NO 3 -anions. N-H•••O hydrogen bonds are represented by black dashed line and Cl•••O interactions are illustrated by magenta dashed line.

  ) bright red areas. The other visible spots on the surfaces correspond to weak C-H•••O and C-H•••Cl hydrogen bonds.

Figure 5 .

 5 Figure 5. 3D Hirshfeld maps with d norm in the range 0.5 to 1.5 Å: A: front view and B back view. Red circle indicating short hydrogen bonds.

Figure 6 .

 6 Figure 6. 2D fingerprint plots of o-ClAN compound for different intermolecular interactions.

Figure 7 .

 7 Figure 7. Left: o-ClAN-pellet before exposure. The pellet measures 5 mm in diameter. Right top: second (red) and third (blue) harmonic emission from LNO (solid lines) and o-ClAN (dashed lines) as a function of time. SHG and THG signal of o-ClAN have been scaled for better visibility. Right bottom: normalized spectral traces for o-ClAN (left) and LNO (right) scaling from yellow (high) to blue (low signal).

Table 1 :

 1 Crystallographic and refinement details for the structures of tri(o-chloroanilinium nitrate) salt.

		tri(o-chloroanilinium nitrate)
	Crystal data
	Chemical formula	(C 6 H 7 ClN + .NO 3	-) 3
	Mr	571.76
	Crystal system, space	Orthorhombic, Pna2 1
	group
	Temperature (K)	100
	a, b, c (Å)	16.8015 (5), 6.5945 (2), 22.4243 (6)
	V (Å3)	2484.56 (12)
	Z	4
	Radiation type	Mo Kα
	µ (mm -1 )	0.43
	Crystal size (mm)	0.1x0.05x0.1
	Data collection
	Diffractometer	Oxford Agilent-SuperNova
	No. of measured,	117559, 13028, 9377
	independent and observed
	with [I > 2σ(I)] reflections
	Rint	0.072
	(sin θ/λ)max (Å -1 )	0.864
	Refinement
	R[F2> 2σ(F2)], wR(F2), S 0.047, 0.099, 1.07
	No. of reflection	13028
	No. of parameters	409
	No. of restraints	1
	H-atom treatment	All H-atom coordinates parameters refined
	Δρmax, Δρmin (e Å-3)	0.45, -0.38
	Absolute structure	Flack x determined using 3510 quotients [(I+)-(I-
		)]/[(I+)+(I-)] (Parsons, Flack and Wagner, Acta
		Cryst. B69 (2013) 249-259).
	Absolute structure
	parameter

Table 2

 2

	selected geometric parameters (Å, °)
	Cl1-C1	1.733(2)	N4-O1	1.278(2)
	Cl2-C7	1.731(2)	N4-O2	1.248(2)
	Cl3-C13	1.729(2)	N4-O3	1.227(2)
	N1-C2	1.453(2)	N5-O4	1.261(2)
	N2-C8	1.459(2)	N5-O5	1.231(2)
	N3-C14	1.458(2)	N5-O6	1.263(2)
	C1-C2	1.387(3)	N6-O7	1.276(2)
	C2-C3	1.390(3)	N6-O8	1.250(2)
	C3-C4	1.388(3)	N6-O9	1.230(2)
	C4-C5	1.374(5)	C12-C7	1.390(3)
	C5-C6	1.392(4)	C13-C14	1.385(3)
	C1-C6	1.389(3)	C14-C15	1.388(3)
	C7-C8	1.386(3)	C15-C16	1.391(3)
	C8-C9	1.387(3)	C16-C17	1.382(5)
	C9-C10	1.390(3)	C17-C18	1.386(4)
	C10-C11	1.385(4)	C18-C13	1.390(3)
	C11-C12	1.387(4)	C1-C2-N1	119.72(18)
	C2-C1-Cl1	119.01(16)	C3-C2-N1	120.17

Table 3 :

 3 Hydrogen-bonding geometry (Å,°)

Acknowledgements

This work was supported by the Université de Lorraine, the CNRS and PHC PROCOPE 40539XA, which are gratefully acknowledged. H.A. is indebted to the Algerian Ministry of Research and Université Abbes Laghrour-Khenchela for a doctoral fellowship. The authors would like to thank the Deutsche Forschungsgemeinschaft (DFG INST 190/165-1 FUGG) and German Academic Exchange Service (DAAD 57139940) for financial support. The authors like to thank Laura Oláh and Zsuzsanna Szaller from the Wigner Research Center for Physics, Budapest, for the preparation of LNO nanopowders within the project MÖB 65056 of the Hungarian Tempus Foundation.