Ameur Douib
email: ameur.douib@inria.fr

David Langlois
email: david.langlois@loria.fr

Kamel Smaïli
email: kamel.smaili@loria.fr

A Translation Evaluation Function based on Neural Network

Keywords: Statistical Machine Translation, Genetic algorithm, Quality estimation, Neural network

In this paper, we study the feasibility of using a neural network to learn a fitness function for a machine translation based on a genetic algorithm termed GAMaT. The neural network is learned on features extracted from pairs of source sentences and translations. The fitness function is trained in order to estimate the BLEU of a translation as precisely as possible. The estimator has been trained on a corpus of more than 1.3 million data. The performance is very promising: the difference between the real BLEU and the one given by the estimator is equal to 0.12 in terms of Mean Absolute Error.

Introduction

Nowadays, a lot of Statistical Machine Translation (SMT) systems use a Beam-search algorithm [START_REF] Koehn | Open source toolkit for statistical machine translation[END_REF] in order to retrieve the best possible translation by taking into account different scores provided by several models: language, translation, distortion, etc. Starting with an empty set, the solution building process consists in producing incrementally a set of complete solutions from partial ones provided by a translation table (T T ). Because the translation is built incrementally, it is then difficult to challenge a previous decision of translation, which can eliminate a partial hypothesis, even if it could propose a good final solution. An alternative to this algorithm is to start with a complete translation hypothesis and try to refine it in order to retrieve the best solution. With complete translation hypotheses, it is possible to revisit each part of the research space and modify it, if necessary. GAMaT [START_REF] Douib | Genetic-based decoder for statistical machine translation[END_REF] is a new decoder for SMT based on a genetic algorithm. It has the advantage to refine several complete solutions in an iterative process and produce acceptable solutions. In fact, a possible solution is encoded as a chromosome, where the chromosome encloses several information (the source sentence segmented into phrases, a translation hypothesis also segmented into phrases, and alignment between source and target segments). Then, from a population of chromosomes, we estimate their fitness (score) in order to keep them, or not, for next generations. To do so, the fitness is a combination of several scores measuring how the different segments of a chromosome are coherent with each others. Nine scores corresponding to nine features are combined to score translations [START_REF] Douib | Genetic-based decoder for statistical machine translation[END_REF]. A weight proportional to the impact of the feature on the evaluation function is assigned to each feature. This combination has been held by a log-linear approach. In GAMaT, the weights corresponding to the nine scores are provided by Moses [START_REF] Koehn | Open source toolkit for statistical machine translation[END_REF]. According to the BLEU [START_REF] Papineni | BLEU: a method for automatic evaluation of machine translation[END_REF] metric results, the translation performance is good, but not better than Moses. To get away from Moses and to propose a relevant solution for GAMaT, we propose to learn the function of the chromosome evaluation by using a neural network (N N ) which predicts a BLEU value of the translation represented in a chromosome. In other words, we would like the fitness function to be correlated to BLEU. The N N is learned on the nine features mentioned before. We opted for BLEU metric because it is commonly used in the MT community to evaluate translations. This kind of learning algorithm is used in the Quality Estimation (QE) community [START_REF] Bojar | Proceedings of the Tenth Workshop on Statistical Machine Translation[END_REF], in order to estimate the translation quality without access to the reference translation. The article is structured as follows. In Section 2 we describe the chromosome features. In Section 3 we present the N N used to learn the fitness function. Then, in Section 4 we describe how we generate the dataset for the N N . We give the results of several configurations in Section 5. Finally, we conclude and give some analyses and perspectives.

Related works

In this paper, we propose a new translation evaluation function for a phrase-based SMT decoders, and which is applied for our genetic-based decoder GAMaT [START_REF] Douib | Genetic-based decoder for statistical machine translation[END_REF]. This function is learned, using a neural network, on nine chromosome features and correlated with the BLEU value of the translation enclosed in the chromosome. Therefore, our work can be classed at the intersection of two research disciplines; The first one is the optimization of decoder parameters for machine translation [START_REF] Moradi | Bi-objective optimization research on integrated fixed time interval preventive maintenance and production for scheduling flexible job-shop problem[END_REF][START_REF] Franz | Minimum error rate training in statistical machine translation[END_REF]. Where, for the majority of decoders, the objective function combines log-linearly a set of translation features to evaluate the translation hypotheses (see equation [START_REF] Bergstra | Deep learning on gpus with python[END_REF]). Optimising the weights of this function allows a better translation accuracy. In the machine translation community to optimise these weights, the proposed algorithms are largely based on a grid search algorithm [START_REF] Franz | Minimum error rate training in statistical machine translation[END_REF]. Where the goal is to find the best set of weights which minimise a loss function adapted for the translation process [START_REF] Moradi | Bi-objective optimization research on integrated fixed time interval preventive maintenance and production for scheduling flexible job-shop problem[END_REF]. The second domain is the Quality Estimation (QE). The main goal in this area is to estimate the translation quality without access to the reference translation [START_REF] Bojar | Proceedings of the Tenth Workshop on Statistical Machine Translation[END_REF]. To this end, in QE community, machine learning algorithms are trained on features extracted from pairs of source sentences and their translations, and they are correlated with an evaluation metric, which can be a metric with binary values (good/bad), or a metric with continuous values (BLEU, TER, ...etc.). In this work, we use a neural network as in QE community, to combine optimally features used in the log-linear approach. The learned function estimate the quality of the translation by predicts its BLEU value without access to the reference translation, which is the case at decoding time.

The features of the chromosomes

In order to evaluate the relevance of a chromosome, we need to evaluate it by combining its different features, such as what was done in [START_REF] Douib | Genetic-based decoder for statistical machine translation[END_REF]. The features are log-linearly combined as follows:

Score(c) = i λ i × log(h i (e, f )) (1)
Where f is the source sentence and e the translation represented in the chromosome c. λ i is the weight of h i determined by Moses and h i is the score related to the i th feature. The value of each weight defines the influence of the corresponding feature in the final score. Nine features related to the construction of a chromosome have been used and are described in the following.

• F 1 : a language model feature, which estimates how the translation e is linguistically correct in the target language. In practice, F 1 is estimated by the likelihood P (e) of a translation yielded by the classical n-gram. In our experiments, n is set to 3.

• F 2 : the second feature concerns the translation probability. Given a chromosome c, F 2 is based on the alignment presented in c. This alignment links each source phrase to a target phrase. Then, F 2 is the product of translation scores (from the source towards the target languages, given by the translation table) between linked source and target phrases. This feature is called the direct translation probability.

• F 3 : this feature concerns the inverse translation probability, which is equivalent to the previous one, except that the translation scores are from the target towards the source languages.

• F 4 : this feature estimates the quality of a pair of segments at word level [START_REF] Koehn | Statistical phrase-based translation[END_REF].

It is defined as the product of lexical probabilities inside one segment and over all the segments of the source sentence. This feature is called a direct lexical probability.

• F 5 : symmetrically to F 4 , an inverse lexical probability is estimated.

• F 6 : this one concerns the length of the target sentence in the chromosome to produce. In fact, the translation should not be too much longer than the source sentence. This feature is set with the difference between the length of the source and the translation in terms of words.

• F 7 : to reinforce the previous feature, a length model is trained [START_REF] Douib | Genetic-based decoder for statistical machine translation[END_REF] that assigns a probability to a pair of sentences depending on their lengths. In other words, this feature is estimated as the probability that a source sentence with a length l s , will produce a translation of length l t .

• F 8 : longer sequences are linguistically more informative than smaller ones. Therefore, a chromosome with a few number of phrases should give a better translation. This feature is called the phrase penalty and is estimated as an exponential function of the number of phrases: e k , where k is the number of phrases in the chromosome.

• F 9 : it is the cost of permutations in the translation at the phrase level, when the target phrases are picked out of order [START_REF] Koehn | Open source toolkit for statistical machine translation[END_REF].

To estimate some of the previous features, we need a translation table, which contains pairs of source and translation phrases with their associated probabilities.

A machine learning algorithm for prediction

As presented in the introduction, the goal of this work is to propose a new chromosome fitness function which combines optimally the features previously presented, and which must be correlated with BLEU. To do so, we decided to train a supervised neural network which takes as input the nine features of a chromosome, and as label in output the BLEU value of the corresponding translation (Figure 1). The learned fitness function is supposed to predict the BLEU of the translation of a chromosome. We experimented different configurations of the neural network by varying the number of the layers and the number of neurons. We used the Sigmoid function for the neuronal activation. The experiments have been conducted by using Keras, a Deep Learning library [START_REF] Bergstra | Deep learning on gpus with python[END_REF].

The dataset for Neural Network

To learn the neural network, a training corpus is necessary. In the following, it will be composed by an important number of pairs < v c , α >. Where v c is a vector of 9 features which characterize a chromosome c and α represents the BLEU value of the translation hypothesis represented in the chromosome c. To this end, we used the French-English parallel corpus of the 9 th task workshop on SMT [START_REF] Bojar | Findings of the 2014 workshop on statistical machine translation[END_REF]. We used this corpus to produce the translation table T T which is necessary to build the chromosomes and also to calculate some features of chromosomes. The corpus contains pairs of French sentences and their English reference translations. We split it into two sets: the former, containing 1,323,382 pairs, is used to build the T T handled by GIZA++ [START_REF] Och | A systematic comparison of various statistical alignment models[END_REF] and the language model; while the latter set (C N N ), containing 165,422 pairs, is used to produce chromosomes. For each source sentence in C N N we built a set of chromosomes, e.g. a set of hypotheses with the segmentation of source and target sentences, and the alignment between source and target segments. We have to produce chromosomes representing perfect, good and bad translations in order to diversify the dataset because the fitness must predict correctly the translation quality of chromosomes, whatever this quality is. To produce chromosomes, we used several functions of GAMaT [START_REF] Douib | Genetic-based decoder for statistical machine translation[END_REF] which take a source sentence and produce one or more chromosomes.

The methods used to build chromosomes are presented in the following:

• From a source sentence, the builder of chromosomes proposes a segmentation based on the longest phrase within it. This longest phrase can be found anywhere in the source sentence. This phrase is picked up from the T T . The segmentation process is iteratively repeated until the whole sentence is segmented. Then, each source phrase is translated in the target language by choosing the most likely translation from T T . Source sentence and hypothesis have same phrase ordering. This produces one chromosome.

• The process is the same as the previous method, but the source sentence is segmented from left to right. This produces one chromosome.

• The process is the same as the previous method, but the source sentence is segmented from right to left. This produces one chromosome.

• Here, the source sentence is randomly segmented, where all the produced segments must exist in the T T . This random segmentation is done several times, in order to increase the number of chromosomes. This allows to increase the size of the training corpus.

• In this one, the builder performs similarly such as in the previous point, except that instead of selecting the best translation for a source phrase, it chooses randomly from T T one from the possible translations of this phrase.

• The producer proposes chromosomes such as in the previous point, except that the alignment between a source phrase and a target segmentation is not monotone: target phrases are mixed. This allows to build several bad chromosomes.

• For each source sentence, the producer of chromosomes keeps the best solution proposed by GAMaT. This allows to produce for each source sentence one chromosome with a good quality translation.

• Similarly to the previous one, the producer of chromosomes keeps the best solution proposed by Moses for the source sentence.

Using these methods, we produced 1,5 million of chromosomes, from 165,422 source sentences. For each of them, we produced a set v c of 9 features, and we calculated α, which is the BLEU value in this work. The BLEU estimates the similarity between the hypothesis and the reference translation. This produced a training corpus for the N N composed of a set of pairs < v c , α >.

Experiments and results

In this section, we describe our experiments. We describe the different training and test corpus we used, and how we measured the performance of the neural network.

Evaluation metric

To evaluate the precision of the prediction function, the Mean Average Error (MAE) criterion [START_REF] Willmott | Advantages of the mean absolute error (MAE) over the root mean square error (RMSE) in assessing average model performance[END_REF] is calculated on the test set. The error is the difference between the real BLEU and the predicted value:

M AE BLEU = 1 n n i=1 |BLEU r (c i ) -BLEU p (c i )| (2)
Where BLEU r (c i ) is the real BLEU value of the translation in the chromosome c i , and BLEU p (c i ) is the predicted BLEU value for the same translation. n represents the size of the test set.

Training and test corpus

In order to build the training and test corpus, we followed the process we described in Section 5. We segmented this corpus into a training and a test part respectively 90% and 10%. We show in Figure 2 This corpus is not balanced, this could be a drawback, because the N N should estimate the translation quality with the same precision, whatever the correctness of a chromosome is. That is why, we trained also, the N N with a balanced training corpus (the bar Balanced training set). In GAMaT, at the translation time, the distribution presented above is not necessarily respected because the set of chromosomes (population) evolves at each iteration. Therefore, we achieve also experiments with another test corpus (bar GAMaT test set in Figure 2) built from a translation corpus used in [START_REF] Douib | Genetic-based decoder for statistical machine translation[END_REF]. This test corpus is representative of chromosomes obtained at end of the real translation process. This test corpus contains 50,000 chromosomes built from 1,000 source sentences. It contains very few perfect translations, and relatively more very bad translations compared to Test set. In the following, we present the performance of the estimator on Test set and GAMaT test set, trained with Unbalanced training set and Balanced training set.

Impact of the neural network architecture

In this section, we study the performance of the N N according to the number of layers (1 to 7) and neurons (8, 16, 32 and 64) in each layer. Figure 3 shows the MAE performance on Test set according to the number of layers, and to the number of neurons. In the left curve, for each number of layers the plot is the average performance according to the number of neurons. In the right curve, for each number of neurons the plot is the average performance according to the number of layers. The figure shows that the best results are achieved for 4 layers and 32 neurons by layer. As these values are only averages, we give in Figure 4 the performance for each couple (number of layers, number of neurons by layer). This figure shows that the performance is better when the number of neurons grows with the number of layers. In the following experiments, we keep the best configuration: 4 layers, and 32 neurons by layer. The performance on the corpus Test set is not affected by the size of the training corpus according to the results of Figure 5. On the contrary, the curve for GAMaT test set is more chaotic but the performance of the estimator tends to be better, as the size of the training increases. shows that the estimator produces bad results for chromosomes for which the BLEU is greater than 0.75 and estimates correctly the others. This is probably due to the fact that bad chromosomes are more numerous than the good ones in Test set. In Figure 6-(a), we did the same experiments, but with Balanced training set. The performance with good chromosomes is henceforth better, since the training corpus is more balanced. As in previous section, we give more details about MAE performance for test subsets according to translation quality in Figure 6-(b). This figure shows that balancing the training corpus allows to improve the prediction performance for good quality translations, but this is not sufficient to improve the overall performance because these good translations are not numerous in the Test set.

Results and Discussion

The results in the different campaigns of Quality Estimation are very close to each other [START_REF] Bojar | Proceedings of the Tenth Workshop on Statistical Machine Translation[END_REF]. Our results obey to this rule. Two main results emerge from this study. For a training corpus not selected smartly, in other words for the Unbalanced training set, the estimator needs 2.25 million of chromosomes to reach the best result 0.1061 (Table 1) on GAMaT test set. While for the Test set, we need only 1.25 million of chromosomes for a MAE of 0.1212. In spite of these low scores (lower is better for MAE), the estimator evaluates badly the best hypotheses of translation, for which the BLEU is greater than 0.75 (see Figure 5-(b)). When we use Balanced training set, we need only 0.5 million of chromosomes to reach the best performance for GAMaT test set with a MAE of 0.1138. The best result for the Test set is obtained for 1.5 million of chromosomes which leads to a MAE of 0.1257. With this Balanced training set, the best hypotheses are better evaluated than with the first training set (see Figure 6-(b)), but we lose in the quality estimation of the bad hypotheses, which are more numerous in the real translation process. This explains the fact that the performance of Unbalanced training set is better than those of Balanced training set. To study the behavior of the proposed evaluation function in a real translation process, we used it as a chromosome evaluation function in our SMT decoder (GAMaT) in order to translate a set of 1.000 sources sentences. In Table 2 we present some translation results according to BLEU metric. Where for each training set we take the best function, learned by the neural network, according to the MAE score and we used it as a chromosome evaluation function in GAMaT. The best translation performance are obtained when we train the neural network on 1.250.000 data with 5 hidden layers and 64 neurons in each layer. This configuration allowed us to achieve 21.05 in BLEU. Through these results, we notice the fact that the neural network configuration, in terms of number of hidden layers and number of neurons, has the same influence on the quality of the learned function (Figure 5-(a) ) as on the translation accuracy, when we use this function as a chromosome evaluation function in GAMaT. The function which has the best MAE score (Table 1), used as fitness function, it leads GAMaT to produce the best output translations.

  the distribution of this corpus according to the BLEU value. This figure shows the proportion of chromosomes for five BLEU intervals: [0,0.25[ (very bad translation according to reference), [0.25,0.5[, [0.5,0.75[, [0.75,1[ and BLEU=1 (perfect translation). The bar Unbalanced training set corresponds to the proportions of the different qualities of translations presented above. The bar referenced as Test set corresponds to the proportions of the test corpus for which we keep the same distribution as in the training corpus.

Figure 2 .

 2 Figure 2.: Distribution of data in the training sets and test sets.

Figure 3 .:

 3 Figure 3.: The influence of the number of layers and neurons in the MAE score.

Figure 4 .: 6 . 4 .

 464 Figure 4.: Details of the influence of the number of layers and neurons in the MAE score.

Figure 5 -Figure 5 .

 55 Figure 5.: The influence of the number of data in Unbalanced training set.

Figure 6 .

 6 Figure 6.: The influence of the number of data in Balanced training set.As in previous section, we give more details about MAE performance for test subsets according to translation quality in Figure6-(b). This figure shows that balancing the training corpus allows to improve the prediction performance for good quality translations, but this is not sufficient to improve the overall performance because these good translations are not numerous in the Test set.

Table 1 .

 1 : Best MAE scores for Unbalanced training set and Balanced training set.

			Test set
	Training set	# of data	Test	GAMaT
	Unbalanced	1.25m 2.25m	0.1212 0.1242	0.1212 0.1061
	Balanced	1.50m 0.50m	0.1257 0.1276	0.1243 0.1138

Conclusion and perspectives

In this paper, we investigated the use of a new function to evaluate chromosomes in GAMaT. The function is an estimator of BLEU for a chromosome. This estimator has been trained by a neural network which is learned on nine features extracted from a set of chromosomes built by GAMaT. The experiments show that the amount of the training data, and their distribution in terms of translation quality, impact the precision of the function. The prediction function achieved convincing results. In fact, the MAE calculated on a test set of 100,000 chromosomes is around 0.12. Also, the use of this function in GAMaT such as a fitness function achieves an encouraging translation performance. The presented method uses BLEU for evaluating a translation, but could be extended to other measures which might be combined in order to get a more robust fitness function. To improve this prediction function, sampling techniques should be used to select more representative data for the training.


Jan Derkacz
email: derkacz@kt.agh.edu.pl

Mikołaj Leszczuk
email: leszczuk@agh.edu.pl

Michał Grega
email: grega@kt.agh.edu.pl

Arian Koźbiał
email: ariankozbial@gmail.com

• Fernando
email: fernandojorge@deusto.es

Jorge Hernández

Amaia Mendez
email: amaia.mendez@deusto.es

Zorrilla • Begoña

García Zapirain

Kamel Smaïli
email: kamel.smaili@loria.fr

F J Hernández

A Mendez Zorrilla

B García Zapirain

Definition of Requirements for Accessing Multilingual Information Opinions

Keywords: opinion mining, social networks, multimedia analysis, multimedia accessibility, multilingual information, mother tongue language, end user, foreign language

In this paper, we will present a study concerning the understanding of the needs of people using Internet in order to access to multilingual information. In fact, in the framework of AMIS (Accessing Multilingual Information and opinionS), a Chist-Era project, we propose to develop a system which will help to understand the main idea of a video in a foreign language. In order to design a useful system, a survey allowing to specify the profile of potential users of AMIS has been conducted. The study concerned 170 people from different countries: Poland, Spain and France. The sample is composed of people of different ages and different culture and languages.The results, in terms of requirements, achieved from this study show differences depending on how often the people watch the news on TV or review them on the Internet, and on the age of the target group. These concrete results help us in several decisions concerning how to build a realistic architecture of AMIS.

Introduction

Nowadays, the information is widely available on the Internet under different forms and different languages. People communicate by posting information on social networks, by recording and sharing videos, by writing long texts, by recording audio messages and so on. Furthermore, this information is produced in several languages. Consequently, a new issue arises: how to make people accessing to a maximum of information pass the barrier of the language? Indeed, even highly educated people rarely speak sufficiently more than two or three foreign languages, while most know only one, and this significantly limits the access to news, culture, ... Permitting people to access to different information sources, would help them to have their own opinion about a topic by cross-checking the information. This challenge constitutes the objective of AMIS: Access Multilingual Information opinionS, a Chist-Era project, which, among other, has the ambition to present an automatic summary of a video to the end user in his mother tongue. In order to understand the needs of the users, in terms of summarized information extracted from videos, we decided to study the requirement analysis of the end users. In fact, the access to information on the Internet depends on several parameters: age, gender, tongue mother, educational level, etc. This investigation helped us to understand what is expectable in accordance to the different attributes presented above and how the summary should be constructed. Building automatically a summary of a video presented in the mother tongue of the end user necessitates several components: video summary, Text summarization, audio summarization, extraction of overlaid text, automatic speech recognition and machine translation. The complexity of the architecture of such a system makes the requirement analysis necessary; one should really not be mistaken in the specification to succeed the realization of the automatic summarization of videos. To do so, a large panel of people (170) from different countries: Poland, Spain and France have been surveyed. These people are from various cultures and different ages. In this paper, a detailed statistical study based on this panel is proposed to understand what kind of information has to be summarized into the mother tongue of the end user. The remainder of this paper is structured as follows. After Related works Section, user requirements will be defined in Section 3. Based on this functional and architectural requirements, recommendations will be provided in Section 4. Finally, Section 5 summarizes an impact of expected results.

Related Works and Expected Progress Beyond State of the Art

The analysis of patterns of Internet and traditional news media usage were performed already in the first decade of widespread access to Internet.

One of the key questions was: to what extend can web become a supplement to television (and newspaper) news, or a substitute for these media? Based on a survey [START_REF] Scott | Patterns of internet and traditional news media use in a networked community[END_REF] of undergraduate students at a large public university where Internet was used on regular basis, this study implied that use of the Web as a news source was positively related with reading newspapers but has no relationship with viewing television news. Members of the surveyed community used the Web mainly as a source of entertainment. Patterns of Web and traditional media exposure were examined considering, i.e. desire for control and knowledge of political and societal issues.

This study indicates that even when computer skills and Internet access become more widespread in the general population, use of the Internet as a news source seems unlikely to diminish considerably the use of traditional news media. News consumption patterns were analysed within a specific group of people: members of university community, 520 undergraduate students, which is not representative for the larger adult population, and the patterns of media use in this university's student population can not be generalized to patterns of news consumption that might be found in a broader society.

The estimation of hours per day for using traditional media and the Web was resulting from the answers to open-ended questions, concerning days per week or hours per day respectively. Only 20 percent of respondents were more likely to use the Web for news than to use newspapers or television. On the other hand, 60 percent of respondents were keener to rely on television news than the Web.

The average time spent watching news programs on television was equal to approximately 35 minutes per day, and the mean time dedicated to entertainment programs reached almost 100 minutes per day. Two approximations of the time spent on the Web were used: estimation of days per week on which the Web and number of days per week on which the Web was used for surveillance.

Outcomes of the conducted analysis indicate that the use of the Web for watching news supplements rather than substitutes for the use of traditional news media in the analysed community. It also appeared that traditional news media do not seem to compete directly with each other or with the Web for news -the more time the respondents spent with one medium, the more time they spent with the other. The obtained results also show that likelihood of using the Web for watching news has no statistically significant relationship with watching television news programs.

Another approach to study usage of online news compared with news use via traditional media was based on niche theory and the theory of uses and gratifications. Survey presented in [START_REF] Dimmick | Competition between the internet and traditional news media: The gratification-opportunities niche dimension[END_REF] was based on data which were collected in a telephone survey with 211 respondents in the Ohio (USA) metropolitan area. The results indicate that the Internet has a competitive displacement effect on traditional media in the daily news domain with the largest displacements happening for television and newspapers.

The findings also show that there is a relatively high degree of overlap between the niches of the Internet and the traditional media. Moreover, the results suggest that the Internet satisfies more needs than any of the traditional media.

Surveys were also done on the relationship between the Internet use and the individual-level social activity. In work presented in [START_REF] Dhavan | Connecting and disconnecting with civic life: Patterns of internet use and the production of social capital[END_REF] the authors apply a motivational perspective to differentiate among types of Internet use when examining the factors predicting such aspects as civic engagement. The predictive capabilities of new media use are then analyzed relatively to key demographic, contextual and traditional media use variables Studies were done to verify if "digital divide" in Internet connectivity is reflected in Internet usage [START_REF] Goldfarb | Internet adoption and usage patterns are different: Implications for the digital divide[END_REF]. For this purpose, a survey of over 18 thousand American citizens was performed. The main finding was that educated people with relatively high income were more likely to have adopted the Internet earlier in time. However, conditional on adoption, low-income, less-educated people spend more time online. The obtained verification implies that this pattern is best explained by differences in the preferences concerning of leisure time.

In methodology adopted in a survey made by Reuters Institute [START_REF] Newman With Richard | Reuters institute digital news report[END_REF] the collected data were referred to user specific user characteristics, such as age, gender and region. The sample collected is reflective of the population that has access to the Internet. The survey was focused on news utilization; consequently the subjects who declared they had not consumed any news in the past month were filtered out.

An online questionnaire was elaborated in order to cover significant aspects of news consumption -therefore it may under-represent the consumption habits of people who are not online, usually older and with limited education. Additionally, a number of face-to-face focus groups were held in a few European Countries and in the USA for investigating issues related to news consumption.

The main outcomes show that approximately half of respondents use social media as a source of news. Women and young people are predominant in this group.

In addition to online access, most users also continue to access news via TV and radio but the extend of this is significantly affected by age. For every group under 45, online news is now more important than television news. Television news still remains most important for older people however, overall usage of this medium continues to decline. For young people (under 24) social media (28 percent) become more popular than TV (24 percent). There are quite significant differences between some countries regarding news sources: for instance in Ireland, almost 40 percent of citizens start their day with radio news and countries like Finland and Japan significant minority still read printed newspapers in the morning. TV usage ranges from 16 percent in Ireland, up to 51 percent in Japan.

As concerns devices, usage of smartphones for news watching is firmly increasing, especially among young people, reaching over half of respondents. Computer use is declining, while tablet usage seems to be saturating, falling back in some countries like the UK and Japan.

Findings resulting from data across all considered countries show that news articles are still the most consumed type of news online content (59 percent) while online news videos were preferred by 20 percent of users on average (22 percent in France, 27 percent in Spain and 28 percent in Poland).

The above presented studies concerned mainly the patterns of the Internet and traditional news media usage in general. These studies do not refer directly to multilingual help system but give additional information, which can be helpful for the questionnaires don't belong to a known distribution. Thus, non-parametric methods have been chosen, in particular, Kruskal Wallis test [START_REF] Breslow | A generalized kruskal-wallis test for comparing k samples subject to unequal patterns of censorship[END_REF].

In the following subsections, the results obtained from the users are presented, with respect to gender and age, mother tongue as well as education level.

Socio-demographic Analysis

In this study 174 people have answered to the previously described questionnaire, mainly all of them are between 18 and 65 being 60,90% males and 39,1% females (see table 5). Many of them have at least bachelor degree.

(A) Gender and Age

As it can be seen in table 6 the participants were divided in seven age ranges being most of them (81,6%) under 45. In fact, the core of the group is between 18 and 45 (63,3%). With Kruskal Wallis test [START_REF] Breslow | A generalized kruskal-wallis test for comparing k samples subject to unequal patterns of censorship[END_REF] (this non-parametric test check if input data belong to the same population), significant differences are checked between the two groups (male and female) divided by gender independently of the age. A significant difference has been found between the 7 groups studying the relation between the number of trips abroad that they perform per year p = 0.002. The p-value is defined as the probability of obtaining a result equal to or "more extreme" than what was actually observed, when the null hypothesis is true. In this case, the null hypothesis is that a relation between age and number of travels abroad does not exist. The following Fig. 1 shows the box plot for this variable:

(B) Mother Tongue

In relation to mother tongue, there is a significant difference regarding the number of additional languages that the participant speaks and understands in an acceptable way (p = 0.001). Usually the users speak/understand correctly French, Spanish and Polish (due to the origin country of the authors).

The statistics (table 7) show that people whose mother tongue is French or Arabic speak at least one extra language. However, in the case of people from Spain and Poland, this number increases to 2. On average, all the participants speak 2 extra languages. Regarding the educational level of the participants, using the Kruskal Wallis test, we have found significant differences in the number of trips abroad that they perform a year (p = 0.000). There is a significant difference in the number of travels abroad between subjects divided according to education. The following Fig. 2 shows the box plot for this variable. In Fig. 2, several outliers can be observed. They are participants who travel more than the average.

Usage of Sources of News Information on TV and Internet

Participants have been divided according to age ranges specified previously. After making the Kruskal Wallis, significant differences in how often the people watch the news on TV (p = 0.015) were observed. Dividing participants questionnaires by gender, we also found significant differences in how often they watch the news on TV (p = 0.024).

In relation to mother tongue, there is a significant difference in how often the people watch the news on TV per day (p = 0.000), the minutes they spend watching news on television (p = 0.000), and the number of times they review the news on Internet (p = 0.011).

Functionalities of the Proposed System

Preferences of the respondents concerning the functionalities of the proposed system are as follows:

1. Watching summaries of longer content -32,18 percent 2. Automatic translation of foreign language -25,29 percent 3. Searching for subject/keywords within a given content -13,79 percent 4. Interaction with social media (rating and sharing the content) -11,49 percent 5. Further exploring a given subject -10,34 percent 6. Comparison of two different audio-video materials on the same subject -6,90 percent

The table 9 shows the chosen functionalities for each age range. Observing these results, the two first items are the best rated, and it is completely aligned to the objectives of the AMIS project. All the participants, independently of the age range prefer when the system includes functionalities 1: "Watching summaries of longer content", and 2: "Automatic translation of foreign language" (except people above 65).

GUI Preferences

In this subsection, the answers of section D questionnaire are analyzed, mainly, D3 and D4 items.

Taking into account D3 answers, the participants have been divided according to the preferred technology (dedicated application or web browser application) there is a significant difference in number of news that participants review (p = 0.008) and number of minutes spend looking news on the Internet (p = 0.012)

In the case of D4 no significant differences have been found. The table 10 shows the preferred technology for each age range. In the questionnaire, people between 18 and 55 have chosen a web application design of the system.

Overall Architectural Recommendations

The analysis of the questionnaires shows, that the functionalities of summarization of the content and automated translation are considered most important for the users. These findings had influence on our approach to the architecture design.

The proposed architecture will be at the same time effective in delivering summarized and translated content and will be fairly easy to implement and integrate. This approach will allow to analyze future research and engineering challenges in order to adapt the second phase architecture. We have also agreed that at first the system will be lacking some of the planned non-critical features such as full social networks integration and sentiment analysis as those are not chosen as most important by our respondents. Proposed phase one architecture is depicted in Fig. 3 with the main pipeline denoted. We start with a full video file which is summarized to desired length. The audio of the summarized video file is extracted and passed to the speech recognizer. An up-to-date speech recognition toolkit, such as KALDI, will be used. The results of the speech recognition are translated using the machine translation algorithms. For machine translation The system will use the state-of-the art Moses system along with the Giza++ toolkit, as well as the IRSTLM language modeling toolkit, as proposed in the Moses' scripts. The summarized video file with added captions provided by the machine translation is presented to the user. At the same time video summarization, speech recognition and machine translation are supported by information provided by the optical character recognition system that is analyzing the overlaid video text.

In the second phase we plan to implement integration with social media and sentiment analysis to cover other functionalities planned for the system. When moving to the second phase we also plan to enhance functionalities and algorithms used in the first phase in order to deliver higher quality summarizations and translations.

Summary on Impact of Expected Further Results

The paper presents results of the survey made in order to specify profiles of potential users, their habits concerning usage of information as well as their preferences regarding functionality of the multilingual help system supporting the understanding of broadcasting news in foreign language.

Over 170 persons with different cultural and social background were subjects to the analysis. The results of the survey will allow to fine-tune the features of the system.

As a result AMIS will greatly minimize difficulties in the exchange of information, regardless of language differences, which nowadays is extremely important.

Its use concerns mainly social and cultural domains, as it allows one to explore another side of specific event. This will allow users to understand the comments, as well as the culture of other nations.

At the scientific level the project concerns synergies and new research linking summarizing video, audio and text materials, ASR, machine translation and opinion exploration.

Evaluation of project outcomes, multilingual information and opinions, due to its proximity to human experience, will be primarily done subjectively. Besides, some automatic evaluation methodologies are planned to be applied, at least for some of the elements of the proposed solution.

Fig. 1

 1 Fig. 1 Age box plot. Subjects have been divided into 7 groups according to age. The number of travels to other countries show significance differences (p=0.002). Circles in the chart show outliers subjects.

Fig. 2

 2 Fig. 2 Education box plot. Education box plot. Subjects have been divided according to education. Circles in the chart show outliers subjects.

Fig. 3

 3 Fig. 3 First phase architecture

Table 5

 5 Gender data

		Frequency Percent
	Male	106	60.9
	Female	68	39.1
	Total	174	100

Table 6

 6 Age data

		Frequency	Percent
	Below 18	32	18.4
	18 -25	61	35.1
	26 -35	28	16.1
	36 -45	21	12.1
	46 -55	16	9.2
	56 -65	11	6.3
	Above 65 5	2.9
	Total	174	100

Table 7 Mother tongue data Frequency Percent Average of extra lenguages

 7

	Arabic	4	2.3	1.50
	French	39	22.4	1.64
	Spanish	31	17.8	2.61
	Polish	93	53.4	2.09
	German 3	1.7	3
	English	1	0.6	1
	Italian	1	0.6	4
	Others	2	1.2	1.5
	Total	174	100	2.09

table 8 ,

 8 the educational level of each participant can be seen. More than 50% of the users have at least the Bachelor Degree.

Table 8

 8 Education level data

		Frequency	Percent
	Elementary	28	16.1
	High school	53	30.5
	Bachelor	30	17.2
	Master	40	23
	Doctoral	23	13.2
	Total	174	100

Table 9

 9 Functionalities selection for each age range

	Funct. Below 18	18 -25 26 -35 36 -45 46 -55 56 -65 Above 65
	1	6,32%	10,92%	5,75%	4,60%	2,30%	1,15%	1,15%
	2	5,17%	8,05%	4,02%	4,02%	1,72%	1,72%	0,57%
	3	2,87%	3,45%	4,02%	1,15%	1,72%	0,57%	0,00%
	4	1,15%	4,60%	1,15%	0,57%	1,72%	1,15%	1,15%
	5	1,15%	5,17%	0,57%	0,57%	1,15%	1,72%	0,00%
	6	1,72%	2,87%	0,57%	1,15%	0,57%	0,00%	0,00%
	Total	18,39%	35,06%	16,09%	12,07%	9,20%	6,32%	2,87%

Table 10

 10 Preferred technology for each age range

		Below	18 -25 26 -35 36 -45 46 -55	56 -65 Above
		18						65
	Dedicated application	8,05%	13,79%	6,90%	4,60%	3,45%	2,87%	1,72%
	Web application	6,32%	19,54%	8,05%	6,90%	5,75%	1,72%	0,00%
	Others	4,02%	1,72%	1,15%	0,57%	0,00%	1,72%	1,15%

Research work co-funded by the National Science Center, Poland, conferred on the basis of the decision number DEC-2015/16/Z/ST7/00559.

adjusting functionalities and features of multilingual help system to the news usage patterns.

There are not many sources regarding user requirements with respect to video summarisation and machine translation. The analysis presented in this paper focuses specifically on such requirements regarding functionalities combining both: automatic video summarisation and translation of news related content.

User and Services Requirements

In this section, the authors describe the questionnaire designed to extract information for the users and the subsequence statistical analysis.

Questionnaire Design

To define user and AMIS service requirements, the authors designed a questionnaire to extract information from users related to their habits to consume news, either through the TV or the Internet. In tables 1, 2, 3 and 4, the parts of designed questionnaire can be seen. As it can be seen the items proposed in the questionnaire are oriented to characterize the user profile taking into account socio-demographic data (such as: educational level and languages that one speaks), the use of sources of information, the preferences of offered functionalities, and human interfaces.

Section A and B (tables 1 and 2) of the questionnaire ask the user about descriptive information, however, section 3 (table 3) allows the user to rank and order the proposed system functionalities. Section D questions are oriented to extract GUI user preferences to guarantee the system usability and accessibility in the future design.

C. Functionalities

Please rank the functionalities -Watching summaries of a longer content -Searching for subjects/keywords within a given content -Automatic translation of foreign language -Comparison of two different audio-video materials on the same subject -Interaction with social media -Further exploring of a given subject In table 4, the items D.3 and D.4 are specially oriented to decide the hardware and software requirements for AMIS system.

Statistical Analysis

In the statistical analysis performed, the normality of the data by Levene test [START_REF] Glass | Testing homogeneity of variances[END_REF] for homogeneity of variances, and the Kolmogorov-Smirnov test [START_REF] Lilliefors | On the kolmogorov-smirnov test for normality with mean and variance unknown[END_REF] has been previously checked. The previous tests have proved that the data obtained through


Context

Original hermeneutical process

In this situation, we started with some empirical tries

Person 1: "Hey, I have data" Person 2: "Hey, I have tools"

Original hermeneutical process

In this situation, we started with some empirical tries -Napoleonic Wars -War of 1870 -First World War -Second World War

Funding is not granted yet, but we're in a hurry, so we already started to work.

ECRPER in short 1) Diachronic and synchronic analysis:

• how the Self and the Other are being represented • how relationship to events is being elaborated through personal writings 2) Bring together different tools, sets up a solid editorial and hermeneutical workflow and make it available for further research.

• OCR • Annotation • Edition • publication
ECRPER in short 1) Diachronic and synchronic analysis:

• how (the Self and) the Other are being represented • how relationship to events is being elaborated through personal writings 2) Bring together different tools, sets up a solid editorial and hermeneutical workflow and make it available for further research.

• OCR • Annotation • Edition

The representation of the other in the context of war discourses • How Germans see the French and reciprocally.

• Broaden to all the mentions of the conflicts actors (Nations, organisations, persons, person types, …)

• All instances appearing in the discourse will be modelized and brought in relation to one another. • Diachronic analysis: appearance and disappearance of a mention, semantic evolution, spelling variations, ...

Representation of the other

Representation of the other: NLP approach The customisation is a way to specialize the entity resolution for a particular domain/profile, for example selecting the correct Vichy wikipedia entry related to the ww2 context.

Specific dictionaries can then be integrated seamlessly in the tool.

NOTE:

When wikipedia doesn't provide a page for the alternative meaning, this approach cannot be applied.

No wikipedia page, no party.

The dictionary is then used to override the resolution phase to force the correct definition.

This approach is very specific, less frequent and should be only a fallback solution The dependency tree can be used to find expressions and modifiers relative to the entities of interest.

  ALMAnaCH = Automatic Language Modelling and Analysis & Computational Humanities, joint EPHE-Inria team A strange mixture of people with different skills working side by side: • Computer engineering → data mining, information extraction • NLP experts → Semantic and Syntactic analysis, Parsing, etc.. • Digital Humanities (History and Literature background) → data modelling, textual analysis, digital philology Context ALMAnaCH = Automatic Language Modelling and Analysis & Computational Humanities, joint EPHE-Inria team A strange mixture of people with different skills working side by side: • Computer engineering → data mining, information extraction • NLP experts → Semantic and Syntactic analysis, Parsing, etc.. • Digital Humanities (History and Literature background) → data modelling, textual analysis, digital philology Why not combining all the skills available around? Software engineering, Data mining, Machine learning, Knowledge discovery

Franco

  -german personal writings in wartime (19th-20th century) With ALMAnaCH (Inria Paris) -DHI (Paris) -Centre Marc Bloch (Berlin) -IEP Lille Analysing German and French diaries and letters written during the French-German wars since the 19th century:

  up everything (POS and NER) Workflow (1) -Extract Named Entities Nous parlons du procès Pucheu. La question est plus actuelle. (...) J'indique qu'à mon avis tout ce procès a été mal conduit -il fallait (...) proclamer que devant des crimes inouïs, (...) la nation prenait une décision politique, immoler les hommes de la haute trahison : "Jetons à l'Europe, en défi, une tête de roi", jetons aux combinards de Vichy et de Washington, en défi, une tête de traître. (...) Je vois Yves qui m'avait cité l'impatience d'Alger devant le cas Pucheu comme une illustration de la crise du Gaulisme. Journal de Léo Hamon, March 14th 1944 entry

E

  .g. named entity "Vichy" (i.e. Régime de Vichy) and Washington (i.e. the US government) ⇔ nominal modifier "combinards" workflow to a very large and diverse corpus → Multilingual terminology with the specific terms for each war • Add sentiment polarity tagging layer • Nice results visualisation Thank you

Let's try to use tools on some random data (YOLO)"

	But with a research question, it worked a little better:
	Person 1: "Hey, I have data."
	Person 2: "Hey, I have tools."
	Person 1:

Person 1: "Hey, I have data" Person 2: "Hey, I have tools" Person 1 and 2 (ecstatic): "

"Waow, I also have an idea of what I want to do" Person 2: "Maybe my tools could help you shedding some light"

	Original hermeneutical process Original hermeneutical process
	Next step, we started to think about a win-win strategy"
	Person 1: "Hey, I have data and a research question"
	Person 2: "Hey, I have tools"
	Person 1:

"Your tools might need new data and use cases to improve (ensure genericity and cross-domain applicability)"

	Original hermeneutical process
	Finally…
	Person 1: "Hey, I have data."
	Person 2: "Hey, I have tools.

" Person 3: "Hey guys, I also have data, tools and some nice research questions" Person 4: "I want to be part of the group, please" Choir : "Let's create

a research project all together"

  ECRPER project(ANR/DFG, 2018-2021)

Acknowledgement

Anne Baillot (Centre Marc Bloch, Berlin) → German literature and History, Mastermind Hector Martinez Alonso (ALMAnaCH, Inria Paris) → POS tagging, NLP stuff

The corpus: WW2 diaries written in French