Panagiota Kotoula
email: kotoula@ceid.upatras.gr

Christos Makris
email: makri@ceid.upatras.gr

Query Disambiguation based on Clustering Techniques

Keywords: Query Disambiguation, Information Retrieval, Query Reformulation, Clustering, Containment, Semantics

In this paper, we describe a novel framework for improving information retrieval results. At first, relevant documents are organized in clusters utilizing the containment metric along with language modeling tools. Then the final ranked list (ascending/ descending order) of the documents that will be returned to the user for the specific query, is produced. To achieve that, firstly we extract the scores between the clusters and the query representations and then we combine the internal rankings of the documents inside the clusters using these scores as weighting factor. The method employed is based in the exploitation of the inter-documents similarities (lexical and/or semantics) after a sophisticated preprocessing. The experimental evaluation demonstrates that the proposed algorithm has the potential to improve the quality of the retrieved results.

Introduction

One of the most popular research issues has always been the subject of improving the quality of ranking in Information Retrieval results. To this extent, information need is expressed through the form of queries submitted against a search engine or platform with the purpose of receiving any available information related to the query (Baeza-Yates and Ribeiro-Neto, 2011; [START_REF] Manning | Introduction to information retrieval[END_REF]. The problem or the challenge in this process is the search machine's potential and capability to respond and deliver the fittest set of information for the specific query, if this information actually exists.

On the other hand, users that post their queries are not specialists but instead plain, every day users that are not usually aware of the best format to provide their input query either because they do not leverage the full potential of the search platform or because they cannot express their intention clearly. The search engine's greatest challenge is then, to understand this user's intention through this given input, the query, that is to disambiguate the terms that synthesize the query and attempt to satisfy the query request.

The disambiguation process in information retrieval concerns both the way the query is approached as well as the way that the retrieved documents are processed. Effective retrieval functions and techniques have been mostly derived from the class of probabilistic models and several approaches have been successfully implemented towards this direction (Baeza-Yates and Ribeiro-Neto, 2011; Crofy et al. 2009).

Recently, the aforementioned models have been combined with the cluster-based retrieval approach. Clustering depends on the relationship between document and query processing. If documents' processing depends on the query, then clustering can be seen as gathering all documents related to this query under the same cluster. Otherwise, documents can be clustered according to the relation between their content information based on lexical or semantic similarities and independent to the query. The level of similarity decided will provide the different number of clusters for a given corpus and will utilize the overlapping and redundant information present in the documents. This kind of retrieval is based on the hypothesis that similar documents can satisfy the same query or queries [START_REF] Jardine | The use of hierarchical clustering in information retrieval[END_REF][START_REF] Van Rijsbergen | Information Retrieval[END_REF]. Moving in this line of thought recently researches have depicted that if good clusters can be designed then the retrieval performance will be improved [START_REF] Raiber | The Correlation between Cluster Hypothesis Tests and the Effectiveness of Cluster-Based Retrieval[END_REF].

In this paper we propose a framework in order to improve query search results by leveraging specific types on information representation along with selecting the proper clustering algorithm to organize the documents. Our approach examines query -independent document processing and representation resulting to a lexical based inter-document similarity in order to form the clusters. The key of improvement in our results is the use of a generalized structural unit, namely gloss, derived from WordNet's thesaurus hierarchy (Fellbaum, 1988; Princeton WordNet Gloss Corpus, 1979) instead of words, namely senses. Usual problems like frequent irrelevant terms are faced effectively since the generalized approach we design elevates the terms that have the most similar sense(s) between each other throughout the document and when gathered they actually represent its content due to their semantics. Our final result is the list of documents returned for the query as an improved ranking.

The paper is organized as follows: in Section 2 we present a brief survey on similar techniques, while in Section 3 we describe an outline of our approach. In Section 4 we describe the lexical and semantics processing of the documents and in Section 5 we conclude the document processing presenting the clustering along with the techniques for evaluating internal scores of the contained documents. In Section 6 we process the query and explain the reformulation. Finally, in Section 7 we describe our ranking technique along with comparing our result with other techniques and platforms in Section 8 and conclude our research in Section 9.

Related Work

Many researches have been developed regarding document and query processing in terms of query disambiguation techniques. Most of them use additional databases that extend the analysis from the standard search engines. These databases are known as thesaurus or ontologies and are basically lexical databases (mainly for the English language) providing extra information and tools for analyzing content and query processing like sets of synonyms, antonyms, definitions and other data, all derived from a hierarchical structure.

In these researches the various processing rules and techniques are applied mostly in conjunction with the WordNet (Princeton University) (Fellbaum, 1988; Princeton WordNet Gloss Corpus, 1979) ontology, along with probability distribution models to smooth and arrange the results. Most of our techniques were inspired or derived from the work of (Giakoumi, Makris and Plegas, 2015) that raise the question of whether the texts and the query should be approached with dependency between each other or not. In any case, texts are being scored in respect to one global lexicon and clusters are being formed through KL-Divergence [START_REF] Kullback | On information and sufficiency[END_REF]. Throughout the process, TF-IDF and SMLE (Smoothed Maximum Likelihood Estimate) models provide the necessary smoothing of the contained information.

In other articles like (Makris et These techniques leverage greedy algorithms like MAXI-MAXU (Maximum k-Intersection, Maximum k-Union), to allocate the maximum intersection of similar information between pairs of documents. Upon locating the same context, a new document is being created that contains this context once along with any new information contained in the texts that participated in the intersection process over the specified threshold. The newly created content is being checked with the coherence metric that concludes whether the derived text is logical and valid.

In a series of additional papers, [START_REF] Kanavos | Extracting Knowledge from Web Search Engine using Wikipedia[END_REF]Makris et al., 2013; Makris and Plegas, 2013; Raviv, Kurland, and Carmel, 2016), a recent and interesting approach is being explored to provide further information in text and terms annotation. The process of Wikification leverages the structure of Wikipedia pages in order to assign additional weights in the scores assigned to terms through WordNet's disambiguation process. Further clustering of the texts is being performed in these papers too, using ntuples. Finally, using the TAGME [START_REF] Ferragina | TAGME: on-the-fly annotation of short text fragments (by Wikipedia entities)[END_REF] technique in conjunction with WordNet results, the final score is assigned to the texts before being returned for a specific query.

Moreover [START_REF] Levi | Selective Cluster-Based Document Retrieval[END_REF][START_REF] Raviv | Document Retrieval Using Entity-Based Language Models[END_REF] explored language models, clustering and Wikification techniques in order to improve the retrieval performance. Our work can be considered to act complementarily to these attempts since it explores the use of WordNet and its glosses as an extra refined mechanism.

Outline of the approach

The disambiguation process in information retrieval concerns both the way the query is approached as well as the way that the retrieved documents/ pages are handled from the (any) search engine. In order to decide how this process can be better applied, in terms of a source corpus or a web search in respect to a specific query, we define the way to modify and represent these documents appropriately while reformulating the query at the same time.

At first, we process the documents and we consider this step as completely independent from the query. The reason for this decision is an intuitive thought which dictates that given a specific query, the chances to find an answer for it and this answer to be ideal are very low; on the other hand, given a document we can extract one or more queries that can find their ideal answer(s) inside the document they represent.

For text processing we use the WordNet 2.1 ontology by choosing to extract the glosses from the hierarchy. The glosses are sentences that describe the various senses also included in this model. That way we insert the idea of generality in our technique; in other words we define a new generalized structural unit to represent our texts. We examine and evaluate the use of glosses due to their structural position inside the Word-Net model. As such instead of limited and isolated senses, we elevate the use of glosses with the purpose of adding or discovering additional (important per case) information for the terms that are contained in the text.

A significant step in our approach is document clustering based on the containment metric. Through clustering we target to achieve the maximum possible concentration of the same or similar content of all documents under the same interface (representation). This interface/ representation is the lexicon of the cluster, based on the documents it contains. This lexicon is being forwarded later in the process in order to be compared with the various representations of the query. At the same time, the documents inside these clusters also receive a score independently of the query.

The query is being processed in a similar waylike the documentsthrough Word-Net 2.1 ontology information extraction. It is then reformulated and through the glosses, extracted from its terms, various representations are being produced (all possible combinations between glosses of each term).

In the final stage of our processing, we produce the final ranked list (ascending/ descending order) applied on the documents that will be returned to the user for the specific query. To achieve that, firstly we extract the scores between the clusters and the query representations and then we combine them through the Borda Count Method (Kozorovitzky, and Kurland, 2011) providing the final ranking order of the documents to return.

For the implementation of this research's algorithm we created and used tools in JAVA and Python in an end-to-end application so that the largest part of the process to be automated and user's interference to be minimum.

4

Language model processing of documents

Document Representation

We begin our document representation technique using natural language processing based on NLTK tools (Bird, Klein, & Loper, 2016) and OpenNLP (OpenNLP, 2016) trained models to extract the terms from the documents and lemmatize them. The output of these steps are being forwarded to WordNet 2.1 interface, where for each document, the contained words one by one are set as input in the thesaurus.

To select the appropriate sense or senses from the WordNet schema we utilize the Wu & Palmer similarity measure [START_REF] Wu | Verbs semantics and lexical selection[END_REF] which is depicted below:

𝑠𝑖𝑚𝑖𝑙𝑎𝑟𝑖𝑡𝑦 (𝑐 𝑖 , 𝑐 𝑗 ) = 2 𝑑𝑒𝑝𝑡ℎ (𝐿𝐶𝐴(𝑐 𝑖 , 𝑐 𝑗 ))
𝑑𝑒𝑝𝑡ℎ(𝑐 𝑖 ) + 𝑑𝑒𝑝𝑡ℎ(𝑐 𝑗 ) , where we define 𝑑𝑒𝑝𝑡ℎ() as the depth of the node inside the ontology and as 𝐿𝐶𝐴() the lowest common ancestor of the two senses 𝑐 𝑖 , 𝑐 𝑗 in the hierarchy, for the terms 𝑡 𝑖 , 𝑡 𝑗 . We use the above algorithm because we do not know a priori the fittest sense for each incoming term and Wu & Palmer provide us the ones for which the similarity is maximized. The similarity is calculated over the senses but the algorithm moves one step further and retrieves from the WordNet ontology the respective gloss that the sense belongs to.

To this point, we have extracted all the necessary information from the WordNet schema and assigned these sets of glosses (sentences) to each one of the retrieved documents. Each document is now being represented by a set of glosses retrieved for its terms instead of the more common methods that stop to the retrieval of senses only. The main advantage of our approach is that there is no need to perform extensive smoothing techniques like TF-IDF. That is because by extracting a more generalized description for each term, the words inside the document that concern the actual concept will finally be expressed by more similar or the same glosses and as such collect the majority of the content. Instead, words that do not contribute to the overall conceptual representation will eventually be limited to minimum frequency in the following steps.

Language Model for Internal Scoring of documents in a cluster

In the next step, we represent the information contained inside the documents with a language modelling approach that aims to conclude scores for each document given a larger set of glosses contained in a fixed vocabulary 𝑉.

Given a constructed fixed vocabulary 𝑉, the idea is to calculate a probability measure over strings that belong to it [START_REF] Manning | Introduction to information retrieval[END_REF]. In this work, we construct these vocabularies 𝑉 containing a fixed set of sentences and calculate the probability of those sentences to exist inside the document. Since our unit for representation is no longer one word but instead a set of words, we apply the same rules for calculating the probability metric over sentences (sets of strings).

For each document and for each contained sentence we calculate these probabilities using the SMLE model (Smoothed Maximum Likelihood Estimate). SMLE model adjusts the representation of the document in respect to the frequency of its contained sentences, in other words based on the number of occurrences of each sentence 𝑠 in the fixed vocabulary 𝑉:

𝑀 𝑑 𝑆𝑀𝐿𝐸 (𝑠) = { 𝑓 𝑠,𝑑 𝑙 𝑑 -𝑐, 𝑖𝑓 𝑠 ∈ 𝑑 𝑒𝑝𝑠, 𝑖𝑓 𝑠 ∋ 𝑑 , ∀𝑠 ∈ 𝑉
, where 𝑓 𝑠,𝑑 is the number of occurrences of sentence 𝑠 in document 𝑑 and 𝑙 𝑑 is the number of terms contained in both 𝑑 and 𝑉; also, 𝑒𝑝𝑠 is a very small quantity of the order of 10 -10 and c is estimated as:

𝑐 = 𝑒𝑝𝑠 * |𝑉| -𝑛 𝑑 |𝑉|
, where |𝑉| is the number of sentences in vocabulary 𝑉 and 𝑛 𝑑 the number of sentences both in the vocabulary 𝑉 and the document 𝑑.

The Smoothed version of MLE containing quantities 𝑒𝑝𝑠 and c is performed in order to avoid the zero probability problem of MLE, that is:

𝑀 𝑑 𝑀𝐿𝐸 (𝑠) = 0, ∀𝑠 ∈ 𝑉 & ∀𝑠 ∋ 𝑑
Using the SMLEs produced we can then calculate the document's score against the fixed vocabulary 𝑉 by adding these sub-SMLEs for all contained sentences inside the document representation.

Clustering

Containment

Documents are now represented by probability distributions and we can focus on how to leverage the similar ones in order to proceed with further clustering them. For this step we chose to use a simple but efficient metric known as Containment [START_REF] Broder | Syntactic Clustering of the Web, Computer Networks and ISDN Systems[END_REF] accompanied with the K. To examine whether a document 𝑑 1 is contained inside another document 𝑑 2 we use the following formula:

𝐶𝑜𝑛𝑡𝑎𝑖𝑛𝑚𝑒𝑛𝑡(𝑑 1 , 𝑑 2 ) = |𝑆(𝑑 1 ) ∩ 𝑆(𝑑 2 )| |𝑆(𝑑 1 )
| , where 𝑆(𝑑 𝑖 ) is the set of glosses that the document 𝑑 𝑖 contains. Containment is being calculated from both directions and we keep the one that produces the higher score. The maximum value for the containment is 1.0 which means that document 𝑑 1 is fully contained inside document 𝑑 2 . The result in each comparison is rounded up to one decimal and the measure is calculated for all possible pairs of documentsremoving from the final set those with containment equal to zero 0.0.

Containment Based Clustering

Having extracted the containment scores between all pairs of documents and keeping only the directions that produce the largest scores we now proceed to the clusters. Specifically, each cluster will be formed by those documents that the containment score between them belongs to a specific pre-defined set of values. One document may be present in more than one clusters.

For the final clustering of documents we have three tools: the content (containment), the value of the SMLE and the mapping between senses and glosses. We choose to use the simplest approach, namely the containment, to divide the texts into clusters. Specifically, having each time one hundred documents for processing we can put clusters of ten/fifteen documents setting containments limits in the range intervals of 0.1 units to meet separately all possible spaces between 0.1 and 0.9. The number of fifteen documents per cluster emerged after experiments. As for the other two metrics, the SMLE will be used for the internal ranking of documents within the cluster so that to get the final ranking of the representations of the query, in the last step of our processing. Finally, the mapping between concepts and glosses on the subject, can be used for further analysis but we stop at this point because we aim to end use only glosses. Nevertheless, one can combine senses and glosses and produce information more well defined to the nature of documents. Such information can produce more glosses for the representation of documents, to combine concepts from each other so that the glosses to be more targeted. We leave this as a matter of future research.

Returning to the clustering we develop the algorithm to extract clusters according to the analysis we did earlier. First, we define the first cluster with documents having containments between the 0.8 and 0.9 interval. Then we repeat this step every 0.1 units: 0.6 -0.7, 0.4 -0.5, 0.2 -0.3 and 0.1.

In particular we run the list of document pairs and using their containment metric we pick up initially all pairs of values in the content space 0.8-0.9. Then we collect all the documents whose containment has values between 0.6-0.7. There is always the case of a document from the previous calculated interval, to appear and we permit this as case, considering that this raises the chances of finding more similarity between query and representation of a cluster because of the extra vocabulary the document adds to that cluster. We repeat the same procedure for the remaining intervals. When we completed the separation in the first cluster, we can reduce even more the documents per cluster and display different documents for the same intervals.

The distribution of the documents inside the clusters resembles a normal one; clusters produced for scores between 0.3 and 0.7 contain larger numbers of documents than clusters near the edge with scores 0.2 to 0.1 or 0.8 to 1.0 as shown in Figure 5.1:

Figure 5-1 Normal Distribution of contained documents

Finally, based on the SMLE language model d, the score for each document inside the cluster is being produced. The fixed vocabulary 𝑉 for each cluster which is used for this score is derived from the separate vocabularies of all documents contained in this cluster. The glosses occur one time at most since the global cluster vocabulary must not contain redundant information. The score for each document varies per cluster it belongs to.

It is important to mention two key characteristics for the above process:

1. The score for each document is produced based on the fixed vocabulary of each cluster it belongs to as such one document may have more than one scores based on the clusters it belongs to. 2. The option to create sub clusters is also utilized in our approach and the technique aims to create a more uniform representation inside the clusters in respect to the number of contained documents. It proves that the aforementioned technique performs worse than the normal distribution. To process the query we follow the same lexical and language model probability approaches, where we tokenize the input, lemmatize the separate terms and provide them as input to WordNet 2.1 in order to retrieve the respective glosses. In the document processing we chose to keep those glosses, whose senses produced the highest scores for the terms. For the query we choose to keep all glosses produced by the Word-Net 2.1 schema. Our intuition behind this decision is that the user who posed the query might have had in mind any specific intention out of various possible ones; so, if a query can mean a lot of different things we have to keep all those meanings instead of just some.

In the final step of this process we reformulate the query by producing all possible glosses for each of its terms and we check all the possible combinations between them as shown if Figure 6.1. The final output will be a set of different query representations containing glosses. To produce the final list of documents that will be returned to the user, our algorithm calculates a similarity score between the global fixed vocabulary 𝑉 of each cluster and each of the query representations. We annotate the query representations as: 𝑞 𝑖 ,1 < 𝑞 < 𝑄, where 𝑄 is the total number of query representations. Each cluster's vocabulary is annotated as: 𝑐 𝑖 , 1 < 𝑖 < 𝐶, where 𝐶 is the total number of clusters. At the first step we produce the similarity measurement score between the two quantities for each query representation and each cluster's vocabulary.

A lot of techniques have been developed for lexical similarities over the years basically for application over characters between words. In our framework we adjusted a few of them to be performed over sentenceswords instead of characters and sentences instead of words. We concluded in the use of two methods: Positional N-Grams similarity and Levenshtein Distance. The N-Gram similarity was used with position 𝑛 = 4. The advantage of the use of the aforementioned similarity techniques is that they offer the possibility of two sentences to be partially similar and not totally the same. Between the two techniques, N-Gram executes in lesser time than Levenshtein Distance and produces better similarity scores. Based on comparing every representation 𝑞 𝑖 with every cluster 𝐶 𝑗 , there will be produced Similarities: 𝑆 𝑖𝑗 : 𝑆 13 … … The final score of each document is produced through Borda Count that combines the similarity between clusters and query representations with the internal document score as produced in each cluster. In particular the combined lists correspond to every distinct combination of cluster and query representation, the lists are ranked according to the internal scores of the document in each cluster and during the combination we use as extra weight the similarity between the query and the clusters.

Experimental evaluation

To perform our evaluation we explored 20 web queries from TREC 2012 WebTracks. We selected queries with the best characteristics against WordNet 2.1 and in respect to the quality of their corpus, in order to perform the experiments. All tracks use the 1 billion page ClueWeb09 (http://lemurproject.org/clueweb09/) dataset and contain a diversity task that contains a ranked list of documents that covers the query topic avoiding information redundancy. TREC assessors have assigned appropriate relevance judgement scores for the documents related to each of the queries. For our results we have employed Google and AOL search engines and for each we have retained the top 100 pages returned. Common identifier of the quality of the results (both for our algorithm as well as for the Google and AOL engines results) is the N-DCG metric (Baeza Yates & Neto, 2011; [START_REF] Manning | Introduction to information retrieval[END_REF] .

Each query is being examined against WordNet and confirmed that all of its terms extract the necessary informationotherwise the query is dismissed. We retrieved up to 200 documents from the ClueWeb09 corpus related to each query. For the clustering of the documents we used the containment metric and created the clusters based on the following sets of values: 0.8 -1.0, 0.6-0.7, 0.4-0.5, 0.2-0.3 and 0-0.1. To measure the efficiency of our method we used (a) the relevance judgement scores from TREC 2012 specialists and (b) the N-DGC, Normalized Discounted Cumulative Gain metric.

When comparing our algorithm with the ideal list of documents returned for each query we have a success of 75% in a very well controlled environment where: i) the stop words list is extended to contain additional words based on the previous results, ii) clustering is preserved in the values presented above, iii) queries selected have documents with strict normal distribution of scores, iv) we keep all glosses for the query terms and v) we change the threshold in our ranking techniques per case. The following figure 8-1 represents the most representative samples of our results with scores [0, 4]. In the vertical axis 0 indicates no relativity at all between the document and the query and 4 is the highest relativity between them. In the horizontal axis, reside the documents evaluated for the query. Finally, our algorithm was assessed against Google and AOL results for the same queries. The vertical axis in the following figure 8-2, shows the N-DCG metric and the horizontal contains the 20 queries. From the results, our algorithm performs much better than Google (69%) and better than AOL (71%). Google decides the returned list through Page Rank and focuses on user "clicks" as well as the incoming and outgoing links between pages. As such pages with lesser interactions between each other are hardly going to be presented in the list. On the other hand, the close values between our algorithm (75%) and AOLwhich are both higher than Google's -could be explained through the use of a common technique between the two approaches which is the clustering of the pages before they are returned for a specific query. Finally, we perform a soft comparison of our algorithm with the techniques and approaches presented in the related researches (Giakoumi, Makris, and Plegas, 2015; Plegas and Stamou, 2012) to further evaluate our algorithm. Our test range was much more limited than the ones used in the aforementioned paper as such the results are inconclusive. Specifically, our algorithm seems to perform 5% -10% better but the queries used are not the same nor in the same extensive number. At the same time, the range of the corpus used in our experiments is significantly smaller mainly due to the limited capacity and related resources used for our research as such further experiments are necessary to conclude.

Conclusions

In this paper we presented a framework in an attempt to improve the quality in the retrieval performance of search engines. The novel approach of our research is the use of a more generalized structure unit, namely gloss, in order to represent our texts throughout language and probabilistic models in conjunction with a lately very effective techniqueclustering. Our main goal in this approach is to introduce a more generalized way of modeling the input in a conceptual way in order to enforce the inclusion of independent modules, re-usable and easily attached to different methods and techniques. This way a model can be flexible enough to be attached in a variety of approaches. At the same time, our algorithm needs further improvement. In our approach the most significant one concerns the use of WordNet where the interface for the input has limited options; a different lemmatizer of stemmer could be developed to provide the words to WordNet in a better comprehensive and pos-tagged way without losing information. To this extend someone can further enhance the scores by applying the Wikification techniques in the final document ranking.

In a more complex approach, an entirely different thesaurus could also be used, like YAGO which offers a much better framework for conceptual analysis but so far lacks the tools to provide massive input and assess its results.

Figure 6 - 1

 61 Figure 6-1 Normal Distribution of contained documents

Figure 8 - 1

 81 Figure 8-1 Algorithm evaluation against TREC assessment.

Figure 8 - 2

 82 Figure 8-2 Algorithm evaluation against Google and AOL.


Despoina Mouratidis

Katia Lida Kermanidis

Automatic Selection of Parallel Data for Machine Translation

Keywords: Machine Learning, Educational data, Data selection, Machine Translation, Random Forests

Nowadays machine translation is widely used, but the required data for training, tuning and testing a machine translation engine is often not sufficient or not useful. The automatic selection of data that are qualitatively appropriate for building translation models can help improve translation accuracy. In this paper, we used a large parallel corpus of educational video lecture subtitles as well as text posted by students and lecturers on the course fora. The text is quite challenging to translate due to the scientific domains involved and its informal genre. We applied a random forest classification schema on the output of three machine translation models (one based on statistical machine translation and two on neural machine translation) in order to automatically identify the best output. The unorthodox language phenomena observed as well as the richin-terminology scientific domains addressed in the educational video lectures, the language-independent nature of the approach, and the tackled three-class classification problem constitute innovative challenges of the work described herein.

Introduction

In recent years, many people, companies and organizations make use of machine translation (MT) solutions. MT software has been improving, and researchers are trying to generate the best translation of a source text. The use of MT is said to have become an indispensable tool, not only for scientific purposes, but also for the general public. Moreover, automatic translation contributes decisively to the learning process, since it can extend the learning target group by breaking the language barrier and enhancing access to the educational material. To this end, the European project Tra-MOOC (Translation for Massive Open Online Courses) [START_REF] Kordoni | TraMOOC (Translation for Massive Open Online Courses): Providing Reliable MT for MOOCs[END_REF] aims at improving the translation process, and overcoming the language barriers in online educational content.

After almost half a century that statistical approach prevailed in MT [START_REF] Koehn | Statistical phrase-based translation[END_REF], a new method, the neuralbased approach, appeared. This, in contrast to statistical machine translation (SMT) implemented by using parallel text corpora to calculate probabilities, generates much more accurate translations. More specifically, neural machine translation (NMT) implements deep learning techniques to teach text translations by taking on existing statistical models as a basis. Also, NMT is able to use algorithms to train itself with linguistic rules [START_REF] Bentivogli | Neural versus phrase-based machine translation quality: a case study[END_REF].

Certainly, many challenges occur in the translation process. Additionally, it should be noted that there is difficulty in translation out of domain data. Therefore, there is a large amount of data to be translated. For large sentences, an extension of the classical neural encoder-decoder can be used, taking into account only the words which have information relevant to the target word and not the whole sentence [START_REF] Bahdanau | Neural machine translation by jointly learning to align and translate[END_REF]. Post-editing and data selection are two ways to reduce the data (i.e. choosing only quality parallel segments) without hurting translation quality. Many online MT platforms now prompt users to improve the proposed translation themselves [START_REF] Peris | Online Learning for Neural Machine Translation Post-editing[END_REF]. This may be a solution to improve translation models, but it also creates a multitude of data that needs to be evaluated regarding usability. On the other hand, data selection methods can be used to recognize the useful and non-useful features in parallel segments. Research has shown that when models are trained with less, but more accurate, data, their performance improves [START_REF] Sharaf | The UMD Neural Machine Translation Systems at WMT17 Bandit Learning Task[END_REF].

In this paper, we consider data selection as a classification problem and we explore the idea of using three translation prototypes for our experiments, one based on SMT and the other two based on NMT. The contribution of this paper is multi-fold:

 the educational content domain comprises scientific fields that involve a high degree of terminology and unknown words. This phenomenon requires a set of robust learning features to represent the parallel text segments.  the informal genre (spontaneous speech transcriptions and forum text) presents linguistic phenomena that are unorthodox and ungrammatical, like repetitions, interjections, fillers, truncated utterances etc., posing a challenge to the automatic identification of grammatical utterances.  the proposed approach is language independent. All linguistic features are based on string similarity and no morphosyntactic information is incorporated in any form.  a metalearner (Random Forest) is employed for data classification, for the first time for the task at hand to the authors' knowledge, in order to tackle the aforementioned challenges.

Experimental Setup

This section describes the corpora, tools and the classification process used.

Corpus

The parallel corpus we worked on was provided from the TraMOOC project. As already mentioned, the corpus includes lecture speech transcriptions and text posted by class participants on course fora. The source corpus consists of 2,687 segments (sentences) in English (Src). For each of these segments, three translation outputs into Greek are available, generated by three prototypes (Trans1, Trans2, Trans3), whereas one reference translation (Ref) from a professional translator is also provided. Translation model 1 (Trans1) used the open-source phrase-based SMT toolkit Moses [START_REF] Koehn | Moses: Open source toolkit for statistical machine translation[END_REF], the translation models 2 (Trans2) and 3 (Trans3) used the NMT Nematus toolkit [START_REF] Sennrich | Nematus: a toolkit for neural machine translation[END_REF]. Trans1 is a statistical based prototype trained on both in-and out-of-domain data.

Trans2 is trained on the same data as Trans1 and uses labels to identify and remember the domain, while Trans3 is the result of training with more in-domain data providing via crowdsourcing, weight tying, layer normalization, and improved domain adaptation. Out of domain data included widely known corpora e.g. Europal, JRC-Acquis, OPUS, WMT News corpora etc. In domain data included TED, QED corpus, Coursera etc. [START_REF] Miceli Barone | Regularization techniques for ne-tuning in neural machine translation[END_REF]. Data pre-processing included the removal of symbols (for example #, $), and some alignment corrections, so that each segment is mapped to its Src, Ref, Trans1, Trans2 and Trans3 variations.

A challenge was the translation of entities like URLs, mathematical expressions and rare words. The first two entity types were copied to the translation output by some prototypes (Trans2 & Trans3, and Trans3 respectively), while the third type is tackled by the third prototype by word division in order to improve MT output [START_REF] Sennrich | Neural machine translation of rare words with subword units[END_REF].

Annotation

Two Greek linguists have annotated each segment with Α, Β or C depending on whether Trans1, 2 or 3 is more similar to Ref respectively. We observe low annotation percentage for class A (17%) compared with class B (37%) and C (46%). This confirms the superiority of the NMT vs. SMT models. At this point, it's important to notice that the two annotators gave different answers in 82 of the 2,687 segment cases. For the different answers, the annotators had a discussion and finally agreed on one common label. We present five segments and their Trans1-2-3 and Ref translations (Table 1), the sixth segment is an example of disagreement of two linguistics. ii) has been called: the three trans didn't change the passive into the active form. Trans1 and 2 gave the most common meaning (κληθεί). Nevertheless, the more successful translation of Trans3 (ονομαστεί) makes a pleonasm with the object (ονόματα), Ref's choice being the correct (πάρει). ID 2: i) which: None of the three Trans translated correctly this question word, not being the first word of the segment.

ii) Trans1 incorrectly connected which to history giving the same grammatical gender (ιστορία...η οποία). Trans2 chose the sometimes confusing, but very common, που (not the question word πού). Trans3 incorrectly connected which to a bit giving the same grammatical gender (κομμάτι...το οποίο).

iii) now: None of the three Trans translated it as an adjective. iv) Trans1 and Trans2 didn't connect the word game to character as a genitive case (παιχνίδι...χαρακτήρας), as was done correctly by Trans3 (χαρακτήρας βιντεοπαιχνιδιού).

ID 3: i) This is where: no metaphorical sense by the three Trans. ii) studying: the same translation by Trans2 and Trans3 (μελέτη), but not expressing the action, the process, as a verb would have done. Trans1 uses a verb (σπουδάζουν) and the sense of "process" is given also by adding an object but the syntax generates a pleonasm (σπουδάζουν σειρά μαθημάτων) and the syntax of the segment is incorrect.

ID 4: i) Authorship: Trans1 translated this word by the word Niemann that is nonexisting in the source segment. It's important to note that we find the Niemann Statement in Harvard and other contexts and this is relevant with essays and authors. Very interesting, (but the result is completely false) is also the Trans2 translation process: from the basic meanings of the whole word (authorship= origin, source) Trans2 uses a synonym: αρχαίου (ancient), but at the same time it separately translates the second synthetic of the word (-ship) to give the common phrase: αρχαίου πλοίου.

ii) essay: Trans1 gives the main meaning of the word (δοκίμιο), but the Trans2 and Trans3 choices are also correct (εργασία, έκθεση), Trans3 choice being Ref's choice.

iii) zip file: Trans1 translates zip by the common adjective of file, but here it is irrelevant: ταχυδρομικό. The Trans2 translation (φερμουάρ) is completely irrelevant here, but very common in other contexts. Trans3 correctly doesn't translate zip in this context.

ID 5: i) get: Trans1 gives the most common translation (πάρετε), but it is not correct here. Trans2 and Trans3 correctly translate this multi-sense word.

ii) You: Only Trans1 correctly translates You as a plural pronoun. iii) lean: None of the three Trans is correct (λιτή, λυγίσει, γείρει) compared to Ref's correct choice (μπει). The word here has a very special metaphorical meaning: "to enter". The sense of "motion" of the preposition into, in the Source text, is partly conveyed in Trans3 (γείρει).

ID 6: Annotator 1 labeled Trans2 as the better translation for the following reasons: 1) έρευνα is a better translation for exploration in this segment, as the main meaning of the word (εξερεύνηση) here is not precise, 2) προέλευση for the word origins is the best translation in this segment and is the same as the Ref tranlation, 2) εξετάστε is not the best translation for check-out in this segment but is better than ελέγξτε, because its meaning is not the primary one (i.e. check) but closer to other secondary meanings of check, like "note" or "hold". By check-out, in combination with the word whole, the writer here means: "read" or better "study and keep in mind", but it can't be translated so, as it is far from the meaning of check, 3) Trans2 kept the proper noun Hobohm in Latin letters, like Ref, and as it is considered to be good practice for dealing with proper nouns from one language to another.

Annotator 2 labeled Trans3, as the better translation for the following reasons: 1) εξερεύνηση is the exact meaning of "exploration", in combination with its prepositional phrase of the origins, implying "deeper research" (εξερεύνηση being more exploratory than a simple research (έρευνα)), 2) της καταγωγής is the primary and most common meaning for of the origins, as it refers to "the first appearance", to "the creation" of the subjective genitive: storytelling, 3) Trans3 changed the Latin into Greek letters for the proper noun: Χόμπομ, as the target language is Greek, and it is common practice to do so.

Features

We considered the task at hand as a classification problem with three output (class) values, so we represented each segment as a tuple (Src, Trans1-2-3, Ref). Each tuple was modeled as a feature-value vector, while the features are based on string similarity, they contain no form of morhosyntactic information, and are therefore language independent. The feature set was based on the work by Barron-Cedeno et al. [START_REF] Barrón-Cedeño | Identifying useful human correction feedback from an online machine translation service[END_REF] and Pighin and May [START_REF] Pighin | An Analysis (and an Annotated Corpus) of User Responses to Machine Translation Output[END_REF]. Feature values were calculated using MATLAB.

Basic-Simple Features

These are simple string similarity features. Levenshtein distance is a string similarity metric, which calculates the minimum number of single-character changes re-quired to change one word into the other. Also, another string similarity metric was used to determine if Trans 1-2-3 is contained in Ref ( Containment c ) [START_REF] Broder | On the resemblance and containment of documents[END_REF].

Results

We have nominal and numeric features. We normalized the numeric features so that their values range between 0 and 1, by using the Feature scaling method. We decided to use the Weka machine learning workbench [START_REF] Singhal | A study on WEKA tool for data preprocessing, classification and clustering[END_REF] for training and testing our dataset. We used evaluation measures that are common in classification, and adopted from Information Retrieval. The first measure is Precision, that is True Positive / (True Positive + False Positive). The second measure is Sensitivity -(Recall), that is True Positive / (True Positive + False Negative).

Given the challenges governing the genre and the domain of the data, we decided to apply a meta-learner for increased robustness. We chose the Random Forest classifier, due to their using the Law of Large Numbers and the ability to avoid overfitting [START_REF] Breiman | Random forests[END_REF], and achieving high generalization accuracy. Random Forests implement an ensemble learning schema that generates multiple decision trees during training, and constructs a combination of the classification outputs of each tree model for prediction. We set the number of iterations (number of trees to be constructed) to 65. Each tree was constructed while considering 20 random features. We employed 10 fold cross validation as testing mode. The minority class (A) causes problems in the classification process: the classification algorithms give low accuracy as they tend to classify the new unseen segments in the majority class [START_REF] Daskalaki | Evaluation of classifiers for an uneven class distribution problem[END_REF]. In order to improve the accuracy of the classifier for the minority class (precision 49%, recall 22%), we used the Smote filter [START_REF] Chawla | SMOTE: synthetic minority over-sampling technique[END_REF], which is an over-sampling approach for creating new synthetic training data. Smote combines the feature values of minority class examples with the feature values of their nearest neighbor examples (n=5) in order to produce new examples of the minority class. The Smote process is applied only on the training data. Using Smote, the segments of class A doubled in number, and the total number of segments reached 3150. We observed that we had better results when we used RandomForest_Smote including all the features as seen in Table 2. It is noted that the results obtained are satisfactory, given that in our experiment we had three classification values, in contrast to related research that targeted a binary class output [START_REF] Barrón-Cedeño | Identifying useful human correction feedback from an online machine translation service[END_REF]. Moreover, the features we used are simple string comparison features, and they are language independent, including no morphosyntactic information in any form.

It is noted that the results obtained are satisfactory, given that in our experiment we had three classification values, in contrast to the [START_REF] Barrón-Cedeño | Identifying useful human correction feedback from an online machine translation service[END_REF] research. In addition, the features we used are simple string comparison features, and they are language independent.

We observed in the table that when we applied RandomForest before the Smote filtering, the classifier correctly classified 22% of A segments for Class A (Trans1), 36% of B segments for Class B (Trans2) and 70% of C segments for Class C (Trans3). After the Smote process, a major change is observed in Class A, where the percentage increased to 64%. For classes B and C we did not notice any particular changes. What is remarkable is that when the classifier does not sort correctly, it usually classifies the segments from one neural model to another (60% B -> C and 25% C -> B), and a much smaller percentage to the statistical model (7% C -> A and 8% B -> A) as well. In total, we can see in the figure below (Fig. 1) the percentages of incorrectly classified instances. We note that Trans1 does not apply the basic syntactic rules, i.e. the subject-verb agreement, the subject-predicate agreement, as well as the modifiers agreements (attributive adjectives, predicate adjectives). Non-agreement is also observed in genitive constructions (possessive case, subjective and objective genitives), when of course there is not of (genitive case) or by. However, it has been found that Trans1 has, in many cases, a richer vocabulary than Trans2 and Trans3. In addition, Trans1 retains the main names, as Trans2 also does, in the Latin script, as it is considered right, and does the same in words not existing in its vocabulary, avoiding false and unrelated translations, as in Trans2 and Trans3 sometimes occurs. Trans2 applies the above agreements, but not always successfully. Trans2 disposes quite satisfactory vocabulary, but not always about words that having more than two basic meanings. However, as it has been said, Trans2 translates all the common words, even those that do not exist in its vocabulary, breaking up compound words into their components and translating them, but, in some cases, this translation is wrong. Trans3 applies the above agreements more successfully than Trans2, it translates more successfully the components of compound words, but, as it has been said, Trans3 lags somewhat to the vocabulary richness.

It is important to know which features are more important to the classifier, so we tried the attribute evaluator technique (in Weka). Ratio of length in words and ratio of length in characters seem to be functional, as well as the Length Factor (LF), as we have described in section 2.3. On the other hand, comparisons, like if Ref=Trans1-2-3, seem not to be so useful for the classifier.

Conclusions and Future Work

In conclusion, this study aimed at automatic data selection for machine translation. It is based on the processing of a sufficiently large parallel corpora database. In this regard, we considered the data selection task as a classification problem. More specifically, three translation models were used, which represent both the old approach (SMT) and the state of art (NMT) to MT. In this way, differences in the translation process and the approach of the three models become more apparent. 82 characteristics have been calculated and 2,687 segments have been annotated. For proper analysis, we pre-processed our data before using Weka tool. We used Smote to address the class imbalance problem in our data. The results recorded give a better translational prediction to model 3, which does not make much of an impression, as this is a sophisticated translation model. It is worth mentioning that the translation was from English to Greek, which increased the task complexity, since the Greek language is a morohologically rich language with ambiguities. One way to more accurately approach ambiguities in the future might be the use of data categorization. For example, grammatical categorization may prove far superior to the lexical features employed herein, an approach that has already been considered for the Greek language [START_REF] Stamatatos | Automatic text categorization in terms of genre and author[END_REF]. Furthermore, it could be studied whether the use of in-depth features influences the translation process, such as etymology, that is believed to be of great help for the Greek language.

It is worth asking ourselves whether we can find similar results amongst other language pairs, and this may be a new field for study.



  Length (in number of words) of Src-Trans1-Trans2-Trans3-Ref.  Length in words of Trans1, Trans2, Trans3, Ref divided by Src, also for Trans1, Trans2, Trans3 divided by Ref.  Length in characters divided by length in words for Src, Trans1, Trans2, Trans3 and Ref.  Levenshtein distance of Trans1, Trans2, Trans3 divided by Length of words and characters of Trans1, Trans2, Trans3.  Number of words that exist in Trans and do not exist in the Ref divided by the number of words in Trans (and vice versa).  Containment c of Trans1-Ref, Trasn2-Ref, Trans3-Ref [5].  Ratio of (third bullet)'s resulting features between (Trans1-Trans2-Trans3, Src), (Ref, Src), and (Trans1-Trans2-Trans3, Ref).  Longest word for Src, Trans1, Trans2, Trans3 and Ref.  Longest word in Trans1, Trans2, Trans3 divided by Src and Ref and longest word in Ref divided by longest word in Src.  If Ref=Trans1 or Trans2 or Trans3, then True, otherwise False, if Src = Trans1 or Trans2 or Trans3, then True, otherwise False. Noise-based Features  If Src is a one word string then True, otherwise False.  If Src is a string of more than five words then True, otherwise False.  If Src is a string with length six to ten words then True, otherwise False.  If Src is a string with length up to eleven words then True, otherwise False.  If Src, Trans1, Trans2, Trans3, Ref has a word with length 10 to 14 characters then True otherwise False. We did the same with word length 15 to infinity.  If Src or Trans1 or Trans2 or Trans3 has a word of three repeated characters then True, otherwise False. Similarity-based Features  The length factors (LF-defined in [14]), LF(Ref, Trans1), LF(Ref, Trans2) and LF(Ref, Trans3) are calculated.  Using the LF (described above), if LF(Ref, Trans1)>LF(Ref, Tans2) then True, otherwise False. The same comparison is performed on LF(Ref, Trans2) and LF(Ref, Trans3), as well as on LF(Ref, Trans1) and LF(Ref, Trans3) (and vice versa).

Fig. 1 .

 1 Fig. 1. Total percentages for incorrectly classified instances.

Table 1 .

 1 Segment Examples from source, Trans1, Trans2, Trans3 and Ref. No translation by Trans1 (not found). Trans2 and Trans3 correctly separate the two synthetics. Trans2 translates the first synthetic as a main word (αρχαϊκό=archaic), common in historical contexts, but not correct in this segment. Trans3 finds the meaning of the prefix: arch-(archi › αρχή, αρχική).

	ID	Source	Trans1	Trans2	Trans3	Ref
	1	The archplot	Η archplot	Το αρχαϊκό	Η αρχική	Η κύρια
		refers to the	αναφέρεται	σχέδιο αναφέ-	πλοκή αναφέ-	πλοκή ανα-
		classical	στην κλασι-	ρεται στον	ρεται στον	φέρεται
		design of a	κή σχεδια-	κλασικό σχε-	κλασσικό	στην τυπική
		story and has	σμό	μιας	διασμό μιας	σχεδιασμό	διαμόρφω-
		been called	ιστορίας και	ιστορίας και	μιας ιστορίας	ση μιας
		by many	έχει κληθεί	έχει κληθεί	και έχει ονο-	ιστορίας και
		names.	με	πολλά	από πολλά	μαστεί από	έχει πάρει
			ονόματα.	ονόματα.	πολλά ονόμα-	πολλά ονό-
						τα.	ματα.
	2	A bit of	Ένα κομμάτι	Ένα κομμάτι	Ένα κομμάτι	Λίγη ιστο-
		gaming his-	της ιστορίας	της ιστορίας	ιστορίας παι-	ρία παιχνι-

Table 2 .

 2 Precision and Recall of our experiments.

	Classifier : RandomForest
	Class	Precision	Recall	Number of	Number of
				features	instances
	A	49%	22%	82	2687
	B	46%	36%	82	2687
	C	50%	70%	82	2687
	Classifier : RandomForest_Smote
	A	77%	63%	82	3150
	B	44%	32%	82	3150
	C	50%	68%	82	3150


Ioannis Neokosmidis
email: i.neokosmidis@incites.eu

Theodoros Rokkas

Dimitris Xydias

Antonino Albanese

Muhammad Shuaib Siddiqui

Carlos Colman-Meixner

Dimitra Simeonidou

Are 5G Networks and the Neutral Host Model the Solution to the Shrinking Telecom Market

Keywords: 5G networks, business model, cloud computing, neutral host, NFV, SDN, telecom market

5G networks will offer advanced functionalities and features such as high throughput, low latency, security and reliability by leveraging technological advances like network virtualization, edge computing and network slicing. The latter will facilitate the entrance of new players in the value chain. Moreover, they will offer a means for network operators to change their business models in order to address revenue losses. The concept of a neutral host supported by the H2020 5GCity project seems to be a strong candidate for future business models helping network operators thrive in the new telecom era.

Introduction

During the last decade, European telecom operators have been experiencing declines in their revenues, mainly because service prices stagnation, regulations, and an increasing demand of investments in their infrastructures. As a result, operators have been looking for innovative ways to turnaround this negative situation by considering the adoption of 5G networks given their technological, economical, and social values [START_REF] Report | Identification and quantification of key socio-economic data for the Strategic Planning of 5G introduction in Europe[END_REF].

The 5G-PPP association has identified a series of specifications and KPIs [2] for 5G networks to be addressed by several ongoing initiatives and projects. The 5G networks KPIs are ultra-high data rates, low latency, reliability and, host neutrality. The last KPI will give a very important role to 5G technology in enabling new players from the vertical industries to enter the digital value chain, something that has not been done yet with the currently available technologies. This role will create significant social value through cutting-edge technological applications in various industries, such as medical care, transportation, and entertainment.

Hence, 5G networks are expected to offer new business opportunities and business models like neutral host to existing network operators. This can be attributed to the fact that 5G networks will make use of technological advances in network function virtualization [START_REF] Mosharaf | A Survey of Network Virtualization[END_REF], software-defined networks (SDN) [START_REF] Nunes | A Survey of Software-Defined Networking: Past, Present, and Future of Programmable Networks[END_REF], edge computing, end-to-end network slicing and network analytics. Such innovative technologies will improve network performance and facilitate the development of sliceable applications and services.

This paper aims to investigate the neutral host business model, as proposed in the 5GCity project, along with the necessary changes that network operators need to make in their business models to thrive in the new digital era. The obtained results will be a valuable tool for decision makers, so to achieve the maximum exploitation of the technological advances of 5G and accelerate the return of network operators to revenue growth.

The objective of 5GCity [5], an innovation project financed within the 5G Public-Private Partnership (5GPPP) initiative by the European Commission (Horizon 2020 program), is the design, development, deployment and demonstration of a distributed cloud and radio platform for municipalities and infrastructure owners acting as 5G neutral hosts. The project's aim is to build and deploy a common, multi-tenant, open platform that extends the (centralized) cloud model to the extreme edge of the network, with demonstrations in three different cities (Barcelona, Bristol and Lucca). 5GCity will directly impact a large and varied range of actors: (i) telecom providers; (ii) municipalities; and (iii) several different vertical sectors utilizing the city infrastructure.

The rest of the paper is organized as follows: Section 2 presents the status of today's telecom market. Section 3 introduces the expected future of telecom networks along with the involved actors. Section 4 describes the required changes for network operators to increase their revenue. Section 5 introduces the role of a neutral host in a 5G Small Cell network. Finally, section 6 concludes this work.

Status of Today's Telecom Market

The impact of the financial crisis of 2008 has different repercussions in EU based on each country's economic structure. As expected, southern European countries were strongly influenced by this crisis. While the growth of GDP in Greece remains negative, Italy and Spain have recovered the growth last year. However, ICT networks provide the backbone for digital products and services that have the potential to support all aspects of our lives, and drive Europe's economic recovery [START_REF]Communication from the European Commission on "A Digital Single Market Strategy for Europe[END_REF]. Well-functioning markets deliver access to high-performance fixed and wireless broadband infrastructure, at affordable prices. Successive adaptations of the EU's telecoms rules combined with the application of EU competition rules, have been instrumental in ensuring that markets operate more competitively, bringing lower prices and better quality of service to consumers and businesses. Effective competition is a key driver for investment in telecoms networks. Electronic communications sector represented 2,1% of EU's GDP in 2014, compared to 2,3% in 2013 and 2,7% in 2009. In 2014 the revenues of the electronic communications sector in the European Union were estimated at 300 billion euros while investment was about ~12% of the total turnover (Figure 1). This can be attributed to the mobile termination rate cuts and other regulatory interventions (e.g. EU Zero Roaming) leading to ARPU squeeze as well as to the proliferation of bundles and decrease of voice traffic. Figure 3 illustrates the average revenue per user in the retail mobile market. inCITES forecasts the service revenues to grow at a 0.8% CAGR in the mobile case and to drop at a -0.3% CAGR in the fixed case between 2017-22. It is evident that the forecasted increase in revenues is very small if one takes into account the ongoing growth in data traffic. According to Cisco Visual Networking Index (VNI), the global mobile data traffic grew an estimated 74 percent in 2015. Growth rates varied widely by region, with Central and Eastern Europe at 71% followed by Western Europe (52% in 2015). In 2015 mobile data traffic was 545.750 and 432.322 Terabytes per month in Central & Eastern Europe and Western Europe respectively (Statista).

3

The Future of Telecom Networks -Involved Actors

The value creation in the telecommunications market is no longer the case of a closed market limited to few players: new roles and relationships are evolved and become part of the new ecosystem. Each actor must understand its position to increase its value and maximize the potential revenues. The following section describes the initial insights regarding the value chain analysis for future 5G networks. The involved actors and their relative position in the value chain is depicted in Figure 4.

Hardware (Equipment) manufacturers / vendors

This actor is on the beginning of the value chain and provides all the necessary equipment to all other involved. It includes entities that either manufactures and/or sells equipment. This equipment can be ICT related like servers, hard disk drives, RAM etc., networking equipment like routers and switches and radio equipment like small cells. In a broader perspective, we can also include vendors that manufacture equipment associated with the city infrastructure such as lampposts, cabinets, cameras, sensors etc. It also includes vendors providing devices for End Users such as mobile phones, tablets and wearable devices that allow the delivery of enhanced 5G services like augmented reality or UHD video.

Software Developers

These entities develop the necessary software as well as the Virtual Network Functions (VNFs) for the delivery / provision of 5G services. The VNFs can be classified as i) those that are necessary for the delivery of the service and ii) those that are complementary and are tailored to fulfil the requirements of users like the media vertical or immersive services. This type of actor also includes entities who develop applications, like for example video analytics or augmented reality software.

Infrastructure owners

These entities own the necessary infrastructure that can be used for hosting the computing, storage and networking infrastructure. The available resources include space in street cabinets, lampposts or buildings along with power supply and connectivity facilities. These are necessary to power and interconnect the ICT equipment. Municipalities can be included in this category and are expected to play a significant role since they possess both facilities and ICT infrastructure. Municipalities will act as a Neutral Host providing wholesale access to interested parties (network operators). Thus, such entities will be of high importance since they enable network densification and keeping low costs.

Network Operators (ICT infrastructure providers)

This category includes all types of network operators like mobile and fixed, cloud providers, data centers that owns the physical resources of the network. It also includes entities that own spectrum licenses. This entity will receive wholesale access from Neutral Host in order to provide services to end-users.

Content/Service Providers

It includes entities that does not own network resources but use virtual resources to create and provide their own services. By taking advantage of the Neutral Host concept, service providers have a pool of different type of resources that can select and use to develop new and innovative services. The type of services they offer can be connectivity, Internet access, content distribution etc. They can develop their own content or come into agreement with other Content Providers or even End Users. They enter into Service Level Agreements with Network Operators to get access to the physical resources.

End Users

This term covers all short of users that range from simple users that seek only connectivity services to vertical industries that have more specific and tight requirements. They are the ultimate consumers of the services created in the telecommunication ecosystem. Furthermore, End Users can become content producers to Service Providers.

Mandatory Changes in Operators' Business Models

In this section, the necessary changes that will allow network operators to thrive in the new digital era are investigated and presented.

Network operators to exploit their strong position as network owners

In the last years, network operators have seen OTTs and vendors gaining a big part of end-user relationships. Taking also into account that connectivity is no longer profitable, network operators are seeking a role as intermediates between vendors and OTTs facilitating partnerships and adopting a wholesale and a "smart pipe" model towards success. In the so-called neutral host model, OTTs, virtual operators and third parties are leasing access from network operators in order to provide their services while vendors are attracted by operators' large scale. Moreover, manufacturers and vendors will be able to connect their devices in the new Internet of Things era. Network operators should resort to recent technologies such as network analytics, software defined networks and network virtualization to optimize network operation and improve its performance. By leveraging cloud computing, networks operators are aiming to a digital single market where they can provide their services remotely from any part of the value chain leading to significant savings due to economies of scale.

Networks operators to get closer to clients' needs

Networks operators should leverage recent technological achievements in order to provide tailor-made personalized services to their customers. Networks operators are probably the only entity that is able to securely collect and analyze users' data. Using big data analytics, valuable knowledge can be extracted enabling them to adapt their products and investment plans in order to meet B2B and B2C requirements. Using advanced computing capabilities, networks operators can now adopt different (e.g. per usage) and / or hybrid pricing schemes and offer customer and price differentiation.

Required Investments

In order to stay on top of technological advances and achieve the aforementioned transformations, networks operators should also change their investment strategies. On the one hand, networks operators should definitely continue investing in infrastructure in order to expand their networks. Since this is a capital-intensive process, networks operators should resort to alternative funding schemes (e.g. network sharing and coinvestments) and exploit governmental subsidies and PPPs. On the other hand, network investments should be accompanied by software-based investments which are essential for controlling and maintaining the network.

5

The role of Neutral host in 5G networks -Potential Risks

In future 5G network deployments, Small Cells [7]- [START_REF] Mahmoud | A comparative study of different deployment modes for femtocell networks[END_REF] are expected to play a significant role in reaching the KPIs as these are specified from 5G-PPP. However, Small Cells deployment usually requires locations with costly backhaul and power facilities, which might hinder the mass deployment of Small Cells. In addition, it should be highlighted that the deployment of "parallel" access networks will be prohibited especially in dense areas. Within future 5G infrastructures, network sharing should evolve beyond the traditional infrastructure sharing models used in previous generations, i.e. site, mast, RAN and core network sharing, towards cloudification, virtualization and holistic end-to-end network slicing. Given this complex landscape, the business model of a neutral operator owning the infrastructure and providing wholesale access to mobile operators seems to be among the most promising ones. For example, a neutral host operator (i.e., the infrastructure owner) could deploy and manage Small Cells while leasing out slices of network capacity to the different 5G service providers on an equal basis to foster competition. However, the advantages of a neutral host operator model are accompanied by several risks related to the possibility of a monopolistic behavior. This contains the risk for abuse of dominant position. In such case, the neutral host operator may refuse to supply the necessary wholesale input, to proceed to excessive pricing as well as to demonstrate discrimination among operators/customers and / or set barriers to new entrants. In order to address these challenges, careful monitoring mechanisms are necessary. These will ensure the public/general interest as well as maximization of the social welfare. NRAs, NCAs, State Aid/PPP are authorities that can potentially undertake the monitoring processes with the first being the most appropriate due to its expertise on remedies such as price cap regulation, retail minus regulation, etc. that require cost accounting models and relative experience.

Conclusions

European telecom companies faced reductions in their revenue associated to variety of factors. 5G networks through a neutral host business model promises to revive opportunities for telecom operators to increase their revenue and new social and economic values for the society.

In this paper, we investigated the necessary changes that should be adopted by network operators to remain competitive by first reporting the status of the telecom market in decreasing revenues fashion. Then we describe the role of telecom networks and actors in the new digital. Followed by several guidelines that will be useful for network operators to adequate their business model by including the neutral host model supported by the H2020 5GCity project. Finally, we described the role of neutral host to address the deployment limitations of 5G networks along with their potential risks.

Fig. 1 .

 1 Fig. 1. Electronic Communications Sector Revenues and Investments (source: Digital Agenda Scoreboard)According to inCITES Consulting connectivity database, Prognosis, both mobile and fixed service revenues were in decline in Western Europe for nearly a decade till 2016 (figure2).

Fig. 2 .

 2 Fig. 2. Year-on-year service revenue trend for fixed and mobile markets in Western Europe (source: inCITES Consulting)

Fig. 3 .

 3 Fig. 3. Average revenue per user in the retail mobile market (source: Digital Agenda Scoreboard)

Fig. 4 .

 4 Fig. 4. Value chain analysis

Acknowledgment

The research leading to these results has been supported by the EU funded H2020 5G-PPP project 5GCity (grant agreement No. 761508).