Manolis Maragoudakis

Euripidis Loukis
email: eloukis@aegean.gr

Yannis Charalabidis
email: yannisx@aegean.gr

A Review of Opinion Mining Methods for Analyzing Citizens' Contributions in Public Policy Debate

Keywords: Electronic Participation (eParticipation), Electronic Consultation (eConsultation), Web 2.0, Opinion Mining, Sentiment Analysis

des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

Electronic Participation (eParticipation) results in the production of large quantities of textual contributions of citizens concerning government policies and decisions under formation. In the "traditional" Web 1.0 based eParticipation (conducted in official government-initiated eParticipation spaces) are offered to the citizens tools for entering not only simple contributions of yes/no or rating type (e.g. various on-line voting and survey tools), but also more sophisticated contributions of textual type as well (e.g. various on-line discussion forum tools) [START_REF]OECD Handbook on Information, Consultation and Public Participation in Policy-Making[END_REF][START_REF]Engaging Citizens Online for Better Policy-making -Policy Brief[END_REF][START_REF]Promise and Problems of e-Democracy: Challenges of Online Citizen Engagement[END_REF][START_REF] Xenakis | An Investigation of the Use of Structured e-Forum for Enhancing e-Participation in Parliaments[END_REF]. Similarly, in the emerging Web 2.0 based forms of eParticipation (exploiting highly popular social media for publishing policy messages and collecting citizens" feedback on them) is offered to the citizens functionality for providing feedback not only of simple "like/dislike" or rating type, but also more of sophisticated comment type as well (e.g. comments on YouTube videos, or postings in blogs) [START_REF] Osimo | Web 2.0 in Government: Why and How?[END_REF][START_REF] Chadwick | Web 2.0: New Challenges for the Study ofE-Democracy in an Era of InformationalExuberance, I/S[END_REF][START_REF] Mergel | The Transformational Effect of Web 2.0 Technologies on Government[END_REF][START_REF] Charalabidis | Towards a Systematic Exploitation of Web 2.0 and Simulation Modeling Tools in Public Policy Process[END_REF]. So hundreds or even thousands of citizens" textual contributions are generated from these eParticipation channels, in numbers much higher than in the off-line public policy debates; these contain valuable opinions on the government policy/decision under discussion, and knowledge on the societal needs or problems this policy/decision attempts to address and the proposed government actions/interventions for this purpose. While the above simple forms of citizens" contribution can be easily analyzed using statistical methods, so that sound conclusions can be drawn from them, this does not hold for the numerous textual contributions collected. The big effort -and therefore the long time and high cost -required for reading hundreds or thousands of citizens" textual contributions on a public policy under formation (in traditional eParticipation spaces, Web 2.0 social media and also in offline public policy debates), summarizing them and extracting the general attitudes of the contributors and main points and issues they raise (e.g. positive and negative aspects and effects of the policies and decisions under discussion, implementation barriers, improvement suggestions) usually leads to a limited exploitation of them. This results in losses of valuable citizens" knowledge and opinions, which would be quite useful to the competent for public organizations for making better and more socially acceptable policies and decisions. Also, this does not allow feedback to be provided to the citizens as to how and to what extent their contributions have been taken into account, resulting finally in "e-consultation fatigue" and disappointment [START_REF]Promise and Problems of e-Democracy: Challenges of Online Citizen Engagement[END_REF]; providing such a feedback would greatly promote government transparency, accountability and openness.

For these reasons ten years ago in the first report of OECD discussing the potential of eParticipation [START_REF]OECD Handbook on Information, Consultation and Public Participation in Policy-Making[END_REF], and also in the subsequent reports on this topic [START_REF]Engaging Citizens Online for Better Policy-making -Policy Brief[END_REF][START_REF]Promise and Problems of e-Democracy: Challenges of Online Citizen Engagement[END_REF], is recognized as one of the most important challenges for the practical application of eParticipation ideas the analysis of the vast amounts of unstructured information that citizens" contributions (e.g. in "threads" of e-conversations) contain using appropriate technologies. These technologies should aim to "support the summarization and content analysis of contributions", "help highlight areas of agreement and disagreement" and "identify the participants main concerns, their level of support for any draft proposals, or their suggestions for action they think necessary to address problems raised". At the same time it is argued that the lack of such analysis of citizens" textual contributions results in a lack of feedback to the citizens on "how the results of the engagement have influenced the decision-making process and changed policy outcome" and has negative impact on citizens" trust in government and eParticipation. Subsequently Rose & Sanford [START_REF] Rose | MappingeParticipation Research: Four Central Challenges[END_REF] from a comprehensive literature review in the area of eParticipation conclude that one of the main research challenges in this area is to use appropriate tools for the analysis of citizens" input to policymaking. Similarly, Macintosh, Coleman & Schneeberger [START_REF] Macintosh | eParticipation: The Research Gaps, ePart 2009[END_REF] in the introductory paper of the ePart 2009 Conference Proceedings discussing the research gaps in the eParticipation area state that there is a lack of effective and efficient technologies for the analysis of unstructured eParticipation data and note that this poses a significant challenge to on-going research.

This paper contributes to addressing this critical for the large scale application and institutionalization of eParticipation research and practice gap, by critically reviewing a wide range of methods and algorithms developed in the area of opinion mining for the analysis of product opinions and reviews posted on the Web. For this purpose we made a systematic search of relevant journal and conference papers using as keywords "opinion mining" and "sentiment analysis". This literature was reviewed focusing on the capabilities that opinion methods provide for meeting the above challenges in the area of eParticipation in a cost-efficient manner (so that we can use them to the largest possible extent), and ii) the main principles of these methods (so that we can select the most suitable ones and use them effectively and efficiently). Based on the results of this review a basic framework has been formulated for the use of opinion mining methods for analyzing citizens" textual contributions in eParticipation. The content of this paper can be very useful for the numerous researchers and practitioners interested in the analysis and exploitation of citizens" textual contributions in various forms of eParticipation.

It should be mentioned that the research presented in this paper has been conducted as part of the project PADGETS ('Policy Gadgets Mashing Underlying Group Knowledge in Web 2.0 Mediawww.padgets.eu) [START_REF] Charalabidis | Towards a Systematic Exploitation of Web 2.0 and Simulation Modeling Tools in Public Policy Process[END_REF], which is supported by the 'ICT for Governance and Policy Modelling' research initiative of the European Commission. The research objective of this project is to develop a methodology and a technological platform for the systematic and centrally managed exploitation of the emerging Web 2.0 social media by government organizations in the processes of policy and decision making. This platform will enable publishing content and deploying micro-applications (termed "Policy Gadgets" -PADGETS) simultaneously to many different Web 2.0 social media, and also retrieving the corresponding users" interactions (e.g. views, likes/dislikes, ratings, comments, etc.) and processing those using advanced methods. Taking into account the high popularity of the targeted Web 2.0 social media it is expected that a large quantity of valuable textual contributions (e.g. blog postings, comments) will be collected from them, so it is of critical importance to make the best possible analysis and exploitation of them in a costefficient manner.

The paper consists of five sections. In the following section 2 the objectives and basic concepts of opinion mining are presented. In section 3 is reviewed the main research stream of this area dealing with sentiment analysis at the document and sentence level. Then in section 4 is reviewed the second most important research stream of this area dealing with feature-based sentiment analysis. Section 5 discusses issues related to our initial framework for dealing with sentiment analysis under project PADGETS, which incorporates linguistic peculiarities such as extensive use of idioms and ill-formed sentences (i.e. short sentences which do not necessarily follow syntactic rules). The final section 6 includes the conclusions and also a framework for the use of opinion mining methods for analyzing citizens" textual contributions in eParticipation.

Opinion Mining Objectives and Concepts

The Web has dramatically altered the way people express their opinions, offering them the capability to post comments and reviews on commercial products and express their views on a plethora of issues in forums, discussion groups, chat rooms, social networking groups and blogs. This user-generated content has been recognized as a valuable source of commercial and political information. However, the large amount of this information and its natural language form make it difficult to extract the useful elements, such as the general feeling/sentiment (e.g. positive, negative, or neutral) on the particular topic (e.g. a product/service or a new policy proposal) and the specific issues raised about it by the users/visitors of these websites. For these reasons methods started being developed for supporting the above tasks, and this lead to the development of the sentiment analysis or opinion mining research domain, which according to [START_REF] Pang | Opinion mining and sentiment analysis[END_REF] aims to develop methods for the computational processing of opinions, sentiments and emotions found, expressed and implied in text. Its initial motivation has been to enable firms to analyze online reviews and comments entered by users of their products in various review sites, blogs, forums, etc., in order to draw general conclusions as to whether users liked the product or not (sentiment analysis), and also more specific conclusions concerning features (characteristics) of the product that have been commented positively or negatively (features extraction and analysis).In general, opinions can be expressed on anything, e.g., an item, a product, a service, an individual, an organization, an event, or a topic, so we use the collective term "object" to denote the target entity that has been commented on; however, comments can be expressed not only on objects, but also on particular "features" (characteristics) of them [START_REF] Liu | Web Data Mining: Exploring Hyperlinks, Contents, and Usage Data[END_REF], e.g. on the "battery life" feature of a mobile phone.

This research domain can be subdivided into two main streams. The first research stream deals with the classification of an opinionated text (i.e. a document consisting of several statements, such as a forum or blog post) as expressing a positive, negative or neutral opinion. More recent research in this research stream focuses on the sentence level, dealing with the classification of a sentence as objective (fact) or subjective (opinion), and then on the further classification of the subjective sentences as expressing a positive, negative or neutral opinion [START_REF] Wiebe | Annotating expressions of opinions and emotions in language[END_REF]. The second research stream of this area deals with the extraction of the most commented features of the commented object, and for each of them, the classification of relevant opinions as positive, negative or neutral is performed. Throughout the following sections 3 and 4 these two research streams are reviewed.

Document and Sentence Level Sentiment Analysis

Sentiment analysis is perhaps the most widely studied topic in this domain [START_REF] Godbole | Large-scale sentiment analysis for news and blogs[END_REF][START_REF] Liu | Opinion observer: Analyzing and comparing opinions on the web[END_REF][START_REF] Aue | Customizing sentiment classifiers to new domains: A case study[END_REF][START_REF] Choi | Joint extraction of entities and relations for opinion recognition[END_REF][START_REF] Das | Yahoo! for Amazon: Sentiment extraction from small talk on the Web[END_REF][START_REF] Dave | Mining the peanut gallery: Opinion extraction and semantic classification of product reviews[END_REF][START_REF] Devitt | Sentiment analysis in financial news: A cohesion based approach[END_REF][START_REF] Gamon | Sentiment classification on customer feedback data: Noisy data, large feature vectors, and the role of linguistic analysis[END_REF][START_REF] Gamon | Pulse: Mining customer opinions from free text[END_REF]. Sentiment classification is similar in some aspects to, but also different in some other aspects from, standard topic-based text classification, which classifies documents into labeled topic classes, e.g., politics, sciences, sports, etc. In topic-based classification, topic related words are important, while in sentiment-based classification it is opinion words denoting a positive or negative attitude (e.g. great, excellent, amazing, horrible, bad, worst, etc.) that are really important. Initially, sentiment-based classification was focused on the document level.

Document-Level Sentiment Analysis

Existing supervised learning methods have been readily applied to document-level sentiment classification. Pang et al. [START_REF] Pang | Thumbs up? Sentiment classification using machine learning techniques[END_REF] followed this approach to classify movie reviews into two classes, positive and negative. It was shown that using unigrams (a bag of individual words) as features in classification performed well with either naïve Bayesian or SVM classifiers. Subsequent papers used a large variety of features and techniques in learning. As most machine learning applications, the main task of sentiment classification is to find a suitable set of classification features. Some of the most widely used features for this purpose are (for a more comprehensive survey see [START_REF] Pang | Opinion mining and sentiment analysis[END_REF]): -Terms" frequencies and positions: these features are individual words or word ngrams and their frequency counts, while in some cases word positions may also be considered.

-Part-of-speech (POS) tags: It has been observed by a plethora of early research studies that adjectives (e.g. "interesting", "bad", etc.) were important indicators of subjectivities and opinions, so they have been treated as special features.

-Opinion words and phrases: By definition, "opinion words" are words commonly used to express positive or negative sentiments, e.g. "beautiful", "wonderful", "good", and "amazing" denote positive opinion words, whereas "bad", "poor", and "terrible" indicate negative opinion words. Although many opinion words are adjectives (as previously shown), adverbs, nouns (e.g. "rubbish", "junk") as well as verbs (e.g. "hate" and "like") can indicate sentiment as well. Apart from individual words, there are also opinion phrases and idioms, such as in the English language "something cost an arm and a leg" that indicates sentiment. -Syntactic dependency: Word dependencies generated from syntactic parsing or dependency trees are also used as features by several researchers. -Negation: Clearly negation words are important because their appearances often change the opinion orientation, e.g. the sentence "I don"t like this camera" is negative.

An interesting conclusion in this research stream is that the "domain adaptation"is very often of critical importance. In particular, it has been found that sentiment classification is highly sensitive to the domain from which the training data (e.g. positive and negative opinion words) have been extracted. A classifier trained using opinionated texts from one domain often performs poorly when it is applied or tested on opinionated texts from another domain. The reason is that words and even language concepts that are used in different domains for expressing opinions can be substantially different. It is possible that the same word in one domain may mean positive, but in another domain may mean negative. For example, as remarked in [START_REF] Turney | Thumbs up or thumbs down? Semantic orientation applied to unsupervised classification of reviews[END_REF], the word "unpredictable" may have a negative orientation in a car review (e.g., "unpredictable brakes"), but it could have a positive orientation in a movie review (e.g., "unpredictable scenario"). Thus, domain expertise is required. Ontologies are very beneficial in this respect since they encode human domain knowledge in a machine readable format. Nevertheless, since ontology construction is a painstaking process, most articles use labeled opinions originated from one domain and apply general opinion words to unlabeled opinions of another domain [START_REF] Aue | Customizing sentiment classifiers to new domains: A case study[END_REF][START_REF] Blitzer | Biographies, Bollywood, boom-boxes and blenders: Domain adaptation for sentiment classification[END_REF][START_REF] Yang | Knowledge transfer and opinion detection in the TREC2006 blog track[END_REF].

Also, some research has been conducted on the use of unsupervised learning techniques for document-level sentiment analysis. In these techniques, weighted terms and phrases are the main indicators for sentiment classification. The use of unsupervised algorithms, such as Mutual Information, which utilize a neighborhood of k surrounding terms around a standard syntactic element (e.g. adjuncts or adverbs) that are likely to express opinions, can give good results [START_REF] Turney | Thumbs up or thumbs down? Semantic orientation applied to unsupervised classification of reviews[END_REF].

Sentence-Level Sentiment Analysis

In the more recent research in this area the level of granularity increases to the level of sentence [START_REF]Wordnet: An Electronic Lexical Database[END_REF][START_REF] Riloff | Feature subsumption for opinion analysis[END_REF][START_REF] Riloff | Learning extraction patterns for subjective expressions[END_REF], focusing on sentence-level subjectivity and sentiment classification. In particular, the main objective is: given a sentence, one has to perform the following two sub-tasks: i) subjectivity classification: i.e. to determine whether it is a subjective sentence (opinion) or an objective one (fact), ii) sentence-level sentiment classification: if the sentence is subjective determine whether it expresses a positive, negative or neutral opinion.

Traditional supervised learning methods are here applicable as well. For instance, one of the early works reported by Wiebe et al. [START_REF] Wiebe | Annotating expressions of opinions and emotions in language[END_REF] performed subjectivity classification using the naïve Bayesian classifier, while subsequent research has used many other learning algorithms [START_REF] Hatzivassiloglou | Effects of adjective orientation and gradability on sentence subjectivity[END_REF][START_REF] Riloff | Learning extraction patterns for subjective expressions[END_REF][START_REF] Nasukawa | Sentiment analysis: Capturing favorability using natural language processing[END_REF][START_REF] Esuli | SentiWordNet: A publicly available lexical resource for opinion mining[END_REF].One of the main bottlenecks of using supervised learning for the above purposes is the manual effort involved in annotating the very large number of training examples (i.e. labeling the available examples as either positive or negative). To reduce the manual effort a "bootstrapping" approach to label training data automatically is reported in [START_REF] Riloff | Learning extraction patterns for subjective expressions[END_REF][START_REF] Riloff | Learning subjective nouns using extraction pattern bootstrapping[END_REF], using a form of re-sampling that aims to estimate the variability of statistical properties of interest. Yu and Hazivassiloglou [34] propose a technique that identifies subjective (opinion) sentences and also determines their opinion orientations. For the initial identification of subjective sentences it uses supervised learning examining three particular learning methods: sentence similarity, naïve Bayesian classification, and multiple naïve Bayesian classifiers. Then for sentiment classification of each identified subjective sentence it uses a similar method to the one used in [START_REF] Turney | Thumbs up or thumbs down? Semantic orientation applied to unsupervised classification of reviews[END_REF], but with several words (instead of two used in [START_REF] Turney | Thumbs up or thumbs down? Semantic orientation applied to unsupervised classification of reviews[END_REF]), and the score function was the wellknown log-likelihood ratio. It should be noted that inn [START_REF] Turney | Thumbs up or thumbs down? Semantic orientation applied to unsupervised classification of reviews[END_REF] the semantic orientation of a phrase is calculated as the Mutual Information between the given phrase and the word "excellent" minus the Mutual Information between the given phrase and the word "poor". The same issue is examined in [START_REF] Kennedy | Sentiment classification of movie reviews using contextual valence shifters[END_REF] considering gradable adjectives. In [START_REF] Gamon | Pulse: Mining customer opinions from free text[END_REF] a semi-supervised learning method is applied, and in [START_REF] Kim | Determining the sentiment of opinions[END_REF] the decision is made by simply summing up opinion words in a sentence. In [START_REF] Kim | Automatic identification of pro and con reasons in online reviews[END_REF][START_REF] Kim | Crystal: Analyzing predictive opinions on the web[END_REF][START_REF] Kim | Automatically assessing review helpfulness[END_REF] statistical prediction models are built to identify some specific types of opinions in reviews. A significant conclusion of this research is that sentence-level classification is not suitable for compound sentences. Wilson et al. [START_REF] Wilson | Just how mad are you? Finding strong and weak opinion clauses[END_REF] pointed out that a single sentence may contain not only multiple opinions, but also both subjective and factual clauses, making the problem even more difficult. In a more recent work [START_REF] Wilson | Recognizing contextual polarity in phrase-level sentiment analysis[END_REF] the problem of sentence-level sentiment analysis is studied using machine learning by considering contextual sentiment influencers such as negation (e.g. not and never) and contrary (e.g. nevertheless and however). A list of such influencers are provided in [START_REF] Polanyi | Contextual lexical valence shifters[END_REF].

Polar Words

For the application of the above sentiment analysis methods it is important to define particular words that bear a positive or negative meaning, which are referred to as "opinion words" or "polar words" in the literature and form the so-called opinion lexicon. Examples of positive polar words in the English language include "beautiful", "good", "amazing", "astonishing", etc., while negative polar words include "poor", "bad", and "awful". Apart from individual words, there are also opinion phrases and idioms indicating positive or negative sentiments, e.g., "the service cost me an arm and a leg" (indicating a negative sentiment of having paid too much). For developing a polar word list three main approaches have been proposed: -The "manual" approach, which is very time-consuming [START_REF] Das | Yahoo! for Amazon: Sentiment extraction from small talk on the Web[END_REF][START_REF] Morinaga | Mining product reputations on the Web[END_REF][START_REF] Tong | An operational system for detecting and tracking opinions in on-line discussion[END_REF][START_REF] Yi | Sentiment analyzer: Extracting sentiments about a given topic using natural language processing techniques[END_REF] and thus it is usually used in combination with automated approaches as a final check, since such automated methods usually make mistakes.

-The "dictionary-based" approach. One of the simpler techniques in this approach is based on "bootstrapping", using a small set of seed opinion words and an online dictionary such as WordNet [START_REF]Wordnet: An Electronic Lexical Database[END_REF]. The strategy is to first collect a small set of opinion words with known orientations (positive and negative) manually, and then to grow this set by searching within the WordNet for their synonyms and antonyms. The newly found words are added to the seed list, and this can lead to a next iteration, etc.; this iterative process stops when no more new opinion words are found. This approach is successfully used in [START_REF] Andreevskaia | Mining WordNet for a fuzzy sentiment: Sentiment tag extraction from WordNet glosses[END_REF][START_REF] Liu | ARSA: A sentiment-aware model for predicting sales performance using blogs[END_REF]. After the iteration process is completed manual labeling can be carried out to correct errors. Researchers have also used additional information (e.g. glosses) in WordNet and additional techniques (e.g., machine learning) to generate better lists [START_REF] Andreevskaia | Mining WordNet for a fuzzy sentiment: Sentiment tag extraction from WordNet glosses[END_REF][START_REF] Esuli | Determining the semantic orientation of terms through gloss analysis[END_REF][START_REF] Esuli | Determining term subjectivity and term orientation for opinion mining[END_REF][START_REF] Esuli | PageRankingWordNetsynsets: An application to opinion mining[END_REF][START_REF] Kamps | Using WordNet to measure semantic orientation of adjectives[END_REF]. So far several opinion word lists have been generated [START_REF] Ding | A holistic lexicon-based approach to opinion mining[END_REF][START_REF] Qiu | Expanding Domain Sentiment Lexicon through Double Propagation[END_REF]. The dictionary-based approach and the opinion words collected from it have a major shortcoming: it is unable to find opinion words with domain specific orientations. For example, for a speakerphone if it is "quiet" this is usually negative; however, for a car if it is "quiet" this is positive. The corpus-based approach described next can help coping with this problem.

-The "Corpus-based" approach and sentiment consistency: The methods following this approach rely on syntactic or co-occurrence patterns and also a seed list of opinion words in order to find other opinion words in a large corpus (= set of documents). A representative work of this approach is the one of Hazivassiloglou and McKeown [START_REF] Hatzivassiloglou | Predicting the semantic orientation of adjectives[END_REF]. The technique starts with a list of seed opinion adjective words, and uses them and a set of linguistic constraints or conventions on connective words to identify additional adjective opinion words and their orientations. For instance, one of the constraints concerns the conjunction (AND), and says that conjoined adjectives usually have the same orientation, e.g. in the sentence "This vehicle is beautiful and spacious" if "beautiful" is known to be positive it can be inferred that "spacious" is also positive. This is so because people usually express the same opinion on both sides of a conjunction. Similar rules or constraints also hold for other connectives, such as OR, BUT, EITHER-OR, and NEITHER-NOR. We call this idea sentiment consistency. Also, clustering can be used to produce two sets of words: positive and negative ones. In [START_REF] Kanayama | Fully automatic lexicon expansion for domainoriented sentiment analysis[END_REF] Kanayama and Nasukawa expanded this approach by introducing the idea of intra-sentential (within a sentence) and inter-sentential (between neighboring sentences) sentiment consistency (called coherency in [START_REF] Kanayama | Fully automatic lexicon expansion for domainoriented sentiment analysis[END_REF]). This approach has a major advantage in comparison with the dictionary-based approach: it can identify domain specific opinion words and their orientations if a corpus from only the specific domain is used.

Feature-based Sentiment Analysis

Classifying opinionated texts at the document or sentence level is useful, but does not provide a complete view on the commented object: a positive opinionated document does not necessarily mean that the author has positive opinions on all aspects or features of the object; likewise, a negative opinionated document does not mean that the author dislikes everything. In a typical opinionated passage, although the general sentiment on the object may be positive or negative, the author can express opinions on both positive and negative aspects of the object. Document-level and sentencelevel classification does not provide such information, so in order to drill down such details we need to go to the object feature level. Therefore feature-based sentiment analysis includes the following two sub-tasks: -identify object features that have been extensively commented on, -determine whether the opinions on each of these features are positive, negative or neutral.

Feature Extraction

Its first task is to specify the topic or the feature of an object commented, which is essential in order to proceed to the second task of classifying its positive or negative meaning. The most widely known approaches that discuss unsupervised learning for identifying explicit features, such as nouns and noun phrases, involve the following two steps ( [START_REF] Yi | Sentiment analyzer: Extracting sentiments about a given topic using natural language processing techniques[END_REF], [START_REF] Esuli | Determining term subjectivity and term orientation for opinion mining[END_REF]): I) Finding frequent nouns and noun phrases: Nouns and noun phrases are straightforwardly identified by using a POS tagger, which is a linguistic software tool for labeling each word with its part-of-speech. Their occurrence frequencies are counted, and only the frequent ones are kept (a frequency threshold can be decided). This approach is based on the assumption that when people comment on object features the vocabulary that they use usually converges, and most object features are nouns. Thus, those nouns that are frequently talked about are usually genuine and important features. II) Finding infrequent features by making use of opinion words: Opinion words are usually adjectives and adverbs that express positive or negative opinions. The basic idea is that the same opinion word can be used to characterize positively or negatively different object features. Opinion words that characterize frequent features can also modify infrequent features, and thus can be used to extract infrequent features.

The precision of first step of the above algorithm was improved by Popescu and Etzioni in [START_REF] Popescu | Extracting product features and opinions from reviews[END_REF]. Their algorithm tries to remove those noun phrases that may not be object features. It evaluates each noun phrase by computing a pointwise mutual information (PMI) score between the phrase and "meronymy" discriminators associated with the object class, e.g., a scanner class. In WordNet, Y is a meronymy of X if Y is a part of X (e.g. wheel is a meronymy of car). The meronymy discriminators for the scanner class are, "of scanner", "scanner has", "scanner comes with", etc., which are used to find components or parts of scanners by searching on the Web. The algorithm also distinguishes components and parts from attributes and properties using WordNet"s is-a hierarchy (which enumerates different kinds of properties) and morphological cues (e.g., "-iness", "-ity" suffixes).

Other related works on feature extraction mainly use the ideas of topic modeling and clustering to capture topics/features in reviews [START_REF] Liu | ARSA: A sentiment-aware model for predicting sales performance using blogs[END_REF][START_REF] Mcdonald | Structured models for fine-to-coarse sentiment analysis[END_REF][START_REF] Su | Hidden Sentiment Association in Chinese Web Opinion Mining[END_REF]. For example, in [START_REF] Mei | Topic sentiment mixture: Modeling facets and opinions in weblogs[END_REF], Mei et al. proposed a probabilistic model called topic-sentiment mixture to capture the mixture of features and sentiments simultaneously. One topic model and two sentiment models were defined based on language models to capture the probabilistic distribution of words in different topics/features with their associated opinion orientations. Su et al. [START_REF] Su | Hidden Sentiment Association in Chinese Web Opinion Mining[END_REF] also proposed a clustering based method with mutual reinforcement to identify implicit features.

After the extraction of object features two additional issues need to be resolved the synonyms issue. It is common that people use different words or phrases to describe the same feature. For example, photo and picture refer to the same feature in digital camera opinions and reviews. Identifying and grouping synonyms is essential for applications. Although WordNet as well as other thesauri and dictionaries help to some extent, they are far from sufficient due to the fact that many synonyms are domain dependent. For example, picture and movie are synonyms in movie reviews, but they are not synonyms in digital camera reviews as picture is more related to photo while movie refers to video. Carenini et al. [START_REF] Carenini | Extracting knowledge from evaluative text[END_REF] proposed a method based on several similarity metrics similar to those in information integration [START_REF] Qiu | Expanding Domain Sentiment Lexicon through Double Propagation[END_REF]. It requires the use of a taxonomy of features to be given for a particular domain. The algorithm merges each discovered feature to a feature node in the taxonomy. The similarity metrics are defined based on string similarity, synonyms and other distances measured using WordNet. Experiments on digital camera and DVD opinions show interesting outcomes.

Identification of Opinion Orientation

An important issue in feature-based sentiment analysis is how to identify the orientation of opinions expressed on an object feature in a sentence. Clearly, the sentence-level sentiment classification methods discussed previously are also applicable here, i.e. they can be applied to each sentence containing object features. However, there is an additional lexicon-based approach to this problem [START_REF] Ding | A holistic lexicon-based approach to opinion mining[END_REF]. This lexicon-based approach basically uses opinion words and phrases in a sentence to determine the orientation of the opinion. Apart from the opinion lexicon, negations and but-clauses in a sentence are also taken into account. In particular, this approach includes the following four steps: A) Identify opinion words and phrases: Given a sentence that contains an object feature initially are identified all opinion words and phrases. Each positive word is assigned the opinion score of +1, each negative word is assigned the opinion score of -1, and each context dependent word (i.e. word having meaning positive or negative depending on the context) is assigned the opinion score of 0. Suppose for example that we are given the sentence "The picture quality of this camera is not great, but the battery life is long." After this step, the sentence is turned into "The picture quality of this camera is not great [+1], but the battery life is long [0]" because "great" is a positive opinion word and "long" is context dependent. B) Handling negations: Negation words and phrases are used to revise the opinion scores obtained in the previous step based on some negation handling rules. After this step, the above sentence is turned into "The picture quality of this camera is not great [-1], but the battery life is long [0]" due to the negation word "not". C) But-clauses: In English, the word "but" means contrary, so a sentence containing but is handled by applying the following rule: the opinion orientation before but and after but are opposite to each other. After this step, the above sentence is turned into "The picture quality of this camera is not great [-1], but the battery life is long [+1]" due to "but". Apart from but, phrases such as "with the exception of", "except that", and "except for" behave similarly to "but" and are handled in the same way. D) Aggregating opinions: This final step applies an opinion aggregation function to the resulting opinion scores to determine the final orientation of the opinion on each object feature in the sentence.

Ontology-based Sentiment Analysis

One of the most recent developments in feature-based sentiment analysis is the use of domain-specific knowledge through ontologies, which constitute a well-known formalism for representing knowledge in a both human and machine comprehensible manner. Using ontologies for web classification and document extraction has been a successful technique that inspired researchers from the sentiment analysis domain to incorporate such knowledge in order to define a taxonomy (or hierarchy) of object features and then build sentiment analysis subsystems that apply rules denoted by the structures of such taxonomies. Most research and commercial systems that incorporate ontologies [START_REF]I-sieve Technologies, Sentiment Analysis Beyond Impressions[END_REF][START_REF] Wei | Sentiment Learning on Product Reviews via Sentiment Ontology Tree[END_REF] consist mainly of two main modules: an ontologybased extraction module and a sentiment analysis module. The ontology-based extraction module creates a small hierarchical tree using a set of relevant texts from which terminology is extracted. In the analysis module this small tree is compared against the large core ontology tree for analyzing the sentiment labels of its nodes.

Sentiment Analysis in project PADGETS

From the aforementioned review on existing techniques on sentiment analysis in every granularity level, ranging from document-based to feature-based opinion mining, we could state that given a language with a plethora of tools and opinionated text in a well-structured format, the performance of sentiment classification could reach very high percentages in the range of 80%-90%. Furthermore, a significant portion of those techniques have been applied and evaluated in domains where users state their opinions in a formal language and using large textual inputs such as electronic commerce, media presentations (i.e. movies or music albums), etc. Nevertheless, as regards to PADGETS, the application languages would be Greek, Slovenian and Italian, which lack of abundance in linguistic resources. Additionally, the domain of social media is not similar to the aforementioned cases, since users tend to utilize a smaller vocabulary with linguistic idioms or utilize the grammatical phenomenon of ellipsis in the texts. This phenomenon resulted in a novel approach that incorporates Social Network Analysis theory in order to deal with connecting opinions. To our knowledge, this is the first time such an approach is being exploited for sentiment classification. The idea is borrowed from the social network analysis (SNA) domain, which considers humans participating in social networking activities as actors and studies their relationships (see the PageRank algorithm of Google® for further details). SNA views the above process as a graph, where actors are nodes and edges state a kind of relations between two actors (e.g. friends in Facebook, Followers in Twitter, etc.). Opinions in social media may be suffering of short texts, encompassing linguistic idioms that are more difficult to be analyzed using traditional tools. However, if we consider opinions as actors and their inter-reference as links, we could form a common basis between opinion mining and SNA. The achievement of such a correspondence lies to the utilization of the prestige attribute, commonly found in SNA. Prestige is a refined measure of prominence of an actor. An actor is considered prestigious if he/she is the object of many references by other actors. Similarly, an opinion is significant (either positive or negative) if it is being addressed by other opinions. Furthermore, an opinion which is referenced by prestigious opinion is more significant than ones which are not referenced by opinions of great prestige. An analogue to real life is the case where a person who is recommended by a CEO of a company is more prestigious than a person who is recommended by the clerk of the company. By connecting opinions (where applicable) and grading their rank prestige accordingly, we could identify which sub-graph contains nodes of positive of negative sentiment and proceed with the classification of all other nodes. A reference can of course be positive (agreement) or negative (disagreement).

Conclusions

Both the traditional and the emerging eParticipation forms result in the production of large quantities of citizens" textual contributions concerning policies and decisions under formation, in numbers much higher than in the off-line public policy debates. It is of critical importance to use appropriate technologies for analyzing them in a costefficient manner, in order to extract the valuable opinions and knowledge they contain, and then integrate them in the policy/decision making processes and provide feedback to the citizens. In this paper we review methods developed in the area of opinion mining as to the capabilities they provide for meeting the above challenges. From this review it has been concluded that a useful body of knowledge has been developed in this area consisting of methods for addressing mainly the following three problems: -classification of an opinionated text as expressing as a whole a positive, negative or neutral opinion (document-level sentiment analysis), -classification of each sentence of such a text as objective (a fact) or subjective (opinion), and then focus on the latter and classification of each of them as expressing a positive, negative or neutral opinion (sentence-level sentiment analysis), -extraction of the particular features/subtopics commented by the authors of these texts, and for each of them identification of the orientation of the opinions expressed about it as positive, negative or neutral (feature-level sentiment analysis).

Based on the conclusions of this review a basic framework for the use of opinion mining methods in eParticipation can be formulated consisting of five stages: I. Classify each particular posting on the policy/decision under discussion as positive, neutral or negative, using methods of document-level sentiment analysis (section 3.1), and then calculate relative frequencies of positive, neutral and negative postings. II. For each posting identify its subjective sentences (expressing opinions) and classify each of them as positive, neutral or negative using methods of sentence-level sentiment analysis (section 3.2), and then calculate relative frequencies of positive, neutral and negative subjective sentences. III. Compare and integrate findings of the above steps I and II, and also findings from the analysis of other types of citizens" non-textual feedback (e.g. numbers of users who voted in favor or against the policy/decision under discussion in an e-vote tool, or rated it positively or negatively in an e-survey tool, or even liked or disliked a relevant content we published in social media). This will allow conclusions to be drawn as to the citizens" general sentiments/feelings (positive or negative) on this government policy/decision. IV. By further processing all postings on this policy/decision using feature extraction methods (section 4.1) identify the main issues raised and commented by citizens. V. Finally for each issue classify each of the postings" sentences containing it as positive, neutral or negative using methods of sentence-level sentiment analysisopinion orientation (section 4.2), and then calculate relative frequencies of positive, neutral and negative subjective sentences. This will allow the identification of the main issues raised by the citizens and the particular sentiments/feelings on them (e.g. positive and negative aspects and effects of the policies/decisions under discussion, implementation barriers, improvement suggestions, etc.).

It should be noted that for the practical application of the above opinion mining methods it is of critical importance to have sufficient language resources, such as lexicons, POS taggers, name entity recognizers, and tokenizers. The availability of these resources varies among languages. Further research is in progress by the authors for the application of the above framework for analyzing citizens" comments on policy messages posted by government organizations in several social media, as part of the PADGETS project mentioned in the introductory section.


Efthimios Bothos

Dimitris Apostolou

Gregoris Mentzas
email: gmentzas@mail.ntua.gr

Citizen Engagement with Information Aggregation Markets

Keywords: Information aggregation markets, public participation, citizen engagement

Participatory governance entails converting citizens from passive users of administrative decisions into active participants of political procedures. Public institutions and organizations can benefit from methods and tools able to aggregate and interpret information dispersed among citizens. In this paper we explore the use of Information Aggregation Markets (IAMs) for citizen engagement. We identify the benefits of IAMs and explain how markets can be used to aggregate citizens' opinions and views on policy issues. Furthermore we report on two real life cases where we deployed IAMs to assist the decision making processes of public organizations. The positive feedback from participants and decision makers provides empirical evidence on the benefits of IAMs as a method for engaging citizens in public policy decision making.

Introduction

In social systems, information, beliefs and opinions are heterogeneous and dispersed among individuals. Public institutions and organizations can gain a considerable advantage by exploring methods and tools which are able to aggregate and interpret dispersed information. For instance, having precise knowledge of citizens' opinions and viewpoints can support public bodies in effective administration, while at the same time creating an enduring bond between citizens and administrators, based on active participation, trust and transparency.

Participatory governance entails converting citizens from passive users of administrative decisions into active participants of political procedures. In the recent few years there has been an observed trend toward increased involvement of the public in the affairs and decisions of policy-setting bodies [START_REF] Rowe | A Typology of Public Engagement Mechanisms[END_REF]. To this end, a number of Internet technologies which allow users to interact, collaborate as well as express their beliefs and opinions can be utilized. These technologies, termed as Web 2.0 [START_REF] O'reilly | What is Web 2.0: Design Patterns and Business Models for the Next Generation of Software[END_REF], provide wider applicability compared to traditional approaches since they can be used to address issues at different political levels (small, medium or large cities, regions or nations) and can significantly reduce the cost and time of monitoring and aggregating citizens' opinions and preferences.

In the "Web 2.0 era" of services and applications citizens collectively contribute to a Web presence and generate massive content behind their virtual collaboration [START_REF] Lin | Building Web 2.0[END_REF]. An exemplar of this new paradigm is "collective intelligence". Kapetanios [START_REF] Kapetanios | Quo Vadis computer science: From Turing to personal computer, personal content and collective intelligence[END_REF] introduces this paradigm as "human-computer systems in which machines enable the collection and harvesting of large amounts of human-generated knowledge, while enabling emergent knowledge, i.e. computation and inference over the collected information, leading to answers, discoveries, or other results that are not found in the human contributions". Typical examples of collective intelligence include collaborative filtering, folksonomies and Information Aggregation Markets (IAMs). IAMs are speculative markets, the purpose of which is to collect and aggregate information in the price of contracts representing different outcomes of future events [START_REF] Spann | Internet-Based Virtual Stock Markets for Business Forecasting[END_REF]. Contract prices hinge on the probability of an event occurring or not. Individuals influence these prices by buying and selling contract shares based on their prediction of the outcome. IAMs are characterized by their relatively easy deployment and high potential in providing significant value to organizations by leveraging contributions from a broad community of contributors and aggregating information about issues that would otherwise be difficult to capture.

In this paper we propose IAMs as a method which can be used by public institutions and organizations for engaging citizens in public decision making. In section 2 we discuss the various levels of citizen engagement in the Web 2.0 era and explain what citizens contribute and why. Section 3 introduces IAMs whereas section 4 analyses how such markets can be used to support citizen participation. In Section 5 we present the results of two real life cases where we utilized IAMs to support the decision making processes of public organizations by allowing citizens to evaluate policy options as well as to propose new ones. The paper concludes in Section 6 with a summary of our observations and suggestions for further research.

E-Participation and Citizen Engagement

According to OECD [START_REF]Citizens as Partners: Information, consultation and public participation in policy-making[END_REF] democratic political participation must offer the means to be informed, the mechanisms to allow citizens to take part in the decision-making and the ability to contribute and influence the policy agenda. OECD defines three levels of interaction according to the nature and direction of the relationship between public bodies and citizens: a) Information, which is a one-way relationship in which the government produces and delivers information for use by the citizens, b) Consultation, which refers to a two-way relationship with the government in which citizens provide their feedback and c) Active participation, referring to a partnership in which citizens actively engage in defining the process and content of policy making.

Based on the aforementioned scale of citizens' engagement Macintosh [START_REF] Macintosh | Characterizing E-Participation in Policy-Making[END_REF] provides three levels of engagement that characterize e-Participation initiatives:

• e-Enabling citizens is concerned with how technology can be used to reach the wider audience by providing a range of methods to cater for the diverse technical and communicative skills of citizens. • e-Engaging with citizens addresses the issue of consulting a wider audience to enable deeper contributions and support deliberative debate on policy issues. • e-Empowering citizens is concerned with supporting active participation and facilitating citizens' ideas to influence the political agenda.

The role of citizens in public decision making becomes increasingly important in consultation (e-Engaging) and active participation (e-Empowering) initiatives as they can contribute in a variety of different ways, including exploring, validating and reconciling ideas [START_REF] Peters | E-consultation: Enabling democracy between elections[END_REF]. With validating and evaluating ideas the public can support the decision-maker to check ideas with the on-the-ground perspective. With exploring ideas, the focus is on divergent input and conversations, whereby the public can bring ideas, suggestions, information and perspectives into the decision-making process. Another contribution that the public can provide relates to reconciliation of diverse ideas. These are more convergent discussions that emphasize trade-offs and the weighing of values. The aforementioned concepts are not mutually exclusive as one public involvement initiative might be designed to explore an issue and validate (options, while another might encompass all three types of input).

To interact with the public, one must somehow communicate with it. Recent innovations in the technologies of communication have affected the feasibility of various methods for public participation, leading to a renewed interest for engaging the public in decision making with the emergence of information technology applications for e-Participation. This is evident from the number of implementations around the globe such as Estonia's TOM portal (http://tom.riik.ee/) and Singapore's Government Consultation Portal (http://www.ura.gov.sg/econsult/index.htm). The impetus to implement e-Participation can also be attributed to the growing awareness of the need to attain more democratic governance [START_REF] Coleman | Bowling together: Online public engagement in policy deliberation[END_REF]. OECD [START_REF]Citizens as Partners: Information, consultation and public participation in policy-making[END_REF] has indicated that democratic governments are under pressure to adopt a new approach to policymaking, placing greater emphasis on citizen involvement both upstream and downstream to decision-making.

Regarding the question of why citizens participate in policy decision making, the review paper of Phang and Kankanhalli [START_REF] Phang | A Research Framework for Citizen Participation via eConsultation[END_REF] lists five major theories: (1) the Socioeconomic Model of Participation, (2) the Rational Model of Participation, (3) the Civic Voluntarism Model, (4) the General Incentives Model and (5) the Social Capital Theory of Participation. The socio-economic model attempts to explain citizen participation in terms of the social circumstances of individuals, such as age, education level and financial status, which shape their attitude towards participating [START_REF] Parry | Political participation and democracy in Britain[END_REF]. Individuals who are older, better educated and wealthier are more likely to participate than those who are not. The rational choice model of participation views citizen participation as a rational activity that serves to promote or defend the goals of participants with the maximum of benefits and the minimum of costs [START_REF] Green | Pathologies of rational choice theory[END_REF]. Citizens obtain benefits such as the ability to influence policy outcomes in ways that is to their advantage, while costs include effort and financial resources that one needs to incur to participate.

The civic voluntarism model [START_REF] Verba | Voice and equality: Civic voluntarism in American politics[END_REF] explains participation by addressing the question of why people do not participate, and suggests three answers: because they can't due to lack of resources (e.g. money, time, and civic skills); they don't want due to lack motivation; and because nobody asked them to. The general incentives model [START_REF] Seyd | Labor's grass roots: The politics of party membership[END_REF] explains citizen participation by synthesizing social factors (e.g. norms) and individual factors (e.g. perceived costs and benefits). The social capital theory of participation attempts to explain citizen participation from a social network perspective. The main premise of the theory is that a community with stronger bonds between its members has a distinct advantage over a community with poor bonds [START_REF] Pattie | Citizenship and civic engagement: attitudes and behavior[END_REF].

Besides the aforementioned theories, IT features that affect citizens' participation are anonymity, simultaneity capability, connectivity and communality [START_REF] Phang | A Research Framework for Citizen Participation via eConsultation[END_REF]. Research on group support systems has found a positive impact of anonymity on group performance (e.g. [START_REF] Desanctis | A foundation for the study of group decision support systems[END_REF]) as anonymity may reduce unfavourable evaluation apprehension effects due to the social status of certain members. Evaluation apprehension is unwanted as it tends to inhibit participation in traditional face-to-face meetings [START_REF] Mcleod | Persuasive linguistic devices in computersupported group discussion[END_REF] and by avoiding its occurrence, GSS are found to increase participation level of a group (see e.g. [START_REF] Connolly | Effects of Anonymity and Evaluative Tone on Idea Generation in Computer-Mediated Groups[END_REF]). The simultaneity of IT is found to increase participation by overcoming production blocking that inhibits content generation process, as there is no need for one to wait for one's turn to express one's ideas [START_REF] Shepherd | Invoking social comparison to improve electronic brainstorming: Beyond anonymity[END_REF]. It can also reduce the cognitive load and distraction of members in trying to remember their ideas while waiting for their turn.

Connectivity refers to the ability that enables individuals to directly communicate with each other, whereas communality refers to the availability of a commonly accessible pool of information to all ( [START_REF] Fulk | Connective and communal public goods in interactive communication systems[END_REF], [START_REF] Monge | Production of collective action in alliance-based interorganizational communication and information systems[END_REF]), ease individuals' participation in information sharing and exchanges. For example, Monge et al. [START_REF] Monge | Production of collective action in alliance-based interorganizational communication and information systems[END_REF] propose that increased provision of connectivity and communality would lead to an increased amount of information generated in the context of inter-organizational information systems.

Interventions on the non IT related factors that affect citizen participation cannot be easily controlled as they depend on characteristics of the society that require time in order to be enhanced such as educational status, financial status. On the other hand introducing methods and tools that encompass the IT features that positively influence citizen participation, such as Information Aggregation Markets, can be beneficial for public organization on shorter time-frames.

Information Aggregation Markets

Information Aggregation Markets (IAMs) are markets designed and run for the primary purpose of mining and aggregating information scattered among participants and subsequently using this information in the form of market values in order to make predictions about specific future events. IAMs are commonly known as 'prediction markets' because they are often used to predict future events. They are essentially "futures markets", i.e. forums for exchanging contracts whose payoffs are tied to unknown future events. The contracts in IAMs can be considered as a subset of the financial derivative called "future" and differ from those in traditional equity markets in that they are not typically tied to a claim of an ownership stake in a firm. Instead, the final price of these futures contracts depends on the outcome of a future event; upon market end, contracts' price incorporates the available information with respect to that event.

IAMs offer substantial benefits, including real time information aggregation as participants are incentivized immediately to submit new information, extraction of realistic expectations as traders 'put their money where their mouth is', high accuracy in predicting future events, as shown by previous papers cited later in section and increased scalability, due to their resemblance to financial markets.

In recent years, a significant increase has been documented in both the volume of academic literature on the subject [START_REF] Tziralis | Prediction Markets: An Extended Literature Review[END_REF], and enterprises putting the concept into actual use for information aggregation and decision support. In the corporate world, a number of companies including Hewlett Packard, Microsoft, Google, Siemens and Eli Lilly have experimented with IAMs to forecast sales, the success of new products or even estimate projects' completion time [START_REF] Cherry | Bet on it! Can a stock market of ideas help companies predict the future?[END_REF]. A number of information aggregation markets solutions for corporate applications are offered by specialized companies such as Inkling, Consensus Point in the US and Nosco and Pre:Kons in Europe whereas many sites on the Web (e.g. Intrade.com, Betfair.com, Nadex.com) offer information contracts in a number of areas including sports, politics, finance, law, entertainment, and even the weather. In addition, a recent study published at the European e-Participation portal suggests the use of information aggregation markets as a tool, which can leverage citizens' participation in European public policy [START_REF] Millard | European eParticipation Summary Report[END_REF].

Recent research has explored the use of IAMs for preferences' aggregation. In 'preference markets' participants engage in securities trading, the price of which represents the degree of preference for a decision option. Participants reveal their own preferences and their expectations of others' preferences, and converge towards an equilibrium that captures the consensus view [START_REF] Dahan | Preference markets: Organizing securities markets for opinions surveys with infinite scalability[END_REF]. This type of IAMs has been mainly applied in corporate settings for the selection of new ideas (see for example [START_REF] Lacomb | The imagination market[END_REF], [START_REF] Bothos | Collective intelligence for idea management with Internet-based information aggregation markets[END_REF]).

The public sector seems quite reluctant to introduce IAMs to improve public decision making, despite the benefits expected from the design of information markets for governance. In particular, Hanson [START_REF] Hanson | Decision Markets for Policy Advice[END_REF] proposes IAMs as a new tool that 'will revolutionize governance' and Ledyard [START_REF] Ledyard | Design of information markets for policy analysis[END_REF] and Hahn and Tetlock [START_REF] Hahn | Using information markets to improve public decision making[END_REF] describe a framework and identify the main characteristics an IAM should fulfil in order to perform well in situations relevant for policy decision making. In 2003, there was a controversial proposal to deploy information markets in order to predict terrorist events in the US and, based on the market outcome, to design appropriate policies relevant to national security. The so called Policy Analysis Market (PAM) was cancelled because the officials objected to the fact that terrorists could possibly affect the outcome while the government did not wish to disclose the kind of sensitive intelligence that an information market would reveal.

The characteristics of IAMs are in line with IT features that affect participation. With IAMs, the private information and preferences of citizens are reflected in the price of contracts that represent policy options. Citizens buy contracts of policy options they prefer and sell those they do not approve. The target is citizens' motivation and their participation in the decision making process. As an example, suppose that a new policy addressing the problem of excessive CO2 emissions by automobiles is proposed. The IAMs e-Engaging approach would be to model the consequences of either adopting or not the proposed policy, by creating different contracts which reflect e.g. the impact on the percentage of CO2 emissions after 5 years.

IAMs can be configured to allow citizens propose alternative policy options raising the level of participation to e-Empowering. The new options are traded in the market together with the ones introduced by the policy makers. As a result the public, through the market, can indicate policies that experts have not considered yet. Identify the preferred and most promising policy options How?

Participants express their preferences by trading on decision options -they can suggest alternatives Citizens predict the impact of alternative policies or express their preferences through trading Why?

-Aggregate stakeholders' preferences -Identify new decision options -Prediction of policies' impact -Aggregation of citizens' preferences When applying IAMs for citizen engagement, one should specifically consider liquidity, participants' incentives and the synthesis of participants groups. Liquidity, i.e. a significant number of transactions in the market, is needed to generate a reasonable price signal on the underlying value of a contract since transaction prices may not be representative of market participants' beliefs in markets with low number of transactions. In other terms, the outcome of an IAM can be used by decision makers only if prices do not provide biased measures of traders' beliefs. Nonetheless, theoretical analysis has shown that the practice to interpret IAM prices as probabilities that aggregate the information held by traders should be applied cautiously regardless of the liquidity problem. Manski [START_REF] Manski | Interpreting the Predictions of Prediction Markets[END_REF] argues that in an all-ornothing IAM (i.e. markets populated with contracts that pay a fixed amount if a specified event occurs and nothing otherwise) populated with risk-neutral traders endowed with heterogeneous beliefs, the mean belief and the equilibrium price may differ substantially. Gjerstad [START_REF] Gjerstad | Risk Aversion, Beliefs, and Prediction Market Equilibrium[END_REF] and Wolfers and Zitzewitz [START_REF] Wolfers | Prediction markets[END_REF], however, show that the bias diminishes if traders have risk-averse utility function, and that the bias disappears if the coefficient of relative risk aversion is equal to one. These findings demonstrate that interpretation of prices in actual IAMs requires some knowledge of trader's risk preferences which is not always feasible.

With respect to incentives in IAMs, play money can be endowed to participants in order to avoid excess technical, regulatory, and fiduciary costs and redeem best performing participants with prizes as incentives. Past research showed that even play-money IAMs can be a dominant source of information ( [START_REF] Servan-Schreiber | Prediction Markets: Does Money Matter?[END_REF]).

Furthermore, in order for IAMs efficiently to aggregate information, they should attract a sound group of participants. Surowiecki [START_REF] Surowiecki | The Wisdom of Crowds[END_REF] has provided a qualitative analysis of participant characteristics necessary for the market to be trustworthy: diversity of opinion, independence of thought and decentralization of knowledge. Wolfers and Zitchevitz [START_REF] Wolfers | Prediction markets[END_REF] established a theoretical model and provided an account of sufficient conditions under which IAM prices aggregate private information held amongst participants. They concluded that, when participants are typically wellinformed, IAM prices will aggregate information into useful information. In the following sections we present two real-life use-cases of IAMs designed to assist public organizations in decision making.

Real life cases of Information Aggregation Markets for Citizen Engagement

The purpose of our pilots was to deploy IAMs in real settings in order to support the decision making processes of public institutions while allowing citizens to contribute by expressing their opinion. The first case was designed and executed at the State of Bremen in Germany in cooperation with the local ministry of education where citizens were asked to express their preferences on a set of policy options relevant to restructuring the bible history class at schools. In the second case we cooperated with the European Commission in the context of a public consultation held in order to acquire and evaluate future research directions regarding the use of information and communication technologies for enabling energy efficiency.

In both cases, the software IAMs were deployed on was IDeM [START_REF] Lacomb | The imagination market[END_REF]. The system can be utilized for aggregating participants' preferences for alternative policy options in market prices (relevant for e-Enabling initiatives) and with proper configuration it can allow traders to introduce new contracts in the market representing new policy options (relevant for e-Engaging initiatives).

To ensure adequate market liquidity even with low number of traders, IDeM implements a trading algorithm called continuous double auction with market maker (CDAwMM). When there are no matching offers the system acts as an 'always there' buyer and seller ready to accept buy and sell offers at a certain price. A logarithmic price function is used to determine the transaction cost [START_REF] Hanson | Logarithmic market scoring rules for modular combinatorial information aggregation[END_REF].

Participants traded using play money; an initial amount 10.000 play money units was endowed to each trader upon registration together with an initial bundle of 50 contracts per policy option available in the market. Contracts could be traded at a price range of 0 to 100 since the price indicated the potential of success of the related policy option and were initially valued at 50 play money units in order to ensure a fair chance for all options. The duration of each market was fixed and announced at the home page of IDeM. A detailed tutorial explaining the basic principles of IAMs as well IDeM functionalities was included in the 'help section' of the software.

Reforming the Bible History class at the State of Bremen

In cooperation with the ministry of education of the German State of Bremen, we designed a real-life use of IAMs with the goal to evaluate policy options for refactoring the 'Bible history' class in Bremen schools. The officials at the State of Bremen wanted to modernize the class in order to address the needs of a multireligion society while involving citizens in the decision making process. The initiative was carried out in two phases. Initially a web-based forum was setup where the citizens of Bremen could submit ideas and views on the issue while engaging in an online discussion. The purpose of the forum was to gather citizens' views and remained open from 9/3/2009 to 20/3/2009. Next, the submitted views and comments were processed by the officials and a set of seven alternative options were derived.

We created a web-based IAM, registered the seven policy options together with their descriptions (as an example one of the policy options was: Title 'Separate classes for each religion', Description 'BGU is replaced by separate classes of Christian, Jewish and Islamic religious education, as well as philosophy / ethics. Skilled teachers with deep knowledge of the specific religion are responsible for each class') and posted a call for participation on the web-based forum. The multilingual capabilities of IDeM allowed us to use the German language in this case. Participants could not propose new policy options in the market and expressed their opinion by investing on the existing ones. The market remained active for 14 days, between 20/3/2009 and 3/4/2009. 42 users registered in our IAM of which 30 participated actively. The total number of transactions was 302; hence the average number of transactions per participant was 7.1. The maximum number of transactions per contract was 60 and the minimum 28.

The officials agreed that participation was satisfying. Two different questionnaires were prepared, one for the administrators and one for the traders. The former was completed by 3 administrators and the latter by 7 traders. Market administrators were satisfied by user participation and the results of the market. As quoted by one of the decision makers of the pilot in Bremen "Using Information Markets we experienced increased participation. Although we didn't take any specific measures to promote participation, people simply responded promptly".

Furthermore the ranking provided by the market, which was calculated using the weighted average price of the transactions, proved particularly useful and the officials at the ministry of education informed us that they were going to consider it when reaching their final decision.

Traders stated that they participated for the first time in an IAM. All but one agreed that they would be willing to take part in a similar market in the future. The trader who objected expressed the opinion that the capitalistic characteristics of the IAMs could lead to manipulation effects and speculative bubbles like those we are experience in the present economic crisis. Furthermore all traders expressed their concerns whether the outcome of the market would be considered by the officials as they felt that the decisions were directed by the central government and their opinion would not matter the most.

European Commission Consultation on Information and Communication Technologies for Enabling Energy Efficiency

The case was designed for a European Commission (EC) public consultation on Information and Communication Technologies for enabling energy efficiency. This endeavour was part of the interactive policy making initiative of the EC. The purpose of deploying IAMs in this context was to allow people from all over Europe to submit and evaluate future research directions relevant to the use of ICT for enabling energy efficiency, allowing them to express their opinions by proposing and trading idea contracts. Ideas placed in the market had the benefit of being scrutinized by a wide range of peers. In the invitation that was sent, participants were encouraged to especially consider and focus on the potential effects of user-driven open innovation in the area of Structural Change, i.e. ICT-enabled structural changes for a low-carbon society. The reason was that that enabling structural changes in Business/Work/Mobility models across the economy and society is the most challenging, least clear and yet potentially greatest area of opportunity.

An initial number of three ideas were used for the commencement of the market. Traders were able to enter new ideas in the 1st week of the market operation. We asked participants to describe their ideas as clearly as possible but shortly. Once the appropriate information had been entered, the ideas automatically were inserted in the market. We agreed with the stakeholders that no more than 18 ideas should enter the market, so a 'first come first served' approach was enforced. The invitation was sent to approximately 2000 people from across Europe and 63 registered. Non-monetary and monetary incentives were provided to the idea creators and the market winner.

For the idea creators the incentive was that all proposed ideas were to be communicated to the EC officials thus increasing the likelihood of potential funding of the winning ideas. The market winner received a complementary registration for the international conference in a relevant area. The market operation started on 4/7/2008, remained open for approximately 3 weeks until 21/7/2008 and participants were able to trade 24/7. Traders were asked to perform as many transactions as possible, on the basis of what they thought the "winning ideas" could be and at the same time would maximize the valuation of their portfolio. As a minimum, traders were encouraged to visit the market once every day and revise their position accordingly. During the trading period 561 transactions occurred by 34 active members of the market (active members were considered the users that made more than 1 transaction in the market). We managed to attract a diverse group of participants from across Europe (10 difference European countries), with fairly wide age range (28 to over 57 years old), experience (1 to over 21 years of working experience) and professional background (including Engineering, Management and Marketing).

Upon market end participants were asked to fill an online questionnaire; we received a total of 14 completed questionnaires. An overwhelming majority of the respondents (85%) indicated that they use information market as for the first time. 78% of them also stated that they would participate again in a similar market if they were requested. This percentage is quite important and shows the success the market had. It is noteworthy to mention that before participating in the market almost 72% of them did not believe that the concept of IM would be useful. However, after participating in a 78% of them support that they, now, believe that IAMs is a means to successfully evaluate policy options and also propose new options.

Furthermore semi-structured interviews were conducted with decision makers in order to gather feedback on the usefulness and the benefits of our approach and a positive feedback was received. An EU official stated "This endeavour proved very interesting and useful. We received many interesting, diverse and innovative ideas Overall, we were very satisfied with the quality of the contributions, although certain ideas were pretty 'wild' and cannot not be readily utilized". With respect to the submitted ideas, we reached our target to receive 18 ideas by participants fairly easily and all were submitted during the first two weeks of market operation.

Conclusions

In this paper we proposed Information Aggregation Markets as a promising method for citizen engagement in public decision making. We analysed the benefits of such markets for e-Engaging and e-Empowering initiatives and explained how markets can be designed and deployed by public institutions and organizations in order to allow citizens' involvement in the processes of selection and evaluation of new policies. Moreover we reported on two real life cases in which we applied IAMs in cooperation with public bodies. In both cases decision makers where pleased by the market output and appreciated the fact that they gained insight into participants' opinions. Furthermore they stated that they would consider the results before reaching a final decision. Users participated actively in the markets while admitting that IAMs can provide the means to successfully evaluate policy options and also propose new ones.

IAMs provide a sound solution to the incentive problems and can outperform alternative approaches to informing public administrations' decision-making. In addition they are scalable and can support an arbitrary number of participants whereas the output, i.e. the price signal, is simple enough to be directly taken into consideration. Nonetheless markets raise certain practical challenges; they are not suitable for all settings, and need to be designed and implemented carefully and sensitively to be effective. Markets may leak sensitive information in a way that other mechanisms do not (or do less), and this in turn can impact morale and motivation negatively, create legal complications by turning participants into "insiders". Markets may be subject to manipulation and it can be challenging to sustain participation.

However the practical performance of markets should not be compared to some absolute ideal, rather to the benefits or limitations of other alternatives [START_REF] Cherry | Bet on it! Can a stock market of ideas help companies predict the future?[END_REF]. Holding meetings, relying on 'expert' opinion, conducting surveys, or polling citizens are all information mechanisms with costs and potential weaknesses. Often these methods are not sufficiently appreciated and challenged. Our research provides evidence that IAMs can constitute a tool for public organizations in order to engage citizens in the decision making process and tap into citizens' knowledge and private information.

Future research should focus on more systematic comparison of IAMs with alternative mechanisms in real world cases. For example, Graefe [START_REF] Graefe | Group Decision Making -Meetings, Nominal Groups, Delphi, and Prediction Markets Compared[END_REF] compares the Delphi method of structured group deliberation with IAMs in a field experiment and concludes that markets perform as well as the Delphi method. On a more practical level, future integration of IAMs into well known social networks such as twitter and facebook could provide a much larger user base while overtaking the need for registration to new platform, not familiar to most citizens.

Table 1

 1 summarizes who participates, what is being traded, how participants are involved and what the objective of IAMs in e-Empowering and e-Engaging is.

Table 1 .

 1 Information Aggregation Markets for e-Participation.

	e-Empowering	e-Engaging
	Who?	Citizens
	Why?