Gwanghoon Yoo

Jeesun Nam

A Hybrid Approach to Sentiment Analysis Enhanced by Sentiment Lexicons and Polarity Shifting Devices

Keywords: Sentiment Analysis, Sentiment Lexicon, Polarity-Shifting Device, Hybrid Approach

This paper presents a hybrid approach to sentiment classification method for Korean texts. It is based on a cascading system by which lexicon-based classification first conducts the sentiment detection along with the local parsing of sentiment constituents, and a supervised machine learning algorithm sorts the texts out of the lexicon. We use a fine-grained Korean machine-readable dictionary for the lexiconbased classification, dealing with Polarity Shifting Devices (PSDs) which are divided into Intensifier, Switcher, Activator, and Nullifier. By structuring PSDs and polarity values of opinion texts, it is possible to process complex sentiment constituents efficiently, such as a structure resulting from double negation. Through the performance evaluation, we prove this hybrid approach particularly enhanced by sentiment lexicons and PSDs outperforms the baselines.

Introduction

This paper aims to propose a novel hybrid approach for Korean sentiment analysis through enhanced Korean sentiment lexicons and Polarity Shifting Devices (PSDs). Based on a fine-grained Korean electronic lexicon DECO that is conceived and constructed on rigorous linguistic criteria [START_REF] Nam | DECO Korean Electronic Dictionary[END_REF], this study presents a DECO-PSD classifier which incorporates graphs designing Recursive Transition Network to structure opinion corpora, processes complex sentiment constituents efficiently, and makes use of a supervised machine learning classification for the texts uncovered by the linguistic resource.

Since the advent of Web technologies, an enormous amount of data has been flooding the internet, containing opinions or sentiments of the public. To understand the public sentiment from the data, sentiment analysis research has been flourished. Studies conducted on sentiment analysis in English showed explosive growth up to sixfold in 2014 compared with 2010 [START_REF] Piryani | Analytical mapping of opinion mining and sentiment analysis research during 2000-2015[END_REF].

Sentiment analysis focuses mainly on identifying polarity including positive and negative in a document or sentence. It is to detect consumers' feelings and opinions about products or services attributed to typically text-based User Generated Contents. This raises the need to implement automatic tools for identifying the sentiment expressed in text. The classification of a document or a sentence according to its polarity can be conducted by machine learning algorithms, lexicon based methods, or even hybrid methods.

Most of supervised machine learning approaches are based on algorithms such as Naive Bayes, Maximum Entropy, and Support Vector Machine, training on a considerable amount of particular dataset [START_REF] Hatzivassiloglou | Predicating the semantic orientation of adjectives[END_REF][START_REF] Pak | Twitter as a corpus for sentiment analysis and opinion mining[END_REF][START_REF] Wang | Machine learning and radiology[END_REF]. Unsupervised machine learning approaches include algorithms like Pointwise Mutual Information (PMI) which estimates the polarity values of a word by computing the relation to the seed tokens exhibiting the explicit polarity [START_REF] Turney | Thumbs up or thumbs down? Semantic orientation applied to unsupervised classification of reviews[END_REF].

Lexicon-Based methods, on the other hand, depend heavily on linguistic resources including a sentiment lexicon composed of pairs of words and its polarity values. Since particular words exhibit polarity values, it is genuinely essential to construct sentiment lexicon data meticulously. Moreover, lexicon-based methods take into account compositional roles of contextual valence shifting [START_REF] Polanyi | Contextual valence shifters[END_REF]. For example, negating and intensifying words get involved in contextual sentiment valence shifting of a sentence significantly.

Each of two approaches has advantages and disadvantages. In the case of machine learning-based sentiment analysis, the polarity values of sentiment lexicon are primarily computed through the statistic estimation, which is advantageous in that the coverage can be widened depending on the size of training data, and it minimizes human labor to build a sentiment linguistic resource. However, it has a limitation in dealing with the linguistic compositional rules such as negation and intensification [START_REF] Neviarouskaya | Attitude Sensing in Text Based on A Compositional Linguistic Approach[END_REF]. Plus, when the classifier training on a specific dataset is utilized for another domain, its performance is more likely to drop significantly. On the other hand, lexicon-based methods have the advantage of processing the compositional rules and ensuring transparency of classification criteria. Additionally, they show robust performance across domains and texts [START_REF] Taboada | Lexicon-based methods for sentiment analysis[END_REF]. However, manual construction of sentiment lexicons requires extensive headwork with relatively limited coverage on informal forms of sentiment words. In this respect, it is necessary to develop a hybrid approach of two methods which complements the disadvantages of each methodology but combines merits.

The hybrid approach includes the machine learning and lexicon-based method containing manually written linguistic rules [START_REF] Prabowo | Sentiment analysis: A combined approach[END_REF]. Different sentiment classifiers grounded in lexicon-based or machine learning methods are used in a cascade manner so that when one classifier fails, the next one takes a turn to classify, and so on until the remaining document is categorized.

In this paper, we propose sentiment analysis in a sentencelevel, based on the hybrid approach for Korean texts. We make use of DECO-PSD classifier of DecoTex platform1 [START_REF] Yoo | DecoTex Users' Manual, TR-2017-12[END_REF] which processes compositional phenomena by using PSDs and sentiment words registered in DECO-SentLex [START_REF] Nam | DECO Korean Electronic Dictionary[END_REF] as well as takes advantage of a supervised machine learning algorithm. It has a cascading system, which is primarily grounded in a lexicon-based classifier utilizing DECO language resource holding lexical information to process compositional rules. For the sentences uncovered by the DECO dictionarybased classifier, a Naïve Bayes classifier gets involved to expand the scalability of polarity classification as a supervised machine learning algorithm training on datasets.

In this paper, Section 2 describes some related studies to this proposal. Section 3 illustrates composition model dealing with valence shifting. Section 4 presents the way to structure opinion texts for DECO PSD classification. Section 5 explains how DECO PSD classification works, and Section 6 presents the results and the comparative evaluation of the different versions of the classifier. Finally, Section 7 concludes this paper and points to some future works.

Related Studies

By introducing SO-CAL (Sentiment Orientation CALculator), [START_REF] Taboada | Lexicon-based methods for sentiment analysis[END_REF] points out that a lexicon-based method is beneficial in processing local context of a sentiment word. This system analyzes the sentiment based on the structured words, which is annotated with their polarity values, incorporating negation and intensification. The system deals with compositional rules of several linguistic contexts that can have an influence on calculating polarity values. SO-CAL shifts the polarity values to the opposite orientation for negation: for example, 'not good' has -3 polarity value due to 'good' with +3 polarity value. Amplifiers like 'so' in English magnify the sentiment intensity whereas downtoners like 'somewhat' decrease it. SO-CAL processes amplifiers and downtoners as a modifier which shifts the sentiment values, and it deals with some words that are unlikely to fit the purpose of sentiment analysis in a sentence, such as modality verbs. The system is programmed to ignore the polarity values for the sentiment lexicon collocated with them. Its performance turned out to be consistent and robust across domains. However, its coverage of sentiment words is restricted in the handcrafted sentiment dictionary, which limits to process various informal forms of sentiment words or coinages. [START_REF] Moilanen | Sentiment composition[END_REF] describes a composition model, which computes the polarity values of syntactic constituents from the head polarity of their sub-constituents. The sentiment composition model parses sub-constituents to represent the higher constituent and evaluates the output polarity of the composed constituent. It covers polarity reversal, propagation, and polarity conflict resolution within multiple linguistic constituent types. However, its practicality is bounded by the quality of a syntactic parsing performance. Since most of the text data remain highly unstructured with low grammaticality, complex syntactic parsing seems to be hardly efficient or practical. [START_REF] Choi | Learning with compositional semantics as structural inference for subsentential sentiment analysis[END_REF] deals with compositional rules in the orientation of sentiment expressions by computing the polarity values of the constituents of the expressions and applying inference rules to combine the constituents. The inference rules are specialized in a local syntactic pattern of a sentence. In particular, it points out the vital role of content-word negators, which switch the sentiment orientation of collocating words. For example, when the system detects a pattern like '[eliminate]VP [the doubt]NP' the polarity value can be computed by the inference rule 'Compose([eliminate],[doubt])' which flips the negative value of 'doubt'. The result based on compositional semantics shows better performance than baselines without the consideration of compositional semantics.

While the research above focuses mainly on processing compositional semantics of sentiment expressions limited in a sentiment dictionary, [START_REF] Prabowo | Sentiment analysis: A combined approach[END_REF] introduces a hybrid or combined approach which makes use of multiple sentiment classifiers including lexiconbased classifiers and machine learning algorithmic classifiers in a sequence of performing best. When one classifier fails to classify a document, it will pass the document onto the next classifier, until the document is sorted. However, it rather focuses on machine learning based classifier than on the quality of lexicon or rule-based classifier. Its sentiment lexicon contains the limited number of sentiment words (3672 entries) and, the rule-based classifier can process a small set of compositional rules not covering content-word negating and flow-flipping by conjunctions like 'but'. [START_REF] Lu | Combining a large sentiment lexicon and machine learning for subjectivity classification[END_REF] also have an investigation on a hybrid method for sentiment analysis which takes advantage of both the handcrafted sentiment lexicons and annotated corpus to extract sentiments, based on supervised machine learning algorithms. The Chinese sentiment lexicon (31,802 entries) is first adjusted under a machine learning algorithm according to annotated corpora as the training data and then integrated into machine learning models to detect polarity. As a result, the hybrid approach significantly outperforms the baselines. However, it does not take into consideration important compositional rules including intensification or negation. [START_REF] Dhaoui | Social media sentiment analysis: lexicon versus machine learning[END_REF] empirically evaluates the lexiconbased, machine learning and hybrid approaches using a sample (850 comments) of UGC on Facebook fashion brand pages. It shows that the hybrid approach has significantly improved the performance especially in classifying positive orientation. Its lexicon-based classifier is based on a sentiment linguistic resource of Linguistic Inquiry and Word Count 2015 (LIWC), which has a limitation to process compositional rules of sentiment constituents.

Composition Model

The processing of compositional rules, which deals with valence shifting [START_REF] Polanyi | Contextual valence shifters[END_REF], is essential in sentiment analysis. Certain words shift polarity values in a context, called Polarity Shifting Devices (PSD) [START_REF] Nam | Study on automatic recognition of Korean negation markers shifting opinion polarity[END_REF]. [START_REF] Neviarouskaya | Attitude Sensing in Text Based on A Compositional Linguistic Approach[END_REF] 2 shows the samples of compositional rules and their examples. In order to compute the polarity shifting of complex combinations, the PSD processing system iterates five times, thereby parsing the sequences applied by multiple compositional rules such as double negation. For example, in the case of sentence like '안 좋 은 것이 없다/an coh-eun keos-i eopta' (There is no a notgood thing), the polarity values of '좋은/coh-eun' (good) are shifted by double negation but remain same, and PSD DECO classifier process compositional semantics through local parsing as shown the number 4 of Table 2. The DECO dictionary was implemented in a compatible manner with the natural language processing platform, Unitex [START_REF] Paumier | De la reconnaissance des formes linguistiques à l'analyse syntaxique[END_REF]. Based on DECO dictionary, Unitex performs the morphological analysis in the input text. Its automaton processing Korean alphabets analyzes surface forms of the input text based on the lexical information of DECO dictionary, handling the complex morphological inflection in Korean.

DECO Annotation using LGGs

Figure 1: LGG for DecoPolClass Annotation

With the output corpus resulting from morphological analysis, Local-Grammar Graph (LGG) [START_REF] Gross | The Construction of local grammars, in Finite-State language processing[END_REF][START_REF] Gross | Nouvelles applications des graphes d'automates finis àla description linguistique[END_REF] can be used to extract or adjust DECO lexical information.

LGG is a Recursive Transition Network (RTN), converted into Finite-State Automata (FSA) and Finite-State Transducer (FST) to formulize lexical patterns and modify texts [START_REF] Gross | The Construction of local grammars, in Finite-State language processing[END_REF][START_REF] Nam | Understanding Maurice Gross' Language Processing Model and its Computational Applications[END_REF]. Through LGG, a user can generate the marked-up corpus of which tokens are annotated with specific lexical information as DECO codes, referring to as DECO annotation. Figure 1 shows LGG which processes to annotate 12,999 entries of sentiment words with corresponding polarity category codes. The LGG has a total of four paths set to output a specific tag after each path. For example, in the case of the first path, if a strongly positive word allocated to <QXSP> occurs in the input text, the path recognizes the string and attaches '/QXSP' to it.

(1) 재미있지 않다고 하지만 완전 인생 영화였다. caymiissci anhtako haciman wanceon insayng yeonghwayeossta.

(Not funny, but it was really the movie of my life.)

For example, when processing a sentence (1) as an input value, the morphemes of each token is tokenized and analyzed to result in a structured text (2) through the DECO lexicon which assigns various lexical information to them, including morphological, semantic, syntactic information.

( Since it contains too much lexical information, it is efficient to structure the sentence with LGG so that necessary information is extracted for sentiment analysis.

LGGs as shown Figure 2 are in a form of RTN to function as FSA and FST which has transitions from the initial state to the final state. They process some of PSD including Intensifier, Switcher, Activator, and Nullifier as explained in Section 3. When the LGGs are merged in the main graph to process the sentence (2), the following output is obtained.

(3) 재미있/QXPO지 않/SWITB다고 하지만/ZABSO 완 전/UINTF 인생/PACTF 영화/QXZE였다.

In this way, corpus modification is performed to mark up the lexical information necessary for the input corpus by using the category codes to which the sentiment words and function words of the DECO dictionary are allocated. It is used as the structured text, which is input data of DECO PSD classifier computing polarity values as well as processing linguistic compositional rules of the polarity constituents in the document. Through DECO annotation based on LGG, the four types of PSD are structured for local semantic parsing as (4). For better understanding, even though it is not parallel to (4), the parsing mechanism is somewhat similar to the way in English as following (5).

(5) (((Not (funny: +2): -2), but: 0) it was (really (the movie of my life: +2): +3): +3). (1) QXSP: +4, QXPO: +2, QXNG: -2, QXSN: -4

Hybrid Sentiment Analysis Model

Each polarity orientation has five degrees. Intensifier can result in even values such as +1, +3, +5 or -1, -3, -5. Including value zero, a sentiment word can be assigned from +5 to -5.

Results

To measure the performance efficiently, we adopt precision, recall, f-measure, and accuracy. Precision is the fraction of correct instances of a polarity among the classified cases of the polarity, whereas recall is the fraction of correctly classified instances of a polarity over the total correct instances of the polarity. F-measure is the harmonic mean of precision and recall, and accuracy is the fraction of the total of correctly classified opinions over the total opinions submitted to the classifier.

Conclusion

This paper proposes the novel approach, hybrid sentiment classification based on DECO PSD classifier processing Polarity Shifting Devices, outperforming baselines. Based on DECO dictionary and Naïve Bayes classification, it has a cascading system through which a lexicon-based classifier locally parses the sentiment constituents to detect opinions first, and then Naïve Bayes classifier sorts the Out Of Dictionary texts by training on MUSE opinion corpora.

In particular, this paper introduces the efficient composition model and how to process it, dealing with four types of PSD including Intensifier, Switcher, Activator, and Nullifier. With simple but powerful compositional rules, it is possible to compute polarity values of complex sentiment constituents such as 'double negation'.

For future works, it is in high demand to have an in-depth investigation on the lexical items which would be assigned to PSD. Since this paper describes a few examples of them, it is essential to study the various aspects of PSD and expand its lexicon. Additionally, more research should get attention to construct a linguistic resource covering a vast amount of multiword expressions so that the coverage of lexical information can expand to detect the hidden polarity values.

Figure 2 :

 2 Figure 2: Examples of PSD LGG (4) ((((재미있/QXPO지: +2) 않/SWITB다고: -2) 하지만 /ZABSO: 0) (완전/UINTF (인생/PACTF 영화/QXZE였 다: +2): +3): +3)

  DECO PSD classifier parses the local sentiment constituents of the structured texts by DECO annotation and vectorizes the polarity values. By aggregation of the values, it classifies the polarity of each sentence. DecoPolClass 'QXSP' (Strongly-Positive), 'QXPO' (Positive), 'QXNG' (Negative), and 'QXSN' (Strongly-Negative) are assigned to polarity values as (1) below.

Table 1 :

 1 classifies them into two types: 'Intensifying type' which contains adverbial intensifiers like 'very', 'so' and verbs like 'increase' and 'magnify', and 'Reversing type' which includes grammatical negators such as 'not' and 'no', or content-word negators such as 'eliminate' and 'reduce' in English. They have functional roles in sentiment semantics. On top of two types, we add two more types which shift polarity values in Korean. Consequently, in this study, the types of PSD are divided into four categories: Intensifier, Switcher, Nullifier, and Activator as explained below. Codes for PSD

	4. Activator: PSD which activates polarity values out of
	neutral words
	e.g. 너무/neomwu (too) + measuring adjectives, 인생
	/insayng (life) + product nouns
	First, Intensifier magnifies or minifies the polarity values
	of sentiment words in contexts. We take into account
	Intensifier including amplifiers such as '완전/wanceon',
	'진짜/jinjja', '너무/neomwu', etc. and downtoners such as
	'조금/cokeum', '덜/teol', etc. When collocating with
	polarity words, the intensifier-amplifiers add '+1' to the
	polarity values of nearby sentiment words, and the
	intensifier-downtoners add '-1'.
	Second, Switcher reverses the orientation of polarity values.
	It includes the function words classified as a negator.
	Negation can be classified into grammatical negation and

1. Intensifier: PSD which intensifies polarity values, including amplifiers and downtoners e.g. 완전/wanceon (fully), 매우/maywu (so), 조금 /cokeum (little), 덜/teol (less), etc.

2. Switcher: PSD which switches the orientation of polarity, including grammatical negators and lexical (content-word) negators e.g. "not": 않다/anhta, 아니하다/anihata, 못하다 /moshata, 안/an, 아니/ani, 못/mos, 아니다/anita, "there is no": 없다/eopta, 제거하다/ceykeohata (eliminate), etc.

3. Nullifier: PSD which nullifies polarity values, including imperative, suggestive, and interrogative markers or auxiliary verbs e.g. -해야 한다/-hayya hata (should), -면/-myeon (if), -ㄹ 듯/-il teus (seem like) etc. lexical negation in Korean. In the case of grammatical negation, there are the 'Short Negation' (e.g. '안 좋다/an cohta' meaning 'not good') of adverbial negators such as '안/an', '못/mos', and '아니/ani' as well as 'Long Negation' (e.g. '좋지 않다/cohci anhta' meaning 'not good') of like '말 같지도 않다/mal kath-cito anhta' (It does not make sense) or '다신 안 갈 것임/tasin an kal keos-im' (I will not visit again).

Third, Nullifier ignores polarity values. It includes functional markers used to make a sentence imperative or interrogative. In Korean, the question mark after sentiment predicate can function as Nullifier. For example, when it comes to a sentence like '그 호텔 좋음?/keu hotheyl coheum?' (Is the hotel good?), the positive values of '좋음' is more likely to be ignored since the purpose of the sentence is to ask whether the hotel is a pleasant place or not. Additionally, concessive conjunctions function as flow-flipping devices which nullify the polarity values of preceding sentiment words in the range of a sentence. In Korean, concessive ending suffixes combined with predicates are used as flow-flipping Nullifier like 'but' in English, including '-지만/-ciman', '-더라도/-teolato', '-ㄴ 데/-ntey', etc. In a sentence '아름답고 예쁘더라도 싫다 /aleumtap-ko yeppeu-teolato silh-ta' (I hate it although it looks beautiful and pretty), concessive ending suffix '-더라 도/-teolato' (although) nullifies all polarity values of preceding words such as '아름답다/aleumtapta' (beautiful), '예쁘다/yeppeuta' (pretty). hyutayphon, etc.) means 'something of my life' in English, exhibiting explicit positive polarity.

As Negative Activator, on the other hand, '너무/neomwu' is a good example. '너무/neomwu' mainly functions as an intensifier of sentiment lexicon like 'so' in English;

인생 게임/insayng keyim (the game of one's life)

Table 2 :

 2

		type code.
		Table 1 describes the category codes of PSD. Words
		assigned to the four PSD types can be continually updated
		based on the bootstrap approach, which supports
		continuous performance improvement. Except for
		Activator, the other three types of PSD shift polarity values
		of the collocating sentiment words assigned to
		corresponding polarity values of DecoPolClass. The
		intervention of 1 or 2 unassigned tokens (UK) into the
		combination of PSD and polarity word is allowed. Even
		though the code ZABSO belongs to the subcategory of
		Nullifier, it does not depend on the position code, and all
	expressing the length Korean measurement adjectives like '길다/kilta' (long), '짧 of something is excessive.	polarity values of sentiment words preceding a word assigned to ZABSO are nullified in a range of the sentence boundary. Table
	다/ccalpta' (short), '크다/kheuta' (big), etc. are basically
	neutral words, but they possess polarity values when
	modified by '너무/neomwu'. To process the sequence, it is
	essential to list up measuring adjectives. We use 1384
	entries of measuring adjectives registered in DECO
	dictionary as well as their adverbial forms to process the
	sequence.
	In order to formalize PSD, the processing code for each
	type is assigned to the corresponding PSD. Through

Examples of Compositional Rule however, it also means the degree of excessiveness as an adverb like 'too' when collocating with a measuring adjective

[START_REF] Nam | Study on automatic recognition of Korean negation markers shifting opinion polarity[END_REF]

. For example, '좋아요/cohayo' and '나빠요/nappayo' are explicit sentiment words implying 'good' and 'bad'. When '너무' collocates them, it just amplifies the polarity values of the sentiment words like '너 무 좋아요/neomwu cohayo' (so good), '너무 나빠요 /neomwu nappayo' (so bad). However, it activates the negative polarity of a measuring adjective: for instance, '길 어요/kileoy'' means 'long', which is hard to be classified as an explicit sentiment word, but it holds negative polarity through being modified by '너무/neomwu'. '너무 길어요 /neomwu kileoyo' means 'too long', annotating the PDS codes to relevant words, DECO PSD classifier locally parses the sentiment constituents. When collocating with sentiment words, each code shifts polarity values as well as controls the way of polarity shifting by fixing the direction of shifting polarity values of a neighboring sentiment word. It is formalized through the position information code (F/B) attached to the basic PSD

Table 3 :

 3 Lexical entries in four polarity categories by the part-of-speech in the DECO dictionary

	and syntax data for lexical entries. It also includes semantic
	categories such as DecoPolClass and DecoPsyClass. We
	use the sentiment lexicon from the DecoPolClass for
	sentence-level sentiment analysis. The polarity categories
	are divided into seven types ('Strongly-Positive', 'Positive',
	'Strongly-Negative', 'Negative', 'Neutral', 'Dependent
	Polarity' and 'Strongly-Dependent Polarity'). In this paper,
	four categories play a critical role in the polarity
	classification: Strongly-Positive (QXSP), Positive (QXPO),
	Strongly-Negative (QXSN), and Negative (QXNG), which
	have total 12,999 lexical entries. The following table shows
	the distribution of the sentiment entries by parts of speech.
	DecoPolClass Noun Verb Adjective Adverb Total
	Strongly-Positive <QXSP>	174	312	326	550	1362
	Positive <QXPO>	1133 1882	1263	1709	5987
	Strongly-Negative <QXSN>	254	1040	763	1052	3109
	Negative <QXNG>	2887 3531	1649	2097	2541

For the lexicon-based classifier, we utilize DECO dictionary. It is a Korean Machine Readable Dictionary (MRD), a rich language resource containing various semantic information such as inflection, parts of speech,

Table 4 :

 4 Examples of Annotated Text and Text Out of DictionaryTable4shows the examples of annotated sentences as well as those uncovered by DECO annotation. Unlike the sentences assigned to the polarity values, the lexicon-based polarity classification cannot compute a polarity value of the Out Of Dictionary (OOD) sentences. Many of them are attributed to spelling or spacing errors, but some OOD sentences are due to the limitation in processing idiomatic CLO). 70% of each domain corpus is used as the training data, and the rest of corpus is for testing data as shown Table4.

	Domain Annotated Text	Result
	RES	분위기도 나쁘/QXNG지 않/SWITB았고요.
		pwunwikito nappu/QXNGci anh/SWITBasskoyo.	+2
		(The atmosphere was not bad.)
	IPT	단단하다는 느낌은 좋았지만/ZABSO 너무/UINTF
		부담스러울/QXNG 정도로 무게감이 느껴져요.
		tantanhatanun nukkimun cohassciman/ZABSO
		nemwu/UINTF pwutamsulewul/QXNG cengtolo	-3
		mwukeykami nukkyecyeyo.
		(The feeling of its solidity was good but so heavy that I
		even feel it burdensome.)

MUSE opinion corpora are manually annotated with the sentiment classification in the sentence-level. To evaluate the robust performance of multi-domain documents, we make use of the comments about restaurants (RES), ITrelated products (ITP), travel-related services (TRA), clothes (

Table 5 :

 5 Accuracy of five domains

	Domain	NB	Accuracy NB+DECO NB+PSD
	RES	0.805		0.809	0.838
	ITP	0.693		0.752	0.78
	TRA	0.764		0.818	0.807
	CLO	0.845		0.808	0.829
	MOV	0.756		0.763	0.769
	Classifier	Polarity	Recall Precision F-measure
		Positive	0.795	0.908	0.848
	NB	Negative 0.695	0.472	0.562
		Accuracy		0.774
		Positive	0.857	0.919	0.887
	NB+DECO	Negative 0.772	0.643	0.702
		Accuracy		0.787
		Positive	0.871	0.907	0.889
	NB+PSD	Negative 0.763	0.689	0.724
		Accuracy		0.803

Table 6 :

 6 Overall performance evaluationTable5shows the accuracy of each domain, and Table6presents the overall performance of whole domain. NB indicates the Naïve Bayes classifier as a baseline classifier, and NB+DECO refers to a combined classifier without processing PSD. Notably, NB+DECO outweighs a baseline classifier even if it cannot deal with PSD processing. As expected, PSD classifier (NB+PSD) shows the best performance over others, which means the hybrid sentiment classification regarding PSD processing yields the robust performance over various domains.

It is available to download from Digital Language and Knowledge Contents Research Association (DICORA) in HUFS. homepage: http://dicora.hufs.ac.kr/