Yoshinori Hijikata
email: hijikata@sys.es.osaka-u.ac.jp

Keisuke Horie
email: horie@nishilab.sys.es.osaka-u.ac.jp

Shogo Nishida
email: nishida@sys.es.osaka-u.ac.jp

Predictive Input Interface of Mathematical Formulas

Keywords: mathematical input, probabilistic language model, predictive input, N-gram model

Currently, inputting mathematical formulas into a document using a PC requires more effort by users than inputting normal text. This fact inhibits the spreading of mathematical formulas as internet contents. We propose a method for predicting user's inputs of mathematical formulas using an N-gram model: a popular probabilistic language model in natural language processing. Mathematical formulas are usually presented in hierarchical structure. Therefore, our method incorporates hierarchical information of mathematical formulas to create a prediction model. We try to achieve high prediction accuracy of inputting characters for mathematical formulas.

Introduction

Mathematical formulas are helpful tools for representing knowledge in all research fields such as science, engineering, social science, and economics. The World Wide Web Consortium (W3C) has standardized the markup language for representing mathematical formulas, called MathML. Users have become able to present mathematical formulas on web pages. Most web browsers and formula manipulation software comply with MathML standards. Community sites related to mathematics are just beginning to become widespread. Those users share how to solve mathematical problems and teach each other their knowledge of mathematics. In the internet research field, search technologies are beginning to be applied to searching for mathematical formulas on web pages [START_REF] Hashimoto | Incorporating breadth first search for indexing mathml objects[END_REF][START_REF] Zanibbi | Recognition and retrieval of mathematical expressions[END_REF]. It is believed that potential needs for presenting mathematical formulas are becoming greater on the web.

However, mathematical formulas have still not spread to the internet as a representation media to the same degree as other media such as text, graphics, sounds and movies. The currently available bothersome methods of inputting mathematical formulas are a deterrent against formula usage on the web. Although we usually use only a keyboard when inputting natural languages, one must use both a keyboard and a mouse when inputting mathematical formulas. Mathematical formulas are not simp-ly presented as the sequence of numerical numbers, alphabet and other symbols. They are usually presented as a part of some structures such as fraction and exponent, which cannot be input using a keyboard. Users input them by clicking special buttons for the functions on the formula editor with a mouse. We believe that insisting on users using both a keyboard and a mouse engenders irritation when inputting mathematical formulas.

In this paper, we propose an input method incorporating a function to forecast mathematical characters that the user will input next. For predicting the subsequent characters that the user will input, the forecasting method uses the characters in the mathematical formulas that the user has already input. The target of the forecast is limited to mathematical structures or characters that cannot be input using a keyboard. Then, using keyboard, a user selects a prediction proposal given by our input method.

Predictive text entry is popular for inputting natural language, especially for handheld devices. In the research area of natural language processing, dictionaries are invariably used to realize predictive text entry [START_REF] Copestake | Augmented and alternative nlp techniques for augmentative and alternative communication[END_REF][START_REF] Hunnicutt | Input and output alternative in word prediction[END_REF][START_REF] Swiffin | Adaptive and predictive techniques in a communication prosthesis[END_REF]. The dictionary usually covers text elements with high appearance frequency. The forecasting method using the dictionary outputs prediction proposals when the text part the user has input most recently begins with some text in the dictionary. It shows the user the remainder of the matched text parts in a word or a phrase as a prediction proposal. It usually orders prediction proposals according to the most recently used order or the most frequently used order [START_REF] Garay-Vitoria | Text prediction systems: a survey[END_REF].

We need dictionaries specialized for mathematical formulas if we apply this method to mathematical formulas. However, unlike natural language, the subsequent characters are not narrowed down in mathematical formulas when a character is input. For example, "no see" usually comes after "Long time" in English. However, the connections among characters in mathematical formulas are not so definitive. It is difficult to create a dictionary that is effective for forecasting mathematical formulas. Therefore, we propose to make a prediction based on a probability model. We apply a probabilistic language model [START_REF] Kita | Gengo to Keisan -Kakuritsuteki Gengo Model (Language and Computation -Probabilistic Language Model)[END_REF] that is popular in natural language processing, to mathematical formulas and to make a prediction using the probability output from this model. This method outputs prediction proposals in the order of the probability of the next input for the user.

We assume that our input method works on an ordinary formula editor such as Microsoft Office Formula Editor or MathType. These formula editors usually provide an input interface based on What You See Is What You Get (WYSIWYG) 1 . Users can check their input formulas on the screen immediately after they input them. They also provide buttons for inputting characters which cannot be input using a keyboard on the top of the screen. In this study, we implemented our original formula editor to evaluate our proposed input method. Our formula editor follows the characteristics of ordinary formula editors described above.

Related Works

One popular input method of mathematical formulas is TeX. Users cannot see the input formula immediately after they input each character when using TeX. Formula editors are more popular for inputting mathematical formulas. Users can check the input formula directly because it complies with WYSIWYG. The formula editor in Microsoft Office, along with its enhanced version MathType and InftyEditor are popular formula editors. InftyEditor allows users to input a math structure by inputting a command. However, they must learn the commands in advance. Handwriting input has been studied for inputting formulas [START_REF] Laviola | Mathpad2: a system for the creation and exploration of mathematical sketches[END_REF][START_REF] O'connell | A usability evaluation of algosketch: a pen-based application for mathematics[END_REF][START_REF] Zeleznik | Hands-on math: a page-based multi-touch and pen desktop for technical work and problem solving[END_REF]. In these studies, the systems divide the input streams into a token using the user's input stroke (or gesture). Usually, they conduct character recognition by matching the extracted tokens to the characters in the database. Finally, they infer the structure of the formula using the stream of the identified characters. Although handwriting input requires a special input device like pen input, our proposed input interface does not require it.

Input word prediction has been studied since the 1980s in the field of natural language processing. Input characters have usually been predicted in a word unit [START_REF] Garay-Vitoria | Text prediction systems: a survey[END_REF]. Predicted words are usually provided when a user inputs a few beginning characters in the word [START_REF] Copestake | Augmented and alternative nlp techniques for augmentative and alternative communication[END_REF][START_REF] Hunnicutt | Input and output alternative in word prediction[END_REF][START_REF] Swiffin | Adaptive and predictive techniques in a communication prosthesis[END_REF]. Prediction is done by matching the input characters to the dictionary [START_REF] Mackenzie | Scanning ambiguous keyboard for efficient one-key text entry[END_REF]. Especially when users input long characters, they want predictive word entry. However, the next characters to input become various in the mathematical formula. Therefore, it is difficult to apply the dictionary-based prediction for the predictive math entry. We introduce the N-gram model, a popular probabilistic language model, into predictive math entry. An N-gram model is usually used for predictive text entry in the research area of Augmentative and Alternative Communication (AAC) [START_REF] Vertanen | The imagination of crowds: conversational aac language modeling using crowdsourcing and large data sources[END_REF][START_REF] Wandmacher | a system for alternative communication adapting to the context and its user[END_REF]. Reactive Keyboard is a typical study of word prediction using an Ngram model for AAC [START_REF] Hunnicutt | Input and output alternative in word prediction[END_REF]. In that study, a tree is built for the prediction, where one alphabetical character corresponds to a node. Priority is assigned to each node based on the number of occurrences of the N-gram. When a user inputs some characters, the method matches them with nodes in the tree, words in the children of the matched node are provided as prediction proposals. In another research area, Zweig et al. investigate methods for answering sentence completion questions using an N-gram model [START_REF] Zweig | Computational approaches to sentence completion[END_REF]. Although all of their methods do not consider the structure of a sentence, our method considers the structure of mathematical formulas.

In recent years, the predictive entry of natural language has been put to practical use in cell phones and smart phones. POBox is a major predictive entry system of Japanese [START_REF] Masui | An efficient text input method for handheld and ubiquitous computers[END_REF]. It outputs a word that matches the starting few characters. It also outputs characters followed by the recent input words. T9 is a popular predictive entry for cell phones [START_REF] King | Reduced keyboard disambiguating system patent[END_REF]. Characters are divided into nine groups in T9. Each group is assigned to one key in the cell phone. Matched words in the dictionary are shown to the user as prediction proposals if a user pushes a key. Predictive entry is useful for cell phones because they are equipped with limited number of buttons. Our assessment is that it is also useful for inputting mathematical formulas because current keyboards are not equipped with keys for numerous special mathematical characters.

Probabilistic Language Model

Introduction of a Probabilistic Language Model

The basic role of the probabilistic language model is to calculate the string generation probability ) Pw can be transformed to the following formula using the multiplication rule in probability theory [START_REF] Kita | Gengo to Keisan -Kakuritsuteki Gengo Model (Language and Computation -Probabilistic Language Model)[END_REF].

1 1 1 1 ( | ) () n n i i i P w w P w     (1)
An N-gram model is a popular probabilistic language model in the natural language processing. We propose a method for predicting user's inputs of mathematical formulas using an N-gram model.

N-gram model

Generally, when the probability of an event that might occur at some point in time is influenced only by the events which happened at the last N time point, we call this phenomenon an N-th order Markov Process [START_REF] Kita | Gengo to Keisan -Kakuritsuteki Gengo Model (Language and Computation -Probabilistic Language Model)[END_REF]. An N-gram model is a model that approximates word occurrences as an N-1-th order Markov Process. In other words, it is considered that the occurrence of a word at some time point depends only on the last N-1 words. The general prediction model in the N-gram model becomes the following.

The cases of N=1, 2, 3 are respectively called unigram, bigram and trigram. The actual formula for calculating the probability of N-gram model becomes the following.

1 1 1 1 1 ( ) ( | ) ( ) n n nN n n N n nN Cw P w w Cw       (3)
We designate this probability as the N-gram probability. The calculation of this probability presents the process by which the user inputs the next word in the context that the user has input the recent words. This helps the prediction of the next input. Here, 1 () n Cw stands for the number of occurrences of string of words 1 n w in the learning data.

Prediction method of math input

Problems in modeling math input

In predictive text entry, a user inputs characters sequentially. The system predicts a word to be input next, as inferred from the characters that the user has already input. Natural language is a simple time series of data when we specifically examine the apparent sequences of characters. Therefore, it can be modeled appropriately by the probabilistic language model explained in the preceding section. Mathematical formulas have structures in their presentation. They are not simple symbolic sequences. For example, mathematical formulas including fractions or integrals have hierarchical structures. This indicates that mathematical formulas cannot be modeled using simple probabilistic language models.

Hierarchical N-gram model

For solving the problem described above, we propose a hierarchical N-gram model. In this model, the user's log of math input (hereinafter, "log data of math input") is divided in hierarchical levels. A model is constructed in each hierarchical level.

The content and hierarchical level of the user's input are recorded in the log data of math input. Here is an example of mathematical formula for showing the actual log data. In this formula, , , , , ,

frac d t   
is in the first level, 0, , , , , , t sup sin sup t  is in the second level, 2 is in the third level. frac denotes fraction and sup signifies superscript. The hierarchical information is defined in advance in each character containing a hierarchical structure. For example, for the symbol of integral  , it is defined that integral range exists in the lower level of  . The previously defined hierarchical in- formation helps to record logs with hierarchical levels.

For characters that can be input using a keyboard, the unit for recording the log is a variable, numerical value, operator, function like sin and log. The name of function is detected by preparing a dictionary in which popular function names are recorded in advance. For characters that cannot be input using a keyboard, the unit for recording the log is a character that is obtainable by clicking an input button in the formula editor. Characters that cannot be input using a keyboard are Greek alphabet characters, mathematical symbols such as differential  and quantifier  , fraction, operators such as  and  , large operators such as a summation symbol and integral sym- bol, accents such as tildes and circumflexes, script such as subscript and superscript. One of the log data of the above formula becomes the following. The number written in a parenthesis is the character's hierarchical level in the formula.

{ (1), ( 1), (1), 0(2), ( 2), ( 1), ( 2), (2), 2(3), frac t sup

    (2), ( 2
), 2(3), (2), (1), (1)} sin sup t d t
For modeling the math input, the log data are divided by the hierarchical level. The log data corresponding to the k-th level are designated as "k-th level log data". Not only the characters in the k-th level but also those in the higher level are used for representing the k-th level log data because, when predicting the next input in the lower level, the last characters in the upper level can be a trigger here. For example, in the sequence (1), 0(2), ( 2)   appeared in the above log data, 0 and  is often used for integral range. Therefore, these characters occurred by  as a trigger. The log data of each hierarchical level obtained from the above log data are shown below.

 1-st level log data :   , , , , , frac d t     2-nd level log data : { , , ,0, , , , , , , , , } frac t sup sin sup t d t      3-rd level log data : { , , ,0, , , , , 2, , , 2,
, , } frac t sup sin sup t d t

   
The N-gram probability is calculated in each hierarchical level. The N-gram probability corresponding to the k-th level log data is designated as the "k-th level N-gram probability". The model of math input considering hierarchical level is designated as the "hierarchical N-gram model".

Prediction using hierarchical N-gram model

A learning dataset is required for constructing a hierarchical N-gram model. It is ideal to create the model from the user's own log data. However, it is difficult to prepare a massive amount of log data of a target user (a user who will use the predictive math entry) in advance. Therefore, we prepare general log data of math input obtained from several users. The N-gram probability in each hierarchical level is calculated based on the general log data. To reflect the target user's input pattern in the model, the input log obtained while the user uses the predictive math entry is added to the log data. The hierarchical N-gram model is updated using the additional log. The prediction is made based on the N-gram probability according to the hierarchical level where the user sets focus by a keyboard or mouse. Characters with high probability to the last N-1 inputs are output as prediction proposals.

The method predicts characters to be input next in the same unit used for recording the log. It does not predict the combination of several input units at one time. The occurrence patterns become increasingly diverse and the prediction accuracy might become low if we predict the occurrence of the combination of input units. Prediction proposals are limited to characters that cannot be input using a keyboard. For math input, it is easier to input characters directly using a keyboard than to input them using predictive math entry if they can be input using a keyboard.

Smoothing

Generally, an N-gram model has a problem called the zero frequency problem. Because the N-gram probability is calculated using occurrence frequency, the probability for a word pair that does not occur in the learning dataset becomes zero. This fact suggests cases in which no prediction proposal is presented. To solve this problem, we conduct a smoothing of the probability value. We adopt a smoothing method called linear interpolation [START_REF] Kita | Gengo to Keisan -Kakuritsuteki Gengo Model (Language and Computation -Probabilistic Language Model)[END_REF], which calculates the N-gram probability The N-gram probability is calculated as follows if N=2 (bigram).

1 1 ( | ) ( | ) (1 ) ( ) n n n n n P w w P w w P w        (4)
Therein,  is the interpolation coefficient. We set 0.7   from our experience. The N-gram probability is calculated as follows if N=3 (trigram).

 

2 1 3 2 1 2 1 1 ( | ) ( | ) ( | ) n n n n n n n n n
P w w w P w w w ww P P w

           (5)
In that equation, 3

 , 2  and 1  respectively represent interpolation coefficients for a trigram, bigram, and unigram. We set 3 0.5

  , 2 0.3   , 1 0.2  
respectively from our experience. We used the values above for interpolation coefficients in a later experiment.

Interface of predictive math entry

We implemented an interface equipped with our proposed prediction method. The interface was implemented in JavaScript. It runs on major web browsers. Figure 1 portrays our developed interface, which conforms to general formula editors. Therefore, the interface realizes WYSIWYG. It has buttons for inputting characters that cannot be input using a keyboard. The usage of our interface is the following. When a user moves a cursor to the place where the user inputs characters, predictive proposals are shown below the cursor in the predictive proposal display (see Figure 1). The user uses cursor buttons of the keyboard to move the cursor. The user moves a cursor to the predictive proposal display by hitting the space key if the user wants to select a character from the predictive proposals. To select a character from the predictive proposals, the user moves the focus to the character that the user wants to input by hitting the space key consecutively or using the cursor buttons. The selected character is input in the target place in the formula if the user pushes enter key. The user can obtain the input formula as in the form of MathML code by clicking the "get MathML" button.

MathML Code Output

Buttons for inputting characters

Display style with WYSIWYG Prediction Proposal Fig. 1. Image of a mathematical editor with our prediction method of mathematical inputs

Evaluation of prediction accuracy

The objective of this evaluation is to ascertain (1) whether our proposed prediction method outputs more highly accurate prediction results than those provided by baseline methods, and (2) whether the hierarchical N-gram model is effective for math input. As baseline methods, we use a method using only the user's recent inputs, a method using only the user's past inputs, which means that the method uses user's recent inputs, and a method using both methods. These baseline methods can be regarded as the simplest prediction methods. To evaluate the effectiveness of hierarchical modeling, we compare our hierarchical N-gram model to a general N-gram model that uses no hierarchical information.

Evaluation Data Set

We manually selected 1,000 mathematical formulas from a textbook of mathematical analysis [START_REF] Miyake | Introduction of Derivation and Integration[END_REF] for evaluation. We invited six test subjects to input those mathematical formulas to build a dataset. The six test subjects were divided into two groups comprising three persons each. One group inputs half of the mathematical formulas. The other group inputs the remainder. Each test subject in the same group inputs the same 500 mathematical formulas. Consequently, three pieces of log data per mathematical formula are obtained. Our dataset contains 3,000 pieces of log data of math input.

We apply ten-fold cross validation to these datasets and evaluate our proposed method in terms of its prediction accuracy. In detail, the mathematical formulas are divided into 10 subsets. Then nine subsets are used as a learning dataset, another subset is used as an evaluation dataset. To calculate the prediction accuracy, we presume that a user inputs each formula of the evaluation dataset from its head to the tail. The prediction is made in each input position (see Figure 2). We examine which prediction proposal matches the actual character at the input position in the log data of the evaluation dataset. That is to say, at each input position in a mathematical formula in the evaluation dataset, we acquire prediction proposals using N-1 recent inputs extracted from the log data. We regard the character at each input position in the log data as a correct character and examine which prediction proposal corresponds to the correct character. We calculate prediction accuracy as a ratio of the input position where the top k candidates of the prediction proposals include the correct character in the log data to the entire input positions. We use the top 10 candidates for evaluation. This calculation is done for all input positions for all formulas in the test dataset.

Baseline Method

The following prediction methods are compared with our proposed prediction method. We consider these prediction methods as baselines.

-Prediction using recent inputs (Recent): This method considers the character existing at the k-th former position in the input log as a prediction proposal ranking at kth order. It outputs prediction proposals to k=10. When k<10 at the current input, it outputs prediction proposals within the k-th rank.

-Prediction using recent inputs and their frequencies (Rct & Frq): This method orders the prediction proposals output by the recent input method described above according to their frequencies. The frequency of each candidate is calculated by counting its occurrences in the formula that the user is currently inputting.

-Unigram (N=1): This method makes a prediction based on the N-gram model (N=1). The prediction proposals become the top-k characters according to the frequencies in the whole dataset. It always outputs the same prediction proposals.

-Unigram (N=1) + Recent input method (Rct & N=1): This method is a hybrid method of unigram model and the recent input method. It calculates the appearance probability for all the characters using unigram model. For characters which exist in the former k characters in the log data of math input, it calculates the probability that a character at the j-th rank in the recent input method becomes a correct character (the prediction accuracy of the recent input method in Table 1 which can be calculated in advance). It adds the both probabilities and uses the added values for ordering the characters.

Comparison of our Proposed Method and Baseline Methods

We show that our proposed prediction method achieves better prediction accuracy than the baseline methods. We examine our hierarchical N-gram model (N=2, 3) and their smoothing methods (hereinafter, "N-gram model (N=2, Smoothing(Smt))" and "N-gram model (N=3, Smoothing(Smt))". The implementation becomes difficult when building an N-gram model with N = 4 and higher because it requires vast memory capacity. The prediction accuracy is calculated using the mathematical formulas in the evaluation dataset. The results are presented in Table 1. Bold values represent the best prediction accuracy among the prediction methods, which is true also for the latter table. As the results show, the proposed method (N=3 with smoothing) apparently achieves the highest accuracy. Its accuracy becomes about 89% when the number of prediction proposals to show to the user is limited to the top five ranking. Its accuracy becomes about 95% when the number of prediction proposals to show to the user is limited to the top ten ranking.

We compared our proposed method (N=3) and another proposed method (N=2). When N=3, the prediction accuracies of the top three ranking and the smaller rankings (k=1,2) become higher than when N=2. The prediction accuracies of the larger rankings (the top four ranking and the larger rankings ( 5, 6, ,10 k 

)) become worse because of the increase of the N-gram pairs in the learning dataset. However, when N=2, the prediction accuracies of the larger rankings become higher than when N=3. However, the prediction accuracies of the top three ranking and the smaller rankings become worse. The method can use only the latest input when N=2. Therefore, the prediction accuracies of the smaller rankings become worse than N=3. Regarding the result of N=3 with smoothing, the prediction accuracies are high both in the smaller rankings and in the larger rankings. It is apparent that the smoothing treatment compensates the above shortcomings by linear interpolation between the trigram and bigram.

Finally, we provide some insight into the results of the baseline methods. The prediction accuracy of the top five ranking is about 41% in the recent input method. Improvement of the precision is not apparent after the top five ranking. Therefore, characters that are repeated in one formula are limited to five kinds and fewer. Compared to the recent input method, in the recent and frequent input method, the accuracies become higher in the larger rankings. However, the degree of improvement is not great. Although unigram (N=1) always outputs the same prediction proposals, it achieves nearly 80% of accuracy in the top ten ranking. However, the accuracies in the smaller rankings are worse than those achieved using our proposed method. When combining unigram and the recent input method, the accuracies become better in the top three ranking and the larger rankings. However, the accuracies of the top one ranking become worse.

Effectiveness of Hierarchical N-gram

We validate the effectiveness of introducing hierarchical information to an N-gram for predicting the math input. We compare the case which introduces hierarchical information to the model and the case which does not introduce hierarchical information to the model. The comparison is made for N-gram model (N=3, smoothing) that achieves the best accuracy in the previous subsection.

Hierarchical information works well for a case in which the user inputs characters at a shallower level after inputting characters at a deep level. We examine the prediction accuracy for this case. Those cases cover about 15% of input positions in our dataset. Table 2 shows results for those cases. They reveal that we can increase the prediction accuracy using the hierarchical information. It might decrease the overall usability if the prediction accuracy decreases for some input conditions. We infer that incorporating hierarchical information into the model is effective for predictive entry.

Effectiveness for adding the personal log data

This subsection validates the effectiveness for adding the target user's log data to the learning data. This might deal with the inconsistency problem of the input order among users. 400 log data are always used as learning data in this evaluation. One user's 500 input logs are divided in to five sets, each of which has 100 input logs. Each set is used as test data in turn. At first, 400 input logs consist of those of the other two users. As we explained, one formula has three user's input logs. One user is randomly selected from the other two users. The selected user's log is used here. We increase the ratio of the target user's log data to the all log data in the learning data set from 0% to 25, 50, 75, 100% and calculate the prediction accuracy. Note that we do not add the target user's log data to the original 400 learning data but replace the log data for the same formula in the original learning data set. The number of logs in the learning data stays constant. This eliminates the influence of the increase of the learning data to the prediction accuracy. Six users' logs are used for this evaluation.

The average of the six users' prediction accuracies are presented in Table 3. The results show that the prediction accuracy increases a little when the ratio of the personal logs increases. The improvement is not so large. However, we can also say that our prediction method achieves accurate prediction even if it does not use the target user's personal logs. It is expected that our method improves the prediction accuracy when many log data are used as learning data even if they are other users' logs. 0.477 0.596 0.647 0.690 0.729 0.772 0.793 0.815 0.834 0.844 25% 0.476 0.597 0.647 0.688 0.728 0.772 0.794 0.818 0.832 0.844 50% 0.478 0.597 0.648 0.688 0.728 0.775 0.797 0.822 0.837 0.849 75% 0.482 0.600 0.650 0.690 0.733 0.778 0.800 0.820 0.839 0.851 100% 0.484 0.600 0.653 0.695 0.738 0.779 0.800 0.820 0.842 0.853 7

Evaluation of Usability

The previous section showed that our proposed method outperforms other prediction methods in prediction accuracy. However, that fact does not mean that our proposed method helps users to input mathematical formulas. The objective of this section is to show that our interface of predictive math entry (an input interface incorporating our proposed prediction method) helps users' actual inputs by conducting user evaluation according to the interface's usability.

Experimental Condition

The evaluation of usability is accomplished according to quantitative indices and qualitative indices. The number of times of switching between a keyboard and a mouse (hereinafter, "#switching"), input time and the number of incorrect inputs (hereinafter, "#incorrect") are used as quantitative indices. Questionnaires related to the usability of the interface, comprising multiple-choice questions and free description, were used as qualitative indices. In this experiment, 12 graduate and undergraduate students participated. We asked them about their experiences of inputting mathematical formulas on PCs. All users answered that they had some experiences on inputting mathematical formulas on PCs. They answered that they had used general formula editors or TeX. Among the participants, three users were good at inputting formulas using TeX and were able to input most formulas without seeing the reference.

Evaluation on the Differences among Interfaces

Experimental Method

This subsection shows the effectiveness of our interface of predictive math entry. Our interface was compared to a general formula editor and an input interface using TeX. Our interface of predictive math entry used in the experiment is the interface depicted in Figure 1. We eliminated the function of our predictive entry from the above interface. That is used as a general formula editor. We originally implemented an input interface using TeX on JavaScript. The screen shot is depicted in Figure 3. A user inputs mathematical formulas using TeX command in its text area. When the user clicks the compile button, the system outputs the compiled formulas with rendering. This interface does not comply with WYSIWYG.

The users input 60 formulas (20 formulas using each interface). The number of characters in each formula is set within some range. These formulas are selected from a textbook of mathematical analysis [START_REF] Miyake | Introduction of Derivation and Integration[END_REF]. We carefully selected formulas different from the formulas in the learning dataset. All the 3,000 formulas used in evaluation of prediction accuracy are used as the learning data for our interface. The quantitative indices are measured in each input formula. After inputting 60 formulas, users answered the questionnaires for qualitative evaluation. The question items are shown in the 1-st column of Table 4. Q1 and Q3 are provided to ascertain which interface the users can use to input formulas with a good level of comfort. We set Q2 to find the time the user felt for inputting formulas. The actual time might differ from their sensory time. Q4 is provided to elicit how fast the user can learn the input method in each interface. Q5 is provided to find out which interface the user prefers with all evaluation viewpoints considered.

Results on Quantitative Indices

The results related to quantitative indices are presented in Table 5. The number of times necessary for key inputs and those of mouse clicks (hereinafter, #keyinputs and #clicks) are also presented in Table 5. Furthermore, we examined whether a statisti-cally significant difference exists among the interface on these indices using a Student t-test. The results are presented in Table 6. In Tables 5 and6, "ours" means our interface with predictive math entry, "w/o prediction" means the input interface without predictive math entry and "TeX" means the input interface using TeX. We confirmed the statistically-significant difference among interfaces in all indices except #incorrect between our interface and that without predictive entry.

The results showed on input time, it is apparent that our interface achieves the shortest input time. Input time is an important index to show that the user can input formulas smoothly. When comparing our interface and the interface without predictive entry, it is apparent that our interface decreases the #switching that is regarded as a reason that inputting mathematical formulas is bothersome. The reduction rate is about 89.1%. In the TeX interface, #incorrect becomes the three times that of the proposed interface and the interface without predictive entry. From these results, it is apparent that the incorrect inputs decrease when users can see the input formulas immediately after inputting characters. No significant difference was found between our proposed interface and the input interface without predictive math entry. However, the value of #incorrect is lower in the proposed method than the interface without predictive entry. A clear difference might be found if we increase the number of users in the experiment. #keyinputs is higher in the proposed method than the interface without predictive entry because users input characters by selecting the prediction proposal with a keyboard and move the cursor with cursor key.

Results on Qualitative Indices

The results of questionnaires for qualitative evaluation are presented in Table 4. The value for each item is the number of users who selected the item. Values shown in the parentheses are the numbers of users who can input mathematical formulas without seeing any references (hereinafter "TeX users"). Our interface achieves the best evaluation for all questionnaire items (Q1 -Q5). Especially for Q3, all users answered that they can input formulas most intuitively using our interface. In our interface, users can see the input formulas right after they input each character. Our interface also provides the prediction proposals that the users want to input. These characteristics engender the users' high evaluations to our interface. When we checked users' free descriptions, many users gave the opinion that switching a keyboard and a mouse took a burden in inputting mathematical formulas. This result supports our proposed interface that decreases the mouse input using the predictive entry.

However, some users supported the TeX interface in answers to the questionnaires. They gave the opinion that they are comfortable with the interface, and that they are used to it because they usually use TeX for inputting formulas. Actually, they were able to input formulas smoothly when using the TeX interface in the experiment. In the TeX interface, users can input formulas only by a keyboard when they learn the TeX command. Therefore, users who learn the TeX command tend to assign positive opinions to the input interface using TeX.

Evaluation on the Differences in Prediction Accuracy

Experimental Method

This subsection presents results of an examination of the influence of the differences in prediction accuracy on usability. Prediction methods of three types were selected considering the difference in prediction accuracy. The N-gram model (N=3, smoothing) was selected as a method with high prediction accuracy. The recent input and unigram method and the recent input method were selected as a method with medium prediction accuracy and a method with low prediction accuracy, respectively. Hereinafter, we designate these methods as "high-accuracy method (high)", "mediumaccuracy method (medium)", and "low-accuracy method (low)". We conducted a user experiment using our input interface depicted in Figure 1. We changed the prediction method in this interface for the experiment.

The experimental method is the same as that used in the evaluation on the differences among interfaces. Only question items for the qualitative evaluation differ from those used in the previous evaluation. Table 7 presents the question items. Q3 is provided to know that the users noticed the difference in the prediction accuracy. Q1, Q2, and Q4 are the same questions as those used in the prior evaluation.

Results on Quantitative Indices

Table 8 presents the results on quantitative evaluation. #keyinputs and #clicks are also shown in Table 8. We conducted Student t-test to assess differences among the prediction methods. The results are presented in Table 9. We confirmed statistically significant difference among the methods in all indices except input time and #incorrect between the medium-accuracy method and the low-accuracy method. When particularly addressing #switching in Table 8, #switching decreases when the prediction accuracy becomes high. #incorrect decreases in the high-accuracy method compared to the other methods. The input time becomes the shortest in the high-accuracy method. From these results, it is apparent that prediction accuracy influences the usability for inputting formulas.

It is particularly interesting that the input time becomes shorter in the low-accuracy method than in the medium-accuracy method. The reason for this result is the following. The low-accuracy method (recent input method) shows only those characters which the user has input before as prediction proposals. In this case, the user can expect what characters are given as prediction proposals while inputting characters. We think that users use predictive entry by considering what characters are given as prediction proposals next. In fact, when we observed the users' input activities when they used the N-gram model (N=3, smoothing), we found that some users moved the input cursor to the predictive proposal display before checking what predictive proposals are shown there. In N-gram model (N=3, smoothing), the users input formulas under the expectation in which their target character exists in the higher rank in the predictive proposal list because the prediction accuracy is highly sufficient. Actually, they input formulas very smoothly by forecasting the prediction results. With the recent input and unigram method, the users had difficulty forecasting the next prediction proposals because unigram does not achieve the high-accuracy prediction and the prediction proposals change according to the most recent inputs. Based on this result, it is important for users to forecast the next prediction proposals for predictive entry.

Results on Qualitative Indices

The results of questionnaires for qualitative evaluation are presented in Table 7. Most users selected the high-accuracy method as the best method for all question items. This result corresponds to the results on the quantitative evaluation. For Q1 and Q3, one user selected the low-accuracy method as the best method. The reason is related to the user's input activities explained in the previous subsection. The user preferred the prediction that is easily forecasted before shifting the input cursor to the prediction proposal display.

Conclusions

As described in this paper, we proposed an input method that predicts the next input characters for mathematical formulas. N-gram model is applied to our method, which is a popular probabilistic language model. We incorporated hierarchical information in mathematical formulas into an N-gram model. The calculated probability was used for predictive math entry. The proposed method is evaluated using prediction accuracy. Results showed that the prediction accuracy of our method is higher than that of other baseline methods. The proposed input interface was evaluated using a user experiment. Results showed that our interface outperforms the input method without predictive entry and the input method using Tex in the usability. We expect that our interface for predictive math entry shall contribute to the spread of math contents. The ease with which users can forecast the prediction proposals is important for usability. We will examine the relation between user predictability and usability for the predictive math entry.

  the N-gram probability but also using the lower-order M-gram probability (M < N). Actually, it linearly interpolates N-gram probability order M-gram probability in the following equation.

π( 1 )PredictFig. 2 .

 12 Fig. 2. Method for calculating prediction accuracy

Fig. 3 .

 3 Fig. 3. Input interface using TeX

Table 1 . Probability of the correct charcter appearing in the top k candidates

 1

		k=1	2	3	4	5	6	7	8	9	10
	Recent	0.154 0.286 0.353 0.393 0.410 0.416 0.416 0.416 0.416 0.416
	Rct & Frq 0.181 0.295 0.359 0.394 0.411 0.415 0.416 0.416 0.416 0.416
	N=1	0.287 0.496 0.571 0.595 0.638 0.681 0.702 0.728 0.758 0.793
	Rct &N=1 0.242 0.498 0.614 0.668 0.710 0.736 0.787 0.810 0.826 0.844
	N=2	0.546 0.703 0.778 0.826 0.856 0.880 0.898 0.912 0.925 0.935
	N=3	0.627 0.730 0.780 0.806 0.819 0.826 0.833 0.839 0.843 0.845
	N=2, Smt 0.541 0.689 0.758 0.798 0.838 0.873 0.899 0.916 0.930 0.940
	N=3, Smt 0.657 0.770 0.833 0.844 0.888 0.907 0.921 0.934 0.945 0.954

Table 2 . Probability of the correct character appearing in the top k candi- dates :comparing the case with hierarchical information(with) and that without hierar

 2

	-

Table 3 . Probability of the correct character appearing in the top k candidates :changing a ratio of log data inputted by a certain user in the learning dataset

 3

	k=1	2	3	4	5	6	7	8	9	10
	0%

Table 4 . Results of questionnaires for finding differences between each interfaces (The value for each item is the number of users who selected the item. The values shown in parentheses are the numbers of users who can input mathematical formulas without see- ing any references)

 4

	ours	w/o predic-	TeX
		tion

Table 5 . Results of the experiment for finding differences between interfaces

 5

		ours w/o prediction TeX
	#clicks	1.69	13.2	0
	#keyinputs 86.8	52.2	121
	#switching 1.79	16.5	0
	input time 66.0	74.0	89.6
	#incorrect 1.38	1.57	4.43

Table 6 . Results of t-test for the experiment for finding differences between interfaces

 6

		Ours vs. w/o prediction Ours vs. TeX w/o prediction vs. TeX
	#switching	***	***	***
	input time	***	***	***
	#incorrect		***	***
			*** : p<0.01

Table 7 . Results of questionnaires for finding differences between different prediction methods

 7

	high medium low

Table 8 . Results of the experiment for finding differences between different prediction methods

 8

		high medium Low
	#clicks	1.32	4.00	6.99
	#keyinputs 87.5	96.3	73.4
	#switching 1.46	4.20	8.70
	input time 54.3	62.7	61.9
	#incorrect 1.21	1.43	1.53

Table 9 . Results of t-test for the experiment for finding differences between different pre- diction methods

 9

		high vs. medium high vs. low medium vs. low
	#switching	***	***	***
	input time	***	***
	#incorrect	**	*
		*** : p<0.01, ** : p<0.05, * :p< 0.1

WYSIWYG (What You See Is What You Get) is technology which provides an input interface where the content displayed on the screen matches the content of process (especially printed results).


Eleftherios Papachristos

Nikolaos Avouris
email: avouris@upatras.gr

The Influence of Website Category on Aesthetic Preferences

Keywords: Website design, aesthetic evaluation, website categories, visual appeal, preference mapping

This paper investigates whether users' aesthetic impressions about websites vary considerably across different domains. The assumption that aesthetic judgments about websites that belong to diverse domains are based on different visual design aspects has been investigated in three distinct studies in healthcare, tourism, and web design business. In these studies participants expressed their overall preference as well as their judgments on the constructs of visual appeal, perceived usability and novelty. In addition, descriptions about the test websites were obtained by expert panel and objective measures. Preference Mapping (PM), which is a data summarization and visualization technique, has been performed in each study. Attribute projection into the preference maps allowed for the identification of important driver of preference for each individual domain. Even though, visual appeal was the most important predictor of overall preference in all studies, appealing websites had different visual characteristics in each domain. Furthermore the importance of the evaluation constructs varied considerably among studies, indicating that aesthetic perceptions differed considerably across domains. These findings emphasize the need for flexible evaluation methods that can be used to identify important visual design factors within a specific website domain.

Introduction

Since Tractinsky et al. [START_REF] Tractinsky | What is beautiful is usable[END_REF] in 2000 published their seminal paper "What is beautiful is usable" much has changed in the HCI community in regard to aesthetics research. The lively debate that was initiated with the controversial suggestion that aesthetic design could influence perceptions about pragmatic qualities of user interfaces shifted attention to more subjective aspects of interaction. Particularly for websites the construct of visual appeal (used by some authors interchangeably with beauty) has been proven to be a very important factor determining users' overall impressions [START_REF] Schenkman | Aesthetics and preferences of web pages[END_REF]. Furthermore, a series of experiments (e.g. [START_REF] Lindgaard | An exploration of relations between visual appeal, trustworthiness and perceived usability of homepages[END_REF]) have sown that users could form stable visual appeal judgments in time periods of as short as 50 msec. These judgments are based mainly on visual design since other aspects (e.g. content) cannot be recognized during such sort time periods. Surprisingly, relatively stable judgments in the same exposure times could also be found for the constructs of perceived usability and credibility [START_REF] Lindgaard | An exploration of relations between visual appeal, trustworthiness and perceived usability of homepages[END_REF]. These results demonstrate the general importance of websites visual design. Although, actual website use could influence users' perceptions, it is nonetheless important to create positive first impressions, considering that it is more difficult to overcome negative ones. These research findings emphasize the importance of website's visual design and therefore the need for appropriate evaluation methods. The most common evaluation approach is using one of the aesthetics oriented multiple-item instruments that are gaining steadily acceptance in the HCI community; for example the "Classical-Expressive" aesthetic scale [START_REF] Lindgaard | An exploration of relations between visual appeal, trustworthiness and perceived usability of homepages[END_REF], AttrakDiff [START_REF] Schenkman | Aesthetics and preferences of web pages[END_REF] or the more recent visAWI [START_REF] Moshagen | Facets of visual aesthetics[END_REF] questionnaire. Among the advantages, of those multiple-item questionnaires is that they ensure fairly reliable and valid measurements and that they provide common ground for results communication and for between study comparisons. These questionnaires have been created to be sufficiently generic in order to be applicable to most -if not all -websites. This generality has been achieved by the inclusion of a variety of websites from different domains as test stimuli during questionnaire creation. Both the VisAWI and the "Classical-Expressive" aesthetic scale, for example, reported a similar website sampling procedure during questionnaire creation and validation. The result of using websites of different domains as test stimuli is that design factors pertinent to specific domains are canceled out. Visual design evaluation with one of the aforementioned questionnaires means having participants rate a design on a predefined set of factors that have been identified to be important for websites in general. However, websites vary in terms of purpose, target user groups, and therefore visual styles. Although, there is no commonly agreed upon taxonomy, various categorizations schemes of websites have been proposed. Studies have shown that users have distinct mental models for different kinds of websites [START_REF] Roth | Mental models for web objects: Where do users expect to find the most frequent objects in online shops, news portals, and company web pages[END_REF]. According to Norman [START_REF] Norman | Some observations on mentai models[END_REF] users form internal mental models of things with which they have interacted, which in turn creates expectation about similar objects they may encounter. These expectations are becoming stronger as the number of encounters increases and users are gaining more experience with a particular website domain. For example, users may have different expectations about the visual design of news websites or online shops. Tuch, et al. [START_REF] Tuch | The role of visual complexity and prototypicality regarding first impression of websites: Working towards understanding aesthetic judgments[END_REF] showed that users' aesthetic judgment can be strongly influenced by their perceptions of website prototypicality. Prototypicality refers to the amount to which an object is representative of a class of objects and depends heavily on each individuals mental models that are build through experience. Designs that contradict what users typically expect of a website may lead to a negative first impression [START_REF] Tuch | The role of visual complexity and prototypicality regarding first impression of websites: Working towards understanding aesthetic judgments[END_REF]. The fact that prototypicality is an important influencing factor in aesthetic judgments has been shown repeatedly in various empirical studies [START_REF] Tuch | The role of visual complexity and prototypicality regarding first impression of websites: Working towards understanding aesthetic judgments[END_REF]. Thus, relying on predefined questionnaires could mean ignoring the visual design aspects that in the users mind have particular importance in the specific website domain.

Other evaluation methods such us Repertory grid technique (RGT), Multidimensional Scaling (MDS), or Preference Mapping (PM) are based on multiple website evaluations and do not impose a set of predefined evaluation criteria on participants. In addition, conducting evaluation studies with one of these techniques allows for the identification of design factors that are important in specific website domain by using multiple websites from the same domain as study stimuli. In the studies presented in this paper Preference Mapping (PM) has been used for evaluation of visual design of a number of examples of websites of a specific domain. This was repeated in the domains of healthcare, tourism and web design businesses. The main objective of this paper is to demonstrate that different design factors can be important driver of preferences in websites that belong to different domains, based on comparison of the findings of evaluation of the websites of these domains. This research is part of a broader research project that attempts to define guidelines for design and evaluation of various kinds of websites including social media presence of small medium organizations in various fields.

Method

Three evaluation studies have been conducted involving website designs from three distinct domains. In the first study (Healthcare domain) 15 hospital websites were evaluated by 34 participants (29 male, 4 female, mean age = 22.2). In the second study (Tourism domain) 32 participants (21 male, 11 female, mean age = 23.3) were asked to evaluate 18 hotel websites. In the third study (Web design business domain) 12 websites of web design companies were evaluated by 30 participants (17 male, 13 female, mean age = 28). These particular websites types were selected because they represent domains that differ both in terms of characteristics and in purpose. However, given that the goal was to identify important visual design characteristics that can shape user first impressions in each of these categories it was important to make sure that none of the test websites were previously known to our participants. Therefore, test websites were randomly selected from lists of top U.S. hospitals in the first study and New Zealand hotels in the second. Since all of our participants were of European origin it was assumed that these choices would minimize the possibility of prior familiarity with the specific test websites and thus influence the "first impression" effect. None of the participants reported previous experience with any of the test websites. In all cases participants volunteered to take part in the evaluation studies and did not receive any compensation. In all studies screenshots were used instead of actual websites, as our goal was to study the impact of various visual design aspects on participants' first impressions. Although, studies have shown [START_REF] Lindgaard | An exploration of relations between visual appeal, trustworthiness and perceived usability of homepages[END_REF] that perceptions about constructs such as visual appeal are relatively stable over time evaluations after actual website use could be biased by non-visual design related aspects (e.g. content).

Procedure

In each evaluation study participants first viewed screenshots of all the websites in a random order and then rated them according to their overall preference on a linear, unmarked scale (from 0 to 100) with the verbal anchors "least preferred" and "most preferred" at the two ends. In a subsequent evaluation phase participants were asked to rate the websites again on the constructs visual appeal, credibility, perceived usability and novelty. Since participants had to rate multiple websites in each study only a limited number of evaluation constructs had been included in the studies in order to avoid participant fatigue. However, the identification of important design characteristics required a better profiling of the test websites that could be provided by these four constructs alone. For this reason all websites were rated on various descriptive attribute by an expert panel.

Nine experts (visual designers, HCI practitioners, and web developers) identified a list of visual design aspects which could possibly influence user preferences. The goal was to find a comprehensive list of descriptive attributes, such as symmetry or complexity, could reflect variations in website designs. After a literature review and panel discussions, a preliminary list of attributes was tested on a set of generic websites in order to eliminate unsuitable attributes. The criteria for elimination were: limited discrimination ability and disagreement between assessors about meaning. In a subsequent session our experts rated the actual test websites on the final 15 attributes. Before finalizing the descriptive dataset, the attributes that did not discriminate significantly between our actual websites were identified through mixed model ANOVA's (websites as fixed and experts as random factors) and were excluded from further analysis.

In addition to participant and expert ratings we also used 16 objective measures that could be grouped into three categories: a) text related metrics (e.g. number of words, number of visible links), b) area related metrics (e.g. percentage of website used to display images or text), and c) color related metrics (e.g. average brightness, saturation). The text related measurements were taken with the help of optical character recognition (OCR) software and were double checked manually. This technique was used instead of html parsing since contemporary websites use graphics or flash instead of plain text very frequently. Websites fragmentation to specific areas (e.g. navigation, images) was done manually with the help of graphic editing software. For the color related measurements, a color recognition program has been written that parsed the website screenshots and calculated the metrics.

Analysis

In order to identify which design characteristics were most influential in preference formation, a common approach would be to perform multiple regression with preference as the depended variable and the various attribute ratings as the predictors (e.g. [START_REF] Schenkman | Aesthetics and preferences of web pages[END_REF]). However, since a purely exploratory approach was followed in regard to the selection of predictors the number of independent variables (30-35) in most studies was larger than the number of observations. Using all predictors at once would overfit the regression model. In addition most of the predictor variables are highly correlated to each other, which can lead to multicollinearity problems. Principal Component (PCR) or Partial Least Square (PLS) regression models in which a large amount of predictors are transformed into view orthogonal uncorrelated components are better suited for these circumstances. A data analysis method based on PCR called Internal Preference Mapping (IPM) has been used in our study in order to identify important design characteristics in each case. Preference mapping is referred to as a group of multivariate statistical techniques aimed at gaining deeper understanding of participants' preferences toward stimuli [START_REF] Meullenet | Multivariate and probabilistic analyses of sensory science problems[END_REF]. The method is a data summarization and visualization technique that creates low dimensional maps depicting stimuli and individual participant preferences simultaneously. This is usually accomplished by conducting Principal Component Analysis (PCA) on a data matrix consisting of stimuli in rows and participant preferences in columns. Since in PCA the first components account for the maximum possible amount of variance two or three dimensional spaces are usually sufficient to capture the majority of the underlying preference structure. Based on Euclidean distances conclusion can be drawn regarding website similarities as well as individual participant preferences towards them.Interpretation of the resulting dimensions as well as identification of important drivers of preferences can be accomplished by projecting additional website attributes into the preference map. Additional data about websites can be projected in the preference space by using average attribute scores as dependent and website factor scores as independent variables in a regression model. The regression coefficients represent the strength of the relations between the additional attributes and the preference dimensions. Attributes that have no relationship with any of the preference dimension cannot be used for preference interpretation and should therefore be removed from further analysis. Thus, during website data projection insignificant attributes that cannot explain participant's preferences can be identified and discarded.

Participant preferences ratings were submitted to a PM analysis for each study individually. Figure 1 shows the three resulting preference maps. In these maps website designs are represented by capital letters while attributes are depicted as light vectors. The vectors indicate the general direction in which the intensity of each attribute increases. Attributes with vectors that point to a similar direction are positively correlated while attributes with vectors pointing to opposing directions are negatively correlated. Websites that lie in the general direction to which an attribute vector points have high intensities of that attribute while objects in the opposite directions have none or low intensities of the same attribute. The bold vector indicates the average preference direction of the participant sample.

From the initial list of 35 attributes only 9, 13 and 2 could be successfully projected in the 2-dimensional preference spaces of the healthcare, tourism, and web design business studies respectively. Although, analysis has been conducted in higher dimensional spaces the results presented here are primarily focused on the first two dimensions which represent the most important components in participant preferences.

In the healthcare study the first preference dimension represented 34.7% of the total preference variance and was highly correlated with the constructs of visual appeal, perceived usability and credibility. The second dimension which explained 14.7% of preference variance correlated with descriptive attributes such as simplicity, order, brightness on one site and large image area, white space and texture on the other.

In the tourism study the first dimension that captured 25.2% of preference variance differentiated among websites on the left site of the map which were mostly graphics based, had more white space, and larger image areas while designs on the right site were mostly text based. The later was confirmed by trained panel data (mostly graphics attribute) and by objective measures (text area, word count). The second dimension was primarily correlated with perceived usability and credibility as well as to the descriptive attributes texture, colorful and dimensional. Novelty was positively correlated with the first and negatively with the second preference dimension.

In the web design business study only two constructs could be projected in the 2dimensional preference space. Interpretation of the preference dimensions solely based on these constructs indicate that the most important component is more related to visual appeal and the second to novelty. It is noteworthy that none of the descriptive attributes as well as none of the constructs credibility and perceived usability were useful in interpretation of the first two preference components in this study. The attributes order, credibility and perceived usability were actually found to be highly correlated with the third dimension that represented only 13.6 % of preference variance.

Considering the average preference direction of the participant sample the most important predictor of website preference was visual appeal in all studies (as in [START_REF] Schenkman | Aesthetics and preferences of web pages[END_REF]). However, in the healthcare study visual appeal was highly correlated with perceived usability and credibility. In addition, visual characteristics that were common among preferred websites in this category were simplicity, order large image areas and white space. In the tourism study visual appeal was not correlated with credibility or perceived usability. Preferred websites in this category had vivid colors, many images, plenty of with space, and a small text area. In the web design business study only novelty could be recognized as an important design factor apart from visual appeal.

Discussion

Preference mapping analysis revealed that diverse design characteristics can be important drivers of preferences for designs that originate from different website domains. Although, the most important preference predictor in all studies was visual appeal, what constitutes an appealing website differed considerably among the three studied cases. In the healthcare study the constructs perceived usability and credibility were found to be equally important as visual appeal. Generally preferred and appealing websites in this study were simple, ordered, spacious designs with large image areas. In the tourism study participants showed an aversion towards websites that were primarily text based. Designs that used mostly graphics, vivid colors and large images of hotel rooms were the most preferred and appealing websites. Perceptions of usability and credibility were exclusively related to the second dimension and were therefore less important drivers of preference than in the healthcare study. Finally in the web design business study only visual appeal and novelty could be found to be important drivers of preferences in the two dimensional space. Websites in this study varied on more design factors and could be characterized as unusual or extreme compared to designs in the others studies. This was generally expected since visual design in this domain serves as a first showcase of the company's ability to produce cutting edge design. Creativity and unconventionality cannot be appropriately captured by descriptive attributes and therefore none of the expert panel or objective measures were useful in preference interpretation in this study. The constructs of perceived usability and credibility that were found to be important drivers of preference in the other two website domains were less important in this one.

These results demonstrate that different design aspects play a determining role in preference creation towards websites within a specific domain. Therefore, misleading conclusions can be drawn by relying on a fixed set of evaluative or descriptive attributes for evaluation purposes of websites in general. Use of generic questionnaires for website design evaluation could lead to consideration of less relevant attributes into the evaluation process while design aspects that are central to a specific website domain could be ignored. For example, novelty was found to be one of the most important drivers of preference in the third (web business), fairly important in the second (tourism), and not important at all in the first (healthcare) study. Furthermore, symmetry which is an item in the classical aesthetic dimension in the questionnaire of Lavie and Tractinsky [START_REF] Lavie | Assessing dimensions of perceived visual aesthetics of web sites[END_REF] was not found to be an important preference attribute in any of the studies presented in this paper. In addition, use of images that has been found to be an important driver of preference in two out of the three studies is a design aspect that is ignored by all aforementioned multiple item questionnaires [START_REF] Lavie | Assessing dimensions of perceived visual aesthetics of web sites[END_REF] [5][6]. This paper reports results from three different studies in which the influence of website domain on users' aesthetic preferences has been investigated. The results suggest that diverse design aspects can influence participants overall impressions of website designs in different domains. A different set of descriptive and evaluative constructs could be identified as important drivers of preference in three distinct studies involving healthcare, tourism and web design business websites. Visual appeal was the most important predictor of participant preferences towards websites in all case studies. However, appealing websites had different visual characteristics in each domain. To conclude, the results in this paper demonstrate the influence of website domain in shaping users' aesthetic preferences. This finding emphasizes the need for flexible evaluation methods that do not ignore the visual design aspects that are important in specific website domains.

Fig. 1 .

 1 Fig. 1. Preference maps with attribute projection for each of the three studies. Light vectors represent attributes. Attributes that begin with a capital letter are participant construct ratings. The bold vector indicates the average preference direction of the participant sample.

Acknowledgements

This work has been partly funded by the European Territorial Cooperation Operational Programme "Greece-Italy 2007-2013" under the project Intersocial.