Fa ´tima

L Mota

Anto ´nio

J Queimada

Sima ˜o

P Pinho

Euge ´nia

A Macedo

Aqueous Solubility of Some Natural Phenolic Compounds

In this work, the aqueous solubilities of two hydroxybenzoic acids (gallic and salicylic acid) and three phenylpropenoic acids (trans-cinnamic, ferulic, and caffeic acids) are addressed. Measurements were performed, as a function of temperature, between 288.15 and 323.15 K, using the shake-flask method for generating the saturated aqueous solutions, followed by compositional analysis by spectrophotometric and gravimetric methods. The pH values of the saturated aqueous solutions were measured by potentiometry. Additional thermodynamic properties, which are fundamental for a better understanding of the solubilization process, as well as necessary for the modeling studies, such as melting temperatures and fusion enthalpies were determined by differential scanning calorimetry (DSC). Apparent acid dissociation constants (K a ) were obtained by potentiometry titration. The measured data were modeled with the cubic-plus-association (CPA) equation of state (EoS). This EoS is applied, for the first time, for multifunctional associating molecules, and the results indicate that it can adequately be used to represent the measured and other literature data with satisfactory accuracy.

Introduction

Fruits, vegetables, spices, and aromatic herbs are a natural source of phenolic compounds. These can be raw materials for the synthesis of different molecules with industrial interest, such as some drugs, cosmetics, antioxidants, antiseptics, and flavors; they also can be used in the preparation of resins, plasticizers, dyes, inks, and pharmaceutical products. [START_REF] Apelblat | The solubilities of benzene polycarboxylic acids in water[END_REF] Many of these phenolic compounds have potential chemical and biological properties, such as antioxidant, chelating, free-radical scavenging, anti-inflammatory, antialergic, antimicrobial, antiviral, anticarcinogenic, chemoprevention (interfering with a disease process), and ultraviolet (UV) filteriing properties. Most of them are phytochemical (they are not required for the normal functioning of the body, but they do have a beneficial effect on health, or an active role in the treatment of a disease).

Because of their hydrogen-bonding ability and aromaticity, phenolic compounds can frequently act as free-radical scavengers, forming aryloxyl radicals (ArO • ). The stabilization of these radicals by other functional groups enhances the antioxidant activity. [START_REF] Chen | Antioxidant activities of caffeic acid and its related hydroxycinnamic acid compounds[END_REF] They also have ability to scavenge the oxidative radicals that are caused by cumulative and prolonged exposure to UV radiation, which has been recognized to induce harmful reactions in human skin. There are several studies concerning topically applied antioxidant products for this purpose. [START_REF] Saija | In vitro and in vivo evaluation of caffeic and ferulic acids as topical photoprotective agents[END_REF] The low incidence of coronary heart disease, atherosclerosis, and some types of cancer has been correlated by several authors with the phenolic content of olive oil. [START_REF] De Lorgeril | Mediterranean diet, traditional risk factors, and the rate of cardiovascular complications after myocardial infarction;Final report of the Lyon Diet Heart Study[END_REF] This leads to the recognition of the olive phenols (which are present in the fruit, leaves, and oil) as potential targets for the food and pharmaceutical industries. Recent studies have highlighted the potential recovery of some of these compounds from olive mill wastewaters. Because of their acidity and high organic content, these cannot be treated by the regular sewage systems, and their treatment is particularly difficult. The recovery of some of these high-value phenolic compounds has been discussed in the recent literature as a means to promote the following biodegration of the wastewaters, while obtaining economic benefits from the separated high-valued phenolic components. [START_REF] Allouche | Toward a high yield recovery of antioxidants and purified hydroxytyrosol from olive mill wastewaters[END_REF][START_REF] Obied | Investigation of Australian olive mill waste for recovery of biophenols[END_REF][START_REF] Obied | Bioactivity and analysis of biophenols recovered from olive mill waste[END_REF] For the adequate design of processes and products that involve these phenolic compounds, it is important to know some of their physicochemical properties, particularly the solubility in different pure and mixed solvents, because these are solids under ambient conditions.

In this work, the aqueous solubilities of some natural phenolic compounds (such as the hydroxybenzoic acids gallic and salicylic acids) and the phenylpropenoic acids (trans-cinnamic, ferulic, and caffeic acids) are addressed. Their chemical structures are presented in Figure 1.

Salicylic acid is the key additive in many skin care products; in addition to being used as an analgesic and antipyretic, it is used as a plant hormone. In small quantities, it is a food preservative and antiseptic in toothpaste. It is found in several plants, notably in wintergreen leaves and the bark of sweet birch. [START_REF]Ulmann's Encyclopedia of Industrial Chemistry[END_REF] Gallic acid, because of its many significant biological activities (such as its use as an antioxidant and antiviric), [START_REF] Lu | Solubilities of gallic acid and its esters in water[END_REF] is commonly used in the pharmaceutical, food, and pigment industries; however, it can also be used to synthesize the hallucinogenic alkaloid, mescaline. It is found in gallnuts, sumac, witch hazel, tea leaves, oak bark, and other plants. [START_REF]Ulmann's Encyclopedia of Industrial Chemistry[END_REF] Transcinnamic acid is used in flavors, synthetic indigo, and certain pharmaceuticals, but its primary use is in the manufacture of the methyl, ethyl, and benzyl esters for the perfume industry. It occurs naturally in essential oils (for example, those from cassia and cinnamon) and can also be found in shea butter. In addition, it can be obtained from some balsams such as storax. [START_REF]Ulmann's Encyclopedia of Industrial Chemistry[END_REF] Caffeic acid is found in many fruits, vegetables, seasonings, beverages (wine, coffee) and olive oil. [START_REF] Saija | In vitro and in vivo evaluation of caffeic and ferulic acids as topical photoprotective agents[END_REF] It is highly predominant in sunflower seeds and greatly affects the solubility of plant proteins. [START_REF] Chen | Antioxidant activities of caffeic acid and its related hydroxycinnamic acid compounds[END_REF] Ferulic acid is a highly abundant phenolic phytochemical present in plant cell walls, and it is found in vegetables and fruits, and in seeds of plants such as brown rice, whole wheat, and oats, as well as in wine, olive oil, coffee, apple, artichoke, peanut, orange, and pineapple. It is the most abundant hydroxycinnamic acid in plants. It may have significant health benefits, through its antioxidant and anticancer activity, and it may also serve as a precursor in the manufacture of useful aromatic compounds. [START_REF] Mathew | Ferulic acid: An antioxidant found naturally in plant cell walls and feruloyl esterases involved in its release and their applications[END_REF] The work presented here is part of a project concerning the solubility of natural phenolic compounds in water and organic solvents. Its final aim is to understand and predict the solubility of these molecules in different solvents, to adequately understand the fate and separation processes of these molecules. To accomplish this, experimental measurements of solubility, as a function of temperature, were performed and an associating equation of state (EoS) was used for their estimation.

Experimental Section

Materials.

In all experiments, bidistilled water (≈ 2.5-3 µS/cm) was used. Gallic acid (g98% purity; CAS No. 149-91-7), salicylic acid (g99% purity: CAS No. 69-72-7), transcinnamic acid (g99% purity; CAS No. 140-10-3), ferulic acid (99% purity; CAS No. 537-98-4) and caffeic acid (98% purity; CAS No. 331-39-5) were obtained from Sigma-Aldrich. All chemicals were used as received.

2.2. Experimental Procedure. 2.2.1. Solubility Measurements. Aqueous solubilities of the solid phenolic compounds were determined, as a function of temperature, by the analytical shake-flask method, using constant-temperature jacketed glass cells. To prepare saturated solutions, excess solute and distilled water were placed into the cell and stirred in a plate stirrer (IKA RO 15 Power). Temperature was maintained with a constanttemperature ((0.10 K) circulating water bath (Grant LTC1) and monitored with four-wire platinum resistance probes (Pt100) placed in the thermostatic jackets and connected to an Agilent 34970A data acquisition unit ((0.01 K). This temperature The time required to reach equilibrium was determined by repetitively measuring the solubilities until constant data were obtained, which was different for each compound, as presented in Table 1. When the equilibrium was attained, the agitation was stopped and then the solution was kept still, to allow undissolved solid to settle. Settling times are also presented in Table 1.

For the determination of the solubilities, samples were taken using plastic syringes coupled with syringe filters (0.45 µm), previously thermostatized at the equilibrium temperature and analyzed by spectrophotometry (Thermo Electron Corporation UV1).

Calibration curves were generated using standards in the following concentration ranges (in mg/L): salicylic acid, 2.7-4.8; gallic acid, 0.5-2.5; trans-cinnamic acid, 0.3-1.5; caffeic acid, 1.3-6.7; and ferulic acid, 0.5-9.4. The analysis wavelengths are presented in Table 1. Linearity was evaluated by linear regression analysis.

All the weighing procedures were performed using an analytical balance (model AE aaa250L) ((0.0001 g).

To verify the accuracy of the spectrophotometric method, some analyses were performed using the gravimetric method, using the same solubilization and sampling procedure. A sample of clear solution was transferred to a preweighed flask (with a mass m v ). The mass of the flask with saturated solution (with a mass m vs ) was registered and then maintained at 313.15 K for 8 days on a hot plate (Stuart Hotplate SB300) to allow the solvent to evaporate. Afterward, the solid residue was placed in a drying oven (Scientific Series 9000) and the mass of dry residue was measured (m vdr ). Before each weighing, the flasks were placed in a desiccator for 2 h to reach ambient temperature. The solubility, when presented as a ratio of grams of solute per grams of solvent, is calculated using the following equation:

S ) m vdr -m v m vs -m vdr (1)
Some phenylpropenoic acids were reported to suffer photodegradation with daylight. [START_REF] Abdelmoty | Polymorphism of cinnamic and alpha-truxillic acids: New additions to an old story[END_REF][12][START_REF] Cohen | Photochemistry of Trans-Cinnamic Acids[END_REF][START_REF] Muthuramu | Photo-Dimerization of Coumarin in Aqueous and Micellar Media[END_REF][START_REF] Russell | Effect of substitution on the 2 + 2 cycloaddition reaction of phenylpropanoids[END_REF][START_REF] Pattabiraman | Regioselective photodimerization of cinnamic acids in water: Templation with cucurbiturils[END_REF] To prevent that, the cells were isolated with aluminum foil. In these cases, apart from the normal analysis, the phenylpropenoic samples were also analyzed via high-preformance liquid chromatography (HPLC) (VWR-Hitachi Lachrom Elite HPLC) at the same wavelength used in the spectrophotometric method to check if any photodegradation had occurred, which was never found, with a single peak being obtained in all chromatograms.

2.2.2. DSC Measurements. Thermograms of the studied phenolics were obtained using differential scanning calorimetry (DSC) (Netzsch, model DSC 200 F3 Maia). Samples of 4-6 mg were sealed hermetically into aluminum crucibles and heated under a stream of nitrogen in the measuring cell, while using an empty crucible as a reference. Initial estimates of the melting temperatures were obtained from a 10 K/min temperature program in a larger temperature range (from room temperature to 623 K). Afterward, several runs were averaged with a heating rate of 1 K/min around the expected melting temperature. All DSC experiments were conducted at least in triplicate. An indium standard was run on this equipment to ascertain its precision for obtaining melting temperatures and heats of fusion. The deviation in the melting temperature was -0.3%, and in fusion enthalpy, it was 3.7%.

Potentiometric Measurements.

Because the phenolic compounds under study are organic acids, the pH of the saturated aqueous solutions and their acid dissociation constants were determined by potentiometry with a glass electrode in a VWR model SympHony SB70P meter with automatic temperature compensation. The measuring system was calibrated with buffers of pH 4.00 and 7.00 (293.15 K) from Merck, which were traceable to standard reference materials from NIST.

Apparent acid dissociation constants were obtained by potentiometric titration of aqueous solutions of the phenolic acids with aqueous NaOH solutions, at the same temperatures as in the solubilization experiments.

Model

Apart from empirical correlations, different thermodynamic approaches can be found for modeling the solid solubility of phenolic compounds in aqueous systems.

Jia et al. [START_REF] Jia | The effect of temperature on the solubility of benzoic acid derivatives in water[END_REF] determined the water solubility of benzoic acid and some of its derivatives, modeling the data with the Wilson equation and the λH correlation. Noubigh et al. [START_REF] Noubigh | Temperature and salt addition effects on the solubility behaviour of some phenolic compounds in water[END_REF] employed Van't Hoff plots to correlate the solubility of syringic, gallic, ferulic vanillic, and protocatechuic acids, as well as vanillin in water and aqueous salt solutions. Nordstro ¨m and Rasmuson [START_REF] Nordstrom | Solubility and melting properties of salicylic acid[END_REF][START_REF] Nordstrom | Polymorphism and thermodynamics of m-hydroxybenzoic acid[END_REF][START_REF] Nordstrom | Phase equilibria and thermodynamics of p-hydroxybenzoic acid[END_REF] studied the solubility of the three isomers of hydroxybenzoic acid in water and different organic solvents and, as Noubigh et al. [START_REF] Noubigh | Temperature and salt addition effects on the solubility behaviour of some phenolic compounds in water[END_REF] correlated the corresponding Van't Hoff plots, this time using a second-order polynomial. Lu and Lu [START_REF] Lu | Solubilities of gallic acid and its esters in water[END_REF] measured the water solubility of gallic acid and some of its esters and also correlated the data from the Van't Hoff plots. They proposed another empirical correlation to describe the S-type curve of the solubility of gallic acid and octyl gallate as a function of temperature. Panteli et al. [START_REF] Panteli | Measurement and Thermodynamic Modelling of Solubilities of Antioxidants in Organic Solvents[END_REF] modeled the solubility of (+)catechin in ethyl acetate and acetonitrile with UNIQUAC, NRTL, UNIFAC, and Regular Solution Theory and concluded that, for these particular systems, NRTL was the better correlative tool, whereas UNIFAC and regular solution theory gave unsatisfactory solubility predictions.

Besides the importance of aqueous systems and ambient pressure conditions, most of the works concerning the estimation of the phase equilibria of phenolic compounds is concerned with their CO 2 solubility, related with supercritical fluid extraction processes. [START_REF] Bertakis | Measurement and thermodynamic modeling of solid-liquid-gas equilibrium of some organic compounds in the presence of CO2[END_REF][START_REF] Diaz-Reinoso | Supercritical CO 2 extraction and purification of compounds with antioxidant activity[END_REF][START_REF] Chafer | Solubility of the natural antioxidant gallic acid in supercritical CO 2 + ethanol as a cosolvent[END_REF][START_REF] Murga | Solubility of three hydroxycinnamic acids in supercritical carbon dioxide[END_REF][START_REF] Saldana | Determination of vapor pressure and solubility correlation of phenolic compounds in supercritical CO 2[END_REF][START_REF] Fornari | A new development in the application of the group contribution associating equation of state to model solid solubilities of phenolic compounds in SC-CO 2[END_REF][START_REF] Yang | Modeling of the solubility of aromatic compounds in supercritical carbon dioxide-cosolvent systems using SAFT equation of state[END_REF] Among the models used, different classical equations of state such as the Peng-Robinson equation of state (PR-EoS), [START_REF] Bertakis | Measurement and thermodynamic modeling of solid-liquid-gas equilibrium of some organic compounds in the presence of CO2[END_REF][START_REF] Murga | Solubility of three hydroxycinnamic acids in supercritical carbon dioxide[END_REF] the Soave-Redlich-Kwong equation of state (SRK-EoS), [START_REF] Chafer | Solubility of the natural antioxidant gallic acid in supercritical CO 2 + ethanol as a cosolvent[END_REF] the group contribution associating equation of state (GCA-EoS) [START_REF] Chafer | Solubility of the natural antioxidant gallic acid in supercritical CO 2 + ethanol as a cosolvent[END_REF][START_REF] Fornari | A new development in the application of the group contribution associating equation of state to model solid solubilities of phenolic compounds in SC-CO 2[END_REF] and the statistical associating fluid theory equation of state (SAFT-EoS) [START_REF] Yang | Modeling of the solubility of aromatic compounds in supercritical carbon dioxide-cosolvent systems using SAFT equation of state[END_REF] can be found. A recent and broad revision on the different models used to predict solubility of phenolic compounds in supercritical CO 2 is available in the literature. [START_REF] Diaz-Reinoso | Supercritical CO 2 extraction and purification of compounds with antioxidant activity[END_REF] As both the solutes and the solvent (water) can associate (selfand cross-association is present in mixtures), thermodynamic models explicitly dealing with these interactions seem appropriate for evaluation in this work. Among the different associating models, recent works on the mutual solubilities of water + organic systems with the cubic-plus-association equation of state (CPA-EoS) [START_REF] Oliveira | Modeling the liquid-liquid equilibria of water plus fluorocarbons with the cubic-plus-association equation of state[END_REF][START_REF] Oliveira | Mutual solubilities of hydrocarbons and water with the CPA EoS[END_REF] suggested the choice of this equation of state.

The CPA-EoS, in terms of the compressibility factor (Z), can be expressed as the sum of two contributions: one accounting for physical interactions (that in the current work is taken as the SRK-EoS), and another accounting for the association contributions (the Wertheim association term): [START_REF] Kontogeorgis | An equation of state for associating fluids[END_REF][START_REF] Kontogeorgis | Multicomponent phase equilibrium calculations for watermethanol-alkane mixtures[END_REF][START_REF] Derawi | Application of the CPA equation of state to organic acids[END_REF][START_REF] Folas | Recent applications of the cubic-plus-association (CPA) equation of state to industrially important systems[END_REF][START_REF] Kontogeorgis | Ten years with the CPA (Cubic-Plus-Association) equation of state. Part 1. Pure compounds and self-associating systems[END_REF][START_REF] Kontogeorgis | Ten years with the CPA (Cubic-Plus-Association) equation of state. Part 2. Cross-associating and multicomponent systems[END_REF]

Z ) Z phys + Z assoc ) 1 1 -bF - aF RT(1 + bF) - 1 2 ( 1 + F ∂lng ∂F ) ∑ i x i∑ A i (1 -X A ) (2)
where a is the energy parameter, b the co-volume parameter, F the density, g the simplified radial distribution function (see eq 7, presented later in this paper), X Ai the mole fraction of pure component i not bonded at site A, and x i the mole fraction of component i.

The pure component energy parameter is obtained from

a(T) ) a 0 [1 + c 1 (1 -√ T r )] 2 (3)
and the mixture of a and b parameters are obtained through the classical mixing and combining rules. [START_REF] Oliveira | Mutual solubilities of hydrocarbons and water with the CPA EoS[END_REF] X Ai is related to the association strength ∆ AiBj between two sites belonging to two different molecules and is calculated by solving the following set of equations:

X A i ) 1 1 + F ∑ j x j∑ B j X Bj ∆ A i B j (4)
For self-associating molecules, eq 5 is used:

∆ A i B i ) g(F) [ exp ( ε A i B i RT ) -1 ] b ii β A i B i (5)
where ε A i B i and β A i B i are the association energy and the association volume, respectively. For sites that belong to two different associating molecules, the Elliot combining rule is used:

∆ A i B j ) √∆ A i B i × ∆ A j B j (6)
The simplified radial distribution function, g(F) is given by

g(F) ) 1 1 -1.9η (7)
where

η ) ( 1 4 ) bF
CPA has three pure component parameters (a 0 , c 1 , and b) for nonassociating components, whereas, for associating components such as n-alcohols, it has five (a 0 , c 1 , b, ε, and β). In both cases, parameters are regressed simultaneously from vapor pressure and liquid density data.

Starting from the solid-liquid equilibrium conditions, neglecting the heat capacity terms and assuming pure solid phases, the water solubility of each of the phenolic compounds under study can be obtained from the relation 38

x i ) φ i L 0 φ i L exp [ -∆H i fus RT ( 1 -T T i m )] (8)
where x i is the solubility (in mole fraction), φ the fugacity coefficient, and ∆H ifus the fusion enthalpy; the subscript "0" refers to the pure component and the subscript "m" refers to melting. Fugacity coefficients were obtained from the CPA-EoS. Fusion enthalpies and melting temperatures were experimentally determined by DSC, whenever possible; otherwise, they were obtained from the Marrero and Gani group contribution approach. [START_REF] Marrero | Group-contribution based estimation of pure component properties[END_REF]

Results and Discussion

Aqueous solubility results are given in Tables 2 and3. The pH of each saturated solution was also measured, and the mean values are presented on the same tables. As can be seen, the solubility increases with temperature for all compounds.

Among the hydroxybenzoic acids (Table 2), gallic acid has the higher solubilities, because of the higher number of hydroxyl groups, leading to increased association interactions with water. For gallic acid, a comparison of our results with those of Lu and Lu 9 (Figure 2) allows one to observe a slight disagreement at the lower temperatures (298.15 and 303.15 K); however, generally, the results follow the expected trend in the entire temperature range. Comparison with Noubigh et al. [START_REF] Noubigh | Temperature and salt addition effects on the solubility behaviour of some phenolic compounds in water[END_REF] shows that the results are not in good agreement, but the Noubigh et al. [START_REF] Noubigh | Temperature and salt addition effects on the solubility behaviour of some phenolic compounds in water[END_REF] data do not increase significantly with temperature, which is contrary to what would be expected. To verify the reproducibility of the spectrophotometric determination, a gallic acid For salicylic acid, our results are in good agreement with previous work, [START_REF] Nordstrom | Solubility and melting properties of salicylic acid[END_REF][START_REF] Apelblat | Solubility of Ascorbic, 2-Furancarboxylic, Glutaric, Pimelic, Salicyclic, and O-Phthalic Acids in Water from 279[END_REF][START_REF] Pena | Solubility and phase separation of benzocaine and salicyclic acid in 1,4-dioxane-water mixtures at several temperatures[END_REF] as can be seen in Figure 2.

No literature data were found for caffeic acid to compare the results. For ferulic acid, the results are very different from those obtained by Noubigh et al. [START_REF] Noubigh | Temperature and salt addition effects on the solubility behaviour of some phenolic compounds in water[END_REF] (negative deviations between 40% and 63%, being larger at the higher temperatures), opposite to those found previously for our gallic acid measurements. These authors also used the shake flask method but reported equilibrium times of 3 h of stirring, while different equilibrium times for different temperatures were achieved in this work, and they were always between 64 h and 84 h, as reported in Table 1. Ferulic acid is reported to have two polymorphic forms [START_REF] Sohn | Characterization of physicochemical properties of ferulic acid[END_REF] that were obtained by recrystallization from various solvents. Sohn and Oh 42 noticed in DSC thermograms that form I has one endothermic peak, corresponding to melting at 448.15 K, whereas form II has three endothermic peaks, at 378.15, 421.15, and 455.15 K. The equilibrium solubility of form I is approximately twice that of form II, and the dissolution rate of form II is higher than that of form I in an initial step. [START_REF] Sohn | Characterization of physicochemical properties of ferulic acid[END_REF] Thus, we may be comparing different crystalline structures of the same compound, which might explain the different results. As can be seen from Table 4, our solubility measurements correspond to form I, with a single solid phase, melting at 444.65 K. Also for transcinnamic acid, no further temperature-dependent solubility data were found, except for a single point at 293.15 K that was available in the Merck Index (0.4 g/L). [START_REF] O'neil | The Merck Index: an encyclopedia of chemicals, drugs and biologicals[END_REF] In the series of phenylpropenoic compounds, higher solubilities for caffeic acid, followed by ferulic acid and trans-cinnamic acid, were obtained. As observed previously in the studied hydroxybenzoic acids, this order follows the extent of hydrogen bonding of each molecule with water, because caffeic acid has two hydroxyl groups, ferulic one, and trans-cinnamic acid none.

Another important fact about the solubilization of solids in liquids is related to fusion enthalpies and melting temperatures. These data have been determined by DSC and are presented in Table 4, where the number of repetitions (scans) is reported. For all the phenolic acids, except gallic and caffeic acids, a linear baseline and a single peak was observed. The melting peaks of gallic and caffeic acids are not symmetric, with other peaks continuously connected to the melting peak, indicating that decomposition processes may occur upon melting. For this reason, the fusion enthalpies of these two acids could not be determined and, therefore, they are not presented in Table 4.

Data regarding the melting temperature and fusion enthalpies were found for salicylic acid [START_REF] Nordstrom | Solubility and melting properties of salicylic acid[END_REF][START_REF]BYU DIPPR 801 Thermophysical Properties Database[END_REF]45 and trans-cinnamic acid. [START_REF]BYU DIPPR 801 Thermophysical Properties Database[END_REF]45 The measured melting temperatures deviate by less than -0.2%, in comparison with all the collected data. Higher deviations were found for the fusion enthalpies, deviations smaller than 1.8% for trans-cinnamic acid, and a range from -0.7% up to 24% for salicylic acid (respectively, versus the data from the NIST Chemistry Webbook and the DIPPR database). Our salicylic acid fusion enthalpy deviates -9.8% from that reported recently by Nordstrom and Rasmuson. [START_REF] Nordstrom | Solubility and melting properties of salicylic acid[END_REF] From the data presented in Table 4 and the measured solubility data presented in Tables 2 and3, it can thus be concluded that the extent of hydrogen bonding with water is certainly more important than the fusion process for determining the degree of solubilization.

Because the studied phenolic compounds are organic acids that dissociate in water, the apparent dissociation constants (K aapp ) were also obtained from potentiometric titration of solutions of each phenolic acid with NaOH:

K a app ) [A -] (H + ) [HA] ( 9
)
where activities are represented by parentheses and concentrations are represented by square brackets. The H + activity is obtained from the pH meter, and K aapp is given by the measured pH where the added volume of NaOH is equal to the equivalent volume, divided by 2. Although this is not the true thermodynamic dissociation constant (that would require activity coef- ficients of the other species), it is an appropriate quantity for the purposes of this work. Measured K aapp are presented in Table 5. Although measurements were performed at the highest temperature (323.15 K), lowest temperature (288.15 K), and room temperature (298.15 K), all the results indicate that no significant temperature change is obtained in the studied temperature range. Hydroxybenzoic acids are more acidic than phenylpropenoic acids, with salicylic acid being the most acidic of all the studied compounds. Curiously, among the phenylpropenoic acids, the opposite trend found in solubility is obtained in terms of acidity with the less-soluble trans-cinnamic acid being the most acidic of the three, which leads to the conclusion that acid dissociation is not as important of an issue in the water solubilization of these molecules. CPA parameters are frequently obtained from purecomponent vapor-pressure and liquid-density data. In the particular case of the phenolics studied in this work, these data are available only for salicylic and trans-cinnamic acids. Another major drawback is the presence of different associating groups (-COOH, -OH) on the same molecule, some of them repeated in different positions (such as -OH). Thus, it seems appropriate to individually assign association energies and volumes for each group and to consider multiple group substitution, either or not with the same association parameters. Thus, in the remaining of this work, eq 5 will be used to obtain the association strengths (∆) of the associating sites tha belong to the same association group (with the same and β), whereas eq 6 will be used in the other cases, those either being self-or cross-association.

To address this condition, which is explicitly considered in the association contribution for the Helmholtz energy, a database of phenolic compounds was taken with single and multiple substitution of associating groups, namely, phenol, o-benzenediol, m-benzenediol, p-benzenediol, benzoic acid, salicylic acid, and trans-cinnamic acid. All the required data were collected from the DIPPR database. 45 With this database, it was possible to compare the different cubic and associating CPA parameters and thus establish (1) correlations for the cubic parameters and (2) values for the association energies and volumes of carboxylic groups in benzoic and phenylpropenoic acid derivatives, and hydroxyl groups in different positions of the aromatic ring.

The following procedure was established (water CPA parameters were obtained from a previous work: 31 a 0 ) 0.12 Pa m 6 /mol 2 , c 1 ) 0.67, b ) 1.45 × 10 -5 m 3 /mol, ε ) 16555 J/mol, β ) 0.069, four-site model). First, critical temperatures (and pressures) were obtained from a third-order group contribution approach that was proposed by Marrero and Gani [START_REF] Marrero | Group-contribution based estimation of pure component properties[END_REF] (see Table 6). This approach has the advantage of clearly distinguishing among isomers, and it was used previously by Nikitin and co-workers to evaluate alkylbenzoates, [START_REF] Nikitin | Critical point measurements for n-alkyl benzoates (C-8 to C-13)[END_REF] phenylalkanoic acids, [START_REF] Nikitin | Critical-point measurements for phenylethanoic to 7-phenylheptanoic acids[END_REF] phthalates, [START_REF] Nikitin | Critical temperatures and pressures of 12 phthalates using the pulse-heating method[END_REF] phenol, and phenylalkanols. [START_REF] Nikitin | Vapor-liquid critical properties of phenol and (C-8 to C-10) phenylalkanols[END_REF] In a first approach, the previously mentioned database was used with these critical temperatures and all parameters were regressed simultaneously. In a following step, the obtained cubic parameters were regressed with T c 2 /p c (a 0 ), T c (c 1 ), and the van der Waals volume vdWV (b), obtained from Bondi's group contribution approach. [START_REF] Bondi | Der Waals Volumes + Radii[END_REF] The following equations were determined to provide an adequate linear fit of the cubic parameters (Figure 3):

a 0 ) 0.2267 + 24.38 ( T c p c ) (10)
c 1 ) -3.557 + ( 6.289 × 10 -3 ) T c (11) b ) -2.328 × 10 -5 + 1.884(vdWV) (12) where T c is given in Kelvin, p c is given in pascals, and vdWV is expressed in terms of m 3 /mol. These cubic parameters were then fixed and the associating parameters regressed separately (Table 6). For each -OH group, a two-site model was adopted, whereas for the -COOH groups, a single site was considered. It can be seen, in the case of the -OH groups, with the exception of the molecules where intramolecular hydrogen bonding occurs (salicylic acid and o-benzenediol), that the same association energy was obtained (1.8 × 10 4 J/mol). Whenever intramolecular hydrogen bonding occurs, these values are slightly lower, as expected. The values of the association volume (β) are more dependent on the substitution of the ring. In the case of the acid groups, both the association energies and volumes are considerably dependent, whether there is a ring substitution with or without intramolecular association (salicylic acid and benzoic acid) or a phenylpropenoic substitution. From the aforementioned methodology, all the cubic and associating parameters of the other molecules under study (gallic, ferulic and caffeic acids) can be estimated, without requiring experimental pure-component data.

For the modeling of the water solubility, eq 8 was used with the experimentally determined melting temperatures and fusion enthalpies and the CPA-EoS for the calculation of the fugacity coefficients. For gallic and caffeic acids, melting temperatures and fusion enthalpies were obtained from the same group contribution approach used previously for the critical properties (see Table 4). [START_REF] Marrero | Group-contribution based estimation of pure component properties[END_REF] These molecules are organic acids; therefore, to effectively model their water solubility, the acid dissociation must be considered. Because these are weak acids, the adopted procedure was to determine the solubility of the undissociated form, as mentioned previously, and then obtain the total solubility as the sum of this with the amount of the basic form. The latter is obtained from the solubility of the acidic form, given by the model, the K aapp value, and the pH value (see Tables 2,3, and 5). Solid-liquid equilibria results are presented in Figures 4 and5. The solubilities of benzoic acid and phenol are also plotted in these figures. As can be seen, a good description can be achieved with the CPA-EoS, but a small k ij value is always required to obtain smaller deviations. Still, in all cases, only a single, temperature-independent k ij value was required, and this was almost constant for the majority of the studied systems, except the gallic, ferulic, and caffeic acids, which required a slightly higher k ij value (close to 0.1). These three phenolics are those for which pure-component data were not available to correlate the CPA parameters, and thus these higher deviations certainly account for the predictive approach proposed for the EoS parameters. Still, it can be concluded that the CPA-EoS, together with the selected methodology for the estimation of the pure-component parameters, is an adequate tool for the estimation of the solid solubility of phenolic compounds.

Conclusions

In this work, aqueous solubility data were measured for some phenolic compounds (salicylic, trans-cinnamic, gallic, caffeic, and ferulic acids) in pure water in the temperature range from 288.15 K to 323.15 K, using the shake-flask method, coupled with spectrophotometric analysis. For all compounds under study, the solubility follows the general increasing trend with temperature. The studied compounds can be divided in two groups: hydroxybenzoic acids (gallic and salicylic acids), which have the higher solubilities, and phenylpropenoic (trans-cinnamic, caffeic, and ferulic acids), which present the lower solubilities.

Besides solubility data, melting temperatures, enthalpies of fusion, and apparent dissociation constants were also determined, providing a broader knowledge about the solubilization processes of these molecules. From all the measurements, it was concluded that the extent of solubilization is mostly governed by the degree of hydrogen bonding formation with water.

The cubic-plus-association equation of state (CPA-EoS) was used with different association energies and volumes for the different associating groups for the first time. A methodology to obtain the pure-component parameters without requiring experimental data was developed, and the results indicate that CPA is an adequate tool to correlate the water solubility of these molecules.

Nomenclature

Literature Cited

Figure 1 .

 1 Figure 1. Structure of the five phenolic compounds in this study.

Figure 2 . 40 ( 0 )

 2400 Figure 2. Solubility of hydroxybenzoic acids in water. Closed symbols denote data from this work ((b) gallic acid and (9) salicylic acid); open symbols denote data obtained from previous literature ((4) Apelblat et al., 40 (0) Nordstrom and Rasmuson, 19 (x) Pena et al., 41 (]) Lu and Lu, 9 and (!) Noubigh et al. 18 ).

Figure 3 .

 3 Figure 3. Regression of the CPA cubic term parameters: (i) a0 vs Tc 2 / p c, (ii) c1 vs Tc, and (iii) b vs vdWV.

Figure 4 .

 4 Figure 4. Solubility of phenol, gallic acid, and salicylic acid in water. Symbols represent experimental data: (O) phenol, (4) gallic acid, and (0) salicylic acid. Modeling results: phenol, kij ) -0.02; gallic acid, kij ) -0.13; and salicylic acid, k ij ) -0.06.

Figure 5 .

 5 Figure 5. Solubility of benzoic, ferulic, caffeic, and trans-cinnamic acids in water. Symbols represent experimental data: (O) benzoic acid, (4) caffeic acid, (0) trans-cinnamic acid, and (x) ferulic acid. Modeling results: benzoic acid, kij ) -0.04; ferulic acid, kij ) -0.13; caffeic acid, kij ) -0.10; and trans-cinnamic acid, k ij ) -0.05.

  List ofSymbols a 0 ) parameter in the cubic term of CPA (Pa m 6 /mol 2 ) b ) co-volume parameter in CPA (m 3 /mol) c 1 ) parameter in the cubic term of CPA CPA ) cubic plus association EoS ) equation of state g ) simplified radial distribution function m ) mass (g) p ) pressure (Pa) K aapp ) apparent acid dissociation constant R ) gas constant; R ) 8.314 J/(mol K) S ) solubility T ) absolute temperature (K) vdWV ) van der Waals volume (m 3 /mol) x ) mole fraction X Ai ) fraction of molecule i not bonded at site A Z ) compressibility factor Greek Symbols ) association energy (J/mol) β ) association volume ∆ AiBj ) association strength between site A on molecule i and site B on molecule j ∆H fus ) fusion enthalpy (J/mol) F ) mole density (mol/m 3 ) η ) reduced fluid density φ ) fugacity coefficient Subscripts c ) critical i, j ) component index m ) melting r ) reduced v ) vial vs ) vial with saturated solution vdr ) vial with dry residue Superscripts assoc ) association contribution phys ) physical contribution Acknowledgment The auhors are grateful for the support provided by Fundac ¸a ˜o para a Cie ˆncia e a Tecnologia (FCT), through project POCI/ EQU/58239/2004 ("Equilibrium and Interfacial Properties of Water + Oil Systems"), and financial support by LSRE FEDER/ POCI/2010. F. L. M. acknowledges her FCT Ph.D. scholarship (No. SFRH/BD/32372/2006) and A. J. Q. acknowledges financial support from POCI/N010/2006. Prof. Luis M. N. B. F. Santos is acknowledged for the interpretation of some of the DSC thermograms.

Table 1 . Shaking and Settling Times and Absorption Wavelengths for the Solubility Measurements

 1

		shaking time (h)	settling time (h)	λ (nm)
	salicylic acid	64-156	7-26	295.6
	gallic acid	64-117	7-26	270.0
	trans-cinnamic acid	69-156	6-42	270.8
	ferulic acid	64-84	24-75	290.0
	caffeic acid	64-84	25-73	290.0

Table 2 . Experimental Solubilities of Hydroxybenzoic Acids in Pure Water and Corresponding Saturation pH Values

 2

	ohio1/yie-yie/yie-yie/yie99907/yie3699d07z xppws 23:ver.3 4/30/08 12:03 Msc: ie-2007-01452o TEID: jsc00 BATID: ie83a20
	D Ind. Eng. Chem. Res., Vol. xxx, No. xx, XXXX
		Salicylic Acid	Gallic Acid
	temperature,	solubility,		solubility,
	T (K)	S (g/L)	pH	S (g/L)	pH
	323	5.97 ( 0.30	2.23	38.9 ( 2.1	2.56
	313	4.55 ( 0.51	2.24	26.4 ( 1.2, 22.5 ( 0.62 a	2.63
	303	2.18 ( 0.46	2.30	18.6 ( 0.9	2.67
	298	1.14 ( 0.44	2.50	14.7 ( 0.8	2.69
	288	1.00 ( 0.05	2.61	9.1 ( 0.7	2.72
	a The second value listed was measured by gravimetry.
	206
	207
	208
	209
	210
	211
	212
	213
	214
	215
	216
	217
	218
	219
	220
	221
	222
	223
	224
	225
	226
	227
	228
	229
	230
	231
	232
	233
	234
	235
	236
	237
	238
	239
	240
	241
	242
	243
	244
	245
	246
	247
	248
	249
	250
	251
	252

Table 3 . Experimental Solubilities of Phenilpropenoic Acids in Pure Water and Corresponding Saturation pH Values

 3

		Trans-Cinnamic Acid		Ferulic Acid		Caffeic Acid
	temperature, T (K)	solubility, S (g/L)	pH	solubility, S (g/L)	pH	solubility, S (g/L)	pH
	323	0.85 ( 0.02	3.26	2.19 ( 0.03	3.18	2.92 ( 0.03	3.17
	313	0.63 ( 0.02	3.37	1.76 ( 0.02	3.31	2.04 ( 0.02	3.25
	303	0.31 ( 0.01	3.56	0.92 ( 0.01, 0.95 ( 0.01 a	3.40	1.23 ( 0.02	3.34
	298	0.23 ( 0.01, 0.20 ( 0.01 a	3.60	0.78 ( 0.01	3.46	0.98 ( 0.02	3.37
	288	0.21 ( 0.01	3.65	0.57 ( 0.01	3.54	0.55 ( 0.01	3.42

a The second value listed was measured by gravimetry.

Table 4 . Average Melting Point and Enthalpy of Fusion of the Studied Phenolic Acids and Comparison with the Selected Group Contribution Model

 4

			DSC		Marrero and Gani Group Contribution Method 39
	compound	number of scans	T m (K)	∆H fus (J/mol)	T m (K)	∆H fus (J/mol)
	salicylic acid	7	431.3	24448	436.3	26590
	gallic acid	4	535.4	ND a	499.0	38770
	trans-cinnamic acid	8	406.1	22214	382.1	23290
	ferulic acid	7	444.6	33338	442.6	32960
	caffeic acid	ND a	ND a	ND a	464.1	39850

a Not determined, because of decomposition upon melting.

Table 5 . Apparent Acid Dissociation Constants of the Studied Phenolic Acids at 298.2 K

 5

	phenolic acid	pKa app
	salicylic acid	3.21
	gallic acid	4.51
	trans-cinnamic acid	4.55
	ferulic acid	4.61
	caffeic acid	4.77

Table 6 . Critical Properties (Marrero and Gani), 39 vdWV (Bondi) 50 CPA Associating Pure-Component Parameters (Cubic Parameters Obtained from Eqs 10-12) and Deviations in the Estimation of Vapor Pressure and Liquid Densities

 6

	COOH	OH	% AAD

ohio1/yie-yie/yie-yie/yie99907/yie3699d07z xppws 23:ver.3 4/30/08 12:03 Msc: ie-2007-01452o TEID: jsc00 BATID: ie83a20


Michalis Agathocleous
email: michalis.agathocleous.09@ucl.ac.uk

Georgia Christodoulou

Vasilis Promponas

Chris C Christodoulou

Vassilis Vassiliades

Antonis Antoniou
email: a.antoniou@cs.ucy.ac.cy

M Ichalis Agathocleous

V Assilis V Assiliades

Protein Secondary Structure Prediction with Bidirectional Recurrent Neural Nets: Can Weight Updating for Each Residue Enhance Performance?

Keywords: Protein Secondary Structure Prediction, Bidirectional Recurrent Neural Networks, Bioinformatics and Computational Biology

ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Protein Secondary Structure Prediction with Bidirectional Recurrent Neural Nets: can weight updating for each residue enhance performance? [START_REF] Baldi | Exploiting the past and the future in protein secondary structure prediction[END_REF]

Introduction

Proteins are linear polymers of amino acids, and their complement in any living cell provides a vast repertoire of functions required for maintaining life. Their functionality is determined by the detailed three dimensional arrangement of their constituent atoms, which specifies the shape of the molecule and the ways it can potentially interact with other biomolecules. Biochemical methods for experimental determination of protein tertiary structure are expensive, time consuming and frequently ineffective. On the contrary, modern high-throughput molecular biology techniques provide us daily with an increasing number of (putative) amino acid sequences, corresponding to proteins of unknown structure and function. It is widely accepted that protein sequence determines its 3D structure [START_REF] Anfinsen | Studies on the principles that govern the folding of protein chains[END_REF]; however, the exact mapping from the 1D to the 3D space has not yet been elucidated. Nevertheless, knowledge of local regular arrangements of amino acid residues (secondary structural elements), may indicate the structure adopted by a protein chain and provide useful constraints for further structural modelling.

The protein secondary structure prediction (PSSP) problem has been addressed with different computational approaches during the last four decades. Mainly due to the lack of necessary volumes of structural data, only relatively recently were machine learning methods (including artificial neural networks -NNs) introduced to this field. Such methods have resulted in considerably higher predictive performance compared to preceding empirical or statistical methods, with per residue accuracies (Q3 [START_REF] Richards | Identification of structural motifs from proteins coordinate data: Secondary structure and first-level supersecondary structure[END_REF]) ranging from 63% to 76% [START_REF] Pollastri | Improving the prediction of protein secondary structure in three and eight classes using recurrent neural networks and profiles[END_REF], [START_REF] Rost | EVA: large-scale analysis of secondary structure prediction[END_REF]. In particular, a fully connected feed forward NN and a single hidden layer, with a local input window of 13 amino acids using orthogonal (or one-hot) encoding has been applied by Qian and Sejnowski [START_REF] Qian | Predicting the secondary structure of globular proteins using neural network models[END_REF] reaching accuracy performance of Q3 = 64.3%. Based upon this work, Rost and Sander developed the so-called PHD method [START_REF] Rost | Improved prediction of protein secondary structure by use of sequence profiles and neural networks[END_REF], [START_REF] Rost | Combining evolutionary information and neural networks to predict protein secondary structure[END_REF], which used various machine learning methods including early stopping and ensemble of averages of different NNs. Their most substantial improvement was, however, the use of multiple sequence alignments (MSA) that enrich the input with evolutionary information [START_REF] Rost | Combining evolutionary information and neural networks to predict protein secondary structure[END_REF]. Through their work the accuracy performance reached to Q3 = 74%. Another interesting approach, similar to PHD, is the work of Salamov and Soloveyev [START_REF] Salamov | Protein secondary structure prediction using local alignments[END_REF] that introduced a variant of the nearestneighbor approach that could achieve similar accuracy (Q3 = 73.5%) using a single sequence as input. Cuff and Barton [START_REF] Cuff | Evaluation and improvement of multiple sequence for protein secondary structure prediction[END_REF] built another MSA-enabled feed forward NN (of a larger size however) with comparable accuracy.

A major breakthrough in the field of PSSP is considered to be the work of Baldi and colleagues [START_REF] Baldi | Exploiting the past and the future in protein secondary structure prediction[END_REF], who introduced a Bidirectional Recurrent Neural Network (BRNN). The main motivation of this work was the inability of feed forward NNs with a fixed-width sliding window to capture long-range dependencies. The aforementioned BRNN takes as input a fixed-width window centered each time at the residue of interest. However, it attempts to predict the secondary structure for the target residue by considering not only the local context (i.e., neighboring residues in the window) but also residues located on the left and right of the target residue [START_REF] Baldi | Exploiting the past and the future in protein secondary structure prediction[END_REF]. These bidirectional dynamics proved to be able to capture both upstream and downstream information. When the BRNN processed evolutionary information in the form of MSAs, the predictive performance reached up to Q3 = 73.6% [START_REF] Baldi | Exploiting the past and the future in protein secondary structure prediction[END_REF], which was increased to 75.1% on average, when an ensemble of 6 predictors was used. Along the same lines Chen and Chaudhari [START_REF] Chen | Cascaded bidirectional recurrent neural networks for protein secondary structure prediction[END_REF] developed a cascaded architecture, consisting of two BRNNs where the second network fil-ters the output of the first. As they report, their system achieves 74.38% (Q3) accuracy with a Segment Overlap (SOV [START_REF] Zemla | A modified definition of Sov, a segment-based measure for protein secondary structure prediction assessment[END_REF]) score of approximately 66%.

In the introductory work of BRNNs for PSSP [START_REF] Baldi | Exploiting the past and the future in protein secondary structure prediction[END_REF], for each position in a sequence presented to the network an input window is formed centered around this position. During the training phase, the protein is processed in its entirety before the weight updates are made. In this paper, our aim is to investigate how the prediction accuracy could be improved by: (i) updating the weights at every residue, which in a way constitutes a form of dynamic training and is more context-sensitive, and (ii) using different filtering approaches (a novel method based on Hidden Markov Models and a cascaded feed forward Artifical NN).

The rest of the paper is organised as follows: section 2 presents the methodology followed by the results and discussion in section 3; section 4 gives the conclusions and future work.

Methods

Data Collection and Preprocessing

Sequence Similarity-Based Redundancy Reduction: In order to train and validate the BRNN, we need a set of high quality data consisting of proteins with experimentally determined 3D structures deposited in the RSCB Protein Data Bank (PDB) (URL: http://www.pdb.org/, accessed 20 April 2009). Moreover, the resulting dataset should be maximal (in order to capture the knowledge we currently have available on protein structures) but also non-redundant, to avoid poor generalisation of the BRNN. For this purpose we utilised the PDB-Select25 dataset (URL: http://bioinfo.tg.fh-giessen.de/pdbselect/, accessed 20 April 2009), which is regularly produced by analysis of the PDB with the algorithm described in [START_REF] Uwe | Selection of representative protein data sets[END_REF]. This dataset contained 4019 polypeptide chains that shared less than 25% overall pair-wise sequence identity.

Data Selection Criteria: It is of great importance to choose a suitable data set so that the BRNN can be properly trained, and several criteria should be fulfilled for the selection process:

-We retained only entries determined by X-ray diffraction, for which we can apply the resolution as a quantitative selection measure. In particular, a 3.0 Å threshold was used to discard structures of insufficient/questionable quality. -We discarded entries with physical chain breaks, as empirically identified by at least one pair of successive C α atoms with a distance longer than 4.0 Å, as well as proteins with large segments of undefined secondary structure. -Chains with a length of less than 30 amino acids were discarded.

-DSSP [START_REF] Kabsch | Dictionary of protein secondary structure: pattern recognition of hydrogen-bonded and geometrical features[END_REF] (see below), should provide a valid output file for any chain retained in the dataset.

We started with a dataset containing a total of 4017 protein chains, of which 2656 corresponded to structures determined by X-ray crystallography. Following the above procedure we ended up with 612 protein chains.

Secondary Structure Assignment We have relied for secondary structure (SS) assignments on the widely used DSSP program [START_REF] Kabsch | Dictionary of protein secondary structure: pattern recognition of hydrogen-bonded and geometrical features[END_REF]. More specifically, we reduce class assignments from the eight secondary structure (SS) types provided by DSSP (i.e. α-helix (H) , 3 10 -helix (G) , π-helix (I) , β-strand (E) , β-bridge (B) , β-turn (T), bend (S) and 'other' ('.')) into three SS states (Helical: H, G; Extended: E, B; Random coil/Loop: I, T, S, '.'). From here onwards, we refer to these states as H, E and C respectively. DSSP results were fetched from the DSSP website (URL: http://swift.cmbi.kun.nl/gv/dssp/, accessed 20 April 2009) and transformed to the 3-state representation by an in-house parser. Since several protein chains contain segments of disordered regions where DSSP does not produce any output, for the purposes of this work we have decided to exclude any such entries.

Multiple Sequence Alignment Preprocessing MSAs have been shown to significantly increase protein secondary structure prediction accuracy in recent applications [START_REF] Rost | Combining evolutionary information and neural networks to predict protein secondary structure[END_REF]. This is because structure is considered to be more conserved than sequence [START_REF] Rost | Combining evolutionary information and neural networks to predict protein secondary structure[END_REF]. Every position within an alignment contains an evolutionary record. We encode each input residue with a 20-dimensional vector, where ordinates correspond to the frequencies of the different 20 amino acid residues at the respective column of the MSA. Apparently, encoding for single sequences at the input reduces to the orthogonal encoding scheme. For the polypeptide chains collected with the previously described procedure, we have utilised unweighted profiles available from the HSSP database [START_REF] Schneider | The HSSP database of protein structure-sequence alignments[END_REF].

Modified BRNN architecture and training procedure

A NN must accept the amino acid at its input with all the necessary information that it needs in order to produce the right output. Taking into account that the formation of different secondary structural elements depends on the interaction between neighboring-in-space (not necessarily in sequence) amino acid residues, we chose the BRNN architecture for its ability to encapsulate information included in the amino acid residues that are coming before and after the residue at the examined position t; where t denotes the discrete time index in [1, T ], with T being the total length of the protein chain.

The BRNN architecture consists of two Recurrent Neural Networks (RNNs) and a Feed Forward Neural Network (FFNN). The RNNs are used for taking into account the information contained in a local segment of length L s centered around position t. The Forward Recurrent Neural Network (FRNN) processes j = Ls-1 2 amino acid residues located on the left side of the segment, computing iteratively from the far left side of the window (i.e., in position t -j) and moving towards the right until position t (inclusive) by taking into account a sliding sub-segment of length S l . The Backward Recurrent Neural Network (BWRNN) processes the amino acids located on the right side of t, in a similar symmetric way.

During the recurrent network processing, a kind of memory is being formed since the NN correlates each sequence separately and holds an internal temporary knowledge [START_REF] Elman | Finding structure in time[END_REF]. The output from the two recurrent NNs and the output from the FFNN is correlated and predicts the secondary structure state for residue t as indicated in equation 1.

O t = η(F t , B t , I t ) (1)
where F t is the forward (upstream) context, B t is the backward (downstream) context and I t is the input vector at time (sequence position) t. In the current work, we use an input vector encoding a single residue (corresponding to a window size of unity in Baldi's [START_REF] Baldi | Exploiting the past and the future in protein secondary structure prediction[END_REF] implementation). The contextual information from the protein is gathered into a pair of vectors F t and B t . Only after the F t and B t are computed, the algorithm can predict the state (as in [START_REF] Baldi | Exploiting the past and the future in protein secondary structure prediction[END_REF]). In order for the amino acids to be examined, two learnable non-linear state transition functions φ(•) and β(•) are applied (equations 2 and 3):

F t = φ   0 j S l -1 i=0 γF t-j+i , I t   (2)
B t = β   0 j S l -1 i=0 γ -1 B t+j-i , I t   (3)
where γ ∈ (0, 1] is a modified shift operator, which in effect adds a constant weight based on the importance given to the outputs of the FRNN and BWRNN. Intuitively, we chose γ < 1 (thus γ -1 > 1) to reflect the fact that protein chains are synthesised from the N-terminal to the C-terminal (i.e., from the left to the right side of the sequence respectively) with some secondary structural elements forming co-translationally [START_REF] Baram | From peptide-bond formation to cotranslational folding: dynamic, regulatory and evolutionary aspects[END_REF].

Once the data located within the input vector enters the BRNN, the Mean Square Error function is applied and is used by the Backpropagation algorithm [START_REF] Rumelhart | Learning representations by backpropagating errors[END_REF] for the BRNN to be trained. The training is performed based on two alternative output encoding schemes: (i) an orthogonal, and (ii) a 'winner-take-all' (WTA). The former scheme has three output units with binary values giving eight possible combinations, three of which are assigned to the three reduced SS states, and the rest are arbitrarily considered to be classified as random coil. The WTA encoding scheme, has three output units as well (corresponding to the reduced SS states) and assigns the SS state of the winning neuron as the prediction for the examined residue. For both schemes, once the error is calculated, the delta rule is applied to update the weights.

Results and Discussion

A set of optimal NN parameters were empirically found following experimentation. For training the network, the dataset of 612 polypeptide chains was randomly split to 513 proteins for training and 99 proteins for testing.

Firstly, we explored BRNN architectures with two hidden layers. More specifically, the FFNN was composed of two fully interconnected hidden layers with 12 neurons each, whereas the BWRNN and FRNN consist of two hidden layers each, with the first hidden layer having 13 and the second 12 neurons.

Our first experiments, utilising single sequences at the input, achieved a highest result of 66.59% (Q3) with optimal parameters: γ = 0.7, learning rate α = 0.8, momentum m = 0.0, L s = 15, S l = 3, and orthogonal output encoding.

The next set of experiments was performed using the MSA profiles as input. For 11 out of the 612 protein chains of the initial data set an HSSP profile was not available, thus the data set was slightly reduced to a total of 601 chains, which was again randomly split into a training and a test set of 504 and 97 protein chains respectively. Initially, the predictive accuracy value reached on average 70.82% (Q3, with the aforementioned optimal parameters), almost 4% higher than when training with single sequences. In order to improve the performance, we decided to apply a randomisation procedure on the data at every iteration (i.e., when all the training data is passed through the network). We consider randomisation to be equivalent to the insertion of noise during training, which could theoretically improve the results. As predicted, randomisation does indeed improve the prediction accuracy, as illustrated in Table 1. It has to also be noted that large L s values gave better results than smaller ones. We experimented with L s values up to 60 (where data sets were modified accordingly to exclude sequences shorter than L s ), and from the results it was obvious that the optimal L s was 31 (which concurs with the input window size of [START_REF] Baldi | Exploiting the past and the future in protein secondary structure prediction[END_REF]). As it can be seen from Table 1 the best result is 73.92% (Q3) accuracy; the corresponding SOV measure is 63.02%. Experimentation with varying the number of neurons in all the layers of the constituent networks of the BRNN did not give any significant improvement in the results reported above. Secondly, we decided to reduce the complexity of the BRNN architecture by using a single hidden layer. We also changed the output encoding to WTA. In addition, we experimented with different hidden layer sizes and the optimal results (shown in Table 2, columns 1 and 2) were obtained with: (i) a FFNN of 15 hidden neurons combined with RNNs of 17 hidden neurons (rows 1-4), and (ii) a FFNN of 51 hidden neurons combined with RNNs of 41 hidden neurons (rows 5-8). In order to assess whether these predictions are significantly different, we performed four repetitions of each configuration (see table 2), and descriptive statistics were calculated (data not shown). For the Q3s resulting for the two configurations no statistically significant difference could be observed, with a non-parametric Mann-Whitney test (p-value=0.57). As we can see from the results shown in table 2 there was a significant improvement of the Q3 prediction accuracy measure (up to 76.07%) and a SOV of up to 65.36% compared to architectures with two hidden layers. Knowing that filtering the initial sequence-to-structure network outputs improves the prediction accuracy (see [START_REF] Chen | Cascaded bidirectional recurrent neural networks for protein secondary structure prediction[END_REF] and references therein), we decided to explore this possibility on our BRNN architectures which gave the best results (i.e., with one hidden layer). Two filtering approaches were employed, one based on a novel Hidden Markov Model (HMM) and one based on a cascaded Artificial NN (ANN, similar to the structure-to-structure network of [START_REF] Rost | Improved prediction of protein secondary structure by use of sequence profiles and neural networks[END_REF]) for comparison. According to the results (shown in table 2, columns 3-6), both filtering approaches improve the results, in particular there is a significant increase in the SOV values (in the order of up to 7%). More specifically, for both configurations the HMM filtering was producing significantly higher SOV values (Mann-Whitney test: p-value = 0.03 for both cases) compared to the unfiltered results. In addition, the ANN filtering produced significantly higher SOV values only for the second configuration (Mann-Whitney test: p-value =0.03) but not for the first one (p-value=0.2). The increase in SOV was expected, since both filtering procedures are known to eliminate biologically irrelevant predictions, especially in cases where isolated residues are predicted in a SS state.

For our best achieved results, further analysis regarding the details of the prediction in different SS states has been conducted. More specifically, we counted all possible instances of observed versus predicted outcomes, through which confusion matrices were created, as shown in Table 3. Three are the main observations from the confusion matrices of Table 3: 1. Helices (H) and loops (C) are pretty accurately predicted.

2. Extended structures (E) suffer from under-prediction. However, the novel HMM-based filtering method seems to partially overcome this problem. 3. Most of the incorrect predictions involve the loop secondary structure state.

Observation 1 was expected since (i) most other works on PSSP report similar trends, and (ii) H and C are the most populated states in our datasets as opposed to strands (observation 2). For observation 3, we believe this could be an artifact of our output encoding scheme (see Section 2.2). It is worth pointing out that the ANN filtering method (see Table 3d) completely eliminates false predictions in the 'H' class.

Conclusions

In an attempt to tackle the PSSP problem, our modified training procedure for the BRNN, where the main modification is the updating of the weights for each residue, gives a prediction accuracy of Q3=76.07% with respective SOV = 64.32, which is better than the state-of-the-art results of [START_REF] Baldi | Exploiting the past and the future in protein secondary structure prediction[END_REF] (Q3=73.6%) and clearly comparable to the results obtained in [START_REF] Baldi | Exploiting the past and the future in protein secondary structure prediction[END_REF] when an ensemble of 6 BRNNbased predictors was used (i.e., Q3=75.1% on average). Even though we have a slight computational overhead with our approach, where weight updates occur for each residue presented to the BRNN, in practice our BRNNs train (in the worst case) within a couple of hours, and certainly we assume that (for most network parameter sets) our single BRNN would be computationally cheaper than the ensemble of 6 BRNNs reported in [START_REF] Baldi | Exploiting the past and the future in protein secondary structure prediction[END_REF]. Training time is not an issue for this type of applications, since it not anticipated that the BRNNs will have to be trained very frequently; the most important issue in this application is the prediction accuracy. Certainly, an exact comparison with [START_REF] Baldi | Exploiting the past and the future in protein secondary structure prediction[END_REF] cannot be made, unless the same training and testing sets are used, but even with that the optimal connectivity of Baldi's architecture is not available (personal communication with one of the authors of [START_REF] Baldi | Exploiting the past and the future in protein secondary structure prediction[END_REF], G. Pollastri).

In the work of Baldi et al. [START_REF] Baldi | Exploiting the past and the future in protein secondary structure prediction[END_REF], the BRNN has a short input window of residues centered at the prediction site. Although this small window aims to avoid overfitting, it does not capture variable long-range information, which is overcome by unfolding the RNNs throughout the sequence. Despite the fact that we minimise the input vector in order to contain information for a single residue, the novelty of our approach lies firstly on the fact that we use a more elaborate computation within the recurrent context windows (as described in section 2.2), and secondly on updating network weights at every amino acid residue. With the latter, even though we are not able to capture long-range dependencies, we manage to more accurately take into account all available local information and this seems to be justified from our results.

Improved prediction results were obtained when sequence-to-structure output was filtered, in a post-processing step, in order to take higher order SS correlations into account. In particular, our novel HMM-based filtering approach not only improved the unfiltered results, but it was shown to be on average marginally better than a standard feed forward ANN-based filtering approach and much better than the BRNN-based filtering results reported in [START_REF] Chen | Cascaded bidirectional recurrent neural networks for protein secondary structure prediction[END_REF].

We believe that if we were to use a ensemble of BRNN-based predictors with our training scheme and our novel filtering procedures, our results would be even better.

Table 1 .

 1 Prediction accuracy results with a BRNN architecture with two hidden layers for different sizes of the local context window Ls. MSA profiles were used as input (for all parameter values, see text).

	Ls	15	17	19	21	23	29	31
	Q3% 72.94 73.14 72.79 71.80 73.09 73.41 73.92

Table 2 .

 2 Prediction accuracy results with a BRNN architecture with one hidden layer, randomized input, WTA output encoding and MSA profiles as input. The first four rows correspond to a BRNN architecture with a FFNN of 15 hidden neurons and RNNs of 17 hidden neurons and the last four rows to a BRNN architecture with a FFNN of 51 hidden neurons and RNNs of 41 hidden neurons. The rightmost columns correspond to the performance metrics after filtering with an HMM and a feed forward ANN.

	Q3% SOV Q3% SOV Q3% SOV
	HMM HMM ANN ANN
	76.07 64.32 76.57 70.32 76.60 71.90
	75.32 64.66 75.17 67.67 75.47 72.90
	75.14 62.12 76.38 68.13 75.59 63.05
	74.81 65.36 74.69 67.99 75.11 69.88
	75.26 64.53 75.75 69.55 76.04 70.40
	75.49 64.45 75.90 69.51 76.33 72.91
	76.07 62.43 76.84 69.15 76.44 71.19
	75.26 65.21 75.90 69.09 73.61 71.32

Table 3 .

 3 Confusion matrices, showing the distribution of predictions for the three secondary structure states, for the best performing configurations: (a) for the best configuration listed in 1 giving a Q3 of 73.92%, (b) best results of the unfiltered configuration shown in table 2 giving a Q3 of 76.07%, (c) best results of the configuration shown in table 2 filtered with a HMM giving a Q3 of 76.57%, and (d) best results of the configuration shown in table 2 filtered with an ANN giving a Q3 of 76.60%. Note that the displayed values correspond to the fraction of predicted residues with a given observed state, expressed as a percentage.

	Obs vs Pred H	E	C	H	E	C
			(a)			(b)
	H	62.63 3.39 33.98 71.65 5.46 22.89
	E	7.22 45.03 47.74 7.50 59.85 32.65
	C	6.55 4.31 89.12 8.01 7.20 84.79
			(c)			(d)
	H	73.24 4.39 22.37 72.74 4.16 23.37
	E	5.77 60.62 33.61 0.00 57.99 42.01
	C	8.75 7.24 84.01 0.00 14.21 85.79

Acknowledgments

We gratefully acknowledge the support of the Cyprus Research Promotion Foundation, as well as the European Structural Funds for grant TPE/ORIZO/0308 (FR)/05.


Angelos I Markos
email: amarkos@eled.duth.gr

Manolis G Vozalis

Konstantinos G Margaritis

An Optimal Scaling Approach to Collaborative Filtering using Categorical Principal Component Analysis and Neighborhood Formation

Keywords: Collaborative filtering, Recommender systems, Low-rank approximation, Categorical Principal Component Analysis, Optimal scaling

Collaborative Filtering (CF) is a popular technique employed by Recommender Systems, a term used to describe intelligent methods that generate personalized recommendations. The most common and accurate approaches to CF are based on latent factor models. Latent factor models can tackle two fundamental problems of CF, data sparsity and scalability and have received considerable attention in recent literature. In this work, we present an optimal scaling approach to address both of these problems using Categorical Principal Component Analysis for the low-rank approximation of the user-item ratings matrix, followed by a neighborhood formation step. The optimal scaling approach has the advantage that it can be easily extended to the case when there are missing data and restrictions for ordinal and numerical variables can be easily imposed. We considered different measurement levels for the user ratings on items, starting with a multiple nominal and consecutively applying nominal, ordinal and numeric levels. Experiments were executed on the MovieLens dataset, aiming to evaluate the aforementioned options in terms of accuracy. Results indicated that a combined approach (multiple nominal measurement level, "passive" missing data strategy) clearly outperformed the other tested options.

Introduction

Recommender Systems (RSs) are intelligent applications that generate personalized recommendations. RSs often rely on Collaborating Filtering (CF), which is based on the premise that users who have agreed in the past, tend to agree in the future. Among the most common and successful approaches to CF are latent factor models, which directly profile both users and products. The goal of latent factor models is to uncover latent features that explain user preferences by transforming both items and users to the same latent factor space, thus making them directly comparable. Latent factors represent either obvious characteristics of the items or completely uninterpretable dimensions. Each factor measures the degree of preference for each user towards items that score high on the corresponding factor.

Several latent factor models have been successfully applied to CF. Sarwar et al. [START_REF] Sarwar | Application of dimensionality reduction in recommender systems -a case study[END_REF] apply Singular Value Decomposition to reduce the dimensionality of sparse rating matrices and improve the scalability of CF systems. Goldberg et al. [START_REF] Goldberg | Eigentaste: A constant time collaborative filtering algorithm[END_REF] use Principal Component Analysis (PCA) as a preprocessing step for low-rank approximation and then utilize clustering to predict user preferences. Hoffman [START_REF] Hofmann | Latent semantic models for collaborative filtering[END_REF] suggests a collaborative prediction method based on a probabilistic latent variable model. Salakhutdinov and Mnih [START_REF] Salakhutdinov | Probabilistic matrix factorization[END_REF] propose a momentum based, probabilistic matrix factorization algorithm with batch learning which scales linearly with the number of observations. Bell and Koren [START_REF] Bell | Scalable collaborative filtering with jointly derived neighborhood interpolation weights[END_REF] present a neighborhood based approach, which uses alternating least squares and removes the global effect from the data. Tacacs et al. [START_REF] Tacacs | Scalable collaborative filtering approaches for large recommender systems[END_REF] discuss and compare modifications of already published matrix factorization methods. In addition to a straightforward optimization approach for the general low-rank approximation problem, many authors propose extensions of well established algorithms to cope with the missing data problem [START_REF] Kim | Collaborative filtering based on iterative principal component analysis[END_REF][START_REF] Paterek | Improving regularized singular value decomposition for collaborative filtering[END_REF].

In this work, we present an optimal scaling approach to address two fundamentals problems of CF, data sparsity and scalability. Optimal scaling is a general approach to treat multivariate data through the optimal transformation of qualitative scales to quantitative values [START_REF] De Leeuw | Nonlinear principal component analysis and related techniques[END_REF][START_REF] Costantini | Mining performance data through nonlinear pca with optimal scaling[END_REF]. In other words, both nominal and ordinal variables (or features) can be optimally transformed to variables with numeric properties. These optimal transformations of the original variables may be used to overcome the linear assumption underlying many classic dimensionality reduction methods. In this light, PCA has been reviewed and extended to Categorical or Nonlinear Principal Component Analysis (CatPCA) with many potential applications [START_REF] Meulman | Principal components analysis with nonlinear optimal scaling transformations for ordinal and nominal data[END_REF].

Our approach uses CatPCA for the low-rank approximation of the user-item ratings matrix, followed by a neighborhood formation step. The combination of matrix factorization and neighborhood formation can lead to accurate predictions [START_REF] Bell | Scalable collaborative filtering with jointly derived neighborhood interpolation weights[END_REF][START_REF] Tacacs | Scalable collaborative filtering approaches for large recommender systems[END_REF]. CatPCA is based on the Alternating Least Squares (ALS) algorithm utilizing an optimal least squares scaling process where original data are transformed so that their overall variance is maximized. The problem is formulated by means of a loss function and it is solved by the ALS. Eventually, this leads to optimally scaled scores on each factor [START_REF] Michailidis | The gifi system of descriptive multivariate analysis[END_REF]. Unlike other matrix factorization methods, CatPCA does not assume multivariate normality and linear relationships between variables and provides a flexible framework for parametrization. In particular, we consider different measurement levels for the user ratings on items, starting with a multiple nominal and consecutively applying nominal, ordinal and numeric levels. A series of experiments was executed on the MovieLens dataset in order to evaluate the aforementioned options in terms of accuracy. A combined approach (multiple nominal level, "passive" missing data strategy) clearly outperformed the other tested options.

The remainder of this paper is organized as follows. The next section, Section 2, gives a brief presentation of CatPCA with optimal scaling. In Section 3, the proposed approach is thoroughly described, starting with the low-rank approximation of the ratings matrix, which is achieved through CatPCA, and continuing with a user neighborhood formation step. The efficiency of four distinct variations of this approach is demonstrated in Section 4 through a set of experiments on a publicly available dataset. Finally, we give conclusions and provide a discussion over the results in Section 5.

Categorical Principal Component Analysis with Optimal Scaling

CatPCA with optimal scaling or optimal scoring is a general approach to treat multivariate data through the optimal transformation of qualitative scales to quantitative values [START_REF] De Leeuw | Nonlinear principal component analysis and related techniques[END_REF][START_REF] Costantini | Mining performance data through nonlinear pca with optimal scaling[END_REF]. In the optimal transformation process, an appropriate quantification level has to be chosen for each of the variables. The most restricted transformation level is called numeric; it applies a linear transformation to the original integer scale values, so that the resulting variables are standardized. The order of variable categories and the equal distances between category numbers of the original variable are preserved in the optimally scaled variable. When all variables are at a numeric analysis level, the analysis is analogous to standard PCA. The optimal least squares transformations of ordinal data can be handled by means of monotonic transformations, which maintain the order in the original data. In the case of nominal data, categories are not ordered and the only information that is preserved is the grouping of objects in categories. Finally, when all variables are at a multiple nominal level, the only information that is preserved is the grouping of objects in categories. The quantification is called multiple because there is a separate quantification for each dimension. With all variables at a multiple nominal level, the analysis is equivalent to multiple correspondence analysis or homogeneity analysis [START_REF] De Leeuw | Nonlinear principal component analysis and related techniques[END_REF][START_REF] De Leeuw | Gifi methods for optimal scaling in r: The package homals[END_REF].

Algorithm Description

In this Section we describe the proposed optimal least squares approach for CF, which starts with the low-rank approximation of the user-item ratings matrix and is followed by a user neighborhood formation step.

Low-Rank approximation through CatPCA

We start with the following basic definitions. For i = 1, . . . , n users, ratings on j = 1, . . . , m items are collected in the n × m data matrix R. Each of the corresponding items takes on k different rating values (levels or categories) from a given range, i.e. (1, 2, 3, 4, 5). They are coded using m binary indicator or dummy matrices G j of size n × k, with entries G j (i, t) = 1 if user i has given item j a rating of t, and G j (i, t) = 0 for any of the remaining rating values. The whole set of indicator matrices can be collected in a block matrix

G = [G 1 . . . G m ].
Missing item ratings are coded as complete zero row sums: if user i has not rated item j, then the row sum of G j is 0. Otherwise the row sum becomes 1. Let M j denote the n × n binary diagonal matrix with entries M j (i, i) = 1 if user i has rated item j and 0 otherwise. Based on M j we define M as the sum of the M j 's and M • as their average. This missing data strategy is known in the literature as missing data passive, because it leaves the indicator matrix G j incomplete [START_REF] Michailidis | The gifi system of descriptive multivariate analysis[END_REF]. The main advantage of this option is that it ignores the missing ratings without making strong assumptions regarding the pattern of missing data.

Let X be the unknown n × p matrix containing the user scores in the latent factor space R p and Y j be the unknown k ×p matrix containing the rating scores (quantifications) of item j in the same p-dimensional space. Both matrices have to be determined during optimization. Based on these definitions, the following loss function can be established:

σ(X; Y 1 . . . Y m ) = 1 m m j=1 tr(X -G j Y j ) M j (X -G j Y j ) (1)
which is optimized under the normalization conditions u M • X = 0, with u denoting an n vector with ones and X M • X = I such that the trivial solution of complete 0-scores is avoided. From an analytical point of view the loss function represents the sum of squares of (X -G j Y j ) and a simultaneous minimization over X and Y j . Minimizing equation 1 leads to a low dimensional space where users are positioned as close as possible to the ratings they have given to specific items and where an item's rating is the centroid of the users who have given this rating to the item.

The minimization problem can be solved by means of an iterative ALS procedure as follows: At iteration 0 we begin with a random starting solution X 0 for the user scores. Each iteration s consists of three steps:

1. Update item rating quantifications: Y (s) j = D -1 j G j X (s) for j = 1, . . . m, where D j = G j G j the k ×k diagonal matrix with the (marginal) frequencies of item j in its main diagonal. 2. Update user scores:

X (s) = M -1 * m j=1 G j Y (s) j 3. Normalization: X (s+1) = M -1/2 * orth()(M -1/2 * X (s) )
Here orth() denotes the modified Gram-Schmidt process which is used to compute an orthonormal basis for the column space of a matrix. This iterative process is continued until the improvement in subsequent loss values is below a convergence criterion. Note that matrix multiplications using indicator matrices can be implemented efficiently as cumulating the sums of rows over X and Y. The ALS algorithm only computes the first p dimensions of the solution, which leads to an increase in computational efficiency. Moreover, by capitalizing on sparseness of G, the algorithm is able to handle large datasets.

The approach described above is known as homogeneity analysis (homals) with all item ratings at a multiple nominal level of measurement and it has the advantage that it can be easily extended to the case when restrictions can be easily imposed. Nominal, numerical and ordinal item ratings can be incorporated with a rank-1 restriction of the form

Y j = z j a j (2)
where z j is a column vector of length k with item rating quantifications and a j a vector of length p with weights. Using this restriction the ratings of a specific item are forced to be on a line in the p dimensional space. However, this restriction is not sufficient to preserve the order for ordinal ratings or even the relative distance for numerical ratings. Therefore, z j should be transformed in every ALS iteration to satisfy these restrictions. In the case ordinal ratings this transformation is achieved by means of a weighted monotone regression of z j on r j and in the case of numerical ratings z j is replaced by the original item rating r j . For a more detailed description we refer to [START_REF] Michailidis | The gifi system of descriptive multivariate analysis[END_REF][START_REF] De Leeuw | Nonlinear principal component analysis and related techniques[END_REF].

User Neighborhood Formation

At this point, we use the n × p matrix X, which is obtained from the previous step, in order to form each user's neighborhood. Note that X contains the user scores in the p-dimensionality space. To find the proximity between two users, a and i, we utilize the Pearson product-moment correlation coefficient, as a similarity measure between each user and his closest neighbors. This similarity measure has been evaluated in a number of studies and has been found to be reliable and accurate. The coefficient is calculated as follows:

cor ai = l j=1 r aj r ij l j=1 r aj h j=1 r ij (3)
where r ij denotes the rating of user i on item j. The summations over j are calculated over the l items for which both users a and i have expressed their opinions.

After a user neighborhood of size h has been formed for user a, we proceed with prediction generation. A prediction rating p aj for user a on item j is computed using the following equation:

p aj = h i=1 r ij * cor ai h i=1 |cor ai | (4)

Experiments

In this section we provide a brief description of the experiments we executed in order to evaluate and compare the proposed methods, and then we present and comment on their results. For this purpose, we utilized the MovieLens dataset. It consists of 100, 000 ratings which were assigned by 943 users on 1682 movies and is available at http://www.grouplens.org/. Ratings follow the 1(bad)-5(excellent) numerical scale. The sparsity of the dataset is high, at a value of 93.7%. Starting from the original dataset, five distinct splits of training (80%) and test (20%) data were utilized. The proposed algorithms were applied on each split and the average result across all 5 trials was computed.

Mean Absolute Error (MAE) was the metric we employed to evaluate the accuracy of the methods. MAE measures the deviation of predictions generated by the RSs from the true rating values, as they were specified by the user.

For our first experiment, we kept a fixed number of retained dimensions and tried to evaluate the impact of a varying neighborhood size on prediction accuracy. Figure 1 depicts the MAE values for neighborhood sizes ranging between 1 and 400 users. Based on that figure, it is clear that the multiple nominal and numeric approaches performed better than the nominal and ordinal ones. In particular, multiple nominal has generated the most accurate prediction among those tested (MAE=0.7646). This value was achieved for a neighborhood of 170 users, the smallest in size when compared to the number of neighbors required by the rest of the approaches to reach their optimal predictions. Specifically, the lowest MAE values for the numeric, nominal and ordinal approaches were 0.7763, 0.7985, and 0.7993, observed for neighborhoods including 280, 330 and 330 users, respectively.

Once the effect of varying neighborhood sizes was assessed, a second experiment was executed, aiming to evaluate the impact of dimensionality reduction on prediction accuracy. Figure 2

Conclusions

In this paper, we describe an optimal scaling approach for CF using CatPCA, which combines the low-rank approximation of the user-item ratings matrix with user neighborhood formation. We considered four different measurement levels for the user ratings in the Movielens dataset. A combined approach, where the user ratings on each item were handled as multiple nominal, gave the most accurate predictions according to MAE. CatPCA with optimal scaling can offer a versatile set of options suited to CF problems. The main advantage of the proposed approach stems from the fact that CatPCA is able to account for more of the variance in the data compared to linear PCA, when the variables are (or may be) nonlinearly related to each other. Another advantage of this approach is that the "passive" strategy for handling missing data, ignores missing ratings without making strong assumptions for their pattern as is the case for various imputation methods. Additionally, predictions for new users of the system can be obtained by the model without requiring a complete retraining of the whole dataset. This is achieved after optimization by means of a cone restricted SVD [START_REF] De Leeuw | Gifi methods for optimal scaling in r: The package homals[END_REF], without influencing the existing scores computed by ALS.

In order to further explore the effectiveness of the proposed approach, more applications on datasets of different size and sparsity are needed. Except for the optimal least squares transformations described in Section 3, optimal spline transformations can also be utilized. Splines are usually smoother and more robust, albeit at the cost of less goodness of fit with respect to the overall loss function that is minimized [START_REF] Michailidis | The gifi system of descriptive multivariate analysis[END_REF][START_REF] De Leeuw | Gifi methods for optimal scaling in r: The package homals[END_REF]. Another direction for future work is to introduce regularization terms into the loss function of CatPCA in order to improve the stability of the algorithm.

Fig. 1 .

 1 Fig. 1. Comparison of the three CatPCA algorithms for varying neighborhood sizes

  depicts the MAE results for values of retained Comparison of the three CatPCA algorithms for different values of retained dimensions dimensions d, between 30 and 100. Based on that Figure, the advantage of multiple nominal approach over the rest was verified. Furthermore, both multiple nominal and ordinal levels reached their best MAE values for d=70, whereas d was 50 and 60 for numeric and nominal levels, respectively.

		0.815
								mult nom
								nominal
		0.81						numeric
								ordinal
		0.805
		0.8
		0.795
	MAE	0.785 0.79
		0.78
		0.775
		0.77
		0.765
		0.76
		30	40	50	60	70	80	90	100
					retained dimensions
	Fig. 2.