Xavier Rocquefelte

Fabrice Goubin

Yvan Montardi

Nicolas Viadere

Alain Demourgues

Alain Tressaud

Myung-Hwan Whangbo

Ste ´phane Jobic

Analysis of the refractive indices of TiO 2 , TiOF 2 , and TiF 4 : concept of optical channel as a guide to understand and design optical materials

In an effort to gain insight into how optical properties of insulating materials are affected by a change in chemical composition, we investigated the dielectric functions of titanium dioxide, TiO 2 , and its fluorine-substituted phases, TiOF 2 and TiF 4 , by electron energy loss spectroscopy measurements and density functional theory electronic band structure calculations. The refractive indices of these compounds are found to be inversely proportional to their cell volumes per formula unit. This observation was explained by employing the concept of optical channels. Our study indicates that the light-scattering properties of insulating compounds can be controlled by modifying their cell volumes.

Introduction

The relationships between optical properties and chemical composition were recognized at an early age. 1 To some extent, these relationships remain mysterious because it is not fully understood how to design a material with a required opacity (i.e., the light-scattering power) and color strength (i.e., the light-absorbing capacity), which depend on the refractive indices n and the extinction coefficient k, respectively. It is therefore important to explore the factors governing the optical properties of materials and their refractive index in particular. In rationalizing and/or estimating refractive indices of closely related compounds, empirical relations have been widely used. Specifically, refractive indices are correlated with either band gaps (Moss relationship) 2 or mass densities (Gladstone-Dale relationship). 3 Although these relationships work satisfactorily for specific situations, there has been no simple rule with which to explain how structural and compositional changes might affect optical properties.

In describing the interaction of light with matter, one often employs oscillator models such as the Drude and Lorentz models. 4,5 However, a more precise description of this interaction requires a proper consideration of quantum mechanics. Density functional theory (DFT) 6,7 electronic band structure calculations and electron energy loss spectroscopy (EELS) 8 experiments enable one to determine the dielectric function (ω) ) 1 (ω) + i 2 (ω) of a material, from which n and k are deduced. 9,10 These parameters are functions of the incident light energy pω. The imaginary component 2 (ω) represents the absorption power of a material per unit volume at a given incident light energy. The zero-frequency limit of n(ω), commonly used as an indicator of the refractive * To whom correspondence should be addressed. E-mail: stephane.jobic@cnrs-imn.fr (S.J.), mike_whangbo@ncsu.edu (M.-H.W.).

† Institut des Mate ´riaux Jean Rouxel. ‡ Rhodia. § Institut de Chimie de la Matie `re Condense ´e de Bordeaux.

| North Carolina State University. (1) Clottes, J.; Menu, M.; Walter, P. Bull. Soc. Prehist. Fr. 1990, 87, 171 (special issue for the 50th anniversary of the discovery of Lascaux). (2) Moss, T. S. Phys. Status Solidi B 1985, 131, 415. (3) Jaffe, H. W. Am. Mineral. 1956, 41, 757. (4) index in the visible range for insulators, is related to the integration of the extinction coefficient k(ω) weighted by the inverse of the excitation energy, 1/ω. That is

The zero-frequency limit of the real component 1 (ω) is equal to n 0 2 and is related to the integration of 2 (ω) weighted by 1/ω Consequently, as k(ω) or 2 (ω) becomes larger and as the optical gap becomes smaller, the refractive index becomes larger. On the basis of DFT calculations, we recently showed that the refractive indices n 0 of the seven allotropes of TiO 2 follow the Gladstone-Dale relationship, i.e., n 0 ∝ 1/V, where V is the cell volume per formula unit (FU). 11 The refractive indices reported in the literature are commonly determined using the D lines of Na (i.e., ∼2.10 eV). The trends in the n values determined around pω ≈ 2 eV are well approximated by those of n 0 values. 11 For compounds having the same chemical composition, this study showed that the key parameter for controlling the refractive index is the total absorption power per unit volume I( 2 ), i.e., the integrated value of 2 (ω) TiO 2 is widely used as a UV shield in different applications such as solar creams. However, although its bulk optical gap is well suited for UV absorption (E g ≈ 3.1 eV), it scatters visible light, leading to whitening of the surface or the volume to protect, because of its high refractive index (n ≈ 2.7). An important industrial challenge is then to prepare colorless UV shield (second-generation UV shield) materials with n < 2 and E g ≈ 3.1 eV. One way to reduce the extent of light scattering is to replace high-scattering elements with low-scattering ones, e.g., replacing high-scattering oxygen atoms with lowscattering fluorine atoms in TiO 2 . The resulting compounds TiOF 2 and TiF 4 are made up of TiX 6 octahedra (X ) O, F) as in the case of the seven allotropes of TiO 2 . The replacement of one O atom by two F atoms increases the cell volume V per FU. Thus, the substitution of F for O should decrease the refractive index. However, the substitution increases the number of valence electrons per FU, which should increase the refractive index. To gain insight into the overall effect of a chemical substitution on optical properties, we examine in this work the optical properties of TiO 2 , TiOF 2 , and TiF 4 . Our work is organized as follows: We describe our calculations of 2 (ω) and EELS measurements of TiOF 2 in section 2 and introduce the concept of optical channels in section 3. In section 4, we compare the experimental and theoretical 2 (ω) spectra of TiOF 2 and then 16) Kresse, G.; Furthmuller, J. Phys. ReV. B 1996, 54, 11169. (17) Blo ¨chl, P. E. Phys. ReV. B 1994, 50, 17953. (18) Optimized model structure 1 of TiOF 2 is as follows: P4/mmm; a )

3.9348 Å, c ) 3.7143 Å; Ti ) (0, 1 /2, 0), O ) (0, 0, 1 /2), F ) (0, 1 /2, 0). (19) Optimized model structure 2 of TiOF 2 is as follows: P4/mmm; a ) 7.6401 Å, c ) 3.9179 Å; Ti ) (0.761 40, 0.761 40, 0), O ) (0, 0.745 28, 0), F1 ) ( 1 /2, 0.742 67, 0), F2 ) (0.742 67, 0.742 67, 1 /2). (20) Optimized model structure 3 of TiOF 2 is as follows: P1; a ) 7.7145 Å, b ) 7.7176 Å, c ) 7.7475 Å; R ) β ) 89.93, γ ) 90.04°; Ti1 ) (0.
n 0 ≡ n(0) ) 1 + 2 π ∫ 0 +∞k(ω) ω dω (1) n 0 2 ) 1 (0) ) 1 + 2 π ∫ 0 +∞ 2 (ω) ω dω (2) I( 2 ) ) ∫ 0 +∞ 2 (ω) dω (3)
show that the refractive indices of TiOF 2 ,TiF 4 , and the seven TiO 2 allotropes are proportional to their 1/V values. The reason for the latter observation is discussed in section 5 using the concept of optical channels. Finally, important results of our study are summarized in section 6.

Electronic Structure Calculations and Electron Energy Loss Spectroscopy Measurements

2.1. Electronic Structure Calculations and Dielectric Function. First-principles DFT electronic band structure and optical property calculations were carried out on TiOF 2 12 and TiF 4 13 using the WIEN2k 14 code. In both cases, we employed the Perdew-Burke-Ernzerhof generalized-gradient 15 approximation for the exchange and correlation potential. As the crystal structure of TiOF 2 exhibits a cubic symmetry [SG (space group): Pm3 hm] in which the O and F atoms randomly occupy the same site, three ordered model structures were generated from geometry optimization using the VASP 16 code with the projector augmented wave 17 potentials prior to the first-principles calculation of the imaginary part of the dielectric function. All model structures of TiOF 2 were derived from the ReO 3 structure. In model structures 1 and 2, the 1 × 1 × 1 unit cell is conserved but the F and O atoms are ordered: the O atoms are positioned axially in model structure 1 (SG: P4/mmm) 18 but in the equatorial plane in model structure 2 (SG: P4/mmm). 19 Model structure 3 corresponds to a random distribution of the F and O atoms in a 2 × 2 × 2 supercell (SG: P1). 20 The geometry optimizations (i.e., atomic positions and cell parameters) of these model structures give rise to cell parameters that are in good agreement with experiment (a slight overestimation of the cell parameters by less than 2%).

DFT calculations underestimate optical band gaps. There are several ways of correcting this deficiency, 21 the simplest one of which is to use a scissors operator (SO), i.e., shifting the conduction bands rigidly with respect to the valence bands to obtain the correct band gap. We calculated n 0 without using an SO and the n values at 2.1 eV after using an appropriate SO. The values of the SO were adjusted to match the EELS data for TiO 2 (SO ) 0.6 eV) and TiOF 2 (SO ) 2.13, 0.83, and 1.53 eV for model structures 1-3, respectively). For TiF 4 , we chose SO ) 1.15 eV by matching the F 2p f Ti 3d peak positions of the 2 (ω) spectra of TiOF 2 and TiF 4 . As can be seen from Table 1, the n 0 values exhibit a trend similar to that for the n values at 2.1 eV.

2.2. Electron Energy Loss Spectroscopy. TiOF 2 samples were characterized by electron energy loss spectroscopy (EELS) measurements using a transmission electron microscope (TEM; Philips CM 30) with a LaB 6 gun. The TEM was operated at 100 kV and was coupled with a GATAN 666 spectrometer controlled by ELP3 software, which derives the optical constants of samples under investigation. The experimental procedure has been described elsewhere. [22][23][24][25] EELS experiments were not carried out for TiF 4 because the latter is highly sensitive to moisture.

Concept of Optical Channels

In general, n 0 increases with increasing I( 2 ). For the seven different allotropic forms of TiO 2 , the total absorption power I( 2 )V per FU is nearly constant, thereby explaining why the Gladstone-Dale relation works for these systems. 11 Although the refractive indices of allotropic phases are well described by the cell volume, there has been no simple rule that allows one to predict how a chemical substitution might influence refractive index. This is not surprising because chemical substitution influences the chemical bonding, the crystal structure, and the cell volume V simultaneously. To help simplify this multiple-parameter problem, we will introduce the concept of "optical channels". As described in section 1, calculations of optical properties are based on electronic band structures in which the electrons are viewed as completely delocalized. We define optical channels as chemical bonds associated with a chromophore. In compounds such as TiO 2 , TiOF 2 , and TiF 4 in which absorption in the visible range is related to charge-transfer excitation from the ligand p-block bands to the Ti d-block bands (in other words, from the anions O 2-and F -to the cations Ti 4+ ), the optical channels are the Ti-O and Ti-F bonds. With this local view of light absorption, it is straightforward to consider two factors affecting optical properties, i.e., the chromophore and the volume. The chromophore effect depends on the efficiency of chemical bonds for light absorption (i.e., "channel quality"), the number of optical channels per FU (i.e., channel quantity), and the number of valence electrons per FU participating in excitation. The volume effect measures the density of the optical channels per FU. The quantity I( 2 ) reflects both the chromophore and volume effects, but the total absorption power, I( 2 )V, reflects only the chromophore effect. In this work, we show the utility of the concept of optical channels by studying the refractive indices of the titanium oxide fluorides TiOF 2 and TiF 4 on the basis of DFT electronic band structure calculations and EELS experiments. Such an approach can be useful in designing new-generation UV shields.

Effect of Fluorine Substitution on Refractive Index

The electronic band structures of TiO 2 , TiOF 2 , and TiF 4 exhibit similar features. The valence bands are composed largely of the 2p orbitals of O and/or F, and the conduction bands are composed largely of 3d orbitals of Ti. As expected, the F 2p-block bands lie somewhat lower in energy than the O 2p-block bands, but they do overlap in TiOF 2 . The F and O 2s-block bands are well separated from the corresponding 2p-block bands (located approximately 15 eV below the top of the valence band). Thus, the O 2p f Ti 3d and/or F 2p f Ti 3d excitations are primarily responsible for the refractive indices of TiO 2 , TiOF 2 , and TiF 4 . Consequently, 12, 18, and 24 valence electrons per FU are mainly responsible for the refractive indices of TiO 2 , TiOF 2 , and TiF 4 , respectively. In addition, the contributions to the transition dipole moments associated with these excitations are significant only between the 3d orbitals of each Ti atom and the 2p orbitals of those oxygen/fluorine atoms forming the first coordination sphere. Therefore, the optical channels of these compounds are their Ti-O and Ti-F bonds.

Figure 1 presents n(ω) and k(ω) plots for TiOF 2 , deduced from EELS measurements and calculated for model structure 1, as a function of the incident photon energy. The EELS and DFT data are in good agreement, as was also found for the rutile TiO 2 phase. The main features of the 2 (ω) spectra of TiO 2 and TiF 4 (associated with the O 2p f Ti 3d and F 2p f Ti 3d transitions, respectively) can be recognized from the 2 (ω) spectrum of TiOF 2 (Figure 2). The calculated n value of 1.93 is in excellent agreement with the experimental value deduced from our EELS measurements. The three different model structures of TiOF 2 lead essentially to the same value of refractive index, that is, the F/O disorder does not significantly affect the refractive index (Table 1). The n value decreases steadily from rutile TiO 2 (2.9) to TiOF 2 (1.9) 1). This decrease in n is related to the corresponding decrease in the I( 2 ) value from 71.9 to 40.7 to 30.6 (Figure 2, Table 1). Thus, the decrease in n is correlated with a lowering of the total absorption capability when one O is substituted by two F atoms.

For TiOF 2 , TiF 4 , and the seven TiO 2 allotropic phases, the refractive index n varies linearly with the total absorption capability per unit volume I( 2 ) (Figure 3a) and also with the reciprocal volume 1/V (Figure 3b). The latter observation indicates that the trends in the refractive indices are well described by the cell volumes per FU. It is important to understand the origin and meaning of this observation. The cell volume V per FU increases steadily from TiO 2 to TiOF 2 to TiF 4 (Table 1). In the case of the seven TiO 2 allotropes, I( 2 )V is nearly constant. This indicates that the structural changes in the seven TiO 2 allotropes affect mainly their cell volumes but not their chromophores. This is understandable because the natures of the Ti-O bonds, and hence the absorption power of the Ti-O bonds, are similar in the TiO 2 allotropes. It is surprising that the I( 2 )V term is also nearly constant for TiO 2 , TiOF 2 , and TiF 4 although the Ti-O and Ti-F bonds should exhibit considerably different absorption capabilities. The probable reason for this observation is discussed in the next section.

Total Absorption Power and Optical Channel

The fact that the total absorption power per FU is nearly constant for the TiO 2 , TiOF 2 , and TiF 4 phases implies that one Ti-O optical channel is optically equivalent to two Ti-F optical channels. The valence electrons of F are more strongly bound to the nucleus than are those of O. Thus, the extent of covalent interaction is stronger in the Ti-O bond than in the Ti-F bond, so that the extent of overlap between the Ti 3d and ligand 2p orbitals is stronger in the Ti-O bond. Consequently, each Ti-F optical channel contributes less to the optical absorption than does each Ti-O optical channel. Thus, on going from TiO 2 to TiOF 2 to TiF 4 , the quality of the optical channel decreases. However, on going from TiO 2 to TiOF 2 to TiF 4 , the number of valence electrons participating in the optical excitation increases (i.e., 12, 18, and 24 per FU, respectively). The total number of optical channels per FU is the same in TiO 2 , TiOF 2 , and TiF 4 ; i.e., each Ti has six Ti-O channels in TiO 2 , two Ti-O and four Ti-F channels in TiOF 2 , and six Ti-F channels in TiF 4 . As already mentioned, it is expected that one Ti-O channel is equivalent to two Ti-F channels in efficiency. Effectively, therefore, each Ti has six Ti-O channels in TiO 2 , four Ti-O channels in TiOF 2 , and three Ti-O channels in TiF 4 . Thus, the numbers of channels in TiO 2 , TiOF 2 , and TiF 4 have the ratio 6:4:3, whereas the numbers of their valence electrons participating in the excitation have the ratio 12:18:24. That is, the decrease in the efficiency for electrons to be excited from the ligand to the Ti atoms is compensated by the increase in the number of valence electrons that can be excited. As a consequence, the total absorption capability per FU, i.e., I( 2 )V, remains constant in TiO 2 , TiOF 2 , and TiF 4 .

It is of interest to note that the variation in the optical properties of a series of compounds can be described in terms of their total absorption capability per FU, i.e., I( 2 )V, which deals with only the chomophore effect. The scattering property of a material arises from the oscillating dynamic deformation of its electron density induced by an applied electric field. The n value becomes larger as the electron density distribution can be more easily distorted (i.e., is more polarizable). From the viewpoint of quantum mechanics, any deformation of a ground-state electron density requires the mixing of excited-state wave functions into the ground-state wave function (in a sense, by electronic excitations). Consequently, the greater the absorption probability, the higher the refractive index becomes.

Concluding Remarks

In summary, the total absorption capability per FU, I( 2 )-V, is nearly constant for TiOF 2 , TiF 4 , and the seven allotropes of TiO 2 because the effect of decreasing the quality of the optical channel is counterbalanced by the effect of increasing the number of valence electrons participating in the optical excitation. Because the refractive index n increases with increasing I( 2 ), it follows that n ∝ 1/V for all of these systems. Such a linear relationship was unexpected because the substitution of F for O induces dramatic changes in the structure and nature of the chemical bond. The concept of optical channels readily clarifies this seemingly unusual observation by separating the effect of the chromophore from the effect of the volume. Note that, in the optical channels consisting of the Ti-O and Ti-F bonds, the ligands O and F belong to the same row of the periodic table. For the refractive indices of ZnQ and CdQ (Q ) O, S, Se, Te), in which the ligands Q of the optical channels (the Zn-Q and Cd-Q bonds) belong to different rows of the periodic table, the values of the total absorption capability I( 2 )V are not constant but increase steadily as Q changes from O to S to Se to Te. 26 Finally, we note that, on going from rutile TiO 2 to TiOF 2 , the scattering properties is reduced by about 30%, without significantly changing the optical band gap. This observation implies that improved UV shields can be prepared by controlling the "chromophore" volume.

Acknowledgment. The work at NCSU was supported by the Office of Basic Energy Sciences, Division of Materials Sciences, U.S. Department of Energy, under Grant DE-FG02-86ER45259. We thank CCIPL (Centre de Calcul Intensif des Pays de la Loire) for generous computing resources.

Figure 1 .

 1 Figure 1. Comparison of the optical properties of TiOF2 determined from EELS experiments (open circles) and electronic band structure calculations using structure model 1 (solid curves): (a) refractive index n and (b) extinction coefficient k as functions of the energy of the incident light.

Figure 2 .

 2 Figure 2. Isotropic imaginary part 2 of the dielectric function as a function of the energy of the incident light: (a) rutile TiO2, (b) TiOF2 (for the structure model 1), (c) TiF4. The open circles correspond to the experimental 2 values from EELS measurements, and the solid curves to the theoretical 2 values predicted from electronic band structure calculations. The inset in b shows the contributions of the O 2p f Ti 3d and F 2p f Ti 3d transitions to 2.

IC048259W( 26 )

 26 Rocquefelte, X.; Whangbo, M.-H.; Jobic, S. Inorg. Chem. 2005, 44, 3594-3598. (27) CRC Handbook of Chemistry and Physics; Lide, D. R., Ed.; CRC Press: Boca Raton, FL, 2003.

Figure 3 .

 3 Figure 3. Refractive indices of TiO2, TiOF2, and TiF4 as functions of (a) I( 2) and (b) 1/V.

Table 1 .

 1 Values calculated at 2.1 eV after using a proper scissors operator. b Values in parentheses are the EELS experimental n values determined at 2.1 eV. c Values in brackets are the experimental data.27 Cell Volume V per Formula Unit, Refractive Index n, and Total Absorption Power per Unit Volume I( 2) of TiOF2, TiF4, and Seven TiO2 Allotropes and to TiF 4 (1.6) (Table

	(21) Held, K.; Anisimov, V. I.; Eyert, V.; Keller, G.; McMahan, A. K.;
	Nekrasov, I.

A.; Vollhardt, D. AdV. Solid State Phys. 2003, 43, 267. (22)

Egerton, R. F. Electron Energy Loss Spectroscopy in the Electron Microscope; Plenum Press: New York, 1986. (23) Montardi, Y.; Witz, C.; Normand, L.; Thorel A. J. Phys. IV 1999, 9, 217. (24) Goubin, F.; Rocquefelte, X.; Whangbo, M.-H.; Montardi, Y.; Brec, R.; Jobic, S. Chem. Mater. 2004, 16, 662. (25) Goubin, F.; Rocquefelte, X.; Pauwels, D.; Tressaud, A.; Demourgues, A.; Jobic, S.; Montardi, Y. J. Solid State Chem. 2004, 177, 2833.

a


Dictionnaire Montesquieu

Venuti, Filippo

Pierre Musitelli

Bibliographical reference

Musitelli Pierre, « Venuti, Filippo », translated by Philip Stewart, dans Dictionnaire Montesquieu [online], directed by Catherine Volpilhac-Auger, ENS Lyon, September 2013. URL : https://dictionnaire-montesquieu.enslyon.fr/fr/article/dem-1376427555-en/en A scholar born in Cortona in 1706 (and who died there in 1768), trained at the university of Pisa, founder with his brothers Marcello and Ridolfino of the Etruscan Academy of Cortona with which Montesquieu was associated in 1729, Filippo Venuti was destined for an ecclesiastical career. In 1738, failing to obtain a benefice at the pontifical court, he was named by the chapter of Saint Jean de Latran as administrator of the abbey of Clairac, on the banks of the Lot, a rich Vatican property in protestant territory. He thus found himself in a region dear to Montesquieu since the Secondats had acquired much land there and Jeanne de Lartigue was born in the domain of Petit-Vivens, very close to Clairac.

Venuti quickly came into contact with the writer, who was happy to press his election to the flourishing Bordeaux Academy: "you deserve, if the door is closed, for us to make a breach to let you in", he announced to him on 17 March 1739 (OC, t. XIX, letter 492). No sooner was he associated with it than the abbé multiplied his papers on the antiquities of Guyenne. In 1741, he won the prize of the Academy of Inscriptions and Letters, a distinction that led to his being elected its honorary foreign correspondent two years later. But with the financial situation of the abbey of Clairac deteriorating and its influence in a region greatly marked by protestantism being judged too feeble, he was removed from his function in 1742. Montesquieu took it to heart to help his unfortunate protégé. He explained to his colleague the Président Barbor, on 9 July: "They made it a crime, I think, for him to be so loved in the region by people who do not love his master. This man loves France, he breathes learning, he is a nobleman known throughout Europe, young and capable of anything. Let us make him our librarian; what do you think?" ("[…] on lui a fait un crime, je crois, de ce qu'il était trop aimé dans le pays par des gens qui n'aiment pas son maître. Cet homme aime la France, il ne respire que l'étude, c'est un homme de condition connu dans toute l'Europe, jeune et capable de tout. Faisons[-en] notre bibliothécaire; qu'en pensez-vous?", letter 525). On 9 September 1742, Venuti assumed his fonctions as librarian of the Bordeaux Academy. Dividing his time between academic labors and mundane occupations, he could frequent at leisure the cenacle of the Chevalier de Vivens at the château de Barry and the literary evenings organized by the Comtesse de Pontac-Belhade, a friend and correspondent of Montesquieu in her residence at Sauviac, near Montauban. With this contact with these clever minds, the abbé entered fully into this refined provincial milieu and could satisfy his natural penchant for literature.

In 1746, he translated into Italian Lefranc de Pompignan's tragedy Dido before taking on The Temple of Gnidus, published in 1749. In 1750 he published his Trionfo letterario della Francia, an apology in verse for the French literary genius inspired by Petrarch's Triumphs. But Venuti's ambition was to settle in Paris and play there a role as emissary of the learned world within the clergy. In 1748, in order to win the esteem of the bishop de Mirepoix, former preceptor to the Dauphin, he translated into Italian Louis Racine's poem Religion. The enterprise proved an embarrassing failure. It so wounded his vanity that he seems to have precipitated his departure for Bordeaux, which Montesquieu regretted in a letter of 18 May 1750: "I am very sorry, my dear abbé, that you are leaving for Italy, and even more that you are not happy with us. I see however, from what news reached me, that there was no intention to fail in the consideration that is so rightly due you. I certainly hope that you are satisfied with your journey to Italy and I would be most pleased if, after this time of pilgrimage, you could pass a happier transmigration, and such as your personal merit requires" ("Je suis bien fâché, mon cher abbé, que vous partiez pour l'Italie, et encore plus que vous ne soyez pas content de nous. Je vois pourtant sur ce qui m'est revenu qu'on n'a pas pensé à manquer à la considération qui vous est due si légitimement […] je souhaite bien que vous ayez satisfaction dans votre voyage d'Italie. Je souhaiterais bien, qu'après ce temps de pèlerinage, vous passassiez dans une plus heureuse transmigration, et telle que votre mérite personnel le demande").

Named provost of Livorno by the Grand Duke François II of Lorraine upon his return to Tuscany, Venuti took it upon himself to animate the literary life of that port city which was an active center of exchanges and circulation of ideas. From 1754 to 1757, he directed a new monthly, the Magazzino toscano d'istruzione e di piacere, which illustrates the progressive passage from traditional erudition, turned toward the study of the past, to an approach engaging questions of the present, the life of the city, the public good, the diffusion of knowledge. With this success, he went on to participate from 1756 to 1759 in the republication of the Encyclopédie in Lucca , in French, with the addition of original notes. Curious about everything and constantly critical with respect to a type of retrograde, elitist education, he is representative of that class of scholars that constituted the intellectual foundation of the philosophy of the Enlightenment and allowed new ideas to reach aspiring men of letters.

Bibliography

Robert Shacketon, "Filippo Venuti, académicien de Bordeaux et ami de Montesquieu", Actes de l'académie de Bordeaux, 4 e série, t. XX, 1965, p. 53-62. Pierre Musitelli, "Filippo Venuti, ami de Montesquieu et collaborateur de l'édition lucquoise de l'Encyclopédie", Dix-Huitième Siècle 38 (2006), https://www.cairn.info/revue-dix-huitiemesiecle-2006-1-page-429.htm.

-, "Dall'antiquaria all'enciclopedismo: l'itinerario di Filippo Venuti tra Francia e Toscana nel secolo dei Lumi", Annuario dell 'Accademia etrusca 32 (2006-2007), Cortone, 2008, p.117-148.


Laurence A F Park
email: lapark@scem.westernsydney.edu.au

Jesse Read

A Blended Metric for Multi-label Optimisation and Evaluation

In multi-label classification, a large number of evaluation metrics exist, for example Hamming loss, exact match, and Jaccard similarity -but there are many more. In fact, there remains an apparent uncertainty in the multi-label literature about which metrics should be considered and when and how to optimise them. This has given rise to a proliferation of metrics, with some papers carrying out empirical evaluations under 10 or more different metrics in order to analyse method performance. We argue that further understanding of underlying mechanisms is necessary. In this paper we tackle the challenge of having a clearer view of evaluation strategies. We present a blended loss function. This function allows us to evaluate under the properties of several major loss functions with a single parameterisation. Furthermore we demonstrate the successful use of this metric as a surrogate loss for other metrics. We offer experimental investigation and theoretical backing to demonstrate that optimising this surrogate loss offers best results for several different metrics than optimising the metrics directly. It simplifies and provides insight to the task of evaluating multi-label prediction methodologies.

Introduction

The major challenge in multi-label classification is dealing with multiple output labels simultaneously, which has important ramifications on building models, and also evaluating them. There have been an impressive number of new methods proposed in recent years, proposing different ways of modelling labels together, but relatively little investigation into the study of which loss functions methods optimise, which functions they should optimise, and how well they can be expected to achieve this for a given problem. As a result, empirical studies look at up to a dozen evaluation metrics. Our study is targeted at bringing new clarity and insight to this situation.

In multi-label classification, optimisation as part of a predictive model inherently involves multiple dimensions; one for each label. A review of multi-label classification is given in [START_REF] Tsoumakas | Mining multi-label data[END_REF]. A number of common benchmark algorithms are reviewed in [START_REF] Zhang | A review on multi-label learning algorithms[END_REF] and [START_REF] Madjarov | An extensive experimental comparison of methods for multi-label learning[END_REF]. Often, a vector is used to represent the labelling, e.g., y k = [y k1 , y k2 , . . . , y kL ] where y kj = 1 iff the j-th of L labels is relevant to the k-th example x k (and y kj = 0 otherwise). To evaluate the performance of a multi-label classifier, typically predicted vectors y k must be compared the vector of true labels t k over all examples k = 1, . . . , K (for a test set of K examples).

Three of the most common similarity functions used in multi-label learning and evaluation are the Jaccard index, Hamming loss, and 0/1 loss. Jaccard index is known as accuracy in some publications, e.g., [START_REF] Godbole | Discriminative methods for multi-labeled classification[END_REF][START_REF] Read | Classifier chains for multi-label classification[END_REF], Hamming loss and 0/1 loss are known often as Hamming score and exact match in their payoff-form (higher is better), respectively [START_REF] Park | Using entropy as a measure of acceptance for multilabel classification[END_REF]. However the basic principal of all multi-label metrics is the same for any metric: provide a single number indicating the similarity of the set (or vector 3 ) of predicted labels compared to the set of true labels, i.e., a score that may be normalised to between 0 and 1. We henceforth refer to each of these mentioned metrics as a similarity or loss function, interchangeably. There is a large number of multi-label criteria, including rankings and micro and macro evaluation; a recent survey of multi-label metrics and unified view of them is given by [START_REF] Wu | A unified view of multi-label performance measures[END_REF].

Consider the examples in Table 1. The similarity functions are defined, for the k-th instance, as

Hamming := 1 L L j=1 I[y kj = t kj ] (1)
Exact := I[y k = t k ] (2)
Jacard :=

|y k ∧ t k | |y k ∨ t k | (3)
for L possible labels, where ∨ and ∧ are the logical OR and AND operations, applied vector-wise, and I[•] is an indicator function returning 1 if the inner condition holds (0 otherwise). Each of these similarity functions measure different accuracy qualities of a multi-label classification system. Hamming similarity provides the proportion of labels predicted correctly, Exact similarity provides the proportion of label sets predicted correctly, while Jaccard similarity only examines the proportion of correctly predicted positive labels out of the potential positive set (predicted positive and actually positive). Therefore, a multi-label classification system should be optimised according to the desired similarity function.

The problem of optimisation in multi-label classification is complicated by the label dimension and the interdependence of labels. Resulting search spaces are usually non-convex and non-differentiable, and all the difficulties of such a search are inherited. A solution may fall into a local maximum/minimum and provide a non-optimal solution. We can transform the problem into simpler problems which Table 1. An example of multi-label evaluation. The average Hamming, Exact, and Jaccard similarity is 0.80, 0.40, and 0.67, respectively. Note that real-world multi-label data sets are typically much sparser with respect to labelling.

t k y k Hamming Exact Jaccard x1 [1 0 1 0] [1 0 0 1] 0.5 0 0.33 x2 [0 1 0 1] [0 1 0 1] 1 1 1 x3 [1 0 0 1] [1 0 0 1] 1 1 1 x4 [0 1 1 0] [0 1 0 0] 0.75 0 0.5 x5 [1 0 0 0] [1 0 0 1] 0.75 0 0.5
provide convex search spaces, but even in these case, optimisation is typically much more difficult than a traditional single-label problem.

One solution is to optimise each label individually and independently. This approach is known as Hamming similarity, and defines the so-called binary relevance method, widely known across the multi-label literature as a baseline approach. Indeed we see that Hamming similarity is proportional to the sum of individual label similarities, therefore, when optimising for Hamming similarity, we can simply optimise the accuracy of each label. An example of this is to model each label with a logistic regression; resulting in L tasks of convex-optimisation.

However, it is typical to motivate methods that model labels together (e.g., [START_REF] Godbole | Discriminative methods for multi-labeled classification[END_REF][START_REF] Read | Efficient Monte Carlo methods for multidimensional learning with classifier chains[END_REF][START_REF] Read | Classifier chains for multi-label classification[END_REF][START_REF] Zhang | A review on multi-label learning algorithms[END_REF][START_REF] Tsoumakas | Random k-labelsets for multi-label classification[END_REF] and many references therein). As shown thoroughly in those papers and many others, it is desirable to model an explicit or implicit dependence among labels. And this, in turn, motivates the evaluation of labels together also. Exact similarity does not decompose across labels (i.e., it requires optimisation of all labels jointly) and therefore encourages modelling labels together. In fact, for this metric it is theoretically optimal to model label combinations as single class values in a large multi-class problem [START_REF] Dembczyński | On label dependence and loss minimization in multi-label classification[END_REF]. This transformation is often called the label powerset method. Hence each vector/set y k is treated as a single value that may be assigned to each instance. This can also be formulated as a differentiable and convex optimization problem, such as multi-class logistic regression, since the multiple classes can be modelled under a single softmax function. Nevertheless, optimisation is still not expected to be easy or necessarily effective in practice (see, e.g., [START_REF] Read | Multi-label classification with meta labels[END_REF][START_REF] Tsoumakas | Random k-labelsets for multi-label classification[END_REF]): most label combination (class) are likely to be very sparse or not present in the training data and the metric is particularly sensitive to noise in the data, since even 1 of the L labels being incorrectly predicted will lead to a 0 score for that particular instance. Other methods such as classifier chains [START_REF] Read | Classifier chains for multi-label classification[END_REF][START_REF] Dembczyński | Bayes optimal multilabel classification via probabilistic classifier chains[END_REF] provide an alternative, which divides the optimisation across labels. However, unlike the binary relevance method, the labels are linked together in a chain cascade and thus can no longer be optimized separately (at least, not in terms of optimising exact similarity), and thus becomes thus a non-convex problem and remains a difficult task, hence paving the way for several extensions (e.g., [START_REF] Read | Efficient Monte Carlo methods for multidimensional learning with classifier chains[END_REF][START_REF] Dembczyński | Bayes optimal multilabel classification via probabilistic classifier chains[END_REF][START_REF] Dembczyński | On label dependence and loss minimization in multi-label classification[END_REF]) with approximate inference, and alternatives (e.g., [START_REF] Tsoumakas | Random k-labelsets for multi-label classification[END_REF][START_REF] Read | Multi-label classification with meta labels[END_REF]) with exact inference on small sub-problems.

Jaccard similarity is also used for evaluating labels together. It is often preferred since it can be seen as a midpoint between Hamming and exact similarity as two extremes. However, fewer theoretical results exist for its optimisation, although relationships with F-measure has been outlined [START_REF] Waegeman | On the bayes-optimality of f-measure maximizers[END_REF]. It remains popular, and it is typically assumed that good results in both Hamming and exact similarity will reflect good results also in Jaccard similarity. Numerous empirical studies also show this [START_REF] Read | Efficient Monte Carlo methods for multidimensional learning with classifier chains[END_REF][START_REF] Madjarov | An extensive experimental comparison of methods for multi-label learning[END_REF].

Therefore, to summarise so far: Hamming similarity is easy to optimise, but may not correspond to labellings that we would find desirable in real-world problems. On the other hand, optimising a multi-label classification system with respect to Jaccard or Exact similarity is difficult, since it requires optimisation of all labels at once, leading to a large optimisation search space. Solutions involve:

-Problem transformation (i.e., binary relevance, or label powerset) -Non-convex optimisation such as local search and hill-climbing methods -Using a surrogate loss -Smoothing and regularisation of the loss to improve results.

In this paper we propose a blended metric, combining Jaccard and Exact similarity functions with the Hamming function as a surrogate metric for both Jaccard and Exact similarity, providing a simpler search space and thus more efficient and effective optimisation. Under a series of investigations we show the benefit of this blended metric. In particular, we obtain a very important result: best results for exact match can be obtained using our surrogate blended metric, rather than the exact match minimizer (label powerset) itself. In general, we are able to conclude making the recommendation of exploring different surrogate metrics, rather than optimising separately across an ever-growing set of metrics.

Our novel contribution is covered as follows:

-In Section 2 we formulate our surrogate blended metric: a single function that is a blend of the three similarity functions (Hamming, Exact, and Jaccard). -In Section 3 we derive the gradient for optimisation of this function.

-In Section 4 and Section 5 we examine how optimisation over the blended function affects the accuracy with regard to the maximisation of the three similarity functions, on a number of real-world multi-label datasets; and we identify the spectrum where each of the three similarity functions are maximised and we discuss in detail and draw conclusions and recommendations.

Multi-label Evaluation and Undetermined Predictions

To begin our investigation, we formulate the blended Hamming, Jaccard, Exact similarity function and show that it behaves well when using undetermined label values (such as votes or probabilistic predictions ∈ [0, 1] for each label).

A Blended Metric

Hamming and Jaccard similarity can be represented in terms of true/false positive/negative counts4 :

Hamming = TP + TN L Jaccard = TP TP+FP+FN if TN = L 1 otherwise
where TP + TN + FP + FN = L (the total number of labels). We also note that exact similarity can also be represented in terms of Hamming and Jaccard similarity.

Exact = lim

β→∞

Hamming β = lim β→∞ Jaccard β
Of the three similarity functions, Hamming similarity is the simplest to optimise over. Hamming similarity is the mean of the accuracy of each label prediction and therefore, each label can be optimised individually (details in [START_REF] Dembczyński | On label dependence and loss minimization in multi-label classification[END_REF]). Both Jaccard and Exact similarity depend on the collection of labels, so the optimisation is non-decomposable. We reason that a blending of Hamming similarity with either Jaccard or Exact similarity provides a greater optimisation of Jaccard or Exact similarity. The insight to this is that the Hamming optimisation search space is likely to be less chaotic than the Jaccard and Exact optimisation search spaces, providing a simpler optimisation path. On the other hand, this may lead to non-optimal solutions. Therefore we will investigate this effect throughout the rest of this paper.

We propose the blended Hamming, Jaccard similarity function:

s(y, t; α) = TP + αTN TP + FN + FP + αTN (4) = TP + αTN L -(1 -α)TN (5)
given the vector of true labels t, the vector of determined or undetermined predictions y, and a parameter α which controls the blending. This metric provides Hamming similarity when α = 1 and Jaccard when α = 0. We additionally note that this form avoids the divide-by-zero problem of the Jaccard similarity. By taking the limit as α → 0, the equation results in s(y, t; α → 0) = TN/TN = 1, when TP, FN and FP are zero.

Behaviour using undetermined labels

The blended similarity function is a function of true/false positive/negative counts. Optimising over these functions is difficult due to the discrete nature of these counts, which leads to a discontinuous optimisation function and a gradient containing values of either zero of infinity. Of course, many multi-label classifiers compute undetermined label values z kj in the [0, 1] range, that are used to obtain the label predictions y kj ∈ {0, 1}, such that

y kj = I [z kj ≥ τ j ] (6)
is the class prediction for the j-th label (I[•] = 1 if the inner condition holds), where τ j is some threshold, typically set to 0.5 but may also be tuned (see, e.g., [START_REF] Largeron | MCut: A Thresholding Strategy for Multilabel Classification[END_REF][START_REF] Read | Classifier chains for multi-label classification[END_REF]) either per label or the same for all labels j = 1, . . . , L. The vector z k = [z k1 , . . . , z kL ] contains the undetermined label values and in many cases z kj ≈ P (y kj = 1|x k ) is related to probabilistic models or approximations 5 , or is a normalised sum of ensemble votes (e.g., [START_REF] Tsoumakas | Random k-labelsets for multi-label classification[END_REF][START_REF] Read | Multi-label classification with meta labels[END_REF][START_REF] Dembczyński | Bayes optimal multilabel classification via probabilistic classifier chains[END_REF]). Using these undetermined label values z kj to obtain undetermined true/false positive/negative values, provides a continuous optimisation space, and a gradient for gradient-based optimisation. For example, given the three labels y k = [1 0 1] and the predicted undetermined label values z k = [0.6 0.4 0.2], the undetermined values would be rounded to obtain determined the values y k = [1 0 0], giving TP = 1, FP = 0, FN = 1, TN = 1. Using the undetermined values instead, we get TP = 0.8, FP = 0.4, FN = 1.2, TN = 0.6. For both cases TP + TN + FP + FN = 3.

The undetermined values can also be used for evaluation, such as in the case of the log loss metric (applied to multi-label evaluation in, e.g., [START_REF] Read | Classifier chains for multi-label classification[END_REF]). Nevertheless, such metrics are relatively less popular in the multi-label literature, perhaps because not all classifiers can provide them. Nevertheless, they help considerably in the optimisation to smooth out the search space.

Blending the Exact Similarity

TP and TN are dependent only on the correctness of the predictions (the correctly predicted 1 and 0 values), so we instead use the vector of correctness values p k = [p k1 , . . . , p kL ], where p ki (each element of p k ) is equal to

z ki when t ki = 1 1 -z ki when t ki = 0 (7)
where t ki is the true label. The combined Hamming-Jaccard score using undetermined labels can now be represented as

s(z k , t k ; α) = p k a k L + p k b k
where a k contains 1 when true label t ki = 1 and α when true label is 0; and b k contains 0 when the true label t ki = 1 and α -1 when the true label is 0. The set of vectors a k and b k are constant for the optimisation.

To include the Exact similarity into this spectrum, we add the parameter β

s(z k , t k ; α, β) = p k a k L + p k b k β
which is equivalent to Hamming similarity when α = 1, β = 1, and equivalent to Jaccard similarity when α → 0, β = 1 and to Exact when β → ∞. A continuous spectrum exists between the three similarity functions for α ∈ (0, 1] and β ∈ [1, ∞).

Given that we obtain the Exact similarity as β → ∞, let us look at how α effects the rate at which the limit is approached, and if so which value of α provides the fastest rate of convergence. Namely, if α is adjusted to α + δ, where δ > 0, the score changes to:

s(z k , t k ; α + δ, β) = p k a k + δ(1 -t k ) L + p k b k + δ(1 -t k ) β (8)
This value is bound between 0 and 1, so this addition of δ(1-t k ) to the numerator and denominator will increase the score. And, thus,

s(z k , t k ; α, β) ≤ s(y k , t k ; α + δ, β) for 0 ≤ α ≤ [α+δ] ≤ 1.
The limit as β → ∞ will approach the correct Exact score faster when any score that is not 1, is closer to zero (since Exact requires that all fractional scores should be mapped to zero). Since decreasing α decreases the score, the limit will approach the correct score faster for smaller α. Therefore, we expect that the best estimate of Exact will occur when α → 0 and β → ∞.

Optimising the combined metric

In this section, we present the linear model, the optimisation function used to fit the linear model, a derivation of the optimisation gradient and inclusion of a penalty to deter over-fitting.

Optimisation function

Suppose the linear model

log z k 1 -z k = W x k
with unknown matrix W , undetermined prediction vector z k ∈ (0, 1) L and feature vector x k ∈ R M . This model resembles a set of L parallel logistic regressions, but unlike logistic regression, we are not determining the W that maximises the likelihood of the data. We compute W that maximises the chosen evaluation function score s(z k , t k ; α, β) for all k. To optimise the average score, the optimisation problem becomes:

max W 1 K K k=1 s(z k , t k ; α, β) (9)
where W is the weight matrix containing the elements w ij , s k ∈ [0, 1) is the score for each of the K objects.

s(z k , t k ; α, β) = s k = L i=1 p ki a ki L + L i=1 p ki b ki β
L is the number of labels, p ki ∈ (0, 1) is the correctness of the undetermined label prediction z ki (recall Eq. ( 7))

p ki = z t ki ki (1 -z ki ) (1-t ki )
and z ki ∈ (0, 1) is the sigmoid of the mapped feature vector x k

z ki = 1 1 + exp -j w ij x kj
where x kj is an element of the feature vector x k , and the element 1 is appended to x k as a bias term. The constants a ki ∈ {α, 1} and b ki ∈ {0, 1 -α} depend only on the optimisation parameter α and the true label values t ki .

a ki = t ki + α(1 -t ki ) b ki = (α -1)(1 -t ki )
α ∈ (0, 1] and β ∈ [1, ∞) are parameters to set the desired optimisation (e.g. α = 1, β = 1 for Hamming similarity). Also note that a ki -b ki = 1.

The optimisation of Eq. ( 9) provides us with W which we can use to make determined predictions by

y = sign (W x k )
where sign returns the L signs of the L elements in W x k , making sure that 1 is appended to x if it was used to compute a bias term during optimisation. Note that the bias term has the same role as the threshold described in Section 2.2.

Metric gradient

We can optimise the similarity function using gradient descent or stochastic gradient descent for large problems. To do so, we need the gradient of the function with respect to each elements of the matrix W . The derivative of the sigmoid function is

dz ki dw ij = x kj z ki (1 -z ki )
The derivative of the correctness function is

dp ki dz ki = p ki t ki z ki - 1 -t ki 1 -z ki
and the derivative of the score function with respect to the correctness is

ds k dp ki = βs k a ki L i=1 p ki a ki - b ki L + L i=1
p ki b ki Combining the partial derivatives provides the derivative of the score function with respect to the unknown weights;

ds k dw ij = ds k dp ki dp ki dz ki dz ki dw ij = βs k p ki (t ki -z ki ) a ki L i p ki a ki - b ki L + L i p ki b ki x kj (10)
Using gradient descent, we update weights W until the optimisation function stops increasing,

w ij ← w ij + λ ds k dw ij
where λ controls the rate of convergence. Note that if α = 1 and β = 1 (giving Hamming similarity), we obtain

a ki = 1, b ki = 0 and s k = L i p ki /L giving the gradient ds k dw ij = p ki (t ki -z ki )x kj L = z ki (1-z ki )x kj L if t ki = 1 -z ki (1-z ki )x kj L if t ki = 0 = (-1) 1-t ki L dz ki dw ij
Thus we see that the gradient that optimises Hamming similarity with respect to w ij is independent of the weights for other values of i, meaning that each label can be fitted independently.

Optimisation penalty

To reduce the chance of over-fitting the training data, we include a penalisation term. Thus Eq. ( 9) becomes max

W 1 K K k=1 s(z k , t k ; α, β) - γ 2 i=1 j=1
w 2 ij penalty for some positive γ, typically chosen via cross-validation. We make a corresponding minor adjustment to the gradient, hence Eq. ( 10) becomes

ds k dw ij = βs k p ki (t ki -z ki ) a ki L i p ki a ki - b ki L + L i p ki b ki x kj -γw ij

Parameter investigation

Having proposed a parametrisable blended metric, and derived its optimiser, we now turn to explore the question: Does blending the Hamming similarity with either Jaccard or Exact similarity provide a greater optimisation of Jaccard or Exact similarity itself?

We answer this question by optimising the model over different data sets, using different α, β parameter combinations and examining the results using the three similarity functions.

Namely, we use the Emotions, Enron, Scene, Slashdot, Stare and Yeast multilabel data sets 6 . Each of these data sets are commonly used for multi-label machine learning except for Stare, which is a data set for detecting cardiovascular disease from retinal features [START_REF] Nguyen | An automated method for retinal arteriovenous nicking quantification from color fundus images[END_REF]. A 50/50 train/test split is used and the models are optimised over the training portion. We experiment with values of α from 0.1 to 1 in increments of 0.1 and also α = 0.00001 to approximate the limit of α approaching zero (α = 0 was not used to avoid the problems associated to the Jaccard metric). We also experiment with eight values β ∈ {2 0 , . . . , 2 7 } (higher values lead to approximately zero gradients), giving 88 α × β combinations. The penalty parameter γ was computed using cross validation. Each of the 88 optimised models for each data set was evaluated using Hamming, Jaccard and Exact similarity to examine the effect of the parameters on the evaluation scores. The optimisation was performed using undetermined scores, but the evaluation scores (shown in the figures) are computed using the final determined labels. The results for the training data are shown in Figure 1, and the results for the testing data are shown in Figure 2. The top six plots in each figure show the Hamming similarity on each of the six data sets, the middle six show the Jaccard similarity, and the bottom six show the Exact accuracy. The shade of each block in a given plot shows the accuracy of the model optimised using the set α, β parameters. For example, the first plot in Figure 1 shows the Hamming similarity on the training portion of the Emotions data; the bottom row of the plot shows the grey level transitioning from light to dark grey, meaning that when β = 1 (log 2 β = 0) the Hamming accuracy increases as α changes from 0 to 1.

Results and Discussion

The top sections of Figures 1 and2 show the mean Hamming similarity between the predicted and true label sets for the six data sets on the training and testing portions. Each block in the figures show the effect of changing α and β on the Hamming similarity. We would expect that the optimal configuration for Hamming similarity is α = 1, β = 1 (the lower right corner of each block plot) since it optimises Hamming similarity. We can see that this is the case for most of 2. The location (α, β) of the optimal Jaccard scores, and their Base score (α = 0, β = 1). A dash means that the optimal score was located at the Base score position.

the data sets for both training and testing. For the remaining cases, we find that the accuracy difference between the α = 1 and β = 1 and optimal configuration is small (of the order of 0.01 or less).

The middle sections of Figures 1 and2 show the Jaccard scores for each of the training and testing data sets. The model optimises over Jaccard similarity when α → 0 and β = 1. Therefore it is expected that the Jaccard scores are optimal or close to optimal for precisely these values (bottom left corners of the block plots).

The plots show that the Jaccard score is high in the lower left corner, but it also remains high as both α and β increase (moving diagonally along the block plot). Slashdot training is a notable exception, where the lower left corner seems to be a "sweet spot", obtaining an approximate 0.2 increase in Jaccard similarity over the rest of the plot. We note that this is not as dramatic for the Slashdot test data, but the lower left corner still provides a high Jaccard score relative to the remaining configurations.

The lower sections of Figures 1 and2 show the Exact match scores for each of the training and testing sets. Exact similarity is being optimised when β → ∞ (the top of the plots). However, these plots show interesting and surprising behaviour, with many of them being similar to the Jaccard plots. In other cases there does not seem to be consistency in the optimal regions, which seems to be located at about α = 0.5, with fluctuating values of β.

We now proceed to examine the difference in accuracy of the optimal α, β configuration and expected α, β configuration. Tables 2, 3 and 4 contain these details for the Jaccard, Hamming and Exact similarity functions.

Table 2 shows the Jaccard scores, where Base is the score for the expected optimal configuration (α → 0, β = 1). We find that seven of the twelve data sets provide the optimal score at the expected configuration, while the remaining five provide a very slight increase (at most 0.004) over the Base score. Thus we confirm that when evaluating using Jaccard similarity, we should optimise with respect to Jaccard similarity. Table 3 shows the Hamming scores, where Base is the score for the expected optimal configuration (α = 1, β = 1). Similar conclusions can me made for Hamming similarity: when evaluating using this metric, we should likewise optimise using Hamming similarity; as expected.

Table 4 shows the Exact scores, where BaseH and BaseJ give the scores for the expected optimal configuration (α = 1 or α → 0, β = 2 7 ). Only one of the data sets provides the optimal score at the expected β = 2 7 . We also find that there are large differences between the optimal scores and the BaseH and BaseJ scores. There are many values of small α (nine values ≤ 0.2). But it is clearly seen that the selection of α and β for optimal Exact similarity is dependent on the data. Therefore, our findings may be summarised as the following recommendations: if optimising Hamming: use α = 1, β = 1; if Jaccard: use α → 0, β = 1; and -we emphasise -for Exact similarity: we should in fact explore the α and β space. This has important and far-reaching implications, since Exact similarity is a widely used metric, and often used to promote novel classifiers because classifiers that model labels together are more likely to outperform the independent baseline under this metric. With a more careful and exploratory optimisation scheme as we define, we have showed how it is possible to achieve even higher predictive performance.

Conclusions and Future Work

We have analysed multi-label evaluation, and outlined the difficulties in optimising several of the well-known loss metrics. To tackle the issues that arise in multi-label optimisation, we proposed a surrogate loss, in the form of a blended metric. This blended metric forms a smooth spectrum between the Hamming and exact match metrics, where it falls depending on its parameterisation. Using particular parameterisations of this function, we show that optimisation is more effective, on account of its smoothness. Indeed, for example we demonstrated that one can obtain better results under exact match by optimising the blended metric, than by optimising exact match directly (which is difficult to do). This is important because exact match is a common metric in the literature and many empirical evaluations are based around it. We also made a series of other recommendations to multi-label researchers, in reflection of our findings.

To fully evaluate the potential of our proposal under a complete range of contexts, further experimental comparison will be necessary, for example with structured prediction like probabilistic classifier chains under different inference algorithms, and structural SVMs. We cannot claim that our proposal optimises in a Bayes optimal way, due to the approximation used; Further theoretical analysis is needed on this front, particularly under exact similarity, Jaccard, and F-measure. Finally, we can point out that with some modification, the blended metric could be used to maximize F-measure -a promising line of future investigation.

Fig. 1 .

 1 Fig. 1. The Hamming (top section), Jaccard (middle section) and Exact (bottom section) training similarity of the (top row from left to right in each section) Emotions, Enron, Scene (bottom row from left to right in each section) Slashdot, Stare and Yeast data, optimised using the shown values of α and β.

Fig. 2 .

 2 Fig. 2. The Hamming (top section), Jaccard (middle section) and Exact (bottom section) testing similarity of the (top row from left to right in each section) Emotions, Enron, Scene (bottom row from left to right in each section) Slashdot, Stare and Yeast data, optimised using the shown values of α and β.

Table 3 .

 3 The location (α, β) of the optimal Hamming scores, and their Base score (α = 1, β = 1). A dash means that the optimal score was located at the Base score position.

	Data	Type	α	β	Hamming Base
	Emotions	Train Test	1 1	2 1	0.827 -	0.823 0.783
	Stare	Train Test	0.6 0.3	4 2	0.946 0.927	0.944 0.927
	Scene	Train Test	0.7 0.8	4 1	0.916 0.901	0.912 0.899
	Yeast	Train Test	0.9 1	1 2	0.804 0.799	0.804 0.798
	Slashdot	Train Test	1e-05 1e-05	1 1	0.968 0.954	0.959 0.953
	Enron	Train Test	0.6 1e-05	8 1	0.954 0.950	0.952 0.947

Table 4 .

 4 The location (α, β) of the optimal Exact scores, and their Jaccard Base score BaseJ (α = 0, β = 128) and Hamming Base score BaseH (α = 1, β = 128).

	Data	Type	α	β	Exact BaseJ BaseH
	Emotions	Train Test	1 0.5	4 4	0.348 0.296 0.296 0.297 0.265 0.265
	Stare	Train Test	0.1 1e-05	8 4	0.500 0.470 0.490 0.439 0.427 0.427
	Scene	Train Test	1e-05 0.7	2 4	0.624 0.526 0.532 0.545 0.480 0.489
	Yeast	Train Test	0.1 0.2	8 16	0.192 0.185 0.186 0.194 0.191 0.190
	Slashdot	Train Test	1e-05 1e-05	1 1	0.553 0.351 0.351 0.360 0.300 0.301
	Enron	Train Test	0.1 0.2	8 128 0.239 0.205 0.229 0.150 0.115 0.137

Notation alternates among papers, since given a set of labels L, a (sub)set Y ⊆ L can be represented as vector y = [y1, . . . , yL] where yj = 1 ⇔ yj ∈ Y

Throughout this work we refer to F-measure and Jaccard in an instance-wise evaluation context (noting that these metrics can also be used in a micro-or macroaveraging context)

Note, however, that metrics like F-measure and Jaccard measure cannot be optimised only under consideration of marginal probabilities.

All available from http://mulan.sourceforge.net/datasets-mlc.html, https:// sourceforge.net/projects/meka/files/Datasets/ (Slashdot), and http://www. ces.clemson.edu/ ~ahoover/stare/ (Stare).