Mathieu Paquier
email: mathieu.paquier@univ-brest.fr

Vincent Koehl

Discriminability of the placement of supra-aural and circumaural headphones

Keywords: Audibility, Headphone, placement

published or not. The documents may come   L'archive ouverte pluridisciplinaire

INTRODUCTION

Sound reproduction over headphones is used in numerous applications such as sound quality assessment, psychoacoustic experiments, domestic use, audio engineering, binaural rendering, etc [START_REF] Møller | Fundamentals of binaural technology[END_REF][START_REF] Gierlich | The application of binaural technology[END_REF]. When choosing a headphone model according to a specific use, attention is paid to its type and especially to the quality of its transducers. Nevertheless, the coupling between the headphone and the listener's ears is not taken into account apart from the fact that the user might prefer for example a circumaural open headphone over a supra-aural closed one.

The HeadPhone Transfer Function (HPTF) describes both the headphone response and the coupling to a listener's ear. For binaural rendering (based on recordings in ears or synthesis by convolving monophonic signals with HRTFs), the HPTFs can be measured, averaged (for repeated measurements) and inverted to compensate for the headphone influence so that the intended binaural signals are recreated at the listener's ear. According to Pralong and Carlile [START_REF] Pralong | The role of individualized headphone calibration for the generation of high fidelity virtual auditory space[END_REF], the equalization needs to be specific to the listener. They found significant inter-individual differences in the 4 to 10 kHz range when measuring the HPTFs of 10 subjects equipped with the same headphone by using an in-ear recording system. In addition, they showed that the use of non-individualized equalization can lead to errors in localization tasks. On the other hand, Lindau and Brinkmann [START_REF] Lindau | Perceptual Evaluation of Headphone Compensation in Binaural Synthesis Based on Non-Individual Recordings[END_REF] showed that non-individual binaural recordings were surprisingly perceived as most realistic when compensated using the HPTFs of the recording subject.

Nevertheless, the signals being equalized or not -as it is the case for stereo recordings listened to over headphones and even for numerous cases of binaural reproduction -the scattering caused by differences in the headphone placement over the listeners' ears is not taken into account. However, it has been shown that slight modifications in the headphone placement can lead to large spectral differences, especially above 8000 Hz, where the quarter wavelength of sound is less than the length of the ear canal, and where standing wave patterns create large variations in sound pressure at different points within the canal. Moreover the large spectral difference of blocked-ear-canal HPTF at high frequency is caused by the difference in the coupling of headphone and pinna [START_REF] Zhou | Reliability of pure-tone thresholds at high frequencies[END_REF].

Toole [START_REF] Toole | The acoustics and psychoacoustics of headphones[END_REF] reported that these differences are less than 5 dB below 2 kHz, but ranged from 8 to 15 dB above 4-5 kHz. These differences were observed for 3 successive headphone (4 different models) placements on real and dummy heads (3 each). Wightman and Kistler [START_REF] Wightman | Headphone simulation of free-field listening. I: Stimulus synthesis[END_REF] measured the HPTFs on 10 humans for 10 headphone placements and Pralong and Carlile [START_REF] Pralong | The role of individualized headphone calibration for the generation of high fidelity virtual auditory space[END_REF] did the same on 10 humans and 1 manikin for 6 headphone placements. Both studies reported that standard deviations of the magnitudes could reach up to 5 dB from 200 Hz to 14 kHz. McAnally and Martin [START_REF] Mcanally | Variability in the headphone-to-ear-canal transfer function[END_REF] measured HPTFs for 20 headphone placements on 6 human heads. Standard deviations were generally smaller than 2.5 dB for frequencies up to 10 kHz, and were as high as 9 dB above 10 kHz. Kulkarni and Colburn [9] also observed a standard deviation of 9 dB on HPTFs measured for 20 headphone placements on an acoustic manikin for frequencies ranging from 9 to 14 kHz.

The perception of the sound scene might also be altered by HPTF variability if the localization cues are modified as a result. The variability of the HPTF group delays being less than the minimum discriminable interaural time difference [START_REF] Kulkarni | Variability in the characterization of the headphone transfer function[END_REF], the potentially audible modifications of HPTF would be exclusively spectral. Kulkarni and Colburn [START_REF] Kulkarni | Variability in the characterization of the headphone transfer function[END_REF] as well as McAnally and Martin [START_REF] Mcanally | Variability in the headphone-to-ear-canal transfer function[END_REF] showed that HPTF and HRTF can exhibit similar spectral features. Martin et al. [START_REF] Martin | Free-field equivalent localization of virtual audio[END_REF] have assessed the ability of listeners to localize sound presented using a virtual audio display that enabled listener-specific equalization based on a single HPTF measurement. They showed that listeners were able to localize virtual sounds with an accuracy equivalent to free-field conditions for eight headphone placements. The headphone placement seemed thus to have a minor influence on this localization task. The variability observed in the HPTF magnitudes (characterized by high-Q peaks and dips in high frequencies) is highly reduced when applying a cochlear filter model on such frequency responses. McAnally and Martin [START_REF] Mcanally | Variability in the headphone-to-ear-canal transfer function[END_REF] reported that the variability observed in the magnitudes of filtered HRTFs is generally considerably higher than the one observed in the magnitudes of filtered HPTFs. This suggests that the spectral information used by listeners to localize sound is unlikely to be masked by the variability of the HPTF magnitude.

However, even though the variability of HPTFs across headphone placements does not have an adverse effect on localization task, it could still be perceived another way. As an example, since degraded or inadequate HRTF can enhance the "in head" sensation often evoked by headphone listeners [START_REF] Wang | An "out of head" sound field enhancement system for headphone[END_REF][START_REF] Breebaart | Phantom materialization: A novel method to enhance stereo audio reproduction on headphones[END_REF], one can think that modifications of HPTF could have the same effect. As another example, headphone are often used for audiometric testing, and placement variability could possibly affect hearing threshold measurements, which was evidenced several past studies [START_REF] Steinberg | Deviations in the loudness judgments of 100 people[END_REF][START_REF] Atherley | Reliability of auditory threshold determinations using a circumaural-earphone assembly[END_REF], but not by Gauz et al. [START_REF] Gauz | High-frequency Békésy audiometry: III. Reliability and validity revisited[END_REF]. Besides, at higher levels, a modification of the timbre could also be perceived because of resonances [START_REF] Toole | The modification of timbre by resonances: Perception and measurement[END_REF].

The aim of the present study is to evaluate whether realistic changes in the headphone placement (i.e. that could occur when anyone places a headphone on his own ears) can lead to noticeable changes in the sound perception. A blind test in which the listener would have to compare stimuli by placing/replacing the headphone over his own head is inherently impossible. So, in the same way as for loudspeaker comparisons [START_REF] Toole | Binaural record/reproduction systems and their use in psychoacoustic investigations[END_REF][START_REF] Hiekkanen | Virtualized listening tests for loudspeakers[END_REF], the different headphone placements have to be recorded beforehand and played back over a fixed headphone. Three different monophonic sequences (one pink noise and two music excerpts) were played over 4 different headphone models and recorded with a dummy head. The omnidirectional microphones were located at the entrance of the blocked ear canal. The recorded sequences were then played back to expert (experiment I with 4 recorded headphone models) and naïve (experiment II with 2 recorded headphone models) listeners on a unique headphone for the whole test. These sequences were also filtered to compensate for the HPTFs of the test headphone. The listener's task was to compare recordings differing only because of the headphone placement at the recording step.

MATERIAL AND METHODS

2.1.Program material

Three short excerpts were used in this study. The first excerpt was a 3.5 s-long pink noise and two music excerpts were selected from commercially available stereo material. They were extracted from CDs as 16bit, 44.1-kHz Wave format files. The second excerpt (Ben Harper, 5 s) included drums, an acoustic guitar, a male human voice and choir voices. The third excerpt (Leonard Bernstein, 4 s) included a symphonic orchestra. These two music excerpts were chosen as symphonic music appeared to be more discriminant than popular music for resonance detection [START_REF] Fryer | Intermodulation distortion listening tests[END_REF]. Moreover the masking effect is different with these two exerpts [START_REF] Koehl | A comparative study on different assessment procedures applied to loudspeaker sound quality[END_REF]. In order to allow the listener to concentrate on timbral modifications (and not on spatial ones) that could be introduced by differences in the headphone placement, monophonic excerpts were considered. Therefore, the left channels only were kept for the two music excerpts. The spectra of the two music excerpts are depicted in Fig. 1. The 3 excerpts were then diotically displayed to a dummy head using the various headphone models under test.

2.2.Recordings

The recordings were made by using a dummy head (Neumann KU 100) whose microphones (omnidirectional) are located at the entrance of the blocked ear canal. Several studies have highlighted the benefits of this type of recording: from this point and up to the eardrum, the complete spatial information is included in the signal, and there is less inter-individual variability when the recording point is the blocked entrance to the ear canal as compared to the open ear canal entrance and the eardrum because the blocked entrance does not include the effect of the ear canal [START_REF] Hammershøi | Sound transmission to and within the human ear canal[END_REF]. This technique also enables the use of rather large diaphragm microphones having better signal-to-noise ratios than the probe tubes used to measure the sound pressure within the ear canal [START_REF] Møller | Binaural technique: Do we need individual recordings[END_REF].

The headphone earpad is respectively surrounding or pressing the pinna for circumaural or supra-aural types. Usually, a circumaural headphone with a large cavity causes less compressive deformation of pinnae and therefore results in less HPTFs variation in different placements. In contrast, a circumaural headphone with a small cavity or supra-aural headphone inclines to cause large HPTFs variation in different placements [START_REF] Riederer | Repeatability analysis of head-related transfer function measurements[END_REF]. In this paper, 4 different headphone models were under study:

• A: Sennheiser HD497 (supra-aural),

• B: Sony MDR CD580 (circumaural with cushions),

• C: Sennheiser HD600 (circumaural),

• D: Sony MDR CD2000 (circumaural with cushions). These headphones are high-quality ones, especially models C and D which are commonly used by audio professionals and in psychoacoustic experiments. Headphones B and D are equipped with large earpads, surrounding the listener's pinna as for a circumaural model, but also with smooth cushions covering the pinna.

Each headphone was placed and then removed over the dummy head by two different experimenters. The experimenters did not attempt to place the headphones in critical positions but only tried to replace them as they should normally be over a listener's head. The 3 excerpts (plus a 20Hz-16kHz log-swept sine signal) were thus binaurally recorded for the 8 placements of each headphone model. The recordings were made using a laptop computer (MacBookPro) equipped with a MBox2 audio interface (f s = 44100 Hz, 16 bits) and took place in an audiometric booth.

2.3.Normalization of the recording levels

Even though recordings stemming from different headphone types were not to be compared by the subject in the listening test, the mean levels per headphone models were matched. Differences in listening levels among the headphone models could possibly lead to discrepancies in the ability to discriminate between successive placements for a given headphone model. The recording levels were thus normalized so that the average over the 8 placements was alike for the four headphone models. This operation is equivalent to a compensation of the headphone sensitivities. However, the relative levels of the 8 recordings made using one single headphone were not modified. The differences in these levels are caused by differences in the headphone placement and shall not be compensated, as they could be a clue to perceive the placement variability.

2.4.Test headphone equalization

This part deals with the equalization of the unique headphone that was used for playback in the subjective test, denoted test headphone for the rest of the paper, and does not apply to the headphones when used for playback over the dummy head for recording purposes.

Equalization method

The headphone used to perform all the subjective assessments was the Sony MDR CD2000 exhibiting stable and broadband HPTFs (see Fig. 11). Moreover, this headphone obtained the highest sound quality ratings in an informal test conducted with sound engineers.

In order to enable the transparency of the test headphone, it was necessary to compensate for its transfer function An inverse filter needs to be computed according to:

! !"# (!) = !(!)!. ! !"# (!) = 1 ( 1
)
where k is the discrete frequency, H(k) is the complex transfer function of the test headphone, H lin (k) is the linearized (i.e. compensated) transfer function of the test headphone and H inv (k) the complex inverse filter of the test headphone.

In the time domain, the convolution of h and h inv (corresponding respectively to H and H inv ) should result in a system response h lin equivalent to a dirac pulse:

ℎ !"# (!) = ℎ(!) ⋇ ! ℎ !"# (!) = !(!) (2)
The direct inversion of the measured frequency response, i.e.

! !"# (!) = 1 !(!) !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!(3)
could however raise several problems. The direct inversion of mixed-phase systems yields an acausal, infinite and instable impulse response [START_REF] Mourjopoulos | Digital equalization of room acoustics[END_REF]. This can be avoided by splitting the transfer function into a minimum-phase and an allpass component, of which only the former is equalized [START_REF] Mourjopoulos | Digital equalization of room acoustics[END_REF]. This leads to an exact compensation of the amplitude but not of the phase response. However, in some cases the result of this method can be insufficient due to the remaining error energy in the allpass component [START_REF] Karjalainen | About room response equalization and dereverberation[END_REF][START_REF] Nelson | Inverse Filter Design and Equalization Zones in Multichannel Sound Reproduction[END_REF]. This can especially cause the inverse filter to have large boosts at some frequencies, which could overload the system and could also produce a long filter. Several alternative approaches were developed: regularization [START_REF] Kirkeby | Fast Deconvolution of Multichannel Systems Using Regularization[END_REF], least square [START_REF] Kirkeby | Digital Filter Design for Inversion Problems in Sound Reproduction[END_REF], homomorphic techniques [START_REF] Mourjopoulos | A comparative study of least-squares and homomorphic techniques for the inversion of mixed phase signals[END_REF], smoothing techniques (power spectra-or complex smoothing), with a fixed or a fractional-octave smoothing window [START_REF] Hatziantoniou | Time-Frequency Mapping Based on Nonuniform Smoothed Spectral Representations[END_REF][START_REF] Hatziantoniou | Generalized Fractional-Octave Smoothing of Audio and Acoustic Responses[END_REF].

For the present study, two inversion methods were experimented: the first one was a direct inversion of the minimum-phase component (because it is one of the simplest method to implement):

! !"# (!) = 1 ! !"# (!) !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!(4)
realized in the frequency domain (including zero padding).

The second method was a third-octave complex smoothing, which has proven to be perceptually superior in most cases according to Norcross et al. [START_REF] Norcross | Subjective investigations of inverse filtering[END_REF]. The principle is to reduce the severity of the peaks and dips by smoothing them prior to calculating the inverse.

In order to minimize the effect of the position of the test headphone in the inversion process, 8 HPTF measurements were carried by displaying a 5s-long log-swept sine (20 Hz-16 kHz) for each of the 8 headphone placements and by recording using the dummy head microphones (fe= 44100 Hz, 16 bits). Then these 8 HPTF were averaged (in the complex frequency domain). A headphone transfer function is generally characterized by smooth fluctuations in the low frequencies and by individual high-Q peaks and dips at high frequencies. Individualized headphone equalization could be achieved by measuring and considering the HPTFs related to each test subject. Although this kind of equalization seems preferable, an average equalization is acceptable and used in most cases [START_REF] Norcross | Subjective investigations of inverse filtering[END_REF]. Moreover, the goal of this study is the subjective comparison of different recordings for which the restitution artifacts are identically equalized. An exact and individual equalization is thus not mandatory.

A preliminary test was conducted in order to determine the perceptually best of these two inversion methods: recordings of the two music excerpts were convolved with the inverse response of the test headphone obtained with respect to the inversion method. Subjects had then to assess the quality of these compensated excerpts played back on the test headphone. This preliminary test indicated that the two methods resulted in equivalent quality ratings. The good ratings obtained using the direct inversion of the minimum-phase component can be explained by an already smooth appearance of the averaged response (over 8 positions) of the test headphone: the frequencies of the high-Q peaks and dips were slightly different for the successive positions, and the average over positions smoothed these differences. The use of a more sophisticated smoothing method, as the second one described above, was thus not necessary in our case.

As the two inversion methods appeared to have the same perceptual efficiency, the simple inversion of the minimum-phase component was chosen because it preserves better the test headphone HPTFs: Fig. 2 indicates that in high frequencies some recurrent evolutions of the curves for successive positions are preserved by the simple average, but removed by the smoothing. For low frequencies, smoothing curve is similar to the simple average.

Test headphone coupling to the ear

When binaural recordings are carried out at the entrance of the blocked ear canal in order to be played back over a headphone, the coupling of the playback headphone has to be verified prior to inverting its response. According to Møller et al. [START_REF] Møller | Fundamentals of binaural technology[END_REF][START_REF] Møller | Transfer characteristics of headphones measured on human ears[END_REF], an additional equalization would be required for headphones that do not account for a Free-air Equivalent Coupling (FEC) to the ear.

The acoustical excitation from an headphone can be modeled by a Thevenin equivalent at the entrance to the ear canal [START_REF] Møller | Fundamentals of binaural technology[END_REF]:

! ! ! ! = ! !"#!!"#"$ ! !"#!!"#"$ !!! !!"#$!!"# (7)
where: • P 6 is the sound pressure at the entrance to the open ear canal, • P 5 is the Thevenin pressure (does not normally exist physically, but if the ear canal is blocked as, for example, with an earplug, the Thevenin pressure is found at the outer side of the earplug), • Z earcanal is the input acoustic impedance of the ear canal, • Z headphone is the headphone radiation acoustic impedance seen from the ear canal. according to the terminology of Møller et al. [START_REF] Møller | Fundamentals of binaural technology[END_REF].

When the recording is made at the entrance to the blocked ear canal, it can be shown [START_REF] Møller | Fundamentals of binaural technology[END_REF] that the correct eardrum signals are obtained during playback if the electrical gain G of the transmission is given by the expression:

! = ! !.!"#$ !.!! ! !"#$ ! !"#$%&' !"##!!"# ! !"#$ ! !"#$%&' !!"#$!!"# (8)
where :

• M is the transfer function from pressure to voltage of the recording microphone, • HPTF is the electroacoustical transfer function of the headphone, measured at the entrance to the blocked ear canal, • p open is the sound pressure at the entrance of the open ear canal, • p blocked is the sound pressure at the outer side of the blocked ear canal.

The last term of Eq. 8 can thus be defined as the Pressure Division Ratio (PDR). It can be expressed as follows when replacing Z headphone by Z radiation (free-air radiation acoustic impedance seen from the ear canal) in Eq. ( 7) for the free-air situation.

!"# = ! !"#$ ! !"#$%&' !"##!!"# ! !"#$ ! !"#$%&' !!"#$!!"# = ! !"#$"%"& + ! !!"#$!!"# ! !"#$"%"& + ! !!"#!$#%& !!!!!!!!!!!!!!!!!!!!!!!!!!!(9)
The two pressure divisions in Eq. ( 8) and Eq. ( 9) are measured respectively by displaying an excitation through a loudspeaker for free-air conditions or by the headphone itself. If the two pressure divisions are equal, the PDR reduces to unity. A headphone complying to this statement is defined as FEC, and recordings carried out at the entrance of the blocked canal can be considered without any compensation due to PDR.

The FEC-compliance of the test headphone (Sony MDR CD2000) was verified by measuring P open and P blocked using a 8 cm-long probe tube (ext. diameter 1.5 mm, int. diameter 0.5 mm) mounted on a DPA4060 miniature omnidirectional microphone with a coupler prototype. P blocked-headphone was measured on 9 ears by placing the free end of the probe tube at the entrance of blocked ear canals. Measurements were carried out on real heads for which the ear canal was blocked using individual moldings. The headphone was placed on the head by the subject himself. A log-swept sine was then played back over the headphone, and recorded with the probe microphone connected to a computer sound card. This operation was repeated for 8 headphone placements, the position of the probe tube being controlled by the experimenter between two consecutive measurements. A similar operation was carried out (with a repetition) in free air when reproducing the log-swept sine by using a loudspeaker (Genelec 8040) in order to measure P blocked-free-air .

P open-headphone and P open-free-air were measured the same way once the individual moldings were removed.

The log-swept sine signal was used because this technique is acknowledged to provide good signal-tonoise ratio, and is nowadays preferred to other methods (linear sweep, MLS...) for measurement of the transfer function of weakly non-linear (or/and approximately time-invariant) systems [START_REF] Farina | Simultaneous measurement of impulse response and distortion with a swept-sine technique[END_REF].

FFT moduli were respectively averaged over the 8 headphone measurements and over the 2 loudspeaker measurements, and the PDR could then be calculated. Fig. 3 indicates the PDR means and standard deviations (computed on the log-magnitude of the PDRs for the 9 ears), as a function of frequency. The PDRs are approximately flat at frequencies below 2 kHz, and remain within the range of ±2 dB between 2 kHz and 5 kHz, even if deviation across the different ears is not negligible. According to Møller et al. [START_REF] Møller | Transfer characteristics of headphones measured on human ears[END_REF], a headphone with such PDR values can be considered as FEC-compliant. Therefore, blocked ear canal measurements are suitable to invert the test headphone response.

2.5.Listening test procedure

Once compensated, the recordings made using the dummy head were presented during a listening test to determine whether differences due to headphone placement are noticeable. The test took place in an audiometric booth, the listener was sitting in front of a screen on which the answering buttons were displayed and could be activated by using a mouse.

The dichotic stimuli were played back over the Sony MDR CD2000 headphone for the whole test. The subject was asked to place it comfortably on his head and to not modify this placement once the test had started.

The listener's task was to discriminate recordings differing only by the headphone placement using a 3 Interval 3 Alternative Forced Choice (3I3AFC) response paradigm. During a trial, 3 intervals were successively presented: one containing a given recording placement and two containing another recording placement. The recordings were randomly assigned to the intervals. The stimulus that was presented once (referred to as oddball stimulus) could then appear in first, second or third position.

After having listened to the 3 stimuli (no repetition allowed), the subject had to indicate which one of the 3 intervals was the oddball one. The listener had to validate his choice to go on to the next trial. To compare this way all recordings made for one excerpt played over one headphone model (i.e. to compare the 8 placements), 28 trials (number of possible pair combinations to compare 8 stimuli) were proposed. A "sub-session" consisted of all trials needed to compare all the recordings made using one single headphone model. Since 3 different excerpts were recorded for each headphone, a sub-session was made of 84 trials. During a sub-session, the listener had to assess the 28 trials associated with the first excerpt, then the 28 trials for the second one and finally the 28 trials for the third one. The excerpt order was randomized as well as the headphone model was selected at random for each sub-session. The test was made of all subsessions needed to assess the headphones under test.

EXPERIMENT I

In order to assess the recordings stemming from the four headphone models, the test was run in two sessions lasting 45 min each. Each session was made of two sub-sessions, the listener having a 5-min break between them (he could remove his headphone during the break). The two sessions had to take place on two different days. The first of these two sessions was preceded by a 5-min pre-test to familiarize the listener with the answering interface and the stimuli. The listening level was identical for all listeners, stimuli were played back at an average level of 85 dBSPL (according to a long term analysis of recordings realized when placing the test headphone over the dummy head, whose microphones were calibrated).

The listeners involved in the test were 10 sound engineering students from the "Master Image & Son" training program of the University of Brest. The subjects had no particular experience in listening tests. However, because of their experience, personal knowledge and interest, they were considered as competent for this listening test. According to ISO 8586-2 standard [START_REF]Sensory analysis -General guidance for the selection, training and monitoring of assessors -Part 2: Experts[END_REF], they match the definition of "expert". The listeners were paid for their participation and showed normal hearing thresholds based on an audiogram [START_REF]Methods for manual pure-tone threshold audiometry[END_REF] passed in the two months preceding the test.

3.1.Audibility of the placement variability

Using a 3I3AFC response paradigm, a subject answering at random has a one-in-three chance of pointing out the oddball stimulus. A detection rate for the oddball stimuli of 33.33 % would then be equivalent to chance. A t-test was used to determine whether the experimental detection rates were statistically higher than 33.33 %. The t-test indicated that, whatever the headphone or the excerpt, the average detection rates (Fig. 4) were always statistically different from 33.33 %, with p = 0.001 for the least significant cases.

3.2.Headphone effect

As illustrated in Fig 5, a 2-way analysis of variance showed that the headphone effect was statistically significant (F(3,108) = 20.199; p < 0.0001). According to the Fisher LSD test, all headphones obtained statistically different detection rates from one another (p < 0.0001) except for the models C and D (p = 0.345). The detection task for the oddball stimulus appeared then to be statistically more difficult for the recordings made using the headphone A (Sennheiser HD497) than for the three other headphones. On the other hand the audibility of the placement variability was statistically higher for the recordings stemming from headphone B (Sony MDR CD580).

3.3.Excerpt effect

As illustrated in Fig 6, the analysis of variance showed that the excerpt effect was also statistically significant (F(2,108) = 60.603; p < 0.0001). The Fisher LSD test indicated that the detection rate was statistically higher with the pink noise than with the two music excerpts (p < 0.0001). The two music excerpts did not obtain statistically different detection rates (p = 0.056). So the detection task for the oddball stimulus appeared to be statistically easier for the recordings made using pink noise than music.

EXPERIMENT II

As a result to experiment I, the different headphone placements could be statistically distinguished when listening to the recordings carried out whatever the reproduction headphone model and the excerpt. However, it may be argued that such high detection rates could be obtained because of the expert status of the listeners involved in this experiment, who are used to critical listening. It can be thought that the results significance would not have been as high with naïve listeners. In order to investigate this hypothesis, the experiment was conducted with 10 naïve assessors (without music or listening background) according to ISO 8586-2 standard [START_REF]Sensory analysis -General guidance for the selection, training and monitoring of assessors -Part 2: Experts[END_REF]. In order to reduce the test duration, only two headphone models were considered in this second experiment. The headphones under test were then reduced to models A (Sennheiser HD497, supra-aural) and B (Sony MDR580, circumaural with cushions) which respectively provided the worst detection rate and the best one. This second test thus consisted of two sub-sessions and was accomplished in a unique 45-min session proceded by a 5-min pre-test to familiarize the listener with the answering interface and the stimuli.

4.1.Audibility of the placement variability

t-tests indicated that for naïve listeners, the average detection rates (Fig. 7) were always statistically higher (p < 0.003 in the least significant case) than 33.33 % (equivalent to chance with a 3I3AFC response paradigm), except for the case headphone A, excerpt 3 (Leonard Bernstein), for which the mean detection rate (40 %) was not statistically different (p = 0.1) from chance level (33.33 %).

4.2.Headphone effect

As illustrated in Fig 8, a 2-way analysis of variance showed that the headphone effect was statistically significant (F(1, 54) = 42.46; p < 0.0001). The detection task for the oddball stimulus appeared then to be more difficult for the recordings made using the headphone A (Sennheiser HD497) than for the headphone B (Sony MDR CD580).

4.3.Excerpt effect

The analysis of variance showed that the excerpt effect was also statistically significant (F(2, 54) = 15.36; p < 0.0001). The Fisher LSD test indicated that the detection rate was statistically higher with the pink noise than with the two music excerpts (p < 0.0001). The two music excerpts did not obtain statistically different detection rates (p=0.09). So the detection task for the oddball stimulus appeared to be statistically easier for the recordings made using pink noise than music (Fig. 9).

4.4.Comparison with expert listeners

Headphone models A and B were tested by both the expert and naïve listeners. The experiments I and II could then be compared on the basis of the detection rates obtained by these two headphone models to look at the subject group effect. A 3-way mixed design analysis of variance (with "Headphone" and "Excerpt" as within-subject factor and "Subject group" as between-subject factor) was carried out to investigate the effects of the factors and their possible interactions on the detection rate.

The listener group proved to have a statistically significant effect (F(1, 108) = 47.39; p < 0.0001). As can be seen in Fig. 10, the detection rate appeared to be statistically higher for the expert subjects. As was already observed separately for the two listeners groups, the headphone model (F(1, 108) = 98.19; p < 0.0001) and the excerpt (F(2, 108) = 39.75; p < 0.0001) had statistically significant effects which respectively resulted in higher detection rates for headphone B and for pink noise.

In addition, no statistically significant interaction was highlighted between the subject group and the two other factors: the experts distinguished the headphone placements with a higher mean detection rate than the naïve listeners, whatever the headphone model and whatever the excerpt.

5.1.Headphone placement is audible

The most important result regarding this study is that headphone placement variability produces audible differences in almost all cases. The detection rate for the oddball sequence varied significantly across the headphones, excerpts and listener's backgrounds but was statistically higher to the value that could be obtained by chance, except for one case only: the naïve listeners failed to detect the oddball stimulus using recordings carried out when reproducing the Leonard Bernstein excerpt with headphone A. As this headphone appeared to be the less discriminant in experiment I, it can be inferred that headphones C and D would have enabled discrimination for naïve listeners as well. Past studies showed that headphone placement appears to have no consequence on localization performances [START_REF] Mcanally | Variability in the headphone-to-ear-canal transfer function[END_REF] nor -under certain conditionson audiometry [START_REF] Gauz | High-frequency Békésy audiometry: III. Reliability and validity revisited[END_REF]. Nevertheless, as it could be suspected from other past studies [START_REF] Mcanally | Variability in the headphone-to-ear-canal transfer function[END_REF][START_REF] Kulkarni | Variability in the characterization of the headphone transfer function[END_REF], the action of placing and replacing a headphone over a listener's head have audible consequences. So the frequency smoothing applied by the inner ear [START_REF] Mcanally | Variability in the headphone-to-ear-canal transfer function[END_REF] does not totally filter out the differences introduced by successive placements.

Fig. 11 indicates for the 4 headphones the means and standard deviations of non-normalized HPTFs across the 8 placements. These spectra were obtained from the 1024-pts magnitude spectrum of the impulse response obtained using a log-sweep sine signal recorded by using a miniature microphone located at the entrance of the ear canal (f s = 44.1 kHz). These measurements were carried out on the dummy head at the same time as the recordings of the 3 excerpts that were used in the subjective experiment. In order to facilitate the reading, Fig. 12 indicates only the standard deviation per headphone model. These data are globally in agreement with previous studies [START_REF] Pralong | The role of individualized headphone calibration for the generation of high fidelity virtual auditory space[END_REF][START_REF] Mcanally | Variability in the headphone-to-ear-canal transfer function[END_REF][START_REF] Kulkarni | Variability in the characterization of the headphone transfer function[END_REF][START_REF] Møller | Transfer characteristics of headphones measured on human ears[END_REF] between which small differences can be observed in the measurements but might be due to the use of a different frequency step (in our study the 1024-pts FFT with f s = 44.1 kHz provided a 43-Hz frequency step). For most of the headphones under test, the standard deviation remained below 2 dB for frequencies up to 5kHz, and reached up to 6 to 8 dB above 5 kHz.

According to more general studies about spectral peaks detection [START_REF] Hiekkanen | Virtualized listening tests for loudspeakers[END_REF][START_REF] Bücklein | The audibility of frequency response irregularities[END_REF][START_REF] Olive | The detection thresholds of resonances at low frequencies[END_REF] and considering the frequencies, levels, and Qs of spectral peaks observed in the present HPTF measurements, the effects of headphone placement were presumably audible. This has been confirmed by the significance of the results to the present subjective experiments.

Finally, one should keep in mind that these results were obtained using monophonic excerpts (diotically presented to the dummy head for recording but resulting in dichotic signals because of the HPTFs). The possible extension of such conclusions to stereophonic or binaural recording must be taken with care.

5.2.Comparisons between headphones

The comparison of the measurements carried out on different headphones does not provide hints to explain the differences in audibility. As an example the placement variability was statistically better perceived by listeners on headphone B than on headphone A, whereas the standard deviation observed in the HPTF measurements appears to be higher for the headphone A than for the headphone B (in agreement with Riederer [START_REF] Riederer | Repeatability analysis of head-related transfer function measurements[END_REF] who showed that the variability in HPTF measurements was higher for supra-aural headphones than for circumaural ones).

It is worth noting that the standard deviation reached up to 4 dB below 5 kHz on the right ear of headphone A, whereas it was only of 2 dB on the left ear, and on both ear for the other headphone models. This headphone was the cheapest of our selection, and it is possible that the right earpad was somehow misshapen, which was however not visually detectable. Nevertheless, despite such a high physical variability in "low" frequencies, the perceptual effect of placement was lower on headphone A.

For headphone B (the one which provided the easiest discrimination of placements), the standard deviation was particularly small above 5 kHz (lower than 4 dB), except for an isolated 7 dB peak at 8 kHz on the left ear. The ability to detect the effect of placement for this headphone could possibly be explained by this peak only. Finally, one should note that headphone A (Sennheiser HD497), which provided the worst detection rate in the perceptual test, and exhibited thus the highest robustness to placement variability, was a supra-aural headphone. This finding is not in agreement with ANSI S3.21 standard [START_REF] Farina | Simultaneous measurement of impulse response and distortion with a swept-sine technique[END_REF] which states that the use of a circumaural headphone should increase the accuracy of auditory threshold determinations at high frequencies, nor with Riederer [START_REF] Riederer | Repeatability analysis of head-related transfer function measurements[END_REF] and Atherley [START_REF] Atherley | Reliability of auditory threshold determinations using a circumaural-earphone assembly[END_REF], who suggested the use of circumaural heaphones, which generally provide a better placement stability. Nevertheless, in these studies, the authors did not use the same headphones as those chosen in the present study, and the generalization of their finding has to be taken with care. In particular, the headphone A was clearly of a least quality than the three other ones. The least quality of this headphone is not obvious when looking at the HPTFs (Fig. 11), but was clearly audible by any listener. In this headphone was significantly cheaper than the other ones. As the audibility of headphone placement would be exclusively caused by spectral cues [START_REF] Mcanally | Variability in the headphone-to-ear-canal transfer function[END_REF], one can think that the headphone placement, characterized by modifications in high frequencies, would not be so critical for an headphone providing a bad reproduction of the spectrum, especially in high frequencies. On the other hand, the placement could be a more serious issue for higher quality headphones, enabling a more precise reproduction of the whole audible band.

5.3.Excerpt influence

The differences caused by variability in the headphone placement were more easily perceived using pink noise than music excerpts. This statement may have several explanations: the pink noise is an almost steady-state signal, which facilitates a memorization effort that might be tough for music excerpts, even for very short ones. In addition, the spectral content of the pink noise ensures that the spectral modifications caused by a specific placement (with possible high-Q peaks and dips) will be highlighted. This will not necessary be the case for music whose spectral content is much less regular and much more time-varying. Bücklein [START_REF] Bücklein | The audibility of frequency response irregularities[END_REF] and Fryer [START_REF] Fryer | Intermodulation distortion listening tests[END_REF] observed that the detection of resonances was more accurate using white noise than music, symphonic music being more discriminant than popular music. According to them, the broader the spectral content, the better the audibility of the resonances. Toole and Olive [START_REF] Toole | The modification of timbre by resonances: Perception and measurement[END_REF] have also observed that resonance detection at low frequencies was easier using pink noise than using music or speech signals and depended on the temporal behavior of the signals under test.

In this study, only the left channel of music recordings was kept. This could cause a bias toward the high frequencies for the third excerpt, as the left channel of symphonic recordings usually contains the first and second violin groups and substantially high-pitched woodwinds like flute and clarinet. As can be noted in Fig. 1, the long-term spectrum of this excerpt mainly contains frequencies above 300 Hz. Using a monophonic recording or an average of the two channels would have possibly given different results because of the frequency content and the possible masking effects. In addition, it is possible that stereophonic recordings would also reduce discriminability of HPTF differences due to positioning.

5.4.Consideration of headphone placement

The audibility of the modifications introduced by headphone placement variability does not necessary mean that an equalization is mandatory to compensate for these differences. Several applications do not require such compensation: for instance, in binaural synthesis, it has been shown that the peaks characterizing the HRTFs were so high that the HPTF variability was negligible for sound source location [START_REF] Mcanally | Variability in the headphone-to-ear-canal transfer function[END_REF]. However, in psychoacoustic experiments where the listener may have to assess the frequency content, variability due to headphone placement might be an issue. As an example, inter-subject variability observed in a listening test (conducted with headphones) could be partially caused by inter-subject differences in HPTFs and might therefore be increased by odd headphone placements. In addition, for experiments conducted in several sessions and for which the headphone is placed several times on a given listener's head, placement variability could increase intra-subject variability.

Finally, it would be quite hard to compensate for HPTF differences: a real-time headphone equalization would require an in-situ (i.e. in the listener's ears) pressure measurement that would be by definition carried out in open ear canal conditions. This kind of measurement would be accomplished by using a probe tube and would therefore be of poor reliability because of the positioning accuracy of the tube, the frequency response of the probe microphones [START_REF] Okabe | Headphone responses on real ears and a head and torso simulator[END_REF], the difficulty of calibrating them (particularly above 7-8 kHz) and especially the bad signal to noise ratio of these devices [START_REF] Toole | Binaural record/reproduction systems and their use in psychoacoustic investigations[END_REF].

A possible solution could be the use of earphones rather than classic headphones. Earphones were not considered in this study because the microphones in use in the present study were located at the entrance of the blocked ear canal of the dummy head. To record the response of earphones, probe tubes should be used, thus involving the same drawbacks as mentionned above. With earphones, the influence of the pinna is not taken into account, and so it may be thought that variability of narrow peaks and dips would disappear. However the resonance of ear canal could change with the acoustic loading resulting from the earphone insertion, providing another form of HPTF variability, possibly audible as well.

6. CONCLUSION Previous studies indicated that spectral modifications were caused by headphone placement variability. The goal of the present experiment aimed at indicating whether these spectral modifications were audible. At the exception of one condition (a given headphone, and only for one given excerpt and for naïve listeners), spectral modifications led to audible differences. The perception of these differences can be more or less emphasized according to the headphone model, the listener background, and the excerpt.

These audible differences could be of crucial importance in some applications, especially for psychoacoustic experiments conducted over several sessions in which a headphone has to be placed several times over subjects' heads.

To sum up, the main conclusion that can be drawn from the results of this study would be a recommendation towards auditory researchers working with headphones: spectral variability resulting from headphone displacement can be easily discriminated and can possibly bias experiments.

FIG. 1 .

 1 FIG. 1. Long-term spectra (left channel only) of excerpt 2 (Ben Harper) and excerpt 3 (Leonard Bernstein)

FIG. 2 .

 2 FIG. 2. HPTF modulus of the Test headphone (left ear), for each of the 8 measured positions (top), and averaged across the 8 positions, smoothed or not (bottom).

FIG. 3 .

 3 FIG.3. Pressure Division Ratio : mean and standard deviation across the 9 ears.

FIG. 4 .FIG. 5 .

 45 FIG. 4. Mean detection rates for the 4 headphones and the 3 excerpts, within their 95 % confidence intervals, when considering all expert listeners.

FIG. 6 .

 6 FIG.6. Mean detection rates for the 3 excerpts, within their 95 % confidence intervals, when considering all headphone models and all expert listeners.

FIG. 8 .FIG. 7 .

 87 FIG. 8. Mean detection rates for headphones A and B, within their 95 % confidence intervals, when considering all excerpts and all naïve listeners.

FIG. 10 .FIG. 9 .

 109 FIG. 10. Mean detection rates for expert and naïve listeners, within their 95% confidence intervals, when considering the headphones A and B and all excerpts. Expert Naïve 0 10 20 30 40 50 60 70 80 90 100

FIG. 11 .

 11 FIG. 11. Means and standard deviations of the HPTFs of 4 headphones (left and right ears, not normalized) computed on a decibel basis from 8 placements measured at blocked ear canal. Shaded areas mean ±1 standard deviation.

FIG. 12 .

 12 FIG. 12. Standard deviations of the HPTFs of 4 headphones (left ear in black, right ear in grey) computed on a decibel basis from 8 placements measured at blocked ear canal.

ACKNOWLEDGMENTS

The authors wish to thank the staff and students of the "Image & Son" research and training group from the University of Brest for participating in this experiment. The authors would also like to thank Sylvain Marchand, Brice Jantzem (Centre Auditif Entendre Brest, France) for their advices and technical support as well as Rozenn Nicol (Orange Labs Lannion, France) for the loan of several headphone models.


Jordi Cabot
email: jordi.cabot@inria.fr

Javier Luis

Cánovas Izquierdo

Valerio Cosentino
email: valerio.cosentino@inria.fr

Belén Rolandi

Exploring the Use of Labels to Categorize Issues in Open-Source Software Projects

Reporting bugs, asking for new features and in general giving any kind of feedback is a common way to contribute to an Open-Source Software (OSS) project. This feedback is generally reported in the form of new issues for the project, managed by the so-called issue-trackers. One of the features provided by most issue-trackers is the possibility to define a set of labels/tags to classify the issues and, at least in theory, facilitate their management. Nevertheless, there is little empirical evidence to confirm that taking the time to categorize new issues has indeed a beneficial impact on the project evolution. In this paper we analyze a population of more than three million of GitHub projects and give some insights on how labels are used in them. Our preliminary results reveal that, even if the label mechanism is scarcely used, using labels favors the resolution of issues. Our analysis also suggests that not all projects use labels in the same way (e.g., for some labels are only a way to prioritize the project while others use them to signal their temporal evolution as they move along in the development workflow). Further research is needed to precisely characterize these label "families" and learn more the ideal application scenarios for each of them.

I. INTRODUCTION

One of the main advantages of Open-Source Software (OSS) is that it enables anyone to be part of the development process at large [START_REF] Bird | Open Borders? Immigration in Open Source Projects[END_REF]. Among the different ways to contribute (e.g., testing, coding, etc.), maybe the most common one is giving feedback by reporting bugs and requesting either improvements or new features. This feedback is reported in the form of new issues for the project, managed by the socalled issue-trackers.

As the project grows, issues become numerous and an appropriate management process, e.g., to decide which issues must be accepted or prioritize them, must be put in place. A common way to facilitate the organization of issues in projects is based on the use of labels [START_REF] Storey | How Software Developers Use Tagging to Support Reminding and Refinding[END_REF]. Labels are a simple and effective way to attach additional information (e.g., metadata) to project issues. A label can give any user an immediate clue about what sort of topic the issue is about, what development task the issue is related to, or what priority the issue has. Furthermore, labels may also be useful for project administrators, since they can serve both as classification and filtering mechanism, thus facilitating the managing of the project. However, little effort has been devoted to the analysis of the impact of using labels in OSS projects.

In this paper we have analyzed a population of more than three million of GitHub projects in order to study how they use the label mechanism. GitHub has become one the most popular platforms to develop and promote OSS projects. With millions of repositories, GitHub gathers many of the important projects in the scene. Among its different functionalities, GitHub provides a light-weight and flexible issue-tracker which also includes labeling support. Our analysis focuses on the more than three million of original projects (i.e., projects that are not simply the fork of another existing project).

In particular, we have studied [START_REF] Bird | Open Borders? Immigration in Open Source Projects[END_REF] how/which labels are used in GitHub projects and (2) how labels may influence the success of the project (mainly, in terms of the time it takes to address new submitted issues). Our results show that there exists a significant difference in terms of the number of treated issues between projects that do not use labels and projects that do. And the more different labels are used in the project the more this effect is observed. Besides this global analysis, a more fine-grained preliminary study aimed at studying how labels are used together is also conducted.

The rest of the paper is organized as follows. Section II introduces our analysis and the main results. Section III discusses the results and presents further analysis steps on the topic. Section IV reports on the related work and Section V gives some last remarks and future work.

II. USE OF LABELS IN OSS

We perform our analysis over the population of repositories hosted in GitHub. GitHub provides an off-the-self issuetracking system allowing projects to manage their development tasks by means of issues. Issues have a title and a description, and can be tagged with labels, linked to milestones and assigned to developers. The GitHub issue-tracking system comes with seven generic labels (i.e., bug, duplicate, enhancement, help wanted, invalid, question, wontfix), but more interestingly, it provides the capability to create custom labels adapted to the specificities of the project.

For our label analysis we used the dataset provided by GiLA [START_REF] Cánovas Izquierdo | GiLA: GitHub Label Analyzer[END_REF], a tool which generates a set of visualizations to facilitate the analysis of project issues depending on their labelbased categorization. GiLA in turn relies on GHTORRENT, a scalable and offline mirror of data offered through the GitHub REST API [START_REF] Gousios | The GHTorrent Dataset and Tool Suite[END_REF], and extends it with a set of auxiliary tables and views storing semi-digested information to facilitate computations regarding projects (repositories), their users, issues and labels.

In our label analysis we targeted the following research questions:

RQ1

Label usage. How many labels are used in Github? How many labels are used per project? What are the most popular ones?

RQ2

Label influence. Does using labels influence the evolution of the project?

A. Label Usage

As a first step, we studied the ratio of projects with at least one issue labeled. We focused on original projects no matter their purpose (i.e., libraries, documentation, code snippets, etc.), thus allowing us to assess the label usage in the platform. In general, from 3,757,038 studied projects, there are only 122,012 (3.25%) of them containing labeled issues. The rest includes projects that do not use labels (i.e., either do not have issues at all or, if they have, none of them are labeled). This result reveals that the label mechanism is scarcely used in GitHub.

We then studied how many labels on average are used in each project 1 , both in terms of distinct labels per project and per issue in the project, and how many users are involved in labeled issues. Table I shows the main results. Second column (# projects) shows that the vast majority of projects only use 1 or 2 different labels (45.53% and 25.42% respectively). Third column (# labeled issues) gives the average ratio of issues that are classified with at least one label, showing that around 60% of the issues are labeled regardless the number of distinct labels available in the project. Forth column reports on the average number of labels per issue, always around 1, also kind of a constant value regardless of the number of project labels. Finally, last column shows that around 78% of the users participating (e.g., commenting, opening, closing, etc.) in the project issues are involved in the labeled ones. With regard to the whole GitHub user population, the ratio of users involved in at least one labeled issue is 39.93%.

We also studied how labels are used according to the size of the project (in terms of number of issues). Table II shows the main data we gathered. We identified four groups of project sizes (see the four double-columns after the column # labels) and in a similar way as we did before, we calculated the number of projects along with the average ratio of issues that are labeled with at least one label. As can be seen, projects with less than 100 issues use quite regularly the label mechanism (they have around 70% of labeled issues), while projects with more than 100 issues only show an increment in the issues labeled when the projects use more than 10 labels. Another interesting analysis is collecting the most popular labels across all projects. Figure 1 shows the seven most used labels together with the number of issues categorized with that label. As could be expected, some of the default labels are among the most used ones but, interestingly, custom labels such as documentation and feature are also broadly used, which may reveal a common need (at least from a user point of view) to demand more documentation and new features for projects.

Finally, we also studied which labels are normally used together by means of a co-occurrence matrix. We discovered that the most labels used together are bug and enhancement (around 40,000 times). Interestingly enough, other relevant label pairs are bug-question, enhancement-question, wontfixbug and wontfix-enhancement, which are used around 10,000 times together each. The indistinct use of bug and enhancement in these pairs might reveal a misunderstanding of when they should be applied.

B. Label Influence

We have already seen that only a small percentage of projects choose to benefit from the label mechanism. In this section we look at those that do and try to see if labels have a positive impact on their evolution. We measure this positive impact according to two dimensions (1) percentage of issue treatment, that is, the ratio of processed issues (indistinctly, merged or closed since both status reflect the fact that the project members considered and made a final decision on the issue's future); and (2) number of people involved in the discussion/treatment of the issues.

To evaluate this, we study the relationship between the number of distinct labels used in the project and six metrics, concretely: (1) average and (2) median time to resolve a labeled issue in the project, (3) average and (4) median age of labeled issues still pending, (5) percentage of labeled issues solved, and (6) average number of people involved (i.e., commenting, opening, closing or merging) in the project labeled issues. The Spearman's rho (ρ) correlation coefficient is used to measure the strength of monotonic relationships between the considered attributes. The values of ρ are in the range [-1, +1], where a perfect correlation is represented either by a -1 or a +1, meaning that the variables are perfectly monotonically related (either a increasing or decreasing relationship, respectively). Thus, the closer to 0 the ρ is, the more independent the variables are.

Table III shows the results of the correlation analysis for projects using labels2 , including a column for each metric and, as reference, a column with the number of total issues. As can be seen, the ρ values regarding the average and median time to solve labeled issues and the percentage of solved labeled issues reveal high correlations. The latter result reveals that on average the percentage of solved labeled issues tends to increase together with the number of labels used in the project and it may confirm that the effort of categorizing issues is beneficial for the project advancement. The former results say that on average this might come at the cost of taking more time to solve those labeled issues. Such results are also linked with the perfect correlation value in the last column, stating that on average projects with a complex label management system in-place tend to involve more people in the discussion of those issues.

It is important to note that the correlations have been calculated on aggregated data (e.g., average time to resolve issues in each group), although they are valid statistics [START_REF] Monin | Correlated Averages vs. Averaged Correlations: Demonstrating the Warm Glow Heuristic Beyond Aggregation[END_REF], they cannot be used to make inferences at individual level [START_REF] Piantadosi | The Ecological Fallacy[END_REF], thus further investigations are needed to confirm that these correlations stand also for the single issues in each group.

III. DISCUSSION AND FURTHER WORK

While our study allows us to better characterize how labels are used in GitHub and, more importantly, to assess that the use of labels may influence positively the project success; however, looking at the standard deviations in Tab.III and quickly browsing a few projects immediately brings up the fact that projects use labels in many different ways. Some use labels to classify the kind of request, others their priority, yet others use labels to indicate the software component the issue is about. Clearly, these different label usages may bring different benefits (or drawbacks) to the projects and/or work better for different kinds of projects (e.g., depending on the project's size, language, complexity or seniority).

Therefore, we would like to dive deep down into the data to learn more about which projects benefit more from the label usage and how they do it. To this purpose, we start by analyzing if there are some specific label sets (what we call label families) that appear more frequently than others. These results would need to be confirmed later on with the project owners to understand better why they decided to choose that specific set of labels and how it is working for them.

To identify the families, we have performed a clustering analysis to aggregate labels commonly used together. Our aim is to identify clusters which maximize the number of common labels in their members. We applied an adapted version of the algorithm used by Tairas and Cabot [START_REF] Tairas | Cloning in DSLs: Experiments with OCL[END_REF], listed in pseudo-code in Figure 2. Although other classical clustering algorithms could be applied, we decided to employ this one due to its specific support for string-based groups (and the ease of being adapted to our problem). The input of the algorithm is a list of label groups used in GitHub projects (i.e., one label group per project). All label groups are passed through a pre-step which removes those groups that use less labels than the default ones (i.e., lines 2 -6 in Figure 2) to reduce the number of groups considered and focus on the ones more potentially interesting. After this pre-step, each remaining group is evaluated for building the clusters (i.e., lines 8 -20 in Figure 2). The first iteration creates a cluster for the first label group evaluated. The second iteration evaluates the first two label groups. If they do not match, then the second label group is placed in a separate cluster. The third label group is then evaluated against the two original clusters (if the two labels groups did not match) and either added to one of them or placed in a third cluster if there is no match. The same process is iteratively applied to all remaining groups.

The comparisons between two label groups (i.e., function M AT CH in line 11 of Listing 2) is performed as follows. Two label groups are considered the same if either they include the very same labels or the number of different labels between them is less than two. To compare labels individually, we use the Levenshtein distance, thus only those labels with a distance of 2 or less are considered the same.

The top five clusters (in terms of number of projects in the cluster) detected by the algorithm are listed in Table IV. The detection of these clusters helped us to identify four label families (the fifth one being, as expected, the default set of labels provided by GitHub and their derivations, e.g., projects with only small changes with regard to the default set such as renaming the bug label as important bug or ui bug), specifically: (1) priority labels, i.e., labels that denote the priority of the tagged issues (e.g., high-priority or urgent); (2) versioning labels to indicate the version of the software to which the issue is referring to (e.g., 0.0.1 or next-release); (3) workflow labels to define the phase in the development process where the issue is located at the moment (e.g., 0 -backlog or 1 -ready); and (4) architectural labels, denoting the architectural components affected by the issue (e.g., component-ui, component-emacs). Note that the number of families is not related to the number of clusters. The output of the cluster algorithm allowed us to simplify the identification of the labels commonly used together.

To get a better idea of how representative are these families, we followed it up with a coverage analysis to determine how many projects fit into one of the identified label families. Note this number cannot just be taken from the clustering algorithm due to the high variability of label terms. A project can have a set of labels that are not syntactically equal to any of the families but convey the exact same semantic meaning than one of them and therefore should be considered to be covered by it. Therefore, for the coverage analysis, we first semiautomatically classified each label as semantically equivalent to one of the four label families, if the label name could fit in that group. In a second step, projects are assigned to one of the families if its labels have been regarded as equivalent to those in the family. We only focus on four families and discard the family of default ones since, being the default GitHub option, the results we could get with that family would not be so relevant.

Table V shows the coverage of each of the four label families with regard to the label population in GitHub (second column) and the project (third column). The results show that around a 15% of labels are related to one of the families. It is important to note that some labels are very specific and only used in a small set of projects (e.g., labels not written in English) while others are used in a considerable number of projects. Results also indicate that around 13% of the projects adhere to one of the families.

Considering that the easiest option for projects is to limit themselves to use the default label set instead of creating their own labeling strategy we believe the above numbers are significant and highlight that projects doing the effort of accu-rately managing issues by means of the label mechanism ended up following only a few different strategies corresponding to the four clusters identified before. Nevertheless, as mentioned above, more research needs to be done to confirm and expand these preliminary results.

IV. RELATED WORK

Tagging is at the core of social networks such as Twitter, Flickr, Facebook, etc., where users can annotate their contributions. The very own nature of the labeling mechanism allows people to freely define new labels in contrast to other formal classification mechanisms. Several works have analyzed how labels are used in these platforms (e.g., [START_REF] Golder | Usage Patterns of Collaborative Tagging Systems[END_REF], [START_REF] Ames | Why We Tag : Motivations for Annotation in Mobile and Online Media[END_REF], [START_REF] Xia | Tag Recommendation in Software Information Sites[END_REF]).

The use of labels has also been extended to code hosting platforms where tools like issue-trackers (as the ones provided in popular platforms like GitHub or BitBucket) offer this functionality. Nevertheless, little effort has been done on analyzing how labels are used in a population of software projects and on whether their use has any impact on the project evolution and success.

Other works ( [START_REF] Treude | Work Item Tagging: Communicating Concerns in Collaborative Software Development[END_REF] and [START_REF] Treude | How Tagging Helps Bridge the Gap between Social and Technical Aspects in Software Development[END_REF]) study the label usage but in only two projects reporting on the label adoption, label categories and their purpose. Our study however focuses on the set of GitHub projects, thus providing results at a much larger scale.

The use of issue-tracking systems have been studied to assess whether they have an impact in the project, concluding that such impact is disputed [START_REF] Bissyande | Got issues? Who Cares about it? A Large Scale Investigation of Issue Trackers from GitHub[END_REF]. On the other hand, Giger et al. [START_REF] Giger | Predicting the Fix Time of Bugs[END_REF] presents an approach to predict the fix time of bug issues. Our work could complement both studies by factoring in the label dimension in them.

The study presented by Posnett et al. [START_REF] Posnett | Got Issues? Do New Features and Code Improvements Affect Defects?[END_REF] reports that releases with high activity in improvements and new features often come with reduced effort in defect fixing. We believe it would be interesting to generalize this work by considering all kinds of labels.

Storey et al. [START_REF] Storey | TODO or To Bug : Exploring How Task Annotations Play a Role in the Work Practices of Software Developers[END_REF] present an empirical study that explores how task annotations embedded within the source code play a role in managing personal and team tasks. The work relies on the tool called TagSEA [START_REF] Storey | How Software Developers Use Tagging to Support Reminding and Refinding[END_REF] which supports programmers in defining semantically rich annotations to source code comments. Our approach is focused on analyzing issue labels but it would be interesting to extend it to deal with labels directly embedded in the source code.

GitHub data has been the target of a number of research papers. Our approach relies on the GHTORRENT dataset [START_REF] Gousios | The GHTorrent Dataset and Tool Suite[END_REF] but there are other options such as GITHUB ARCHIVE [17] which aim to simplify the mining of GitHub data. Additionally, recommendations on how to mine GitHub data are provided by Bird et al. [START_REF] Bird | The Promises and Perils of Mining Git[END_REF]; we have taken them into consideration when performing our analysis.

V. CONCLUSION

In this paper we have studied how labels are used in GitHub projects. Our results show that even if they are used scarcely in the platform, when they are they may have a positive impact on the project. We have also reported that projects employ labels in different ways classifying issues according to priority, affected component, workflow process, etc., according to our preliminary analysis of the top "label families". Tool support for a deeper analysis of the label usage in specific projects is also provided.

Next steps will focus on continuing our investigations around the concept of label families. In particular, we would like to study how each family may influence the project success, according to various definitions of success, and to better understand why projects choose a specific label set instead of another (or none at all) depending on the project characteristics (size, domain, language, community organization, etc). Besides this, we would like to study how labels evolve during the life-cycle of the project, including the profile of users adding/removing them. Finally, we plan to extend our approach to other web-based code hosting services (e.g., BitBucket, SourceForge, etc.) to see if there are significant differences across them.

Fig. 1 :

 1 Fig. 1: Main labels used in GitHub.

Fig. 2 :

 2 Fig. 2: Clustering algorithm.

TABLE I :

 I Label usage in GitHub projects.

	# labels	# projects	% labeled issues	avg. labels/issue	% users involved in labeled issues
	1	55,561 (45.53%)	59.87% (σ = 0.37)	1.00 (σ = 0.02)	80.98% (σ = 0.30)
	2	31,026 (25.42%)	61.00% (σ = 0.32)	1.02 (σ = 0.14)	72.06% (σ = 0.28)
	3	13,390 (10.97%)	58.89% (σ = 0.30)	1.04 (σ = 0.21)	77.73% (σ = 0.30)
	4	6,910 (5.66%)	58.94% (σ = 0.29)	1.06 (σ = 0.27)	75.81% (σ = 0.30)
	5	4,206 (3.44%)	59.72% (σ = 0.29)	1.09 (σ = 0.33)	75.22% (σ = 0.30)
	6	3,011 (2.46%)	56.16% (σ = 0.30)	1.08 (σ = 0.36)	72.05% (σ = 0.31)
	7	1,934 (1.58%)	57.83% (σ = 0.29)	1.13 (σ = 0.41)	72.87% (σ = 0.30)
	8	1,378 (1.13%)	58.99% (σ = 0.28)	1.18 (σ = 0.48)	72.52% (σ = 0.29)
	9	955 (0.78%)	58.83% (σ = 0.28)	1.20 (σ = 0.51)	72.06% (σ = 0.28)
	10	723 (0.59%)	55.06% (σ = 0.27)	1.25 (σ = 0.53)	69.25% (σ = 0.28)
	>10	2,918 (2.60%)	55.88% (σ = 0.27)	1.52 (σ = 0.85)	70.43% (σ = 0.27)
	Total	122,012	58.29% (σ = 0.30)	1.14 (σ = 0.37)	78.72% (σ = 0.30)

TABLE II :

 II Label usage according to the number of issues in GitHub projects (σ < 0.33 for all the values of % LI).

					# projects with X issues and % of labeled issues (LI)
	# labels	X ∈ (0, 10)	X ∈ [10, 100)	X ∈ [100, 1000)	X ∈ [1000, ∞)
		# proj.	% LI	# proj.	% LI	# proj.	% LI	# proj.	% LI
	1		45,150	69.55%	9,996	18.39%	407	6.03%	8	28.67%
	2		17,268	75.94%	13,203	43.48%	545	11.81%	10	14.41%
	3		3,915	79.65%	8,771	52.64%	694	21.15%	10	5.78%
	4		1,071	82.18%	5,121	58.36%	703	28.68%	15	14.45%
	5		421	84.31%	3,115	62.72%	656	30.52%	14	17.28%
	6		223	78.10%	1,995	63.68%	773	31.52%	20	12.80%
	7		84	84.65%	1,177	66.88%	651	39.11%	22	24.43%
	8		49	84.64%	823	67.81%	481	43.11%	25	22.83%
	9		19	83.57%	518	72.16%	394	41.96%	24	28.31%
	10	4	85.00%	337	69.74%	363	42.89%	19	21.60%
	>10	12	82.64%	780	73.85%	1,765	52.25%	361	33.95%
	Total	68,216	70.10%	45,836	45.68%	7,432	34.81%	528	29.54%
		30,000
			26,020
		25,000
	Number of issues	20,000 15,000 10,000
					7,070
		5,000				3,103	1,401	1,154	918	724
		0

TABLE III :

 III Label influence correlation analysis.

	# labels used in the project	avg. time to solve (hours)	med. time to solve (hours)	avg. issue age (hours)	med. issue age (hours)	% solved	avg. people involved	# issues
	0	786.43 (σ = 2,498.07)	26.93	7,927 (σ = 6,466.29)	6,774	22.53	3.35	2,408,224
	1	795.40 (σ = 2,343.27)	46.18	4,516 (σ = 4,861.38)	2,577	43.51	3.40	111,839
	2	808.60 (σ = 2,199.81)	74.92	5,867 (σ = 6,005.09)	3,747	48.76	5.44	187,705
	3	937.30 (σ = 2,418.90)	101.30	7,540 (σ = 7,233.70)	6,346	53.21	9.73	158,172
	4	998.50 (σ = 2,600.54)	111.80	8,646 (σ = 7,724.24)	7,427	55.27	15.61	134,617
	5	1,111.00 (σ = 2,671.35)	145.70	9,196 (σ = 7,996.22)	8,081	56.30	20.93	111,186
	6	1,060.00 (σ = 2,640.91)	116.40	9,729 (σ = 8,416.60)	8,335	58.82	25.91	99,749
	7	1,152.00 (σ = 2,761.60)	127.20	9,330 (σ = 7,769.62)	8,268	57.95	35.18	91,584
	8	1,139.00 (σ = 2,831.19)	116.40	10,610 (σ = 8,539.56)	9,154	59.28	46.79	81,657
	9	982.90 (σ = 2,599.07)	70.40	10,100 (σ = 8,669.41)	8,612	63.23	51.24	70,146
	10	1,425.00 (σ = 2,811.23)	306.40	8,321 (σ = 6,418.98)	7,276	47.59	71.67	84,193
	> 10	1,148.00 (σ = 2,789.25)	148.10	8,302 (σ = 8,607.11)	5,918	60.19	129.18	727,024
	ρ	0.80	0.74	0.54	0.47	0.73	1.00

TABLE IV :

 IV Excerpt of the clusters detected.

	Cluster	Labels
		0 -backlog 1 -ready 2 -working 3 -done breaking
	A	bug build contribution documentation duplicate easy fix enhancement invalid jump in p1 p2 p3 question refactoring
		removal taken wontifx
	B	bug change docs duplicate enhancement feature fixed invalid migrations question wontfix
		activity bug discuss documentation duplicate enhancement
	C	feature improvement incompatible invalid milestone question
		remove task urgent wontfix
		bug component-keymap-emacs component-keymap-vim
		component-logic component-mode-haskell component-mode-perl
		component-notyi component-ui component-ui-cocoa component-ui-gtk
		component-ui-pango component-ui-vty duplicate enhancement
	D	frontend-gtk frontend-pango frontend-vty imported invalid
		keymap-emacs keymap-vi milestone-release0.4 milestone-release0.7
		opsys-all opsys-osx osx performance priority-critical
		priority-high priority-low priority-medium
		type-cleanup-refactor type-other type-task usability wontfix
	E	0.0.1 0.0.3 0.1.0 0.2.0 0.3.0 0.4.0 0.5.0 1.0.0 1.0.0.rc1 breaking bug doc enhancement invalid new update

TABLE V :

 V Coverage analysis of the label families.

	Family	# labels	% projects
	Priority	1,027 (2.33%)	4.33%
	Version	2,703 (6.14%)	1.68%
	Workflow	1,972 (4.48%)	5.67%
	Architecture	1,104 (2.51%)	2.00%

Note that several labels can be defined at project level. We only consider those labels being used in the project issues.

The row regarding those projects not using labels is provided for the sake of comparison. Note that in this case we considered non-labeled issues to calculate the metrics.


Javier Luis

Cánovas Izquierdo

Valerio Cosentino
email: valerio.cosentino@inria.fr

Belén Rolandi

Alexandre Bergel
email: alexandre.bergel@me.com

Jordi Cabot
email: jordi.cabot@inria.fr

Gila

GiLA: GitHub Label Analyzer

à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

I. INTRODUCTION

The development of Open-Source Software (OSS) relies on a community of users who freely contribute to their advance [START_REF] Bird | Open Borders? Immigration in Open Source Projects[END_REF]. Among the different ways to contribute to OSS projects (e.g., testing, code contributions, etc.), probably the most common one (at least one of the easiest to start with) consists in providing feedback by reporting bugs or requesting improvements and new features. This feedback is typically expressed as new issues for the project managed by an issuetracking system.

OSS projects are hosted in web-based code hosting platforms providing a source code repository for the project plus other project management tools. Among those platforms, GitHub 1 has become, with more than 10 millions of repositories, one of the most popular ones. Similar to the other platforms, GitHub provides a light-weight and flexible issuetracker which also includes labeling support. Labels (sometimes also known as tags) are a simple yet effective way to attach additional information (e.g., metadata) to project issues [START_REF] Storey | How Software Developers Use Tagging to Support Reminding and Refinding[END_REF]. A label can give any user an immediate clue about what sort of topic the issue is about, what development task the issue is related to, or what priority the issue has. Furthermore, labels are also useful for project administrators, since they can serve both as classification and filtering mechanism, thus facilitating the managing of the project. The GitHub issue-tracking system comes with a few generic labels (i.e., bug, duplicate, etc.), but more interestingly, 1 http://github.com it provides the capability to create new custom labels that are adapted to the specificities of the project2 .

Labeling is regarded as a useful technique to organize and manage projects (see some anecdotical evidence 3 ) but as the project grows, the number of labels and issues also increases and their analysis and management becomes more and more difficult. As a consequence, it may be no longer feasible to infer relevant information by manually browsing them, risking losing useful information for the optimal evolution of the project.

We believe that visualization techniques could be applied here to overcome this challenge. In particular, this paper proposes a tool to better understand how labels are being used in GitHub projects, with the aim of providing more insights into how such projects are being managed. Our tool provides three visualizations addressing three different viewpoints, specifically:

V1

Label usage, which helps to identify the most used labels and which ones are commonly used together.

V2

User involvement, which allows discovering the most active and knowledgeable users around each label.

V3

Typical Label timeline, which provides some insights about how issues under that label evolve over time (e.g., time to be treated).

The tool, called GiLA, is available online [START_REF]GiLA website[END_REF] (together with a demonstration video [START_REF]GiLA demo video[END_REF]) and can be used to explore these viewpoints on all the original projects (i.e., projects that are not a fork of a previous project) in GitHub. We believe that the results favour not only a better comprehension of the project but also help in its advancement, e.g., by helping to quickly identify experts on a particular topic/label. The remainder of the paper is organized as follows. Section II reports on the existing related work. Section III describes and illustrates the three visualizations while Section IV presents the tool functionalities, its architecture and inner workings. Finally, Section V concludes the paper and presents some future work.

II. RELATED WORK

In the last years, we have witnessed the development of many tools and services focused on the exploration and analysis of software repositories, mainly GitHub hosted ones. Even GitHub itself promotes it by means of organizing the so-called GitHub Data Challenge. Some examples would be:

(1) Octoboard4 , which provides a dashboard showing overall activity metrics in GitHub; (2) Popular Coding Convention5 , which analyzes the main programming conventions used in GitHub; or (3) The Open Source Report Card6 , which provides an enriched version of developer profiles. Commercial tools such as KeyLines are also starting to incorporate GitHub analysis as part of their offering or at least as part of their marketing efforts 7 .

Despite all these existing tools, to the best of our knowledge our tool is the only one focusing on the analysis and visualization of how issue labels are used in a project. Other works have been focused on recommending the most suitable labels for software artefacts (e.g., [START_REF] Al-Kofahi | Fuzzy Set Approach for Automatic Tagging in Evolving Software[END_REF], [START_REF] Xia | Tag Recommendation in Software Information Sites[END_REF], [START_REF] Wang | EnTagRec: An Enhanced Tag Recommendation System for Software Information Sites[END_REF]) or have studied the role of issue-tracking systems in software development (e.g., [START_REF] Bissyande | Got issues? Who Cares About it? A Large Scale Investigation of Issue Trackers from GitHub[END_REF], [START_REF] Posnett | Got Issues? Do New Features and Code Improvements Affect Defects?[END_REF]), however, little attention has paid to study how issue labels are used, in particular, with the aim of proposing useful visualization techniques for them.

III. VISUALIZING HOW LABELS ARE USED

This section motivates and explains the three visualizations calculated and rendered by our tool. In particular, such visualizations give insights about (1) the general use of labels in the project, (2) the user involvement on each label category, and (3) the evolution of issues classified under a given label. We illustrate all of them using the Netty project8 , an eventdriven asynchronous network application framework developed in GitHub, as a running example.

A. Label Usage Visualization

Motivation. Having an overview of the labels used in the project and their distribution among issues provides some insights about the relevance of specific topics in the project community and, since an issue in GitHub can be tagged with one or more labels, the relationships between them. Therefore, with this visualization, you can quickly identify the most popular labels and which ones are commonly used together.

Visualization. Label usage information is displayed using a graph-based visualization. Nodes represent labels and edges represent relations between two labels. We say that labels L 1 and L 2 are related if there is at least one issue annotated with both L 1 and L 2 in the project. The strength of a relation between L 1 and L 2 is measured in terms of the number of issues using both labels. The thickness of the edge is used to visualize such strength factor (the greater the number the thicker the edge). Finally, the size of each node indicates the number of issues classified with that label (the greater the number the larger the node). Example. Figure 1 shows this visualization on the example project. According to the node size, we can see that the most used label is defect followed by feature and improvement. The visualization also shows that improvement and feature are the labels most strongly connected, that is, the labels most frequently appearing together. Improvement can also be commonly found together with defect, won't fix and cleanup while feature is often used with won't fix issues. On the other hand, the label gsoc is always used alone. These results may help project developers to identify that the development effort is normally spent in improving and fixing issues of the project while little effort seems to be required to other activites such as documentation or security. It also suggests that quite a number of improvements and feature requests are discarded (relation with won't fix label).

B. User Involvement Visualization

Motivation. Visualizing the people actively participating (i.e., opening, closing, commenting) on issues in a given label category can quickly unveil the community leaders for the different aspects of the project. In particular, it can help identifying the collaborators that most frequently take decisions (e.g., accepting or rejecting issues) with regards to issues affecting a given topic. Similarly, we could use it to uncover potential experts on the topic and therefore people we could assign new issues on that same category in the future.

Visualization. User involvement is shown in a graph-based visualization that highlights the contribution of users on issues tagged with a given label. Users are represented as boxes while the label is represented as a circle in the center. The box size and edge thickness indicate the number of times such user has contributed to issues with that label. In particular, the box width and height are proportional to the number of created and closed issues, respectively. Additionally, box colors allow distinguishing between collaborators (i.e., administrators or members with high-level privileges, as white boxes) and users (i.e., black boxes). Besides, the edge between a user and the label indicates the number of comments such user has made on issues for that label. As before, the thickness indicates the degree of contribution. Example. Figure 2 shows this visualization for the label improvement in the example project. From the size and color of the user rectangles we can see that normanmaurer and trustin are the collaborators that most often participate on improvement issues, specially normanmaurer. The fact that, for this user, the rectangle height is larger than its width, shows that such user has closed issues more times than opened them. The edge connecting the user rectangle with the label also reveals that normanmaurer and trustin are the users more actively commenting this kind of issues.

C. Label Timeline Visualization

Motivation. Analyzing how issues in each label typically evolve over time provides some insights about the main priorities and interests in a project (e.g., what type of issues receives the fastest initial response, which are usually solved quickly and what is the expected end resolution for issues classified under a certain label).

Visualization. Issue evolution is represented as a tree-like visualization that shows the average time for some important events in the lifecycle of issues tagged with a given label. The tree has a main path which includes two events: (1) average time for the first comment and (2) average time for the first comment from a collaborator of the project. This path then forks into three subpaths to represent (1) the percentage of issues closed (and the average time to be closed), (2) the percentage of issues merged (and the average time to be merged) and (3) the percentage of issues still open (and their average age).

Example. Figure 3 shows the resulting visualization for the label feature of the Netty project. The white and black stopping points tell that, in average, it takes around 7 days until an issue with this label receives the first comment (both from any user and a collaborator). As for the resolution, we can see that 5.73% of the issues are merged, 79.74% are closed and 14.54% are still open. The ordering of the branches reflects that issues are merged faster (7.96 days) than they are closed (34.67 days in average) whereas the average age for pending issues is 222.13 days.

IV. TOOL SUPPORT These visualiations have been implemented as part of the a web application, called GiLA [START_REF]GiLA website[END_REF]. This section describes its main functionalities and architecture.

A. Functionalities

To use the tool, users have to access to the GiLA website [START_REF]GiLA website[END_REF]. Figure 6 shows the landing page of GiLA, shortly describing the available visualizations and together with a combobox to select the project to analyze. The combobox is initialized with a set of popular projects but users can just type the name of the project they would like to see. Once one project is selected, the user can access to the results page by clicking on the Show me! button.

The results page includes first three basic metrics concerning the labels usage in the project and then the three visualizations described before. Figure 7 shows an screenshot of the results page for the Netty project. While the first visualization is shown as soon as the user loads the page, the second and third visualizations (i.e., user involvement and label timeline visualizations) are initially empty since the user has to first choose the specific label to use in the visualization. Once done, the corresponding visualization is rendered and shown. Additionally, visualizations include tooltips to help the user to understand the results. For instance, in the last visualization the user can pass the mouse over the elements to visualize extra information.

B. Tool Architecture

Figure 5 depicts the main components of the GiLA architecture. As can be seen, GiLA is decomposed into a server side, which includes the dataset and the web services; and the client side with the website.

As dataset to perform the analysis, GiLA uses GHTorrent, an scalable and offline mirror of data offered through the GitHub REST API [START_REF] Gousios | The ghtorrent dataset and tool suite[END_REF]. The dataset is provided as both relational and NoSQL databases. The former is the one used as source for the analysis. This database is loaded into our server and preprocessed to speed up the queries later on. In short, once a visitor selects a project, the tool does not trigger the computation of the data required for the visualization but directly queries the calculated results stored in auxiliary tables.

As a trade-off, the results show the label usage in the project until the latest snapshot of GHTorrent uploaded to the tool. New snapshots can be imported by means of a set of developed scripts which update the database and create the auxiliary tables needed. Projects that are a fork of other projects are filtered out.

An excerpt of the core tables in the database schema is shown in Figure 4. The database stores information about projects, members, issues and labels. A project (see projects table) is managed by two kinds of collaborators: the owner 9 and the project members (see project_members table), the former has full control over the repository, whereas the latter are granted full management permissions. Each project keeps track of the submitted issues (see issues table), along with their labels (see issue_labels table), events (see issue_events table) and comments (see issue_comments table). Any user (see users table) can submit and comment issues, however, only collaborators can close (merge or reject) or reopen them. In addition, they are the only ones that can define labels (see repo_labels table) and assign them to issues.

On the server side, a set of web services (one per visualization plus one returning the list of GitHub projects) implemented as Java servlets are in charge of querying the database data and give it back to the client side for its rendering. For this rendering phase, and after evaluating several alternatives (among them D3 10 , Google Charts 11 , pygal 12 or Processing 13 ) we opted for D3, a JavaScript library for manipulating documents based on data which facilitates the creation of SVG-based graphics. We believe this was the most suitable approach in our case given its efficiency and broad support.

V. CONCLUSION

In this paper we presented a tool to help managers and developers to better understand how issue labels are employed in their OSS projects. The tool comes with three visualizations to highlight the frequency and relationships between labels, which users are involved in the creation/resolution of issues categorized with such labels and their typical evolution along the development process. The tool is provided as a web service available at [START_REF]GiLA website[END_REF] for the analysis of any GitHub project. A demonstration video has also been made available [START_REF]GiLA demo video[END_REF].

Our future plans for the tool include adding information about the evolution of a set of labels associated to an issue (when they are added/removed and by who) and adding some label-based metrics that help compare your project with regard to other GitHub projects using similar sets of labels. We are also interested in evaluating the scalability of the approach (e.g., how the database update performs when new GHTorrent snapshots are released) as well as conducting a user study to collect developer comments that may improve our visualizations. Finally, we would like to extend our approach to other web-based code hosting services (e.g., BitBucket, SourceForge, etc.) to identify whether there exist similarities in the way of managing projects across different social coding platforms.

Fig. 1 :

 1 Fig. 1: Label usage visualization for the Netty project.

Fig. 2 :

 2 Fig. 2: User involvement visualization for label improvement in the Netty project.

Fig. 3 :

 3 Fig. 3: Label timeline visualization for label feature in the Netty project.

Fig. 4 :

 4 Fig. 4: Excerpt of the database schema considered.

Fig. 5 :

 5 Fig. 5: Tool Architecture.

Fig. 6 :

 6 Fig. 6: Landing page of GiLA.

Fig. 7 :

 7 Fig. 7: GiLA results page for the Netty project.

Browsing issues for any project quickly uncovers the usage of custom labels, some of them such as documentation or feature even more popular than some of the default ones, see: http://modeling-languages.com/use-ofissue-labels-in-github(accessed 11-26-14)

3 http://www.stateofcode.com/2013/06/using-github-issues-effectively (accessed 11-26-14) http://www.ianbicking.org/blog/2014/03/use-github-issues-to-organize-aproject.html (accessed 11-26-14) https://twitter.com/briantford/status/492431188578275328 (accessed 11-26-14)

http://octoboard.com(accessed 11-26-14)

http://sideeffect.kr/popularconvention (accessed 11-26-14)

https://osrc.dfm.io(accessed 11-26-14)

http://keylines.com/network-visualization/graphing-github (accessed 11-26-14)

[START_REF] Bissyande | Got issues? Who Cares About it? A Large Scale Investigation of Issue Trackers from GitHub[END_REF] https://github.com/netty/netty (accessed 11-26-14)