OBJECTIVE CHARACTERIZATION OF AUDIO SIGNAL QUALITY: APPLICATIONS TO MUSIC COLLECTION DESCRIPTION

Dominique Fourer Geoffroy Peeters

UMR STMS (IRCAM -CNRS -UPMC) dominique@fourer.fr, geoffroy.peeters@ircam.fr

In this paper, we propose a set of audio features to describe the quality of an audio signal. Audio quality is here considered as being modified by the chain of processes/effects applied to the individual instrument tracks to obtain the final mix of a musical piece. Thus, the quality also depends on the mastering processes applied to the final mix or the signal degradation caused by MP3 compression. To evaluate our proposal, we created a large set of artificial mixes and also used real-world studio mixes. Using unsupervised and supervised classification methods, we show that our proposed audio features can detect the processing chain. Since this processing chain applied in professional studio has evolved over the years, we use our audio features to directly predict the decade during which a music track was recorded.

Index Termsaudio quality, music information retrieval, audio reverse-engineering, database indexing, music remixing.

INTRODUCTION

Audio signal quality can be related to subjective and objective audio signal attributes resulting from a sophisticated digital signal processing chain. Despite, a consistent definition of audio quality has not yet been offered, researchers agree to say that it depends on a combination of transformations applied to audio signal since studio recording (or pure synthesis) to the resulting final mix obtained after mastering [START_REF] Duarte Pestana | Spectral characteristics of popular commercial recordings 1950-2010[END_REF]. Knowing the audio quality of a music track is full of interest for applications such as music streaming or playlist generation since it allows to decide which sound file (when several occurrences of the same music track exist in a database) has the better quality or should be discarded.

Among the first works related to the objective description of the audio quality, the standard ISO/IEC 15938-4 (MPEG-7 Audio) [START_REF] Bitzer | Coding of moving pictures and audio ISO/IEC JTC 1/SC 29/WG 11[END_REF] proposes a set of informative features to describe the audio content and the signal quality. More recently, audio quality has re-gained interest. In 2011, [START_REF] Tardieu | Production effect: Audio features for recordings techniques description and decade prediction[END_REF][START_REF] Duarte Pestana | Spectral characteristics of popular commercial recordings 1950-2010[END_REF] propose to use a set of audio quality features to estimate the decade during which a musical piece was recorded and help the navigation in large music collection. In 2015, [START_REF] Wilson | A lexicon of audio quality[END_REF] performs a set of per- ceptual experiments in which users judge the audio quality through listening tests. This leads to an audio quality lexicon. [START_REF] Kendrick | Perceived audio quality of sounds degraded by nonlinear distortions and single-ended assessment using hasqi[END_REF][START_REF] Fazenda | Perception and automated assessment of audio quality in user generated content[END_REF][START_REF] Wilson | Variation in multitrack mixes : analysis of low-level audio signal features[END_REF] propose a set of audio quality features to predict the results of the perceptual experiments using a machine learning approach.

In this paper, we extend this approach, i.e. we propose an objective description of the audio quality. We aim at describing the audio signal content related to the mixing process and the signal quality. Hence, this approach is directly related to the audio signal reverse engineering problem [START_REF] Reiss | Reverse engineering of a mix[END_REF][START_REF] Gorlow | Reverse engineering stereo music recordings pursuing an informed twostage approach[END_REF] which finds applications in music description, audio branding [START_REF] Bronner | Audio Branding. Brands, Sound and Communication[END_REF], automatic playlist generation or automatic music mixing [START_REF] Barchiesi | Automatic target mixing using least-squares optimization of gains and equalization settings[END_REF].

The paper is organized as follows. In Section 2, we address problem of the objective description of the audio quality by first describing the set of considered alteration effects. We then present the set of proposed audio signal quality features. In Section 3, we apply this framework to automatically predict the audio signal alterations and to automatically predict the music decade. We finally discuss the results and present future works in Section 4.

TOWARDS OBJECTIVE AUDIO QUALITY ASSESSMENT

A music audio signal is the result of the mixing of a set of effects and transformations applied on separated tracks (elementary signals) in order to obtain an artistic mixture [START_REF] Barchiesi | Automatic target mixing using least-squares optimization of gains and equalization settings[END_REF]. These transformations, which are subjectively applied in studio by sound engineers, depend on the targeted music medium and are often difficult to reverse. Because of this, audio mixing reverse engineering recently gained interest. It was addressed for example in [START_REF] Reiss | Reverse engineering of a mix[END_REF][START_REF] Gorlow | Reverse engineering stereo music recordings pursuing an informed twostage approach[END_REF]. Furthermore, after studio mixing, audio signals can also be degraded by signal transformations resulting from users manipulation (e.g. remixing, resampling, lossy compression, etc.). This results in a loss of quality, which can be characterized for example, by an addition of noise or a reduction of the content frequency bandwidth.

Hence, the audio quality characterization problem addressed here consists in either obtaining cues about the signal mixing process or (ideal case) recovering the exact signal properties related to the transformations which have been applied to the signal.

Considered audio signal alteration effects

In the present work we only consider a restricted set of signal alterations. Those however cover a wide range of commonly used audio transformations, as often addressed in the music processing literature [START_REF] Zölzer | Digital Audio Signal Processing[END_REF][START_REF] Barchiesi | Automatic target mixing using least-squares optimization of gains and equalization settings[END_REF][START_REF] Reiss | Reverse engineering of a mix[END_REF][START_REF] Luo | Compression history identification for digital audio signal[END_REF][START_REF] Duarte Pestana | Spectral characteristics of popular commercial recordings 1950-2010[END_REF][START_REF] Avendano | Frequency-domain source identification and manipulation in stereo mixes for enhancement, suppression and re-panning applications[END_REF].

Dynamic range control

Dynamic range control is a non-linear effect, which modifies the overall loudness perception of the audio signal. This can be done by amplifying the volume of quiet sounds and reducing the volume of loud sounds. The result is a transformation of the dynamic range of the input signal. A compressor or an expander, are often parameterized by a detection threshold L and a gain ratio R. Delay parameters τ att v , τ rel v and τ att g , τ rel g can also be used to obtain smoothed detection and gain functions. Furthermore, different compressor parameters can also be separately applied on arbitrary chosen signal frequency bands [START_REF] Zölzer | Digital Audio Signal Processing[END_REF]. In our experiments, we apply a compressor-expander (compander) on a Linear Instantaneous (LI) mixture using SoX with parameters depending on the signal profile, as detailed in Table 1.

Table 1. SoX profiles used for dynamic range control.

Profile name

SoX parameters speech compand 0.02,0. • Mono effect consists in duplicating both channels:

s 2 [n] = s 1 [n].
• Amplitude panning aims at simulating the direction of arrival of a mixture by changing its amplitude on each channel. For a given azimuth θ ∈ [-π 2 , + π 2 ] and a monophonic input signal x[n], the left and right channels are given by • Head Related Transfer Function (HRTF) filtering aims at simulating the binaural perception of a source signal which arrives from a given direction. This effect is simply obtained by convolving the source signal x[n] by the left and right impulse responses corresponding to the given azimuth θ. Our experiments use the CIPIC HRTF database [START_REF] Algazi | The cipic hrtf database[END_REF].

s 1 [n] s 2 [n] = sin θ 2 + π 4 cos θ 2 + π 4 x[n]. (1)
If θ = 0 then s 1 [n] = s 2 [n], if θ = -π 2 then s 1 [n] = x[n], s 2 [n] = 0, if θ = π 2 then s 1 [n] = 0, s 2 [n] = x[n].

Lossy audio compression

We simulate this alteration by encoding the original audio mixture in the MP3 audio file format [START_REF]Information technology generic coding of moving pictures and associated audio information ISO/IEC 13818-3 -part 3: Audio[END_REF] (which is the most popular format for audio storage, transfer and playback). For this, we used the LAME encoder with four different quality profiles corresponding to the following bitrates: 16 kbs, 64 kbs, 128 kbs and 320 kbs.

Content alteration

We also consider two content alteration effects:

• Resampling consists in changing the number of samples. Down-sampling reduces it while up-sampling increases it. This results in an increase or a decrease of the original signal frequency bandwidth. It is often related to a loss of signal quality (in particular for downsampling). Our original sampling rate is 44.1 kHz. We consider four profiles in our experiments: 8 kHz (down), 16 kHz (down), 32 kHz (down), 96 kHz (up).

• Noise addition is simply achieved by merging (addition) the original signal with a white Gaussian noise signal. We used five different Signal-to-Noise Ratio (SNR) values: -15 dB, -5 dB, 10 dB, 20 dB and 45 dB.

Audio quality features

For the purpose of describing the audio quality, we collect previously proposed audio quality features from - [START_REF] Duarte Pestana | Spectral characteristics of popular commercial recordings 1950-2010[END_REF] (average spectrum), - [START_REF] Bitzer | Coding of moving pictures and audio ISO/IEC JTC 1/SC 29/WG 11[END_REF] (monophony detector, cross-channel correlation, relative delay, balance, DC-offset, frequency bandwidth, background noise-level), - [START_REF] Tardieu | Production effect: Audio features for recordings techniques description and decade prediction[END_REF] (dynamic histogram, cochleagram difference, spectral stereo phase spread) and - [START_REF] Widmer | Automatic music detection in television productions[END_REF] (spectral entropy). For reason of restricted length of the paper, we refer the readers to the respective publications for a detailed description. The entire set of features used in this study is summarized in Table 2. Features corresponding to a time series (DH, AS, SE and BW) are statistically summarized by their mean, median, Inter-Quartile Range (IQR), standard deviation, skewness, kurtosis, minimum, maximum, entropy and slope over time. This leads to 10 scalars for each framebased feature. For DH and AS, we also compute the centroid and the position of the maximum. The CD feature is represented by a matrix D of size M × N (M denotes frequency bands and N time-frames). It is summarized by 5 scalars expressed as Each sound file is represented by a vector of features, which is used as the input of a classification method.

CD 1 = 1 M N M -1 m=0 N -1 n=0 |D m,n |, CD 2 = σ 1 N N -1 n=0 |D m,n | , CD 3 = 1 M N N -1 n=0 M -1 m=0 D m,n

NUMERICAL EXPERIMENTS

In order to validate our proposed audio quality features, we use the following two experimental scenarios.

Scenario 1: prediction of audio signal alterations

In this scenario, we used the Medley dataset [START_REF] Bittner | Medleydb: A multitrack dataset for annotation-intensive MIR research[END_REF] which provides 122 music pieces available in two forms: 1) studio mix (which is a stereo high-quality artistic mixture with effects applied to the various instrument tracks), 2) the set of separate multi-track instruments which allows to build a flat monophonic mix (named LI mix). On those, we apply a set of 27 different signal alteration effects. Those are detailed in Section 2.1 and summarized in Table 3. It should be noted that several sets of parameters can be used for a given effect to obtain several instances (e.g. dynamic range compression uses 5 different profiles). The goal of our experiment is to automatically recognize the alteration effects applied to the audio, using our audio features (cf. Table 2). We consider this as a set of classification tasks with 7 classes for the dynamic range control, 5 for spatialization, 5 for lossy compression and 10 for content alteration. We try to solve these tasks using both supervised and unsupervised classification.

Supervised classification

We tested the following supervised classification algorithms: K-Nearest Neighbor (KNN), Linear Discriminant Analysis (LDA) [START_REF] Anderson | An Introduction to Multivariate Statistical Analysis[END_REF] and multiclass (one-against-all) Radial Basis Function (RBF) kernel Support Vector Machine (SVM) [START_REF] Lauer | MSVMpack: a multiclass support vector machine package[END_REF]. We performed a 3-fold cross-validation, with randomly partitioned equal sized folds. For the KNN method, we used K = 9 (which was empirically found to provide the best results) and a city-block distance (also named Manhattan) which provided better results than an Euclidean one.

The classification results are indicated in Table 4 in terms of class-recall and global accuracy. The corresponding confusion matrices are indicated in Tables 5 (a)-(d). The table shows that SVM and LDA outperform the KNN method in all cases. Best results are obtained to predict spatialization classes (98%), then lossy compression (88%), then content alteration (85%) and finally dynamic range (71%). It should be noted that discriminating the dynamic range classes is a much harder problem but interestingly, our method achieves (but with difficulties according to Table . 5 (a)) to discriminate the "streaming" and the "music 2" profiles which use almost identical SoX parameters.

Unsupervised classification

For the unsupervised classification, we use a simple k-means algorithm [START_REF] Seber | Multivariate Observations[END_REF] using again the city-block distance. The number K of clusters is fitted to the number of classes to discriminate (cf. Table 3). For the unsupervised case, we used an automatic feature selection algorithm in order to detect the most relevant features to be used for each classification task. For this, we used the Inertia Ratio Maximization using Feature Space Projection (IRMFSP) [START_REF] Peeters | Automatic classification of large musical instrument databases using hierarchical classifiers with inertia ratio maximization[END_REF] algorithm. Thus, the most informative features are indicated in Table 7.

In Table 6 we indicate the results of the unsupervised classification in terms of cluster purity [START_REF] Manning | Introduction to Information Retrieval[END_REF] (a cluster containing a single class has a purity of 1). Using less than 7 signal features, we obtain purities above 0.6 for all classification tasks. As for the supervised case, the spatialization classes are also the best predicted in the unsupervised case. These results are very promising for future unsupervised applications.

Scenario 2: Decade prediction

Since the processing chain applied in professional studio has evolved over the years, and since our audio features allows describing this chain, we test the use our audio features to predict directly the decades during which a music track was recorded. For this, we consider the dataset of 1980 music tracks previously used in [START_REF] Tardieu | Production effect: Audio features for recordings techniques description and decade prediction[END_REF]. It covers the years from 1960 to 2000. We subdivide them into 5 decades considered as classes. Each musical piece is resampled at 44.1 kHz and only the 60 first seconds are analyzed. As before, we tested the following supervised classification algorithms: KNN, LDA and SVM using a 3-fold cross-validation scheme. We also applied an artist filter in order to ensure that the same artist is not present both in the training and testing set (it allows to prevent over-fitting [START_REF] Flexer | A closer look on artist filters for musical genre classification[END_REF]).

Results are indicated in Table 8 in terms of recall and accuracy. The best results are obtained using SVM (63%). In comparison, the results obtained in [START_REF] Tardieu | Production effect: Audio features for recordings techniques description and decade prediction[END_REF] were 61% without adding Mel-Frequency Cepstrum Coefficients (MFCC) to the set of audio quality features and 64% including them. It should be noted however, that the splitting between train and test used here is not the same as the one used in [START_REF] Tardieu | Production effect: Audio features for recordings techniques description and decade prediction[END_REF].

CONCLUSION

In this paper, we proposed a set of audio features for the automatic characterization of the audio quality. During an experiment with artificially generated mixing, we showed that the proposed approach can efficiently predict the type of audio effects and alterations applied to the original audio signal. With real commercial music tracks, we also showed that the same approach can be used to predict the decade during which the track was recorded. This approach paves the way of more sophisticated systems designed for automatic mixing, playlist generation or database indexing. Future works will consist in further investigating a larger set of "realistic" signal alterations and using audio quality annotated dataset as in [START_REF] Wilson | Perception of audio quality in productions of popular music[END_REF].

20 5 :

 5 -60,-40,-10 -5 -90 0.1 streaming compand 0.3,1 6:-70,-60,-20 -5 -90 speech/music compand 0.1,0.3 -60,-60,-30,-15,-20,-12,-4,-8,-2,-7 In our experiment we only consider two-channels mixtures. We denote by s 1 [n] and s 2 [n] the left and right channels discrete-time signals and by S 1 [n, m] and S 2 [n, m] their discrete Short-Time Fourier Transforms (STFT). The considered stereophonic transformations are described as follows.

1 M M - 1 m=0

 11 LAME encoder: http://lame.sourceforge.net/ and CD 4 = σ D m,n , where σ(x) denotes the standard deviation of the time series x.

•

  Phase panning changes the delay between both chan-

nels. It can be implemented by transforming the STFT phase. If we denote by ∆φ, the phase lag parameter, this effect is obtained by S 2 [n, m] = S 1 [n, m]e j∆φ . s 2 [n] is then obtained by inversion of the STFT. SoX -Sound eXchange: http://sox.sourceforge.net/

Table 2 .

 2 List of proposed audio signal quality features.

	Feature name	Label	Designation	#
	Dynamic histogram [3]	DH	mixture dynamic range 12
	Average spectrum[1]	AS		12
	Cochleagram difference [3]	CD	stereo quality	5
	Spectral Stereo Phase Spread [3] SSPS		1
	Monophony detector[2]	isMono		1
	Cross-channel correlation[2]	CCCor		1
	Relative delay[2]	RDelay		1
	Balance [2]	Bal		1
	DC-offset [2]	DCOff	signal content	1
	Root Mean Squared amplitude	aRMS		1
	Spectral Entropy [17]	SE		10
	Frequency bandwidth[2]	BW		10
	Background noise level [2]	BNL		1
	Total number of features	57

Table 3 .

 3 List of considered simulated audio alteration.

	Effect name (# of classes) Profiles	#
		no compression (LI mix)	1
	Dynamic range control (7)	reference studio mix	1
		dynamic range compression (SoX)	5
		reference studio mix	1
		mono	1
	Spatialization (5)	amplitude panning	4
		phase panning	4
		HRTF	4
	Lossy compression (5)	original WAV file MP3 compression (LAME encoder) 4 1
	Content alteration (10)	resampling addition of a white Gaussian noise	5 5

Table 4 .

 4 Supervised classification results for each task in terms of recall and of accuracy.

	Method	Dynamic range control class name no co. stud. spee. stream. spe./mus. mus.1 mus.2	Accuracy
	KNN	0.36	0.80 0.23 0.08	0.26		0.44	0.06	0.32
	LDA	0.72	0.98 0.65 0.48	0.89		0.96	0.27	0.71
	SVM	0.90	0.99 0.48 0.37	0.23		0.95	0.09	0.57
	Method	Spatialization class name stud. mix mono amp. pan. phs. pan. HRTF	Accuracy
	KNN		0.31	0.34	0.90	0.85	0.98	0.83
	LDA		0.94	1		0.97	0.57	1		0.86
	SVM		0.96	0.89	1	0.97	0.99	0.98
	Method	Lossy compression class name orig. wav mp3 320kbs mp3 128kbs mp3 64kbs mp3 16kbs	Accuracy
	KNN	0.34	0.20		0.20	0.99		1		0.55
	LDA	0.73	0.80		0.85	1		1		0.88
	SVM	0.75	0.59		0.43	1		0.99	0.75
	Method	Content alteration class name 8kHz 16kHz 32kHz 44kHz 96kHz -15dB -5dB 10dB 20dB 45 dB	Acc.
	KNN	0.83	0.72	0.51	0.25	0.32	1	1	0.90	0.61	0.24	0.64
	LDA	0.87	0.89	0.81	0.55	0.68	1	1	0.98	0.94	0.77	0.85
	SVM	0.90	0.80	0.70	0.57	0.65	0.99	1	0.89	0.66	0.46	0.76

Table 5 .

 5 Confusion matrices obtained using the best classification method for each prediction task.(a) LDA method applied to dynamic range control effect prediction.

										(b) LDA method applied to dynamic range control effect prediction.
	ref. class	Classified as no comp. stud. mix spee. stream. spe./mus. mus.1 mus.2		ref. class	Classified as 8kHz 16kHz 32kHz 44kHz 96kHz -15dB -5dB 10dB 20dB 45 dB
	no comp. 88 stud. mix	119	1		4 1	30 1			8kHz 16kHz 32kHz	106 4	13 119 2	99	6 5	1 14	1	3 3 2
	spee.			79	14	8		21		44kHz	1	6	7	67	19		22
	stream. spe./mus. mus.1 mus.2			9 4 40	59 49	1 109 5	117	53 33		96kHz -15dB -5dB 10dB 20dB		2	15	19	83	122	122	119 2	3 115	3 5
										45dB		3		20			5	94
	(c) LDA method applied to lossy compression detect.	(d) SVM method applied to spatialization effect detect.		(e) SVM method applied to decade predict.
	ref. class	Classified as wav 320kbs 128kbs 64kbs 16kbs		ref. class	Classified as stud.mix mono amp. pan. phs. pan. HRTF		ref. class	60s 70s	Classified as 80s 90s 2000s
	orig. wav	89	25	8			stud.mix	117			1		4		60s	327 49	8	7	5
	mp3 320kbs 18	97	7			mono		108		14			70s	73	123 136 45	19
	mp3 128kbs 5	13	104			amp. pan.			487	1				80s	9	74	272 35	6
	mp3 64kbs mp3 16kbs			122	122		phs. pan. HRTF	1	2 2	11	475	485		90s 2000s	26 11	29 11	61 16	216 64 44 314

Table 6 .

 6 Unsupervised classification results for each task in terms of cluster purity, optimal number of features, number of clusters K.

	Task	Cluster purity # of feat. K
	Dynamic range control	0.62	4	7
	Spatialization	0.80	5	5
	Lossy compression	0.78	3	5
	Resampling	0.71	2	5
	Noise add.	0.78	7	5
	Noise add.+Resampling 0.63	6	10

Table 7 .

 7 Top-10 features sorted by descending order of the Fisher score (FS)[START_REF] Peeters | Automatic classification of large musical instrument databases using hierarchical classifiers with inertia ratio maximization[END_REF] for each classification task.

	rank	dynamic range control feat. name FS	spatialization feat. name FS	lossy compression content alteration feat. name FS feat. name FS
	1	aRMS	0.80	isMono	0.71 mean AS	1	median AS 1
	2	SSPS	0.70	CCCor	0.60 slope AS	0.96 mean BW	0.74
	3	min DH	0.42	CD5	0.53 max BW	0.39 max BW	0.69
	4	CCCor	0.23	SSPS	0.45 mean BW	0.22 min SE	0.33
	5	DH pk. pos. 0.13	CD1	0.23 median AS 0.06 mean SE	0.28
	6	CD1	0.05	CD4	0.07 std BW	0.06 skew. SE	0.18
	7	entropy DH 0.04	aRMS	0.05 std AS	0.05 median SE 0.16
	8	skew. DH	0.03	CD3	0.04 max AS	0.02 max SE	0.14
	9	std. DH	0.02	Bal	0.02 skew. BW	0.02 entropy SE 0.12
	10	slope DH	0.01	slope AS	0.02 iqr BW	0.02 min DH	0.10

Table 8 .

 8 Supervised classification results for decade prediction in terms of recall and of accuracy.

	Method	60s	70s	Class name 80s 90s	2000s	Accuracy
	KNN	0.77 0.38 0.63 0.49 0.71	0.60
	LDA	0.69 0.43 0.62 0.52 0.77	0.60
	SVM	0.83 0.31 0.69 0.55 0.79	0.63