Alexey Ozerov

Ngoc Q K Duong

Louis Chevallier

Weighted nonnegative tensor factorization: on monotonicity of multiplicative update rules and application to user-guided audio source separation

Keywords: Weighted nonnegative matrix factorization, weighted nonnegative tensor factorization, multiplicative updates, monotonicity, user-guided audio source separation

This report focuses on so-called weighted variants of nonnnegative matrix factorization (NMF) and more generally nonnnegative tensor factorization (NTF) approximations. First, we consider multiplicative update (MU) rules to optimize these approximations, and we prove that some results on monotonicity of MU rules for NMF generalize without restrictions to both the NTF and the WNTF cases. Second, we propose new weighting strategies for an existing NTF-based user-guided audio source separation method. Experimental evaluation shows that these weightings lead to better source separation than the same model without using the weights. The best configuration of the proposed method was entered into the fourth community-based Signal Separation Evaluation Campaign (SiSEC 2013).

I. INTRODUCTION

Nonnegative matrix factorization (NMF) [START_REF] Lee | Algorithms for non-negative matrix factorization[END_REF] or more generally nonnegative tensor factorization (NTF) [START_REF] Cichocki | Nonnegative Matrix and Tensor Factorizations[END_REF] approaches consist in approximating nonnegative matrices (tensors) by lower rank structured matrices (tensors) composed by nonnegative latent factors. These approximations can be useful for revealing some latent data structure [START_REF] Févotte | Nonnegative matrix factorization with the Itakura-Saito divergence. With application to music analysis[END_REF] or for compressing the data [START_REF] Ozerov | Coding-based informed source separation: Nonnegative tensor factorization approach[END_REF]. Thus they have recently gained a great popularity in both machine learning [START_REF] Sivic | Discovering objects and their location in images[END_REF], [START_REF] Mairal | Online learning for matrix factorization and sparse coding[END_REF] and signal processing [START_REF] Smaragdis | Non-negative matrix factorization for polyphonic music transcription[END_REF]- [START_REF] Cemgil | Probabilistic latent tensor factorization framework for audio modeling[END_REF] communities. As such, they were applied for non-supervised image classification [START_REF] Sivic | Discovering objects and their location in images[END_REF], image inpainting [START_REF] Mairal | Online learning for matrix factorization and sparse coding[END_REF], polyphonic music transcription [START_REF] Smaragdis | Non-negative matrix factorization for polyphonic music transcription[END_REF], audio source separation [START_REF] Virtanen | Monaural sound source separation by perceptually weighted non-negative matrix factorization[END_REF]- [START_REF] Lefèvre | Semi-supervised NMF with time-frequency annotations for single-channel source separation[END_REF], audio coding [START_REF] Ozerov | Coding-based informed source separation: Nonnegative tensor factorization approach[END_REF], etc.

In this report we focus on a particular kind of NMF (NTF) methods called weighted NMF (WNMF) (weighted NTF (WNTF)), where the contribution of each data point to the approximation is weighted by a nonnegative weight. Weighted NMF was already used for face feature extraction [START_REF] Blondel | Weighted non-negative matrix factorization and face feature extraction[END_REF], ratings prediction [START_REF] Kim | Weighted nonnegative matrix factorization[END_REF], [START_REF] Gu | Collaborative filtering weighted nonnegative matrix factorization incorporating user and item graphs[END_REF], mass spectrometry analysis [START_REF] Dubroca | Weighted NMF for high-resolution mass spectrometry analysis[END_REF], as well as for audio source separation with perceptual modeling [START_REF] Virtanen | Monaural sound source separation by perceptually weighted non-negative matrix factorization[END_REF], [START_REF] Kirbiz | Perceptually weighted non-negative matrix factorization for blind single-channel music source separation[END_REF]. A particular WNTF modeling, namely a weighted three-way "PARAFAC" factor analysis, was considered in [START_REF] Paatero | A weighted non-negative least squares algorithm for three-way "PARAFAC" factor analysis[END_REF]. The NTF modeling for tensors with possibly missing entries [START_REF] Yilmaz | Probabilistic latent tensor factorisation[END_REF] could be considered as a partial case of WNTF, where the weights can be either ones (observed) or zeros (missing).

Concerning the algorithms to compute NMF (NTF) decompositions, one of the most popular choice among the others [START_REF] Cichocki | Nonnegative Matrix and Tensor Factorizations[END_REF] are the multiplicative update (MU) rules [START_REF] Lee | Algorithms for non-negative matrix factorization[END_REF], [START_REF] Févotte | Nonnegative matrix factorization with the Itakura-Saito divergence. With application to music analysis[END_REF]. While in terms of convergence speed MU is not the fastest approach [START_REF] Cichocki | Nonnegative Matrix and Tensor Factorizations[END_REF], its popularity can be explained by the simplicity of the derivation and implementation, as well as by the fact that the nonnegativity constraints are inherently taken into account. Deriving MU rules for WNMF (WNTF) is quite straightforward, and it was already done for WNMF, e.g., in [START_REF] Kim | Weighted nonnegative matrix factorization[END_REF], [START_REF] Dubroca | Weighted NMF for high-resolution mass spectrometry analysis[END_REF]. However, few works analyse their convergence properties in terms of monotonicity of the optimized criterion, i.e., by theoretically studying whether the criterion to be minimized remains non-increasing at each update. While several results on monotonicity of MU rules for NMF exist [START_REF] Lee | Algorithms for non-negative matrix factorization[END_REF], [START_REF] Févotte | Algorithms for nonnegative matrix factorization with the beta-divergence[END_REF], [START_REF] Yang | Unified development of multiplicative algorithms for linear and quadratic nonnegative matrix factorization[END_REF], less work (e.g., [START_REF] Blondel | Weighted non-negative matrix factorization and face feature extraction[END_REF], [START_REF] Gu | Collaborative filtering weighted nonnegative matrix factorization incorporating user and item graphs[END_REF]) just reports the results on monotonicity of MU rules for WNMF in particular cases of the Euclidean (EU) distance and the Kullback-Leibler (KL) divergence. To our best knowledge there are no such results either for WNTF, or for other divergences, such as α or β-divergences [START_REF] Févotte | Nonnegative matrix factorization with the Itakura-Saito divergence. With application to music analysis[END_REF], [START_REF] Cichocki | Nonnegative matrix factorization with alpha-divergence[END_REF].

The contribution of this work is two-fold. First, we provide some theoretical results on monotonicity of MU rules for WNMF and WNTF. Second, we propose new WNTF strategies applied to a user-guided audio source separation framework inspired by [START_REF] Ozerov | Multichannel nonnegative tensor factorization with structured constraints for user-guided audio source separation[END_REF] and using a particular NTF model (multichannel NMF model from [START_REF] Ozerov | Multichannel nonnegative matrix factorization in convolutive mixtures for audio source separation[END_REF]). The user-guided source separation approach from [START_REF] Ozerov | Multichannel nonnegative tensor factorization with structured constraints for user-guided audio source separation[END_REF] is based on a temporal user-specified annotation of source activities (or time codes) that are further used to constrain NTF model learning. Our original WNTF framework targets to (i) re-weight the data so as to enhance the importance of segments with less sources (or less NTF components), and to (ii) re-weight the data so as to re-equilibrate the impact of different data type segments that may have different lengths.

The remaining of this report is organized as follows. WNMF and WNTF together with a general formulation of the MU rules are presented in section II. New results on monotonicity of these rules for both WNMF and WNTF are given in section III. Section IV is devoted to the description of the proposed WNTF-based user-guided audio source separation framework, which is then evaluated in section V on the development data of the fourth community-based Signal Separation Evaluation Campaign (SiSEC 2013) "professionally produced music recordings" (PPMRs) task. The best configuration according to this evaluation was then entered into the evaluation campaign. Some conclusions are drawn in section VI.

II. WEIGHTED NMF AND WEIGHTED NTF

A. Weighted NMF Let V ∈ R F ×N + a nonnegative matrix of data that is approximated by a nonnegative matrix V ∈ R F ×N + being a product of two nonnegative latent matrices W ∈ R F ×K + and H ∈ R K×N + as V ≈ V = WH, which can be rewritten in a scalar form as v f n ≈ vfn = k w f k h kn , (1)
where v f n , vfn , w f k and h kn denote, respectively, the entries of V, V, W and H. The goal of NMF consists in finding the latent parameters Z {W, H} minimizing the following criterion:

C NMF (Z) = D(V| V) = f,n d(v f n |v f n ), (2)
where vfn is given by (1) and d(x|y) is some divergence (e.g., α-divergence [START_REF] Cichocki | Nonnegative matrix factorization with alpha-divergence[END_REF] or β-divergence [START_REF] Févotte | Nonnegative matrix factorization with the Itakura-Saito divergence. With application to music analysis[END_REF]). As such, we here consider only the case of separable matrix divergences D(V| V), i.e., those computed by element-wise summing of scalar divergences [START_REF] Yang | Unified development of multiplicative algorithms for linear and quadratic nonnegative matrix factorization[END_REF], [START_REF] Dhillon | Generalized nonnegative matrix approximations with Bregman divergences[END_REF]. However, we believe that our results below can be easily generalized to some nonseparable divergences.

Let B = [b f n ] f,n ∈ R F ×N +
a matrix of nonnegative weights, the goal of WNMF is to optimize the same criterion as [START_REF] Cichocki | Nonnegative Matrix and Tensor Factorizations[END_REF], except that all the entries in summation are weighted by b f n :

C WNMF (Z) = f,n b f n d(v f n |v f n ). (3)
The MU rules [START_REF] Lee | Algorithms for non-negative matrix factorization[END_REF]- [START_REF] Févotte | Nonnegative matrix factorization with the Itakura-Saito divergence. With application to music analysis[END_REF], [START_REF] Yang | Unified development of multiplicative algorithms for linear and quadratic nonnegative matrix factorization[END_REF] consist in updating in turn each scalar parameter z as follows:

z ← z ([∇ z C(Z)] -/[∇ z C(Z)] + ) η , (4)
where η > 0, C(Z) is the cost function to be minimized, its derivative with respect to (w.r.t.) the parameter writes

∇ z C(Z) = [∇ z C(Z)] + -[∇ z C(Z)] -, (5)
and the summands are both nonnegative. Note that the decomposition (5) is not unique and this algorithm is rather a heuristic one. Thus, neither its convergence, no its monotonicity is guaranteed and should be studied case by case [START_REF] Lee | Algorithms for non-negative matrix factorization[END_REF], [START_REF] Févotte | Algorithms for nonnegative matrix factorization with the beta-divergence[END_REF], [START_REF] Yang | Unified development of multiplicative algorithms for linear and quadratic nonnegative matrix factorization[END_REF].

Assuming the derivative over the second argument of our divergence d(x|y) can be written

d ′ (x|y) = d ′ + (x|y) -d ′ -(x|y), (6)
where d ′ + (x|y) and d ′ -(x|y) are both nonnegative, one can write the following MU rules for WNMF:

w f k ← w f k n b f n d ′ -(v f n |v f n )h kn n b f n d ′ + (v f n |v f n )h kn η , (7)
h kn ← h kn f b f n d ′ -(v f n |v f n )w f k f b f n d ′ + (v f n |v f n )w f k η . (8)
Note that this is not the only way to write the MU rules, since the decomposition (5) could be probably obtained differently. However, this is the way the MU rules are derived in many cases, e.g., for the β-divergence as in [START_REF] Févotte | Nonnegative matrix factorization with the Itakura-Saito divergence. With application to music analysis[END_REF], [START_REF] Févotte | Algorithms for nonnegative matrix factorization with the beta-divergence[END_REF] and for all separable divergences (including α-divergence and αβ-Bregman divergence) considered in [START_REF] Yang | Unified development of multiplicative algorithms for linear and quadratic nonnegative matrix factorization[END_REF].

B. Weighted NTF

We build our presentation following a general formulation of tensor decompositions originally called probabilistic latent tensor factorization (PLTF) [START_REF] Cemgil | Probabilistic latent tensor factorization framework for audio modeling[END_REF], [START_REF] Yilmaz | Probabilistic latent tensor factorisation[END_REF]. However, we rather call it here NTF, since we do not push its probabilistic aspect. Our presentation follows very closely [START_REF] Cemgil | Probabilistic latent tensor factorization framework for audio modeling[END_REF], [START_REF] Yilmaz | Probabilistic latent tensor factorisation[END_REF], except that we are using slightly different notations and we consider the weighted case.

Instead of matrices (so-called 2-way arrays) we now consider tensors (so-called multi-way arrays) that are all assumed nonnegative. For example

E = [e f nk ] f,n,k ∈ R F,N,K
+ is a 3-way array. However, for the sake of conciseness and following [START_REF] Cemgil | Probabilistic latent tensor factorization framework for audio modeling[END_REF], [START_REF] Yilmaz | Probabilistic latent tensor factorisation[END_REF] we use single-letter notations for both tensor indices and their domains of definition, e.g., j = f nk and J = {f, k, n} f,n,k in the example above.

Let us introduce the following notations:

• I is the set of all indices, • V = [v i0 ] i0∈I0 is the data tensor and I 0 ⊂ I is the set of visible indices, • Z α = [z α iα ] iα∈Iα (α = 1, .
. . , T ) are T latent factors (tensors), I α ⊂ I, and we also require

I = I 0 ∪I 1 ∪. . .∪I T . • Īα = I\I α denotes the set of indices that are not in I α .
With these conventions the matrix approximation (1) can be extended to

v i0 ≈ vi0 = ī0∈ Ī0 T α=1 z α iα . (9)
This formulation generalizes in fact many existing models. Let us give some examples for a better understanding. Assuming Z 1 = W, Z 2 = H, I = {f, n, k}, I 0 = {f, n}, I 1 = {f, k} and I 2 = {n, k}, we get back to the NMF decomposition [START_REF] Lee | Algorithms for non-negative matrix factorization[END_REF]. The TUCKER3 decomposition [START_REF] Kiers | Towards a standardized notation and terminology in multiway analysis[END_REF] (this example is from [START_REF] Yilmaz | Probabilistic latent tensor factorisation[END_REF])

v jkl ≈ vjkl = p,q,r z 1 jp z 2 kq z 3 lr z 4 pqr (10)
can be represented as ( 9) by defining I = {j, k, l, p, q, r}, I 0 = {j, k, l}, I 1 = {j, p}, I 2 = {k, q}, I 3 = {l, r} and I 4 = {p, q, r}. These models can be also visualized by graphical representations from [START_REF] Cemgil | Probabilistic latent tensor factorization framework for audio modeling[END_REF], [START_REF] Yilmaz | Probabilistic latent tensor factorisation[END_REF], where all indices are represented by graph nodes and the latent factors by corresponding non-oriented graph vertices. NMF (1) and TUCKER3 (10) models are represented by such graphs on Fig. 1 (A) and (B), where the visible indices are shaded and the non-visible ones are white. Let Z = {Z α } α=1,...,T set of all latent factors and B = [b i0 ] i0∈I0 a tensor of nonnegative weights. WNTF criterion to be minimized writes:

C WNTF (Z) = i0 b i0 d(v i0 |v i0 ). (11)
Finally, relying on the decomposition (6), as in the WNMF case, one can derive the following MU rules for WNTF:

z α iα ← z α iα īα b i0 d ′ -(v i0 |v i0 ) α ′ =α z α ′ i ′ α īα b i0 d ′ + (v i0 |v i0 ) α ′ =α z α ′ i ′ α η . (12)

III. NEW RESULTS ON MONOTONICITY OF MU RULES FOR WNMF AND WNTF

There exist several results on monotonicity of MU rules for NMF with β-divergence [START_REF] Févotte | Algorithms for nonnegative matrix factorization with the beta-divergence[END_REF] and NMF with other divergences (e.g., α-divergence or αβ-Bregman divergence) [START_REF] Yang | Unified development of multiplicative algorithms for linear and quadratic nonnegative matrix factorization[END_REF]. However, these results are not really extended neither to a general NTF case, nor to the WNMF or WNTF cases. Some results for WNMF exist [START_REF] Blondel | Weighted non-negative matrix factorization and face feature extraction[END_REF], [START_REF] Gu | Collaborative filtering weighted nonnegative matrix factorization incorporating user and item graphs[END_REF], but only in particular cases of the EU distance and the KL divergence. In order to fill in these gaps in the state of the art we here provide the insite on the monotonicity of the NMF MU rules for NTF, WNMF and WNTF cases. We start with the following lemma. Lemma 1. When updating one latent factor Z α , given all other factors {Z α ′ } α ′ =α fixed, criterion (3) is nonincreasing under WNMF MU rules [START_REF] Ozerov | Multichannel nonnegative matrix factorization in convolutive mixtures for audio source separation[END_REF] if and only if for each i 0 ∩ i α criterion

C WNTF (Z α i0∩iα ) = i0∩ īα b i0∩ īα d(v i0∩ īα |v i0∩ īα ), (13)
(where Z α i0∩iα = [z α i0∩iα,i0∩ īα ] i0∩ īα ) is non-increasing under these rules. Proof:: The sufficiency is evident, the necessity follows from the fact that two sets of entries of Z α involved in two different criteria [START_REF] Ozerov | Multichannel nonnegative tensor factorization with structured constraints for user-guided audio source separation[END_REF]

(corresponding to two different indices i 0 ∩ i α = i ′ 0 ∩ i ′ α )
do not overlap. Proposition 1 (WNMF monotonicity ⇔ WNTF monotonicity). Assume WNMF MU rules [START_REF] Smaragdis | Non-negative matrix factorization for polyphonic music transcription[END_REF], [START_REF] Virtanen | Monaural sound source separation by non-negative matrix factorization with temporal continuity and sparseness criteria[END_REF] and WNTF MU rules [START_REF] Ozerov | Multichannel nonnegative matrix factorization in convolutive mixtures for audio source separation[END_REF] are derived for the same η, for the same divergence d(x|y) and under the same decomposition [START_REF] Mairal | Online learning for matrix factorization and sparse coding[END_REF]. WNMF criterion (3) is non-increasing under the WNMF MU rules if and only if WNTF criterion (11) is non-increasing under the WNTF MU rules.

Proof:: The sufficiency follows from the fact that WNTF generalizes WNMF. To prove the necessity, it is enough to show, thanks to lemma 1, that for updating one latent sub-factor Z α i0∩iα , given i 0 ∩ i α ∈ I 0 ∩ I α and given all other factors {Z α ′ } α ′ =α fixed, expressions (9), ( 13) and ( 12) can be recast into the form of expressions (1), ( 3) and (8) for WNMF.

Let us define

F = |I 0 ∩ Īα |, N = 1 and K = | Ī0 ∩ I α |,
where |A| denotes cardinality of a set A. We can now unfold multi-way index sets I 0 ∩ Īα and Ī0 ∩ I α onto 1-way index sets {f } = {1, . . . , F } and {k} = {1, . . . , K} using some bijections f → [i 0 ∩ īα ](f ) and k → [ ī0 ∩i α ](k). We then define elements of matrices/vectors

V ∈ R F ×1 + , B ∈ R F ×1 + , W ∈ R F ×K + and H ∈ R K×1 + as v f 1 = v i0∩iα,[i0∩ īα](f ) , b f 1 = b i0∩iα,[i0∩ īα](f ) , h k1 = z α i0∩iα,[ ī0∩iα](k)
and

w f k = w[i0∩ īα](f )[ ī0∩iα](k) , where wi0∩ īα, ī0∩iα = ī0∩ īα α ′ =α z α ′ i ′ α .
It can be easily checked that with these notations expressions (9), ( 13) and ( 12) rewrite as expressions (1), ( 3) and ( 8) for WNMF.

Even if according to proposition 1 the monotonicity of the WNMF MU rules implies that of the WNTF MU rules, the monotonicity of the WNMF MU rules has not been guaranteed yet. To obtain such results, let us first formulate the following lemma, which strictly speaking is not a direct consequence of proposition 1, but it is very similar to it. Lemma 2 (NMF monotonicity ⇔ NTF monotonicity). This lemma formulates exactly as proposition 1, but without weighting, i.e., with trivial weighting: b i0 = 1 (i 0 ∈ I 0 ). Proof:: The proof is exactly as that of proposition 1, except with trivial weighting. Proposition 2 (NMF monotonicity ⇔ WNMF monotonicity). Assume that WNMF MU rules [START_REF] Smaragdis | Non-negative matrix factorization for polyphonic music transcription[END_REF], [START_REF] Virtanen | Monaural sound source separation by non-negative matrix factorization with temporal continuity and sparseness criteria[END_REF] are derived for some η, some divergence d(x|y) and under some decomposition [START_REF] Mairal | Online learning for matrix factorization and sparse coding[END_REF]. WNMF criterion (3) is non-increasing under the WNMF MU rules for a trivial weighting B 0 = [START_REF] Lee | Algorithms for non-negative matrix factorization[END_REF] f,n (making WNMF (3) equivalent standard NMF (2)) if and only if WNMF criterion (3) is non-increasing under the WNMF MU rules for any weighting B.

Proof:: The sufficiency being evident, let us prove the necessity. We carry the proof for H update (8), given W fixed. According to lemma 1, it is enough to show that for each n = ñ the following criterion is non-increasing under the WNMF MU updates of h ñ = [h ñk ] k (the same trick is used in proofs in [START_REF] Févotte | Algorithms for nonnegative matrix factorization with the beta-divergence[END_REF]):

C WNMF (h ñ) = f b f ñ d(v f ñ|v f ñ). ( 14
)
Thus, in the following we fix n = ñ and consider ( 14) as target criterion.

We first assume b f ñ ∈ N (f = 1, . . . , F ). Let us introduce binary matrices A f (ñ) ∈ {0, 1} bf ñ ×F . Each matrix A f (ñ) is zero everywhere except the f -th column that contains ones. We then define a binary matrix A(ñ) = [a lf (ñ)] l,f ∈ {0, 1} L×F , where L = f b f ñ, that stacks vertically matrices A f (ñ) as follows:

A(ñ) = [A 1 (ñ) T , . . . , A F (ñ) T ] T . (15)
Using A(ñ) we rewrite approximation (1) as follows:

v ′ lñ ≈ v′ lñ = f,k a lf (ñ)w f k h kñ , (16)
where v ′ lñ = f a lf (ñ)v f ñ. Let us first remark that ( 16) is an NTF approximation (w.r.t. v ′ lñ and v′ lñ ) according to our general formulation [START_REF] Cemgil | Probabilistic latent tensor factorization framework for audio modeling[END_REF]. Non-weighted NTF criterion (11) (i.e., [START_REF] Kirbiz | Perceptually weighted non-negative matrix factorization for blind single-channel music source separation[END_REF] with trivial weighting b ′ lñ = 1) for approximation [START_REF] Blondel | Weighted non-negative matrix factorization and face feature extraction[END_REF] writes

C NTF (h ñ) = l d(v ′ lñ |v ′ lñ ). (17)
It can be easily shown that criterion [START_REF] Kim | Weighted nonnegative matrix factorization[END_REF] is strictly equivalent to criterion [START_REF] Kim | Nonnegative matrix partial co-factorization for spectral and temporal drum source separation[END_REF] and that the corresponding MU updates w.r.t. h ñ are the same. Moreover, since the monotonicity of non-weighted NMF is assumed, it implies, according to lemma 2, the monotonicity of non-weighted NTF criterion [START_REF] Kim | Weighted nonnegative matrix factorization[END_REF], and thus that of criterion [START_REF] Kim | Nonnegative matrix partial co-factorization for spectral and temporal drum source separation[END_REF].

We have proven the result for b f ñ ∈ N. Since multiplying all the weights by a positive constant factor does not affect monotonicity of MU updates, the result is proven for b f ñ ∈ Q + . Finally, since Q + is dense in R + and since both MU updates and the corresponding criteria are all continuous w.r.t. weights and parameters, the result is proven for b f ñ ∈ R + .

Propositions 2 and 1 can be summarized by the following theorem.

Theorem 1 (NMF monotonicity ⇔ WNTF monotonicity). Assume WNMF MU rules (7), ( 8) and WNTF MU rules (12) are derived for the same η, for the same divergence d(x|y) and under the same decomposition [START_REF] Mairal | Online learning for matrix factorization and sparse coding[END_REF]. WNMF criterion (3) is non-increasing under the WNMF MU rules for a trivial weighting B 0 = [1] f,n (making WNMF (3) equivalent standard NMF (2)) if and only if WNTF criterion (11) is non-increasing under the WNTF MU rules.

We have shown that the results on the monotonicity of NMF MU rules derived as in section II-A (in particular, those from [START_REF] Févotte | Algorithms for nonnegative matrix factorization with the beta-divergence[END_REF] for β-divergence and those from [START_REF] Yang | Unified development of multiplicative algorithms for linear and quadratic nonnegative matrix factorization[END_REF] for separable divergences, e.g., for α and αβ-Bregman divergences) generalize to WNMF, NTF and WNTF cases.

IV. APPLICATION TO USER-GUIDED AUDIO SOURCE SEPARATION

The considered user-guided audio source separation framework is similar to the one from [START_REF] Ozerov | Multichannel nonnegative tensor factorization with structured constraints for user-guided audio source separation[END_REF], except that here we assume all model parameters, including mixing coefficients, nonnegative. Thus, we rather adopt the multichannel NMF modeling from [START_REF] Ozerov | Multichannel nonnegative matrix factorization in convolutive mixtures for audio source separation[END_REF] (the one referred as "MU rules" in [START_REF] Ozerov | Multichannel nonnegative matrix factorization in convolutive mixtures for audio source separation[END_REF]). We then introduce two complementary weighting strategies.

A. WNTF model for source separation

Let X = [x lf n ] l,f,n ∈ C L×F ×N the short-time Fourier transform (STFT) of the multichannel mixture, where l, f and n denote, respectively, the channel, the frequency and the time indices. We consider its power-spectrogram

V = [v lf n ] l,f,n ∈ R L×F ×N + ([v lf n ] l,f,n = |x lf n | 2
l,f,n ) as data tensor. Assuming there are J spatial sources altogether represented by K latent additive components, the model approximation writes

v lf n ≈ vlfn = j,k q ljf u jk w f k h kn , (18)
where tensor Q = [q ljf ] l,j,f ∈ R L×J×F + models the time-invariant spatial filters of J sources, matrix U = [u jk ] j,k ∈ R J×K + represent the correspondence between J sources and K latent additive components, and matrices W =

[w f k ] f,k ∈ R F ×K + and H = [h kn ] k,n ∈ R K×N +
, model, respectively, the spectral patterns and the time activations of latent additive components. A graphical representation of this model is given on Fig. 1 (C).

Let B = [b lf n ] l,f,n ∈ R L×F ×N +
a tensor of nonnegative weights and d(x|y) β-divergence (see, e.g., [START_REF] Févotte | Nonnegative matrix factorization with the Itakura-Saito divergence. With application to music analysis[END_REF] for a definition). According to [START_REF] Févotte | Nonnegative matrix factorization with the Itakura-Saito divergence. With application to music analysis[END_REF] d ′ (x|y) = y β-2 (y -x), thus it can be represented as ( 6) with d ′ + (x|y) = y β-1 and d ′ -(x|y) = y β-2 x, and general WNTF MU rules (12) become:

q ljf ← q ljf n,k b lf n v lf n vβ-2 lf n u jk w f k h kn n,k b lf n vβ-1 lf n u jk w f k h kn η , (19)
u jk ← u jk l,f,n b lf n v lf n vβ-2 lf n q ljf w f k h kn l,f,n b lf n vβ-1 lf n q ljf w f k h kn η , (20)
w f k ← w f k l,j,n b lf n v lf n vβ-2 lf n q ljf u jk h kn l,j,n b lf n vβ-1 lf n q ljf u jk h kn η , (21)
h kn ← h kn l,j,f b lf n v lf n vβ-2 lf n q ljf u jk w f k l,j,f b lf n vβ-1 lf n q ljf u jk w f k η . (22)

B. User-guided approach

User-guided source separation approach we consider here is the one from [START_REF] Ozerov | Multichannel nonnegative tensor factorization with structured constraints for user-guided audio source separation[END_REF] developed for separation of PPMRs. The difference lies in the fact that a weighting is used within this modeling. This approach consists in introducing some structure into approximation [START_REF] Gu | Collaborative filtering weighted nonnegative matrix factorization incorporating user and item graphs[END_REF] by setting to zero some factor entries, which then remain zero under multiplicative updates ( 19) - [START_REF] Févotte | Algorithms for nonnegative matrix factorization with the beta-divergence[END_REF]. The main steps of this approach are summarized below, and more details can be found in [START_REF] Ozerov | Multichannel nonnegative tensor factorization with structured constraints for user-guided audio source separation[END_REF].

1) The user (manually) performs a temporal segmentation of the sources to separate (see Fig. 2 (A) for an example) and decides on the number of latent components (indexed by k in section IV-A) per source (e.g., 2 for bass, 7 for vocals, etc.).

2) The temporal segmentation and the number of components per source are reflected in matrix H in the forms of zeros. E.g., if source 1 is assigned two first components and is silent between frames 100 and 200, then we set h 1n = h 2n = 0 for n = 100, . . . , 200. The other coefficients are randomly initialized to positive values.

3) The remaining parameters are initialized using and ad-hoc procedure described in Section IV.H of [START_REF] Ozerov | Multichannel nonnegative matrix factorization in convolutive mixtures for audio source separation[END_REF], based on WNMF of the stacked channel spectrograms (initialization of W) and clustering of the spatial cues (initialization of Q and U). This WNMF is performed with a similar temporal weighting as the one described in section IV-C and used in the next step. 4) Iterate WNMF MU rules ( 19) -( 22) with weights B specified by one of strategies from section IV-C below. 5) Compute K latent components estimates in the STFT domain by Wiener filtering as follows:

ĉklfn = v-1 lf n j q ljf u jk w f k h kn x lf n , (23)
where vlfn is defined in [START_REF] Gu | Collaborative filtering weighted nonnegative matrix factorization incorporating user and item graphs[END_REF]. 6) As discussed in [START_REF] Ozerov | Multichannel nonnegative tensor factorization with structured constraints for user-guided audio source separation[END_REF], for PPMR separation, neither temporal segmentation, nor spatial cues can completely disambiguate component -source assignments. Thus, if needed, the user can listen to the components and manually fine-tune their grouping.

C. Proposed temporal weighting strategies

We introduce two new temporal weighting strategies for our model, where the weights b lf n vary only over the temporal dimension indexed by n and are constant over l and f . We also explain how these strategies, being complementary, can be combined.

1) Weighting to enhance data purity: Some source separation approaches, e.g., [START_REF] Wang | Investigating single-channel audio source separation methods based on non-negative matrix factorization[END_REF], first learn source spectral patterns W from some training data consisting of clean source examples, and then perform NMF decomposition on a test mixture, while keeping pre-learned spectral patterns W fixed. Other approaches, e.g., [START_REF] Ozerov | Multichannel nonnegative tensor factorization with structured constraints for user-guided audio source separation[END_REF] summarized in section IV-B, introduce information about source activities via zeros in H, and, assuming trivial weights (b lf n = 1) as in [START_REF] Ozerov | Multichannel nonnegative tensor factorization with structured constraints for user-guided audio source separation[END_REF], spectral patterns W are learned from all data without making any distinction between segments with clean sources and those with mixed sources.

We propose a weighting providing a continuum of strategies between these two, thus possibly allowing to choose a better intermediate strategy. Given a temporal segmentation, as explained above, we define the weights for estimating parameters of (18) as one of the following two options:

b src lf n (λ) = [1/#(act_src n )] λ , (24)
b cmp lf n (λ) = [1/#(act_cmp n )] λ , (25)
where λ ∈ [0, +∞) is a fixed parameter, and #(act_src n ) and #(act_cmp n ) are, respectively, the number of active sources and the number of latent NTF components in n-th frame. An example of such weighting is represented on Fig. 2 (B). One can see that, in case of two sources in the mixture and for λ → +∞, this approach is equivalent to the first state-of-the-art strategy [START_REF] Wang | Investigating single-channel audio source separation methods based on non-negative matrix factorization[END_REF] (for more than two sources, it generalizes [START_REF] Wang | Investigating single-channel audio source separation methods based on non-negative matrix factorization[END_REF] to a slightly more tricky learning). For λ = 0 this approach becomes equivalent to the second strategy [START_REF] Ozerov | Multichannel nonnegative tensor factorization with structured constraints for user-guided audio source separation[END_REF]. The proposed weighting is also related to [START_REF] Kim | Nonnegative matrix partial co-factorization for spectral and temporal drum source separation[END_REF], [START_REF] Lefèvre | Semi-supervised NMF with time-frequency annotations for single-channel source separation[END_REF], where clean and mixed sources are re-weighted with one weight in case of two sources, but extends these approaches to multi-source scenario with segmentation and multiple weights.

2) Weighting to equilibrate segment types: Another potential problem of learning strategy presented in section IV-B is that the types of segment activities for an arbitrary segmentation can have different sizes (e.g., "bass"only segment is too long as compared to "piano"-only segment or "bass +vocals"-only segment is too long as compared to "piano+vocals"-only segment). Such a "dis-equilibrated" structure could lead to over-fitting of the NTF model on a particular source, while modeling poorly other sources. To overcome this issue we propose the following weighting b

lf n (µ) = [1/segm_len n ] µ , (26)
where µ ∈ [0, 1] is a fixed parameter, and segm_len n denotes the number of frames of the same segment type as the n-th frame (e.g., "piano+vocals" or "bass+vocals"). An example is shown on Fig. 2 (C).

3) Combined weightings: Finally, the two above weighting strategies being complementary, they can be combined as

b src lf n (λ, µ) = b src lf n (λ) b lf n (µ), (27)
b cmp lf n (λ, µ) = b cmp lf n (λ) b lf n (µ). (28)
V. EXPERIMENTS A. Data

We evaluate our approach on dev2 subset of SiSEC 20131 PPMRs task development dataset including three 18 -25 second 44100 Hz sampled stereo mixtures to be separated together with the corresponding full 2 -4 minute recordings. The dev1 subset is not considered since it does not include full recordings, and thus the segmental information is very poor (almost all sources are active at the same time).

B. Parameter setting

The STFT is computed using a half-overlapping 93 ms (4096 samples) length sine window. Parameters of MU rules ( 19) -( 22) are set to η = 1 and β = 0, which correspond to the IS divergence [START_REF] Févotte | Nonnegative matrix factorization with the Itakura-Saito divergence. With application to music analysis[END_REF]. Thanks to the results reported in [START_REF] Févotte | Algorithms for nonnegative matrix factorization with the beta-divergence[END_REF] on NMF and to our generalization of these results to WNTF (Sec. III), criterion ( 11) is non-increasing under these updates for any weighting B.

C. Oracle components grouping

Our goal is to assess the separation performance for different considered weighting strategies we proposed. Within the user-guides source separation approach described in section IV-B, a manual user intervention is needed in the first step (to create temporal segmentation) and in the last step (to fine-tune component grouping). While the segmentation step is independent on the weighting, which is itself built on segmentation, the grouping step depends on the weighting. Given a considerable number of weightings we would like to assess, performing manual grouping for each weighting is too time consuming. Thus, inspired by [START_REF] Virtanen | Monaural sound source separation by non-negative matrix factorization with temporal continuity and sparseness criteria[END_REF], and given that for development data the corresponding clean sources are available, the grouping is performed in an oracle greedy manner, as follows. First, source estimates are initialized by zero and reconstructed components [START_REF] Yang | Unified development of multiplicative algorithms for linear and quadratic nonnegative matrix factorization[END_REF] are ranged in order of decreasing energy. Then, while iterating over k = 1, . . . , K, source estimates are updated by adding each time component ĉklfn to only one source so as the total mean squared error between the current source estimates and the clean sources is minimum.

D. Simulation results

The approach was run for both combined weighting types ( 27) and [START_REF] Vincent | Performance measurement in blind audio source separation[END_REF], for different parameters (λ, µ) ∈ {0, 1, 3, 9, 27} × {0, 0.33, 0.66, 0.83, 1}, and assessed in terms of signal-to-distortion ratio (SDR) [START_REF] Vincent | Performance measurement in blind audio source separation[END_REF] and overall perceptual score (OPS) [START_REF] Emiya | Subjective and objective quality assessment of audio source separation[END_REF] source separation measures. The results are given in table I. The best average results in terms of SDR are achieved with b cmp lf n (λ = 3, µ = 0.66), and those in terms of OPS with b cmp lf n (λ = 9, µ = 0). The corresponding configurations outperform the baseline non-weighted approach (λ = µ = 0) by 0.61 dB for SDR and by 2.96 for OPS. For our SiSEC 2013 PPMRs task submission, we have chosen b cmp lf n (λ = 3, µ = 0.66) weighting. With this weighting we have also assessed the results on the same dev2 subset, but with manual component grouping instead of the oracle one. Average SDR and OPS are, respectively, 3.28 dB and 27.98, which is not very different compared to the oracle results (SDR = 3.10 dB and OPS = 28.02, see table I

VI. CONCLUSION

Fig. 1 .

 1 Fig. 1. Graphical representations of different NTF models. (A): NMF model (1); (B): TUCKER3 model (10); (C): Multichannel NMF model (18).

Fig. 2 .

 2 Fig. 2. Examples of proposed weighting strategies. (A): temporal segmentation; (B): weighting enhancing data purity b src lf n (λ), (λ = 2); (C): weighting equilibrating segment types b lf n (µ), (µ = 0.8); (D): combined weighting b src lf n (λ, µ) (λ = 2, µ = 0.8).

  ). .[START_REF] Paatero | A weighted non-negative least squares algorithm for three-way "PARAFAC" factor analysis[END_REF] 1.96 2.36 2.42 25.87 22.60 26.09 24.88 24.76 1 1.81 2.37 2.45 2.51 2.42 27.73 24.39 24.86 24.08 23.81 b src lf n (λ, µ) 3 2.42 2.74 2.59 2.09 3.09 24.62 25.93 25.16 23.86 24.06 9 2.15 2.66 2.44 2.49 2.26 25.53 23.26 22.65 22.12 20.75 27 1.99 2.22 1.85 1.97 2.24 21.17 20.08 19.84 19.75 20.12 0 2.49 2.20 1.96 2.36 2.42 25.87 22.60 26.09 24.88 24.76 1 2.15 2.36 2.54 2.38 2.59 26.86 24.80 26.78 25.68 24.30 b cmp lf n (λ, µ) 3 2.61 2.43 3.10 3.00 1.83 24.55 25.31 28.02 24.49 21.80 9 2.81 1.79 2.13 2.57 2.34 28.83 25.55 21.27 24.36 22.69 27 2.24 2.10 1.71 2.00 1.98 21.50 23.15 23.15 23.22 22.42 TABLE I AVERAGE SDRS / OPSS FOR DIFFERENT WEIGHTING STRATEGIES ON THREE MIXTURES OF DEV2 DATASET OF SISEC 2013 PPMRS TASK.

		Measure		Average SDR (dB)			Average OPS (0 -100)
	Weighting type	λ \ µ	0	0.33 0.66 0.83	1	0	0.33	0.66	0.83	1
		0	2.49 2

http://sisec2013.wiki.irisa.fr

We have proven that certain results on monotonicity of MU rules for NMF generalize to WNMF, NTF and WNTF cases. Moreover, we have proposed new parametric weighting strategies for NTF-based user-guided separation of PPMRs. Some of the parametrizations of these weighting have shown improving source separation performance over the baseline non-weighted case. We have chosen the best weighting setup of the proposed approach to enter the fourth community-based Signal Separation Evaluation Campaign (SiSEC 2013). Future work will include the development and investigation of other weighting strategies for audio source separation, as well as applying WNTF for other applications.


Fergal Mccann
email: galjmccann@gmail.com.

Anders Akerman

Agnès Bénassy-Quéré

Matthieu Crozet

Ron Davies

Benny Jung

Sebastian Krautheim

Eóin Mcguirk

Indirect Exporters

Keywords:

Indirect Exporters are defined as firms exporting through a trade intermediary. Despite numerous recent appearances in theoretical work, empirical evidence comparing these firms to uniquely domestic firms and Direct Exporters does not exist. I show that in Eastern Europe these firms do, as predicted by the theoretical literature, lie between domestic firms and Direct Exporters for a range of performance measures. The "Direct Exporter premium" is the more robust finding, while certain ambiguity surrounding the productivity gap between Indirect Exporters and Domestic firms indicates that these two groups may not be as significantly different.

Introduction

The role played by intermediaries in international trade is a topic of growing interest. The literature has provided ample evidence, across countries of varying levels of economic development, that these firms account for a significant portion of trade flows. [START_REF] Abel-Koch | Firm size and the choice of export mode[END_REF] These studies have been accompanied by a range of papers modeling international trade as involving more than uniquely exporting firms in one country and consumers in another.

This theoretical treatment of the role of trade intermediaries has usually involved either network or matching frameworks [START_REF] Ahn | The role of intermediaries in facilitating trade[END_REF] or extensions of the model of Melitz (2003). [START_REF] Akerman | A theory on the role of wholesalers in international trade based on economies of scope[END_REF] In Blum et al. (2009), the largest firms choose a direct distribution technology to reach foreign consumers themselves. Less productive firms choose an intermediation technology by pairing up with large trading firms to export indirectly. In Ahn et al. (2010) the fixed cost of selling to an intermediary in the firm's own country is lower than the fixed cost of exporting directly. This leads to an identical sorting to that in Blum et al. (2009) where the most productive firms export directly, less productive firms export through intermediaries, and the least productive active firms sell on the domestic market only (termed here "Domestic firms"). Akerman (2010) models wholesalers as having an advantage through economies of scope, i.e. they smooth the fixed cost of selling abroad across many products. He then shows that wholesalers will export a lower volume but more products, and predicts that the sorting mentioned above will hold. Felbermayer and Jung (2009) present a slightly different set-up, focusing on the hold-up problem. They also predict the same sorting pattern as the above three papers.

To the best of my knowledge, empirical evidence on the productivity sorting present in the above-mentioned models does not exist. [START_REF] Antràs | Intermediated trade[END_REF] In each paper mentioned above, the empirical analysis focuses on issues relating to the intermediaries themselves, rather than to the indirectly exporting firms. The aim of this paper is to inform this growing literature on the validity of the productivity sorting results generated by each model. Due to a lack of panel data, I cannot ascertain cleanly whether the productivity sorting is due to selection into export modes or learning from exporting. The purpose of this study is rather to inform the literature on the relative performance of these three types of firms.

We see from probit analysis that Indirect Exporters are more likely than Domestic firms to import, to be foreign owned, to license foreign technology, to be multi-product firms, and to engage in R&D. We also see that Direct Exporters are more likely to engage in most of the above than Indirect Exporters. This suggests support for the performance hierarchy mentioned above. On productivity, the results presented here again suggest that it is valid to assume that Direct Exporters are more productive than both Indirect Exporters and Domestic firms. On the other hand, the assumption that Indirect Exporters are more productive than Domestic firms receives weaker support. In terms of sales and domestic sales, these firms exporting through intermediaries are indeed shown to perform more strongly than purely domestic firms. Using logged output per worker as a measure of productivity, however, the hypothesis that Indirect Exporters are more productive than Domestic firms is shown to not hold. Using a broader definition of Indirect Exporter, however, the productivity hierarchy presented by the theoretical literature is shown to hold perfectly as predicted.

The overall message of the paper is that, while there is a certain ambiguity on the advantage of Indirect Exporters, the overwhelming evidence is in support of the hierarchy predicted by the emerging theoretical literature.

The paper proceeds with a description of the data (Section 2), empirical analysis (Section 3) and a conclusion (Section 4).

Data

The data used come from the Business Environment and Enterprise Performance Survey (BEEPS), which is collected by The European Bank for Reconstruction and Development (EBRD) and The World Bank. This database collects information on, inter alia, a firm's sales, exports, imports, outsourcing, employment, wages, ownership, investment and opinions on corruption and institutional quality. Data are collected for most countries in Eastern Europe for 2002, 2005, 2007, 2008 and 2009. The authors (EBRD and World Bank, 2010) state that "the survey universe was defined as commercial, service or industrial business establishments with at least five full-time employees". The statistical sampling technique used is stratified random sampling. The three levels of stratification used were industry, establishment size and region. Table 1 of the latest report (EBRD and World Bank, 2010) shows that for the total country sample, over 98 percent of the target number of interviews were achieved.

Figures for nominal monetary variables are given in local currency units. I normalize these variables by converting them to US dollar figures using the mean yearly exchange rate from the IMF International Financial Statistics database. The data show that full country coverage was only carried out in 2002, 2005 and 2009. There are just under 30,000 observations in the data altogether, with wider coverage for large countries such as Russia, Turkey and Poland.

Empirical evidence on characteristics of Indirect Exporters

The aim of this paper is to give a first portrait of Indirect Exporters (IE hereon). Firms in the BEEPS are asked "what percentage of establishment's sales were indirect exports (sold domestically to a third party that exports products)", as well as "what percentage of sales were direct exports". For the purposes of the current paper, a firm is Domestic (D hereon) if it has 100 percent of its sales in the domestic market, an IE if some of its sales are in the form of indirect exports but none are direct exports, and a Direct Exporter (DE hereon) if it has some sales in the form of direct exports. Table 1 presents the occurrence of each of these three types of firm. Among the DE are 943 firms that have some direct and some indirect exports. I code these as DE due to the fact that, in the world of Melitz-type models, a firm exporting though both modes must have overcome the higher fixed costs of exporting directly. We see that the majority of firms are domestic firms, as would be expected given previous evidence. [START_REF] Bernard | Intermediaries in international trade: Direct versus indirect modes of export[END_REF] Among exporting firms, the majority of these are DE, with only 3 percent of firms exporting through a trade intermediary. This suggests that the intermediate productivity range in which IE lie in the theoretical literature is a small one in reality. Looking at these firms' share in total sales, we see that IE are twice as important as their frequency would suggest, and unsurprisingly, DE account for a much larger share of sales than their frequency would suggest. We also see that, for both IE and DE, indirect and direct exports, respectively, account for roughly 40 percent of total sales, while DE sell on average 3 percent of their total sales through indirect exports. This indicates that once firms have overcome the costs of exporting directly, indirect exports form a negligible part of their activity. If we relax our definition of IE, we see that firms engaging in indirect exports are more important than as appears in Table 1. If we define IE as any firm engaging in indirect exports (even if they also export directly), and DE as firms that only export directly, we see that 7% of firms are IE and 18% are DE. Changing this definition, however, only alters the sales shares of the two groups by one percent each. If we remove the firms that export both directly and through a wholesaler and include only "pure" IE and DE, we again see that frequencies and sales shares change very little.

Distributions of Exporters

I now present Kernel density plots of firm performance measures for my three categories of exporter. In Figure 1 we see that IE lie clearly between D and DE for logged total sales. As a more accurate proxy for firm productivity, I use logged output per worker. The Kernel density plots for logged output per worker in Figure 2 show that, in the lower part of the distribution, the sorting pattern predicted by the theoretical literature holds. The lower productivity of domestic firms relative to both types of exporter is confirmed at all points in the distribution. As we move up to larger values of logged output per worker, however, we see that there are some points at which the distributions of IE and DE touch, and even cross. The broad pattern, however, is still one in keeping with the predictions of the theoretical literature. Table 4 in the Appendix reports Kolmogorov-Smirnov tests for equality of distributions. We see from this table that the distributions for both IE and DE lie significantly to the right of those for the rest of the sample, and that the distribution for DE lies to the right of that for IE. This offers continued support for the performance hierarchy set out throughout the paper.

Simple Regression Analysis

In this section I engage in simple regression analysis to estimate the "performance premia" for each exporter type. Table 2 gives a more in-depth picture of the characteristics of the different type of exporters. I run five probit regressions which give the probability for IE and DE relative to D firms of participating in the following: importing, having a foreign owner, licensing of foreign technology, Research and Development (R&D) and multi-product sales. Formally I run

P r(Y it = 1) = IndirectExp it + DirectExp it + δ c + δ t + δ s (1)
where Y = 1 if a firm engages in each of the five variables mentioned above and 0 otherwise, IndirectExp it is a dummy for IE, DirectExp it is a dummy for DE, and the other controls are country, time and sector dummies. I then run four OLS regressions of the form

X it = IndirectExp it + DirectExp it + δ c + δ t + δ s (2)
where X it will be logged sales, logged domestic sales, logged output per worker and employment. From the probit regressions in Table 2 we see that IE and DE are more likely to be engaged in all the activities mentioned above than domestic firms. This confirms the theoretical literature's prediction that domestic firms sit at the bottom of a performance hierarchy. These probit regressions also give credence to the predictions of the theoretical literature on the relation between IE and DE. For importing, being foreign owned and engaging in R&D, the likelihood of participation is statistically significantly higher for DE than IE. For licensing of foreign technology and for being a multi-product firm, the marginal effect is larger for IE, but the chi-squared test cannot reject equality of coefficients between the two types of exporter. Using OLS regressions, Table 2 tells us again that for sales, local sales and employment (l), the hierarchy predicted by the theoretical literature holds in its entirety, with the regression coefficients showing that DE and IE lie both significantly above D firms, and the F test showing that the difference in coefficients, with DE being greater than IE, is statistically significant. For logged output per worker, we find that DE are statistically significantly more productive than both IE and D firms, but the data do not give a significant difference in logged output per worker between IE and D firms. Table 2 gives the strong overall impression of a performance hierarchy of firms: DE are the most productive firms, followed by IE, followed by D firms. These results suggest that the theoretical result of an intermediate productivity for those exporting through a trade intermediary seems an accurate approximation, although the "Indirect Exporter premium" is less robust, as evidenced by column (9). The extremely robust nature of the "Direct Exporter premium" suggests that trade intermediaries do indeed lower entry costs to exporting, so that only the top performing firms enter as DE. On the other hand, the less robust nature of the "Indirect Exporter premium" may suggest that intermediaries lower entry costs to such an extent that entry costs for IE are not statistically different to those to domestic production.

We now relax our definition of IE, to allow all firms that export indirectly, regardless of whether they also export directly, to be considered IE, and restrict our definition of DE to firms that only export directly. In Table 3, the regressions of Table 2 are replicated with this new definition of IE and DE. In probit regressions, DE are still more likely to be foreign-owned than IE, who have a higher likelihood than D firms. For R&D and importing, we no longer see a significant difference between IE and DE. IE are now more likely to be multi-product firms than DE. The OLS regressions return the same results as before: for logged sales, logged local sales and employment (l), the expected hierarchy holds. The key difference between Table 3 and Table 2 lies in Column (9), where the coefficient on logged output per worker now tells us that IE, when defined in this more broad way, are indeed more productive than D firms, although only at the five percent level of significance. When moving from the narrow definition of IE, we do indeed see that the hierarchy proposed by the current theoretical literature holds in the data even for our best proxy for productivity. The weaker statistical significance again suggests that the "Indirect Exporter premium" is the least robust of the relationships in the hierarchy.

From Tables 2 and3, the overwhelming picture is one in which firms exporting directly are more productive than firms exporting through a trade intermediary, who are in turn more productive than firms serving only the domestic market. The less robust nature of the latter relationship indicates that a certain caution should be exerted when assuming such a productivity hierarchy, but that on the whole the sorting of firms predicted by the new theoretical literature on intermediated trade is largely valid.

As a final robustness check, I address the worry that intermediary firms themselves may be driving some of the results. A reader may worry that wholesaling firms are more likely to be DE and thus bias results. The data show that the distribution of D, IE and DE firms in the wholesaling industry (NACE2 51) is similar to that shown across the dataset. I run all regressions and re-plot all densities documented above, and see that absolutely all results are robust to the exclusion of firms categorized in NACE2 industry 51.

Conclusion

I have presented first evidence on the performance of Indirect Exporters. In concordance with the predictions of recent theoretical models, Indirect Exporters are found to lie between Domestic firms and Direct Exporters for a number of firm performance measures. The premium of Direct Exporters is unambiguous throughout the paper, whereas that of Indirect Exporters over Domestic firms is supported, but slightly less robust. This suggests, first of all, that intermediaries do certainly lower the fixed cost of exporting, as purported in the literature. Secondly, the fixed cost reductions made possible by trade intermediaries may be hugely economically significant, with the potential that they leave Indirect Exporters and firms serving only the domestic market not significantly differing. These findings can help motivate and validate current and future research regarding the role of intermediaries in mediating international trade.

Figure 1 :Figure 2 :

 12 Figure 1: Kernel densities of logged total sales

Table 1 :

 1 Observations, Relative Frequency, Group Share in Total Sample Sales, Share of Export Types in Firms' Sales

	Firm Type	Observations (Rel. Freq.) Share of Total Ind. Exp./Sales Dir. Exp./Sales
			Sample Sales
	Domestic	20,848 (.76)	.3796	0	0
	Indirect Exporters	887 (.03)	.0610	.3867	0
	Direct Exporters	5,825 (.21)	.5593	.0341	.4032
	Source: BEEPS

Table 2 :

 2 Portrait of exporter types, Base Category is Domestic firms

		(1)	(2)	(3)	(4)	(5)	(6)	(7)	(8)	(9)
		Probit	Probit	Probit	Probit	Probit	OLS	OLS	OLS	OLS
		Importer Foreign For. Tech Multi Prod.	R&D	l	ln(Sales) ln(Dom Sales) Lab. Prod.
	Indirect Exporter 0.264*** 0.116*** 0.043***	0.103***	0.159*** .774***	.781***	.366***	.032
		(16.01)	(7.12)	(4.22)	( 4.92)	(7.04)	(14.85)	(10.07)	(4.37)	(0.58)
	Direct Exporter 0.307*** 0.187*** 0.038***	0.081***	0.204*** 1.168*** 1.437***	.724***	.290***
		(37.19)	(25.92)	(8.48)	(8.35)	(19.61)	(49.08)	(39.19)	(18.76)	(11.04)
	N	18715	26036	13937	20544	17338	26403	21015	20486	20977
	Chi-Sq (probit) or F-test (OLS) for equality of coefficients between IE and DE
	Chi-Sq or F	5.59	16.01	0.16	0.87	3.37	52.95	67.33	17.04	20.28
	p-value	0.018**	0.000***	0.689	0.351	0.066*	0.000*** 0.000***	0.000***	0.001***
	t statistics in parentheses
	* p < 0.05, * * p < 0.01, * * * p < 0.001
	Year, Country, Sector dummies included in all regressions
	marginal effects reported for probit regressions

Table 3 :

 3 Portrait of exporter types, Base Category is Domestic firms. Indirect Exporters defined more broadly. parentheses * p < 0.05, * * p < 0.01, * * * p < 0.001 Year, Country, Sector dummies included in all regressions marginal effects reported for probit regressions

		(1)	(2)	(3)	(4)	(5)	(6)	(7)	(8)	(9)
		Probit	Probit	Probit	Probit	Probit	OLS	OLS	OLS	OLS
		Importer Foreign For. Tech Multi Prod.	R&D	l	ln(Sales) ln(Dom Sales) Lab. Prod.
	Indirect Exporter 0.309*** 0.136*** 0.046***	0.140***	0.198*** 1.012*** 1.082***	0.437***	0.105*
		(27.21)	(11.10)	(5.98)	(9.06)	(12.03)	(26.31)	(18.74)	(7.13)	(2.54)
	Direct Exporter 0.293*** 0.198*** 0.038***	0.064***	0.202*** 1.152*** 1.442***	0.761***	0.308***
		(34.07)	(25.25)	(7.82)	(6.19)	(18.03)	(45.87)	(37.40)	(18.72)	(11.16)
	Chi-Sq (probit) or F-test (OLS) for equality of coefficients between IE and DE
	Chi-Sq or F	1.39	18.23	.66	16.81	.03	11.21	33.92	24.10	21.02
	p-value	0.2389	0.000***	0.4168	0.000***	0.8611	0.000*** 0.000***	0.000***	0.000***
	t statistics in

[START_REF] Ahn | The role of intermediaries in facilitating trade[END_REF] find that intermediaries account for

percent of Chinese exports in 2005. Blum et al. (2010) report that around 35 percent of imports into Chile from Argentina are mediated through wholesalers, with 6 percent through retailers. Akerman (2010) shows that in Sweden in 2005, roughly half of firms exporting goods were wholesalers, while these wholesalers accounted for 15 percent of export volume. Bernard et al. (2010) show that in Italy, 27 per cent of manufacturing exporters are wholesalers, accounting for 11 percent of export volume in 2003. Felbermayr and Jung (2009) show at industry level for the US that the ratio of exports to intermediaries over exports to foreign affiliates is almost always larger than one, and often by orders of magnitude. Bernard, Jensen et al. (2010) show that in the US, "mixed wholesalerretailers", i.e. firms with more than 75 percent of output in those categories, account for two thirds of US exports in 2002. 2 See Antràs and Costinot (2010), Blum et al. (2009), who use a Melitz setting, Petropoulou (2007) or Rauch and Watson (2004).3 See Abel-Koch (2010),[START_REF] Ahn | The role of intermediaries in facilitating trade[END_REF],[START_REF] Akerman | A theory on the role of wholesalers in international trade based on economies of scope[END_REF] or Felbermayr and Jung (2009).[START_REF] Antràs | Intermediated trade[END_REF] Abel-Koch (2010) is the most closely related study to the current paper. She shows that firm size negatively predicts the share of indirect exports in total sales, suggesting that as fims get larger they are more likely to export directly.

Bernard et al. (2007) show that in the US 18 percent of manufacturing firms engaged in exporting in 2002.

A Appendix