Benjamin Dollet

Detlef Lohse
email: d.lohse@utwente.nl

Pinning stabilizes neighboring surface nanobubbles against Ostwald ripening

Pinning of the contact line and gas oversaturation explain the stability of single surface nanobubbles. In this article we theoretically show that the pinning also suppresses the Ostwald ripening process between neighboring surface nanobubbles, thus explaining why in a population of neighboring surface nanobubbles different radii of curvature of the nanobubbles can be observed.

Introduction

When a solid is immersed in a gas-oversaturated liquid, stable nanoscopic spherical-cap shaped so-called surface nanobubbles can be formed at the interface. [START_REF] Lohse | Surface nanobubble and surface nanodroplets[END_REF][START_REF] Craig | Very small bubbles at surfaces -the nanobubble puzzle[END_REF][START_REF] Hampton | Nanobubbles and the nanobubble bridging capillary force[END_REF][START_REF] Attard | Nanobubbles: the big picture[END_REF][START_REF] Attard | The stability of nanobubbles[END_REF] These were first speculated to exist based on stepwise features in atomic forces curves between hydrophobic surfaces [START_REF] Parker | cavities and the long-ranged attraction between hydrophobic surfaces[END_REF][START_REF] Carambassis | Forces Measured between Hydrophobic Surfaces due to a Submicroscopic Bridging Bubble[END_REF][START_REF] Yakubov | Interaction forces between hydrophobic surfaces. Attractive jump as an indication of formation of "stable" submicrocavities[END_REF] and later seen in atomic force microscopy. [START_REF] Lou | Nanobubbles on solid surface imaged by atomic force microscopy[END_REF][START_REF] Ishida | Nano bubbles on a hydrophobic surface in water observed by tapping-mode atomic force microscopy[END_REF][START_REF] Tyrrell | Images of nanobubbles on hydrophobic surfaces and their interactions[END_REF] Meanwhile it has been theoretically understood and supported by numerical simulations that the reason for the on first sight surprising stability of a single surface nanobubbles lies in the balance between gas overpressure thanks to the gas oversaturation in the liquid and the Laplace pressure [START_REF] Lohse | Surface nanobubble and surface nanodroplets[END_REF][START_REF] Liu | A unified mechanism for the stability of surface nanobubbles: Contact line pinning and supersaturation[END_REF][START_REF] Lohse | Pinning and gas oversaturation imply stable single surface nanobubble[END_REF][START_REF] Maheshwari | Stability of Surface Nanobubbles: A Molecular Dynamics Study[END_REF] and that pinning of the contact line [START_REF] Zhang | Stability of Interfacial Nanobubbles[END_REF][START_REF] Liu | Nanobubble stability induced by contact line pinning[END_REF][START_REF] Liu | Contact line pinning and the relationship between nanobubbles and substrates[END_REF][START_REF] Weijs | Why Surface Nanobubbles Live for Hours[END_REF] is a necessary condition to achieve this stability for a single surface nanobubble.

What however happens when there are several surface nanobubbles sitting next to each other on a surface? On one hand the theory of Lohse and Zhang [START_REF] Lohse | Pinning and gas oversaturation imply stable single surface nanobubble[END_REF] predicts that for given gas oversaturation ζ = c ∞ /c s -1 > 0 the equilibrium contact angle θ eq of all these surface bubbles should obey

sin θ eq = ζ L L c , (1)
where L is the footprint diameter of the pinned bubble (assumed to be of spherical cap-shape)

and L c = 4σ/P 0 following from the surface tension σ and the ambient pressure P 0 . Moreover, c s is the saturation concentration and c ∞ is the given gas concentration far away from the bubble.

Eq. ( 1) implies that the equilibrium radius of curvature R eq = L/(2 sin θ eq ) should be the same for all surface nanobubbles, namely

R eq = L c 2ζ eq . (2)
On the other hand one would expect that neighboring surface nanobubbles are subjected to Ostwald ripening, [START_REF] Voorhees | the theory of Ostwald ripening[END_REF] leading to shrinkage of bubbles with small radius of curvature and growth of neighboring ones with large radius of curvature. Indeed, experimental studies do reveal neighboring surface nanobubbles with different radii of curvature. [START_REF] Zhang | Stability of Interfacial Nanobubbles[END_REF][START_REF] Zhang | Physical properties of nanobubbles on hydrophobic surface in water and aqueous solutions[END_REF][START_REF] Yang | Characterization of nanobubbles on hydrophobic surfaces in water[END_REF][START_REF] Zhang | Nanobubbles at the interface between water and a hydrophobic solid[END_REF][START_REF] Yang | Electrolytically generated nanobubbles on highly orientated pyrolytic graphite surfaces[END_REF][START_REF] Zhang | Effects of surfactants on the formation and the stability of interfacial nanobubbles[END_REF][START_REF] Brotchie | Response of interfacial nanobubbles to ultrasound irradiation[END_REF][START_REF] German | Interfacial Nanobubbles Are Leaky: Permeability of the Gas/Water Interface[END_REF] The solution to this apparent paradox has qualitatively been given in the last paragraph of the seminal paper by Zhang and coworker, [START_REF] Zhang | Stability of Interfacial Nanobubbles[END_REF] who not only identified contact line pinning as reason for the stabilization of single surface nanobubbles against dissolution, but also as reason for the stabilization against Ostwald ripening within a population of surface nanobubbles. The aim of this paper is to theoretically and quantitatively work out this idea. We will study the stability of two different neighboring bubbles with distance d against Ostwald ripening, where d is assumed to be much larger than the footprint diameter of the bubbles. The problem is mathematically formulated in the next section. Then the results of the linear stability analysis are presented and discussed. The paper closes with conclusions and an outlook.

Derivation of the evolution equation for two diffusively coupled surface bubbes

We state the problem as follows: consider two pinned surface nanobubbles, labelled 1 and 2, of constant footprint lengths L 1 and L 2 and variable contact angle θ 1 and θ 2 , separated by a distance d (Fig. 1). Let c(x, t) be the concentration field of dissolved gas in the liquid surrounding the nanobubbles. The boundary conditions are: c → c ∞ far from the nanobubbles, ∂c/∂n = 0 at the solid boundary (i.e., no flux through it), where n is the normal direction to the boundary, and Henry's law at S i , the surface of nanobubble i:

1 c| S i = c s 1 + 4σ sin θ i L i P 0 , ( 3
)
where σ is the surface tension, and P 0 the ambient pressure. In the absence of bulk flow, the concentration field obeys the diffusion equation: ∂c/∂t = D∆c, with D the diffusion coefficient of the dissolved gas. We will further assume that the evolution of the nanobubbles is slow enough to be considered as quasistatic for the diffusion problem, so that the concentration field simply obeys the Laplace equation:

∆c = 0. (4)
This quasistatic approximation does not rule out the possibility of Ostwald ripening, [START_REF] Voorhees | the theory of Ostwald ripening[END_REF] provided that the time scale of this process is much larger than the diffusive time scale τ D = L 2 /D, where the footprint L is taken as a relevant characteristic length scale of the nanobubble. Actually, the quasistatic approximation is well suited for nanobubbles (say, L < 1 µm), for which τ D is below 1 ms, using D ≈ 10 -9 m 2 /s as the typical diffusion coefficient of gases in water. In general, this problem must be solved numerically. However, like for other interaction problems involving potential flows, such as the relative translation of two spheres [START_REF] Lamb | [END_REF] or the coupled dynamics of two bubbles in an acoustic field, [START_REF] Doinikov | On the mutual interaction of two gas bubbles in a sound field[END_REF] we can resort to analytical approximations when the nanobubbles are far enough from each other so that L i /d 1. Formally, we introduce the small parameter

ε = L 1 d , (5)
and we assume that L 2 /L 1 is of order 1, so that L 2 /d is of the same order ε. In this case, the potential field (here, the concentration field) can be expanded in a power series in ε, which dominant terms represent single objects without interaction, and with correction terms accounting for the interaction. A way to construct such a series is to consider successive images of a sphere with respect to the other. [START_REF] Lamb | [END_REF] However, we cannot use such an approach in our case, except for the very special case θ 1 = θ 2 = π/2. Actually, this case can be treated exactly using bispherical coordinates.

We will use a rougher approximation, but fully consistent as a leading-order correction.

Since the Laplace equation ( 4) is linear, we can use the principle of superposition of solutions.

We split the concentration field as follows, where the constant term c ∞ is substracted off for convenience:

c = c ∞ + c (0) 1 + c (0) 2 + ε(c (1) 1 + c (1) 2 ), (6)
where each subfield c (j)

i is of order 1, and obeys Laplace equation: ∆c

i = 0, with the following boundary conditions:

c (j) i = 0 at ∞, ∂c (j) i ∂n = 0 at the solid boundary,
for the bubbles i = 1, 2 and the orders j = 0, 1. Hence, the concentration field defined by (6) obeys the boundary conditions at the wall and at infinity. At S i , we specify the following boundary condition:

c (0) i = c s 1 + 4σ sin θ i L i P 0 -c ∞ . (7)
We do not specify any boundary condition for c (0) i at S 3-i . Hence, the problems labelled with

j = 0 neglect interaction: c ∞ + c (0) i
is exactly the concentration of the nanobubble i if it were isolated. As a result, the fields c (0) i take a nonzero value at S 3-i . Therefore, the field

c (0) = c ∞ + c (0) 1 + c (0)
2 does not respect the boundary conditions (3). This discrepancy is of order ε. Hence, the fields c

(1) i are introduced to compensate this discrepancy at order ε, by imposing as boundary condition over S i :

c (1) i = -ε -1 c (0) 3-i | d , (8)
where c 3-i along S i in (8) leads to terms of order ε 2 , which we neglect. Here again, we do not specify boundary conditions for c

(1)

i at S 3-i . Hence,
as before, the field (6) does not respect the boundary condition ( 3), but the discrepancy is now of order ε 2 . Hence, (6) gives the correct concentration field up to order ε included.

To compute the evolution equation of nanobubble i, we need the diffusive mass flux Ṁi at its surface S i . Consistently with (6), we write Ṁi = Ṁ (0

) i + ε Ṁ (1) i , where Ṁ (j) i = S i (-D∇c (j) i • n)| S i dS i ,
with n the outwards unit normal vector on S i .

The fields c (j) i share similar boundary conditions: zero concentration at infinity, a constant concentration c 0 prescribed at the nanobubble surface S i , and zero flux at the solid boundary.

Popov [START_REF] Popov | Evaporative deposition patterns: Spatial dimensions of the deposit[END_REF] gave the general exact solution of such a problem, using toroidal coordinates (Fig. 2).

In particular, we can directly quote the mass flux (A8) calculated by Popov:

30 Ṁi = - π 2 L i Dc 0 f (θ i ), (9)
where D is the diffusion coefficient, and:

f (θ) = sin θ 1 + cos θ + 4 ∞ 0 1 + cosh 2θτ sinh 2πτ tanh[(π -θ)τ ]dτ.
The first application of this result is to the problems (0): there, substituting the expression (7) for c 0 in (9), we recover the result for a single nanobubble: 13

Ṁ (0) i = - π 2 L i D c s 1 + 4σ sin θ i L i P 0 -c ∞ f (θ i ). (10)
To compute c are obtained by rotation about the vertical symmetry axis (dashed-dotted line). The bipolar coordinates are obtained from two points called foci, separated by a distance L. The curves at constant β are the circles passing through the two foci (dashed curves). They interest the axis joining the foci with an angle β. Hence, this axis can be mapped to the wall, and the nanobubble of contact angle θ can be identified with the curve β = θ[2π] (plain curve). The curves at constant α are the circles perpendicular to the family of circles at constant β (dotted curves). The coordinate β is defined on any interval of length 2π, and the coordinate α is defined between 0 and +∞; as α → +∞, the circles at constant α tend towards the foci, and as α → 0 + , their radius tend towards ∞.

zero at large distance from the nanobubbles. We use the exact concentration field (A3) given by Popov:

30 c (0) i (α i , β i ) = c s 1 + 4σ sin θ i L i P 0 -c ∞ 2(cosh α i -cos β i ) × ∞ 0 cosh θ i τ cosh(2π -β i )τ cosh πτ cosh(π -θ)τ P -1/2+iτ (cosh α i )dτ, (11)
where c 0 takes the value given by Eq. ( 7), and where the toroidal coordinates (α i , β i ) are defined with respect to nanobubble i. Despite its complex index -1/2 + iτ , the Legendre function P -1/2+iτ is real valued. [START_REF] Lebedev | Special Functions and their Applications[END_REF] In this solution, β i is assumed to vary between π -θ i , which defines S i , and 3π -θ i . To compute the far-field asymptotic behavior of ( 11), we notice that the far field corresponds to α i → 0, cos β i → 1, hence β i = 2π. From the property of the Legendre function P ν (1) = 1, 31 (11) has the following far-field behavior:

c (0) i ∼ c s 1 + 4σ sin θ i L i P 0 -c ∞ L i r ϕ(θ i ), (12)
where r is the distance from the center of the footprint, and ϕ is defined as:

ϕ(θ) = ∞ 0 cosh θτ cosh πτ cosh(π -θ)τ dτ. (13)
A plot of this auxiliary function (where the integral must be performed numerically) is shown in Fig. 3. It is an increasing function of the contact angle, which diverges as θ → π. By direct integration, we have the two useful exact values: ϕ(0) = 1/π, and ϕ(π/2) = 1/2. The latter value enables us to cross check our calculations with the well-known solution of the Laplace equation around a sphere of radius R: c(r) = c 0 R/r, with r the distance to the center of the sphere. Finally, (12) yields the following value:

c (0) i | d = c s 1 + 4σ sin θ i L i P 0 -c ∞ L i d ϕ(θ i ) = c s 1 + 4σ sin θ i L i P 0 -c ∞ ε L i L 1 ϕ(θ i ),
where c Hence, the boundary condition (8) of the problem at order 1 becomes:

c (1) i = -ε -1 c (0) 3-i | d = - L 3-i L 1 c s 1 + 4σ sin θ 3-i L 3-i P 0 -c ∞ ϕ(θ 3-i ).
Substituting this value for c 0 into (9), we get the first-order correction for the mass flux towards bubble i:

Ṁ (1) i = - π 2 L i Dc (1) i f (θ i ) = π 2 L i D L 3-i L 1 c s 1 + 4σ sin θ 3-i L 3-i P 0 -c ∞ ϕ(θ 3-i ) f (θ i ). (14)
Hence, combining ( 10) and ( 14), the mass flux towards nanobubble i writes:

Ṁi = Ṁ (0) i + ε Ṁ (1) i = - π 2 L i Df (θ i ) c s 1 + 4σ sin θ i L i P 0 -c ∞ -ε L 3-i L 1 c s 1 + 4σ sin θ 3-i L 3-i P 0 -c ∞ ϕ(θ 3-i ) .
Now, from the simple geometry of a spherical cap with fixed footprint diameter L i , we get:

Ṁi = π 8 ρ g L 3 i θi (1 + cos θ i ) 2 .
Hence, using the oversaturation parameter ζ and the length scale L c we get the coupled dy-namical system driving the evolution of the two contact angles:

     θ1 = 4Dcs ρgL 2 1 (1 + cos θ 1 ) 2 f (θ 1 ) ζ -Lc L 1 sin θ 1 -ε L 2 L 1 ζ -Lc L 2 sin θ 2 ϕ(θ 2 ) θ2 = 4Dcs ρgL 2 2 (1 + cos θ 2 ) 2 f (θ 2 ) ζ -Lc L 2 sin θ 2 -ε ζ -Lc L 1 sin θ 1 ϕ(θ 1 ) . (15)
We stress again that this system ( 15) is valid up to order ε included. For large nanobubble distance ε → 0, the two equations decouple and reduce to the evolution equation recently derived by Lohse and Zhang:

13 θ = 4Dc s ρ g L 2 (1 + cos θ) 2 f (θ) ζ - L c L sin θ . (16)
Equilibrium and stability

The system (15) is the central result of our theoretical study, as it allows us to study the equilibrium and the stability of the coupled system of two surface nanobubbles. Their equilibrium is such that θ1 = θ2 = 0, hence:

     0 = ζ -Lc L 1 sin θ 1 -ε L 2 L 1 ζ -Lc L 2 sin θ 2 ϕ(θ 2 ) 0 = ζ -Lc L 2 sin θ 2 -ε ζ -Lc L 1 sin θ 1 ϕ(θ 1 ) (17)
Hence, the equilibrium contact angle θ (0) i of a single nanobubble obeys sin θ

(0) i = ζL i /L c , in
agreement with the result of ref. [START_REF] Lohse | Pinning and gas oversaturation imply stable single surface nanobubble[END_REF] If we set:

θ eq i = θ (0) i + εθ (1) i
+ . . . and insert in (17), it immediately follows that θ

= 0. Hence, the equilibrium contact angle is not modified at order ε.

The most interesting question is whether the presence of a second nanobubble will affect the stability of the first. It was shown by Lohse & Zhang 13 that a pinned nanobubble was stable against dissolution, contrary to a bulk bubble, because its Laplace pressure is an increasing function of its contact angle; mathematically, θ = g(θ) with (dg/dθ)| θ eq < 0. This stability criterion actually holds for θ < π/2, which corresponds to the experimental range of very small contact angles reported in the literature 11 . To study the stability of the two-bubble equilibrium, we linearise the system (15) close to the equilibrium: writing θ i = θ eq i + δ i with |δ i | 1, and δ = (δ 1 , δ 2 ) T , we get δ = (4Dc s /ρ g )Aδ with the matrix A:

A =    -Lc L 3 1
(1 + cos θ eq 1 ) 2 f (θ eq 1 ) cos θ eq 1 ε Lc L 3 1

(1 + cos θ eq 1 ) 2 f (θ eq 1 )ϕ(θ eq 2 ) cos θ eq 2 ε Lc L 3

2

(1 + cos θ eq 2 ) 2 f (θ eq 2 )ϕ(θ eq 1 ) cos θ eq 1

-Lc L 3 2 (1 + cos θ eq 2 ) 2 f (θ eq 2 ) cos θ eq 2    .
The stability of equilibrium depends on the sign of the eigenvalues of this matrix: it is stable if and only if both are negative. Now, the eigenvalues obey the polynomial equation:

0 = det(A -λI) = λ 2 + L c 1 L 3 1
(1 + cos θ eq 1 ) 2 f (θ eq 1 ) cos θ eq 1

+ 1 L 3 2 (1 + cos θ eq 2 ) 2 f (θ eq 2 ) cos θ eq 2 λ + L 2 c L 3 1 L 3 2 1 -ε 2 L 2 L 1 ϕ(θ eq 1 )ϕ(θ eq 2 )
×(1 + cos θ eq 1 ) 2 (1 + cos θ eq 2 ) 2 f (θ eq 1 )f (θ eq 2 ) cos θ eq 1 cos θ eq 2 .

To discuss only the stability and not the growth or decay rates, we only need to discuss the sign of the coefficients of this second-order polynomial. Indeed, if we write it λ 2 + aλ + b, a equals the sum of its two roots, and b their product. The assumption θ eq 1 < π/2 and θ eq 2 < π/2

guarantees that a > 0. Hence, if b > 0 with b = L 2 c L 3 1 L 3 2
(1 + cos θ eq 1 ) [START_REF] Craig | Very small bubbles at surfaces -the nanobubble puzzle[END_REF] (1 + cos θ eq 2 ) 2 f (θ eq 1 )f (θ eq 2 ) cos θ eq 1 cos θ

eq 2 × 1 -ε 2 L 2 L 1 ϕ(θ eq 1 )ϕ(θ eq 2 ) , (19)
both eigenvalues are positive and the two-nanobubble equilibrium is stable. The sign of b is determined by the sign of the term 1 -ϕ(θ eq 1 )ϕ(θ eq 2 )ε 2 L 2 /L 1 in eq. (19). It is positive

except if ϕ(θ eq 1 )ϕ(θ eq 2 )L 1 L 2 /d 2 > 1.
Since ϕ(θ) increases from 1/π to 1/2 when θ increases from 0 to π/2 (Fig. 3), a necessary condition for this inequality to hold is:

L 1 L 2 /d 2 > 4 or, equivalently, ε 2 L 2 /L 1 >

Conclusions and outlook

In summary, we have theoretically shown that in the framework of our analytical linear approximation, the equilibrium of a nanobubble and its stability are not affected by the presence of a second, neighboring nanobubble. This may on first sight appear puzzling when thinking in terms of Ostwald ripening, where a bigger bubble grows at the expense of smaller neighbours and which clearly holds for microbubbles in the bulk. However, once more, as in the case of a single surface nanobubbles, [START_REF] Lohse | Pinning and gas oversaturation imply stable single surface nanobubble[END_REF] this bulk-bubble based intuition fails because of the pinning condition for surface nanobubbles.

The next steps to be taken are obvious: On the numerical side, just as we numerically confirmed the stability theory of ref. [START_REF] Lohse | Pinning and gas oversaturation imply stable single surface nanobubble[END_REF] by molecular dynamics (MD) simulations 14 for a single surface nanobubbles, we will numerically confirm the present theory for the stability of a pair of diffusively interacting (pinned) surface nanobubbles. The MD simulations will also allow us to go beyond the linear approximation and explore higher order Ostwald ripening effects. On the theoretical side, the present calculation for two diffusively interacting surface nanobubbles can straightforwardly be extended to three and more surface nanobubbles. On the experimental side, the community is in urgent need of controlled and quantitative experiments for diffusively interacting surface nanobubbles. In particular, the gas oversaturation ζ should be known in these experiments, which clearly poses a challenge and may require highly controlled devices as that described in refs. [START_REF] Enriquez | Growing bubbles in a slightly oversaturated liquid solution[END_REF]

27

 27

Figure 1 :

 1 Figure 1: Sketch of the two nanobubbles.

  | d is the concentration field induced by the presence of nanobubble 3 -i evaluated at the location of the other nanobubble i. At order ε, it suffices to evaluate this term at a constant distance d; accounting for the spatial variations of c (0)

i

  | d , we need the asymptotic expression of the concentration field at order L φ α = const. β = const.

Figure 2 :

 2 Figure2: Sketch of the bipolar coordinates (α, β) from which the toroidal coordinates (α, β, φ) are obtained by rotation about the vertical symmetry axis (dashed-dotted line). The bipolar coordinates are obtained from two points called foci, separated by a distance L. The curves at constant β are the circles passing through the two foci (dashed curves). They interest the axis joining the foci with an angle β. Hence, this axis can be mapped to the wall, and the nanobubble of contact angle θ can be identified with the curve β = θ[2π] (plain curve). The curves at constant α are the circles perpendicular to the family of circles at constant β (dotted curves). The coordinate β is defined on any interval of length 2π, and the coordinate α is defined between 0 and +∞; as α → +∞, the circles at constant α tend towards the foci, and as α → 0 + , their radius tend towards ∞.

iFigure 3 :

 3 Figure 3: Plot of ϕ(θ) defined by (13).

For the case θ > π/2 stability and instability of course would be reversed.

Acknowledgments

We gratefully acknowledge numerous illuminating discussions with Xuehua Zhang over the years. We also acknowledge helpful discussions with Vince Craig and Ing-Shouh Hwang and financial support from the NWO-MCEC programme and an ERC Advanced Grant.


Petr Hájek
email: petr.hajek@upce.cz

Vladimír Olej
email: vladimir.olej@upce.cz

Predicting Firms' Credit Ratings using Ensembles of Artificial Immune Systems and Machine Learning -An Over-Sampling Approach

Keywords: Credit rating, artificial immune systems, machine learning, neural networks, classification performance, balanced and imbalanced dataset, SMOTE, AdaBoost. 1

This paper examines the classification performance of artificial immune systems on the one hand and machine learning and neural networks on the other hand on the problem of forecasting credit ratings of firms. The problem is realized as a two-class problem, for investment and non-investment rating grades. The dataset is usually imbalanced in credit rating predictions. We address the issue by over-sampling the minority class in the training dataset. The experimental results show that this approach leads to significantly higher classification accuracy. Additionally, the use of the ensembles of classifiers makes the prediction even more accurate.

Introduction

Credit rating addresses an issuer's overall capacity and willingness to meet its financial obligations, thus reducing the information asymmetry between issuers and investors. However, credit ratings are based on costly analysis performed by professionals, which is why credit rating forecasting has attracted considerable recent interest. Various artificial intelligence (AI) methods have been applied to model the complex nonlinear relations between input variables and target classes (see [START_REF] Hajek | Credit Rating Modelling by Kernel-Based Approaches with Supervised and Semi-Supervised Learning[END_REF] for an example of a review). Recent efforts have shown that approaches integrating feature selection process and an appropriate AI method provide the best classification performance [START_REF] Hajek | Municipal Credit Rating Modelling by Neural Networks[END_REF]. However, although the ensembles of classifiers have shown promising results in related fields such as credit risk and bankruptcy forecasting [START_REF] Kumar | Bankruptcy Prediction in Banks and Firms via Statistical and Intelligent Techniques -A Review[END_REF], significantly insufficient attention has been paid to them in credit rating forecasting. In addition, no research has been found that examines the effect of over-sampling the minority class in credit rating data. Therefore, the aim of this paper is to examine the effect of: (1) oversampling the minority class in credit rating data; and (2) ensembling base classifiers. The difficulty in predicting credit ratings also stems from the fact that a multitude of sources is used in the credit rating analysis, involving both quantitative data (accounting and financial data drawn from financial statements) and qualitative assessments. The sentiment analysis of corporate annual reports has recently been used to address the issue of qualitative assessment [START_REF] Hajek | Evaluating Sentiment in Annual Reports for Financial Distress Prediction using Neural Networks and Support Vector Machines[END_REF]. We follow this approach and use the chosen sentiment categories in addition to corporate financial indicators as input variables.

We employ two categories of AI methods to examine the given aims, (1) artificial immune systems (AISs) and (2) machine learning (ML). AISs mimic the processes and mechanisms of biological immune systems and we specifically use Artificial Immune Recognition Systems (AIRSs) [START_REF] Watkins | Artificial Immune Recognition System (AIRS): An Immune Inspired Supervised Learning Algorithm[END_REF] and the Clonal Selection Classification Algorithm (CSCA) [START_REF] Brownlee | Clonal Selection Theory Algorithm and CLONALG: The Clonal Selection Classification Algorithm (CSCA). Victoria Australia, Centre for Intelligent Systems and Complex Processes[END_REF]. Out of the second category, we use C4.5 decision trees (DTs) [START_REF] Quinlan | C4. 5: Programs for Machine Learning[END_REF], Support Vector Machines (SVMs) [START_REF] Platt | Fast Training of Support Vector Machines using Sequential Minimal Optimization[END_REF], Radial Basis Function Neural Networks (RBFs) [START_REF] Haykin | Neural Networks: A Comprehensive Foundation[END_REF] and Multilayer Perceptrons (MLPs) [START_REF] Haykin | Neural Networks: A Comprehensive Foundation[END_REF]. The research to date has tended to focus on the ensembles of classifiers of the latter category while little attention has been paid to the ensembles of AISs [START_REF] García-Pedrajas | Construction of Classifier Ensembles by means of Artificial Immune Systems[END_REF].

This paper is organized as follows. In the next section, we provide the description of the dataset. Given the fact that the classes are imbalanced in the dataset, we use the Synthetic Minority Oversampling Technique (SMOTE) algorithm [START_REF] Chawla | SMOTE: Synthetic Minority Over-sampling Technique[END_REF] to modify the training dataset. The third section presents the results of experiments. First, the effect of over-sampling is tested and then we employ the AdaBoost algorithm [START_REF] Freund | Experiments with a New Boosting Algorithm[END_REF] to generate the ensembles of the base classifiers (AIRS1, AIRS2, AIRS2-p, CSCA; and DT, SVM, RBF, MLP). For both categories, the classification performance is examined on (imbalanced) testing dataset depending on the proportion of training dataset generated. The last section discusses the results and concludes the paper.

Problem Formulation and Dataset

The prediction of firms' credit ratings was realized as a two-class problem. The classes were represented by investment grade (IG, low default risk) and noninvestment grade (NG, high default risk), assigned by a highly regarded Standard & Poor's rating agency in 2011. The investment grade position is critical to many investors due to the restrictions imposed on investment instruments. We used two main groups of input variables in this study, financial indicators and sentiment indicators (see Table 1). More specifically, we used several subgroups of financial indicators such as size, profitability ratios, liquidity ratios, leverage ratios and market value ratios. Sentiment indicators refer to the business position of a company (business risk, character (reputation), organizational problems, management evaluation, accounting quality, etc.). The financial indicators were drawn from the Value Line database, while sentiment indicators were drawn from annual reports available at the U.S. Securities and Exchange Commission EDGAR System. Both groups of input variables were collected for 520 U.S. companies (selected from the Standard & Poor's database) in the year 2010, 195 classified as IG and 325 as NG. Mining and financial companies were excluded from the dataset since they require specific input variables.

The sentiment indicators required linguistic pre-processing (tokenization and lemmatization) and subsequent comparison with the financial dictionary provided by [START_REF] Loughran | When Is a Liability Not a Liability? Textual Analysis, Dictionaries, and 10-Ks[END_REF]. Then, the tf.idf term weighting scheme was applied to obtain the importance of terms and an average weight was calculated for each sentiment category (negative, positive, uncertainty, litigious, modal strong and modal weak). See [START_REF] Hajek | Evaluating Sentiment in Annual Reports for Financial Distress Prediction using Neural Networks and Support Vector Machines[END_REF] for the detailed information on the collection of data. High to low stock price class {IG, NG}

The given dataset was randomly divided into training and testing dataset (2:1). This procedure was repeated five times. Thus, the training dataset O train contained 347 companies, 217 classified as IG and 130 as NG. The testing dataset O test covered 173 companies, 108 as IG and 65 as NG. Both datasets were imbalanced, with the less frequent NG category (minority class). There are several approaches to handle this issue. The under-sampling of the majority class may represent a good way to reduce the sensitivity of classifiers, but in our case this approach resulted in a decrease in classification performance. This may be related to both the small size of the dataset and important decision information stored in most of the objects in the majority class. Another way is to apply the over-sampling of the minority class so that all classes are represented equally in the training dataset. We used the SMOTE procedure [START_REF] Chawla | SMOTE: Synthetic Minority Over-sampling Technique[END_REF]

Modelling Financial and Sentiment Indicators

In this section we employed commonly used artificial immune classification algorithms, AIRS1, AIRS2, AIRS2-p and CSCA. Further, the AISs were compared with DTs, SVMs, RBFs and MLPs. We used 10-fold cross-validation on training data to find the optimum settings of the classifiers' parameters. The measures of classification performance are represented by the averages of standard statistics applied in classification tasks [START_REF] Powers | Evaluation: from Precision, Recall and F-measure to ROC, Informedness, Markedness and Correlation[END_REF]: true positives (TP rate), false positives (FP rate), precision (Pre) and recall (Re), F-measure (F-m), the area under the receiver operating characteristic (ROC) curve and misclassification cost (MC). Fm is the weighted harmonic mean of Pre and Re, or the Matthews correlation coefficient, which is a geometric mean of the chance-corrected variants. A ROC is a graphical plot which illustrates the performance of a binary classifier system, which represents a standard technique for summarization classifier performance over a range of tradeoffs between TP and FP error rates. In particular, FP rate is important in this study owing to its possible serious financial consequences. This is due to the significant difference between default rates of IG and NG firms. For example, Standard & Poor's reported 0.03 % for IG and 1.71 % for NG in 2011. Therefore, we designed an MC matrix, where MC = 0.03 was assigned to each false classified IG firm and MC = 1.71 to each false classified NG firm, respectively.

Artificial Immune Classification Algorithms

Methods

The group of AIRS algorithms represents AISs using populations [START_REF] Dasgupta | Artificial Immune Systems and their Applications[END_REF][START_REF] De Castro | Artificial Immune Systems: A New Computational Intelligence Approach[END_REF]. The algorithms are based on the principle of Recognition Ball (RB) or Artificial Recognition Ball (ARB), which can be described as recognition areas or artificial recognition areas that combine feature vector (antibody) and vector class. The principle solves the issue of the completeness of AISs. Each antibody is surrounded (in the sense of antibody representation in the state space) by a small area called the RB, in which the antibody recognizes all antigens (training dataset). Further, the AIRS algorithms use the principle of a limited resource. Each ARB area competes for limited resource according to its stimulation level. The classification performance of the AIRS algorithms proposed by [START_REF] Watkins | Artificial Immune Recognition System (AIRS): An Immune Inspired Supervised Learning Algorithm[END_REF] depends on several user-defined parameters. We used the AIRS1, AIRS2 and AIRS2-p algorithms to predict the firms' credit ratings. The following parameters of the AIRSs were examined to obtain the best classification performance: affinity threshold scalar = {0.1,0.2, … ,0.9}, clonal rate = {1,2,4, … ,32}, hypermutation rate = {1,2, … ,10}, number of k nearest neighbors = {1,2, … ,10}, initial memory cell pool size = 50, number of instances to compute the affinity threshold = all, stimulation threshold = 0.9 and total resources = 150.

The CSCA [START_REF] Brownlee | Clonal Selection Theory Algorithm and CLONALG: The Clonal Selection Classification Algorithm (CSCA). Victoria Australia, Centre for Intelligent Systems and Complex Processes[END_REF] uses a fitness function to maximize classification accuracy (minimize misclassification accuracy). The performance of the CSCA depends on the following user-defined parameters: clonal scale factor = 1.0, initial population size = {10,20, … ,100}, number of nearest neighbors = {1,2, … ,10}, minimum fitness threshold = 1, number of partitions = 1 and total generations = {1,5, 10, ... ,100}.

Results

The best classification results for the AIRSs and CSCA simulations on testing dataset are shown in Table 2.

Here, we present the average performance measures over the five datasets. The measures are represented by the averages of standard statistics applied in classification tasks [START_REF] Powers | Evaluation: from Precision, Recall and F-measure to ROC, Informedness, Markedness and Correlation[END_REF]. The results in Table 2 show that the AIRS2 performed best while the CSCA performed significantly worse (using paired t-test on P < 0.01). Further, the sizes of training datasets suggest that the AIRSs require larger datasets to achieve good classification performance (see Fig. 1).

Table 2.

Best results for AIRS1, AIRS2, AIRS2-p and CSCA on testing dataset. cation performance of the DT depends on the following parameters: the confidence factor used for pruning (= 0.25 in this study) and the minimum number of instances per leaf = 2. SVMs represent an essential kernel-based method with many modifications proposed recently. SVMs use kernel functions to separate the hyperplane between two classes by maximizing the margin between the closest data points. This is done in a higher-dimensional space where the data become linearly separable. We used the SVMs trained by the sequential minimal optimization (SMO) algorithm [START_REF] Platt | Fast Training of Support Vector Machines using Sequential Minimal Optimization[END_REF]. The classification performance of the SVM was tested for the following user-defined parameters: kernel functions = {polynomial, RBF},  = 0.01, the level of polynomial function = 2, complexity parameter C = {1,2,4, … ,256}, round-off error  = 1.0E-12 and tolerance parameter = 0.001. The RBF was trained with the Broyden-Fletcher-Goldfarb-Shanno method. The initial centres for the Gaussian RBFs were found using a k-means algorithm. The initial sigma values were set to the maximum distance between any centre and its nearest neighbor in the set of centres. The classification performance of the RBF depends on the following user-defined parameters: maximum number iteration = not limited, minimum standard deviation for the clusters = {0.1,0.2,0.3}, the number of clusters for k-means = {1,2,4, … ,64} and ridge factor = 1.0E-8.

The MLP was trained using the backpropagation algorithm with momentum. The following parameters of the MLP were examined to achieve the best classification performance: the number of neurons in the hidden layer = {5,10,15, … ,30}, learning rate = {0.01,0.05,0.1}, momentum = 0.2 and the number of epochs = {100,500,1000}.

Results

The best classification results for the DT, SVM, RBF and MLP simulations on testing dataset are shown in Table 3. Here, the best classification performance was achieved by MLP (Accuracy) and DT (MC). Compared with the AISs, the MLP provided significantly better results in terms of both classification accuracy and ROCs (again, tested using paired t-test at P < 0.01). Furthermore, less O train were required to be generated in order to learn the MLP compared with the AIRS2 (see Fig. 2). On the other hand, DT provided the lowest MC of all classifiers.

Comparison Across Classifiers

In Fig. 3 We further examined the effect of the ensembles of classifiers. We employed the AdaBoost method [START_REF] Freund | Experiments with a New Boosting Algorithm[END_REF] that combines many 'weak' classifiers to obtain an accurate learning algorithm. More precisely, AdaBoost is a meta-algorithm that can be used in conjunction with many other learning algorithms to improve their performance. This meta-algorithm is adaptive in the sense that subsequent classifiers built are tweaked in favor of those instances misclassified by previous classifiers. The number of iterations to be performed was set to 10. Again, we tested the previously described classification algorithms as base learners.

The results are depicted in Fig. 4. Here, the balanced training dataset was used from prior experiments. The ROC was significantly higher (at P < 0.05) for all the classifiers except for the MLP. AdaBoost provided noteworthy improvements for the DTs, AISs and SVMs especially. The results of the classification of the O test for the original imbalanced and balanced O train are shown in Table 5. DT with balanced data provided significantly lower MC (at P < 0.05) for both individual (Table 4) and ensemble classifiers (Table 5). For details, see confusion matrix in Table 6.

Conclusion

The aim of this paper was to examine the performance of AISs and ML on a complex classification task where financial indicators are combined with sentiment analysis. We attempted to address an important issue of imbalanced dataset. On the one hand, the over-sampling of the minority class showed promising classification improvement, yet on the other hand, the under-sampling of the majority class resulted in weaker classification performance. We performed many experiments to find the best setting of the learning parameters of individual classifiers to achieve the best classification performance. The results also confirmed that an ensemble of weaker base classifiers may perform better than the individual classifiers, especially in terms of ROC measure. More importantly, the decrease in MC may lead to the substantial financial savings of investors. For example, the ensemble of DTs trained on balanced data reduced the MC from 21.29 to 17.28 (18.8% savings). Thus, this study represents a good basis for further experiments in the field of financial distress forecasting, where the problem of imbalanced datasets is usually to be addressed. Additionally, we encourage future research in multi-class datasets. The experiments in this study were carried out in Statistica 10 (linguistic preprocessing) and Weka 3.7.5 (Artificial Immune Classification Algorithms and ML) in MS Windows 7 operating system.

Fig. 1 .

 1 Fig. 1. Classification accuracy of AIRS1, AIRS2, AIRS2-p and CSCA on O test depending on O train size (Source: own)

  we compare the ROC achieved for the balanced and imbalanced datasets. Except for the AIRS2-p and RBF, the performance of the classifiers improved significantly (at P < 0.05) when the O train were balanced using the SMOTE (O train 100 ). Depending upon the amount of over-sampling required, neighbors from k nearest neighbors are randomly chosen. In our experiments we used k = 5 nearest neighbors. The results of the classification of O test for the original imbalanced and balanced O train are shown in Table4.

Fig. 2 .Fig. 3 .

 23 Fig. 2. Classification accuracy of DT, SVM, RBF and MLP on O test depending on O train size (Source: own)

Fig. 4 .Table 6 .

 46 Fig. 4. ROC for base classifiers and AdaBoost (Source: own)Table 6. Confusion matrix of the O test for the balanced O train trained using DT with AdaBoost.

Table 1 .

 1 Input and output variables describing the dataset.

		Variable		Variable
	x 1	Enterprise value	x 11	Dividend yield
	x 2	Cash	x 12	Payout ratio
	x 3	Revenues	x 13	Standard deviation of stock price
	x 4	Earnings per share	x 14	Frequency of negative terms
	x 5	Return on equity	x 15	Frequency of positive terms
	x 6	Price to book value	x 16	Frequency of uncertainty terms
	x 7	Enterprise value/earnings	x 17	Frequency of litigious terms
	x 8	Price to earnings per share	x 18	Frequency of strong modal terms
	x 9	Market debt / total capital	x 19	Frequency of weak modal terms
	x 10

  to generate additional objects (firms) for the NG class to make the training dataset balanced. Thus, O train contained 433 (O train 100 ) companies, 217 classified as IG and 216 as NG. These training datasets O train 100 were considered to be the base balanced training datasets with 100 % of companies. We further examined the effect of generating additional training datasets, increasing the number by 25 % up to 300 % (O train

					125 , O train	150 ,
	... ,O train	300 ). The testing set remained imbalanced and fixed for all training sets (O t-
	rain 100 , O train	125 , O train	150 , ... ,O train	300 ).

Table 3 .

 3 Best results for DT, SVM, RBF and MLP algorithms on testing dataset.

		DT		SVM		RBF		MLP
	O train	O train 125		O train 150		O train 150		O train 125
	Accuracy [%]	86.13		87.28		84.39		88.44
	MC	17.59		30.96		36.01		24.19
	Class	IG	NG	IG	NG	IG	NG	IG	NG
	TP rate	0.785 0.907	0.938	0.833	0.908	0.806	0.908	0.870
	FP rate	0.093 0.215	0.167	0.062	0.194	0.092	0.130	0.092
	Precision	0.836 0.875	0.772	0.957	0.738	0.935	0.808	0.940
	Recall	0.785 0.907	0.938	0.833	0.908	0.806	0.908	0.870
	F-m	0.810 0.891	0.847	0.891	0.814	0.866	0.855	0.904
	ROC	0.871 0.871	0.886	0.886	0.884	0.881	0.947	0.947

Table 4 .

 4 Classification performance on O test for the original imbalanced (i) and balanced (b) O train (weighted average for IG and NG).

Table 5 .

 5 Classification performance of the O test for the original imbalanced (i) and balanced (b) O train (weighted average for IG and NG) with AdaBoost.

			AIRS1	AIRS2	AIRS2-p CSCA	DT	SVM	RBF	MLP
		i	0.821	0.814	0.834	0.814	0.861 0.851 0.799 0.845
	TP	b	0.853	0.801	0.798	0.801	0.875 0.853 0.786 0.834
		i	0.143	0.201	0.174	0.241	0.138 0.163 0.202 0.161
	FP	b	0.148	0.160	0.155	0.160	0.133 0.148 0.196 0.148
		i	0.822	0.814	0.834	0.810	0.862 0.851 0.800 0.846
	F-m	b	0.854	0.804	0.801	0.804	0.876 0.854 0.788 0.835
		i	42.42	45.99	50.80	21.56	21.29 27.99 44.80 36.14
	MC	b	26.96	30.54	29.10	19.80	17.89 24.00 37.33 27.33
			AIRS1	AIRS2 AIRS2-p CSCA	DT	SVM	RBF	MLP
		i	0.839	0.845	0.850	0.819	0.848	0.846	0.828	0.852
	TP	b	0.845	0.836	0.858	0.809	0.872	0.831	0.798	0.836
		i	0.171	0.166	0.144	0.233	0.155	0.141	0.180	0.150
	FP	b	0.141	0.147	0.138	0.191	0.134	0.140	0.179	0.138
		i	0.840	0.857	0.852	0.816	0.849	0.848	0.829	0.853
	F-m	b	0.846	0.837	0.860	0.803	0.872	0.833	0.800	0.838
		i	32.70	35.14	26.97	38.99	22.51	39.52	44.09	37.83
	MC	b	27.03	25.29	28.97	19.04	17.28	32.04	35.26	26.93

Acknowledgments. This work was supported by the scientific research project of the Czech Sciences Foundation Grant No: 13-10331S.


Jin Zhang
email: jin.zhang@unibas.ch

Automating transition functions: a way to improve trading profits with recurrent reinforcement learning

Keywords: Daily equity trading, Recurrent reinforcement learning, Transition variable selection, Automated transition functions

This paper presents an application of the logistic smooth transition function and recurrent reinforcement learning for designing financial trading systems. We propose a trading system which is an upgraded version of the regime-switching recurrent reinforcement learning (RS-RRL) trading system referred to in the literature. In our proposed system (RS-RRL 2.0), we use an automated transition function to model the regime switches in equity returns. Unlike the original RS-RRL trading system, the dynamic of the transition function in our trading system is driven by utility maximization, which is in line with the trading purpose. Volume, relative strength index, price-to-earnings ratio, moving average prices from technical analysis, and the conditional volatility from a GARCH model are considered as possible options for the transition variable in RS-RRL type trading systems. The significance of Sharpe ratios, the choice of transition variables, and the stability of the trading system are examined by using the daily data of 20 Swiss SPI stocks for the period April 2009 to September 2013. The results from our experiment show that our proposed trading system outperforms the original RS-RRL and RRL trading systems suggested in the literature in terms of better Sharpe ratios recorded in three consecutive out-of-sample periods.

Introduction

Recent developments in algorithmic trading have shown that people never stop their effort of searching and developing trading strategies. In the computer science literature, artificial intelligence (AI) techniques have been increasingly applied in the field of technical analysis for equity trading, e.g. the application of a genetic algorithm (GA) in technical trading rules optimization [START_REF] Allen | Using genetic algorithms to find technical trading rules[END_REF], the adaptive reinforcement learning system which uses 14 commonly-used technical indicators as a part of the system inputs [START_REF] Dempster | An automated FX trading system using adaptive reinforcement learning[END_REF], and a Logitboost method to select combinations of technical trading rules for stocks and futures trading [START_REF] Creamer | Model calibration and automated trading agent for euro futures[END_REF].

Although AI has been used widely in research on financial trading platform design, these studies have concentrated on the optimization of trading signals in technical analysis, in isolation from financial fundamentals and other available tools. Various approaches based on technical analysis [START_REF] Murphy | Technical analysis of the financial markets : a comprehensive guide to trading methods and applications[END_REF] and fundamental analysis [START_REF] Graham | The Intelligent Investor: The Definitive Book on Value Investing[END_REF] have been developed to forecast future trends in stock prices. Additionally, tools from financial engineering have been used to facilitate price movement analysis. In real world, financial news, quotes, company earnings, pre-market and after-hours data, market research, company analysis are commonly used for price movement analysis. For example, researchers suggest to use trading systems which use fundamental indicators to select equity and technical indicators to produce trading signals [START_REF] Silva | Portfolio optimization using fundamental indicators based on multi-objective ea[END_REF].

Recurrent reinforcement learning (RRL), an online learning technique which finds approximate solutions to stochastic dynamic programming problems, was used by researchers to tune financial trading systems for the purpose of utility maximization [START_REF] Moody | Performance functions and reinforcement learning for trading systems and portfolios[END_REF]. In the literature, most discussions of RRL trading have been in the context of high-frequency FX and equity indices trading (see [START_REF] Gold | FX trading via recurrent reinforcement learning[END_REF][START_REF] Dempster | An automated FX trading system using adaptive reinforcement learning[END_REF][START_REF] Bertoluzzo | Making financial trading by recurrent reinforcement learning[END_REF][START_REF] Gorse | Application of stochastic recurrent reinforcement learning to index trading[END_REF]). Due to the effectiveness of RRL in training financial trading systems, the technique has been adopted by researchers to build more sophisticated financial trading systems. For example, the multi-layer trading system integrated a risk management module with a RRL trading system for utility maximization [START_REF] Dempster | An automated FX trading system using adaptive reinforcement learning[END_REF].

Over the last few decades, models such as regime switching (RS) have become popular in econometrics. For example, the threshold autoregressive (TAR) model [START_REF] Tong | On a threshold model[END_REF] and the smooth transition autoregressive (STAR) model allowing for smooth transition [START_REF] Chan | On estimating thresholds in autoregressive models[END_REF] draw people's attention. Although interest in RS models has grown, most papers on RS have focused on model development, with only a few applications of RS models concerned with artificial intelligence being found in the financial trading field.

In the literature, it has been found that transitions between economic regimes are often signaled by price patterns. Researchers proposed a regime-switching RRL trading system to cope with the price changes which is non-linear due to regime switches [START_REF] Maringer | Regime-switching recurrent reinforcement learning for investment decision making[END_REF][START_REF] Maringer | Regime-switching recurrent reinforcement learning in automated trading[END_REF]. The basic RS-RRL trading system consists of two independent RRL systems which produce two trading signals for two scenarios, i.e. for low and high volatility regimes. The trading signal is made on the basis of the weighted total value of the trading signals from the two RRL systems. The weight is an output from a logistic smooth transition autoregressive (LSTAR) function, in which the conditional variance from a GARCH model is used as the transition variable.

In this paper, we aim to add to the existing literature on RS-RRL. We do so by studying the transition variable selection problem in the RS-RRL trading system. Five indicator variables (i.e. volume, relative strength index, price to earnings ratio, conditional volatility and moving average prices) are considered as possible transition variables in the RS-RRL trading system. We also introduce an upgraded version of the original RS-RRL trading system for the purpose of enhancing trading profits. Our upgraded version, the RS-RRL 2.0 trading system uses an automated transition function which is different from the constant transition function in the original RS-RRL trading system.

The paper is organized as follows. Section 2 provides a brief overview of RRLtype trading systems and introduces the our proposed trading system: RS-RRL 2.0. Section 3 provides the results of the experiment. Section 4 concludes the paper.

RRL-type trading systems

Recurrent reinforcement learning

Recurrent reinforcement learning (RRL) was a technique to tune financial trading systems for the purpose of utility maximization [START_REF] Moody | Performance functions and reinforcement learning for trading systems and portfolios[END_REF]. The RRL technique is a stochastic gradient ascent algorithm which continuously optimizes a utility measure by using new market information. Although, in most discussions on RRL, the market information usually comprises a series of lagged price returns. The basic RRL trading system in [START_REF] Moody | Performance functions and reinforcement learning for trading systems and portfolios[END_REF] is designed to trade a single-asset with a two-position action (long/short), which is produced by using linear combinations of returns and a tanh function (see Figure 1). w denotes a series parameters of the input signals. F t-1 refers to the current holding position; and F t denotes the asset's holding position on the following day. The goal of the RRL trading system is to maximize a wealth measure U t by adjusting the parameter set w t in a continuous manner:

max U t (R t ; w t ), (1)
R t = ν • (sgn(F t-1 ) • r t -δ • | sgn(F t ) -sgn(F t-1 )|), (2)
and

F t = tanh (w t × I t ) , (3)
where ν is the number of shares and δ is the transactions cost rate. I t denotes a series of inputs, including F t-1 , a constant v with a value of 1, and a set of lagged returns r t , r t-1 , r t-2 , . . . , r t-l+1 , t = 1, . . . , T , and l is an integer number representing the length of the lags.

The signal parameters w t are updated by RRL based on stochastic gradients. The gradients of U t with respect to the signal parameter set w t can be written as:

dU t (w t ) dw t = dU t dR t dR t dF t dF t dw t + dR t dF t-1 dF t-1 dw t-1 , (4)
dF t dw t ≈ ∂F t ∂w t + ∂F t ∂F t-1 dF t-1 dw t-1 , (5)
dR t dF t-1 = ν • (r t + δ • sgn(F t -F t-1 )), ( 6
)
dR t dF t = -ν • δ • sgn(F t -F t-1 ). ( 7
)
In the literature, the utility measure used in RRL trading systems is the differential Sharpe ratio (DSR), which is the first-order term after taking the Taylor series expansion of a performance measure, namely the exponential moving average Sharpe ratio (EMSR) at η → 0:

EM SR t = A t K η • (B t -A 2 t ) 1/2 , (8)
where

A t = A t-1 + η • (R t -A t-1 ), B t = B t-1 + η • (R 2 t -B t-1
), and

K η = ( 1-η/2 1-η ) 1/2 .
In other words, U t can be written as:

DSR t = B t-1 • (R t -A t-1 ) -1 2 • A t-1 • (R 2 t -B t-1 ) (B t-1 -A 2 t-1 ) 3 2 , (9)
and the derivative of U t with respect to R t in Eq. ( 4) can be written as:

dU t dR t = B t-1 -A t-1 • R t (B t-1 -A 2 t-1 ) 3/2 . ( 10
)
At time t, the signal parameter set w t is updated by using

w t = w t-1 + ρ • dUt(wt)
dwt , where ρ is the learning step.

Regime-switching RRL

Although the basic RRL trading system uses linear combinations to generate trading signals, nonlinear behavior in financial time series has been documented in the literature. To model the nonlinearities in price changes caused by regime switches, researchers proposed a RS-RRL trading system [START_REF] Maringer | Regime-switching recurrent reinforcement learning for investment decision making[END_REF], in which the trading signal F t is defined as:

F t = F A t • (1 -G t ) + F B t • G t , (11)
F A t = tanh w A t-1 × I t-1 , (12)
F B t = tanh w B t-1 × I t-1 . (13)
In other words, the final output F t of the system is a weighted sum of F A t and F B t from two individual RRL systems. The weight is a value generated by using a logistic function:

G t (s t ; γ, c) = 1 1 + exp(-γ • (s t -c)) , (14)
where s t , γ and c refer to the transition variable, the transition rate and the threshold value respectively. In the econometric literature, G t refers to the transition function in a LSTAR model. The LSTAR models a univariate return series r t as:

r t = (φ 0,1 + φ 1,1 r t-1 + . . . + φ l,1 r t-l ) • (1 -G(s t ; γ, c)) + (φ 0,2 + φ 1,2 r t-1 + . . . + φ l,2 r t-l ) • G(s t ; γ, c) + ε t . ( 15
)
It is found that the conditional volatility variable from a GARCH model is a suitable transition variable for RS-RRL trading; and the transition rate and the threshold value in Eq. ( 14) are derived by using a quasi-maximum likelihood estimation (for further details, see [START_REF] Maringer | Transition variable selection for regime switching recurrent reinforcement learning[END_REF]).

Regime-switching RRL 2.0

There are two main concerns with the original RS-RRL trading system. The first is that, if users do not manually update the parameters in G t , the transition function will be constant. Constant transition models may not work well in the real world as the transition rate and the threshold value can be time-varying. The second concern is that, the likelihood based inference for the transition function does not serve the purpose of RRL in the trading context. In other words, the derivation of LSTAR parameters with quasi-maximum likelihood estimation (i.e. minimizing error terms), does not fit well with the purpose of trading, which is utility maximization. We therefore propose a new RS-RRL trading system which we call RS-RRL 2.0 (see Figure 2), in order to improve the trading performance of the RS-RRL trading system. In our proposed trading system, in addition to the two RRL trading systems, we use a control unit which consists of a summation function and a sigmoid function to mimic the transition function G t in the LSTAR model. Inputs of the control unit include the transition variable S t , the threshold value c, and the current holding position F t-1 . To maximize the utility function, RRL updates the signal parameters in the trading units and the control unit. The design of RS-RRL 2.0 allows the transition function G t and the utility measure to interact in an automated fashion in real time.

In the proposed RS-RRL 2.0 trading system, F t is the weighted sum of the outputs F A t and F B t from the two individual RRL trading systems, which are given by Eq. ( 11), Eq. ( 12) and Eq. ( 13). G t is a function of the transition variable, the threshold value, and the signal parameters w G :

G t = sigmoid(w G t-1 × I t-1 ) = 1 1 + exp(-w G t-1 × I t-1 ) , ( 16
)
F t tanh Previous Action F t-1 - Transition Variable Series w s G w c G w F A Input Series x t Input Series x t-L w x A sigmoid Threshold Value C F B t tanh v w x B w v A w F G w v B w F B . . . Previous Action F t-1 G t F A t F B t w' B ⋅ x . . . Input Series x t-L Input Series x t v Previous Action F t-1 F A t *G t + F B t *(1-G t ) w' G ⋅ x G w' A ⋅ x
Fig. 2. The regime-switching recurrent reinforcement learning 2.0 which is similar to the logistic transition function in LSTAR models. We consider only one transition function in this paper; other transition functions from the econometric literature, such as exponential STAR (ESTAR) functions, also can be modified and used in the system. Gradients of U t with respect to the signal parameter sets w A t , w B t and w G t can be written as:

dU t (w A t ) dw A t = dU t dR t dR t dF t dF t dw A t + dR t dF t-1 dF t-1 dw A t-1 , (17)
dF t-1 dw G t-1 . (19)
We use the reward measure and the utility function given in Eq. ( 2) and Eq. ( 9), so that the derivatives dUt dRt , dRt dFt and dRt dFt-1 are the same as Eq. ( 10), Eq. ( 6) and Eq. [START_REF] Gorse | Application of stochastic recurrent reinforcement learning to index trading[END_REF]. The other components in Eq. (17), Eq. (18) and Eq. ( 19) can be found in the following:

dF t dw A t ≈ ∂F A t ∂w A t + ∂F t ∂F t-1 dF t-1 dw A t-1 , ( 20
)
dF t dw B t ≈ ∂F B t ∂w B t + ∂F t ∂F t-1 dF t-1 dw B t-1 , (21)
and

dF t dw G t ≈ ∂G t ∂w G t + ∂F t ∂F t-1 dF t-1 dw G t-1 , (22)
where

∂F t ∂F t-1 = (G t ∂F A t ∂F t-1 + F A t ∂G t ∂F t-1 ) -(G t ∂F B t ∂F t-1 + F B t ∂G t ∂F t-1 ) + ∂F B t ∂F t-1 ,
and all the other required components can be derived using the chain rule of derivatives.

3 Results of the experiment The RS-RRL 2.0 trading system is designed to trade a single asset, there are 20 trading systems to trade the 20 SPI stocks. Every trading system consists of a group of 500 simulated traders (sim-traders). In the real world, as different traders may have different levels of information asymmetry, we initialize the sim-traders by using different numbers (i.e. random numbers from a Gaussian distribution with a mean of 0 and a standard deviation of 0.05). The sim-traders are then trained with the first 750 samples. At the end of the training period, we select the best-performing trader (referred to as the elitist trader hereafter) from the 500 sim-traders according to their Sharpe ratio rankings in an evaluation period (i.e. the last 125 trades before out-of-sample trading). The elitist trader is reselected at the beginning of each out-of-sample period.

Based on preliminary tests, the following parameters were found to be suitable settings for the daily equity trading problem: the number of shares traded ν = 1; the learning rate ρ = 0.15; the adaption rate η = 0.05. We expect that new information will be reflected in stock prices in a maximum period of two weeks, therefore we use a value of l = 10. The transaction cost δ has a value of 3 bps. The parameters of the transition variable and the threshold value have an initial value of 1. We use an optimization approach, differential evolution to estimate the threshold value (see [START_REF] Maringer | Transition variable selection for regime switching recurrent reinforcement learning[END_REF]), other parameters of the system are initialized using random numbers.

Out-of-sample trading performance

Profitability and stability are two particularly important factors in a financial trading system. In this study, we use the Sharpe ratio to measure the profitability and we calculate the Sharpe ratio using daily returns. It should be noted that the trading performance of RRL-type trading systems relates directly to the initialization of signal parameters. Therefore, stability refers to the consistency of the Sharpe ratios recorded from independent restarts of the trading system.

We restart the RS-RRL 2.0 trading system 100 times, saving the daily Sharpe ratio of the elitist trader in each out-of-sample period from each trial. We collect a number of 100 Sharpe ratios for each stock. The mean value and the standard deviation (SD) of the 100 Sharpe ratios are considered as the measures for profitability and stability respectively. We check the daily Sharpe ratio statistics which are produced by using the basic RRL, the RS-RRL 1.0, and the RS-RRL 2.0 trading systems for each stock. We collect these Sharpe ratio means and SDs based on 100 restarts of the RRL and the RS-RRL 1.0 trading systems from the three out-of-sample periods. To make a fair comparison, the system parameters used in the RRL and RS-RRL 1.0 trading systems are the same as that used in RS-RRL 2.0. Both of the RRL and RS-RRL trading systems consist of a group of 500 sim-traders. We select an elitist trader from the 500 sim-traders for outof-sample trading on the basis of their Sharpe ratio rankings from evaluation.

The means and standard deviations which are computed based the 100 restarts from the three out-of-sample periods are reported in Table 1 and Table 2. As these tables show that, the RS-RRL 2.0 trading system tends to perform at least as well as the RRL and the RS-RRL 1.0 trading system, and often better with respect to higher means and lower standard deviations. We perform the one-way analysis of variance (ANOVA) to compare the means of the Sharpe ratios produced by the RS-RRL 1.0 and the RS-RRL 2.0 trading systems. In Table 1, the Star symbols indicate the cases where the two Sharpe ratio means are significantly different from each other at a significance level of 5%, given a same transition variable. The results of the experiment show that most of the SR means from the RS-RRL 2.0 trading system are statistically greater than those from the RS-RRL 1.0 trading system at a significance level of 5%. Although most Sharpe ratio means are not statistically greater than zero, we can still produce a positive return after netting the profits and losses from trading the 20 stocks by using the RS-RRL 2.0, which confirms the same findings in [START_REF] Maringer | Transition variable selection for regime switching recurrent reinforcement learning[END_REF].

Conclusions

This paper presents an upgraded version of the RS-RRL trading system referred to in the literature. In the proposed RS-RRL 2.0 trading system, we use an automated transition function to model regime switches in equity returns. Unlike the original RS-RRL trading system, the dynamic of the transition function in our system is driven by utility maximization. Although the proposed system looks similar to an artificial neural network (ANN), we use a performance measure, i.e. the differential Sharpe ratio, which is different from the error function in traditional ANNs to implement the online learning for equity trading. We are able to reveal the time-varying feature of the threshold value by studying the parameter w G C (as shown in Figure 2), which provides a deeper insight into the impact of regime switches on stock returns comparing to the constant transition function suggested in the literature.

The results of our experiment show that regime-switching trading system brings value-added to trading performance. The RS-RRL 1.0 trading system referred to in the literature, produces higher Sharpe ratio means than the RRL trading system without impairing the stability of trading results. We also show that most the Sharpe ratios from the proposed RS-RRL 2.0 trading system are statistically greater than that from the RS-RRL 1.0 trading system at a significance level of 5%, given a same transition variable. With respect to transition variable selection, it seems that the price-to-earnings ratio is apt to produce higher Sharpe ratio means comparing the other four indicators; however, it is difficult to tell a 'one size fits all' indicator which improves the Sharpe ratio in general based on such a small sample of stocks.

3. 1

 1 The context of the experimentThe companies selected for this experiment are Swiss SPI stocks (1 April 2009 -26 September 2013, each series containing 1126 observations). We consider volume, relative strength index (RSI), price-to-earnings (P/E) ratio and moving average price changes from technical analysis, and conditional volatility from a GARCH model as options for selecting the transition variable. Of over 100 SPI stocks, 20 have full historical information (downloaded from Bloomberg) of the above variables. The 1126 observations of each stock are partitioned into an initial training set consisting of the first 750 samples. We consider three consecutive out-of-sample periods, with each out-of-sample period consisting of 125 observations.

Fig. 3 .

 3 Fig. 3. Training and trading

Table 1 .

 1 Sharpe ratio means

	Volume	Con.Vol	RSI 9		PE	MV 30
	RRL RSRRL1 RSRRL2 RSRRL1 RSRRL2 RSRRL1 RSRRL2 RSRRL1 RSRRL2 RSRRL1 RSRRL2
	1 0.033 0.004	0.098*	0.026	0.096*	0.028	0.094*	0.026	0.095*	0.040	0.101*
	2 0.020 0.027	0.031	0.005	0.029*	0.016	0.029	0.024	0.025*	0.046	0.025
	3 0.056 0.076	0.046*	0.026	0.025	0.060	0.055	0.033	0.027	0.013	0.036*
	4 0.040 0.041	0.033	0.038	0.039	0.041	0.033	0.043	0.040	0.006	0.023*
	5 0.008 -0.007	0.070*	0.007	0.070*	0.006	0.059*	0.018	0.072*	0.038	0.076*
	6 0.057 0.023	0.113*	0.060	0.112*	0.061	0.107*	0.097	0.112*	0.009	0.110*
	7 0.033 0.016	0.020	-0.003	0.042*	0.039	0.055*	0.009	0.021	0.009	0.014
	8 0.014 0.020	0.079*	0.021	0.076*	0.011	0.059*	0.023	0.080*	0.044	0.077*
	9 0.027 0.052	0.028*	0.035	0.031	0.024	0.023	0.067	0.041*	-0.006	0.019*
	10 0.032 0.029	0.047*	0.062	0.074	0.025	0.019	0.016	0.034*	-0.002	0.054*
	11 0.026 0.028	0.024	0.007	0.037*	0.017	0.031*	0.003	0.042*	0.023	0.036*
	12 0.050 0.069	0.078	0.075	0.071	0.051	0.058	0.027	0.073*	0.048	0.080*
	13 0.034 0.030	0.023	0.013	0.014	0.030	0.025	0.014	0.024*	0.029	0.015*
	14 0.064 0.072	0.082	0.060	0.066	0.077	0.094*	0.056	0.073*	0.069	0.087*
	15 0.032 0.043	0.068*	0.033	0.072*	0.036	0.061*	0.063	0.065	0.031	0.051*
	16 0.020 0.014	-0.015*	-0.026	-0.022	0.023	0.001*	-0.003	0.045*	-0.011	0.008*
	17 0.035 0.026	0.104*	0.029	0.087*	0.045	0.053	0.062	0.111*	0.004	0.106*
	18 0.019 0.019	0.006*	0.001	0.011	0.025	0.002*	0.028	-0.009*	0.024	0.004*
	19 0.035 0.042	0.067*	0.049	0.089*	0.043	0.087*	0.044	0.084*	0.036	0.081*
	20 0.066 0.005	0.049*	0.048	0.055	0.062	0.090*	0.082	0.048*	0.040	0.055*

Table 2 .

 2 Sharpe ratio standard deviations

	Volume	Con.Vol	RSI 9	PE		MV 30
	RRL RSRRL1 RSRRL2 RSRRL1 RSRRL2 RSRRL1 RSRRL2 RSRRL1 RSRRL2 RSRRL1 RSRRL2
	1 0.081 0.077	0.044	0.077	0.043	0.078	0.052	0.089	0.037	0.075	0.039
	2 0.070 0.072	0.060	0.069	0.053	0.067	0.059	0.065	0.047	0.081	0.049
	3 0.072 0.075	0.051	0.075	0.050	0.068	0.065	0.063	0.050	0.083	0.052
	4 0.071 0.070	0.042	0.069	0.043	0.069	0.051	0.060	0.043	0.059	0.055
	5 0.078 0.077	0.052	0.069	0.049	0.077	0.078	0.065	0.054	0.083	0.054
	6 0.073 0.072	0.037	0.077	0.039	0.075	0.050	0.064	0.036	0.095	0.038
	7 0.091 0.085	0.068	0.083	0.077	0.095	0.087	0.092	0.075	0.077	0.062
	8 0.099 0.092	0.105	0.094	0.112	0.099	0.107	0.086	0.111	0.104	0.112
	9 0.079 0.071	0.057	0.075	0.063	0.076	0.070	0.070	0.061	0.077	0.069
	10 0.081 0.081	0.126	0.099	0.154	0.085	0.110	0.076	0.131	0.094	0.145
	11 0.079 0.091	0.056	0.078	0.065	0.080	0.055	0.065	0.073	0.077	0.068
	12 0.068 0.081	0.060	0.072	0.063	0.074	0.054	0.080	0.076	0.075	0.080
	13 0.071 0.076	0.047	0.072	0.055	0.073	0.044	0.056	0.045	0.076	0.057
	14 0.072 0.073	0.064	0.078	0.067	0.076	0.058	0.069	0.069	0.076	0.079
	15 0.080 0.073	0.050	0.070	0.051	0.074	0.055	0.065	0.045	0.084	0.056
	16 0.082 0.095	0.085	0.077	0.080	0.077	0.082	0.086	0.080	0.079	0.096
	17 0.075 0.087	0.047	0.076	0.059	0.078	0.066	0.085	0.048	0.077	0.055
	18 0.078 0.073	0.083	0.086	0.072	0.072	0.090	0.071	0.083	0.090	0.068
	19 0.074 0.068	0.077	0.074	0.078	0.076	0.093	0.083	0.078	0.067	0.077
	20 0.095 0.085	0.092	0.082	0.073	0.094	0.097	0.093	0.079	0.076	0.087

Acknowledgements

This work has been partially supported by the Swiss National Science Foundation project SNSF-138095.


Mohamed Abou-Zleikha

Zheng-Hua Tan

Mads Graesbøll Christensen

Søren Holdt Jensen

Utilising Tree-Based Ensemble Learning for Speaker Segmentation

In audio and speech processing, accurate detection of the changing points between multiple speakers in speech segments is an important stage for several applications such as speaker identification and tracking. Bayesian Information Criteria (BIC)-based approaches are the most traditionally used ones as they proved to be very effective for such task. The main criticism levelled against BIC-based approaches is the use of a penalty parameter in the BIC function. The use of this parameters consequently means that a fine tuning is required for each variation of the acoustic conditions. When tuned for a certain condition, the model becomes biased to the data used for training limiting the model's generalisation ability. In this paper, we propose a BIC-based tuning-free approach for speaker segmentation through the use of ensemble-based learning. A forest of segmentation trees is constructed in which each tree is trained using a sampled version of the speech segment. During the tree construction process, a set of randomly selected points in the input sequence is examined as potential segmentation points. The point that yields the highest ∆BIC is chosen and the same process is repeated for the resultant left and right segments. The tree is constructed where each node corresponds to the highest ∆BIC with the associated point index. After building the forest and using all trees, the accumulated ∆BIC for each point is calculated and the positions of the local maximums are considered as speaker changing points. The proposed approach is tested on artificially created conversations from the TIMIT database. The approach proposed show very accurate results comparable to those achieved by the-state-of-the-art methods with a 9% (absolute) higher F1 compared with the standard ∆BIC with optimally tuned penalty parameter.

Introduction

Speaker segmentation is the process of determining the speaker switching points in a speech signal leading to an accurate separation of the signal into speaker homogeneous subsegments. This is an essential initial stage in several speech and audio applications such as speaker tracking [START_REF] Bonastre | A speaker tracking system based on speaker turn detection for nist evaluation[END_REF], audio classification [START_REF] Meinedo | Audio segmentation, classification and clustering in a broadcast news task[END_REF] and speaker diarization systems [START_REF] Anguera Miro | Speaker diarization: A review of recent research[END_REF]. Segmentation is usually performed without any prior knowledge about who or how many speakers are present in the speech segment, with an unknown maximum and minimum length of the speakers' segments and with no prior information about the acoustic conditions and the noise level and type. Having no or very limited prior information about these conditions makes speaker segmentation one of the very challenging task in the speech processing domain.

Several approaches have been proposed for speaker segmentation [START_REF] Chen | Speaker, environment and channel change detection and clustering via the bayesian information criterion[END_REF][START_REF] Li | Improving speaker segmentation via speaker identification and text segmentation[END_REF][START_REF] Ben | Speaker diarization using bottom-up clustering based on a parameter-derived distance between adapted gmms[END_REF][START_REF] Kotti | Automatic speaker change detection with the bayesian information criterion using mpeg-7 features and a fusion scheme[END_REF][START_REF] Cheng | Bic-based speaker segmentation using divide-andconquer strategies with application to speaker diarization[END_REF][START_REF] Mohammadi | Knndist: A non-parametric distance measure for speaker segmentation[END_REF]] (an extensive review can be find in [START_REF] Anguera Miro | Speaker diarization: A review of recent research[END_REF]). Among these methods, ∆BIC metric is the most widely used for this task [START_REF] Chen | Speaker, environment and channel change detection and clustering via the bayesian information criterion[END_REF][START_REF] Li | Improving speaker segmentation via speaker identification and text segmentation[END_REF][START_REF] Ben | Speaker diarization using bottom-up clustering based on a parameter-derived distance between adapted gmms[END_REF][START_REF] Kotti | Automatic speaker change detection with the bayesian information criterion using mpeg-7 features and a fusion scheme[END_REF][START_REF] Cheng | Bic-based speaker segmentation using divide-andconquer strategies with application to speaker diarization[END_REF]. The main drawback of BIC-based approaches is that they require fine tuning of a penalty factor that highly affects the quality of segmentation. This penalty factor is related to the number of parameters those are used to model the speech segment. Traditionally, the penalty factor is tuned for each specific acoustic conditions which limits the generalisation capabilities of the system [START_REF] Mori | Speaker change detection and speaker clustering using vq distortion for broadcast news speech recognition[END_REF][START_REF] Vandecatseye | The cost278 pan-european broadcast news database[END_REF]. An alternative BIC function has been proposed which aims at eliminating the effect of the number of parameters in the BIC calculation [START_REF] Ajmera | Robust speaker change detection[END_REF]. This is accomplished by estimating the speech segment by two mixtures of the Gaussian mixture model and estimating the left and right segments around the segmentation point by one mixtures of the Gaussian mixture model.

In this paper, we introduce a novel approach based on the ensemble-based mechanism to estimate the speakers changing points in multi-speakers segments. The main advantage of the proposed approach is that it eliminates the need to tune the penalty factor originally employed in the BIC-based segmentation function. Through using the proposed method we were able to increase the robustness of the traditional BIC-based techniques when used in various acoustic conditions.

The approach works by building a set of segmentation trees called segmentation forest; each tree is built using an approach similar to the one proposed in [START_REF] Cheng | Bic-based speaker segmentation using divide-andconquer strategies with application to speaker diarization[END_REF]. Each node in each segmentation tree examines a set of randomly selected points as potential segmentation points. The gain function in each tree is the value of the BIC and the goal is to select the point that gives the highest BIC. The value of the penalty factor is randomly assigned for each BIC calculation. Each segmentation tree assigns a BIC value for the examined points. The final resultant BIC values for the sequence points are the accumulated values using all segmentation trees in the forest.

The paper is organised as follows: In section 3, the standard BIC-based speaker changing point detection is presented, followed by a BIC-based speaker segmentation using divide-and-conquer strategy in Section 3. In Section 4, the proposed approach is explained. The experiments and evaluations conducted to validate the proposed approach are presented in Section 5 and finally a conclusion is presented in section 6.

Detecting Speaker Changing Points Using BIC

The BIC is a asymptotically optimal Bayesian-based model-selection criterion traditionally used to determine the parametric model that best fits a set of data samples X = x 1 , ..., x N , where x i ∈ R d , where d is the dimension of the feature space [START_REF] Chen | Speaker, environment and channel change detection and clustering via the bayesian information criterion[END_REF]. According to the BIC approach, the model M j that best fits a set of data samples is the model that maximises the function:

BIC j = logP (X|M j ) -λ 1 2 k j logN (1)
where logP (X|M j ) is the log likelihood of the data X for M j , λ is the weight of the second term that relates to the number of parameters k j in the model M j and the number of samples N in the data. Suppose we have two models representing the data X namely M 1 and M 2 , the model M 1 is chosen over M 2 to fit the data X if ∆BIC = BIC 1 -BIC 2 is positive.

For the speaker segmentation task, suppose a segment of speech X. To check if the speaker changes at a point index i, M 1 represents the model drawn from two fullcovariance Gaussians using the separated segments at point i and M 2 is the model drawn from one full-covariance Gaussian. We calculate the ∆BIC between M 1 and M 2 : the number of parameters k 1 in M 1 is twice the number of parameters in M 2 ,

where

k 1 = d + d(d+1)

2

. ∆BIC is calculated as:

∆BIC = N * log|Σ|-i * log|Σ lef t |-(N -i+1) * log|Σ right |- 1 2 λ(d+ d(d + 1) 2 )logN (2
) where |.| is the determinant of the covariance matrix, λ is the penalty factor that tunes the segmentation sensitivity [START_REF] Grašič | Online speaker segmentation and clustering using crosslikelihood ratio calculation with reference criterion selection[END_REF], ideally this parameter is equal to 1. Point i is considered a speaker switching point, if ∆BIC > 0.

Applying this approach on a speech segment detects a single changing point. In order to be able to detect multiple-switching points, a window growing mechanism is usually employed [START_REF] Tritschler | Improved speaker segmentation and segments clustering using the bayesian information criterion[END_REF]. This approach starts by processing a small window N init and tries to detect a changing point within this window. It then extends this window by a N g until a changing point is detected or the window size reaches a maximum size N max . If the search reaches N max with no detection of a changing point, the window is shifted by N s . Otherwise, the search process is repeated starting from the newly discovered switching point.

BIC-Based Speaker Segmentation Using Divide-and-Conquer

This approach uses the ∆BIC function to perform a hierarchical splitting of a speech sequence into its most two dissimilar parts [START_REF] Cheng | Bic-based speaker segmentation using divide-andconquer strategies with application to speaker diarization[END_REF]. This is done by scanning the whole segment and choosing the point that gives the highest ∆BIC. The resultant point is then considered as a potential speaker switching point (called i). The same approach is then repeated on the left and right segments of the point i. This process is applied recursively until the size of the segment becomes smaller than a threshold.

After processing the left and right segments, ∆BIC is checked, if it is positive, point i is considered as a switching point; otherwise, the leftmost sub-segment from the right segment and the rightmost sub-segment from the left segment are considered as one segment and the highest ∆BIC is calculated. If ∆BIC is positive, the new point is added to the set of changing points.

One can note that the process described can be seen as generating a tree-like structure, where each internal node in that tree is a segmentation point. It is also worth noticing that this approach still uses the standard ∆BIC where the tuning process of λ is still required.

The approach proposed in this paper employs a similar mechanism for generating a tree structure for the segmentation. These resultant trees are used to detect the changing points. In the next section, the approach proposed for speaker segmentation using a segmentation forest is explained.

BIC-based Speaker Segmentation Using Segmentation Forest

The ensemble-based learning approaches such as random forest and density forest [START_REF] Breiman | Random forests[END_REF] have been successfully employed for several tasks in the audio domain such as emotion recognition [START_REF] Schuller | The relevance of feature type for the automatic classification of emotional user states: low level descriptors and functionals[END_REF][START_REF] Rong | Acoustic feature selection for automatic emotion recognition from speech[END_REF], paralinguistic event detection [START_REF] Abou-Zleikha | Non-linguistic vocal event detection and localisation using online random forest[END_REF] and audio event detection [START_REF] Kumar | Audio event detection from acoustic unit occurrence patterns[END_REF]. In this work, a segmentation forest is utilised as a special case of the random forest approach. Random forest is a tree-based non-parametric classification and regression approach. The principle is to grow an assemble of trees on a random selection of samples in a training set. Each tree is a classification and regression tree (CART). While constructing the trees and at each tree node, a randomly selected set of features is considered and the features are investigated as potential predictors that decide the split of the data. The splitting robustness is calculated as the information gain resulted from the splitting.

The proposed approach applies a similar mechanism to the one used by the random forest. However, instead of building decision trees, we build a set of segmentation trees, where the randomly selected features to be examined at each node are the potential segmentation positions, the gain function in our case is the ∆BIC.

Formally, a segmentation forest SF is a set of segmentation trees

SF = {t i } : i = [1...T ] (3)
where t i is the i th individual tree and T is the total number of trees. Each tree t i , in the forest is trained independently in a similar manner to the one used for building the tree in the divide-and-conquer approach explained in Section 3. The main differences between the two processes are:

-Instead of scanning the full set of positions in the sequence, a randomly selected set of points is examined. -The value of λ in the ∆BIC function is randomly chosen between a min and a max values. -We do not recheck internal points if the value of ∆BIC is negative, instead we assign the negative value obtained for ∆BIC to the corresponding point. -Each tree is trained using a sampling from the original sequence without replacement.

A detailed description of the tree building process is presented in Algorithm 1.

The result of the pervious process is a set of segmentation trees where each node in each tree is associated with a position and a ∆BIC value. The final ∆BIC for each point is the accumulated ∆BIC from all trees. As a result, positive accumulated Algorithm 1 Building a segmentation tree n ← train(X) -X : the speech segment n : the node that represents the segmentation of the input segment if ( length(X) < threshold) then return empty end if pp ← get round of (length(X)) randomly chosen points from X as potential segmentation points to examine -Find the point i from pp that gives the highest ∆BIC (called mBIC), where λ is randomly chosen for each point -Split X at position i into X l and Xr -call the same process on X l to get n.child l -call the same process on Xr to get n.childr -store the position i in n.position --store the value mBIC in n.∆BIC return n ∆BIC values are produced on and around the speaker segmentation points. To detect the exact changing point positions, the resultant ∆BIC value sequence is grouped forming a set of local regions according to the distance between the points. The position of the maxima of each local region represents a speaker changing point as illustrated in Figure 1.

Experiment And Evaluation

The purpose of the experiments conducted is to validate the proposed approach and to check its efficiency for the speaker segmentation task. For this purpose, The performance and execution time of the proposed method is compared with several other BIC-based approaches reported in the literature.

Experiment Setup

In order to evaluate the proposed approach, a comparison between its performance and several other BIC-based approaches is performed. A artificially created conversations by concatenating speech from the TIMIT database are used for evaluation [START_REF] Lamel | Speech database development: Design and analysis of the acoustic-phonetic corpus[END_REF]. TIMIT is an acoustic-phonetic databased which consists of 6300 utterances for 630 english speakers. Two conversation sets (A and B) are generated, the first set is for penalty parameter tuning in the standard and divide-and-conquer BIC-based approaches. It consists of 20 conversation, each contains 2 to 6 speakers and the length of each speaker segment varies between 2 and 6 seconds. The testing dataset, B, consists of 100 conversation. The number of speakers per conversation changes from 2 to 6 and the length of each speaker segment is between 2 and 6 seconds. The testing set contains 338 switching points. The feature vectors used were 23-dimensional mel-frequency cepstral coefficients (MFCC) and log energy extracted every 10 ms, with a window size of 25 ms.

In the process of detecting the changing points in the speaker segmentation modules, two types of errors occur: the first is due to missing a true speaker changing point. This type of errors can be measured using the precision (PRC), which is calculated as:

P RC =
number of correctly f ound changes total number of changes f ound [START_REF] Ben | Speaker diarization using bottom-up clustering based on a parameter-derived distance between adapted gmms[END_REF] and the other measurement is the Missed Detection Rate (MDR), which is calculated as:

M DR = 100 * M D RC ( 5
)
where M D is the number of missed changing points and RC is the number of true changing points. The second error type occurs when a false changing point is detected. This type can be measured using the recall (RCL), which is defined as: RCL = number of correctly f ound changes total number of correct changes [START_REF] Breiman | Random forests[END_REF] and the other measurement is False Alarm Rate (FAR), which is calculated as:

F AR = 100 * (1 -RCL) (7)
Another measurement that combines the PRC and RCL in one value is the F 1measure. This measurement is defined as: The value of F 1 is between 0 and 1, the closer the value to 1 is, the better the system. For M DR and F AR measurements, the values are between 0 and 100, the smaller the value is, the better the system.

F 1 = 2 * P RC * RCL P RC + RCL (8)
Four approaches are evaluated using the conversation set. The first approach is the standard window-growing approach (referred to as BIC) as described in Section 2. The λ value is tuned using the set A and the result value is set to λ = 2.8. The second approach is the divide-and-conquer approach (referred to as DACBIC) as described in Section 3 and the λ value is also tuned using the set A and it gives λ = 3.2. The third one is the window-growing approach using one mixture per GMM for separated segments and two mixture per GMM for combined segments to estimate the covariance matrices [START_REF] Ajmera | Robust speaker change detection[END_REF]. This removes the effect of the penalty parameter (referred to as GMM-BIC). Diagonal covariance GMM is used for GMMBIC implementation. The fourth approach is the proposed approach (referred to as SF-BIC). The number of trees in our approach is set to 50 and the stopping criterion threshold (minimum segment length) is 500. The number of points to examine at each node is length(segment) Nmin where N min is the shifting factor. λ value was generated randomly between [0..6]. The tolerance value for the detected points is 0.5 second, i.e. if a detected point is positioned within 0.5 distance around a reference point, it is classified as a correctly detected point.

Evaluation

The five accuracy measures discussed are calculated for each examined approach. We used the same evaluation protocol proposed in [START_REF] Moschou | Systematic comparison of bic-based speaker segmentation systems[END_REF]. According to this protocol, the previously discussed error metrics are first calculated followed by applying ANOVA and Tukey test.

Table 1 presents the mean and standard divination values of each measure for each of the examined system.

The results show that a better performance is obtained by the proposed method compared with the other approaches with respect to the overall performance of the system as depicted by the F 1 measure. The results also indicate that our approach is able to more accurately predict the true changing points according to the RCL and M DR measures. The DICBIC detects less false alarm points compared with the other approaches as the results of P RC and F AR show.

To check if the performance obtained is significantly different among the compared approaches, a One-way ANOVA is applied for a 95% confidence level. The null hypothesis tested is that the groups means are equal, i.e. the approaches are not significantly different. The alternative hypothesis states that the groups means are unequal, which consequently means that at least one of the systems differs from the rest with respect to the examined measure. The F -statistic values and the p -values of all approaches are calculated and presented in Table 2. The results show that the performance is significantly different among all five measures. Since ANOVA test does not provide any information about which system is different from the other, the Tukey range test, or honestly significant differences method, is employed. Tukeys method provides a pair-wise comparison of the means while maintaining the confidence level at a predefined value. If the confidence interval includes zero, the differences are not significant, otherwise, the differences are significant.

The Tukey test is applied between the proposed approach and the three other approaches and the results obtained are presented in Table 3. The results show that the differences in the performance between all approaches are significant except for the F 1 measure between the proposed approach and the DICBIC approach. The results obtained from the statistical analysis performed indicate that the performance of our approach is comparable to those achieved by the-state-of-the-art method as demonstrated by the insignificant difference between the accuracies obtained.

The execution time for each approach (shown in Table 4) shows that proposed approach has the highest execution time. This is due to the time required to build a set of segmentation models instead of one. However, since each tree is built independently, this time consumption issue can be solved by constructing the trees in parallel.

Conclusion

In this paper, a tree-based ensemble method for speaker changing point detection is proposed. The approach trains a set of trees using a sampled version of the speech segment. To build a node in the tree, a randomly selected points are examined as potential segmentation points. The point that gives the highest ∆BIC is chosen. This process is recursively applied on the left and right subsegments until a stopping criterion is reached. As a result, a tree is constructed where each node stores the highest ∆BIC with the associated point index. Once the model is built, the accumulated ∆BIC for each point is calculated using the all trees in the forest. The final changing points are then calculated as the positions of the local maximums after grouping the points those have a positive ∆BIC into groups according to the distance between them.

We conduct a set of experiments to test the proposed approach and analyse its performance. For this purpose, a comparison is performed with three other state-of-the-art methods. The comparison shows that the proposed approach achieves better average results an insignificant performance difference from the best models reported in the literature. Our model, however, has the advantage of eliminating the need of parameter tuning and thereafter demonstrates more robustness and generalisation capability over changes in the acoustic conditions.

The future work includes building an interactive speaker changing point detection, where the user can modify, add or delete a set of changing points and the model uses this information to adapt itself.

Fig. 1 .

 1 Fig. 1. An illustration of a accumulated ∆BIC and the local regions with their maxima

Table 1 .

 1 Average and standard deviation values for the results obtained from each approach using the five error measurements

		BIC DICBIC GM M BIC SF -BIC
	P RC(mean) 0.77	0.89	0.59	0.83
	P RC(std)	0.30	0.27	0.28	0.24
	RCL(mean)	0.74	0.75	0.75	0.84
	RCL(std)	0.31	0.29	0.31	0.25
	F1(mean)	0.74	0.79	0.65	0.83
	F1(std)	0.29	0.27	0.23	0.23
	F AR(mean) 13.31% 3.97%	50.67%	10.78 %
	F AR(std)	16.94	11.00	23.88	14.22
	M DR(mean) 26.03% 25.33%	35.03%	16.47%
	M DR(std)	30.48	28.82	30.61	24.85

Table 2 .

 2 F -statistic and p -value results obtained from ANOVA for each approach using the five measurements

		F -statistic p -value
	P RC	66.64	1.42e-34
	RCL	6.59	2.36e-04
	F1	44.85	8.45e-25
	F AR	142.39	1.22e-61
	M DR	6.59	2.36e-04

Table 3 .

 3 The Tukey test results between the proposed approach and the other examined approaches.SF -BIC vs GM M BIC SF -BIC vs DICBIC SF -BIC vs BIC

	P RC	0.374,0.448	-0.126,-0.0540	-0.0130 , -0.0635
	RCL	0.105 , 0.1856	0.01201, 0.0886	0.01655 , 0.09561
	F1	0.3151 , 0.3805	-0.0401, 0.0302	0.0077 , 0.0820
	F AR	-45.477 , -39.888	3.197 , 6.8120	-6.977 , -2.530
	M DR	-26.489 , -18.561	-16.512, -8.860	-17.468 , -9.561

Table 4 .

 4 Average and standard deviation values for the execution time of each approach

		BIC DICBIC GM M BIC SF -BIC
	time(mean)(s) 50.16 19.82	91.81	124.77
	time(std)	25.76 12.26	50.24	54.18

This work was supported in part by the Danish Council for Strategic Research of the Danish Agency for Science Technology and Innovation under the CoSound project, case number 11-115328. This publication only reflects the authors' views.