Adrien Meynard

Bruno Torrésani

TIME-SCALE SYNTHESIS FOR LOCALLY STATIONARY SIGNALS

Keywords: Wavelet transform, time warping, probabilistic synthesis model

We develop a time-scale synthesis-based probabilistic approach for the modeling of locally stationary signals. Inspired by our previous work, the model involves zero-mean, complex Gaussian wavelet coefficients, whose distribution varies as a function of time by time dependent translations on the scale axis. In a maximum a posteriori approach, we propose an estimator for the model parameters, namely the timevarying scale translation and an underlying power spectrum. The proposed approach is illustrated on a denoising example. It is also shown that the model can handle locally stationary signals with fast frequency variations, and provide in this case very sharp time-scale representations more concentrated than synchrosqueezed or reassigned wavelet transform.

INTRODUCTION

Classical time-frequency analysis is generally used for building signal representations from which relevant information can be extracted (see e.g. [START_REF] Carmona | Practical time-frequency analysis: Gabor and Wavelet Transforms With an Implementation in S[END_REF][START_REF] Flandrin | Explorations in Time-Frequency Analysis[END_REF][START_REF] Gröchenig | Foundations of time-frequency analysis[END_REF] for reviews). Under suitable assumptions, linear transforms such as the STFT, wavelet transform or generalizations are invertible, which also leads to so-called synthesis approaches [START_REF] Balazs | Adapted and adaptive linear timefrequency representations: a synthesis point of view[END_REF]. The latter express signals as linear combinations of time-frequency atoms, and the corresponding time-frequency coefficients provide another type of time-frequency representation, which is less constrained by consistency requirements and uncertainty principles.

Statistical approaches to time-frequency analysis often rely on ad hoc statistical models for time-frequency transforms. Information extraction is then formulated as a statistical estimation problem. Examples include non-negative matrix factorization methods (see [START_REF] Févotte | Nonnegative matrix factorisations as probabilistic inference in composite models[END_REF]), detection of timefrequency components [START_REF] Reinhold | Objective detection and time-frequency localization of components within transient signals[END_REF][START_REF] Huillery | On the description of spectrogram probabilities with a chisquared law[END_REF], and several other tasks. In most situations, modeling appears as a post-processing stage after computation of a time-frequency transform. However, statistical models are generally not compatible with consistency conditions satisfied by time-frequency transforms.

Recently, Turner and Sahani [START_REF] Turner | Timefrequency analysis as probabilistic inference[END_REF] introduced a new Bayesian paradigm, under the name of probabilistic time-frequency representations. The idea is to express signals as the result of a synthesis from a random time-frequency representation, for which a prior distribution is chosen. This is applied to various contexts, such as the synthesis of stationary signals, and several non-stationary situations, including NMF-based component estimation, non-stationary noise. A similar point of view has already been taken by various authors in the past, see for example [START_REF] Davy | Bayesian Analysis of Polyphonic Western Tonal Music[END_REF] and references therein. We rely here on the Turner-Sahani model, which we revisit in a slightly different way, assuming a generative model of the form

y(t) = Re s (ψ s * W s )(t) + ǫ(t) , (1)
where {ψ s , s} is a filter bank, labeled by a frequency (or scale) index s, the W s (t) are random subband (time-frequency or time-scale) coefficients, and ǫ(t) is a noise. Our focus is here on non-stationary signals, more precisely locally stationary signals for which time-dependent spectral characteristics can be defined. Following our earlier JEFAS approach [START_REF] Meynard | Spectral Analysis for Nonstationary Audio[END_REF][START_REF] Meynard | Séparation de sources doublement non stationnaire[END_REF][START_REF] Omer | Time-frequency and time-scale analysis of deformed stationary processes, with application to non-stationary sound modeling[END_REF], we investigate a class of nonstationarity generated by time-dependent shifts in the timescale domain. JEFAS is analysis based, i.e. post-processing of time-scale representation. We introduce JEFAS-S, a Bayesian synthesis approach that provides adaptive time-scale signal representation, together with corresponding parameter estimation. While JEFAS based estimation was based on approximations of the time-scale transform, an exact estimation is possible here, and we provide a corresponding EM algorithm. In addition, in some situations, the model is flexible enough to provide extremely concentrated time-scale representations that can be sharper than reassigned transforms [START_REF] Franc | Improving the readability of time-frequency and time-scale representations by the reassignment method[END_REF].

THE SYNTHESIS MODEL

In this paper, we limit to time-scale representation, i.e. wavelet synthesis. We denote by ψ the analysis wavelet, and by ψ s scaled wavelets defined by ψ s (t) = q -s/2 ψ (q -s t), for some constant q > 1. s ∈ {s 1 , ... s M } is a finite set of scales.

The discrete model

We consider the finite periodic case: assume we have N time values τ = (τ 1 • • • τ N ) T and the corresponding sampled signal y = (y(τ 1 ) • • • y(τ N )) T with sampling frequency F s . We then focus on a corresponding discretized wavelet transform. For n ∈ {1, ... N }, m ∈ {1, ... M }, denote by ψ nm ∈ C N the vector (ψ sm (τ

1-n ) • • • ψ sm (τ N -n )) T ,
y = y 0 + ǫ = Re N n=1 Ψ n w n + ǫ , (2)
where the w n ∈ C M , n = 1, . . . N are vectors of synthesis coefficients. This model can also be written in matrix form as y = Re(DW + ǫ), where the dictionary matrix D is the concatenation of matrices Ψ n , and W = vec(w 1 , . . . w N ).

In this paper, ǫ will be modeled as a Gaussian white noise, with variance σ 2 , as in [START_REF] Turner | Timefrequency analysis as probabilistic inference[END_REF]. Non-stationarity will be introduced via a suitable prior on W, that intends to describe locally time-warped situations as introduced in [START_REF] Clerc | Estimating deformations of stationary processes[END_REF].

A class of non-stationary priors: time warping

When all subband signals W s in (1) are stationary, the resulting signal y is stationary. We are interested here in a specific situation where non-stationarity induces a time-dependent shift on the scale axis, as studied in [START_REF] Clerc | Estimating deformations of stationary processes[END_REF][START_REF] Omer | Time-frequency and time-scale analysis of deformed stationary processes, with application to non-stationary sound modeling[END_REF][START_REF] Meynard | Spectral Analysis for Nonstationary Audio[END_REF]

y(t) = (D γ x)(t) ∆ = γ ′ (t)x(γ(t)) , (3)
where x is a wide sense stationary random signal, and γ is a smooth, strictly increasing function.

To build the prior distribution on discrete subband coefficients, we make the following assumptions

• The vectors w n are decorrelated, zero-mean, circular complex Gaussian vectors:

w n ∼ CN c (0, C n ) • The corresponding covariance matrices C n are trans-
lates of a fixed function f as shown in [START_REF] Meynard | Spectral Analysis for Nonstationary Audio[END_REF], namely

[C n ] mm ′ ∆ = [C(θ n )] mm ′ = f (s m +θ n , s m ′ +θ n ) , (4)
where f : R * + 2 → C is Hermitian and positivesemidefinite, and θ n ∈ R is the shift parameter.

In [START_REF] Meynard | Spectral Analysis for Nonstationary Audio[END_REF], it was shown that the wavelet coefficients of a stationary random signal modified by time warping transform can be approximated by random vectors satisfying the above assumptions. There, the parameter θ n represents a local dilation factor at time τ n (derivative γ ′ (τ n ) of the time warping function at τ n ), and f involves the power spectrum S of the underlying signal and the Fourier transform of the wavelet:

f (s, s ′ ) = q s+s ′ 2 ∞ 0 S (ξ) ψ(q s ξ) ψ(q s ′ ξ) dξ . (5)

ESTIMATION PROCEDURE

Bayesian inference

The estimation of the subband coefficient matrix W relies on the evaluation of the corresponding posterior distribution. The latter depends on the following parameters, which are supposed to be known at this point: the dilation factors θ n , and the covariance function f . Let Γ 0 ∈ C MN ×MN be the block diagonal matrix with blocks C 1 , . . . C N . The posterior distribution of the subband coefficient is a complex Gaussian law p(W|y) ∼ CN (µ, Γ, R), with mean and covariance

µ = Γ 0 D H C -1 y y , Γ = Γ 0 - 1 4 Γ 0 D H C -1 y DΓ 0 ,
(the relation matrix R, not useful here is not provided) where

C y = σ 2 I + 1 2 Re DΓ 0 D H . ( 6
)
Therefore the posterior expectation wn of w n reads

wn = 1 2 C n Ψ H n C -1 y y , (7)
where the matrix C y can be expressed as

C y = σ 2 I + 1 2 Re N n=1 Ψ n C n Ψ H n . (8)
Remark 1. It is worth mentioning that unlike the prior distribution, the posterior distribution of subband coefficients involves time correlations. Indeed, given any n, n ′ = 1 . . . N ,

E{ wn wH n ′ } = δ nn ′ C n - 1 4 C n Ψ H n C -1 y Ψ n ′ C n ′ ,
which generally does not vanish when n ′ = n.

Parameter selection and reconstruction

We now discuss the choice of the two model parameters, i.e. the scaling factors θ n and the covariance function f . We first notice that the expression in equation ( 5) provides a natural choice for the covariance function f . The latter involves the wavelet, which is known, and the power spectrum S of the underlying stationary process, which is unknown. In this setting, we then have to provide the vector θ = (θ 1 , . . . θ N ) of scaling factors and the power spectrum.

The JEFAS algorithm introduced in [START_REF] Meynard | Spectral Analysis for Nonstationary Audio[END_REF] provides a valuable, analysis-based approach for estimating θ and S . We now describe an alternative algorithm, based on the EM (Expectation Maximization) principle [START_REF] Dempster | Maximum Likelihood from Incomplete Data via the EM Algorithm[END_REF], especially tailored for the synthesis approach developed in this paper. Here, y is the observation, θ the parameter, and W the latent variable.

Proposition 1 (EM steps). Denote by θ(k-1) the vector of dilation factors at iteration k -1 of the algorithm. Let W(k-1) be the matrix of subband coefficients at iteration k -1. Then the update at iteration k relies on the following two steps:

1. For n ∈ {1, . . . , N }, the estimate [START_REF] Huillery | On the description of spectrogram probabilities with a chisquared law[END_REF]

at time τ n reads w(k) n = 1 2 C θ(k-1) n Ψ H n C y θ(k-1) -1 y . (9)
2. The scaling factor θ is re-estimated by solving

θ(k) n = arg min θ Q kn (θ) , (10)
Q kn (θ) = log|det(C(θ))|+ w(k)H n C(θ) -1 w(k) n +Trace C(θ) -1 Γ n θ(k-1) , Γ n θ(k-1) ∈ C M×M being the n-th diagonal block
of the posterior covariance matrix Γ θ(k-1) .

After running the corresponding algorithm (described in more details below), an estimate for the time-scale coefficients w is available, and a corresponding estimate ỹ0 for the signal y 0 can be obtained as

ỹ0 = Re N n=1 Ψ n wn . ( 11
)
Notice that the reconstruction expression (11) combined with [START_REF] Huillery | On the description of spectrogram probabilities with a chisquared law[END_REF] can be interpreted as a Wiener filtering. The bias and variance of the estimator can be evaluated.

Proposition 2. With the above notation, the bias of the estimator ỹ0 is given by

B ∆ = E {ỹ 0 |y 0 } -y 0 = -σ 2 C -1 y y 0 , (12)
and the corresponding error variance reads

R(ỹ 0 |y 0 ) ∆ = E (ỹ 0 -E{ỹ 0 |y 0 }) (ỹ 0 -E{ỹ 0 |y 0 }) T y 0 = σ 2 I -σ 2 C -1 y 2 . ( 13
)

Algorithm: JEFAS-Synthesis

The steps of the estimation algorithm are given in Algorithm 1. The latter takes as input the signal y, the noise variance σ 2 , a precision parameter Λ for the stopping criterion and a bandwidth parameter N ′ (see below).

Initialization. The algorithm requires initial estimates θ (0) for the parameters, and the function f in (4). In JEFAS-S, we use the expression [START_REF] Févotte | Nonnegative matrix factorisations as probabilistic inference in composite models[END_REF], for which an initial estimate of S has to be provided. When successful, JEFAS [START_REF] Meynard | Spectral Analysis for Nonstationary Audio[END_REF] provides such an estimate. Otherwise, a rough estimate can be obtained from the Welch periodogram of the input signal y.

Stopping criterion. EM guarantees the monotonicity of the Likelihood function L(θ). The increment of the latter is used as a stopping criterion: EM will stop when the condition

L(θ (k) ) -L(θ (k-1) ) < Λ ( 14
)
is true. Here Λ > 0 is a parameter fixed by the user.

Algorithm 1 ( W, θ, SX ) = JEFAS-S(y, σ 2 , Λ, N ′ )

• Initialization: estimate θ(0) and S (0) using JEFAS. • k ← 1. while stopping criterion (14) = FALSE do for all n ∈ {1, . . . , N } do

• Restrict Ψ n , C y θ(k-1)
and y to the interval [n-

N ′ /2, n + N ′ /2]. • Compute w(k) n using (9). end for • Estimate θ(k)
by solving [START_REF] Meynard | Spectral Analysis for Nonstationary Audio[END_REF]. • Estimate S (k) using the wavelet based estimate.

• k ← k + 1.
end while Dimension reduction. The matrix C y of dimension N M × N M can be extremely large. However, it generally has fast off-diagonal decay. This can be exploited to speed up the evaluation of wn in ( 9) by restricting to a neighborhood

[n -N ′ /2, n + N ′ /2] of n of given bandwidth N ′ .
Optimization. The optimization problem ( 10) is solved using a standard quasi-Newton scheme.

Spectrum estimate update. The spectrum update from the current estimate of W is performed in two steps: first correct for the translation by θ n , to obtain an approximately stationary subband transform, then average over time to obtain a wavelet based spectral estimate as in [START_REF] Meynard | Spectral Analysis for Nonstationary Audio[END_REF].

Remark 2. Other choices can be made for the function f , which can lead to different estimates for subband coefficients, while preserving reconstruction (see section 4.2).

NUMERICAL RESULTS

Illustration on denoising of a synthetic signal

We first evaluate the performances of JEFAS-S on a denoising problem. A synthetic non-stationary signal y is built as follows: start from a stationary signal x, with power spectrum S equal to the sum of two non-overlapping Hann windows, and apply the time warping deformation D γ to x, with γ ′ an exponentially damped sine wave. Here, x is one second long, sampled at F s = 8192 Hz.

We denote by SNR y and SNR ỹ0 the input and output signal-to-noise ratios. Numerical results show that SNR ỹ0 is larger than SNR y as long as SNR y is in the range [2 dB, 25 dB], with maximal improvement of 8 dB. The 25 dB upper limit for SNR ỹ0 is presumably due the distortion intrinsically introduced by the reconstruction formula [START_REF] Meynard | Séparation de sources doublement non stationnaire[END_REF]: bias and variability in the time warping estimation.

In the specific case where the input SNR is 16 dB, and after initializing with the output of JEFAS, JEFAS-S converges in 3 iterations (CPU time: 347 seconds on a computer running an Intel Xeon E5-2680 v4 processor). JEFAS-S does not significantly improve the quality of the estimated time warping function. Indeed, the mean square error on the time warping function estimation decreases by about 0.5% from JEFAS to JEFAS-S. We display the estimated adapted time-scale representation W in Fig. 1 (left). As expected, it is very similar to the wavelet transform (right), though a bit sharper. Indeed, the choice of the expression (5) for the covariance function f yields a wavelet-like representation. The main visible difference concerns the temporal oscillations of W, due to the prior assumption of temporal decorrelation between w n .

Locally harmonic signal with fast varying frequency

We now consider a locally harmonic signal, of the form

y(t) = A(t) cos(2πφ(t)) ,
where the instantaneous frequency φ ′ is a fast varying function chosen as the measurement of the heart rate of a person suffering from atrial fibrillation (real data). The synthetic instantaneous amplitude A is a slowly varying function, the signal is termed "semi-real". Apart from the amplitude modulation, this signal follows the model (3): the time warping function derivative is the instantaneous frequency, and the underlying stationary signal x is sinusoidal. The signal duration is 83.1 seconds, sampled at F s = 10 Hz (N = 832 samples). Because of the fast instantaneous frequency variations, the wavelet transform of y (not shown here) contains interference patterns, the model in [START_REF] Meynard | Spectral Analysis for Nonstationary Audio[END_REF] is not adequate and JEFAS does not converge. We initialized JEFAS-S to θ = 0, and a constant function for S . Given these initial values (far from actual values), JEFAS-S converges slowly (72 iterations). Results are displayed in Fig. 2, where the estimated instantaneous frequency is superimposed on the ground-truth. This shows that JEFAS-S is indeed able to estimate fast varying frequency modulations.

In addition, one can take advantage of this result to obtain a sharper time-scale representation. To that end, we choose a sharply concentrated prior covariance function f ♯ , of the form f ♯ (s, s ′ ) = exp{-(s -ς) 2 /σ 2 s }δ ss ′ , where ν 1 denotes the central frequency of the sine wave, ς = log q (ξ 0 /ν 1 ), and σ s is a tuning parameter for the scale concentration. We display in the top of Fig. 3 the covariance matrices C(0) corresponding to the expression (5) (left) and f ♯ (right), which is indeed Furthermore, we stress that the quality of the reconstruction is not degraded.

CONCLUSION

We have described an alternative to the JEFAS model of [START_REF] Meynard | Spectral Analysis for Nonstationary Audio[END_REF] for locally deformed signals. Unlike JEFAS, which is an analysis based approach (i.e. post-processing of wavelet transform), JEFAS-S is synthesis-based and therefore less constrained by uncertainty principles. We illustrated the JEFAS-S on a denoising example. Our numerical results also show that JEFAS-S is able to handle locally stationary signals with fast varying instantaneous frequency, and can provide very sharp time-scale representations. While the current paper was focused on wavelet transform, the JEFAS-S model can handle arbitrary subband decompositions (such as the NSDGT [START_REF] Balazs | Theory, implementation and applications of nonstationary Gabor frames[END_REF]). Such extensions will be discussed in a forthcoming publication, together with additional numerical results and complete proofs. JEFAS-S can also be extended to more general transformations, for example involving amplitude modulations or filtering posterior to time warping. This is an ongoing work.

  and by Ψ n ∈ C N ×M the matrix obtained by concatenation of vectors ψ nm , m ∈ {1, ... M }. The observation equation (1) then reads

Fig. 1 .

 1 Fig. 1. Synthetic signal. Left: representation given by JEFAS-S. Right: scalogram (wavelet transform).

Fig. 2 .Fig. 3 .

 23 Fig. 2. Semi-real signal. Estimated time warping function compared with the normalized instantaneous frequency φ ′ .

  . It was shown there that such a model can account for signals obtained by time warping stationary signals, namely signals of the form


Larry Karp
email: karp@are.berkeley.edu

Thierry Paul

Cross-sectoral pollution externalities and multiple equilibria

Keywords: coordination games, costs of adjustment, cross-sectoral pollution, factor reallocation, learningby-doing, intersectoral migration, multiple equilibria JEL Classification: C61, J21,O13, R23

In familiar models, a decrease in the friction facing mobile factors (e.g., lowering their adjustment costs) increases a coordination problem, leading to more circumstances where there are multiple equilibria. We show that a decrease in friction can decrease coordination problems when a production externality arises from a changing stock, e.g. of pollution or knowledge. In general, the relation between the amount of friction that mobile factors face and the likelihood of multiple equilibria is non-monotonic.

Introduction

Nonconvex technologies can give rise to multiple competitive equilibria. We extend intuition about multiplicity in models of intersectoral labor migration in two circumstances. First, production in a sector might create pollution that lowers productivity in the other sector. Second, cumulative production in one sector might lower costs there due to learning-by-doing that is external to firms. Our main contribution is to explain why multiplicity may be very unlikely in exactly the circumstances where previous models would suggest that it is likely to occur.

In some circumstances, the payoff from moving to a different sector ('migrating') is higher if many other agents also migrate. In this situation, actions are strategic complements and there may be multiple equilibria. For example, in [START_REF] Matsuyama | Increasing returns, industrialization, and indeterminancy of equilibrium[END_REF] and [START_REF] Krugman | History versus expectations[END_REF] migration models, agents decide whether to work in the Agricultural or Manufacturing sector. For some range of labor allocations, an externality causes the benefit of working in a particular sector to increase with the number of workers there. Agents' decisions depend on their beliefs about what other agents will do, rather than merely on exogenous or predetermined economic fundamentals. This model has been used to explain why similar countries might follow completely different development paths.

If it is very costly for agents to change previous decisions -i.e., if the amount of friction is extreme -then there is little scope for agents' current decisions to depend on their beliefs about what other agents will do. In this case, the multiplicity of equilibria is unlikely. If the cost of changing previous decisions is negligible (and given that actions are strategic complements), it is natural to think that beliefs are a greater factor in the decision-making process, making the multiplicity of equilibria more likely. This type of reasoning leads to the conjecture that lower friction (cheaper migration costs) increases the 'likelihood' of multiplicity. We show that this conjecture is not true in general.

Our model can be used to describe two types of situations. This flexibility is important because it shows that our results are relevant beyond the case we emphasize, arising from cross-sectoral pollution externalities. There, Manufacturing output creates pollution flows that raise pollution stocks, damaging natural capital and lowering labor productivity in Agriculture, as in [START_REF] Copeland | Trade, spatial separation, and the environment[END_REF]. At a point in time the wage differential depends only on the stock of natural capital, which is inversely related to the pollution stock. Increased Manufacturing output increases the pollution stock, lowering future labor productivity in Agriculture, altering the future wage differential.

By relabelling variables, the model can be interpreted as an extension of [START_REF] Krugman | History versus expectations[END_REF] migration model to include learning-by-doing in Manufacturing, as in [START_REF] Matsuyama | Agricultural productivity, comparative advantage, and economic growth[END_REF]. With this interpretation, labor productivity in Manufacturing depends on the amount of knowledge in that sector. Manufacturing production increases the stock of knowledge via learning-by-doing; this knowledge decays if the level of production falls. Changes in the stock of knowledge alter the wage differential, changing incentives to migrate.

We describe our research question and results, and then review related literature. Then in section 2 we present the model and in section 3 the results.

Research question and results

Dynamic models with complementarities can give rise to two or more stable steady states. Each stable steady state has a basin of attraction, defined as the set of initial conditions from which there is an equilibrium trajectory that approaches that steady state. The intersection of two basins of attraction is the 'Region of Multiplicity' , or ROM. If the ROM is empty, there exists a unique equilibrium trajectory for all initial conditions. In this case, we regard the equilibrium as unique, despite the existence of multiple steady states. In our usage, 'equilibrium' always refers to a trajectory, not simply to a steady state. If the ROM has positive measure, then there exists a set with positive measure such that from any point in this set there are multiple equilibrium trajectories. In this case, the model exhibits multiple equilibria for some initial conditions.

How does a change in intersectoral labor migration costs affect the multiplicity of equilibria? There are two ways to interpret the statement that a parameter change makes multiplicity 'more likely' :

Interpretation 1: The change increases the measure of the set of other parameters for which multiplicity occurs. Interpretation 2: The change increases the measure of the ROM, holding fixed other parameters.

Interpretation 2 means that there are more initial conditions for which there exist multiple equilibria. The model contains two state variables. The first of these is the fraction, L, of labor in Manufacturing. We denote the second state variable as K. In both interpretations of the model, a larger K increases the Manufacturing-Agriculture wage differential, making it more attractive to work in the manufacturing sector. The mechanism behind this relation differs in the two interpretations of the model.

In the pollution setting, we interpret K as a pollution stock. An increase in L raises manufacturing output, increasing the flow of pollution generated in that sector, thereby increasing the pollution stock. The higher pollution stock lowers productivity in Agriculture: the pollution externality is 'cross-sectoral' . In the learning-by-doing setting, we can interpret K as the stock or knowledge in Manufacturing. An increase in L again raises manufacturing output, but now this higher output contributes to learning-by-doing, increasing K. The higher stock of knowledge raises Manufacturing productivity, increasing the Manufacturing-Agriculture wage differential. To avoid repetition, we focus on the cross-sectoral pollution interpretation.

Figure 1 illustrates the relation between the models with one and with two state variables, and the distinction between the two interpretations of 'more likely'. In the model with one state variable (L), illustrated by panels (a1) and (b1), the state space is [0, 1]; there are two steady states, L = 0 and L = 1 where, respectively, all labor is in Agriculture or in Manufacturing. In the model with two state variables, (L, K), illustrated by panels (a2) and (b2), the state space is the unit square. Here there are two stable steady states, (0, 0) and (1, 1). In the first, all labor is in Agriculture and the pollution stock is zero, so agricultural productivity is high. In the second, all labor is in Manufacturing and pollution is at its maximum level, equal to 1, causing agricultural productivity to be low.

In panels (a1) and (a2), the basins of attraction for the steady states, in both the onestate and the two-state models, are the sets A and B. These sets are intervals in the onestate model and regions in the two-state model. The sets A and B have no intersection except for the boundary, which is of measure 0. A typical interior point a ∈ A (or b ∈ B) has a unique trajectory to the steady state where all labor is in Agriculture (respectively, Manufacturing). Panels (a1) and (a2) illustrate the situation where the equilibrium is unique and there is hysteresis: the steady state depends on the initial condition.

In contrast, in panels (b1) and (b2) the state space consists of three sets, A, B and C. The basin of attraction for the steady state with all labor in Agriculture is A ∪ C and the basin of attraction for the steady state with all labor in Manufacturing is B ∪ C. The intersection of these two sets, C, is the ROM. From point c ∈ C there are two trajectories, approaching different steady states. The equilibrium is not unique in the models represented by panels (b1) and (b2), because there is a set of initial conditions, with positive measure, for which there are multiple equilibrium trajectories.

According to Interpretation 1, a decrease in friction makes multiplicity more likely if it increases the measure of the set of parameters for which panels (b1) or (b2) rather than panels (a1) or (a2) describes the dynamics. According to Interpretation 2, the decrease in friction makes multiplicity more likely if it increases the measure of the ROM, C.

The chief result from our two-dimensional model is that there is a non-monotonic relation between the measure of the ROM and friction in the adjustment for labor. Thus, according to Interpretation 2, a decrease in labor adjustment costs may make multiplicity either more or less likely. A decrease in labor adjustment costs does, however, increase the set of other parameter values for which the ROM is positive. Thus, according to Interpretation I, lower adjustment costs make multiplicity more likely, just as in the standard one-state models. But according to Interpretation 2, lower adjustment costs might make multiplicity either more or less likely. Taken together, the two results imply that if labor adjustment costs are extremely small, the measure of the ROM is positive for a wide range of parameter values, but the measure is always extremely small. Multiplicity in this case is possible, but it occurs only for a small set of initial conditions and in that sense is not likely.

For the type of model we consider, this non-monotonicity requires the second state variable, e.g. a stock pollutant instead of a flow pollutant. However, the nonmonotonicity survives even if the stock pollutant decays very quickly. In that case, the amount of damage-creating pollution remaining over a small interval of time closely tracks the emissions flow during that interval -because emissions flows over previous periods have already largely dissipated. In this respect, the two-state model with rapidly decaying stock appears to approximate a one-state model with a flow pollutant. Nevertheless, the non-monotonicty does not arise under a flow pollutant. Thus, two models that appear similar might lead to different types of results.

Even if the pollution stock decays rapidly (causing the stock to closely track the pollution flow), it remains possible for the labor allocation to change still more quickly. Rapid adjustment of labor requires that migration costs be small. When the speed of adjustment in the labor market is fast relative to the speed of adjustment of the pollution stock, a worker can move between sectors quickly enough to insure that she remains in the high wage sector 'most of the time' . In this case, for most initial conditions, the worker's migration decision is not sensitive to her beliefs about what other workers will do. Multiplicity is possible, but the ROM is small.

We use a deterministic model and assume that agents have rational point expectations: they correctly anticipate the evolution of the economy. If agents were myopic, there would be no role for expectations to influence behavior. There might nevertheless be multiple stable steady states, each with its basin of attraction. In this case, there would be hysteresis, but not multiplicity of equilibria (as we use that term).

Literature review

Previous literature, excepting [START_REF] Krugman | History versus expectations[END_REF], [START_REF] Fukao | History versus expectations: a comment[END_REF] and [START_REF] Karp | Indeterminacy with environmental and labor dynamics[END_REF], neglects the relation between parameters of the model and the measure of the ROM, and concentrates on the relation between parameter values and the existence of a ROM with positive measure. There are two likely reasons for this emphasis. First, although it is sometimes straightforward to determine conditions under which the ROM has positive measure, the comparative statics of this measure are complicated.

Second, intuition (supported by Krugman's model) may have encouraged the idea that the two senses in which a parameter change can make multiplicity 'more likely' are essentially the same.

Multiplicity of equilibria arises when there are increasing returns to scale, or some other feature that makes the economy non-convex. Greater convexity of adjustment costs (more generally, increased friction) appears to counteract the effect of crosssectoral pollution or increasing returns to scale. Therefore, a natural conjecture is that more convex adjustment costs make multiplicity less likely. [START_REF] Karp | Indeterminacy with environmental and labor dynamics[END_REF]use numerical methods to show that this conjecture is not true in general in a two-state version of [START_REF] Krugman | History versus expectations[END_REF]. Here we use a simpler framework and obtain analytic results, thereby improving our intuition for this type of model.

There is little empirical evidence regarding the type of multiplicity we described above. Davis andWeinstein (2002, 2004) find that Japanese data is consistent with increasing returns to scale, but that the data is inconsistent with the existence of multiple stable steady states -a necessary condition for multiplicity of equilibria in this setting. [START_REF] Moro | The effect of statistical discrimination on Black-White wage inequality: estimating a model with multiple equilibria[END_REF] estimates a multiple equilibrium model of wage inequality; Brock andDurlauf (2001, 2002) discuss the estimation of discrete choice models with social interactions, a situation that can lead to multiple equilibria.

There is a different type of multiplicity, often referred to as 'indeterminacy', that occurs when there exists a continuum of equilibrium trajectories to a single steady state. Indeterminacy is an important topic in macro-economics, where the role of costs of adjustment (friction) is significant. [START_REF] Benhabib | Indeterminacy and sunspots in macroeconomics[END_REF] review this literature; recent contributions include [START_REF] Cooper | Evidence on macroeconomic complementarities[END_REF], [START_REF] Cooper | Dynamic complementarities[END_REF], Wen (1998a,b), [START_REF] Benhabib | Indeterminacy under constant returns to scale in multisector economies[END_REF], [START_REF] Nishimura | Indeterminacy in a dynamic small open economy[END_REF] and [START_REF] Lubik | Testing for indeterminacy: an application to U.S. monetary policy[END_REF]. A recurring question concerns the specification for which plausible estimates of adjustment costs and returns to scale are consistent with indeterminacy. In these models, lower costs of adjustment (less friction for the mobile factor) mean that indeterminacy is more likely, using Interpretation 1. Since the analysis of these models (typically) examines behavior in the neighborhood of the steady state, no attention is paid to Interpretation 2.

Recent theoretical papers show that changing an assumption of migration models may eliminate the multiplicity of equilibria. [START_REF] Frankel | Resolving indeterminacy in dynamic settings: the role of shocks[END_REF] show that multiplicity disappears in a variation of Matsuyama's model where the wage differential is subject to Brownian motion and there exist 'dominance regions'. In another variation of this model, [START_REF] Herrendorf | Ruling out multiplicity and indeterminancy: the role of heterogeneity[END_REF] show that there is a unique equilibrium if agents are sufficiently heterogeneous. We provide another explanation that might either increase or diminish the importance of coordination problems.

The model

The discrete time model makes it easy to obtain the comparative static results in section 3. To introduce our model and also to establish the connection between our discrete time formulation and previous literature, we begin with a one-state, two-period formulation (section 2.1). This model reproduces the major insights from [START_REF] Krugman | History versus expectations[END_REF]who considers increasing returns to scale that are external to firms, whereas we focus on crosssectoral pollution, where both sectors have constant returns to scale. This difference changes the interpretation of the one-state model but not the nature of the results.

Our contribution uses a two-state, three-period model where all migration occurs during the first two periods (section 2.2). We use the two-state, three-period model in section 3 to show analytically that an increase in intersectoral labor mobility (a decrease in friction) can reduce the set of initial conditions from which there emanate multiple equilibrium trajectories. That is, an increase in mobility can decrease the measure of the ROM, and in this respect make multiplicity less likely.

The two-period, one-state model

This section obtains the insights from Krugman's continuous time formulation using a two-period model. The initial stock of labor in Manufacturing is 0 ≤ L ≤ 1 and the measure of entrants into Manufacturing is u, so the amount of labor in Manufacturing in the next period is L + u. Workers who move incur a cost of migration in the current period and (possibly) obtain a higher wage in the next. The discount factor is β. The manufacturing wage is fixed at α M . Pollution arising from the manufacturing sector in the second period equals L + u. Average and marginal productivity (and thus the wage) in Agriculture in this period equals α Ab(L + u); each unit of pollution lowers the wage in Agriculture by b units. This one-state variable model describes a flow pollutant. The Manufacturing-Agriculture wage differential in the second period therefore equals a + b(L + u),witha ≡ α M -α A . We assume that a < 0 and a + b > 0: if all workers are in the same sector, that is the high-wage sector.

The social cost of migration is u 2 /2γ ,with γ>0. Migration services are competitively supplied, so the price of migration (the amount that a migrant pays in order to switch sectors) is |u|/γ . A higher value of γ means that adjustment costs are lower: there is less friction. An agent who decides to migrate pays the migration cost in the current period, in order to be in a different sector at the next period.

The present value of the benefit minus the cost of migrating to Manufacture (u > 0) or to Agriculture (u < 0) equals the present value of the wage differential minus the cost of migration:

n(L, u) ≡ 1 γ (βγ (a + b(L + u)) -u).
( 1 )

For bγβ < 1, the net benefit of migration decreases with the number of other agents migrating. In this case, actions are strategic substitutes and the equilibrium is unique. Agents play a coordination game (i.e. actions are strategic complements), and there are multiple equilibria, if and only if bγβ > 1, as we hereafter assume.

Figure 2 graphs the migration constraints 0 ≤ L + u ≤ 1 (the dotted lines) and the solution to n = 0 for bγβ > 1 (the solid line). The L coordinates of the points of intersection between the graph of n = 0 and the migration constraints (the heavy dots in figure 2) define the interval [1 -(a + b)βγ , -βaγ ]. This interval corresponds to the interval C in figure 1, panel (b1). The ROM consists of the intersection of this interval and the set of feasible initial conditions, [0, 1]. For an initial condition (a value of L) in the ROM, the value of u that satisfies n(L, u) = 0 (i.e., a point on the solid line) is an unstable equilibrium.1 At initial conditions inside the ROM there are two stable equilibria: all labor moves to Manufacturing or to Agriculture. For example, at u = 1 -L for L ∈ ROM, the benefit of moving to Manufacturing exceeds the cost (n(L, 1 -L) > 0) so it is a stable equilibrium for all labor to move to Manufacturing. The length of the ROM is

Length of ROM = max{0, min{1, -βaγ }) -max{0, 1 -(a + b)βγ }}.
(2)

For initial conditions to the left of the ROM, the unique equilibrium sends all labor to Agriculture. For initial conditions to the right of the ROM, the unique equilibrium sends all labor to Manufacturing. The existence of multiplicity requires a combination of patience, a large externality, and low adjustment costs (large values of β, b, γ ), just as in [START_REF] Krugman | History versus expectations[END_REF]. An increase in any of these factors increases the length of the ROM when this is positive and less than 1. Thus, lower adjustment cost makes multiplicity ' more likely' in both senses described in the Introduction.

For this model we do not need any assumptions about the measure used to assess the likelihood of multiplicity; that is, the statement that multiplicity becomes more likely is independent of the priors on the initial condition and on b and β. For example, an increase in γ can cause initial conditions to enter the ROM but never cause initial conditions to leave the ROM. Here, there is no loss in generality in using the length of the ROM to measure the likelihood of multiplicity. This measure corresponds to a uniform prior over initial conditions.

The three-period, two-state model

This section introduces the model that we use to establish the non-monotonicity of the ROM with respect to labor adjustment costs. Current migration affects the labor allocation in the next period, and current pollution affects the pollution stock in the next period. The period index is t ∈{0, 1, 2}. The predetermined stock of pollution in period t equals K t and the flow of pollution in period t is GL t ,with G > 0. Each unit of labor in Manufacturing in the current period adds G units to the next-period pollution stock. The stock of pollution in t + 1is

K t+1 = K t + G(L t -K t ) ⇒ K t+1 -K t = G(L t -K t ).
( 3 ) Each unit of this stock reduces productivity in Agriculture by b units. The pollution stock at t is predetermined, so the Agriculture wage at t is α A -bK t . The Manufacturing wage is fixed at α M , so the Manufacturing-Agriculture wage differential is α A -α M -bK t = a -bK t .

In order to focus on our research question, we hereafter set a = 0.5 and b = 1, so the wage differential is 0.5 -K t . With this choice of parameters, the wage differential equals 0 at K = 0.5. Using equation (3), the pollution stock is constant at K = 0.5 only for L = 0.5. Thus, K = L = 0.5 is the only interior steady state for this linear model. Just as in the one-state model, this interior equilibrium is unstable. To further simplify notation, we define the state variables as deviations from these values: l t ≡ L t -0.5 and k t ≡ K t -0.5. By construction, k t equals the Manufacturing-Agricultural wage differential in period t. The state space for the model is the square

-0.5 ≤ l t ≤ 0.5, -0.5 ≤ k t ≤ 0.5, (4)
and the equation of motion for the transformed pollution stock is

k t+1 = k t + G(l t -k t ). ( 5
)
Migration decisions are made at t = 0 and at t = 1. Agents base their migration decisions on their beliefs about wages in periods t = 1 and t = 2 and on migration costs in periods t = 0 and t = 1. Denote the amount of migration to Manufacturing in period 0 as u (as in the one-period model) and the amount of migration to Manufacturing in period 1 as v. Negative values mean that migration is into Agriculture. The social cost of migration is quadratic in migration, and the price that a migrant pays is | u |/γ in period 0 and | v |/γ in period 1. In the initial period, (t = 0), the state variables L 0 , K 0 are given.

We adopt the following parameter restriction:

0 < G ≤ 1. ( 6
)
This restriction and equation (3) imply that the next-period pollution stock is a convex combination of the current pollution stock and the current flow. Because our objective is to demonstrate and explain a counter-intuitive result, it is sufficient to show that the result holds for reasonable -not for all -parameterizations.2 Section 3.2 notes that inequality (6) has a physical interpretation. If we think of the discrete time model as an approximation to a continuous time model, the value G = 1 corresponds to the fastest possible speed of adjustment of the pollution stock. The special case G = 1 also leads to particularly sharp analytic results.

Results

We first state our principal result. The next section contrasts the comparative statics of the ROM, with respect to γ , in the one-state and the two-state models. A final section offers a different perspective, sketching a continuous time model. Our principal results use the following definitions:

X(k 0 ) = - φ χ k 0 -0.5 γβ 2 G -1 χ Y(k 0 ) = - φ χ k 0 + 0.5 γβ 2 G -1 χ , ( 7 )
where

χ = γβG(γ Gβ + 3 + β -βG -G) + 1 φ = γβ(γβG + 2.0 -1.0γβG 2 -3.0G + G 2 -2.0βG + β + βG 2 ).
The following Proposition summarizes our main results; the appendix contains the proof.

Proposition 1. Suppose that inequality (6) holds in our three-period model.

(i) β 2 Gγ > 1 is necessary and sufficient for the ROM to have positive measure. Thus, an increase in γ (i.e., a decrease in friction) increases the range of other parameter values (β and G) for which the ROM has positive measure. (ii) The ROM is defined by the following set:

ROM = {(k, l) : -.5 ≤ k ≤ .5 ∩ -.5 ≤ l ≤ .5 ∩ Y(k) ≤ l ≤ X(k)}. (iii) For β 2 Gγ > 1, the area of the ROM is non-monotonic in γ .
The condition for the ROM to have positive measure, β 2 Gγ > 1, is essentially the same as for the one-state variable model (with G playing a role analogous to b) except that here the condition involves β 2 rather than β. This difference arises because in the two-state variable model migration in period t affects the wage differential in period t + 2 rather than in period t + 1 (as was the case in the two-period model). The lag occurs because here the pollution stock mediates the effect of a change in labor allocation. In the one-state variable model, which describes a flow pollutant, the change in labor allocation has an immediate effect on the wage differential. Figure 3 shows an example of the ROM. (Compare to figure 1, panel (b2).) The lower line is the graph of X and the upper line is the graph of Y, defined in equation ( 7). The area between these two lines is the ROM. Restriction (6) and β 2 Gγ > 1 imply that χ > 0. This fact, equation ( 7), and the assumption that γβ 2 G > 1 imply that X < Y. For any k 0 , the vertical distance between Y and X is

M ≡ Y -X = γβ 2 G -1 χ > 0. ( 8
)
In general, we do not have a simple closed form expression for the area of the ROM. However, if Y(0.5) < -0.5 or X(0.5) > -0.5, then the ROM does not include the NW or SE corners of state space, and the ROM is a parallelogram. If X(0.5) > -0.5, the area of the ROM is simply M.3 Interpretation 2 states that a parameter change increases the measure of the ROM. In general, checking this condition requires that we define the 'measure', i.e. a probability distribution for the set of initial conditions. For example, if the initial conditions are uniformly distributed over the unit square (the state space), then an increase in the area of the ROM unambiguously raises the likelihood that an arbitrarily chosen initial condition lies in the ROM. However, for a non-uniform distribution, an increase in the area of the ROM does not, in general, imply that it is likely' that an initial condition lies in the ROM. For general parameter values, a change in γ causes the ROM to rotate as its area changes. Therefore, for two values of γ , γ 1 = γ 2 the inequality area(ROM(γ 1 )) > area(ROM(γ 2 ) does not imply that ROM(γ 2 ) ⊂ ROM(γ 1 ). Consequently, unless the distribution of initial conditions is uniform over state space, the fact that area(ROM(γ 1 )) > area(ROM(γ 2 ) does not necessarily mean that multiplicity is more likely under γ 1 than under γ 2 .

In contrast, for the special case G = 1, equation ( 5) implies that the wage differential in the next period does not depend on the current wage differential (the current value of k). When G = 1 the ROM is flat; its boundaries are independent of k 0 .For G = 1, the magnitude of γ affects only the vertical distance between the two lines in figure 3. In this case, area(ROM(γ 1 )) > area(ROM(γ 2 ) does imply that Rom(γ 2 ) ⊂ Rom(γ 1 ). Thus, for G = 1, a larger area of the ROM means that multiplicity is 'more likely' , regardless of the distribution of the initial condition.

Comparison of one-state and two-state models

We noted that the presence of a second state variable (the stock of pollution) changes the relation between the amount of friction in labor adjustment and the likelihood of multiplicity of equilibria, as captured by the measure of the ROM. There are two reasons why the case G = 1 is particularly useful for illustrating this difference. First, as noted above, in this situation there is no loss in generality in using a uniform prior for initial conditions. Second, the case G = 1 is of special interest because it leads to a model that appears to approximate the one-state variable model. Recall that G = 1 corresponds to complete adjustment of the pollution stock, following changes in L, within a single period. That is, when G = 1 the stock of pollution adjusts very rapidly -precisely the situation where we might expect that a one-dimensional model provides a good approximation to the two-dimensional model. When G = 1 the area of the ROM is max{0, M}, with M = γβ 2 -1/γ 2 β 2 -1. Thus, γ > β -2 is necessary and sufficient for the ROM to have positive measure when G = 1. For β = 1, M = 1/γ + 1, which is strictly decreasing in γ . For β < 1, M is first increasing in γ (in the neighborhood β -2 ) and then decreasing. The measure reaches its maximum at γ m ≡ 1/2β 2 (2 + 2 1 -β 2 ) and thereafter decreases. The maximum point γ m converges to β -2 as β → 1.

Figure 4 shows the graph of γ m (the solid curve) and of β -2 (the dotted curve). For combinations of (β, γ) below the dotted curve, the measure of the ROM is 0; for a point between the two curves, the measure of the ROM is increasing in γ , holding β fixed; and for points above the solid curve, the measure is decreasing in γ , holding β fixed.

In presenting the model with two state variables, we set b = 1 and a =-0.5. Therefore, to compare the two models we need to use these values in the model with one state variable. Here we also set β = 1, so that the necessary and sufficient condition for a positive measure of the ROM is γ>1 in both models. With these restrictions, figure 5 graphs the measure of the ROM in the two models, for γ>1. This figure uses equations ( 2) and ( 8). The figure shows that the friction parameter γ has the opposite effect on the measure of the ROM in the two models.

A different perspective

Here we sketch a continuous time model to make three points. First, it provides a physical interpretation of restriction (6). Second, it shows that our discrete time model can be viewed as an approximation of a continuous time model. Third, the idea of two state variables adjusting at different speeds is transparent in the continuous time setting, but not in the discrete time setting. The online appendix describes a more general model.

The continuous time analog of equation ( 3) is

K = g(L t -K t ). ( 9
)
If we hold L t in equation ( 9) constant for one unit of time and solve that equation, to rewrite it in the same form as equation ( 3), we see that the relation between the two parameters is G = 1e -g . Because g ≤ ∞, the restriction G ≤ 1 is reasonable. Of course G > 0 because the pollution flow affects the stock. Define a 'provisional' steady state of pollution as the steady state corresponding to a fixed value of labor allocation. For example, consider two economies. The first has been largely rural, with only 5 per cent of its labor force in Manufacturing (L = 0.05), and the second has been industrialized, with 95 per cent of labor in the Manufacturing sector (L = 0.95). If the two economies have been at their respective stages for a long period of time, then in both economies the respective pollution stocks will be close to their 'provisional' steady states: K = 0.05 for the first economy and K = 0.95 for the second. These are the values of K that solve K = 0 in equation ( 9). The relatively low level of manufacturing in the first economy corresponds to a relatively cleaner environment.

In the continuous time setting, adjustment to a provisional steady state can be made arbitrarily rapid, by increasing the speed of adjustment parameter g in equation ( 9). As g → ∞, the pollution stock adjusts instantaneously. For large g, the pollution stock never deviates far from the pollution flow. Therefore, for large g a flow pollutant model approximates a stock pollutant pollutant model. In contrast, in the discrete time setting with our timing convention, adjustment of pollution to a provisional steady state takes at least one period, i.e. it always occurs with a lag. The two-state discrete time model cannot exactly reproduce the one-state model.

Define u as the flow of labor into Manufacturing (for u > 0) or into Agriculture (for u < 0), so dL/dt = u. The flow cost of migration is u 2 t /2γ , so a migrant at time t pays the migration cost |u t |/γ . As in Krugman's one-state model, γ is inversely related to friction; γ is a speed-of-adjustment parameter for labor. When g and γ differ by orders of magnitude, the time scales over which the two state variables change are different: there are 'slow-fast dynamics'. Equation ( 9) can be rewritten as

K g = L t -K t . ( 10
)
In the limit, as g → ∞, equation (10) implies that K t = L t , which implies that the Manufacturing-Agricultural wage differential in period t is L t -0.5. This formula for the wage differential is the same as in Krugman's setting, using a = -0.5 and b = 1. (We adopt these two parameter restrictions only to simplify the exposition.) More generally, when g is large (for fixed γ ), K adjusts rapidly, relative to the speed of adjustment of L. Here, the pollution stock closely tracks L: K ≈ L for g large. Thus, it would appear that the one-state model should provide a good approximation to the two-state model when the pollution stock adjusts rapidly. However, the two models have different implications concerning Interpretation 2. Provided that g < ∞, by making γ large migration becomes cheap enough that a worker can be sure of being in the high wage sector 'most of the time', regardless of other workers' decisions. In this case, a worker bases her migration decision primarily on the state variables, not on her beliefs about what other workers will do. Multiplicity is still possible, but it occurs for a small set of initial conditions: it is unlikely.

When g is finite, current migration affects the second time-derivative of the wage differential,

d 2 (K -0.5) dt 2 = g(u -g(L -K)).
In contrast, in Krugman's model (g =∞), current migration affects the first timederivative of the wage differential. The presence of the second state variable mediates migration's effect on the wage differential. In a discrete time setting, we saw that the presence of the second state variable causes migration to affect the wage differential with alag.

Conclusion and discussion

Many dynamic models can be viewed as extensions of static models, obtained by introducing a payoff-relevant state variable that adjusts over time. There are usually many ways of making a model dynamic. If we think that the friction in the adjustment of the mobile factor is the single most important source of dynamics, it is essential to include the labor allocation as a state variable. The additional complexity that comes from including a second state variable is a powerful argument in favor of the one-state model.

In addition, we may think that the one-state model is adequate. For example, we might agree that the cross-sectoral externality is not literally associated with the pollution flow, and that instead it is associated with a pollution stock. However, if the pollution stock tracks the flow very closely, it might appear that there would be little loss of economic insight in replacing the pollution stock with the flow, thereby using a one-state model. In a standard optimal control setting (i.e., a game against nature), when the speed of adjustment of different state variables differs by orders of magnitude, a higher dimensional problem can often be well-approximated using a lower dimensional state space. Where technology is convex, a competitive equilibrium can be obtained by solving a planner's optimization problem.

However, in nonconvex settings, an equilibrium problem involving agents with rational expectations can give rise to multiple equilibria. In this situation, it is typically not possible to obtain the equilibrium by solving a planner's problem. Here, an apparently innocuous reduced-dimensional 'approximation' of a higher dimensional state space might change important features of the problem. We used a simple model with two state variables to illustrate and explain this possibility.

When we ask how an increase in labor adjustment costs affects the likelihood of multiplicity, we might have in mind two different relations. Friction affects the measure of the set of (other) parameter values under which multiplicity is a possibility, and it affects the measure of the set of initial conditions for which multiplicity actually occurs. In Krugman's model and in our two-period simplification, the answer to the comparative statics question is the same, regardless of which of these two interpretations we have in mind. The theoretical and empirical literature on indeterminacy emphasizes the first interpretation. There is little discussion in the literature involving the second interpretation, which is arguably as important as the first.

We showed that when the labor allocation affects the wage differential with a lag, either because of an environmental stock externality or due to a stock of knowledge arising from learning-by-doing, then the answer to the comparative statics question may differ, depending on which interpretation one adopts. Under the first interpretation, a decrease in friction always makes multiplicity more likely in our model, just as in the onestate setting. Under the second interpretation, the relation is non-monotonic, unlike in the one-state setting.

When it is harder for an agent to take an action, such as moving to a new sector, it seems that there would be fewer initial conditions under which her decision would depend on beliefs about what other agents will do. This conclusion is (by now) so well-established that it seems obvious. However, when the current wage differential is sluggish, as occurs in our two-state model, the relation is reversed for low cost of labor adjustment. With low adjustment costs, for most initial states the agent's equilibrium migration decision does not depend on beliefs about what others will do; whatever their actions, it is cheap for an agent to move in future periods in order to remain in the high wage sector. That is, for most initial conditions, agents have a dominant strategy when adjustment costs are low.

This theoretical point shows the danger of drawing conclusions about the importance of multiplicity based on estimates of structural parameters of the model (i.e., based on Interpretation 1). Parameter estimates might suggest, for example, that there are significant cross-sectoral negative externalities or within-sectoral increasing returns to scale (or some other source of non-convexity); that factor adjustment costs are very low; and that a one-state variable model apparently provides a good approximation to the economy (because other state variables adjust rapidly). The conventional wisdom is that in these circumstances the ROM is likely to have positive measure. Our results agree with this conclusion, but also suggest that the measure of the ROM is likely to be very small, and therefore the economy is unlikely to have multiple equilibria.

An economy that has multiple steady states but a unique equilibrium might evolve in very different ways, depending on the initial condition. For example, an exogenous change in pollution control technology or environmental policy can change incentives about where to invest. Those kinds of exogenous changes can alter an interior steady state, leading to standard comparative statics results. If the changes in pollution control or policy are small, we expect the change in the steady state to also be small. However, those exogenous changes might shift the basins of attraction. If the shift causes a predetermined state variable to now reside in a different basin of attraction, the small exogenous change can cause the system to approach a different steady state. In that case, the exogenous change causes a large change in the equilibrium trajectory, even though each of the steady states has changed by a small amount.

Matters are even more complicated if there are multiple steady states and also multiple equilibria, i.e. if the ROM has positive measure. Here, where the outcome depends on agents' beliefs as well as economic fundamentals, the standard application of comparative statics might tell us very little. We would have to know how the exogenous change alters agents' beliefs.

Figure 1 .

 1 Figure 1. Phase space for different models. (a1) one-state, unique equilibrium; (a2) two-state, unique equilibrium; (b1) one-state, multiple equilibria; (b2) two-state, multiple equilibria.

Figure 2 .

 2 Figure 2. The ROM in a one-period model with βγ G > 1. The heavy dots, given by the L coordinates of the solid and the dotted lines, show the boundary of the ROM.

Figure 3 .

 3 Figure 3. The ROM (area between lines) for β = 0.8, G = 0.5, γ = 2.5.

Figure 4 .

 4 Figure 4. For G = 1, measure of ROM is 0 below the dotted curve, increasing in γ between curves, and decreasing in γ above solid curve.

Figure 5 .

 5 Figure 5. Meausure of ROM, with β = G = b = 1. The solid curve shows that for the model with one state variable, the measure increases with γ . The dotted curve shows that for the model with two state variables, the measure decreases with γ .

We use the standard notion of stability. At an interior equilibrium, n = 0 and dn/du > 0. If a small measure of agents 'deviate' (e.g., they migrate to Manufacturing when their equilibrium action is to remain in Agriculture), then other agents would want to follow the deviation. The interior equilibrium is therefore unstable.

If we did not impose the upper limit in equation (6), our proof of Proposition 1 would require distinguishing between the two cases where G ≷ 1 + √ γβ + 1/γβ. The restriction G ≤ 1 means that we need consider only one of these two cases.

If Y(0.5) < -0.5, the measure of the ROM is the horizontal rather than the vertical distance between the lines X and Y.If the ROM includes the corner of state space (i.e., if X(0.5) < -0.5 < Y(0.5)), then in computing its area we need to account for the 'missing triangles' at the corners, and the formula for the measure becomes more complicated. It is easy to confirm that any of these three configurations are possible, depending on parameter values.

The agent who migrates in period 0 avoids paying the period 1 migration costs. If migration in period 1 is at an interior level, period 1 migration costs equal the present value of the wage differential in period 2.

Acknowledgements. We thank Tasos Xepapadeas and two anonymous referees for comments on an earlier draft. The usual disclaimer applies.

Supplementary material. The supplementary material for this article can be found at https://doi.org/10. 1017/S1355770X19000147.

Appendix A: Proof of Proposition 1

We construct the equilibrium by working backwards, beginning with the agents' problem in period 1, the last period during which they can migrate.

Using equation ( 5)and l 1 = l 0 + u, we write the present value at t = 1ofbeingin Manufacturing in period 2, as

Our timing conventions imply that this value is predetermined at period 1. The equilibrium for the subgame beginning in period 1 is therefore unique. Agents are indifferent between migrating and staying in their current sector if and only if βk 2 = v/γ , i.e. if v = γ f (u; l 0 , k 0 ). The speed of adjustment parameter affects the magnitude but not the sign of the quantity γ f (•),andf (•) is increasing in u for all G > 0.

Taking into account the labor supply constraint, the equilibrium value of v is

Figure A6 shows an example of the graph of v(u), given particular values l 0 = 0 and k 0 > 0.

The u coordinate of the left and the right kink in this graph are, respectively left kink:

For all l 0 and k 0 , it is always the case that p < q. Inequality (6) implies that ρ is a decreasing function of l 0 , so p and q are decreasing functions of l 0 -a fact that we use below.

Using these definitions and equation (A1) implies

Thus, an increase in u increases the equilibrium v, provided that v is interior. In contrast, an increase in u decreases the equilibrium v when this variable is on the boundary of the labor supply constraint, as figure A6 illustrates.

In period 1 an agent is either indifferent between migrating and staying in her current sector (at an interior equilibrium) or she strictly prefers to migrate (at a boundary equilibrium). Agents with rational expectations understand this fact in period 0. Therefore, the benefit of migrating to Manufacturing in period 0 is the present value of the wage differential in period 1 (βk 1 ), plus the present value of migration costs in period 1 (βv(u)/γ ). 4 The present value of migrating to Manufacturing in period 0 is therefore

If the value of this expression is negative, its absolute value is the value of migrating to Agriculture. For u > 0, the cost of moving to Manufacturing in period 0 is u/γ ; for u < 0, the cost of moving to Agriculture is -u/γ . Define the difference between benefits and costs of moving to Manufacturing in the first period as

(Again, if h < 0, then -h is the value of moving to Agriculture.) Using equation (A2), we have

Period 0 actions are always strategic substitutes for u < p and for u > q. For q < u < p, period 0 actions are strategic complements if and only if β 2 Gγ > 1. When actions are strategic substitutes (for all values of the state variable) the equilibrium is generically unique; β 2 Gγ > 1 is therefore necessary for the ROM to have positive measure, as part (i) of the Proposition states.

Since we are interested in the measure of the ROM as a function of γ , we hereafter assume that β 2 Gγ > 1. Given this condition, we want to characterize the ROM, i.e. the region of the (k, l) plane such that if (k 0 , l 0 ) is in this region, there are multiple equilibria in period 0.

An interior equilibrium requires that h = 0 and a stable interior equilibrium requires in addition that dh/du < 0, evaluated at the equilibrium. (See footnote 1.) Since we are interested only in stable equilibria, equation (A4) means that we can rule out the possibility of interior equilibria where p < u < q. We are left with three possibilities: (i) The equilibrium is interior with 0.5l 0 > u > q, (ii) The equilibrium is interior with -0.5l 0 < u < p, and (iii) The equilibrium is on the boundary, i.e. u = -0.5l 0 or u = 0.5l 0 .

In order to construct the equilibrium, we determine the values of u for which h(u; l 0 , k 0 ) = 0 at a stable equilibrium. We first consider the case where u ≥ q; here, by equation (A1), v = 0.5l 0u. We substitute v = 0.5l 0u into the function h(•) defined in equation (A3), and solve h(•) = 0 to obtain an expression for u as a function of l 0 , k 0 . Denote this function as x(l 0 , k 0 ). Next, we consider the case u ≤ p, where v = -0.5l 0u. We use this relation in the equation h(•) = 0 and solve for u to obtain a function that we denote as y(l 0 , k 0 ). These functions x(•) and y(•) are

using the definition

With this notation, we write the equilibrium correspondence:

The first line states that if x ≥ q, then a stable equilibrium is u = x, provided that this value is less than the upper limit of migration, 0.5l 0 ; otherwise the labor supply constraint is binding, and all labor moves to Manufacturing. The second line has a similar interpretation. Thus, there are two equilibria if the initial condition satisfies both q ≤ x and y ≤ p. Using previous definitions, these two inequalities can be rewritten as

This inequality defines the ROM.

Figure A7 shows the graph of the equilibrium migration correspondence for k 0 = 0.2, G = 0.7, β = 0.8, and γ = 5.

The top kinked line is the graph of min{x, .5l 0 } over the interval where x ≥ q. The kink occurs where x = .5l 0 . The top straight line is the graph of q. The bottom kinked line and the bottom straight line are the graphs of max{y, -.5l 0 } and of p, respectively. The overlap of the two kinked lines defines the ROM, given k 0 = 0.2. If, for example, l 0 = -0.05, the two equilibrium values of migration are u = -0.144 (a movement to Agriculture) and u = 0.3 (a movement to Manufacturing).

Our assumptions β 2 Gγ > 1 and β ≤ 1 imply that Gγ > 1, so the slope x and y (as functions of l 0 ) are always positive, as shown. We noted above that inequality (6) implies that the slope of p and q (graphed as functions of l 0 ) is negative. Therefore, if x ≥ p is satisfied, it holds for large values of l 0 ; if y ≤ q is satisfied, it holds for small l 0 .

The boundaries of the overlap are determined by the solution to x = q and y = p. Denote X(k 0 ) as the value of l 0 that satisfies x = q, and denote Y(k 0 ) as the value of l 0 that satisfies y = p. Some calculation yields the formulae in equation ( 7) of the text. This step establishes part (ii) of the Proposition. The vertical distance between the boundaries of the ROM is M, defined in equation ( 8). We noted in the text that inequality (6) and the assumption γβ 2 G > 1 imply that χ > 0. Therefore, when these two inequalities hold, the ROM has positive measure. This fact establishes sufficiency in part (i) of the Proposition. The denominator of M is quadratic in γ and the numerator is linear, so M → 0 as γ → ∞. Thus, the measure of the ROM approaches 0 as γ → ∞. Since the measure is 0 for γβ 2 G < 1, positive for γβ 2 G > 1 and approaches 0 as γ → ∞, it is non-monotonic in γ , as part (iii) of the Proposition states.


Kiyu Nishida
email: nishida.kiyu.621@s.kyushu-u.ac.jp

Akishige Yuguchi

Kazuhiro Jo
email: jo@design.kyushu-u.ac.jp

Paul Modler
email: pmodler@hfg-karlsruhe.de

Markus Noisternig
email: markus.noisternig@ircam.fr

Border: A Live Performance Based on Web AR and a Gesture-Controlled Virtual Instrument

Keywords: Web AR, Gesture Control, Collaborative Media CCS Concepts, Human-centered computing → Mixed / augmented reality, Web-based interaction, •Applied computing

Recent technological advances, such as increased CPU/GPU processing speed, along with the miniaturization of devices and sensors, have created new possibilities for integrating immersive technologies in music and performance art. Virtual and Augmented Reality (VR/AR) have become increasingly interesting as mobile device platforms, such as up-to-date smartphones, with necessary CPU resources entered the consumer market. In combination with recent web technologies, any mobile device can simply connect with a browser to a local server to access the latest technology. The web platform also eases the integration of collaborative situated media in participatory artwork. In this paper, we present the interactive music improvisation piece 'Border,' premiered in 2018 at the Beyond Festival at the Center for Art and Media Karlsruhe (ZKM). This piece explores the interaction between a performer and the audience using web-based applications -including AR, real-time 3D audio/video streaming, advanced web audio, and gesturecontrolled virtual instruments -on smart mobile devices.

INTRODUCTION

In recent years, the number of live performances experimenting with media-based technologies has rapidly increased, raising many questions regarding musical expression and aesthetics [START_REF] Oh | Audience-participation techniques based on social mobile computing[END_REF]. Collaborative situated and interactive media, for example, questions the relationship of a performer with the audience, as the audience members change their roles from passive listeners to active participants [START_REF] Freeman | Large audience participation, technology, and orchestral performance[END_REF][START_REF] Jo | ) performing technology: User content and the new digital media[END_REF].

A wide range of different ways to achieve interactively and collaborated live performances have been proposed [START_REF] Hindle | Swarmed: Captive portals, mobile devices, and audience participation in multi-user music performance[END_REF][START_REF] Zeitbloom | [END_REF]. Some of them are based on intensive research and development focusing on various technical, scenographic, musical and overall aesthetic aspects. The next paradigm shift in this area involves mobile devices connected to the webfrom simple sensors to smartphones and wearables -that create a much more connected environment and provide users with easy access to multi-media content [START_REF] Turchet | Internet of musical things: Vision and challenges[END_REF].

Recent technological advances, such as increased CPU and GPU processing speed, along with the miniaturization of devices and sensors, have changed the way mobile devices are used in interactive and participatory artwork. Mobile devices not only receive media streams from servers, but also help to integrate local environmental information through motion sensors, cameras, and microphones. This enables Augmented Reality (AR) on mobile devices, which, in contrast to Virtual Reality (VR), layers computergenerated content on top of the existing reality. Using motion and gesture tracking provides natural human-computer interaction methods and enables embodied generative music and sound control. With such a system, the listener becomes a performer playing the virtual world such as a 'virtual instrument' [START_REF] Egozy | 12*: Mobile phone-based audience participation in a chamber music performance[END_REF], and AR is expected to enhance audience participation in a concert situation [START_REF] Mazzanti | Augmented stage for participatory performances[END_REF].

One of the main remaining challenges in AR is to provide the listener with a consistent immersive audio experience. Higher-order Ambisonics (HOA), for example, is a spatial audio format that is widely used for VR/AR applications. It can be decoded over both loudspeakers [START_REF] Zotter | Energy-preserving ambisonic decoding[END_REF] and headphones [START_REF] Daniel | 3d binaural sound reproduction using a virtual ambisonic approach[END_REF][START_REF] Noisternig | 3d binaural sound reproduction using a virtual ambisonic approach[END_REF] and combines good sound localization with a high level of immersion. Up-to-date software tools for spatial audio production provide advanced 3D real-time audio processing methods for interactive sound source control and room effect simulation in VR/AR environments [START_REF] Carpentier | Panoramix: 3d mixing and post-production workstation[END_REF][START_REF] Carpentier | Twenty years of ircam spat: looking back, looking forward[END_REF].

In combination with web technologies, such as HTML5 1 and WebGL 2 , any mobile device with a browser can simply connect to a server without needing to download and install an app prior to the performance. Web-based applications are easy to maintain and offer seamless functionality on different platforms and operating systems (although not all web browsers support VR/AR content).

MOTIVATION

Several artworks have explored ways to integrate 3D virtual content into live performances providing new experiences for both audience members and performers. The collaborative mixed reality environment for Reflets [START_REF] Berthaut | Reflets: Combining and revealing spaces for musical performances[END_REF], for example, displays virtual objects onto semi-transparent panels placed between the audience and the performer. Con i piedi per terra [START_REF] Mazzanti | Augmented stage for participatory performances[END_REF] is another example of a participatory performance that uses AR. A specific application was developed for this performance that provides interactive AR content on smartphones and enables audience members to contribute to the performance. Our piece, 'Border,' on the contrary, gives a single performer the ability to add multiple layers of immersive audiovisual content to a live music performance using AR and interactive technologies. This is achieved by extending the timeline of the performance through live looping. The performer recorded his/her movements over a certain time interval, and the recordings were then continuously repeated in the AR environment. Each 'instance' of the performer (i.e. each of the recorded video clips) was then displayed at a certain position in the AR environment. The audience members could access the AR video stream by connecting their smart devices to a web-based application. Figure 1 shows the AR view on a smartphone. To enable the performer to control sound and music in this AR environment, we developed a gesture-controlled virtual instrument, which uses Kinect body motion tracking.

live loop recorded video

ARCHITECTURE AND DESIGN

In this section, we describe the web-based application, including the AR and audience interaction, and the gesturecontrolled virtual instrument.

Web-Based Application

We decided to design a web-based application so that a performer could communicate and interact with the audience members. The web application was built on the crossplatform JavaScript run-time environment Node.js3 . To maintain high bandwidth access and good network stability, an isolated local area network (LAN) was installed using a central Wi-Fi router. Audience members could access the web-based application by connecting their mobile devices to the wireless LAN and then opening the web address indicated on the screen before the start of the performance.

Augmented Reality Environment

In our piece, the web-based AR environment was implemented using the cross-platform JavaScript library AR.js 4which runs on every mobile device browser supporting We-bGL and WebRTC5 . AR.js works with specific visual markers (i.e. a sort of simplified QR-code) to insert specific 3D scenes in the video. Figure 2 shows the processing chart of our AR system for inserting the live performer into the AR stream on a mobile device. The Kinect sensor provides video and depth information. Using the obtained depth information, the performer's body shape area is extracted from the video stream, and the background of this layer mask is filled with solid white in Processing6 . This live video stream is then sent through Max7 to CamTwist8 . The video from CamTwist is recognized as a web camera source by a web browser on the server and is transmitted in real time using WebRTC. At the same time, the live video stream was recorded for a certain time interval in Max, then stored on the server and uploaded -in the background -to the mobile devices through the web interface, where it is played back in a loop as an additional layer to already playing videos, as shown in Figure 1. Each frame of both the loaded loops and the transmitted video is copied onto an HTML5 canvas, a container element used to process graphics. Deleting every white pixel of the layer mask extracts the performer instance, which is then displayed in the AR stream.

Interaction with Audience Members

To interact with audience members, the performer sends an audio stream to a mobile device in real time. The implementation of this functionality combines tools such as WebRTC, CamTwist, Max, and the Web Audio API 9 . Audience members controlled the playback volume of the looped sounds by tilting their smartphones. This means the performer can use the mobile devices in the audience to add an additional layer of musical expression to his/her performance and the audience members become active performers, using their mobile devices as musical instruments.

Gesture-Controlled Virtual Instrument

To support various kinds of instruments and timbres with the performer's motions, a gesture-controlled virtual instrument was designed based on body tracking with a Kinect sensor. As shown in Figure 3, with this instrument, a performer can switch between two modes using simple movements, the 'selection mode' for choosing an instrument, and the 'play mode' for playing the selected instrument. Each selected virtual instrument represents a unique timbre, a sampled sound, or an audio effect. In the play mode, shown in the right sub-figure, the performer's personal space (i.e. the region of space immediately surrounding the body) is partitioned into a limited number of areas representing MIDI notes which trigger sounds each time the performer 'touches' one of these areas. Figure 4 shows the system architecture of the gesture-controlled virtual instrument. The interface of the virtual instrument was implemented in Processing and Max, and controlled through the Kinect motion tracking; Ableton Live 10 was used for audio recording and playback, and Panoramix 11 for real-time spatial audio processing. According to the gestures tracked by the Kinect, Processing detected when a MIDI key was activated and provided the associated video stream. The video was then streamed in real-time to Max using Syphon 12 and projected on a video screen on stage to help the audience understand the computer-generated AR scenes on their mobile devices. In parallel, the video stream was sent to a computer screen on stage to provide the performer with some visual feedback. All building blocks communicated via the Open Sound Control (OSC) protocol [START_REF] Wright | Open sound control: a new protocol for communicating with sound synthesizers[END_REF] and User Datagram Protocol (UDP). The MIDI notes were interpreted in Max, triggering the sounds in the digital audio workstation Ableton Live. The Ableton Live audio outputs were connected to Panoramix using Soundflower 13 , although the sound source positions are sent as OSC message bundles using ToscA 14 . In addition, an iPad was set up for extended remote control of Panoramix, Ableton Live, and Max.

Moreover, HOA was used for 3-D audio rendering in realtime in order to provide an immersive audio experience. The audio was played back over a 43-speaker dome using the 'energy preserving' decoder [START_REF] Zotter | Energy-preserving ambisonic decoding[END_REF].

LIVE PERFORMANCE

We performed 'Border' at the Beyond Festival 2018 'Future Sound' at the ZKM in Karlsruhe, Germany. Figures 5 and6 show the stage plan, without the surrounding speaker array, and a photo of the stage. The photo shows the five AR markers, each with a dimension of 1x1 square meters to guarantee that smartphones at the far end of the auditorium can recognize them. Each AR marker defines the position of a 'visual instance' of the performer. An audience of 120 attended the concert, and on average, 80 participated actively 10 Ableton live, https://www.ableton.com/en/ 11 Panoramix [START_REF] Carpentier | Panoramix: 3d mixing and post-production workstation[END_REF], https://forumnet.ircam.fr/product/ panoramix-en/ 12 Syphon, http://syphon.v002.info/ 13 Soundflower, https://soundflower.en.softonic.com/ 14 ToscA, http://forumnet.ircam.fr/product/tosca-en/ with their smartphones. At the beginning of the concert, instructions were given to the audience on how to set up their smartphones and on how to interact with the AR environment. Then, the audience held their smartphones in their hands. During the performance, the performer introduced the way to interact with a sound using the smartphone, and ways to actively participate in the show were demonstrated.

DISCUSSION

This section discusses the interactive AR environment, the gesture-controlled virtual instrument, and the experience of the audience members.

Augmented Reality

We designed our interactive AR environment as a first step in integrating a virtual world with the real world. There are still many steps to achieve this concept completely. We describe two key considerations here.

Management of Computers

Both processing a video and displaying a video as a virtual object take a large amount of CPU and GPU power.

In terms of that and the bandwidth for the communication lines, we had to use a quarter of the video resolution compared to the original resolution from the Kinect sensor, which caused a significant drop in the sense of immersion.

Limits of a web-based application

AR.js displays virtual objects on real-world video streams based on the position of an AR maker. One problem with this method is that maintaining the spatial relationship between markers is difficult, specifically, synchronizing the virtual positions with the absolute real ones. For this reason, we displayed the performer instances using the relative positions of the AR markers, which limited the expression of AR in the web browser. Extending the web browser's AR library with the ability to synthesize a virtual world in sync with the real world more fluently would allow us to create more dynamic and rich effects. In fact, this also could be achieved through a dedicated native application; however, as described above, such an application would be less useful in a concert with a heterogeneous audience.

Gesture-Controlled Virtual Instrument

In this performance, the sound was mostly controlled by gestural movements of the performer, although for certain instructions, a tablet interface was used. The performer experienced this separation of the interface into two parts as less practical for control. An alternative method could be to use an additional tracking system such as eye movements [START_REF] Hornof | The prospects for eye-controlled musical performance[END_REF] or certain hand gestures [START_REF] Han | Lessons learned in exploring the leap motion sensor for gesture-based instrument design[END_REF][START_REF] Modler | Video based recognition of hand gestures by neural networks for the control of sound and music[END_REF]. In combination with the motion tracking used in this performance, the integration of these options would improve the control, and so expand the expressiveness of the performer significantly.

The performer also said the visual feedback provided by the monitor was not easy to see because he had to look down, and it causes to lose visual communication with the audience. A solution to this problem is to use a mixed reality panel as mentioned in [START_REF] Berthaut | Reflets: Combining and revealing spaces for musical performances[END_REF]. Such a display enables the performer to get feedback and visual communication. It also enables the audience members to get more immersion, combining with AR provided by their mobile devices.

On the other hand, the latency between the moment the performer hits a key with his gesture on the virtual instrument and the actual occurrence of the sound was not perceived as disruptive, because the performance did not have a dominant beat such as a drum track with a strict tempo. However, special care must be taken in such an environment if it follows a dominant beat.

Perspective of Audience Members

Some people reported after the concert that holding up a smartphone during the performance was exhausting and fatiguing. Others reported that they watched the AR through the smartphone for only a very short period, compared to the total length of the performance. A possible solution is using smartphones on cardboard carriers. Although this may not replace a head-mounted display (e.g., Vive, HoloLens or Oculus Rift), it provides inexpensive accessible technologies for integrating a large number of audience members in immersive and participatory music and performance art, with a low physical effort and minimum fatigue.

CONCLUSIONS

This paper presents the immersive and interactive performance 'Border' which explores the performer/audience interaction using web-based applications including augmented reality, real-time 3D audio/video streaming, advanced web audio technologies, and a gesture-controlled virtual instrument. Many artists have explored the interaction between a performer and the audience, and it is obvious that interactive VR/AR changes the role of the audience members from passive listeners to active participants. 'Border' highlights how the use of web technologies and mobile devices can provide easy access to collaborative VR/AR in art performances. A video of the performance presented in this paper is available at: https://youtu.be/_4yrxCEap1M

Figure 1 :

 1 Figure 1: The AR view through the audience members' smartphones (post-edit)

Figure 2 :

 2 Figure 2: AR system processing chart

Figure 3 :Figure 4 :

 34 Figure 3: Gesture-controlled virtual instrument: selection mode (left) and play mode (right)

Figure 5 :Figure 6 :

 56 Figure 5: Stage setting (the surrounding 43-channel speaker dome is not depicted)

HTML5, https://dev.w3.org/html5/spec-LC/

WebGL, https://www.khronos.org/registry/webgl

Node.js, https://nodejs.org/en/

AR.js, https://github.com/jeromeetienne/AR.js

WebRTC, https://webrtc.org/

Processing, https://processing.org/

Max, https://cycling74.com/

CamTwist, http://camtwiststudio.com/

Web Audio API, https://developer.mozilla.org/ en-US/docs/Web/API/Web_Audio_API

ACKNOWLEDGMENTS

This project was supported in part by JSPS KAKENHI (Grant No. JP17H04772) and the Japanese Public-Private Partnership Student Study Abroad Program 'TOBITATE! Young Ambassador Program'. The authors further thank Christian Sünkeberg for IT support, the colleagues of the Media Art Department of the Karlsruhe University of Arts and Design for fruitful discussions, as well as the Center for Art and Media Karlsruhe (ZKM) and the Beyond Festival 2018 for the invitation to premiere this piece.