Victor Violier

THE MILITARIZATION THEORY IN POST-SOVIET RUSSIA: DISPELLING THE PATHOLOGICAL LOOK AT POLITICAL AND ADMINISTRATIVE ELITES

Keywords: Militarization, law, legitimity, state building, political and administrative elites, political culture

This paper aims, through the specific example of the plenipotentiary envoys À a.k.a. the polpredy, at questioning Law as a legitimate knowledge of the political elite in post-Soviet Russia. The term legitimate has to be understood as both a legitimated and a legitimating knowledge.

A new level of administration was set by Vladimir Putin right after his election in May 2000 and has become a symbol of the militarization of political elites in Russia, concretized by a massive recruitment of people from the so-called power ministries. Beyond this, in the context of a closed institutional game and the power's will to neutralize a whole bunch of the political game's rules, law also becomes a ground on which to build a control of the political and administrative elites' recruitment. Our approach blends a critical overview of the literature and a prosopographical study of more than 20 members of Russian top political elites between 2000 and 2012, corresponding to Putin's three first mandates as the head of the Russian state À two as President and a third one as Prime Minister under Dmitri Medvedev's Presidency.

Our study led us to the conclusion that, not only should we regard Law as esteemed but also, and above all, as invested with an instrumental function by the power in place, but also those who long to be in power.

THE OSCILLATIONS OF RUSSIAN FEDERALISM

The evolution of Russian federalism since the end of the USSR may be compared to that of a pendulum (Gel'man, 2006, p. 104). During the 1990s and the Presidency of Boris Yeltsin À from December 25, 1991 to December 31, 1999 À the power of the different "subjects of the Federation," as the various entities of the Federation are called, continually expanded while the Central power became weaker and weaker. To maintain his power, Yeltsin concluded agreements with the local barons and thus delegated large elements of the state sovereignty to the provinces. This process was particularly accurate concerning the Republics within the Russian Federation. The central power even started to sign bilateral treaties with more and more subjects of the Federation. It started, for instance, with The Republic of Tatarstan on February 15, 1994 (Gazier, 2002, pp. 194À195;Lallemand, 2010, p. 130). In this context, a large number of subjects felt free to adopt local laws, rules, regulations, and policies that were not in line with national laws. Some of those laws were even unconstitutional, regarding the Constitution of the Russian Federation adopted on the December 25, 1993. Therefore, a "transactional federalism" (Gel'man,op. cit.,p. 107) or "Federalism on a-case-by-case basis" was born. This led to the conclusion that "During the 1990's, Russia had as many regional political regimes as provinces" (Lallemand,op. cit.,p. 131). This asymmetry (Dauce´, 2008, p. 46;Lallemand, 2010, p. 130) reached its peak with the financial and economic crisis that occurred in August 1998. The regional executives had become extremely powerful and the obvious inability of the central power to deal with the multifaceted crisis À that was symbolized by the collapse of the rouble which resulted in the Russian government and Central Bank devaluing the rouble and defaulting on the debt À raised the question of the federal power's future (Mendras, 2007, pp. 127À132). The nomination of Yevgeny Primakov as Prime Minister in September 1998 was one of the last attempts to restore central power over the provinces. Less than a year later, Yeltsin and his entourage chose Vladimir Putin as the new Prime Minister and, even more, the designated successor to become the second President of the Russian Federation (Gazier, 2002, p. 181;Sakwa, 2008, pp. 121, 123À124).

The Putin Presidency has consisted from the beginning in an authoritative recentralization of power. After being chosen to act as interim President on the evening of the January 31, 1999, Vladimir Putin's position was then confirmed by the presidential election held in March 2000. In fact, after competing as an independent, he was elected in the first round with more than 53% of the vote out of a total of 11 candidates. Since he came to power, Vladimir Putin has shown his intention to stop the movement of decentralization initiated under Yeltsin's Presidency (Gazier, 2002, p. 203). It was even illustrated while he was still Prime Minister by the invasion of the secessionist Republic of Chechnya that took place on August 26, 1999 and started the second Chechen war in response to the invasion of Dagestan by the Islamic International Brigade (Le [START_REF] Hue´rou | Un Etat en guerre. La Russie et le conflit russo-tche´tche`ne[END_REF][START_REF] Merlin | Le Nord-Caucase au miroir du fe´de´ralisme russe[END_REF].

THE ESTABLISHMENT OF THE POLPREDY, HOW THE ROT HAS SET IN …

As a newly elected President, Vladimir Putin then put forward two main concepts to symbolize the re-ordering of the State order, and, consequently, of Russian federalism. The first concept is the so-called "power vertical." It consists of implementing a straight hierarchy between all levels of administration, each one obeying to the upper one [START_REF] Gazier | Le bouleversement des institutions et de la vie politique[END_REF][START_REF] Gazier | Le Pre´sident de la Fe´de´ration de Russie et l'instauration d'une "verticale executive[END_REF][START_REF] Kossolov | Vertical administrative: Fondements et obstacles[END_REF]. The second concept is the "dictatorship of law," which comprises two lines. On the one hand, it means that all local and regional laws, rules, and policies now have to match the Constitution of the Russian Federation. On the other hand, it also means that every administration, every company or individual is submitted to the law. This second part has been almost theoretical but has constituted a convenient political weapon for President Putin. A relevant illustration of this was the so-called "Khodorkovsky case" that started after Mikhail Khodorkovsky was arrested on October 25, 2003 in Novosibirsk (Siberia) and his company forced into administration by the State. Indeed, the trial of the wealthiest man in Russia at that time À according to Forbes magazine À was a clear demonstration of the State power over individuals and local or national lobbies (Dauce´, op. cit.;[START_REF] Favarel-Garrigues | La dictature de la loi. Paris: Alternatives e´conomiques[END_REF].

As for policies, Vladimir Putin has introduced several State reforms since he took office. The creation and appointment of the so-called polpredy on the May 13, 2000 is one of them. In fact, he used the presidential decrees (Ukaz) N°849 to 855 "O polnomocˇnom predstavitele prezidenta rossijskoj federacii v federal'nom okruge" to create this new level of the state administration. The polpredy are part of the presidential administration, regularly considered the most powerful corps within the State. There were seven delegates for the entire territory at the very beginning. An eighth one was appointed to the North Caucasian Federal District which resulted from the division of the Southern District decided in 2010 by Dmitri Medvedev after he succeeded Vladimir Putin. The polpredy are supposed to control the actions of the governors and the presidents of the Republics and to ensure that local legislation complies with the State Constitution. We can add that the creation of the polpredy was complemented by additional measures tending to a recentralization of the State power, such as the suppression of direct universal suffrage for the election of the Governors and Presidents of Republics.

In a nutshell, presidential plenipotentiary envoys became the champions of the new centralism [START_REF] Gel'man | Возвращение Левиафана? Политика рецентрализации в современной России[END_REF] and the vertical of power since they were nominated to tame the regional entities emancipated during the Yeltsin years. They also serve as an example of the reshaping of the Russian political and administrative elite, from which we can observe both the configuration and evolution of Russian elites since the fall of the Soviet Union.

THE SHORT-SIGHTEDNESS OF THE MILITARIZATION THEORY

An Attractive Paradigm … What we see as the "militarization theory" consists in a widespread paradigm dominating a large amount of issues about Russian elites and revolving around the issue of a "militarization of power" (Raviot, 2008, p. 48).

Russia is considered a hybrid political regime [START_REF] Diamond | Thinking about hybrid regimes[END_REF]. Its "degree of democracy" is regularly debated, The Russian political regime being thus usually classified between the western liberal "consolidated democracy" [START_REF] Linz | Toward consolidated democracies[END_REF] and the politically "closed regimes" [START_REF] Levitsky | The rise of competitive authoritarianism[END_REF][START_REF] Schedler | The menu of manipulation[END_REF]. It is also considered as a "democracy with adjective" [START_REF] Collier | Democracy with adjectives: Conceptual innovation in comparative research[END_REF]: administrated, directed, Potemkin etc. Thus, one of the key aspects that has been widely portrayed by both scientific and media analysts during the last fifteen years would be a so-called militarization of Russian elites. A few months after Vladimir Putin came to power, what we decided to call "The militocratic paradigm" was developed. Indeed, according to most analysts, the arrival of Putin to power has entailed the appointment of an increasing number of people from military structures to positions of power. Thus, according to this analysis, the entire political regime of contemporary Russia has been affected by the massive penetration of military people. The term 'militocracy' was used for the first time À to our knowledge À by Olga Krychtanovskaya and Stephen White. In an article published in 2003 in Post-Soviet Affairs and entitled "Putin's Militocracy," they clearly insist on what they consider a major transformation in Russia since the 2000s. Their statement is doubly linked to the incoming of numerous military personnel. Firstly, they consider that Vladimir Putin has played a major role in the recruitment of his previous colleagues at every level of administration. Therefore, this penetration would be the result of a well-planned policy from the newly elected President to secure his position by reshuffling the administration main body. Secondly, this reshuffling would affect even the form of the state and the political regime of Putin's Russia. It would put an end to Russia's transition to democracy and create a military regime. As an example of this decisive conceptual shift in meaning, we can quote the title of an article from The Economist magazine published in 2007: "The Making of a New KGB State." At the core of the article the author asserts "Men from the FSB and its sister organisations control the Kremlin, the government, the media and large parts of the economy À as well as the military and security forces. (…) The KGB provided a crucial service of surveillance and suppression; it was a state within a state. Now, however, it has become the state itself" (The Economist, 2007, §4, §6). It might also be added that the edition concerned is even entitled "Putin's people. The spies who run Russia." In the same way, some researchers went so far as to warn against "a peaceful military takeover" [START_REF] Petrov | Seven faces of Putin's Russia: Federal districts as the new level of state territorial composition[END_REF] happening in Russia as soon as "the institution of the envoys is one of the major elements within the new Putin model of replacing state institutions with substitutes" (Petrov, 2005, p. 56): "A year and a half may be too short a period for definite conclusions, but one thing that is already sufficiently clear is that, whatever the Kremlin's initial plans and words, what is really happening is the insertion of police-state mechanisms into a delegative and declarative democratic state (…) Putin's aim is to create a strong state under his leadership; everything else À including de-democratization and de-federalization À is mere side-effect" (Petrov, 2002, p. 82).

Thus, the characteristics of the military people would influence their way of ruling the administrations and, therefore, the whole state. The values and attitudes that they acquired within the security forces determine their actions whilst in power. This matches with the work of [START_REF] Mills | The power elite[END_REF], who wrote in this respect, in The Power Elite, that military people did not lose the effects of both the education and training that modeled their personality and their vision of the world. Meanwhile, this perspective could also be reversed. Indeed, instead of considering only a militarization of politics, couldn't we think of a civilization À in the sense of a "politization" À of the military area? It would result in us taking into account the politization of military people and security forces as well as law enforcement organizations. We think here of Jean Joana's reflexion about the position of military personnel and their relation to power mentioning among other Alfred Stepan's writings about Brazil and the example of the Escuela superior de Guerra (1971Guerra ( , 1973)). Actually, the military might become so bogged down in politics that it would neutralize their ability to be a problem for the government. For instance, the sociology of comparative politics has been reluctant to consider "military regimes" as a category of regime in itself. Indeed, [START_REF] Linz | Totalitarian and authoritarian regimes[END_REF] has already suggested, the nature of these regimes cannot be reduced to the military origin of those who rule, since their influence tends to fade while the regime is stabilizing (Joana, 2006, pp. 7À9).

The quantitative dimension of the "militocratic paradigm" has been demonstrated. The calculations have certainly been well overestimated at the very beginning and have favored a catastrophic outlook (see [START_REF] Rivera | The Militarization of the Russian Elite Under Putin: How wide and how deep? Department Of Government[END_REF] revising data of [START_REF] Kryshtanovskaı¨a | Putin's militocracy[END_REF]. Still, it seems that this dimension is globally attested. According to the data of [START_REF] Rivera | The Militarization of the Russian Elite Under Putin: How wide and how deep? Department Of Government[END_REF] one in eight members of the Russian administrations has served in the power ministries. And this number has increased by 25% since Vladimir Putin came to power. As a consequence, the aim of our paper is not to contradict the data but to question the qualitative dimension of it. In other words, to what extent has Russia become a militocratic regime?

… But a Distorting Prism to Look at Russian Elites Beyond a normative and pathological look, a closer look at plenipotentiary envoys' career paths shows variety and diversity. Our empirical research consisted of a prosopography of all the polpredy (24) who have been in office since the creation of the function in 2000 until May 2012. The study of the biographical characteristics of these individuals is a good way of analyzing the role of the cultural factor in the process of distinction of Russian elites since Vladimir Putin came to power. In fact, they are both the personification of the "new centralism" and the "vertical of power" and an example of the reshuffling of Russian elites since the end of the Soviet era. Thus, the empirical work consisted of compiling information and data from various sources to compose, as far as possible, the most detailed biography for every individual that has been in office within this administration between 2000 and 2012. The year 2000 refers to the first election of Vladimir Putin and the creation of the function of the polpredy in May. May 2012 refers to the third election of Vladimir Putin after Dmitri Medvedev's term. The election took place in March, he then entered into office on May 7th. This period includes a total of 24 people.

We started using the biographical information we could find form the two main sources edited in Russian [START_REF] Chchegolev | Kto est' kto v Rossii. Ispolnitel'naıä vlast[END_REF][START_REF] Muhin | Federal'naıä i reguional'naıä elita Rossii À Kto est' kto v politike i ekonomike[END_REF]. Then, we updated and cross-checked these informations with both institutional and non-governmental websites À such as the official websites of the federal districts administration and independent journalists' websites À such as kompromat.ru, for instance. Concerning the nature of the two principal sources, we would like to assert that they are different from the main Anglo-Saxon reference which is the biographical dictionary entitled The Who's Who. Actually, if they both highlight identity-building strategies of the politicians, our sources are more based on a bureaucratic than "worldly" model [START_REF] Collovald | Identite´s strate´giques[END_REF]. They include all the high-level officials and bureaucrats and detail their careers in a chronological and very official way.

The analysis of our empirical materials allows us to contrast sharply with the caricatural and homogeneous vision of Russian elites given by the militocratic theory. To begin with, it is important to underline that not only the career paths but even the mandate trajectories of the polpredy have been very diversified since these representatives were nominated and dismissed on the decision of the only President of the Federation. Firstly, some of the polpredy have been nominated in two different districts over the 12-year period of time we are considering. Secondly, the duration of the 24 polpredy's term of office has varied between 6 months and more than 11 years. The average duration of the term of office has been three and a half years.

We paid attention to the recruitment and departure conditions. Indeed, the function of the polpred is unequally linked to governmental positions. Seven of them occupied a position within the government when they were nominated as a polpred. Nevertheless, they didn't have similar positions. For instance, Oleg Safonov (polpred in the Far-eastern district from 2007 to 2009), deputy Minister of the Interior (MVD) before his nomination, was far lower in governmental hierarchy than Valentina Matvyenko (polpred in the North-Western district in 2003), since she was herself deputy Prime Minister. The future of the polpredy after their mandate is also very different. In fact, only four of them have been directly nominated in the government while some started running state-controlled agencies and companies or were nominated in the regions, for instance as governors.

CenterÀregion or regionÀcenter pathways are also hard to unify. In fact, three of the 24 polpredy used to be governors of some subjects of the Federation, that is, part of the regional administration before being nominated in the Presidential administration. V. Ishaev has served as Governor of the Kraı¨of Khabarovsk from 1991 to 2009 before becoming polpred of the Far-Eastern federal district. A. Khloponin was in office in Krasnoyarsk for 8 years when he was nominated in 2010 to rule the North Caucasian federal district. Finally, V. Tolokonsky was the head of the oblast of Novosibirsk for 11 years before his nomination as the new polpred of the Siberian federal district in 2010. On the contrary, two polpredy were transferred from the presidential À central À administration to the regional administration. Thus, V. Matvyenko, the only woman who has occupied the post of polpred, became Governor of St-Petersburg after her very short term in office as the polpred of the North-Western federal district in 2003. G. Poltavchenko occupied the same function as Matvienko after he was dismissed from the post of polpred of the Central Federal District. As a consequence, it is hardly possible to think in terms of "political spoils" [START_REF] Bailey | Stratagems and spoils: A social anthropology of politics[END_REF], and establish a hierarchy of national and regional functions in which we could replace with precision the polpredy.

The most important aspect highlighted by the large diversity of the polpredy's career pathways is certainly their professional background. Indeed, it results from a detailed analysis of the various professional positions occupied by the polpredy before their coming in office that three major groups could be distinguished. This strongly undermines the militocratic theory, not just because only eight of the twenty four polpredy have been working within the power ministries but also because even these eight individuals are far from having such a common professional identity.

Therefore, the first group of individuals that can be identified is composed of eight individuals who have worked in the so-called power ministries. Thus, they are considered as being part of the so-called siloviki. Among these eight individuals that have worked in the power ministries, four have served in the KGB and/or for its successor the FSB. V. Tcherkesov is a former colonel general of the KGB and was the Deputy Director of the FSB. G. Poltavchenko is a former officer of the KGB and was the head of the Direction of the Federal tax police in Saint-Petersburg. G. Rapota worked in the First General Direction of the FSB (foreign intelligence) and was involved in long-term missions in western countries. Finally, O. Safonov, a former KGB agent, has been decorated with the Order of the Red Flag of the KGB Institute of the Soviet Union. Then, three individuals are former professional soldiers. Indeed, V. Kazantsev, K. Pulikovskiı¨, and A. Kvashnin are respectively General in Chief, Lieutenant-General, and Colonel General of the Russian army and have served in various conflicts such as the war in Afghanistan or the two Chechen conflicts. Lastly, we can add P. Latyshev, who is a former Colonel General of the Ministry of Interior of the Russian Federation (MVD).

Sixteen of the polpredy (2/3) thus have a different professional origin. Indeed, nine of the plenipotentiary representatives occupied an intellectual profession or were former senior managers in the Civil service. Among them were four former magistrates, three former University Professors, and two former senior civil servants. They consist of the second identifiable group. The third and last group is composed of five plenipotentiary representatives who came from the business sector. They were all leading banks and/or industries À such as Noris Oı¨l or Noris Nickel À during the 1990s. Finally, we could notice here that two trajectories do not match with our categorization: V. Matvienko and K. Ishakov had particular career paths that we will analyze further.

The close look we just had at the plenipotentiary representatives career path allows us to have a more nuanced picture of these individuals than the one depicted after V. Putin's coming into office and that we described before. It also invites us to consider siloviki as a polysemic notion that should be debated. Therefore, if five out of the seven first polpredy were former members of the power ministries, there were "only" eight out of 24 twelve years after the introduction of their function. Nevertheless, it still corresponds to a third of the total. Consequently, it would still be possible to consider that this new level of administration is a testimony of the "military takeover" (Petrov, op. cit.) that has occurred since V. Putin came to power in 2000, without any further examination. This is the reason why we currently need to develop our analysis on the nine key individuals known as former power ministries members, that is, siloviki.

Firstly, the notion of siloviki itself À as well as the one of power ministries À should be debated [START_REF] Bacon | Securitising Russia: The domestic politics of Vladimir Putin[END_REF]. The concept of siloviki products a homogeneous reality. It thus tends to lead to the conclusion that, for instance in the case we are interested in, the eight individuals who have worked in the power ministries are broadly similar to each other. Nevertheless, it is now time to assess this problematic notions of "silovik" and "power minister." To summarize on this issue, a relatively consensual and operational definition has already been established as follows: "The power ministries in Russia are those state agencies in which the personnel generally wear uniforms and in which some people carry guns. More precisely, those bodies are military, security or law enforcement bodies that possess armed units of formations. People with power ministry backgrounds are referred to as siloviki" (Taylor, 2007, p. vii). However, as Edwin Bacon, Julian Cooper and Bettina Renz noted: "The concept of a silovik, though an attractive shorthand term, masks wide-ranging differences between those who are labelled with this epithet. A silovik could have served in a diverse group of force structure (for examples, the military, the police, the intelligence services), in a diverse range of roles, at different level of seniority, in different periods, and for differing lengths of time. There are democratic siloviki, and authoritarian non-siloviki" (Bacon et al., 2007, pp. 42-43). We could not agree more to the three authors' statement in this case. The notion of silovik actually functions as a stereotype that smoothes out asperities and erases the differences between trajectories that can be particularly various, providing a distorted vision of the reality. As a consequence, there might be more differences than similarities between some or the totality of the individuals categorized as siloviki.

Secondly, a careful look at the recent political history of the Russian Federation might also help us to put this particular recruitment back into its context. Indeed, when Vladimir Putin was chosen as his successor by Boris Yeltsin and the so-called Family À as the Yeltsin entourage has been nicknamed À he only had a few months experience on the national political stage (Dauce´, 2008, pp. 60À62). Besides, he was not affiliated to any of the political parties existing in Russia. Therefore, he could not recruit in a political party's reserve of individuals and had no other choice but to rely on the individuals he already knew through other ways (Solovev quot. in [START_REF] Renz | Putin's militocracy? An alternative interpretation of siloviki in contemporary Russian politics[END_REF]. That is why he decided to rely on people he used to know by working with them either in the security forces or in the St-Petersburg City council where he started his political career, after the fall of the Berlin wall. We can add that when Boris Yeltsin came to power, he proceeded the same way, nominating people from his hometown, Sverdlovsk, to assume various strategic positions [START_REF] Kryshtanovskaı¨a | Putin's militocracy[END_REF]. Actually, we can use the comparison of Alekseı¨Salmin, Director of the Russian Centre of Public Policies: "Of course Putin relies on those he understands and knows. This is absolutely natural. If a professor happened to become president, he, too would invite some people from his environment" (quot. in [START_REF] Renz | Putin's militocracy? An alternative interpretation of siloviki in contemporary Russian politics[END_REF].

According to this point of view, the professional socialization of Vladimir Putin would explain the nature of his recruitment. That might be a reasonable explanation. Nevertheless, this cannot be the only valuable explanation for the large amount of people from the power ministries coming to power on both national and local scales. Consequently, the introduction of the cultural factor has to go further than an insight into Vladimir Putin's professional background.

CONSIDERING RUSSIAN ELITES BEYOND THE MILITARIZATION

The Cultural Factor as an Alternative Clue

The cultural factor will be studied in two steps in order to put forward an alternative explanation to the militarization theory À which makes the incursion of the military in Russian politics a concerted plan. On the one hand, the elite culture of liberal democracy À especially in western countries but also among Russian intellectual elites À explains the development of the militarization theory. On the other hand, the Soviet cultural legacy in Russia might be a clue to understand the reshuffle of Russian elites about the law around the beginning of the XXIst century.

The culture of liberal democracy has been playing a major role in the construction of the militarization theory. Thus, before discussing the findings of the prosopography concerning the cultural factor, we have to consider the cultural aspect of the knowledge on democracy and more specifically on transitions to democracy. In fact, the topic of elite political culture À understood as the circulation of cultural knowledge concerning transitions to democracy À helps to have a better understanding of both the construction of Russia's attitude toward democracy and the future collective political behavior in Putin's Russia. Thus, considering the cultural dimension of politics allows to complete the deconstruction of the militocratic paradigm. That is the reason why we need to identify deeply both its theoretical and contextual origins, that are closely related to the production of democratic knowledge. The militocratic paradigm is, by and large, a construct of analysts' pathological look at the Russian political regime and, therefore, at the Russian elites. Although transitology as well as all theories of democratization have been widely discarded, the fact remains that they still have effects on the field of comparative politics and political regimes analysis. The main objections that have been raised against transitology and its grip on comparative analysis of political regime change might apply to the militocratic paradigm. Indeed, transitology suggests that liberal democracy is the end of the transition process which, by the way, is placed at the very core of the analysis. Its normativity, that comprises sometimes a very prescriptive dimension, has had a strong influence on the production of knowledge on the new-born Russian political regime since the end of the 1980s. Yet, if liberal democracy is considered an ideal in terms of political regime, the Russian political regime is to be regarded, at least, as an incomplete or imperfect one (1990s), or even as an antithesis or a concurrent model (2000s). As a consequence, Putin's Russia has been gradually observed from a pathological standpoint. According to it, Russia got lost on its way toward democracy. This has become particularly obvious since Vladimir Putin came to power because he departed from the liberal and pro-western policy of his predecessor.

It seems that some of the main objections made to transitological approaches (see for instance Carothers, 2002;Dobry, 2000a;[START_REF] Zalewski | Les transitions de´mocratiques en Europe centrale et orientale[END_REF] should be reinvested to discuss the pathological look at both Russia's political regime and the elites. For instance, the tree structure used by [START_REF] Przeworski | Some problems in the study of the transition to democracy[END_REF] in The Games of Transition to describe the sequence of choice-situations on the path to democracy represents the finalist aspect of transitology. It has been demonstrated that one should rather question the process of social and political reproduction explaining the original forms of the State in postcommunist societies (Dobry, 2000b). It is a matter of restoring the importance of political structures while transitology is only centered on political actors and their strategies. Transitology also emphasizes the gap between the historical path of postcommunist societies toward democracy and the ideal democratic path by conceptualizing it. This gap approaches the gap that [START_REF] Bailey | Les re`gles du jeu politique: e´tude anthropologique[END_REF] put in evidence between normative and pragmatic rules. On the one hand there are general political guidelines. On the other, there are goal-oriented leverages and tactics.

Last but not least concerning the issue of democratic knowledge, a careful look should be given to the study [START_REF] Gue´nard | La promotion de la de´mocratie: une impasse the´orique? La vie des ide´es[END_REF] made on the debates held in the field of democratization studies in the United States and more particularly within the Journal of Democracy. The author first notes that the Journal was created on R. Reagan's initiative at the beginning of the 1980s and was intended to be the fundamental guide during the war of ideas with the Soviet Union. Thus it aimed at supporting the export of the "democratic model." Indeed, its founders M. J. Plattner and L. J. Diamond, both close to the American neo-cons, wanted to challenge the ideologies of authoritarian and totalitarian States with a militant support of democracy. The journal was supposed to favor the expression of a "science of democracy." Yet, this ideological end encouraged theoretical reshuffles such as the creation of the concept of consolidation that renewed and consolidated the transitologic paradigm. Nevertheless, the theoretical inputs that came with the notion of consolidation did not modify the objections we already expressed. For instance, the confirmation of the existence of "hybrid regimes" [START_REF] Diamond | Thinking about hybrid regimes[END_REF] does not change the fact that the analysis is relying on a one best way. Political elites are still regarded as pluralistic and stratified elites remain the norm of democratization as, for instance, in the analysis of the relations between elites and political regimes (see, e.g., Carothers, 2002, p. 301;Genieys, 2011, p. 266 where Russia is associated with China, Cuba, North Korea, and Iran as a country where the "political elite is (until then) ideologically unified," p. 300). Elites are indeed, in this conception, made to compete with a large variety of capitals. It might sound quite normative to consider that Russian elites are supposed to become pluralistic. It results in two main effects. It first suggest that they are perfectly ideologically unified at the moment, which deeply underestimates the complexity of relations and interactions not only between the various configurations of elites but also within each of them. It secondly implies that Russia is lagging behind comparing to the West which is supposed to embody political modernity.

Nonetheless, it must be acknowledged that, after a period of hesitation about the way Russia should follow to make its transition, the new team that came to power in the end of 1999 quickly ensured the shutdown of the political game. Following on from Juan J. Linz about authoritarian regimes (1985), academics such as William Genieys (op. cit.) tend to attach authoritarian regimes to a limited elite pluralism. This reinforces the pathological look at Russian elites as a consequence of the analysis of Russia's political regime. We in fact assume that Russian political elites are commonly evaluated in reference to the incompleteness of its democracy. Furthermore, all these developments fit in perfectly with the militarization theory which is essentially based on the idea that Russian elites have become strikingly homogenized À since Vladimir Putin came to power in 1999 À around a common military ethos. It has been finally corroborated in the course of both journalistic and scientific narratives focused on the victory of the former KGB members over the liberal democrats and the reformers. As a consequence, the analysis of elites in post-Soviet Russia, where the competition is restricted and favors the reproduction of an oligarchic elite linked by stable collusions, makes Russia a black sheep among modern western democracies that are legitimated by a knowledge they developed across the world's political, diplomatic, and scientific fields [START_REF] Wagner | Les, classes sociales dans la mondialisation[END_REF].

Reconsidering the Reshuffle of Russian Elites

Considering the issue of the cultural factor to think about both political elites and political regime changes in Post-Soviet Russia is to establish the preliminary basis of a more nuanced reading of their evolution since the fall of the USSR. The cultural factor is not only a way to identify how the militocratic paradigm is an obstacle to the analysis of Russian elites. It is also a clue to identify a more relevant binding agent between them. As we saw, one of the interests of the analysis of the career paths of the 24 polpredy who have been in function from 2000 to 2012 is to underline the heterogeneity of a particular body of the presidential administration behind a homogeneous representation. Besides, it allows us to go beyond the identification of the militocratic paradigm. In fact, asserting that answering Robert Dahl's famous interrogation À "Who governs?" (1961) À is a harder that it looks concerning contemporary Russia, does not mean that we cannot find some clue.

Back to our corpus, some career paths make it clear that a continuity exists among Russian political elites between the Yeltsin era and the Putin era. This is in line with some major theoretical contributions provided on central and eastern Europe [START_REF] Mink | La grande reconversion[END_REF] and Russia [START_REF] Szelenyi | Circulation or reproduction of elites during post-communist transformation in Russia and Eastern Europe[END_REF] which analyses former high-ranking communist cadres' career paths after the fall of the Berlin wall and more specifically their retraining in politics and economy. For instance, trajectories such as the one of S. Kiriyenko confirm the continuity between the Yeltsin and the Putin years. A former high-ranking official of the CPSU during the 1980s and then close to the late B. Nemtsov À former liberal politician who was one of Yeltsin's lieutenants À in Nizhny-Novgorod, he became Prime Minister in 1998 when Yeltsin was President of the Russian Federation. He was one of the founders of the Union of Right Forces (SPS) which was the main political movement of the right-wing liberals created in 1999. He then became the first polpred of the Siberian Federal District after V. Putin came to power. We can consider his term was a success since V. Putin then named him to become the head of the Federal Atomic Energy Agency (Rosatom) when it was created in 2005. He is now 52 years old and is still in office. This example of a liberal politician of the early 1990s recruited by V. Putin fights against the idea that military people erased the past, shunting Yeltsin years' liberals into early retirement. We might also give the example of G. Poltavchenko who was the first polpred in office in the Central Federal District. He had been chosen, before, to act on behalf of Yeltsin in the Leningrad oblast under the previous administration.

Highlighting these trajectories, we cannot help trying to understand how V. Putin gathered such different people in this state administration body and globally in his political entourage. What kind of vector of "the unity in diversity" (Highley & Pakulski, 2000, p. 658) À did V. Putin find to make everyone pull together? It seems to us that law is part of the answer even beyond the concepts of dictatorship of law and power vertical (see below). Keeping in mind the importance of both historical and geographical context in the study of transitions and regime change [START_REF] Bunce | Comparative democratization. Big and bounded generalizations[END_REF] a look should be taken at the political, economic and social developments of the late 1990s in Russia. In the generalized crisis context corresponding to Putin's coming to office, the power elites have been developing a legal discourse to insure their own legitimation. This part of the work has been especially symbolized by Vladislav Surkov's ideological work within the Kremlin and the two main concepts we already talked about À the vertical of power and the dictatorship of law À aimed at building a "sovereign" or "managed" democracy [START_REF] Laruelle | Le nouveau nationalisme russe[END_REF].

The concept of "sovereign democracy" set up by Vladislav Surkov is indeed helpful to portray not only what is assumed to be the practical gap between Russian regime and the ones in western countries but also the theoretical conceptualization of differences between democracy in Western understanding and the Russian regime. The epithet concentrates on the Surkov ideological construction intended to justify Russia's choice to take a different path, leading away from the Western conception of the liberal democracy À as claimed in particular by the US and the European Union State members. The sovereign democracy should firstly be seen as a reaction À not to say reflex À from the Russian head of state to the colored revolutions that happened in Georgia, Ukraine and Kyrgyzstan between 2003 and 2005. This revolutionary wave in several former Soviet Union countries was indeed interpreted by the Russian power as the result of various external interferences from the West. It is then supposed to show everyone À and especially the United States À that no foreign interference will ever be tolerated. Then, Surkov's concept has to be understood as a subtle blend of Russia's political past and historical conquests. It actually insists on the nationalist dimension and is based on a positive vision of the country's past, highlighting for instance the successes and achievements of the Soviet era. As M. Laruelle details it (op. cit.,p. 228): the Soviet Union was sovereign but not democratic, Russia during the 1990s was democratic but lost its sovereignty, Vladimir Putin's Russia is supposed to reconciliate these two main orientations. Finally, this ideological construction aims at embodying political modernity and the future of the Russian society in a globalized and evolving world which requires every country to be open to develop. To conclude on this point, we can contend that Vladislav Surkov's concept of sovereign democracy serves two main goals: on the domestic side, it is used to justify and defend its own path to democracy; at an international level, it can be considered a competing offer, especially intended to Russia's partners that might be willing to escape the US or EU influence such as India, China or even some countries in eastern Europe, as Polish political events of the beginning of 2016 suggest.

Accordingly, recruiting people who have competencies in both law and law enforcement would have made sense for the team in power in addition to the questions of prestige and sociability networks (Raviot, 2006, pp. 48À52). Then, the recruitment of people from law enforcement bodies would not be due to their military background but to the fact that the power in place actually believes that they master knowledge and know-how related to law. This hypothesis actually corresponds to some classic theories of the State's construction in Political science and refers to the classic notion of the rational-legal authority [START_REF] Weber | E ´conomie et Socie´te´[END_REF]. Law is thus valued and the power in place needs personnel that are familiar with it, not to say specialists. Accordingly, specific knowledge and know-how connected with legal training and sociability networks as well as a particular position within judicial field would make some individuals particularly valuable resource persons. In this respect, judicial and legal fields are considered as "positional" spaces or areas [START_REF] Boltanski | L'espace positionnel: Multiplicite´des positions institutionnelles et habitus de classe[END_REF][START_REF] Passeron | Biographies, flux, itine´raires, trajectoires[END_REF] and the construction of a monopoly on a complex knowledge À legal, in this case À must be regarded as a key factor to ensure social power [START_REF] Konrad | The Intellectuals on the road to class power[END_REF]. Considering our corpus, we can mention that not only is law at the core of the education and training in law enforcement bodies, but also four of the polpredy are former magistrates. N. Vinnichenko, V. Ustinov, D. Kozak, and A. Konovalov all worked in the General Prosecution both in Moscow and the regions. This hypothesis might be further investigated on the basis of the model provided by some research conducted on the EU staff members (see for instance [START_REF] Cohen | Les juristes et l'ordre politique europe´en[END_REF].

In fact, not only do we consider knowledge stricto sensu but also knowhow and skills as well as interpersonal relationships, cultural background etc. that generate a "entre-soi." It results from this that individuals and groups of individuals revolving around law in Russia could not be limited to lawyers or other legal experts. Those knowledge lato sensu then logically comprise everything referring to the notion of habitus [START_REF] Bourdieu | Le sens pratique[END_REF] of theses elites. Such resources provide cultural capital/asset, that comes in three "forms": "incorpore´"/"embedded," "objective"/"objectified," and "institutionnalise´"/"institutionalized" (Bourdieu, 1979, pp. 3À5). Elites are arising, in this regard, from this shared bedrock of cultural asset: "Belonging to an Elite implies mastering its behavior codes, its language, its relationships that shape modes of sociability (…) It also implies having a specific field of competence and being endowed with a particular knowhow"… (Leferme-Falguie`res & Van Renterghem, 2000, p. 63).

Furthermore, the power's ideological and political discourse emphasizing the importance of law and legality seems to be in line with political modernity and must be endorsed by international political elites. A sample of the international reactions to D. Medvedev's coming to power in 2008 À which was actually a highly fictional succession À is indicative of this wideranged acceptance. Lawyer by training and former University teacher of roman and civil law, he was supposed to be a very good match for the globalized intellectual elite since they were assuming that he was a part of it. Nevertheless, Dmitri Medvedev's policy as a President of the Federation showed the relativity of his modernity discourse. This is especially true knowing that V. Putin introduces himself as a modernizer since his very first term in office.

Furthermore, we might postulate the relevance of the connection with Charles ' Tilly's work (1992) on coercion À associated with capital À in state development in Europe and the leading role of war or, at least, its preparation À whether from a practical point of view or at a symbolic and discursive level. According to C. Tilly, the establishment of an army generated a stable state structure and the preparation of war has been the source of internal structures of the European states. However, he highlighted a mili-tarizationÀcivilization process after which the civilian bureaucracies and assemblies regain control over military people and structures. Yet, this process is not supposed to consist of a compulsory step of the states' development. In this perspective, Tilly's work emphasizes the parallel existence of states where militaries remain on the top of power, especially when institutions led by civilian people have failed. In the case of Russia, the 1990s can be considered as the failure and disrepute of civilian and democratic institutions. Russia's contemporary regime is then, again, considered in pathological perspective since the civilization is regarded as missing or failed. The constant references to the rule of law might be seen as a countervailing measure to ensure the existence of a civil State.

LAW AS A RELEVANT ELEMENT OF POLITICAL ELITES CULTURE TO BUILD A "SOVEREIGN DEMOCRACY"

According to N. Elias' fundamentally relational conception of the society (1981), the social equilibrium of forces results in the interdependent relationships that occur among elites and are related to the law. We are prone to think that although the existence of a common habitus of silovikis making them a homogeneous category is hardly relevant, the power in place may have developed a common belief in the existence of a common ethos relying on law and legality among political and administrative elites.

In other words, whether or not these common characteristics and shared values are an illusion of the team in power, it seems that they bet on building their authority and power on it. This belief might be scrutinized in the light of M. Dobry's concept of "confidence in the habitus" (1992). It means that people in power and V. Putin in particular are betting on what they assume to be the habitus of the individuals they recruit. According to the representation that they have of them, they are "the men for the job," the ones that should be trusted in a context of crisis À that is, a context of overturning of the institutional references.

It is hard to identify a coherent ideological construction in today's Russia except for a common denominator based on a massive and multifaceted nationalist reinvestment [START_REF] Laruelle | Le nouveau nationalisme russe[END_REF]. It is worth noting in this regard that scholar books are currently being rewritten, as demonstrated by V. Putin's recent project to edit a common textbook for all Russian students of classes nine and eleven (that students usually attend when they are 15 and 17). This emerging enterprise already aims at presenting an idealized and pacified vision of the Soviet era [START_REF] Konkka | La recherche d'un mode`le de pre´sentation de l'histoire sovie´tique dans les anne´es 1990-2000: l'exemple du manuel scolaire Histoire de la Russie XXe`me À de´but XXIe`me sie`cles d'A[END_REF]. According to the same logic, Law might play a role as the "lowest common denominator" and a symbolic resource for the elites in Putin's Russia. Putin came to power while Russia was in the middle of a political, economic and social crisis that followed the collapse of the USSR. Both the discourse and action increasingly valued law and legality, partly through revaluation of the Soviet cultural heritage. For instance, Vladislav Surkov, who was for a long time the personal spin doctor of V. Putin and is considered to be "the man behind" the Presidential party United Russia as well as the concept of the sovereign democracy, has made clear that the power wants to repeat not the warrior rhetoric but the discourse of modernization of the Soviet Union [START_REF] Surkov | Soverennaıä demokratıä. Ot idei k doctrine[END_REF]. Since 2000, the social and political consensus around V. Putin resulted in both the comeback of economic growth À mostly due to the huge increase of oil prices À and the promise of a return of State authority. The team in power has been gradually looking for an ideological contribution to replace the economic aspect of the pact. Indeed, the economic crisis of 2008 and the first massive demonstrations of winter 2011 revealed the need of such a bind, not only among elites, but also between elites and the rest of society [START_REF] Hue´rou | Les mouvements de protestation: une nouvelle ge´ne´ration dans la rue. Fe´vrier[END_REF].

Considering this ideological construction and the bet of the team in power on a common commitment to law and legality, the Soviet legacy must be taken into account. Actually, this may particularly be seen as a generational effect among elites. Almost all of the 24 polpredy come not only from the secondary and higher education system but also from the training bodies of the USSR's and CPSU's cadres. Two of them even made all of the "Soviet part" of their career within the CPSU organs. In fact, V. Mativienko started in the komsomols and gradually rose to the top of the Party from 1972 to 1986 when she became Deputy President of a Soviet District Executive Committee in Leningrad. Khamil Ishakov also made a name for himself within the Party by becoming the Head of the City of Kazan. He even stayed in office after the fall of the Soviet Union since he was the mayor of Kazan until 2005.

The 24 polpredy were on an average 35 years old when the USSR collapsed. They were then at the middle of their professional careers and already deeply involved in their personal paths. A majority of them have been awarded by Soviet honorary prices and medals that made them "Heroes of the Soviet Union" for instance. Their personal and individual socializations have been deeply influenced by their experience of the communist regime and the main role assigned to legality and the law. The Socialist legality is a legacy of Lenin's "revolutionary legality" and was renamed as soon as Stalin came to power. Its concept was to give to an undemocratic political system a formal democratic frame [START_REF] Ginsburgs | Socialist legality' in the USSR. since the XXth party congress[END_REF][START_REF] Lavigne | La le´galite´socialiste et le de´veloppement de la pre´occupation juridique en Union Sovie´tique[END_REF]. As far as we are concerned, the continual reference to law in today's Russia can be seen as a legacy of this design. Indeed, the Soviet concept of socialist legality helps us to understand both how political culture is significant in unifying political and administrative elites and how it may be identified as a common denominator beyond the opposition of democratic and autocratic.

The militarization theory we highlighted might distract from a more complex À and classical, assuming it follows Max Weber's conception of legalrational authority À process of legitimation of power in post-Soviet Russia: using law as an instrumental resource. As French sociologist B. Pudal, drawing on various works of historians, wrote about the concept of totalitarianism in the specific case of Soviet Union, we consider the militarization theory to be a "smokescreen-concept" (Pudal, 2009, p. 164). The establishment of the plenipotentiary envoys in 2000 has clearly been regarded as a personification of the movement of "recentralization" of power. The prosopographical review though strongly shades the militarization thesis and allows to escape the "militocratic paradigm" as we understand it. In order to legitimate its policy, the power in place instrumentalizes both the law and the relation to legality in the political and administrative fields. By historicizing the process of reshuffling of the elites in post-Soviet Russia since the end of the 1980s and the Gorbatchevian perestroı¨ka, we can bring up the idea of a reinvestment of the socialist legality by the power in place. According to this concept, we could assume that the group in power bets on sort of an elite ethos grounded on a common commitment to law and considers this bet an efficient political strategy.


Simon Leglaive

Xavier Alameda-Pineda

Laurent Girin

Radu Horaud

A RECURRENT VARIATIONAL AUTOENCODER FOR SPEECH ENHANCEMENT

Keywords: Speech enhancement, recurrent variational autoencoders, nonnegative matrix factorization, variational inference

This paper presents a generative approach to speech enhancement based on a recurrent variational autoencoder (RVAE). The deep generative speech model is trained using clean speech signals only, and it is combined with a nonnegative matrix factorization noise model for speech enhancement. We propose a variational expectationmaximization algorithm where the encoder of the RVAE is finetuned at test time, to approximate the distribution of the latent variables given the noisy speech observations. Compared with previous approaches based on feed-forward fully-connected architectures, the proposed recurrent deep generative speech model induces a posterior temporal dynamic over the latent variables, which is shown to improve the speech enhancement results.

INTRODUCTION

Speech enhancement is an important problem in audio signal processing [START_REF] Loizou | Speech enhancement: theory and practice[END_REF]. The objective is to recover a clean speech signal from a noisy mixture signal. In this work, we focus on single-channel (i.e. single-microphone) speech enhancement.

Discriminative approaches based on deep neural networks have been extensively used for speech enhancement. They try to estimate a clean speech spectrogram or a time-frequency mask from a noisy speech spectrogram, see e.g. [START_REF] Xu | A regression approach to speech enhancement based on deep neural networks[END_REF][START_REF] Weninger | Speech enhancement with LSTM recurrent neural networks and its application to noise-robust ASR[END_REF][START_REF] Wang | Supervised speech separation based on deep learning: An overview[END_REF][START_REF] Li | Audio-noise power spectral density estimation using long short-term memory[END_REF][START_REF] Li | Multichannel speech enhancement based on time-frequency masking using subband long short-term memory[END_REF]. Recently, deep generative speech models based on variational autoencoders (VAEs) [START_REF] Kingma | Auto-encoding variational Bayes[END_REF] have been investigated for single-channel [START_REF] Bando | Statistical speech enhancement based on probabilistic integration of variational autoencoder and non-negative matrix factorization[END_REF][START_REF] Leglaive | A variance modeling framework based on variational autoencoders for speech enhancement[END_REF][START_REF] Leglaive | Speech enhancement with variational autoencoders and alpha-stable distributions[END_REF][START_REF] Pariente | A statistically principled and computationally efficient approach to speech enhancement using variational autoencoders[END_REF] and multichannel speech enhancement [START_REF] Sekiguchi | Bayesian multichannel speech enhancement with a deep speech prior[END_REF][START_REF] Leglaive | Semi-supervised multichannel speech enhancement with variational autoencoders and non-negative matrix factorization[END_REF][START_REF] Fontaine | Cauchy multichannel speech enhancement with a deep speech prior[END_REF]. A pre-trained deep generative speech model is combined with a nonnegative matrix factorization (NMF) [START_REF] Févotte | Nonnegative matrix factorization with the Itakura-Saito divergence: With application to music analysis[END_REF] noise model whose parameters are estimated at test time, from the observation of the noisy mixture signal only. Compared with discriminative approaches, these generative methods do not require pairs of clean and noisy speech signal for training. This setting was referred to as "semi-supervised source separation" in previous works [START_REF] Smaragdis | Supervised and semisupervised separation of sounds from single-channel mixtures[END_REF][START_REF] Mysore | A non-negative approach to semisupervised separation of speech from noise with the use of temporal dynamics[END_REF][START_REF] Mohammadiha | Supervised and unsupervised speech enhancement using nonnegative matrix factorization[END_REF], which should not be confused with the supervised/unsupervised terminology of machine learning.

To the best of our knowledge, the aforementioned works on VAE-based deep generative models for speech enhancement have only considered an independent modeling of the speech time frames, through the use of feed-forward and fully connected architectures. In this work, we propose a recurrent VAE (RVAE) for modeling the speech signal. The generative model is a special case of the one proposed in [START_REF] Chung | A recurrent latent variable model for sequential data[END_REF], but the inference model for training is different. At test time, we develop a variational expectation-maximization algorithm (VEM) [START_REF] Neal | A view of the EM algorithm that justifies incremental, sparse, and other variants[END_REF] to perform speech enhancement. The encoder Xavier Alameda-Pineda acknowledges the French National Research Agency (ANR) for funding the ML3RI project. This work has been partially supported by MIAI @ Grenoble Alpes, (ANR-19-P3IA-0003).

of the RVAE is fine-tuned to approximate the posterior distribution of the latent variables, given the noisy speech observations. This model induces a posterior temporal dynamic over the latent variables, which is further propagated to the speech estimate. Experimental results show that this approach outperforms its feed-forward and fully-connected counterpart.

DEEP GENERATIVE SPEECH MODEL

Definition

Let s = {sn ∈ C F } N -1
n=0 denote a sequence of short-time Fourier transform (STFT) speech time frames, and z = {zn ∈ R L } N -1 n=0 a corresponding sequence of latent random vectors. We define the following hierarchical generative speech model independently for all time frames n ∈ {0, ..., N -1}:

sn z ∼ Nc (0, diag {vs,n(z)}) , with zn i.i.d ∼ N (0, I) , (1)
and where vs,n(z) ∈ R F + will be defined by means of a decoder neural network. N denotes the multivariate Gaussian distribution for a real-valued random vector and Nc denotes the multivariate complex proper Gaussian distribution [START_REF] Neeser | Proper complex random processes with applications to information theory[END_REF]. Multiple choices are possible to define the neural network corresponding to vs,n(z), which will lead to different probabilistic graphical models represented in Fig. 1.

FFNN generative speech model vs,n(z) = ϕ FFNN dec (zn; θdec) where ϕ FFNN dec (⋅ ; θdec) ∶ R L ↦ R F + denotes a feed-forward fullyconnected neural network (FFNN) of parameters θdec. Such an architecture was used in [START_REF] Bando | Statistical speech enhancement based on probabilistic integration of variational autoencoder and non-negative matrix factorization[END_REF][START_REF] Leglaive | A variance modeling framework based on variational autoencoders for speech enhancement[END_REF][START_REF] Leglaive | Speech enhancement with variational autoencoders and alpha-stable distributions[END_REF][START_REF] Pariente | A statistically principled and computationally efficient approach to speech enhancement using variational autoencoders[END_REF][START_REF] Sekiguchi | Bayesian multichannel speech enhancement with a deep speech prior[END_REF][START_REF] Leglaive | Semi-supervised multichannel speech enhancement with variational autoencoders and non-negative matrix factorization[END_REF][START_REF] Fontaine | Cauchy multichannel speech enhancement with a deep speech prior[END_REF]. As represented in Fig. 1a, this model results in the following factorization of the complete-data likelihood:

p(s, z; θdec) = N -1 n=0 p(sn zn; θdec)p(zn). (2)
Note that in this case, the speech STFT time frames are not only conditionally independent, but also marginally independent, i.e. p(s; θdec) = ∏ N -1 n=0 p(sn; θdec). RNN generative speech model vs,n(z) = ϕ RNN dec,n (z0∶n; θdec) where ϕ RNN dec,n (⋅ ; θdec) ∶ R L×(n+1) ↦ R F + denotes the output at time frame n of a recurrent neural network (RNN), taking as input the sequence of latent random vectors z0∶n = {z n ′ ∈ R L } n n ′ =0 . As represented in Fig. 1b, we have the following factorization of the complete-data likelihood:

p(s, z; θdec) = N -1 n=0 p(sn z0∶n; θdec)p(zn). (3)
Note that for this RNN-based model, the speech STFT time frames are still conditionally independent, but not marginally independent.

BRNN generative speech model vs,n(z) = ϕ BRNN dec,n (z; θdec) where ϕ BRNN dec,n (⋅ ; θdec) ∶ R L×N ↦ R F + denotes the output at time frame n of a bidirectional RNN (BRNN) taking as input the complete sequence of latent random vectors z. As represented in Fig. 1c, we end up with the following factorization of the complete-data likelihood:

p(s, z; θdec) = N -1 n=0 p(sn z; θdec)p(zn). (4)
As for the RNN-based model, the speech STFT time frames are conditionally independent but not marginally. Note that for avoiding cluttered notations, the variance vs,n(z) in the generative speech model ( 1) is not made explicitly dependent on the decoder network parameters θdec, but it clearly is.

Training

We would like to estimate the decoder parameters θdec in the maximum likelihood sense, i.e. by maximizing ∑ I i=1 ln p s (i) ; θdec , where {s (i) ∈ C F ×N } I i=1 is a training dataset consisting of I i.i.d sequences of N STFT speech time frames. In the following, because it simplifies the presentation, we simply omit the sum over the I sequences and the associated subscript (i).

Due to the non-linear relationship between s and z, the marginal likelihood p(s; θdec) = ∫ p(s z; θdec)p(z)dz is analytically intractable, and it cannot be straightforwardly optimized. We therefore resort to the framework of variational autoencoders [START_REF] Kingma | Auto-encoding variational Bayes[END_REF] for parameters estimation, which builds upon stochastic fixed-form variational inference [START_REF] Jordan | An introduction to variational methods for graphical models[END_REF][START_REF] Honkela | Approximate Riemannian conjugate gradient learning for fixed-form variational Bayes[END_REF][START_REF] Salimans | Fixed-form variational posterior approximation through stochastic linear regression[END_REF][START_REF] Hoffman | Stochastic variational inference[END_REF][START_REF] Blei | Variational inference: A review for statisticians[END_REF]. This latter methodology first introduces a variational distribution q(z s; θenc) (or inference model) parametrized by θenc, which is an approximation of the true intractable posterior distribution p(z s; θdec). For any variational distribution, we have the following decomposition of log-marginal likelihood: ln p(s; θdec) = Ls(θenc, θdec) + DKL q(z s; θenc) ∥ p(z s; θdec) ,

(5) where Ls(θenc, θdec) is the variational free energy (VFE) (also referred to as the evidence lower bound) defined by: Ls(θenc, θdec) = E q(z s;θenc) [ln p(s, z; θdec)ln q(z s; θenc)] = E q(z s;θenc) [ln p(s z; θdec)] -DKL q(z s; θenc) ∥ p(z) , [START_REF] Li | Multichannel speech enhancement based on time-frequency masking using subband long short-term memory[END_REF] and DKL(q ∥ p) = Eq[ln qln p] is the Kullback-Leibler (KL) divergence. As the latter is always non-negative, we see from [START_REF] Li | Audio-noise power spectral density estimation using long short-term memory[END_REF] that the VFE is a lower bound of the intractable log-marginal likelihood. Moreover, we see that it is tight if and only if q(z s; θenc) = p(z s; θdec). Therefore, our objective is now to maximize the VFE with respect to (w.r.t) both θenc and θdec. But in order to fully define the VFE in [START_REF] Li | Multichannel speech enhancement based on time-frequency masking using subband long short-term memory[END_REF], we have to define the form of the variational distribution q(z s; θenc).

Using the chain rule for joint distributions, the posterior distribution of the latent vectors can be exactly expressed as follows:

p(z s; θdec) = N -1 n=0 p(zn z0∶n-1, s; θdec), (7)
where we considered p(z0 z-1, s; θdec) = p(z0 s; θdec). The variational distribution q(z s; θenc) is naturally also expressed as:

q(z s; θenc) = N -1 n=0 q(zn z0∶n-1, s; θenc). (8)
In this work, q(zn z0∶n-1, s; θenc) denotes to the probability density function (pdf) of the following Gaussian inference model:

zn z0∶n-1, s ∼ N µ z,n (z0∶n-1, s), diag vz,n(z0∶n-1, s) , (9)
where {µ z,n , vz,n}(z0∶n-1, s) ∈ R L × R L + will be defined by means of an encoder neural network.

Inference model for the BRNN generative speech model For the BRNN generative speech model, the parameters of the variational distribution in ( 9) are defined by

{µ z,n , vz,n}(z0∶n-1, s) = ϕ BRNN enc,n (z0∶n-1, s; θenc), (10)
where

ϕ BRNN enc,n (⋅, ⋅ ; θenc) ∶ R L×n × C F ×N ↦ R L × R L +
denotes the output at time frame n of a neural network whose parameters are denoted by θenc. It is composed of: 1. "Prediction block": a causal recurrent block processing z0∶n-1; 2. "Observation block": a bidirectional recurrent block processing the complete sequence of STFT speech time frames s;

3. "Update block": a feed-forward fully-connected block processing the outputs at time-frame n of the two previous blocks.

If we want to sample from q(z s; θenc) in ( 8), we have to sample recursively each zn, starting from n = 0 up to N -1. Interestingly, the posterior is formed by running forward over the latent vectors, and both forward and backward over the input sequence of STFT speech time-frames. In other words, the latent vector at a given time frame is inferred by taking into account not only the latent vectors at the previous time steps, but also all the speech STFT frames at the current, past and future time steps. The anti-causal relationships were not taken into account in the RVAE model [START_REF] Chung | A recurrent latent variable model for sequential data[END_REF].

Inference model for the RNN generative speech model Using the fact that zn is conditionally independent of all other nodes in Fig. 1b given its Markov blanket (defined as the set of parents, children and co-parents of that node) [START_REF] Bishop | Pattern Recognition and Machine Learning[END_REF], [START_REF] Kingma | Auto-encoding variational Bayes[END_REF] can be simplified as:

p(zn z0∶n-1, s; θdec) = p(zn z0∶n-1, s n∶N -1 ; θdec), (11)
where

s n∶N -1 = {s n ′ ∈ C F } N -1 n ′ =n .
This conditional independence also applies to the variational distribution in [START_REF] Leglaive | A variance modeling framework based on variational autoencoders for speech enhancement[END_REF], whose parameters are now given by:

{µ z,n , vz,n}(z0∶n-1, s) = ϕ RNN enc,n (z0∶n-1, s n∶N -1 ; θenc), (12)
where

ϕ RNN enc,n (⋅, ⋅ ; θenc) ∶ R L×n × C F ×(N -n) ↦ R L × R L
+ denotes the same neural network as for the BRNN-based model, except that the observation block is not a bidirectional recurrent block anymore, but an anti-causal recurrent one. The full approximate posterior is now formed by running forward over the latent vectors, and backward over the input sequence of STFT speech time-frames.

Inference model for the FFNN generative speech model For the same reason as before, by studying the Markov blanket of zn in Fig. 1a, the dependencies in [START_REF] Kingma | Auto-encoding variational Bayes[END_REF] can be simplified as follows: p(zn z0∶n-1, s; θdec) = p(zn sn; θdec). [START_REF] Leglaive | Semi-supervised multichannel speech enhancement with variational autoencoders and non-negative matrix factorization[END_REF] This simplification also applies to the variational distribution in [START_REF] Leglaive | A variance modeling framework based on variational autoencoders for speech enhancement[END_REF], whose parameters are now given by:

{µ z,n , vz,n}(z0∶n-1, s) = ϕ FFNN enc (sn; θenc), (14)
where ϕ FFNN enc (⋅ ; θenc) ∶ C F ↦ R L × R L + denotes the output of an FFNN. Such an architecture was used in [START_REF] Bando | Statistical speech enhancement based on probabilistic integration of variational autoencoder and non-negative matrix factorization[END_REF][START_REF] Leglaive | A variance modeling framework based on variational autoencoders for speech enhancement[END_REF][START_REF] Leglaive | Speech enhancement with variational autoencoders and alpha-stable distributions[END_REF][START_REF] Pariente | A statistically principled and computationally efficient approach to speech enhancement using variational autoencoders[END_REF][START_REF] Sekiguchi | Bayesian multichannel speech enhancement with a deep speech prior[END_REF][START_REF] Leglaive | Semi-supervised multichannel speech enhancement with variational autoencoders and non-negative matrix factorization[END_REF][START_REF] Fontaine | Cauchy multichannel speech enhancement with a deep speech prior[END_REF]. This is the only case where, from the approximate posterior, we can sample all latent vectors in parallel for all time frames, without further approximation.

Here also, the mean and variance vectors in the inference model [START_REF] Leglaive | A variance modeling framework based on variational autoencoders for speech enhancement[END_REF] are not made explicitly dependent on the encoder network parameters θenc, but they clearly are.

Variational free energy Given the generative model [START_REF] Loizou | Speech enhancement: theory and practice[END_REF] and the general inference model ( 9), we can develop the VFE defined in [START_REF] Li | Multichannel speech enhancement based on time-frequency masking using subband long short-term memory[END_REF] as follows (derivation details are provided in Appendix A.1):

Ls(θenc, θdec) c = - F -1 f =0 N -1 n=0 E q(z s;θenc) dIS s f n 2 , v s,f n (z) + 1 2 L-1 l=0 N -1 n=0 E q(z 0∶n-1 s;θenc) ln v z,ln (z0∶n-1, s) -µ 2 z,ln (z0∶n-1, s) -v z,ln (z0∶n-1, s) , (15)
where c = denotes equality up to an additive constant w.r.t θenc and θdec, dIS(a, b) = a bln(a b) -1 is the Itakura-Saito (IS) divergence [START_REF] Févotte | Nonnegative matrix factorization with the Itakura-Saito divergence: With application to music analysis[END_REF], s f n ∈ C and v s,f n (z) ∈ R+ denote respectively the f -th entries of sn and vs,n(z), and µ z,ln (z0∶n-1, s) ∈ R and v z,ln (z0∶n-1, s) ∈ R+ denote respectively the l-th entry of µ z,n (z0∶n-1, s) and vz,n(z0∶n-1, s).

The expectations in [START_REF] Févotte | Nonnegative matrix factorization with the Itakura-Saito divergence: With application to music analysis[END_REF] are analytically intractable, so we compute unbiased Monte Carlo estimates using a set {z (r) } R r=1 of i.i.d. realizations drawn from q(z s; θenc). For that purpose, we use the "reparametrization trick" introduced in [START_REF] Kingma | Auto-encoding variational Bayes[END_REF]. The obtained objective function is differentiable w.r.t to both θdec and θenc, and it can be optimized using gradient-ascent-based algorithms. Finally, we recall that in the final expression of the VFE, there should actually be an additional sum over the I i.i.d. sequences in the training dataset {s (i) } I i=1 . For stochastic or mini-batch optimization algorithms, we would only consider a subset of these training sequences for each update of the model parameters.

SPEECH ENHANCEMENT: MODEL AND ALGORITHM

Speech, noise and mixture model

The deep generative clean speech model along with its parameters learning procedure were defined in the previous section. For speech enhancement, we now consider a Gaussian noise model based on an NMF parametrization of the variance [START_REF] Févotte | Nonnegative matrix factorization with the Itakura-Saito divergence: With application to music analysis[END_REF]. Independently for all time frames n ∈ {0, ..., N -1}, we have:

bn ∼ Nc(0, diag{v b,n }), (16)
where

v b,n = (W b H b )∶,n with W b ∈ R F ×K + and H b ∈ R K×N + .
The noisy mixture signal is modeled as xn = √ gnsn+bn, where gn ∈ R+ is a gain parameter scaling the level of the speech signal at each time frame [START_REF] Leglaive | A variance modeling framework based on variational autoencoders for speech enhancement[END_REF]. We further consider the independence of the speech and noise signals so that the likelihood is defined by:

xn z ∼ Nc (0, diag{vx,n(z)}) , (17)
where vx,n(z) = gnvs,n(z) + v b,n .

Speech enhancement algorithm

We consider that the speech model parameters θdec which have been learned during the training stage are fixed, so we omit them in the rest of this section. We now need to estimate the remaining model parameters φ = g = [g0, ..., g N -1 ] ⊺ , W b , H b from the observation of the noisy mixture signal x = {xn ∈ C F } N -1 n=0 . However, very similarly as for the training stage (see Section 2.2), the marginal likelihood p(x; φ) is intractable, and we resort again to variational inference. The VFE at test time is defined by: Lx(θenc, φ) = E q(z x;θenc) [ln p(x z; φ)]-DKL q(z x; θenc) ∥ p(z) .

(18) Following a VEM algorithm [START_REF] Neal | A view of the EM algorithm that justifies incremental, sparse, and other variants[END_REF], we will maximize this criterion alternatively w.r.t θenc at the E-step, and φ at the M-step. Note that here also, we have Lx(θenc, φ) ≤ ln p(x; φ) with equality if and only if q(z x; θenc) = p(z x; φ).

Variational E-

Step with fine-tuned encoder We consider a fixedform variational inference strategy, reusing the inference model learned during the training stage. More precisely, the variational distribution q(z x; θenc) is defined exactly as q(z s; θenc) in ( 9) and (8) except that s is replaced with x. Remember that the mean and variance vectors µ z,n (⋅, ⋅) and vz,n(⋅, ⋅) in ( 9) correspond to the VAE encoder network, whose parameters θenc were estimated along with the parameters θdec of the generative speech model. During the training stage, this encoder network took clean speech signals as input. It is now fine-tuned with a noisy speech signal as input. For that purpose, we maximize Lx(θenc, φ) w.r.t θenc only, with fixed φ. This criterion takes the exact same form as [START_REF] Févotte | Nonnegative matrix factorization with the Itakura-Saito divergence: With application to music analysis[END_REF] except that s f n 2 is replaced with x f n 2 where x f n ∈ C denotes the f -th entry of xn, s is replaced with x, and v s,f n (z) is replaced with v x,f n (z), the f -th entry of vx,n(z) which was defined along with [START_REF] Mysore | A non-negative approach to semisupervised separation of speech from noise with the use of temporal dynamics[END_REF]. Exactly as in Section 2.2, intractable expectations are replaced with a Monte Carlo estimate and the VFE is maximized w.r.t. θenc by means of gradient-based optimization techniques. In summary, we use the framework of VAEs [START_REF] Kingma | Auto-encoding variational Bayes[END_REF] both at training for estimating θdec and θenc from clean speech signals, and at testing for fine-tuning θenc from the noisy speech signal, and with θdec fixed. The idea of refitting the encoder was also proposed in [START_REF] Mattei | Refit your encoder when new data comes by[END_REF] in a different context.

Point-estimate E-Step

In the experiments, we will compare this variational E-step with an alternative proposed in [START_REF] Kameoka | Supervised determined source separation with multichannel variational autoencoder[END_REF], which consists in relying only on a point estimate of the latent variables. In our framework, this approach can be understood as assuming that the approximate posterior q(z x; θenc) is a dirac delta function centered at the maximum a posteriori estimate z ⋆ . Maximization of p(z x; φ) ∝ p(x z; φ)p(z) w.r.t z can be achieved by means of gradient-based techniques, where backpropagation is used to compute the gradient w.r.t. the input of the generative decoder network.

M-

Step For both the VEM algorithm and the point-estimate alternative, the M-Step consists in maximizing Lx(θenc, φ) w.r.t. φ under a non-negativity constraint and with θenc fixed. Replacing intractable expectations with Monte Carlo estimates, the M-step can be recast as minimizing the following criterion [START_REF] Leglaive | A variance modeling framework based on variational autoencoders for speech enhancement[END_REF]:

C(φ) = R r=1 F -1 f =0 N -1 n=0 dIS x f n 2 , v x,f n z (r) , (19)
where v x,f n (z (r) ) implicitly depends on φ. For the VEM algorithm, {z (r) } R r=1 is a set of i.i.d. sequences drawn from q(z x; θenc) using the current value of the parameters θenc. For the point estimate approach, R = 1 and z (1) corresponds to the maximum a posteriori estimate. This optimization problem can be tackled using a majorize-minimize approach [START_REF] Hunter | A tutorial on MM algorithms[END_REF], which leads to the multiplicative update rules derived in [START_REF] Leglaive | A variance modeling framework based on variational autoencoders for speech enhancement[END_REF] using the methodology proposed in [START_REF] Févotte | Algorithms for nonnegative matrix factorization with the β-divergence[END_REF] (these updates are recalled in Appendix A.2).

Speech reconstruction Given the estimated model parameters, we want to compute the posterior mean of the speech coefficients:

ŝfn = E p(s f n x f n ;φ) [s f n ] = E p(z x;φ) √ gnv s,f n (z) v x,f n (z) x f n . ( 20
)
In practice, the speech estimate is actually given by the scaled coefficients √ gnŝ f n . Note that (20) corresponds to a Wiener-like filtering, averaged over all possible realizations of the latent variables according to their posterior distribution. As before, this expectation is intractable, but we approximate it by a Monte Carlo estimate using samples drawn from q(z x; θenc) for the VEM algorithm. For the point-estimate approach, p(z x; φ) is approximated by a dirac delta function centered at the maximum a posteriori.

In the case of the RNN-and BRNN-based generative speech models (see Section 2.1), it is important to remember that sampling from q(z x; θenc) is actually done recursively, by sampling q(zn z0∶n-1, x; θenc) from n = 0 to N -1 (see Section 2.2). Therefore, there is a posterior temporal dynamic that will be propagated from the latent vectors to the estimated speech signal, through the expectation in the Wiener-like filtering of [START_REF] Neal | A view of the EM algorithm that justifies incremental, sparse, and other variants[END_REF]. This temporal dynamic is expected to be beneficial compared with the FFNN generative speech model, where the speech estimate is built independently for all time frames.

EXPERIMENTS Dataset

The deep generative speech models are trained using around 25 hours of clean speech data, from the "si_tr_s" subset of the Wall Street Journal (WSJ0) dataset [START_REF] Garofalo | CSR-I (WSJ0) Sennheiser LDC93S6B[END_REF]. Early stopping with a patience of 20 epochs is performed using the subset "si_dt_05" (around 2 hours of speech). We removed the trailing and leading silences for each utterance. For testing, we used around 1.5 hours of noisy speech, corresponding to 651 synthetic mixtures. The clean speech signals are taken from the "si_et_05" subset of WSJ0 (unseen speakers), and the noise signals from the "verification" subset of the QUT-NOISE dataset [START_REF] Dean | The QUT-NOISE-SRE protocol for the evaluation of noisy speaker recognition[END_REF]. Each mixture is created by uniformly sampling a noise type among {"café", "home", "street", "car"} and a signal-to-noise ratio (SNR) among {-5, 0, 5} dB. The intensity of each signal for creating a mixture at a given SNR is computed using the ITU-R BS.1770-4 protocol [START_REF]Algorithms to measure audio programme loudness and true-peak audio level[END_REF]. Note that an SNR computed with this protocol is here 2.5 dB lower (in average) than with a simple sum of the squared signal coefficients. Finally, all signals have a 16 kHz-sampling rate, and the STFT is computed using a 64-ms sine window (i.e. F = 513) with 75%-overlap.

Network architecture and training parameters All details regarding the encoder and decoder network architectures and their training procedure are provided in Appendix A.3.

Speech enhancement parameters

The dimension of the latent space for the deep generative speech model is fixed to L = 16. The rank of the NMF-based noise model is fixed to K = 8. W b and H b are randomly initialized (with a fixed seed to ensure fair comparisons), and g is initialized with an all-ones vector. For computing [START_REF] Chung | A recurrent latent variable model for sequential data[END_REF], we fix the number of samples to R = 1, which is also the case for building the Monte Carlo estimate of [START_REF] Neal | A view of the EM algorithm that justifies incremental, sparse, and other variants[END_REF]. The VEM algorithm and its "point estimate" alternative (referred to as PEEM) are run for 500 iterations. We used Adam [START_REF] Kingma | Adam: A method for stochastic optimization[END_REF] with a step size of 10 the gradient-based iterative optimization technique involved at the E-step. For the FFNN deep generative speech model, it was found that an insufficient number of gradient steps had a strong negative impact on the results, so it was fixed to 10. For the (B)RNN model, this choice had a much lesser impact so it was fixed to 1, thus limiting the computational burden.

Results

We compare the performance of the VEM and PEEM algorithms for the three types of deep generative speech model. For the FFNN model only, we also compare with the Monte Carlo EM (MCEM) algorithm proposed in [START_REF] Leglaive | A variance modeling framework based on variational autoencoders for speech enhancement[END_REF] (which cannot be straightforwardly adapted to the (B)RNN model). The enhanced speech quality is evaluated in terms of scale-invariant signal-to-distortion ratio (SI-SDR) in dB [START_REF] Le Roux | SDR -Halfbaked or Well Done?[END_REF], perceptual evaluation of speech quality (PESQ) measure (between -0.5 and 4.5) [START_REF] Rix | Perceptual evaluation of speech quality (PESQ)-a new method for speech quality assessment of telephone networks and codecs[END_REF] and extended short-time objective intelligibility (ESTOI) measure (between 0 and 1) [START_REF] Taal | An algorithm for intelligibility prediction of time-frequency weighted noisy speech[END_REF]. For all measures, the higher the better. The median results for all SNRs along with their confidence interval are presented in Table 1. Best results are in black-color-bold font, while gray-color-bold font indicates results that are not significantly different. As a reference, we also provide the results obtained with the noisy mixture signal as the speech estimate, and with oracle Wiener filtering. Note that oracle results are here particularly low, which shows the difficulty of the dataset. Oracle SI-SDR is for instance 7 dB lower than the one in [START_REF] Leglaive | A variance modeling framework based on variational autoencoders for speech enhancement[END_REF]. Therefore, the VEM and PEEM results should not be directly compared with the MCEM results provided in [START_REF] Leglaive | A variance modeling framework based on variational autoencoders for speech enhancement[END_REF], but only with the ones provided here.

From Table 1, we can draw the following conclusions: First, we observe that for the FFNN model, the VEM algorithm performs poorly. In this setting, the performance measures actually strongly decrease after the first 50-to-100 iterations of the algorithm. We did not observe this behavior for the (B)RNN model. We argue that the posterior temporal dynamic over the latent variables helps the VEM algorithm finding a satisfactory estimate of the overparametrized posterior model q(z x; θenc). Second, the superiority of the RNN model over the FFNN one is confirmed for all algorithms in this comparison. However, the bidirectional model (BRNN) does not perform significantly better than the unidirectional one. Third, the VEM algorithm outperforms the PEEM one, which shows the interest of using the full (approximate) posterior distribution of the latent variables and not only the maximum-a-posteriori point estimate for estimating the noise and mixture model parameters. Audio examples and code are available online [START_REF]Companion website[END_REF].

CONCLUSION

In this work, we proposed a recurrent deep generative speech model and a variational EM algorithm for speech enhancement. We showed that introducing a temporal dynamic is clearly beneficial in terms of speech enhancement. Future works include developing a Markov chain EM algorithm to measure the quality of the proposed variational approximation of the intractable true posterior distribution.

A. APPENDIX

A.1. Variational free energy derivation details

In this section we give derivation details for obtaining the expression of the variational free energy in [START_REF] Févotte | Nonnegative matrix factorization with the Itakura-Saito divergence: With application to music analysis[END_REF]. We will develop the two terms involved in the definition of the variational free energy in [START_REF] Li | Multichannel speech enhancement based on time-frequency masking using subband long short-term memory[END_REF].

Data-fidelity term From the generative model defined in (1) we have:

E q(z s;θenc) [ln p(s z; θdec)] = N -1 n=0 E q(z s;θenc) [ln p(sn z; θdec)] = - F -1 f =0 N -1 n=0 E q(z s;θenc) ln v s,f n (z) + s f n 2 v s,f n (z) -F N ln(π) (21)
Regularization term From the inference model defined in ( 9) and ( 8) we have:

DKL q(z s; θenc) ∥ p(z) = E q(z s;θenc) [ln q(z s; θenc) -ln p(z)] = N -1 n=0 E q(z s;θenc) [ln q(zn z0∶n-1, s; θenc) -ln p(zn)] = N -1 n=0 E q(z 0∶n s;θenc) [ln q(zn z0∶n-1, s; θenc) -ln p(zn)] = N -1 n=0
E q(z 0∶n-1 s;θenc) E q(zn z 0∶n-1 ,s;θenc) ln q(zn z0∶n-1, s; θenc)

-ln p(zn) = N -1 n=0
E q(z 0∶n-1 s;θenc) DKL q(zn z0∶n-1, s; θenc) ∥ p(zn)

= - 1 2 L-1 l=0 N -1 n=0 E q(z 0∶n-1 s;θenc) ln v z,ln (z0∶n-1, s) -µ 2 z,ln (z0∶n-1, s) -v z,ln (z0∶n-1, s) - N L 2 . (22)
Summing up ( 21) and ( 22) and recognizing the IS divergence we end up with the expression of the variational free energy in [START_REF] Févotte | Nonnegative matrix factorization with the Itakura-Saito divergence: With application to music analysis[END_REF].

A.2. Update rules for the M-step

The multiplicative update rules for minimizing (19) using a majorizeminimize technique [START_REF] Hunter | A tutorial on MM algorithms[END_REF][START_REF] Févotte | Algorithms for nonnegative matrix factorization with the β-divergence[END_REF] are given by (see [START_REF] Leglaive | A variance modeling framework based on variational autoencoders for speech enhancement[END_REF] for derivation details):

H b ← H b ⊙ ⎡ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎣ W ⊺ b X ⊙2 ⊙ R ∑ r=1 V (r) x ⊙-2 W ⊺ b R ∑ r=1 V (r) x ⊙-1 ⎤ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎦ ⊙1 2 ; ( 23
) W b ← W b ⊙ ⎡ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎣ X ⊙2 ⊙ R ∑ r=1 V (r) x ⊙-2 H ⊺ b R ∑ r=1 V (r) x ⊙-1 H ⊺ b ⎤ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎦ ⊙1 2 ; ( 24
)
g ⊺ ← g ⊺ ⊙ ⎡ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎣ 1 ⊺ X ⊙2 ⊙ R ∑ r=1 V (r) s ⊙ V (r) x ⊙-2 1 ⊺ R ∑ r=1 V (r) s ⊙ V (r) x ⊙-1 ⎤ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎦ ⊙1 2 , (25)
where ⊙ denotes element-wise multiplication and exponentiation, matrix division is also element-wise, V

s , V

∈ R F ×N + are the matrices of entries v s,f n z (r) and v x,f n z (r) respectively, X ∈ C F ×N is the matrix of entries x f n and 1 is an all-ones column vector of dimension F . Note that non-negativity of H b , W b and g is ensured provided that they are initialized with non-negative values.

A.3. Neural network architectures and training

The decoder and encoder network architectures are represented in Fig. 2 and Fig. 3 respectively. The "dense" (i.e. feed-forward fullyconnected) output layers are of dimension L = 16 and F = 513 for the encoder and decoder, respectively. The dimension of all other layers was arbitrarily fixed to 128. RNN layers correspond to long short-term memory (LSTM) ones [START_REF] Hochreiter | Long short-term memory[END_REF]. For the FFNN generative model, a batch is made of 128 time frames of clean speech power spectrogram. For the (B)RNN generative model, a batch is made 32 sequences of 50 time frames. Given an input sequence, all LSTM hidden states for the encoder and decoder networks are initialized to zero. For training, we use the Adam optimizer [START_REF] Kingma | Adam: A method for stochastic optimization[END_REF] with a step size of 10 -3 , exponential decay rates of 0.9 and 0.999 for the first and second moment estimates, respectively, and an epsilon of 10 -8 for preventing division by zero.

Fig. 1 :

 1 Fig. 1: Probabilistic graphical models for N = 3.

Fig. 2 :Fig. 3 :

 23 Fig. 2: Decoder network architectures corresponding to the speech generative models in Fig. 1.

Table 1 :

 1 -2 for Median results and confidence intervals.

	Algorithm	Model	SI-SDR (dB)	PESQ	ESTOI
	MCEM [9]	FFNN	5.4 ± 0.4	2.22 ± 0.04 0.60 ± 0.01
	PEEM	FFNN RNN BRNN	4.4 ± 0.4 5.8 ± 0.5 5.4 ± 0.5	2.21 ± 0.04 0.58 ± 0.01 2.33 ± 0.04 0.63 ± 0.01 2.30 ± 0.04 0.62 ± 0.01
	VEM	FFNN RNN BRNN	4.4 ± 0.4 6.8 ± 0.4 6.9 ± 0.5	1.93 ± 0.05 0.53 ± 0.01 2.33 ± 0.04 0.67 ± 0.01 2.35 ± 0.04 0.67 ± 0.01
	noisy mixture oracle Wiener filtering	-2.6 ± 0.5 12.1 ± 0.3	1.82 ± 0.03 0.49 ± 0.01 3.13 ± 0.02 0.88 ± 0.01