Academic Entrepreneurship in France: the promotion of economic returns of public research and its political and scientific challenges

de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

In France, the contribution of public research to economic development has given rise to a vast number of laws and public measures at the national, regional and local levels. The 1999 Law on innovation and research [START_REF]Law of 12[END_REF] clearly illustrates the spirit of the successive governments in this matter 2 . This law makes it easier for researcherscivil servants to move to the private sector in order to contribute to the development of technological activities in existing or new firms. Hence, in addition to the more common possibilities for technology transfer and public-private partnerships, researchers-civil servants are also incited to take part in economic and labour activities. Although being able to be both 'civil servants' and 'entrepreneurs' seems unusual, it is now encouraged by the government.

The issue is one of academic entrepreneurship (Mustar, 2003) 3 , i.e. the creation of firms from public service research and the statutory staff or recent graduates as a phenomenon that is indicative of the on-going changes in the public research system in France. Far from choosing a perspective that analyses the conditions of R & D and technological innovation, the purpose is to open the black boxes that the academic configurations [START_REF]Academic configurations constitute pertinent social training for the actors who find themselves there, heads of institutions and directors' teams, teaching and research professionals[END_REF] often constitute in this debate. Academic entrepreneurship will therefore be examined from the standpoint of the activities of the responsible ministry and the research professionals which combines the contributions from sociology of organisations, the professions and public action. Seen from this angle, academic entrepreneurship is not only understood as the use of the potential of public research in the service of technological economic development, but also as a way of increasing the resources of university institutions. Academic entrepreneurship is integrated in a series of activities known as the 'economic returns' of public research. We will show that these activities confusedly combine the external drivers of the development of a knowledge economy and the internal drivers of the commercialisation and protection of the products of public research. This latter logic is common in the national government and has also deeply penetrated the minds of the researchers.

This article not only analyses the content of official reports and national statistics, but is also based on the case study of the creation of innovative technological firms and on interviews with the coordinators of university projects.The first part deals with the theoretical debate in which this research was embedded. The second part is devoted to the analysis of present public policies that support academic entrepreneurship. The third part, which targets the recipients of the European Journal of Education, Vol. 43, No. 3, 2008 public policies, deals with the place of entrepreneurial practices in French universities and in the careers of the researchers.

The Economic Returns of Public Research: the political and sectorial challenges and the theoretical debate

Analysing the economic promotion of the activities of those who work in public research in France leads us first to discuss the new characteristics of these practices. There is nothing new about professionals' commercial activities in public research. But in the past they were not very visible or quantifiable and included individuals or peripheral structures of an associative or private kind rather than public research institutions (universities, agencies), laboratories and academic researchers active in the framework of their main activity.

Encouraged by the ministry and launched at the European level in a context of strong economic and social demand, promoting the economic value of research is now imposed on universities and their staff and on public research agencies. The instituts nationaux polytechniques and the scientific faculties introduced it before the year 2000 and the humanities and law faculties did so after this date, with a notable acceleration between 2002 and 2003 [START_REF] Curie | Les activités de valorisation dans les établissements universitaires français[END_REF] .

But it is easier to identify than to quantify the growing efforts of institutions to encourage and organise economic promotion. First of all, there is a lack of budgetary and longitudinal data and there are great differences between the institutions and the disciplines. Second, the objective data seem to put into perspective rather than to support the discourse about the significant development of economic promotion in the public research institutions. At the institutional level, there are sometimes substantial evolutions. The scientific university of Lyon (Lyon I), which has 27,000 students, managed 408 private contracts in 2002, representing some 10 million euros [START_REF]Chambre régionale des comptes de Rhône-Alpes[END_REF] . The European contracts represented about the same amount. The number of contracts (European, private . . .) doubled in five years. They represented 2.5 times more than the funding from the Ministry, whereas in 1999 they were only 12% higher 7 . Globally speaking, the resources of the institutions generated by research are increasing. The CNRS (Centre national de la recherche scientifique) thus saw its budget increase by 56% between 2000 and 2005 [START_REF]Cour des comptes[END_REF] . Such increases, however, must be put into perspective. On the one hand, these resources remain marginal, even if they are increasing: the 2006 budget of the CNRS amounted to 2,738 million euros, 75% of which came from state subsidies (81% in 2000) and a quarter from contracts, royalties from patents and services (495 million euros). On the other hand, they increasingly come from public funds and correspond to an evolution of the steering modes of research through funding that is no longer globalised, but granted for projects. Hence, of all the CNRS contracts, only 6% were signed with firms, mostly with public administrations (Europe, State, regions . . .) [START_REF]Le CNRS en chiffres[END_REF] . In 2003, the total resources of the university research teams (outside staff salaries) amounted to 1,174 million euros. 32% came directly from the Ministry of Research (contractual allocations) and 22% had transited via organisations (CNRS . . .). The European contribution amounted to 7%, that of the regions to 11% and that of the expertise contracts and other resources to slightly over 25% 10 .

The mixed nature of these results leads one to question the meaning actors give to the term 'economic returns'. Indeed, it varies: research contracts for or in collaboration with firms, different services, royalties from the exploitation licences of patents, mobility of researchers between the private and public sectors, creation of firms born of public research. Yet in the political discourses and the figures, the frontiers of economic promotion are not necessarily clearly fixed. It can be considered as a policy of patents and licences (IGAENR, 2000) or also englobe the contracts, the creation of high tech firms and the transfer of technology. It can give rise to controversy on whether consultancy and services to firms, whose added value in terms of technological innovation is weak, should be taken into account. Research contracts funded by public administrationsthe European Union, the regions and the State -are not always taken into consideration in the same way, since they are supposed to both promote applied research and ensure the permanence of a public service of autonomous research. But these unfixed definitions seem to reveal that economic promotion underpins two political objectives that are often associated yet distinct. On the one hand, it aims to promote and develop economic activities that are based on qualifications and technologies, i.e. 'a knowledge economy'. On the other, it has a commercial purpose, i.e. a 'commercialisation' of knowledge in the service of the continuity and development of academic activities. This ambiguity of the policies and practices is echoed in the theoretical debate. Hence, a large set of work, dominated by the economy of innovation or the sociology of science, deals with economic promotion as a revealer of the slow adaptation of academic research to a knowledge economy (Gibbons, 1994), i.e. a mode of production of knowledge that focuses on problems to be solved as they are defined by industry or the public authorities, which is at odds with the academic and watertight organisation of disciplines and universities and in harmony with action schemes and a hybrid, transdisciplinary and heterogeneous organisation (Grossetti, 2003, Champenois, 2006). The model of the 'triple helice' (Etzkowitz & Leydesdorff, 1997) evokes a hybrid space where science, industry and government try respectively to take the place of the other, thus erasing their differences. Entrepreneurial universities take on a new mission of economic development and technological innovation and resemble start-up incubators and technological poles. High tech firms come closer to the academic model by carrying out research activities and creating networks to share knowledge. Governments act as entrepreneurs and investors in risk-capital and encourage links between researchers and firms via the financing of partnership projects or of territorial systems of innovation. Many studies analyse these phenomena and show that, far from the figure of the isolated researcher, the researcher's creative capacities are often underpinned by a system of varied solicitations and 'academic research in partnership' (Laredo & Mustar, 2001; 2003). This thesis of hybrid university organisation does not take into account the many actors involved in the 'scientific policy', their logics and the compromises that result from this (Cassier, 1996; Pestre & Jacq, 1996; Tuunainen, 2005) nor the 'field effects' (Bourdieu, 1997). Although laboratories and universities are changing under the pressure of their environment and the significant increase in economic, social and political demands, these studies lead one to consider these external factors -especially the economic factor -as objective needs that impose themselves on the actors of research. But the universities, their disciplinary and scientific components, and the professionals that work there develop strategies to reduce their dependence on external constraints.

Another series of studies deals more specifically with the reconciliation of academic practices and the values and logics of the commercial sector. These analyses apply as much to the research programmes as to the training offer or the university sport. In this conception, economic promotion activities are embedded in a commercialisation process of university activities, i.e. the efforts made within the universities to make a profit from teaching, research and other university services (Bok, 2003). The concept of academic capitalism (Slaughter & Leslie, 1997) refers to the commercialisation efforts made by the universities and academics to ensure the continuity of the funding of their activities: 'institutional and professorial market or market like efforts to secure external funds' (Slaughter & Leslie, 1997, p. 209). These trends are not without effect, as academic communities are turning towards activities 'with a commercial potential' (Slaughter & Leslie, 1997; Bok, 2003) to the detriment of basic research or 'true' higher education (Aronowitz, 2000).

Many of these authors privilege neo-liberal interpretations of the growth of these practices. Hence, academic capitalism is an answer to economic globalisation and the reduction in traditional public subventions of a structural type in favour of selective and project funds. The budgetary restrictions imposed by national governments on the sector oblige the latter to develop entrepreneurial attitudes to find resources elsewhere and to maintain and develop their activities (Slaughter & Leslie, 1997; Welsh & Glenna, 2006). Other authors stress the external pressures (governmental, economic, social) that oblige universities -the knowledge factoryto meet the challenges of contemporary societies that are faced with globalisation and are characterised by rapid technological changes, as well as high levels of unemployment or poverty (Aronowitz, 2000). For Bok (2003), the commercialisation of university products and activities must also be linked to a dynamic that is internal to the academic field whose values of competition and excellence require more and more resources.

It thus appears that the acceleration of economic promotion of research in American and European universities cannot be explained primarily by direct collaboration and hybrid forms of organisation between the public and private sector of research in the framework of a knowledge economy, but is more globally the result of high inter-dependencies between this sector of higher education and research and the economic and social demands that are partly built and relayed by the governmental sphere. Moreover, these promotion practices do not systematically develop in opposition to academic values, but, on the contrary, to support them. We propose to defend this perspective in order to analyse French government policy in matters of economic returns.

Economic Returns from the Standpoint of Public Authorities: between the justification of economic development and financial stakes

The French policy of support for innovation and public research is part of building a knowledge economy. But it is possible to go beyond this first stage of analysis by looking at public action not so much from the angle of its objectives but of its instruments, a position that defends the deconstruction of political activity (Lascoumes & Le Gales, 2004). A cross-analysis of the public measures and the criteria for their evaluation shows that the government policy of supporting tech-nological innovation in French universities underscores two logics: the development of a knowledge economy and the growth of the institutions' own resources.

Putting Public Research at the Service of a Knowledge Economy

Building a national and European knowledge economy -where the creation of wealth and the productivity of commercial activities are determined by human capital and innovative and technological products -is determined by the quantitative objective to bring the share of the gross domestic product devoted to research expenditure to 3% by 2010, 2% of which should come from the private sector. If we refer to Eurostat and OECD data, France invested 2.19% of its GDP in R&D in 2003, thanks to a rather high share of the public contribution (0.83% of the GDP), whereas the contribution of the private sector was rather low (1.36%). The corrective measures of the French government targeted firms (research tax credit, aids given by the OSEO group 11 , projects supported by the Agence de l'Innovation industrielle 12 ) as much as the public research sector, although the present study only deals with the latter.

The law of July 1999 13 on Innovation and Research was meant to make it easier for civil servant researchers to move to the private sector in order to create a firm, bring in scientific competence or hold shares in the capital of a firm. It also created favourable legal conditions, encouraged the implementation of regional incubators and launched a national support competition to create innovative technological firms. 28 incubators accredited by the State were created in all the French regions to host projects to help create innovative firms, mainly linked to public research. These incubators were financed by the Ministry (50%), the regions and Europe (European Social Fund). The calls for proposals help the project coordinators and mainly target doctorate holders, engineers and other young university graduates who wish to develop the work they have carried out in their laboratories. The regions associate themselves with this national policy in favour of technological innovation as it echoes their concerns about the qualification of regional and local economic systems. They contribute to the funding of interface structures between the firms and research teams such as regional centres for innovation and the transfer of industrial technology (CRITT) or technological zones and, more recently, the competitivity poles. They also create their own support measures for technological entrepreneurship, be it academic or not: incubators, calls for projects, competition for innovation, and support for project development.

Making the Public Sector Research Prosper and Profitable

In parallel to the growing focus on technological innovation and academic entrepreneurship in universities, a system of expertise and evaluation of practices in this field was introduced at the national level.The technological and economic value of the projects of the researchers-entrepreneurs thus comes second after the preservation and defence of intellectual and economic ownership of the public service of research.

a) The evaluation bodies of academic entrepreneurship

In the framework of the 1999 Law, the IGAENR (Inspection générale de l'administration de l'Éducation nationale et de la recherche) saw its evaluation competences extended to research 14 and the former CNE (Comité national de l'évaluation) 15 also integrated, as from 2003, evaluation criteria that enabled it to better appreciate whether the institutions that were evaluated gave special attention to promotion.Two bodies with substantial expertise in matters of entrepreneurship within the institutions came to the fore: the ethics commission; and the national audit office and, by delegation, the regional audit offices.

The ethics commission was created in the framework of the law of 1993 to complement the legislation on research defined by the Law of 1982. It became an essential pillar supporting the implementation of the 1999 Law on innovation and research, seeking to reconcile 'the needs for the public research staff to participate in the creation and development of firms with the general principles that guaranteed the regular functioning of the public services and the moral behaviour of their agents' 16 . Since then, all the requests of researcher-civil servants relating to the 1999 law must be presented to the commission, which gives an opinion. Researchers' projects are co-validated by their institution and the commission. The institution gives them authorisation, following the opinion of the commission. At a later stage, the commission must be informed of the contracts and conventions signed between the firm that has been created and the university.

The national audit office and, by delegation, the regional audit offices 17 , have become essential actors in the evaluation of promotion activities in the universities. The national audit office published reports on this topic, first in 1997, then in 2000 and decided in 2003 to launch a broader survey on the management of research activities in universities together with several regional audit offices.

b) The evaluation criteria of economic returns in universities Although entrepreneurial initiatives in universities are encouraged, they are also controlled according to precise criteria. As its name indicates, the ethics commission does not evaluate researchers-civil servants' projects on the basis of their economic and commercial pertinence 18 . But it does measure the compatibility between the commercial project and the researcher's civil servant status. This is a difficult problem because the very activity of academic entrepreneurship is always close to illegality.

The general code of the civil service forbids civil servants (non researchers) to exercise a lucrative private occupation, have any interests in a firm which is related to their administration and be a member of an administrative board or president of a public company -except if it is not-for-profit. In other words, everything the 1999 law allows with respect to innovation and research is forbidden by the general code of the civil service. This exceptional status of civil servants-researchers is justified by the specificity of their mission, which consists in taking part in the development of knowledge and the social and economic development of the country. The analysis of the ethics commission's reports shows that the moral principles of the civil service and the issue of the protection of public property are confusedly intertwined and that the researchers' duties towards the general community tend to be replaced by those those to their institution. In matters of production, management and use of patents and other research products, the law gives the university institutions broad prerogatives for them to be able to have their rights, including their financial rights, recognised. The intellectual property code specifies that, once a patent has been presented following an intervention during the execution of functions for the benefit of an employer, employers have the right to become the owner or beneficiary of all or parts of the rights attached to the patent that protects the invention of their employees. From then on, the academic researchers' inventions in the framework of their missions belong to the university or the agency for which they are carrying out their research. It is only if the institution decides not to proceed with the promotion of the invention that the researcher who is its author can benefit from its patrimonial rights according to the conditions of the convention that has been signed with the institution. Hence, researchers are not the main beneficiaries of their products -patents or other scientific products -whereas the intellectual property is an automatic right of public employers, unless they renounce it.

These principles of protection of the public heritage of research and defence of the financial interests of the university institutions are the object of a particularly dense jurisprudence in the reports of the commission 19 . The issue is first to verify whether the products to be promoted were produced in the framework of the public service research and in the institutions to which the researchers-civil servants are attached (Commission de déontologie, 2005). When the entrepreneurial demand concerns new work that is not part of a promotion process, the commission gives a negative opinion. The commission also evaluates whether it is the researchers making the request who produced the research activities they wish to enhance. Hence, the director of a thesis can promote the work of one of his doctoral students if his own work has an obvious link with this thesis.

Once the 'public' property character of the product to be promoted has been established, the commission makes sure that the researcher and a fortiori the institution are not unjustly dispossessed of their property in the framework of the promotion activity, both at the legal level -through the renouncing of patent ownership for example -and at the financial level -through contracts or concessions that under-rate the value of the product that is exchanged. Indeed, when researchers take the initiative to contribute to the commercial exploitation of their work, they not only commit themselves, but also their institution and more generally the community. Hence, a contract of transfer of know-how stipulating how promotion will be undertaken for each party must be established between the institution to which the researcher is attached and the partner firm. A contract of patent ownership must accompany it if need be and must specify the conditions of exploitation and financial reward for the institution. If the researcher's project foresees the transfer of several patents and licences to the partner firm -without having even envisaged a contract between the university and the firm -or if it concerns the transfer of all the know-how of a researcher accompanied by a just as important transfer of ownership rights by the university, with no or little counterpart, the commission will give an unfavourable opinion.

The conditions of reciprocity of the public-private exchange are another concern of evaluators who have given a negative answer to a request for scientific competition when they believed that the licence contract that accompanied it did not foresee the payment of a sum 'that could not be considered as sufficiently ensuring the enhancement of the research work (. . .) and could jeopardise the material and moral interests of the public service' 20 . It also contested the request to host enhancement activities, considering that the enhancement contract that accompanied it 'would only contribute a very weak financial counterpart to the university' 21 . Hence, the commission carefully studies the conditions of the contract that links the institution of the researcher and the firm and makes sure that there are 'balanced and not unfair clauses that benefit the firm' 22 . The transfer of ownership rights is evaluated with care. It must be accompanied by an annual financial counterpart for the institution and be justified, especially when the products (software, patents) have a clear commercial potential. The contracts must include strict protection clauses, such as the university's right to control the allowance of licences by firms to third parties. The financial counterpart that is granted to the university must appear in the know-how transfer contracts and the commission also specifies the rates of retribution of the university which must neither be a percentage that is taken out of the complement of payment of the researcher nor a percentage on the benefits -'whose consistence is not defined and whose amount may be low' 23 -but must be defined on the basis of the turnover. As a minimum guarantee must be respected, the commission may consider a rate of 1% of the turnover as too little and therefore give a reserved favourable opinion until this rate is considerably increased.

This evaluation grid of entrepreneurial practices within universities is valid for the national audit office and the regional audit offices, which highlights even more clearly the weaknesses of the management of economic returns in French universities. According to the audit office, universities often have little knowledge of their returns activities and leave their laboratories completely free in this matter 24 . The 'bad' pupils are the universities that tend to abandon their prerogatives in matters of patent management, or to treat them without enough rigour and a lack of authority regarding their components and staff. The universities, which, until recently, tended to systematically reject all commercial relations with the private partners are now explicitly encouraged to do the contrary.

Hence, in a specific case, the national audit office reproaches the university of 'abandoning' the intellectual property of certain patents, specifying that this was done 'without (. . .) any resistance' when the patent presented 'a true commercial potential' 25 . Moreover, the university in question has no means to oversee the use made of its goods by third parties. The audit office concludes from this that 'the university is disarmed in relation to the owners of patents who know of their exploitation and the profits that are generated' 26 . It takes into account the constraints that are associated with the management of patents and more especially their costs which could be dissuasive for the institutions. The deposit of a patent in France is evaluated between 4,500 and 7,600 euros in 1997, its extension to other countries costs 15,000 to 22,000 euros in Europe and its annual renewal between 400 and 3,000 euros, whereas the potential benefits do not come for several years. The universities of Aix-Marseille 2 and Paris 7 are the only ones that seemed to be able to balance their accounts in matters of patent management in 2005 27 . Hence, the regional audit offices specify that the institution must 'be careful in its use of its patent ownership rights' but that these rights 'must not be abandoned, especially if the patents have a generic field of application (. . .)' 28 .

The issue of equipping the national and regional economy to face the competition of emerging countries and other developed countries is an essential justification for French national policy in support of academic entrepreneurship, helped in this respect by the regional policies. But the analysis of the measures and means of the evaluation of academic entrepreneurship says more about the driver of this national policy. The economic and technological pertinence of the projects is not the subject of the evaluation. The entrepreneurial initiatives of the researchers are evaluated on the basis of principles of promotion, protection and defence of the intellectual and commercial interests of the public service of research in general and of the universities in particular and call upon criteria of judgement that are more often considered to be the privilege of the private sphere: importance given to the value of property, marketing of the products of public research, estimate of the costs and fair estimate of the market value of public goods, reciprocity of exchanges and protection of innovations.

Economic Returns from the Standpoint of Those Concerned

In matters of academic entrepreneurship, the public measures may seem disproportionate considering the volume of the practices. Indeed, a rapid assessment at the national level shows that the practice of economic activity remains very limited. The study of researchers-entrepreneurs sheds light on ordinary professional motivations that are in close coherence with ambitions of a scientific nature (intellectual emulation, publication . . .).

Entrepreneurship in French Universities

Despite widespread communication around the 1999 law on innovation and research and its entrepreneurial value, its social impact seems weak, given the limited use that is made of it by the target population 29 . The appropriation of the legal measures by the researchers-entrepreneurs shows that the stakes on the frontier between the public sector of research and the world of private enterprise do not lead to its dissolution. a) Researchers' requests for mobility 98 mobility requests were addressed to the Commission in 2005 and, between 1999 and 2005, they concerned 533 civil servants. If we take into account the number of teachers-researchers (55,673 FTE) and researchers (73,500 FTE), one can conclude that the 1999 law affects less than 0.5% of its target audience. Moreover, researchers do not adhere in the same way to the three possibilities for mobility offered by the 1999 law, i.e.:

-participation on a personal basis as an associate or director in the creation of a firm(creation of an economic activity); -scientific aid brought to a firm that ensures, within the framework of a contract with the university, the promotion of its work (reinforced consultancy); -participation, on a personal basis, on an administrative board or a monitoring council (double activity).

These three possibilities are part of individual entrepreneurial practices which are specific and have a different impact in terms of production of economic value.That is why we define them distinctly as 'creation of economic activity', 'reinforced consultancy' and 'double activity'. But they are not used in the same way by researchers, who prefer 'reinforced consultancy'. Researchers remain in their public institution, with their usual research and teaching activities and salary and only transmit their competences and intellectual knowledge. 84% of the requests made between 1999 and 2005, i.e. 372 persons, were orientated towards this option.The only difference with more traditional services offered by researchers that do not need the authorisation of the ethics commission is that this consultancy is on a long term basis and requires the active support of the researcher for the enhancement process of his work in the firm. Reinforced consultancy can therefore be exercised on a five-year basis that can be renewed several times and the researcher can devote up to one day a week to the firm. Moreover, intellectual participation and a financial participation in the social capital of the firm can be cumulated within the 15% limit authorised, thus acquiring the dividends from the activities if there are any. But he/she cannot play a strategic role in the firm.

Then come, but far behind, the requests for the 'creation of economic activities'but these are decreasing: 11% of the requests in 2005, i.e. 132 persons, as against 21% in 2002. This option leads the researcher to leave the public sector for two years, renewable twice, for a project to create a firm for a limited, or even a more sustainable period. Researchers are seconded. They continue to be paid but 'must cease all activity in the public sector of which they are a member' 30 . They must be able to devote their time to the creation of a firm, whilst avoiding a conflict of interest between the firm and the public institution.

Last, the option that is defined here as a 'double activity' is the least attractive -5.4% of the requests, i.e. 29 persons -because of the ambivalence of its status or the double competences it implies and for which the researcher is not necessarily prepared. More than the two previous options, this one can create a 'conflict of interests'. Whilst carrying out their usual missions of research and teaching, researchers take part in directing the firm. The justification of this transgression of the code of the civil service is to 'sensitise firms to innovation and make them more attentive to the progress of basic research and its applications' 31 . The purpose is therefore to encourage the entry of those who are aware of the issues around research in the management teams of French firms.

b) Measures accompanying academic entrepreneurship

The 1999 Law on innovation and research also contained a chapter on measures that accompany these entrepreneurial practices in public research institutions: national tenders to support the creation of high tech firms, incubators. But it should be noted that, whereas these measures were originally meant to promote research activities that originate in public research 32 or are linked 33 to it, their missions have been gradually broadened to projects that may come from the private sector and to support projects from the public service of research (researchers-civil servants, recently graduated doctors and engineers), as well as creative people who are not close to the academic world, such as the highly qualified unemployed (bac + 5 to bac + 8). This is the situation for the national tenders for aid for the creation of innovative firms of which only 41% of the winners in 2005 promoted the work of public research 34 whereas 50% hold a doctorate and 34% a degree in engineering. Out of the total number of winners, 39% were job seekers. Their number has doubled in six years (20% in 1999).The same is true for incubators, which, originally, were to promote the technologies that emerged from and were associated with public research, whereas the projects that have received support since 2000 (1415 at the end of 2005) are projects that are first linked to public research (51%) and then emerge from it (45%), or even come exclusively from the private sector without any cooperation being necessarily established with public research (4%). Moreover, the share of public research projects decreases each year (75% in 2001, 60% in 2002, 50% in 2003, 49% in 2004 and 45% in 2005). Almost all those who submit projects (91%) are graduates of long higher education studies (bac + 5 at least). 45% hold a doctorate and 36% an engineering degree. Of the 1446 project leaders who benefited from the help of incubators, only 229 were affected by the 1999 law, i.e. 16% of the total. Even if this share increases for the projects originating from public research(36%), it is clear that most people who submitted incubated projects in 2005 were not civil servants only. 50% came from public research institutions. They came from the industrial world (55%), mostly from small and medium-sized firms(38%) and large companies (17%). Of the 50% who did not come from public institutions, 33% were unemployed.

The broadening of the purposes of the measures to accompany innovation in public research institutions is probably due to several factors, four of which seem to be particularly significant. First, a lack of entrepreneurial projects from public research and a low demand for mobility on the part of the public research staff create an imbalance between a weak entrepreneurial demand and a broad offer in all the French regions. Secondly, with the crisis in the recruitment in public service research, the creation of firms constitutes a professional alternative for those who have held a doctorate or an engineering degree for more than one year. Thirdly, in a context of economic sluggishness associated with a high rate of unemployment, including for qualified people, the creation of firms constitutes an alternative. Fourth and last, one should not neglect the important role of the regions at every level in funding measures that encourage academic entrepreneurship and more especially their financial contribution to the incubators.They contribute up to 50% or even 75% to the budgets of the latter and their funding criteria only partly tie in with those of the Ministry. It is the criteria of the creation of economic activity, and more especially technological activity and of regional development that is the priority, whether the projects that are funded come from the public sector or the private sector, as this convention between a regional incubator and a region shows: 'the projects will be identified primarily in the economic activities of the region.They can emanate from academic or industrial extension, from individual initiatives or existing firms. Their origin can be endogenous or exogenous to the region'. 35 In other words, the institutional origin of the innovative project does not really matter, but the criterion of its implantation is crucial.

The Creation of Firms by Researchers-Entrepreneurs

The sociological curiosity is heightened when the social practices analysed are the least shared: are these 'exceptional practices' engendered by exceptional individuals? The general survey carried out is indeed limited but it brings an answer that deserves attention: for the researchers-entrepreneurs who were questioned, the creation of firms was closely linked to the scientific career and did not constitute an alternative, a parenthesis or a new start. But this strong interdependence between academic career and the entrepreneurial project presents an unknown element in the practice of economic returns in universities as a way of developing academic activities.

Christelle Manifet 291 a) Two firms set up by a French university

The survey reported on is of a limited nature in more ways than one. Carried out in July 2006, its aim was to collect the accounts of 'exceptional' researchers because they had taken part in the creation of a firm. It was decided to limit the survey to a small number of experiences whilst seeking to better understand the meaning of the testimonies and individual pathways. Hence the unity of place was emphasised by choosing two cases of firms being created in a single university which, for the sake of confidentiality will be called the blue University 36 . The two firms had acquired a certain longevity, which made it possible to reflect on the entrepreneurial process in its professional, institutional and political dynamic and on the obstacles the projects had encountered. Both promoted research in biology or process engineering. The survey was carried out by means of interviews.

The sample

Firm 1

Firm 2

• 2 teacher-researchers who had developed a project and minority shareholders (30%)

• 1 teacher-researcher who had developed a project and minority shareholder (15%) • 2 private associates, majority stakeholders (70%) • 6 associate private shareholders (85%) • 1 manager, 2 employees We shall begin by retracing the main stages followed by the researchersentrepreneurs who were interviewed.

Sylvain, a Professor of Biology who Set up a Firm

Sylvain is a researcher who created a firm but did not lead the project. The promotion of the work he carried out with his team consisted for many years in classical contracts with firms.When the firm had been established and had pursued its activities with difficulty, Sylvain wished to come back to this more classical formula, which seemed to him less strenuous. Sylvain learned to carry out research with a thesis director and in a laboratory and a university that were familiar with the industrial application of public research.The question developed in his thesis was a question of basic science: what is the use of water on a protein molecule? The day he defended his thesis an eminent member of his jury said to him: 'your method could have a certain success with industrialists'. Having become a senior lecturer, he wanted to promote his work and make his laboratory benefit from it. Taking into account the economic characteristics of its region, the team consulted the major regional firms which, although they were interested, did not want to take any risks. Two years went by and the two researchers were rather discouraged: 'we felt that the industrialists were not ready to take part in the innovative adventure'.

Finally, it was a firm manager outside their region who contacted them. He wanted to gradually convert the traditional industrial activity of his firm into a technological activity around acid molecules, which had been marginal until then.

It was with this entrepreneur that a first contract was signed whereby the laboratory would supply the samples that the firm would commercialise. One year after these promising beginnings, the firm had more and more problems in its heavy industry branch and had to accelerate its reconversion. Sylvain and his partner then committed themselves to giving another dimension to the project. They elaborated a European project for which they obtained 680,000 euros, half for their laboratory, a quarter for the firm and the rest for the third European partner who had been contacted on this occasion.The project was to build a machine that could produce on a large scale and had to act quickly because obtaining the European funds depended on engaging the expenses. The two academics rapidly made contact with consultancy offices to design this machine, and, despite several problems with their aid, finally succeeded.

The localisation of this machine was the first factor of tension between the two researchers-entrepreneurs and their industrial partner. The two researchers wanted to let their region benefit from it, according to a principle of reciprocity with regard to the firm support brought to them by the region, but also to monitor the use that was made of their discovery. The firm manager wanted his employees and his firm that was located elsewhere to benefit from it too. Legal and economic problems would soon end the conflict. The firm was put in receivership, which shifted the problem. The issue was no longer to decide as to the localisation of the machine and the future technological activity of the bankrupt firm, but to find new partners to pursue the promotion of the work of the two researchers. It was then in 2001, whilst seeking another solution and stimulated by a political context that encouraged academic entrepreneurship, that the idea of creating a firm came to the two partners. Thanks to the quality relationship they had built with the former manager, they contacted two of their former managers and quickly came together to set up a firm. These two private associates became major shareholders and the two researchers shared 30% of the capital of the firm. But this time there was greater tension in the partnership: 'we quickly realised that there was no reciprocity between our laboratory and the firm that had been set up. The two shareholders insisted that the firm was not making any profit, which was true . . . But were they doing their utmost?'. The two academics began to be suspicious, especially since one of the shareholders had in the meantime been recruited by another firm which was a competitor of the first industrial partner. For Sylvain, things were clear: 'they wanted the ownership of our patent (. . .).Their strategy was to ruin the firm'. But the two academics were not ready to give up the patent of their invention. The cooperation failed, and since then, the firm has been on standby.

Sylvain does not abandon his aim of promoting his process, but he wants to come back to more classical formulae: 'from the technological intervention, the rental of machines, the management of the intellectual property of our patent, the training of staff from the private sector, answering invitations to tender . . . We wanted to go back to a classical contract logic!'.

Karim, Professor of Physics, Head of a Laboratory and Creator of a Firm

Karim could be the ideal figure of the academic entrepreneur researcher. He created a firm to which he now brings his scientific support but his entrepreneurial activities do not stop there. He is head of a laboratory and is a co-inventor of a technological process that gave rise to no less than 17 patents whose economic applications are numerous and affect common consumer products. Karim has also contributed to the creation of several other firms that are carried by industrial partners or former students, three of which were rewarded at the national innovation contest. In short, Karim systematically thinks about how to give added value to the basic research carried out in his laboratory right up to the commercial outlets and organises all the activities of his laboratory with this purpose. This frame of mind can only satisfy his institution and ministry.Yet Karim's dynamism is upset by an organisational environment he perceives as hostile.

He began his career in a technological university, in an atmosphere that was favourable to promotion and, with his team, he co-invented the technology that would lead to some 20 patents. Solicited by the blue University, he accepted move there, feeling that it would be an ally. Yet he was soon disillusioned. Everything that was supported in his former environment now seemed to be criticised. Although the president of the university tended to be on his side, his authority was weak and above all he did not have the 'entrepreneurial' culture that characterised the management of his former institution. Obliged to cope on his own, Karim pursued his innovative activities going so far as to deposit a patent on his own. In the mid-1990s, he already caressed the thought of setting up a firm. All his activities led him to this, e.g. when he looked for an agency that could industrialise his technology, when he negotiated with his industrial partners the transfer of the licences or the deposit of patents, when the experts of the European Commission ensured him of the 'strategic' character for Europe of his work and encouraged him to install himself in the technological zones in Germany or Spain.

But the implementation of the 1999 law on innovation and research made the local situation evolve in a positive way. The regional incubator was created and his university now placed promotion at the heart of its activities. This was the window of opportunity that Karim was waiting for. He very quickly set up his firm and had a 15% participation in its capital. This was rather complex because it associated members of his family, financers and a local entrepreneur. For Karim, the originality of this enterprise was that its aim was not to produce consumer goods. It was at the confluent of research and industrial production because it was devoted to the industrialisation of the technological processes developed by his laboratory: 'I wanted an engineering structure that prolonged the work of the laboratory but on another scale and I wanted other structures for the production'.

The pressure from his peers in his institution runs through Karim's testimony. He is suspected of 'making money' when his activities flourish and then is strongly criticised when some of the projects go wrong. Karim feels isolated. The conclusions of an evaluation report on his entrepreneurial practices only increase these tensions. Not only his institution, but Karim himself are directly accused of lacking rigour in the management of his patents and contracts, of not taking into account the interests of his institution, and even of not respecting the deontological rules of his status to the benefit of his private and family interests. Facing adversity, Karim carries on and the firm he had created still exists after five years.

Sylvain and Karim's paths are very different. In one case, the entrepreneurial project is the result of a combination of circumstances. In the other, it is a previous driver. For one, the problems are linked to the partnership with the private sector, for the other to the social control of the local academic community. But for both, entrepreneurship is integrated in a personal academic and career project and in a collective enterprise of development of a team and a laboratory.

c) The motives of the researchers-entrepreneurs: making their activities and teams prosper and protecting their invention The personal, family and financial interests that these two researchers can obtain from their entrepreneurship do not seem to be the essential motive. Moreover, as the two experiences that have been analysed have shown, the personal investment is as important as the risks. First, the financial risks, as Sylvain said afterwards, not without humour: 'mind you, the loss in capital will enable us to reduce our taxes!'.Then, there are the professional risks, as is seen in Karim's marginalisation within his institution. At first, there may be a social predisposition or a difference with these researchers that link them to the Schumpertian model of the entrepreneur (Champenois, 2006, p. 364). It is Karim who highlights the difficulties he has in sharing his ideas: 'an academic should not do this type of thing'. It is Sylvain who, from the outset of the interview, positions himself in an unusual academic path: 'I did try a general licence but I gave up after six months'.Yet both these entrepreneurs consider themselves as researchers and their scientific production, expressed in terms of publications, shelters them from all criticisms.The pertinence of economic returns is not questioned in either Sylvain's or Karims' case and seems to constitute a necessary element of their creative capacity and therefore of their scientific recognition.That is why when entrepreneurial activity can endanger associated scientific interests, such as the appropriation of a patent by a private actor, the researcher prefers to renounce, regardless of the considerations for the economic success of the partner. That is also why when the local academic community marginalises it because of its entrepreneurial activities, the researcher-entrepreneur accepts it with difficulty and compensates through the search for international scientific recognition.

But it is true that academic entrepreneurship differs from applied research, technological transfer or the public-private partnership. Two reasons account for Sylvain and Karim's entrepreneurial initiatives: to support their team and to protect their invention.

The driver for entrepreneurship is the need to nurture both one's own research activities and that of one's team. The commercial interest is obvious but remains a means to pursue research activities. The two interviews evoke the fact that the research community, through state redistribution, does not cover the entire needs of contemporary research structures which must find other ways to finance their activities and strengthen their teams. Awareness of the collective interest is strong, although it does not apply to the impersonal whole that makes up the public sector of research. The whole is not the institution but the laboratory or even the team which is a true enterprise that must be made to prosper for the benefit of all others.

The other essential reason is to protect one's inventions. Our two researchersentrepreneurs are also researchers-inventors who have a keen sense of their intellectual property. Although they are newcomers in market negotiations, they know full well how to protect their intellectual property. Their fears seem somewhat irrational. Indeed, the invention has usually led to a patent and it is enough to give a licence to frame the exploitation of their inventions by the industrialists. The classical promotion process, however, leads to a form of loss of control of the invention.The industrialists, who are also very concerned by the ownership of their production processes, often ask for exclusive licences, which reduces the freedom to exploit the patents. Moreover, there is always the risk of opportunistic and/or disloyal behaviour where, for example, a firm must pay the royalties of the exploitation of the licences. By involving themselves directly in the economic promotion of their inventions, the researchers have the feeling that they control the process. The outcome of Sylvain's experience shows, however, that reality is more complex and that participation in the creation of a firm does not protect the invention, the researcher, the laboratory and the institution from opportunistic and disloyal behaviour or commercial failures.

In this last section, we have looked at the entrepreneurial practices of academic researchers first from a statistical and general standpoint and then from a comprehensive standpoint. The first helped to highlight three elements -the limited involvement in entrepreneurial practices of staff of French public research, the differentiated appropriation of the possibilities offered by law, the shift in missions of the public measures to support innovation in public research. From these, we can draw more general lessons. First, marginal practices in universities cannot illustrate alone a global transformation of the sector.The development of a political offer of support to individual entrepreneurial practices in universities does not express itself in a large increase in these practices, but rather in better organisation and institutional embedding. Then, the researchers who turn to entrepreneurial activities privilege those that maintain the boundaries between the sectors of public research and private enterprise, rather than formulae that encourage hybrid practices. The comprehensive outlook is a useful complement to these general results. First, although entrepreneurial practices are exceptional and concern above all the researchers-inventors, they also reveal rather ordinary scientific strategies. The creation of firms is part of an individual scientific career, as well as of a collective enterprise, that of a team of researchers.

Conclusion

In this article, we wanted to link academic entrepreneurship to a series of activities in the field of economic promotion of public research that can take on different meanings according to the actors in the production chain. For the ministry and its agencies, economic promotion oscillates between two alternatives while trying a policy of support for an innovative and technological economy and a managerial policy to make the public service of research more profitable. For the regions, the economic promotion of public research means creating qualified and regional economic activity. For the researchers-entrepreneurs, it generates money and nurtures the scientific production and publication process. The creation of firms is also, more specifically, a way of both protecting and exploiting the individual and collective benefits of an invention.

Whoever wishes to interpret the activities of the economic promotion of public research cannot ignore the diversity of the modes of qualification it gives rise to nor the weight of strictly political and academic logics in their development. These activities therefore do not correspond to the use of the public research potential in the service of economic development and job creation but to the commercialisation of knowledge, i.e. professional practices and modes of organisation of the public research space which, in certain conditions, analyse knowledge as a private good, seeking to make profitable the costs of production and reap the benefits. The indices gathered here have shown that there is always an academic or administrative issue at stake. As it has been shown elsewhere (Champenois, 2006), identifying an opportunity for the industrial application of the products of research is the source of a circuit to fund research. More significantly, the testimonies of the researchers-entrepreneurs seem to show the almost necessary nature of economic promotion as if the scientific abilities of the researchers depended on it. On the side of the government agencies, the researchers' enterprise is evaluated on its capacity to protect and defend the products of public research rather than on the pertinence of the economic model. Although the entrepreneurial spirit, autonomy and the personal ability to take risks are encouraged, researchers-entrepreneurs are far from being dispensed from all collective responsibility regarding the 'firmuniversity' whose interests they must defend in the market.

Hence, the debate on the birth of an economy that is based on knowledge, that is fabourable to hybrids and the insertion of research, be it public or private, in the very processes of economic production may have evacuated another debate on the autonomous and parallel transformation of the public sector of research. The concepts of academic capitalism (Slaughter & Leslie, 1997) and commercialisation (Bok, 2003) aim at incarnating this ambiguous transformation that is characterised by the partial transfer and strategic appropriation of mercantile logics within science and university organisations (Tuunainen, 2005) and that contributes to maintaining and recreating rather than erasing the boundaries between the research and industry sectors, and the public and private sector.

11. The OSEO group was created on 17 January 2005 following the closening of the competences of the Banque de développement des PME and its subsidiary SOLARIS and Anvar in order to increase the support of public research to the small and medium-sized firms. 12. The Agence de l'innovation industrielle (AII), which was created in 2005, aims to finance large mobilising research programmes that involve public research laboratories and large industrial groups and small and mediumsized firms. 13. The possibilities for researchers-civil servants to be mobile already existed in the loi d'orientation de l'enseignement supérieur de 1982. But the 1999 Law clearly illustrates the government's wish to promote and simplify these practices. 14. Proof of this is the latest report of the IGAENR, Rapport sur la valorisation la recherche au sein des universités, 2007, 250 p. 15. The mission of the Comité national d'Évaluation de l'enseignement supérieur (CNÉ) is to evaluate all scientific, cultural and professional institutions. In the framework of the reform of the national administration of research and higher education it has been fragilised and then integrated as from January 2007 in the Agence d'évaluation de l'enseignement supérieur et de la recherche (AERES), which was created in 2006. 16. Extract from the circular note of 7 October 1999 concerning the implementation of the measures of the law of 12 July 1999. 17. A delegation that the regional audit offices obtained with their creation in the law on political decentralisation of March 1982 concerning the rights and freedom of the communities, departments and regions. 18. This is mainly the role of incubators and competitive examinations. 19. The commission can give the following opinions: 'favourable', 'favourable with reserves', 'negative', 'negative as it stands', 'postponed', 'withdrawal' or 'rejection'. In 2005, it gave 98 opinions, most of which were favourable or favourable with reserves (76%). 20. Commission de déontologie, Rapport d 'activité, 2005, p. 71. 21. Ibid. 22. Ibid. 23. Ibid. 24. Cour des comptes, La gestion de la recherche dans les universités, octobre 2005, op. cit.: pp. 99-107 and pp. 148-151. 25. Extract from university evaluation reports carried out by the regional audit offices. We do not specify the references out of respect for the anonymity of the institutions and persons concerned. 26. Ibid. 27. Cour des comptes, La gestion de la recherche dans les universités, octobre 2005, op. cit., p. 104. 28. Extract from an evaluation report of a regional audit office, op. cit. 29. The deontology commission carries out annual reports on its submissions of cases to the court since 2000. 'Normally', it is the compulsory reference for all requests that refer to the 1999 law. Its cover improved as the institutions and staff better mastered the texts. The former directorate for technology of the ministry also carried out a state of the art of the measures and activities in matters of innovation and technological research in 2006: Direction de la

  • 1 manager, 3 employees • In activity for 3 years, on hold for 2 years (at the time of the survey in July 2006) • In activity for 5 years b) Two examples of researchers-entrepreneurs

European Journal of Education © 2008 The Author. Journal compilation © 2008 Blackwell Publishing Ltd. 14653435, 2008, 3, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/j.1465-3435.2008.00360.x by Universite Toulouse II Jean Jaures, Wiley Online Library on [09/02/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License

© 2008 The Author. Journal compilation © 2008 Blackwell Publishing Ltd.14653435, 2008, 3, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/j.1465-3435.2008.00360.x by Universite Toulouse II Jean Jaures, Wiley Online Library on [09/02/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License

European Journal of Education © 2008 The Author. Journal compilation © 2008 Blackwell Publishing Ltd. 14653435, 2008, 3, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/j.1465-3435.2008.00360.x by Universite Toulouse II Jean Jaures, Wiley Online Library on [09/02/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License

European Journal of Education © 2008 The Author. Journal compilation © 2008 Blackwell Publishing Ltd. 14653435, 2008, 3, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/j.1465-3435.2008.00360.x by Universite Toulouse II Jean Jaures, Wiley Online Library on [09/02/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License

European Journal of Education © 2008 The Author. Journal compilation © 2008 Blackwell Publishing Ltd. 14653435, 2008, 3, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/j.1465-3435.2008.00360.x by Universite Toulouse II Jean Jaures, Wiley Online Library on [09/02/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License

European Journal of Education © 2008 The Author. Journal compilation © 2008 Blackwell Publishing Ltd. 14653435, 2008, 3, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/j.1465-3435.2008.00360.x by Universite Toulouse II Jean Jaures, Wiley Online Library on [09/02/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License

. Extract from a convention between an incubator and a region that was subject to a general investigation and whose anonymity we wish to preserve. 36. At the beginning, the survey covered three cases, including a firm that closed down after three years. It was not possible to contact the doctorate holder and the engineer who had created it.


domain: qfin math stat

text:
Pierre Bras
email: pierre.bras@sorbonne-universite.fr

Gilles Pagès
email: gilles.pages@sorbonne-universite.fr.

Langevin algorithms for Markovian Neural Networks and Deep Stochastic control

Keywords: Langevin algorithm, SGLD, Markovian neural network, Stochastic control, Deep neural network, Stochastic optimization

Stochastic Gradient Descent Langevin Dynamics (SGLD) algorithms, which add noise to the classic gradient descent, are known to improve the training of neural networks in some cases where the neural network is very deep. In this paper we study the possibilities of training acceleration for the numerical resolution of stochastic control problems through gradient descent, where the control is parametrized by a neural network. If the control is applied at many discretization times then solving the stochastic control problem reduces to minimizing the loss of a very deep neural network. We numerically show that Langevin algorithms improve the training on various stochastic control problems like hedging and resource management, and for different choices of gradient descent methods.

Introduction

Stochastic Optimal Control (SOC), which consists in optimizing a functional of a trajectory of a controlled Stochastic Differential Equation (SDE) has applications in a wide range of problems: management of resources, queuing systems, epidemic and population processes, pricing of financial derivatives, portfolio allocation... In comparison with classic optimal control, SOC models include a random noise with known probability distribution that affects the evolution or the observation of the system. SOC also aims at managing the risk induced by this noise.

SOC problems are usually solved using specific strategies, such as Forward-Backward SDEs (FBSDEs) [START_REF] Peng | Fully coupled forward-backward stochastic differential equations and applications to optimal control[END_REF], or by solving Hamilton-Jacobi-Bellman (HJB) optimality conditions [START_REF] Bellman | Dynamic programming[END_REF] through partial differential equations methods using appropriate numerical schemes or by stochastic dynamic programming [START_REF] Kushner | Numerical methods for stochastic control problems in continuous time[END_REF]. Such problems can also be solved using Neural Networks calibrated by SGD techniques [GM05, HE16, WLP + 19, CL21, BHLP22].

More specifically, in this article we consider the numerical resolution of a SOC problem where the control is parametrized by a neural network calibrated by gradient descent. This method implies to compute the pathwise derivatives along the trajectory of the SDE of the objective function with respect to the parameters of the neural network, as introduced in [START_REF] Giles | Smoking adjoints: fast evaluation of greeks in monte carlo calculations[END_REF][START_REF] Michael | Monte Carlo evaluation of sensitivities in computational finance[END_REF]. Stochastic gradient descent is a very general approach that can be applied to a wide range of problems and which does not need to be specifically adapted to each problem under consideration. Moreover, SGD scales very efficiently to high-dimensional problems, in contrast with HJB-based methods, and has proved its efficiency on highly non-convex problems [START_REF] Dauphin | Equilibrated adaptive learning rates for non-convex optimization[END_REF].

However, if the neural control is applied at many time steps as it is the case for the Euler-Maruyama scheme where the (discrete) control is taken as an approximation of a continuous control, then the SOC problem reads as the optimization of a very deep neural network, which is roughly as deep as the number of instants at which the control can be applied (see Figures 1 and2). Very deep neural networks are known to be considerably more difficult to train [GB10, DPG + 14] and may run into vanishing gradient problems [START_REF] Hochreiter | Untersuchungen zu dynamischen neuronalen netzen[END_REF][START_REF] Hanin | Which neural net architectures give rise to exploding and vanishing gradients?[END_REF]. Indeed, the deeper the neural network is, the more non-linear it is, thus increasing the number of local traps for the gradient descent such as local (but not global) and saddle points. In image analysis where very deep convolutional neural networks are commonly used, residual [START_REF] He | Deep residual learning for image recognition[END_REF] and convolutional dense [START_REF] Huang | Densely connected convolutional networks[END_REF] networks were introduced to deal with this issue. These networks are based on architectures with residual connections to propagate the gradient information through the numerous successive layers.

As it comes to deep SOC, we cannot freely change the structure of the neural network since it is fixed by the equations defining the SOC problem and therefore we cannot directly use residual connections. We can only freely choose the structure of the neural network returning the control, for which a few layers is often enough (see for example [START_REF] Buehler | Deep hedging[END_REF][START_REF] Bachouch | Deep neural networks algorithms for stochastic control problems on finite horizon: numerical applications[END_REF]).

A way to improve the learning is to replace SGD algorithms by Stochastic Gradient Langevin Dynamics (SGLD) algorithms. Such optimizers add an exogenous white noise to the gradient descent, providing regularization and allowing to escape from traps. It has indeed been observed that adding noise improves the learning for very deep neural networks [NVL + 15, Ani19, GMDB16, SLH + 19]. Moreover, [START_REF] Bras | Langevin algorithms for very deep Neural Networks with application to image classification[END_REF] compares side-by-side Langevin with non-Langevin algorithms on networks with increasing depth and shows that for shallow neural networks, Langevin algorithms do nothing else than adding noise to the gradient descent, however the deeper the network is, the greater the gains provided by Langevin algorithms are.

In the present article we study the performances of Langevin optimizers on SOC problems where the number of discretization times where the control is applied is large enough. We use the preconditioned versions of SGD and SGLD [START_REF] Li | Preconditioned stochastic gradient langevin dynamics for deep neural networks[END_REF] for various choices of preconditioners. We compare side-by-side Langevin and non-Langevin algorithms and we show that Langevin optimizers can significantly improve the training procedure on various problems: fishing quotas [START_REF] Mathieu Laurière | Performance of a Markovian neural network versus dynamic programming on a fishing control problem[END_REF], deep hedging [START_REF] Buehler | Deep hedging[END_REF], oil drilling and resource management [START_REF] Gaïgi | Optimal risk management problem of natural resources: application to oil drilling[END_REF]. We mainly consider two different approaches for numerical resolution of SOCs. In the first approach, the control is a single neural network which is applied to every time step and which may depend on the running time t (see Figure 1). This approach leads to a model with fewer trainable parameters, which is critical in some data-driven financial applications where the amount of data is limited, and which is more able to capture the specific Markovian features of the problem. In the second approach, a different neural network is used for each control time (see Figure 2). This last setting is also suitable for applying the Layer Langevin algorithm, which is a variant of the Langevin algorithm introduced in [START_REF] Bras | Langevin algorithms for very deep Neural Networks with application to image classification[END_REF] and which proved to be more adapted to the training of very deep neural networks than the Langevin algorithm itself.

We observe that the gains of Langevin algorithms depend on the preconditioner however. While the Adam [KB15] and the Adadelta [START_REF] Matthew | ADADELTA: An Adaptive Learning Rate Method[END_REF] algorithms can be substantially accelerated by Langevin training, the gains are more limited or sometimes null for RMSprop [START_REF] Tieleman | Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent magnitude[END_REF]. The code for the numerical experiments is available at https://github.com/Bras-P/langevin-for-stochastic-contro It includes in particular ready-to-use Langevin optimizers and Layer Langevin optimizers as instances of the TensorFlow Optimizer base class, a framework for algorithm comparison in a SOC setting with GPU support and a demonstration notebook.

Notations: For x and y two vectors we define the Schur product x * y as the vector (x i y i ) i . We also sometimes use the notation * for scalar-vector multiplication. For a, b ∈ N we denote M a,b (R) the set of a × b matrices with real-valued coefficients. For x ∈ R we define the positive part of x denoted x + as max(x, 0). We consider multivariate (F t )-Brownian motions W and B defined on some filtered probability space (Ω, F , (F t ) t∈[0,T ] ), P).

Stochastic control through gradient descent 2.1 Stochastic optimal control

We consider the following SOC problem in continuous time:

min u J(u) := E T 0 G(t, X t )dt + F (X T ) ,
(2.1)

dX t = b(X t , u t )dt + σ(X t , u t )dW t , t ∈ [0, T ] (2.2) where b : R d1 × R d3 → R d1 , σ : R d1 × R d3 → M d1,d2 (R), W is a R d2 -valued Brownian motion and u is a R d3 -valued continuous adapted process, T > 0, G : [0, T ] × R d1 → R and F : R d1 → R.
We first approximate the continuous SDE (X t ) t∈[0,T ] with its Euler-Maruyama scheme and the control u t with a discrete-time control. For N ∈ N being the number of discretization times, we consider the regular subdivision of [0, T ]:

t k := kT /N, k ∈ {0, . . . , N }, h := T /N (2.3)
and we approximate the control applied at times t 0 , . . . , t N -1 as the output of a single neural network depending on t, or as the output of N neural networks, one for each discretization instant t k :

u t k = ūθ (t k , X t k ) or u t k = ūθ k (X t k ) (2.4)
where ūθ is a neural function with finite-dimensional parameter θ ∈ R d . Indeed, since (2.2) defines a Markovian process, we can assume that u t depends only on t and on X t instead of t and (X s ) s∈[0,t] .

The SOC problem (2.1) is numerically approximated by:

min θ J(ū θ ) := N -1 k=0 (t k+1 -t k )G(t k+1 , Xθ t k+1 ) + F ( Xθ tN ), (2.5) Xθ t k+1 = Xθ t k + (t k+1 -t k )b Xθ t k , ūk,θ ( Xθ t k ) + t k+1 -t k σ Xθ t k , ūk,θ ( Xθ t k ) ξ k+1 , (2.6) ξ k ∼ i.i.d. N (0, I d2 ) (2.7)
where θ ∈ R d and ūk,θ = ūθ (t k , •) in the first case of (2.4) and θ = (θ 0 , . . . , θ N -1 ) ∈ (R d ) N and ūk,θ = ūθ k in the second case of (2.4).

For every θ, ∇ θ J can be computed by automatic differentiation as the gradient w.r.t. to θ is tracked all along the trajectory through the recursive relation (2.6) [START_REF] Giles | Smoking adjoints: fast evaluation of greeks in monte carlo calculations[END_REF][START_REF] Michael | Monte Carlo evaluation of sensitivities in computational finance[END_REF]. Then the SGD algorithm reads

θ n+1 = θ n -γ n+1 1 n batch n batch i=1 ∇ θ J(ū θn , (ξ i,n+1 k ) 1≤k≤N ) =: θ n -γ n+1 g n+1 (2.8)
where (ξ i,n k ) 1≤k≤N,1≤i≤n batch ,n∈N is an array of i.i.d. random vectors N (0, I d2 )-distributed, (γ n ) n∈N is a nonincreasing positive step sequence and where the dependence of J in (ξ i,n k ) is made explicit. If the number of Euler-Maruyama steps N is large, then the optimization problem in (2.5) consists in the training of a very deep neural network that can be difficult to train directly (see the Introduction). Both cases are illustrated in Figures 1 and2.

Preconditioned stochastic gradient Langevin dynamics

We consider preconditioned stochastic gradient algorithms i.e. for (P n ) a preconditioner rule the update reads

θ n+1 = θ n -γ n+1 P n+1 • g n+1 (2.9)
where g n+1 is defined in (2.8). We use the Adam [START_REF] Diederik | Adam: A method for stochastic optimization[END_REF], the RMSprop [TH12] and the Adadelta [START_REF] Matthew | ADADELTA: An Adaptive Learning Rate Method[END_REF] preconditioners, which are detailed in Algorithms 1, 2 and 3 respectively. For some algorithm name, the corresponding Langevin algorithm denoted L-name reads

θ n+1 = θ n -γ n+1 P n+1 • g n+1 + σ n+1 √ γ n+1 N (0, P n+1 ) (2.10)
where (σ n ) is a constant or non-decreasing sequence controlling the amount of injected noise.

Algorithm 1 Adam update Parameters:

β 1 , β 2 , λ > 0 M n+1 = β 1 M n + (1 -β 1 )g n+1 MS n+1 = β 2 MS n +(1 -β 2 )g n+1 ⊙ g n+1 M n+1 = M n+1 /(1 -β n+1 1 ) MS n+1 = MS n+1 /(1 -β n+1 2 ) P n+1 = diag 1 ⊘ λ1 + MS n+1 θ n+1 = θ n -γ n+1 P n+1 • M n+1 . Algorithm 2 RMSprop update Parameters: α, λ > 0 MS n+1 = α MS n +(1 -α)g n+1 ⊙ g n+1 P n+1 = diag 1 ⊘ λ1 + MS n+1 θ n+1 = θ n -γ n+1 P n+1 • g n+1 Algorithm 3 Adadelta update Parameters: β 1 , β 2 , λ > 0 MS n+1 = β 1 MS n +(1 -β 1 )g n+1 ⊙ g n+1 P n+1 = diag (λ1 + MS n ) ⊘ λ1 + MS n θ n+1 = θ n -γ n+1 P n+1 • g n+1 . MS n+1 = β 2 MS n +(1 -β 2 )(θ n+1 -θ n ) ⊙ (θ n+1 -θ n ).
The Layer Langevin algorithm, introduced in [Bra22] consists in updating with Langevin noise only some layers of the network. It relies on the heuristic that for a deep neural network, the non-linearities of the network are mostly contained in the deepest layers and adds Langevin noise to these layers only.

Choosing a preconditioner rule P called name, the Layer Langevin algorithm denoted LL-name reads

θ (i) n+1 = θ (i) n -γ n+1 [P n+1 • g n+1 ] (i) + 1 i∈J σ n+1 √ γ n+1 N (0, P n+1 ) (i) , (2.11)
where J is a subset of weight indices. In particular, we denote LL-name p% the Layer Langevin name algorithm where the Langevin layers are chosen to be the first p% layers.

u • • • X t k SDE X t k+1 SDE X t k+2 • • • G G G • • • + + + • • •
Figure 1: Depth of Markovian neural networks. The control u acts on X t k , which itself acts on X t k+1 , X t k+2 , . . ., X tN , hence the depth of the network.

u t k u t k+1 • • • X t k SDE X t k+1 SDE X t k+2 • • • G G G • • • + + + • • • Figure 2:
Markovian neural network with one control for every time step.

Experimental setting

We proceed to side-by-side comparison of Langevin algorithms (2.10) with their non-Langevin counterparts (2.9) on various SOC problems. We consider a first case where the control is given by only one neural network depending on t and a second case where a different neural network is used for each control time (2.4). In this second case, since we can expect two consecutive control networks to have close parameters, one usual way of performing the training procedure is to first train networks for a small amount of control times, then to perform the whole training through transfer learning. We do not expect Langevin algorithms to be suitable for the fine tuning, but since the first step of the training still consists in training a deep neural network, we analyse the benefits of Langevin algorithms for this first step.

Unless stated otherwise, the batch size is set to n batch = 512 i.e. stochastic gradient iterations are performed by averaging the gradient over 512 random trajectories. In the plot, each epoch consists in 5 batches i.e. the average loss J(u θ ) is plotted every 5 iterations of the stochastic gradient. After each epoch, J(u θ ) is estimated over 25 × 512 trajectories and 95% confidence intervals are indicated, although for some plots the intervals are too small to be visible. While comparing some algorithm with its Langevin or Layer Langevin counterpart, we ensure that both training procedures start with the same initial weights.

Fishing quotas

We consider the fishing quota problem introduced in [START_REF] Mathieu Laurière | Performance of a Markovian neural network versus dynamic programming on a fishing control problem[END_REF]. Let X t ∈ R d1 be the fish biomass for every fish species; we wish to keep it close to an ideal state X t ∈ R d1 . The dynamics of X are given by

dX t = X t * ((r -u t -κX t )dt + ηdW t ) , t ∈ [0, T ], (3.1)
where r ∈ R d1 is the growth rate for each species, u t ∈ R d1 is the controlled fishing (with

d 3 = d 1 ), κ ∈ M d1,d1 (R)
is the interaction matrix between the fish species, η ∈ M d1,d2 (R), W is a R d2 -valued Brownian motion. The Figure 3: Example of a trajectory of X t ∈ R 5 along with the controlled fishing u t with N = 100. We recall that the objective biomass is X t ≡ 1.

control u is constrained to take its values in [u m , u M ] d1 . The objective is

J(u) = E T 0 (|X t -X t | 2 -α, u t )dt + β[u] 0,T , (3.2)
where α ∈ R d1 , β ∈ R + , [u] 0,T denotes the quadratic variation of u on [0, T ]. The term α, u penalizes small fishing quotas while the term β[u] 0,T penalizes too many daily changes.

In the experiments, following [START_REF] Mathieu Laurière | Performance of a Markovian neural network versus dynamic programming on a fishing control problem[END_REF] we choose

d 1 = d 2 = 5, T = 1, X ≡ 1, r = 2 * 1, η = 0.1 * I d1 , α = 0.01 * 1, β = 0.1, u m = 0.1, u M = 1. (3.3) and κ =       1.2 -0.1 0 0 -0.1 0.2 1.2 0 0 -0.1 0 0.2 1.2 -0.1 0 0 0 0.1 1.2 0 0.1 0.1 0 0 1.2       . (3.4)
The initial state X 0 is randomly generated following N 1, (1/2)I d1 clipped to [0.2, 2] d1 . The quadratic variation [u] 0,T is approximated in the discretized setting by

[u] 0,T ≃ N -1 k=0 |u t k+1 -u t k | 2 . (3.5)
Each control u θ is given by a feedforward neural network with two hidden layers with 32 units each and with ReLU activation while the output layer has sigmoid activation in order to fulfil the constraint on u. An example of controlled trajectory is plotted in Figure 3. The results are given in Figure 4 for the Adam optimizer with an increasing number of Euler-Maruyama steps and with one single control, in Figure 5 for the RMSprop and L-Adadelta optimizers and with one single control and in Figure 6 for the training with multiple neural networks.

Deep hedging

We consider the problem of hedging portfolio of derivatives as a SOC problem as in [START_REF] Buehler | Deep hedging[END_REF]. We aim to replicate a F T -measurable payoff Z defined on some portfolio S t ∈ R d1 by trading (at least some of) the assets contained in S t at times (t k ). The control is given by u t ∈ R d1 representing the amount held for each asset. The objective is

J(u) = ν -Z + N -1 k=0 u t k , S t k+1 -S t k - N k=0 c tr , S t k * |u t k -u t k-1 | (4.1)
where ν : L 1 (Ω) → R is a convex risk measure (see [BGTW19, Definition 3.1]), c tr ∈ R d1 represents proportional transaction costs and we fix u t-1 = u tN = 0, implying full liquidation in T . We furthermore assume that ν can be written as (1 e -2) for RMSprop (Adadelta resp.) for epochs 0 to 40 and γ n is divided by 10 and σ n is set to 0 beyond. At the end of each epoch, J is estimated over 50 × 512 trajectories. A zoom on the last epochs for RMSprop is given. where the loss function ℓ : R → R is continuous, non-decreasing and convex. This is the case in particular for the entropic risk measure where ℓ(x) =exp(-λx) and the conditional value at risk measure where ℓ(x) = (1α) -1 max(x, 0). Then (4.1) can be rewritten as inf u,w J(u, w)

ν(X) = inf w∈R (w + E[ℓ(-X -w)]) (4.2)
:= E w + ℓ Z - N -1 k=0 u t k , S t k+1 -S t k + N k=0 c tr , S t k * |u t k -u t k-1 | -w . (4.3)
In the numerical experiments, we analyse the problem of hedging in a Heston model as described in [BGTW19, Section 5]. For even d 1 , we consider d ′ 1 := d 1 /2 independent Heston models where the price and volatility processes are described by the following SDEs for 1 ≤ i ≤ d ′ 1 :

dS 1,i t = V i t S 1,i t dB i t , S 1,i 0 = s i 0 , (4.4)
dV i t = a i (b i -V i t )dt + η i V i t dW i t , V i 0 = v i 0 , (4.5)
where a, b, η, s 0 , v i ∈ (R + ) d ′ 1 and for each 1 ≤ i ≤ d ′ 1 , B i and W i are standard Brownian motions with correlation ρ i ∈ [-1, 1]. The volatility V itself is not tradable directly but only through options on variance modelled by the following variance swap:

S 2,i t := E T 0 V i s ds F t = t 0 V i s ds + L i (t, V i t ), (4.6)
L i (t, v) := v -b i a i 1 -e a i (T -t) + b i (T -t). (4.7)
The payoff is given by

Z = d ′ 1 i=1 S 1,i T -K i + where K ∈ (R + ) d ′ 1 .
We consider the convex risk measure associated to the value-at-risk i.e. associated to the loss function ℓ(x) = 1 1α max(x, 0).

In the experiments we choose

d ′ 1 = 5, T = 1, a = 1, b = 0.04 * 1, η = 2 * 1, ρ = -0.
7 * 1, α = 0.9, (4.8)

s 0 = K = 1, v 0 = 0.1 * 1, c tr = 5 e -4 * 1. (4.9)
Each control u θ is given by a feedforward neural network with two hidden layers with 32 units each and with ReLU activation while the output layer has ReLU activation too in order to forbid short-selling. As recommended in [START_REF] Buehler | Deep hedging[END_REF], since transaction costs are involved the control u θ at time t k is a function of log(S 1 t k ), V t k and u t k-1 . An example of controlled trajectory showing only one of the five Heston models is plotted in Figure 7.

The results are given in Figure 8 for the comparison of Langevin and non-Langevin algorithms with a single control and in Figure 9 for the training with multiple controls.

Oil drilling

We consider the control problem in the management of natural resources applied to oil drilling introduced in [GKL18] and extended in [START_REF] Gaïgi | Optimal risk management problem of natural resources: application to oil drilling[END_REF]. The objective is for an oil driller, to balance the costs of extraction, storage in a volatile energy market. The oil price P t ∈ R is assumed to be a Black-Scholes process:

dP t = µP t dt + ηP t dW t .
(5.1)

The control is given by q t = (q v t , q s t , q v,s t ) ∈ R 3 where q v t is the quantity of extracted oil immediately sold on the market per time unit, q s t is the quantity of extracted oil that is stored per time unit, q v,s t is the quantity of stored oil that is sold per time unit. The cumulated quantities of extracted and stored oil at time t are respectively given by E t = t 0 (q v r + q s r )dr, S t = t 0 (q s rq v,s r )dr.

(5.

2)

The extraction and storage prices are respectively given by c e (E t ) = exp (ξ e E t ) , c s (S t ) = exp (ξ s S t ) -1.

(5.

3)

The constraints on the control are the following: q v t , q s t , q v,s t ≥ 0, q v,s t ≤ q S , q v t + q s t ≤ K 0 , 0 ≤ S t ≤ Q S , (5.4) where q S , K 0 and Q S are operational bounds. The objective is J(q) = -E T 0 e -ρr U q v r P r + q v,s r (1ε)P r -(q v r + q s r )c e (E r )c s (S r ) dr , (5.5)

where U : R → R is the utility function.

In the experiments we take T = 1, µ = 0.01, η = 0.2, ρ = 0.01, ε = 0, K 0 = 5, ξ e = 1 e -2, ξ s = 5 e -3, q S = 10, P 0 = 1, U (x) = x.

(5.6)

The control q t is given by a feedforward neural network with two hidden layers with 32 units and with ReLU activation while the output layer has several ReLU activations such that the constraints on q (5.4) are fulfilled1 . An example of controlled trajectory is given in Figure 10.

The results are given in Figure 11 for the comparison of Langevin and non-Langevin algorithms with a single control. We do not display the results for the training with multiple controls however as we could not obtain satisfying results neither with Langevin nor-with non-Langevin methods.

Comments on the numerical experiments

We observe that in many cases and in various SOC problems, Langevin and Layer Langevin algorithms show improvement when compared with their respective non-Langevin counterparts, provided that N is large enough,

Figure 4 :

 4 Figure 4: Comparison of Adam et L-Adam algorithms during the training for the fishing control problem with N = 20, 50, 100 respectively. The schedules are γ n = 2 e -3 and σ n = 1 e -3 (5 e -3 for N = 100) for epochs 0 to 40 and γ n = 2 e -4 and σ n = 0 beyond. At the end of each epoch, J is estimated over 50 × 512 trajectories. A zoom on the last epochs is given.

Figure 5 :

 5 Figure 5: Comparison of Langevin algorithms with their non-Langevin counterparts during the training for the fishing control problem with N = 50 respectively. The schedules are γ n = 2 e -3 (5 e -1) and σ n = 5 e -3(1 e -2) for RMSprop (Adadelta resp.) for epochs 0 to 40 and γ n is divided by 10 and σ n is set to 0 beyond. At the end of each epoch, J is estimated over 50 × 512 trajectories. A zoom on the last epochs for RMSprop is given.

Figure 6 :

 6 Figure 6: Training of the fishing problem with multiple controls with N = 10. The schedules are γ n = 2 e -3 and σ n = 2 e -3 for Adam and γ n = 5 e -1 and σ n = 5 e -3 for Adadelta, for epochs 0 to 40 γ n is divided by 10 and σ n is set to 0 beyond.

Figure 8 :Figure 9 :Figure 10 :

 8910 Figure 8: Comparison of algorithms during the training for the deep hedging control problem with N = 30, 50, 50 respectively. The schedules are γ n = 2 e -3(5 e -1) and σ n = 2 e -3 (5 e -3) for Adam (resp. Adadelta) for epochs 0 to 80 and γ n is divided by 10 and σ n is set to 0 beyond.

We remark that max(q, K) = K -ReLU(-q + K).

Acknowledgements

The authors thank Idris Kharroubi for helpful discussions.

(2 e -3, 5 e -3) for Adam (resp. RMSprop, Adadelta) for epochs 0 to 60 (resp. 80, 80) and γ is divided by 10 and σ is set to 0 beyond.

which is remarkable for randomized algorithms. Langevin algorithms converge faster and/or toward a lower loss value. This is particularly visible for the Adadelta method. The gains are limited in some cases (see Figures 5 and11 for RMSprop) but still the optimization procedure is improved.

The gains for the L-RMSprop algorithm remain limited however. In particular, we did not observe any significant improvement for fishing SOC with multiple controls, for deep hedging SOC with a single control and for oil drilling SOC. We do not have explanation for this fact.

The gains brought by Langevin algorithm increase with the depth of the network as shown in Figures 4 and8. However, contrary to [START_REF] Bras | Langevin algorithms for very deep Neural Networks with application to image classification[END_REF], we did not observe overwhelming gains as N becomes very large. We believe that this is due (in part) to the particular structure of the deep SOC problem where the same control is repeated all along the trajectory.

As for SOC with multiple controls, we observe that Layer Langevin algorithms with a small number of Langevin layers (10%-30%) generally outperforms Vanilla Langevin methods while Vanilla Langevin may bring limited gains or be less efficient than the standard non-Langevin methods, see Figures 6 and9 for Adam.