Patricia Laurens
email: p.laurens@esiee.fr

Christian Le Bas
email: clebas@univ-catholyon.fr

Antoine Schoen
email: antoine.schoen@esiee.fr

Stéphane Lhuillery
email: stephane.lhuillery@icn-groupe.fr

Technological contribution of MNEs to the growth of energy-greentech sector in the early post-Kyoto period

Keywords: Innovation, MNE, Energy cleantech, Patent, Kyoto Protocol O33, Q55

We consider the commitment of large firms with high R&D investments to the development of technologies of climate change mitigation related to the production or storage of energy. We analyse such Climate Change Mitigation Technologies focused on energy production and storage (energy CCMT) across the globe with the aim of assessing whether the Kyoto Protocol fosters the diffusion of inventive activity in energy greentech. Using patents as the key dataset, we give an empirical description of the corporate patenting activity and assess its contribution to the overall energy CCMT inventions across countries and sectors of energy greentech before and after the signing of the Kyoto Protocol (1997). Our observations

Introduction

The world conference held in Kyoto (Dec. 1997) is acknowledged as opening a new area in the battle for climate change. It will have durably changed the firm vision about the environmental concerns. How has the greentech technological activity progressed in large firms in the post-Kyoto period? How much is greentech technology being patented by MNEs? In which type of energy clean technologies are the large firms most active after Kyoto (energy sources vs. energy storage, « traditional » nuclear fission vs. renewable energy production)? The paper at hand tries to answer these questions.

The paper contributes to the literature dealing with the firms' commitment to the bulk of inventions in the field of energy green technologies (energy greentech or greentech hereafter) by quantifying the innovative production in that field before and after the signing of the Kyoto Protocol (1997). We focus on large innovative firms and observe if and how they have reacted to the Kyoto Protocol by tracking the change of their patenting behaviour from a pre-Kyoto period (1994)(1995)(1996) to an early post-Kyoto period (2003)(2004)(2005). When a large firm starts to invest or to intensify its investments in the knowledge related to energy greentech, it diversifies its technological knowledge base. Such a process differs across large firms due to the history of the firm, its own initial and past specialization, the specific national institutional setting in which companies are embedded (see [START_REF] Breschi | Knowledge relatedness in firm technological diversification[END_REF]. Large firm invention activity lies at the core of our study but an analysis based on the country of origin of the firm is preferred for delineating the main quantitative trends. Two evidences justify this approach: 1) the home domestic base still stays important for large often multinational firms [START_REF] Laurens | The rate and motives of the internationalisation of large firm R&D (1994-2005): Toward a turning point?[END_REF], 2) for this kind of firm, national energy policy creates strong incentives in favour of green energy technologies (see for instance [START_REF] Popp | Technology and the diffusion of renewable energy[END_REF]. This research is part of the general theoretical framework of the theory of diffusion of technology. Clean technologies related to energy can be considered to be general purpose technologies shaping a wide array of environmental innovations related to energy -much like information and communication technologies do, to an even wider extent. Such technologies diffuse through a wide variety of industrial sectors and technologies all over the world.

According to this perspective, we do not study the diffusion of a given technological novelty through a group of firms belonging to the same industry. Thus we cannot use the well-known models (such as epidemics models, S-shaped curves, saturation or percolation structures) for analysing the scale, rhythm and determinants of innovation diffusion (for a list of these models see for instance [START_REF] Swann | The Economics of Innovation[END_REF]. Nevertheless, we can support our research on several theoretical results that help frame the diffusion of clean technologies related to energy. For instance according to the directed technological change approach proposed by Acemoglu (2002), energy prices and market size plays a crucial role for reorienting R&D towards greener technology projects. Nevertheless the two effects act in opposite direction: the price effect encourages innovation directed at scarce factors, the market effect lead to innovation favouring abundant factors. The degree of substitution between these two determinants plays a role: when they are substitutable le market size effect is larger. The analysis is more complex in the sense Acemoglu considers as well the degree of R&D state-dependence or in other words past-dependence (see [START_REF] Acemoglu | The Environment and Directed Technical Change[END_REF]. Temporary taxes and R&D subsidies redirect innovation towards clean energy inputs [START_REF] Acemoglu | The Environment and Directed Technical Change[END_REF] and promote the diffusion of energy green technologies. Specific taxes (that make the "dirty product" more costly) and R&D subsidies (for supporting "clean" projects) provide incentives to create new "clean products" and foster the introduction of cleaner technologies and environmental improvements [START_REF] Rennings | Redefining innovation --eco-innovation research and the contribution from ecological economics[END_REF]. [START_REF] Verdolini | At home and abroad: an empirical analysis of innovation and diffusion in energy technologies[END_REF] have thus confirmed empirically the role of energy prices effects coupled to technological opportunities effects. [START_REF] Popp | Technology and the diffusion of renewable energy[END_REF] when analysing renewable energy have confirmed that the role of policy-induced substitution is larger than technology push dynamics. The energy technology process has specific features that impede the identification of a simple trend. On the one hand the supplier/user relationship is more complex in that the suppliers of energy equipment are not always linked to the final user, on the other, this activity has a strong systemic dimension [START_REF] Oltra | Sectoral system of environmental innovation: An application to the French automotive industry[END_REF]. Moreover, evolutionary economics of environmental innovation [START_REF] Horbach | Determinants of Eco-innovations by Type of Environmental Impact: The Role of Regulatory Push/Pull, Technology Push and Market Pull[END_REF][START_REF] Mowery | Technology policy and global warming: Why new policy models are needed (or why putting new wine in old bottles won't work)[END_REF][START_REF] Ghisetti | Innovation Complementarity and Environmental Productivity Effects: Reality or Delusion? Evidence from the EU[END_REF] put forth the role and variety of institution types and the notion of technological trajectory (Antonelli, 2011).

The paper is organized as follows. The first section delineates the context of energy greentech at the world and nation levels. The role of national energy policies is pointed as the impact of the different actors (industrial firms in particular). Section 2 presents the research questions. Section 3 offers information on the data set, the characteristics of the sample of firms and the definition of the indicators. Section 4 provides results and shows new trends related to the large firms' contribution to cleaner environment. In the last section we discuss our findings, highlighting the role of major firms from Japan, USA and Europe.

: The energy greentech context: a brief overview from national policy to corporate technology activity

Climate change issues have become a global concern putting pressure on decision makers within governments and corporations. Since 1992 obligations were taken to address climate change issues (Borghesi & al., 2002). The Kyoto Protocol (1997), which established emission reduction targets, was rejected by the US in 2001 arguing of absence of obligations for all GHG emitters and possible negative effects on the US economy2 . Many new regional and national policies followed the signing of the Kyoto Protocol where developed countries agreed to limit emissions of greenhouse gases.

: First steps in greentech

The extent and scope of State support differs across countries. Several European countries have emerged as early movers in promoting the supply and demand of clean energy and EU policies have reinforced member States' programs (promoting high R&D levels, the pre-emption of the EU internal market with an early positioning in countries such as Germany, the UK and Italy, and public incentives and standards promoting EU technology deployment).

Japan was also an early mover in cleantech. Without national energy resources, it got engaged in environmental innovation policy after the oil shocks (the Sunshine Programme was designed to introduce solar power in the 1970s). These developments were partly responsible for establishing major Japanese solar photovoltaic (PV) manufacturers and a thriving solar industry [START_REF] Foster | Solar Energy: Renewable energy and the environment[END_REF] 3 . Conversely the United States has suffered from the lack of a coherent national energy policy and of an effective legislation creating incentives for renewables development although, at the state level, policies fostering clean energy industries have been implemented.

: Evidence for technological development in greentech

The correlation between political decisions resulting from the signing of the Kyoto Protocol and the take-off of clean-energy technologies was documented [START_REF] Jaffe | Technological change and the environment[END_REF][START_REF] Johnstone | Renewable energy policies and technological innovation: evidence based on patent counts[END_REF][START_REF] Veugelers | Which intruments to induce clean innovating[END_REF]. In order to measure the stimulation of greentech, many scholars used surveys 4 or patent data [START_REF] Popp | Lesson from patents: Using patents to measure technological change in environmental models[END_REF]. The latter is considered to correctly reflect the level of R&D investments [START_REF] Griliches | Patent Statistics as Economic Indicators: A Survey[END_REF] and are adapted to investigate the consequences of the public policy framework put in place to support the development of cleantech. Patent data offers accessibility over long periods of time, provide information regarding applicants' name and location while covering worldwide inventive activity.

3 Japan was the first country to reach 1 GW of installed solar capacity in 2004 thanks to the first subsidy programme started in 1994 for residential solar panels. 4 Since 2006 the Community Innovation Survey reports information concerning the adoption of energy-saving technologies in firms. Many papers now make use of the data stemming from this survey.

Several studies have investigated the rise of greentech patenting since the end of the 1990s and put forward the leading role of developed countries, in particular Japan (plus United States, and Germany) but also the significant contribution of new comers like Korea, China or Russia at USPTO (Dechezleprêtre, 2009 and[START_REF] Veugelers | Which intruments to induce clean innovating[END_REF][START_REF] Mark | Green Growth, Technology and Innovation[END_REF]. Using patent data, Dechezleprêtre (2009) (like [START_REF] Lanjouw | Innovation and the international diffusion of environmentally responsive technology[END_REF] finds that US, Japan, and Germany account for two-thirds of the climate-friendly inventions. They emphasize the role of policy in the greentech innovation increase that occurred after the Kyoto Protocol in all countries except in the US and Australia that had not ratified the Kyoto Protocol 5 . Using a panel of patent data from OECD countries for exploring the development of renewable energy technologies, [START_REF] Johnstone | Renewable energy policies and technological innovation: evidence based on patent counts[END_REF] show a rapid growth in wind and solar energy patent activity, particularly since the mid-1990s and find that public funded R&D programs increased patenting activity for the sponsored technologies. However their effects on innovation in renewable energy differ across technologies.

: Large firms and greentech

Scarce information is available on the contribution of the different actors from the business sector to green technology innovation. The quest for social and environmental sustainability has transformed the landscape of global competition [START_REF] Nidumolu | Why Sustainability Is Now the Key Driver of Innovation[END_REF]. Large firms accused of being major contributors to various environmental problems as a result of their worldwide operations [START_REF] Christmann | Multinational companies and the natural environment: Determinants of global environmental policy[END_REF][START_REF] Christmann | Globalization and the environment: Determinants of firm self-regulation in China[END_REF][START_REF] Strike | The (IR) Responsibility of multinational enterprises[END_REF] offer potential market opportunities in new products and technologies and save cost from lower energy use [START_REF] Mcwilliams | Corporate Social Responsibility and Financial Performance: Correlation or Misspecification?[END_REF][START_REF] Cohen | Absorptive capacity: A new perspective on learning and innovation[END_REF][START_REF] Grant | Prospering in Dynamically-Competitive Environments: Organizational Capability as Knowledge Integration[END_REF][START_REF] Begg | The business of climate change: Corporate responses to Kyoto[END_REF][START_REF] Margolick | Corporate Greenhouse Gas Reduction Targets[END_REF][START_REF] Reinhardt | Global Climate Change and BP Amoco[END_REF].

Despite the fact that they face common threats and opportunities, there was a striking variation in the responses of companies across sectors and countries [START_REF] Falkner | Business and Global Climate Governance: A Neo-Pluralist Perspective in Business and global governance, Chapter 5[END_REF]. As a whole, European industry has displayed a readiness to invest in technologies that might reduce greenhouse gas emissions. In the 1990s, US-based companies were very active in challenging the climate change mitigation. Large firms on both sides of the Atlantic appeared to converge in the 2000s toward a shared acknowledgement of the role of GHGs in climate change and the need for some action. The impact of MNEs' home country policies on corporate strategies is likely to diminish over time as industries become more international in scope [START_REF] Levy | CSR and Theories of Global Governance: Strategic Contestation in Global Issue Arenas. The Oxford Handbook of CSR[END_REF].

Since the early 2000s many illustrations of the commitment of MNEs in greentech can be found in the academic literature and in the media. Pernick and Wilder (2007) present examples that show that the "clean tech revolution" is already under way. Very large corporations such as Intel, General Electric, Toyota, Sharp, Total, Chevron, Daimler reported large investments in clean technology, through R&D programmes, partnerships or start-ups acquisitions. Traditional energy companies like Total became involved in solar power (acquiring SunPower, Silicon Valley's dominant solar-panel maker). Among the dominant players in corporate cleantech are: ABB (world leader in power grids), Siemens and Schneider Electric that specializes in energy management6 . The adaptive responses of firms to the climate change challenge also depend on the industry itself. Companies from petroleum-gas, automotive, energy, home appliances and metal/mining industries, which are closely related to climate change issues due to their energy needs, their processes and their outputs -are more sensitive to the climate change [START_REF] Deegan | A study of the environmental disclosure policies of Australian corporations[END_REF][START_REF] Lang | Cross sectional determinants of analyst rating corporate disclosures[END_REF]. However the ability of sectors to develop green competitiveness based on their existing comparative advantages, skills and production patterns is uneven across countries (Frankhauser, 2013). Multinational companies are more responsive to climate change when compared with national enterprises [START_REF] Kaya | Companies Responses to Climate Change: The Case of Turkey[END_REF].

However, it has also to be kept in mind that R&D investments in the field of energy/electricity have dramatically declined in the developed countries over the last decades to the exception of state-owned enterprises [START_REF] Sterlacchini | Energy R&D in private and state-owned utilities: An analysis of the major world electric companies[END_REF].

: Research questions

To date, there is no global picture targeting the diffusion of energy greentech in large firms. Our aim is to qualify, from the pre-Kyoto to the post-Kyoto periods, the trends in the contribution of large firms to the global greentech innovative production across countries. We focus on the energy sector and consider technologies related to energy production and energy storage that mitigate climate change7 . We use the terms cleantech or greentech when referring to these technologies.

In the context of R&D public policy and large firm strategy in the area of energy greentech, we address four questions:

1. The first investigation aims at measuring the contribution of large firms involved in energy green technologies arcoss countries and investigate their evolution in order to assess the likely diversification in favour of green technologies. We explore how many firms are involved with energy greentech in the pre-Kyoto period and how many have entered a process of diversification towards energy greentech after the adoption of the Kyoto Protocol.

2. The second aspect aims to determine if (and to which extent) large innovative firms increased their inventive production in energy greentech after the Kyoto protocol. In order to appreciate if large firms get more and more involved in green technologies, we compare their relative volume of patents in energy greentech in the pre-and post-Kyoto periods of time.

3. In order to compare the efforts of large firms, we rely on their specialization in energy greentech. We assess the scale and evolution of the large firm relative specialization in green technologies through a green specialization index. It reveals in which countries firms have done more efforts to reinforce their specialization in energy greentech.

4. Finally, we identify the sectors of energy greentech in which the large firms are the most active. For this, we detail the green specialization of firms at the level of green subsectors and follow their evolution over time.

: Data and sampling

: Database building

This research uses patent information extracted from the Patstat database (2011), which includes all patents applied for through the world in one of the 180 national, regional and international patent offices. We retrieve first filings (priority patent applications 8 , i.e. the very first patent application for a novelty) without any patent office restriction from 1994 to 2005 and select patents filed by firms from 1994 to 1996 as the set of patents of a pre-Kyoto period. In a similar way, we select patents filed by from 2003 to 2005 as the set of patents of a post-Kyoto period 9 .

We select a corpus of firm patents by gathering together those patents filed by any legal entity which belongs to the set of 946 large firms which have filed at least 5 priority patents in any technology during both the pre-Kyoto (1994)(1995)(1996) and post-Kyoto (2003post-Kyoto ( -2005) ) periods. These firms were selected from an initial list of more than 2000 large industrial companies with the highest annual R&D investments provided by the Industrial R&D Investment Scoreboard 2008 (1000 European firms and 1000 non European firms) 10 . For each firm, the patent portfolio was built using the Orbis database 11 by including applications originating from the firm and from all identified subsidiaries in which the firm had more than 50.01% of shares (see [START_REF] Laurens | The rate and motives of the internationalisation of large firm R&D (1994-2005): Toward a turning point?[END_REF] for methodological details on delineation of the consolidated corporate patent portfolios).

8 Considering priority patents applied without any patent office restriction we encompass all greentech activity (de Rassenfosse et al., 2013) while most of published studies select valuable patents (i.e. PCT patents or patents applied or granted at USPTO). 9 The number of energy greentech patents applied for by large firms each year is very low in many countries before the signing of the Kyoto Protocol in 1997 (around 150 in Germany but only around 40 in France, 15 in the UK …). Therefore we calculate indicators on a 3 year period of time to obtain more robust indicators in the pre-Kyoto period that we compare with those calculated on a 3 year period of time in the post-Kyoto period. 10 The list of the firms included in the Industrial R&D Investments Scoreboard (edition 2008) is given in http://iri.jrc.ec.europa.eu/scoreboard.html. 11 Edited by Bureau van Dijk Electronic Publishing.

Each firm is assigned to a country according to the location of its headquarters and its patents are assigned to the country of the headquarters of the firm to whom the applicant belongs12 .

The 946 firms are roughly equally distributed between the US (34%), Asia (27.2%) and Europe (35.7%) (Table 2). In North America, firms in Technology (33%), Industrial goods and services (18%) and Health cares (13%) dominate. In Europe, Industrial goods and services (27%), Consumer goods (17%) and Industrials (13%) are the most represented industries. In Asia, the principal industrial sectors are Industrial goods and services (24%), Consumer goods (21%) and Basic Materials (19%).

Among the overall firm priority patent portfolios, we retrieve energy cleantech patents from the Y02E subclass of the CPC classification 13 . It covers technologies dealing with the reduction of GHG emission, related to energy generation, transmission or distribution [START_REF] Veefkind | A new EPO classification scheme for climate change mitigation technologies[END_REF] and includes Technologies with contribution to GHG emissions mitigation (energy storage (batteries), fuel cells, hydrogen technology), Renewable energy sources (photovoltaic, wind, thermal solar, hydro, oceanic and geothermal energies), Technologies for the production of fuel of non-fossil origin (biofuel from energy using wastes), Combustion technologies with mitigation potential (Combined Heat and Power …); Nuclear Energy and Technologies for efficient electrical power generation, transmission or distribution.

: Indicators of energy greentech specialization

Let us use i= 1, 2, 3….n to index firms; j= 1, 2, 3….m to index countries and g =1 to 7 to index sub sectors in energy greentech.

We formally consider the firm patenting at time t as: Pij (t) = firm i from country j new patents at time t (green technology patents included) Pgij (t) = firm i from country j new patents in energy green technologies sub sector g at time t

We measure the volume of firm i patenting in energy greentech inventions (whatever the type of sub sector) as

We measure the firm i contribution to energy green tech as:

The index of specialization in energy green technologies is defined for firm i from country j and the time period t as:

where ∑i Pgij indicates all energy green patents in our data set (for time t) and ∑∑ij Pij (t ) the overall set of patents (whatever the technological field and for the overall number of firms and for time t).

Firm i has a relative specialisation in energy greentech when Gij (t) >1 14 . The higher Gij (t), the more specialised firm i is.

The index of specialization in a subsector g of energy green technologies, SGgij relates to the firm i (from country j) contribution to a subsector g of energy greentech inventions among energy greentech inventions. It is expressed as: Firm i has a relative specialisation in the subsector g when SGgij (t) >1. The higher SGgij (t), the more specialised firm i is.

: Overall characterisation of the database

Between 1994 and 2005, the 946 firms applied for 3 590 310 priority patents, among which 72 565 are greentech patents. Technologies included relate to energy storage, hydrogen and fuel cells dominate in firm greentech patents (Table 1). Then come technologies for renewable energies, combustion and nuclear energy. Japanese firms representing about 23% of the firms in our dataset dominate the worldwide corporate greentech priority patenting by producing more than 55% of greentech patents applied for worldwide from 1994 to 2005 (Table 2). Europe and US, housing each about 1/3 of the firms, contribute each to 15% of worldwide corporate greentech patenting. This overwhelming contribution of Japan priority patents reflects a bias due to the heterogeneity in patent office rules that is particularly striking when considering patents at the Japan Patent Office 15 . Using transnational priority patents (patents applied for in at least two distinct patent offices) provides a less biased picture and alleviates the contribution of Japanese firms. However, Japanese firms still dominates to a large extend (45%) the worldwide corporate energy greentech invention.

Chinese and Korean firms are also large patentees with each 5% of the worldwide greentech patenting, a high share compared to their presence in the firm set. German firms contribute approximately to 2/3 of European corporate greentech patents (around 10%), French firms to 2.3% and the British firms to 0.8%.

15 Until recently, patents applied for at JPO had only one claim while in main western patent offices, each patent include several claims. Extending patents in other countries usually requires to group several patents applied for at JPO into a single patent applied for in another country.

The ratio was estimated to be 3:1 or 5:1 (de Rassenfosse, 2013).

Table 2: Contextual data on the contribution of firms to energy cleantech patenting across countries (1994)(1995)(1996)(1997)(1998)(1999)(2000)(2001)(2002)(2003)(2004)(2005). The contribution of large firms to their home country total greentech patenting shows strong variations across countries (Table 2). Overall, large firms contribute to more than 6 over 10 patents in energy greentech. In Japan (or Switzerland), the greentech technological activity is performed almost exclusively by large firms while in the US and Europe, they are involved in 1/4 to 1/3 of the greentech patents of their home country. This share varies across EU countries:

Country of firms

in France large firms contribute to approximately 50% of greentech but this share is below 10% in countries like Denmark or Austria.

In the pre-Kyoto period, the 946 firms filed 706 524 priority patents; 11 445 of them were energy greentech ones. In the post-Kyoto period, 882 895 patents were applied for with 20 273 energy greentech patents. In the decade, the number of energy greentech patents of firms has almost doubled and their distribution by greentech sectors has changed. The largest one "Energy storage, fuel cells and hydrogen technologies" received more and more attention from large firms: it concentrated 62% of the energy greentech patents in the pre-Kyoto period and 3/4 of them ten years later. The second largest sector "Renewable energy" has declined from 20% to 16%. To the exception of "Fuel of non-fossil origin", whose share remained stable to 2%, the shares of other sectors have decreased. The drop was most marked in "Nuclear energy" (from 7.9% to less than 1%) and "Technologies for efficient electrical power generation, transmission or distribution" (4% to 1%).

: Trends of the technological contribution of MNEs to energy greentech

This section details the trends of the corporate greentech activity from the pre-Kyoto to post-Kyoto periods of time based on the 4 indicators previously defined.

: Share of energy greentech firms across countries

In the pre-Kyoto period, 36% of the large firms were green firms (i.e. they applied for greentech patents) (Table 3). This share has gained 10 points and reached 46% in the post-Kyoto period.

The diffusion of energy greentech inventive activity among firms is more pronounced in European countries (+50%) than in US (+12%) or Asia (10%). With 70% of its large firms already active in energy greentech in the 1990s and 3/4 of firms in the mid-2000s, Japan is a special case. This (large) share of Japanese green firms is about twice what is observed in western countries where approximately 1/3 of large firms are green. In France and Italy the green activity of large firms is high. Conversely, Nordic countries and the UK show a low share of large green firms, like in the US. In Asia, it is worth noting that the share of green firms has sharply increased in the post-Kyoto period in Korea and Taiwan. the share of greentech patents in firm portfolios has progressed from 1.62 % to 2.30 % within ten years (Table 4). This increasing commitment of firms to greentech is not homogenous across countries. Firms from Japan, France, Netherlands, Belgium and Denmark show the strongest growth. US and German firms hardly progressed at all and decreasing trends are even observed in Switzerland, Canada or Finland (the 2 first had already a high commitment in the pre-Kyoto period). In the mid-2000s, Japan, France, Belgium, Canada and Italy are the only countries for which large firms have more than 2% of energy greentech patents in their patent portfolios. Firms from Germany, the US, the UK or Korea lag behind. In Asia, Japanese firms have a pronounced and durable specialization in greentech that has intensified over time: their specialisation index has grown from 1.13 to 1.34 (Table 5).

Conversely, Korean or Taiwanese firms show an absence of greentech specialisation and have hardly progressed over the decade (G has increased from 0.20 to 0.25 but remained far below 1). US firms that were not not specialized in greentech technologies in the pre-Kyoto period have not intensified but reduced their overall efforts later (their greentech specialisation index being respectively equal to 0.62 and 0.50). In Europe, diverse trends coexist. German firms that contribute approximately to more than half of European corporate greentech patents have a specialisation index reduced by 1/3 in the post-Kyoto period (energy greentech was not among their main priorities). Such a drop in the greentech specialization index is also observed in Italy and in most of the small or Nordic European countries (except Belgium, Netherlands and Norway). Conversely, French firms have significantly reinforced their overall greentech specialisation and become specialised in energy greentech (G=1.13) in the post-Kyoto period. Our results show the leading position of Japanese firms in terms of their net contribution to greentech patents compared with their foreign competitors. They reveal as well that Japanese firms gave green technological activity a high and early priority as shown by their relatively high specialization already in the pre-Kyoto period. Being the greenest firms and still reinforcing their commitment to this field (the percentage of energy green patent in their patent portfolios has risen from 1.8% to 3.1% on average) Japanese firms are responsible for most of the progression of the worldwide corporate technological activity in energy greentech after the signing of the Kyoto Protocol. Large firms from a few other countries also performed well as they intensified their efforts in greentech. This is the case in France, Italy, Belgium and Canada.

Firm specialisation in subsectors of energy greentech

Firm greentech specialisation index across countries differ according to sectors of greentech (Table 6). In the pre-Kyoto period, US and EU firms were centred in Combustion and Nuclear These data show that the post-Kyoto period is all over the world a period of rising commitment of firms to all sectors of energy greentech. However most European firms remained more active in traditional energy sector (except in a few small countries) and increased their activity in transport energy while US ones were more prone to develop technological skills in renewable energy.

: Discussion of findings: role and strategy of major firms

Based on the firm's home location, we follow the contribution of the firms to energy greentech in the pre-Kyoto and post-Kyoto periods and investigate if its intensity increased after the signing of the Kyoto Protocol. We aim at exploring if progress of large firms' commitment to energy greentech is linked to regional strategic policy regarding climate change issues or if, energy cleantech innovation being a global challenge, business competition for international market between large firms may be the major motivation for large firm involvement. In some sectors, the firm main responsibility could be linked to its capacity to propose technological solutions for the home country to respect its engagement towards greenhouse gas emission restriction (for example in the case of large public firms); in others, intensive competition for international market may be the dominant incentive (for example, the race for electric vehicles in the car industry).

5.1 : The pre-Kyoto period highlights the leadership of Japanese firms in energy greentech

In the pre-Kyoto period, Japanese large firms concentrated almost all the greentech inventive capacities of the country and all manufacturing industries were committed. Their early high greentech specialisation gave them an obvious advantage in the competition of large firms for cleantech innovation leadership over European and US corporations. Moreover Japanese MNEs were already specialized in new energy greentech technologies for energy production and energy storage (Renewable energies, Biofuel, Batteries, Fuel cell and Hydrogen technologies). This early and massive commitment of Japanese firms to greentech is at the origin of the leading role of the country in greentech as evidenced in the literature. It finds its roots in early strategic national R&D programmes 16 set up to limit the country dependence on petroleum but also to maintain Japan as a high-tech superpower in the green 21th century. In this perspective, firm technological greentech capacity fulfilled national strategic objective and built a dominating position in the new and promising international market of energy greentech.

The commitment of firms from the US and most of the European countries differed strikingly from the Japanese firms. Overall, they were much less committed (lower share of greentech patents in patent portfolios, lower share of firms applying for greentech patents). Their contribution to the national greentech effort was much smaller and their greentech activities were centred on traditional energies such as nuclear energy and combustion. A large group of European countries including France, Germany, the UK, Switzerland and Sweden were strongly over-specialised in Nuclear Energy, Combustion and Electrical Efficiency but under-specialised in Renewable Energies (with the exception of the UK 17 ). The 16 For exemple, Sunshine or WE-NET programmes for photovoltaic or hydrogen technologies. 17 The UK (like the Netherlands) hosts several international MNEs that locate their headquarter in the UK or in Caraïbian countries of in Commonwealth countries due to fiscal incentives (Shell, Tyco International …) that may have strategies that differ from continental EU firms.

Netherlands is the only western country among the main patentees where the firm specialisation in Renewable energy exceeded that of Japanese firms.

:

The early post-Kyoto period: a slightly modified landscape

The leading position of Japanese firms in the mid-1990s has not been really challenged in the early post-Kyoto period. Neither European nor US firms have reinforced their contribution that still stagnated at 6% to 7% of the total large firms production in the post-Kyoto period; the US firm share has even declined. Japanese firms have confirmed and even intensified after Kyoto, the strategic orientations defined in the earlier period.

European and US firms did not follow a similar trend and the gap widened between Japanese and western firms. In 2003-2005, Japanese specialisation in energy greentech was twice that of European firms (only 1.5 time ten years before) and 2.7 that of US firms (twice ten years before). However, firms in western countries have started to move: the share of firms filing greentech patents and the share of greentech patents in firm portfolios have significantly increased in most countries (and industries). So firms from most western countries have diversified towards energy greentech and increased their inventive production in green technologies. However this process was slow and they still have stayed below the level reached in Japan ten years previous.

We find different strategies of greentech specialisation to comply with climate change mitigation set by the Kyoto protocol. The priorities of Japanese firms remained unchanged in the post-Kyoto period: they still focused on Renewable energies, Biofuel, Batteries and fuel cells. In US and Europe, different pictures emerged. Some countries remained highly engaged in traditional sources of energy (Nuclear energy, Combustion with SG far above 1) like

France and the UK while most others have strongly decommitted from these sectors and focused on other sources of energy. This is the case in the US where large firms have massively decommitted from Nuclear Energy (and to a lower extent from Combustion) 18 Italy are balanced between wind energy, PV and solar energy 19 . In smaller countries, they are more specialized: wind energy in Sweden; solar energy in the Netherlands. The post-Kyoto evolution also concerns the sector of transport energy (batteries, hydrogen technology and fuel cell) where large firms from most EU countries have reinforced their contributions. German firms took the lead and the new specialisation in batteries, fuel cells and hydrogen technologies emerged from the car industries, a sector of growing importance in patenting in the 2000s [START_REF] Laurens | The rate and motives of the internationalisation of large firm R&D (1994-2005): Toward a turning point?[END_REF].

5.3 : Regional responses of firms to the Kyoto Protocol

In the early post-Kyoto period, the issue of the impact of energy on climate change has been addressed differently by firms according to their geographic locations. Coming back to our four initial questions, the evolutions in the decade from the pre-to the post Kyoto period shown in



The patent portfolios of firms became greener in the post-Kyoto period. However its trend remained "light green" in western countries as it turned comparatively to "intense green" in Japan.



Compared to Japan, highly specialized in energy greentech and still reinforcing it in the post Kyoto period, EU firms and even more US firms are losing ground in the battle for green from the pre-to the post-Kyoto period. As noted by [START_REF] Acemoglu | The Environment and Directed Technical Change[END_REF] American firms addressing climate change at the governance level were catching up with their international competitors. They are, however, in accordance with the situation depicted by [START_REF] Falkner | Business and Global Climate Governance: A Neo-Pluralist Perspective in Business and global governance, Chapter 5[END_REF] where many large US companies (Ford, Chryler, General Motors, Texaco) started to align their strategies at the end of the 1990s by taking into consideration the signature of the Kyoto protocol despite the official opposition of the US government. This overall evolution of US large firms could suggest that large firms are less sensitive to the fluctuating regional political signals in terms of engagement of the government fostering local technological choice and more prone to investigate new technological sectors as soon as they detect international market opportunities. This relates to the competitive risk linked to the regulatory risk in the global and domestic marketplaces described by [START_REF] Cogan | Corporate Governance and Climate Change: Making the Connection[END_REF].

US industry needs to close the spending gap with counterparts in Japan to remain competitive at the cutting edge of technology and innovation in energy greentech, a gap that not all EU firms were aware of. Our results on EU firms fit well the statements of [START_REF] Aghion | Cold start for the green innovation machine[END_REF] according to which the "private innovation machine" has not taken off in the case of EU MNEs due to the design of political instruments that do not provide enough incentives to invest in clean innovation and due to the lack of coordination among EU countries 21 .

Additionally, a lack of coordination at the EU level in the sector of energy that contrasts with the focused energy technology policy in the US and Japan (Wiesenthal, 2011) 21 Using data from CIS 2006 survey they have found that among possible motives for innovation, those related to improving energy effiency and reducing the enrironmental impact were the less often cited motives for explaining innovation. 22 It is for example well documented that Utility industries or other industries extensively relying on energy are more prone to be committed to greentech. We have observed similar trends in our data and the sectoral profile of countries matters and more strikingly in small countries with a small number of large firms (for example the dominating Industrial Machinery sector in Austria, Norway or Sweden explains the green affinity of these countries). This is also the case of the energy supplying industry, for large public firms like Electricité de France, Areva (France), Vatterfall (Sweden) or Hydro-Quebec (Canada).

23 However one main difference relates to the predominent role of Utilities in our study that did not emerge in their multisector greentech work.

of large firms innovation to the global greentech innovation in the firm's country. Our main findings are the following 26 :

1. The contribution of large firms to greentech patenting varies according to countries and greentech sectors. In Japan all the greentech technological activity originates from large firms. In US and Europe, large firms are involved in 1/4 to 1/3 of the greentech patents 27 .

2. The sector of the transport energy (battery, fuel cell) is the sector where the firm commitment is the strongest.

3. Japanese and US firms are highly committed to renewable energies.

4. The post-Kyoto period is a period of growing commitment of firms to all sectors of energy greentech. European firms tend to remain more active in traditional energy sector and transport energy while US ones develop skills in renewable energy.

5. There is a large heterogeneity across European countries in terms of shares of greentech, index and profile of greentech specialisation

Coming back to the diffusion perspective presented in the introduction, our more important finding is that the Kyoto Protocol has deeply transformed the game. Our sample of large multinational firms makes appear a massive trend in favour of the diffusion of energy greentech. A growing number of firms produce energy greentech inventions and the overall 26 We do not provide any assessment of the enviromental benefits of the MNEs technological activity in clean energy sector (as realized by Gilli et al., 2013;[START_REF] Ghisetti | Innovation Complementarity and Environmental Productivity Effects: Reality or Delusion? Evidence from the EU[END_REF][START_REF] Mazzanti | Environmental efficiency and labour productivity: Trade-off or joint dynamics? A theoretical investigation and empirical evidence from Italy using NAMEA[END_REF].

27 More than 83% of corporate greentech patents produced worldwide originate from Japanese firms while US and European firms contribute to 6%-7% of corporate greentech. But this counting is clearly due to the bias provided by the use of priority patenting.

share of greentech has increased significantly -a tendency that has already been identified by Veugelers (2014) 28 for the total population of patentees. Kyoto protocol may be interpreted as a signal opening a new area of a stronger environmental regulation. As such it has triggered a wave of environmental inventions [START_REF] Borghesi | Brown Sunsets and Green Dawns in the Industrial Sector: Eco Innovations, Firm Behavior and the European Emission Trading[END_REF]. This result has an important meaning: overall, the firms have started a virtuous development of their knowledge capacity in the field of energy greentech, entering a learning process in energy green technology. Our work confirms the study by [START_REF] Nicolli | Heterogeneous policies, heterogenous technologies : the case of renewable energy[END_REF] stating that the ratification of the Kyoto protocol -determining a more stable and less uncertain policy framework -has amplified the green innovation inducement effect of both energy policy and market liberalization.

In the technological competition between green and "dirty" technologies all factors providing additional benefits to the green energy technologies are worthy. We can thus expect the firms to switch to effective green energy technologies in a near future. Our index of specialisation (that is less an indicator of diffusion than a marker of internal specialisation) provides a less clear picture. That is in line with what [START_REF] Popp | Technology and the diffusion of renewable energy[END_REF] report on the diffusion of renewable energy that reveals leaders and laggards. Maybe the most important sources of differences across countries will stem from national differences in environmental policy (what report [START_REF] Popp | Technology and the diffusion of renewable energy[END_REF] as well). Our work confirms the private green energy machine cannot be effective without any public support [START_REF] Veugelers | Which intruments to induce clean innovating[END_REF].

  give a high priority to the reduction and eventual elimination of GHG emissions. Climate change mitigation efforts are positive signals towards the consumers, can 5 Australia ratified the Kyoto Protocol only in 2007.

  energy (except Finland and the Netherlands for the latter), sectors where Japanese and Korean firms were not specialised. At that time Japanese firms were focused on new technologies in Transports (Batteries, fuel cells), Renewable energy and Non fossil fuel and their profile remains unchanged after the Kyoto protocol. In Korea, a high growth of specialisation in Combustion and Nuclear energy is visible.One striking feature of the post-Kyoto period is the pronounced decommitment to Nuclear Energy among US and EU firms (to the exception of France and the UK). Similarly, a despecialisation in Combustion emerges in most of the EU countries. The other significant feature emerging in the post-Kyoto protocol is the high growth of specialisation of US and EU firms in Renewable energy, in particular in small EU countries and in the UK. The specialisation index in renewable energy has also progressed in France and Germany but firms from these countries remain under-specialised (SG <1) compared to their competitors. Firms in most EU countries show also an increasing specialisation in Transport (batteries, fuel cell).

  could have delayed the EU large firm commitment to new energy cleantech. Firms answer to the climate change challenge is also dependent on the industrial sectors themselves 22 . We have yet to consider this dimension in our geographical analysis at the country level. Different national industry profiles could explain the different observed behaviours concerning the greentech commitment of large firms. The commitment of industrial sectors among countries was already the subject of an extensive study carried out by Frankhauser (2013) analysing all greentechs (and not only in the energy sector) between 2005 and 2007. Our study while restricted to the energy greentech sector and covering a previous period of time evidences rather similar results 23 .

:

  Acemoglu, D. (2002) Directed Technical Change. Review of Economic Studies 69(4): 781-810

Table 1 :

 1 Energy greentech patents of firms across greentech sectors(1994)(1995)(1996)(1997)(1998)(1999)(2000)(2001)(2002)(2003)(2004)(2005)

	Energy greentech subsectors	Distribution
		(%)
	Renewable energy sources	17.71
	Combustion technologies (CHP, CCPP)	3.13

14 Specialisation is calculated considering the patents of the whole set of large firms as a reference. When firm i is specialised in energy greentech, it means that its relative patenting activity in energy greentech (eg ) exceeds the average activity of the whole set of large firms.

Table 3 :

 3 Share of firms active in energy greentech across countries before and after the Kyoto

	Protocol (%)
		Share of large firms in greentech (%)
	Country*
		1994-1996	2003-2005
	United States	27	32
	Japan	70	77
	Germany	29	44
	United Kingdom	19	27
	France	36	46
	Switzerland	22	44
	Sweden	11	26
	Netherlands	25	33
	Finland	22	33
	Korea	57	86

4.2 Share of energy greentech in firms across countries before and after the Kyoto Protocol Overall energy greentech technological activities were given more attention by firms over time:

Table 4 :

 4 Share of greentech patents (e g ) in firm patent portfolios across countries before and after the Kyoto Protocol

		Share of green patents (%)
	Country*
		1994-1996	2003-2005
	United States	1.01	1.14
	Japan	1.82	3.08
	Germany	1.48	1.51
	United Kingdom	0.64	0.87
	France	1.43	2.59
	Switzerland	2.88	1.08
	Sweden	0.71	0.56
	Netherlands	0.54	1.36
	Finland	0.70	0.36
	Korea	0.40	0.57
	Belgium	0.13	2.58
	Canada	5.09	2.19
	Denmark	0.82	1.40
	Italy	2.24	2.14
	Taiwan	0.29	0.48
	Total	1.62	2.30

*: only countries with more than 5 firms are shown 4.3 Distribution of firm energy greentech specialisation across firm countries

Table 5 :

 5 Firm greentech specialization index (G) across countries before and after the Kyoto

	Protocol
		Greentech specialisation index of
	Country*	large firms
		1994-1996	2003-2005
	United States	0.62	0.50
	Japan	1.13	1.34
	Germany	0.91	0.66
	United Kingdom 0.40	0.38
	France	0.88	1.13
	Switzerland	1.78	0.47
	Sweden	0.44	0.24
	Netherlands	0.33	0.59
	Finland	0.43	0.16
	Korea	0.25	0.25
	Belgium	0.08	1.12
	Canada	3.14	0.95

Table 6 :

 6 Firm greentech specialisation index (SG) in main subsectors of energy greentech Bold numbers indicate relative specialisation in subsectors of energy greentech.

					Batteries,
		Renewable	Nuclear	Non	fossil
	Country	Combustion			fuel	cell,
		energy	energy	fuel
					hydrogen
		94_96 03_05 94_96 03_05 94_96 03_05 94_96 03_05 94_96 03_05
	United States	0.54 1.23 1.95 1.78 2.56 0.84 0.24 0.33 0.93 0.95
	Japan	1.09 1.03 0.61 0.73 0.64 0.75 1.12 1.10 1.04 1.01
	Germany	0.71 0.65 3.75 1.90 3.08 0.00 0.80 0.68 0.61 1.03
	United Kingdom 1.20 2.98 6.44 4.92 1.48 1.39 1.04 0.00 0.54 0.51
	France	0.42 0.70 4.57 4.03 4.50 9.58 0.31 0.23 0.51 0.85
	Switzerland	0.43 1.20 6.61 0.74 5.54 0.00 0.23 1.69 0.22 0.48
	Sweden	0.00 3.37 3.24 10.31 1.48 0.00 0.00 0.00 1.23 0.24
	Netherlands	2.07 1.09 2.10 1.05 0.00 0.00 0.00 0.60 0.75 1.01
	Finland	0.34 1.09 6.36 2.32 0.00 0.00 3.24 10.60 0.96 0.65
	Korea	0.80 0.45 0.00 2.04 0.51 3.90 0.00 0.00 1.24 1.06

  and significantly increased their involvement in Renewable Energy (SG has decreased by a factor 3 in Nuclear Energy and progressed by a factor 2.5 in Renewable Energy). Similar changes occur in small European countries (except in the Netherlands where firms were already highly involved in Renewable Energy in the 1990s). However the overall commitment of European firms to Renewable Energies is less marked than in US ones: while the specialisation index of the latter more than doubled, that of EU firms increased only by one-third on average at the ) had conducted countries like Italy, UK or Germany to reconsider their programme in Nuclear energy but, on the other hand, oil prices, interest of developing countries and environmental concerns lead to a nuclear renaissance in the late 1990s. Our results indicate that in 2003-2005, US large firms followed the first trend.

	continent level.
	Firms from Germany, France and the UK did follow different trends. German firms have totally
	decommitted from Nuclear Energy but not from Combustion. In the UK, firms still focused on
	traditional energy. French firms show a weak investment in new energy greentech for
	energy production and remain under specialised (SG <1), a trait which reflects the strong
	reliance of that country on nuclear resources (Cosatea, 2014). Besides this trend of sustained
	reliance on traditional energies, all EU countries have increased their specialisation index in
	new sources of energy or energy storage. The investments of firms in Germany, France and
	18 The disengagement of US large firms from Nuclear Energy in the mid-2000s was also
	observed by Albino (2014) on a smaller set of patents. He explains how opposite trends
	coexisted: one the one hand, nuclear accidents (Three Miles Islands (1979) and Tchernobyl
	(1986)

Table 6

 6

		can be summarized as follow:
		Diversification of corporate patent portfolios in favour of green technologies is a
		global trend shared worldwide. On average, European firms were more prone to
		diversification than US ones but they still lag far behind Japanese firms that strongly
		diversified much earlier.

  delay in intervention in favor of green technologies is costly. A soon and strong policy response is far better.This apparent relative decreasing specialization of US and EU firms has to be considered with care. It results mainly from the decommitment from Nuclear energy and Combustion in many (but not all) countries. Signals of an early commitment to Renewable energy and Energy storage is evident in most western countries in the post Kyoto period. Its intensity is more pronounced in the US compared to EU.However EU mixes diverging trends and small EU countries adapt faster 20 .

Table 6 :

 6 Changes from the pre-to the post Kyoto period according to four indicators

Indicator Number of firms in energy green technology Share of greentech in patent portfolio (e g ) Greentech specialisation index (G) Specialisation in greentech sub- sector (SG)

	Japan	+10%		+ 66% Netherlands,		+20% specialisation	Unmodified (except in France)
		Already high in	Starting from Denmark),	a	Already (France,	Specialisation in
				stagnation (Italy)	Netherlands,	Involvement in
		the first period	high share in the or severe decrease	specialised in the Denmark)	renewables, Renewable or	and
				(Switzerland,		De-specialisation	energy storage
		incremental	first period Sweden, Finland)		first period (Germany, UK,	new energy storage
							Sweden, Italy)
		growth between
	the two periods Our findings, showing that US firms have taken on the issues of renewable energy before most
	US of those from EU countries, contradict Cogan (2006) who states that in the mid-2000s +20% +10% -20% Changed
		Moderate in the	Moderate in the		Reinforcing its	Strong
		first period	first period			under-	decommitment
		Slight	increase				specialisation in	from Nuclear
		the second period				the first period	Strong
							involvement	in
							Renewable
	Europe	+30%		+10%			Global stability	Situation	varies
		High or weak	Diverse evolutions:	but	variation	across countries
		growth according	large	increase	across countries	Decommitment
		to the countries	(Germany, France,	Reinforced	from	Nuclear

In 2011, Canada also withdrew from the Accord.

All these large firms are included in the set of large firms with the highest R&D investments used in this work.

Technologies related to smart grids and the capture of CO 2 are not included.

For example if the applicant of a patent is located in country B and is a subsidiary of a large firm with headquarters in country A, the patent is attributed to the corporate patent portfolio of country A.

YO2 class was set up by EPO in 2010 to tag technologies which "control, reduce or prevent GHG emissions of anthropogenic origin" as set forth by the Kyoto Protocol.

The firms' commitment in the subsector Renewables was investigated more in details. Data are not shown here.

The size of the market that plays an important role favoring abundant inputs (according toAcemoglu, 2002) should have a positive effect on fossile energy research in large countries as the USA. Surprisingly it appears not in our findings.

See among others:[START_REF] Johnstone | Renewable energy policies and technological innovation: evidence based on patent counts[END_REF]Johnstone ( , 2011) ) andDechezleprêtre (2009)

Our study targeting large firms does not give information on country's overall activity in energy greentech nor allow any comparison between countries.

The upward trend is particularly strong for solar PV, wind and carbon capture.

Conclusion: the role of policies and firm strategies

In this study, we contribute to the literature on greentech inventions across countries and in different technologies 24 by focusing attention on the role played by large multinational firms. The leading role of multinational companies in the growth of the green sector was assessed by OECD [START_REF] Kalamova | Sources of Finance, Investment Policies and Plant Entry in the Renewable Energy Sector[END_REF][START_REF] Corsatea | Technological capabilities for innovation activities across Europe: Evidence from wind, solar and bioenergy technologies[END_REF] as well as the importance of clear and continuous national public policies to promote both greentech technology development and market demands. However to our knowledge no research has ever studied to which extend such large contributors to the technological progress have participated in the take off of the "green innovation machine" [START_REF] Aghion | Cold start for the green innovation machine[END_REF] 25 .

We use a unique set of corporate patent data applied for by the firms with the largest R&D investments among the world and restrict the study to those that have a sustained patenting activity. Conversely to most studies that used restricted sets of patents meeting a certain quality threshold when selecting triadic patents, PCT patents, patents applied at USPTO or EPO for monitoring technological developments, we analyse a broader set of patents by including all priority patents applied for, without any restriction. In order to deal with institutional bias associated to priority patent, we most often use relative specialization index and compare corporate activities in greentech and non greentech to assess the distribution of greentech invention among large firms depending on the firm country and on the contribution


Mateusz Budnik

Laurent Besacier

Ali Khodabakhsh
email: ali.khodabakhsh@ozu.edu.tr

Cenk Demiroglu
email: cenk.demiroglu@ozyegin.edu.tr

Deep complementary features for speaker identification in TV broadcast data

This work tries to investigate the use of a Convolutional Neural Network approach and its fusion with more traditional systems such as Total Variability Space for speaker identification in TV broadcast data. The former uses spectrograms for training, while the latter is based on MFCC features. The dataset poses several challenges such as significant class imbalance or background noise and music. Even though the performance of the Convolutional Neural Network is lower than the state-of-the-art, it is able to complement it and give better results through fusion. Different fusion techniques are evaluated using both early and late fusion.

Introduction

In the past few years, Convolutional Neural Networks (CNN) became widely used in image related domains providing stateof-the-art performance [START_REF] Szegedy | Going deeper with convolutions[END_REF]. At the same time Deep Neural Networks (DNN) were being applied more and more to monodimensional signals for tasks like language recognition [START_REF] Matejka | Neural network bottleneck features for language identification[END_REF], speech recognition [START_REF] Deng | Recent advances in deep learning for speech research at microsoft[END_REF] or speaker identification [START_REF] Richardson | Deep neural network approaches to speaker and language recognition[END_REF]. Lately, there was an increasing number of studies trying to address some of the related tasks (notably automatic speech recognition) with the use of CNN based systems with only spectrograms as input [START_REF] Ganapathy | Robust language identification using convolutional neural network features[END_REF][START_REF] Uzan | I know that voice: Identifying the voice actor behind the voice[END_REF]. However, such systems have not yet been widely explored for speaker identification. This study tries to give additional insight into the efficient use of CNN for this particular biometrics task.

Contribution In this paper, the use of convolutional neural networks (CNNs) for speaker identification is investigated. Additionally, a challenging dataset was selected, containing both noise and un-balanced speaker data. This CNN approach is compared to more traditional methods. Despite the lower CNN performance, the use of this deep complementary features in fusion improves on the state-of-the-art.

Outline The structure of this paper is as follows. Section 2 presents a short overview of the recent developments in connection to this study. Section 3 gives the overview of the methods proposed. This is followed by the presentation of the experimental framework in Section 4. The results are presented in Section 5. Section 6 contains the concluding remarks.

Related Works

In [START_REF] Palaz | Analysis of cnnbased speech recognition system using raw speech as input[END_REF] a speech recognition system is presented that uses raw speech as input, which is then processed by a 1D convolutional layer. The use of CNN with spectrograms was also explored previously: in [START_REF] Deng | A deep convolutional neural network using heterogeneous pooling for trading acoustic invariance with phonetic confusion[END_REF] a system dealing with phonetic confu-sion is presented, however convolution is done only along the frequency axis. An interesting approach was presented in [START_REF] Lee | Unsupervised feature learning for audio classification using convolutional deep belief networks[END_REF] where an unsupervised approach to feature learning is proposed. The algorithm is based on a convolutional deep belief network. Apart from that, Recurrent Neural Networks (RNN) were also successfully used with speech spectrograms, as it is suggested in [START_REF] Hannun | Deepspeech: Scaling up end-to-end speech recognition[END_REF] for automatic speech recognition.

An approach similar to this study is presented in [START_REF] Uzan | I know that voice: Identifying the voice actor behind the voice[END_REF]. In their paper, a CNN is trained using spectrograms in order to identify disguised voices. However, compared to our paper, this study considered a significantly smaller number of speakers with the explicit goal of identifying fraudulent behavior and different identities. In addition, no fusion approaches were explored. Apart from a more noiseless and balanced dataset, a different CNN architecture (based on AlexNet [START_REF] Krizhevsky | Imagenet classification with deep convolutional neural networks[END_REF]) is also used. Conversely, the base architecture used in our study is simpler and provides better performance [START_REF] Simonyan | Very deep convolutional networks for large-scale image recognition[END_REF]. Some attempts were also made to use CNNs in noisy conditions. A recent study [START_REF] Mclaren | Application of convolutional neural networks to speaker recognition in noisy conditions[END_REF] uses 1D convolutions on filter banks. Surrounding frames are taken into account and serve as context to reduce noise impact.

In [START_REF] Anand | Convoluted feelings convolutional and recurrent nets for detecting emotion from audio data[END_REF], a study was done on emotion recognition by combination of audio and visual features. Similar to this study, spectrograms were used, and 1D and 2D CNNs were evaluated. In [START_REF] Ganapathy | Robust language identification using convolutional neural network features[END_REF], CNNs were used for the language identification task. Finally, speaker and language recognition experiments using neural networks were also presented in [START_REF] Richardson | Deep neural network approaches to speaker and language recognition[END_REF].

Method description

Speaker Identification Systems

Gaussian Mixture Model-Universal Background Model (GMM-UBM) [START_REF] Douglas A Reynolds | Speaker verification using adapted gaussian mixture models[END_REF] and Total Variability Space (TVS) [START_REF] Dehak | Front-end factor analysis for speaker verification[END_REF] speaker recognition systems are used in this study.

In the GMM-UBM approach, a Universal Background Model (UBM) is first trained on speech features extracted from multiple speakers using the Expectation-Maximization (EM) algorithm. Speaker-specific models are then obtained using Maximum a Posteriori (MAP) mean adaptation. Similarity scoring is done by calculation of log-likelihood ratio (LLR) on these models. Given a sequence of feature vectors X extracted from a test segment, LLR is computed as Λ(X) = log p(X|λ hyp ) -log p(X|λ ubm ), where λ hyp and λ ubm represent speaker-specific GMM and UBM model respectively.

After adaptation, models can also be represented as highdimensional supervectors of means of distributions. These supervectors can be represented as low-dimensional identity vec-Figure 1: A simplified overview of system, which highlights how the convolutional neural network was used in this study in contrast to a more traditional approach. tors (i-vector) using factor analysis. In this approach, mean supervectors Ms and M0 representing speaker-specific model and UBM respectively are extracted, and an i-vector, ws, is calculated using Ms = M0 + T ws. The low-rank rectangular matrix, T , representing the variability space of the i-vectors, is learned in an unsupervised manner using the Expectation-Maximization (EM) algorithm. In the case of having multiple tracks for speaker modeling, i-vectors extracted from each speech track are typically averaged and the average i-vector is used as the speaker model. Scoring can be done with cosine similarity metric, while pre-processing i-vectors before scoring can result in better performance.

In our setup, speaker identification is done by scoring a test segment versus all the speaker models. The speaker identity corresponding to the highest similarity score is chosen as the result of the identification test.

A UBM consisting of 1024 gaussians is trained on the training data for both systems. T matrix is then trained on the segmented training data. Segmentation outputs of conventional BIC-criterion [START_REF] Delacourt | Distbic: A speaker-based segmentation for audio data indexing[END_REF] are used. The dimension of output i-vectors is set to 500. MSR Identity Toolbox [START_REF] Seyed | Msr identity toolbox v1. 0: A matlab toolbox for speaker recognition research[END_REF] is used for all experiments. Similarity scoring is done by cosine similarity scoring between the test segment and the i-vector representing target identity. Similarity scoring is also done using probabilistic linear discriminant analysis (PLDA) [START_REF] Garcia | Analysis of i-vector length normalization in speaker recognition systems[END_REF] for additional comparison using the same toolkit. A wide range of different parameter values was tested to ensure the best possible performance. Length normalization [START_REF] Garcia | Analysis of i-vector length normalization in speaker recognition systems[END_REF] is used for increased performance.

Convolutional Neural Networks

In Figure 1 the general way in which the CNN algorithm is applied can be seen. For any given speech segment (Fig. 1a) the spectrograms are first extracted. Because they have a fixed size there are usually several overlapping spectrograms representing each segment. Next (Fig. 1b) , each spectrogram is fed to the convolutional neural network separately. This in turn produces an individual vector of potential speaker identities for every input (Fig 1c). Finally, to obtain a single vector for the speech track, the individual vectors are averaged.

The network used in this study is inspired by the general design proposed in [START_REF] Simonyan | Very deep convolutional networks for large-scale image recognition[END_REF] for image recognition. It was chosen as a starting point due to its relative structure simplicity and stateof-the-art performance. However, several changes were made in order to adapt it to this specific speaker identification task. The significant differences between the ImageNet dataset (containing images of everyday objects and animals among others), on which the reference network was originally trained, and the spectrogram data made it necessary to retrain the whole algo-rithm from scratch. Due to a less complex nature of the spectrogram data (monochromatic with similar patterns) a reduced version of the original model was introduced with no visible change in performance. The detailed structure can be found in Table 1. The visualization of this network is shown in Figure 2. full conn

1 × 1 × 2048 fc7 full conn 1 × 1 × 2048 fc8 full conn 1 × 1 × 821
Table 1: The structure of the network.

The network was trained from scratch on a set of grayscale spectrogram images with non-square dimensions. The model was trained for around 12 epochs. Every convolutional layer was followed by a rectified linear unit (ReLU), which serves as an activation function and is defined as f (x) = max(0, x). The first two fully connected layers (fc6, fc7) are followed by ReLU and dropout with the rate of 0.5. The output of the last fully connected layer (fc8) is used with the softmax function and corresponds to the target speakers (821 individual speakers in total).

Different to the initial design for image recognition, the proposed structure has fewer convolutional layers (from 8 down to 5), however the filter size for the first two is expanded. Adding additional convolutional layers did not improve performance. Average pooling layers were chosen instead of max pooling. The input to the network is a 48 × 128 pixel grayscale image of a spectrogram. Due to the overlap between the images, no random cropping or rotation is used during training. Caffe framework [START_REF] Jia | Caffe: Convolutional architecture for fast feature embedding[END_REF] was used for training and testing the net.

The network gives predictions based on individual spectrograms. In order to be able to fuse the output with the output of the TVS system, the spectrograms are mapped to bigger speech segments. The mapping is done by averaging the scores of every spectrogram contained within a given segment. In Figure 3a an example of a spectrogram used for training is shown. Figure 3b represents the saliency map, i.e. a heatmap representing the most significant regions of the image used by the CNN to predict a given speaker. In this case it represents speaker with the highest response from the top layer. This was obtained by backpropagating the correct output from the last layer to the input layer in a similar way as it was done in [START_REF] Matthew | Visualizing and understanding convolutional networks[END_REF]. Note the heavy reliance on horizontal patterns.

Fusion

Fusion is often used to enhance results for speaker recognition systems, for example in [START_REF] Mclaren | Application of convolutional neural networks to speaker recognition in noisy conditions[END_REF]. Even if by itself a system gives inferior results, it still can help to improve the baseline performance. In this article, several attempts were made to fuse the CNN results with the output of the TVS system. Both early and late fusions were considered.

Late Fusion

A conventional late fusion of normalized predictions taken from both systems was used, which is the average of the two outputs.

A weighted sum of both outputs was also tested.

Duration-based Late Fusion

This strategy was proposed to give the CNN scores higher weights for short duration segments and lower ones for the long speech segments. CNN seems to produce comparable results to the TVS system on short segments, while the difference in performance grows with increasing duration. Fusion on the longer segments also seems to be less beneficial. This is illustrated in the bottom plot of Figure 5. In this case, fusion was calculated as s = (1 -tanh(d))scnn + sivec , where s corresponds to the scores provided by each system and d is the segment duration.

Early Fusion with Support Vector Machines

This strategy serves as early fusion, where a linear SVM is used for the final classification decision based on a concatenated normalized outputs of CNN's last hidden layer and i-vectors. Principal component analysis is applied to the CNN output in order to match the i-vector dimensionality (500 for each).

Experimental Setup 4.1. Dataset

The REPERE corpus [START_REF] Giraudel | The repere corpus: a multimodal corpus for person recognition[END_REF] was used in this study. The dataset contains a set of videos from two French television channels (LCP and BFM). There are 7 types of videos, ranging from news shows, debates to celebrity gossip and culture programs.

Only the audio track was used in the experiments. The dataset is quite challenging. The recording takes place both inside a studio setting and outside in public and noisy environments. Apart from this, music is often played in the background during certain presentations or interviews. Additionally, there is a significant imbalance between speakers, with anchors and top politicians both being often over-represented in the dataset. Total amount of speech per speaker for speakers present in both train / test sets helps to illustrate this and it is shown in Figure 4. Normalized histogram of the number of segments for each duration bin is also shown in Figure 5 for training and test data. It is important to mention that a big portion of speech segments fall below 2 seconds of speech. For the test set, 24.8% of speech segments are shorter than 2 seconds, and 70.4% are shorter than 10 seconds.

Experiments were done in a closed-set manner, where all the speakers in the training data are used for training models, while performance is evaluated only on test segments from speakers overlapping between training and test data. There are 821 speakers available in the training data, from which only 113 are observed in the test data.

Training data includes 9377 speech segments from 148 videos, while the test data contains 2410 segments from 57 videos. Training data and test data contain around 22 hours and around 6 hours of active speech respectively.

Features

Mel-Frequency Cepstral Coefficients (GMM-UBM, TVS)

Energy feature and Mel-Frequency Cepstral Coefficients (MFCCs) of 19 dimensions are extracted every 10 ms with a window length of 20 ms. These features along with their delta and delta-delta coefficients are concatenated. Static energy is used for silence removal using bi-gaussian distribution of frame log-energies. For each frame, a 59 dimensional feature vector is then obtained after application of feature warping [START_REF] Pelecanos | Feature warping for robust speaker verification[END_REF] on remaining features.

Spectrograms (CNN)

Spectrograms were extracted on 240 ms duration with a frequency of 25 Hz. This results in an overlap of 200 ms (83%) between neighboring spectrograms. For each spectrogram, first the corresponding audio segment was windowed every 5 ms with a window length of 20 ms. Then on each window, after applying Hamming windowing, log-spectral amplitude values were extracted. By discarding the symmetric part and the value corresponding to the highest frequency, a 48 by 128 matrix of values was obtained, which was used as input to the CNN (as an image). Additionally, a wider version of the spectrograms (having 640 ms in duration) were tested. However, their use with the CNN network led to the same performance. Therefore, they are omitted in this paper. In both training and testing phases, spectrograms containing speech from multiple speakers were discarded along with the ones not containing speech.

Results and Discussion

Table 2 gives the accuracy for the baseline systems and the CNN. The segment accuracy for the CNN is generated as explained in Section 3.2. We see that CNN is slightly lower in performance than baseline approaches for speaker identification. In Table 3 the results of fusion are presented. The last two columns give partial results for segments shorter and longer than 2 seconds, respectively. Apart from the standard accuracy, a duration based accuracy is also given, i.e. the duration of the data predicted correctly versus the total duration of the data.

The rather poor performance of the single CNN approach when compared to TVS may be attributed to several different factors. First of all, the unbalanced speaker dataset where some speakers (like high level politicians or news anchors and presenters) are heavily over represented, while others may appear for just a few seconds. The second factor could be connected to the nature of the corpus. Live and mostly spontaneous (especially in the case of debates) TV broadcasts usually come with significant noise (street noises, crowds, other voices) or background music. This may, in fact, disproportionally affect the raw spectrograms over the MFCC features.

A relatively low performance was given also by the PLDA approach, even though a grid search was done in order to choose the best hyper-parameters possible. This can be explained by the dependency of PLDA performance on the availability of a large training set (as discussed for example in [START_REF] Aronowitz | Inter dataset variability compensation for speaker recognition[END_REF]). In the training data used in this study, only 375 speakers out of 821 had more than two segments, whereas thousands of multi session speakers are usually used for successful estimation of the PLDA hyper-parameters. The late fusion approaches represented by CNN+TVS and durCNN+TVS seem to work much better than the early fusion based on the SVM. Both late fusion approaches were able to be better than both the CNN and the TVS. Based on the partial accuracy results, it seems that the main improvement of fusion is for the shorter speech segments. The duration based accuracy reveals the underlining imbalance of the dataset, where the improvement of the number of segments correctly classified does not necessarily imply a higher duration score.

Method

Conclusion and Future Work

In this paper, we proposed an approach which uses the output of a CNN network trained on spectrograms to improve the performance of a TVS system based on MFCC features. The tests were carried out on a broadcast TV dataset, which included reallife issues such as environmental noise and imbalance between speakers, with encouraging results.

As for future work, a multimodal CNN system using both speech and faces extracted from the video may be used to try to enhance the performance. We also intend to use Recurrent Neural Networks (RNNs) for better use of temporal information and possible improvement over longer speech segments. Artificially increasing the CNN training set is another perspective since it has been proven efficient for several image related tasks. Also, the influence of time-and frequency-based features extracted by the CNN may require further insight.

  name type filter size output size / stride input: 48 × 128 grayscale image of spectro. conv1 convolution 7 × 7/1 40 × 122 × 64 pool1 ave pooling 2 × 2/2 20 × 61 × 64 conv2 convolution 5 × 5/1 18 × 59 × 128 pool2 ave pooling 2 × 2/2 9 × 30 × 128 conv3 convolution 3 × 3/1 9 × 30 × 256 conv4 convolution 3 × 3/1 9 × 30 × 256 pool3 ave pooling 2 × 2/2 5 × 15 × 256 conv5 convolution 3 × 3/1 5 × 15 × 512 fc6

Figure 2 :Figure 3 :

 23 Figure 2: The visualization of the CNN used in this study. A spectrogram is taken as input and is convolved with 64 different filters (with the size of 7×7) at the first layer with the stride equal to 1. The resulting 64 feature maps are then passed through the ReLU function (not visible here) and downsampled using average pooling. A similar process continues up to the fully connected layers (f6, which takes conv5 as input, and fc7) and the final output layer (corresponding to the number of speakers in the train set).

Figure 4 :Figure 5 :

 45 Figure 4: Total amount of speech per speaker for speakers present in both train / test sets of REPERE corpus. Speakers are sorted according to total speech duration in training set.

Table 2 :

 2 CNN and baseline accuracy (% on the test set) estimated at the speaker segments level.

		Accuracy	Trained on
	CNN	67.41	Spectrograms
	PLDA	70.50	MFCC
	GMM-UBM	71.16	MFCC
	TVS	72.78	MFCC
	Method	Stand.	Dur.	<= 2s > 2s
		Acc.	Acc.	Acc.	Acc.
	CNN	67.41 76.00	40.93	76.32
	TVS	72.78 83.74	48.99	81.58
	early:SVM:CNN+TVS 69.05 75.27	51.63	75.41
	late:CNN+TVS	75.89 83.61	58.45	82.27
	lat:dur:CNN+TVS	75.10 84.07	56.12	82.04

Table 3 :

 3 Fusion results with standard accuracy and duration based accuracy (on test set).

Acknowledgements: this work was partially supported by the ANR (Agence Nationale de la Recherche, France) and by the T ÜB İTAK (T Ürkiye B İlimsel ve Teknolojik Aras ¸tırma Kurumu, Turkey) grant No 112E176.